[2023-10-10 16:33:57,731][122664] Saving configuration to ./train_atari/atari_demonattack_APPO/config.json... [2023-10-10 16:33:58,048][122664] Rollout worker 0 uses device cpu [2023-10-10 16:33:58,049][122664] Rollout worker 1 uses device cpu [2023-10-10 16:33:58,050][122664] Rollout worker 2 uses device cpu [2023-10-10 16:33:58,050][122664] Rollout worker 3 uses device cpu [2023-10-10 16:33:58,051][122664] Rollout worker 4 uses device cpu [2023-10-10 16:33:58,051][122664] Rollout worker 5 uses device cpu [2023-10-10 16:33:58,052][122664] Rollout worker 6 uses device cpu [2023-10-10 16:33:58,052][122664] Rollout worker 7 uses device cpu [2023-10-10 16:33:58,052][122664] Rollout worker 8 uses device cpu [2023-10-10 16:33:58,053][122664] Rollout worker 9 uses device cpu [2023-10-10 16:33:58,053][122664] Rollout worker 10 uses device cpu [2023-10-10 16:33:58,054][122664] Rollout worker 11 uses device cpu [2023-10-10 16:33:58,054][122664] Rollout worker 12 uses device cpu [2023-10-10 16:33:58,055][122664] Rollout worker 13 uses device cpu [2023-10-10 16:33:58,055][122664] Rollout worker 14 uses device cpu [2023-10-10 16:33:58,056][122664] Rollout worker 15 uses device cpu [2023-10-10 16:33:58,342][122664] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-10-10 16:33:58,343][122664] InferenceWorker_p0-w0: min num requests: 2 [2023-10-10 16:33:58,346][122664] Using GPUs [1] for process 1 (actually maps to GPUs [1]) [2023-10-10 16:33:58,346][122664] InferenceWorker_p1-w0: min num requests: 2 [2023-10-10 16:33:58,392][122664] Starting all processes... [2023-10-10 16:33:58,393][122664] Starting process learner_proc0 [2023-10-10 16:34:00,076][122664] Starting process learner_proc1 [2023-10-10 16:34:00,079][123247] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-10-10 16:34:00,079][123247] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 [2023-10-10 16:34:00,097][123247] Num visible devices: 1 [2023-10-10 16:34:00,114][123247] Setting fixed seed 1234 [2023-10-10 16:34:00,115][123247] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-10-10 16:34:00,115][123247] Initializing actor-critic model on device cuda:0 [2023-10-10 16:34:00,115][123247] RunningMeanStd input shape: (4, 84, 84) [2023-10-10 16:34:00,116][123247] RunningMeanStd input shape: (1,) [2023-10-10 16:34:00,127][123247] ConvEncoder: input_channels=4 [2023-10-10 16:34:00,295][123247] Conv encoder output size: 512 [2023-10-10 16:34:00,296][123247] Created Actor Critic model with architecture: [2023-10-10 16:34:00,297][123247] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): MultiInputEncoder( (encoders): ModuleDict( (obs): ConvEncoder( (enc): RecursiveScriptModule( original_name=ConvEncoderImpl (conv_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Conv2d) (1): RecursiveScriptModule(original_name=ReLU) (2): RecursiveScriptModule(original_name=Conv2d) (3): RecursiveScriptModule(original_name=ReLU) (4): RecursiveScriptModule(original_name=Conv2d) (5): RecursiveScriptModule(original_name=ReLU) ) (mlp_layers): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=ReLU) ) ) ) ) ) (core): ModelCoreIdentity() (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=512, out_features=1, bias=True) (action_parameterization): ActionParameterizationDefault( (distribution_linear): Linear(in_features=512, out_features=6, bias=True) ) ) [2023-10-10 16:34:00,870][123247] Using optimizer [2023-10-10 16:34:00,870][123247] No checkpoints found [2023-10-10 16:34:00,871][123247] Did not load from checkpoint, starting from scratch! [2023-10-10 16:34:00,871][123247] Initialized policy 0 weights for model version 0 [2023-10-10 16:34:00,872][123247] LearnerWorker_p0 finished initialization! [2023-10-10 16:34:00,873][123247] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-10-10 16:34:01,830][122664] Starting all processes... [2023-10-10 16:34:01,833][123465] Using GPUs [1] for process 1 (actually maps to GPUs [1]) [2023-10-10 16:34:01,833][123465] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [1]) for learning process 1 [2023-10-10 16:34:01,838][122664] Starting process inference_proc0-0 [2023-10-10 16:34:01,839][122664] Starting process inference_proc1-0 [2023-10-10 16:34:01,839][122664] Starting process rollout_proc0 [2023-10-10 16:34:01,851][123465] Num visible devices: 1 [2023-10-10 16:34:01,839][122664] Starting process rollout_proc1 [2023-10-10 16:34:01,839][122664] Starting process rollout_proc2 [2023-10-10 16:34:01,867][123465] Setting fixed seed 1234 [2023-10-10 16:34:01,868][123465] Using GPUs [0] for process 1 (actually maps to GPUs [1]) [2023-10-10 16:34:01,869][123465] Initializing actor-critic model on device cuda:0 [2023-10-10 16:34:01,869][123465] RunningMeanStd input shape: (4, 84, 84) [2023-10-10 16:34:01,839][122664] Starting process rollout_proc3 [2023-10-10 16:34:01,870][123465] RunningMeanStd input shape: (1,) [2023-10-10 16:34:01,840][122664] Starting process rollout_proc4 [2023-10-10 16:34:01,845][122664] Starting process rollout_proc5 [2023-10-10 16:34:01,845][122664] Starting process rollout_proc6 [2023-10-10 16:34:01,846][122664] Starting process rollout_proc7 [2023-10-10 16:34:01,882][123465] ConvEncoder: input_channels=4 [2023-10-10 16:34:01,847][122664] Starting process rollout_proc8 [2023-10-10 16:34:01,851][122664] Starting process rollout_proc9 [2023-10-10 16:34:01,852][122664] Starting process rollout_proc10 [2023-10-10 16:34:01,853][122664] Starting process rollout_proc11 [2023-10-10 16:34:01,866][122664] Starting process rollout_proc12 [2023-10-10 16:34:01,867][122664] Starting process rollout_proc13 [2023-10-10 16:34:02,341][123465] Conv encoder output size: 512 [2023-10-10 16:34:02,344][123465] Created Actor Critic model with architecture: [2023-10-10 16:34:02,344][123465] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): MultiInputEncoder( (encoders): ModuleDict( (obs): ConvEncoder( (enc): RecursiveScriptModule( original_name=ConvEncoderImpl (conv_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Conv2d) (1): RecursiveScriptModule(original_name=ReLU) (2): RecursiveScriptModule(original_name=Conv2d) (3): RecursiveScriptModule(original_name=ReLU) (4): RecursiveScriptModule(original_name=Conv2d) (5): RecursiveScriptModule(original_name=ReLU) ) (mlp_layers): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=ReLU) ) ) ) ) ) (core): ModelCoreIdentity() (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=512, out_features=1, bias=True) (action_parameterization): ActionParameterizationDefault( (distribution_linear): Linear(in_features=512, out_features=6, bias=True) ) ) [2023-10-10 16:34:03,190][123465] Using optimizer [2023-10-10 16:34:03,191][123465] No checkpoints found [2023-10-10 16:34:03,191][123465] Did not load from checkpoint, starting from scratch! [2023-10-10 16:34:03,191][123465] Initialized policy 1 weights for model version 0 [2023-10-10 16:34:03,193][123465] LearnerWorker_p1 finished initialization! [2023-10-10 16:34:03,193][123465] Using GPUs [0] for process 1 (actually maps to GPUs [1]) [2023-10-10 16:34:03,973][122664] Starting process rollout_proc14 [2023-10-10 16:34:03,978][123620] Worker 2 uses CPU cores [4, 5] [2023-10-10 16:34:03,999][122664] Starting process rollout_proc15 [2023-10-10 16:34:04,004][123619] Worker 1 uses CPU cores [2, 3] [2023-10-10 16:34:04,037][123629] Worker 11 uses CPU cores [22, 23] [2023-10-10 16:34:04,213][123615] Worker 0 uses CPU cores [0, 1] [2023-10-10 16:34:04,368][123630] Worker 12 uses CPU cores [24, 25] [2023-10-10 16:34:04,384][123622] Worker 4 uses CPU cores [8, 9] [2023-10-10 16:34:04,406][123623] Worker 5 uses CPU cores [10, 11] [2023-10-10 16:34:04,480][123631] Worker 13 uses CPU cores [26, 27] [2023-10-10 16:34:04,495][123628] Worker 8 uses CPU cores [16, 17] [2023-10-10 16:34:04,601][123582] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-10-10 16:34:04,601][123582] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 [2023-10-10 16:34:04,602][123621] Worker 3 uses CPU cores [6, 7] [2023-10-10 16:34:04,606][123627] Worker 9 uses CPU cores [18, 19] [2023-10-10 16:34:04,623][123582] Num visible devices: 1 [2023-10-10 16:34:04,641][123625] Worker 7 uses CPU cores [14, 15] [2023-10-10 16:34:04,706][123624] Worker 6 uses CPU cores [12, 13] [2023-10-10 16:34:04,710][123626] Worker 10 uses CPU cores [20, 21] [2023-10-10 16:34:04,872][123614] Using GPUs [1] for process 1 (actually maps to GPUs [1]) [2023-10-10 16:34:04,872][123614] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [1]) for inference process 1 [2023-10-10 16:34:04,891][123614] Num visible devices: 1 [2023-10-10 16:34:05,249][123582] RunningMeanStd input shape: (4, 84, 84) [2023-10-10 16:34:05,249][123582] RunningMeanStd input shape: (1,) [2023-10-10 16:34:05,261][123582] ConvEncoder: input_channels=4 [2023-10-10 16:34:05,367][123582] Conv encoder output size: 512 [2023-10-10 16:34:05,515][123614] RunningMeanStd input shape: (4, 84, 84) [2023-10-10 16:34:05,516][123614] RunningMeanStd input shape: (1,) [2023-10-10 16:34:05,527][123614] ConvEncoder: input_channels=4 [2023-10-10 16:34:05,626][123614] Conv encoder output size: 512 [2023-10-10 16:34:05,972][124221] Worker 15 uses CPU cores [30, 31] [2023-10-10 16:34:06,031][122664] Inference worker 0-0 is ready! [2023-10-10 16:34:06,032][122664] Inference worker 1-0 is ready! [2023-10-10 16:34:06,033][122664] All inference workers are ready! Signal rollout workers to start! [2023-10-10 16:34:06,034][124189] Worker 14 uses CPU cores [28, 29] [2023-10-10 16:34:06,034][123625] EnvRunner 7-0 uses policy 1 [2023-10-10 16:34:06,034][123628] EnvRunner 8-0 uses policy 0 [2023-10-10 16:34:06,034][123630] EnvRunner 12-0 uses policy 0 [2023-10-10 16:34:06,034][122664] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan, 1: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-10-10 16:34:06,034][123620] EnvRunner 2-0 uses policy 0 [2023-10-10 16:34:06,034][123615] EnvRunner 0-0 uses policy 0 [2023-10-10 16:34:06,034][123619] EnvRunner 1-0 uses policy 1 [2023-10-10 16:34:06,034][123624] EnvRunner 6-0 uses policy 0 [2023-10-10 16:34:06,034][123629] EnvRunner 11-0 uses policy 1 [2023-10-10 16:34:06,034][123627] EnvRunner 9-0 uses policy 1 [2023-10-10 16:34:06,034][123626] EnvRunner 10-0 uses policy 0 [2023-10-10 16:34:06,035][123622] EnvRunner 4-0 uses policy 0 [2023-10-10 16:34:06,035][123623] EnvRunner 5-0 uses policy 1 [2023-10-10 16:34:06,035][123631] EnvRunner 13-0 uses policy 1 [2023-10-10 16:34:06,035][123621] EnvRunner 3-0 uses policy 1 [2023-10-10 16:34:06,203][124189] EnvRunner 14-0 uses policy 0 [2023-10-10 16:34:06,211][124221] EnvRunner 15-0 uses policy 1 [2023-10-10 16:34:08,330][122664] Heartbeat connected on Batcher_0 [2023-10-10 16:34:08,332][122664] Heartbeat connected on LearnerWorker_p0 [2023-10-10 16:34:08,335][122664] Heartbeat connected on Batcher_1 [2023-10-10 16:34:08,338][122664] Heartbeat connected on LearnerWorker_p1 [2023-10-10 16:34:08,345][122664] Heartbeat connected on InferenceWorker_p0-w0 [2023-10-10 16:34:08,349][122664] Heartbeat connected on InferenceWorker_p1-w0 [2023-10-10 16:34:08,351][122664] Heartbeat connected on RolloutWorker_w0 [2023-10-10 16:34:08,355][122664] Heartbeat connected on RolloutWorker_w2 [2023-10-10 16:34:08,356][122664] Heartbeat connected on RolloutWorker_w1 [2023-10-10 16:34:08,361][122664] Heartbeat connected on RolloutWorker_w4 [2023-10-10 16:34:08,362][122664] Heartbeat connected on RolloutWorker_w3 [2023-10-10 16:34:08,365][122664] Heartbeat connected on RolloutWorker_w5 [2023-10-10 16:34:08,369][122664] Heartbeat connected on RolloutWorker_w7 [2023-10-10 16:34:08,370][122664] Heartbeat connected on RolloutWorker_w6 [2023-10-10 16:34:08,374][122664] Heartbeat connected on RolloutWorker_w8 [2023-10-10 16:34:08,378][122664] Heartbeat connected on RolloutWorker_w9 [2023-10-10 16:34:08,379][122664] Heartbeat connected on RolloutWorker_w10 [2023-10-10 16:34:08,380][122664] Heartbeat connected on RolloutWorker_w11 [2023-10-10 16:34:08,385][122664] Heartbeat connected on RolloutWorker_w12 [2023-10-10 16:34:08,389][122664] Heartbeat connected on RolloutWorker_w14 [2023-10-10 16:34:08,390][122664] Heartbeat connected on RolloutWorker_w13 [2023-10-10 16:34:08,397][122664] Heartbeat connected on RolloutWorker_w15 [2023-10-10 16:34:08,788][122664] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 635.5, 1: 785.8. Samples: 3914. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-10-10 16:34:08,789][122664] Avg episode reward: [(0, '1.444'), (1, '1.000')] [2023-10-10 16:34:13,788][122664] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 1053.7, 1: 1095.7. Samples: 16666. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-10-10 16:34:13,789][122664] Avg episode reward: [(0, '2.804'), (1, '2.814')] [2023-10-10 16:34:15,767][123614] Updated weights for policy 1, policy_version 10 (0.0009) [2023-10-10 16:34:16,079][123582] Updated weights for policy 0, policy_version 10 (0.0008) [2023-10-10 16:34:16,135][123614] Updated weights for policy 1, policy_version 20 (0.0008) [2023-10-10 16:34:16,435][123582] Updated weights for policy 0, policy_version 20 (0.0008) [2023-10-10 16:34:16,501][123614] Updated weights for policy 1, policy_version 30 (0.0007) [2023-10-10 16:34:16,809][123582] Updated weights for policy 0, policy_version 30 (0.0007) [2023-10-10 16:34:18,788][122664] Fps is (10 sec: 6553.7, 60 sec: 5138.6, 300 sec: 5138.6). Total num frames: 65536. Throughput: 0: 1325.3, 1: 1343.0. Samples: 34030. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-10 16:34:18,788][122664] Avg episode reward: [(0, '3.443'), (1, '3.024')] [2023-10-10 16:34:18,939][123614] Updated weights for policy 1, policy_version 40 (0.0010) [2023-10-10 16:34:18,943][123582] Updated weights for policy 0, policy_version 40 (0.0009) [2023-10-10 16:34:19,302][123614] Updated weights for policy 1, policy_version 50 (0.0009) [2023-10-10 16:34:19,312][123582] Updated weights for policy 0, policy_version 50 (0.0008) [2023-10-10 16:34:19,670][123614] Updated weights for policy 1, policy_version 60 (0.0009) [2023-10-10 16:34:19,682][123582] Updated weights for policy 0, policy_version 60 (0.0008) [2023-10-10 16:34:22,948][123582] Updated weights for policy 0, policy_version 70 (0.0009) [2023-10-10 16:34:23,008][123614] Updated weights for policy 1, policy_version 70 (0.0008) [2023-10-10 16:34:23,328][123582] Updated weights for policy 0, policy_version 80 (0.0007) [2023-10-10 16:34:23,376][123614] Updated weights for policy 1, policy_version 80 (0.0008) [2023-10-10 16:34:23,699][123582] Updated weights for policy 0, policy_version 90 (0.0009) [2023-10-10 16:34:23,735][123614] Updated weights for policy 1, policy_version 90 (0.0007) [2023-10-10 16:34:23,788][122664] Fps is (10 sec: 13107.2, 60 sec: 7382.8, 300 sec: 7382.8). Total num frames: 131072. Throughput: 0: 1524.4, 1: 1528.5. Samples: 54200. Policy #0 lag: (min: 33.0, avg: 33.0, max: 33.0) [2023-10-10 16:34:23,789][122664] Avg episode reward: [(0, '3.570'), (1, '3.800')] [2023-10-10 16:34:27,121][123582] Updated weights for policy 0, policy_version 100 (0.0009) [2023-10-10 16:34:27,133][123614] Updated weights for policy 1, policy_version 100 (0.0009) [2023-10-10 16:34:27,489][123582] Updated weights for policy 0, policy_version 110 (0.0008) [2023-10-10 16:34:27,493][123614] Updated weights for policy 1, policy_version 110 (0.0009) [2023-10-10 16:34:27,860][123582] Updated weights for policy 0, policy_version 120 (0.0008) [2023-10-10 16:34:27,861][123614] Updated weights for policy 1, policy_version 120 (0.0008) [2023-10-10 16:34:28,788][122664] Fps is (10 sec: 19660.7, 60 sec: 11521.0, 300 sec: 11521.0). Total num frames: 262144. Throughput: 0: 1446.0, 1: 1455.2. Samples: 66012. Policy #0 lag: (min: 22.0, avg: 30.0, max: 54.0) [2023-10-10 16:34:28,788][122664] Avg episode reward: [(0, '4.030'), (1, '3.510')] [2023-10-10 16:34:28,789][123247] Saving new best policy, reward=4.030! [2023-10-10 16:34:28,789][123465] Saving new best policy, reward=3.510! [2023-10-10 16:34:31,550][123582] Updated weights for policy 0, policy_version 130 (0.0007) [2023-10-10 16:34:31,579][123614] Updated weights for policy 1, policy_version 130 (0.0009) [2023-10-10 16:34:31,915][123582] Updated weights for policy 0, policy_version 140 (0.0009) [2023-10-10 16:34:31,941][123614] Updated weights for policy 1, policy_version 140 (0.0007) [2023-10-10 16:34:32,280][123582] Updated weights for policy 0, policy_version 150 (0.0007) [2023-10-10 16:34:32,306][123614] Updated weights for policy 1, policy_version 150 (0.0007) [2023-10-10 16:34:32,650][123582] Updated weights for policy 0, policy_version 160 (0.0007) [2023-10-10 16:34:32,673][123614] Updated weights for policy 1, policy_version 160 (0.0007) [2023-10-10 16:34:33,788][122664] Fps is (10 sec: 19660.7, 60 sec: 11806.7, 300 sec: 11806.7). Total num frames: 327680. Throughput: 0: 1540.8, 1: 1549.1. Samples: 85758. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-10 16:34:33,789][122664] Avg episode reward: [(0, '4.240'), (1, '3.260')] [2023-10-10 16:34:33,790][123247] Saving new best policy, reward=4.240! [2023-10-10 16:34:36,299][123582] Updated weights for policy 0, policy_version 170 (0.0007) [2023-10-10 16:34:36,553][123614] Updated weights for policy 1, policy_version 170 (0.0009) [2023-10-10 16:34:36,676][123582] Updated weights for policy 0, policy_version 180 (0.0008) [2023-10-10 16:34:36,909][123614] Updated weights for policy 1, policy_version 180 (0.0007) [2023-10-10 16:34:37,033][123582] Updated weights for policy 0, policy_version 190 (0.0008) [2023-10-10 16:34:37,272][123614] Updated weights for policy 1, policy_version 190 (0.0009) [2023-10-10 16:34:38,788][122664] Fps is (10 sec: 13107.1, 60 sec: 12005.2, 300 sec: 12005.2). Total num frames: 393216. Throughput: 0: 1644.8, 1: 1655.3. Samples: 108090. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-10 16:34:38,789][122664] Avg episode reward: [(0, '3.850'), (1, '2.960')] [2023-10-10 16:34:40,664][123582] Updated weights for policy 0, policy_version 200 (0.0008) [2023-10-10 16:34:41,029][123582] Updated weights for policy 0, policy_version 210 (0.0008) [2023-10-10 16:34:41,036][123614] Updated weights for policy 1, policy_version 200 (0.0008) [2023-10-10 16:34:41,394][123582] Updated weights for policy 0, policy_version 220 (0.0009) [2023-10-10 16:34:41,404][123614] Updated weights for policy 1, policy_version 210 (0.0007) [2023-10-10 16:34:41,771][123614] Updated weights for policy 1, policy_version 220 (0.0008) [2023-10-10 16:34:43,788][122664] Fps is (10 sec: 13107.3, 60 sec: 12151.2, 300 sec: 12151.2). Total num frames: 458752. Throughput: 0: 1563.2, 1: 1573.9. Samples: 118438. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:34:43,789][122664] Avg episode reward: [(0, '3.420'), (1, '3.320')] [2023-10-10 16:34:45,191][123582] Updated weights for policy 0, policy_version 230 (0.0008) [2023-10-10 16:34:45,562][123582] Updated weights for policy 0, policy_version 240 (0.0007) [2023-10-10 16:34:45,602][123614] Updated weights for policy 1, policy_version 230 (0.0009) [2023-10-10 16:34:45,919][123582] Updated weights for policy 0, policy_version 250 (0.0007) [2023-10-10 16:34:45,968][123614] Updated weights for policy 1, policy_version 240 (0.0008) [2023-10-10 16:34:46,324][123614] Updated weights for policy 1, policy_version 250 (0.0007) [2023-10-10 16:34:48,788][122664] Fps is (10 sec: 13107.3, 60 sec: 12263.0, 300 sec: 12263.0). Total num frames: 524288. Throughput: 0: 1640.6, 1: 1637.2. Samples: 140140. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-10 16:34:48,789][122664] Avg episode reward: [(0, '4.060'), (1, '3.580')] [2023-10-10 16:34:48,789][123465] Saving new best policy, reward=3.580! [2023-10-10 16:34:49,708][123582] Updated weights for policy 0, policy_version 260 (0.0007) [2023-10-10 16:34:50,081][123582] Updated weights for policy 0, policy_version 270 (0.0008) [2023-10-10 16:34:50,176][123614] Updated weights for policy 1, policy_version 260 (0.0007) [2023-10-10 16:34:50,451][123582] Updated weights for policy 0, policy_version 280 (0.0009) [2023-10-10 16:34:50,544][123614] Updated weights for policy 1, policy_version 270 (0.0007) [2023-10-10 16:34:50,906][123614] Updated weights for policy 1, policy_version 280 (0.0007) [2023-10-10 16:34:53,788][122664] Fps is (10 sec: 13107.2, 60 sec: 12351.4, 300 sec: 12351.4). Total num frames: 589824. Throughput: 0: 1773.2, 1: 1756.6. Samples: 162758. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-10-10 16:34:53,789][122664] Avg episode reward: [(0, '3.870'), (1, '3.740')] [2023-10-10 16:34:53,794][123465] Saving new best policy, reward=3.740! [2023-10-10 16:34:54,054][123582] Updated weights for policy 0, policy_version 290 (0.0007) [2023-10-10 16:34:54,423][123582] Updated weights for policy 0, policy_version 300 (0.0010) [2023-10-10 16:34:54,754][123614] Updated weights for policy 1, policy_version 290 (0.0007) [2023-10-10 16:34:54,789][123582] Updated weights for policy 0, policy_version 310 (0.0007) [2023-10-10 16:34:55,110][123614] Updated weights for policy 1, policy_version 300 (0.0007) [2023-10-10 16:34:55,158][123582] Updated weights for policy 0, policy_version 320 (0.0008) [2023-10-10 16:34:55,474][123614] Updated weights for policy 1, policy_version 310 (0.0007) [2023-10-10 16:34:55,840][123614] Updated weights for policy 1, policy_version 320 (0.0008) [2023-10-10 16:34:58,788][122664] Fps is (10 sec: 13107.0, 60 sec: 12423.0, 300 sec: 12423.0). Total num frames: 655360. Throughput: 0: 1745.0, 1: 1728.4. Samples: 172966. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-10-10 16:34:58,789][122664] Avg episode reward: [(0, '3.340'), (1, '4.000')] [2023-10-10 16:34:58,790][123465] Saving new best policy, reward=4.000! [2023-10-10 16:34:58,945][123582] Updated weights for policy 0, policy_version 330 (0.0008) [2023-10-10 16:34:59,323][123582] Updated weights for policy 0, policy_version 340 (0.0008) [2023-10-10 16:34:59,440][123614] Updated weights for policy 1, policy_version 330 (0.0007) [2023-10-10 16:34:59,693][123582] Updated weights for policy 0, policy_version 350 (0.0009) [2023-10-10 16:34:59,809][123614] Updated weights for policy 1, policy_version 340 (0.0009) [2023-10-10 16:35:00,177][123614] Updated weights for policy 1, policy_version 350 (0.0009) [2023-10-10 16:35:03,500][123582] Updated weights for policy 0, policy_version 360 (0.0009) [2023-10-10 16:35:03,788][122664] Fps is (10 sec: 13106.9, 60 sec: 12482.2, 300 sec: 12482.2). Total num frames: 720896. Throughput: 0: 1797.5, 1: 1789.1. Samples: 195424. Policy #0 lag: (min: 26.0, avg: 27.0, max: 48.0) [2023-10-10 16:35:03,789][122664] Avg episode reward: [(0, '3.530'), (1, '3.580')] [2023-10-10 16:35:03,868][123582] Updated weights for policy 0, policy_version 370 (0.0008) [2023-10-10 16:35:03,960][123614] Updated weights for policy 1, policy_version 360 (0.0007) [2023-10-10 16:35:04,230][123582] Updated weights for policy 0, policy_version 380 (0.0009) [2023-10-10 16:35:04,332][123614] Updated weights for policy 1, policy_version 370 (0.0008) [2023-10-10 16:35:04,703][123614] Updated weights for policy 1, policy_version 380 (0.0009) [2023-10-10 16:35:07,946][123582] Updated weights for policy 0, policy_version 390 (0.0009) [2023-10-10 16:35:08,316][123582] Updated weights for policy 0, policy_version 400 (0.0010) [2023-10-10 16:35:08,368][123614] Updated weights for policy 1, policy_version 390 (0.0008) [2023-10-10 16:35:08,686][123582] Updated weights for policy 0, policy_version 410 (0.0007) [2023-10-10 16:35:08,739][123614] Updated weights for policy 1, policy_version 400 (0.0007) [2023-10-10 16:35:08,788][122664] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 12532.1). Total num frames: 786432. Throughput: 0: 1801.1, 1: 1800.3. Samples: 216262. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:35:08,788][122664] Avg episode reward: [(0, '3.220'), (1, '3.480')] [2023-10-10 16:35:09,104][123614] Updated weights for policy 1, policy_version 410 (0.0008) [2023-10-10 16:35:12,399][123582] Updated weights for policy 0, policy_version 420 (0.0008) [2023-10-10 16:35:12,770][123582] Updated weights for policy 0, policy_version 430 (0.0007) [2023-10-10 16:35:12,784][123614] Updated weights for policy 1, policy_version 420 (0.0009) [2023-10-10 16:35:13,138][123582] Updated weights for policy 0, policy_version 440 (0.0008) [2023-10-10 16:35:13,144][123614] Updated weights for policy 1, policy_version 430 (0.0008) [2023-10-10 16:35:13,497][123614] Updated weights for policy 1, policy_version 440 (0.0007) [2023-10-10 16:35:13,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 13058.1). Total num frames: 884736. Throughput: 0: 1797.0, 1: 1791.1. Samples: 227476. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-10-10 16:35:13,789][122664] Avg episode reward: [(0, '3.750'), (1, '3.660')] [2023-10-10 16:35:16,800][123582] Updated weights for policy 0, policy_version 450 (0.0008) [2023-10-10 16:35:17,165][123582] Updated weights for policy 0, policy_version 460 (0.0008) [2023-10-10 16:35:17,237][123614] Updated weights for policy 1, policy_version 450 (0.0009) [2023-10-10 16:35:17,526][123582] Updated weights for policy 0, policy_version 470 (0.0010) [2023-10-10 16:35:17,611][123614] Updated weights for policy 1, policy_version 460 (0.0008) [2023-10-10 16:35:17,894][123582] Updated weights for policy 0, policy_version 480 (0.0009) [2023-10-10 16:35:17,982][123614] Updated weights for policy 1, policy_version 470 (0.0008) [2023-10-10 16:35:18,339][123614] Updated weights for policy 1, policy_version 480 (0.0007) [2023-10-10 16:35:18,788][122664] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 13511.9). Total num frames: 983040. Throughput: 0: 1815.8, 1: 1810.8. Samples: 248956. Policy #0 lag: (min: 1.0, avg: 12.8, max: 33.0) [2023-10-10 16:35:18,789][122664] Avg episode reward: [(0, '4.370'), (1, '3.930')] [2023-10-10 16:35:18,790][123247] Saving new best policy, reward=4.370! [2023-10-10 16:35:21,634][123582] Updated weights for policy 0, policy_version 490 (0.0010) [2023-10-10 16:35:22,014][123582] Updated weights for policy 0, policy_version 500 (0.0007) [2023-10-10 16:35:22,157][123614] Updated weights for policy 1, policy_version 490 (0.0009) [2023-10-10 16:35:22,387][123582] Updated weights for policy 0, policy_version 510 (0.0008) [2023-10-10 16:35:22,528][123614] Updated weights for policy 1, policy_version 500 (0.0008) [2023-10-10 16:35:22,893][123614] Updated weights for policy 1, policy_version 510 (0.0009) [2023-10-10 16:35:23,788][122664] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 13485.9). Total num frames: 1048576. Throughput: 0: 1809.8, 1: 1793.2. Samples: 270228. Policy #0 lag: (min: 26.0, avg: 27.8, max: 55.0) [2023-10-10 16:35:23,789][122664] Avg episode reward: [(0, '4.360'), (1, '4.290')] [2023-10-10 16:35:23,797][123465] Saving new best policy, reward=4.290! [2023-10-10 16:35:26,048][123582] Updated weights for policy 0, policy_version 520 (0.0007) [2023-10-10 16:35:26,406][123582] Updated weights for policy 0, policy_version 530 (0.0008) [2023-10-10 16:35:26,600][123614] Updated weights for policy 1, policy_version 520 (0.0007) [2023-10-10 16:35:26,779][123582] Updated weights for policy 0, policy_version 540 (0.0007) [2023-10-10 16:35:26,967][123614] Updated weights for policy 1, policy_version 530 (0.0008) [2023-10-10 16:35:27,328][123614] Updated weights for policy 1, policy_version 540 (0.0011) [2023-10-10 16:35:28,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13463.0). Total num frames: 1114112. Throughput: 0: 1819.8, 1: 1806.7. Samples: 281630. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:35:28,789][122664] Avg episode reward: [(0, '3.540'), (1, '3.790')] [2023-10-10 16:35:30,442][123582] Updated weights for policy 0, policy_version 550 (0.0009) [2023-10-10 16:35:30,816][123582] Updated weights for policy 0, policy_version 560 (0.0008) [2023-10-10 16:35:31,003][123614] Updated weights for policy 1, policy_version 550 (0.0010) [2023-10-10 16:35:31,189][123582] Updated weights for policy 0, policy_version 570 (0.0007) [2023-10-10 16:35:31,365][123614] Updated weights for policy 1, policy_version 560 (0.0007) [2023-10-10 16:35:31,738][123614] Updated weights for policy 1, policy_version 570 (0.0009) [2023-10-10 16:35:33,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13442.7). Total num frames: 1179648. Throughput: 0: 1808.4, 1: 1802.9. Samples: 302650. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:35:33,789][122664] Avg episode reward: [(0, '3.940'), (1, '3.690')] [2023-10-10 16:35:35,073][123582] Updated weights for policy 0, policy_version 580 (0.0009) [2023-10-10 16:35:35,380][123614] Updated weights for policy 1, policy_version 580 (0.0007) [2023-10-10 16:35:35,441][123582] Updated weights for policy 0, policy_version 590 (0.0008) [2023-10-10 16:35:35,739][123614] Updated weights for policy 1, policy_version 590 (0.0008) [2023-10-10 16:35:35,819][123582] Updated weights for policy 0, policy_version 600 (0.0007) [2023-10-10 16:35:36,116][123614] Updated weights for policy 1, policy_version 600 (0.0007) [2023-10-10 16:35:38,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13424.6). Total num frames: 1245184. Throughput: 0: 1800.0, 1: 1812.2. Samples: 325304. Policy #0 lag: (min: 17.0, avg: 21.2, max: 49.0) [2023-10-10 16:35:38,789][122664] Avg episode reward: [(0, '3.840'), (1, '4.250')] [2023-10-10 16:35:39,598][123582] Updated weights for policy 0, policy_version 610 (0.0007) [2023-10-10 16:35:39,788][123614] Updated weights for policy 1, policy_version 610 (0.0008) [2023-10-10 16:35:39,970][123582] Updated weights for policy 0, policy_version 620 (0.0010) [2023-10-10 16:35:40,151][123614] Updated weights for policy 1, policy_version 620 (0.0008) [2023-10-10 16:35:40,343][123582] Updated weights for policy 0, policy_version 630 (0.0009) [2023-10-10 16:35:40,513][123614] Updated weights for policy 1, policy_version 630 (0.0007) [2023-10-10 16:35:40,710][123582] Updated weights for policy 0, policy_version 640 (0.0010) [2023-10-10 16:35:40,881][123614] Updated weights for policy 1, policy_version 640 (0.0008) [2023-10-10 16:35:43,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13408.4). Total num frames: 1310720. Throughput: 0: 1792.6, 1: 1807.1. Samples: 334954. Policy #0 lag: (min: 28.0, avg: 35.4, max: 60.0) [2023-10-10 16:35:43,789][122664] Avg episode reward: [(0, '3.560'), (1, '4.220')] [2023-10-10 16:35:44,446][123582] Updated weights for policy 0, policy_version 650 (0.0007) [2023-10-10 16:35:44,571][123614] Updated weights for policy 1, policy_version 650 (0.0009) [2023-10-10 16:35:44,813][123582] Updated weights for policy 0, policy_version 660 (0.0008) [2023-10-10 16:35:44,943][123614] Updated weights for policy 1, policy_version 660 (0.0008) [2023-10-10 16:35:45,181][123582] Updated weights for policy 0, policy_version 670 (0.0008) [2023-10-10 16:35:45,318][123614] Updated weights for policy 1, policy_version 670 (0.0009) [2023-10-10 16:35:48,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13393.7). Total num frames: 1376256. Throughput: 0: 1795.7, 1: 1802.8. Samples: 357356. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 16:35:48,789][122664] Avg episode reward: [(0, '3.830'), (1, '3.850')] [2023-10-10 16:35:49,075][123582] Updated weights for policy 0, policy_version 680 (0.0008) [2023-10-10 16:35:49,277][123614] Updated weights for policy 1, policy_version 680 (0.0009) [2023-10-10 16:35:49,442][123582] Updated weights for policy 0, policy_version 690 (0.0010) [2023-10-10 16:35:49,663][123614] Updated weights for policy 1, policy_version 690 (0.0009) [2023-10-10 16:35:49,811][123582] Updated weights for policy 0, policy_version 700 (0.0008) [2023-10-10 16:35:50,030][123614] Updated weights for policy 1, policy_version 700 (0.0009) [2023-10-10 16:35:53,393][123582] Updated weights for policy 0, policy_version 710 (0.0008) [2023-10-10 16:35:53,573][123614] Updated weights for policy 1, policy_version 710 (0.0008) [2023-10-10 16:35:53,758][123582] Updated weights for policy 0, policy_version 720 (0.0007) [2023-10-10 16:35:53,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13380.4). Total num frames: 1441792. Throughput: 0: 1811.6, 1: 1805.3. Samples: 379024. Policy #0 lag: (min: 15.0, avg: 18.4, max: 47.0) [2023-10-10 16:35:53,789][122664] Avg episode reward: [(0, '4.230'), (1, '3.880')] [2023-10-10 16:35:53,938][123614] Updated weights for policy 1, policy_version 720 (0.0007) [2023-10-10 16:35:54,124][123582] Updated weights for policy 0, policy_version 730 (0.0007) [2023-10-10 16:35:54,292][123614] Updated weights for policy 1, policy_version 730 (0.0008) [2023-10-10 16:35:54,343][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000000736_753664.pth... [2023-10-10 16:35:54,509][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000000736_753664.pth... [2023-10-10 16:35:57,850][123582] Updated weights for policy 0, policy_version 740 (0.0009) [2023-10-10 16:35:58,061][123614] Updated weights for policy 1, policy_version 740 (0.0010) [2023-10-10 16:35:58,219][123582] Updated weights for policy 0, policy_version 750 (0.0008) [2023-10-10 16:35:58,422][123614] Updated weights for policy 1, policy_version 750 (0.0008) [2023-10-10 16:35:58,584][123582] Updated weights for policy 0, policy_version 760 (0.0007) [2023-10-10 16:35:58,787][123614] Updated weights for policy 1, policy_version 760 (0.0008) [2023-10-10 16:35:58,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13368.3). Total num frames: 1507328. Throughput: 0: 1800.5, 1: 1798.9. Samples: 389450. Policy #0 lag: (min: 4.0, avg: 8.3, max: 36.0) [2023-10-10 16:35:58,788][122664] Avg episode reward: [(0, '4.200'), (1, '3.510')] [2023-10-10 16:36:02,316][123582] Updated weights for policy 0, policy_version 770 (0.0008) [2023-10-10 16:36:02,586][123614] Updated weights for policy 1, policy_version 770 (0.0008) [2023-10-10 16:36:02,686][123582] Updated weights for policy 0, policy_version 780 (0.0007) [2023-10-10 16:36:02,948][123614] Updated weights for policy 1, policy_version 780 (0.0007) [2023-10-10 16:36:03,056][123582] Updated weights for policy 0, policy_version 790 (0.0008) [2023-10-10 16:36:03,314][123614] Updated weights for policy 1, policy_version 790 (0.0008) [2023-10-10 16:36:03,422][123582] Updated weights for policy 0, policy_version 800 (0.0010) [2023-10-10 16:36:03,686][123614] Updated weights for policy 1, policy_version 800 (0.0007) [2023-10-10 16:36:03,788][122664] Fps is (10 sec: 19660.7, 60 sec: 15291.8, 300 sec: 13913.8). Total num frames: 1638400. Throughput: 0: 1808.5, 1: 1807.9. Samples: 411692. Policy #0 lag: (min: 15.0, avg: 15.2, max: 25.0) [2023-10-10 16:36:03,789][122664] Avg episode reward: [(0, '4.930'), (1, '4.160')] [2023-10-10 16:36:03,790][123247] Saving new best policy, reward=4.930! [2023-10-10 16:36:07,045][123582] Updated weights for policy 0, policy_version 810 (0.0010) [2023-10-10 16:36:07,421][123582] Updated weights for policy 0, policy_version 820 (0.0008) [2023-10-10 16:36:07,523][123614] Updated weights for policy 1, policy_version 810 (0.0007) [2023-10-10 16:36:07,794][123582] Updated weights for policy 0, policy_version 830 (0.0008) [2023-10-10 16:36:07,886][123614] Updated weights for policy 1, policy_version 820 (0.0008) [2023-10-10 16:36:08,254][123614] Updated weights for policy 1, policy_version 830 (0.0008) [2023-10-10 16:36:08,788][122664] Fps is (10 sec: 19660.3, 60 sec: 15291.7, 300 sec: 13880.9). Total num frames: 1703936. Throughput: 0: 1796.3, 1: 1797.5. Samples: 431946. Policy #0 lag: (min: 12.0, avg: 18.7, max: 44.0) [2023-10-10 16:36:08,789][122664] Avg episode reward: [(0, '4.640'), (1, '4.100')] [2023-10-10 16:36:11,491][123582] Updated weights for policy 0, policy_version 840 (0.0010) [2023-10-10 16:36:11,858][123582] Updated weights for policy 0, policy_version 850 (0.0010) [2023-10-10 16:36:11,922][123614] Updated weights for policy 1, policy_version 840 (0.0008) [2023-10-10 16:36:12,231][123582] Updated weights for policy 0, policy_version 860 (0.0008) [2023-10-10 16:36:12,283][123614] Updated weights for policy 1, policy_version 850 (0.0009) [2023-10-10 16:36:12,644][123614] Updated weights for policy 1, policy_version 860 (0.0008) [2023-10-10 16:36:13,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 13850.6). Total num frames: 1769472. Throughput: 0: 1806.8, 1: 1806.0. Samples: 444206. Policy #0 lag: (min: 15.0, avg: 36.3, max: 40.0) [2023-10-10 16:36:13,789][122664] Avg episode reward: [(0, '4.490'), (1, '3.820')] [2023-10-10 16:36:16,049][123582] Updated weights for policy 0, policy_version 870 (0.0007) [2023-10-10 16:36:16,419][123582] Updated weights for policy 0, policy_version 880 (0.0007) [2023-10-10 16:36:16,548][123614] Updated weights for policy 1, policy_version 870 (0.0010) [2023-10-10 16:36:16,793][123582] Updated weights for policy 0, policy_version 890 (0.0008) [2023-10-10 16:36:16,920][123614] Updated weights for policy 1, policy_version 880 (0.0007) [2023-10-10 16:36:17,287][123614] Updated weights for policy 1, policy_version 890 (0.0007) [2023-10-10 16:36:18,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13822.6). Total num frames: 1835008. Throughput: 0: 1796.7, 1: 1796.4. Samples: 464340. Policy #0 lag: (min: 16.0, avg: 35.1, max: 48.0) [2023-10-10 16:36:18,789][122664] Avg episode reward: [(0, '4.610'), (1, '4.390')] [2023-10-10 16:36:18,791][123465] Saving new best policy, reward=4.390! [2023-10-10 16:36:20,485][123582] Updated weights for policy 0, policy_version 900 (0.0007) [2023-10-10 16:36:20,833][123614] Updated weights for policy 1, policy_version 900 (0.0007) [2023-10-10 16:36:20,862][123582] Updated weights for policy 0, policy_version 910 (0.0008) [2023-10-10 16:36:21,200][123614] Updated weights for policy 1, policy_version 910 (0.0007) [2023-10-10 16:36:21,235][123582] Updated weights for policy 0, policy_version 920 (0.0008) [2023-10-10 16:36:21,565][123614] Updated weights for policy 1, policy_version 920 (0.0008) [2023-10-10 16:36:23,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13796.7). Total num frames: 1900544. Throughput: 0: 1801.3, 1: 1790.6. Samples: 486942. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-10 16:36:23,789][122664] Avg episode reward: [(0, '4.760'), (1, '4.860')] [2023-10-10 16:36:23,798][123465] Saving new best policy, reward=4.860! [2023-10-10 16:36:25,056][123582] Updated weights for policy 0, policy_version 930 (0.0007) [2023-10-10 16:36:25,352][123614] Updated weights for policy 1, policy_version 930 (0.0010) [2023-10-10 16:36:25,418][123582] Updated weights for policy 0, policy_version 940 (0.0008) [2023-10-10 16:36:25,725][123614] Updated weights for policy 1, policy_version 940 (0.0010) [2023-10-10 16:36:25,788][123582] Updated weights for policy 0, policy_version 950 (0.0007) [2023-10-10 16:36:26,092][123614] Updated weights for policy 1, policy_version 950 (0.0007) [2023-10-10 16:36:26,156][123582] Updated weights for policy 0, policy_version 960 (0.0007) [2023-10-10 16:36:26,470][123614] Updated weights for policy 1, policy_version 960 (0.0007) [2023-10-10 16:36:28,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13772.5). Total num frames: 1966080. Throughput: 0: 1801.3, 1: 1793.4. Samples: 496714. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:36:28,788][122664] Avg episode reward: [(0, '5.140'), (1, '4.470')] [2023-10-10 16:36:28,789][123247] Saving new best policy, reward=5.140! [2023-10-10 16:36:29,812][123582] Updated weights for policy 0, policy_version 970 (0.0008) [2023-10-10 16:36:30,104][123614] Updated weights for policy 1, policy_version 970 (0.0008) [2023-10-10 16:36:30,181][123582] Updated weights for policy 0, policy_version 980 (0.0007) [2023-10-10 16:36:30,465][123614] Updated weights for policy 1, policy_version 980 (0.0010) [2023-10-10 16:36:30,564][123582] Updated weights for policy 0, policy_version 990 (0.0007) [2023-10-10 16:36:30,839][123614] Updated weights for policy 1, policy_version 990 (0.0010) [2023-10-10 16:36:33,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13750.0). Total num frames: 2031616. Throughput: 0: 1802.0, 1: 1801.2. Samples: 519504. Policy #0 lag: (min: 21.0, avg: 26.1, max: 53.0) [2023-10-10 16:36:33,789][122664] Avg episode reward: [(0, '4.870'), (1, '4.410')] [2023-10-10 16:36:34,302][123582] Updated weights for policy 0, policy_version 1000 (0.0009) [2023-10-10 16:36:34,663][123582] Updated weights for policy 0, policy_version 1010 (0.0009) [2023-10-10 16:36:34,725][123614] Updated weights for policy 1, policy_version 1000 (0.0009) [2023-10-10 16:36:35,040][123582] Updated weights for policy 0, policy_version 1020 (0.0009) [2023-10-10 16:36:35,104][123614] Updated weights for policy 1, policy_version 1010 (0.0007) [2023-10-10 16:36:35,470][123614] Updated weights for policy 1, policy_version 1020 (0.0011) [2023-10-10 16:36:38,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 13729.0). Total num frames: 2097152. Throughput: 0: 1805.6, 1: 1814.7. Samples: 541936. Policy #0 lag: (min: 13.0, avg: 16.3, max: 45.0) [2023-10-10 16:36:38,789][122664] Avg episode reward: [(0, '4.680'), (1, '4.680')] [2023-10-10 16:36:38,843][123582] Updated weights for policy 0, policy_version 1030 (0.0007) [2023-10-10 16:36:38,942][123614] Updated weights for policy 1, policy_version 1030 (0.0008) [2023-10-10 16:36:39,213][123582] Updated weights for policy 0, policy_version 1040 (0.0009) [2023-10-10 16:36:39,308][123614] Updated weights for policy 1, policy_version 1040 (0.0007) [2023-10-10 16:36:39,580][123582] Updated weights for policy 0, policy_version 1050 (0.0007) [2023-10-10 16:36:39,678][123614] Updated weights for policy 1, policy_version 1050 (0.0007) [2023-10-10 16:36:43,403][123582] Updated weights for policy 0, policy_version 1060 (0.0007) [2023-10-10 16:36:43,430][123614] Updated weights for policy 1, policy_version 1060 (0.0008) [2023-10-10 16:36:43,766][123582] Updated weights for policy 0, policy_version 1070 (0.0007) [2023-10-10 16:36:43,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13709.3). Total num frames: 2162688. Throughput: 0: 1797.6, 1: 1807.7. Samples: 551690. Policy #0 lag: (min: 3.0, avg: 7.6, max: 35.0) [2023-10-10 16:36:43,788][122664] Avg episode reward: [(0, '4.930'), (1, '4.270')] [2023-10-10 16:36:43,801][123614] Updated weights for policy 1, policy_version 1070 (0.0008) [2023-10-10 16:36:44,145][123582] Updated weights for policy 0, policy_version 1080 (0.0007) [2023-10-10 16:36:44,170][123614] Updated weights for policy 1, policy_version 1080 (0.0009) [2023-10-10 16:36:47,866][123582] Updated weights for policy 0, policy_version 1090 (0.0010) [2023-10-10 16:36:48,027][123614] Updated weights for policy 1, policy_version 1090 (0.0009) [2023-10-10 16:36:48,232][123582] Updated weights for policy 0, policy_version 1100 (0.0009) [2023-10-10 16:36:48,394][123614] Updated weights for policy 1, policy_version 1100 (0.0008) [2023-10-10 16:36:48,605][123582] Updated weights for policy 0, policy_version 1110 (0.0007) [2023-10-10 16:36:48,761][123614] Updated weights for policy 1, policy_version 1110 (0.0008) [2023-10-10 16:36:48,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13690.8). Total num frames: 2228224. Throughput: 0: 1805.6, 1: 1802.8. Samples: 574072. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-10-10 16:36:48,789][122664] Avg episode reward: [(0, '5.100'), (1, '3.960')] [2023-10-10 16:36:48,981][123582] Updated weights for policy 0, policy_version 1120 (0.0007) [2023-10-10 16:36:49,135][123614] Updated weights for policy 1, policy_version 1120 (0.0008) [2023-10-10 16:36:52,658][123582] Updated weights for policy 0, policy_version 1130 (0.0009) [2023-10-10 16:36:52,927][123614] Updated weights for policy 1, policy_version 1130 (0.0008) [2023-10-10 16:36:53,027][123582] Updated weights for policy 0, policy_version 1140 (0.0007) [2023-10-10 16:36:53,292][123614] Updated weights for policy 1, policy_version 1140 (0.0009) [2023-10-10 16:36:53,394][123582] Updated weights for policy 0, policy_version 1150 (0.0007) [2023-10-10 16:36:53,647][123614] Updated weights for policy 1, policy_version 1150 (0.0009) [2023-10-10 16:36:53,788][122664] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14064.0). Total num frames: 2359296. Throughput: 0: 1798.4, 1: 1801.1. Samples: 593922. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 16:36:53,789][122664] Avg episode reward: [(0, '5.100'), (1, '4.590')] [2023-10-10 16:36:57,187][123582] Updated weights for policy 0, policy_version 1160 (0.0008) [2023-10-10 16:36:57,394][123614] Updated weights for policy 1, policy_version 1160 (0.0007) [2023-10-10 16:36:57,558][123582] Updated weights for policy 0, policy_version 1170 (0.0008) [2023-10-10 16:36:57,769][123614] Updated weights for policy 1, policy_version 1170 (0.0008) [2023-10-10 16:36:57,927][123582] Updated weights for policy 0, policy_version 1180 (0.0010) [2023-10-10 16:36:58,133][123614] Updated weights for policy 1, policy_version 1180 (0.0007) [2023-10-10 16:36:58,788][122664] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14036.4). Total num frames: 2424832. Throughput: 0: 1802.0, 1: 1804.2. Samples: 606484. Policy #0 lag: (min: 31.0, avg: 32.4, max: 56.0) [2023-10-10 16:36:58,789][122664] Avg episode reward: [(0, '4.990'), (1, '5.090')] [2023-10-10 16:36:58,789][123465] Saving new best policy, reward=5.090! [2023-10-10 16:37:01,657][123582] Updated weights for policy 0, policy_version 1190 (0.0007) [2023-10-10 16:37:01,816][123614] Updated weights for policy 1, policy_version 1190 (0.0008) [2023-10-10 16:37:02,029][123582] Updated weights for policy 0, policy_version 1200 (0.0008) [2023-10-10 16:37:02,194][123614] Updated weights for policy 1, policy_version 1200 (0.0010) [2023-10-10 16:37:02,399][123582] Updated weights for policy 0, policy_version 1210 (0.0007) [2023-10-10 16:37:02,568][123614] Updated weights for policy 1, policy_version 1210 (0.0008) [2023-10-10 16:37:03,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14010.2). Total num frames: 2490368. Throughput: 0: 1802.4, 1: 1805.7. Samples: 626706. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:37:03,789][122664] Avg episode reward: [(0, '4.810'), (1, '5.240')] [2023-10-10 16:37:03,790][123465] Saving new best policy, reward=5.240! [2023-10-10 16:37:06,117][123582] Updated weights for policy 0, policy_version 1220 (0.0009) [2023-10-10 16:37:06,352][123614] Updated weights for policy 1, policy_version 1220 (0.0008) [2023-10-10 16:37:06,492][123582] Updated weights for policy 0, policy_version 1230 (0.0008) [2023-10-10 16:37:06,716][123614] Updated weights for policy 1, policy_version 1230 (0.0007) [2023-10-10 16:37:06,857][123582] Updated weights for policy 0, policy_version 1240 (0.0008) [2023-10-10 16:37:07,094][123614] Updated weights for policy 1, policy_version 1240 (0.0008) [2023-10-10 16:37:08,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 13985.5). Total num frames: 2555904. Throughput: 0: 1797.1, 1: 1799.6. Samples: 648794. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) [2023-10-10 16:37:08,789][122664] Avg episode reward: [(0, '4.790'), (1, '5.030')] [2023-10-10 16:37:10,604][123582] Updated weights for policy 0, policy_version 1250 (0.0008) [2023-10-10 16:37:10,823][123614] Updated weights for policy 1, policy_version 1250 (0.0010) [2023-10-10 16:37:10,970][123582] Updated weights for policy 0, policy_version 1260 (0.0008) [2023-10-10 16:37:11,193][123614] Updated weights for policy 1, policy_version 1260 (0.0009) [2023-10-10 16:37:11,335][123582] Updated weights for policy 0, policy_version 1270 (0.0008) [2023-10-10 16:37:11,565][123614] Updated weights for policy 1, policy_version 1270 (0.0008) [2023-10-10 16:37:11,710][123582] Updated weights for policy 0, policy_version 1280 (0.0008) [2023-10-10 16:37:11,932][123614] Updated weights for policy 1, policy_version 1280 (0.0009) [2023-10-10 16:37:13,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13962.1). Total num frames: 2621440. Throughput: 0: 1806.8, 1: 1805.3. Samples: 659260. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:37:13,789][122664] Avg episode reward: [(0, '5.510'), (1, '5.030')] [2023-10-10 16:37:13,790][123247] Saving new best policy, reward=5.510! [2023-10-10 16:37:15,416][123582] Updated weights for policy 0, policy_version 1290 (0.0010) [2023-10-10 16:37:15,735][123614] Updated weights for policy 1, policy_version 1290 (0.0008) [2023-10-10 16:37:15,779][123582] Updated weights for policy 0, policy_version 1300 (0.0009) [2023-10-10 16:37:16,099][123614] Updated weights for policy 1, policy_version 1300 (0.0008) [2023-10-10 16:37:16,149][123582] Updated weights for policy 0, policy_version 1310 (0.0010) [2023-10-10 16:37:16,463][123614] Updated weights for policy 1, policy_version 1310 (0.0010) [2023-10-10 16:37:18,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13939.9). Total num frames: 2686976. Throughput: 0: 1800.4, 1: 1795.3. Samples: 681310. Policy #0 lag: (min: 26.0, avg: 26.7, max: 44.0) [2023-10-10 16:37:18,788][122664] Avg episode reward: [(0, '5.760'), (1, '4.600')] [2023-10-10 16:37:18,789][123247] Saving new best policy, reward=5.760! [2023-10-10 16:37:19,940][123582] Updated weights for policy 0, policy_version 1320 (0.0010) [2023-10-10 16:37:20,300][123614] Updated weights for policy 1, policy_version 1320 (0.0009) [2023-10-10 16:37:20,326][123582] Updated weights for policy 0, policy_version 1330 (0.0009) [2023-10-10 16:37:20,668][123614] Updated weights for policy 1, policy_version 1330 (0.0009) [2023-10-10 16:37:20,702][123582] Updated weights for policy 0, policy_version 1340 (0.0008) [2023-10-10 16:37:21,041][123614] Updated weights for policy 1, policy_version 1340 (0.0008) [2023-10-10 16:37:23,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13918.9). Total num frames: 2752512. Throughput: 0: 1799.7, 1: 1797.1. Samples: 703790. Policy #0 lag: (min: 28.0, avg: 34.5, max: 60.0) [2023-10-10 16:37:23,788][122664] Avg episode reward: [(0, '5.540'), (1, '4.610')] [2023-10-10 16:37:24,382][123582] Updated weights for policy 0, policy_version 1350 (0.0008) [2023-10-10 16:37:24,523][123614] Updated weights for policy 1, policy_version 1350 (0.0008) [2023-10-10 16:37:24,758][123582] Updated weights for policy 0, policy_version 1360 (0.0008) [2023-10-10 16:37:24,889][123614] Updated weights for policy 1, policy_version 1360 (0.0009) [2023-10-10 16:37:25,119][123582] Updated weights for policy 0, policy_version 1370 (0.0009) [2023-10-10 16:37:25,256][123614] Updated weights for policy 1, policy_version 1370 (0.0010) [2023-10-10 16:37:28,753][123582] Updated weights for policy 0, policy_version 1380 (0.0010) [2023-10-10 16:37:28,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13898.9). Total num frames: 2818048. Throughput: 0: 1803.5, 1: 1794.8. Samples: 713616. Policy #0 lag: (min: 10.0, avg: 10.2, max: 19.0) [2023-10-10 16:37:28,789][122664] Avg episode reward: [(0, '5.560'), (1, '4.510')] [2023-10-10 16:37:29,125][123582] Updated weights for policy 0, policy_version 1390 (0.0008) [2023-10-10 16:37:29,158][123614] Updated weights for policy 1, policy_version 1380 (0.0007) [2023-10-10 16:37:29,504][123582] Updated weights for policy 0, policy_version 1400 (0.0009) [2023-10-10 16:37:29,520][123614] Updated weights for policy 1, policy_version 1390 (0.0007) [2023-10-10 16:37:29,890][123614] Updated weights for policy 1, policy_version 1400 (0.0010) [2023-10-10 16:37:33,279][123582] Updated weights for policy 0, policy_version 1410 (0.0009) [2023-10-10 16:37:33,479][123614] Updated weights for policy 1, policy_version 1410 (0.0008) [2023-10-10 16:37:33,656][123582] Updated weights for policy 0, policy_version 1420 (0.0008) [2023-10-10 16:37:33,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 13879.8). Total num frames: 2883584. Throughput: 0: 1800.5, 1: 1801.6. Samples: 736170. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-10 16:37:33,789][122664] Avg episode reward: [(0, '5.670'), (1, '4.340')] [2023-10-10 16:37:33,851][123614] Updated weights for policy 1, policy_version 1420 (0.0007) [2023-10-10 16:37:34,020][123582] Updated weights for policy 0, policy_version 1430 (0.0007) [2023-10-10 16:37:34,214][123614] Updated weights for policy 1, policy_version 1430 (0.0007) [2023-10-10 16:37:34,390][123582] Updated weights for policy 0, policy_version 1440 (0.0007) [2023-10-10 16:37:34,585][123614] Updated weights for policy 1, policy_version 1440 (0.0007) [2023-10-10 16:37:38,190][123582] Updated weights for policy 0, policy_version 1450 (0.0008) [2023-10-10 16:37:38,364][123614] Updated weights for policy 1, policy_version 1450 (0.0008) [2023-10-10 16:37:38,558][123582] Updated weights for policy 0, policy_version 1460 (0.0007) [2023-10-10 16:37:38,733][123614] Updated weights for policy 1, policy_version 1460 (0.0007) [2023-10-10 16:37:38,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13861.7). Total num frames: 2949120. Throughput: 0: 1813.4, 1: 1809.7. Samples: 756960. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 16:37:38,789][122664] Avg episode reward: [(0, '5.550'), (1, '4.770')] [2023-10-10 16:37:38,926][123582] Updated weights for policy 0, policy_version 1470 (0.0009) [2023-10-10 16:37:39,100][123614] Updated weights for policy 1, policy_version 1470 (0.0007) [2023-10-10 16:37:42,627][123582] Updated weights for policy 0, policy_version 1480 (0.0007) [2023-10-10 16:37:42,917][123614] Updated weights for policy 1, policy_version 1480 (0.0009) [2023-10-10 16:37:42,998][123582] Updated weights for policy 0, policy_version 1490 (0.0008) [2023-10-10 16:37:43,281][123614] Updated weights for policy 1, policy_version 1490 (0.0008) [2023-10-10 16:37:43,368][123582] Updated weights for policy 0, policy_version 1500 (0.0010) [2023-10-10 16:37:43,655][123614] Updated weights for policy 1, policy_version 1500 (0.0008) [2023-10-10 16:37:43,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 13994.8). Total num frames: 3047424. Throughput: 0: 1800.4, 1: 1794.4. Samples: 768252. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:37:43,788][122664] Avg episode reward: [(0, '5.370'), (1, '5.150')] [2023-10-10 16:37:47,003][123582] Updated weights for policy 0, policy_version 1510 (0.0010) [2023-10-10 16:37:47,375][123582] Updated weights for policy 0, policy_version 1520 (0.0008) [2023-10-10 16:37:47,505][123614] Updated weights for policy 1, policy_version 1510 (0.0008) [2023-10-10 16:37:47,744][123582] Updated weights for policy 0, policy_version 1530 (0.0008) [2023-10-10 16:37:47,877][123614] Updated weights for policy 1, policy_version 1520 (0.0008) [2023-10-10 16:37:48,241][123614] Updated weights for policy 1, policy_version 1530 (0.0009) [2023-10-10 16:37:48,788][122664] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14122.0). Total num frames: 3145728. Throughput: 0: 1810.6, 1: 1806.4. Samples: 789468. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-10 16:37:48,789][122664] Avg episode reward: [(0, '5.370'), (1, '5.220')] [2023-10-10 16:37:51,438][123582] Updated weights for policy 0, policy_version 1540 (0.0009) [2023-10-10 16:37:51,807][123582] Updated weights for policy 0, policy_version 1550 (0.0008) [2023-10-10 16:37:52,057][123614] Updated weights for policy 1, policy_version 1540 (0.0010) [2023-10-10 16:37:52,175][123582] Updated weights for policy 0, policy_version 1560 (0.0007) [2023-10-10 16:37:52,425][123614] Updated weights for policy 1, policy_version 1550 (0.0008) [2023-10-10 16:37:52,789][123614] Updated weights for policy 1, policy_version 1560 (0.0007) [2023-10-10 16:37:53,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14199.5, 300 sec: 14099.7). Total num frames: 3211264. Throughput: 0: 1805.2, 1: 1790.1. Samples: 810582. Policy #0 lag: (min: 17.0, avg: 27.1, max: 49.0) [2023-10-10 16:37:53,789][122664] Avg episode reward: [(0, '5.790'), (1, '5.140')] [2023-10-10 16:37:53,799][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000001568_1605632.pth... [2023-10-10 16:37:53,799][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000001568_1605632.pth... [2023-10-10 16:37:53,835][123247] Saving new best policy, reward=5.790! [2023-10-10 16:37:55,911][123582] Updated weights for policy 0, policy_version 1570 (0.0008) [2023-10-10 16:37:56,276][123582] Updated weights for policy 0, policy_version 1580 (0.0008) [2023-10-10 16:37:56,407][123614] Updated weights for policy 1, policy_version 1570 (0.0010) [2023-10-10 16:37:56,650][123582] Updated weights for policy 0, policy_version 1590 (0.0007) [2023-10-10 16:37:56,772][123614] Updated weights for policy 1, policy_version 1580 (0.0007) [2023-10-10 16:37:57,017][123582] Updated weights for policy 0, policy_version 1600 (0.0008) [2023-10-10 16:37:57,141][123614] Updated weights for policy 1, policy_version 1590 (0.0009) [2023-10-10 16:37:57,520][123614] Updated weights for policy 1, policy_version 1600 (0.0010) [2023-10-10 16:37:58,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14078.4). Total num frames: 3276800. Throughput: 0: 1813.2, 1: 1807.7. Samples: 822200. Policy #0 lag: (min: 30.0, avg: 32.7, max: 62.0) [2023-10-10 16:37:58,789][122664] Avg episode reward: [(0, '5.980'), (1, '5.280')] [2023-10-10 16:37:58,789][123465] Saving new best policy, reward=5.280! [2023-10-10 16:37:58,790][123247] Saving new best policy, reward=5.980! [2023-10-10 16:38:00,864][123582] Updated weights for policy 0, policy_version 1610 (0.0010) [2023-10-10 16:38:01,233][123582] Updated weights for policy 0, policy_version 1620 (0.0007) [2023-10-10 16:38:01,253][123614] Updated weights for policy 1, policy_version 1610 (0.0007) [2023-10-10 16:38:01,616][123582] Updated weights for policy 0, policy_version 1630 (0.0008) [2023-10-10 16:38:01,621][123614] Updated weights for policy 1, policy_version 1620 (0.0009) [2023-10-10 16:38:01,987][123614] Updated weights for policy 1, policy_version 1630 (0.0009) [2023-10-10 16:38:03,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14058.0). Total num frames: 3342336. Throughput: 0: 1803.2, 1: 1784.4. Samples: 842754. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-10 16:38:03,789][122664] Avg episode reward: [(0, '6.020'), (1, '5.390')] [2023-10-10 16:38:03,790][123247] Saving new best policy, reward=6.020! [2023-10-10 16:38:03,791][123465] Saving new best policy, reward=5.390! [2023-10-10 16:38:05,421][123582] Updated weights for policy 0, policy_version 1640 (0.0010) [2023-10-10 16:38:05,787][123582] Updated weights for policy 0, policy_version 1650 (0.0009) [2023-10-10 16:38:06,022][123614] Updated weights for policy 1, policy_version 1640 (0.0008) [2023-10-10 16:38:06,156][123582] Updated weights for policy 0, policy_version 1660 (0.0010) [2023-10-10 16:38:06,404][123614] Updated weights for policy 1, policy_version 1650 (0.0007) [2023-10-10 16:38:06,771][123614] Updated weights for policy 1, policy_version 1660 (0.0008) [2023-10-10 16:38:08,788][122664] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14038.4). Total num frames: 3407872. Throughput: 0: 1803.1, 1: 1771.5. Samples: 864644. Policy #0 lag: (min: 4.0, avg: 4.5, max: 20.0) [2023-10-10 16:38:08,789][122664] Avg episode reward: [(0, '6.260'), (1, '5.730')] [2023-10-10 16:38:08,802][123247] Saving new best policy, reward=6.260! [2023-10-10 16:38:08,803][123465] Saving new best policy, reward=5.730! [2023-10-10 16:38:09,744][123582] Updated weights for policy 0, policy_version 1670 (0.0008) [2023-10-10 16:38:10,110][123582] Updated weights for policy 0, policy_version 1680 (0.0009) [2023-10-10 16:38:10,482][123582] Updated weights for policy 0, policy_version 1690 (0.0007) [2023-10-10 16:38:10,543][123614] Updated weights for policy 1, policy_version 1670 (0.0009) [2023-10-10 16:38:10,917][123614] Updated weights for policy 1, policy_version 1680 (0.0007) [2023-10-10 16:38:11,283][123614] Updated weights for policy 1, policy_version 1690 (0.0008) [2023-10-10 16:38:13,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14019.6). Total num frames: 3473408. Throughput: 0: 1804.9, 1: 1775.4. Samples: 874730. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:38:13,788][122664] Avg episode reward: [(0, '6.150'), (1, '5.590')] [2023-10-10 16:38:14,088][123582] Updated weights for policy 0, policy_version 1700 (0.0007) [2023-10-10 16:38:14,455][123582] Updated weights for policy 0, policy_version 1710 (0.0008) [2023-10-10 16:38:14,821][123582] Updated weights for policy 0, policy_version 1720 (0.0009) [2023-10-10 16:38:15,020][123614] Updated weights for policy 1, policy_version 1700 (0.0008) [2023-10-10 16:38:15,388][123614] Updated weights for policy 1, policy_version 1710 (0.0008) [2023-10-10 16:38:15,767][123614] Updated weights for policy 1, policy_version 1720 (0.0009) [2023-10-10 16:38:18,452][123582] Updated weights for policy 0, policy_version 1730 (0.0009) [2023-10-10 16:38:18,788][122664] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14001.6). Total num frames: 3538944. Throughput: 0: 1813.2, 1: 1776.8. Samples: 897718. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-10-10 16:38:18,788][122664] Avg episode reward: [(0, '6.050'), (1, '6.170')] [2023-10-10 16:38:18,789][123465] Saving new best policy, reward=6.170! [2023-10-10 16:38:18,818][123582] Updated weights for policy 0, policy_version 1740 (0.0010) [2023-10-10 16:38:19,202][123582] Updated weights for policy 0, policy_version 1750 (0.0010) [2023-10-10 16:38:19,354][123614] Updated weights for policy 1, policy_version 1730 (0.0007) [2023-10-10 16:38:19,570][123582] Updated weights for policy 0, policy_version 1760 (0.0007) [2023-10-10 16:38:19,716][123614] Updated weights for policy 1, policy_version 1740 (0.0008) [2023-10-10 16:38:20,088][123614] Updated weights for policy 1, policy_version 1750 (0.0012) [2023-10-10 16:38:20,463][123614] Updated weights for policy 1, policy_version 1760 (0.0010) [2023-10-10 16:38:23,266][123582] Updated weights for policy 0, policy_version 1770 (0.0008) [2023-10-10 16:38:23,647][123582] Updated weights for policy 0, policy_version 1780 (0.0010) [2023-10-10 16:38:23,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13984.2). Total num frames: 3604480. Throughput: 0: 1817.1, 1: 1805.2. Samples: 919964. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-10-10 16:38:23,789][122664] Avg episode reward: [(0, '5.930'), (1, '6.080')] [2023-10-10 16:38:24,018][123582] Updated weights for policy 0, policy_version 1790 (0.0007) [2023-10-10 16:38:24,127][123614] Updated weights for policy 1, policy_version 1770 (0.0009) [2023-10-10 16:38:24,497][123614] Updated weights for policy 1, policy_version 1780 (0.0011) [2023-10-10 16:38:24,871][123614] Updated weights for policy 1, policy_version 1790 (0.0010) [2023-10-10 16:38:27,732][123582] Updated weights for policy 0, policy_version 1800 (0.0008) [2023-10-10 16:38:28,100][123582] Updated weights for policy 0, policy_version 1810 (0.0008) [2023-10-10 16:38:28,474][123582] Updated weights for policy 0, policy_version 1820 (0.0009) [2023-10-10 16:38:28,575][123614] Updated weights for policy 1, policy_version 1800 (0.0009) [2023-10-10 16:38:28,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14092.2). Total num frames: 3702784. Throughput: 0: 1815.5, 1: 1786.6. Samples: 930346. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-10 16:38:28,788][122664] Avg episode reward: [(0, '6.090'), (1, '5.840')] [2023-10-10 16:38:28,944][123614] Updated weights for policy 1, policy_version 1810 (0.0010) [2023-10-10 16:38:29,313][123614] Updated weights for policy 1, policy_version 1820 (0.0009) [2023-10-10 16:38:32,173][123582] Updated weights for policy 0, policy_version 1830 (0.0009) [2023-10-10 16:38:32,544][123582] Updated weights for policy 0, policy_version 1840 (0.0009) [2023-10-10 16:38:32,911][123582] Updated weights for policy 0, policy_version 1850 (0.0008) [2023-10-10 16:38:33,155][123614] Updated weights for policy 1, policy_version 1830 (0.0009) [2023-10-10 16:38:33,521][123614] Updated weights for policy 1, policy_version 1840 (0.0008) [2023-10-10 16:38:33,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14073.8). Total num frames: 3768320. Throughput: 0: 1816.7, 1: 1800.1. Samples: 952224. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:38:33,789][122664] Avg episode reward: [(0, '6.760'), (1, '5.730')] [2023-10-10 16:38:33,790][123247] Saving new best policy, reward=6.760! [2023-10-10 16:38:33,890][123614] Updated weights for policy 1, policy_version 1850 (0.0009) [2023-10-10 16:38:36,674][123582] Updated weights for policy 0, policy_version 1860 (0.0008) [2023-10-10 16:38:37,038][123582] Updated weights for policy 0, policy_version 1870 (0.0008) [2023-10-10 16:38:37,417][123582] Updated weights for policy 0, policy_version 1880 (0.0008) [2023-10-10 16:38:37,584][123614] Updated weights for policy 1, policy_version 1860 (0.0008) [2023-10-10 16:38:37,957][123614] Updated weights for policy 1, policy_version 1870 (0.0008) [2023-10-10 16:38:38,321][123614] Updated weights for policy 1, policy_version 1880 (0.0007) [2023-10-10 16:38:38,788][122664] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14176.2). Total num frames: 3866624. Throughput: 0: 1810.0, 1: 1790.7. Samples: 972612. Policy #0 lag: (min: 26.0, avg: 29.6, max: 58.0) [2023-10-10 16:38:38,788][122664] Avg episode reward: [(0, '5.850'), (1, '5.790')] [2023-10-10 16:38:41,161][123582] Updated weights for policy 0, policy_version 1890 (0.0007) [2023-10-10 16:38:41,537][123582] Updated weights for policy 0, policy_version 1900 (0.0007) [2023-10-10 16:38:41,913][123582] Updated weights for policy 0, policy_version 1910 (0.0007) [2023-10-10 16:38:42,088][123614] Updated weights for policy 1, policy_version 1890 (0.0009) [2023-10-10 16:38:42,287][123582] Updated weights for policy 0, policy_version 1920 (0.0009) [2023-10-10 16:38:42,458][123614] Updated weights for policy 1, policy_version 1900 (0.0008) [2023-10-10 16:38:42,822][123614] Updated weights for policy 1, policy_version 1910 (0.0009) [2023-10-10 16:38:43,199][123614] Updated weights for policy 1, policy_version 1920 (0.0007) [2023-10-10 16:38:43,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14157.0). Total num frames: 3932160. Throughput: 0: 1814.7, 1: 1800.6. Samples: 984888. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) [2023-10-10 16:38:43,788][122664] Avg episode reward: [(0, '6.250'), (1, '6.120')] [2023-10-10 16:38:45,930][123582] Updated weights for policy 0, policy_version 1930 (0.0007) [2023-10-10 16:38:46,305][123582] Updated weights for policy 0, policy_version 1940 (0.0007) [2023-10-10 16:38:46,678][123582] Updated weights for policy 0, policy_version 1950 (0.0009) [2023-10-10 16:38:46,958][123614] Updated weights for policy 1, policy_version 1930 (0.0008) [2023-10-10 16:38:47,325][123614] Updated weights for policy 1, policy_version 1940 (0.0008) [2023-10-10 16:38:47,691][123614] Updated weights for policy 1, policy_version 1950 (0.0010) [2023-10-10 16:38:48,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14138.4). Total num frames: 3997696. Throughput: 0: 1812.2, 1: 1795.8. Samples: 1005114. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:38:48,789][122664] Avg episode reward: [(0, '6.090'), (1, '5.920')] [2023-10-10 16:38:50,412][123582] Updated weights for policy 0, policy_version 1960 (0.0009) [2023-10-10 16:38:50,789][123582] Updated weights for policy 0, policy_version 1970 (0.0009) [2023-10-10 16:38:51,155][123582] Updated weights for policy 0, policy_version 1980 (0.0008) [2023-10-10 16:38:51,511][123614] Updated weights for policy 1, policy_version 1960 (0.0008) [2023-10-10 16:38:51,889][123614] Updated weights for policy 1, policy_version 1970 (0.0010) [2023-10-10 16:38:52,259][123614] Updated weights for policy 1, policy_version 1980 (0.0009) [2023-10-10 16:38:53,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14120.5). Total num frames: 4063232. Throughput: 0: 1810.6, 1: 1800.2. Samples: 1027128. Policy #0 lag: (min: 31.0, avg: 31.7, max: 50.0) [2023-10-10 16:38:53,788][122664] Avg episode reward: [(0, '6.700'), (1, '5.990')] [2023-10-10 16:38:54,942][123582] Updated weights for policy 0, policy_version 1990 (0.0009) [2023-10-10 16:38:55,307][123582] Updated weights for policy 0, policy_version 2000 (0.0007) [2023-10-10 16:38:55,683][123582] Updated weights for policy 0, policy_version 2010 (0.0007) [2023-10-10 16:38:55,994][123614] Updated weights for policy 1, policy_version 1990 (0.0009) [2023-10-10 16:38:56,361][123614] Updated weights for policy 1, policy_version 2000 (0.0008) [2023-10-10 16:38:56,726][123614] Updated weights for policy 1, policy_version 2010 (0.0011) [2023-10-10 16:38:58,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14103.2). Total num frames: 4128768. Throughput: 0: 1806.5, 1: 1806.0. Samples: 1037294. Policy #0 lag: (min: 1.0, avg: 11.1, max: 33.0) [2023-10-10 16:38:58,788][122664] Avg episode reward: [(0, '8.150'), (1, '6.120')] [2023-10-10 16:38:58,789][123247] Saving new best policy, reward=8.150! [2023-10-10 16:38:59,360][123582] Updated weights for policy 0, policy_version 2020 (0.0009) [2023-10-10 16:38:59,726][123582] Updated weights for policy 0, policy_version 2030 (0.0010) [2023-10-10 16:39:00,108][123582] Updated weights for policy 0, policy_version 2040 (0.0009) [2023-10-10 16:39:00,485][123614] Updated weights for policy 1, policy_version 2020 (0.0009) [2023-10-10 16:39:00,853][123614] Updated weights for policy 1, policy_version 2030 (0.0010) [2023-10-10 16:39:01,223][123614] Updated weights for policy 1, policy_version 2040 (0.0008) [2023-10-10 16:39:03,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 4194304. Throughput: 0: 1797.5, 1: 1799.4. Samples: 1059578. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-10 16:39:03,789][122664] Avg episode reward: [(0, '7.060'), (1, '5.870')] [2023-10-10 16:39:03,794][123582] Updated weights for policy 0, policy_version 2050 (0.0008) [2023-10-10 16:39:04,158][123582] Updated weights for policy 0, policy_version 2060 (0.0007) [2023-10-10 16:39:04,533][123582] Updated weights for policy 0, policy_version 2070 (0.0008) [2023-10-10 16:39:04,901][123582] Updated weights for policy 0, policy_version 2080 (0.0007) [2023-10-10 16:39:04,992][123614] Updated weights for policy 1, policy_version 2050 (0.0009) [2023-10-10 16:39:05,368][123614] Updated weights for policy 1, policy_version 2060 (0.0007) [2023-10-10 16:39:05,743][123614] Updated weights for policy 1, policy_version 2070 (0.0011) [2023-10-10 16:39:06,108][123614] Updated weights for policy 1, policy_version 2080 (0.0008) [2023-10-10 16:39:08,653][123582] Updated weights for policy 0, policy_version 2090 (0.0008) [2023-10-10 16:39:08,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 4259840. Throughput: 0: 1810.2, 1: 1791.6. Samples: 1082046. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:39:08,789][122664] Avg episode reward: [(0, '7.200'), (1, '5.960')] [2023-10-10 16:39:09,023][123582] Updated weights for policy 0, policy_version 2100 (0.0010) [2023-10-10 16:39:09,394][123582] Updated weights for policy 0, policy_version 2110 (0.0009) [2023-10-10 16:39:09,673][123614] Updated weights for policy 1, policy_version 2090 (0.0009) [2023-10-10 16:39:10,039][123614] Updated weights for policy 1, policy_version 2100 (0.0008) [2023-10-10 16:39:10,404][123614] Updated weights for policy 1, policy_version 2110 (0.0007) [2023-10-10 16:39:13,024][123582] Updated weights for policy 0, policy_version 2120 (0.0008) [2023-10-10 16:39:13,406][123582] Updated weights for policy 0, policy_version 2130 (0.0008) [2023-10-10 16:39:13,779][123582] Updated weights for policy 0, policy_version 2140 (0.0010) [2023-10-10 16:39:13,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 4325376. Throughput: 0: 1800.2, 1: 1794.1. Samples: 1092088. Policy #0 lag: (min: 26.0, avg: 28.4, max: 55.0) [2023-10-10 16:39:13,788][122664] Avg episode reward: [(0, '6.970'), (1, '6.500')] [2023-10-10 16:39:14,061][123614] Updated weights for policy 1, policy_version 2120 (0.0008) [2023-10-10 16:39:14,430][123614] Updated weights for policy 1, policy_version 2130 (0.0008) [2023-10-10 16:39:14,798][123614] Updated weights for policy 1, policy_version 2140 (0.0010) [2023-10-10 16:39:14,945][123465] Saving new best policy, reward=6.500! [2023-10-10 16:39:17,426][123582] Updated weights for policy 0, policy_version 2150 (0.0009) [2023-10-10 16:39:17,800][123582] Updated weights for policy 0, policy_version 2160 (0.0008) [2023-10-10 16:39:18,169][123582] Updated weights for policy 0, policy_version 2170 (0.0010) [2023-10-10 16:39:18,651][123614] Updated weights for policy 1, policy_version 2150 (0.0010) [2023-10-10 16:39:18,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 4423680. Throughput: 0: 1810.5, 1: 1797.7. Samples: 1114594. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-10-10 16:39:18,789][122664] Avg episode reward: [(0, '7.280'), (1, '7.490')] [2023-10-10 16:39:19,015][123614] Updated weights for policy 1, policy_version 2160 (0.0011) [2023-10-10 16:39:19,386][123614] Updated weights for policy 1, policy_version 2170 (0.0010) [2023-10-10 16:39:19,606][123465] Saving new best policy, reward=7.490! [2023-10-10 16:39:22,060][123582] Updated weights for policy 0, policy_version 2180 (0.0010) [2023-10-10 16:39:22,429][123582] Updated weights for policy 0, policy_version 2190 (0.0008) [2023-10-10 16:39:22,803][123582] Updated weights for policy 0, policy_version 2200 (0.0008) [2023-10-10 16:39:22,976][123614] Updated weights for policy 1, policy_version 2180 (0.0009) [2023-10-10 16:39:23,337][123614] Updated weights for policy 1, policy_version 2190 (0.0009) [2023-10-10 16:39:23,705][123614] Updated weights for policy 1, policy_version 2200 (0.0008) [2023-10-10 16:39:23,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 4489216. Throughput: 0: 1800.4, 1: 1808.3. Samples: 1135004. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:39:23,789][122664] Avg episode reward: [(0, '7.780'), (1, '7.540')] [2023-10-10 16:39:24,001][123465] Saving new best policy, reward=7.540! [2023-10-10 16:39:26,438][123582] Updated weights for policy 0, policy_version 2210 (0.0009) [2023-10-10 16:39:26,806][123582] Updated weights for policy 0, policy_version 2220 (0.0010) [2023-10-10 16:39:27,175][123582] Updated weights for policy 0, policy_version 2230 (0.0009) [2023-10-10 16:39:27,432][123614] Updated weights for policy 1, policy_version 2210 (0.0009) [2023-10-10 16:39:27,553][123582] Updated weights for policy 0, policy_version 2240 (0.0009) [2023-10-10 16:39:27,799][123614] Updated weights for policy 1, policy_version 2220 (0.0008) [2023-10-10 16:39:28,168][123614] Updated weights for policy 1, policy_version 2230 (0.0007) [2023-10-10 16:39:28,544][123614] Updated weights for policy 1, policy_version 2240 (0.0010) [2023-10-10 16:39:28,788][122664] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 4587520. Throughput: 0: 1812.4, 1: 1799.8. Samples: 1147440. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-10 16:39:28,788][122664] Avg episode reward: [(0, '7.250'), (1, '7.800')] [2023-10-10 16:39:28,789][123465] Saving new best policy, reward=7.800! [2023-10-10 16:39:31,289][123582] Updated weights for policy 0, policy_version 2250 (0.0011) [2023-10-10 16:39:31,653][123582] Updated weights for policy 0, policy_version 2260 (0.0009) [2023-10-10 16:39:32,023][123582] Updated weights for policy 0, policy_version 2270 (0.0010) [2023-10-10 16:39:32,318][123614] Updated weights for policy 1, policy_version 2250 (0.0009) [2023-10-10 16:39:32,684][123614] Updated weights for policy 1, policy_version 2260 (0.0008) [2023-10-10 16:39:33,058][123614] Updated weights for policy 1, policy_version 2270 (0.0009) [2023-10-10 16:39:33,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 4653056. Throughput: 0: 1799.7, 1: 1813.3. Samples: 1167702. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-10-10 16:39:33,789][122664] Avg episode reward: [(0, '7.440'), (1, '7.510')] [2023-10-10 16:39:35,869][123582] Updated weights for policy 0, policy_version 2280 (0.0007) [2023-10-10 16:39:36,241][123582] Updated weights for policy 0, policy_version 2290 (0.0009) [2023-10-10 16:39:36,622][123582] Updated weights for policy 0, policy_version 2300 (0.0007) [2023-10-10 16:39:36,907][123614] Updated weights for policy 1, policy_version 2280 (0.0009) [2023-10-10 16:39:37,280][123614] Updated weights for policy 1, policy_version 2290 (0.0007) [2023-10-10 16:39:37,660][123614] Updated weights for policy 1, policy_version 2300 (0.0008) [2023-10-10 16:39:38,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 4718592. Throughput: 0: 1803.0, 1: 1805.4. Samples: 1189504. Policy #0 lag: (min: 9.0, avg: 16.2, max: 41.0) [2023-10-10 16:39:38,788][122664] Avg episode reward: [(0, '7.510'), (1, '7.710')] [2023-10-10 16:39:40,243][123582] Updated weights for policy 0, policy_version 2310 (0.0007) [2023-10-10 16:39:40,624][123582] Updated weights for policy 0, policy_version 2320 (0.0009) [2023-10-10 16:39:40,986][123582] Updated weights for policy 0, policy_version 2330 (0.0010) [2023-10-10 16:39:41,246][123614] Updated weights for policy 1, policy_version 2310 (0.0008) [2023-10-10 16:39:41,614][123614] Updated weights for policy 1, policy_version 2320 (0.0008) [2023-10-10 16:39:41,990][123614] Updated weights for policy 1, policy_version 2330 (0.0007) [2023-10-10 16:39:43,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 4784128. Throughput: 0: 1807.1, 1: 1813.0. Samples: 1200198. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:39:43,789][122664] Avg episode reward: [(0, '7.370'), (1, '7.980')] [2023-10-10 16:39:43,791][123465] Saving new best policy, reward=7.980! [2023-10-10 16:39:44,623][123582] Updated weights for policy 0, policy_version 2340 (0.0008) [2023-10-10 16:39:44,995][123582] Updated weights for policy 0, policy_version 2350 (0.0008) [2023-10-10 16:39:45,358][123582] Updated weights for policy 0, policy_version 2360 (0.0008) [2023-10-10 16:39:45,835][123614] Updated weights for policy 1, policy_version 2340 (0.0009) [2023-10-10 16:39:46,208][123614] Updated weights for policy 1, policy_version 2350 (0.0008) [2023-10-10 16:39:46,568][123614] Updated weights for policy 1, policy_version 2360 (0.0007) [2023-10-10 16:39:48,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 4849664. Throughput: 0: 1817.0, 1: 1798.1. Samples: 1222256. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:39:48,789][122664] Avg episode reward: [(0, '7.880'), (1, '7.170')] [2023-10-10 16:39:49,004][123582] Updated weights for policy 0, policy_version 2370 (0.0008) [2023-10-10 16:39:49,372][123582] Updated weights for policy 0, policy_version 2380 (0.0008) [2023-10-10 16:39:49,747][123582] Updated weights for policy 0, policy_version 2390 (0.0007) [2023-10-10 16:39:50,122][123582] Updated weights for policy 0, policy_version 2400 (0.0008) [2023-10-10 16:39:50,302][123614] Updated weights for policy 1, policy_version 2370 (0.0008) [2023-10-10 16:39:50,675][123614] Updated weights for policy 1, policy_version 2380 (0.0008) [2023-10-10 16:39:51,050][123614] Updated weights for policy 1, policy_version 2390 (0.0009) [2023-10-10 16:39:51,422][123614] Updated weights for policy 1, policy_version 2400 (0.0008) [2023-10-10 16:39:53,713][123582] Updated weights for policy 0, policy_version 2410 (0.0010) [2023-10-10 16:39:53,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 4915200. Throughput: 0: 1818.3, 1: 1801.0. Samples: 1244914. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-10-10 16:39:53,788][122664] Avg episode reward: [(0, '8.660'), (1, '6.940')] [2023-10-10 16:39:53,796][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000002400_2457600.pth... [2023-10-10 16:39:53,832][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000000736_753664.pth [2023-10-10 16:39:54,080][123582] Updated weights for policy 0, policy_version 2420 (0.0010) [2023-10-10 16:39:54,441][123582] Updated weights for policy 0, policy_version 2430 (0.0009) [2023-10-10 16:39:54,505][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000002432_2490368.pth... [2023-10-10 16:39:54,539][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000000736_753664.pth [2023-10-10 16:39:54,543][123247] Saving new best policy, reward=8.660! [2023-10-10 16:39:55,196][123614] Updated weights for policy 1, policy_version 2410 (0.0011) [2023-10-10 16:39:55,569][123614] Updated weights for policy 1, policy_version 2420 (0.0007) [2023-10-10 16:39:55,939][123614] Updated weights for policy 1, policy_version 2430 (0.0009) [2023-10-10 16:39:58,217][123582] Updated weights for policy 0, policy_version 2440 (0.0009) [2023-10-10 16:39:58,594][123582] Updated weights for policy 0, policy_version 2450 (0.0009) [2023-10-10 16:39:58,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 4980736. Throughput: 0: 1818.8, 1: 1802.3. Samples: 1255038. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-10-10 16:39:58,788][122664] Avg episode reward: [(0, '8.670'), (1, '6.320')] [2023-10-10 16:39:58,966][123582] Updated weights for policy 0, policy_version 2460 (0.0007) [2023-10-10 16:39:59,111][123247] Saving new best policy, reward=8.670! [2023-10-10 16:39:59,768][123614] Updated weights for policy 1, policy_version 2440 (0.0009) [2023-10-10 16:40:00,138][123614] Updated weights for policy 1, policy_version 2450 (0.0008) [2023-10-10 16:40:00,513][123614] Updated weights for policy 1, policy_version 2460 (0.0008) [2023-10-10 16:40:02,535][123582] Updated weights for policy 0, policy_version 2470 (0.0009) [2023-10-10 16:40:02,907][123582] Updated weights for policy 0, policy_version 2480 (0.0009) [2023-10-10 16:40:03,270][123582] Updated weights for policy 0, policy_version 2490 (0.0009) [2023-10-10 16:40:03,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 5079040. Throughput: 0: 1822.4, 1: 1796.5. Samples: 1277442. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:40:03,789][122664] Avg episode reward: [(0, '8.870'), (1, '7.210')] [2023-10-10 16:40:03,791][123247] Saving new best policy, reward=8.870! [2023-10-10 16:40:04,286][123614] Updated weights for policy 1, policy_version 2470 (0.0008) [2023-10-10 16:40:04,661][123614] Updated weights for policy 1, policy_version 2480 (0.0008) [2023-10-10 16:40:05,034][123614] Updated weights for policy 1, policy_version 2490 (0.0009) [2023-10-10 16:40:06,905][123582] Updated weights for policy 0, policy_version 2500 (0.0008) [2023-10-10 16:40:07,283][123582] Updated weights for policy 0, policy_version 2510 (0.0009) [2023-10-10 16:40:07,651][123582] Updated weights for policy 0, policy_version 2520 (0.0010) [2023-10-10 16:40:08,705][123614] Updated weights for policy 1, policy_version 2500 (0.0008) [2023-10-10 16:40:08,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 5144576. Throughput: 0: 1821.3, 1: 1816.0. Samples: 1298684. Policy #0 lag: (min: 18.0, avg: 23.4, max: 50.0) [2023-10-10 16:40:08,789][122664] Avg episode reward: [(0, '8.100'), (1, '7.530')] [2023-10-10 16:40:09,079][123614] Updated weights for policy 1, policy_version 2510 (0.0010) [2023-10-10 16:40:09,458][123614] Updated weights for policy 1, policy_version 2520 (0.0009) [2023-10-10 16:40:11,195][123582] Updated weights for policy 0, policy_version 2530 (0.0008) [2023-10-10 16:40:11,572][123582] Updated weights for policy 0, policy_version 2540 (0.0009) [2023-10-10 16:40:11,954][123582] Updated weights for policy 0, policy_version 2550 (0.0008) [2023-10-10 16:40:12,320][123582] Updated weights for policy 0, policy_version 2560 (0.0008) [2023-10-10 16:40:13,029][123614] Updated weights for policy 1, policy_version 2530 (0.0007) [2023-10-10 16:40:13,391][123614] Updated weights for policy 1, policy_version 2540 (0.0007) [2023-10-10 16:40:13,757][123614] Updated weights for policy 1, policy_version 2550 (0.0009) [2023-10-10 16:40:13,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 5210112. Throughput: 0: 1820.4, 1: 1794.9. Samples: 1310130. Policy #0 lag: (min: 31.0, avg: 42.7, max: 63.0) [2023-10-10 16:40:13,789][122664] Avg episode reward: [(0, '7.540'), (1, '8.320')] [2023-10-10 16:40:14,122][123465] Saving new best policy, reward=8.320! [2023-10-10 16:40:14,126][123614] Updated weights for policy 1, policy_version 2560 (0.0009) [2023-10-10 16:40:15,867][123582] Updated weights for policy 0, policy_version 2570 (0.0007) [2023-10-10 16:40:16,242][123582] Updated weights for policy 0, policy_version 2580 (0.0008) [2023-10-10 16:40:16,625][123582] Updated weights for policy 0, policy_version 2590 (0.0009) [2023-10-10 16:40:17,902][123614] Updated weights for policy 1, policy_version 2570 (0.0009) [2023-10-10 16:40:18,264][123614] Updated weights for policy 1, policy_version 2580 (0.0009) [2023-10-10 16:40:18,643][123614] Updated weights for policy 1, policy_version 2590 (0.0008) [2023-10-10 16:40:18,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 5308416. Throughput: 0: 1838.7, 1: 1816.4. Samples: 1332180. Policy #0 lag: (min: 31.0, avg: 42.7, max: 63.0) [2023-10-10 16:40:18,788][122664] Avg episode reward: [(0, '7.810'), (1, '8.170')] [2023-10-10 16:40:20,359][123582] Updated weights for policy 0, policy_version 2600 (0.0008) [2023-10-10 16:40:20,732][123582] Updated weights for policy 0, policy_version 2610 (0.0009) [2023-10-10 16:40:21,104][123582] Updated weights for policy 0, policy_version 2620 (0.0008) [2023-10-10 16:40:22,380][123614] Updated weights for policy 1, policy_version 2600 (0.0009) [2023-10-10 16:40:22,755][123614] Updated weights for policy 1, policy_version 2610 (0.0008) [2023-10-10 16:40:23,123][123614] Updated weights for policy 1, policy_version 2620 (0.0008) [2023-10-10 16:40:23,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 5373952. Throughput: 0: 1841.6, 1: 1799.6. Samples: 1353358. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:40:23,788][122664] Avg episode reward: [(0, '8.380'), (1, '8.190')] [2023-10-10 16:40:24,866][123582] Updated weights for policy 0, policy_version 2630 (0.0007) [2023-10-10 16:40:25,235][123582] Updated weights for policy 0, policy_version 2640 (0.0007) [2023-10-10 16:40:25,612][123582] Updated weights for policy 0, policy_version 2650 (0.0007) [2023-10-10 16:40:26,821][123614] Updated weights for policy 1, policy_version 2630 (0.0008) [2023-10-10 16:40:27,198][123614] Updated weights for policy 1, policy_version 2640 (0.0009) [2023-10-10 16:40:27,575][123614] Updated weights for policy 1, policy_version 2650 (0.0007) [2023-10-10 16:40:28,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 5439488. Throughput: 0: 1835.4, 1: 1812.9. Samples: 1364372. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:40:28,789][122664] Avg episode reward: [(0, '8.070'), (1, '7.770')] [2023-10-10 16:40:29,135][123582] Updated weights for policy 0, policy_version 2661 (0.0008) [2023-10-10 16:40:29,507][123582] Updated weights for policy 0, policy_version 2671 (0.0010) [2023-10-10 16:40:29,874][123582] Updated weights for policy 0, policy_version 2681 (0.0009) [2023-10-10 16:40:31,253][123614] Updated weights for policy 1, policy_version 2660 (0.0008) [2023-10-10 16:40:31,628][123614] Updated weights for policy 1, policy_version 2670 (0.0007) [2023-10-10 16:40:31,998][123614] Updated weights for policy 1, policy_version 2680 (0.0008) [2023-10-10 16:40:33,566][123582] Updated weights for policy 0, policy_version 2691 (0.0008) [2023-10-10 16:40:33,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 5505024. Throughput: 0: 1836.1, 1: 1802.8. Samples: 1386010. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:40:33,789][122664] Avg episode reward: [(0, '8.030'), (1, '7.600')] [2023-10-10 16:40:33,934][123582] Updated weights for policy 0, policy_version 2701 (0.0007) [2023-10-10 16:40:34,305][123582] Updated weights for policy 0, policy_version 2711 (0.0007) [2023-10-10 16:40:35,681][123614] Updated weights for policy 1, policy_version 2690 (0.0008) [2023-10-10 16:40:36,049][123614] Updated weights for policy 1, policy_version 2700 (0.0008) [2023-10-10 16:40:36,415][123614] Updated weights for policy 1, policy_version 2710 (0.0008) [2023-10-10 16:40:36,786][123614] Updated weights for policy 1, policy_version 2720 (0.0009) [2023-10-10 16:40:38,043][123582] Updated weights for policy 0, policy_version 2721 (0.0008) [2023-10-10 16:40:38,421][123582] Updated weights for policy 0, policy_version 2731 (0.0008) [2023-10-10 16:40:38,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 5570560. Throughput: 0: 1822.4, 1: 1802.1. Samples: 1408018. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:40:38,789][122664] Avg episode reward: [(0, '8.120'), (1, '7.570')] [2023-10-10 16:40:38,799][123582] Updated weights for policy 0, policy_version 2741 (0.0008) [2023-10-10 16:40:39,175][123582] Updated weights for policy 0, policy_version 2751 (0.0010) [2023-10-10 16:40:40,650][123614] Updated weights for policy 1, policy_version 2730 (0.0009) [2023-10-10 16:40:41,020][123614] Updated weights for policy 1, policy_version 2740 (0.0007) [2023-10-10 16:40:41,394][123614] Updated weights for policy 1, policy_version 2750 (0.0007) [2023-10-10 16:40:42,867][123582] Updated weights for policy 0, policy_version 2761 (0.0008) [2023-10-10 16:40:43,246][123582] Updated weights for policy 0, policy_version 2771 (0.0008) [2023-10-10 16:40:43,621][123582] Updated weights for policy 0, policy_version 2781 (0.0008) [2023-10-10 16:40:43,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 5668864. Throughput: 0: 1831.4, 1: 1798.3. Samples: 1418374. Policy #0 lag: (min: 25.0, avg: 28.6, max: 56.0) [2023-10-10 16:40:43,789][122664] Avg episode reward: [(0, '8.140'), (1, '8.980')] [2023-10-10 16:40:43,791][123465] Saving new best policy, reward=8.980! [2023-10-10 16:40:44,998][123614] Updated weights for policy 1, policy_version 2760 (0.0007) [2023-10-10 16:40:45,370][123614] Updated weights for policy 1, policy_version 2770 (0.0008) [2023-10-10 16:40:45,747][123614] Updated weights for policy 1, policy_version 2780 (0.0008) [2023-10-10 16:40:47,225][123582] Updated weights for policy 0, policy_version 2791 (0.0009) [2023-10-10 16:40:47,594][123582] Updated weights for policy 0, policy_version 2801 (0.0010) [2023-10-10 16:40:47,979][123582] Updated weights for policy 0, policy_version 2811 (0.0009) [2023-10-10 16:40:48,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 5734400. Throughput: 0: 1826.4, 1: 1803.6. Samples: 1440792. Policy #0 lag: (min: 25.0, avg: 28.6, max: 56.0) [2023-10-10 16:40:48,789][122664] Avg episode reward: [(0, '8.330'), (1, '9.370')] [2023-10-10 16:40:48,791][123465] Saving new best policy, reward=9.370! [2023-10-10 16:40:49,564][123614] Updated weights for policy 1, policy_version 2790 (0.0008) [2023-10-10 16:40:49,930][123614] Updated weights for policy 1, policy_version 2800 (0.0010) [2023-10-10 16:40:50,308][123614] Updated weights for policy 1, policy_version 2810 (0.0007) [2023-10-10 16:40:51,615][123582] Updated weights for policy 0, policy_version 2821 (0.0008) [2023-10-10 16:40:51,993][123582] Updated weights for policy 0, policy_version 2831 (0.0009) [2023-10-10 16:40:52,356][123582] Updated weights for policy 0, policy_version 2841 (0.0007) [2023-10-10 16:40:53,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 5799936. Throughput: 0: 1839.6, 1: 1809.8. Samples: 1462906. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:40:53,789][122664] Avg episode reward: [(0, '9.160'), (1, '10.490')] [2023-10-10 16:40:53,796][123247] Saving new best policy, reward=9.160! [2023-10-10 16:40:53,947][123614] Updated weights for policy 1, policy_version 2820 (0.0007) [2023-10-10 16:40:54,311][123614] Updated weights for policy 1, policy_version 2830 (0.0009) [2023-10-10 16:40:54,682][123614] Updated weights for policy 1, policy_version 2840 (0.0010) [2023-10-10 16:40:54,985][123465] Saving new best policy, reward=10.490! [2023-10-10 16:40:56,005][123582] Updated weights for policy 0, policy_version 2851 (0.0007) [2023-10-10 16:40:56,367][123582] Updated weights for policy 0, policy_version 2861 (0.0008) [2023-10-10 16:40:56,739][123582] Updated weights for policy 0, policy_version 2871 (0.0010) [2023-10-10 16:40:58,351][123614] Updated weights for policy 1, policy_version 2850 (0.0009) [2023-10-10 16:40:58,723][123614] Updated weights for policy 1, policy_version 2860 (0.0007) [2023-10-10 16:40:58,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 5865472. Throughput: 0: 1830.0, 1: 1806.5. Samples: 1473776. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:40:58,788][122664] Avg episode reward: [(0, '9.290'), (1, '10.690')] [2023-10-10 16:40:58,789][123247] Saving new best policy, reward=9.290! [2023-10-10 16:40:59,090][123614] Updated weights for policy 1, policy_version 2870 (0.0008) [2023-10-10 16:40:59,455][123465] Saving new best policy, reward=10.690! [2023-10-10 16:40:59,455][123614] Updated weights for policy 1, policy_version 2880 (0.0009) [2023-10-10 16:41:00,504][123582] Updated weights for policy 0, policy_version 2881 (0.0010) [2023-10-10 16:41:00,871][123582] Updated weights for policy 0, policy_version 2891 (0.0008) [2023-10-10 16:41:01,238][123582] Updated weights for policy 0, policy_version 2901 (0.0010) [2023-10-10 16:41:01,615][123582] Updated weights for policy 0, policy_version 2911 (0.0007) [2023-10-10 16:41:03,234][123614] Updated weights for policy 1, policy_version 2890 (0.0010) [2023-10-10 16:41:03,595][123614] Updated weights for policy 1, policy_version 2900 (0.0010) [2023-10-10 16:41:03,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 5931008. Throughput: 0: 1823.7, 1: 1804.6. Samples: 1495454. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 16:41:03,789][122664] Avg episode reward: [(0, '8.780'), (1, '10.140')] [2023-10-10 16:41:03,965][123614] Updated weights for policy 1, policy_version 2910 (0.0007) [2023-10-10 16:41:05,412][123582] Updated weights for policy 0, policy_version 2921 (0.0007) [2023-10-10 16:41:05,789][123582] Updated weights for policy 0, policy_version 2931 (0.0007) [2023-10-10 16:41:06,160][123582] Updated weights for policy 0, policy_version 2941 (0.0007) [2023-10-10 16:41:07,764][123614] Updated weights for policy 1, policy_version 2920 (0.0009) [2023-10-10 16:41:08,149][123614] Updated weights for policy 1, policy_version 2930 (0.0010) [2023-10-10 16:41:08,523][123614] Updated weights for policy 1, policy_version 2940 (0.0007) [2023-10-10 16:41:08,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 6029312. Throughput: 0: 1828.0, 1: 1802.5. Samples: 1516730. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-10 16:41:08,788][122664] Avg episode reward: [(0, '7.990'), (1, '10.250')] [2023-10-10 16:41:09,818][123582] Updated weights for policy 0, policy_version 2951 (0.0009) [2023-10-10 16:41:10,195][123582] Updated weights for policy 0, policy_version 2961 (0.0010) [2023-10-10 16:41:10,567][123582] Updated weights for policy 0, policy_version 2971 (0.0008) [2023-10-10 16:41:12,179][123614] Updated weights for policy 1, policy_version 2950 (0.0010) [2023-10-10 16:41:12,544][123614] Updated weights for policy 1, policy_version 2960 (0.0010) [2023-10-10 16:41:12,906][123614] Updated weights for policy 1, policy_version 2970 (0.0009) [2023-10-10 16:41:13,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 6094848. Throughput: 0: 1829.5, 1: 1807.3. Samples: 1528030. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:41:13,789][122664] Avg episode reward: [(0, '8.150'), (1, '10.090')] [2023-10-10 16:41:14,193][123582] Updated weights for policy 0, policy_version 2981 (0.0009) [2023-10-10 16:41:14,567][123582] Updated weights for policy 0, policy_version 2991 (0.0009) [2023-10-10 16:41:14,937][123582] Updated weights for policy 0, policy_version 3001 (0.0008) [2023-10-10 16:41:16,635][123614] Updated weights for policy 1, policy_version 2980 (0.0009) [2023-10-10 16:41:17,003][123614] Updated weights for policy 1, policy_version 2990 (0.0011) [2023-10-10 16:41:17,369][123614] Updated weights for policy 1, policy_version 3000 (0.0008) [2023-10-10 16:41:18,684][123582] Updated weights for policy 0, policy_version 3011 (0.0009) [2023-10-10 16:41:18,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 6160384. Throughput: 0: 1822.2, 1: 1805.8. Samples: 1549272. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:41:18,788][122664] Avg episode reward: [(0, '8.600'), (1, '10.280')] [2023-10-10 16:41:19,062][123582] Updated weights for policy 0, policy_version 3021 (0.0007) [2023-10-10 16:41:19,432][123582] Updated weights for policy 0, policy_version 3031 (0.0008) [2023-10-10 16:41:20,985][123614] Updated weights for policy 1, policy_version 3010 (0.0008) [2023-10-10 16:41:21,353][123614] Updated weights for policy 1, policy_version 3020 (0.0012) [2023-10-10 16:41:21,719][123614] Updated weights for policy 1, policy_version 3030 (0.0008) [2023-10-10 16:41:22,086][123614] Updated weights for policy 1, policy_version 3040 (0.0008) [2023-10-10 16:41:23,249][123582] Updated weights for policy 0, policy_version 3041 (0.0010) [2023-10-10 16:41:23,620][123582] Updated weights for policy 0, policy_version 3051 (0.0007) [2023-10-10 16:41:23,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 6225920. Throughput: 0: 1825.9, 1: 1812.3. Samples: 1571738. Policy #0 lag: (min: 31.0, avg: 33.0, max: 62.0) [2023-10-10 16:41:23,788][122664] Avg episode reward: [(0, '9.780'), (1, '10.510')] [2023-10-10 16:41:23,992][123582] Updated weights for policy 0, policy_version 3061 (0.0009) [2023-10-10 16:41:24,365][123582] Updated weights for policy 0, policy_version 3071 (0.0008) [2023-10-10 16:41:24,395][123247] Saving new best policy, reward=9.780! [2023-10-10 16:41:25,757][123614] Updated weights for policy 1, policy_version 3050 (0.0007) [2023-10-10 16:41:26,118][123614] Updated weights for policy 1, policy_version 3060 (0.0008) [2023-10-10 16:41:26,488][123614] Updated weights for policy 1, policy_version 3070 (0.0007) [2023-10-10 16:41:28,027][123582] Updated weights for policy 0, policy_version 3081 (0.0008) [2023-10-10 16:41:28,400][123582] Updated weights for policy 0, policy_version 3091 (0.0007) [2023-10-10 16:41:28,774][123582] Updated weights for policy 0, policy_version 3101 (0.0008) [2023-10-10 16:41:28,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 6291456. Throughput: 0: 1818.7, 1: 1816.7. Samples: 1581966. Policy #0 lag: (min: 31.0, avg: 33.0, max: 62.0) [2023-10-10 16:41:28,788][122664] Avg episode reward: [(0, '10.650'), (1, '10.280')] [2023-10-10 16:41:28,882][123247] Saving new best policy, reward=10.650! [2023-10-10 16:41:30,193][123614] Updated weights for policy 1, policy_version 3080 (0.0009) [2023-10-10 16:41:30,572][123614] Updated weights for policy 1, policy_version 3090 (0.0010) [2023-10-10 16:41:30,933][123614] Updated weights for policy 1, policy_version 3100 (0.0008) [2023-10-10 16:41:32,375][123582] Updated weights for policy 0, policy_version 3111 (0.0009) [2023-10-10 16:41:32,752][123582] Updated weights for policy 0, policy_version 3121 (0.0008) [2023-10-10 16:41:33,127][123582] Updated weights for policy 0, policy_version 3131 (0.0010) [2023-10-10 16:41:33,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 6389760. Throughput: 0: 1816.5, 1: 1816.0. Samples: 1604252. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:41:33,789][122664] Avg episode reward: [(0, '10.660'), (1, '10.420')] [2023-10-10 16:41:33,789][123247] Saving new best policy, reward=10.660! [2023-10-10 16:41:34,682][123614] Updated weights for policy 1, policy_version 3110 (0.0009) [2023-10-10 16:41:35,048][123614] Updated weights for policy 1, policy_version 3120 (0.0008) [2023-10-10 16:41:35,412][123614] Updated weights for policy 1, policy_version 3130 (0.0008) [2023-10-10 16:41:36,767][123582] Updated weights for policy 0, policy_version 3141 (0.0009) [2023-10-10 16:41:37,135][123582] Updated weights for policy 0, policy_version 3151 (0.0010) [2023-10-10 16:41:37,507][123582] Updated weights for policy 0, policy_version 3161 (0.0008) [2023-10-10 16:41:38,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 6455296. Throughput: 0: 1808.9, 1: 1809.6. Samples: 1625738. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:41:38,789][122664] Avg episode reward: [(0, '10.430'), (1, '10.070')] [2023-10-10 16:41:39,187][123614] Updated weights for policy 1, policy_version 3140 (0.0009) [2023-10-10 16:41:39,552][123614] Updated weights for policy 1, policy_version 3150 (0.0007) [2023-10-10 16:41:39,921][123614] Updated weights for policy 1, policy_version 3160 (0.0010) [2023-10-10 16:41:41,180][123582] Updated weights for policy 0, policy_version 3171 (0.0008) [2023-10-10 16:41:41,547][123582] Updated weights for policy 0, policy_version 3181 (0.0009) [2023-10-10 16:41:41,919][123582] Updated weights for policy 0, policy_version 3191 (0.0008) [2023-10-10 16:41:43,545][123614] Updated weights for policy 1, policy_version 3170 (0.0008) [2023-10-10 16:41:43,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 6520832. Throughput: 0: 1815.9, 1: 1810.0. Samples: 1636940. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-10 16:41:43,788][122664] Avg episode reward: [(0, '10.070'), (1, '10.590')] [2023-10-10 16:41:43,914][123614] Updated weights for policy 1, policy_version 3180 (0.0008) [2023-10-10 16:41:44,281][123614] Updated weights for policy 1, policy_version 3190 (0.0007) [2023-10-10 16:41:44,655][123614] Updated weights for policy 1, policy_version 3200 (0.0008) [2023-10-10 16:41:45,583][123582] Updated weights for policy 0, policy_version 3201 (0.0008) [2023-10-10 16:41:45,954][123582] Updated weights for policy 0, policy_version 3211 (0.0007) [2023-10-10 16:41:46,329][123582] Updated weights for policy 0, policy_version 3221 (0.0007) [2023-10-10 16:41:46,719][123582] Updated weights for policy 0, policy_version 3231 (0.0007) [2023-10-10 16:41:48,484][123614] Updated weights for policy 1, policy_version 3210 (0.0007) [2023-10-10 16:41:48,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 6586368. Throughput: 0: 1817.3, 1: 1809.9. Samples: 1658674. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-10 16:41:48,788][122664] Avg episode reward: [(0, '10.040'), (1, '10.290')] [2023-10-10 16:41:48,857][123614] Updated weights for policy 1, policy_version 3220 (0.0008) [2023-10-10 16:41:49,228][123614] Updated weights for policy 1, policy_version 3230 (0.0008) [2023-10-10 16:41:50,293][123582] Updated weights for policy 0, policy_version 3241 (0.0010) [2023-10-10 16:41:50,659][123582] Updated weights for policy 0, policy_version 3251 (0.0010) [2023-10-10 16:41:51,034][123582] Updated weights for policy 0, policy_version 3261 (0.0010) [2023-10-10 16:41:52,978][123614] Updated weights for policy 1, policy_version 3240 (0.0010) [2023-10-10 16:41:53,350][123614] Updated weights for policy 1, policy_version 3250 (0.0010) [2023-10-10 16:41:53,723][123614] Updated weights for policy 1, policy_version 3260 (0.0007) [2023-10-10 16:41:53,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 6651904. Throughput: 0: 1816.5, 1: 1816.1. Samples: 1680198. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 16:41:53,789][122664] Avg episode reward: [(0, '9.670'), (1, '10.880')] [2023-10-10 16:41:53,796][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000003264_3342336.pth... [2023-10-10 16:41:53,826][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000001568_1605632.pth [2023-10-10 16:41:53,864][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000003264_3342336.pth... [2023-10-10 16:41:53,893][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000001568_1605632.pth [2023-10-10 16:41:53,896][123465] Saving new best policy, reward=10.880! [2023-10-10 16:41:54,787][123582] Updated weights for policy 0, policy_version 3271 (0.0010) [2023-10-10 16:41:55,162][123582] Updated weights for policy 0, policy_version 3281 (0.0008) [2023-10-10 16:41:55,530][123582] Updated weights for policy 0, policy_version 3291 (0.0008) [2023-10-10 16:41:57,438][123614] Updated weights for policy 1, policy_version 3270 (0.0010) [2023-10-10 16:41:57,805][123614] Updated weights for policy 1, policy_version 3280 (0.0010) [2023-10-10 16:41:58,179][123614] Updated weights for policy 1, policy_version 3290 (0.0010) [2023-10-10 16:41:58,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 6750208. Throughput: 0: 1816.5, 1: 1809.6. Samples: 1691206. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 16:41:58,789][122664] Avg episode reward: [(0, '10.200'), (1, '10.640')] [2023-10-10 16:41:59,383][123582] Updated weights for policy 0, policy_version 3301 (0.0009) [2023-10-10 16:41:59,773][123582] Updated weights for policy 0, policy_version 3311 (0.0009) [2023-10-10 16:42:00,144][123582] Updated weights for policy 0, policy_version 3321 (0.0010) [2023-10-10 16:42:01,836][123614] Updated weights for policy 1, policy_version 3300 (0.0009) [2023-10-10 16:42:02,199][123614] Updated weights for policy 1, policy_version 3310 (0.0007) [2023-10-10 16:42:02,570][123614] Updated weights for policy 1, policy_version 3320 (0.0008) [2023-10-10 16:42:03,711][123582] Updated weights for policy 0, policy_version 3331 (0.0008) [2023-10-10 16:42:03,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 6815744. Throughput: 0: 1814.7, 1: 1813.1. Samples: 1712524. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:42:03,788][122664] Avg episode reward: [(0, '10.780'), (1, '9.910')] [2023-10-10 16:42:04,078][123582] Updated weights for policy 0, policy_version 3341 (0.0007) [2023-10-10 16:42:04,451][123582] Updated weights for policy 0, policy_version 3351 (0.0009) [2023-10-10 16:42:04,785][123247] Saving new best policy, reward=10.780! [2023-10-10 16:42:06,223][123614] Updated weights for policy 1, policy_version 3330 (0.0009) [2023-10-10 16:42:06,586][123614] Updated weights for policy 1, policy_version 3340 (0.0008) [2023-10-10 16:42:06,955][123614] Updated weights for policy 1, policy_version 3350 (0.0010) [2023-10-10 16:42:07,322][123614] Updated weights for policy 1, policy_version 3360 (0.0010) [2023-10-10 16:42:07,966][123582] Updated weights for policy 0, policy_version 3361 (0.0007) [2023-10-10 16:42:08,338][123582] Updated weights for policy 0, policy_version 3371 (0.0008) [2023-10-10 16:42:08,709][123582] Updated weights for policy 0, policy_version 3381 (0.0010) [2023-10-10 16:42:08,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 6881280. Throughput: 0: 1815.2, 1: 1805.8. Samples: 1734686. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:42:08,788][122664] Avg episode reward: [(0, '11.040'), (1, '10.080')] [2023-10-10 16:42:09,075][123582] Updated weights for policy 0, policy_version 3391 (0.0009) [2023-10-10 16:42:09,115][123247] Saving new best policy, reward=11.040! [2023-10-10 16:42:11,064][123614] Updated weights for policy 1, policy_version 3370 (0.0007) [2023-10-10 16:42:11,431][123614] Updated weights for policy 1, policy_version 3380 (0.0007) [2023-10-10 16:42:11,798][123614] Updated weights for policy 1, policy_version 3390 (0.0007) [2023-10-10 16:42:12,705][123582] Updated weights for policy 0, policy_version 3401 (0.0008) [2023-10-10 16:42:13,068][123582] Updated weights for policy 0, policy_version 3411 (0.0010) [2023-10-10 16:42:13,434][123582] Updated weights for policy 0, policy_version 3421 (0.0009) [2023-10-10 16:42:13,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 6979584. Throughput: 0: 1821.6, 1: 1811.7. Samples: 1745464. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:42:13,788][122664] Avg episode reward: [(0, '11.200'), (1, '9.810')] [2023-10-10 16:42:13,789][123247] Saving new best policy, reward=11.200! [2023-10-10 16:42:15,444][123614] Updated weights for policy 1, policy_version 3400 (0.0009) [2023-10-10 16:42:15,812][123614] Updated weights for policy 1, policy_version 3410 (0.0007) [2023-10-10 16:42:16,190][123614] Updated weights for policy 1, policy_version 3420 (0.0009) [2023-10-10 16:42:17,156][123582] Updated weights for policy 0, policy_version 3431 (0.0009) [2023-10-10 16:42:17,524][123582] Updated weights for policy 0, policy_version 3441 (0.0009) [2023-10-10 16:42:17,899][123582] Updated weights for policy 0, policy_version 3451 (0.0008) [2023-10-10 16:42:18,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 7045120. Throughput: 0: 1819.6, 1: 1809.8. Samples: 1767574. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-10-10 16:42:18,789][122664] Avg episode reward: [(0, '10.720'), (1, '10.980')] [2023-10-10 16:42:18,790][123465] Saving new best policy, reward=10.980! [2023-10-10 16:42:19,739][123614] Updated weights for policy 1, policy_version 3430 (0.0008) [2023-10-10 16:42:20,101][123614] Updated weights for policy 1, policy_version 3440 (0.0007) [2023-10-10 16:42:20,471][123614] Updated weights for policy 1, policy_version 3450 (0.0007) [2023-10-10 16:42:21,528][123582] Updated weights for policy 0, policy_version 3461 (0.0008) [2023-10-10 16:42:21,898][123582] Updated weights for policy 0, policy_version 3471 (0.0007) [2023-10-10 16:42:22,270][123582] Updated weights for policy 0, policy_version 3481 (0.0008) [2023-10-10 16:42:23,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 7110656. Throughput: 0: 1827.9, 1: 1814.8. Samples: 1789658. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-10-10 16:42:23,788][122664] Avg episode reward: [(0, '9.390'), (1, '11.220')] [2023-10-10 16:42:23,796][123465] Saving new best policy, reward=11.220! [2023-10-10 16:42:24,173][123614] Updated weights for policy 1, policy_version 3460 (0.0010) [2023-10-10 16:42:24,545][123614] Updated weights for policy 1, policy_version 3470 (0.0009) [2023-10-10 16:42:24,909][123614] Updated weights for policy 1, policy_version 3480 (0.0008) [2023-10-10 16:42:25,992][123582] Updated weights for policy 0, policy_version 3491 (0.0009) [2023-10-10 16:42:26,374][123582] Updated weights for policy 0, policy_version 3501 (0.0009) [2023-10-10 16:42:26,749][123582] Updated weights for policy 0, policy_version 3511 (0.0008) [2023-10-10 16:42:28,619][123614] Updated weights for policy 1, policy_version 3490 (0.0008) [2023-10-10 16:42:28,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 7176192. Throughput: 0: 1816.0, 1: 1811.6. Samples: 1800180. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-10 16:42:28,788][122664] Avg episode reward: [(0, '10.010'), (1, '11.410')] [2023-10-10 16:42:28,988][123614] Updated weights for policy 1, policy_version 3500 (0.0010) [2023-10-10 16:42:29,359][123614] Updated weights for policy 1, policy_version 3510 (0.0012) [2023-10-10 16:42:29,731][123614] Updated weights for policy 1, policy_version 3520 (0.0012) [2023-10-10 16:42:29,731][123465] Saving new best policy, reward=11.410! [2023-10-10 16:42:30,504][123582] Updated weights for policy 0, policy_version 3521 (0.0010) [2023-10-10 16:42:30,885][123582] Updated weights for policy 0, policy_version 3531 (0.0008) [2023-10-10 16:42:31,241][123582] Updated weights for policy 0, policy_version 3541 (0.0007) [2023-10-10 16:42:31,620][123582] Updated weights for policy 0, policy_version 3551 (0.0008) [2023-10-10 16:42:33,425][123614] Updated weights for policy 1, policy_version 3530 (0.0008) [2023-10-10 16:42:33,785][123614] Updated weights for policy 1, policy_version 3540 (0.0007) [2023-10-10 16:42:33,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 7241728. Throughput: 0: 1814.5, 1: 1818.5. Samples: 1822160. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-10 16:42:33,788][122664] Avg episode reward: [(0, '9.180'), (1, '11.160')] [2023-10-10 16:42:34,156][123614] Updated weights for policy 1, policy_version 3550 (0.0008) [2023-10-10 16:42:35,537][123582] Updated weights for policy 0, policy_version 3561 (0.0010) [2023-10-10 16:42:35,901][123582] Updated weights for policy 0, policy_version 3571 (0.0007) [2023-10-10 16:42:36,275][123582] Updated weights for policy 0, policy_version 3581 (0.0007) [2023-10-10 16:42:38,040][123614] Updated weights for policy 1, policy_version 3560 (0.0009) [2023-10-10 16:42:38,405][123614] Updated weights for policy 1, policy_version 3570 (0.0008) [2023-10-10 16:42:38,772][123614] Updated weights for policy 1, policy_version 3580 (0.0007) [2023-10-10 16:42:38,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 7307264. Throughput: 0: 1806.9, 1: 1820.4. Samples: 1843426. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-10 16:42:38,788][122664] Avg episode reward: [(0, '9.880'), (1, '11.510')] [2023-10-10 16:42:38,919][123465] Saving new best policy, reward=11.510! [2023-10-10 16:42:40,004][123582] Updated weights for policy 0, policy_version 3591 (0.0010) [2023-10-10 16:42:40,378][123582] Updated weights for policy 0, policy_version 3601 (0.0009) [2023-10-10 16:42:40,753][123582] Updated weights for policy 0, policy_version 3611 (0.0008) [2023-10-10 16:42:42,348][123614] Updated weights for policy 1, policy_version 3590 (0.0009) [2023-10-10 16:42:42,707][123614] Updated weights for policy 1, policy_version 3600 (0.0009) [2023-10-10 16:42:43,083][123614] Updated weights for policy 1, policy_version 3610 (0.0008) [2023-10-10 16:42:43,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 7405568. Throughput: 0: 1806.5, 1: 1818.3. Samples: 1854320. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-10 16:42:43,788][122664] Avg episode reward: [(0, '10.210'), (1, '11.570')] [2023-10-10 16:42:43,789][123465] Saving new best policy, reward=11.570! [2023-10-10 16:42:44,413][123582] Updated weights for policy 0, policy_version 3621 (0.0009) [2023-10-10 16:42:44,784][123582] Updated weights for policy 0, policy_version 3631 (0.0010) [2023-10-10 16:42:45,168][123582] Updated weights for policy 0, policy_version 3641 (0.0009) [2023-10-10 16:42:46,764][123614] Updated weights for policy 1, policy_version 3620 (0.0007) [2023-10-10 16:42:47,139][123614] Updated weights for policy 1, policy_version 3630 (0.0008) [2023-10-10 16:42:47,494][123614] Updated weights for policy 1, policy_version 3640 (0.0007) [2023-10-10 16:42:48,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 7471104. Throughput: 0: 1815.8, 1: 1822.8. Samples: 1876258. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:42:48,788][122664] Avg episode reward: [(0, '9.900'), (1, '13.180')] [2023-10-10 16:42:48,789][123465] Saving new best policy, reward=13.180! [2023-10-10 16:42:48,798][123582] Updated weights for policy 0, policy_version 3651 (0.0009) [2023-10-10 16:42:49,180][123582] Updated weights for policy 0, policy_version 3661 (0.0007) [2023-10-10 16:42:49,551][123582] Updated weights for policy 0, policy_version 3671 (0.0008) [2023-10-10 16:42:51,307][123614] Updated weights for policy 1, policy_version 3650 (0.0008) [2023-10-10 16:42:51,679][123614] Updated weights for policy 1, policy_version 3660 (0.0009) [2023-10-10 16:42:52,040][123614] Updated weights for policy 1, policy_version 3670 (0.0008) [2023-10-10 16:42:52,412][123614] Updated weights for policy 1, policy_version 3680 (0.0009) [2023-10-10 16:42:53,184][123582] Updated weights for policy 0, policy_version 3681 (0.0008) [2023-10-10 16:42:53,548][123582] Updated weights for policy 0, policy_version 3691 (0.0009) [2023-10-10 16:42:53,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 7536640. Throughput: 0: 1819.8, 1: 1817.6. Samples: 1898368. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:42:53,789][122664] Avg episode reward: [(0, '10.710'), (1, '12.860')] [2023-10-10 16:42:53,923][123582] Updated weights for policy 0, policy_version 3701 (0.0008) [2023-10-10 16:42:54,297][123582] Updated weights for policy 0, policy_version 3711 (0.0009) [2023-10-10 16:42:56,127][123614] Updated weights for policy 1, policy_version 3690 (0.0008) [2023-10-10 16:42:56,504][123614] Updated weights for policy 1, policy_version 3700 (0.0007) [2023-10-10 16:42:56,876][123614] Updated weights for policy 1, policy_version 3710 (0.0008) [2023-10-10 16:42:57,908][123582] Updated weights for policy 0, policy_version 3721 (0.0008) [2023-10-10 16:42:58,277][123582] Updated weights for policy 0, policy_version 3731 (0.0009) [2023-10-10 16:42:58,649][123582] Updated weights for policy 0, policy_version 3741 (0.0009) [2023-10-10 16:42:58,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 7634944. Throughput: 0: 1814.5, 1: 1821.6. Samples: 1909090. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:42:58,788][122664] Avg episode reward: [(0, '10.950'), (1, '14.280')] [2023-10-10 16:42:58,789][123465] Saving new best policy, reward=14.280! [2023-10-10 16:43:00,570][123614] Updated weights for policy 1, policy_version 3720 (0.0009) [2023-10-10 16:43:00,937][123614] Updated weights for policy 1, policy_version 3730 (0.0007) [2023-10-10 16:43:01,311][123614] Updated weights for policy 1, policy_version 3740 (0.0008) [2023-10-10 16:43:02,298][123582] Updated weights for policy 0, policy_version 3751 (0.0008) [2023-10-10 16:43:02,676][123582] Updated weights for policy 0, policy_version 3761 (0.0011) [2023-10-10 16:43:03,047][123582] Updated weights for policy 0, policy_version 3771 (0.0011) [2023-10-10 16:43:03,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 7700480. Throughput: 0: 1817.5, 1: 1814.2. Samples: 1931000. Policy #0 lag: (min: 16.0, avg: 37.8, max: 48.0) [2023-10-10 16:43:03,789][122664] Avg episode reward: [(0, '10.420'), (1, '14.190')] [2023-10-10 16:43:05,164][123614] Updated weights for policy 1, policy_version 3750 (0.0009) [2023-10-10 16:43:05,533][123614] Updated weights for policy 1, policy_version 3760 (0.0007) [2023-10-10 16:43:05,905][123614] Updated weights for policy 1, policy_version 3770 (0.0007) [2023-10-10 16:43:06,764][123582] Updated weights for policy 0, policy_version 3781 (0.0010) [2023-10-10 16:43:07,130][123582] Updated weights for policy 0, policy_version 3791 (0.0008) [2023-10-10 16:43:07,515][123582] Updated weights for policy 0, policy_version 3801 (0.0008) [2023-10-10 16:43:08,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 7766016. Throughput: 0: 1812.0, 1: 1813.4. Samples: 1952802. Policy #0 lag: (min: 16.0, avg: 37.8, max: 48.0) [2023-10-10 16:43:08,789][122664] Avg episode reward: [(0, '10.330'), (1, '13.410')] [2023-10-10 16:43:09,434][123614] Updated weights for policy 1, policy_version 3780 (0.0010) [2023-10-10 16:43:09,800][123614] Updated weights for policy 1, policy_version 3790 (0.0009) [2023-10-10 16:43:10,167][123614] Updated weights for policy 1, policy_version 3800 (0.0011) [2023-10-10 16:43:11,247][123582] Updated weights for policy 0, policy_version 3811 (0.0009) [2023-10-10 16:43:11,625][123582] Updated weights for policy 0, policy_version 3821 (0.0008) [2023-10-10 16:43:11,996][123582] Updated weights for policy 0, policy_version 3831 (0.0009) [2023-10-10 16:43:13,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 7831552. Throughput: 0: 1824.4, 1: 1819.0. Samples: 1964130. Policy #0 lag: (min: 29.0, avg: 33.4, max: 61.0) [2023-10-10 16:43:13,788][122664] Avg episode reward: [(0, '9.630'), (1, '12.160')] [2023-10-10 16:43:13,867][123614] Updated weights for policy 1, policy_version 3810 (0.0010) [2023-10-10 16:43:14,231][123614] Updated weights for policy 1, policy_version 3820 (0.0009) [2023-10-10 16:43:14,606][123614] Updated weights for policy 1, policy_version 3830 (0.0009) [2023-10-10 16:43:14,981][123614] Updated weights for policy 1, policy_version 3840 (0.0008) [2023-10-10 16:43:15,681][123582] Updated weights for policy 0, policy_version 3841 (0.0010) [2023-10-10 16:43:16,052][123582] Updated weights for policy 0, policy_version 3851 (0.0007) [2023-10-10 16:43:16,421][123582] Updated weights for policy 0, policy_version 3861 (0.0008) [2023-10-10 16:43:16,797][123582] Updated weights for policy 0, policy_version 3871 (0.0008) [2023-10-10 16:43:18,683][123614] Updated weights for policy 1, policy_version 3850 (0.0007) [2023-10-10 16:43:18,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 7897088. Throughput: 0: 1821.1, 1: 1814.2. Samples: 1985746. Policy #0 lag: (min: 29.0, avg: 33.4, max: 61.0) [2023-10-10 16:43:18,789][122664] Avg episode reward: [(0, '10.980'), (1, '12.870')] [2023-10-10 16:43:19,049][123614] Updated weights for policy 1, policy_version 3860 (0.0007) [2023-10-10 16:43:19,425][123614] Updated weights for policy 1, policy_version 3870 (0.0008) [2023-10-10 16:43:20,433][123582] Updated weights for policy 0, policy_version 3881 (0.0008) [2023-10-10 16:43:20,810][123582] Updated weights for policy 0, policy_version 3891 (0.0009) [2023-10-10 16:43:21,185][123582] Updated weights for policy 0, policy_version 3901 (0.0007) [2023-10-10 16:43:23,201][123614] Updated weights for policy 1, policy_version 3880 (0.0009) [2023-10-10 16:43:23,588][123614] Updated weights for policy 1, policy_version 3890 (0.0009) [2023-10-10 16:43:23,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 7962624. Throughput: 0: 1828.0, 1: 1815.5. Samples: 2007384. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-10 16:43:23,789][122664] Avg episode reward: [(0, '11.810'), (1, '11.570')] [2023-10-10 16:43:23,797][123247] Saving new best policy, reward=11.810! [2023-10-10 16:43:23,948][123614] Updated weights for policy 1, policy_version 3900 (0.0007) [2023-10-10 16:43:24,944][123582] Updated weights for policy 0, policy_version 3911 (0.0008) [2023-10-10 16:43:25,321][123582] Updated weights for policy 0, policy_version 3921 (0.0008) [2023-10-10 16:43:25,692][123582] Updated weights for policy 0, policy_version 3931 (0.0007) [2023-10-10 16:43:27,505][123614] Updated weights for policy 1, policy_version 3910 (0.0007) [2023-10-10 16:43:27,868][123614] Updated weights for policy 1, policy_version 3920 (0.0008) [2023-10-10 16:43:28,248][123614] Updated weights for policy 1, policy_version 3930 (0.0007) [2023-10-10 16:43:28,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 8060928. Throughput: 0: 1830.5, 1: 1816.9. Samples: 2018456. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-10 16:43:28,788][122664] Avg episode reward: [(0, '11.850'), (1, '11.310')] [2023-10-10 16:43:28,789][123247] Saving new best policy, reward=11.850! [2023-10-10 16:43:29,380][123582] Updated weights for policy 0, policy_version 3941 (0.0007) [2023-10-10 16:43:29,754][123582] Updated weights for policy 0, policy_version 3951 (0.0007) [2023-10-10 16:43:30,127][123582] Updated weights for policy 0, policy_version 3961 (0.0007) [2023-10-10 16:43:31,835][123614] Updated weights for policy 1, policy_version 3940 (0.0008) [2023-10-10 16:43:32,215][123614] Updated weights for policy 1, policy_version 3950 (0.0008) [2023-10-10 16:43:32,588][123614] Updated weights for policy 1, policy_version 3960 (0.0007) [2023-10-10 16:43:33,766][123582] Updated weights for policy 0, policy_version 3971 (0.0007) [2023-10-10 16:43:33,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 8126464. Throughput: 0: 1824.3, 1: 1817.6. Samples: 2040146. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-10 16:43:33,788][122664] Avg episode reward: [(0, '11.080'), (1, '12.600')] [2023-10-10 16:43:34,137][123582] Updated weights for policy 0, policy_version 3981 (0.0007) [2023-10-10 16:43:34,508][123582] Updated weights for policy 0, policy_version 3991 (0.0007) [2023-10-10 16:43:36,338][123614] Updated weights for policy 1, policy_version 3970 (0.0009) [2023-10-10 16:43:36,707][123614] Updated weights for policy 1, policy_version 3980 (0.0009) [2023-10-10 16:43:37,071][123614] Updated weights for policy 1, policy_version 3990 (0.0011) [2023-10-10 16:43:37,436][123614] Updated weights for policy 1, policy_version 4000 (0.0010) [2023-10-10 16:43:38,169][123582] Updated weights for policy 0, policy_version 4001 (0.0010) [2023-10-10 16:43:38,531][123582] Updated weights for policy 0, policy_version 4011 (0.0008) [2023-10-10 16:43:38,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 8192000. Throughput: 0: 1828.2, 1: 1817.8. Samples: 2062436. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-10 16:43:38,788][122664] Avg episode reward: [(0, '9.850'), (1, '12.860')] [2023-10-10 16:43:38,913][123582] Updated weights for policy 0, policy_version 4021 (0.0008) [2023-10-10 16:43:39,281][123582] Updated weights for policy 0, policy_version 4031 (0.0009) [2023-10-10 16:43:41,212][123614] Updated weights for policy 1, policy_version 4010 (0.0007) [2023-10-10 16:43:41,575][123614] Updated weights for policy 1, policy_version 4020 (0.0007) [2023-10-10 16:43:41,949][123614] Updated weights for policy 1, policy_version 4030 (0.0007) [2023-10-10 16:43:43,048][123582] Updated weights for policy 0, policy_version 4041 (0.0007) [2023-10-10 16:43:43,425][123582] Updated weights for policy 0, policy_version 4051 (0.0007) [2023-10-10 16:43:43,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 8257536. Throughput: 0: 1830.3, 1: 1818.5. Samples: 2073286. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-10 16:43:43,789][122664] Avg episode reward: [(0, '9.840'), (1, '12.700')] [2023-10-10 16:43:43,797][123582] Updated weights for policy 0, policy_version 4061 (0.0007) [2023-10-10 16:43:45,765][123614] Updated weights for policy 1, policy_version 4040 (0.0010) [2023-10-10 16:43:46,127][123614] Updated weights for policy 1, policy_version 4050 (0.0008) [2023-10-10 16:43:46,495][123614] Updated weights for policy 1, policy_version 4060 (0.0008) [2023-10-10 16:43:47,472][123582] Updated weights for policy 0, policy_version 4071 (0.0009) [2023-10-10 16:43:47,850][123582] Updated weights for policy 0, policy_version 4081 (0.0008) [2023-10-10 16:43:48,205][123582] Updated weights for policy 0, policy_version 4091 (0.0008) [2023-10-10 16:43:48,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 8355840. Throughput: 0: 1834.7, 1: 1814.7. Samples: 2095222. Policy #0 lag: (min: 2.0, avg: 13.1, max: 34.0) [2023-10-10 16:43:48,789][122664] Avg episode reward: [(0, '9.980'), (1, '14.050')] [2023-10-10 16:43:50,308][123614] Updated weights for policy 1, policy_version 4070 (0.0010) [2023-10-10 16:43:50,665][123614] Updated weights for policy 1, policy_version 4080 (0.0009) [2023-10-10 16:43:51,044][123614] Updated weights for policy 1, policy_version 4090 (0.0009) [2023-10-10 16:43:51,927][123582] Updated weights for policy 0, policy_version 4101 (0.0008) [2023-10-10 16:43:52,295][123582] Updated weights for policy 0, policy_version 4111 (0.0008) [2023-10-10 16:43:52,683][123582] Updated weights for policy 0, policy_version 4121 (0.0011) [2023-10-10 16:43:53,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 8421376. Throughput: 0: 1830.5, 1: 1813.0. Samples: 2116762. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 16:43:53,789][122664] Avg episode reward: [(0, '10.130'), (1, '12.630')] [2023-10-10 16:43:53,800][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000004096_4194304.pth... [2023-10-10 16:43:53,800][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000004128_4227072.pth... [2023-10-10 16:43:53,837][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000002432_2490368.pth [2023-10-10 16:43:53,840][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000002400_2457600.pth [2023-10-10 16:43:54,626][123614] Updated weights for policy 1, policy_version 4100 (0.0008) [2023-10-10 16:43:55,003][123614] Updated weights for policy 1, policy_version 4110 (0.0007) [2023-10-10 16:43:55,372][123614] Updated weights for policy 1, policy_version 4120 (0.0008) [2023-10-10 16:43:56,338][123582] Updated weights for policy 0, policy_version 4131 (0.0011) [2023-10-10 16:43:56,709][123582] Updated weights for policy 0, policy_version 4141 (0.0010) [2023-10-10 16:43:57,085][123582] Updated weights for policy 0, policy_version 4151 (0.0009) [2023-10-10 16:43:58,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 8486912. Throughput: 0: 1829.3, 1: 1807.2. Samples: 2127774. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 16:43:58,788][122664] Avg episode reward: [(0, '10.700'), (1, '11.650')] [2023-10-10 16:43:59,198][123614] Updated weights for policy 1, policy_version 4130 (0.0008) [2023-10-10 16:43:59,569][123614] Updated weights for policy 1, policy_version 4140 (0.0010) [2023-10-10 16:43:59,932][123614] Updated weights for policy 1, policy_version 4150 (0.0011) [2023-10-10 16:44:00,308][123614] Updated weights for policy 1, policy_version 4160 (0.0010) [2023-10-10 16:44:00,539][123582] Updated weights for policy 0, policy_version 4161 (0.0008) [2023-10-10 16:44:00,915][123582] Updated weights for policy 0, policy_version 4171 (0.0008) [2023-10-10 16:44:01,278][123582] Updated weights for policy 0, policy_version 4181 (0.0009) [2023-10-10 16:44:01,654][123582] Updated weights for policy 0, policy_version 4191 (0.0007) [2023-10-10 16:44:03,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 8552448. Throughput: 0: 1832.8, 1: 1805.7. Samples: 2149480. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-10 16:44:03,789][122664] Avg episode reward: [(0, '10.570'), (1, '12.390')] [2023-10-10 16:44:03,945][123614] Updated weights for policy 1, policy_version 4170 (0.0009) [2023-10-10 16:44:04,310][123614] Updated weights for policy 1, policy_version 4180 (0.0007) [2023-10-10 16:44:04,679][123614] Updated weights for policy 1, policy_version 4190 (0.0007) [2023-10-10 16:44:05,375][123582] Updated weights for policy 0, policy_version 4201 (0.0008) [2023-10-10 16:44:05,748][123582] Updated weights for policy 0, policy_version 4211 (0.0008) [2023-10-10 16:44:06,126][123582] Updated weights for policy 0, policy_version 4221 (0.0007) [2023-10-10 16:44:08,312][123614] Updated weights for policy 1, policy_version 4200 (0.0010) [2023-10-10 16:44:08,690][123614] Updated weights for policy 1, policy_version 4210 (0.0010) [2023-10-10 16:44:08,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 8617984. Throughput: 0: 1828.2, 1: 1816.9. Samples: 2171414. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-10 16:44:08,788][122664] Avg episode reward: [(0, '10.270'), (1, '12.500')] [2023-10-10 16:44:09,063][123614] Updated weights for policy 1, policy_version 4220 (0.0010) [2023-10-10 16:44:09,812][123582] Updated weights for policy 0, policy_version 4231 (0.0008) [2023-10-10 16:44:10,183][123582] Updated weights for policy 0, policy_version 4241 (0.0010) [2023-10-10 16:44:10,553][123582] Updated weights for policy 0, policy_version 4251 (0.0009) [2023-10-10 16:44:12,624][123614] Updated weights for policy 1, policy_version 4230 (0.0008) [2023-10-10 16:44:12,989][123614] Updated weights for policy 1, policy_version 4240 (0.0008) [2023-10-10 16:44:13,363][123614] Updated weights for policy 1, policy_version 4250 (0.0008) [2023-10-10 16:44:13,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 8716288. Throughput: 0: 1828.5, 1: 1807.0. Samples: 2182052. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 16:44:13,788][122664] Avg episode reward: [(0, '10.660'), (1, '12.030')] [2023-10-10 16:44:14,192][123582] Updated weights for policy 0, policy_version 4261 (0.0008) [2023-10-10 16:44:14,570][123582] Updated weights for policy 0, policy_version 4271 (0.0008) [2023-10-10 16:44:14,943][123582] Updated weights for policy 0, policy_version 4281 (0.0008) [2023-10-10 16:44:17,213][123614] Updated weights for policy 1, policy_version 4260 (0.0008) [2023-10-10 16:44:17,574][123614] Updated weights for policy 1, policy_version 4270 (0.0008) [2023-10-10 16:44:17,945][123614] Updated weights for policy 1, policy_version 4280 (0.0009) [2023-10-10 16:44:18,533][123582] Updated weights for policy 0, policy_version 4291 (0.0008) [2023-10-10 16:44:18,788][122664] Fps is (10 sec: 16383.4, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 8781824. Throughput: 0: 1831.8, 1: 1810.5. Samples: 2204048. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 16:44:18,790][122664] Avg episode reward: [(0, '10.980'), (1, '12.650')] [2023-10-10 16:44:18,906][123582] Updated weights for policy 0, policy_version 4301 (0.0008) [2023-10-10 16:44:19,277][123582] Updated weights for policy 0, policy_version 4311 (0.0008) [2023-10-10 16:44:21,652][123614] Updated weights for policy 1, policy_version 4290 (0.0008) [2023-10-10 16:44:22,015][123614] Updated weights for policy 1, policy_version 4300 (0.0007) [2023-10-10 16:44:22,382][123614] Updated weights for policy 1, policy_version 4310 (0.0010) [2023-10-10 16:44:22,751][123614] Updated weights for policy 1, policy_version 4320 (0.0010) [2023-10-10 16:44:22,925][123582] Updated weights for policy 0, policy_version 4321 (0.0010) [2023-10-10 16:44:23,307][123582] Updated weights for policy 0, policy_version 4331 (0.0010) [2023-10-10 16:44:23,678][123582] Updated weights for policy 0, policy_version 4341 (0.0009) [2023-10-10 16:44:23,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 8847360. Throughput: 0: 1823.4, 1: 1801.9. Samples: 2225574. Policy #0 lag: (min: 9.0, avg: 30.5, max: 32.0) [2023-10-10 16:44:23,789][122664] Avg episode reward: [(0, '11.420'), (1, '11.980')] [2023-10-10 16:44:24,054][123582] Updated weights for policy 0, policy_version 4351 (0.0007) [2023-10-10 16:44:26,388][123614] Updated weights for policy 1, policy_version 4330 (0.0009) [2023-10-10 16:44:26,755][123614] Updated weights for policy 1, policy_version 4340 (0.0008) [2023-10-10 16:44:27,124][123614] Updated weights for policy 1, policy_version 4350 (0.0008) [2023-10-10 16:44:27,615][123582] Updated weights for policy 0, policy_version 4361 (0.0008) [2023-10-10 16:44:27,988][123582] Updated weights for policy 0, policy_version 4371 (0.0008) [2023-10-10 16:44:28,364][123582] Updated weights for policy 0, policy_version 4381 (0.0011) [2023-10-10 16:44:28,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 8945664. Throughput: 0: 1832.3, 1: 1806.3. Samples: 2237022. Policy #0 lag: (min: 31.0, avg: 32.7, max: 59.0) [2023-10-10 16:44:28,789][122664] Avg episode reward: [(0, '13.000'), (1, '11.000')] [2023-10-10 16:44:28,790][123247] Saving new best policy, reward=13.000! [2023-10-10 16:44:30,846][123614] Updated weights for policy 1, policy_version 4360 (0.0008) [2023-10-10 16:44:31,219][123614] Updated weights for policy 1, policy_version 4370 (0.0010) [2023-10-10 16:44:31,582][123614] Updated weights for policy 1, policy_version 4380 (0.0009) [2023-10-10 16:44:32,053][123582] Updated weights for policy 0, policy_version 4391 (0.0009) [2023-10-10 16:44:32,418][123582] Updated weights for policy 0, policy_version 4401 (0.0008) [2023-10-10 16:44:32,790][123582] Updated weights for policy 0, policy_version 4411 (0.0008) [2023-10-10 16:44:33,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 9011200. Throughput: 0: 1816.2, 1: 1805.1. Samples: 2258180. Policy #0 lag: (min: 31.0, avg: 32.7, max: 59.0) [2023-10-10 16:44:33,788][122664] Avg episode reward: [(0, '13.940'), (1, '12.320')] [2023-10-10 16:44:33,789][123247] Saving new best policy, reward=13.940! [2023-10-10 16:44:35,215][123614] Updated weights for policy 1, policy_version 4390 (0.0010) [2023-10-10 16:44:35,590][123614] Updated weights for policy 1, policy_version 4400 (0.0007) [2023-10-10 16:44:35,954][123614] Updated weights for policy 1, policy_version 4410 (0.0008) [2023-10-10 16:44:36,468][123582] Updated weights for policy 0, policy_version 4421 (0.0009) [2023-10-10 16:44:36,837][123582] Updated weights for policy 0, policy_version 4431 (0.0009) [2023-10-10 16:44:37,200][123582] Updated weights for policy 0, policy_version 4441 (0.0010) [2023-10-10 16:44:38,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 9076736. Throughput: 0: 1835.8, 1: 1807.2. Samples: 2280696. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:44:38,788][122664] Avg episode reward: [(0, '14.210'), (1, '11.880')] [2023-10-10 16:44:38,796][123247] Saving new best policy, reward=14.210! [2023-10-10 16:44:39,721][123614] Updated weights for policy 1, policy_version 4420 (0.0010) [2023-10-10 16:44:40,090][123614] Updated weights for policy 1, policy_version 4430 (0.0008) [2023-10-10 16:44:40,459][123614] Updated weights for policy 1, policy_version 4440 (0.0007) [2023-10-10 16:44:40,828][123582] Updated weights for policy 0, policy_version 4451 (0.0009) [2023-10-10 16:44:41,196][123582] Updated weights for policy 0, policy_version 4461 (0.0009) [2023-10-10 16:44:41,570][123582] Updated weights for policy 0, policy_version 4471 (0.0007) [2023-10-10 16:44:43,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 9142272. Throughput: 0: 1824.7, 1: 1807.7. Samples: 2291232. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:44:43,789][122664] Avg episode reward: [(0, '14.550'), (1, '13.660')] [2023-10-10 16:44:43,790][123247] Saving new best policy, reward=14.550! [2023-10-10 16:44:44,195][123614] Updated weights for policy 1, policy_version 4450 (0.0008) [2023-10-10 16:44:44,563][123614] Updated weights for policy 1, policy_version 4460 (0.0009) [2023-10-10 16:44:44,931][123614] Updated weights for policy 1, policy_version 4470 (0.0008) [2023-10-10 16:44:45,302][123614] Updated weights for policy 1, policy_version 4480 (0.0008) [2023-10-10 16:44:45,304][123582] Updated weights for policy 0, policy_version 4481 (0.0007) [2023-10-10 16:44:45,683][123582] Updated weights for policy 0, policy_version 4491 (0.0009) [2023-10-10 16:44:46,055][123582] Updated weights for policy 0, policy_version 4501 (0.0007) [2023-10-10 16:44:46,428][123582] Updated weights for policy 0, policy_version 4511 (0.0010) [2023-10-10 16:44:48,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 9207808. Throughput: 0: 1831.7, 1: 1808.7. Samples: 2313298. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:44:48,788][122664] Avg episode reward: [(0, '12.340'), (1, '13.820')] [2023-10-10 16:44:49,046][123614] Updated weights for policy 1, policy_version 4490 (0.0008) [2023-10-10 16:44:49,411][123614] Updated weights for policy 1, policy_version 4500 (0.0008) [2023-10-10 16:44:49,776][123614] Updated weights for policy 1, policy_version 4510 (0.0009) [2023-10-10 16:44:50,219][123582] Updated weights for policy 0, policy_version 4521 (0.0010) [2023-10-10 16:44:50,590][123582] Updated weights for policy 0, policy_version 4531 (0.0009) [2023-10-10 16:44:50,957][123582] Updated weights for policy 0, policy_version 4541 (0.0009) [2023-10-10 16:44:53,460][123614] Updated weights for policy 1, policy_version 4520 (0.0010) [2023-10-10 16:44:53,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 9273344. Throughput: 0: 1833.4, 1: 1810.3. Samples: 2335382. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:44:53,789][122664] Avg episode reward: [(0, '10.990'), (1, '14.190')] [2023-10-10 16:44:53,848][123614] Updated weights for policy 1, policy_version 4530 (0.0009) [2023-10-10 16:44:54,213][123614] Updated weights for policy 1, policy_version 4540 (0.0010) [2023-10-10 16:44:54,547][123582] Updated weights for policy 0, policy_version 4551 (0.0007) [2023-10-10 16:44:54,923][123582] Updated weights for policy 0, policy_version 4561 (0.0008) [2023-10-10 16:44:55,297][123582] Updated weights for policy 0, policy_version 4571 (0.0007) [2023-10-10 16:44:57,984][123614] Updated weights for policy 1, policy_version 4550 (0.0007) [2023-10-10 16:44:58,349][123614] Updated weights for policy 1, policy_version 4560 (0.0007) [2023-10-10 16:44:58,720][123614] Updated weights for policy 1, policy_version 4570 (0.0007) [2023-10-10 16:44:58,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 9338880. Throughput: 0: 1839.9, 1: 1807.2. Samples: 2346174. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-10 16:44:58,789][122664] Avg episode reward: [(0, '11.400'), (1, '14.050')] [2023-10-10 16:44:58,947][123582] Updated weights for policy 0, policy_version 4581 (0.0008) [2023-10-10 16:44:59,320][123582] Updated weights for policy 0, policy_version 4591 (0.0009) [2023-10-10 16:44:59,688][123582] Updated weights for policy 0, policy_version 4601 (0.0009) [2023-10-10 16:45:02,477][123614] Updated weights for policy 1, policy_version 4580 (0.0008) [2023-10-10 16:45:02,838][123614] Updated weights for policy 1, policy_version 4590 (0.0010) [2023-10-10 16:45:03,215][123614] Updated weights for policy 1, policy_version 4600 (0.0009) [2023-10-10 16:45:03,492][123582] Updated weights for policy 0, policy_version 4611 (0.0009) [2023-10-10 16:45:03,788][122664] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 9437184. Throughput: 0: 1826.2, 1: 1818.6. Samples: 2368062. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-10 16:45:03,788][122664] Avg episode reward: [(0, '10.870'), (1, '14.190')] [2023-10-10 16:45:03,875][123582] Updated weights for policy 0, policy_version 4621 (0.0008) [2023-10-10 16:45:04,247][123582] Updated weights for policy 0, policy_version 4631 (0.0008) [2023-10-10 16:45:06,984][123614] Updated weights for policy 1, policy_version 4610 (0.0007) [2023-10-10 16:45:07,341][123614] Updated weights for policy 1, policy_version 4620 (0.0007) [2023-10-10 16:45:07,717][123614] Updated weights for policy 1, policy_version 4630 (0.0007) [2023-10-10 16:45:07,804][123582] Updated weights for policy 0, policy_version 4641 (0.0008) [2023-10-10 16:45:08,078][123614] Updated weights for policy 1, policy_version 4640 (0.0009) [2023-10-10 16:45:08,178][123582] Updated weights for policy 0, policy_version 4651 (0.0008) [2023-10-10 16:45:08,548][123582] Updated weights for policy 0, policy_version 4661 (0.0010) [2023-10-10 16:45:08,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 9502720. Throughput: 0: 1825.1, 1: 1812.2. Samples: 2389252. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 16:45:08,788][122664] Avg episode reward: [(0, '11.970'), (1, '13.440')] [2023-10-10 16:45:08,913][123582] Updated weights for policy 0, policy_version 4671 (0.0010) [2023-10-10 16:45:11,718][123614] Updated weights for policy 1, policy_version 4650 (0.0007) [2023-10-10 16:45:12,092][123614] Updated weights for policy 1, policy_version 4660 (0.0010) [2023-10-10 16:45:12,475][123614] Updated weights for policy 1, policy_version 4670 (0.0009) [2023-10-10 16:45:12,765][123582] Updated weights for policy 0, policy_version 4681 (0.0009) [2023-10-10 16:45:13,125][123582] Updated weights for policy 0, policy_version 4691 (0.0010) [2023-10-10 16:45:13,501][123582] Updated weights for policy 0, policy_version 4701 (0.0007) [2023-10-10 16:45:13,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 9601024. Throughput: 0: 1818.5, 1: 1820.0. Samples: 2400752. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:45:13,788][122664] Avg episode reward: [(0, '13.490'), (1, '14.450')] [2023-10-10 16:45:13,789][123465] Saving new best policy, reward=14.450! [2023-10-10 16:45:16,275][123614] Updated weights for policy 1, policy_version 4680 (0.0009) [2023-10-10 16:45:16,648][123614] Updated weights for policy 1, policy_version 4690 (0.0009) [2023-10-10 16:45:17,023][123614] Updated weights for policy 1, policy_version 4700 (0.0008) [2023-10-10 16:45:17,332][123582] Updated weights for policy 0, policy_version 4711 (0.0010) [2023-10-10 16:45:17,698][123582] Updated weights for policy 0, policy_version 4721 (0.0009) [2023-10-10 16:45:18,068][123582] Updated weights for policy 0, policy_version 4731 (0.0009) [2023-10-10 16:45:18,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 9666560. Throughput: 0: 1824.8, 1: 1803.5. Samples: 2421458. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:45:18,789][122664] Avg episode reward: [(0, '13.830'), (1, '13.970')] [2023-10-10 16:45:20,826][123614] Updated weights for policy 1, policy_version 4710 (0.0008) [2023-10-10 16:45:21,197][123614] Updated weights for policy 1, policy_version 4720 (0.0010) [2023-10-10 16:45:21,564][123614] Updated weights for policy 1, policy_version 4730 (0.0009) [2023-10-10 16:45:21,795][123582] Updated weights for policy 0, policy_version 4741 (0.0009) [2023-10-10 16:45:22,165][123582] Updated weights for policy 0, policy_version 4751 (0.0007) [2023-10-10 16:45:22,537][123582] Updated weights for policy 0, policy_version 4761 (0.0008) [2023-10-10 16:45:23,788][122664] Fps is (10 sec: 13106.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 9732096. Throughput: 0: 1810.7, 1: 1799.8. Samples: 2443168. Policy #0 lag: (min: 19.0, avg: 31.2, max: 32.0) [2023-10-10 16:45:23,789][122664] Avg episode reward: [(0, '15.090'), (1, '15.170')] [2023-10-10 16:45:23,801][123247] Saving new best policy, reward=15.090! [2023-10-10 16:45:23,801][123465] Saving new best policy, reward=15.170! [2023-10-10 16:45:25,371][123614] Updated weights for policy 1, policy_version 4740 (0.0008) [2023-10-10 16:45:25,725][123614] Updated weights for policy 1, policy_version 4750 (0.0009) [2023-10-10 16:45:26,084][123582] Updated weights for policy 0, policy_version 4771 (0.0009) [2023-10-10 16:45:26,100][123614] Updated weights for policy 1, policy_version 4760 (0.0008) [2023-10-10 16:45:26,449][123582] Updated weights for policy 0, policy_version 4781 (0.0007) [2023-10-10 16:45:26,817][123582] Updated weights for policy 0, policy_version 4791 (0.0007) [2023-10-10 16:45:28,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 9797632. Throughput: 0: 1816.6, 1: 1798.8. Samples: 2453926. Policy #0 lag: (min: 19.0, avg: 31.2, max: 32.0) [2023-10-10 16:45:28,789][122664] Avg episode reward: [(0, '16.110'), (1, '14.930')] [2023-10-10 16:45:28,791][123247] Saving new best policy, reward=16.110! [2023-10-10 16:45:29,721][123614] Updated weights for policy 1, policy_version 4770 (0.0007) [2023-10-10 16:45:30,089][123614] Updated weights for policy 1, policy_version 4780 (0.0008) [2023-10-10 16:45:30,450][123582] Updated weights for policy 0, policy_version 4801 (0.0010) [2023-10-10 16:45:30,457][123614] Updated weights for policy 1, policy_version 4790 (0.0007) [2023-10-10 16:45:30,818][123614] Updated weights for policy 1, policy_version 4800 (0.0009) [2023-10-10 16:45:30,824][123582] Updated weights for policy 0, policy_version 4811 (0.0008) [2023-10-10 16:45:31,199][123582] Updated weights for policy 0, policy_version 4821 (0.0007) [2023-10-10 16:45:31,567][123582] Updated weights for policy 0, policy_version 4831 (0.0007) [2023-10-10 16:45:33,788][122664] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 9863168. Throughput: 0: 1810.1, 1: 1801.1. Samples: 2475804. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-10-10 16:45:33,791][122664] Avg episode reward: [(0, '15.300'), (1, '14.350')] [2023-10-10 16:45:34,533][123614] Updated weights for policy 1, policy_version 4810 (0.0009) [2023-10-10 16:45:34,902][123614] Updated weights for policy 1, policy_version 4820 (0.0009) [2023-10-10 16:45:35,279][123614] Updated weights for policy 1, policy_version 4830 (0.0007) [2023-10-10 16:45:35,310][123582] Updated weights for policy 0, policy_version 4841 (0.0007) [2023-10-10 16:45:35,686][123582] Updated weights for policy 0, policy_version 4851 (0.0007) [2023-10-10 16:45:36,064][123582] Updated weights for policy 0, policy_version 4861 (0.0009) [2023-10-10 16:45:38,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 9928704. Throughput: 0: 1810.1, 1: 1812.8. Samples: 2498410. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-10-10 16:45:38,788][122664] Avg episode reward: [(0, '14.580'), (1, '12.410')] [2023-10-10 16:45:39,156][123614] Updated weights for policy 1, policy_version 4840 (0.0009) [2023-10-10 16:45:39,535][123614] Updated weights for policy 1, policy_version 4850 (0.0008) [2023-10-10 16:45:39,885][123582] Updated weights for policy 0, policy_version 4871 (0.0007) [2023-10-10 16:45:39,896][123614] Updated weights for policy 1, policy_version 4860 (0.0007) [2023-10-10 16:45:40,247][123582] Updated weights for policy 0, policy_version 4881 (0.0008) [2023-10-10 16:45:40,617][123582] Updated weights for policy 0, policy_version 4891 (0.0009) [2023-10-10 16:45:43,554][123614] Updated weights for policy 1, policy_version 4870 (0.0007) [2023-10-10 16:45:43,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 9994240. Throughput: 0: 1799.6, 1: 1795.1. Samples: 2507934. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-10-10 16:45:43,789][122664] Avg episode reward: [(0, '13.410'), (1, '13.650')] [2023-10-10 16:45:43,922][123614] Updated weights for policy 1, policy_version 4880 (0.0007) [2023-10-10 16:45:44,254][123582] Updated weights for policy 0, policy_version 4901 (0.0009) [2023-10-10 16:45:44,291][123614] Updated weights for policy 1, policy_version 4890 (0.0008) [2023-10-10 16:45:44,631][123582] Updated weights for policy 0, policy_version 4911 (0.0009) [2023-10-10 16:45:45,015][123582] Updated weights for policy 0, policy_version 4921 (0.0009) [2023-10-10 16:45:48,074][123614] Updated weights for policy 1, policy_version 4900 (0.0008) [2023-10-10 16:45:48,441][123614] Updated weights for policy 1, policy_version 4910 (0.0009) [2023-10-10 16:45:48,690][123582] Updated weights for policy 0, policy_version 4931 (0.0008) [2023-10-10 16:45:48,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 10059776. Throughput: 0: 1811.7, 1: 1806.7. Samples: 2530890. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-10-10 16:45:48,788][122664] Avg episode reward: [(0, '12.170'), (1, '13.340')] [2023-10-10 16:45:48,809][123614] Updated weights for policy 1, policy_version 4920 (0.0009) [2023-10-10 16:45:49,054][123582] Updated weights for policy 0, policy_version 4941 (0.0007) [2023-10-10 16:45:49,434][123582] Updated weights for policy 0, policy_version 4951 (0.0009) [2023-10-10 16:45:52,453][123614] Updated weights for policy 1, policy_version 4930 (0.0009) [2023-10-10 16:45:52,821][123614] Updated weights for policy 1, policy_version 4940 (0.0008) [2023-10-10 16:45:53,187][123614] Updated weights for policy 1, policy_version 4950 (0.0007) [2023-10-10 16:45:53,220][123582] Updated weights for policy 0, policy_version 4961 (0.0010) [2023-10-10 16:45:53,558][123614] Updated weights for policy 1, policy_version 4960 (0.0007) [2023-10-10 16:45:53,580][123582] Updated weights for policy 0, policy_version 4971 (0.0007) [2023-10-10 16:45:53,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 10158080. Throughput: 0: 1813.1, 1: 1796.0. Samples: 2551662. Policy #0 lag: (min: 5.0, avg: 6.4, max: 32.0) [2023-10-10 16:45:53,789][122664] Avg episode reward: [(0, '11.840'), (1, '13.450')] [2023-10-10 16:45:53,799][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000004960_5079040.pth... [2023-10-10 16:45:53,828][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000003264_3342336.pth [2023-10-10 16:45:53,960][123582] Updated weights for policy 0, policy_version 4981 (0.0008) [2023-10-10 16:45:54,334][123582] Updated weights for policy 0, policy_version 4991 (0.0008) [2023-10-10 16:45:54,367][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000004992_5111808.pth... [2023-10-10 16:45:54,407][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000003264_3342336.pth [2023-10-10 16:45:57,276][123614] Updated weights for policy 1, policy_version 4970 (0.0010) [2023-10-10 16:45:57,645][123614] Updated weights for policy 1, policy_version 4980 (0.0008) [2023-10-10 16:45:57,811][123582] Updated weights for policy 0, policy_version 5001 (0.0011) [2023-10-10 16:45:58,018][123614] Updated weights for policy 1, policy_version 4990 (0.0009) [2023-10-10 16:45:58,188][123582] Updated weights for policy 0, policy_version 5011 (0.0008) [2023-10-10 16:45:58,572][123582] Updated weights for policy 0, policy_version 5021 (0.0008) [2023-10-10 16:45:58,788][122664] Fps is (10 sec: 19660.7, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 10256384. Throughput: 0: 1812.0, 1: 1804.2. Samples: 2563480. Policy #0 lag: (min: 22.0, avg: 30.0, max: 54.0) [2023-10-10 16:45:58,789][122664] Avg episode reward: [(0, '11.790'), (1, '14.640')] [2023-10-10 16:46:01,770][123614] Updated weights for policy 1, policy_version 5000 (0.0009) [2023-10-10 16:46:02,140][123614] Updated weights for policy 1, policy_version 5010 (0.0008) [2023-10-10 16:46:02,301][123582] Updated weights for policy 0, policy_version 5031 (0.0007) [2023-10-10 16:46:02,502][123614] Updated weights for policy 1, policy_version 5020 (0.0007) [2023-10-10 16:46:02,666][123582] Updated weights for policy 0, policy_version 5041 (0.0008) [2023-10-10 16:46:03,033][123582] Updated weights for policy 0, policy_version 5051 (0.0009) [2023-10-10 16:46:03,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 10321920. Throughput: 0: 1815.3, 1: 1800.7. Samples: 2584178. Policy #0 lag: (min: 22.0, avg: 30.0, max: 54.0) [2023-10-10 16:46:03,789][122664] Avg episode reward: [(0, '11.590'), (1, '14.880')] [2023-10-10 16:46:06,076][123614] Updated weights for policy 1, policy_version 5030 (0.0008) [2023-10-10 16:46:06,453][123614] Updated weights for policy 1, policy_version 5040 (0.0007) [2023-10-10 16:46:06,775][123582] Updated weights for policy 0, policy_version 5061 (0.0008) [2023-10-10 16:46:06,814][123614] Updated weights for policy 1, policy_version 5050 (0.0008) [2023-10-10 16:46:07,143][123582] Updated weights for policy 0, policy_version 5071 (0.0010) [2023-10-10 16:46:07,521][123582] Updated weights for policy 0, policy_version 5081 (0.0010) [2023-10-10 16:46:08,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 10387456. Throughput: 0: 1811.0, 1: 1804.7. Samples: 2605874. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:46:08,789][122664] Avg episode reward: [(0, '12.280'), (1, '16.370')] [2023-10-10 16:46:08,800][123465] Saving new best policy, reward=16.370! [2023-10-10 16:46:10,637][123614] Updated weights for policy 1, policy_version 5060 (0.0008) [2023-10-10 16:46:11,000][123614] Updated weights for policy 1, policy_version 5070 (0.0008) [2023-10-10 16:46:11,297][123582] Updated weights for policy 0, policy_version 5091 (0.0009) [2023-10-10 16:46:11,377][123614] Updated weights for policy 1, policy_version 5080 (0.0007) [2023-10-10 16:46:11,662][123582] Updated weights for policy 0, policy_version 5101 (0.0007) [2023-10-10 16:46:12,032][123582] Updated weights for policy 0, policy_version 5111 (0.0008) [2023-10-10 16:46:13,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 10452992. Throughput: 0: 1811.6, 1: 1806.2. Samples: 2616726. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:46:13,789][122664] Avg episode reward: [(0, '13.230'), (1, '14.940')] [2023-10-10 16:46:15,184][123614] Updated weights for policy 1, policy_version 5090 (0.0007) [2023-10-10 16:46:15,553][123614] Updated weights for policy 1, policy_version 5100 (0.0009) [2023-10-10 16:46:15,755][123582] Updated weights for policy 0, policy_version 5121 (0.0008) [2023-10-10 16:46:15,926][123614] Updated weights for policy 1, policy_version 5110 (0.0008) [2023-10-10 16:46:16,116][123582] Updated weights for policy 0, policy_version 5131 (0.0007) [2023-10-10 16:46:16,288][123614] Updated weights for policy 1, policy_version 5120 (0.0008) [2023-10-10 16:46:16,494][123582] Updated weights for policy 0, policy_version 5141 (0.0007) [2023-10-10 16:46:16,870][123582] Updated weights for policy 0, policy_version 5151 (0.0008) [2023-10-10 16:46:18,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 10518528. Throughput: 0: 1805.6, 1: 1801.2. Samples: 2638110. Policy #0 lag: (min: 1.0, avg: 28.8, max: 32.0) [2023-10-10 16:46:18,788][122664] Avg episode reward: [(0, '12.830'), (1, '15.280')] [2023-10-10 16:46:19,910][123614] Updated weights for policy 1, policy_version 5130 (0.0007) [2023-10-10 16:46:20,272][123614] Updated weights for policy 1, policy_version 5140 (0.0007) [2023-10-10 16:46:20,633][123614] Updated weights for policy 1, policy_version 5150 (0.0008) [2023-10-10 16:46:20,698][123582] Updated weights for policy 0, policy_version 5161 (0.0007) [2023-10-10 16:46:21,080][123582] Updated weights for policy 0, policy_version 5171 (0.0011) [2023-10-10 16:46:21,467][123582] Updated weights for policy 0, policy_version 5181 (0.0011) [2023-10-10 16:46:23,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 10584064. Throughput: 0: 1799.9, 1: 1813.8. Samples: 2661026. Policy #0 lag: (min: 1.0, avg: 28.8, max: 32.0) [2023-10-10 16:46:23,789][122664] Avg episode reward: [(0, '13.680'), (1, '15.600')] [2023-10-10 16:46:24,171][123614] Updated weights for policy 1, policy_version 5160 (0.0010) [2023-10-10 16:46:24,552][123614] Updated weights for policy 1, policy_version 5170 (0.0010) [2023-10-10 16:46:24,912][123614] Updated weights for policy 1, policy_version 5180 (0.0009) [2023-10-10 16:46:25,212][123582] Updated weights for policy 0, policy_version 5191 (0.0009) [2023-10-10 16:46:25,584][123582] Updated weights for policy 0, policy_version 5201 (0.0009) [2023-10-10 16:46:25,958][123582] Updated weights for policy 0, policy_version 5211 (0.0007) [2023-10-10 16:46:28,732][123614] Updated weights for policy 1, policy_version 5190 (0.0009) [2023-10-10 16:46:28,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 10649600. Throughput: 0: 1801.5, 1: 1816.2. Samples: 2670728. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:46:28,789][122664] Avg episode reward: [(0, '14.910'), (1, '15.480')] [2023-10-10 16:46:29,108][123614] Updated weights for policy 1, policy_version 5200 (0.0010) [2023-10-10 16:46:29,468][123614] Updated weights for policy 1, policy_version 5210 (0.0009) [2023-10-10 16:46:29,667][123582] Updated weights for policy 0, policy_version 5221 (0.0008) [2023-10-10 16:46:30,046][123582] Updated weights for policy 0, policy_version 5231 (0.0010) [2023-10-10 16:46:30,414][123582] Updated weights for policy 0, policy_version 5241 (0.0007) [2023-10-10 16:46:33,104][123614] Updated weights for policy 1, policy_version 5220 (0.0008) [2023-10-10 16:46:33,476][123614] Updated weights for policy 1, policy_version 5230 (0.0007) [2023-10-10 16:46:33,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 10715136. Throughput: 0: 1801.7, 1: 1815.9. Samples: 2693682. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:46:33,788][122664] Avg episode reward: [(0, '15.100'), (1, '14.330')] [2023-10-10 16:46:33,843][123614] Updated weights for policy 1, policy_version 5240 (0.0007) [2023-10-10 16:46:33,935][123582] Updated weights for policy 0, policy_version 5251 (0.0008) [2023-10-10 16:46:34,306][123582] Updated weights for policy 0, policy_version 5261 (0.0008) [2023-10-10 16:46:34,672][123582] Updated weights for policy 0, policy_version 5271 (0.0010) [2023-10-10 16:46:37,521][123614] Updated weights for policy 1, policy_version 5250 (0.0008) [2023-10-10 16:46:37,898][123614] Updated weights for policy 1, policy_version 5260 (0.0008) [2023-10-10 16:46:38,266][123614] Updated weights for policy 1, policy_version 5270 (0.0008) [2023-10-10 16:46:38,362][123582] Updated weights for policy 0, policy_version 5281 (0.0008) [2023-10-10 16:46:38,636][123614] Updated weights for policy 1, policy_version 5280 (0.0007) [2023-10-10 16:46:38,726][123582] Updated weights for policy 0, policy_version 5291 (0.0008) [2023-10-10 16:46:38,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 10813440. Throughput: 0: 1811.7, 1: 1814.9. Samples: 2714858. Policy #0 lag: (min: 5.0, avg: 12.9, max: 37.0) [2023-10-10 16:46:38,789][122664] Avg episode reward: [(0, '14.770'), (1, '16.960')] [2023-10-10 16:46:38,796][123465] Saving new best policy, reward=16.960! [2023-10-10 16:46:39,099][123582] Updated weights for policy 0, policy_version 5301 (0.0007) [2023-10-10 16:46:39,470][123582] Updated weights for policy 0, policy_version 5311 (0.0008) [2023-10-10 16:46:42,244][123614] Updated weights for policy 1, policy_version 5290 (0.0010) [2023-10-10 16:46:42,616][123614] Updated weights for policy 1, policy_version 5300 (0.0007) [2023-10-10 16:46:42,974][123614] Updated weights for policy 1, policy_version 5310 (0.0007) [2023-10-10 16:46:43,078][123582] Updated weights for policy 0, policy_version 5321 (0.0008) [2023-10-10 16:46:43,456][123582] Updated weights for policy 0, policy_version 5331 (0.0007) [2023-10-10 16:46:43,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 10878976. Throughput: 0: 1802.2, 1: 1819.2. Samples: 2726444. Policy #0 lag: (min: 5.0, avg: 12.9, max: 37.0) [2023-10-10 16:46:43,789][122664] Avg episode reward: [(0, '15.750'), (1, '15.160')] [2023-10-10 16:46:43,823][123582] Updated weights for policy 0, policy_version 5341 (0.0007) [2023-10-10 16:46:46,684][123614] Updated weights for policy 1, policy_version 5320 (0.0007) [2023-10-10 16:46:47,060][123614] Updated weights for policy 1, policy_version 5330 (0.0007) [2023-10-10 16:46:47,429][123614] Updated weights for policy 1, policy_version 5340 (0.0009) [2023-10-10 16:46:47,645][123582] Updated weights for policy 0, policy_version 5351 (0.0009) [2023-10-10 16:46:48,011][123582] Updated weights for policy 0, policy_version 5361 (0.0009) [2023-10-10 16:46:48,380][123582] Updated weights for policy 0, policy_version 5371 (0.0011) [2023-10-10 16:46:48,788][122664] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 10977280. Throughput: 0: 1815.2, 1: 1827.7. Samples: 2748108. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 16:46:48,789][122664] Avg episode reward: [(0, '16.620'), (1, '13.770')] [2023-10-10 16:46:48,789][123247] Saving new best policy, reward=16.620! [2023-10-10 16:46:51,155][123614] Updated weights for policy 1, policy_version 5350 (0.0008) [2023-10-10 16:46:51,523][123614] Updated weights for policy 1, policy_version 5360 (0.0007) [2023-10-10 16:46:51,893][123614] Updated weights for policy 1, policy_version 5370 (0.0007) [2023-10-10 16:46:52,060][123582] Updated weights for policy 0, policy_version 5381 (0.0010) [2023-10-10 16:46:52,434][123582] Updated weights for policy 0, policy_version 5391 (0.0010) [2023-10-10 16:46:52,795][123582] Updated weights for policy 0, policy_version 5401 (0.0010) [2023-10-10 16:46:53,788][122664] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 11042816. Throughput: 0: 1809.2, 1: 1825.8. Samples: 2769448. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-10 16:46:53,790][122664] Avg episode reward: [(0, '15.650'), (1, '13.800')] [2023-10-10 16:46:55,595][123614] Updated weights for policy 1, policy_version 5380 (0.0007) [2023-10-10 16:46:55,971][123614] Updated weights for policy 1, policy_version 5390 (0.0007) [2023-10-10 16:46:56,339][123614] Updated weights for policy 1, policy_version 5400 (0.0007) [2023-10-10 16:46:56,501][123582] Updated weights for policy 0, policy_version 5411 (0.0009) [2023-10-10 16:46:56,871][123582] Updated weights for policy 0, policy_version 5421 (0.0007) [2023-10-10 16:46:57,251][123582] Updated weights for policy 0, policy_version 5431 (0.0008) [2023-10-10 16:46:58,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 11108352. Throughput: 0: 1819.1, 1: 1824.4. Samples: 2780686. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-10 16:46:58,789][122664] Avg episode reward: [(0, '14.240'), (1, '12.080')] [2023-10-10 16:47:00,005][123614] Updated weights for policy 1, policy_version 5410 (0.0008) [2023-10-10 16:47:00,374][123614] Updated weights for policy 1, policy_version 5420 (0.0009) [2023-10-10 16:47:00,742][123614] Updated weights for policy 1, policy_version 5430 (0.0011) [2023-10-10 16:47:01,019][123582] Updated weights for policy 0, policy_version 5441 (0.0009) [2023-10-10 16:47:01,114][123614] Updated weights for policy 1, policy_version 5440 (0.0008) [2023-10-10 16:47:01,393][123582] Updated weights for policy 0, policy_version 5451 (0.0007) [2023-10-10 16:47:01,761][123582] Updated weights for policy 0, policy_version 5461 (0.0007) [2023-10-10 16:47:02,134][123582] Updated weights for policy 0, policy_version 5471 (0.0007) [2023-10-10 16:47:03,788][122664] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 11173888. Throughput: 0: 1813.2, 1: 1826.0. Samples: 2801872. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:47:03,789][122664] Avg episode reward: [(0, '14.270'), (1, '13.280')] [2023-10-10 16:47:04,836][123614] Updated weights for policy 1, policy_version 5450 (0.0008) [2023-10-10 16:47:05,213][123614] Updated weights for policy 1, policy_version 5460 (0.0007) [2023-10-10 16:47:05,576][123614] Updated weights for policy 1, policy_version 5470 (0.0008) [2023-10-10 16:47:05,879][123582] Updated weights for policy 0, policy_version 5481 (0.0009) [2023-10-10 16:47:06,248][123582] Updated weights for policy 0, policy_version 5491 (0.0008) [2023-10-10 16:47:06,621][123582] Updated weights for policy 0, policy_version 5501 (0.0009) [2023-10-10 16:47:08,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 11239424. Throughput: 0: 1816.4, 1: 1815.9. Samples: 2824478. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:47:08,789][122664] Avg episode reward: [(0, '13.020'), (1, '14.440')] [2023-10-10 16:47:09,408][123614] Updated weights for policy 1, policy_version 5480 (0.0007) [2023-10-10 16:47:09,796][123614] Updated weights for policy 1, policy_version 5490 (0.0007) [2023-10-10 16:47:10,163][123614] Updated weights for policy 1, policy_version 5500 (0.0008) [2023-10-10 16:47:10,362][123582] Updated weights for policy 0, policy_version 5511 (0.0009) [2023-10-10 16:47:10,741][123582] Updated weights for policy 0, policy_version 5521 (0.0009) [2023-10-10 16:47:11,104][123582] Updated weights for policy 0, policy_version 5531 (0.0009) [2023-10-10 16:47:13,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 11304960. Throughput: 0: 1817.0, 1: 1812.9. Samples: 2834074. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:47:13,789][122664] Avg episode reward: [(0, '14.020'), (1, '14.970')] [2023-10-10 16:47:13,933][123614] Updated weights for policy 1, policy_version 5510 (0.0009) [2023-10-10 16:47:14,296][123614] Updated weights for policy 1, policy_version 5520 (0.0011) [2023-10-10 16:47:14,663][123614] Updated weights for policy 1, policy_version 5530 (0.0009) [2023-10-10 16:47:14,751][123582] Updated weights for policy 0, policy_version 5541 (0.0008) [2023-10-10 16:47:15,127][123582] Updated weights for policy 0, policy_version 5551 (0.0009) [2023-10-10 16:47:15,497][123582] Updated weights for policy 0, policy_version 5561 (0.0007) [2023-10-10 16:47:18,327][123614] Updated weights for policy 1, policy_version 5540 (0.0007) [2023-10-10 16:47:18,700][123614] Updated weights for policy 1, policy_version 5550 (0.0007) [2023-10-10 16:47:18,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 11370496. Throughput: 0: 1815.5, 1: 1809.2. Samples: 2856790. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:47:18,789][122664] Avg episode reward: [(0, '12.810'), (1, '14.940')] [2023-10-10 16:47:19,080][123614] Updated weights for policy 1, policy_version 5560 (0.0009) [2023-10-10 16:47:19,244][123582] Updated weights for policy 0, policy_version 5571 (0.0008) [2023-10-10 16:47:19,619][123582] Updated weights for policy 0, policy_version 5581 (0.0008) [2023-10-10 16:47:19,984][123582] Updated weights for policy 0, policy_version 5591 (0.0009) [2023-10-10 16:47:22,663][123614] Updated weights for policy 1, policy_version 5570 (0.0008) [2023-10-10 16:47:23,029][123614] Updated weights for policy 1, policy_version 5580 (0.0007) [2023-10-10 16:47:23,404][123614] Updated weights for policy 1, policy_version 5590 (0.0008) [2023-10-10 16:47:23,582][123582] Updated weights for policy 0, policy_version 5601 (0.0008) [2023-10-10 16:47:23,769][123614] Updated weights for policy 1, policy_version 5600 (0.0008) [2023-10-10 16:47:23,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 11468800. Throughput: 0: 1818.9, 1: 1814.4. Samples: 2878354. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:47:23,788][122664] Avg episode reward: [(0, '13.880'), (1, '16.200')] [2023-10-10 16:47:23,940][123582] Updated weights for policy 0, policy_version 5611 (0.0008) [2023-10-10 16:47:24,311][123582] Updated weights for policy 0, policy_version 5621 (0.0010) [2023-10-10 16:47:24,679][123582] Updated weights for policy 0, policy_version 5631 (0.0009) [2023-10-10 16:47:27,503][123614] Updated weights for policy 1, policy_version 5610 (0.0009) [2023-10-10 16:47:27,871][123614] Updated weights for policy 1, policy_version 5620 (0.0007) [2023-10-10 16:47:28,248][123614] Updated weights for policy 1, policy_version 5630 (0.0008) [2023-10-10 16:47:28,376][123582] Updated weights for policy 0, policy_version 5641 (0.0008) [2023-10-10 16:47:28,752][123582] Updated weights for policy 0, policy_version 5651 (0.0011) [2023-10-10 16:47:28,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 11534336. Throughput: 0: 1815.8, 1: 1808.2. Samples: 2889526. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:47:28,789][122664] Avg episode reward: [(0, '13.840'), (1, '16.090')] [2023-10-10 16:47:29,117][123582] Updated weights for policy 0, policy_version 5661 (0.0010) [2023-10-10 16:47:32,094][123614] Updated weights for policy 1, policy_version 5640 (0.0007) [2023-10-10 16:47:32,471][123614] Updated weights for policy 1, policy_version 5650 (0.0007) [2023-10-10 16:47:32,725][123582] Updated weights for policy 0, policy_version 5671 (0.0008) [2023-10-10 16:47:32,841][123614] Updated weights for policy 1, policy_version 5660 (0.0007) [2023-10-10 16:47:33,094][123582] Updated weights for policy 0, policy_version 5681 (0.0008) [2023-10-10 16:47:33,466][123582] Updated weights for policy 0, policy_version 5691 (0.0008) [2023-10-10 16:47:33,788][122664] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 11632640. Throughput: 0: 1815.0, 1: 1802.9. Samples: 2910912. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) [2023-10-10 16:47:33,789][122664] Avg episode reward: [(0, '13.970'), (1, '15.900')] [2023-10-10 16:47:36,600][123614] Updated weights for policy 1, policy_version 5670 (0.0008) [2023-10-10 16:47:36,972][123614] Updated weights for policy 1, policy_version 5680 (0.0007) [2023-10-10 16:47:37,066][123582] Updated weights for policy 0, policy_version 5701 (0.0008) [2023-10-10 16:47:37,341][123614] Updated weights for policy 1, policy_version 5690 (0.0008) [2023-10-10 16:47:37,441][123582] Updated weights for policy 0, policy_version 5711 (0.0008) [2023-10-10 16:47:37,802][123582] Updated weights for policy 0, policy_version 5721 (0.0009) [2023-10-10 16:47:38,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 11698176. Throughput: 0: 1814.5, 1: 1791.3. Samples: 2931710. Policy #0 lag: (min: 1.0, avg: 3.7, max: 25.0) [2023-10-10 16:47:38,789][122664] Avg episode reward: [(0, '14.210'), (1, '16.440')] [2023-10-10 16:47:41,147][123614] Updated weights for policy 1, policy_version 5700 (0.0008) [2023-10-10 16:47:41,413][123582] Updated weights for policy 0, policy_version 5731 (0.0008) [2023-10-10 16:47:41,508][123614] Updated weights for policy 1, policy_version 5710 (0.0007) [2023-10-10 16:47:41,795][123582] Updated weights for policy 0, policy_version 5741 (0.0009) [2023-10-10 16:47:41,873][123614] Updated weights for policy 1, policy_version 5720 (0.0007) [2023-10-10 16:47:42,160][123582] Updated weights for policy 0, policy_version 5751 (0.0009) [2023-10-10 16:47:43,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 11763712. Throughput: 0: 1815.2, 1: 1805.4. Samples: 2943612. Policy #0 lag: (min: 1.0, avg: 3.7, max: 25.0) [2023-10-10 16:47:43,789][122664] Avg episode reward: [(0, '14.580'), (1, '15.880')] [2023-10-10 16:47:45,864][123614] Updated weights for policy 1, policy_version 5730 (0.0007) [2023-10-10 16:47:45,952][123582] Updated weights for policy 0, policy_version 5761 (0.0007) [2023-10-10 16:47:46,229][123614] Updated weights for policy 1, policy_version 5740 (0.0009) [2023-10-10 16:47:46,324][123582] Updated weights for policy 0, policy_version 5771 (0.0007) [2023-10-10 16:47:46,593][123614] Updated weights for policy 1, policy_version 5750 (0.0010) [2023-10-10 16:47:46,698][123582] Updated weights for policy 0, policy_version 5781 (0.0007) [2023-10-10 16:47:46,964][123614] Updated weights for policy 1, policy_version 5760 (0.0009) [2023-10-10 16:47:47,062][123582] Updated weights for policy 0, policy_version 5791 (0.0010) [2023-10-10 16:47:48,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 11829248. Throughput: 0: 1813.4, 1: 1786.0. Samples: 2963844. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:47:48,788][122664] Avg episode reward: [(0, '14.260'), (1, '13.580')] [2023-10-10 16:47:50,638][123614] Updated weights for policy 1, policy_version 5770 (0.0009) [2023-10-10 16:47:50,879][123582] Updated weights for policy 0, policy_version 5801 (0.0007) [2023-10-10 16:47:51,008][123614] Updated weights for policy 1, policy_version 5780 (0.0010) [2023-10-10 16:47:51,249][123582] Updated weights for policy 0, policy_version 5811 (0.0007) [2023-10-10 16:47:51,381][123614] Updated weights for policy 1, policy_version 5790 (0.0008) [2023-10-10 16:47:51,617][123582] Updated weights for policy 0, policy_version 5821 (0.0009) [2023-10-10 16:47:53,788][122664] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 11894784. Throughput: 0: 1813.2, 1: 1787.3. Samples: 2986504. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:47:53,790][122664] Avg episode reward: [(0, '15.890'), (1, '12.800')] [2023-10-10 16:47:53,799][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000005792_5931008.pth... [2023-10-10 16:47:53,800][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000005824_5963776.pth... [2023-10-10 16:47:53,829][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000004096_4194304.pth [2023-10-10 16:47:53,836][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000004128_4227072.pth [2023-10-10 16:47:55,118][123614] Updated weights for policy 1, policy_version 5800 (0.0009) [2023-10-10 16:47:55,442][123582] Updated weights for policy 0, policy_version 5831 (0.0008) [2023-10-10 16:47:55,487][123614] Updated weights for policy 1, policy_version 5810 (0.0008) [2023-10-10 16:47:55,817][123582] Updated weights for policy 0, policy_version 5841 (0.0007) [2023-10-10 16:47:55,867][123614] Updated weights for policy 1, policy_version 5820 (0.0009) [2023-10-10 16:47:56,195][123582] Updated weights for policy 0, policy_version 5851 (0.0010) [2023-10-10 16:47:58,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.6, 300 sec: 14440.1). Total num frames: 11960320. Throughput: 0: 1813.5, 1: 1787.1. Samples: 2996098. Policy #0 lag: (min: 17.0, avg: 31.0, max: 49.0) [2023-10-10 16:47:58,788][122664] Avg episode reward: [(0, '15.870'), (1, '12.010')] [2023-10-10 16:47:59,549][123614] Updated weights for policy 1, policy_version 5830 (0.0009) [2023-10-10 16:47:59,876][123582] Updated weights for policy 0, policy_version 5861 (0.0010) [2023-10-10 16:47:59,910][123614] Updated weights for policy 1, policy_version 5840 (0.0007) [2023-10-10 16:48:00,254][123582] Updated weights for policy 0, policy_version 5871 (0.0009) [2023-10-10 16:48:00,279][123614] Updated weights for policy 1, policy_version 5850 (0.0009) [2023-10-10 16:48:00,624][123582] Updated weights for policy 0, policy_version 5881 (0.0011) [2023-10-10 16:48:03,788][122664] Fps is (10 sec: 13107.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 12025856. Throughput: 0: 1810.4, 1: 1785.6. Samples: 3018606. Policy #0 lag: (min: 17.0, avg: 31.0, max: 49.0) [2023-10-10 16:48:03,788][122664] Avg episode reward: [(0, '16.530'), (1, '13.350')] [2023-10-10 16:48:03,981][123614] Updated weights for policy 1, policy_version 5860 (0.0008) [2023-10-10 16:48:04,285][123582] Updated weights for policy 0, policy_version 5891 (0.0010) [2023-10-10 16:48:04,346][123614] Updated weights for policy 1, policy_version 5870 (0.0007) [2023-10-10 16:48:04,654][123582] Updated weights for policy 0, policy_version 5901 (0.0010) [2023-10-10 16:48:04,706][123614] Updated weights for policy 1, policy_version 5880 (0.0009) [2023-10-10 16:48:05,032][123582] Updated weights for policy 0, policy_version 5911 (0.0008) [2023-10-10 16:48:08,304][123614] Updated weights for policy 1, policy_version 5890 (0.0007) [2023-10-10 16:48:08,645][123582] Updated weights for policy 0, policy_version 5921 (0.0009) [2023-10-10 16:48:08,669][123614] Updated weights for policy 1, policy_version 5900 (0.0008) [2023-10-10 16:48:08,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 12091392. Throughput: 0: 1807.9, 1: 1808.8. Samples: 3041102. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 16:48:08,789][122664] Avg episode reward: [(0, '16.350'), (1, '13.160')] [2023-10-10 16:48:09,021][123582] Updated weights for policy 0, policy_version 5931 (0.0008) [2023-10-10 16:48:09,037][123614] Updated weights for policy 1, policy_version 5910 (0.0009) [2023-10-10 16:48:09,387][123582] Updated weights for policy 0, policy_version 5941 (0.0007) [2023-10-10 16:48:09,410][123614] Updated weights for policy 1, policy_version 5920 (0.0008) [2023-10-10 16:48:09,758][123582] Updated weights for policy 0, policy_version 5951 (0.0007) [2023-10-10 16:48:13,275][123614] Updated weights for policy 1, policy_version 5930 (0.0007) [2023-10-10 16:48:13,401][123582] Updated weights for policy 0, policy_version 5961 (0.0007) [2023-10-10 16:48:13,641][123614] Updated weights for policy 1, policy_version 5940 (0.0008) [2023-10-10 16:48:13,775][123582] Updated weights for policy 0, policy_version 5971 (0.0007) [2023-10-10 16:48:13,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 12156928. Throughput: 0: 1808.4, 1: 1788.1. Samples: 3051366. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 16:48:13,788][122664] Avg episode reward: [(0, '17.210'), (1, '13.770')] [2023-10-10 16:48:14,004][123614] Updated weights for policy 1, policy_version 5950 (0.0007) [2023-10-10 16:48:14,144][123582] Updated weights for policy 0, policy_version 5981 (0.0007) [2023-10-10 16:48:14,260][123247] Saving new best policy, reward=17.210! [2023-10-10 16:48:17,604][123614] Updated weights for policy 1, policy_version 5960 (0.0008) [2023-10-10 16:48:17,887][123582] Updated weights for policy 0, policy_version 5991 (0.0010) [2023-10-10 16:48:17,967][123614] Updated weights for policy 1, policy_version 5970 (0.0008) [2023-10-10 16:48:18,264][123582] Updated weights for policy 0, policy_version 6001 (0.0009) [2023-10-10 16:48:18,345][123614] Updated weights for policy 1, policy_version 5980 (0.0007) [2023-10-10 16:48:18,629][123582] Updated weights for policy 0, policy_version 6011 (0.0008) [2023-10-10 16:48:18,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 12255232. Throughput: 0: 1807.6, 1: 1813.5. Samples: 3073862. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:48:18,789][122664] Avg episode reward: [(0, '16.080'), (1, '16.290')] [2023-10-10 16:48:22,161][123614] Updated weights for policy 1, policy_version 5990 (0.0009) [2023-10-10 16:48:22,200][123582] Updated weights for policy 0, policy_version 6021 (0.0008) [2023-10-10 16:48:22,524][123614] Updated weights for policy 1, policy_version 6000 (0.0008) [2023-10-10 16:48:22,567][123582] Updated weights for policy 0, policy_version 6031 (0.0009) [2023-10-10 16:48:22,891][123614] Updated weights for policy 1, policy_version 6010 (0.0008) [2023-10-10 16:48:22,934][123582] Updated weights for policy 0, policy_version 6041 (0.0008) [2023-10-10 16:48:23,788][122664] Fps is (10 sec: 19660.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 12353536. Throughput: 0: 1810.7, 1: 1795.2. Samples: 3093974. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:48:23,788][122664] Avg episode reward: [(0, '15.960'), (1, '15.720')] [2023-10-10 16:48:26,699][123582] Updated weights for policy 0, policy_version 6051 (0.0008) [2023-10-10 16:48:26,731][123614] Updated weights for policy 1, policy_version 6020 (0.0008) [2023-10-10 16:48:27,069][123582] Updated weights for policy 0, policy_version 6061 (0.0008) [2023-10-10 16:48:27,093][123614] Updated weights for policy 1, policy_version 6030 (0.0008) [2023-10-10 16:48:27,444][123582] Updated weights for policy 0, policy_version 6071 (0.0008) [2023-10-10 16:48:27,457][123614] Updated weights for policy 1, policy_version 6040 (0.0007) [2023-10-10 16:48:28,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 12419072. Throughput: 0: 1812.6, 1: 1808.1. Samples: 3106544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:48:28,788][122664] Avg episode reward: [(0, '15.290'), (1, '15.920')] [2023-10-10 16:48:31,152][123582] Updated weights for policy 0, policy_version 6081 (0.0008) [2023-10-10 16:48:31,171][123614] Updated weights for policy 1, policy_version 6050 (0.0009) [2023-10-10 16:48:31,529][123582] Updated weights for policy 0, policy_version 6091 (0.0010) [2023-10-10 16:48:31,540][123614] Updated weights for policy 1, policy_version 6060 (0.0009) [2023-10-10 16:48:31,895][123582] Updated weights for policy 0, policy_version 6101 (0.0009) [2023-10-10 16:48:31,915][123614] Updated weights for policy 1, policy_version 6070 (0.0007) [2023-10-10 16:48:32,253][123582] Updated weights for policy 0, policy_version 6111 (0.0008) [2023-10-10 16:48:32,284][123614] Updated weights for policy 1, policy_version 6080 (0.0008) [2023-10-10 16:48:33,788][122664] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 12484608. Throughput: 0: 1809.9, 1: 1795.6. Samples: 3126096. Policy #0 lag: (min: 24.0, avg: 47.3, max: 56.0) [2023-10-10 16:48:33,789][122664] Avg episode reward: [(0, '15.050'), (1, '15.560')] [2023-10-10 16:48:36,058][123614] Updated weights for policy 1, policy_version 6090 (0.0007) [2023-10-10 16:48:36,202][123582] Updated weights for policy 0, policy_version 6121 (0.0007) [2023-10-10 16:48:36,437][123614] Updated weights for policy 1, policy_version 6100 (0.0007) [2023-10-10 16:48:36,578][123582] Updated weights for policy 0, policy_version 6131 (0.0007) [2023-10-10 16:48:36,796][123614] Updated weights for policy 1, policy_version 6110 (0.0009) [2023-10-10 16:48:36,945][123582] Updated weights for policy 0, policy_version 6141 (0.0009) [2023-10-10 16:48:38,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 12550144. Throughput: 0: 1800.8, 1: 1797.5. Samples: 3148426. Policy #0 lag: (min: 24.0, avg: 47.3, max: 56.0) [2023-10-10 16:48:38,789][122664] Avg episode reward: [(0, '14.830'), (1, '16.840')] [2023-10-10 16:48:40,515][123614] Updated weights for policy 1, policy_version 6120 (0.0008) [2023-10-10 16:48:40,627][123582] Updated weights for policy 0, policy_version 6151 (0.0009) [2023-10-10 16:48:40,897][123614] Updated weights for policy 1, policy_version 6130 (0.0007) [2023-10-10 16:48:41,000][123582] Updated weights for policy 0, policy_version 6161 (0.0009) [2023-10-10 16:48:41,266][123614] Updated weights for policy 1, policy_version 6140 (0.0009) [2023-10-10 16:48:41,376][123582] Updated weights for policy 0, policy_version 6171 (0.0007) [2023-10-10 16:48:43,788][122664] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 12615680. Throughput: 0: 1807.8, 1: 1798.8. Samples: 3158396. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-10-10 16:48:43,788][122664] Avg episode reward: [(0, '14.780'), (1, '16.500')] [2023-10-10 16:48:44,980][123614] Updated weights for policy 1, policy_version 6150 (0.0007) [2023-10-10 16:48:45,103][123582] Updated weights for policy 0, policy_version 6181 (0.0007) [2023-10-10 16:48:45,355][123614] Updated weights for policy 1, policy_version 6160 (0.0007) [2023-10-10 16:48:45,474][123582] Updated weights for policy 0, policy_version 6191 (0.0007) [2023-10-10 16:48:45,724][123614] Updated weights for policy 1, policy_version 6170 (0.0007) [2023-10-10 16:48:45,853][123582] Updated weights for policy 0, policy_version 6201 (0.0008) [2023-10-10 16:48:48,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 12681216. Throughput: 0: 1801.6, 1: 1805.5. Samples: 3180926. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-10-10 16:48:48,789][122664] Avg episode reward: [(0, '15.230'), (1, '18.000')] [2023-10-10 16:48:48,791][123465] Saving new best policy, reward=18.000! [2023-10-10 16:48:49,332][123614] Updated weights for policy 1, policy_version 6180 (0.0008) [2023-10-10 16:48:49,666][123582] Updated weights for policy 0, policy_version 6211 (0.0008) [2023-10-10 16:48:49,700][123614] Updated weights for policy 1, policy_version 6190 (0.0009) [2023-10-10 16:48:50,026][123582] Updated weights for policy 0, policy_version 6221 (0.0008) [2023-10-10 16:48:50,067][123614] Updated weights for policy 1, policy_version 6200 (0.0008) [2023-10-10 16:48:50,396][123582] Updated weights for policy 0, policy_version 6231 (0.0009) [2023-10-10 16:48:53,727][123614] Updated weights for policy 1, policy_version 6210 (0.0008) [2023-10-10 16:48:53,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.6, 300 sec: 14440.1). Total num frames: 12746752. Throughput: 0: 1799.4, 1: 1808.2. Samples: 3203446. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-10 16:48:53,789][122664] Avg episode reward: [(0, '15.500'), (1, '17.250')] [2023-10-10 16:48:54,088][123614] Updated weights for policy 1, policy_version 6220 (0.0008) [2023-10-10 16:48:54,246][123582] Updated weights for policy 0, policy_version 6241 (0.0009) [2023-10-10 16:48:54,459][123614] Updated weights for policy 1, policy_version 6230 (0.0008) [2023-10-10 16:48:54,613][123582] Updated weights for policy 0, policy_version 6251 (0.0009) [2023-10-10 16:48:54,825][123614] Updated weights for policy 1, policy_version 6240 (0.0008) [2023-10-10 16:48:54,987][123582] Updated weights for policy 0, policy_version 6261 (0.0009) [2023-10-10 16:48:55,372][123582] Updated weights for policy 0, policy_version 6271 (0.0011) [2023-10-10 16:48:58,517][123614] Updated weights for policy 1, policy_version 6250 (0.0007) [2023-10-10 16:48:58,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 12812288. Throughput: 0: 1798.8, 1: 1800.9. Samples: 3213350. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-10 16:48:58,788][122664] Avg episode reward: [(0, '17.010'), (1, '17.730')] [2023-10-10 16:48:58,885][123614] Updated weights for policy 1, policy_version 6260 (0.0008) [2023-10-10 16:48:59,006][123582] Updated weights for policy 0, policy_version 6281 (0.0008) [2023-10-10 16:48:59,252][123614] Updated weights for policy 1, policy_version 6270 (0.0009) [2023-10-10 16:48:59,376][123582] Updated weights for policy 0, policy_version 6291 (0.0008) [2023-10-10 16:48:59,745][123582] Updated weights for policy 0, policy_version 6301 (0.0008) [2023-10-10 16:49:02,968][123614] Updated weights for policy 1, policy_version 6280 (0.0009) [2023-10-10 16:49:03,340][123614] Updated weights for policy 1, policy_version 6290 (0.0009) [2023-10-10 16:49:03,433][123582] Updated weights for policy 0, policy_version 6311 (0.0008) [2023-10-10 16:49:03,712][123614] Updated weights for policy 1, policy_version 6300 (0.0008) [2023-10-10 16:49:03,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 12877824. Throughput: 0: 1795.8, 1: 1808.0. Samples: 3236036. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:49:03,788][122664] Avg episode reward: [(0, '16.900'), (1, '18.290')] [2023-10-10 16:49:03,805][123582] Updated weights for policy 0, policy_version 6321 (0.0007) [2023-10-10 16:49:03,850][123465] Saving new best policy, reward=18.290! [2023-10-10 16:49:04,189][123582] Updated weights for policy 0, policy_version 6331 (0.0011) [2023-10-10 16:49:07,469][123614] Updated weights for policy 1, policy_version 6310 (0.0008) [2023-10-10 16:49:07,847][123614] Updated weights for policy 1, policy_version 6320 (0.0009) [2023-10-10 16:49:07,997][123582] Updated weights for policy 0, policy_version 6341 (0.0008) [2023-10-10 16:49:08,209][123614] Updated weights for policy 1, policy_version 6330 (0.0009) [2023-10-10 16:49:08,366][123582] Updated weights for policy 0, policy_version 6351 (0.0008) [2023-10-10 16:49:08,745][123582] Updated weights for policy 0, policy_version 6361 (0.0008) [2023-10-10 16:49:08,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 12976128. Throughput: 0: 1806.1, 1: 1798.8. Samples: 3256198. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:49:08,789][122664] Avg episode reward: [(0, '16.880'), (1, '18.410')] [2023-10-10 16:49:08,797][123465] Saving new best policy, reward=18.410! [2023-10-10 16:49:12,102][123614] Updated weights for policy 1, policy_version 6340 (0.0008) [2023-10-10 16:49:12,406][123582] Updated weights for policy 0, policy_version 6371 (0.0009) [2023-10-10 16:49:12,473][123614] Updated weights for policy 1, policy_version 6350 (0.0008) [2023-10-10 16:49:12,778][123582] Updated weights for policy 0, policy_version 6381 (0.0008) [2023-10-10 16:49:12,844][123614] Updated weights for policy 1, policy_version 6360 (0.0009) [2023-10-10 16:49:13,149][123582] Updated weights for policy 0, policy_version 6391 (0.0008) [2023-10-10 16:49:13,788][122664] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 13074432. Throughput: 0: 1787.3, 1: 1803.2. Samples: 3268114. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 16:49:13,789][122664] Avg episode reward: [(0, '16.880'), (1, '17.560')] [2023-10-10 16:49:16,416][123614] Updated weights for policy 1, policy_version 6370 (0.0008) [2023-10-10 16:49:16,781][123614] Updated weights for policy 1, policy_version 6380 (0.0008) [2023-10-10 16:49:16,890][123582] Updated weights for policy 0, policy_version 6401 (0.0007) [2023-10-10 16:49:17,152][123614] Updated weights for policy 1, policy_version 6390 (0.0007) [2023-10-10 16:49:17,258][123582] Updated weights for policy 0, policy_version 6411 (0.0007) [2023-10-10 16:49:17,515][123614] Updated weights for policy 1, policy_version 6400 (0.0007) [2023-10-10 16:49:17,642][123582] Updated weights for policy 0, policy_version 6421 (0.0007) [2023-10-10 16:49:18,016][123582] Updated weights for policy 0, policy_version 6431 (0.0007) [2023-10-10 16:49:18,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 13139968. Throughput: 0: 1810.5, 1: 1805.7. Samples: 3288828. Policy #0 lag: (min: 31.0, avg: 31.3, max: 41.0) [2023-10-10 16:49:18,789][122664] Avg episode reward: [(0, '17.000'), (1, '17.840')] [2023-10-10 16:49:21,363][123614] Updated weights for policy 1, policy_version 6410 (0.0010) [2023-10-10 16:49:21,733][123614] Updated weights for policy 1, policy_version 6420 (0.0008) [2023-10-10 16:49:21,753][123582] Updated weights for policy 0, policy_version 6441 (0.0008) [2023-10-10 16:49:22,098][123614] Updated weights for policy 1, policy_version 6430 (0.0007) [2023-10-10 16:49:22,131][123582] Updated weights for policy 0, policy_version 6451 (0.0007) [2023-10-10 16:49:22,512][123582] Updated weights for policy 0, policy_version 6461 (0.0007) [2023-10-10 16:49:23,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 13205504. Throughput: 0: 1800.3, 1: 1801.5. Samples: 3310506. Policy #0 lag: (min: 31.0, avg: 31.3, max: 41.0) [2023-10-10 16:49:23,789][122664] Avg episode reward: [(0, '17.240'), (1, '17.890')] [2023-10-10 16:49:23,801][123247] Saving new best policy, reward=17.240! [2023-10-10 16:49:25,956][123614] Updated weights for policy 1, policy_version 6440 (0.0007) [2023-10-10 16:49:26,236][123582] Updated weights for policy 0, policy_version 6471 (0.0008) [2023-10-10 16:49:26,329][123614] Updated weights for policy 1, policy_version 6450 (0.0007) [2023-10-10 16:49:26,614][123582] Updated weights for policy 0, policy_version 6481 (0.0008) [2023-10-10 16:49:26,704][123614] Updated weights for policy 1, policy_version 6460 (0.0008) [2023-10-10 16:49:26,978][123582] Updated weights for policy 0, policy_version 6491 (0.0009) [2023-10-10 16:49:28,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 13271040. Throughput: 0: 1811.4, 1: 1808.9. Samples: 3321310. Policy #0 lag: (min: 31.0, avg: 32.1, max: 53.0) [2023-10-10 16:49:28,789][122664] Avg episode reward: [(0, '17.700'), (1, '17.000')] [2023-10-10 16:49:28,790][123247] Saving new best policy, reward=17.700! [2023-10-10 16:49:30,341][123614] Updated weights for policy 1, policy_version 6470 (0.0010) [2023-10-10 16:49:30,706][123614] Updated weights for policy 1, policy_version 6480 (0.0008) [2023-10-10 16:49:30,814][123582] Updated weights for policy 0, policy_version 6501 (0.0008) [2023-10-10 16:49:31,085][123614] Updated weights for policy 1, policy_version 6490 (0.0007) [2023-10-10 16:49:31,187][123582] Updated weights for policy 0, policy_version 6511 (0.0007) [2023-10-10 16:49:31,566][123582] Updated weights for policy 0, policy_version 6521 (0.0010) [2023-10-10 16:49:33,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 13336576. Throughput: 0: 1794.9, 1: 1802.9. Samples: 3342824. Policy #0 lag: (min: 31.0, avg: 32.1, max: 53.0) [2023-10-10 16:49:33,789][122664] Avg episode reward: [(0, '17.410'), (1, '15.690')] [2023-10-10 16:49:34,821][123614] Updated weights for policy 1, policy_version 6500 (0.0008) [2023-10-10 16:49:35,189][123614] Updated weights for policy 1, policy_version 6510 (0.0007) [2023-10-10 16:49:35,272][123582] Updated weights for policy 0, policy_version 6531 (0.0010) [2023-10-10 16:49:35,557][123614] Updated weights for policy 1, policy_version 6520 (0.0007) [2023-10-10 16:49:35,647][123582] Updated weights for policy 0, policy_version 6541 (0.0008) [2023-10-10 16:49:36,014][123582] Updated weights for policy 0, policy_version 6551 (0.0007) [2023-10-10 16:49:38,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 13402112. Throughput: 0: 1794.2, 1: 1804.6. Samples: 3365392. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:49:38,789][122664] Avg episode reward: [(0, '17.940'), (1, '14.890')] [2023-10-10 16:49:38,796][123247] Saving new best policy, reward=17.940! [2023-10-10 16:49:39,241][123614] Updated weights for policy 1, policy_version 6530 (0.0007) [2023-10-10 16:49:39,615][123614] Updated weights for policy 1, policy_version 6540 (0.0008) [2023-10-10 16:49:39,672][123582] Updated weights for policy 0, policy_version 6561 (0.0007) [2023-10-10 16:49:39,972][123614] Updated weights for policy 1, policy_version 6550 (0.0010) [2023-10-10 16:49:40,035][123582] Updated weights for policy 0, policy_version 6571 (0.0008) [2023-10-10 16:49:40,332][123614] Updated weights for policy 1, policy_version 6560 (0.0009) [2023-10-10 16:49:40,407][123582] Updated weights for policy 0, policy_version 6581 (0.0009) [2023-10-10 16:49:40,788][123582] Updated weights for policy 0, policy_version 6591 (0.0009) [2023-10-10 16:49:43,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 13467648. Throughput: 0: 1796.9, 1: 1802.5. Samples: 3375324. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:49:43,789][122664] Avg episode reward: [(0, '17.570'), (1, '15.410')] [2023-10-10 16:49:43,895][123614] Updated weights for policy 1, policy_version 6570 (0.0009) [2023-10-10 16:49:44,263][123614] Updated weights for policy 1, policy_version 6580 (0.0007) [2023-10-10 16:49:44,494][123582] Updated weights for policy 0, policy_version 6601 (0.0008) [2023-10-10 16:49:44,627][123614] Updated weights for policy 1, policy_version 6590 (0.0007) [2023-10-10 16:49:44,866][123582] Updated weights for policy 0, policy_version 6611 (0.0009) [2023-10-10 16:49:45,235][123582] Updated weights for policy 0, policy_version 6621 (0.0008) [2023-10-10 16:49:48,310][123614] Updated weights for policy 1, policy_version 6600 (0.0007) [2023-10-10 16:49:48,679][123614] Updated weights for policy 1, policy_version 6610 (0.0009) [2023-10-10 16:49:48,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 13533184. Throughput: 0: 1801.3, 1: 1803.9. Samples: 3398270. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 16:49:48,788][122664] Avg episode reward: [(0, '17.940'), (1, '15.740')] [2023-10-10 16:49:48,863][123582] Updated weights for policy 0, policy_version 6631 (0.0007) [2023-10-10 16:49:49,035][123614] Updated weights for policy 1, policy_version 6620 (0.0007) [2023-10-10 16:49:49,220][123582] Updated weights for policy 0, policy_version 6641 (0.0010) [2023-10-10 16:49:49,591][123582] Updated weights for policy 0, policy_version 6651 (0.0009) [2023-10-10 16:49:52,864][123614] Updated weights for policy 1, policy_version 6630 (0.0008) [2023-10-10 16:49:53,223][123614] Updated weights for policy 1, policy_version 6640 (0.0009) [2023-10-10 16:49:53,373][123582] Updated weights for policy 0, policy_version 6661 (0.0009) [2023-10-10 16:49:53,591][123614] Updated weights for policy 1, policy_version 6650 (0.0007) [2023-10-10 16:49:53,753][123582] Updated weights for policy 0, policy_version 6671 (0.0007) [2023-10-10 16:49:53,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 13598720. Throughput: 0: 1812.5, 1: 1810.8. Samples: 3419244. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 16:49:53,789][122664] Avg episode reward: [(0, '17.320'), (1, '15.990')] [2023-10-10 16:49:53,804][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000006656_6815744.pth... [2023-10-10 16:49:53,833][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000004960_5079040.pth [2023-10-10 16:49:54,120][123582] Updated weights for policy 0, policy_version 6681 (0.0011) [2023-10-10 16:49:54,379][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000006688_6848512.pth... [2023-10-10 16:49:54,417][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000004992_5111808.pth [2023-10-10 16:49:57,366][123614] Updated weights for policy 1, policy_version 6660 (0.0008) [2023-10-10 16:49:57,733][123614] Updated weights for policy 1, policy_version 6670 (0.0010) [2023-10-10 16:49:57,852][123582] Updated weights for policy 0, policy_version 6691 (0.0009) [2023-10-10 16:49:58,098][123614] Updated weights for policy 1, policy_version 6680 (0.0008) [2023-10-10 16:49:58,226][123582] Updated weights for policy 0, policy_version 6701 (0.0008) [2023-10-10 16:49:58,596][123582] Updated weights for policy 0, policy_version 6711 (0.0008) [2023-10-10 16:49:58,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 13697024. Throughput: 0: 1801.3, 1: 1809.6. Samples: 3430608. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-10 16:49:58,789][122664] Avg episode reward: [(0, '17.830'), (1, '15.800')] [2023-10-10 16:50:01,703][123614] Updated weights for policy 1, policy_version 6690 (0.0007) [2023-10-10 16:50:02,072][123614] Updated weights for policy 1, policy_version 6700 (0.0007) [2023-10-10 16:50:02,269][123582] Updated weights for policy 0, policy_version 6721 (0.0007) [2023-10-10 16:50:02,461][123614] Updated weights for policy 1, policy_version 6710 (0.0009) [2023-10-10 16:50:02,646][123582] Updated weights for policy 0, policy_version 6731 (0.0007) [2023-10-10 16:50:02,814][123614] Updated weights for policy 1, policy_version 6720 (0.0009) [2023-10-10 16:50:03,021][123582] Updated weights for policy 0, policy_version 6741 (0.0009) [2023-10-10 16:50:03,411][123582] Updated weights for policy 0, policy_version 6751 (0.0009) [2023-10-10 16:50:03,788][122664] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 13795328. Throughput: 0: 1808.1, 1: 1811.9. Samples: 3451726. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 16:50:03,789][122664] Avg episode reward: [(0, '18.180'), (1, '17.700')] [2023-10-10 16:50:03,790][123247] Saving new best policy, reward=18.180! [2023-10-10 16:50:06,804][123614] Updated weights for policy 1, policy_version 6730 (0.0007) [2023-10-10 16:50:06,987][123582] Updated weights for policy 0, policy_version 6761 (0.0008) [2023-10-10 16:50:07,164][123614] Updated weights for policy 1, policy_version 6740 (0.0007) [2023-10-10 16:50:07,362][123582] Updated weights for policy 0, policy_version 6771 (0.0009) [2023-10-10 16:50:07,534][123614] Updated weights for policy 1, policy_version 6750 (0.0008) [2023-10-10 16:50:07,730][123582] Updated weights for policy 0, policy_version 6781 (0.0010) [2023-10-10 16:50:08,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 13860864. Throughput: 0: 1803.7, 1: 1802.7. Samples: 3472790. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 16:50:08,788][122664] Avg episode reward: [(0, '17.260'), (1, '16.420')] [2023-10-10 16:50:11,380][123582] Updated weights for policy 0, policy_version 6791 (0.0008) [2023-10-10 16:50:11,550][123614] Updated weights for policy 1, policy_version 6760 (0.0009) [2023-10-10 16:50:11,758][123582] Updated weights for policy 0, policy_version 6801 (0.0009) [2023-10-10 16:50:11,926][123614] Updated weights for policy 1, policy_version 6770 (0.0009) [2023-10-10 16:50:12,127][123582] Updated weights for policy 0, policy_version 6811 (0.0009) [2023-10-10 16:50:12,280][123614] Updated weights for policy 1, policy_version 6780 (0.0007) [2023-10-10 16:50:13,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 13926400. Throughput: 0: 1815.7, 1: 1813.5. Samples: 3484626. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:50:13,789][122664] Avg episode reward: [(0, '16.910'), (1, '15.840')] [2023-10-10 16:50:15,887][123614] Updated weights for policy 1, policy_version 6790 (0.0009) [2023-10-10 16:50:15,927][123582] Updated weights for policy 0, policy_version 6821 (0.0007) [2023-10-10 16:50:16,248][123614] Updated weights for policy 1, policy_version 6800 (0.0008) [2023-10-10 16:50:16,307][123582] Updated weights for policy 0, policy_version 6831 (0.0008) [2023-10-10 16:50:16,615][123614] Updated weights for policy 1, policy_version 6810 (0.0008) [2023-10-10 16:50:16,687][123582] Updated weights for policy 0, policy_version 6841 (0.0007) [2023-10-10 16:50:18,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 13991936. Throughput: 0: 1809.9, 1: 1795.3. Samples: 3505058. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:50:18,788][122664] Avg episode reward: [(0, '17.100'), (1, '14.220')] [2023-10-10 16:50:20,317][123614] Updated weights for policy 1, policy_version 6820 (0.0008) [2023-10-10 16:50:20,355][123582] Updated weights for policy 0, policy_version 6851 (0.0008) [2023-10-10 16:50:20,697][123614] Updated weights for policy 1, policy_version 6830 (0.0007) [2023-10-10 16:50:20,724][123582] Updated weights for policy 0, policy_version 6861 (0.0008) [2023-10-10 16:50:21,061][123614] Updated weights for policy 1, policy_version 6840 (0.0007) [2023-10-10 16:50:21,104][123582] Updated weights for policy 0, policy_version 6871 (0.0009) [2023-10-10 16:50:23,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 14057472. Throughput: 0: 1813.6, 1: 1792.6. Samples: 3527670. Policy #0 lag: (min: 8.0, avg: 17.7, max: 40.0) [2023-10-10 16:50:23,789][122664] Avg episode reward: [(0, '15.730'), (1, '14.690')] [2023-10-10 16:50:24,885][123582] Updated weights for policy 0, policy_version 6881 (0.0009) [2023-10-10 16:50:24,908][123614] Updated weights for policy 1, policy_version 6850 (0.0008) [2023-10-10 16:50:25,260][123582] Updated weights for policy 0, policy_version 6891 (0.0008) [2023-10-10 16:50:25,280][123614] Updated weights for policy 1, policy_version 6860 (0.0009) [2023-10-10 16:50:25,638][123614] Updated weights for policy 1, policy_version 6870 (0.0008) [2023-10-10 16:50:25,638][123582] Updated weights for policy 0, policy_version 6901 (0.0009) [2023-10-10 16:50:26,015][123582] Updated weights for policy 0, policy_version 6911 (0.0008) [2023-10-10 16:50:26,019][123614] Updated weights for policy 1, policy_version 6880 (0.0007) [2023-10-10 16:50:28,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 14123008. Throughput: 0: 1809.6, 1: 1787.6. Samples: 3537198. Policy #0 lag: (min: 8.0, avg: 17.7, max: 40.0) [2023-10-10 16:50:28,789][122664] Avg episode reward: [(0, '16.330'), (1, '13.500')] [2023-10-10 16:50:29,702][123582] Updated weights for policy 0, policy_version 6921 (0.0009) [2023-10-10 16:50:29,787][123614] Updated weights for policy 1, policy_version 6890 (0.0008) [2023-10-10 16:50:30,079][123582] Updated weights for policy 0, policy_version 6931 (0.0009) [2023-10-10 16:50:30,161][123614] Updated weights for policy 1, policy_version 6900 (0.0009) [2023-10-10 16:50:30,454][123582] Updated weights for policy 0, policy_version 6941 (0.0007) [2023-10-10 16:50:30,527][123614] Updated weights for policy 1, policy_version 6910 (0.0008) [2023-10-10 16:50:33,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 14188544. Throughput: 0: 1806.4, 1: 1788.6. Samples: 3560046. Policy #0 lag: (min: 11.0, avg: 11.2, max: 20.0) [2023-10-10 16:50:33,788][122664] Avg episode reward: [(0, '16.520'), (1, '13.640')] [2023-10-10 16:50:34,086][123614] Updated weights for policy 1, policy_version 6920 (0.0008) [2023-10-10 16:50:34,158][123582] Updated weights for policy 0, policy_version 6951 (0.0008) [2023-10-10 16:50:34,456][123614] Updated weights for policy 1, policy_version 6930 (0.0007) [2023-10-10 16:50:34,527][123582] Updated weights for policy 0, policy_version 6961 (0.0008) [2023-10-10 16:50:34,835][123614] Updated weights for policy 1, policy_version 6940 (0.0008) [2023-10-10 16:50:34,897][123582] Updated weights for policy 0, policy_version 6971 (0.0008) [2023-10-10 16:50:38,524][123614] Updated weights for policy 1, policy_version 6950 (0.0008) [2023-10-10 16:50:38,701][123582] Updated weights for policy 0, policy_version 6981 (0.0010) [2023-10-10 16:50:38,788][122664] Fps is (10 sec: 13106.6, 60 sec: 14199.3, 300 sec: 14440.1). Total num frames: 14254080. Throughput: 0: 1807.4, 1: 1805.2. Samples: 3581812. Policy #0 lag: (min: 11.0, avg: 11.2, max: 20.0) [2023-10-10 16:50:38,790][122664] Avg episode reward: [(0, '17.270'), (1, '14.860')] [2023-10-10 16:50:38,890][123614] Updated weights for policy 1, policy_version 6960 (0.0007) [2023-10-10 16:50:39,073][123582] Updated weights for policy 0, policy_version 6991 (0.0007) [2023-10-10 16:50:39,256][123614] Updated weights for policy 1, policy_version 6970 (0.0008) [2023-10-10 16:50:39,444][123582] Updated weights for policy 0, policy_version 7001 (0.0008) [2023-10-10 16:50:43,000][123614] Updated weights for policy 1, policy_version 6980 (0.0008) [2023-10-10 16:50:43,097][123582] Updated weights for policy 0, policy_version 7011 (0.0008) [2023-10-10 16:50:43,364][123614] Updated weights for policy 1, policy_version 6990 (0.0008) [2023-10-10 16:50:43,469][123582] Updated weights for policy 0, policy_version 7021 (0.0009) [2023-10-10 16:50:43,722][123614] Updated weights for policy 1, policy_version 7000 (0.0009) [2023-10-10 16:50:43,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 14319616. Throughput: 0: 1802.9, 1: 1786.7. Samples: 3592140. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:50:43,788][122664] Avg episode reward: [(0, '16.610'), (1, '16.260')] [2023-10-10 16:50:43,839][123582] Updated weights for policy 0, policy_version 7031 (0.0007) [2023-10-10 16:50:47,363][123614] Updated weights for policy 1, policy_version 7010 (0.0009) [2023-10-10 16:50:47,573][123582] Updated weights for policy 0, policy_version 7041 (0.0007) [2023-10-10 16:50:47,730][123614] Updated weights for policy 1, policy_version 7020 (0.0008) [2023-10-10 16:50:47,946][123582] Updated weights for policy 0, policy_version 7051 (0.0008) [2023-10-10 16:50:48,107][123614] Updated weights for policy 1, policy_version 7030 (0.0010) [2023-10-10 16:50:48,319][123582] Updated weights for policy 0, policy_version 7061 (0.0008) [2023-10-10 16:50:48,472][123614] Updated weights for policy 1, policy_version 7040 (0.0007) [2023-10-10 16:50:48,682][123582] Updated weights for policy 0, policy_version 7071 (0.0007) [2023-10-10 16:50:48,788][122664] Fps is (10 sec: 19661.5, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 14450688. Throughput: 0: 1807.5, 1: 1804.9. Samples: 3614284. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:50:48,789][122664] Avg episode reward: [(0, '17.180'), (1, '15.630')] [2023-10-10 16:50:52,183][123614] Updated weights for policy 1, policy_version 7050 (0.0007) [2023-10-10 16:50:52,503][123582] Updated weights for policy 0, policy_version 7081 (0.0007) [2023-10-10 16:50:52,548][123614] Updated weights for policy 1, policy_version 7060 (0.0007) [2023-10-10 16:50:52,865][123582] Updated weights for policy 0, policy_version 7091 (0.0007) [2023-10-10 16:50:52,907][123614] Updated weights for policy 1, policy_version 7070 (0.0008) [2023-10-10 16:50:53,245][123582] Updated weights for policy 0, policy_version 7101 (0.0009) [2023-10-10 16:50:53,788][122664] Fps is (10 sec: 19660.2, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 14516224. Throughput: 0: 1797.2, 1: 1789.9. Samples: 3634210. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:50:53,789][122664] Avg episode reward: [(0, '16.560'), (1, '14.450')] [2023-10-10 16:50:56,736][123614] Updated weights for policy 1, policy_version 7080 (0.0009) [2023-10-10 16:50:56,889][123582] Updated weights for policy 0, policy_version 7111 (0.0008) [2023-10-10 16:50:57,116][123614] Updated weights for policy 1, policy_version 7090 (0.0009) [2023-10-10 16:50:57,254][123582] Updated weights for policy 0, policy_version 7121 (0.0007) [2023-10-10 16:50:57,480][123614] Updated weights for policy 1, policy_version 7100 (0.0008) [2023-10-10 16:50:57,624][123582] Updated weights for policy 0, policy_version 7131 (0.0007) [2023-10-10 16:50:58,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 14581760. Throughput: 0: 1804.4, 1: 1801.4. Samples: 3646886. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:50:58,789][122664] Avg episode reward: [(0, '15.610'), (1, '15.550')] [2023-10-10 16:51:01,313][123614] Updated weights for policy 1, policy_version 7110 (0.0009) [2023-10-10 16:51:01,478][123582] Updated weights for policy 0, policy_version 7141 (0.0008) [2023-10-10 16:51:01,682][123614] Updated weights for policy 1, policy_version 7120 (0.0007) [2023-10-10 16:51:01,841][123582] Updated weights for policy 0, policy_version 7151 (0.0008) [2023-10-10 16:51:02,047][123614] Updated weights for policy 1, policy_version 7130 (0.0007) [2023-10-10 16:51:02,216][123582] Updated weights for policy 0, policy_version 7161 (0.0008) [2023-10-10 16:51:03,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 14647296. Throughput: 0: 1800.0, 1: 1793.7. Samples: 3666776. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:51:03,789][122664] Avg episode reward: [(0, '15.160'), (1, '16.910')] [2023-10-10 16:51:05,874][123614] Updated weights for policy 1, policy_version 7140 (0.0007) [2023-10-10 16:51:05,943][123582] Updated weights for policy 0, policy_version 7171 (0.0008) [2023-10-10 16:51:06,235][123614] Updated weights for policy 1, policy_version 7150 (0.0008) [2023-10-10 16:51:06,313][123582] Updated weights for policy 0, policy_version 7181 (0.0007) [2023-10-10 16:51:06,595][123614] Updated weights for policy 1, policy_version 7160 (0.0008) [2023-10-10 16:51:06,684][123582] Updated weights for policy 0, policy_version 7191 (0.0009) [2023-10-10 16:51:08,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 14712832. Throughput: 0: 1793.5, 1: 1792.5. Samples: 3689044. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 16:51:08,789][122664] Avg episode reward: [(0, '16.970'), (1, '17.330')] [2023-10-10 16:51:10,409][123614] Updated weights for policy 1, policy_version 7170 (0.0008) [2023-10-10 16:51:10,426][123582] Updated weights for policy 0, policy_version 7201 (0.0009) [2023-10-10 16:51:10,775][123614] Updated weights for policy 1, policy_version 7180 (0.0007) [2023-10-10 16:51:10,791][123582] Updated weights for policy 0, policy_version 7211 (0.0007) [2023-10-10 16:51:11,150][123614] Updated weights for policy 1, policy_version 7190 (0.0007) [2023-10-10 16:51:11,161][123582] Updated weights for policy 0, policy_version 7221 (0.0008) [2023-10-10 16:51:11,523][123614] Updated weights for policy 1, policy_version 7200 (0.0008) [2023-10-10 16:51:11,525][123582] Updated weights for policy 0, policy_version 7231 (0.0008) [2023-10-10 16:51:13,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 14778368. Throughput: 0: 1803.5, 1: 1797.0. Samples: 3699218. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 16:51:13,789][122664] Avg episode reward: [(0, '18.700'), (1, '17.800')] [2023-10-10 16:51:13,790][123247] Saving new best policy, reward=18.700! [2023-10-10 16:51:15,253][123614] Updated weights for policy 1, policy_version 7210 (0.0008) [2023-10-10 16:51:15,298][123582] Updated weights for policy 0, policy_version 7241 (0.0007) [2023-10-10 16:51:15,621][123614] Updated weights for policy 1, policy_version 7220 (0.0007) [2023-10-10 16:51:15,671][123582] Updated weights for policy 0, policy_version 7251 (0.0007) [2023-10-10 16:51:15,989][123614] Updated weights for policy 1, policy_version 7230 (0.0007) [2023-10-10 16:51:16,044][123582] Updated weights for policy 0, policy_version 7261 (0.0007) [2023-10-10 16:51:18,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 14843904. Throughput: 0: 1800.2, 1: 1793.6. Samples: 3721770. Policy #0 lag: (min: 9.0, avg: 17.2, max: 41.0) [2023-10-10 16:51:18,789][122664] Avg episode reward: [(0, '18.340'), (1, '20.260')] [2023-10-10 16:51:18,790][123465] Saving new best policy, reward=20.260! [2023-10-10 16:51:19,636][123582] Updated weights for policy 0, policy_version 7271 (0.0009) [2023-10-10 16:51:19,833][123614] Updated weights for policy 1, policy_version 7240 (0.0008) [2023-10-10 16:51:20,014][123582] Updated weights for policy 0, policy_version 7281 (0.0009) [2023-10-10 16:51:20,196][123614] Updated weights for policy 1, policy_version 7250 (0.0008) [2023-10-10 16:51:20,384][123582] Updated weights for policy 0, policy_version 7291 (0.0008) [2023-10-10 16:51:20,566][123614] Updated weights for policy 1, policy_version 7260 (0.0009) [2023-10-10 16:51:23,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 14909440. Throughput: 0: 1805.4, 1: 1809.5. Samples: 3744480. Policy #0 lag: (min: 9.0, avg: 17.2, max: 41.0) [2023-10-10 16:51:23,789][122664] Avg episode reward: [(0, '18.220'), (1, '20.380')] [2023-10-10 16:51:23,801][123465] Saving new best policy, reward=20.380! [2023-10-10 16:51:24,181][123582] Updated weights for policy 0, policy_version 7301 (0.0008) [2023-10-10 16:51:24,191][123614] Updated weights for policy 1, policy_version 7270 (0.0007) [2023-10-10 16:51:24,552][123582] Updated weights for policy 0, policy_version 7311 (0.0008) [2023-10-10 16:51:24,559][123614] Updated weights for policy 1, policy_version 7280 (0.0007) [2023-10-10 16:51:24,923][123582] Updated weights for policy 0, policy_version 7321 (0.0009) [2023-10-10 16:51:24,931][123614] Updated weights for policy 1, policy_version 7290 (0.0009) [2023-10-10 16:51:28,628][123582] Updated weights for policy 0, policy_version 7331 (0.0007) [2023-10-10 16:51:28,734][123614] Updated weights for policy 1, policy_version 7300 (0.0008) [2023-10-10 16:51:28,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 14974976. Throughput: 0: 1801.7, 1: 1800.4. Samples: 3754236. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:51:28,789][122664] Avg episode reward: [(0, '16.700'), (1, '19.540')] [2023-10-10 16:51:28,998][123582] Updated weights for policy 0, policy_version 7341 (0.0008) [2023-10-10 16:51:29,095][123614] Updated weights for policy 1, policy_version 7310 (0.0008) [2023-10-10 16:51:29,369][123582] Updated weights for policy 0, policy_version 7351 (0.0007) [2023-10-10 16:51:29,453][123614] Updated weights for policy 1, policy_version 7320 (0.0010) [2023-10-10 16:51:33,007][123582] Updated weights for policy 0, policy_version 7361 (0.0009) [2023-10-10 16:51:33,151][123614] Updated weights for policy 1, policy_version 7330 (0.0009) [2023-10-10 16:51:33,382][123582] Updated weights for policy 0, policy_version 7371 (0.0009) [2023-10-10 16:51:33,516][123614] Updated weights for policy 1, policy_version 7340 (0.0007) [2023-10-10 16:51:33,740][123582] Updated weights for policy 0, policy_version 7381 (0.0008) [2023-10-10 16:51:33,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 15040512. Throughput: 0: 1808.8, 1: 1809.4. Samples: 3777102. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:51:33,788][122664] Avg episode reward: [(0, '15.190'), (1, '18.850')] [2023-10-10 16:51:33,883][123614] Updated weights for policy 1, policy_version 7350 (0.0007) [2023-10-10 16:51:34,115][123582] Updated weights for policy 0, policy_version 7391 (0.0010) [2023-10-10 16:51:34,251][123614] Updated weights for policy 1, policy_version 7360 (0.0007) [2023-10-10 16:51:37,876][123582] Updated weights for policy 0, policy_version 7401 (0.0009) [2023-10-10 16:51:38,045][123614] Updated weights for policy 1, policy_version 7370 (0.0009) [2023-10-10 16:51:38,240][123582] Updated weights for policy 0, policy_version 7411 (0.0008) [2023-10-10 16:51:38,413][123614] Updated weights for policy 1, policy_version 7380 (0.0008) [2023-10-10 16:51:38,620][123582] Updated weights for policy 0, policy_version 7421 (0.0009) [2023-10-10 16:51:38,781][123614] Updated weights for policy 1, policy_version 7390 (0.0009) [2023-10-10 16:51:38,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 15138816. Throughput: 0: 1818.3, 1: 1800.8. Samples: 3797070. Policy #0 lag: (min: 28.0, avg: 36.0, max: 60.0) [2023-10-10 16:51:38,790][122664] Avg episode reward: [(0, '14.940'), (1, '18.180')] [2023-10-10 16:51:42,377][123582] Updated weights for policy 0, policy_version 7431 (0.0008) [2023-10-10 16:51:42,498][123614] Updated weights for policy 1, policy_version 7400 (0.0007) [2023-10-10 16:51:42,750][123582] Updated weights for policy 0, policy_version 7441 (0.0007) [2023-10-10 16:51:42,868][123614] Updated weights for policy 1, policy_version 7410 (0.0007) [2023-10-10 16:51:43,126][123582] Updated weights for policy 0, policy_version 7451 (0.0007) [2023-10-10 16:51:43,244][123614] Updated weights for policy 1, policy_version 7420 (0.0007) [2023-10-10 16:51:43,788][122664] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 15237120. Throughput: 0: 1805.7, 1: 1804.5. Samples: 3809344. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:51:43,788][122664] Avg episode reward: [(0, '15.600'), (1, '18.360')] [2023-10-10 16:51:46,780][123614] Updated weights for policy 1, policy_version 7430 (0.0007) [2023-10-10 16:51:47,012][123582] Updated weights for policy 0, policy_version 7461 (0.0008) [2023-10-10 16:51:47,151][123614] Updated weights for policy 1, policy_version 7440 (0.0009) [2023-10-10 16:51:47,379][123582] Updated weights for policy 0, policy_version 7471 (0.0008) [2023-10-10 16:51:47,514][123614] Updated weights for policy 1, policy_version 7450 (0.0007) [2023-10-10 16:51:47,751][123582] Updated weights for policy 0, policy_version 7481 (0.0008) [2023-10-10 16:51:48,788][122664] Fps is (10 sec: 16384.7, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 15302656. Throughput: 0: 1816.7, 1: 1805.3. Samples: 3829768. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:51:48,788][122664] Avg episode reward: [(0, '15.160'), (1, '17.760')] [2023-10-10 16:51:51,354][123614] Updated weights for policy 1, policy_version 7460 (0.0008) [2023-10-10 16:51:51,518][123582] Updated weights for policy 0, policy_version 7491 (0.0008) [2023-10-10 16:51:51,715][123614] Updated weights for policy 1, policy_version 7470 (0.0007) [2023-10-10 16:51:51,890][123582] Updated weights for policy 0, policy_version 7501 (0.0007) [2023-10-10 16:51:52,094][123614] Updated weights for policy 1, policy_version 7480 (0.0009) [2023-10-10 16:51:52,264][123582] Updated weights for policy 0, policy_version 7511 (0.0008) [2023-10-10 16:51:53,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 15368192. Throughput: 0: 1802.5, 1: 1805.8. Samples: 3851416. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:51:53,789][122664] Avg episode reward: [(0, '16.210'), (1, '17.310')] [2023-10-10 16:51:53,799][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000007488_7667712.pth... [2023-10-10 16:51:53,800][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000007520_7700480.pth... [2023-10-10 16:51:53,836][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000005792_5931008.pth [2023-10-10 16:51:53,843][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000005824_5963776.pth [2023-10-10 16:51:55,817][123614] Updated weights for policy 1, policy_version 7490 (0.0008) [2023-10-10 16:51:55,880][123582] Updated weights for policy 0, policy_version 7521 (0.0007) [2023-10-10 16:51:56,186][123614] Updated weights for policy 1, policy_version 7500 (0.0007) [2023-10-10 16:51:56,246][123582] Updated weights for policy 0, policy_version 7531 (0.0007) [2023-10-10 16:51:56,561][123614] Updated weights for policy 1, policy_version 7510 (0.0007) [2023-10-10 16:51:56,619][123582] Updated weights for policy 0, policy_version 7541 (0.0007) [2023-10-10 16:51:56,930][123614] Updated weights for policy 1, policy_version 7520 (0.0007) [2023-10-10 16:51:56,987][123582] Updated weights for policy 0, policy_version 7551 (0.0008) [2023-10-10 16:51:58,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 15433728. Throughput: 0: 1814.0, 1: 1812.4. Samples: 3862406. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:51:58,789][122664] Avg episode reward: [(0, '18.780'), (1, '16.240')] [2023-10-10 16:51:58,790][123247] Saving new best policy, reward=18.780! [2023-10-10 16:52:00,675][123582] Updated weights for policy 0, policy_version 7561 (0.0007) [2023-10-10 16:52:00,690][123614] Updated weights for policy 1, policy_version 7530 (0.0008) [2023-10-10 16:52:01,051][123614] Updated weights for policy 1, policy_version 7540 (0.0008) [2023-10-10 16:52:01,054][123582] Updated weights for policy 0, policy_version 7571 (0.0010) [2023-10-10 16:52:01,423][123614] Updated weights for policy 1, policy_version 7550 (0.0007) [2023-10-10 16:52:01,428][123582] Updated weights for policy 0, policy_version 7581 (0.0008) [2023-10-10 16:52:03,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 15499264. Throughput: 0: 1794.5, 1: 1803.9. Samples: 3883698. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:52:03,789][122664] Avg episode reward: [(0, '20.000'), (1, '17.560')] [2023-10-10 16:52:03,791][123247] Saving new best policy, reward=20.000! [2023-10-10 16:52:05,088][123582] Updated weights for policy 0, policy_version 7591 (0.0008) [2023-10-10 16:52:05,217][123614] Updated weights for policy 1, policy_version 7560 (0.0007) [2023-10-10 16:52:05,457][123582] Updated weights for policy 0, policy_version 7601 (0.0008) [2023-10-10 16:52:05,590][123614] Updated weights for policy 1, policy_version 7570 (0.0007) [2023-10-10 16:52:05,819][123582] Updated weights for policy 0, policy_version 7611 (0.0008) [2023-10-10 16:52:05,953][123614] Updated weights for policy 1, policy_version 7580 (0.0007) [2023-10-10 16:52:08,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 15564800. Throughput: 0: 1796.2, 1: 1802.4. Samples: 3906414. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:52:08,789][122664] Avg episode reward: [(0, '18.150'), (1, '18.280')] [2023-10-10 16:52:09,357][123582] Updated weights for policy 0, policy_version 7621 (0.0007) [2023-10-10 16:52:09,631][123614] Updated weights for policy 1, policy_version 7590 (0.0009) [2023-10-10 16:52:09,739][123582] Updated weights for policy 0, policy_version 7631 (0.0008) [2023-10-10 16:52:10,003][123614] Updated weights for policy 1, policy_version 7600 (0.0009) [2023-10-10 16:52:10,108][123582] Updated weights for policy 0, policy_version 7641 (0.0007) [2023-10-10 16:52:10,362][123614] Updated weights for policy 1, policy_version 7610 (0.0007) [2023-10-10 16:52:13,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 15630336. Throughput: 0: 1804.0, 1: 1803.6. Samples: 3916574. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:52:13,789][122664] Avg episode reward: [(0, '18.100'), (1, '16.310')] [2023-10-10 16:52:13,817][123582] Updated weights for policy 0, policy_version 7651 (0.0008) [2023-10-10 16:52:13,983][123614] Updated weights for policy 1, policy_version 7620 (0.0008) [2023-10-10 16:52:14,186][123582] Updated weights for policy 0, policy_version 7661 (0.0007) [2023-10-10 16:52:14,347][123614] Updated weights for policy 1, policy_version 7630 (0.0008) [2023-10-10 16:52:14,553][123582] Updated weights for policy 0, policy_version 7671 (0.0008) [2023-10-10 16:52:14,710][123614] Updated weights for policy 1, policy_version 7640 (0.0008) [2023-10-10 16:52:18,176][123582] Updated weights for policy 0, policy_version 7681 (0.0007) [2023-10-10 16:52:18,549][123582] Updated weights for policy 0, policy_version 7691 (0.0009) [2023-10-10 16:52:18,580][123614] Updated weights for policy 1, policy_version 7650 (0.0009) [2023-10-10 16:52:18,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 15695872. Throughput: 0: 1800.9, 1: 1801.0. Samples: 3939188. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:52:18,788][122664] Avg episode reward: [(0, '18.630'), (1, '16.720')] [2023-10-10 16:52:18,908][123582] Updated weights for policy 0, policy_version 7701 (0.0009) [2023-10-10 16:52:18,948][123614] Updated weights for policy 1, policy_version 7660 (0.0007) [2023-10-10 16:52:19,283][123582] Updated weights for policy 0, policy_version 7711 (0.0008) [2023-10-10 16:52:19,311][123614] Updated weights for policy 1, policy_version 7670 (0.0007) [2023-10-10 16:52:19,676][123614] Updated weights for policy 1, policy_version 7680 (0.0009) [2023-10-10 16:52:23,210][123582] Updated weights for policy 0, policy_version 7721 (0.0008) [2023-10-10 16:52:23,431][123614] Updated weights for policy 1, policy_version 7690 (0.0009) [2023-10-10 16:52:23,584][123582] Updated weights for policy 0, policy_version 7731 (0.0007) [2023-10-10 16:52:23,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 15761408. Throughput: 0: 1808.5, 1: 1813.6. Samples: 3960060. Policy #0 lag: (min: 14.0, avg: 20.9, max: 46.0) [2023-10-10 16:52:23,788][122664] Avg episode reward: [(0, '16.430'), (1, '17.450')] [2023-10-10 16:52:23,805][123614] Updated weights for policy 1, policy_version 7700 (0.0009) [2023-10-10 16:52:23,958][123582] Updated weights for policy 0, policy_version 7741 (0.0008) [2023-10-10 16:52:24,168][123614] Updated weights for policy 1, policy_version 7710 (0.0007) [2023-10-10 16:52:27,619][123582] Updated weights for policy 0, policy_version 7751 (0.0009) [2023-10-10 16:52:27,850][123614] Updated weights for policy 1, policy_version 7720 (0.0008) [2023-10-10 16:52:27,990][123582] Updated weights for policy 0, policy_version 7761 (0.0009) [2023-10-10 16:52:28,231][123614] Updated weights for policy 1, policy_version 7730 (0.0008) [2023-10-10 16:52:28,359][123582] Updated weights for policy 0, policy_version 7771 (0.0008) [2023-10-10 16:52:28,601][123614] Updated weights for policy 1, policy_version 7740 (0.0009) [2023-10-10 16:52:28,788][122664] Fps is (10 sec: 19660.7, 60 sec: 15291.8, 300 sec: 14440.1). Total num frames: 15892480. Throughput: 0: 1801.4, 1: 1805.2. Samples: 3971644. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-10 16:52:28,788][122664] Avg episode reward: [(0, '14.980'), (1, '16.970')] [2023-10-10 16:52:32,143][123582] Updated weights for policy 0, policy_version 7781 (0.0007) [2023-10-10 16:52:32,293][123614] Updated weights for policy 1, policy_version 7750 (0.0010) [2023-10-10 16:52:32,520][123582] Updated weights for policy 0, policy_version 7791 (0.0007) [2023-10-10 16:52:32,669][123614] Updated weights for policy 1, policy_version 7760 (0.0008) [2023-10-10 16:52:32,891][123582] Updated weights for policy 0, policy_version 7801 (0.0007) [2023-10-10 16:52:33,028][123614] Updated weights for policy 1, policy_version 7770 (0.0008) [2023-10-10 16:52:33,788][122664] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 15958016. Throughput: 0: 1809.0, 1: 1811.2. Samples: 3992678. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-10 16:52:33,789][122664] Avg episode reward: [(0, '17.740'), (1, '17.360')] [2023-10-10 16:52:36,513][123582] Updated weights for policy 0, policy_version 7811 (0.0008) [2023-10-10 16:52:36,768][123614] Updated weights for policy 1, policy_version 7780 (0.0009) [2023-10-10 16:52:36,889][123582] Updated weights for policy 0, policy_version 7821 (0.0008) [2023-10-10 16:52:37,138][123614] Updated weights for policy 1, policy_version 7790 (0.0007) [2023-10-10 16:52:37,335][123582] Updated weights for policy 0, policy_version 7833 (0.0008) [2023-10-10 16:52:37,502][123614] Updated weights for policy 1, policy_version 7800 (0.0007) [2023-10-10 16:52:38,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 16023552. Throughput: 0: 1805.7, 1: 1800.5. Samples: 4013692. Policy #0 lag: (min: 23.0, avg: 28.4, max: 55.0) [2023-10-10 16:52:38,788][122664] Avg episode reward: [(0, '17.220'), (1, '16.970')] [2023-10-10 16:52:40,943][123582] Updated weights for policy 0, policy_version 7843 (0.0009) [2023-10-10 16:52:41,315][123582] Updated weights for policy 0, policy_version 7853 (0.0007) [2023-10-10 16:52:41,326][123614] Updated weights for policy 1, policy_version 7810 (0.0008) [2023-10-10 16:52:41,687][123582] Updated weights for policy 0, policy_version 7863 (0.0007) [2023-10-10 16:52:41,696][123614] Updated weights for policy 1, policy_version 7820 (0.0008) [2023-10-10 16:52:42,059][123614] Updated weights for policy 1, policy_version 7830 (0.0009) [2023-10-10 16:52:42,438][123614] Updated weights for policy 1, policy_version 7840 (0.0010) [2023-10-10 16:52:43,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 16089088. Throughput: 0: 1803.1, 1: 1814.0. Samples: 4025174. Policy #0 lag: (min: 23.0, avg: 28.4, max: 55.0) [2023-10-10 16:52:43,789][122664] Avg episode reward: [(0, '16.380'), (1, '19.260')] [2023-10-10 16:52:45,501][123582] Updated weights for policy 0, policy_version 7873 (0.0008) [2023-10-10 16:52:45,880][123582] Updated weights for policy 0, policy_version 7883 (0.0007) [2023-10-10 16:52:46,091][123614] Updated weights for policy 1, policy_version 7850 (0.0008) [2023-10-10 16:52:46,252][123582] Updated weights for policy 0, policy_version 7893 (0.0010) [2023-10-10 16:52:46,464][123614] Updated weights for policy 1, policy_version 7860 (0.0008) [2023-10-10 16:52:46,628][123582] Updated weights for policy 0, policy_version 7903 (0.0008) [2023-10-10 16:52:46,839][123614] Updated weights for policy 1, policy_version 7870 (0.0007) [2023-10-10 16:52:48,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 16154624. Throughput: 0: 1804.7, 1: 1801.7. Samples: 4045986. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:52:48,788][122664] Avg episode reward: [(0, '16.900'), (1, '17.320')] [2023-10-10 16:52:50,375][123582] Updated weights for policy 0, policy_version 7913 (0.0010) [2023-10-10 16:52:50,573][123614] Updated weights for policy 1, policy_version 7880 (0.0008) [2023-10-10 16:52:50,743][123582] Updated weights for policy 0, policy_version 7923 (0.0008) [2023-10-10 16:52:50,942][123614] Updated weights for policy 1, policy_version 7890 (0.0007) [2023-10-10 16:52:51,110][123582] Updated weights for policy 0, policy_version 7933 (0.0007) [2023-10-10 16:52:51,315][123614] Updated weights for policy 1, policy_version 7900 (0.0009) [2023-10-10 16:52:53,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 16220160. Throughput: 0: 1809.6, 1: 1799.6. Samples: 4068826. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:52:53,789][122664] Avg episode reward: [(0, '16.890'), (1, '18.690')] [2023-10-10 16:52:54,927][123582] Updated weights for policy 0, policy_version 7943 (0.0009) [2023-10-10 16:52:54,994][123614] Updated weights for policy 1, policy_version 7910 (0.0008) [2023-10-10 16:52:55,296][123582] Updated weights for policy 0, policy_version 7953 (0.0007) [2023-10-10 16:52:55,363][123614] Updated weights for policy 1, policy_version 7920 (0.0008) [2023-10-10 16:52:55,666][123582] Updated weights for policy 0, policy_version 7963 (0.0007) [2023-10-10 16:52:55,741][123614] Updated weights for policy 1, policy_version 7930 (0.0008) [2023-10-10 16:52:58,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 16285696. Throughput: 0: 1804.4, 1: 1793.3. Samples: 4078468. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:52:58,788][122664] Avg episode reward: [(0, '18.440'), (1, '17.460')] [2023-10-10 16:52:59,322][123582] Updated weights for policy 0, policy_version 7973 (0.0010) [2023-10-10 16:52:59,463][123614] Updated weights for policy 1, policy_version 7940 (0.0009) [2023-10-10 16:52:59,707][123582] Updated weights for policy 0, policy_version 7983 (0.0007) [2023-10-10 16:52:59,832][123614] Updated weights for policy 1, policy_version 7950 (0.0007) [2023-10-10 16:53:00,077][123582] Updated weights for policy 0, policy_version 7993 (0.0008) [2023-10-10 16:53:00,192][123614] Updated weights for policy 1, policy_version 7960 (0.0008) [2023-10-10 16:53:03,694][123582] Updated weights for policy 0, policy_version 8003 (0.0009) [2023-10-10 16:53:03,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.6, 300 sec: 14440.1). Total num frames: 16351232. Throughput: 0: 1800.8, 1: 1796.7. Samples: 4101076. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:53:03,788][122664] Avg episode reward: [(0, '17.650'), (1, '18.780')] [2023-10-10 16:53:04,058][123614] Updated weights for policy 1, policy_version 7970 (0.0009) [2023-10-10 16:53:04,070][123582] Updated weights for policy 0, policy_version 8013 (0.0009) [2023-10-10 16:53:04,427][123614] Updated weights for policy 1, policy_version 7980 (0.0008) [2023-10-10 16:53:04,441][123582] Updated weights for policy 0, policy_version 8023 (0.0008) [2023-10-10 16:53:04,800][123614] Updated weights for policy 1, policy_version 7990 (0.0008) [2023-10-10 16:53:05,177][123614] Updated weights for policy 1, policy_version 8000 (0.0007) [2023-10-10 16:53:08,301][123582] Updated weights for policy 0, policy_version 8033 (0.0008) [2023-10-10 16:53:08,709][123582] Updated weights for policy 0, policy_version 8043 (0.0007) [2023-10-10 16:53:08,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 16416768. Throughput: 0: 1812.7, 1: 1808.0. Samples: 4122992. Policy #0 lag: (min: 27.0, avg: 31.5, max: 59.0) [2023-10-10 16:53:08,789][122664] Avg episode reward: [(0, '17.970'), (1, '19.210')] [2023-10-10 16:53:08,852][123614] Updated weights for policy 1, policy_version 8010 (0.0007) [2023-10-10 16:53:09,077][123582] Updated weights for policy 0, policy_version 8053 (0.0007) [2023-10-10 16:53:09,221][123614] Updated weights for policy 1, policy_version 8020 (0.0007) [2023-10-10 16:53:09,444][123582] Updated weights for policy 0, policy_version 8063 (0.0007) [2023-10-10 16:53:09,586][123614] Updated weights for policy 1, policy_version 8030 (0.0007) [2023-10-10 16:53:13,222][123582] Updated weights for policy 0, policy_version 8073 (0.0008) [2023-10-10 16:53:13,393][123614] Updated weights for policy 1, policy_version 8040 (0.0008) [2023-10-10 16:53:13,603][123582] Updated weights for policy 0, policy_version 8083 (0.0009) [2023-10-10 16:53:13,754][123614] Updated weights for policy 1, policy_version 8050 (0.0007) [2023-10-10 16:53:13,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 16482304. Throughput: 0: 1794.3, 1: 1793.6. Samples: 4133098. Policy #0 lag: (min: 27.0, avg: 31.5, max: 59.0) [2023-10-10 16:53:13,788][122664] Avg episode reward: [(0, '18.090'), (1, '17.980')] [2023-10-10 16:53:13,984][123582] Updated weights for policy 0, policy_version 8093 (0.0008) [2023-10-10 16:53:14,120][123614] Updated weights for policy 1, policy_version 8060 (0.0009) [2023-10-10 16:53:17,790][123582] Updated weights for policy 0, policy_version 8103 (0.0009) [2023-10-10 16:53:17,846][123614] Updated weights for policy 1, policy_version 8070 (0.0009) [2023-10-10 16:53:18,166][123582] Updated weights for policy 0, policy_version 8113 (0.0007) [2023-10-10 16:53:18,211][123614] Updated weights for policy 1, policy_version 8080 (0.0009) [2023-10-10 16:53:18,535][123582] Updated weights for policy 0, policy_version 8123 (0.0008) [2023-10-10 16:53:18,577][123614] Updated weights for policy 1, policy_version 8090 (0.0008) [2023-10-10 16:53:18,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.5, 300 sec: 14329.0). Total num frames: 16580608. Throughput: 0: 1804.7, 1: 1805.5. Samples: 4155136. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:53:18,789][122664] Avg episode reward: [(0, '19.600'), (1, '17.670')] [2023-10-10 16:53:22,265][123582] Updated weights for policy 0, policy_version 8133 (0.0008) [2023-10-10 16:53:22,395][123614] Updated weights for policy 1, policy_version 8100 (0.0008) [2023-10-10 16:53:22,639][123582] Updated weights for policy 0, policy_version 8143 (0.0008) [2023-10-10 16:53:22,751][123614] Updated weights for policy 1, policy_version 8110 (0.0008) [2023-10-10 16:53:23,012][123582] Updated weights for policy 0, policy_version 8153 (0.0007) [2023-10-10 16:53:23,123][123614] Updated weights for policy 1, policy_version 8120 (0.0009) [2023-10-10 16:53:23,788][122664] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 16678912. Throughput: 0: 1785.9, 1: 1785.2. Samples: 4174392. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:53:23,789][122664] Avg episode reward: [(0, '18.250'), (1, '16.670')] [2023-10-10 16:53:26,674][123582] Updated weights for policy 0, policy_version 8163 (0.0008) [2023-10-10 16:53:26,896][123614] Updated weights for policy 1, policy_version 8130 (0.0008) [2023-10-10 16:53:27,039][123582] Updated weights for policy 0, policy_version 8173 (0.0008) [2023-10-10 16:53:27,269][123614] Updated weights for policy 1, policy_version 8140 (0.0007) [2023-10-10 16:53:27,408][123582] Updated weights for policy 0, policy_version 8183 (0.0008) [2023-10-10 16:53:27,634][123614] Updated weights for policy 1, policy_version 8150 (0.0008) [2023-10-10 16:53:27,998][123614] Updated weights for policy 1, policy_version 8160 (0.0009) [2023-10-10 16:53:28,788][122664] Fps is (10 sec: 16384.4, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 16744448. Throughput: 0: 1807.9, 1: 1798.3. Samples: 4187450. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:53:28,788][122664] Avg episode reward: [(0, '17.360'), (1, '16.460')] [2023-10-10 16:53:31,172][123582] Updated weights for policy 0, policy_version 8193 (0.0008) [2023-10-10 16:53:31,544][123582] Updated weights for policy 0, policy_version 8203 (0.0010) [2023-10-10 16:53:31,814][123614] Updated weights for policy 1, policy_version 8170 (0.0008) [2023-10-10 16:53:31,916][123582] Updated weights for policy 0, policy_version 8213 (0.0007) [2023-10-10 16:53:32,185][123614] Updated weights for policy 1, policy_version 8180 (0.0008) [2023-10-10 16:53:32,293][123582] Updated weights for policy 0, policy_version 8223 (0.0008) [2023-10-10 16:53:32,549][123614] Updated weights for policy 1, policy_version 8190 (0.0008) [2023-10-10 16:53:33,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 16809984. Throughput: 0: 1792.3, 1: 1782.3. Samples: 4206846. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:53:33,789][122664] Avg episode reward: [(0, '15.630'), (1, '14.630')] [2023-10-10 16:53:36,048][123582] Updated weights for policy 0, policy_version 8233 (0.0010) [2023-10-10 16:53:36,279][123614] Updated weights for policy 1, policy_version 8200 (0.0007) [2023-10-10 16:53:36,421][123582] Updated weights for policy 0, policy_version 8243 (0.0009) [2023-10-10 16:53:36,643][123614] Updated weights for policy 1, policy_version 8210 (0.0007) [2023-10-10 16:53:36,784][123582] Updated weights for policy 0, policy_version 8253 (0.0008) [2023-10-10 16:53:37,005][123614] Updated weights for policy 1, policy_version 8220 (0.0007) [2023-10-10 16:53:38,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 16875520. Throughput: 0: 1787.2, 1: 1786.2. Samples: 4229626. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:53:38,789][122664] Avg episode reward: [(0, '16.590'), (1, '15.230')] [2023-10-10 16:53:40,598][123582] Updated weights for policy 0, policy_version 8263 (0.0009) [2023-10-10 16:53:40,701][123614] Updated weights for policy 1, policy_version 8230 (0.0008) [2023-10-10 16:53:40,979][123582] Updated weights for policy 0, policy_version 8273 (0.0008) [2023-10-10 16:53:41,067][123614] Updated weights for policy 1, policy_version 8240 (0.0008) [2023-10-10 16:53:41,360][123582] Updated weights for policy 0, policy_version 8283 (0.0009) [2023-10-10 16:53:41,438][123614] Updated weights for policy 1, policy_version 8250 (0.0008) [2023-10-10 16:53:43,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 16941056. Throughput: 0: 1788.9, 1: 1790.9. Samples: 4239558. Policy #0 lag: (min: 8.0, avg: 30.5, max: 40.0) [2023-10-10 16:53:43,789][122664] Avg episode reward: [(0, '16.520'), (1, '17.890')] [2023-10-10 16:53:45,114][123582] Updated weights for policy 0, policy_version 8293 (0.0008) [2023-10-10 16:53:45,276][123614] Updated weights for policy 1, policy_version 8260 (0.0008) [2023-10-10 16:53:45,486][123582] Updated weights for policy 0, policy_version 8303 (0.0009) [2023-10-10 16:53:45,641][123614] Updated weights for policy 1, policy_version 8270 (0.0007) [2023-10-10 16:53:45,858][123582] Updated weights for policy 0, policy_version 8313 (0.0009) [2023-10-10 16:53:46,010][123614] Updated weights for policy 1, policy_version 8280 (0.0007) [2023-10-10 16:53:48,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 17006592. Throughput: 0: 1785.1, 1: 1785.1. Samples: 4261736. Policy #0 lag: (min: 8.0, avg: 30.5, max: 40.0) [2023-10-10 16:53:48,789][122664] Avg episode reward: [(0, '16.550'), (1, '18.200')] [2023-10-10 16:53:49,629][123582] Updated weights for policy 0, policy_version 8323 (0.0008) [2023-10-10 16:53:49,697][123614] Updated weights for policy 1, policy_version 8290 (0.0008) [2023-10-10 16:53:49,996][123582] Updated weights for policy 0, policy_version 8333 (0.0008) [2023-10-10 16:53:50,067][123614] Updated weights for policy 1, policy_version 8300 (0.0008) [2023-10-10 16:53:50,360][123582] Updated weights for policy 0, policy_version 8343 (0.0010) [2023-10-10 16:53:50,430][123614] Updated weights for policy 1, policy_version 8310 (0.0009) [2023-10-10 16:53:50,805][123614] Updated weights for policy 1, policy_version 8320 (0.0009) [2023-10-10 16:53:53,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 17072128. Throughput: 0: 1792.8, 1: 1796.3. Samples: 4284506. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-10 16:53:53,789][122664] Avg episode reward: [(0, '16.780'), (1, '18.030')] [2023-10-10 16:53:53,800][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000008352_8552448.pth... [2023-10-10 16:53:53,800][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000008320_8519680.pth... [2023-10-10 16:53:53,835][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000006688_6848512.pth [2023-10-10 16:53:53,838][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000006656_6815744.pth [2023-10-10 16:53:53,839][123247] Saving a milestone ./train_atari/atari_demonattack_APPO/checkpoint_p0/milestones/checkpoint_000008352_8552448.pth [2023-10-10 16:53:53,842][123465] Saving a milestone ./train_atari/atari_demonattack_APPO/checkpoint_p1/milestones/checkpoint_000008320_8519680.pth [2023-10-10 16:53:54,106][123582] Updated weights for policy 0, policy_version 8353 (0.0007) [2023-10-10 16:53:54,374][123614] Updated weights for policy 1, policy_version 8330 (0.0009) [2023-10-10 16:53:54,517][123582] Updated weights for policy 0, policy_version 8363 (0.0008) [2023-10-10 16:53:54,738][123614] Updated weights for policy 1, policy_version 8340 (0.0008) [2023-10-10 16:53:54,889][123582] Updated weights for policy 0, policy_version 8373 (0.0008) [2023-10-10 16:53:55,104][123614] Updated weights for policy 1, policy_version 8350 (0.0008) [2023-10-10 16:53:55,265][123582] Updated weights for policy 0, policy_version 8383 (0.0008) [2023-10-10 16:53:58,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 17137664. Throughput: 0: 1792.3, 1: 1788.7. Samples: 4294242. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-10 16:53:58,789][122664] Avg episode reward: [(0, '18.370'), (1, '18.010')] [2023-10-10 16:53:58,997][123582] Updated weights for policy 0, policy_version 8393 (0.0007) [2023-10-10 16:53:59,029][123614] Updated weights for policy 1, policy_version 8360 (0.0007) [2023-10-10 16:53:59,374][123582] Updated weights for policy 0, policy_version 8403 (0.0007) [2023-10-10 16:53:59,396][123614] Updated weights for policy 1, policy_version 8370 (0.0008) [2023-10-10 16:53:59,746][123582] Updated weights for policy 0, policy_version 8413 (0.0009) [2023-10-10 16:53:59,761][123614] Updated weights for policy 1, policy_version 8380 (0.0008) [2023-10-10 16:54:03,359][123582] Updated weights for policy 0, policy_version 8423 (0.0009) [2023-10-10 16:54:03,571][123614] Updated weights for policy 1, policy_version 8390 (0.0008) [2023-10-10 16:54:03,730][123582] Updated weights for policy 0, policy_version 8433 (0.0008) [2023-10-10 16:54:03,788][122664] Fps is (10 sec: 13107.9, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 17203200. Throughput: 0: 1796.4, 1: 1796.0. Samples: 4316790. Policy #0 lag: (min: 5.0, avg: 30.1, max: 32.0) [2023-10-10 16:54:03,788][122664] Avg episode reward: [(0, '19.040'), (1, '18.350')] [2023-10-10 16:54:03,930][123614] Updated weights for policy 1, policy_version 8400 (0.0007) [2023-10-10 16:54:04,111][123582] Updated weights for policy 0, policy_version 8443 (0.0009) [2023-10-10 16:54:04,297][123614] Updated weights for policy 1, policy_version 8410 (0.0008) [2023-10-10 16:54:07,694][123582] Updated weights for policy 0, policy_version 8453 (0.0009) [2023-10-10 16:54:08,057][123614] Updated weights for policy 1, policy_version 8420 (0.0009) [2023-10-10 16:54:08,062][123582] Updated weights for policy 0, policy_version 8463 (0.0009) [2023-10-10 16:54:08,420][123614] Updated weights for policy 1, policy_version 8430 (0.0007) [2023-10-10 16:54:08,430][123582] Updated weights for policy 0, policy_version 8473 (0.0010) [2023-10-10 16:54:08,786][123614] Updated weights for policy 1, policy_version 8440 (0.0007) [2023-10-10 16:54:08,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 17301504. Throughput: 0: 1812.0, 1: 1807.2. Samples: 4337252. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:54:08,789][122664] Avg episode reward: [(0, '16.510'), (1, '18.840')] [2023-10-10 16:54:12,212][123582] Updated weights for policy 0, policy_version 8483 (0.0009) [2023-10-10 16:54:12,447][123614] Updated weights for policy 1, policy_version 8450 (0.0007) [2023-10-10 16:54:12,575][123582] Updated weights for policy 0, policy_version 8493 (0.0007) [2023-10-10 16:54:12,815][123614] Updated weights for policy 1, policy_version 8460 (0.0008) [2023-10-10 16:54:12,943][123582] Updated weights for policy 0, policy_version 8503 (0.0007) [2023-10-10 16:54:13,174][123614] Updated weights for policy 1, policy_version 8470 (0.0007) [2023-10-10 16:54:13,543][123614] Updated weights for policy 1, policy_version 8480 (0.0007) [2023-10-10 16:54:13,788][122664] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 17399808. Throughput: 0: 1800.0, 1: 1795.2. Samples: 4349236. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:54:13,789][122664] Avg episode reward: [(0, '16.610'), (1, '17.040')] [2023-10-10 16:54:16,529][123582] Updated weights for policy 0, policy_version 8513 (0.0008) [2023-10-10 16:54:16,896][123582] Updated weights for policy 0, policy_version 8523 (0.0010) [2023-10-10 16:54:17,261][123582] Updated weights for policy 0, policy_version 8533 (0.0008) [2023-10-10 16:54:17,303][123614] Updated weights for policy 1, policy_version 8490 (0.0007) [2023-10-10 16:54:17,624][123582] Updated weights for policy 0, policy_version 8543 (0.0007) [2023-10-10 16:54:17,674][123614] Updated weights for policy 1, policy_version 8500 (0.0009) [2023-10-10 16:54:18,038][123614] Updated weights for policy 1, policy_version 8510 (0.0010) [2023-10-10 16:54:18,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 17465344. Throughput: 0: 1816.6, 1: 1812.4. Samples: 4370150. Policy #0 lag: (min: 6.0, avg: 10.8, max: 38.0) [2023-10-10 16:54:18,789][122664] Avg episode reward: [(0, '17.810'), (1, '17.310')] [2023-10-10 16:54:21,272][123582] Updated weights for policy 0, policy_version 8553 (0.0009) [2023-10-10 16:54:21,635][123614] Updated weights for policy 1, policy_version 8520 (0.0008) [2023-10-10 16:54:21,637][123582] Updated weights for policy 0, policy_version 8563 (0.0009) [2023-10-10 16:54:22,009][123614] Updated weights for policy 1, policy_version 8530 (0.0007) [2023-10-10 16:54:22,010][123582] Updated weights for policy 0, policy_version 8573 (0.0010) [2023-10-10 16:54:22,387][123614] Updated weights for policy 1, policy_version 8540 (0.0009) [2023-10-10 16:54:23,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 17530880. Throughput: 0: 1813.4, 1: 1800.8. Samples: 4392262. Policy #0 lag: (min: 6.0, avg: 10.8, max: 38.0) [2023-10-10 16:54:23,789][122664] Avg episode reward: [(0, '18.870'), (1, '19.730')] [2023-10-10 16:54:25,668][123582] Updated weights for policy 0, policy_version 8583 (0.0009) [2023-10-10 16:54:26,043][123582] Updated weights for policy 0, policy_version 8593 (0.0008) [2023-10-10 16:54:26,142][123614] Updated weights for policy 1, policy_version 8550 (0.0008) [2023-10-10 16:54:26,411][123582] Updated weights for policy 0, policy_version 8603 (0.0008) [2023-10-10 16:54:26,511][123614] Updated weights for policy 1, policy_version 8560 (0.0008) [2023-10-10 16:54:26,878][123614] Updated weights for policy 1, policy_version 8570 (0.0009) [2023-10-10 16:54:28,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 17596416. Throughput: 0: 1818.7, 1: 1813.1. Samples: 4402988. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:54:28,789][122664] Avg episode reward: [(0, '18.560'), (1, '17.780')] [2023-10-10 16:54:30,281][123582] Updated weights for policy 0, policy_version 8613 (0.0008) [2023-10-10 16:54:30,657][123582] Updated weights for policy 0, policy_version 8623 (0.0007) [2023-10-10 16:54:30,720][123614] Updated weights for policy 1, policy_version 8580 (0.0008) [2023-10-10 16:54:31,027][123582] Updated weights for policy 0, policy_version 8633 (0.0007) [2023-10-10 16:54:31,095][123614] Updated weights for policy 1, policy_version 8590 (0.0008) [2023-10-10 16:54:31,461][123614] Updated weights for policy 1, policy_version 8600 (0.0009) [2023-10-10 16:54:33,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 17661952. Throughput: 0: 1812.6, 1: 1800.9. Samples: 4424346. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:54:33,789][122664] Avg episode reward: [(0, '18.600'), (1, '16.440')] [2023-10-10 16:54:34,731][123582] Updated weights for policy 0, policy_version 8643 (0.0008) [2023-10-10 16:54:35,081][123614] Updated weights for policy 1, policy_version 8610 (0.0008) [2023-10-10 16:54:35,113][123582] Updated weights for policy 0, policy_version 8653 (0.0008) [2023-10-10 16:54:35,455][123614] Updated weights for policy 1, policy_version 8620 (0.0008) [2023-10-10 16:54:35,487][123582] Updated weights for policy 0, policy_version 8663 (0.0007) [2023-10-10 16:54:35,821][123614] Updated weights for policy 1, policy_version 8630 (0.0009) [2023-10-10 16:54:36,185][123614] Updated weights for policy 1, policy_version 8640 (0.0007) [2023-10-10 16:54:38,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 17727488. Throughput: 0: 1814.7, 1: 1804.0. Samples: 4447346. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 16:54:38,788][122664] Avg episode reward: [(0, '19.350'), (1, '18.010')] [2023-10-10 16:54:39,166][123582] Updated weights for policy 0, policy_version 8673 (0.0009) [2023-10-10 16:54:39,582][123582] Updated weights for policy 0, policy_version 8683 (0.0011) [2023-10-10 16:54:39,944][123582] Updated weights for policy 0, policy_version 8693 (0.0008) [2023-10-10 16:54:39,954][123614] Updated weights for policy 1, policy_version 8650 (0.0007) [2023-10-10 16:54:40,307][123582] Updated weights for policy 0, policy_version 8703 (0.0008) [2023-10-10 16:54:40,331][123614] Updated weights for policy 1, policy_version 8660 (0.0007) [2023-10-10 16:54:40,693][123614] Updated weights for policy 1, policy_version 8670 (0.0009) [2023-10-10 16:54:43,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 17793024. Throughput: 0: 1816.2, 1: 1804.0. Samples: 4457148. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 16:54:43,789][122664] Avg episode reward: [(0, '19.390'), (1, '17.020')] [2023-10-10 16:54:44,048][123582] Updated weights for policy 0, policy_version 8713 (0.0007) [2023-10-10 16:54:44,431][123582] Updated weights for policy 0, policy_version 8723 (0.0008) [2023-10-10 16:54:44,547][123614] Updated weights for policy 1, policy_version 8680 (0.0008) [2023-10-10 16:54:44,803][123582] Updated weights for policy 0, policy_version 8733 (0.0008) [2023-10-10 16:54:44,935][123614] Updated weights for policy 1, policy_version 8690 (0.0009) [2023-10-10 16:54:45,297][123614] Updated weights for policy 1, policy_version 8700 (0.0009) [2023-10-10 16:54:48,437][123582] Updated weights for policy 0, policy_version 8743 (0.0008) [2023-10-10 16:54:48,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 17858560. Throughput: 0: 1820.6, 1: 1798.2. Samples: 4479638. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-10-10 16:54:48,789][122664] Avg episode reward: [(0, '19.990'), (1, '17.610')] [2023-10-10 16:54:48,812][123582] Updated weights for policy 0, policy_version 8753 (0.0007) [2023-10-10 16:54:48,972][123614] Updated weights for policy 1, policy_version 8710 (0.0007) [2023-10-10 16:54:49,187][123582] Updated weights for policy 0, policy_version 8763 (0.0009) [2023-10-10 16:54:49,330][123614] Updated weights for policy 1, policy_version 8720 (0.0007) [2023-10-10 16:54:49,702][123614] Updated weights for policy 1, policy_version 8730 (0.0009) [2023-10-10 16:54:53,016][123582] Updated weights for policy 0, policy_version 8773 (0.0010) [2023-10-10 16:54:53,392][123582] Updated weights for policy 0, policy_version 8783 (0.0008) [2023-10-10 16:54:53,507][123614] Updated weights for policy 1, policy_version 8740 (0.0009) [2023-10-10 16:54:53,762][123582] Updated weights for policy 0, policy_version 8793 (0.0007) [2023-10-10 16:54:53,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.6, 300 sec: 14329.1). Total num frames: 17924096. Throughput: 0: 1822.4, 1: 1814.4. Samples: 4500910. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-10-10 16:54:53,789][122664] Avg episode reward: [(0, '19.870'), (1, '16.550')] [2023-10-10 16:54:53,878][123614] Updated weights for policy 1, policy_version 8750 (0.0007) [2023-10-10 16:54:54,239][123614] Updated weights for policy 1, policy_version 8760 (0.0007) [2023-10-10 16:54:57,358][123582] Updated weights for policy 0, policy_version 8803 (0.0008) [2023-10-10 16:54:57,731][123582] Updated weights for policy 0, policy_version 8813 (0.0007) [2023-10-10 16:54:57,941][123614] Updated weights for policy 1, policy_version 8770 (0.0008) [2023-10-10 16:54:58,108][123582] Updated weights for policy 0, policy_version 8823 (0.0008) [2023-10-10 16:54:58,308][123614] Updated weights for policy 1, policy_version 8780 (0.0008) [2023-10-10 16:54:58,678][123614] Updated weights for policy 1, policy_version 8790 (0.0007) [2023-10-10 16:54:58,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 18022400. Throughput: 0: 1817.4, 1: 1805.7. Samples: 4512276. Policy #0 lag: (min: 10.0, avg: 10.6, max: 26.0) [2023-10-10 16:54:58,789][122664] Avg episode reward: [(0, '20.320'), (1, '19.230')] [2023-10-10 16:54:58,790][123247] Saving new best policy, reward=20.320! [2023-10-10 16:54:59,046][123614] Updated weights for policy 1, policy_version 8800 (0.0011) [2023-10-10 16:55:01,882][123582] Updated weights for policy 0, policy_version 8833 (0.0008) [2023-10-10 16:55:02,246][123582] Updated weights for policy 0, policy_version 8843 (0.0011) [2023-10-10 16:55:02,619][123582] Updated weights for policy 0, policy_version 8853 (0.0009) [2023-10-10 16:55:02,830][123614] Updated weights for policy 1, policy_version 8810 (0.0008) [2023-10-10 16:55:02,991][123582] Updated weights for policy 0, policy_version 8863 (0.0010) [2023-10-10 16:55:03,197][123614] Updated weights for policy 1, policy_version 8820 (0.0009) [2023-10-10 16:55:03,557][123614] Updated weights for policy 1, policy_version 8830 (0.0010) [2023-10-10 16:55:03,788][122664] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 18120704. Throughput: 0: 1821.5, 1: 1813.7. Samples: 4533736. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:55:03,789][122664] Avg episode reward: [(0, '20.680'), (1, '19.390')] [2023-10-10 16:55:03,790][123247] Saving new best policy, reward=20.680! [2023-10-10 16:55:06,740][123582] Updated weights for policy 0, policy_version 8873 (0.0008) [2023-10-10 16:55:07,094][123614] Updated weights for policy 1, policy_version 8840 (0.0009) [2023-10-10 16:55:07,121][123582] Updated weights for policy 0, policy_version 8883 (0.0009) [2023-10-10 16:55:07,462][123614] Updated weights for policy 1, policy_version 8850 (0.0008) [2023-10-10 16:55:07,486][123582] Updated weights for policy 0, policy_version 8893 (0.0008) [2023-10-10 16:55:07,837][123614] Updated weights for policy 1, policy_version 8860 (0.0007) [2023-10-10 16:55:08,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 18186240. Throughput: 0: 1805.6, 1: 1800.7. Samples: 4554546. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:55:08,789][122664] Avg episode reward: [(0, '20.730'), (1, '19.450')] [2023-10-10 16:55:08,799][123247] Saving new best policy, reward=20.730! [2023-10-10 16:55:11,077][123582] Updated weights for policy 0, policy_version 8903 (0.0010) [2023-10-10 16:55:11,445][123582] Updated weights for policy 0, policy_version 8913 (0.0010) [2023-10-10 16:55:11,635][123614] Updated weights for policy 1, policy_version 8870 (0.0008) [2023-10-10 16:55:11,813][123582] Updated weights for policy 0, policy_version 8923 (0.0008) [2023-10-10 16:55:12,004][123614] Updated weights for policy 1, policy_version 8880 (0.0008) [2023-10-10 16:55:12,384][123614] Updated weights for policy 1, policy_version 8890 (0.0007) [2023-10-10 16:55:13,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 18251776. Throughput: 0: 1812.8, 1: 1811.5. Samples: 4566080. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-10-10 16:55:13,789][122664] Avg episode reward: [(0, '21.280'), (1, '18.490')] [2023-10-10 16:55:13,790][123247] Saving new best policy, reward=21.280! [2023-10-10 16:55:15,681][123582] Updated weights for policy 0, policy_version 8933 (0.0010) [2023-10-10 16:55:16,051][123582] Updated weights for policy 0, policy_version 8943 (0.0010) [2023-10-10 16:55:16,148][123614] Updated weights for policy 1, policy_version 8900 (0.0008) [2023-10-10 16:55:16,421][123582] Updated weights for policy 0, policy_version 8953 (0.0008) [2023-10-10 16:55:16,512][123614] Updated weights for policy 1, policy_version 8910 (0.0009) [2023-10-10 16:55:16,876][123614] Updated weights for policy 1, policy_version 8920 (0.0007) [2023-10-10 16:55:18,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 18317312. Throughput: 0: 1807.1, 1: 1800.6. Samples: 4586692. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-10-10 16:55:18,789][122664] Avg episode reward: [(0, '20.400'), (1, '17.910')] [2023-10-10 16:55:20,025][123582] Updated weights for policy 0, policy_version 8963 (0.0009) [2023-10-10 16:55:20,398][123582] Updated weights for policy 0, policy_version 8973 (0.0008) [2023-10-10 16:55:20,460][123614] Updated weights for policy 1, policy_version 8930 (0.0009) [2023-10-10 16:55:20,771][123582] Updated weights for policy 0, policy_version 8983 (0.0007) [2023-10-10 16:55:20,836][123614] Updated weights for policy 1, policy_version 8940 (0.0009) [2023-10-10 16:55:21,204][123614] Updated weights for policy 1, policy_version 8950 (0.0008) [2023-10-10 16:55:21,573][123614] Updated weights for policy 1, policy_version 8960 (0.0008) [2023-10-10 16:55:23,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 18382848. Throughput: 0: 1805.8, 1: 1799.1. Samples: 4609564. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-10 16:55:23,788][122664] Avg episode reward: [(0, '19.840'), (1, '17.180')] [2023-10-10 16:55:24,555][123582] Updated weights for policy 0, policy_version 8993 (0.0010) [2023-10-10 16:55:24,967][123582] Updated weights for policy 0, policy_version 9003 (0.0008) [2023-10-10 16:55:25,317][123614] Updated weights for policy 1, policy_version 8970 (0.0007) [2023-10-10 16:55:25,337][123582] Updated weights for policy 0, policy_version 9013 (0.0008) [2023-10-10 16:55:25,692][123614] Updated weights for policy 1, policy_version 8980 (0.0008) [2023-10-10 16:55:25,708][123582] Updated weights for policy 0, policy_version 9023 (0.0007) [2023-10-10 16:55:26,060][123614] Updated weights for policy 1, policy_version 8990 (0.0008) [2023-10-10 16:55:28,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 18448384. Throughput: 0: 1804.5, 1: 1801.6. Samples: 4619426. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-10 16:55:28,789][122664] Avg episode reward: [(0, '18.160'), (1, '17.630')] [2023-10-10 16:55:29,330][123582] Updated weights for policy 0, policy_version 9033 (0.0010) [2023-10-10 16:55:29,709][123582] Updated weights for policy 0, policy_version 9043 (0.0009) [2023-10-10 16:55:29,859][123614] Updated weights for policy 1, policy_version 9000 (0.0008) [2023-10-10 16:55:30,080][123582] Updated weights for policy 0, policy_version 9053 (0.0008) [2023-10-10 16:55:30,223][123614] Updated weights for policy 1, policy_version 9010 (0.0007) [2023-10-10 16:55:30,586][123614] Updated weights for policy 1, policy_version 9020 (0.0009) [2023-10-10 16:55:33,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.2). Total num frames: 18513920. Throughput: 0: 1803.4, 1: 1805.5. Samples: 4642038. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:55:33,789][122664] Avg episode reward: [(0, '16.790'), (1, '18.300')] [2023-10-10 16:55:33,796][123582] Updated weights for policy 0, policy_version 9063 (0.0007) [2023-10-10 16:55:34,169][123582] Updated weights for policy 0, policy_version 9073 (0.0009) [2023-10-10 16:55:34,223][123614] Updated weights for policy 1, policy_version 9030 (0.0008) [2023-10-10 16:55:34,540][123582] Updated weights for policy 0, policy_version 9083 (0.0009) [2023-10-10 16:55:34,612][123614] Updated weights for policy 1, policy_version 9040 (0.0008) [2023-10-10 16:55:34,973][123614] Updated weights for policy 1, policy_version 9050 (0.0009) [2023-10-10 16:55:38,148][123582] Updated weights for policy 0, policy_version 9093 (0.0010) [2023-10-10 16:55:38,528][123582] Updated weights for policy 0, policy_version 9103 (0.0009) [2023-10-10 16:55:38,635][123614] Updated weights for policy 1, policy_version 9060 (0.0007) [2023-10-10 16:55:38,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 18579456. Throughput: 0: 1813.9, 1: 1807.8. Samples: 4663886. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:55:38,789][122664] Avg episode reward: [(0, '16.920'), (1, '17.940')] [2023-10-10 16:55:38,896][123582] Updated weights for policy 0, policy_version 9113 (0.0008) [2023-10-10 16:55:39,001][123614] Updated weights for policy 1, policy_version 9070 (0.0010) [2023-10-10 16:55:39,376][123614] Updated weights for policy 1, policy_version 9080 (0.0008) [2023-10-10 16:55:42,695][123582] Updated weights for policy 0, policy_version 9123 (0.0008) [2023-10-10 16:55:43,064][123582] Updated weights for policy 0, policy_version 9133 (0.0007) [2023-10-10 16:55:43,122][123614] Updated weights for policy 1, policy_version 9090 (0.0009) [2023-10-10 16:55:43,438][123582] Updated weights for policy 0, policy_version 9143 (0.0008) [2023-10-10 16:55:43,491][123614] Updated weights for policy 1, policy_version 9100 (0.0008) [2023-10-10 16:55:43,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 18677760. Throughput: 0: 1805.3, 1: 1799.0. Samples: 4674466. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 16:55:43,788][122664] Avg episode reward: [(0, '17.520'), (1, '18.050')] [2023-10-10 16:55:43,858][123614] Updated weights for policy 1, policy_version 9110 (0.0008) [2023-10-10 16:55:44,229][123614] Updated weights for policy 1, policy_version 9120 (0.0010) [2023-10-10 16:55:47,076][123582] Updated weights for policy 0, policy_version 9153 (0.0007) [2023-10-10 16:55:47,450][123582] Updated weights for policy 0, policy_version 9163 (0.0008) [2023-10-10 16:55:47,819][123582] Updated weights for policy 0, policy_version 9173 (0.0008) [2023-10-10 16:55:48,070][123614] Updated weights for policy 1, policy_version 9130 (0.0008) [2023-10-10 16:55:48,188][123582] Updated weights for policy 0, policy_version 9183 (0.0008) [2023-10-10 16:55:48,442][123614] Updated weights for policy 1, policy_version 9140 (0.0008) [2023-10-10 16:55:48,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 18743296. Throughput: 0: 1810.9, 1: 1807.3. Samples: 4696556. Policy #0 lag: (min: 17.0, avg: 32.2, max: 49.0) [2023-10-10 16:55:48,789][122664] Avg episode reward: [(0, '16.440'), (1, '18.290')] [2023-10-10 16:55:48,808][123614] Updated weights for policy 1, policy_version 9150 (0.0008) [2023-10-10 16:55:51,895][123582] Updated weights for policy 0, policy_version 9193 (0.0007) [2023-10-10 16:55:52,262][123582] Updated weights for policy 0, policy_version 9203 (0.0008) [2023-10-10 16:55:52,638][123582] Updated weights for policy 0, policy_version 9213 (0.0008) [2023-10-10 16:55:52,663][123614] Updated weights for policy 1, policy_version 9160 (0.0009) [2023-10-10 16:55:53,032][123614] Updated weights for policy 1, policy_version 9170 (0.0009) [2023-10-10 16:55:53,413][123614] Updated weights for policy 1, policy_version 9180 (0.0007) [2023-10-10 16:55:53,788][122664] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 18841600. Throughput: 0: 1804.8, 1: 1792.8. Samples: 4716438. Policy #0 lag: (min: 17.0, avg: 32.2, max: 49.0) [2023-10-10 16:55:53,789][122664] Avg episode reward: [(0, '15.990'), (1, '17.140')] [2023-10-10 16:55:53,798][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000009216_9437184.pth... [2023-10-10 16:55:53,798][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000009184_9404416.pth... [2023-10-10 16:55:53,835][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000007488_7667712.pth [2023-10-10 16:55:53,836][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000007520_7700480.pth [2023-10-10 16:55:56,326][123582] Updated weights for policy 0, policy_version 9223 (0.0008) [2023-10-10 16:55:56,703][123582] Updated weights for policy 0, policy_version 9233 (0.0008) [2023-10-10 16:55:57,073][123582] Updated weights for policy 0, policy_version 9243 (0.0008) [2023-10-10 16:55:57,157][123614] Updated weights for policy 1, policy_version 9190 (0.0009) [2023-10-10 16:55:57,529][123614] Updated weights for policy 1, policy_version 9200 (0.0010) [2023-10-10 16:55:57,893][123614] Updated weights for policy 1, policy_version 9210 (0.0008) [2023-10-10 16:55:58,788][122664] Fps is (10 sec: 16384.6, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 18907136. Throughput: 0: 1816.1, 1: 1803.5. Samples: 4728962. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:55:58,788][122664] Avg episode reward: [(0, '16.410'), (1, '17.680')] [2023-10-10 16:56:00,812][123582] Updated weights for policy 0, policy_version 9253 (0.0009) [2023-10-10 16:56:01,181][123582] Updated weights for policy 0, policy_version 9263 (0.0009) [2023-10-10 16:56:01,552][123582] Updated weights for policy 0, policy_version 9273 (0.0008) [2023-10-10 16:56:01,566][123614] Updated weights for policy 1, policy_version 9220 (0.0007) [2023-10-10 16:56:01,932][123614] Updated weights for policy 1, policy_version 9230 (0.0007) [2023-10-10 16:56:02,307][123614] Updated weights for policy 1, policy_version 9240 (0.0009) [2023-10-10 16:56:03,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 18972672. Throughput: 0: 1811.4, 1: 1800.8. Samples: 4749242. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:56:03,789][122664] Avg episode reward: [(0, '15.540'), (1, '16.600')] [2023-10-10 16:56:05,162][123582] Updated weights for policy 0, policy_version 9283 (0.0007) [2023-10-10 16:56:05,528][123582] Updated weights for policy 0, policy_version 9293 (0.0007) [2023-10-10 16:56:05,912][123582] Updated weights for policy 0, policy_version 9303 (0.0007) [2023-10-10 16:56:05,980][123614] Updated weights for policy 1, policy_version 9250 (0.0007) [2023-10-10 16:56:06,351][123614] Updated weights for policy 1, policy_version 9260 (0.0008) [2023-10-10 16:56:06,717][123614] Updated weights for policy 1, policy_version 9270 (0.0009) [2023-10-10 16:56:07,090][123614] Updated weights for policy 1, policy_version 9280 (0.0008) [2023-10-10 16:56:08,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 19038208. Throughput: 0: 1820.2, 1: 1793.5. Samples: 4772182. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 16:56:08,788][122664] Avg episode reward: [(0, '17.870'), (1, '15.620')] [2023-10-10 16:56:09,589][123582] Updated weights for policy 0, policy_version 9313 (0.0007) [2023-10-10 16:56:09,996][123582] Updated weights for policy 0, policy_version 9323 (0.0010) [2023-10-10 16:56:10,365][123582] Updated weights for policy 0, policy_version 9333 (0.0008) [2023-10-10 16:56:10,743][123582] Updated weights for policy 0, policy_version 9343 (0.0008) [2023-10-10 16:56:10,775][123614] Updated weights for policy 1, policy_version 9290 (0.0007) [2023-10-10 16:56:11,146][123614] Updated weights for policy 1, policy_version 9300 (0.0007) [2023-10-10 16:56:11,512][123614] Updated weights for policy 1, policy_version 9310 (0.0008) [2023-10-10 16:56:13,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 19103744. Throughput: 0: 1819.3, 1: 1792.2. Samples: 4781944. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 16:56:13,789][122664] Avg episode reward: [(0, '17.450'), (1, '15.700')] [2023-10-10 16:56:14,415][123582] Updated weights for policy 0, policy_version 9353 (0.0010) [2023-10-10 16:56:14,774][123582] Updated weights for policy 0, policy_version 9363 (0.0010) [2023-10-10 16:56:15,153][123582] Updated weights for policy 0, policy_version 9373 (0.0008) [2023-10-10 16:56:15,251][123614] Updated weights for policy 1, policy_version 9320 (0.0007) [2023-10-10 16:56:15,632][123614] Updated weights for policy 1, policy_version 9330 (0.0009) [2023-10-10 16:56:16,004][123614] Updated weights for policy 1, policy_version 9340 (0.0007) [2023-10-10 16:56:18,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 19169280. Throughput: 0: 1814.9, 1: 1795.4. Samples: 4804498. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) [2023-10-10 16:56:18,788][122664] Avg episode reward: [(0, '16.960'), (1, '17.150')] [2023-10-10 16:56:18,983][123582] Updated weights for policy 0, policy_version 9383 (0.0009) [2023-10-10 16:56:19,357][123582] Updated weights for policy 0, policy_version 9393 (0.0011) [2023-10-10 16:56:19,730][123582] Updated weights for policy 0, policy_version 9403 (0.0009) [2023-10-10 16:56:19,732][123614] Updated weights for policy 1, policy_version 9350 (0.0008) [2023-10-10 16:56:20,098][123614] Updated weights for policy 1, policy_version 9360 (0.0007) [2023-10-10 16:56:20,473][123614] Updated weights for policy 1, policy_version 9370 (0.0007) [2023-10-10 16:56:23,457][123582] Updated weights for policy 0, policy_version 9413 (0.0007) [2023-10-10 16:56:23,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 19234816. Throughput: 0: 1821.5, 1: 1806.2. Samples: 4827134. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) [2023-10-10 16:56:23,789][122664] Avg episode reward: [(0, '17.660'), (1, '17.970')] [2023-10-10 16:56:23,819][123582] Updated weights for policy 0, policy_version 9423 (0.0008) [2023-10-10 16:56:24,164][123614] Updated weights for policy 1, policy_version 9380 (0.0009) [2023-10-10 16:56:24,193][123582] Updated weights for policy 0, policy_version 9433 (0.0008) [2023-10-10 16:56:24,538][123614] Updated weights for policy 1, policy_version 9390 (0.0008) [2023-10-10 16:56:24,913][123614] Updated weights for policy 1, policy_version 9400 (0.0007) [2023-10-10 16:56:27,765][123582] Updated weights for policy 0, policy_version 9443 (0.0007) [2023-10-10 16:56:28,148][123582] Updated weights for policy 0, policy_version 9453 (0.0008) [2023-10-10 16:56:28,472][123614] Updated weights for policy 1, policy_version 9410 (0.0008) [2023-10-10 16:56:28,510][123582] Updated weights for policy 0, policy_version 9463 (0.0008) [2023-10-10 16:56:28,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 19300352. Throughput: 0: 1816.9, 1: 1802.5. Samples: 4837338. Policy #0 lag: (min: 31.0, avg: 31.6, max: 47.0) [2023-10-10 16:56:28,788][122664] Avg episode reward: [(0, '19.140'), (1, '18.030')] [2023-10-10 16:56:28,838][123614] Updated weights for policy 1, policy_version 9420 (0.0007) [2023-10-10 16:56:29,202][123614] Updated weights for policy 1, policy_version 9430 (0.0009) [2023-10-10 16:56:29,570][123614] Updated weights for policy 1, policy_version 9440 (0.0009) [2023-10-10 16:56:32,125][123582] Updated weights for policy 0, policy_version 9473 (0.0007) [2023-10-10 16:56:32,487][123582] Updated weights for policy 0, policy_version 9483 (0.0008) [2023-10-10 16:56:32,863][123582] Updated weights for policy 0, policy_version 9493 (0.0008) [2023-10-10 16:56:33,229][123582] Updated weights for policy 0, policy_version 9503 (0.0008) [2023-10-10 16:56:33,333][123614] Updated weights for policy 1, policy_version 9450 (0.0009) [2023-10-10 16:56:33,707][123614] Updated weights for policy 1, policy_version 9460 (0.0010) [2023-10-10 16:56:33,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14440.2). Total num frames: 19398656. Throughput: 0: 1819.0, 1: 1810.1. Samples: 4859866. Policy #0 lag: (min: 21.0, avg: 22.8, max: 49.0) [2023-10-10 16:56:33,788][122664] Avg episode reward: [(0, '20.600'), (1, '19.170')] [2023-10-10 16:56:34,074][123614] Updated weights for policy 1, policy_version 9470 (0.0007) [2023-10-10 16:56:36,706][123582] Updated weights for policy 0, policy_version 9513 (0.0007) [2023-10-10 16:56:37,072][123582] Updated weights for policy 0, policy_version 9523 (0.0008) [2023-10-10 16:56:37,448][123582] Updated weights for policy 0, policy_version 9533 (0.0011) [2023-10-10 16:56:37,868][123614] Updated weights for policy 1, policy_version 9480 (0.0009) [2023-10-10 16:56:38,234][123614] Updated weights for policy 1, policy_version 9490 (0.0008) [2023-10-10 16:56:38,613][123614] Updated weights for policy 1, policy_version 9500 (0.0007) [2023-10-10 16:56:38,788][122664] Fps is (10 sec: 19660.7, 60 sec: 15291.8, 300 sec: 14440.1). Total num frames: 19496960. Throughput: 0: 1831.5, 1: 1814.2. Samples: 4880494. Policy #0 lag: (min: 21.0, avg: 22.8, max: 49.0) [2023-10-10 16:56:38,788][122664] Avg episode reward: [(0, '17.240'), (1, '20.410')] [2023-10-10 16:56:38,799][123465] Saving new best policy, reward=20.410! [2023-10-10 16:56:41,163][123582] Updated weights for policy 0, policy_version 9543 (0.0008) [2023-10-10 16:56:41,545][123582] Updated weights for policy 0, policy_version 9553 (0.0009) [2023-10-10 16:56:41,914][123582] Updated weights for policy 0, policy_version 9563 (0.0009) [2023-10-10 16:56:42,174][123614] Updated weights for policy 1, policy_version 9510 (0.0009) [2023-10-10 16:56:42,557][123614] Updated weights for policy 1, policy_version 9520 (0.0009) [2023-10-10 16:56:42,927][123614] Updated weights for policy 1, policy_version 9530 (0.0008) [2023-10-10 16:56:43,788][122664] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 19562496. Throughput: 0: 1821.3, 1: 1814.9. Samples: 4892590. Policy #0 lag: (min: 18.0, avg: 20.7, max: 50.0) [2023-10-10 16:56:43,789][122664] Avg episode reward: [(0, '17.890'), (1, '19.820')] [2023-10-10 16:56:45,718][123582] Updated weights for policy 0, policy_version 9573 (0.0008) [2023-10-10 16:56:46,082][123582] Updated weights for policy 0, policy_version 9583 (0.0011) [2023-10-10 16:56:46,455][123582] Updated weights for policy 0, policy_version 9593 (0.0009) [2023-10-10 16:56:46,693][123614] Updated weights for policy 1, policy_version 9540 (0.0008) [2023-10-10 16:56:47,060][123614] Updated weights for policy 1, policy_version 9550 (0.0008) [2023-10-10 16:56:47,430][123614] Updated weights for policy 1, policy_version 9560 (0.0009) [2023-10-10 16:56:48,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 19628032. Throughput: 0: 1820.8, 1: 1815.9. Samples: 4912890. Policy #0 lag: (min: 18.0, avg: 20.7, max: 50.0) [2023-10-10 16:56:48,789][122664] Avg episode reward: [(0, '18.640'), (1, '18.930')] [2023-10-10 16:56:50,138][123582] Updated weights for policy 0, policy_version 9603 (0.0007) [2023-10-10 16:56:50,521][123582] Updated weights for policy 0, policy_version 9613 (0.0009) [2023-10-10 16:56:50,899][123582] Updated weights for policy 0, policy_version 9623 (0.0009) [2023-10-10 16:56:51,191][123614] Updated weights for policy 1, policy_version 9570 (0.0007) [2023-10-10 16:56:51,560][123614] Updated weights for policy 1, policy_version 9580 (0.0009) [2023-10-10 16:56:51,925][123614] Updated weights for policy 1, policy_version 9590 (0.0007) [2023-10-10 16:56:52,296][123614] Updated weights for policy 1, policy_version 9600 (0.0008) [2023-10-10 16:56:53,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 19693568. Throughput: 0: 1809.7, 1: 1818.3. Samples: 4935442. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-10 16:56:53,790][122664] Avg episode reward: [(0, '19.910'), (1, '18.590')] [2023-10-10 16:56:54,769][123582] Updated weights for policy 0, policy_version 9633 (0.0008) [2023-10-10 16:56:55,177][123582] Updated weights for policy 0, policy_version 9643 (0.0010) [2023-10-10 16:56:55,553][123582] Updated weights for policy 0, policy_version 9653 (0.0008) [2023-10-10 16:56:55,931][123582] Updated weights for policy 0, policy_version 9663 (0.0008) [2023-10-10 16:56:56,086][123614] Updated weights for policy 1, policy_version 9610 (0.0008) [2023-10-10 16:56:56,454][123614] Updated weights for policy 1, policy_version 9620 (0.0007) [2023-10-10 16:56:56,830][123614] Updated weights for policy 1, policy_version 9630 (0.0007) [2023-10-10 16:56:58,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14440.2). Total num frames: 19759104. Throughput: 0: 1807.4, 1: 1822.2. Samples: 4945278. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-10 16:56:58,789][122664] Avg episode reward: [(0, '18.050'), (1, '19.200')] [2023-10-10 16:56:59,553][123582] Updated weights for policy 0, policy_version 9673 (0.0009) [2023-10-10 16:56:59,917][123582] Updated weights for policy 0, policy_version 9683 (0.0008) [2023-10-10 16:57:00,291][123582] Updated weights for policy 0, policy_version 9693 (0.0009) [2023-10-10 16:57:00,530][123614] Updated weights for policy 1, policy_version 9640 (0.0007) [2023-10-10 16:57:00,900][123614] Updated weights for policy 1, policy_version 9650 (0.0008) [2023-10-10 16:57:01,276][123614] Updated weights for policy 1, policy_version 9660 (0.0007) [2023-10-10 16:57:03,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 19824640. Throughput: 0: 1811.9, 1: 1815.3. Samples: 4967724. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:57:03,789][122664] Avg episode reward: [(0, '19.380'), (1, '21.150')] [2023-10-10 16:57:03,791][123465] Saving new best policy, reward=21.150! [2023-10-10 16:57:04,029][123582] Updated weights for policy 0, policy_version 9703 (0.0009) [2023-10-10 16:57:04,402][123582] Updated weights for policy 0, policy_version 9713 (0.0009) [2023-10-10 16:57:04,775][123582] Updated weights for policy 0, policy_version 9723 (0.0009) [2023-10-10 16:57:05,012][123614] Updated weights for policy 1, policy_version 9670 (0.0009) [2023-10-10 16:57:05,379][123614] Updated weights for policy 1, policy_version 9680 (0.0008) [2023-10-10 16:57:05,752][123614] Updated weights for policy 1, policy_version 9690 (0.0007) [2023-10-10 16:57:08,365][123582] Updated weights for policy 0, policy_version 9733 (0.0008) [2023-10-10 16:57:08,734][123582] Updated weights for policy 0, policy_version 9743 (0.0008) [2023-10-10 16:57:08,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 19890176. Throughput: 0: 1813.0, 1: 1813.4. Samples: 4990322. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:57:08,789][122664] Avg episode reward: [(0, '19.050'), (1, '20.410')] [2023-10-10 16:57:09,103][123582] Updated weights for policy 0, policy_version 9753 (0.0007) [2023-10-10 16:57:09,308][123614] Updated weights for policy 1, policy_version 9700 (0.0008) [2023-10-10 16:57:09,674][123614] Updated weights for policy 1, policy_version 9710 (0.0008) [2023-10-10 16:57:10,048][123614] Updated weights for policy 1, policy_version 9720 (0.0011) [2023-10-10 16:57:12,695][123582] Updated weights for policy 0, policy_version 9763 (0.0008) [2023-10-10 16:57:13,069][123582] Updated weights for policy 0, policy_version 9773 (0.0008) [2023-10-10 16:57:13,438][123582] Updated weights for policy 0, policy_version 9783 (0.0008) [2023-10-10 16:57:13,762][123614] Updated weights for policy 1, policy_version 9730 (0.0011) [2023-10-10 16:57:13,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 19988480. Throughput: 0: 1812.5, 1: 1812.3. Samples: 5000452. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:57:13,789][122664] Avg episode reward: [(0, '19.430'), (1, '22.510')] [2023-10-10 16:57:14,133][123614] Updated weights for policy 1, policy_version 9740 (0.0008) [2023-10-10 16:57:14,508][123614] Updated weights for policy 1, policy_version 9750 (0.0010) [2023-10-10 16:57:14,868][123465] Saving new best policy, reward=22.510! [2023-10-10 16:57:14,873][123614] Updated weights for policy 1, policy_version 9760 (0.0009) [2023-10-10 16:57:17,181][123582] Updated weights for policy 0, policy_version 9793 (0.0009) [2023-10-10 16:57:17,549][123582] Updated weights for policy 0, policy_version 9803 (0.0009) [2023-10-10 16:57:17,920][123582] Updated weights for policy 0, policy_version 9813 (0.0011) [2023-10-10 16:57:18,293][123582] Updated weights for policy 0, policy_version 9823 (0.0008) [2023-10-10 16:57:18,526][123614] Updated weights for policy 1, policy_version 9770 (0.0008) [2023-10-10 16:57:18,788][122664] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 20054016. Throughput: 0: 1811.1, 1: 1811.2. Samples: 5022870. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-10 16:57:18,788][122664] Avg episode reward: [(0, '20.590'), (1, '19.970')] [2023-10-10 16:57:18,900][123614] Updated weights for policy 1, policy_version 9780 (0.0007) [2023-10-10 16:57:19,274][123614] Updated weights for policy 1, policy_version 9790 (0.0008) [2023-10-10 16:57:22,172][123582] Updated weights for policy 0, policy_version 9833 (0.0007) [2023-10-10 16:57:22,538][123582] Updated weights for policy 0, policy_version 9843 (0.0008) [2023-10-10 16:57:22,804][123614] Updated weights for policy 1, policy_version 9800 (0.0009) [2023-10-10 16:57:22,910][123582] Updated weights for policy 0, policy_version 9853 (0.0008) [2023-10-10 16:57:23,181][123614] Updated weights for policy 1, policy_version 9810 (0.0008) [2023-10-10 16:57:23,556][123614] Updated weights for policy 1, policy_version 9820 (0.0009) [2023-10-10 16:57:23,788][122664] Fps is (10 sec: 16383.2, 60 sec: 15291.6, 300 sec: 14440.1). Total num frames: 20152320. Throughput: 0: 1794.5, 1: 1814.7. Samples: 5042908. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-10 16:57:23,790][122664] Avg episode reward: [(0, '19.300'), (1, '21.390')] [2023-10-10 16:57:26,762][123582] Updated weights for policy 0, policy_version 9863 (0.0007) [2023-10-10 16:57:27,137][123614] Updated weights for policy 1, policy_version 9830 (0.0009) [2023-10-10 16:57:27,143][123582] Updated weights for policy 0, policy_version 9873 (0.0010) [2023-10-10 16:57:27,505][123614] Updated weights for policy 1, policy_version 9840 (0.0007) [2023-10-10 16:57:27,519][123582] Updated weights for policy 0, policy_version 9883 (0.0008) [2023-10-10 16:57:27,870][123614] Updated weights for policy 1, policy_version 9850 (0.0008) [2023-10-10 16:57:28,788][122664] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 20217856. Throughput: 0: 1809.2, 1: 1818.4. Samples: 5055830. Policy #0 lag: (min: 26.0, avg: 26.8, max: 45.0) [2023-10-10 16:57:28,789][122664] Avg episode reward: [(0, '18.610'), (1, '20.210')] [2023-10-10 16:57:31,361][123582] Updated weights for policy 0, policy_version 9893 (0.0008) [2023-10-10 16:57:31,680][123614] Updated weights for policy 1, policy_version 9860 (0.0008) [2023-10-10 16:57:31,744][123582] Updated weights for policy 0, policy_version 9903 (0.0008) [2023-10-10 16:57:32,053][123614] Updated weights for policy 1, policy_version 9870 (0.0008) [2023-10-10 16:57:32,113][123582] Updated weights for policy 0, policy_version 9913 (0.0009) [2023-10-10 16:57:32,415][123614] Updated weights for policy 1, policy_version 9880 (0.0007) [2023-10-10 16:57:33,788][122664] Fps is (10 sec: 13107.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 20283392. Throughput: 0: 1795.1, 1: 1816.4. Samples: 5075404. Policy #0 lag: (min: 26.0, avg: 26.8, max: 45.0) [2023-10-10 16:57:33,789][122664] Avg episode reward: [(0, '17.330'), (1, '18.410')] [2023-10-10 16:57:35,747][123582] Updated weights for policy 0, policy_version 9923 (0.0009) [2023-10-10 16:57:36,113][123582] Updated weights for policy 0, policy_version 9933 (0.0009) [2023-10-10 16:57:36,126][123614] Updated weights for policy 1, policy_version 9890 (0.0009) [2023-10-10 16:57:36,483][123582] Updated weights for policy 0, policy_version 9943 (0.0008) [2023-10-10 16:57:36,496][123614] Updated weights for policy 1, policy_version 9900 (0.0009) [2023-10-10 16:57:36,865][123614] Updated weights for policy 1, policy_version 9910 (0.0008) [2023-10-10 16:57:37,229][123614] Updated weights for policy 1, policy_version 9920 (0.0008) [2023-10-10 16:57:38,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 20348928. Throughput: 0: 1793.7, 1: 1817.5. Samples: 5097946. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-10 16:57:38,789][122664] Avg episode reward: [(0, '17.530'), (1, '19.000')] [2023-10-10 16:57:40,474][123582] Updated weights for policy 0, policy_version 9953 (0.0008) [2023-10-10 16:57:40,891][123582] Updated weights for policy 0, policy_version 9963 (0.0008) [2023-10-10 16:57:40,969][123614] Updated weights for policy 1, policy_version 9930 (0.0008) [2023-10-10 16:57:41,271][123582] Updated weights for policy 0, policy_version 9973 (0.0007) [2023-10-10 16:57:41,340][123614] Updated weights for policy 1, policy_version 9940 (0.0008) [2023-10-10 16:57:41,642][123582] Updated weights for policy 0, policy_version 9983 (0.0009) [2023-10-10 16:57:41,712][123614] Updated weights for policy 1, policy_version 9950 (0.0008) [2023-10-10 16:57:43,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 20414464. Throughput: 0: 1800.0, 1: 1817.5. Samples: 5108068. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-10 16:57:43,789][122664] Avg episode reward: [(0, '18.810'), (1, '18.190')] [2023-10-10 16:57:45,324][123582] Updated weights for policy 0, policy_version 9993 (0.0007) [2023-10-10 16:57:45,438][123614] Updated weights for policy 1, policy_version 9960 (0.0008) [2023-10-10 16:57:45,688][123582] Updated weights for policy 0, policy_version 10003 (0.0009) [2023-10-10 16:57:45,816][123614] Updated weights for policy 1, policy_version 9970 (0.0007) [2023-10-10 16:57:46,059][123582] Updated weights for policy 0, policy_version 10013 (0.0008) [2023-10-10 16:57:46,180][123614] Updated weights for policy 1, policy_version 9980 (0.0007) [2023-10-10 16:57:48,788][122664] Fps is (10 sec: 13107.8, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 20480000. Throughput: 0: 1786.7, 1: 1823.5. Samples: 5130180. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-10 16:57:48,788][122664] Avg episode reward: [(0, '18.790'), (1, '19.010')] [2023-10-10 16:57:49,770][123582] Updated weights for policy 0, policy_version 10023 (0.0007) [2023-10-10 16:57:50,100][123614] Updated weights for policy 1, policy_version 9990 (0.0007) [2023-10-10 16:57:50,146][123582] Updated weights for policy 0, policy_version 10033 (0.0009) [2023-10-10 16:57:50,478][123614] Updated weights for policy 1, policy_version 10000 (0.0010) [2023-10-10 16:57:50,520][123582] Updated weights for policy 0, policy_version 10043 (0.0007) [2023-10-10 16:57:50,851][123614] Updated weights for policy 1, policy_version 10010 (0.0010) [2023-10-10 16:57:53,788][122664] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 20545536. Throughput: 0: 1795.6, 1: 1813.5. Samples: 5152732. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-10 16:57:53,790][122664] Avg episode reward: [(0, '18.590'), (1, '19.880')] [2023-10-10 16:57:53,802][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000010016_10256384.pth... [2023-10-10 16:57:53,841][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000008320_8519680.pth [2023-10-10 16:57:54,037][123582] Updated weights for policy 0, policy_version 10053 (0.0007) [2023-10-10 16:57:54,416][123582] Updated weights for policy 0, policy_version 10063 (0.0008) [2023-10-10 16:57:54,482][123614] Updated weights for policy 1, policy_version 10020 (0.0010) [2023-10-10 16:57:54,775][123582] Updated weights for policy 0, policy_version 10073 (0.0009) [2023-10-10 16:57:54,852][123614] Updated weights for policy 1, policy_version 10030 (0.0008) [2023-10-10 16:57:55,033][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000010080_10321920.pth... [2023-10-10 16:57:55,062][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000008352_8552448.pth [2023-10-10 16:57:55,218][123614] Updated weights for policy 1, policy_version 10040 (0.0009) [2023-10-10 16:57:58,505][123582] Updated weights for policy 0, policy_version 10083 (0.0009) [2023-10-10 16:57:58,788][122664] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 20611072. Throughput: 0: 1788.7, 1: 1818.4. Samples: 5162772. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-10 16:57:58,789][122664] Avg episode reward: [(0, '19.120'), (1, '21.470')] [2023-10-10 16:57:58,858][123614] Updated weights for policy 1, policy_version 10050 (0.0008) [2023-10-10 16:57:58,871][123582] Updated weights for policy 0, policy_version 10093 (0.0007) [2023-10-10 16:57:59,234][123614] Updated weights for policy 1, policy_version 10060 (0.0007) [2023-10-10 16:57:59,247][123582] Updated weights for policy 0, policy_version 10103 (0.0008) [2023-10-10 16:57:59,606][123614] Updated weights for policy 1, policy_version 10070 (0.0007) [2023-10-10 16:57:59,971][123614] Updated weights for policy 1, policy_version 10080 (0.0009) [2023-10-10 16:58:03,006][123582] Updated weights for policy 0, policy_version 10113 (0.0008) [2023-10-10 16:58:03,373][123582] Updated weights for policy 0, policy_version 10123 (0.0007) [2023-10-10 16:58:03,656][123614] Updated weights for policy 1, policy_version 10090 (0.0008) [2023-10-10 16:58:03,746][123582] Updated weights for policy 0, policy_version 10133 (0.0008) [2023-10-10 16:58:03,788][122664] Fps is (10 sec: 13107.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 20676608. Throughput: 0: 1803.0, 1: 1813.7. Samples: 5185624. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:58:03,788][122664] Avg episode reward: [(0, '20.480'), (1, '21.760')] [2023-10-10 16:58:04,016][123614] Updated weights for policy 1, policy_version 10100 (0.0008) [2023-10-10 16:58:04,124][123582] Updated weights for policy 0, policy_version 10143 (0.0008) [2023-10-10 16:58:04,380][123614] Updated weights for policy 1, policy_version 10110 (0.0009) [2023-10-10 16:58:07,775][123582] Updated weights for policy 0, policy_version 10153 (0.0010) [2023-10-10 16:58:08,081][123614] Updated weights for policy 1, policy_version 10120 (0.0009) [2023-10-10 16:58:08,153][123582] Updated weights for policy 0, policy_version 10163 (0.0007) [2023-10-10 16:58:08,444][123614] Updated weights for policy 1, policy_version 10130 (0.0007) [2023-10-10 16:58:08,523][123582] Updated weights for policy 0, policy_version 10173 (0.0008) [2023-10-10 16:58:08,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 20774912. Throughput: 0: 1807.0, 1: 1820.7. Samples: 5206152. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:58:08,789][122664] Avg episode reward: [(0, '21.470'), (1, '21.810')] [2023-10-10 16:58:08,798][123247] Saving new best policy, reward=21.470! [2023-10-10 16:58:08,818][123614] Updated weights for policy 1, policy_version 10140 (0.0009) [2023-10-10 16:58:12,061][123582] Updated weights for policy 0, policy_version 10183 (0.0009) [2023-10-10 16:58:12,428][123582] Updated weights for policy 0, policy_version 10193 (0.0008) [2023-10-10 16:58:12,589][123614] Updated weights for policy 1, policy_version 10150 (0.0008) [2023-10-10 16:58:12,806][123582] Updated weights for policy 0, policy_version 10203 (0.0009) [2023-10-10 16:58:12,948][123614] Updated weights for policy 1, policy_version 10160 (0.0008) [2023-10-10 16:58:13,321][123614] Updated weights for policy 1, policy_version 10170 (0.0008) [2023-10-10 16:58:13,788][122664] Fps is (10 sec: 19660.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 20873216. Throughput: 0: 1807.6, 1: 1803.5. Samples: 5218330. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:58:13,789][122664] Avg episode reward: [(0, '21.070'), (1, '21.980')] [2023-10-10 16:58:16,537][123582] Updated weights for policy 0, policy_version 10213 (0.0010) [2023-10-10 16:58:16,917][123582] Updated weights for policy 0, policy_version 10223 (0.0008) [2023-10-10 16:58:16,973][123614] Updated weights for policy 1, policy_version 10180 (0.0007) [2023-10-10 16:58:17,281][123582] Updated weights for policy 0, policy_version 10233 (0.0011) [2023-10-10 16:58:17,348][123614] Updated weights for policy 1, policy_version 10190 (0.0007) [2023-10-10 16:58:17,720][123614] Updated weights for policy 1, policy_version 10200 (0.0007) [2023-10-10 16:58:18,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 20938752. Throughput: 0: 1814.6, 1: 1812.0. Samples: 5238602. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:58:18,789][122664] Avg episode reward: [(0, '20.050'), (1, '21.820')] [2023-10-10 16:58:21,047][123582] Updated weights for policy 0, policy_version 10243 (0.0008) [2023-10-10 16:58:21,410][123614] Updated weights for policy 1, policy_version 10210 (0.0009) [2023-10-10 16:58:21,412][123582] Updated weights for policy 0, policy_version 10253 (0.0007) [2023-10-10 16:58:21,783][123614] Updated weights for policy 1, policy_version 10220 (0.0007) [2023-10-10 16:58:21,788][123582] Updated weights for policy 0, policy_version 10263 (0.0007) [2023-10-10 16:58:22,155][123614] Updated weights for policy 1, policy_version 10230 (0.0007) [2023-10-10 16:58:22,517][123614] Updated weights for policy 1, policy_version 10240 (0.0009) [2023-10-10 16:58:23,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 21004288. Throughput: 0: 1816.1, 1: 1801.3. Samples: 5260730. Policy #0 lag: (min: 10.0, avg: 10.1, max: 14.0) [2023-10-10 16:58:23,789][122664] Avg episode reward: [(0, '22.440'), (1, '21.140')] [2023-10-10 16:58:23,800][123247] Saving new best policy, reward=22.440! [2023-10-10 16:58:25,496][123582] Updated weights for policy 0, policy_version 10273 (0.0009) [2023-10-10 16:58:25,913][123582] Updated weights for policy 0, policy_version 10283 (0.0008) [2023-10-10 16:58:26,157][123614] Updated weights for policy 1, policy_version 10250 (0.0010) [2023-10-10 16:58:26,285][123582] Updated weights for policy 0, policy_version 10293 (0.0007) [2023-10-10 16:58:26,533][123614] Updated weights for policy 1, policy_version 10260 (0.0009) [2023-10-10 16:58:26,652][123582] Updated weights for policy 0, policy_version 10303 (0.0008) [2023-10-10 16:58:26,895][123614] Updated weights for policy 1, policy_version 10270 (0.0008) [2023-10-10 16:58:28,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 21069824. Throughput: 0: 1820.8, 1: 1804.0. Samples: 5271188. Policy #0 lag: (min: 10.0, avg: 10.1, max: 14.0) [2023-10-10 16:58:28,789][122664] Avg episode reward: [(0, '23.470'), (1, '20.270')] [2023-10-10 16:58:28,790][123247] Saving new best policy, reward=23.470! [2023-10-10 16:58:30,287][123582] Updated weights for policy 0, policy_version 10313 (0.0009) [2023-10-10 16:58:30,667][123582] Updated weights for policy 0, policy_version 10323 (0.0007) [2023-10-10 16:58:30,768][123614] Updated weights for policy 1, policy_version 10280 (0.0009) [2023-10-10 16:58:31,037][123582] Updated weights for policy 0, policy_version 10333 (0.0009) [2023-10-10 16:58:31,132][123614] Updated weights for policy 1, policy_version 10290 (0.0007) [2023-10-10 16:58:31,503][123614] Updated weights for policy 1, policy_version 10300 (0.0009) [2023-10-10 16:58:33,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 21135360. Throughput: 0: 1827.3, 1: 1797.3. Samples: 5293288. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:58:33,789][122664] Avg episode reward: [(0, '22.370'), (1, '21.010')] [2023-10-10 16:58:34,570][123582] Updated weights for policy 0, policy_version 10343 (0.0008) [2023-10-10 16:58:34,938][123582] Updated weights for policy 0, policy_version 10353 (0.0007) [2023-10-10 16:58:35,086][123614] Updated weights for policy 1, policy_version 10310 (0.0008) [2023-10-10 16:58:35,319][123582] Updated weights for policy 0, policy_version 10363 (0.0008) [2023-10-10 16:58:35,471][123614] Updated weights for policy 1, policy_version 10320 (0.0007) [2023-10-10 16:58:35,827][123614] Updated weights for policy 1, policy_version 10330 (0.0010) [2023-10-10 16:58:38,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 21200896. Throughput: 0: 1820.6, 1: 1812.6. Samples: 5316224. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:58:38,789][122664] Avg episode reward: [(0, '20.720'), (1, '20.370')] [2023-10-10 16:58:39,075][123582] Updated weights for policy 0, policy_version 10373 (0.0007) [2023-10-10 16:58:39,412][123614] Updated weights for policy 1, policy_version 10340 (0.0008) [2023-10-10 16:58:39,441][123582] Updated weights for policy 0, policy_version 10383 (0.0010) [2023-10-10 16:58:39,772][123614] Updated weights for policy 1, policy_version 10350 (0.0007) [2023-10-10 16:58:39,823][123582] Updated weights for policy 0, policy_version 10393 (0.0011) [2023-10-10 16:58:40,138][123614] Updated weights for policy 1, policy_version 10360 (0.0008) [2023-10-10 16:58:43,535][123582] Updated weights for policy 0, policy_version 10403 (0.0009) [2023-10-10 16:58:43,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 21266432. Throughput: 0: 1819.8, 1: 1809.7. Samples: 5326100. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:58:43,789][122664] Avg episode reward: [(0, '18.660'), (1, '18.950')] [2023-10-10 16:58:43,838][123614] Updated weights for policy 1, policy_version 10370 (0.0008) [2023-10-10 16:58:43,900][123582] Updated weights for policy 0, policy_version 10413 (0.0007) [2023-10-10 16:58:44,200][123614] Updated weights for policy 1, policy_version 10380 (0.0007) [2023-10-10 16:58:44,271][123582] Updated weights for policy 0, policy_version 10423 (0.0007) [2023-10-10 16:58:44,569][123614] Updated weights for policy 1, policy_version 10390 (0.0009) [2023-10-10 16:58:44,943][123614] Updated weights for policy 1, policy_version 10400 (0.0008) [2023-10-10 16:58:47,956][123582] Updated weights for policy 0, policy_version 10433 (0.0009) [2023-10-10 16:58:48,323][123582] Updated weights for policy 0, policy_version 10443 (0.0010) [2023-10-10 16:58:48,697][123582] Updated weights for policy 0, policy_version 10453 (0.0008) [2023-10-10 16:58:48,705][123614] Updated weights for policy 1, policy_version 10410 (0.0007) [2023-10-10 16:58:48,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.2). Total num frames: 21331968. Throughput: 0: 1814.5, 1: 1811.8. Samples: 5348806. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:58:48,788][122664] Avg episode reward: [(0, '17.050'), (1, '20.190')] [2023-10-10 16:58:49,070][123582] Updated weights for policy 0, policy_version 10463 (0.0008) [2023-10-10 16:58:49,075][123614] Updated weights for policy 1, policy_version 10420 (0.0007) [2023-10-10 16:58:49,445][123614] Updated weights for policy 1, policy_version 10430 (0.0008) [2023-10-10 16:58:52,929][123582] Updated weights for policy 0, policy_version 10473 (0.0008) [2023-10-10 16:58:53,178][123614] Updated weights for policy 1, policy_version 10440 (0.0008) [2023-10-10 16:58:53,293][123582] Updated weights for policy 0, policy_version 10483 (0.0008) [2023-10-10 16:58:53,546][123614] Updated weights for policy 1, policy_version 10450 (0.0007) [2023-10-10 16:58:53,664][123582] Updated weights for policy 0, policy_version 10493 (0.0008) [2023-10-10 16:58:53,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 21430272. Throughput: 0: 1810.0, 1: 1814.2. Samples: 5369242. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:58:53,789][122664] Avg episode reward: [(0, '16.130'), (1, '18.150')] [2023-10-10 16:58:53,913][123614] Updated weights for policy 1, policy_version 10460 (0.0009) [2023-10-10 16:58:57,441][123582] Updated weights for policy 0, policy_version 10503 (0.0008) [2023-10-10 16:58:57,639][123614] Updated weights for policy 1, policy_version 10470 (0.0010) [2023-10-10 16:58:57,816][123582] Updated weights for policy 0, policy_version 10513 (0.0007) [2023-10-10 16:58:58,014][123614] Updated weights for policy 1, policy_version 10480 (0.0008) [2023-10-10 16:58:58,196][123582] Updated weights for policy 0, policy_version 10523 (0.0007) [2023-10-10 16:58:58,377][123614] Updated weights for policy 1, policy_version 10490 (0.0008) [2023-10-10 16:58:58,788][122664] Fps is (10 sec: 19660.6, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 21528576. Throughput: 0: 1803.4, 1: 1812.2. Samples: 5381032. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 16:58:58,789][122664] Avg episode reward: [(0, '17.510'), (1, '19.460')] [2023-10-10 16:59:01,877][123582] Updated weights for policy 0, policy_version 10533 (0.0008) [2023-10-10 16:59:02,192][123614] Updated weights for policy 1, policy_version 10500 (0.0008) [2023-10-10 16:59:02,246][123582] Updated weights for policy 0, policy_version 10543 (0.0008) [2023-10-10 16:59:02,558][123614] Updated weights for policy 1, policy_version 10510 (0.0008) [2023-10-10 16:59:02,622][123582] Updated weights for policy 0, policy_version 10553 (0.0007) [2023-10-10 16:59:02,927][123614] Updated weights for policy 1, policy_version 10520 (0.0009) [2023-10-10 16:59:03,788][122664] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 21594112. Throughput: 0: 1811.7, 1: 1821.7. Samples: 5402108. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 16:59:03,789][122664] Avg episode reward: [(0, '18.180'), (1, '21.710')] [2023-10-10 16:59:06,339][123582] Updated weights for policy 0, policy_version 10563 (0.0007) [2023-10-10 16:59:06,537][123614] Updated weights for policy 1, policy_version 10530 (0.0009) [2023-10-10 16:59:06,713][123582] Updated weights for policy 0, policy_version 10573 (0.0008) [2023-10-10 16:59:06,904][123614] Updated weights for policy 1, policy_version 10540 (0.0007) [2023-10-10 16:59:07,083][123582] Updated weights for policy 0, policy_version 10583 (0.0008) [2023-10-10 16:59:07,271][123614] Updated weights for policy 1, policy_version 10550 (0.0007) [2023-10-10 16:59:07,636][123614] Updated weights for policy 1, policy_version 10560 (0.0007) [2023-10-10 16:59:08,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 21659648. Throughput: 0: 1801.4, 1: 1819.2. Samples: 5423656. Policy #0 lag: (min: 25.0, avg: 34.4, max: 57.0) [2023-10-10 16:59:08,789][122664] Avg episode reward: [(0, '19.640'), (1, '20.870')] [2023-10-10 16:59:10,804][123582] Updated weights for policy 0, policy_version 10593 (0.0009) [2023-10-10 16:59:11,208][123582] Updated weights for policy 0, policy_version 10603 (0.0009) [2023-10-10 16:59:11,418][123614] Updated weights for policy 1, policy_version 10570 (0.0007) [2023-10-10 16:59:11,573][123582] Updated weights for policy 0, policy_version 10613 (0.0010) [2023-10-10 16:59:11,784][123614] Updated weights for policy 1, policy_version 10580 (0.0007) [2023-10-10 16:59:11,949][123582] Updated weights for policy 0, policy_version 10623 (0.0008) [2023-10-10 16:59:12,152][123614] Updated weights for policy 1, policy_version 10590 (0.0008) [2023-10-10 16:59:13,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 21725184. Throughput: 0: 1808.4, 1: 1826.2. Samples: 5434744. Policy #0 lag: (min: 25.0, avg: 34.4, max: 57.0) [2023-10-10 16:59:13,789][122664] Avg episode reward: [(0, '21.870'), (1, '20.960')] [2023-10-10 16:59:15,612][123582] Updated weights for policy 0, policy_version 10633 (0.0007) [2023-10-10 16:59:15,845][123614] Updated weights for policy 1, policy_version 10600 (0.0009) [2023-10-10 16:59:15,981][123582] Updated weights for policy 0, policy_version 10643 (0.0007) [2023-10-10 16:59:16,208][123614] Updated weights for policy 1, policy_version 10610 (0.0009) [2023-10-10 16:59:16,356][123582] Updated weights for policy 0, policy_version 10653 (0.0007) [2023-10-10 16:59:16,581][123614] Updated weights for policy 1, policy_version 10620 (0.0007) [2023-10-10 16:59:18,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 21790720. Throughput: 0: 1797.4, 1: 1821.1. Samples: 5456122. Policy #0 lag: (min: 25.0, avg: 34.4, max: 57.0) [2023-10-10 16:59:18,788][122664] Avg episode reward: [(0, '22.610'), (1, '20.380')] [2023-10-10 16:59:20,105][123582] Updated weights for policy 0, policy_version 10663 (0.0009) [2023-10-10 16:59:20,460][123614] Updated weights for policy 1, policy_version 10630 (0.0007) [2023-10-10 16:59:20,482][123582] Updated weights for policy 0, policy_version 10673 (0.0007) [2023-10-10 16:59:20,838][123614] Updated weights for policy 1, policy_version 10640 (0.0008) [2023-10-10 16:59:20,847][123582] Updated weights for policy 0, policy_version 10683 (0.0009) [2023-10-10 16:59:21,214][123614] Updated weights for policy 1, policy_version 10650 (0.0008) [2023-10-10 16:59:23,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 21856256. Throughput: 0: 1799.5, 1: 1808.2. Samples: 5478568. Policy #0 lag: (min: 27.0, avg: 27.3, max: 39.0) [2023-10-10 16:59:23,788][122664] Avg episode reward: [(0, '21.830'), (1, '22.780')] [2023-10-10 16:59:23,800][123465] Saving new best policy, reward=22.780! [2023-10-10 16:59:24,541][123582] Updated weights for policy 0, policy_version 10693 (0.0008) [2023-10-10 16:59:24,901][123614] Updated weights for policy 1, policy_version 10660 (0.0008) [2023-10-10 16:59:24,918][123582] Updated weights for policy 0, policy_version 10703 (0.0008) [2023-10-10 16:59:25,282][123614] Updated weights for policy 1, policy_version 10670 (0.0007) [2023-10-10 16:59:25,298][123582] Updated weights for policy 0, policy_version 10713 (0.0009) [2023-10-10 16:59:25,653][123614] Updated weights for policy 1, policy_version 10680 (0.0008) [2023-10-10 16:59:28,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 21921792. Throughput: 0: 1797.5, 1: 1809.2. Samples: 5488402. Policy #0 lag: (min: 27.0, avg: 27.3, max: 39.0) [2023-10-10 16:59:28,789][122664] Avg episode reward: [(0, '19.540'), (1, '23.790')] [2023-10-10 16:59:28,790][123465] Saving new best policy, reward=23.790! [2023-10-10 16:59:29,119][123582] Updated weights for policy 0, policy_version 10723 (0.0007) [2023-10-10 16:59:29,286][123614] Updated weights for policy 1, policy_version 10690 (0.0008) [2023-10-10 16:59:29,497][123582] Updated weights for policy 0, policy_version 10733 (0.0008) [2023-10-10 16:59:29,646][123614] Updated weights for policy 1, policy_version 10700 (0.0009) [2023-10-10 16:59:29,858][123582] Updated weights for policy 0, policy_version 10743 (0.0008) [2023-10-10 16:59:30,018][123614] Updated weights for policy 1, policy_version 10710 (0.0009) [2023-10-10 16:59:30,383][123614] Updated weights for policy 1, policy_version 10720 (0.0009) [2023-10-10 16:59:33,530][123582] Updated weights for policy 0, policy_version 10753 (0.0008) [2023-10-10 16:59:33,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 21987328. Throughput: 0: 1793.7, 1: 1807.7. Samples: 5510870. Policy #0 lag: (min: 27.0, avg: 27.3, max: 39.0) [2023-10-10 16:59:33,788][122664] Avg episode reward: [(0, '19.340'), (1, '24.030')] [2023-10-10 16:59:33,894][123582] Updated weights for policy 0, policy_version 10763 (0.0008) [2023-10-10 16:59:34,106][123614] Updated weights for policy 1, policy_version 10730 (0.0007) [2023-10-10 16:59:34,261][123582] Updated weights for policy 0, policy_version 10773 (0.0009) [2023-10-10 16:59:34,471][123614] Updated weights for policy 1, policy_version 10740 (0.0010) [2023-10-10 16:59:34,641][123582] Updated weights for policy 0, policy_version 10783 (0.0010) [2023-10-10 16:59:34,841][123614] Updated weights for policy 1, policy_version 10750 (0.0009) [2023-10-10 16:59:34,915][123465] Saving new best policy, reward=24.030! [2023-10-10 16:59:38,379][123582] Updated weights for policy 0, policy_version 10793 (0.0008) [2023-10-10 16:59:38,520][123614] Updated weights for policy 1, policy_version 10760 (0.0010) [2023-10-10 16:59:38,754][123582] Updated weights for policy 0, policy_version 10803 (0.0008) [2023-10-10 16:59:38,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 22052864. Throughput: 0: 1812.6, 1: 1814.8. Samples: 5532472. Policy #0 lag: (min: 5.0, avg: 9.2, max: 37.0) [2023-10-10 16:59:38,789][122664] Avg episode reward: [(0, '19.040'), (1, '24.110')] [2023-10-10 16:59:38,892][123614] Updated weights for policy 1, policy_version 10770 (0.0009) [2023-10-10 16:59:39,123][123582] Updated weights for policy 0, policy_version 10813 (0.0008) [2023-10-10 16:59:39,259][123614] Updated weights for policy 1, policy_version 10780 (0.0007) [2023-10-10 16:59:39,406][123465] Saving new best policy, reward=24.110! [2023-10-10 16:59:42,822][123614] Updated weights for policy 1, policy_version 10790 (0.0008) [2023-10-10 16:59:42,910][123582] Updated weights for policy 0, policy_version 10823 (0.0010) [2023-10-10 16:59:43,185][123614] Updated weights for policy 1, policy_version 10800 (0.0008) [2023-10-10 16:59:43,283][123582] Updated weights for policy 0, policy_version 10833 (0.0007) [2023-10-10 16:59:43,556][123614] Updated weights for policy 1, policy_version 10810 (0.0008) [2023-10-10 16:59:43,650][123582] Updated weights for policy 0, policy_version 10843 (0.0008) [2023-10-10 16:59:43,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 22151168. Throughput: 0: 1797.4, 1: 1809.4. Samples: 5543336. Policy #0 lag: (min: 5.0, avg: 9.2, max: 37.0) [2023-10-10 16:59:43,788][122664] Avg episode reward: [(0, '18.480'), (1, '22.740')] [2023-10-10 16:59:47,163][123614] Updated weights for policy 1, policy_version 10820 (0.0008) [2023-10-10 16:59:47,370][123582] Updated weights for policy 0, policy_version 10853 (0.0009) [2023-10-10 16:59:47,535][123614] Updated weights for policy 1, policy_version 10830 (0.0007) [2023-10-10 16:59:47,749][123582] Updated weights for policy 0, policy_version 10863 (0.0008) [2023-10-10 16:59:47,899][123614] Updated weights for policy 1, policy_version 10840 (0.0008) [2023-10-10 16:59:48,120][123582] Updated weights for policy 0, policy_version 10873 (0.0008) [2023-10-10 16:59:48,788][122664] Fps is (10 sec: 19661.1, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 22249472. Throughput: 0: 1807.3, 1: 1813.1. Samples: 5565024. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:59:48,788][122664] Avg episode reward: [(0, '19.050'), (1, '23.620')] [2023-10-10 16:59:51,529][123614] Updated weights for policy 1, policy_version 10850 (0.0009) [2023-10-10 16:59:51,894][123614] Updated weights for policy 1, policy_version 10860 (0.0008) [2023-10-10 16:59:51,935][123582] Updated weights for policy 0, policy_version 10883 (0.0009) [2023-10-10 16:59:52,266][123614] Updated weights for policy 1, policy_version 10870 (0.0008) [2023-10-10 16:59:52,309][123582] Updated weights for policy 0, policy_version 10893 (0.0007) [2023-10-10 16:59:52,630][123614] Updated weights for policy 1, policy_version 10880 (0.0008) [2023-10-10 16:59:52,673][123582] Updated weights for policy 0, policy_version 10903 (0.0008) [2023-10-10 16:59:53,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 22315008. Throughput: 0: 1788.7, 1: 1812.7. Samples: 5585720. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 16:59:53,789][122664] Avg episode reward: [(0, '18.910'), (1, '21.350')] [2023-10-10 16:59:53,799][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000010912_11173888.pth... [2023-10-10 16:59:53,800][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000010880_11141120.pth... [2023-10-10 16:59:53,828][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000009216_9437184.pth [2023-10-10 16:59:53,834][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000009184_9404416.pth [2023-10-10 16:59:56,351][123582] Updated weights for policy 0, policy_version 10913 (0.0007) [2023-10-10 16:59:56,360][123614] Updated weights for policy 1, policy_version 10890 (0.0008) [2023-10-10 16:59:56,734][123614] Updated weights for policy 1, policy_version 10900 (0.0007) [2023-10-10 16:59:56,758][123582] Updated weights for policy 0, policy_version 10923 (0.0008) [2023-10-10 16:59:57,104][123614] Updated weights for policy 1, policy_version 10910 (0.0008) [2023-10-10 16:59:57,124][123582] Updated weights for policy 0, policy_version 10933 (0.0008) [2023-10-10 16:59:57,496][123582] Updated weights for policy 0, policy_version 10943 (0.0009) [2023-10-10 16:59:58,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 22380544. Throughput: 0: 1807.4, 1: 1813.8. Samples: 5597696. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-10 16:59:58,788][122664] Avg episode reward: [(0, '18.590'), (1, '18.800')] [2023-10-10 17:00:00,829][123614] Updated weights for policy 1, policy_version 10920 (0.0008) [2023-10-10 17:00:01,025][123582] Updated weights for policy 0, policy_version 10953 (0.0008) [2023-10-10 17:00:01,196][123614] Updated weights for policy 1, policy_version 10930 (0.0008) [2023-10-10 17:00:01,396][123582] Updated weights for policy 0, policy_version 10963 (0.0008) [2023-10-10 17:00:01,559][123614] Updated weights for policy 1, policy_version 10940 (0.0007) [2023-10-10 17:00:01,764][123582] Updated weights for policy 0, policy_version 10973 (0.0008) [2023-10-10 17:00:03,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 22446080. Throughput: 0: 1795.5, 1: 1811.5. Samples: 5618438. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-10 17:00:03,789][122664] Avg episode reward: [(0, '17.340'), (1, '19.750')] [2023-10-10 17:00:05,407][123582] Updated weights for policy 0, policy_version 10983 (0.0008) [2023-10-10 17:00:05,426][123614] Updated weights for policy 1, policy_version 10950 (0.0008) [2023-10-10 17:00:05,778][123582] Updated weights for policy 0, policy_version 10993 (0.0008) [2023-10-10 17:00:05,816][123614] Updated weights for policy 1, policy_version 10960 (0.0009) [2023-10-10 17:00:06,160][123582] Updated weights for policy 0, policy_version 11003 (0.0010) [2023-10-10 17:00:06,189][123614] Updated weights for policy 1, policy_version 10970 (0.0007) [2023-10-10 17:00:08,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 22511616. Throughput: 0: 1800.0, 1: 1805.6. Samples: 5640822. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-10 17:00:08,789][122664] Avg episode reward: [(0, '16.890'), (1, '19.520')] [2023-10-10 17:00:09,813][123582] Updated weights for policy 0, policy_version 11013 (0.0009) [2023-10-10 17:00:09,997][123614] Updated weights for policy 1, policy_version 10980 (0.0008) [2023-10-10 17:00:10,175][123582] Updated weights for policy 0, policy_version 11023 (0.0009) [2023-10-10 17:00:10,374][123614] Updated weights for policy 1, policy_version 10990 (0.0008) [2023-10-10 17:00:10,542][123582] Updated weights for policy 0, policy_version 11033 (0.0008) [2023-10-10 17:00:10,741][123614] Updated weights for policy 1, policy_version 11000 (0.0009) [2023-10-10 17:00:13,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 22577152. Throughput: 0: 1800.8, 1: 1802.2. Samples: 5650538. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-10 17:00:13,789][122664] Avg episode reward: [(0, '17.550'), (1, '20.370')] [2023-10-10 17:00:14,194][123582] Updated weights for policy 0, policy_version 11043 (0.0009) [2023-10-10 17:00:14,480][123614] Updated weights for policy 1, policy_version 11010 (0.0007) [2023-10-10 17:00:14,567][123582] Updated weights for policy 0, policy_version 11053 (0.0008) [2023-10-10 17:00:14,850][123614] Updated weights for policy 1, policy_version 11020 (0.0008) [2023-10-10 17:00:14,931][123582] Updated weights for policy 0, policy_version 11063 (0.0009) [2023-10-10 17:00:15,215][123614] Updated weights for policy 1, policy_version 11030 (0.0007) [2023-10-10 17:00:15,588][123614] Updated weights for policy 1, policy_version 11040 (0.0008) [2023-10-10 17:00:18,769][123582] Updated weights for policy 0, policy_version 11073 (0.0009) [2023-10-10 17:00:18,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 22642688. Throughput: 0: 1803.3, 1: 1804.8. Samples: 5673236. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-10 17:00:18,789][122664] Avg episode reward: [(0, '16.850'), (1, '20.440')] [2023-10-10 17:00:19,138][123582] Updated weights for policy 0, policy_version 11083 (0.0008) [2023-10-10 17:00:19,293][123614] Updated weights for policy 1, policy_version 11050 (0.0007) [2023-10-10 17:00:19,509][123582] Updated weights for policy 0, policy_version 11093 (0.0008) [2023-10-10 17:00:19,651][123614] Updated weights for policy 1, policy_version 11060 (0.0008) [2023-10-10 17:00:19,875][123582] Updated weights for policy 0, policy_version 11103 (0.0008) [2023-10-10 17:00:20,022][123614] Updated weights for policy 1, policy_version 11070 (0.0009) [2023-10-10 17:00:23,630][123582] Updated weights for policy 0, policy_version 11113 (0.0007) [2023-10-10 17:00:23,717][123614] Updated weights for policy 1, policy_version 11080 (0.0009) [2023-10-10 17:00:23,788][122664] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 22708224. Throughput: 0: 1808.6, 1: 1814.1. Samples: 5695490. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-10 17:00:23,788][122664] Avg episode reward: [(0, '16.730'), (1, '22.460')] [2023-10-10 17:00:24,007][123582] Updated weights for policy 0, policy_version 11123 (0.0010) [2023-10-10 17:00:24,089][123614] Updated weights for policy 1, policy_version 11090 (0.0009) [2023-10-10 17:00:24,381][123582] Updated weights for policy 0, policy_version 11133 (0.0008) [2023-10-10 17:00:24,455][123614] Updated weights for policy 1, policy_version 11100 (0.0007) [2023-10-10 17:00:28,141][123614] Updated weights for policy 1, policy_version 11110 (0.0008) [2023-10-10 17:00:28,273][123582] Updated weights for policy 0, policy_version 11143 (0.0007) [2023-10-10 17:00:28,510][123614] Updated weights for policy 1, policy_version 11120 (0.0009) [2023-10-10 17:00:28,643][123582] Updated weights for policy 0, policy_version 11153 (0.0009) [2023-10-10 17:00:28,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 22773760. Throughput: 0: 1799.4, 1: 1807.1. Samples: 5705630. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:00:28,788][122664] Avg episode reward: [(0, '19.440'), (1, '21.660')] [2023-10-10 17:00:28,882][123614] Updated weights for policy 1, policy_version 11130 (0.0008) [2023-10-10 17:00:29,030][123582] Updated weights for policy 0, policy_version 11163 (0.0010) [2023-10-10 17:00:32,610][123614] Updated weights for policy 1, policy_version 11140 (0.0009) [2023-10-10 17:00:32,881][123582] Updated weights for policy 0, policy_version 11173 (0.0008) [2023-10-10 17:00:32,976][123614] Updated weights for policy 1, policy_version 11150 (0.0008) [2023-10-10 17:00:33,258][123582] Updated weights for policy 0, policy_version 11183 (0.0008) [2023-10-10 17:00:33,349][123614] Updated weights for policy 1, policy_version 11160 (0.0007) [2023-10-10 17:00:33,637][123582] Updated weights for policy 0, policy_version 11193 (0.0007) [2023-10-10 17:00:33,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 22872064. Throughput: 0: 1809.5, 1: 1810.4. Samples: 5727924. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:00:33,789][122664] Avg episode reward: [(0, '20.180'), (1, '21.060')] [2023-10-10 17:00:37,053][123614] Updated weights for policy 1, policy_version 11170 (0.0007) [2023-10-10 17:00:37,396][123582] Updated weights for policy 0, policy_version 11203 (0.0009) [2023-10-10 17:00:37,415][123614] Updated weights for policy 1, policy_version 11180 (0.0008) [2023-10-10 17:00:37,763][123582] Updated weights for policy 0, policy_version 11213 (0.0009) [2023-10-10 17:00:37,785][123614] Updated weights for policy 1, policy_version 11190 (0.0007) [2023-10-10 17:00:38,145][123614] Updated weights for policy 1, policy_version 11200 (0.0008) [2023-10-10 17:00:38,147][123582] Updated weights for policy 0, policy_version 11223 (0.0008) [2023-10-10 17:00:38,788][122664] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 22970368. Throughput: 0: 1804.8, 1: 1799.7. Samples: 5747924. Policy #0 lag: (min: 17.0, avg: 20.1, max: 49.0) [2023-10-10 17:00:38,789][122664] Avg episode reward: [(0, '20.060'), (1, '20.700')] [2023-10-10 17:00:41,836][123582] Updated weights for policy 0, policy_version 11233 (0.0010) [2023-10-10 17:00:41,898][123614] Updated weights for policy 1, policy_version 11210 (0.0007) [2023-10-10 17:00:42,249][123582] Updated weights for policy 0, policy_version 11243 (0.0007) [2023-10-10 17:00:42,261][123614] Updated weights for policy 1, policy_version 11220 (0.0008) [2023-10-10 17:00:42,623][123582] Updated weights for policy 0, policy_version 11253 (0.0007) [2023-10-10 17:00:42,633][123614] Updated weights for policy 1, policy_version 11230 (0.0010) [2023-10-10 17:00:42,999][123582] Updated weights for policy 0, policy_version 11263 (0.0009) [2023-10-10 17:00:43,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 23035904. Throughput: 0: 1802.2, 1: 1811.6. Samples: 5760320. Policy #0 lag: (min: 17.0, avg: 20.1, max: 49.0) [2023-10-10 17:00:43,789][122664] Avg episode reward: [(0, '22.760'), (1, '20.860')] [2023-10-10 17:00:46,247][123614] Updated weights for policy 1, policy_version 11240 (0.0008) [2023-10-10 17:00:46,616][123614] Updated weights for policy 1, policy_version 11250 (0.0007) [2023-10-10 17:00:46,658][123582] Updated weights for policy 0, policy_version 11273 (0.0008) [2023-10-10 17:00:46,976][123614] Updated weights for policy 1, policy_version 11260 (0.0008) [2023-10-10 17:00:47,029][123582] Updated weights for policy 0, policy_version 11283 (0.0008) [2023-10-10 17:00:47,399][123582] Updated weights for policy 0, policy_version 11293 (0.0011) [2023-10-10 17:00:48,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 23101440. Throughput: 0: 1797.5, 1: 1803.3. Samples: 5780474. Policy #0 lag: (min: 17.0, avg: 20.1, max: 49.0) [2023-10-10 17:00:48,788][122664] Avg episode reward: [(0, '22.360'), (1, '22.490')] [2023-10-10 17:00:50,876][123614] Updated weights for policy 1, policy_version 11270 (0.0008) [2023-10-10 17:00:51,124][123582] Updated weights for policy 0, policy_version 11303 (0.0008) [2023-10-10 17:00:51,253][123614] Updated weights for policy 1, policy_version 11280 (0.0009) [2023-10-10 17:00:51,505][123582] Updated weights for policy 0, policy_version 11313 (0.0009) [2023-10-10 17:00:51,621][123614] Updated weights for policy 1, policy_version 11290 (0.0009) [2023-10-10 17:00:51,870][123582] Updated weights for policy 0, policy_version 11323 (0.0010) [2023-10-10 17:00:53,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 23166976. Throughput: 0: 1788.9, 1: 1808.7. Samples: 5802714. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-10 17:00:53,789][122664] Avg episode reward: [(0, '23.550'), (1, '20.550')] [2023-10-10 17:00:53,801][123247] Saving new best policy, reward=23.550! [2023-10-10 17:00:55,275][123614] Updated weights for policy 1, policy_version 11300 (0.0008) [2023-10-10 17:00:55,604][123582] Updated weights for policy 0, policy_version 11333 (0.0010) [2023-10-10 17:00:55,640][123614] Updated weights for policy 1, policy_version 11310 (0.0007) [2023-10-10 17:00:55,970][123582] Updated weights for policy 0, policy_version 11343 (0.0008) [2023-10-10 17:00:56,006][123614] Updated weights for policy 1, policy_version 11320 (0.0007) [2023-10-10 17:00:56,342][123582] Updated weights for policy 0, policy_version 11353 (0.0007) [2023-10-10 17:00:58,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 23232512. Throughput: 0: 1795.0, 1: 1808.6. Samples: 5812700. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-10 17:00:58,789][122664] Avg episode reward: [(0, '21.610'), (1, '20.170')] [2023-10-10 17:00:59,851][123614] Updated weights for policy 1, policy_version 11330 (0.0008) [2023-10-10 17:01:00,139][123582] Updated weights for policy 0, policy_version 11363 (0.0008) [2023-10-10 17:01:00,221][123614] Updated weights for policy 1, policy_version 11340 (0.0008) [2023-10-10 17:01:00,508][123582] Updated weights for policy 0, policy_version 11373 (0.0009) [2023-10-10 17:01:00,595][123614] Updated weights for policy 1, policy_version 11350 (0.0008) [2023-10-10 17:01:00,872][123582] Updated weights for policy 0, policy_version 11383 (0.0009) [2023-10-10 17:01:00,958][123614] Updated weights for policy 1, policy_version 11360 (0.0008) [2023-10-10 17:01:03,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 23298048. Throughput: 0: 1785.7, 1: 1803.2. Samples: 5834736. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-10 17:01:03,789][122664] Avg episode reward: [(0, '21.160'), (1, '22.460')] [2023-10-10 17:01:04,587][123582] Updated weights for policy 0, policy_version 11393 (0.0010) [2023-10-10 17:01:04,639][123614] Updated weights for policy 1, policy_version 11370 (0.0008) [2023-10-10 17:01:04,944][123582] Updated weights for policy 0, policy_version 11403 (0.0008) [2023-10-10 17:01:05,012][123614] Updated weights for policy 1, policy_version 11380 (0.0007) [2023-10-10 17:01:05,319][123582] Updated weights for policy 0, policy_version 11413 (0.0008) [2023-10-10 17:01:05,382][123614] Updated weights for policy 1, policy_version 11390 (0.0007) [2023-10-10 17:01:05,685][123582] Updated weights for policy 0, policy_version 11423 (0.0009) [2023-10-10 17:01:08,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 23363584. Throughput: 0: 1791.1, 1: 1809.6. Samples: 5857524. Policy #0 lag: (min: 24.0, avg: 47.0, max: 56.0) [2023-10-10 17:01:08,788][122664] Avg episode reward: [(0, '19.200'), (1, '20.750')] [2023-10-10 17:01:09,079][123614] Updated weights for policy 1, policy_version 11400 (0.0009) [2023-10-10 17:01:09,412][123582] Updated weights for policy 0, policy_version 11433 (0.0009) [2023-10-10 17:01:09,446][123614] Updated weights for policy 1, policy_version 11410 (0.0008) [2023-10-10 17:01:09,780][123582] Updated weights for policy 0, policy_version 11443 (0.0008) [2023-10-10 17:01:09,815][123614] Updated weights for policy 1, policy_version 11420 (0.0008) [2023-10-10 17:01:10,145][123582] Updated weights for policy 0, policy_version 11453 (0.0010) [2023-10-10 17:01:13,471][123614] Updated weights for policy 1, policy_version 11430 (0.0008) [2023-10-10 17:01:13,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 23429120. Throughput: 0: 1790.8, 1: 1804.2. Samples: 5867406. Policy #0 lag: (min: 24.0, avg: 47.0, max: 56.0) [2023-10-10 17:01:13,788][122664] Avg episode reward: [(0, '19.710'), (1, '21.220')] [2023-10-10 17:01:13,836][123614] Updated weights for policy 1, policy_version 11440 (0.0007) [2023-10-10 17:01:13,875][123582] Updated weights for policy 0, policy_version 11463 (0.0008) [2023-10-10 17:01:14,212][123614] Updated weights for policy 1, policy_version 11450 (0.0007) [2023-10-10 17:01:14,248][123582] Updated weights for policy 0, policy_version 11473 (0.0007) [2023-10-10 17:01:14,615][123582] Updated weights for policy 0, policy_version 11483 (0.0008) [2023-10-10 17:01:17,937][123614] Updated weights for policy 1, policy_version 11460 (0.0009) [2023-10-10 17:01:18,310][123614] Updated weights for policy 1, policy_version 11470 (0.0009) [2023-10-10 17:01:18,344][123582] Updated weights for policy 0, policy_version 11493 (0.0010) [2023-10-10 17:01:18,678][123614] Updated weights for policy 1, policy_version 11480 (0.0007) [2023-10-10 17:01:18,713][123582] Updated weights for policy 0, policy_version 11503 (0.0010) [2023-10-10 17:01:18,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 23494656. Throughput: 0: 1792.6, 1: 1810.6. Samples: 5890066. Policy #0 lag: (min: 24.0, avg: 47.0, max: 56.0) [2023-10-10 17:01:18,789][122664] Avg episode reward: [(0, '19.160'), (1, '20.990')] [2023-10-10 17:01:19,083][123582] Updated weights for policy 0, policy_version 11513 (0.0008) [2023-10-10 17:01:22,305][123614] Updated weights for policy 1, policy_version 11490 (0.0008) [2023-10-10 17:01:22,675][123614] Updated weights for policy 1, policy_version 11500 (0.0007) [2023-10-10 17:01:22,723][123582] Updated weights for policy 0, policy_version 11523 (0.0007) [2023-10-10 17:01:23,044][123614] Updated weights for policy 1, policy_version 11510 (0.0007) [2023-10-10 17:01:23,080][123582] Updated weights for policy 0, policy_version 11533 (0.0008) [2023-10-10 17:01:23,422][123614] Updated weights for policy 1, policy_version 11520 (0.0007) [2023-10-10 17:01:23,458][123582] Updated weights for policy 0, policy_version 11543 (0.0010) [2023-10-10 17:01:23,788][122664] Fps is (10 sec: 19660.0, 60 sec: 15291.6, 300 sec: 14662.3). Total num frames: 23625728. Throughput: 0: 1806.1, 1: 1805.7. Samples: 5910454. Policy #0 lag: (min: 4.0, avg: 12.0, max: 36.0) [2023-10-10 17:01:23,789][122664] Avg episode reward: [(0, '18.740'), (1, '21.000')] [2023-10-10 17:01:27,090][123614] Updated weights for policy 1, policy_version 11530 (0.0009) [2023-10-10 17:01:27,356][123582] Updated weights for policy 0, policy_version 11553 (0.0008) [2023-10-10 17:01:27,463][123614] Updated weights for policy 1, policy_version 11540 (0.0008) [2023-10-10 17:01:27,768][123582] Updated weights for policy 0, policy_version 11563 (0.0008) [2023-10-10 17:01:27,830][123614] Updated weights for policy 1, policy_version 11550 (0.0008) [2023-10-10 17:01:28,139][123582] Updated weights for policy 0, policy_version 11573 (0.0010) [2023-10-10 17:01:28,508][123582] Updated weights for policy 0, policy_version 11583 (0.0010) [2023-10-10 17:01:28,788][122664] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 23691264. Throughput: 0: 1797.2, 1: 1810.4. Samples: 5922658. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:01:28,789][122664] Avg episode reward: [(0, '20.440'), (1, '20.680')] [2023-10-10 17:01:31,639][123614] Updated weights for policy 1, policy_version 11560 (0.0010) [2023-10-10 17:01:32,000][123614] Updated weights for policy 1, policy_version 11570 (0.0007) [2023-10-10 17:01:32,211][123582] Updated weights for policy 0, policy_version 11593 (0.0007) [2023-10-10 17:01:32,369][123614] Updated weights for policy 1, policy_version 11580 (0.0010) [2023-10-10 17:01:32,573][123582] Updated weights for policy 0, policy_version 11603 (0.0007) [2023-10-10 17:01:32,947][123582] Updated weights for policy 0, policy_version 11613 (0.0008) [2023-10-10 17:01:33,788][122664] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 23756800. Throughput: 0: 1809.1, 1: 1798.3. Samples: 5942806. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:01:33,789][122664] Avg episode reward: [(0, '18.780'), (1, '21.190')] [2023-10-10 17:01:36,063][123614] Updated weights for policy 1, policy_version 11590 (0.0008) [2023-10-10 17:01:36,437][123614] Updated weights for policy 1, policy_version 11600 (0.0009) [2023-10-10 17:01:36,598][123582] Updated weights for policy 0, policy_version 11623 (0.0009) [2023-10-10 17:01:36,801][123614] Updated weights for policy 1, policy_version 11610 (0.0008) [2023-10-10 17:01:36,967][123582] Updated weights for policy 0, policy_version 11633 (0.0009) [2023-10-10 17:01:37,346][123582] Updated weights for policy 0, policy_version 11643 (0.0009) [2023-10-10 17:01:38,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 23822336. Throughput: 0: 1798.1, 1: 1799.9. Samples: 5964624. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:01:38,789][122664] Avg episode reward: [(0, '19.190'), (1, '20.240')] [2023-10-10 17:01:40,466][123614] Updated weights for policy 1, policy_version 11620 (0.0010) [2023-10-10 17:01:40,832][123614] Updated weights for policy 1, policy_version 11630 (0.0008) [2023-10-10 17:01:40,966][123582] Updated weights for policy 0, policy_version 11653 (0.0008) [2023-10-10 17:01:41,203][123614] Updated weights for policy 1, policy_version 11640 (0.0008) [2023-10-10 17:01:41,329][123582] Updated weights for policy 0, policy_version 11663 (0.0009) [2023-10-10 17:01:41,697][123582] Updated weights for policy 0, policy_version 11673 (0.0010) [2023-10-10 17:01:43,788][122664] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 23887872. Throughput: 0: 1813.6, 1: 1804.1. Samples: 5975496. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:01:43,789][122664] Avg episode reward: [(0, '19.500'), (1, '21.130')] [2023-10-10 17:01:45,043][123614] Updated weights for policy 1, policy_version 11650 (0.0009) [2023-10-10 17:01:45,282][123582] Updated weights for policy 0, policy_version 11683 (0.0009) [2023-10-10 17:01:45,408][123614] Updated weights for policy 1, policy_version 11660 (0.0008) [2023-10-10 17:01:45,652][123582] Updated weights for policy 0, policy_version 11693 (0.0009) [2023-10-10 17:01:45,779][123614] Updated weights for policy 1, policy_version 11670 (0.0007) [2023-10-10 17:01:46,020][123582] Updated weights for policy 0, policy_version 11703 (0.0008) [2023-10-10 17:01:46,144][123614] Updated weights for policy 1, policy_version 11680 (0.0007) [2023-10-10 17:01:48,788][122664] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 23953408. Throughput: 0: 1814.2, 1: 1809.7. Samples: 5997810. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:01:48,788][122664] Avg episode reward: [(0, '19.840'), (1, '23.410')] [2023-10-10 17:01:49,629][123582] Updated weights for policy 0, policy_version 11713 (0.0007) [2023-10-10 17:01:49,898][123614] Updated weights for policy 1, policy_version 11690 (0.0009) [2023-10-10 17:01:50,002][123582] Updated weights for policy 0, policy_version 11723 (0.0007) [2023-10-10 17:01:50,260][123614] Updated weights for policy 1, policy_version 11700 (0.0008) [2023-10-10 17:01:50,380][123582] Updated weights for policy 0, policy_version 11733 (0.0008) [2023-10-10 17:01:50,624][123614] Updated weights for policy 1, policy_version 11710 (0.0008) [2023-10-10 17:01:50,756][123582] Updated weights for policy 0, policy_version 11743 (0.0008) [2023-10-10 17:01:53,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 24018944. Throughput: 0: 1813.9, 1: 1815.8. Samples: 6020864. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:01:53,789][122664] Avg episode reward: [(0, '16.530'), (1, '24.440')] [2023-10-10 17:01:53,803][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000011744_12025856.pth... [2023-10-10 17:01:53,804][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000011712_11993088.pth... [2023-10-10 17:01:53,836][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000010080_10321920.pth [2023-10-10 17:01:53,841][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000010016_10256384.pth [2023-10-10 17:01:53,844][123465] Saving new best policy, reward=24.440! [2023-10-10 17:01:54,331][123614] Updated weights for policy 1, policy_version 11720 (0.0007) [2023-10-10 17:01:54,528][123582] Updated weights for policy 0, policy_version 11753 (0.0010) [2023-10-10 17:01:54,701][123614] Updated weights for policy 1, policy_version 11730 (0.0007) [2023-10-10 17:01:54,895][123582] Updated weights for policy 0, policy_version 11763 (0.0008) [2023-10-10 17:01:55,063][123614] Updated weights for policy 1, policy_version 11740 (0.0007) [2023-10-10 17:01:55,266][123582] Updated weights for policy 0, policy_version 11773 (0.0007) [2023-10-10 17:01:58,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 24084480. Throughput: 0: 1819.9, 1: 1817.9. Samples: 6031106. Policy #0 lag: (min: 9.0, avg: 21.8, max: 41.0) [2023-10-10 17:01:58,789][122664] Avg episode reward: [(0, '18.640'), (1, '23.820')] [2023-10-10 17:01:58,882][123614] Updated weights for policy 1, policy_version 11750 (0.0007) [2023-10-10 17:01:58,954][123582] Updated weights for policy 0, policy_version 11783 (0.0008) [2023-10-10 17:01:59,256][123614] Updated weights for policy 1, policy_version 11760 (0.0009) [2023-10-10 17:01:59,324][123582] Updated weights for policy 0, policy_version 11793 (0.0008) [2023-10-10 17:01:59,629][123614] Updated weights for policy 1, policy_version 11770 (0.0009) [2023-10-10 17:01:59,693][123582] Updated weights for policy 0, policy_version 11803 (0.0007) [2023-10-10 17:02:03,298][123582] Updated weights for policy 0, policy_version 11813 (0.0008) [2023-10-10 17:02:03,337][123614] Updated weights for policy 1, policy_version 11780 (0.0007) [2023-10-10 17:02:03,659][123582] Updated weights for policy 0, policy_version 11823 (0.0008) [2023-10-10 17:02:03,702][123614] Updated weights for policy 1, policy_version 11790 (0.0007) [2023-10-10 17:02:03,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 24150016. Throughput: 0: 1824.7, 1: 1810.7. Samples: 6053658. Policy #0 lag: (min: 9.0, avg: 21.8, max: 41.0) [2023-10-10 17:02:03,788][122664] Avg episode reward: [(0, '19.440'), (1, '23.320')] [2023-10-10 17:02:04,033][123582] Updated weights for policy 0, policy_version 11833 (0.0009) [2023-10-10 17:02:04,078][123614] Updated weights for policy 1, policy_version 11800 (0.0007) [2023-10-10 17:02:07,798][123614] Updated weights for policy 1, policy_version 11810 (0.0007) [2023-10-10 17:02:07,927][123582] Updated weights for policy 0, policy_version 11843 (0.0009) [2023-10-10 17:02:08,162][123614] Updated weights for policy 1, policy_version 11820 (0.0009) [2023-10-10 17:02:08,297][123582] Updated weights for policy 0, policy_version 11853 (0.0007) [2023-10-10 17:02:08,535][123614] Updated weights for policy 1, policy_version 11830 (0.0008) [2023-10-10 17:02:08,674][123582] Updated weights for policy 0, policy_version 11863 (0.0007) [2023-10-10 17:02:08,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 24215552. Throughput: 0: 1822.9, 1: 1814.9. Samples: 6074156. Policy #0 lag: (min: 9.0, avg: 21.8, max: 41.0) [2023-10-10 17:02:08,789][122664] Avg episode reward: [(0, '19.470'), (1, '24.190')] [2023-10-10 17:02:08,900][123614] Updated weights for policy 1, policy_version 11840 (0.0008) [2023-10-10 17:02:12,292][123582] Updated weights for policy 0, policy_version 11873 (0.0009) [2023-10-10 17:02:12,533][123614] Updated weights for policy 1, policy_version 11850 (0.0008) [2023-10-10 17:02:12,685][123582] Updated weights for policy 0, policy_version 11883 (0.0007) [2023-10-10 17:02:12,895][123614] Updated weights for policy 1, policy_version 11860 (0.0008) [2023-10-10 17:02:13,044][123582] Updated weights for policy 0, policy_version 11893 (0.0008) [2023-10-10 17:02:13,260][123614] Updated weights for policy 1, policy_version 11870 (0.0007) [2023-10-10 17:02:13,421][123582] Updated weights for policy 0, policy_version 11903 (0.0009) [2023-10-10 17:02:13,788][122664] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 24346624. Throughput: 0: 1824.9, 1: 1809.1. Samples: 6086188. Policy #0 lag: (min: 26.0, avg: 28.3, max: 58.0) [2023-10-10 17:02:13,789][122664] Avg episode reward: [(0, '19.010'), (1, '25.900')] [2023-10-10 17:02:13,790][123465] Saving new best policy, reward=25.900! [2023-10-10 17:02:17,025][123582] Updated weights for policy 0, policy_version 11913 (0.0008) [2023-10-10 17:02:17,065][123614] Updated weights for policy 1, policy_version 11880 (0.0007) [2023-10-10 17:02:17,395][123582] Updated weights for policy 0, policy_version 11923 (0.0009) [2023-10-10 17:02:17,424][123614] Updated weights for policy 1, policy_version 11890 (0.0008) [2023-10-10 17:02:17,768][123582] Updated weights for policy 0, policy_version 11933 (0.0009) [2023-10-10 17:02:17,788][123614] Updated weights for policy 1, policy_version 11900 (0.0007) [2023-10-10 17:02:18,788][122664] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14440.2). Total num frames: 24412160. Throughput: 0: 1826.4, 1: 1816.8. Samples: 6106748. Policy #0 lag: (min: 26.0, avg: 28.3, max: 58.0) [2023-10-10 17:02:18,789][122664] Avg episode reward: [(0, '19.210'), (1, '25.800')] [2023-10-10 17:02:21,471][123582] Updated weights for policy 0, policy_version 11943 (0.0010) [2023-10-10 17:02:21,674][123614] Updated weights for policy 1, policy_version 11910 (0.0009) [2023-10-10 17:02:21,841][123582] Updated weights for policy 0, policy_version 11953 (0.0010) [2023-10-10 17:02:22,059][123614] Updated weights for policy 1, policy_version 11920 (0.0007) [2023-10-10 17:02:22,221][123582] Updated weights for policy 0, policy_version 11963 (0.0008) [2023-10-10 17:02:22,428][123614] Updated weights for policy 1, policy_version 11930 (0.0008) [2023-10-10 17:02:23,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 24477696. Throughput: 0: 1826.9, 1: 1803.8. Samples: 6128002. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-10 17:02:23,788][122664] Avg episode reward: [(0, '18.740'), (1, '22.790')] [2023-10-10 17:02:26,005][123582] Updated weights for policy 0, policy_version 11973 (0.0011) [2023-10-10 17:02:26,171][123614] Updated weights for policy 1, policy_version 11940 (0.0010) [2023-10-10 17:02:26,380][123582] Updated weights for policy 0, policy_version 11983 (0.0007) [2023-10-10 17:02:26,537][123614] Updated weights for policy 1, policy_version 11950 (0.0007) [2023-10-10 17:02:26,760][123582] Updated weights for policy 0, policy_version 11993 (0.0009) [2023-10-10 17:02:26,909][123614] Updated weights for policy 1, policy_version 11960 (0.0009) [2023-10-10 17:02:28,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 24543232. Throughput: 0: 1823.9, 1: 1814.0. Samples: 6139200. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-10 17:02:28,789][122664] Avg episode reward: [(0, '20.010'), (1, '22.160')] [2023-10-10 17:02:30,448][123582] Updated weights for policy 0, policy_version 12003 (0.0010) [2023-10-10 17:02:30,749][123614] Updated weights for policy 1, policy_version 11970 (0.0009) [2023-10-10 17:02:30,817][123582] Updated weights for policy 0, policy_version 12013 (0.0009) [2023-10-10 17:02:31,114][123614] Updated weights for policy 1, policy_version 11980 (0.0010) [2023-10-10 17:02:31,195][123582] Updated weights for policy 0, policy_version 12023 (0.0007) [2023-10-10 17:02:31,485][123614] Updated weights for policy 1, policy_version 11990 (0.0007) [2023-10-10 17:02:31,857][123614] Updated weights for policy 1, policy_version 12000 (0.0007) [2023-10-10 17:02:33,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 24608768. Throughput: 0: 1822.2, 1: 1791.6. Samples: 6160430. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-10 17:02:33,789][122664] Avg episode reward: [(0, '20.620'), (1, '22.690')] [2023-10-10 17:02:34,885][123582] Updated weights for policy 0, policy_version 12033 (0.0008) [2023-10-10 17:02:35,259][123582] Updated weights for policy 0, policy_version 12043 (0.0009) [2023-10-10 17:02:35,580][123614] Updated weights for policy 1, policy_version 12010 (0.0008) [2023-10-10 17:02:35,628][123582] Updated weights for policy 0, policy_version 12053 (0.0008) [2023-10-10 17:02:35,945][123614] Updated weights for policy 1, policy_version 12020 (0.0008) [2023-10-10 17:02:36,013][123582] Updated weights for policy 0, policy_version 12063 (0.0009) [2023-10-10 17:02:36,314][123614] Updated weights for policy 1, policy_version 12030 (0.0009) [2023-10-10 17:02:38,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 24674304. Throughput: 0: 1822.6, 1: 1776.1. Samples: 6182806. Policy #0 lag: (min: 13.0, avg: 20.5, max: 45.0) [2023-10-10 17:02:38,789][122664] Avg episode reward: [(0, '20.450'), (1, '22.210')] [2023-10-10 17:02:39,578][123582] Updated weights for policy 0, policy_version 12073 (0.0008) [2023-10-10 17:02:39,939][123582] Updated weights for policy 0, policy_version 12083 (0.0009) [2023-10-10 17:02:40,222][123614] Updated weights for policy 1, policy_version 12040 (0.0008) [2023-10-10 17:02:40,311][123582] Updated weights for policy 0, policy_version 12093 (0.0008) [2023-10-10 17:02:40,593][123614] Updated weights for policy 1, policy_version 12050 (0.0008) [2023-10-10 17:02:40,962][123614] Updated weights for policy 1, policy_version 12060 (0.0009) [2023-10-10 17:02:43,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 24739840. Throughput: 0: 1820.2, 1: 1776.0. Samples: 6192934. Policy #0 lag: (min: 13.0, avg: 20.5, max: 45.0) [2023-10-10 17:02:43,788][122664] Avg episode reward: [(0, '19.100'), (1, '21.190')] [2023-10-10 17:02:43,918][123582] Updated weights for policy 0, policy_version 12103 (0.0011) [2023-10-10 17:02:44,299][123582] Updated weights for policy 0, policy_version 12113 (0.0011) [2023-10-10 17:02:44,638][123614] Updated weights for policy 1, policy_version 12070 (0.0010) [2023-10-10 17:02:44,661][123582] Updated weights for policy 0, policy_version 12123 (0.0009) [2023-10-10 17:02:44,998][123614] Updated weights for policy 1, policy_version 12080 (0.0008) [2023-10-10 17:02:45,365][123614] Updated weights for policy 1, policy_version 12090 (0.0010) [2023-10-10 17:02:48,423][123582] Updated weights for policy 0, policy_version 12133 (0.0009) [2023-10-10 17:02:48,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 24805376. Throughput: 0: 1815.4, 1: 1790.5. Samples: 6215922. Policy #0 lag: (min: 13.0, avg: 20.5, max: 45.0) [2023-10-10 17:02:48,788][122664] Avg episode reward: [(0, '19.570'), (1, '20.850')] [2023-10-10 17:02:48,796][123582] Updated weights for policy 0, policy_version 12143 (0.0008) [2023-10-10 17:02:49,003][123614] Updated weights for policy 1, policy_version 12100 (0.0008) [2023-10-10 17:02:49,176][123582] Updated weights for policy 0, policy_version 12153 (0.0008) [2023-10-10 17:02:49,373][123614] Updated weights for policy 1, policy_version 12110 (0.0007) [2023-10-10 17:02:49,740][123614] Updated weights for policy 1, policy_version 12120 (0.0008) [2023-10-10 17:02:52,743][123582] Updated weights for policy 0, policy_version 12163 (0.0008) [2023-10-10 17:02:53,111][123582] Updated weights for policy 0, policy_version 12173 (0.0009) [2023-10-10 17:02:53,406][123614] Updated weights for policy 1, policy_version 12130 (0.0008) [2023-10-10 17:02:53,478][123582] Updated weights for policy 0, policy_version 12183 (0.0010) [2023-10-10 17:02:53,776][123614] Updated weights for policy 1, policy_version 12140 (0.0007) [2023-10-10 17:02:53,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 24870912. Throughput: 0: 1821.8, 1: 1808.8. Samples: 6237532. Policy #0 lag: (min: 22.0, avg: 27.8, max: 54.0) [2023-10-10 17:02:53,788][122664] Avg episode reward: [(0, '18.220'), (1, '21.620')] [2023-10-10 17:02:54,153][123614] Updated weights for policy 1, policy_version 12150 (0.0010) [2023-10-10 17:02:54,522][123614] Updated weights for policy 1, policy_version 12160 (0.0012) [2023-10-10 17:02:57,121][123582] Updated weights for policy 0, policy_version 12193 (0.0009) [2023-10-10 17:02:57,534][123582] Updated weights for policy 0, policy_version 12203 (0.0009) [2023-10-10 17:02:57,904][123582] Updated weights for policy 0, policy_version 12213 (0.0007) [2023-10-10 17:02:58,236][123614] Updated weights for policy 1, policy_version 12170 (0.0008) [2023-10-10 17:02:58,276][123582] Updated weights for policy 0, policy_version 12223 (0.0007) [2023-10-10 17:02:58,608][123614] Updated weights for policy 1, policy_version 12180 (0.0007) [2023-10-10 17:02:58,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 24969216. Throughput: 0: 1823.9, 1: 1788.0. Samples: 6248720. Policy #0 lag: (min: 22.0, avg: 27.8, max: 54.0) [2023-10-10 17:02:58,788][122664] Avg episode reward: [(0, '19.640'), (1, '21.330')] [2023-10-10 17:02:58,973][123614] Updated weights for policy 1, policy_version 12190 (0.0009) [2023-10-10 17:03:02,109][123582] Updated weights for policy 0, policy_version 12233 (0.0007) [2023-10-10 17:03:02,487][123582] Updated weights for policy 0, policy_version 12243 (0.0008) [2023-10-10 17:03:02,731][123614] Updated weights for policy 1, policy_version 12200 (0.0008) [2023-10-10 17:03:02,852][123582] Updated weights for policy 0, policy_version 12253 (0.0007) [2023-10-10 17:03:03,099][123614] Updated weights for policy 1, policy_version 12210 (0.0009) [2023-10-10 17:03:03,474][123614] Updated weights for policy 1, policy_version 12220 (0.0009) [2023-10-10 17:03:03,788][122664] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 25067520. Throughput: 0: 1818.4, 1: 1812.4. Samples: 6270138. Policy #0 lag: (min: 8.0, avg: 32.9, max: 40.0) [2023-10-10 17:03:03,788][122664] Avg episode reward: [(0, '19.610'), (1, '21.800')] [2023-10-10 17:03:06,458][123582] Updated weights for policy 0, policy_version 12263 (0.0008) [2023-10-10 17:03:06,835][123582] Updated weights for policy 0, policy_version 12273 (0.0010) [2023-10-10 17:03:07,133][123614] Updated weights for policy 1, policy_version 12230 (0.0010) [2023-10-10 17:03:07,208][123582] Updated weights for policy 0, policy_version 12283 (0.0009) [2023-10-10 17:03:07,502][123614] Updated weights for policy 1, policy_version 12240 (0.0008) [2023-10-10 17:03:07,874][123614] Updated weights for policy 1, policy_version 12250 (0.0009) [2023-10-10 17:03:08,788][122664] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 25133056. Throughput: 0: 1827.9, 1: 1801.6. Samples: 6291330. Policy #0 lag: (min: 8.0, avg: 32.9, max: 40.0) [2023-10-10 17:03:08,789][122664] Avg episode reward: [(0, '22.130'), (1, '22.290')] [2023-10-10 17:03:10,909][123582] Updated weights for policy 0, policy_version 12293 (0.0009) [2023-10-10 17:03:11,288][123582] Updated weights for policy 0, policy_version 12303 (0.0007) [2023-10-10 17:03:11,611][123614] Updated weights for policy 1, policy_version 12260 (0.0008) [2023-10-10 17:03:11,669][123582] Updated weights for policy 0, policy_version 12313 (0.0007) [2023-10-10 17:03:11,982][123614] Updated weights for policy 1, policy_version 12270 (0.0007) [2023-10-10 17:03:12,348][123614] Updated weights for policy 1, policy_version 12280 (0.0009) [2023-10-10 17:03:13,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 25198592. Throughput: 0: 1824.4, 1: 1814.4. Samples: 6302946. Policy #0 lag: (min: 8.0, avg: 32.9, max: 40.0) [2023-10-10 17:03:13,789][122664] Avg episode reward: [(0, '22.790'), (1, '22.900')] [2023-10-10 17:03:15,302][123582] Updated weights for policy 0, policy_version 12323 (0.0008) [2023-10-10 17:03:15,684][123582] Updated weights for policy 0, policy_version 12333 (0.0009) [2023-10-10 17:03:16,060][123582] Updated weights for policy 0, policy_version 12343 (0.0009) [2023-10-10 17:03:16,093][123614] Updated weights for policy 1, policy_version 12290 (0.0010) [2023-10-10 17:03:16,453][123614] Updated weights for policy 1, policy_version 12300 (0.0009) [2023-10-10 17:03:16,825][123614] Updated weights for policy 1, policy_version 12310 (0.0009) [2023-10-10 17:03:17,201][123614] Updated weights for policy 1, policy_version 12320 (0.0008) [2023-10-10 17:03:18,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 25264128. Throughput: 0: 1822.8, 1: 1808.3. Samples: 6323832. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:03:18,789][122664] Avg episode reward: [(0, '22.970'), (1, '24.130')] [2023-10-10 17:03:19,809][123582] Updated weights for policy 0, policy_version 12353 (0.0009) [2023-10-10 17:03:20,184][123582] Updated weights for policy 0, policy_version 12363 (0.0010) [2023-10-10 17:03:20,564][123582] Updated weights for policy 0, policy_version 12373 (0.0009) [2023-10-10 17:03:20,878][123614] Updated weights for policy 1, policy_version 12330 (0.0008) [2023-10-10 17:03:20,928][123582] Updated weights for policy 0, policy_version 12383 (0.0010) [2023-10-10 17:03:21,250][123614] Updated weights for policy 1, policy_version 12340 (0.0010) [2023-10-10 17:03:21,621][123614] Updated weights for policy 1, policy_version 12350 (0.0008) [2023-10-10 17:03:23,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 25329664. Throughput: 0: 1819.6, 1: 1815.8. Samples: 6346398. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:03:23,789][122664] Avg episode reward: [(0, '22.950'), (1, '23.590')] [2023-10-10 17:03:24,641][123582] Updated weights for policy 0, policy_version 12393 (0.0011) [2023-10-10 17:03:25,020][123582] Updated weights for policy 0, policy_version 12403 (0.0009) [2023-10-10 17:03:25,251][123614] Updated weights for policy 1, policy_version 12360 (0.0009) [2023-10-10 17:03:25,388][123582] Updated weights for policy 0, policy_version 12413 (0.0009) [2023-10-10 17:03:25,627][123614] Updated weights for policy 1, policy_version 12370 (0.0008) [2023-10-10 17:03:25,997][123614] Updated weights for policy 1, policy_version 12380 (0.0007) [2023-10-10 17:03:28,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 25395200. Throughput: 0: 1819.2, 1: 1815.4. Samples: 6356492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:03:28,788][122664] Avg episode reward: [(0, '21.870'), (1, '23.310')] [2023-10-10 17:03:28,990][123582] Updated weights for policy 0, policy_version 12423 (0.0008) [2023-10-10 17:03:29,375][123582] Updated weights for policy 0, policy_version 12433 (0.0010) [2023-10-10 17:03:29,557][123614] Updated weights for policy 1, policy_version 12390 (0.0007) [2023-10-10 17:03:29,739][123582] Updated weights for policy 0, policy_version 12443 (0.0008) [2023-10-10 17:03:29,915][123614] Updated weights for policy 1, policy_version 12400 (0.0007) [2023-10-10 17:03:30,293][123614] Updated weights for policy 1, policy_version 12410 (0.0010) [2023-10-10 17:03:33,373][123582] Updated weights for policy 0, policy_version 12453 (0.0008) [2023-10-10 17:03:33,744][123582] Updated weights for policy 0, policy_version 12463 (0.0007) [2023-10-10 17:03:33,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 25460736. Throughput: 0: 1819.6, 1: 1807.7. Samples: 6379152. Policy #0 lag: (min: 8.0, avg: 36.8, max: 40.0) [2023-10-10 17:03:33,789][122664] Avg episode reward: [(0, '19.040'), (1, '23.840')] [2023-10-10 17:03:34,105][123582] Updated weights for policy 0, policy_version 12473 (0.0007) [2023-10-10 17:03:34,195][123614] Updated weights for policy 1, policy_version 12420 (0.0010) [2023-10-10 17:03:34,574][123614] Updated weights for policy 1, policy_version 12430 (0.0009) [2023-10-10 17:03:34,941][123614] Updated weights for policy 1, policy_version 12440 (0.0008) [2023-10-10 17:03:37,825][123582] Updated weights for policy 0, policy_version 12483 (0.0007) [2023-10-10 17:03:38,204][123582] Updated weights for policy 0, policy_version 12493 (0.0007) [2023-10-10 17:03:38,510][123614] Updated weights for policy 1, policy_version 12450 (0.0008) [2023-10-10 17:03:38,579][123582] Updated weights for policy 0, policy_version 12503 (0.0007) [2023-10-10 17:03:38,788][122664] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 25526272. Throughput: 0: 1816.4, 1: 1813.7. Samples: 6400890. Policy #0 lag: (min: 8.0, avg: 36.8, max: 40.0) [2023-10-10 17:03:38,789][122664] Avg episode reward: [(0, '19.160'), (1, '24.550')] [2023-10-10 17:03:38,880][123614] Updated weights for policy 1, policy_version 12460 (0.0009) [2023-10-10 17:03:39,255][123614] Updated weights for policy 1, policy_version 12470 (0.0010) [2023-10-10 17:03:39,628][123614] Updated weights for policy 1, policy_version 12480 (0.0009) [2023-10-10 17:03:42,287][123582] Updated weights for policy 0, policy_version 12513 (0.0009) [2023-10-10 17:03:42,698][123582] Updated weights for policy 0, policy_version 12523 (0.0012) [2023-10-10 17:03:43,077][123582] Updated weights for policy 0, policy_version 12533 (0.0008) [2023-10-10 17:03:43,333][123614] Updated weights for policy 1, policy_version 12490 (0.0008) [2023-10-10 17:03:43,454][123582] Updated weights for policy 0, policy_version 12543 (0.0008) [2023-10-10 17:03:43,704][123614] Updated weights for policy 1, policy_version 12500 (0.0008) [2023-10-10 17:03:43,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 25624576. Throughput: 0: 1812.4, 1: 1818.6. Samples: 6412118. Policy #0 lag: (min: 17.0, avg: 26.4, max: 49.0) [2023-10-10 17:03:43,789][122664] Avg episode reward: [(0, '18.870'), (1, '24.020')] [2023-10-10 17:03:44,079][123614] Updated weights for policy 1, policy_version 12510 (0.0007) [2023-10-10 17:03:47,198][123582] Updated weights for policy 0, policy_version 12553 (0.0008) [2023-10-10 17:03:47,574][123582] Updated weights for policy 0, policy_version 12563 (0.0007) [2023-10-10 17:03:47,843][123614] Updated weights for policy 1, policy_version 12520 (0.0007) [2023-10-10 17:03:47,941][123582] Updated weights for policy 0, policy_version 12573 (0.0009) [2023-10-10 17:03:48,212][123614] Updated weights for policy 1, policy_version 12530 (0.0007) [2023-10-10 17:03:48,589][123614] Updated weights for policy 1, policy_version 12540 (0.0008) [2023-10-10 17:03:48,788][122664] Fps is (10 sec: 19661.3, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 25722880. Throughput: 0: 1816.8, 1: 1818.0. Samples: 6433702. Policy #0 lag: (min: 17.0, avg: 26.4, max: 49.0) [2023-10-10 17:03:48,789][122664] Avg episode reward: [(0, '18.680'), (1, '23.800')] [2023-10-10 17:03:51,632][123582] Updated weights for policy 0, policy_version 12583 (0.0010) [2023-10-10 17:03:52,002][123582] Updated weights for policy 0, policy_version 12593 (0.0009) [2023-10-10 17:03:52,177][123614] Updated weights for policy 1, policy_version 12550 (0.0008) [2023-10-10 17:03:52,372][123582] Updated weights for policy 0, policy_version 12603 (0.0007) [2023-10-10 17:03:52,555][123614] Updated weights for policy 1, policy_version 12560 (0.0008) [2023-10-10 17:03:52,912][123614] Updated weights for policy 1, policy_version 12570 (0.0007) [2023-10-10 17:03:53,788][122664] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 25788416. Throughput: 0: 1807.4, 1: 1820.2. Samples: 6454570. Policy #0 lag: (min: 17.0, avg: 26.4, max: 49.0) [2023-10-10 17:03:53,789][122664] Avg episode reward: [(0, '19.720'), (1, '24.320')] [2023-10-10 17:03:53,798][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000012576_12877824.pth... [2023-10-10 17:03:53,799][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000012608_12910592.pth... [2023-10-10 17:03:53,828][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000010880_11141120.pth [2023-10-10 17:03:53,838][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000010912_11173888.pth [2023-10-10 17:03:55,949][123582] Updated weights for policy 0, policy_version 12613 (0.0008) [2023-10-10 17:03:56,324][123582] Updated weights for policy 0, policy_version 12623 (0.0010) [2023-10-10 17:03:56,638][123614] Updated weights for policy 1, policy_version 12580 (0.0007) [2023-10-10 17:03:56,686][123582] Updated weights for policy 0, policy_version 12633 (0.0008) [2023-10-10 17:03:57,018][123614] Updated weights for policy 1, policy_version 12590 (0.0007) [2023-10-10 17:03:57,372][123614] Updated weights for policy 1, policy_version 12600 (0.0009) [2023-10-10 17:03:58,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 25853952. Throughput: 0: 1814.1, 1: 1820.5. Samples: 6466500. Policy #0 lag: (min: 19.0, avg: 25.3, max: 51.0) [2023-10-10 17:03:58,789][122664] Avg episode reward: [(0, '19.060'), (1, '22.650')] [2023-10-10 17:04:00,465][123582] Updated weights for policy 0, policy_version 12643 (0.0009) [2023-10-10 17:04:00,834][123582] Updated weights for policy 0, policy_version 12653 (0.0010) [2023-10-10 17:04:01,134][123614] Updated weights for policy 1, policy_version 12610 (0.0009) [2023-10-10 17:04:01,204][123582] Updated weights for policy 0, policy_version 12663 (0.0008) [2023-10-10 17:04:01,486][123614] Updated weights for policy 1, policy_version 12620 (0.0008) [2023-10-10 17:04:01,855][123614] Updated weights for policy 1, policy_version 12630 (0.0010) [2023-10-10 17:04:02,224][123614] Updated weights for policy 1, policy_version 12640 (0.0008) [2023-10-10 17:04:03,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 25919488. Throughput: 0: 1813.9, 1: 1814.8. Samples: 6487122. Policy #0 lag: (min: 19.0, avg: 25.3, max: 51.0) [2023-10-10 17:04:03,789][122664] Avg episode reward: [(0, '21.140'), (1, '21.680')] [2023-10-10 17:04:04,863][123582] Updated weights for policy 0, policy_version 12673 (0.0008) [2023-10-10 17:04:05,230][123582] Updated weights for policy 0, policy_version 12683 (0.0010) [2023-10-10 17:04:05,597][123582] Updated weights for policy 0, policy_version 12693 (0.0010) [2023-10-10 17:04:05,972][123582] Updated weights for policy 0, policy_version 12703 (0.0008) [2023-10-10 17:04:06,020][123614] Updated weights for policy 1, policy_version 12650 (0.0008) [2023-10-10 17:04:06,395][123614] Updated weights for policy 1, policy_version 12660 (0.0010) [2023-10-10 17:04:06,762][123614] Updated weights for policy 1, policy_version 12670 (0.0010) [2023-10-10 17:04:08,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 25985024. Throughput: 0: 1814.8, 1: 1805.7. Samples: 6509322. Policy #0 lag: (min: 19.0, avg: 25.3, max: 51.0) [2023-10-10 17:04:08,789][122664] Avg episode reward: [(0, '19.680'), (1, '22.080')] [2023-10-10 17:04:09,773][123582] Updated weights for policy 0, policy_version 12713 (0.0009) [2023-10-10 17:04:10,136][123582] Updated weights for policy 0, policy_version 12723 (0.0007) [2023-10-10 17:04:10,499][123614] Updated weights for policy 1, policy_version 12680 (0.0008) [2023-10-10 17:04:10,513][123582] Updated weights for policy 0, policy_version 12733 (0.0008) [2023-10-10 17:04:10,870][123614] Updated weights for policy 1, policy_version 12690 (0.0008) [2023-10-10 17:04:11,232][123614] Updated weights for policy 1, policy_version 12700 (0.0008) [2023-10-10 17:04:13,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 26050560. Throughput: 0: 1813.2, 1: 1803.8. Samples: 6519256. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:04:13,788][122664] Avg episode reward: [(0, '19.620'), (1, '21.820')] [2023-10-10 17:04:14,178][123582] Updated weights for policy 0, policy_version 12743 (0.0008) [2023-10-10 17:04:14,545][123582] Updated weights for policy 0, policy_version 12753 (0.0008) [2023-10-10 17:04:14,791][123614] Updated weights for policy 1, policy_version 12710 (0.0010) [2023-10-10 17:04:14,923][123582] Updated weights for policy 0, policy_version 12763 (0.0009) [2023-10-10 17:04:15,157][123614] Updated weights for policy 1, policy_version 12720 (0.0008) [2023-10-10 17:04:15,531][123614] Updated weights for policy 1, policy_version 12730 (0.0009) [2023-10-10 17:04:18,528][123582] Updated weights for policy 0, policy_version 12773 (0.0008) [2023-10-10 17:04:18,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 26116096. Throughput: 0: 1819.5, 1: 1812.3. Samples: 6542580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:04:18,789][122664] Avg episode reward: [(0, '21.110'), (1, '21.510')] [2023-10-10 17:04:18,912][123582] Updated weights for policy 0, policy_version 12783 (0.0009) [2023-10-10 17:04:19,215][123614] Updated weights for policy 1, policy_version 12740 (0.0008) [2023-10-10 17:04:19,284][123582] Updated weights for policy 0, policy_version 12793 (0.0009) [2023-10-10 17:04:19,592][123614] Updated weights for policy 1, policy_version 12750 (0.0007) [2023-10-10 17:04:19,957][123614] Updated weights for policy 1, policy_version 12760 (0.0010) [2023-10-10 17:04:22,930][123582] Updated weights for policy 0, policy_version 12803 (0.0009) [2023-10-10 17:04:23,300][123582] Updated weights for policy 0, policy_version 12813 (0.0011) [2023-10-10 17:04:23,679][123582] Updated weights for policy 0, policy_version 12823 (0.0008) [2023-10-10 17:04:23,706][123614] Updated weights for policy 1, policy_version 12770 (0.0010) [2023-10-10 17:04:23,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 26181632. Throughput: 0: 1821.6, 1: 1811.1. Samples: 6564360. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:04:23,789][122664] Avg episode reward: [(0, '20.650'), (1, '22.830')] [2023-10-10 17:04:24,074][123614] Updated weights for policy 1, policy_version 12780 (0.0008) [2023-10-10 17:04:24,442][123614] Updated weights for policy 1, policy_version 12790 (0.0010) [2023-10-10 17:04:24,818][123614] Updated weights for policy 1, policy_version 12800 (0.0010) [2023-10-10 17:04:27,405][123582] Updated weights for policy 0, policy_version 12833 (0.0008) [2023-10-10 17:04:27,802][123582] Updated weights for policy 0, policy_version 12843 (0.0007) [2023-10-10 17:04:28,174][123582] Updated weights for policy 0, policy_version 12853 (0.0008) [2023-10-10 17:04:28,375][123614] Updated weights for policy 1, policy_version 12810 (0.0008) [2023-10-10 17:04:28,543][123582] Updated weights for policy 0, policy_version 12863 (0.0009) [2023-10-10 17:04:28,747][123614] Updated weights for policy 1, policy_version 12820 (0.0008) [2023-10-10 17:04:28,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 26279936. Throughput: 0: 1817.6, 1: 1804.5. Samples: 6575112. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:04:28,789][122664] Avg episode reward: [(0, '19.220'), (1, '21.360')] [2023-10-10 17:04:29,124][123614] Updated weights for policy 1, policy_version 12830 (0.0008) [2023-10-10 17:04:32,342][123582] Updated weights for policy 0, policy_version 12873 (0.0007) [2023-10-10 17:04:32,709][123582] Updated weights for policy 0, policy_version 12883 (0.0008) [2023-10-10 17:04:32,934][123614] Updated weights for policy 1, policy_version 12840 (0.0009) [2023-10-10 17:04:33,086][123582] Updated weights for policy 0, policy_version 12893 (0.0007) [2023-10-10 17:04:33,301][123614] Updated weights for policy 1, policy_version 12850 (0.0009) [2023-10-10 17:04:33,672][123614] Updated weights for policy 1, policy_version 12860 (0.0007) [2023-10-10 17:04:33,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 26345472. Throughput: 0: 1816.6, 1: 1810.0. Samples: 6596900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:04:33,788][122664] Avg episode reward: [(0, '19.380'), (1, '20.890')] [2023-10-10 17:04:36,701][123582] Updated weights for policy 0, policy_version 12903 (0.0008) [2023-10-10 17:04:37,072][123582] Updated weights for policy 0, policy_version 12913 (0.0010) [2023-10-10 17:04:37,444][123582] Updated weights for policy 0, policy_version 12923 (0.0009) [2023-10-10 17:04:37,528][123614] Updated weights for policy 1, policy_version 12870 (0.0008) [2023-10-10 17:04:37,912][123614] Updated weights for policy 1, policy_version 12880 (0.0009) [2023-10-10 17:04:38,276][123614] Updated weights for policy 1, policy_version 12890 (0.0009) [2023-10-10 17:04:38,788][122664] Fps is (10 sec: 16384.3, 60 sec: 15291.8, 300 sec: 14551.2). Total num frames: 26443776. Throughput: 0: 1814.2, 1: 1800.7. Samples: 6617242. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-10 17:04:38,788][122664] Avg episode reward: [(0, '19.320'), (1, '19.460')] [2023-10-10 17:04:41,185][123582] Updated weights for policy 0, policy_version 12933 (0.0009) [2023-10-10 17:04:41,564][123582] Updated weights for policy 0, policy_version 12943 (0.0009) [2023-10-10 17:04:41,908][123614] Updated weights for policy 1, policy_version 12900 (0.0007) [2023-10-10 17:04:41,936][123582] Updated weights for policy 0, policy_version 12953 (0.0008) [2023-10-10 17:04:42,282][123614] Updated weights for policy 1, policy_version 12910 (0.0009) [2023-10-10 17:04:42,653][123614] Updated weights for policy 1, policy_version 12920 (0.0009) [2023-10-10 17:04:43,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 26509312. Throughput: 0: 1817.2, 1: 1812.1. Samples: 6629822. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-10 17:04:43,789][122664] Avg episode reward: [(0, '20.350'), (1, '16.890')] [2023-10-10 17:04:45,607][123582] Updated weights for policy 0, policy_version 12963 (0.0007) [2023-10-10 17:04:45,982][123582] Updated weights for policy 0, policy_version 12973 (0.0007) [2023-10-10 17:04:46,247][123614] Updated weights for policy 1, policy_version 12930 (0.0008) [2023-10-10 17:04:46,347][123582] Updated weights for policy 0, policy_version 12983 (0.0007) [2023-10-10 17:04:46,621][123614] Updated weights for policy 1, policy_version 12940 (0.0008) [2023-10-10 17:04:46,982][123614] Updated weights for policy 1, policy_version 12950 (0.0010) [2023-10-10 17:04:47,346][123614] Updated weights for policy 1, policy_version 12960 (0.0009) [2023-10-10 17:04:48,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 26574848. Throughput: 0: 1809.2, 1: 1815.6. Samples: 6650240. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-10 17:04:48,789][122664] Avg episode reward: [(0, '20.530'), (1, '17.370')] [2023-10-10 17:04:49,995][123582] Updated weights for policy 0, policy_version 12993 (0.0008) [2023-10-10 17:04:50,370][123582] Updated weights for policy 0, policy_version 13003 (0.0007) [2023-10-10 17:04:50,749][123582] Updated weights for policy 0, policy_version 13013 (0.0008) [2023-10-10 17:04:51,115][123582] Updated weights for policy 0, policy_version 13023 (0.0008) [2023-10-10 17:04:51,240][123614] Updated weights for policy 1, policy_version 12970 (0.0007) [2023-10-10 17:04:51,605][123614] Updated weights for policy 1, policy_version 12980 (0.0009) [2023-10-10 17:04:51,978][123614] Updated weights for policy 1, policy_version 12990 (0.0007) [2023-10-10 17:04:53,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 26640384. Throughput: 0: 1811.9, 1: 1821.2. Samples: 6672812. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:04:53,788][122664] Avg episode reward: [(0, '21.770'), (1, '16.250')] [2023-10-10 17:04:54,867][123582] Updated weights for policy 0, policy_version 13033 (0.0007) [2023-10-10 17:04:55,237][123582] Updated weights for policy 0, policy_version 13043 (0.0008) [2023-10-10 17:04:55,612][123582] Updated weights for policy 0, policy_version 13053 (0.0008) [2023-10-10 17:04:55,633][123614] Updated weights for policy 1, policy_version 13000 (0.0008) [2023-10-10 17:04:56,000][123614] Updated weights for policy 1, policy_version 13010 (0.0007) [2023-10-10 17:04:56,366][123614] Updated weights for policy 1, policy_version 13020 (0.0009) [2023-10-10 17:04:58,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 26705920. Throughput: 0: 1811.7, 1: 1819.0. Samples: 6682638. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:04:58,789][122664] Avg episode reward: [(0, '24.430'), (1, '17.900')] [2023-10-10 17:04:58,789][123247] Saving new best policy, reward=24.430! [2023-10-10 17:04:59,439][123582] Updated weights for policy 0, policy_version 13063 (0.0009) [2023-10-10 17:04:59,805][123582] Updated weights for policy 0, policy_version 13073 (0.0009) [2023-10-10 17:05:00,137][123614] Updated weights for policy 1, policy_version 13030 (0.0009) [2023-10-10 17:05:00,177][123582] Updated weights for policy 0, policy_version 13083 (0.0007) [2023-10-10 17:05:00,497][123614] Updated weights for policy 1, policy_version 13040 (0.0007) [2023-10-10 17:05:00,863][123614] Updated weights for policy 1, policy_version 13050 (0.0009) [2023-10-10 17:05:03,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 26771456. Throughput: 0: 1802.5, 1: 1813.6. Samples: 6705306. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:05:03,788][122664] Avg episode reward: [(0, '25.610'), (1, '18.480')] [2023-10-10 17:05:03,986][123582] Updated weights for policy 0, policy_version 13093 (0.0008) [2023-10-10 17:05:04,370][123582] Updated weights for policy 0, policy_version 13103 (0.0010) [2023-10-10 17:05:04,437][123614] Updated weights for policy 1, policy_version 13060 (0.0007) [2023-10-10 17:05:04,746][123582] Updated weights for policy 0, policy_version 13113 (0.0007) [2023-10-10 17:05:04,815][123614] Updated weights for policy 1, policy_version 13070 (0.0008) [2023-10-10 17:05:05,001][123247] Saving new best policy, reward=25.610! [2023-10-10 17:05:05,185][123614] Updated weights for policy 1, policy_version 13080 (0.0009) [2023-10-10 17:05:08,267][123582] Updated weights for policy 0, policy_version 13123 (0.0007) [2023-10-10 17:05:08,649][123582] Updated weights for policy 0, policy_version 13133 (0.0007) [2023-10-10 17:05:08,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 26836992. Throughput: 0: 1815.1, 1: 1814.6. Samples: 6727696. Policy #0 lag: (min: 31.0, avg: 32.5, max: 58.0) [2023-10-10 17:05:08,789][122664] Avg episode reward: [(0, '25.610'), (1, '18.990')] [2023-10-10 17:05:08,919][123614] Updated weights for policy 1, policy_version 13090 (0.0007) [2023-10-10 17:05:09,023][123582] Updated weights for policy 0, policy_version 13143 (0.0007) [2023-10-10 17:05:09,286][123614] Updated weights for policy 1, policy_version 13100 (0.0007) [2023-10-10 17:05:09,648][123614] Updated weights for policy 1, policy_version 13110 (0.0010) [2023-10-10 17:05:10,018][123614] Updated weights for policy 1, policy_version 13120 (0.0009) [2023-10-10 17:05:12,871][123582] Updated weights for policy 0, policy_version 13153 (0.0008) [2023-10-10 17:05:13,285][123582] Updated weights for policy 0, policy_version 13163 (0.0008) [2023-10-10 17:05:13,617][123614] Updated weights for policy 1, policy_version 13130 (0.0009) [2023-10-10 17:05:13,662][123582] Updated weights for policy 0, policy_version 13173 (0.0009) [2023-10-10 17:05:13,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 26902528. Throughput: 0: 1804.3, 1: 1812.6. Samples: 6737872. Policy #0 lag: (min: 31.0, avg: 32.5, max: 58.0) [2023-10-10 17:05:13,788][122664] Avg episode reward: [(0, '24.340'), (1, '19.710')] [2023-10-10 17:05:13,977][123614] Updated weights for policy 1, policy_version 13140 (0.0007) [2023-10-10 17:05:14,032][123582] Updated weights for policy 0, policy_version 13183 (0.0008) [2023-10-10 17:05:14,358][123614] Updated weights for policy 1, policy_version 13150 (0.0009) [2023-10-10 17:05:17,623][123582] Updated weights for policy 0, policy_version 13193 (0.0008) [2023-10-10 17:05:17,998][123582] Updated weights for policy 0, policy_version 13203 (0.0009) [2023-10-10 17:05:18,024][123614] Updated weights for policy 1, policy_version 13160 (0.0010) [2023-10-10 17:05:18,368][123582] Updated weights for policy 0, policy_version 13213 (0.0008) [2023-10-10 17:05:18,398][123614] Updated weights for policy 1, policy_version 13170 (0.0009) [2023-10-10 17:05:18,765][123614] Updated weights for policy 1, policy_version 13180 (0.0007) [2023-10-10 17:05:18,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 27000832. Throughput: 0: 1818.5, 1: 1813.8. Samples: 6760356. Policy #0 lag: (min: 11.0, avg: 13.9, max: 43.0) [2023-10-10 17:05:18,789][122664] Avg episode reward: [(0, '24.600'), (1, '19.370')] [2023-10-10 17:05:22,096][123582] Updated weights for policy 0, policy_version 13223 (0.0008) [2023-10-10 17:05:22,474][123582] Updated weights for policy 0, policy_version 13233 (0.0009) [2023-10-10 17:05:22,508][123614] Updated weights for policy 1, policy_version 13190 (0.0007) [2023-10-10 17:05:22,845][123582] Updated weights for policy 0, policy_version 13243 (0.0010) [2023-10-10 17:05:22,887][123614] Updated weights for policy 1, policy_version 13200 (0.0007) [2023-10-10 17:05:23,253][123614] Updated weights for policy 1, policy_version 13210 (0.0008) [2023-10-10 17:05:23,788][122664] Fps is (10 sec: 19660.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 27099136. Throughput: 0: 1804.4, 1: 1816.0. Samples: 6780162. Policy #0 lag: (min: 11.0, avg: 13.9, max: 43.0) [2023-10-10 17:05:23,789][122664] Avg episode reward: [(0, '24.300'), (1, '19.760')] [2023-10-10 17:05:26,537][123582] Updated weights for policy 0, policy_version 13253 (0.0008) [2023-10-10 17:05:26,831][123614] Updated weights for policy 1, policy_version 13220 (0.0009) [2023-10-10 17:05:26,915][123582] Updated weights for policy 0, policy_version 13263 (0.0008) [2023-10-10 17:05:27,189][123614] Updated weights for policy 1, policy_version 13230 (0.0007) [2023-10-10 17:05:27,278][123582] Updated weights for policy 0, policy_version 13273 (0.0007) [2023-10-10 17:05:27,554][123614] Updated weights for policy 1, policy_version 13240 (0.0008) [2023-10-10 17:05:28,788][122664] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 27164672. Throughput: 0: 1811.3, 1: 1812.0. Samples: 6792874. Policy #0 lag: (min: 11.0, avg: 13.9, max: 43.0) [2023-10-10 17:05:28,788][122664] Avg episode reward: [(0, '23.460'), (1, '17.560')] [2023-10-10 17:05:31,042][123582] Updated weights for policy 0, policy_version 13283 (0.0008) [2023-10-10 17:05:31,417][123582] Updated weights for policy 0, policy_version 13293 (0.0011) [2023-10-10 17:05:31,536][123614] Updated weights for policy 1, policy_version 13250 (0.0007) [2023-10-10 17:05:31,781][123582] Updated weights for policy 0, policy_version 13303 (0.0007) [2023-10-10 17:05:31,900][123614] Updated weights for policy 1, policy_version 13260 (0.0007) [2023-10-10 17:05:32,264][123614] Updated weights for policy 1, policy_version 13270 (0.0007) [2023-10-10 17:05:32,629][123614] Updated weights for policy 1, policy_version 13280 (0.0009) [2023-10-10 17:05:33,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 27230208. Throughput: 0: 1800.6, 1: 1804.3. Samples: 6812458. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:05:33,789][122664] Avg episode reward: [(0, '22.510'), (1, '17.220')] [2023-10-10 17:05:35,583][123582] Updated weights for policy 0, policy_version 13313 (0.0008) [2023-10-10 17:05:35,952][123582] Updated weights for policy 0, policy_version 13323 (0.0008) [2023-10-10 17:05:36,271][123614] Updated weights for policy 1, policy_version 13290 (0.0009) [2023-10-10 17:05:36,325][123582] Updated weights for policy 0, policy_version 13333 (0.0008) [2023-10-10 17:05:36,643][123614] Updated weights for policy 1, policy_version 13300 (0.0009) [2023-10-10 17:05:36,711][123582] Updated weights for policy 0, policy_version 13343 (0.0008) [2023-10-10 17:05:37,013][123614] Updated weights for policy 1, policy_version 13310 (0.0008) [2023-10-10 17:05:38,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 27295744. Throughput: 0: 1794.0, 1: 1816.4. Samples: 6835280. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:05:38,789][122664] Avg episode reward: [(0, '22.290'), (1, '16.990')] [2023-10-10 17:05:40,435][123582] Updated weights for policy 0, policy_version 13353 (0.0009) [2023-10-10 17:05:40,622][123614] Updated weights for policy 1, policy_version 13320 (0.0008) [2023-10-10 17:05:40,798][123582] Updated weights for policy 0, policy_version 13363 (0.0009) [2023-10-10 17:05:40,991][123614] Updated weights for policy 1, policy_version 13330 (0.0008) [2023-10-10 17:05:41,168][123582] Updated weights for policy 0, policy_version 13373 (0.0009) [2023-10-10 17:05:41,352][123614] Updated weights for policy 1, policy_version 13340 (0.0008) [2023-10-10 17:05:43,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 27361280. Throughput: 0: 1791.2, 1: 1815.1. Samples: 6844920. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:05:43,789][122664] Avg episode reward: [(0, '21.770'), (1, '16.940')] [2023-10-10 17:05:44,854][123582] Updated weights for policy 0, policy_version 13383 (0.0010) [2023-10-10 17:05:45,138][123614] Updated weights for policy 1, policy_version 13350 (0.0008) [2023-10-10 17:05:45,222][123582] Updated weights for policy 0, policy_version 13393 (0.0008) [2023-10-10 17:05:45,517][123614] Updated weights for policy 1, policy_version 13360 (0.0009) [2023-10-10 17:05:45,601][123582] Updated weights for policy 0, policy_version 13403 (0.0008) [2023-10-10 17:05:45,887][123614] Updated weights for policy 1, policy_version 13370 (0.0009) [2023-10-10 17:05:48,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 27426816. Throughput: 0: 1795.1, 1: 1809.2. Samples: 6867498. Policy #0 lag: (min: 20.0, avg: 20.0, max: 22.0) [2023-10-10 17:05:48,789][122664] Avg episode reward: [(0, '21.310'), (1, '15.890')] [2023-10-10 17:05:49,354][123582] Updated weights for policy 0, policy_version 13413 (0.0008) [2023-10-10 17:05:49,655][123614] Updated weights for policy 1, policy_version 13380 (0.0008) [2023-10-10 17:05:49,730][123582] Updated weights for policy 0, policy_version 13423 (0.0008) [2023-10-10 17:05:50,028][123614] Updated weights for policy 1, policy_version 13390 (0.0009) [2023-10-10 17:05:50,110][123582] Updated weights for policy 0, policy_version 13433 (0.0007) [2023-10-10 17:05:50,386][123614] Updated weights for policy 1, policy_version 13400 (0.0009) [2023-10-10 17:05:53,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 27492352. Throughput: 0: 1793.1, 1: 1813.4. Samples: 6889988. Policy #0 lag: (min: 20.0, avg: 20.0, max: 22.0) [2023-10-10 17:05:53,789][122664] Avg episode reward: [(0, '21.270'), (1, '16.690')] [2023-10-10 17:05:53,797][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000013408_13729792.pth... [2023-10-10 17:05:53,832][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000011712_11993088.pth [2023-10-10 17:05:53,950][123582] Updated weights for policy 0, policy_version 13443 (0.0009) [2023-10-10 17:05:54,181][123614] Updated weights for policy 1, policy_version 13410 (0.0009) [2023-10-10 17:05:54,332][123582] Updated weights for policy 0, policy_version 13453 (0.0009) [2023-10-10 17:05:54,546][123614] Updated weights for policy 1, policy_version 13420 (0.0007) [2023-10-10 17:05:54,699][123582] Updated weights for policy 0, policy_version 13463 (0.0007) [2023-10-10 17:05:54,923][123614] Updated weights for policy 1, policy_version 13430 (0.0008) [2023-10-10 17:05:55,027][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000013472_13795328.pth... [2023-10-10 17:05:55,060][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000011744_12025856.pth [2023-10-10 17:05:55,281][123614] Updated weights for policy 1, policy_version 13440 (0.0009) [2023-10-10 17:05:58,435][123582] Updated weights for policy 0, policy_version 13473 (0.0008) [2023-10-10 17:05:58,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 27557888. Throughput: 0: 1787.5, 1: 1815.5. Samples: 6900008. Policy #0 lag: (min: 20.0, avg: 20.0, max: 22.0) [2023-10-10 17:05:58,789][122664] Avg episode reward: [(0, '21.530'), (1, '18.510')] [2023-10-10 17:05:58,842][123582] Updated weights for policy 0, policy_version 13483 (0.0008) [2023-10-10 17:05:58,916][123614] Updated weights for policy 1, policy_version 13450 (0.0007) [2023-10-10 17:05:59,211][123582] Updated weights for policy 0, policy_version 13493 (0.0008) [2023-10-10 17:05:59,287][123614] Updated weights for policy 1, policy_version 13460 (0.0008) [2023-10-10 17:05:59,580][123582] Updated weights for policy 0, policy_version 13503 (0.0009) [2023-10-10 17:05:59,646][123614] Updated weights for policy 1, policy_version 13470 (0.0009) [2023-10-10 17:06:03,266][123582] Updated weights for policy 0, policy_version 13513 (0.0010) [2023-10-10 17:06:03,353][123614] Updated weights for policy 1, policy_version 13480 (0.0007) [2023-10-10 17:06:03,625][123582] Updated weights for policy 0, policy_version 13523 (0.0009) [2023-10-10 17:06:03,717][123614] Updated weights for policy 1, policy_version 13490 (0.0010) [2023-10-10 17:06:03,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 27623424. Throughput: 0: 1789.7, 1: 1818.0. Samples: 6922700. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:06:03,788][122664] Avg episode reward: [(0, '20.790'), (1, '17.610')] [2023-10-10 17:06:03,992][123582] Updated weights for policy 0, policy_version 13533 (0.0008) [2023-10-10 17:06:04,089][123614] Updated weights for policy 1, policy_version 13500 (0.0008) [2023-10-10 17:06:07,615][123582] Updated weights for policy 0, policy_version 13543 (0.0007) [2023-10-10 17:06:07,771][123614] Updated weights for policy 1, policy_version 13510 (0.0008) [2023-10-10 17:06:07,995][123582] Updated weights for policy 0, policy_version 13553 (0.0007) [2023-10-10 17:06:08,155][123614] Updated weights for policy 1, policy_version 13520 (0.0010) [2023-10-10 17:06:08,369][123582] Updated weights for policy 0, policy_version 13563 (0.0008) [2023-10-10 17:06:08,513][123614] Updated weights for policy 1, policy_version 13530 (0.0009) [2023-10-10 17:06:08,788][122664] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 27754496. Throughput: 0: 1799.8, 1: 1816.8. Samples: 6942908. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:06:08,789][122664] Avg episode reward: [(0, '21.330'), (1, '18.870')] [2023-10-10 17:06:12,048][123582] Updated weights for policy 0, policy_version 13573 (0.0009) [2023-10-10 17:06:12,216][123614] Updated weights for policy 1, policy_version 13540 (0.0010) [2023-10-10 17:06:12,415][123582] Updated weights for policy 0, policy_version 13583 (0.0007) [2023-10-10 17:06:12,585][123614] Updated weights for policy 1, policy_version 13550 (0.0007) [2023-10-10 17:06:12,795][123582] Updated weights for policy 0, policy_version 13593 (0.0007) [2023-10-10 17:06:12,949][123614] Updated weights for policy 1, policy_version 13560 (0.0007) [2023-10-10 17:06:13,788][122664] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 27820032. Throughput: 0: 1797.6, 1: 1813.4. Samples: 6955368. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:06:13,789][122664] Avg episode reward: [(0, '21.890'), (1, '21.980')] [2023-10-10 17:06:16,525][123582] Updated weights for policy 0, policy_version 13603 (0.0008) [2023-10-10 17:06:16,619][123614] Updated weights for policy 1, policy_version 13570 (0.0007) [2023-10-10 17:06:16,900][123582] Updated weights for policy 0, policy_version 13613 (0.0009) [2023-10-10 17:06:16,981][123614] Updated weights for policy 1, policy_version 13580 (0.0007) [2023-10-10 17:06:17,274][123582] Updated weights for policy 0, policy_version 13623 (0.0009) [2023-10-10 17:06:17,345][123614] Updated weights for policy 1, policy_version 13590 (0.0008) [2023-10-10 17:06:17,719][123614] Updated weights for policy 1, policy_version 13600 (0.0007) [2023-10-10 17:06:18,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 27885568. Throughput: 0: 1800.9, 1: 1824.3. Samples: 6975592. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:06:18,789][122664] Avg episode reward: [(0, '23.700'), (1, '22.070')] [2023-10-10 17:06:20,872][123582] Updated weights for policy 0, policy_version 13633 (0.0007) [2023-10-10 17:06:21,246][123582] Updated weights for policy 0, policy_version 13643 (0.0007) [2023-10-10 17:06:21,486][123614] Updated weights for policy 1, policy_version 13610 (0.0008) [2023-10-10 17:06:21,611][123582] Updated weights for policy 0, policy_version 13653 (0.0009) [2023-10-10 17:06:21,855][123614] Updated weights for policy 1, policy_version 13620 (0.0007) [2023-10-10 17:06:21,981][123582] Updated weights for policy 0, policy_version 13663 (0.0008) [2023-10-10 17:06:22,219][123614] Updated weights for policy 1, policy_version 13630 (0.0007) [2023-10-10 17:06:23,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 27951104. Throughput: 0: 1805.3, 1: 1815.7. Samples: 6998226. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:06:23,789][122664] Avg episode reward: [(0, '24.270'), (1, '22.620')] [2023-10-10 17:06:25,756][123582] Updated weights for policy 0, policy_version 13673 (0.0007) [2023-10-10 17:06:25,921][123614] Updated weights for policy 1, policy_version 13640 (0.0007) [2023-10-10 17:06:26,128][123582] Updated weights for policy 0, policy_version 13683 (0.0007) [2023-10-10 17:06:26,300][123614] Updated weights for policy 1, policy_version 13650 (0.0007) [2023-10-10 17:06:26,498][123582] Updated weights for policy 0, policy_version 13693 (0.0007) [2023-10-10 17:06:26,658][123614] Updated weights for policy 1, policy_version 13660 (0.0007) [2023-10-10 17:06:28,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 28016640. Throughput: 0: 1811.9, 1: 1822.8. Samples: 7008480. Policy #0 lag: (min: 21.0, avg: 31.7, max: 53.0) [2023-10-10 17:06:28,789][122664] Avg episode reward: [(0, '23.510'), (1, '23.450')] [2023-10-10 17:06:30,255][123582] Updated weights for policy 0, policy_version 13703 (0.0009) [2023-10-10 17:06:30,329][123614] Updated weights for policy 1, policy_version 13670 (0.0008) [2023-10-10 17:06:30,621][123582] Updated weights for policy 0, policy_version 13713 (0.0008) [2023-10-10 17:06:30,687][123614] Updated weights for policy 1, policy_version 13680 (0.0008) [2023-10-10 17:06:30,993][123582] Updated weights for policy 0, policy_version 13723 (0.0008) [2023-10-10 17:06:31,064][123614] Updated weights for policy 1, policy_version 13690 (0.0009) [2023-10-10 17:06:33,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 28082176. Throughput: 0: 1801.3, 1: 1821.6. Samples: 7030528. Policy #0 lag: (min: 21.0, avg: 31.7, max: 53.0) [2023-10-10 17:06:33,788][122664] Avg episode reward: [(0, '23.400'), (1, '20.460')] [2023-10-10 17:06:34,815][123614] Updated weights for policy 1, policy_version 13700 (0.0009) [2023-10-10 17:06:34,816][123582] Updated weights for policy 0, policy_version 13733 (0.0009) [2023-10-10 17:06:35,181][123614] Updated weights for policy 1, policy_version 13710 (0.0008) [2023-10-10 17:06:35,189][123582] Updated weights for policy 0, policy_version 13743 (0.0009) [2023-10-10 17:06:35,552][123614] Updated weights for policy 1, policy_version 13720 (0.0008) [2023-10-10 17:06:35,563][123582] Updated weights for policy 0, policy_version 13753 (0.0008) [2023-10-10 17:06:38,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 28147712. Throughput: 0: 1810.8, 1: 1813.0. Samples: 7053060. Policy #0 lag: (min: 21.0, avg: 31.7, max: 53.0) [2023-10-10 17:06:38,789][122664] Avg episode reward: [(0, '21.980'), (1, '21.320')] [2023-10-10 17:06:39,260][123582] Updated weights for policy 0, policy_version 13763 (0.0008) [2023-10-10 17:06:39,276][123614] Updated weights for policy 1, policy_version 13730 (0.0008) [2023-10-10 17:06:39,632][123582] Updated weights for policy 0, policy_version 13773 (0.0010) [2023-10-10 17:06:39,642][123614] Updated weights for policy 1, policy_version 13740 (0.0009) [2023-10-10 17:06:40,003][123582] Updated weights for policy 0, policy_version 13783 (0.0008) [2023-10-10 17:06:40,008][123614] Updated weights for policy 1, policy_version 13750 (0.0009) [2023-10-10 17:06:40,374][123614] Updated weights for policy 1, policy_version 13760 (0.0007) [2023-10-10 17:06:43,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 28213248. Throughput: 0: 1809.5, 1: 1808.6. Samples: 7062820. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-10 17:06:43,788][122664] Avg episode reward: [(0, '23.590'), (1, '20.910')] [2023-10-10 17:06:43,801][123582] Updated weights for policy 0, policy_version 13793 (0.0009) [2023-10-10 17:06:44,137][123614] Updated weights for policy 1, policy_version 13770 (0.0007) [2023-10-10 17:06:44,217][123582] Updated weights for policy 0, policy_version 13803 (0.0009) [2023-10-10 17:06:44,507][123614] Updated weights for policy 1, policy_version 13780 (0.0010) [2023-10-10 17:06:44,579][123582] Updated weights for policy 0, policy_version 13813 (0.0008) [2023-10-10 17:06:44,864][123614] Updated weights for policy 1, policy_version 13790 (0.0009) [2023-10-10 17:06:44,958][123582] Updated weights for policy 0, policy_version 13823 (0.0009) [2023-10-10 17:06:48,523][123582] Updated weights for policy 0, policy_version 13833 (0.0010) [2023-10-10 17:06:48,599][123614] Updated weights for policy 1, policy_version 13800 (0.0007) [2023-10-10 17:06:48,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 28278784. Throughput: 0: 1809.4, 1: 1805.8. Samples: 7085384. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-10 17:06:48,789][122664] Avg episode reward: [(0, '22.980'), (1, '19.890')] [2023-10-10 17:06:48,890][123582] Updated weights for policy 0, policy_version 13843 (0.0008) [2023-10-10 17:06:48,970][123614] Updated weights for policy 1, policy_version 13810 (0.0008) [2023-10-10 17:06:49,258][123582] Updated weights for policy 0, policy_version 13853 (0.0008) [2023-10-10 17:06:49,333][123614] Updated weights for policy 1, policy_version 13820 (0.0009) [2023-10-10 17:06:52,916][123582] Updated weights for policy 0, policy_version 13863 (0.0009) [2023-10-10 17:06:53,074][123614] Updated weights for policy 1, policy_version 13830 (0.0008) [2023-10-10 17:06:53,288][123582] Updated weights for policy 0, policy_version 13873 (0.0010) [2023-10-10 17:06:53,467][123614] Updated weights for policy 1, policy_version 13840 (0.0008) [2023-10-10 17:06:53,671][123582] Updated weights for policy 0, policy_version 13883 (0.0009) [2023-10-10 17:06:53,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 28344320. Throughput: 0: 1813.1, 1: 1812.7. Samples: 7106068. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-10 17:06:53,789][122664] Avg episode reward: [(0, '26.820'), (1, '20.540')] [2023-10-10 17:06:53,831][123614] Updated weights for policy 1, policy_version 13850 (0.0007) [2023-10-10 17:06:53,852][123247] Saving new best policy, reward=26.820! [2023-10-10 17:06:57,414][123582] Updated weights for policy 0, policy_version 13893 (0.0008) [2023-10-10 17:06:57,503][123614] Updated weights for policy 1, policy_version 13860 (0.0008) [2023-10-10 17:06:57,787][123582] Updated weights for policy 0, policy_version 13903 (0.0007) [2023-10-10 17:06:57,870][123614] Updated weights for policy 1, policy_version 13870 (0.0008) [2023-10-10 17:06:58,155][123582] Updated weights for policy 0, policy_version 13913 (0.0010) [2023-10-10 17:06:58,230][123614] Updated weights for policy 1, policy_version 13880 (0.0008) [2023-10-10 17:06:58,788][122664] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 28475392. Throughput: 0: 1806.1, 1: 1805.6. Samples: 7117896. Policy #0 lag: (min: 18.0, avg: 19.1, max: 40.0) [2023-10-10 17:06:58,789][122664] Avg episode reward: [(0, '24.890'), (1, '21.870')] [2023-10-10 17:07:01,877][123582] Updated weights for policy 0, policy_version 13923 (0.0010) [2023-10-10 17:07:02,011][123614] Updated weights for policy 1, policy_version 13890 (0.0008) [2023-10-10 17:07:02,252][123582] Updated weights for policy 0, policy_version 13933 (0.0008) [2023-10-10 17:07:02,368][123614] Updated weights for policy 1, policy_version 13900 (0.0007) [2023-10-10 17:07:02,635][123582] Updated weights for policy 0, policy_version 13943 (0.0008) [2023-10-10 17:07:02,748][123614] Updated weights for policy 1, policy_version 13910 (0.0007) [2023-10-10 17:07:03,115][123614] Updated weights for policy 1, policy_version 13920 (0.0008) [2023-10-10 17:07:03,788][122664] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 28540928. Throughput: 0: 1813.7, 1: 1810.7. Samples: 7138690. Policy #0 lag: (min: 18.0, avg: 19.1, max: 40.0) [2023-10-10 17:07:03,789][122664] Avg episode reward: [(0, '25.620'), (1, '21.870')] [2023-10-10 17:07:06,348][123582] Updated weights for policy 0, policy_version 13953 (0.0008) [2023-10-10 17:07:06,725][123582] Updated weights for policy 0, policy_version 13963 (0.0008) [2023-10-10 17:07:06,747][123614] Updated weights for policy 1, policy_version 13930 (0.0008) [2023-10-10 17:07:07,089][123582] Updated weights for policy 0, policy_version 13973 (0.0009) [2023-10-10 17:07:07,117][123614] Updated weights for policy 1, policy_version 13940 (0.0007) [2023-10-10 17:07:07,459][123582] Updated weights for policy 0, policy_version 13983 (0.0007) [2023-10-10 17:07:07,482][123614] Updated weights for policy 1, policy_version 13950 (0.0007) [2023-10-10 17:07:08,788][122664] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 28606464. Throughput: 0: 1798.1, 1: 1797.3. Samples: 7160018. Policy #0 lag: (min: 24.0, avg: 50.4, max: 56.0) [2023-10-10 17:07:08,788][122664] Avg episode reward: [(0, '25.600'), (1, '24.030')] [2023-10-10 17:07:11,172][123582] Updated weights for policy 0, policy_version 13993 (0.0009) [2023-10-10 17:07:11,246][123614] Updated weights for policy 1, policy_version 13960 (0.0007) [2023-10-10 17:07:11,546][123582] Updated weights for policy 0, policy_version 14003 (0.0007) [2023-10-10 17:07:11,630][123614] Updated weights for policy 1, policy_version 13970 (0.0009) [2023-10-10 17:07:11,920][123582] Updated weights for policy 0, policy_version 14013 (0.0007) [2023-10-10 17:07:12,000][123614] Updated weights for policy 1, policy_version 13980 (0.0008) [2023-10-10 17:07:13,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 28672000. Throughput: 0: 1810.2, 1: 1804.8. Samples: 7171156. Policy #0 lag: (min: 24.0, avg: 50.4, max: 56.0) [2023-10-10 17:07:13,789][122664] Avg episode reward: [(0, '26.760'), (1, '24.820')] [2023-10-10 17:07:15,711][123614] Updated weights for policy 1, policy_version 13990 (0.0007) [2023-10-10 17:07:15,739][123582] Updated weights for policy 0, policy_version 14023 (0.0008) [2023-10-10 17:07:16,065][123614] Updated weights for policy 1, policy_version 14000 (0.0007) [2023-10-10 17:07:16,101][123582] Updated weights for policy 0, policy_version 14033 (0.0007) [2023-10-10 17:07:16,434][123614] Updated weights for policy 1, policy_version 14010 (0.0008) [2023-10-10 17:07:16,477][123582] Updated weights for policy 0, policy_version 14043 (0.0009) [2023-10-10 17:07:18,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 28737536. Throughput: 0: 1802.8, 1: 1799.3. Samples: 7192626. Policy #0 lag: (min: 24.0, avg: 50.4, max: 56.0) [2023-10-10 17:07:18,789][122664] Avg episode reward: [(0, '23.700'), (1, '25.000')] [2023-10-10 17:07:20,091][123582] Updated weights for policy 0, policy_version 14053 (0.0009) [2023-10-10 17:07:20,182][123614] Updated weights for policy 1, policy_version 14020 (0.0008) [2023-10-10 17:07:20,461][123582] Updated weights for policy 0, policy_version 14063 (0.0007) [2023-10-10 17:07:20,557][123614] Updated weights for policy 1, policy_version 14030 (0.0008) [2023-10-10 17:07:20,830][123582] Updated weights for policy 0, policy_version 14073 (0.0009) [2023-10-10 17:07:20,924][123614] Updated weights for policy 1, policy_version 14040 (0.0008) [2023-10-10 17:07:23,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 28803072. Throughput: 0: 1809.1, 1: 1800.4. Samples: 7215484. Policy #0 lag: (min: 3.0, avg: 11.0, max: 35.0) [2023-10-10 17:07:23,789][122664] Avg episode reward: [(0, '20.720'), (1, '25.420')] [2023-10-10 17:07:24,343][123582] Updated weights for policy 0, policy_version 14083 (0.0007) [2023-10-10 17:07:24,708][123582] Updated weights for policy 0, policy_version 14093 (0.0009) [2023-10-10 17:07:24,760][123614] Updated weights for policy 1, policy_version 14050 (0.0008) [2023-10-10 17:07:25,077][123582] Updated weights for policy 0, policy_version 14103 (0.0007) [2023-10-10 17:07:25,122][123614] Updated weights for policy 1, policy_version 14060 (0.0008) [2023-10-10 17:07:25,501][123614] Updated weights for policy 1, policy_version 14070 (0.0008) [2023-10-10 17:07:25,864][123614] Updated weights for policy 1, policy_version 14080 (0.0008) [2023-10-10 17:07:28,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 28868608. Throughput: 0: 1810.9, 1: 1798.3. Samples: 7225236. Policy #0 lag: (min: 3.0, avg: 11.0, max: 35.0) [2023-10-10 17:07:28,788][122664] Avg episode reward: [(0, '22.530'), (1, '23.900')] [2023-10-10 17:07:28,951][123582] Updated weights for policy 0, policy_version 14113 (0.0007) [2023-10-10 17:07:29,348][123582] Updated weights for policy 0, policy_version 14123 (0.0007) [2023-10-10 17:07:29,584][123614] Updated weights for policy 1, policy_version 14090 (0.0007) [2023-10-10 17:07:29,711][123582] Updated weights for policy 0, policy_version 14133 (0.0007) [2023-10-10 17:07:29,952][123614] Updated weights for policy 1, policy_version 14100 (0.0008) [2023-10-10 17:07:30,081][123582] Updated weights for policy 0, policy_version 14143 (0.0007) [2023-10-10 17:07:30,321][123614] Updated weights for policy 1, policy_version 14110 (0.0008) [2023-10-10 17:07:33,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 28934144. Throughput: 0: 1809.7, 1: 1793.7. Samples: 7247538. Policy #0 lag: (min: 3.0, avg: 11.0, max: 35.0) [2023-10-10 17:07:33,789][122664] Avg episode reward: [(0, '21.960'), (1, '22.530')] [2023-10-10 17:07:33,821][123582] Updated weights for policy 0, policy_version 14153 (0.0009) [2023-10-10 17:07:34,013][123614] Updated weights for policy 1, policy_version 14120 (0.0010) [2023-10-10 17:07:34,198][123582] Updated weights for policy 0, policy_version 14163 (0.0009) [2023-10-10 17:07:34,393][123614] Updated weights for policy 1, policy_version 14130 (0.0007) [2023-10-10 17:07:34,568][123582] Updated weights for policy 0, policy_version 14173 (0.0009) [2023-10-10 17:07:34,755][123614] Updated weights for policy 1, policy_version 14140 (0.0009) [2023-10-10 17:07:38,333][123582] Updated weights for policy 0, policy_version 14183 (0.0008) [2023-10-10 17:07:38,420][123614] Updated weights for policy 1, policy_version 14150 (0.0008) [2023-10-10 17:07:38,699][123582] Updated weights for policy 0, policy_version 14193 (0.0008) [2023-10-10 17:07:38,785][123614] Updated weights for policy 1, policy_version 14160 (0.0008) [2023-10-10 17:07:38,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 28999680. Throughput: 0: 1814.6, 1: 1807.6. Samples: 7269066. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-10 17:07:38,789][122664] Avg episode reward: [(0, '21.680'), (1, '23.160')] [2023-10-10 17:07:39,077][123582] Updated weights for policy 0, policy_version 14203 (0.0008) [2023-10-10 17:07:39,163][123614] Updated weights for policy 1, policy_version 14170 (0.0009) [2023-10-10 17:07:42,634][123582] Updated weights for policy 0, policy_version 14213 (0.0009) [2023-10-10 17:07:42,996][123614] Updated weights for policy 1, policy_version 14180 (0.0007) [2023-10-10 17:07:43,003][123582] Updated weights for policy 0, policy_version 14223 (0.0010) [2023-10-10 17:07:43,363][123614] Updated weights for policy 1, policy_version 14190 (0.0007) [2023-10-10 17:07:43,378][123582] Updated weights for policy 0, policy_version 14233 (0.0007) [2023-10-10 17:07:43,745][123614] Updated weights for policy 1, policy_version 14200 (0.0008) [2023-10-10 17:07:43,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 29097984. Throughput: 0: 1804.1, 1: 1794.9. Samples: 7279850. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-10 17:07:43,789][122664] Avg episode reward: [(0, '22.680'), (1, '21.840')] [2023-10-10 17:07:46,957][123582] Updated weights for policy 0, policy_version 14243 (0.0007) [2023-10-10 17:07:47,326][123582] Updated weights for policy 0, policy_version 14253 (0.0008) [2023-10-10 17:07:47,375][123614] Updated weights for policy 1, policy_version 14210 (0.0008) [2023-10-10 17:07:47,701][123582] Updated weights for policy 0, policy_version 14263 (0.0010) [2023-10-10 17:07:47,739][123614] Updated weights for policy 1, policy_version 14220 (0.0009) [2023-10-10 17:07:48,113][123614] Updated weights for policy 1, policy_version 14230 (0.0009) [2023-10-10 17:07:48,475][123614] Updated weights for policy 1, policy_version 14240 (0.0009) [2023-10-10 17:07:48,788][122664] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 29196288. Throughput: 0: 1811.2, 1: 1807.9. Samples: 7301548. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:07:48,789][122664] Avg episode reward: [(0, '25.330'), (1, '22.740')] [2023-10-10 17:07:51,453][123582] Updated weights for policy 0, policy_version 14273 (0.0008) [2023-10-10 17:07:51,825][123582] Updated weights for policy 0, policy_version 14283 (0.0010) [2023-10-10 17:07:52,198][123582] Updated weights for policy 0, policy_version 14293 (0.0007) [2023-10-10 17:07:52,284][123614] Updated weights for policy 1, policy_version 14250 (0.0008) [2023-10-10 17:07:52,571][123582] Updated weights for policy 0, policy_version 14303 (0.0008) [2023-10-10 17:07:52,656][123614] Updated weights for policy 1, policy_version 14260 (0.0008) [2023-10-10 17:07:53,023][123614] Updated weights for policy 1, policy_version 14270 (0.0010) [2023-10-10 17:07:53,788][122664] Fps is (10 sec: 16384.4, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 29261824. Throughput: 0: 1808.4, 1: 1796.4. Samples: 7322232. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:07:53,788][122664] Avg episode reward: [(0, '26.470'), (1, '21.830')] [2023-10-10 17:07:53,796][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000014272_14614528.pth... [2023-10-10 17:07:53,796][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000014304_14647296.pth... [2023-10-10 17:07:53,826][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000012608_12910592.pth [2023-10-10 17:07:53,835][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000012576_12877824.pth [2023-10-10 17:07:56,309][123582] Updated weights for policy 0, policy_version 14313 (0.0010) [2023-10-10 17:07:56,673][123614] Updated weights for policy 1, policy_version 14280 (0.0007) [2023-10-10 17:07:56,679][123582] Updated weights for policy 0, policy_version 14323 (0.0010) [2023-10-10 17:07:57,042][123614] Updated weights for policy 1, policy_version 14290 (0.0007) [2023-10-10 17:07:57,056][123582] Updated weights for policy 0, policy_version 14333 (0.0008) [2023-10-10 17:07:57,412][123614] Updated weights for policy 1, policy_version 14300 (0.0010) [2023-10-10 17:07:58,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 29327360. Throughput: 0: 1814.6, 1: 1812.4. Samples: 7334372. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:07:58,789][122664] Avg episode reward: [(0, '26.160'), (1, '24.130')] [2023-10-10 17:08:00,806][123582] Updated weights for policy 0, policy_version 14343 (0.0007) [2023-10-10 17:08:01,195][123582] Updated weights for policy 0, policy_version 14353 (0.0009) [2023-10-10 17:08:01,205][123614] Updated weights for policy 1, policy_version 14310 (0.0008) [2023-10-10 17:08:01,567][123582] Updated weights for policy 0, policy_version 14363 (0.0010) [2023-10-10 17:08:01,569][123614] Updated weights for policy 1, policy_version 14320 (0.0008) [2023-10-10 17:08:01,937][123614] Updated weights for policy 1, policy_version 14330 (0.0008) [2023-10-10 17:08:03,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 29392896. Throughput: 0: 1809.3, 1: 1794.5. Samples: 7354794. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-10 17:08:03,788][122664] Avg episode reward: [(0, '24.220'), (1, '23.540')] [2023-10-10 17:08:05,343][123582] Updated weights for policy 0, policy_version 14373 (0.0008) [2023-10-10 17:08:05,712][123582] Updated weights for policy 0, policy_version 14383 (0.0009) [2023-10-10 17:08:05,825][123614] Updated weights for policy 1, policy_version 14340 (0.0008) [2023-10-10 17:08:06,086][123582] Updated weights for policy 0, policy_version 14393 (0.0009) [2023-10-10 17:08:06,193][123614] Updated weights for policy 1, policy_version 14350 (0.0008) [2023-10-10 17:08:06,563][123614] Updated weights for policy 1, policy_version 14360 (0.0007) [2023-10-10 17:08:08,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 29458432. Throughput: 0: 1803.2, 1: 1791.7. Samples: 7377254. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-10 17:08:08,789][122664] Avg episode reward: [(0, '22.010'), (1, '25.160')] [2023-10-10 17:08:09,734][123582] Updated weights for policy 0, policy_version 14403 (0.0008) [2023-10-10 17:08:10,109][123582] Updated weights for policy 0, policy_version 14413 (0.0009) [2023-10-10 17:08:10,451][123614] Updated weights for policy 1, policy_version 14370 (0.0007) [2023-10-10 17:08:10,485][123582] Updated weights for policy 0, policy_version 14423 (0.0008) [2023-10-10 17:08:10,817][123614] Updated weights for policy 1, policy_version 14380 (0.0008) [2023-10-10 17:08:11,187][123614] Updated weights for policy 1, policy_version 14390 (0.0009) [2023-10-10 17:08:11,567][123614] Updated weights for policy 1, policy_version 14400 (0.0008) [2023-10-10 17:08:13,788][122664] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 29523968. Throughput: 0: 1803.9, 1: 1791.3. Samples: 7387018. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-10 17:08:13,789][122664] Avg episode reward: [(0, '18.190'), (1, '23.680')] [2023-10-10 17:08:14,082][123582] Updated weights for policy 0, policy_version 14433 (0.0008) [2023-10-10 17:08:14,492][123582] Updated weights for policy 0, policy_version 14443 (0.0009) [2023-10-10 17:08:14,860][123582] Updated weights for policy 0, policy_version 14453 (0.0009) [2023-10-10 17:08:15,202][123614] Updated weights for policy 1, policy_version 14410 (0.0008) [2023-10-10 17:08:15,235][123582] Updated weights for policy 0, policy_version 14463 (0.0010) [2023-10-10 17:08:15,567][123614] Updated weights for policy 1, policy_version 14420 (0.0010) [2023-10-10 17:08:15,947][123614] Updated weights for policy 1, policy_version 14430 (0.0008) [2023-10-10 17:08:18,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 29589504. Throughput: 0: 1809.2, 1: 1795.8. Samples: 7409762. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 17:08:18,788][122664] Avg episode reward: [(0, '16.900'), (1, '26.740')] [2023-10-10 17:08:18,789][123465] Saving new best policy, reward=26.740! [2023-10-10 17:08:18,987][123582] Updated weights for policy 0, policy_version 14473 (0.0007) [2023-10-10 17:08:19,350][123582] Updated weights for policy 0, policy_version 14483 (0.0008) [2023-10-10 17:08:19,683][123614] Updated weights for policy 1, policy_version 14440 (0.0007) [2023-10-10 17:08:19,732][123582] Updated weights for policy 0, policy_version 14493 (0.0009) [2023-10-10 17:08:20,057][123614] Updated weights for policy 1, policy_version 14450 (0.0009) [2023-10-10 17:08:20,424][123614] Updated weights for policy 1, policy_version 14460 (0.0008) [2023-10-10 17:08:23,407][123582] Updated weights for policy 0, policy_version 14503 (0.0011) [2023-10-10 17:08:23,775][123582] Updated weights for policy 0, policy_version 14513 (0.0011) [2023-10-10 17:08:23,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 29655040. Throughput: 0: 1814.4, 1: 1808.5. Samples: 7432098. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 17:08:23,789][122664] Avg episode reward: [(0, '17.130'), (1, '25.780')] [2023-10-10 17:08:24,143][123582] Updated weights for policy 0, policy_version 14523 (0.0009) [2023-10-10 17:08:24,243][123614] Updated weights for policy 1, policy_version 14470 (0.0009) [2023-10-10 17:08:24,613][123614] Updated weights for policy 1, policy_version 14480 (0.0008) [2023-10-10 17:08:24,977][123614] Updated weights for policy 1, policy_version 14490 (0.0007) [2023-10-10 17:08:27,824][123582] Updated weights for policy 0, policy_version 14533 (0.0008) [2023-10-10 17:08:28,199][123582] Updated weights for policy 0, policy_version 14543 (0.0008) [2023-10-10 17:08:28,570][123582] Updated weights for policy 0, policy_version 14553 (0.0007) [2023-10-10 17:08:28,667][123614] Updated weights for policy 1, policy_version 14500 (0.0007) [2023-10-10 17:08:28,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 29720576. Throughput: 0: 1813.1, 1: 1798.7. Samples: 7442380. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 17:08:28,789][122664] Avg episode reward: [(0, '17.900'), (1, '23.390')] [2023-10-10 17:08:29,028][123614] Updated weights for policy 1, policy_version 14510 (0.0009) [2023-10-10 17:08:29,397][123614] Updated weights for policy 1, policy_version 14520 (0.0010) [2023-10-10 17:08:32,338][123582] Updated weights for policy 0, policy_version 14563 (0.0009) [2023-10-10 17:08:32,718][123582] Updated weights for policy 0, policy_version 14573 (0.0007) [2023-10-10 17:08:33,031][123614] Updated weights for policy 1, policy_version 14530 (0.0009) [2023-10-10 17:08:33,083][123582] Updated weights for policy 0, policy_version 14583 (0.0008) [2023-10-10 17:08:33,392][123614] Updated weights for policy 1, policy_version 14540 (0.0007) [2023-10-10 17:08:33,764][123614] Updated weights for policy 1, policy_version 14550 (0.0007) [2023-10-10 17:08:33,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 29818880. Throughput: 0: 1820.7, 1: 1805.3. Samples: 7464714. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:08:33,788][122664] Avg episode reward: [(0, '19.530'), (1, '21.570')] [2023-10-10 17:08:34,130][123614] Updated weights for policy 1, policy_version 14560 (0.0007) [2023-10-10 17:08:36,784][123582] Updated weights for policy 0, policy_version 14593 (0.0008) [2023-10-10 17:08:37,155][123582] Updated weights for policy 0, policy_version 14603 (0.0010) [2023-10-10 17:08:37,525][123582] Updated weights for policy 0, policy_version 14613 (0.0009) [2023-10-10 17:08:37,695][123614] Updated weights for policy 1, policy_version 14570 (0.0008) [2023-10-10 17:08:37,903][123582] Updated weights for policy 0, policy_version 14623 (0.0010) [2023-10-10 17:08:38,063][123614] Updated weights for policy 1, policy_version 14580 (0.0008) [2023-10-10 17:08:38,419][123614] Updated weights for policy 1, policy_version 14590 (0.0009) [2023-10-10 17:08:38,788][122664] Fps is (10 sec: 19661.2, 60 sec: 15291.8, 300 sec: 14551.2). Total num frames: 29917184. Throughput: 0: 1810.0, 1: 1800.0. Samples: 7484680. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:08:38,788][122664] Avg episode reward: [(0, '20.950'), (1, '22.790')] [2023-10-10 17:08:41,710][123582] Updated weights for policy 0, policy_version 14633 (0.0010) [2023-10-10 17:08:42,083][123582] Updated weights for policy 0, policy_version 14643 (0.0009) [2023-10-10 17:08:42,265][123614] Updated weights for policy 1, policy_version 14600 (0.0007) [2023-10-10 17:08:42,448][123582] Updated weights for policy 0, policy_version 14653 (0.0008) [2023-10-10 17:08:42,621][123614] Updated weights for policy 1, policy_version 14610 (0.0007) [2023-10-10 17:08:42,993][123614] Updated weights for policy 1, policy_version 14620 (0.0009) [2023-10-10 17:08:43,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 29982720. Throughput: 0: 1816.4, 1: 1800.0. Samples: 7497108. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 17:08:43,789][122664] Avg episode reward: [(0, '21.110'), (1, '22.520')] [2023-10-10 17:08:46,264][123582] Updated weights for policy 0, policy_version 14663 (0.0009) [2023-10-10 17:08:46,646][123582] Updated weights for policy 0, policy_version 14673 (0.0009) [2023-10-10 17:08:46,794][123614] Updated weights for policy 1, policy_version 14630 (0.0009) [2023-10-10 17:08:47,028][123582] Updated weights for policy 0, policy_version 14683 (0.0009) [2023-10-10 17:08:47,161][123614] Updated weights for policy 1, policy_version 14640 (0.0007) [2023-10-10 17:08:47,530][123614] Updated weights for policy 1, policy_version 14650 (0.0007) [2023-10-10 17:08:48,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 30048256. Throughput: 0: 1803.9, 1: 1799.6. Samples: 7516950. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 17:08:48,789][122664] Avg episode reward: [(0, '23.310'), (1, '23.240')] [2023-10-10 17:08:50,761][123582] Updated weights for policy 0, policy_version 14693 (0.0007) [2023-10-10 17:08:51,106][123614] Updated weights for policy 1, policy_version 14660 (0.0010) [2023-10-10 17:08:51,130][123582] Updated weights for policy 0, policy_version 14703 (0.0008) [2023-10-10 17:08:51,471][123614] Updated weights for policy 1, policy_version 14670 (0.0009) [2023-10-10 17:08:51,517][123582] Updated weights for policy 0, policy_version 14713 (0.0007) [2023-10-10 17:08:51,834][123614] Updated weights for policy 1, policy_version 14680 (0.0008) [2023-10-10 17:08:53,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 30113792. Throughput: 0: 1802.4, 1: 1800.1. Samples: 7539366. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 17:08:53,788][122664] Avg episode reward: [(0, '23.360'), (1, '21.860')] [2023-10-10 17:08:55,274][123582] Updated weights for policy 0, policy_version 14723 (0.0008) [2023-10-10 17:08:55,654][123582] Updated weights for policy 0, policy_version 14733 (0.0007) [2023-10-10 17:08:55,720][123614] Updated weights for policy 1, policy_version 14690 (0.0010) [2023-10-10 17:08:56,016][123582] Updated weights for policy 0, policy_version 14743 (0.0009) [2023-10-10 17:08:56,081][123614] Updated weights for policy 1, policy_version 14700 (0.0007) [2023-10-10 17:08:56,446][123614] Updated weights for policy 1, policy_version 14710 (0.0007) [2023-10-10 17:08:56,814][123614] Updated weights for policy 1, policy_version 14720 (0.0007) [2023-10-10 17:08:58,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 30179328. Throughput: 0: 1798.2, 1: 1807.3. Samples: 7549262. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-10 17:08:58,789][122664] Avg episode reward: [(0, '22.400'), (1, '23.600')] [2023-10-10 17:08:59,619][123582] Updated weights for policy 0, policy_version 14753 (0.0009) [2023-10-10 17:08:59,983][123582] Updated weights for policy 0, policy_version 14763 (0.0009) [2023-10-10 17:09:00,362][123582] Updated weights for policy 0, policy_version 14773 (0.0007) [2023-10-10 17:09:00,531][123614] Updated weights for policy 1, policy_version 14730 (0.0008) [2023-10-10 17:09:00,732][123582] Updated weights for policy 0, policy_version 14783 (0.0007) [2023-10-10 17:09:00,891][123614] Updated weights for policy 1, policy_version 14740 (0.0009) [2023-10-10 17:09:01,265][123614] Updated weights for policy 1, policy_version 14750 (0.0009) [2023-10-10 17:09:03,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 30244864. Throughput: 0: 1806.1, 1: 1793.0. Samples: 7571722. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-10 17:09:03,788][122664] Avg episode reward: [(0, '20.790'), (1, '22.340')] [2023-10-10 17:09:04,413][123582] Updated weights for policy 0, policy_version 14793 (0.0009) [2023-10-10 17:09:04,779][123582] Updated weights for policy 0, policy_version 14803 (0.0009) [2023-10-10 17:09:05,064][123614] Updated weights for policy 1, policy_version 14760 (0.0007) [2023-10-10 17:09:05,151][123582] Updated weights for policy 0, policy_version 14813 (0.0007) [2023-10-10 17:09:05,429][123614] Updated weights for policy 1, policy_version 14770 (0.0009) [2023-10-10 17:09:05,797][123614] Updated weights for policy 1, policy_version 14780 (0.0007) [2023-10-10 17:09:08,693][123582] Updated weights for policy 0, policy_version 14823 (0.0008) [2023-10-10 17:09:08,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 30310400. Throughput: 0: 1819.4, 1: 1793.4. Samples: 7594674. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-10 17:09:08,789][122664] Avg episode reward: [(0, '22.140'), (1, '22.800')] [2023-10-10 17:09:09,066][123582] Updated weights for policy 0, policy_version 14833 (0.0007) [2023-10-10 17:09:09,441][123582] Updated weights for policy 0, policy_version 14843 (0.0008) [2023-10-10 17:09:09,576][123614] Updated weights for policy 1, policy_version 14790 (0.0008) [2023-10-10 17:09:09,956][123614] Updated weights for policy 1, policy_version 14800 (0.0007) [2023-10-10 17:09:10,327][123614] Updated weights for policy 1, policy_version 14810 (0.0007) [2023-10-10 17:09:13,059][123582] Updated weights for policy 0, policy_version 14853 (0.0010) [2023-10-10 17:09:13,429][123582] Updated weights for policy 0, policy_version 14863 (0.0008) [2023-10-10 17:09:13,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 30375936. Throughput: 0: 1815.4, 1: 1789.1. Samples: 7604584. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:09:13,789][122664] Avg episode reward: [(0, '21.820'), (1, '22.280')] [2023-10-10 17:09:13,806][123582] Updated weights for policy 0, policy_version 14873 (0.0007) [2023-10-10 17:09:14,098][123614] Updated weights for policy 1, policy_version 14820 (0.0008) [2023-10-10 17:09:14,464][123614] Updated weights for policy 1, policy_version 14830 (0.0009) [2023-10-10 17:09:14,827][123614] Updated weights for policy 1, policy_version 14840 (0.0007) [2023-10-10 17:09:17,494][123582] Updated weights for policy 0, policy_version 14883 (0.0007) [2023-10-10 17:09:17,867][123582] Updated weights for policy 0, policy_version 14893 (0.0009) [2023-10-10 17:09:18,237][123582] Updated weights for policy 0, policy_version 14903 (0.0009) [2023-10-10 17:09:18,523][123614] Updated weights for policy 1, policy_version 14850 (0.0007) [2023-10-10 17:09:18,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 30474240. Throughput: 0: 1821.5, 1: 1799.8. Samples: 7627672. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:09:18,789][122664] Avg episode reward: [(0, '18.010'), (1, '22.570')] [2023-10-10 17:09:18,893][123614] Updated weights for policy 1, policy_version 14860 (0.0011) [2023-10-10 17:09:19,274][123614] Updated weights for policy 1, policy_version 14870 (0.0008) [2023-10-10 17:09:19,639][123614] Updated weights for policy 1, policy_version 14880 (0.0008) [2023-10-10 17:09:21,840][123582] Updated weights for policy 0, policy_version 14913 (0.0009) [2023-10-10 17:09:22,207][123582] Updated weights for policy 0, policy_version 14923 (0.0008) [2023-10-10 17:09:22,573][123582] Updated weights for policy 0, policy_version 14933 (0.0007) [2023-10-10 17:09:22,947][123582] Updated weights for policy 0, policy_version 14943 (0.0009) [2023-10-10 17:09:23,301][123614] Updated weights for policy 1, policy_version 14890 (0.0008) [2023-10-10 17:09:23,679][123614] Updated weights for policy 1, policy_version 14900 (0.0008) [2023-10-10 17:09:23,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 30539776. Throughput: 0: 1821.1, 1: 1812.7. Samples: 7648204. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-10 17:09:23,789][122664] Avg episode reward: [(0, '19.570'), (1, '23.210')] [2023-10-10 17:09:24,043][123614] Updated weights for policy 1, policy_version 14910 (0.0008) [2023-10-10 17:09:26,713][123582] Updated weights for policy 0, policy_version 14953 (0.0011) [2023-10-10 17:09:27,087][123582] Updated weights for policy 0, policy_version 14963 (0.0009) [2023-10-10 17:09:27,457][123582] Updated weights for policy 0, policy_version 14973 (0.0007) [2023-10-10 17:09:27,769][123614] Updated weights for policy 1, policy_version 14920 (0.0008) [2023-10-10 17:09:28,143][123614] Updated weights for policy 1, policy_version 14930 (0.0007) [2023-10-10 17:09:28,515][123614] Updated weights for policy 1, policy_version 14940 (0.0007) [2023-10-10 17:09:28,788][122664] Fps is (10 sec: 16384.4, 60 sec: 15291.8, 300 sec: 14551.2). Total num frames: 30638080. Throughput: 0: 1825.0, 1: 1805.2. Samples: 7660466. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-10 17:09:28,788][122664] Avg episode reward: [(0, '19.900'), (1, '23.140')] [2023-10-10 17:09:31,092][123582] Updated weights for policy 0, policy_version 14983 (0.0010) [2023-10-10 17:09:31,473][123582] Updated weights for policy 0, policy_version 14993 (0.0010) [2023-10-10 17:09:31,857][123582] Updated weights for policy 0, policy_version 15003 (0.0007) [2023-10-10 17:09:32,142][123614] Updated weights for policy 1, policy_version 14950 (0.0008) [2023-10-10 17:09:32,509][123614] Updated weights for policy 1, policy_version 14960 (0.0008) [2023-10-10 17:09:32,876][123614] Updated weights for policy 1, policy_version 14970 (0.0010) [2023-10-10 17:09:33,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 30703616. Throughput: 0: 1830.0, 1: 1812.7. Samples: 7680876. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-10 17:09:33,789][122664] Avg episode reward: [(0, '19.250'), (1, '22.250')] [2023-10-10 17:09:35,404][123582] Updated weights for policy 0, policy_version 15013 (0.0008) [2023-10-10 17:09:35,778][123582] Updated weights for policy 0, policy_version 15023 (0.0007) [2023-10-10 17:09:36,154][123582] Updated weights for policy 0, policy_version 15033 (0.0010) [2023-10-10 17:09:36,467][123614] Updated weights for policy 1, policy_version 14980 (0.0011) [2023-10-10 17:09:36,834][123614] Updated weights for policy 1, policy_version 14990 (0.0008) [2023-10-10 17:09:37,206][123614] Updated weights for policy 1, policy_version 15000 (0.0009) [2023-10-10 17:09:38,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 30769152. Throughput: 0: 1835.2, 1: 1806.2. Samples: 7703230. Policy #0 lag: (min: 31.0, avg: 32.9, max: 61.0) [2023-10-10 17:09:38,789][122664] Avg episode reward: [(0, '19.090'), (1, '24.210')] [2023-10-10 17:09:39,774][123582] Updated weights for policy 0, policy_version 15043 (0.0009) [2023-10-10 17:09:40,140][123582] Updated weights for policy 0, policy_version 15053 (0.0007) [2023-10-10 17:09:40,506][123582] Updated weights for policy 0, policy_version 15063 (0.0007) [2023-10-10 17:09:40,936][123614] Updated weights for policy 1, policy_version 15010 (0.0010) [2023-10-10 17:09:41,310][123614] Updated weights for policy 1, policy_version 15020 (0.0007) [2023-10-10 17:09:41,676][123614] Updated weights for policy 1, policy_version 15030 (0.0011) [2023-10-10 17:09:42,053][123614] Updated weights for policy 1, policy_version 15040 (0.0007) [2023-10-10 17:09:43,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 30834688. Throughput: 0: 1839.9, 1: 1813.4. Samples: 7713662. Policy #0 lag: (min: 31.0, avg: 32.9, max: 61.0) [2023-10-10 17:09:43,789][122664] Avg episode reward: [(0, '19.910'), (1, '25.430')] [2023-10-10 17:09:44,212][123582] Updated weights for policy 0, policy_version 15073 (0.0010) [2023-10-10 17:09:44,580][123582] Updated weights for policy 0, policy_version 15083 (0.0009) [2023-10-10 17:09:44,951][123582] Updated weights for policy 0, policy_version 15093 (0.0008) [2023-10-10 17:09:45,321][123582] Updated weights for policy 0, policy_version 15103 (0.0008) [2023-10-10 17:09:45,702][123614] Updated weights for policy 1, policy_version 15050 (0.0010) [2023-10-10 17:09:46,074][123614] Updated weights for policy 1, policy_version 15060 (0.0009) [2023-10-10 17:09:46,454][123614] Updated weights for policy 1, policy_version 15070 (0.0009) [2023-10-10 17:09:48,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 30900224. Throughput: 0: 1838.0, 1: 1811.1. Samples: 7735932. Policy #0 lag: (min: 31.0, avg: 32.9, max: 61.0) [2023-10-10 17:09:48,789][122664] Avg episode reward: [(0, '20.610'), (1, '25.260')] [2023-10-10 17:09:48,808][123582] Updated weights for policy 0, policy_version 15113 (0.0011) [2023-10-10 17:09:49,174][123582] Updated weights for policy 0, policy_version 15123 (0.0010) [2023-10-10 17:09:49,551][123582] Updated weights for policy 0, policy_version 15133 (0.0011) [2023-10-10 17:09:50,411][123614] Updated weights for policy 1, policy_version 15080 (0.0009) [2023-10-10 17:09:50,787][123614] Updated weights for policy 1, policy_version 15090 (0.0009) [2023-10-10 17:09:51,156][123614] Updated weights for policy 1, policy_version 15100 (0.0007) [2023-10-10 17:09:53,231][123582] Updated weights for policy 0, policy_version 15143 (0.0009) [2023-10-10 17:09:53,618][123582] Updated weights for policy 0, policy_version 15153 (0.0009) [2023-10-10 17:09:53,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 30965760. Throughput: 0: 1825.3, 1: 1816.8. Samples: 7758570. Policy #0 lag: (min: 32.0, avg: 53.8, max: 56.0) [2023-10-10 17:09:53,789][122664] Avg episode reward: [(0, '23.150'), (1, '25.800')] [2023-10-10 17:09:53,800][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000015104_15466496.pth... [2023-10-10 17:09:53,835][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000013408_13729792.pth [2023-10-10 17:09:53,991][123582] Updated weights for policy 0, policy_version 15163 (0.0007) [2023-10-10 17:09:54,172][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000015168_15532032.pth... [2023-10-10 17:09:54,203][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000013472_13795328.pth [2023-10-10 17:09:54,930][123614] Updated weights for policy 1, policy_version 15110 (0.0009) [2023-10-10 17:09:55,313][123614] Updated weights for policy 1, policy_version 15120 (0.0010) [2023-10-10 17:09:55,675][123614] Updated weights for policy 1, policy_version 15130 (0.0010) [2023-10-10 17:09:57,729][123582] Updated weights for policy 0, policy_version 15173 (0.0010) [2023-10-10 17:09:58,101][123582] Updated weights for policy 0, policy_version 15183 (0.0008) [2023-10-10 17:09:58,472][123582] Updated weights for policy 0, policy_version 15193 (0.0007) [2023-10-10 17:09:58,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 31064064. Throughput: 0: 1830.1, 1: 1818.8. Samples: 7768788. Policy #0 lag: (min: 32.0, avg: 53.8, max: 56.0) [2023-10-10 17:09:58,789][122664] Avg episode reward: [(0, '24.280'), (1, '24.030')] [2023-10-10 17:09:59,311][123614] Updated weights for policy 1, policy_version 15140 (0.0011) [2023-10-10 17:09:59,682][123614] Updated weights for policy 1, policy_version 15150 (0.0011) [2023-10-10 17:10:00,047][123614] Updated weights for policy 1, policy_version 15160 (0.0010) [2023-10-10 17:10:02,144][123582] Updated weights for policy 0, policy_version 15203 (0.0010) [2023-10-10 17:10:02,510][123582] Updated weights for policy 0, policy_version 15213 (0.0007) [2023-10-10 17:10:02,882][123582] Updated weights for policy 0, policy_version 15223 (0.0009) [2023-10-10 17:10:03,763][123614] Updated weights for policy 1, policy_version 15170 (0.0010) [2023-10-10 17:10:03,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 31129600. Throughput: 0: 1820.1, 1: 1810.1. Samples: 7791034. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:10:03,789][122664] Avg episode reward: [(0, '24.420'), (1, '25.500')] [2023-10-10 17:10:04,138][123614] Updated weights for policy 1, policy_version 15180 (0.0010) [2023-10-10 17:10:04,499][123614] Updated weights for policy 1, policy_version 15190 (0.0009) [2023-10-10 17:10:04,869][123614] Updated weights for policy 1, policy_version 15200 (0.0008) [2023-10-10 17:10:06,695][123582] Updated weights for policy 0, policy_version 15233 (0.0009) [2023-10-10 17:10:07,074][123582] Updated weights for policy 0, policy_version 15243 (0.0008) [2023-10-10 17:10:07,444][123582] Updated weights for policy 0, policy_version 15253 (0.0008) [2023-10-10 17:10:07,821][123582] Updated weights for policy 0, policy_version 15263 (0.0008) [2023-10-10 17:10:08,522][123614] Updated weights for policy 1, policy_version 15210 (0.0008) [2023-10-10 17:10:08,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 31195136. Throughput: 0: 1823.8, 1: 1812.6. Samples: 7811842. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:10:08,789][122664] Avg episode reward: [(0, '25.440'), (1, '25.250')] [2023-10-10 17:10:08,887][123614] Updated weights for policy 1, policy_version 15220 (0.0008) [2023-10-10 17:10:09,263][123614] Updated weights for policy 1, policy_version 15230 (0.0008) [2023-10-10 17:10:11,510][123582] Updated weights for policy 0, policy_version 15273 (0.0008) [2023-10-10 17:10:11,882][123582] Updated weights for policy 0, policy_version 15283 (0.0010) [2023-10-10 17:10:12,261][123582] Updated weights for policy 0, policy_version 15293 (0.0008) [2023-10-10 17:10:12,865][123614] Updated weights for policy 1, policy_version 15240 (0.0010) [2023-10-10 17:10:13,231][123614] Updated weights for policy 1, policy_version 15250 (0.0011) [2023-10-10 17:10:13,602][123614] Updated weights for policy 1, policy_version 15260 (0.0007) [2023-10-10 17:10:13,788][122664] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 31293440. Throughput: 0: 1817.1, 1: 1807.7. Samples: 7823586. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:10:13,789][122664] Avg episode reward: [(0, '24.960'), (1, '23.640')] [2023-10-10 17:10:16,013][123582] Updated weights for policy 0, policy_version 15303 (0.0010) [2023-10-10 17:10:16,397][123582] Updated weights for policy 0, policy_version 15313 (0.0009) [2023-10-10 17:10:16,771][123582] Updated weights for policy 0, policy_version 15323 (0.0010) [2023-10-10 17:10:17,379][123614] Updated weights for policy 1, policy_version 15270 (0.0009) [2023-10-10 17:10:17,754][123614] Updated weights for policy 1, policy_version 15280 (0.0011) [2023-10-10 17:10:18,125][123614] Updated weights for policy 1, policy_version 15290 (0.0011) [2023-10-10 17:10:18,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 31358976. Throughput: 0: 1819.4, 1: 1811.4. Samples: 7844260. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:10:18,789][122664] Avg episode reward: [(0, '23.560'), (1, '21.760')] [2023-10-10 17:10:20,462][123582] Updated weights for policy 0, policy_version 15333 (0.0010) [2023-10-10 17:10:20,835][123582] Updated weights for policy 0, policy_version 15343 (0.0009) [2023-10-10 17:10:21,203][123582] Updated weights for policy 0, policy_version 15353 (0.0008) [2023-10-10 17:10:21,851][123614] Updated weights for policy 1, policy_version 15300 (0.0010) [2023-10-10 17:10:22,220][123614] Updated weights for policy 1, policy_version 15310 (0.0008) [2023-10-10 17:10:22,592][123614] Updated weights for policy 1, policy_version 15320 (0.0010) [2023-10-10 17:10:23,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 31424512. Throughput: 0: 1817.6, 1: 1804.3. Samples: 7866214. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:10:23,789][122664] Avg episode reward: [(0, '24.020'), (1, '21.750')] [2023-10-10 17:10:24,957][123582] Updated weights for policy 0, policy_version 15363 (0.0010) [2023-10-10 17:10:25,324][123582] Updated weights for policy 0, policy_version 15373 (0.0009) [2023-10-10 17:10:25,697][123582] Updated weights for policy 0, policy_version 15383 (0.0010) [2023-10-10 17:10:26,237][123614] Updated weights for policy 1, policy_version 15330 (0.0011) [2023-10-10 17:10:26,614][123614] Updated weights for policy 1, policy_version 15340 (0.0009) [2023-10-10 17:10:26,974][123614] Updated weights for policy 1, policy_version 15350 (0.0008) [2023-10-10 17:10:27,338][123614] Updated weights for policy 1, policy_version 15360 (0.0009) [2023-10-10 17:10:28,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 31490048. Throughput: 0: 1814.0, 1: 1816.6. Samples: 7877038. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:10:28,789][122664] Avg episode reward: [(0, '22.480'), (1, '20.250')] [2023-10-10 17:10:29,277][123582] Updated weights for policy 0, policy_version 15393 (0.0010) [2023-10-10 17:10:29,641][123582] Updated weights for policy 0, policy_version 15403 (0.0008) [2023-10-10 17:10:30,020][123582] Updated weights for policy 0, policy_version 15413 (0.0007) [2023-10-10 17:10:30,392][123582] Updated weights for policy 0, policy_version 15423 (0.0008) [2023-10-10 17:10:31,168][123614] Updated weights for policy 1, policy_version 15370 (0.0008) [2023-10-10 17:10:31,543][123614] Updated weights for policy 1, policy_version 15380 (0.0009) [2023-10-10 17:10:31,906][123614] Updated weights for policy 1, policy_version 15390 (0.0007) [2023-10-10 17:10:33,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 31555584. Throughput: 0: 1813.7, 1: 1811.6. Samples: 7899068. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:10:33,788][122664] Avg episode reward: [(0, '24.580'), (1, '19.830')] [2023-10-10 17:10:33,982][123582] Updated weights for policy 0, policy_version 15433 (0.0008) [2023-10-10 17:10:34,355][123582] Updated weights for policy 0, policy_version 15443 (0.0008) [2023-10-10 17:10:34,734][123582] Updated weights for policy 0, policy_version 15453 (0.0011) [2023-10-10 17:10:35,484][123614] Updated weights for policy 1, policy_version 15400 (0.0007) [2023-10-10 17:10:35,838][123614] Updated weights for policy 1, policy_version 15410 (0.0007) [2023-10-10 17:10:36,211][123614] Updated weights for policy 1, policy_version 15420 (0.0009) [2023-10-10 17:10:38,482][123582] Updated weights for policy 0, policy_version 15463 (0.0007) [2023-10-10 17:10:38,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 31621120. Throughput: 0: 1817.8, 1: 1810.2. Samples: 7921830. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:10:38,789][122664] Avg episode reward: [(0, '24.520'), (1, '19.600')] [2023-10-10 17:10:38,856][123582] Updated weights for policy 0, policy_version 15473 (0.0009) [2023-10-10 17:10:39,234][123582] Updated weights for policy 0, policy_version 15483 (0.0010) [2023-10-10 17:10:39,908][123614] Updated weights for policy 1, policy_version 15430 (0.0008) [2023-10-10 17:10:40,293][123614] Updated weights for policy 1, policy_version 15440 (0.0007) [2023-10-10 17:10:40,657][123614] Updated weights for policy 1, policy_version 15450 (0.0008) [2023-10-10 17:10:42,963][123582] Updated weights for policy 0, policy_version 15493 (0.0010) [2023-10-10 17:10:43,339][123582] Updated weights for policy 0, policy_version 15503 (0.0009) [2023-10-10 17:10:43,715][123582] Updated weights for policy 0, policy_version 15513 (0.0007) [2023-10-10 17:10:43,788][122664] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 31686656. Throughput: 0: 1810.4, 1: 1809.6. Samples: 7931690. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:10:43,789][122664] Avg episode reward: [(0, '23.100'), (1, '22.640')] [2023-10-10 17:10:44,277][123614] Updated weights for policy 1, policy_version 15460 (0.0010) [2023-10-10 17:10:44,640][123614] Updated weights for policy 1, policy_version 15470 (0.0008) [2023-10-10 17:10:45,007][123614] Updated weights for policy 1, policy_version 15480 (0.0008) [2023-10-10 17:10:47,360][123582] Updated weights for policy 0, policy_version 15523 (0.0007) [2023-10-10 17:10:47,736][123582] Updated weights for policy 0, policy_version 15533 (0.0007) [2023-10-10 17:10:48,106][123582] Updated weights for policy 0, policy_version 15543 (0.0009) [2023-10-10 17:10:48,737][123614] Updated weights for policy 1, policy_version 15490 (0.0007) [2023-10-10 17:10:48,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 31784960. Throughput: 0: 1820.6, 1: 1812.7. Samples: 7954532. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:10:48,788][122664] Avg episode reward: [(0, '24.660'), (1, '25.770')] [2023-10-10 17:10:49,097][123614] Updated weights for policy 1, policy_version 15500 (0.0007) [2023-10-10 17:10:49,464][123614] Updated weights for policy 1, policy_version 15510 (0.0009) [2023-10-10 17:10:49,835][123614] Updated weights for policy 1, policy_version 15520 (0.0009) [2023-10-10 17:10:51,735][123582] Updated weights for policy 0, policy_version 15553 (0.0008) [2023-10-10 17:10:52,101][123582] Updated weights for policy 0, policy_version 15563 (0.0011) [2023-10-10 17:10:52,476][123582] Updated weights for policy 0, policy_version 15573 (0.0008) [2023-10-10 17:10:52,846][123582] Updated weights for policy 0, policy_version 15583 (0.0009) [2023-10-10 17:10:53,448][123614] Updated weights for policy 1, policy_version 15530 (0.0009) [2023-10-10 17:10:53,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 31850496. Throughput: 0: 1824.1, 1: 1812.8. Samples: 7975504. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:10:53,789][122664] Avg episode reward: [(0, '25.410'), (1, '25.650')] [2023-10-10 17:10:53,819][123614] Updated weights for policy 1, policy_version 15540 (0.0009) [2023-10-10 17:10:54,185][123614] Updated weights for policy 1, policy_version 15550 (0.0008) [2023-10-10 17:10:56,358][123582] Updated weights for policy 0, policy_version 15593 (0.0009) [2023-10-10 17:10:56,733][123582] Updated weights for policy 0, policy_version 15603 (0.0007) [2023-10-10 17:10:57,106][123582] Updated weights for policy 0, policy_version 15613 (0.0009) [2023-10-10 17:10:57,989][123614] Updated weights for policy 1, policy_version 15560 (0.0009) [2023-10-10 17:10:58,357][123614] Updated weights for policy 1, policy_version 15570 (0.0007) [2023-10-10 17:10:58,732][123614] Updated weights for policy 1, policy_version 15580 (0.0007) [2023-10-10 17:10:58,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 31916032. Throughput: 0: 1828.2, 1: 1814.5. Samples: 7987508. Policy #0 lag: (min: 21.0, avg: 24.1, max: 53.0) [2023-10-10 17:10:58,788][122664] Avg episode reward: [(0, '25.690'), (1, '26.400')] [2023-10-10 17:11:00,848][123582] Updated weights for policy 0, policy_version 15623 (0.0009) [2023-10-10 17:11:01,216][123582] Updated weights for policy 0, policy_version 15633 (0.0007) [2023-10-10 17:11:01,590][123582] Updated weights for policy 0, policy_version 15643 (0.0009) [2023-10-10 17:11:02,490][123614] Updated weights for policy 1, policy_version 15590 (0.0008) [2023-10-10 17:11:02,861][123614] Updated weights for policy 1, policy_version 15600 (0.0009) [2023-10-10 17:11:03,233][123614] Updated weights for policy 1, policy_version 15610 (0.0008) [2023-10-10 17:11:03,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 32014336. Throughput: 0: 1829.1, 1: 1822.0. Samples: 8008558. Policy #0 lag: (min: 21.0, avg: 24.1, max: 53.0) [2023-10-10 17:11:03,789][122664] Avg episode reward: [(0, '26.560'), (1, '26.230')] [2023-10-10 17:11:05,441][123582] Updated weights for policy 0, policy_version 15653 (0.0007) [2023-10-10 17:11:05,821][123582] Updated weights for policy 0, policy_version 15663 (0.0009) [2023-10-10 17:11:06,206][123582] Updated weights for policy 0, policy_version 15673 (0.0008) [2023-10-10 17:11:07,070][123614] Updated weights for policy 1, policy_version 15620 (0.0009) [2023-10-10 17:11:07,435][123614] Updated weights for policy 1, policy_version 15630 (0.0008) [2023-10-10 17:11:07,804][123614] Updated weights for policy 1, policy_version 15640 (0.0008) [2023-10-10 17:11:08,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 32079872. Throughput: 0: 1821.7, 1: 1819.8. Samples: 8030082. Policy #0 lag: (min: 21.0, avg: 24.1, max: 53.0) [2023-10-10 17:11:08,788][122664] Avg episode reward: [(0, '24.980'), (1, '25.940')] [2023-10-10 17:11:09,933][123582] Updated weights for policy 0, policy_version 15683 (0.0008) [2023-10-10 17:11:10,303][123582] Updated weights for policy 0, policy_version 15693 (0.0008) [2023-10-10 17:11:10,678][123582] Updated weights for policy 0, policy_version 15703 (0.0009) [2023-10-10 17:11:11,598][123614] Updated weights for policy 1, policy_version 15650 (0.0009) [2023-10-10 17:11:11,965][123614] Updated weights for policy 1, policy_version 15660 (0.0007) [2023-10-10 17:11:12,339][123614] Updated weights for policy 1, policy_version 15670 (0.0007) [2023-10-10 17:11:12,711][123614] Updated weights for policy 1, policy_version 15680 (0.0007) [2023-10-10 17:11:13,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 32145408. Throughput: 0: 1823.7, 1: 1824.6. Samples: 8041214. Policy #0 lag: (min: 1.0, avg: 14.1, max: 33.0) [2023-10-10 17:11:13,788][122664] Avg episode reward: [(0, '24.910'), (1, '25.980')] [2023-10-10 17:11:14,257][123582] Updated weights for policy 0, policy_version 15713 (0.0007) [2023-10-10 17:11:14,625][123582] Updated weights for policy 0, policy_version 15723 (0.0009) [2023-10-10 17:11:15,009][123582] Updated weights for policy 0, policy_version 15733 (0.0010) [2023-10-10 17:11:15,381][123582] Updated weights for policy 0, policy_version 15743 (0.0010) [2023-10-10 17:11:16,358][123614] Updated weights for policy 1, policy_version 15690 (0.0011) [2023-10-10 17:11:16,719][123614] Updated weights for policy 1, policy_version 15700 (0.0010) [2023-10-10 17:11:17,092][123614] Updated weights for policy 1, policy_version 15710 (0.0008) [2023-10-10 17:11:18,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 32210944. Throughput: 0: 1828.1, 1: 1817.8. Samples: 8063136. Policy #0 lag: (min: 1.0, avg: 14.1, max: 33.0) [2023-10-10 17:11:18,788][122664] Avg episode reward: [(0, '23.920'), (1, '25.070')] [2023-10-10 17:11:19,009][123582] Updated weights for policy 0, policy_version 15753 (0.0010) [2023-10-10 17:11:19,386][123582] Updated weights for policy 0, policy_version 15763 (0.0011) [2023-10-10 17:11:19,765][123582] Updated weights for policy 0, policy_version 15773 (0.0007) [2023-10-10 17:11:20,725][123614] Updated weights for policy 1, policy_version 15720 (0.0009) [2023-10-10 17:11:21,097][123614] Updated weights for policy 1, policy_version 15730 (0.0010) [2023-10-10 17:11:21,465][123614] Updated weights for policy 1, policy_version 15740 (0.0009) [2023-10-10 17:11:23,530][123582] Updated weights for policy 0, policy_version 15783 (0.0007) [2023-10-10 17:11:23,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 32276480. Throughput: 0: 1824.1, 1: 1812.9. Samples: 8085498. Policy #0 lag: (min: 1.0, avg: 14.1, max: 33.0) [2023-10-10 17:11:23,789][122664] Avg episode reward: [(0, '26.340'), (1, '23.890')] [2023-10-10 17:11:23,902][123582] Updated weights for policy 0, policy_version 15793 (0.0007) [2023-10-10 17:11:24,273][123582] Updated weights for policy 0, policy_version 15803 (0.0008) [2023-10-10 17:11:25,141][123614] Updated weights for policy 1, policy_version 15750 (0.0008) [2023-10-10 17:11:25,506][123614] Updated weights for policy 1, policy_version 15760 (0.0007) [2023-10-10 17:11:25,867][123614] Updated weights for policy 1, policy_version 15770 (0.0007) [2023-10-10 17:11:27,799][123582] Updated weights for policy 0, policy_version 15813 (0.0011) [2023-10-10 17:11:28,182][123582] Updated weights for policy 0, policy_version 15823 (0.0011) [2023-10-10 17:11:28,557][123582] Updated weights for policy 0, policy_version 15833 (0.0010) [2023-10-10 17:11:28,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 32342016. Throughput: 0: 1831.6, 1: 1813.8. Samples: 8095732. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-10 17:11:28,788][122664] Avg episode reward: [(0, '25.920'), (1, '24.600')] [2023-10-10 17:11:29,422][123614] Updated weights for policy 1, policy_version 15780 (0.0010) [2023-10-10 17:11:29,786][123614] Updated weights for policy 1, policy_version 15790 (0.0011) [2023-10-10 17:11:30,160][123614] Updated weights for policy 1, policy_version 15800 (0.0010) [2023-10-10 17:11:32,427][123582] Updated weights for policy 0, policy_version 15843 (0.0010) [2023-10-10 17:11:32,789][123582] Updated weights for policy 0, policy_version 15853 (0.0008) [2023-10-10 17:11:33,161][123582] Updated weights for policy 0, policy_version 15863 (0.0008) [2023-10-10 17:11:33,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 32440320. Throughput: 0: 1824.3, 1: 1816.3. Samples: 8118360. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-10 17:11:33,789][122664] Avg episode reward: [(0, '23.680'), (1, '26.150')] [2023-10-10 17:11:33,844][123614] Updated weights for policy 1, policy_version 15810 (0.0009) [2023-10-10 17:11:34,211][123614] Updated weights for policy 1, policy_version 15820 (0.0008) [2023-10-10 17:11:34,586][123614] Updated weights for policy 1, policy_version 15830 (0.0008) [2023-10-10 17:11:34,948][123614] Updated weights for policy 1, policy_version 15840 (0.0007) [2023-10-10 17:11:36,841][123582] Updated weights for policy 0, policy_version 15873 (0.0007) [2023-10-10 17:11:37,207][123582] Updated weights for policy 0, policy_version 15883 (0.0010) [2023-10-10 17:11:37,575][123582] Updated weights for policy 0, policy_version 15893 (0.0010) [2023-10-10 17:11:37,940][123582] Updated weights for policy 0, policy_version 15903 (0.0010) [2023-10-10 17:11:38,658][123614] Updated weights for policy 1, policy_version 15850 (0.0010) [2023-10-10 17:11:38,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 32505856. Throughput: 0: 1815.7, 1: 1822.2. Samples: 8139208. Policy #0 lag: (min: 8.0, avg: 30.7, max: 32.0) [2023-10-10 17:11:38,788][122664] Avg episode reward: [(0, '21.930'), (1, '25.470')] [2023-10-10 17:11:39,030][123614] Updated weights for policy 1, policy_version 15860 (0.0010) [2023-10-10 17:11:39,392][123614] Updated weights for policy 1, policy_version 15870 (0.0008) [2023-10-10 17:11:41,712][123582] Updated weights for policy 0, policy_version 15913 (0.0010) [2023-10-10 17:11:42,085][123582] Updated weights for policy 0, policy_version 15923 (0.0010) [2023-10-10 17:11:42,461][123582] Updated weights for policy 0, policy_version 15933 (0.0007) [2023-10-10 17:11:42,933][123614] Updated weights for policy 1, policy_version 15880 (0.0009) [2023-10-10 17:11:43,300][123614] Updated weights for policy 1, policy_version 15890 (0.0008) [2023-10-10 17:11:43,674][123614] Updated weights for policy 1, policy_version 15900 (0.0008) [2023-10-10 17:11:43,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 32571392. Throughput: 0: 1816.3, 1: 1819.2. Samples: 8151108. Policy #0 lag: (min: 8.0, avg: 30.7, max: 32.0) [2023-10-10 17:11:43,789][122664] Avg episode reward: [(0, '22.120'), (1, '23.940')] [2023-10-10 17:11:46,106][123582] Updated weights for policy 0, policy_version 15943 (0.0009) [2023-10-10 17:11:46,475][123582] Updated weights for policy 0, policy_version 15953 (0.0008) [2023-10-10 17:11:46,851][123582] Updated weights for policy 0, policy_version 15963 (0.0007) [2023-10-10 17:11:47,425][123614] Updated weights for policy 1, policy_version 15910 (0.0007) [2023-10-10 17:11:47,793][123614] Updated weights for policy 1, policy_version 15920 (0.0007) [2023-10-10 17:11:48,164][123614] Updated weights for policy 1, policy_version 15930 (0.0007) [2023-10-10 17:11:48,788][122664] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 32669696. Throughput: 0: 1812.0, 1: 1823.1. Samples: 8172136. Policy #0 lag: (min: 8.0, avg: 30.7, max: 32.0) [2023-10-10 17:11:48,789][122664] Avg episode reward: [(0, '22.850'), (1, '26.120')] [2023-10-10 17:11:50,471][123582] Updated weights for policy 0, policy_version 15973 (0.0009) [2023-10-10 17:11:50,854][123582] Updated weights for policy 0, policy_version 15983 (0.0009) [2023-10-10 17:11:51,226][123582] Updated weights for policy 0, policy_version 15993 (0.0009) [2023-10-10 17:11:51,900][123614] Updated weights for policy 1, policy_version 15940 (0.0009) [2023-10-10 17:11:52,267][123614] Updated weights for policy 1, policy_version 15950 (0.0009) [2023-10-10 17:11:52,635][123614] Updated weights for policy 1, policy_version 15960 (0.0007) [2023-10-10 17:11:53,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14440.2). Total num frames: 32735232. Throughput: 0: 1815.7, 1: 1823.9. Samples: 8193864. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:11:53,788][122664] Avg episode reward: [(0, '25.050'), (1, '25.930')] [2023-10-10 17:11:53,798][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000016000_16384000.pth... [2023-10-10 17:11:53,798][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000015968_16351232.pth... [2023-10-10 17:11:53,835][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000014272_14614528.pth [2023-10-10 17:11:53,838][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000014304_14647296.pth [2023-10-10 17:11:54,831][123582] Updated weights for policy 0, policy_version 16003 (0.0009) [2023-10-10 17:11:55,214][123582] Updated weights for policy 0, policy_version 16013 (0.0009) [2023-10-10 17:11:55,583][123582] Updated weights for policy 0, policy_version 16023 (0.0009) [2023-10-10 17:11:56,447][123614] Updated weights for policy 1, policy_version 15970 (0.0009) [2023-10-10 17:11:56,809][123614] Updated weights for policy 1, policy_version 15980 (0.0008) [2023-10-10 17:11:57,174][123614] Updated weights for policy 1, policy_version 15990 (0.0010) [2023-10-10 17:11:57,545][123614] Updated weights for policy 1, policy_version 16000 (0.0010) [2023-10-10 17:11:58,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 32800768. Throughput: 0: 1813.8, 1: 1818.5. Samples: 8204668. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:11:58,788][122664] Avg episode reward: [(0, '26.630'), (1, '25.320')] [2023-10-10 17:11:59,346][123582] Updated weights for policy 0, policy_version 16033 (0.0009) [2023-10-10 17:11:59,720][123582] Updated weights for policy 0, policy_version 16043 (0.0010) [2023-10-10 17:12:00,088][123582] Updated weights for policy 0, policy_version 16053 (0.0011) [2023-10-10 17:12:00,461][123582] Updated weights for policy 0, policy_version 16063 (0.0008) [2023-10-10 17:12:01,137][123614] Updated weights for policy 1, policy_version 16010 (0.0010) [2023-10-10 17:12:01,514][123614] Updated weights for policy 1, policy_version 16020 (0.0012) [2023-10-10 17:12:01,879][123614] Updated weights for policy 1, policy_version 16030 (0.0010) [2023-10-10 17:12:03,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 32866304. Throughput: 0: 1808.6, 1: 1819.6. Samples: 8226402. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:12:03,789][122664] Avg episode reward: [(0, '25.160'), (1, '24.910')] [2023-10-10 17:12:04,179][123582] Updated weights for policy 0, policy_version 16073 (0.0008) [2023-10-10 17:12:04,542][123582] Updated weights for policy 0, policy_version 16083 (0.0008) [2023-10-10 17:12:04,920][123582] Updated weights for policy 0, policy_version 16093 (0.0009) [2023-10-10 17:12:05,613][123614] Updated weights for policy 1, policy_version 16040 (0.0009) [2023-10-10 17:12:05,985][123614] Updated weights for policy 1, policy_version 16050 (0.0009) [2023-10-10 17:12:06,355][123614] Updated weights for policy 1, policy_version 16060 (0.0007) [2023-10-10 17:12:08,732][123582] Updated weights for policy 0, policy_version 16103 (0.0009) [2023-10-10 17:12:08,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 32931840. Throughput: 0: 1812.0, 1: 1822.1. Samples: 8249032. Policy #0 lag: (min: 31.0, avg: 32.3, max: 57.0) [2023-10-10 17:12:08,789][122664] Avg episode reward: [(0, '26.880'), (1, '22.730')] [2023-10-10 17:12:09,117][123582] Updated weights for policy 0, policy_version 16113 (0.0010) [2023-10-10 17:12:09,492][123582] Updated weights for policy 0, policy_version 16123 (0.0008) [2023-10-10 17:12:09,675][123247] Saving new best policy, reward=26.880! [2023-10-10 17:12:10,086][123614] Updated weights for policy 1, policy_version 16070 (0.0008) [2023-10-10 17:12:10,466][123614] Updated weights for policy 1, policy_version 16080 (0.0009) [2023-10-10 17:12:10,835][123614] Updated weights for policy 1, policy_version 16090 (0.0009) [2023-10-10 17:12:13,233][123582] Updated weights for policy 0, policy_version 16133 (0.0009) [2023-10-10 17:12:13,602][123582] Updated weights for policy 0, policy_version 16143 (0.0010) [2023-10-10 17:12:13,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 32997376. Throughput: 0: 1800.5, 1: 1818.8. Samples: 8258602. Policy #0 lag: (min: 31.0, avg: 32.3, max: 57.0) [2023-10-10 17:12:13,789][122664] Avg episode reward: [(0, '26.520'), (1, '23.230')] [2023-10-10 17:12:13,982][123582] Updated weights for policy 0, policy_version 16153 (0.0007) [2023-10-10 17:12:14,576][123614] Updated weights for policy 1, policy_version 16100 (0.0008) [2023-10-10 17:12:14,936][123614] Updated weights for policy 1, policy_version 16110 (0.0009) [2023-10-10 17:12:15,313][123614] Updated weights for policy 1, policy_version 16120 (0.0008) [2023-10-10 17:12:17,578][123582] Updated weights for policy 0, policy_version 16163 (0.0008) [2023-10-10 17:12:17,951][123582] Updated weights for policy 0, policy_version 16173 (0.0008) [2023-10-10 17:12:18,327][123582] Updated weights for policy 0, policy_version 16183 (0.0009) [2023-10-10 17:12:18,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 33095680. Throughput: 0: 1810.6, 1: 1815.1. Samples: 8281514. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:12:18,789][122664] Avg episode reward: [(0, '27.380'), (1, '23.700')] [2023-10-10 17:12:18,790][123247] Saving new best policy, reward=27.380! [2023-10-10 17:12:19,002][123614] Updated weights for policy 1, policy_version 16130 (0.0009) [2023-10-10 17:12:19,377][123614] Updated weights for policy 1, policy_version 16140 (0.0009) [2023-10-10 17:12:19,746][123614] Updated weights for policy 1, policy_version 16150 (0.0011) [2023-10-10 17:12:20,114][123614] Updated weights for policy 1, policy_version 16160 (0.0011) [2023-10-10 17:12:22,104][123582] Updated weights for policy 0, policy_version 16193 (0.0008) [2023-10-10 17:12:22,484][123582] Updated weights for policy 0, policy_version 16203 (0.0008) [2023-10-10 17:12:22,849][123582] Updated weights for policy 0, policy_version 16213 (0.0009) [2023-10-10 17:12:23,218][123582] Updated weights for policy 0, policy_version 16223 (0.0011) [2023-10-10 17:12:23,742][123614] Updated weights for policy 1, policy_version 16170 (0.0009) [2023-10-10 17:12:23,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 33161216. Throughput: 0: 1805.0, 1: 1820.2. Samples: 8302342. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:12:23,789][122664] Avg episode reward: [(0, '28.670'), (1, '22.690')] [2023-10-10 17:12:23,797][123247] Saving new best policy, reward=28.670! [2023-10-10 17:12:24,102][123614] Updated weights for policy 1, policy_version 16180 (0.0010) [2023-10-10 17:12:24,470][123614] Updated weights for policy 1, policy_version 16190 (0.0010) [2023-10-10 17:12:26,923][123582] Updated weights for policy 0, policy_version 16233 (0.0008) [2023-10-10 17:12:27,300][123582] Updated weights for policy 0, policy_version 16243 (0.0009) [2023-10-10 17:12:27,665][123582] Updated weights for policy 0, policy_version 16253 (0.0008) [2023-10-10 17:12:28,174][123614] Updated weights for policy 1, policy_version 16200 (0.0008) [2023-10-10 17:12:28,552][123614] Updated weights for policy 1, policy_version 16210 (0.0008) [2023-10-10 17:12:28,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 33226752. Throughput: 0: 1808.2, 1: 1812.5. Samples: 8314040. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:12:28,789][122664] Avg episode reward: [(0, '28.870'), (1, '23.050')] [2023-10-10 17:12:28,789][123247] Saving new best policy, reward=28.870! [2023-10-10 17:12:28,926][123614] Updated weights for policy 1, policy_version 16220 (0.0009) [2023-10-10 17:12:31,422][123582] Updated weights for policy 0, policy_version 16263 (0.0009) [2023-10-10 17:12:31,799][123582] Updated weights for policy 0, policy_version 16273 (0.0010) [2023-10-10 17:12:32,177][123582] Updated weights for policy 0, policy_version 16283 (0.0010) [2023-10-10 17:12:32,797][123614] Updated weights for policy 1, policy_version 16230 (0.0009) [2023-10-10 17:12:33,171][123614] Updated weights for policy 1, policy_version 16240 (0.0008) [2023-10-10 17:12:33,532][123614] Updated weights for policy 1, policy_version 16250 (0.0007) [2023-10-10 17:12:33,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 33325056. Throughput: 0: 1804.4, 1: 1815.3. Samples: 8335022. Policy #0 lag: (min: 27.0, avg: 27.1, max: 32.0) [2023-10-10 17:12:33,789][122664] Avg episode reward: [(0, '26.800'), (1, '22.660')] [2023-10-10 17:12:35,850][123582] Updated weights for policy 0, policy_version 16293 (0.0008) [2023-10-10 17:12:36,221][123582] Updated weights for policy 0, policy_version 16303 (0.0007) [2023-10-10 17:12:36,589][123582] Updated weights for policy 0, policy_version 16313 (0.0010) [2023-10-10 17:12:37,267][123614] Updated weights for policy 1, policy_version 16260 (0.0008) [2023-10-10 17:12:37,634][123614] Updated weights for policy 1, policy_version 16270 (0.0008) [2023-10-10 17:12:37,991][123614] Updated weights for policy 1, policy_version 16280 (0.0009) [2023-10-10 17:12:38,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 33390592. Throughput: 0: 1804.3, 1: 1806.4. Samples: 8356346. Policy #0 lag: (min: 27.0, avg: 27.1, max: 32.0) [2023-10-10 17:12:38,788][122664] Avg episode reward: [(0, '28.410'), (1, '23.520')] [2023-10-10 17:12:40,300][123582] Updated weights for policy 0, policy_version 16323 (0.0011) [2023-10-10 17:12:40,686][123582] Updated weights for policy 0, policy_version 16333 (0.0008) [2023-10-10 17:12:41,056][123582] Updated weights for policy 0, policy_version 16343 (0.0009) [2023-10-10 17:12:41,757][123614] Updated weights for policy 1, policy_version 16290 (0.0007) [2023-10-10 17:12:42,127][123614] Updated weights for policy 1, policy_version 16300 (0.0008) [2023-10-10 17:12:42,491][123614] Updated weights for policy 1, policy_version 16310 (0.0007) [2023-10-10 17:12:42,860][123614] Updated weights for policy 1, policy_version 16320 (0.0008) [2023-10-10 17:12:43,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 33456128. Throughput: 0: 1802.8, 1: 1813.7. Samples: 8367414. Policy #0 lag: (min: 27.0, avg: 27.1, max: 32.0) [2023-10-10 17:12:43,789][122664] Avg episode reward: [(0, '28.730'), (1, '24.860')] [2023-10-10 17:12:44,773][123582] Updated weights for policy 0, policy_version 16353 (0.0011) [2023-10-10 17:12:45,155][123582] Updated weights for policy 0, policy_version 16363 (0.0009) [2023-10-10 17:12:45,524][123582] Updated weights for policy 0, policy_version 16373 (0.0010) [2023-10-10 17:12:45,908][123582] Updated weights for policy 0, policy_version 16383 (0.0011) [2023-10-10 17:12:46,525][123614] Updated weights for policy 1, policy_version 16330 (0.0010) [2023-10-10 17:12:46,897][123614] Updated weights for policy 1, policy_version 16340 (0.0010) [2023-10-10 17:12:47,265][123614] Updated weights for policy 1, policy_version 16350 (0.0010) [2023-10-10 17:12:48,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 33521664. Throughput: 0: 1799.8, 1: 1804.9. Samples: 8388612. Policy #0 lag: (min: 8.0, avg: 32.0, max: 40.0) [2023-10-10 17:12:48,789][122664] Avg episode reward: [(0, '29.190'), (1, '24.470')] [2023-10-10 17:12:48,790][123247] Saving new best policy, reward=29.190! [2023-10-10 17:12:49,602][123582] Updated weights for policy 0, policy_version 16393 (0.0008) [2023-10-10 17:12:49,977][123582] Updated weights for policy 0, policy_version 16403 (0.0008) [2023-10-10 17:12:50,356][123582] Updated weights for policy 0, policy_version 16413 (0.0008) [2023-10-10 17:12:51,078][123614] Updated weights for policy 1, policy_version 16360 (0.0008) [2023-10-10 17:12:51,447][123614] Updated weights for policy 1, policy_version 16370 (0.0007) [2023-10-10 17:12:51,821][123614] Updated weights for policy 1, policy_version 16380 (0.0007) [2023-10-10 17:12:53,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 33587200. Throughput: 0: 1802.0, 1: 1796.9. Samples: 8410984. Policy #0 lag: (min: 8.0, avg: 32.0, max: 40.0) [2023-10-10 17:12:53,788][122664] Avg episode reward: [(0, '30.220'), (1, '25.300')] [2023-10-10 17:12:54,122][123582] Updated weights for policy 0, policy_version 16423 (0.0008) [2023-10-10 17:12:54,494][123582] Updated weights for policy 0, policy_version 16433 (0.0010) [2023-10-10 17:12:54,875][123582] Updated weights for policy 0, policy_version 16443 (0.0009) [2023-10-10 17:12:55,060][123247] Saving new best policy, reward=30.220! [2023-10-10 17:12:55,610][123614] Updated weights for policy 1, policy_version 16390 (0.0007) [2023-10-10 17:12:55,993][123614] Updated weights for policy 1, policy_version 16400 (0.0007) [2023-10-10 17:12:56,370][123614] Updated weights for policy 1, policy_version 16410 (0.0008) [2023-10-10 17:12:58,445][123582] Updated weights for policy 0, policy_version 16453 (0.0008) [2023-10-10 17:12:58,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 33652736. Throughput: 0: 1806.5, 1: 1801.3. Samples: 8420952. Policy #0 lag: (min: 8.0, avg: 32.0, max: 40.0) [2023-10-10 17:12:58,788][122664] Avg episode reward: [(0, '31.420'), (1, '27.840')] [2023-10-10 17:12:58,789][123465] Saving new best policy, reward=27.840! [2023-10-10 17:12:58,821][123582] Updated weights for policy 0, policy_version 16463 (0.0009) [2023-10-10 17:12:59,203][123582] Updated weights for policy 0, policy_version 16473 (0.0008) [2023-10-10 17:12:59,461][123247] Saving new best policy, reward=31.420! [2023-10-10 17:13:00,078][123614] Updated weights for policy 1, policy_version 16420 (0.0008) [2023-10-10 17:13:00,443][123614] Updated weights for policy 1, policy_version 16430 (0.0008) [2023-10-10 17:13:00,808][123614] Updated weights for policy 1, policy_version 16440 (0.0007) [2023-10-10 17:13:02,969][123582] Updated weights for policy 0, policy_version 16483 (0.0009) [2023-10-10 17:13:03,338][123582] Updated weights for policy 0, policy_version 16493 (0.0007) [2023-10-10 17:13:03,704][123582] Updated weights for policy 0, policy_version 16503 (0.0008) [2023-10-10 17:13:03,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 33718272. Throughput: 0: 1800.6, 1: 1806.8. Samples: 8443850. Policy #0 lag: (min: 6.0, avg: 8.4, max: 38.0) [2023-10-10 17:13:03,789][122664] Avg episode reward: [(0, '33.030'), (1, '28.160')] [2023-10-10 17:13:03,791][123465] Saving new best policy, reward=28.160! [2023-10-10 17:13:04,043][123247] Saving new best policy, reward=33.030! [2023-10-10 17:13:04,351][123614] Updated weights for policy 1, policy_version 16450 (0.0007) [2023-10-10 17:13:04,726][123614] Updated weights for policy 1, policy_version 16460 (0.0008) [2023-10-10 17:13:05,084][123614] Updated weights for policy 1, policy_version 16470 (0.0008) [2023-10-10 17:13:05,454][123614] Updated weights for policy 1, policy_version 16480 (0.0008) [2023-10-10 17:13:07,391][123582] Updated weights for policy 0, policy_version 16513 (0.0008) [2023-10-10 17:13:07,767][123582] Updated weights for policy 0, policy_version 16523 (0.0009) [2023-10-10 17:13:08,136][123582] Updated weights for policy 0, policy_version 16533 (0.0007) [2023-10-10 17:13:08,514][123582] Updated weights for policy 0, policy_version 16543 (0.0007) [2023-10-10 17:13:08,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 33816576. Throughput: 0: 1811.5, 1: 1817.5. Samples: 8465646. Policy #0 lag: (min: 6.0, avg: 8.4, max: 38.0) [2023-10-10 17:13:08,788][122664] Avg episode reward: [(0, '30.820'), (1, '28.560')] [2023-10-10 17:13:09,028][123614] Updated weights for policy 1, policy_version 16490 (0.0007) [2023-10-10 17:13:09,397][123614] Updated weights for policy 1, policy_version 16500 (0.0007) [2023-10-10 17:13:09,763][123614] Updated weights for policy 1, policy_version 16510 (0.0010) [2023-10-10 17:13:09,834][123465] Saving new best policy, reward=28.560! [2023-10-10 17:13:12,186][123582] Updated weights for policy 0, policy_version 16553 (0.0007) [2023-10-10 17:13:12,566][123582] Updated weights for policy 0, policy_version 16563 (0.0009) [2023-10-10 17:13:12,931][123582] Updated weights for policy 0, policy_version 16573 (0.0007) [2023-10-10 17:13:13,389][123614] Updated weights for policy 1, policy_version 16520 (0.0008) [2023-10-10 17:13:13,756][123614] Updated weights for policy 1, policy_version 16530 (0.0008) [2023-10-10 17:13:13,788][122664] Fps is (10 sec: 16384.5, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 33882112. Throughput: 0: 1806.7, 1: 1810.8. Samples: 8476824. Policy #0 lag: (min: 9.0, avg: 11.8, max: 41.0) [2023-10-10 17:13:13,788][122664] Avg episode reward: [(0, '31.010'), (1, '28.460')] [2023-10-10 17:13:14,122][123614] Updated weights for policy 1, policy_version 16540 (0.0007) [2023-10-10 17:13:16,552][123582] Updated weights for policy 0, policy_version 16583 (0.0007) [2023-10-10 17:13:16,933][123582] Updated weights for policy 0, policy_version 16593 (0.0008) [2023-10-10 17:13:17,303][123582] Updated weights for policy 0, policy_version 16603 (0.0009) [2023-10-10 17:13:17,887][123614] Updated weights for policy 1, policy_version 16550 (0.0008) [2023-10-10 17:13:18,256][123614] Updated weights for policy 1, policy_version 16560 (0.0008) [2023-10-10 17:13:18,629][123614] Updated weights for policy 1, policy_version 16570 (0.0007) [2023-10-10 17:13:18,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 33947648. Throughput: 0: 1814.8, 1: 1815.1. Samples: 8498370. Policy #0 lag: (min: 9.0, avg: 11.8, max: 41.0) [2023-10-10 17:13:18,788][122664] Avg episode reward: [(0, '27.780'), (1, '27.290')] [2023-10-10 17:13:21,125][123582] Updated weights for policy 0, policy_version 16613 (0.0008) [2023-10-10 17:13:21,496][123582] Updated weights for policy 0, policy_version 16623 (0.0010) [2023-10-10 17:13:21,873][123582] Updated weights for policy 0, policy_version 16633 (0.0009) [2023-10-10 17:13:22,216][123614] Updated weights for policy 1, policy_version 16580 (0.0007) [2023-10-10 17:13:22,590][123614] Updated weights for policy 1, policy_version 16590 (0.0009) [2023-10-10 17:13:22,959][123614] Updated weights for policy 1, policy_version 16600 (0.0007) [2023-10-10 17:13:23,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 34045952. Throughput: 0: 1806.8, 1: 1817.6. Samples: 8519444. Policy #0 lag: (min: 9.0, avg: 11.8, max: 41.0) [2023-10-10 17:13:23,789][122664] Avg episode reward: [(0, '29.050'), (1, '26.430')] [2023-10-10 17:13:25,489][123582] Updated weights for policy 0, policy_version 16643 (0.0012) [2023-10-10 17:13:25,864][123582] Updated weights for policy 0, policy_version 16653 (0.0011) [2023-10-10 17:13:26,229][123582] Updated weights for policy 0, policy_version 16663 (0.0008) [2023-10-10 17:13:26,619][123614] Updated weights for policy 1, policy_version 16610 (0.0009) [2023-10-10 17:13:26,986][123614] Updated weights for policy 1, policy_version 16620 (0.0009) [2023-10-10 17:13:27,356][123614] Updated weights for policy 1, policy_version 16630 (0.0007) [2023-10-10 17:13:27,723][123614] Updated weights for policy 1, policy_version 16640 (0.0007) [2023-10-10 17:13:28,788][122664] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 34111488. Throughput: 0: 1812.9, 1: 1818.7. Samples: 8530836. Policy #0 lag: (min: 23.0, avg: 35.2, max: 55.0) [2023-10-10 17:13:28,789][122664] Avg episode reward: [(0, '26.700'), (1, '25.380')] [2023-10-10 17:13:29,945][123582] Updated weights for policy 0, policy_version 16673 (0.0007) [2023-10-10 17:13:30,316][123582] Updated weights for policy 0, policy_version 16683 (0.0007) [2023-10-10 17:13:30,687][123582] Updated weights for policy 0, policy_version 16693 (0.0008) [2023-10-10 17:13:31,053][123582] Updated weights for policy 0, policy_version 16703 (0.0008) [2023-10-10 17:13:31,354][123614] Updated weights for policy 1, policy_version 16650 (0.0011) [2023-10-10 17:13:31,721][123614] Updated weights for policy 1, policy_version 16660 (0.0010) [2023-10-10 17:13:32,094][123614] Updated weights for policy 1, policy_version 16670 (0.0011) [2023-10-10 17:13:33,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 34177024. Throughput: 0: 1808.1, 1: 1828.0. Samples: 8552238. Policy #0 lag: (min: 23.0, avg: 35.2, max: 55.0) [2023-10-10 17:13:33,789][122664] Avg episode reward: [(0, '24.300'), (1, '25.420')] [2023-10-10 17:13:34,714][123582] Updated weights for policy 0, policy_version 16713 (0.0010) [2023-10-10 17:13:35,095][123582] Updated weights for policy 0, policy_version 16723 (0.0009) [2023-10-10 17:13:35,467][123582] Updated weights for policy 0, policy_version 16733 (0.0010) [2023-10-10 17:13:35,795][123614] Updated weights for policy 1, policy_version 16680 (0.0010) [2023-10-10 17:13:36,173][123614] Updated weights for policy 1, policy_version 16690 (0.0009) [2023-10-10 17:13:36,545][123614] Updated weights for policy 1, policy_version 16700 (0.0008) [2023-10-10 17:13:38,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 34242560. Throughput: 0: 1813.9, 1: 1831.7. Samples: 8575036. Policy #0 lag: (min: 23.0, avg: 35.2, max: 55.0) [2023-10-10 17:13:38,789][122664] Avg episode reward: [(0, '25.170'), (1, '23.680')] [2023-10-10 17:13:39,245][123582] Updated weights for policy 0, policy_version 16743 (0.0009) [2023-10-10 17:13:39,628][123582] Updated weights for policy 0, policy_version 16753 (0.0008) [2023-10-10 17:13:39,987][123582] Updated weights for policy 0, policy_version 16763 (0.0007) [2023-10-10 17:13:40,352][123614] Updated weights for policy 1, policy_version 16710 (0.0009) [2023-10-10 17:13:40,725][123614] Updated weights for policy 1, policy_version 16720 (0.0009) [2023-10-10 17:13:41,092][123614] Updated weights for policy 1, policy_version 16730 (0.0009) [2023-10-10 17:13:43,716][123582] Updated weights for policy 0, policy_version 16773 (0.0008) [2023-10-10 17:13:43,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 34308096. Throughput: 0: 1809.6, 1: 1830.0. Samples: 8584730. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:13:43,789][122664] Avg episode reward: [(0, '25.640'), (1, '22.820')] [2023-10-10 17:13:44,088][123582] Updated weights for policy 0, policy_version 16783 (0.0008) [2023-10-10 17:13:44,461][123582] Updated weights for policy 0, policy_version 16793 (0.0009) [2023-10-10 17:13:44,829][123614] Updated weights for policy 1, policy_version 16740 (0.0007) [2023-10-10 17:13:45,202][123614] Updated weights for policy 1, policy_version 16750 (0.0009) [2023-10-10 17:13:45,570][123614] Updated weights for policy 1, policy_version 16760 (0.0008) [2023-10-10 17:13:47,961][123582] Updated weights for policy 0, policy_version 16803 (0.0007) [2023-10-10 17:13:48,326][123582] Updated weights for policy 0, policy_version 16813 (0.0008) [2023-10-10 17:13:48,708][123582] Updated weights for policy 0, policy_version 16823 (0.0010) [2023-10-10 17:13:48,788][122664] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 34373632. Throughput: 0: 1819.2, 1: 1819.8. Samples: 8607602. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:13:48,788][122664] Avg episode reward: [(0, '27.400'), (1, '24.330')] [2023-10-10 17:13:49,295][123614] Updated weights for policy 1, policy_version 16770 (0.0008) [2023-10-10 17:13:49,663][123614] Updated weights for policy 1, policy_version 16780 (0.0008) [2023-10-10 17:13:50,039][123614] Updated weights for policy 1, policy_version 16790 (0.0009) [2023-10-10 17:13:50,403][123614] Updated weights for policy 1, policy_version 16800 (0.0009) [2023-10-10 17:13:52,496][123582] Updated weights for policy 0, policy_version 16833 (0.0010) [2023-10-10 17:13:52,859][123582] Updated weights for policy 0, policy_version 16843 (0.0008) [2023-10-10 17:13:53,232][123582] Updated weights for policy 0, policy_version 16853 (0.0008) [2023-10-10 17:13:53,610][123582] Updated weights for policy 0, policy_version 16863 (0.0009) [2023-10-10 17:13:53,788][122664] Fps is (10 sec: 16383.4, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 34471936. Throughput: 0: 1818.8, 1: 1811.7. Samples: 8629018. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:13:53,789][122664] Avg episode reward: [(0, '27.400'), (1, '24.830')] [2023-10-10 17:13:53,801][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000016864_17268736.pth... [2023-10-10 17:13:53,832][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000015168_15532032.pth [2023-10-10 17:13:53,836][123247] Saving a milestone ./train_atari/atari_demonattack_APPO/checkpoint_p0/milestones/checkpoint_000016864_17268736.pth [2023-10-10 17:13:54,181][123614] Updated weights for policy 1, policy_version 16810 (0.0008) [2023-10-10 17:13:54,541][123614] Updated weights for policy 1, policy_version 16820 (0.0010) [2023-10-10 17:13:54,905][123614] Updated weights for policy 1, policy_version 16830 (0.0010) [2023-10-10 17:13:54,979][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000016832_17235968.pth... [2023-10-10 17:13:55,014][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000015104_15466496.pth [2023-10-10 17:13:55,018][123465] Saving a milestone ./train_atari/atari_demonattack_APPO/checkpoint_p1/milestones/checkpoint_000016832_17235968.pth [2023-10-10 17:13:57,205][123582] Updated weights for policy 0, policy_version 16873 (0.0008) [2023-10-10 17:13:57,580][123582] Updated weights for policy 0, policy_version 16883 (0.0010) [2023-10-10 17:13:57,960][123582] Updated weights for policy 0, policy_version 16893 (0.0010) [2023-10-10 17:13:58,599][123614] Updated weights for policy 1, policy_version 16840 (0.0008) [2023-10-10 17:13:58,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 34537472. Throughput: 0: 1816.3, 1: 1814.6. Samples: 8640216. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:13:58,789][122664] Avg episode reward: [(0, '25.880'), (1, '25.600')] [2023-10-10 17:13:58,969][123614] Updated weights for policy 1, policy_version 16850 (0.0009) [2023-10-10 17:13:59,333][123614] Updated weights for policy 1, policy_version 16860 (0.0010) [2023-10-10 17:14:01,604][123582] Updated weights for policy 0, policy_version 16903 (0.0009) [2023-10-10 17:14:01,983][123582] Updated weights for policy 0, policy_version 16913 (0.0008) [2023-10-10 17:14:02,352][123582] Updated weights for policy 0, policy_version 16923 (0.0008) [2023-10-10 17:14:02,907][123614] Updated weights for policy 1, policy_version 16870 (0.0009) [2023-10-10 17:14:03,288][123614] Updated weights for policy 1, policy_version 16880 (0.0008) [2023-10-10 17:14:03,654][123614] Updated weights for policy 1, policy_version 16890 (0.0009) [2023-10-10 17:14:03,788][122664] Fps is (10 sec: 13107.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 34603008. Throughput: 0: 1819.2, 1: 1814.4. Samples: 8661884. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:14:03,789][122664] Avg episode reward: [(0, '26.910'), (1, '27.190')] [2023-10-10 17:14:06,107][123582] Updated weights for policy 0, policy_version 16933 (0.0010) [2023-10-10 17:14:06,482][123582] Updated weights for policy 0, policy_version 16943 (0.0010) [2023-10-10 17:14:06,857][123582] Updated weights for policy 0, policy_version 16953 (0.0007) [2023-10-10 17:14:07,374][123614] Updated weights for policy 1, policy_version 16900 (0.0009) [2023-10-10 17:14:07,746][123614] Updated weights for policy 1, policy_version 16910 (0.0010) [2023-10-10 17:14:08,116][123614] Updated weights for policy 1, policy_version 16920 (0.0007) [2023-10-10 17:14:08,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 34701312. Throughput: 0: 1824.3, 1: 1807.0. Samples: 8682854. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-10 17:14:08,789][122664] Avg episode reward: [(0, '27.350'), (1, '27.190')] [2023-10-10 17:14:10,628][123582] Updated weights for policy 0, policy_version 16963 (0.0010) [2023-10-10 17:14:11,005][123582] Updated weights for policy 0, policy_version 16973 (0.0010) [2023-10-10 17:14:11,375][123582] Updated weights for policy 0, policy_version 16983 (0.0008) [2023-10-10 17:14:11,950][123614] Updated weights for policy 1, policy_version 16930 (0.0008) [2023-10-10 17:14:12,320][123614] Updated weights for policy 1, policy_version 16940 (0.0008) [2023-10-10 17:14:12,693][123614] Updated weights for policy 1, policy_version 16950 (0.0011) [2023-10-10 17:14:13,062][123614] Updated weights for policy 1, policy_version 16960 (0.0009) [2023-10-10 17:14:13,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 34766848. Throughput: 0: 1826.9, 1: 1807.6. Samples: 8694384. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-10 17:14:13,789][122664] Avg episode reward: [(0, '25.340'), (1, '29.340')] [2023-10-10 17:14:13,789][123465] Saving new best policy, reward=29.340! [2023-10-10 17:14:15,191][123582] Updated weights for policy 0, policy_version 16993 (0.0008) [2023-10-10 17:14:15,552][123582] Updated weights for policy 0, policy_version 17003 (0.0010) [2023-10-10 17:14:15,926][123582] Updated weights for policy 0, policy_version 17013 (0.0009) [2023-10-10 17:14:16,300][123582] Updated weights for policy 0, policy_version 17023 (0.0007) [2023-10-10 17:14:16,747][123614] Updated weights for policy 1, policy_version 16970 (0.0009) [2023-10-10 17:14:17,113][123614] Updated weights for policy 1, policy_version 16980 (0.0010) [2023-10-10 17:14:17,486][123614] Updated weights for policy 1, policy_version 16990 (0.0007) [2023-10-10 17:14:18,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 34832384. Throughput: 0: 1820.8, 1: 1800.3. Samples: 8715188. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-10 17:14:18,789][122664] Avg episode reward: [(0, '23.940'), (1, '29.070')] [2023-10-10 17:14:19,840][123582] Updated weights for policy 0, policy_version 17033 (0.0010) [2023-10-10 17:14:20,214][123582] Updated weights for policy 0, policy_version 17043 (0.0007) [2023-10-10 17:14:20,586][123582] Updated weights for policy 0, policy_version 17053 (0.0007) [2023-10-10 17:14:21,281][123614] Updated weights for policy 1, policy_version 17000 (0.0007) [2023-10-10 17:14:21,658][123614] Updated weights for policy 1, policy_version 17010 (0.0008) [2023-10-10 17:14:22,028][123614] Updated weights for policy 1, policy_version 17020 (0.0008) [2023-10-10 17:14:23,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 34897920. Throughput: 0: 1822.9, 1: 1801.6. Samples: 8738138. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 17:14:23,788][122664] Avg episode reward: [(0, '22.920'), (1, '30.200')] [2023-10-10 17:14:23,796][123465] Saving new best policy, reward=30.200! [2023-10-10 17:14:24,204][123582] Updated weights for policy 0, policy_version 17063 (0.0009) [2023-10-10 17:14:24,582][123582] Updated weights for policy 0, policy_version 17073 (0.0008) [2023-10-10 17:14:24,959][123582] Updated weights for policy 0, policy_version 17083 (0.0008) [2023-10-10 17:14:25,673][123614] Updated weights for policy 1, policy_version 17030 (0.0008) [2023-10-10 17:14:26,056][123614] Updated weights for policy 1, policy_version 17040 (0.0007) [2023-10-10 17:14:26,430][123614] Updated weights for policy 1, policy_version 17050 (0.0009) [2023-10-10 17:14:28,426][123582] Updated weights for policy 0, policy_version 17093 (0.0008) [2023-10-10 17:14:28,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 34963456. Throughput: 0: 1824.3, 1: 1801.8. Samples: 8747906. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 17:14:28,789][122664] Avg episode reward: [(0, '23.630'), (1, '29.850')] [2023-10-10 17:14:28,796][123582] Updated weights for policy 0, policy_version 17103 (0.0009) [2023-10-10 17:14:29,147][123582] Updated weights for policy 0, policy_version 17113 (0.0008) [2023-10-10 17:14:30,092][123614] Updated weights for policy 1, policy_version 17060 (0.0011) [2023-10-10 17:14:30,464][123614] Updated weights for policy 1, policy_version 17070 (0.0008) [2023-10-10 17:14:30,837][123614] Updated weights for policy 1, policy_version 17080 (0.0008) [2023-10-10 17:14:32,889][123582] Updated weights for policy 0, policy_version 17123 (0.0008) [2023-10-10 17:14:33,266][123582] Updated weights for policy 0, policy_version 17133 (0.0010) [2023-10-10 17:14:33,635][123582] Updated weights for policy 0, policy_version 17143 (0.0008) [2023-10-10 17:14:33,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 35028992. Throughput: 0: 1824.8, 1: 1803.1. Samples: 8770860. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 17:14:33,789][122664] Avg episode reward: [(0, '24.870'), (1, '31.030')] [2023-10-10 17:14:33,790][123465] Saving new best policy, reward=31.030! [2023-10-10 17:14:34,502][123614] Updated weights for policy 1, policy_version 17090 (0.0008) [2023-10-10 17:14:34,886][123614] Updated weights for policy 1, policy_version 17100 (0.0009) [2023-10-10 17:14:35,248][123614] Updated weights for policy 1, policy_version 17110 (0.0010) [2023-10-10 17:14:35,616][123614] Updated weights for policy 1, policy_version 17120 (0.0007) [2023-10-10 17:14:37,393][123582] Updated weights for policy 0, policy_version 17153 (0.0008) [2023-10-10 17:14:37,773][123582] Updated weights for policy 0, policy_version 17163 (0.0009) [2023-10-10 17:14:38,152][123582] Updated weights for policy 0, policy_version 17173 (0.0009) [2023-10-10 17:14:38,524][123582] Updated weights for policy 0, policy_version 17183 (0.0009) [2023-10-10 17:14:38,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 35127296. Throughput: 0: 1820.9, 1: 1808.2. Samples: 8792328. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-10-10 17:14:38,789][122664] Avg episode reward: [(0, '22.840'), (1, '27.820')] [2023-10-10 17:14:39,378][123614] Updated weights for policy 1, policy_version 17130 (0.0009) [2023-10-10 17:14:39,750][123614] Updated weights for policy 1, policy_version 17140 (0.0008) [2023-10-10 17:14:40,122][123614] Updated weights for policy 1, policy_version 17150 (0.0008) [2023-10-10 17:14:42,224][123582] Updated weights for policy 0, policy_version 17193 (0.0008) [2023-10-10 17:14:42,600][123582] Updated weights for policy 0, policy_version 17203 (0.0008) [2023-10-10 17:14:42,968][123582] Updated weights for policy 0, policy_version 17213 (0.0009) [2023-10-10 17:14:43,756][123614] Updated weights for policy 1, policy_version 17160 (0.0007) [2023-10-10 17:14:43,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 35192832. Throughput: 0: 1822.7, 1: 1804.2. Samples: 8803424. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-10-10 17:14:43,789][122664] Avg episode reward: [(0, '22.470'), (1, '27.170')] [2023-10-10 17:14:44,122][123614] Updated weights for policy 1, policy_version 17170 (0.0009) [2023-10-10 17:14:44,498][123614] Updated weights for policy 1, policy_version 17180 (0.0008) [2023-10-10 17:14:46,601][123582] Updated weights for policy 0, policy_version 17223 (0.0008) [2023-10-10 17:14:46,984][123582] Updated weights for policy 0, policy_version 17233 (0.0008) [2023-10-10 17:14:47,352][123582] Updated weights for policy 0, policy_version 17243 (0.0011) [2023-10-10 17:14:48,209][123614] Updated weights for policy 1, policy_version 17190 (0.0010) [2023-10-10 17:14:48,582][123614] Updated weights for policy 1, policy_version 17200 (0.0007) [2023-10-10 17:14:48,788][122664] Fps is (10 sec: 13107.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 35258368. Throughput: 0: 1819.8, 1: 1809.1. Samples: 8825186. Policy #0 lag: (min: 26.0, avg: 40.9, max: 58.0) [2023-10-10 17:14:48,788][122664] Avg episode reward: [(0, '22.480'), (1, '25.730')] [2023-10-10 17:14:48,962][123614] Updated weights for policy 1, policy_version 17210 (0.0008) [2023-10-10 17:14:51,148][123582] Updated weights for policy 0, policy_version 17253 (0.0009) [2023-10-10 17:14:51,525][123582] Updated weights for policy 0, policy_version 17263 (0.0009) [2023-10-10 17:14:51,904][123582] Updated weights for policy 0, policy_version 17273 (0.0007) [2023-10-10 17:14:52,565][123614] Updated weights for policy 1, policy_version 17220 (0.0008) [2023-10-10 17:14:52,934][123614] Updated weights for policy 1, policy_version 17230 (0.0008) [2023-10-10 17:14:53,312][123614] Updated weights for policy 1, policy_version 17240 (0.0009) [2023-10-10 17:14:53,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 35356672. Throughput: 0: 1814.8, 1: 1814.2. Samples: 8846158. Policy #0 lag: (min: 26.0, avg: 40.9, max: 58.0) [2023-10-10 17:14:53,789][122664] Avg episode reward: [(0, '23.730'), (1, '24.720')] [2023-10-10 17:14:55,575][123582] Updated weights for policy 0, policy_version 17283 (0.0008) [2023-10-10 17:14:55,944][123582] Updated weights for policy 0, policy_version 17293 (0.0011) [2023-10-10 17:14:56,325][123582] Updated weights for policy 0, policy_version 17303 (0.0010) [2023-10-10 17:14:56,939][123614] Updated weights for policy 1, policy_version 17250 (0.0009) [2023-10-10 17:14:57,294][123614] Updated weights for policy 1, policy_version 17260 (0.0008) [2023-10-10 17:14:57,662][123614] Updated weights for policy 1, policy_version 17270 (0.0009) [2023-10-10 17:14:58,036][123614] Updated weights for policy 1, policy_version 17280 (0.0009) [2023-10-10 17:14:58,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 35422208. Throughput: 0: 1815.5, 1: 1818.0. Samples: 8857890. Policy #0 lag: (min: 26.0, avg: 40.9, max: 58.0) [2023-10-10 17:14:58,788][122664] Avg episode reward: [(0, '24.550'), (1, '24.570')] [2023-10-10 17:15:00,113][123582] Updated weights for policy 0, policy_version 17313 (0.0011) [2023-10-10 17:15:00,495][123582] Updated weights for policy 0, policy_version 17323 (0.0011) [2023-10-10 17:15:00,876][123582] Updated weights for policy 0, policy_version 17333 (0.0008) [2023-10-10 17:15:01,243][123582] Updated weights for policy 0, policy_version 17343 (0.0008) [2023-10-10 17:15:01,689][123614] Updated weights for policy 1, policy_version 17290 (0.0009) [2023-10-10 17:15:02,058][123614] Updated weights for policy 1, policy_version 17300 (0.0007) [2023-10-10 17:15:02,419][123614] Updated weights for policy 1, policy_version 17310 (0.0007) [2023-10-10 17:15:03,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 35487744. Throughput: 0: 1817.0, 1: 1819.9. Samples: 8878846. Policy #0 lag: (min: 15.0, avg: 26.4, max: 47.0) [2023-10-10 17:15:03,789][122664] Avg episode reward: [(0, '25.160'), (1, '26.370')] [2023-10-10 17:15:04,854][123582] Updated weights for policy 0, policy_version 17353 (0.0009) [2023-10-10 17:15:05,226][123582] Updated weights for policy 0, policy_version 17363 (0.0009) [2023-10-10 17:15:05,599][123582] Updated weights for policy 0, policy_version 17373 (0.0011) [2023-10-10 17:15:06,117][123614] Updated weights for policy 1, policy_version 17320 (0.0008) [2023-10-10 17:15:06,483][123614] Updated weights for policy 1, policy_version 17330 (0.0009) [2023-10-10 17:15:06,863][123614] Updated weights for policy 1, policy_version 17340 (0.0009) [2023-10-10 17:15:08,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 35553280. Throughput: 0: 1806.8, 1: 1823.1. Samples: 8901484. Policy #0 lag: (min: 15.0, avg: 26.4, max: 47.0) [2023-10-10 17:15:08,788][122664] Avg episode reward: [(0, '25.410'), (1, '25.450')] [2023-10-10 17:15:09,473][123582] Updated weights for policy 0, policy_version 17383 (0.0008) [2023-10-10 17:15:09,850][123582] Updated weights for policy 0, policy_version 17393 (0.0010) [2023-10-10 17:15:10,220][123582] Updated weights for policy 0, policy_version 17403 (0.0008) [2023-10-10 17:15:10,680][123614] Updated weights for policy 1, policy_version 17350 (0.0008) [2023-10-10 17:15:11,069][123614] Updated weights for policy 1, policy_version 17360 (0.0010) [2023-10-10 17:15:11,429][123614] Updated weights for policy 1, policy_version 17370 (0.0008) [2023-10-10 17:15:13,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 35618816. Throughput: 0: 1805.5, 1: 1822.9. Samples: 8911182. Policy #0 lag: (min: 15.0, avg: 26.4, max: 47.0) [2023-10-10 17:15:13,788][122664] Avg episode reward: [(0, '25.870'), (1, '25.210')] [2023-10-10 17:15:13,836][123582] Updated weights for policy 0, policy_version 17413 (0.0010) [2023-10-10 17:15:14,207][123582] Updated weights for policy 0, policy_version 17423 (0.0010) [2023-10-10 17:15:14,586][123582] Updated weights for policy 0, policy_version 17433 (0.0009) [2023-10-10 17:15:15,086][123614] Updated weights for policy 1, policy_version 17380 (0.0008) [2023-10-10 17:15:15,441][123614] Updated weights for policy 1, policy_version 17390 (0.0008) [2023-10-10 17:15:15,810][123614] Updated weights for policy 1, policy_version 17400 (0.0009) [2023-10-10 17:15:18,212][123582] Updated weights for policy 0, policy_version 17443 (0.0009) [2023-10-10 17:15:18,585][123582] Updated weights for policy 0, policy_version 17453 (0.0007) [2023-10-10 17:15:18,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 35684352. Throughput: 0: 1803.3, 1: 1825.7. Samples: 8934164. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:15:18,788][122664] Avg episode reward: [(0, '27.410'), (1, '26.980')] [2023-10-10 17:15:18,948][123582] Updated weights for policy 0, policy_version 17463 (0.0008) [2023-10-10 17:15:19,542][123614] Updated weights for policy 1, policy_version 17410 (0.0009) [2023-10-10 17:15:19,909][123614] Updated weights for policy 1, policy_version 17420 (0.0009) [2023-10-10 17:15:20,285][123614] Updated weights for policy 1, policy_version 17430 (0.0007) [2023-10-10 17:15:20,654][123614] Updated weights for policy 1, policy_version 17440 (0.0007) [2023-10-10 17:15:22,581][123582] Updated weights for policy 0, policy_version 17473 (0.0009) [2023-10-10 17:15:22,947][123582] Updated weights for policy 0, policy_version 17483 (0.0009) [2023-10-10 17:15:23,320][123582] Updated weights for policy 0, policy_version 17493 (0.0010) [2023-10-10 17:15:23,689][123582] Updated weights for policy 0, policy_version 17503 (0.0011) [2023-10-10 17:15:23,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 35782656. Throughput: 0: 1815.5, 1: 1821.6. Samples: 8955994. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:15:23,789][122664] Avg episode reward: [(0, '27.690'), (1, '29.280')] [2023-10-10 17:15:24,227][123614] Updated weights for policy 1, policy_version 17450 (0.0008) [2023-10-10 17:15:24,585][123614] Updated weights for policy 1, policy_version 17460 (0.0010) [2023-10-10 17:15:24,961][123614] Updated weights for policy 1, policy_version 17470 (0.0007) [2023-10-10 17:15:27,251][123582] Updated weights for policy 0, policy_version 17513 (0.0009) [2023-10-10 17:15:27,619][123582] Updated weights for policy 0, policy_version 17523 (0.0010) [2023-10-10 17:15:27,992][123582] Updated weights for policy 0, policy_version 17533 (0.0009) [2023-10-10 17:15:28,622][123614] Updated weights for policy 1, policy_version 17480 (0.0010) [2023-10-10 17:15:28,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 35848192. Throughput: 0: 1818.0, 1: 1822.3. Samples: 8967240. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:15:28,789][122664] Avg episode reward: [(0, '30.170'), (1, '27.380')] [2023-10-10 17:15:28,990][123614] Updated weights for policy 1, policy_version 17490 (0.0008) [2023-10-10 17:15:29,365][123614] Updated weights for policy 1, policy_version 17500 (0.0007) [2023-10-10 17:15:31,737][123582] Updated weights for policy 0, policy_version 17543 (0.0010) [2023-10-10 17:15:32,122][123582] Updated weights for policy 0, policy_version 17553 (0.0008) [2023-10-10 17:15:32,486][123582] Updated weights for policy 0, policy_version 17563 (0.0007) [2023-10-10 17:15:33,233][123614] Updated weights for policy 1, policy_version 17510 (0.0007) [2023-10-10 17:15:33,607][123614] Updated weights for policy 1, policy_version 17520 (0.0007) [2023-10-10 17:15:33,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 35913728. Throughput: 0: 1816.9, 1: 1818.5. Samples: 8988780. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:15:33,789][122664] Avg episode reward: [(0, '29.130'), (1, '29.020')] [2023-10-10 17:15:33,982][123614] Updated weights for policy 1, policy_version 17530 (0.0008) [2023-10-10 17:15:36,116][123582] Updated weights for policy 0, policy_version 17573 (0.0010) [2023-10-10 17:15:36,494][123582] Updated weights for policy 0, policy_version 17583 (0.0007) [2023-10-10 17:15:36,859][123582] Updated weights for policy 0, policy_version 17593 (0.0007) [2023-10-10 17:15:37,733][123614] Updated weights for policy 1, policy_version 17540 (0.0008) [2023-10-10 17:15:38,098][123614] Updated weights for policy 1, policy_version 17550 (0.0010) [2023-10-10 17:15:38,457][123614] Updated weights for policy 1, policy_version 17560 (0.0012) [2023-10-10 17:15:38,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 36012032. Throughput: 0: 1821.2, 1: 1818.2. Samples: 9009928. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:15:38,789][122664] Avg episode reward: [(0, '28.930'), (1, '29.640')] [2023-10-10 17:15:40,561][123582] Updated weights for policy 0, policy_version 17603 (0.0007) [2023-10-10 17:15:40,938][123582] Updated weights for policy 0, policy_version 17613 (0.0008) [2023-10-10 17:15:41,314][123582] Updated weights for policy 0, policy_version 17623 (0.0010) [2023-10-10 17:15:42,242][123614] Updated weights for policy 1, policy_version 17570 (0.0009) [2023-10-10 17:15:42,615][123614] Updated weights for policy 1, policy_version 17580 (0.0007) [2023-10-10 17:15:42,988][123614] Updated weights for policy 1, policy_version 17590 (0.0009) [2023-10-10 17:15:43,350][123614] Updated weights for policy 1, policy_version 17600 (0.0008) [2023-10-10 17:15:43,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 36077568. Throughput: 0: 1822.1, 1: 1811.2. Samples: 9021390. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:15:43,789][122664] Avg episode reward: [(0, '26.990'), (1, '29.470')] [2023-10-10 17:15:44,814][123582] Updated weights for policy 0, policy_version 17633 (0.0007) [2023-10-10 17:15:45,185][123582] Updated weights for policy 0, policy_version 17643 (0.0008) [2023-10-10 17:15:45,558][123582] Updated weights for policy 0, policy_version 17653 (0.0010) [2023-10-10 17:15:45,923][123582] Updated weights for policy 0, policy_version 17663 (0.0010) [2023-10-10 17:15:47,117][123614] Updated weights for policy 1, policy_version 17610 (0.0007) [2023-10-10 17:15:47,476][123614] Updated weights for policy 1, policy_version 17620 (0.0008) [2023-10-10 17:15:47,852][123614] Updated weights for policy 1, policy_version 17630 (0.0008) [2023-10-10 17:15:48,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 36143104. Throughput: 0: 1829.2, 1: 1816.2. Samples: 9042892. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:15:48,789][122664] Avg episode reward: [(0, '25.360'), (1, '28.940')] [2023-10-10 17:15:49,720][123582] Updated weights for policy 0, policy_version 17673 (0.0008) [2023-10-10 17:15:50,098][123582] Updated weights for policy 0, policy_version 17683 (0.0007) [2023-10-10 17:15:50,471][123582] Updated weights for policy 0, policy_version 17693 (0.0009) [2023-10-10 17:15:51,414][123614] Updated weights for policy 1, policy_version 17640 (0.0009) [2023-10-10 17:15:51,785][123614] Updated weights for policy 1, policy_version 17650 (0.0008) [2023-10-10 17:15:52,148][123614] Updated weights for policy 1, policy_version 17660 (0.0007) [2023-10-10 17:15:53,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 36208640. Throughput: 0: 1836.9, 1: 1811.3. Samples: 9065656. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-10 17:15:53,789][122664] Avg episode reward: [(0, '24.630'), (1, '29.250')] [2023-10-10 17:15:53,797][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000017664_18087936.pth... [2023-10-10 17:15:53,825][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000015968_16351232.pth [2023-10-10 17:15:54,145][123582] Updated weights for policy 0, policy_version 17703 (0.0009) [2023-10-10 17:15:54,529][123582] Updated weights for policy 0, policy_version 17713 (0.0009) [2023-10-10 17:15:54,893][123582] Updated weights for policy 0, policy_version 17723 (0.0008) [2023-10-10 17:15:55,081][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000017728_18153472.pth... [2023-10-10 17:15:55,120][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000016000_16384000.pth [2023-10-10 17:15:55,982][123614] Updated weights for policy 1, policy_version 17670 (0.0007) [2023-10-10 17:15:56,360][123614] Updated weights for policy 1, policy_version 17680 (0.0010) [2023-10-10 17:15:56,728][123614] Updated weights for policy 1, policy_version 17690 (0.0009) [2023-10-10 17:15:58,572][123582] Updated weights for policy 0, policy_version 17733 (0.0009) [2023-10-10 17:15:58,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 36274176. Throughput: 0: 1837.9, 1: 1816.4. Samples: 9075628. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-10 17:15:58,788][122664] Avg episode reward: [(0, '23.750'), (1, '26.230')] [2023-10-10 17:15:58,943][123582] Updated weights for policy 0, policy_version 17743 (0.0007) [2023-10-10 17:15:59,309][123582] Updated weights for policy 0, policy_version 17753 (0.0009) [2023-10-10 17:16:00,481][123614] Updated weights for policy 1, policy_version 17700 (0.0009) [2023-10-10 17:16:00,850][123614] Updated weights for policy 1, policy_version 17710 (0.0009) [2023-10-10 17:16:01,220][123614] Updated weights for policy 1, policy_version 17720 (0.0008) [2023-10-10 17:16:02,964][123582] Updated weights for policy 0, policy_version 17763 (0.0007) [2023-10-10 17:16:03,341][123582] Updated weights for policy 0, policy_version 17773 (0.0008) [2023-10-10 17:16:03,721][123582] Updated weights for policy 0, policy_version 17783 (0.0010) [2023-10-10 17:16:03,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 36339712. Throughput: 0: 1835.6, 1: 1802.8. Samples: 9097896. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-10 17:16:03,789][122664] Avg episode reward: [(0, '24.590'), (1, '25.030')] [2023-10-10 17:16:04,907][123614] Updated weights for policy 1, policy_version 17730 (0.0008) [2023-10-10 17:16:05,279][123614] Updated weights for policy 1, policy_version 17740 (0.0009) [2023-10-10 17:16:05,650][123614] Updated weights for policy 1, policy_version 17750 (0.0007) [2023-10-10 17:16:06,025][123614] Updated weights for policy 1, policy_version 17760 (0.0009) [2023-10-10 17:16:07,419][123582] Updated weights for policy 0, policy_version 17793 (0.0009) [2023-10-10 17:16:07,796][123582] Updated weights for policy 0, policy_version 17803 (0.0010) [2023-10-10 17:16:08,167][123582] Updated weights for policy 0, policy_version 17813 (0.0008) [2023-10-10 17:16:08,548][123582] Updated weights for policy 0, policy_version 17823 (0.0008) [2023-10-10 17:16:08,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 36438016. Throughput: 0: 1827.9, 1: 1801.8. Samples: 9119330. Policy #0 lag: (min: 10.0, avg: 10.1, max: 15.0) [2023-10-10 17:16:08,789][122664] Avg episode reward: [(0, '25.490'), (1, '25.330')] [2023-10-10 17:16:09,750][123614] Updated weights for policy 1, policy_version 17770 (0.0009) [2023-10-10 17:16:10,109][123614] Updated weights for policy 1, policy_version 17780 (0.0011) [2023-10-10 17:16:10,483][123614] Updated weights for policy 1, policy_version 17790 (0.0011) [2023-10-10 17:16:12,220][123582] Updated weights for policy 0, policy_version 17833 (0.0007) [2023-10-10 17:16:12,600][123582] Updated weights for policy 0, policy_version 17843 (0.0007) [2023-10-10 17:16:12,968][123582] Updated weights for policy 0, policy_version 17853 (0.0009) [2023-10-10 17:16:13,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 36503552. Throughput: 0: 1825.4, 1: 1801.9. Samples: 9130468. Policy #0 lag: (min: 10.0, avg: 10.1, max: 15.0) [2023-10-10 17:16:13,789][122664] Avg episode reward: [(0, '26.860'), (1, '22.530')] [2023-10-10 17:16:14,252][123614] Updated weights for policy 1, policy_version 17800 (0.0009) [2023-10-10 17:16:14,619][123614] Updated weights for policy 1, policy_version 17810 (0.0010) [2023-10-10 17:16:14,977][123614] Updated weights for policy 1, policy_version 17820 (0.0010) [2023-10-10 17:16:16,709][123582] Updated weights for policy 0, policy_version 17863 (0.0009) [2023-10-10 17:16:17,088][123582] Updated weights for policy 0, policy_version 17873 (0.0008) [2023-10-10 17:16:17,462][123582] Updated weights for policy 0, policy_version 17883 (0.0008) [2023-10-10 17:16:18,702][123614] Updated weights for policy 1, policy_version 17830 (0.0008) [2023-10-10 17:16:18,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 36569088. Throughput: 0: 1826.9, 1: 1805.2. Samples: 9152222. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-10 17:16:18,788][122664] Avg episode reward: [(0, '26.960'), (1, '23.760')] [2023-10-10 17:16:19,077][123614] Updated weights for policy 1, policy_version 17840 (0.0008) [2023-10-10 17:16:19,449][123614] Updated weights for policy 1, policy_version 17850 (0.0007) [2023-10-10 17:16:21,173][123582] Updated weights for policy 0, policy_version 17893 (0.0010) [2023-10-10 17:16:21,541][123582] Updated weights for policy 0, policy_version 17903 (0.0008) [2023-10-10 17:16:21,911][123582] Updated weights for policy 0, policy_version 17913 (0.0007) [2023-10-10 17:16:23,207][123614] Updated weights for policy 1, policy_version 17860 (0.0009) [2023-10-10 17:16:23,572][123614] Updated weights for policy 1, policy_version 17870 (0.0010) [2023-10-10 17:16:23,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 36634624. Throughput: 0: 1822.7, 1: 1819.8. Samples: 9173840. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-10 17:16:23,788][122664] Avg episode reward: [(0, '25.410'), (1, '23.810')] [2023-10-10 17:16:23,958][123614] Updated weights for policy 1, policy_version 17880 (0.0012) [2023-10-10 17:16:25,744][123582] Updated weights for policy 0, policy_version 17923 (0.0008) [2023-10-10 17:16:26,109][123582] Updated weights for policy 0, policy_version 17933 (0.0007) [2023-10-10 17:16:26,487][123582] Updated weights for policy 0, policy_version 17943 (0.0009) [2023-10-10 17:16:27,483][123614] Updated weights for policy 1, policy_version 17890 (0.0010) [2023-10-10 17:16:27,855][123614] Updated weights for policy 1, policy_version 17900 (0.0007) [2023-10-10 17:16:28,223][123614] Updated weights for policy 1, policy_version 17910 (0.0007) [2023-10-10 17:16:28,597][123614] Updated weights for policy 1, policy_version 17920 (0.0007) [2023-10-10 17:16:28,788][122664] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 36732928. Throughput: 0: 1824.2, 1: 1814.7. Samples: 9185140. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-10 17:16:28,789][122664] Avg episode reward: [(0, '26.580'), (1, '24.870')] [2023-10-10 17:16:30,138][123582] Updated weights for policy 0, policy_version 17953 (0.0008) [2023-10-10 17:16:30,507][123582] Updated weights for policy 0, policy_version 17963 (0.0007) [2023-10-10 17:16:30,872][123582] Updated weights for policy 0, policy_version 17973 (0.0008) [2023-10-10 17:16:31,247][123582] Updated weights for policy 0, policy_version 17983 (0.0007) [2023-10-10 17:16:32,390][123614] Updated weights for policy 1, policy_version 17930 (0.0009) [2023-10-10 17:16:32,760][123614] Updated weights for policy 1, policy_version 17940 (0.0011) [2023-10-10 17:16:33,133][123614] Updated weights for policy 1, policy_version 17950 (0.0009) [2023-10-10 17:16:33,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 36798464. Throughput: 0: 1812.5, 1: 1820.5. Samples: 9206378. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 17:16:33,789][122664] Avg episode reward: [(0, '28.050'), (1, '25.280')] [2023-10-10 17:16:34,916][123582] Updated weights for policy 0, policy_version 17993 (0.0008) [2023-10-10 17:16:35,291][123582] Updated weights for policy 0, policy_version 18003 (0.0007) [2023-10-10 17:16:35,661][123582] Updated weights for policy 0, policy_version 18013 (0.0008) [2023-10-10 17:16:36,740][123614] Updated weights for policy 1, policy_version 17960 (0.0009) [2023-10-10 17:16:37,101][123614] Updated weights for policy 1, policy_version 17970 (0.0011) [2023-10-10 17:16:37,469][123614] Updated weights for policy 1, policy_version 17980 (0.0010) [2023-10-10 17:16:38,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 36864000. Throughput: 0: 1810.1, 1: 1810.5. Samples: 9228584. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 17:16:38,789][122664] Avg episode reward: [(0, '26.630'), (1, '27.000')] [2023-10-10 17:16:39,348][123582] Updated weights for policy 0, policy_version 18023 (0.0008) [2023-10-10 17:16:39,730][123582] Updated weights for policy 0, policy_version 18033 (0.0008) [2023-10-10 17:16:40,097][123582] Updated weights for policy 0, policy_version 18043 (0.0012) [2023-10-10 17:16:41,146][123614] Updated weights for policy 1, policy_version 17990 (0.0008) [2023-10-10 17:16:41,527][123614] Updated weights for policy 1, policy_version 18000 (0.0007) [2023-10-10 17:16:41,893][123614] Updated weights for policy 1, policy_version 18010 (0.0007) [2023-10-10 17:16:43,741][123582] Updated weights for policy 0, policy_version 18053 (0.0010) [2023-10-10 17:16:43,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 36929536. Throughput: 0: 1807.4, 1: 1822.0. Samples: 9238950. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 17:16:43,789][122664] Avg episode reward: [(0, '28.560'), (1, '26.490')] [2023-10-10 17:16:44,120][123582] Updated weights for policy 0, policy_version 18063 (0.0008) [2023-10-10 17:16:44,494][123582] Updated weights for policy 0, policy_version 18073 (0.0008) [2023-10-10 17:16:45,415][123614] Updated weights for policy 1, policy_version 18020 (0.0008) [2023-10-10 17:16:45,788][123614] Updated weights for policy 1, policy_version 18030 (0.0008) [2023-10-10 17:16:46,163][123614] Updated weights for policy 1, policy_version 18040 (0.0010) [2023-10-10 17:16:48,124][123582] Updated weights for policy 0, policy_version 18083 (0.0007) [2023-10-10 17:16:48,506][123582] Updated weights for policy 0, policy_version 18093 (0.0008) [2023-10-10 17:16:48,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 36995072. Throughput: 0: 1810.4, 1: 1821.6. Samples: 9261336. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 17:16:48,788][122664] Avg episode reward: [(0, '25.620'), (1, '25.060')] [2023-10-10 17:16:48,866][123582] Updated weights for policy 0, policy_version 18103 (0.0007) [2023-10-10 17:16:49,756][123614] Updated weights for policy 1, policy_version 18050 (0.0008) [2023-10-10 17:16:50,122][123614] Updated weights for policy 1, policy_version 18060 (0.0007) [2023-10-10 17:16:50,499][123614] Updated weights for policy 1, policy_version 18070 (0.0007) [2023-10-10 17:16:50,862][123614] Updated weights for policy 1, policy_version 18080 (0.0009) [2023-10-10 17:16:52,626][123582] Updated weights for policy 0, policy_version 18113 (0.0007) [2023-10-10 17:16:53,001][123582] Updated weights for policy 0, policy_version 18123 (0.0011) [2023-10-10 17:16:53,367][123582] Updated weights for policy 0, policy_version 18133 (0.0008) [2023-10-10 17:16:53,740][123582] Updated weights for policy 0, policy_version 18143 (0.0010) [2023-10-10 17:16:53,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 37093376. Throughput: 0: 1809.6, 1: 1823.7. Samples: 9282832. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:16:53,788][122664] Avg episode reward: [(0, '25.220'), (1, '24.280')] [2023-10-10 17:16:54,594][123614] Updated weights for policy 1, policy_version 18090 (0.0008) [2023-10-10 17:16:54,967][123614] Updated weights for policy 1, policy_version 18100 (0.0008) [2023-10-10 17:16:55,334][123614] Updated weights for policy 1, policy_version 18110 (0.0009) [2023-10-10 17:16:57,499][123582] Updated weights for policy 0, policy_version 18153 (0.0009) [2023-10-10 17:16:57,865][123582] Updated weights for policy 0, policy_version 18163 (0.0009) [2023-10-10 17:16:58,237][123582] Updated weights for policy 0, policy_version 18173 (0.0007) [2023-10-10 17:16:58,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 37158912. Throughput: 0: 1805.3, 1: 1825.6. Samples: 9293858. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:16:58,789][122664] Avg episode reward: [(0, '25.950'), (1, '24.490')] [2023-10-10 17:16:59,216][123614] Updated weights for policy 1, policy_version 18120 (0.0008) [2023-10-10 17:16:59,586][123614] Updated weights for policy 1, policy_version 18130 (0.0010) [2023-10-10 17:16:59,954][123614] Updated weights for policy 1, policy_version 18140 (0.0008) [2023-10-10 17:17:01,851][123582] Updated weights for policy 0, policy_version 18183 (0.0008) [2023-10-10 17:17:02,236][123582] Updated weights for policy 0, policy_version 18193 (0.0010) [2023-10-10 17:17:02,605][123582] Updated weights for policy 0, policy_version 18203 (0.0011) [2023-10-10 17:17:03,640][123614] Updated weights for policy 1, policy_version 18150 (0.0010) [2023-10-10 17:17:03,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 37224448. Throughput: 0: 1812.2, 1: 1821.3. Samples: 9315732. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:17:03,789][122664] Avg episode reward: [(0, '25.400'), (1, '24.310')] [2023-10-10 17:17:04,008][123614] Updated weights for policy 1, policy_version 18160 (0.0009) [2023-10-10 17:17:04,386][123614] Updated weights for policy 1, policy_version 18170 (0.0010) [2023-10-10 17:17:06,196][123582] Updated weights for policy 0, policy_version 18213 (0.0008) [2023-10-10 17:17:06,573][123582] Updated weights for policy 0, policy_version 18223 (0.0008) [2023-10-10 17:17:06,952][123582] Updated weights for policy 0, policy_version 18233 (0.0007) [2023-10-10 17:17:08,088][123614] Updated weights for policy 1, policy_version 18180 (0.0011) [2023-10-10 17:17:08,456][123614] Updated weights for policy 1, policy_version 18190 (0.0008) [2023-10-10 17:17:08,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 37289984. Throughput: 0: 1811.2, 1: 1813.7. Samples: 9336962. Policy #0 lag: (min: 0.0, avg: 27.8, max: 32.0) [2023-10-10 17:17:08,789][122664] Avg episode reward: [(0, '24.000'), (1, '25.170')] [2023-10-10 17:17:08,830][123614] Updated weights for policy 1, policy_version 18200 (0.0008) [2023-10-10 17:17:10,631][123582] Updated weights for policy 0, policy_version 18243 (0.0008) [2023-10-10 17:17:11,008][123582] Updated weights for policy 0, policy_version 18253 (0.0010) [2023-10-10 17:17:11,384][123582] Updated weights for policy 0, policy_version 18263 (0.0008) [2023-10-10 17:17:12,434][123614] Updated weights for policy 1, policy_version 18210 (0.0008) [2023-10-10 17:17:12,809][123614] Updated weights for policy 1, policy_version 18220 (0.0008) [2023-10-10 17:17:13,175][123614] Updated weights for policy 1, policy_version 18230 (0.0007) [2023-10-10 17:17:13,544][123614] Updated weights for policy 1, policy_version 18240 (0.0009) [2023-10-10 17:17:13,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 37388288. Throughput: 0: 1812.2, 1: 1811.3. Samples: 9348198. Policy #0 lag: (min: 0.0, avg: 27.8, max: 32.0) [2023-10-10 17:17:13,789][122664] Avg episode reward: [(0, '24.430'), (1, '23.990')] [2023-10-10 17:17:15,003][123582] Updated weights for policy 0, policy_version 18273 (0.0008) [2023-10-10 17:17:15,378][123582] Updated weights for policy 0, policy_version 18283 (0.0011) [2023-10-10 17:17:15,750][123582] Updated weights for policy 0, policy_version 18293 (0.0007) [2023-10-10 17:17:16,118][123582] Updated weights for policy 0, policy_version 18303 (0.0007) [2023-10-10 17:17:17,143][123614] Updated weights for policy 1, policy_version 18250 (0.0009) [2023-10-10 17:17:17,514][123614] Updated weights for policy 1, policy_version 18260 (0.0010) [2023-10-10 17:17:17,889][123614] Updated weights for policy 1, policy_version 18270 (0.0010) [2023-10-10 17:17:18,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 37453824. Throughput: 0: 1817.4, 1: 1811.6. Samples: 9369680. Policy #0 lag: (min: 0.0, avg: 27.8, max: 32.0) [2023-10-10 17:17:18,789][122664] Avg episode reward: [(0, '23.260'), (1, '24.640')] [2023-10-10 17:17:19,736][123582] Updated weights for policy 0, policy_version 18313 (0.0011) [2023-10-10 17:17:20,110][123582] Updated weights for policy 0, policy_version 18323 (0.0010) [2023-10-10 17:17:20,484][123582] Updated weights for policy 0, policy_version 18333 (0.0010) [2023-10-10 17:17:21,713][123614] Updated weights for policy 1, policy_version 18280 (0.0008) [2023-10-10 17:17:22,094][123614] Updated weights for policy 1, policy_version 18290 (0.0008) [2023-10-10 17:17:22,458][123614] Updated weights for policy 1, policy_version 18300 (0.0007) [2023-10-10 17:17:23,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 37519360. Throughput: 0: 1827.0, 1: 1814.2. Samples: 9392436. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-10 17:17:23,789][122664] Avg episode reward: [(0, '22.620'), (1, '24.570')] [2023-10-10 17:17:24,165][123582] Updated weights for policy 0, policy_version 18343 (0.0007) [2023-10-10 17:17:24,537][123582] Updated weights for policy 0, policy_version 18353 (0.0009) [2023-10-10 17:17:24,922][123582] Updated weights for policy 0, policy_version 18363 (0.0009) [2023-10-10 17:17:26,359][123614] Updated weights for policy 1, policy_version 18310 (0.0007) [2023-10-10 17:17:26,741][123614] Updated weights for policy 1, policy_version 18320 (0.0009) [2023-10-10 17:17:27,114][123614] Updated weights for policy 1, policy_version 18330 (0.0010) [2023-10-10 17:17:28,487][123582] Updated weights for policy 0, policy_version 18373 (0.0008) [2023-10-10 17:17:28,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 37584896. Throughput: 0: 1828.1, 1: 1814.1. Samples: 9402852. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-10 17:17:28,789][122664] Avg episode reward: [(0, '22.510'), (1, '24.760')] [2023-10-10 17:17:28,848][123582] Updated weights for policy 0, policy_version 18383 (0.0008) [2023-10-10 17:17:29,218][123582] Updated weights for policy 0, policy_version 18393 (0.0007) [2023-10-10 17:17:30,812][123614] Updated weights for policy 1, policy_version 18340 (0.0010) [2023-10-10 17:17:31,183][123614] Updated weights for policy 1, policy_version 18350 (0.0010) [2023-10-10 17:17:31,548][123614] Updated weights for policy 1, policy_version 18360 (0.0008) [2023-10-10 17:17:32,894][123582] Updated weights for policy 0, policy_version 18403 (0.0008) [2023-10-10 17:17:33,278][123582] Updated weights for policy 0, policy_version 18413 (0.0009) [2023-10-10 17:17:33,644][123582] Updated weights for policy 0, policy_version 18423 (0.0008) [2023-10-10 17:17:33,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 37650432. Throughput: 0: 1829.6, 1: 1807.6. Samples: 9425010. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-10 17:17:33,788][122664] Avg episode reward: [(0, '22.330'), (1, '23.700')] [2023-10-10 17:17:35,308][123614] Updated weights for policy 1, policy_version 18370 (0.0009) [2023-10-10 17:17:35,680][123614] Updated weights for policy 1, policy_version 18380 (0.0011) [2023-10-10 17:17:36,058][123614] Updated weights for policy 1, policy_version 18390 (0.0010) [2023-10-10 17:17:36,423][123614] Updated weights for policy 1, policy_version 18400 (0.0008) [2023-10-10 17:17:37,580][123582] Updated weights for policy 0, policy_version 18433 (0.0010) [2023-10-10 17:17:37,956][123582] Updated weights for policy 0, policy_version 18443 (0.0011) [2023-10-10 17:17:38,319][123582] Updated weights for policy 0, policy_version 18453 (0.0010) [2023-10-10 17:17:38,683][123582] Updated weights for policy 0, policy_version 18463 (0.0010) [2023-10-10 17:17:38,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 37748736. Throughput: 0: 1830.0, 1: 1812.7. Samples: 9446756. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-10 17:17:38,788][122664] Avg episode reward: [(0, '23.670'), (1, '24.410')] [2023-10-10 17:17:39,842][123614] Updated weights for policy 1, policy_version 18410 (0.0008) [2023-10-10 17:17:40,218][123614] Updated weights for policy 1, policy_version 18420 (0.0008) [2023-10-10 17:17:40,595][123614] Updated weights for policy 1, policy_version 18430 (0.0008) [2023-10-10 17:17:42,292][123582] Updated weights for policy 0, policy_version 18473 (0.0010) [2023-10-10 17:17:42,667][123582] Updated weights for policy 0, policy_version 18483 (0.0010) [2023-10-10 17:17:43,038][123582] Updated weights for policy 0, policy_version 18493 (0.0010) [2023-10-10 17:17:43,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 37814272. Throughput: 0: 1832.9, 1: 1811.4. Samples: 9457852. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-10 17:17:43,789][122664] Avg episode reward: [(0, '25.780'), (1, '25.650')] [2023-10-10 17:17:44,098][123614] Updated weights for policy 1, policy_version 18440 (0.0007) [2023-10-10 17:17:44,458][123614] Updated weights for policy 1, policy_version 18450 (0.0009) [2023-10-10 17:17:44,831][123614] Updated weights for policy 1, policy_version 18460 (0.0007) [2023-10-10 17:17:46,588][123582] Updated weights for policy 0, policy_version 18503 (0.0011) [2023-10-10 17:17:46,963][123582] Updated weights for policy 0, policy_version 18513 (0.0010) [2023-10-10 17:17:47,323][123582] Updated weights for policy 0, policy_version 18523 (0.0008) [2023-10-10 17:17:48,625][123614] Updated weights for policy 1, policy_version 18470 (0.0007) [2023-10-10 17:17:48,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 37879808. Throughput: 0: 1822.6, 1: 1816.1. Samples: 9479474. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-10 17:17:48,789][122664] Avg episode reward: [(0, '28.430'), (1, '26.610')] [2023-10-10 17:17:48,991][123614] Updated weights for policy 1, policy_version 18480 (0.0007) [2023-10-10 17:17:49,365][123614] Updated weights for policy 1, policy_version 18490 (0.0008) [2023-10-10 17:17:51,067][123582] Updated weights for policy 0, policy_version 18533 (0.0009) [2023-10-10 17:17:51,450][123582] Updated weights for policy 0, policy_version 18543 (0.0009) [2023-10-10 17:17:51,822][123582] Updated weights for policy 0, policy_version 18553 (0.0008) [2023-10-10 17:17:52,975][123614] Updated weights for policy 1, policy_version 18500 (0.0009) [2023-10-10 17:17:53,347][123614] Updated weights for policy 1, policy_version 18510 (0.0008) [2023-10-10 17:17:53,717][123614] Updated weights for policy 1, policy_version 18520 (0.0007) [2023-10-10 17:17:53,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 37945344. Throughput: 0: 1829.2, 1: 1817.1. Samples: 9501044. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-10 17:17:53,789][122664] Avg episode reward: [(0, '25.650'), (1, '25.520')] [2023-10-10 17:17:53,798][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000018560_19005440.pth... [2023-10-10 17:17:53,833][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000016864_17268736.pth [2023-10-10 17:17:54,012][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000018528_18972672.pth... [2023-10-10 17:17:54,041][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000016832_17235968.pth [2023-10-10 17:17:55,415][123582] Updated weights for policy 0, policy_version 18563 (0.0008) [2023-10-10 17:17:55,782][123582] Updated weights for policy 0, policy_version 18573 (0.0007) [2023-10-10 17:17:56,162][123582] Updated weights for policy 0, policy_version 18583 (0.0007) [2023-10-10 17:17:57,285][123614] Updated weights for policy 1, policy_version 18530 (0.0007) [2023-10-10 17:17:57,647][123614] Updated weights for policy 1, policy_version 18540 (0.0008) [2023-10-10 17:17:58,028][123614] Updated weights for policy 1, policy_version 18550 (0.0009) [2023-10-10 17:17:58,393][123614] Updated weights for policy 1, policy_version 18560 (0.0008) [2023-10-10 17:17:58,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 38043648. Throughput: 0: 1822.5, 1: 1824.9. Samples: 9512332. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:17:58,789][122664] Avg episode reward: [(0, '25.600'), (1, '25.370')] [2023-10-10 17:17:59,874][123582] Updated weights for policy 0, policy_version 18593 (0.0007) [2023-10-10 17:18:00,242][123582] Updated weights for policy 0, policy_version 18603 (0.0008) [2023-10-10 17:18:00,617][123582] Updated weights for policy 0, policy_version 18613 (0.0007) [2023-10-10 17:18:01,006][123582] Updated weights for policy 0, policy_version 18623 (0.0007) [2023-10-10 17:18:02,168][123614] Updated weights for policy 1, policy_version 18570 (0.0007) [2023-10-10 17:18:02,541][123614] Updated weights for policy 1, policy_version 18580 (0.0009) [2023-10-10 17:18:02,909][123614] Updated weights for policy 1, policy_version 18590 (0.0007) [2023-10-10 17:18:03,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 38109184. Throughput: 0: 1830.8, 1: 1821.5. Samples: 9534034. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:18:03,789][122664] Avg episode reward: [(0, '26.660'), (1, '28.130')] [2023-10-10 17:18:04,725][123582] Updated weights for policy 0, policy_version 18633 (0.0009) [2023-10-10 17:18:05,100][123582] Updated weights for policy 0, policy_version 18643 (0.0009) [2023-10-10 17:18:05,475][123582] Updated weights for policy 0, policy_version 18653 (0.0008) [2023-10-10 17:18:06,631][123614] Updated weights for policy 1, policy_version 18600 (0.0008) [2023-10-10 17:18:07,004][123614] Updated weights for policy 1, policy_version 18610 (0.0010) [2023-10-10 17:18:07,377][123614] Updated weights for policy 1, policy_version 18620 (0.0010) [2023-10-10 17:18:08,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 38174720. Throughput: 0: 1817.9, 1: 1824.1. Samples: 9556328. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:18:08,788][122664] Avg episode reward: [(0, '27.380'), (1, '28.650')] [2023-10-10 17:18:09,359][123582] Updated weights for policy 0, policy_version 18663 (0.0010) [2023-10-10 17:18:09,741][123582] Updated weights for policy 0, policy_version 18673 (0.0009) [2023-10-10 17:18:10,118][123582] Updated weights for policy 0, policy_version 18683 (0.0010) [2023-10-10 17:18:11,125][123614] Updated weights for policy 1, policy_version 18630 (0.0010) [2023-10-10 17:18:11,510][123614] Updated weights for policy 1, policy_version 18640 (0.0007) [2023-10-10 17:18:11,876][123614] Updated weights for policy 1, policy_version 18650 (0.0009) [2023-10-10 17:18:13,756][123582] Updated weights for policy 0, policy_version 18693 (0.0009) [2023-10-10 17:18:13,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 38240256. Throughput: 0: 1814.3, 1: 1821.9. Samples: 9566482. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:18:13,789][122664] Avg episode reward: [(0, '27.990'), (1, '28.280')] [2023-10-10 17:18:14,134][123582] Updated weights for policy 0, policy_version 18703 (0.0008) [2023-10-10 17:18:14,498][123582] Updated weights for policy 0, policy_version 18713 (0.0008) [2023-10-10 17:18:15,558][123614] Updated weights for policy 1, policy_version 18660 (0.0009) [2023-10-10 17:18:15,933][123614] Updated weights for policy 1, policy_version 18670 (0.0007) [2023-10-10 17:18:16,305][123614] Updated weights for policy 1, policy_version 18680 (0.0009) [2023-10-10 17:18:18,093][123582] Updated weights for policy 0, policy_version 18723 (0.0009) [2023-10-10 17:18:18,472][123582] Updated weights for policy 0, policy_version 18733 (0.0009) [2023-10-10 17:18:18,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 38305792. Throughput: 0: 1814.4, 1: 1828.4. Samples: 9588934. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-10 17:18:18,789][122664] Avg episode reward: [(0, '26.960'), (1, '26.660')] [2023-10-10 17:18:18,834][123582] Updated weights for policy 0, policy_version 18743 (0.0008) [2023-10-10 17:18:19,971][123614] Updated weights for policy 1, policy_version 18690 (0.0008) [2023-10-10 17:18:20,333][123614] Updated weights for policy 1, policy_version 18700 (0.0008) [2023-10-10 17:18:20,705][123614] Updated weights for policy 1, policy_version 18710 (0.0007) [2023-10-10 17:18:21,070][123614] Updated weights for policy 1, policy_version 18720 (0.0007) [2023-10-10 17:18:22,472][123582] Updated weights for policy 0, policy_version 18753 (0.0008) [2023-10-10 17:18:22,843][123582] Updated weights for policy 0, policy_version 18763 (0.0008) [2023-10-10 17:18:23,221][123582] Updated weights for policy 0, policy_version 18773 (0.0010) [2023-10-10 17:18:23,600][123582] Updated weights for policy 0, policy_version 18783 (0.0007) [2023-10-10 17:18:23,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 38404096. Throughput: 0: 1818.9, 1: 1821.9. Samples: 9610592. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-10 17:18:23,789][122664] Avg episode reward: [(0, '26.760'), (1, '24.520')] [2023-10-10 17:18:24,859][123614] Updated weights for policy 1, policy_version 18730 (0.0009) [2023-10-10 17:18:25,224][123614] Updated weights for policy 1, policy_version 18740 (0.0008) [2023-10-10 17:18:25,603][123614] Updated weights for policy 1, policy_version 18750 (0.0008) [2023-10-10 17:18:27,311][123582] Updated weights for policy 0, policy_version 18793 (0.0010) [2023-10-10 17:18:27,690][123582] Updated weights for policy 0, policy_version 18803 (0.0010) [2023-10-10 17:18:28,072][123582] Updated weights for policy 0, policy_version 18813 (0.0009) [2023-10-10 17:18:28,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 38469632. Throughput: 0: 1818.1, 1: 1820.9. Samples: 9621610. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-10 17:18:28,789][122664] Avg episode reward: [(0, '25.620'), (1, '24.520')] [2023-10-10 17:18:29,391][123614] Updated weights for policy 1, policy_version 18760 (0.0008) [2023-10-10 17:18:29,762][123614] Updated weights for policy 1, policy_version 18770 (0.0008) [2023-10-10 17:18:30,125][123614] Updated weights for policy 1, policy_version 18780 (0.0009) [2023-10-10 17:18:31,805][123582] Updated weights for policy 0, policy_version 18823 (0.0010) [2023-10-10 17:18:32,180][123582] Updated weights for policy 0, policy_version 18833 (0.0007) [2023-10-10 17:18:32,546][123582] Updated weights for policy 0, policy_version 18843 (0.0007) [2023-10-10 17:18:33,437][123614] Updated weights for policy 1, policy_version 18790 (0.0009) [2023-10-10 17:18:33,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 38535168. Throughput: 0: 1818.2, 1: 1828.7. Samples: 9643584. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:18:33,789][122664] Avg episode reward: [(0, '27.550'), (1, '22.660')] [2023-10-10 17:18:33,809][123614] Updated weights for policy 1, policy_version 18800 (0.0010) [2023-10-10 17:18:34,168][123614] Updated weights for policy 1, policy_version 18810 (0.0009) [2023-10-10 17:18:36,148][123582] Updated weights for policy 0, policy_version 18853 (0.0010) [2023-10-10 17:18:36,518][123582] Updated weights for policy 0, policy_version 18863 (0.0008) [2023-10-10 17:18:36,896][123582] Updated weights for policy 0, policy_version 18873 (0.0009) [2023-10-10 17:18:38,100][123614] Updated weights for policy 1, policy_version 18820 (0.0010) [2023-10-10 17:18:38,467][123614] Updated weights for policy 1, policy_version 18830 (0.0007) [2023-10-10 17:18:38,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 38600704. Throughput: 0: 1815.3, 1: 1827.7. Samples: 9664980. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:18:38,789][122664] Avg episode reward: [(0, '28.120'), (1, '23.480')] [2023-10-10 17:18:38,833][123614] Updated weights for policy 1, policy_version 18840 (0.0007) [2023-10-10 17:18:40,444][123582] Updated weights for policy 0, policy_version 18883 (0.0011) [2023-10-10 17:18:40,814][123582] Updated weights for policy 0, policy_version 18893 (0.0009) [2023-10-10 17:18:41,187][123582] Updated weights for policy 0, policy_version 18903 (0.0008) [2023-10-10 17:18:42,445][123614] Updated weights for policy 1, policy_version 18850 (0.0008) [2023-10-10 17:18:42,823][123614] Updated weights for policy 1, policy_version 18860 (0.0008) [2023-10-10 17:18:43,203][123614] Updated weights for policy 1, policy_version 18870 (0.0008) [2023-10-10 17:18:43,562][123614] Updated weights for policy 1, policy_version 18880 (0.0007) [2023-10-10 17:18:43,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 38699008. Throughput: 0: 1819.4, 1: 1820.5. Samples: 9676128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:18:43,789][122664] Avg episode reward: [(0, '27.030'), (1, '23.050')] [2023-10-10 17:18:44,933][123582] Updated weights for policy 0, policy_version 18913 (0.0008) [2023-10-10 17:18:45,306][123582] Updated weights for policy 0, policy_version 18923 (0.0007) [2023-10-10 17:18:45,678][123582] Updated weights for policy 0, policy_version 18933 (0.0007) [2023-10-10 17:18:46,064][123582] Updated weights for policy 0, policy_version 18943 (0.0009) [2023-10-10 17:18:47,068][123614] Updated weights for policy 1, policy_version 18890 (0.0008) [2023-10-10 17:18:47,438][123614] Updated weights for policy 1, policy_version 18900 (0.0007) [2023-10-10 17:18:47,817][123614] Updated weights for policy 1, policy_version 18910 (0.0008) [2023-10-10 17:18:48,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 38764544. Throughput: 0: 1812.4, 1: 1820.8. Samples: 9697528. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:18:48,789][122664] Avg episode reward: [(0, '27.940'), (1, '24.060')] [2023-10-10 17:18:49,889][123582] Updated weights for policy 0, policy_version 18953 (0.0009) [2023-10-10 17:18:50,267][123582] Updated weights for policy 0, policy_version 18963 (0.0011) [2023-10-10 17:18:50,636][123582] Updated weights for policy 0, policy_version 18973 (0.0010) [2023-10-10 17:18:51,535][123614] Updated weights for policy 1, policy_version 18920 (0.0007) [2023-10-10 17:18:51,901][123614] Updated weights for policy 1, policy_version 18930 (0.0007) [2023-10-10 17:18:52,270][123614] Updated weights for policy 1, policy_version 18940 (0.0007) [2023-10-10 17:18:53,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 38830080. Throughput: 0: 1814.7, 1: 1821.0. Samples: 9719936. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:18:53,789][122664] Avg episode reward: [(0, '29.540'), (1, '26.200')] [2023-10-10 17:18:54,310][123582] Updated weights for policy 0, policy_version 18983 (0.0008) [2023-10-10 17:18:54,691][123582] Updated weights for policy 0, policy_version 18993 (0.0011) [2023-10-10 17:18:55,061][123582] Updated weights for policy 0, policy_version 19003 (0.0010) [2023-10-10 17:18:56,082][123614] Updated weights for policy 1, policy_version 18950 (0.0010) [2023-10-10 17:18:56,464][123614] Updated weights for policy 1, policy_version 18960 (0.0008) [2023-10-10 17:18:56,825][123614] Updated weights for policy 1, policy_version 18970 (0.0007) [2023-10-10 17:18:58,686][123582] Updated weights for policy 0, policy_version 19013 (0.0009) [2023-10-10 17:18:58,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 38895616. Throughput: 0: 1820.3, 1: 1817.4. Samples: 9730176. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:18:58,788][122664] Avg episode reward: [(0, '28.920'), (1, '29.690')] [2023-10-10 17:18:59,051][123582] Updated weights for policy 0, policy_version 19023 (0.0009) [2023-10-10 17:18:59,420][123582] Updated weights for policy 0, policy_version 19033 (0.0009) [2023-10-10 17:19:00,602][123614] Updated weights for policy 1, policy_version 18980 (0.0008) [2023-10-10 17:19:00,972][123614] Updated weights for policy 1, policy_version 18990 (0.0009) [2023-10-10 17:19:01,333][123614] Updated weights for policy 1, policy_version 19000 (0.0007) [2023-10-10 17:19:03,075][123582] Updated weights for policy 0, policy_version 19043 (0.0010) [2023-10-10 17:19:03,457][123582] Updated weights for policy 0, policy_version 19053 (0.0007) [2023-10-10 17:19:03,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 38961152. Throughput: 0: 1816.1, 1: 1822.6. Samples: 9752674. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:19:03,789][122664] Avg episode reward: [(0, '29.050'), (1, '30.070')] [2023-10-10 17:19:03,833][123582] Updated weights for policy 0, policy_version 19063 (0.0008) [2023-10-10 17:19:04,923][123614] Updated weights for policy 1, policy_version 19010 (0.0008) [2023-10-10 17:19:05,295][123614] Updated weights for policy 1, policy_version 19020 (0.0011) [2023-10-10 17:19:05,671][123614] Updated weights for policy 1, policy_version 19030 (0.0010) [2023-10-10 17:19:06,041][123614] Updated weights for policy 1, policy_version 19040 (0.0008) [2023-10-10 17:19:07,519][123582] Updated weights for policy 0, policy_version 19073 (0.0010) [2023-10-10 17:19:07,885][123582] Updated weights for policy 0, policy_version 19083 (0.0007) [2023-10-10 17:19:08,255][123582] Updated weights for policy 0, policy_version 19093 (0.0007) [2023-10-10 17:19:08,641][123582] Updated weights for policy 0, policy_version 19103 (0.0007) [2023-10-10 17:19:08,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 39059456. Throughput: 0: 1814.0, 1: 1828.3. Samples: 9774494. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-10 17:19:08,789][122664] Avg episode reward: [(0, '27.700'), (1, '29.880')] [2023-10-10 17:19:09,669][123614] Updated weights for policy 1, policy_version 19050 (0.0008) [2023-10-10 17:19:10,031][123614] Updated weights for policy 1, policy_version 19060 (0.0009) [2023-10-10 17:19:10,408][123614] Updated weights for policy 1, policy_version 19070 (0.0009) [2023-10-10 17:19:12,266][123582] Updated weights for policy 0, policy_version 19113 (0.0008) [2023-10-10 17:19:12,636][123582] Updated weights for policy 0, policy_version 19123 (0.0007) [2023-10-10 17:19:13,015][123582] Updated weights for policy 0, policy_version 19133 (0.0008) [2023-10-10 17:19:13,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 39124992. Throughput: 0: 1819.4, 1: 1827.7. Samples: 9785726. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-10 17:19:13,788][122664] Avg episode reward: [(0, '26.410'), (1, '31.850')] [2023-10-10 17:19:14,115][123614] Updated weights for policy 1, policy_version 19080 (0.0008) [2023-10-10 17:19:14,486][123614] Updated weights for policy 1, policy_version 19090 (0.0008) [2023-10-10 17:19:14,845][123614] Updated weights for policy 1, policy_version 19100 (0.0009) [2023-10-10 17:19:14,991][123465] Saving new best policy, reward=31.850! [2023-10-10 17:19:16,850][123582] Updated weights for policy 0, policy_version 19143 (0.0008) [2023-10-10 17:19:17,224][123582] Updated weights for policy 0, policy_version 19153 (0.0008) [2023-10-10 17:19:17,590][123582] Updated weights for policy 0, policy_version 19163 (0.0008) [2023-10-10 17:19:18,399][123614] Updated weights for policy 1, policy_version 19110 (0.0008) [2023-10-10 17:19:18,769][123614] Updated weights for policy 1, policy_version 19120 (0.0007) [2023-10-10 17:19:18,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 39190528. Throughput: 0: 1822.0, 1: 1823.5. Samples: 9807630. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-10 17:19:18,789][122664] Avg episode reward: [(0, '22.260'), (1, '32.220')] [2023-10-10 17:19:19,134][123614] Updated weights for policy 1, policy_version 19130 (0.0007) [2023-10-10 17:19:19,354][123465] Saving new best policy, reward=32.220! [2023-10-10 17:19:21,379][123582] Updated weights for policy 0, policy_version 19173 (0.0010) [2023-10-10 17:19:21,740][123582] Updated weights for policy 0, policy_version 19183 (0.0010) [2023-10-10 17:19:22,116][123582] Updated weights for policy 0, policy_version 19193 (0.0011) [2023-10-10 17:19:22,843][123614] Updated weights for policy 1, policy_version 19140 (0.0008) [2023-10-10 17:19:23,217][123614] Updated weights for policy 1, policy_version 19150 (0.0010) [2023-10-10 17:19:23,588][123614] Updated weights for policy 1, policy_version 19160 (0.0010) [2023-10-10 17:19:23,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 39256064. Throughput: 0: 1815.2, 1: 1818.3. Samples: 9828486. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-10 17:19:23,788][122664] Avg episode reward: [(0, '18.800'), (1, '30.180')] [2023-10-10 17:19:25,764][123582] Updated weights for policy 0, policy_version 19203 (0.0009) [2023-10-10 17:19:26,129][123582] Updated weights for policy 0, policy_version 19213 (0.0009) [2023-10-10 17:19:26,508][123582] Updated weights for policy 0, policy_version 19223 (0.0011) [2023-10-10 17:19:27,386][123614] Updated weights for policy 1, policy_version 19170 (0.0008) [2023-10-10 17:19:27,761][123614] Updated weights for policy 1, policy_version 19180 (0.0009) [2023-10-10 17:19:28,138][123614] Updated weights for policy 1, policy_version 19190 (0.0009) [2023-10-10 17:19:28,502][123614] Updated weights for policy 1, policy_version 19200 (0.0007) [2023-10-10 17:19:28,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 39354368. Throughput: 0: 1821.3, 1: 1823.1. Samples: 9840128. Policy #0 lag: (min: 26.0, avg: 33.8, max: 58.0) [2023-10-10 17:19:28,789][122664] Avg episode reward: [(0, '19.200'), (1, '29.600')] [2023-10-10 17:19:30,338][123582] Updated weights for policy 0, policy_version 19233 (0.0010) [2023-10-10 17:19:30,708][123582] Updated weights for policy 0, policy_version 19243 (0.0008) [2023-10-10 17:19:31,075][123582] Updated weights for policy 0, policy_version 19253 (0.0007) [2023-10-10 17:19:31,440][123582] Updated weights for policy 0, policy_version 19263 (0.0008) [2023-10-10 17:19:32,214][123614] Updated weights for policy 1, policy_version 19210 (0.0008) [2023-10-10 17:19:32,584][123614] Updated weights for policy 1, policy_version 19220 (0.0008) [2023-10-10 17:19:32,956][123614] Updated weights for policy 1, policy_version 19230 (0.0007) [2023-10-10 17:19:33,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 39419904. Throughput: 0: 1811.2, 1: 1820.6. Samples: 9860958. Policy #0 lag: (min: 26.0, avg: 33.8, max: 58.0) [2023-10-10 17:19:33,789][122664] Avg episode reward: [(0, '20.070'), (1, '28.310')] [2023-10-10 17:19:35,140][123582] Updated weights for policy 0, policy_version 19273 (0.0008) [2023-10-10 17:19:35,512][123582] Updated weights for policy 0, policy_version 19283 (0.0008) [2023-10-10 17:19:35,885][123582] Updated weights for policy 0, policy_version 19293 (0.0009) [2023-10-10 17:19:36,626][123614] Updated weights for policy 1, policy_version 19240 (0.0008) [2023-10-10 17:19:36,990][123614] Updated weights for policy 1, policy_version 19250 (0.0010) [2023-10-10 17:19:37,358][123614] Updated weights for policy 1, policy_version 19260 (0.0011) [2023-10-10 17:19:38,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 39485440. Throughput: 0: 1809.2, 1: 1818.8. Samples: 9883198. Policy #0 lag: (min: 26.0, avg: 33.8, max: 58.0) [2023-10-10 17:19:38,789][122664] Avg episode reward: [(0, '22.050'), (1, '27.960')] [2023-10-10 17:19:39,563][123582] Updated weights for policy 0, policy_version 19303 (0.0010) [2023-10-10 17:19:39,937][123582] Updated weights for policy 0, policy_version 19313 (0.0009) [2023-10-10 17:19:40,310][123582] Updated weights for policy 0, policy_version 19323 (0.0009) [2023-10-10 17:19:41,218][123614] Updated weights for policy 1, policy_version 19270 (0.0010) [2023-10-10 17:19:41,603][123614] Updated weights for policy 1, policy_version 19280 (0.0010) [2023-10-10 17:19:41,977][123614] Updated weights for policy 1, policy_version 19290 (0.0009) [2023-10-10 17:19:43,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 39550976. Throughput: 0: 1810.4, 1: 1823.2. Samples: 9893684. Policy #0 lag: (min: 26.0, avg: 33.8, max: 58.0) [2023-10-10 17:19:43,789][122664] Avg episode reward: [(0, '22.130'), (1, '26.030')] [2023-10-10 17:19:43,881][123582] Updated weights for policy 0, policy_version 19333 (0.0007) [2023-10-10 17:19:44,245][123582] Updated weights for policy 0, policy_version 19343 (0.0011) [2023-10-10 17:19:44,624][123582] Updated weights for policy 0, policy_version 19353 (0.0010) [2023-10-10 17:19:45,826][123614] Updated weights for policy 1, policy_version 19300 (0.0010) [2023-10-10 17:19:46,188][123614] Updated weights for policy 1, policy_version 19310 (0.0008) [2023-10-10 17:19:46,550][123614] Updated weights for policy 1, policy_version 19320 (0.0010) [2023-10-10 17:19:48,347][123582] Updated weights for policy 0, policy_version 19363 (0.0008) [2023-10-10 17:19:48,724][123582] Updated weights for policy 0, policy_version 19373 (0.0007) [2023-10-10 17:19:48,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 39616512. Throughput: 0: 1813.0, 1: 1811.4. Samples: 9915770. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:19:48,789][122664] Avg episode reward: [(0, '23.720'), (1, '27.330')] [2023-10-10 17:19:49,094][123582] Updated weights for policy 0, policy_version 19383 (0.0008) [2023-10-10 17:19:50,219][123614] Updated weights for policy 1, policy_version 19330 (0.0011) [2023-10-10 17:19:50,580][123614] Updated weights for policy 1, policy_version 19340 (0.0008) [2023-10-10 17:19:50,947][123614] Updated weights for policy 1, policy_version 19350 (0.0010) [2023-10-10 17:19:51,318][123614] Updated weights for policy 1, policy_version 19360 (0.0010) [2023-10-10 17:19:52,745][123582] Updated weights for policy 0, policy_version 19393 (0.0008) [2023-10-10 17:19:53,113][123582] Updated weights for policy 0, policy_version 19403 (0.0009) [2023-10-10 17:19:53,485][123582] Updated weights for policy 0, policy_version 19413 (0.0007) [2023-10-10 17:19:53,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 39682048. Throughput: 0: 1821.2, 1: 1808.8. Samples: 9937846. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:19:53,789][122664] Avg episode reward: [(0, '25.000'), (1, '25.770')] [2023-10-10 17:19:53,799][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000019360_19824640.pth... [2023-10-10 17:19:53,827][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000017664_18087936.pth [2023-10-10 17:19:53,856][123582] Updated weights for policy 0, policy_version 19423 (0.0007) [2023-10-10 17:19:53,890][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000019424_19890176.pth... [2023-10-10 17:19:53,918][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000017728_18153472.pth [2023-10-10 17:19:54,957][123614] Updated weights for policy 1, policy_version 19370 (0.0009) [2023-10-10 17:19:55,327][123614] Updated weights for policy 1, policy_version 19380 (0.0008) [2023-10-10 17:19:55,693][123614] Updated weights for policy 1, policy_version 19390 (0.0007) [2023-10-10 17:19:57,474][123582] Updated weights for policy 0, policy_version 19433 (0.0008) [2023-10-10 17:19:57,853][123582] Updated weights for policy 0, policy_version 19443 (0.0007) [2023-10-10 17:19:58,225][123582] Updated weights for policy 0, policy_version 19453 (0.0010) [2023-10-10 17:19:58,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 39780352. Throughput: 0: 1813.8, 1: 1809.4. Samples: 9948770. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:19:58,789][122664] Avg episode reward: [(0, '26.280'), (1, '25.610')] [2023-10-10 17:19:59,416][123614] Updated weights for policy 1, policy_version 19400 (0.0008) [2023-10-10 17:19:59,781][123614] Updated weights for policy 1, policy_version 19410 (0.0009) [2023-10-10 17:20:00,152][123614] Updated weights for policy 1, policy_version 19420 (0.0011) [2023-10-10 17:20:01,914][123582] Updated weights for policy 0, policy_version 19463 (0.0010) [2023-10-10 17:20:02,284][123582] Updated weights for policy 0, policy_version 19473 (0.0011) [2023-10-10 17:20:02,665][123582] Updated weights for policy 0, policy_version 19483 (0.0009) [2023-10-10 17:20:03,735][123614] Updated weights for policy 1, policy_version 19430 (0.0010) [2023-10-10 17:20:03,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 39845888. Throughput: 0: 1819.1, 1: 1807.6. Samples: 9970830. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:20:03,789][122664] Avg episode reward: [(0, '27.380'), (1, '26.940')] [2023-10-10 17:20:04,118][123614] Updated weights for policy 1, policy_version 19440 (0.0010) [2023-10-10 17:20:04,488][123614] Updated weights for policy 1, policy_version 19450 (0.0009) [2023-10-10 17:20:06,418][123582] Updated weights for policy 0, policy_version 19493 (0.0009) [2023-10-10 17:20:06,794][123582] Updated weights for policy 0, policy_version 19503 (0.0009) [2023-10-10 17:20:07,171][123582] Updated weights for policy 0, policy_version 19513 (0.0008) [2023-10-10 17:20:08,267][123614] Updated weights for policy 1, policy_version 19460 (0.0010) [2023-10-10 17:20:08,633][123614] Updated weights for policy 1, policy_version 19470 (0.0011) [2023-10-10 17:20:08,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 39911424. Throughput: 0: 1820.0, 1: 1818.2. Samples: 9992204. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:20:08,789][122664] Avg episode reward: [(0, '28.970'), (1, '28.890')] [2023-10-10 17:20:09,006][123614] Updated weights for policy 1, policy_version 19480 (0.0010) [2023-10-10 17:20:10,920][123582] Updated weights for policy 0, policy_version 19523 (0.0008) [2023-10-10 17:20:11,301][123582] Updated weights for policy 0, policy_version 19533 (0.0007) [2023-10-10 17:20:11,673][123582] Updated weights for policy 0, policy_version 19543 (0.0008) [2023-10-10 17:20:12,665][123614] Updated weights for policy 1, policy_version 19490 (0.0010) [2023-10-10 17:20:13,023][123614] Updated weights for policy 1, policy_version 19500 (0.0007) [2023-10-10 17:20:13,388][123614] Updated weights for policy 1, policy_version 19510 (0.0007) [2023-10-10 17:20:13,771][123614] Updated weights for policy 1, policy_version 19520 (0.0008) [2023-10-10 17:20:13,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 40009728. Throughput: 0: 1821.8, 1: 1805.5. Samples: 10003356. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:20:13,789][122664] Avg episode reward: [(0, '28.280'), (1, '27.930')] [2023-10-10 17:20:15,298][123582] Updated weights for policy 0, policy_version 19553 (0.0009) [2023-10-10 17:20:15,669][123582] Updated weights for policy 0, policy_version 19563 (0.0009) [2023-10-10 17:20:16,044][123582] Updated weights for policy 0, policy_version 19573 (0.0010) [2023-10-10 17:20:16,416][123582] Updated weights for policy 0, policy_version 19583 (0.0008) [2023-10-10 17:20:17,385][123614] Updated weights for policy 1, policy_version 19530 (0.0008) [2023-10-10 17:20:17,765][123614] Updated weights for policy 1, policy_version 19540 (0.0010) [2023-10-10 17:20:18,132][123614] Updated weights for policy 1, policy_version 19550 (0.0010) [2023-10-10 17:20:18,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 40075264. Throughput: 0: 1821.7, 1: 1819.9. Samples: 10024830. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:20:18,789][122664] Avg episode reward: [(0, '27.740'), (1, '27.310')] [2023-10-10 17:20:19,994][123582] Updated weights for policy 0, policy_version 19593 (0.0007) [2023-10-10 17:20:20,364][123582] Updated weights for policy 0, policy_version 19603 (0.0007) [2023-10-10 17:20:20,746][123582] Updated weights for policy 0, policy_version 19613 (0.0009) [2023-10-10 17:20:21,778][123614] Updated weights for policy 1, policy_version 19560 (0.0009) [2023-10-10 17:20:22,149][123614] Updated weights for policy 1, policy_version 19570 (0.0009) [2023-10-10 17:20:22,514][123614] Updated weights for policy 1, policy_version 19580 (0.0008) [2023-10-10 17:20:23,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 40140800. Throughput: 0: 1833.2, 1: 1811.2. Samples: 10047194. Policy #0 lag: (min: 24.0, avg: 49.1, max: 56.0) [2023-10-10 17:20:23,789][122664] Avg episode reward: [(0, '29.570'), (1, '26.290')] [2023-10-10 17:20:24,371][123582] Updated weights for policy 0, policy_version 19623 (0.0008) [2023-10-10 17:20:24,741][123582] Updated weights for policy 0, policy_version 19633 (0.0009) [2023-10-10 17:20:25,117][123582] Updated weights for policy 0, policy_version 19643 (0.0007) [2023-10-10 17:20:26,312][123614] Updated weights for policy 1, policy_version 19590 (0.0009) [2023-10-10 17:20:26,682][123614] Updated weights for policy 1, policy_version 19600 (0.0010) [2023-10-10 17:20:27,052][123614] Updated weights for policy 1, policy_version 19610 (0.0010) [2023-10-10 17:20:28,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 40206336. Throughput: 0: 1829.5, 1: 1814.5. Samples: 10057664. Policy #0 lag: (min: 24.0, avg: 49.1, max: 56.0) [2023-10-10 17:20:28,789][122664] Avg episode reward: [(0, '30.430'), (1, '27.900')] [2023-10-10 17:20:28,943][123582] Updated weights for policy 0, policy_version 19653 (0.0011) [2023-10-10 17:20:29,309][123582] Updated weights for policy 0, policy_version 19663 (0.0009) [2023-10-10 17:20:29,691][123582] Updated weights for policy 0, policy_version 19673 (0.0008) [2023-10-10 17:20:30,960][123614] Updated weights for policy 1, policy_version 19620 (0.0010) [2023-10-10 17:20:31,317][123614] Updated weights for policy 1, policy_version 19630 (0.0009) [2023-10-10 17:20:31,695][123614] Updated weights for policy 1, policy_version 19640 (0.0009) [2023-10-10 17:20:33,324][123582] Updated weights for policy 0, policy_version 19683 (0.0009) [2023-10-10 17:20:33,699][123582] Updated weights for policy 0, policy_version 19693 (0.0008) [2023-10-10 17:20:33,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 40271872. Throughput: 0: 1821.6, 1: 1810.3. Samples: 10079206. Policy #0 lag: (min: 24.0, avg: 49.1, max: 56.0) [2023-10-10 17:20:33,788][122664] Avg episode reward: [(0, '30.840'), (1, '29.940')] [2023-10-10 17:20:34,079][123582] Updated weights for policy 0, policy_version 19703 (0.0008) [2023-10-10 17:20:35,331][123614] Updated weights for policy 1, policy_version 19650 (0.0008) [2023-10-10 17:20:35,702][123614] Updated weights for policy 1, policy_version 19660 (0.0011) [2023-10-10 17:20:36,062][123614] Updated weights for policy 1, policy_version 19670 (0.0010) [2023-10-10 17:20:36,429][123614] Updated weights for policy 1, policy_version 19680 (0.0010) [2023-10-10 17:20:37,807][123582] Updated weights for policy 0, policy_version 19713 (0.0010) [2023-10-10 17:20:38,172][123582] Updated weights for policy 0, policy_version 19723 (0.0010) [2023-10-10 17:20:38,539][123582] Updated weights for policy 0, policy_version 19733 (0.0010) [2023-10-10 17:20:38,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 40337408. Throughput: 0: 1823.7, 1: 1806.4. Samples: 10101200. Policy #0 lag: (min: 24.0, avg: 49.1, max: 56.0) [2023-10-10 17:20:38,788][122664] Avg episode reward: [(0, '30.810'), (1, '30.530')] [2023-10-10 17:20:38,917][123582] Updated weights for policy 0, policy_version 19743 (0.0007) [2023-10-10 17:20:40,099][123614] Updated weights for policy 1, policy_version 19690 (0.0009) [2023-10-10 17:20:40,462][123614] Updated weights for policy 1, policy_version 19700 (0.0008) [2023-10-10 17:20:40,834][123614] Updated weights for policy 1, policy_version 19710 (0.0008) [2023-10-10 17:20:42,526][123582] Updated weights for policy 0, policy_version 19753 (0.0009) [2023-10-10 17:20:42,903][123582] Updated weights for policy 0, policy_version 19763 (0.0009) [2023-10-10 17:20:43,278][123582] Updated weights for policy 0, policy_version 19773 (0.0008) [2023-10-10 17:20:43,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 40435712. Throughput: 0: 1820.5, 1: 1805.1. Samples: 10111920. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-10-10 17:20:43,789][122664] Avg episode reward: [(0, '30.780'), (1, '29.040')] [2023-10-10 17:20:44,605][123614] Updated weights for policy 1, policy_version 19720 (0.0009) [2023-10-10 17:20:44,972][123614] Updated weights for policy 1, policy_version 19730 (0.0010) [2023-10-10 17:20:45,344][123614] Updated weights for policy 1, policy_version 19740 (0.0009) [2023-10-10 17:20:46,747][123582] Updated weights for policy 0, policy_version 19783 (0.0007) [2023-10-10 17:20:47,124][123582] Updated weights for policy 0, policy_version 19793 (0.0007) [2023-10-10 17:20:47,501][123582] Updated weights for policy 0, policy_version 19803 (0.0009) [2023-10-10 17:20:48,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 40501248. Throughput: 0: 1824.2, 1: 1801.4. Samples: 10133982. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-10-10 17:20:48,789][122664] Avg episode reward: [(0, '31.620'), (1, '28.030')] [2023-10-10 17:20:49,017][123614] Updated weights for policy 1, policy_version 19750 (0.0008) [2023-10-10 17:20:49,385][123614] Updated weights for policy 1, policy_version 19760 (0.0008) [2023-10-10 17:20:49,755][123614] Updated weights for policy 1, policy_version 19770 (0.0008) [2023-10-10 17:20:51,114][123582] Updated weights for policy 0, policy_version 19813 (0.0007) [2023-10-10 17:20:51,491][123582] Updated weights for policy 0, policy_version 19823 (0.0007) [2023-10-10 17:20:51,856][123582] Updated weights for policy 0, policy_version 19833 (0.0008) [2023-10-10 17:20:53,451][123614] Updated weights for policy 1, policy_version 19780 (0.0008) [2023-10-10 17:20:53,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 40566784. Throughput: 0: 1828.1, 1: 1810.3. Samples: 10155930. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-10-10 17:20:53,789][122664] Avg episode reward: [(0, '31.300'), (1, '28.420')] [2023-10-10 17:20:53,832][123614] Updated weights for policy 1, policy_version 19790 (0.0008) [2023-10-10 17:20:54,202][123614] Updated weights for policy 1, policy_version 19800 (0.0009) [2023-10-10 17:20:55,492][123582] Updated weights for policy 0, policy_version 19843 (0.0008) [2023-10-10 17:20:55,864][123582] Updated weights for policy 0, policy_version 19853 (0.0009) [2023-10-10 17:20:56,242][123582] Updated weights for policy 0, policy_version 19863 (0.0010) [2023-10-10 17:20:57,878][123614] Updated weights for policy 1, policy_version 19810 (0.0008) [2023-10-10 17:20:58,250][123614] Updated weights for policy 1, policy_version 19820 (0.0010) [2023-10-10 17:20:58,613][123614] Updated weights for policy 1, policy_version 19830 (0.0008) [2023-10-10 17:20:58,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 40632320. Throughput: 0: 1821.0, 1: 1806.1. Samples: 10166572. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:20:58,789][122664] Avg episode reward: [(0, '29.140'), (1, '28.890')] [2023-10-10 17:20:58,979][123614] Updated weights for policy 1, policy_version 19840 (0.0009) [2023-10-10 17:20:59,909][123582] Updated weights for policy 0, policy_version 19873 (0.0009) [2023-10-10 17:21:00,289][123582] Updated weights for policy 0, policy_version 19883 (0.0008) [2023-10-10 17:21:00,663][123582] Updated weights for policy 0, policy_version 19893 (0.0008) [2023-10-10 17:21:01,024][123582] Updated weights for policy 0, policy_version 19903 (0.0007) [2023-10-10 17:21:02,693][123614] Updated weights for policy 1, policy_version 19850 (0.0010) [2023-10-10 17:21:03,057][123614] Updated weights for policy 1, policy_version 19860 (0.0009) [2023-10-10 17:21:03,435][123614] Updated weights for policy 1, policy_version 19870 (0.0008) [2023-10-10 17:21:03,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 40730624. Throughput: 0: 1830.7, 1: 1810.1. Samples: 10188666. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:21:03,789][122664] Avg episode reward: [(0, '27.550'), (1, '30.090')] [2023-10-10 17:21:04,816][123582] Updated weights for policy 0, policy_version 19913 (0.0008) [2023-10-10 17:21:05,184][123582] Updated weights for policy 0, policy_version 19923 (0.0008) [2023-10-10 17:21:05,561][123582] Updated weights for policy 0, policy_version 19933 (0.0009) [2023-10-10 17:21:07,054][123614] Updated weights for policy 1, policy_version 19880 (0.0008) [2023-10-10 17:21:07,426][123614] Updated weights for policy 1, policy_version 19890 (0.0009) [2023-10-10 17:21:07,793][123614] Updated weights for policy 1, policy_version 19900 (0.0011) [2023-10-10 17:21:08,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 40796160. Throughput: 0: 1818.8, 1: 1810.3. Samples: 10210506. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:21:08,788][122664] Avg episode reward: [(0, '26.210'), (1, '31.440')] [2023-10-10 17:21:09,399][123582] Updated weights for policy 0, policy_version 19943 (0.0007) [2023-10-10 17:21:09,776][123582] Updated weights for policy 0, policy_version 19953 (0.0008) [2023-10-10 17:21:10,147][123582] Updated weights for policy 0, policy_version 19963 (0.0008) [2023-10-10 17:21:11,639][123614] Updated weights for policy 1, policy_version 19910 (0.0009) [2023-10-10 17:21:12,023][123614] Updated weights for policy 1, policy_version 19920 (0.0010) [2023-10-10 17:21:12,391][123614] Updated weights for policy 1, policy_version 19930 (0.0011) [2023-10-10 17:21:13,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 40861696. Throughput: 0: 1816.5, 1: 1816.5. Samples: 10221152. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:21:13,788][122664] Avg episode reward: [(0, '26.170'), (1, '32.710')] [2023-10-10 17:21:13,789][123465] Saving new best policy, reward=32.710! [2023-10-10 17:21:13,867][123582] Updated weights for policy 0, policy_version 19973 (0.0008) [2023-10-10 17:21:14,233][123582] Updated weights for policy 0, policy_version 19983 (0.0011) [2023-10-10 17:21:14,614][123582] Updated weights for policy 0, policy_version 19993 (0.0011) [2023-10-10 17:21:16,085][123614] Updated weights for policy 1, policy_version 19940 (0.0009) [2023-10-10 17:21:16,447][123614] Updated weights for policy 1, policy_version 19950 (0.0009) [2023-10-10 17:21:16,816][123614] Updated weights for policy 1, policy_version 19960 (0.0010) [2023-10-10 17:21:18,385][123582] Updated weights for policy 0, policy_version 20003 (0.0008) [2023-10-10 17:21:18,763][123582] Updated weights for policy 0, policy_version 20013 (0.0008) [2023-10-10 17:21:18,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 40927232. Throughput: 0: 1822.0, 1: 1816.8. Samples: 10242956. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:21:18,788][122664] Avg episode reward: [(0, '25.490'), (1, '31.520')] [2023-10-10 17:21:19,123][123582] Updated weights for policy 0, policy_version 20023 (0.0009) [2023-10-10 17:21:20,583][123614] Updated weights for policy 1, policy_version 19970 (0.0008) [2023-10-10 17:21:20,957][123614] Updated weights for policy 1, policy_version 19980 (0.0010) [2023-10-10 17:21:21,320][123614] Updated weights for policy 1, policy_version 19990 (0.0008) [2023-10-10 17:21:21,691][123614] Updated weights for policy 1, policy_version 20000 (0.0007) [2023-10-10 17:21:23,009][123582] Updated weights for policy 0, policy_version 20033 (0.0010) [2023-10-10 17:21:23,397][123582] Updated weights for policy 0, policy_version 20043 (0.0011) [2023-10-10 17:21:23,763][123582] Updated weights for policy 0, policy_version 20053 (0.0009) [2023-10-10 17:21:23,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 40992768. Throughput: 0: 1817.0, 1: 1814.8. Samples: 10264632. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:21:23,788][122664] Avg episode reward: [(0, '28.810'), (1, '32.640')] [2023-10-10 17:21:24,140][123582] Updated weights for policy 0, policy_version 20063 (0.0012) [2023-10-10 17:21:25,317][123614] Updated weights for policy 1, policy_version 20010 (0.0008) [2023-10-10 17:21:25,677][123614] Updated weights for policy 1, policy_version 20020 (0.0009) [2023-10-10 17:21:26,053][123614] Updated weights for policy 1, policy_version 20030 (0.0009) [2023-10-10 17:21:27,689][123582] Updated weights for policy 0, policy_version 20073 (0.0008) [2023-10-10 17:21:28,071][123582] Updated weights for policy 0, policy_version 20083 (0.0009) [2023-10-10 17:21:28,446][123582] Updated weights for policy 0, policy_version 20093 (0.0009) [2023-10-10 17:21:28,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 41091072. Throughput: 0: 1813.1, 1: 1815.5. Samples: 10275206. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:21:28,789][122664] Avg episode reward: [(0, '30.080'), (1, '30.780')] [2023-10-10 17:21:29,833][123614] Updated weights for policy 1, policy_version 20040 (0.0009) [2023-10-10 17:21:30,207][123614] Updated weights for policy 1, policy_version 20050 (0.0007) [2023-10-10 17:21:30,584][123614] Updated weights for policy 1, policy_version 20060 (0.0008) [2023-10-10 17:21:32,009][123582] Updated weights for policy 0, policy_version 20103 (0.0010) [2023-10-10 17:21:32,376][123582] Updated weights for policy 0, policy_version 20113 (0.0007) [2023-10-10 17:21:32,755][123582] Updated weights for policy 0, policy_version 20123 (0.0008) [2023-10-10 17:21:33,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 41156608. Throughput: 0: 1815.9, 1: 1812.9. Samples: 10297276. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:21:33,788][122664] Avg episode reward: [(0, '29.820'), (1, '29.940')] [2023-10-10 17:21:34,298][123614] Updated weights for policy 1, policy_version 20070 (0.0007) [2023-10-10 17:21:34,657][123614] Updated weights for policy 1, policy_version 20080 (0.0008) [2023-10-10 17:21:35,024][123614] Updated weights for policy 1, policy_version 20090 (0.0011) [2023-10-10 17:21:36,425][123582] Updated weights for policy 0, policy_version 20133 (0.0007) [2023-10-10 17:21:36,788][123582] Updated weights for policy 0, policy_version 20143 (0.0011) [2023-10-10 17:21:37,161][123582] Updated weights for policy 0, policy_version 20153 (0.0010) [2023-10-10 17:21:38,576][123614] Updated weights for policy 1, policy_version 20100 (0.0009) [2023-10-10 17:21:38,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 41222144. Throughput: 0: 1808.7, 1: 1820.9. Samples: 10319262. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:21:38,789][122664] Avg episode reward: [(0, '29.940'), (1, '26.970')] [2023-10-10 17:21:38,941][123614] Updated weights for policy 1, policy_version 20110 (0.0007) [2023-10-10 17:21:39,310][123614] Updated weights for policy 1, policy_version 20120 (0.0008) [2023-10-10 17:21:40,842][123582] Updated weights for policy 0, policy_version 20163 (0.0011) [2023-10-10 17:21:41,214][123582] Updated weights for policy 0, policy_version 20173 (0.0010) [2023-10-10 17:21:41,597][123582] Updated weights for policy 0, policy_version 20183 (0.0009) [2023-10-10 17:21:43,004][123614] Updated weights for policy 1, policy_version 20130 (0.0011) [2023-10-10 17:21:43,373][123614] Updated weights for policy 1, policy_version 20140 (0.0010) [2023-10-10 17:21:43,753][123614] Updated weights for policy 1, policy_version 20150 (0.0009) [2023-10-10 17:21:43,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 41287680. Throughput: 0: 1818.1, 1: 1820.8. Samples: 10330324. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:21:43,789][122664] Avg episode reward: [(0, '29.860'), (1, '24.530')] [2023-10-10 17:21:44,116][123614] Updated weights for policy 1, policy_version 20160 (0.0010) [2023-10-10 17:21:45,188][123582] Updated weights for policy 0, policy_version 20193 (0.0009) [2023-10-10 17:21:45,560][123582] Updated weights for policy 0, policy_version 20203 (0.0009) [2023-10-10 17:21:45,934][123582] Updated weights for policy 0, policy_version 20213 (0.0010) [2023-10-10 17:21:46,313][123582] Updated weights for policy 0, policy_version 20223 (0.0008) [2023-10-10 17:21:47,748][123614] Updated weights for policy 1, policy_version 20170 (0.0011) [2023-10-10 17:21:48,127][123614] Updated weights for policy 1, policy_version 20180 (0.0010) [2023-10-10 17:21:48,495][123614] Updated weights for policy 1, policy_version 20190 (0.0008) [2023-10-10 17:21:48,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 41385984. Throughput: 0: 1810.2, 1: 1823.8. Samples: 10352194. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:21:48,790][122664] Avg episode reward: [(0, '29.480'), (1, '23.850')] [2023-10-10 17:21:49,979][123582] Updated weights for policy 0, policy_version 20233 (0.0009) [2023-10-10 17:21:50,348][123582] Updated weights for policy 0, policy_version 20243 (0.0010) [2023-10-10 17:21:50,711][123582] Updated weights for policy 0, policy_version 20253 (0.0009) [2023-10-10 17:21:52,149][123614] Updated weights for policy 1, policy_version 20200 (0.0008) [2023-10-10 17:21:52,510][123614] Updated weights for policy 1, policy_version 20210 (0.0007) [2023-10-10 17:21:52,881][123614] Updated weights for policy 1, policy_version 20220 (0.0009) [2023-10-10 17:21:53,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 41451520. Throughput: 0: 1820.3, 1: 1817.5. Samples: 10374206. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:21:53,789][122664] Avg episode reward: [(0, '29.420'), (1, '26.190')] [2023-10-10 17:21:53,799][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000020224_20709376.pth... [2023-10-10 17:21:53,800][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000020256_20742144.pth... [2023-10-10 17:21:53,831][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000018528_18972672.pth [2023-10-10 17:21:53,838][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000018560_19005440.pth [2023-10-10 17:21:54,292][123582] Updated weights for policy 0, policy_version 20263 (0.0008) [2023-10-10 17:21:54,664][123582] Updated weights for policy 0, policy_version 20273 (0.0010) [2023-10-10 17:21:55,028][123582] Updated weights for policy 0, policy_version 20283 (0.0007) [2023-10-10 17:21:56,652][123614] Updated weights for policy 1, policy_version 20230 (0.0009) [2023-10-10 17:21:57,046][123614] Updated weights for policy 1, policy_version 20240 (0.0007) [2023-10-10 17:21:57,423][123614] Updated weights for policy 1, policy_version 20250 (0.0008) [2023-10-10 17:21:58,729][123582] Updated weights for policy 0, policy_version 20293 (0.0010) [2023-10-10 17:21:58,788][122664] Fps is (10 sec: 13107.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 41517056. Throughput: 0: 1823.4, 1: 1819.2. Samples: 10385070. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:21:58,788][122664] Avg episode reward: [(0, '28.680'), (1, '27.150')] [2023-10-10 17:21:59,106][123582] Updated weights for policy 0, policy_version 20303 (0.0011) [2023-10-10 17:21:59,475][123582] Updated weights for policy 0, policy_version 20313 (0.0009) [2023-10-10 17:22:01,101][123614] Updated weights for policy 1, policy_version 20260 (0.0009) [2023-10-10 17:22:01,471][123614] Updated weights for policy 1, policy_version 20270 (0.0008) [2023-10-10 17:22:01,840][123614] Updated weights for policy 1, policy_version 20280 (0.0008) [2023-10-10 17:22:03,253][123582] Updated weights for policy 0, policy_version 20323 (0.0010) [2023-10-10 17:22:03,624][123582] Updated weights for policy 0, policy_version 20333 (0.0009) [2023-10-10 17:22:03,788][122664] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 41582592. Throughput: 0: 1818.7, 1: 1815.6. Samples: 10406498. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:22:03,788][122664] Avg episode reward: [(0, '30.220'), (1, '29.310')] [2023-10-10 17:22:03,996][123582] Updated weights for policy 0, policy_version 20343 (0.0008) [2023-10-10 17:22:05,379][123614] Updated weights for policy 1, policy_version 20290 (0.0008) [2023-10-10 17:22:05,744][123614] Updated weights for policy 1, policy_version 20300 (0.0008) [2023-10-10 17:22:06,117][123614] Updated weights for policy 1, policy_version 20310 (0.0008) [2023-10-10 17:22:06,483][123614] Updated weights for policy 1, policy_version 20320 (0.0007) [2023-10-10 17:22:07,702][123582] Updated weights for policy 0, policy_version 20353 (0.0007) [2023-10-10 17:22:08,076][123582] Updated weights for policy 0, policy_version 20363 (0.0010) [2023-10-10 17:22:08,458][123582] Updated weights for policy 0, policy_version 20373 (0.0010) [2023-10-10 17:22:08,788][122664] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 41648128. Throughput: 0: 1821.1, 1: 1824.3. Samples: 10428678. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:22:08,789][122664] Avg episode reward: [(0, '28.840'), (1, '28.400')] [2023-10-10 17:22:08,823][123582] Updated weights for policy 0, policy_version 20383 (0.0011) [2023-10-10 17:22:10,179][123614] Updated weights for policy 1, policy_version 20330 (0.0008) [2023-10-10 17:22:10,544][123614] Updated weights for policy 1, policy_version 20340 (0.0010) [2023-10-10 17:22:10,913][123614] Updated weights for policy 1, policy_version 20350 (0.0009) [2023-10-10 17:22:12,558][123582] Updated weights for policy 0, policy_version 20393 (0.0010) [2023-10-10 17:22:12,932][123582] Updated weights for policy 0, policy_version 20403 (0.0010) [2023-10-10 17:22:13,301][123582] Updated weights for policy 0, policy_version 20413 (0.0010) [2023-10-10 17:22:13,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 41746432. Throughput: 0: 1825.9, 1: 1826.5. Samples: 10439564. Policy #0 lag: (min: 18.0, avg: 23.6, max: 50.0) [2023-10-10 17:22:13,788][122664] Avg episode reward: [(0, '29.420'), (1, '28.630')] [2023-10-10 17:22:14,777][123614] Updated weights for policy 1, policy_version 20360 (0.0010) [2023-10-10 17:22:15,148][123614] Updated weights for policy 1, policy_version 20370 (0.0008) [2023-10-10 17:22:15,518][123614] Updated weights for policy 1, policy_version 20380 (0.0007) [2023-10-10 17:22:17,048][123582] Updated weights for policy 0, policy_version 20423 (0.0008) [2023-10-10 17:22:17,416][123582] Updated weights for policy 0, policy_version 20433 (0.0007) [2023-10-10 17:22:17,786][123582] Updated weights for policy 0, policy_version 20443 (0.0009) [2023-10-10 17:22:18,788][122664] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 41811968. Throughput: 0: 1827.4, 1: 1824.9. Samples: 10461630. Policy #0 lag: (min: 18.0, avg: 23.6, max: 50.0) [2023-10-10 17:22:18,788][122664] Avg episode reward: [(0, '28.280'), (1, '27.770')] [2023-10-10 17:22:19,109][123614] Updated weights for policy 1, policy_version 20390 (0.0008) [2023-10-10 17:22:19,476][123614] Updated weights for policy 1, policy_version 20400 (0.0010) [2023-10-10 17:22:19,848][123614] Updated weights for policy 1, policy_version 20410 (0.0008) [2023-10-10 17:22:21,407][123582] Updated weights for policy 0, policy_version 20453 (0.0007) [2023-10-10 17:22:21,769][123582] Updated weights for policy 0, policy_version 20463 (0.0008) [2023-10-10 17:22:22,140][123582] Updated weights for policy 0, policy_version 20473 (0.0007) [2023-10-10 17:22:23,476][123614] Updated weights for policy 1, policy_version 20420 (0.0009) [2023-10-10 17:22:23,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 41877504. Throughput: 0: 1831.1, 1: 1823.2. Samples: 10483702. Policy #0 lag: (min: 18.0, avg: 23.6, max: 50.0) [2023-10-10 17:22:23,789][122664] Avg episode reward: [(0, '27.700'), (1, '25.750')] [2023-10-10 17:22:23,851][123614] Updated weights for policy 1, policy_version 20430 (0.0007) [2023-10-10 17:22:24,223][123614] Updated weights for policy 1, policy_version 20440 (0.0008) [2023-10-10 17:22:25,776][123582] Updated weights for policy 0, policy_version 20483 (0.0009) [2023-10-10 17:22:26,136][123582] Updated weights for policy 0, policy_version 20493 (0.0009) [2023-10-10 17:22:26,513][123582] Updated weights for policy 0, policy_version 20503 (0.0010) [2023-10-10 17:22:27,825][123614] Updated weights for policy 1, policy_version 20450 (0.0009) [2023-10-10 17:22:28,190][123614] Updated weights for policy 1, policy_version 20460 (0.0007) [2023-10-10 17:22:28,560][123614] Updated weights for policy 1, policy_version 20470 (0.0010) [2023-10-10 17:22:28,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 41943040. Throughput: 0: 1826.2, 1: 1824.9. Samples: 10494624. Policy #0 lag: (min: 30.0, avg: 33.0, max: 62.0) [2023-10-10 17:22:28,789][122664] Avg episode reward: [(0, '27.240'), (1, '22.130')] [2023-10-10 17:22:28,918][123614] Updated weights for policy 1, policy_version 20480 (0.0010) [2023-10-10 17:22:30,271][123582] Updated weights for policy 0, policy_version 20513 (0.0010) [2023-10-10 17:22:30,638][123582] Updated weights for policy 0, policy_version 20523 (0.0009) [2023-10-10 17:22:31,006][123582] Updated weights for policy 0, policy_version 20533 (0.0008) [2023-10-10 17:22:31,382][123582] Updated weights for policy 0, policy_version 20543 (0.0011) [2023-10-10 17:22:32,633][123614] Updated weights for policy 1, policy_version 20490 (0.0007) [2023-10-10 17:22:32,999][123614] Updated weights for policy 1, policy_version 20500 (0.0007) [2023-10-10 17:22:33,362][123614] Updated weights for policy 1, policy_version 20510 (0.0008) [2023-10-10 17:22:33,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 42041344. Throughput: 0: 1827.1, 1: 1816.5. Samples: 10516152. Policy #0 lag: (min: 30.0, avg: 33.0, max: 62.0) [2023-10-10 17:22:33,789][122664] Avg episode reward: [(0, '27.680'), (1, '20.360')] [2023-10-10 17:22:34,908][123582] Updated weights for policy 0, policy_version 20553 (0.0011) [2023-10-10 17:22:35,288][123582] Updated weights for policy 0, policy_version 20563 (0.0007) [2023-10-10 17:22:35,658][123582] Updated weights for policy 0, policy_version 20573 (0.0008) [2023-10-10 17:22:36,956][123614] Updated weights for policy 1, policy_version 20520 (0.0007) [2023-10-10 17:22:37,322][123614] Updated weights for policy 1, policy_version 20530 (0.0009) [2023-10-10 17:22:37,690][123614] Updated weights for policy 1, policy_version 20540 (0.0010) [2023-10-10 17:22:38,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 42106880. Throughput: 0: 1819.9, 1: 1825.4. Samples: 10538242. Policy #0 lag: (min: 30.0, avg: 33.0, max: 62.0) [2023-10-10 17:22:38,788][122664] Avg episode reward: [(0, '26.380'), (1, '20.160')] [2023-10-10 17:22:39,467][123582] Updated weights for policy 0, policy_version 20583 (0.0008) [2023-10-10 17:22:39,846][123582] Updated weights for policy 0, policy_version 20593 (0.0007) [2023-10-10 17:22:40,222][123582] Updated weights for policy 0, policy_version 20603 (0.0007) [2023-10-10 17:22:41,495][123614] Updated weights for policy 1, policy_version 20550 (0.0008) [2023-10-10 17:22:41,882][123614] Updated weights for policy 1, policy_version 20560 (0.0008) [2023-10-10 17:22:42,258][123614] Updated weights for policy 1, policy_version 20570 (0.0011) [2023-10-10 17:22:43,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 42172416. Throughput: 0: 1819.1, 1: 1819.6. Samples: 10548810. Policy #0 lag: (min: 30.0, avg: 33.0, max: 62.0) [2023-10-10 17:22:43,789][122664] Avg episode reward: [(0, '21.850'), (1, '21.480')] [2023-10-10 17:22:43,848][123582] Updated weights for policy 0, policy_version 20613 (0.0007) [2023-10-10 17:22:44,222][123582] Updated weights for policy 0, policy_version 20623 (0.0010) [2023-10-10 17:22:44,590][123582] Updated weights for policy 0, policy_version 20633 (0.0009) [2023-10-10 17:22:46,011][123614] Updated weights for policy 1, policy_version 20580 (0.0009) [2023-10-10 17:22:46,377][123614] Updated weights for policy 1, policy_version 20590 (0.0008) [2023-10-10 17:22:46,741][123614] Updated weights for policy 1, policy_version 20600 (0.0009) [2023-10-10 17:22:48,240][123582] Updated weights for policy 0, policy_version 20643 (0.0009) [2023-10-10 17:22:48,608][123582] Updated weights for policy 0, policy_version 20653 (0.0008) [2023-10-10 17:22:48,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.6, 300 sec: 14551.2). Total num frames: 42237952. Throughput: 0: 1830.0, 1: 1821.6. Samples: 10570820. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:22:48,788][122664] Avg episode reward: [(0, '22.870'), (1, '23.030')] [2023-10-10 17:22:48,984][123582] Updated weights for policy 0, policy_version 20663 (0.0009) [2023-10-10 17:22:50,548][123614] Updated weights for policy 1, policy_version 20610 (0.0008) [2023-10-10 17:22:50,912][123614] Updated weights for policy 1, policy_version 20620 (0.0010) [2023-10-10 17:22:51,281][123614] Updated weights for policy 1, policy_version 20630 (0.0010) [2023-10-10 17:22:51,643][123614] Updated weights for policy 1, policy_version 20640 (0.0008) [2023-10-10 17:22:52,606][123582] Updated weights for policy 0, policy_version 20673 (0.0008) [2023-10-10 17:22:52,980][123582] Updated weights for policy 0, policy_version 20683 (0.0008) [2023-10-10 17:22:53,359][123582] Updated weights for policy 0, policy_version 20693 (0.0007) [2023-10-10 17:22:53,731][123582] Updated weights for policy 0, policy_version 20703 (0.0009) [2023-10-10 17:22:53,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 42336256. Throughput: 0: 1829.8, 1: 1817.4. Samples: 10592800. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:22:53,789][122664] Avg episode reward: [(0, '23.410'), (1, '24.490')] [2023-10-10 17:22:55,281][123614] Updated weights for policy 1, policy_version 20650 (0.0009) [2023-10-10 17:22:55,647][123614] Updated weights for policy 1, policy_version 20660 (0.0011) [2023-10-10 17:22:56,023][123614] Updated weights for policy 1, policy_version 20670 (0.0009) [2023-10-10 17:22:57,463][123582] Updated weights for policy 0, policy_version 20713 (0.0010) [2023-10-10 17:22:57,831][123582] Updated weights for policy 0, policy_version 20723 (0.0007) [2023-10-10 17:22:58,209][123582] Updated weights for policy 0, policy_version 20733 (0.0008) [2023-10-10 17:22:58,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 42401792. Throughput: 0: 1828.6, 1: 1814.0. Samples: 10603480. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:22:58,789][122664] Avg episode reward: [(0, '25.580'), (1, '24.590')] [2023-10-10 17:22:59,851][123614] Updated weights for policy 1, policy_version 20680 (0.0008) [2023-10-10 17:23:00,230][123614] Updated weights for policy 1, policy_version 20690 (0.0008) [2023-10-10 17:23:00,598][123614] Updated weights for policy 1, policy_version 20700 (0.0010) [2023-10-10 17:23:01,940][123582] Updated weights for policy 0, policy_version 20743 (0.0008) [2023-10-10 17:23:02,313][123582] Updated weights for policy 0, policy_version 20753 (0.0007) [2023-10-10 17:23:02,681][123582] Updated weights for policy 0, policy_version 20763 (0.0007) [2023-10-10 17:23:03,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 42467328. Throughput: 0: 1818.9, 1: 1820.5. Samples: 10625404. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:23:03,789][122664] Avg episode reward: [(0, '27.080'), (1, '26.740')] [2023-10-10 17:23:04,088][123614] Updated weights for policy 1, policy_version 20710 (0.0009) [2023-10-10 17:23:04,450][123614] Updated weights for policy 1, policy_version 20720 (0.0009) [2023-10-10 17:23:04,811][123614] Updated weights for policy 1, policy_version 20730 (0.0011) [2023-10-10 17:23:06,281][123582] Updated weights for policy 0, policy_version 20773 (0.0008) [2023-10-10 17:23:06,658][123582] Updated weights for policy 0, policy_version 20783 (0.0008) [2023-10-10 17:23:07,031][123582] Updated weights for policy 0, policy_version 20793 (0.0009) [2023-10-10 17:23:08,352][123614] Updated weights for policy 1, policy_version 20740 (0.0008) [2023-10-10 17:23:08,724][123614] Updated weights for policy 1, policy_version 20750 (0.0007) [2023-10-10 17:23:08,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 42532864. Throughput: 0: 1819.2, 1: 1816.9. Samples: 10647326. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:23:08,789][122664] Avg episode reward: [(0, '24.040'), (1, '30.170')] [2023-10-10 17:23:09,092][123614] Updated weights for policy 1, policy_version 20760 (0.0009) [2023-10-10 17:23:10,692][123582] Updated weights for policy 0, policy_version 20803 (0.0009) [2023-10-10 17:23:11,058][123582] Updated weights for policy 0, policy_version 20813 (0.0009) [2023-10-10 17:23:11,433][123582] Updated weights for policy 0, policy_version 20823 (0.0007) [2023-10-10 17:23:12,717][123614] Updated weights for policy 1, policy_version 20770 (0.0008) [2023-10-10 17:23:13,085][123614] Updated weights for policy 1, policy_version 20780 (0.0008) [2023-10-10 17:23:13,449][123614] Updated weights for policy 1, policy_version 20790 (0.0007) [2023-10-10 17:23:13,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 42598400. Throughput: 0: 1813.9, 1: 1818.3. Samples: 10658070. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:23:13,789][122664] Avg episode reward: [(0, '23.840'), (1, '29.820')] [2023-10-10 17:23:13,816][123614] Updated weights for policy 1, policy_version 20800 (0.0007) [2023-10-10 17:23:15,071][123582] Updated weights for policy 0, policy_version 20833 (0.0007) [2023-10-10 17:23:15,435][123582] Updated weights for policy 0, policy_version 20843 (0.0008) [2023-10-10 17:23:15,813][123582] Updated weights for policy 0, policy_version 20853 (0.0007) [2023-10-10 17:23:16,188][123582] Updated weights for policy 0, policy_version 20863 (0.0010) [2023-10-10 17:23:17,541][123614] Updated weights for policy 1, policy_version 20810 (0.0010) [2023-10-10 17:23:17,910][123614] Updated weights for policy 1, policy_version 20820 (0.0009) [2023-10-10 17:23:18,293][123614] Updated weights for policy 1, policy_version 20830 (0.0010) [2023-10-10 17:23:18,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 42696704. Throughput: 0: 1820.6, 1: 1817.9. Samples: 10679882. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:23:18,789][122664] Avg episode reward: [(0, '24.580'), (1, '30.510')] [2023-10-10 17:23:19,889][123582] Updated weights for policy 0, policy_version 20873 (0.0011) [2023-10-10 17:23:20,256][123582] Updated weights for policy 0, policy_version 20883 (0.0010) [2023-10-10 17:23:20,627][123582] Updated weights for policy 0, policy_version 20893 (0.0010) [2023-10-10 17:23:22,165][123614] Updated weights for policy 1, policy_version 20840 (0.0008) [2023-10-10 17:23:22,539][123614] Updated weights for policy 1, policy_version 20850 (0.0008) [2023-10-10 17:23:22,919][123614] Updated weights for policy 1, policy_version 20860 (0.0008) [2023-10-10 17:23:23,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 42762240. Throughput: 0: 1822.4, 1: 1808.9. Samples: 10701652. Policy #0 lag: (min: 3.0, avg: 3.0, max: 5.0) [2023-10-10 17:23:23,788][122664] Avg episode reward: [(0, '23.610'), (1, '30.400')] [2023-10-10 17:23:24,341][123582] Updated weights for policy 0, policy_version 20903 (0.0009) [2023-10-10 17:23:24,709][123582] Updated weights for policy 0, policy_version 20913 (0.0009) [2023-10-10 17:23:25,080][123582] Updated weights for policy 0, policy_version 20923 (0.0009) [2023-10-10 17:23:26,611][123614] Updated weights for policy 1, policy_version 20870 (0.0009) [2023-10-10 17:23:26,995][123614] Updated weights for policy 1, policy_version 20880 (0.0009) [2023-10-10 17:23:27,363][123614] Updated weights for policy 1, policy_version 20890 (0.0008) [2023-10-10 17:23:28,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 42827776. Throughput: 0: 1822.0, 1: 1815.1. Samples: 10712478. Policy #0 lag: (min: 3.0, avg: 3.0, max: 5.0) [2023-10-10 17:23:28,789][122664] Avg episode reward: [(0, '22.390'), (1, '30.370')] [2023-10-10 17:23:28,825][123582] Updated weights for policy 0, policy_version 20933 (0.0010) [2023-10-10 17:23:29,191][123582] Updated weights for policy 0, policy_version 20943 (0.0008) [2023-10-10 17:23:29,560][123582] Updated weights for policy 0, policy_version 20953 (0.0009) [2023-10-10 17:23:31,010][123614] Updated weights for policy 1, policy_version 20900 (0.0009) [2023-10-10 17:23:31,377][123614] Updated weights for policy 1, policy_version 20910 (0.0009) [2023-10-10 17:23:31,751][123614] Updated weights for policy 1, policy_version 20920 (0.0009) [2023-10-10 17:23:33,391][123582] Updated weights for policy 0, policy_version 20963 (0.0009) [2023-10-10 17:23:33,768][123582] Updated weights for policy 0, policy_version 20973 (0.0011) [2023-10-10 17:23:33,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 42893312. Throughput: 0: 1810.6, 1: 1813.0. Samples: 10733884. Policy #0 lag: (min: 3.0, avg: 3.0, max: 5.0) [2023-10-10 17:23:33,789][122664] Avg episode reward: [(0, '22.530'), (1, '30.760')] [2023-10-10 17:23:34,139][123582] Updated weights for policy 0, policy_version 20983 (0.0007) [2023-10-10 17:23:35,444][123614] Updated weights for policy 1, policy_version 20930 (0.0010) [2023-10-10 17:23:35,805][123614] Updated weights for policy 1, policy_version 20940 (0.0008) [2023-10-10 17:23:36,178][123614] Updated weights for policy 1, policy_version 20950 (0.0010) [2023-10-10 17:23:36,548][123614] Updated weights for policy 1, policy_version 20960 (0.0010) [2023-10-10 17:23:37,735][123582] Updated weights for policy 0, policy_version 20993 (0.0008) [2023-10-10 17:23:38,109][123582] Updated weights for policy 0, policy_version 21003 (0.0008) [2023-10-10 17:23:38,492][123582] Updated weights for policy 0, policy_version 21013 (0.0008) [2023-10-10 17:23:38,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 42958848. Throughput: 0: 1810.1, 1: 1813.9. Samples: 10755880. Policy #0 lag: (min: 3.0, avg: 3.0, max: 5.0) [2023-10-10 17:23:38,789][122664] Avg episode reward: [(0, '22.010'), (1, '31.470')] [2023-10-10 17:23:38,854][123582] Updated weights for policy 0, policy_version 21023 (0.0007) [2023-10-10 17:23:40,439][123614] Updated weights for policy 1, policy_version 20970 (0.0007) [2023-10-10 17:23:40,802][123614] Updated weights for policy 1, policy_version 20980 (0.0007) [2023-10-10 17:23:41,175][123614] Updated weights for policy 1, policy_version 20990 (0.0010) [2023-10-10 17:23:42,565][123582] Updated weights for policy 0, policy_version 21033 (0.0010) [2023-10-10 17:23:42,936][123582] Updated weights for policy 0, policy_version 21043 (0.0009) [2023-10-10 17:23:43,309][123582] Updated weights for policy 0, policy_version 21053 (0.0010) [2023-10-10 17:23:43,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 43057152. Throughput: 0: 1809.2, 1: 1813.8. Samples: 10766512. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-10 17:23:43,789][122664] Avg episode reward: [(0, '23.790'), (1, '28.220')] [2023-10-10 17:23:44,988][123614] Updated weights for policy 1, policy_version 21000 (0.0007) [2023-10-10 17:23:45,356][123614] Updated weights for policy 1, policy_version 21010 (0.0008) [2023-10-10 17:23:45,730][123614] Updated weights for policy 1, policy_version 21020 (0.0008) [2023-10-10 17:23:46,960][123582] Updated weights for policy 0, policy_version 21063 (0.0008) [2023-10-10 17:23:47,339][123582] Updated weights for policy 0, policy_version 21073 (0.0009) [2023-10-10 17:23:47,710][123582] Updated weights for policy 0, policy_version 21083 (0.0011) [2023-10-10 17:23:48,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 43122688. Throughput: 0: 1813.7, 1: 1807.0. Samples: 10788334. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-10 17:23:48,789][122664] Avg episode reward: [(0, '24.300'), (1, '27.600')] [2023-10-10 17:23:49,457][123614] Updated weights for policy 1, policy_version 21030 (0.0008) [2023-10-10 17:23:49,826][123614] Updated weights for policy 1, policy_version 21040 (0.0009) [2023-10-10 17:23:50,204][123614] Updated weights for policy 1, policy_version 21050 (0.0008) [2023-10-10 17:23:51,245][123582] Updated weights for policy 0, policy_version 21093 (0.0008) [2023-10-10 17:23:51,621][123582] Updated weights for policy 0, policy_version 21103 (0.0008) [2023-10-10 17:23:51,998][123582] Updated weights for policy 0, policy_version 21113 (0.0007) [2023-10-10 17:23:53,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 43188224. Throughput: 0: 1812.0, 1: 1815.0. Samples: 10810544. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-10 17:23:53,789][122664] Avg episode reward: [(0, '23.210'), (1, '28.790')] [2023-10-10 17:23:53,802][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000021120_21626880.pth... [2023-10-10 17:23:53,838][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000019424_19890176.pth [2023-10-10 17:23:53,911][123614] Updated weights for policy 1, policy_version 21060 (0.0008) [2023-10-10 17:23:54,288][123614] Updated weights for policy 1, policy_version 21070 (0.0008) [2023-10-10 17:23:54,648][123614] Updated weights for policy 1, policy_version 21080 (0.0007) [2023-10-10 17:23:54,946][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000021088_21594112.pth... [2023-10-10 17:23:54,975][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000019360_19824640.pth [2023-10-10 17:23:55,772][123582] Updated weights for policy 0, policy_version 21123 (0.0009) [2023-10-10 17:23:56,144][123582] Updated weights for policy 0, policy_version 21133 (0.0010) [2023-10-10 17:23:56,522][123582] Updated weights for policy 0, policy_version 21143 (0.0011) [2023-10-10 17:23:58,118][123614] Updated weights for policy 1, policy_version 21090 (0.0008) [2023-10-10 17:23:58,498][123614] Updated weights for policy 1, policy_version 21100 (0.0010) [2023-10-10 17:23:58,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 43253760. Throughput: 0: 1816.4, 1: 1807.2. Samples: 10821134. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-10 17:23:58,789][122664] Avg episode reward: [(0, '23.100'), (1, '27.110')] [2023-10-10 17:23:58,863][123614] Updated weights for policy 1, policy_version 21110 (0.0009) [2023-10-10 17:23:59,230][123614] Updated weights for policy 1, policy_version 21120 (0.0010) [2023-10-10 17:24:00,315][123582] Updated weights for policy 0, policy_version 21153 (0.0010) [2023-10-10 17:24:00,686][123582] Updated weights for policy 0, policy_version 21163 (0.0008) [2023-10-10 17:24:01,062][123582] Updated weights for policy 0, policy_version 21173 (0.0007) [2023-10-10 17:24:01,441][123582] Updated weights for policy 0, policy_version 21183 (0.0007) [2023-10-10 17:24:02,950][123614] Updated weights for policy 1, policy_version 21130 (0.0010) [2023-10-10 17:24:03,320][123614] Updated weights for policy 1, policy_version 21140 (0.0011) [2023-10-10 17:24:03,686][123614] Updated weights for policy 1, policy_version 21150 (0.0008) [2023-10-10 17:24:03,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 43352064. Throughput: 0: 1813.2, 1: 1820.1. Samples: 10843384. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-10 17:24:03,789][122664] Avg episode reward: [(0, '22.870'), (1, '25.300')] [2023-10-10 17:24:04,917][123582] Updated weights for policy 0, policy_version 21193 (0.0008) [2023-10-10 17:24:05,279][123582] Updated weights for policy 0, policy_version 21203 (0.0009) [2023-10-10 17:24:05,653][123582] Updated weights for policy 0, policy_version 21213 (0.0008) [2023-10-10 17:24:07,257][123614] Updated weights for policy 1, policy_version 21160 (0.0008) [2023-10-10 17:24:07,639][123614] Updated weights for policy 1, policy_version 21170 (0.0008) [2023-10-10 17:24:08,008][123614] Updated weights for policy 1, policy_version 21180 (0.0008) [2023-10-10 17:24:08,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 43417600. Throughput: 0: 1817.6, 1: 1816.9. Samples: 10865208. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-10 17:24:08,789][122664] Avg episode reward: [(0, '25.100'), (1, '24.210')] [2023-10-10 17:24:09,332][123582] Updated weights for policy 0, policy_version 21223 (0.0008) [2023-10-10 17:24:09,711][123582] Updated weights for policy 0, policy_version 21233 (0.0010) [2023-10-10 17:24:10,090][123582] Updated weights for policy 0, policy_version 21243 (0.0010) [2023-10-10 17:24:11,748][123614] Updated weights for policy 1, policy_version 21190 (0.0007) [2023-10-10 17:24:12,142][123614] Updated weights for policy 1, policy_version 21200 (0.0007) [2023-10-10 17:24:12,504][123614] Updated weights for policy 1, policy_version 21210 (0.0007) [2023-10-10 17:24:13,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 43483136. Throughput: 0: 1813.6, 1: 1819.7. Samples: 10875976. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-10 17:24:13,788][122664] Avg episode reward: [(0, '24.250'), (1, '24.700')] [2023-10-10 17:24:13,849][123582] Updated weights for policy 0, policy_version 21253 (0.0010) [2023-10-10 17:24:14,225][123582] Updated weights for policy 0, policy_version 21263 (0.0009) [2023-10-10 17:24:14,589][123582] Updated weights for policy 0, policy_version 21273 (0.0008) [2023-10-10 17:24:16,169][123614] Updated weights for policy 1, policy_version 21220 (0.0008) [2023-10-10 17:24:16,539][123614] Updated weights for policy 1, policy_version 21230 (0.0007) [2023-10-10 17:24:16,912][123614] Updated weights for policy 1, policy_version 21240 (0.0010) [2023-10-10 17:24:18,290][123582] Updated weights for policy 0, policy_version 21283 (0.0008) [2023-10-10 17:24:18,667][123582] Updated weights for policy 0, policy_version 21293 (0.0009) [2023-10-10 17:24:18,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 43548672. Throughput: 0: 1819.3, 1: 1818.3. Samples: 10897574. Policy #0 lag: (min: 31.0, avg: 39.1, max: 63.0) [2023-10-10 17:24:18,788][122664] Avg episode reward: [(0, '24.300'), (1, '25.060')] [2023-10-10 17:24:19,040][123582] Updated weights for policy 0, policy_version 21303 (0.0008) [2023-10-10 17:24:20,529][123614] Updated weights for policy 1, policy_version 21250 (0.0008) [2023-10-10 17:24:20,905][123614] Updated weights for policy 1, policy_version 21260 (0.0008) [2023-10-10 17:24:21,273][123614] Updated weights for policy 1, policy_version 21270 (0.0009) [2023-10-10 17:24:21,647][123614] Updated weights for policy 1, policy_version 21280 (0.0009) [2023-10-10 17:24:22,748][123582] Updated weights for policy 0, policy_version 21313 (0.0008) [2023-10-10 17:24:23,124][123582] Updated weights for policy 0, policy_version 21323 (0.0010) [2023-10-10 17:24:23,501][123582] Updated weights for policy 0, policy_version 21333 (0.0010) [2023-10-10 17:24:23,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 43614208. Throughput: 0: 1821.2, 1: 1816.8. Samples: 10919590. Policy #0 lag: (min: 31.0, avg: 39.1, max: 63.0) [2023-10-10 17:24:23,788][122664] Avg episode reward: [(0, '24.580'), (1, '23.840')] [2023-10-10 17:24:23,876][123582] Updated weights for policy 0, policy_version 21343 (0.0011) [2023-10-10 17:24:25,487][123614] Updated weights for policy 1, policy_version 21290 (0.0009) [2023-10-10 17:24:25,859][123614] Updated weights for policy 1, policy_version 21300 (0.0010) [2023-10-10 17:24:26,234][123614] Updated weights for policy 1, policy_version 21310 (0.0008) [2023-10-10 17:24:27,676][123582] Updated weights for policy 0, policy_version 21353 (0.0008) [2023-10-10 17:24:28,058][123582] Updated weights for policy 0, policy_version 21363 (0.0008) [2023-10-10 17:24:28,435][123582] Updated weights for policy 0, policy_version 21373 (0.0008) [2023-10-10 17:24:28,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 43712512. Throughput: 0: 1821.3, 1: 1818.1. Samples: 10930286. Policy #0 lag: (min: 31.0, avg: 39.1, max: 63.0) [2023-10-10 17:24:28,788][122664] Avg episode reward: [(0, '25.780'), (1, '26.570')] [2023-10-10 17:24:30,082][123614] Updated weights for policy 1, policy_version 21320 (0.0010) [2023-10-10 17:24:30,452][123614] Updated weights for policy 1, policy_version 21330 (0.0008) [2023-10-10 17:24:30,814][123614] Updated weights for policy 1, policy_version 21340 (0.0007) [2023-10-10 17:24:32,104][123582] Updated weights for policy 0, policy_version 21383 (0.0007) [2023-10-10 17:24:32,481][123582] Updated weights for policy 0, policy_version 21393 (0.0007) [2023-10-10 17:24:32,853][123582] Updated weights for policy 0, policy_version 21403 (0.0008) [2023-10-10 17:24:33,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 43778048. Throughput: 0: 1822.3, 1: 1818.8. Samples: 10952182. Policy #0 lag: (min: 12.0, avg: 20.0, max: 44.0) [2023-10-10 17:24:33,789][122664] Avg episode reward: [(0, '25.370'), (1, '26.070')] [2023-10-10 17:24:34,490][123614] Updated weights for policy 1, policy_version 21350 (0.0008) [2023-10-10 17:24:34,861][123614] Updated weights for policy 1, policy_version 21360 (0.0008) [2023-10-10 17:24:35,226][123614] Updated weights for policy 1, policy_version 21370 (0.0008) [2023-10-10 17:24:36,460][123582] Updated weights for policy 0, policy_version 21413 (0.0008) [2023-10-10 17:24:36,836][123582] Updated weights for policy 0, policy_version 21423 (0.0007) [2023-10-10 17:24:37,219][123582] Updated weights for policy 0, policy_version 21433 (0.0008) [2023-10-10 17:24:38,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 43843584. Throughput: 0: 1815.8, 1: 1819.6. Samples: 10974136. Policy #0 lag: (min: 12.0, avg: 20.0, max: 44.0) [2023-10-10 17:24:38,788][122664] Avg episode reward: [(0, '23.660'), (1, '27.250')] [2023-10-10 17:24:38,800][123614] Updated weights for policy 1, policy_version 21380 (0.0007) [2023-10-10 17:24:39,157][123614] Updated weights for policy 1, policy_version 21390 (0.0009) [2023-10-10 17:24:39,520][123614] Updated weights for policy 1, policy_version 21400 (0.0009) [2023-10-10 17:24:41,040][123582] Updated weights for policy 0, policy_version 21443 (0.0008) [2023-10-10 17:24:41,412][123582] Updated weights for policy 0, policy_version 21453 (0.0007) [2023-10-10 17:24:41,785][123582] Updated weights for policy 0, policy_version 21463 (0.0008) [2023-10-10 17:24:43,267][123614] Updated weights for policy 1, policy_version 21410 (0.0008) [2023-10-10 17:24:43,631][123614] Updated weights for policy 1, policy_version 21420 (0.0007) [2023-10-10 17:24:43,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 43909120. Throughput: 0: 1825.2, 1: 1815.9. Samples: 10984980. Policy #0 lag: (min: 12.0, avg: 20.0, max: 44.0) [2023-10-10 17:24:43,789][122664] Avg episode reward: [(0, '23.890'), (1, '26.040')] [2023-10-10 17:24:43,997][123614] Updated weights for policy 1, policy_version 21430 (0.0008) [2023-10-10 17:24:44,364][123614] Updated weights for policy 1, policy_version 21440 (0.0010) [2023-10-10 17:24:45,483][123582] Updated weights for policy 0, policy_version 21473 (0.0009) [2023-10-10 17:24:45,853][123582] Updated weights for policy 0, policy_version 21483 (0.0008) [2023-10-10 17:24:46,222][123582] Updated weights for policy 0, policy_version 21493 (0.0009) [2023-10-10 17:24:46,589][123582] Updated weights for policy 0, policy_version 21503 (0.0009) [2023-10-10 17:24:48,044][123614] Updated weights for policy 1, policy_version 21450 (0.0007) [2023-10-10 17:24:48,413][123614] Updated weights for policy 1, policy_version 21460 (0.0007) [2023-10-10 17:24:48,789][123614] Updated weights for policy 1, policy_version 21470 (0.0009) [2023-10-10 17:24:48,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 43974656. Throughput: 0: 1812.0, 1: 1822.9. Samples: 11006954. Policy #0 lag: (min: 12.0, avg: 20.0, max: 44.0) [2023-10-10 17:24:48,789][122664] Avg episode reward: [(0, '22.950'), (1, '29.800')] [2023-10-10 17:24:50,324][123582] Updated weights for policy 0, policy_version 21513 (0.0007) [2023-10-10 17:24:50,693][123582] Updated weights for policy 0, policy_version 21523 (0.0008) [2023-10-10 17:24:51,060][123582] Updated weights for policy 0, policy_version 21533 (0.0008) [2023-10-10 17:24:52,528][123614] Updated weights for policy 1, policy_version 21480 (0.0008) [2023-10-10 17:24:52,902][123614] Updated weights for policy 1, policy_version 21490 (0.0008) [2023-10-10 17:24:53,266][123614] Updated weights for policy 1, policy_version 21500 (0.0010) [2023-10-10 17:24:53,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 44072960. Throughput: 0: 1807.7, 1: 1813.5. Samples: 11028160. Policy #0 lag: (min: 10.0, avg: 10.1, max: 17.0) [2023-10-10 17:24:53,789][122664] Avg episode reward: [(0, '20.900'), (1, '29.270')] [2023-10-10 17:24:54,690][123582] Updated weights for policy 0, policy_version 21543 (0.0008) [2023-10-10 17:24:55,066][123582] Updated weights for policy 0, policy_version 21553 (0.0009) [2023-10-10 17:24:55,439][123582] Updated weights for policy 0, policy_version 21563 (0.0008) [2023-10-10 17:24:56,807][123614] Updated weights for policy 1, policy_version 21510 (0.0010) [2023-10-10 17:24:57,188][123614] Updated weights for policy 1, policy_version 21520 (0.0009) [2023-10-10 17:24:57,555][123614] Updated weights for policy 1, policy_version 21530 (0.0007) [2023-10-10 17:24:58,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 44138496. Throughput: 0: 1813.3, 1: 1821.6. Samples: 11039550. Policy #0 lag: (min: 10.0, avg: 10.1, max: 17.0) [2023-10-10 17:24:58,789][122664] Avg episode reward: [(0, '20.260'), (1, '30.610')] [2023-10-10 17:24:59,228][123582] Updated weights for policy 0, policy_version 21573 (0.0008) [2023-10-10 17:24:59,599][123582] Updated weights for policy 0, policy_version 21583 (0.0007) [2023-10-10 17:24:59,973][123582] Updated weights for policy 0, policy_version 21593 (0.0007) [2023-10-10 17:25:01,364][123614] Updated weights for policy 1, policy_version 21540 (0.0008) [2023-10-10 17:25:01,732][123614] Updated weights for policy 1, policy_version 21550 (0.0008) [2023-10-10 17:25:02,101][123614] Updated weights for policy 1, policy_version 21560 (0.0011) [2023-10-10 17:25:03,772][123582] Updated weights for policy 0, policy_version 21603 (0.0009) [2023-10-10 17:25:03,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 44204032. Throughput: 0: 1807.5, 1: 1814.7. Samples: 11060570. Policy #0 lag: (min: 10.0, avg: 10.1, max: 17.0) [2023-10-10 17:25:03,789][122664] Avg episode reward: [(0, '21.570'), (1, '32.810')] [2023-10-10 17:25:03,790][123465] Saving new best policy, reward=32.810! [2023-10-10 17:25:04,157][123582] Updated weights for policy 0, policy_version 21613 (0.0008) [2023-10-10 17:25:04,532][123582] Updated weights for policy 0, policy_version 21623 (0.0008) [2023-10-10 17:25:05,776][123614] Updated weights for policy 1, policy_version 21570 (0.0008) [2023-10-10 17:25:06,147][123614] Updated weights for policy 1, policy_version 21580 (0.0008) [2023-10-10 17:25:06,524][123614] Updated weights for policy 1, policy_version 21590 (0.0010) [2023-10-10 17:25:06,891][123614] Updated weights for policy 1, policy_version 21600 (0.0010) [2023-10-10 17:25:08,053][123582] Updated weights for policy 0, policy_version 21633 (0.0008) [2023-10-10 17:25:08,422][123582] Updated weights for policy 0, policy_version 21643 (0.0009) [2023-10-10 17:25:08,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 44269568. Throughput: 0: 1810.0, 1: 1815.9. Samples: 11082754. Policy #0 lag: (min: 10.0, avg: 10.1, max: 17.0) [2023-10-10 17:25:08,789][122664] Avg episode reward: [(0, '22.370'), (1, '30.790')] [2023-10-10 17:25:08,796][123582] Updated weights for policy 0, policy_version 21653 (0.0010) [2023-10-10 17:25:09,163][123582] Updated weights for policy 0, policy_version 21663 (0.0011) [2023-10-10 17:25:10,585][123614] Updated weights for policy 1, policy_version 21610 (0.0008) [2023-10-10 17:25:10,954][123614] Updated weights for policy 1, policy_version 21620 (0.0007) [2023-10-10 17:25:11,322][123614] Updated weights for policy 1, policy_version 21630 (0.0007) [2023-10-10 17:25:12,982][123582] Updated weights for policy 0, policy_version 21673 (0.0008) [2023-10-10 17:25:13,350][123582] Updated weights for policy 0, policy_version 21683 (0.0010) [2023-10-10 17:25:13,725][123582] Updated weights for policy 0, policy_version 21693 (0.0008) [2023-10-10 17:25:13,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 44335104. Throughput: 0: 1802.8, 1: 1818.5. Samples: 11093244. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-10 17:25:13,788][122664] Avg episode reward: [(0, '22.970'), (1, '29.420')] [2023-10-10 17:25:14,821][123614] Updated weights for policy 1, policy_version 21640 (0.0008) [2023-10-10 17:25:15,203][123614] Updated weights for policy 1, policy_version 21650 (0.0007) [2023-10-10 17:25:15,569][123614] Updated weights for policy 1, policy_version 21660 (0.0007) [2023-10-10 17:25:17,342][123582] Updated weights for policy 0, policy_version 21703 (0.0008) [2023-10-10 17:25:17,710][123582] Updated weights for policy 0, policy_version 21713 (0.0008) [2023-10-10 17:25:18,085][123582] Updated weights for policy 0, policy_version 21723 (0.0011) [2023-10-10 17:25:18,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 44433408. Throughput: 0: 1810.3, 1: 1825.9. Samples: 11115810. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-10 17:25:18,788][122664] Avg episode reward: [(0, '23.160'), (1, '30.030')] [2023-10-10 17:25:19,271][123614] Updated weights for policy 1, policy_version 21670 (0.0009) [2023-10-10 17:25:19,640][123614] Updated weights for policy 1, policy_version 21680 (0.0007) [2023-10-10 17:25:20,007][123614] Updated weights for policy 1, policy_version 21690 (0.0010) [2023-10-10 17:25:21,875][123582] Updated weights for policy 0, policy_version 21733 (0.0010) [2023-10-10 17:25:22,246][123582] Updated weights for policy 0, policy_version 21743 (0.0010) [2023-10-10 17:25:22,626][123582] Updated weights for policy 0, policy_version 21753 (0.0010) [2023-10-10 17:25:23,756][123614] Updated weights for policy 1, policy_version 21700 (0.0009) [2023-10-10 17:25:23,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 44498944. Throughput: 0: 1804.6, 1: 1824.4. Samples: 11137440. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-10 17:25:23,789][122664] Avg episode reward: [(0, '24.690'), (1, '28.820')] [2023-10-10 17:25:24,131][123614] Updated weights for policy 1, policy_version 21710 (0.0008) [2023-10-10 17:25:24,504][123614] Updated weights for policy 1, policy_version 21720 (0.0010) [2023-10-10 17:25:26,382][123582] Updated weights for policy 0, policy_version 21763 (0.0008) [2023-10-10 17:25:26,759][123582] Updated weights for policy 0, policy_version 21773 (0.0008) [2023-10-10 17:25:27,132][123582] Updated weights for policy 0, policy_version 21783 (0.0010) [2023-10-10 17:25:28,093][123614] Updated weights for policy 1, policy_version 21730 (0.0010) [2023-10-10 17:25:28,466][123614] Updated weights for policy 1, policy_version 21740 (0.0008) [2023-10-10 17:25:28,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 44564480. Throughput: 0: 1811.7, 1: 1829.5. Samples: 11148834. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:25:28,789][122664] Avg episode reward: [(0, '27.550'), (1, '30.080')] [2023-10-10 17:25:28,833][123614] Updated weights for policy 1, policy_version 21750 (0.0007) [2023-10-10 17:25:29,203][123614] Updated weights for policy 1, policy_version 21760 (0.0007) [2023-10-10 17:25:30,788][123582] Updated weights for policy 0, policy_version 21793 (0.0010) [2023-10-10 17:25:31,172][123582] Updated weights for policy 0, policy_version 21803 (0.0011) [2023-10-10 17:25:31,547][123582] Updated weights for policy 0, policy_version 21813 (0.0008) [2023-10-10 17:25:31,923][123582] Updated weights for policy 0, policy_version 21823 (0.0007) [2023-10-10 17:25:32,920][123614] Updated weights for policy 1, policy_version 21770 (0.0010) [2023-10-10 17:25:33,293][123614] Updated weights for policy 1, policy_version 21780 (0.0010) [2023-10-10 17:25:33,658][123614] Updated weights for policy 1, policy_version 21790 (0.0008) [2023-10-10 17:25:33,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 44662784. Throughput: 0: 1804.3, 1: 1820.8. Samples: 11170084. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:25:33,788][122664] Avg episode reward: [(0, '31.010'), (1, '28.390')] [2023-10-10 17:25:35,384][123582] Updated weights for policy 0, policy_version 21833 (0.0008) [2023-10-10 17:25:35,765][123582] Updated weights for policy 0, policy_version 21843 (0.0007) [2023-10-10 17:25:36,139][123582] Updated weights for policy 0, policy_version 21853 (0.0008) [2023-10-10 17:25:37,278][123614] Updated weights for policy 1, policy_version 21800 (0.0008) [2023-10-10 17:25:37,657][123614] Updated weights for policy 1, policy_version 21810 (0.0008) [2023-10-10 17:25:38,021][123614] Updated weights for policy 1, policy_version 21820 (0.0008) [2023-10-10 17:25:38,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 44728320. Throughput: 0: 1813.2, 1: 1830.4. Samples: 11192122. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:25:38,788][122664] Avg episode reward: [(0, '33.910'), (1, '28.460')] [2023-10-10 17:25:38,795][123247] Saving new best policy, reward=33.910! [2023-10-10 17:25:39,911][123582] Updated weights for policy 0, policy_version 21863 (0.0009) [2023-10-10 17:25:40,280][123582] Updated weights for policy 0, policy_version 21873 (0.0010) [2023-10-10 17:25:40,657][123582] Updated weights for policy 0, policy_version 21883 (0.0008) [2023-10-10 17:25:41,861][123614] Updated weights for policy 1, policy_version 21830 (0.0009) [2023-10-10 17:25:42,239][123614] Updated weights for policy 1, policy_version 21840 (0.0009) [2023-10-10 17:25:42,618][123614] Updated weights for policy 1, policy_version 21850 (0.0008) [2023-10-10 17:25:43,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 44793856. Throughput: 0: 1808.3, 1: 1820.9. Samples: 11202864. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:25:43,789][122664] Avg episode reward: [(0, '32.620'), (1, '28.430')] [2023-10-10 17:25:44,357][123582] Updated weights for policy 0, policy_version 21893 (0.0009) [2023-10-10 17:25:44,733][123582] Updated weights for policy 0, policy_version 21903 (0.0009) [2023-10-10 17:25:45,117][123582] Updated weights for policy 0, policy_version 21913 (0.0008) [2023-10-10 17:25:46,291][123614] Updated weights for policy 1, policy_version 21860 (0.0011) [2023-10-10 17:25:46,657][123614] Updated weights for policy 1, policy_version 21870 (0.0011) [2023-10-10 17:25:47,034][123614] Updated weights for policy 1, policy_version 21880 (0.0011) [2023-10-10 17:25:48,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 44859392. Throughput: 0: 1812.8, 1: 1821.5. Samples: 11224112. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 17:25:48,789][122664] Avg episode reward: [(0, '32.220'), (1, '28.340')] [2023-10-10 17:25:48,819][123582] Updated weights for policy 0, policy_version 21923 (0.0008) [2023-10-10 17:25:49,197][123582] Updated weights for policy 0, policy_version 21933 (0.0007) [2023-10-10 17:25:49,572][123582] Updated weights for policy 0, policy_version 21943 (0.0010) [2023-10-10 17:25:50,731][123614] Updated weights for policy 1, policy_version 21890 (0.0007) [2023-10-10 17:25:51,099][123614] Updated weights for policy 1, policy_version 21900 (0.0009) [2023-10-10 17:25:51,468][123614] Updated weights for policy 1, policy_version 21910 (0.0010) [2023-10-10 17:25:51,833][123614] Updated weights for policy 1, policy_version 21920 (0.0007) [2023-10-10 17:25:53,183][123582] Updated weights for policy 0, policy_version 21953 (0.0009) [2023-10-10 17:25:53,558][123582] Updated weights for policy 0, policy_version 21963 (0.0009) [2023-10-10 17:25:53,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 44924928. Throughput: 0: 1824.4, 1: 1820.2. Samples: 11246760. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 17:25:53,788][122664] Avg episode reward: [(0, '30.340'), (1, '28.620')] [2023-10-10 17:25:53,796][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000021920_22446080.pth... [2023-10-10 17:25:53,837][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000020224_20709376.pth [2023-10-10 17:25:53,933][123582] Updated weights for policy 0, policy_version 21973 (0.0007) [2023-10-10 17:25:54,305][123582] Updated weights for policy 0, policy_version 21983 (0.0007) [2023-10-10 17:25:54,334][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000021984_22511616.pth... [2023-10-10 17:25:54,364][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000020256_20742144.pth [2023-10-10 17:25:55,532][123614] Updated weights for policy 1, policy_version 21930 (0.0008) [2023-10-10 17:25:55,907][123614] Updated weights for policy 1, policy_version 21940 (0.0007) [2023-10-10 17:25:56,277][123614] Updated weights for policy 1, policy_version 21950 (0.0009) [2023-10-10 17:25:57,930][123582] Updated weights for policy 0, policy_version 21993 (0.0008) [2023-10-10 17:25:58,298][123582] Updated weights for policy 0, policy_version 22003 (0.0008) [2023-10-10 17:25:58,662][123582] Updated weights for policy 0, policy_version 22013 (0.0008) [2023-10-10 17:25:58,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 45023232. Throughput: 0: 1822.7, 1: 1818.6. Samples: 11257102. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 17:25:58,789][122664] Avg episode reward: [(0, '30.140'), (1, '30.210')] [2023-10-10 17:25:59,986][123614] Updated weights for policy 1, policy_version 21960 (0.0007) [2023-10-10 17:26:00,355][123614] Updated weights for policy 1, policy_version 21970 (0.0010) [2023-10-10 17:26:00,724][123614] Updated weights for policy 1, policy_version 21980 (0.0008) [2023-10-10 17:26:02,401][123582] Updated weights for policy 0, policy_version 22023 (0.0009) [2023-10-10 17:26:02,772][123582] Updated weights for policy 0, policy_version 22033 (0.0009) [2023-10-10 17:26:03,151][123582] Updated weights for policy 0, policy_version 22043 (0.0009) [2023-10-10 17:26:03,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 45088768. Throughput: 0: 1823.8, 1: 1816.0. Samples: 11279600. Policy #0 lag: (min: 21.0, avg: 22.9, max: 52.0) [2023-10-10 17:26:03,788][122664] Avg episode reward: [(0, '29.700'), (1, '28.530')] [2023-10-10 17:26:04,288][123614] Updated weights for policy 1, policy_version 21990 (0.0008) [2023-10-10 17:26:04,652][123614] Updated weights for policy 1, policy_version 22000 (0.0010) [2023-10-10 17:26:05,026][123614] Updated weights for policy 1, policy_version 22010 (0.0009) [2023-10-10 17:26:06,685][123582] Updated weights for policy 0, policy_version 22053 (0.0009) [2023-10-10 17:26:07,049][123582] Updated weights for policy 0, policy_version 22063 (0.0010) [2023-10-10 17:26:07,424][123582] Updated weights for policy 0, policy_version 22073 (0.0010) [2023-10-10 17:26:08,743][123614] Updated weights for policy 1, policy_version 22020 (0.0008) [2023-10-10 17:26:08,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 45154304. Throughput: 0: 1826.9, 1: 1814.3. Samples: 11301296. Policy #0 lag: (min: 21.0, avg: 22.9, max: 52.0) [2023-10-10 17:26:08,788][122664] Avg episode reward: [(0, '25.580'), (1, '28.500')] [2023-10-10 17:26:09,112][123614] Updated weights for policy 1, policy_version 22030 (0.0010) [2023-10-10 17:26:09,487][123614] Updated weights for policy 1, policy_version 22040 (0.0009) [2023-10-10 17:26:11,171][123582] Updated weights for policy 0, policy_version 22083 (0.0010) [2023-10-10 17:26:11,538][123582] Updated weights for policy 0, policy_version 22093 (0.0008) [2023-10-10 17:26:11,915][123582] Updated weights for policy 0, policy_version 22103 (0.0008) [2023-10-10 17:26:13,344][123614] Updated weights for policy 1, policy_version 22050 (0.0008) [2023-10-10 17:26:13,716][123614] Updated weights for policy 1, policy_version 22060 (0.0008) [2023-10-10 17:26:13,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 45219840. Throughput: 0: 1824.4, 1: 1809.0. Samples: 11312338. Policy #0 lag: (min: 21.0, avg: 22.9, max: 52.0) [2023-10-10 17:26:13,789][122664] Avg episode reward: [(0, '24.110'), (1, '31.110')] [2023-10-10 17:26:14,085][123614] Updated weights for policy 1, policy_version 22070 (0.0009) [2023-10-10 17:26:14,459][123614] Updated weights for policy 1, policy_version 22080 (0.0008) [2023-10-10 17:26:15,484][123582] Updated weights for policy 0, policy_version 22113 (0.0009) [2023-10-10 17:26:15,864][123582] Updated weights for policy 0, policy_version 22123 (0.0008) [2023-10-10 17:26:16,229][123582] Updated weights for policy 0, policy_version 22133 (0.0008) [2023-10-10 17:26:16,604][123582] Updated weights for policy 0, policy_version 22143 (0.0007) [2023-10-10 17:26:18,147][123614] Updated weights for policy 1, policy_version 22090 (0.0008) [2023-10-10 17:26:18,519][123614] Updated weights for policy 1, policy_version 22100 (0.0008) [2023-10-10 17:26:18,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 45285376. Throughput: 0: 1835.0, 1: 1813.6. Samples: 11334272. Policy #0 lag: (min: 21.0, avg: 22.9, max: 52.0) [2023-10-10 17:26:18,789][122664] Avg episode reward: [(0, '26.820'), (1, '33.380')] [2023-10-10 17:26:18,887][123614] Updated weights for policy 1, policy_version 22110 (0.0008) [2023-10-10 17:26:18,957][123465] Saving new best policy, reward=33.380! [2023-10-10 17:26:20,301][123582] Updated weights for policy 0, policy_version 22153 (0.0009) [2023-10-10 17:26:20,675][123582] Updated weights for policy 0, policy_version 22163 (0.0008) [2023-10-10 17:26:21,051][123582] Updated weights for policy 0, policy_version 22173 (0.0009) [2023-10-10 17:26:22,526][123614] Updated weights for policy 1, policy_version 22120 (0.0007) [2023-10-10 17:26:22,900][123614] Updated weights for policy 1, policy_version 22130 (0.0009) [2023-10-10 17:26:23,274][123614] Updated weights for policy 1, policy_version 22140 (0.0008) [2023-10-10 17:26:23,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 45383680. Throughput: 0: 1822.5, 1: 1811.4. Samples: 11355648. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:26:23,789][122664] Avg episode reward: [(0, '26.160'), (1, '30.810')] [2023-10-10 17:26:24,737][123582] Updated weights for policy 0, policy_version 22183 (0.0009) [2023-10-10 17:26:25,118][123582] Updated weights for policy 0, policy_version 22193 (0.0010) [2023-10-10 17:26:25,495][123582] Updated weights for policy 0, policy_version 22203 (0.0009) [2023-10-10 17:26:27,068][123614] Updated weights for policy 1, policy_version 22150 (0.0010) [2023-10-10 17:26:27,449][123614] Updated weights for policy 1, policy_version 22160 (0.0008) [2023-10-10 17:26:27,825][123614] Updated weights for policy 1, policy_version 22170 (0.0009) [2023-10-10 17:26:28,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 45449216. Throughput: 0: 1826.3, 1: 1819.9. Samples: 11366942. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:26:28,789][122664] Avg episode reward: [(0, '26.570'), (1, '31.490')] [2023-10-10 17:26:29,104][123582] Updated weights for policy 0, policy_version 22213 (0.0008) [2023-10-10 17:26:29,475][123582] Updated weights for policy 0, policy_version 22223 (0.0008) [2023-10-10 17:26:29,856][123582] Updated weights for policy 0, policy_version 22233 (0.0010) [2023-10-10 17:26:31,478][123614] Updated weights for policy 1, policy_version 22180 (0.0008) [2023-10-10 17:26:31,851][123614] Updated weights for policy 1, policy_version 22190 (0.0007) [2023-10-10 17:26:32,219][123614] Updated weights for policy 1, policy_version 22200 (0.0008) [2023-10-10 17:26:33,439][123582] Updated weights for policy 0, policy_version 22243 (0.0008) [2023-10-10 17:26:33,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 45514752. Throughput: 0: 1829.5, 1: 1816.8. Samples: 11388192. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:26:33,789][122664] Avg episode reward: [(0, '27.850'), (1, '32.040')] [2023-10-10 17:26:33,807][123582] Updated weights for policy 0, policy_version 22253 (0.0010) [2023-10-10 17:26:34,176][123582] Updated weights for policy 0, policy_version 22263 (0.0007) [2023-10-10 17:26:35,865][123614] Updated weights for policy 1, policy_version 22210 (0.0008) [2023-10-10 17:26:36,229][123614] Updated weights for policy 1, policy_version 22220 (0.0008) [2023-10-10 17:26:36,593][123614] Updated weights for policy 1, policy_version 22230 (0.0009) [2023-10-10 17:26:36,968][123614] Updated weights for policy 1, policy_version 22240 (0.0008) [2023-10-10 17:26:37,913][123582] Updated weights for policy 0, policy_version 22273 (0.0010) [2023-10-10 17:26:38,279][123582] Updated weights for policy 0, policy_version 22283 (0.0009) [2023-10-10 17:26:38,655][123582] Updated weights for policy 0, policy_version 22293 (0.0008) [2023-10-10 17:26:38,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 45580288. Throughput: 0: 1820.6, 1: 1821.3. Samples: 11410644. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:26:38,788][122664] Avg episode reward: [(0, '28.090'), (1, '30.650')] [2023-10-10 17:26:39,020][123582] Updated weights for policy 0, policy_version 22303 (0.0008) [2023-10-10 17:26:40,650][123614] Updated weights for policy 1, policy_version 22250 (0.0008) [2023-10-10 17:26:41,017][123614] Updated weights for policy 1, policy_version 22260 (0.0008) [2023-10-10 17:26:41,388][123614] Updated weights for policy 1, policy_version 22270 (0.0007) [2023-10-10 17:26:42,622][123582] Updated weights for policy 0, policy_version 22313 (0.0010) [2023-10-10 17:26:42,990][123582] Updated weights for policy 0, policy_version 22323 (0.0009) [2023-10-10 17:26:43,366][123582] Updated weights for policy 0, policy_version 22333 (0.0009) [2023-10-10 17:26:43,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 45678592. Throughput: 0: 1826.4, 1: 1822.0. Samples: 11421276. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-10 17:26:43,789][122664] Avg episode reward: [(0, '29.180'), (1, '31.170')] [2023-10-10 17:26:45,017][123614] Updated weights for policy 1, policy_version 22280 (0.0007) [2023-10-10 17:26:45,382][123614] Updated weights for policy 1, policy_version 22290 (0.0008) [2023-10-10 17:26:45,744][123614] Updated weights for policy 1, policy_version 22300 (0.0010) [2023-10-10 17:26:46,983][123582] Updated weights for policy 0, policy_version 22343 (0.0009) [2023-10-10 17:26:47,354][123582] Updated weights for policy 0, policy_version 22353 (0.0011) [2023-10-10 17:26:47,726][123582] Updated weights for policy 0, policy_version 22363 (0.0010) [2023-10-10 17:26:48,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 45744128. Throughput: 0: 1818.1, 1: 1823.0. Samples: 11443450. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-10 17:26:48,788][122664] Avg episode reward: [(0, '30.360'), (1, '28.510')] [2023-10-10 17:26:49,328][123614] Updated weights for policy 1, policy_version 22310 (0.0008) [2023-10-10 17:26:49,702][123614] Updated weights for policy 1, policy_version 22320 (0.0008) [2023-10-10 17:26:50,073][123614] Updated weights for policy 1, policy_version 22330 (0.0008) [2023-10-10 17:26:51,520][123582] Updated weights for policy 0, policy_version 22373 (0.0009) [2023-10-10 17:26:51,886][123582] Updated weights for policy 0, policy_version 22383 (0.0009) [2023-10-10 17:26:52,264][123582] Updated weights for policy 0, policy_version 22393 (0.0007) [2023-10-10 17:26:53,772][123614] Updated weights for policy 1, policy_version 22340 (0.0009) [2023-10-10 17:26:53,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 45809664. Throughput: 0: 1825.3, 1: 1826.5. Samples: 11465630. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-10 17:26:53,789][122664] Avg episode reward: [(0, '31.540'), (1, '27.660')] [2023-10-10 17:26:54,142][123614] Updated weights for policy 1, policy_version 22350 (0.0009) [2023-10-10 17:26:54,520][123614] Updated weights for policy 1, policy_version 22360 (0.0009) [2023-10-10 17:26:55,986][123582] Updated weights for policy 0, policy_version 22403 (0.0007) [2023-10-10 17:26:56,366][123582] Updated weights for policy 0, policy_version 22413 (0.0010) [2023-10-10 17:26:56,736][123582] Updated weights for policy 0, policy_version 22423 (0.0009) [2023-10-10 17:26:58,169][123614] Updated weights for policy 1, policy_version 22370 (0.0008) [2023-10-10 17:26:58,532][123614] Updated weights for policy 1, policy_version 22380 (0.0011) [2023-10-10 17:26:58,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 45875200. Throughput: 0: 1819.2, 1: 1830.8. Samples: 11476590. Policy #0 lag: (min: 23.0, avg: 25.9, max: 55.0) [2023-10-10 17:26:58,788][122664] Avg episode reward: [(0, '30.830'), (1, '29.660')] [2023-10-10 17:26:58,894][123614] Updated weights for policy 1, policy_version 22390 (0.0008) [2023-10-10 17:26:59,264][123614] Updated weights for policy 1, policy_version 22400 (0.0008) [2023-10-10 17:27:00,457][123582] Updated weights for policy 0, policy_version 22433 (0.0011) [2023-10-10 17:27:00,827][123582] Updated weights for policy 0, policy_version 22443 (0.0007) [2023-10-10 17:27:01,194][123582] Updated weights for policy 0, policy_version 22453 (0.0007) [2023-10-10 17:27:01,576][123582] Updated weights for policy 0, policy_version 22463 (0.0007) [2023-10-10 17:27:02,996][123614] Updated weights for policy 1, policy_version 22410 (0.0008) [2023-10-10 17:27:03,363][123614] Updated weights for policy 1, policy_version 22420 (0.0009) [2023-10-10 17:27:03,729][123614] Updated weights for policy 1, policy_version 22430 (0.0009) [2023-10-10 17:27:03,788][122664] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 45940736. Throughput: 0: 1819.4, 1: 1825.5. Samples: 11498292. Policy #0 lag: (min: 23.0, avg: 25.9, max: 55.0) [2023-10-10 17:27:03,788][122664] Avg episode reward: [(0, '32.990'), (1, '28.440')] [2023-10-10 17:27:05,171][123582] Updated weights for policy 0, policy_version 22473 (0.0008) [2023-10-10 17:27:05,544][123582] Updated weights for policy 0, policy_version 22483 (0.0007) [2023-10-10 17:27:05,917][123582] Updated weights for policy 0, policy_version 22493 (0.0007) [2023-10-10 17:27:07,443][123614] Updated weights for policy 1, policy_version 22440 (0.0010) [2023-10-10 17:27:07,817][123614] Updated weights for policy 1, policy_version 22450 (0.0009) [2023-10-10 17:27:08,175][123614] Updated weights for policy 1, policy_version 22460 (0.0007) [2023-10-10 17:27:08,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 46039040. Throughput: 0: 1825.7, 1: 1828.3. Samples: 11520074. Policy #0 lag: (min: 23.0, avg: 25.9, max: 55.0) [2023-10-10 17:27:08,788][122664] Avg episode reward: [(0, '33.940'), (1, '28.460')] [2023-10-10 17:27:08,799][123247] Saving new best policy, reward=33.940! [2023-10-10 17:27:09,627][123582] Updated weights for policy 0, policy_version 22503 (0.0010) [2023-10-10 17:27:10,007][123582] Updated weights for policy 0, policy_version 22513 (0.0010) [2023-10-10 17:27:10,386][123582] Updated weights for policy 0, policy_version 22523 (0.0008) [2023-10-10 17:27:11,999][123614] Updated weights for policy 1, policy_version 22470 (0.0007) [2023-10-10 17:27:12,378][123614] Updated weights for policy 1, policy_version 22480 (0.0009) [2023-10-10 17:27:12,752][123614] Updated weights for policy 1, policy_version 22490 (0.0011) [2023-10-10 17:27:13,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 46104576. Throughput: 0: 1825.0, 1: 1827.2. Samples: 11531294. Policy #0 lag: (min: 23.0, avg: 25.9, max: 55.0) [2023-10-10 17:27:13,789][122664] Avg episode reward: [(0, '34.070'), (1, '28.390')] [2023-10-10 17:27:14,048][123582] Updated weights for policy 0, policy_version 22533 (0.0010) [2023-10-10 17:27:14,419][123582] Updated weights for policy 0, policy_version 22543 (0.0012) [2023-10-10 17:27:14,788][123582] Updated weights for policy 0, policy_version 22553 (0.0011) [2023-10-10 17:27:15,054][123247] Saving new best policy, reward=34.070! [2023-10-10 17:27:16,428][123614] Updated weights for policy 1, policy_version 22500 (0.0009) [2023-10-10 17:27:16,793][123614] Updated weights for policy 1, policy_version 22510 (0.0008) [2023-10-10 17:27:17,157][123614] Updated weights for policy 1, policy_version 22520 (0.0009) [2023-10-10 17:27:18,609][123582] Updated weights for policy 0, policy_version 22563 (0.0009) [2023-10-10 17:27:18,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 46170112. Throughput: 0: 1817.2, 1: 1826.3. Samples: 11552148. Policy #0 lag: (min: 31.0, avg: 31.2, max: 39.0) [2023-10-10 17:27:18,789][122664] Avg episode reward: [(0, '33.510'), (1, '27.340')] [2023-10-10 17:27:18,985][123582] Updated weights for policy 0, policy_version 22573 (0.0008) [2023-10-10 17:27:19,357][123582] Updated weights for policy 0, policy_version 22583 (0.0010) [2023-10-10 17:27:20,955][123614] Updated weights for policy 1, policy_version 22530 (0.0010) [2023-10-10 17:27:21,323][123614] Updated weights for policy 1, policy_version 22540 (0.0009) [2023-10-10 17:27:21,703][123614] Updated weights for policy 1, policy_version 22550 (0.0010) [2023-10-10 17:27:22,076][123614] Updated weights for policy 1, policy_version 22560 (0.0007) [2023-10-10 17:27:22,849][123582] Updated weights for policy 0, policy_version 22593 (0.0010) [2023-10-10 17:27:23,226][123582] Updated weights for policy 0, policy_version 22603 (0.0008) [2023-10-10 17:27:23,607][123582] Updated weights for policy 0, policy_version 22613 (0.0007) [2023-10-10 17:27:23,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 46235648. Throughput: 0: 1818.3, 1: 1820.8. Samples: 11574402. Policy #0 lag: (min: 31.0, avg: 31.2, max: 39.0) [2023-10-10 17:27:23,789][122664] Avg episode reward: [(0, '32.770'), (1, '28.360')] [2023-10-10 17:27:23,987][123582] Updated weights for policy 0, policy_version 22623 (0.0009) [2023-10-10 17:27:25,622][123614] Updated weights for policy 1, policy_version 22570 (0.0008) [2023-10-10 17:27:25,997][123614] Updated weights for policy 1, policy_version 22580 (0.0008) [2023-10-10 17:27:26,382][123614] Updated weights for policy 1, policy_version 22590 (0.0007) [2023-10-10 17:27:27,791][123582] Updated weights for policy 0, policy_version 22633 (0.0009) [2023-10-10 17:27:28,168][123582] Updated weights for policy 0, policy_version 22643 (0.0008) [2023-10-10 17:27:28,546][123582] Updated weights for policy 0, policy_version 22653 (0.0009) [2023-10-10 17:27:28,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 46333952. Throughput: 0: 1816.2, 1: 1819.3. Samples: 11584874. Policy #0 lag: (min: 31.0, avg: 31.2, max: 39.0) [2023-10-10 17:27:28,789][122664] Avg episode reward: [(0, '32.240'), (1, '29.200')] [2023-10-10 17:27:29,983][123614] Updated weights for policy 1, policy_version 22600 (0.0009) [2023-10-10 17:27:30,350][123614] Updated weights for policy 1, policy_version 22610 (0.0009) [2023-10-10 17:27:30,728][123614] Updated weights for policy 1, policy_version 22620 (0.0008) [2023-10-10 17:27:32,283][123582] Updated weights for policy 0, policy_version 22663 (0.0009) [2023-10-10 17:27:32,649][123582] Updated weights for policy 0, policy_version 22673 (0.0008) [2023-10-10 17:27:33,013][123582] Updated weights for policy 0, policy_version 22683 (0.0008) [2023-10-10 17:27:33,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 46399488. Throughput: 0: 1820.9, 1: 1816.1. Samples: 11607118. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-10 17:27:33,789][122664] Avg episode reward: [(0, '32.200'), (1, '29.500')] [2023-10-10 17:27:34,449][123614] Updated weights for policy 1, policy_version 22630 (0.0008) [2023-10-10 17:27:34,812][123614] Updated weights for policy 1, policy_version 22640 (0.0008) [2023-10-10 17:27:35,186][123614] Updated weights for policy 1, policy_version 22650 (0.0007) [2023-10-10 17:27:36,674][123582] Updated weights for policy 0, policy_version 22693 (0.0007) [2023-10-10 17:27:37,048][123582] Updated weights for policy 0, policy_version 22703 (0.0009) [2023-10-10 17:27:37,412][123582] Updated weights for policy 0, policy_version 22713 (0.0009) [2023-10-10 17:27:38,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 46465024. Throughput: 0: 1814.1, 1: 1817.3. Samples: 11629044. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-10 17:27:38,790][122664] Avg episode reward: [(0, '30.490'), (1, '29.570')] [2023-10-10 17:27:38,860][123614] Updated weights for policy 1, policy_version 22660 (0.0008) [2023-10-10 17:27:39,233][123614] Updated weights for policy 1, policy_version 22670 (0.0008) [2023-10-10 17:27:39,603][123614] Updated weights for policy 1, policy_version 22680 (0.0008) [2023-10-10 17:27:41,083][123582] Updated weights for policy 0, policy_version 22723 (0.0008) [2023-10-10 17:27:41,454][123582] Updated weights for policy 0, policy_version 22733 (0.0008) [2023-10-10 17:27:41,836][123582] Updated weights for policy 0, policy_version 22743 (0.0008) [2023-10-10 17:27:43,295][123614] Updated weights for policy 1, policy_version 22690 (0.0009) [2023-10-10 17:27:43,656][123614] Updated weights for policy 1, policy_version 22700 (0.0012) [2023-10-10 17:27:43,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 46530560. Throughput: 0: 1818.9, 1: 1814.9. Samples: 11640112. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-10 17:27:43,788][122664] Avg episode reward: [(0, '30.470'), (1, '29.440')] [2023-10-10 17:27:44,031][123614] Updated weights for policy 1, policy_version 22710 (0.0008) [2023-10-10 17:27:44,393][123614] Updated weights for policy 1, policy_version 22720 (0.0009) [2023-10-10 17:27:45,495][123582] Updated weights for policy 0, policy_version 22753 (0.0008) [2023-10-10 17:27:45,866][123582] Updated weights for policy 0, policy_version 22763 (0.0011) [2023-10-10 17:27:46,236][123582] Updated weights for policy 0, policy_version 22773 (0.0010) [2023-10-10 17:27:46,619][123582] Updated weights for policy 0, policy_version 22783 (0.0010) [2023-10-10 17:27:48,002][123614] Updated weights for policy 1, policy_version 22730 (0.0008) [2023-10-10 17:27:48,374][123614] Updated weights for policy 1, policy_version 22740 (0.0009) [2023-10-10 17:27:48,738][123614] Updated weights for policy 1, policy_version 22750 (0.0009) [2023-10-10 17:27:48,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 46596096. Throughput: 0: 1815.7, 1: 1817.3. Samples: 11661780. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-10 17:27:48,789][122664] Avg episode reward: [(0, '31.840'), (1, '29.150')] [2023-10-10 17:27:50,303][123582] Updated weights for policy 0, policy_version 22793 (0.0010) [2023-10-10 17:27:50,672][123582] Updated weights for policy 0, policy_version 22803 (0.0011) [2023-10-10 17:27:51,037][123582] Updated weights for policy 0, policy_version 22813 (0.0010) [2023-10-10 17:27:52,431][123614] Updated weights for policy 1, policy_version 22760 (0.0008) [2023-10-10 17:27:52,793][123614] Updated weights for policy 1, policy_version 22770 (0.0010) [2023-10-10 17:27:53,163][123614] Updated weights for policy 1, policy_version 22780 (0.0010) [2023-10-10 17:27:53,788][122664] Fps is (10 sec: 16383.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 46694400. Throughput: 0: 1811.7, 1: 1811.0. Samples: 11683098. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:27:53,789][122664] Avg episode reward: [(0, '34.630'), (1, '32.020')] [2023-10-10 17:27:53,799][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000022784_23330816.pth... [2023-10-10 17:27:53,799][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000022816_23363584.pth... [2023-10-10 17:27:53,832][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000021088_21594112.pth [2023-10-10 17:27:53,844][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000021120_21626880.pth [2023-10-10 17:27:53,849][123247] Saving new best policy, reward=34.630! [2023-10-10 17:27:54,881][123582] Updated weights for policy 0, policy_version 22823 (0.0009) [2023-10-10 17:27:55,262][123582] Updated weights for policy 0, policy_version 22833 (0.0010) [2023-10-10 17:27:55,640][123582] Updated weights for policy 0, policy_version 22843 (0.0011) [2023-10-10 17:27:56,951][123614] Updated weights for policy 1, policy_version 22790 (0.0009) [2023-10-10 17:27:57,339][123614] Updated weights for policy 1, policy_version 22800 (0.0007) [2023-10-10 17:27:57,722][123614] Updated weights for policy 1, policy_version 22810 (0.0009) [2023-10-10 17:27:58,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 46759936. Throughput: 0: 1808.7, 1: 1812.1. Samples: 11694230. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:27:58,789][122664] Avg episode reward: [(0, '33.010'), (1, '32.140')] [2023-10-10 17:27:59,334][123582] Updated weights for policy 0, policy_version 22853 (0.0010) [2023-10-10 17:27:59,699][123582] Updated weights for policy 0, policy_version 22863 (0.0009) [2023-10-10 17:28:00,077][123582] Updated weights for policy 0, policy_version 22873 (0.0010) [2023-10-10 17:28:01,459][123614] Updated weights for policy 1, policy_version 22820 (0.0009) [2023-10-10 17:28:01,838][123614] Updated weights for policy 1, policy_version 22830 (0.0008) [2023-10-10 17:28:02,201][123614] Updated weights for policy 1, policy_version 22840 (0.0009) [2023-10-10 17:28:03,757][123582] Updated weights for policy 0, policy_version 22883 (0.0008) [2023-10-10 17:28:03,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 46825472. Throughput: 0: 1815.7, 1: 1814.9. Samples: 11715526. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:28:03,789][122664] Avg episode reward: [(0, '34.920'), (1, '33.190')] [2023-10-10 17:28:04,137][123582] Updated weights for policy 0, policy_version 22893 (0.0012) [2023-10-10 17:28:04,521][123582] Updated weights for policy 0, policy_version 22903 (0.0010) [2023-10-10 17:28:04,850][123247] Saving new best policy, reward=34.920! [2023-10-10 17:28:05,817][123614] Updated weights for policy 1, policy_version 22850 (0.0010) [2023-10-10 17:28:06,186][123614] Updated weights for policy 1, policy_version 22860 (0.0007) [2023-10-10 17:28:06,567][123614] Updated weights for policy 1, policy_version 22870 (0.0007) [2023-10-10 17:28:06,929][123614] Updated weights for policy 1, policy_version 22880 (0.0008) [2023-10-10 17:28:08,185][123582] Updated weights for policy 0, policy_version 22913 (0.0010) [2023-10-10 17:28:08,548][123582] Updated weights for policy 0, policy_version 22923 (0.0007) [2023-10-10 17:28:08,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 46891008. Throughput: 0: 1817.2, 1: 1812.4. Samples: 11737730. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:28:08,789][122664] Avg episode reward: [(0, '35.430'), (1, '33.720')] [2023-10-10 17:28:08,800][123465] Saving new best policy, reward=33.720! [2023-10-10 17:28:08,930][123582] Updated weights for policy 0, policy_version 22933 (0.0009) [2023-10-10 17:28:09,308][123582] Updated weights for policy 0, policy_version 22943 (0.0008) [2023-10-10 17:28:09,340][123247] Saving new best policy, reward=35.430! [2023-10-10 17:28:10,730][123614] Updated weights for policy 1, policy_version 22890 (0.0007) [2023-10-10 17:28:11,103][123614] Updated weights for policy 1, policy_version 22900 (0.0007) [2023-10-10 17:28:11,472][123614] Updated weights for policy 1, policy_version 22910 (0.0007) [2023-10-10 17:28:12,996][123582] Updated weights for policy 0, policy_version 22953 (0.0009) [2023-10-10 17:28:13,364][123582] Updated weights for policy 0, policy_version 22963 (0.0008) [2023-10-10 17:28:13,742][123582] Updated weights for policy 0, policy_version 22973 (0.0009) [2023-10-10 17:28:13,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 46956544. Throughput: 0: 1813.8, 1: 1813.2. Samples: 11748088. Policy #0 lag: (min: 13.0, avg: 16.1, max: 45.0) [2023-10-10 17:28:13,788][122664] Avg episode reward: [(0, '34.090'), (1, '32.630')] [2023-10-10 17:28:15,149][123614] Updated weights for policy 1, policy_version 22920 (0.0007) [2023-10-10 17:28:15,515][123614] Updated weights for policy 1, policy_version 22930 (0.0007) [2023-10-10 17:28:15,895][123614] Updated weights for policy 1, policy_version 22940 (0.0010) [2023-10-10 17:28:17,466][123582] Updated weights for policy 0, policy_version 22983 (0.0008) [2023-10-10 17:28:17,847][123582] Updated weights for policy 0, policy_version 22993 (0.0008) [2023-10-10 17:28:18,214][123582] Updated weights for policy 0, policy_version 23003 (0.0012) [2023-10-10 17:28:18,788][122664] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 47054848. Throughput: 0: 1819.7, 1: 1808.9. Samples: 11770406. Policy #0 lag: (min: 13.0, avg: 16.1, max: 45.0) [2023-10-10 17:28:18,789][122664] Avg episode reward: [(0, '32.420'), (1, '32.670')] [2023-10-10 17:28:19,637][123614] Updated weights for policy 1, policy_version 22950 (0.0012) [2023-10-10 17:28:20,002][123614] Updated weights for policy 1, policy_version 22960 (0.0010) [2023-10-10 17:28:20,366][123614] Updated weights for policy 1, policy_version 22970 (0.0008) [2023-10-10 17:28:21,919][123582] Updated weights for policy 0, policy_version 23013 (0.0009) [2023-10-10 17:28:22,296][123582] Updated weights for policy 0, policy_version 23023 (0.0009) [2023-10-10 17:28:22,657][123582] Updated weights for policy 0, policy_version 23033 (0.0007) [2023-10-10 17:28:23,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 47120384. Throughput: 0: 1813.9, 1: 1803.2. Samples: 11791814. Policy #0 lag: (min: 13.0, avg: 16.1, max: 45.0) [2023-10-10 17:28:23,789][122664] Avg episode reward: [(0, '33.130'), (1, '34.630')] [2023-10-10 17:28:23,800][123465] Saving new best policy, reward=34.630! [2023-10-10 17:28:24,389][123614] Updated weights for policy 1, policy_version 22980 (0.0007) [2023-10-10 17:28:24,758][123614] Updated weights for policy 1, policy_version 22990 (0.0009) [2023-10-10 17:28:25,129][123614] Updated weights for policy 1, policy_version 23000 (0.0009) [2023-10-10 17:28:26,363][123582] Updated weights for policy 0, policy_version 23043 (0.0008) [2023-10-10 17:28:26,746][123582] Updated weights for policy 0, policy_version 23053 (0.0009) [2023-10-10 17:28:27,117][123582] Updated weights for policy 0, policy_version 23063 (0.0010) [2023-10-10 17:28:28,660][123614] Updated weights for policy 1, policy_version 23010 (0.0008) [2023-10-10 17:28:28,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 47185920. Throughput: 0: 1820.0, 1: 1799.8. Samples: 11803006. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:28:28,789][122664] Avg episode reward: [(0, '33.840'), (1, '34.370')] [2023-10-10 17:28:29,014][123614] Updated weights for policy 1, policy_version 23020 (0.0007) [2023-10-10 17:28:29,385][123614] Updated weights for policy 1, policy_version 23030 (0.0008) [2023-10-10 17:28:29,745][123614] Updated weights for policy 1, policy_version 23040 (0.0009) [2023-10-10 17:28:30,803][123582] Updated weights for policy 0, policy_version 23073 (0.0010) [2023-10-10 17:28:31,172][123582] Updated weights for policy 0, policy_version 23083 (0.0008) [2023-10-10 17:28:31,541][123582] Updated weights for policy 0, policy_version 23093 (0.0010) [2023-10-10 17:28:31,911][123582] Updated weights for policy 0, policy_version 23103 (0.0011) [2023-10-10 17:28:33,407][123614] Updated weights for policy 1, policy_version 23050 (0.0008) [2023-10-10 17:28:33,772][123614] Updated weights for policy 1, policy_version 23060 (0.0008) [2023-10-10 17:28:33,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 47251456. Throughput: 0: 1809.0, 1: 1806.9. Samples: 11824498. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:28:33,789][122664] Avg episode reward: [(0, '34.210'), (1, '33.560')] [2023-10-10 17:28:34,143][123614] Updated weights for policy 1, policy_version 23070 (0.0010) [2023-10-10 17:28:35,633][123582] Updated weights for policy 0, policy_version 23113 (0.0008) [2023-10-10 17:28:36,012][123582] Updated weights for policy 0, policy_version 23123 (0.0010) [2023-10-10 17:28:36,389][123582] Updated weights for policy 0, policy_version 23133 (0.0010) [2023-10-10 17:28:37,859][123614] Updated weights for policy 1, policy_version 23080 (0.0010) [2023-10-10 17:28:38,228][123614] Updated weights for policy 1, policy_version 23090 (0.0007) [2023-10-10 17:28:38,599][123614] Updated weights for policy 1, policy_version 23100 (0.0008) [2023-10-10 17:28:38,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 47349760. Throughput: 0: 1807.5, 1: 1808.4. Samples: 11845814. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:28:38,789][122664] Avg episode reward: [(0, '33.420'), (1, '29.870')] [2023-10-10 17:28:40,206][123582] Updated weights for policy 0, policy_version 23143 (0.0007) [2023-10-10 17:28:40,592][123582] Updated weights for policy 0, policy_version 23153 (0.0007) [2023-10-10 17:28:40,963][123582] Updated weights for policy 0, policy_version 23163 (0.0007) [2023-10-10 17:28:42,257][123614] Updated weights for policy 1, policy_version 23110 (0.0009) [2023-10-10 17:28:42,626][123614] Updated weights for policy 1, policy_version 23120 (0.0007) [2023-10-10 17:28:42,992][123614] Updated weights for policy 1, policy_version 23130 (0.0007) [2023-10-10 17:28:43,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 47415296. Throughput: 0: 1811.8, 1: 1809.5. Samples: 11857190. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:28:43,789][122664] Avg episode reward: [(0, '33.590'), (1, '27.390')] [2023-10-10 17:28:44,550][123582] Updated weights for policy 0, policy_version 23173 (0.0007) [2023-10-10 17:28:44,911][123582] Updated weights for policy 0, policy_version 23183 (0.0010) [2023-10-10 17:28:45,297][123582] Updated weights for policy 0, policy_version 23193 (0.0009) [2023-10-10 17:28:46,757][123614] Updated weights for policy 1, policy_version 23140 (0.0008) [2023-10-10 17:28:47,125][123614] Updated weights for policy 1, policy_version 23150 (0.0007) [2023-10-10 17:28:47,481][123614] Updated weights for policy 1, policy_version 23160 (0.0009) [2023-10-10 17:28:48,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 47480832. Throughput: 0: 1812.2, 1: 1814.9. Samples: 11878748. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-10 17:28:48,789][122664] Avg episode reward: [(0, '32.570'), (1, '26.930')] [2023-10-10 17:28:48,958][123582] Updated weights for policy 0, policy_version 23203 (0.0009) [2023-10-10 17:28:49,331][123582] Updated weights for policy 0, policy_version 23213 (0.0009) [2023-10-10 17:28:49,691][123582] Updated weights for policy 0, policy_version 23223 (0.0009) [2023-10-10 17:28:51,366][123614] Updated weights for policy 1, policy_version 23170 (0.0007) [2023-10-10 17:28:51,731][123614] Updated weights for policy 1, policy_version 23180 (0.0009) [2023-10-10 17:28:52,098][123614] Updated weights for policy 1, policy_version 23190 (0.0008) [2023-10-10 17:28:52,475][123614] Updated weights for policy 1, policy_version 23200 (0.0009) [2023-10-10 17:28:53,256][123582] Updated weights for policy 0, policy_version 23233 (0.0008) [2023-10-10 17:28:53,631][123582] Updated weights for policy 0, policy_version 23243 (0.0007) [2023-10-10 17:28:53,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 47546368. Throughput: 0: 1819.3, 1: 1811.2. Samples: 11901104. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-10 17:28:53,789][122664] Avg episode reward: [(0, '30.980'), (1, '27.320')] [2023-10-10 17:28:54,011][123582] Updated weights for policy 0, policy_version 23253 (0.0007) [2023-10-10 17:28:54,373][123582] Updated weights for policy 0, policy_version 23263 (0.0009) [2023-10-10 17:28:56,191][123614] Updated weights for policy 1, policy_version 23210 (0.0008) [2023-10-10 17:28:56,562][123614] Updated weights for policy 1, policy_version 23220 (0.0008) [2023-10-10 17:28:56,933][123614] Updated weights for policy 1, policy_version 23230 (0.0008) [2023-10-10 17:28:58,131][123582] Updated weights for policy 0, policy_version 23273 (0.0010) [2023-10-10 17:28:58,511][123582] Updated weights for policy 0, policy_version 23283 (0.0008) [2023-10-10 17:28:58,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 47611904. Throughput: 0: 1812.7, 1: 1820.4. Samples: 11911580. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-10 17:28:58,789][122664] Avg episode reward: [(0, '34.280'), (1, '27.870')] [2023-10-10 17:28:58,884][123582] Updated weights for policy 0, policy_version 23293 (0.0009) [2023-10-10 17:29:00,586][123614] Updated weights for policy 1, policy_version 23240 (0.0012) [2023-10-10 17:29:00,961][123614] Updated weights for policy 1, policy_version 23250 (0.0009) [2023-10-10 17:29:01,330][123614] Updated weights for policy 1, policy_version 23260 (0.0010) [2023-10-10 17:29:02,574][123582] Updated weights for policy 0, policy_version 23303 (0.0008) [2023-10-10 17:29:02,948][123582] Updated weights for policy 0, policy_version 23313 (0.0008) [2023-10-10 17:29:03,313][123582] Updated weights for policy 0, policy_version 23323 (0.0008) [2023-10-10 17:29:03,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 47710208. Throughput: 0: 1810.3, 1: 1817.1. Samples: 11933644. Policy #0 lag: (min: 6.0, avg: 20.8, max: 38.0) [2023-10-10 17:29:03,789][122664] Avg episode reward: [(0, '36.260'), (1, '27.120')] [2023-10-10 17:29:03,790][123247] Saving new best policy, reward=36.260! [2023-10-10 17:29:04,884][123614] Updated weights for policy 1, policy_version 23270 (0.0010) [2023-10-10 17:29:05,255][123614] Updated weights for policy 1, policy_version 23280 (0.0010) [2023-10-10 17:29:05,624][123614] Updated weights for policy 1, policy_version 23290 (0.0009) [2023-10-10 17:29:07,076][123582] Updated weights for policy 0, policy_version 23333 (0.0009) [2023-10-10 17:29:07,439][123582] Updated weights for policy 0, policy_version 23343 (0.0008) [2023-10-10 17:29:07,814][123582] Updated weights for policy 0, policy_version 23353 (0.0008) [2023-10-10 17:29:08,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 47775744. Throughput: 0: 1802.9, 1: 1826.7. Samples: 11955148. Policy #0 lag: (min: 6.0, avg: 20.8, max: 38.0) [2023-10-10 17:29:08,789][122664] Avg episode reward: [(0, '32.420'), (1, '29.120')] [2023-10-10 17:29:09,143][123614] Updated weights for policy 1, policy_version 23300 (0.0008) [2023-10-10 17:29:09,511][123614] Updated weights for policy 1, policy_version 23310 (0.0008) [2023-10-10 17:29:09,876][123614] Updated weights for policy 1, policy_version 23320 (0.0009) [2023-10-10 17:29:11,441][123582] Updated weights for policy 0, policy_version 23363 (0.0009) [2023-10-10 17:29:11,816][123582] Updated weights for policy 0, policy_version 23373 (0.0009) [2023-10-10 17:29:12,178][123582] Updated weights for policy 0, policy_version 23383 (0.0011) [2023-10-10 17:29:13,432][123614] Updated weights for policy 1, policy_version 23330 (0.0009) [2023-10-10 17:29:13,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 47841280. Throughput: 0: 1807.6, 1: 1827.5. Samples: 11966588. Policy #0 lag: (min: 6.0, avg: 20.8, max: 38.0) [2023-10-10 17:29:13,788][122664] Avg episode reward: [(0, '29.760'), (1, '29.540')] [2023-10-10 17:29:13,803][123614] Updated weights for policy 1, policy_version 23340 (0.0008) [2023-10-10 17:29:14,181][123614] Updated weights for policy 1, policy_version 23350 (0.0007) [2023-10-10 17:29:14,543][123614] Updated weights for policy 1, policy_version 23360 (0.0008) [2023-10-10 17:29:15,819][123582] Updated weights for policy 0, policy_version 23393 (0.0011) [2023-10-10 17:29:16,194][123582] Updated weights for policy 0, policy_version 23403 (0.0008) [2023-10-10 17:29:16,565][123582] Updated weights for policy 0, policy_version 23413 (0.0008) [2023-10-10 17:29:16,933][123582] Updated weights for policy 0, policy_version 23423 (0.0009) [2023-10-10 17:29:18,180][123614] Updated weights for policy 1, policy_version 23370 (0.0011) [2023-10-10 17:29:18,539][123614] Updated weights for policy 1, policy_version 23380 (0.0010) [2023-10-10 17:29:18,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 47906816. Throughput: 0: 1812.0, 1: 1829.5. Samples: 11988368. Policy #0 lag: (min: 6.0, avg: 20.8, max: 38.0) [2023-10-10 17:29:18,789][122664] Avg episode reward: [(0, '29.510'), (1, '30.250')] [2023-10-10 17:29:18,905][123614] Updated weights for policy 1, policy_version 23390 (0.0010) [2023-10-10 17:29:20,541][123582] Updated weights for policy 0, policy_version 23433 (0.0007) [2023-10-10 17:29:20,924][123582] Updated weights for policy 0, policy_version 23443 (0.0009) [2023-10-10 17:29:21,294][123582] Updated weights for policy 0, policy_version 23453 (0.0009) [2023-10-10 17:29:22,750][123614] Updated weights for policy 1, policy_version 23400 (0.0007) [2023-10-10 17:29:23,120][123614] Updated weights for policy 1, policy_version 23410 (0.0009) [2023-10-10 17:29:23,488][123614] Updated weights for policy 1, policy_version 23420 (0.0008) [2023-10-10 17:29:23,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 48005120. Throughput: 0: 1820.9, 1: 1823.6. Samples: 12009820. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-10 17:29:23,789][122664] Avg episode reward: [(0, '31.360'), (1, '33.120')] [2023-10-10 17:29:24,871][123582] Updated weights for policy 0, policy_version 23463 (0.0009) [2023-10-10 17:29:25,242][123582] Updated weights for policy 0, policy_version 23473 (0.0007) [2023-10-10 17:29:25,618][123582] Updated weights for policy 0, policy_version 23483 (0.0010) [2023-10-10 17:29:27,164][123614] Updated weights for policy 1, policy_version 23430 (0.0009) [2023-10-10 17:29:27,534][123614] Updated weights for policy 1, policy_version 23440 (0.0009) [2023-10-10 17:29:27,904][123614] Updated weights for policy 1, policy_version 23450 (0.0009) [2023-10-10 17:29:28,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 48070656. Throughput: 0: 1824.2, 1: 1829.4. Samples: 12021604. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-10 17:29:28,789][122664] Avg episode reward: [(0, '32.960'), (1, '32.200')] [2023-10-10 17:29:29,378][123582] Updated weights for policy 0, policy_version 23493 (0.0007) [2023-10-10 17:29:29,752][123582] Updated weights for policy 0, policy_version 23503 (0.0007) [2023-10-10 17:29:30,132][123582] Updated weights for policy 0, policy_version 23513 (0.0007) [2023-10-10 17:29:31,506][123614] Updated weights for policy 1, policy_version 23460 (0.0008) [2023-10-10 17:29:31,880][123614] Updated weights for policy 1, policy_version 23470 (0.0007) [2023-10-10 17:29:32,243][123614] Updated weights for policy 1, policy_version 23480 (0.0008) [2023-10-10 17:29:33,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 48136192. Throughput: 0: 1817.6, 1: 1829.3. Samples: 12042856. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-10 17:29:33,789][122664] Avg episode reward: [(0, '29.440'), (1, '32.790')] [2023-10-10 17:29:33,927][123582] Updated weights for policy 0, policy_version 23523 (0.0010) [2023-10-10 17:29:34,331][123582] Updated weights for policy 0, policy_version 23533 (0.0010) [2023-10-10 17:29:34,698][123582] Updated weights for policy 0, policy_version 23543 (0.0011) [2023-10-10 17:29:35,912][123614] Updated weights for policy 1, policy_version 23490 (0.0008) [2023-10-10 17:29:36,293][123614] Updated weights for policy 1, policy_version 23500 (0.0008) [2023-10-10 17:29:36,660][123614] Updated weights for policy 1, policy_version 23510 (0.0009) [2023-10-10 17:29:37,021][123614] Updated weights for policy 1, policy_version 23520 (0.0008) [2023-10-10 17:29:38,444][123582] Updated weights for policy 0, policy_version 23553 (0.0011) [2023-10-10 17:29:38,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 48201728. Throughput: 0: 1812.0, 1: 1834.9. Samples: 12065216. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-10 17:29:38,789][122664] Avg episode reward: [(0, '28.310'), (1, '33.350')] [2023-10-10 17:29:38,817][123582] Updated weights for policy 0, policy_version 23563 (0.0007) [2023-10-10 17:29:39,183][123582] Updated weights for policy 0, policy_version 23573 (0.0011) [2023-10-10 17:29:39,554][123582] Updated weights for policy 0, policy_version 23583 (0.0010) [2023-10-10 17:29:40,693][123614] Updated weights for policy 1, policy_version 23530 (0.0009) [2023-10-10 17:29:41,065][123614] Updated weights for policy 1, policy_version 23540 (0.0008) [2023-10-10 17:29:41,426][123614] Updated weights for policy 1, policy_version 23550 (0.0009) [2023-10-10 17:29:43,228][123582] Updated weights for policy 0, policy_version 23593 (0.0011) [2023-10-10 17:29:43,598][123582] Updated weights for policy 0, policy_version 23603 (0.0011) [2023-10-10 17:29:43,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 48267264. Throughput: 0: 1810.1, 1: 1824.6. Samples: 12075140. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-10 17:29:43,789][122664] Avg episode reward: [(0, '28.710'), (1, '31.310')] [2023-10-10 17:29:43,975][123582] Updated weights for policy 0, policy_version 23613 (0.0008) [2023-10-10 17:29:45,151][123614] Updated weights for policy 1, policy_version 23560 (0.0009) [2023-10-10 17:29:45,516][123614] Updated weights for policy 1, policy_version 23570 (0.0007) [2023-10-10 17:29:45,890][123614] Updated weights for policy 1, policy_version 23580 (0.0007) [2023-10-10 17:29:47,797][123582] Updated weights for policy 0, policy_version 23623 (0.0010) [2023-10-10 17:29:48,159][123582] Updated weights for policy 0, policy_version 23633 (0.0011) [2023-10-10 17:29:48,537][123582] Updated weights for policy 0, policy_version 23643 (0.0011) [2023-10-10 17:29:48,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 48365568. Throughput: 0: 1817.7, 1: 1835.1. Samples: 12098020. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-10 17:29:48,789][122664] Avg episode reward: [(0, '29.180'), (1, '30.100')] [2023-10-10 17:29:49,538][123614] Updated weights for policy 1, policy_version 23590 (0.0009) [2023-10-10 17:29:49,901][123614] Updated weights for policy 1, policy_version 23600 (0.0010) [2023-10-10 17:29:50,275][123614] Updated weights for policy 1, policy_version 23610 (0.0011) [2023-10-10 17:29:52,128][123582] Updated weights for policy 0, policy_version 23653 (0.0008) [2023-10-10 17:29:52,504][123582] Updated weights for policy 0, policy_version 23663 (0.0010) [2023-10-10 17:29:52,866][123582] Updated weights for policy 0, policy_version 23673 (0.0008) [2023-10-10 17:29:53,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 48431104. Throughput: 0: 1820.8, 1: 1831.0. Samples: 12119478. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-10 17:29:53,788][122664] Avg episode reward: [(0, '35.260'), (1, '31.000')] [2023-10-10 17:29:53,798][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000023680_24248320.pth... [2023-10-10 17:29:53,831][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000021984_22511616.pth [2023-10-10 17:29:53,832][123614] Updated weights for policy 1, policy_version 23620 (0.0008) [2023-10-10 17:29:54,206][123614] Updated weights for policy 1, policy_version 23630 (0.0010) [2023-10-10 17:29:54,575][123614] Updated weights for policy 1, policy_version 23640 (0.0012) [2023-10-10 17:29:54,865][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000023648_24215552.pth... [2023-10-10 17:29:54,899][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000021920_22446080.pth [2023-10-10 17:29:56,683][123582] Updated weights for policy 0, policy_version 23683 (0.0009) [2023-10-10 17:29:57,062][123582] Updated weights for policy 0, policy_version 23693 (0.0007) [2023-10-10 17:29:57,428][123582] Updated weights for policy 0, policy_version 23703 (0.0008) [2023-10-10 17:29:58,239][123614] Updated weights for policy 1, policy_version 23650 (0.0009) [2023-10-10 17:29:58,610][123614] Updated weights for policy 1, policy_version 23660 (0.0008) [2023-10-10 17:29:58,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 48496640. Throughput: 0: 1818.2, 1: 1832.4. Samples: 12130868. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-10 17:29:58,789][122664] Avg episode reward: [(0, '35.070'), (1, '32.030')] [2023-10-10 17:29:58,975][123614] Updated weights for policy 1, policy_version 23670 (0.0009) [2023-10-10 17:29:59,340][123614] Updated weights for policy 1, policy_version 23680 (0.0008) [2023-10-10 17:30:01,229][123582] Updated weights for policy 0, policy_version 23713 (0.0009) [2023-10-10 17:30:01,590][123582] Updated weights for policy 0, policy_version 23723 (0.0008) [2023-10-10 17:30:01,969][123582] Updated weights for policy 0, policy_version 23733 (0.0008) [2023-10-10 17:30:02,343][123582] Updated weights for policy 0, policy_version 23743 (0.0007) [2023-10-10 17:30:03,029][123614] Updated weights for policy 1, policy_version 23690 (0.0010) [2023-10-10 17:30:03,399][123614] Updated weights for policy 1, policy_version 23700 (0.0010) [2023-10-10 17:30:03,772][123614] Updated weights for policy 1, policy_version 23710 (0.0010) [2023-10-10 17:30:03,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 48562176. Throughput: 0: 1809.4, 1: 1826.9. Samples: 12152002. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-10 17:30:03,788][122664] Avg episode reward: [(0, '35.570'), (1, '30.040')] [2023-10-10 17:30:05,950][123582] Updated weights for policy 0, policy_version 23753 (0.0008) [2023-10-10 17:30:06,314][123582] Updated weights for policy 0, policy_version 23763 (0.0007) [2023-10-10 17:30:06,693][123582] Updated weights for policy 0, policy_version 23773 (0.0008) [2023-10-10 17:30:07,504][123614] Updated weights for policy 1, policy_version 23720 (0.0010) [2023-10-10 17:30:07,875][123614] Updated weights for policy 1, policy_version 23730 (0.0008) [2023-10-10 17:30:08,243][123614] Updated weights for policy 1, policy_version 23740 (0.0007) [2023-10-10 17:30:08,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 48660480. Throughput: 0: 1807.3, 1: 1827.1. Samples: 12173366. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-10 17:30:08,789][122664] Avg episode reward: [(0, '35.820'), (1, '29.200')] [2023-10-10 17:30:10,436][123582] Updated weights for policy 0, policy_version 23783 (0.0011) [2023-10-10 17:30:10,818][123582] Updated weights for policy 0, policy_version 23793 (0.0010) [2023-10-10 17:30:11,190][123582] Updated weights for policy 0, policy_version 23803 (0.0008) [2023-10-10 17:30:12,118][123614] Updated weights for policy 1, policy_version 23750 (0.0007) [2023-10-10 17:30:12,494][123614] Updated weights for policy 1, policy_version 23760 (0.0008) [2023-10-10 17:30:12,866][123614] Updated weights for policy 1, policy_version 23770 (0.0010) [2023-10-10 17:30:13,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 48726016. Throughput: 0: 1801.6, 1: 1818.6. Samples: 12184510. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-10 17:30:13,788][122664] Avg episode reward: [(0, '34.580'), (1, '27.620')] [2023-10-10 17:30:14,686][123582] Updated weights for policy 0, policy_version 23813 (0.0010) [2023-10-10 17:30:15,054][123582] Updated weights for policy 0, policy_version 23823 (0.0009) [2023-10-10 17:30:15,434][123582] Updated weights for policy 0, policy_version 23833 (0.0010) [2023-10-10 17:30:16,563][123614] Updated weights for policy 1, policy_version 23780 (0.0009) [2023-10-10 17:30:16,925][123614] Updated weights for policy 1, policy_version 23790 (0.0009) [2023-10-10 17:30:17,288][123614] Updated weights for policy 1, policy_version 23800 (0.0010) [2023-10-10 17:30:18,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 48791552. Throughput: 0: 1814.6, 1: 1805.9. Samples: 12205780. Policy #0 lag: (min: 31.0, avg: 44.5, max: 63.0) [2023-10-10 17:30:18,788][122664] Avg episode reward: [(0, '35.290'), (1, '28.290')] [2023-10-10 17:30:19,170][123582] Updated weights for policy 0, policy_version 23843 (0.0007) [2023-10-10 17:30:19,565][123582] Updated weights for policy 0, policy_version 23853 (0.0007) [2023-10-10 17:30:19,946][123582] Updated weights for policy 0, policy_version 23863 (0.0011) [2023-10-10 17:30:21,017][123614] Updated weights for policy 1, policy_version 23810 (0.0009) [2023-10-10 17:30:21,390][123614] Updated weights for policy 1, policy_version 23820 (0.0007) [2023-10-10 17:30:21,758][123614] Updated weights for policy 1, policy_version 23830 (0.0007) [2023-10-10 17:30:22,126][123614] Updated weights for policy 1, policy_version 23840 (0.0008) [2023-10-10 17:30:23,452][123582] Updated weights for policy 0, policy_version 23873 (0.0010) [2023-10-10 17:30:23,788][122664] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 48857088. Throughput: 0: 1823.6, 1: 1807.0. Samples: 12228594. Policy #0 lag: (min: 31.0, avg: 44.5, max: 63.0) [2023-10-10 17:30:23,789][122664] Avg episode reward: [(0, '38.950'), (1, '26.340')] [2023-10-10 17:30:23,827][123582] Updated weights for policy 0, policy_version 23883 (0.0008) [2023-10-10 17:30:24,206][123582] Updated weights for policy 0, policy_version 23893 (0.0008) [2023-10-10 17:30:24,579][123582] Updated weights for policy 0, policy_version 23903 (0.0009) [2023-10-10 17:30:24,608][123247] Saving new best policy, reward=38.950! [2023-10-10 17:30:25,903][123614] Updated weights for policy 1, policy_version 23850 (0.0008) [2023-10-10 17:30:26,276][123614] Updated weights for policy 1, policy_version 23860 (0.0007) [2023-10-10 17:30:26,641][123614] Updated weights for policy 1, policy_version 23870 (0.0007) [2023-10-10 17:30:28,306][123582] Updated weights for policy 0, policy_version 23913 (0.0008) [2023-10-10 17:30:28,673][123582] Updated weights for policy 0, policy_version 23923 (0.0008) [2023-10-10 17:30:28,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 48922624. Throughput: 0: 1822.4, 1: 1810.0. Samples: 12238596. Policy #0 lag: (min: 31.0, avg: 44.5, max: 63.0) [2023-10-10 17:30:28,788][122664] Avg episode reward: [(0, '37.130'), (1, '26.020')] [2023-10-10 17:30:29,044][123582] Updated weights for policy 0, policy_version 23933 (0.0008) [2023-10-10 17:30:30,327][123614] Updated weights for policy 1, policy_version 23880 (0.0008) [2023-10-10 17:30:30,690][123614] Updated weights for policy 1, policy_version 23890 (0.0007) [2023-10-10 17:30:31,071][123614] Updated weights for policy 1, policy_version 23900 (0.0007) [2023-10-10 17:30:32,650][123582] Updated weights for policy 0, policy_version 23943 (0.0010) [2023-10-10 17:30:33,025][123582] Updated weights for policy 0, policy_version 23953 (0.0008) [2023-10-10 17:30:33,398][123582] Updated weights for policy 0, policy_version 23963 (0.0008) [2023-10-10 17:30:33,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 49020928. Throughput: 0: 1827.3, 1: 1809.0. Samples: 12261656. Policy #0 lag: (min: 30.0, avg: 37.8, max: 62.0) [2023-10-10 17:30:33,789][122664] Avg episode reward: [(0, '35.230'), (1, '25.070')] [2023-10-10 17:30:34,731][123614] Updated weights for policy 1, policy_version 23910 (0.0009) [2023-10-10 17:30:35,106][123614] Updated weights for policy 1, policy_version 23920 (0.0010) [2023-10-10 17:30:35,461][123614] Updated weights for policy 1, policy_version 23930 (0.0009) [2023-10-10 17:30:36,997][123582] Updated weights for policy 0, policy_version 23973 (0.0009) [2023-10-10 17:30:37,371][123582] Updated weights for policy 0, policy_version 23983 (0.0008) [2023-10-10 17:30:37,745][123582] Updated weights for policy 0, policy_version 23993 (0.0007) [2023-10-10 17:30:38,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 49086464. Throughput: 0: 1825.0, 1: 1808.9. Samples: 12283006. Policy #0 lag: (min: 30.0, avg: 37.8, max: 62.0) [2023-10-10 17:30:38,789][122664] Avg episode reward: [(0, '32.750'), (1, '25.260')] [2023-10-10 17:30:39,164][123614] Updated weights for policy 1, policy_version 23940 (0.0010) [2023-10-10 17:30:39,531][123614] Updated weights for policy 1, policy_version 23950 (0.0010) [2023-10-10 17:30:39,897][123614] Updated weights for policy 1, policy_version 23960 (0.0007) [2023-10-10 17:30:41,474][123582] Updated weights for policy 0, policy_version 24003 (0.0008) [2023-10-10 17:30:41,843][123582] Updated weights for policy 0, policy_version 24013 (0.0009) [2023-10-10 17:30:42,206][123582] Updated weights for policy 0, policy_version 24023 (0.0009) [2023-10-10 17:30:43,467][123614] Updated weights for policy 1, policy_version 23970 (0.0007) [2023-10-10 17:30:43,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 49152000. Throughput: 0: 1825.0, 1: 1805.6. Samples: 12294244. Policy #0 lag: (min: 30.0, avg: 37.8, max: 62.0) [2023-10-10 17:30:43,789][122664] Avg episode reward: [(0, '34.010'), (1, '27.040')] [2023-10-10 17:30:43,835][123614] Updated weights for policy 1, policy_version 23980 (0.0008) [2023-10-10 17:30:44,205][123614] Updated weights for policy 1, policy_version 23990 (0.0008) [2023-10-10 17:30:44,570][123614] Updated weights for policy 1, policy_version 24000 (0.0008) [2023-10-10 17:30:45,972][123582] Updated weights for policy 0, policy_version 24033 (0.0009) [2023-10-10 17:30:46,351][123582] Updated weights for policy 0, policy_version 24043 (0.0010) [2023-10-10 17:30:46,718][123582] Updated weights for policy 0, policy_version 24053 (0.0010) [2023-10-10 17:30:47,090][123582] Updated weights for policy 0, policy_version 24063 (0.0009) [2023-10-10 17:30:48,159][123614] Updated weights for policy 1, policy_version 24010 (0.0008) [2023-10-10 17:30:48,516][123614] Updated weights for policy 1, policy_version 24020 (0.0008) [2023-10-10 17:30:48,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 49217536. Throughput: 0: 1830.0, 1: 1814.5. Samples: 12316006. Policy #0 lag: (min: 30.0, avg: 37.8, max: 62.0) [2023-10-10 17:30:48,789][122664] Avg episode reward: [(0, '35.040'), (1, '27.340')] [2023-10-10 17:30:48,883][123614] Updated weights for policy 1, policy_version 24030 (0.0007) [2023-10-10 17:30:50,921][123582] Updated weights for policy 0, policy_version 24073 (0.0010) [2023-10-10 17:30:51,293][123582] Updated weights for policy 0, policy_version 24083 (0.0008) [2023-10-10 17:30:51,663][123582] Updated weights for policy 0, policy_version 24093 (0.0010) [2023-10-10 17:30:52,638][123614] Updated weights for policy 1, policy_version 24040 (0.0008) [2023-10-10 17:30:53,019][123614] Updated weights for policy 1, policy_version 24050 (0.0009) [2023-10-10 17:30:53,394][123614] Updated weights for policy 1, policy_version 24060 (0.0011) [2023-10-10 17:30:53,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 49315840. Throughput: 0: 1822.5, 1: 1816.5. Samples: 12337124. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-10 17:30:53,789][122664] Avg episode reward: [(0, '32.230'), (1, '28.080')] [2023-10-10 17:30:55,227][123582] Updated weights for policy 0, policy_version 24103 (0.0008) [2023-10-10 17:30:55,594][123582] Updated weights for policy 0, policy_version 24113 (0.0008) [2023-10-10 17:30:55,971][123582] Updated weights for policy 0, policy_version 24123 (0.0008) [2023-10-10 17:30:57,265][123614] Updated weights for policy 1, policy_version 24070 (0.0008) [2023-10-10 17:30:57,651][123614] Updated weights for policy 1, policy_version 24080 (0.0009) [2023-10-10 17:30:58,025][123614] Updated weights for policy 1, policy_version 24090 (0.0009) [2023-10-10 17:30:58,788][122664] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 49381376. Throughput: 0: 1826.1, 1: 1817.9. Samples: 12348490. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-10 17:30:58,788][122664] Avg episode reward: [(0, '33.820'), (1, '27.710')] [2023-10-10 17:30:59,686][123582] Updated weights for policy 0, policy_version 24133 (0.0010) [2023-10-10 17:31:00,062][123582] Updated weights for policy 0, policy_version 24143 (0.0008) [2023-10-10 17:31:00,442][123582] Updated weights for policy 0, policy_version 24153 (0.0010) [2023-10-10 17:31:01,457][123614] Updated weights for policy 1, policy_version 24100 (0.0011) [2023-10-10 17:31:01,826][123614] Updated weights for policy 1, policy_version 24110 (0.0007) [2023-10-10 17:31:02,203][123614] Updated weights for policy 1, policy_version 24120 (0.0009) [2023-10-10 17:31:03,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 49446912. Throughput: 0: 1812.1, 1: 1827.2. Samples: 12369548. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-10 17:31:03,789][122664] Avg episode reward: [(0, '32.980'), (1, '29.220')] [2023-10-10 17:31:04,248][123582] Updated weights for policy 0, policy_version 24163 (0.0009) [2023-10-10 17:31:04,635][123582] Updated weights for policy 0, policy_version 24173 (0.0010) [2023-10-10 17:31:05,008][123582] Updated weights for policy 0, policy_version 24183 (0.0009) [2023-10-10 17:31:05,802][123614] Updated weights for policy 1, policy_version 24130 (0.0009) [2023-10-10 17:31:06,171][123614] Updated weights for policy 1, policy_version 24140 (0.0007) [2023-10-10 17:31:06,545][123614] Updated weights for policy 1, policy_version 24150 (0.0010) [2023-10-10 17:31:06,924][123614] Updated weights for policy 1, policy_version 24160 (0.0009) [2023-10-10 17:31:08,678][123582] Updated weights for policy 0, policy_version 24193 (0.0008) [2023-10-10 17:31:08,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 49512448. Throughput: 0: 1812.6, 1: 1824.9. Samples: 12392278. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-10 17:31:08,789][122664] Avg episode reward: [(0, '34.920'), (1, '29.340')] [2023-10-10 17:31:09,047][123582] Updated weights for policy 0, policy_version 24203 (0.0008) [2023-10-10 17:31:09,432][123582] Updated weights for policy 0, policy_version 24213 (0.0009) [2023-10-10 17:31:09,797][123582] Updated weights for policy 0, policy_version 24223 (0.0010) [2023-10-10 17:31:10,823][123614] Updated weights for policy 1, policy_version 24170 (0.0007) [2023-10-10 17:31:11,193][123614] Updated weights for policy 1, policy_version 24180 (0.0007) [2023-10-10 17:31:11,555][123614] Updated weights for policy 1, policy_version 24190 (0.0007) [2023-10-10 17:31:13,645][123582] Updated weights for policy 0, policy_version 24233 (0.0009) [2023-10-10 17:31:13,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 49577984. Throughput: 0: 1811.1, 1: 1822.9. Samples: 12402126. Policy #0 lag: (min: 0.0, avg: 22.4, max: 32.0) [2023-10-10 17:31:13,788][122664] Avg episode reward: [(0, '35.380'), (1, '30.130')] [2023-10-10 17:31:14,020][123582] Updated weights for policy 0, policy_version 24243 (0.0009) [2023-10-10 17:31:14,397][123582] Updated weights for policy 0, policy_version 24253 (0.0009) [2023-10-10 17:31:15,224][123614] Updated weights for policy 1, policy_version 24200 (0.0008) [2023-10-10 17:31:15,590][123614] Updated weights for policy 1, policy_version 24210 (0.0007) [2023-10-10 17:31:15,967][123614] Updated weights for policy 1, policy_version 24220 (0.0008) [2023-10-10 17:31:18,025][123582] Updated weights for policy 0, policy_version 24263 (0.0010) [2023-10-10 17:31:18,399][123582] Updated weights for policy 0, policy_version 24273 (0.0008) [2023-10-10 17:31:18,777][123582] Updated weights for policy 0, policy_version 24283 (0.0009) [2023-10-10 17:31:18,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 49643520. Throughput: 0: 1809.7, 1: 1819.0. Samples: 12424948. Policy #0 lag: (min: 0.0, avg: 22.4, max: 32.0) [2023-10-10 17:31:18,789][122664] Avg episode reward: [(0, '34.890'), (1, '33.600')] [2023-10-10 17:31:19,685][123614] Updated weights for policy 1, policy_version 24230 (0.0009) [2023-10-10 17:31:20,046][123614] Updated weights for policy 1, policy_version 24240 (0.0008) [2023-10-10 17:31:20,409][123614] Updated weights for policy 1, policy_version 24250 (0.0008) [2023-10-10 17:31:22,320][123582] Updated weights for policy 0, policy_version 24293 (0.0008) [2023-10-10 17:31:22,696][123582] Updated weights for policy 0, policy_version 24303 (0.0008) [2023-10-10 17:31:23,065][123582] Updated weights for policy 0, policy_version 24313 (0.0009) [2023-10-10 17:31:23,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 49741824. Throughput: 0: 1810.1, 1: 1816.5. Samples: 12446202. Policy #0 lag: (min: 0.0, avg: 22.4, max: 32.0) [2023-10-10 17:31:23,788][122664] Avg episode reward: [(0, '35.050'), (1, '31.930')] [2023-10-10 17:31:24,280][123614] Updated weights for policy 1, policy_version 24260 (0.0009) [2023-10-10 17:31:24,646][123614] Updated weights for policy 1, policy_version 24270 (0.0010) [2023-10-10 17:31:25,018][123614] Updated weights for policy 1, policy_version 24280 (0.0008) [2023-10-10 17:31:26,894][123582] Updated weights for policy 0, policy_version 24323 (0.0008) [2023-10-10 17:31:27,273][123582] Updated weights for policy 0, policy_version 24333 (0.0008) [2023-10-10 17:31:27,640][123582] Updated weights for policy 0, policy_version 24343 (0.0007) [2023-10-10 17:31:28,659][123614] Updated weights for policy 1, policy_version 24290 (0.0007) [2023-10-10 17:31:28,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 49807360. Throughput: 0: 1806.3, 1: 1819.6. Samples: 12457412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:31:28,788][122664] Avg episode reward: [(0, '31.950'), (1, '32.500')] [2023-10-10 17:31:29,032][123614] Updated weights for policy 1, policy_version 24300 (0.0007) [2023-10-10 17:31:29,394][123614] Updated weights for policy 1, policy_version 24310 (0.0009) [2023-10-10 17:31:29,767][123614] Updated weights for policy 1, policy_version 24320 (0.0009) [2023-10-10 17:31:31,200][123582] Updated weights for policy 0, policy_version 24353 (0.0008) [2023-10-10 17:31:31,573][123582] Updated weights for policy 0, policy_version 24363 (0.0008) [2023-10-10 17:31:31,943][123582] Updated weights for policy 0, policy_version 24373 (0.0009) [2023-10-10 17:31:32,314][123582] Updated weights for policy 0, policy_version 24383 (0.0008) [2023-10-10 17:31:33,517][123614] Updated weights for policy 1, policy_version 24330 (0.0007) [2023-10-10 17:31:33,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 49872896. Throughput: 0: 1806.1, 1: 1812.0. Samples: 12478820. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:31:33,789][122664] Avg episode reward: [(0, '31.830'), (1, '32.050')] [2023-10-10 17:31:33,886][123614] Updated weights for policy 1, policy_version 24340 (0.0007) [2023-10-10 17:31:34,255][123614] Updated weights for policy 1, policy_version 24350 (0.0007) [2023-10-10 17:31:35,983][123582] Updated weights for policy 0, policy_version 24393 (0.0008) [2023-10-10 17:31:36,358][123582] Updated weights for policy 0, policy_version 24403 (0.0008) [2023-10-10 17:31:36,731][123582] Updated weights for policy 0, policy_version 24413 (0.0008) [2023-10-10 17:31:37,927][123614] Updated weights for policy 1, policy_version 24360 (0.0008) [2023-10-10 17:31:38,294][123614] Updated weights for policy 1, policy_version 24370 (0.0010) [2023-10-10 17:31:38,663][123614] Updated weights for policy 1, policy_version 24380 (0.0009) [2023-10-10 17:31:38,788][122664] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 49938432. Throughput: 0: 1808.4, 1: 1812.8. Samples: 12500078. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:31:38,789][122664] Avg episode reward: [(0, '30.310'), (1, '34.230')] [2023-10-10 17:31:40,477][123582] Updated weights for policy 0, policy_version 24423 (0.0008) [2023-10-10 17:31:40,845][123582] Updated weights for policy 0, policy_version 24433 (0.0010) [2023-10-10 17:31:41,222][123582] Updated weights for policy 0, policy_version 24443 (0.0008) [2023-10-10 17:31:42,415][123614] Updated weights for policy 1, policy_version 24390 (0.0009) [2023-10-10 17:31:42,788][123614] Updated weights for policy 1, policy_version 24400 (0.0008) [2023-10-10 17:31:43,151][123614] Updated weights for policy 1, policy_version 24410 (0.0008) [2023-10-10 17:31:43,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 50036736. Throughput: 0: 1811.6, 1: 1802.7. Samples: 12511132. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:31:43,788][122664] Avg episode reward: [(0, '31.170'), (1, '31.250')] [2023-10-10 17:31:44,881][123582] Updated weights for policy 0, policy_version 24453 (0.0007) [2023-10-10 17:31:45,252][123582] Updated weights for policy 0, policy_version 24463 (0.0008) [2023-10-10 17:31:45,633][123582] Updated weights for policy 0, policy_version 24473 (0.0009) [2023-10-10 17:31:46,877][123614] Updated weights for policy 1, policy_version 24420 (0.0007) [2023-10-10 17:31:47,248][123614] Updated weights for policy 1, policy_version 24430 (0.0007) [2023-10-10 17:31:47,613][123614] Updated weights for policy 1, policy_version 24440 (0.0007) [2023-10-10 17:31:48,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 50102272. Throughput: 0: 1821.1, 1: 1806.7. Samples: 12532796. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-10-10 17:31:48,789][122664] Avg episode reward: [(0, '30.460'), (1, '30.450')] [2023-10-10 17:31:49,339][123582] Updated weights for policy 0, policy_version 24483 (0.0008) [2023-10-10 17:31:49,733][123582] Updated weights for policy 0, policy_version 24493 (0.0007) [2023-10-10 17:31:50,109][123582] Updated weights for policy 0, policy_version 24503 (0.0008) [2023-10-10 17:31:51,263][123614] Updated weights for policy 1, policy_version 24450 (0.0008) [2023-10-10 17:31:51,632][123614] Updated weights for policy 1, policy_version 24460 (0.0007) [2023-10-10 17:31:52,004][123614] Updated weights for policy 1, policy_version 24470 (0.0007) [2023-10-10 17:31:52,373][123614] Updated weights for policy 1, policy_version 24480 (0.0007) [2023-10-10 17:31:53,727][123582] Updated weights for policy 0, policy_version 24513 (0.0007) [2023-10-10 17:31:53,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 50167808. Throughput: 0: 1819.7, 1: 1809.6. Samples: 12555596. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-10-10 17:31:53,788][122664] Avg episode reward: [(0, '32.540'), (1, '29.420')] [2023-10-10 17:31:53,797][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000024480_25067520.pth... [2023-10-10 17:31:53,836][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000022784_23330816.pth [2023-10-10 17:31:54,090][123582] Updated weights for policy 0, policy_version 24523 (0.0009) [2023-10-10 17:31:54,464][123582] Updated weights for policy 0, policy_version 24533 (0.0007) [2023-10-10 17:31:54,827][123582] Updated weights for policy 0, policy_version 24543 (0.0008) [2023-10-10 17:31:54,864][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000024544_25133056.pth... [2023-10-10 17:31:54,902][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000022816_23363584.pth [2023-10-10 17:31:56,039][123614] Updated weights for policy 1, policy_version 24490 (0.0007) [2023-10-10 17:31:56,402][123614] Updated weights for policy 1, policy_version 24500 (0.0007) [2023-10-10 17:31:56,781][123614] Updated weights for policy 1, policy_version 24510 (0.0009) [2023-10-10 17:31:58,444][123582] Updated weights for policy 0, policy_version 24553 (0.0008) [2023-10-10 17:31:58,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 50233344. Throughput: 0: 1821.0, 1: 1815.8. Samples: 12565782. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-10-10 17:31:58,788][122664] Avg episode reward: [(0, '33.550'), (1, '27.890')] [2023-10-10 17:31:58,821][123582] Updated weights for policy 0, policy_version 24563 (0.0007) [2023-10-10 17:31:59,201][123582] Updated weights for policy 0, policy_version 24573 (0.0009) [2023-10-10 17:32:00,448][123614] Updated weights for policy 1, policy_version 24520 (0.0008) [2023-10-10 17:32:00,810][123614] Updated weights for policy 1, policy_version 24530 (0.0007) [2023-10-10 17:32:01,168][123614] Updated weights for policy 1, policy_version 24540 (0.0007) [2023-10-10 17:32:02,846][123582] Updated weights for policy 0, policy_version 24583 (0.0009) [2023-10-10 17:32:03,227][123582] Updated weights for policy 0, policy_version 24593 (0.0009) [2023-10-10 17:32:03,590][123582] Updated weights for policy 0, policy_version 24603 (0.0008) [2023-10-10 17:32:03,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 50331648. Throughput: 0: 1821.7, 1: 1807.4. Samples: 12588258. Policy #0 lag: (min: 19.0, avg: 22.9, max: 51.0) [2023-10-10 17:32:03,788][122664] Avg episode reward: [(0, '34.160'), (1, '30.080')] [2023-10-10 17:32:04,880][123614] Updated weights for policy 1, policy_version 24550 (0.0007) [2023-10-10 17:32:05,246][123614] Updated weights for policy 1, policy_version 24560 (0.0007) [2023-10-10 17:32:05,613][123614] Updated weights for policy 1, policy_version 24570 (0.0008) [2023-10-10 17:32:07,368][123582] Updated weights for policy 0, policy_version 24613 (0.0009) [2023-10-10 17:32:07,743][123582] Updated weights for policy 0, policy_version 24623 (0.0008) [2023-10-10 17:32:08,119][123582] Updated weights for policy 0, policy_version 24633 (0.0008) [2023-10-10 17:32:08,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 50397184. Throughput: 0: 1820.1, 1: 1806.0. Samples: 12609376. Policy #0 lag: (min: 19.0, avg: 22.9, max: 51.0) [2023-10-10 17:32:08,789][122664] Avg episode reward: [(0, '32.980'), (1, '31.420')] [2023-10-10 17:32:09,422][123614] Updated weights for policy 1, policy_version 24580 (0.0008) [2023-10-10 17:32:09,785][123614] Updated weights for policy 1, policy_version 24590 (0.0008) [2023-10-10 17:32:10,160][123614] Updated weights for policy 1, policy_version 24600 (0.0008) [2023-10-10 17:32:11,896][123582] Updated weights for policy 0, policy_version 24643 (0.0010) [2023-10-10 17:32:12,261][123582] Updated weights for policy 0, policy_version 24653 (0.0011) [2023-10-10 17:32:12,630][123582] Updated weights for policy 0, policy_version 24663 (0.0009) [2023-10-10 17:32:13,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 50462720. Throughput: 0: 1822.1, 1: 1803.1. Samples: 12620548. Policy #0 lag: (min: 19.0, avg: 22.9, max: 51.0) [2023-10-10 17:32:13,788][122664] Avg episode reward: [(0, '33.600'), (1, '31.130')] [2023-10-10 17:32:13,842][123614] Updated weights for policy 1, policy_version 24610 (0.0007) [2023-10-10 17:32:14,218][123614] Updated weights for policy 1, policy_version 24620 (0.0010) [2023-10-10 17:32:14,584][123614] Updated weights for policy 1, policy_version 24630 (0.0009) [2023-10-10 17:32:14,946][123614] Updated weights for policy 1, policy_version 24640 (0.0007) [2023-10-10 17:32:16,363][123582] Updated weights for policy 0, policy_version 24673 (0.0008) [2023-10-10 17:32:16,738][123582] Updated weights for policy 0, policy_version 24683 (0.0008) [2023-10-10 17:32:17,115][123582] Updated weights for policy 0, policy_version 24693 (0.0007) [2023-10-10 17:32:17,496][123582] Updated weights for policy 0, policy_version 24703 (0.0008) [2023-10-10 17:32:18,673][123614] Updated weights for policy 1, policy_version 24650 (0.0009) [2023-10-10 17:32:18,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 50528256. Throughput: 0: 1825.0, 1: 1806.3. Samples: 12642228. Policy #0 lag: (min: 19.0, avg: 22.9, max: 51.0) [2023-10-10 17:32:18,788][122664] Avg episode reward: [(0, '34.470'), (1, '31.770')] [2023-10-10 17:32:19,038][123614] Updated weights for policy 1, policy_version 24660 (0.0007) [2023-10-10 17:32:19,407][123614] Updated weights for policy 1, policy_version 24670 (0.0008) [2023-10-10 17:32:21,204][123582] Updated weights for policy 0, policy_version 24713 (0.0009) [2023-10-10 17:32:21,576][123582] Updated weights for policy 0, policy_version 24723 (0.0008) [2023-10-10 17:32:21,947][123582] Updated weights for policy 0, policy_version 24733 (0.0007) [2023-10-10 17:32:23,094][123614] Updated weights for policy 1, policy_version 24680 (0.0008) [2023-10-10 17:32:23,463][123614] Updated weights for policy 1, policy_version 24690 (0.0008) [2023-10-10 17:32:23,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 50593792. Throughput: 0: 1813.7, 1: 1817.5. Samples: 12663480. Policy #0 lag: (min: 3.0, avg: 13.8, max: 35.0) [2023-10-10 17:32:23,789][122664] Avg episode reward: [(0, '36.250'), (1, '31.430')] [2023-10-10 17:32:23,835][123614] Updated weights for policy 1, policy_version 24700 (0.0011) [2023-10-10 17:32:25,791][123582] Updated weights for policy 0, policy_version 24743 (0.0007) [2023-10-10 17:32:26,159][123582] Updated weights for policy 0, policy_version 24753 (0.0008) [2023-10-10 17:32:26,532][123582] Updated weights for policy 0, policy_version 24763 (0.0008) [2023-10-10 17:32:27,480][123614] Updated weights for policy 1, policy_version 24710 (0.0007) [2023-10-10 17:32:27,856][123614] Updated weights for policy 1, policy_version 24720 (0.0007) [2023-10-10 17:32:28,222][123614] Updated weights for policy 1, policy_version 24730 (0.0009) [2023-10-10 17:32:28,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 50692096. Throughput: 0: 1821.7, 1: 1821.1. Samples: 12675058. Policy #0 lag: (min: 3.0, avg: 13.8, max: 35.0) [2023-10-10 17:32:28,789][122664] Avg episode reward: [(0, '34.970'), (1, '35.360')] [2023-10-10 17:32:28,790][123465] Saving new best policy, reward=35.360! [2023-10-10 17:32:30,219][123582] Updated weights for policy 0, policy_version 24773 (0.0008) [2023-10-10 17:32:30,582][123582] Updated weights for policy 0, policy_version 24783 (0.0008) [2023-10-10 17:32:30,952][123582] Updated weights for policy 0, policy_version 24793 (0.0009) [2023-10-10 17:32:31,920][123614] Updated weights for policy 1, policy_version 24740 (0.0007) [2023-10-10 17:32:32,293][123614] Updated weights for policy 1, policy_version 24750 (0.0008) [2023-10-10 17:32:32,668][123614] Updated weights for policy 1, policy_version 24760 (0.0008) [2023-10-10 17:32:33,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 50757632. Throughput: 0: 1811.0, 1: 1825.5. Samples: 12696440. Policy #0 lag: (min: 3.0, avg: 13.8, max: 35.0) [2023-10-10 17:32:33,788][122664] Avg episode reward: [(0, '33.280'), (1, '36.400')] [2023-10-10 17:32:33,789][123465] Saving new best policy, reward=36.400! [2023-10-10 17:32:34,743][123582] Updated weights for policy 0, policy_version 24803 (0.0008) [2023-10-10 17:32:35,143][123582] Updated weights for policy 0, policy_version 24813 (0.0007) [2023-10-10 17:32:35,509][123582] Updated weights for policy 0, policy_version 24823 (0.0007) [2023-10-10 17:32:36,278][123614] Updated weights for policy 1, policy_version 24770 (0.0009) [2023-10-10 17:32:36,652][123614] Updated weights for policy 1, policy_version 24780 (0.0008) [2023-10-10 17:32:37,023][123614] Updated weights for policy 1, policy_version 24790 (0.0009) [2023-10-10 17:32:37,389][123614] Updated weights for policy 1, policy_version 24800 (0.0008) [2023-10-10 17:32:38,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 50823168. Throughput: 0: 1807.2, 1: 1817.0. Samples: 12718688. Policy #0 lag: (min: 3.0, avg: 13.8, max: 35.0) [2023-10-10 17:32:38,789][122664] Avg episode reward: [(0, '31.500'), (1, '35.300')] [2023-10-10 17:32:39,233][123582] Updated weights for policy 0, policy_version 24833 (0.0010) [2023-10-10 17:32:39,605][123582] Updated weights for policy 0, policy_version 24843 (0.0007) [2023-10-10 17:32:39,980][123582] Updated weights for policy 0, policy_version 24853 (0.0009) [2023-10-10 17:32:40,350][123582] Updated weights for policy 0, policy_version 24863 (0.0008) [2023-10-10 17:32:41,103][123614] Updated weights for policy 1, policy_version 24810 (0.0011) [2023-10-10 17:32:41,474][123614] Updated weights for policy 1, policy_version 24820 (0.0008) [2023-10-10 17:32:41,854][123614] Updated weights for policy 1, policy_version 24830 (0.0011) [2023-10-10 17:32:43,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 50888704. Throughput: 0: 1806.0, 1: 1817.3. Samples: 12728830. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-10 17:32:43,788][122664] Avg episode reward: [(0, '30.150'), (1, '35.040')] [2023-10-10 17:32:43,987][123582] Updated weights for policy 0, policy_version 24873 (0.0008) [2023-10-10 17:32:44,349][123582] Updated weights for policy 0, policy_version 24883 (0.0008) [2023-10-10 17:32:44,720][123582] Updated weights for policy 0, policy_version 24893 (0.0008) [2023-10-10 17:32:45,565][123614] Updated weights for policy 1, policy_version 24840 (0.0010) [2023-10-10 17:32:45,940][123614] Updated weights for policy 1, policy_version 24850 (0.0009) [2023-10-10 17:32:46,305][123614] Updated weights for policy 1, policy_version 24860 (0.0007) [2023-10-10 17:32:48,198][123582] Updated weights for policy 0, policy_version 24903 (0.0008) [2023-10-10 17:32:48,575][123582] Updated weights for policy 0, policy_version 24913 (0.0007) [2023-10-10 17:32:48,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 50954240. Throughput: 0: 1812.5, 1: 1815.0. Samples: 12751496. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-10 17:32:48,788][122664] Avg episode reward: [(0, '31.850'), (1, '34.330')] [2023-10-10 17:32:48,940][123582] Updated weights for policy 0, policy_version 24923 (0.0007) [2023-10-10 17:32:50,007][123614] Updated weights for policy 1, policy_version 24870 (0.0008) [2023-10-10 17:32:50,369][123614] Updated weights for policy 1, policy_version 24880 (0.0008) [2023-10-10 17:32:50,735][123614] Updated weights for policy 1, policy_version 24890 (0.0009) [2023-10-10 17:32:52,650][123582] Updated weights for policy 0, policy_version 24933 (0.0007) [2023-10-10 17:32:53,021][123582] Updated weights for policy 0, policy_version 24943 (0.0008) [2023-10-10 17:32:53,395][123582] Updated weights for policy 0, policy_version 24953 (0.0008) [2023-10-10 17:32:53,788][122664] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 51052544. Throughput: 0: 1822.1, 1: 1817.9. Samples: 12773176. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-10 17:32:53,789][122664] Avg episode reward: [(0, '34.270'), (1, '33.840')] [2023-10-10 17:32:54,459][123614] Updated weights for policy 1, policy_version 24900 (0.0009) [2023-10-10 17:32:54,829][123614] Updated weights for policy 1, policy_version 24910 (0.0008) [2023-10-10 17:32:55,194][123614] Updated weights for policy 1, policy_version 24920 (0.0007) [2023-10-10 17:32:56,940][123582] Updated weights for policy 0, policy_version 24963 (0.0008) [2023-10-10 17:32:57,308][123582] Updated weights for policy 0, policy_version 24973 (0.0008) [2023-10-10 17:32:57,678][123582] Updated weights for policy 0, policy_version 24983 (0.0009) [2023-10-10 17:32:58,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 51118080. Throughput: 0: 1822.3, 1: 1818.2. Samples: 12784372. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-10 17:32:58,789][122664] Avg episode reward: [(0, '34.720'), (1, '34.440')] [2023-10-10 17:32:58,927][123614] Updated weights for policy 1, policy_version 24930 (0.0007) [2023-10-10 17:32:59,293][123614] Updated weights for policy 1, policy_version 24940 (0.0009) [2023-10-10 17:32:59,667][123614] Updated weights for policy 1, policy_version 24950 (0.0008) [2023-10-10 17:33:00,025][123614] Updated weights for policy 1, policy_version 24960 (0.0007) [2023-10-10 17:33:01,300][123582] Updated weights for policy 0, policy_version 24993 (0.0008) [2023-10-10 17:33:01,677][123582] Updated weights for policy 0, policy_version 25003 (0.0009) [2023-10-10 17:33:02,053][123582] Updated weights for policy 0, policy_version 25013 (0.0007) [2023-10-10 17:33:02,429][123582] Updated weights for policy 0, policy_version 25023 (0.0008) [2023-10-10 17:33:03,643][123614] Updated weights for policy 1, policy_version 24970 (0.0010) [2023-10-10 17:33:03,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 51183616. Throughput: 0: 1824.3, 1: 1819.1. Samples: 12806178. Policy #0 lag: (min: 13.0, avg: 21.0, max: 45.0) [2023-10-10 17:33:03,789][122664] Avg episode reward: [(0, '33.990'), (1, '35.400')] [2023-10-10 17:33:04,013][123614] Updated weights for policy 1, policy_version 24980 (0.0010) [2023-10-10 17:33:04,379][123614] Updated weights for policy 1, policy_version 24990 (0.0010) [2023-10-10 17:33:06,088][123582] Updated weights for policy 0, policy_version 25033 (0.0008) [2023-10-10 17:33:06,471][123582] Updated weights for policy 0, policy_version 25043 (0.0007) [2023-10-10 17:33:06,845][123582] Updated weights for policy 0, policy_version 25053 (0.0007) [2023-10-10 17:33:08,100][123614] Updated weights for policy 1, policy_version 25000 (0.0011) [2023-10-10 17:33:08,469][123614] Updated weights for policy 1, policy_version 25010 (0.0009) [2023-10-10 17:33:08,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 51249152. Throughput: 0: 1832.1, 1: 1823.9. Samples: 12827998. Policy #0 lag: (min: 13.0, avg: 21.0, max: 45.0) [2023-10-10 17:33:08,789][122664] Avg episode reward: [(0, '32.240'), (1, '37.710')] [2023-10-10 17:33:08,847][123614] Updated weights for policy 1, policy_version 25020 (0.0009) [2023-10-10 17:33:08,998][123465] Saving new best policy, reward=37.710! [2023-10-10 17:33:10,415][123582] Updated weights for policy 0, policy_version 25063 (0.0008) [2023-10-10 17:33:10,789][123582] Updated weights for policy 0, policy_version 25073 (0.0010) [2023-10-10 17:33:11,152][123582] Updated weights for policy 0, policy_version 25083 (0.0008) [2023-10-10 17:33:12,738][123614] Updated weights for policy 1, policy_version 25030 (0.0010) [2023-10-10 17:33:13,117][123614] Updated weights for policy 1, policy_version 25040 (0.0008) [2023-10-10 17:33:13,486][123614] Updated weights for policy 1, policy_version 25050 (0.0007) [2023-10-10 17:33:13,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 51347456. Throughput: 0: 1822.0, 1: 1817.4. Samples: 12838828. Policy #0 lag: (min: 13.0, avg: 21.0, max: 45.0) [2023-10-10 17:33:13,788][122664] Avg episode reward: [(0, '34.270'), (1, '42.280')] [2023-10-10 17:33:13,789][123465] Saving new best policy, reward=42.280! [2023-10-10 17:33:14,788][123582] Updated weights for policy 0, policy_version 25093 (0.0007) [2023-10-10 17:33:15,173][123582] Updated weights for policy 0, policy_version 25103 (0.0008) [2023-10-10 17:33:15,544][123582] Updated weights for policy 0, policy_version 25113 (0.0009) [2023-10-10 17:33:17,179][123614] Updated weights for policy 1, policy_version 25060 (0.0008) [2023-10-10 17:33:17,541][123614] Updated weights for policy 1, policy_version 25070 (0.0008) [2023-10-10 17:33:17,908][123614] Updated weights for policy 1, policy_version 25080 (0.0007) [2023-10-10 17:33:18,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 51412992. Throughput: 0: 1828.9, 1: 1821.1. Samples: 12860690. Policy #0 lag: (min: 13.0, avg: 21.0, max: 45.0) [2023-10-10 17:33:18,788][122664] Avg episode reward: [(0, '31.810'), (1, '40.890')] [2023-10-10 17:33:19,257][123582] Updated weights for policy 0, policy_version 25123 (0.0009) [2023-10-10 17:33:19,654][123582] Updated weights for policy 0, policy_version 25133 (0.0011) [2023-10-10 17:33:20,020][123582] Updated weights for policy 0, policy_version 25143 (0.0009) [2023-10-10 17:33:21,637][123614] Updated weights for policy 1, policy_version 25090 (0.0007) [2023-10-10 17:33:22,004][123614] Updated weights for policy 1, policy_version 25100 (0.0009) [2023-10-10 17:33:22,371][123614] Updated weights for policy 1, policy_version 25110 (0.0009) [2023-10-10 17:33:22,739][123614] Updated weights for policy 1, policy_version 25120 (0.0008) [2023-10-10 17:33:23,730][123582] Updated weights for policy 0, policy_version 25153 (0.0009) [2023-10-10 17:33:23,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 51478528. Throughput: 0: 1834.8, 1: 1808.7. Samples: 12882646. Policy #0 lag: (min: 12.0, avg: 12.0, max: 14.0) [2023-10-10 17:33:23,788][122664] Avg episode reward: [(0, '30.440'), (1, '39.140')] [2023-10-10 17:33:24,113][123582] Updated weights for policy 0, policy_version 25163 (0.0009) [2023-10-10 17:33:24,492][123582] Updated weights for policy 0, policy_version 25173 (0.0008) [2023-10-10 17:33:24,855][123582] Updated weights for policy 0, policy_version 25183 (0.0008) [2023-10-10 17:33:26,396][123614] Updated weights for policy 1, policy_version 25130 (0.0007) [2023-10-10 17:33:26,759][123614] Updated weights for policy 1, policy_version 25140 (0.0009) [2023-10-10 17:33:27,127][123614] Updated weights for policy 1, policy_version 25150 (0.0007) [2023-10-10 17:33:28,350][123582] Updated weights for policy 0, policy_version 25193 (0.0009) [2023-10-10 17:33:28,730][123582] Updated weights for policy 0, policy_version 25203 (0.0009) [2023-10-10 17:33:28,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 51544064. Throughput: 0: 1835.3, 1: 1818.9. Samples: 12893272. Policy #0 lag: (min: 12.0, avg: 12.0, max: 14.0) [2023-10-10 17:33:28,788][122664] Avg episode reward: [(0, '34.680'), (1, '38.680')] [2023-10-10 17:33:29,100][123582] Updated weights for policy 0, policy_version 25213 (0.0008) [2023-10-10 17:33:30,838][123614] Updated weights for policy 1, policy_version 25160 (0.0008) [2023-10-10 17:33:31,200][123614] Updated weights for policy 1, policy_version 25170 (0.0007) [2023-10-10 17:33:31,567][123614] Updated weights for policy 1, policy_version 25180 (0.0008) [2023-10-10 17:33:32,886][123582] Updated weights for policy 0, policy_version 25223 (0.0009) [2023-10-10 17:33:33,264][123582] Updated weights for policy 0, policy_version 25233 (0.0007) [2023-10-10 17:33:33,638][123582] Updated weights for policy 0, policy_version 25243 (0.0009) [2023-10-10 17:33:33,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 51609600. Throughput: 0: 1824.9, 1: 1814.9. Samples: 12915286. Policy #0 lag: (min: 12.0, avg: 12.0, max: 14.0) [2023-10-10 17:33:33,789][122664] Avg episode reward: [(0, '36.390'), (1, '40.180')] [2023-10-10 17:33:35,096][123614] Updated weights for policy 1, policy_version 25190 (0.0009) [2023-10-10 17:33:35,460][123614] Updated weights for policy 1, policy_version 25200 (0.0008) [2023-10-10 17:33:35,830][123614] Updated weights for policy 1, policy_version 25210 (0.0008) [2023-10-10 17:33:37,434][123582] Updated weights for policy 0, policy_version 25253 (0.0008) [2023-10-10 17:33:37,815][123582] Updated weights for policy 0, policy_version 25263 (0.0008) [2023-10-10 17:33:38,182][123582] Updated weights for policy 0, policy_version 25273 (0.0009) [2023-10-10 17:33:38,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 51707904. Throughput: 0: 1813.8, 1: 1819.3. Samples: 12936662. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 17:33:38,788][122664] Avg episode reward: [(0, '36.450'), (1, '38.310')] [2023-10-10 17:33:39,565][123614] Updated weights for policy 1, policy_version 25220 (0.0009) [2023-10-10 17:33:39,940][123614] Updated weights for policy 1, policy_version 25230 (0.0010) [2023-10-10 17:33:40,301][123614] Updated weights for policy 1, policy_version 25240 (0.0007) [2023-10-10 17:33:42,054][123582] Updated weights for policy 0, policy_version 25283 (0.0009) [2023-10-10 17:33:42,422][123582] Updated weights for policy 0, policy_version 25293 (0.0010) [2023-10-10 17:33:42,801][123582] Updated weights for policy 0, policy_version 25303 (0.0008) [2023-10-10 17:33:43,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 51773440. Throughput: 0: 1812.6, 1: 1818.4. Samples: 12947764. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 17:33:43,789][122664] Avg episode reward: [(0, '36.520'), (1, '39.100')] [2023-10-10 17:33:44,000][123614] Updated weights for policy 1, policy_version 25250 (0.0008) [2023-10-10 17:33:44,371][123614] Updated weights for policy 1, policy_version 25260 (0.0011) [2023-10-10 17:33:44,737][123614] Updated weights for policy 1, policy_version 25270 (0.0007) [2023-10-10 17:33:45,111][123614] Updated weights for policy 1, policy_version 25280 (0.0008) [2023-10-10 17:33:46,329][123582] Updated weights for policy 0, policy_version 25313 (0.0007) [2023-10-10 17:33:46,695][123582] Updated weights for policy 0, policy_version 25323 (0.0009) [2023-10-10 17:33:47,074][123582] Updated weights for policy 0, policy_version 25333 (0.0009) [2023-10-10 17:33:47,444][123582] Updated weights for policy 0, policy_version 25343 (0.0007) [2023-10-10 17:33:48,783][123614] Updated weights for policy 1, policy_version 25290 (0.0008) [2023-10-10 17:33:48,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 51838976. Throughput: 0: 1814.4, 1: 1811.0. Samples: 12969320. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 17:33:48,789][122664] Avg episode reward: [(0, '36.980'), (1, '33.720')] [2023-10-10 17:33:49,152][123614] Updated weights for policy 1, policy_version 25300 (0.0007) [2023-10-10 17:33:49,524][123614] Updated weights for policy 1, policy_version 25310 (0.0007) [2023-10-10 17:33:51,022][123582] Updated weights for policy 0, policy_version 25353 (0.0007) [2023-10-10 17:33:51,405][123582] Updated weights for policy 0, policy_version 25363 (0.0008) [2023-10-10 17:33:51,774][123582] Updated weights for policy 0, policy_version 25373 (0.0009) [2023-10-10 17:33:53,200][123614] Updated weights for policy 1, policy_version 25320 (0.0009) [2023-10-10 17:33:53,577][123614] Updated weights for policy 1, policy_version 25330 (0.0011) [2023-10-10 17:33:53,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 51904512. Throughput: 0: 1813.8, 1: 1810.8. Samples: 12991106. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 17:33:53,788][122664] Avg episode reward: [(0, '38.520'), (1, '32.640')] [2023-10-10 17:33:53,798][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000025376_25985024.pth... [2023-10-10 17:33:53,833][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000023680_24248320.pth [2023-10-10 17:33:53,837][123247] Saving a milestone ./train_atari/atari_demonattack_APPO/checkpoint_p0/milestones/checkpoint_000025376_25985024.pth [2023-10-10 17:33:53,939][123614] Updated weights for policy 1, policy_version 25340 (0.0009) [2023-10-10 17:33:54,085][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000025344_25952256.pth... [2023-10-10 17:33:54,123][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000023648_24215552.pth [2023-10-10 17:33:54,128][123465] Saving a milestone ./train_atari/atari_demonattack_APPO/checkpoint_p1/milestones/checkpoint_000025344_25952256.pth [2023-10-10 17:33:55,457][123582] Updated weights for policy 0, policy_version 25383 (0.0010) [2023-10-10 17:33:55,839][123582] Updated weights for policy 0, policy_version 25393 (0.0008) [2023-10-10 17:33:56,216][123582] Updated weights for policy 0, policy_version 25403 (0.0007) [2023-10-10 17:33:57,723][123614] Updated weights for policy 1, policy_version 25350 (0.0007) [2023-10-10 17:33:58,090][123614] Updated weights for policy 1, policy_version 25360 (0.0010) [2023-10-10 17:33:58,465][123614] Updated weights for policy 1, policy_version 25370 (0.0010) [2023-10-10 17:33:58,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 52002816. Throughput: 0: 1815.2, 1: 1813.5. Samples: 13002116. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-10 17:33:58,789][122664] Avg episode reward: [(0, '36.530'), (1, '30.240')] [2023-10-10 17:33:59,904][123582] Updated weights for policy 0, policy_version 25413 (0.0008) [2023-10-10 17:34:00,272][123582] Updated weights for policy 0, policy_version 25423 (0.0007) [2023-10-10 17:34:00,641][123582] Updated weights for policy 0, policy_version 25433 (0.0009) [2023-10-10 17:34:02,225][123614] Updated weights for policy 1, policy_version 25380 (0.0008) [2023-10-10 17:34:02,603][123614] Updated weights for policy 1, policy_version 25390 (0.0009) [2023-10-10 17:34:02,975][123614] Updated weights for policy 1, policy_version 25400 (0.0009) [2023-10-10 17:34:03,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 52068352. Throughput: 0: 1809.2, 1: 1816.1. Samples: 13023826. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-10 17:34:03,788][122664] Avg episode reward: [(0, '38.980'), (1, '28.620')] [2023-10-10 17:34:03,789][123247] Saving new best policy, reward=38.980! [2023-10-10 17:34:04,529][123582] Updated weights for policy 0, policy_version 25443 (0.0009) [2023-10-10 17:34:04,936][123582] Updated weights for policy 0, policy_version 25453 (0.0008) [2023-10-10 17:34:05,300][123582] Updated weights for policy 0, policy_version 25463 (0.0008) [2023-10-10 17:34:06,766][123614] Updated weights for policy 1, policy_version 25410 (0.0009) [2023-10-10 17:34:07,132][123614] Updated weights for policy 1, policy_version 25420 (0.0007) [2023-10-10 17:34:07,498][123614] Updated weights for policy 1, policy_version 25430 (0.0007) [2023-10-10 17:34:07,872][123614] Updated weights for policy 1, policy_version 25440 (0.0007) [2023-10-10 17:34:08,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 52133888. Throughput: 0: 1806.3, 1: 1818.1. Samples: 13045746. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-10 17:34:08,788][122664] Avg episode reward: [(0, '36.810'), (1, '28.230')] [2023-10-10 17:34:08,853][123582] Updated weights for policy 0, policy_version 25473 (0.0007) [2023-10-10 17:34:09,224][123582] Updated weights for policy 0, policy_version 25483 (0.0010) [2023-10-10 17:34:09,596][123582] Updated weights for policy 0, policy_version 25493 (0.0011) [2023-10-10 17:34:09,968][123582] Updated weights for policy 0, policy_version 25503 (0.0010) [2023-10-10 17:34:11,501][123614] Updated weights for policy 1, policy_version 25450 (0.0008) [2023-10-10 17:34:11,875][123614] Updated weights for policy 1, policy_version 25460 (0.0007) [2023-10-10 17:34:12,244][123614] Updated weights for policy 1, policy_version 25470 (0.0009) [2023-10-10 17:34:13,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 52199424. Throughput: 0: 1801.5, 1: 1823.4. Samples: 13056394. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-10 17:34:13,789][122664] Avg episode reward: [(0, '33.880'), (1, '29.090')] [2023-10-10 17:34:13,847][123582] Updated weights for policy 0, policy_version 25513 (0.0008) [2023-10-10 17:34:14,223][123582] Updated weights for policy 0, policy_version 25523 (0.0007) [2023-10-10 17:34:14,595][123582] Updated weights for policy 0, policy_version 25533 (0.0009) [2023-10-10 17:34:15,836][123614] Updated weights for policy 1, policy_version 25480 (0.0008) [2023-10-10 17:34:16,203][123614] Updated weights for policy 1, policy_version 25490 (0.0007) [2023-10-10 17:34:16,570][123614] Updated weights for policy 1, policy_version 25500 (0.0007) [2023-10-10 17:34:18,286][123582] Updated weights for policy 0, policy_version 25543 (0.0008) [2023-10-10 17:34:18,654][123582] Updated weights for policy 0, policy_version 25553 (0.0007) [2023-10-10 17:34:18,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 52264960. Throughput: 0: 1802.0, 1: 1816.3. Samples: 13078108. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-10-10 17:34:18,788][122664] Avg episode reward: [(0, '30.640'), (1, '28.660')] [2023-10-10 17:34:19,026][123582] Updated weights for policy 0, policy_version 25563 (0.0007) [2023-10-10 17:34:20,337][123614] Updated weights for policy 1, policy_version 25510 (0.0007) [2023-10-10 17:34:20,702][123614] Updated weights for policy 1, policy_version 25520 (0.0007) [2023-10-10 17:34:21,071][123614] Updated weights for policy 1, policy_version 25530 (0.0007) [2023-10-10 17:34:22,702][123582] Updated weights for policy 0, policy_version 25573 (0.0009) [2023-10-10 17:34:23,073][123582] Updated weights for policy 0, policy_version 25583 (0.0007) [2023-10-10 17:34:23,447][123582] Updated weights for policy 0, policy_version 25593 (0.0010) [2023-10-10 17:34:23,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 52363264. Throughput: 0: 1814.9, 1: 1815.0. Samples: 13100008. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-10-10 17:34:23,789][122664] Avg episode reward: [(0, '30.870'), (1, '27.590')] [2023-10-10 17:34:24,702][123614] Updated weights for policy 1, policy_version 25540 (0.0008) [2023-10-10 17:34:25,075][123614] Updated weights for policy 1, policy_version 25550 (0.0008) [2023-10-10 17:34:25,451][123614] Updated weights for policy 1, policy_version 25560 (0.0008) [2023-10-10 17:34:27,165][123582] Updated weights for policy 0, policy_version 25603 (0.0009) [2023-10-10 17:34:27,539][123582] Updated weights for policy 0, policy_version 25613 (0.0008) [2023-10-10 17:34:27,903][123582] Updated weights for policy 0, policy_version 25623 (0.0009) [2023-10-10 17:34:28,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 52428800. Throughput: 0: 1811.7, 1: 1815.3. Samples: 13110980. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-10-10 17:34:28,789][122664] Avg episode reward: [(0, '30.160'), (1, '30.240')] [2023-10-10 17:34:29,115][123614] Updated weights for policy 1, policy_version 25570 (0.0010) [2023-10-10 17:34:29,485][123614] Updated weights for policy 1, policy_version 25580 (0.0011) [2023-10-10 17:34:29,862][123614] Updated weights for policy 1, policy_version 25590 (0.0007) [2023-10-10 17:34:30,224][123614] Updated weights for policy 1, policy_version 25600 (0.0009) [2023-10-10 17:34:31,572][123582] Updated weights for policy 0, policy_version 25633 (0.0009) [2023-10-10 17:34:31,938][123582] Updated weights for policy 0, policy_version 25643 (0.0008) [2023-10-10 17:34:32,313][123582] Updated weights for policy 0, policy_version 25653 (0.0007) [2023-10-10 17:34:32,695][123582] Updated weights for policy 0, policy_version 25663 (0.0007) [2023-10-10 17:34:33,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 52494336. Throughput: 0: 1813.3, 1: 1817.1. Samples: 13132686. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-10-10 17:34:33,789][122664] Avg episode reward: [(0, '30.940'), (1, '29.770')] [2023-10-10 17:34:33,834][123614] Updated weights for policy 1, policy_version 25610 (0.0007) [2023-10-10 17:34:34,210][123614] Updated weights for policy 1, policy_version 25620 (0.0008) [2023-10-10 17:34:34,571][123614] Updated weights for policy 1, policy_version 25630 (0.0007) [2023-10-10 17:34:36,476][123582] Updated weights for policy 0, policy_version 25673 (0.0009) [2023-10-10 17:34:36,841][123582] Updated weights for policy 0, policy_version 25683 (0.0011) [2023-10-10 17:34:37,215][123582] Updated weights for policy 0, policy_version 25693 (0.0010) [2023-10-10 17:34:38,376][123614] Updated weights for policy 1, policy_version 25640 (0.0008) [2023-10-10 17:34:38,746][123614] Updated weights for policy 1, policy_version 25650 (0.0008) [2023-10-10 17:34:38,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 52559872. Throughput: 0: 1804.8, 1: 1818.2. Samples: 13154138. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-10-10 17:34:38,788][122664] Avg episode reward: [(0, '30.930'), (1, '30.310')] [2023-10-10 17:34:39,099][123614] Updated weights for policy 1, policy_version 25660 (0.0009) [2023-10-10 17:34:40,769][123582] Updated weights for policy 0, policy_version 25703 (0.0009) [2023-10-10 17:34:41,125][123582] Updated weights for policy 0, policy_version 25713 (0.0009) [2023-10-10 17:34:41,501][123582] Updated weights for policy 0, policy_version 25723 (0.0008) [2023-10-10 17:34:42,799][123614] Updated weights for policy 1, policy_version 25670 (0.0009) [2023-10-10 17:34:43,187][123614] Updated weights for policy 1, policy_version 25680 (0.0007) [2023-10-10 17:34:43,556][123614] Updated weights for policy 1, policy_version 25690 (0.0008) [2023-10-10 17:34:43,788][122664] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 52658176. Throughput: 0: 1814.4, 1: 1815.5. Samples: 13165460. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-10-10 17:34:43,788][122664] Avg episode reward: [(0, '31.810'), (1, '30.380')] [2023-10-10 17:34:45,158][123582] Updated weights for policy 0, policy_version 25733 (0.0009) [2023-10-10 17:34:45,536][123582] Updated weights for policy 0, policy_version 25743 (0.0009) [2023-10-10 17:34:45,907][123582] Updated weights for policy 0, policy_version 25753 (0.0010) [2023-10-10 17:34:47,305][123614] Updated weights for policy 1, policy_version 25700 (0.0009) [2023-10-10 17:34:47,675][123614] Updated weights for policy 1, policy_version 25710 (0.0007) [2023-10-10 17:34:48,039][123614] Updated weights for policy 1, policy_version 25720 (0.0007) [2023-10-10 17:34:48,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 52723712. Throughput: 0: 1815.9, 1: 1812.1. Samples: 13187086. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-10-10 17:34:48,789][122664] Avg episode reward: [(0, '32.080'), (1, '29.160')] [2023-10-10 17:34:49,656][123582] Updated weights for policy 0, policy_version 25763 (0.0008) [2023-10-10 17:34:50,044][123582] Updated weights for policy 0, policy_version 25773 (0.0009) [2023-10-10 17:34:50,421][123582] Updated weights for policy 0, policy_version 25783 (0.0008) [2023-10-10 17:34:51,871][123614] Updated weights for policy 1, policy_version 25730 (0.0009) [2023-10-10 17:34:52,246][123614] Updated weights for policy 1, policy_version 25740 (0.0009) [2023-10-10 17:34:52,615][123614] Updated weights for policy 1, policy_version 25750 (0.0007) [2023-10-10 17:34:52,984][123614] Updated weights for policy 1, policy_version 25760 (0.0008) [2023-10-10 17:34:53,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 52789248. Throughput: 0: 1814.8, 1: 1808.0. Samples: 13208774. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-10-10 17:34:53,789][122664] Avg episode reward: [(0, '32.920'), (1, '27.520')] [2023-10-10 17:34:54,044][123582] Updated weights for policy 0, policy_version 25793 (0.0009) [2023-10-10 17:34:54,415][123582] Updated weights for policy 0, policy_version 25803 (0.0008) [2023-10-10 17:34:54,781][123582] Updated weights for policy 0, policy_version 25813 (0.0008) [2023-10-10 17:34:55,166][123582] Updated weights for policy 0, policy_version 25823 (0.0008) [2023-10-10 17:34:56,656][123614] Updated weights for policy 1, policy_version 25770 (0.0007) [2023-10-10 17:34:57,018][123614] Updated weights for policy 1, policy_version 25780 (0.0010) [2023-10-10 17:34:57,383][123614] Updated weights for policy 1, policy_version 25790 (0.0010) [2023-10-10 17:34:58,725][123582] Updated weights for policy 0, policy_version 25833 (0.0008) [2023-10-10 17:34:58,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 52854784. Throughput: 0: 1823.2, 1: 1808.6. Samples: 13219826. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-10-10 17:34:58,789][122664] Avg episode reward: [(0, '32.910'), (1, '26.720')] [2023-10-10 17:34:59,099][123582] Updated weights for policy 0, policy_version 25843 (0.0009) [2023-10-10 17:34:59,469][123582] Updated weights for policy 0, policy_version 25853 (0.0011) [2023-10-10 17:35:01,172][123614] Updated weights for policy 1, policy_version 25800 (0.0009) [2023-10-10 17:35:01,548][123614] Updated weights for policy 1, policy_version 25810 (0.0008) [2023-10-10 17:35:01,918][123614] Updated weights for policy 1, policy_version 25820 (0.0010) [2023-10-10 17:35:03,296][123582] Updated weights for policy 0, policy_version 25863 (0.0009) [2023-10-10 17:35:03,677][123582] Updated weights for policy 0, policy_version 25873 (0.0007) [2023-10-10 17:35:03,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 52920320. Throughput: 0: 1820.2, 1: 1802.9. Samples: 13241146. Policy #0 lag: (min: 5.0, avg: 5.1, max: 12.0) [2023-10-10 17:35:03,788][122664] Avg episode reward: [(0, '32.630'), (1, '25.230')] [2023-10-10 17:35:04,058][123582] Updated weights for policy 0, policy_version 25883 (0.0007) [2023-10-10 17:35:05,769][123614] Updated weights for policy 1, policy_version 25830 (0.0008) [2023-10-10 17:35:06,142][123614] Updated weights for policy 1, policy_version 25840 (0.0009) [2023-10-10 17:35:06,505][123614] Updated weights for policy 1, policy_version 25850 (0.0010) [2023-10-10 17:35:07,723][123582] Updated weights for policy 0, policy_version 25893 (0.0008) [2023-10-10 17:35:08,091][123582] Updated weights for policy 0, policy_version 25903 (0.0010) [2023-10-10 17:35:08,474][123582] Updated weights for policy 0, policy_version 25913 (0.0009) [2023-10-10 17:35:08,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 53018624. Throughput: 0: 1821.5, 1: 1799.7. Samples: 13262960. Policy #0 lag: (min: 5.0, avg: 5.1, max: 12.0) [2023-10-10 17:35:08,789][122664] Avg episode reward: [(0, '33.690'), (1, '26.010')] [2023-10-10 17:35:10,280][123614] Updated weights for policy 1, policy_version 25860 (0.0009) [2023-10-10 17:35:10,646][123614] Updated weights for policy 1, policy_version 25870 (0.0008) [2023-10-10 17:35:11,008][123614] Updated weights for policy 1, policy_version 25880 (0.0007) [2023-10-10 17:35:12,339][123582] Updated weights for policy 0, policy_version 25923 (0.0008) [2023-10-10 17:35:12,718][123582] Updated weights for policy 0, policy_version 25933 (0.0008) [2023-10-10 17:35:13,072][123582] Updated weights for policy 0, policy_version 25943 (0.0008) [2023-10-10 17:35:13,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 53084160. Throughput: 0: 1820.1, 1: 1798.2. Samples: 13273804. Policy #0 lag: (min: 5.0, avg: 5.1, max: 12.0) [2023-10-10 17:35:13,789][122664] Avg episode reward: [(0, '32.230'), (1, '26.820')] [2023-10-10 17:35:14,786][123614] Updated weights for policy 1, policy_version 25890 (0.0009) [2023-10-10 17:35:15,152][123614] Updated weights for policy 1, policy_version 25900 (0.0007) [2023-10-10 17:35:15,511][123614] Updated weights for policy 1, policy_version 25910 (0.0008) [2023-10-10 17:35:15,883][123614] Updated weights for policy 1, policy_version 25920 (0.0008) [2023-10-10 17:35:16,680][123582] Updated weights for policy 0, policy_version 25953 (0.0007) [2023-10-10 17:35:17,052][123582] Updated weights for policy 0, policy_version 25963 (0.0010) [2023-10-10 17:35:17,420][123582] Updated weights for policy 0, policy_version 25973 (0.0008) [2023-10-10 17:35:17,788][123582] Updated weights for policy 0, policy_version 25983 (0.0008) [2023-10-10 17:35:18,788][122664] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 53149696. Throughput: 0: 1821.3, 1: 1796.8. Samples: 13295502. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:35:18,788][122664] Avg episode reward: [(0, '31.040'), (1, '28.000')] [2023-10-10 17:35:19,517][123614] Updated weights for policy 1, policy_version 25930 (0.0008) [2023-10-10 17:35:19,879][123614] Updated weights for policy 1, policy_version 25940 (0.0007) [2023-10-10 17:35:20,251][123614] Updated weights for policy 1, policy_version 25950 (0.0009) [2023-10-10 17:35:21,492][123582] Updated weights for policy 0, policy_version 25993 (0.0010) [2023-10-10 17:35:21,865][123582] Updated weights for policy 0, policy_version 26003 (0.0009) [2023-10-10 17:35:22,229][123582] Updated weights for policy 0, policy_version 26013 (0.0008) [2023-10-10 17:35:23,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 53215232. Throughput: 0: 1820.8, 1: 1813.4. Samples: 13317676. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:35:23,789][122664] Avg episode reward: [(0, '34.430'), (1, '29.630')] [2023-10-10 17:35:23,848][123614] Updated weights for policy 1, policy_version 25960 (0.0007) [2023-10-10 17:35:24,222][123614] Updated weights for policy 1, policy_version 25970 (0.0009) [2023-10-10 17:35:24,598][123614] Updated weights for policy 1, policy_version 25980 (0.0010) [2023-10-10 17:35:26,009][123582] Updated weights for policy 0, policy_version 26023 (0.0008) [2023-10-10 17:35:26,381][123582] Updated weights for policy 0, policy_version 26033 (0.0007) [2023-10-10 17:35:26,761][123582] Updated weights for policy 0, policy_version 26043 (0.0007) [2023-10-10 17:35:28,387][123614] Updated weights for policy 1, policy_version 25990 (0.0008) [2023-10-10 17:35:28,768][123614] Updated weights for policy 1, policy_version 26000 (0.0008) [2023-10-10 17:35:28,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 53280768. Throughput: 0: 1824.1, 1: 1797.6. Samples: 13328436. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:35:28,788][122664] Avg episode reward: [(0, '34.590'), (1, '30.230')] [2023-10-10 17:35:29,140][123614] Updated weights for policy 1, policy_version 26010 (0.0008) [2023-10-10 17:35:30,543][123582] Updated weights for policy 0, policy_version 26053 (0.0008) [2023-10-10 17:35:30,902][123582] Updated weights for policy 0, policy_version 26063 (0.0009) [2023-10-10 17:35:31,275][123582] Updated weights for policy 0, policy_version 26073 (0.0010) [2023-10-10 17:35:32,695][123614] Updated weights for policy 1, policy_version 26020 (0.0008) [2023-10-10 17:35:33,069][123614] Updated weights for policy 1, policy_version 26030 (0.0009) [2023-10-10 17:35:33,430][123614] Updated weights for policy 1, policy_version 26040 (0.0010) [2023-10-10 17:35:33,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 53379072. Throughput: 0: 1806.9, 1: 1814.7. Samples: 13350058. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:35:33,789][122664] Avg episode reward: [(0, '35.090'), (1, '32.850')] [2023-10-10 17:35:35,003][123582] Updated weights for policy 0, policy_version 26083 (0.0010) [2023-10-10 17:35:35,395][123582] Updated weights for policy 0, policy_version 26093 (0.0008) [2023-10-10 17:35:35,752][123582] Updated weights for policy 0, policy_version 26103 (0.0007) [2023-10-10 17:35:36,937][123614] Updated weights for policy 1, policy_version 26050 (0.0009) [2023-10-10 17:35:37,314][123614] Updated weights for policy 1, policy_version 26060 (0.0008) [2023-10-10 17:35:37,680][123614] Updated weights for policy 1, policy_version 26070 (0.0008) [2023-10-10 17:35:38,044][123614] Updated weights for policy 1, policy_version 26080 (0.0010) [2023-10-10 17:35:38,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 53444608. Throughput: 0: 1809.6, 1: 1811.5. Samples: 13371724. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:35:38,789][122664] Avg episode reward: [(0, '35.180'), (1, '33.720')] [2023-10-10 17:35:39,339][123582] Updated weights for policy 0, policy_version 26113 (0.0007) [2023-10-10 17:35:39,707][123582] Updated weights for policy 0, policy_version 26123 (0.0010) [2023-10-10 17:35:40,087][123582] Updated weights for policy 0, policy_version 26133 (0.0010) [2023-10-10 17:35:40,446][123582] Updated weights for policy 0, policy_version 26143 (0.0010) [2023-10-10 17:35:41,862][123614] Updated weights for policy 1, policy_version 26090 (0.0010) [2023-10-10 17:35:42,240][123614] Updated weights for policy 1, policy_version 26100 (0.0009) [2023-10-10 17:35:42,620][123614] Updated weights for policy 1, policy_version 26110 (0.0007) [2023-10-10 17:35:43,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 53510144. Throughput: 0: 1803.9, 1: 1813.3. Samples: 13382598. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-10 17:35:43,789][122664] Avg episode reward: [(0, '35.060'), (1, '36.150')] [2023-10-10 17:35:44,262][123582] Updated weights for policy 0, policy_version 26153 (0.0009) [2023-10-10 17:35:44,635][123582] Updated weights for policy 0, policy_version 26163 (0.0009) [2023-10-10 17:35:45,006][123582] Updated weights for policy 0, policy_version 26173 (0.0010) [2023-10-10 17:35:46,172][123614] Updated weights for policy 1, policy_version 26120 (0.0011) [2023-10-10 17:35:46,552][123614] Updated weights for policy 1, policy_version 26130 (0.0009) [2023-10-10 17:35:46,915][123614] Updated weights for policy 1, policy_version 26140 (0.0010) [2023-10-10 17:35:48,588][123582] Updated weights for policy 0, policy_version 26183 (0.0007) [2023-10-10 17:35:48,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 53575680. Throughput: 0: 1807.8, 1: 1814.5. Samples: 13404152. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-10 17:35:48,789][122664] Avg episode reward: [(0, '37.940'), (1, '35.710')] [2023-10-10 17:35:48,967][123582] Updated weights for policy 0, policy_version 26193 (0.0009) [2023-10-10 17:35:49,341][123582] Updated weights for policy 0, policy_version 26203 (0.0009) [2023-10-10 17:35:50,784][123614] Updated weights for policy 1, policy_version 26150 (0.0009) [2023-10-10 17:35:51,150][123614] Updated weights for policy 1, policy_version 26160 (0.0009) [2023-10-10 17:35:51,517][123614] Updated weights for policy 1, policy_version 26170 (0.0008) [2023-10-10 17:35:52,922][123582] Updated weights for policy 0, policy_version 26213 (0.0008) [2023-10-10 17:35:53,295][123582] Updated weights for policy 0, policy_version 26223 (0.0007) [2023-10-10 17:35:53,677][123582] Updated weights for policy 0, policy_version 26233 (0.0009) [2023-10-10 17:35:53,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 53641216. Throughput: 0: 1813.8, 1: 1811.5. Samples: 13426098. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-10 17:35:53,788][122664] Avg episode reward: [(0, '39.110'), (1, '34.370')] [2023-10-10 17:35:53,797][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000026176_26804224.pth... [2023-10-10 17:35:53,831][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000024480_25067520.pth [2023-10-10 17:35:53,926][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000026240_26869760.pth... [2023-10-10 17:35:53,955][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000024544_25133056.pth [2023-10-10 17:35:53,958][123247] Saving new best policy, reward=39.110! [2023-10-10 17:35:55,180][123614] Updated weights for policy 1, policy_version 26180 (0.0009) [2023-10-10 17:35:55,555][123614] Updated weights for policy 1, policy_version 26190 (0.0007) [2023-10-10 17:35:55,927][123614] Updated weights for policy 1, policy_version 26200 (0.0010) [2023-10-10 17:35:57,298][123582] Updated weights for policy 0, policy_version 26243 (0.0008) [2023-10-10 17:35:57,669][123582] Updated weights for policy 0, policy_version 26253 (0.0008) [2023-10-10 17:35:58,039][123582] Updated weights for policy 0, policy_version 26263 (0.0007) [2023-10-10 17:35:58,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 53739520. Throughput: 0: 1811.7, 1: 1814.8. Samples: 13436998. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-10 17:35:58,789][122664] Avg episode reward: [(0, '38.850'), (1, '34.800')] [2023-10-10 17:35:59,642][123614] Updated weights for policy 1, policy_version 26210 (0.0008) [2023-10-10 17:36:00,014][123614] Updated weights for policy 1, policy_version 26220 (0.0010) [2023-10-10 17:36:00,382][123614] Updated weights for policy 1, policy_version 26230 (0.0010) [2023-10-10 17:36:00,750][123614] Updated weights for policy 1, policy_version 26240 (0.0008) [2023-10-10 17:36:01,752][123582] Updated weights for policy 0, policy_version 26273 (0.0007) [2023-10-10 17:36:02,120][123582] Updated weights for policy 0, policy_version 26283 (0.0007) [2023-10-10 17:36:02,498][123582] Updated weights for policy 0, policy_version 26293 (0.0008) [2023-10-10 17:36:02,870][123582] Updated weights for policy 0, policy_version 26303 (0.0010) [2023-10-10 17:36:03,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 53805056. Throughput: 0: 1818.4, 1: 1812.8. Samples: 13458902. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:36:03,789][122664] Avg episode reward: [(0, '38.890'), (1, '33.960')] [2023-10-10 17:36:04,409][123614] Updated weights for policy 1, policy_version 26250 (0.0010) [2023-10-10 17:36:04,779][123614] Updated weights for policy 1, policy_version 26260 (0.0010) [2023-10-10 17:36:05,152][123614] Updated weights for policy 1, policy_version 26270 (0.0011) [2023-10-10 17:36:06,481][123582] Updated weights for policy 0, policy_version 26313 (0.0008) [2023-10-10 17:36:06,852][123582] Updated weights for policy 0, policy_version 26323 (0.0010) [2023-10-10 17:36:07,230][123582] Updated weights for policy 0, policy_version 26333 (0.0007) [2023-10-10 17:36:08,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 53870592. Throughput: 0: 1817.6, 1: 1812.1. Samples: 13481014. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:36:08,789][122664] Avg episode reward: [(0, '38.380'), (1, '32.870')] [2023-10-10 17:36:08,917][123614] Updated weights for policy 1, policy_version 26280 (0.0009) [2023-10-10 17:36:09,276][123614] Updated weights for policy 1, policy_version 26290 (0.0008) [2023-10-10 17:36:09,657][123614] Updated weights for policy 1, policy_version 26300 (0.0009) [2023-10-10 17:36:10,934][123582] Updated weights for policy 0, policy_version 26343 (0.0007) [2023-10-10 17:36:11,318][123582] Updated weights for policy 0, policy_version 26353 (0.0008) [2023-10-10 17:36:11,688][123582] Updated weights for policy 0, policy_version 26363 (0.0008) [2023-10-10 17:36:13,490][123614] Updated weights for policy 1, policy_version 26310 (0.0008) [2023-10-10 17:36:13,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 53936128. Throughput: 0: 1818.4, 1: 1808.9. Samples: 13491662. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:36:13,788][122664] Avg episode reward: [(0, '38.310'), (1, '30.680')] [2023-10-10 17:36:13,875][123614] Updated weights for policy 1, policy_version 26320 (0.0007) [2023-10-10 17:36:14,246][123614] Updated weights for policy 1, policy_version 26330 (0.0007) [2023-10-10 17:36:15,480][123582] Updated weights for policy 0, policy_version 26373 (0.0009) [2023-10-10 17:36:15,856][123582] Updated weights for policy 0, policy_version 26383 (0.0008) [2023-10-10 17:36:16,229][123582] Updated weights for policy 0, policy_version 26393 (0.0008) [2023-10-10 17:36:17,974][123614] Updated weights for policy 1, policy_version 26340 (0.0008) [2023-10-10 17:36:18,348][123614] Updated weights for policy 1, policy_version 26350 (0.0010) [2023-10-10 17:36:18,726][123614] Updated weights for policy 1, policy_version 26360 (0.0009) [2023-10-10 17:36:18,788][122664] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 54001664. Throughput: 0: 1824.9, 1: 1810.6. Samples: 13513656. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:36:18,788][122664] Avg episode reward: [(0, '38.900'), (1, '31.110')] [2023-10-10 17:36:19,880][123582] Updated weights for policy 0, policy_version 26403 (0.0010) [2023-10-10 17:36:20,270][123582] Updated weights for policy 0, policy_version 26413 (0.0008) [2023-10-10 17:36:20,642][123582] Updated weights for policy 0, policy_version 26423 (0.0007) [2023-10-10 17:36:22,484][123614] Updated weights for policy 1, policy_version 26370 (0.0010) [2023-10-10 17:36:22,846][123614] Updated weights for policy 1, policy_version 26380 (0.0010) [2023-10-10 17:36:23,214][123614] Updated weights for policy 1, policy_version 26390 (0.0011) [2023-10-10 17:36:23,581][123614] Updated weights for policy 1, policy_version 26400 (0.0009) [2023-10-10 17:36:23,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 54099968. Throughput: 0: 1828.0, 1: 1800.4. Samples: 13535004. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:36:23,789][122664] Avg episode reward: [(0, '38.870'), (1, '31.920')] [2023-10-10 17:36:24,068][123582] Updated weights for policy 0, policy_version 26433 (0.0011) [2023-10-10 17:36:24,438][123582] Updated weights for policy 0, policy_version 26443 (0.0009) [2023-10-10 17:36:24,814][123582] Updated weights for policy 0, policy_version 26453 (0.0008) [2023-10-10 17:36:25,189][123582] Updated weights for policy 0, policy_version 26463 (0.0007) [2023-10-10 17:36:27,359][123614] Updated weights for policy 1, policy_version 26410 (0.0009) [2023-10-10 17:36:27,732][123614] Updated weights for policy 1, policy_version 26420 (0.0008) [2023-10-10 17:36:28,098][123614] Updated weights for policy 1, policy_version 26430 (0.0010) [2023-10-10 17:36:28,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 54165504. Throughput: 0: 1831.3, 1: 1807.2. Samples: 13546328. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-10 17:36:28,789][122664] Avg episode reward: [(0, '37.120'), (1, '32.280')] [2023-10-10 17:36:28,810][123582] Updated weights for policy 0, policy_version 26473 (0.0007) [2023-10-10 17:36:29,181][123582] Updated weights for policy 0, policy_version 26483 (0.0007) [2023-10-10 17:36:29,553][123582] Updated weights for policy 0, policy_version 26493 (0.0009) [2023-10-10 17:36:31,870][123614] Updated weights for policy 1, policy_version 26440 (0.0010) [2023-10-10 17:36:32,239][123614] Updated weights for policy 1, policy_version 26450 (0.0009) [2023-10-10 17:36:32,615][123614] Updated weights for policy 1, policy_version 26460 (0.0007) [2023-10-10 17:36:33,217][123582] Updated weights for policy 0, policy_version 26503 (0.0007) [2023-10-10 17:36:33,582][123582] Updated weights for policy 0, policy_version 26513 (0.0008) [2023-10-10 17:36:33,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 54231040. Throughput: 0: 1834.8, 1: 1797.1. Samples: 13567586. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-10 17:36:33,788][122664] Avg episode reward: [(0, '32.950'), (1, '34.800')] [2023-10-10 17:36:33,967][123582] Updated weights for policy 0, policy_version 26523 (0.0010) [2023-10-10 17:36:36,488][123614] Updated weights for policy 1, policy_version 26470 (0.0010) [2023-10-10 17:36:36,860][123614] Updated weights for policy 1, policy_version 26480 (0.0007) [2023-10-10 17:36:37,219][123614] Updated weights for policy 1, policy_version 26490 (0.0009) [2023-10-10 17:36:37,749][123582] Updated weights for policy 0, policy_version 26533 (0.0010) [2023-10-10 17:36:38,118][123582] Updated weights for policy 0, policy_version 26543 (0.0010) [2023-10-10 17:36:38,495][123582] Updated weights for policy 0, policy_version 26553 (0.0009) [2023-10-10 17:36:38,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 54329344. Throughput: 0: 1826.0, 1: 1796.2. Samples: 13589096. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-10 17:36:38,788][122664] Avg episode reward: [(0, '34.110'), (1, '32.870')] [2023-10-10 17:36:40,941][123614] Updated weights for policy 1, policy_version 26500 (0.0011) [2023-10-10 17:36:41,307][123614] Updated weights for policy 1, policy_version 26510 (0.0009) [2023-10-10 17:36:41,688][123614] Updated weights for policy 1, policy_version 26520 (0.0011) [2023-10-10 17:36:42,225][123582] Updated weights for policy 0, policy_version 26563 (0.0007) [2023-10-10 17:36:42,608][123582] Updated weights for policy 0, policy_version 26573 (0.0009) [2023-10-10 17:36:42,974][123582] Updated weights for policy 0, policy_version 26583 (0.0009) [2023-10-10 17:36:43,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 54394880. Throughput: 0: 1828.6, 1: 1801.1. Samples: 13600332. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-10 17:36:43,788][122664] Avg episode reward: [(0, '36.350'), (1, '33.120')] [2023-10-10 17:36:45,533][123614] Updated weights for policy 1, policy_version 26530 (0.0011) [2023-10-10 17:36:45,900][123614] Updated weights for policy 1, policy_version 26540 (0.0007) [2023-10-10 17:36:46,266][123614] Updated weights for policy 1, policy_version 26550 (0.0008) [2023-10-10 17:36:46,641][123614] Updated weights for policy 1, policy_version 26560 (0.0008) [2023-10-10 17:36:46,689][123582] Updated weights for policy 0, policy_version 26593 (0.0007) [2023-10-10 17:36:47,045][123582] Updated weights for policy 0, policy_version 26603 (0.0007) [2023-10-10 17:36:47,418][123582] Updated weights for policy 0, policy_version 26613 (0.0009) [2023-10-10 17:36:47,788][123582] Updated weights for policy 0, policy_version 26623 (0.0007) [2023-10-10 17:36:48,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 54460416. Throughput: 0: 1826.1, 1: 1790.1. Samples: 13621630. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-10-10 17:36:48,789][122664] Avg episode reward: [(0, '35.550'), (1, '33.500')] [2023-10-10 17:36:50,396][123614] Updated weights for policy 1, policy_version 26570 (0.0009) [2023-10-10 17:36:50,767][123614] Updated weights for policy 1, policy_version 26580 (0.0008) [2023-10-10 17:36:51,138][123614] Updated weights for policy 1, policy_version 26590 (0.0008) [2023-10-10 17:36:51,458][123582] Updated weights for policy 0, policy_version 26633 (0.0009) [2023-10-10 17:36:51,841][123582] Updated weights for policy 0, policy_version 26643 (0.0008) [2023-10-10 17:36:52,221][123582] Updated weights for policy 0, policy_version 26653 (0.0007) [2023-10-10 17:36:53,788][122664] Fps is (10 sec: 13106.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 54525952. Throughput: 0: 1829.2, 1: 1792.0. Samples: 13643972. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-10-10 17:36:53,789][122664] Avg episode reward: [(0, '35.010'), (1, '31.800')] [2023-10-10 17:36:54,753][123614] Updated weights for policy 1, policy_version 26600 (0.0008) [2023-10-10 17:36:55,132][123614] Updated weights for policy 1, policy_version 26610 (0.0009) [2023-10-10 17:36:55,502][123614] Updated weights for policy 1, policy_version 26620 (0.0008) [2023-10-10 17:36:55,715][123582] Updated weights for policy 0, policy_version 26663 (0.0009) [2023-10-10 17:36:56,090][123582] Updated weights for policy 0, policy_version 26673 (0.0009) [2023-10-10 17:36:56,467][123582] Updated weights for policy 0, policy_version 26683 (0.0008) [2023-10-10 17:36:58,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 54591488. Throughput: 0: 1824.3, 1: 1790.8. Samples: 13654338. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-10-10 17:36:58,788][122664] Avg episode reward: [(0, '35.340'), (1, '29.150')] [2023-10-10 17:36:59,232][123614] Updated weights for policy 1, policy_version 26630 (0.0008) [2023-10-10 17:36:59,591][123614] Updated weights for policy 1, policy_version 26640 (0.0009) [2023-10-10 17:36:59,961][123614] Updated weights for policy 1, policy_version 26650 (0.0008) [2023-10-10 17:37:00,137][123582] Updated weights for policy 0, policy_version 26693 (0.0010) [2023-10-10 17:37:00,521][123582] Updated weights for policy 0, policy_version 26703 (0.0009) [2023-10-10 17:37:00,891][123582] Updated weights for policy 0, policy_version 26713 (0.0007) [2023-10-10 17:37:03,741][123614] Updated weights for policy 1, policy_version 26660 (0.0007) [2023-10-10 17:37:03,788][122664] Fps is (10 sec: 13107.8, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 54657024. Throughput: 0: 1824.6, 1: 1791.7. Samples: 13676388. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-10-10 17:37:03,788][122664] Avg episode reward: [(0, '35.510'), (1, '30.650')] [2023-10-10 17:37:04,107][123614] Updated weights for policy 1, policy_version 26670 (0.0009) [2023-10-10 17:37:04,481][123614] Updated weights for policy 1, policy_version 26680 (0.0008) [2023-10-10 17:37:04,711][123582] Updated weights for policy 0, policy_version 26723 (0.0010) [2023-10-10 17:37:05,108][123582] Updated weights for policy 0, policy_version 26733 (0.0007) [2023-10-10 17:37:05,474][123582] Updated weights for policy 0, policy_version 26743 (0.0007) [2023-10-10 17:37:08,356][123614] Updated weights for policy 1, policy_version 26690 (0.0008) [2023-10-10 17:37:08,723][123614] Updated weights for policy 1, policy_version 26700 (0.0008) [2023-10-10 17:37:08,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 54722560. Throughput: 0: 1816.5, 1: 1808.1. Samples: 13698110. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-10-10 17:37:08,789][122664] Avg episode reward: [(0, '35.610'), (1, '30.510')] [2023-10-10 17:37:09,108][123614] Updated weights for policy 1, policy_version 26710 (0.0008) [2023-10-10 17:37:09,116][123582] Updated weights for policy 0, policy_version 26753 (0.0007) [2023-10-10 17:37:09,468][123614] Updated weights for policy 1, policy_version 26720 (0.0007) [2023-10-10 17:37:09,487][123582] Updated weights for policy 0, policy_version 26763 (0.0009) [2023-10-10 17:37:09,865][123582] Updated weights for policy 0, policy_version 26773 (0.0012) [2023-10-10 17:37:10,244][123582] Updated weights for policy 0, policy_version 26783 (0.0010) [2023-10-10 17:37:12,929][123614] Updated weights for policy 1, policy_version 26730 (0.0010) [2023-10-10 17:37:13,294][123614] Updated weights for policy 1, policy_version 26740 (0.0008) [2023-10-10 17:37:13,659][123614] Updated weights for policy 1, policy_version 26750 (0.0008) [2023-10-10 17:37:13,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 54820864. Throughput: 0: 1813.6, 1: 1791.7. Samples: 13708564. Policy #0 lag: (min: 28.0, avg: 31.8, max: 60.0) [2023-10-10 17:37:13,789][122664] Avg episode reward: [(0, '37.430'), (1, '30.390')] [2023-10-10 17:37:13,806][123582] Updated weights for policy 0, policy_version 26793 (0.0009) [2023-10-10 17:37:14,175][123582] Updated weights for policy 0, policy_version 26803 (0.0009) [2023-10-10 17:37:14,548][123582] Updated weights for policy 0, policy_version 26813 (0.0009) [2023-10-10 17:37:17,264][123614] Updated weights for policy 1, policy_version 26760 (0.0010) [2023-10-10 17:37:17,626][123614] Updated weights for policy 1, policy_version 26770 (0.0008) [2023-10-10 17:37:18,000][123614] Updated weights for policy 1, policy_version 26780 (0.0007) [2023-10-10 17:37:18,316][123582] Updated weights for policy 0, policy_version 26823 (0.0008) [2023-10-10 17:37:18,700][123582] Updated weights for policy 0, policy_version 26833 (0.0011) [2023-10-10 17:37:18,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 54886400. Throughput: 0: 1813.5, 1: 1808.2. Samples: 13730566. Policy #0 lag: (min: 28.0, avg: 31.8, max: 60.0) [2023-10-10 17:37:18,789][122664] Avg episode reward: [(0, '39.800'), (1, '32.530')] [2023-10-10 17:37:19,069][123582] Updated weights for policy 0, policy_version 26843 (0.0007) [2023-10-10 17:37:19,251][123247] Saving new best policy, reward=39.800! [2023-10-10 17:37:21,655][123614] Updated weights for policy 1, policy_version 26790 (0.0010) [2023-10-10 17:37:22,026][123614] Updated weights for policy 1, policy_version 26800 (0.0009) [2023-10-10 17:37:22,401][123614] Updated weights for policy 1, policy_version 26810 (0.0007) [2023-10-10 17:37:22,691][123582] Updated weights for policy 0, policy_version 26853 (0.0009) [2023-10-10 17:37:23,057][123582] Updated weights for policy 0, policy_version 26863 (0.0009) [2023-10-10 17:37:23,428][123582] Updated weights for policy 0, policy_version 26873 (0.0008) [2023-10-10 17:37:23,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 54984704. Throughput: 0: 1815.2, 1: 1798.1. Samples: 13751696. Policy #0 lag: (min: 28.0, avg: 31.8, max: 60.0) [2023-10-10 17:37:23,789][122664] Avg episode reward: [(0, '41.170'), (1, '32.890')] [2023-10-10 17:37:23,801][123247] Saving new best policy, reward=41.170! [2023-10-10 17:37:26,198][123614] Updated weights for policy 1, policy_version 26820 (0.0008) [2023-10-10 17:37:26,573][123614] Updated weights for policy 1, policy_version 26830 (0.0008) [2023-10-10 17:37:26,947][123614] Updated weights for policy 1, policy_version 26840 (0.0007) [2023-10-10 17:37:27,208][123582] Updated weights for policy 0, policy_version 26883 (0.0009) [2023-10-10 17:37:27,587][123582] Updated weights for policy 0, policy_version 26893 (0.0008) [2023-10-10 17:37:27,959][123582] Updated weights for policy 0, policy_version 26903 (0.0007) [2023-10-10 17:37:28,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 55050240. Throughput: 0: 1815.3, 1: 1808.0. Samples: 13763382. Policy #0 lag: (min: 28.0, avg: 31.8, max: 60.0) [2023-10-10 17:37:28,789][122664] Avg episode reward: [(0, '41.200'), (1, '31.820')] [2023-10-10 17:37:28,790][123247] Saving new best policy, reward=41.200! [2023-10-10 17:37:30,715][123614] Updated weights for policy 1, policy_version 26850 (0.0008) [2023-10-10 17:37:31,079][123614] Updated weights for policy 1, policy_version 26860 (0.0009) [2023-10-10 17:37:31,453][123614] Updated weights for policy 1, policy_version 26870 (0.0007) [2023-10-10 17:37:31,627][123582] Updated weights for policy 0, policy_version 26913 (0.0009) [2023-10-10 17:37:31,819][123614] Updated weights for policy 1, policy_version 26880 (0.0007) [2023-10-10 17:37:32,004][123582] Updated weights for policy 0, policy_version 26923 (0.0007) [2023-10-10 17:37:32,371][123582] Updated weights for policy 0, policy_version 26933 (0.0007) [2023-10-10 17:37:32,740][123582] Updated weights for policy 0, policy_version 26943 (0.0008) [2023-10-10 17:37:33,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 55115776. Throughput: 0: 1816.0, 1: 1805.6. Samples: 13784602. Policy #0 lag: (min: 5.0, avg: 9.0, max: 37.0) [2023-10-10 17:37:33,789][122664] Avg episode reward: [(0, '41.560'), (1, '31.450')] [2023-10-10 17:37:33,790][123247] Saving new best policy, reward=41.560! [2023-10-10 17:37:35,612][123614] Updated weights for policy 1, policy_version 26890 (0.0009) [2023-10-10 17:37:35,989][123614] Updated weights for policy 1, policy_version 26900 (0.0009) [2023-10-10 17:37:36,347][123614] Updated weights for policy 1, policy_version 26910 (0.0008) [2023-10-10 17:37:36,586][123582] Updated weights for policy 0, policy_version 26953 (0.0007) [2023-10-10 17:37:36,965][123582] Updated weights for policy 0, policy_version 26963 (0.0007) [2023-10-10 17:37:37,328][123582] Updated weights for policy 0, policy_version 26973 (0.0007) [2023-10-10 17:37:38,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 55181312. Throughput: 0: 1811.1, 1: 1808.4. Samples: 13806848. Policy #0 lag: (min: 5.0, avg: 9.0, max: 37.0) [2023-10-10 17:37:38,788][122664] Avg episode reward: [(0, '39.950'), (1, '30.850')] [2023-10-10 17:37:39,988][123614] Updated weights for policy 1, policy_version 26920 (0.0008) [2023-10-10 17:37:40,354][123614] Updated weights for policy 1, policy_version 26930 (0.0009) [2023-10-10 17:37:40,722][123614] Updated weights for policy 1, policy_version 26940 (0.0011) [2023-10-10 17:37:40,965][123582] Updated weights for policy 0, policy_version 26983 (0.0007) [2023-10-10 17:37:41,344][123582] Updated weights for policy 0, policy_version 26993 (0.0009) [2023-10-10 17:37:41,714][123582] Updated weights for policy 0, policy_version 27003 (0.0008) [2023-10-10 17:37:43,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 55246848. Throughput: 0: 1816.8, 1: 1807.8. Samples: 13817444. Policy #0 lag: (min: 5.0, avg: 9.0, max: 37.0) [2023-10-10 17:37:43,788][122664] Avg episode reward: [(0, '37.690'), (1, '30.450')] [2023-10-10 17:37:44,524][123614] Updated weights for policy 1, policy_version 26950 (0.0008) [2023-10-10 17:37:44,903][123614] Updated weights for policy 1, policy_version 26960 (0.0009) [2023-10-10 17:37:45,269][123614] Updated weights for policy 1, policy_version 26970 (0.0008) [2023-10-10 17:37:45,377][123582] Updated weights for policy 0, policy_version 27013 (0.0009) [2023-10-10 17:37:45,750][123582] Updated weights for policy 0, policy_version 27023 (0.0009) [2023-10-10 17:37:46,131][123582] Updated weights for policy 0, policy_version 27033 (0.0008) [2023-10-10 17:37:48,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 55312384. Throughput: 0: 1815.6, 1: 1804.8. Samples: 13839306. Policy #0 lag: (min: 5.0, avg: 9.0, max: 37.0) [2023-10-10 17:37:48,789][122664] Avg episode reward: [(0, '38.480'), (1, '31.230')] [2023-10-10 17:37:49,055][123614] Updated weights for policy 1, policy_version 26980 (0.0007) [2023-10-10 17:37:49,425][123614] Updated weights for policy 1, policy_version 26990 (0.0007) [2023-10-10 17:37:49,790][123614] Updated weights for policy 1, policy_version 27000 (0.0008) [2023-10-10 17:37:49,828][123582] Updated weights for policy 0, policy_version 27043 (0.0010) [2023-10-10 17:37:50,222][123582] Updated weights for policy 0, policy_version 27053 (0.0009) [2023-10-10 17:37:50,588][123582] Updated weights for policy 0, policy_version 27063 (0.0008) [2023-10-10 17:37:53,594][123614] Updated weights for policy 1, policy_version 27010 (0.0008) [2023-10-10 17:37:53,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 55377920. Throughput: 0: 1816.7, 1: 1818.9. Samples: 13861710. Policy #0 lag: (min: 12.0, avg: 15.3, max: 44.0) [2023-10-10 17:37:53,789][122664] Avg episode reward: [(0, '40.410'), (1, '30.000')] [2023-10-10 17:37:53,801][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000027072_27721728.pth... [2023-10-10 17:37:53,835][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000025376_25985024.pth [2023-10-10 17:37:53,977][123614] Updated weights for policy 1, policy_version 27020 (0.0009) [2023-10-10 17:37:54,288][123582] Updated weights for policy 0, policy_version 27073 (0.0010) [2023-10-10 17:37:54,347][123614] Updated weights for policy 1, policy_version 27030 (0.0010) [2023-10-10 17:37:54,662][123582] Updated weights for policy 0, policy_version 27083 (0.0008) [2023-10-10 17:37:54,705][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000027040_27688960.pth... [2023-10-10 17:37:54,707][123614] Updated weights for policy 1, policy_version 27040 (0.0009) [2023-10-10 17:37:54,738][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000025344_25952256.pth [2023-10-10 17:37:55,033][123582] Updated weights for policy 0, policy_version 27093 (0.0010) [2023-10-10 17:37:55,405][123582] Updated weights for policy 0, policy_version 27103 (0.0008) [2023-10-10 17:37:58,369][123614] Updated weights for policy 1, policy_version 27050 (0.0009) [2023-10-10 17:37:58,735][123614] Updated weights for policy 1, policy_version 27060 (0.0010) [2023-10-10 17:37:58,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 55443456. Throughput: 0: 1820.9, 1: 1810.0. Samples: 13871958. Policy #0 lag: (min: 12.0, avg: 15.3, max: 44.0) [2023-10-10 17:37:58,788][122664] Avg episode reward: [(0, '36.030'), (1, '31.410')] [2023-10-10 17:37:59,088][123582] Updated weights for policy 0, policy_version 27113 (0.0007) [2023-10-10 17:37:59,110][123614] Updated weights for policy 1, policy_version 27070 (0.0007) [2023-10-10 17:37:59,450][123582] Updated weights for policy 0, policy_version 27123 (0.0010) [2023-10-10 17:37:59,829][123582] Updated weights for policy 0, policy_version 27133 (0.0009) [2023-10-10 17:38:02,693][123614] Updated weights for policy 1, policy_version 27080 (0.0009) [2023-10-10 17:38:03,066][123614] Updated weights for policy 1, policy_version 27090 (0.0007) [2023-10-10 17:38:03,428][123614] Updated weights for policy 1, policy_version 27100 (0.0007) [2023-10-10 17:38:03,499][123582] Updated weights for policy 0, policy_version 27143 (0.0008) [2023-10-10 17:38:03,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 55541760. Throughput: 0: 1823.8, 1: 1822.0. Samples: 13894626. Policy #0 lag: (min: 12.0, avg: 15.3, max: 44.0) [2023-10-10 17:38:03,788][122664] Avg episode reward: [(0, '36.710'), (1, '30.490')] [2023-10-10 17:38:03,875][123582] Updated weights for policy 0, policy_version 27153 (0.0009) [2023-10-10 17:38:04,255][123582] Updated weights for policy 0, policy_version 27163 (0.0008) [2023-10-10 17:38:07,427][123614] Updated weights for policy 1, policy_version 27110 (0.0007) [2023-10-10 17:38:07,795][123614] Updated weights for policy 1, policy_version 27120 (0.0009) [2023-10-10 17:38:07,875][123582] Updated weights for policy 0, policy_version 27173 (0.0009) [2023-10-10 17:38:08,163][123614] Updated weights for policy 1, policy_version 27130 (0.0008) [2023-10-10 17:38:08,240][123582] Updated weights for policy 0, policy_version 27183 (0.0008) [2023-10-10 17:38:08,618][123582] Updated weights for policy 0, policy_version 27193 (0.0008) [2023-10-10 17:38:08,788][122664] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 55607296. Throughput: 0: 1830.3, 1: 1802.5. Samples: 13915176. Policy #0 lag: (min: 12.0, avg: 15.3, max: 44.0) [2023-10-10 17:38:08,789][122664] Avg episode reward: [(0, '30.870'), (1, '31.690')] [2023-10-10 17:38:11,858][123614] Updated weights for policy 1, policy_version 27140 (0.0008) [2023-10-10 17:38:12,210][123582] Updated weights for policy 0, policy_version 27203 (0.0007) [2023-10-10 17:38:12,232][123614] Updated weights for policy 1, policy_version 27150 (0.0008) [2023-10-10 17:38:12,575][123582] Updated weights for policy 0, policy_version 27213 (0.0008) [2023-10-10 17:38:12,601][123614] Updated weights for policy 1, policy_version 27160 (0.0008) [2023-10-10 17:38:12,956][123582] Updated weights for policy 0, policy_version 27223 (0.0008) [2023-10-10 17:38:13,788][122664] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 55705600. Throughput: 0: 1825.4, 1: 1818.8. Samples: 13927372. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:38:13,789][122664] Avg episode reward: [(0, '29.780'), (1, '34.500')] [2023-10-10 17:38:16,414][123614] Updated weights for policy 1, policy_version 27170 (0.0008) [2023-10-10 17:38:16,588][123582] Updated weights for policy 0, policy_version 27233 (0.0011) [2023-10-10 17:38:16,787][123614] Updated weights for policy 1, policy_version 27180 (0.0009) [2023-10-10 17:38:16,959][123582] Updated weights for policy 0, policy_version 27243 (0.0008) [2023-10-10 17:38:17,157][123614] Updated weights for policy 1, policy_version 27190 (0.0007) [2023-10-10 17:38:17,332][123582] Updated weights for policy 0, policy_version 27253 (0.0007) [2023-10-10 17:38:17,524][123614] Updated weights for policy 1, policy_version 27200 (0.0007) [2023-10-10 17:38:17,695][123582] Updated weights for policy 0, policy_version 27263 (0.0007) [2023-10-10 17:38:18,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 55771136. Throughput: 0: 1822.7, 1: 1795.7. Samples: 13947430. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:38:18,789][122664] Avg episode reward: [(0, '31.970'), (1, '37.660')] [2023-10-10 17:38:21,257][123614] Updated weights for policy 1, policy_version 27210 (0.0008) [2023-10-10 17:38:21,357][123582] Updated weights for policy 0, policy_version 27273 (0.0007) [2023-10-10 17:38:21,633][123614] Updated weights for policy 1, policy_version 27220 (0.0009) [2023-10-10 17:38:21,722][123582] Updated weights for policy 0, policy_version 27283 (0.0009) [2023-10-10 17:38:21,998][123614] Updated weights for policy 1, policy_version 27230 (0.0009) [2023-10-10 17:38:22,089][123582] Updated weights for policy 0, policy_version 27293 (0.0007) [2023-10-10 17:38:23,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 55836672. Throughput: 0: 1825.4, 1: 1789.2. Samples: 13969502. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:38:23,788][122664] Avg episode reward: [(0, '31.740'), (1, '37.980')] [2023-10-10 17:38:25,655][123614] Updated weights for policy 1, policy_version 27240 (0.0008) [2023-10-10 17:38:25,782][123582] Updated weights for policy 0, policy_version 27303 (0.0008) [2023-10-10 17:38:26,027][123614] Updated weights for policy 1, policy_version 27250 (0.0007) [2023-10-10 17:38:26,158][123582] Updated weights for policy 0, policy_version 27313 (0.0007) [2023-10-10 17:38:26,392][123614] Updated weights for policy 1, policy_version 27260 (0.0007) [2023-10-10 17:38:26,530][123582] Updated weights for policy 0, policy_version 27323 (0.0007) [2023-10-10 17:38:28,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 55902208. Throughput: 0: 1819.1, 1: 1790.5. Samples: 13979874. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:38:28,788][122664] Avg episode reward: [(0, '32.840'), (1, '38.170')] [2023-10-10 17:38:30,165][123614] Updated weights for policy 1, policy_version 27270 (0.0008) [2023-10-10 17:38:30,254][123582] Updated weights for policy 0, policy_version 27333 (0.0008) [2023-10-10 17:38:30,536][123614] Updated weights for policy 1, policy_version 27280 (0.0008) [2023-10-10 17:38:30,619][123582] Updated weights for policy 0, policy_version 27343 (0.0009) [2023-10-10 17:38:30,902][123614] Updated weights for policy 1, policy_version 27290 (0.0008) [2023-10-10 17:38:30,992][123582] Updated weights for policy 0, policy_version 27353 (0.0009) [2023-10-10 17:38:33,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 55967744. Throughput: 0: 1825.5, 1: 1795.4. Samples: 14002246. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:38:33,789][122664] Avg episode reward: [(0, '32.640'), (1, '36.250')] [2023-10-10 17:38:34,647][123614] Updated weights for policy 1, policy_version 27300 (0.0007) [2023-10-10 17:38:34,787][123582] Updated weights for policy 0, policy_version 27363 (0.0008) [2023-10-10 17:38:35,053][123614] Updated weights for policy 1, policy_version 27310 (0.0007) [2023-10-10 17:38:35,182][123582] Updated weights for policy 0, policy_version 27373 (0.0007) [2023-10-10 17:38:35,420][123614] Updated weights for policy 1, policy_version 27320 (0.0008) [2023-10-10 17:38:35,549][123582] Updated weights for policy 0, policy_version 27383 (0.0007) [2023-10-10 17:38:38,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 56033280. Throughput: 0: 1820.3, 1: 1797.4. Samples: 14024504. Policy #0 lag: (min: 1.0, avg: 10.8, max: 33.0) [2023-10-10 17:38:38,788][122664] Avg episode reward: [(0, '32.900'), (1, '34.710')] [2023-10-10 17:38:39,030][123614] Updated weights for policy 1, policy_version 27330 (0.0007) [2023-10-10 17:38:39,264][123582] Updated weights for policy 0, policy_version 27393 (0.0007) [2023-10-10 17:38:39,400][123614] Updated weights for policy 1, policy_version 27340 (0.0007) [2023-10-10 17:38:39,627][123582] Updated weights for policy 0, policy_version 27403 (0.0008) [2023-10-10 17:38:39,767][123614] Updated weights for policy 1, policy_version 27350 (0.0008) [2023-10-10 17:38:40,002][123582] Updated weights for policy 0, policy_version 27413 (0.0007) [2023-10-10 17:38:40,139][123614] Updated weights for policy 1, policy_version 27360 (0.0007) [2023-10-10 17:38:40,372][123582] Updated weights for policy 0, policy_version 27423 (0.0007) [2023-10-10 17:38:43,667][123614] Updated weights for policy 1, policy_version 27370 (0.0008) [2023-10-10 17:38:43,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 56098816. Throughput: 0: 1818.6, 1: 1792.6. Samples: 14034462. Policy #0 lag: (min: 1.0, avg: 10.8, max: 33.0) [2023-10-10 17:38:43,788][122664] Avg episode reward: [(0, '33.390'), (1, '35.060')] [2023-10-10 17:38:44,030][123614] Updated weights for policy 1, policy_version 27380 (0.0008) [2023-10-10 17:38:44,179][123582] Updated weights for policy 0, policy_version 27433 (0.0009) [2023-10-10 17:38:44,402][123614] Updated weights for policy 1, policy_version 27390 (0.0007) [2023-10-10 17:38:44,551][123582] Updated weights for policy 0, policy_version 27443 (0.0009) [2023-10-10 17:38:44,927][123582] Updated weights for policy 0, policy_version 27453 (0.0010) [2023-10-10 17:38:48,160][123614] Updated weights for policy 1, policy_version 27400 (0.0008) [2023-10-10 17:38:48,527][123614] Updated weights for policy 1, policy_version 27410 (0.0007) [2023-10-10 17:38:48,585][123582] Updated weights for policy 0, policy_version 27463 (0.0008) [2023-10-10 17:38:48,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 56164352. Throughput: 0: 1813.8, 1: 1807.0. Samples: 14057562. Policy #0 lag: (min: 1.0, avg: 10.8, max: 33.0) [2023-10-10 17:38:48,789][122664] Avg episode reward: [(0, '37.170'), (1, '37.180')] [2023-10-10 17:38:48,889][123614] Updated weights for policy 1, policy_version 27420 (0.0007) [2023-10-10 17:38:48,946][123582] Updated weights for policy 0, policy_version 27473 (0.0008) [2023-10-10 17:38:49,328][123582] Updated weights for policy 0, policy_version 27483 (0.0010) [2023-10-10 17:38:52,927][123614] Updated weights for policy 1, policy_version 27430 (0.0008) [2023-10-10 17:38:52,981][123582] Updated weights for policy 0, policy_version 27493 (0.0010) [2023-10-10 17:38:53,300][123614] Updated weights for policy 1, policy_version 27440 (0.0008) [2023-10-10 17:38:53,350][123582] Updated weights for policy 0, policy_version 27503 (0.0008) [2023-10-10 17:38:53,669][123614] Updated weights for policy 1, policy_version 27450 (0.0008) [2023-10-10 17:38:53,717][123582] Updated weights for policy 0, policy_version 27513 (0.0007) [2023-10-10 17:38:53,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 56229888. Throughput: 0: 1817.0, 1: 1797.7. Samples: 14077838. Policy #0 lag: (min: 1.0, avg: 10.8, max: 33.0) [2023-10-10 17:38:53,789][122664] Avg episode reward: [(0, '38.300'), (1, '34.170')] [2023-10-10 17:38:57,275][123614] Updated weights for policy 1, policy_version 27460 (0.0008) [2023-10-10 17:38:57,596][123582] Updated weights for policy 0, policy_version 27523 (0.0008) [2023-10-10 17:38:57,645][123614] Updated weights for policy 1, policy_version 27470 (0.0007) [2023-10-10 17:38:57,971][123582] Updated weights for policy 0, policy_version 27533 (0.0007) [2023-10-10 17:38:58,011][123614] Updated weights for policy 1, policy_version 27480 (0.0008) [2023-10-10 17:38:58,330][123582] Updated weights for policy 0, policy_version 27543 (0.0010) [2023-10-10 17:38:58,788][122664] Fps is (10 sec: 19661.0, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 56360960. Throughput: 0: 1809.3, 1: 1797.4. Samples: 14089674. Policy #0 lag: (min: 31.0, avg: 32.9, max: 60.0) [2023-10-10 17:38:58,789][122664] Avg episode reward: [(0, '39.650'), (1, '36.000')] [2023-10-10 17:39:01,740][123614] Updated weights for policy 1, policy_version 27490 (0.0007) [2023-10-10 17:39:01,994][123582] Updated weights for policy 0, policy_version 27553 (0.0010) [2023-10-10 17:39:02,120][123614] Updated weights for policy 1, policy_version 27500 (0.0009) [2023-10-10 17:39:02,365][123582] Updated weights for policy 0, policy_version 27563 (0.0007) [2023-10-10 17:39:02,487][123614] Updated weights for policy 1, policy_version 27510 (0.0008) [2023-10-10 17:39:02,734][123582] Updated weights for policy 0, policy_version 27573 (0.0007) [2023-10-10 17:39:02,851][123614] Updated weights for policy 1, policy_version 27520 (0.0008) [2023-10-10 17:39:03,107][123582] Updated weights for policy 0, policy_version 27583 (0.0007) [2023-10-10 17:39:03,788][122664] Fps is (10 sec: 19661.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 56426496. Throughput: 0: 1817.9, 1: 1805.3. Samples: 14110474. Policy #0 lag: (min: 31.0, avg: 32.9, max: 60.0) [2023-10-10 17:39:03,788][122664] Avg episode reward: [(0, '41.150'), (1, '32.160')] [2023-10-10 17:39:06,647][123614] Updated weights for policy 1, policy_version 27530 (0.0007) [2023-10-10 17:39:06,697][123582] Updated weights for policy 0, policy_version 27593 (0.0008) [2023-10-10 17:39:07,017][123614] Updated weights for policy 1, policy_version 27540 (0.0007) [2023-10-10 17:39:07,064][123582] Updated weights for policy 0, policy_version 27603 (0.0009) [2023-10-10 17:39:07,382][123614] Updated weights for policy 1, policy_version 27550 (0.0008) [2023-10-10 17:39:07,442][123582] Updated weights for policy 0, policy_version 27613 (0.0008) [2023-10-10 17:39:08,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 56492032. Throughput: 0: 1810.7, 1: 1798.7. Samples: 14131924. Policy #0 lag: (min: 31.0, avg: 32.9, max: 60.0) [2023-10-10 17:39:08,789][122664] Avg episode reward: [(0, '40.840'), (1, '32.280')] [2023-10-10 17:39:11,042][123614] Updated weights for policy 1, policy_version 27560 (0.0009) [2023-10-10 17:39:11,245][123582] Updated weights for policy 0, policy_version 27623 (0.0007) [2023-10-10 17:39:11,414][123614] Updated weights for policy 1, policy_version 27570 (0.0008) [2023-10-10 17:39:11,618][123582] Updated weights for policy 0, policy_version 27633 (0.0008) [2023-10-10 17:39:11,777][123614] Updated weights for policy 1, policy_version 27580 (0.0007) [2023-10-10 17:39:11,995][123582] Updated weights for policy 0, policy_version 27643 (0.0007) [2023-10-10 17:39:13,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 56557568. Throughput: 0: 1819.3, 1: 1806.1. Samples: 14143018. Policy #0 lag: (min: 31.0, avg: 32.9, max: 60.0) [2023-10-10 17:39:13,789][122664] Avg episode reward: [(0, '41.180'), (1, '31.570')] [2023-10-10 17:39:15,525][123614] Updated weights for policy 1, policy_version 27590 (0.0007) [2023-10-10 17:39:15,614][123582] Updated weights for policy 0, policy_version 27653 (0.0007) [2023-10-10 17:39:15,891][123614] Updated weights for policy 1, policy_version 27600 (0.0007) [2023-10-10 17:39:15,981][123582] Updated weights for policy 0, policy_version 27663 (0.0007) [2023-10-10 17:39:16,254][123614] Updated weights for policy 1, policy_version 27610 (0.0009) [2023-10-10 17:39:16,366][123582] Updated weights for policy 0, policy_version 27673 (0.0007) [2023-10-10 17:39:18,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 56623104. Throughput: 0: 1803.8, 1: 1797.2. Samples: 14164294. Policy #0 lag: (min: 20.0, avg: 32.9, max: 52.0) [2023-10-10 17:39:18,789][122664] Avg episode reward: [(0, '39.590'), (1, '31.620')] [2023-10-10 17:39:19,997][123614] Updated weights for policy 1, policy_version 27620 (0.0008) [2023-10-10 17:39:20,147][123582] Updated weights for policy 0, policy_version 27683 (0.0009) [2023-10-10 17:39:20,396][123614] Updated weights for policy 1, policy_version 27630 (0.0007) [2023-10-10 17:39:20,543][123582] Updated weights for policy 0, policy_version 27693 (0.0009) [2023-10-10 17:39:20,763][123614] Updated weights for policy 1, policy_version 27640 (0.0008) [2023-10-10 17:39:20,917][123582] Updated weights for policy 0, policy_version 27703 (0.0009) [2023-10-10 17:39:23,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 56688640. Throughput: 0: 1804.4, 1: 1796.0. Samples: 14186522. Policy #0 lag: (min: 20.0, avg: 32.9, max: 52.0) [2023-10-10 17:39:23,789][122664] Avg episode reward: [(0, '39.740'), (1, '31.600')] [2023-10-10 17:39:24,444][123614] Updated weights for policy 1, policy_version 27650 (0.0008) [2023-10-10 17:39:24,667][123582] Updated weights for policy 0, policy_version 27713 (0.0008) [2023-10-10 17:39:24,810][123614] Updated weights for policy 1, policy_version 27660 (0.0009) [2023-10-10 17:39:25,030][123582] Updated weights for policy 0, policy_version 27723 (0.0008) [2023-10-10 17:39:25,176][123614] Updated weights for policy 1, policy_version 27670 (0.0009) [2023-10-10 17:39:25,399][123582] Updated weights for policy 0, policy_version 27733 (0.0007) [2023-10-10 17:39:25,536][123614] Updated weights for policy 1, policy_version 27680 (0.0008) [2023-10-10 17:39:25,772][123582] Updated weights for policy 0, policy_version 27743 (0.0009) [2023-10-10 17:39:28,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 56754176. Throughput: 0: 1805.4, 1: 1798.2. Samples: 14196624. Policy #0 lag: (min: 20.0, avg: 32.9, max: 52.0) [2023-10-10 17:39:28,789][122664] Avg episode reward: [(0, '40.330'), (1, '28.910')] [2023-10-10 17:39:29,283][123614] Updated weights for policy 1, policy_version 27690 (0.0007) [2023-10-10 17:39:29,473][123582] Updated weights for policy 0, policy_version 27753 (0.0009) [2023-10-10 17:39:29,646][123614] Updated weights for policy 1, policy_version 27700 (0.0007) [2023-10-10 17:39:29,849][123582] Updated weights for policy 0, policy_version 27763 (0.0009) [2023-10-10 17:39:30,020][123614] Updated weights for policy 1, policy_version 27710 (0.0008) [2023-10-10 17:39:30,216][123582] Updated weights for policy 0, policy_version 27773 (0.0007) [2023-10-10 17:39:33,560][123614] Updated weights for policy 1, policy_version 27720 (0.0009) [2023-10-10 17:39:33,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 56819712. Throughput: 0: 1803.3, 1: 1799.6. Samples: 14219692. Policy #0 lag: (min: 20.0, avg: 32.9, max: 52.0) [2023-10-10 17:39:33,788][122664] Avg episode reward: [(0, '39.500'), (1, '28.970')] [2023-10-10 17:39:33,872][123582] Updated weights for policy 0, policy_version 27783 (0.0008) [2023-10-10 17:39:33,937][123614] Updated weights for policy 1, policy_version 27730 (0.0007) [2023-10-10 17:39:34,242][123582] Updated weights for policy 0, policy_version 27793 (0.0009) [2023-10-10 17:39:34,296][123614] Updated weights for policy 1, policy_version 27740 (0.0008) [2023-10-10 17:39:34,610][123582] Updated weights for policy 0, policy_version 27803 (0.0008) [2023-10-10 17:39:38,087][123614] Updated weights for policy 1, policy_version 27750 (0.0008) [2023-10-10 17:39:38,333][123582] Updated weights for policy 0, policy_version 27813 (0.0009) [2023-10-10 17:39:38,452][123614] Updated weights for policy 1, policy_version 27760 (0.0008) [2023-10-10 17:39:38,710][123582] Updated weights for policy 0, policy_version 27823 (0.0008) [2023-10-10 17:39:38,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 56885248. Throughput: 0: 1807.5, 1: 1816.2. Samples: 14240904. Policy #0 lag: (min: 20.0, avg: 32.9, max: 52.0) [2023-10-10 17:39:38,788][122664] Avg episode reward: [(0, '37.160'), (1, '27.830')] [2023-10-10 17:39:38,819][123614] Updated weights for policy 1, policy_version 27770 (0.0009) [2023-10-10 17:39:39,076][123582] Updated weights for policy 0, policy_version 27833 (0.0008) [2023-10-10 17:39:42,561][123614] Updated weights for policy 1, policy_version 27780 (0.0011) [2023-10-10 17:39:42,883][123582] Updated weights for policy 0, policy_version 27843 (0.0008) [2023-10-10 17:39:42,923][123614] Updated weights for policy 1, policy_version 27790 (0.0009) [2023-10-10 17:39:43,256][123582] Updated weights for policy 0, policy_version 27853 (0.0007) [2023-10-10 17:39:43,302][123614] Updated weights for policy 1, policy_version 27800 (0.0008) [2023-10-10 17:39:43,627][123582] Updated weights for policy 0, policy_version 27863 (0.0007) [2023-10-10 17:39:43,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 56983552. Throughput: 0: 1801.9, 1: 1804.7. Samples: 14251974. Policy #0 lag: (min: 4.0, avg: 12.5, max: 36.0) [2023-10-10 17:39:43,789][122664] Avg episode reward: [(0, '38.090'), (1, '31.620')] [2023-10-10 17:39:46,852][123614] Updated weights for policy 1, policy_version 27810 (0.0009) [2023-10-10 17:39:47,229][123614] Updated weights for policy 1, policy_version 27820 (0.0008) [2023-10-10 17:39:47,255][123582] Updated weights for policy 0, policy_version 27873 (0.0008) [2023-10-10 17:39:47,590][123614] Updated weights for policy 1, policy_version 27830 (0.0007) [2023-10-10 17:39:47,629][123582] Updated weights for policy 0, policy_version 27883 (0.0009) [2023-10-10 17:39:47,961][123614] Updated weights for policy 1, policy_version 27840 (0.0007) [2023-10-10 17:39:48,002][123582] Updated weights for policy 0, policy_version 27893 (0.0009) [2023-10-10 17:39:48,363][123582] Updated weights for policy 0, policy_version 27903 (0.0008) [2023-10-10 17:39:48,788][122664] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 57081856. Throughput: 0: 1811.3, 1: 1813.1. Samples: 14273570. Policy #0 lag: (min: 4.0, avg: 12.5, max: 36.0) [2023-10-10 17:39:48,789][122664] Avg episode reward: [(0, '40.540'), (1, '34.300')] [2023-10-10 17:39:51,699][123614] Updated weights for policy 1, policy_version 27850 (0.0008) [2023-10-10 17:39:52,032][123582] Updated weights for policy 0, policy_version 27913 (0.0008) [2023-10-10 17:39:52,056][123614] Updated weights for policy 1, policy_version 27860 (0.0007) [2023-10-10 17:39:52,409][123582] Updated weights for policy 0, policy_version 27923 (0.0007) [2023-10-10 17:39:52,431][123614] Updated weights for policy 1, policy_version 27870 (0.0008) [2023-10-10 17:39:52,777][123582] Updated weights for policy 0, policy_version 27933 (0.0008) [2023-10-10 17:39:53,788][122664] Fps is (10 sec: 16384.3, 60 sec: 15291.8, 300 sec: 14551.2). Total num frames: 57147392. Throughput: 0: 1806.2, 1: 1815.9. Samples: 14294918. Policy #0 lag: (min: 4.0, avg: 12.5, max: 36.0) [2023-10-10 17:39:53,789][122664] Avg episode reward: [(0, '40.950'), (1, '35.160')] [2023-10-10 17:39:53,801][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000027872_28540928.pth... [2023-10-10 17:39:53,802][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000027936_28606464.pth... [2023-10-10 17:39:53,841][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000026240_26869760.pth [2023-10-10 17:39:53,841][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000026176_26804224.pth [2023-10-10 17:39:56,160][123614] Updated weights for policy 1, policy_version 27880 (0.0009) [2023-10-10 17:39:56,526][123582] Updated weights for policy 0, policy_version 27943 (0.0007) [2023-10-10 17:39:56,529][123614] Updated weights for policy 1, policy_version 27890 (0.0009) [2023-10-10 17:39:56,894][123614] Updated weights for policy 1, policy_version 27900 (0.0007) [2023-10-10 17:39:56,898][123582] Updated weights for policy 0, policy_version 27953 (0.0008) [2023-10-10 17:39:57,271][123582] Updated weights for policy 0, policy_version 27963 (0.0009) [2023-10-10 17:39:58,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 57212928. Throughput: 0: 1815.8, 1: 1817.4. Samples: 14306514. Policy #0 lag: (min: 4.0, avg: 12.5, max: 36.0) [2023-10-10 17:39:58,789][122664] Avg episode reward: [(0, '42.090'), (1, '37.770')] [2023-10-10 17:39:58,790][123247] Saving new best policy, reward=42.090! [2023-10-10 17:40:00,507][123614] Updated weights for policy 1, policy_version 27910 (0.0008) [2023-10-10 17:40:00,868][123614] Updated weights for policy 1, policy_version 27920 (0.0007) [2023-10-10 17:40:01,075][123582] Updated weights for policy 0, policy_version 27973 (0.0007) [2023-10-10 17:40:01,243][123614] Updated weights for policy 1, policy_version 27930 (0.0007) [2023-10-10 17:40:01,441][123582] Updated weights for policy 0, policy_version 27983 (0.0008) [2023-10-10 17:40:01,809][123582] Updated weights for policy 0, policy_version 27993 (0.0008) [2023-10-10 17:40:03,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 57278464. Throughput: 0: 1804.7, 1: 1821.0. Samples: 14327448. Policy #0 lag: (min: 2.0, avg: 9.6, max: 34.0) [2023-10-10 17:40:03,789][122664] Avg episode reward: [(0, '41.300'), (1, '37.980')] [2023-10-10 17:40:04,920][123614] Updated weights for policy 1, policy_version 27940 (0.0007) [2023-10-10 17:40:05,316][123614] Updated weights for policy 1, policy_version 27950 (0.0010) [2023-10-10 17:40:05,514][123582] Updated weights for policy 0, policy_version 28003 (0.0009) [2023-10-10 17:40:05,687][123614] Updated weights for policy 1, policy_version 27960 (0.0008) [2023-10-10 17:40:05,915][123582] Updated weights for policy 0, policy_version 28013 (0.0007) [2023-10-10 17:40:06,285][123582] Updated weights for policy 0, policy_version 28023 (0.0010) [2023-10-10 17:40:08,788][122664] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 57344000. Throughput: 0: 1806.7, 1: 1823.3. Samples: 14349876. Policy #0 lag: (min: 2.0, avg: 9.6, max: 34.0) [2023-10-10 17:40:08,789][122664] Avg episode reward: [(0, '42.550'), (1, '36.510')] [2023-10-10 17:40:08,796][123247] Saving new best policy, reward=42.550! [2023-10-10 17:40:09,327][123614] Updated weights for policy 1, policy_version 27970 (0.0007) [2023-10-10 17:40:09,708][123614] Updated weights for policy 1, policy_version 27980 (0.0008) [2023-10-10 17:40:10,001][123582] Updated weights for policy 0, policy_version 28033 (0.0011) [2023-10-10 17:40:10,078][123614] Updated weights for policy 1, policy_version 27990 (0.0007) [2023-10-10 17:40:10,372][123582] Updated weights for policy 0, policy_version 28043 (0.0008) [2023-10-10 17:40:10,451][123614] Updated weights for policy 1, policy_version 28000 (0.0009) [2023-10-10 17:40:10,741][123582] Updated weights for policy 0, policy_version 28053 (0.0009) [2023-10-10 17:40:11,117][123582] Updated weights for policy 0, policy_version 28063 (0.0007) [2023-10-10 17:40:13,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 57409536. Throughput: 0: 1805.2, 1: 1819.4. Samples: 14359730. Policy #0 lag: (min: 2.0, avg: 9.6, max: 34.0) [2023-10-10 17:40:13,789][122664] Avg episode reward: [(0, '42.550'), (1, '37.770')] [2023-10-10 17:40:14,208][123614] Updated weights for policy 1, policy_version 28010 (0.0008) [2023-10-10 17:40:14,574][123614] Updated weights for policy 1, policy_version 28020 (0.0009) [2023-10-10 17:40:14,907][123582] Updated weights for policy 0, policy_version 28073 (0.0007) [2023-10-10 17:40:14,943][123614] Updated weights for policy 1, policy_version 28030 (0.0008) [2023-10-10 17:40:15,266][123582] Updated weights for policy 0, policy_version 28083 (0.0007) [2023-10-10 17:40:15,641][123582] Updated weights for policy 0, policy_version 28093 (0.0007) [2023-10-10 17:40:18,752][123614] Updated weights for policy 1, policy_version 28040 (0.0007) [2023-10-10 17:40:18,788][122664] Fps is (10 sec: 13107.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 57475072. Throughput: 0: 1802.8, 1: 1809.2. Samples: 14382228. Policy #0 lag: (min: 2.0, avg: 9.6, max: 34.0) [2023-10-10 17:40:18,788][122664] Avg episode reward: [(0, '44.500'), (1, '36.630')] [2023-10-10 17:40:18,789][123247] Saving new best policy, reward=44.500! [2023-10-10 17:40:19,115][123614] Updated weights for policy 1, policy_version 28050 (0.0007) [2023-10-10 17:40:19,296][123582] Updated weights for policy 0, policy_version 28103 (0.0008) [2023-10-10 17:40:19,484][123614] Updated weights for policy 1, policy_version 28060 (0.0007) [2023-10-10 17:40:19,665][123582] Updated weights for policy 0, policy_version 28113 (0.0009) [2023-10-10 17:40:20,032][123582] Updated weights for policy 0, policy_version 28123 (0.0010) [2023-10-10 17:40:23,335][123614] Updated weights for policy 1, policy_version 28070 (0.0009) [2023-10-10 17:40:23,715][123614] Updated weights for policy 1, policy_version 28080 (0.0009) [2023-10-10 17:40:23,772][123582] Updated weights for policy 0, policy_version 28133 (0.0010) [2023-10-10 17:40:23,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 57540608. Throughput: 0: 1808.8, 1: 1811.6. Samples: 14403822. Policy #0 lag: (min: 2.0, avg: 9.6, max: 34.0) [2023-10-10 17:40:23,789][122664] Avg episode reward: [(0, '41.510'), (1, '33.670')] [2023-10-10 17:40:24,075][123614] Updated weights for policy 1, policy_version 28090 (0.0008) [2023-10-10 17:40:24,156][123582] Updated weights for policy 0, policy_version 28143 (0.0008) [2023-10-10 17:40:24,520][123582] Updated weights for policy 0, policy_version 28153 (0.0009) [2023-10-10 17:40:27,621][123614] Updated weights for policy 1, policy_version 28100 (0.0009) [2023-10-10 17:40:27,992][123614] Updated weights for policy 1, policy_version 28110 (0.0009) [2023-10-10 17:40:28,250][123582] Updated weights for policy 0, policy_version 28163 (0.0010) [2023-10-10 17:40:28,369][123614] Updated weights for policy 1, policy_version 28120 (0.0009) [2023-10-10 17:40:28,623][123582] Updated weights for policy 0, policy_version 28173 (0.0007) [2023-10-10 17:40:28,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 57638912. Throughput: 0: 1801.3, 1: 1808.6. Samples: 14414418. Policy #0 lag: (min: 1.0, avg: 12.8, max: 33.0) [2023-10-10 17:40:28,789][122664] Avg episode reward: [(0, '40.500'), (1, '31.550')] [2023-10-10 17:40:28,986][123582] Updated weights for policy 0, policy_version 28183 (0.0007) [2023-10-10 17:40:32,151][123614] Updated weights for policy 1, policy_version 28130 (0.0009) [2023-10-10 17:40:32,524][123614] Updated weights for policy 1, policy_version 28140 (0.0007) [2023-10-10 17:40:32,733][123582] Updated weights for policy 0, policy_version 28193 (0.0007) [2023-10-10 17:40:32,890][123614] Updated weights for policy 1, policy_version 28150 (0.0008) [2023-10-10 17:40:33,095][123582] Updated weights for policy 0, policy_version 28203 (0.0007) [2023-10-10 17:40:33,255][123614] Updated weights for policy 1, policy_version 28160 (0.0010) [2023-10-10 17:40:33,475][123582] Updated weights for policy 0, policy_version 28213 (0.0008) [2023-10-10 17:40:33,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 57704448. Throughput: 0: 1805.4, 1: 1811.2. Samples: 14436316. Policy #0 lag: (min: 1.0, avg: 12.8, max: 33.0) [2023-10-10 17:40:33,789][122664] Avg episode reward: [(0, '39.190'), (1, '31.380')] [2023-10-10 17:40:33,850][123582] Updated weights for policy 0, policy_version 28223 (0.0008) [2023-10-10 17:40:36,967][123614] Updated weights for policy 1, policy_version 28170 (0.0008) [2023-10-10 17:40:37,337][123614] Updated weights for policy 1, policy_version 28180 (0.0007) [2023-10-10 17:40:37,418][123582] Updated weights for policy 0, policy_version 28233 (0.0010) [2023-10-10 17:40:37,698][123614] Updated weights for policy 1, policy_version 28190 (0.0007) [2023-10-10 17:40:37,797][123582] Updated weights for policy 0, policy_version 28243 (0.0008) [2023-10-10 17:40:38,179][123582] Updated weights for policy 0, policy_version 28253 (0.0011) [2023-10-10 17:40:38,788][122664] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 57802752. Throughput: 0: 1796.8, 1: 1796.4. Samples: 14456614. Policy #0 lag: (min: 1.0, avg: 12.8, max: 33.0) [2023-10-10 17:40:38,789][122664] Avg episode reward: [(0, '39.830'), (1, '31.980')] [2023-10-10 17:40:41,431][123614] Updated weights for policy 1, policy_version 28200 (0.0010) [2023-10-10 17:40:41,805][123614] Updated weights for policy 1, policy_version 28210 (0.0008) [2023-10-10 17:40:41,990][123582] Updated weights for policy 0, policy_version 28263 (0.0010) [2023-10-10 17:40:42,174][123614] Updated weights for policy 1, policy_version 28220 (0.0009) [2023-10-10 17:40:42,353][123582] Updated weights for policy 0, policy_version 28273 (0.0007) [2023-10-10 17:40:42,723][123582] Updated weights for policy 0, policy_version 28283 (0.0008) [2023-10-10 17:40:43,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 57868288. Throughput: 0: 1802.5, 1: 1802.0. Samples: 14468720. Policy #0 lag: (min: 1.0, avg: 12.8, max: 33.0) [2023-10-10 17:40:43,789][122664] Avg episode reward: [(0, '38.740'), (1, '30.360')] [2023-10-10 17:40:45,830][123614] Updated weights for policy 1, policy_version 28230 (0.0007) [2023-10-10 17:40:46,201][123614] Updated weights for policy 1, policy_version 28240 (0.0007) [2023-10-10 17:40:46,464][123582] Updated weights for policy 0, policy_version 28293 (0.0009) [2023-10-10 17:40:46,572][123614] Updated weights for policy 1, policy_version 28250 (0.0008) [2023-10-10 17:40:46,841][123582] Updated weights for policy 0, policy_version 28303 (0.0007) [2023-10-10 17:40:47,205][123582] Updated weights for policy 0, policy_version 28313 (0.0007) [2023-10-10 17:40:48,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 57933824. Throughput: 0: 1802.7, 1: 1792.4. Samples: 14489224. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 17:40:48,789][122664] Avg episode reward: [(0, '38.040'), (1, '31.390')] [2023-10-10 17:40:50,430][123614] Updated weights for policy 1, policy_version 28260 (0.0007) [2023-10-10 17:40:50,812][123614] Updated weights for policy 1, policy_version 28270 (0.0009) [2023-10-10 17:40:51,089][123582] Updated weights for policy 0, policy_version 28323 (0.0008) [2023-10-10 17:40:51,187][123614] Updated weights for policy 1, policy_version 28280 (0.0007) [2023-10-10 17:40:51,494][123582] Updated weights for policy 0, policy_version 28333 (0.0008) [2023-10-10 17:40:51,864][123582] Updated weights for policy 0, policy_version 28343 (0.0007) [2023-10-10 17:40:53,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 57999360. Throughput: 0: 1792.7, 1: 1794.0. Samples: 14511274. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 17:40:53,789][122664] Avg episode reward: [(0, '37.240'), (1, '33.450')] [2023-10-10 17:40:54,791][123614] Updated weights for policy 1, policy_version 28290 (0.0007) [2023-10-10 17:40:55,156][123614] Updated weights for policy 1, policy_version 28300 (0.0007) [2023-10-10 17:40:55,528][123614] Updated weights for policy 1, policy_version 28310 (0.0007) [2023-10-10 17:40:55,574][123582] Updated weights for policy 0, policy_version 28353 (0.0011) [2023-10-10 17:40:55,894][123614] Updated weights for policy 1, policy_version 28320 (0.0007) [2023-10-10 17:40:55,950][123582] Updated weights for policy 0, policy_version 28363 (0.0010) [2023-10-10 17:40:56,326][123582] Updated weights for policy 0, policy_version 28373 (0.0010) [2023-10-10 17:40:56,707][123582] Updated weights for policy 0, policy_version 28383 (0.0008) [2023-10-10 17:40:58,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 58064896. Throughput: 0: 1802.8, 1: 1798.0. Samples: 14521766. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 17:40:58,789][122664] Avg episode reward: [(0, '38.730'), (1, '33.790')] [2023-10-10 17:40:59,590][123614] Updated weights for policy 1, policy_version 28330 (0.0007) [2023-10-10 17:40:59,966][123614] Updated weights for policy 1, policy_version 28340 (0.0007) [2023-10-10 17:41:00,331][123614] Updated weights for policy 1, policy_version 28350 (0.0007) [2023-10-10 17:41:00,454][123582] Updated weights for policy 0, policy_version 28393 (0.0007) [2023-10-10 17:41:00,832][123582] Updated weights for policy 0, policy_version 28403 (0.0009) [2023-10-10 17:41:01,198][123582] Updated weights for policy 0, policy_version 28413 (0.0008) [2023-10-10 17:41:03,788][122664] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 58130432. Throughput: 0: 1794.9, 1: 1801.9. Samples: 14544086. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 17:41:03,788][122664] Avg episode reward: [(0, '39.680'), (1, '32.240')] [2023-10-10 17:41:04,005][123614] Updated weights for policy 1, policy_version 28360 (0.0007) [2023-10-10 17:41:04,369][123614] Updated weights for policy 1, policy_version 28370 (0.0009) [2023-10-10 17:41:04,734][123614] Updated weights for policy 1, policy_version 28380 (0.0009) [2023-10-10 17:41:04,919][123582] Updated weights for policy 0, policy_version 28423 (0.0009) [2023-10-10 17:41:05,307][123582] Updated weights for policy 0, policy_version 28433 (0.0008) [2023-10-10 17:41:05,685][123582] Updated weights for policy 0, policy_version 28443 (0.0009) [2023-10-10 17:41:08,476][123614] Updated weights for policy 1, policy_version 28390 (0.0008) [2023-10-10 17:41:08,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.6, 300 sec: 14440.1). Total num frames: 58195968. Throughput: 0: 1792.2, 1: 1811.2. Samples: 14565976. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 17:41:08,788][122664] Avg episode reward: [(0, '37.150'), (1, '33.110')] [2023-10-10 17:41:08,854][123614] Updated weights for policy 1, policy_version 28400 (0.0007) [2023-10-10 17:41:09,223][123614] Updated weights for policy 1, policy_version 28410 (0.0008) [2023-10-10 17:41:09,420][123582] Updated weights for policy 0, policy_version 28453 (0.0009) [2023-10-10 17:41:09,789][123582] Updated weights for policy 0, policy_version 28463 (0.0012) [2023-10-10 17:41:10,165][123582] Updated weights for policy 0, policy_version 28473 (0.0010) [2023-10-10 17:41:12,925][123614] Updated weights for policy 1, policy_version 28420 (0.0008) [2023-10-10 17:41:13,291][123614] Updated weights for policy 1, policy_version 28430 (0.0008) [2023-10-10 17:41:13,652][123614] Updated weights for policy 1, policy_version 28440 (0.0009) [2023-10-10 17:41:13,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 58261504. Throughput: 0: 1790.0, 1: 1804.9. Samples: 14576186. Policy #0 lag: (min: 31.0, avg: 45.0, max: 63.0) [2023-10-10 17:41:13,788][122664] Avg episode reward: [(0, '39.040'), (1, '31.450')] [2023-10-10 17:41:13,946][123582] Updated weights for policy 0, policy_version 28483 (0.0007) [2023-10-10 17:41:14,333][123582] Updated weights for policy 0, policy_version 28493 (0.0009) [2023-10-10 17:41:14,700][123582] Updated weights for policy 0, policy_version 28503 (0.0007) [2023-10-10 17:41:17,221][123614] Updated weights for policy 1, policy_version 28450 (0.0008) [2023-10-10 17:41:17,595][123614] Updated weights for policy 1, policy_version 28460 (0.0009) [2023-10-10 17:41:17,961][123614] Updated weights for policy 1, policy_version 28470 (0.0007) [2023-10-10 17:41:18,329][123614] Updated weights for policy 1, policy_version 28480 (0.0009) [2023-10-10 17:41:18,339][123582] Updated weights for policy 0, policy_version 28513 (0.0007) [2023-10-10 17:41:18,705][123582] Updated weights for policy 0, policy_version 28523 (0.0010) [2023-10-10 17:41:18,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 58359808. Throughput: 0: 1791.5, 1: 1814.2. Samples: 14598572. Policy #0 lag: (min: 31.0, avg: 45.0, max: 63.0) [2023-10-10 17:41:18,788][122664] Avg episode reward: [(0, '38.090'), (1, '33.140')] [2023-10-10 17:41:19,090][123582] Updated weights for policy 0, policy_version 28533 (0.0010) [2023-10-10 17:41:19,455][123582] Updated weights for policy 0, policy_version 28543 (0.0009) [2023-10-10 17:41:22,039][123614] Updated weights for policy 1, policy_version 28490 (0.0011) [2023-10-10 17:41:22,407][123614] Updated weights for policy 1, policy_version 28500 (0.0008) [2023-10-10 17:41:22,768][123614] Updated weights for policy 1, policy_version 28510 (0.0011) [2023-10-10 17:41:23,221][123582] Updated weights for policy 0, policy_version 28553 (0.0008) [2023-10-10 17:41:23,595][123582] Updated weights for policy 0, policy_version 28563 (0.0008) [2023-10-10 17:41:23,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 58425344. Throughput: 0: 1807.5, 1: 1818.9. Samples: 14619804. Policy #0 lag: (min: 31.0, avg: 45.0, max: 63.0) [2023-10-10 17:41:23,788][122664] Avg episode reward: [(0, '41.260'), (1, '30.760')] [2023-10-10 17:41:23,968][123582] Updated weights for policy 0, policy_version 28573 (0.0009) [2023-10-10 17:41:26,570][123614] Updated weights for policy 1, policy_version 28520 (0.0009) [2023-10-10 17:41:26,945][123614] Updated weights for policy 1, policy_version 28530 (0.0009) [2023-10-10 17:41:27,310][123614] Updated weights for policy 1, policy_version 28540 (0.0009) [2023-10-10 17:41:27,502][123582] Updated weights for policy 0, policy_version 28583 (0.0009) [2023-10-10 17:41:27,878][123582] Updated weights for policy 0, policy_version 28593 (0.0010) [2023-10-10 17:41:28,243][123582] Updated weights for policy 0, policy_version 28603 (0.0011) [2023-10-10 17:41:28,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 58523648. Throughput: 0: 1788.8, 1: 1822.2. Samples: 14631216. Policy #0 lag: (min: 31.0, avg: 45.0, max: 63.0) [2023-10-10 17:41:28,788][122664] Avg episode reward: [(0, '41.430'), (1, '29.100')] [2023-10-10 17:41:31,087][123614] Updated weights for policy 1, policy_version 28550 (0.0010) [2023-10-10 17:41:31,449][123614] Updated weights for policy 1, policy_version 28560 (0.0008) [2023-10-10 17:41:31,822][123614] Updated weights for policy 1, policy_version 28570 (0.0008) [2023-10-10 17:41:31,906][123582] Updated weights for policy 0, policy_version 28613 (0.0009) [2023-10-10 17:41:32,280][123582] Updated weights for policy 0, policy_version 28623 (0.0007) [2023-10-10 17:41:32,645][123582] Updated weights for policy 0, policy_version 28633 (0.0008) [2023-10-10 17:41:33,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 58589184. Throughput: 0: 1803.4, 1: 1814.0. Samples: 14652008. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 17:41:33,789][122664] Avg episode reward: [(0, '39.520'), (1, '30.750')] [2023-10-10 17:41:35,792][123614] Updated weights for policy 1, policy_version 28580 (0.0008) [2023-10-10 17:41:36,175][123614] Updated weights for policy 1, policy_version 28590 (0.0007) [2023-10-10 17:41:36,380][123582] Updated weights for policy 0, policy_version 28643 (0.0007) [2023-10-10 17:41:36,550][123614] Updated weights for policy 1, policy_version 28600 (0.0008) [2023-10-10 17:41:36,771][123582] Updated weights for policy 0, policy_version 28653 (0.0007) [2023-10-10 17:41:37,152][123582] Updated weights for policy 0, policy_version 28663 (0.0009) [2023-10-10 17:41:38,788][122664] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 58654720. Throughput: 0: 1801.3, 1: 1806.0. Samples: 14673602. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 17:41:38,789][122664] Avg episode reward: [(0, '40.260'), (1, '31.300')] [2023-10-10 17:41:40,287][123614] Updated weights for policy 1, policy_version 28610 (0.0008) [2023-10-10 17:41:40,658][123614] Updated weights for policy 1, policy_version 28620 (0.0007) [2023-10-10 17:41:40,875][123582] Updated weights for policy 0, policy_version 28673 (0.0010) [2023-10-10 17:41:41,028][123614] Updated weights for policy 1, policy_version 28630 (0.0008) [2023-10-10 17:41:41,245][123582] Updated weights for policy 0, policy_version 28683 (0.0008) [2023-10-10 17:41:41,393][123614] Updated weights for policy 1, policy_version 28640 (0.0009) [2023-10-10 17:41:41,614][123582] Updated weights for policy 0, policy_version 28693 (0.0007) [2023-10-10 17:41:41,986][123582] Updated weights for policy 0, policy_version 28703 (0.0009) [2023-10-10 17:41:43,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 58720256. Throughput: 0: 1808.4, 1: 1803.7. Samples: 14684312. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 17:41:43,788][122664] Avg episode reward: [(0, '40.030'), (1, '31.000')] [2023-10-10 17:41:45,021][123614] Updated weights for policy 1, policy_version 28650 (0.0008) [2023-10-10 17:41:45,386][123614] Updated weights for policy 1, policy_version 28660 (0.0007) [2023-10-10 17:41:45,756][123614] Updated weights for policy 1, policy_version 28670 (0.0008) [2023-10-10 17:41:45,763][123582] Updated weights for policy 0, policy_version 28713 (0.0008) [2023-10-10 17:41:46,123][123582] Updated weights for policy 0, policy_version 28723 (0.0010) [2023-10-10 17:41:46,509][123582] Updated weights for policy 0, policy_version 28733 (0.0010) [2023-10-10 17:41:48,788][122664] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 58785792. Throughput: 0: 1798.1, 1: 1804.9. Samples: 14706222. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 17:41:48,788][122664] Avg episode reward: [(0, '41.630'), (1, '32.240')] [2023-10-10 17:41:49,706][123614] Updated weights for policy 1, policy_version 28680 (0.0010) [2023-10-10 17:41:50,076][123614] Updated weights for policy 1, policy_version 28690 (0.0009) [2023-10-10 17:41:50,380][123582] Updated weights for policy 0, policy_version 28743 (0.0010) [2023-10-10 17:41:50,438][123614] Updated weights for policy 1, policy_version 28700 (0.0007) [2023-10-10 17:41:50,753][123582] Updated weights for policy 0, policy_version 28753 (0.0008) [2023-10-10 17:41:51,129][123582] Updated weights for policy 0, policy_version 28763 (0.0007) [2023-10-10 17:41:53,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 58851328. Throughput: 0: 1802.6, 1: 1812.4. Samples: 14728654. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 17:41:53,789][122664] Avg episode reward: [(0, '39.440'), (1, '32.640')] [2023-10-10 17:41:53,803][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000028704_29392896.pth... [2023-10-10 17:41:53,804][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000028768_29458432.pth... [2023-10-10 17:41:53,838][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000027040_27688960.pth [2023-10-10 17:41:53,839][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000027072_27721728.pth [2023-10-10 17:41:54,196][123614] Updated weights for policy 1, policy_version 28710 (0.0007) [2023-10-10 17:41:54,568][123614] Updated weights for policy 1, policy_version 28720 (0.0009) [2023-10-10 17:41:54,772][123582] Updated weights for policy 0, policy_version 28773 (0.0008) [2023-10-10 17:41:54,943][123614] Updated weights for policy 1, policy_version 28730 (0.0007) [2023-10-10 17:41:55,137][123582] Updated weights for policy 0, policy_version 28783 (0.0009) [2023-10-10 17:41:55,513][123582] Updated weights for policy 0, policy_version 28793 (0.0009) [2023-10-10 17:41:58,511][123614] Updated weights for policy 1, policy_version 28740 (0.0009) [2023-10-10 17:41:58,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 58916864. Throughput: 0: 1807.4, 1: 1804.4. Samples: 14738716. Policy #0 lag: (min: 24.0, avg: 51.0, max: 56.0) [2023-10-10 17:41:58,789][122664] Avg episode reward: [(0, '39.320'), (1, '34.160')] [2023-10-10 17:41:58,881][123614] Updated weights for policy 1, policy_version 28750 (0.0007) [2023-10-10 17:41:59,250][123614] Updated weights for policy 1, policy_version 28760 (0.0007) [2023-10-10 17:41:59,273][123582] Updated weights for policy 0, policy_version 28803 (0.0007) [2023-10-10 17:41:59,638][123582] Updated weights for policy 0, policy_version 28813 (0.0007) [2023-10-10 17:42:00,012][123582] Updated weights for policy 0, policy_version 28823 (0.0009) [2023-10-10 17:42:02,996][123614] Updated weights for policy 1, policy_version 28770 (0.0009) [2023-10-10 17:42:03,374][123614] Updated weights for policy 1, policy_version 28780 (0.0009) [2023-10-10 17:42:03,587][123582] Updated weights for policy 0, policy_version 28833 (0.0008) [2023-10-10 17:42:03,735][123614] Updated weights for policy 1, policy_version 28790 (0.0008) [2023-10-10 17:42:03,788][122664] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 58982400. Throughput: 0: 1805.2, 1: 1816.4. Samples: 14761546. Policy #0 lag: (min: 24.0, avg: 51.0, max: 56.0) [2023-10-10 17:42:03,788][122664] Avg episode reward: [(0, '43.310'), (1, '35.820')] [2023-10-10 17:42:03,968][123582] Updated weights for policy 0, policy_version 28843 (0.0009) [2023-10-10 17:42:04,100][123614] Updated weights for policy 1, policy_version 28800 (0.0008) [2023-10-10 17:42:04,336][123582] Updated weights for policy 0, policy_version 28853 (0.0008) [2023-10-10 17:42:04,711][123582] Updated weights for policy 0, policy_version 28863 (0.0008) [2023-10-10 17:42:07,883][123614] Updated weights for policy 1, policy_version 28810 (0.0008) [2023-10-10 17:42:08,245][123614] Updated weights for policy 1, policy_version 28820 (0.0007) [2023-10-10 17:42:08,466][123582] Updated weights for policy 0, policy_version 28873 (0.0008) [2023-10-10 17:42:08,619][123614] Updated weights for policy 1, policy_version 28830 (0.0008) [2023-10-10 17:42:08,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 59080704. Throughput: 0: 1810.1, 1: 1797.8. Samples: 14782160. Policy #0 lag: (min: 24.0, avg: 51.0, max: 56.0) [2023-10-10 17:42:08,789][122664] Avg episode reward: [(0, '43.720'), (1, '33.920')] [2023-10-10 17:42:08,837][123582] Updated weights for policy 0, policy_version 28883 (0.0007) [2023-10-10 17:42:09,212][123582] Updated weights for policy 0, policy_version 28893 (0.0008) [2023-10-10 17:42:12,373][123614] Updated weights for policy 1, policy_version 28840 (0.0008) [2023-10-10 17:42:12,747][123614] Updated weights for policy 1, policy_version 28850 (0.0008) [2023-10-10 17:42:12,873][123582] Updated weights for policy 0, policy_version 28903 (0.0007) [2023-10-10 17:42:13,114][123614] Updated weights for policy 1, policy_version 28860 (0.0007) [2023-10-10 17:42:13,244][123582] Updated weights for policy 0, policy_version 28913 (0.0008) [2023-10-10 17:42:13,624][123582] Updated weights for policy 0, policy_version 28923 (0.0008) [2023-10-10 17:42:13,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 59146240. Throughput: 0: 1803.3, 1: 1807.6. Samples: 14793708. Policy #0 lag: (min: 24.0, avg: 51.0, max: 56.0) [2023-10-10 17:42:13,788][122664] Avg episode reward: [(0, '42.450'), (1, '30.170')] [2023-10-10 17:42:16,694][123614] Updated weights for policy 1, policy_version 28870 (0.0007) [2023-10-10 17:42:17,065][123614] Updated weights for policy 1, policy_version 28880 (0.0008) [2023-10-10 17:42:17,330][123582] Updated weights for policy 0, policy_version 28933 (0.0009) [2023-10-10 17:42:17,426][123614] Updated weights for policy 1, policy_version 28890 (0.0009) [2023-10-10 17:42:17,696][123582] Updated weights for policy 0, policy_version 28943 (0.0007) [2023-10-10 17:42:18,071][123582] Updated weights for policy 0, policy_version 28953 (0.0010) [2023-10-10 17:42:18,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 59244544. Throughput: 0: 1820.6, 1: 1799.2. Samples: 14814898. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 17:42:18,789][122664] Avg episode reward: [(0, '39.350'), (1, '30.300')] [2023-10-10 17:42:21,172][123614] Updated weights for policy 1, policy_version 28900 (0.0008) [2023-10-10 17:42:21,560][123614] Updated weights for policy 1, policy_version 28910 (0.0009) [2023-10-10 17:42:21,679][123582] Updated weights for policy 0, policy_version 28963 (0.0008) [2023-10-10 17:42:21,923][123614] Updated weights for policy 1, policy_version 28920 (0.0009) [2023-10-10 17:42:22,072][123582] Updated weights for policy 0, policy_version 28973 (0.0009) [2023-10-10 17:42:22,444][123582] Updated weights for policy 0, policy_version 28983 (0.0010) [2023-10-10 17:42:23,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 59310080. Throughput: 0: 1813.5, 1: 1802.1. Samples: 14836306. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 17:42:23,789][122664] Avg episode reward: [(0, '42.650'), (1, '30.520')] [2023-10-10 17:42:25,633][123614] Updated weights for policy 1, policy_version 28930 (0.0008) [2023-10-10 17:42:26,001][123614] Updated weights for policy 1, policy_version 28940 (0.0009) [2023-10-10 17:42:26,223][123582] Updated weights for policy 0, policy_version 28993 (0.0010) [2023-10-10 17:42:26,367][123614] Updated weights for policy 1, policy_version 28950 (0.0008) [2023-10-10 17:42:26,586][123582] Updated weights for policy 0, policy_version 29003 (0.0008) [2023-10-10 17:42:26,732][123614] Updated weights for policy 1, policy_version 28960 (0.0008) [2023-10-10 17:42:26,963][123582] Updated weights for policy 0, policy_version 29013 (0.0008) [2023-10-10 17:42:27,331][123582] Updated weights for policy 0, policy_version 29023 (0.0008) [2023-10-10 17:42:28,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 59375616. Throughput: 0: 1821.1, 1: 1803.4. Samples: 14847412. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 17:42:28,789][122664] Avg episode reward: [(0, '42.430'), (1, '30.240')] [2023-10-10 17:42:30,474][123614] Updated weights for policy 1, policy_version 28970 (0.0009) [2023-10-10 17:42:30,837][123614] Updated weights for policy 1, policy_version 28980 (0.0008) [2023-10-10 17:42:30,906][123582] Updated weights for policy 0, policy_version 29033 (0.0008) [2023-10-10 17:42:31,204][123614] Updated weights for policy 1, policy_version 28990 (0.0009) [2023-10-10 17:42:31,276][123582] Updated weights for policy 0, policy_version 29043 (0.0007) [2023-10-10 17:42:31,645][123582] Updated weights for policy 0, policy_version 29053 (0.0008) [2023-10-10 17:42:33,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 59441152. Throughput: 0: 1820.4, 1: 1794.2. Samples: 14868880. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 17:42:33,789][122664] Avg episode reward: [(0, '44.960'), (1, '28.930')] [2023-10-10 17:42:33,790][123247] Saving new best policy, reward=44.960! [2023-10-10 17:42:34,951][123614] Updated weights for policy 1, policy_version 29000 (0.0009) [2023-10-10 17:42:35,327][123614] Updated weights for policy 1, policy_version 29010 (0.0009) [2023-10-10 17:42:35,407][123582] Updated weights for policy 0, policy_version 29063 (0.0008) [2023-10-10 17:42:35,684][123614] Updated weights for policy 1, policy_version 29020 (0.0008) [2023-10-10 17:42:35,777][123582] Updated weights for policy 0, policy_version 29073 (0.0009) [2023-10-10 17:42:36,152][123582] Updated weights for policy 0, policy_version 29083 (0.0010) [2023-10-10 17:42:38,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 59506688. Throughput: 0: 1814.9, 1: 1801.6. Samples: 14891394. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 17:42:38,789][122664] Avg episode reward: [(0, '43.740'), (1, '26.440')] [2023-10-10 17:42:39,383][123614] Updated weights for policy 1, policy_version 29030 (0.0009) [2023-10-10 17:42:39,751][123614] Updated weights for policy 1, policy_version 29040 (0.0009) [2023-10-10 17:42:39,981][123582] Updated weights for policy 0, policy_version 29093 (0.0009) [2023-10-10 17:42:40,118][123614] Updated weights for policy 1, policy_version 29050 (0.0007) [2023-10-10 17:42:40,350][123582] Updated weights for policy 0, policy_version 29103 (0.0008) [2023-10-10 17:42:40,717][123582] Updated weights for policy 0, policy_version 29113 (0.0008) [2023-10-10 17:42:43,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 59572224. Throughput: 0: 1811.3, 1: 1804.9. Samples: 14901446. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:42:43,789][122664] Avg episode reward: [(0, '45.360'), (1, '29.440')] [2023-10-10 17:42:43,790][123247] Saving new best policy, reward=45.360! [2023-10-10 17:42:43,813][123614] Updated weights for policy 1, policy_version 29060 (0.0008) [2023-10-10 17:42:44,186][123614] Updated weights for policy 1, policy_version 29070 (0.0008) [2023-10-10 17:42:44,384][123582] Updated weights for policy 0, policy_version 29123 (0.0009) [2023-10-10 17:42:44,557][123614] Updated weights for policy 1, policy_version 29080 (0.0007) [2023-10-10 17:42:44,751][123582] Updated weights for policy 0, policy_version 29133 (0.0009) [2023-10-10 17:42:45,127][123582] Updated weights for policy 0, policy_version 29143 (0.0010) [2023-10-10 17:42:48,354][123614] Updated weights for policy 1, policy_version 29090 (0.0009) [2023-10-10 17:42:48,722][123614] Updated weights for policy 1, policy_version 29100 (0.0007) [2023-10-10 17:42:48,761][123582] Updated weights for policy 0, policy_version 29153 (0.0011) [2023-10-10 17:42:48,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 59637760. Throughput: 0: 1814.0, 1: 1800.0. Samples: 14924178. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:42:48,789][122664] Avg episode reward: [(0, '46.890'), (1, '30.010')] [2023-10-10 17:42:49,084][123614] Updated weights for policy 1, policy_version 29110 (0.0008) [2023-10-10 17:42:49,120][123582] Updated weights for policy 0, policy_version 29163 (0.0008) [2023-10-10 17:42:49,454][123614] Updated weights for policy 1, policy_version 29120 (0.0009) [2023-10-10 17:42:49,503][123582] Updated weights for policy 0, policy_version 29173 (0.0008) [2023-10-10 17:42:49,870][123582] Updated weights for policy 0, policy_version 29183 (0.0008) [2023-10-10 17:42:49,898][123247] Saving new best policy, reward=46.890! [2023-10-10 17:42:53,211][123614] Updated weights for policy 1, policy_version 29130 (0.0007) [2023-10-10 17:42:53,547][123582] Updated weights for policy 0, policy_version 29193 (0.0007) [2023-10-10 17:42:53,573][123614] Updated weights for policy 1, policy_version 29140 (0.0007) [2023-10-10 17:42:53,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 59703296. Throughput: 0: 1819.5, 1: 1808.2. Samples: 14945406. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:42:53,789][122664] Avg episode reward: [(0, '43.380'), (1, '29.250')] [2023-10-10 17:42:53,924][123582] Updated weights for policy 0, policy_version 29203 (0.0008) [2023-10-10 17:42:53,944][123614] Updated weights for policy 1, policy_version 29150 (0.0008) [2023-10-10 17:42:54,297][123582] Updated weights for policy 0, policy_version 29213 (0.0008) [2023-10-10 17:42:57,491][123614] Updated weights for policy 1, policy_version 29160 (0.0010) [2023-10-10 17:42:57,853][123614] Updated weights for policy 1, policy_version 29170 (0.0009) [2023-10-10 17:42:57,920][123582] Updated weights for policy 0, policy_version 29223 (0.0008) [2023-10-10 17:42:58,220][123614] Updated weights for policy 1, policy_version 29180 (0.0007) [2023-10-10 17:42:58,298][123582] Updated weights for policy 0, policy_version 29233 (0.0008) [2023-10-10 17:42:58,664][123582] Updated weights for policy 0, policy_version 29243 (0.0008) [2023-10-10 17:42:58,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 59801600. Throughput: 0: 1819.2, 1: 1803.9. Samples: 14956744. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:42:58,788][122664] Avg episode reward: [(0, '38.970'), (1, '28.740')] [2023-10-10 17:43:01,989][123614] Updated weights for policy 1, policy_version 29190 (0.0007) [2023-10-10 17:43:02,358][123614] Updated weights for policy 1, policy_version 29200 (0.0009) [2023-10-10 17:43:02,477][123582] Updated weights for policy 0, policy_version 29253 (0.0009) [2023-10-10 17:43:02,729][123614] Updated weights for policy 1, policy_version 29210 (0.0007) [2023-10-10 17:43:02,854][123582] Updated weights for policy 0, policy_version 29263 (0.0009) [2023-10-10 17:43:03,235][123582] Updated weights for policy 0, policy_version 29273 (0.0010) [2023-10-10 17:43:03,788][122664] Fps is (10 sec: 19661.1, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 59899904. Throughput: 0: 1814.6, 1: 1811.9. Samples: 14978090. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:43:03,788][122664] Avg episode reward: [(0, '39.000'), (1, '29.640')] [2023-10-10 17:43:06,414][123614] Updated weights for policy 1, policy_version 29220 (0.0009) [2023-10-10 17:43:06,746][123582] Updated weights for policy 0, policy_version 29283 (0.0009) [2023-10-10 17:43:06,792][123614] Updated weights for policy 1, policy_version 29230 (0.0009) [2023-10-10 17:43:07,131][123582] Updated weights for policy 0, policy_version 29293 (0.0008) [2023-10-10 17:43:07,150][123614] Updated weights for policy 1, policy_version 29240 (0.0007) [2023-10-10 17:43:07,503][123582] Updated weights for policy 0, policy_version 29303 (0.0009) [2023-10-10 17:43:08,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 59965440. Throughput: 0: 1815.7, 1: 1805.6. Samples: 14999268. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:43:08,789][122664] Avg episode reward: [(0, '37.010'), (1, '31.590')] [2023-10-10 17:43:10,886][123614] Updated weights for policy 1, policy_version 29250 (0.0007) [2023-10-10 17:43:11,142][123582] Updated weights for policy 0, policy_version 29313 (0.0008) [2023-10-10 17:43:11,257][123614] Updated weights for policy 1, policy_version 29260 (0.0009) [2023-10-10 17:43:11,505][123582] Updated weights for policy 0, policy_version 29323 (0.0008) [2023-10-10 17:43:11,624][123614] Updated weights for policy 1, policy_version 29270 (0.0008) [2023-10-10 17:43:11,879][123582] Updated weights for policy 0, policy_version 29333 (0.0008) [2023-10-10 17:43:12,003][123614] Updated weights for policy 1, policy_version 29280 (0.0007) [2023-10-10 17:43:12,253][123582] Updated weights for policy 0, policy_version 29343 (0.0008) [2023-10-10 17:43:13,788][122664] Fps is (10 sec: 13106.8, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 60030976. Throughput: 0: 1814.3, 1: 1812.5. Samples: 15010618. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:43:13,789][122664] Avg episode reward: [(0, '34.480'), (1, '33.650')] [2023-10-10 17:43:15,738][123614] Updated weights for policy 1, policy_version 29290 (0.0008) [2023-10-10 17:43:16,070][123582] Updated weights for policy 0, policy_version 29353 (0.0008) [2023-10-10 17:43:16,105][123614] Updated weights for policy 1, policy_version 29300 (0.0008) [2023-10-10 17:43:16,434][123582] Updated weights for policy 0, policy_version 29363 (0.0009) [2023-10-10 17:43:16,465][123614] Updated weights for policy 1, policy_version 29310 (0.0007) [2023-10-10 17:43:16,811][123582] Updated weights for policy 0, policy_version 29373 (0.0008) [2023-10-10 17:43:18,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 60096512. Throughput: 0: 1808.4, 1: 1811.7. Samples: 15031784. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:43:18,789][122664] Avg episode reward: [(0, '32.790'), (1, '35.260')] [2023-10-10 17:43:20,267][123614] Updated weights for policy 1, policy_version 29320 (0.0008) [2023-10-10 17:43:20,415][123582] Updated weights for policy 0, policy_version 29383 (0.0009) [2023-10-10 17:43:20,642][123614] Updated weights for policy 1, policy_version 29330 (0.0009) [2023-10-10 17:43:20,773][123582] Updated weights for policy 0, policy_version 29393 (0.0008) [2023-10-10 17:43:21,004][123614] Updated weights for policy 1, policy_version 29340 (0.0009) [2023-10-10 17:43:21,144][123582] Updated weights for policy 0, policy_version 29403 (0.0009) [2023-10-10 17:43:23,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 60162048. Throughput: 0: 1821.8, 1: 1804.0. Samples: 15054556. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:43:23,789][122664] Avg episode reward: [(0, '29.950'), (1, '37.760')] [2023-10-10 17:43:24,725][123614] Updated weights for policy 1, policy_version 29350 (0.0008) [2023-10-10 17:43:24,868][123582] Updated weights for policy 0, policy_version 29413 (0.0010) [2023-10-10 17:43:25,100][123614] Updated weights for policy 1, policy_version 29360 (0.0007) [2023-10-10 17:43:25,241][123582] Updated weights for policy 0, policy_version 29423 (0.0008) [2023-10-10 17:43:25,473][123614] Updated weights for policy 1, policy_version 29370 (0.0008) [2023-10-10 17:43:25,608][123582] Updated weights for policy 0, policy_version 29433 (0.0007) [2023-10-10 17:43:28,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 60227584. Throughput: 0: 1821.1, 1: 1801.4. Samples: 15064458. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 17:43:28,789][122664] Avg episode reward: [(0, '28.300'), (1, '37.690')] [2023-10-10 17:43:29,170][123614] Updated weights for policy 1, policy_version 29380 (0.0007) [2023-10-10 17:43:29,407][123582] Updated weights for policy 0, policy_version 29443 (0.0010) [2023-10-10 17:43:29,542][123614] Updated weights for policy 1, policy_version 29390 (0.0010) [2023-10-10 17:43:29,773][123582] Updated weights for policy 0, policy_version 29453 (0.0009) [2023-10-10 17:43:29,911][123614] Updated weights for policy 1, policy_version 29400 (0.0008) [2023-10-10 17:43:30,149][123582] Updated weights for policy 0, policy_version 29463 (0.0008) [2023-10-10 17:43:33,726][123614] Updated weights for policy 1, policy_version 29410 (0.0007) [2023-10-10 17:43:33,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 60293120. Throughput: 0: 1812.3, 1: 1805.9. Samples: 15086996. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 17:43:33,788][122664] Avg episode reward: [(0, '29.530'), (1, '37.130')] [2023-10-10 17:43:33,955][123582] Updated weights for policy 0, policy_version 29473 (0.0008) [2023-10-10 17:43:34,093][123614] Updated weights for policy 1, policy_version 29420 (0.0008) [2023-10-10 17:43:34,324][123582] Updated weights for policy 0, policy_version 29483 (0.0009) [2023-10-10 17:43:34,453][123614] Updated weights for policy 1, policy_version 29430 (0.0008) [2023-10-10 17:43:34,689][123582] Updated weights for policy 0, policy_version 29493 (0.0009) [2023-10-10 17:43:34,821][123614] Updated weights for policy 1, policy_version 29440 (0.0008) [2023-10-10 17:43:35,066][123582] Updated weights for policy 0, policy_version 29503 (0.0007) [2023-10-10 17:43:38,596][123582] Updated weights for policy 0, policy_version 29513 (0.0008) [2023-10-10 17:43:38,658][123614] Updated weights for policy 1, policy_version 29450 (0.0007) [2023-10-10 17:43:38,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 60358656. Throughput: 0: 1812.9, 1: 1818.8. Samples: 15108834. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 17:43:38,788][122664] Avg episode reward: [(0, '31.190'), (1, '36.490')] [2023-10-10 17:43:38,967][123582] Updated weights for policy 0, policy_version 29523 (0.0007) [2023-10-10 17:43:39,028][123614] Updated weights for policy 1, policy_version 29460 (0.0007) [2023-10-10 17:43:39,340][123582] Updated weights for policy 0, policy_version 29533 (0.0007) [2023-10-10 17:43:39,391][123614] Updated weights for policy 1, policy_version 29470 (0.0008) [2023-10-10 17:43:42,992][123582] Updated weights for policy 0, policy_version 29543 (0.0007) [2023-10-10 17:43:43,269][123614] Updated weights for policy 1, policy_version 29480 (0.0007) [2023-10-10 17:43:43,357][123582] Updated weights for policy 0, policy_version 29553 (0.0008) [2023-10-10 17:43:43,635][123614] Updated weights for policy 1, policy_version 29490 (0.0007) [2023-10-10 17:43:43,730][123582] Updated weights for policy 0, policy_version 29563 (0.0007) [2023-10-10 17:43:43,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 60424192. Throughput: 0: 1810.3, 1: 1801.1. Samples: 15119258. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 17:43:43,788][122664] Avg episode reward: [(0, '32.120'), (1, '36.770')] [2023-10-10 17:43:43,998][123614] Updated weights for policy 1, policy_version 29500 (0.0008) [2023-10-10 17:43:47,509][123614] Updated weights for policy 1, policy_version 29510 (0.0007) [2023-10-10 17:43:47,573][123582] Updated weights for policy 0, policy_version 29573 (0.0008) [2023-10-10 17:43:47,879][123614] Updated weights for policy 1, policy_version 29520 (0.0009) [2023-10-10 17:43:47,941][123582] Updated weights for policy 0, policy_version 29583 (0.0008) [2023-10-10 17:43:48,247][123614] Updated weights for policy 1, policy_version 29530 (0.0007) [2023-10-10 17:43:48,312][123582] Updated weights for policy 0, policy_version 29593 (0.0007) [2023-10-10 17:43:48,788][122664] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 60555264. Throughput: 0: 1814.9, 1: 1814.0. Samples: 15141392. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:43:48,789][122664] Avg episode reward: [(0, '33.990'), (1, '37.970')] [2023-10-10 17:43:51,980][123582] Updated weights for policy 0, policy_version 29603 (0.0008) [2023-10-10 17:43:52,126][123614] Updated weights for policy 1, policy_version 29540 (0.0008) [2023-10-10 17:43:52,370][123582] Updated weights for policy 0, policy_version 29613 (0.0009) [2023-10-10 17:43:52,502][123614] Updated weights for policy 1, policy_version 29550 (0.0007) [2023-10-10 17:43:52,742][123582] Updated weights for policy 0, policy_version 29623 (0.0008) [2023-10-10 17:43:52,870][123614] Updated weights for policy 1, policy_version 29560 (0.0008) [2023-10-10 17:43:53,788][122664] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 60620800. Throughput: 0: 1807.3, 1: 1799.4. Samples: 15161568. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:43:53,789][122664] Avg episode reward: [(0, '33.560'), (1, '39.280')] [2023-10-10 17:43:53,798][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000029568_30277632.pth... [2023-10-10 17:43:53,798][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000029632_30343168.pth... [2023-10-10 17:43:53,843][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000027936_28606464.pth [2023-10-10 17:43:53,844][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000027872_28540928.pth [2023-10-10 17:43:56,406][123582] Updated weights for policy 0, policy_version 29633 (0.0007) [2023-10-10 17:43:56,603][123614] Updated weights for policy 1, policy_version 29570 (0.0007) [2023-10-10 17:43:56,778][123582] Updated weights for policy 0, policy_version 29643 (0.0007) [2023-10-10 17:43:56,985][123614] Updated weights for policy 1, policy_version 29580 (0.0010) [2023-10-10 17:43:57,157][123582] Updated weights for policy 0, policy_version 29653 (0.0008) [2023-10-10 17:43:57,343][123614] Updated weights for policy 1, policy_version 29590 (0.0009) [2023-10-10 17:43:57,532][123582] Updated weights for policy 0, policy_version 29663 (0.0008) [2023-10-10 17:43:57,710][123614] Updated weights for policy 1, policy_version 29600 (0.0007) [2023-10-10 17:43:58,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 60686336. Throughput: 0: 1817.1, 1: 1814.9. Samples: 15174054. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:43:58,788][122664] Avg episode reward: [(0, '33.760'), (1, '36.460')] [2023-10-10 17:44:01,326][123582] Updated weights for policy 0, policy_version 29673 (0.0008) [2023-10-10 17:44:01,346][123614] Updated weights for policy 1, policy_version 29610 (0.0009) [2023-10-10 17:44:01,690][123582] Updated weights for policy 0, policy_version 29683 (0.0009) [2023-10-10 17:44:01,723][123614] Updated weights for policy 1, policy_version 29620 (0.0008) [2023-10-10 17:44:02,066][123582] Updated weights for policy 0, policy_version 29693 (0.0009) [2023-10-10 17:44:02,087][123614] Updated weights for policy 1, policy_version 29630 (0.0008) [2023-10-10 17:44:03,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 60751872. Throughput: 0: 1811.6, 1: 1789.5. Samples: 15193834. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:44:03,789][122664] Avg episode reward: [(0, '34.590'), (1, '37.550')] [2023-10-10 17:44:05,805][123614] Updated weights for policy 1, policy_version 29640 (0.0009) [2023-10-10 17:44:05,811][123582] Updated weights for policy 0, policy_version 29703 (0.0007) [2023-10-10 17:44:06,167][123614] Updated weights for policy 1, policy_version 29650 (0.0008) [2023-10-10 17:44:06,178][123582] Updated weights for policy 0, policy_version 29713 (0.0009) [2023-10-10 17:44:06,530][123614] Updated weights for policy 1, policy_version 29660 (0.0007) [2023-10-10 17:44:06,542][123582] Updated weights for policy 0, policy_version 29723 (0.0007) [2023-10-10 17:44:08,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 60817408. Throughput: 0: 1805.0, 1: 1795.6. Samples: 15216580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:44:08,788][122664] Avg episode reward: [(0, '35.440'), (1, '38.760')] [2023-10-10 17:44:10,121][123582] Updated weights for policy 0, policy_version 29733 (0.0007) [2023-10-10 17:44:10,456][123614] Updated weights for policy 1, policy_version 29670 (0.0007) [2023-10-10 17:44:10,487][123582] Updated weights for policy 0, policy_version 29743 (0.0009) [2023-10-10 17:44:10,823][123614] Updated weights for policy 1, policy_version 29680 (0.0009) [2023-10-10 17:44:10,868][123582] Updated weights for policy 0, policy_version 29753 (0.0008) [2023-10-10 17:44:11,182][123614] Updated weights for policy 1, policy_version 29690 (0.0009) [2023-10-10 17:44:13,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 60882944. Throughput: 0: 1806.7, 1: 1789.5. Samples: 15226284. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 17:44:13,788][122664] Avg episode reward: [(0, '35.900'), (1, '37.890')] [2023-10-10 17:44:14,413][123582] Updated weights for policy 0, policy_version 29763 (0.0008) [2023-10-10 17:44:14,784][123582] Updated weights for policy 0, policy_version 29773 (0.0008) [2023-10-10 17:44:14,834][123614] Updated weights for policy 1, policy_version 29700 (0.0009) [2023-10-10 17:44:15,161][123582] Updated weights for policy 0, policy_version 29783 (0.0007) [2023-10-10 17:44:15,205][123614] Updated weights for policy 1, policy_version 29710 (0.0007) [2023-10-10 17:44:15,569][123614] Updated weights for policy 1, policy_version 29720 (0.0007) [2023-10-10 17:44:18,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 60948480. Throughput: 0: 1813.7, 1: 1791.7. Samples: 15249238. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 17:44:18,788][122664] Avg episode reward: [(0, '34.460'), (1, '37.450')] [2023-10-10 17:44:18,973][123582] Updated weights for policy 0, policy_version 29793 (0.0007) [2023-10-10 17:44:19,216][123614] Updated weights for policy 1, policy_version 29730 (0.0007) [2023-10-10 17:44:19,346][123582] Updated weights for policy 0, policy_version 29803 (0.0011) [2023-10-10 17:44:19,583][123614] Updated weights for policy 1, policy_version 29740 (0.0008) [2023-10-10 17:44:19,715][123582] Updated weights for policy 0, policy_version 29813 (0.0009) [2023-10-10 17:44:19,955][123614] Updated weights for policy 1, policy_version 29750 (0.0010) [2023-10-10 17:44:20,099][123582] Updated weights for policy 0, policy_version 29823 (0.0010) [2023-10-10 17:44:20,325][123614] Updated weights for policy 1, policy_version 29760 (0.0008) [2023-10-10 17:44:23,784][123582] Updated weights for policy 0, policy_version 29833 (0.0009) [2023-10-10 17:44:23,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 61014016. Throughput: 0: 1818.7, 1: 1804.8. Samples: 15271890. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 17:44:23,788][122664] Avg episode reward: [(0, '34.360'), (1, '39.130')] [2023-10-10 17:44:24,017][123614] Updated weights for policy 1, policy_version 29770 (0.0008) [2023-10-10 17:44:24,160][123582] Updated weights for policy 0, policy_version 29843 (0.0008) [2023-10-10 17:44:24,384][123614] Updated weights for policy 1, policy_version 29780 (0.0010) [2023-10-10 17:44:24,522][123582] Updated weights for policy 0, policy_version 29853 (0.0008) [2023-10-10 17:44:24,753][123614] Updated weights for policy 1, policy_version 29790 (0.0009) [2023-10-10 17:44:28,200][123582] Updated weights for policy 0, policy_version 29863 (0.0008) [2023-10-10 17:44:28,371][123614] Updated weights for policy 1, policy_version 29800 (0.0008) [2023-10-10 17:44:28,564][123582] Updated weights for policy 0, policy_version 29873 (0.0008) [2023-10-10 17:44:28,739][123614] Updated weights for policy 1, policy_version 29810 (0.0008) [2023-10-10 17:44:28,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 61079552. Throughput: 0: 1816.8, 1: 1800.2. Samples: 15282024. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 17:44:28,788][122664] Avg episode reward: [(0, '36.370'), (1, '36.170')] [2023-10-10 17:44:28,935][123582] Updated weights for policy 0, policy_version 29883 (0.0008) [2023-10-10 17:44:29,107][123614] Updated weights for policy 1, policy_version 29820 (0.0008) [2023-10-10 17:44:32,617][123582] Updated weights for policy 0, policy_version 29893 (0.0007) [2023-10-10 17:44:32,852][123614] Updated weights for policy 1, policy_version 29830 (0.0008) [2023-10-10 17:44:32,988][123582] Updated weights for policy 0, policy_version 29903 (0.0007) [2023-10-10 17:44:33,222][123614] Updated weights for policy 1, policy_version 29840 (0.0008) [2023-10-10 17:44:33,361][123582] Updated weights for policy 0, policy_version 29913 (0.0007) [2023-10-10 17:44:33,582][123614] Updated weights for policy 1, policy_version 29850 (0.0009) [2023-10-10 17:44:33,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 61177856. Throughput: 0: 1823.2, 1: 1808.9. Samples: 15304834. Policy #0 lag: (min: 12.0, avg: 23.5, max: 44.0) [2023-10-10 17:44:33,788][122664] Avg episode reward: [(0, '36.310'), (1, '33.370')] [2023-10-10 17:44:37,086][123582] Updated weights for policy 0, policy_version 29923 (0.0008) [2023-10-10 17:44:37,257][123614] Updated weights for policy 1, policy_version 29860 (0.0009) [2023-10-10 17:44:37,491][123582] Updated weights for policy 0, policy_version 29933 (0.0008) [2023-10-10 17:44:37,647][123614] Updated weights for policy 1, policy_version 29870 (0.0008) [2023-10-10 17:44:37,861][123582] Updated weights for policy 0, policy_version 29943 (0.0008) [2023-10-10 17:44:38,020][123614] Updated weights for policy 1, policy_version 29880 (0.0010) [2023-10-10 17:44:38,788][122664] Fps is (10 sec: 19660.3, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 61276160. Throughput: 0: 1821.0, 1: 1805.1. Samples: 15324740. Policy #0 lag: (min: 12.0, avg: 23.5, max: 44.0) [2023-10-10 17:44:38,789][122664] Avg episode reward: [(0, '35.960'), (1, '33.420')] [2023-10-10 17:44:41,484][123582] Updated weights for policy 0, policy_version 29953 (0.0007) [2023-10-10 17:44:41,725][123614] Updated weights for policy 1, policy_version 29890 (0.0007) [2023-10-10 17:44:41,854][123582] Updated weights for policy 0, policy_version 29963 (0.0009) [2023-10-10 17:44:42,101][123614] Updated weights for policy 1, policy_version 29900 (0.0007) [2023-10-10 17:44:42,226][123582] Updated weights for policy 0, policy_version 29973 (0.0007) [2023-10-10 17:44:42,473][123614] Updated weights for policy 1, policy_version 29910 (0.0008) [2023-10-10 17:44:42,599][123582] Updated weights for policy 0, policy_version 29983 (0.0008) [2023-10-10 17:44:42,842][123614] Updated weights for policy 1, policy_version 29920 (0.0009) [2023-10-10 17:44:43,788][122664] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 61341696. Throughput: 0: 1824.2, 1: 1808.9. Samples: 15337546. Policy #0 lag: (min: 12.0, avg: 23.5, max: 44.0) [2023-10-10 17:44:43,789][122664] Avg episode reward: [(0, '36.360'), (1, '33.220')] [2023-10-10 17:44:46,298][123582] Updated weights for policy 0, policy_version 29993 (0.0007) [2023-10-10 17:44:46,484][123614] Updated weights for policy 1, policy_version 29930 (0.0007) [2023-10-10 17:44:46,674][123582] Updated weights for policy 0, policy_version 30003 (0.0008) [2023-10-10 17:44:46,852][123614] Updated weights for policy 1, policy_version 29940 (0.0008) [2023-10-10 17:44:47,034][123582] Updated weights for policy 0, policy_version 30013 (0.0009) [2023-10-10 17:44:47,217][123614] Updated weights for policy 1, policy_version 29950 (0.0010) [2023-10-10 17:44:48,788][122664] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 61407232. Throughput: 0: 1820.8, 1: 1810.6. Samples: 15357248. Policy #0 lag: (min: 12.0, avg: 23.5, max: 44.0) [2023-10-10 17:44:48,791][122664] Avg episode reward: [(0, '35.800'), (1, '32.440')] [2023-10-10 17:44:50,800][123582] Updated weights for policy 0, policy_version 30023 (0.0008) [2023-10-10 17:44:50,897][123614] Updated weights for policy 1, policy_version 29960 (0.0007) [2023-10-10 17:44:51,163][123582] Updated weights for policy 0, policy_version 30033 (0.0008) [2023-10-10 17:44:51,267][123614] Updated weights for policy 1, policy_version 29970 (0.0007) [2023-10-10 17:44:51,541][123582] Updated weights for policy 0, policy_version 30043 (0.0007) [2023-10-10 17:44:51,633][123614] Updated weights for policy 1, policy_version 29980 (0.0007) [2023-10-10 17:44:53,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 61472768. Throughput: 0: 1820.9, 1: 1809.1. Samples: 15379932. Policy #0 lag: (min: 12.0, avg: 23.5, max: 44.0) [2023-10-10 17:44:53,789][122664] Avg episode reward: [(0, '36.490'), (1, '32.560')] [2023-10-10 17:44:55,202][123582] Updated weights for policy 0, policy_version 30053 (0.0007) [2023-10-10 17:44:55,311][123614] Updated weights for policy 1, policy_version 29990 (0.0007) [2023-10-10 17:44:55,568][123582] Updated weights for policy 0, policy_version 30063 (0.0007) [2023-10-10 17:44:55,687][123614] Updated weights for policy 1, policy_version 30000 (0.0009) [2023-10-10 17:44:55,933][123582] Updated weights for policy 0, policy_version 30073 (0.0007) [2023-10-10 17:44:56,046][123614] Updated weights for policy 1, policy_version 30010 (0.0007) [2023-10-10 17:44:58,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 61538304. Throughput: 0: 1820.0, 1: 1812.3. Samples: 15389734. Policy #0 lag: (min: 30.0, avg: 34.4, max: 62.0) [2023-10-10 17:44:58,789][122664] Avg episode reward: [(0, '35.060'), (1, '33.440')] [2023-10-10 17:44:59,784][123582] Updated weights for policy 0, policy_version 30083 (0.0008) [2023-10-10 17:44:59,837][123614] Updated weights for policy 1, policy_version 30020 (0.0007) [2023-10-10 17:45:00,154][123582] Updated weights for policy 0, policy_version 30093 (0.0009) [2023-10-10 17:45:00,208][123614] Updated weights for policy 1, policy_version 30030 (0.0008) [2023-10-10 17:45:00,520][123582] Updated weights for policy 0, policy_version 30103 (0.0007) [2023-10-10 17:45:00,567][123614] Updated weights for policy 1, policy_version 30040 (0.0008) [2023-10-10 17:45:03,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 61603840. Throughput: 0: 1812.9, 1: 1813.2. Samples: 15412412. Policy #0 lag: (min: 30.0, avg: 34.4, max: 62.0) [2023-10-10 17:45:03,789][122664] Avg episode reward: [(0, '34.250'), (1, '31.490')] [2023-10-10 17:45:04,090][123614] Updated weights for policy 1, policy_version 30050 (0.0007) [2023-10-10 17:45:04,111][123582] Updated weights for policy 0, policy_version 30113 (0.0010) [2023-10-10 17:45:04,458][123614] Updated weights for policy 1, policy_version 30060 (0.0007) [2023-10-10 17:45:04,473][123582] Updated weights for policy 0, policy_version 30123 (0.0008) [2023-10-10 17:45:04,821][123614] Updated weights for policy 1, policy_version 30070 (0.0008) [2023-10-10 17:45:04,855][123582] Updated weights for policy 0, policy_version 30133 (0.0009) [2023-10-10 17:45:05,197][123614] Updated weights for policy 1, policy_version 30080 (0.0009) [2023-10-10 17:45:05,230][123582] Updated weights for policy 0, policy_version 30143 (0.0007) [2023-10-10 17:45:08,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 61669376. Throughput: 0: 1819.0, 1: 1813.1. Samples: 15435332. Policy #0 lag: (min: 30.0, avg: 34.4, max: 62.0) [2023-10-10 17:45:08,788][122664] Avg episode reward: [(0, '33.560'), (1, '33.760')] [2023-10-10 17:45:08,934][123614] Updated weights for policy 1, policy_version 30090 (0.0007) [2023-10-10 17:45:08,988][123582] Updated weights for policy 0, policy_version 30153 (0.0008) [2023-10-10 17:45:09,291][123614] Updated weights for policy 1, policy_version 30100 (0.0010) [2023-10-10 17:45:09,356][123582] Updated weights for policy 0, policy_version 30163 (0.0007) [2023-10-10 17:45:09,659][123614] Updated weights for policy 1, policy_version 30110 (0.0007) [2023-10-10 17:45:09,726][123582] Updated weights for policy 0, policy_version 30173 (0.0008) [2023-10-10 17:45:13,369][123614] Updated weights for policy 1, policy_version 30120 (0.0007) [2023-10-10 17:45:13,442][123582] Updated weights for policy 0, policy_version 30183 (0.0009) [2023-10-10 17:45:13,730][123614] Updated weights for policy 1, policy_version 30130 (0.0008) [2023-10-10 17:45:13,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 61734912. Throughput: 0: 1815.3, 1: 1814.4. Samples: 15445364. Policy #0 lag: (min: 30.0, avg: 34.4, max: 62.0) [2023-10-10 17:45:13,789][122664] Avg episode reward: [(0, '35.200'), (1, '34.790')] [2023-10-10 17:45:13,814][123582] Updated weights for policy 0, policy_version 30193 (0.0007) [2023-10-10 17:45:14,092][123614] Updated weights for policy 1, policy_version 30140 (0.0007) [2023-10-10 17:45:14,190][123582] Updated weights for policy 0, policy_version 30203 (0.0007) [2023-10-10 17:45:17,872][123582] Updated weights for policy 0, policy_version 30213 (0.0007) [2023-10-10 17:45:17,962][123614] Updated weights for policy 1, policy_version 30150 (0.0009) [2023-10-10 17:45:18,247][123582] Updated weights for policy 0, policy_version 30223 (0.0008) [2023-10-10 17:45:18,323][123614] Updated weights for policy 1, policy_version 30160 (0.0007) [2023-10-10 17:45:18,630][123582] Updated weights for policy 0, policy_version 30233 (0.0007) [2023-10-10 17:45:18,709][123614] Updated weights for policy 1, policy_version 30170 (0.0007) [2023-10-10 17:45:18,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 61800448. Throughput: 0: 1809.6, 1: 1816.1. Samples: 15467990. Policy #0 lag: (min: 30.0, avg: 34.4, max: 62.0) [2023-10-10 17:45:18,788][122664] Avg episode reward: [(0, '35.450'), (1, '33.920')] [2023-10-10 17:45:22,303][123582] Updated weights for policy 0, policy_version 30243 (0.0009) [2023-10-10 17:45:22,636][123614] Updated weights for policy 1, policy_version 30180 (0.0008) [2023-10-10 17:45:22,705][123582] Updated weights for policy 0, policy_version 30253 (0.0008) [2023-10-10 17:45:23,027][123614] Updated weights for policy 1, policy_version 30190 (0.0007) [2023-10-10 17:45:23,077][123582] Updated weights for policy 0, policy_version 30263 (0.0007) [2023-10-10 17:45:23,391][123614] Updated weights for policy 1, policy_version 30200 (0.0007) [2023-10-10 17:45:23,788][122664] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 61931520. Throughput: 0: 1805.8, 1: 1805.8. Samples: 15487260. Policy #0 lag: (min: 10.0, avg: 13.8, max: 42.0) [2023-10-10 17:45:23,788][122664] Avg episode reward: [(0, '34.750'), (1, '39.110')] [2023-10-10 17:45:26,641][123582] Updated weights for policy 0, policy_version 30273 (0.0008) [2023-10-10 17:45:27,023][123582] Updated weights for policy 0, policy_version 30283 (0.0008) [2023-10-10 17:45:27,179][123614] Updated weights for policy 1, policy_version 30210 (0.0007) [2023-10-10 17:45:27,386][123582] Updated weights for policy 0, policy_version 30293 (0.0008) [2023-10-10 17:45:27,542][123614] Updated weights for policy 1, policy_version 30220 (0.0008) [2023-10-10 17:45:27,750][123582] Updated weights for policy 0, policy_version 30303 (0.0007) [2023-10-10 17:45:27,907][123614] Updated weights for policy 1, policy_version 30230 (0.0008) [2023-10-10 17:45:28,275][123614] Updated weights for policy 1, policy_version 30240 (0.0009) [2023-10-10 17:45:28,788][122664] Fps is (10 sec: 19660.2, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 61997056. Throughput: 0: 1805.6, 1: 1810.3. Samples: 15500258. Policy #0 lag: (min: 10.0, avg: 13.8, max: 42.0) [2023-10-10 17:45:28,789][122664] Avg episode reward: [(0, '37.620'), (1, '39.450')] [2023-10-10 17:45:31,465][123582] Updated weights for policy 0, policy_version 30313 (0.0007) [2023-10-10 17:45:31,837][123582] Updated weights for policy 0, policy_version 30323 (0.0008) [2023-10-10 17:45:31,986][123614] Updated weights for policy 1, policy_version 30250 (0.0008) [2023-10-10 17:45:32,207][123582] Updated weights for policy 0, policy_version 30333 (0.0007) [2023-10-10 17:45:32,358][123614] Updated weights for policy 1, policy_version 30260 (0.0008) [2023-10-10 17:45:32,723][123614] Updated weights for policy 1, policy_version 30270 (0.0008) [2023-10-10 17:45:33,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 62062592. Throughput: 0: 1807.8, 1: 1807.5. Samples: 15519936. Policy #0 lag: (min: 10.0, avg: 13.8, max: 42.0) [2023-10-10 17:45:33,789][122664] Avg episode reward: [(0, '41.560'), (1, '40.570')] [2023-10-10 17:45:35,868][123582] Updated weights for policy 0, policy_version 30343 (0.0007) [2023-10-10 17:45:36,237][123582] Updated weights for policy 0, policy_version 30353 (0.0007) [2023-10-10 17:45:36,509][123614] Updated weights for policy 1, policy_version 30280 (0.0008) [2023-10-10 17:45:36,607][123582] Updated weights for policy 0, policy_version 30363 (0.0008) [2023-10-10 17:45:36,878][123614] Updated weights for policy 1, policy_version 30290 (0.0009) [2023-10-10 17:45:37,237][123614] Updated weights for policy 1, policy_version 30300 (0.0009) [2023-10-10 17:45:38,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 62128128. Throughput: 0: 1805.7, 1: 1797.8. Samples: 15542088. Policy #0 lag: (min: 10.0, avg: 13.8, max: 42.0) [2023-10-10 17:45:38,788][122664] Avg episode reward: [(0, '38.780'), (1, '41.920')] [2023-10-10 17:45:40,266][123582] Updated weights for policy 0, policy_version 30373 (0.0008) [2023-10-10 17:45:40,637][123582] Updated weights for policy 0, policy_version 30383 (0.0008) [2023-10-10 17:45:40,899][123614] Updated weights for policy 1, policy_version 30310 (0.0009) [2023-10-10 17:45:41,005][123582] Updated weights for policy 0, policy_version 30393 (0.0007) [2023-10-10 17:45:41,273][123614] Updated weights for policy 1, policy_version 30320 (0.0007) [2023-10-10 17:45:41,633][123614] Updated weights for policy 1, policy_version 30330 (0.0007) [2023-10-10 17:45:43,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 62193664. Throughput: 0: 1810.0, 1: 1805.7. Samples: 15552442. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-10 17:45:43,789][122664] Avg episode reward: [(0, '39.780'), (1, '41.540')] [2023-10-10 17:45:44,810][123582] Updated weights for policy 0, policy_version 30403 (0.0009) [2023-10-10 17:45:45,189][123582] Updated weights for policy 0, policy_version 30413 (0.0008) [2023-10-10 17:45:45,381][123614] Updated weights for policy 1, policy_version 30340 (0.0008) [2023-10-10 17:45:45,556][123582] Updated weights for policy 0, policy_version 30423 (0.0007) [2023-10-10 17:45:45,742][123614] Updated weights for policy 1, policy_version 30350 (0.0009) [2023-10-10 17:45:46,114][123614] Updated weights for policy 1, policy_version 30360 (0.0010) [2023-10-10 17:45:48,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 62259200. Throughput: 0: 1816.8, 1: 1793.5. Samples: 15574876. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-10 17:45:48,789][122664] Avg episode reward: [(0, '40.350'), (1, '40.840')] [2023-10-10 17:45:49,158][123582] Updated weights for policy 0, policy_version 30433 (0.0007) [2023-10-10 17:45:49,526][123582] Updated weights for policy 0, policy_version 30443 (0.0007) [2023-10-10 17:45:49,830][123614] Updated weights for policy 1, policy_version 30370 (0.0008) [2023-10-10 17:45:49,905][123582] Updated weights for policy 0, policy_version 30453 (0.0010) [2023-10-10 17:45:50,207][123614] Updated weights for policy 1, policy_version 30380 (0.0007) [2023-10-10 17:45:50,277][123582] Updated weights for policy 0, policy_version 30463 (0.0009) [2023-10-10 17:45:50,575][123614] Updated weights for policy 1, policy_version 30390 (0.0010) [2023-10-10 17:45:50,938][123614] Updated weights for policy 1, policy_version 30400 (0.0007) [2023-10-10 17:45:53,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 62324736. Throughput: 0: 1815.5, 1: 1796.6. Samples: 15597880. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-10 17:45:53,789][122664] Avg episode reward: [(0, '40.290'), (1, '43.810')] [2023-10-10 17:45:53,796][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000030400_31129600.pth... [2023-10-10 17:45:53,829][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000028704_29392896.pth [2023-10-10 17:45:53,833][123465] Saving new best policy, reward=43.810! [2023-10-10 17:45:53,849][123582] Updated weights for policy 0, policy_version 30473 (0.0008) [2023-10-10 17:45:54,227][123582] Updated weights for policy 0, policy_version 30483 (0.0008) [2023-10-10 17:45:54,540][123614] Updated weights for policy 1, policy_version 30410 (0.0009) [2023-10-10 17:45:54,597][123582] Updated weights for policy 0, policy_version 30493 (0.0008) [2023-10-10 17:45:54,706][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000030496_31227904.pth... [2023-10-10 17:45:54,737][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000028768_29458432.pth [2023-10-10 17:45:54,904][123614] Updated weights for policy 1, policy_version 30420 (0.0009) [2023-10-10 17:45:55,271][123614] Updated weights for policy 1, policy_version 30430 (0.0011) [2023-10-10 17:45:58,255][123582] Updated weights for policy 0, policy_version 30503 (0.0007) [2023-10-10 17:45:58,633][123582] Updated weights for policy 0, policy_version 30513 (0.0008) [2023-10-10 17:45:58,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 62390272. Throughput: 0: 1821.6, 1: 1792.9. Samples: 15608016. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-10 17:45:58,788][122664] Avg episode reward: [(0, '39.460'), (1, '43.630')] [2023-10-10 17:45:58,943][123614] Updated weights for policy 1, policy_version 30440 (0.0009) [2023-10-10 17:45:58,997][123582] Updated weights for policy 0, policy_version 30523 (0.0009) [2023-10-10 17:45:59,315][123614] Updated weights for policy 1, policy_version 30450 (0.0009) [2023-10-10 17:45:59,677][123614] Updated weights for policy 1, policy_version 30460 (0.0008) [2023-10-10 17:46:02,653][123582] Updated weights for policy 0, policy_version 30533 (0.0008) [2023-10-10 17:46:03,023][123582] Updated weights for policy 0, policy_version 30543 (0.0008) [2023-10-10 17:46:03,385][123582] Updated weights for policy 0, policy_version 30553 (0.0009) [2023-10-10 17:46:03,411][123614] Updated weights for policy 1, policy_version 30470 (0.0009) [2023-10-10 17:46:03,782][123614] Updated weights for policy 1, policy_version 30480 (0.0008) [2023-10-10 17:46:03,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 62488576. Throughput: 0: 1823.9, 1: 1794.7. Samples: 15630832. Policy #0 lag: (min: 28.0, avg: 34.3, max: 60.0) [2023-10-10 17:46:03,789][122664] Avg episode reward: [(0, '39.960'), (1, '44.930')] [2023-10-10 17:46:04,145][123614] Updated weights for policy 1, policy_version 30490 (0.0008) [2023-10-10 17:46:04,361][123465] Saving new best policy, reward=44.930! [2023-10-10 17:46:07,243][123582] Updated weights for policy 0, policy_version 30563 (0.0009) [2023-10-10 17:46:07,633][123582] Updated weights for policy 0, policy_version 30573 (0.0011) [2023-10-10 17:46:07,860][123614] Updated weights for policy 1, policy_version 30500 (0.0009) [2023-10-10 17:46:08,017][123582] Updated weights for policy 0, policy_version 30583 (0.0009) [2023-10-10 17:46:08,248][123614] Updated weights for policy 1, policy_version 30510 (0.0007) [2023-10-10 17:46:08,616][123614] Updated weights for policy 1, policy_version 30520 (0.0007) [2023-10-10 17:46:08,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 62554112. Throughput: 0: 1826.0, 1: 1808.0. Samples: 15650788. Policy #0 lag: (min: 28.0, avg: 34.3, max: 60.0) [2023-10-10 17:46:08,788][122664] Avg episode reward: [(0, '37.940'), (1, '45.550')] [2023-10-10 17:46:08,908][123465] Saving new best policy, reward=45.550! [2023-10-10 17:46:11,648][123582] Updated weights for policy 0, policy_version 30593 (0.0008) [2023-10-10 17:46:12,024][123582] Updated weights for policy 0, policy_version 30603 (0.0011) [2023-10-10 17:46:12,305][123614] Updated weights for policy 1, policy_version 30530 (0.0010) [2023-10-10 17:46:12,403][123582] Updated weights for policy 0, policy_version 30613 (0.0011) [2023-10-10 17:46:12,673][123614] Updated weights for policy 1, policy_version 30540 (0.0008) [2023-10-10 17:46:12,775][123582] Updated weights for policy 0, policy_version 30623 (0.0008) [2023-10-10 17:46:13,044][123614] Updated weights for policy 1, policy_version 30550 (0.0009) [2023-10-10 17:46:13,419][123614] Updated weights for policy 1, policy_version 30560 (0.0008) [2023-10-10 17:46:13,788][122664] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 62652416. Throughput: 0: 1819.3, 1: 1796.8. Samples: 15662984. Policy #0 lag: (min: 28.0, avg: 34.3, max: 60.0) [2023-10-10 17:46:13,789][122664] Avg episode reward: [(0, '36.480'), (1, '41.160')] [2023-10-10 17:46:16,569][123582] Updated weights for policy 0, policy_version 30633 (0.0009) [2023-10-10 17:46:16,939][123582] Updated weights for policy 0, policy_version 30643 (0.0007) [2023-10-10 17:46:17,279][123614] Updated weights for policy 1, policy_version 30570 (0.0008) [2023-10-10 17:46:17,316][123582] Updated weights for policy 0, policy_version 30653 (0.0008) [2023-10-10 17:46:17,653][123614] Updated weights for policy 1, policy_version 30580 (0.0007) [2023-10-10 17:46:18,015][123614] Updated weights for policy 1, policy_version 30590 (0.0009) [2023-10-10 17:46:18,788][122664] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 62717952. Throughput: 0: 1817.9, 1: 1804.0. Samples: 15682922. Policy #0 lag: (min: 28.0, avg: 34.3, max: 60.0) [2023-10-10 17:46:18,789][122664] Avg episode reward: [(0, '35.300'), (1, '42.250')] [2023-10-10 17:46:20,832][123582] Updated weights for policy 0, policy_version 30663 (0.0010) [2023-10-10 17:46:21,209][123582] Updated weights for policy 0, policy_version 30673 (0.0007) [2023-10-10 17:46:21,590][123582] Updated weights for policy 0, policy_version 30683 (0.0008) [2023-10-10 17:46:21,796][123614] Updated weights for policy 1, policy_version 30600 (0.0008) [2023-10-10 17:46:22,157][123614] Updated weights for policy 1, policy_version 30610 (0.0010) [2023-10-10 17:46:22,525][123614] Updated weights for policy 1, policy_version 30620 (0.0008) [2023-10-10 17:46:23,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 62783488. Throughput: 0: 1824.8, 1: 1802.5. Samples: 15705320. Policy #0 lag: (min: 28.0, avg: 34.3, max: 60.0) [2023-10-10 17:46:23,789][122664] Avg episode reward: [(0, '34.700'), (1, '39.510')] [2023-10-10 17:46:25,387][123582] Updated weights for policy 0, policy_version 30693 (0.0009) [2023-10-10 17:46:25,757][123582] Updated weights for policy 0, policy_version 30703 (0.0007) [2023-10-10 17:46:26,136][123582] Updated weights for policy 0, policy_version 30713 (0.0010) [2023-10-10 17:46:26,258][123614] Updated weights for policy 1, policy_version 30630 (0.0009) [2023-10-10 17:46:26,628][123614] Updated weights for policy 1, policy_version 30640 (0.0010) [2023-10-10 17:46:26,997][123614] Updated weights for policy 1, policy_version 30650 (0.0009) [2023-10-10 17:46:28,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 62849024. Throughput: 0: 1817.9, 1: 1808.8. Samples: 15715642. Policy #0 lag: (min: 31.0, avg: 32.0, max: 53.0) [2023-10-10 17:46:28,788][122664] Avg episode reward: [(0, '38.850'), (1, '38.180')] [2023-10-10 17:46:29,727][123582] Updated weights for policy 0, policy_version 30723 (0.0008) [2023-10-10 17:46:30,098][123582] Updated weights for policy 0, policy_version 30733 (0.0010) [2023-10-10 17:46:30,481][123582] Updated weights for policy 0, policy_version 30743 (0.0009) [2023-10-10 17:46:30,789][123614] Updated weights for policy 1, policy_version 30660 (0.0008) [2023-10-10 17:46:31,157][123614] Updated weights for policy 1, policy_version 30670 (0.0009) [2023-10-10 17:46:31,533][123614] Updated weights for policy 1, policy_version 30680 (0.0009) [2023-10-10 17:46:33,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 62914560. Throughput: 0: 1819.8, 1: 1796.5. Samples: 15737610. Policy #0 lag: (min: 31.0, avg: 32.0, max: 53.0) [2023-10-10 17:46:33,789][122664] Avg episode reward: [(0, '39.440'), (1, '38.460')] [2023-10-10 17:46:34,320][123582] Updated weights for policy 0, policy_version 30753 (0.0008) [2023-10-10 17:46:34,685][123582] Updated weights for policy 0, policy_version 30763 (0.0012) [2023-10-10 17:46:35,052][123582] Updated weights for policy 0, policy_version 30773 (0.0008) [2023-10-10 17:46:35,369][123614] Updated weights for policy 1, policy_version 30690 (0.0009) [2023-10-10 17:46:35,426][123582] Updated weights for policy 0, policy_version 30783 (0.0008) [2023-10-10 17:46:35,748][123614] Updated weights for policy 1, policy_version 30700 (0.0008) [2023-10-10 17:46:36,125][123614] Updated weights for policy 1, policy_version 30710 (0.0008) [2023-10-10 17:46:36,503][123614] Updated weights for policy 1, policy_version 30720 (0.0010) [2023-10-10 17:46:38,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 62980096. Throughput: 0: 1815.6, 1: 1795.2. Samples: 15760370. Policy #0 lag: (min: 31.0, avg: 32.0, max: 53.0) [2023-10-10 17:46:38,789][122664] Avg episode reward: [(0, '42.650'), (1, '36.600')] [2023-10-10 17:46:39,140][123582] Updated weights for policy 0, policy_version 30793 (0.0008) [2023-10-10 17:46:39,521][123582] Updated weights for policy 0, policy_version 30803 (0.0008) [2023-10-10 17:46:39,897][123582] Updated weights for policy 0, policy_version 30813 (0.0009) [2023-10-10 17:46:40,226][123614] Updated weights for policy 1, policy_version 30730 (0.0010) [2023-10-10 17:46:40,597][123614] Updated weights for policy 1, policy_version 30740 (0.0007) [2023-10-10 17:46:40,953][123614] Updated weights for policy 1, policy_version 30750 (0.0009) [2023-10-10 17:46:43,568][123582] Updated weights for policy 0, policy_version 30823 (0.0009) [2023-10-10 17:46:43,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 63045632. Throughput: 0: 1811.3, 1: 1793.2. Samples: 15770220. Policy #0 lag: (min: 31.0, avg: 32.0, max: 53.0) [2023-10-10 17:46:43,789][122664] Avg episode reward: [(0, '43.380'), (1, '39.210')] [2023-10-10 17:46:43,936][123582] Updated weights for policy 0, policy_version 30833 (0.0009) [2023-10-10 17:46:44,318][123582] Updated weights for policy 0, policy_version 30843 (0.0009) [2023-10-10 17:46:44,581][123614] Updated weights for policy 1, policy_version 30760 (0.0008) [2023-10-10 17:46:44,948][123614] Updated weights for policy 1, policy_version 30770 (0.0008) [2023-10-10 17:46:45,308][123614] Updated weights for policy 1, policy_version 30780 (0.0009) [2023-10-10 17:46:47,953][123582] Updated weights for policy 0, policy_version 30853 (0.0010) [2023-10-10 17:46:48,327][123582] Updated weights for policy 0, policy_version 30863 (0.0008) [2023-10-10 17:46:48,700][123582] Updated weights for policy 0, policy_version 30873 (0.0008) [2023-10-10 17:46:48,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 63111168. Throughput: 0: 1815.3, 1: 1792.3. Samples: 15793174. Policy #0 lag: (min: 31.0, avg: 32.0, max: 53.0) [2023-10-10 17:46:48,789][122664] Avg episode reward: [(0, '43.670'), (1, '39.970')] [2023-10-10 17:46:48,959][123614] Updated weights for policy 1, policy_version 30790 (0.0008) [2023-10-10 17:46:49,323][123614] Updated weights for policy 1, policy_version 30800 (0.0008) [2023-10-10 17:46:49,696][123614] Updated weights for policy 1, policy_version 30810 (0.0007) [2023-10-10 17:46:52,382][123582] Updated weights for policy 0, policy_version 30883 (0.0008) [2023-10-10 17:46:52,764][123582] Updated weights for policy 0, policy_version 30893 (0.0011) [2023-10-10 17:46:53,126][123582] Updated weights for policy 0, policy_version 30903 (0.0009) [2023-10-10 17:46:53,621][123614] Updated weights for policy 1, policy_version 30820 (0.0009) [2023-10-10 17:46:53,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 63209472. Throughput: 0: 1815.3, 1: 1810.8. Samples: 15813962. Policy #0 lag: (min: 29.0, avg: 30.9, max: 59.0) [2023-10-10 17:46:53,789][122664] Avg episode reward: [(0, '45.190'), (1, '38.880')] [2023-10-10 17:46:54,016][123614] Updated weights for policy 1, policy_version 30830 (0.0011) [2023-10-10 17:46:54,377][123614] Updated weights for policy 1, policy_version 30840 (0.0009) [2023-10-10 17:46:56,789][123582] Updated weights for policy 0, policy_version 30913 (0.0010) [2023-10-10 17:46:57,161][123582] Updated weights for policy 0, policy_version 30923 (0.0008) [2023-10-10 17:46:57,533][123582] Updated weights for policy 0, policy_version 30933 (0.0007) [2023-10-10 17:46:57,884][123614] Updated weights for policy 1, policy_version 30850 (0.0010) [2023-10-10 17:46:57,911][123582] Updated weights for policy 0, policy_version 30943 (0.0009) [2023-10-10 17:46:58,254][123614] Updated weights for policy 1, policy_version 30860 (0.0007) [2023-10-10 17:46:58,618][123614] Updated weights for policy 1, policy_version 30870 (0.0007) [2023-10-10 17:46:58,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 63275008. Throughput: 0: 1815.5, 1: 1796.0. Samples: 15825500. Policy #0 lag: (min: 29.0, avg: 30.9, max: 59.0) [2023-10-10 17:46:58,789][122664] Avg episode reward: [(0, '45.980'), (1, '40.100')] [2023-10-10 17:46:58,985][123614] Updated weights for policy 1, policy_version 30880 (0.0008) [2023-10-10 17:47:01,648][123582] Updated weights for policy 0, policy_version 30953 (0.0008) [2023-10-10 17:47:02,027][123582] Updated weights for policy 0, policy_version 30963 (0.0008) [2023-10-10 17:47:02,393][123582] Updated weights for policy 0, policy_version 30973 (0.0008) [2023-10-10 17:47:02,848][123614] Updated weights for policy 1, policy_version 30890 (0.0008) [2023-10-10 17:47:03,219][123614] Updated weights for policy 1, policy_version 30900 (0.0008) [2023-10-10 17:47:03,594][123614] Updated weights for policy 1, policy_version 30910 (0.0007) [2023-10-10 17:47:03,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 63373312. Throughput: 0: 1820.1, 1: 1817.2. Samples: 15846604. Policy #0 lag: (min: 29.0, avg: 30.9, max: 59.0) [2023-10-10 17:47:03,789][122664] Avg episode reward: [(0, '46.070'), (1, '40.810')] [2023-10-10 17:47:06,277][123582] Updated weights for policy 0, policy_version 30983 (0.0010) [2023-10-10 17:47:06,642][123582] Updated weights for policy 0, policy_version 30993 (0.0008) [2023-10-10 17:47:07,017][123582] Updated weights for policy 0, policy_version 31003 (0.0007) [2023-10-10 17:47:07,212][123614] Updated weights for policy 1, policy_version 30920 (0.0008) [2023-10-10 17:47:07,587][123614] Updated weights for policy 1, policy_version 30930 (0.0008) [2023-10-10 17:47:07,947][123614] Updated weights for policy 1, policy_version 30940 (0.0007) [2023-10-10 17:47:08,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 63438848. Throughput: 0: 1806.8, 1: 1806.7. Samples: 15867926. Policy #0 lag: (min: 29.0, avg: 30.9, max: 59.0) [2023-10-10 17:47:08,788][122664] Avg episode reward: [(0, '44.700'), (1, '39.240')] [2023-10-10 17:47:10,764][123582] Updated weights for policy 0, policy_version 31013 (0.0008) [2023-10-10 17:47:11,132][123582] Updated weights for policy 0, policy_version 31023 (0.0009) [2023-10-10 17:47:11,506][123582] Updated weights for policy 0, policy_version 31033 (0.0008) [2023-10-10 17:47:11,645][123614] Updated weights for policy 1, policy_version 30950 (0.0009) [2023-10-10 17:47:12,020][123614] Updated weights for policy 1, policy_version 30960 (0.0009) [2023-10-10 17:47:12,394][123614] Updated weights for policy 1, policy_version 30970 (0.0009) [2023-10-10 17:47:13,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 63504384. Throughput: 0: 1818.5, 1: 1817.1. Samples: 15879242. Policy #0 lag: (min: 30.0, avg: 33.0, max: 62.0) [2023-10-10 17:47:13,788][122664] Avg episode reward: [(0, '46.620'), (1, '39.610')] [2023-10-10 17:47:15,324][123582] Updated weights for policy 0, policy_version 31043 (0.0008) [2023-10-10 17:47:15,694][123582] Updated weights for policy 0, policy_version 31053 (0.0010) [2023-10-10 17:47:16,034][123614] Updated weights for policy 1, policy_version 30980 (0.0008) [2023-10-10 17:47:16,071][123582] Updated weights for policy 0, policy_version 31063 (0.0009) [2023-10-10 17:47:16,401][123614] Updated weights for policy 1, policy_version 30990 (0.0007) [2023-10-10 17:47:16,778][123614] Updated weights for policy 1, policy_version 31000 (0.0007) [2023-10-10 17:47:18,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 63569920. Throughput: 0: 1805.1, 1: 1810.8. Samples: 15900326. Policy #0 lag: (min: 30.0, avg: 33.0, max: 62.0) [2023-10-10 17:47:18,788][122664] Avg episode reward: [(0, '48.070'), (1, '43.850')] [2023-10-10 17:47:18,789][123247] Saving new best policy, reward=48.070! [2023-10-10 17:47:19,703][123582] Updated weights for policy 0, policy_version 31073 (0.0008) [2023-10-10 17:47:20,070][123582] Updated weights for policy 0, policy_version 31083 (0.0009) [2023-10-10 17:47:20,445][123582] Updated weights for policy 0, policy_version 31093 (0.0010) [2023-10-10 17:47:20,570][123614] Updated weights for policy 1, policy_version 31010 (0.0008) [2023-10-10 17:47:20,816][123582] Updated weights for policy 0, policy_version 31103 (0.0009) [2023-10-10 17:47:20,943][123614] Updated weights for policy 1, policy_version 31020 (0.0007) [2023-10-10 17:47:21,310][123614] Updated weights for policy 1, policy_version 31030 (0.0008) [2023-10-10 17:47:21,679][123614] Updated weights for policy 1, policy_version 31040 (0.0008) [2023-10-10 17:47:23,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 63635456. Throughput: 0: 1807.8, 1: 1811.2. Samples: 15923226. Policy #0 lag: (min: 30.0, avg: 33.0, max: 62.0) [2023-10-10 17:47:23,789][122664] Avg episode reward: [(0, '47.290'), (1, '43.190')] [2023-10-10 17:47:24,270][123582] Updated weights for policy 0, policy_version 31113 (0.0011) [2023-10-10 17:47:24,645][123582] Updated weights for policy 0, policy_version 31123 (0.0008) [2023-10-10 17:47:25,016][123582] Updated weights for policy 0, policy_version 31133 (0.0009) [2023-10-10 17:47:25,291][123614] Updated weights for policy 1, policy_version 31050 (0.0007) [2023-10-10 17:47:25,659][123614] Updated weights for policy 1, policy_version 31060 (0.0008) [2023-10-10 17:47:26,022][123614] Updated weights for policy 1, policy_version 31070 (0.0009) [2023-10-10 17:47:28,658][123582] Updated weights for policy 0, policy_version 31143 (0.0008) [2023-10-10 17:47:28,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 63700992. Throughput: 0: 1808.3, 1: 1812.8. Samples: 15933168. Policy #0 lag: (min: 30.0, avg: 33.0, max: 62.0) [2023-10-10 17:47:28,788][122664] Avg episode reward: [(0, '44.390'), (1, '41.870')] [2023-10-10 17:47:29,032][123582] Updated weights for policy 0, policy_version 31153 (0.0007) [2023-10-10 17:47:29,401][123582] Updated weights for policy 0, policy_version 31163 (0.0011) [2023-10-10 17:47:29,894][123614] Updated weights for policy 1, policy_version 31080 (0.0011) [2023-10-10 17:47:30,259][123614] Updated weights for policy 1, policy_version 31090 (0.0010) [2023-10-10 17:47:30,621][123614] Updated weights for policy 1, policy_version 31100 (0.0011) [2023-10-10 17:47:33,145][123582] Updated weights for policy 0, policy_version 31173 (0.0008) [2023-10-10 17:47:33,511][123582] Updated weights for policy 0, policy_version 31183 (0.0008) [2023-10-10 17:47:33,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 63766528. Throughput: 0: 1802.7, 1: 1803.8. Samples: 15955464. Policy #0 lag: (min: 30.0, avg: 33.0, max: 62.0) [2023-10-10 17:47:33,789][122664] Avg episode reward: [(0, '41.980'), (1, '43.410')] [2023-10-10 17:47:33,879][123582] Updated weights for policy 0, policy_version 31193 (0.0007) [2023-10-10 17:47:34,465][123614] Updated weights for policy 1, policy_version 31110 (0.0008) [2023-10-10 17:47:34,835][123614] Updated weights for policy 1, policy_version 31120 (0.0008) [2023-10-10 17:47:35,206][123614] Updated weights for policy 1, policy_version 31130 (0.0007) [2023-10-10 17:47:37,641][123582] Updated weights for policy 0, policy_version 31203 (0.0010) [2023-10-10 17:47:38,030][123582] Updated weights for policy 0, policy_version 31213 (0.0008) [2023-10-10 17:47:38,399][123582] Updated weights for policy 0, policy_version 31223 (0.0009) [2023-10-10 17:47:38,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 63864832. Throughput: 0: 1808.1, 1: 1814.1. Samples: 15976956. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:47:38,788][122664] Avg episode reward: [(0, '43.980'), (1, '41.820')] [2023-10-10 17:47:38,839][123614] Updated weights for policy 1, policy_version 31140 (0.0007) [2023-10-10 17:47:39,217][123614] Updated weights for policy 1, policy_version 31150 (0.0009) [2023-10-10 17:47:39,584][123614] Updated weights for policy 1, policy_version 31160 (0.0010) [2023-10-10 17:47:42,178][123582] Updated weights for policy 0, policy_version 31233 (0.0009) [2023-10-10 17:47:42,549][123582] Updated weights for policy 0, policy_version 31243 (0.0008) [2023-10-10 17:47:42,920][123582] Updated weights for policy 0, policy_version 31253 (0.0009) [2023-10-10 17:47:43,294][123582] Updated weights for policy 0, policy_version 31263 (0.0010) [2023-10-10 17:47:43,401][123614] Updated weights for policy 1, policy_version 31170 (0.0010) [2023-10-10 17:47:43,767][123614] Updated weights for policy 1, policy_version 31180 (0.0007) [2023-10-10 17:47:43,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 63930368. Throughput: 0: 1803.7, 1: 1802.0. Samples: 15987756. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:47:43,789][122664] Avg episode reward: [(0, '41.910'), (1, '42.770')] [2023-10-10 17:47:44,140][123614] Updated weights for policy 1, policy_version 31190 (0.0009) [2023-10-10 17:47:44,507][123614] Updated weights for policy 1, policy_version 31200 (0.0010) [2023-10-10 17:47:46,880][123582] Updated weights for policy 0, policy_version 31273 (0.0011) [2023-10-10 17:47:47,256][123582] Updated weights for policy 0, policy_version 31283 (0.0008) [2023-10-10 17:47:47,625][123582] Updated weights for policy 0, policy_version 31293 (0.0007) [2023-10-10 17:47:48,234][123614] Updated weights for policy 1, policy_version 31210 (0.0011) [2023-10-10 17:47:48,602][123614] Updated weights for policy 1, policy_version 31220 (0.0011) [2023-10-10 17:47:48,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 63995904. Throughput: 0: 1814.3, 1: 1806.8. Samples: 16009554. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:47:48,789][122664] Avg episode reward: [(0, '41.770'), (1, '39.780')] [2023-10-10 17:47:48,974][123614] Updated weights for policy 1, policy_version 31230 (0.0007) [2023-10-10 17:47:51,341][123582] Updated weights for policy 0, policy_version 31303 (0.0007) [2023-10-10 17:47:51,713][123582] Updated weights for policy 0, policy_version 31313 (0.0007) [2023-10-10 17:47:52,103][123582] Updated weights for policy 0, policy_version 31323 (0.0010) [2023-10-10 17:47:52,599][123614] Updated weights for policy 1, policy_version 31240 (0.0009) [2023-10-10 17:47:52,975][123614] Updated weights for policy 1, policy_version 31250 (0.0008) [2023-10-10 17:47:53,343][123614] Updated weights for policy 1, policy_version 31260 (0.0007) [2023-10-10 17:47:53,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 64094208. Throughput: 0: 1811.1, 1: 1798.8. Samples: 16030372. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:47:53,789][122664] Avg episode reward: [(0, '43.240'), (1, '40.550')] [2023-10-10 17:47:53,799][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000031328_32079872.pth... [2023-10-10 17:47:53,799][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000031264_32014336.pth... [2023-10-10 17:47:53,835][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000029568_30277632.pth [2023-10-10 17:47:53,839][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000029632_30343168.pth [2023-10-10 17:47:55,907][123582] Updated weights for policy 0, policy_version 31333 (0.0008) [2023-10-10 17:47:56,277][123582] Updated weights for policy 0, policy_version 31343 (0.0008) [2023-10-10 17:47:56,650][123582] Updated weights for policy 0, policy_version 31353 (0.0007) [2023-10-10 17:47:57,068][123614] Updated weights for policy 1, policy_version 31270 (0.0008) [2023-10-10 17:47:57,431][123614] Updated weights for policy 1, policy_version 31280 (0.0007) [2023-10-10 17:47:57,802][123614] Updated weights for policy 1, policy_version 31290 (0.0007) [2023-10-10 17:47:58,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 64159744. Throughput: 0: 1816.7, 1: 1809.1. Samples: 16042404. Policy #0 lag: (min: 5.0, avg: 12.1, max: 37.0) [2023-10-10 17:47:58,789][122664] Avg episode reward: [(0, '42.680'), (1, '36.110')] [2023-10-10 17:48:00,380][123582] Updated weights for policy 0, policy_version 31363 (0.0007) [2023-10-10 17:48:00,756][123582] Updated weights for policy 0, policy_version 31373 (0.0008) [2023-10-10 17:48:01,128][123582] Updated weights for policy 0, policy_version 31383 (0.0009) [2023-10-10 17:48:01,547][123614] Updated weights for policy 1, policy_version 31300 (0.0010) [2023-10-10 17:48:01,925][123614] Updated weights for policy 1, policy_version 31310 (0.0008) [2023-10-10 17:48:02,303][123614] Updated weights for policy 1, policy_version 31320 (0.0008) [2023-10-10 17:48:03,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 64225280. Throughput: 0: 1811.8, 1: 1804.5. Samples: 16063062. Policy #0 lag: (min: 5.0, avg: 12.1, max: 37.0) [2023-10-10 17:48:03,789][122664] Avg episode reward: [(0, '42.270'), (1, '33.000')] [2023-10-10 17:48:04,783][123582] Updated weights for policy 0, policy_version 31393 (0.0008) [2023-10-10 17:48:05,147][123582] Updated weights for policy 0, policy_version 31403 (0.0009) [2023-10-10 17:48:05,522][123582] Updated weights for policy 0, policy_version 31413 (0.0009) [2023-10-10 17:48:05,894][123582] Updated weights for policy 0, policy_version 31423 (0.0007) [2023-10-10 17:48:06,033][123614] Updated weights for policy 1, policy_version 31330 (0.0007) [2023-10-10 17:48:06,401][123614] Updated weights for policy 1, policy_version 31340 (0.0008) [2023-10-10 17:48:06,766][123614] Updated weights for policy 1, policy_version 31350 (0.0007) [2023-10-10 17:48:07,138][123614] Updated weights for policy 1, policy_version 31360 (0.0007) [2023-10-10 17:48:08,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 64290816. Throughput: 0: 1810.9, 1: 1803.7. Samples: 16085880. Policy #0 lag: (min: 5.0, avg: 12.1, max: 37.0) [2023-10-10 17:48:08,788][122664] Avg episode reward: [(0, '41.220'), (1, '30.170')] [2023-10-10 17:48:09,697][123582] Updated weights for policy 0, policy_version 31433 (0.0008) [2023-10-10 17:48:10,063][123582] Updated weights for policy 0, policy_version 31443 (0.0008) [2023-10-10 17:48:10,434][123582] Updated weights for policy 0, policy_version 31453 (0.0008) [2023-10-10 17:48:10,876][123614] Updated weights for policy 1, policy_version 31370 (0.0008) [2023-10-10 17:48:11,231][123614] Updated weights for policy 1, policy_version 31380 (0.0007) [2023-10-10 17:48:11,610][123614] Updated weights for policy 1, policy_version 31390 (0.0008) [2023-10-10 17:48:13,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 64356352. Throughput: 0: 1809.9, 1: 1808.2. Samples: 16095986. Policy #0 lag: (min: 5.0, avg: 12.1, max: 37.0) [2023-10-10 17:48:13,789][122664] Avg episode reward: [(0, '40.900'), (1, '30.580')] [2023-10-10 17:48:14,076][123582] Updated weights for policy 0, policy_version 31463 (0.0008) [2023-10-10 17:48:14,448][123582] Updated weights for policy 0, policy_version 31473 (0.0007) [2023-10-10 17:48:14,821][123582] Updated weights for policy 0, policy_version 31483 (0.0008) [2023-10-10 17:48:15,294][123614] Updated weights for policy 1, policy_version 31400 (0.0009) [2023-10-10 17:48:15,663][123614] Updated weights for policy 1, policy_version 31410 (0.0008) [2023-10-10 17:48:16,036][123614] Updated weights for policy 1, policy_version 31420 (0.0007) [2023-10-10 17:48:18,340][123582] Updated weights for policy 0, policy_version 31493 (0.0008) [2023-10-10 17:48:18,712][123582] Updated weights for policy 0, policy_version 31503 (0.0007) [2023-10-10 17:48:18,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 64421888. Throughput: 0: 1810.9, 1: 1815.6. Samples: 16118660. Policy #0 lag: (min: 5.0, avg: 12.1, max: 37.0) [2023-10-10 17:48:18,789][122664] Avg episode reward: [(0, '39.620'), (1, '31.970')] [2023-10-10 17:48:19,080][123582] Updated weights for policy 0, policy_version 31513 (0.0009) [2023-10-10 17:48:19,752][123614] Updated weights for policy 1, policy_version 31430 (0.0008) [2023-10-10 17:48:20,114][123614] Updated weights for policy 1, policy_version 31440 (0.0008) [2023-10-10 17:48:20,473][123614] Updated weights for policy 1, policy_version 31450 (0.0007) [2023-10-10 17:48:22,877][123582] Updated weights for policy 0, policy_version 31523 (0.0009) [2023-10-10 17:48:23,257][123582] Updated weights for policy 0, policy_version 31533 (0.0010) [2023-10-10 17:48:23,638][123582] Updated weights for policy 0, policy_version 31543 (0.0009) [2023-10-10 17:48:23,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 64487424. Throughput: 0: 1820.1, 1: 1818.3. Samples: 16140686. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-10 17:48:23,788][122664] Avg episode reward: [(0, '37.110'), (1, '32.090')] [2023-10-10 17:48:24,154][123614] Updated weights for policy 1, policy_version 31460 (0.0010) [2023-10-10 17:48:24,542][123614] Updated weights for policy 1, policy_version 31470 (0.0008) [2023-10-10 17:48:24,912][123614] Updated weights for policy 1, policy_version 31480 (0.0009) [2023-10-10 17:48:27,298][123582] Updated weights for policy 0, policy_version 31553 (0.0009) [2023-10-10 17:48:27,670][123582] Updated weights for policy 0, policy_version 31563 (0.0008) [2023-10-10 17:48:28,044][123582] Updated weights for policy 0, policy_version 31573 (0.0008) [2023-10-10 17:48:28,424][123582] Updated weights for policy 0, policy_version 31583 (0.0009) [2023-10-10 17:48:28,759][123614] Updated weights for policy 1, policy_version 31490 (0.0008) [2023-10-10 17:48:28,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 64585728. Throughput: 0: 1814.2, 1: 1816.4. Samples: 16151136. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-10 17:48:28,789][122664] Avg episode reward: [(0, '36.030'), (1, '33.940')] [2023-10-10 17:48:29,123][123614] Updated weights for policy 1, policy_version 31500 (0.0008) [2023-10-10 17:48:29,495][123614] Updated weights for policy 1, policy_version 31510 (0.0009) [2023-10-10 17:48:29,865][123614] Updated weights for policy 1, policy_version 31520 (0.0009) [2023-10-10 17:48:32,081][123582] Updated weights for policy 0, policy_version 31593 (0.0009) [2023-10-10 17:48:32,453][123582] Updated weights for policy 0, policy_version 31603 (0.0011) [2023-10-10 17:48:32,816][123582] Updated weights for policy 0, policy_version 31613 (0.0010) [2023-10-10 17:48:33,395][123614] Updated weights for policy 1, policy_version 31530 (0.0010) [2023-10-10 17:48:33,759][123614] Updated weights for policy 1, policy_version 31540 (0.0007) [2023-10-10 17:48:33,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 64651264. Throughput: 0: 1810.3, 1: 1817.1. Samples: 16172786. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-10 17:48:33,789][122664] Avg episode reward: [(0, '35.200'), (1, '35.020')] [2023-10-10 17:48:34,123][123614] Updated weights for policy 1, policy_version 31550 (0.0008) [2023-10-10 17:48:36,657][123582] Updated weights for policy 0, policy_version 31623 (0.0010) [2023-10-10 17:48:37,041][123582] Updated weights for policy 0, policy_version 31633 (0.0008) [2023-10-10 17:48:37,404][123582] Updated weights for policy 0, policy_version 31643 (0.0008) [2023-10-10 17:48:37,814][123614] Updated weights for policy 1, policy_version 31560 (0.0010) [2023-10-10 17:48:38,188][123614] Updated weights for policy 1, policy_version 31570 (0.0009) [2023-10-10 17:48:38,562][123614] Updated weights for policy 1, policy_version 31580 (0.0008) [2023-10-10 17:48:38,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 64749568. Throughput: 0: 1803.4, 1: 1816.3. Samples: 16193258. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-10 17:48:38,789][122664] Avg episode reward: [(0, '36.320'), (1, '33.650')] [2023-10-10 17:48:41,221][123582] Updated weights for policy 0, policy_version 31653 (0.0007) [2023-10-10 17:48:41,589][123582] Updated weights for policy 0, policy_version 31663 (0.0007) [2023-10-10 17:48:41,972][123582] Updated weights for policy 0, policy_version 31673 (0.0009) [2023-10-10 17:48:42,188][123614] Updated weights for policy 1, policy_version 31590 (0.0009) [2023-10-10 17:48:42,558][123614] Updated weights for policy 1, policy_version 31600 (0.0008) [2023-10-10 17:48:42,924][123614] Updated weights for policy 1, policy_version 31610 (0.0010) [2023-10-10 17:48:43,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 64815104. Throughput: 0: 1810.2, 1: 1813.9. Samples: 16205488. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-10-10 17:48:43,789][122664] Avg episode reward: [(0, '36.490'), (1, '35.360')] [2023-10-10 17:48:45,772][123582] Updated weights for policy 0, policy_version 31683 (0.0009) [2023-10-10 17:48:46,142][123582] Updated weights for policy 0, policy_version 31693 (0.0009) [2023-10-10 17:48:46,512][123582] Updated weights for policy 0, policy_version 31703 (0.0008) [2023-10-10 17:48:46,580][123614] Updated weights for policy 1, policy_version 31620 (0.0009) [2023-10-10 17:48:46,942][123614] Updated weights for policy 1, policy_version 31630 (0.0007) [2023-10-10 17:48:47,308][123614] Updated weights for policy 1, policy_version 31640 (0.0009) [2023-10-10 17:48:48,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 64880640. Throughput: 0: 1801.7, 1: 1814.4. Samples: 16225790. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-10-10 17:48:48,789][122664] Avg episode reward: [(0, '39.560'), (1, '36.140')] [2023-10-10 17:48:50,256][123582] Updated weights for policy 0, policy_version 31713 (0.0008) [2023-10-10 17:48:50,638][123582] Updated weights for policy 0, policy_version 31723 (0.0011) [2023-10-10 17:48:51,003][123582] Updated weights for policy 0, policy_version 31733 (0.0008) [2023-10-10 17:48:51,099][123614] Updated weights for policy 1, policy_version 31650 (0.0009) [2023-10-10 17:48:51,375][123582] Updated weights for policy 0, policy_version 31743 (0.0007) [2023-10-10 17:48:51,465][123614] Updated weights for policy 1, policy_version 31660 (0.0007) [2023-10-10 17:48:51,832][123614] Updated weights for policy 1, policy_version 31670 (0.0009) [2023-10-10 17:48:52,203][123614] Updated weights for policy 1, policy_version 31680 (0.0007) [2023-10-10 17:48:53,788][122664] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 64946176. Throughput: 0: 1797.8, 1: 1811.4. Samples: 16248294. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-10-10 17:48:53,789][122664] Avg episode reward: [(0, '40.370'), (1, '36.980')] [2023-10-10 17:48:55,147][123582] Updated weights for policy 0, policy_version 31753 (0.0007) [2023-10-10 17:48:55,519][123582] Updated weights for policy 0, policy_version 31763 (0.0008) [2023-10-10 17:48:55,884][123582] Updated weights for policy 0, policy_version 31773 (0.0007) [2023-10-10 17:48:55,935][123614] Updated weights for policy 1, policy_version 31690 (0.0008) [2023-10-10 17:48:56,307][123614] Updated weights for policy 1, policy_version 31700 (0.0009) [2023-10-10 17:48:56,679][123614] Updated weights for policy 1, policy_version 31710 (0.0007) [2023-10-10 17:48:58,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 65011712. Throughput: 0: 1796.2, 1: 1808.0. Samples: 16258176. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-10-10 17:48:58,788][122664] Avg episode reward: [(0, '39.620'), (1, '39.380')] [2023-10-10 17:48:59,519][123582] Updated weights for policy 0, policy_version 31783 (0.0008) [2023-10-10 17:48:59,897][123582] Updated weights for policy 0, policy_version 31793 (0.0008) [2023-10-10 17:49:00,261][123582] Updated weights for policy 0, policy_version 31803 (0.0007) [2023-10-10 17:49:00,375][123614] Updated weights for policy 1, policy_version 31720 (0.0008) [2023-10-10 17:49:00,747][123614] Updated weights for policy 1, policy_version 31730 (0.0010) [2023-10-10 17:49:01,114][123614] Updated weights for policy 1, policy_version 31740 (0.0008) [2023-10-10 17:49:03,744][123582] Updated weights for policy 0, policy_version 31813 (0.0008) [2023-10-10 17:49:03,788][122664] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 65077248. Throughput: 0: 1802.8, 1: 1806.9. Samples: 16281098. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-10-10 17:49:03,788][122664] Avg episode reward: [(0, '44.950'), (1, '37.750')] [2023-10-10 17:49:04,117][123582] Updated weights for policy 0, policy_version 31823 (0.0009) [2023-10-10 17:49:04,488][123582] Updated weights for policy 0, policy_version 31833 (0.0008) [2023-10-10 17:49:04,824][123614] Updated weights for policy 1, policy_version 31750 (0.0008) [2023-10-10 17:49:05,198][123614] Updated weights for policy 1, policy_version 31760 (0.0010) [2023-10-10 17:49:05,561][123614] Updated weights for policy 1, policy_version 31770 (0.0010) [2023-10-10 17:49:08,311][123582] Updated weights for policy 0, policy_version 31843 (0.0007) [2023-10-10 17:49:08,703][123582] Updated weights for policy 0, policy_version 31853 (0.0007) [2023-10-10 17:49:08,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 65142784. Throughput: 0: 1816.2, 1: 1801.1. Samples: 16303466. Policy #0 lag: (min: 24.0, avg: 52.0, max: 56.0) [2023-10-10 17:49:08,789][122664] Avg episode reward: [(0, '45.230'), (1, '36.190')] [2023-10-10 17:49:09,076][123582] Updated weights for policy 0, policy_version 31863 (0.0008) [2023-10-10 17:49:09,382][123614] Updated weights for policy 1, policy_version 31780 (0.0008) [2023-10-10 17:49:09,776][123614] Updated weights for policy 1, policy_version 31790 (0.0008) [2023-10-10 17:49:10,146][123614] Updated weights for policy 1, policy_version 31800 (0.0008) [2023-10-10 17:49:12,621][123582] Updated weights for policy 0, policy_version 31873 (0.0007) [2023-10-10 17:49:12,987][123582] Updated weights for policy 0, policy_version 31883 (0.0009) [2023-10-10 17:49:13,367][123582] Updated weights for policy 0, policy_version 31893 (0.0009) [2023-10-10 17:49:13,734][123614] Updated weights for policy 1, policy_version 31810 (0.0007) [2023-10-10 17:49:13,736][123582] Updated weights for policy 0, policy_version 31903 (0.0008) [2023-10-10 17:49:13,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 65241088. Throughput: 0: 1805.0, 1: 1806.7. Samples: 16313662. Policy #0 lag: (min: 24.0, avg: 52.0, max: 56.0) [2023-10-10 17:49:13,789][122664] Avg episode reward: [(0, '46.090'), (1, '38.460')] [2023-10-10 17:49:14,107][123614] Updated weights for policy 1, policy_version 31820 (0.0007) [2023-10-10 17:49:14,468][123614] Updated weights for policy 1, policy_version 31830 (0.0008) [2023-10-10 17:49:14,842][123614] Updated weights for policy 1, policy_version 31840 (0.0008) [2023-10-10 17:49:17,528][123582] Updated weights for policy 0, policy_version 31913 (0.0007) [2023-10-10 17:49:17,893][123582] Updated weights for policy 0, policy_version 31923 (0.0009) [2023-10-10 17:49:18,266][123582] Updated weights for policy 0, policy_version 31933 (0.0008) [2023-10-10 17:49:18,561][123614] Updated weights for policy 1, policy_version 31850 (0.0010) [2023-10-10 17:49:18,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 65306624. Throughput: 0: 1819.3, 1: 1804.5. Samples: 16335858. Policy #0 lag: (min: 24.0, avg: 52.0, max: 56.0) [2023-10-10 17:49:18,788][122664] Avg episode reward: [(0, '48.820'), (1, '39.370')] [2023-10-10 17:49:18,789][123247] Saving new best policy, reward=48.820! [2023-10-10 17:49:18,923][123614] Updated weights for policy 1, policy_version 31860 (0.0008) [2023-10-10 17:49:19,290][123614] Updated weights for policy 1, policy_version 31870 (0.0009) [2023-10-10 17:49:21,899][123582] Updated weights for policy 0, policy_version 31943 (0.0007) [2023-10-10 17:49:22,284][123582] Updated weights for policy 0, policy_version 31953 (0.0007) [2023-10-10 17:49:22,660][123582] Updated weights for policy 0, policy_version 31963 (0.0008) [2023-10-10 17:49:22,958][123614] Updated weights for policy 1, policy_version 31880 (0.0008) [2023-10-10 17:49:23,316][123614] Updated weights for policy 1, policy_version 31890 (0.0008) [2023-10-10 17:49:23,691][123614] Updated weights for policy 1, policy_version 31900 (0.0007) [2023-10-10 17:49:23,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 65372160. Throughput: 0: 1810.5, 1: 1811.9. Samples: 16356266. Policy #0 lag: (min: 24.0, avg: 52.0, max: 56.0) [2023-10-10 17:49:23,789][122664] Avg episode reward: [(0, '53.180'), (1, '39.450')] [2023-10-10 17:49:23,799][123247] Saving new best policy, reward=53.180! [2023-10-10 17:49:26,357][123582] Updated weights for policy 0, policy_version 31973 (0.0009) [2023-10-10 17:49:26,725][123582] Updated weights for policy 0, policy_version 31983 (0.0009) [2023-10-10 17:49:27,097][123582] Updated weights for policy 0, policy_version 31993 (0.0009) [2023-10-10 17:49:27,396][123614] Updated weights for policy 1, policy_version 31910 (0.0008) [2023-10-10 17:49:27,766][123614] Updated weights for policy 1, policy_version 31920 (0.0010) [2023-10-10 17:49:28,139][123614] Updated weights for policy 1, policy_version 31930 (0.0008) [2023-10-10 17:49:28,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 65470464. Throughput: 0: 1816.3, 1: 1807.9. Samples: 16368578. Policy #0 lag: (min: 19.0, avg: 44.4, max: 48.0) [2023-10-10 17:49:28,789][122664] Avg episode reward: [(0, '53.490'), (1, '42.370')] [2023-10-10 17:49:28,789][123247] Saving new best policy, reward=53.490! [2023-10-10 17:49:30,733][123582] Updated weights for policy 0, policy_version 32003 (0.0010) [2023-10-10 17:49:31,115][123582] Updated weights for policy 0, policy_version 32013 (0.0007) [2023-10-10 17:49:31,486][123582] Updated weights for policy 0, policy_version 32023 (0.0009) [2023-10-10 17:49:31,935][123614] Updated weights for policy 1, policy_version 31940 (0.0008) [2023-10-10 17:49:32,302][123614] Updated weights for policy 1, policy_version 31950 (0.0009) [2023-10-10 17:49:32,683][123614] Updated weights for policy 1, policy_version 31960 (0.0015) [2023-10-10 17:49:33,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 65536000. Throughput: 0: 1815.1, 1: 1814.3. Samples: 16389112. Policy #0 lag: (min: 19.0, avg: 44.4, max: 48.0) [2023-10-10 17:49:33,788][122664] Avg episode reward: [(0, '54.840'), (1, '41.650')] [2023-10-10 17:49:33,789][123247] Saving new best policy, reward=54.840! [2023-10-10 17:49:35,284][123582] Updated weights for policy 0, policy_version 32033 (0.0010) [2023-10-10 17:49:35,659][123582] Updated weights for policy 0, policy_version 32043 (0.0008) [2023-10-10 17:49:36,028][123582] Updated weights for policy 0, policy_version 32053 (0.0008) [2023-10-10 17:49:36,399][123582] Updated weights for policy 0, policy_version 32063 (0.0007) [2023-10-10 17:49:36,455][123614] Updated weights for policy 1, policy_version 31970 (0.0010) [2023-10-10 17:49:36,820][123614] Updated weights for policy 1, policy_version 31980 (0.0008) [2023-10-10 17:49:37,202][123614] Updated weights for policy 1, policy_version 31990 (0.0011) [2023-10-10 17:49:37,575][123614] Updated weights for policy 1, policy_version 32000 (0.0010) [2023-10-10 17:49:38,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 65601536. Throughput: 0: 1815.2, 1: 1804.0. Samples: 16411156. Policy #0 lag: (min: 19.0, avg: 44.4, max: 48.0) [2023-10-10 17:49:38,789][122664] Avg episode reward: [(0, '52.910'), (1, '39.860')] [2023-10-10 17:49:40,124][123582] Updated weights for policy 0, policy_version 32073 (0.0008) [2023-10-10 17:49:40,495][123582] Updated weights for policy 0, policy_version 32083 (0.0008) [2023-10-10 17:49:40,874][123582] Updated weights for policy 0, policy_version 32093 (0.0007) [2023-10-10 17:49:41,195][123614] Updated weights for policy 1, policy_version 32010 (0.0008) [2023-10-10 17:49:41,562][123614] Updated weights for policy 1, policy_version 32020 (0.0007) [2023-10-10 17:49:41,936][123614] Updated weights for policy 1, policy_version 32030 (0.0008) [2023-10-10 17:49:43,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 65667072. Throughput: 0: 1814.7, 1: 1816.4. Samples: 16421576. Policy #0 lag: (min: 19.0, avg: 44.4, max: 48.0) [2023-10-10 17:49:43,789][122664] Avg episode reward: [(0, '53.390'), (1, '41.060')] [2023-10-10 17:49:44,361][123582] Updated weights for policy 0, policy_version 32103 (0.0009) [2023-10-10 17:49:44,726][123582] Updated weights for policy 0, policy_version 32113 (0.0008) [2023-10-10 17:49:45,102][123582] Updated weights for policy 0, policy_version 32123 (0.0008) [2023-10-10 17:49:45,798][123614] Updated weights for policy 1, policy_version 32040 (0.0010) [2023-10-10 17:49:46,169][123614] Updated weights for policy 1, policy_version 32050 (0.0011) [2023-10-10 17:49:46,541][123614] Updated weights for policy 1, policy_version 32060 (0.0010) [2023-10-10 17:49:48,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 65732608. Throughput: 0: 1812.3, 1: 1805.7. Samples: 16443906. Policy #0 lag: (min: 19.0, avg: 44.4, max: 48.0) [2023-10-10 17:49:48,789][122664] Avg episode reward: [(0, '51.650'), (1, '42.990')] [2023-10-10 17:49:48,912][123582] Updated weights for policy 0, policy_version 32133 (0.0007) [2023-10-10 17:49:49,290][123582] Updated weights for policy 0, policy_version 32143 (0.0008) [2023-10-10 17:49:49,660][123582] Updated weights for policy 0, policy_version 32153 (0.0007) [2023-10-10 17:49:50,311][123614] Updated weights for policy 1, policy_version 32070 (0.0008) [2023-10-10 17:49:50,683][123614] Updated weights for policy 1, policy_version 32080 (0.0008) [2023-10-10 17:49:51,059][123614] Updated weights for policy 1, policy_version 32090 (0.0007) [2023-10-10 17:49:53,245][123582] Updated weights for policy 0, policy_version 32163 (0.0007) [2023-10-10 17:49:53,648][123582] Updated weights for policy 0, policy_version 32173 (0.0007) [2023-10-10 17:49:53,788][122664] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 65798144. Throughput: 0: 1811.3, 1: 1806.5. Samples: 16466268. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-10 17:49:53,789][122664] Avg episode reward: [(0, '53.350'), (1, '42.380')] [2023-10-10 17:49:53,803][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000032096_32866304.pth... [2023-10-10 17:49:53,834][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000030400_31129600.pth [2023-10-10 17:49:54,020][123582] Updated weights for policy 0, policy_version 32183 (0.0007) [2023-10-10 17:49:54,352][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000032192_32964608.pth... [2023-10-10 17:49:54,381][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000030496_31227904.pth [2023-10-10 17:49:54,732][123614] Updated weights for policy 1, policy_version 32100 (0.0009) [2023-10-10 17:49:55,119][123614] Updated weights for policy 1, policy_version 32110 (0.0008) [2023-10-10 17:49:55,499][123614] Updated weights for policy 1, policy_version 32120 (0.0010) [2023-10-10 17:49:57,869][123582] Updated weights for policy 0, policy_version 32193 (0.0010) [2023-10-10 17:49:58,247][123582] Updated weights for policy 0, policy_version 32203 (0.0011) [2023-10-10 17:49:58,624][123582] Updated weights for policy 0, policy_version 32213 (0.0009) [2023-10-10 17:49:58,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 65863680. Throughput: 0: 1811.7, 1: 1804.8. Samples: 16476404. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-10 17:49:58,789][122664] Avg episode reward: [(0, '54.660'), (1, '43.230')] [2023-10-10 17:49:59,007][123582] Updated weights for policy 0, policy_version 32223 (0.0008) [2023-10-10 17:49:59,329][123614] Updated weights for policy 1, policy_version 32130 (0.0008) [2023-10-10 17:49:59,696][123614] Updated weights for policy 1, policy_version 32140 (0.0007) [2023-10-10 17:50:00,064][123614] Updated weights for policy 1, policy_version 32150 (0.0008) [2023-10-10 17:50:00,434][123614] Updated weights for policy 1, policy_version 32160 (0.0009) [2023-10-10 17:50:02,749][123582] Updated weights for policy 0, policy_version 32233 (0.0008) [2023-10-10 17:50:03,127][123582] Updated weights for policy 0, policy_version 32243 (0.0008) [2023-10-10 17:50:03,491][123582] Updated weights for policy 0, policy_version 32253 (0.0007) [2023-10-10 17:50:03,788][122664] Fps is (10 sec: 16384.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 65961984. Throughput: 0: 1815.3, 1: 1807.7. Samples: 16498892. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-10 17:50:03,788][122664] Avg episode reward: [(0, '53.410'), (1, '41.710')] [2023-10-10 17:50:04,006][123614] Updated weights for policy 1, policy_version 32170 (0.0008) [2023-10-10 17:50:04,380][123614] Updated weights for policy 1, policy_version 32180 (0.0008) [2023-10-10 17:50:04,749][123614] Updated weights for policy 1, policy_version 32190 (0.0009) [2023-10-10 17:50:07,036][123582] Updated weights for policy 0, policy_version 32263 (0.0007) [2023-10-10 17:50:07,405][123582] Updated weights for policy 0, policy_version 32273 (0.0008) [2023-10-10 17:50:07,772][123582] Updated weights for policy 0, policy_version 32283 (0.0008) [2023-10-10 17:50:08,505][123614] Updated weights for policy 1, policy_version 32200 (0.0008) [2023-10-10 17:50:08,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 66027520. Throughput: 0: 1808.9, 1: 1816.7. Samples: 16519420. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-10 17:50:08,789][122664] Avg episode reward: [(0, '49.510'), (1, '40.060')] [2023-10-10 17:50:08,872][123614] Updated weights for policy 1, policy_version 32210 (0.0008) [2023-10-10 17:50:09,252][123614] Updated weights for policy 1, policy_version 32220 (0.0008) [2023-10-10 17:50:11,548][123582] Updated weights for policy 0, policy_version 32293 (0.0010) [2023-10-10 17:50:11,922][123582] Updated weights for policy 0, policy_version 32303 (0.0009) [2023-10-10 17:50:12,294][123582] Updated weights for policy 0, policy_version 32313 (0.0011) [2023-10-10 17:50:12,945][123614] Updated weights for policy 1, policy_version 32230 (0.0008) [2023-10-10 17:50:13,312][123614] Updated weights for policy 1, policy_version 32240 (0.0008) [2023-10-10 17:50:13,687][123614] Updated weights for policy 1, policy_version 32250 (0.0009) [2023-10-10 17:50:13,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 66093056. Throughput: 0: 1812.3, 1: 1801.2. Samples: 16531184. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:50:13,789][122664] Avg episode reward: [(0, '49.970'), (1, '38.810')] [2023-10-10 17:50:16,105][123582] Updated weights for policy 0, policy_version 32323 (0.0009) [2023-10-10 17:50:16,479][123582] Updated weights for policy 0, policy_version 32333 (0.0008) [2023-10-10 17:50:16,846][123582] Updated weights for policy 0, policy_version 32343 (0.0007) [2023-10-10 17:50:17,290][123614] Updated weights for policy 1, policy_version 32260 (0.0010) [2023-10-10 17:50:17,661][123614] Updated weights for policy 1, policy_version 32270 (0.0010) [2023-10-10 17:50:18,022][123614] Updated weights for policy 1, policy_version 32280 (0.0010) [2023-10-10 17:50:18,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 66191360. Throughput: 0: 1804.0, 1: 1812.1. Samples: 16551838. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:50:18,788][122664] Avg episode reward: [(0, '49.050'), (1, '37.450')] [2023-10-10 17:50:20,438][123582] Updated weights for policy 0, policy_version 32353 (0.0007) [2023-10-10 17:50:20,809][123582] Updated weights for policy 0, policy_version 32363 (0.0009) [2023-10-10 17:50:21,183][123582] Updated weights for policy 0, policy_version 32373 (0.0010) [2023-10-10 17:50:21,549][123582] Updated weights for policy 0, policy_version 32383 (0.0010) [2023-10-10 17:50:21,827][123614] Updated weights for policy 1, policy_version 32290 (0.0008) [2023-10-10 17:50:22,195][123614] Updated weights for policy 1, policy_version 32300 (0.0010) [2023-10-10 17:50:22,565][123614] Updated weights for policy 1, policy_version 32310 (0.0011) [2023-10-10 17:50:22,943][123614] Updated weights for policy 1, policy_version 32320 (0.0007) [2023-10-10 17:50:23,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 66256896. Throughput: 0: 1808.7, 1: 1805.3. Samples: 16573782. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:50:23,788][122664] Avg episode reward: [(0, '50.140'), (1, '41.040')] [2023-10-10 17:50:25,299][123582] Updated weights for policy 0, policy_version 32393 (0.0008) [2023-10-10 17:50:25,669][123582] Updated weights for policy 0, policy_version 32403 (0.0007) [2023-10-10 17:50:26,048][123582] Updated weights for policy 0, policy_version 32413 (0.0007) [2023-10-10 17:50:26,638][123614] Updated weights for policy 1, policy_version 32330 (0.0008) [2023-10-10 17:50:27,004][123614] Updated weights for policy 1, policy_version 32340 (0.0009) [2023-10-10 17:50:27,375][123614] Updated weights for policy 1, policy_version 32350 (0.0010) [2023-10-10 17:50:28,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 66322432. Throughput: 0: 1807.2, 1: 1814.7. Samples: 16584562. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:50:28,789][122664] Avg episode reward: [(0, '51.610'), (1, '40.690')] [2023-10-10 17:50:29,775][123582] Updated weights for policy 0, policy_version 32423 (0.0008) [2023-10-10 17:50:30,142][123582] Updated weights for policy 0, policy_version 32433 (0.0010) [2023-10-10 17:50:30,504][123582] Updated weights for policy 0, policy_version 32443 (0.0007) [2023-10-10 17:50:31,179][123614] Updated weights for policy 1, policy_version 32360 (0.0007) [2023-10-10 17:50:31,555][123614] Updated weights for policy 1, policy_version 32370 (0.0007) [2023-10-10 17:50:31,926][123614] Updated weights for policy 1, policy_version 32380 (0.0007) [2023-10-10 17:50:33,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 66387968. Throughput: 0: 1810.2, 1: 1804.7. Samples: 16606578. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:50:33,788][122664] Avg episode reward: [(0, '50.930'), (1, '38.430')] [2023-10-10 17:50:34,125][123582] Updated weights for policy 0, policy_version 32453 (0.0010) [2023-10-10 17:50:34,487][123582] Updated weights for policy 0, policy_version 32463 (0.0010) [2023-10-10 17:50:34,862][123582] Updated weights for policy 0, policy_version 32473 (0.0009) [2023-10-10 17:50:35,487][123614] Updated weights for policy 1, policy_version 32390 (0.0007) [2023-10-10 17:50:35,844][123614] Updated weights for policy 1, policy_version 32400 (0.0010) [2023-10-10 17:50:36,217][123614] Updated weights for policy 1, policy_version 32410 (0.0007) [2023-10-10 17:50:38,764][123582] Updated weights for policy 0, policy_version 32483 (0.0010) [2023-10-10 17:50:38,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 66453504. Throughput: 0: 1813.1, 1: 1810.0. Samples: 16629306. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-10 17:50:38,788][122664] Avg episode reward: [(0, '50.210'), (1, '39.320')] [2023-10-10 17:50:39,162][123582] Updated weights for policy 0, policy_version 32493 (0.0008) [2023-10-10 17:50:39,530][123582] Updated weights for policy 0, policy_version 32503 (0.0008) [2023-10-10 17:50:39,973][123614] Updated weights for policy 1, policy_version 32420 (0.0009) [2023-10-10 17:50:40,358][123614] Updated weights for policy 1, policy_version 32430 (0.0007) [2023-10-10 17:50:40,721][123614] Updated weights for policy 1, policy_version 32440 (0.0007) [2023-10-10 17:50:43,317][123582] Updated weights for policy 0, policy_version 32513 (0.0009) [2023-10-10 17:50:43,687][123582] Updated weights for policy 0, policy_version 32523 (0.0010) [2023-10-10 17:50:43,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 66519040. Throughput: 0: 1799.5, 1: 1814.0. Samples: 16639010. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-10 17:50:43,789][122664] Avg episode reward: [(0, '52.500'), (1, '40.030')] [2023-10-10 17:50:44,054][123582] Updated weights for policy 0, policy_version 32533 (0.0009) [2023-10-10 17:50:44,263][123614] Updated weights for policy 1, policy_version 32450 (0.0007) [2023-10-10 17:50:44,432][123582] Updated weights for policy 0, policy_version 32543 (0.0008) [2023-10-10 17:50:44,626][123614] Updated weights for policy 1, policy_version 32460 (0.0008) [2023-10-10 17:50:44,997][123614] Updated weights for policy 1, policy_version 32470 (0.0008) [2023-10-10 17:50:45,368][123614] Updated weights for policy 1, policy_version 32480 (0.0007) [2023-10-10 17:50:48,145][123582] Updated weights for policy 0, policy_version 32553 (0.0010) [2023-10-10 17:50:48,517][123582] Updated weights for policy 0, policy_version 32563 (0.0011) [2023-10-10 17:50:48,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 66584576. Throughput: 0: 1807.0, 1: 1810.3. Samples: 16661670. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-10 17:50:48,788][122664] Avg episode reward: [(0, '52.450'), (1, '40.500')] [2023-10-10 17:50:48,891][123582] Updated weights for policy 0, policy_version 32573 (0.0009) [2023-10-10 17:50:49,158][123614] Updated weights for policy 1, policy_version 32490 (0.0007) [2023-10-10 17:50:49,522][123614] Updated weights for policy 1, policy_version 32500 (0.0009) [2023-10-10 17:50:49,890][123614] Updated weights for policy 1, policy_version 32510 (0.0009) [2023-10-10 17:50:52,691][123582] Updated weights for policy 0, policy_version 32583 (0.0008) [2023-10-10 17:50:53,064][123582] Updated weights for policy 0, policy_version 32593 (0.0009) [2023-10-10 17:50:53,423][123582] Updated weights for policy 0, policy_version 32603 (0.0010) [2023-10-10 17:50:53,460][123614] Updated weights for policy 1, policy_version 32520 (0.0008) [2023-10-10 17:50:53,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 66682880. Throughput: 0: 1809.8, 1: 1812.9. Samples: 16682442. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-10 17:50:53,789][122664] Avg episode reward: [(0, '49.420'), (1, '39.210')] [2023-10-10 17:50:53,843][123614] Updated weights for policy 1, policy_version 32530 (0.0008) [2023-10-10 17:50:54,208][123614] Updated weights for policy 1, policy_version 32540 (0.0008) [2023-10-10 17:50:57,166][123582] Updated weights for policy 0, policy_version 32613 (0.0008) [2023-10-10 17:50:57,529][123582] Updated weights for policy 0, policy_version 32623 (0.0009) [2023-10-10 17:50:57,849][123614] Updated weights for policy 1, policy_version 32550 (0.0009) [2023-10-10 17:50:57,906][123582] Updated weights for policy 0, policy_version 32633 (0.0009) [2023-10-10 17:50:58,218][123614] Updated weights for policy 1, policy_version 32560 (0.0007) [2023-10-10 17:50:58,587][123614] Updated weights for policy 1, policy_version 32570 (0.0010) [2023-10-10 17:50:58,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 66748416. Throughput: 0: 1809.4, 1: 1815.2. Samples: 16694292. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:50:58,789][122664] Avg episode reward: [(0, '48.550'), (1, '37.760')] [2023-10-10 17:51:01,621][123582] Updated weights for policy 0, policy_version 32643 (0.0008) [2023-10-10 17:51:01,994][123582] Updated weights for policy 0, policy_version 32653 (0.0007) [2023-10-10 17:51:02,275][123614] Updated weights for policy 1, policy_version 32580 (0.0009) [2023-10-10 17:51:02,363][123582] Updated weights for policy 0, policy_version 32663 (0.0008) [2023-10-10 17:51:02,645][123614] Updated weights for policy 1, policy_version 32590 (0.0007) [2023-10-10 17:51:03,008][123614] Updated weights for policy 1, policy_version 32600 (0.0008) [2023-10-10 17:51:03,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 66846720. Throughput: 0: 1814.5, 1: 1815.9. Samples: 16715208. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:51:03,789][122664] Avg episode reward: [(0, '55.010'), (1, '37.890')] [2023-10-10 17:51:03,790][123247] Saving new best policy, reward=55.010! [2023-10-10 17:51:05,955][123582] Updated weights for policy 0, policy_version 32673 (0.0009) [2023-10-10 17:51:06,340][123582] Updated weights for policy 0, policy_version 32683 (0.0009) [2023-10-10 17:51:06,530][123614] Updated weights for policy 1, policy_version 32610 (0.0007) [2023-10-10 17:51:06,707][123582] Updated weights for policy 0, policy_version 32693 (0.0009) [2023-10-10 17:51:06,901][123614] Updated weights for policy 1, policy_version 32620 (0.0008) [2023-10-10 17:51:07,086][123582] Updated weights for policy 0, policy_version 32703 (0.0008) [2023-10-10 17:51:07,264][123614] Updated weights for policy 1, policy_version 32630 (0.0009) [2023-10-10 17:51:07,633][123614] Updated weights for policy 1, policy_version 32640 (0.0009) [2023-10-10 17:51:08,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 66912256. Throughput: 0: 1806.2, 1: 1825.7. Samples: 16737218. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:51:08,789][122664] Avg episode reward: [(0, '56.470'), (1, '38.240')] [2023-10-10 17:51:08,798][123247] Saving new best policy, reward=56.470! [2023-10-10 17:51:10,737][123582] Updated weights for policy 0, policy_version 32713 (0.0008) [2023-10-10 17:51:11,119][123582] Updated weights for policy 0, policy_version 32723 (0.0007) [2023-10-10 17:51:11,487][123582] Updated weights for policy 0, policy_version 32733 (0.0007) [2023-10-10 17:51:11,501][123614] Updated weights for policy 1, policy_version 32650 (0.0007) [2023-10-10 17:51:11,867][123614] Updated weights for policy 1, policy_version 32660 (0.0008) [2023-10-10 17:51:12,235][123614] Updated weights for policy 1, policy_version 32670 (0.0007) [2023-10-10 17:51:13,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 66977792. Throughput: 0: 1814.6, 1: 1819.2. Samples: 16748084. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:51:13,789][122664] Avg episode reward: [(0, '56.940'), (1, '38.380')] [2023-10-10 17:51:13,790][123247] Saving new best policy, reward=56.940! [2023-10-10 17:51:15,098][123582] Updated weights for policy 0, policy_version 32743 (0.0008) [2023-10-10 17:51:15,468][123582] Updated weights for policy 0, policy_version 32753 (0.0008) [2023-10-10 17:51:15,850][123582] Updated weights for policy 0, policy_version 32763 (0.0008) [2023-10-10 17:51:15,945][123614] Updated weights for policy 1, policy_version 32680 (0.0008) [2023-10-10 17:51:16,310][123614] Updated weights for policy 1, policy_version 32690 (0.0009) [2023-10-10 17:51:16,676][123614] Updated weights for policy 1, policy_version 32700 (0.0007) [2023-10-10 17:51:18,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 67043328. Throughput: 0: 1806.4, 1: 1821.8. Samples: 16769846. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:51:18,789][122664] Avg episode reward: [(0, '52.310'), (1, '41.780')] [2023-10-10 17:51:19,462][123582] Updated weights for policy 0, policy_version 32773 (0.0007) [2023-10-10 17:51:19,835][123582] Updated weights for policy 0, policy_version 32783 (0.0009) [2023-10-10 17:51:20,213][123582] Updated weights for policy 0, policy_version 32793 (0.0008) [2023-10-10 17:51:20,427][123614] Updated weights for policy 1, policy_version 32710 (0.0009) [2023-10-10 17:51:20,793][123614] Updated weights for policy 1, policy_version 32720 (0.0008) [2023-10-10 17:51:21,161][123614] Updated weights for policy 1, policy_version 32730 (0.0008) [2023-10-10 17:51:23,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 67108864. Throughput: 0: 1812.9, 1: 1819.4. Samples: 16792760. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:51:23,788][122664] Avg episode reward: [(0, '48.800'), (1, '42.310')] [2023-10-10 17:51:24,033][123582] Updated weights for policy 0, policy_version 32803 (0.0008) [2023-10-10 17:51:24,428][123582] Updated weights for policy 0, policy_version 32813 (0.0008) [2023-10-10 17:51:24,800][123582] Updated weights for policy 0, policy_version 32823 (0.0007) [2023-10-10 17:51:24,877][123614] Updated weights for policy 1, policy_version 32740 (0.0009) [2023-10-10 17:51:25,255][123614] Updated weights for policy 1, policy_version 32750 (0.0008) [2023-10-10 17:51:25,636][123614] Updated weights for policy 1, policy_version 32760 (0.0007) [2023-10-10 17:51:28,552][123582] Updated weights for policy 0, policy_version 32833 (0.0009) [2023-10-10 17:51:28,788][122664] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 67174400. Throughput: 0: 1816.5, 1: 1817.3. Samples: 16802530. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:51:28,788][122664] Avg episode reward: [(0, '48.860'), (1, '39.780')] [2023-10-10 17:51:28,932][123582] Updated weights for policy 0, policy_version 32843 (0.0010) [2023-10-10 17:51:29,300][123582] Updated weights for policy 0, policy_version 32853 (0.0008) [2023-10-10 17:51:29,317][123614] Updated weights for policy 1, policy_version 32770 (0.0007) [2023-10-10 17:51:29,681][123582] Updated weights for policy 0, policy_version 32863 (0.0010) [2023-10-10 17:51:29,683][123614] Updated weights for policy 1, policy_version 32780 (0.0008) [2023-10-10 17:51:30,059][123614] Updated weights for policy 1, policy_version 32790 (0.0009) [2023-10-10 17:51:30,426][123614] Updated weights for policy 1, policy_version 32800 (0.0008) [2023-10-10 17:51:33,261][123582] Updated weights for policy 0, policy_version 32873 (0.0009) [2023-10-10 17:51:33,628][123582] Updated weights for policy 0, policy_version 32883 (0.0008) [2023-10-10 17:51:33,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 67239936. Throughput: 0: 1809.1, 1: 1820.8. Samples: 16825014. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:51:33,788][122664] Avg episode reward: [(0, '50.930'), (1, '39.330')] [2023-10-10 17:51:33,999][123582] Updated weights for policy 0, policy_version 32893 (0.0008) [2023-10-10 17:51:34,098][123614] Updated weights for policy 1, policy_version 32810 (0.0009) [2023-10-10 17:51:34,474][123614] Updated weights for policy 1, policy_version 32820 (0.0007) [2023-10-10 17:51:34,840][123614] Updated weights for policy 1, policy_version 32830 (0.0007) [2023-10-10 17:51:37,821][123582] Updated weights for policy 0, policy_version 32903 (0.0009) [2023-10-10 17:51:38,190][123582] Updated weights for policy 0, policy_version 32913 (0.0009) [2023-10-10 17:51:38,468][123614] Updated weights for policy 1, policy_version 32840 (0.0009) [2023-10-10 17:51:38,557][123582] Updated weights for policy 0, policy_version 32923 (0.0008) [2023-10-10 17:51:38,788][122664] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 67338240. Throughput: 0: 1812.0, 1: 1822.4. Samples: 16845992. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:51:38,789][122664] Avg episode reward: [(0, '48.850'), (1, '39.790')] [2023-10-10 17:51:38,839][123614] Updated weights for policy 1, policy_version 32850 (0.0009) [2023-10-10 17:51:39,198][123614] Updated weights for policy 1, policy_version 32860 (0.0009) [2023-10-10 17:51:42,197][123582] Updated weights for policy 0, policy_version 32933 (0.0008) [2023-10-10 17:51:42,559][123582] Updated weights for policy 0, policy_version 32943 (0.0011) [2023-10-10 17:51:42,803][123614] Updated weights for policy 1, policy_version 32870 (0.0008) [2023-10-10 17:51:42,932][123582] Updated weights for policy 0, policy_version 32953 (0.0009) [2023-10-10 17:51:43,165][123614] Updated weights for policy 1, policy_version 32880 (0.0009) [2023-10-10 17:51:43,537][123614] Updated weights for policy 1, policy_version 32890 (0.0010) [2023-10-10 17:51:43,788][122664] Fps is (10 sec: 19660.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 67436544. Throughput: 0: 1806.3, 1: 1820.9. Samples: 16857516. Policy #0 lag: (min: 6.0, avg: 13.5, max: 38.0) [2023-10-10 17:51:43,789][122664] Avg episode reward: [(0, '50.860'), (1, '43.300')] [2023-10-10 17:51:46,735][123582] Updated weights for policy 0, policy_version 32963 (0.0008) [2023-10-10 17:51:47,116][123582] Updated weights for policy 0, policy_version 32973 (0.0009) [2023-10-10 17:51:47,338][123614] Updated weights for policy 1, policy_version 32900 (0.0009) [2023-10-10 17:51:47,480][123582] Updated weights for policy 0, policy_version 32983 (0.0008) [2023-10-10 17:51:47,706][123614] Updated weights for policy 1, policy_version 32910 (0.0007) [2023-10-10 17:51:48,065][123614] Updated weights for policy 1, policy_version 32920 (0.0008) [2023-10-10 17:51:48,788][122664] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 67502080. Throughput: 0: 1808.2, 1: 1821.2. Samples: 16878532. Policy #0 lag: (min: 6.0, avg: 13.5, max: 38.0) [2023-10-10 17:51:48,789][122664] Avg episode reward: [(0, '50.380'), (1, '41.370')] [2023-10-10 17:51:51,224][123582] Updated weights for policy 0, policy_version 32993 (0.0009) [2023-10-10 17:51:51,586][123582] Updated weights for policy 0, policy_version 33003 (0.0008) [2023-10-10 17:51:51,798][123614] Updated weights for policy 1, policy_version 32930 (0.0008) [2023-10-10 17:51:51,959][123582] Updated weights for policy 0, policy_version 33013 (0.0007) [2023-10-10 17:51:52,164][123614] Updated weights for policy 1, policy_version 32940 (0.0009) [2023-10-10 17:51:52,335][123582] Updated weights for policy 0, policy_version 33023 (0.0008) [2023-10-10 17:51:52,537][123614] Updated weights for policy 1, policy_version 32950 (0.0008) [2023-10-10 17:51:52,901][123614] Updated weights for policy 1, policy_version 32960 (0.0011) [2023-10-10 17:51:53,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 67567616. Throughput: 0: 1802.0, 1: 1812.7. Samples: 16899884. Policy #0 lag: (min: 6.0, avg: 13.5, max: 38.0) [2023-10-10 17:51:53,789][122664] Avg episode reward: [(0, '45.000'), (1, '41.720')] [2023-10-10 17:51:53,801][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000033024_33816576.pth... [2023-10-10 17:51:53,801][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000032960_33751040.pth... [2023-10-10 17:51:53,838][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000031264_32014336.pth [2023-10-10 17:51:53,838][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000031328_32079872.pth [2023-10-10 17:51:56,087][123582] Updated weights for policy 0, policy_version 33033 (0.0007) [2023-10-10 17:51:56,456][123582] Updated weights for policy 0, policy_version 33043 (0.0008) [2023-10-10 17:51:56,652][123614] Updated weights for policy 1, policy_version 32970 (0.0008) [2023-10-10 17:51:56,824][123582] Updated weights for policy 0, policy_version 33053 (0.0009) [2023-10-10 17:51:57,018][123614] Updated weights for policy 1, policy_version 32980 (0.0008) [2023-10-10 17:51:57,381][123614] Updated weights for policy 1, policy_version 32990 (0.0011) [2023-10-10 17:51:58,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 67633152. Throughput: 0: 1808.8, 1: 1814.4. Samples: 16911128. Policy #0 lag: (min: 6.0, avg: 13.5, max: 38.0) [2023-10-10 17:51:58,789][122664] Avg episode reward: [(0, '42.270'), (1, '40.060')] [2023-10-10 17:52:00,621][123582] Updated weights for policy 0, policy_version 33063 (0.0009) [2023-10-10 17:52:01,005][123582] Updated weights for policy 0, policy_version 33073 (0.0007) [2023-10-10 17:52:01,191][123614] Updated weights for policy 1, policy_version 33000 (0.0010) [2023-10-10 17:52:01,365][123582] Updated weights for policy 0, policy_version 33083 (0.0007) [2023-10-10 17:52:01,568][123614] Updated weights for policy 1, policy_version 33010 (0.0009) [2023-10-10 17:52:01,940][123614] Updated weights for policy 1, policy_version 33020 (0.0008) [2023-10-10 17:52:03,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 67698688. Throughput: 0: 1791.3, 1: 1810.4. Samples: 16931922. Policy #0 lag: (min: 6.0, avg: 13.5, max: 38.0) [2023-10-10 17:52:03,789][122664] Avg episode reward: [(0, '40.250'), (1, '36.790')] [2023-10-10 17:52:05,119][123582] Updated weights for policy 0, policy_version 33093 (0.0008) [2023-10-10 17:52:05,483][123582] Updated weights for policy 0, policy_version 33103 (0.0007) [2023-10-10 17:52:05,550][123614] Updated weights for policy 1, policy_version 33030 (0.0009) [2023-10-10 17:52:05,856][123582] Updated weights for policy 0, policy_version 33113 (0.0009) [2023-10-10 17:52:05,911][123614] Updated weights for policy 1, policy_version 33040 (0.0007) [2023-10-10 17:52:06,276][123614] Updated weights for policy 1, policy_version 33050 (0.0009) [2023-10-10 17:52:08,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 67764224. Throughput: 0: 1786.4, 1: 1811.4. Samples: 16954658. Policy #0 lag: (min: 1.0, avg: 11.4, max: 33.0) [2023-10-10 17:52:08,788][122664] Avg episode reward: [(0, '37.940'), (1, '38.360')] [2023-10-10 17:52:09,557][123582] Updated weights for policy 0, policy_version 33123 (0.0008) [2023-10-10 17:52:09,947][123582] Updated weights for policy 0, policy_version 33133 (0.0009) [2023-10-10 17:52:10,046][123614] Updated weights for policy 1, policy_version 33060 (0.0008) [2023-10-10 17:52:10,311][123582] Updated weights for policy 0, policy_version 33143 (0.0009) [2023-10-10 17:52:10,431][123614] Updated weights for policy 1, policy_version 33070 (0.0008) [2023-10-10 17:52:10,797][123614] Updated weights for policy 1, policy_version 33080 (0.0008) [2023-10-10 17:52:13,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 67829760. Throughput: 0: 1786.1, 1: 1811.6. Samples: 16964428. Policy #0 lag: (min: 1.0, avg: 11.4, max: 33.0) [2023-10-10 17:52:13,789][122664] Avg episode reward: [(0, '36.910'), (1, '40.070')] [2023-10-10 17:52:13,938][123582] Updated weights for policy 0, policy_version 33153 (0.0009) [2023-10-10 17:52:14,311][123582] Updated weights for policy 0, policy_version 33163 (0.0010) [2023-10-10 17:52:14,604][123614] Updated weights for policy 1, policy_version 33090 (0.0008) [2023-10-10 17:52:14,681][123582] Updated weights for policy 0, policy_version 33173 (0.0008) [2023-10-10 17:52:14,980][123614] Updated weights for policy 1, policy_version 33100 (0.0008) [2023-10-10 17:52:15,058][123582] Updated weights for policy 0, policy_version 33183 (0.0008) [2023-10-10 17:52:15,352][123614] Updated weights for policy 1, policy_version 33110 (0.0008) [2023-10-10 17:52:15,723][123614] Updated weights for policy 1, policy_version 33120 (0.0007) [2023-10-10 17:52:18,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 67895296. Throughput: 0: 1796.0, 1: 1807.9. Samples: 16987188. Policy #0 lag: (min: 1.0, avg: 11.4, max: 33.0) [2023-10-10 17:52:18,788][122664] Avg episode reward: [(0, '35.060'), (1, '41.700')] [2023-10-10 17:52:18,960][123582] Updated weights for policy 0, policy_version 33193 (0.0007) [2023-10-10 17:52:19,339][123582] Updated weights for policy 0, policy_version 33203 (0.0009) [2023-10-10 17:52:19,607][123614] Updated weights for policy 1, policy_version 33130 (0.0008) [2023-10-10 17:52:19,716][123582] Updated weights for policy 0, policy_version 33213 (0.0009) [2023-10-10 17:52:19,977][123614] Updated weights for policy 1, policy_version 33140 (0.0010) [2023-10-10 17:52:20,347][123614] Updated weights for policy 1, policy_version 33150 (0.0009) [2023-10-10 17:52:23,417][123582] Updated weights for policy 0, policy_version 33223 (0.0007) [2023-10-10 17:52:23,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 67960832. Throughput: 0: 1811.2, 1: 1818.8. Samples: 17009344. Policy #0 lag: (min: 1.0, avg: 11.4, max: 33.0) [2023-10-10 17:52:23,789][122664] Avg episode reward: [(0, '34.610'), (1, '42.180')] [2023-10-10 17:52:23,793][123582] Updated weights for policy 0, policy_version 33233 (0.0007) [2023-10-10 17:52:24,123][123614] Updated weights for policy 1, policy_version 33160 (0.0008) [2023-10-10 17:52:24,164][123582] Updated weights for policy 0, policy_version 33243 (0.0007) [2023-10-10 17:52:24,495][123614] Updated weights for policy 1, policy_version 33170 (0.0009) [2023-10-10 17:52:24,864][123614] Updated weights for policy 1, policy_version 33180 (0.0008) [2023-10-10 17:52:27,821][123582] Updated weights for policy 0, policy_version 33253 (0.0008) [2023-10-10 17:52:28,202][123582] Updated weights for policy 0, policy_version 33263 (0.0009) [2023-10-10 17:52:28,394][123614] Updated weights for policy 1, policy_version 33190 (0.0007) [2023-10-10 17:52:28,567][123582] Updated weights for policy 0, policy_version 33273 (0.0008) [2023-10-10 17:52:28,770][123614] Updated weights for policy 1, policy_version 33200 (0.0008) [2023-10-10 17:52:28,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 68026368. Throughput: 0: 1795.8, 1: 1808.7. Samples: 17019718. Policy #0 lag: (min: 1.0, avg: 11.4, max: 33.0) [2023-10-10 17:52:28,788][122664] Avg episode reward: [(0, '33.900'), (1, '42.710')] [2023-10-10 17:52:29,137][123614] Updated weights for policy 1, policy_version 33210 (0.0008) [2023-10-10 17:52:32,248][123582] Updated weights for policy 0, policy_version 33283 (0.0007) [2023-10-10 17:52:32,625][123582] Updated weights for policy 0, policy_version 33293 (0.0008) [2023-10-10 17:52:32,726][123614] Updated weights for policy 1, policy_version 33220 (0.0009) [2023-10-10 17:52:33,009][123582] Updated weights for policy 0, policy_version 33303 (0.0009) [2023-10-10 17:52:33,101][123614] Updated weights for policy 1, policy_version 33230 (0.0009) [2023-10-10 17:52:33,465][123614] Updated weights for policy 1, policy_version 33240 (0.0009) [2023-10-10 17:52:33,788][122664] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 68157440. Throughput: 0: 1815.9, 1: 1818.3. Samples: 17042070. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:52:33,789][122664] Avg episode reward: [(0, '32.300'), (1, '42.410')] [2023-10-10 17:52:36,608][123582] Updated weights for policy 0, policy_version 33313 (0.0007) [2023-10-10 17:52:36,986][123582] Updated weights for policy 0, policy_version 33323 (0.0009) [2023-10-10 17:52:37,157][123614] Updated weights for policy 1, policy_version 33250 (0.0011) [2023-10-10 17:52:37,359][123582] Updated weights for policy 0, policy_version 33333 (0.0009) [2023-10-10 17:52:37,531][123614] Updated weights for policy 1, policy_version 33260 (0.0007) [2023-10-10 17:52:37,720][123582] Updated weights for policy 0, policy_version 33343 (0.0008) [2023-10-10 17:52:37,896][123614] Updated weights for policy 1, policy_version 33270 (0.0008) [2023-10-10 17:52:38,260][123614] Updated weights for policy 1, policy_version 33280 (0.0007) [2023-10-10 17:52:38,788][122664] Fps is (10 sec: 19660.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 68222976. Throughput: 0: 1809.6, 1: 1808.1. Samples: 17062682. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:52:38,789][122664] Avg episode reward: [(0, '29.850'), (1, '40.410')] [2023-10-10 17:52:41,377][123582] Updated weights for policy 0, policy_version 33353 (0.0010) [2023-10-10 17:52:41,758][123582] Updated weights for policy 0, policy_version 33363 (0.0008) [2023-10-10 17:52:42,028][123614] Updated weights for policy 1, policy_version 33290 (0.0007) [2023-10-10 17:52:42,136][123582] Updated weights for policy 0, policy_version 33373 (0.0009) [2023-10-10 17:52:42,392][123614] Updated weights for policy 1, policy_version 33300 (0.0008) [2023-10-10 17:52:42,760][123614] Updated weights for policy 1, policy_version 33310 (0.0007) [2023-10-10 17:52:43,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 68288512. Throughput: 0: 1816.6, 1: 1821.7. Samples: 17074848. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:52:43,789][122664] Avg episode reward: [(0, '32.900'), (1, '44.470')] [2023-10-10 17:52:45,881][123582] Updated weights for policy 0, policy_version 33383 (0.0008) [2023-10-10 17:52:46,257][123582] Updated weights for policy 0, policy_version 33393 (0.0007) [2023-10-10 17:52:46,545][123614] Updated weights for policy 1, policy_version 33320 (0.0009) [2023-10-10 17:52:46,616][123582] Updated weights for policy 0, policy_version 33403 (0.0008) [2023-10-10 17:52:46,910][123614] Updated weights for policy 1, policy_version 33330 (0.0007) [2023-10-10 17:52:47,278][123614] Updated weights for policy 1, policy_version 33340 (0.0009) [2023-10-10 17:52:48,788][122664] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 68354048. Throughput: 0: 1809.7, 1: 1809.7. Samples: 17094794. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:52:48,788][122664] Avg episode reward: [(0, '34.340'), (1, '48.620')] [2023-10-10 17:52:48,789][123465] Saving new best policy, reward=48.620! [2023-10-10 17:52:50,303][123582] Updated weights for policy 0, policy_version 33413 (0.0008) [2023-10-10 17:52:50,673][123582] Updated weights for policy 0, policy_version 33423 (0.0008) [2023-10-10 17:52:51,038][123582] Updated weights for policy 0, policy_version 33433 (0.0009) [2023-10-10 17:52:51,039][123614] Updated weights for policy 1, policy_version 33350 (0.0008) [2023-10-10 17:52:51,400][123614] Updated weights for policy 1, policy_version 33360 (0.0008) [2023-10-10 17:52:51,772][123614] Updated weights for policy 1, policy_version 33370 (0.0007) [2023-10-10 17:52:53,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 68419584. Throughput: 0: 1815.0, 1: 1805.0. Samples: 17117560. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:52:53,789][122664] Avg episode reward: [(0, '35.530'), (1, '47.330')] [2023-10-10 17:52:54,785][123582] Updated weights for policy 0, policy_version 33443 (0.0009) [2023-10-10 17:52:55,155][123582] Updated weights for policy 0, policy_version 33453 (0.0011) [2023-10-10 17:52:55,535][123582] Updated weights for policy 0, policy_version 33463 (0.0009) [2023-10-10 17:52:55,554][123614] Updated weights for policy 1, policy_version 33380 (0.0007) [2023-10-10 17:52:55,944][123614] Updated weights for policy 1, policy_version 33390 (0.0008) [2023-10-10 17:52:56,309][123614] Updated weights for policy 1, policy_version 33400 (0.0008) [2023-10-10 17:52:58,788][122664] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 68485120. Throughput: 0: 1813.6, 1: 1804.8. Samples: 17127256. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-10 17:52:58,789][122664] Avg episode reward: [(0, '39.500'), (1, '47.440')] [2023-10-10 17:52:59,295][123582] Updated weights for policy 0, policy_version 33473 (0.0008) [2023-10-10 17:52:59,671][123582] Updated weights for policy 0, policy_version 33483 (0.0009) [2023-10-10 17:52:59,946][123614] Updated weights for policy 1, policy_version 33410 (0.0008) [2023-10-10 17:53:00,034][123582] Updated weights for policy 0, policy_version 33493 (0.0007) [2023-10-10 17:53:00,310][123614] Updated weights for policy 1, policy_version 33420 (0.0008) [2023-10-10 17:53:00,403][123582] Updated weights for policy 0, policy_version 33503 (0.0007) [2023-10-10 17:53:00,679][123614] Updated weights for policy 1, policy_version 33430 (0.0009) [2023-10-10 17:53:01,051][123614] Updated weights for policy 1, policy_version 33440 (0.0008) [2023-10-10 17:53:03,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 68550656. Throughput: 0: 1808.7, 1: 1806.7. Samples: 17149880. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-10 17:53:03,789][122664] Avg episode reward: [(0, '42.230'), (1, '44.690')] [2023-10-10 17:53:04,067][123582] Updated weights for policy 0, policy_version 33513 (0.0008) [2023-10-10 17:53:04,441][123582] Updated weights for policy 0, policy_version 33523 (0.0009) [2023-10-10 17:53:04,770][123614] Updated weights for policy 1, policy_version 33450 (0.0010) [2023-10-10 17:53:04,801][123582] Updated weights for policy 0, policy_version 33533 (0.0009) [2023-10-10 17:53:05,142][123614] Updated weights for policy 1, policy_version 33460 (0.0010) [2023-10-10 17:53:05,512][123614] Updated weights for policy 1, policy_version 33470 (0.0010) [2023-10-10 17:53:08,420][123582] Updated weights for policy 0, policy_version 33543 (0.0007) [2023-10-10 17:53:08,788][122664] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 68616192. Throughput: 0: 1816.9, 1: 1811.2. Samples: 17172612. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-10 17:53:08,789][122664] Avg episode reward: [(0, '39.450'), (1, '43.960')] [2023-10-10 17:53:08,803][123582] Updated weights for policy 0, policy_version 33553 (0.0010) [2023-10-10 17:53:09,118][123614] Updated weights for policy 1, policy_version 33480 (0.0008) [2023-10-10 17:53:09,170][123582] Updated weights for policy 0, policy_version 33563 (0.0009) [2023-10-10 17:53:09,492][123614] Updated weights for policy 1, policy_version 33490 (0.0008) [2023-10-10 17:53:09,863][123614] Updated weights for policy 1, policy_version 33500 (0.0009) [2023-10-10 17:53:12,954][123582] Updated weights for policy 0, policy_version 33573 (0.0007) [2023-10-10 17:53:13,329][123582] Updated weights for policy 0, policy_version 33583 (0.0008) [2023-10-10 17:53:13,596][123614] Updated weights for policy 1, policy_version 33510 (0.0007) [2023-10-10 17:53:13,696][123582] Updated weights for policy 0, policy_version 33593 (0.0007) [2023-10-10 17:53:13,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 68681728. Throughput: 0: 1815.7, 1: 1805.9. Samples: 17182688. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-10 17:53:13,788][122664] Avg episode reward: [(0, '38.550'), (1, '44.700')] [2023-10-10 17:53:13,967][123614] Updated weights for policy 1, policy_version 33520 (0.0007) [2023-10-10 17:53:14,340][123614] Updated weights for policy 1, policy_version 33530 (0.0008) [2023-10-10 17:53:17,415][123582] Updated weights for policy 0, policy_version 33603 (0.0008) [2023-10-10 17:53:17,789][123582] Updated weights for policy 0, policy_version 33613 (0.0008) [2023-10-10 17:53:17,932][123614] Updated weights for policy 1, policy_version 33540 (0.0008) [2023-10-10 17:53:18,162][123582] Updated weights for policy 0, policy_version 33623 (0.0007) [2023-10-10 17:53:18,296][123614] Updated weights for policy 1, policy_version 33550 (0.0009) [2023-10-10 17:53:18,667][123614] Updated weights for policy 1, policy_version 33560 (0.0007) [2023-10-10 17:53:18,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 68780032. Throughput: 0: 1814.0, 1: 1812.4. Samples: 17205256. Policy #0 lag: (min: 17.0, avg: 25.7, max: 49.0) [2023-10-10 17:53:18,789][122664] Avg episode reward: [(0, '37.350'), (1, '46.010')] [2023-10-10 17:53:21,728][123582] Updated weights for policy 0, policy_version 33633 (0.0008) [2023-10-10 17:53:22,109][123582] Updated weights for policy 0, policy_version 33643 (0.0009) [2023-10-10 17:53:22,429][123614] Updated weights for policy 1, policy_version 33570 (0.0007) [2023-10-10 17:53:22,476][123582] Updated weights for policy 0, policy_version 33653 (0.0009) [2023-10-10 17:53:22,791][123614] Updated weights for policy 1, policy_version 33580 (0.0007) [2023-10-10 17:53:22,845][123582] Updated weights for policy 0, policy_version 33663 (0.0007) [2023-10-10 17:53:23,153][123614] Updated weights for policy 1, policy_version 33590 (0.0007) [2023-10-10 17:53:23,522][123614] Updated weights for policy 1, policy_version 33600 (0.0008) [2023-10-10 17:53:23,788][122664] Fps is (10 sec: 19660.7, 60 sec: 15291.8, 300 sec: 14551.2). Total num frames: 68878336. Throughput: 0: 1807.2, 1: 1807.2. Samples: 17225330. Policy #0 lag: (min: 17.0, avg: 25.7, max: 49.0) [2023-10-10 17:53:23,788][122664] Avg episode reward: [(0, '38.570'), (1, '44.530')] [2023-10-10 17:53:26,501][123582] Updated weights for policy 0, policy_version 33673 (0.0010) [2023-10-10 17:53:26,867][123582] Updated weights for policy 0, policy_version 33683 (0.0008) [2023-10-10 17:53:27,171][123614] Updated weights for policy 1, policy_version 33610 (0.0007) [2023-10-10 17:53:27,243][123582] Updated weights for policy 0, policy_version 33693 (0.0009) [2023-10-10 17:53:27,546][123614] Updated weights for policy 1, policy_version 33620 (0.0008) [2023-10-10 17:53:27,923][123614] Updated weights for policy 1, policy_version 33630 (0.0009) [2023-10-10 17:53:28,788][122664] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 68943872. Throughput: 0: 1816.1, 1: 1808.3. Samples: 17237948. Policy #0 lag: (min: 17.0, avg: 25.7, max: 49.0) [2023-10-10 17:53:28,789][122664] Avg episode reward: [(0, '43.910'), (1, '44.080')] [2023-10-10 17:53:31,005][123582] Updated weights for policy 0, policy_version 33703 (0.0009) [2023-10-10 17:53:31,371][123582] Updated weights for policy 0, policy_version 33713 (0.0008) [2023-10-10 17:53:31,702][123614] Updated weights for policy 1, policy_version 33640 (0.0009) [2023-10-10 17:53:31,743][123582] Updated weights for policy 0, policy_version 33723 (0.0008) [2023-10-10 17:53:32,068][123614] Updated weights for policy 1, policy_version 33650 (0.0008) [2023-10-10 17:53:32,431][123614] Updated weights for policy 1, policy_version 33660 (0.0008) [2023-10-10 17:53:33,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 69009408. Throughput: 0: 1814.8, 1: 1810.5. Samples: 17257932. Policy #0 lag: (min: 17.0, avg: 25.7, max: 49.0) [2023-10-10 17:53:33,789][122664] Avg episode reward: [(0, '44.190'), (1, '44.500')] [2023-10-10 17:53:35,513][123582] Updated weights for policy 0, policy_version 33733 (0.0008) [2023-10-10 17:53:35,886][123582] Updated weights for policy 0, policy_version 33743 (0.0008) [2023-10-10 17:53:36,254][123582] Updated weights for policy 0, policy_version 33753 (0.0009) [2023-10-10 17:53:36,257][123614] Updated weights for policy 1, policy_version 33670 (0.0008) [2023-10-10 17:53:36,620][123614] Updated weights for policy 1, policy_version 33680 (0.0009) [2023-10-10 17:53:36,992][123614] Updated weights for policy 1, policy_version 33690 (0.0008) [2023-10-10 17:53:38,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 69074944. Throughput: 0: 1810.1, 1: 1811.2. Samples: 17280514. Policy #0 lag: (min: 17.0, avg: 25.7, max: 49.0) [2023-10-10 17:53:38,789][122664] Avg episode reward: [(0, '43.830'), (1, '41.510')] [2023-10-10 17:53:40,003][123582] Updated weights for policy 0, policy_version 33763 (0.0007) [2023-10-10 17:53:40,397][123582] Updated weights for policy 0, policy_version 33773 (0.0008) [2023-10-10 17:53:40,731][123614] Updated weights for policy 1, policy_version 33700 (0.0009) [2023-10-10 17:53:40,773][123582] Updated weights for policy 0, policy_version 33783 (0.0008) [2023-10-10 17:53:41,125][123614] Updated weights for policy 1, policy_version 33710 (0.0007) [2023-10-10 17:53:41,489][123614] Updated weights for policy 1, policy_version 33720 (0.0007) [2023-10-10 17:53:43,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 69140480. Throughput: 0: 1809.4, 1: 1815.8. Samples: 17290388. Policy #0 lag: (min: 21.0, avg: 26.3, max: 53.0) [2023-10-10 17:53:43,789][122664] Avg episode reward: [(0, '45.830'), (1, '41.340')] [2023-10-10 17:53:44,572][123582] Updated weights for policy 0, policy_version 33793 (0.0009) [2023-10-10 17:53:44,929][123582] Updated weights for policy 0, policy_version 33803 (0.0007) [2023-10-10 17:53:45,230][123614] Updated weights for policy 1, policy_version 33730 (0.0007) [2023-10-10 17:53:45,301][123582] Updated weights for policy 0, policy_version 33813 (0.0008) [2023-10-10 17:53:45,606][123614] Updated weights for policy 1, policy_version 33740 (0.0007) [2023-10-10 17:53:45,675][123582] Updated weights for policy 0, policy_version 33823 (0.0008) [2023-10-10 17:53:45,969][123614] Updated weights for policy 1, policy_version 33750 (0.0008) [2023-10-10 17:53:46,336][123614] Updated weights for policy 1, policy_version 33760 (0.0007) [2023-10-10 17:53:48,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 69206016. Throughput: 0: 1807.1, 1: 1811.8. Samples: 17312728. Policy #0 lag: (min: 21.0, avg: 26.3, max: 53.0) [2023-10-10 17:53:48,789][122664] Avg episode reward: [(0, '42.640'), (1, '43.330')] [2023-10-10 17:53:49,430][123582] Updated weights for policy 0, policy_version 33833 (0.0007) [2023-10-10 17:53:49,795][123582] Updated weights for policy 0, policy_version 33843 (0.0008) [2023-10-10 17:53:50,056][123614] Updated weights for policy 1, policy_version 33770 (0.0009) [2023-10-10 17:53:50,167][123582] Updated weights for policy 0, policy_version 33853 (0.0008) [2023-10-10 17:53:50,413][123614] Updated weights for policy 1, policy_version 33780 (0.0007) [2023-10-10 17:53:50,785][123614] Updated weights for policy 1, policy_version 33790 (0.0010) [2023-10-10 17:53:53,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 69271552. Throughput: 0: 1806.3, 1: 1801.7. Samples: 17334970. Policy #0 lag: (min: 21.0, avg: 26.3, max: 53.0) [2023-10-10 17:53:53,789][122664] Avg episode reward: [(0, '42.300'), (1, '45.400')] [2023-10-10 17:53:53,801][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000033792_34603008.pth... [2023-10-10 17:53:53,832][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000032096_32866304.pth [2023-10-10 17:53:53,836][123465] Saving a milestone ./train_atari/atari_demonattack_APPO/checkpoint_p1/milestones/checkpoint_000033792_34603008.pth [2023-10-10 17:53:54,050][123582] Updated weights for policy 0, policy_version 33863 (0.0008) [2023-10-10 17:53:54,412][123582] Updated weights for policy 0, policy_version 33873 (0.0008) [2023-10-10 17:53:54,574][123614] Updated weights for policy 1, policy_version 33800 (0.0008) [2023-10-10 17:53:54,786][123582] Updated weights for policy 0, policy_version 33883 (0.0009) [2023-10-10 17:53:54,937][123614] Updated weights for policy 1, policy_version 33810 (0.0008) [2023-10-10 17:53:54,970][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000033888_34701312.pth... [2023-10-10 17:53:54,998][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000032192_32964608.pth [2023-10-10 17:53:55,002][123247] Saving a milestone ./train_atari/atari_demonattack_APPO/checkpoint_p0/milestones/checkpoint_000033888_34701312.pth [2023-10-10 17:53:55,309][123614] Updated weights for policy 1, policy_version 33820 (0.0009) [2023-10-10 17:53:58,393][123582] Updated weights for policy 0, policy_version 33893 (0.0008) [2023-10-10 17:53:58,762][123582] Updated weights for policy 0, policy_version 33903 (0.0008) [2023-10-10 17:53:58,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 69337088. Throughput: 0: 1804.2, 1: 1803.1. Samples: 17345016. Policy #0 lag: (min: 21.0, avg: 26.3, max: 53.0) [2023-10-10 17:53:58,788][122664] Avg episode reward: [(0, '41.960'), (1, '43.540')] [2023-10-10 17:53:58,801][123614] Updated weights for policy 1, policy_version 33830 (0.0007) [2023-10-10 17:53:59,138][123582] Updated weights for policy 0, policy_version 33913 (0.0009) [2023-10-10 17:53:59,166][123614] Updated weights for policy 1, policy_version 33840 (0.0008) [2023-10-10 17:53:59,540][123614] Updated weights for policy 1, policy_version 33850 (0.0009) [2023-10-10 17:54:02,783][123582] Updated weights for policy 0, policy_version 33923 (0.0008) [2023-10-10 17:54:03,157][123582] Updated weights for policy 0, policy_version 33933 (0.0007) [2023-10-10 17:54:03,316][123614] Updated weights for policy 1, policy_version 33860 (0.0009) [2023-10-10 17:54:03,526][123582] Updated weights for policy 0, policy_version 33943 (0.0007) [2023-10-10 17:54:03,677][123614] Updated weights for policy 1, policy_version 33870 (0.0007) [2023-10-10 17:54:03,788][122664] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 69402624. Throughput: 0: 1807.8, 1: 1798.8. Samples: 17367554. Policy #0 lag: (min: 21.0, avg: 26.3, max: 53.0) [2023-10-10 17:54:03,788][122664] Avg episode reward: [(0, '43.900'), (1, '43.270')] [2023-10-10 17:54:04,041][123614] Updated weights for policy 1, policy_version 33880 (0.0008) [2023-10-10 17:54:07,158][123582] Updated weights for policy 0, policy_version 33953 (0.0008) [2023-10-10 17:54:07,533][123582] Updated weights for policy 0, policy_version 33963 (0.0008) [2023-10-10 17:54:07,798][123614] Updated weights for policy 1, policy_version 33890 (0.0008) [2023-10-10 17:54:07,902][123582] Updated weights for policy 0, policy_version 33973 (0.0008) [2023-10-10 17:54:08,169][123614] Updated weights for policy 1, policy_version 33900 (0.0007) [2023-10-10 17:54:08,263][123582] Updated weights for policy 0, policy_version 33983 (0.0007) [2023-10-10 17:54:08,530][123614] Updated weights for policy 1, policy_version 33910 (0.0009) [2023-10-10 17:54:08,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 69500928. Throughput: 0: 1801.0, 1: 1807.3. Samples: 17387702. Policy #0 lag: (min: 18.0, avg: 31.4, max: 32.0) [2023-10-10 17:54:08,789][122664] Avg episode reward: [(0, '44.860'), (1, '43.230')] [2023-10-10 17:54:08,899][123614] Updated weights for policy 1, policy_version 33920 (0.0009) [2023-10-10 17:54:11,979][123582] Updated weights for policy 0, policy_version 33993 (0.0008) [2023-10-10 17:54:12,354][123582] Updated weights for policy 0, policy_version 34003 (0.0008) [2023-10-10 17:54:12,678][123614] Updated weights for policy 1, policy_version 33930 (0.0009) [2023-10-10 17:54:12,727][123582] Updated weights for policy 0, policy_version 34013 (0.0008) [2023-10-10 17:54:13,044][123614] Updated weights for policy 1, policy_version 33940 (0.0008) [2023-10-10 17:54:13,408][123614] Updated weights for policy 1, policy_version 33950 (0.0009) [2023-10-10 17:54:13,788][122664] Fps is (10 sec: 19660.3, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 69599232. Throughput: 0: 1806.8, 1: 1799.1. Samples: 17400212. Policy #0 lag: (min: 18.0, avg: 31.4, max: 32.0) [2023-10-10 17:54:13,789][122664] Avg episode reward: [(0, '43.850'), (1, '43.650')] [2023-10-10 17:54:16,424][123582] Updated weights for policy 0, policy_version 34023 (0.0010) [2023-10-10 17:54:16,804][123582] Updated weights for policy 0, policy_version 34033 (0.0010) [2023-10-10 17:54:17,153][123614] Updated weights for policy 1, policy_version 33960 (0.0008) [2023-10-10 17:54:17,176][123582] Updated weights for policy 0, policy_version 34043 (0.0008) [2023-10-10 17:54:17,515][123614] Updated weights for policy 1, policy_version 33970 (0.0008) [2023-10-10 17:54:17,892][123614] Updated weights for policy 1, policy_version 33980 (0.0008) [2023-10-10 17:54:18,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 69664768. Throughput: 0: 1798.3, 1: 1807.6. Samples: 17420196. Policy #0 lag: (min: 18.0, avg: 31.4, max: 32.0) [2023-10-10 17:54:18,788][122664] Avg episode reward: [(0, '41.470'), (1, '41.970')] [2023-10-10 17:54:20,751][123582] Updated weights for policy 0, policy_version 34053 (0.0008) [2023-10-10 17:54:21,124][123582] Updated weights for policy 0, policy_version 34063 (0.0009) [2023-10-10 17:54:21,490][123582] Updated weights for policy 0, policy_version 34073 (0.0007) [2023-10-10 17:54:21,588][123614] Updated weights for policy 1, policy_version 33990 (0.0007) [2023-10-10 17:54:21,950][123614] Updated weights for policy 1, policy_version 34000 (0.0007) [2023-10-10 17:54:22,325][123614] Updated weights for policy 1, policy_version 34010 (0.0010) [2023-10-10 17:54:23,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 69730304. Throughput: 0: 1803.4, 1: 1797.3. Samples: 17442546. Policy #0 lag: (min: 18.0, avg: 31.4, max: 32.0) [2023-10-10 17:54:23,789][122664] Avg episode reward: [(0, '40.130'), (1, '40.880')] [2023-10-10 17:54:25,106][123582] Updated weights for policy 0, policy_version 34083 (0.0008) [2023-10-10 17:54:25,515][123582] Updated weights for policy 0, policy_version 34093 (0.0008) [2023-10-10 17:54:25,893][123582] Updated weights for policy 0, policy_version 34103 (0.0008) [2023-10-10 17:54:26,011][123614] Updated weights for policy 1, policy_version 34020 (0.0008) [2023-10-10 17:54:26,394][123614] Updated weights for policy 1, policy_version 34030 (0.0009) [2023-10-10 17:54:26,772][123614] Updated weights for policy 1, policy_version 34040 (0.0009) [2023-10-10 17:54:28,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 69795840. Throughput: 0: 1806.0, 1: 1805.9. Samples: 17452926. Policy #0 lag: (min: 10.0, avg: 11.5, max: 37.0) [2023-10-10 17:54:28,789][122664] Avg episode reward: [(0, '41.780'), (1, '43.080')] [2023-10-10 17:54:29,582][123582] Updated weights for policy 0, policy_version 34113 (0.0008) [2023-10-10 17:54:29,954][123582] Updated weights for policy 0, policy_version 34123 (0.0009) [2023-10-10 17:54:30,327][123582] Updated weights for policy 0, policy_version 34133 (0.0009) [2023-10-10 17:54:30,550][123614] Updated weights for policy 1, policy_version 34050 (0.0009) [2023-10-10 17:54:30,698][123582] Updated weights for policy 0, policy_version 34143 (0.0009) [2023-10-10 17:54:30,908][123614] Updated weights for policy 1, policy_version 34060 (0.0008) [2023-10-10 17:54:31,283][123614] Updated weights for policy 1, policy_version 34070 (0.0010) [2023-10-10 17:54:31,653][123614] Updated weights for policy 1, policy_version 34080 (0.0009) [2023-10-10 17:54:33,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 69861376. Throughput: 0: 1808.0, 1: 1794.3. Samples: 17474832. Policy #0 lag: (min: 10.0, avg: 11.5, max: 37.0) [2023-10-10 17:54:33,788][122664] Avg episode reward: [(0, '41.960'), (1, '40.880')] [2023-10-10 17:54:34,495][123582] Updated weights for policy 0, policy_version 34153 (0.0008) [2023-10-10 17:54:34,877][123582] Updated weights for policy 0, policy_version 34163 (0.0009) [2023-10-10 17:54:35,258][123582] Updated weights for policy 0, policy_version 34173 (0.0010) [2023-10-10 17:54:35,456][123614] Updated weights for policy 1, policy_version 34090 (0.0007) [2023-10-10 17:54:35,817][123614] Updated weights for policy 1, policy_version 34100 (0.0008) [2023-10-10 17:54:36,186][123614] Updated weights for policy 1, policy_version 34110 (0.0008) [2023-10-10 17:54:38,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 69926912. Throughput: 0: 1813.9, 1: 1796.4. Samples: 17497434. Policy #0 lag: (min: 10.0, avg: 11.5, max: 37.0) [2023-10-10 17:54:38,788][122664] Avg episode reward: [(0, '43.760'), (1, '41.480')] [2023-10-10 17:54:38,941][123582] Updated weights for policy 0, policy_version 34183 (0.0009) [2023-10-10 17:54:39,300][123582] Updated weights for policy 0, policy_version 34193 (0.0009) [2023-10-10 17:54:39,681][123582] Updated weights for policy 0, policy_version 34203 (0.0007) [2023-10-10 17:54:39,907][123614] Updated weights for policy 1, policy_version 34120 (0.0009) [2023-10-10 17:54:40,265][123614] Updated weights for policy 1, policy_version 34130 (0.0009) [2023-10-10 17:54:40,640][123614] Updated weights for policy 1, policy_version 34140 (0.0009) [2023-10-10 17:54:43,271][123582] Updated weights for policy 0, policy_version 34213 (0.0008) [2023-10-10 17:54:43,650][123582] Updated weights for policy 0, policy_version 34223 (0.0010) [2023-10-10 17:54:43,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 69992448. Throughput: 0: 1811.8, 1: 1799.3. Samples: 17507516. Policy #0 lag: (min: 10.0, avg: 11.5, max: 37.0) [2023-10-10 17:54:43,788][122664] Avg episode reward: [(0, '43.310'), (1, '40.160')] [2023-10-10 17:54:44,021][123582] Updated weights for policy 0, policy_version 34233 (0.0009) [2023-10-10 17:54:44,183][123614] Updated weights for policy 1, policy_version 34150 (0.0007) [2023-10-10 17:54:44,547][123614] Updated weights for policy 1, policy_version 34160 (0.0008) [2023-10-10 17:54:44,911][123614] Updated weights for policy 1, policy_version 34170 (0.0011) [2023-10-10 17:54:47,715][123582] Updated weights for policy 0, policy_version 34243 (0.0009) [2023-10-10 17:54:48,082][123582] Updated weights for policy 0, policy_version 34253 (0.0009) [2023-10-10 17:54:48,457][123582] Updated weights for policy 0, policy_version 34263 (0.0008) [2023-10-10 17:54:48,667][123614] Updated weights for policy 1, policy_version 34180 (0.0009) [2023-10-10 17:54:48,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 70057984. Throughput: 0: 1818.7, 1: 1802.3. Samples: 17530498. Policy #0 lag: (min: 10.0, avg: 11.5, max: 37.0) [2023-10-10 17:54:48,788][122664] Avg episode reward: [(0, '43.970'), (1, '41.700')] [2023-10-10 17:54:49,033][123614] Updated weights for policy 1, policy_version 34190 (0.0007) [2023-10-10 17:54:49,401][123614] Updated weights for policy 1, policy_version 34200 (0.0008) [2023-10-10 17:54:52,130][123582] Updated weights for policy 0, policy_version 34273 (0.0008) [2023-10-10 17:54:52,487][123582] Updated weights for policy 0, policy_version 34283 (0.0009) [2023-10-10 17:54:52,860][123582] Updated weights for policy 0, policy_version 34293 (0.0010) [2023-10-10 17:54:53,174][123614] Updated weights for policy 1, policy_version 34210 (0.0010) [2023-10-10 17:54:53,232][123582] Updated weights for policy 0, policy_version 34303 (0.0007) [2023-10-10 17:54:53,545][123614] Updated weights for policy 1, policy_version 34220 (0.0008) [2023-10-10 17:54:53,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 70156288. Throughput: 0: 1818.8, 1: 1808.8. Samples: 17550946. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:54:53,789][122664] Avg episode reward: [(0, '42.580'), (1, '41.310')] [2023-10-10 17:54:53,910][123614] Updated weights for policy 1, policy_version 34230 (0.0009) [2023-10-10 17:54:54,274][123614] Updated weights for policy 1, policy_version 34240 (0.0008) [2023-10-10 17:54:56,854][123582] Updated weights for policy 0, policy_version 34313 (0.0007) [2023-10-10 17:54:57,228][123582] Updated weights for policy 0, policy_version 34323 (0.0009) [2023-10-10 17:54:57,598][123582] Updated weights for policy 0, policy_version 34333 (0.0008) [2023-10-10 17:54:57,990][123614] Updated weights for policy 1, policy_version 34250 (0.0008) [2023-10-10 17:54:58,370][123614] Updated weights for policy 1, policy_version 34260 (0.0009) [2023-10-10 17:54:58,735][123614] Updated weights for policy 1, policy_version 34270 (0.0009) [2023-10-10 17:54:58,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 70221824. Throughput: 0: 1820.1, 1: 1802.5. Samples: 17563232. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:54:58,788][122664] Avg episode reward: [(0, '43.390'), (1, '40.980')] [2023-10-10 17:55:01,306][123582] Updated weights for policy 0, policy_version 34343 (0.0008) [2023-10-10 17:55:01,685][123582] Updated weights for policy 0, policy_version 34353 (0.0007) [2023-10-10 17:55:02,051][123582] Updated weights for policy 0, policy_version 34363 (0.0007) [2023-10-10 17:55:02,309][123614] Updated weights for policy 1, policy_version 34280 (0.0007) [2023-10-10 17:55:02,683][123614] Updated weights for policy 1, policy_version 34290 (0.0009) [2023-10-10 17:55:03,048][123614] Updated weights for policy 1, policy_version 34300 (0.0008) [2023-10-10 17:55:03,788][122664] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 70320128. Throughput: 0: 1822.7, 1: 1811.8. Samples: 17583748. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:55:03,788][122664] Avg episode reward: [(0, '45.370'), (1, '40.950')] [2023-10-10 17:55:05,728][123582] Updated weights for policy 0, policy_version 34373 (0.0009) [2023-10-10 17:55:06,099][123582] Updated weights for policy 0, policy_version 34383 (0.0009) [2023-10-10 17:55:06,487][123582] Updated weights for policy 0, policy_version 34393 (0.0009) [2023-10-10 17:55:06,715][123614] Updated weights for policy 1, policy_version 34310 (0.0008) [2023-10-10 17:55:07,085][123614] Updated weights for policy 1, policy_version 34320 (0.0009) [2023-10-10 17:55:07,455][123614] Updated weights for policy 1, policy_version 34330 (0.0010) [2023-10-10 17:55:08,788][122664] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 70385664. Throughput: 0: 1822.0, 1: 1809.2. Samples: 17605946. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:55:08,789][122664] Avg episode reward: [(0, '46.210'), (1, '44.350')] [2023-10-10 17:55:10,335][123582] Updated weights for policy 0, policy_version 34403 (0.0008) [2023-10-10 17:55:10,725][123582] Updated weights for policy 0, policy_version 34413 (0.0009) [2023-10-10 17:55:11,101][123582] Updated weights for policy 0, policy_version 34423 (0.0009) [2023-10-10 17:55:11,254][123614] Updated weights for policy 1, policy_version 34340 (0.0009) [2023-10-10 17:55:11,643][123614] Updated weights for policy 1, policy_version 34350 (0.0008) [2023-10-10 17:55:12,011][123614] Updated weights for policy 1, policy_version 34360 (0.0007) [2023-10-10 17:55:13,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 70451200. Throughput: 0: 1815.3, 1: 1814.1. Samples: 17616246. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:55:13,788][122664] Avg episode reward: [(0, '48.020'), (1, '48.540')] [2023-10-10 17:55:14,787][123582] Updated weights for policy 0, policy_version 34433 (0.0008) [2023-10-10 17:55:15,155][123582] Updated weights for policy 0, policy_version 34443 (0.0007) [2023-10-10 17:55:15,529][123582] Updated weights for policy 0, policy_version 34453 (0.0007) [2023-10-10 17:55:15,689][123614] Updated weights for policy 1, policy_version 34370 (0.0010) [2023-10-10 17:55:15,911][123582] Updated weights for policy 0, policy_version 34463 (0.0007) [2023-10-10 17:55:16,057][123614] Updated weights for policy 1, policy_version 34380 (0.0008) [2023-10-10 17:55:16,424][123614] Updated weights for policy 1, policy_version 34390 (0.0007) [2023-10-10 17:55:16,779][123614] Updated weights for policy 1, policy_version 34400 (0.0011) [2023-10-10 17:55:18,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 70516736. Throughput: 0: 1814.3, 1: 1809.6. Samples: 17637910. Policy #0 lag: (min: 20.0, avg: 20.6, max: 37.0) [2023-10-10 17:55:18,789][122664] Avg episode reward: [(0, '46.710'), (1, '47.050')] [2023-10-10 17:55:19,515][123582] Updated weights for policy 0, policy_version 34473 (0.0008) [2023-10-10 17:55:19,883][123582] Updated weights for policy 0, policy_version 34483 (0.0008) [2023-10-10 17:55:20,266][123582] Updated weights for policy 0, policy_version 34493 (0.0009) [2023-10-10 17:55:20,581][123614] Updated weights for policy 1, policy_version 34410 (0.0007) [2023-10-10 17:55:20,939][123614] Updated weights for policy 1, policy_version 34420 (0.0010) [2023-10-10 17:55:21,310][123614] Updated weights for policy 1, policy_version 34430 (0.0010) [2023-10-10 17:55:23,788][122664] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 70582272. Throughput: 0: 1815.1, 1: 1812.9. Samples: 17660692. Policy #0 lag: (min: 20.0, avg: 20.6, max: 37.0) [2023-10-10 17:55:23,789][122664] Avg episode reward: [(0, '45.480'), (1, '45.930')] [2023-10-10 17:55:24,084][123582] Updated weights for policy 0, policy_version 34503 (0.0009) [2023-10-10 17:55:24,456][123582] Updated weights for policy 0, policy_version 34513 (0.0009) [2023-10-10 17:55:24,829][123582] Updated weights for policy 0, policy_version 34523 (0.0010) [2023-10-10 17:55:25,035][123614] Updated weights for policy 1, policy_version 34440 (0.0009) [2023-10-10 17:55:25,405][123614] Updated weights for policy 1, policy_version 34450 (0.0009) [2023-10-10 17:55:25,774][123614] Updated weights for policy 1, policy_version 34460 (0.0010) [2023-10-10 17:55:28,470][123582] Updated weights for policy 0, policy_version 34533 (0.0008) [2023-10-10 17:55:28,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 70647808. Throughput: 0: 1817.4, 1: 1809.0. Samples: 17670706. Policy #0 lag: (min: 20.0, avg: 20.6, max: 37.0) [2023-10-10 17:55:28,789][122664] Avg episode reward: [(0, '50.110'), (1, '44.970')] [2023-10-10 17:55:28,849][123582] Updated weights for policy 0, policy_version 34543 (0.0007) [2023-10-10 17:55:29,213][123582] Updated weights for policy 0, policy_version 34553 (0.0007) [2023-10-10 17:55:29,453][123614] Updated weights for policy 1, policy_version 34470 (0.0009) [2023-10-10 17:55:29,814][123614] Updated weights for policy 1, policy_version 34480 (0.0010) [2023-10-10 17:55:30,189][123614] Updated weights for policy 1, policy_version 34490 (0.0010) [2023-10-10 17:55:32,806][123582] Updated weights for policy 0, policy_version 34563 (0.0008) [2023-10-10 17:55:33,182][123582] Updated weights for policy 0, policy_version 34573 (0.0010) [2023-10-10 17:55:33,555][123582] Updated weights for policy 0, policy_version 34583 (0.0009) [2023-10-10 17:55:33,788][122664] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 70713344. Throughput: 0: 1821.8, 1: 1811.2. Samples: 17693982. Policy #0 lag: (min: 20.0, avg: 20.6, max: 37.0) [2023-10-10 17:55:33,788][122664] Avg episode reward: [(0, '48.100'), (1, '46.170')] [2023-10-10 17:55:33,970][123614] Updated weights for policy 1, policy_version 34500 (0.0008) [2023-10-10 17:55:34,331][123614] Updated weights for policy 1, policy_version 34510 (0.0008) [2023-10-10 17:55:34,704][123614] Updated weights for policy 1, policy_version 34520 (0.0010) [2023-10-10 17:55:37,196][123582] Updated weights for policy 0, policy_version 34593 (0.0010) [2023-10-10 17:55:37,571][123582] Updated weights for policy 0, policy_version 34603 (0.0010) [2023-10-10 17:55:37,940][123582] Updated weights for policy 0, policy_version 34613 (0.0008) [2023-10-10 17:55:38,313][123582] Updated weights for policy 0, policy_version 34623 (0.0007) [2023-10-10 17:55:38,429][123614] Updated weights for policy 1, policy_version 34530 (0.0008) [2023-10-10 17:55:38,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 70811648. Throughput: 0: 1817.8, 1: 1821.9. Samples: 17714732. Policy #0 lag: (min: 20.0, avg: 20.6, max: 37.0) [2023-10-10 17:55:38,788][122664] Avg episode reward: [(0, '48.140'), (1, '42.400')] [2023-10-10 17:55:38,790][123614] Updated weights for policy 1, policy_version 34540 (0.0007) [2023-10-10 17:55:39,147][123614] Updated weights for policy 1, policy_version 34550 (0.0010) [2023-10-10 17:55:39,511][123614] Updated weights for policy 1, policy_version 34560 (0.0011) [2023-10-10 17:55:41,813][123582] Updated weights for policy 0, policy_version 34633 (0.0009) [2023-10-10 17:55:42,190][123582] Updated weights for policy 0, policy_version 34643 (0.0008) [2023-10-10 17:55:42,561][123582] Updated weights for policy 0, policy_version 34653 (0.0008) [2023-10-10 17:55:43,208][123614] Updated weights for policy 1, policy_version 34570 (0.0010) [2023-10-10 17:55:43,570][123614] Updated weights for policy 1, policy_version 34580 (0.0009) [2023-10-10 17:55:43,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 70877184. Throughput: 0: 1825.0, 1: 1811.4. Samples: 17726870. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-10 17:55:43,789][122664] Avg episode reward: [(0, '49.290'), (1, '42.760')] [2023-10-10 17:55:43,938][123614] Updated weights for policy 1, policy_version 34590 (0.0008) [2023-10-10 17:55:46,327][123582] Updated weights for policy 0, policy_version 34663 (0.0010) [2023-10-10 17:55:46,695][123582] Updated weights for policy 0, policy_version 34673 (0.0010) [2023-10-10 17:55:47,064][123582] Updated weights for policy 0, policy_version 34683 (0.0009) [2023-10-10 17:55:47,692][123614] Updated weights for policy 1, policy_version 34600 (0.0008) [2023-10-10 17:55:48,054][123614] Updated weights for policy 1, policy_version 34610 (0.0009) [2023-10-10 17:55:48,428][123614] Updated weights for policy 1, policy_version 34620 (0.0008) [2023-10-10 17:55:48,788][122664] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 70975488. Throughput: 0: 1823.5, 1: 1821.0. Samples: 17747754. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-10 17:55:48,789][122664] Avg episode reward: [(0, '49.060'), (1, '43.220')] [2023-10-10 17:55:50,790][123582] Updated weights for policy 0, policy_version 34693 (0.0010) [2023-10-10 17:55:51,170][123582] Updated weights for policy 0, policy_version 34703 (0.0007) [2023-10-10 17:55:51,547][123582] Updated weights for policy 0, policy_version 34713 (0.0007) [2023-10-10 17:55:52,290][123614] Updated weights for policy 1, policy_version 34630 (0.0008) [2023-10-10 17:55:52,664][123614] Updated weights for policy 1, policy_version 34640 (0.0008) [2023-10-10 17:55:53,039][123614] Updated weights for policy 1, policy_version 34650 (0.0007) [2023-10-10 17:55:53,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 71041024. Throughput: 0: 1819.4, 1: 1803.2. Samples: 17768962. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-10 17:55:53,789][122664] Avg episode reward: [(0, '50.760'), (1, '40.830')] [2023-10-10 17:55:53,799][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000034656_35487744.pth... [2023-10-10 17:55:53,799][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000034720_35553280.pth... [2023-10-10 17:55:53,835][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000033024_33816576.pth [2023-10-10 17:55:53,839][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000032960_33751040.pth [2023-10-10 17:55:55,452][123582] Updated weights for policy 0, policy_version 34723 (0.0007) [2023-10-10 17:55:55,836][123582] Updated weights for policy 0, policy_version 34733 (0.0008) [2023-10-10 17:55:56,206][123582] Updated weights for policy 0, policy_version 34743 (0.0007) [2023-10-10 17:55:56,732][123614] Updated weights for policy 1, policy_version 34660 (0.0007) [2023-10-10 17:55:57,122][123614] Updated weights for policy 1, policy_version 34670 (0.0008) [2023-10-10 17:55:57,499][123614] Updated weights for policy 1, policy_version 34680 (0.0009) [2023-10-10 17:55:58,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 71106560. Throughput: 0: 1828.1, 1: 1817.9. Samples: 17780320. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-10 17:55:58,789][122664] Avg episode reward: [(0, '49.990'), (1, '37.090')] [2023-10-10 17:55:59,860][123582] Updated weights for policy 0, policy_version 34753 (0.0008) [2023-10-10 17:56:00,225][123582] Updated weights for policy 0, policy_version 34763 (0.0008) [2023-10-10 17:56:00,597][123582] Updated weights for policy 0, policy_version 34773 (0.0007) [2023-10-10 17:56:00,964][123582] Updated weights for policy 0, policy_version 34783 (0.0010) [2023-10-10 17:56:01,131][123614] Updated weights for policy 1, policy_version 34690 (0.0009) [2023-10-10 17:56:01,493][123614] Updated weights for policy 1, policy_version 34700 (0.0007) [2023-10-10 17:56:01,866][123614] Updated weights for policy 1, policy_version 34710 (0.0008) [2023-10-10 17:56:02,231][123614] Updated weights for policy 1, policy_version 34720 (0.0010) [2023-10-10 17:56:03,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 71172096. Throughput: 0: 1829.2, 1: 1806.1. Samples: 17801500. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-10 17:56:03,789][122664] Avg episode reward: [(0, '54.040'), (1, '39.820')] [2023-10-10 17:56:04,649][123582] Updated weights for policy 0, policy_version 34793 (0.0009) [2023-10-10 17:56:05,016][123582] Updated weights for policy 0, policy_version 34803 (0.0009) [2023-10-10 17:56:05,392][123582] Updated weights for policy 0, policy_version 34813 (0.0007) [2023-10-10 17:56:06,017][123614] Updated weights for policy 1, policy_version 34730 (0.0007) [2023-10-10 17:56:06,390][123614] Updated weights for policy 1, policy_version 34740 (0.0007) [2023-10-10 17:56:06,765][123614] Updated weights for policy 1, policy_version 34750 (0.0007) [2023-10-10 17:56:08,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 71237632. Throughput: 0: 1833.6, 1: 1805.3. Samples: 17824444. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-10 17:56:08,788][122664] Avg episode reward: [(0, '51.040'), (1, '38.150')] [2023-10-10 17:56:09,033][123582] Updated weights for policy 0, policy_version 34823 (0.0007) [2023-10-10 17:56:09,398][123582] Updated weights for policy 0, policy_version 34833 (0.0010) [2023-10-10 17:56:09,772][123582] Updated weights for policy 0, policy_version 34843 (0.0011) [2023-10-10 17:56:10,393][123614] Updated weights for policy 1, policy_version 34760 (0.0007) [2023-10-10 17:56:10,756][123614] Updated weights for policy 1, policy_version 34770 (0.0008) [2023-10-10 17:56:11,133][123614] Updated weights for policy 1, policy_version 34780 (0.0009) [2023-10-10 17:56:13,418][123582] Updated weights for policy 0, policy_version 34853 (0.0008) [2023-10-10 17:56:13,786][123582] Updated weights for policy 0, policy_version 34863 (0.0008) [2023-10-10 17:56:13,788][122664] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 71303168. Throughput: 0: 1828.1, 1: 1808.7. Samples: 17834360. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-10 17:56:13,789][122664] Avg episode reward: [(0, '51.500'), (1, '41.320')] [2023-10-10 17:56:14,155][123582] Updated weights for policy 0, policy_version 34873 (0.0007) [2023-10-10 17:56:14,641][123614] Updated weights for policy 1, policy_version 34790 (0.0010) [2023-10-10 17:56:15,009][123614] Updated weights for policy 1, policy_version 34800 (0.0011) [2023-10-10 17:56:15,378][123614] Updated weights for policy 1, policy_version 34810 (0.0009) [2023-10-10 17:56:17,895][123582] Updated weights for policy 0, policy_version 34883 (0.0010) [2023-10-10 17:56:18,269][123582] Updated weights for policy 0, policy_version 34893 (0.0008) [2023-10-10 17:56:18,645][123582] Updated weights for policy 0, policy_version 34903 (0.0007) [2023-10-10 17:56:18,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 71368704. Throughput: 0: 1818.9, 1: 1811.3. Samples: 17857342. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-10 17:56:18,789][122664] Avg episode reward: [(0, '52.940'), (1, '39.500')] [2023-10-10 17:56:19,072][123614] Updated weights for policy 1, policy_version 34820 (0.0009) [2023-10-10 17:56:19,436][123614] Updated weights for policy 1, policy_version 34830 (0.0007) [2023-10-10 17:56:19,816][123614] Updated weights for policy 1, policy_version 34840 (0.0009) [2023-10-10 17:56:22,337][123582] Updated weights for policy 0, policy_version 34913 (0.0009) [2023-10-10 17:56:22,711][123582] Updated weights for policy 0, policy_version 34923 (0.0009) [2023-10-10 17:56:23,087][123582] Updated weights for policy 0, policy_version 34933 (0.0008) [2023-10-10 17:56:23,447][123582] Updated weights for policy 0, policy_version 34943 (0.0007) [2023-10-10 17:56:23,545][123614] Updated weights for policy 1, policy_version 34850 (0.0010) [2023-10-10 17:56:23,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 71467008. Throughput: 0: 1822.8, 1: 1817.8. Samples: 17878560. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-10 17:56:23,788][122664] Avg episode reward: [(0, '46.940'), (1, '39.430')] [2023-10-10 17:56:23,910][123614] Updated weights for policy 1, policy_version 34860 (0.0011) [2023-10-10 17:56:24,275][123614] Updated weights for policy 1, policy_version 34870 (0.0010) [2023-10-10 17:56:24,642][123614] Updated weights for policy 1, policy_version 34880 (0.0008) [2023-10-10 17:56:27,078][123582] Updated weights for policy 0, policy_version 34953 (0.0008) [2023-10-10 17:56:27,455][123582] Updated weights for policy 0, policy_version 34963 (0.0008) [2023-10-10 17:56:27,823][123582] Updated weights for policy 0, policy_version 34973 (0.0008) [2023-10-10 17:56:28,389][123614] Updated weights for policy 1, policy_version 34890 (0.0007) [2023-10-10 17:56:28,758][123614] Updated weights for policy 1, policy_version 34900 (0.0008) [2023-10-10 17:56:28,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 71532544. Throughput: 0: 1807.5, 1: 1815.5. Samples: 17889906. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-10 17:56:28,789][122664] Avg episode reward: [(0, '44.910'), (1, '40.450')] [2023-10-10 17:56:29,134][123614] Updated weights for policy 1, policy_version 34910 (0.0007) [2023-10-10 17:56:31,351][123582] Updated weights for policy 0, policy_version 34983 (0.0008) [2023-10-10 17:56:31,715][123582] Updated weights for policy 0, policy_version 34993 (0.0007) [2023-10-10 17:56:32,080][123582] Updated weights for policy 0, policy_version 35003 (0.0008) [2023-10-10 17:56:32,764][123614] Updated weights for policy 1, policy_version 34920 (0.0008) [2023-10-10 17:56:33,135][123614] Updated weights for policy 1, policy_version 34930 (0.0009) [2023-10-10 17:56:33,514][123614] Updated weights for policy 1, policy_version 34940 (0.0009) [2023-10-10 17:56:33,788][122664] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 71630848. Throughput: 0: 1810.6, 1: 1820.0. Samples: 17911130. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-10 17:56:33,788][122664] Avg episode reward: [(0, '44.640'), (1, '40.060')] [2023-10-10 17:56:35,858][123582] Updated weights for policy 0, policy_version 35013 (0.0009) [2023-10-10 17:56:36,245][123582] Updated weights for policy 0, policy_version 35023 (0.0012) [2023-10-10 17:56:36,611][123582] Updated weights for policy 0, policy_version 35033 (0.0008) [2023-10-10 17:56:37,180][123614] Updated weights for policy 1, policy_version 34950 (0.0007) [2023-10-10 17:56:37,549][123614] Updated weights for policy 1, policy_version 34960 (0.0009) [2023-10-10 17:56:37,914][123614] Updated weights for policy 1, policy_version 34970 (0.0007) [2023-10-10 17:56:38,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 71696384. Throughput: 0: 1812.7, 1: 1832.2. Samples: 17932984. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-10 17:56:38,789][122664] Avg episode reward: [(0, '46.560'), (1, '42.770')] [2023-10-10 17:56:40,434][123582] Updated weights for policy 0, policy_version 35043 (0.0009) [2023-10-10 17:56:40,828][123582] Updated weights for policy 0, policy_version 35053 (0.0009) [2023-10-10 17:56:41,193][123582] Updated weights for policy 0, policy_version 35063 (0.0008) [2023-10-10 17:56:41,534][123614] Updated weights for policy 1, policy_version 34980 (0.0007) [2023-10-10 17:56:41,896][123614] Updated weights for policy 1, policy_version 34990 (0.0009) [2023-10-10 17:56:42,270][123614] Updated weights for policy 1, policy_version 35000 (0.0008) [2023-10-10 17:56:43,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 71761920. Throughput: 0: 1815.2, 1: 1827.6. Samples: 17944244. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-10 17:56:43,789][122664] Avg episode reward: [(0, '45.790'), (1, '42.190')] [2023-10-10 17:56:44,802][123582] Updated weights for policy 0, policy_version 35073 (0.0008) [2023-10-10 17:56:45,172][123582] Updated weights for policy 0, policy_version 35083 (0.0010) [2023-10-10 17:56:45,546][123582] Updated weights for policy 0, policy_version 35093 (0.0009) [2023-10-10 17:56:45,903][123582] Updated weights for policy 0, policy_version 35103 (0.0010) [2023-10-10 17:56:46,055][123614] Updated weights for policy 1, policy_version 35010 (0.0009) [2023-10-10 17:56:46,456][123614] Updated weights for policy 1, policy_version 35020 (0.0010) [2023-10-10 17:56:46,826][123614] Updated weights for policy 1, policy_version 35030 (0.0007) [2023-10-10 17:56:47,194][123614] Updated weights for policy 1, policy_version 35040 (0.0008) [2023-10-10 17:56:48,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 71827456. Throughput: 0: 1817.3, 1: 1830.6. Samples: 17965654. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-10 17:56:48,789][122664] Avg episode reward: [(0, '45.470'), (1, '39.480')] [2023-10-10 17:56:49,678][123582] Updated weights for policy 0, policy_version 35113 (0.0011) [2023-10-10 17:56:50,044][123582] Updated weights for policy 0, policy_version 35123 (0.0010) [2023-10-10 17:56:50,420][123582] Updated weights for policy 0, policy_version 35133 (0.0008) [2023-10-10 17:56:50,883][123614] Updated weights for policy 1, policy_version 35050 (0.0010) [2023-10-10 17:56:51,259][123614] Updated weights for policy 1, policy_version 35060 (0.0007) [2023-10-10 17:56:51,641][123614] Updated weights for policy 1, policy_version 35070 (0.0008) [2023-10-10 17:56:53,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 71892992. Throughput: 0: 1807.8, 1: 1836.1. Samples: 17988420. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-10 17:56:53,788][122664] Avg episode reward: [(0, '45.160'), (1, '40.040')] [2023-10-10 17:56:54,135][123582] Updated weights for policy 0, policy_version 35143 (0.0009) [2023-10-10 17:56:54,502][123582] Updated weights for policy 0, policy_version 35153 (0.0008) [2023-10-10 17:56:54,877][123582] Updated weights for policy 0, policy_version 35163 (0.0007) [2023-10-10 17:56:55,101][123614] Updated weights for policy 1, policy_version 35080 (0.0007) [2023-10-10 17:56:55,476][123614] Updated weights for policy 1, policy_version 35090 (0.0011) [2023-10-10 17:56:55,837][123614] Updated weights for policy 1, policy_version 35100 (0.0007) [2023-10-10 17:56:58,720][123582] Updated weights for policy 0, policy_version 35173 (0.0008) [2023-10-10 17:56:58,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 71958528. Throughput: 0: 1808.2, 1: 1836.8. Samples: 17998386. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-10 17:56:58,789][122664] Avg episode reward: [(0, '43.340'), (1, '39.860')] [2023-10-10 17:56:59,092][123582] Updated weights for policy 0, policy_version 35183 (0.0007) [2023-10-10 17:56:59,459][123614] Updated weights for policy 1, policy_version 35110 (0.0009) [2023-10-10 17:56:59,466][123582] Updated weights for policy 0, policy_version 35193 (0.0007) [2023-10-10 17:56:59,824][123614] Updated weights for policy 1, policy_version 35120 (0.0008) [2023-10-10 17:57:00,189][123614] Updated weights for policy 1, policy_version 35130 (0.0007) [2023-10-10 17:57:03,022][123582] Updated weights for policy 0, policy_version 35203 (0.0008) [2023-10-10 17:57:03,389][123582] Updated weights for policy 0, policy_version 35213 (0.0009) [2023-10-10 17:57:03,760][123582] Updated weights for policy 0, policy_version 35223 (0.0010) [2023-10-10 17:57:03,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.6, 300 sec: 14440.1). Total num frames: 72024064. Throughput: 0: 1812.9, 1: 1831.2. Samples: 18021330. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-10 17:57:03,788][122664] Avg episode reward: [(0, '45.150'), (1, '41.870')] [2023-10-10 17:57:03,861][123614] Updated weights for policy 1, policy_version 35140 (0.0008) [2023-10-10 17:57:04,225][123614] Updated weights for policy 1, policy_version 35150 (0.0008) [2023-10-10 17:57:04,590][123614] Updated weights for policy 1, policy_version 35160 (0.0010) [2023-10-10 17:57:07,309][123582] Updated weights for policy 0, policy_version 35233 (0.0007) [2023-10-10 17:57:07,680][123582] Updated weights for policy 0, policy_version 35243 (0.0007) [2023-10-10 17:57:08,059][123582] Updated weights for policy 0, policy_version 35253 (0.0011) [2023-10-10 17:57:08,270][123614] Updated weights for policy 1, policy_version 35170 (0.0007) [2023-10-10 17:57:08,429][123582] Updated weights for policy 0, policy_version 35263 (0.0009) [2023-10-10 17:57:08,643][123614] Updated weights for policy 1, policy_version 35180 (0.0008) [2023-10-10 17:57:08,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 72122368. Throughput: 0: 1821.4, 1: 1820.2. Samples: 18042430. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-10 17:57:08,789][122664] Avg episode reward: [(0, '47.500'), (1, '42.110')] [2023-10-10 17:57:09,008][123614] Updated weights for policy 1, policy_version 35190 (0.0009) [2023-10-10 17:57:09,372][123614] Updated weights for policy 1, policy_version 35200 (0.0009) [2023-10-10 17:57:12,136][123582] Updated weights for policy 0, policy_version 35273 (0.0012) [2023-10-10 17:57:12,511][123582] Updated weights for policy 0, policy_version 35283 (0.0008) [2023-10-10 17:57:12,878][123582] Updated weights for policy 0, policy_version 35293 (0.0009) [2023-10-10 17:57:13,044][123614] Updated weights for policy 1, policy_version 35210 (0.0008) [2023-10-10 17:57:13,415][123614] Updated weights for policy 1, policy_version 35220 (0.0009) [2023-10-10 17:57:13,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 72187904. Throughput: 0: 1821.8, 1: 1828.5. Samples: 18054170. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-10 17:57:13,789][122664] Avg episode reward: [(0, '47.090'), (1, '41.060')] [2023-10-10 17:57:13,795][123614] Updated weights for policy 1, policy_version 35230 (0.0009) [2023-10-10 17:57:16,627][123582] Updated weights for policy 0, policy_version 35303 (0.0007) [2023-10-10 17:57:17,008][123582] Updated weights for policy 0, policy_version 35313 (0.0007) [2023-10-10 17:57:17,387][123582] Updated weights for policy 0, policy_version 35323 (0.0009) [2023-10-10 17:57:17,525][123614] Updated weights for policy 1, policy_version 35240 (0.0007) [2023-10-10 17:57:17,897][123614] Updated weights for policy 1, policy_version 35250 (0.0009) [2023-10-10 17:57:18,273][123614] Updated weights for policy 1, policy_version 35260 (0.0009) [2023-10-10 17:57:18,788][122664] Fps is (10 sec: 16384.4, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 72286208. Throughput: 0: 1824.0, 1: 1820.5. Samples: 18075132. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-10 17:57:18,788][122664] Avg episode reward: [(0, '47.620'), (1, '41.240')] [2023-10-10 17:57:21,188][123582] Updated weights for policy 0, policy_version 35333 (0.0008) [2023-10-10 17:57:21,545][123582] Updated weights for policy 0, policy_version 35343 (0.0010) [2023-10-10 17:57:21,925][123582] Updated weights for policy 0, policy_version 35353 (0.0009) [2023-10-10 17:57:21,999][123614] Updated weights for policy 1, policy_version 35270 (0.0007) [2023-10-10 17:57:22,369][123614] Updated weights for policy 1, policy_version 35280 (0.0009) [2023-10-10 17:57:22,745][123614] Updated weights for policy 1, policy_version 35290 (0.0008) [2023-10-10 17:57:23,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 72351744. Throughput: 0: 1819.6, 1: 1821.8. Samples: 18096848. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-10 17:57:23,789][122664] Avg episode reward: [(0, '49.300'), (1, '39.540')] [2023-10-10 17:57:25,563][123582] Updated weights for policy 0, policy_version 35363 (0.0008) [2023-10-10 17:57:25,963][123582] Updated weights for policy 0, policy_version 35373 (0.0009) [2023-10-10 17:57:26,336][123582] Updated weights for policy 0, policy_version 35383 (0.0009) [2023-10-10 17:57:26,508][123614] Updated weights for policy 1, policy_version 35300 (0.0008) [2023-10-10 17:57:26,891][123614] Updated weights for policy 1, policy_version 35310 (0.0008) [2023-10-10 17:57:27,251][123614] Updated weights for policy 1, policy_version 35320 (0.0007) [2023-10-10 17:57:28,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 72417280. Throughput: 0: 1822.4, 1: 1817.1. Samples: 18108024. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-10 17:57:28,789][122664] Avg episode reward: [(0, '52.780'), (1, '42.580')] [2023-10-10 17:57:29,994][123582] Updated weights for policy 0, policy_version 35393 (0.0007) [2023-10-10 17:57:30,370][123582] Updated weights for policy 0, policy_version 35403 (0.0008) [2023-10-10 17:57:30,748][123582] Updated weights for policy 0, policy_version 35413 (0.0007) [2023-10-10 17:57:30,863][123614] Updated weights for policy 1, policy_version 35330 (0.0007) [2023-10-10 17:57:31,116][123582] Updated weights for policy 0, policy_version 35423 (0.0007) [2023-10-10 17:57:31,235][123614] Updated weights for policy 1, policy_version 35340 (0.0007) [2023-10-10 17:57:31,607][123614] Updated weights for policy 1, policy_version 35350 (0.0008) [2023-10-10 17:57:31,973][123614] Updated weights for policy 1, policy_version 35360 (0.0007) [2023-10-10 17:57:33,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 72482816. Throughput: 0: 1815.4, 1: 1828.7. Samples: 18129638. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-10 17:57:33,788][122664] Avg episode reward: [(0, '54.870'), (1, '41.130')] [2023-10-10 17:57:34,864][123582] Updated weights for policy 0, policy_version 35433 (0.0010) [2023-10-10 17:57:35,223][123582] Updated weights for policy 0, policy_version 35443 (0.0010) [2023-10-10 17:57:35,586][123582] Updated weights for policy 0, policy_version 35453 (0.0010) [2023-10-10 17:57:35,879][123614] Updated weights for policy 1, policy_version 35370 (0.0009) [2023-10-10 17:57:36,254][123614] Updated weights for policy 1, policy_version 35380 (0.0010) [2023-10-10 17:57:36,634][123614] Updated weights for policy 1, policy_version 35390 (0.0009) [2023-10-10 17:57:38,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 72548352. Throughput: 0: 1820.3, 1: 1824.5. Samples: 18152434. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-10 17:57:38,789][122664] Avg episode reward: [(0, '56.220'), (1, '40.080')] [2023-10-10 17:57:39,140][123582] Updated weights for policy 0, policy_version 35463 (0.0008) [2023-10-10 17:57:39,512][123582] Updated weights for policy 0, policy_version 35473 (0.0009) [2023-10-10 17:57:39,888][123582] Updated weights for policy 0, policy_version 35483 (0.0009) [2023-10-10 17:57:40,209][123614] Updated weights for policy 1, policy_version 35400 (0.0007) [2023-10-10 17:57:40,583][123614] Updated weights for policy 1, policy_version 35410 (0.0007) [2023-10-10 17:57:40,943][123614] Updated weights for policy 1, policy_version 35420 (0.0007) [2023-10-10 17:57:43,511][123582] Updated weights for policy 0, policy_version 35493 (0.0009) [2023-10-10 17:57:43,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 72613888. Throughput: 0: 1819.4, 1: 1824.7. Samples: 18162370. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-10 17:57:43,789][122664] Avg episode reward: [(0, '52.860'), (1, '38.550')] [2023-10-10 17:57:43,876][123582] Updated weights for policy 0, policy_version 35503 (0.0009) [2023-10-10 17:57:44,241][123582] Updated weights for policy 0, policy_version 35513 (0.0009) [2023-10-10 17:57:44,542][123614] Updated weights for policy 1, policy_version 35430 (0.0008) [2023-10-10 17:57:44,918][123614] Updated weights for policy 1, policy_version 35440 (0.0011) [2023-10-10 17:57:45,282][123614] Updated weights for policy 1, policy_version 35450 (0.0009) [2023-10-10 17:57:47,979][123582] Updated weights for policy 0, policy_version 35523 (0.0008) [2023-10-10 17:57:48,351][123582] Updated weights for policy 0, policy_version 35533 (0.0008) [2023-10-10 17:57:48,717][123582] Updated weights for policy 0, policy_version 35543 (0.0009) [2023-10-10 17:57:48,788][122664] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 72679424. Throughput: 0: 1819.3, 1: 1818.2. Samples: 18185018. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-10 17:57:48,788][122664] Avg episode reward: [(0, '50.650'), (1, '39.910')] [2023-10-10 17:57:49,000][123614] Updated weights for policy 1, policy_version 35460 (0.0009) [2023-10-10 17:57:49,371][123614] Updated weights for policy 1, policy_version 35470 (0.0009) [2023-10-10 17:57:49,740][123614] Updated weights for policy 1, policy_version 35480 (0.0008) [2023-10-10 17:57:52,473][123582] Updated weights for policy 0, policy_version 35553 (0.0009) [2023-10-10 17:57:52,848][123582] Updated weights for policy 0, policy_version 35563 (0.0010) [2023-10-10 17:57:53,229][123582] Updated weights for policy 0, policy_version 35573 (0.0007) [2023-10-10 17:57:53,436][123614] Updated weights for policy 1, policy_version 35490 (0.0007) [2023-10-10 17:57:53,604][123582] Updated weights for policy 0, policy_version 35583 (0.0007) [2023-10-10 17:57:53,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 72777728. Throughput: 0: 1811.9, 1: 1821.7. Samples: 18205942. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-10 17:57:53,788][122664] Avg episode reward: [(0, '48.530'), (1, '41.310')] [2023-10-10 17:57:53,796][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000035584_36438016.pth... [2023-10-10 17:57:53,812][123614] Updated weights for policy 1, policy_version 35500 (0.0007) [2023-10-10 17:57:53,834][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000033888_34701312.pth [2023-10-10 17:57:54,184][123614] Updated weights for policy 1, policy_version 35510 (0.0008) [2023-10-10 17:57:54,549][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000035520_36372480.pth... [2023-10-10 17:57:54,549][123614] Updated weights for policy 1, policy_version 35520 (0.0008) [2023-10-10 17:57:54,587][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000033792_34603008.pth [2023-10-10 17:57:57,476][123582] Updated weights for policy 0, policy_version 35593 (0.0010) [2023-10-10 17:57:57,842][123582] Updated weights for policy 0, policy_version 35603 (0.0008) [2023-10-10 17:57:57,982][123614] Updated weights for policy 1, policy_version 35530 (0.0009) [2023-10-10 17:57:58,214][123582] Updated weights for policy 0, policy_version 35613 (0.0008) [2023-10-10 17:57:58,355][123614] Updated weights for policy 1, policy_version 35540 (0.0008) [2023-10-10 17:57:58,727][123614] Updated weights for policy 1, policy_version 35550 (0.0009) [2023-10-10 17:57:58,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 72843264. Throughput: 0: 1805.7, 1: 1820.8. Samples: 18217358. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:57:58,788][122664] Avg episode reward: [(0, '46.360'), (1, '43.340')] [2023-10-10 17:58:02,039][123582] Updated weights for policy 0, policy_version 35623 (0.0008) [2023-10-10 17:58:02,415][123582] Updated weights for policy 0, policy_version 35633 (0.0008) [2023-10-10 17:58:02,547][123614] Updated weights for policy 1, policy_version 35560 (0.0007) [2023-10-10 17:58:02,787][123582] Updated weights for policy 0, policy_version 35643 (0.0008) [2023-10-10 17:58:02,921][123614] Updated weights for policy 1, policy_version 35570 (0.0008) [2023-10-10 17:58:03,285][123614] Updated weights for policy 1, policy_version 35580 (0.0007) [2023-10-10 17:58:03,788][122664] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 72941568. Throughput: 0: 1809.8, 1: 1815.0. Samples: 18238248. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:58:03,789][122664] Avg episode reward: [(0, '50.170'), (1, '45.820')] [2023-10-10 17:58:06,461][123582] Updated weights for policy 0, policy_version 35653 (0.0008) [2023-10-10 17:58:06,838][123582] Updated weights for policy 0, policy_version 35663 (0.0007) [2023-10-10 17:58:06,975][123614] Updated weights for policy 1, policy_version 35590 (0.0008) [2023-10-10 17:58:07,210][123582] Updated weights for policy 0, policy_version 35673 (0.0007) [2023-10-10 17:58:07,342][123614] Updated weights for policy 1, policy_version 35600 (0.0008) [2023-10-10 17:58:07,707][123614] Updated weights for policy 1, policy_version 35610 (0.0007) [2023-10-10 17:58:08,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 73007104. Throughput: 0: 1805.0, 1: 1808.7. Samples: 18259464. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:58:08,789][122664] Avg episode reward: [(0, '50.160'), (1, '47.570')] [2023-10-10 17:58:10,907][123582] Updated weights for policy 0, policy_version 35683 (0.0008) [2023-10-10 17:58:11,312][123582] Updated weights for policy 0, policy_version 35693 (0.0009) [2023-10-10 17:58:11,515][123614] Updated weights for policy 1, policy_version 35620 (0.0008) [2023-10-10 17:58:11,687][123582] Updated weights for policy 0, policy_version 35703 (0.0008) [2023-10-10 17:58:11,880][123614] Updated weights for policy 1, policy_version 35630 (0.0009) [2023-10-10 17:58:12,244][123614] Updated weights for policy 1, policy_version 35640 (0.0008) [2023-10-10 17:58:13,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 73072640. Throughput: 0: 1810.3, 1: 1810.5. Samples: 18270960. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:58:13,789][122664] Avg episode reward: [(0, '49.480'), (1, '47.290')] [2023-10-10 17:58:15,302][123582] Updated weights for policy 0, policy_version 35713 (0.0008) [2023-10-10 17:58:15,677][123582] Updated weights for policy 0, policy_version 35723 (0.0007) [2023-10-10 17:58:15,991][123614] Updated weights for policy 1, policy_version 35650 (0.0008) [2023-10-10 17:58:16,050][123582] Updated weights for policy 0, policy_version 35733 (0.0007) [2023-10-10 17:58:16,351][123614] Updated weights for policy 1, policy_version 35660 (0.0009) [2023-10-10 17:58:16,423][123582] Updated weights for policy 0, policy_version 35743 (0.0008) [2023-10-10 17:58:16,714][123614] Updated weights for policy 1, policy_version 35670 (0.0010) [2023-10-10 17:58:17,082][123614] Updated weights for policy 1, policy_version 35680 (0.0010) [2023-10-10 17:58:18,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 73138176. Throughput: 0: 1801.8, 1: 1803.2. Samples: 18291864. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:58:18,789][122664] Avg episode reward: [(0, '46.860'), (1, '45.450')] [2023-10-10 17:58:20,109][123582] Updated weights for policy 0, policy_version 35753 (0.0008) [2023-10-10 17:58:20,479][123582] Updated weights for policy 0, policy_version 35763 (0.0009) [2023-10-10 17:58:20,849][123582] Updated weights for policy 0, policy_version 35773 (0.0008) [2023-10-10 17:58:20,983][123614] Updated weights for policy 1, policy_version 35690 (0.0008) [2023-10-10 17:58:21,351][123614] Updated weights for policy 1, policy_version 35700 (0.0008) [2023-10-10 17:58:21,717][123614] Updated weights for policy 1, policy_version 35710 (0.0008) [2023-10-10 17:58:23,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 73203712. Throughput: 0: 1801.7, 1: 1801.3. Samples: 18314570. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:58:23,789][122664] Avg episode reward: [(0, '42.430'), (1, '44.230')] [2023-10-10 17:58:24,512][123582] Updated weights for policy 0, policy_version 35783 (0.0008) [2023-10-10 17:58:24,882][123582] Updated weights for policy 0, policy_version 35793 (0.0009) [2023-10-10 17:58:25,255][123582] Updated weights for policy 0, policy_version 35803 (0.0008) [2023-10-10 17:58:25,264][123614] Updated weights for policy 1, policy_version 35720 (0.0008) [2023-10-10 17:58:25,629][123614] Updated weights for policy 1, policy_version 35730 (0.0011) [2023-10-10 17:58:25,999][123614] Updated weights for policy 1, policy_version 35740 (0.0011) [2023-10-10 17:58:28,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 73269248. Throughput: 0: 1804.4, 1: 1797.0. Samples: 18324436. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:58:28,789][122664] Avg episode reward: [(0, '43.990'), (1, '44.440')] [2023-10-10 17:58:28,983][123582] Updated weights for policy 0, policy_version 35813 (0.0009) [2023-10-10 17:58:29,349][123582] Updated weights for policy 0, policy_version 35823 (0.0008) [2023-10-10 17:58:29,728][123582] Updated weights for policy 0, policy_version 35833 (0.0009) [2023-10-10 17:58:29,767][123614] Updated weights for policy 1, policy_version 35750 (0.0008) [2023-10-10 17:58:30,139][123614] Updated weights for policy 1, policy_version 35760 (0.0010) [2023-10-10 17:58:30,513][123614] Updated weights for policy 1, policy_version 35770 (0.0011) [2023-10-10 17:58:33,471][123582] Updated weights for policy 0, policy_version 35843 (0.0009) [2023-10-10 17:58:33,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 73334784. Throughput: 0: 1803.5, 1: 1794.7. Samples: 18346936. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:58:33,788][122664] Avg episode reward: [(0, '44.870'), (1, '45.250')] [2023-10-10 17:58:33,846][123582] Updated weights for policy 0, policy_version 35853 (0.0008) [2023-10-10 17:58:34,223][123582] Updated weights for policy 0, policy_version 35863 (0.0007) [2023-10-10 17:58:34,307][123614] Updated weights for policy 1, policy_version 35780 (0.0010) [2023-10-10 17:58:34,677][123614] Updated weights for policy 1, policy_version 35790 (0.0009) [2023-10-10 17:58:35,058][123614] Updated weights for policy 1, policy_version 35800 (0.0010) [2023-10-10 17:58:37,705][123582] Updated weights for policy 0, policy_version 35873 (0.0007) [2023-10-10 17:58:38,080][123582] Updated weights for policy 0, policy_version 35883 (0.0009) [2023-10-10 17:58:38,457][123582] Updated weights for policy 0, policy_version 35893 (0.0007) [2023-10-10 17:58:38,725][123614] Updated weights for policy 1, policy_version 35810 (0.0010) [2023-10-10 17:58:38,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.6, 300 sec: 14440.1). Total num frames: 73400320. Throughput: 0: 1817.4, 1: 1803.8. Samples: 18368894. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:58:38,788][122664] Avg episode reward: [(0, '43.360'), (1, '45.750')] [2023-10-10 17:58:38,831][123582] Updated weights for policy 0, policy_version 35903 (0.0009) [2023-10-10 17:58:39,097][123614] Updated weights for policy 1, policy_version 35820 (0.0008) [2023-10-10 17:58:39,464][123614] Updated weights for policy 1, policy_version 35830 (0.0010) [2023-10-10 17:58:39,838][123614] Updated weights for policy 1, policy_version 35840 (0.0010) [2023-10-10 17:58:42,639][123582] Updated weights for policy 0, policy_version 35913 (0.0009) [2023-10-10 17:58:43,004][123582] Updated weights for policy 0, policy_version 35923 (0.0009) [2023-10-10 17:58:43,377][123582] Updated weights for policy 0, policy_version 35933 (0.0011) [2023-10-10 17:58:43,586][123614] Updated weights for policy 1, policy_version 35850 (0.0008) [2023-10-10 17:58:43,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 73498624. Throughput: 0: 1818.5, 1: 1792.4. Samples: 18379850. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:58:43,789][122664] Avg episode reward: [(0, '46.820'), (1, '47.530')] [2023-10-10 17:58:43,960][123614] Updated weights for policy 1, policy_version 35860 (0.0007) [2023-10-10 17:58:44,325][123614] Updated weights for policy 1, policy_version 35870 (0.0007) [2023-10-10 17:58:46,998][123582] Updated weights for policy 0, policy_version 35943 (0.0011) [2023-10-10 17:58:47,361][123582] Updated weights for policy 0, policy_version 35953 (0.0010) [2023-10-10 17:58:47,734][123582] Updated weights for policy 0, policy_version 35963 (0.0009) [2023-10-10 17:58:48,097][123614] Updated weights for policy 1, policy_version 35880 (0.0009) [2023-10-10 17:58:48,460][123614] Updated weights for policy 1, policy_version 35890 (0.0007) [2023-10-10 17:58:48,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 73564160. Throughput: 0: 1822.7, 1: 1810.3. Samples: 18401730. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 17:58:48,789][122664] Avg episode reward: [(0, '47.670'), (1, '43.750')] [2023-10-10 17:58:48,827][123614] Updated weights for policy 1, policy_version 35900 (0.0010) [2023-10-10 17:58:51,607][123582] Updated weights for policy 0, policy_version 35973 (0.0008) [2023-10-10 17:58:51,982][123582] Updated weights for policy 0, policy_version 35983 (0.0009) [2023-10-10 17:58:52,366][123582] Updated weights for policy 0, policy_version 35993 (0.0008) [2023-10-10 17:58:52,630][123614] Updated weights for policy 1, policy_version 35910 (0.0010) [2023-10-10 17:58:52,991][123614] Updated weights for policy 1, policy_version 35920 (0.0010) [2023-10-10 17:58:53,369][123614] Updated weights for policy 1, policy_version 35930 (0.0010) [2023-10-10 17:58:53,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 73662464. Throughput: 0: 1818.3, 1: 1795.6. Samples: 18422088. Policy #0 lag: (min: 4.0, avg: 27.8, max: 32.0) [2023-10-10 17:58:53,789][122664] Avg episode reward: [(0, '48.110'), (1, '39.790')] [2023-10-10 17:58:56,041][123582] Updated weights for policy 0, policy_version 36003 (0.0007) [2023-10-10 17:58:56,427][123582] Updated weights for policy 0, policy_version 36013 (0.0009) [2023-10-10 17:58:56,805][123582] Updated weights for policy 0, policy_version 36023 (0.0010) [2023-10-10 17:58:57,132][123614] Updated weights for policy 1, policy_version 35940 (0.0009) [2023-10-10 17:58:57,501][123614] Updated weights for policy 1, policy_version 35950 (0.0010) [2023-10-10 17:58:57,870][123614] Updated weights for policy 1, policy_version 35960 (0.0008) [2023-10-10 17:58:58,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 73728000. Throughput: 0: 1823.6, 1: 1803.6. Samples: 18434180. Policy #0 lag: (min: 4.0, avg: 27.8, max: 32.0) [2023-10-10 17:58:58,789][122664] Avg episode reward: [(0, '44.260'), (1, '40.750')] [2023-10-10 17:59:00,350][123582] Updated weights for policy 0, policy_version 36033 (0.0009) [2023-10-10 17:59:00,720][123582] Updated weights for policy 0, policy_version 36043 (0.0008) [2023-10-10 17:59:01,094][123582] Updated weights for policy 0, policy_version 36053 (0.0009) [2023-10-10 17:59:01,468][123582] Updated weights for policy 0, policy_version 36063 (0.0010) [2023-10-10 17:59:01,742][123614] Updated weights for policy 1, policy_version 35970 (0.0009) [2023-10-10 17:59:02,101][123614] Updated weights for policy 1, policy_version 35980 (0.0007) [2023-10-10 17:59:02,469][123614] Updated weights for policy 1, policy_version 35990 (0.0007) [2023-10-10 17:59:02,836][123614] Updated weights for policy 1, policy_version 36000 (0.0009) [2023-10-10 17:59:03,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 73793536. Throughput: 0: 1818.9, 1: 1801.0. Samples: 18454762. Policy #0 lag: (min: 4.0, avg: 27.8, max: 32.0) [2023-10-10 17:59:03,788][122664] Avg episode reward: [(0, '42.250'), (1, '40.370')] [2023-10-10 17:59:05,065][123582] Updated weights for policy 0, policy_version 36073 (0.0007) [2023-10-10 17:59:05,443][123582] Updated weights for policy 0, policy_version 36083 (0.0007) [2023-10-10 17:59:05,809][123582] Updated weights for policy 0, policy_version 36093 (0.0007) [2023-10-10 17:59:06,515][123614] Updated weights for policy 1, policy_version 36010 (0.0009) [2023-10-10 17:59:06,885][123614] Updated weights for policy 1, policy_version 36020 (0.0007) [2023-10-10 17:59:07,262][123614] Updated weights for policy 1, policy_version 36030 (0.0007) [2023-10-10 17:59:08,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 73859072. Throughput: 0: 1829.2, 1: 1797.6. Samples: 18477778. Policy #0 lag: (min: 4.0, avg: 27.8, max: 32.0) [2023-10-10 17:59:08,789][122664] Avg episode reward: [(0, '48.560'), (1, '42.790')] [2023-10-10 17:59:09,218][123582] Updated weights for policy 0, policy_version 36103 (0.0008) [2023-10-10 17:59:09,592][123582] Updated weights for policy 0, policy_version 36113 (0.0009) [2023-10-10 17:59:09,956][123582] Updated weights for policy 0, policy_version 36123 (0.0011) [2023-10-10 17:59:10,845][123614] Updated weights for policy 1, policy_version 36040 (0.0008) [2023-10-10 17:59:11,212][123614] Updated weights for policy 1, policy_version 36050 (0.0007) [2023-10-10 17:59:11,585][123614] Updated weights for policy 1, policy_version 36060 (0.0009) [2023-10-10 17:59:13,718][123582] Updated weights for policy 0, policy_version 36133 (0.0009) [2023-10-10 17:59:13,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 73924608. Throughput: 0: 1828.5, 1: 1803.3. Samples: 18487864. Policy #0 lag: (min: 4.0, avg: 27.8, max: 32.0) [2023-10-10 17:59:13,788][122664] Avg episode reward: [(0, '47.170'), (1, '43.420')] [2023-10-10 17:59:14,097][123582] Updated weights for policy 0, policy_version 36143 (0.0010) [2023-10-10 17:59:14,477][123582] Updated weights for policy 0, policy_version 36153 (0.0010) [2023-10-10 17:59:15,414][123614] Updated weights for policy 1, policy_version 36070 (0.0009) [2023-10-10 17:59:15,783][123614] Updated weights for policy 1, policy_version 36080 (0.0010) [2023-10-10 17:59:16,147][123614] Updated weights for policy 1, policy_version 36090 (0.0009) [2023-10-10 17:59:18,187][123582] Updated weights for policy 0, policy_version 36163 (0.0010) [2023-10-10 17:59:18,572][123582] Updated weights for policy 0, policy_version 36173 (0.0011) [2023-10-10 17:59:18,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 73990144. Throughput: 0: 1832.3, 1: 1801.6. Samples: 18510460. Policy #0 lag: (min: 4.0, avg: 27.8, max: 32.0) [2023-10-10 17:59:18,789][122664] Avg episode reward: [(0, '49.190'), (1, '43.140')] [2023-10-10 17:59:18,941][123582] Updated weights for policy 0, policy_version 36183 (0.0008) [2023-10-10 17:59:19,868][123614] Updated weights for policy 1, policy_version 36100 (0.0010) [2023-10-10 17:59:20,228][123614] Updated weights for policy 1, policy_version 36110 (0.0009) [2023-10-10 17:59:20,605][123614] Updated weights for policy 1, policy_version 36120 (0.0011) [2023-10-10 17:59:22,672][123582] Updated weights for policy 0, policy_version 36193 (0.0009) [2023-10-10 17:59:23,040][123582] Updated weights for policy 0, policy_version 36203 (0.0010) [2023-10-10 17:59:23,417][123582] Updated weights for policy 0, policy_version 36213 (0.0008) [2023-10-10 17:59:23,783][123582] Updated weights for policy 0, policy_version 36223 (0.0007) [2023-10-10 17:59:23,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 74055680. Throughput: 0: 1825.2, 1: 1800.8. Samples: 18532064. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-10 17:59:23,788][122664] Avg episode reward: [(0, '48.140'), (1, '43.130')] [2023-10-10 17:59:24,309][123614] Updated weights for policy 1, policy_version 36130 (0.0008) [2023-10-10 17:59:24,674][123614] Updated weights for policy 1, policy_version 36140 (0.0010) [2023-10-10 17:59:25,054][123614] Updated weights for policy 1, policy_version 36150 (0.0009) [2023-10-10 17:59:25,417][123614] Updated weights for policy 1, policy_version 36160 (0.0007) [2023-10-10 17:59:27,377][123582] Updated weights for policy 0, policy_version 36233 (0.0007) [2023-10-10 17:59:27,748][123582] Updated weights for policy 0, policy_version 36243 (0.0007) [2023-10-10 17:59:28,114][123582] Updated weights for policy 0, policy_version 36253 (0.0010) [2023-10-10 17:59:28,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 74153984. Throughput: 0: 1828.1, 1: 1797.8. Samples: 18543016. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-10 17:59:28,788][122664] Avg episode reward: [(0, '45.190'), (1, '43.130')] [2023-10-10 17:59:29,168][123614] Updated weights for policy 1, policy_version 36170 (0.0009) [2023-10-10 17:59:29,542][123614] Updated weights for policy 1, policy_version 36180 (0.0009) [2023-10-10 17:59:29,909][123614] Updated weights for policy 1, policy_version 36190 (0.0007) [2023-10-10 17:59:31,866][123582] Updated weights for policy 0, policy_version 36263 (0.0008) [2023-10-10 17:59:32,241][123582] Updated weights for policy 0, policy_version 36273 (0.0009) [2023-10-10 17:59:32,615][123582] Updated weights for policy 0, policy_version 36283 (0.0009) [2023-10-10 17:59:33,611][123614] Updated weights for policy 1, policy_version 36200 (0.0010) [2023-10-10 17:59:33,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 74219520. Throughput: 0: 1828.4, 1: 1797.3. Samples: 18564888. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-10 17:59:33,788][122664] Avg episode reward: [(0, '44.340'), (1, '40.430')] [2023-10-10 17:59:33,988][123614] Updated weights for policy 1, policy_version 36210 (0.0008) [2023-10-10 17:59:34,353][123614] Updated weights for policy 1, policy_version 36220 (0.0009) [2023-10-10 17:59:36,047][123582] Updated weights for policy 0, policy_version 36293 (0.0010) [2023-10-10 17:59:36,415][123582] Updated weights for policy 0, policy_version 36303 (0.0010) [2023-10-10 17:59:36,787][123582] Updated weights for policy 0, policy_version 36313 (0.0007) [2023-10-10 17:59:37,944][123614] Updated weights for policy 1, policy_version 36230 (0.0009) [2023-10-10 17:59:38,319][123614] Updated weights for policy 1, policy_version 36240 (0.0008) [2023-10-10 17:59:38,681][123614] Updated weights for policy 1, policy_version 36250 (0.0007) [2023-10-10 17:59:38,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 74285056. Throughput: 0: 1835.9, 1: 1811.8. Samples: 18586234. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-10 17:59:38,788][122664] Avg episode reward: [(0, '48.210'), (1, '43.520')] [2023-10-10 17:59:40,623][123582] Updated weights for policy 0, policy_version 36323 (0.0009) [2023-10-10 17:59:41,007][123582] Updated weights for policy 0, policy_version 36333 (0.0009) [2023-10-10 17:59:41,378][123582] Updated weights for policy 0, policy_version 36343 (0.0010) [2023-10-10 17:59:42,487][123614] Updated weights for policy 1, policy_version 36260 (0.0009) [2023-10-10 17:59:42,863][123614] Updated weights for policy 1, policy_version 36270 (0.0008) [2023-10-10 17:59:43,228][123614] Updated weights for policy 1, policy_version 36280 (0.0007) [2023-10-10 17:59:43,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 74383360. Throughput: 0: 1824.2, 1: 1809.8. Samples: 18597710. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-10 17:59:43,788][122664] Avg episode reward: [(0, '47.400'), (1, '43.010')] [2023-10-10 17:59:44,974][123582] Updated weights for policy 0, policy_version 36353 (0.0010) [2023-10-10 17:59:45,346][123582] Updated weights for policy 0, policy_version 36363 (0.0007) [2023-10-10 17:59:45,719][123582] Updated weights for policy 0, policy_version 36373 (0.0007) [2023-10-10 17:59:46,102][123582] Updated weights for policy 0, policy_version 36383 (0.0009) [2023-10-10 17:59:46,892][123614] Updated weights for policy 1, policy_version 36290 (0.0007) [2023-10-10 17:59:47,259][123614] Updated weights for policy 1, policy_version 36300 (0.0009) [2023-10-10 17:59:47,626][123614] Updated weights for policy 1, policy_version 36310 (0.0008) [2023-10-10 17:59:47,994][123614] Updated weights for policy 1, policy_version 36320 (0.0010) [2023-10-10 17:59:48,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 74448896. Throughput: 0: 1836.2, 1: 1814.0. Samples: 18619020. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-10-10 17:59:48,789][122664] Avg episode reward: [(0, '49.160'), (1, '41.930')] [2023-10-10 17:59:49,801][123582] Updated weights for policy 0, policy_version 36393 (0.0008) [2023-10-10 17:59:50,180][123582] Updated weights for policy 0, policy_version 36403 (0.0007) [2023-10-10 17:59:50,557][123582] Updated weights for policy 0, policy_version 36413 (0.0009) [2023-10-10 17:59:51,819][123614] Updated weights for policy 1, policy_version 36330 (0.0008) [2023-10-10 17:59:52,196][123614] Updated weights for policy 1, policy_version 36340 (0.0007) [2023-10-10 17:59:52,560][123614] Updated weights for policy 1, policy_version 36350 (0.0008) [2023-10-10 17:59:53,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 74514432. Throughput: 0: 1824.5, 1: 1807.5. Samples: 18641220. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-10-10 17:59:53,789][122664] Avg episode reward: [(0, '46.860'), (1, '42.680')] [2023-10-10 17:59:53,799][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000036352_37224448.pth... [2023-10-10 17:59:53,799][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000036416_37289984.pth... [2023-10-10 17:59:53,835][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000034656_35487744.pth [2023-10-10 17:59:53,840][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000034720_35553280.pth [2023-10-10 17:59:54,250][123582] Updated weights for policy 0, policy_version 36423 (0.0009) [2023-10-10 17:59:54,628][123582] Updated weights for policy 0, policy_version 36433 (0.0010) [2023-10-10 17:59:55,002][123582] Updated weights for policy 0, policy_version 36443 (0.0008) [2023-10-10 17:59:56,234][123614] Updated weights for policy 1, policy_version 36360 (0.0007) [2023-10-10 17:59:56,606][123614] Updated weights for policy 1, policy_version 36370 (0.0009) [2023-10-10 17:59:56,967][123614] Updated weights for policy 1, policy_version 36380 (0.0008) [2023-10-10 17:59:58,710][123582] Updated weights for policy 0, policy_version 36453 (0.0010) [2023-10-10 17:59:58,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 74579968. Throughput: 0: 1826.8, 1: 1816.2. Samples: 18651800. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-10-10 17:59:58,789][122664] Avg episode reward: [(0, '41.450'), (1, '39.920')] [2023-10-10 17:59:59,081][123582] Updated weights for policy 0, policy_version 36463 (0.0011) [2023-10-10 17:59:59,452][123582] Updated weights for policy 0, policy_version 36473 (0.0011) [2023-10-10 18:00:00,641][123614] Updated weights for policy 1, policy_version 36390 (0.0009) [2023-10-10 18:00:01,011][123614] Updated weights for policy 1, policy_version 36400 (0.0010) [2023-10-10 18:00:01,373][123614] Updated weights for policy 1, policy_version 36410 (0.0007) [2023-10-10 18:00:03,031][123582] Updated weights for policy 0, policy_version 36483 (0.0007) [2023-10-10 18:00:03,400][123582] Updated weights for policy 0, policy_version 36493 (0.0007) [2023-10-10 18:00:03,775][123582] Updated weights for policy 0, policy_version 36503 (0.0007) [2023-10-10 18:00:03,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 74645504. Throughput: 0: 1826.2, 1: 1808.2. Samples: 18674006. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-10-10 18:00:03,789][122664] Avg episode reward: [(0, '42.210'), (1, '40.620')] [2023-10-10 18:00:05,135][123614] Updated weights for policy 1, policy_version 36420 (0.0008) [2023-10-10 18:00:05,501][123614] Updated weights for policy 1, policy_version 36430 (0.0007) [2023-10-10 18:00:05,863][123614] Updated weights for policy 1, policy_version 36440 (0.0007) [2023-10-10 18:00:07,368][123582] Updated weights for policy 0, policy_version 36513 (0.0007) [2023-10-10 18:00:07,739][123582] Updated weights for policy 0, policy_version 36523 (0.0009) [2023-10-10 18:00:08,106][123582] Updated weights for policy 0, policy_version 36533 (0.0010) [2023-10-10 18:00:08,467][123582] Updated weights for policy 0, policy_version 36543 (0.0010) [2023-10-10 18:00:08,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 74743808. Throughput: 0: 1821.8, 1: 1810.5. Samples: 18695518. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-10-10 18:00:08,789][122664] Avg episode reward: [(0, '40.290'), (1, '40.340')] [2023-10-10 18:00:09,677][123614] Updated weights for policy 1, policy_version 36450 (0.0010) [2023-10-10 18:00:10,049][123614] Updated weights for policy 1, policy_version 36460 (0.0009) [2023-10-10 18:00:10,424][123614] Updated weights for policy 1, policy_version 36470 (0.0009) [2023-10-10 18:00:10,781][123614] Updated weights for policy 1, policy_version 36480 (0.0010) [2023-10-10 18:00:12,225][123582] Updated weights for policy 0, policy_version 36553 (0.0008) [2023-10-10 18:00:12,593][123582] Updated weights for policy 0, policy_version 36563 (0.0008) [2023-10-10 18:00:12,967][123582] Updated weights for policy 0, policy_version 36573 (0.0010) [2023-10-10 18:00:13,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 74809344. Throughput: 0: 1827.5, 1: 1810.3. Samples: 18706716. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) [2023-10-10 18:00:13,788][122664] Avg episode reward: [(0, '41.130'), (1, '44.270')] [2023-10-10 18:00:14,364][123614] Updated weights for policy 1, policy_version 36490 (0.0010) [2023-10-10 18:00:14,736][123614] Updated weights for policy 1, policy_version 36500 (0.0011) [2023-10-10 18:00:15,112][123614] Updated weights for policy 1, policy_version 36510 (0.0009) [2023-10-10 18:00:16,571][123582] Updated weights for policy 0, policy_version 36583 (0.0010) [2023-10-10 18:00:16,944][123582] Updated weights for policy 0, policy_version 36593 (0.0010) [2023-10-10 18:00:17,319][123582] Updated weights for policy 0, policy_version 36603 (0.0011) [2023-10-10 18:00:18,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 74874880. Throughput: 0: 1816.1, 1: 1811.0. Samples: 18728108. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) [2023-10-10 18:00:18,788][122664] Avg episode reward: [(0, '43.210'), (1, '42.910')] [2023-10-10 18:00:18,906][123614] Updated weights for policy 1, policy_version 36520 (0.0010) [2023-10-10 18:00:19,272][123614] Updated weights for policy 1, policy_version 36530 (0.0008) [2023-10-10 18:00:19,646][123614] Updated weights for policy 1, policy_version 36540 (0.0010) [2023-10-10 18:00:20,912][123582] Updated weights for policy 0, policy_version 36613 (0.0007) [2023-10-10 18:00:21,284][123582] Updated weights for policy 0, policy_version 36623 (0.0010) [2023-10-10 18:00:21,662][123582] Updated weights for policy 0, policy_version 36633 (0.0010) [2023-10-10 18:00:23,453][123614] Updated weights for policy 1, policy_version 36550 (0.0008) [2023-10-10 18:00:23,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 74940416. Throughput: 0: 1817.2, 1: 1813.5. Samples: 18749616. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) [2023-10-10 18:00:23,788][122664] Avg episode reward: [(0, '40.440'), (1, '42.210')] [2023-10-10 18:00:23,812][123614] Updated weights for policy 1, policy_version 36560 (0.0007) [2023-10-10 18:00:24,182][123614] Updated weights for policy 1, policy_version 36570 (0.0008) [2023-10-10 18:00:25,454][123582] Updated weights for policy 0, policy_version 36643 (0.0010) [2023-10-10 18:00:25,830][123582] Updated weights for policy 0, policy_version 36653 (0.0009) [2023-10-10 18:00:26,205][123582] Updated weights for policy 0, policy_version 36663 (0.0008) [2023-10-10 18:00:27,716][123614] Updated weights for policy 1, policy_version 36580 (0.0008) [2023-10-10 18:00:28,086][123614] Updated weights for policy 1, policy_version 36590 (0.0010) [2023-10-10 18:00:28,458][123614] Updated weights for policy 1, policy_version 36600 (0.0010) [2023-10-10 18:00:28,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 75038720. Throughput: 0: 1811.7, 1: 1799.7. Samples: 18760222. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) [2023-10-10 18:00:28,789][122664] Avg episode reward: [(0, '36.410'), (1, '42.080')] [2023-10-10 18:00:29,877][123582] Updated weights for policy 0, policy_version 36673 (0.0008) [2023-10-10 18:00:30,253][123582] Updated weights for policy 0, policy_version 36683 (0.0007) [2023-10-10 18:00:30,619][123582] Updated weights for policy 0, policy_version 36693 (0.0007) [2023-10-10 18:00:30,991][123582] Updated weights for policy 0, policy_version 36703 (0.0007) [2023-10-10 18:00:32,211][123614] Updated weights for policy 1, policy_version 36610 (0.0008) [2023-10-10 18:00:32,585][123614] Updated weights for policy 1, policy_version 36620 (0.0008) [2023-10-10 18:00:32,958][123614] Updated weights for policy 1, policy_version 36630 (0.0008) [2023-10-10 18:00:33,321][123614] Updated weights for policy 1, policy_version 36640 (0.0007) [2023-10-10 18:00:33,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 75104256. Throughput: 0: 1820.8, 1: 1805.5. Samples: 18782204. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) [2023-10-10 18:00:33,788][122664] Avg episode reward: [(0, '35.130'), (1, '44.880')] [2023-10-10 18:00:34,564][123582] Updated weights for policy 0, policy_version 36713 (0.0008) [2023-10-10 18:00:34,930][123582] Updated weights for policy 0, policy_version 36723 (0.0008) [2023-10-10 18:00:35,308][123582] Updated weights for policy 0, policy_version 36733 (0.0008) [2023-10-10 18:00:37,246][123614] Updated weights for policy 1, policy_version 36650 (0.0008) [2023-10-10 18:00:37,620][123614] Updated weights for policy 1, policy_version 36660 (0.0009) [2023-10-10 18:00:37,992][123614] Updated weights for policy 1, policy_version 36670 (0.0009) [2023-10-10 18:00:38,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 75169792. Throughput: 0: 1826.3, 1: 1794.3. Samples: 18804144. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) [2023-10-10 18:00:38,789][122664] Avg episode reward: [(0, '35.450'), (1, '46.460')] [2023-10-10 18:00:38,953][123582] Updated weights for policy 0, policy_version 36743 (0.0008) [2023-10-10 18:00:39,324][123582] Updated weights for policy 0, policy_version 36753 (0.0009) [2023-10-10 18:00:39,689][123582] Updated weights for policy 0, policy_version 36763 (0.0008) [2023-10-10 18:00:41,601][123614] Updated weights for policy 1, policy_version 36680 (0.0008) [2023-10-10 18:00:41,967][123614] Updated weights for policy 1, policy_version 36690 (0.0008) [2023-10-10 18:00:42,348][123614] Updated weights for policy 1, policy_version 36700 (0.0011) [2023-10-10 18:00:43,305][123582] Updated weights for policy 0, policy_version 36773 (0.0009) [2023-10-10 18:00:43,683][123582] Updated weights for policy 0, policy_version 36783 (0.0008) [2023-10-10 18:00:43,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 75235328. Throughput: 0: 1823.8, 1: 1808.1. Samples: 18815236. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 18:00:43,789][122664] Avg episode reward: [(0, '37.710'), (1, '46.210')] [2023-10-10 18:00:44,051][123582] Updated weights for policy 0, policy_version 36793 (0.0009) [2023-10-10 18:00:45,939][123614] Updated weights for policy 1, policy_version 36710 (0.0010) [2023-10-10 18:00:46,312][123614] Updated weights for policy 1, policy_version 36720 (0.0011) [2023-10-10 18:00:46,676][123614] Updated weights for policy 1, policy_version 36730 (0.0008) [2023-10-10 18:00:47,627][123582] Updated weights for policy 0, policy_version 36803 (0.0008) [2023-10-10 18:00:47,994][123582] Updated weights for policy 0, policy_version 36813 (0.0008) [2023-10-10 18:00:48,363][123582] Updated weights for policy 0, policy_version 36823 (0.0008) [2023-10-10 18:00:48,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 75333632. Throughput: 0: 1828.1, 1: 1801.4. Samples: 18837334. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 18:00:48,789][122664] Avg episode reward: [(0, '37.810'), (1, '49.070')] [2023-10-10 18:00:48,790][123465] Saving new best policy, reward=49.070! [2023-10-10 18:00:50,411][123614] Updated weights for policy 1, policy_version 36740 (0.0008) [2023-10-10 18:00:50,798][123614] Updated weights for policy 1, policy_version 36750 (0.0009) [2023-10-10 18:00:51,153][123614] Updated weights for policy 1, policy_version 36760 (0.0008) [2023-10-10 18:00:52,123][123582] Updated weights for policy 0, policy_version 36833 (0.0009) [2023-10-10 18:00:52,509][123582] Updated weights for policy 0, policy_version 36843 (0.0011) [2023-10-10 18:00:52,874][123582] Updated weights for policy 0, policy_version 36853 (0.0009) [2023-10-10 18:00:53,241][123582] Updated weights for policy 0, policy_version 36863 (0.0009) [2023-10-10 18:00:53,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 75399168. Throughput: 0: 1820.9, 1: 1800.9. Samples: 18858496. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 18:00:53,788][122664] Avg episode reward: [(0, '37.060'), (1, '50.300')] [2023-10-10 18:00:53,800][123465] Saving new best policy, reward=50.300! [2023-10-10 18:00:55,064][123614] Updated weights for policy 1, policy_version 36770 (0.0007) [2023-10-10 18:00:55,428][123614] Updated weights for policy 1, policy_version 36780 (0.0007) [2023-10-10 18:00:55,805][123614] Updated weights for policy 1, policy_version 36790 (0.0007) [2023-10-10 18:00:56,177][123614] Updated weights for policy 1, policy_version 36800 (0.0008) [2023-10-10 18:00:57,158][123582] Updated weights for policy 0, policy_version 36873 (0.0009) [2023-10-10 18:00:57,525][123582] Updated weights for policy 0, policy_version 36883 (0.0009) [2023-10-10 18:00:57,901][123582] Updated weights for policy 0, policy_version 36893 (0.0010) [2023-10-10 18:00:58,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 75464704. Throughput: 0: 1821.3, 1: 1799.4. Samples: 18869648. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 18:00:58,789][122664] Avg episode reward: [(0, '37.900'), (1, '52.350')] [2023-10-10 18:00:58,790][123465] Saving new best policy, reward=52.350! [2023-10-10 18:00:59,811][123614] Updated weights for policy 1, policy_version 36810 (0.0007) [2023-10-10 18:01:00,177][123614] Updated weights for policy 1, policy_version 36820 (0.0008) [2023-10-10 18:01:00,541][123614] Updated weights for policy 1, policy_version 36830 (0.0009) [2023-10-10 18:01:01,552][123582] Updated weights for policy 0, policy_version 36903 (0.0009) [2023-10-10 18:01:01,924][123582] Updated weights for policy 0, policy_version 36913 (0.0008) [2023-10-10 18:01:02,297][123582] Updated weights for policy 0, policy_version 36923 (0.0008) [2023-10-10 18:01:03,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 75530240. Throughput: 0: 1819.8, 1: 1809.1. Samples: 18891408. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 18:01:03,789][122664] Avg episode reward: [(0, '37.150'), (1, '51.860')] [2023-10-10 18:01:04,091][123614] Updated weights for policy 1, policy_version 36840 (0.0007) [2023-10-10 18:01:04,451][123614] Updated weights for policy 1, policy_version 36850 (0.0008) [2023-10-10 18:01:04,823][123614] Updated weights for policy 1, policy_version 36860 (0.0007) [2023-10-10 18:01:06,122][123582] Updated weights for policy 0, policy_version 36933 (0.0009) [2023-10-10 18:01:06,491][123582] Updated weights for policy 0, policy_version 36943 (0.0007) [2023-10-10 18:01:06,868][123582] Updated weights for policy 0, policy_version 36953 (0.0007) [2023-10-10 18:01:08,433][123614] Updated weights for policy 1, policy_version 36870 (0.0008) [2023-10-10 18:01:08,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 75595776. Throughput: 0: 1821.1, 1: 1821.6. Samples: 18913542. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-10 18:01:08,789][122664] Avg episode reward: [(0, '37.400'), (1, '51.610')] [2023-10-10 18:01:08,805][123614] Updated weights for policy 1, policy_version 36880 (0.0007) [2023-10-10 18:01:09,174][123614] Updated weights for policy 1, policy_version 36890 (0.0008) [2023-10-10 18:01:10,655][123582] Updated weights for policy 0, policy_version 36963 (0.0008) [2023-10-10 18:01:11,051][123582] Updated weights for policy 0, policy_version 36973 (0.0009) [2023-10-10 18:01:11,428][123582] Updated weights for policy 0, policy_version 36983 (0.0010) [2023-10-10 18:01:12,904][123614] Updated weights for policy 1, policy_version 36900 (0.0007) [2023-10-10 18:01:13,280][123614] Updated weights for policy 1, policy_version 36910 (0.0009) [2023-10-10 18:01:13,654][123614] Updated weights for policy 1, policy_version 36920 (0.0009) [2023-10-10 18:01:13,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 75661312. Throughput: 0: 1828.2, 1: 1821.7. Samples: 18924470. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-10 18:01:13,788][122664] Avg episode reward: [(0, '38.200'), (1, '48.820')] [2023-10-10 18:01:15,172][123582] Updated weights for policy 0, policy_version 36993 (0.0009) [2023-10-10 18:01:15,552][123582] Updated weights for policy 0, policy_version 37003 (0.0009) [2023-10-10 18:01:15,925][123582] Updated weights for policy 0, policy_version 37013 (0.0010) [2023-10-10 18:01:16,301][123582] Updated weights for policy 0, policy_version 37023 (0.0010) [2023-10-10 18:01:17,250][123614] Updated weights for policy 1, policy_version 36930 (0.0008) [2023-10-10 18:01:17,625][123614] Updated weights for policy 1, policy_version 36940 (0.0008) [2023-10-10 18:01:17,988][123614] Updated weights for policy 1, policy_version 36950 (0.0010) [2023-10-10 18:01:18,361][123614] Updated weights for policy 1, policy_version 36960 (0.0010) [2023-10-10 18:01:18,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 75759616. Throughput: 0: 1810.0, 1: 1824.0. Samples: 18945730. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-10 18:01:18,789][122664] Avg episode reward: [(0, '37.560'), (1, '50.380')] [2023-10-10 18:01:19,962][123582] Updated weights for policy 0, policy_version 37033 (0.0008) [2023-10-10 18:01:20,335][123582] Updated weights for policy 0, policy_version 37043 (0.0009) [2023-10-10 18:01:20,704][123582] Updated weights for policy 0, policy_version 37053 (0.0008) [2023-10-10 18:01:22,124][123614] Updated weights for policy 1, policy_version 36970 (0.0007) [2023-10-10 18:01:22,491][123614] Updated weights for policy 1, policy_version 36980 (0.0008) [2023-10-10 18:01:22,858][123614] Updated weights for policy 1, policy_version 36990 (0.0010) [2023-10-10 18:01:23,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 75825152. Throughput: 0: 1809.2, 1: 1824.4. Samples: 18967652. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-10 18:01:23,788][122664] Avg episode reward: [(0, '41.340'), (1, '46.250')] [2023-10-10 18:01:24,404][123582] Updated weights for policy 0, policy_version 37063 (0.0010) [2023-10-10 18:01:24,774][123582] Updated weights for policy 0, policy_version 37073 (0.0009) [2023-10-10 18:01:25,158][123582] Updated weights for policy 0, policy_version 37083 (0.0008) [2023-10-10 18:01:26,576][123614] Updated weights for policy 1, policy_version 37000 (0.0008) [2023-10-10 18:01:26,957][123614] Updated weights for policy 1, policy_version 37010 (0.0010) [2023-10-10 18:01:27,322][123614] Updated weights for policy 1, policy_version 37020 (0.0007) [2023-10-10 18:01:28,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 75890688. Throughput: 0: 1808.6, 1: 1820.6. Samples: 18978550. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-10 18:01:28,789][122664] Avg episode reward: [(0, '39.640'), (1, '45.580')] [2023-10-10 18:01:28,854][123582] Updated weights for policy 0, policy_version 37093 (0.0009) [2023-10-10 18:01:29,224][123582] Updated weights for policy 0, policy_version 37103 (0.0009) [2023-10-10 18:01:29,602][123582] Updated weights for policy 0, policy_version 37113 (0.0010) [2023-10-10 18:01:31,016][123614] Updated weights for policy 1, policy_version 37030 (0.0007) [2023-10-10 18:01:31,388][123614] Updated weights for policy 1, policy_version 37040 (0.0008) [2023-10-10 18:01:31,756][123614] Updated weights for policy 1, policy_version 37050 (0.0007) [2023-10-10 18:01:33,249][123582] Updated weights for policy 0, policy_version 37123 (0.0010) [2023-10-10 18:01:33,625][123582] Updated weights for policy 0, policy_version 37133 (0.0007) [2023-10-10 18:01:33,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 75956224. Throughput: 0: 1804.7, 1: 1820.6. Samples: 19000470. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-10 18:01:33,788][122664] Avg episode reward: [(0, '37.630'), (1, '44.120')] [2023-10-10 18:01:34,000][123582] Updated weights for policy 0, policy_version 37143 (0.0008) [2023-10-10 18:01:35,381][123614] Updated weights for policy 1, policy_version 37060 (0.0009) [2023-10-10 18:01:35,752][123614] Updated weights for policy 1, policy_version 37070 (0.0010) [2023-10-10 18:01:36,114][123614] Updated weights for policy 1, policy_version 37080 (0.0010) [2023-10-10 18:01:37,534][123582] Updated weights for policy 0, policy_version 37153 (0.0008) [2023-10-10 18:01:37,899][123582] Updated weights for policy 0, policy_version 37163 (0.0009) [2023-10-10 18:01:38,275][123582] Updated weights for policy 0, policy_version 37173 (0.0011) [2023-10-10 18:01:38,647][123582] Updated weights for policy 0, policy_version 37183 (0.0008) [2023-10-10 18:01:38,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 76054528. Throughput: 0: 1815.2, 1: 1821.2. Samples: 19022132. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-10 18:01:38,788][122664] Avg episode reward: [(0, '38.940'), (1, '39.930')] [2023-10-10 18:01:39,921][123614] Updated weights for policy 1, policy_version 37090 (0.0010) [2023-10-10 18:01:40,295][123614] Updated weights for policy 1, policy_version 37100 (0.0008) [2023-10-10 18:01:40,671][123614] Updated weights for policy 1, policy_version 37110 (0.0008) [2023-10-10 18:01:41,050][123614] Updated weights for policy 1, policy_version 37120 (0.0009) [2023-10-10 18:01:42,369][123582] Updated weights for policy 0, policy_version 37193 (0.0008) [2023-10-10 18:01:42,746][123582] Updated weights for policy 0, policy_version 37203 (0.0007) [2023-10-10 18:01:43,125][123582] Updated weights for policy 0, policy_version 37213 (0.0009) [2023-10-10 18:01:43,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 76120064. Throughput: 0: 1811.1, 1: 1822.8. Samples: 19033172. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-10 18:01:43,788][122664] Avg episode reward: [(0, '40.200'), (1, '38.000')] [2023-10-10 18:01:44,666][123614] Updated weights for policy 1, policy_version 37130 (0.0009) [2023-10-10 18:01:45,036][123614] Updated weights for policy 1, policy_version 37140 (0.0010) [2023-10-10 18:01:45,414][123614] Updated weights for policy 1, policy_version 37150 (0.0010) [2023-10-10 18:01:46,891][123582] Updated weights for policy 0, policy_version 37223 (0.0008) [2023-10-10 18:01:47,264][123582] Updated weights for policy 0, policy_version 37233 (0.0009) [2023-10-10 18:01:47,648][123582] Updated weights for policy 0, policy_version 37243 (0.0010) [2023-10-10 18:01:48,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 76185600. Throughput: 0: 1821.6, 1: 1817.2. Samples: 19055156. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-10 18:01:48,788][122664] Avg episode reward: [(0, '39.960'), (1, '39.070')] [2023-10-10 18:01:49,181][123614] Updated weights for policy 1, policy_version 37160 (0.0011) [2023-10-10 18:01:49,556][123614] Updated weights for policy 1, policy_version 37170 (0.0009) [2023-10-10 18:01:49,922][123614] Updated weights for policy 1, policy_version 37180 (0.0008) [2023-10-10 18:01:51,344][123582] Updated weights for policy 0, policy_version 37253 (0.0008) [2023-10-10 18:01:51,709][123582] Updated weights for policy 0, policy_version 37263 (0.0008) [2023-10-10 18:01:52,091][123582] Updated weights for policy 0, policy_version 37273 (0.0007) [2023-10-10 18:01:53,547][123614] Updated weights for policy 1, policy_version 37190 (0.0008) [2023-10-10 18:01:53,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 76251136. Throughput: 0: 1812.8, 1: 1814.8. Samples: 19076784. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-10 18:01:53,788][122664] Avg episode reward: [(0, '37.480'), (1, '40.790')] [2023-10-10 18:01:53,796][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000037280_38174720.pth... [2023-10-10 18:01:53,832][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000035584_36438016.pth [2023-10-10 18:01:53,918][123614] Updated weights for policy 1, policy_version 37200 (0.0009) [2023-10-10 18:01:54,293][123614] Updated weights for policy 1, policy_version 37210 (0.0011) [2023-10-10 18:01:54,501][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000037216_38109184.pth... [2023-10-10 18:01:54,544][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000035520_36372480.pth [2023-10-10 18:01:55,725][123582] Updated weights for policy 0, policy_version 37283 (0.0007) [2023-10-10 18:01:56,120][123582] Updated weights for policy 0, policy_version 37293 (0.0009) [2023-10-10 18:01:56,492][123582] Updated weights for policy 0, policy_version 37303 (0.0008) [2023-10-10 18:01:58,035][123614] Updated weights for policy 1, policy_version 37220 (0.0009) [2023-10-10 18:01:58,405][123614] Updated weights for policy 1, policy_version 37230 (0.0010) [2023-10-10 18:01:58,769][123614] Updated weights for policy 1, policy_version 37240 (0.0008) [2023-10-10 18:01:58,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 76316672. Throughput: 0: 1811.5, 1: 1810.5. Samples: 19087464. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-10 18:01:58,789][122664] Avg episode reward: [(0, '39.470'), (1, '41.130')] [2023-10-10 18:02:00,189][123582] Updated weights for policy 0, policy_version 37313 (0.0009) [2023-10-10 18:02:00,568][123582] Updated weights for policy 0, policy_version 37323 (0.0009) [2023-10-10 18:02:00,934][123582] Updated weights for policy 0, policy_version 37333 (0.0011) [2023-10-10 18:02:01,301][123582] Updated weights for policy 0, policy_version 37343 (0.0010) [2023-10-10 18:02:02,509][123614] Updated weights for policy 1, policy_version 37250 (0.0007) [2023-10-10 18:02:02,869][123614] Updated weights for policy 1, policy_version 37260 (0.0009) [2023-10-10 18:02:03,239][123614] Updated weights for policy 1, policy_version 37270 (0.0008) [2023-10-10 18:02:03,606][123614] Updated weights for policy 1, policy_version 37280 (0.0009) [2023-10-10 18:02:03,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 76414976. Throughput: 0: 1813.4, 1: 1817.8. Samples: 19109134. Policy #0 lag: (min: 14.0, avg: 18.4, max: 46.0) [2023-10-10 18:02:03,789][122664] Avg episode reward: [(0, '39.020'), (1, '41.390')] [2023-10-10 18:02:04,916][123582] Updated weights for policy 0, policy_version 37353 (0.0009) [2023-10-10 18:02:05,295][123582] Updated weights for policy 0, policy_version 37363 (0.0009) [2023-10-10 18:02:05,668][123582] Updated weights for policy 0, policy_version 37373 (0.0009) [2023-10-10 18:02:07,292][123614] Updated weights for policy 1, policy_version 37290 (0.0010) [2023-10-10 18:02:07,657][123614] Updated weights for policy 1, policy_version 37300 (0.0010) [2023-10-10 18:02:08,032][123614] Updated weights for policy 1, policy_version 37310 (0.0010) [2023-10-10 18:02:08,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 76480512. Throughput: 0: 1814.6, 1: 1812.4. Samples: 19130868. Policy #0 lag: (min: 14.0, avg: 18.4, max: 46.0) [2023-10-10 18:02:08,789][122664] Avg episode reward: [(0, '40.390'), (1, '44.390')] [2023-10-10 18:02:09,276][123582] Updated weights for policy 0, policy_version 37383 (0.0009) [2023-10-10 18:02:09,641][123582] Updated weights for policy 0, policy_version 37393 (0.0011) [2023-10-10 18:02:10,015][123582] Updated weights for policy 0, policy_version 37403 (0.0010) [2023-10-10 18:02:11,678][123614] Updated weights for policy 1, policy_version 37320 (0.0009) [2023-10-10 18:02:12,056][123614] Updated weights for policy 1, policy_version 37330 (0.0008) [2023-10-10 18:02:12,414][123614] Updated weights for policy 1, policy_version 37340 (0.0009) [2023-10-10 18:02:13,604][123582] Updated weights for policy 0, policy_version 37413 (0.0010) [2023-10-10 18:02:13,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 76546048. Throughput: 0: 1813.9, 1: 1815.1. Samples: 19141856. Policy #0 lag: (min: 14.0, avg: 18.4, max: 46.0) [2023-10-10 18:02:13,788][122664] Avg episode reward: [(0, '43.180'), (1, '42.740')] [2023-10-10 18:02:13,970][123582] Updated weights for policy 0, policy_version 37423 (0.0009) [2023-10-10 18:02:14,345][123582] Updated weights for policy 0, policy_version 37433 (0.0009) [2023-10-10 18:02:16,127][123614] Updated weights for policy 1, policy_version 37350 (0.0009) [2023-10-10 18:02:16,494][123614] Updated weights for policy 1, policy_version 37360 (0.0009) [2023-10-10 18:02:16,867][123614] Updated weights for policy 1, policy_version 37370 (0.0008) [2023-10-10 18:02:17,916][123582] Updated weights for policy 0, policy_version 37443 (0.0010) [2023-10-10 18:02:18,290][123582] Updated weights for policy 0, policy_version 37453 (0.0007) [2023-10-10 18:02:18,665][123582] Updated weights for policy 0, policy_version 37463 (0.0010) [2023-10-10 18:02:18,788][122664] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 76611584. Throughput: 0: 1822.6, 1: 1809.5. Samples: 19163914. Policy #0 lag: (min: 14.0, avg: 18.4, max: 46.0) [2023-10-10 18:02:18,788][122664] Avg episode reward: [(0, '43.490'), (1, '49.480')] [2023-10-10 18:02:20,692][123614] Updated weights for policy 1, policy_version 37380 (0.0009) [2023-10-10 18:02:21,059][123614] Updated weights for policy 1, policy_version 37390 (0.0009) [2023-10-10 18:02:21,438][123614] Updated weights for policy 1, policy_version 37400 (0.0010) [2023-10-10 18:02:22,327][123582] Updated weights for policy 0, policy_version 37473 (0.0008) [2023-10-10 18:02:22,697][123582] Updated weights for policy 0, policy_version 37483 (0.0008) [2023-10-10 18:02:23,067][123582] Updated weights for policy 0, policy_version 37493 (0.0011) [2023-10-10 18:02:23,454][123582] Updated weights for policy 0, policy_version 37503 (0.0011) [2023-10-10 18:02:23,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 76709888. Throughput: 0: 1821.3, 1: 1812.0. Samples: 19185630. Policy #0 lag: (min: 14.0, avg: 18.4, max: 46.0) [2023-10-10 18:02:23,789][122664] Avg episode reward: [(0, '46.310'), (1, '49.260')] [2023-10-10 18:02:25,289][123614] Updated weights for policy 1, policy_version 37410 (0.0009) [2023-10-10 18:02:25,658][123614] Updated weights for policy 1, policy_version 37420 (0.0011) [2023-10-10 18:02:26,033][123614] Updated weights for policy 1, policy_version 37430 (0.0011) [2023-10-10 18:02:26,398][123614] Updated weights for policy 1, policy_version 37440 (0.0009) [2023-10-10 18:02:27,176][123582] Updated weights for policy 0, policy_version 37513 (0.0008) [2023-10-10 18:02:27,539][123582] Updated weights for policy 0, policy_version 37523 (0.0008) [2023-10-10 18:02:27,917][123582] Updated weights for policy 0, policy_version 37533 (0.0009) [2023-10-10 18:02:28,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 76775424. Throughput: 0: 1824.0, 1: 1811.1. Samples: 19196754. Policy #0 lag: (min: 24.0, avg: 44.2, max: 56.0) [2023-10-10 18:02:28,789][122664] Avg episode reward: [(0, '45.760'), (1, '49.630')] [2023-10-10 18:02:30,055][123614] Updated weights for policy 1, policy_version 37450 (0.0007) [2023-10-10 18:02:30,429][123614] Updated weights for policy 1, policy_version 37460 (0.0008) [2023-10-10 18:02:30,793][123614] Updated weights for policy 1, policy_version 37470 (0.0007) [2023-10-10 18:02:31,651][123582] Updated weights for policy 0, policy_version 37543 (0.0010) [2023-10-10 18:02:32,019][123582] Updated weights for policy 0, policy_version 37553 (0.0008) [2023-10-10 18:02:32,393][123582] Updated weights for policy 0, policy_version 37563 (0.0007) [2023-10-10 18:02:33,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 76840960. Throughput: 0: 1820.1, 1: 1809.8. Samples: 19218500. Policy #0 lag: (min: 24.0, avg: 44.2, max: 56.0) [2023-10-10 18:02:33,789][122664] Avg episode reward: [(0, '46.460'), (1, '48.510')] [2023-10-10 18:02:34,554][123614] Updated weights for policy 1, policy_version 37480 (0.0008) [2023-10-10 18:02:34,930][123614] Updated weights for policy 1, policy_version 37490 (0.0008) [2023-10-10 18:02:35,302][123614] Updated weights for policy 1, policy_version 37500 (0.0007) [2023-10-10 18:02:35,994][123582] Updated weights for policy 0, policy_version 37573 (0.0010) [2023-10-10 18:02:36,363][123582] Updated weights for policy 0, policy_version 37583 (0.0010) [2023-10-10 18:02:36,736][123582] Updated weights for policy 0, policy_version 37593 (0.0009) [2023-10-10 18:02:38,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 76906496. Throughput: 0: 1829.9, 1: 1819.3. Samples: 19240998. Policy #0 lag: (min: 24.0, avg: 44.2, max: 56.0) [2023-10-10 18:02:38,789][122664] Avg episode reward: [(0, '42.550'), (1, '47.370')] [2023-10-10 18:02:38,887][123614] Updated weights for policy 1, policy_version 37510 (0.0008) [2023-10-10 18:02:39,256][123614] Updated weights for policy 1, policy_version 37520 (0.0008) [2023-10-10 18:02:39,627][123614] Updated weights for policy 1, policy_version 37530 (0.0009) [2023-10-10 18:02:40,518][123582] Updated weights for policy 0, policy_version 37603 (0.0009) [2023-10-10 18:02:40,917][123582] Updated weights for policy 0, policy_version 37613 (0.0010) [2023-10-10 18:02:41,303][123582] Updated weights for policy 0, policy_version 37623 (0.0009) [2023-10-10 18:02:43,413][123614] Updated weights for policy 1, policy_version 37540 (0.0008) [2023-10-10 18:02:43,781][123614] Updated weights for policy 1, policy_version 37550 (0.0008) [2023-10-10 18:02:43,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 76972032. Throughput: 0: 1829.2, 1: 1807.9. Samples: 19251130. Policy #0 lag: (min: 24.0, avg: 44.2, max: 56.0) [2023-10-10 18:02:43,788][122664] Avg episode reward: [(0, '43.730'), (1, '48.610')] [2023-10-10 18:02:44,145][123614] Updated weights for policy 1, policy_version 37560 (0.0009) [2023-10-10 18:02:45,035][123582] Updated weights for policy 0, policy_version 37633 (0.0011) [2023-10-10 18:02:45,405][123582] Updated weights for policy 0, policy_version 37643 (0.0010) [2023-10-10 18:02:45,775][123582] Updated weights for policy 0, policy_version 37653 (0.0009) [2023-10-10 18:02:46,150][123582] Updated weights for policy 0, policy_version 37663 (0.0010) [2023-10-10 18:02:47,850][123614] Updated weights for policy 1, policy_version 37570 (0.0010) [2023-10-10 18:02:48,217][123614] Updated weights for policy 1, policy_version 37580 (0.0009) [2023-10-10 18:02:48,590][123614] Updated weights for policy 1, policy_version 37590 (0.0009) [2023-10-10 18:02:48,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 77037568. Throughput: 0: 1837.0, 1: 1820.0. Samples: 19273698. Policy #0 lag: (min: 24.0, avg: 44.2, max: 56.0) [2023-10-10 18:02:48,788][122664] Avg episode reward: [(0, '45.160'), (1, '47.430')] [2023-10-10 18:02:48,951][123614] Updated weights for policy 1, policy_version 37600 (0.0009) [2023-10-10 18:02:49,926][123582] Updated weights for policy 0, policy_version 37673 (0.0010) [2023-10-10 18:02:50,296][123582] Updated weights for policy 0, policy_version 37683 (0.0010) [2023-10-10 18:02:50,672][123582] Updated weights for policy 0, policy_version 37693 (0.0010) [2023-10-10 18:02:52,805][123614] Updated weights for policy 1, policy_version 37610 (0.0008) [2023-10-10 18:02:53,181][123614] Updated weights for policy 1, policy_version 37620 (0.0009) [2023-10-10 18:02:53,557][123614] Updated weights for policy 1, policy_version 37630 (0.0007) [2023-10-10 18:02:53,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 77135872. Throughput: 0: 1830.5, 1: 1808.0. Samples: 19294598. Policy #0 lag: (min: 24.0, avg: 44.2, max: 56.0) [2023-10-10 18:02:53,788][122664] Avg episode reward: [(0, '48.270'), (1, '48.660')] [2023-10-10 18:02:54,413][123582] Updated weights for policy 0, policy_version 37703 (0.0011) [2023-10-10 18:02:54,782][123582] Updated weights for policy 0, policy_version 37713 (0.0007) [2023-10-10 18:02:55,158][123582] Updated weights for policy 0, policy_version 37723 (0.0007) [2023-10-10 18:02:57,208][123614] Updated weights for policy 1, policy_version 37640 (0.0010) [2023-10-10 18:02:57,569][123614] Updated weights for policy 1, policy_version 37650 (0.0008) [2023-10-10 18:02:57,943][123614] Updated weights for policy 1, policy_version 37660 (0.0008) [2023-10-10 18:02:58,775][123582] Updated weights for policy 0, policy_version 37733 (0.0008) [2023-10-10 18:02:58,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 77201408. Throughput: 0: 1831.0, 1: 1814.7. Samples: 19305914. Policy #0 lag: (min: 7.0, avg: 23.0, max: 39.0) [2023-10-10 18:02:58,789][122664] Avg episode reward: [(0, '49.420'), (1, '46.750')] [2023-10-10 18:02:59,156][123582] Updated weights for policy 0, policy_version 37743 (0.0008) [2023-10-10 18:02:59,521][123582] Updated weights for policy 0, policy_version 37753 (0.0008) [2023-10-10 18:03:01,590][123614] Updated weights for policy 1, policy_version 37670 (0.0008) [2023-10-10 18:03:01,957][123614] Updated weights for policy 1, policy_version 37680 (0.0010) [2023-10-10 18:03:02,325][123614] Updated weights for policy 1, policy_version 37690 (0.0010) [2023-10-10 18:03:03,035][123582] Updated weights for policy 0, policy_version 37763 (0.0008) [2023-10-10 18:03:03,405][123582] Updated weights for policy 0, policy_version 37773 (0.0009) [2023-10-10 18:03:03,769][123582] Updated weights for policy 0, policy_version 37783 (0.0008) [2023-10-10 18:03:03,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 77266944. Throughput: 0: 1824.7, 1: 1809.8. Samples: 19327466. Policy #0 lag: (min: 7.0, avg: 23.0, max: 39.0) [2023-10-10 18:03:03,789][122664] Avg episode reward: [(0, '46.490'), (1, '49.540')] [2023-10-10 18:03:06,089][123614] Updated weights for policy 1, policy_version 37700 (0.0011) [2023-10-10 18:03:06,457][123614] Updated weights for policy 1, policy_version 37710 (0.0009) [2023-10-10 18:03:06,833][123614] Updated weights for policy 1, policy_version 37720 (0.0008) [2023-10-10 18:03:07,393][123582] Updated weights for policy 0, policy_version 37793 (0.0008) [2023-10-10 18:03:07,775][123582] Updated weights for policy 0, policy_version 37803 (0.0010) [2023-10-10 18:03:08,141][123582] Updated weights for policy 0, policy_version 37813 (0.0009) [2023-10-10 18:03:08,518][123582] Updated weights for policy 0, policy_version 37823 (0.0007) [2023-10-10 18:03:08,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 77365248. Throughput: 0: 1825.9, 1: 1806.4. Samples: 19349080. Policy #0 lag: (min: 7.0, avg: 23.0, max: 39.0) [2023-10-10 18:03:08,789][122664] Avg episode reward: [(0, '44.920'), (1, '46.140')] [2023-10-10 18:03:10,536][123614] Updated weights for policy 1, policy_version 37730 (0.0008) [2023-10-10 18:03:10,904][123614] Updated weights for policy 1, policy_version 37740 (0.0008) [2023-10-10 18:03:11,275][123614] Updated weights for policy 1, policy_version 37750 (0.0007) [2023-10-10 18:03:11,642][123614] Updated weights for policy 1, policy_version 37760 (0.0007) [2023-10-10 18:03:12,376][123582] Updated weights for policy 0, policy_version 37833 (0.0012) [2023-10-10 18:03:12,749][123582] Updated weights for policy 0, policy_version 37843 (0.0010) [2023-10-10 18:03:13,119][123582] Updated weights for policy 0, policy_version 37853 (0.0011) [2023-10-10 18:03:13,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 77430784. Throughput: 0: 1821.4, 1: 1805.5. Samples: 19359962. Policy #0 lag: (min: 7.0, avg: 23.0, max: 39.0) [2023-10-10 18:03:13,789][122664] Avg episode reward: [(0, '43.210'), (1, '48.540')] [2023-10-10 18:03:15,315][123614] Updated weights for policy 1, policy_version 37770 (0.0009) [2023-10-10 18:03:15,681][123614] Updated weights for policy 1, policy_version 37780 (0.0007) [2023-10-10 18:03:16,043][123614] Updated weights for policy 1, policy_version 37790 (0.0007) [2023-10-10 18:03:16,818][123582] Updated weights for policy 0, policy_version 37863 (0.0008) [2023-10-10 18:03:17,189][123582] Updated weights for policy 0, policy_version 37873 (0.0010) [2023-10-10 18:03:17,556][123582] Updated weights for policy 0, policy_version 37883 (0.0009) [2023-10-10 18:03:18,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 77496320. Throughput: 0: 1821.2, 1: 1801.1. Samples: 19381504. Policy #0 lag: (min: 7.0, avg: 23.0, max: 39.0) [2023-10-10 18:03:18,789][122664] Avg episode reward: [(0, '43.480'), (1, '48.330')] [2023-10-10 18:03:19,862][123614] Updated weights for policy 1, policy_version 37800 (0.0009) [2023-10-10 18:03:20,231][123614] Updated weights for policy 1, policy_version 37810 (0.0010) [2023-10-10 18:03:20,598][123614] Updated weights for policy 1, policy_version 37820 (0.0010) [2023-10-10 18:03:21,295][123582] Updated weights for policy 0, policy_version 37893 (0.0007) [2023-10-10 18:03:21,666][123582] Updated weights for policy 0, policy_version 37903 (0.0008) [2023-10-10 18:03:22,047][123582] Updated weights for policy 0, policy_version 37913 (0.0009) [2023-10-10 18:03:23,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 77561856. Throughput: 0: 1810.8, 1: 1801.6. Samples: 19403560. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:03:23,789][122664] Avg episode reward: [(0, '40.670'), (1, '47.760')] [2023-10-10 18:03:24,209][123614] Updated weights for policy 1, policy_version 37830 (0.0009) [2023-10-10 18:03:24,577][123614] Updated weights for policy 1, policy_version 37840 (0.0011) [2023-10-10 18:03:24,952][123614] Updated weights for policy 1, policy_version 37850 (0.0011) [2023-10-10 18:03:25,813][123582] Updated weights for policy 0, policy_version 37923 (0.0007) [2023-10-10 18:03:26,229][123582] Updated weights for policy 0, policy_version 37933 (0.0007) [2023-10-10 18:03:26,600][123582] Updated weights for policy 0, policy_version 37943 (0.0008) [2023-10-10 18:03:28,727][123614] Updated weights for policy 1, policy_version 37860 (0.0008) [2023-10-10 18:03:28,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 77627392. Throughput: 0: 1816.4, 1: 1803.6. Samples: 19414026. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:03:28,788][122664] Avg episode reward: [(0, '43.490'), (1, '48.620')] [2023-10-10 18:03:29,094][123614] Updated weights for policy 1, policy_version 37870 (0.0007) [2023-10-10 18:03:29,464][123614] Updated weights for policy 1, policy_version 37880 (0.0008) [2023-10-10 18:03:30,293][123582] Updated weights for policy 0, policy_version 37953 (0.0008) [2023-10-10 18:03:30,672][123582] Updated weights for policy 0, policy_version 37963 (0.0008) [2023-10-10 18:03:31,045][123582] Updated weights for policy 0, policy_version 37973 (0.0008) [2023-10-10 18:03:31,424][123582] Updated weights for policy 0, policy_version 37983 (0.0008) [2023-10-10 18:03:33,237][123614] Updated weights for policy 1, policy_version 37890 (0.0007) [2023-10-10 18:03:33,607][123614] Updated weights for policy 1, policy_version 37900 (0.0007) [2023-10-10 18:03:33,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 77692928. Throughput: 0: 1807.4, 1: 1795.6. Samples: 19435832. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:03:33,788][122664] Avg episode reward: [(0, '43.620'), (1, '48.540')] [2023-10-10 18:03:33,979][123614] Updated weights for policy 1, policy_version 37910 (0.0007) [2023-10-10 18:03:34,346][123614] Updated weights for policy 1, policy_version 37920 (0.0007) [2023-10-10 18:03:35,159][123582] Updated weights for policy 0, policy_version 37993 (0.0009) [2023-10-10 18:03:35,527][123582] Updated weights for policy 0, policy_version 38003 (0.0008) [2023-10-10 18:03:35,901][123582] Updated weights for policy 0, policy_version 38013 (0.0008) [2023-10-10 18:03:38,074][123614] Updated weights for policy 1, policy_version 37930 (0.0010) [2023-10-10 18:03:38,452][123614] Updated weights for policy 1, policy_version 37940 (0.0008) [2023-10-10 18:03:38,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 77758464. Throughput: 0: 1811.6, 1: 1806.5. Samples: 19457416. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:03:38,789][122664] Avg episode reward: [(0, '43.450'), (1, '52.140')] [2023-10-10 18:03:38,820][123614] Updated weights for policy 1, policy_version 37950 (0.0008) [2023-10-10 18:03:39,486][123582] Updated weights for policy 0, policy_version 38023 (0.0010) [2023-10-10 18:03:39,848][123582] Updated weights for policy 0, policy_version 38033 (0.0008) [2023-10-10 18:03:40,217][123582] Updated weights for policy 0, policy_version 38043 (0.0009) [2023-10-10 18:03:42,585][123614] Updated weights for policy 1, policy_version 37960 (0.0007) [2023-10-10 18:03:42,959][123614] Updated weights for policy 1, policy_version 37970 (0.0009) [2023-10-10 18:03:43,336][123614] Updated weights for policy 1, policy_version 37980 (0.0007) [2023-10-10 18:03:43,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 77856768. Throughput: 0: 1811.2, 1: 1799.9. Samples: 19468416. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:03:43,789][122664] Avg episode reward: [(0, '39.650'), (1, '51.930')] [2023-10-10 18:03:43,919][123582] Updated weights for policy 0, policy_version 38053 (0.0008) [2023-10-10 18:03:44,299][123582] Updated weights for policy 0, policy_version 38063 (0.0008) [2023-10-10 18:03:44,664][123582] Updated weights for policy 0, policy_version 38073 (0.0010) [2023-10-10 18:03:47,052][123614] Updated weights for policy 1, policy_version 37990 (0.0008) [2023-10-10 18:03:47,415][123614] Updated weights for policy 1, policy_version 38000 (0.0007) [2023-10-10 18:03:47,785][123614] Updated weights for policy 1, policy_version 38010 (0.0008) [2023-10-10 18:03:48,331][123582] Updated weights for policy 0, policy_version 38083 (0.0012) [2023-10-10 18:03:48,703][123582] Updated weights for policy 0, policy_version 38093 (0.0007) [2023-10-10 18:03:48,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 77922304. Throughput: 0: 1805.2, 1: 1806.4. Samples: 19489990. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:03:48,789][122664] Avg episode reward: [(0, '38.920'), (1, '50.950')] [2023-10-10 18:03:49,075][123582] Updated weights for policy 0, policy_version 38103 (0.0008) [2023-10-10 18:03:51,505][123614] Updated weights for policy 1, policy_version 38020 (0.0008) [2023-10-10 18:03:51,884][123614] Updated weights for policy 1, policy_version 38030 (0.0007) [2023-10-10 18:03:52,253][123614] Updated weights for policy 1, policy_version 38040 (0.0009) [2023-10-10 18:03:52,750][123582] Updated weights for policy 0, policy_version 38113 (0.0008) [2023-10-10 18:03:53,113][123582] Updated weights for policy 0, policy_version 38123 (0.0009) [2023-10-10 18:03:53,494][123582] Updated weights for policy 0, policy_version 38133 (0.0009) [2023-10-10 18:03:53,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 77987840. Throughput: 0: 1812.7, 1: 1795.5. Samples: 19511446. Policy #0 lag: (min: 2.0, avg: 7.2, max: 34.0) [2023-10-10 18:03:53,789][122664] Avg episode reward: [(0, '39.190'), (1, '46.360')] [2023-10-10 18:03:53,799][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000038048_38961152.pth... [2023-10-10 18:03:53,832][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000036352_37224448.pth [2023-10-10 18:03:53,861][123582] Updated weights for policy 0, policy_version 38143 (0.0007) [2023-10-10 18:03:53,895][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000038144_39059456.pth... [2023-10-10 18:03:53,935][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000036416_37289984.pth [2023-10-10 18:03:55,937][123614] Updated weights for policy 1, policy_version 38050 (0.0009) [2023-10-10 18:03:56,307][123614] Updated weights for policy 1, policy_version 38060 (0.0007) [2023-10-10 18:03:56,679][123614] Updated weights for policy 1, policy_version 38070 (0.0008) [2023-10-10 18:03:57,042][123614] Updated weights for policy 1, policy_version 38080 (0.0007) [2023-10-10 18:03:57,457][123582] Updated weights for policy 0, policy_version 38153 (0.0007) [2023-10-10 18:03:57,820][123582] Updated weights for policy 0, policy_version 38163 (0.0009) [2023-10-10 18:03:58,200][123582] Updated weights for policy 0, policy_version 38173 (0.0010) [2023-10-10 18:03:58,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 78086144. Throughput: 0: 1812.0, 1: 1809.4. Samples: 19522926. Policy #0 lag: (min: 2.0, avg: 7.2, max: 34.0) [2023-10-10 18:03:58,788][122664] Avg episode reward: [(0, '43.070'), (1, '47.310')] [2023-10-10 18:04:00,800][123614] Updated weights for policy 1, policy_version 38090 (0.0007) [2023-10-10 18:04:01,166][123614] Updated weights for policy 1, policy_version 38100 (0.0008) [2023-10-10 18:04:01,539][123614] Updated weights for policy 1, policy_version 38110 (0.0008) [2023-10-10 18:04:01,826][123582] Updated weights for policy 0, policy_version 38183 (0.0009) [2023-10-10 18:04:02,189][123582] Updated weights for policy 0, policy_version 38193 (0.0008) [2023-10-10 18:04:02,562][123582] Updated weights for policy 0, policy_version 38203 (0.0008) [2023-10-10 18:04:03,788][122664] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 78151680. Throughput: 0: 1818.7, 1: 1800.6. Samples: 19544372. Policy #0 lag: (min: 2.0, avg: 7.2, max: 34.0) [2023-10-10 18:04:03,789][122664] Avg episode reward: [(0, '43.030'), (1, '45.130')] [2023-10-10 18:04:05,181][123614] Updated weights for policy 1, policy_version 38120 (0.0008) [2023-10-10 18:04:05,548][123614] Updated weights for policy 1, policy_version 38130 (0.0007) [2023-10-10 18:04:05,913][123614] Updated weights for policy 1, policy_version 38140 (0.0007) [2023-10-10 18:04:06,220][123582] Updated weights for policy 0, policy_version 38213 (0.0010) [2023-10-10 18:04:06,587][123582] Updated weights for policy 0, policy_version 38223 (0.0007) [2023-10-10 18:04:06,963][123582] Updated weights for policy 0, policy_version 38233 (0.0008) [2023-10-10 18:04:08,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 78217216. Throughput: 0: 1827.3, 1: 1806.0. Samples: 19567060. Policy #0 lag: (min: 2.0, avg: 7.2, max: 34.0) [2023-10-10 18:04:08,789][122664] Avg episode reward: [(0, '41.060'), (1, '47.360')] [2023-10-10 18:04:09,581][123614] Updated weights for policy 1, policy_version 38150 (0.0009) [2023-10-10 18:04:09,952][123614] Updated weights for policy 1, policy_version 38160 (0.0010) [2023-10-10 18:04:10,316][123614] Updated weights for policy 1, policy_version 38170 (0.0009) [2023-10-10 18:04:10,601][123582] Updated weights for policy 0, policy_version 38243 (0.0009) [2023-10-10 18:04:11,005][123582] Updated weights for policy 0, policy_version 38253 (0.0008) [2023-10-10 18:04:11,382][123582] Updated weights for policy 0, policy_version 38263 (0.0007) [2023-10-10 18:04:13,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 78282752. Throughput: 0: 1825.0, 1: 1804.3. Samples: 19577346. Policy #0 lag: (min: 2.0, avg: 7.2, max: 34.0) [2023-10-10 18:04:13,789][122664] Avg episode reward: [(0, '42.860'), (1, '46.240')] [2023-10-10 18:04:14,072][123614] Updated weights for policy 1, policy_version 38180 (0.0008) [2023-10-10 18:04:14,445][123614] Updated weights for policy 1, policy_version 38190 (0.0008) [2023-10-10 18:04:14,810][123614] Updated weights for policy 1, policy_version 38200 (0.0008) [2023-10-10 18:04:15,030][123582] Updated weights for policy 0, policy_version 38273 (0.0008) [2023-10-10 18:04:15,410][123582] Updated weights for policy 0, policy_version 38283 (0.0009) [2023-10-10 18:04:15,786][123582] Updated weights for policy 0, policy_version 38293 (0.0007) [2023-10-10 18:04:16,161][123582] Updated weights for policy 0, policy_version 38303 (0.0007) [2023-10-10 18:04:18,432][123614] Updated weights for policy 1, policy_version 38210 (0.0009) [2023-10-10 18:04:18,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 78348288. Throughput: 0: 1835.6, 1: 1812.8. Samples: 19600012. Policy #0 lag: (min: 31.0, avg: 41.7, max: 63.0) [2023-10-10 18:04:18,789][122664] Avg episode reward: [(0, '45.710'), (1, '47.980')] [2023-10-10 18:04:18,802][123614] Updated weights for policy 1, policy_version 38220 (0.0007) [2023-10-10 18:04:19,169][123614] Updated weights for policy 1, policy_version 38230 (0.0009) [2023-10-10 18:04:19,537][123614] Updated weights for policy 1, policy_version 38240 (0.0010) [2023-10-10 18:04:19,820][123582] Updated weights for policy 0, policy_version 38313 (0.0009) [2023-10-10 18:04:20,191][123582] Updated weights for policy 0, policy_version 38323 (0.0008) [2023-10-10 18:04:20,558][123582] Updated weights for policy 0, policy_version 38333 (0.0009) [2023-10-10 18:04:23,240][123614] Updated weights for policy 1, policy_version 38250 (0.0008) [2023-10-10 18:04:23,615][123614] Updated weights for policy 1, policy_version 38260 (0.0008) [2023-10-10 18:04:23,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 78413824. Throughput: 0: 1835.2, 1: 1817.8. Samples: 19621800. Policy #0 lag: (min: 31.0, avg: 41.7, max: 63.0) [2023-10-10 18:04:23,789][122664] Avg episode reward: [(0, '44.860'), (1, '49.490')] [2023-10-10 18:04:23,989][123614] Updated weights for policy 1, policy_version 38270 (0.0008) [2023-10-10 18:04:24,132][123582] Updated weights for policy 0, policy_version 38343 (0.0008) [2023-10-10 18:04:24,510][123582] Updated weights for policy 0, policy_version 38353 (0.0008) [2023-10-10 18:04:24,879][123582] Updated weights for policy 0, policy_version 38363 (0.0009) [2023-10-10 18:04:27,788][123614] Updated weights for policy 1, policy_version 38280 (0.0007) [2023-10-10 18:04:28,153][123614] Updated weights for policy 1, policy_version 38290 (0.0010) [2023-10-10 18:04:28,488][123582] Updated weights for policy 0, policy_version 38373 (0.0009) [2023-10-10 18:04:28,511][123614] Updated weights for policy 1, policy_version 38300 (0.0008) [2023-10-10 18:04:28,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 78512128. Throughput: 0: 1837.2, 1: 1813.6. Samples: 19632700. Policy #0 lag: (min: 31.0, avg: 41.7, max: 63.0) [2023-10-10 18:04:28,788][122664] Avg episode reward: [(0, '42.010'), (1, '49.780')] [2023-10-10 18:04:28,861][123582] Updated weights for policy 0, policy_version 38383 (0.0008) [2023-10-10 18:04:29,222][123582] Updated weights for policy 0, policy_version 38393 (0.0008) [2023-10-10 18:04:32,307][123614] Updated weights for policy 1, policy_version 38310 (0.0007) [2023-10-10 18:04:32,677][123614] Updated weights for policy 1, policy_version 38320 (0.0008) [2023-10-10 18:04:32,813][123582] Updated weights for policy 0, policy_version 38403 (0.0008) [2023-10-10 18:04:33,047][123614] Updated weights for policy 1, policy_version 38330 (0.0007) [2023-10-10 18:04:33,179][123582] Updated weights for policy 0, policy_version 38413 (0.0008) [2023-10-10 18:04:33,554][123582] Updated weights for policy 0, policy_version 38423 (0.0009) [2023-10-10 18:04:33,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 78577664. Throughput: 0: 1844.5, 1: 1822.4. Samples: 19655002. Policy #0 lag: (min: 31.0, avg: 41.7, max: 63.0) [2023-10-10 18:04:33,788][122664] Avg episode reward: [(0, '42.200'), (1, '45.890')] [2023-10-10 18:04:36,537][123614] Updated weights for policy 1, policy_version 38340 (0.0008) [2023-10-10 18:04:36,910][123614] Updated weights for policy 1, policy_version 38350 (0.0008) [2023-10-10 18:04:37,179][123582] Updated weights for policy 0, policy_version 38433 (0.0008) [2023-10-10 18:04:37,287][123614] Updated weights for policy 1, policy_version 38360 (0.0008) [2023-10-10 18:04:37,546][123582] Updated weights for policy 0, policy_version 38443 (0.0007) [2023-10-10 18:04:37,920][123582] Updated weights for policy 0, policy_version 38453 (0.0007) [2023-10-10 18:04:38,291][123582] Updated weights for policy 0, policy_version 38463 (0.0009) [2023-10-10 18:04:38,788][122664] Fps is (10 sec: 16383.8, 60 sec: 15291.8, 300 sec: 14551.2). Total num frames: 78675968. Throughput: 0: 1828.0, 1: 1820.6. Samples: 19675632. Policy #0 lag: (min: 31.0, avg: 41.7, max: 63.0) [2023-10-10 18:04:38,788][122664] Avg episode reward: [(0, '36.810'), (1, '45.360')] [2023-10-10 18:04:41,213][123614] Updated weights for policy 1, policy_version 38370 (0.0009) [2023-10-10 18:04:41,576][123614] Updated weights for policy 1, policy_version 38380 (0.0009) [2023-10-10 18:04:41,954][123614] Updated weights for policy 1, policy_version 38390 (0.0009) [2023-10-10 18:04:42,146][123582] Updated weights for policy 0, policy_version 38473 (0.0009) [2023-10-10 18:04:42,327][123614] Updated weights for policy 1, policy_version 38400 (0.0008) [2023-10-10 18:04:42,520][123582] Updated weights for policy 0, policy_version 38483 (0.0008) [2023-10-10 18:04:42,895][123582] Updated weights for policy 0, policy_version 38493 (0.0009) [2023-10-10 18:04:43,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 78741504. Throughput: 0: 1836.5, 1: 1825.3. Samples: 19687706. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-10 18:04:43,789][122664] Avg episode reward: [(0, '35.570'), (1, '45.690')] [2023-10-10 18:04:45,882][123614] Updated weights for policy 1, policy_version 38410 (0.0009) [2023-10-10 18:04:46,252][123614] Updated weights for policy 1, policy_version 38420 (0.0009) [2023-10-10 18:04:46,622][123582] Updated weights for policy 0, policy_version 38503 (0.0009) [2023-10-10 18:04:46,627][123614] Updated weights for policy 1, policy_version 38430 (0.0008) [2023-10-10 18:04:46,994][123582] Updated weights for policy 0, policy_version 38513 (0.0008) [2023-10-10 18:04:47,364][123582] Updated weights for policy 0, policy_version 38523 (0.0009) [2023-10-10 18:04:48,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 78807040. Throughput: 0: 1829.1, 1: 1816.4. Samples: 19708420. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-10 18:04:48,789][122664] Avg episode reward: [(0, '37.660'), (1, '49.690')] [2023-10-10 18:04:50,303][123614] Updated weights for policy 1, policy_version 38440 (0.0008) [2023-10-10 18:04:50,670][123614] Updated weights for policy 1, policy_version 38450 (0.0007) [2023-10-10 18:04:51,033][123614] Updated weights for policy 1, policy_version 38460 (0.0008) [2023-10-10 18:04:51,103][123582] Updated weights for policy 0, policy_version 38533 (0.0009) [2023-10-10 18:04:51,476][123582] Updated weights for policy 0, policy_version 38543 (0.0007) [2023-10-10 18:04:51,855][123582] Updated weights for policy 0, policy_version 38553 (0.0008) [2023-10-10 18:04:53,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 78872576. Throughput: 0: 1820.3, 1: 1812.4. Samples: 19730528. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-10 18:04:53,789][122664] Avg episode reward: [(0, '40.240'), (1, '46.210')] [2023-10-10 18:04:54,660][123614] Updated weights for policy 1, policy_version 38470 (0.0008) [2023-10-10 18:04:55,030][123614] Updated weights for policy 1, policy_version 38480 (0.0009) [2023-10-10 18:04:55,401][123614] Updated weights for policy 1, policy_version 38490 (0.0009) [2023-10-10 18:04:55,539][123582] Updated weights for policy 0, policy_version 38563 (0.0008) [2023-10-10 18:04:55,915][123582] Updated weights for policy 0, policy_version 38573 (0.0009) [2023-10-10 18:04:56,280][123582] Updated weights for policy 0, policy_version 38583 (0.0009) [2023-10-10 18:04:58,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 78938112. Throughput: 0: 1822.5, 1: 1814.3. Samples: 19741002. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-10 18:04:58,789][122664] Avg episode reward: [(0, '41.840'), (1, '49.000')] [2023-10-10 18:04:59,018][123614] Updated weights for policy 1, policy_version 38500 (0.0009) [2023-10-10 18:04:59,383][123614] Updated weights for policy 1, policy_version 38510 (0.0009) [2023-10-10 18:04:59,760][123614] Updated weights for policy 1, policy_version 38520 (0.0008) [2023-10-10 18:04:59,913][123582] Updated weights for policy 0, policy_version 38593 (0.0009) [2023-10-10 18:05:00,287][123582] Updated weights for policy 0, policy_version 38603 (0.0010) [2023-10-10 18:05:00,666][123582] Updated weights for policy 0, policy_version 38613 (0.0010) [2023-10-10 18:05:01,032][123582] Updated weights for policy 0, policy_version 38623 (0.0010) [2023-10-10 18:05:03,513][123614] Updated weights for policy 1, policy_version 38530 (0.0008) [2023-10-10 18:05:03,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 79003648. Throughput: 0: 1816.5, 1: 1810.2. Samples: 19763214. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-10 18:05:03,788][122664] Avg episode reward: [(0, '39.270'), (1, '47.180')] [2023-10-10 18:05:03,887][123614] Updated weights for policy 1, policy_version 38540 (0.0008) [2023-10-10 18:05:04,251][123614] Updated weights for policy 1, policy_version 38550 (0.0007) [2023-10-10 18:05:04,611][123614] Updated weights for policy 1, policy_version 38560 (0.0008) [2023-10-10 18:05:04,996][123582] Updated weights for policy 0, policy_version 38633 (0.0010) [2023-10-10 18:05:05,374][123582] Updated weights for policy 0, policy_version 38643 (0.0010) [2023-10-10 18:05:05,742][123582] Updated weights for policy 0, policy_version 38653 (0.0009) [2023-10-10 18:05:08,433][123614] Updated weights for policy 1, policy_version 38570 (0.0009) [2023-10-10 18:05:08,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 79069184. Throughput: 0: 1812.8, 1: 1814.1. Samples: 19785012. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-10 18:05:08,789][122664] Avg episode reward: [(0, '40.140'), (1, '46.270')] [2023-10-10 18:05:08,817][123614] Updated weights for policy 1, policy_version 38580 (0.0008) [2023-10-10 18:05:09,185][123614] Updated weights for policy 1, policy_version 38590 (0.0009) [2023-10-10 18:05:09,349][123582] Updated weights for policy 0, policy_version 38663 (0.0009) [2023-10-10 18:05:09,718][123582] Updated weights for policy 0, policy_version 38673 (0.0009) [2023-10-10 18:05:10,098][123582] Updated weights for policy 0, policy_version 38683 (0.0010) [2023-10-10 18:05:12,897][123614] Updated weights for policy 1, policy_version 38600 (0.0010) [2023-10-10 18:05:13,272][123614] Updated weights for policy 1, policy_version 38610 (0.0009) [2023-10-10 18:05:13,641][123614] Updated weights for policy 1, policy_version 38620 (0.0007) [2023-10-10 18:05:13,647][123582] Updated weights for policy 0, policy_version 38693 (0.0008) [2023-10-10 18:05:13,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 79167488. Throughput: 0: 1811.9, 1: 1808.4. Samples: 19795616. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-10 18:05:13,789][122664] Avg episode reward: [(0, '41.470'), (1, '44.040')] [2023-10-10 18:05:14,025][123582] Updated weights for policy 0, policy_version 38703 (0.0008) [2023-10-10 18:05:14,386][123582] Updated weights for policy 0, policy_version 38713 (0.0009) [2023-10-10 18:05:17,330][123614] Updated weights for policy 1, policy_version 38630 (0.0007) [2023-10-10 18:05:17,694][123614] Updated weights for policy 1, policy_version 38640 (0.0008) [2023-10-10 18:05:17,959][123582] Updated weights for policy 0, policy_version 38723 (0.0009) [2023-10-10 18:05:18,069][123614] Updated weights for policy 1, policy_version 38650 (0.0008) [2023-10-10 18:05:18,329][123582] Updated weights for policy 0, policy_version 38733 (0.0009) [2023-10-10 18:05:18,706][123582] Updated weights for policy 0, policy_version 38743 (0.0008) [2023-10-10 18:05:18,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 79233024. Throughput: 0: 1810.4, 1: 1811.8. Samples: 19818002. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-10 18:05:18,788][122664] Avg episode reward: [(0, '42.140'), (1, '43.390')] [2023-10-10 18:05:21,914][123614] Updated weights for policy 1, policy_version 38660 (0.0007) [2023-10-10 18:05:22,275][123614] Updated weights for policy 1, policy_version 38670 (0.0007) [2023-10-10 18:05:22,540][123582] Updated weights for policy 0, policy_version 38753 (0.0007) [2023-10-10 18:05:22,653][123614] Updated weights for policy 1, policy_version 38680 (0.0007) [2023-10-10 18:05:22,900][123582] Updated weights for policy 0, policy_version 38763 (0.0009) [2023-10-10 18:05:23,271][123582] Updated weights for policy 0, policy_version 38773 (0.0010) [2023-10-10 18:05:23,640][123582] Updated weights for policy 0, policy_version 38783 (0.0011) [2023-10-10 18:05:23,788][122664] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14551.2). Total num frames: 79331328. Throughput: 0: 1813.6, 1: 1807.0. Samples: 19838556. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-10 18:05:23,789][122664] Avg episode reward: [(0, '42.400'), (1, '42.930')] [2023-10-10 18:05:26,281][123614] Updated weights for policy 1, policy_version 38690 (0.0008) [2023-10-10 18:05:26,646][123614] Updated weights for policy 1, policy_version 38700 (0.0011) [2023-10-10 18:05:27,011][123614] Updated weights for policy 1, policy_version 38710 (0.0008) [2023-10-10 18:05:27,337][123582] Updated weights for policy 0, policy_version 38793 (0.0010) [2023-10-10 18:05:27,376][123614] Updated weights for policy 1, policy_version 38720 (0.0008) [2023-10-10 18:05:27,711][123582] Updated weights for policy 0, policy_version 38803 (0.0009) [2023-10-10 18:05:28,093][123582] Updated weights for policy 0, policy_version 38813 (0.0010) [2023-10-10 18:05:28,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 79396864. Throughput: 0: 1807.7, 1: 1809.0. Samples: 19850460. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-10 18:05:28,789][122664] Avg episode reward: [(0, '43.740'), (1, '42.730')] [2023-10-10 18:05:31,291][123614] Updated weights for policy 1, policy_version 38730 (0.0008) [2023-10-10 18:05:31,656][123614] Updated weights for policy 1, policy_version 38740 (0.0009) [2023-10-10 18:05:31,909][123582] Updated weights for policy 0, policy_version 38823 (0.0009) [2023-10-10 18:05:32,017][123614] Updated weights for policy 1, policy_version 38750 (0.0007) [2023-10-10 18:05:32,276][123582] Updated weights for policy 0, policy_version 38833 (0.0008) [2023-10-10 18:05:32,660][123582] Updated weights for policy 0, policy_version 38843 (0.0008) [2023-10-10 18:05:33,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 79462400. Throughput: 0: 1809.2, 1: 1801.2. Samples: 19870886. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-10 18:05:33,788][122664] Avg episode reward: [(0, '46.800'), (1, '42.610')] [2023-10-10 18:05:35,635][123614] Updated weights for policy 1, policy_version 38760 (0.0008) [2023-10-10 18:05:35,999][123614] Updated weights for policy 1, policy_version 38770 (0.0007) [2023-10-10 18:05:36,328][123582] Updated weights for policy 0, policy_version 38853 (0.0008) [2023-10-10 18:05:36,363][123614] Updated weights for policy 1, policy_version 38780 (0.0007) [2023-10-10 18:05:36,708][123582] Updated weights for policy 0, policy_version 38863 (0.0012) [2023-10-10 18:05:37,075][123582] Updated weights for policy 0, policy_version 38873 (0.0011) [2023-10-10 18:05:38,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 79527936. Throughput: 0: 1811.6, 1: 1809.6. Samples: 19893482. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-10 18:05:38,789][122664] Avg episode reward: [(0, '47.100'), (1, '43.630')] [2023-10-10 18:05:39,753][123614] Updated weights for policy 1, policy_version 38790 (0.0008) [2023-10-10 18:05:40,125][123614] Updated weights for policy 1, policy_version 38800 (0.0010) [2023-10-10 18:05:40,507][123614] Updated weights for policy 1, policy_version 38810 (0.0009) [2023-10-10 18:05:40,665][123582] Updated weights for policy 0, policy_version 38883 (0.0008) [2023-10-10 18:05:41,057][123582] Updated weights for policy 0, policy_version 38893 (0.0008) [2023-10-10 18:05:41,426][123582] Updated weights for policy 0, policy_version 38903 (0.0007) [2023-10-10 18:05:43,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 79593472. Throughput: 0: 1812.3, 1: 1809.3. Samples: 19903976. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-10 18:05:43,788][122664] Avg episode reward: [(0, '47.010'), (1, '43.380')] [2023-10-10 18:05:44,350][123614] Updated weights for policy 1, policy_version 38820 (0.0009) [2023-10-10 18:05:44,729][123614] Updated weights for policy 1, policy_version 38830 (0.0010) [2023-10-10 18:05:45,002][123582] Updated weights for policy 0, policy_version 38913 (0.0007) [2023-10-10 18:05:45,096][123614] Updated weights for policy 1, policy_version 38840 (0.0007) [2023-10-10 18:05:45,365][123582] Updated weights for policy 0, policy_version 38923 (0.0008) [2023-10-10 18:05:45,749][123582] Updated weights for policy 0, policy_version 38933 (0.0010) [2023-10-10 18:05:46,120][123582] Updated weights for policy 0, policy_version 38943 (0.0009) [2023-10-10 18:05:48,646][123614] Updated weights for policy 1, policy_version 38850 (0.0007) [2023-10-10 18:05:48,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 79659008. Throughput: 0: 1813.3, 1: 1810.0. Samples: 19926262. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-10 18:05:48,788][122664] Avg episode reward: [(0, '42.150'), (1, '44.480')] [2023-10-10 18:05:49,022][123614] Updated weights for policy 1, policy_version 38860 (0.0007) [2023-10-10 18:05:49,391][123614] Updated weights for policy 1, policy_version 38870 (0.0007) [2023-10-10 18:05:49,762][123614] Updated weights for policy 1, policy_version 38880 (0.0009) [2023-10-10 18:05:49,971][123582] Updated weights for policy 0, policy_version 38953 (0.0008) [2023-10-10 18:05:50,341][123582] Updated weights for policy 0, policy_version 38963 (0.0007) [2023-10-10 18:05:50,710][123582] Updated weights for policy 0, policy_version 38973 (0.0007) [2023-10-10 18:05:53,669][123614] Updated weights for policy 1, policy_version 38890 (0.0007) [2023-10-10 18:05:53,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 79724544. Throughput: 0: 1811.1, 1: 1810.8. Samples: 19947996. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-10 18:05:53,789][122664] Avg episode reward: [(0, '41.250'), (1, '49.460')] [2023-10-10 18:05:53,801][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000038976_39911424.pth... [2023-10-10 18:05:53,836][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000037280_38174720.pth [2023-10-10 18:05:54,042][123614] Updated weights for policy 1, policy_version 38900 (0.0007) [2023-10-10 18:05:54,384][123582] Updated weights for policy 0, policy_version 38983 (0.0008) [2023-10-10 18:05:54,404][123614] Updated weights for policy 1, policy_version 38910 (0.0007) [2023-10-10 18:05:54,475][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000038912_39845888.pth... [2023-10-10 18:05:54,508][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000037216_38109184.pth [2023-10-10 18:05:54,762][123582] Updated weights for policy 0, policy_version 38993 (0.0010) [2023-10-10 18:05:55,142][123582] Updated weights for policy 0, policy_version 39003 (0.0010) [2023-10-10 18:05:58,056][123614] Updated weights for policy 1, policy_version 38920 (0.0008) [2023-10-10 18:05:58,430][123614] Updated weights for policy 1, policy_version 38930 (0.0008) [2023-10-10 18:05:58,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 79790080. Throughput: 0: 1805.9, 1: 1810.8. Samples: 19958366. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-10 18:05:58,789][122664] Avg episode reward: [(0, '41.450'), (1, '48.170')] [2023-10-10 18:05:58,804][123614] Updated weights for policy 1, policy_version 38940 (0.0010) [2023-10-10 18:05:59,016][123582] Updated weights for policy 0, policy_version 39013 (0.0009) [2023-10-10 18:05:59,391][123582] Updated weights for policy 0, policy_version 39023 (0.0007) [2023-10-10 18:05:59,764][123582] Updated weights for policy 0, policy_version 39033 (0.0008) [2023-10-10 18:06:02,292][123614] Updated weights for policy 1, policy_version 38950 (0.0010) [2023-10-10 18:06:02,660][123614] Updated weights for policy 1, policy_version 38960 (0.0007) [2023-10-10 18:06:03,033][123614] Updated weights for policy 1, policy_version 38970 (0.0011) [2023-10-10 18:06:03,490][123582] Updated weights for policy 0, policy_version 39043 (0.0010) [2023-10-10 18:06:03,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 79888384. Throughput: 0: 1794.0, 1: 1813.2. Samples: 19980330. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-10 18:06:03,789][122664] Avg episode reward: [(0, '42.500'), (1, '49.160')] [2023-10-10 18:06:03,867][123582] Updated weights for policy 0, policy_version 39053 (0.0009) [2023-10-10 18:06:04,241][123582] Updated weights for policy 0, policy_version 39063 (0.0010) [2023-10-10 18:06:06,595][123614] Updated weights for policy 1, policy_version 38980 (0.0008) [2023-10-10 18:06:06,964][123614] Updated weights for policy 1, policy_version 38990 (0.0008) [2023-10-10 18:06:07,331][123614] Updated weights for policy 1, policy_version 39000 (0.0007) [2023-10-10 18:06:07,993][123582] Updated weights for policy 0, policy_version 39073 (0.0009) [2023-10-10 18:06:08,355][123582] Updated weights for policy 0, policy_version 39083 (0.0009) [2023-10-10 18:06:08,731][123582] Updated weights for policy 0, policy_version 39093 (0.0009) [2023-10-10 18:06:08,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 79953920. Throughput: 0: 1811.1, 1: 1820.7. Samples: 20001988. Policy #0 lag: (min: 14.0, avg: 21.2, max: 46.0) [2023-10-10 18:06:08,788][122664] Avg episode reward: [(0, '44.070'), (1, '48.880')] [2023-10-10 18:06:09,105][123582] Updated weights for policy 0, policy_version 39103 (0.0009) [2023-10-10 18:06:11,097][123614] Updated weights for policy 1, policy_version 39010 (0.0008) [2023-10-10 18:06:11,465][123614] Updated weights for policy 1, policy_version 39020 (0.0011) [2023-10-10 18:06:11,835][123614] Updated weights for policy 1, policy_version 39030 (0.0011) [2023-10-10 18:06:12,194][123614] Updated weights for policy 1, policy_version 39040 (0.0010) [2023-10-10 18:06:12,936][123582] Updated weights for policy 0, policy_version 39113 (0.0009) [2023-10-10 18:06:13,308][123582] Updated weights for policy 0, policy_version 39123 (0.0008) [2023-10-10 18:06:13,682][123582] Updated weights for policy 0, policy_version 39133 (0.0007) [2023-10-10 18:06:13,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 80019456. Throughput: 0: 1796.8, 1: 1814.8. Samples: 20012980. Policy #0 lag: (min: 14.0, avg: 21.2, max: 46.0) [2023-10-10 18:06:13,788][122664] Avg episode reward: [(0, '45.900'), (1, '52.540')] [2023-10-10 18:06:13,789][123465] Saving new best policy, reward=52.540! [2023-10-10 18:06:16,006][123614] Updated weights for policy 1, policy_version 39050 (0.0010) [2023-10-10 18:06:16,378][123614] Updated weights for policy 1, policy_version 39060 (0.0009) [2023-10-10 18:06:16,748][123614] Updated weights for policy 1, policy_version 39070 (0.0008) [2023-10-10 18:06:17,221][123582] Updated weights for policy 0, policy_version 39143 (0.0008) [2023-10-10 18:06:17,589][123582] Updated weights for policy 0, policy_version 39153 (0.0009) [2023-10-10 18:06:17,960][123582] Updated weights for policy 0, policy_version 39163 (0.0008) [2023-10-10 18:06:18,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 80117760. Throughput: 0: 1820.0, 1: 1823.0. Samples: 20034820. Policy #0 lag: (min: 14.0, avg: 21.2, max: 46.0) [2023-10-10 18:06:18,789][122664] Avg episode reward: [(0, '46.760'), (1, '55.510')] [2023-10-10 18:06:18,791][123465] Saving new best policy, reward=55.510! [2023-10-10 18:06:20,667][123614] Updated weights for policy 1, policy_version 39080 (0.0008) [2023-10-10 18:06:21,040][123614] Updated weights for policy 1, policy_version 39090 (0.0008) [2023-10-10 18:06:21,408][123614] Updated weights for policy 1, policy_version 39100 (0.0009) [2023-10-10 18:06:21,780][123582] Updated weights for policy 0, policy_version 39173 (0.0009) [2023-10-10 18:06:22,156][123582] Updated weights for policy 0, policy_version 39183 (0.0012) [2023-10-10 18:06:22,527][123582] Updated weights for policy 0, policy_version 39193 (0.0007) [2023-10-10 18:06:23,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 80183296. Throughput: 0: 1809.1, 1: 1817.2. Samples: 20056664. Policy #0 lag: (min: 14.0, avg: 21.2, max: 46.0) [2023-10-10 18:06:23,789][122664] Avg episode reward: [(0, '48.130'), (1, '57.330')] [2023-10-10 18:06:23,802][123465] Saving new best policy, reward=57.330! [2023-10-10 18:06:24,936][123614] Updated weights for policy 1, policy_version 39110 (0.0007) [2023-10-10 18:06:25,308][123614] Updated weights for policy 1, policy_version 39120 (0.0007) [2023-10-10 18:06:25,682][123614] Updated weights for policy 1, policy_version 39130 (0.0007) [2023-10-10 18:06:26,183][123582] Updated weights for policy 0, policy_version 39203 (0.0009) [2023-10-10 18:06:26,570][123582] Updated weights for policy 0, policy_version 39213 (0.0008) [2023-10-10 18:06:26,933][123582] Updated weights for policy 0, policy_version 39223 (0.0007) [2023-10-10 18:06:28,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 80248832. Throughput: 0: 1821.9, 1: 1815.8. Samples: 20067674. Policy #0 lag: (min: 14.0, avg: 21.2, max: 46.0) [2023-10-10 18:06:28,789][122664] Avg episode reward: [(0, '48.520'), (1, '59.830')] [2023-10-10 18:06:28,790][123465] Saving new best policy, reward=59.830! [2023-10-10 18:06:29,293][123614] Updated weights for policy 1, policy_version 39140 (0.0008) [2023-10-10 18:06:29,656][123614] Updated weights for policy 1, policy_version 39150 (0.0010) [2023-10-10 18:06:30,027][123614] Updated weights for policy 1, policy_version 39160 (0.0008) [2023-10-10 18:06:30,658][123582] Updated weights for policy 0, policy_version 39233 (0.0009) [2023-10-10 18:06:31,026][123582] Updated weights for policy 0, policy_version 39243 (0.0007) [2023-10-10 18:06:31,407][123582] Updated weights for policy 0, policy_version 39253 (0.0009) [2023-10-10 18:06:31,776][123582] Updated weights for policy 0, policy_version 39263 (0.0009) [2023-10-10 18:06:33,763][123614] Updated weights for policy 1, policy_version 39170 (0.0010) [2023-10-10 18:06:33,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 80314368. Throughput: 0: 1808.2, 1: 1819.4. Samples: 20089504. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-10 18:06:33,788][122664] Avg episode reward: [(0, '46.990'), (1, '59.540')] [2023-10-10 18:06:34,122][123614] Updated weights for policy 1, policy_version 39180 (0.0008) [2023-10-10 18:06:34,498][123614] Updated weights for policy 1, policy_version 39190 (0.0008) [2023-10-10 18:06:34,864][123614] Updated weights for policy 1, policy_version 39200 (0.0010) [2023-10-10 18:06:35,439][123582] Updated weights for policy 0, policy_version 39273 (0.0009) [2023-10-10 18:06:35,817][123582] Updated weights for policy 0, policy_version 39283 (0.0010) [2023-10-10 18:06:36,196][123582] Updated weights for policy 0, policy_version 39293 (0.0009) [2023-10-10 18:06:38,665][123614] Updated weights for policy 1, policy_version 39210 (0.0007) [2023-10-10 18:06:38,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 80379904. Throughput: 0: 1810.4, 1: 1823.8. Samples: 20111536. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-10 18:06:38,788][122664] Avg episode reward: [(0, '48.050'), (1, '59.720')] [2023-10-10 18:06:39,040][123614] Updated weights for policy 1, policy_version 39220 (0.0008) [2023-10-10 18:06:39,409][123614] Updated weights for policy 1, policy_version 39230 (0.0007) [2023-10-10 18:06:39,721][123582] Updated weights for policy 0, policy_version 39303 (0.0009) [2023-10-10 18:06:40,101][123582] Updated weights for policy 0, policy_version 39313 (0.0009) [2023-10-10 18:06:40,462][123582] Updated weights for policy 0, policy_version 39323 (0.0008) [2023-10-10 18:06:42,910][123614] Updated weights for policy 1, policy_version 39240 (0.0009) [2023-10-10 18:06:43,282][123614] Updated weights for policy 1, policy_version 39250 (0.0007) [2023-10-10 18:06:43,655][123614] Updated weights for policy 1, policy_version 39260 (0.0010) [2023-10-10 18:06:43,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 80445440. Throughput: 0: 1819.5, 1: 1822.6. Samples: 20122262. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-10 18:06:43,789][122664] Avg episode reward: [(0, '48.130'), (1, '59.690')] [2023-10-10 18:06:44,165][123582] Updated weights for policy 0, policy_version 39333 (0.0007) [2023-10-10 18:06:44,536][123582] Updated weights for policy 0, policy_version 39343 (0.0010) [2023-10-10 18:06:44,903][123582] Updated weights for policy 0, policy_version 39353 (0.0008) [2023-10-10 18:06:47,477][123614] Updated weights for policy 1, policy_version 39270 (0.0008) [2023-10-10 18:06:47,846][123614] Updated weights for policy 1, policy_version 39280 (0.0011) [2023-10-10 18:06:48,212][123614] Updated weights for policy 1, policy_version 39290 (0.0010) [2023-10-10 18:06:48,655][123582] Updated weights for policy 0, policy_version 39363 (0.0007) [2023-10-10 18:06:48,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 80543744. Throughput: 0: 1828.4, 1: 1821.3. Samples: 20144566. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-10 18:06:48,789][122664] Avg episode reward: [(0, '47.430'), (1, '60.070')] [2023-10-10 18:06:48,791][123465] Saving new best policy, reward=60.070! [2023-10-10 18:06:49,025][123582] Updated weights for policy 0, policy_version 39373 (0.0008) [2023-10-10 18:06:49,397][123582] Updated weights for policy 0, policy_version 39383 (0.0009) [2023-10-10 18:06:51,970][123614] Updated weights for policy 1, policy_version 39300 (0.0007) [2023-10-10 18:06:52,343][123614] Updated weights for policy 1, policy_version 39310 (0.0008) [2023-10-10 18:06:52,714][123614] Updated weights for policy 1, policy_version 39320 (0.0008) [2023-10-10 18:06:53,029][123582] Updated weights for policy 0, policy_version 39393 (0.0008) [2023-10-10 18:06:53,409][123582] Updated weights for policy 0, policy_version 39403 (0.0008) [2023-10-10 18:06:53,782][123582] Updated weights for policy 0, policy_version 39413 (0.0009) [2023-10-10 18:06:53,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 80609280. Throughput: 0: 1828.4, 1: 1812.3. Samples: 20165820. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-10 18:06:53,788][122664] Avg episode reward: [(0, '47.260'), (1, '57.300')] [2023-10-10 18:06:54,157][123582] Updated weights for policy 0, policy_version 39423 (0.0007) [2023-10-10 18:06:56,417][123614] Updated weights for policy 1, policy_version 39330 (0.0010) [2023-10-10 18:06:56,786][123614] Updated weights for policy 1, policy_version 39340 (0.0011) [2023-10-10 18:06:57,147][123614] Updated weights for policy 1, policy_version 39350 (0.0010) [2023-10-10 18:06:57,513][123614] Updated weights for policy 1, policy_version 39360 (0.0009) [2023-10-10 18:06:57,791][123582] Updated weights for policy 0, policy_version 39433 (0.0007) [2023-10-10 18:06:58,177][123582] Updated weights for policy 0, policy_version 39443 (0.0009) [2023-10-10 18:06:58,557][123582] Updated weights for policy 0, policy_version 39453 (0.0009) [2023-10-10 18:06:58,788][122664] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 80707584. Throughput: 0: 1825.5, 1: 1825.1. Samples: 20177262. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-10 18:06:58,789][122664] Avg episode reward: [(0, '48.730'), (1, '57.010')] [2023-10-10 18:07:01,203][123614] Updated weights for policy 1, policy_version 39370 (0.0008) [2023-10-10 18:07:01,572][123614] Updated weights for policy 1, policy_version 39380 (0.0008) [2023-10-10 18:07:01,938][123614] Updated weights for policy 1, policy_version 39390 (0.0007) [2023-10-10 18:07:02,260][123582] Updated weights for policy 0, policy_version 39463 (0.0009) [2023-10-10 18:07:02,639][123582] Updated weights for policy 0, policy_version 39473 (0.0010) [2023-10-10 18:07:03,017][123582] Updated weights for policy 0, policy_version 39483 (0.0009) [2023-10-10 18:07:03,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 80773120. Throughput: 0: 1817.3, 1: 1821.7. Samples: 20198576. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-10 18:07:03,789][122664] Avg episode reward: [(0, '48.940'), (1, '56.030')] [2023-10-10 18:07:05,595][123614] Updated weights for policy 1, policy_version 39400 (0.0008) [2023-10-10 18:07:05,960][123614] Updated weights for policy 1, policy_version 39410 (0.0010) [2023-10-10 18:07:06,325][123614] Updated weights for policy 1, policy_version 39420 (0.0009) [2023-10-10 18:07:06,695][123582] Updated weights for policy 0, policy_version 39493 (0.0008) [2023-10-10 18:07:07,070][123582] Updated weights for policy 0, policy_version 39503 (0.0008) [2023-10-10 18:07:07,433][123582] Updated weights for policy 0, policy_version 39513 (0.0009) [2023-10-10 18:07:08,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 80838656. Throughput: 0: 1817.1, 1: 1821.4. Samples: 20220396. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-10 18:07:08,789][122664] Avg episode reward: [(0, '48.530'), (1, '59.080')] [2023-10-10 18:07:09,928][123614] Updated weights for policy 1, policy_version 39430 (0.0009) [2023-10-10 18:07:10,296][123614] Updated weights for policy 1, policy_version 39440 (0.0007) [2023-10-10 18:07:10,662][123614] Updated weights for policy 1, policy_version 39450 (0.0010) [2023-10-10 18:07:11,135][123582] Updated weights for policy 0, policy_version 39523 (0.0010) [2023-10-10 18:07:11,520][123582] Updated weights for policy 0, policy_version 39533 (0.0007) [2023-10-10 18:07:11,894][123582] Updated weights for policy 0, policy_version 39543 (0.0007) [2023-10-10 18:07:13,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 80904192. Throughput: 0: 1814.7, 1: 1820.9. Samples: 20231274. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-10 18:07:13,788][122664] Avg episode reward: [(0, '47.320'), (1, '55.680')] [2023-10-10 18:07:14,284][123614] Updated weights for policy 1, policy_version 39460 (0.0009) [2023-10-10 18:07:14,656][123614] Updated weights for policy 1, policy_version 39470 (0.0008) [2023-10-10 18:07:15,017][123614] Updated weights for policy 1, policy_version 39480 (0.0008) [2023-10-10 18:07:15,502][123582] Updated weights for policy 0, policy_version 39553 (0.0008) [2023-10-10 18:07:15,873][123582] Updated weights for policy 0, policy_version 39563 (0.0009) [2023-10-10 18:07:16,242][123582] Updated weights for policy 0, policy_version 39573 (0.0009) [2023-10-10 18:07:16,618][123582] Updated weights for policy 0, policy_version 39583 (0.0008) [2023-10-10 18:07:18,658][123614] Updated weights for policy 1, policy_version 39490 (0.0009) [2023-10-10 18:07:18,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 80969728. Throughput: 0: 1816.3, 1: 1826.5. Samples: 20253430. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-10 18:07:18,788][122664] Avg episode reward: [(0, '45.030'), (1, '55.710')] [2023-10-10 18:07:19,037][123614] Updated weights for policy 1, policy_version 39500 (0.0010) [2023-10-10 18:07:19,406][123614] Updated weights for policy 1, policy_version 39510 (0.0009) [2023-10-10 18:07:19,775][123614] Updated weights for policy 1, policy_version 39520 (0.0011) [2023-10-10 18:07:20,221][123582] Updated weights for policy 0, policy_version 39593 (0.0008) [2023-10-10 18:07:20,590][123582] Updated weights for policy 0, policy_version 39603 (0.0009) [2023-10-10 18:07:20,969][123582] Updated weights for policy 0, policy_version 39613 (0.0011) [2023-10-10 18:07:23,554][123614] Updated weights for policy 1, policy_version 39530 (0.0011) [2023-10-10 18:07:23,788][122664] Fps is (10 sec: 13106.6, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 81035264. Throughput: 0: 1817.6, 1: 1824.9. Samples: 20275450. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-10 18:07:23,790][122664] Avg episode reward: [(0, '44.890'), (1, '57.330')] [2023-10-10 18:07:23,925][123614] Updated weights for policy 1, policy_version 39540 (0.0010) [2023-10-10 18:07:24,293][123614] Updated weights for policy 1, policy_version 39550 (0.0008) [2023-10-10 18:07:24,761][123582] Updated weights for policy 0, policy_version 39623 (0.0008) [2023-10-10 18:07:25,132][123582] Updated weights for policy 0, policy_version 39633 (0.0008) [2023-10-10 18:07:25,505][123582] Updated weights for policy 0, policy_version 39643 (0.0007) [2023-10-10 18:07:28,141][123614] Updated weights for policy 1, policy_version 39560 (0.0010) [2023-10-10 18:07:28,505][123614] Updated weights for policy 1, policy_version 39570 (0.0008) [2023-10-10 18:07:28,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 81100800. Throughput: 0: 1812.9, 1: 1822.8. Samples: 20285868. Policy #0 lag: (min: 13.0, avg: 20.9, max: 45.0) [2023-10-10 18:07:28,789][122664] Avg episode reward: [(0, '43.820'), (1, '53.620')] [2023-10-10 18:07:28,872][123614] Updated weights for policy 1, policy_version 39580 (0.0010) [2023-10-10 18:07:29,224][123582] Updated weights for policy 0, policy_version 39653 (0.0007) [2023-10-10 18:07:29,597][123582] Updated weights for policy 0, policy_version 39663 (0.0009) [2023-10-10 18:07:29,963][123582] Updated weights for policy 0, policy_version 39673 (0.0009) [2023-10-10 18:07:32,645][123614] Updated weights for policy 1, policy_version 39590 (0.0009) [2023-10-10 18:07:33,006][123614] Updated weights for policy 1, policy_version 39600 (0.0008) [2023-10-10 18:07:33,372][123614] Updated weights for policy 1, policy_version 39610 (0.0007) [2023-10-10 18:07:33,568][123582] Updated weights for policy 0, policy_version 39683 (0.0008) [2023-10-10 18:07:33,788][122664] Fps is (10 sec: 16384.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 81199104. Throughput: 0: 1811.6, 1: 1824.9. Samples: 20308206. Policy #0 lag: (min: 13.0, avg: 20.9, max: 45.0) [2023-10-10 18:07:33,788][122664] Avg episode reward: [(0, '46.290'), (1, '54.900')] [2023-10-10 18:07:33,940][123582] Updated weights for policy 0, policy_version 39693 (0.0007) [2023-10-10 18:07:34,316][123582] Updated weights for policy 0, policy_version 39703 (0.0007) [2023-10-10 18:07:37,085][123614] Updated weights for policy 1, policy_version 39620 (0.0008) [2023-10-10 18:07:37,453][123614] Updated weights for policy 1, policy_version 39630 (0.0011) [2023-10-10 18:07:37,818][123614] Updated weights for policy 1, policy_version 39640 (0.0009) [2023-10-10 18:07:37,900][123582] Updated weights for policy 0, policy_version 39713 (0.0007) [2023-10-10 18:07:38,274][123582] Updated weights for policy 0, policy_version 39723 (0.0009) [2023-10-10 18:07:38,643][123582] Updated weights for policy 0, policy_version 39733 (0.0009) [2023-10-10 18:07:38,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 81264640. Throughput: 0: 1815.6, 1: 1822.1. Samples: 20329518. Policy #0 lag: (min: 13.0, avg: 20.9, max: 45.0) [2023-10-10 18:07:38,788][122664] Avg episode reward: [(0, '48.390'), (1, '54.690')] [2023-10-10 18:07:39,011][123582] Updated weights for policy 0, policy_version 39743 (0.0007) [2023-10-10 18:07:41,536][123614] Updated weights for policy 1, policy_version 39650 (0.0008) [2023-10-10 18:07:41,906][123614] Updated weights for policy 1, policy_version 39660 (0.0007) [2023-10-10 18:07:42,265][123614] Updated weights for policy 1, policy_version 39670 (0.0008) [2023-10-10 18:07:42,636][123614] Updated weights for policy 1, policy_version 39680 (0.0010) [2023-10-10 18:07:42,803][123582] Updated weights for policy 0, policy_version 39753 (0.0007) [2023-10-10 18:07:43,167][123582] Updated weights for policy 0, policy_version 39763 (0.0010) [2023-10-10 18:07:43,535][123582] Updated weights for policy 0, policy_version 39773 (0.0011) [2023-10-10 18:07:43,788][122664] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 81362944. Throughput: 0: 1819.2, 1: 1827.1. Samples: 20341344. Policy #0 lag: (min: 13.0, avg: 20.9, max: 45.0) [2023-10-10 18:07:43,789][122664] Avg episode reward: [(0, '50.530'), (1, '53.840')] [2023-10-10 18:07:46,396][123614] Updated weights for policy 1, policy_version 39690 (0.0007) [2023-10-10 18:07:46,769][123614] Updated weights for policy 1, policy_version 39700 (0.0009) [2023-10-10 18:07:47,144][123614] Updated weights for policy 1, policy_version 39710 (0.0007) [2023-10-10 18:07:47,291][123582] Updated weights for policy 0, policy_version 39783 (0.0007) [2023-10-10 18:07:47,663][123582] Updated weights for policy 0, policy_version 39793 (0.0007) [2023-10-10 18:07:48,035][123582] Updated weights for policy 0, policy_version 39803 (0.0007) [2023-10-10 18:07:48,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 81428480. Throughput: 0: 1817.7, 1: 1818.5. Samples: 20362206. Policy #0 lag: (min: 13.0, avg: 20.9, max: 45.0) [2023-10-10 18:07:48,788][122664] Avg episode reward: [(0, '51.910'), (1, '55.270')] [2023-10-10 18:07:50,740][123614] Updated weights for policy 1, policy_version 39720 (0.0010) [2023-10-10 18:07:51,110][123614] Updated weights for policy 1, policy_version 39730 (0.0008) [2023-10-10 18:07:51,483][123614] Updated weights for policy 1, policy_version 39740 (0.0008) [2023-10-10 18:07:51,611][123582] Updated weights for policy 0, policy_version 39813 (0.0008) [2023-10-10 18:07:51,980][123582] Updated weights for policy 0, policy_version 39823 (0.0009) [2023-10-10 18:07:52,344][123582] Updated weights for policy 0, policy_version 39833 (0.0010) [2023-10-10 18:07:53,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 81494016. Throughput: 0: 1818.1, 1: 1824.9. Samples: 20384332. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:07:53,789][122664] Avg episode reward: [(0, '47.990'), (1, '55.430')] [2023-10-10 18:07:53,800][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000039840_40796160.pth... [2023-10-10 18:07:53,800][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000039744_40697856.pth... [2023-10-10 18:07:53,837][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000038048_38961152.pth [2023-10-10 18:07:53,840][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000038144_39059456.pth [2023-10-10 18:07:55,062][123614] Updated weights for policy 1, policy_version 39750 (0.0007) [2023-10-10 18:07:55,430][123614] Updated weights for policy 1, policy_version 39760 (0.0007) [2023-10-10 18:07:55,799][123614] Updated weights for policy 1, policy_version 39770 (0.0007) [2023-10-10 18:07:56,056][123582] Updated weights for policy 0, policy_version 39843 (0.0010) [2023-10-10 18:07:56,437][123582] Updated weights for policy 0, policy_version 39853 (0.0008) [2023-10-10 18:07:56,807][123582] Updated weights for policy 0, policy_version 39863 (0.0007) [2023-10-10 18:07:58,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 81559552. Throughput: 0: 1817.5, 1: 1826.5. Samples: 20395258. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:07:58,789][122664] Avg episode reward: [(0, '46.730'), (1, '54.480')] [2023-10-10 18:07:59,536][123614] Updated weights for policy 1, policy_version 39780 (0.0008) [2023-10-10 18:07:59,910][123614] Updated weights for policy 1, policy_version 39790 (0.0008) [2023-10-10 18:08:00,263][123614] Updated weights for policy 1, policy_version 39800 (0.0007) [2023-10-10 18:08:00,543][123582] Updated weights for policy 0, policy_version 39873 (0.0008) [2023-10-10 18:08:00,909][123582] Updated weights for policy 0, policy_version 39883 (0.0008) [2023-10-10 18:08:01,276][123582] Updated weights for policy 0, policy_version 39893 (0.0008) [2023-10-10 18:08:01,652][123582] Updated weights for policy 0, policy_version 39903 (0.0008) [2023-10-10 18:08:03,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 81625088. Throughput: 0: 1820.0, 1: 1812.5. Samples: 20416890. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:08:03,789][122664] Avg episode reward: [(0, '47.460'), (1, '57.450')] [2023-10-10 18:08:04,089][123614] Updated weights for policy 1, policy_version 39810 (0.0007) [2023-10-10 18:08:04,459][123614] Updated weights for policy 1, policy_version 39820 (0.0010) [2023-10-10 18:08:04,832][123614] Updated weights for policy 1, policy_version 39830 (0.0008) [2023-10-10 18:08:05,196][123614] Updated weights for policy 1, policy_version 39840 (0.0010) [2023-10-10 18:08:05,423][123582] Updated weights for policy 0, policy_version 39913 (0.0010) [2023-10-10 18:08:05,791][123582] Updated weights for policy 0, policy_version 39923 (0.0008) [2023-10-10 18:08:06,177][123582] Updated weights for policy 0, policy_version 39933 (0.0009) [2023-10-10 18:08:08,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 81690624. Throughput: 0: 1819.7, 1: 1819.0. Samples: 20439190. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:08:08,788][122664] Avg episode reward: [(0, '49.180'), (1, '59.430')] [2023-10-10 18:08:09,093][123614] Updated weights for policy 1, policy_version 39850 (0.0008) [2023-10-10 18:08:09,463][123614] Updated weights for policy 1, policy_version 39860 (0.0008) [2023-10-10 18:08:09,827][123614] Updated weights for policy 1, policy_version 39870 (0.0008) [2023-10-10 18:08:09,951][123582] Updated weights for policy 0, policy_version 39943 (0.0008) [2023-10-10 18:08:10,322][123582] Updated weights for policy 0, policy_version 39953 (0.0008) [2023-10-10 18:08:10,683][123582] Updated weights for policy 0, policy_version 39963 (0.0009) [2023-10-10 18:08:13,353][123614] Updated weights for policy 1, policy_version 39880 (0.0008) [2023-10-10 18:08:13,718][123614] Updated weights for policy 1, policy_version 39890 (0.0007) [2023-10-10 18:08:13,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 81756160. Throughput: 0: 1820.4, 1: 1807.6. Samples: 20449128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:08:13,788][122664] Avg episode reward: [(0, '52.240'), (1, '55.830')] [2023-10-10 18:08:14,090][123614] Updated weights for policy 1, policy_version 39900 (0.0008) [2023-10-10 18:08:14,125][123582] Updated weights for policy 0, policy_version 39973 (0.0008) [2023-10-10 18:08:14,497][123582] Updated weights for policy 0, policy_version 39983 (0.0010) [2023-10-10 18:08:14,881][123582] Updated weights for policy 0, policy_version 39993 (0.0008) [2023-10-10 18:08:17,756][123614] Updated weights for policy 1, policy_version 39910 (0.0010) [2023-10-10 18:08:18,113][123614] Updated weights for policy 1, policy_version 39920 (0.0007) [2023-10-10 18:08:18,483][123614] Updated weights for policy 1, policy_version 39930 (0.0008) [2023-10-10 18:08:18,608][123582] Updated weights for policy 0, policy_version 40003 (0.0008) [2023-10-10 18:08:18,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 81854464. Throughput: 0: 1823.0, 1: 1813.1. Samples: 20471830. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:08:18,788][122664] Avg episode reward: [(0, '51.020'), (1, '57.400')] [2023-10-10 18:08:18,981][123582] Updated weights for policy 0, policy_version 40013 (0.0009) [2023-10-10 18:08:19,359][123582] Updated weights for policy 0, policy_version 40023 (0.0009) [2023-10-10 18:08:22,237][123614] Updated weights for policy 1, policy_version 39940 (0.0007) [2023-10-10 18:08:22,605][123614] Updated weights for policy 1, policy_version 39950 (0.0007) [2023-10-10 18:08:22,965][123614] Updated weights for policy 1, policy_version 39960 (0.0008) [2023-10-10 18:08:23,055][123582] Updated weights for policy 0, policy_version 40033 (0.0009) [2023-10-10 18:08:23,423][123582] Updated weights for policy 0, policy_version 40043 (0.0008) [2023-10-10 18:08:23,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 81920000. Throughput: 0: 1818.9, 1: 1806.1. Samples: 20492642. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-10 18:08:23,789][122664] Avg episode reward: [(0, '48.840'), (1, '57.420')] [2023-10-10 18:08:23,803][123582] Updated weights for policy 0, policy_version 40053 (0.0007) [2023-10-10 18:08:24,172][123582] Updated weights for policy 0, policy_version 40063 (0.0007) [2023-10-10 18:08:26,543][123614] Updated weights for policy 1, policy_version 39970 (0.0009) [2023-10-10 18:08:26,914][123614] Updated weights for policy 1, policy_version 39980 (0.0007) [2023-10-10 18:08:27,285][123614] Updated weights for policy 1, policy_version 39990 (0.0009) [2023-10-10 18:08:27,654][123614] Updated weights for policy 1, policy_version 40000 (0.0007) [2023-10-10 18:08:27,799][123582] Updated weights for policy 0, policy_version 40073 (0.0008) [2023-10-10 18:08:28,163][123582] Updated weights for policy 0, policy_version 40083 (0.0012) [2023-10-10 18:08:28,547][123582] Updated weights for policy 0, policy_version 40093 (0.0010) [2023-10-10 18:08:28,788][122664] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 82018304. Throughput: 0: 1819.2, 1: 1805.5. Samples: 20504452. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-10 18:08:28,789][122664] Avg episode reward: [(0, '47.180'), (1, '55.930')] [2023-10-10 18:08:31,366][123614] Updated weights for policy 1, policy_version 40010 (0.0008) [2023-10-10 18:08:31,727][123614] Updated weights for policy 1, policy_version 40020 (0.0007) [2023-10-10 18:08:32,104][123614] Updated weights for policy 1, policy_version 40030 (0.0009) [2023-10-10 18:08:32,333][123582] Updated weights for policy 0, policy_version 40103 (0.0010) [2023-10-10 18:08:32,701][123582] Updated weights for policy 0, policy_version 40113 (0.0007) [2023-10-10 18:08:33,080][123582] Updated weights for policy 0, policy_version 40123 (0.0008) [2023-10-10 18:08:33,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 82083840. Throughput: 0: 1824.3, 1: 1808.7. Samples: 20525688. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-10 18:08:33,789][122664] Avg episode reward: [(0, '47.550'), (1, '53.830')] [2023-10-10 18:08:35,866][123614] Updated weights for policy 1, policy_version 40040 (0.0009) [2023-10-10 18:08:36,228][123614] Updated weights for policy 1, policy_version 40050 (0.0012) [2023-10-10 18:08:36,597][123614] Updated weights for policy 1, policy_version 40060 (0.0010) [2023-10-10 18:08:36,789][123582] Updated weights for policy 0, policy_version 40133 (0.0007) [2023-10-10 18:08:37,160][123582] Updated weights for policy 0, policy_version 40143 (0.0009) [2023-10-10 18:08:37,540][123582] Updated weights for policy 0, policy_version 40153 (0.0009) [2023-10-10 18:08:38,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 82149376. Throughput: 0: 1820.8, 1: 1798.8. Samples: 20547216. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-10 18:08:38,789][122664] Avg episode reward: [(0, '44.610'), (1, '56.120')] [2023-10-10 18:08:40,327][123614] Updated weights for policy 1, policy_version 40070 (0.0008) [2023-10-10 18:08:40,692][123614] Updated weights for policy 1, policy_version 40080 (0.0009) [2023-10-10 18:08:41,061][123614] Updated weights for policy 1, policy_version 40090 (0.0008) [2023-10-10 18:08:41,249][123582] Updated weights for policy 0, policy_version 40163 (0.0008) [2023-10-10 18:08:41,633][123582] Updated weights for policy 0, policy_version 40173 (0.0009) [2023-10-10 18:08:42,002][123582] Updated weights for policy 0, policy_version 40183 (0.0011) [2023-10-10 18:08:43,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 82214912. Throughput: 0: 1824.8, 1: 1796.9. Samples: 20558238. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-10 18:08:43,789][122664] Avg episode reward: [(0, '43.060'), (1, '56.720')] [2023-10-10 18:08:44,584][123614] Updated weights for policy 1, policy_version 40100 (0.0007) [2023-10-10 18:08:44,955][123614] Updated weights for policy 1, policy_version 40110 (0.0009) [2023-10-10 18:08:45,325][123614] Updated weights for policy 1, policy_version 40120 (0.0007) [2023-10-10 18:08:45,634][123582] Updated weights for policy 0, policy_version 40193 (0.0009) [2023-10-10 18:08:46,052][123582] Updated weights for policy 0, policy_version 40203 (0.0009) [2023-10-10 18:08:46,426][123582] Updated weights for policy 0, policy_version 40213 (0.0009) [2023-10-10 18:08:46,796][123582] Updated weights for policy 0, policy_version 40223 (0.0009) [2023-10-10 18:08:48,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 82280448. Throughput: 0: 1816.0, 1: 1812.9. Samples: 20580190. Policy #0 lag: (min: 9.0, avg: 22.1, max: 41.0) [2023-10-10 18:08:48,789][122664] Avg episode reward: [(0, '45.150'), (1, '54.390')] [2023-10-10 18:08:49,003][123614] Updated weights for policy 1, policy_version 40130 (0.0009) [2023-10-10 18:08:49,368][123614] Updated weights for policy 1, policy_version 40140 (0.0009) [2023-10-10 18:08:49,726][123614] Updated weights for policy 1, policy_version 40150 (0.0009) [2023-10-10 18:08:50,093][123614] Updated weights for policy 1, policy_version 40160 (0.0009) [2023-10-10 18:08:50,501][123582] Updated weights for policy 0, policy_version 40233 (0.0008) [2023-10-10 18:08:50,872][123582] Updated weights for policy 0, policy_version 40243 (0.0008) [2023-10-10 18:08:51,250][123582] Updated weights for policy 0, policy_version 40253 (0.0008) [2023-10-10 18:08:53,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 82345984. Throughput: 0: 1815.0, 1: 1812.9. Samples: 20602444. Policy #0 lag: (min: 9.0, avg: 22.1, max: 41.0) [2023-10-10 18:08:53,788][122664] Avg episode reward: [(0, '43.130'), (1, '54.040')] [2023-10-10 18:08:53,798][123614] Updated weights for policy 1, policy_version 40170 (0.0008) [2023-10-10 18:08:54,162][123614] Updated weights for policy 1, policy_version 40180 (0.0011) [2023-10-10 18:08:54,526][123614] Updated weights for policy 1, policy_version 40190 (0.0009) [2023-10-10 18:08:54,986][123582] Updated weights for policy 0, policy_version 40263 (0.0009) [2023-10-10 18:08:55,355][123582] Updated weights for policy 0, policy_version 40273 (0.0010) [2023-10-10 18:08:55,726][123582] Updated weights for policy 0, policy_version 40283 (0.0009) [2023-10-10 18:08:58,216][123614] Updated weights for policy 1, policy_version 40200 (0.0010) [2023-10-10 18:08:58,608][123614] Updated weights for policy 1, policy_version 40210 (0.0009) [2023-10-10 18:08:58,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 82411520. Throughput: 0: 1814.0, 1: 1824.9. Samples: 20612882. Policy #0 lag: (min: 9.0, avg: 22.1, max: 41.0) [2023-10-10 18:08:58,788][122664] Avg episode reward: [(0, '43.320'), (1, '53.200')] [2023-10-10 18:08:58,974][123614] Updated weights for policy 1, policy_version 40220 (0.0009) [2023-10-10 18:08:59,373][123582] Updated weights for policy 0, policy_version 40293 (0.0009) [2023-10-10 18:08:59,752][123582] Updated weights for policy 0, policy_version 40303 (0.0008) [2023-10-10 18:09:00,128][123582] Updated weights for policy 0, policy_version 40313 (0.0008) [2023-10-10 18:09:02,780][123614] Updated weights for policy 1, policy_version 40230 (0.0007) [2023-10-10 18:09:03,147][123614] Updated weights for policy 1, policy_version 40240 (0.0008) [2023-10-10 18:09:03,526][123614] Updated weights for policy 1, policy_version 40250 (0.0007) [2023-10-10 18:09:03,729][123582] Updated weights for policy 0, policy_version 40323 (0.0007) [2023-10-10 18:09:03,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 82509824. Throughput: 0: 1813.2, 1: 1820.0. Samples: 20635328. Policy #0 lag: (min: 9.0, avg: 22.1, max: 41.0) [2023-10-10 18:09:03,788][122664] Avg episode reward: [(0, '41.760'), (1, '51.080')] [2023-10-10 18:09:04,103][123582] Updated weights for policy 0, policy_version 40333 (0.0008) [2023-10-10 18:09:04,470][123582] Updated weights for policy 0, policy_version 40343 (0.0011) [2023-10-10 18:09:07,422][123614] Updated weights for policy 1, policy_version 40260 (0.0008) [2023-10-10 18:09:07,788][123614] Updated weights for policy 1, policy_version 40270 (0.0008) [2023-10-10 18:09:08,160][123614] Updated weights for policy 1, policy_version 40280 (0.0009) [2023-10-10 18:09:08,356][123582] Updated weights for policy 0, policy_version 40353 (0.0011) [2023-10-10 18:09:08,729][123582] Updated weights for policy 0, policy_version 40363 (0.0008) [2023-10-10 18:09:08,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 82575360. Throughput: 0: 1822.0, 1: 1814.1. Samples: 20656264. Policy #0 lag: (min: 9.0, avg: 22.1, max: 41.0) [2023-10-10 18:09:08,788][122664] Avg episode reward: [(0, '42.690'), (1, '52.360')] [2023-10-10 18:09:09,097][123582] Updated weights for policy 0, policy_version 40373 (0.0009) [2023-10-10 18:09:09,476][123582] Updated weights for policy 0, policy_version 40383 (0.0009) [2023-10-10 18:09:11,835][123614] Updated weights for policy 1, policy_version 40290 (0.0007) [2023-10-10 18:09:12,199][123614] Updated weights for policy 1, policy_version 40300 (0.0008) [2023-10-10 18:09:12,566][123614] Updated weights for policy 1, policy_version 40310 (0.0007) [2023-10-10 18:09:12,932][123614] Updated weights for policy 1, policy_version 40320 (0.0007) [2023-10-10 18:09:13,101][123582] Updated weights for policy 0, policy_version 40393 (0.0009) [2023-10-10 18:09:13,477][123582] Updated weights for policy 0, policy_version 40403 (0.0009) [2023-10-10 18:09:13,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 82640896. Throughput: 0: 1809.2, 1: 1824.1. Samples: 20667950. Policy #0 lag: (min: 9.0, avg: 22.1, max: 41.0) [2023-10-10 18:09:13,788][122664] Avg episode reward: [(0, '39.990'), (1, '53.140')] [2023-10-10 18:09:13,846][123582] Updated weights for policy 0, policy_version 40413 (0.0008) [2023-10-10 18:09:16,616][123614] Updated weights for policy 1, policy_version 40330 (0.0007) [2023-10-10 18:09:16,984][123614] Updated weights for policy 1, policy_version 40340 (0.0009) [2023-10-10 18:09:17,344][123614] Updated weights for policy 1, policy_version 40350 (0.0008) [2023-10-10 18:09:17,586][123582] Updated weights for policy 0, policy_version 40423 (0.0010) [2023-10-10 18:09:17,962][123582] Updated weights for policy 0, policy_version 40433 (0.0009) [2023-10-10 18:09:18,344][123582] Updated weights for policy 0, policy_version 40443 (0.0008) [2023-10-10 18:09:18,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 82739200. Throughput: 0: 1813.9, 1: 1816.4. Samples: 20689052. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-10-10 18:09:18,788][122664] Avg episode reward: [(0, '41.020'), (1, '52.050')] [2023-10-10 18:09:21,042][123614] Updated weights for policy 1, policy_version 40360 (0.0010) [2023-10-10 18:09:21,408][123614] Updated weights for policy 1, policy_version 40370 (0.0007) [2023-10-10 18:09:21,784][123614] Updated weights for policy 1, policy_version 40380 (0.0008) [2023-10-10 18:09:22,163][123582] Updated weights for policy 0, policy_version 40453 (0.0008) [2023-10-10 18:09:22,534][123582] Updated weights for policy 0, policy_version 40463 (0.0011) [2023-10-10 18:09:22,895][123582] Updated weights for policy 0, policy_version 40473 (0.0010) [2023-10-10 18:09:23,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 82804736. Throughput: 0: 1806.1, 1: 1820.2. Samples: 20710402. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-10-10 18:09:23,789][122664] Avg episode reward: [(0, '38.940'), (1, '52.240')] [2023-10-10 18:09:25,416][123614] Updated weights for policy 1, policy_version 40390 (0.0008) [2023-10-10 18:09:25,783][123614] Updated weights for policy 1, policy_version 40400 (0.0007) [2023-10-10 18:09:26,155][123614] Updated weights for policy 1, policy_version 40410 (0.0007) [2023-10-10 18:09:26,463][123582] Updated weights for policy 0, policy_version 40483 (0.0010) [2023-10-10 18:09:26,840][123582] Updated weights for policy 0, policy_version 40493 (0.0012) [2023-10-10 18:09:27,205][123582] Updated weights for policy 0, policy_version 40503 (0.0008) [2023-10-10 18:09:28,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 82870272. Throughput: 0: 1818.0, 1: 1820.0. Samples: 20721948. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-10-10 18:09:28,789][122664] Avg episode reward: [(0, '40.800'), (1, '55.070')] [2023-10-10 18:09:29,717][123614] Updated weights for policy 1, policy_version 40420 (0.0008) [2023-10-10 18:09:30,079][123614] Updated weights for policy 1, policy_version 40430 (0.0010) [2023-10-10 18:09:30,444][123614] Updated weights for policy 1, policy_version 40440 (0.0008) [2023-10-10 18:09:30,815][123582] Updated weights for policy 0, policy_version 40513 (0.0009) [2023-10-10 18:09:31,191][123582] Updated weights for policy 0, policy_version 40523 (0.0009) [2023-10-10 18:09:31,565][123582] Updated weights for policy 0, policy_version 40533 (0.0010) [2023-10-10 18:09:31,935][123582] Updated weights for policy 0, policy_version 40543 (0.0011) [2023-10-10 18:09:33,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 82935808. Throughput: 0: 1810.9, 1: 1815.0. Samples: 20743356. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-10-10 18:09:33,789][122664] Avg episode reward: [(0, '41.910'), (1, '55.220')] [2023-10-10 18:09:34,115][123614] Updated weights for policy 1, policy_version 40450 (0.0008) [2023-10-10 18:09:34,484][123614] Updated weights for policy 1, policy_version 40460 (0.0008) [2023-10-10 18:09:34,859][123614] Updated weights for policy 1, policy_version 40470 (0.0008) [2023-10-10 18:09:35,228][123614] Updated weights for policy 1, policy_version 40480 (0.0008) [2023-10-10 18:09:35,725][123582] Updated weights for policy 0, policy_version 40553 (0.0010) [2023-10-10 18:09:36,108][123582] Updated weights for policy 0, policy_version 40563 (0.0010) [2023-10-10 18:09:36,477][123582] Updated weights for policy 0, policy_version 40573 (0.0009) [2023-10-10 18:09:38,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 83001344. Throughput: 0: 1810.1, 1: 1822.1. Samples: 20765892. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-10-10 18:09:38,789][122664] Avg episode reward: [(0, '41.630'), (1, '56.260')] [2023-10-10 18:09:38,872][123614] Updated weights for policy 1, policy_version 40490 (0.0007) [2023-10-10 18:09:39,241][123614] Updated weights for policy 1, policy_version 40500 (0.0008) [2023-10-10 18:09:39,607][123614] Updated weights for policy 1, policy_version 40510 (0.0010) [2023-10-10 18:09:40,147][123582] Updated weights for policy 0, policy_version 40583 (0.0009) [2023-10-10 18:09:40,513][123582] Updated weights for policy 0, policy_version 40593 (0.0007) [2023-10-10 18:09:40,888][123582] Updated weights for policy 0, policy_version 40603 (0.0009) [2023-10-10 18:09:43,316][123614] Updated weights for policy 1, policy_version 40520 (0.0009) [2023-10-10 18:09:43,686][123614] Updated weights for policy 1, policy_version 40530 (0.0007) [2023-10-10 18:09:43,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 83066880. Throughput: 0: 1811.4, 1: 1816.9. Samples: 20776156. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-10 18:09:43,789][122664] Avg episode reward: [(0, '42.750'), (1, '54.350')] [2023-10-10 18:09:44,062][123614] Updated weights for policy 1, policy_version 40540 (0.0008) [2023-10-10 18:09:44,574][123582] Updated weights for policy 0, policy_version 40613 (0.0009) [2023-10-10 18:09:44,947][123582] Updated weights for policy 0, policy_version 40623 (0.0007) [2023-10-10 18:09:45,332][123582] Updated weights for policy 0, policy_version 40633 (0.0010) [2023-10-10 18:09:47,618][123614] Updated weights for policy 1, policy_version 40550 (0.0009) [2023-10-10 18:09:47,989][123614] Updated weights for policy 1, policy_version 40560 (0.0009) [2023-10-10 18:09:48,369][123614] Updated weights for policy 1, policy_version 40570 (0.0008) [2023-10-10 18:09:48,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 83165184. Throughput: 0: 1806.4, 1: 1822.2. Samples: 20798616. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-10 18:09:48,789][122664] Avg episode reward: [(0, '44.720'), (1, '50.900')] [2023-10-10 18:09:48,983][123582] Updated weights for policy 0, policy_version 40643 (0.0008) [2023-10-10 18:09:49,365][123582] Updated weights for policy 0, policy_version 40653 (0.0009) [2023-10-10 18:09:49,730][123582] Updated weights for policy 0, policy_version 40663 (0.0009) [2023-10-10 18:09:52,041][123614] Updated weights for policy 1, policy_version 40580 (0.0007) [2023-10-10 18:09:52,418][123614] Updated weights for policy 1, policy_version 40590 (0.0009) [2023-10-10 18:09:52,783][123614] Updated weights for policy 1, policy_version 40600 (0.0010) [2023-10-10 18:09:53,468][123582] Updated weights for policy 0, policy_version 40673 (0.0009) [2023-10-10 18:09:53,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 83230720. Throughput: 0: 1812.7, 1: 1834.7. Samples: 20820394. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-10 18:09:53,789][122664] Avg episode reward: [(0, '44.640'), (1, '52.230')] [2023-10-10 18:09:53,796][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000040608_41582592.pth... [2023-10-10 18:09:53,837][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000038912_39845888.pth [2023-10-10 18:09:53,837][123582] Updated weights for policy 0, policy_version 40683 (0.0010) [2023-10-10 18:09:54,225][123582] Updated weights for policy 0, policy_version 40693 (0.0010) [2023-10-10 18:09:54,599][123582] Updated weights for policy 0, policy_version 40703 (0.0011) [2023-10-10 18:09:54,633][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000040704_41680896.pth... [2023-10-10 18:09:54,661][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000038976_39911424.pth [2023-10-10 18:09:56,418][123614] Updated weights for policy 1, policy_version 40610 (0.0010) [2023-10-10 18:09:56,795][123614] Updated weights for policy 1, policy_version 40620 (0.0007) [2023-10-10 18:09:57,152][123614] Updated weights for policy 1, policy_version 40630 (0.0008) [2023-10-10 18:09:57,526][123614] Updated weights for policy 1, policy_version 40640 (0.0009) [2023-10-10 18:09:58,298][123582] Updated weights for policy 0, policy_version 40713 (0.0011) [2023-10-10 18:09:58,671][123582] Updated weights for policy 0, policy_version 40723 (0.0008) [2023-10-10 18:09:58,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 83296256. Throughput: 0: 1810.3, 1: 1823.2. Samples: 20831456. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-10 18:09:58,788][122664] Avg episode reward: [(0, '43.240'), (1, '50.370')] [2023-10-10 18:09:59,037][123582] Updated weights for policy 0, policy_version 40733 (0.0008) [2023-10-10 18:10:01,178][123614] Updated weights for policy 1, policy_version 40650 (0.0009) [2023-10-10 18:10:01,542][123614] Updated weights for policy 1, policy_version 40660 (0.0011) [2023-10-10 18:10:01,918][123614] Updated weights for policy 1, policy_version 40670 (0.0009) [2023-10-10 18:10:02,578][123582] Updated weights for policy 0, policy_version 40743 (0.0009) [2023-10-10 18:10:02,952][123582] Updated weights for policy 0, policy_version 40753 (0.0008) [2023-10-10 18:10:03,324][123582] Updated weights for policy 0, policy_version 40763 (0.0009) [2023-10-10 18:10:03,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 83394560. Throughput: 0: 1815.8, 1: 1835.0. Samples: 20853336. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-10 18:10:03,789][122664] Avg episode reward: [(0, '43.790'), (1, '49.320')] [2023-10-10 18:10:05,559][123614] Updated weights for policy 1, policy_version 40680 (0.0009) [2023-10-10 18:10:05,930][123614] Updated weights for policy 1, policy_version 40690 (0.0007) [2023-10-10 18:10:06,305][123614] Updated weights for policy 1, policy_version 40700 (0.0008) [2023-10-10 18:10:06,989][123582] Updated weights for policy 0, policy_version 40773 (0.0008) [2023-10-10 18:10:07,372][123582] Updated weights for policy 0, policy_version 40783 (0.0009) [2023-10-10 18:10:07,738][123582] Updated weights for policy 0, policy_version 40793 (0.0009) [2023-10-10 18:10:08,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 83460096. Throughput: 0: 1819.7, 1: 1835.2. Samples: 20874872. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:10:08,789][122664] Avg episode reward: [(0, '48.190'), (1, '50.310')] [2023-10-10 18:10:09,975][123614] Updated weights for policy 1, policy_version 40710 (0.0008) [2023-10-10 18:10:10,332][123614] Updated weights for policy 1, policy_version 40720 (0.0009) [2023-10-10 18:10:10,699][123614] Updated weights for policy 1, policy_version 40730 (0.0009) [2023-10-10 18:10:11,440][123582] Updated weights for policy 0, policy_version 40803 (0.0010) [2023-10-10 18:10:11,821][123582] Updated weights for policy 0, policy_version 40813 (0.0007) [2023-10-10 18:10:12,196][123582] Updated weights for policy 0, policy_version 40823 (0.0009) [2023-10-10 18:10:13,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 83525632. Throughput: 0: 1815.2, 1: 1833.9. Samples: 20886160. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:10:13,789][122664] Avg episode reward: [(0, '46.930'), (1, '58.480')] [2023-10-10 18:10:14,419][123614] Updated weights for policy 1, policy_version 40740 (0.0009) [2023-10-10 18:10:14,790][123614] Updated weights for policy 1, policy_version 40750 (0.0007) [2023-10-10 18:10:15,157][123614] Updated weights for policy 1, policy_version 40760 (0.0007) [2023-10-10 18:10:15,840][123582] Updated weights for policy 0, policy_version 40833 (0.0009) [2023-10-10 18:10:16,218][123582] Updated weights for policy 0, policy_version 40843 (0.0010) [2023-10-10 18:10:16,592][123582] Updated weights for policy 0, policy_version 40853 (0.0007) [2023-10-10 18:10:16,960][123582] Updated weights for policy 0, policy_version 40863 (0.0007) [2023-10-10 18:10:18,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 83591168. Throughput: 0: 1822.0, 1: 1830.8. Samples: 20907728. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:10:18,789][122664] Avg episode reward: [(0, '50.950'), (1, '58.120')] [2023-10-10 18:10:18,853][123614] Updated weights for policy 1, policy_version 40770 (0.0009) [2023-10-10 18:10:19,224][123614] Updated weights for policy 1, policy_version 40780 (0.0008) [2023-10-10 18:10:19,600][123614] Updated weights for policy 1, policy_version 40790 (0.0008) [2023-10-10 18:10:19,970][123614] Updated weights for policy 1, policy_version 40800 (0.0009) [2023-10-10 18:10:20,778][123582] Updated weights for policy 0, policy_version 40873 (0.0007) [2023-10-10 18:10:21,143][123582] Updated weights for policy 0, policy_version 40883 (0.0008) [2023-10-10 18:10:21,514][123582] Updated weights for policy 0, policy_version 40893 (0.0011) [2023-10-10 18:10:23,762][123614] Updated weights for policy 1, policy_version 40810 (0.0008) [2023-10-10 18:10:23,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 83656704. Throughput: 0: 1817.7, 1: 1823.6. Samples: 20929752. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:10:23,788][122664] Avg episode reward: [(0, '51.260'), (1, '55.240')] [2023-10-10 18:10:24,131][123614] Updated weights for policy 1, policy_version 40820 (0.0007) [2023-10-10 18:10:24,498][123614] Updated weights for policy 1, policy_version 40830 (0.0009) [2023-10-10 18:10:25,188][123582] Updated weights for policy 0, policy_version 40903 (0.0009) [2023-10-10 18:10:25,561][123582] Updated weights for policy 0, policy_version 40913 (0.0008) [2023-10-10 18:10:25,926][123582] Updated weights for policy 0, policy_version 40923 (0.0008) [2023-10-10 18:10:28,046][123614] Updated weights for policy 1, policy_version 40840 (0.0008) [2023-10-10 18:10:28,412][123614] Updated weights for policy 1, policy_version 40850 (0.0010) [2023-10-10 18:10:28,780][123614] Updated weights for policy 1, policy_version 40860 (0.0007) [2023-10-10 18:10:28,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 83722240. Throughput: 0: 1814.9, 1: 1830.1. Samples: 20940182. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:10:28,789][122664] Avg episode reward: [(0, '49.570'), (1, '52.980')] [2023-10-10 18:10:29,594][123582] Updated weights for policy 0, policy_version 40933 (0.0008) [2023-10-10 18:10:29,976][123582] Updated weights for policy 0, policy_version 40943 (0.0011) [2023-10-10 18:10:30,340][123582] Updated weights for policy 0, policy_version 40953 (0.0008) [2023-10-10 18:10:32,423][123614] Updated weights for policy 1, policy_version 40870 (0.0010) [2023-10-10 18:10:32,782][123614] Updated weights for policy 1, policy_version 40880 (0.0008) [2023-10-10 18:10:33,147][123614] Updated weights for policy 1, policy_version 40890 (0.0009) [2023-10-10 18:10:33,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 83820544. Throughput: 0: 1813.5, 1: 1820.4. Samples: 20962144. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:10:33,789][122664] Avg episode reward: [(0, '49.850'), (1, '52.660')] [2023-10-10 18:10:34,037][123582] Updated weights for policy 0, policy_version 40963 (0.0010) [2023-10-10 18:10:34,408][123582] Updated weights for policy 0, policy_version 40973 (0.0010) [2023-10-10 18:10:34,789][123582] Updated weights for policy 0, policy_version 40983 (0.0009) [2023-10-10 18:10:36,961][123614] Updated weights for policy 1, policy_version 40900 (0.0008) [2023-10-10 18:10:37,330][123614] Updated weights for policy 1, policy_version 40910 (0.0010) [2023-10-10 18:10:37,698][123614] Updated weights for policy 1, policy_version 40920 (0.0010) [2023-10-10 18:10:38,425][123582] Updated weights for policy 0, policy_version 40993 (0.0007) [2023-10-10 18:10:38,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 83886080. Throughput: 0: 1812.4, 1: 1820.2. Samples: 20983860. Policy #0 lag: (min: 3.0, avg: 17.7, max: 35.0) [2023-10-10 18:10:38,788][122664] Avg episode reward: [(0, '48.370'), (1, '52.770')] [2023-10-10 18:10:38,797][123582] Updated weights for policy 0, policy_version 41003 (0.0008) [2023-10-10 18:10:39,166][123582] Updated weights for policy 0, policy_version 41013 (0.0008) [2023-10-10 18:10:39,536][123582] Updated weights for policy 0, policy_version 41023 (0.0009) [2023-10-10 18:10:41,311][123614] Updated weights for policy 1, policy_version 40930 (0.0010) [2023-10-10 18:10:41,685][123614] Updated weights for policy 1, policy_version 40940 (0.0007) [2023-10-10 18:10:42,060][123614] Updated weights for policy 1, policy_version 40950 (0.0007) [2023-10-10 18:10:42,425][123614] Updated weights for policy 1, policy_version 40960 (0.0007) [2023-10-10 18:10:43,237][123582] Updated weights for policy 0, policy_version 41033 (0.0007) [2023-10-10 18:10:43,608][123582] Updated weights for policy 0, policy_version 41043 (0.0007) [2023-10-10 18:10:43,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 83951616. Throughput: 0: 1813.7, 1: 1814.1. Samples: 20994708. Policy #0 lag: (min: 3.0, avg: 17.7, max: 35.0) [2023-10-10 18:10:43,788][122664] Avg episode reward: [(0, '47.770'), (1, '53.720')] [2023-10-10 18:10:43,970][123582] Updated weights for policy 0, policy_version 41053 (0.0009) [2023-10-10 18:10:46,030][123614] Updated weights for policy 1, policy_version 40970 (0.0008) [2023-10-10 18:10:46,405][123614] Updated weights for policy 1, policy_version 40980 (0.0009) [2023-10-10 18:10:46,772][123614] Updated weights for policy 1, policy_version 40990 (0.0007) [2023-10-10 18:10:47,602][123582] Updated weights for policy 0, policy_version 41063 (0.0010) [2023-10-10 18:10:47,972][123582] Updated weights for policy 0, policy_version 41073 (0.0010) [2023-10-10 18:10:48,343][123582] Updated weights for policy 0, policy_version 41083 (0.0007) [2023-10-10 18:10:48,788][122664] Fps is (10 sec: 16383.2, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 84049920. Throughput: 0: 1816.7, 1: 1817.6. Samples: 21016882. Policy #0 lag: (min: 3.0, avg: 17.7, max: 35.0) [2023-10-10 18:10:48,790][122664] Avg episode reward: [(0, '48.990'), (1, '56.910')] [2023-10-10 18:10:50,556][123614] Updated weights for policy 1, policy_version 41000 (0.0008) [2023-10-10 18:10:50,923][123614] Updated weights for policy 1, policy_version 41010 (0.0008) [2023-10-10 18:10:51,299][123614] Updated weights for policy 1, policy_version 41020 (0.0009) [2023-10-10 18:10:52,040][123582] Updated weights for policy 0, policy_version 41093 (0.0009) [2023-10-10 18:10:52,414][123582] Updated weights for policy 0, policy_version 41103 (0.0007) [2023-10-10 18:10:52,793][123582] Updated weights for policy 0, policy_version 41113 (0.0007) [2023-10-10 18:10:53,788][122664] Fps is (10 sec: 16383.4, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 84115456. Throughput: 0: 1814.7, 1: 1813.6. Samples: 21038148. Policy #0 lag: (min: 3.0, avg: 17.7, max: 35.0) [2023-10-10 18:10:53,789][122664] Avg episode reward: [(0, '48.120'), (1, '57.680')] [2023-10-10 18:10:54,990][123614] Updated weights for policy 1, policy_version 41030 (0.0008) [2023-10-10 18:10:55,357][123614] Updated weights for policy 1, policy_version 41040 (0.0007) [2023-10-10 18:10:55,725][123614] Updated weights for policy 1, policy_version 41050 (0.0007) [2023-10-10 18:10:56,387][123582] Updated weights for policy 0, policy_version 41123 (0.0008) [2023-10-10 18:10:56,760][123582] Updated weights for policy 0, policy_version 41133 (0.0007) [2023-10-10 18:10:57,135][123582] Updated weights for policy 0, policy_version 41143 (0.0008) [2023-10-10 18:10:58,788][122664] Fps is (10 sec: 13107.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 84180992. Throughput: 0: 1811.7, 1: 1814.8. Samples: 21049356. Policy #0 lag: (min: 3.0, avg: 17.7, max: 35.0) [2023-10-10 18:10:58,789][122664] Avg episode reward: [(0, '54.720'), (1, '59.160')] [2023-10-10 18:10:59,519][123614] Updated weights for policy 1, policy_version 41060 (0.0010) [2023-10-10 18:10:59,876][123614] Updated weights for policy 1, policy_version 41070 (0.0008) [2023-10-10 18:11:00,246][123614] Updated weights for policy 1, policy_version 41080 (0.0008) [2023-10-10 18:11:00,950][123582] Updated weights for policy 0, policy_version 41153 (0.0009) [2023-10-10 18:11:01,326][123582] Updated weights for policy 0, policy_version 41163 (0.0010) [2023-10-10 18:11:01,699][123582] Updated weights for policy 0, policy_version 41173 (0.0009) [2023-10-10 18:11:02,061][123582] Updated weights for policy 0, policy_version 41183 (0.0010) [2023-10-10 18:11:03,780][123614] Updated weights for policy 1, policy_version 41090 (0.0007) [2023-10-10 18:11:03,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 84246528. Throughput: 0: 1805.8, 1: 1816.6. Samples: 21070734. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 18:11:03,789][122664] Avg episode reward: [(0, '55.570'), (1, '57.230')] [2023-10-10 18:11:04,147][123614] Updated weights for policy 1, policy_version 41100 (0.0007) [2023-10-10 18:11:04,510][123614] Updated weights for policy 1, policy_version 41110 (0.0008) [2023-10-10 18:11:04,884][123614] Updated weights for policy 1, policy_version 41120 (0.0008) [2023-10-10 18:11:05,951][123582] Updated weights for policy 0, policy_version 41193 (0.0007) [2023-10-10 18:11:06,329][123582] Updated weights for policy 0, policy_version 41203 (0.0007) [2023-10-10 18:11:06,706][123582] Updated weights for policy 0, policy_version 41213 (0.0010) [2023-10-10 18:11:08,611][123614] Updated weights for policy 1, policy_version 41130 (0.0008) [2023-10-10 18:11:08,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 84312064. Throughput: 0: 1809.0, 1: 1821.7. Samples: 21093136. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 18:11:08,789][122664] Avg episode reward: [(0, '52.850'), (1, '53.370')] [2023-10-10 18:11:08,983][123614] Updated weights for policy 1, policy_version 41140 (0.0007) [2023-10-10 18:11:09,344][123614] Updated weights for policy 1, policy_version 41150 (0.0008) [2023-10-10 18:11:10,232][123582] Updated weights for policy 0, policy_version 41223 (0.0009) [2023-10-10 18:11:10,608][123582] Updated weights for policy 0, policy_version 41233 (0.0008) [2023-10-10 18:11:10,981][123582] Updated weights for policy 0, policy_version 41243 (0.0007) [2023-10-10 18:11:13,188][123614] Updated weights for policy 1, policy_version 41160 (0.0008) [2023-10-10 18:11:13,573][123614] Updated weights for policy 1, policy_version 41170 (0.0009) [2023-10-10 18:11:13,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 84377600. Throughput: 0: 1813.2, 1: 1820.4. Samples: 21103698. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 18:11:13,788][122664] Avg episode reward: [(0, '55.180'), (1, '50.600')] [2023-10-10 18:11:13,947][123614] Updated weights for policy 1, policy_version 41180 (0.0010) [2023-10-10 18:11:14,649][123582] Updated weights for policy 0, policy_version 41253 (0.0010) [2023-10-10 18:11:15,024][123582] Updated weights for policy 0, policy_version 41263 (0.0008) [2023-10-10 18:11:15,396][123582] Updated weights for policy 0, policy_version 41273 (0.0007) [2023-10-10 18:11:17,612][123614] Updated weights for policy 1, policy_version 41190 (0.0009) [2023-10-10 18:11:17,984][123614] Updated weights for policy 1, policy_version 41200 (0.0009) [2023-10-10 18:11:18,366][123614] Updated weights for policy 1, policy_version 41210 (0.0009) [2023-10-10 18:11:18,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 84475904. Throughput: 0: 1821.7, 1: 1822.8. Samples: 21126144. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 18:11:18,789][122664] Avg episode reward: [(0, '52.670'), (1, '50.850')] [2023-10-10 18:11:18,962][123582] Updated weights for policy 0, policy_version 41283 (0.0007) [2023-10-10 18:11:19,331][123582] Updated weights for policy 0, policy_version 41293 (0.0009) [2023-10-10 18:11:19,699][123582] Updated weights for policy 0, policy_version 41303 (0.0010) [2023-10-10 18:11:22,065][123614] Updated weights for policy 1, policy_version 41220 (0.0010) [2023-10-10 18:11:22,429][123614] Updated weights for policy 1, policy_version 41230 (0.0008) [2023-10-10 18:11:22,796][123614] Updated weights for policy 1, policy_version 41240 (0.0010) [2023-10-10 18:11:23,322][123582] Updated weights for policy 0, policy_version 41313 (0.0008) [2023-10-10 18:11:23,691][123582] Updated weights for policy 0, policy_version 41323 (0.0008) [2023-10-10 18:11:23,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 84541440. Throughput: 0: 1820.5, 1: 1819.0. Samples: 21147638. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 18:11:23,788][122664] Avg episode reward: [(0, '52.220'), (1, '50.060')] [2023-10-10 18:11:24,066][123582] Updated weights for policy 0, policy_version 41333 (0.0008) [2023-10-10 18:11:24,430][123582] Updated weights for policy 0, policy_version 41343 (0.0009) [2023-10-10 18:11:26,352][123614] Updated weights for policy 1, policy_version 41250 (0.0010) [2023-10-10 18:11:26,720][123614] Updated weights for policy 1, policy_version 41260 (0.0009) [2023-10-10 18:11:27,096][123614] Updated weights for policy 1, policy_version 41270 (0.0009) [2023-10-10 18:11:27,456][123614] Updated weights for policy 1, policy_version 41280 (0.0010) [2023-10-10 18:11:28,199][123582] Updated weights for policy 0, policy_version 41353 (0.0008) [2023-10-10 18:11:28,575][123582] Updated weights for policy 0, policy_version 41363 (0.0007) [2023-10-10 18:11:28,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 84606976. Throughput: 0: 1821.5, 1: 1824.8. Samples: 21158792. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 18:11:28,788][122664] Avg episode reward: [(0, '52.020'), (1, '50.150')] [2023-10-10 18:11:28,943][123582] Updated weights for policy 0, policy_version 41373 (0.0007) [2023-10-10 18:11:31,257][123614] Updated weights for policy 1, policy_version 41290 (0.0009) [2023-10-10 18:11:31,628][123614] Updated weights for policy 1, policy_version 41300 (0.0007) [2023-10-10 18:11:32,006][123614] Updated weights for policy 1, policy_version 41310 (0.0008) [2023-10-10 18:11:32,646][123582] Updated weights for policy 0, policy_version 41383 (0.0008) [2023-10-10 18:11:33,005][123582] Updated weights for policy 0, policy_version 41393 (0.0009) [2023-10-10 18:11:33,385][123582] Updated weights for policy 0, policy_version 41403 (0.0008) [2023-10-10 18:11:33,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 84705280. Throughput: 0: 1816.6, 1: 1819.1. Samples: 21180490. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 18:11:33,789][122664] Avg episode reward: [(0, '52.030'), (1, '52.320')] [2023-10-10 18:11:35,523][123614] Updated weights for policy 1, policy_version 41320 (0.0009) [2023-10-10 18:11:35,892][123614] Updated weights for policy 1, policy_version 41330 (0.0008) [2023-10-10 18:11:36,261][123614] Updated weights for policy 1, policy_version 41340 (0.0008) [2023-10-10 18:11:37,060][123582] Updated weights for policy 0, policy_version 41413 (0.0009) [2023-10-10 18:11:37,432][123582] Updated weights for policy 0, policy_version 41423 (0.0009) [2023-10-10 18:11:37,810][123582] Updated weights for policy 0, policy_version 41433 (0.0010) [2023-10-10 18:11:38,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 84770816. Throughput: 0: 1815.7, 1: 1824.4. Samples: 21201950. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 18:11:38,789][122664] Avg episode reward: [(0, '52.320'), (1, '55.800')] [2023-10-10 18:11:39,955][123614] Updated weights for policy 1, policy_version 41350 (0.0009) [2023-10-10 18:11:40,325][123614] Updated weights for policy 1, policy_version 41360 (0.0009) [2023-10-10 18:11:40,693][123614] Updated weights for policy 1, policy_version 41370 (0.0009) [2023-10-10 18:11:41,569][123582] Updated weights for policy 0, policy_version 41443 (0.0008) [2023-10-10 18:11:41,941][123582] Updated weights for policy 0, policy_version 41453 (0.0010) [2023-10-10 18:11:42,323][123582] Updated weights for policy 0, policy_version 41463 (0.0007) [2023-10-10 18:11:43,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 84836352. Throughput: 0: 1824.6, 1: 1823.6. Samples: 21213522. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 18:11:43,788][122664] Avg episode reward: [(0, '52.720'), (1, '54.060')] [2023-10-10 18:11:44,329][123614] Updated weights for policy 1, policy_version 41380 (0.0007) [2023-10-10 18:11:44,696][123614] Updated weights for policy 1, policy_version 41390 (0.0007) [2023-10-10 18:11:45,062][123614] Updated weights for policy 1, policy_version 41400 (0.0008) [2023-10-10 18:11:45,927][123582] Updated weights for policy 0, policy_version 41473 (0.0007) [2023-10-10 18:11:46,301][123582] Updated weights for policy 0, policy_version 41483 (0.0009) [2023-10-10 18:11:46,671][123582] Updated weights for policy 0, policy_version 41493 (0.0007) [2023-10-10 18:11:47,034][123582] Updated weights for policy 0, policy_version 41503 (0.0010) [2023-10-10 18:11:48,711][123614] Updated weights for policy 1, policy_version 41410 (0.0007) [2023-10-10 18:11:48,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.6, 300 sec: 14551.2). Total num frames: 84901888. Throughput: 0: 1828.6, 1: 1832.9. Samples: 21235504. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 18:11:48,789][122664] Avg episode reward: [(0, '49.010'), (1, '53.270')] [2023-10-10 18:11:49,093][123614] Updated weights for policy 1, policy_version 41420 (0.0010) [2023-10-10 18:11:49,469][123614] Updated weights for policy 1, policy_version 41430 (0.0008) [2023-10-10 18:11:49,831][123614] Updated weights for policy 1, policy_version 41440 (0.0008) [2023-10-10 18:11:50,682][123582] Updated weights for policy 0, policy_version 41513 (0.0008) [2023-10-10 18:11:51,056][123582] Updated weights for policy 0, policy_version 41523 (0.0007) [2023-10-10 18:11:51,429][123582] Updated weights for policy 0, policy_version 41533 (0.0008) [2023-10-10 18:11:53,698][123614] Updated weights for policy 1, policy_version 41450 (0.0008) [2023-10-10 18:11:53,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 84967424. Throughput: 0: 1834.7, 1: 1820.4. Samples: 21257616. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 18:11:53,788][122664] Avg episode reward: [(0, '48.450'), (1, '51.560')] [2023-10-10 18:11:53,795][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000041536_42532864.pth... [2023-10-10 18:11:53,836][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000039840_40796160.pth [2023-10-10 18:11:54,069][123614] Updated weights for policy 1, policy_version 41460 (0.0009) [2023-10-10 18:11:54,438][123614] Updated weights for policy 1, policy_version 41470 (0.0009) [2023-10-10 18:11:54,504][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000041472_42467328.pth... [2023-10-10 18:11:54,546][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000039744_40697856.pth [2023-10-10 18:11:55,022][123582] Updated weights for policy 0, policy_version 41543 (0.0008) [2023-10-10 18:11:55,395][123582] Updated weights for policy 0, policy_version 41553 (0.0009) [2023-10-10 18:11:55,766][123582] Updated weights for policy 0, policy_version 41563 (0.0009) [2023-10-10 18:11:58,244][123614] Updated weights for policy 1, policy_version 41480 (0.0009) [2023-10-10 18:11:58,620][123614] Updated weights for policy 1, policy_version 41490 (0.0010) [2023-10-10 18:11:58,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 85032960. Throughput: 0: 1833.6, 1: 1819.9. Samples: 21268106. Policy #0 lag: (min: 8.0, avg: 26.6, max: 40.0) [2023-10-10 18:11:58,788][122664] Avg episode reward: [(0, '47.830'), (1, '54.700')] [2023-10-10 18:11:59,000][123614] Updated weights for policy 1, policy_version 41500 (0.0007) [2023-10-10 18:11:59,341][123582] Updated weights for policy 0, policy_version 41573 (0.0009) [2023-10-10 18:11:59,701][123582] Updated weights for policy 0, policy_version 41583 (0.0009) [2023-10-10 18:12:00,086][123582] Updated weights for policy 0, policy_version 41593 (0.0009) [2023-10-10 18:12:02,626][123614] Updated weights for policy 1, policy_version 41510 (0.0009) [2023-10-10 18:12:02,990][123614] Updated weights for policy 1, policy_version 41520 (0.0008) [2023-10-10 18:12:03,366][123614] Updated weights for policy 1, policy_version 41530 (0.0009) [2023-10-10 18:12:03,713][123582] Updated weights for policy 0, policy_version 41603 (0.0009) [2023-10-10 18:12:03,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 85131264. Throughput: 0: 1829.8, 1: 1815.9. Samples: 21290202. Policy #0 lag: (min: 8.0, avg: 26.6, max: 40.0) [2023-10-10 18:12:03,789][122664] Avg episode reward: [(0, '46.460'), (1, '50.560')] [2023-10-10 18:12:04,087][123582] Updated weights for policy 0, policy_version 41613 (0.0009) [2023-10-10 18:12:04,457][123582] Updated weights for policy 0, policy_version 41623 (0.0010) [2023-10-10 18:12:07,190][123614] Updated weights for policy 1, policy_version 41540 (0.0009) [2023-10-10 18:12:07,566][123614] Updated weights for policy 1, policy_version 41550 (0.0008) [2023-10-10 18:12:07,934][123614] Updated weights for policy 1, policy_version 41560 (0.0008) [2023-10-10 18:12:08,146][123582] Updated weights for policy 0, policy_version 41633 (0.0009) [2023-10-10 18:12:08,520][123582] Updated weights for policy 0, policy_version 41643 (0.0007) [2023-10-10 18:12:08,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 85196800. Throughput: 0: 1826.4, 1: 1811.2. Samples: 21311328. Policy #0 lag: (min: 8.0, avg: 26.6, max: 40.0) [2023-10-10 18:12:08,788][122664] Avg episode reward: [(0, '46.170'), (1, '50.180')] [2023-10-10 18:12:08,896][123582] Updated weights for policy 0, policy_version 41653 (0.0007) [2023-10-10 18:12:09,273][123582] Updated weights for policy 0, policy_version 41663 (0.0008) [2023-10-10 18:12:11,534][123614] Updated weights for policy 1, policy_version 41570 (0.0009) [2023-10-10 18:12:11,905][123614] Updated weights for policy 1, policy_version 41580 (0.0008) [2023-10-10 18:12:12,270][123614] Updated weights for policy 1, policy_version 41590 (0.0007) [2023-10-10 18:12:12,643][123614] Updated weights for policy 1, policy_version 41600 (0.0008) [2023-10-10 18:12:13,131][123582] Updated weights for policy 0, policy_version 41673 (0.0009) [2023-10-10 18:12:13,508][123582] Updated weights for policy 0, policy_version 41683 (0.0009) [2023-10-10 18:12:13,788][122664] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 85262336. Throughput: 0: 1827.9, 1: 1812.8. Samples: 21322622. Policy #0 lag: (min: 8.0, avg: 26.6, max: 40.0) [2023-10-10 18:12:13,788][122664] Avg episode reward: [(0, '47.470'), (1, '51.310')] [2023-10-10 18:12:13,883][123582] Updated weights for policy 0, policy_version 41693 (0.0009) [2023-10-10 18:12:16,410][123614] Updated weights for policy 1, policy_version 41610 (0.0007) [2023-10-10 18:12:16,780][123614] Updated weights for policy 1, policy_version 41620 (0.0007) [2023-10-10 18:12:17,143][123614] Updated weights for policy 1, policy_version 41630 (0.0009) [2023-10-10 18:12:17,616][123582] Updated weights for policy 0, policy_version 41703 (0.0010) [2023-10-10 18:12:17,992][123582] Updated weights for policy 0, policy_version 41713 (0.0008) [2023-10-10 18:12:18,370][123582] Updated weights for policy 0, policy_version 41723 (0.0007) [2023-10-10 18:12:18,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 85360640. Throughput: 0: 1822.3, 1: 1811.5. Samples: 21344010. Policy #0 lag: (min: 8.0, avg: 26.6, max: 40.0) [2023-10-10 18:12:18,788][122664] Avg episode reward: [(0, '49.240'), (1, '49.330')] [2023-10-10 18:12:20,912][123614] Updated weights for policy 1, policy_version 41640 (0.0009) [2023-10-10 18:12:21,287][123614] Updated weights for policy 1, policy_version 41650 (0.0008) [2023-10-10 18:12:21,652][123614] Updated weights for policy 1, policy_version 41660 (0.0008) [2023-10-10 18:12:22,085][123582] Updated weights for policy 0, policy_version 41733 (0.0008) [2023-10-10 18:12:22,459][123582] Updated weights for policy 0, policy_version 41743 (0.0008) [2023-10-10 18:12:22,841][123582] Updated weights for policy 0, policy_version 41753 (0.0009) [2023-10-10 18:12:23,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 85426176. Throughput: 0: 1823.0, 1: 1811.1. Samples: 21365484. Policy #0 lag: (min: 25.0, avg: 25.0, max: 29.0) [2023-10-10 18:12:23,788][122664] Avg episode reward: [(0, '48.380'), (1, '48.650')] [2023-10-10 18:12:25,345][123614] Updated weights for policy 1, policy_version 41670 (0.0009) [2023-10-10 18:12:25,717][123614] Updated weights for policy 1, policy_version 41680 (0.0009) [2023-10-10 18:12:26,079][123614] Updated weights for policy 1, policy_version 41690 (0.0009) [2023-10-10 18:12:26,463][123582] Updated weights for policy 0, policy_version 41763 (0.0009) [2023-10-10 18:12:26,834][123582] Updated weights for policy 0, policy_version 41773 (0.0007) [2023-10-10 18:12:27,204][123582] Updated weights for policy 0, policy_version 41783 (0.0008) [2023-10-10 18:12:28,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 85491712. Throughput: 0: 1819.5, 1: 1810.3. Samples: 21376866. Policy #0 lag: (min: 25.0, avg: 25.0, max: 29.0) [2023-10-10 18:12:28,789][122664] Avg episode reward: [(0, '49.490'), (1, '49.710')] [2023-10-10 18:12:29,843][123614] Updated weights for policy 1, policy_version 41700 (0.0008) [2023-10-10 18:12:30,201][123614] Updated weights for policy 1, policy_version 41710 (0.0009) [2023-10-10 18:12:30,574][123614] Updated weights for policy 1, policy_version 41720 (0.0010) [2023-10-10 18:12:30,816][123582] Updated weights for policy 0, policy_version 41793 (0.0008) [2023-10-10 18:12:31,184][123582] Updated weights for policy 0, policy_version 41803 (0.0007) [2023-10-10 18:12:31,562][123582] Updated weights for policy 0, policy_version 41813 (0.0007) [2023-10-10 18:12:31,928][123582] Updated weights for policy 0, policy_version 41823 (0.0007) [2023-10-10 18:12:33,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 85557248. Throughput: 0: 1823.7, 1: 1802.5. Samples: 21398684. Policy #0 lag: (min: 25.0, avg: 25.0, max: 29.0) [2023-10-10 18:12:33,789][122664] Avg episode reward: [(0, '49.270'), (1, '45.640')] [2023-10-10 18:12:34,211][123614] Updated weights for policy 1, policy_version 41730 (0.0007) [2023-10-10 18:12:34,574][123614] Updated weights for policy 1, policy_version 41740 (0.0011) [2023-10-10 18:12:34,948][123614] Updated weights for policy 1, policy_version 41750 (0.0009) [2023-10-10 18:12:35,323][123614] Updated weights for policy 1, policy_version 41760 (0.0011) [2023-10-10 18:12:35,821][123582] Updated weights for policy 0, policy_version 41833 (0.0007) [2023-10-10 18:12:36,188][123582] Updated weights for policy 0, policy_version 41843 (0.0008) [2023-10-10 18:12:36,563][123582] Updated weights for policy 0, policy_version 41853 (0.0009) [2023-10-10 18:12:38,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 85622784. Throughput: 0: 1814.3, 1: 1813.7. Samples: 21420878. Policy #0 lag: (min: 25.0, avg: 25.0, max: 29.0) [2023-10-10 18:12:38,788][122664] Avg episode reward: [(0, '48.780'), (1, '47.570')] [2023-10-10 18:12:38,893][123614] Updated weights for policy 1, policy_version 41770 (0.0012) [2023-10-10 18:12:39,257][123614] Updated weights for policy 1, policy_version 41780 (0.0010) [2023-10-10 18:12:39,627][123614] Updated weights for policy 1, policy_version 41790 (0.0008) [2023-10-10 18:12:40,300][123582] Updated weights for policy 0, policy_version 41863 (0.0009) [2023-10-10 18:12:40,671][123582] Updated weights for policy 0, policy_version 41873 (0.0008) [2023-10-10 18:12:41,034][123582] Updated weights for policy 0, policy_version 41883 (0.0008) [2023-10-10 18:12:43,430][123614] Updated weights for policy 1, policy_version 41800 (0.0009) [2023-10-10 18:12:43,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 85688320. Throughput: 0: 1812.8, 1: 1808.9. Samples: 21431086. Policy #0 lag: (min: 25.0, avg: 25.0, max: 29.0) [2023-10-10 18:12:43,789][122664] Avg episode reward: [(0, '50.200'), (1, '48.220')] [2023-10-10 18:12:43,804][123614] Updated weights for policy 1, policy_version 41810 (0.0010) [2023-10-10 18:12:44,173][123614] Updated weights for policy 1, policy_version 41820 (0.0008) [2023-10-10 18:12:44,688][123582] Updated weights for policy 0, policy_version 41893 (0.0008) [2023-10-10 18:12:45,054][123582] Updated weights for policy 0, policy_version 41903 (0.0009) [2023-10-10 18:12:45,418][123582] Updated weights for policy 0, policy_version 41913 (0.0011) [2023-10-10 18:12:47,827][123614] Updated weights for policy 1, policy_version 41830 (0.0010) [2023-10-10 18:12:48,200][123614] Updated weights for policy 1, policy_version 41840 (0.0010) [2023-10-10 18:12:48,564][123614] Updated weights for policy 1, policy_version 41850 (0.0007) [2023-10-10 18:12:48,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 85753856. Throughput: 0: 1815.9, 1: 1820.9. Samples: 21453858. Policy #0 lag: (min: 25.0, avg: 25.0, max: 29.0) [2023-10-10 18:12:48,789][122664] Avg episode reward: [(0, '48.020'), (1, '48.130')] [2023-10-10 18:12:49,072][123582] Updated weights for policy 0, policy_version 41923 (0.0009) [2023-10-10 18:12:49,439][123582] Updated weights for policy 0, policy_version 41933 (0.0007) [2023-10-10 18:12:49,812][123582] Updated weights for policy 0, policy_version 41943 (0.0011) [2023-10-10 18:12:52,207][123614] Updated weights for policy 1, policy_version 41860 (0.0009) [2023-10-10 18:12:52,577][123614] Updated weights for policy 1, policy_version 41870 (0.0010) [2023-10-10 18:12:52,949][123614] Updated weights for policy 1, policy_version 41880 (0.0008) [2023-10-10 18:12:53,500][123582] Updated weights for policy 0, policy_version 41953 (0.0011) [2023-10-10 18:12:53,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 85852160. Throughput: 0: 1821.2, 1: 1825.2. Samples: 21475418. Policy #0 lag: (min: 25.0, avg: 25.0, max: 29.0) [2023-10-10 18:12:53,788][122664] Avg episode reward: [(0, '48.110'), (1, '52.060')] [2023-10-10 18:12:53,877][123582] Updated weights for policy 0, policy_version 41963 (0.0008) [2023-10-10 18:12:54,244][123582] Updated weights for policy 0, policy_version 41973 (0.0008) [2023-10-10 18:12:54,616][123582] Updated weights for policy 0, policy_version 41983 (0.0008) [2023-10-10 18:12:56,616][123614] Updated weights for policy 1, policy_version 41890 (0.0008) [2023-10-10 18:12:56,992][123614] Updated weights for policy 1, policy_version 41900 (0.0009) [2023-10-10 18:12:57,358][123614] Updated weights for policy 1, policy_version 41910 (0.0007) [2023-10-10 18:12:57,738][123614] Updated weights for policy 1, policy_version 41920 (0.0008) [2023-10-10 18:12:58,200][123582] Updated weights for policy 0, policy_version 41993 (0.0009) [2023-10-10 18:12:58,575][123582] Updated weights for policy 0, policy_version 42003 (0.0007) [2023-10-10 18:12:58,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 85917696. Throughput: 0: 1816.0, 1: 1828.0. Samples: 21486606. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 18:12:58,789][122664] Avg episode reward: [(0, '48.250'), (1, '50.770')] [2023-10-10 18:12:58,953][123582] Updated weights for policy 0, policy_version 42013 (0.0007) [2023-10-10 18:13:01,587][123614] Updated weights for policy 1, policy_version 41930 (0.0007) [2023-10-10 18:13:01,957][123614] Updated weights for policy 1, policy_version 41940 (0.0007) [2023-10-10 18:13:02,316][123614] Updated weights for policy 1, policy_version 41950 (0.0009) [2023-10-10 18:13:02,673][123582] Updated weights for policy 0, policy_version 42023 (0.0007) [2023-10-10 18:13:03,045][123582] Updated weights for policy 0, policy_version 42033 (0.0010) [2023-10-10 18:13:03,423][123582] Updated weights for policy 0, policy_version 42043 (0.0009) [2023-10-10 18:13:03,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 86016000. Throughput: 0: 1823.9, 1: 1820.4. Samples: 21508002. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 18:13:03,788][122664] Avg episode reward: [(0, '47.930'), (1, '57.660')] [2023-10-10 18:13:06,018][123614] Updated weights for policy 1, policy_version 41960 (0.0008) [2023-10-10 18:13:06,384][123614] Updated weights for policy 1, policy_version 41970 (0.0009) [2023-10-10 18:13:06,753][123614] Updated weights for policy 1, policy_version 41980 (0.0007) [2023-10-10 18:13:07,039][123582] Updated weights for policy 0, policy_version 42053 (0.0008) [2023-10-10 18:13:07,415][123582] Updated weights for policy 0, policy_version 42063 (0.0009) [2023-10-10 18:13:07,774][123582] Updated weights for policy 0, policy_version 42073 (0.0008) [2023-10-10 18:13:08,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 86081536. Throughput: 0: 1828.3, 1: 1818.5. Samples: 21529594. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 18:13:08,789][122664] Avg episode reward: [(0, '43.360'), (1, '57.610')] [2023-10-10 18:13:10,510][123614] Updated weights for policy 1, policy_version 41990 (0.0009) [2023-10-10 18:13:10,869][123614] Updated weights for policy 1, policy_version 42000 (0.0010) [2023-10-10 18:13:11,231][123614] Updated weights for policy 1, policy_version 42010 (0.0009) [2023-10-10 18:13:11,507][123582] Updated weights for policy 0, policy_version 42083 (0.0010) [2023-10-10 18:13:11,876][123582] Updated weights for policy 0, policy_version 42093 (0.0008) [2023-10-10 18:13:12,244][123582] Updated weights for policy 0, policy_version 42103 (0.0007) [2023-10-10 18:13:13,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 86147072. Throughput: 0: 1825.5, 1: 1815.6. Samples: 21540716. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 18:13:13,789][122664] Avg episode reward: [(0, '42.500'), (1, '55.150')] [2023-10-10 18:13:15,013][123614] Updated weights for policy 1, policy_version 42020 (0.0008) [2023-10-10 18:13:15,378][123614] Updated weights for policy 1, policy_version 42030 (0.0010) [2023-10-10 18:13:15,747][123614] Updated weights for policy 1, policy_version 42040 (0.0009) [2023-10-10 18:13:15,854][123582] Updated weights for policy 0, policy_version 42113 (0.0007) [2023-10-10 18:13:16,218][123582] Updated weights for policy 0, policy_version 42123 (0.0008) [2023-10-10 18:13:16,587][123582] Updated weights for policy 0, policy_version 42133 (0.0008) [2023-10-10 18:13:16,970][123582] Updated weights for policy 0, policy_version 42143 (0.0010) [2023-10-10 18:13:18,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 86212608. Throughput: 0: 1825.1, 1: 1812.9. Samples: 21562394. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 18:13:18,789][122664] Avg episode reward: [(0, '44.080'), (1, '56.600')] [2023-10-10 18:13:19,399][123614] Updated weights for policy 1, policy_version 42050 (0.0008) [2023-10-10 18:13:19,754][123614] Updated weights for policy 1, policy_version 42060 (0.0010) [2023-10-10 18:13:20,120][123614] Updated weights for policy 1, policy_version 42070 (0.0008) [2023-10-10 18:13:20,489][123614] Updated weights for policy 1, policy_version 42080 (0.0009) [2023-10-10 18:13:20,873][123582] Updated weights for policy 0, policy_version 42153 (0.0011) [2023-10-10 18:13:21,256][123582] Updated weights for policy 0, policy_version 42163 (0.0010) [2023-10-10 18:13:21,625][123582] Updated weights for policy 0, policy_version 42173 (0.0008) [2023-10-10 18:13:23,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 86278144. Throughput: 0: 1822.3, 1: 1814.9. Samples: 21584552. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 18:13:23,789][122664] Avg episode reward: [(0, '43.680'), (1, '59.800')] [2023-10-10 18:13:24,004][123614] Updated weights for policy 1, policy_version 42090 (0.0010) [2023-10-10 18:13:24,371][123614] Updated weights for policy 1, policy_version 42100 (0.0011) [2023-10-10 18:13:24,751][123614] Updated weights for policy 1, policy_version 42110 (0.0011) [2023-10-10 18:13:25,171][123582] Updated weights for policy 0, policy_version 42183 (0.0007) [2023-10-10 18:13:25,547][123582] Updated weights for policy 0, policy_version 42193 (0.0007) [2023-10-10 18:13:25,924][123582] Updated weights for policy 0, policy_version 42203 (0.0009) [2023-10-10 18:13:28,601][123614] Updated weights for policy 1, policy_version 42120 (0.0007) [2023-10-10 18:13:28,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 86343680. Throughput: 0: 1824.8, 1: 1811.0. Samples: 21594698. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) [2023-10-10 18:13:28,788][122664] Avg episode reward: [(0, '43.370'), (1, '62.020')] [2023-10-10 18:13:28,967][123614] Updated weights for policy 1, policy_version 42130 (0.0008) [2023-10-10 18:13:29,336][123614] Updated weights for policy 1, policy_version 42140 (0.0009) [2023-10-10 18:13:29,474][123465] Saving new best policy, reward=62.020! [2023-10-10 18:13:29,563][123582] Updated weights for policy 0, policy_version 42213 (0.0009) [2023-10-10 18:13:29,941][123582] Updated weights for policy 0, policy_version 42223 (0.0009) [2023-10-10 18:13:30,303][123582] Updated weights for policy 0, policy_version 42233 (0.0009) [2023-10-10 18:13:32,988][123614] Updated weights for policy 1, policy_version 42150 (0.0011) [2023-10-10 18:13:33,355][123614] Updated weights for policy 1, policy_version 42160 (0.0010) [2023-10-10 18:13:33,714][123614] Updated weights for policy 1, policy_version 42170 (0.0008) [2023-10-10 18:13:33,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 86409216. Throughput: 0: 1824.9, 1: 1814.3. Samples: 21617624. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) [2023-10-10 18:13:33,788][122664] Avg episode reward: [(0, '44.370'), (1, '63.380')] [2023-10-10 18:13:33,931][123465] Saving new best policy, reward=63.380! [2023-10-10 18:13:33,932][123582] Updated weights for policy 0, policy_version 42243 (0.0008) [2023-10-10 18:13:34,305][123582] Updated weights for policy 0, policy_version 42253 (0.0008) [2023-10-10 18:13:34,670][123582] Updated weights for policy 0, policy_version 42263 (0.0009) [2023-10-10 18:13:37,567][123614] Updated weights for policy 1, policy_version 42180 (0.0008) [2023-10-10 18:13:37,935][123614] Updated weights for policy 1, policy_version 42190 (0.0010) [2023-10-10 18:13:38,307][123614] Updated weights for policy 1, policy_version 42200 (0.0009) [2023-10-10 18:13:38,453][123582] Updated weights for policy 0, policy_version 42273 (0.0010) [2023-10-10 18:13:38,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 86507520. Throughput: 0: 1825.3, 1: 1804.2. Samples: 21638746. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) [2023-10-10 18:13:38,788][122664] Avg episode reward: [(0, '43.130'), (1, '64.020')] [2023-10-10 18:13:38,797][123465] Saving new best policy, reward=64.020! [2023-10-10 18:13:38,841][123582] Updated weights for policy 0, policy_version 42283 (0.0007) [2023-10-10 18:13:39,210][123582] Updated weights for policy 0, policy_version 42293 (0.0008) [2023-10-10 18:13:39,576][123582] Updated weights for policy 0, policy_version 42303 (0.0008) [2023-10-10 18:13:41,972][123614] Updated weights for policy 1, policy_version 42210 (0.0010) [2023-10-10 18:13:42,339][123614] Updated weights for policy 1, policy_version 42220 (0.0008) [2023-10-10 18:13:42,697][123614] Updated weights for policy 1, policy_version 42230 (0.0007) [2023-10-10 18:13:43,065][123614] Updated weights for policy 1, policy_version 42240 (0.0009) [2023-10-10 18:13:43,340][123582] Updated weights for policy 0, policy_version 42313 (0.0009) [2023-10-10 18:13:43,714][123582] Updated weights for policy 0, policy_version 42323 (0.0008) [2023-10-10 18:13:43,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 86573056. Throughput: 0: 1824.3, 1: 1809.9. Samples: 21650144. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) [2023-10-10 18:13:43,788][122664] Avg episode reward: [(0, '43.190'), (1, '63.810')] [2023-10-10 18:13:44,088][123582] Updated weights for policy 0, policy_version 42333 (0.0009) [2023-10-10 18:13:46,784][123614] Updated weights for policy 1, policy_version 42250 (0.0008) [2023-10-10 18:13:47,155][123614] Updated weights for policy 1, policy_version 42260 (0.0009) [2023-10-10 18:13:47,519][123614] Updated weights for policy 1, policy_version 42270 (0.0008) [2023-10-10 18:13:47,610][123582] Updated weights for policy 0, policy_version 42343 (0.0009) [2023-10-10 18:13:47,973][123582] Updated weights for policy 0, policy_version 42353 (0.0008) [2023-10-10 18:13:48,361][123582] Updated weights for policy 0, policy_version 42363 (0.0009) [2023-10-10 18:13:48,788][122664] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 86671360. Throughput: 0: 1825.6, 1: 1812.1. Samples: 21671698. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) [2023-10-10 18:13:48,789][122664] Avg episode reward: [(0, '43.150'), (1, '62.600')] [2023-10-10 18:13:51,280][123614] Updated weights for policy 1, policy_version 42280 (0.0007) [2023-10-10 18:13:51,649][123614] Updated weights for policy 1, policy_version 42290 (0.0007) [2023-10-10 18:13:52,010][123614] Updated weights for policy 1, policy_version 42300 (0.0008) [2023-10-10 18:13:52,018][123582] Updated weights for policy 0, policy_version 42373 (0.0008) [2023-10-10 18:13:52,388][123582] Updated weights for policy 0, policy_version 42383 (0.0008) [2023-10-10 18:13:52,768][123582] Updated weights for policy 0, policy_version 42393 (0.0009) [2023-10-10 18:13:53,788][122664] Fps is (10 sec: 16383.4, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 86736896. Throughput: 0: 1821.9, 1: 1813.7. Samples: 21693196. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) [2023-10-10 18:13:53,789][122664] Avg episode reward: [(0, '45.980'), (1, '59.380')] [2023-10-10 18:13:53,801][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000042304_43319296.pth... [2023-10-10 18:13:53,802][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000042400_43417600.pth... [2023-10-10 18:13:53,837][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000040608_41582592.pth [2023-10-10 18:13:53,837][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000040704_41680896.pth [2023-10-10 18:13:53,841][123465] Saving a milestone ./train_atari/atari_demonattack_APPO/checkpoint_p1/milestones/checkpoint_000042304_43319296.pth [2023-10-10 18:13:53,841][123247] Saving a milestone ./train_atari/atari_demonattack_APPO/checkpoint_p0/milestones/checkpoint_000042400_43417600.pth [2023-10-10 18:13:55,766][123614] Updated weights for policy 1, policy_version 42310 (0.0008) [2023-10-10 18:13:56,126][123614] Updated weights for policy 1, policy_version 42320 (0.0007) [2023-10-10 18:13:56,424][123582] Updated weights for policy 0, policy_version 42403 (0.0009) [2023-10-10 18:13:56,496][123614] Updated weights for policy 1, policy_version 42330 (0.0008) [2023-10-10 18:13:56,794][123582] Updated weights for policy 0, policy_version 42413 (0.0009) [2023-10-10 18:13:57,171][123582] Updated weights for policy 0, policy_version 42423 (0.0010) [2023-10-10 18:13:58,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 86802432. Throughput: 0: 1819.7, 1: 1816.4. Samples: 21704338. Policy #0 lag: (min: 24.0, avg: 49.2, max: 56.0) [2023-10-10 18:13:58,789][122664] Avg episode reward: [(0, '49.870'), (1, '49.290')] [2023-10-10 18:14:00,034][123614] Updated weights for policy 1, policy_version 42340 (0.0007) [2023-10-10 18:14:00,398][123614] Updated weights for policy 1, policy_version 42350 (0.0010) [2023-10-10 18:14:00,768][123614] Updated weights for policy 1, policy_version 42360 (0.0008) [2023-10-10 18:14:00,776][123582] Updated weights for policy 0, policy_version 42433 (0.0008) [2023-10-10 18:14:01,145][123582] Updated weights for policy 0, policy_version 42443 (0.0007) [2023-10-10 18:14:01,524][123582] Updated weights for policy 0, policy_version 42453 (0.0007) [2023-10-10 18:14:01,907][123582] Updated weights for policy 0, policy_version 42463 (0.0009) [2023-10-10 18:14:03,788][122664] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 86867968. Throughput: 0: 1817.2, 1: 1817.3. Samples: 21725946. Policy #0 lag: (min: 24.0, avg: 49.2, max: 56.0) [2023-10-10 18:14:03,788][122664] Avg episode reward: [(0, '52.770'), (1, '50.730')] [2023-10-10 18:14:04,483][123614] Updated weights for policy 1, policy_version 42370 (0.0008) [2023-10-10 18:14:04,857][123614] Updated weights for policy 1, policy_version 42380 (0.0008) [2023-10-10 18:14:05,230][123614] Updated weights for policy 1, policy_version 42390 (0.0009) [2023-10-10 18:14:05,594][123614] Updated weights for policy 1, policy_version 42400 (0.0008) [2023-10-10 18:14:05,674][123582] Updated weights for policy 0, policy_version 42473 (0.0009) [2023-10-10 18:14:06,054][123582] Updated weights for policy 0, policy_version 42483 (0.0009) [2023-10-10 18:14:06,431][123582] Updated weights for policy 0, policy_version 42493 (0.0009) [2023-10-10 18:14:08,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 86933504. Throughput: 0: 1826.2, 1: 1819.7. Samples: 21748616. Policy #0 lag: (min: 24.0, avg: 49.2, max: 56.0) [2023-10-10 18:14:08,789][122664] Avg episode reward: [(0, '56.410'), (1, '52.240')] [2023-10-10 18:14:09,392][123614] Updated weights for policy 1, policy_version 42410 (0.0009) [2023-10-10 18:14:09,764][123614] Updated weights for policy 1, policy_version 42420 (0.0007) [2023-10-10 18:14:09,951][123582] Updated weights for policy 0, policy_version 42503 (0.0010) [2023-10-10 18:14:10,130][123614] Updated weights for policy 1, policy_version 42430 (0.0007) [2023-10-10 18:14:10,324][123582] Updated weights for policy 0, policy_version 42513 (0.0009) [2023-10-10 18:14:10,690][123582] Updated weights for policy 0, policy_version 42523 (0.0008) [2023-10-10 18:14:13,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 86999040. Throughput: 0: 1826.3, 1: 1816.0. Samples: 21758600. Policy #0 lag: (min: 24.0, avg: 49.2, max: 56.0) [2023-10-10 18:14:13,789][122664] Avg episode reward: [(0, '58.770'), (1, '51.970')] [2023-10-10 18:14:13,790][123247] Saving new best policy, reward=58.770! [2023-10-10 18:14:13,895][123614] Updated weights for policy 1, policy_version 42440 (0.0011) [2023-10-10 18:14:14,268][123614] Updated weights for policy 1, policy_version 42450 (0.0007) [2023-10-10 18:14:14,330][123582] Updated weights for policy 0, policy_version 42533 (0.0010) [2023-10-10 18:14:14,641][123614] Updated weights for policy 1, policy_version 42460 (0.0008) [2023-10-10 18:14:14,700][123582] Updated weights for policy 0, policy_version 42543 (0.0008) [2023-10-10 18:14:15,078][123582] Updated weights for policy 0, policy_version 42553 (0.0010) [2023-10-10 18:14:18,433][123614] Updated weights for policy 1, policy_version 42470 (0.0007) [2023-10-10 18:14:18,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 87064576. Throughput: 0: 1823.6, 1: 1811.4. Samples: 21781198. Policy #0 lag: (min: 24.0, avg: 49.2, max: 56.0) [2023-10-10 18:14:18,788][122664] Avg episode reward: [(0, '60.750'), (1, '48.340')] [2023-10-10 18:14:18,798][123614] Updated weights for policy 1, policy_version 42480 (0.0007) [2023-10-10 18:14:18,874][123582] Updated weights for policy 0, policy_version 42563 (0.0008) [2023-10-10 18:14:19,173][123614] Updated weights for policy 1, policy_version 42490 (0.0007) [2023-10-10 18:14:19,246][123582] Updated weights for policy 0, policy_version 42573 (0.0008) [2023-10-10 18:14:19,637][123582] Updated weights for policy 0, policy_version 42583 (0.0009) [2023-10-10 18:14:19,972][123247] Saving new best policy, reward=60.750! [2023-10-10 18:14:22,721][123614] Updated weights for policy 1, policy_version 42500 (0.0007) [2023-10-10 18:14:23,092][123614] Updated weights for policy 1, policy_version 42510 (0.0010) [2023-10-10 18:14:23,155][123582] Updated weights for policy 0, policy_version 42593 (0.0009) [2023-10-10 18:14:23,469][123614] Updated weights for policy 1, policy_version 42520 (0.0008) [2023-10-10 18:14:23,530][123582] Updated weights for policy 0, policy_version 42603 (0.0008) [2023-10-10 18:14:23,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 87162880. Throughput: 0: 1816.9, 1: 1820.6. Samples: 21802434. Policy #0 lag: (min: 24.0, avg: 49.2, max: 56.0) [2023-10-10 18:14:23,788][122664] Avg episode reward: [(0, '58.470'), (1, '43.560')] [2023-10-10 18:14:23,897][123582] Updated weights for policy 0, policy_version 42613 (0.0009) [2023-10-10 18:14:24,273][123582] Updated weights for policy 0, policy_version 42623 (0.0010) [2023-10-10 18:14:26,996][123614] Updated weights for policy 1, policy_version 42530 (0.0007) [2023-10-10 18:14:27,363][123614] Updated weights for policy 1, policy_version 42540 (0.0008) [2023-10-10 18:14:27,734][123614] Updated weights for policy 1, policy_version 42550 (0.0008) [2023-10-10 18:14:27,976][123582] Updated weights for policy 0, policy_version 42633 (0.0007) [2023-10-10 18:14:28,093][123614] Updated weights for policy 1, policy_version 42560 (0.0009) [2023-10-10 18:14:28,349][123582] Updated weights for policy 0, policy_version 42643 (0.0010) [2023-10-10 18:14:28,724][123582] Updated weights for policy 0, policy_version 42653 (0.0007) [2023-10-10 18:14:28,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 87228416. Throughput: 0: 1828.5, 1: 1818.8. Samples: 21814274. Policy #0 lag: (min: 24.0, avg: 49.2, max: 56.0) [2023-10-10 18:14:28,789][122664] Avg episode reward: [(0, '61.460'), (1, '41.240')] [2023-10-10 18:14:28,835][123247] Saving new best policy, reward=61.460! [2023-10-10 18:14:31,763][123614] Updated weights for policy 1, policy_version 42570 (0.0009) [2023-10-10 18:14:32,122][123614] Updated weights for policy 1, policy_version 42580 (0.0008) [2023-10-10 18:14:32,493][123582] Updated weights for policy 0, policy_version 42663 (0.0008) [2023-10-10 18:14:32,495][123614] Updated weights for policy 1, policy_version 42590 (0.0008) [2023-10-10 18:14:32,856][123582] Updated weights for policy 0, policy_version 42673 (0.0007) [2023-10-10 18:14:33,238][123582] Updated weights for policy 0, policy_version 42683 (0.0011) [2023-10-10 18:14:33,788][122664] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 87326720. Throughput: 0: 1819.7, 1: 1816.7. Samples: 21835336. Policy #0 lag: (min: 31.0, avg: 32.4, max: 56.0) [2023-10-10 18:14:33,789][122664] Avg episode reward: [(0, '62.630'), (1, '42.790')] [2023-10-10 18:14:33,790][123247] Saving new best policy, reward=62.630! [2023-10-10 18:14:36,209][123614] Updated weights for policy 1, policy_version 42600 (0.0007) [2023-10-10 18:14:36,589][123614] Updated weights for policy 1, policy_version 42610 (0.0008) [2023-10-10 18:14:36,932][123582] Updated weights for policy 0, policy_version 42693 (0.0008) [2023-10-10 18:14:36,959][123614] Updated weights for policy 1, policy_version 42620 (0.0008) [2023-10-10 18:14:37,312][123582] Updated weights for policy 0, policy_version 42703 (0.0009) [2023-10-10 18:14:37,683][123582] Updated weights for policy 0, policy_version 42713 (0.0009) [2023-10-10 18:14:38,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 87392256. Throughput: 0: 1819.3, 1: 1818.9. Samples: 21856914. Policy #0 lag: (min: 31.0, avg: 32.4, max: 56.0) [2023-10-10 18:14:38,789][122664] Avg episode reward: [(0, '64.870'), (1, '43.380')] [2023-10-10 18:14:38,799][123247] Saving new best policy, reward=64.870! [2023-10-10 18:14:40,647][123614] Updated weights for policy 1, policy_version 42630 (0.0007) [2023-10-10 18:14:41,015][123614] Updated weights for policy 1, policy_version 42640 (0.0007) [2023-10-10 18:14:41,282][123582] Updated weights for policy 0, policy_version 42723 (0.0009) [2023-10-10 18:14:41,384][123614] Updated weights for policy 1, policy_version 42650 (0.0007) [2023-10-10 18:14:41,660][123582] Updated weights for policy 0, policy_version 42733 (0.0009) [2023-10-10 18:14:42,021][123582] Updated weights for policy 0, policy_version 42743 (0.0007) [2023-10-10 18:14:43,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 87457792. Throughput: 0: 1816.9, 1: 1819.1. Samples: 21867958. Policy #0 lag: (min: 31.0, avg: 32.4, max: 56.0) [2023-10-10 18:14:43,789][122664] Avg episode reward: [(0, '65.920'), (1, '45.970')] [2023-10-10 18:14:43,789][123247] Saving new best policy, reward=65.920! [2023-10-10 18:14:45,080][123614] Updated weights for policy 1, policy_version 42660 (0.0010) [2023-10-10 18:14:45,452][123614] Updated weights for policy 1, policy_version 42670 (0.0009) [2023-10-10 18:14:45,645][123582] Updated weights for policy 0, policy_version 42753 (0.0009) [2023-10-10 18:14:45,829][123614] Updated weights for policy 1, policy_version 42680 (0.0008) [2023-10-10 18:14:46,005][123582] Updated weights for policy 0, policy_version 42763 (0.0008) [2023-10-10 18:14:46,376][123582] Updated weights for policy 0, policy_version 42773 (0.0008) [2023-10-10 18:14:46,742][123582] Updated weights for policy 0, policy_version 42783 (0.0009) [2023-10-10 18:14:48,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 87523328. Throughput: 0: 1820.4, 1: 1815.5. Samples: 21889560. Policy #0 lag: (min: 31.0, avg: 32.4, max: 56.0) [2023-10-10 18:14:48,789][122664] Avg episode reward: [(0, '67.640'), (1, '46.860')] [2023-10-10 18:14:48,790][123247] Saving new best policy, reward=67.640! [2023-10-10 18:14:49,614][123614] Updated weights for policy 1, policy_version 42690 (0.0007) [2023-10-10 18:14:49,980][123614] Updated weights for policy 1, policy_version 42700 (0.0009) [2023-10-10 18:14:50,349][123614] Updated weights for policy 1, policy_version 42710 (0.0009) [2023-10-10 18:14:50,575][123582] Updated weights for policy 0, policy_version 42793 (0.0009) [2023-10-10 18:14:50,720][123614] Updated weights for policy 1, policy_version 42720 (0.0008) [2023-10-10 18:14:50,943][123582] Updated weights for policy 0, policy_version 42803 (0.0009) [2023-10-10 18:14:51,309][123582] Updated weights for policy 0, policy_version 42813 (0.0011) [2023-10-10 18:14:53,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 87588864. Throughput: 0: 1813.8, 1: 1814.1. Samples: 21911870. Policy #0 lag: (min: 31.0, avg: 32.4, max: 56.0) [2023-10-10 18:14:53,789][122664] Avg episode reward: [(0, '67.490'), (1, '44.130')] [2023-10-10 18:14:54,365][123614] Updated weights for policy 1, policy_version 42730 (0.0008) [2023-10-10 18:14:54,737][123614] Updated weights for policy 1, policy_version 42740 (0.0008) [2023-10-10 18:14:55,097][123614] Updated weights for policy 1, policy_version 42750 (0.0009) [2023-10-10 18:14:55,296][123582] Updated weights for policy 0, policy_version 42823 (0.0008) [2023-10-10 18:14:55,673][123582] Updated weights for policy 0, policy_version 42833 (0.0009) [2023-10-10 18:14:56,055][123582] Updated weights for policy 0, policy_version 42843 (0.0010) [2023-10-10 18:14:58,744][123614] Updated weights for policy 1, policy_version 42760 (0.0007) [2023-10-10 18:14:58,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 87654400. Throughput: 0: 1808.4, 1: 1816.4. Samples: 21921716. Policy #0 lag: (min: 31.0, avg: 32.4, max: 56.0) [2023-10-10 18:14:58,788][122664] Avg episode reward: [(0, '68.680'), (1, '42.390')] [2023-10-10 18:14:58,789][123247] Saving new best policy, reward=68.680! [2023-10-10 18:14:59,121][123614] Updated weights for policy 1, policy_version 42770 (0.0009) [2023-10-10 18:14:59,489][123614] Updated weights for policy 1, policy_version 42780 (0.0010) [2023-10-10 18:14:59,863][123582] Updated weights for policy 0, policy_version 42853 (0.0009) [2023-10-10 18:15:00,235][123582] Updated weights for policy 0, policy_version 42863 (0.0009) [2023-10-10 18:15:00,619][123582] Updated weights for policy 0, policy_version 42873 (0.0007) [2023-10-10 18:15:03,321][123614] Updated weights for policy 1, policy_version 42790 (0.0009) [2023-10-10 18:15:03,684][123614] Updated weights for policy 1, policy_version 42800 (0.0010) [2023-10-10 18:15:03,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 87719936. Throughput: 0: 1802.0, 1: 1816.8. Samples: 21944048. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:15:03,789][122664] Avg episode reward: [(0, '65.570'), (1, '40.700')] [2023-10-10 18:15:04,053][123614] Updated weights for policy 1, policy_version 42810 (0.0007) [2023-10-10 18:15:04,106][123582] Updated weights for policy 0, policy_version 42883 (0.0009) [2023-10-10 18:15:04,473][123582] Updated weights for policy 0, policy_version 42893 (0.0009) [2023-10-10 18:15:04,844][123582] Updated weights for policy 0, policy_version 42903 (0.0007) [2023-10-10 18:15:07,720][123614] Updated weights for policy 1, policy_version 42820 (0.0008) [2023-10-10 18:15:08,088][123614] Updated weights for policy 1, policy_version 42830 (0.0008) [2023-10-10 18:15:08,453][123614] Updated weights for policy 1, policy_version 42840 (0.0008) [2023-10-10 18:15:08,524][123582] Updated weights for policy 0, policy_version 42913 (0.0008) [2023-10-10 18:15:08,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 87818240. Throughput: 0: 1812.9, 1: 1814.6. Samples: 21965674. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:15:08,789][122664] Avg episode reward: [(0, '60.330'), (1, '42.990')] [2023-10-10 18:15:08,894][123582] Updated weights for policy 0, policy_version 42923 (0.0010) [2023-10-10 18:15:09,270][123582] Updated weights for policy 0, policy_version 42933 (0.0008) [2023-10-10 18:15:09,648][123582] Updated weights for policy 0, policy_version 42943 (0.0010) [2023-10-10 18:15:12,152][123614] Updated weights for policy 1, policy_version 42850 (0.0008) [2023-10-10 18:15:12,518][123614] Updated weights for policy 1, policy_version 42860 (0.0007) [2023-10-10 18:15:12,888][123614] Updated weights for policy 1, policy_version 42870 (0.0007) [2023-10-10 18:15:13,259][123614] Updated weights for policy 1, policy_version 42880 (0.0008) [2023-10-10 18:15:13,307][123582] Updated weights for policy 0, policy_version 42953 (0.0007) [2023-10-10 18:15:13,677][123582] Updated weights for policy 0, policy_version 42963 (0.0007) [2023-10-10 18:15:13,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 87883776. Throughput: 0: 1803.3, 1: 1808.8. Samples: 21976816. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:15:13,789][122664] Avg episode reward: [(0, '57.870'), (1, '42.670')] [2023-10-10 18:15:14,053][123582] Updated weights for policy 0, policy_version 42973 (0.0008) [2023-10-10 18:15:16,910][123614] Updated weights for policy 1, policy_version 42890 (0.0007) [2023-10-10 18:15:17,274][123614] Updated weights for policy 1, policy_version 42900 (0.0008) [2023-10-10 18:15:17,646][123614] Updated weights for policy 1, policy_version 42910 (0.0010) [2023-10-10 18:15:17,754][123582] Updated weights for policy 0, policy_version 42983 (0.0009) [2023-10-10 18:15:18,126][123582] Updated weights for policy 0, policy_version 42993 (0.0009) [2023-10-10 18:15:18,498][123582] Updated weights for policy 0, policy_version 43003 (0.0007) [2023-10-10 18:15:18,788][122664] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 87982080. Throughput: 0: 1809.0, 1: 1813.9. Samples: 21998366. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:15:18,789][122664] Avg episode reward: [(0, '59.140'), (1, '45.300')] [2023-10-10 18:15:21,274][123614] Updated weights for policy 1, policy_version 42920 (0.0010) [2023-10-10 18:15:21,642][123614] Updated weights for policy 1, policy_version 42930 (0.0009) [2023-10-10 18:15:22,009][123614] Updated weights for policy 1, policy_version 42940 (0.0008) [2023-10-10 18:15:22,233][123582] Updated weights for policy 0, policy_version 43013 (0.0008) [2023-10-10 18:15:22,602][123582] Updated weights for policy 0, policy_version 43023 (0.0008) [2023-10-10 18:15:22,976][123582] Updated weights for policy 0, policy_version 43033 (0.0011) [2023-10-10 18:15:23,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 88047616. Throughput: 0: 1806.2, 1: 1809.8. Samples: 22019636. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:15:23,789][122664] Avg episode reward: [(0, '60.240'), (1, '45.110')] [2023-10-10 18:15:25,770][123614] Updated weights for policy 1, policy_version 42950 (0.0008) [2023-10-10 18:15:26,136][123614] Updated weights for policy 1, policy_version 42960 (0.0009) [2023-10-10 18:15:26,506][123614] Updated weights for policy 1, policy_version 42970 (0.0009) [2023-10-10 18:15:26,527][123582] Updated weights for policy 0, policy_version 43043 (0.0009) [2023-10-10 18:15:26,901][123582] Updated weights for policy 0, policy_version 43053 (0.0008) [2023-10-10 18:15:27,279][123582] Updated weights for policy 0, policy_version 43063 (0.0011) [2023-10-10 18:15:28,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 88113152. Throughput: 0: 1815.9, 1: 1811.4. Samples: 22031188. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:15:28,788][122664] Avg episode reward: [(0, '58.890'), (1, '44.450')] [2023-10-10 18:15:30,179][123614] Updated weights for policy 1, policy_version 42980 (0.0008) [2023-10-10 18:15:30,551][123614] Updated weights for policy 1, policy_version 42990 (0.0008) [2023-10-10 18:15:30,911][123614] Updated weights for policy 1, policy_version 43000 (0.0007) [2023-10-10 18:15:31,071][123582] Updated weights for policy 0, policy_version 43073 (0.0008) [2023-10-10 18:15:31,446][123582] Updated weights for policy 0, policy_version 43083 (0.0008) [2023-10-10 18:15:31,828][123582] Updated weights for policy 0, policy_version 43093 (0.0009) [2023-10-10 18:15:32,211][123582] Updated weights for policy 0, policy_version 43103 (0.0008) [2023-10-10 18:15:33,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 88178688. Throughput: 0: 1806.4, 1: 1809.2. Samples: 22052258. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-10 18:15:33,789][122664] Avg episode reward: [(0, '57.850'), (1, '46.050')] [2023-10-10 18:15:34,728][123614] Updated weights for policy 1, policy_version 43010 (0.0008) [2023-10-10 18:15:35,085][123614] Updated weights for policy 1, policy_version 43020 (0.0007) [2023-10-10 18:15:35,464][123614] Updated weights for policy 1, policy_version 43030 (0.0007) [2023-10-10 18:15:35,825][123614] Updated weights for policy 1, policy_version 43040 (0.0008) [2023-10-10 18:15:35,869][123582] Updated weights for policy 0, policy_version 43113 (0.0008) [2023-10-10 18:15:36,244][123582] Updated weights for policy 0, policy_version 43123 (0.0008) [2023-10-10 18:15:36,614][123582] Updated weights for policy 0, policy_version 43133 (0.0009) [2023-10-10 18:15:38,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 88244224. Throughput: 0: 1818.1, 1: 1811.0. Samples: 22075176. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-10 18:15:38,788][122664] Avg episode reward: [(0, '56.000'), (1, '45.270')] [2023-10-10 18:15:39,534][123614] Updated weights for policy 1, policy_version 43050 (0.0009) [2023-10-10 18:15:39,896][123614] Updated weights for policy 1, policy_version 43060 (0.0009) [2023-10-10 18:15:40,260][123614] Updated weights for policy 1, policy_version 43070 (0.0009) [2023-10-10 18:15:40,311][123582] Updated weights for policy 0, policy_version 43143 (0.0008) [2023-10-10 18:15:40,684][123582] Updated weights for policy 0, policy_version 43153 (0.0008) [2023-10-10 18:15:41,057][123582] Updated weights for policy 0, policy_version 43163 (0.0009) [2023-10-10 18:15:43,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 88309760. Throughput: 0: 1822.6, 1: 1808.3. Samples: 22085104. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-10 18:15:43,788][122664] Avg episode reward: [(0, '55.350'), (1, '45.150')] [2023-10-10 18:15:43,989][123614] Updated weights for policy 1, policy_version 43080 (0.0008) [2023-10-10 18:15:44,361][123614] Updated weights for policy 1, policy_version 43090 (0.0008) [2023-10-10 18:15:44,730][123614] Updated weights for policy 1, policy_version 43100 (0.0008) [2023-10-10 18:15:44,770][123582] Updated weights for policy 0, policy_version 43173 (0.0007) [2023-10-10 18:15:45,139][123582] Updated weights for policy 0, policy_version 43183 (0.0007) [2023-10-10 18:15:45,515][123582] Updated weights for policy 0, policy_version 43193 (0.0007) [2023-10-10 18:15:48,399][123614] Updated weights for policy 1, policy_version 43110 (0.0010) [2023-10-10 18:15:48,764][123614] Updated weights for policy 1, policy_version 43120 (0.0011) [2023-10-10 18:15:48,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 88375296. Throughput: 0: 1825.5, 1: 1808.6. Samples: 22107580. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-10 18:15:48,789][122664] Avg episode reward: [(0, '58.280'), (1, '46.140')] [2023-10-10 18:15:49,134][123614] Updated weights for policy 1, policy_version 43130 (0.0010) [2023-10-10 18:15:49,247][123582] Updated weights for policy 0, policy_version 43203 (0.0009) [2023-10-10 18:15:49,612][123582] Updated weights for policy 0, policy_version 43213 (0.0007) [2023-10-10 18:15:49,983][123582] Updated weights for policy 0, policy_version 43223 (0.0010) [2023-10-10 18:15:52,916][123614] Updated weights for policy 1, policy_version 43140 (0.0008) [2023-10-10 18:15:53,287][123614] Updated weights for policy 1, policy_version 43150 (0.0007) [2023-10-10 18:15:53,657][123614] Updated weights for policy 1, policy_version 43160 (0.0007) [2023-10-10 18:15:53,707][123582] Updated weights for policy 0, policy_version 43233 (0.0009) [2023-10-10 18:15:53,788][122664] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 88440832. Throughput: 0: 1818.1, 1: 1812.9. Samples: 22129068. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-10 18:15:53,789][122664] Avg episode reward: [(0, '60.300'), (1, '45.330')] [2023-10-10 18:15:53,949][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000043168_44204032.pth... [2023-10-10 18:15:53,977][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000041472_42467328.pth [2023-10-10 18:15:54,079][123582] Updated weights for policy 0, policy_version 43243 (0.0007) [2023-10-10 18:15:54,444][123582] Updated weights for policy 0, policy_version 43253 (0.0008) [2023-10-10 18:15:54,816][123582] Updated weights for policy 0, policy_version 43263 (0.0007) [2023-10-10 18:15:54,851][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000043264_44302336.pth... [2023-10-10 18:15:54,888][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000041536_42532864.pth [2023-10-10 18:15:57,380][123614] Updated weights for policy 1, policy_version 43170 (0.0007) [2023-10-10 18:15:57,740][123614] Updated weights for policy 1, policy_version 43180 (0.0008) [2023-10-10 18:15:58,100][123614] Updated weights for policy 1, policy_version 43190 (0.0010) [2023-10-10 18:15:58,477][123614] Updated weights for policy 1, policy_version 43200 (0.0008) [2023-10-10 18:15:58,598][123582] Updated weights for policy 0, policy_version 43273 (0.0009) [2023-10-10 18:15:58,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 88539136. Throughput: 0: 1819.6, 1: 1810.2. Samples: 22140160. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-10 18:15:58,789][122664] Avg episode reward: [(0, '59.000'), (1, '50.000')] [2023-10-10 18:15:58,961][123582] Updated weights for policy 0, policy_version 43283 (0.0008) [2023-10-10 18:15:59,341][123582] Updated weights for policy 0, policy_version 43293 (0.0009) [2023-10-10 18:16:02,308][123614] Updated weights for policy 1, policy_version 43210 (0.0007) [2023-10-10 18:16:02,683][123614] Updated weights for policy 1, policy_version 43220 (0.0008) [2023-10-10 18:16:02,922][123582] Updated weights for policy 0, policy_version 43303 (0.0008) [2023-10-10 18:16:03,047][123614] Updated weights for policy 1, policy_version 43230 (0.0009) [2023-10-10 18:16:03,299][123582] Updated weights for policy 0, policy_version 43313 (0.0009) [2023-10-10 18:16:03,676][123582] Updated weights for policy 0, policy_version 43323 (0.0009) [2023-10-10 18:16:03,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 88604672. Throughput: 0: 1821.9, 1: 1810.6. Samples: 22161828. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-10 18:16:03,789][122664] Avg episode reward: [(0, '59.710'), (1, '51.720')] [2023-10-10 18:16:06,794][123614] Updated weights for policy 1, policy_version 43240 (0.0010) [2023-10-10 18:16:07,166][123614] Updated weights for policy 1, policy_version 43250 (0.0010) [2023-10-10 18:16:07,357][123582] Updated weights for policy 0, policy_version 43333 (0.0008) [2023-10-10 18:16:07,530][123614] Updated weights for policy 1, policy_version 43260 (0.0008) [2023-10-10 18:16:07,724][123582] Updated weights for policy 0, policy_version 43343 (0.0008) [2023-10-10 18:16:08,096][123582] Updated weights for policy 0, policy_version 43353 (0.0007) [2023-10-10 18:16:08,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 88702976. Throughput: 0: 1822.5, 1: 1794.1. Samples: 22182384. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:16:08,789][122664] Avg episode reward: [(0, '60.950'), (1, '51.770')] [2023-10-10 18:16:11,367][123614] Updated weights for policy 1, policy_version 43270 (0.0007) [2023-10-10 18:16:11,727][123582] Updated weights for policy 0, policy_version 43363 (0.0008) [2023-10-10 18:16:11,742][123614] Updated weights for policy 1, policy_version 43280 (0.0008) [2023-10-10 18:16:12,092][123582] Updated weights for policy 0, policy_version 43373 (0.0008) [2023-10-10 18:16:12,105][123614] Updated weights for policy 1, policy_version 43290 (0.0009) [2023-10-10 18:16:12,463][123582] Updated weights for policy 0, policy_version 43383 (0.0008) [2023-10-10 18:16:13,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 88768512. Throughput: 0: 1822.4, 1: 1807.5. Samples: 22194534. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:16:13,788][122664] Avg episode reward: [(0, '61.730'), (1, '49.330')] [2023-10-10 18:16:15,900][123614] Updated weights for policy 1, policy_version 43300 (0.0009) [2023-10-10 18:16:16,111][123582] Updated weights for policy 0, policy_version 43393 (0.0008) [2023-10-10 18:16:16,262][123614] Updated weights for policy 1, policy_version 43310 (0.0008) [2023-10-10 18:16:16,473][123582] Updated weights for policy 0, policy_version 43403 (0.0010) [2023-10-10 18:16:16,629][123614] Updated weights for policy 1, policy_version 43320 (0.0009) [2023-10-10 18:16:16,848][123582] Updated weights for policy 0, policy_version 43413 (0.0007) [2023-10-10 18:16:17,215][123582] Updated weights for policy 0, policy_version 43423 (0.0008) [2023-10-10 18:16:18,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 88834048. Throughput: 0: 1823.6, 1: 1791.5. Samples: 22214938. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:16:18,789][122664] Avg episode reward: [(0, '61.500'), (1, '50.750')] [2023-10-10 18:16:20,361][123614] Updated weights for policy 1, policy_version 43330 (0.0009) [2023-10-10 18:16:20,734][123614] Updated weights for policy 1, policy_version 43340 (0.0008) [2023-10-10 18:16:21,098][123582] Updated weights for policy 0, policy_version 43433 (0.0007) [2023-10-10 18:16:21,103][123614] Updated weights for policy 1, policy_version 43350 (0.0008) [2023-10-10 18:16:21,475][123582] Updated weights for policy 0, policy_version 43443 (0.0008) [2023-10-10 18:16:21,478][123614] Updated weights for policy 1, policy_version 43360 (0.0008) [2023-10-10 18:16:21,837][123582] Updated weights for policy 0, policy_version 43453 (0.0010) [2023-10-10 18:16:23,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 88899584. Throughput: 0: 1818.9, 1: 1795.4. Samples: 22237820. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:16:23,788][122664] Avg episode reward: [(0, '60.920'), (1, '51.930')] [2023-10-10 18:16:25,038][123614] Updated weights for policy 1, policy_version 43370 (0.0009) [2023-10-10 18:16:25,406][123614] Updated weights for policy 1, policy_version 43380 (0.0009) [2023-10-10 18:16:25,455][123582] Updated weights for policy 0, policy_version 43463 (0.0007) [2023-10-10 18:16:25,777][123614] Updated weights for policy 1, policy_version 43390 (0.0009) [2023-10-10 18:16:25,826][123582] Updated weights for policy 0, policy_version 43473 (0.0009) [2023-10-10 18:16:26,206][123582] Updated weights for policy 0, policy_version 43483 (0.0008) [2023-10-10 18:16:28,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 88965120. Throughput: 0: 1812.5, 1: 1796.1. Samples: 22247494. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:16:28,789][122664] Avg episode reward: [(0, '59.930'), (1, '51.150')] [2023-10-10 18:16:29,544][123614] Updated weights for policy 1, policy_version 43400 (0.0009) [2023-10-10 18:16:29,840][123582] Updated weights for policy 0, policy_version 43493 (0.0008) [2023-10-10 18:16:29,916][123614] Updated weights for policy 1, policy_version 43410 (0.0009) [2023-10-10 18:16:30,199][123582] Updated weights for policy 0, policy_version 43503 (0.0008) [2023-10-10 18:16:30,275][123614] Updated weights for policy 1, policy_version 43420 (0.0008) [2023-10-10 18:16:30,569][123582] Updated weights for policy 0, policy_version 43513 (0.0008) [2023-10-10 18:16:33,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 89030656. Throughput: 0: 1809.6, 1: 1799.1. Samples: 22269970. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:16:33,788][122664] Avg episode reward: [(0, '59.730'), (1, '52.070')] [2023-10-10 18:16:33,985][123614] Updated weights for policy 1, policy_version 43430 (0.0008) [2023-10-10 18:16:34,208][123582] Updated weights for policy 0, policy_version 43523 (0.0008) [2023-10-10 18:16:34,366][123614] Updated weights for policy 1, policy_version 43440 (0.0007) [2023-10-10 18:16:34,577][123582] Updated weights for policy 0, policy_version 43533 (0.0007) [2023-10-10 18:16:34,735][123614] Updated weights for policy 1, policy_version 43450 (0.0007) [2023-10-10 18:16:34,953][123582] Updated weights for policy 0, policy_version 43543 (0.0007) [2023-10-10 18:16:38,374][123614] Updated weights for policy 1, policy_version 43460 (0.0008) [2023-10-10 18:16:38,742][123614] Updated weights for policy 1, policy_version 43470 (0.0008) [2023-10-10 18:16:38,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 89096192. Throughput: 0: 1808.5, 1: 1813.0. Samples: 22292036. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:16:38,789][122664] Avg episode reward: [(0, '61.010'), (1, '53.540')] [2023-10-10 18:16:38,854][123582] Updated weights for policy 0, policy_version 43553 (0.0011) [2023-10-10 18:16:39,110][123614] Updated weights for policy 1, policy_version 43480 (0.0007) [2023-10-10 18:16:39,215][123582] Updated weights for policy 0, policy_version 43563 (0.0008) [2023-10-10 18:16:39,586][123582] Updated weights for policy 0, policy_version 43573 (0.0008) [2023-10-10 18:16:39,960][123582] Updated weights for policy 0, policy_version 43583 (0.0010) [2023-10-10 18:16:42,836][123614] Updated weights for policy 1, policy_version 43490 (0.0008) [2023-10-10 18:16:43,203][123614] Updated weights for policy 1, policy_version 43500 (0.0010) [2023-10-10 18:16:43,543][123582] Updated weights for policy 0, policy_version 43593 (0.0007) [2023-10-10 18:16:43,579][123614] Updated weights for policy 1, policy_version 43510 (0.0007) [2023-10-10 18:16:43,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 89161728. Throughput: 0: 1805.7, 1: 1798.4. Samples: 22302344. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:16:43,788][122664] Avg episode reward: [(0, '59.610'), (1, '52.930')] [2023-10-10 18:16:43,916][123582] Updated weights for policy 0, policy_version 43603 (0.0008) [2023-10-10 18:16:43,938][123614] Updated weights for policy 1, policy_version 43520 (0.0008) [2023-10-10 18:16:44,293][123582] Updated weights for policy 0, policy_version 43613 (0.0010) [2023-10-10 18:16:47,797][123614] Updated weights for policy 1, policy_version 43530 (0.0009) [2023-10-10 18:16:48,005][123582] Updated weights for policy 0, policy_version 43623 (0.0008) [2023-10-10 18:16:48,161][123614] Updated weights for policy 1, policy_version 43540 (0.0009) [2023-10-10 18:16:48,379][123582] Updated weights for policy 0, policy_version 43633 (0.0007) [2023-10-10 18:16:48,540][123614] Updated weights for policy 1, policy_version 43550 (0.0010) [2023-10-10 18:16:48,748][123582] Updated weights for policy 0, policy_version 43643 (0.0007) [2023-10-10 18:16:48,788][122664] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 89260032. Throughput: 0: 1808.0, 1: 1813.1. Samples: 22324776. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:16:48,789][122664] Avg episode reward: [(0, '60.190'), (1, '53.570')] [2023-10-10 18:16:52,312][123614] Updated weights for policy 1, policy_version 43560 (0.0009) [2023-10-10 18:16:52,492][123582] Updated weights for policy 0, policy_version 43653 (0.0008) [2023-10-10 18:16:52,678][123614] Updated weights for policy 1, policy_version 43570 (0.0008) [2023-10-10 18:16:52,860][123582] Updated weights for policy 0, policy_version 43663 (0.0008) [2023-10-10 18:16:53,053][123614] Updated weights for policy 1, policy_version 43580 (0.0008) [2023-10-10 18:16:53,236][123582] Updated weights for policy 0, policy_version 43673 (0.0008) [2023-10-10 18:16:53,788][122664] Fps is (10 sec: 19660.5, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 89358336. Throughput: 0: 1812.0, 1: 1802.0. Samples: 22345014. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:16:53,789][122664] Avg episode reward: [(0, '52.550'), (1, '57.420')] [2023-10-10 18:16:56,806][123614] Updated weights for policy 1, policy_version 43590 (0.0009) [2023-10-10 18:16:57,129][123582] Updated weights for policy 0, policy_version 43683 (0.0010) [2023-10-10 18:16:57,174][123614] Updated weights for policy 1, policy_version 43600 (0.0008) [2023-10-10 18:16:57,508][123582] Updated weights for policy 0, policy_version 43693 (0.0008) [2023-10-10 18:16:57,540][123614] Updated weights for policy 1, policy_version 43610 (0.0008) [2023-10-10 18:16:57,870][123582] Updated weights for policy 0, policy_version 43703 (0.0007) [2023-10-10 18:16:58,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 89423872. Throughput: 0: 1805.3, 1: 1816.4. Samples: 22357512. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:16:58,789][122664] Avg episode reward: [(0, '53.150'), (1, '55.570')] [2023-10-10 18:17:01,366][123582] Updated weights for policy 0, policy_version 43713 (0.0007) [2023-10-10 18:17:01,412][123614] Updated weights for policy 1, policy_version 43620 (0.0010) [2023-10-10 18:17:01,728][123582] Updated weights for policy 0, policy_version 43723 (0.0009) [2023-10-10 18:17:01,782][123614] Updated weights for policy 1, policy_version 43630 (0.0009) [2023-10-10 18:17:02,094][123582] Updated weights for policy 0, policy_version 43733 (0.0007) [2023-10-10 18:17:02,151][123614] Updated weights for policy 1, policy_version 43640 (0.0008) [2023-10-10 18:17:02,462][123582] Updated weights for policy 0, policy_version 43743 (0.0007) [2023-10-10 18:17:03,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 89489408. Throughput: 0: 1807.8, 1: 1797.0. Samples: 22377156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:17:03,789][122664] Avg episode reward: [(0, '49.660'), (1, '52.720')] [2023-10-10 18:17:05,721][123614] Updated weights for policy 1, policy_version 43650 (0.0007) [2023-10-10 18:17:06,066][123582] Updated weights for policy 0, policy_version 43753 (0.0007) [2023-10-10 18:17:06,091][123614] Updated weights for policy 1, policy_version 43660 (0.0008) [2023-10-10 18:17:06,433][123582] Updated weights for policy 0, policy_version 43763 (0.0008) [2023-10-10 18:17:06,469][123614] Updated weights for policy 1, policy_version 43670 (0.0009) [2023-10-10 18:17:06,804][123582] Updated weights for policy 0, policy_version 43773 (0.0009) [2023-10-10 18:17:06,828][123614] Updated weights for policy 1, policy_version 43680 (0.0008) [2023-10-10 18:17:08,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 89554944. Throughput: 0: 1803.0, 1: 1795.2. Samples: 22399740. Policy #0 lag: (min: 11.0, avg: 18.0, max: 43.0) [2023-10-10 18:17:08,788][122664] Avg episode reward: [(0, '50.080'), (1, '52.520')] [2023-10-10 18:17:10,487][123614] Updated weights for policy 1, policy_version 43690 (0.0007) [2023-10-10 18:17:10,627][123582] Updated weights for policy 0, policy_version 43783 (0.0008) [2023-10-10 18:17:10,855][123614] Updated weights for policy 1, policy_version 43700 (0.0007) [2023-10-10 18:17:10,993][123582] Updated weights for policy 0, policy_version 43793 (0.0008) [2023-10-10 18:17:11,223][123614] Updated weights for policy 1, policy_version 43710 (0.0010) [2023-10-10 18:17:11,361][123582] Updated weights for policy 0, policy_version 43803 (0.0009) [2023-10-10 18:17:13,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 89620480. Throughput: 0: 1812.2, 1: 1794.0. Samples: 22409772. Policy #0 lag: (min: 11.0, avg: 18.0, max: 43.0) [2023-10-10 18:17:13,789][122664] Avg episode reward: [(0, '53.890'), (1, '57.110')] [2023-10-10 18:17:14,863][123614] Updated weights for policy 1, policy_version 43720 (0.0008) [2023-10-10 18:17:15,159][123582] Updated weights for policy 0, policy_version 43813 (0.0007) [2023-10-10 18:17:15,239][123614] Updated weights for policy 1, policy_version 43730 (0.0008) [2023-10-10 18:17:15,533][123582] Updated weights for policy 0, policy_version 43823 (0.0008) [2023-10-10 18:17:15,607][123614] Updated weights for policy 1, policy_version 43740 (0.0008) [2023-10-10 18:17:15,904][123582] Updated weights for policy 0, policy_version 43833 (0.0009) [2023-10-10 18:17:18,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 89686016. Throughput: 0: 1803.1, 1: 1796.9. Samples: 22431972. Policy #0 lag: (min: 11.0, avg: 18.0, max: 43.0) [2023-10-10 18:17:18,789][122664] Avg episode reward: [(0, '53.410'), (1, '54.600')] [2023-10-10 18:17:19,497][123614] Updated weights for policy 1, policy_version 43750 (0.0010) [2023-10-10 18:17:19,760][123582] Updated weights for policy 0, policy_version 43843 (0.0009) [2023-10-10 18:17:19,877][123614] Updated weights for policy 1, policy_version 43760 (0.0009) [2023-10-10 18:17:20,123][123582] Updated weights for policy 0, policy_version 43853 (0.0008) [2023-10-10 18:17:20,242][123614] Updated weights for policy 1, policy_version 43770 (0.0007) [2023-10-10 18:17:20,502][123582] Updated weights for policy 0, policy_version 43863 (0.0010) [2023-10-10 18:17:23,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 89751552. Throughput: 0: 1796.4, 1: 1806.8. Samples: 22454180. Policy #0 lag: (min: 11.0, avg: 18.0, max: 43.0) [2023-10-10 18:17:23,789][122664] Avg episode reward: [(0, '54.590'), (1, '51.810')] [2023-10-10 18:17:24,060][123614] Updated weights for policy 1, policy_version 43780 (0.0009) [2023-10-10 18:17:24,384][123582] Updated weights for policy 0, policy_version 43873 (0.0009) [2023-10-10 18:17:24,419][123614] Updated weights for policy 1, policy_version 43790 (0.0009) [2023-10-10 18:17:24,759][123582] Updated weights for policy 0, policy_version 43883 (0.0009) [2023-10-10 18:17:24,785][123614] Updated weights for policy 1, policy_version 43800 (0.0010) [2023-10-10 18:17:25,120][123582] Updated weights for policy 0, policy_version 43893 (0.0007) [2023-10-10 18:17:25,487][123582] Updated weights for policy 0, policy_version 43903 (0.0008) [2023-10-10 18:17:28,545][123614] Updated weights for policy 1, policy_version 43810 (0.0008) [2023-10-10 18:17:28,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 89817088. Throughput: 0: 1797.8, 1: 1791.5. Samples: 22463862. Policy #0 lag: (min: 11.0, avg: 18.0, max: 43.0) [2023-10-10 18:17:28,789][122664] Avg episode reward: [(0, '54.510'), (1, '48.220')] [2023-10-10 18:17:28,913][123614] Updated weights for policy 1, policy_version 43820 (0.0007) [2023-10-10 18:17:29,291][123614] Updated weights for policy 1, policy_version 43830 (0.0007) [2023-10-10 18:17:29,294][123582] Updated weights for policy 0, policy_version 43913 (0.0009) [2023-10-10 18:17:29,655][123614] Updated weights for policy 1, policy_version 43840 (0.0009) [2023-10-10 18:17:29,666][123582] Updated weights for policy 0, policy_version 43923 (0.0008) [2023-10-10 18:17:30,035][123582] Updated weights for policy 0, policy_version 43933 (0.0010) [2023-10-10 18:17:33,282][123614] Updated weights for policy 1, policy_version 43850 (0.0008) [2023-10-10 18:17:33,583][123582] Updated weights for policy 0, policy_version 43943 (0.0009) [2023-10-10 18:17:33,656][123614] Updated weights for policy 1, policy_version 43860 (0.0007) [2023-10-10 18:17:33,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 89882624. Throughput: 0: 1790.0, 1: 1802.9. Samples: 22486456. Policy #0 lag: (min: 11.0, avg: 18.0, max: 43.0) [2023-10-10 18:17:33,789][122664] Avg episode reward: [(0, '52.880'), (1, '48.730')] [2023-10-10 18:17:33,960][123582] Updated weights for policy 0, policy_version 43953 (0.0009) [2023-10-10 18:17:34,019][123614] Updated weights for policy 1, policy_version 43870 (0.0007) [2023-10-10 18:17:34,322][123582] Updated weights for policy 0, policy_version 43963 (0.0009) [2023-10-10 18:17:37,704][123614] Updated weights for policy 1, policy_version 43880 (0.0008) [2023-10-10 18:17:38,072][123614] Updated weights for policy 1, policy_version 43890 (0.0007) [2023-10-10 18:17:38,116][123582] Updated weights for policy 0, policy_version 43973 (0.0010) [2023-10-10 18:17:38,433][123614] Updated weights for policy 1, policy_version 43900 (0.0007) [2023-10-10 18:17:38,489][123582] Updated weights for policy 0, policy_version 43983 (0.0008) [2023-10-10 18:17:38,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 89980928. Throughput: 0: 1806.2, 1: 1797.2. Samples: 22507166. Policy #0 lag: (min: 11.0, avg: 18.0, max: 43.0) [2023-10-10 18:17:38,788][122664] Avg episode reward: [(0, '55.250'), (1, '44.450')] [2023-10-10 18:17:38,869][123582] Updated weights for policy 0, policy_version 43993 (0.0008) [2023-10-10 18:17:42,149][123614] Updated weights for policy 1, policy_version 43910 (0.0007) [2023-10-10 18:17:42,512][123614] Updated weights for policy 1, policy_version 43920 (0.0007) [2023-10-10 18:17:42,516][123582] Updated weights for policy 0, policy_version 44003 (0.0008) [2023-10-10 18:17:42,878][123582] Updated weights for policy 0, policy_version 44013 (0.0010) [2023-10-10 18:17:42,884][123614] Updated weights for policy 1, policy_version 43930 (0.0007) [2023-10-10 18:17:43,247][123582] Updated weights for policy 0, policy_version 44023 (0.0007) [2023-10-10 18:17:43,788][122664] Fps is (10 sec: 19661.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 90079232. Throughput: 0: 1788.4, 1: 1799.6. Samples: 22518968. Policy #0 lag: (min: 31.0, avg: 32.7, max: 53.0) [2023-10-10 18:17:43,788][122664] Avg episode reward: [(0, '54.970'), (1, '41.780')] [2023-10-10 18:17:46,640][123614] Updated weights for policy 1, policy_version 43940 (0.0008) [2023-10-10 18:17:46,986][123582] Updated weights for policy 0, policy_version 44033 (0.0008) [2023-10-10 18:17:47,010][123614] Updated weights for policy 1, policy_version 43950 (0.0008) [2023-10-10 18:17:47,351][123582] Updated weights for policy 0, policy_version 44043 (0.0007) [2023-10-10 18:17:47,377][123614] Updated weights for policy 1, policy_version 43960 (0.0008) [2023-10-10 18:17:47,731][123582] Updated weights for policy 0, policy_version 44053 (0.0011) [2023-10-10 18:17:48,093][123582] Updated weights for policy 0, policy_version 44063 (0.0008) [2023-10-10 18:17:48,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 90144768. Throughput: 0: 1807.8, 1: 1800.5. Samples: 22539528. Policy #0 lag: (min: 31.0, avg: 32.7, max: 53.0) [2023-10-10 18:17:48,788][122664] Avg episode reward: [(0, '58.060'), (1, '41.220')] [2023-10-10 18:17:51,271][123614] Updated weights for policy 1, policy_version 43970 (0.0008) [2023-10-10 18:17:51,642][123614] Updated weights for policy 1, policy_version 43980 (0.0007) [2023-10-10 18:17:51,918][123582] Updated weights for policy 0, policy_version 44073 (0.0008) [2023-10-10 18:17:52,011][123614] Updated weights for policy 1, policy_version 43990 (0.0008) [2023-10-10 18:17:52,283][123582] Updated weights for policy 0, policy_version 44083 (0.0008) [2023-10-10 18:17:52,381][123614] Updated weights for policy 1, policy_version 44000 (0.0007) [2023-10-10 18:17:52,662][123582] Updated weights for policy 0, policy_version 44093 (0.0010) [2023-10-10 18:17:53,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 90210304. Throughput: 0: 1788.9, 1: 1794.9. Samples: 22561014. Policy #0 lag: (min: 31.0, avg: 32.7, max: 53.0) [2023-10-10 18:17:53,789][122664] Avg episode reward: [(0, '57.630'), (1, '41.520')] [2023-10-10 18:17:53,797][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000044000_45056000.pth... [2023-10-10 18:17:53,797][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000044096_45154304.pth... [2023-10-10 18:17:53,836][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000042400_43417600.pth [2023-10-10 18:17:53,838][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000042304_43319296.pth [2023-10-10 18:17:56,049][123614] Updated weights for policy 1, policy_version 44010 (0.0010) [2023-10-10 18:17:56,420][123614] Updated weights for policy 1, policy_version 44020 (0.0008) [2023-10-10 18:17:56,548][123582] Updated weights for policy 0, policy_version 44103 (0.0008) [2023-10-10 18:17:56,788][123614] Updated weights for policy 1, policy_version 44030 (0.0009) [2023-10-10 18:17:56,921][123582] Updated weights for policy 0, policy_version 44113 (0.0008) [2023-10-10 18:17:57,302][123582] Updated weights for policy 0, policy_version 44123 (0.0010) [2023-10-10 18:17:58,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 90275840. Throughput: 0: 1805.1, 1: 1804.7. Samples: 22572214. Policy #0 lag: (min: 31.0, avg: 32.7, max: 53.0) [2023-10-10 18:17:58,789][122664] Avg episode reward: [(0, '57.220'), (1, '39.770')] [2023-10-10 18:18:00,576][123614] Updated weights for policy 1, policy_version 44040 (0.0009) [2023-10-10 18:18:00,941][123614] Updated weights for policy 1, policy_version 44050 (0.0008) [2023-10-10 18:18:01,043][123582] Updated weights for policy 0, policy_version 44133 (0.0009) [2023-10-10 18:18:01,316][123614] Updated weights for policy 1, policy_version 44060 (0.0009) [2023-10-10 18:18:01,420][123582] Updated weights for policy 0, policy_version 44143 (0.0007) [2023-10-10 18:18:01,797][123582] Updated weights for policy 0, policy_version 44153 (0.0009) [2023-10-10 18:18:03,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 90341376. Throughput: 0: 1786.4, 1: 1800.7. Samples: 22593394. Policy #0 lag: (min: 31.0, avg: 32.7, max: 53.0) [2023-10-10 18:18:03,789][122664] Avg episode reward: [(0, '56.450'), (1, '39.030')] [2023-10-10 18:18:04,999][123614] Updated weights for policy 1, policy_version 44070 (0.0008) [2023-10-10 18:18:05,386][123614] Updated weights for policy 1, policy_version 44080 (0.0008) [2023-10-10 18:18:05,427][123582] Updated weights for policy 0, policy_version 44163 (0.0007) [2023-10-10 18:18:05,743][123614] Updated weights for policy 1, policy_version 44090 (0.0008) [2023-10-10 18:18:05,795][123582] Updated weights for policy 0, policy_version 44173 (0.0008) [2023-10-10 18:18:06,170][123582] Updated weights for policy 0, policy_version 44183 (0.0008) [2023-10-10 18:18:08,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 90406912. Throughput: 0: 1799.8, 1: 1799.8. Samples: 22616162. Policy #0 lag: (min: 31.0, avg: 32.7, max: 53.0) [2023-10-10 18:18:08,789][122664] Avg episode reward: [(0, '59.600'), (1, '39.330')] [2023-10-10 18:18:09,371][123614] Updated weights for policy 1, policy_version 44100 (0.0009) [2023-10-10 18:18:09,739][123614] Updated weights for policy 1, policy_version 44110 (0.0009) [2023-10-10 18:18:09,936][123582] Updated weights for policy 0, policy_version 44193 (0.0007) [2023-10-10 18:18:10,103][123614] Updated weights for policy 1, policy_version 44120 (0.0009) [2023-10-10 18:18:10,311][123582] Updated weights for policy 0, policy_version 44203 (0.0009) [2023-10-10 18:18:10,677][123582] Updated weights for policy 0, policy_version 44213 (0.0009) [2023-10-10 18:18:11,048][123582] Updated weights for policy 0, policy_version 44223 (0.0008) [2023-10-10 18:18:13,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 90472448. Throughput: 0: 1800.5, 1: 1802.8. Samples: 22626010. Policy #0 lag: (min: 1.0, avg: 12.7, max: 33.0) [2023-10-10 18:18:13,788][122664] Avg episode reward: [(0, '55.950'), (1, '41.350')] [2023-10-10 18:18:13,849][123614] Updated weights for policy 1, policy_version 44130 (0.0008) [2023-10-10 18:18:14,209][123614] Updated weights for policy 1, policy_version 44140 (0.0008) [2023-10-10 18:18:14,574][123614] Updated weights for policy 1, policy_version 44150 (0.0008) [2023-10-10 18:18:14,712][123582] Updated weights for policy 0, policy_version 44233 (0.0007) [2023-10-10 18:18:14,942][123614] Updated weights for policy 1, policy_version 44160 (0.0009) [2023-10-10 18:18:15,081][123582] Updated weights for policy 0, policy_version 44243 (0.0008) [2023-10-10 18:18:15,448][123582] Updated weights for policy 0, policy_version 44253 (0.0009) [2023-10-10 18:18:18,682][123614] Updated weights for policy 1, policy_version 44170 (0.0008) [2023-10-10 18:18:18,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 90537984. Throughput: 0: 1806.6, 1: 1803.4. Samples: 22648908. Policy #0 lag: (min: 1.0, avg: 12.7, max: 33.0) [2023-10-10 18:18:18,788][122664] Avg episode reward: [(0, '55.810'), (1, '39.430')] [2023-10-10 18:18:19,041][123582] Updated weights for policy 0, policy_version 44263 (0.0007) [2023-10-10 18:18:19,053][123614] Updated weights for policy 1, policy_version 44180 (0.0009) [2023-10-10 18:18:19,410][123582] Updated weights for policy 0, policy_version 44273 (0.0008) [2023-10-10 18:18:19,416][123614] Updated weights for policy 1, policy_version 44190 (0.0008) [2023-10-10 18:18:19,781][123582] Updated weights for policy 0, policy_version 44283 (0.0007) [2023-10-10 18:18:23,197][123614] Updated weights for policy 1, policy_version 44200 (0.0008) [2023-10-10 18:18:23,291][123582] Updated weights for policy 0, policy_version 44293 (0.0007) [2023-10-10 18:18:23,561][123614] Updated weights for policy 1, policy_version 44210 (0.0008) [2023-10-10 18:18:23,670][123582] Updated weights for policy 0, policy_version 44303 (0.0009) [2023-10-10 18:18:23,788][122664] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 90603520. Throughput: 0: 1817.1, 1: 1811.6. Samples: 22670458. Policy #0 lag: (min: 1.0, avg: 12.7, max: 33.0) [2023-10-10 18:18:23,789][122664] Avg episode reward: [(0, '52.420'), (1, '44.870')] [2023-10-10 18:18:23,939][123614] Updated weights for policy 1, policy_version 44220 (0.0009) [2023-10-10 18:18:24,039][123582] Updated weights for policy 0, policy_version 44313 (0.0007) [2023-10-10 18:18:27,711][123614] Updated weights for policy 1, policy_version 44230 (0.0008) [2023-10-10 18:18:27,808][123582] Updated weights for policy 0, policy_version 44323 (0.0008) [2023-10-10 18:18:28,080][123614] Updated weights for policy 1, policy_version 44240 (0.0008) [2023-10-10 18:18:28,176][123582] Updated weights for policy 0, policy_version 44333 (0.0007) [2023-10-10 18:18:28,448][123614] Updated weights for policy 1, policy_version 44250 (0.0008) [2023-10-10 18:18:28,548][123582] Updated weights for policy 0, policy_version 44343 (0.0008) [2023-10-10 18:18:28,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 90701824. Throughput: 0: 1810.6, 1: 1801.9. Samples: 22681528. Policy #0 lag: (min: 1.0, avg: 12.7, max: 33.0) [2023-10-10 18:18:28,789][122664] Avg episode reward: [(0, '55.180'), (1, '46.330')] [2023-10-10 18:18:32,380][123614] Updated weights for policy 1, policy_version 44260 (0.0007) [2023-10-10 18:18:32,443][123582] Updated weights for policy 0, policy_version 44353 (0.0009) [2023-10-10 18:18:32,744][123614] Updated weights for policy 1, policy_version 44270 (0.0008) [2023-10-10 18:18:32,811][123582] Updated weights for policy 0, policy_version 44363 (0.0009) [2023-10-10 18:18:33,109][123614] Updated weights for policy 1, policy_version 44280 (0.0009) [2023-10-10 18:18:33,185][123582] Updated weights for policy 0, policy_version 44373 (0.0009) [2023-10-10 18:18:33,543][123582] Updated weights for policy 0, policy_version 44383 (0.0008) [2023-10-10 18:18:33,788][122664] Fps is (10 sec: 19661.1, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 90800128. Throughput: 0: 1816.2, 1: 1815.0. Samples: 22702932. Policy #0 lag: (min: 1.0, avg: 12.7, max: 33.0) [2023-10-10 18:18:33,789][122664] Avg episode reward: [(0, '49.360'), (1, '46.770')] [2023-10-10 18:18:36,797][123614] Updated weights for policy 1, policy_version 44290 (0.0008) [2023-10-10 18:18:37,169][123614] Updated weights for policy 1, policy_version 44300 (0.0008) [2023-10-10 18:18:37,215][123582] Updated weights for policy 0, policy_version 44393 (0.0009) [2023-10-10 18:18:37,535][123614] Updated weights for policy 1, policy_version 44310 (0.0008) [2023-10-10 18:18:37,589][123582] Updated weights for policy 0, policy_version 44403 (0.0009) [2023-10-10 18:18:37,903][123614] Updated weights for policy 1, policy_version 44320 (0.0007) [2023-10-10 18:18:37,962][123582] Updated weights for policy 0, policy_version 44413 (0.0007) [2023-10-10 18:18:38,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 90865664. Throughput: 0: 1807.6, 1: 1800.8. Samples: 22723392. Policy #0 lag: (min: 1.0, avg: 12.7, max: 33.0) [2023-10-10 18:18:38,789][122664] Avg episode reward: [(0, '49.100'), (1, '51.170')] [2023-10-10 18:18:41,687][123614] Updated weights for policy 1, policy_version 44330 (0.0007) [2023-10-10 18:18:41,812][123582] Updated weights for policy 0, policy_version 44423 (0.0010) [2023-10-10 18:18:42,056][123614] Updated weights for policy 1, policy_version 44340 (0.0008) [2023-10-10 18:18:42,186][123582] Updated weights for policy 0, policy_version 44433 (0.0008) [2023-10-10 18:18:42,414][123614] Updated weights for policy 1, policy_version 44350 (0.0008) [2023-10-10 18:18:42,561][123582] Updated weights for policy 0, policy_version 44443 (0.0008) [2023-10-10 18:18:43,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 90931200. Throughput: 0: 1821.9, 1: 1813.4. Samples: 22735802. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:18:43,789][122664] Avg episode reward: [(0, '47.950'), (1, '50.960')] [2023-10-10 18:18:46,168][123614] Updated weights for policy 1, policy_version 44360 (0.0008) [2023-10-10 18:18:46,211][123582] Updated weights for policy 0, policy_version 44453 (0.0010) [2023-10-10 18:18:46,535][123614] Updated weights for policy 1, policy_version 44370 (0.0007) [2023-10-10 18:18:46,586][123582] Updated weights for policy 0, policy_version 44463 (0.0007) [2023-10-10 18:18:46,906][123614] Updated weights for policy 1, policy_version 44380 (0.0007) [2023-10-10 18:18:46,967][123582] Updated weights for policy 0, policy_version 44473 (0.0008) [2023-10-10 18:18:48,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 90996736. Throughput: 0: 1817.6, 1: 1792.4. Samples: 22755848. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:18:48,789][122664] Avg episode reward: [(0, '48.260'), (1, '52.510')] [2023-10-10 18:18:50,748][123614] Updated weights for policy 1, policy_version 44390 (0.0008) [2023-10-10 18:18:50,761][123582] Updated weights for policy 0, policy_version 44483 (0.0009) [2023-10-10 18:18:51,127][123614] Updated weights for policy 1, policy_version 44400 (0.0007) [2023-10-10 18:18:51,128][123582] Updated weights for policy 0, policy_version 44493 (0.0007) [2023-10-10 18:18:51,485][123614] Updated weights for policy 1, policy_version 44410 (0.0007) [2023-10-10 18:18:51,503][123582] Updated weights for policy 0, policy_version 44503 (0.0007) [2023-10-10 18:18:53,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 91062272. Throughput: 0: 1810.5, 1: 1787.7. Samples: 22778082. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:18:53,788][122664] Avg episode reward: [(0, '52.150'), (1, '52.970')] [2023-10-10 18:18:55,147][123614] Updated weights for policy 1, policy_version 44420 (0.0007) [2023-10-10 18:18:55,192][123582] Updated weights for policy 0, policy_version 44513 (0.0007) [2023-10-10 18:18:55,507][123614] Updated weights for policy 1, policy_version 44430 (0.0007) [2023-10-10 18:18:55,560][123582] Updated weights for policy 0, policy_version 44523 (0.0007) [2023-10-10 18:18:55,871][123614] Updated weights for policy 1, policy_version 44440 (0.0007) [2023-10-10 18:18:55,923][123582] Updated weights for policy 0, policy_version 44533 (0.0010) [2023-10-10 18:18:56,291][123582] Updated weights for policy 0, policy_version 44543 (0.0008) [2023-10-10 18:18:58,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 91127808. Throughput: 0: 1808.9, 1: 1789.1. Samples: 22787920. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:18:58,789][122664] Avg episode reward: [(0, '50.490'), (1, '56.200')] [2023-10-10 18:18:59,585][123614] Updated weights for policy 1, policy_version 44450 (0.0008) [2023-10-10 18:18:59,950][123614] Updated weights for policy 1, policy_version 44460 (0.0009) [2023-10-10 18:19:00,064][123582] Updated weights for policy 0, policy_version 44553 (0.0009) [2023-10-10 18:19:00,315][123614] Updated weights for policy 1, policy_version 44470 (0.0008) [2023-10-10 18:19:00,435][123582] Updated weights for policy 0, policy_version 44563 (0.0008) [2023-10-10 18:19:00,689][123614] Updated weights for policy 1, policy_version 44480 (0.0009) [2023-10-10 18:19:00,808][123582] Updated weights for policy 0, policy_version 44573 (0.0009) [2023-10-10 18:19:03,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 91193344. Throughput: 0: 1800.3, 1: 1786.6. Samples: 22810320. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:19:03,789][122664] Avg episode reward: [(0, '49.640'), (1, '55.480')] [2023-10-10 18:19:04,444][123614] Updated weights for policy 1, policy_version 44490 (0.0009) [2023-10-10 18:19:04,590][123582] Updated weights for policy 0, policy_version 44583 (0.0007) [2023-10-10 18:19:04,802][123614] Updated weights for policy 1, policy_version 44500 (0.0009) [2023-10-10 18:19:04,962][123582] Updated weights for policy 0, policy_version 44593 (0.0009) [2023-10-10 18:19:05,181][123614] Updated weights for policy 1, policy_version 44510 (0.0009) [2023-10-10 18:19:05,330][123582] Updated weights for policy 0, policy_version 44603 (0.0009) [2023-10-10 18:19:08,788][122664] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 91258880. Throughput: 0: 1797.0, 1: 1807.7. Samples: 22832672. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:19:08,790][122664] Avg episode reward: [(0, '50.590'), (1, '56.370')] [2023-10-10 18:19:08,822][123614] Updated weights for policy 1, policy_version 44520 (0.0007) [2023-10-10 18:19:09,047][123582] Updated weights for policy 0, policy_version 44613 (0.0008) [2023-10-10 18:19:09,189][123614] Updated weights for policy 1, policy_version 44530 (0.0007) [2023-10-10 18:19:09,419][123582] Updated weights for policy 0, policy_version 44623 (0.0008) [2023-10-10 18:19:09,563][123614] Updated weights for policy 1, policy_version 44540 (0.0009) [2023-10-10 18:19:09,785][123582] Updated weights for policy 0, policy_version 44633 (0.0009) [2023-10-10 18:19:13,239][123614] Updated weights for policy 1, policy_version 44550 (0.0009) [2023-10-10 18:19:13,437][123582] Updated weights for policy 0, policy_version 44643 (0.0009) [2023-10-10 18:19:13,605][123614] Updated weights for policy 1, policy_version 44560 (0.0008) [2023-10-10 18:19:13,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 91324416. Throughput: 0: 1788.5, 1: 1794.4. Samples: 22842760. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:19:13,789][122664] Avg episode reward: [(0, '52.070'), (1, '55.760')] [2023-10-10 18:19:13,806][123582] Updated weights for policy 0, policy_version 44653 (0.0008) [2023-10-10 18:19:13,977][123614] Updated weights for policy 1, policy_version 44570 (0.0008) [2023-10-10 18:19:14,175][123582] Updated weights for policy 0, policy_version 44663 (0.0009) [2023-10-10 18:19:17,555][123614] Updated weights for policy 1, policy_version 44580 (0.0009) [2023-10-10 18:19:17,923][123614] Updated weights for policy 1, policy_version 44590 (0.0010) [2023-10-10 18:19:17,997][123582] Updated weights for policy 0, policy_version 44673 (0.0009) [2023-10-10 18:19:18,281][123614] Updated weights for policy 1, policy_version 44600 (0.0007) [2023-10-10 18:19:18,364][123582] Updated weights for policy 0, policy_version 44683 (0.0008) [2023-10-10 18:19:18,734][123582] Updated weights for policy 0, policy_version 44693 (0.0008) [2023-10-10 18:19:18,788][122664] Fps is (10 sec: 16384.6, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 91422720. Throughput: 0: 1798.1, 1: 1814.4. Samples: 22865494. Policy #0 lag: (min: 31.0, avg: 32.1, max: 55.0) [2023-10-10 18:19:18,788][122664] Avg episode reward: [(0, '53.390'), (1, '58.950')] [2023-10-10 18:19:19,100][123582] Updated weights for policy 0, policy_version 44703 (0.0008) [2023-10-10 18:19:21,998][123614] Updated weights for policy 1, policy_version 44610 (0.0008) [2023-10-10 18:19:22,374][123614] Updated weights for policy 1, policy_version 44620 (0.0008) [2023-10-10 18:19:22,735][123614] Updated weights for policy 1, policy_version 44630 (0.0008) [2023-10-10 18:19:22,947][123582] Updated weights for policy 0, policy_version 44713 (0.0008) [2023-10-10 18:19:23,096][123614] Updated weights for policy 1, policy_version 44640 (0.0007) [2023-10-10 18:19:23,324][123582] Updated weights for policy 0, policy_version 44723 (0.0008) [2023-10-10 18:19:23,699][123582] Updated weights for policy 0, policy_version 44733 (0.0009) [2023-10-10 18:19:23,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 91488256. Throughput: 0: 1802.9, 1: 1808.6. Samples: 22885908. Policy #0 lag: (min: 31.0, avg: 32.1, max: 55.0) [2023-10-10 18:19:23,789][122664] Avg episode reward: [(0, '59.210'), (1, '57.060')] [2023-10-10 18:19:26,769][123614] Updated weights for policy 1, policy_version 44650 (0.0007) [2023-10-10 18:19:27,129][123614] Updated weights for policy 1, policy_version 44660 (0.0008) [2023-10-10 18:19:27,493][123582] Updated weights for policy 0, policy_version 44743 (0.0007) [2023-10-10 18:19:27,504][123614] Updated weights for policy 1, policy_version 44670 (0.0009) [2023-10-10 18:19:27,867][123582] Updated weights for policy 0, policy_version 44753 (0.0011) [2023-10-10 18:19:28,241][123582] Updated weights for policy 0, policy_version 44763 (0.0009) [2023-10-10 18:19:28,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 91586560. Throughput: 0: 1791.0, 1: 1810.4. Samples: 22897866. Policy #0 lag: (min: 31.0, avg: 32.1, max: 55.0) [2023-10-10 18:19:28,789][122664] Avg episode reward: [(0, '58.490'), (1, '56.450')] [2023-10-10 18:19:31,239][123614] Updated weights for policy 1, policy_version 44680 (0.0009) [2023-10-10 18:19:31,610][123614] Updated weights for policy 1, policy_version 44690 (0.0010) [2023-10-10 18:19:31,928][123582] Updated weights for policy 0, policy_version 44773 (0.0007) [2023-10-10 18:19:31,974][123614] Updated weights for policy 1, policy_version 44700 (0.0008) [2023-10-10 18:19:32,302][123582] Updated weights for policy 0, policy_version 44783 (0.0007) [2023-10-10 18:19:32,675][123582] Updated weights for policy 0, policy_version 44793 (0.0011) [2023-10-10 18:19:33,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 91652096. Throughput: 0: 1804.9, 1: 1813.1. Samples: 22918658. Policy #0 lag: (min: 31.0, avg: 32.1, max: 55.0) [2023-10-10 18:19:33,789][122664] Avg episode reward: [(0, '61.370'), (1, '55.810')] [2023-10-10 18:19:35,773][123614] Updated weights for policy 1, policy_version 44710 (0.0008) [2023-10-10 18:19:36,151][123614] Updated weights for policy 1, policy_version 44720 (0.0007) [2023-10-10 18:19:36,167][123582] Updated weights for policy 0, policy_version 44803 (0.0009) [2023-10-10 18:19:36,516][123614] Updated weights for policy 1, policy_version 44730 (0.0007) [2023-10-10 18:19:36,528][123582] Updated weights for policy 0, policy_version 44813 (0.0008) [2023-10-10 18:19:36,901][123582] Updated weights for policy 0, policy_version 44823 (0.0007) [2023-10-10 18:19:38,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 91717632. Throughput: 0: 1798.8, 1: 1817.4. Samples: 22940808. Policy #0 lag: (min: 31.0, avg: 32.1, max: 55.0) [2023-10-10 18:19:38,788][122664] Avg episode reward: [(0, '59.360'), (1, '53.350')] [2023-10-10 18:19:40,154][123614] Updated weights for policy 1, policy_version 44740 (0.0008) [2023-10-10 18:19:40,492][123582] Updated weights for policy 0, policy_version 44833 (0.0010) [2023-10-10 18:19:40,524][123614] Updated weights for policy 1, policy_version 44750 (0.0007) [2023-10-10 18:19:40,864][123582] Updated weights for policy 0, policy_version 44843 (0.0008) [2023-10-10 18:19:40,885][123614] Updated weights for policy 1, policy_version 44760 (0.0009) [2023-10-10 18:19:41,239][123582] Updated weights for policy 0, policy_version 44853 (0.0009) [2023-10-10 18:19:41,613][123582] Updated weights for policy 0, policy_version 44863 (0.0007) [2023-10-10 18:19:43,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 91783168. Throughput: 0: 1808.5, 1: 1816.3. Samples: 22951034. Policy #0 lag: (min: 31.0, avg: 32.1, max: 55.0) [2023-10-10 18:19:43,789][122664] Avg episode reward: [(0, '60.010'), (1, '54.450')] [2023-10-10 18:19:44,335][123614] Updated weights for policy 1, policy_version 44770 (0.0008) [2023-10-10 18:19:44,703][123614] Updated weights for policy 1, policy_version 44780 (0.0010) [2023-10-10 18:19:45,081][123614] Updated weights for policy 1, policy_version 44790 (0.0009) [2023-10-10 18:19:45,429][123582] Updated weights for policy 0, policy_version 44873 (0.0008) [2023-10-10 18:19:45,448][123614] Updated weights for policy 1, policy_version 44800 (0.0008) [2023-10-10 18:19:45,787][123582] Updated weights for policy 0, policy_version 44883 (0.0010) [2023-10-10 18:19:46,160][123582] Updated weights for policy 0, policy_version 44893 (0.0009) [2023-10-10 18:19:48,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 91848704. Throughput: 0: 1803.9, 1: 1827.3. Samples: 22973722. Policy #0 lag: (min: 7.0, avg: 7.6, max: 23.0) [2023-10-10 18:19:48,789][122664] Avg episode reward: [(0, '62.830'), (1, '51.510')] [2023-10-10 18:19:49,177][123614] Updated weights for policy 1, policy_version 44810 (0.0008) [2023-10-10 18:19:49,537][123614] Updated weights for policy 1, policy_version 44820 (0.0008) [2023-10-10 18:19:49,811][123582] Updated weights for policy 0, policy_version 44903 (0.0008) [2023-10-10 18:19:49,901][123614] Updated weights for policy 1, policy_version 44830 (0.0009) [2023-10-10 18:19:50,189][123582] Updated weights for policy 0, policy_version 44913 (0.0009) [2023-10-10 18:19:50,555][123582] Updated weights for policy 0, policy_version 44923 (0.0009) [2023-10-10 18:19:53,626][123614] Updated weights for policy 1, policy_version 44840 (0.0007) [2023-10-10 18:19:53,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 91914240. Throughput: 0: 1812.7, 1: 1816.1. Samples: 22995968. Policy #0 lag: (min: 7.0, avg: 7.6, max: 23.0) [2023-10-10 18:19:53,788][122664] Avg episode reward: [(0, '65.490'), (1, '51.480')] [2023-10-10 18:19:53,797][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000044928_46006272.pth... [2023-10-10 18:19:53,827][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000043264_44302336.pth [2023-10-10 18:19:54,001][123614] Updated weights for policy 1, policy_version 44850 (0.0009) [2023-10-10 18:19:54,174][123582] Updated weights for policy 0, policy_version 44933 (0.0007) [2023-10-10 18:19:54,368][123614] Updated weights for policy 1, policy_version 44860 (0.0008) [2023-10-10 18:19:54,505][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000044864_45940736.pth... [2023-10-10 18:19:54,534][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000043168_44204032.pth [2023-10-10 18:19:54,545][123582] Updated weights for policy 0, policy_version 44943 (0.0008) [2023-10-10 18:19:54,918][123582] Updated weights for policy 0, policy_version 44953 (0.0008) [2023-10-10 18:19:58,189][123614] Updated weights for policy 1, policy_version 44870 (0.0009) [2023-10-10 18:19:58,481][123582] Updated weights for policy 0, policy_version 44963 (0.0008) [2023-10-10 18:19:58,558][123614] Updated weights for policy 1, policy_version 44880 (0.0009) [2023-10-10 18:19:58,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 91979776. Throughput: 0: 1816.4, 1: 1818.3. Samples: 23006320. Policy #0 lag: (min: 7.0, avg: 7.6, max: 23.0) [2023-10-10 18:19:58,788][122664] Avg episode reward: [(0, '65.820'), (1, '51.680')] [2023-10-10 18:19:58,863][123582] Updated weights for policy 0, policy_version 44973 (0.0007) [2023-10-10 18:19:58,933][123614] Updated weights for policy 1, policy_version 44890 (0.0008) [2023-10-10 18:19:59,225][123582] Updated weights for policy 0, policy_version 44983 (0.0009) [2023-10-10 18:20:02,624][123614] Updated weights for policy 1, policy_version 44900 (0.0007) [2023-10-10 18:20:02,954][123582] Updated weights for policy 0, policy_version 44993 (0.0009) [2023-10-10 18:20:02,992][123614] Updated weights for policy 1, policy_version 44910 (0.0011) [2023-10-10 18:20:03,320][123582] Updated weights for policy 0, policy_version 45003 (0.0009) [2023-10-10 18:20:03,363][123614] Updated weights for policy 1, policy_version 44920 (0.0009) [2023-10-10 18:20:03,704][123582] Updated weights for policy 0, policy_version 45013 (0.0009) [2023-10-10 18:20:03,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 92078080. Throughput: 0: 1808.8, 1: 1810.5. Samples: 23028364. Policy #0 lag: (min: 7.0, avg: 7.6, max: 23.0) [2023-10-10 18:20:03,789][122664] Avg episode reward: [(0, '62.540'), (1, '49.890')] [2023-10-10 18:20:04,068][123582] Updated weights for policy 0, policy_version 45023 (0.0010) [2023-10-10 18:20:07,125][123614] Updated weights for policy 1, policy_version 44930 (0.0008) [2023-10-10 18:20:07,482][123614] Updated weights for policy 1, policy_version 44940 (0.0009) [2023-10-10 18:20:07,564][123582] Updated weights for policy 0, policy_version 45033 (0.0008) [2023-10-10 18:20:07,849][123614] Updated weights for policy 1, policy_version 44950 (0.0007) [2023-10-10 18:20:07,938][123582] Updated weights for policy 0, policy_version 45043 (0.0008) [2023-10-10 18:20:08,217][123614] Updated weights for policy 1, policy_version 44960 (0.0009) [2023-10-10 18:20:08,300][123582] Updated weights for policy 0, policy_version 45053 (0.0007) [2023-10-10 18:20:08,788][122664] Fps is (10 sec: 19660.3, 60 sec: 15291.8, 300 sec: 14551.2). Total num frames: 92176384. Throughput: 0: 1809.6, 1: 1802.6. Samples: 23048456. Policy #0 lag: (min: 7.0, avg: 7.6, max: 23.0) [2023-10-10 18:20:08,789][122664] Avg episode reward: [(0, '62.600'), (1, '46.840')] [2023-10-10 18:20:11,947][123614] Updated weights for policy 1, policy_version 44970 (0.0008) [2023-10-10 18:20:12,060][123582] Updated weights for policy 0, policy_version 45063 (0.0008) [2023-10-10 18:20:12,313][123614] Updated weights for policy 1, policy_version 44980 (0.0008) [2023-10-10 18:20:12,439][123582] Updated weights for policy 0, policy_version 45073 (0.0010) [2023-10-10 18:20:12,686][123614] Updated weights for policy 1, policy_version 44990 (0.0008) [2023-10-10 18:20:12,814][123582] Updated weights for policy 0, policy_version 45083 (0.0008) [2023-10-10 18:20:13,788][122664] Fps is (10 sec: 16384.4, 60 sec: 15291.8, 300 sec: 14440.1). Total num frames: 92241920. Throughput: 0: 1820.7, 1: 1809.1. Samples: 23061206. Policy #0 lag: (min: 7.0, avg: 7.6, max: 23.0) [2023-10-10 18:20:13,788][122664] Avg episode reward: [(0, '63.310'), (1, '43.050')] [2023-10-10 18:20:16,416][123614] Updated weights for policy 1, policy_version 45000 (0.0009) [2023-10-10 18:20:16,479][123582] Updated weights for policy 0, policy_version 45093 (0.0009) [2023-10-10 18:20:16,789][123614] Updated weights for policy 1, policy_version 45010 (0.0009) [2023-10-10 18:20:16,855][123582] Updated weights for policy 0, policy_version 45103 (0.0008) [2023-10-10 18:20:17,145][123614] Updated weights for policy 1, policy_version 45020 (0.0008) [2023-10-10 18:20:17,228][123582] Updated weights for policy 0, policy_version 45113 (0.0009) [2023-10-10 18:20:18,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 92307456. Throughput: 0: 1807.1, 1: 1798.5. Samples: 23080908. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:20:18,789][122664] Avg episode reward: [(0, '64.020'), (1, '44.010')] [2023-10-10 18:20:20,862][123614] Updated weights for policy 1, policy_version 45030 (0.0010) [2023-10-10 18:20:21,063][123582] Updated weights for policy 0, policy_version 45123 (0.0009) [2023-10-10 18:20:21,232][123614] Updated weights for policy 1, policy_version 45040 (0.0010) [2023-10-10 18:20:21,431][123582] Updated weights for policy 0, policy_version 45133 (0.0008) [2023-10-10 18:20:21,592][123614] Updated weights for policy 1, policy_version 45050 (0.0008) [2023-10-10 18:20:21,808][123582] Updated weights for policy 0, policy_version 45143 (0.0008) [2023-10-10 18:20:23,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 92372992. Throughput: 0: 1810.8, 1: 1804.3. Samples: 23103490. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:20:23,788][122664] Avg episode reward: [(0, '63.990'), (1, '44.910')] [2023-10-10 18:20:25,239][123614] Updated weights for policy 1, policy_version 45060 (0.0007) [2023-10-10 18:20:25,565][123582] Updated weights for policy 0, policy_version 45153 (0.0008) [2023-10-10 18:20:25,605][123614] Updated weights for policy 1, policy_version 45070 (0.0008) [2023-10-10 18:20:25,935][123582] Updated weights for policy 0, policy_version 45163 (0.0007) [2023-10-10 18:20:25,971][123614] Updated weights for policy 1, policy_version 45080 (0.0008) [2023-10-10 18:20:26,303][123582] Updated weights for policy 0, policy_version 45173 (0.0007) [2023-10-10 18:20:26,675][123582] Updated weights for policy 0, policy_version 45183 (0.0007) [2023-10-10 18:20:28,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 92438528. Throughput: 0: 1810.5, 1: 1808.2. Samples: 23113876. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:20:28,789][122664] Avg episode reward: [(0, '64.280'), (1, '40.920')] [2023-10-10 18:20:29,726][123614] Updated weights for policy 1, policy_version 45090 (0.0007) [2023-10-10 18:20:30,085][123614] Updated weights for policy 1, policy_version 45100 (0.0008) [2023-10-10 18:20:30,449][123614] Updated weights for policy 1, policy_version 45110 (0.0009) [2023-10-10 18:20:30,510][123582] Updated weights for policy 0, policy_version 45193 (0.0007) [2023-10-10 18:20:30,817][123614] Updated weights for policy 1, policy_version 45120 (0.0007) [2023-10-10 18:20:30,882][123582] Updated weights for policy 0, policy_version 45203 (0.0008) [2023-10-10 18:20:31,269][123582] Updated weights for policy 0, policy_version 45213 (0.0010) [2023-10-10 18:20:33,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 92504064. Throughput: 0: 1808.2, 1: 1805.0. Samples: 23136316. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:20:33,788][122664] Avg episode reward: [(0, '65.850'), (1, '41.540')] [2023-10-10 18:20:34,439][123614] Updated weights for policy 1, policy_version 45130 (0.0008) [2023-10-10 18:20:34,802][123614] Updated weights for policy 1, policy_version 45140 (0.0009) [2023-10-10 18:20:35,035][123582] Updated weights for policy 0, policy_version 45223 (0.0008) [2023-10-10 18:20:35,166][123614] Updated weights for policy 1, policy_version 45150 (0.0008) [2023-10-10 18:20:35,403][123582] Updated weights for policy 0, policy_version 45233 (0.0007) [2023-10-10 18:20:35,771][123582] Updated weights for policy 0, policy_version 45243 (0.0007) [2023-10-10 18:20:38,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 92569600. Throughput: 0: 1804.4, 1: 1815.4. Samples: 23158858. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:20:38,788][122664] Avg episode reward: [(0, '61.260'), (1, '42.910')] [2023-10-10 18:20:38,808][123614] Updated weights for policy 1, policy_version 45160 (0.0007) [2023-10-10 18:20:39,180][123614] Updated weights for policy 1, policy_version 45170 (0.0009) [2023-10-10 18:20:39,521][123582] Updated weights for policy 0, policy_version 45253 (0.0008) [2023-10-10 18:20:39,549][123614] Updated weights for policy 1, policy_version 45180 (0.0010) [2023-10-10 18:20:39,891][123582] Updated weights for policy 0, policy_version 45263 (0.0008) [2023-10-10 18:20:40,262][123582] Updated weights for policy 0, policy_version 45273 (0.0008) [2023-10-10 18:20:43,291][123614] Updated weights for policy 1, policy_version 45190 (0.0009) [2023-10-10 18:20:43,667][123614] Updated weights for policy 1, policy_version 45200 (0.0008) [2023-10-10 18:20:43,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 92635136. Throughput: 0: 1801.3, 1: 1812.8. Samples: 23168956. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:20:43,789][122664] Avg episode reward: [(0, '59.540'), (1, '41.550')] [2023-10-10 18:20:43,992][123582] Updated weights for policy 0, policy_version 45283 (0.0010) [2023-10-10 18:20:44,034][123614] Updated weights for policy 1, policy_version 45210 (0.0009) [2023-10-10 18:20:44,364][123582] Updated weights for policy 0, policy_version 45293 (0.0010) [2023-10-10 18:20:44,734][123582] Updated weights for policy 0, policy_version 45303 (0.0009) [2023-10-10 18:20:47,702][123614] Updated weights for policy 1, policy_version 45220 (0.0008) [2023-10-10 18:20:48,071][123614] Updated weights for policy 1, policy_version 45230 (0.0008) [2023-10-10 18:20:48,442][123614] Updated weights for policy 1, policy_version 45240 (0.0010) [2023-10-10 18:20:48,452][123582] Updated weights for policy 0, policy_version 45313 (0.0010) [2023-10-10 18:20:48,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 92733440. Throughput: 0: 1805.7, 1: 1820.6. Samples: 23191550. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:20:48,788][122664] Avg episode reward: [(0, '57.530'), (1, '42.600')] [2023-10-10 18:20:48,835][123582] Updated weights for policy 0, policy_version 45323 (0.0010) [2023-10-10 18:20:49,197][123582] Updated weights for policy 0, policy_version 45333 (0.0011) [2023-10-10 18:20:49,569][123582] Updated weights for policy 0, policy_version 45343 (0.0010) [2023-10-10 18:20:52,090][123614] Updated weights for policy 1, policy_version 45250 (0.0008) [2023-10-10 18:20:52,461][123614] Updated weights for policy 1, policy_version 45260 (0.0010) [2023-10-10 18:20:52,840][123614] Updated weights for policy 1, policy_version 45270 (0.0011) [2023-10-10 18:20:53,200][123614] Updated weights for policy 1, policy_version 45280 (0.0009) [2023-10-10 18:20:53,371][123582] Updated weights for policy 0, policy_version 45353 (0.0008) [2023-10-10 18:20:53,740][123582] Updated weights for policy 0, policy_version 45363 (0.0007) [2023-10-10 18:20:53,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 92798976. Throughput: 0: 1819.4, 1: 1826.4. Samples: 23212516. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-10-10 18:20:53,789][122664] Avg episode reward: [(0, '56.380'), (1, '44.590')] [2023-10-10 18:20:54,114][123582] Updated weights for policy 0, policy_version 45373 (0.0007) [2023-10-10 18:20:57,070][123614] Updated weights for policy 1, policy_version 45290 (0.0008) [2023-10-10 18:20:57,437][123614] Updated weights for policy 1, policy_version 45300 (0.0010) [2023-10-10 18:20:57,810][123614] Updated weights for policy 1, policy_version 45310 (0.0009) [2023-10-10 18:20:57,931][123582] Updated weights for policy 0, policy_version 45383 (0.0009) [2023-10-10 18:20:58,310][123582] Updated weights for policy 0, policy_version 45393 (0.0009) [2023-10-10 18:20:58,687][123582] Updated weights for policy 0, policy_version 45403 (0.0007) [2023-10-10 18:20:58,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 92864512. Throughput: 0: 1796.0, 1: 1827.5. Samples: 23224266. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-10-10 18:20:58,788][122664] Avg episode reward: [(0, '53.340'), (1, '44.920')] [2023-10-10 18:21:01,479][123614] Updated weights for policy 1, policy_version 45320 (0.0008) [2023-10-10 18:21:01,848][123614] Updated weights for policy 1, policy_version 45330 (0.0007) [2023-10-10 18:21:02,218][123614] Updated weights for policy 1, policy_version 45340 (0.0008) [2023-10-10 18:21:02,308][123582] Updated weights for policy 0, policy_version 45413 (0.0007) [2023-10-10 18:21:02,687][123582] Updated weights for policy 0, policy_version 45423 (0.0008) [2023-10-10 18:21:03,059][123582] Updated weights for policy 0, policy_version 45433 (0.0007) [2023-10-10 18:21:03,788][122664] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 92962816. Throughput: 0: 1817.2, 1: 1827.6. Samples: 23244922. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-10-10 18:21:03,788][122664] Avg episode reward: [(0, '56.420'), (1, '46.970')] [2023-10-10 18:21:06,050][123614] Updated weights for policy 1, policy_version 45350 (0.0009) [2023-10-10 18:21:06,415][123614] Updated weights for policy 1, policy_version 45360 (0.0011) [2023-10-10 18:21:06,725][123582] Updated weights for policy 0, policy_version 45443 (0.0007) [2023-10-10 18:21:06,785][123614] Updated weights for policy 1, policy_version 45370 (0.0009) [2023-10-10 18:21:07,100][123582] Updated weights for policy 0, policy_version 45453 (0.0009) [2023-10-10 18:21:07,480][123582] Updated weights for policy 0, policy_version 45463 (0.0011) [2023-10-10 18:21:08,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 93028352. Throughput: 0: 1799.5, 1: 1823.5. Samples: 23266524. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-10-10 18:21:08,788][122664] Avg episode reward: [(0, '55.660'), (1, '44.760')] [2023-10-10 18:21:10,322][123614] Updated weights for policy 1, policy_version 45380 (0.0008) [2023-10-10 18:21:10,694][123614] Updated weights for policy 1, policy_version 45390 (0.0008) [2023-10-10 18:21:11,070][123614] Updated weights for policy 1, policy_version 45400 (0.0009) [2023-10-10 18:21:11,355][123582] Updated weights for policy 0, policy_version 45473 (0.0010) [2023-10-10 18:21:11,729][123582] Updated weights for policy 0, policy_version 45483 (0.0010) [2023-10-10 18:21:12,112][123582] Updated weights for policy 0, policy_version 45493 (0.0009) [2023-10-10 18:21:12,484][123582] Updated weights for policy 0, policy_version 45503 (0.0009) [2023-10-10 18:21:13,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 93093888. Throughput: 0: 1815.7, 1: 1819.6. Samples: 23277464. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-10-10 18:21:13,789][122664] Avg episode reward: [(0, '56.910'), (1, '45.130')] [2023-10-10 18:21:14,826][123614] Updated weights for policy 1, policy_version 45410 (0.0007) [2023-10-10 18:21:15,191][123614] Updated weights for policy 1, policy_version 45420 (0.0008) [2023-10-10 18:21:15,570][123614] Updated weights for policy 1, policy_version 45430 (0.0007) [2023-10-10 18:21:15,927][123614] Updated weights for policy 1, policy_version 45440 (0.0007) [2023-10-10 18:21:16,233][123582] Updated weights for policy 0, policy_version 45513 (0.0007) [2023-10-10 18:21:16,610][123582] Updated weights for policy 0, policy_version 45523 (0.0008) [2023-10-10 18:21:16,980][123582] Updated weights for policy 0, policy_version 45533 (0.0007) [2023-10-10 18:21:18,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 93159424. Throughput: 0: 1797.0, 1: 1815.7. Samples: 23298888. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-10-10 18:21:18,789][122664] Avg episode reward: [(0, '58.130'), (1, '47.030')] [2023-10-10 18:21:19,525][123614] Updated weights for policy 1, policy_version 45450 (0.0007) [2023-10-10 18:21:19,904][123614] Updated weights for policy 1, policy_version 45460 (0.0008) [2023-10-10 18:21:20,270][123614] Updated weights for policy 1, policy_version 45470 (0.0008) [2023-10-10 18:21:20,542][123582] Updated weights for policy 0, policy_version 45543 (0.0008) [2023-10-10 18:21:20,915][123582] Updated weights for policy 0, policy_version 45553 (0.0007) [2023-10-10 18:21:21,285][123582] Updated weights for policy 0, policy_version 45563 (0.0008) [2023-10-10 18:21:23,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 93224960. Throughput: 0: 1802.0, 1: 1818.8. Samples: 23321796. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:21:23,789][122664] Avg episode reward: [(0, '58.740'), (1, '48.120')] [2023-10-10 18:21:24,105][123614] Updated weights for policy 1, policy_version 45480 (0.0007) [2023-10-10 18:21:24,474][123614] Updated weights for policy 1, policy_version 45490 (0.0008) [2023-10-10 18:21:24,831][123614] Updated weights for policy 1, policy_version 45500 (0.0007) [2023-10-10 18:21:24,937][123582] Updated weights for policy 0, policy_version 45573 (0.0007) [2023-10-10 18:21:25,307][123582] Updated weights for policy 0, policy_version 45583 (0.0007) [2023-10-10 18:21:25,676][123582] Updated weights for policy 0, policy_version 45593 (0.0007) [2023-10-10 18:21:28,563][123614] Updated weights for policy 1, policy_version 45510 (0.0008) [2023-10-10 18:21:28,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 93290496. Throughput: 0: 1802.8, 1: 1814.0. Samples: 23331716. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:21:28,789][122664] Avg episode reward: [(0, '61.720'), (1, '46.680')] [2023-10-10 18:21:28,939][123614] Updated weights for policy 1, policy_version 45520 (0.0007) [2023-10-10 18:21:29,304][123614] Updated weights for policy 1, policy_version 45530 (0.0008) [2023-10-10 18:21:29,376][123582] Updated weights for policy 0, policy_version 45603 (0.0008) [2023-10-10 18:21:29,744][123582] Updated weights for policy 0, policy_version 45613 (0.0008) [2023-10-10 18:21:30,124][123582] Updated weights for policy 0, policy_version 45623 (0.0010) [2023-10-10 18:21:33,030][123614] Updated weights for policy 1, policy_version 45540 (0.0007) [2023-10-10 18:21:33,411][123614] Updated weights for policy 1, policy_version 45550 (0.0010) [2023-10-10 18:21:33,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 93356032. Throughput: 0: 1804.5, 1: 1815.0. Samples: 23354430. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:21:33,789][122664] Avg episode reward: [(0, '60.480'), (1, '44.420')] [2023-10-10 18:21:33,795][123614] Updated weights for policy 1, policy_version 45560 (0.0008) [2023-10-10 18:21:33,850][123582] Updated weights for policy 0, policy_version 45633 (0.0011) [2023-10-10 18:21:34,209][123582] Updated weights for policy 0, policy_version 45643 (0.0011) [2023-10-10 18:21:34,589][123582] Updated weights for policy 0, policy_version 45653 (0.0010) [2023-10-10 18:21:34,968][123582] Updated weights for policy 0, policy_version 45663 (0.0008) [2023-10-10 18:21:37,534][123614] Updated weights for policy 1, policy_version 45570 (0.0008) [2023-10-10 18:21:37,903][123614] Updated weights for policy 1, policy_version 45580 (0.0011) [2023-10-10 18:21:38,269][123614] Updated weights for policy 1, policy_version 45590 (0.0008) [2023-10-10 18:21:38,624][123582] Updated weights for policy 0, policy_version 45673 (0.0007) [2023-10-10 18:21:38,647][123614] Updated weights for policy 1, policy_version 45600 (0.0007) [2023-10-10 18:21:38,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 93454336. Throughput: 0: 1817.4, 1: 1809.0. Samples: 23375706. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:21:38,789][122664] Avg episode reward: [(0, '60.810'), (1, '44.620')] [2023-10-10 18:21:38,996][123582] Updated weights for policy 0, policy_version 45683 (0.0008) [2023-10-10 18:21:39,370][123582] Updated weights for policy 0, policy_version 45693 (0.0009) [2023-10-10 18:21:42,319][123614] Updated weights for policy 1, policy_version 45610 (0.0008) [2023-10-10 18:21:42,682][123614] Updated weights for policy 1, policy_version 45620 (0.0008) [2023-10-10 18:21:43,042][123614] Updated weights for policy 1, policy_version 45630 (0.0008) [2023-10-10 18:21:43,081][123582] Updated weights for policy 0, policy_version 45703 (0.0009) [2023-10-10 18:21:43,458][123582] Updated weights for policy 0, policy_version 45713 (0.0009) [2023-10-10 18:21:43,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 93519872. Throughput: 0: 1808.0, 1: 1809.4. Samples: 23387050. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:21:43,788][122664] Avg episode reward: [(0, '61.210'), (1, '44.200')] [2023-10-10 18:21:43,820][123582] Updated weights for policy 0, policy_version 45723 (0.0007) [2023-10-10 18:21:46,819][123614] Updated weights for policy 1, policy_version 45640 (0.0007) [2023-10-10 18:21:47,181][123614] Updated weights for policy 1, policy_version 45650 (0.0009) [2023-10-10 18:21:47,302][123582] Updated weights for policy 0, policy_version 45733 (0.0007) [2023-10-10 18:21:47,545][123614] Updated weights for policy 1, policy_version 45660 (0.0007) [2023-10-10 18:21:47,669][123582] Updated weights for policy 0, policy_version 45743 (0.0007) [2023-10-10 18:21:48,051][123582] Updated weights for policy 0, policy_version 45753 (0.0009) [2023-10-10 18:21:48,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 93618176. Throughput: 0: 1820.2, 1: 1805.3. Samples: 23408070. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:21:48,789][122664] Avg episode reward: [(0, '62.130'), (1, '46.920')] [2023-10-10 18:21:51,376][123614] Updated weights for policy 1, policy_version 45670 (0.0007) [2023-10-10 18:21:51,748][123582] Updated weights for policy 0, policy_version 45763 (0.0008) [2023-10-10 18:21:51,751][123614] Updated weights for policy 1, policy_version 45680 (0.0009) [2023-10-10 18:21:52,120][123614] Updated weights for policy 1, policy_version 45690 (0.0007) [2023-10-10 18:21:52,124][123582] Updated weights for policy 0, policy_version 45773 (0.0008) [2023-10-10 18:21:52,502][123582] Updated weights for policy 0, policy_version 45783 (0.0008) [2023-10-10 18:21:53,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 93683712. Throughput: 0: 1820.0, 1: 1802.4. Samples: 23429534. Policy #0 lag: (min: 5.0, avg: 6.3, max: 28.0) [2023-10-10 18:21:53,789][122664] Avg episode reward: [(0, '60.500'), (1, '47.030')] [2023-10-10 18:21:53,797][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000045696_46792704.pth... [2023-10-10 18:21:53,797][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000045792_46891008.pth... [2023-10-10 18:21:53,829][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000044000_45056000.pth [2023-10-10 18:21:53,837][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000044096_45154304.pth [2023-10-10 18:21:55,747][123614] Updated weights for policy 1, policy_version 45700 (0.0008) [2023-10-10 18:21:56,073][123582] Updated weights for policy 0, policy_version 45793 (0.0009) [2023-10-10 18:21:56,110][123614] Updated weights for policy 1, policy_version 45710 (0.0007) [2023-10-10 18:21:56,442][123582] Updated weights for policy 0, policy_version 45803 (0.0008) [2023-10-10 18:21:56,482][123614] Updated weights for policy 1, policy_version 45720 (0.0008) [2023-10-10 18:21:56,829][123582] Updated weights for policy 0, policy_version 45813 (0.0009) [2023-10-10 18:21:57,199][123582] Updated weights for policy 0, policy_version 45823 (0.0007) [2023-10-10 18:21:58,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 93749248. Throughput: 0: 1818.4, 1: 1809.8. Samples: 23440730. Policy #0 lag: (min: 5.0, avg: 6.3, max: 28.0) [2023-10-10 18:21:58,789][122664] Avg episode reward: [(0, '65.910'), (1, '51.680')] [2023-10-10 18:22:00,153][123614] Updated weights for policy 1, policy_version 45730 (0.0008) [2023-10-10 18:22:00,529][123614] Updated weights for policy 1, policy_version 45740 (0.0008) [2023-10-10 18:22:00,886][123614] Updated weights for policy 1, policy_version 45750 (0.0008) [2023-10-10 18:22:00,999][123582] Updated weights for policy 0, policy_version 45833 (0.0008) [2023-10-10 18:22:01,259][123614] Updated weights for policy 1, policy_version 45760 (0.0009) [2023-10-10 18:22:01,366][123582] Updated weights for policy 0, policy_version 45843 (0.0008) [2023-10-10 18:22:01,739][123582] Updated weights for policy 0, policy_version 45853 (0.0007) [2023-10-10 18:22:03,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 93814784. Throughput: 0: 1826.4, 1: 1807.2. Samples: 23462398. Policy #0 lag: (min: 5.0, avg: 6.3, max: 28.0) [2023-10-10 18:22:03,789][122664] Avg episode reward: [(0, '69.300'), (1, '51.620')] [2023-10-10 18:22:03,789][123247] Saving new best policy, reward=69.300! [2023-10-10 18:22:04,938][123614] Updated weights for policy 1, policy_version 45770 (0.0007) [2023-10-10 18:22:05,231][123582] Updated weights for policy 0, policy_version 45863 (0.0008) [2023-10-10 18:22:05,303][123614] Updated weights for policy 1, policy_version 45780 (0.0008) [2023-10-10 18:22:05,600][123582] Updated weights for policy 0, policy_version 45873 (0.0008) [2023-10-10 18:22:05,673][123614] Updated weights for policy 1, policy_version 45790 (0.0008) [2023-10-10 18:22:05,976][123582] Updated weights for policy 0, policy_version 45883 (0.0008) [2023-10-10 18:22:08,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 93880320. Throughput: 0: 1822.0, 1: 1801.8. Samples: 23484868. Policy #0 lag: (min: 5.0, avg: 6.3, max: 28.0) [2023-10-10 18:22:08,789][122664] Avg episode reward: [(0, '68.900'), (1, '48.370')] [2023-10-10 18:22:09,485][123614] Updated weights for policy 1, policy_version 45800 (0.0008) [2023-10-10 18:22:09,798][123582] Updated weights for policy 0, policy_version 45893 (0.0010) [2023-10-10 18:22:09,855][123614] Updated weights for policy 1, policy_version 45810 (0.0010) [2023-10-10 18:22:10,176][123582] Updated weights for policy 0, policy_version 45903 (0.0010) [2023-10-10 18:22:10,226][123614] Updated weights for policy 1, policy_version 45820 (0.0007) [2023-10-10 18:22:10,558][123582] Updated weights for policy 0, policy_version 45913 (0.0010) [2023-10-10 18:22:13,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 93945856. Throughput: 0: 1820.1, 1: 1800.0. Samples: 23494624. Policy #0 lag: (min: 5.0, avg: 6.3, max: 28.0) [2023-10-10 18:22:13,789][122664] Avg episode reward: [(0, '66.700'), (1, '48.630')] [2023-10-10 18:22:13,960][123614] Updated weights for policy 1, policy_version 45830 (0.0007) [2023-10-10 18:22:14,273][123582] Updated weights for policy 0, policy_version 45923 (0.0010) [2023-10-10 18:22:14,325][123614] Updated weights for policy 1, policy_version 45840 (0.0010) [2023-10-10 18:22:14,649][123582] Updated weights for policy 0, policy_version 45933 (0.0007) [2023-10-10 18:22:14,692][123614] Updated weights for policy 1, policy_version 45850 (0.0007) [2023-10-10 18:22:15,006][123582] Updated weights for policy 0, policy_version 45943 (0.0008) [2023-10-10 18:22:18,361][123614] Updated weights for policy 1, policy_version 45860 (0.0007) [2023-10-10 18:22:18,472][123582] Updated weights for policy 0, policy_version 45953 (0.0007) [2023-10-10 18:22:18,723][123614] Updated weights for policy 1, policy_version 45870 (0.0007) [2023-10-10 18:22:18,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 94011392. Throughput: 0: 1825.7, 1: 1799.6. Samples: 23517566. Policy #0 lag: (min: 5.0, avg: 6.3, max: 28.0) [2023-10-10 18:22:18,789][122664] Avg episode reward: [(0, '66.670'), (1, '51.010')] [2023-10-10 18:22:18,837][123582] Updated weights for policy 0, policy_version 45963 (0.0008) [2023-10-10 18:22:19,098][123614] Updated weights for policy 1, policy_version 45880 (0.0007) [2023-10-10 18:22:19,208][123582] Updated weights for policy 0, policy_version 45973 (0.0009) [2023-10-10 18:22:19,582][123582] Updated weights for policy 0, policy_version 45983 (0.0007) [2023-10-10 18:22:22,850][123614] Updated weights for policy 1, policy_version 45890 (0.0011) [2023-10-10 18:22:23,216][123614] Updated weights for policy 1, policy_version 45900 (0.0009) [2023-10-10 18:22:23,484][123582] Updated weights for policy 0, policy_version 45993 (0.0008) [2023-10-10 18:22:23,582][123614] Updated weights for policy 1, policy_version 45910 (0.0007) [2023-10-10 18:22:23,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 94076928. Throughput: 0: 1813.8, 1: 1808.6. Samples: 23538716. Policy #0 lag: (min: 5.0, avg: 6.3, max: 28.0) [2023-10-10 18:22:23,788][122664] Avg episode reward: [(0, '67.210'), (1, '50.240')] [2023-10-10 18:22:23,850][123582] Updated weights for policy 0, policy_version 46003 (0.0007) [2023-10-10 18:22:23,955][123614] Updated weights for policy 1, policy_version 45920 (0.0008) [2023-10-10 18:22:24,222][123582] Updated weights for policy 0, policy_version 46013 (0.0008) [2023-10-10 18:22:27,644][123614] Updated weights for policy 1, policy_version 45930 (0.0008) [2023-10-10 18:22:28,021][123614] Updated weights for policy 1, policy_version 45940 (0.0008) [2023-10-10 18:22:28,134][123582] Updated weights for policy 0, policy_version 46023 (0.0009) [2023-10-10 18:22:28,388][123614] Updated weights for policy 1, policy_version 45950 (0.0009) [2023-10-10 18:22:28,510][123582] Updated weights for policy 0, policy_version 46033 (0.0008) [2023-10-10 18:22:28,788][122664] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 94175232. Throughput: 0: 1820.7, 1: 1802.5. Samples: 23550092. Policy #0 lag: (min: 4.0, avg: 4.1, max: 11.0) [2023-10-10 18:22:28,788][122664] Avg episode reward: [(0, '67.630'), (1, '50.880')] [2023-10-10 18:22:28,881][123582] Updated weights for policy 0, policy_version 46043 (0.0007) [2023-10-10 18:22:32,140][123614] Updated weights for policy 1, policy_version 45960 (0.0010) [2023-10-10 18:22:32,466][123582] Updated weights for policy 0, policy_version 46053 (0.0007) [2023-10-10 18:22:32,513][123614] Updated weights for policy 1, policy_version 45970 (0.0008) [2023-10-10 18:22:32,837][123582] Updated weights for policy 0, policy_version 46063 (0.0009) [2023-10-10 18:22:32,870][123614] Updated weights for policy 1, policy_version 45980 (0.0008) [2023-10-10 18:22:33,209][123582] Updated weights for policy 0, policy_version 46073 (0.0009) [2023-10-10 18:22:33,788][122664] Fps is (10 sec: 19661.0, 60 sec: 15291.8, 300 sec: 14551.2). Total num frames: 94273536. Throughput: 0: 1819.7, 1: 1812.4. Samples: 23571510. Policy #0 lag: (min: 4.0, avg: 4.1, max: 11.0) [2023-10-10 18:22:33,788][122664] Avg episode reward: [(0, '69.310'), (1, '48.280')] [2023-10-10 18:22:33,789][123247] Saving new best policy, reward=69.310! [2023-10-10 18:22:36,696][123614] Updated weights for policy 1, policy_version 45990 (0.0010) [2023-10-10 18:22:37,045][123582] Updated weights for policy 0, policy_version 46083 (0.0010) [2023-10-10 18:22:37,077][123614] Updated weights for policy 1, policy_version 46000 (0.0008) [2023-10-10 18:22:37,424][123582] Updated weights for policy 0, policy_version 46093 (0.0009) [2023-10-10 18:22:37,446][123614] Updated weights for policy 1, policy_version 46010 (0.0008) [2023-10-10 18:22:37,794][123582] Updated weights for policy 0, policy_version 46103 (0.0009) [2023-10-10 18:22:38,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 94339072. Throughput: 0: 1810.2, 1: 1803.1. Samples: 23592134. Policy #0 lag: (min: 4.0, avg: 4.1, max: 11.0) [2023-10-10 18:22:38,789][122664] Avg episode reward: [(0, '66.660'), (1, '48.000')] [2023-10-10 18:22:41,155][123614] Updated weights for policy 1, policy_version 46020 (0.0008) [2023-10-10 18:22:41,509][123582] Updated weights for policy 0, policy_version 46113 (0.0007) [2023-10-10 18:22:41,526][123614] Updated weights for policy 1, policy_version 46030 (0.0007) [2023-10-10 18:22:41,869][123582] Updated weights for policy 0, policy_version 46123 (0.0010) [2023-10-10 18:22:41,892][123614] Updated weights for policy 1, policy_version 46040 (0.0007) [2023-10-10 18:22:42,248][123582] Updated weights for policy 0, policy_version 46133 (0.0008) [2023-10-10 18:22:42,608][123582] Updated weights for policy 0, policy_version 46143 (0.0007) [2023-10-10 18:22:43,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 94404608. Throughput: 0: 1821.5, 1: 1809.2. Samples: 23604112. Policy #0 lag: (min: 4.0, avg: 4.1, max: 11.0) [2023-10-10 18:22:43,788][122664] Avg episode reward: [(0, '64.560'), (1, '40.840')] [2023-10-10 18:22:45,658][123614] Updated weights for policy 1, policy_version 46050 (0.0007) [2023-10-10 18:22:46,021][123614] Updated weights for policy 1, policy_version 46060 (0.0007) [2023-10-10 18:22:46,365][123582] Updated weights for policy 0, policy_version 46153 (0.0008) [2023-10-10 18:22:46,391][123614] Updated weights for policy 1, policy_version 46070 (0.0007) [2023-10-10 18:22:46,725][123582] Updated weights for policy 0, policy_version 46163 (0.0009) [2023-10-10 18:22:46,752][123614] Updated weights for policy 1, policy_version 46080 (0.0008) [2023-10-10 18:22:47,101][123582] Updated weights for policy 0, policy_version 46173 (0.0009) [2023-10-10 18:22:48,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 94470144. Throughput: 0: 1806.2, 1: 1795.6. Samples: 23624480. Policy #0 lag: (min: 4.0, avg: 4.1, max: 11.0) [2023-10-10 18:22:48,788][122664] Avg episode reward: [(0, '61.140'), (1, '41.060')] [2023-10-10 18:22:50,515][123614] Updated weights for policy 1, policy_version 46090 (0.0008) [2023-10-10 18:22:50,747][123582] Updated weights for policy 0, policy_version 46183 (0.0009) [2023-10-10 18:22:50,889][123614] Updated weights for policy 1, policy_version 46100 (0.0008) [2023-10-10 18:22:51,120][123582] Updated weights for policy 0, policy_version 46193 (0.0007) [2023-10-10 18:22:51,252][123614] Updated weights for policy 1, policy_version 46110 (0.0007) [2023-10-10 18:22:51,489][123582] Updated weights for policy 0, policy_version 46203 (0.0009) [2023-10-10 18:22:53,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 94535680. Throughput: 0: 1810.2, 1: 1793.7. Samples: 23647044. Policy #0 lag: (min: 4.0, avg: 4.1, max: 11.0) [2023-10-10 18:22:53,789][122664] Avg episode reward: [(0, '64.130'), (1, '43.610')] [2023-10-10 18:22:54,974][123614] Updated weights for policy 1, policy_version 46120 (0.0009) [2023-10-10 18:22:55,085][123582] Updated weights for policy 0, policy_version 46213 (0.0010) [2023-10-10 18:22:55,340][123614] Updated weights for policy 1, policy_version 46130 (0.0010) [2023-10-10 18:22:55,457][123582] Updated weights for policy 0, policy_version 46223 (0.0008) [2023-10-10 18:22:55,705][123614] Updated weights for policy 1, policy_version 46140 (0.0008) [2023-10-10 18:22:55,830][123582] Updated weights for policy 0, policy_version 46233 (0.0008) [2023-10-10 18:22:58,788][122664] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 94601216. Throughput: 0: 1810.1, 1: 1792.1. Samples: 23656724. Policy #0 lag: (min: 23.0, avg: 32.4, max: 55.0) [2023-10-10 18:22:58,789][122664] Avg episode reward: [(0, '59.990'), (1, '43.930')] [2023-10-10 18:22:59,572][123614] Updated weights for policy 1, policy_version 46150 (0.0008) [2023-10-10 18:22:59,579][123582] Updated weights for policy 0, policy_version 46243 (0.0008) [2023-10-10 18:22:59,939][123614] Updated weights for policy 1, policy_version 46160 (0.0008) [2023-10-10 18:22:59,941][123582] Updated weights for policy 0, policy_version 46253 (0.0008) [2023-10-10 18:23:00,311][123614] Updated weights for policy 1, policy_version 46170 (0.0008) [2023-10-10 18:23:00,317][123582] Updated weights for policy 0, policy_version 46263 (0.0007) [2023-10-10 18:23:03,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 94666752. Throughput: 0: 1798.7, 1: 1791.4. Samples: 23679122. Policy #0 lag: (min: 23.0, avg: 32.4, max: 55.0) [2023-10-10 18:23:03,789][122664] Avg episode reward: [(0, '55.230'), (1, '43.960')] [2023-10-10 18:23:03,942][123614] Updated weights for policy 1, policy_version 46180 (0.0007) [2023-10-10 18:23:04,122][123582] Updated weights for policy 0, policy_version 46273 (0.0007) [2023-10-10 18:23:04,308][123614] Updated weights for policy 1, policy_version 46190 (0.0007) [2023-10-10 18:23:04,493][123582] Updated weights for policy 0, policy_version 46283 (0.0008) [2023-10-10 18:23:04,674][123614] Updated weights for policy 1, policy_version 46200 (0.0008) [2023-10-10 18:23:04,860][123582] Updated weights for policy 0, policy_version 46293 (0.0009) [2023-10-10 18:23:05,228][123582] Updated weights for policy 0, policy_version 46303 (0.0009) [2023-10-10 18:23:08,542][123614] Updated weights for policy 1, policy_version 46210 (0.0009) [2023-10-10 18:23:08,788][122664] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 94732288. Throughput: 0: 1808.3, 1: 1806.4. Samples: 23701378. Policy #0 lag: (min: 23.0, avg: 32.4, max: 55.0) [2023-10-10 18:23:08,788][122664] Avg episode reward: [(0, '54.780'), (1, '43.320')] [2023-10-10 18:23:08,918][123614] Updated weights for policy 1, policy_version 46220 (0.0008) [2023-10-10 18:23:09,113][123582] Updated weights for policy 0, policy_version 46313 (0.0007) [2023-10-10 18:23:09,295][123614] Updated weights for policy 1, policy_version 46230 (0.0009) [2023-10-10 18:23:09,473][123582] Updated weights for policy 0, policy_version 46323 (0.0008) [2023-10-10 18:23:09,647][123614] Updated weights for policy 1, policy_version 46240 (0.0008) [2023-10-10 18:23:09,853][123582] Updated weights for policy 0, policy_version 46333 (0.0008) [2023-10-10 18:23:13,330][123614] Updated weights for policy 1, policy_version 46250 (0.0007) [2023-10-10 18:23:13,473][123582] Updated weights for policy 0, policy_version 46343 (0.0007) [2023-10-10 18:23:13,693][123614] Updated weights for policy 1, policy_version 46260 (0.0008) [2023-10-10 18:23:13,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 94797824. Throughput: 0: 1801.1, 1: 1788.7. Samples: 23711632. Policy #0 lag: (min: 23.0, avg: 32.4, max: 55.0) [2023-10-10 18:23:13,788][122664] Avg episode reward: [(0, '56.300'), (1, '42.870')] [2023-10-10 18:23:13,859][123582] Updated weights for policy 0, policy_version 46353 (0.0007) [2023-10-10 18:23:14,059][123614] Updated weights for policy 1, policy_version 46270 (0.0007) [2023-10-10 18:23:14,221][123582] Updated weights for policy 0, policy_version 46363 (0.0009) [2023-10-10 18:23:17,657][123614] Updated weights for policy 1, policy_version 46280 (0.0010) [2023-10-10 18:23:17,947][123582] Updated weights for policy 0, policy_version 46373 (0.0009) [2023-10-10 18:23:18,024][123614] Updated weights for policy 1, policy_version 46290 (0.0008) [2023-10-10 18:23:18,310][123582] Updated weights for policy 0, policy_version 46383 (0.0008) [2023-10-10 18:23:18,400][123614] Updated weights for policy 1, policy_version 46300 (0.0009) [2023-10-10 18:23:18,682][123582] Updated weights for policy 0, policy_version 46393 (0.0007) [2023-10-10 18:23:18,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 94896128. Throughput: 0: 1799.4, 1: 1807.9. Samples: 23733836. Policy #0 lag: (min: 23.0, avg: 32.4, max: 55.0) [2023-10-10 18:23:18,788][122664] Avg episode reward: [(0, '56.560'), (1, '43.360')] [2023-10-10 18:23:22,034][123614] Updated weights for policy 1, policy_version 46310 (0.0008) [2023-10-10 18:23:22,366][123582] Updated weights for policy 0, policy_version 46403 (0.0009) [2023-10-10 18:23:22,419][123614] Updated weights for policy 1, policy_version 46320 (0.0007) [2023-10-10 18:23:22,741][123582] Updated weights for policy 0, policy_version 46413 (0.0007) [2023-10-10 18:23:22,788][123614] Updated weights for policy 1, policy_version 46330 (0.0008) [2023-10-10 18:23:23,126][123582] Updated weights for policy 0, policy_version 46423 (0.0008) [2023-10-10 18:23:23,788][122664] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 94994432. Throughput: 0: 1799.5, 1: 1796.6. Samples: 23753956. Policy #0 lag: (min: 23.0, avg: 32.4, max: 55.0) [2023-10-10 18:23:23,789][122664] Avg episode reward: [(0, '57.930'), (1, '45.440')] [2023-10-10 18:23:26,378][123614] Updated weights for policy 1, policy_version 46340 (0.0008) [2023-10-10 18:23:26,742][123582] Updated weights for policy 0, policy_version 46433 (0.0007) [2023-10-10 18:23:26,753][123614] Updated weights for policy 1, policy_version 46350 (0.0007) [2023-10-10 18:23:27,115][123614] Updated weights for policy 1, policy_version 46360 (0.0010) [2023-10-10 18:23:27,119][123582] Updated weights for policy 0, policy_version 46443 (0.0007) [2023-10-10 18:23:27,483][123582] Updated weights for policy 0, policy_version 46453 (0.0009) [2023-10-10 18:23:27,857][123582] Updated weights for policy 0, policy_version 46463 (0.0008) [2023-10-10 18:23:28,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 95059968. Throughput: 0: 1799.5, 1: 1806.7. Samples: 23766392. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:23:28,789][122664] Avg episode reward: [(0, '57.480'), (1, '45.290')] [2023-10-10 18:23:30,838][123614] Updated weights for policy 1, policy_version 46370 (0.0007) [2023-10-10 18:23:31,204][123614] Updated weights for policy 1, policy_version 46380 (0.0011) [2023-10-10 18:23:31,572][123614] Updated weights for policy 1, policy_version 46390 (0.0009) [2023-10-10 18:23:31,594][123582] Updated weights for policy 0, policy_version 46473 (0.0007) [2023-10-10 18:23:31,938][123614] Updated weights for policy 1, policy_version 46400 (0.0008) [2023-10-10 18:23:31,967][123582] Updated weights for policy 0, policy_version 46483 (0.0009) [2023-10-10 18:23:32,342][123582] Updated weights for policy 0, policy_version 46493 (0.0007) [2023-10-10 18:23:33,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 95125504. Throughput: 0: 1800.2, 1: 1803.5. Samples: 23786650. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:23:33,789][122664] Avg episode reward: [(0, '62.110'), (1, '47.130')] [2023-10-10 18:23:35,660][123614] Updated weights for policy 1, policy_version 46410 (0.0007) [2023-10-10 18:23:36,029][123614] Updated weights for policy 1, policy_version 46420 (0.0008) [2023-10-10 18:23:36,115][123582] Updated weights for policy 0, policy_version 46503 (0.0007) [2023-10-10 18:23:36,397][123614] Updated weights for policy 1, policy_version 46430 (0.0008) [2023-10-10 18:23:36,478][123582] Updated weights for policy 0, policy_version 46513 (0.0007) [2023-10-10 18:23:36,851][123582] Updated weights for policy 0, policy_version 46523 (0.0009) [2023-10-10 18:23:38,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 95191040. Throughput: 0: 1797.7, 1: 1811.0. Samples: 23809434. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:23:38,789][122664] Avg episode reward: [(0, '60.940'), (1, '48.210')] [2023-10-10 18:23:40,149][123614] Updated weights for policy 1, policy_version 46440 (0.0008) [2023-10-10 18:23:40,498][123582] Updated weights for policy 0, policy_version 46533 (0.0007) [2023-10-10 18:23:40,521][123614] Updated weights for policy 1, policy_version 46450 (0.0010) [2023-10-10 18:23:40,867][123582] Updated weights for policy 0, policy_version 46543 (0.0009) [2023-10-10 18:23:40,894][123614] Updated weights for policy 1, policy_version 46460 (0.0007) [2023-10-10 18:23:41,241][123582] Updated weights for policy 0, policy_version 46553 (0.0010) [2023-10-10 18:23:43,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 95256576. Throughput: 0: 1805.6, 1: 1811.0. Samples: 23819474. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:23:43,789][122664] Avg episode reward: [(0, '60.080'), (1, '42.440')] [2023-10-10 18:23:44,504][123614] Updated weights for policy 1, policy_version 46470 (0.0008) [2023-10-10 18:23:44,868][123614] Updated weights for policy 1, policy_version 46480 (0.0008) [2023-10-10 18:23:45,097][123582] Updated weights for policy 0, policy_version 46563 (0.0009) [2023-10-10 18:23:45,230][123614] Updated weights for policy 1, policy_version 46490 (0.0007) [2023-10-10 18:23:45,461][123582] Updated weights for policy 0, policy_version 46573 (0.0008) [2023-10-10 18:23:45,839][123582] Updated weights for policy 0, policy_version 46583 (0.0007) [2023-10-10 18:23:48,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 95322112. Throughput: 0: 1804.5, 1: 1822.8. Samples: 23842350. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:23:48,789][122664] Avg episode reward: [(0, '61.890'), (1, '43.150')] [2023-10-10 18:23:48,872][123614] Updated weights for policy 1, policy_version 46500 (0.0007) [2023-10-10 18:23:49,243][123614] Updated weights for policy 1, policy_version 46510 (0.0009) [2023-10-10 18:23:49,450][123582] Updated weights for policy 0, policy_version 46593 (0.0009) [2023-10-10 18:23:49,613][123614] Updated weights for policy 1, policy_version 46520 (0.0008) [2023-10-10 18:23:49,820][123582] Updated weights for policy 0, policy_version 46603 (0.0008) [2023-10-10 18:23:50,196][123582] Updated weights for policy 0, policy_version 46613 (0.0009) [2023-10-10 18:23:50,570][123582] Updated weights for policy 0, policy_version 46623 (0.0008) [2023-10-10 18:23:53,346][123614] Updated weights for policy 1, policy_version 46530 (0.0008) [2023-10-10 18:23:53,709][123614] Updated weights for policy 1, policy_version 46540 (0.0007) [2023-10-10 18:23:53,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 95387648. Throughput: 0: 1809.7, 1: 1818.7. Samples: 23864658. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:23:53,788][122664] Avg episode reward: [(0, '63.100'), (1, '47.880')] [2023-10-10 18:23:53,796][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000046624_47742976.pth... [2023-10-10 18:23:53,832][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000044928_46006272.pth [2023-10-10 18:23:54,073][123614] Updated weights for policy 1, policy_version 46550 (0.0008) [2023-10-10 18:23:54,317][123582] Updated weights for policy 0, policy_version 46633 (0.0009) [2023-10-10 18:23:54,431][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000046560_47677440.pth... [2023-10-10 18:23:54,436][123614] Updated weights for policy 1, policy_version 46560 (0.0009) [2023-10-10 18:23:54,467][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000044864_45940736.pth [2023-10-10 18:23:54,684][123582] Updated weights for policy 0, policy_version 46643 (0.0009) [2023-10-10 18:23:55,060][123582] Updated weights for policy 0, policy_version 46653 (0.0007) [2023-10-10 18:23:58,259][123614] Updated weights for policy 1, policy_version 46570 (0.0011) [2023-10-10 18:23:58,629][123614] Updated weights for policy 1, policy_version 46580 (0.0008) [2023-10-10 18:23:58,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 95453184. Throughput: 0: 1807.0, 1: 1822.8. Samples: 23874974. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:23:58,788][122664] Avg episode reward: [(0, '62.570'), (1, '46.490')] [2023-10-10 18:23:58,849][123582] Updated weights for policy 0, policy_version 46663 (0.0007) [2023-10-10 18:23:58,983][123614] Updated weights for policy 1, policy_version 46590 (0.0008) [2023-10-10 18:23:59,211][123582] Updated weights for policy 0, policy_version 46673 (0.0008) [2023-10-10 18:23:59,581][123582] Updated weights for policy 0, policy_version 46683 (0.0010) [2023-10-10 18:24:02,721][123614] Updated weights for policy 1, policy_version 46600 (0.0008) [2023-10-10 18:24:03,088][123614] Updated weights for policy 1, policy_version 46610 (0.0007) [2023-10-10 18:24:03,105][123582] Updated weights for policy 0, policy_version 46693 (0.0009) [2023-10-10 18:24:03,464][123614] Updated weights for policy 1, policy_version 46620 (0.0007) [2023-10-10 18:24:03,483][123582] Updated weights for policy 0, policy_version 46703 (0.0007) [2023-10-10 18:24:03,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 95551488. Throughput: 0: 1808.0, 1: 1820.5. Samples: 23897120. Policy #0 lag: (min: 18.0, avg: 26.0, max: 50.0) [2023-10-10 18:24:03,789][122664] Avg episode reward: [(0, '62.550'), (1, '45.160')] [2023-10-10 18:24:03,852][123582] Updated weights for policy 0, policy_version 46713 (0.0009) [2023-10-10 18:24:07,329][123614] Updated weights for policy 1, policy_version 46630 (0.0009) [2023-10-10 18:24:07,595][123582] Updated weights for policy 0, policy_version 46723 (0.0009) [2023-10-10 18:24:07,701][123614] Updated weights for policy 1, policy_version 46640 (0.0008) [2023-10-10 18:24:07,956][123582] Updated weights for policy 0, policy_version 46733 (0.0007) [2023-10-10 18:24:08,069][123614] Updated weights for policy 1, policy_version 46650 (0.0009) [2023-10-10 18:24:08,324][123582] Updated weights for policy 0, policy_version 46743 (0.0009) [2023-10-10 18:24:08,788][122664] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 95649792. Throughput: 0: 1814.7, 1: 1819.7. Samples: 23917504. Policy #0 lag: (min: 18.0, avg: 26.0, max: 50.0) [2023-10-10 18:24:08,788][122664] Avg episode reward: [(0, '66.930'), (1, '44.220')] [2023-10-10 18:24:11,829][123614] Updated weights for policy 1, policy_version 46660 (0.0009) [2023-10-10 18:24:12,065][123582] Updated weights for policy 0, policy_version 46753 (0.0009) [2023-10-10 18:24:12,201][123614] Updated weights for policy 1, policy_version 46670 (0.0009) [2023-10-10 18:24:12,441][123582] Updated weights for policy 0, policy_version 46763 (0.0008) [2023-10-10 18:24:12,559][123614] Updated weights for policy 1, policy_version 46680 (0.0007) [2023-10-10 18:24:12,813][123582] Updated weights for policy 0, policy_version 46773 (0.0008) [2023-10-10 18:24:13,183][123582] Updated weights for policy 0, policy_version 46783 (0.0010) [2023-10-10 18:24:13,788][122664] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 95715328. Throughput: 0: 1807.1, 1: 1822.7. Samples: 23929732. Policy #0 lag: (min: 18.0, avg: 26.0, max: 50.0) [2023-10-10 18:24:13,789][122664] Avg episode reward: [(0, '65.230'), (1, '43.250')] [2023-10-10 18:24:16,274][123614] Updated weights for policy 1, policy_version 46690 (0.0008) [2023-10-10 18:24:16,640][123614] Updated weights for policy 1, policy_version 46700 (0.0008) [2023-10-10 18:24:16,783][123582] Updated weights for policy 0, policy_version 46793 (0.0007) [2023-10-10 18:24:17,012][123614] Updated weights for policy 1, policy_version 46710 (0.0008) [2023-10-10 18:24:17,146][123582] Updated weights for policy 0, policy_version 46803 (0.0008) [2023-10-10 18:24:17,371][123614] Updated weights for policy 1, policy_version 46720 (0.0008) [2023-10-10 18:24:17,522][123582] Updated weights for policy 0, policy_version 46813 (0.0008) [2023-10-10 18:24:18,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 95780864. Throughput: 0: 1822.3, 1: 1807.7. Samples: 23949998. Policy #0 lag: (min: 18.0, avg: 26.0, max: 50.0) [2023-10-10 18:24:18,788][122664] Avg episode reward: [(0, '67.370'), (1, '43.550')] [2023-10-10 18:24:21,099][123614] Updated weights for policy 1, policy_version 46730 (0.0007) [2023-10-10 18:24:21,185][123582] Updated weights for policy 0, policy_version 46823 (0.0008) [2023-10-10 18:24:21,468][123614] Updated weights for policy 1, policy_version 46740 (0.0007) [2023-10-10 18:24:21,552][123582] Updated weights for policy 0, policy_version 46833 (0.0009) [2023-10-10 18:24:21,836][123614] Updated weights for policy 1, policy_version 46750 (0.0007) [2023-10-10 18:24:21,921][123582] Updated weights for policy 0, policy_version 46843 (0.0009) [2023-10-10 18:24:23,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 95846400. Throughput: 0: 1811.3, 1: 1808.9. Samples: 23972342. Policy #0 lag: (min: 18.0, avg: 26.0, max: 50.0) [2023-10-10 18:24:23,788][122664] Avg episode reward: [(0, '70.570'), (1, '45.620')] [2023-10-10 18:24:23,797][123247] Saving new best policy, reward=70.570! [2023-10-10 18:24:25,617][123614] Updated weights for policy 1, policy_version 46760 (0.0007) [2023-10-10 18:24:25,669][123582] Updated weights for policy 0, policy_version 46853 (0.0009) [2023-10-10 18:24:25,988][123614] Updated weights for policy 1, policy_version 46770 (0.0007) [2023-10-10 18:24:26,040][123582] Updated weights for policy 0, policy_version 46863 (0.0007) [2023-10-10 18:24:26,355][123614] Updated weights for policy 1, policy_version 46780 (0.0008) [2023-10-10 18:24:26,407][123582] Updated weights for policy 0, policy_version 46873 (0.0009) [2023-10-10 18:24:28,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 95911936. Throughput: 0: 1814.3, 1: 1809.7. Samples: 23982552. Policy #0 lag: (min: 18.0, avg: 26.0, max: 50.0) [2023-10-10 18:24:28,789][122664] Avg episode reward: [(0, '71.550'), (1, '47.710')] [2023-10-10 18:24:28,789][123247] Saving new best policy, reward=71.550! [2023-10-10 18:24:30,047][123614] Updated weights for policy 1, policy_version 46790 (0.0007) [2023-10-10 18:24:30,324][123582] Updated weights for policy 0, policy_version 46883 (0.0008) [2023-10-10 18:24:30,406][123614] Updated weights for policy 1, policy_version 46800 (0.0008) [2023-10-10 18:24:30,696][123582] Updated weights for policy 0, policy_version 46893 (0.0009) [2023-10-10 18:24:30,779][123614] Updated weights for policy 1, policy_version 46810 (0.0007) [2023-10-10 18:24:31,069][123582] Updated weights for policy 0, policy_version 46903 (0.0010) [2023-10-10 18:24:33,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 95977472. Throughput: 0: 1805.1, 1: 1796.6. Samples: 24004426. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:24:33,789][122664] Avg episode reward: [(0, '70.670'), (1, '45.980')] [2023-10-10 18:24:34,397][123614] Updated weights for policy 1, policy_version 46820 (0.0008) [2023-10-10 18:24:34,768][123614] Updated weights for policy 1, policy_version 46830 (0.0008) [2023-10-10 18:24:34,889][123582] Updated weights for policy 0, policy_version 46913 (0.0007) [2023-10-10 18:24:35,136][123614] Updated weights for policy 1, policy_version 46840 (0.0007) [2023-10-10 18:24:35,270][123582] Updated weights for policy 0, policy_version 46923 (0.0008) [2023-10-10 18:24:35,634][123582] Updated weights for policy 0, policy_version 46933 (0.0007) [2023-10-10 18:24:36,007][123582] Updated weights for policy 0, policy_version 46943 (0.0008) [2023-10-10 18:24:38,725][123614] Updated weights for policy 1, policy_version 46850 (0.0008) [2023-10-10 18:24:38,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 96043008. Throughput: 0: 1800.1, 1: 1807.9. Samples: 24027020. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:24:38,789][122664] Avg episode reward: [(0, '68.770'), (1, '50.580')] [2023-10-10 18:24:39,090][123614] Updated weights for policy 1, policy_version 46860 (0.0009) [2023-10-10 18:24:39,456][123614] Updated weights for policy 1, policy_version 46870 (0.0008) [2023-10-10 18:24:39,709][123582] Updated weights for policy 0, policy_version 46953 (0.0011) [2023-10-10 18:24:39,818][123614] Updated weights for policy 1, policy_version 46880 (0.0008) [2023-10-10 18:24:40,091][123582] Updated weights for policy 0, policy_version 46963 (0.0008) [2023-10-10 18:24:40,460][123582] Updated weights for policy 0, policy_version 46973 (0.0008) [2023-10-10 18:24:43,523][123614] Updated weights for policy 1, policy_version 46890 (0.0008) [2023-10-10 18:24:43,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 96108544. Throughput: 0: 1799.7, 1: 1798.1. Samples: 24036876. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:24:43,789][122664] Avg episode reward: [(0, '66.230'), (1, '49.790')] [2023-10-10 18:24:43,890][123614] Updated weights for policy 1, policy_version 46900 (0.0011) [2023-10-10 18:24:44,268][123614] Updated weights for policy 1, policy_version 46910 (0.0008) [2023-10-10 18:24:44,302][123582] Updated weights for policy 0, policy_version 46983 (0.0009) [2023-10-10 18:24:44,687][123582] Updated weights for policy 0, policy_version 46993 (0.0010) [2023-10-10 18:24:45,054][123582] Updated weights for policy 0, policy_version 47003 (0.0013) [2023-10-10 18:24:48,017][123614] Updated weights for policy 1, policy_version 46920 (0.0010) [2023-10-10 18:24:48,384][123614] Updated weights for policy 1, policy_version 46930 (0.0009) [2023-10-10 18:24:48,639][123582] Updated weights for policy 0, policy_version 47013 (0.0009) [2023-10-10 18:24:48,758][123614] Updated weights for policy 1, policy_version 46940 (0.0007) [2023-10-10 18:24:48,788][122664] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 96174080. Throughput: 0: 1805.6, 1: 1806.5. Samples: 24059664. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:24:48,788][122664] Avg episode reward: [(0, '61.070'), (1, '47.760')] [2023-10-10 18:24:49,022][123582] Updated weights for policy 0, policy_version 47023 (0.0009) [2023-10-10 18:24:49,392][123582] Updated weights for policy 0, policy_version 47033 (0.0008) [2023-10-10 18:24:52,758][123614] Updated weights for policy 1, policy_version 46950 (0.0007) [2023-10-10 18:24:53,127][123614] Updated weights for policy 1, policy_version 46960 (0.0009) [2023-10-10 18:24:53,148][123582] Updated weights for policy 0, policy_version 47043 (0.0008) [2023-10-10 18:24:53,496][123614] Updated weights for policy 1, policy_version 46970 (0.0009) [2023-10-10 18:24:53,508][123582] Updated weights for policy 0, policy_version 47053 (0.0009) [2023-10-10 18:24:53,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 96272384. Throughput: 0: 1816.1, 1: 1793.0. Samples: 24079912. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:24:53,789][122664] Avg episode reward: [(0, '61.070'), (1, '46.810')] [2023-10-10 18:24:53,876][123582] Updated weights for policy 0, policy_version 47063 (0.0009) [2023-10-10 18:24:57,182][123614] Updated weights for policy 1, policy_version 46980 (0.0009) [2023-10-10 18:24:57,477][123582] Updated weights for policy 0, policy_version 47073 (0.0008) [2023-10-10 18:24:57,555][123614] Updated weights for policy 1, policy_version 46990 (0.0009) [2023-10-10 18:24:57,848][123582] Updated weights for policy 0, policy_version 47083 (0.0008) [2023-10-10 18:24:57,920][123614] Updated weights for policy 1, policy_version 47000 (0.0008) [2023-10-10 18:24:58,224][123582] Updated weights for policy 0, policy_version 47093 (0.0009) [2023-10-10 18:24:58,604][123582] Updated weights for policy 0, policy_version 47103 (0.0009) [2023-10-10 18:24:58,788][122664] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 96370688. Throughput: 0: 1802.6, 1: 1800.2. Samples: 24091860. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:24:58,788][122664] Avg episode reward: [(0, '63.530'), (1, '51.450')] [2023-10-10 18:25:01,666][123614] Updated weights for policy 1, policy_version 47010 (0.0008) [2023-10-10 18:25:02,041][123614] Updated weights for policy 1, policy_version 47020 (0.0009) [2023-10-10 18:25:02,220][123582] Updated weights for policy 0, policy_version 47113 (0.0008) [2023-10-10 18:25:02,409][123614] Updated weights for policy 1, policy_version 47030 (0.0007) [2023-10-10 18:25:02,592][123582] Updated weights for policy 0, policy_version 47123 (0.0007) [2023-10-10 18:25:02,768][123614] Updated weights for policy 1, policy_version 47040 (0.0008) [2023-10-10 18:25:02,973][123582] Updated weights for policy 0, policy_version 47133 (0.0009) [2023-10-10 18:25:03,788][122664] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 96436224. Throughput: 0: 1813.2, 1: 1802.5. Samples: 24112704. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-10 18:25:03,788][122664] Avg episode reward: [(0, '66.260'), (1, '55.910')] [2023-10-10 18:25:06,401][123614] Updated weights for policy 1, policy_version 47050 (0.0008) [2023-10-10 18:25:06,660][123582] Updated weights for policy 0, policy_version 47143 (0.0009) [2023-10-10 18:25:06,768][123614] Updated weights for policy 1, policy_version 47060 (0.0007) [2023-10-10 18:25:07,037][123582] Updated weights for policy 0, policy_version 47153 (0.0009) [2023-10-10 18:25:07,137][123614] Updated weights for policy 1, policy_version 47070 (0.0008) [2023-10-10 18:25:07,410][123582] Updated weights for policy 0, policy_version 47163 (0.0008) [2023-10-10 18:25:08,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 96501760. Throughput: 0: 1802.4, 1: 1797.4. Samples: 24134336. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-10 18:25:08,788][122664] Avg episode reward: [(0, '66.340'), (1, '55.340')] [2023-10-10 18:25:10,915][123614] Updated weights for policy 1, policy_version 47080 (0.0009) [2023-10-10 18:25:11,193][123582] Updated weights for policy 0, policy_version 47173 (0.0008) [2023-10-10 18:25:11,276][123614] Updated weights for policy 1, policy_version 47090 (0.0009) [2023-10-10 18:25:11,566][123582] Updated weights for policy 0, policy_version 47183 (0.0008) [2023-10-10 18:25:11,648][123614] Updated weights for policy 1, policy_version 47100 (0.0007) [2023-10-10 18:25:11,933][123582] Updated weights for policy 0, policy_version 47193 (0.0008) [2023-10-10 18:25:13,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 96567296. Throughput: 0: 1815.7, 1: 1803.3. Samples: 24145408. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-10 18:25:13,789][122664] Avg episode reward: [(0, '63.860'), (1, '57.410')] [2023-10-10 18:25:15,284][123614] Updated weights for policy 1, policy_version 47110 (0.0009) [2023-10-10 18:25:15,641][123582] Updated weights for policy 0, policy_version 47203 (0.0008) [2023-10-10 18:25:15,649][123614] Updated weights for policy 1, policy_version 47120 (0.0009) [2023-10-10 18:25:16,017][123614] Updated weights for policy 1, policy_version 47130 (0.0007) [2023-10-10 18:25:16,021][123582] Updated weights for policy 0, policy_version 47213 (0.0009) [2023-10-10 18:25:16,379][123582] Updated weights for policy 0, policy_version 47223 (0.0009) [2023-10-10 18:25:18,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 96632832. Throughput: 0: 1809.8, 1: 1804.7. Samples: 24167080. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-10 18:25:18,789][122664] Avg episode reward: [(0, '69.930'), (1, '57.000')] [2023-10-10 18:25:19,619][123614] Updated weights for policy 1, policy_version 47140 (0.0008) [2023-10-10 18:25:19,983][123614] Updated weights for policy 1, policy_version 47150 (0.0007) [2023-10-10 18:25:20,090][123582] Updated weights for policy 0, policy_version 47233 (0.0007) [2023-10-10 18:25:20,354][123614] Updated weights for policy 1, policy_version 47160 (0.0008) [2023-10-10 18:25:20,465][123582] Updated weights for policy 0, policy_version 47243 (0.0009) [2023-10-10 18:25:20,818][123582] Updated weights for policy 0, policy_version 47253 (0.0009) [2023-10-10 18:25:21,189][123582] Updated weights for policy 0, policy_version 47263 (0.0007) [2023-10-10 18:25:23,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 96698368. Throughput: 0: 1816.1, 1: 1811.2. Samples: 24190248. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-10 18:25:23,788][122664] Avg episode reward: [(0, '67.310'), (1, '57.970')] [2023-10-10 18:25:24,111][123614] Updated weights for policy 1, policy_version 47170 (0.0008) [2023-10-10 18:25:24,485][123614] Updated weights for policy 1, policy_version 47180 (0.0008) [2023-10-10 18:25:24,839][123582] Updated weights for policy 0, policy_version 47273 (0.0009) [2023-10-10 18:25:24,855][123614] Updated weights for policy 1, policy_version 47190 (0.0009) [2023-10-10 18:25:25,212][123614] Updated weights for policy 1, policy_version 47200 (0.0008) [2023-10-10 18:25:25,220][123582] Updated weights for policy 0, policy_version 47283 (0.0008) [2023-10-10 18:25:25,589][123582] Updated weights for policy 0, policy_version 47293 (0.0008) [2023-10-10 18:25:28,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 96763904. Throughput: 0: 1817.9, 1: 1811.7. Samples: 24200204. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-10 18:25:28,788][122664] Avg episode reward: [(0, '62.810'), (1, '59.970')] [2023-10-10 18:25:28,899][123614] Updated weights for policy 1, policy_version 47210 (0.0008) [2023-10-10 18:25:29,249][123582] Updated weights for policy 0, policy_version 47303 (0.0007) [2023-10-10 18:25:29,262][123614] Updated weights for policy 1, policy_version 47220 (0.0008) [2023-10-10 18:25:29,620][123582] Updated weights for policy 0, policy_version 47313 (0.0007) [2023-10-10 18:25:29,620][123614] Updated weights for policy 1, policy_version 47230 (0.0009) [2023-10-10 18:25:29,991][123582] Updated weights for policy 0, policy_version 47323 (0.0008) [2023-10-10 18:25:33,430][123614] Updated weights for policy 1, policy_version 47240 (0.0007) [2023-10-10 18:25:33,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 96829440. Throughput: 0: 1810.2, 1: 1814.4. Samples: 24222772. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-10 18:25:33,788][122664] Avg episode reward: [(0, '62.760'), (1, '61.010')] [2023-10-10 18:25:33,808][123614] Updated weights for policy 1, policy_version 47250 (0.0007) [2023-10-10 18:25:33,827][123582] Updated weights for policy 0, policy_version 47333 (0.0009) [2023-10-10 18:25:34,166][123614] Updated weights for policy 1, policy_version 47260 (0.0008) [2023-10-10 18:25:34,203][123582] Updated weights for policy 0, policy_version 47343 (0.0007) [2023-10-10 18:25:34,558][123582] Updated weights for policy 0, policy_version 47353 (0.0010) [2023-10-10 18:25:37,849][123614] Updated weights for policy 1, policy_version 47270 (0.0008) [2023-10-10 18:25:38,216][123614] Updated weights for policy 1, policy_version 47280 (0.0009) [2023-10-10 18:25:38,338][123582] Updated weights for policy 0, policy_version 47363 (0.0008) [2023-10-10 18:25:38,582][123614] Updated weights for policy 1, policy_version 47290 (0.0008) [2023-10-10 18:25:38,713][123582] Updated weights for policy 0, policy_version 47373 (0.0008) [2023-10-10 18:25:38,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 96894976. Throughput: 0: 1813.1, 1: 1823.8. Samples: 24243574. Policy #0 lag: (min: 31.0, avg: 44.2, max: 63.0) [2023-10-10 18:25:38,788][122664] Avg episode reward: [(0, '63.960'), (1, '63.450')] [2023-10-10 18:25:39,087][123582] Updated weights for policy 0, policy_version 47383 (0.0009) [2023-10-10 18:25:42,385][123614] Updated weights for policy 1, policy_version 47300 (0.0008) [2023-10-10 18:25:42,755][123614] Updated weights for policy 1, policy_version 47310 (0.0008) [2023-10-10 18:25:42,832][123582] Updated weights for policy 0, policy_version 47393 (0.0010) [2023-10-10 18:25:43,123][123614] Updated weights for policy 1, policy_version 47320 (0.0007) [2023-10-10 18:25:43,205][123582] Updated weights for policy 0, policy_version 47403 (0.0008) [2023-10-10 18:25:43,579][123582] Updated weights for policy 0, policy_version 47413 (0.0007) [2023-10-10 18:25:43,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 96993280. Throughput: 0: 1806.2, 1: 1814.5. Samples: 24254794. Policy #0 lag: (min: 31.0, avg: 44.2, max: 63.0) [2023-10-10 18:25:43,788][122664] Avg episode reward: [(0, '64.940'), (1, '66.050')] [2023-10-10 18:25:43,789][123465] Saving new best policy, reward=66.050! [2023-10-10 18:25:43,953][123582] Updated weights for policy 0, policy_version 47423 (0.0009) [2023-10-10 18:25:46,739][123614] Updated weights for policy 1, policy_version 47330 (0.0007) [2023-10-10 18:25:47,113][123614] Updated weights for policy 1, policy_version 47340 (0.0008) [2023-10-10 18:25:47,476][123614] Updated weights for policy 1, policy_version 47350 (0.0008) [2023-10-10 18:25:47,640][123582] Updated weights for policy 0, policy_version 47433 (0.0008) [2023-10-10 18:25:47,844][123614] Updated weights for policy 1, policy_version 47360 (0.0008) [2023-10-10 18:25:48,004][123582] Updated weights for policy 0, policy_version 47443 (0.0009) [2023-10-10 18:25:48,374][123582] Updated weights for policy 0, policy_version 47453 (0.0008) [2023-10-10 18:25:48,788][122664] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 97091584. Throughput: 0: 1812.0, 1: 1823.5. Samples: 24276300. Policy #0 lag: (min: 31.0, avg: 44.2, max: 63.0) [2023-10-10 18:25:48,788][122664] Avg episode reward: [(0, '67.040'), (1, '69.900')] [2023-10-10 18:25:48,789][123465] Saving new best policy, reward=69.900! [2023-10-10 18:25:51,708][123614] Updated weights for policy 1, policy_version 47370 (0.0008) [2023-10-10 18:25:52,081][123614] Updated weights for policy 1, policy_version 47380 (0.0008) [2023-10-10 18:25:52,169][123582] Updated weights for policy 0, policy_version 47463 (0.0008) [2023-10-10 18:25:52,450][123614] Updated weights for policy 1, policy_version 47390 (0.0007) [2023-10-10 18:25:52,542][123582] Updated weights for policy 0, policy_version 47473 (0.0009) [2023-10-10 18:25:52,910][123582] Updated weights for policy 0, policy_version 47483 (0.0009) [2023-10-10 18:25:53,788][122664] Fps is (10 sec: 16383.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 97157120. Throughput: 0: 1799.6, 1: 1816.5. Samples: 24297062. Policy #0 lag: (min: 31.0, avg: 44.2, max: 63.0) [2023-10-10 18:25:53,789][122664] Avg episode reward: [(0, '69.240'), (1, '70.600')] [2023-10-10 18:25:53,801][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000047392_48529408.pth... [2023-10-10 18:25:53,801][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000047488_48627712.pth... [2023-10-10 18:25:53,832][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000045792_46891008.pth [2023-10-10 18:25:53,843][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000045696_46792704.pth [2023-10-10 18:25:53,848][123465] Saving new best policy, reward=70.600! [2023-10-10 18:25:56,064][123614] Updated weights for policy 1, policy_version 47400 (0.0007) [2023-10-10 18:25:56,436][123614] Updated weights for policy 1, policy_version 47410 (0.0007) [2023-10-10 18:25:56,498][123582] Updated weights for policy 0, policy_version 47493 (0.0009) [2023-10-10 18:25:56,806][123614] Updated weights for policy 1, policy_version 47420 (0.0007) [2023-10-10 18:25:56,874][123582] Updated weights for policy 0, policy_version 47503 (0.0009) [2023-10-10 18:25:57,240][123582] Updated weights for policy 0, policy_version 47513 (0.0011) [2023-10-10 18:25:58,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 97222656. Throughput: 0: 1810.2, 1: 1817.6. Samples: 24308658. Policy #0 lag: (min: 31.0, avg: 44.2, max: 63.0) [2023-10-10 18:25:58,789][122664] Avg episode reward: [(0, '68.690'), (1, '73.310')] [2023-10-10 18:25:58,789][123465] Saving new best policy, reward=73.310! [2023-10-10 18:26:00,394][123614] Updated weights for policy 1, policy_version 47430 (0.0007) [2023-10-10 18:26:00,760][123614] Updated weights for policy 1, policy_version 47440 (0.0009) [2023-10-10 18:26:00,958][123582] Updated weights for policy 0, policy_version 47523 (0.0008) [2023-10-10 18:26:01,117][123614] Updated weights for policy 1, policy_version 47450 (0.0008) [2023-10-10 18:26:01,322][123582] Updated weights for policy 0, policy_version 47533 (0.0008) [2023-10-10 18:26:01,699][123582] Updated weights for policy 0, policy_version 47543 (0.0008) [2023-10-10 18:26:03,788][122664] Fps is (10 sec: 13107.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 97288192. Throughput: 0: 1798.9, 1: 1810.4. Samples: 24329500. Policy #0 lag: (min: 31.0, avg: 44.2, max: 63.0) [2023-10-10 18:26:03,788][122664] Avg episode reward: [(0, '70.880'), (1, '73.130')] [2023-10-10 18:26:04,906][123614] Updated weights for policy 1, policy_version 47460 (0.0009) [2023-10-10 18:26:05,230][123582] Updated weights for policy 0, policy_version 47553 (0.0009) [2023-10-10 18:26:05,271][123614] Updated weights for policy 1, policy_version 47470 (0.0008) [2023-10-10 18:26:05,605][123582] Updated weights for policy 0, policy_version 47563 (0.0008) [2023-10-10 18:26:05,647][123614] Updated weights for policy 1, policy_version 47480 (0.0008) [2023-10-10 18:26:05,972][123582] Updated weights for policy 0, policy_version 47573 (0.0009) [2023-10-10 18:26:06,349][123582] Updated weights for policy 0, policy_version 47583 (0.0007) [2023-10-10 18:26:08,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 97353728. Throughput: 0: 1799.8, 1: 1811.7. Samples: 24352766. Policy #0 lag: (min: 25.0, avg: 35.0, max: 57.0) [2023-10-10 18:26:08,788][122664] Avg episode reward: [(0, '74.840'), (1, '69.690')] [2023-10-10 18:26:08,799][123247] Saving new best policy, reward=74.840! [2023-10-10 18:26:09,307][123614] Updated weights for policy 1, policy_version 47490 (0.0008) [2023-10-10 18:26:09,676][123614] Updated weights for policy 1, policy_version 47500 (0.0008) [2023-10-10 18:26:10,044][123614] Updated weights for policy 1, policy_version 47510 (0.0008) [2023-10-10 18:26:10,102][123582] Updated weights for policy 0, policy_version 47593 (0.0009) [2023-10-10 18:26:10,410][123614] Updated weights for policy 1, policy_version 47520 (0.0009) [2023-10-10 18:26:10,482][123582] Updated weights for policy 0, policy_version 47603 (0.0009) [2023-10-10 18:26:10,856][123582] Updated weights for policy 0, policy_version 47613 (0.0010) [2023-10-10 18:26:13,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 97419264. Throughput: 0: 1801.3, 1: 1809.3. Samples: 24362684. Policy #0 lag: (min: 25.0, avg: 35.0, max: 57.0) [2023-10-10 18:26:13,788][122664] Avg episode reward: [(0, '69.950'), (1, '70.260')] [2023-10-10 18:26:14,202][123614] Updated weights for policy 1, policy_version 47530 (0.0007) [2023-10-10 18:26:14,562][123582] Updated weights for policy 0, policy_version 47623 (0.0008) [2023-10-10 18:26:14,569][123614] Updated weights for policy 1, policy_version 47540 (0.0008) [2023-10-10 18:26:14,937][123582] Updated weights for policy 0, policy_version 47633 (0.0007) [2023-10-10 18:26:14,940][123614] Updated weights for policy 1, policy_version 47550 (0.0007) [2023-10-10 18:26:15,305][123582] Updated weights for policy 0, policy_version 47643 (0.0007) [2023-10-10 18:26:18,574][123614] Updated weights for policy 1, policy_version 47560 (0.0009) [2023-10-10 18:26:18,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 97484800. Throughput: 0: 1804.8, 1: 1810.3. Samples: 24385450. Policy #0 lag: (min: 25.0, avg: 35.0, max: 57.0) [2023-10-10 18:26:18,789][122664] Avg episode reward: [(0, '68.040'), (1, '68.950')] [2023-10-10 18:26:18,950][123614] Updated weights for policy 1, policy_version 47570 (0.0008) [2023-10-10 18:26:19,040][123582] Updated weights for policy 0, policy_version 47653 (0.0008) [2023-10-10 18:26:19,315][123614] Updated weights for policy 1, policy_version 47580 (0.0007) [2023-10-10 18:26:19,418][123582] Updated weights for policy 0, policy_version 47663 (0.0008) [2023-10-10 18:26:19,788][123582] Updated weights for policy 0, policy_version 47673 (0.0008) [2023-10-10 18:26:23,104][123614] Updated weights for policy 1, policy_version 47590 (0.0009) [2023-10-10 18:26:23,441][123582] Updated weights for policy 0, policy_version 47683 (0.0007) [2023-10-10 18:26:23,486][123614] Updated weights for policy 1, policy_version 47600 (0.0008) [2023-10-10 18:26:23,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 97550336. Throughput: 0: 1815.0, 1: 1811.0. Samples: 24406744. Policy #0 lag: (min: 25.0, avg: 35.0, max: 57.0) [2023-10-10 18:26:23,789][122664] Avg episode reward: [(0, '67.830'), (1, '70.170')] [2023-10-10 18:26:23,809][123582] Updated weights for policy 0, policy_version 47693 (0.0008) [2023-10-10 18:26:23,855][123614] Updated weights for policy 1, policy_version 47610 (0.0008) [2023-10-10 18:26:24,184][123582] Updated weights for policy 0, policy_version 47703 (0.0009) [2023-10-10 18:26:27,462][123614] Updated weights for policy 1, policy_version 47620 (0.0008) [2023-10-10 18:26:27,795][123582] Updated weights for policy 0, policy_version 47713 (0.0009) [2023-10-10 18:26:27,829][123614] Updated weights for policy 1, policy_version 47630 (0.0007) [2023-10-10 18:26:28,166][123582] Updated weights for policy 0, policy_version 47723 (0.0007) [2023-10-10 18:26:28,195][123614] Updated weights for policy 1, policy_version 47640 (0.0008) [2023-10-10 18:26:28,536][123582] Updated weights for policy 0, policy_version 47733 (0.0009) [2023-10-10 18:26:28,788][122664] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 97648640. Throughput: 0: 1809.6, 1: 1809.9. Samples: 24417674. Policy #0 lag: (min: 25.0, avg: 35.0, max: 57.0) [2023-10-10 18:26:28,788][122664] Avg episode reward: [(0, '70.320'), (1, '71.950')] [2023-10-10 18:26:28,912][123582] Updated weights for policy 0, policy_version 47743 (0.0008) [2023-10-10 18:26:32,009][123614] Updated weights for policy 1, policy_version 47650 (0.0009) [2023-10-10 18:26:32,371][123614] Updated weights for policy 1, policy_version 47660 (0.0008) [2023-10-10 18:26:32,677][123582] Updated weights for policy 0, policy_version 47753 (0.0007) [2023-10-10 18:26:32,731][123614] Updated weights for policy 1, policy_version 47670 (0.0008) [2023-10-10 18:26:33,045][123582] Updated weights for policy 0, policy_version 47763 (0.0008) [2023-10-10 18:26:33,103][123614] Updated weights for policy 1, policy_version 47680 (0.0008) [2023-10-10 18:26:33,413][123582] Updated weights for policy 0, policy_version 47773 (0.0008) [2023-10-10 18:26:33,788][122664] Fps is (10 sec: 19661.0, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 97746944. Throughput: 0: 1807.4, 1: 1810.3. Samples: 24439096. Policy #0 lag: (min: 25.0, avg: 35.0, max: 57.0) [2023-10-10 18:26:33,789][122664] Avg episode reward: [(0, '68.210'), (1, '72.540')] [2023-10-10 18:26:36,924][123614] Updated weights for policy 1, policy_version 47690 (0.0009) [2023-10-10 18:26:37,169][123582] Updated weights for policy 0, policy_version 47783 (0.0007) [2023-10-10 18:26:37,289][123614] Updated weights for policy 1, policy_version 47700 (0.0008) [2023-10-10 18:26:37,538][123582] Updated weights for policy 0, policy_version 47793 (0.0008) [2023-10-10 18:26:37,652][123614] Updated weights for policy 1, policy_version 47710 (0.0008) [2023-10-10 18:26:37,900][123582] Updated weights for policy 0, policy_version 47803 (0.0009) [2023-10-10 18:26:38,788][122664] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 97812480. Throughput: 0: 1806.9, 1: 1809.7. Samples: 24459806. Policy #0 lag: (min: 24.0, avg: 38.1, max: 56.0) [2023-10-10 18:26:38,789][122664] Avg episode reward: [(0, '70.360'), (1, '77.420')] [2023-10-10 18:26:38,798][123465] Saving new best policy, reward=77.420! [2023-10-10 18:26:41,523][123614] Updated weights for policy 1, policy_version 47720 (0.0009) [2023-10-10 18:26:41,742][123582] Updated weights for policy 0, policy_version 47813 (0.0010) [2023-10-10 18:26:41,899][123614] Updated weights for policy 1, policy_version 47730 (0.0008) [2023-10-10 18:26:42,114][123582] Updated weights for policy 0, policy_version 47823 (0.0008) [2023-10-10 18:26:42,266][123614] Updated weights for policy 1, policy_version 47740 (0.0007) [2023-10-10 18:26:42,489][123582] Updated weights for policy 0, policy_version 47833 (0.0007) [2023-10-10 18:26:43,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 97878016. Throughput: 0: 1805.3, 1: 1823.1. Samples: 24471936. Policy #0 lag: (min: 24.0, avg: 38.1, max: 56.0) [2023-10-10 18:26:43,789][122664] Avg episode reward: [(0, '70.240'), (1, '78.250')] [2023-10-10 18:26:43,791][123465] Saving new best policy, reward=78.250! [2023-10-10 18:26:45,828][123614] Updated weights for policy 1, policy_version 47750 (0.0007) [2023-10-10 18:26:46,193][123614] Updated weights for policy 1, policy_version 47760 (0.0009) [2023-10-10 18:26:46,336][123582] Updated weights for policy 0, policy_version 47843 (0.0009) [2023-10-10 18:26:46,560][123614] Updated weights for policy 1, policy_version 47770 (0.0008) [2023-10-10 18:26:46,696][123582] Updated weights for policy 0, policy_version 47853 (0.0007) [2023-10-10 18:26:47,080][123582] Updated weights for policy 0, policy_version 47863 (0.0007) [2023-10-10 18:26:48,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 97943552. Throughput: 0: 1800.3, 1: 1815.4. Samples: 24492208. Policy #0 lag: (min: 24.0, avg: 38.1, max: 56.0) [2023-10-10 18:26:48,788][122664] Avg episode reward: [(0, '65.590'), (1, '76.110')] [2023-10-10 18:26:50,440][123614] Updated weights for policy 1, policy_version 47780 (0.0008) [2023-10-10 18:26:50,807][123614] Updated weights for policy 1, policy_version 47790 (0.0007) [2023-10-10 18:26:50,817][123582] Updated weights for policy 0, policy_version 47873 (0.0010) [2023-10-10 18:26:51,173][123614] Updated weights for policy 1, policy_version 47800 (0.0008) [2023-10-10 18:26:51,192][123582] Updated weights for policy 0, policy_version 47883 (0.0007) [2023-10-10 18:26:51,560][123582] Updated weights for policy 0, policy_version 47893 (0.0008) [2023-10-10 18:26:51,926][123582] Updated weights for policy 0, policy_version 47903 (0.0011) [2023-10-10 18:26:53,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.6, 300 sec: 14440.1). Total num frames: 98009088. Throughput: 0: 1798.4, 1: 1807.7. Samples: 24515040. Policy #0 lag: (min: 24.0, avg: 38.1, max: 56.0) [2023-10-10 18:26:53,789][122664] Avg episode reward: [(0, '65.490'), (1, '77.490')] [2023-10-10 18:26:54,755][123614] Updated weights for policy 1, policy_version 47810 (0.0009) [2023-10-10 18:26:55,123][123614] Updated weights for policy 1, policy_version 47820 (0.0008) [2023-10-10 18:26:55,488][123614] Updated weights for policy 1, policy_version 47830 (0.0007) [2023-10-10 18:26:55,538][123582] Updated weights for policy 0, policy_version 47913 (0.0008) [2023-10-10 18:26:55,855][123614] Updated weights for policy 1, policy_version 47840 (0.0008) [2023-10-10 18:26:55,905][123582] Updated weights for policy 0, policy_version 47923 (0.0008) [2023-10-10 18:26:56,285][123582] Updated weights for policy 0, policy_version 47933 (0.0009) [2023-10-10 18:26:58,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 98074624. Throughput: 0: 1796.7, 1: 1806.7. Samples: 24524838. Policy #0 lag: (min: 24.0, avg: 38.1, max: 56.0) [2023-10-10 18:26:58,789][122664] Avg episode reward: [(0, '71.280'), (1, '70.880')] [2023-10-10 18:26:59,639][123614] Updated weights for policy 1, policy_version 47850 (0.0009) [2023-10-10 18:26:59,978][123582] Updated weights for policy 0, policy_version 47943 (0.0010) [2023-10-10 18:27:00,007][123614] Updated weights for policy 1, policy_version 47860 (0.0009) [2023-10-10 18:27:00,335][123582] Updated weights for policy 0, policy_version 47953 (0.0007) [2023-10-10 18:27:00,365][123614] Updated weights for policy 1, policy_version 47870 (0.0007) [2023-10-10 18:27:00,701][123582] Updated weights for policy 0, policy_version 47963 (0.0009) [2023-10-10 18:27:03,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 98140160. Throughput: 0: 1798.0, 1: 1805.4. Samples: 24547604. Policy #0 lag: (min: 24.0, avg: 38.1, max: 56.0) [2023-10-10 18:27:03,788][122664] Avg episode reward: [(0, '69.500'), (1, '65.820')] [2023-10-10 18:27:03,918][123614] Updated weights for policy 1, policy_version 47880 (0.0011) [2023-10-10 18:27:04,290][123614] Updated weights for policy 1, policy_version 47890 (0.0008) [2023-10-10 18:27:04,360][123582] Updated weights for policy 0, policy_version 47973 (0.0010) [2023-10-10 18:27:04,653][123614] Updated weights for policy 1, policy_version 47900 (0.0008) [2023-10-10 18:27:04,749][123582] Updated weights for policy 0, policy_version 47983 (0.0009) [2023-10-10 18:27:05,117][123582] Updated weights for policy 0, policy_version 47993 (0.0009) [2023-10-10 18:27:08,308][123614] Updated weights for policy 1, policy_version 47910 (0.0010) [2023-10-10 18:27:08,705][123614] Updated weights for policy 1, policy_version 47920 (0.0010) [2023-10-10 18:27:08,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 98205696. Throughput: 0: 1802.8, 1: 1815.2. Samples: 24569550. Policy #0 lag: (min: 24.0, avg: 38.1, max: 56.0) [2023-10-10 18:27:08,788][122664] Avg episode reward: [(0, '67.670'), (1, '67.240')] [2023-10-10 18:27:08,865][123582] Updated weights for policy 0, policy_version 48003 (0.0010) [2023-10-10 18:27:09,061][123614] Updated weights for policy 1, policy_version 47930 (0.0007) [2023-10-10 18:27:09,237][123582] Updated weights for policy 0, policy_version 48013 (0.0010) [2023-10-10 18:27:09,599][123582] Updated weights for policy 0, policy_version 48023 (0.0008) [2023-10-10 18:27:12,757][123614] Updated weights for policy 1, policy_version 47940 (0.0008) [2023-10-10 18:27:13,119][123614] Updated weights for policy 1, policy_version 47950 (0.0007) [2023-10-10 18:27:13,437][123582] Updated weights for policy 0, policy_version 48033 (0.0009) [2023-10-10 18:27:13,483][123614] Updated weights for policy 1, policy_version 47960 (0.0008) [2023-10-10 18:27:13,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 98304000. Throughput: 0: 1797.1, 1: 1809.1. Samples: 24579952. Policy #0 lag: (min: 11.0, avg: 15.4, max: 43.0) [2023-10-10 18:27:13,788][122664] Avg episode reward: [(0, '65.820'), (1, '66.790')] [2023-10-10 18:27:13,812][123582] Updated weights for policy 0, policy_version 48043 (0.0008) [2023-10-10 18:27:14,185][123582] Updated weights for policy 0, policy_version 48053 (0.0008) [2023-10-10 18:27:14,549][123582] Updated weights for policy 0, policy_version 48063 (0.0008) [2023-10-10 18:27:17,200][123614] Updated weights for policy 1, policy_version 47970 (0.0008) [2023-10-10 18:27:17,562][123614] Updated weights for policy 1, policy_version 47980 (0.0009) [2023-10-10 18:27:17,926][123614] Updated weights for policy 1, policy_version 47990 (0.0007) [2023-10-10 18:27:18,213][123582] Updated weights for policy 0, policy_version 48073 (0.0009) [2023-10-10 18:27:18,288][123614] Updated weights for policy 1, policy_version 48000 (0.0008) [2023-10-10 18:27:18,580][123582] Updated weights for policy 0, policy_version 48083 (0.0008) [2023-10-10 18:27:18,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 98369536. Throughput: 0: 1805.1, 1: 1816.6. Samples: 24602070. Policy #0 lag: (min: 11.0, avg: 15.4, max: 43.0) [2023-10-10 18:27:18,789][122664] Avg episode reward: [(0, '67.870'), (1, '67.940')] [2023-10-10 18:27:18,956][123582] Updated weights for policy 0, policy_version 48093 (0.0007) [2023-10-10 18:27:22,028][123614] Updated weights for policy 1, policy_version 48010 (0.0009) [2023-10-10 18:27:22,386][123614] Updated weights for policy 1, policy_version 48020 (0.0009) [2023-10-10 18:27:22,647][123582] Updated weights for policy 0, policy_version 48103 (0.0008) [2023-10-10 18:27:22,754][123614] Updated weights for policy 1, policy_version 48030 (0.0008) [2023-10-10 18:27:23,018][123582] Updated weights for policy 0, policy_version 48113 (0.0008) [2023-10-10 18:27:23,391][123582] Updated weights for policy 0, policy_version 48123 (0.0008) [2023-10-10 18:27:23,788][122664] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 98467840. Throughput: 0: 1812.6, 1: 1813.8. Samples: 24622996. Policy #0 lag: (min: 11.0, avg: 15.4, max: 43.0) [2023-10-10 18:27:23,789][122664] Avg episode reward: [(0, '69.280'), (1, '66.680')] [2023-10-10 18:27:26,453][123614] Updated weights for policy 1, policy_version 48040 (0.0008) [2023-10-10 18:27:26,818][123614] Updated weights for policy 1, policy_version 48050 (0.0009) [2023-10-10 18:27:27,068][123582] Updated weights for policy 0, policy_version 48133 (0.0008) [2023-10-10 18:27:27,183][123614] Updated weights for policy 1, policy_version 48060 (0.0008) [2023-10-10 18:27:27,432][123582] Updated weights for policy 0, policy_version 48143 (0.0009) [2023-10-10 18:27:27,809][123582] Updated weights for policy 0, policy_version 48153 (0.0008) [2023-10-10 18:27:28,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 98533376. Throughput: 0: 1814.4, 1: 1813.2. Samples: 24635180. Policy #0 lag: (min: 11.0, avg: 15.4, max: 43.0) [2023-10-10 18:27:28,789][122664] Avg episode reward: [(0, '69.710'), (1, '66.320')] [2023-10-10 18:27:31,039][123614] Updated weights for policy 1, policy_version 48070 (0.0009) [2023-10-10 18:27:31,414][123614] Updated weights for policy 1, policy_version 48080 (0.0010) [2023-10-10 18:27:31,498][123582] Updated weights for policy 0, policy_version 48163 (0.0007) [2023-10-10 18:27:31,774][123614] Updated weights for policy 1, policy_version 48090 (0.0008) [2023-10-10 18:27:31,861][123582] Updated weights for policy 0, policy_version 48173 (0.0009) [2023-10-10 18:27:32,239][123582] Updated weights for policy 0, policy_version 48183 (0.0009) [2023-10-10 18:27:33,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 98598912. Throughput: 0: 1822.0, 1: 1805.4. Samples: 24655442. Policy #0 lag: (min: 11.0, avg: 15.4, max: 43.0) [2023-10-10 18:27:33,789][122664] Avg episode reward: [(0, '72.220'), (1, '64.590')] [2023-10-10 18:27:35,492][123614] Updated weights for policy 1, policy_version 48100 (0.0008) [2023-10-10 18:27:35,852][123614] Updated weights for policy 1, policy_version 48110 (0.0009) [2023-10-10 18:27:36,000][123582] Updated weights for policy 0, policy_version 48193 (0.0008) [2023-10-10 18:27:36,225][123614] Updated weights for policy 1, policy_version 48120 (0.0007) [2023-10-10 18:27:36,358][123582] Updated weights for policy 0, policy_version 48203 (0.0007) [2023-10-10 18:27:36,740][123582] Updated weights for policy 0, policy_version 48213 (0.0009) [2023-10-10 18:27:37,100][123582] Updated weights for policy 0, policy_version 48223 (0.0008) [2023-10-10 18:27:38,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 98664448. Throughput: 0: 1811.3, 1: 1807.4. Samples: 24677882. Policy #0 lag: (min: 11.0, avg: 15.4, max: 43.0) [2023-10-10 18:27:38,788][122664] Avg episode reward: [(0, '70.670'), (1, '66.560')] [2023-10-10 18:27:39,827][123614] Updated weights for policy 1, policy_version 48130 (0.0008) [2023-10-10 18:27:40,192][123614] Updated weights for policy 1, policy_version 48140 (0.0010) [2023-10-10 18:27:40,557][123614] Updated weights for policy 1, policy_version 48150 (0.0007) [2023-10-10 18:27:40,913][123582] Updated weights for policy 0, policy_version 48233 (0.0008) [2023-10-10 18:27:40,923][123614] Updated weights for policy 1, policy_version 48160 (0.0008) [2023-10-10 18:27:41,285][123582] Updated weights for policy 0, policy_version 48243 (0.0009) [2023-10-10 18:27:41,649][123582] Updated weights for policy 0, policy_version 48253 (0.0011) [2023-10-10 18:27:43,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 98729984. Throughput: 0: 1821.4, 1: 1809.5. Samples: 24688228. Policy #0 lag: (min: 18.0, avg: 18.6, max: 31.0) [2023-10-10 18:27:43,789][122664] Avg episode reward: [(0, '70.920'), (1, '67.830')] [2023-10-10 18:27:44,595][123614] Updated weights for policy 1, policy_version 48170 (0.0009) [2023-10-10 18:27:44,958][123614] Updated weights for policy 1, policy_version 48180 (0.0007) [2023-10-10 18:27:45,325][123614] Updated weights for policy 1, policy_version 48190 (0.0007) [2023-10-10 18:27:45,381][123582] Updated weights for policy 0, policy_version 48263 (0.0008) [2023-10-10 18:27:45,753][123582] Updated weights for policy 0, policy_version 48273 (0.0009) [2023-10-10 18:27:46,125][123582] Updated weights for policy 0, policy_version 48283 (0.0008) [2023-10-10 18:27:48,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 98795520. Throughput: 0: 1811.8, 1: 1808.6. Samples: 24710522. Policy #0 lag: (min: 18.0, avg: 18.6, max: 31.0) [2023-10-10 18:27:48,788][122664] Avg episode reward: [(0, '69.850'), (1, '62.580')] [2023-10-10 18:27:49,070][123614] Updated weights for policy 1, policy_version 48200 (0.0009) [2023-10-10 18:27:49,448][123614] Updated weights for policy 1, policy_version 48210 (0.0009) [2023-10-10 18:27:49,812][123614] Updated weights for policy 1, policy_version 48220 (0.0009) [2023-10-10 18:27:49,937][123582] Updated weights for policy 0, policy_version 48293 (0.0008) [2023-10-10 18:27:50,314][123582] Updated weights for policy 0, policy_version 48303 (0.0011) [2023-10-10 18:27:50,681][123582] Updated weights for policy 0, policy_version 48313 (0.0009) [2023-10-10 18:27:53,672][123614] Updated weights for policy 1, policy_version 48230 (0.0009) [2023-10-10 18:27:53,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 98861056. Throughput: 0: 1809.1, 1: 1815.9. Samples: 24732674. Policy #0 lag: (min: 18.0, avg: 18.6, max: 31.0) [2023-10-10 18:27:53,789][122664] Avg episode reward: [(0, '73.500'), (1, '64.560')] [2023-10-10 18:27:53,798][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000048320_49479680.pth... [2023-10-10 18:27:53,837][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000046624_47742976.pth [2023-10-10 18:27:54,052][123614] Updated weights for policy 1, policy_version 48240 (0.0008) [2023-10-10 18:27:54,222][123582] Updated weights for policy 0, policy_version 48323 (0.0009) [2023-10-10 18:27:54,413][123614] Updated weights for policy 1, policy_version 48250 (0.0007) [2023-10-10 18:27:54,590][123582] Updated weights for policy 0, policy_version 48333 (0.0008) [2023-10-10 18:27:54,629][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000048256_49414144.pth... [2023-10-10 18:27:54,662][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000046560_47677440.pth [2023-10-10 18:27:54,963][123582] Updated weights for policy 0, policy_version 48343 (0.0009) [2023-10-10 18:27:58,024][123614] Updated weights for policy 1, policy_version 48260 (0.0009) [2023-10-10 18:27:58,384][123614] Updated weights for policy 1, policy_version 48270 (0.0010) [2023-10-10 18:27:58,520][123582] Updated weights for policy 0, policy_version 48353 (0.0008) [2023-10-10 18:27:58,760][123614] Updated weights for policy 1, policy_version 48280 (0.0007) [2023-10-10 18:27:58,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 98926592. Throughput: 0: 1820.6, 1: 1806.4. Samples: 24743168. Policy #0 lag: (min: 18.0, avg: 18.6, max: 31.0) [2023-10-10 18:27:58,789][122664] Avg episode reward: [(0, '75.330'), (1, '64.350')] [2023-10-10 18:27:58,891][123582] Updated weights for policy 0, policy_version 48363 (0.0008) [2023-10-10 18:27:59,264][123582] Updated weights for policy 0, policy_version 48373 (0.0008) [2023-10-10 18:27:59,633][123582] Updated weights for policy 0, policy_version 48383 (0.0008) [2023-10-10 18:27:59,668][123247] Saving new best policy, reward=75.330! [2023-10-10 18:28:02,473][123614] Updated weights for policy 1, policy_version 48290 (0.0009) [2023-10-10 18:28:02,839][123614] Updated weights for policy 1, policy_version 48300 (0.0008) [2023-10-10 18:28:03,211][123614] Updated weights for policy 1, policy_version 48310 (0.0008) [2023-10-10 18:28:03,278][123582] Updated weights for policy 0, policy_version 48393 (0.0007) [2023-10-10 18:28:03,580][123614] Updated weights for policy 1, policy_version 48320 (0.0009) [2023-10-10 18:28:03,658][123582] Updated weights for policy 0, policy_version 48403 (0.0007) [2023-10-10 18:28:03,788][122664] Fps is (10 sec: 16384.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 99024896. Throughput: 0: 1824.4, 1: 1813.6. Samples: 24765778. Policy #0 lag: (min: 18.0, avg: 18.6, max: 31.0) [2023-10-10 18:28:03,788][122664] Avg episode reward: [(0, '76.210'), (1, '64.180')] [2023-10-10 18:28:04,020][123582] Updated weights for policy 0, policy_version 48413 (0.0008) [2023-10-10 18:28:04,130][123247] Saving new best policy, reward=76.210! [2023-10-10 18:28:07,343][123614] Updated weights for policy 1, policy_version 48330 (0.0010) [2023-10-10 18:28:07,606][123582] Updated weights for policy 0, policy_version 48423 (0.0009) [2023-10-10 18:28:07,715][123614] Updated weights for policy 1, policy_version 48340 (0.0009) [2023-10-10 18:28:07,974][123582] Updated weights for policy 0, policy_version 48433 (0.0007) [2023-10-10 18:28:08,081][123614] Updated weights for policy 1, policy_version 48350 (0.0009) [2023-10-10 18:28:08,345][123582] Updated weights for policy 0, policy_version 48443 (0.0007) [2023-10-10 18:28:08,788][122664] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 99123200. Throughput: 0: 1822.4, 1: 1804.2. Samples: 24786190. Policy #0 lag: (min: 18.0, avg: 18.6, max: 31.0) [2023-10-10 18:28:08,789][122664] Avg episode reward: [(0, '76.170'), (1, '65.830')] [2023-10-10 18:28:11,747][123614] Updated weights for policy 1, policy_version 48360 (0.0007) [2023-10-10 18:28:12,052][123582] Updated weights for policy 0, policy_version 48453 (0.0008) [2023-10-10 18:28:12,129][123614] Updated weights for policy 1, policy_version 48370 (0.0007) [2023-10-10 18:28:12,417][123582] Updated weights for policy 0, policy_version 48463 (0.0008) [2023-10-10 18:28:12,487][123614] Updated weights for policy 1, policy_version 48380 (0.0008) [2023-10-10 18:28:12,792][123582] Updated weights for policy 0, policy_version 48473 (0.0008) [2023-10-10 18:28:13,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 99188736. Throughput: 0: 1816.2, 1: 1814.8. Samples: 24798574. Policy #0 lag: (min: 25.0, avg: 26.1, max: 39.0) [2023-10-10 18:28:13,788][122664] Avg episode reward: [(0, '78.200'), (1, '70.800')] [2023-10-10 18:28:13,789][123247] Saving new best policy, reward=78.200! [2023-10-10 18:28:16,256][123614] Updated weights for policy 1, policy_version 48390 (0.0008) [2023-10-10 18:28:16,500][123582] Updated weights for policy 0, policy_version 48483 (0.0010) [2023-10-10 18:28:16,621][123614] Updated weights for policy 1, policy_version 48400 (0.0008) [2023-10-10 18:28:16,870][123582] Updated weights for policy 0, policy_version 48493 (0.0007) [2023-10-10 18:28:16,980][123614] Updated weights for policy 1, policy_version 48410 (0.0008) [2023-10-10 18:28:17,242][123582] Updated weights for policy 0, policy_version 48503 (0.0008) [2023-10-10 18:28:18,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 99254272. Throughput: 0: 1817.9, 1: 1812.5. Samples: 24818808. Policy #0 lag: (min: 25.0, avg: 26.1, max: 39.0) [2023-10-10 18:28:18,789][122664] Avg episode reward: [(0, '75.150'), (1, '69.010')] [2023-10-10 18:28:20,704][123614] Updated weights for policy 1, policy_version 48420 (0.0009) [2023-10-10 18:28:21,037][123582] Updated weights for policy 0, policy_version 48513 (0.0007) [2023-10-10 18:28:21,072][123614] Updated weights for policy 1, policy_version 48430 (0.0011) [2023-10-10 18:28:21,413][123582] Updated weights for policy 0, policy_version 48523 (0.0008) [2023-10-10 18:28:21,443][123614] Updated weights for policy 1, policy_version 48440 (0.0009) [2023-10-10 18:28:21,773][123582] Updated weights for policy 0, policy_version 48533 (0.0009) [2023-10-10 18:28:22,143][123582] Updated weights for policy 0, policy_version 48543 (0.0007) [2023-10-10 18:28:23,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 99319808. Throughput: 0: 1813.5, 1: 1813.0. Samples: 24841074. Policy #0 lag: (min: 25.0, avg: 26.1, max: 39.0) [2023-10-10 18:28:23,789][122664] Avg episode reward: [(0, '77.440'), (1, '68.600')] [2023-10-10 18:28:25,165][123614] Updated weights for policy 1, policy_version 48450 (0.0008) [2023-10-10 18:28:25,540][123614] Updated weights for policy 1, policy_version 48460 (0.0008) [2023-10-10 18:28:25,912][123582] Updated weights for policy 0, policy_version 48553 (0.0008) [2023-10-10 18:28:25,913][123614] Updated weights for policy 1, policy_version 48470 (0.0009) [2023-10-10 18:28:26,281][123582] Updated weights for policy 0, policy_version 48563 (0.0008) [2023-10-10 18:28:26,283][123614] Updated weights for policy 1, policy_version 48480 (0.0008) [2023-10-10 18:28:26,655][123582] Updated weights for policy 0, policy_version 48573 (0.0009) [2023-10-10 18:28:28,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 99385344. Throughput: 0: 1814.7, 1: 1808.4. Samples: 24851268. Policy #0 lag: (min: 25.0, avg: 26.1, max: 39.0) [2023-10-10 18:28:28,789][122664] Avg episode reward: [(0, '77.720'), (1, '67.480')] [2023-10-10 18:28:30,166][123614] Updated weights for policy 1, policy_version 48490 (0.0009) [2023-10-10 18:28:30,316][123582] Updated weights for policy 0, policy_version 48583 (0.0007) [2023-10-10 18:28:30,541][123614] Updated weights for policy 1, policy_version 48500 (0.0008) [2023-10-10 18:28:30,682][123582] Updated weights for policy 0, policy_version 48593 (0.0008) [2023-10-10 18:28:30,900][123614] Updated weights for policy 1, policy_version 48510 (0.0009) [2023-10-10 18:28:31,056][123582] Updated weights for policy 0, policy_version 48603 (0.0008) [2023-10-10 18:28:33,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 99450880. Throughput: 0: 1813.2, 1: 1802.7. Samples: 24873240. Policy #0 lag: (min: 25.0, avg: 26.1, max: 39.0) [2023-10-10 18:28:33,788][122664] Avg episode reward: [(0, '75.660'), (1, '65.730')] [2023-10-10 18:28:34,525][123614] Updated weights for policy 1, policy_version 48520 (0.0007) [2023-10-10 18:28:34,803][123582] Updated weights for policy 0, policy_version 48613 (0.0009) [2023-10-10 18:28:34,888][123614] Updated weights for policy 1, policy_version 48530 (0.0008) [2023-10-10 18:28:35,179][123582] Updated weights for policy 0, policy_version 48623 (0.0008) [2023-10-10 18:28:35,259][123614] Updated weights for policy 1, policy_version 48540 (0.0008) [2023-10-10 18:28:35,546][123582] Updated weights for policy 0, policy_version 48633 (0.0008) [2023-10-10 18:28:38,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 99516416. Throughput: 0: 1813.0, 1: 1809.8. Samples: 24895702. Policy #0 lag: (min: 25.0, avg: 26.1, max: 39.0) [2023-10-10 18:28:38,789][122664] Avg episode reward: [(0, '70.150'), (1, '64.470')] [2023-10-10 18:28:39,170][123582] Updated weights for policy 0, policy_version 48643 (0.0009) [2023-10-10 18:28:39,214][123614] Updated weights for policy 1, policy_version 48550 (0.0009) [2023-10-10 18:28:39,543][123582] Updated weights for policy 0, policy_version 48653 (0.0009) [2023-10-10 18:28:39,606][123614] Updated weights for policy 1, policy_version 48560 (0.0008) [2023-10-10 18:28:39,911][123582] Updated weights for policy 0, policy_version 48663 (0.0010) [2023-10-10 18:28:39,966][123614] Updated weights for policy 1, policy_version 48570 (0.0008) [2023-10-10 18:28:43,494][123614] Updated weights for policy 1, policy_version 48580 (0.0008) [2023-10-10 18:28:43,663][123582] Updated weights for policy 0, policy_version 48673 (0.0009) [2023-10-10 18:28:43,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 99581952. Throughput: 0: 1807.7, 1: 1799.4. Samples: 24905488. Policy #0 lag: (min: 25.0, avg: 26.1, max: 39.0) [2023-10-10 18:28:43,788][122664] Avg episode reward: [(0, '69.430'), (1, '59.370')] [2023-10-10 18:28:43,860][123614] Updated weights for policy 1, policy_version 48590 (0.0010) [2023-10-10 18:28:44,028][123582] Updated weights for policy 0, policy_version 48683 (0.0009) [2023-10-10 18:28:44,227][123614] Updated weights for policy 1, policy_version 48600 (0.0007) [2023-10-10 18:28:44,398][123582] Updated weights for policy 0, policy_version 48693 (0.0008) [2023-10-10 18:28:44,767][123582] Updated weights for policy 0, policy_version 48703 (0.0009) [2023-10-10 18:28:48,046][123614] Updated weights for policy 1, policy_version 48610 (0.0008) [2023-10-10 18:28:48,415][123614] Updated weights for policy 1, policy_version 48620 (0.0007) [2023-10-10 18:28:48,615][123582] Updated weights for policy 0, policy_version 48713 (0.0008) [2023-10-10 18:28:48,785][123614] Updated weights for policy 1, policy_version 48630 (0.0009) [2023-10-10 18:28:48,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 99647488. Throughput: 0: 1800.3, 1: 1806.1. Samples: 24928066. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:28:48,788][122664] Avg episode reward: [(0, '68.730'), (1, '59.760')] [2023-10-10 18:28:48,994][123582] Updated weights for policy 0, policy_version 48723 (0.0008) [2023-10-10 18:28:49,156][123614] Updated weights for policy 1, policy_version 48640 (0.0008) [2023-10-10 18:28:49,363][123582] Updated weights for policy 0, policy_version 48733 (0.0009) [2023-10-10 18:28:52,781][123614] Updated weights for policy 1, policy_version 48650 (0.0008) [2023-10-10 18:28:52,931][123582] Updated weights for policy 0, policy_version 48743 (0.0008) [2023-10-10 18:28:53,156][123614] Updated weights for policy 1, policy_version 48660 (0.0007) [2023-10-10 18:28:53,306][123582] Updated weights for policy 0, policy_version 48753 (0.0007) [2023-10-10 18:28:53,518][123614] Updated weights for policy 1, policy_version 48670 (0.0007) [2023-10-10 18:28:53,674][123582] Updated weights for policy 0, policy_version 48763 (0.0009) [2023-10-10 18:28:53,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 99745792. Throughput: 0: 1808.8, 1: 1798.1. Samples: 24948496. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:28:53,788][122664] Avg episode reward: [(0, '64.410'), (1, '56.220')] [2023-10-10 18:28:57,315][123582] Updated weights for policy 0, policy_version 48773 (0.0008) [2023-10-10 18:28:57,349][123614] Updated weights for policy 1, policy_version 48680 (0.0010) [2023-10-10 18:28:57,684][123582] Updated weights for policy 0, policy_version 48783 (0.0009) [2023-10-10 18:28:57,708][123614] Updated weights for policy 1, policy_version 48690 (0.0008) [2023-10-10 18:28:58,061][123582] Updated weights for policy 0, policy_version 48793 (0.0009) [2023-10-10 18:28:58,083][123614] Updated weights for policy 1, policy_version 48700 (0.0007) [2023-10-10 18:28:58,788][122664] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 99844096. Throughput: 0: 1805.2, 1: 1796.7. Samples: 24960660. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:28:58,789][122664] Avg episode reward: [(0, '62.290'), (1, '56.410')] [2023-10-10 18:29:01,752][123582] Updated weights for policy 0, policy_version 48803 (0.0007) [2023-10-10 18:29:01,830][123614] Updated weights for policy 1, policy_version 48710 (0.0008) [2023-10-10 18:29:02,117][123582] Updated weights for policy 0, policy_version 48813 (0.0008) [2023-10-10 18:29:02,193][123614] Updated weights for policy 1, policy_version 48720 (0.0008) [2023-10-10 18:29:02,483][123582] Updated weights for policy 0, policy_version 48823 (0.0009) [2023-10-10 18:29:02,569][123614] Updated weights for policy 1, policy_version 48730 (0.0008) [2023-10-10 18:29:03,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 99909632. Throughput: 0: 1813.6, 1: 1793.2. Samples: 24981116. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:29:03,789][122664] Avg episode reward: [(0, '60.230'), (1, '52.070')] [2023-10-10 18:29:06,288][123614] Updated weights for policy 1, policy_version 48740 (0.0007) [2023-10-10 18:29:06,328][123582] Updated weights for policy 0, policy_version 48833 (0.0008) [2023-10-10 18:29:06,661][123614] Updated weights for policy 1, policy_version 48750 (0.0007) [2023-10-10 18:29:06,700][123582] Updated weights for policy 0, policy_version 48843 (0.0008) [2023-10-10 18:29:07,037][123614] Updated weights for policy 1, policy_version 48760 (0.0008) [2023-10-10 18:29:07,068][123582] Updated weights for policy 0, policy_version 48853 (0.0008) [2023-10-10 18:29:07,436][123582] Updated weights for policy 0, policy_version 48863 (0.0009) [2023-10-10 18:29:08,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 99975168. Throughput: 0: 1807.0, 1: 1787.3. Samples: 25002818. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:29:08,788][122664] Avg episode reward: [(0, '57.200'), (1, '49.150')] [2023-10-10 18:29:10,774][123614] Updated weights for policy 1, policy_version 48770 (0.0007) [2023-10-10 18:29:11,144][123582] Updated weights for policy 0, policy_version 48873 (0.0008) [2023-10-10 18:29:11,146][123614] Updated weights for policy 1, policy_version 48780 (0.0008) [2023-10-10 18:29:11,511][123614] Updated weights for policy 1, policy_version 48790 (0.0008) [2023-10-10 18:29:11,518][123582] Updated weights for policy 0, policy_version 48883 (0.0008) [2023-10-10 18:29:11,869][123614] Updated weights for policy 1, policy_version 48800 (0.0009) [2023-10-10 18:29:11,885][123582] Updated weights for policy 0, policy_version 48893 (0.0008) [2023-10-10 18:29:13,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 100040704. Throughput: 0: 1811.9, 1: 1797.1. Samples: 25013672. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:29:13,789][122664] Avg episode reward: [(0, '58.970'), (1, '50.710')] [2023-10-10 18:29:15,671][123582] Updated weights for policy 0, policy_version 48903 (0.0008) [2023-10-10 18:29:15,698][123614] Updated weights for policy 1, policy_version 48810 (0.0007) [2023-10-10 18:29:16,038][123582] Updated weights for policy 0, policy_version 48913 (0.0007) [2023-10-10 18:29:16,065][123614] Updated weights for policy 1, policy_version 48820 (0.0008) [2023-10-10 18:29:16,408][123582] Updated weights for policy 0, policy_version 48923 (0.0007) [2023-10-10 18:29:16,435][123614] Updated weights for policy 1, policy_version 48830 (0.0008) [2023-10-10 18:29:18,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 100106240. Throughput: 0: 1804.9, 1: 1792.6. Samples: 25035126. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-10 18:29:18,788][122664] Avg episode reward: [(0, '59.590'), (1, '52.860')] [2023-10-10 18:29:20,089][123614] Updated weights for policy 1, policy_version 48840 (0.0009) [2023-10-10 18:29:20,155][123582] Updated weights for policy 0, policy_version 48933 (0.0010) [2023-10-10 18:29:20,457][123614] Updated weights for policy 1, policy_version 48850 (0.0007) [2023-10-10 18:29:20,517][123582] Updated weights for policy 0, policy_version 48943 (0.0007) [2023-10-10 18:29:20,821][123614] Updated weights for policy 1, policy_version 48860 (0.0008) [2023-10-10 18:29:20,887][123582] Updated weights for policy 0, policy_version 48953 (0.0008) [2023-10-10 18:29:23,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 100171776. Throughput: 0: 1804.4, 1: 1800.5. Samples: 25057922. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-10 18:29:23,789][122664] Avg episode reward: [(0, '58.260'), (1, '51.380')] [2023-10-10 18:29:24,482][123614] Updated weights for policy 1, policy_version 48870 (0.0009) [2023-10-10 18:29:24,605][123582] Updated weights for policy 0, policy_version 48963 (0.0007) [2023-10-10 18:29:24,859][123614] Updated weights for policy 1, policy_version 48880 (0.0007) [2023-10-10 18:29:24,975][123582] Updated weights for policy 0, policy_version 48973 (0.0009) [2023-10-10 18:29:25,229][123614] Updated weights for policy 1, policy_version 48890 (0.0008) [2023-10-10 18:29:25,347][123582] Updated weights for policy 0, policy_version 48983 (0.0008) [2023-10-10 18:29:28,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 100237312. Throughput: 0: 1801.1, 1: 1800.5. Samples: 25067560. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-10 18:29:28,789][122664] Avg episode reward: [(0, '56.850'), (1, '54.760')] [2023-10-10 18:29:29,017][123614] Updated weights for policy 1, policy_version 48900 (0.0007) [2023-10-10 18:29:29,142][123582] Updated weights for policy 0, policy_version 48993 (0.0008) [2023-10-10 18:29:29,381][123614] Updated weights for policy 1, policy_version 48910 (0.0008) [2023-10-10 18:29:29,517][123582] Updated weights for policy 0, policy_version 49003 (0.0009) [2023-10-10 18:29:29,757][123614] Updated weights for policy 1, policy_version 48920 (0.0008) [2023-10-10 18:29:29,890][123582] Updated weights for policy 0, policy_version 49013 (0.0007) [2023-10-10 18:29:30,261][123582] Updated weights for policy 0, policy_version 49023 (0.0007) [2023-10-10 18:29:33,483][123614] Updated weights for policy 1, policy_version 48930 (0.0008) [2023-10-10 18:29:33,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 100302848. Throughput: 0: 1797.3, 1: 1800.5. Samples: 25089968. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-10 18:29:33,788][122664] Avg episode reward: [(0, '59.090'), (1, '55.690')] [2023-10-10 18:29:33,852][123614] Updated weights for policy 1, policy_version 48940 (0.0007) [2023-10-10 18:29:33,969][123582] Updated weights for policy 0, policy_version 49033 (0.0007) [2023-10-10 18:29:34,216][123614] Updated weights for policy 1, policy_version 48950 (0.0008) [2023-10-10 18:29:34,346][123582] Updated weights for policy 0, policy_version 49043 (0.0009) [2023-10-10 18:29:34,583][123614] Updated weights for policy 1, policy_version 48960 (0.0008) [2023-10-10 18:29:34,713][123582] Updated weights for policy 0, policy_version 49053 (0.0008) [2023-10-10 18:29:38,348][123614] Updated weights for policy 1, policy_version 48970 (0.0009) [2023-10-10 18:29:38,619][123582] Updated weights for policy 0, policy_version 49063 (0.0008) [2023-10-10 18:29:38,723][123614] Updated weights for policy 1, policy_version 48980 (0.0008) [2023-10-10 18:29:38,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 100368384. Throughput: 0: 1812.7, 1: 1808.2. Samples: 25111436. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-10 18:29:38,789][122664] Avg episode reward: [(0, '57.420'), (1, '54.530')] [2023-10-10 18:29:38,990][123582] Updated weights for policy 0, policy_version 49073 (0.0008) [2023-10-10 18:29:39,098][123614] Updated weights for policy 1, policy_version 48990 (0.0008) [2023-10-10 18:29:39,348][123582] Updated weights for policy 0, policy_version 49083 (0.0011) [2023-10-10 18:29:42,808][123614] Updated weights for policy 1, policy_version 49000 (0.0008) [2023-10-10 18:29:42,949][123582] Updated weights for policy 0, policy_version 49093 (0.0009) [2023-10-10 18:29:43,180][123614] Updated weights for policy 1, policy_version 49010 (0.0007) [2023-10-10 18:29:43,319][123582] Updated weights for policy 0, policy_version 49103 (0.0008) [2023-10-10 18:29:43,546][123614] Updated weights for policy 1, policy_version 49020 (0.0008) [2023-10-10 18:29:43,691][123582] Updated weights for policy 0, policy_version 49113 (0.0010) [2023-10-10 18:29:43,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 100466688. Throughput: 0: 1792.5, 1: 1796.6. Samples: 25122170. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-10 18:29:43,789][122664] Avg episode reward: [(0, '58.080'), (1, '54.510')] [2023-10-10 18:29:47,310][123582] Updated weights for policy 0, policy_version 49123 (0.0010) [2023-10-10 18:29:47,344][123614] Updated weights for policy 1, policy_version 49030 (0.0009) [2023-10-10 18:29:47,682][123582] Updated weights for policy 0, policy_version 49133 (0.0008) [2023-10-10 18:29:47,711][123614] Updated weights for policy 1, policy_version 49040 (0.0008) [2023-10-10 18:29:48,058][123582] Updated weights for policy 0, policy_version 49143 (0.0008) [2023-10-10 18:29:48,080][123614] Updated weights for policy 1, policy_version 49050 (0.0008) [2023-10-10 18:29:48,788][122664] Fps is (10 sec: 19661.2, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 100564992. Throughput: 0: 1811.6, 1: 1811.5. Samples: 25144154. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-10 18:29:48,788][122664] Avg episode reward: [(0, '58.060'), (1, '55.670')] [2023-10-10 18:29:51,837][123582] Updated weights for policy 0, policy_version 49153 (0.0008) [2023-10-10 18:29:51,913][123614] Updated weights for policy 1, policy_version 49060 (0.0007) [2023-10-10 18:29:52,215][123582] Updated weights for policy 0, policy_version 49163 (0.0009) [2023-10-10 18:29:52,287][123614] Updated weights for policy 1, policy_version 49070 (0.0009) [2023-10-10 18:29:52,580][123582] Updated weights for policy 0, policy_version 49173 (0.0008) [2023-10-10 18:29:52,662][123614] Updated weights for policy 1, policy_version 49080 (0.0007) [2023-10-10 18:29:52,958][123582] Updated weights for policy 0, policy_version 49183 (0.0007) [2023-10-10 18:29:53,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 100630528. Throughput: 0: 1799.4, 1: 1794.5. Samples: 25164542. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-10 18:29:53,789][122664] Avg episode reward: [(0, '60.280'), (1, '58.130')] [2023-10-10 18:29:53,798][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000049088_50266112.pth... [2023-10-10 18:29:53,799][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000049184_50364416.pth... [2023-10-10 18:29:53,835][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000047488_48627712.pth [2023-10-10 18:29:53,837][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000047392_48529408.pth [2023-10-10 18:29:56,362][123614] Updated weights for policy 1, policy_version 49090 (0.0008) [2023-10-10 18:29:56,690][123582] Updated weights for policy 0, policy_version 49193 (0.0008) [2023-10-10 18:29:56,734][123614] Updated weights for policy 1, policy_version 49100 (0.0008) [2023-10-10 18:29:57,056][123582] Updated weights for policy 0, policy_version 49203 (0.0007) [2023-10-10 18:29:57,096][123614] Updated weights for policy 1, policy_version 49110 (0.0008) [2023-10-10 18:29:57,422][123582] Updated weights for policy 0, policy_version 49213 (0.0011) [2023-10-10 18:29:57,470][123614] Updated weights for policy 1, policy_version 49120 (0.0008) [2023-10-10 18:29:58,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 100696064. Throughput: 0: 1813.4, 1: 1807.7. Samples: 25176620. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-10 18:29:58,788][122664] Avg episode reward: [(0, '59.960'), (1, '61.270')] [2023-10-10 18:30:01,074][123582] Updated weights for policy 0, policy_version 49223 (0.0008) [2023-10-10 18:30:01,216][123614] Updated weights for policy 1, policy_version 49130 (0.0009) [2023-10-10 18:30:01,439][123582] Updated weights for policy 0, policy_version 49233 (0.0008) [2023-10-10 18:30:01,589][123614] Updated weights for policy 1, policy_version 49140 (0.0008) [2023-10-10 18:30:01,819][123582] Updated weights for policy 0, policy_version 49243 (0.0008) [2023-10-10 18:30:01,947][123614] Updated weights for policy 1, policy_version 49150 (0.0009) [2023-10-10 18:30:03,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 100761600. Throughput: 0: 1804.8, 1: 1793.5. Samples: 25197052. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-10 18:30:03,788][122664] Avg episode reward: [(0, '58.280'), (1, '66.180')] [2023-10-10 18:30:05,543][123582] Updated weights for policy 0, policy_version 49253 (0.0007) [2023-10-10 18:30:05,632][123614] Updated weights for policy 1, policy_version 49160 (0.0007) [2023-10-10 18:30:05,923][123582] Updated weights for policy 0, policy_version 49263 (0.0009) [2023-10-10 18:30:06,001][123614] Updated weights for policy 1, policy_version 49170 (0.0007) [2023-10-10 18:30:06,298][123582] Updated weights for policy 0, policy_version 49273 (0.0007) [2023-10-10 18:30:06,361][123614] Updated weights for policy 1, policy_version 49180 (0.0009) [2023-10-10 18:30:08,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 100827136. Throughput: 0: 1802.6, 1: 1790.3. Samples: 25219600. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-10 18:30:08,789][122664] Avg episode reward: [(0, '57.140'), (1, '66.240')] [2023-10-10 18:30:10,085][123582] Updated weights for policy 0, policy_version 49283 (0.0007) [2023-10-10 18:30:10,279][123614] Updated weights for policy 1, policy_version 49190 (0.0007) [2023-10-10 18:30:10,455][123582] Updated weights for policy 0, policy_version 49293 (0.0008) [2023-10-10 18:30:10,661][123614] Updated weights for policy 1, policy_version 49200 (0.0007) [2023-10-10 18:30:10,820][123582] Updated weights for policy 0, policy_version 49303 (0.0008) [2023-10-10 18:30:11,030][123614] Updated weights for policy 1, policy_version 49210 (0.0008) [2023-10-10 18:30:13,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 100892672. Throughput: 0: 1803.9, 1: 1786.5. Samples: 25229130. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-10 18:30:13,789][122664] Avg episode reward: [(0, '59.250'), (1, '65.210')] [2023-10-10 18:30:14,552][123582] Updated weights for policy 0, policy_version 49313 (0.0007) [2023-10-10 18:30:14,711][123614] Updated weights for policy 1, policy_version 49220 (0.0008) [2023-10-10 18:30:14,922][123582] Updated weights for policy 0, policy_version 49323 (0.0007) [2023-10-10 18:30:15,074][123614] Updated weights for policy 1, policy_version 49230 (0.0009) [2023-10-10 18:30:15,297][123582] Updated weights for policy 0, policy_version 49333 (0.0009) [2023-10-10 18:30:15,448][123614] Updated weights for policy 1, policy_version 49240 (0.0008) [2023-10-10 18:30:15,665][123582] Updated weights for policy 0, policy_version 49343 (0.0007) [2023-10-10 18:30:18,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 100958208. Throughput: 0: 1804.3, 1: 1785.9. Samples: 25251526. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-10 18:30:18,789][122664] Avg episode reward: [(0, '58.440'), (1, '67.730')] [2023-10-10 18:30:19,138][123614] Updated weights for policy 1, policy_version 49250 (0.0007) [2023-10-10 18:30:19,336][123582] Updated weights for policy 0, policy_version 49353 (0.0007) [2023-10-10 18:30:19,514][123614] Updated weights for policy 1, policy_version 49260 (0.0009) [2023-10-10 18:30:19,700][123582] Updated weights for policy 0, policy_version 49363 (0.0007) [2023-10-10 18:30:19,873][123614] Updated weights for policy 1, policy_version 49270 (0.0008) [2023-10-10 18:30:20,076][123582] Updated weights for policy 0, policy_version 49373 (0.0009) [2023-10-10 18:30:20,252][123614] Updated weights for policy 1, policy_version 49280 (0.0008) [2023-10-10 18:30:23,777][123582] Updated weights for policy 0, policy_version 49383 (0.0008) [2023-10-10 18:30:23,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 101023744. Throughput: 0: 1810.3, 1: 1803.6. Samples: 25274058. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:30:23,789][122664] Avg episode reward: [(0, '58.550'), (1, '71.750')] [2023-10-10 18:30:23,978][123614] Updated weights for policy 1, policy_version 49290 (0.0007) [2023-10-10 18:30:24,155][123582] Updated weights for policy 0, policy_version 49393 (0.0007) [2023-10-10 18:30:24,347][123614] Updated weights for policy 1, policy_version 49300 (0.0008) [2023-10-10 18:30:24,521][123582] Updated weights for policy 0, policy_version 49403 (0.0010) [2023-10-10 18:30:24,713][123614] Updated weights for policy 1, policy_version 49310 (0.0007) [2023-10-10 18:30:28,115][123582] Updated weights for policy 0, policy_version 49413 (0.0007) [2023-10-10 18:30:28,375][123614] Updated weights for policy 1, policy_version 49320 (0.0007) [2023-10-10 18:30:28,489][123582] Updated weights for policy 0, policy_version 49423 (0.0009) [2023-10-10 18:30:28,741][123614] Updated weights for policy 1, policy_version 49330 (0.0010) [2023-10-10 18:30:28,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 101089280. Throughput: 0: 1807.9, 1: 1792.7. Samples: 25284196. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:30:28,789][122664] Avg episode reward: [(0, '55.710'), (1, '70.980')] [2023-10-10 18:30:28,863][123582] Updated weights for policy 0, policy_version 49433 (0.0009) [2023-10-10 18:30:29,119][123614] Updated weights for policy 1, policy_version 49340 (0.0009) [2023-10-10 18:30:32,614][123582] Updated weights for policy 0, policy_version 49443 (0.0008) [2023-10-10 18:30:32,948][123614] Updated weights for policy 1, policy_version 49350 (0.0009) [2023-10-10 18:30:32,974][123582] Updated weights for policy 0, policy_version 49453 (0.0009) [2023-10-10 18:30:33,317][123614] Updated weights for policy 1, policy_version 49360 (0.0008) [2023-10-10 18:30:33,342][123582] Updated weights for policy 0, policy_version 49463 (0.0007) [2023-10-10 18:30:33,689][123614] Updated weights for policy 1, policy_version 49370 (0.0009) [2023-10-10 18:30:33,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 101187584. Throughput: 0: 1808.4, 1: 1801.0. Samples: 25306574. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:30:33,789][122664] Avg episode reward: [(0, '52.090'), (1, '70.450')] [2023-10-10 18:30:37,081][123582] Updated weights for policy 0, policy_version 49473 (0.0007) [2023-10-10 18:30:37,441][123582] Updated weights for policy 0, policy_version 49483 (0.0008) [2023-10-10 18:30:37,494][123614] Updated weights for policy 1, policy_version 49380 (0.0008) [2023-10-10 18:30:37,814][123582] Updated weights for policy 0, policy_version 49493 (0.0007) [2023-10-10 18:30:37,857][123614] Updated weights for policy 1, policy_version 49390 (0.0008) [2023-10-10 18:30:38,189][123582] Updated weights for policy 0, policy_version 49503 (0.0009) [2023-10-10 18:30:38,224][123614] Updated weights for policy 1, policy_version 49400 (0.0009) [2023-10-10 18:30:38,788][122664] Fps is (10 sec: 19660.8, 60 sec: 15291.8, 300 sec: 14551.2). Total num frames: 101285888. Throughput: 0: 1804.8, 1: 1788.6. Samples: 25326244. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:30:38,788][122664] Avg episode reward: [(0, '52.470'), (1, '72.860')] [2023-10-10 18:30:41,897][123614] Updated weights for policy 1, policy_version 49410 (0.0009) [2023-10-10 18:30:41,916][123582] Updated weights for policy 0, policy_version 49513 (0.0008) [2023-10-10 18:30:42,271][123614] Updated weights for policy 1, policy_version 49420 (0.0009) [2023-10-10 18:30:42,283][123582] Updated weights for policy 0, policy_version 49523 (0.0008) [2023-10-10 18:30:42,636][123614] Updated weights for policy 1, policy_version 49430 (0.0007) [2023-10-10 18:30:42,656][123582] Updated weights for policy 0, policy_version 49533 (0.0008) [2023-10-10 18:30:43,000][123614] Updated weights for policy 1, policy_version 49440 (0.0008) [2023-10-10 18:30:43,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 101351424. Throughput: 0: 1811.6, 1: 1802.1. Samples: 25339238. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:30:43,788][122664] Avg episode reward: [(0, '52.590'), (1, '75.090')] [2023-10-10 18:30:46,397][123582] Updated weights for policy 0, policy_version 49543 (0.0009) [2023-10-10 18:30:46,736][123614] Updated weights for policy 1, policy_version 49450 (0.0008) [2023-10-10 18:30:46,768][123582] Updated weights for policy 0, policy_version 49553 (0.0009) [2023-10-10 18:30:47,103][123614] Updated weights for policy 1, policy_version 49460 (0.0007) [2023-10-10 18:30:47,145][123582] Updated weights for policy 0, policy_version 49563 (0.0008) [2023-10-10 18:30:47,474][123614] Updated weights for policy 1, policy_version 49470 (0.0008) [2023-10-10 18:30:48,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.2). Total num frames: 101416960. Throughput: 0: 1802.2, 1: 1791.0. Samples: 25358746. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:30:48,789][122664] Avg episode reward: [(0, '60.050'), (1, '73.190')] [2023-10-10 18:30:50,999][123582] Updated weights for policy 0, policy_version 49573 (0.0009) [2023-10-10 18:30:51,281][123614] Updated weights for policy 1, policy_version 49480 (0.0007) [2023-10-10 18:30:51,376][123582] Updated weights for policy 0, policy_version 49583 (0.0008) [2023-10-10 18:30:51,645][123614] Updated weights for policy 1, policy_version 49490 (0.0010) [2023-10-10 18:30:51,750][123582] Updated weights for policy 0, policy_version 49593 (0.0008) [2023-10-10 18:30:52,017][123614] Updated weights for policy 1, policy_version 49500 (0.0007) [2023-10-10 18:30:53,788][122664] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 101482496. Throughput: 0: 1801.6, 1: 1789.1. Samples: 25381186. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:30:53,789][122664] Avg episode reward: [(0, '55.700'), (1, '76.110')] [2023-10-10 18:30:55,382][123582] Updated weights for policy 0, policy_version 49603 (0.0008) [2023-10-10 18:30:55,748][123582] Updated weights for policy 0, policy_version 49613 (0.0009) [2023-10-10 18:30:55,833][123614] Updated weights for policy 1, policy_version 49510 (0.0008) [2023-10-10 18:30:56,129][123582] Updated weights for policy 0, policy_version 49623 (0.0007) [2023-10-10 18:30:56,222][123614] Updated weights for policy 1, policy_version 49520 (0.0008) [2023-10-10 18:30:56,598][123614] Updated weights for policy 1, policy_version 49530 (0.0010) [2023-10-10 18:30:58,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 101548032. Throughput: 0: 1804.3, 1: 1794.2. Samples: 25391062. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-10 18:30:58,789][122664] Avg episode reward: [(0, '57.620'), (1, '78.200')] [2023-10-10 18:30:59,857][123582] Updated weights for policy 0, policy_version 49633 (0.0007) [2023-10-10 18:31:00,219][123582] Updated weights for policy 0, policy_version 49643 (0.0010) [2023-10-10 18:31:00,319][123614] Updated weights for policy 1, policy_version 49540 (0.0010) [2023-10-10 18:31:00,591][123582] Updated weights for policy 0, policy_version 49653 (0.0008) [2023-10-10 18:31:00,683][123614] Updated weights for policy 1, policy_version 49550 (0.0009) [2023-10-10 18:31:00,965][123582] Updated weights for policy 0, policy_version 49663 (0.0009) [2023-10-10 18:31:01,056][123614] Updated weights for policy 1, policy_version 49560 (0.0008) [2023-10-10 18:31:03,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 101613568. Throughput: 0: 1801.3, 1: 1787.0. Samples: 25413000. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-10 18:31:03,789][122664] Avg episode reward: [(0, '58.090'), (1, '79.430')] [2023-10-10 18:31:03,790][123465] Saving new best policy, reward=79.430! [2023-10-10 18:31:04,815][123614] Updated weights for policy 1, policy_version 49570 (0.0008) [2023-10-10 18:31:04,874][123582] Updated weights for policy 0, policy_version 49673 (0.0009) [2023-10-10 18:31:05,176][123614] Updated weights for policy 1, policy_version 49580 (0.0007) [2023-10-10 18:31:05,249][123582] Updated weights for policy 0, policy_version 49683 (0.0009) [2023-10-10 18:31:05,539][123614] Updated weights for policy 1, policy_version 49590 (0.0008) [2023-10-10 18:31:05,624][123582] Updated weights for policy 0, policy_version 49693 (0.0009) [2023-10-10 18:31:05,905][123614] Updated weights for policy 1, policy_version 49600 (0.0008) [2023-10-10 18:31:08,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 101679104. Throughput: 0: 1794.2, 1: 1785.2. Samples: 25435132. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-10 18:31:08,789][122664] Avg episode reward: [(0, '57.070'), (1, '79.530')] [2023-10-10 18:31:08,796][123465] Saving new best policy, reward=79.530! [2023-10-10 18:31:09,459][123582] Updated weights for policy 0, policy_version 49703 (0.0008) [2023-10-10 18:31:09,824][123582] Updated weights for policy 0, policy_version 49713 (0.0008) [2023-10-10 18:31:09,848][123614] Updated weights for policy 1, policy_version 49610 (0.0007) [2023-10-10 18:31:10,191][123582] Updated weights for policy 0, policy_version 49723 (0.0008) [2023-10-10 18:31:10,213][123614] Updated weights for policy 1, policy_version 49620 (0.0007) [2023-10-10 18:31:10,585][123614] Updated weights for policy 1, policy_version 49630 (0.0007) [2023-10-10 18:31:13,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 101744640. Throughput: 0: 1791.4, 1: 1778.8. Samples: 25444854. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-10 18:31:13,788][122664] Avg episode reward: [(0, '55.230'), (1, '82.020')] [2023-10-10 18:31:13,836][123582] Updated weights for policy 0, policy_version 49733 (0.0010) [2023-10-10 18:31:14,086][123614] Updated weights for policy 1, policy_version 49640 (0.0008) [2023-10-10 18:31:14,212][123582] Updated weights for policy 0, policy_version 49743 (0.0009) [2023-10-10 18:31:14,445][123614] Updated weights for policy 1, policy_version 49650 (0.0010) [2023-10-10 18:31:14,575][123582] Updated weights for policy 0, policy_version 49753 (0.0009) [2023-10-10 18:31:14,817][123614] Updated weights for policy 1, policy_version 49660 (0.0007) [2023-10-10 18:31:14,955][123465] Saving new best policy, reward=82.020! [2023-10-10 18:31:18,390][123582] Updated weights for policy 0, policy_version 49763 (0.0007) [2023-10-10 18:31:18,641][123614] Updated weights for policy 1, policy_version 49670 (0.0007) [2023-10-10 18:31:18,750][123582] Updated weights for policy 0, policy_version 49773 (0.0007) [2023-10-10 18:31:18,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 101810176. Throughput: 0: 1788.0, 1: 1788.8. Samples: 25467532. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-10 18:31:18,788][122664] Avg episode reward: [(0, '55.710'), (1, '81.900')] [2023-10-10 18:31:19,012][123614] Updated weights for policy 1, policy_version 49680 (0.0007) [2023-10-10 18:31:19,125][123582] Updated weights for policy 0, policy_version 49783 (0.0007) [2023-10-10 18:31:19,380][123614] Updated weights for policy 1, policy_version 49690 (0.0008) [2023-10-10 18:31:22,722][123582] Updated weights for policy 0, policy_version 49793 (0.0007) [2023-10-10 18:31:23,089][123582] Updated weights for policy 0, policy_version 49803 (0.0008) [2023-10-10 18:31:23,241][123614] Updated weights for policy 1, policy_version 49700 (0.0009) [2023-10-10 18:31:23,461][123582] Updated weights for policy 0, policy_version 49813 (0.0009) [2023-10-10 18:31:23,615][123614] Updated weights for policy 1, policy_version 49710 (0.0007) [2023-10-10 18:31:23,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 101875712. Throughput: 0: 1803.8, 1: 1797.6. Samples: 25488308. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-10 18:31:23,788][122664] Avg episode reward: [(0, '54.130'), (1, '80.020')] [2023-10-10 18:31:23,831][123582] Updated weights for policy 0, policy_version 49823 (0.0007) [2023-10-10 18:31:23,991][123614] Updated weights for policy 1, policy_version 49720 (0.0007) [2023-10-10 18:31:27,589][123582] Updated weights for policy 0, policy_version 49833 (0.0009) [2023-10-10 18:31:27,652][123614] Updated weights for policy 1, policy_version 49730 (0.0008) [2023-10-10 18:31:27,956][123582] Updated weights for policy 0, policy_version 49843 (0.0009) [2023-10-10 18:31:28,013][123614] Updated weights for policy 1, policy_version 49740 (0.0007) [2023-10-10 18:31:28,337][123582] Updated weights for policy 0, policy_version 49853 (0.0009) [2023-10-10 18:31:28,385][123614] Updated weights for policy 1, policy_version 49750 (0.0010) [2023-10-10 18:31:28,754][123614] Updated weights for policy 1, policy_version 49760 (0.0009) [2023-10-10 18:31:28,788][122664] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 102006784. Throughput: 0: 1789.2, 1: 1779.5. Samples: 25499828. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-10 18:31:28,789][122664] Avg episode reward: [(0, '52.280'), (1, '78.190')] [2023-10-10 18:31:32,224][123582] Updated weights for policy 0, policy_version 49863 (0.0008) [2023-10-10 18:31:32,517][123614] Updated weights for policy 1, policy_version 49770 (0.0008) [2023-10-10 18:31:32,611][123582] Updated weights for policy 0, policy_version 49873 (0.0009) [2023-10-10 18:31:32,895][123614] Updated weights for policy 1, policy_version 49780 (0.0008) [2023-10-10 18:31:32,982][123582] Updated weights for policy 0, policy_version 49883 (0.0009) [2023-10-10 18:31:33,259][123614] Updated weights for policy 1, policy_version 49790 (0.0007) [2023-10-10 18:31:33,788][122664] Fps is (10 sec: 19660.6, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 102072320. Throughput: 0: 1807.3, 1: 1796.1. Samples: 25520900. Policy #0 lag: (min: 24.0, avg: 41.7, max: 56.0) [2023-10-10 18:31:33,789][122664] Avg episode reward: [(0, '53.670'), (1, '75.810')] [2023-10-10 18:31:36,772][123582] Updated weights for policy 0, policy_version 49893 (0.0008) [2023-10-10 18:31:36,883][123614] Updated weights for policy 1, policy_version 49800 (0.0007) [2023-10-10 18:31:37,157][123582] Updated weights for policy 0, policy_version 49903 (0.0011) [2023-10-10 18:31:37,250][123614] Updated weights for policy 1, policy_version 49810 (0.0007) [2023-10-10 18:31:37,527][123582] Updated weights for policy 0, policy_version 49913 (0.0008) [2023-10-10 18:31:37,618][123614] Updated weights for policy 1, policy_version 49820 (0.0007) [2023-10-10 18:31:38,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 102137856. Throughput: 0: 1788.1, 1: 1784.1. Samples: 25541934. Policy #0 lag: (min: 24.0, avg: 41.7, max: 56.0) [2023-10-10 18:31:38,789][122664] Avg episode reward: [(0, '53.280'), (1, '75.540')] [2023-10-10 18:31:41,240][123582] Updated weights for policy 0, policy_version 49923 (0.0009) [2023-10-10 18:31:41,407][123614] Updated weights for policy 1, policy_version 49830 (0.0008) [2023-10-10 18:31:41,617][123582] Updated weights for policy 0, policy_version 49933 (0.0008) [2023-10-10 18:31:41,783][123614] Updated weights for policy 1, policy_version 49840 (0.0008) [2023-10-10 18:31:41,985][123582] Updated weights for policy 0, policy_version 49943 (0.0008) [2023-10-10 18:31:42,161][123614] Updated weights for policy 1, policy_version 49850 (0.0008) [2023-10-10 18:31:43,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 102203392. Throughput: 0: 1808.8, 1: 1801.8. Samples: 25553540. Policy #0 lag: (min: 24.0, avg: 41.7, max: 56.0) [2023-10-10 18:31:43,788][122664] Avg episode reward: [(0, '52.950'), (1, '74.390')] [2023-10-10 18:31:45,729][123582] Updated weights for policy 0, policy_version 49953 (0.0009) [2023-10-10 18:31:45,815][123614] Updated weights for policy 1, policy_version 49860 (0.0008) [2023-10-10 18:31:46,095][123582] Updated weights for policy 0, policy_version 49963 (0.0007) [2023-10-10 18:31:46,187][123614] Updated weights for policy 1, policy_version 49870 (0.0008) [2023-10-10 18:31:46,466][123582] Updated weights for policy 0, policy_version 49973 (0.0008) [2023-10-10 18:31:46,554][123614] Updated weights for policy 1, policy_version 49880 (0.0008) [2023-10-10 18:31:46,845][123582] Updated weights for policy 0, policy_version 49983 (0.0008) [2023-10-10 18:31:48,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 102268928. Throughput: 0: 1790.7, 1: 1788.1. Samples: 25574044. Policy #0 lag: (min: 24.0, avg: 41.7, max: 56.0) [2023-10-10 18:31:48,788][122664] Avg episode reward: [(0, '53.060'), (1, '69.290')] [2023-10-10 18:31:50,391][123614] Updated weights for policy 1, policy_version 49890 (0.0009) [2023-10-10 18:31:50,653][123582] Updated weights for policy 0, policy_version 49993 (0.0007) [2023-10-10 18:31:50,746][123614] Updated weights for policy 1, policy_version 49900 (0.0009) [2023-10-10 18:31:51,030][123582] Updated weights for policy 0, policy_version 50003 (0.0008) [2023-10-10 18:31:51,115][123614] Updated weights for policy 1, policy_version 49910 (0.0007) [2023-10-10 18:31:51,399][123582] Updated weights for policy 0, policy_version 50013 (0.0008) [2023-10-10 18:31:51,482][123614] Updated weights for policy 1, policy_version 49920 (0.0007) [2023-10-10 18:31:53,788][122664] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 102334464. Throughput: 0: 1790.5, 1: 1793.2. Samples: 25596398. Policy #0 lag: (min: 24.0, avg: 41.7, max: 56.0) [2023-10-10 18:31:53,789][122664] Avg episode reward: [(0, '55.400'), (1, '68.170')] [2023-10-10 18:31:53,801][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000050016_51216384.pth... [2023-10-10 18:31:53,801][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000049920_51118080.pth... [2023-10-10 18:31:53,835][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000048320_49479680.pth [2023-10-10 18:31:53,837][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000048256_49414144.pth [2023-10-10 18:31:55,155][123582] Updated weights for policy 0, policy_version 50023 (0.0008) [2023-10-10 18:31:55,332][123614] Updated weights for policy 1, policy_version 49930 (0.0007) [2023-10-10 18:31:55,536][123582] Updated weights for policy 0, policy_version 50033 (0.0007) [2023-10-10 18:31:55,694][123614] Updated weights for policy 1, policy_version 49940 (0.0010) [2023-10-10 18:31:55,904][123582] Updated weights for policy 0, policy_version 50043 (0.0008) [2023-10-10 18:31:56,065][123614] Updated weights for policy 1, policy_version 49950 (0.0008) [2023-10-10 18:31:58,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 102400000. Throughput: 0: 1789.0, 1: 1793.3. Samples: 25606056. Policy #0 lag: (min: 24.0, avg: 41.7, max: 56.0) [2023-10-10 18:31:58,789][122664] Avg episode reward: [(0, '57.300'), (1, '66.510')] [2023-10-10 18:31:59,608][123582] Updated weights for policy 0, policy_version 50053 (0.0007) [2023-10-10 18:31:59,796][123614] Updated weights for policy 1, policy_version 49960 (0.0008) [2023-10-10 18:31:59,985][123582] Updated weights for policy 0, policy_version 50063 (0.0009) [2023-10-10 18:32:00,165][123614] Updated weights for policy 1, policy_version 49970 (0.0009) [2023-10-10 18:32:00,344][123582] Updated weights for policy 0, policy_version 50073 (0.0007) [2023-10-10 18:32:00,531][123614] Updated weights for policy 1, policy_version 49980 (0.0008) [2023-10-10 18:32:03,788][122664] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 102465536. Throughput: 0: 1797.3, 1: 1794.5. Samples: 25629166. Policy #0 lag: (min: 24.0, avg: 41.7, max: 56.0) [2023-10-10 18:32:03,788][122664] Avg episode reward: [(0, '57.280'), (1, '60.820')] [2023-10-10 18:32:04,021][123582] Updated weights for policy 0, policy_version 50083 (0.0009) [2023-10-10 18:32:04,199][123614] Updated weights for policy 1, policy_version 49990 (0.0008) [2023-10-10 18:32:04,400][123582] Updated weights for policy 0, policy_version 50093 (0.0009) [2023-10-10 18:32:04,557][123614] Updated weights for policy 1, policy_version 50000 (0.0009) [2023-10-10 18:32:04,763][123582] Updated weights for policy 0, policy_version 50103 (0.0007) [2023-10-10 18:32:04,918][123614] Updated weights for policy 1, policy_version 50010 (0.0009) [2023-10-10 18:32:08,448][123582] Updated weights for policy 0, policy_version 50113 (0.0008) [2023-10-10 18:32:08,657][123614] Updated weights for policy 1, policy_version 50020 (0.0009) [2023-10-10 18:32:08,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 102531072. Throughput: 0: 1811.3, 1: 1813.4. Samples: 25651422. Policy #0 lag: (min: 24.0, avg: 41.7, max: 56.0) [2023-10-10 18:32:08,788][122664] Avg episode reward: [(0, '56.270'), (1, '59.000')] [2023-10-10 18:32:08,813][123582] Updated weights for policy 0, policy_version 50123 (0.0008) [2023-10-10 18:32:09,031][123614] Updated weights for policy 1, policy_version 50030 (0.0007) [2023-10-10 18:32:09,193][123582] Updated weights for policy 0, policy_version 50133 (0.0008) [2023-10-10 18:32:09,393][123614] Updated weights for policy 1, policy_version 50040 (0.0007) [2023-10-10 18:32:09,555][123582] Updated weights for policy 0, policy_version 50143 (0.0008) [2023-10-10 18:32:13,111][123614] Updated weights for policy 1, policy_version 50050 (0.0009) [2023-10-10 18:32:13,189][123582] Updated weights for policy 0, policy_version 50153 (0.0009) [2023-10-10 18:32:13,488][123614] Updated weights for policy 1, policy_version 50060 (0.0009) [2023-10-10 18:32:13,570][123582] Updated weights for policy 0, policy_version 50163 (0.0008) [2023-10-10 18:32:13,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 102596608. Throughput: 0: 1787.9, 1: 1804.8. Samples: 25661498. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 18:32:13,788][122664] Avg episode reward: [(0, '59.020'), (1, '55.850')] [2023-10-10 18:32:13,850][123614] Updated weights for policy 1, policy_version 50070 (0.0008) [2023-10-10 18:32:13,938][123582] Updated weights for policy 0, policy_version 50173 (0.0008) [2023-10-10 18:32:14,219][123614] Updated weights for policy 1, policy_version 50080 (0.0009) [2023-10-10 18:32:17,597][123582] Updated weights for policy 0, policy_version 50183 (0.0007) [2023-10-10 18:32:17,885][123614] Updated weights for policy 1, policy_version 50090 (0.0007) [2023-10-10 18:32:17,969][123582] Updated weights for policy 0, policy_version 50193 (0.0007) [2023-10-10 18:32:18,255][123614] Updated weights for policy 1, policy_version 50100 (0.0009) [2023-10-10 18:32:18,336][123582] Updated weights for policy 0, policy_version 50203 (0.0007) [2023-10-10 18:32:18,620][123614] Updated weights for policy 1, policy_version 50110 (0.0008) [2023-10-10 18:32:18,788][122664] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 102727680. Throughput: 0: 1802.6, 1: 1817.9. Samples: 25683820. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 18:32:18,789][122664] Avg episode reward: [(0, '57.740'), (1, '54.080')] [2023-10-10 18:32:22,175][123582] Updated weights for policy 0, policy_version 50213 (0.0008) [2023-10-10 18:32:22,308][123614] Updated weights for policy 1, policy_version 50120 (0.0009) [2023-10-10 18:32:22,560][123582] Updated weights for policy 0, policy_version 50223 (0.0008) [2023-10-10 18:32:22,680][123614] Updated weights for policy 1, policy_version 50130 (0.0007) [2023-10-10 18:32:22,937][123582] Updated weights for policy 0, policy_version 50233 (0.0007) [2023-10-10 18:32:23,044][123614] Updated weights for policy 1, policy_version 50140 (0.0008) [2023-10-10 18:32:23,788][122664] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 102793216. Throughput: 0: 1792.9, 1: 1800.0. Samples: 25703618. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 18:32:23,789][122664] Avg episode reward: [(0, '56.580'), (1, '53.210')] [2023-10-10 18:32:26,564][123582] Updated weights for policy 0, policy_version 50243 (0.0008) [2023-10-10 18:32:26,774][123614] Updated weights for policy 1, policy_version 50150 (0.0007) [2023-10-10 18:32:26,931][123582] Updated weights for policy 0, policy_version 50253 (0.0007) [2023-10-10 18:32:27,139][123614] Updated weights for policy 1, policy_version 50160 (0.0008) [2023-10-10 18:32:27,296][123582] Updated weights for policy 0, policy_version 50263 (0.0010) [2023-10-10 18:32:27,509][123614] Updated weights for policy 1, policy_version 50170 (0.0007) [2023-10-10 18:32:28,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 102858752. Throughput: 0: 1809.1, 1: 1811.8. Samples: 25716480. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 18:32:28,789][122664] Avg episode reward: [(0, '60.670'), (1, '53.050')] [2023-10-10 18:32:31,018][123582] Updated weights for policy 0, policy_version 50273 (0.0008) [2023-10-10 18:32:31,373][123614] Updated weights for policy 1, policy_version 50180 (0.0007) [2023-10-10 18:32:31,401][123582] Updated weights for policy 0, policy_version 50283 (0.0009) [2023-10-10 18:32:31,738][123614] Updated weights for policy 1, policy_version 50190 (0.0007) [2023-10-10 18:32:31,780][123582] Updated weights for policy 0, policy_version 50293 (0.0008) [2023-10-10 18:32:32,102][123614] Updated weights for policy 1, policy_version 50200 (0.0007) [2023-10-10 18:32:32,153][123582] Updated weights for policy 0, policy_version 50303 (0.0010) [2023-10-10 18:32:33,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 102924288. Throughput: 0: 1804.6, 1: 1799.0. Samples: 25736206. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 18:32:33,788][122664] Avg episode reward: [(0, '61.940'), (1, '56.550')] [2023-10-10 18:32:35,874][123614] Updated weights for policy 1, policy_version 50210 (0.0008) [2023-10-10 18:32:35,920][123582] Updated weights for policy 0, policy_version 50313 (0.0009) [2023-10-10 18:32:36,232][123614] Updated weights for policy 1, policy_version 50220 (0.0008) [2023-10-10 18:32:36,294][123582] Updated weights for policy 0, policy_version 50323 (0.0008) [2023-10-10 18:32:36,601][123614] Updated weights for policy 1, policy_version 50230 (0.0007) [2023-10-10 18:32:36,667][123582] Updated weights for policy 0, policy_version 50333 (0.0007) [2023-10-10 18:32:36,965][123614] Updated weights for policy 1, policy_version 50240 (0.0008) [2023-10-10 18:32:38,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 102989824. Throughput: 0: 1812.4, 1: 1804.0. Samples: 25759138. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 18:32:38,789][122664] Avg episode reward: [(0, '62.070'), (1, '56.550')] [2023-10-10 18:32:40,340][123582] Updated weights for policy 0, policy_version 50343 (0.0009) [2023-10-10 18:32:40,716][123582] Updated weights for policy 0, policy_version 50353 (0.0009) [2023-10-10 18:32:40,738][123614] Updated weights for policy 1, policy_version 50250 (0.0007) [2023-10-10 18:32:41,087][123582] Updated weights for policy 0, policy_version 50363 (0.0009) [2023-10-10 18:32:41,096][123614] Updated weights for policy 1, policy_version 50260 (0.0008) [2023-10-10 18:32:41,462][123614] Updated weights for policy 1, policy_version 50270 (0.0008) [2023-10-10 18:32:43,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 103055360. Throughput: 0: 1814.9, 1: 1805.1. Samples: 25768954. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 18:32:43,789][122664] Avg episode reward: [(0, '63.730'), (1, '57.740')] [2023-10-10 18:32:44,802][123582] Updated weights for policy 0, policy_version 50373 (0.0008) [2023-10-10 18:32:45,179][123582] Updated weights for policy 0, policy_version 50383 (0.0009) [2023-10-10 18:32:45,203][123614] Updated weights for policy 1, policy_version 50280 (0.0008) [2023-10-10 18:32:45,534][123582] Updated weights for policy 0, policy_version 50393 (0.0009) [2023-10-10 18:32:45,578][123614] Updated weights for policy 1, policy_version 50290 (0.0007) [2023-10-10 18:32:45,943][123614] Updated weights for policy 1, policy_version 50300 (0.0008) [2023-10-10 18:32:48,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 103120896. Throughput: 0: 1810.4, 1: 1802.4. Samples: 25791744. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:32:48,788][122664] Avg episode reward: [(0, '59.570'), (1, '57.790')] [2023-10-10 18:32:49,229][123582] Updated weights for policy 0, policy_version 50403 (0.0008) [2023-10-10 18:32:49,588][123614] Updated weights for policy 1, policy_version 50310 (0.0007) [2023-10-10 18:32:49,609][123582] Updated weights for policy 0, policy_version 50413 (0.0008) [2023-10-10 18:32:49,955][123614] Updated weights for policy 1, policy_version 50320 (0.0008) [2023-10-10 18:32:49,973][123582] Updated weights for policy 0, policy_version 50423 (0.0007) [2023-10-10 18:32:50,327][123614] Updated weights for policy 1, policy_version 50330 (0.0007) [2023-10-10 18:32:53,635][123582] Updated weights for policy 0, policy_version 50433 (0.0007) [2023-10-10 18:32:53,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 103186432. Throughput: 0: 1816.6, 1: 1805.4. Samples: 25814414. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:32:53,789][122664] Avg episode reward: [(0, '60.210'), (1, '56.150')] [2023-10-10 18:32:54,007][123582] Updated weights for policy 0, policy_version 50443 (0.0008) [2023-10-10 18:32:54,216][123614] Updated weights for policy 1, policy_version 50340 (0.0008) [2023-10-10 18:32:54,379][123582] Updated weights for policy 0, policy_version 50453 (0.0007) [2023-10-10 18:32:54,587][123614] Updated weights for policy 1, policy_version 50350 (0.0008) [2023-10-10 18:32:54,753][123582] Updated weights for policy 0, policy_version 50463 (0.0007) [2023-10-10 18:32:54,948][123614] Updated weights for policy 1, policy_version 50360 (0.0008) [2023-10-10 18:32:58,523][123582] Updated weights for policy 0, policy_version 50473 (0.0008) [2023-10-10 18:32:58,617][123614] Updated weights for policy 1, policy_version 50370 (0.0010) [2023-10-10 18:32:58,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 103251968. Throughput: 0: 1817.1, 1: 1803.1. Samples: 25824406. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:32:58,788][122664] Avg episode reward: [(0, '55.680'), (1, '56.720')] [2023-10-10 18:32:58,893][123582] Updated weights for policy 0, policy_version 50483 (0.0008) [2023-10-10 18:32:58,981][123614] Updated weights for policy 1, policy_version 50380 (0.0008) [2023-10-10 18:32:59,265][123582] Updated weights for policy 0, policy_version 50493 (0.0009) [2023-10-10 18:32:59,351][123614] Updated weights for policy 1, policy_version 50390 (0.0007) [2023-10-10 18:32:59,715][123614] Updated weights for policy 1, policy_version 50400 (0.0008) [2023-10-10 18:33:03,067][123582] Updated weights for policy 0, policy_version 50503 (0.0008) [2023-10-10 18:33:03,383][123614] Updated weights for policy 1, policy_version 50410 (0.0008) [2023-10-10 18:33:03,432][123582] Updated weights for policy 0, policy_version 50513 (0.0008) [2023-10-10 18:33:03,752][123614] Updated weights for policy 1, policy_version 50420 (0.0008) [2023-10-10 18:33:03,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 103317504. Throughput: 0: 1815.9, 1: 1811.6. Samples: 25847058. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:33:03,788][122664] Avg episode reward: [(0, '59.420'), (1, '55.650')] [2023-10-10 18:33:03,797][123582] Updated weights for policy 0, policy_version 50523 (0.0008) [2023-10-10 18:33:04,122][123614] Updated weights for policy 1, policy_version 50430 (0.0008) [2023-10-10 18:33:07,445][123582] Updated weights for policy 0, policy_version 50533 (0.0009) [2023-10-10 18:33:07,816][123614] Updated weights for policy 1, policy_version 50440 (0.0007) [2023-10-10 18:33:07,837][123582] Updated weights for policy 0, policy_version 50543 (0.0009) [2023-10-10 18:33:08,183][123614] Updated weights for policy 1, policy_version 50450 (0.0007) [2023-10-10 18:33:08,201][123582] Updated weights for policy 0, policy_version 50553 (0.0009) [2023-10-10 18:33:08,557][123614] Updated weights for policy 1, policy_version 50460 (0.0008) [2023-10-10 18:33:08,788][122664] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 103448576. Throughput: 0: 1816.3, 1: 1811.8. Samples: 25866882. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:33:08,789][122664] Avg episode reward: [(0, '61.490'), (1, '57.630')] [2023-10-10 18:33:12,016][123582] Updated weights for policy 0, policy_version 50563 (0.0009) [2023-10-10 18:33:12,335][123614] Updated weights for policy 1, policy_version 50470 (0.0008) [2023-10-10 18:33:12,376][123582] Updated weights for policy 0, policy_version 50573 (0.0009) [2023-10-10 18:33:12,728][123614] Updated weights for policy 1, policy_version 50480 (0.0009) [2023-10-10 18:33:12,751][123582] Updated weights for policy 0, policy_version 50583 (0.0009) [2023-10-10 18:33:13,088][123614] Updated weights for policy 1, policy_version 50490 (0.0008) [2023-10-10 18:33:13,788][122664] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 103514112. Throughput: 0: 1805.6, 1: 1816.7. Samples: 25879484. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:33:13,789][122664] Avg episode reward: [(0, '60.700'), (1, '55.560')] [2023-10-10 18:33:16,322][123582] Updated weights for policy 0, policy_version 50593 (0.0008) [2023-10-10 18:33:16,691][123582] Updated weights for policy 0, policy_version 50603 (0.0008) [2023-10-10 18:33:16,772][123614] Updated weights for policy 1, policy_version 50500 (0.0007) [2023-10-10 18:33:17,064][123582] Updated weights for policy 0, policy_version 50613 (0.0009) [2023-10-10 18:33:17,137][123614] Updated weights for policy 1, policy_version 50510 (0.0010) [2023-10-10 18:33:17,430][123582] Updated weights for policy 0, policy_version 50623 (0.0009) [2023-10-10 18:33:17,510][123614] Updated weights for policy 1, policy_version 50520 (0.0008) [2023-10-10 18:33:18,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 103579648. Throughput: 0: 1810.0, 1: 1816.6. Samples: 25899404. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:33:18,789][122664] Avg episode reward: [(0, '59.170'), (1, '54.040')] [2023-10-10 18:33:21,121][123582] Updated weights for policy 0, policy_version 50633 (0.0009) [2023-10-10 18:33:21,182][123614] Updated weights for policy 1, policy_version 50530 (0.0007) [2023-10-10 18:33:21,502][123582] Updated weights for policy 0, policy_version 50643 (0.0007) [2023-10-10 18:33:21,552][123614] Updated weights for policy 1, policy_version 50540 (0.0007) [2023-10-10 18:33:21,865][123582] Updated weights for policy 0, policy_version 50653 (0.0007) [2023-10-10 18:33:21,922][123614] Updated weights for policy 1, policy_version 50550 (0.0008) [2023-10-10 18:33:22,299][123614] Updated weights for policy 1, policy_version 50560 (0.0009) [2023-10-10 18:33:23,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 103645184. Throughput: 0: 1800.4, 1: 1814.0. Samples: 25921786. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:33:23,789][122664] Avg episode reward: [(0, '62.320'), (1, '55.100')] [2023-10-10 18:33:25,661][123582] Updated weights for policy 0, policy_version 50663 (0.0010) [2023-10-10 18:33:26,032][123582] Updated weights for policy 0, policy_version 50673 (0.0008) [2023-10-10 18:33:26,084][123614] Updated weights for policy 1, policy_version 50570 (0.0008) [2023-10-10 18:33:26,401][123582] Updated weights for policy 0, policy_version 50683 (0.0007) [2023-10-10 18:33:26,442][123614] Updated weights for policy 1, policy_version 50580 (0.0008) [2023-10-10 18:33:26,821][123614] Updated weights for policy 1, policy_version 50590 (0.0008) [2023-10-10 18:33:28,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 103710720. Throughput: 0: 1806.3, 1: 1820.3. Samples: 25932152. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:33:28,788][122664] Avg episode reward: [(0, '62.290'), (1, '54.440')] [2023-10-10 18:33:29,923][123582] Updated weights for policy 0, policy_version 50693 (0.0008) [2023-10-10 18:33:30,297][123582] Updated weights for policy 0, policy_version 50703 (0.0010) [2023-10-10 18:33:30,442][123614] Updated weights for policy 1, policy_version 50600 (0.0007) [2023-10-10 18:33:30,661][123582] Updated weights for policy 0, policy_version 50713 (0.0009) [2023-10-10 18:33:30,811][123614] Updated weights for policy 1, policy_version 50610 (0.0007) [2023-10-10 18:33:31,184][123614] Updated weights for policy 1, policy_version 50620 (0.0009) [2023-10-10 18:33:33,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 103776256. Throughput: 0: 1807.1, 1: 1814.9. Samples: 25954736. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:33:33,789][122664] Avg episode reward: [(0, '61.070'), (1, '55.050')] [2023-10-10 18:33:34,300][123582] Updated weights for policy 0, policy_version 50723 (0.0009) [2023-10-10 18:33:34,648][123614] Updated weights for policy 1, policy_version 50630 (0.0007) [2023-10-10 18:33:34,667][123582] Updated weights for policy 0, policy_version 50733 (0.0008) [2023-10-10 18:33:35,023][123614] Updated weights for policy 1, policy_version 50640 (0.0007) [2023-10-10 18:33:35,047][123582] Updated weights for policy 0, policy_version 50743 (0.0009) [2023-10-10 18:33:35,391][123614] Updated weights for policy 1, policy_version 50650 (0.0007) [2023-10-10 18:33:38,743][123582] Updated weights for policy 0, policy_version 50753 (0.0008) [2023-10-10 18:33:38,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 103841792. Throughput: 0: 1808.1, 1: 1822.0. Samples: 25977768. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:33:38,788][122664] Avg episode reward: [(0, '61.910'), (1, '53.900')] [2023-10-10 18:33:39,072][123614] Updated weights for policy 1, policy_version 50660 (0.0008) [2023-10-10 18:33:39,113][123582] Updated weights for policy 0, policy_version 50763 (0.0009) [2023-10-10 18:33:39,437][123614] Updated weights for policy 1, policy_version 50670 (0.0010) [2023-10-10 18:33:39,490][123582] Updated weights for policy 0, policy_version 50773 (0.0009) [2023-10-10 18:33:39,802][123614] Updated weights for policy 1, policy_version 50680 (0.0009) [2023-10-10 18:33:39,862][123582] Updated weights for policy 0, policy_version 50783 (0.0009) [2023-10-10 18:33:43,470][123614] Updated weights for policy 1, policy_version 50690 (0.0009) [2023-10-10 18:33:43,545][123582] Updated weights for policy 0, policy_version 50793 (0.0008) [2023-10-10 18:33:43,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 103907328. Throughput: 0: 1809.6, 1: 1815.5. Samples: 25987536. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:33:43,788][122664] Avg episode reward: [(0, '61.410'), (1, '54.020')] [2023-10-10 18:33:43,840][123614] Updated weights for policy 1, policy_version 50700 (0.0009) [2023-10-10 18:33:43,921][123582] Updated weights for policy 0, policy_version 50803 (0.0007) [2023-10-10 18:33:44,204][123614] Updated weights for policy 1, policy_version 50710 (0.0008) [2023-10-10 18:33:44,287][123582] Updated weights for policy 0, policy_version 50813 (0.0008) [2023-10-10 18:33:44,568][123614] Updated weights for policy 1, policy_version 50720 (0.0009) [2023-10-10 18:33:47,909][123582] Updated weights for policy 0, policy_version 50823 (0.0009) [2023-10-10 18:33:48,282][123582] Updated weights for policy 0, policy_version 50833 (0.0010) [2023-10-10 18:33:48,353][123614] Updated weights for policy 1, policy_version 50730 (0.0007) [2023-10-10 18:33:48,660][123582] Updated weights for policy 0, policy_version 50843 (0.0007) [2023-10-10 18:33:48,719][123614] Updated weights for policy 1, policy_version 50740 (0.0008) [2023-10-10 18:33:48,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 103972864. Throughput: 0: 1817.9, 1: 1808.8. Samples: 26010256. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:33:48,788][122664] Avg episode reward: [(0, '64.430'), (1, '54.530')] [2023-10-10 18:33:49,087][123614] Updated weights for policy 1, policy_version 50750 (0.0008) [2023-10-10 18:33:52,558][123582] Updated weights for policy 0, policy_version 50853 (0.0007) [2023-10-10 18:33:52,890][123614] Updated weights for policy 1, policy_version 50760 (0.0007) [2023-10-10 18:33:52,947][123582] Updated weights for policy 0, policy_version 50863 (0.0009) [2023-10-10 18:33:53,247][123614] Updated weights for policy 1, policy_version 50770 (0.0009) [2023-10-10 18:33:53,319][123582] Updated weights for policy 0, policy_version 50873 (0.0009) [2023-10-10 18:33:53,616][123614] Updated weights for policy 1, policy_version 50780 (0.0009) [2023-10-10 18:33:53,788][122664] Fps is (10 sec: 19660.6, 60 sec: 15291.8, 300 sec: 14440.1). Total num frames: 104103936. Throughput: 0: 1813.3, 1: 1809.3. Samples: 26029896. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:33:53,788][122664] Avg episode reward: [(0, '64.320'), (1, '55.820')] [2023-10-10 18:33:53,797][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000050784_52002816.pth... [2023-10-10 18:33:53,797][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000050880_52101120.pth... [2023-10-10 18:33:53,830][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000049184_50364416.pth [2023-10-10 18:33:53,834][123247] Saving a milestone ./train_atari/atari_demonattack_APPO/checkpoint_p0/milestones/checkpoint_000050880_52101120.pth [2023-10-10 18:33:53,835][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000049088_50266112.pth [2023-10-10 18:33:53,839][123465] Saving a milestone ./train_atari/atari_demonattack_APPO/checkpoint_p1/milestones/checkpoint_000050784_52002816.pth [2023-10-10 18:33:57,078][123582] Updated weights for policy 0, policy_version 50883 (0.0008) [2023-10-10 18:33:57,369][123614] Updated weights for policy 1, policy_version 50790 (0.0008) [2023-10-10 18:33:57,447][123582] Updated weights for policy 0, policy_version 50893 (0.0007) [2023-10-10 18:33:57,736][123614] Updated weights for policy 1, policy_version 50800 (0.0008) [2023-10-10 18:33:57,816][123582] Updated weights for policy 0, policy_version 50903 (0.0009) [2023-10-10 18:33:58,095][123614] Updated weights for policy 1, policy_version 50810 (0.0009) [2023-10-10 18:33:58,788][122664] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 104169472. Throughput: 0: 1810.6, 1: 1806.1. Samples: 26042238. Policy #0 lag: (min: 17.0, avg: 28.8, max: 49.0) [2023-10-10 18:33:58,789][122664] Avg episode reward: [(0, '68.030'), (1, '59.550')] [2023-10-10 18:34:01,322][123582] Updated weights for policy 0, policy_version 50913 (0.0007) [2023-10-10 18:34:01,696][123582] Updated weights for policy 0, policy_version 50923 (0.0007) [2023-10-10 18:34:01,869][123614] Updated weights for policy 1, policy_version 50820 (0.0010) [2023-10-10 18:34:02,068][123582] Updated weights for policy 0, policy_version 50933 (0.0008) [2023-10-10 18:34:02,231][123614] Updated weights for policy 1, policy_version 50830 (0.0008) [2023-10-10 18:34:02,438][123582] Updated weights for policy 0, policy_version 50943 (0.0008) [2023-10-10 18:34:02,597][123614] Updated weights for policy 1, policy_version 50840 (0.0007) [2023-10-10 18:34:03,788][122664] Fps is (10 sec: 13107.1, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 104235008. Throughput: 0: 1815.6, 1: 1814.4. Samples: 26062754. Policy #0 lag: (min: 17.0, avg: 28.8, max: 49.0) [2023-10-10 18:34:03,788][122664] Avg episode reward: [(0, '69.220'), (1, '61.530')] [2023-10-10 18:34:06,119][123582] Updated weights for policy 0, policy_version 50953 (0.0010) [2023-10-10 18:34:06,266][123614] Updated weights for policy 1, policy_version 50850 (0.0010) [2023-10-10 18:34:06,491][123582] Updated weights for policy 0, policy_version 50963 (0.0008) [2023-10-10 18:34:06,622][123614] Updated weights for policy 1, policy_version 50860 (0.0008) [2023-10-10 18:34:06,862][123582] Updated weights for policy 0, policy_version 50973 (0.0010) [2023-10-10 18:34:06,986][123614] Updated weights for policy 1, policy_version 50870 (0.0007) [2023-10-10 18:34:07,360][123614] Updated weights for policy 1, policy_version 50880 (0.0008) [2023-10-10 18:34:08,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 104300544. Throughput: 0: 1820.1, 1: 1806.5. Samples: 26084982. Policy #0 lag: (min: 17.0, avg: 28.8, max: 49.0) [2023-10-10 18:34:08,788][122664] Avg episode reward: [(0, '68.810'), (1, '65.520')] [2023-10-10 18:34:10,651][123582] Updated weights for policy 0, policy_version 50983 (0.0009) [2023-10-10 18:34:11,020][123582] Updated weights for policy 0, policy_version 50993 (0.0009) [2023-10-10 18:34:11,168][123614] Updated weights for policy 1, policy_version 50890 (0.0007) [2023-10-10 18:34:11,382][123582] Updated weights for policy 0, policy_version 51003 (0.0007) [2023-10-10 18:34:11,537][123614] Updated weights for policy 1, policy_version 50900 (0.0008) [2023-10-10 18:34:11,898][123614] Updated weights for policy 1, policy_version 50910 (0.0007) [2023-10-10 18:34:13,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 104366080. Throughput: 0: 1820.7, 1: 1808.2. Samples: 26095452. Policy #0 lag: (min: 17.0, avg: 28.8, max: 49.0) [2023-10-10 18:34:13,789][122664] Avg episode reward: [(0, '73.870'), (1, '68.160')] [2023-10-10 18:34:15,087][123582] Updated weights for policy 0, policy_version 51013 (0.0008) [2023-10-10 18:34:15,451][123582] Updated weights for policy 0, policy_version 51023 (0.0008) [2023-10-10 18:34:15,548][123614] Updated weights for policy 1, policy_version 50920 (0.0008) [2023-10-10 18:34:15,828][123582] Updated weights for policy 0, policy_version 51033 (0.0010) [2023-10-10 18:34:15,913][123614] Updated weights for policy 1, policy_version 50930 (0.0007) [2023-10-10 18:34:16,285][123614] Updated weights for policy 1, policy_version 50940 (0.0008) [2023-10-10 18:34:18,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 104431616. Throughput: 0: 1814.4, 1: 1804.4. Samples: 26117582. Policy #0 lag: (min: 17.0, avg: 28.8, max: 49.0) [2023-10-10 18:34:18,789][122664] Avg episode reward: [(0, '72.760'), (1, '67.530')] [2023-10-10 18:34:19,521][123582] Updated weights for policy 0, policy_version 51043 (0.0009) [2023-10-10 18:34:19,884][123582] Updated weights for policy 0, policy_version 51053 (0.0009) [2023-10-10 18:34:19,907][123614] Updated weights for policy 1, policy_version 50950 (0.0008) [2023-10-10 18:34:20,264][123582] Updated weights for policy 0, policy_version 51063 (0.0009) [2023-10-10 18:34:20,291][123614] Updated weights for policy 1, policy_version 50960 (0.0008) [2023-10-10 18:34:20,657][123614] Updated weights for policy 1, policy_version 50970 (0.0008) [2023-10-10 18:34:23,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 104497152. Throughput: 0: 1809.7, 1: 1802.8. Samples: 26140334. Policy #0 lag: (min: 17.0, avg: 28.8, max: 49.0) [2023-10-10 18:34:23,789][122664] Avg episode reward: [(0, '77.240'), (1, '69.860')] [2023-10-10 18:34:23,980][123582] Updated weights for policy 0, policy_version 51073 (0.0008) [2023-10-10 18:34:24,346][123582] Updated weights for policy 0, policy_version 51083 (0.0009) [2023-10-10 18:34:24,361][123614] Updated weights for policy 1, policy_version 50980 (0.0011) [2023-10-10 18:34:24,725][123582] Updated weights for policy 0, policy_version 51093 (0.0009) [2023-10-10 18:34:24,727][123614] Updated weights for policy 1, policy_version 50990 (0.0008) [2023-10-10 18:34:25,083][123614] Updated weights for policy 1, policy_version 51000 (0.0009) [2023-10-10 18:34:25,090][123582] Updated weights for policy 0, policy_version 51103 (0.0008) [2023-10-10 18:34:28,755][123614] Updated weights for policy 1, policy_version 51010 (0.0008) [2023-10-10 18:34:28,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 104562688. Throughput: 0: 1811.0, 1: 1804.7. Samples: 26150246. Policy #0 lag: (min: 17.0, avg: 28.8, max: 49.0) [2023-10-10 18:34:28,789][122664] Avg episode reward: [(0, '75.520'), (1, '69.140')] [2023-10-10 18:34:28,833][123582] Updated weights for policy 0, policy_version 51113 (0.0007) [2023-10-10 18:34:29,127][123614] Updated weights for policy 1, policy_version 51020 (0.0009) [2023-10-10 18:34:29,213][123582] Updated weights for policy 0, policy_version 51123 (0.0008) [2023-10-10 18:34:29,484][123614] Updated weights for policy 1, policy_version 51030 (0.0007) [2023-10-10 18:34:29,585][123582] Updated weights for policy 0, policy_version 51133 (0.0007) [2023-10-10 18:34:29,852][123614] Updated weights for policy 1, policy_version 51040 (0.0009) [2023-10-10 18:34:33,228][123582] Updated weights for policy 0, policy_version 51143 (0.0008) [2023-10-10 18:34:33,580][123614] Updated weights for policy 1, policy_version 51050 (0.0008) [2023-10-10 18:34:33,605][123582] Updated weights for policy 0, policy_version 51153 (0.0007) [2023-10-10 18:34:33,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 104628224. Throughput: 0: 1808.8, 1: 1809.7. Samples: 26173086. Policy #0 lag: (min: 17.0, avg: 28.8, max: 49.0) [2023-10-10 18:34:33,789][122664] Avg episode reward: [(0, '70.100'), (1, '66.910')] [2023-10-10 18:34:33,943][123614] Updated weights for policy 1, policy_version 51060 (0.0007) [2023-10-10 18:34:33,966][123582] Updated weights for policy 0, policy_version 51163 (0.0008) [2023-10-10 18:34:34,315][123614] Updated weights for policy 1, policy_version 51070 (0.0008) [2023-10-10 18:34:37,732][123582] Updated weights for policy 0, policy_version 51173 (0.0010) [2023-10-10 18:34:37,948][123614] Updated weights for policy 1, policy_version 51080 (0.0008) [2023-10-10 18:34:38,112][123582] Updated weights for policy 0, policy_version 51183 (0.0008) [2023-10-10 18:34:38,314][123614] Updated weights for policy 1, policy_version 51090 (0.0009) [2023-10-10 18:34:38,484][123582] Updated weights for policy 0, policy_version 51193 (0.0009) [2023-10-10 18:34:38,679][123614] Updated weights for policy 1, policy_version 51100 (0.0008) [2023-10-10 18:34:38,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 104726528. Throughput: 0: 1818.0, 1: 1818.0. Samples: 26193514. Policy #0 lag: (min: 31.0, avg: 32.0, max: 51.0) [2023-10-10 18:34:38,788][122664] Avg episode reward: [(0, '69.700'), (1, '64.570')] [2023-10-10 18:34:42,077][123582] Updated weights for policy 0, policy_version 51203 (0.0008) [2023-10-10 18:34:42,423][123614] Updated weights for policy 1, policy_version 51110 (0.0007) [2023-10-10 18:34:42,457][123582] Updated weights for policy 0, policy_version 51213 (0.0010) [2023-10-10 18:34:42,802][123614] Updated weights for policy 1, policy_version 51120 (0.0009) [2023-10-10 18:34:42,830][123582] Updated weights for policy 0, policy_version 51223 (0.0008) [2023-10-10 18:34:43,173][123614] Updated weights for policy 1, policy_version 51130 (0.0010) [2023-10-10 18:34:43,788][122664] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 104824832. Throughput: 0: 1814.8, 1: 1813.7. Samples: 26205522. Policy #0 lag: (min: 31.0, avg: 32.0, max: 51.0) [2023-10-10 18:34:43,789][122664] Avg episode reward: [(0, '72.000'), (1, '68.330')] [2023-10-10 18:34:46,478][123582] Updated weights for policy 0, policy_version 51233 (0.0008) [2023-10-10 18:34:46,846][123582] Updated weights for policy 0, policy_version 51243 (0.0010) [2023-10-10 18:34:46,975][123614] Updated weights for policy 1, policy_version 51140 (0.0010) [2023-10-10 18:34:47,217][123582] Updated weights for policy 0, policy_version 51253 (0.0009) [2023-10-10 18:34:47,350][123614] Updated weights for policy 1, policy_version 51150 (0.0009) [2023-10-10 18:34:47,593][123582] Updated weights for policy 0, policy_version 51263 (0.0007) [2023-10-10 18:34:47,718][123614] Updated weights for policy 1, policy_version 51160 (0.0009) [2023-10-10 18:34:48,788][122664] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 104890368. Throughput: 0: 1813.4, 1: 1811.2. Samples: 26225864. Policy #0 lag: (min: 31.0, avg: 32.0, max: 51.0) [2023-10-10 18:34:48,789][122664] Avg episode reward: [(0, '74.990'), (1, '64.560')] [2023-10-10 18:34:51,355][123582] Updated weights for policy 0, policy_version 51273 (0.0009) [2023-10-10 18:34:51,406][123614] Updated weights for policy 1, policy_version 51170 (0.0009) [2023-10-10 18:34:51,722][123582] Updated weights for policy 0, policy_version 51283 (0.0009) [2023-10-10 18:34:51,773][123614] Updated weights for policy 1, policy_version 51180 (0.0008) [2023-10-10 18:34:52,103][123582] Updated weights for policy 0, policy_version 51293 (0.0008) [2023-10-10 18:34:52,140][123614] Updated weights for policy 1, policy_version 51190 (0.0008) [2023-10-10 18:34:52,512][123614] Updated weights for policy 1, policy_version 51200 (0.0010) [2023-10-10 18:34:53,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 104955904. Throughput: 0: 1806.4, 1: 1813.0. Samples: 26247856. Policy #0 lag: (min: 31.0, avg: 32.0, max: 51.0) [2023-10-10 18:34:53,788][122664] Avg episode reward: [(0, '78.840'), (1, '65.790')] [2023-10-10 18:34:53,799][123247] Saving new best policy, reward=78.840! [2023-10-10 18:34:55,920][123582] Updated weights for policy 0, policy_version 51303 (0.0009) [2023-10-10 18:34:56,288][123582] Updated weights for policy 0, policy_version 51313 (0.0009) [2023-10-10 18:34:56,356][123614] Updated weights for policy 1, policy_version 51210 (0.0007) [2023-10-10 18:34:56,650][123582] Updated weights for policy 0, policy_version 51323 (0.0008) [2023-10-10 18:34:56,721][123614] Updated weights for policy 1, policy_version 51220 (0.0007) [2023-10-10 18:34:57,084][123614] Updated weights for policy 1, policy_version 51230 (0.0008) [2023-10-10 18:34:58,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 105021440. Throughput: 0: 1808.8, 1: 1816.3. Samples: 26258584. Policy #0 lag: (min: 31.0, avg: 32.0, max: 51.0) [2023-10-10 18:34:58,789][122664] Avg episode reward: [(0, '85.390'), (1, '65.230')] [2023-10-10 18:34:58,790][123247] Saving new best policy, reward=85.390! [2023-10-10 18:35:00,390][123582] Updated weights for policy 0, policy_version 51333 (0.0009) [2023-10-10 18:35:00,765][123582] Updated weights for policy 0, policy_version 51343 (0.0010) [2023-10-10 18:35:00,934][123614] Updated weights for policy 1, policy_version 51240 (0.0007) [2023-10-10 18:35:01,134][123582] Updated weights for policy 0, policy_version 51353 (0.0008) [2023-10-10 18:35:01,303][123614] Updated weights for policy 1, policy_version 51250 (0.0009) [2023-10-10 18:35:01,678][123614] Updated weights for policy 1, policy_version 51260 (0.0008) [2023-10-10 18:35:03,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 105086976. Throughput: 0: 1800.7, 1: 1805.6. Samples: 26279866. Policy #0 lag: (min: 31.0, avg: 32.0, max: 51.0) [2023-10-10 18:35:03,788][122664] Avg episode reward: [(0, '88.550'), (1, '59.470')] [2023-10-10 18:35:03,789][123247] Saving new best policy, reward=88.550! [2023-10-10 18:35:04,732][123582] Updated weights for policy 0, policy_version 51363 (0.0009) [2023-10-10 18:35:05,114][123582] Updated weights for policy 0, policy_version 51373 (0.0011) [2023-10-10 18:35:05,422][123614] Updated weights for policy 1, policy_version 51270 (0.0008) [2023-10-10 18:35:05,475][123582] Updated weights for policy 0, policy_version 51383 (0.0008) [2023-10-10 18:35:05,793][123614] Updated weights for policy 1, policy_version 51280 (0.0008) [2023-10-10 18:35:06,171][123614] Updated weights for policy 1, policy_version 51290 (0.0009) [2023-10-10 18:35:08,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 105152512. Throughput: 0: 1806.9, 1: 1797.2. Samples: 26302520. Policy #0 lag: (min: 31.0, avg: 32.0, max: 51.0) [2023-10-10 18:35:08,789][122664] Avg episode reward: [(0, '88.320'), (1, '58.550')] [2023-10-10 18:35:09,224][123582] Updated weights for policy 0, policy_version 51393 (0.0008) [2023-10-10 18:35:09,597][123582] Updated weights for policy 0, policy_version 51403 (0.0008) [2023-10-10 18:35:09,972][123582] Updated weights for policy 0, policy_version 51413 (0.0008) [2023-10-10 18:35:10,013][123614] Updated weights for policy 1, policy_version 51300 (0.0009) [2023-10-10 18:35:10,346][123582] Updated weights for policy 0, policy_version 51423 (0.0009) [2023-10-10 18:35:10,387][123614] Updated weights for policy 1, policy_version 51310 (0.0009) [2023-10-10 18:35:10,759][123614] Updated weights for policy 1, policy_version 51320 (0.0011) [2023-10-10 18:35:13,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 105218048. Throughput: 0: 1806.0, 1: 1797.7. Samples: 26312414. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:35:13,789][122664] Avg episode reward: [(0, '88.170'), (1, '55.140')] [2023-10-10 18:35:13,907][123582] Updated weights for policy 0, policy_version 51433 (0.0011) [2023-10-10 18:35:14,277][123582] Updated weights for policy 0, policy_version 51443 (0.0009) [2023-10-10 18:35:14,474][123614] Updated weights for policy 1, policy_version 51330 (0.0010) [2023-10-10 18:35:14,658][123582] Updated weights for policy 0, policy_version 51453 (0.0010) [2023-10-10 18:35:14,836][123614] Updated weights for policy 1, policy_version 51340 (0.0009) [2023-10-10 18:35:15,203][123614] Updated weights for policy 1, policy_version 51350 (0.0009) [2023-10-10 18:35:15,573][123614] Updated weights for policy 1, policy_version 51360 (0.0008) [2023-10-10 18:35:18,417][123582] Updated weights for policy 0, policy_version 51463 (0.0008) [2023-10-10 18:35:18,787][123582] Updated weights for policy 0, policy_version 51473 (0.0010) [2023-10-10 18:35:18,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 105283584. Throughput: 0: 1808.7, 1: 1792.8. Samples: 26335150. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:35:18,789][122664] Avg episode reward: [(0, '92.190'), (1, '56.980')] [2023-10-10 18:35:19,164][123582] Updated weights for policy 0, policy_version 51483 (0.0009) [2023-10-10 18:35:19,307][123614] Updated weights for policy 1, policy_version 51370 (0.0008) [2023-10-10 18:35:19,336][123247] Saving new best policy, reward=92.190! [2023-10-10 18:35:19,675][123614] Updated weights for policy 1, policy_version 51380 (0.0007) [2023-10-10 18:35:20,031][123614] Updated weights for policy 1, policy_version 51390 (0.0008) [2023-10-10 18:35:23,083][123582] Updated weights for policy 0, policy_version 51493 (0.0007) [2023-10-10 18:35:23,465][123582] Updated weights for policy 0, policy_version 51503 (0.0007) [2023-10-10 18:35:23,771][123614] Updated weights for policy 1, policy_version 51400 (0.0009) [2023-10-10 18:35:23,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 105349120. Throughput: 0: 1815.3, 1: 1810.7. Samples: 26356682. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:35:23,789][122664] Avg episode reward: [(0, '97.260'), (1, '56.770')] [2023-10-10 18:35:23,831][123582] Updated weights for policy 0, policy_version 51513 (0.0007) [2023-10-10 18:35:24,092][123247] Saving new best policy, reward=97.260! [2023-10-10 18:35:24,137][123614] Updated weights for policy 1, policy_version 51410 (0.0009) [2023-10-10 18:35:24,513][123614] Updated weights for policy 1, policy_version 51420 (0.0008) [2023-10-10 18:35:27,413][123582] Updated weights for policy 0, policy_version 51523 (0.0007) [2023-10-10 18:35:27,780][123582] Updated weights for policy 0, policy_version 51533 (0.0009) [2023-10-10 18:35:28,154][123582] Updated weights for policy 0, policy_version 51543 (0.0007) [2023-10-10 18:35:28,346][123614] Updated weights for policy 1, policy_version 51430 (0.0007) [2023-10-10 18:35:28,719][123614] Updated weights for policy 1, policy_version 51440 (0.0007) [2023-10-10 18:35:28,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 105447424. Throughput: 0: 1810.5, 1: 1792.2. Samples: 26367642. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:35:28,788][122664] Avg episode reward: [(0, '98.440'), (1, '57.810')] [2023-10-10 18:35:28,789][123247] Saving new best policy, reward=98.440! [2023-10-10 18:35:29,095][123614] Updated weights for policy 1, policy_version 51450 (0.0007) [2023-10-10 18:35:31,951][123582] Updated weights for policy 0, policy_version 51553 (0.0008) [2023-10-10 18:35:32,318][123582] Updated weights for policy 0, policy_version 51563 (0.0008) [2023-10-10 18:35:32,696][123582] Updated weights for policy 0, policy_version 51573 (0.0008) [2023-10-10 18:35:32,760][123614] Updated weights for policy 1, policy_version 51460 (0.0008) [2023-10-10 18:35:33,075][123582] Updated weights for policy 0, policy_version 51583 (0.0007) [2023-10-10 18:35:33,128][123614] Updated weights for policy 1, policy_version 51470 (0.0008) [2023-10-10 18:35:33,498][123614] Updated weights for policy 1, policy_version 51480 (0.0007) [2023-10-10 18:35:33,788][122664] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 105512960. Throughput: 0: 1814.9, 1: 1812.0. Samples: 26389076. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:35:33,788][122664] Avg episode reward: [(0, '102.800'), (1, '55.260')] [2023-10-10 18:35:33,789][123247] Saving new best policy, reward=102.800! [2023-10-10 18:35:36,721][123582] Updated weights for policy 0, policy_version 51593 (0.0009) [2023-10-10 18:35:37,098][123582] Updated weights for policy 0, policy_version 51603 (0.0008) [2023-10-10 18:35:37,263][123614] Updated weights for policy 1, policy_version 51490 (0.0007) [2023-10-10 18:35:37,462][123582] Updated weights for policy 0, policy_version 51613 (0.0008) [2023-10-10 18:35:37,624][123614] Updated weights for policy 1, policy_version 51500 (0.0008) [2023-10-10 18:35:37,996][123614] Updated weights for policy 1, policy_version 51510 (0.0010) [2023-10-10 18:35:38,367][123614] Updated weights for policy 1, policy_version 51520 (0.0009) [2023-10-10 18:35:38,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 105611264. Throughput: 0: 1807.0, 1: 1786.9. Samples: 26409584. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:35:38,788][122664] Avg episode reward: [(0, '99.800'), (1, '54.140')] [2023-10-10 18:35:41,142][123582] Updated weights for policy 0, policy_version 51623 (0.0008) [2023-10-10 18:35:41,507][123582] Updated weights for policy 0, policy_version 51633 (0.0008) [2023-10-10 18:35:41,876][123582] Updated weights for policy 0, policy_version 51643 (0.0007) [2023-10-10 18:35:42,176][123614] Updated weights for policy 1, policy_version 51530 (0.0008) [2023-10-10 18:35:42,543][123614] Updated weights for policy 1, policy_version 51540 (0.0009) [2023-10-10 18:35:42,911][123614] Updated weights for policy 1, policy_version 51550 (0.0009) [2023-10-10 18:35:43,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 105676800. Throughput: 0: 1817.8, 1: 1805.9. Samples: 26421650. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:35:43,789][122664] Avg episode reward: [(0, '105.280'), (1, '52.310')] [2023-10-10 18:35:43,790][123247] Saving new best policy, reward=105.280! [2023-10-10 18:35:45,584][123582] Updated weights for policy 0, policy_version 51653 (0.0007) [2023-10-10 18:35:45,951][123582] Updated weights for policy 0, policy_version 51663 (0.0007) [2023-10-10 18:35:46,319][123582] Updated weights for policy 0, policy_version 51673 (0.0008) [2023-10-10 18:35:46,668][123614] Updated weights for policy 1, policy_version 51560 (0.0010) [2023-10-10 18:35:47,035][123614] Updated weights for policy 1, policy_version 51570 (0.0011) [2023-10-10 18:35:47,402][123614] Updated weights for policy 1, policy_version 51580 (0.0010) [2023-10-10 18:35:48,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 105742336. Throughput: 0: 1810.4, 1: 1791.3. Samples: 26441944. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:35:48,788][122664] Avg episode reward: [(0, '102.560'), (1, '51.630')] [2023-10-10 18:35:49,948][123582] Updated weights for policy 0, policy_version 51683 (0.0008) [2023-10-10 18:35:50,319][123582] Updated weights for policy 0, policy_version 51693 (0.0008) [2023-10-10 18:35:50,694][123582] Updated weights for policy 0, policy_version 51703 (0.0008) [2023-10-10 18:35:51,099][123614] Updated weights for policy 1, policy_version 51590 (0.0008) [2023-10-10 18:35:51,478][123614] Updated weights for policy 1, policy_version 51600 (0.0008) [2023-10-10 18:35:51,842][123614] Updated weights for policy 1, policy_version 51610 (0.0007) [2023-10-10 18:35:53,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 105807872. Throughput: 0: 1809.1, 1: 1796.4. Samples: 26464770. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:35:53,789][122664] Avg episode reward: [(0, '101.590'), (1, '49.530')] [2023-10-10 18:35:53,801][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000051712_52953088.pth... [2023-10-10 18:35:53,802][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000051616_52854784.pth... [2023-10-10 18:35:53,834][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000050016_51216384.pth [2023-10-10 18:35:53,837][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000049920_51118080.pth [2023-10-10 18:35:54,310][123582] Updated weights for policy 0, policy_version 51713 (0.0009) [2023-10-10 18:35:54,690][123582] Updated weights for policy 0, policy_version 51723 (0.0010) [2023-10-10 18:35:55,071][123582] Updated weights for policy 0, policy_version 51733 (0.0007) [2023-10-10 18:35:55,441][123582] Updated weights for policy 0, policy_version 51743 (0.0007) [2023-10-10 18:35:55,670][123614] Updated weights for policy 1, policy_version 51620 (0.0010) [2023-10-10 18:35:56,042][123614] Updated weights for policy 1, policy_version 51630 (0.0007) [2023-10-10 18:35:56,414][123614] Updated weights for policy 1, policy_version 51640 (0.0007) [2023-10-10 18:35:58,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 105873408. Throughput: 0: 1810.5, 1: 1799.1. Samples: 26474846. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:35:58,789][122664] Avg episode reward: [(0, '96.100'), (1, '51.180')] [2023-10-10 18:35:59,170][123582] Updated weights for policy 0, policy_version 51753 (0.0009) [2023-10-10 18:35:59,543][123582] Updated weights for policy 0, policy_version 51763 (0.0009) [2023-10-10 18:35:59,922][123582] Updated weights for policy 0, policy_version 51773 (0.0009) [2023-10-10 18:36:00,029][123614] Updated weights for policy 1, policy_version 51650 (0.0009) [2023-10-10 18:36:00,403][123614] Updated weights for policy 1, policy_version 51660 (0.0010) [2023-10-10 18:36:00,768][123614] Updated weights for policy 1, policy_version 51670 (0.0011) [2023-10-10 18:36:01,133][123614] Updated weights for policy 1, policy_version 51680 (0.0007) [2023-10-10 18:36:03,593][123582] Updated weights for policy 0, policy_version 51783 (0.0009) [2023-10-10 18:36:03,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 105938944. Throughput: 0: 1805.7, 1: 1804.0. Samples: 26497586. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:36:03,789][122664] Avg episode reward: [(0, '100.130'), (1, '52.410')] [2023-10-10 18:36:03,967][123582] Updated weights for policy 0, policy_version 51793 (0.0010) [2023-10-10 18:36:04,340][123582] Updated weights for policy 0, policy_version 51803 (0.0009) [2023-10-10 18:36:04,711][123614] Updated weights for policy 1, policy_version 51690 (0.0011) [2023-10-10 18:36:05,073][123614] Updated weights for policy 1, policy_version 51700 (0.0008) [2023-10-10 18:36:05,449][123614] Updated weights for policy 1, policy_version 51710 (0.0008) [2023-10-10 18:36:08,127][123582] Updated weights for policy 0, policy_version 51813 (0.0010) [2023-10-10 18:36:08,508][123582] Updated weights for policy 0, policy_version 51823 (0.0010) [2023-10-10 18:36:08,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 106004480. Throughput: 0: 1808.4, 1: 1809.9. Samples: 26519508. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:36:08,789][122664] Avg episode reward: [(0, '98.570'), (1, '49.430')] [2023-10-10 18:36:08,874][123582] Updated weights for policy 0, policy_version 51833 (0.0011) [2023-10-10 18:36:09,120][123614] Updated weights for policy 1, policy_version 51720 (0.0007) [2023-10-10 18:36:09,481][123614] Updated weights for policy 1, policy_version 51730 (0.0007) [2023-10-10 18:36:09,848][123614] Updated weights for policy 1, policy_version 51740 (0.0008) [2023-10-10 18:36:12,493][123582] Updated weights for policy 0, policy_version 51843 (0.0010) [2023-10-10 18:36:12,864][123582] Updated weights for policy 0, policy_version 51853 (0.0008) [2023-10-10 18:36:13,231][123582] Updated weights for policy 0, policy_version 51863 (0.0008) [2023-10-10 18:36:13,694][123614] Updated weights for policy 1, policy_version 51750 (0.0009) [2023-10-10 18:36:13,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 106102784. Throughput: 0: 1804.5, 1: 1799.9. Samples: 26529840. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:36:13,789][122664] Avg episode reward: [(0, '95.570'), (1, '52.580')] [2023-10-10 18:36:14,058][123614] Updated weights for policy 1, policy_version 51760 (0.0008) [2023-10-10 18:36:14,426][123614] Updated weights for policy 1, policy_version 51770 (0.0008) [2023-10-10 18:36:16,946][123582] Updated weights for policy 0, policy_version 51873 (0.0009) [2023-10-10 18:36:17,331][123582] Updated weights for policy 0, policy_version 51883 (0.0009) [2023-10-10 18:36:17,692][123582] Updated weights for policy 0, policy_version 51893 (0.0008) [2023-10-10 18:36:18,052][123614] Updated weights for policy 1, policy_version 51780 (0.0010) [2023-10-10 18:36:18,067][123582] Updated weights for policy 0, policy_version 51903 (0.0009) [2023-10-10 18:36:18,419][123614] Updated weights for policy 1, policy_version 51790 (0.0009) [2023-10-10 18:36:18,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 106168320. Throughput: 0: 1812.6, 1: 1814.5. Samples: 26552296. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:36:18,788][122664] Avg episode reward: [(0, '90.110'), (1, '50.900')] [2023-10-10 18:36:18,795][123614] Updated weights for policy 1, policy_version 51800 (0.0010) [2023-10-10 18:36:21,849][123582] Updated weights for policy 0, policy_version 51913 (0.0010) [2023-10-10 18:36:22,229][123582] Updated weights for policy 0, policy_version 51923 (0.0011) [2023-10-10 18:36:22,439][123614] Updated weights for policy 1, policy_version 51810 (0.0008) [2023-10-10 18:36:22,596][123582] Updated weights for policy 0, policy_version 51933 (0.0008) [2023-10-10 18:36:22,816][123614] Updated weights for policy 1, policy_version 51820 (0.0008) [2023-10-10 18:36:23,173][123614] Updated weights for policy 1, policy_version 51830 (0.0007) [2023-10-10 18:36:23,546][123614] Updated weights for policy 1, policy_version 51840 (0.0011) [2023-10-10 18:36:23,788][122664] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14440.1). Total num frames: 106266624. Throughput: 0: 1812.6, 1: 1811.2. Samples: 26572654. Policy #0 lag: (min: 22.0, avg: 24.7, max: 54.0) [2023-10-10 18:36:23,789][122664] Avg episode reward: [(0, '82.750'), (1, '52.800')] [2023-10-10 18:36:26,299][123582] Updated weights for policy 0, policy_version 51943 (0.0008) [2023-10-10 18:36:26,673][123582] Updated weights for policy 0, policy_version 51953 (0.0007) [2023-10-10 18:36:27,039][123582] Updated weights for policy 0, policy_version 51963 (0.0007) [2023-10-10 18:36:27,204][123614] Updated weights for policy 1, policy_version 51850 (0.0008) [2023-10-10 18:36:27,571][123614] Updated weights for policy 1, policy_version 51860 (0.0008) [2023-10-10 18:36:27,943][123614] Updated weights for policy 1, policy_version 51870 (0.0008) [2023-10-10 18:36:28,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 106332160. Throughput: 0: 1811.2, 1: 1814.9. Samples: 26584826. Policy #0 lag: (min: 22.0, avg: 24.7, max: 54.0) [2023-10-10 18:36:28,789][122664] Avg episode reward: [(0, '86.670'), (1, '53.460')] [2023-10-10 18:36:30,741][123582] Updated weights for policy 0, policy_version 51973 (0.0009) [2023-10-10 18:36:31,111][123582] Updated weights for policy 0, policy_version 51983 (0.0010) [2023-10-10 18:36:31,483][123582] Updated weights for policy 0, policy_version 51993 (0.0009) [2023-10-10 18:36:31,593][123614] Updated weights for policy 1, policy_version 51880 (0.0008) [2023-10-10 18:36:31,949][123614] Updated weights for policy 1, policy_version 51890 (0.0007) [2023-10-10 18:36:32,319][123614] Updated weights for policy 1, policy_version 51900 (0.0007) [2023-10-10 18:36:33,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 106397696. Throughput: 0: 1814.2, 1: 1816.6. Samples: 26605330. Policy #0 lag: (min: 22.0, avg: 24.7, max: 54.0) [2023-10-10 18:36:33,788][122664] Avg episode reward: [(0, '88.000'), (1, '57.500')] [2023-10-10 18:36:35,063][123582] Updated weights for policy 0, policy_version 52003 (0.0008) [2023-10-10 18:36:35,431][123582] Updated weights for policy 0, policy_version 52013 (0.0008) [2023-10-10 18:36:35,800][123582] Updated weights for policy 0, policy_version 52023 (0.0009) [2023-10-10 18:36:36,063][123614] Updated weights for policy 1, policy_version 51910 (0.0008) [2023-10-10 18:36:36,434][123614] Updated weights for policy 1, policy_version 51920 (0.0008) [2023-10-10 18:36:36,807][123614] Updated weights for policy 1, policy_version 51930 (0.0008) [2023-10-10 18:36:38,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 106463232. Throughput: 0: 1814.5, 1: 1819.9. Samples: 26628316. Policy #0 lag: (min: 22.0, avg: 24.7, max: 54.0) [2023-10-10 18:36:38,789][122664] Avg episode reward: [(0, '75.520'), (1, '56.270')] [2023-10-10 18:36:39,603][123582] Updated weights for policy 0, policy_version 52033 (0.0008) [2023-10-10 18:36:39,966][123582] Updated weights for policy 0, policy_version 52043 (0.0010) [2023-10-10 18:36:40,339][123582] Updated weights for policy 0, policy_version 52053 (0.0009) [2023-10-10 18:36:40,393][123614] Updated weights for policy 1, policy_version 51940 (0.0007) [2023-10-10 18:36:40,713][123582] Updated weights for policy 0, policy_version 52063 (0.0008) [2023-10-10 18:36:40,774][123614] Updated weights for policy 1, policy_version 51950 (0.0008) [2023-10-10 18:36:41,148][123614] Updated weights for policy 1, policy_version 51960 (0.0009) [2023-10-10 18:36:43,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 106528768. Throughput: 0: 1813.3, 1: 1816.4. Samples: 26638184. Policy #0 lag: (min: 22.0, avg: 24.7, max: 54.0) [2023-10-10 18:36:43,789][122664] Avg episode reward: [(0, '70.060'), (1, '57.980')] [2023-10-10 18:36:44,305][123582] Updated weights for policy 0, policy_version 52073 (0.0008) [2023-10-10 18:36:44,674][123582] Updated weights for policy 0, policy_version 52083 (0.0007) [2023-10-10 18:36:44,973][123614] Updated weights for policy 1, policy_version 51970 (0.0008) [2023-10-10 18:36:45,047][123582] Updated weights for policy 0, policy_version 52093 (0.0007) [2023-10-10 18:36:45,345][123614] Updated weights for policy 1, policy_version 51980 (0.0008) [2023-10-10 18:36:45,708][123614] Updated weights for policy 1, policy_version 51990 (0.0011) [2023-10-10 18:36:46,082][123614] Updated weights for policy 1, policy_version 52000 (0.0010) [2023-10-10 18:36:48,601][123582] Updated weights for policy 0, policy_version 52103 (0.0008) [2023-10-10 18:36:48,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 106594304. Throughput: 0: 1818.5, 1: 1810.5. Samples: 26660892. Policy #0 lag: (min: 22.0, avg: 24.7, max: 54.0) [2023-10-10 18:36:48,789][122664] Avg episode reward: [(0, '64.390'), (1, '56.760')] [2023-10-10 18:36:48,971][123582] Updated weights for policy 0, policy_version 52113 (0.0009) [2023-10-10 18:36:49,347][123582] Updated weights for policy 0, policy_version 52123 (0.0009) [2023-10-10 18:36:49,868][123614] Updated weights for policy 1, policy_version 52010 (0.0010) [2023-10-10 18:36:50,240][123614] Updated weights for policy 1, policy_version 52020 (0.0007) [2023-10-10 18:36:50,605][123614] Updated weights for policy 1, policy_version 52030 (0.0007) [2023-10-10 18:36:53,020][123582] Updated weights for policy 0, policy_version 52133 (0.0007) [2023-10-10 18:36:53,403][123582] Updated weights for policy 0, policy_version 52143 (0.0007) [2023-10-10 18:36:53,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 106659840. Throughput: 0: 1820.9, 1: 1811.2. Samples: 26682954. Policy #0 lag: (min: 22.0, avg: 24.7, max: 54.0) [2023-10-10 18:36:53,788][123582] Updated weights for policy 0, policy_version 52153 (0.0008) [2023-10-10 18:36:53,788][122664] Avg episode reward: [(0, '65.160'), (1, '55.430')] [2023-10-10 18:36:54,167][123614] Updated weights for policy 1, policy_version 52040 (0.0009) [2023-10-10 18:36:54,531][123614] Updated weights for policy 1, policy_version 52050 (0.0009) [2023-10-10 18:36:54,909][123614] Updated weights for policy 1, policy_version 52060 (0.0010) [2023-10-10 18:36:57,547][123582] Updated weights for policy 0, policy_version 52163 (0.0008) [2023-10-10 18:36:57,919][123582] Updated weights for policy 0, policy_version 52173 (0.0009) [2023-10-10 18:36:58,303][123582] Updated weights for policy 0, policy_version 52183 (0.0008) [2023-10-10 18:36:58,571][123614] Updated weights for policy 1, policy_version 52070 (0.0009) [2023-10-10 18:36:58,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 106758144. Throughput: 0: 1822.6, 1: 1815.5. Samples: 26693554. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:36:58,788][122664] Avg episode reward: [(0, '65.890'), (1, '55.200')] [2023-10-10 18:36:58,945][123614] Updated weights for policy 1, policy_version 52080 (0.0008) [2023-10-10 18:36:59,320][123614] Updated weights for policy 1, policy_version 52090 (0.0009) [2023-10-10 18:37:02,059][123582] Updated weights for policy 0, policy_version 52193 (0.0007) [2023-10-10 18:37:02,432][123582] Updated weights for policy 0, policy_version 52203 (0.0008) [2023-10-10 18:37:02,811][123582] Updated weights for policy 0, policy_version 52213 (0.0009) [2023-10-10 18:37:03,177][123582] Updated weights for policy 0, policy_version 52223 (0.0010) [2023-10-10 18:37:03,232][123614] Updated weights for policy 1, policy_version 52100 (0.0010) [2023-10-10 18:37:03,604][123614] Updated weights for policy 1, policy_version 52110 (0.0010) [2023-10-10 18:37:03,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 106823680. Throughput: 0: 1819.5, 1: 1806.3. Samples: 26715456. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:37:03,789][122664] Avg episode reward: [(0, '66.250'), (1, '49.590')] [2023-10-10 18:37:03,982][123614] Updated weights for policy 1, policy_version 52120 (0.0011) [2023-10-10 18:37:07,060][123582] Updated weights for policy 0, policy_version 52233 (0.0007) [2023-10-10 18:37:07,426][123582] Updated weights for policy 0, policy_version 52243 (0.0007) [2023-10-10 18:37:07,655][123614] Updated weights for policy 1, policy_version 52130 (0.0010) [2023-10-10 18:37:07,792][123582] Updated weights for policy 0, policy_version 52253 (0.0007) [2023-10-10 18:37:08,020][123614] Updated weights for policy 1, policy_version 52140 (0.0007) [2023-10-10 18:37:08,399][123614] Updated weights for policy 1, policy_version 52150 (0.0009) [2023-10-10 18:37:08,766][123614] Updated weights for policy 1, policy_version 52160 (0.0010) [2023-10-10 18:37:08,788][122664] Fps is (10 sec: 16383.8, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 106921984. Throughput: 0: 1810.9, 1: 1808.3. Samples: 26735520. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:37:08,789][122664] Avg episode reward: [(0, '64.600'), (1, '50.180')] [2023-10-10 18:37:11,463][123582] Updated weights for policy 0, policy_version 52263 (0.0008) [2023-10-10 18:37:11,841][123582] Updated weights for policy 0, policy_version 52273 (0.0009) [2023-10-10 18:37:12,206][123582] Updated weights for policy 0, policy_version 52283 (0.0008) [2023-10-10 18:37:12,341][123614] Updated weights for policy 1, policy_version 52170 (0.0007) [2023-10-10 18:37:12,702][123614] Updated weights for policy 1, policy_version 52180 (0.0009) [2023-10-10 18:37:13,068][123614] Updated weights for policy 1, policy_version 52190 (0.0007) [2023-10-10 18:37:13,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 106987520. Throughput: 0: 1822.4, 1: 1809.2. Samples: 26748248. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:37:13,788][122664] Avg episode reward: [(0, '69.300'), (1, '47.470')] [2023-10-10 18:37:15,845][123582] Updated weights for policy 0, policy_version 52293 (0.0009) [2023-10-10 18:37:16,211][123582] Updated weights for policy 0, policy_version 52303 (0.0010) [2023-10-10 18:37:16,585][123582] Updated weights for policy 0, policy_version 52313 (0.0008) [2023-10-10 18:37:16,778][123614] Updated weights for policy 1, policy_version 52200 (0.0008) [2023-10-10 18:37:17,147][123614] Updated weights for policy 1, policy_version 52210 (0.0010) [2023-10-10 18:37:17,512][123614] Updated weights for policy 1, policy_version 52220 (0.0008) [2023-10-10 18:37:18,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 107053056. Throughput: 0: 1812.8, 1: 1809.9. Samples: 26768356. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:37:18,789][122664] Avg episode reward: [(0, '69.130'), (1, '47.450')] [2023-10-10 18:37:20,369][123582] Updated weights for policy 0, policy_version 52323 (0.0010) [2023-10-10 18:37:20,730][123582] Updated weights for policy 0, policy_version 52333 (0.0007) [2023-10-10 18:37:21,102][123582] Updated weights for policy 0, policy_version 52343 (0.0009) [2023-10-10 18:37:21,288][123614] Updated weights for policy 1, policy_version 52230 (0.0008) [2023-10-10 18:37:21,643][123614] Updated weights for policy 1, policy_version 52240 (0.0009) [2023-10-10 18:37:22,013][123614] Updated weights for policy 1, policy_version 52250 (0.0011) [2023-10-10 18:37:23,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 107118592. Throughput: 0: 1805.2, 1: 1803.9. Samples: 26790726. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:37:23,789][122664] Avg episode reward: [(0, '68.950'), (1, '50.350')] [2023-10-10 18:37:24,877][123582] Updated weights for policy 0, policy_version 52353 (0.0009) [2023-10-10 18:37:25,252][123582] Updated weights for policy 0, policy_version 52363 (0.0009) [2023-10-10 18:37:25,622][123582] Updated weights for policy 0, policy_version 52373 (0.0008) [2023-10-10 18:37:25,758][123614] Updated weights for policy 1, policy_version 52260 (0.0009) [2023-10-10 18:37:26,003][123582] Updated weights for policy 0, policy_version 52383 (0.0007) [2023-10-10 18:37:26,125][123614] Updated weights for policy 1, policy_version 52270 (0.0007) [2023-10-10 18:37:26,500][123614] Updated weights for policy 1, policy_version 52280 (0.0007) [2023-10-10 18:37:28,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 107184128. Throughput: 0: 1802.8, 1: 1809.2. Samples: 26800724. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:37:28,789][122664] Avg episode reward: [(0, '68.850'), (1, '54.200')] [2023-10-10 18:37:29,742][123582] Updated weights for policy 0, policy_version 52393 (0.0008) [2023-10-10 18:37:30,120][123582] Updated weights for policy 0, policy_version 52403 (0.0008) [2023-10-10 18:37:30,192][123614] Updated weights for policy 1, policy_version 52290 (0.0009) [2023-10-10 18:37:30,496][123582] Updated weights for policy 0, policy_version 52413 (0.0008) [2023-10-10 18:37:30,561][123614] Updated weights for policy 1, policy_version 52300 (0.0009) [2023-10-10 18:37:30,941][123614] Updated weights for policy 1, policy_version 52310 (0.0010) [2023-10-10 18:37:31,302][123614] Updated weights for policy 1, policy_version 52320 (0.0009) [2023-10-10 18:37:33,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 107249664. Throughput: 0: 1797.4, 1: 1814.4. Samples: 26823424. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:37:33,789][122664] Avg episode reward: [(0, '67.460'), (1, '50.900')] [2023-10-10 18:37:34,126][123582] Updated weights for policy 0, policy_version 52423 (0.0010) [2023-10-10 18:37:34,495][123582] Updated weights for policy 0, policy_version 52433 (0.0010) [2023-10-10 18:37:34,859][123582] Updated weights for policy 0, policy_version 52443 (0.0009) [2023-10-10 18:37:35,005][123614] Updated weights for policy 1, policy_version 52330 (0.0008) [2023-10-10 18:37:35,374][123614] Updated weights for policy 1, policy_version 52340 (0.0009) [2023-10-10 18:37:35,748][123614] Updated weights for policy 1, policy_version 52350 (0.0010) [2023-10-10 18:37:38,677][123582] Updated weights for policy 0, policy_version 52453 (0.0009) [2023-10-10 18:37:38,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 107315200. Throughput: 0: 1811.2, 1: 1811.7. Samples: 26845982. Policy #0 lag: (min: 24.0, avg: 49.2, max: 56.0) [2023-10-10 18:37:38,789][122664] Avg episode reward: [(0, '70.740'), (1, '52.740')] [2023-10-10 18:37:39,060][123582] Updated weights for policy 0, policy_version 52463 (0.0009) [2023-10-10 18:37:39,422][123614] Updated weights for policy 1, policy_version 52360 (0.0007) [2023-10-10 18:37:39,432][123582] Updated weights for policy 0, policy_version 52473 (0.0008) [2023-10-10 18:37:39,786][123614] Updated weights for policy 1, policy_version 52370 (0.0008) [2023-10-10 18:37:40,153][123614] Updated weights for policy 1, policy_version 52380 (0.0009) [2023-10-10 18:37:43,139][123582] Updated weights for policy 0, policy_version 52483 (0.0008) [2023-10-10 18:37:43,524][123582] Updated weights for policy 0, policy_version 52493 (0.0009) [2023-10-10 18:37:43,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 107380736. Throughput: 0: 1794.0, 1: 1808.7. Samples: 26855674. Policy #0 lag: (min: 24.0, avg: 49.2, max: 56.0) [2023-10-10 18:37:43,789][122664] Avg episode reward: [(0, '70.140'), (1, '49.420')] [2023-10-10 18:37:43,889][123582] Updated weights for policy 0, policy_version 52503 (0.0008) [2023-10-10 18:37:44,008][123614] Updated weights for policy 1, policy_version 52390 (0.0007) [2023-10-10 18:37:44,389][123614] Updated weights for policy 1, policy_version 52400 (0.0010) [2023-10-10 18:37:44,764][123614] Updated weights for policy 1, policy_version 52410 (0.0008) [2023-10-10 18:37:47,714][123582] Updated weights for policy 0, policy_version 52513 (0.0007) [2023-10-10 18:37:48,094][123582] Updated weights for policy 0, policy_version 52523 (0.0010) [2023-10-10 18:37:48,471][123582] Updated weights for policy 0, policy_version 52533 (0.0008) [2023-10-10 18:37:48,495][123614] Updated weights for policy 1, policy_version 52420 (0.0009) [2023-10-10 18:37:48,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 107446272. Throughput: 0: 1810.8, 1: 1807.5. Samples: 26878280. Policy #0 lag: (min: 24.0, avg: 49.2, max: 56.0) [2023-10-10 18:37:48,788][122664] Avg episode reward: [(0, '69.180'), (1, '51.580')] [2023-10-10 18:37:48,837][123582] Updated weights for policy 0, policy_version 52543 (0.0007) [2023-10-10 18:37:48,855][123614] Updated weights for policy 1, policy_version 52430 (0.0007) [2023-10-10 18:37:49,228][123614] Updated weights for policy 1, policy_version 52440 (0.0010) [2023-10-10 18:37:52,531][123582] Updated weights for policy 0, policy_version 52553 (0.0009) [2023-10-10 18:37:52,902][123582] Updated weights for policy 0, policy_version 52563 (0.0008) [2023-10-10 18:37:52,905][123614] Updated weights for policy 1, policy_version 52450 (0.0008) [2023-10-10 18:37:53,269][123614] Updated weights for policy 1, policy_version 52460 (0.0008) [2023-10-10 18:37:53,279][123582] Updated weights for policy 0, policy_version 52573 (0.0008) [2023-10-10 18:37:53,643][123614] Updated weights for policy 1, policy_version 52470 (0.0010) [2023-10-10 18:37:53,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 107544576. Throughput: 0: 1800.7, 1: 1820.0. Samples: 26898450. Policy #0 lag: (min: 24.0, avg: 49.2, max: 56.0) [2023-10-10 18:37:53,788][122664] Avg episode reward: [(0, '70.160'), (1, '53.420')] [2023-10-10 18:37:53,797][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000052576_53837824.pth... [2023-10-10 18:37:53,834][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000050880_52101120.pth [2023-10-10 18:37:54,014][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000052480_53739520.pth... [2023-10-10 18:37:54,019][123614] Updated weights for policy 1, policy_version 52480 (0.0010) [2023-10-10 18:37:54,044][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000050784_52002816.pth [2023-10-10 18:37:56,898][123582] Updated weights for policy 0, policy_version 52583 (0.0007) [2023-10-10 18:37:57,283][123582] Updated weights for policy 0, policy_version 52593 (0.0010) [2023-10-10 18:37:57,644][123582] Updated weights for policy 0, policy_version 52603 (0.0009) [2023-10-10 18:37:57,821][123614] Updated weights for policy 1, policy_version 52490 (0.0008) [2023-10-10 18:37:58,194][123614] Updated weights for policy 1, policy_version 52500 (0.0009) [2023-10-10 18:37:58,563][123614] Updated weights for policy 1, policy_version 52510 (0.0007) [2023-10-10 18:37:58,788][122664] Fps is (10 sec: 19660.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 107642880. Throughput: 0: 1810.0, 1: 1803.1. Samples: 26910834. Policy #0 lag: (min: 24.0, avg: 49.2, max: 56.0) [2023-10-10 18:37:58,789][122664] Avg episode reward: [(0, '72.640'), (1, '58.360')] [2023-10-10 18:38:01,455][123582] Updated weights for policy 0, policy_version 52613 (0.0009) [2023-10-10 18:38:01,835][123582] Updated weights for policy 0, policy_version 52623 (0.0009) [2023-10-10 18:38:02,197][123582] Updated weights for policy 0, policy_version 52633 (0.0008) [2023-10-10 18:38:02,237][123614] Updated weights for policy 1, policy_version 52520 (0.0007) [2023-10-10 18:38:02,607][123614] Updated weights for policy 1, policy_version 52530 (0.0008) [2023-10-10 18:38:02,978][123614] Updated weights for policy 1, policy_version 52540 (0.0007) [2023-10-10 18:38:03,788][122664] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 107708416. Throughput: 0: 1800.2, 1: 1814.7. Samples: 26931026. Policy #0 lag: (min: 24.0, avg: 49.2, max: 56.0) [2023-10-10 18:38:03,789][122664] Avg episode reward: [(0, '73.830'), (1, '57.590')] [2023-10-10 18:38:05,802][123582] Updated weights for policy 0, policy_version 52643 (0.0008) [2023-10-10 18:38:06,170][123582] Updated weights for policy 0, policy_version 52653 (0.0009) [2023-10-10 18:38:06,553][123582] Updated weights for policy 0, policy_version 52663 (0.0008) [2023-10-10 18:38:06,586][123614] Updated weights for policy 1, policy_version 52550 (0.0007) [2023-10-10 18:38:06,954][123614] Updated weights for policy 1, policy_version 52560 (0.0007) [2023-10-10 18:38:07,325][123614] Updated weights for policy 1, policy_version 52570 (0.0009) [2023-10-10 18:38:08,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 107773952. Throughput: 0: 1806.0, 1: 1813.0. Samples: 26953580. Policy #0 lag: (min: 24.0, avg: 49.2, max: 56.0) [2023-10-10 18:38:08,789][122664] Avg episode reward: [(0, '74.860'), (1, '61.100')] [2023-10-10 18:38:10,217][123582] Updated weights for policy 0, policy_version 52673 (0.0007) [2023-10-10 18:38:10,589][123582] Updated weights for policy 0, policy_version 52683 (0.0007) [2023-10-10 18:38:10,967][123582] Updated weights for policy 0, policy_version 52693 (0.0008) [2023-10-10 18:38:11,027][123614] Updated weights for policy 1, policy_version 52580 (0.0007) [2023-10-10 18:38:11,331][123582] Updated weights for policy 0, policy_version 52703 (0.0009) [2023-10-10 18:38:11,405][123614] Updated weights for policy 1, policy_version 52590 (0.0009) [2023-10-10 18:38:11,772][123614] Updated weights for policy 1, policy_version 52600 (0.0008) [2023-10-10 18:38:13,788][122664] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 107839488. Throughput: 0: 1811.3, 1: 1818.8. Samples: 26964078. Policy #0 lag: (min: 31.0, avg: 32.6, max: 58.0) [2023-10-10 18:38:13,789][122664] Avg episode reward: [(0, '75.670'), (1, '63.400')] [2023-10-10 18:38:15,128][123582] Updated weights for policy 0, policy_version 52713 (0.0011) [2023-10-10 18:38:15,465][123614] Updated weights for policy 1, policy_version 52610 (0.0009) [2023-10-10 18:38:15,498][123582] Updated weights for policy 0, policy_version 52723 (0.0010) [2023-10-10 18:38:15,832][123614] Updated weights for policy 1, policy_version 52620 (0.0008) [2023-10-10 18:38:15,878][123582] Updated weights for policy 0, policy_version 52733 (0.0009) [2023-10-10 18:38:16,203][123614] Updated weights for policy 1, policy_version 52630 (0.0007) [2023-10-10 18:38:16,568][123614] Updated weights for policy 1, policy_version 52640 (0.0008) [2023-10-10 18:38:18,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 107905024. Throughput: 0: 1812.5, 1: 1805.1. Samples: 26986218. Policy #0 lag: (min: 31.0, avg: 32.6, max: 58.0) [2023-10-10 18:38:18,789][122664] Avg episode reward: [(0, '73.400'), (1, '65.850')] [2023-10-10 18:38:19,529][123582] Updated weights for policy 0, policy_version 52743 (0.0008) [2023-10-10 18:38:19,897][123582] Updated weights for policy 0, policy_version 52753 (0.0008) [2023-10-10 18:38:20,227][123614] Updated weights for policy 1, policy_version 52650 (0.0008) [2023-10-10 18:38:20,274][123582] Updated weights for policy 0, policy_version 52763 (0.0008) [2023-10-10 18:38:20,595][123614] Updated weights for policy 1, policy_version 52660 (0.0010) [2023-10-10 18:38:20,964][123614] Updated weights for policy 1, policy_version 52670 (0.0007) [2023-10-10 18:38:23,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 107970560. Throughput: 0: 1819.3, 1: 1809.0. Samples: 27009258. Policy #0 lag: (min: 31.0, avg: 32.6, max: 58.0) [2023-10-10 18:38:23,789][122664] Avg episode reward: [(0, '75.230'), (1, '70.940')] [2023-10-10 18:38:23,896][123582] Updated weights for policy 0, policy_version 52773 (0.0008) [2023-10-10 18:38:24,283][123582] Updated weights for policy 0, policy_version 52783 (0.0009) [2023-10-10 18:38:24,610][123614] Updated weights for policy 1, policy_version 52680 (0.0008) [2023-10-10 18:38:24,649][123582] Updated weights for policy 0, policy_version 52793 (0.0007) [2023-10-10 18:38:24,981][123614] Updated weights for policy 1, policy_version 52690 (0.0007) [2023-10-10 18:38:25,352][123614] Updated weights for policy 1, policy_version 52700 (0.0010) [2023-10-10 18:38:28,109][123582] Updated weights for policy 0, policy_version 52803 (0.0007) [2023-10-10 18:38:28,480][123582] Updated weights for policy 0, policy_version 52813 (0.0007) [2023-10-10 18:38:28,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 108036096. Throughput: 0: 1819.3, 1: 1809.6. Samples: 27018976. Policy #0 lag: (min: 31.0, avg: 32.6, max: 58.0) [2023-10-10 18:38:28,788][122664] Avg episode reward: [(0, '75.530'), (1, '67.940')] [2023-10-10 18:38:28,844][123582] Updated weights for policy 0, policy_version 52823 (0.0008) [2023-10-10 18:38:29,172][123614] Updated weights for policy 1, policy_version 52710 (0.0009) [2023-10-10 18:38:29,548][123614] Updated weights for policy 1, policy_version 52720 (0.0008) [2023-10-10 18:38:29,917][123614] Updated weights for policy 1, policy_version 52730 (0.0008) [2023-10-10 18:38:32,546][123582] Updated weights for policy 0, policy_version 52833 (0.0009) [2023-10-10 18:38:32,919][123582] Updated weights for policy 0, policy_version 52843 (0.0008) [2023-10-10 18:38:33,292][123582] Updated weights for policy 0, policy_version 52853 (0.0007) [2023-10-10 18:38:33,574][123614] Updated weights for policy 1, policy_version 52740 (0.0009) [2023-10-10 18:38:33,661][123582] Updated weights for policy 0, policy_version 52863 (0.0008) [2023-10-10 18:38:33,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 108134400. Throughput: 0: 1811.8, 1: 1815.1. Samples: 27041494. Policy #0 lag: (min: 31.0, avg: 32.6, max: 58.0) [2023-10-10 18:38:33,789][122664] Avg episode reward: [(0, '71.990'), (1, '66.470')] [2023-10-10 18:38:33,936][123614] Updated weights for policy 1, policy_version 52750 (0.0007) [2023-10-10 18:38:34,304][123614] Updated weights for policy 1, policy_version 52760 (0.0007) [2023-10-10 18:38:37,403][123582] Updated weights for policy 0, policy_version 52873 (0.0007) [2023-10-10 18:38:37,771][123582] Updated weights for policy 0, policy_version 52883 (0.0009) [2023-10-10 18:38:37,980][123614] Updated weights for policy 1, policy_version 52770 (0.0009) [2023-10-10 18:38:38,135][123582] Updated weights for policy 0, policy_version 52893 (0.0010) [2023-10-10 18:38:38,352][123614] Updated weights for policy 1, policy_version 52780 (0.0009) [2023-10-10 18:38:38,719][123614] Updated weights for policy 1, policy_version 52790 (0.0009) [2023-10-10 18:38:38,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 108199936. Throughput: 0: 1813.2, 1: 1817.2. Samples: 27061816. Policy #0 lag: (min: 31.0, avg: 32.6, max: 58.0) [2023-10-10 18:38:38,788][122664] Avg episode reward: [(0, '71.930'), (1, '66.630')] [2023-10-10 18:38:39,091][123614] Updated weights for policy 1, policy_version 52800 (0.0008) [2023-10-10 18:38:42,015][123582] Updated weights for policy 0, policy_version 52903 (0.0007) [2023-10-10 18:38:42,390][123582] Updated weights for policy 0, policy_version 52913 (0.0008) [2023-10-10 18:38:42,758][123582] Updated weights for policy 0, policy_version 52923 (0.0008) [2023-10-10 18:38:42,905][123614] Updated weights for policy 1, policy_version 52810 (0.0009) [2023-10-10 18:38:43,277][123614] Updated weights for policy 1, policy_version 52820 (0.0010) [2023-10-10 18:38:43,643][123614] Updated weights for policy 1, policy_version 52830 (0.0009) [2023-10-10 18:38:43,788][122664] Fps is (10 sec: 16384.4, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 108298240. Throughput: 0: 1807.9, 1: 1818.2. Samples: 27074006. Policy #0 lag: (min: 31.0, avg: 32.6, max: 58.0) [2023-10-10 18:38:43,788][122664] Avg episode reward: [(0, '74.500'), (1, '66.140')] [2023-10-10 18:38:46,349][123582] Updated weights for policy 0, policy_version 52933 (0.0008) [2023-10-10 18:38:46,726][123582] Updated weights for policy 0, policy_version 52943 (0.0008) [2023-10-10 18:38:47,110][123582] Updated weights for policy 0, policy_version 52953 (0.0009) [2023-10-10 18:38:47,201][123614] Updated weights for policy 1, policy_version 52840 (0.0007) [2023-10-10 18:38:47,566][123614] Updated weights for policy 1, policy_version 52850 (0.0009) [2023-10-10 18:38:47,937][123614] Updated weights for policy 1, policy_version 52860 (0.0009) [2023-10-10 18:38:48,788][122664] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 108363776. Throughput: 0: 1815.8, 1: 1822.6. Samples: 27094754. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:38:48,789][122664] Avg episode reward: [(0, '75.810'), (1, '67.500')] [2023-10-10 18:38:50,846][123582] Updated weights for policy 0, policy_version 52963 (0.0007) [2023-10-10 18:38:51,220][123582] Updated weights for policy 0, policy_version 52973 (0.0007) [2023-10-10 18:38:51,588][123582] Updated weights for policy 0, policy_version 52983 (0.0007) [2023-10-10 18:38:51,655][123614] Updated weights for policy 1, policy_version 52870 (0.0007) [2023-10-10 18:38:52,028][123614] Updated weights for policy 1, policy_version 52880 (0.0008) [2023-10-10 18:38:52,384][123614] Updated weights for policy 1, policy_version 52890 (0.0007) [2023-10-10 18:38:53,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 108429312. Throughput: 0: 1812.6, 1: 1813.5. Samples: 27116754. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:38:53,789][122664] Avg episode reward: [(0, '74.420'), (1, '68.500')] [2023-10-10 18:38:55,393][123582] Updated weights for policy 0, policy_version 52993 (0.0007) [2023-10-10 18:38:55,767][123582] Updated weights for policy 0, policy_version 53003 (0.0009) [2023-10-10 18:38:56,142][123582] Updated weights for policy 0, policy_version 53013 (0.0009) [2023-10-10 18:38:56,188][123614] Updated weights for policy 1, policy_version 52900 (0.0009) [2023-10-10 18:38:56,517][123582] Updated weights for policy 0, policy_version 53023 (0.0008) [2023-10-10 18:38:56,559][123614] Updated weights for policy 1, policy_version 52910 (0.0010) [2023-10-10 18:38:56,931][123614] Updated weights for policy 1, policy_version 52920 (0.0008) [2023-10-10 18:38:58,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 108494848. Throughput: 0: 1811.9, 1: 1818.3. Samples: 27127440. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:38:58,789][122664] Avg episode reward: [(0, '76.520'), (1, '62.610')] [2023-10-10 18:39:00,292][123582] Updated weights for policy 0, policy_version 53033 (0.0008) [2023-10-10 18:39:00,627][123614] Updated weights for policy 1, policy_version 52930 (0.0007) [2023-10-10 18:39:00,660][123582] Updated weights for policy 0, policy_version 53043 (0.0008) [2023-10-10 18:39:00,986][123614] Updated weights for policy 1, policy_version 52940 (0.0008) [2023-10-10 18:39:01,027][123582] Updated weights for policy 0, policy_version 53053 (0.0009) [2023-10-10 18:39:01,359][123614] Updated weights for policy 1, policy_version 52950 (0.0009) [2023-10-10 18:39:01,730][123614] Updated weights for policy 1, policy_version 52960 (0.0008) [2023-10-10 18:39:03,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 108560384. Throughput: 0: 1810.0, 1: 1811.7. Samples: 27149194. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:39:03,789][122664] Avg episode reward: [(0, '76.720'), (1, '66.820')] [2023-10-10 18:39:04,629][123582] Updated weights for policy 0, policy_version 53063 (0.0009) [2023-10-10 18:39:05,002][123582] Updated weights for policy 0, policy_version 53073 (0.0009) [2023-10-10 18:39:05,381][123582] Updated weights for policy 0, policy_version 53083 (0.0009) [2023-10-10 18:39:05,442][123614] Updated weights for policy 1, policy_version 52970 (0.0008) [2023-10-10 18:39:05,819][123614] Updated weights for policy 1, policy_version 52980 (0.0011) [2023-10-10 18:39:06,176][123614] Updated weights for policy 1, policy_version 52990 (0.0010) [2023-10-10 18:39:08,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 108625920. Throughput: 0: 1808.3, 1: 1808.9. Samples: 27172032. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:39:08,789][122664] Avg episode reward: [(0, '73.140'), (1, '65.490')] [2023-10-10 18:39:09,132][123582] Updated weights for policy 0, policy_version 53093 (0.0007) [2023-10-10 18:39:09,510][123582] Updated weights for policy 0, policy_version 53103 (0.0009) [2023-10-10 18:39:09,886][123582] Updated weights for policy 0, policy_version 53113 (0.0009) [2023-10-10 18:39:09,974][123614] Updated weights for policy 1, policy_version 53000 (0.0009) [2023-10-10 18:39:10,338][123614] Updated weights for policy 1, policy_version 53010 (0.0008) [2023-10-10 18:39:10,706][123614] Updated weights for policy 1, policy_version 53020 (0.0008) [2023-10-10 18:39:13,647][123582] Updated weights for policy 0, policy_version 53123 (0.0008) [2023-10-10 18:39:13,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 108691456. Throughput: 0: 1807.2, 1: 1810.8. Samples: 27181786. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:39:13,788][122664] Avg episode reward: [(0, '76.120'), (1, '60.500')] [2023-10-10 18:39:14,010][123582] Updated weights for policy 0, policy_version 53133 (0.0007) [2023-10-10 18:39:14,386][123582] Updated weights for policy 0, policy_version 53143 (0.0008) [2023-10-10 18:39:14,455][123614] Updated weights for policy 1, policy_version 53030 (0.0008) [2023-10-10 18:39:14,821][123614] Updated weights for policy 1, policy_version 53040 (0.0008) [2023-10-10 18:39:15,187][123614] Updated weights for policy 1, policy_version 53050 (0.0007) [2023-10-10 18:39:18,173][123582] Updated weights for policy 0, policy_version 53153 (0.0009) [2023-10-10 18:39:18,556][123582] Updated weights for policy 0, policy_version 53163 (0.0008) [2023-10-10 18:39:18,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 108756992. Throughput: 0: 1811.5, 1: 1814.1. Samples: 27204646. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:39:18,788][122664] Avg episode reward: [(0, '76.880'), (1, '58.160')] [2023-10-10 18:39:18,797][123614] Updated weights for policy 1, policy_version 53060 (0.0007) [2023-10-10 18:39:18,922][123582] Updated weights for policy 0, policy_version 53173 (0.0008) [2023-10-10 18:39:19,169][123614] Updated weights for policy 1, policy_version 53070 (0.0007) [2023-10-10 18:39:19,287][123582] Updated weights for policy 0, policy_version 53183 (0.0008) [2023-10-10 18:39:19,536][123614] Updated weights for policy 1, policy_version 53080 (0.0007) [2023-10-10 18:39:23,033][123582] Updated weights for policy 0, policy_version 53193 (0.0009) [2023-10-10 18:39:23,217][123614] Updated weights for policy 1, policy_version 53090 (0.0008) [2023-10-10 18:39:23,398][123582] Updated weights for policy 0, policy_version 53203 (0.0010) [2023-10-10 18:39:23,590][123614] Updated weights for policy 1, policy_version 53100 (0.0009) [2023-10-10 18:39:23,765][123582] Updated weights for policy 0, policy_version 53213 (0.0007) [2023-10-10 18:39:23,788][122664] Fps is (10 sec: 13106.6, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 108822528. Throughput: 0: 1822.9, 1: 1816.2. Samples: 27225576. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:39:23,789][122664] Avg episode reward: [(0, '73.480'), (1, '57.700')] [2023-10-10 18:39:23,966][123614] Updated weights for policy 1, policy_version 53110 (0.0007) [2023-10-10 18:39:24,328][123614] Updated weights for policy 1, policy_version 53120 (0.0008) [2023-10-10 18:39:27,344][123582] Updated weights for policy 0, policy_version 53223 (0.0010) [2023-10-10 18:39:27,715][123582] Updated weights for policy 0, policy_version 53233 (0.0008) [2023-10-10 18:39:27,948][123614] Updated weights for policy 1, policy_version 53130 (0.0008) [2023-10-10 18:39:28,082][123582] Updated weights for policy 0, policy_version 53243 (0.0007) [2023-10-10 18:39:28,305][123614] Updated weights for policy 1, policy_version 53140 (0.0007) [2023-10-10 18:39:28,680][123614] Updated weights for policy 1, policy_version 53150 (0.0010) [2023-10-10 18:39:28,788][122664] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 108953600. Throughput: 0: 1814.3, 1: 1812.1. Samples: 27237194. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:39:28,788][122664] Avg episode reward: [(0, '73.840'), (1, '57.750')] [2023-10-10 18:39:31,758][123582] Updated weights for policy 0, policy_version 53253 (0.0008) [2023-10-10 18:39:32,133][123582] Updated weights for policy 0, policy_version 53263 (0.0008) [2023-10-10 18:39:32,394][123614] Updated weights for policy 1, policy_version 53160 (0.0007) [2023-10-10 18:39:32,506][123582] Updated weights for policy 0, policy_version 53273 (0.0008) [2023-10-10 18:39:32,755][123614] Updated weights for policy 1, policy_version 53170 (0.0007) [2023-10-10 18:39:33,126][123614] Updated weights for policy 1, policy_version 53180 (0.0009) [2023-10-10 18:39:33,788][122664] Fps is (10 sec: 19661.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 109019136. Throughput: 0: 1821.0, 1: 1815.8. Samples: 27258412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:39:33,789][122664] Avg episode reward: [(0, '70.650'), (1, '57.960')] [2023-10-10 18:39:36,287][123582] Updated weights for policy 0, policy_version 53283 (0.0008) [2023-10-10 18:39:36,644][123582] Updated weights for policy 0, policy_version 53293 (0.0008) [2023-10-10 18:39:36,970][123614] Updated weights for policy 1, policy_version 53190 (0.0009) [2023-10-10 18:39:37,022][123582] Updated weights for policy 0, policy_version 53303 (0.0008) [2023-10-10 18:39:37,339][123614] Updated weights for policy 1, policy_version 53200 (0.0007) [2023-10-10 18:39:37,707][123614] Updated weights for policy 1, policy_version 53210 (0.0010) [2023-10-10 18:39:38,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 109084672. Throughput: 0: 1809.5, 1: 1806.3. Samples: 27279462. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:39:38,789][122664] Avg episode reward: [(0, '72.730'), (1, '52.650')] [2023-10-10 18:39:40,589][123582] Updated weights for policy 0, policy_version 53313 (0.0008) [2023-10-10 18:39:40,968][123582] Updated weights for policy 0, policy_version 53323 (0.0008) [2023-10-10 18:39:41,341][123582] Updated weights for policy 0, policy_version 53333 (0.0008) [2023-10-10 18:39:41,381][123614] Updated weights for policy 1, policy_version 53220 (0.0008) [2023-10-10 18:39:41,713][123582] Updated weights for policy 0, policy_version 53343 (0.0008) [2023-10-10 18:39:41,749][123614] Updated weights for policy 1, policy_version 53230 (0.0009) [2023-10-10 18:39:42,117][123614] Updated weights for policy 1, policy_version 53240 (0.0007) [2023-10-10 18:39:43,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 109150208. Throughput: 0: 1817.1, 1: 1812.7. Samples: 27290780. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:39:43,789][122664] Avg episode reward: [(0, '72.910'), (1, '52.440')] [2023-10-10 18:39:45,426][123582] Updated weights for policy 0, policy_version 53353 (0.0009) [2023-10-10 18:39:45,797][123582] Updated weights for policy 0, policy_version 53363 (0.0010) [2023-10-10 18:39:45,843][123614] Updated weights for policy 1, policy_version 53250 (0.0008) [2023-10-10 18:39:46,165][123582] Updated weights for policy 0, policy_version 53373 (0.0009) [2023-10-10 18:39:46,217][123614] Updated weights for policy 1, policy_version 53260 (0.0009) [2023-10-10 18:39:46,573][123614] Updated weights for policy 1, policy_version 53270 (0.0010) [2023-10-10 18:39:46,945][123614] Updated weights for policy 1, policy_version 53280 (0.0009) [2023-10-10 18:39:48,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 109215744. Throughput: 0: 1808.9, 1: 1809.3. Samples: 27312012. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:39:48,789][122664] Avg episode reward: [(0, '80.390'), (1, '55.520')] [2023-10-10 18:39:49,840][123582] Updated weights for policy 0, policy_version 53383 (0.0008) [2023-10-10 18:39:50,213][123582] Updated weights for policy 0, policy_version 53393 (0.0008) [2023-10-10 18:39:50,592][123582] Updated weights for policy 0, policy_version 53403 (0.0009) [2023-10-10 18:39:50,640][123614] Updated weights for policy 1, policy_version 53290 (0.0007) [2023-10-10 18:39:51,010][123614] Updated weights for policy 1, policy_version 53300 (0.0009) [2023-10-10 18:39:51,388][123614] Updated weights for policy 1, policy_version 53310 (0.0008) [2023-10-10 18:39:53,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 109281280. Throughput: 0: 1805.7, 1: 1812.0. Samples: 27334828. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:39:53,788][122664] Avg episode reward: [(0, '83.730'), (1, '60.150')] [2023-10-10 18:39:53,796][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000053408_54689792.pth... [2023-10-10 18:39:53,797][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000053312_54591488.pth... [2023-10-10 18:39:53,832][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000051616_52854784.pth [2023-10-10 18:39:53,838][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000051712_52953088.pth [2023-10-10 18:39:54,283][123582] Updated weights for policy 0, policy_version 53413 (0.0008) [2023-10-10 18:39:54,684][123582] Updated weights for policy 0, policy_version 53423 (0.0009) [2023-10-10 18:39:55,007][123614] Updated weights for policy 1, policy_version 53320 (0.0008) [2023-10-10 18:39:55,050][123582] Updated weights for policy 0, policy_version 53433 (0.0007) [2023-10-10 18:39:55,369][123614] Updated weights for policy 1, policy_version 53330 (0.0007) [2023-10-10 18:39:55,745][123614] Updated weights for policy 1, policy_version 53340 (0.0009) [2023-10-10 18:39:58,657][123582] Updated weights for policy 0, policy_version 53443 (0.0009) [2023-10-10 18:39:58,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 109346816. Throughput: 0: 1808.9, 1: 1814.0. Samples: 27344820. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:39:58,788][122664] Avg episode reward: [(0, '83.130'), (1, '59.250')] [2023-10-10 18:39:59,026][123582] Updated weights for policy 0, policy_version 53453 (0.0008) [2023-10-10 18:39:59,393][123582] Updated weights for policy 0, policy_version 53463 (0.0008) [2023-10-10 18:39:59,407][123614] Updated weights for policy 1, policy_version 53350 (0.0008) [2023-10-10 18:39:59,775][123614] Updated weights for policy 1, policy_version 53360 (0.0007) [2023-10-10 18:40:00,148][123614] Updated weights for policy 1, policy_version 53370 (0.0008) [2023-10-10 18:40:03,152][123582] Updated weights for policy 0, policy_version 53473 (0.0007) [2023-10-10 18:40:03,530][123582] Updated weights for policy 0, policy_version 53483 (0.0008) [2023-10-10 18:40:03,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 109412352. Throughput: 0: 1809.6, 1: 1815.3. Samples: 27367766. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:40:03,788][122664] Avg episode reward: [(0, '78.690'), (1, '57.250')] [2023-10-10 18:40:03,902][123582] Updated weights for policy 0, policy_version 53493 (0.0009) [2023-10-10 18:40:03,932][123614] Updated weights for policy 1, policy_version 53380 (0.0008) [2023-10-10 18:40:04,276][123582] Updated weights for policy 0, policy_version 53503 (0.0008) [2023-10-10 18:40:04,326][123614] Updated weights for policy 1, policy_version 53390 (0.0008) [2023-10-10 18:40:04,696][123614] Updated weights for policy 1, policy_version 53400 (0.0010) [2023-10-10 18:40:07,923][123582] Updated weights for policy 0, policy_version 53513 (0.0010) [2023-10-10 18:40:08,267][123614] Updated weights for policy 1, policy_version 53410 (0.0008) [2023-10-10 18:40:08,286][123582] Updated weights for policy 0, policy_version 53523 (0.0010) [2023-10-10 18:40:08,630][123614] Updated weights for policy 1, policy_version 53420 (0.0007) [2023-10-10 18:40:08,657][123582] Updated weights for policy 0, policy_version 53533 (0.0007) [2023-10-10 18:40:08,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 109510656. Throughput: 0: 1808.6, 1: 1816.8. Samples: 27388718. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:40:08,789][122664] Avg episode reward: [(0, '82.700'), (1, '55.260')] [2023-10-10 18:40:08,985][123614] Updated weights for policy 1, policy_version 53430 (0.0007) [2023-10-10 18:40:09,350][123614] Updated weights for policy 1, policy_version 53440 (0.0009) [2023-10-10 18:40:12,442][123582] Updated weights for policy 0, policy_version 53543 (0.0008) [2023-10-10 18:40:12,810][123582] Updated weights for policy 0, policy_version 53553 (0.0009) [2023-10-10 18:40:13,176][123614] Updated weights for policy 1, policy_version 53450 (0.0008) [2023-10-10 18:40:13,181][123582] Updated weights for policy 0, policy_version 53563 (0.0008) [2023-10-10 18:40:13,550][123614] Updated weights for policy 1, policy_version 53460 (0.0008) [2023-10-10 18:40:13,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 109576192. Throughput: 0: 1806.5, 1: 1815.9. Samples: 27400204. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:40:13,788][122664] Avg episode reward: [(0, '80.510'), (1, '58.260')] [2023-10-10 18:40:13,908][123614] Updated weights for policy 1, policy_version 53470 (0.0008) [2023-10-10 18:40:16,914][123582] Updated weights for policy 0, policy_version 53573 (0.0008) [2023-10-10 18:40:17,273][123582] Updated weights for policy 0, policy_version 53583 (0.0009) [2023-10-10 18:40:17,537][123614] Updated weights for policy 1, policy_version 53480 (0.0009) [2023-10-10 18:40:17,645][123582] Updated weights for policy 0, policy_version 53593 (0.0008) [2023-10-10 18:40:17,905][123614] Updated weights for policy 1, policy_version 53490 (0.0009) [2023-10-10 18:40:18,277][123614] Updated weights for policy 1, policy_version 53500 (0.0007) [2023-10-10 18:40:18,788][122664] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 109674496. Throughput: 0: 1808.7, 1: 1815.3. Samples: 27421492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:40:18,788][122664] Avg episode reward: [(0, '80.280'), (1, '60.590')] [2023-10-10 18:40:21,328][123582] Updated weights for policy 0, policy_version 53603 (0.0008) [2023-10-10 18:40:21,704][123582] Updated weights for policy 0, policy_version 53613 (0.0008) [2023-10-10 18:40:22,048][123614] Updated weights for policy 1, policy_version 53510 (0.0009) [2023-10-10 18:40:22,074][123582] Updated weights for policy 0, policy_version 53623 (0.0007) [2023-10-10 18:40:22,409][123614] Updated weights for policy 1, policy_version 53520 (0.0008) [2023-10-10 18:40:22,784][123614] Updated weights for policy 1, policy_version 53530 (0.0008) [2023-10-10 18:40:23,788][122664] Fps is (10 sec: 16383.7, 60 sec: 15291.8, 300 sec: 14551.2). Total num frames: 109740032. Throughput: 0: 1807.7, 1: 1814.0. Samples: 27442442. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:40:23,789][122664] Avg episode reward: [(0, '80.280'), (1, '60.610')] [2023-10-10 18:40:25,815][123582] Updated weights for policy 0, policy_version 53633 (0.0008) [2023-10-10 18:40:26,190][123582] Updated weights for policy 0, policy_version 53643 (0.0007) [2023-10-10 18:40:26,558][123614] Updated weights for policy 1, policy_version 53540 (0.0010) [2023-10-10 18:40:26,559][123582] Updated weights for policy 0, policy_version 53653 (0.0008) [2023-10-10 18:40:26,927][123614] Updated weights for policy 1, policy_version 53550 (0.0009) [2023-10-10 18:40:26,936][123582] Updated weights for policy 0, policy_version 53663 (0.0007) [2023-10-10 18:40:27,299][123614] Updated weights for policy 1, policy_version 53560 (0.0009) [2023-10-10 18:40:28,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 109805568. Throughput: 0: 1811.6, 1: 1816.8. Samples: 27454058. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:40:28,789][122664] Avg episode reward: [(0, '82.280'), (1, '59.070')] [2023-10-10 18:40:30,752][123582] Updated weights for policy 0, policy_version 53673 (0.0008) [2023-10-10 18:40:30,961][123614] Updated weights for policy 1, policy_version 53570 (0.0009) [2023-10-10 18:40:31,128][123582] Updated weights for policy 0, policy_version 53683 (0.0008) [2023-10-10 18:40:31,330][123614] Updated weights for policy 1, policy_version 53580 (0.0008) [2023-10-10 18:40:31,494][123582] Updated weights for policy 0, policy_version 53693 (0.0008) [2023-10-10 18:40:31,695][123614] Updated weights for policy 1, policy_version 53590 (0.0007) [2023-10-10 18:40:32,057][123614] Updated weights for policy 1, policy_version 53600 (0.0009) [2023-10-10 18:40:33,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 109871104. Throughput: 0: 1802.4, 1: 1811.4. Samples: 27474630. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:40:33,788][122664] Avg episode reward: [(0, '84.160'), (1, '60.440')] [2023-10-10 18:40:35,230][123582] Updated weights for policy 0, policy_version 53703 (0.0008) [2023-10-10 18:40:35,604][123582] Updated weights for policy 0, policy_version 53713 (0.0008) [2023-10-10 18:40:35,823][123614] Updated weights for policy 1, policy_version 53610 (0.0008) [2023-10-10 18:40:35,977][123582] Updated weights for policy 0, policy_version 53723 (0.0008) [2023-10-10 18:40:36,188][123614] Updated weights for policy 1, policy_version 53620 (0.0008) [2023-10-10 18:40:36,554][123614] Updated weights for policy 1, policy_version 53630 (0.0008) [2023-10-10 18:40:38,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 109936640. Throughput: 0: 1799.2, 1: 1805.1. Samples: 27497022. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:40:38,788][122664] Avg episode reward: [(0, '84.520'), (1, '59.760')] [2023-10-10 18:40:39,715][123582] Updated weights for policy 0, policy_version 53733 (0.0010) [2023-10-10 18:40:40,102][123582] Updated weights for policy 0, policy_version 53743 (0.0007) [2023-10-10 18:40:40,299][123614] Updated weights for policy 1, policy_version 53640 (0.0007) [2023-10-10 18:40:40,471][123582] Updated weights for policy 0, policy_version 53753 (0.0008) [2023-10-10 18:40:40,664][123614] Updated weights for policy 1, policy_version 53650 (0.0007) [2023-10-10 18:40:41,036][123614] Updated weights for policy 1, policy_version 53660 (0.0008) [2023-10-10 18:40:43,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 110002176. Throughput: 0: 1799.9, 1: 1800.4. Samples: 27506832. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:40:43,789][122664] Avg episode reward: [(0, '79.210'), (1, '61.340')] [2023-10-10 18:40:44,106][123582] Updated weights for policy 0, policy_version 53763 (0.0008) [2023-10-10 18:40:44,468][123582] Updated weights for policy 0, policy_version 53773 (0.0008) [2023-10-10 18:40:44,753][123614] Updated weights for policy 1, policy_version 53670 (0.0009) [2023-10-10 18:40:44,835][123582] Updated weights for policy 0, policy_version 53783 (0.0008) [2023-10-10 18:40:45,117][123614] Updated weights for policy 1, policy_version 53680 (0.0009) [2023-10-10 18:40:45,490][123614] Updated weights for policy 1, policy_version 53690 (0.0009) [2023-10-10 18:40:48,572][123582] Updated weights for policy 0, policy_version 53793 (0.0007) [2023-10-10 18:40:48,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 110067712. Throughput: 0: 1801.1, 1: 1798.4. Samples: 27529742. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:40:48,789][122664] Avg episode reward: [(0, '82.140'), (1, '56.930')] [2023-10-10 18:40:48,934][123582] Updated weights for policy 0, policy_version 53803 (0.0008) [2023-10-10 18:40:49,304][123582] Updated weights for policy 0, policy_version 53813 (0.0008) [2023-10-10 18:40:49,325][123614] Updated weights for policy 1, policy_version 53700 (0.0008) [2023-10-10 18:40:49,670][123582] Updated weights for policy 0, policy_version 53823 (0.0008) [2023-10-10 18:40:49,709][123614] Updated weights for policy 1, policy_version 53710 (0.0009) [2023-10-10 18:40:50,081][123614] Updated weights for policy 1, policy_version 53720 (0.0009) [2023-10-10 18:40:53,306][123582] Updated weights for policy 0, policy_version 53833 (0.0008) [2023-10-10 18:40:53,675][123582] Updated weights for policy 0, policy_version 53843 (0.0009) [2023-10-10 18:40:53,740][123614] Updated weights for policy 1, policy_version 53730 (0.0007) [2023-10-10 18:40:53,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 110133248. Throughput: 0: 1817.4, 1: 1810.1. Samples: 27551954. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:40:53,789][122664] Avg episode reward: [(0, '86.330'), (1, '53.380')] [2023-10-10 18:40:54,041][123582] Updated weights for policy 0, policy_version 53853 (0.0008) [2023-10-10 18:40:54,117][123614] Updated weights for policy 1, policy_version 53740 (0.0010) [2023-10-10 18:40:54,476][123614] Updated weights for policy 1, policy_version 53750 (0.0010) [2023-10-10 18:40:54,844][123614] Updated weights for policy 1, policy_version 53760 (0.0009) [2023-10-10 18:40:57,920][123582] Updated weights for policy 0, policy_version 53863 (0.0007) [2023-10-10 18:40:58,297][123582] Updated weights for policy 0, policy_version 53873 (0.0007) [2023-10-10 18:40:58,553][123614] Updated weights for policy 1, policy_version 53770 (0.0007) [2023-10-10 18:40:58,673][123582] Updated weights for policy 0, policy_version 53883 (0.0007) [2023-10-10 18:40:58,788][122664] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 110198784. Throughput: 0: 1805.0, 1: 1795.8. Samples: 27562240. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:40:58,788][122664] Avg episode reward: [(0, '81.850'), (1, '58.310')] [2023-10-10 18:40:58,923][123614] Updated weights for policy 1, policy_version 53780 (0.0008) [2023-10-10 18:40:59,300][123614] Updated weights for policy 1, policy_version 53790 (0.0008) [2023-10-10 18:41:02,226][123582] Updated weights for policy 0, policy_version 53893 (0.0007) [2023-10-10 18:41:02,605][123582] Updated weights for policy 0, policy_version 53903 (0.0007) [2023-10-10 18:41:02,969][123582] Updated weights for policy 0, policy_version 53913 (0.0008) [2023-10-10 18:41:02,973][123614] Updated weights for policy 1, policy_version 53800 (0.0010) [2023-10-10 18:41:03,347][123614] Updated weights for policy 1, policy_version 53810 (0.0009) [2023-10-10 18:41:03,725][123614] Updated weights for policy 1, policy_version 53820 (0.0008) [2023-10-10 18:41:03,788][122664] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 110297088. Throughput: 0: 1816.6, 1: 1809.8. Samples: 27584680. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:41:03,788][122664] Avg episode reward: [(0, '80.380'), (1, '59.570')] [2023-10-10 18:41:06,739][123582] Updated weights for policy 0, policy_version 53923 (0.0007) [2023-10-10 18:41:07,116][123582] Updated weights for policy 0, policy_version 53933 (0.0008) [2023-10-10 18:41:07,485][123582] Updated weights for policy 0, policy_version 53943 (0.0009) [2023-10-10 18:41:07,486][123614] Updated weights for policy 1, policy_version 53830 (0.0008) [2023-10-10 18:41:07,853][123614] Updated weights for policy 1, policy_version 53840 (0.0007) [2023-10-10 18:41:08,223][123614] Updated weights for policy 1, policy_version 53850 (0.0010) [2023-10-10 18:41:08,788][122664] Fps is (10 sec: 19660.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 110395392. Throughput: 0: 1804.1, 1: 1798.9. Samples: 27604580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:41:08,789][122664] Avg episode reward: [(0, '82.940'), (1, '59.690')] [2023-10-10 18:41:11,205][123582] Updated weights for policy 0, policy_version 53953 (0.0009) [2023-10-10 18:41:11,587][123582] Updated weights for policy 0, policy_version 53963 (0.0010) [2023-10-10 18:41:11,952][123582] Updated weights for policy 0, policy_version 53973 (0.0008) [2023-10-10 18:41:12,025][123614] Updated weights for policy 1, policy_version 53860 (0.0010) [2023-10-10 18:41:12,319][123582] Updated weights for policy 0, policy_version 53983 (0.0008) [2023-10-10 18:41:12,391][123614] Updated weights for policy 1, policy_version 53870 (0.0009) [2023-10-10 18:41:12,754][123614] Updated weights for policy 1, policy_version 53880 (0.0008) [2023-10-10 18:41:13,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 110460928. Throughput: 0: 1815.0, 1: 1808.3. Samples: 27617106. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:41:13,789][122664] Avg episode reward: [(0, '79.550'), (1, '59.730')] [2023-10-10 18:41:15,974][123582] Updated weights for policy 0, policy_version 53993 (0.0009) [2023-10-10 18:41:16,352][123582] Updated weights for policy 0, policy_version 54003 (0.0009) [2023-10-10 18:41:16,552][123614] Updated weights for policy 1, policy_version 53890 (0.0007) [2023-10-10 18:41:16,721][123582] Updated weights for policy 0, policy_version 54013 (0.0007) [2023-10-10 18:41:16,921][123614] Updated weights for policy 1, policy_version 53900 (0.0007) [2023-10-10 18:41:17,290][123614] Updated weights for policy 1, policy_version 53910 (0.0008) [2023-10-10 18:41:17,653][123614] Updated weights for policy 1, policy_version 53920 (0.0012) [2023-10-10 18:41:18,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 110526464. Throughput: 0: 1812.0, 1: 1807.7. Samples: 27637520. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:41:18,788][122664] Avg episode reward: [(0, '76.540'), (1, '59.270')] [2023-10-10 18:41:20,445][123582] Updated weights for policy 0, policy_version 54023 (0.0007) [2023-10-10 18:41:20,815][123582] Updated weights for policy 0, policy_version 54033 (0.0007) [2023-10-10 18:41:21,176][123582] Updated weights for policy 0, policy_version 54043 (0.0007) [2023-10-10 18:41:21,270][123614] Updated weights for policy 1, policy_version 53930 (0.0007) [2023-10-10 18:41:21,636][123614] Updated weights for policy 1, policy_version 53940 (0.0007) [2023-10-10 18:41:22,015][123614] Updated weights for policy 1, policy_version 53950 (0.0007) [2023-10-10 18:41:23,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 110592000. Throughput: 0: 1818.2, 1: 1815.7. Samples: 27660548. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:41:23,789][122664] Avg episode reward: [(0, '82.140'), (1, '58.840')] [2023-10-10 18:41:24,885][123582] Updated weights for policy 0, policy_version 54053 (0.0009) [2023-10-10 18:41:25,274][123582] Updated weights for policy 0, policy_version 54063 (0.0007) [2023-10-10 18:41:25,558][123614] Updated weights for policy 1, policy_version 53960 (0.0008) [2023-10-10 18:41:25,646][123582] Updated weights for policy 0, policy_version 54073 (0.0007) [2023-10-10 18:41:25,923][123614] Updated weights for policy 1, policy_version 53970 (0.0008) [2023-10-10 18:41:26,298][123614] Updated weights for policy 1, policy_version 53980 (0.0009) [2023-10-10 18:41:28,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 110657536. Throughput: 0: 1814.8, 1: 1817.1. Samples: 27670268. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:41:28,788][122664] Avg episode reward: [(0, '84.620'), (1, '58.450')] [2023-10-10 18:41:29,330][123582] Updated weights for policy 0, policy_version 54083 (0.0008) [2023-10-10 18:41:29,693][123582] Updated weights for policy 0, policy_version 54093 (0.0007) [2023-10-10 18:41:29,868][123614] Updated weights for policy 1, policy_version 53990 (0.0007) [2023-10-10 18:41:30,072][123582] Updated weights for policy 0, policy_version 54103 (0.0008) [2023-10-10 18:41:30,230][123614] Updated weights for policy 1, policy_version 54000 (0.0007) [2023-10-10 18:41:30,597][123614] Updated weights for policy 1, policy_version 54010 (0.0009) [2023-10-10 18:41:33,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 110723072. Throughput: 0: 1811.3, 1: 1816.2. Samples: 27692976. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:41:33,788][122664] Avg episode reward: [(0, '88.070'), (1, '60.080')] [2023-10-10 18:41:33,875][123582] Updated weights for policy 0, policy_version 54113 (0.0008) [2023-10-10 18:41:34,250][123582] Updated weights for policy 0, policy_version 54123 (0.0008) [2023-10-10 18:41:34,463][123614] Updated weights for policy 1, policy_version 54020 (0.0008) [2023-10-10 18:41:34,616][123582] Updated weights for policy 0, policy_version 54133 (0.0009) [2023-10-10 18:41:34,859][123614] Updated weights for policy 1, policy_version 54030 (0.0008) [2023-10-10 18:41:35,001][123582] Updated weights for policy 0, policy_version 54143 (0.0009) [2023-10-10 18:41:35,234][123614] Updated weights for policy 1, policy_version 54040 (0.0008) [2023-10-10 18:41:38,719][123582] Updated weights for policy 0, policy_version 54153 (0.0009) [2023-10-10 18:41:38,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 110788608. Throughput: 0: 1815.6, 1: 1814.4. Samples: 27715302. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:41:38,788][122664] Avg episode reward: [(0, '86.410'), (1, '60.580')] [2023-10-10 18:41:38,844][123614] Updated weights for policy 1, policy_version 54050 (0.0009) [2023-10-10 18:41:39,100][123582] Updated weights for policy 0, policy_version 54163 (0.0007) [2023-10-10 18:41:39,210][123614] Updated weights for policy 1, policy_version 54060 (0.0009) [2023-10-10 18:41:39,464][123582] Updated weights for policy 0, policy_version 54173 (0.0008) [2023-10-10 18:41:39,585][123614] Updated weights for policy 1, policy_version 54070 (0.0008) [2023-10-10 18:41:39,945][123614] Updated weights for policy 1, policy_version 54080 (0.0008) [2023-10-10 18:41:43,214][123582] Updated weights for policy 0, policy_version 54183 (0.0007) [2023-10-10 18:41:43,590][123582] Updated weights for policy 0, policy_version 54193 (0.0007) [2023-10-10 18:41:43,637][123614] Updated weights for policy 1, policy_version 54090 (0.0007) [2023-10-10 18:41:43,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 110854144. Throughput: 0: 1807.2, 1: 1817.0. Samples: 27725330. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:41:43,789][122664] Avg episode reward: [(0, '83.090'), (1, '59.130')] [2023-10-10 18:41:43,965][123582] Updated weights for policy 0, policy_version 54203 (0.0007) [2023-10-10 18:41:43,996][123614] Updated weights for policy 1, policy_version 54100 (0.0008) [2023-10-10 18:41:44,363][123614] Updated weights for policy 1, policy_version 54110 (0.0007) [2023-10-10 18:41:47,715][123582] Updated weights for policy 0, policy_version 54213 (0.0007) [2023-10-10 18:41:48,002][123614] Updated weights for policy 1, policy_version 54120 (0.0009) [2023-10-10 18:41:48,089][123582] Updated weights for policy 0, policy_version 54223 (0.0010) [2023-10-10 18:41:48,372][123614] Updated weights for policy 1, policy_version 54130 (0.0007) [2023-10-10 18:41:48,463][123582] Updated weights for policy 0, policy_version 54233 (0.0008) [2023-10-10 18:41:48,736][123614] Updated weights for policy 1, policy_version 54140 (0.0008) [2023-10-10 18:41:48,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 110952448. Throughput: 0: 1816.3, 1: 1817.7. Samples: 27748214. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:41:48,789][122664] Avg episode reward: [(0, '79.670'), (1, '60.420')] [2023-10-10 18:41:52,062][123582] Updated weights for policy 0, policy_version 54243 (0.0007) [2023-10-10 18:41:52,430][123614] Updated weights for policy 1, policy_version 54150 (0.0009) [2023-10-10 18:41:52,437][123582] Updated weights for policy 0, policy_version 54253 (0.0009) [2023-10-10 18:41:52,797][123582] Updated weights for policy 0, policy_version 54263 (0.0009) [2023-10-10 18:41:52,803][123614] Updated weights for policy 1, policy_version 54160 (0.0007) [2023-10-10 18:41:53,159][123614] Updated weights for policy 1, policy_version 54170 (0.0007) [2023-10-10 18:41:53,788][122664] Fps is (10 sec: 19660.9, 60 sec: 15291.8, 300 sec: 14551.2). Total num frames: 111050752. Throughput: 0: 1811.5, 1: 1820.1. Samples: 27768002. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:41:53,788][122664] Avg episode reward: [(0, '79.550'), (1, '63.510')] [2023-10-10 18:41:53,799][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000054176_55476224.pth... [2023-10-10 18:41:53,799][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000054272_55574528.pth... [2023-10-10 18:41:53,828][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000052480_53739520.pth [2023-10-10 18:41:53,834][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000052576_53837824.pth [2023-10-10 18:41:56,511][123582] Updated weights for policy 0, policy_version 54273 (0.0007) [2023-10-10 18:41:56,882][123582] Updated weights for policy 0, policy_version 54283 (0.0008) [2023-10-10 18:41:57,008][123614] Updated weights for policy 1, policy_version 54180 (0.0007) [2023-10-10 18:41:57,249][123582] Updated weights for policy 0, policy_version 54293 (0.0008) [2023-10-10 18:41:57,371][123614] Updated weights for policy 1, policy_version 54190 (0.0008) [2023-10-10 18:41:57,618][123582] Updated weights for policy 0, policy_version 54303 (0.0008) [2023-10-10 18:41:57,735][123614] Updated weights for policy 1, policy_version 54200 (0.0008) [2023-10-10 18:41:58,788][122664] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 111116288. Throughput: 0: 1823.1, 1: 1813.0. Samples: 27780730. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:41:58,789][122664] Avg episode reward: [(0, '80.870'), (1, '60.000')] [2023-10-10 18:42:01,412][123582] Updated weights for policy 0, policy_version 54313 (0.0010) [2023-10-10 18:42:01,540][123614] Updated weights for policy 1, policy_version 54210 (0.0009) [2023-10-10 18:42:01,767][123582] Updated weights for policy 0, policy_version 54323 (0.0009) [2023-10-10 18:42:01,914][123614] Updated weights for policy 1, policy_version 54220 (0.0007) [2023-10-10 18:42:02,152][123582] Updated weights for policy 0, policy_version 54333 (0.0007) [2023-10-10 18:42:02,285][123614] Updated weights for policy 1, policy_version 54230 (0.0008) [2023-10-10 18:42:02,650][123614] Updated weights for policy 1, policy_version 54240 (0.0009) [2023-10-10 18:42:03,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 111181824. Throughput: 0: 1806.7, 1: 1810.3. Samples: 27800290. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:42:03,789][122664] Avg episode reward: [(0, '80.660'), (1, '55.660')] [2023-10-10 18:42:05,840][123582] Updated weights for policy 0, policy_version 54343 (0.0008) [2023-10-10 18:42:06,209][123582] Updated weights for policy 0, policy_version 54353 (0.0009) [2023-10-10 18:42:06,250][123614] Updated weights for policy 1, policy_version 54250 (0.0008) [2023-10-10 18:42:06,575][123582] Updated weights for policy 0, policy_version 54363 (0.0009) [2023-10-10 18:42:06,617][123614] Updated weights for policy 1, policy_version 54260 (0.0008) [2023-10-10 18:42:06,983][123614] Updated weights for policy 1, policy_version 54270 (0.0009) [2023-10-10 18:42:08,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 111247360. Throughput: 0: 1801.8, 1: 1804.2. Samples: 27822820. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:42:08,789][122664] Avg episode reward: [(0, '82.060'), (1, '51.000')] [2023-10-10 18:42:10,391][123582] Updated weights for policy 0, policy_version 54373 (0.0008) [2023-10-10 18:42:10,659][123614] Updated weights for policy 1, policy_version 54280 (0.0010) [2023-10-10 18:42:10,775][123582] Updated weights for policy 0, policy_version 54383 (0.0008) [2023-10-10 18:42:11,017][123614] Updated weights for policy 1, policy_version 54290 (0.0009) [2023-10-10 18:42:11,156][123582] Updated weights for policy 0, policy_version 54393 (0.0009) [2023-10-10 18:42:11,386][123614] Updated weights for policy 1, policy_version 54300 (0.0008) [2023-10-10 18:42:13,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 111312896. Throughput: 0: 1803.6, 1: 1803.3. Samples: 27832580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:42:13,789][122664] Avg episode reward: [(0, '81.630'), (1, '49.800')] [2023-10-10 18:42:14,754][123582] Updated weights for policy 0, policy_version 54403 (0.0008) [2023-10-10 18:42:15,137][123582] Updated weights for policy 0, policy_version 54413 (0.0010) [2023-10-10 18:42:15,319][123614] Updated weights for policy 1, policy_version 54310 (0.0009) [2023-10-10 18:42:15,510][123582] Updated weights for policy 0, policy_version 54423 (0.0009) [2023-10-10 18:42:15,681][123614] Updated weights for policy 1, policy_version 54320 (0.0008) [2023-10-10 18:42:16,058][123614] Updated weights for policy 1, policy_version 54330 (0.0009) [2023-10-10 18:42:18,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 111378432. Throughput: 0: 1802.7, 1: 1794.3. Samples: 27854842. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:42:18,789][122664] Avg episode reward: [(0, '79.480'), (1, '49.780')] [2023-10-10 18:42:19,163][123582] Updated weights for policy 0, policy_version 54433 (0.0007) [2023-10-10 18:42:19,537][123582] Updated weights for policy 0, policy_version 54443 (0.0007) [2023-10-10 18:42:19,901][123582] Updated weights for policy 0, policy_version 54453 (0.0007) [2023-10-10 18:42:19,927][123614] Updated weights for policy 1, policy_version 54340 (0.0009) [2023-10-10 18:42:20,272][123582] Updated weights for policy 0, policy_version 54463 (0.0008) [2023-10-10 18:42:20,307][123614] Updated weights for policy 1, policy_version 54350 (0.0007) [2023-10-10 18:42:20,674][123614] Updated weights for policy 1, policy_version 54360 (0.0008) [2023-10-10 18:42:23,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 111443968. Throughput: 0: 1810.9, 1: 1798.3. Samples: 27877714. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:42:23,789][122664] Avg episode reward: [(0, '84.270'), (1, '49.870')] [2023-10-10 18:42:23,873][123582] Updated weights for policy 0, policy_version 54473 (0.0010) [2023-10-10 18:42:24,247][123582] Updated weights for policy 0, policy_version 54483 (0.0009) [2023-10-10 18:42:24,336][123614] Updated weights for policy 1, policy_version 54370 (0.0009) [2023-10-10 18:42:24,624][123582] Updated weights for policy 0, policy_version 54493 (0.0009) [2023-10-10 18:42:24,708][123614] Updated weights for policy 1, policy_version 54380 (0.0009) [2023-10-10 18:42:25,069][123614] Updated weights for policy 1, policy_version 54390 (0.0009) [2023-10-10 18:42:25,438][123614] Updated weights for policy 1, policy_version 54400 (0.0007) [2023-10-10 18:42:28,234][123582] Updated weights for policy 0, policy_version 54503 (0.0007) [2023-10-10 18:42:28,609][123582] Updated weights for policy 0, policy_version 54513 (0.0010) [2023-10-10 18:42:28,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 111509504. Throughput: 0: 1811.2, 1: 1797.4. Samples: 27887720. Policy #0 lag: (min: 30.0, avg: 33.3, max: 62.0) [2023-10-10 18:42:28,788][122664] Avg episode reward: [(0, '88.130'), (1, '47.800')] [2023-10-10 18:42:28,992][123582] Updated weights for policy 0, policy_version 54523 (0.0009) [2023-10-10 18:42:29,124][123614] Updated weights for policy 1, policy_version 54410 (0.0007) [2023-10-10 18:42:29,497][123614] Updated weights for policy 1, policy_version 54420 (0.0009) [2023-10-10 18:42:29,857][123614] Updated weights for policy 1, policy_version 54430 (0.0008) [2023-10-10 18:42:32,702][123582] Updated weights for policy 0, policy_version 54533 (0.0008) [2023-10-10 18:42:33,071][123582] Updated weights for policy 0, policy_version 54543 (0.0010) [2023-10-10 18:42:33,447][123582] Updated weights for policy 0, policy_version 54553 (0.0008) [2023-10-10 18:42:33,604][123614] Updated weights for policy 1, policy_version 54440 (0.0008) [2023-10-10 18:42:33,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 111607808. Throughput: 0: 1812.8, 1: 1798.9. Samples: 27910738. Policy #0 lag: (min: 30.0, avg: 33.3, max: 62.0) [2023-10-10 18:42:33,788][122664] Avg episode reward: [(0, '92.530'), (1, '47.920')] [2023-10-10 18:42:33,967][123614] Updated weights for policy 1, policy_version 54450 (0.0007) [2023-10-10 18:42:34,335][123614] Updated weights for policy 1, policy_version 54460 (0.0009) [2023-10-10 18:42:37,033][123582] Updated weights for policy 0, policy_version 54563 (0.0009) [2023-10-10 18:42:37,406][123582] Updated weights for policy 0, policy_version 54573 (0.0011) [2023-10-10 18:42:37,783][123582] Updated weights for policy 0, policy_version 54583 (0.0008) [2023-10-10 18:42:37,911][123614] Updated weights for policy 1, policy_version 54470 (0.0008) [2023-10-10 18:42:38,276][123614] Updated weights for policy 1, policy_version 54480 (0.0007) [2023-10-10 18:42:38,647][123614] Updated weights for policy 1, policy_version 54490 (0.0008) [2023-10-10 18:42:38,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 111673344. Throughput: 0: 1815.2, 1: 1806.7. Samples: 27930992. Policy #0 lag: (min: 30.0, avg: 33.3, max: 62.0) [2023-10-10 18:42:38,789][122664] Avg episode reward: [(0, '91.900'), (1, '46.350')] [2023-10-10 18:42:41,484][123582] Updated weights for policy 0, policy_version 54593 (0.0007) [2023-10-10 18:42:41,853][123582] Updated weights for policy 0, policy_version 54603 (0.0008) [2023-10-10 18:42:42,219][123582] Updated weights for policy 0, policy_version 54613 (0.0009) [2023-10-10 18:42:42,552][123614] Updated weights for policy 1, policy_version 54500 (0.0008) [2023-10-10 18:42:42,589][123582] Updated weights for policy 0, policy_version 54623 (0.0007) [2023-10-10 18:42:42,914][123614] Updated weights for policy 1, policy_version 54510 (0.0009) [2023-10-10 18:42:43,280][123614] Updated weights for policy 1, policy_version 54520 (0.0010) [2023-10-10 18:42:43,788][122664] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 111771648. Throughput: 0: 1810.1, 1: 1805.6. Samples: 27943438. Policy #0 lag: (min: 30.0, avg: 33.3, max: 62.0) [2023-10-10 18:42:43,788][122664] Avg episode reward: [(0, '86.850'), (1, '45.990')] [2023-10-10 18:42:46,345][123582] Updated weights for policy 0, policy_version 54633 (0.0007) [2023-10-10 18:42:46,709][123582] Updated weights for policy 0, policy_version 54643 (0.0007) [2023-10-10 18:42:47,083][123582] Updated weights for policy 0, policy_version 54653 (0.0008) [2023-10-10 18:42:47,165][123614] Updated weights for policy 1, policy_version 54530 (0.0009) [2023-10-10 18:42:47,530][123614] Updated weights for policy 1, policy_version 54540 (0.0008) [2023-10-10 18:42:47,895][123614] Updated weights for policy 1, policy_version 54550 (0.0008) [2023-10-10 18:42:48,260][123614] Updated weights for policy 1, policy_version 54560 (0.0009) [2023-10-10 18:42:48,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 111837184. Throughput: 0: 1815.6, 1: 1817.6. Samples: 27963784. Policy #0 lag: (min: 30.0, avg: 33.3, max: 62.0) [2023-10-10 18:42:48,789][122664] Avg episode reward: [(0, '88.410'), (1, '46.070')] [2023-10-10 18:42:50,716][123582] Updated weights for policy 0, policy_version 54663 (0.0009) [2023-10-10 18:42:51,087][123582] Updated weights for policy 0, policy_version 54673 (0.0008) [2023-10-10 18:42:51,456][123582] Updated weights for policy 0, policy_version 54683 (0.0007) [2023-10-10 18:42:52,032][123614] Updated weights for policy 1, policy_version 54570 (0.0011) [2023-10-10 18:42:52,392][123614] Updated weights for policy 1, policy_version 54580 (0.0009) [2023-10-10 18:42:52,766][123614] Updated weights for policy 1, policy_version 54590 (0.0007) [2023-10-10 18:42:53,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 111902720. Throughput: 0: 1820.5, 1: 1802.7. Samples: 27985862. Policy #0 lag: (min: 30.0, avg: 33.3, max: 62.0) [2023-10-10 18:42:53,788][122664] Avg episode reward: [(0, '84.440'), (1, '46.770')] [2023-10-10 18:42:55,258][123582] Updated weights for policy 0, policy_version 54693 (0.0009) [2023-10-10 18:42:55,640][123582] Updated weights for policy 0, policy_version 54703 (0.0008) [2023-10-10 18:42:56,020][123582] Updated weights for policy 0, policy_version 54713 (0.0009) [2023-10-10 18:42:56,453][123614] Updated weights for policy 1, policy_version 54600 (0.0007) [2023-10-10 18:42:56,822][123614] Updated weights for policy 1, policy_version 54610 (0.0009) [2023-10-10 18:42:57,187][123614] Updated weights for policy 1, policy_version 54620 (0.0010) [2023-10-10 18:42:58,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 111968256. Throughput: 0: 1817.6, 1: 1821.5. Samples: 27996338. Policy #0 lag: (min: 30.0, avg: 33.3, max: 62.0) [2023-10-10 18:42:58,788][122664] Avg episode reward: [(0, '84.770'), (1, '52.090')] [2023-10-10 18:42:59,585][123582] Updated weights for policy 0, policy_version 54723 (0.0008) [2023-10-10 18:42:59,952][123582] Updated weights for policy 0, policy_version 54733 (0.0007) [2023-10-10 18:43:00,325][123582] Updated weights for policy 0, policy_version 54743 (0.0007) [2023-10-10 18:43:01,050][123614] Updated weights for policy 1, policy_version 54630 (0.0009) [2023-10-10 18:43:01,419][123614] Updated weights for policy 1, policy_version 54640 (0.0010) [2023-10-10 18:43:01,778][123614] Updated weights for policy 1, policy_version 54650 (0.0009) [2023-10-10 18:43:03,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 112033792. Throughput: 0: 1824.6, 1: 1808.8. Samples: 28018342. Policy #0 lag: (min: 13.0, avg: 15.8, max: 45.0) [2023-10-10 18:43:03,789][122664] Avg episode reward: [(0, '85.750'), (1, '52.970')] [2023-10-10 18:43:04,031][123582] Updated weights for policy 0, policy_version 54753 (0.0007) [2023-10-10 18:43:04,394][123582] Updated weights for policy 0, policy_version 54763 (0.0011) [2023-10-10 18:43:04,777][123582] Updated weights for policy 0, policy_version 54773 (0.0008) [2023-10-10 18:43:05,153][123582] Updated weights for policy 0, policy_version 54783 (0.0010) [2023-10-10 18:43:05,315][123614] Updated weights for policy 1, policy_version 54660 (0.0008) [2023-10-10 18:43:05,693][123614] Updated weights for policy 1, policy_version 54670 (0.0008) [2023-10-10 18:43:06,063][123614] Updated weights for policy 1, policy_version 54680 (0.0007) [2023-10-10 18:43:08,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 112099328. Throughput: 0: 1821.5, 1: 1810.1. Samples: 28041138. Policy #0 lag: (min: 13.0, avg: 15.8, max: 45.0) [2023-10-10 18:43:08,789][122664] Avg episode reward: [(0, '86.220'), (1, '54.000')] [2023-10-10 18:43:08,877][123582] Updated weights for policy 0, policy_version 54793 (0.0008) [2023-10-10 18:43:09,250][123582] Updated weights for policy 0, policy_version 54803 (0.0009) [2023-10-10 18:43:09,629][123582] Updated weights for policy 0, policy_version 54813 (0.0009) [2023-10-10 18:43:09,644][123614] Updated weights for policy 1, policy_version 54690 (0.0007) [2023-10-10 18:43:10,018][123614] Updated weights for policy 1, policy_version 54700 (0.0008) [2023-10-10 18:43:10,386][123614] Updated weights for policy 1, policy_version 54710 (0.0009) [2023-10-10 18:43:10,764][123614] Updated weights for policy 1, policy_version 54720 (0.0008) [2023-10-10 18:43:13,347][123582] Updated weights for policy 0, policy_version 54823 (0.0007) [2023-10-10 18:43:13,719][123582] Updated weights for policy 0, policy_version 54833 (0.0009) [2023-10-10 18:43:13,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 112164864. Throughput: 0: 1821.3, 1: 1811.5. Samples: 28051200. Policy #0 lag: (min: 13.0, avg: 15.8, max: 45.0) [2023-10-10 18:43:13,789][122664] Avg episode reward: [(0, '86.350'), (1, '54.050')] [2023-10-10 18:43:14,090][123582] Updated weights for policy 0, policy_version 54843 (0.0009) [2023-10-10 18:43:14,347][123614] Updated weights for policy 1, policy_version 54730 (0.0009) [2023-10-10 18:43:14,712][123614] Updated weights for policy 1, policy_version 54740 (0.0011) [2023-10-10 18:43:15,075][123614] Updated weights for policy 1, policy_version 54750 (0.0010) [2023-10-10 18:43:17,922][123582] Updated weights for policy 0, policy_version 54853 (0.0008) [2023-10-10 18:43:18,300][123582] Updated weights for policy 0, policy_version 54863 (0.0007) [2023-10-10 18:43:18,666][123582] Updated weights for policy 0, policy_version 54873 (0.0007) [2023-10-10 18:43:18,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 112230400. Throughput: 0: 1816.2, 1: 1808.7. Samples: 28073860. Policy #0 lag: (min: 13.0, avg: 15.8, max: 45.0) [2023-10-10 18:43:18,788][122664] Avg episode reward: [(0, '91.560'), (1, '53.640')] [2023-10-10 18:43:18,815][123614] Updated weights for policy 1, policy_version 54760 (0.0007) [2023-10-10 18:43:19,187][123614] Updated weights for policy 1, policy_version 54770 (0.0009) [2023-10-10 18:43:19,541][123614] Updated weights for policy 1, policy_version 54780 (0.0008) [2023-10-10 18:43:22,311][123582] Updated weights for policy 0, policy_version 54883 (0.0007) [2023-10-10 18:43:22,679][123582] Updated weights for policy 0, policy_version 54893 (0.0007) [2023-10-10 18:43:23,043][123582] Updated weights for policy 0, policy_version 54903 (0.0007) [2023-10-10 18:43:23,204][123614] Updated weights for policy 1, policy_version 54790 (0.0009) [2023-10-10 18:43:23,569][123614] Updated weights for policy 1, policy_version 54800 (0.0008) [2023-10-10 18:43:23,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 112328704. Throughput: 0: 1812.6, 1: 1817.4. Samples: 28094342. Policy #0 lag: (min: 13.0, avg: 15.8, max: 45.0) [2023-10-10 18:43:23,789][122664] Avg episode reward: [(0, '97.040'), (1, '54.490')] [2023-10-10 18:43:23,944][123614] Updated weights for policy 1, policy_version 54810 (0.0009) [2023-10-10 18:43:26,627][123582] Updated weights for policy 0, policy_version 54913 (0.0007) [2023-10-10 18:43:27,000][123582] Updated weights for policy 0, policy_version 54923 (0.0008) [2023-10-10 18:43:27,382][123582] Updated weights for policy 0, policy_version 54933 (0.0010) [2023-10-10 18:43:27,660][123614] Updated weights for policy 1, policy_version 54820 (0.0007) [2023-10-10 18:43:27,750][123582] Updated weights for policy 0, policy_version 54943 (0.0010) [2023-10-10 18:43:28,033][123614] Updated weights for policy 1, policy_version 54830 (0.0008) [2023-10-10 18:43:28,399][123614] Updated weights for policy 1, policy_version 54840 (0.0008) [2023-10-10 18:43:28,788][122664] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 112427008. Throughput: 0: 1820.0, 1: 1814.2. Samples: 28106978. Policy #0 lag: (min: 13.0, avg: 15.8, max: 45.0) [2023-10-10 18:43:28,788][122664] Avg episode reward: [(0, '94.400'), (1, '54.710')] [2023-10-10 18:43:31,436][123582] Updated weights for policy 0, policy_version 54953 (0.0008) [2023-10-10 18:43:31,796][123582] Updated weights for policy 0, policy_version 54963 (0.0010) [2023-10-10 18:43:32,180][123582] Updated weights for policy 0, policy_version 54973 (0.0009) [2023-10-10 18:43:32,332][123614] Updated weights for policy 1, policy_version 54850 (0.0007) [2023-10-10 18:43:32,702][123614] Updated weights for policy 1, policy_version 54860 (0.0010) [2023-10-10 18:43:33,070][123614] Updated weights for policy 1, policy_version 54870 (0.0008) [2023-10-10 18:43:33,430][123614] Updated weights for policy 1, policy_version 54880 (0.0007) [2023-10-10 18:43:33,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 112492544. Throughput: 0: 1818.5, 1: 1816.8. Samples: 28127374. Policy #0 lag: (min: 13.0, avg: 15.8, max: 45.0) [2023-10-10 18:43:33,789][122664] Avg episode reward: [(0, '90.210'), (1, '54.650')] [2023-10-10 18:43:35,875][123582] Updated weights for policy 0, policy_version 54983 (0.0009) [2023-10-10 18:43:36,245][123582] Updated weights for policy 0, policy_version 54993 (0.0009) [2023-10-10 18:43:36,625][123582] Updated weights for policy 0, policy_version 55003 (0.0010) [2023-10-10 18:43:37,017][123614] Updated weights for policy 1, policy_version 54890 (0.0009) [2023-10-10 18:43:37,382][123614] Updated weights for policy 1, policy_version 54900 (0.0011) [2023-10-10 18:43:37,749][123614] Updated weights for policy 1, policy_version 54910 (0.0009) [2023-10-10 18:43:38,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 112558080. Throughput: 0: 1815.7, 1: 1811.1. Samples: 28149072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:43:38,789][122664] Avg episode reward: [(0, '87.680'), (1, '56.320')] [2023-10-10 18:43:40,359][123582] Updated weights for policy 0, policy_version 55013 (0.0009) [2023-10-10 18:43:40,733][123582] Updated weights for policy 0, policy_version 55023 (0.0010) [2023-10-10 18:43:41,113][123582] Updated weights for policy 0, policy_version 55033 (0.0009) [2023-10-10 18:43:41,377][123614] Updated weights for policy 1, policy_version 54920 (0.0009) [2023-10-10 18:43:41,739][123614] Updated weights for policy 1, policy_version 54930 (0.0010) [2023-10-10 18:43:42,113][123614] Updated weights for policy 1, policy_version 54940 (0.0010) [2023-10-10 18:43:43,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 112623616. Throughput: 0: 1815.5, 1: 1812.0. Samples: 28159580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:43:43,789][122664] Avg episode reward: [(0, '90.490'), (1, '52.940')] [2023-10-10 18:43:44,912][123582] Updated weights for policy 0, policy_version 55043 (0.0008) [2023-10-10 18:43:45,300][123582] Updated weights for policy 0, policy_version 55053 (0.0009) [2023-10-10 18:43:45,675][123582] Updated weights for policy 0, policy_version 55063 (0.0009) [2023-10-10 18:43:45,781][123614] Updated weights for policy 1, policy_version 54950 (0.0009) [2023-10-10 18:43:46,144][123614] Updated weights for policy 1, policy_version 54960 (0.0009) [2023-10-10 18:43:46,508][123614] Updated weights for policy 1, policy_version 54970 (0.0008) [2023-10-10 18:43:48,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 112689152. Throughput: 0: 1808.6, 1: 1814.8. Samples: 28181392. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:43:48,789][122664] Avg episode reward: [(0, '89.800'), (1, '53.680')] [2023-10-10 18:43:49,273][123582] Updated weights for policy 0, policy_version 55073 (0.0009) [2023-10-10 18:43:49,654][123582] Updated weights for policy 0, policy_version 55083 (0.0010) [2023-10-10 18:43:50,026][123582] Updated weights for policy 0, policy_version 55093 (0.0011) [2023-10-10 18:43:50,382][123614] Updated weights for policy 1, policy_version 54980 (0.0008) [2023-10-10 18:43:50,398][123582] Updated weights for policy 0, policy_version 55103 (0.0008) [2023-10-10 18:43:50,774][123614] Updated weights for policy 1, policy_version 54990 (0.0009) [2023-10-10 18:43:51,144][123614] Updated weights for policy 1, policy_version 55000 (0.0008) [2023-10-10 18:43:53,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 112754688. Throughput: 0: 1803.7, 1: 1807.4. Samples: 28203638. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:43:53,788][122664] Avg episode reward: [(0, '86.360'), (1, '53.120')] [2023-10-10 18:43:53,796][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000055008_56328192.pth... [2023-10-10 18:43:53,826][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000053312_54591488.pth [2023-10-10 18:43:54,148][123582] Updated weights for policy 0, policy_version 55113 (0.0009) [2023-10-10 18:43:54,529][123582] Updated weights for policy 0, policy_version 55123 (0.0009) [2023-10-10 18:43:54,893][123582] Updated weights for policy 0, policy_version 55133 (0.0010) [2023-10-10 18:43:54,912][123614] Updated weights for policy 1, policy_version 55010 (0.0008) [2023-10-10 18:43:55,001][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000055136_56459264.pth... [2023-10-10 18:43:55,037][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000053408_54689792.pth [2023-10-10 18:43:55,293][123614] Updated weights for policy 1, policy_version 55020 (0.0010) [2023-10-10 18:43:55,666][123614] Updated weights for policy 1, policy_version 55030 (0.0010) [2023-10-10 18:43:56,032][123614] Updated weights for policy 1, policy_version 55040 (0.0010) [2023-10-10 18:43:58,465][123582] Updated weights for policy 0, policy_version 55143 (0.0008) [2023-10-10 18:43:58,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 112820224. Throughput: 0: 1808.6, 1: 1802.6. Samples: 28213704. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:43:58,788][122664] Avg episode reward: [(0, '86.350'), (1, '52.960')] [2023-10-10 18:43:58,847][123582] Updated weights for policy 0, policy_version 55153 (0.0009) [2023-10-10 18:43:59,228][123582] Updated weights for policy 0, policy_version 55163 (0.0008) [2023-10-10 18:43:59,704][123614] Updated weights for policy 1, policy_version 55050 (0.0009) [2023-10-10 18:44:00,072][123614] Updated weights for policy 1, policy_version 55060 (0.0009) [2023-10-10 18:44:00,440][123614] Updated weights for policy 1, policy_version 55070 (0.0008) [2023-10-10 18:44:02,986][123582] Updated weights for policy 0, policy_version 55173 (0.0010) [2023-10-10 18:44:03,367][123582] Updated weights for policy 0, policy_version 55183 (0.0008) [2023-10-10 18:44:03,746][123582] Updated weights for policy 0, policy_version 55193 (0.0009) [2023-10-10 18:44:03,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 112885760. Throughput: 0: 1809.1, 1: 1801.9. Samples: 28236352. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:44:03,788][122664] Avg episode reward: [(0, '87.340'), (1, '52.600')] [2023-10-10 18:44:04,081][123614] Updated weights for policy 1, policy_version 55080 (0.0007) [2023-10-10 18:44:04,450][123614] Updated weights for policy 1, policy_version 55090 (0.0008) [2023-10-10 18:44:04,822][123614] Updated weights for policy 1, policy_version 55100 (0.0007) [2023-10-10 18:44:07,326][123582] Updated weights for policy 0, policy_version 55203 (0.0007) [2023-10-10 18:44:07,700][123582] Updated weights for policy 0, policy_version 55213 (0.0009) [2023-10-10 18:44:08,070][123582] Updated weights for policy 0, policy_version 55223 (0.0008) [2023-10-10 18:44:08,471][123614] Updated weights for policy 1, policy_version 55110 (0.0008) [2023-10-10 18:44:08,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 112984064. Throughput: 0: 1809.3, 1: 1813.5. Samples: 28257368. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:44:08,788][122664] Avg episode reward: [(0, '84.950'), (1, '53.110')] [2023-10-10 18:44:08,843][123614] Updated weights for policy 1, policy_version 55120 (0.0009) [2023-10-10 18:44:09,214][123614] Updated weights for policy 1, policy_version 55130 (0.0008) [2023-10-10 18:44:11,757][123582] Updated weights for policy 0, policy_version 55233 (0.0007) [2023-10-10 18:44:12,121][123582] Updated weights for policy 0, policy_version 55243 (0.0008) [2023-10-10 18:44:12,494][123582] Updated weights for policy 0, policy_version 55253 (0.0008) [2023-10-10 18:44:12,868][123582] Updated weights for policy 0, policy_version 55263 (0.0009) [2023-10-10 18:44:13,027][123614] Updated weights for policy 1, policy_version 55140 (0.0007) [2023-10-10 18:44:13,406][123614] Updated weights for policy 1, policy_version 55150 (0.0010) [2023-10-10 18:44:13,778][123614] Updated weights for policy 1, policy_version 55160 (0.0009) [2023-10-10 18:44:13,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 113049600. Throughput: 0: 1807.7, 1: 1800.2. Samples: 28269332. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:44:13,788][122664] Avg episode reward: [(0, '84.560'), (1, '52.860')] [2023-10-10 18:44:16,416][123582] Updated weights for policy 0, policy_version 55273 (0.0009) [2023-10-10 18:44:16,788][123582] Updated weights for policy 0, policy_version 55283 (0.0007) [2023-10-10 18:44:17,163][123582] Updated weights for policy 0, policy_version 55293 (0.0009) [2023-10-10 18:44:17,449][123614] Updated weights for policy 1, policy_version 55170 (0.0008) [2023-10-10 18:44:17,824][123614] Updated weights for policy 1, policy_version 55180 (0.0007) [2023-10-10 18:44:18,201][123614] Updated weights for policy 1, policy_version 55190 (0.0008) [2023-10-10 18:44:18,565][123614] Updated weights for policy 1, policy_version 55200 (0.0010) [2023-10-10 18:44:18,788][122664] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 113147904. Throughput: 0: 1811.3, 1: 1811.8. Samples: 28290412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:44:18,788][122664] Avg episode reward: [(0, '83.490'), (1, '56.030')] [2023-10-10 18:44:20,936][123582] Updated weights for policy 0, policy_version 55303 (0.0009) [2023-10-10 18:44:21,305][123582] Updated weights for policy 0, policy_version 55313 (0.0008) [2023-10-10 18:44:21,688][123582] Updated weights for policy 0, policy_version 55323 (0.0008) [2023-10-10 18:44:22,299][123614] Updated weights for policy 1, policy_version 55210 (0.0011) [2023-10-10 18:44:22,658][123614] Updated weights for policy 1, policy_version 55220 (0.0010) [2023-10-10 18:44:23,026][123614] Updated weights for policy 1, policy_version 55230 (0.0009) [2023-10-10 18:44:23,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 113213440. Throughput: 0: 1813.4, 1: 1807.5. Samples: 28312012. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:44:23,789][122664] Avg episode reward: [(0, '79.570'), (1, '56.320')] [2023-10-10 18:44:25,507][123582] Updated weights for policy 0, policy_version 55333 (0.0009) [2023-10-10 18:44:25,901][123582] Updated weights for policy 0, policy_version 55343 (0.0009) [2023-10-10 18:44:26,267][123582] Updated weights for policy 0, policy_version 55353 (0.0008) [2023-10-10 18:44:26,672][123614] Updated weights for policy 1, policy_version 55240 (0.0009) [2023-10-10 18:44:27,038][123614] Updated weights for policy 1, policy_version 55250 (0.0010) [2023-10-10 18:44:27,401][123614] Updated weights for policy 1, policy_version 55260 (0.0008) [2023-10-10 18:44:28,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 113278976. Throughput: 0: 1814.3, 1: 1816.2. Samples: 28322952. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:44:28,788][122664] Avg episode reward: [(0, '76.280'), (1, '55.200')] [2023-10-10 18:44:29,918][123582] Updated weights for policy 0, policy_version 55363 (0.0008) [2023-10-10 18:44:30,290][123582] Updated weights for policy 0, policy_version 55373 (0.0008) [2023-10-10 18:44:30,670][123582] Updated weights for policy 0, policy_version 55383 (0.0008) [2023-10-10 18:44:31,153][123614] Updated weights for policy 1, policy_version 55270 (0.0007) [2023-10-10 18:44:31,522][123614] Updated weights for policy 1, policy_version 55280 (0.0007) [2023-10-10 18:44:31,888][123614] Updated weights for policy 1, policy_version 55290 (0.0007) [2023-10-10 18:44:33,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 113344512. Throughput: 0: 1816.7, 1: 1807.3. Samples: 28344474. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:44:33,789][122664] Avg episode reward: [(0, '73.880'), (1, '54.760')] [2023-10-10 18:44:34,401][123582] Updated weights for policy 0, policy_version 55393 (0.0008) [2023-10-10 18:44:34,774][123582] Updated weights for policy 0, policy_version 55403 (0.0010) [2023-10-10 18:44:35,150][123582] Updated weights for policy 0, policy_version 55413 (0.0009) [2023-10-10 18:44:35,515][123582] Updated weights for policy 0, policy_version 55423 (0.0008) [2023-10-10 18:44:35,672][123614] Updated weights for policy 1, policy_version 55300 (0.0007) [2023-10-10 18:44:36,070][123614] Updated weights for policy 1, policy_version 55310 (0.0007) [2023-10-10 18:44:36,440][123614] Updated weights for policy 1, policy_version 55320 (0.0007) [2023-10-10 18:44:38,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 113410048. Throughput: 0: 1820.0, 1: 1818.4. Samples: 28367364. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:44:38,788][122664] Avg episode reward: [(0, '68.430'), (1, '55.860')] [2023-10-10 18:44:39,183][123582] Updated weights for policy 0, policy_version 55433 (0.0011) [2023-10-10 18:44:39,551][123582] Updated weights for policy 0, policy_version 55443 (0.0009) [2023-10-10 18:44:39,922][123582] Updated weights for policy 0, policy_version 55453 (0.0009) [2023-10-10 18:44:39,971][123614] Updated weights for policy 1, policy_version 55330 (0.0007) [2023-10-10 18:44:40,341][123614] Updated weights for policy 1, policy_version 55340 (0.0007) [2023-10-10 18:44:40,707][123614] Updated weights for policy 1, policy_version 55350 (0.0007) [2023-10-10 18:44:41,071][123614] Updated weights for policy 1, policy_version 55360 (0.0008) [2023-10-10 18:44:43,701][123582] Updated weights for policy 0, policy_version 55463 (0.0010) [2023-10-10 18:44:43,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 113475584. Throughput: 0: 1813.9, 1: 1820.8. Samples: 28377268. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:44:43,789][122664] Avg episode reward: [(0, '70.810'), (1, '55.060')] [2023-10-10 18:44:44,071][123582] Updated weights for policy 0, policy_version 55473 (0.0007) [2023-10-10 18:44:44,436][123582] Updated weights for policy 0, policy_version 55483 (0.0008) [2023-10-10 18:44:44,821][123614] Updated weights for policy 1, policy_version 55370 (0.0009) [2023-10-10 18:44:45,194][123614] Updated weights for policy 1, policy_version 55380 (0.0009) [2023-10-10 18:44:45,558][123614] Updated weights for policy 1, policy_version 55390 (0.0007) [2023-10-10 18:44:47,970][123582] Updated weights for policy 0, policy_version 55493 (0.0008) [2023-10-10 18:44:48,342][123582] Updated weights for policy 0, policy_version 55503 (0.0007) [2023-10-10 18:44:48,725][123582] Updated weights for policy 0, policy_version 55513 (0.0010) [2023-10-10 18:44:48,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 113541120. Throughput: 0: 1818.3, 1: 1824.0. Samples: 28400256. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:44:48,788][122664] Avg episode reward: [(0, '69.640'), (1, '56.060')] [2023-10-10 18:44:49,136][123614] Updated weights for policy 1, policy_version 55400 (0.0007) [2023-10-10 18:44:49,499][123614] Updated weights for policy 1, policy_version 55410 (0.0008) [2023-10-10 18:44:49,872][123614] Updated weights for policy 1, policy_version 55420 (0.0007) [2023-10-10 18:44:52,477][123582] Updated weights for policy 0, policy_version 55523 (0.0008) [2023-10-10 18:44:52,852][123582] Updated weights for policy 0, policy_version 55533 (0.0009) [2023-10-10 18:44:53,218][123582] Updated weights for policy 0, policy_version 55543 (0.0010) [2023-10-10 18:44:53,481][123614] Updated weights for policy 1, policy_version 55430 (0.0008) [2023-10-10 18:44:53,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 113639424. Throughput: 0: 1817.8, 1: 1817.5. Samples: 28420960. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 18:44:53,789][122664] Avg episode reward: [(0, '69.580'), (1, '55.990')] [2023-10-10 18:44:53,839][123614] Updated weights for policy 1, policy_version 55440 (0.0009) [2023-10-10 18:44:54,213][123614] Updated weights for policy 1, policy_version 55450 (0.0011) [2023-10-10 18:44:56,949][123582] Updated weights for policy 0, policy_version 55553 (0.0009) [2023-10-10 18:44:57,317][123582] Updated weights for policy 0, policy_version 55563 (0.0008) [2023-10-10 18:44:57,701][123582] Updated weights for policy 0, policy_version 55573 (0.0009) [2023-10-10 18:44:57,993][123614] Updated weights for policy 1, policy_version 55460 (0.0008) [2023-10-10 18:44:58,064][123582] Updated weights for policy 0, policy_version 55583 (0.0007) [2023-10-10 18:44:58,365][123614] Updated weights for policy 1, policy_version 55470 (0.0008) [2023-10-10 18:44:58,740][123614] Updated weights for policy 1, policy_version 55480 (0.0008) [2023-10-10 18:44:58,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 113704960. Throughput: 0: 1807.7, 1: 1818.5. Samples: 28432512. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 18:44:58,789][122664] Avg episode reward: [(0, '64.170'), (1, '59.270')] [2023-10-10 18:45:01,860][123582] Updated weights for policy 0, policy_version 55593 (0.0009) [2023-10-10 18:45:02,243][123582] Updated weights for policy 0, policy_version 55603 (0.0008) [2023-10-10 18:45:02,468][123614] Updated weights for policy 1, policy_version 55490 (0.0009) [2023-10-10 18:45:02,605][123582] Updated weights for policy 0, policy_version 55613 (0.0010) [2023-10-10 18:45:02,828][123614] Updated weights for policy 1, policy_version 55500 (0.0008) [2023-10-10 18:45:03,202][123614] Updated weights for policy 1, policy_version 55510 (0.0008) [2023-10-10 18:45:03,565][123614] Updated weights for policy 1, policy_version 55520 (0.0008) [2023-10-10 18:45:03,788][122664] Fps is (10 sec: 16384.4, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 113803264. Throughput: 0: 1813.0, 1: 1811.6. Samples: 28453518. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 18:45:03,789][122664] Avg episode reward: [(0, '62.470'), (1, '62.260')] [2023-10-10 18:45:06,347][123582] Updated weights for policy 0, policy_version 55623 (0.0008) [2023-10-10 18:45:06,720][123582] Updated weights for policy 0, policy_version 55633 (0.0008) [2023-10-10 18:45:07,085][123582] Updated weights for policy 0, policy_version 55643 (0.0008) [2023-10-10 18:45:07,222][123614] Updated weights for policy 1, policy_version 55530 (0.0008) [2023-10-10 18:45:07,584][123614] Updated weights for policy 1, policy_version 55540 (0.0009) [2023-10-10 18:45:07,960][123614] Updated weights for policy 1, policy_version 55550 (0.0008) [2023-10-10 18:45:08,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 113868800. Throughput: 0: 1805.4, 1: 1813.8. Samples: 28474876. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 18:45:08,789][122664] Avg episode reward: [(0, '59.550'), (1, '62.160')] [2023-10-10 18:45:10,781][123582] Updated weights for policy 0, policy_version 55653 (0.0008) [2023-10-10 18:45:11,159][123582] Updated weights for policy 0, policy_version 55663 (0.0008) [2023-10-10 18:45:11,527][123582] Updated weights for policy 0, policy_version 55673 (0.0010) [2023-10-10 18:45:11,790][123614] Updated weights for policy 1, policy_version 55560 (0.0007) [2023-10-10 18:45:12,157][123614] Updated weights for policy 1, policy_version 55570 (0.0007) [2023-10-10 18:45:12,528][123614] Updated weights for policy 1, policy_version 55580 (0.0007) [2023-10-10 18:45:13,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 113934336. Throughput: 0: 1817.7, 1: 1814.2. Samples: 28486388. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 18:45:13,789][122664] Avg episode reward: [(0, '57.040'), (1, '62.850')] [2023-10-10 18:45:15,169][123582] Updated weights for policy 0, policy_version 55683 (0.0007) [2023-10-10 18:45:15,547][123582] Updated weights for policy 0, policy_version 55693 (0.0008) [2023-10-10 18:45:15,923][123582] Updated weights for policy 0, policy_version 55703 (0.0010) [2023-10-10 18:45:16,146][123614] Updated weights for policy 1, policy_version 55590 (0.0008) [2023-10-10 18:45:16,512][123614] Updated weights for policy 1, policy_version 55600 (0.0007) [2023-10-10 18:45:16,872][123614] Updated weights for policy 1, policy_version 55610 (0.0008) [2023-10-10 18:45:18,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 113999872. Throughput: 0: 1810.2, 1: 1812.4. Samples: 28507492. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 18:45:18,789][122664] Avg episode reward: [(0, '54.130'), (1, '65.590')] [2023-10-10 18:45:19,635][123582] Updated weights for policy 0, policy_version 55713 (0.0009) [2023-10-10 18:45:19,998][123582] Updated weights for policy 0, policy_version 55723 (0.0010) [2023-10-10 18:45:20,377][123582] Updated weights for policy 0, policy_version 55733 (0.0008) [2023-10-10 18:45:20,729][123614] Updated weights for policy 1, policy_version 55620 (0.0010) [2023-10-10 18:45:20,751][123582] Updated weights for policy 0, policy_version 55743 (0.0008) [2023-10-10 18:45:21,104][123614] Updated weights for policy 1, policy_version 55630 (0.0008) [2023-10-10 18:45:21,475][123614] Updated weights for policy 1, policy_version 55640 (0.0008) [2023-10-10 18:45:23,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 114065408. Throughput: 0: 1806.8, 1: 1802.7. Samples: 28529794. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 18:45:23,789][122664] Avg episode reward: [(0, '55.440'), (1, '63.500')] [2023-10-10 18:45:24,587][123582] Updated weights for policy 0, policy_version 55753 (0.0010) [2023-10-10 18:45:24,949][123582] Updated weights for policy 0, policy_version 55763 (0.0008) [2023-10-10 18:45:25,198][123614] Updated weights for policy 1, policy_version 55650 (0.0009) [2023-10-10 18:45:25,318][123582] Updated weights for policy 0, policy_version 55773 (0.0008) [2023-10-10 18:45:25,565][123614] Updated weights for policy 1, policy_version 55660 (0.0008) [2023-10-10 18:45:25,934][123614] Updated weights for policy 1, policy_version 55670 (0.0010) [2023-10-10 18:45:26,309][123614] Updated weights for policy 1, policy_version 55680 (0.0008) [2023-10-10 18:45:28,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 114130944. Throughput: 0: 1808.2, 1: 1804.2. Samples: 28539824. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 18:45:28,788][122664] Avg episode reward: [(0, '60.430'), (1, '64.100')] [2023-10-10 18:45:28,934][123582] Updated weights for policy 0, policy_version 55783 (0.0009) [2023-10-10 18:45:29,303][123582] Updated weights for policy 0, policy_version 55793 (0.0008) [2023-10-10 18:45:29,673][123582] Updated weights for policy 0, policy_version 55803 (0.0009) [2023-10-10 18:45:30,129][123614] Updated weights for policy 1, policy_version 55690 (0.0009) [2023-10-10 18:45:30,493][123614] Updated weights for policy 1, policy_version 55700 (0.0008) [2023-10-10 18:45:30,867][123614] Updated weights for policy 1, policy_version 55710 (0.0009) [2023-10-10 18:45:33,391][123582] Updated weights for policy 0, policy_version 55813 (0.0009) [2023-10-10 18:45:33,755][123582] Updated weights for policy 0, policy_version 55823 (0.0008) [2023-10-10 18:45:33,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 114196480. Throughput: 0: 1810.2, 1: 1795.4. Samples: 28562508. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 18:45:33,789][122664] Avg episode reward: [(0, '60.840'), (1, '64.270')] [2023-10-10 18:45:34,135][123582] Updated weights for policy 0, policy_version 55833 (0.0010) [2023-10-10 18:45:34,631][123614] Updated weights for policy 1, policy_version 55720 (0.0008) [2023-10-10 18:45:35,007][123614] Updated weights for policy 1, policy_version 55730 (0.0007) [2023-10-10 18:45:35,375][123614] Updated weights for policy 1, policy_version 55740 (0.0008) [2023-10-10 18:45:37,818][123582] Updated weights for policy 0, policy_version 55843 (0.0008) [2023-10-10 18:45:38,186][123582] Updated weights for policy 0, policy_version 55853 (0.0009) [2023-10-10 18:45:38,564][123582] Updated weights for policy 0, policy_version 55863 (0.0011) [2023-10-10 18:45:38,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 114262016. Throughput: 0: 1819.2, 1: 1809.5. Samples: 28584250. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 18:45:38,789][122664] Avg episode reward: [(0, '60.620'), (1, '63.900')] [2023-10-10 18:45:38,995][123614] Updated weights for policy 1, policy_version 55750 (0.0008) [2023-10-10 18:45:39,364][123614] Updated weights for policy 1, policy_version 55760 (0.0008) [2023-10-10 18:45:39,736][123614] Updated weights for policy 1, policy_version 55770 (0.0009) [2023-10-10 18:45:42,256][123582] Updated weights for policy 0, policy_version 55873 (0.0008) [2023-10-10 18:45:42,627][123582] Updated weights for policy 0, policy_version 55883 (0.0008) [2023-10-10 18:45:43,003][123582] Updated weights for policy 0, policy_version 55893 (0.0011) [2023-10-10 18:45:43,373][123582] Updated weights for policy 0, policy_version 55903 (0.0008) [2023-10-10 18:45:43,374][123614] Updated weights for policy 1, policy_version 55780 (0.0008) [2023-10-10 18:45:43,737][123614] Updated weights for policy 1, policy_version 55790 (0.0007) [2023-10-10 18:45:43,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 114360320. Throughput: 0: 1809.8, 1: 1800.8. Samples: 28594988. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 18:45:43,789][122664] Avg episode reward: [(0, '62.430'), (1, '68.350')] [2023-10-10 18:45:44,109][123614] Updated weights for policy 1, policy_version 55800 (0.0008) [2023-10-10 18:45:46,961][123582] Updated weights for policy 0, policy_version 55913 (0.0008) [2023-10-10 18:45:47,342][123582] Updated weights for policy 0, policy_version 55923 (0.0009) [2023-10-10 18:45:47,713][123582] Updated weights for policy 0, policy_version 55933 (0.0008) [2023-10-10 18:45:47,940][123614] Updated weights for policy 1, policy_version 55810 (0.0007) [2023-10-10 18:45:48,309][123614] Updated weights for policy 1, policy_version 55820 (0.0007) [2023-10-10 18:45:48,682][123614] Updated weights for policy 1, policy_version 55830 (0.0008) [2023-10-10 18:45:48,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 114425856. Throughput: 0: 1815.2, 1: 1815.7. Samples: 28616906. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 18:45:48,789][122664] Avg episode reward: [(0, '64.000'), (1, '68.880')] [2023-10-10 18:45:49,059][123614] Updated weights for policy 1, policy_version 55840 (0.0009) [2023-10-10 18:45:51,545][123582] Updated weights for policy 0, policy_version 55943 (0.0008) [2023-10-10 18:45:51,920][123582] Updated weights for policy 0, policy_version 55953 (0.0007) [2023-10-10 18:45:52,301][123582] Updated weights for policy 0, policy_version 55963 (0.0009) [2023-10-10 18:45:52,831][123614] Updated weights for policy 1, policy_version 55850 (0.0008) [2023-10-10 18:45:53,191][123614] Updated weights for policy 1, policy_version 55860 (0.0009) [2023-10-10 18:45:53,555][123614] Updated weights for policy 1, policy_version 55870 (0.0011) [2023-10-10 18:45:53,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 114524160. Throughput: 0: 1803.9, 1: 1809.1. Samples: 28637466. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 18:45:53,789][122664] Avg episode reward: [(0, '68.310'), (1, '67.410')] [2023-10-10 18:45:53,801][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000055872_57212928.pth... [2023-10-10 18:45:53,801][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000055968_57311232.pth... [2023-10-10 18:45:53,838][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000054272_55574528.pth [2023-10-10 18:45:53,841][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000054176_55476224.pth [2023-10-10 18:45:56,173][123582] Updated weights for policy 0, policy_version 55973 (0.0009) [2023-10-10 18:45:56,549][123582] Updated weights for policy 0, policy_version 55983 (0.0008) [2023-10-10 18:45:56,931][123582] Updated weights for policy 0, policy_version 55993 (0.0008) [2023-10-10 18:45:57,276][123614] Updated weights for policy 1, policy_version 55880 (0.0007) [2023-10-10 18:45:57,643][123614] Updated weights for policy 1, policy_version 55890 (0.0007) [2023-10-10 18:45:58,008][123614] Updated weights for policy 1, policy_version 55900 (0.0010) [2023-10-10 18:45:58,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 114589696. Throughput: 0: 1814.7, 1: 1816.0. Samples: 28649768. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 18:45:58,789][122664] Avg episode reward: [(0, '68.600'), (1, '69.650')] [2023-10-10 18:46:00,619][123582] Updated weights for policy 0, policy_version 56003 (0.0008) [2023-10-10 18:46:00,994][123582] Updated weights for policy 0, policy_version 56013 (0.0009) [2023-10-10 18:46:01,370][123582] Updated weights for policy 0, policy_version 56023 (0.0010) [2023-10-10 18:46:01,659][123614] Updated weights for policy 1, policy_version 55910 (0.0009) [2023-10-10 18:46:02,031][123614] Updated weights for policy 1, policy_version 55920 (0.0008) [2023-10-10 18:46:02,399][123614] Updated weights for policy 1, policy_version 55930 (0.0010) [2023-10-10 18:46:03,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 114655232. Throughput: 0: 1802.0, 1: 1810.2. Samples: 28670042. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:46:03,789][122664] Avg episode reward: [(0, '67.020'), (1, '65.560')] [2023-10-10 18:46:04,944][123582] Updated weights for policy 0, policy_version 56033 (0.0008) [2023-10-10 18:46:05,312][123582] Updated weights for policy 0, policy_version 56043 (0.0008) [2023-10-10 18:46:05,686][123582] Updated weights for policy 0, policy_version 56053 (0.0007) [2023-10-10 18:46:06,061][123582] Updated weights for policy 0, policy_version 56063 (0.0007) [2023-10-10 18:46:06,103][123614] Updated weights for policy 1, policy_version 55940 (0.0010) [2023-10-10 18:46:06,480][123614] Updated weights for policy 1, policy_version 55950 (0.0010) [2023-10-10 18:46:06,855][123614] Updated weights for policy 1, policy_version 55960 (0.0010) [2023-10-10 18:46:08,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 114720768. Throughput: 0: 1808.8, 1: 1818.8. Samples: 28693036. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:46:08,788][122664] Avg episode reward: [(0, '67.800'), (1, '62.920')] [2023-10-10 18:46:09,702][123582] Updated weights for policy 0, policy_version 56073 (0.0008) [2023-10-10 18:46:10,078][123582] Updated weights for policy 0, policy_version 56083 (0.0007) [2023-10-10 18:46:10,450][123614] Updated weights for policy 1, policy_version 55970 (0.0009) [2023-10-10 18:46:10,454][123582] Updated weights for policy 0, policy_version 56093 (0.0008) [2023-10-10 18:46:10,821][123614] Updated weights for policy 1, policy_version 55980 (0.0009) [2023-10-10 18:46:11,183][123614] Updated weights for policy 1, policy_version 55990 (0.0009) [2023-10-10 18:46:11,563][123614] Updated weights for policy 1, policy_version 56000 (0.0007) [2023-10-10 18:46:13,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 114786304. Throughput: 0: 1809.7, 1: 1816.5. Samples: 28703006. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:46:13,788][122664] Avg episode reward: [(0, '66.910'), (1, '66.410')] [2023-10-10 18:46:14,029][123582] Updated weights for policy 0, policy_version 56103 (0.0008) [2023-10-10 18:46:14,401][123582] Updated weights for policy 0, policy_version 56113 (0.0008) [2023-10-10 18:46:14,778][123582] Updated weights for policy 0, policy_version 56123 (0.0007) [2023-10-10 18:46:15,394][123614] Updated weights for policy 1, policy_version 56010 (0.0008) [2023-10-10 18:46:15,761][123614] Updated weights for policy 1, policy_version 56020 (0.0009) [2023-10-10 18:46:16,131][123614] Updated weights for policy 1, policy_version 56030 (0.0007) [2023-10-10 18:46:18,306][123582] Updated weights for policy 0, policy_version 56133 (0.0010) [2023-10-10 18:46:18,671][123582] Updated weights for policy 0, policy_version 56143 (0.0010) [2023-10-10 18:46:18,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 114851840. Throughput: 0: 1814.3, 1: 1820.8. Samples: 28726088. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:46:18,788][122664] Avg episode reward: [(0, '69.420'), (1, '68.460')] [2023-10-10 18:46:19,041][123582] Updated weights for policy 0, policy_version 56153 (0.0007) [2023-10-10 18:46:19,657][123614] Updated weights for policy 1, policy_version 56040 (0.0010) [2023-10-10 18:46:20,026][123614] Updated weights for policy 1, policy_version 56050 (0.0009) [2023-10-10 18:46:20,400][123614] Updated weights for policy 1, policy_version 56060 (0.0007) [2023-10-10 18:46:22,755][123582] Updated weights for policy 0, policy_version 56163 (0.0008) [2023-10-10 18:46:23,135][123582] Updated weights for policy 0, policy_version 56173 (0.0009) [2023-10-10 18:46:23,508][123582] Updated weights for policy 0, policy_version 56183 (0.0010) [2023-10-10 18:46:23,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 114917376. Throughput: 0: 1815.5, 1: 1818.0. Samples: 28747760. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:46:23,788][122664] Avg episode reward: [(0, '70.290'), (1, '67.140')] [2023-10-10 18:46:24,252][123614] Updated weights for policy 1, policy_version 56070 (0.0009) [2023-10-10 18:46:24,619][123614] Updated weights for policy 1, policy_version 56080 (0.0011) [2023-10-10 18:46:24,984][123614] Updated weights for policy 1, policy_version 56090 (0.0008) [2023-10-10 18:46:27,229][123582] Updated weights for policy 0, policy_version 56193 (0.0007) [2023-10-10 18:46:27,601][123582] Updated weights for policy 0, policy_version 56203 (0.0008) [2023-10-10 18:46:27,976][123582] Updated weights for policy 0, policy_version 56213 (0.0007) [2023-10-10 18:46:28,346][123582] Updated weights for policy 0, policy_version 56223 (0.0008) [2023-10-10 18:46:28,653][123614] Updated weights for policy 1, policy_version 56100 (0.0008) [2023-10-10 18:46:28,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 115015680. Throughput: 0: 1821.2, 1: 1816.8. Samples: 28758696. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:46:28,788][122664] Avg episode reward: [(0, '77.050'), (1, '67.300')] [2023-10-10 18:46:29,020][123614] Updated weights for policy 1, policy_version 56110 (0.0008) [2023-10-10 18:46:29,396][123614] Updated weights for policy 1, policy_version 56120 (0.0009) [2023-10-10 18:46:32,216][123582] Updated weights for policy 0, policy_version 56233 (0.0009) [2023-10-10 18:46:32,578][123582] Updated weights for policy 0, policy_version 56243 (0.0008) [2023-10-10 18:46:32,953][123582] Updated weights for policy 0, policy_version 56253 (0.0008) [2023-10-10 18:46:33,099][123614] Updated weights for policy 1, policy_version 56130 (0.0008) [2023-10-10 18:46:33,473][123614] Updated weights for policy 1, policy_version 56140 (0.0008) [2023-10-10 18:46:33,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 115081216. Throughput: 0: 1817.8, 1: 1817.6. Samples: 28780498. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:46:33,788][122664] Avg episode reward: [(0, '76.370'), (1, '67.970')] [2023-10-10 18:46:33,836][123614] Updated weights for policy 1, policy_version 56150 (0.0008) [2023-10-10 18:46:34,214][123614] Updated weights for policy 1, policy_version 56160 (0.0007) [2023-10-10 18:46:36,745][123582] Updated weights for policy 0, policy_version 56263 (0.0007) [2023-10-10 18:46:37,110][123582] Updated weights for policy 0, policy_version 56273 (0.0009) [2023-10-10 18:46:37,482][123582] Updated weights for policy 0, policy_version 56283 (0.0009) [2023-10-10 18:46:37,818][123614] Updated weights for policy 1, policy_version 56170 (0.0007) [2023-10-10 18:46:38,186][123614] Updated weights for policy 1, policy_version 56180 (0.0007) [2023-10-10 18:46:38,557][123614] Updated weights for policy 1, policy_version 56190 (0.0007) [2023-10-10 18:46:38,788][122664] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 115179520. Throughput: 0: 1819.9, 1: 1817.4. Samples: 28801144. Policy #0 lag: (min: 11.0, avg: 11.0, max: 16.0) [2023-10-10 18:46:38,789][122664] Avg episode reward: [(0, '69.350'), (1, '64.080')] [2023-10-10 18:46:41,260][123582] Updated weights for policy 0, policy_version 56293 (0.0008) [2023-10-10 18:46:41,651][123582] Updated weights for policy 0, policy_version 56303 (0.0008) [2023-10-10 18:46:42,018][123582] Updated weights for policy 0, policy_version 56313 (0.0009) [2023-10-10 18:46:42,232][123614] Updated weights for policy 1, policy_version 56200 (0.0007) [2023-10-10 18:46:42,596][123614] Updated weights for policy 1, policy_version 56210 (0.0007) [2023-10-10 18:46:42,963][123614] Updated weights for policy 1, policy_version 56220 (0.0007) [2023-10-10 18:46:43,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 115245056. Throughput: 0: 1820.8, 1: 1818.2. Samples: 28813524. Policy #0 lag: (min: 11.0, avg: 11.0, max: 16.0) [2023-10-10 18:46:43,789][122664] Avg episode reward: [(0, '70.180'), (1, '68.410')] [2023-10-10 18:46:45,677][123582] Updated weights for policy 0, policy_version 56323 (0.0010) [2023-10-10 18:46:46,045][123582] Updated weights for policy 0, policy_version 56333 (0.0007) [2023-10-10 18:46:46,415][123582] Updated weights for policy 0, policy_version 56343 (0.0008) [2023-10-10 18:46:46,610][123614] Updated weights for policy 1, policy_version 56230 (0.0008) [2023-10-10 18:46:46,985][123614] Updated weights for policy 1, policy_version 56240 (0.0009) [2023-10-10 18:46:47,348][123614] Updated weights for policy 1, policy_version 56250 (0.0007) [2023-10-10 18:46:48,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 115310592. Throughput: 0: 1818.8, 1: 1822.3. Samples: 28833892. Policy #0 lag: (min: 11.0, avg: 11.0, max: 16.0) [2023-10-10 18:46:48,789][122664] Avg episode reward: [(0, '68.500'), (1, '70.380')] [2023-10-10 18:46:50,153][123582] Updated weights for policy 0, policy_version 56353 (0.0009) [2023-10-10 18:46:50,524][123582] Updated weights for policy 0, policy_version 56363 (0.0007) [2023-10-10 18:46:50,889][123582] Updated weights for policy 0, policy_version 56373 (0.0010) [2023-10-10 18:46:51,130][123614] Updated weights for policy 1, policy_version 56260 (0.0008) [2023-10-10 18:46:51,260][123582] Updated weights for policy 0, policy_version 56383 (0.0009) [2023-10-10 18:46:51,517][123614] Updated weights for policy 1, policy_version 56270 (0.0007) [2023-10-10 18:46:51,889][123614] Updated weights for policy 1, policy_version 56280 (0.0008) [2023-10-10 18:46:53,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 115376128. Throughput: 0: 1816.1, 1: 1814.4. Samples: 28856412. Policy #0 lag: (min: 11.0, avg: 11.0, max: 16.0) [2023-10-10 18:46:53,789][122664] Avg episode reward: [(0, '61.680'), (1, '64.790')] [2023-10-10 18:46:54,874][123582] Updated weights for policy 0, policy_version 56393 (0.0008) [2023-10-10 18:46:55,249][123582] Updated weights for policy 0, policy_version 56403 (0.0008) [2023-10-10 18:46:55,552][123614] Updated weights for policy 1, policy_version 56290 (0.0009) [2023-10-10 18:46:55,609][123582] Updated weights for policy 0, policy_version 56413 (0.0008) [2023-10-10 18:46:55,917][123614] Updated weights for policy 1, policy_version 56300 (0.0010) [2023-10-10 18:46:56,282][123614] Updated weights for policy 1, policy_version 56310 (0.0007) [2023-10-10 18:46:56,650][123614] Updated weights for policy 1, policy_version 56320 (0.0010) [2023-10-10 18:46:58,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 115441664. Throughput: 0: 1812.8, 1: 1814.8. Samples: 28866248. Policy #0 lag: (min: 11.0, avg: 11.0, max: 16.0) [2023-10-10 18:46:58,788][122664] Avg episode reward: [(0, '62.670'), (1, '66.360')] [2023-10-10 18:46:59,452][123582] Updated weights for policy 0, policy_version 56423 (0.0008) [2023-10-10 18:46:59,817][123582] Updated weights for policy 0, policy_version 56433 (0.0008) [2023-10-10 18:47:00,183][123582] Updated weights for policy 0, policy_version 56443 (0.0009) [2023-10-10 18:47:00,352][123614] Updated weights for policy 1, policy_version 56330 (0.0008) [2023-10-10 18:47:00,719][123614] Updated weights for policy 1, policy_version 56340 (0.0008) [2023-10-10 18:47:01,080][123614] Updated weights for policy 1, policy_version 56350 (0.0008) [2023-10-10 18:47:03,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 115507200. Throughput: 0: 1800.5, 1: 1814.9. Samples: 28888780. Policy #0 lag: (min: 11.0, avg: 11.0, max: 16.0) [2023-10-10 18:47:03,789][122664] Avg episode reward: [(0, '62.890'), (1, '65.750')] [2023-10-10 18:47:03,949][123582] Updated weights for policy 0, policy_version 56453 (0.0008) [2023-10-10 18:47:04,320][123582] Updated weights for policy 0, policy_version 56463 (0.0007) [2023-10-10 18:47:04,696][123582] Updated weights for policy 0, policy_version 56473 (0.0008) [2023-10-10 18:47:04,860][123614] Updated weights for policy 1, policy_version 56360 (0.0007) [2023-10-10 18:47:05,230][123614] Updated weights for policy 1, policy_version 56370 (0.0008) [2023-10-10 18:47:05,599][123614] Updated weights for policy 1, policy_version 56380 (0.0008) [2023-10-10 18:47:08,383][123582] Updated weights for policy 0, policy_version 56483 (0.0009) [2023-10-10 18:47:08,744][123582] Updated weights for policy 0, policy_version 56493 (0.0008) [2023-10-10 18:47:08,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 115572736. Throughput: 0: 1817.9, 1: 1817.6. Samples: 28911358. Policy #0 lag: (min: 11.0, avg: 11.0, max: 16.0) [2023-10-10 18:47:08,789][122664] Avg episode reward: [(0, '63.910'), (1, '62.970')] [2023-10-10 18:47:09,117][123582] Updated weights for policy 0, policy_version 56503 (0.0008) [2023-10-10 18:47:09,269][123614] Updated weights for policy 1, policy_version 56390 (0.0008) [2023-10-10 18:47:09,645][123614] Updated weights for policy 1, policy_version 56400 (0.0010) [2023-10-10 18:47:10,011][123614] Updated weights for policy 1, policy_version 56410 (0.0011) [2023-10-10 18:47:12,914][123582] Updated weights for policy 0, policy_version 56513 (0.0007) [2023-10-10 18:47:13,284][123582] Updated weights for policy 0, policy_version 56523 (0.0011) [2023-10-10 18:47:13,650][123582] Updated weights for policy 0, policy_version 56533 (0.0010) [2023-10-10 18:47:13,705][123614] Updated weights for policy 1, policy_version 56420 (0.0009) [2023-10-10 18:47:13,788][122664] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 115638272. Throughput: 0: 1798.8, 1: 1818.0. Samples: 28921448. Policy #0 lag: (min: 11.0, avg: 11.0, max: 16.0) [2023-10-10 18:47:13,788][122664] Avg episode reward: [(0, '61.030'), (1, '65.320')] [2023-10-10 18:47:14,021][123582] Updated weights for policy 0, policy_version 56543 (0.0007) [2023-10-10 18:47:14,074][123614] Updated weights for policy 1, policy_version 56430 (0.0007) [2023-10-10 18:47:14,443][123614] Updated weights for policy 1, policy_version 56440 (0.0007) [2023-10-10 18:47:17,682][123582] Updated weights for policy 0, policy_version 56553 (0.0007) [2023-10-10 18:47:18,054][123582] Updated weights for policy 0, policy_version 56563 (0.0010) [2023-10-10 18:47:18,198][123614] Updated weights for policy 1, policy_version 56450 (0.0008) [2023-10-10 18:47:18,421][123582] Updated weights for policy 0, policy_version 56573 (0.0008) [2023-10-10 18:47:18,561][123614] Updated weights for policy 1, policy_version 56460 (0.0008) [2023-10-10 18:47:18,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 115736576. Throughput: 0: 1814.7, 1: 1814.3. Samples: 28943804. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-10 18:47:18,788][122664] Avg episode reward: [(0, '63.980'), (1, '65.450')] [2023-10-10 18:47:18,930][123614] Updated weights for policy 1, policy_version 56470 (0.0010) [2023-10-10 18:47:19,302][123614] Updated weights for policy 1, policy_version 56480 (0.0008) [2023-10-10 18:47:22,019][123582] Updated weights for policy 0, policy_version 56583 (0.0009) [2023-10-10 18:47:22,389][123582] Updated weights for policy 0, policy_version 56593 (0.0008) [2023-10-10 18:47:22,766][123582] Updated weights for policy 0, policy_version 56603 (0.0008) [2023-10-10 18:47:23,031][123614] Updated weights for policy 1, policy_version 56490 (0.0007) [2023-10-10 18:47:23,395][123614] Updated weights for policy 1, policy_version 56500 (0.0011) [2023-10-10 18:47:23,761][123614] Updated weights for policy 1, policy_version 56510 (0.0009) [2023-10-10 18:47:23,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 115802112. Throughput: 0: 1805.7, 1: 1819.2. Samples: 28964266. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-10 18:47:23,789][122664] Avg episode reward: [(0, '64.350'), (1, '65.330')] [2023-10-10 18:47:26,502][123582] Updated weights for policy 0, policy_version 56613 (0.0007) [2023-10-10 18:47:26,896][123582] Updated weights for policy 0, policy_version 56623 (0.0009) [2023-10-10 18:47:27,217][123614] Updated weights for policy 1, policy_version 56520 (0.0008) [2023-10-10 18:47:27,266][123582] Updated weights for policy 0, policy_version 56633 (0.0009) [2023-10-10 18:47:27,579][123614] Updated weights for policy 1, policy_version 56530 (0.0010) [2023-10-10 18:47:27,942][123614] Updated weights for policy 1, policy_version 56540 (0.0009) [2023-10-10 18:47:28,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 115900416. Throughput: 0: 1813.2, 1: 1817.0. Samples: 28976884. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-10 18:47:28,789][122664] Avg episode reward: [(0, '66.800'), (1, '60.050')] [2023-10-10 18:47:30,947][123582] Updated weights for policy 0, policy_version 56643 (0.0007) [2023-10-10 18:47:31,316][123582] Updated weights for policy 0, policy_version 56653 (0.0009) [2023-10-10 18:47:31,687][123582] Updated weights for policy 0, policy_version 56663 (0.0010) [2023-10-10 18:47:31,786][123614] Updated weights for policy 1, policy_version 56550 (0.0008) [2023-10-10 18:47:32,143][123614] Updated weights for policy 1, policy_version 56560 (0.0008) [2023-10-10 18:47:32,519][123614] Updated weights for policy 1, policy_version 56570 (0.0008) [2023-10-10 18:47:33,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 115965952. Throughput: 0: 1806.4, 1: 1814.5. Samples: 28996832. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-10 18:47:33,789][122664] Avg episode reward: [(0, '65.380'), (1, '61.340')] [2023-10-10 18:47:35,308][123582] Updated weights for policy 0, policy_version 56673 (0.0008) [2023-10-10 18:47:35,677][123582] Updated weights for policy 0, policy_version 56683 (0.0009) [2023-10-10 18:47:36,049][123582] Updated weights for policy 0, policy_version 56693 (0.0007) [2023-10-10 18:47:36,201][123614] Updated weights for policy 1, policy_version 56580 (0.0008) [2023-10-10 18:47:36,422][123582] Updated weights for policy 0, policy_version 56703 (0.0008) [2023-10-10 18:47:36,582][123614] Updated weights for policy 1, policy_version 56590 (0.0009) [2023-10-10 18:47:36,949][123614] Updated weights for policy 1, policy_version 56600 (0.0009) [2023-10-10 18:47:38,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 116031488. Throughput: 0: 1808.0, 1: 1812.1. Samples: 29019314. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-10 18:47:38,788][122664] Avg episode reward: [(0, '63.220'), (1, '64.150')] [2023-10-10 18:47:40,142][123582] Updated weights for policy 0, policy_version 56713 (0.0010) [2023-10-10 18:47:40,518][123582] Updated weights for policy 0, policy_version 56723 (0.0009) [2023-10-10 18:47:40,827][123614] Updated weights for policy 1, policy_version 56610 (0.0008) [2023-10-10 18:47:40,892][123582] Updated weights for policy 0, policy_version 56733 (0.0009) [2023-10-10 18:47:41,197][123614] Updated weights for policy 1, policy_version 56620 (0.0008) [2023-10-10 18:47:41,572][123614] Updated weights for policy 1, policy_version 56630 (0.0008) [2023-10-10 18:47:41,938][123614] Updated weights for policy 1, policy_version 56640 (0.0009) [2023-10-10 18:47:43,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 116097024. Throughput: 0: 1806.6, 1: 1815.3. Samples: 29029236. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-10 18:47:43,788][122664] Avg episode reward: [(0, '66.010'), (1, '65.160')] [2023-10-10 18:47:44,620][123582] Updated weights for policy 0, policy_version 56743 (0.0010) [2023-10-10 18:47:45,003][123582] Updated weights for policy 0, policy_version 56753 (0.0010) [2023-10-10 18:47:45,369][123582] Updated weights for policy 0, policy_version 56763 (0.0009) [2023-10-10 18:47:45,753][123614] Updated weights for policy 1, policy_version 56650 (0.0008) [2023-10-10 18:47:46,130][123614] Updated weights for policy 1, policy_version 56660 (0.0008) [2023-10-10 18:47:46,494][123614] Updated weights for policy 1, policy_version 56670 (0.0008) [2023-10-10 18:47:48,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 116162560. Throughput: 0: 1809.5, 1: 1801.8. Samples: 29051290. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-10 18:47:48,789][122664] Avg episode reward: [(0, '64.550'), (1, '61.000')] [2023-10-10 18:47:49,071][123582] Updated weights for policy 0, policy_version 56773 (0.0009) [2023-10-10 18:47:49,443][123582] Updated weights for policy 0, policy_version 56783 (0.0008) [2023-10-10 18:47:49,820][123582] Updated weights for policy 0, policy_version 56793 (0.0009) [2023-10-10 18:47:50,247][123614] Updated weights for policy 1, policy_version 56680 (0.0008) [2023-10-10 18:47:50,613][123614] Updated weights for policy 1, policy_version 56690 (0.0009) [2023-10-10 18:47:50,986][123614] Updated weights for policy 1, policy_version 56700 (0.0008) [2023-10-10 18:47:53,569][123582] Updated weights for policy 0, policy_version 56803 (0.0008) [2023-10-10 18:47:53,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 116228096. Throughput: 0: 1813.1, 1: 1796.0. Samples: 29073766. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-10 18:47:53,788][122664] Avg episode reward: [(0, '67.640'), (1, '58.720')] [2023-10-10 18:47:53,796][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000056704_58064896.pth... [2023-10-10 18:47:53,838][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000055008_56328192.pth [2023-10-10 18:47:53,934][123582] Updated weights for policy 0, policy_version 56813 (0.0008) [2023-10-10 18:47:54,308][123582] Updated weights for policy 0, policy_version 56823 (0.0010) [2023-10-10 18:47:54,620][123614] Updated weights for policy 1, policy_version 56710 (0.0008) [2023-10-10 18:47:54,633][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000056832_58195968.pth... [2023-10-10 18:47:54,671][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000055136_56459264.pth [2023-10-10 18:47:54,982][123614] Updated weights for policy 1, policy_version 56720 (0.0009) [2023-10-10 18:47:55,356][123614] Updated weights for policy 1, policy_version 56730 (0.0008) [2023-10-10 18:47:58,045][123582] Updated weights for policy 0, policy_version 56833 (0.0009) [2023-10-10 18:47:58,414][123582] Updated weights for policy 0, policy_version 56843 (0.0008) [2023-10-10 18:47:58,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 116293632. Throughput: 0: 1809.1, 1: 1796.9. Samples: 29083716. Policy #0 lag: (min: 26.0, avg: 26.1, max: 33.0) [2023-10-10 18:47:58,788][122664] Avg episode reward: [(0, '69.330'), (1, '57.770')] [2023-10-10 18:47:58,790][123582] Updated weights for policy 0, policy_version 56853 (0.0011) [2023-10-10 18:47:59,098][123614] Updated weights for policy 1, policy_version 56740 (0.0009) [2023-10-10 18:47:59,161][123582] Updated weights for policy 0, policy_version 56863 (0.0010) [2023-10-10 18:47:59,466][123614] Updated weights for policy 1, policy_version 56750 (0.0007) [2023-10-10 18:47:59,846][123614] Updated weights for policy 1, policy_version 56760 (0.0010) [2023-10-10 18:48:02,849][123582] Updated weights for policy 0, policy_version 56873 (0.0008) [2023-10-10 18:48:03,221][123582] Updated weights for policy 0, policy_version 56883 (0.0009) [2023-10-10 18:48:03,535][123614] Updated weights for policy 1, policy_version 56770 (0.0010) [2023-10-10 18:48:03,597][123582] Updated weights for policy 0, policy_version 56893 (0.0009) [2023-10-10 18:48:03,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 116391936. Throughput: 0: 1817.2, 1: 1793.7. Samples: 29106296. Policy #0 lag: (min: 26.0, avg: 26.1, max: 33.0) [2023-10-10 18:48:03,789][122664] Avg episode reward: [(0, '67.550'), (1, '55.060')] [2023-10-10 18:48:03,901][123614] Updated weights for policy 1, policy_version 56780 (0.0007) [2023-10-10 18:48:04,269][123614] Updated weights for policy 1, policy_version 56790 (0.0007) [2023-10-10 18:48:04,628][123614] Updated weights for policy 1, policy_version 56800 (0.0010) [2023-10-10 18:48:07,238][123582] Updated weights for policy 0, policy_version 56903 (0.0011) [2023-10-10 18:48:07,614][123582] Updated weights for policy 0, policy_version 56913 (0.0010) [2023-10-10 18:48:07,981][123582] Updated weights for policy 0, policy_version 56923 (0.0009) [2023-10-10 18:48:08,316][123614] Updated weights for policy 1, policy_version 56810 (0.0008) [2023-10-10 18:48:08,680][123614] Updated weights for policy 1, policy_version 56820 (0.0010) [2023-10-10 18:48:08,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 116457472. Throughput: 0: 1807.7, 1: 1802.9. Samples: 29126746. Policy #0 lag: (min: 26.0, avg: 26.1, max: 33.0) [2023-10-10 18:48:08,789][122664] Avg episode reward: [(0, '62.060'), (1, '59.020')] [2023-10-10 18:48:09,046][123614] Updated weights for policy 1, policy_version 56830 (0.0010) [2023-10-10 18:48:11,710][123582] Updated weights for policy 0, policy_version 56933 (0.0007) [2023-10-10 18:48:12,106][123582] Updated weights for policy 0, policy_version 56943 (0.0007) [2023-10-10 18:48:12,471][123582] Updated weights for policy 0, policy_version 56953 (0.0009) [2023-10-10 18:48:12,661][123614] Updated weights for policy 1, policy_version 56840 (0.0008) [2023-10-10 18:48:13,035][123614] Updated weights for policy 1, policy_version 56850 (0.0009) [2023-10-10 18:48:13,406][123614] Updated weights for policy 1, policy_version 56860 (0.0010) [2023-10-10 18:48:13,788][122664] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 116555776. Throughput: 0: 1813.6, 1: 1792.1. Samples: 29139140. Policy #0 lag: (min: 26.0, avg: 26.1, max: 33.0) [2023-10-10 18:48:13,789][122664] Avg episode reward: [(0, '59.810'), (1, '58.590')] [2023-10-10 18:48:16,029][123582] Updated weights for policy 0, policy_version 56963 (0.0010) [2023-10-10 18:48:16,399][123582] Updated weights for policy 0, policy_version 56973 (0.0007) [2023-10-10 18:48:16,768][123582] Updated weights for policy 0, policy_version 56983 (0.0009) [2023-10-10 18:48:17,103][123614] Updated weights for policy 1, policy_version 56870 (0.0007) [2023-10-10 18:48:17,470][123614] Updated weights for policy 1, policy_version 56880 (0.0011) [2023-10-10 18:48:17,835][123614] Updated weights for policy 1, policy_version 56890 (0.0010) [2023-10-10 18:48:18,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 116621312. Throughput: 0: 1813.1, 1: 1803.4. Samples: 29159574. Policy #0 lag: (min: 26.0, avg: 26.1, max: 33.0) [2023-10-10 18:48:18,789][122664] Avg episode reward: [(0, '58.560'), (1, '59.240')] [2023-10-10 18:48:20,553][123582] Updated weights for policy 0, policy_version 56993 (0.0008) [2023-10-10 18:48:20,919][123582] Updated weights for policy 0, policy_version 57003 (0.0009) [2023-10-10 18:48:21,292][123582] Updated weights for policy 0, policy_version 57013 (0.0007) [2023-10-10 18:48:21,653][123582] Updated weights for policy 0, policy_version 57023 (0.0007) [2023-10-10 18:48:21,681][123614] Updated weights for policy 1, policy_version 56900 (0.0009) [2023-10-10 18:48:22,070][123614] Updated weights for policy 1, policy_version 56910 (0.0007) [2023-10-10 18:48:22,441][123614] Updated weights for policy 1, policy_version 56920 (0.0007) [2023-10-10 18:48:23,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 116686848. Throughput: 0: 1813.1, 1: 1796.2. Samples: 29181734. Policy #0 lag: (min: 26.0, avg: 26.1, max: 33.0) [2023-10-10 18:48:23,788][122664] Avg episode reward: [(0, '57.790'), (1, '58.210')] [2023-10-10 18:48:25,250][123582] Updated weights for policy 0, policy_version 57033 (0.0008) [2023-10-10 18:48:25,619][123582] Updated weights for policy 0, policy_version 57043 (0.0009) [2023-10-10 18:48:25,996][123582] Updated weights for policy 0, policy_version 57053 (0.0008) [2023-10-10 18:48:26,087][123614] Updated weights for policy 1, policy_version 56930 (0.0008) [2023-10-10 18:48:26,455][123614] Updated weights for policy 1, policy_version 56940 (0.0009) [2023-10-10 18:48:26,829][123614] Updated weights for policy 1, policy_version 56950 (0.0008) [2023-10-10 18:48:27,194][123614] Updated weights for policy 1, policy_version 56960 (0.0010) [2023-10-10 18:48:28,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 116752384. Throughput: 0: 1814.6, 1: 1813.0. Samples: 29192478. Policy #0 lag: (min: 26.0, avg: 26.1, max: 33.0) [2023-10-10 18:48:28,789][122664] Avg episode reward: [(0, '59.770'), (1, '61.850')] [2023-10-10 18:48:29,507][123582] Updated weights for policy 0, policy_version 57063 (0.0007) [2023-10-10 18:48:29,870][123582] Updated weights for policy 0, policy_version 57073 (0.0009) [2023-10-10 18:48:30,241][123582] Updated weights for policy 0, policy_version 57083 (0.0011) [2023-10-10 18:48:30,998][123614] Updated weights for policy 1, policy_version 56970 (0.0008) [2023-10-10 18:48:31,366][123614] Updated weights for policy 1, policy_version 56980 (0.0007) [2023-10-10 18:48:31,721][123614] Updated weights for policy 1, policy_version 56990 (0.0008) [2023-10-10 18:48:33,788][123582] Updated weights for policy 0, policy_version 57093 (0.0009) [2023-10-10 18:48:33,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 116817920. Throughput: 0: 1824.3, 1: 1811.6. Samples: 29214904. Policy #0 lag: (min: 26.0, avg: 26.1, max: 33.0) [2023-10-10 18:48:33,789][122664] Avg episode reward: [(0, '58.090'), (1, '60.280')] [2023-10-10 18:48:34,159][123582] Updated weights for policy 0, policy_version 57103 (0.0008) [2023-10-10 18:48:34,531][123582] Updated weights for policy 0, policy_version 57113 (0.0008) [2023-10-10 18:48:35,441][123614] Updated weights for policy 1, policy_version 57000 (0.0008) [2023-10-10 18:48:35,804][123614] Updated weights for policy 1, policy_version 57010 (0.0007) [2023-10-10 18:48:36,182][123614] Updated weights for policy 1, policy_version 57020 (0.0009) [2023-10-10 18:48:38,203][123582] Updated weights for policy 0, policy_version 57123 (0.0009) [2023-10-10 18:48:38,582][123582] Updated weights for policy 0, policy_version 57133 (0.0009) [2023-10-10 18:48:38,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 116883456. Throughput: 0: 1821.9, 1: 1817.3. Samples: 29237530. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-10 18:48:38,789][122664] Avg episode reward: [(0, '61.850'), (1, '62.210')] [2023-10-10 18:48:38,958][123582] Updated weights for policy 0, policy_version 57143 (0.0009) [2023-10-10 18:48:39,868][123614] Updated weights for policy 1, policy_version 57030 (0.0008) [2023-10-10 18:48:40,228][123614] Updated weights for policy 1, policy_version 57040 (0.0010) [2023-10-10 18:48:40,605][123614] Updated weights for policy 1, policy_version 57050 (0.0011) [2023-10-10 18:48:42,601][123582] Updated weights for policy 0, policy_version 57153 (0.0009) [2023-10-10 18:48:42,978][123582] Updated weights for policy 0, policy_version 57163 (0.0010) [2023-10-10 18:48:43,336][123582] Updated weights for policy 0, policy_version 57173 (0.0011) [2023-10-10 18:48:43,709][123582] Updated weights for policy 0, policy_version 57183 (0.0009) [2023-10-10 18:48:43,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 116981760. Throughput: 0: 1831.0, 1: 1814.4. Samples: 29247756. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-10 18:48:43,789][122664] Avg episode reward: [(0, '63.630'), (1, '65.390')] [2023-10-10 18:48:44,266][123614] Updated weights for policy 1, policy_version 57060 (0.0010) [2023-10-10 18:48:44,640][123614] Updated weights for policy 1, policy_version 57070 (0.0007) [2023-10-10 18:48:45,011][123614] Updated weights for policy 1, policy_version 57080 (0.0007) [2023-10-10 18:48:47,467][123582] Updated weights for policy 0, policy_version 57193 (0.0008) [2023-10-10 18:48:47,842][123582] Updated weights for policy 0, policy_version 57203 (0.0008) [2023-10-10 18:48:48,212][123582] Updated weights for policy 0, policy_version 57213 (0.0007) [2023-10-10 18:48:48,550][123614] Updated weights for policy 1, policy_version 57090 (0.0009) [2023-10-10 18:48:48,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 117047296. Throughput: 0: 1822.4, 1: 1825.6. Samples: 29270458. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-10 18:48:48,789][122664] Avg episode reward: [(0, '63.960'), (1, '67.360')] [2023-10-10 18:48:48,917][123614] Updated weights for policy 1, policy_version 57100 (0.0007) [2023-10-10 18:48:49,292][123614] Updated weights for policy 1, policy_version 57110 (0.0008) [2023-10-10 18:48:49,658][123614] Updated weights for policy 1, policy_version 57120 (0.0008) [2023-10-10 18:48:51,973][123582] Updated weights for policy 0, policy_version 57223 (0.0009) [2023-10-10 18:48:52,343][123582] Updated weights for policy 0, policy_version 57233 (0.0010) [2023-10-10 18:48:52,721][123582] Updated weights for policy 0, policy_version 57243 (0.0008) [2023-10-10 18:48:53,234][123614] Updated weights for policy 1, policy_version 57130 (0.0008) [2023-10-10 18:48:53,609][123614] Updated weights for policy 1, policy_version 57140 (0.0007) [2023-10-10 18:48:53,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 117112832. Throughput: 0: 1827.4, 1: 1824.6. Samples: 29291088. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-10 18:48:53,789][122664] Avg episode reward: [(0, '59.680'), (1, '64.050')] [2023-10-10 18:48:53,976][123614] Updated weights for policy 1, policy_version 57150 (0.0008) [2023-10-10 18:48:56,401][123582] Updated weights for policy 0, policy_version 57253 (0.0009) [2023-10-10 18:48:56,770][123582] Updated weights for policy 0, policy_version 57263 (0.0008) [2023-10-10 18:48:57,152][123582] Updated weights for policy 0, policy_version 57273 (0.0012) [2023-10-10 18:48:57,640][123614] Updated weights for policy 1, policy_version 57160 (0.0008) [2023-10-10 18:48:58,009][123614] Updated weights for policy 1, policy_version 57170 (0.0010) [2023-10-10 18:48:58,377][123614] Updated weights for policy 1, policy_version 57180 (0.0008) [2023-10-10 18:48:58,788][122664] Fps is (10 sec: 16384.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 117211136. Throughput: 0: 1823.6, 1: 1830.4. Samples: 29303570. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-10 18:48:58,789][122664] Avg episode reward: [(0, '55.970'), (1, '62.360')] [2023-10-10 18:49:01,040][123582] Updated weights for policy 0, policy_version 57283 (0.0009) [2023-10-10 18:49:01,423][123582] Updated weights for policy 0, policy_version 57293 (0.0009) [2023-10-10 18:49:01,796][123582] Updated weights for policy 0, policy_version 57303 (0.0009) [2023-10-10 18:49:02,104][123614] Updated weights for policy 1, policy_version 57190 (0.0008) [2023-10-10 18:49:02,469][123614] Updated weights for policy 1, policy_version 57200 (0.0008) [2023-10-10 18:49:02,841][123614] Updated weights for policy 1, policy_version 57210 (0.0007) [2023-10-10 18:49:03,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 117276672. Throughput: 0: 1823.1, 1: 1828.1. Samples: 29323876. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-10 18:49:03,789][122664] Avg episode reward: [(0, '58.820'), (1, '65.740')] [2023-10-10 18:49:05,483][123582] Updated weights for policy 0, policy_version 57313 (0.0008) [2023-10-10 18:49:05,896][123582] Updated weights for policy 0, policy_version 57323 (0.0010) [2023-10-10 18:49:06,259][123582] Updated weights for policy 0, policy_version 57333 (0.0007) [2023-10-10 18:49:06,581][123614] Updated weights for policy 1, policy_version 57220 (0.0007) [2023-10-10 18:49:06,635][123582] Updated weights for policy 0, policy_version 57343 (0.0010) [2023-10-10 18:49:06,970][123614] Updated weights for policy 1, policy_version 57230 (0.0007) [2023-10-10 18:49:07,335][123614] Updated weights for policy 1, policy_version 57240 (0.0008) [2023-10-10 18:49:08,788][122664] Fps is (10 sec: 13106.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 117342208. Throughput: 0: 1817.8, 1: 1824.8. Samples: 29345650. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-10 18:49:08,789][122664] Avg episode reward: [(0, '60.430'), (1, '70.930')] [2023-10-10 18:49:10,442][123582] Updated weights for policy 0, policy_version 57353 (0.0007) [2023-10-10 18:49:10,813][123582] Updated weights for policy 0, policy_version 57363 (0.0007) [2023-10-10 18:49:11,097][123614] Updated weights for policy 1, policy_version 57250 (0.0010) [2023-10-10 18:49:11,177][123582] Updated weights for policy 0, policy_version 57373 (0.0008) [2023-10-10 18:49:11,467][123614] Updated weights for policy 1, policy_version 57260 (0.0007) [2023-10-10 18:49:11,848][123614] Updated weights for policy 1, policy_version 57270 (0.0008) [2023-10-10 18:49:12,214][123614] Updated weights for policy 1, policy_version 57280 (0.0009) [2023-10-10 18:49:13,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 117407744. Throughput: 0: 1818.8, 1: 1819.4. Samples: 29356196. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:49:13,789][122664] Avg episode reward: [(0, '60.850'), (1, '71.580')] [2023-10-10 18:49:14,864][123582] Updated weights for policy 0, policy_version 57383 (0.0008) [2023-10-10 18:49:15,230][123582] Updated weights for policy 0, policy_version 57393 (0.0008) [2023-10-10 18:49:15,609][123582] Updated weights for policy 0, policy_version 57403 (0.0010) [2023-10-10 18:49:16,144][123614] Updated weights for policy 1, policy_version 57290 (0.0008) [2023-10-10 18:49:16,516][123614] Updated weights for policy 1, policy_version 57300 (0.0008) [2023-10-10 18:49:16,886][123614] Updated weights for policy 1, policy_version 57310 (0.0009) [2023-10-10 18:49:18,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 117473280. Throughput: 0: 1813.3, 1: 1808.3. Samples: 29377878. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:49:18,789][122664] Avg episode reward: [(0, '63.280'), (1, '75.770')] [2023-10-10 18:49:19,330][123582] Updated weights for policy 0, policy_version 57413 (0.0007) [2023-10-10 18:49:19,704][123582] Updated weights for policy 0, policy_version 57423 (0.0008) [2023-10-10 18:49:20,085][123582] Updated weights for policy 0, policy_version 57433 (0.0010) [2023-10-10 18:49:20,598][123614] Updated weights for policy 1, policy_version 57320 (0.0010) [2023-10-10 18:49:20,978][123614] Updated weights for policy 1, policy_version 57330 (0.0012) [2023-10-10 18:49:21,342][123614] Updated weights for policy 1, policy_version 57340 (0.0010) [2023-10-10 18:49:23,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 117538816. Throughput: 0: 1819.1, 1: 1803.4. Samples: 29400540. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:49:23,788][122664] Avg episode reward: [(0, '64.190'), (1, '79.640')] [2023-10-10 18:49:23,800][123582] Updated weights for policy 0, policy_version 57443 (0.0009) [2023-10-10 18:49:24,168][123582] Updated weights for policy 0, policy_version 57453 (0.0010) [2023-10-10 18:49:24,550][123582] Updated weights for policy 0, policy_version 57463 (0.0011) [2023-10-10 18:49:25,063][123614] Updated weights for policy 1, policy_version 57350 (0.0009) [2023-10-10 18:49:25,433][123614] Updated weights for policy 1, policy_version 57360 (0.0008) [2023-10-10 18:49:25,798][123614] Updated weights for policy 1, policy_version 57370 (0.0010) [2023-10-10 18:49:28,204][123582] Updated weights for policy 0, policy_version 57473 (0.0009) [2023-10-10 18:49:28,577][123582] Updated weights for policy 0, policy_version 57483 (0.0009) [2023-10-10 18:49:28,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 117604352. Throughput: 0: 1808.9, 1: 1806.1. Samples: 29410434. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:49:28,789][122664] Avg episode reward: [(0, '65.190'), (1, '80.290')] [2023-10-10 18:49:28,951][123582] Updated weights for policy 0, policy_version 57493 (0.0010) [2023-10-10 18:49:29,323][123582] Updated weights for policy 0, policy_version 57503 (0.0007) [2023-10-10 18:49:29,496][123614] Updated weights for policy 1, policy_version 57380 (0.0008) [2023-10-10 18:49:29,852][123614] Updated weights for policy 1, policy_version 57390 (0.0008) [2023-10-10 18:49:30,216][123614] Updated weights for policy 1, policy_version 57400 (0.0007) [2023-10-10 18:49:32,950][123582] Updated weights for policy 0, policy_version 57513 (0.0008) [2023-10-10 18:49:33,329][123582] Updated weights for policy 0, policy_version 57523 (0.0009) [2023-10-10 18:49:33,707][123582] Updated weights for policy 0, policy_version 57533 (0.0009) [2023-10-10 18:49:33,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 117669888. Throughput: 0: 1810.3, 1: 1796.6. Samples: 29432768. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:49:33,789][122664] Avg episode reward: [(0, '64.020'), (1, '80.170')] [2023-10-10 18:49:33,936][123614] Updated weights for policy 1, policy_version 57410 (0.0007) [2023-10-10 18:49:34,301][123614] Updated weights for policy 1, policy_version 57420 (0.0008) [2023-10-10 18:49:34,671][123614] Updated weights for policy 1, policy_version 57430 (0.0008) [2023-10-10 18:49:35,040][123614] Updated weights for policy 1, policy_version 57440 (0.0009) [2023-10-10 18:49:37,380][123582] Updated weights for policy 0, policy_version 57543 (0.0008) [2023-10-10 18:49:37,750][123582] Updated weights for policy 0, policy_version 57553 (0.0008) [2023-10-10 18:49:38,130][123582] Updated weights for policy 0, policy_version 57563 (0.0007) [2023-10-10 18:49:38,614][123614] Updated weights for policy 1, policy_version 57450 (0.0007) [2023-10-10 18:49:38,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 117768192. Throughput: 0: 1808.2, 1: 1804.9. Samples: 29453680. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:49:38,788][122664] Avg episode reward: [(0, '63.950'), (1, '77.150')] [2023-10-10 18:49:38,980][123614] Updated weights for policy 1, policy_version 57460 (0.0008) [2023-10-10 18:49:39,351][123614] Updated weights for policy 1, policy_version 57470 (0.0008) [2023-10-10 18:49:41,816][123582] Updated weights for policy 0, policy_version 57573 (0.0008) [2023-10-10 18:49:42,194][123582] Updated weights for policy 0, policy_version 57583 (0.0008) [2023-10-10 18:49:42,579][123582] Updated weights for policy 0, policy_version 57593 (0.0009) [2023-10-10 18:49:43,074][123614] Updated weights for policy 1, policy_version 57480 (0.0008) [2023-10-10 18:49:43,438][123614] Updated weights for policy 1, policy_version 57490 (0.0008) [2023-10-10 18:49:43,788][122664] Fps is (10 sec: 16384.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 117833728. Throughput: 0: 1811.7, 1: 1787.6. Samples: 29465538. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:49:43,788][122664] Avg episode reward: [(0, '56.310'), (1, '72.240')] [2023-10-10 18:49:43,813][123614] Updated weights for policy 1, policy_version 57500 (0.0007) [2023-10-10 18:49:46,292][123582] Updated weights for policy 0, policy_version 57603 (0.0011) [2023-10-10 18:49:46,665][123582] Updated weights for policy 0, policy_version 57613 (0.0008) [2023-10-10 18:49:47,036][123582] Updated weights for policy 0, policy_version 57623 (0.0010) [2023-10-10 18:49:47,532][123614] Updated weights for policy 1, policy_version 57510 (0.0008) [2023-10-10 18:49:47,896][123614] Updated weights for policy 1, policy_version 57520 (0.0010) [2023-10-10 18:49:48,267][123614] Updated weights for policy 1, policy_version 57530 (0.0010) [2023-10-10 18:49:48,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 117932032. Throughput: 0: 1809.0, 1: 1799.5. Samples: 29486258. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:49:48,788][122664] Avg episode reward: [(0, '54.420'), (1, '71.540')] [2023-10-10 18:49:50,890][123582] Updated weights for policy 0, policy_version 57633 (0.0007) [2023-10-10 18:49:51,292][123582] Updated weights for policy 0, policy_version 57643 (0.0009) [2023-10-10 18:49:51,660][123582] Updated weights for policy 0, policy_version 57653 (0.0008) [2023-10-10 18:49:52,006][123614] Updated weights for policy 1, policy_version 57540 (0.0008) [2023-10-10 18:49:52,029][123582] Updated weights for policy 0, policy_version 57663 (0.0007) [2023-10-10 18:49:52,404][123614] Updated weights for policy 1, policy_version 57550 (0.0008) [2023-10-10 18:49:52,766][123614] Updated weights for policy 1, policy_version 57560 (0.0010) [2023-10-10 18:49:53,788][122664] Fps is (10 sec: 16383.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 117997568. Throughput: 0: 1806.8, 1: 1794.0. Samples: 29507690. Policy #0 lag: (min: 8.0, avg: 29.9, max: 40.0) [2023-10-10 18:49:53,789][122664] Avg episode reward: [(0, '53.930'), (1, '70.950')] [2023-10-10 18:49:53,803][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000057568_58949632.pth... [2023-10-10 18:49:53,804][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000057664_59047936.pth... [2023-10-10 18:49:53,842][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000055872_57212928.pth [2023-10-10 18:49:53,845][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000055968_57311232.pth [2023-10-10 18:49:55,597][123582] Updated weights for policy 0, policy_version 57673 (0.0007) [2023-10-10 18:49:55,971][123582] Updated weights for policy 0, policy_version 57683 (0.0007) [2023-10-10 18:49:56,339][123582] Updated weights for policy 0, policy_version 57693 (0.0007) [2023-10-10 18:49:56,416][123614] Updated weights for policy 1, policy_version 57570 (0.0007) [2023-10-10 18:49:56,793][123614] Updated weights for policy 1, policy_version 57580 (0.0009) [2023-10-10 18:49:57,158][123614] Updated weights for policy 1, policy_version 57590 (0.0010) [2023-10-10 18:49:57,528][123614] Updated weights for policy 1, policy_version 57600 (0.0011) [2023-10-10 18:49:58,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 118063104. Throughput: 0: 1809.9, 1: 1802.6. Samples: 29518760. Policy #0 lag: (min: 8.0, avg: 29.9, max: 40.0) [2023-10-10 18:49:58,789][122664] Avg episode reward: [(0, '55.950'), (1, '68.770')] [2023-10-10 18:50:00,024][123582] Updated weights for policy 0, policy_version 57703 (0.0009) [2023-10-10 18:50:00,393][123582] Updated weights for policy 0, policy_version 57713 (0.0010) [2023-10-10 18:50:00,762][123582] Updated weights for policy 0, policy_version 57723 (0.0010) [2023-10-10 18:50:01,322][123614] Updated weights for policy 1, policy_version 57610 (0.0010) [2023-10-10 18:50:01,693][123614] Updated weights for policy 1, policy_version 57620 (0.0009) [2023-10-10 18:50:02,051][123614] Updated weights for policy 1, policy_version 57630 (0.0008) [2023-10-10 18:50:03,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 118128640. Throughput: 0: 1809.8, 1: 1804.4. Samples: 29540518. Policy #0 lag: (min: 8.0, avg: 29.9, max: 40.0) [2023-10-10 18:50:03,789][122664] Avg episode reward: [(0, '54.550'), (1, '66.710')] [2023-10-10 18:50:04,341][123582] Updated weights for policy 0, policy_version 57733 (0.0008) [2023-10-10 18:50:04,703][123582] Updated weights for policy 0, policy_version 57743 (0.0009) [2023-10-10 18:50:05,084][123582] Updated weights for policy 0, policy_version 57753 (0.0008) [2023-10-10 18:50:05,737][123614] Updated weights for policy 1, policy_version 57640 (0.0007) [2023-10-10 18:50:06,115][123614] Updated weights for policy 1, policy_version 57650 (0.0008) [2023-10-10 18:50:06,484][123614] Updated weights for policy 1, policy_version 57660 (0.0008) [2023-10-10 18:50:08,757][123582] Updated weights for policy 0, policy_version 57763 (0.0009) [2023-10-10 18:50:08,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 118194176. Throughput: 0: 1810.7, 1: 1806.7. Samples: 29563322. Policy #0 lag: (min: 8.0, avg: 29.9, max: 40.0) [2023-10-10 18:50:08,789][122664] Avg episode reward: [(0, '58.670'), (1, '63.960')] [2023-10-10 18:50:09,131][123582] Updated weights for policy 0, policy_version 57773 (0.0007) [2023-10-10 18:50:09,500][123582] Updated weights for policy 0, policy_version 57783 (0.0007) [2023-10-10 18:50:10,160][123614] Updated weights for policy 1, policy_version 57670 (0.0009) [2023-10-10 18:50:10,525][123614] Updated weights for policy 1, policy_version 57680 (0.0008) [2023-10-10 18:50:10,894][123614] Updated weights for policy 1, policy_version 57690 (0.0009) [2023-10-10 18:50:13,217][123582] Updated weights for policy 0, policy_version 57793 (0.0009) [2023-10-10 18:50:13,591][123582] Updated weights for policy 0, policy_version 57803 (0.0009) [2023-10-10 18:50:13,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 118259712. Throughput: 0: 1813.4, 1: 1807.1. Samples: 29573356. Policy #0 lag: (min: 8.0, avg: 29.9, max: 40.0) [2023-10-10 18:50:13,789][122664] Avg episode reward: [(0, '57.090'), (1, '65.180')] [2023-10-10 18:50:13,964][123582] Updated weights for policy 0, policy_version 57813 (0.0008) [2023-10-10 18:50:14,338][123582] Updated weights for policy 0, policy_version 57823 (0.0008) [2023-10-10 18:50:14,736][123614] Updated weights for policy 1, policy_version 57700 (0.0009) [2023-10-10 18:50:15,112][123614] Updated weights for policy 1, policy_version 57710 (0.0009) [2023-10-10 18:50:15,483][123614] Updated weights for policy 1, policy_version 57720 (0.0007) [2023-10-10 18:50:17,952][123582] Updated weights for policy 0, policy_version 57833 (0.0007) [2023-10-10 18:50:18,320][123582] Updated weights for policy 0, policy_version 57843 (0.0009) [2023-10-10 18:50:18,694][123582] Updated weights for policy 0, policy_version 57853 (0.0007) [2023-10-10 18:50:18,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 118325248. Throughput: 0: 1823.9, 1: 1801.6. Samples: 29595914. Policy #0 lag: (min: 8.0, avg: 29.9, max: 40.0) [2023-10-10 18:50:18,788][122664] Avg episode reward: [(0, '63.930'), (1, '64.540')] [2023-10-10 18:50:19,281][123614] Updated weights for policy 1, policy_version 57730 (0.0008) [2023-10-10 18:50:19,656][123614] Updated weights for policy 1, policy_version 57740 (0.0009) [2023-10-10 18:50:20,024][123614] Updated weights for policy 1, policy_version 57750 (0.0008) [2023-10-10 18:50:20,395][123614] Updated weights for policy 1, policy_version 57760 (0.0007) [2023-10-10 18:50:22,495][123582] Updated weights for policy 0, policy_version 57863 (0.0008) [2023-10-10 18:50:22,867][123582] Updated weights for policy 0, policy_version 57873 (0.0010) [2023-10-10 18:50:23,239][123582] Updated weights for policy 0, policy_version 57883 (0.0009) [2023-10-10 18:50:23,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 118423552. Throughput: 0: 1821.3, 1: 1810.9. Samples: 29617128. Policy #0 lag: (min: 8.0, avg: 29.9, max: 40.0) [2023-10-10 18:50:23,789][122664] Avg episode reward: [(0, '62.150'), (1, '64.470')] [2023-10-10 18:50:24,029][123614] Updated weights for policy 1, policy_version 57770 (0.0010) [2023-10-10 18:50:24,402][123614] Updated weights for policy 1, policy_version 57780 (0.0011) [2023-10-10 18:50:24,769][123614] Updated weights for policy 1, policy_version 57790 (0.0009) [2023-10-10 18:50:26,932][123582] Updated weights for policy 0, policy_version 57893 (0.0008) [2023-10-10 18:50:27,305][123582] Updated weights for policy 0, policy_version 57903 (0.0008) [2023-10-10 18:50:27,676][123582] Updated weights for policy 0, policy_version 57913 (0.0009) [2023-10-10 18:50:28,481][123614] Updated weights for policy 1, policy_version 57800 (0.0009) [2023-10-10 18:50:28,788][122664] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 118489088. Throughput: 0: 1821.1, 1: 1802.5. Samples: 29628602. Policy #0 lag: (min: 8.0, avg: 29.9, max: 40.0) [2023-10-10 18:50:28,789][122664] Avg episode reward: [(0, '61.950'), (1, '60.160')] [2023-10-10 18:50:28,854][123614] Updated weights for policy 1, policy_version 57810 (0.0007) [2023-10-10 18:50:29,224][123614] Updated weights for policy 1, policy_version 57820 (0.0007) [2023-10-10 18:50:31,240][123582] Updated weights for policy 0, policy_version 57923 (0.0009) [2023-10-10 18:50:31,607][123582] Updated weights for policy 0, policy_version 57933 (0.0010) [2023-10-10 18:50:31,985][123582] Updated weights for policy 0, policy_version 57943 (0.0010) [2023-10-10 18:50:33,011][123614] Updated weights for policy 1, policy_version 57830 (0.0008) [2023-10-10 18:50:33,373][123614] Updated weights for policy 1, policy_version 57840 (0.0007) [2023-10-10 18:50:33,752][123614] Updated weights for policy 1, policy_version 57850 (0.0007) [2023-10-10 18:50:33,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 118554624. Throughput: 0: 1818.0, 1: 1810.7. Samples: 29649552. Policy #0 lag: (min: 31.0, avg: 41.9, max: 63.0) [2023-10-10 18:50:33,789][122664] Avg episode reward: [(0, '63.590'), (1, '61.750')] [2023-10-10 18:50:35,640][123582] Updated weights for policy 0, policy_version 57953 (0.0008) [2023-10-10 18:50:36,059][123582] Updated weights for policy 0, policy_version 57963 (0.0011) [2023-10-10 18:50:36,426][123582] Updated weights for policy 0, policy_version 57973 (0.0011) [2023-10-10 18:50:36,803][123582] Updated weights for policy 0, policy_version 57983 (0.0011) [2023-10-10 18:50:37,664][123614] Updated weights for policy 1, policy_version 57860 (0.0007) [2023-10-10 18:50:38,053][123614] Updated weights for policy 1, policy_version 57870 (0.0009) [2023-10-10 18:50:38,416][123614] Updated weights for policy 1, policy_version 57880 (0.0009) [2023-10-10 18:50:38,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 118652928. Throughput: 0: 1827.8, 1: 1799.0. Samples: 29670898. Policy #0 lag: (min: 31.0, avg: 41.9, max: 63.0) [2023-10-10 18:50:38,789][122664] Avg episode reward: [(0, '70.930'), (1, '60.790')] [2023-10-10 18:50:40,547][123582] Updated weights for policy 0, policy_version 57993 (0.0008) [2023-10-10 18:50:40,911][123582] Updated weights for policy 0, policy_version 58003 (0.0009) [2023-10-10 18:50:41,287][123582] Updated weights for policy 0, policy_version 58013 (0.0007) [2023-10-10 18:50:42,199][123614] Updated weights for policy 1, policy_version 57890 (0.0008) [2023-10-10 18:50:42,572][123614] Updated weights for policy 1, policy_version 57900 (0.0008) [2023-10-10 18:50:42,941][123614] Updated weights for policy 1, policy_version 57910 (0.0009) [2023-10-10 18:50:43,309][123614] Updated weights for policy 1, policy_version 57920 (0.0011) [2023-10-10 18:50:43,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 118718464. Throughput: 0: 1821.9, 1: 1808.1. Samples: 29682110. Policy #0 lag: (min: 31.0, avg: 41.9, max: 63.0) [2023-10-10 18:50:43,789][122664] Avg episode reward: [(0, '72.430'), (1, '60.520')] [2023-10-10 18:50:44,900][123582] Updated weights for policy 0, policy_version 58023 (0.0009) [2023-10-10 18:50:45,272][123582] Updated weights for policy 0, policy_version 58033 (0.0009) [2023-10-10 18:50:45,647][123582] Updated weights for policy 0, policy_version 58043 (0.0008) [2023-10-10 18:50:47,053][123614] Updated weights for policy 1, policy_version 57930 (0.0008) [2023-10-10 18:50:47,424][123614] Updated weights for policy 1, policy_version 57940 (0.0007) [2023-10-10 18:50:47,795][123614] Updated weights for policy 1, policy_version 57950 (0.0009) [2023-10-10 18:50:48,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 118784000. Throughput: 0: 1818.4, 1: 1805.7. Samples: 29703602. Policy #0 lag: (min: 31.0, avg: 41.9, max: 63.0) [2023-10-10 18:50:48,789][122664] Avg episode reward: [(0, '72.750'), (1, '59.360')] [2023-10-10 18:50:49,421][123582] Updated weights for policy 0, policy_version 58053 (0.0008) [2023-10-10 18:50:49,784][123582] Updated weights for policy 0, policy_version 58063 (0.0008) [2023-10-10 18:50:50,160][123582] Updated weights for policy 0, policy_version 58073 (0.0009) [2023-10-10 18:50:51,393][123614] Updated weights for policy 1, policy_version 57960 (0.0009) [2023-10-10 18:50:51,768][123614] Updated weights for policy 1, policy_version 57970 (0.0010) [2023-10-10 18:50:52,130][123614] Updated weights for policy 1, policy_version 57980 (0.0008) [2023-10-10 18:50:53,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 118849536. Throughput: 0: 1820.6, 1: 1799.6. Samples: 29726232. Policy #0 lag: (min: 31.0, avg: 41.9, max: 63.0) [2023-10-10 18:50:53,788][122664] Avg episode reward: [(0, '73.260'), (1, '58.450')] [2023-10-10 18:50:53,811][123582] Updated weights for policy 0, policy_version 58083 (0.0008) [2023-10-10 18:50:54,171][123582] Updated weights for policy 0, policy_version 58093 (0.0007) [2023-10-10 18:50:54,545][123582] Updated weights for policy 0, policy_version 58103 (0.0008) [2023-10-10 18:50:55,973][123614] Updated weights for policy 1, policy_version 57990 (0.0009) [2023-10-10 18:50:56,349][123614] Updated weights for policy 1, policy_version 58000 (0.0010) [2023-10-10 18:50:56,717][123614] Updated weights for policy 1, policy_version 58010 (0.0009) [2023-10-10 18:50:58,103][123582] Updated weights for policy 0, policy_version 58113 (0.0009) [2023-10-10 18:50:58,481][123582] Updated weights for policy 0, policy_version 58123 (0.0008) [2023-10-10 18:50:58,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 118915072. Throughput: 0: 1814.3, 1: 1807.1. Samples: 29736320. Policy #0 lag: (min: 31.0, avg: 41.9, max: 63.0) [2023-10-10 18:50:58,789][122664] Avg episode reward: [(0, '76.260'), (1, '57.510')] [2023-10-10 18:50:58,852][123582] Updated weights for policy 0, policy_version 58133 (0.0007) [2023-10-10 18:50:59,231][123582] Updated weights for policy 0, policy_version 58143 (0.0008) [2023-10-10 18:51:00,432][123614] Updated weights for policy 1, policy_version 58020 (0.0010) [2023-10-10 18:51:00,796][123614] Updated weights for policy 1, policy_version 58030 (0.0008) [2023-10-10 18:51:01,164][123614] Updated weights for policy 1, policy_version 58040 (0.0009) [2023-10-10 18:51:03,005][123582] Updated weights for policy 0, policy_version 58153 (0.0007) [2023-10-10 18:51:03,370][123582] Updated weights for policy 0, policy_version 58163 (0.0009) [2023-10-10 18:51:03,738][123582] Updated weights for policy 0, policy_version 58173 (0.0010) [2023-10-10 18:51:03,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 118980608. Throughput: 0: 1811.4, 1: 1807.0. Samples: 29758740. Policy #0 lag: (min: 31.0, avg: 41.9, max: 63.0) [2023-10-10 18:51:03,788][122664] Avg episode reward: [(0, '73.390'), (1, '54.480')] [2023-10-10 18:51:04,675][123614] Updated weights for policy 1, policy_version 58050 (0.0008) [2023-10-10 18:51:05,041][123614] Updated weights for policy 1, policy_version 58060 (0.0010) [2023-10-10 18:51:05,418][123614] Updated weights for policy 1, policy_version 58070 (0.0010) [2023-10-10 18:51:05,775][123614] Updated weights for policy 1, policy_version 58080 (0.0010) [2023-10-10 18:51:07,549][123582] Updated weights for policy 0, policy_version 58183 (0.0010) [2023-10-10 18:51:07,920][123582] Updated weights for policy 0, policy_version 58193 (0.0010) [2023-10-10 18:51:08,286][123582] Updated weights for policy 0, policy_version 58203 (0.0008) [2023-10-10 18:51:08,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 119078912. Throughput: 0: 1814.5, 1: 1812.9. Samples: 29780362. Policy #0 lag: (min: 31.0, avg: 41.9, max: 63.0) [2023-10-10 18:51:08,789][122664] Avg episode reward: [(0, '74.250'), (1, '54.520')] [2023-10-10 18:51:09,399][123614] Updated weights for policy 1, policy_version 58090 (0.0011) [2023-10-10 18:51:09,772][123614] Updated weights for policy 1, policy_version 58100 (0.0011) [2023-10-10 18:51:10,136][123614] Updated weights for policy 1, policy_version 58110 (0.0010) [2023-10-10 18:51:12,011][123582] Updated weights for policy 0, policy_version 58213 (0.0008) [2023-10-10 18:51:12,365][123582] Updated weights for policy 0, policy_version 58223 (0.0009) [2023-10-10 18:51:12,728][123582] Updated weights for policy 0, policy_version 58233 (0.0010) [2023-10-10 18:51:13,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 119144448. Throughput: 0: 1812.8, 1: 1808.4. Samples: 29791558. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:51:13,789][122664] Avg episode reward: [(0, '73.450'), (1, '55.040')] [2023-10-10 18:51:13,944][123614] Updated weights for policy 1, policy_version 58120 (0.0008) [2023-10-10 18:51:14,313][123614] Updated weights for policy 1, policy_version 58130 (0.0008) [2023-10-10 18:51:14,678][123614] Updated weights for policy 1, policy_version 58140 (0.0011) [2023-10-10 18:51:16,276][123582] Updated weights for policy 0, policy_version 58243 (0.0010) [2023-10-10 18:51:16,648][123582] Updated weights for policy 0, policy_version 58253 (0.0007) [2023-10-10 18:51:17,016][123582] Updated weights for policy 0, policy_version 58263 (0.0007) [2023-10-10 18:51:18,376][123614] Updated weights for policy 1, policy_version 58150 (0.0009) [2023-10-10 18:51:18,747][123614] Updated weights for policy 1, policy_version 58160 (0.0008) [2023-10-10 18:51:18,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 119209984. Throughput: 0: 1819.5, 1: 1810.6. Samples: 29812906. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:51:18,789][122664] Avg episode reward: [(0, '74.750'), (1, '57.250')] [2023-10-10 18:51:19,131][123614] Updated weights for policy 1, policy_version 58170 (0.0007) [2023-10-10 18:51:20,791][123582] Updated weights for policy 0, policy_version 58273 (0.0008) [2023-10-10 18:51:21,192][123582] Updated weights for policy 0, policy_version 58283 (0.0008) [2023-10-10 18:51:21,568][123582] Updated weights for policy 0, policy_version 58293 (0.0007) [2023-10-10 18:51:21,940][123582] Updated weights for policy 0, policy_version 58303 (0.0008) [2023-10-10 18:51:22,850][123614] Updated weights for policy 1, policy_version 58180 (0.0011) [2023-10-10 18:51:23,246][123614] Updated weights for policy 1, policy_version 58190 (0.0008) [2023-10-10 18:51:23,613][123614] Updated weights for policy 1, policy_version 58200 (0.0008) [2023-10-10 18:51:23,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 119275520. Throughput: 0: 1812.6, 1: 1816.1. Samples: 29834188. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:51:23,789][122664] Avg episode reward: [(0, '67.280'), (1, '58.400')] [2023-10-10 18:51:25,512][123582] Updated weights for policy 0, policy_version 58313 (0.0007) [2023-10-10 18:51:25,888][123582] Updated weights for policy 0, policy_version 58323 (0.0009) [2023-10-10 18:51:26,247][123582] Updated weights for policy 0, policy_version 58333 (0.0009) [2023-10-10 18:51:27,312][123614] Updated weights for policy 1, policy_version 58210 (0.0009) [2023-10-10 18:51:27,675][123614] Updated weights for policy 1, policy_version 58220 (0.0008) [2023-10-10 18:51:28,041][123614] Updated weights for policy 1, policy_version 58230 (0.0008) [2023-10-10 18:51:28,416][123614] Updated weights for policy 1, policy_version 58240 (0.0007) [2023-10-10 18:51:28,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 119373824. Throughput: 0: 1815.7, 1: 1806.7. Samples: 29845120. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:51:28,789][122664] Avg episode reward: [(0, '67.720'), (1, '56.960')] [2023-10-10 18:51:29,989][123582] Updated weights for policy 0, policy_version 58343 (0.0009) [2023-10-10 18:51:30,367][123582] Updated weights for policy 0, policy_version 58353 (0.0009) [2023-10-10 18:51:30,739][123582] Updated weights for policy 0, policy_version 58363 (0.0007) [2023-10-10 18:51:32,126][123614] Updated weights for policy 1, policy_version 58250 (0.0009) [2023-10-10 18:51:32,500][123614] Updated weights for policy 1, policy_version 58260 (0.0009) [2023-10-10 18:51:32,868][123614] Updated weights for policy 1, policy_version 58270 (0.0008) [2023-10-10 18:51:33,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 119439360. Throughput: 0: 1815.1, 1: 1811.5. Samples: 29866798. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:51:33,789][122664] Avg episode reward: [(0, '74.830'), (1, '55.240')] [2023-10-10 18:51:34,438][123582] Updated weights for policy 0, policy_version 58373 (0.0010) [2023-10-10 18:51:34,805][123582] Updated weights for policy 0, policy_version 58383 (0.0009) [2023-10-10 18:51:35,183][123582] Updated weights for policy 0, policy_version 58393 (0.0010) [2023-10-10 18:51:36,554][123614] Updated weights for policy 1, policy_version 58280 (0.0007) [2023-10-10 18:51:36,934][123614] Updated weights for policy 1, policy_version 58290 (0.0008) [2023-10-10 18:51:37,306][123614] Updated weights for policy 1, policy_version 58300 (0.0010) [2023-10-10 18:51:38,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 119504896. Throughput: 0: 1809.4, 1: 1813.0. Samples: 29889242. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:51:38,789][122664] Avg episode reward: [(0, '73.360'), (1, '54.820')] [2023-10-10 18:51:38,893][123582] Updated weights for policy 0, policy_version 58403 (0.0009) [2023-10-10 18:51:39,268][123582] Updated weights for policy 0, policy_version 58413 (0.0009) [2023-10-10 18:51:39,640][123582] Updated weights for policy 0, policy_version 58423 (0.0009) [2023-10-10 18:51:40,890][123614] Updated weights for policy 1, policy_version 58310 (0.0008) [2023-10-10 18:51:41,253][123614] Updated weights for policy 1, policy_version 58320 (0.0009) [2023-10-10 18:51:41,616][123614] Updated weights for policy 1, policy_version 58330 (0.0008) [2023-10-10 18:51:43,312][123582] Updated weights for policy 0, policy_version 58433 (0.0008) [2023-10-10 18:51:43,672][123582] Updated weights for policy 0, policy_version 58443 (0.0009) [2023-10-10 18:51:43,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 119570432. Throughput: 0: 1815.6, 1: 1812.8. Samples: 29899598. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:51:43,789][122664] Avg episode reward: [(0, '66.720'), (1, '54.280')] [2023-10-10 18:51:44,039][123582] Updated weights for policy 0, policy_version 58453 (0.0008) [2023-10-10 18:51:44,413][123582] Updated weights for policy 0, policy_version 58463 (0.0010) [2023-10-10 18:51:45,478][123614] Updated weights for policy 1, policy_version 58340 (0.0008) [2023-10-10 18:51:45,854][123614] Updated weights for policy 1, policy_version 58350 (0.0008) [2023-10-10 18:51:46,226][123614] Updated weights for policy 1, policy_version 58360 (0.0010) [2023-10-10 18:51:48,084][123582] Updated weights for policy 0, policy_version 58473 (0.0011) [2023-10-10 18:51:48,458][123582] Updated weights for policy 0, policy_version 58483 (0.0008) [2023-10-10 18:51:48,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 119635968. Throughput: 0: 1819.5, 1: 1809.5. Samples: 29922046. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:51:48,788][122664] Avg episode reward: [(0, '64.370'), (1, '57.860')] [2023-10-10 18:51:48,838][123582] Updated weights for policy 0, policy_version 58493 (0.0008) [2023-10-10 18:51:49,942][123614] Updated weights for policy 1, policy_version 58370 (0.0010) [2023-10-10 18:51:50,304][123614] Updated weights for policy 1, policy_version 58380 (0.0010) [2023-10-10 18:51:50,665][123614] Updated weights for policy 1, policy_version 58390 (0.0010) [2023-10-10 18:51:51,033][123614] Updated weights for policy 1, policy_version 58400 (0.0010) [2023-10-10 18:51:52,607][123582] Updated weights for policy 0, policy_version 58503 (0.0008) [2023-10-10 18:51:52,979][123582] Updated weights for policy 0, policy_version 58513 (0.0010) [2023-10-10 18:51:53,350][123582] Updated weights for policy 0, policy_version 58523 (0.0007) [2023-10-10 18:51:53,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 119734272. Throughput: 0: 1821.3, 1: 1809.7. Samples: 29943756. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:51:53,789][122664] Avg episode reward: [(0, '64.950'), (1, '67.780')] [2023-10-10 18:51:53,796][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000058528_59932672.pth... [2023-10-10 18:51:53,796][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000058400_59801600.pth... [2023-10-10 18:51:53,827][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000056832_58195968.pth [2023-10-10 18:51:53,832][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000056704_58064896.pth [2023-10-10 18:51:54,691][123614] Updated weights for policy 1, policy_version 58410 (0.0007) [2023-10-10 18:51:55,063][123614] Updated weights for policy 1, policy_version 58420 (0.0009) [2023-10-10 18:51:55,431][123614] Updated weights for policy 1, policy_version 58430 (0.0009) [2023-10-10 18:51:57,027][123582] Updated weights for policy 0, policy_version 58533 (0.0007) [2023-10-10 18:51:57,392][123582] Updated weights for policy 0, policy_version 58543 (0.0007) [2023-10-10 18:51:57,765][123582] Updated weights for policy 0, policy_version 58553 (0.0008) [2023-10-10 18:51:58,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 119799808. Throughput: 0: 1820.1, 1: 1811.0. Samples: 29954954. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:51:58,788][122664] Avg episode reward: [(0, '66.670'), (1, '70.740')] [2023-10-10 18:51:59,187][123614] Updated weights for policy 1, policy_version 58440 (0.0010) [2023-10-10 18:51:59,554][123614] Updated weights for policy 1, policy_version 58450 (0.0009) [2023-10-10 18:51:59,920][123614] Updated weights for policy 1, policy_version 58460 (0.0008) [2023-10-10 18:52:01,274][123582] Updated weights for policy 0, policy_version 58563 (0.0011) [2023-10-10 18:52:01,647][123582] Updated weights for policy 0, policy_version 58573 (0.0007) [2023-10-10 18:52:02,022][123582] Updated weights for policy 0, policy_version 58583 (0.0009) [2023-10-10 18:52:03,518][123614] Updated weights for policy 1, policy_version 58470 (0.0007) [2023-10-10 18:52:03,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 119865344. Throughput: 0: 1817.6, 1: 1811.0. Samples: 29976192. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:52:03,789][122664] Avg episode reward: [(0, '70.240'), (1, '70.690')] [2023-10-10 18:52:03,882][123614] Updated weights for policy 1, policy_version 58480 (0.0007) [2023-10-10 18:52:04,251][123614] Updated weights for policy 1, policy_version 58490 (0.0007) [2023-10-10 18:52:05,753][123582] Updated weights for policy 0, policy_version 58593 (0.0011) [2023-10-10 18:52:06,165][123582] Updated weights for policy 0, policy_version 58603 (0.0007) [2023-10-10 18:52:06,544][123582] Updated weights for policy 0, policy_version 58613 (0.0008) [2023-10-10 18:52:06,924][123582] Updated weights for policy 0, policy_version 58623 (0.0007) [2023-10-10 18:52:08,058][123614] Updated weights for policy 1, policy_version 58500 (0.0007) [2023-10-10 18:52:08,445][123614] Updated weights for policy 1, policy_version 58510 (0.0008) [2023-10-10 18:52:08,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 119930880. Throughput: 0: 1825.7, 1: 1819.1. Samples: 29998204. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:52:08,789][122664] Avg episode reward: [(0, '68.850'), (1, '71.870')] [2023-10-10 18:52:08,816][123614] Updated weights for policy 1, policy_version 58520 (0.0008) [2023-10-10 18:52:10,693][123582] Updated weights for policy 0, policy_version 58633 (0.0008) [2023-10-10 18:52:11,058][123582] Updated weights for policy 0, policy_version 58643 (0.0007) [2023-10-10 18:52:11,437][123582] Updated weights for policy 0, policy_version 58653 (0.0007) [2023-10-10 18:52:12,380][123614] Updated weights for policy 1, policy_version 58530 (0.0007) [2023-10-10 18:52:12,745][123614] Updated weights for policy 1, policy_version 58540 (0.0008) [2023-10-10 18:52:13,114][123614] Updated weights for policy 1, policy_version 58550 (0.0008) [2023-10-10 18:52:13,486][123614] Updated weights for policy 1, policy_version 58560 (0.0008) [2023-10-10 18:52:13,788][122664] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 120029184. Throughput: 0: 1826.3, 1: 1818.9. Samples: 30009152. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:52:13,790][122664] Avg episode reward: [(0, '69.410'), (1, '72.530')] [2023-10-10 18:52:14,934][123582] Updated weights for policy 0, policy_version 58663 (0.0007) [2023-10-10 18:52:15,300][123582] Updated weights for policy 0, policy_version 58673 (0.0007) [2023-10-10 18:52:15,675][123582] Updated weights for policy 0, policy_version 58683 (0.0007) [2023-10-10 18:52:17,299][123614] Updated weights for policy 1, policy_version 58570 (0.0009) [2023-10-10 18:52:17,667][123614] Updated weights for policy 1, policy_version 58580 (0.0007) [2023-10-10 18:52:18,032][123614] Updated weights for policy 1, policy_version 58590 (0.0010) [2023-10-10 18:52:18,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 120094720. Throughput: 0: 1832.9, 1: 1816.6. Samples: 30031024. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:52:18,789][122664] Avg episode reward: [(0, '69.920'), (1, '75.320')] [2023-10-10 18:52:19,214][123582] Updated weights for policy 0, policy_version 58693 (0.0007) [2023-10-10 18:52:19,584][123582] Updated weights for policy 0, policy_version 58703 (0.0009) [2023-10-10 18:52:19,951][123582] Updated weights for policy 0, policy_version 58713 (0.0011) [2023-10-10 18:52:21,772][123614] Updated weights for policy 1, policy_version 58600 (0.0009) [2023-10-10 18:52:22,148][123614] Updated weights for policy 1, policy_version 58610 (0.0009) [2023-10-10 18:52:22,515][123614] Updated weights for policy 1, policy_version 58620 (0.0008) [2023-10-10 18:52:23,722][123582] Updated weights for policy 0, policy_version 58723 (0.0008) [2023-10-10 18:52:23,788][122664] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 120160256. Throughput: 0: 1838.7, 1: 1807.9. Samples: 30053336. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:52:23,788][122664] Avg episode reward: [(0, '69.040'), (1, '75.570')] [2023-10-10 18:52:24,088][123582] Updated weights for policy 0, policy_version 58733 (0.0007) [2023-10-10 18:52:24,455][123582] Updated weights for policy 0, policy_version 58743 (0.0009) [2023-10-10 18:52:26,360][123614] Updated weights for policy 1, policy_version 58630 (0.0007) [2023-10-10 18:52:26,726][123614] Updated weights for policy 1, policy_version 58640 (0.0008) [2023-10-10 18:52:27,101][123614] Updated weights for policy 1, policy_version 58650 (0.0008) [2023-10-10 18:52:28,079][123582] Updated weights for policy 0, policy_version 58753 (0.0008) [2023-10-10 18:52:28,451][123582] Updated weights for policy 0, policy_version 58763 (0.0010) [2023-10-10 18:52:28,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 120225792. Throughput: 0: 1838.4, 1: 1816.7. Samples: 30064078. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:52:28,789][122664] Avg episode reward: [(0, '68.260'), (1, '79.740')] [2023-10-10 18:52:28,820][123582] Updated weights for policy 0, policy_version 58773 (0.0011) [2023-10-10 18:52:29,190][123582] Updated weights for policy 0, policy_version 58783 (0.0010) [2023-10-10 18:52:30,777][123614] Updated weights for policy 1, policy_version 58660 (0.0009) [2023-10-10 18:52:31,153][123614] Updated weights for policy 1, policy_version 58670 (0.0009) [2023-10-10 18:52:31,532][123614] Updated weights for policy 1, policy_version 58680 (0.0007) [2023-10-10 18:52:32,851][123582] Updated weights for policy 0, policy_version 58793 (0.0009) [2023-10-10 18:52:33,215][123582] Updated weights for policy 0, policy_version 58803 (0.0010) [2023-10-10 18:52:33,594][123582] Updated weights for policy 0, policy_version 58813 (0.0008) [2023-10-10 18:52:33,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 120324096. Throughput: 0: 1839.4, 1: 1807.0. Samples: 30086134. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:52:33,789][122664] Avg episode reward: [(0, '67.710'), (1, '76.060')] [2023-10-10 18:52:35,279][123614] Updated weights for policy 1, policy_version 58690 (0.0009) [2023-10-10 18:52:35,651][123614] Updated weights for policy 1, policy_version 58700 (0.0008) [2023-10-10 18:52:36,018][123614] Updated weights for policy 1, policy_version 58710 (0.0007) [2023-10-10 18:52:36,377][123614] Updated weights for policy 1, policy_version 58720 (0.0007) [2023-10-10 18:52:37,323][123582] Updated weights for policy 0, policy_version 58823 (0.0008) [2023-10-10 18:52:37,689][123582] Updated weights for policy 0, policy_version 58833 (0.0008) [2023-10-10 18:52:38,066][123582] Updated weights for policy 0, policy_version 58843 (0.0008) [2023-10-10 18:52:38,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 120389632. Throughput: 0: 1831.1, 1: 1811.3. Samples: 30107664. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:52:38,788][122664] Avg episode reward: [(0, '63.670'), (1, '77.390')] [2023-10-10 18:52:39,917][123614] Updated weights for policy 1, policy_version 58730 (0.0008) [2023-10-10 18:52:40,285][123614] Updated weights for policy 1, policy_version 58740 (0.0009) [2023-10-10 18:52:40,652][123614] Updated weights for policy 1, policy_version 58750 (0.0008) [2023-10-10 18:52:41,629][123582] Updated weights for policy 0, policy_version 58853 (0.0009) [2023-10-10 18:52:41,995][123582] Updated weights for policy 0, policy_version 58863 (0.0010) [2023-10-10 18:52:42,372][123582] Updated weights for policy 0, policy_version 58873 (0.0008) [2023-10-10 18:52:43,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 120455168. Throughput: 0: 1835.3, 1: 1812.3. Samples: 30119098. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:52:43,789][122664] Avg episode reward: [(0, '64.980'), (1, '79.710')] [2023-10-10 18:52:44,451][123614] Updated weights for policy 1, policy_version 58760 (0.0007) [2023-10-10 18:52:44,815][123614] Updated weights for policy 1, policy_version 58770 (0.0008) [2023-10-10 18:52:45,189][123614] Updated weights for policy 1, policy_version 58780 (0.0011) [2023-10-10 18:52:46,025][123582] Updated weights for policy 0, policy_version 58883 (0.0007) [2023-10-10 18:52:46,394][123582] Updated weights for policy 0, policy_version 58893 (0.0008) [2023-10-10 18:52:46,762][123582] Updated weights for policy 0, policy_version 58903 (0.0009) [2023-10-10 18:52:48,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 120520704. Throughput: 0: 1836.9, 1: 1813.7. Samples: 30140470. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:52:48,789][122664] Avg episode reward: [(0, '65.100'), (1, '78.290')] [2023-10-10 18:52:48,873][123614] Updated weights for policy 1, policy_version 58790 (0.0010) [2023-10-10 18:52:49,244][123614] Updated weights for policy 1, policy_version 58800 (0.0010) [2023-10-10 18:52:49,606][123614] Updated weights for policy 1, policy_version 58810 (0.0010) [2023-10-10 18:52:50,667][123582] Updated weights for policy 0, policy_version 58913 (0.0009) [2023-10-10 18:52:51,070][123582] Updated weights for policy 0, policy_version 58923 (0.0009) [2023-10-10 18:52:51,444][123582] Updated weights for policy 0, policy_version 58933 (0.0008) [2023-10-10 18:52:51,816][123582] Updated weights for policy 0, policy_version 58943 (0.0008) [2023-10-10 18:52:53,430][123614] Updated weights for policy 1, policy_version 58820 (0.0009) [2023-10-10 18:52:53,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 120586240. Throughput: 0: 1828.4, 1: 1818.7. Samples: 30162324. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:52:53,789][122664] Avg episode reward: [(0, '64.930'), (1, '77.210')] [2023-10-10 18:52:53,825][123614] Updated weights for policy 1, policy_version 58830 (0.0007) [2023-10-10 18:52:54,194][123614] Updated weights for policy 1, policy_version 58840 (0.0008) [2023-10-10 18:52:55,495][123582] Updated weights for policy 0, policy_version 58953 (0.0008) [2023-10-10 18:52:55,861][123582] Updated weights for policy 0, policy_version 58963 (0.0007) [2023-10-10 18:52:56,233][123582] Updated weights for policy 0, policy_version 58973 (0.0008) [2023-10-10 18:52:57,799][123614] Updated weights for policy 1, policy_version 58850 (0.0012) [2023-10-10 18:52:58,171][123614] Updated weights for policy 1, policy_version 58860 (0.0009) [2023-10-10 18:52:58,536][123614] Updated weights for policy 1, policy_version 58870 (0.0008) [2023-10-10 18:52:58,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 120651776. Throughput: 0: 1825.6, 1: 1809.6. Samples: 30172736. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:52:58,788][122664] Avg episode reward: [(0, '60.840'), (1, '71.540')] [2023-10-10 18:52:58,899][123614] Updated weights for policy 1, policy_version 58880 (0.0008) [2023-10-10 18:52:59,821][123582] Updated weights for policy 0, policy_version 58983 (0.0008) [2023-10-10 18:53:00,194][123582] Updated weights for policy 0, policy_version 58993 (0.0007) [2023-10-10 18:53:00,572][123582] Updated weights for policy 0, policy_version 59003 (0.0007) [2023-10-10 18:53:02,609][123614] Updated weights for policy 1, policy_version 58890 (0.0007) [2023-10-10 18:53:02,974][123614] Updated weights for policy 1, policy_version 58900 (0.0009) [2023-10-10 18:53:03,345][123614] Updated weights for policy 1, policy_version 58910 (0.0007) [2023-10-10 18:53:03,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 120750080. Throughput: 0: 1826.0, 1: 1823.7. Samples: 30195258. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:53:03,788][122664] Avg episode reward: [(0, '59.940'), (1, '66.680')] [2023-10-10 18:53:04,181][123582] Updated weights for policy 0, policy_version 59013 (0.0009) [2023-10-10 18:53:04,564][123582] Updated weights for policy 0, policy_version 59023 (0.0010) [2023-10-10 18:53:04,933][123582] Updated weights for policy 0, policy_version 59033 (0.0010) [2023-10-10 18:53:06,893][123614] Updated weights for policy 1, policy_version 58920 (0.0010) [2023-10-10 18:53:07,261][123614] Updated weights for policy 1, policy_version 58930 (0.0008) [2023-10-10 18:53:07,626][123614] Updated weights for policy 1, policy_version 58940 (0.0007) [2023-10-10 18:53:08,610][123582] Updated weights for policy 0, policy_version 59043 (0.0007) [2023-10-10 18:53:08,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 120815616. Throughput: 0: 1823.0, 1: 1816.3. Samples: 30217106. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:53:08,789][122664] Avg episode reward: [(0, '57.400'), (1, '68.640')] [2023-10-10 18:53:08,979][123582] Updated weights for policy 0, policy_version 59053 (0.0008) [2023-10-10 18:53:09,356][123582] Updated weights for policy 0, policy_version 59063 (0.0009) [2023-10-10 18:53:11,386][123614] Updated weights for policy 1, policy_version 58950 (0.0007) [2023-10-10 18:53:11,764][123614] Updated weights for policy 1, policy_version 58960 (0.0008) [2023-10-10 18:53:12,129][123614] Updated weights for policy 1, policy_version 58970 (0.0010) [2023-10-10 18:53:13,109][123582] Updated weights for policy 0, policy_version 59073 (0.0008) [2023-10-10 18:53:13,481][123582] Updated weights for policy 0, policy_version 59083 (0.0007) [2023-10-10 18:53:13,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.6, 300 sec: 14440.1). Total num frames: 120881152. Throughput: 0: 1819.2, 1: 1818.4. Samples: 30227766. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:53:13,788][122664] Avg episode reward: [(0, '60.810'), (1, '68.930')] [2023-10-10 18:53:13,855][123582] Updated weights for policy 0, policy_version 59093 (0.0007) [2023-10-10 18:53:14,223][123582] Updated weights for policy 0, policy_version 59103 (0.0009) [2023-10-10 18:53:15,746][123614] Updated weights for policy 1, policy_version 58980 (0.0008) [2023-10-10 18:53:16,110][123614] Updated weights for policy 1, policy_version 58990 (0.0009) [2023-10-10 18:53:16,475][123614] Updated weights for policy 1, policy_version 59000 (0.0010) [2023-10-10 18:53:17,854][123582] Updated weights for policy 0, policy_version 59113 (0.0007) [2023-10-10 18:53:18,233][123582] Updated weights for policy 0, policy_version 59123 (0.0009) [2023-10-10 18:53:18,599][123582] Updated weights for policy 0, policy_version 59133 (0.0009) [2023-10-10 18:53:18,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 120979456. Throughput: 0: 1816.4, 1: 1820.5. Samples: 30249792. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:53:18,789][122664] Avg episode reward: [(0, '62.360'), (1, '69.180')] [2023-10-10 18:53:20,149][123614] Updated weights for policy 1, policy_version 59010 (0.0009) [2023-10-10 18:53:20,511][123614] Updated weights for policy 1, policy_version 59020 (0.0009) [2023-10-10 18:53:20,881][123614] Updated weights for policy 1, policy_version 59030 (0.0007) [2023-10-10 18:53:21,244][123614] Updated weights for policy 1, policy_version 59040 (0.0007) [2023-10-10 18:53:22,225][123582] Updated weights for policy 0, policy_version 59143 (0.0010) [2023-10-10 18:53:22,596][123582] Updated weights for policy 0, policy_version 59153 (0.0008) [2023-10-10 18:53:22,971][123582] Updated weights for policy 0, policy_version 59163 (0.0010) [2023-10-10 18:53:23,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 121044992. Throughput: 0: 1821.8, 1: 1818.0. Samples: 30271454. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:53:23,789][122664] Avg episode reward: [(0, '61.510'), (1, '69.220')] [2023-10-10 18:53:24,964][123614] Updated weights for policy 1, policy_version 59050 (0.0009) [2023-10-10 18:53:25,340][123614] Updated weights for policy 1, policy_version 59060 (0.0008) [2023-10-10 18:53:25,704][123614] Updated weights for policy 1, policy_version 59070 (0.0007) [2023-10-10 18:53:26,605][123582] Updated weights for policy 0, policy_version 59173 (0.0007) [2023-10-10 18:53:26,969][123582] Updated weights for policy 0, policy_version 59183 (0.0008) [2023-10-10 18:53:27,343][123582] Updated weights for policy 0, policy_version 59193 (0.0008) [2023-10-10 18:53:28,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 121110528. Throughput: 0: 1824.2, 1: 1814.2. Samples: 30282824. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:53:28,789][122664] Avg episode reward: [(0, '62.130'), (1, '70.180')] [2023-10-10 18:53:29,367][123614] Updated weights for policy 1, policy_version 59080 (0.0008) [2023-10-10 18:53:29,738][123614] Updated weights for policy 1, policy_version 59090 (0.0009) [2023-10-10 18:53:30,112][123614] Updated weights for policy 1, policy_version 59100 (0.0008) [2023-10-10 18:53:30,902][123582] Updated weights for policy 0, policy_version 59203 (0.0009) [2023-10-10 18:53:31,278][123582] Updated weights for policy 0, policy_version 59213 (0.0011) [2023-10-10 18:53:31,644][123582] Updated weights for policy 0, policy_version 59223 (0.0009) [2023-10-10 18:53:33,737][123614] Updated weights for policy 1, policy_version 59110 (0.0008) [2023-10-10 18:53:33,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 121176064. Throughput: 0: 1824.3, 1: 1816.8. Samples: 30304318. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:53:33,789][122664] Avg episode reward: [(0, '64.690'), (1, '70.610')] [2023-10-10 18:53:34,105][123614] Updated weights for policy 1, policy_version 59120 (0.0009) [2023-10-10 18:53:34,470][123614] Updated weights for policy 1, policy_version 59130 (0.0008) [2023-10-10 18:53:35,384][123582] Updated weights for policy 0, policy_version 59233 (0.0009) [2023-10-10 18:53:35,754][123582] Updated weights for policy 0, policy_version 59243 (0.0010) [2023-10-10 18:53:36,124][123582] Updated weights for policy 0, policy_version 59253 (0.0009) [2023-10-10 18:53:36,510][123582] Updated weights for policy 0, policy_version 59263 (0.0008) [2023-10-10 18:53:38,099][123614] Updated weights for policy 1, policy_version 59140 (0.0011) [2023-10-10 18:53:38,502][123614] Updated weights for policy 1, policy_version 59150 (0.0009) [2023-10-10 18:53:38,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 121241600. Throughput: 0: 1827.1, 1: 1812.4. Samples: 30326100. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:53:38,788][122664] Avg episode reward: [(0, '64.180'), (1, '73.470')] [2023-10-10 18:53:38,878][123614] Updated weights for policy 1, policy_version 59160 (0.0009) [2023-10-10 18:53:40,230][123582] Updated weights for policy 0, policy_version 59273 (0.0007) [2023-10-10 18:53:40,596][123582] Updated weights for policy 0, policy_version 59283 (0.0008) [2023-10-10 18:53:40,971][123582] Updated weights for policy 0, policy_version 59293 (0.0009) [2023-10-10 18:53:42,539][123614] Updated weights for policy 1, policy_version 59170 (0.0008) [2023-10-10 18:53:42,908][123614] Updated weights for policy 1, policy_version 59180 (0.0011) [2023-10-10 18:53:43,281][123614] Updated weights for policy 1, policy_version 59190 (0.0009) [2023-10-10 18:53:43,654][123614] Updated weights for policy 1, policy_version 59200 (0.0008) [2023-10-10 18:53:43,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 121339904. Throughput: 0: 1826.9, 1: 1819.5. Samples: 30336824. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:53:43,789][122664] Avg episode reward: [(0, '66.300'), (1, '72.390')] [2023-10-10 18:53:44,765][123582] Updated weights for policy 0, policy_version 59303 (0.0008) [2023-10-10 18:53:45,127][123582] Updated weights for policy 0, policy_version 59313 (0.0008) [2023-10-10 18:53:45,506][123582] Updated weights for policy 0, policy_version 59323 (0.0007) [2023-10-10 18:53:47,317][123614] Updated weights for policy 1, policy_version 59210 (0.0009) [2023-10-10 18:53:47,682][123614] Updated weights for policy 1, policy_version 59220 (0.0010) [2023-10-10 18:53:48,048][123614] Updated weights for policy 1, policy_version 59230 (0.0010) [2023-10-10 18:53:48,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 121405440. Throughput: 0: 1818.8, 1: 1812.3. Samples: 30358656. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:53:48,789][122664] Avg episode reward: [(0, '65.760'), (1, '73.170')] [2023-10-10 18:53:49,254][123582] Updated weights for policy 0, policy_version 59333 (0.0009) [2023-10-10 18:53:49,619][123582] Updated weights for policy 0, policy_version 59343 (0.0010) [2023-10-10 18:53:49,988][123582] Updated weights for policy 0, policy_version 59353 (0.0007) [2023-10-10 18:53:51,833][123614] Updated weights for policy 1, policy_version 59240 (0.0008) [2023-10-10 18:53:52,197][123614] Updated weights for policy 1, policy_version 59250 (0.0007) [2023-10-10 18:53:52,561][123614] Updated weights for policy 1, policy_version 59260 (0.0008) [2023-10-10 18:53:53,673][123582] Updated weights for policy 0, policy_version 59363 (0.0007) [2023-10-10 18:53:53,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 121470976. Throughput: 0: 1823.8, 1: 1821.5. Samples: 30381148. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:53:53,789][122664] Avg episode reward: [(0, '62.530'), (1, '67.190')] [2023-10-10 18:53:53,801][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000059264_60686336.pth... [2023-10-10 18:53:53,842][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000057568_58949632.pth [2023-10-10 18:53:53,847][123465] Saving a milestone ./train_atari/atari_demonattack_APPO/checkpoint_p1/milestones/checkpoint_000059264_60686336.pth [2023-10-10 18:53:54,056][123582] Updated weights for policy 0, policy_version 59373 (0.0008) [2023-10-10 18:53:54,420][123582] Updated weights for policy 0, policy_version 59383 (0.0008) [2023-10-10 18:53:54,750][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000059392_60817408.pth... [2023-10-10 18:53:54,782][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000057664_59047936.pth [2023-10-10 18:53:54,787][123247] Saving a milestone ./train_atari/atari_demonattack_APPO/checkpoint_p0/milestones/checkpoint_000059392_60817408.pth [2023-10-10 18:53:56,081][123614] Updated weights for policy 1, policy_version 59270 (0.0008) [2023-10-10 18:53:56,452][123614] Updated weights for policy 1, policy_version 59280 (0.0008) [2023-10-10 18:53:56,831][123614] Updated weights for policy 1, policy_version 59290 (0.0007) [2023-10-10 18:53:57,921][123582] Updated weights for policy 0, policy_version 59393 (0.0008) [2023-10-10 18:53:58,298][123582] Updated weights for policy 0, policy_version 59403 (0.0008) [2023-10-10 18:53:58,673][123582] Updated weights for policy 0, policy_version 59413 (0.0009) [2023-10-10 18:53:58,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14440.2). Total num frames: 121536512. Throughput: 0: 1826.4, 1: 1818.4. Samples: 30391778. Policy #0 lag: (min: 29.0, avg: 34.9, max: 61.0) [2023-10-10 18:53:58,788][122664] Avg episode reward: [(0, '63.620'), (1, '68.900')] [2023-10-10 18:53:59,043][123582] Updated weights for policy 0, policy_version 59423 (0.0008) [2023-10-10 18:54:00,397][123614] Updated weights for policy 1, policy_version 59300 (0.0008) [2023-10-10 18:54:00,768][123614] Updated weights for policy 1, policy_version 59310 (0.0008) [2023-10-10 18:54:01,141][123614] Updated weights for policy 1, policy_version 59320 (0.0009) [2023-10-10 18:54:02,708][123582] Updated weights for policy 0, policy_version 59433 (0.0009) [2023-10-10 18:54:03,083][123582] Updated weights for policy 0, policy_version 59443 (0.0009) [2023-10-10 18:54:03,459][123582] Updated weights for policy 0, policy_version 59453 (0.0007) [2023-10-10 18:54:03,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 121634816. Throughput: 0: 1825.4, 1: 1829.5. Samples: 30414264. Policy #0 lag: (min: 29.0, avg: 34.9, max: 61.0) [2023-10-10 18:54:03,789][122664] Avg episode reward: [(0, '62.290'), (1, '68.670')] [2023-10-10 18:54:04,898][123614] Updated weights for policy 1, policy_version 59330 (0.0008) [2023-10-10 18:54:05,272][123614] Updated weights for policy 1, policy_version 59340 (0.0008) [2023-10-10 18:54:05,637][123614] Updated weights for policy 1, policy_version 59350 (0.0010) [2023-10-10 18:54:06,011][123614] Updated weights for policy 1, policy_version 59360 (0.0009) [2023-10-10 18:54:07,058][123582] Updated weights for policy 0, policy_version 59463 (0.0007) [2023-10-10 18:54:07,436][123582] Updated weights for policy 0, policy_version 59473 (0.0007) [2023-10-10 18:54:07,809][123582] Updated weights for policy 0, policy_version 59483 (0.0007) [2023-10-10 18:54:08,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 121700352. Throughput: 0: 1825.4, 1: 1826.0. Samples: 30435770. Policy #0 lag: (min: 29.0, avg: 34.9, max: 61.0) [2023-10-10 18:54:08,789][122664] Avg episode reward: [(0, '60.760'), (1, '71.220')] [2023-10-10 18:54:09,655][123614] Updated weights for policy 1, policy_version 59370 (0.0009) [2023-10-10 18:54:10,013][123614] Updated weights for policy 1, policy_version 59380 (0.0009) [2023-10-10 18:54:10,390][123614] Updated weights for policy 1, policy_version 59390 (0.0010) [2023-10-10 18:54:11,595][123582] Updated weights for policy 0, policy_version 59493 (0.0007) [2023-10-10 18:54:11,962][123582] Updated weights for policy 0, policy_version 59503 (0.0008) [2023-10-10 18:54:12,341][123582] Updated weights for policy 0, policy_version 59513 (0.0008) [2023-10-10 18:54:13,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 121765888. Throughput: 0: 1820.2, 1: 1830.1. Samples: 30447090. Policy #0 lag: (min: 29.0, avg: 34.9, max: 61.0) [2023-10-10 18:54:13,789][122664] Avg episode reward: [(0, '58.780'), (1, '72.710')] [2023-10-10 18:54:14,182][123614] Updated weights for policy 1, policy_version 59400 (0.0009) [2023-10-10 18:54:14,545][123614] Updated weights for policy 1, policy_version 59410 (0.0009) [2023-10-10 18:54:14,921][123614] Updated weights for policy 1, policy_version 59420 (0.0008) [2023-10-10 18:54:16,168][123582] Updated weights for policy 0, policy_version 59523 (0.0009) [2023-10-10 18:54:16,533][123582] Updated weights for policy 0, policy_version 59533 (0.0009) [2023-10-10 18:54:16,907][123582] Updated weights for policy 0, policy_version 59543 (0.0007) [2023-10-10 18:54:18,642][123614] Updated weights for policy 1, policy_version 59430 (0.0007) [2023-10-10 18:54:18,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 121831424. Throughput: 0: 1815.9, 1: 1826.4. Samples: 30468220. Policy #0 lag: (min: 29.0, avg: 34.9, max: 61.0) [2023-10-10 18:54:18,789][122664] Avg episode reward: [(0, '57.020'), (1, '71.350')] [2023-10-10 18:54:18,997][123614] Updated weights for policy 1, policy_version 59440 (0.0008) [2023-10-10 18:54:19,367][123614] Updated weights for policy 1, policy_version 59450 (0.0008) [2023-10-10 18:54:20,591][123582] Updated weights for policy 0, policy_version 59553 (0.0008) [2023-10-10 18:54:20,961][123582] Updated weights for policy 0, policy_version 59563 (0.0008) [2023-10-10 18:54:21,328][123582] Updated weights for policy 0, policy_version 59573 (0.0008) [2023-10-10 18:54:21,691][123582] Updated weights for policy 0, policy_version 59583 (0.0009) [2023-10-10 18:54:23,025][123614] Updated weights for policy 1, policy_version 59460 (0.0010) [2023-10-10 18:54:23,418][123614] Updated weights for policy 1, policy_version 59470 (0.0009) [2023-10-10 18:54:23,781][123614] Updated weights for policy 1, policy_version 59480 (0.0007) [2023-10-10 18:54:23,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 121896960. Throughput: 0: 1817.8, 1: 1827.5. Samples: 30490136. Policy #0 lag: (min: 29.0, avg: 34.9, max: 61.0) [2023-10-10 18:54:23,788][122664] Avg episode reward: [(0, '59.550'), (1, '70.740')] [2023-10-10 18:54:25,428][123582] Updated weights for policy 0, policy_version 59593 (0.0007) [2023-10-10 18:54:25,800][123582] Updated weights for policy 0, policy_version 59603 (0.0008) [2023-10-10 18:54:26,180][123582] Updated weights for policy 0, policy_version 59613 (0.0007) [2023-10-10 18:54:27,415][123614] Updated weights for policy 1, policy_version 59490 (0.0009) [2023-10-10 18:54:27,785][123614] Updated weights for policy 1, policy_version 59500 (0.0008) [2023-10-10 18:54:28,147][123614] Updated weights for policy 1, policy_version 59510 (0.0007) [2023-10-10 18:54:28,507][123614] Updated weights for policy 1, policy_version 59520 (0.0007) [2023-10-10 18:54:28,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 121995264. Throughput: 0: 1818.4, 1: 1831.5. Samples: 30501066. Policy #0 lag: (min: 29.0, avg: 34.9, max: 61.0) [2023-10-10 18:54:28,789][122664] Avg episode reward: [(0, '55.860'), (1, '68.560')] [2023-10-10 18:54:29,747][123582] Updated weights for policy 0, policy_version 59623 (0.0009) [2023-10-10 18:54:30,113][123582] Updated weights for policy 0, policy_version 59633 (0.0009) [2023-10-10 18:54:30,489][123582] Updated weights for policy 0, policy_version 59643 (0.0010) [2023-10-10 18:54:32,137][123614] Updated weights for policy 1, policy_version 59530 (0.0008) [2023-10-10 18:54:32,504][123614] Updated weights for policy 1, policy_version 59540 (0.0007) [2023-10-10 18:54:32,866][123614] Updated weights for policy 1, policy_version 59550 (0.0008) [2023-10-10 18:54:33,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 122060800. Throughput: 0: 1824.7, 1: 1825.3. Samples: 30522904. Policy #0 lag: (min: 29.0, avg: 34.9, max: 61.0) [2023-10-10 18:54:33,789][122664] Avg episode reward: [(0, '53.820'), (1, '69.310')] [2023-10-10 18:54:34,120][123582] Updated weights for policy 0, policy_version 59653 (0.0009) [2023-10-10 18:54:34,493][123582] Updated weights for policy 0, policy_version 59663 (0.0010) [2023-10-10 18:54:34,863][123582] Updated weights for policy 0, policy_version 59673 (0.0008) [2023-10-10 18:54:36,718][123614] Updated weights for policy 1, policy_version 59560 (0.0010) [2023-10-10 18:54:37,094][123614] Updated weights for policy 1, policy_version 59570 (0.0008) [2023-10-10 18:54:37,461][123614] Updated weights for policy 1, policy_version 59580 (0.0008) [2023-10-10 18:54:38,684][123582] Updated weights for policy 0, policy_version 59683 (0.0008) [2023-10-10 18:54:38,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 122126336. Throughput: 0: 1816.4, 1: 1828.7. Samples: 30545174. Policy #0 lag: (min: 19.0, avg: 25.5, max: 51.0) [2023-10-10 18:54:38,789][122664] Avg episode reward: [(0, '48.760'), (1, '70.780')] [2023-10-10 18:54:39,058][123582] Updated weights for policy 0, policy_version 59693 (0.0008) [2023-10-10 18:54:39,429][123582] Updated weights for policy 0, policy_version 59703 (0.0011) [2023-10-10 18:54:41,086][123614] Updated weights for policy 1, policy_version 59590 (0.0008) [2023-10-10 18:54:41,449][123614] Updated weights for policy 1, policy_version 59600 (0.0010) [2023-10-10 18:54:41,816][123614] Updated weights for policy 1, policy_version 59610 (0.0011) [2023-10-10 18:54:43,047][123582] Updated weights for policy 0, policy_version 59713 (0.0008) [2023-10-10 18:54:43,416][123582] Updated weights for policy 0, policy_version 59723 (0.0008) [2023-10-10 18:54:43,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 122191872. Throughput: 0: 1813.8, 1: 1824.9. Samples: 30555518. Policy #0 lag: (min: 19.0, avg: 25.5, max: 51.0) [2023-10-10 18:54:43,789][122664] Avg episode reward: [(0, '50.410'), (1, '64.430')] [2023-10-10 18:54:43,798][123582] Updated weights for policy 0, policy_version 59733 (0.0008) [2023-10-10 18:54:44,181][123582] Updated weights for policy 0, policy_version 59743 (0.0009) [2023-10-10 18:54:45,741][123614] Updated weights for policy 1, policy_version 59620 (0.0008) [2023-10-10 18:54:46,114][123614] Updated weights for policy 1, policy_version 59630 (0.0009) [2023-10-10 18:54:46,471][123614] Updated weights for policy 1, policy_version 59640 (0.0010) [2023-10-10 18:54:47,952][123582] Updated weights for policy 0, policy_version 59753 (0.0010) [2023-10-10 18:54:48,324][123582] Updated weights for policy 0, policy_version 59763 (0.0007) [2023-10-10 18:54:48,691][123582] Updated weights for policy 0, policy_version 59773 (0.0008) [2023-10-10 18:54:48,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 122257408. Throughput: 0: 1816.4, 1: 1813.8. Samples: 30577624. Policy #0 lag: (min: 19.0, avg: 25.5, max: 51.0) [2023-10-10 18:54:48,789][122664] Avg episode reward: [(0, '51.790'), (1, '64.010')] [2023-10-10 18:54:50,370][123614] Updated weights for policy 1, policy_version 59650 (0.0010) [2023-10-10 18:54:50,729][123614] Updated weights for policy 1, policy_version 59660 (0.0008) [2023-10-10 18:54:51,102][123614] Updated weights for policy 1, policy_version 59670 (0.0008) [2023-10-10 18:54:51,472][123614] Updated weights for policy 1, policy_version 59680 (0.0008) [2023-10-10 18:54:52,286][123582] Updated weights for policy 0, policy_version 59783 (0.0007) [2023-10-10 18:54:52,656][123582] Updated weights for policy 0, policy_version 59793 (0.0007) [2023-10-10 18:54:53,028][123582] Updated weights for policy 0, policy_version 59803 (0.0010) [2023-10-10 18:54:53,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 122355712. Throughput: 0: 1817.0, 1: 1813.6. Samples: 30599148. Policy #0 lag: (min: 19.0, avg: 25.5, max: 51.0) [2023-10-10 18:54:53,789][122664] Avg episode reward: [(0, '50.660'), (1, '64.760')] [2023-10-10 18:54:54,882][123614] Updated weights for policy 1, policy_version 59690 (0.0008) [2023-10-10 18:54:55,244][123614] Updated weights for policy 1, policy_version 59700 (0.0009) [2023-10-10 18:54:55,615][123614] Updated weights for policy 1, policy_version 59710 (0.0007) [2023-10-10 18:54:56,673][123582] Updated weights for policy 0, policy_version 59813 (0.0008) [2023-10-10 18:54:57,045][123582] Updated weights for policy 0, policy_version 59823 (0.0008) [2023-10-10 18:54:57,424][123582] Updated weights for policy 0, policy_version 59833 (0.0007) [2023-10-10 18:54:58,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 122421248. Throughput: 0: 1823.3, 1: 1813.0. Samples: 30610726. Policy #0 lag: (min: 19.0, avg: 25.5, max: 51.0) [2023-10-10 18:54:58,789][122664] Avg episode reward: [(0, '51.280'), (1, '67.510')] [2023-10-10 18:54:59,383][123614] Updated weights for policy 1, policy_version 59720 (0.0009) [2023-10-10 18:54:59,760][123614] Updated weights for policy 1, policy_version 59730 (0.0010) [2023-10-10 18:55:00,122][123614] Updated weights for policy 1, policy_version 59740 (0.0008) [2023-10-10 18:55:01,235][123582] Updated weights for policy 0, policy_version 59843 (0.0009) [2023-10-10 18:55:01,602][123582] Updated weights for policy 0, policy_version 59853 (0.0009) [2023-10-10 18:55:01,977][123582] Updated weights for policy 0, policy_version 59863 (0.0007) [2023-10-10 18:55:03,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 122486784. Throughput: 0: 1822.7, 1: 1814.8. Samples: 30631904. Policy #0 lag: (min: 19.0, avg: 25.5, max: 51.0) [2023-10-10 18:55:03,788][122664] Avg episode reward: [(0, '52.520'), (1, '72.170')] [2023-10-10 18:55:03,884][123614] Updated weights for policy 1, policy_version 59750 (0.0010) [2023-10-10 18:55:04,254][123614] Updated weights for policy 1, policy_version 59760 (0.0009) [2023-10-10 18:55:04,625][123614] Updated weights for policy 1, policy_version 59770 (0.0010) [2023-10-10 18:55:05,716][123582] Updated weights for policy 0, policy_version 59873 (0.0007) [2023-10-10 18:55:06,083][123582] Updated weights for policy 0, policy_version 59883 (0.0008) [2023-10-10 18:55:06,454][123582] Updated weights for policy 0, policy_version 59893 (0.0007) [2023-10-10 18:55:06,824][123582] Updated weights for policy 0, policy_version 59903 (0.0007) [2023-10-10 18:55:08,362][123614] Updated weights for policy 1, policy_version 59780 (0.0007) [2023-10-10 18:55:08,760][123614] Updated weights for policy 1, policy_version 59790 (0.0008) [2023-10-10 18:55:08,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 122552320. Throughput: 0: 1815.2, 1: 1817.6. Samples: 30653612. Policy #0 lag: (min: 19.0, avg: 25.5, max: 51.0) [2023-10-10 18:55:08,789][122664] Avg episode reward: [(0, '53.330'), (1, '72.610')] [2023-10-10 18:55:09,137][123614] Updated weights for policy 1, policy_version 59800 (0.0007) [2023-10-10 18:55:10,601][123582] Updated weights for policy 0, policy_version 59913 (0.0007) [2023-10-10 18:55:10,972][123582] Updated weights for policy 0, policy_version 59923 (0.0007) [2023-10-10 18:55:11,348][123582] Updated weights for policy 0, policy_version 59933 (0.0007) [2023-10-10 18:55:12,740][123614] Updated weights for policy 1, policy_version 59810 (0.0008) [2023-10-10 18:55:13,107][123614] Updated weights for policy 1, policy_version 59820 (0.0008) [2023-10-10 18:55:13,473][123614] Updated weights for policy 1, policy_version 59830 (0.0010) [2023-10-10 18:55:13,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 122617856. Throughput: 0: 1818.1, 1: 1805.7. Samples: 30664140. Policy #0 lag: (min: 19.0, avg: 25.5, max: 51.0) [2023-10-10 18:55:13,788][122664] Avg episode reward: [(0, '56.880'), (1, '69.630')] [2023-10-10 18:55:13,845][123614] Updated weights for policy 1, policy_version 59840 (0.0007) [2023-10-10 18:55:14,906][123582] Updated weights for policy 0, policy_version 59943 (0.0009) [2023-10-10 18:55:15,262][123582] Updated weights for policy 0, policy_version 59953 (0.0008) [2023-10-10 18:55:15,634][123582] Updated weights for policy 0, policy_version 59963 (0.0009) [2023-10-10 18:55:17,508][123614] Updated weights for policy 1, policy_version 59850 (0.0009) [2023-10-10 18:55:17,885][123614] Updated weights for policy 1, policy_version 59860 (0.0010) [2023-10-10 18:55:18,241][123614] Updated weights for policy 1, policy_version 59870 (0.0009) [2023-10-10 18:55:18,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 122716160. Throughput: 0: 1808.9, 1: 1815.3. Samples: 30685992. Policy #0 lag: (min: 29.0, avg: 31.7, max: 61.0) [2023-10-10 18:55:18,788][122664] Avg episode reward: [(0, '57.730'), (1, '70.330')] [2023-10-10 18:55:19,520][123582] Updated weights for policy 0, policy_version 59973 (0.0009) [2023-10-10 18:55:19,884][123582] Updated weights for policy 0, policy_version 59983 (0.0009) [2023-10-10 18:55:20,263][123582] Updated weights for policy 0, policy_version 59993 (0.0010) [2023-10-10 18:55:22,111][123614] Updated weights for policy 1, policy_version 59880 (0.0008) [2023-10-10 18:55:22,480][123614] Updated weights for policy 1, policy_version 59890 (0.0009) [2023-10-10 18:55:22,852][123614] Updated weights for policy 1, policy_version 59900 (0.0007) [2023-10-10 18:55:23,788][122664] Fps is (10 sec: 16383.4, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 122781696. Throughput: 0: 1812.6, 1: 1804.0. Samples: 30707924. Policy #0 lag: (min: 29.0, avg: 31.7, max: 61.0) [2023-10-10 18:55:23,789][122664] Avg episode reward: [(0, '56.230'), (1, '71.290')] [2023-10-10 18:55:23,931][123582] Updated weights for policy 0, policy_version 60003 (0.0008) [2023-10-10 18:55:24,299][123582] Updated weights for policy 0, policy_version 60013 (0.0007) [2023-10-10 18:55:24,678][123582] Updated weights for policy 0, policy_version 60023 (0.0009) [2023-10-10 18:55:26,565][123614] Updated weights for policy 1, policy_version 59910 (0.0007) [2023-10-10 18:55:26,929][123614] Updated weights for policy 1, policy_version 59920 (0.0007) [2023-10-10 18:55:27,295][123614] Updated weights for policy 1, policy_version 59930 (0.0009) [2023-10-10 18:55:28,302][123582] Updated weights for policy 0, policy_version 60033 (0.0009) [2023-10-10 18:55:28,672][123582] Updated weights for policy 0, policy_version 60043 (0.0008) [2023-10-10 18:55:28,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 122847232. Throughput: 0: 1812.9, 1: 1815.8. Samples: 30718812. Policy #0 lag: (min: 29.0, avg: 31.7, max: 61.0) [2023-10-10 18:55:28,789][122664] Avg episode reward: [(0, '57.440'), (1, '70.400')] [2023-10-10 18:55:29,048][123582] Updated weights for policy 0, policy_version 60053 (0.0009) [2023-10-10 18:55:29,425][123582] Updated weights for policy 0, policy_version 60063 (0.0008) [2023-10-10 18:55:31,051][123614] Updated weights for policy 1, policy_version 59940 (0.0007) [2023-10-10 18:55:31,419][123614] Updated weights for policy 1, policy_version 59950 (0.0010) [2023-10-10 18:55:31,786][123614] Updated weights for policy 1, policy_version 59960 (0.0008) [2023-10-10 18:55:33,148][123582] Updated weights for policy 0, policy_version 60073 (0.0009) [2023-10-10 18:55:33,524][123582] Updated weights for policy 0, policy_version 60083 (0.0009) [2023-10-10 18:55:33,788][122664] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 122912768. Throughput: 0: 1810.0, 1: 1812.4. Samples: 30740634. Policy #0 lag: (min: 29.0, avg: 31.7, max: 61.0) [2023-10-10 18:55:33,788][122664] Avg episode reward: [(0, '56.770'), (1, '71.560')] [2023-10-10 18:55:33,901][123582] Updated weights for policy 0, policy_version 60093 (0.0009) [2023-10-10 18:55:35,443][123614] Updated weights for policy 1, policy_version 59970 (0.0007) [2023-10-10 18:55:35,814][123614] Updated weights for policy 1, policy_version 59980 (0.0008) [2023-10-10 18:55:36,181][123614] Updated weights for policy 1, policy_version 59990 (0.0009) [2023-10-10 18:55:36,537][123614] Updated weights for policy 1, policy_version 60000 (0.0009) [2023-10-10 18:55:37,563][123582] Updated weights for policy 0, policy_version 60103 (0.0008) [2023-10-10 18:55:37,931][123582] Updated weights for policy 0, policy_version 60113 (0.0009) [2023-10-10 18:55:38,307][123582] Updated weights for policy 0, policy_version 60123 (0.0010) [2023-10-10 18:55:38,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 123011072. Throughput: 0: 1811.0, 1: 1808.3. Samples: 30762016. Policy #0 lag: (min: 29.0, avg: 31.7, max: 61.0) [2023-10-10 18:55:38,788][122664] Avg episode reward: [(0, '60.030'), (1, '75.710')] [2023-10-10 18:55:40,296][123614] Updated weights for policy 1, policy_version 60010 (0.0008) [2023-10-10 18:55:40,664][123614] Updated weights for policy 1, policy_version 60020 (0.0009) [2023-10-10 18:55:41,038][123614] Updated weights for policy 1, policy_version 60030 (0.0007) [2023-10-10 18:55:42,048][123582] Updated weights for policy 0, policy_version 60133 (0.0009) [2023-10-10 18:55:42,435][123582] Updated weights for policy 0, policy_version 60143 (0.0010) [2023-10-10 18:55:42,811][123582] Updated weights for policy 0, policy_version 60153 (0.0007) [2023-10-10 18:55:43,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 123076608. Throughput: 0: 1801.8, 1: 1807.5. Samples: 30773142. Policy #0 lag: (min: 29.0, avg: 31.7, max: 61.0) [2023-10-10 18:55:43,789][122664] Avg episode reward: [(0, '69.260'), (1, '76.820')] [2023-10-10 18:55:44,791][123614] Updated weights for policy 1, policy_version 60040 (0.0009) [2023-10-10 18:55:45,156][123614] Updated weights for policy 1, policy_version 60050 (0.0008) [2023-10-10 18:55:45,527][123614] Updated weights for policy 1, policy_version 60060 (0.0009) [2023-10-10 18:55:46,312][123582] Updated weights for policy 0, policy_version 60163 (0.0009) [2023-10-10 18:55:46,686][123582] Updated weights for policy 0, policy_version 60173 (0.0010) [2023-10-10 18:55:47,062][123582] Updated weights for policy 0, policy_version 60183 (0.0007) [2023-10-10 18:55:48,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 123142144. Throughput: 0: 1807.8, 1: 1805.4. Samples: 30794498. Policy #0 lag: (min: 29.0, avg: 31.7, max: 61.0) [2023-10-10 18:55:48,788][122664] Avg episode reward: [(0, '69.600'), (1, '79.980')] [2023-10-10 18:55:49,338][123614] Updated weights for policy 1, policy_version 60070 (0.0007) [2023-10-10 18:55:49,709][123614] Updated weights for policy 1, policy_version 60080 (0.0008) [2023-10-10 18:55:50,076][123614] Updated weights for policy 1, policy_version 60090 (0.0007) [2023-10-10 18:55:50,659][123582] Updated weights for policy 0, policy_version 60193 (0.0008) [2023-10-10 18:55:51,031][123582] Updated weights for policy 0, policy_version 60203 (0.0007) [2023-10-10 18:55:51,410][123582] Updated weights for policy 0, policy_version 60213 (0.0008) [2023-10-10 18:55:51,794][123582] Updated weights for policy 0, policy_version 60223 (0.0010) [2023-10-10 18:55:53,784][123614] Updated weights for policy 1, policy_version 60100 (0.0008) [2023-10-10 18:55:53,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 123207680. Throughput: 0: 1817.4, 1: 1819.3. Samples: 30817262. Policy #0 lag: (min: 29.0, avg: 31.7, max: 61.0) [2023-10-10 18:55:53,788][122664] Avg episode reward: [(0, '66.880'), (1, '79.400')] [2023-10-10 18:55:53,798][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000060224_61669376.pth... [2023-10-10 18:55:53,837][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000058528_59932672.pth [2023-10-10 18:55:54,164][123614] Updated weights for policy 1, policy_version 60110 (0.0009) [2023-10-10 18:55:54,521][123614] Updated weights for policy 1, policy_version 60120 (0.0008) [2023-10-10 18:55:54,816][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000060128_61571072.pth... [2023-10-10 18:55:54,853][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000058400_59801600.pth [2023-10-10 18:55:55,552][123582] Updated weights for policy 0, policy_version 60233 (0.0009) [2023-10-10 18:55:55,924][123582] Updated weights for policy 0, policy_version 60243 (0.0010) [2023-10-10 18:55:56,296][123582] Updated weights for policy 0, policy_version 60253 (0.0007) [2023-10-10 18:55:58,140][123614] Updated weights for policy 1, policy_version 60130 (0.0008) [2023-10-10 18:55:58,505][123614] Updated weights for policy 1, policy_version 60140 (0.0009) [2023-10-10 18:55:58,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 123273216. Throughput: 0: 1815.0, 1: 1814.4. Samples: 30827464. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:55:58,789][122664] Avg episode reward: [(0, '68.250'), (1, '81.200')] [2023-10-10 18:55:58,879][123614] Updated weights for policy 1, policy_version 60150 (0.0008) [2023-10-10 18:55:59,238][123614] Updated weights for policy 1, policy_version 60160 (0.0007) [2023-10-10 18:56:00,056][123582] Updated weights for policy 0, policy_version 60263 (0.0010) [2023-10-10 18:56:00,433][123582] Updated weights for policy 0, policy_version 60273 (0.0010) [2023-10-10 18:56:00,810][123582] Updated weights for policy 0, policy_version 60283 (0.0010) [2023-10-10 18:56:02,823][123614] Updated weights for policy 1, policy_version 60170 (0.0008) [2023-10-10 18:56:03,191][123614] Updated weights for policy 1, policy_version 60180 (0.0009) [2023-10-10 18:56:03,570][123614] Updated weights for policy 1, policy_version 60190 (0.0007) [2023-10-10 18:56:03,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 123371520. Throughput: 0: 1814.4, 1: 1823.8. Samples: 30849712. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:56:03,789][122664] Avg episode reward: [(0, '72.740'), (1, '81.130')] [2023-10-10 18:56:04,514][123582] Updated weights for policy 0, policy_version 60293 (0.0011) [2023-10-10 18:56:04,884][123582] Updated weights for policy 0, policy_version 60303 (0.0011) [2023-10-10 18:56:05,257][123582] Updated weights for policy 0, policy_version 60313 (0.0011) [2023-10-10 18:56:07,196][123614] Updated weights for policy 1, policy_version 60200 (0.0008) [2023-10-10 18:56:07,571][123614] Updated weights for policy 1, policy_version 60210 (0.0008) [2023-10-10 18:56:07,945][123614] Updated weights for policy 1, policy_version 60220 (0.0009) [2023-10-10 18:56:08,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 123437056. Throughput: 0: 1809.9, 1: 1816.9. Samples: 30871130. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:56:08,789][122664] Avg episode reward: [(0, '71.640'), (1, '78.230')] [2023-10-10 18:56:09,032][123582] Updated weights for policy 0, policy_version 60323 (0.0009) [2023-10-10 18:56:09,398][123582] Updated weights for policy 0, policy_version 60333 (0.0007) [2023-10-10 18:56:09,765][123582] Updated weights for policy 0, policy_version 60343 (0.0007) [2023-10-10 18:56:11,574][123614] Updated weights for policy 1, policy_version 60230 (0.0010) [2023-10-10 18:56:11,946][123614] Updated weights for policy 1, policy_version 60240 (0.0010) [2023-10-10 18:56:12,321][123614] Updated weights for policy 1, policy_version 60250 (0.0008) [2023-10-10 18:56:13,369][123582] Updated weights for policy 0, policy_version 60353 (0.0010) [2023-10-10 18:56:13,739][123582] Updated weights for policy 0, policy_version 60363 (0.0008) [2023-10-10 18:56:13,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 123502592. Throughput: 0: 1809.0, 1: 1817.1. Samples: 30881986. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:56:13,788][122664] Avg episode reward: [(0, '70.070'), (1, '75.810')] [2023-10-10 18:56:14,115][123582] Updated weights for policy 0, policy_version 60373 (0.0008) [2023-10-10 18:56:14,487][123582] Updated weights for policy 0, policy_version 60383 (0.0009) [2023-10-10 18:56:15,944][123614] Updated weights for policy 1, policy_version 60260 (0.0009) [2023-10-10 18:56:16,303][123614] Updated weights for policy 1, policy_version 60270 (0.0009) [2023-10-10 18:56:16,680][123614] Updated weights for policy 1, policy_version 60280 (0.0009) [2023-10-10 18:56:18,238][123582] Updated weights for policy 0, policy_version 60393 (0.0008) [2023-10-10 18:56:18,616][123582] Updated weights for policy 0, policy_version 60403 (0.0007) [2023-10-10 18:56:18,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 123568128. Throughput: 0: 1806.6, 1: 1815.4. Samples: 30903624. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:56:18,789][122664] Avg episode reward: [(0, '73.760'), (1, '72.200')] [2023-10-10 18:56:18,988][123582] Updated weights for policy 0, policy_version 60413 (0.0009) [2023-10-10 18:56:20,474][123614] Updated weights for policy 1, policy_version 60290 (0.0010) [2023-10-10 18:56:20,835][123614] Updated weights for policy 1, policy_version 60300 (0.0009) [2023-10-10 18:56:21,209][123614] Updated weights for policy 1, policy_version 60310 (0.0008) [2023-10-10 18:56:21,580][123614] Updated weights for policy 1, policy_version 60320 (0.0009) [2023-10-10 18:56:22,611][123582] Updated weights for policy 0, policy_version 60423 (0.0009) [2023-10-10 18:56:22,985][123582] Updated weights for policy 0, policy_version 60433 (0.0010) [2023-10-10 18:56:23,361][123582] Updated weights for policy 0, policy_version 60443 (0.0009) [2023-10-10 18:56:23,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 123666432. Throughput: 0: 1808.3, 1: 1819.1. Samples: 30925248. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:56:23,789][122664] Avg episode reward: [(0, '70.750'), (1, '76.800')] [2023-10-10 18:56:25,199][123614] Updated weights for policy 1, policy_version 60330 (0.0007) [2023-10-10 18:56:25,573][123614] Updated weights for policy 1, policy_version 60340 (0.0009) [2023-10-10 18:56:25,944][123614] Updated weights for policy 1, policy_version 60350 (0.0008) [2023-10-10 18:56:27,099][123582] Updated weights for policy 0, policy_version 60453 (0.0009) [2023-10-10 18:56:27,475][123582] Updated weights for policy 0, policy_version 60463 (0.0008) [2023-10-10 18:56:27,844][123582] Updated weights for policy 0, policy_version 60473 (0.0008) [2023-10-10 18:56:28,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 123731968. Throughput: 0: 1810.3, 1: 1820.4. Samples: 30936526. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:56:28,789][122664] Avg episode reward: [(0, '72.840'), (1, '77.240')] [2023-10-10 18:56:29,751][123614] Updated weights for policy 1, policy_version 60360 (0.0008) [2023-10-10 18:56:30,120][123614] Updated weights for policy 1, policy_version 60370 (0.0008) [2023-10-10 18:56:30,480][123614] Updated weights for policy 1, policy_version 60380 (0.0007) [2023-10-10 18:56:31,679][123582] Updated weights for policy 0, policy_version 60483 (0.0009) [2023-10-10 18:56:32,053][123582] Updated weights for policy 0, policy_version 60493 (0.0007) [2023-10-10 18:56:32,429][123582] Updated weights for policy 0, policy_version 60503 (0.0009) [2023-10-10 18:56:33,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 123797504. Throughput: 0: 1811.2, 1: 1819.1. Samples: 30957864. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:56:33,788][122664] Avg episode reward: [(0, '70.230'), (1, '78.030')] [2023-10-10 18:56:33,940][123614] Updated weights for policy 1, policy_version 60390 (0.0007) [2023-10-10 18:56:34,311][123614] Updated weights for policy 1, policy_version 60400 (0.0009) [2023-10-10 18:56:34,689][123614] Updated weights for policy 1, policy_version 60410 (0.0007) [2023-10-10 18:56:36,319][123582] Updated weights for policy 0, policy_version 60513 (0.0009) [2023-10-10 18:56:36,703][123582] Updated weights for policy 0, policy_version 60523 (0.0008) [2023-10-10 18:56:37,075][123582] Updated weights for policy 0, policy_version 60533 (0.0011) [2023-10-10 18:56:37,444][123582] Updated weights for policy 0, policy_version 60543 (0.0009) [2023-10-10 18:56:38,483][123614] Updated weights for policy 1, policy_version 60420 (0.0009) [2023-10-10 18:56:38,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 123863040. Throughput: 0: 1791.0, 1: 1808.9. Samples: 30979260. Policy #0 lag: (min: 2.0, avg: 4.3, max: 29.0) [2023-10-10 18:56:38,789][122664] Avg episode reward: [(0, '70.250'), (1, '74.410')] [2023-10-10 18:56:38,874][123614] Updated weights for policy 1, policy_version 60430 (0.0007) [2023-10-10 18:56:39,239][123614] Updated weights for policy 1, policy_version 60440 (0.0008) [2023-10-10 18:56:41,213][123582] Updated weights for policy 0, policy_version 60553 (0.0009) [2023-10-10 18:56:41,585][123582] Updated weights for policy 0, policy_version 60563 (0.0007) [2023-10-10 18:56:41,958][123582] Updated weights for policy 0, policy_version 60573 (0.0007) [2023-10-10 18:56:42,971][123614] Updated weights for policy 1, policy_version 60450 (0.0010) [2023-10-10 18:56:43,342][123614] Updated weights for policy 1, policy_version 60460 (0.0008) [2023-10-10 18:56:43,710][123614] Updated weights for policy 1, policy_version 60470 (0.0007) [2023-10-10 18:56:43,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 123928576. Throughput: 0: 1807.6, 1: 1808.7. Samples: 30990196. Policy #0 lag: (min: 2.0, avg: 4.3, max: 29.0) [2023-10-10 18:56:43,789][122664] Avg episode reward: [(0, '69.040'), (1, '69.570')] [2023-10-10 18:56:44,078][123614] Updated weights for policy 1, policy_version 60480 (0.0007) [2023-10-10 18:56:45,516][123582] Updated weights for policy 0, policy_version 60583 (0.0010) [2023-10-10 18:56:45,890][123582] Updated weights for policy 0, policy_version 60593 (0.0010) [2023-10-10 18:56:46,267][123582] Updated weights for policy 0, policy_version 60603 (0.0011) [2023-10-10 18:56:47,707][123614] Updated weights for policy 1, policy_version 60490 (0.0010) [2023-10-10 18:56:48,071][123614] Updated weights for policy 1, policy_version 60500 (0.0010) [2023-10-10 18:56:48,444][123614] Updated weights for policy 1, policy_version 60510 (0.0010) [2023-10-10 18:56:48,788][122664] Fps is (10 sec: 16384.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 124026880. Throughput: 0: 1796.0, 1: 1806.2. Samples: 31011812. Policy #0 lag: (min: 2.0, avg: 4.3, max: 29.0) [2023-10-10 18:56:48,788][122664] Avg episode reward: [(0, '63.900'), (1, '67.050')] [2023-10-10 18:56:50,059][123582] Updated weights for policy 0, policy_version 60613 (0.0009) [2023-10-10 18:56:50,434][123582] Updated weights for policy 0, policy_version 60623 (0.0010) [2023-10-10 18:56:50,799][123582] Updated weights for policy 0, policy_version 60633 (0.0009) [2023-10-10 18:56:52,195][123614] Updated weights for policy 1, policy_version 60520 (0.0008) [2023-10-10 18:56:52,577][123614] Updated weights for policy 1, policy_version 60530 (0.0010) [2023-10-10 18:56:52,953][123614] Updated weights for policy 1, policy_version 60540 (0.0011) [2023-10-10 18:56:53,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 124092416. Throughput: 0: 1800.0, 1: 1808.9. Samples: 31033530. Policy #0 lag: (min: 2.0, avg: 4.3, max: 29.0) [2023-10-10 18:56:53,789][122664] Avg episode reward: [(0, '58.930'), (1, '63.920')] [2023-10-10 18:56:54,461][123582] Updated weights for policy 0, policy_version 60643 (0.0010) [2023-10-10 18:56:54,829][123582] Updated weights for policy 0, policy_version 60653 (0.0009) [2023-10-10 18:56:55,200][123582] Updated weights for policy 0, policy_version 60663 (0.0008) [2023-10-10 18:56:56,551][123614] Updated weights for policy 1, policy_version 60550 (0.0009) [2023-10-10 18:56:56,933][123614] Updated weights for policy 1, policy_version 60560 (0.0008) [2023-10-10 18:56:57,293][123614] Updated weights for policy 1, policy_version 60570 (0.0009) [2023-10-10 18:56:58,788][122664] Fps is (10 sec: 13106.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 124157952. Throughput: 0: 1801.0, 1: 1812.8. Samples: 31044610. Policy #0 lag: (min: 2.0, avg: 4.3, max: 29.0) [2023-10-10 18:56:58,789][122664] Avg episode reward: [(0, '58.650'), (1, '64.530')] [2023-10-10 18:56:58,918][123582] Updated weights for policy 0, policy_version 60673 (0.0010) [2023-10-10 18:56:59,283][123582] Updated weights for policy 0, policy_version 60683 (0.0008) [2023-10-10 18:56:59,660][123582] Updated weights for policy 0, policy_version 60693 (0.0009) [2023-10-10 18:57:00,025][123582] Updated weights for policy 0, policy_version 60703 (0.0009) [2023-10-10 18:57:00,986][123614] Updated weights for policy 1, policy_version 60580 (0.0007) [2023-10-10 18:57:01,350][123614] Updated weights for policy 1, policy_version 60590 (0.0008) [2023-10-10 18:57:01,712][123614] Updated weights for policy 1, policy_version 60600 (0.0007) [2023-10-10 18:57:03,777][123582] Updated weights for policy 0, policy_version 60713 (0.0007) [2023-10-10 18:57:03,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 124223488. Throughput: 0: 1803.3, 1: 1813.0. Samples: 31066356. Policy #0 lag: (min: 2.0, avg: 4.3, max: 29.0) [2023-10-10 18:57:03,788][122664] Avg episode reward: [(0, '63.130'), (1, '60.890')] [2023-10-10 18:57:04,149][123582] Updated weights for policy 0, policy_version 60723 (0.0007) [2023-10-10 18:57:04,512][123582] Updated weights for policy 0, policy_version 60733 (0.0008) [2023-10-10 18:57:05,418][123614] Updated weights for policy 1, policy_version 60610 (0.0008) [2023-10-10 18:57:05,788][123614] Updated weights for policy 1, policy_version 60620 (0.0008) [2023-10-10 18:57:06,158][123614] Updated weights for policy 1, policy_version 60630 (0.0007) [2023-10-10 18:57:06,520][123614] Updated weights for policy 1, policy_version 60640 (0.0007) [2023-10-10 18:57:08,003][123582] Updated weights for policy 0, policy_version 60743 (0.0010) [2023-10-10 18:57:08,382][123582] Updated weights for policy 0, policy_version 60753 (0.0009) [2023-10-10 18:57:08,744][123582] Updated weights for policy 0, policy_version 60763 (0.0007) [2023-10-10 18:57:08,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 124289024. Throughput: 0: 1817.7, 1: 1814.5. Samples: 31088698. Policy #0 lag: (min: 2.0, avg: 4.3, max: 29.0) [2023-10-10 18:57:08,789][122664] Avg episode reward: [(0, '60.750'), (1, '63.010')] [2023-10-10 18:57:10,183][123614] Updated weights for policy 1, policy_version 60650 (0.0008) [2023-10-10 18:57:10,560][123614] Updated weights for policy 1, policy_version 60660 (0.0009) [2023-10-10 18:57:10,924][123614] Updated weights for policy 1, policy_version 60670 (0.0008) [2023-10-10 18:57:12,564][123582] Updated weights for policy 0, policy_version 60773 (0.0007) [2023-10-10 18:57:12,929][123582] Updated weights for policy 0, policy_version 60783 (0.0008) [2023-10-10 18:57:13,311][123582] Updated weights for policy 0, policy_version 60793 (0.0009) [2023-10-10 18:57:13,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 124387328. Throughput: 0: 1805.3, 1: 1815.7. Samples: 31099470. Policy #0 lag: (min: 2.0, avg: 4.3, max: 29.0) [2023-10-10 18:57:13,788][122664] Avg episode reward: [(0, '63.240'), (1, '64.730')] [2023-10-10 18:57:14,708][123614] Updated weights for policy 1, policy_version 60680 (0.0008) [2023-10-10 18:57:15,081][123614] Updated weights for policy 1, policy_version 60690 (0.0009) [2023-10-10 18:57:15,444][123614] Updated weights for policy 1, policy_version 60700 (0.0008) [2023-10-10 18:57:16,908][123582] Updated weights for policy 0, policy_version 60803 (0.0009) [2023-10-10 18:57:17,292][123582] Updated weights for policy 0, policy_version 60813 (0.0008) [2023-10-10 18:57:17,662][123582] Updated weights for policy 0, policy_version 60823 (0.0010) [2023-10-10 18:57:18,788][122664] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 124452864. Throughput: 0: 1815.9, 1: 1818.4. Samples: 31121408. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:57:18,789][122664] Avg episode reward: [(0, '63.860'), (1, '66.890')] [2023-10-10 18:57:19,166][123614] Updated weights for policy 1, policy_version 60710 (0.0010) [2023-10-10 18:57:19,538][123614] Updated weights for policy 1, policy_version 60720 (0.0010) [2023-10-10 18:57:19,905][123614] Updated weights for policy 1, policy_version 60730 (0.0010) [2023-10-10 18:57:21,483][123582] Updated weights for policy 0, policy_version 60833 (0.0009) [2023-10-10 18:57:21,861][123582] Updated weights for policy 0, policy_version 60843 (0.0009) [2023-10-10 18:57:22,233][123582] Updated weights for policy 0, policy_version 60853 (0.0008) [2023-10-10 18:57:22,606][123582] Updated weights for policy 0, policy_version 60863 (0.0009) [2023-10-10 18:57:23,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 124518400. Throughput: 0: 1810.1, 1: 1828.3. Samples: 31142986. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:57:23,789][122664] Avg episode reward: [(0, '62.660'), (1, '66.300')] [2023-10-10 18:57:23,835][123614] Updated weights for policy 1, policy_version 60740 (0.0009) [2023-10-10 18:57:24,227][123614] Updated weights for policy 1, policy_version 60750 (0.0008) [2023-10-10 18:57:24,582][123614] Updated weights for policy 1, policy_version 60760 (0.0008) [2023-10-10 18:57:26,288][123582] Updated weights for policy 0, policy_version 60873 (0.0009) [2023-10-10 18:57:26,662][123582] Updated weights for policy 0, policy_version 60883 (0.0008) [2023-10-10 18:57:27,030][123582] Updated weights for policy 0, policy_version 60893 (0.0010) [2023-10-10 18:57:28,408][123614] Updated weights for policy 1, policy_version 60770 (0.0009) [2023-10-10 18:57:28,769][123614] Updated weights for policy 1, policy_version 60780 (0.0007) [2023-10-10 18:57:28,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 124583936. Throughput: 0: 1812.5, 1: 1819.3. Samples: 31153626. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:57:28,788][122664] Avg episode reward: [(0, '66.590'), (1, '65.650')] [2023-10-10 18:57:29,139][123614] Updated weights for policy 1, policy_version 60790 (0.0007) [2023-10-10 18:57:29,515][123614] Updated weights for policy 1, policy_version 60800 (0.0007) [2023-10-10 18:57:30,912][123582] Updated weights for policy 0, policy_version 60903 (0.0007) [2023-10-10 18:57:31,281][123582] Updated weights for policy 0, policy_version 60913 (0.0007) [2023-10-10 18:57:31,655][123582] Updated weights for policy 0, policy_version 60923 (0.0009) [2023-10-10 18:57:33,285][123614] Updated weights for policy 1, policy_version 60810 (0.0007) [2023-10-10 18:57:33,656][123614] Updated weights for policy 1, policy_version 60820 (0.0007) [2023-10-10 18:57:33,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 124649472. Throughput: 0: 1802.7, 1: 1826.1. Samples: 31175106. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:57:33,788][122664] Avg episode reward: [(0, '66.180'), (1, '64.840')] [2023-10-10 18:57:34,023][123614] Updated weights for policy 1, policy_version 60830 (0.0009) [2023-10-10 18:57:35,340][123582] Updated weights for policy 0, policy_version 60933 (0.0009) [2023-10-10 18:57:35,717][123582] Updated weights for policy 0, policy_version 60943 (0.0011) [2023-10-10 18:57:36,100][123582] Updated weights for policy 0, policy_version 60953 (0.0011) [2023-10-10 18:57:37,533][123614] Updated weights for policy 1, policy_version 60840 (0.0010) [2023-10-10 18:57:37,900][123614] Updated weights for policy 1, policy_version 60850 (0.0011) [2023-10-10 18:57:38,271][123614] Updated weights for policy 1, policy_version 60860 (0.0007) [2023-10-10 18:57:38,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 124747776. Throughput: 0: 1800.5, 1: 1816.9. Samples: 31196314. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:57:38,789][122664] Avg episode reward: [(0, '64.690'), (1, '64.160')] [2023-10-10 18:57:39,805][123582] Updated weights for policy 0, policy_version 60963 (0.0010) [2023-10-10 18:57:40,178][123582] Updated weights for policy 0, policy_version 60973 (0.0011) [2023-10-10 18:57:40,550][123582] Updated weights for policy 0, policy_version 60983 (0.0010) [2023-10-10 18:57:41,781][123614] Updated weights for policy 1, policy_version 60870 (0.0010) [2023-10-10 18:57:42,148][123614] Updated weights for policy 1, policy_version 60880 (0.0008) [2023-10-10 18:57:42,510][123614] Updated weights for policy 1, policy_version 60890 (0.0008) [2023-10-10 18:57:43,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 124813312. Throughput: 0: 1797.6, 1: 1820.2. Samples: 31207414. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:57:43,789][122664] Avg episode reward: [(0, '66.600'), (1, '65.560')] [2023-10-10 18:57:44,212][123582] Updated weights for policy 0, policy_version 60993 (0.0010) [2023-10-10 18:57:44,584][123582] Updated weights for policy 0, policy_version 61003 (0.0008) [2023-10-10 18:57:44,956][123582] Updated weights for policy 0, policy_version 61013 (0.0008) [2023-10-10 18:57:45,321][123582] Updated weights for policy 0, policy_version 61023 (0.0008) [2023-10-10 18:57:46,130][123614] Updated weights for policy 1, policy_version 60900 (0.0008) [2023-10-10 18:57:46,504][123614] Updated weights for policy 1, policy_version 60910 (0.0007) [2023-10-10 18:57:46,872][123614] Updated weights for policy 1, policy_version 60920 (0.0008) [2023-10-10 18:57:48,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 124878848. Throughput: 0: 1799.6, 1: 1815.4. Samples: 31229032. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:57:48,789][122664] Avg episode reward: [(0, '67.570'), (1, '70.450')] [2023-10-10 18:57:49,111][123582] Updated weights for policy 0, policy_version 61033 (0.0011) [2023-10-10 18:57:49,478][123582] Updated weights for policy 0, policy_version 61043 (0.0011) [2023-10-10 18:57:49,852][123582] Updated weights for policy 0, policy_version 61053 (0.0010) [2023-10-10 18:57:50,619][123614] Updated weights for policy 1, policy_version 60930 (0.0008) [2023-10-10 18:57:50,974][123614] Updated weights for policy 1, policy_version 60940 (0.0010) [2023-10-10 18:57:51,340][123614] Updated weights for policy 1, policy_version 60950 (0.0009) [2023-10-10 18:57:51,711][123614] Updated weights for policy 1, policy_version 60960 (0.0007) [2023-10-10 18:57:53,485][123582] Updated weights for policy 0, policy_version 61063 (0.0008) [2023-10-10 18:57:53,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 124944384. Throughput: 0: 1805.5, 1: 1812.1. Samples: 31251492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:57:53,789][122664] Avg episode reward: [(0, '67.730'), (1, '67.100')] [2023-10-10 18:57:53,803][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000060960_62423040.pth... [2023-10-10 18:57:53,833][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000059264_60686336.pth [2023-10-10 18:57:53,867][123582] Updated weights for policy 0, policy_version 61073 (0.0009) [2023-10-10 18:57:54,246][123582] Updated weights for policy 0, policy_version 61083 (0.0008) [2023-10-10 18:57:54,426][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000061088_62554112.pth... [2023-10-10 18:57:54,464][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000059392_60817408.pth [2023-10-10 18:57:55,359][123614] Updated weights for policy 1, policy_version 60970 (0.0008) [2023-10-10 18:57:55,725][123614] Updated weights for policy 1, policy_version 60980 (0.0010) [2023-10-10 18:57:56,097][123614] Updated weights for policy 1, policy_version 60990 (0.0009) [2023-10-10 18:57:57,935][123582] Updated weights for policy 0, policy_version 61093 (0.0010) [2023-10-10 18:57:58,311][123582] Updated weights for policy 0, policy_version 61103 (0.0011) [2023-10-10 18:57:58,682][123582] Updated weights for policy 0, policy_version 61113 (0.0010) [2023-10-10 18:57:58,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 125009920. Throughput: 0: 1796.0, 1: 1809.6. Samples: 31261720. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:57:58,789][122664] Avg episode reward: [(0, '66.760'), (1, '67.770')] [2023-10-10 18:58:00,040][123614] Updated weights for policy 1, policy_version 61000 (0.0008) [2023-10-10 18:58:00,410][123614] Updated weights for policy 1, policy_version 61010 (0.0007) [2023-10-10 18:58:00,778][123614] Updated weights for policy 1, policy_version 61020 (0.0008) [2023-10-10 18:58:02,386][123582] Updated weights for policy 0, policy_version 61123 (0.0009) [2023-10-10 18:58:02,759][123582] Updated weights for policy 0, policy_version 61133 (0.0008) [2023-10-10 18:58:03,141][123582] Updated weights for policy 0, policy_version 61143 (0.0011) [2023-10-10 18:58:03,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 125108224. Throughput: 0: 1807.8, 1: 1805.3. Samples: 31284000. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:58:03,788][122664] Avg episode reward: [(0, '64.100'), (1, '64.340')] [2023-10-10 18:58:04,513][123614] Updated weights for policy 1, policy_version 61030 (0.0011) [2023-10-10 18:58:04,883][123614] Updated weights for policy 1, policy_version 61040 (0.0010) [2023-10-10 18:58:05,254][123614] Updated weights for policy 1, policy_version 61050 (0.0009) [2023-10-10 18:58:06,853][123582] Updated weights for policy 0, policy_version 61153 (0.0009) [2023-10-10 18:58:07,216][123582] Updated weights for policy 0, policy_version 61163 (0.0008) [2023-10-10 18:58:07,588][123582] Updated weights for policy 0, policy_version 61173 (0.0007) [2023-10-10 18:58:07,964][123582] Updated weights for policy 0, policy_version 61183 (0.0007) [2023-10-10 18:58:08,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 125173760. Throughput: 0: 1798.1, 1: 1808.0. Samples: 31305258. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:58:08,788][122664] Avg episode reward: [(0, '61.650'), (1, '68.740')] [2023-10-10 18:58:08,995][123614] Updated weights for policy 1, policy_version 61060 (0.0010) [2023-10-10 18:58:09,376][123614] Updated weights for policy 1, policy_version 61070 (0.0008) [2023-10-10 18:58:09,758][123614] Updated weights for policy 1, policy_version 61080 (0.0008) [2023-10-10 18:58:11,818][123582] Updated weights for policy 0, policy_version 61193 (0.0007) [2023-10-10 18:58:12,182][123582] Updated weights for policy 0, policy_version 61203 (0.0007) [2023-10-10 18:58:12,561][123582] Updated weights for policy 0, policy_version 61213 (0.0007) [2023-10-10 18:58:13,410][123614] Updated weights for policy 1, policy_version 61090 (0.0010) [2023-10-10 18:58:13,777][123614] Updated weights for policy 1, policy_version 61100 (0.0010) [2023-10-10 18:58:13,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 125239296. Throughput: 0: 1812.0, 1: 1804.3. Samples: 31316358. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:58:13,789][122664] Avg episode reward: [(0, '61.600'), (1, '68.110')] [2023-10-10 18:58:14,152][123614] Updated weights for policy 1, policy_version 61110 (0.0010) [2023-10-10 18:58:14,507][123614] Updated weights for policy 1, policy_version 61120 (0.0010) [2023-10-10 18:58:16,268][123582] Updated weights for policy 0, policy_version 61223 (0.0008) [2023-10-10 18:58:16,637][123582] Updated weights for policy 0, policy_version 61233 (0.0007) [2023-10-10 18:58:17,011][123582] Updated weights for policy 0, policy_version 61243 (0.0008) [2023-10-10 18:58:18,207][123614] Updated weights for policy 1, policy_version 61130 (0.0009) [2023-10-10 18:58:18,582][123614] Updated weights for policy 1, policy_version 61140 (0.0008) [2023-10-10 18:58:18,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 125304832. Throughput: 0: 1804.0, 1: 1809.1. Samples: 31337696. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:58:18,788][122664] Avg episode reward: [(0, '62.020'), (1, '72.410')] [2023-10-10 18:58:18,941][123614] Updated weights for policy 1, policy_version 61150 (0.0008) [2023-10-10 18:58:20,737][123582] Updated weights for policy 0, policy_version 61253 (0.0010) [2023-10-10 18:58:21,105][123582] Updated weights for policy 0, policy_version 61263 (0.0010) [2023-10-10 18:58:21,469][123582] Updated weights for policy 0, policy_version 61273 (0.0009) [2023-10-10 18:58:22,618][123614] Updated weights for policy 1, policy_version 61160 (0.0009) [2023-10-10 18:58:22,985][123614] Updated weights for policy 1, policy_version 61170 (0.0010) [2023-10-10 18:58:23,349][123614] Updated weights for policy 1, policy_version 61180 (0.0008) [2023-10-10 18:58:23,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 125403136. Throughput: 0: 1810.9, 1: 1808.4. Samples: 31359184. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:58:23,788][122664] Avg episode reward: [(0, '63.380'), (1, '73.780')] [2023-10-10 18:58:25,037][123582] Updated weights for policy 0, policy_version 61283 (0.0007) [2023-10-10 18:58:25,398][123582] Updated weights for policy 0, policy_version 61293 (0.0009) [2023-10-10 18:58:25,780][123582] Updated weights for policy 0, policy_version 61303 (0.0008) [2023-10-10 18:58:26,959][123614] Updated weights for policy 1, policy_version 61190 (0.0008) [2023-10-10 18:58:27,333][123614] Updated weights for policy 1, policy_version 61200 (0.0007) [2023-10-10 18:58:27,694][123614] Updated weights for policy 1, policy_version 61210 (0.0009) [2023-10-10 18:58:28,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 125468672. Throughput: 0: 1818.3, 1: 1813.8. Samples: 31370858. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:58:28,788][122664] Avg episode reward: [(0, '64.250'), (1, '75.710')] [2023-10-10 18:58:29,493][123582] Updated weights for policy 0, policy_version 61313 (0.0007) [2023-10-10 18:58:29,855][123582] Updated weights for policy 0, policy_version 61323 (0.0009) [2023-10-10 18:58:30,226][123582] Updated weights for policy 0, policy_version 61333 (0.0007) [2023-10-10 18:58:30,598][123582] Updated weights for policy 0, policy_version 61343 (0.0009) [2023-10-10 18:58:31,477][123614] Updated weights for policy 1, policy_version 61220 (0.0008) [2023-10-10 18:58:31,845][123614] Updated weights for policy 1, policy_version 61230 (0.0009) [2023-10-10 18:58:32,208][123614] Updated weights for policy 1, policy_version 61240 (0.0010) [2023-10-10 18:58:33,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 125534208. Throughput: 0: 1814.6, 1: 1804.4. Samples: 31391890. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:58:33,789][122664] Avg episode reward: [(0, '58.840'), (1, '74.230')] [2023-10-10 18:58:34,293][123582] Updated weights for policy 0, policy_version 61353 (0.0008) [2023-10-10 18:58:34,662][123582] Updated weights for policy 0, policy_version 61363 (0.0009) [2023-10-10 18:58:35,034][123582] Updated weights for policy 0, policy_version 61373 (0.0009) [2023-10-10 18:58:36,009][123614] Updated weights for policy 1, policy_version 61250 (0.0011) [2023-10-10 18:58:36,369][123614] Updated weights for policy 1, policy_version 61260 (0.0008) [2023-10-10 18:58:36,738][123614] Updated weights for policy 1, policy_version 61270 (0.0008) [2023-10-10 18:58:37,105][123614] Updated weights for policy 1, policy_version 61280 (0.0007) [2023-10-10 18:58:38,621][123582] Updated weights for policy 0, policy_version 61383 (0.0007) [2023-10-10 18:58:38,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 125599744. Throughput: 0: 1825.7, 1: 1801.8. Samples: 31414730. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 18:58:38,788][122664] Avg episode reward: [(0, '58.510'), (1, '75.190')] [2023-10-10 18:58:38,988][123582] Updated weights for policy 0, policy_version 61393 (0.0008) [2023-10-10 18:58:39,358][123582] Updated weights for policy 0, policy_version 61403 (0.0009) [2023-10-10 18:58:40,740][123614] Updated weights for policy 1, policy_version 61290 (0.0009) [2023-10-10 18:58:41,111][123614] Updated weights for policy 1, policy_version 61300 (0.0009) [2023-10-10 18:58:41,479][123614] Updated weights for policy 1, policy_version 61310 (0.0010) [2023-10-10 18:58:43,000][123582] Updated weights for policy 0, policy_version 61413 (0.0008) [2023-10-10 18:58:43,369][123582] Updated weights for policy 0, policy_version 61423 (0.0008) [2023-10-10 18:58:43,740][123582] Updated weights for policy 0, policy_version 61433 (0.0009) [2023-10-10 18:58:43,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 125665280. Throughput: 0: 1818.1, 1: 1803.8. Samples: 31424708. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 18:58:43,789][122664] Avg episode reward: [(0, '58.480'), (1, '73.730')] [2023-10-10 18:58:45,218][123614] Updated weights for policy 1, policy_version 61320 (0.0010) [2023-10-10 18:58:45,588][123614] Updated weights for policy 1, policy_version 61330 (0.0008) [2023-10-10 18:58:45,962][123614] Updated weights for policy 1, policy_version 61340 (0.0007) [2023-10-10 18:58:47,419][123582] Updated weights for policy 0, policy_version 61443 (0.0009) [2023-10-10 18:58:47,791][123582] Updated weights for policy 0, policy_version 61453 (0.0008) [2023-10-10 18:58:48,157][123582] Updated weights for policy 0, policy_version 61463 (0.0009) [2023-10-10 18:58:48,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 125763584. Throughput: 0: 1827.1, 1: 1809.2. Samples: 31447632. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 18:58:48,789][122664] Avg episode reward: [(0, '60.760'), (1, '74.830')] [2023-10-10 18:58:49,677][123614] Updated weights for policy 1, policy_version 61350 (0.0008) [2023-10-10 18:58:50,042][123614] Updated weights for policy 1, policy_version 61360 (0.0008) [2023-10-10 18:58:50,417][123614] Updated weights for policy 1, policy_version 61370 (0.0009) [2023-10-10 18:58:51,879][123582] Updated weights for policy 0, policy_version 61473 (0.0008) [2023-10-10 18:58:52,264][123582] Updated weights for policy 0, policy_version 61483 (0.0009) [2023-10-10 18:58:52,627][123582] Updated weights for policy 0, policy_version 61493 (0.0010) [2023-10-10 18:58:53,005][123582] Updated weights for policy 0, policy_version 61503 (0.0009) [2023-10-10 18:58:53,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 125829120. Throughput: 0: 1826.7, 1: 1817.1. Samples: 31469228. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 18:58:53,789][122664] Avg episode reward: [(0, '60.850'), (1, '73.600')] [2023-10-10 18:58:54,193][123614] Updated weights for policy 1, policy_version 61380 (0.0008) [2023-10-10 18:58:54,581][123614] Updated weights for policy 1, policy_version 61390 (0.0010) [2023-10-10 18:58:54,950][123614] Updated weights for policy 1, policy_version 61400 (0.0010) [2023-10-10 18:58:56,662][123582] Updated weights for policy 0, policy_version 61513 (0.0011) [2023-10-10 18:58:57,028][123582] Updated weights for policy 0, policy_version 61523 (0.0011) [2023-10-10 18:58:57,406][123582] Updated weights for policy 0, policy_version 61533 (0.0007) [2023-10-10 18:58:58,631][123614] Updated weights for policy 1, policy_version 61410 (0.0009) [2023-10-10 18:58:58,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 125894656. Throughput: 0: 1828.5, 1: 1816.9. Samples: 31480400. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 18:58:58,788][122664] Avg episode reward: [(0, '61.700'), (1, '75.990')] [2023-10-10 18:58:59,006][123614] Updated weights for policy 1, policy_version 61420 (0.0011) [2023-10-10 18:58:59,367][123614] Updated weights for policy 1, policy_version 61430 (0.0008) [2023-10-10 18:58:59,732][123614] Updated weights for policy 1, policy_version 61440 (0.0007) [2023-10-10 18:59:01,046][123582] Updated weights for policy 0, policy_version 61543 (0.0008) [2023-10-10 18:59:01,412][123582] Updated weights for policy 0, policy_version 61553 (0.0011) [2023-10-10 18:59:01,787][123582] Updated weights for policy 0, policy_version 61563 (0.0007) [2023-10-10 18:59:03,349][123614] Updated weights for policy 1, policy_version 61450 (0.0008) [2023-10-10 18:59:03,713][123614] Updated weights for policy 1, policy_version 61460 (0.0010) [2023-10-10 18:59:03,788][122664] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 125960192. Throughput: 0: 1835.7, 1: 1814.8. Samples: 31501966. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 18:59:03,788][122664] Avg episode reward: [(0, '63.450'), (1, '76.510')] [2023-10-10 18:59:04,083][123614] Updated weights for policy 1, policy_version 61470 (0.0010) [2023-10-10 18:59:05,515][123582] Updated weights for policy 0, policy_version 61573 (0.0009) [2023-10-10 18:59:05,885][123582] Updated weights for policy 0, policy_version 61583 (0.0011) [2023-10-10 18:59:06,268][123582] Updated weights for policy 0, policy_version 61593 (0.0012) [2023-10-10 18:59:08,075][123614] Updated weights for policy 1, policy_version 61480 (0.0009) [2023-10-10 18:59:08,443][123614] Updated weights for policy 1, policy_version 61490 (0.0007) [2023-10-10 18:59:08,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 126025728. Throughput: 0: 1828.5, 1: 1819.7. Samples: 31523354. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 18:59:08,789][122664] Avg episode reward: [(0, '63.760'), (1, '74.090')] [2023-10-10 18:59:08,803][123614] Updated weights for policy 1, policy_version 61500 (0.0009) [2023-10-10 18:59:09,972][123582] Updated weights for policy 0, policy_version 61603 (0.0010) [2023-10-10 18:59:10,359][123582] Updated weights for policy 0, policy_version 61613 (0.0009) [2023-10-10 18:59:10,722][123582] Updated weights for policy 0, policy_version 61623 (0.0009) [2023-10-10 18:59:12,509][123614] Updated weights for policy 1, policy_version 61510 (0.0009) [2023-10-10 18:59:12,877][123614] Updated weights for policy 1, policy_version 61520 (0.0009) [2023-10-10 18:59:13,248][123614] Updated weights for policy 1, policy_version 61530 (0.0010) [2023-10-10 18:59:13,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 126124032. Throughput: 0: 1823.1, 1: 1804.9. Samples: 31534118. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 18:59:13,789][122664] Avg episode reward: [(0, '66.150'), (1, '72.030')] [2023-10-10 18:59:14,321][123582] Updated weights for policy 0, policy_version 61633 (0.0008) [2023-10-10 18:59:14,686][123582] Updated weights for policy 0, policy_version 61643 (0.0010) [2023-10-10 18:59:15,054][123582] Updated weights for policy 0, policy_version 61653 (0.0007) [2023-10-10 18:59:15,427][123582] Updated weights for policy 0, policy_version 61663 (0.0008) [2023-10-10 18:59:17,135][123614] Updated weights for policy 1, policy_version 61540 (0.0010) [2023-10-10 18:59:17,512][123614] Updated weights for policy 1, policy_version 61550 (0.0009) [2023-10-10 18:59:17,875][123614] Updated weights for policy 1, policy_version 61560 (0.0008) [2023-10-10 18:59:18,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 126189568. Throughput: 0: 1826.4, 1: 1821.8. Samples: 31556058. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 18:59:18,789][122664] Avg episode reward: [(0, '65.700'), (1, '65.970')] [2023-10-10 18:59:19,076][123582] Updated weights for policy 0, policy_version 61673 (0.0010) [2023-10-10 18:59:19,447][123582] Updated weights for policy 0, policy_version 61683 (0.0009) [2023-10-10 18:59:19,822][123582] Updated weights for policy 0, policy_version 61693 (0.0009) [2023-10-10 18:59:21,359][123614] Updated weights for policy 1, policy_version 61570 (0.0008) [2023-10-10 18:59:21,730][123614] Updated weights for policy 1, policy_version 61580 (0.0008) [2023-10-10 18:59:22,101][123614] Updated weights for policy 1, policy_version 61590 (0.0009) [2023-10-10 18:59:22,475][123614] Updated weights for policy 1, policy_version 61600 (0.0008) [2023-10-10 18:59:23,506][123582] Updated weights for policy 0, policy_version 61703 (0.0008) [2023-10-10 18:59:23,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 126255104. Throughput: 0: 1818.9, 1: 1813.2. Samples: 31578174. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-10 18:59:23,788][122664] Avg episode reward: [(0, '65.100'), (1, '68.470')] [2023-10-10 18:59:23,876][123582] Updated weights for policy 0, policy_version 61713 (0.0010) [2023-10-10 18:59:24,250][123582] Updated weights for policy 0, policy_version 61723 (0.0008) [2023-10-10 18:59:26,094][123614] Updated weights for policy 1, policy_version 61610 (0.0009) [2023-10-10 18:59:26,460][123614] Updated weights for policy 1, policy_version 61620 (0.0010) [2023-10-10 18:59:26,829][123614] Updated weights for policy 1, policy_version 61630 (0.0010) [2023-10-10 18:59:27,867][123582] Updated weights for policy 0, policy_version 61733 (0.0010) [2023-10-10 18:59:28,234][123582] Updated weights for policy 0, policy_version 61743 (0.0008) [2023-10-10 18:59:28,610][123582] Updated weights for policy 0, policy_version 61753 (0.0008) [2023-10-10 18:59:28,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 126320640. Throughput: 0: 1824.0, 1: 1818.4. Samples: 31588618. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-10 18:59:28,789][122664] Avg episode reward: [(0, '65.460'), (1, '65.690')] [2023-10-10 18:59:30,591][123614] Updated weights for policy 1, policy_version 61640 (0.0008) [2023-10-10 18:59:30,960][123614] Updated weights for policy 1, policy_version 61650 (0.0009) [2023-10-10 18:59:31,321][123614] Updated weights for policy 1, policy_version 61660 (0.0007) [2023-10-10 18:59:32,396][123582] Updated weights for policy 0, policy_version 61763 (0.0010) [2023-10-10 18:59:32,768][123582] Updated weights for policy 0, policy_version 61773 (0.0009) [2023-10-10 18:59:33,134][123582] Updated weights for policy 0, policy_version 61783 (0.0011) [2023-10-10 18:59:33,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 126418944. Throughput: 0: 1817.6, 1: 1811.2. Samples: 31610930. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-10 18:59:33,789][122664] Avg episode reward: [(0, '66.780'), (1, '63.310')] [2023-10-10 18:59:35,031][123614] Updated weights for policy 1, policy_version 61670 (0.0009) [2023-10-10 18:59:35,392][123614] Updated weights for policy 1, policy_version 61680 (0.0007) [2023-10-10 18:59:35,763][123614] Updated weights for policy 1, policy_version 61690 (0.0008) [2023-10-10 18:59:36,910][123582] Updated weights for policy 0, policy_version 61793 (0.0009) [2023-10-10 18:59:37,281][123582] Updated weights for policy 0, policy_version 61803 (0.0011) [2023-10-10 18:59:37,644][123582] Updated weights for policy 0, policy_version 61813 (0.0010) [2023-10-10 18:59:38,020][123582] Updated weights for policy 0, policy_version 61823 (0.0008) [2023-10-10 18:59:38,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 126484480. Throughput: 0: 1818.2, 1: 1808.5. Samples: 31632428. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-10 18:59:38,788][122664] Avg episode reward: [(0, '71.000'), (1, '61.030')] [2023-10-10 18:59:39,421][123614] Updated weights for policy 1, policy_version 61700 (0.0009) [2023-10-10 18:59:39,810][123614] Updated weights for policy 1, policy_version 61710 (0.0009) [2023-10-10 18:59:40,176][123614] Updated weights for policy 1, policy_version 61720 (0.0007) [2023-10-10 18:59:41,731][123582] Updated weights for policy 0, policy_version 61833 (0.0008) [2023-10-10 18:59:42,108][123582] Updated weights for policy 0, policy_version 61843 (0.0010) [2023-10-10 18:59:42,470][123582] Updated weights for policy 0, policy_version 61853 (0.0010) [2023-10-10 18:59:43,706][123614] Updated weights for policy 1, policy_version 61730 (0.0007) [2023-10-10 18:59:43,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 126550016. Throughput: 0: 1818.3, 1: 1814.7. Samples: 31643882. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-10 18:59:43,789][122664] Avg episode reward: [(0, '71.600'), (1, '60.600')] [2023-10-10 18:59:44,071][123614] Updated weights for policy 1, policy_version 61740 (0.0007) [2023-10-10 18:59:44,445][123614] Updated weights for policy 1, policy_version 61750 (0.0009) [2023-10-10 18:59:44,809][123614] Updated weights for policy 1, policy_version 61760 (0.0008) [2023-10-10 18:59:46,117][123582] Updated weights for policy 0, policy_version 61863 (0.0009) [2023-10-10 18:59:46,497][123582] Updated weights for policy 0, policy_version 61873 (0.0010) [2023-10-10 18:59:46,867][123582] Updated weights for policy 0, policy_version 61883 (0.0008) [2023-10-10 18:59:48,597][123614] Updated weights for policy 1, policy_version 61770 (0.0007) [2023-10-10 18:59:48,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 126615552. Throughput: 0: 1810.9, 1: 1817.6. Samples: 31665250. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-10 18:59:48,788][122664] Avg episode reward: [(0, '71.690'), (1, '61.590')] [2023-10-10 18:59:48,964][123614] Updated weights for policy 1, policy_version 61780 (0.0010) [2023-10-10 18:59:49,332][123614] Updated weights for policy 1, policy_version 61790 (0.0009) [2023-10-10 18:59:50,756][123582] Updated weights for policy 0, policy_version 61893 (0.0007) [2023-10-10 18:59:51,131][123582] Updated weights for policy 0, policy_version 61903 (0.0009) [2023-10-10 18:59:51,505][123582] Updated weights for policy 0, policy_version 61913 (0.0008) [2023-10-10 18:59:52,968][123614] Updated weights for policy 1, policy_version 61800 (0.0008) [2023-10-10 18:59:53,334][123614] Updated weights for policy 1, policy_version 61810 (0.0008) [2023-10-10 18:59:53,699][123614] Updated weights for policy 1, policy_version 61820 (0.0007) [2023-10-10 18:59:53,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 126681088. Throughput: 0: 1814.0, 1: 1819.9. Samples: 31686876. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-10 18:59:53,788][122664] Avg episode reward: [(0, '72.060'), (1, '62.760')] [2023-10-10 18:59:53,794][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000061920_63406080.pth... [2023-10-10 18:59:53,833][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000060224_61669376.pth [2023-10-10 18:59:53,836][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000061824_63307776.pth... [2023-10-10 18:59:53,865][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000060128_61571072.pth [2023-10-10 18:59:55,096][123582] Updated weights for policy 0, policy_version 61923 (0.0008) [2023-10-10 18:59:55,462][123582] Updated weights for policy 0, policy_version 61933 (0.0007) [2023-10-10 18:59:55,831][123582] Updated weights for policy 0, policy_version 61943 (0.0007) [2023-10-10 18:59:57,346][123614] Updated weights for policy 1, policy_version 61830 (0.0009) [2023-10-10 18:59:57,714][123614] Updated weights for policy 1, policy_version 61840 (0.0008) [2023-10-10 18:59:58,088][123614] Updated weights for policy 1, policy_version 61850 (0.0008) [2023-10-10 18:59:58,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 126779392. Throughput: 0: 1813.7, 1: 1826.8. Samples: 31697940. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-10 18:59:58,789][122664] Avg episode reward: [(0, '73.310'), (1, '60.190')] [2023-10-10 18:59:59,499][123582] Updated weights for policy 0, policy_version 61953 (0.0009) [2023-10-10 18:59:59,867][123582] Updated weights for policy 0, policy_version 61963 (0.0010) [2023-10-10 19:00:00,246][123582] Updated weights for policy 0, policy_version 61973 (0.0007) [2023-10-10 19:00:00,628][123582] Updated weights for policy 0, policy_version 61983 (0.0009) [2023-10-10 19:00:01,796][123614] Updated weights for policy 1, policy_version 61860 (0.0009) [2023-10-10 19:00:02,164][123614] Updated weights for policy 1, policy_version 61870 (0.0009) [2023-10-10 19:00:02,534][123614] Updated weights for policy 1, policy_version 61880 (0.0009) [2023-10-10 19:00:03,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 126844928. Throughput: 0: 1819.6, 1: 1817.6. Samples: 31719728. Policy #0 lag: (min: 7.0, avg: 21.3, max: 39.0) [2023-10-10 19:00:03,788][122664] Avg episode reward: [(0, '71.560'), (1, '59.710')] [2023-10-10 19:00:04,158][123582] Updated weights for policy 0, policy_version 61993 (0.0008) [2023-10-10 19:00:04,529][123582] Updated weights for policy 0, policy_version 62003 (0.0008) [2023-10-10 19:00:04,908][123582] Updated weights for policy 0, policy_version 62013 (0.0011) [2023-10-10 19:00:06,146][123614] Updated weights for policy 1, policy_version 61890 (0.0008) [2023-10-10 19:00:06,514][123614] Updated weights for policy 1, policy_version 61900 (0.0010) [2023-10-10 19:00:06,882][123614] Updated weights for policy 1, policy_version 61910 (0.0009) [2023-10-10 19:00:07,250][123614] Updated weights for policy 1, policy_version 61920 (0.0008) [2023-10-10 19:00:08,643][123582] Updated weights for policy 0, policy_version 62023 (0.0009) [2023-10-10 19:00:08,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 126910464. Throughput: 0: 1821.5, 1: 1829.7. Samples: 31742478. Policy #0 lag: (min: 7.0, avg: 21.3, max: 39.0) [2023-10-10 19:00:08,789][122664] Avg episode reward: [(0, '72.790'), (1, '62.340')] [2023-10-10 19:00:09,020][123582] Updated weights for policy 0, policy_version 62033 (0.0010) [2023-10-10 19:00:09,394][123582] Updated weights for policy 0, policy_version 62043 (0.0011) [2023-10-10 19:00:10,905][123614] Updated weights for policy 1, policy_version 61930 (0.0010) [2023-10-10 19:00:11,272][123614] Updated weights for policy 1, policy_version 61940 (0.0009) [2023-10-10 19:00:11,633][123614] Updated weights for policy 1, policy_version 61950 (0.0010) [2023-10-10 19:00:13,262][123582] Updated weights for policy 0, policy_version 62053 (0.0010) [2023-10-10 19:00:13,631][123582] Updated weights for policy 0, policy_version 62063 (0.0007) [2023-10-10 19:00:13,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 126976000. Throughput: 0: 1813.6, 1: 1826.9. Samples: 31752444. Policy #0 lag: (min: 7.0, avg: 21.3, max: 39.0) [2023-10-10 19:00:13,789][122664] Avg episode reward: [(0, '74.660'), (1, '61.770')] [2023-10-10 19:00:14,003][123582] Updated weights for policy 0, policy_version 62073 (0.0007) [2023-10-10 19:00:15,344][123614] Updated weights for policy 1, policy_version 61960 (0.0008) [2023-10-10 19:00:15,713][123614] Updated weights for policy 1, policy_version 61970 (0.0007) [2023-10-10 19:00:16,079][123614] Updated weights for policy 1, policy_version 61980 (0.0007) [2023-10-10 19:00:17,473][123582] Updated weights for policy 0, policy_version 62083 (0.0009) [2023-10-10 19:00:17,839][123582] Updated weights for policy 0, policy_version 62093 (0.0007) [2023-10-10 19:00:18,209][123582] Updated weights for policy 0, policy_version 62103 (0.0009) [2023-10-10 19:00:18,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 127074304. Throughput: 0: 1818.2, 1: 1831.5. Samples: 31775168. Policy #0 lag: (min: 7.0, avg: 21.3, max: 39.0) [2023-10-10 19:00:18,788][122664] Avg episode reward: [(0, '72.940'), (1, '65.030')] [2023-10-10 19:00:19,774][123614] Updated weights for policy 1, policy_version 61990 (0.0007) [2023-10-10 19:00:20,152][123614] Updated weights for policy 1, policy_version 62000 (0.0007) [2023-10-10 19:00:20,514][123614] Updated weights for policy 1, policy_version 62010 (0.0010) [2023-10-10 19:00:21,686][123582] Updated weights for policy 0, policy_version 62113 (0.0008) [2023-10-10 19:00:22,060][123582] Updated weights for policy 0, policy_version 62123 (0.0008) [2023-10-10 19:00:22,442][123582] Updated weights for policy 0, policy_version 62133 (0.0009) [2023-10-10 19:00:22,826][123582] Updated weights for policy 0, policy_version 62143 (0.0009) [2023-10-10 19:00:23,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 127139840. Throughput: 0: 1820.1, 1: 1829.6. Samples: 31796668. Policy #0 lag: (min: 7.0, avg: 21.3, max: 39.0) [2023-10-10 19:00:23,789][122664] Avg episode reward: [(0, '73.770'), (1, '69.730')] [2023-10-10 19:00:24,286][123614] Updated weights for policy 1, policy_version 62020 (0.0009) [2023-10-10 19:00:24,672][123614] Updated weights for policy 1, policy_version 62030 (0.0009) [2023-10-10 19:00:25,039][123614] Updated weights for policy 1, policy_version 62040 (0.0008) [2023-10-10 19:00:26,643][123582] Updated weights for policy 0, policy_version 62153 (0.0007) [2023-10-10 19:00:27,017][123582] Updated weights for policy 0, policy_version 62163 (0.0009) [2023-10-10 19:00:27,392][123582] Updated weights for policy 0, policy_version 62173 (0.0010) [2023-10-10 19:00:28,604][123614] Updated weights for policy 1, policy_version 62050 (0.0008) [2023-10-10 19:00:28,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 127205376. Throughput: 0: 1814.6, 1: 1825.1. Samples: 31807668. Policy #0 lag: (min: 7.0, avg: 21.3, max: 39.0) [2023-10-10 19:00:28,788][122664] Avg episode reward: [(0, '72.020'), (1, '73.240')] [2023-10-10 19:00:28,973][123614] Updated weights for policy 1, policy_version 62060 (0.0009) [2023-10-10 19:00:29,326][123614] Updated weights for policy 1, policy_version 62070 (0.0007) [2023-10-10 19:00:29,696][123614] Updated weights for policy 1, policy_version 62080 (0.0008) [2023-10-10 19:00:31,052][123582] Updated weights for policy 0, policy_version 62183 (0.0007) [2023-10-10 19:00:31,415][123582] Updated weights for policy 0, policy_version 62193 (0.0008) [2023-10-10 19:00:31,790][123582] Updated weights for policy 0, policy_version 62203 (0.0008) [2023-10-10 19:00:33,297][123614] Updated weights for policy 1, policy_version 62090 (0.0010) [2023-10-10 19:00:33,668][123614] Updated weights for policy 1, policy_version 62100 (0.0008) [2023-10-10 19:00:33,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 127270912. Throughput: 0: 1821.6, 1: 1825.3. Samples: 31829362. Policy #0 lag: (min: 7.0, avg: 21.3, max: 39.0) [2023-10-10 19:00:33,789][122664] Avg episode reward: [(0, '75.140'), (1, '74.560')] [2023-10-10 19:00:34,045][123614] Updated weights for policy 1, policy_version 62110 (0.0007) [2023-10-10 19:00:35,609][123582] Updated weights for policy 0, policy_version 62213 (0.0007) [2023-10-10 19:00:36,005][123582] Updated weights for policy 0, policy_version 62223 (0.0009) [2023-10-10 19:00:36,368][123582] Updated weights for policy 0, policy_version 62233 (0.0008) [2023-10-10 19:00:37,627][123614] Updated weights for policy 1, policy_version 62120 (0.0008) [2023-10-10 19:00:37,991][123614] Updated weights for policy 1, policy_version 62130 (0.0009) [2023-10-10 19:00:38,358][123614] Updated weights for policy 1, policy_version 62140 (0.0010) [2023-10-10 19:00:38,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 127369216. Throughput: 0: 1816.1, 1: 1821.3. Samples: 31850560. Policy #0 lag: (min: 7.0, avg: 21.3, max: 39.0) [2023-10-10 19:00:38,789][122664] Avg episode reward: [(0, '74.150'), (1, '76.660')] [2023-10-10 19:00:40,186][123582] Updated weights for policy 0, policy_version 62243 (0.0009) [2023-10-10 19:00:40,562][123582] Updated weights for policy 0, policy_version 62253 (0.0011) [2023-10-10 19:00:40,928][123582] Updated weights for policy 0, policy_version 62263 (0.0009) [2023-10-10 19:00:42,148][123614] Updated weights for policy 1, policy_version 62150 (0.0008) [2023-10-10 19:00:42,518][123614] Updated weights for policy 1, policy_version 62160 (0.0009) [2023-10-10 19:00:42,891][123614] Updated weights for policy 1, policy_version 62170 (0.0009) [2023-10-10 19:00:43,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 127434752. Throughput: 0: 1812.8, 1: 1822.1. Samples: 31861510. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:00:43,788][122664] Avg episode reward: [(0, '69.320'), (1, '76.020')] [2023-10-10 19:00:44,710][123582] Updated weights for policy 0, policy_version 62273 (0.0009) [2023-10-10 19:00:45,070][123582] Updated weights for policy 0, policy_version 62283 (0.0009) [2023-10-10 19:00:45,448][123582] Updated weights for policy 0, policy_version 62293 (0.0010) [2023-10-10 19:00:45,818][123582] Updated weights for policy 0, policy_version 62303 (0.0008) [2023-10-10 19:00:46,559][123614] Updated weights for policy 1, policy_version 62180 (0.0008) [2023-10-10 19:00:46,934][123614] Updated weights for policy 1, policy_version 62190 (0.0009) [2023-10-10 19:00:47,310][123614] Updated weights for policy 1, policy_version 62200 (0.0007) [2023-10-10 19:00:48,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 127500288. Throughput: 0: 1804.5, 1: 1816.0. Samples: 31882652. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:00:48,789][122664] Avg episode reward: [(0, '67.320'), (1, '74.470')] [2023-10-10 19:00:49,520][123582] Updated weights for policy 0, policy_version 62313 (0.0008) [2023-10-10 19:00:49,901][123582] Updated weights for policy 0, policy_version 62323 (0.0009) [2023-10-10 19:00:50,270][123582] Updated weights for policy 0, policy_version 62333 (0.0009) [2023-10-10 19:00:51,118][123614] Updated weights for policy 1, policy_version 62210 (0.0009) [2023-10-10 19:00:51,486][123614] Updated weights for policy 1, policy_version 62220 (0.0010) [2023-10-10 19:00:51,860][123614] Updated weights for policy 1, policy_version 62230 (0.0008) [2023-10-10 19:00:52,220][123614] Updated weights for policy 1, policy_version 62240 (0.0010) [2023-10-10 19:00:53,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 127565824. Throughput: 0: 1799.8, 1: 1811.3. Samples: 31904978. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:00:53,788][122664] Avg episode reward: [(0, '67.850'), (1, '79.080')] [2023-10-10 19:00:54,046][123582] Updated weights for policy 0, policy_version 62343 (0.0009) [2023-10-10 19:00:54,414][123582] Updated weights for policy 0, policy_version 62353 (0.0008) [2023-10-10 19:00:54,788][123582] Updated weights for policy 0, policy_version 62363 (0.0007) [2023-10-10 19:00:56,080][123614] Updated weights for policy 1, policy_version 62250 (0.0008) [2023-10-10 19:00:56,435][123614] Updated weights for policy 1, policy_version 62260 (0.0008) [2023-10-10 19:00:56,806][123614] Updated weights for policy 1, policy_version 62270 (0.0008) [2023-10-10 19:00:58,530][123582] Updated weights for policy 0, policy_version 62373 (0.0008) [2023-10-10 19:00:58,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 127631360. Throughput: 0: 1801.2, 1: 1812.4. Samples: 31915054. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:00:58,789][122664] Avg episode reward: [(0, '64.890'), (1, '80.930')] [2023-10-10 19:00:58,890][123582] Updated weights for policy 0, policy_version 62383 (0.0007) [2023-10-10 19:00:59,255][123582] Updated weights for policy 0, policy_version 62393 (0.0008) [2023-10-10 19:01:00,672][123614] Updated weights for policy 1, policy_version 62280 (0.0007) [2023-10-10 19:01:01,037][123614] Updated weights for policy 1, policy_version 62290 (0.0008) [2023-10-10 19:01:01,413][123614] Updated weights for policy 1, policy_version 62300 (0.0009) [2023-10-10 19:01:02,837][123582] Updated weights for policy 0, policy_version 62403 (0.0008) [2023-10-10 19:01:03,205][123582] Updated weights for policy 0, policy_version 62413 (0.0007) [2023-10-10 19:01:03,578][123582] Updated weights for policy 0, policy_version 62423 (0.0008) [2023-10-10 19:01:03,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 127696896. Throughput: 0: 1808.3, 1: 1798.6. Samples: 31937478. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:01:03,789][122664] Avg episode reward: [(0, '62.150'), (1, '81.520')] [2023-10-10 19:01:05,194][123614] Updated weights for policy 1, policy_version 62310 (0.0009) [2023-10-10 19:01:05,573][123614] Updated weights for policy 1, policy_version 62320 (0.0007) [2023-10-10 19:01:05,937][123614] Updated weights for policy 1, policy_version 62330 (0.0010) [2023-10-10 19:01:07,194][123582] Updated weights for policy 0, policy_version 62433 (0.0010) [2023-10-10 19:01:07,569][123582] Updated weights for policy 0, policy_version 62443 (0.0009) [2023-10-10 19:01:07,933][123582] Updated weights for policy 0, policy_version 62453 (0.0009) [2023-10-10 19:01:08,312][123582] Updated weights for policy 0, policy_version 62463 (0.0008) [2023-10-10 19:01:08,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 127795200. Throughput: 0: 1811.1, 1: 1797.3. Samples: 31959046. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:01:08,789][122664] Avg episode reward: [(0, '63.340'), (1, '80.340')] [2023-10-10 19:01:09,807][123614] Updated weights for policy 1, policy_version 62340 (0.0011) [2023-10-10 19:01:10,179][123614] Updated weights for policy 1, policy_version 62350 (0.0010) [2023-10-10 19:01:10,552][123614] Updated weights for policy 1, policy_version 62360 (0.0011) [2023-10-10 19:01:11,962][123582] Updated weights for policy 0, policy_version 62473 (0.0011) [2023-10-10 19:01:12,330][123582] Updated weights for policy 0, policy_version 62483 (0.0011) [2023-10-10 19:01:12,706][123582] Updated weights for policy 0, policy_version 62493 (0.0010) [2023-10-10 19:01:13,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 127860736. Throughput: 0: 1818.1, 1: 1795.6. Samples: 31970288. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:01:13,789][122664] Avg episode reward: [(0, '59.360'), (1, '76.760')] [2023-10-10 19:01:14,187][123614] Updated weights for policy 1, policy_version 62370 (0.0009) [2023-10-10 19:01:14,552][123614] Updated weights for policy 1, policy_version 62380 (0.0009) [2023-10-10 19:01:14,912][123614] Updated weights for policy 1, policy_version 62390 (0.0008) [2023-10-10 19:01:15,278][123614] Updated weights for policy 1, policy_version 62400 (0.0008) [2023-10-10 19:01:16,493][123582] Updated weights for policy 0, policy_version 62503 (0.0009) [2023-10-10 19:01:16,867][123582] Updated weights for policy 0, policy_version 62513 (0.0008) [2023-10-10 19:01:17,232][123582] Updated weights for policy 0, policy_version 62523 (0.0009) [2023-10-10 19:01:18,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 127926272. Throughput: 0: 1810.3, 1: 1796.7. Samples: 31991676. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:01:18,789][122664] Avg episode reward: [(0, '59.390'), (1, '74.160')] [2023-10-10 19:01:18,932][123614] Updated weights for policy 1, policy_version 62410 (0.0009) [2023-10-10 19:01:19,309][123614] Updated weights for policy 1, policy_version 62420 (0.0008) [2023-10-10 19:01:19,677][123614] Updated weights for policy 1, policy_version 62430 (0.0007) [2023-10-10 19:01:20,995][123582] Updated weights for policy 0, policy_version 62533 (0.0008) [2023-10-10 19:01:21,375][123582] Updated weights for policy 0, policy_version 62543 (0.0008) [2023-10-10 19:01:21,747][123582] Updated weights for policy 0, policy_version 62553 (0.0007) [2023-10-10 19:01:23,213][123614] Updated weights for policy 1, policy_version 62440 (0.0009) [2023-10-10 19:01:23,581][123614] Updated weights for policy 1, policy_version 62450 (0.0010) [2023-10-10 19:01:23,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 127991808. Throughput: 0: 1813.2, 1: 1808.1. Samples: 32013522. Policy #0 lag: (min: 17.0, avg: 36.1, max: 49.0) [2023-10-10 19:01:23,789][122664] Avg episode reward: [(0, '60.020'), (1, '73.270')] [2023-10-10 19:01:23,946][123614] Updated weights for policy 1, policy_version 62460 (0.0010) [2023-10-10 19:01:25,314][123582] Updated weights for policy 0, policy_version 62563 (0.0007) [2023-10-10 19:01:25,686][123582] Updated weights for policy 0, policy_version 62573 (0.0007) [2023-10-10 19:01:26,062][123582] Updated weights for policy 0, policy_version 62583 (0.0009) [2023-10-10 19:01:27,639][123614] Updated weights for policy 1, policy_version 62470 (0.0009) [2023-10-10 19:01:28,010][123614] Updated weights for policy 1, policy_version 62480 (0.0011) [2023-10-10 19:01:28,381][123614] Updated weights for policy 1, policy_version 62490 (0.0009) [2023-10-10 19:01:28,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 128090112. Throughput: 0: 1818.8, 1: 1800.5. Samples: 32024382. Policy #0 lag: (min: 17.0, avg: 36.1, max: 49.0) [2023-10-10 19:01:28,788][122664] Avg episode reward: [(0, '59.490'), (1, '69.490')] [2023-10-10 19:01:29,771][123582] Updated weights for policy 0, policy_version 62593 (0.0008) [2023-10-10 19:01:30,142][123582] Updated weights for policy 0, policy_version 62603 (0.0009) [2023-10-10 19:01:30,510][123582] Updated weights for policy 0, policy_version 62613 (0.0010) [2023-10-10 19:01:30,880][123582] Updated weights for policy 0, policy_version 62623 (0.0010) [2023-10-10 19:01:32,110][123614] Updated weights for policy 1, policy_version 62500 (0.0008) [2023-10-10 19:01:32,477][123614] Updated weights for policy 1, policy_version 62510 (0.0009) [2023-10-10 19:01:32,844][123614] Updated weights for policy 1, policy_version 62520 (0.0008) [2023-10-10 19:01:33,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 128155648. Throughput: 0: 1817.8, 1: 1816.4. Samples: 32046190. Policy #0 lag: (min: 17.0, avg: 36.1, max: 49.0) [2023-10-10 19:01:33,788][122664] Avg episode reward: [(0, '60.890'), (1, '63.320')] [2023-10-10 19:01:34,553][123582] Updated weights for policy 0, policy_version 62633 (0.0009) [2023-10-10 19:01:34,929][123582] Updated weights for policy 0, policy_version 62643 (0.0009) [2023-10-10 19:01:35,308][123582] Updated weights for policy 0, policy_version 62653 (0.0008) [2023-10-10 19:01:36,667][123614] Updated weights for policy 1, policy_version 62530 (0.0008) [2023-10-10 19:01:37,034][123614] Updated weights for policy 1, policy_version 62540 (0.0008) [2023-10-10 19:01:37,409][123614] Updated weights for policy 1, policy_version 62550 (0.0009) [2023-10-10 19:01:37,779][123614] Updated weights for policy 1, policy_version 62560 (0.0008) [2023-10-10 19:01:38,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 128221184. Throughput: 0: 1826.9, 1: 1800.3. Samples: 32068200. Policy #0 lag: (min: 17.0, avg: 36.1, max: 49.0) [2023-10-10 19:01:38,788][122664] Avg episode reward: [(0, '62.480'), (1, '66.250')] [2023-10-10 19:01:38,807][123582] Updated weights for policy 0, policy_version 62663 (0.0007) [2023-10-10 19:01:39,181][123582] Updated weights for policy 0, policy_version 62673 (0.0007) [2023-10-10 19:01:39,555][123582] Updated weights for policy 0, policy_version 62683 (0.0010) [2023-10-10 19:01:41,506][123614] Updated weights for policy 1, policy_version 62570 (0.0007) [2023-10-10 19:01:41,871][123614] Updated weights for policy 1, policy_version 62580 (0.0009) [2023-10-10 19:01:42,235][123614] Updated weights for policy 1, policy_version 62590 (0.0007) [2023-10-10 19:01:43,285][123582] Updated weights for policy 0, policy_version 62693 (0.0007) [2023-10-10 19:01:43,666][123582] Updated weights for policy 0, policy_version 62703 (0.0009) [2023-10-10 19:01:43,788][122664] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 128286720. Throughput: 0: 1826.5, 1: 1815.3. Samples: 32078934. Policy #0 lag: (min: 17.0, avg: 36.1, max: 49.0) [2023-10-10 19:01:43,789][122664] Avg episode reward: [(0, '69.700'), (1, '61.790')] [2023-10-10 19:01:44,049][123582] Updated weights for policy 0, policy_version 62713 (0.0010) [2023-10-10 19:01:45,939][123614] Updated weights for policy 1, policy_version 62600 (0.0009) [2023-10-10 19:01:46,311][123614] Updated weights for policy 1, policy_version 62610 (0.0008) [2023-10-10 19:01:46,681][123614] Updated weights for policy 1, policy_version 62620 (0.0010) [2023-10-10 19:01:47,809][123582] Updated weights for policy 0, policy_version 62723 (0.0007) [2023-10-10 19:01:48,180][123582] Updated weights for policy 0, policy_version 62733 (0.0008) [2023-10-10 19:01:48,561][123582] Updated weights for policy 0, policy_version 62743 (0.0007) [2023-10-10 19:01:48,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 128352256. Throughput: 0: 1822.2, 1: 1809.3. Samples: 32100896. Policy #0 lag: (min: 17.0, avg: 36.1, max: 49.0) [2023-10-10 19:01:48,789][122664] Avg episode reward: [(0, '64.440'), (1, '63.860')] [2023-10-10 19:01:50,394][123614] Updated weights for policy 1, policy_version 62630 (0.0010) [2023-10-10 19:01:50,764][123614] Updated weights for policy 1, policy_version 62640 (0.0009) [2023-10-10 19:01:51,127][123614] Updated weights for policy 1, policy_version 62650 (0.0009) [2023-10-10 19:01:52,125][123582] Updated weights for policy 0, policy_version 62753 (0.0008) [2023-10-10 19:01:52,496][123582] Updated weights for policy 0, policy_version 62763 (0.0008) [2023-10-10 19:01:52,871][123582] Updated weights for policy 0, policy_version 62773 (0.0009) [2023-10-10 19:01:53,227][123582] Updated weights for policy 0, policy_version 62783 (0.0007) [2023-10-10 19:01:53,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 128450560. Throughput: 0: 1819.1, 1: 1818.0. Samples: 32122714. Policy #0 lag: (min: 17.0, avg: 36.1, max: 49.0) [2023-10-10 19:01:53,789][122664] Avg episode reward: [(0, '67.470'), (1, '58.670')] [2023-10-10 19:01:53,799][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000062656_64159744.pth... [2023-10-10 19:01:53,799][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000062784_64290816.pth... [2023-10-10 19:01:53,829][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000060960_62423040.pth [2023-10-10 19:01:53,837][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000061088_62554112.pth [2023-10-10 19:01:54,732][123614] Updated weights for policy 1, policy_version 62660 (0.0010) [2023-10-10 19:01:55,114][123614] Updated weights for policy 1, policy_version 62670 (0.0010) [2023-10-10 19:01:55,479][123614] Updated weights for policy 1, policy_version 62680 (0.0008) [2023-10-10 19:01:56,961][123582] Updated weights for policy 0, policy_version 62793 (0.0007) [2023-10-10 19:01:57,326][123582] Updated weights for policy 0, policy_version 62803 (0.0007) [2023-10-10 19:01:57,700][123582] Updated weights for policy 0, policy_version 62813 (0.0007) [2023-10-10 19:01:58,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 128516096. Throughput: 0: 1818.5, 1: 1820.9. Samples: 32134060. Policy #0 lag: (min: 17.0, avg: 36.1, max: 49.0) [2023-10-10 19:01:58,788][122664] Avg episode reward: [(0, '72.900'), (1, '57.220')] [2023-10-10 19:01:59,090][123614] Updated weights for policy 1, policy_version 62690 (0.0009) [2023-10-10 19:01:59,461][123614] Updated weights for policy 1, policy_version 62700 (0.0010) [2023-10-10 19:01:59,833][123614] Updated weights for policy 1, policy_version 62710 (0.0009) [2023-10-10 19:02:00,210][123614] Updated weights for policy 1, policy_version 62720 (0.0008) [2023-10-10 19:02:01,384][123582] Updated weights for policy 0, policy_version 62823 (0.0007) [2023-10-10 19:02:01,764][123582] Updated weights for policy 0, policy_version 62833 (0.0007) [2023-10-10 19:02:02,138][123582] Updated weights for policy 0, policy_version 62843 (0.0007) [2023-10-10 19:02:03,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 128581632. Throughput: 0: 1819.2, 1: 1814.8. Samples: 32155204. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 19:02:03,788][122664] Avg episode reward: [(0, '73.860'), (1, '60.910')] [2023-10-10 19:02:03,940][123614] Updated weights for policy 1, policy_version 62730 (0.0009) [2023-10-10 19:02:04,306][123614] Updated weights for policy 1, policy_version 62740 (0.0009) [2023-10-10 19:02:04,672][123614] Updated weights for policy 1, policy_version 62750 (0.0008) [2023-10-10 19:02:05,791][123582] Updated weights for policy 0, policy_version 62853 (0.0008) [2023-10-10 19:02:06,160][123582] Updated weights for policy 0, policy_version 62863 (0.0007) [2023-10-10 19:02:06,533][123582] Updated weights for policy 0, policy_version 62873 (0.0008) [2023-10-10 19:02:08,292][123614] Updated weights for policy 1, policy_version 62760 (0.0007) [2023-10-10 19:02:08,670][123614] Updated weights for policy 1, policy_version 62770 (0.0008) [2023-10-10 19:02:08,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 128647168. Throughput: 0: 1821.2, 1: 1817.4. Samples: 32177258. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 19:02:08,789][122664] Avg episode reward: [(0, '75.750'), (1, '61.060')] [2023-10-10 19:02:09,040][123614] Updated weights for policy 1, policy_version 62780 (0.0007) [2023-10-10 19:02:10,171][123582] Updated weights for policy 0, policy_version 62883 (0.0008) [2023-10-10 19:02:10,542][123582] Updated weights for policy 0, policy_version 62893 (0.0007) [2023-10-10 19:02:10,908][123582] Updated weights for policy 0, policy_version 62903 (0.0010) [2023-10-10 19:02:12,810][123614] Updated weights for policy 1, policy_version 62790 (0.0007) [2023-10-10 19:02:13,177][123614] Updated weights for policy 1, policy_version 62800 (0.0008) [2023-10-10 19:02:13,545][123614] Updated weights for policy 1, policy_version 62810 (0.0007) [2023-10-10 19:02:13,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 128745472. Throughput: 0: 1822.4, 1: 1811.6. Samples: 32187912. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 19:02:13,789][122664] Avg episode reward: [(0, '73.210'), (1, '62.870')] [2023-10-10 19:02:14,488][123582] Updated weights for policy 0, policy_version 62913 (0.0011) [2023-10-10 19:02:14,869][123582] Updated weights for policy 0, policy_version 62923 (0.0009) [2023-10-10 19:02:15,231][123582] Updated weights for policy 0, policy_version 62933 (0.0008) [2023-10-10 19:02:15,609][123582] Updated weights for policy 0, policy_version 62943 (0.0007) [2023-10-10 19:02:17,267][123614] Updated weights for policy 1, policy_version 62820 (0.0008) [2023-10-10 19:02:17,639][123614] Updated weights for policy 1, policy_version 62830 (0.0008) [2023-10-10 19:02:18,010][123614] Updated weights for policy 1, policy_version 62840 (0.0009) [2023-10-10 19:02:18,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 128811008. Throughput: 0: 1826.9, 1: 1817.6. Samples: 32210196. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 19:02:18,789][122664] Avg episode reward: [(0, '76.520'), (1, '61.190')] [2023-10-10 19:02:19,393][123582] Updated weights for policy 0, policy_version 62953 (0.0009) [2023-10-10 19:02:19,765][123582] Updated weights for policy 0, policy_version 62963 (0.0008) [2023-10-10 19:02:20,136][123582] Updated weights for policy 0, policy_version 62973 (0.0007) [2023-10-10 19:02:21,748][123614] Updated weights for policy 1, policy_version 62850 (0.0008) [2023-10-10 19:02:22,117][123614] Updated weights for policy 1, policy_version 62860 (0.0007) [2023-10-10 19:02:22,483][123614] Updated weights for policy 1, policy_version 62870 (0.0007) [2023-10-10 19:02:22,851][123614] Updated weights for policy 1, policy_version 62880 (0.0008) [2023-10-10 19:02:23,781][123582] Updated weights for policy 0, policy_version 62983 (0.0008) [2023-10-10 19:02:23,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 128876544. Throughput: 0: 1824.2, 1: 1815.9. Samples: 32232004. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 19:02:23,788][122664] Avg episode reward: [(0, '80.330'), (1, '61.930')] [2023-10-10 19:02:24,154][123582] Updated weights for policy 0, policy_version 62993 (0.0007) [2023-10-10 19:02:24,523][123582] Updated weights for policy 0, policy_version 63003 (0.0009) [2023-10-10 19:02:26,586][123614] Updated weights for policy 1, policy_version 62890 (0.0009) [2023-10-10 19:02:26,958][123614] Updated weights for policy 1, policy_version 62900 (0.0008) [2023-10-10 19:02:27,327][123614] Updated weights for policy 1, policy_version 62910 (0.0008) [2023-10-10 19:02:28,266][123582] Updated weights for policy 0, policy_version 63013 (0.0007) [2023-10-10 19:02:28,637][123582] Updated weights for policy 0, policy_version 63023 (0.0007) [2023-10-10 19:02:28,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 128942080. Throughput: 0: 1823.7, 1: 1817.1. Samples: 32242770. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 19:02:28,788][122664] Avg episode reward: [(0, '84.030'), (1, '62.100')] [2023-10-10 19:02:29,004][123582] Updated weights for policy 0, policy_version 63033 (0.0008) [2023-10-10 19:02:31,143][123614] Updated weights for policy 1, policy_version 62920 (0.0008) [2023-10-10 19:02:31,520][123614] Updated weights for policy 1, policy_version 62930 (0.0009) [2023-10-10 19:02:31,887][123614] Updated weights for policy 1, policy_version 62940 (0.0007) [2023-10-10 19:02:32,605][123582] Updated weights for policy 0, policy_version 63043 (0.0010) [2023-10-10 19:02:32,976][123582] Updated weights for policy 0, policy_version 63053 (0.0011) [2023-10-10 19:02:33,347][123582] Updated weights for policy 0, policy_version 63063 (0.0008) [2023-10-10 19:02:33,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 129040384. Throughput: 0: 1824.1, 1: 1814.0. Samples: 32264610. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 19:02:33,789][122664] Avg episode reward: [(0, '80.180'), (1, '59.290')] [2023-10-10 19:02:35,517][123614] Updated weights for policy 1, policy_version 62950 (0.0008) [2023-10-10 19:02:35,880][123614] Updated weights for policy 1, policy_version 62960 (0.0007) [2023-10-10 19:02:36,258][123614] Updated weights for policy 1, policy_version 62970 (0.0007) [2023-10-10 19:02:37,068][123582] Updated weights for policy 0, policy_version 63073 (0.0007) [2023-10-10 19:02:37,441][123582] Updated weights for policy 0, policy_version 63083 (0.0008) [2023-10-10 19:02:37,808][123582] Updated weights for policy 0, policy_version 63093 (0.0008) [2023-10-10 19:02:38,182][123582] Updated weights for policy 0, policy_version 63103 (0.0008) [2023-10-10 19:02:38,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 129105920. Throughput: 0: 1818.7, 1: 1806.1. Samples: 32285828. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 19:02:38,789][122664] Avg episode reward: [(0, '84.260'), (1, '59.800')] [2023-10-10 19:02:40,026][123614] Updated weights for policy 1, policy_version 62980 (0.0007) [2023-10-10 19:02:40,423][123614] Updated weights for policy 1, policy_version 62990 (0.0007) [2023-10-10 19:02:40,780][123614] Updated weights for policy 1, policy_version 63000 (0.0007) [2023-10-10 19:02:41,879][123582] Updated weights for policy 0, policy_version 63113 (0.0008) [2023-10-10 19:02:42,250][123582] Updated weights for policy 0, policy_version 63123 (0.0010) [2023-10-10 19:02:42,631][123582] Updated weights for policy 0, policy_version 63133 (0.0008) [2023-10-10 19:02:43,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 129171456. Throughput: 0: 1818.0, 1: 1804.5. Samples: 32297074. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:02:43,789][122664] Avg episode reward: [(0, '82.630'), (1, '59.150')] [2023-10-10 19:02:44,438][123614] Updated weights for policy 1, policy_version 63010 (0.0008) [2023-10-10 19:02:44,804][123614] Updated weights for policy 1, policy_version 63020 (0.0010) [2023-10-10 19:02:45,170][123614] Updated weights for policy 1, policy_version 63030 (0.0009) [2023-10-10 19:02:45,545][123614] Updated weights for policy 1, policy_version 63040 (0.0008) [2023-10-10 19:02:46,343][123582] Updated weights for policy 0, policy_version 63143 (0.0008) [2023-10-10 19:02:46,716][123582] Updated weights for policy 0, policy_version 63153 (0.0009) [2023-10-10 19:02:47,094][123582] Updated weights for policy 0, policy_version 63163 (0.0008) [2023-10-10 19:02:48,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 129236992. Throughput: 0: 1814.6, 1: 1803.9. Samples: 32318038. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:02:48,789][122664] Avg episode reward: [(0, '82.730'), (1, '57.700')] [2023-10-10 19:02:49,368][123614] Updated weights for policy 1, policy_version 63050 (0.0009) [2023-10-10 19:02:49,732][123614] Updated weights for policy 1, policy_version 63060 (0.0010) [2023-10-10 19:02:50,096][123614] Updated weights for policy 1, policy_version 63070 (0.0010) [2023-10-10 19:02:50,850][123582] Updated weights for policy 0, policy_version 63173 (0.0008) [2023-10-10 19:02:51,235][123582] Updated weights for policy 0, policy_version 63183 (0.0009) [2023-10-10 19:02:51,615][123582] Updated weights for policy 0, policy_version 63193 (0.0010) [2023-10-10 19:02:53,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 129302528. Throughput: 0: 1809.4, 1: 1817.5. Samples: 32340468. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:02:53,789][122664] Avg episode reward: [(0, '83.890'), (1, '57.760')] [2023-10-10 19:02:53,791][123614] Updated weights for policy 1, policy_version 63080 (0.0008) [2023-10-10 19:02:54,156][123614] Updated weights for policy 1, policy_version 63090 (0.0007) [2023-10-10 19:02:54,529][123614] Updated weights for policy 1, policy_version 63100 (0.0010) [2023-10-10 19:02:55,302][123582] Updated weights for policy 0, policy_version 63203 (0.0010) [2023-10-10 19:02:55,680][123582] Updated weights for policy 0, policy_version 63213 (0.0009) [2023-10-10 19:02:56,059][123582] Updated weights for policy 0, policy_version 63223 (0.0009) [2023-10-10 19:02:58,145][123614] Updated weights for policy 1, policy_version 63110 (0.0008) [2023-10-10 19:02:58,517][123614] Updated weights for policy 1, policy_version 63120 (0.0009) [2023-10-10 19:02:58,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 129368064. Throughput: 0: 1809.4, 1: 1810.0. Samples: 32350784. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:02:58,789][122664] Avg episode reward: [(0, '81.660'), (1, '55.890')] [2023-10-10 19:02:58,873][123614] Updated weights for policy 1, policy_version 63130 (0.0008) [2023-10-10 19:02:59,859][123582] Updated weights for policy 0, policy_version 63233 (0.0008) [2023-10-10 19:03:00,231][123582] Updated weights for policy 0, policy_version 63243 (0.0009) [2023-10-10 19:03:00,595][123582] Updated weights for policy 0, policy_version 63253 (0.0007) [2023-10-10 19:03:00,966][123582] Updated weights for policy 0, policy_version 63263 (0.0011) [2023-10-10 19:03:02,517][123614] Updated weights for policy 1, policy_version 63140 (0.0009) [2023-10-10 19:03:02,876][123614] Updated weights for policy 1, policy_version 63150 (0.0009) [2023-10-10 19:03:03,241][123614] Updated weights for policy 1, policy_version 63160 (0.0008) [2023-10-10 19:03:03,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 129466368. Throughput: 0: 1803.2, 1: 1814.5. Samples: 32372992. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:03:03,789][122664] Avg episode reward: [(0, '78.060'), (1, '52.290')] [2023-10-10 19:03:04,564][123582] Updated weights for policy 0, policy_version 63273 (0.0009) [2023-10-10 19:03:04,937][123582] Updated weights for policy 0, policy_version 63283 (0.0008) [2023-10-10 19:03:05,312][123582] Updated weights for policy 0, policy_version 63293 (0.0008) [2023-10-10 19:03:07,088][123614] Updated weights for policy 1, policy_version 63170 (0.0008) [2023-10-10 19:03:07,456][123614] Updated weights for policy 1, policy_version 63180 (0.0007) [2023-10-10 19:03:07,824][123614] Updated weights for policy 1, policy_version 63190 (0.0007) [2023-10-10 19:03:08,192][123614] Updated weights for policy 1, policy_version 63200 (0.0007) [2023-10-10 19:03:08,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 129531904. Throughput: 0: 1808.0, 1: 1808.0. Samples: 32394724. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:03:08,789][122664] Avg episode reward: [(0, '77.460'), (1, '50.670')] [2023-10-10 19:03:09,069][123582] Updated weights for policy 0, policy_version 63303 (0.0007) [2023-10-10 19:03:09,439][123582] Updated weights for policy 0, policy_version 63313 (0.0008) [2023-10-10 19:03:09,805][123582] Updated weights for policy 0, policy_version 63323 (0.0008) [2023-10-10 19:03:11,757][123614] Updated weights for policy 1, policy_version 63210 (0.0009) [2023-10-10 19:03:12,132][123614] Updated weights for policy 1, policy_version 63220 (0.0008) [2023-10-10 19:03:12,511][123614] Updated weights for policy 1, policy_version 63230 (0.0009) [2023-10-10 19:03:13,537][123582] Updated weights for policy 0, policy_version 63333 (0.0008) [2023-10-10 19:03:13,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 129597440. Throughput: 0: 1808.7, 1: 1818.0. Samples: 32405968. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:03:13,789][122664] Avg episode reward: [(0, '71.610'), (1, '52.130')] [2023-10-10 19:03:13,913][123582] Updated weights for policy 0, policy_version 63343 (0.0008) [2023-10-10 19:03:14,295][123582] Updated weights for policy 0, policy_version 63353 (0.0008) [2023-10-10 19:03:16,256][123614] Updated weights for policy 1, policy_version 63240 (0.0008) [2023-10-10 19:03:16,625][123614] Updated weights for policy 1, policy_version 63250 (0.0009) [2023-10-10 19:03:16,993][123614] Updated weights for policy 1, policy_version 63260 (0.0008) [2023-10-10 19:03:17,947][123582] Updated weights for policy 0, policy_version 63363 (0.0008) [2023-10-10 19:03:18,318][123582] Updated weights for policy 0, policy_version 63373 (0.0012) [2023-10-10 19:03:18,685][123582] Updated weights for policy 0, policy_version 63383 (0.0010) [2023-10-10 19:03:18,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 129662976. Throughput: 0: 1804.5, 1: 1817.4. Samples: 32427598. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:03:18,789][122664] Avg episode reward: [(0, '69.740'), (1, '51.760')] [2023-10-10 19:03:20,637][123614] Updated weights for policy 1, policy_version 63270 (0.0010) [2023-10-10 19:03:21,003][123614] Updated weights for policy 1, policy_version 63280 (0.0011) [2023-10-10 19:03:21,366][123614] Updated weights for policy 1, policy_version 63290 (0.0010) [2023-10-10 19:03:22,452][123582] Updated weights for policy 0, policy_version 63393 (0.0007) [2023-10-10 19:03:22,821][123582] Updated weights for policy 0, policy_version 63403 (0.0008) [2023-10-10 19:03:23,189][123582] Updated weights for policy 0, policy_version 63413 (0.0008) [2023-10-10 19:03:23,557][123582] Updated weights for policy 0, policy_version 63423 (0.0008) [2023-10-10 19:03:23,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 129761280. Throughput: 0: 1809.7, 1: 1816.2. Samples: 32448994. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-10 19:03:23,789][122664] Avg episode reward: [(0, '71.450'), (1, '51.160')] [2023-10-10 19:03:25,140][123614] Updated weights for policy 1, policy_version 63300 (0.0009) [2023-10-10 19:03:25,517][123614] Updated weights for policy 1, policy_version 63310 (0.0011) [2023-10-10 19:03:25,874][123614] Updated weights for policy 1, policy_version 63320 (0.0008) [2023-10-10 19:03:27,312][123582] Updated weights for policy 0, policy_version 63433 (0.0009) [2023-10-10 19:03:27,693][123582] Updated weights for policy 0, policy_version 63443 (0.0008) [2023-10-10 19:03:28,053][123582] Updated weights for policy 0, policy_version 63453 (0.0008) [2023-10-10 19:03:28,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 129826816. Throughput: 0: 1803.9, 1: 1819.9. Samples: 32460142. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-10 19:03:28,789][122664] Avg episode reward: [(0, '72.900'), (1, '53.800')] [2023-10-10 19:03:29,491][123614] Updated weights for policy 1, policy_version 63330 (0.0007) [2023-10-10 19:03:29,869][123614] Updated weights for policy 1, policy_version 63340 (0.0008) [2023-10-10 19:03:30,232][123614] Updated weights for policy 1, policy_version 63350 (0.0007) [2023-10-10 19:03:30,603][123614] Updated weights for policy 1, policy_version 63360 (0.0008) [2023-10-10 19:03:31,771][123582] Updated weights for policy 0, policy_version 63463 (0.0007) [2023-10-10 19:03:32,151][123582] Updated weights for policy 0, policy_version 63473 (0.0007) [2023-10-10 19:03:32,525][123582] Updated weights for policy 0, policy_version 63483 (0.0007) [2023-10-10 19:03:33,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 129892352. Throughput: 0: 1815.1, 1: 1821.1. Samples: 32481666. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-10 19:03:33,789][122664] Avg episode reward: [(0, '74.110'), (1, '53.430')] [2023-10-10 19:03:34,313][123614] Updated weights for policy 1, policy_version 63370 (0.0007) [2023-10-10 19:03:34,681][123614] Updated weights for policy 1, policy_version 63380 (0.0009) [2023-10-10 19:03:35,055][123614] Updated weights for policy 1, policy_version 63390 (0.0008) [2023-10-10 19:03:36,418][123582] Updated weights for policy 0, policy_version 63493 (0.0007) [2023-10-10 19:03:36,801][123582] Updated weights for policy 0, policy_version 63503 (0.0009) [2023-10-10 19:03:37,182][123582] Updated weights for policy 0, policy_version 63513 (0.0008) [2023-10-10 19:03:38,750][123614] Updated weights for policy 1, policy_version 63400 (0.0009) [2023-10-10 19:03:38,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 129957888. Throughput: 0: 1805.3, 1: 1818.6. Samples: 32503544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-10 19:03:38,789][122664] Avg episode reward: [(0, '68.830'), (1, '50.930')] [2023-10-10 19:03:39,119][123614] Updated weights for policy 1, policy_version 63410 (0.0008) [2023-10-10 19:03:39,490][123614] Updated weights for policy 1, policy_version 63420 (0.0009) [2023-10-10 19:03:40,778][123582] Updated weights for policy 0, policy_version 63523 (0.0008) [2023-10-10 19:03:41,156][123582] Updated weights for policy 0, policy_version 63533 (0.0009) [2023-10-10 19:03:41,529][123582] Updated weights for policy 0, policy_version 63543 (0.0008) [2023-10-10 19:03:43,054][123614] Updated weights for policy 1, policy_version 63430 (0.0010) [2023-10-10 19:03:43,428][123614] Updated weights for policy 1, policy_version 63440 (0.0010) [2023-10-10 19:03:43,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 130023424. Throughput: 0: 1821.4, 1: 1816.4. Samples: 32514486. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-10 19:03:43,789][122664] Avg episode reward: [(0, '69.980'), (1, '57.490')] [2023-10-10 19:03:43,798][123614] Updated weights for policy 1, policy_version 63450 (0.0010) [2023-10-10 19:03:45,257][123582] Updated weights for policy 0, policy_version 63553 (0.0009) [2023-10-10 19:03:45,621][123582] Updated weights for policy 0, policy_version 63563 (0.0007) [2023-10-10 19:03:45,991][123582] Updated weights for policy 0, policy_version 63573 (0.0007) [2023-10-10 19:03:46,369][123582] Updated weights for policy 0, policy_version 63583 (0.0007) [2023-10-10 19:03:47,438][123614] Updated weights for policy 1, policy_version 63460 (0.0009) [2023-10-10 19:03:47,812][123614] Updated weights for policy 1, policy_version 63470 (0.0009) [2023-10-10 19:03:48,184][123614] Updated weights for policy 1, policy_version 63480 (0.0007) [2023-10-10 19:03:48,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 130121728. Throughput: 0: 1813.2, 1: 1817.9. Samples: 32536392. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-10 19:03:48,788][122664] Avg episode reward: [(0, '75.040'), (1, '58.430')] [2023-10-10 19:03:49,965][123582] Updated weights for policy 0, policy_version 63593 (0.0009) [2023-10-10 19:03:50,337][123582] Updated weights for policy 0, policy_version 63603 (0.0010) [2023-10-10 19:03:50,716][123582] Updated weights for policy 0, policy_version 63613 (0.0008) [2023-10-10 19:03:51,910][123614] Updated weights for policy 1, policy_version 63490 (0.0007) [2023-10-10 19:03:52,278][123614] Updated weights for policy 1, policy_version 63500 (0.0007) [2023-10-10 19:03:52,653][123614] Updated weights for policy 1, policy_version 63510 (0.0007) [2023-10-10 19:03:53,018][123614] Updated weights for policy 1, policy_version 63520 (0.0008) [2023-10-10 19:03:53,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 130187264. Throughput: 0: 1815.7, 1: 1823.6. Samples: 32558494. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-10 19:03:53,789][122664] Avg episode reward: [(0, '77.410'), (1, '56.370')] [2023-10-10 19:03:53,799][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000063520_65044480.pth... [2023-10-10 19:03:53,799][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000063616_65142784.pth... [2023-10-10 19:03:53,835][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000061920_63406080.pth [2023-10-10 19:03:53,836][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000061824_63307776.pth [2023-10-10 19:03:54,312][123582] Updated weights for policy 0, policy_version 63623 (0.0009) [2023-10-10 19:03:54,673][123582] Updated weights for policy 0, policy_version 63633 (0.0010) [2023-10-10 19:03:55,043][123582] Updated weights for policy 0, policy_version 63643 (0.0009) [2023-10-10 19:03:56,642][123614] Updated weights for policy 1, policy_version 63530 (0.0009) [2023-10-10 19:03:57,011][123614] Updated weights for policy 1, policy_version 63540 (0.0008) [2023-10-10 19:03:57,382][123614] Updated weights for policy 1, policy_version 63550 (0.0009) [2023-10-10 19:03:58,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 130252800. Throughput: 0: 1815.3, 1: 1819.0. Samples: 32569510. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-10 19:03:58,789][122664] Avg episode reward: [(0, '77.570'), (1, '55.770')] [2023-10-10 19:03:58,803][123582] Updated weights for policy 0, policy_version 63653 (0.0010) [2023-10-10 19:03:59,178][123582] Updated weights for policy 0, policy_version 63663 (0.0011) [2023-10-10 19:03:59,556][123582] Updated weights for policy 0, policy_version 63673 (0.0011) [2023-10-10 19:04:00,948][123614] Updated weights for policy 1, policy_version 63560 (0.0007) [2023-10-10 19:04:01,318][123614] Updated weights for policy 1, policy_version 63570 (0.0007) [2023-10-10 19:04:01,685][123614] Updated weights for policy 1, policy_version 63580 (0.0008) [2023-10-10 19:04:03,149][123582] Updated weights for policy 0, policy_version 63683 (0.0009) [2023-10-10 19:04:03,526][123582] Updated weights for policy 0, policy_version 63693 (0.0009) [2023-10-10 19:04:03,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 130318336. Throughput: 0: 1818.8, 1: 1823.8. Samples: 32591516. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-10 19:04:03,788][122664] Avg episode reward: [(0, '75.730'), (1, '54.950')] [2023-10-10 19:04:03,901][123582] Updated weights for policy 0, policy_version 63703 (0.0008) [2023-10-10 19:04:05,369][123614] Updated weights for policy 1, policy_version 63590 (0.0008) [2023-10-10 19:04:05,735][123614] Updated weights for policy 1, policy_version 63600 (0.0008) [2023-10-10 19:04:06,106][123614] Updated weights for policy 1, policy_version 63610 (0.0009) [2023-10-10 19:04:07,554][123582] Updated weights for policy 0, policy_version 63713 (0.0008) [2023-10-10 19:04:07,919][123582] Updated weights for policy 0, policy_version 63723 (0.0011) [2023-10-10 19:04:08,285][123582] Updated weights for policy 0, policy_version 63733 (0.0009) [2023-10-10 19:04:08,652][123582] Updated weights for policy 0, policy_version 63743 (0.0009) [2023-10-10 19:04:08,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 130416640. Throughput: 0: 1824.8, 1: 1821.8. Samples: 32613094. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 19:04:08,789][122664] Avg episode reward: [(0, '77.600'), (1, '55.040')] [2023-10-10 19:04:10,076][123614] Updated weights for policy 1, policy_version 63620 (0.0008) [2023-10-10 19:04:10,477][123614] Updated weights for policy 1, policy_version 63630 (0.0008) [2023-10-10 19:04:10,843][123614] Updated weights for policy 1, policy_version 63640 (0.0008) [2023-10-10 19:04:12,459][123582] Updated weights for policy 0, policy_version 63753 (0.0011) [2023-10-10 19:04:12,833][123582] Updated weights for policy 0, policy_version 63763 (0.0011) [2023-10-10 19:04:13,212][123582] Updated weights for policy 0, policy_version 63773 (0.0008) [2023-10-10 19:04:13,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 130482176. Throughput: 0: 1818.1, 1: 1818.3. Samples: 32623780. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 19:04:13,788][122664] Avg episode reward: [(0, '77.330'), (1, '59.820')] [2023-10-10 19:04:14,349][123614] Updated weights for policy 1, policy_version 63650 (0.0009) [2023-10-10 19:04:14,719][123614] Updated weights for policy 1, policy_version 63660 (0.0011) [2023-10-10 19:04:15,076][123614] Updated weights for policy 1, policy_version 63670 (0.0010) [2023-10-10 19:04:15,443][123614] Updated weights for policy 1, policy_version 63680 (0.0009) [2023-10-10 19:04:16,816][123582] Updated weights for policy 0, policy_version 63783 (0.0008) [2023-10-10 19:04:17,188][123582] Updated weights for policy 0, policy_version 63793 (0.0009) [2023-10-10 19:04:17,556][123582] Updated weights for policy 0, policy_version 63803 (0.0007) [2023-10-10 19:04:18,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 130547712. Throughput: 0: 1824.4, 1: 1825.0. Samples: 32645888. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 19:04:18,788][122664] Avg episode reward: [(0, '78.460'), (1, '62.340')] [2023-10-10 19:04:19,073][123614] Updated weights for policy 1, policy_version 63690 (0.0010) [2023-10-10 19:04:19,436][123614] Updated weights for policy 1, policy_version 63700 (0.0009) [2023-10-10 19:04:19,807][123614] Updated weights for policy 1, policy_version 63710 (0.0008) [2023-10-10 19:04:21,211][123582] Updated weights for policy 0, policy_version 63813 (0.0011) [2023-10-10 19:04:21,593][123582] Updated weights for policy 0, policy_version 63823 (0.0008) [2023-10-10 19:04:21,969][123582] Updated weights for policy 0, policy_version 63833 (0.0009) [2023-10-10 19:04:23,468][123614] Updated weights for policy 1, policy_version 63720 (0.0010) [2023-10-10 19:04:23,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 130613248. Throughput: 0: 1827.1, 1: 1824.2. Samples: 32667852. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 19:04:23,789][122664] Avg episode reward: [(0, '78.210'), (1, '60.250')] [2023-10-10 19:04:23,832][123614] Updated weights for policy 1, policy_version 63730 (0.0007) [2023-10-10 19:04:24,204][123614] Updated weights for policy 1, policy_version 63740 (0.0008) [2023-10-10 19:04:25,596][123582] Updated weights for policy 0, policy_version 63843 (0.0008) [2023-10-10 19:04:25,973][123582] Updated weights for policy 0, policy_version 63853 (0.0009) [2023-10-10 19:04:26,354][123582] Updated weights for policy 0, policy_version 63863 (0.0008) [2023-10-10 19:04:27,846][123614] Updated weights for policy 1, policy_version 63750 (0.0007) [2023-10-10 19:04:28,217][123614] Updated weights for policy 1, policy_version 63760 (0.0008) [2023-10-10 19:04:28,583][123614] Updated weights for policy 1, policy_version 63770 (0.0009) [2023-10-10 19:04:28,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 130678784. Throughput: 0: 1815.6, 1: 1837.7. Samples: 32678884. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 19:04:28,788][122664] Avg episode reward: [(0, '85.800'), (1, '59.580')] [2023-10-10 19:04:29,919][123582] Updated weights for policy 0, policy_version 63873 (0.0010) [2023-10-10 19:04:30,295][123582] Updated weights for policy 0, policy_version 63883 (0.0008) [2023-10-10 19:04:30,673][123582] Updated weights for policy 0, policy_version 63893 (0.0008) [2023-10-10 19:04:31,041][123582] Updated weights for policy 0, policy_version 63903 (0.0009) [2023-10-10 19:04:32,216][123614] Updated weights for policy 1, policy_version 63780 (0.0008) [2023-10-10 19:04:32,587][123614] Updated weights for policy 1, policy_version 63790 (0.0009) [2023-10-10 19:04:32,958][123614] Updated weights for policy 1, policy_version 63800 (0.0008) [2023-10-10 19:04:33,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 130777088. Throughput: 0: 1827.9, 1: 1832.4. Samples: 32701108. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 19:04:33,789][122664] Avg episode reward: [(0, '85.710'), (1, '60.590')] [2023-10-10 19:04:34,652][123582] Updated weights for policy 0, policy_version 63913 (0.0010) [2023-10-10 19:04:35,020][123582] Updated weights for policy 0, policy_version 63923 (0.0009) [2023-10-10 19:04:35,400][123582] Updated weights for policy 0, policy_version 63933 (0.0010) [2023-10-10 19:04:36,657][123614] Updated weights for policy 1, policy_version 63810 (0.0008) [2023-10-10 19:04:37,027][123614] Updated weights for policy 1, policy_version 63820 (0.0008) [2023-10-10 19:04:37,391][123614] Updated weights for policy 1, policy_version 63830 (0.0008) [2023-10-10 19:04:37,759][123614] Updated weights for policy 1, policy_version 63840 (0.0008) [2023-10-10 19:04:38,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 130842624. Throughput: 0: 1821.1, 1: 1835.7. Samples: 32723052. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 19:04:38,789][122664] Avg episode reward: [(0, '89.360'), (1, '57.950')] [2023-10-10 19:04:39,215][123582] Updated weights for policy 0, policy_version 63943 (0.0007) [2023-10-10 19:04:39,582][123582] Updated weights for policy 0, policy_version 63953 (0.0009) [2023-10-10 19:04:39,958][123582] Updated weights for policy 0, policy_version 63963 (0.0009) [2023-10-10 19:04:41,566][123614] Updated weights for policy 1, policy_version 63850 (0.0010) [2023-10-10 19:04:41,935][123614] Updated weights for policy 1, policy_version 63860 (0.0011) [2023-10-10 19:04:42,298][123614] Updated weights for policy 1, policy_version 63870 (0.0009) [2023-10-10 19:04:43,572][123582] Updated weights for policy 0, policy_version 63973 (0.0008) [2023-10-10 19:04:43,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 130908160. Throughput: 0: 1820.9, 1: 1830.5. Samples: 32733826. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 19:04:43,789][122664] Avg episode reward: [(0, '87.500'), (1, '57.140')] [2023-10-10 19:04:43,948][123582] Updated weights for policy 0, policy_version 63983 (0.0008) [2023-10-10 19:04:44,316][123582] Updated weights for policy 0, policy_version 63993 (0.0007) [2023-10-10 19:04:46,042][123614] Updated weights for policy 1, policy_version 63880 (0.0009) [2023-10-10 19:04:46,415][123614] Updated weights for policy 1, policy_version 63890 (0.0008) [2023-10-10 19:04:46,790][123614] Updated weights for policy 1, policy_version 63900 (0.0009) [2023-10-10 19:04:47,911][123582] Updated weights for policy 0, policy_version 64003 (0.0008) [2023-10-10 19:04:48,291][123582] Updated weights for policy 0, policy_version 64013 (0.0011) [2023-10-10 19:04:48,651][123582] Updated weights for policy 0, policy_version 64023 (0.0011) [2023-10-10 19:04:48,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 130973696. Throughput: 0: 1826.3, 1: 1822.8. Samples: 32755726. Policy #0 lag: (min: 31.0, avg: 31.8, max: 50.0) [2023-10-10 19:04:48,788][122664] Avg episode reward: [(0, '85.810'), (1, '58.580')] [2023-10-10 19:04:50,410][123614] Updated weights for policy 1, policy_version 63910 (0.0009) [2023-10-10 19:04:50,771][123614] Updated weights for policy 1, policy_version 63920 (0.0009) [2023-10-10 19:04:51,144][123614] Updated weights for policy 1, policy_version 63930 (0.0008) [2023-10-10 19:04:52,379][123582] Updated weights for policy 0, policy_version 64033 (0.0010) [2023-10-10 19:04:52,743][123582] Updated weights for policy 0, policy_version 64043 (0.0007) [2023-10-10 19:04:53,114][123582] Updated weights for policy 0, policy_version 64053 (0.0008) [2023-10-10 19:04:53,479][123582] Updated weights for policy 0, policy_version 64063 (0.0008) [2023-10-10 19:04:53,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 131072000. Throughput: 0: 1820.1, 1: 1836.6. Samples: 32777646. Policy #0 lag: (min: 31.0, avg: 31.8, max: 50.0) [2023-10-10 19:04:53,789][122664] Avg episode reward: [(0, '81.990'), (1, '60.280')] [2023-10-10 19:04:54,828][123614] Updated weights for policy 1, policy_version 63940 (0.0009) [2023-10-10 19:04:55,217][123614] Updated weights for policy 1, policy_version 63950 (0.0009) [2023-10-10 19:04:55,582][123614] Updated weights for policy 1, policy_version 63960 (0.0008) [2023-10-10 19:04:57,055][123582] Updated weights for policy 0, policy_version 64073 (0.0008) [2023-10-10 19:04:57,433][123582] Updated weights for policy 0, policy_version 64083 (0.0008) [2023-10-10 19:04:57,806][123582] Updated weights for policy 0, policy_version 64093 (0.0007) [2023-10-10 19:04:58,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 131137536. Throughput: 0: 1832.6, 1: 1834.5. Samples: 32788802. Policy #0 lag: (min: 31.0, avg: 31.8, max: 50.0) [2023-10-10 19:04:58,788][122664] Avg episode reward: [(0, '77.180'), (1, '59.350')] [2023-10-10 19:04:59,264][123614] Updated weights for policy 1, policy_version 63970 (0.0010) [2023-10-10 19:04:59,637][123614] Updated weights for policy 1, policy_version 63980 (0.0008) [2023-10-10 19:05:00,003][123614] Updated weights for policy 1, policy_version 63990 (0.0009) [2023-10-10 19:05:00,374][123614] Updated weights for policy 1, policy_version 64000 (0.0008) [2023-10-10 19:05:01,712][123582] Updated weights for policy 0, policy_version 64103 (0.0009) [2023-10-10 19:05:02,086][123582] Updated weights for policy 0, policy_version 64113 (0.0007) [2023-10-10 19:05:02,454][123582] Updated weights for policy 0, policy_version 64123 (0.0010) [2023-10-10 19:05:03,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 131203072. Throughput: 0: 1819.9, 1: 1826.4. Samples: 32809972. Policy #0 lag: (min: 31.0, avg: 31.8, max: 50.0) [2023-10-10 19:05:03,789][122664] Avg episode reward: [(0, '79.180'), (1, '67.480')] [2023-10-10 19:05:04,101][123614] Updated weights for policy 1, policy_version 64010 (0.0007) [2023-10-10 19:05:04,471][123614] Updated weights for policy 1, policy_version 64020 (0.0009) [2023-10-10 19:05:04,831][123614] Updated weights for policy 1, policy_version 64030 (0.0007) [2023-10-10 19:05:06,176][123582] Updated weights for policy 0, policy_version 64133 (0.0010) [2023-10-10 19:05:06,538][123582] Updated weights for policy 0, policy_version 64143 (0.0009) [2023-10-10 19:05:06,916][123582] Updated weights for policy 0, policy_version 64153 (0.0007) [2023-10-10 19:05:08,498][123614] Updated weights for policy 1, policy_version 64040 (0.0009) [2023-10-10 19:05:08,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 131268608. Throughput: 0: 1822.0, 1: 1820.2. Samples: 32831750. Policy #0 lag: (min: 31.0, avg: 31.8, max: 50.0) [2023-10-10 19:05:08,788][122664] Avg episode reward: [(0, '75.190'), (1, '71.120')] [2023-10-10 19:05:08,868][123614] Updated weights for policy 1, policy_version 64050 (0.0007) [2023-10-10 19:05:09,229][123614] Updated weights for policy 1, policy_version 64060 (0.0008) [2023-10-10 19:05:10,498][123582] Updated weights for policy 0, policy_version 64163 (0.0010) [2023-10-10 19:05:10,875][123582] Updated weights for policy 0, policy_version 64173 (0.0010) [2023-10-10 19:05:11,236][123582] Updated weights for policy 0, policy_version 64183 (0.0009) [2023-10-10 19:05:12,891][123614] Updated weights for policy 1, policy_version 64070 (0.0008) [2023-10-10 19:05:13,251][123614] Updated weights for policy 1, policy_version 64080 (0.0008) [2023-10-10 19:05:13,622][123614] Updated weights for policy 1, policy_version 64090 (0.0007) [2023-10-10 19:05:13,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 131334144. Throughput: 0: 1822.0, 1: 1817.9. Samples: 32842682. Policy #0 lag: (min: 31.0, avg: 31.8, max: 50.0) [2023-10-10 19:05:13,789][122664] Avg episode reward: [(0, '74.550'), (1, '69.750')] [2023-10-10 19:05:14,984][123582] Updated weights for policy 0, policy_version 64193 (0.0010) [2023-10-10 19:05:15,363][123582] Updated weights for policy 0, policy_version 64203 (0.0008) [2023-10-10 19:05:15,720][123582] Updated weights for policy 0, policy_version 64213 (0.0007) [2023-10-10 19:05:16,096][123582] Updated weights for policy 0, policy_version 64223 (0.0007) [2023-10-10 19:05:17,238][123614] Updated weights for policy 1, policy_version 64100 (0.0009) [2023-10-10 19:05:17,603][123614] Updated weights for policy 1, policy_version 64110 (0.0009) [2023-10-10 19:05:17,970][123614] Updated weights for policy 1, policy_version 64120 (0.0010) [2023-10-10 19:05:18,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 131432448. Throughput: 0: 1818.2, 1: 1819.8. Samples: 32864820. Policy #0 lag: (min: 31.0, avg: 31.8, max: 50.0) [2023-10-10 19:05:18,789][122664] Avg episode reward: [(0, '73.640'), (1, '75.710')] [2023-10-10 19:05:19,780][123582] Updated weights for policy 0, policy_version 64233 (0.0009) [2023-10-10 19:05:20,159][123582] Updated weights for policy 0, policy_version 64243 (0.0009) [2023-10-10 19:05:20,547][123582] Updated weights for policy 0, policy_version 64253 (0.0009) [2023-10-10 19:05:21,691][123614] Updated weights for policy 1, policy_version 64130 (0.0007) [2023-10-10 19:05:22,061][123614] Updated weights for policy 1, policy_version 64140 (0.0011) [2023-10-10 19:05:22,433][123614] Updated weights for policy 1, policy_version 64150 (0.0008) [2023-10-10 19:05:22,804][123614] Updated weights for policy 1, policy_version 64160 (0.0008) [2023-10-10 19:05:23,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 131497984. Throughput: 0: 1819.7, 1: 1817.6. Samples: 32886728. Policy #0 lag: (min: 31.0, avg: 31.8, max: 50.0) [2023-10-10 19:05:23,789][122664] Avg episode reward: [(0, '71.540'), (1, '76.220')] [2023-10-10 19:05:24,263][123582] Updated weights for policy 0, policy_version 64263 (0.0011) [2023-10-10 19:05:24,645][123582] Updated weights for policy 0, policy_version 64273 (0.0010) [2023-10-10 19:05:25,016][123582] Updated weights for policy 0, policy_version 64283 (0.0010) [2023-10-10 19:05:26,514][123614] Updated weights for policy 1, policy_version 64170 (0.0007) [2023-10-10 19:05:26,876][123614] Updated weights for policy 1, policy_version 64180 (0.0007) [2023-10-10 19:05:27,251][123614] Updated weights for policy 1, policy_version 64190 (0.0009) [2023-10-10 19:05:28,552][123582] Updated weights for policy 0, policy_version 64293 (0.0008) [2023-10-10 19:05:28,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 131563520. Throughput: 0: 1819.0, 1: 1811.4. Samples: 32897196. Policy #0 lag: (min: 25.0, avg: 33.8, max: 57.0) [2023-10-10 19:05:28,789][122664] Avg episode reward: [(0, '68.750'), (1, '76.510')] [2023-10-10 19:05:28,926][123582] Updated weights for policy 0, policy_version 64303 (0.0007) [2023-10-10 19:05:29,293][123582] Updated weights for policy 0, policy_version 64313 (0.0009) [2023-10-10 19:05:31,056][123614] Updated weights for policy 1, policy_version 64200 (0.0008) [2023-10-10 19:05:31,414][123614] Updated weights for policy 1, policy_version 64210 (0.0008) [2023-10-10 19:05:31,790][123614] Updated weights for policy 1, policy_version 64220 (0.0009) [2023-10-10 19:05:32,990][123582] Updated weights for policy 0, policy_version 64323 (0.0010) [2023-10-10 19:05:33,361][123582] Updated weights for policy 0, policy_version 64333 (0.0009) [2023-10-10 19:05:33,723][123582] Updated weights for policy 0, policy_version 64343 (0.0008) [2023-10-10 19:05:33,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 131629056. Throughput: 0: 1818.2, 1: 1820.0. Samples: 32919442. Policy #0 lag: (min: 25.0, avg: 33.8, max: 57.0) [2023-10-10 19:05:33,788][122664] Avg episode reward: [(0, '63.580'), (1, '68.870')] [2023-10-10 19:05:35,523][123614] Updated weights for policy 1, policy_version 64230 (0.0008) [2023-10-10 19:05:35,884][123614] Updated weights for policy 1, policy_version 64240 (0.0008) [2023-10-10 19:05:36,258][123614] Updated weights for policy 1, policy_version 64250 (0.0009) [2023-10-10 19:05:37,352][123582] Updated weights for policy 0, policy_version 64353 (0.0008) [2023-10-10 19:05:37,729][123582] Updated weights for policy 0, policy_version 64363 (0.0009) [2023-10-10 19:05:38,099][123582] Updated weights for policy 0, policy_version 64373 (0.0010) [2023-10-10 19:05:38,465][123582] Updated weights for policy 0, policy_version 64383 (0.0012) [2023-10-10 19:05:38,788][122664] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 131727360. Throughput: 0: 1819.8, 1: 1807.3. Samples: 32940866. Policy #0 lag: (min: 25.0, avg: 33.8, max: 57.0) [2023-10-10 19:05:38,788][122664] Avg episode reward: [(0, '60.960'), (1, '71.420')] [2023-10-10 19:05:40,018][123614] Updated weights for policy 1, policy_version 64260 (0.0009) [2023-10-10 19:05:40,407][123614] Updated weights for policy 1, policy_version 64270 (0.0010) [2023-10-10 19:05:40,779][123614] Updated weights for policy 1, policy_version 64280 (0.0009) [2023-10-10 19:05:42,226][123582] Updated weights for policy 0, policy_version 64393 (0.0008) [2023-10-10 19:05:42,594][123582] Updated weights for policy 0, policy_version 64403 (0.0008) [2023-10-10 19:05:42,969][123582] Updated weights for policy 0, policy_version 64413 (0.0008) [2023-10-10 19:05:43,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 131792896. Throughput: 0: 1813.1, 1: 1813.2. Samples: 32951986. Policy #0 lag: (min: 25.0, avg: 33.8, max: 57.0) [2023-10-10 19:05:43,789][122664] Avg episode reward: [(0, '60.610'), (1, '71.580')] [2023-10-10 19:05:44,315][123614] Updated weights for policy 1, policy_version 64290 (0.0008) [2023-10-10 19:05:44,680][123614] Updated weights for policy 1, policy_version 64300 (0.0010) [2023-10-10 19:05:45,054][123614] Updated weights for policy 1, policy_version 64310 (0.0009) [2023-10-10 19:05:45,416][123614] Updated weights for policy 1, policy_version 64320 (0.0007) [2023-10-10 19:05:46,631][123582] Updated weights for policy 0, policy_version 64423 (0.0007) [2023-10-10 19:05:46,999][123582] Updated weights for policy 0, policy_version 64433 (0.0008) [2023-10-10 19:05:47,364][123582] Updated weights for policy 0, policy_version 64443 (0.0007) [2023-10-10 19:05:48,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 131858432. Throughput: 0: 1815.2, 1: 1815.3. Samples: 32973344. Policy #0 lag: (min: 25.0, avg: 33.8, max: 57.0) [2023-10-10 19:05:48,789][122664] Avg episode reward: [(0, '58.730'), (1, '70.960')] [2023-10-10 19:05:49,245][123614] Updated weights for policy 1, policy_version 64330 (0.0008) [2023-10-10 19:05:49,608][123614] Updated weights for policy 1, policy_version 64340 (0.0010) [2023-10-10 19:05:49,970][123614] Updated weights for policy 1, policy_version 64350 (0.0008) [2023-10-10 19:05:51,239][123582] Updated weights for policy 0, policy_version 64453 (0.0010) [2023-10-10 19:05:51,622][123582] Updated weights for policy 0, policy_version 64463 (0.0009) [2023-10-10 19:05:52,003][123582] Updated weights for policy 0, policy_version 64473 (0.0008) [2023-10-10 19:05:53,629][123614] Updated weights for policy 1, policy_version 64360 (0.0010) [2023-10-10 19:05:53,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 131923968. Throughput: 0: 1815.6, 1: 1818.7. Samples: 32995296. Policy #0 lag: (min: 25.0, avg: 33.8, max: 57.0) [2023-10-10 19:05:53,789][122664] Avg episode reward: [(0, '59.110'), (1, '71.620')] [2023-10-10 19:05:53,799][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000064480_66027520.pth... [2023-10-10 19:05:53,835][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000062784_64290816.pth [2023-10-10 19:05:53,996][123614] Updated weights for policy 1, policy_version 64370 (0.0010) [2023-10-10 19:05:54,357][123614] Updated weights for policy 1, policy_version 64380 (0.0010) [2023-10-10 19:05:54,503][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000064384_65929216.pth... [2023-10-10 19:05:54,541][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000062656_64159744.pth [2023-10-10 19:05:55,584][123582] Updated weights for policy 0, policy_version 64483 (0.0008) [2023-10-10 19:05:55,954][123582] Updated weights for policy 0, policy_version 64493 (0.0008) [2023-10-10 19:05:56,317][123582] Updated weights for policy 0, policy_version 64503 (0.0008) [2023-10-10 19:05:58,088][123614] Updated weights for policy 1, policy_version 64390 (0.0011) [2023-10-10 19:05:58,460][123614] Updated weights for policy 1, policy_version 64400 (0.0010) [2023-10-10 19:05:58,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 131989504. Throughput: 0: 1818.8, 1: 1816.3. Samples: 33006262. Policy #0 lag: (min: 25.0, avg: 33.8, max: 57.0) [2023-10-10 19:05:58,789][122664] Avg episode reward: [(0, '65.770'), (1, '75.190')] [2023-10-10 19:05:58,827][123614] Updated weights for policy 1, policy_version 64410 (0.0010) [2023-10-10 19:06:00,146][123582] Updated weights for policy 0, policy_version 64513 (0.0008) [2023-10-10 19:06:00,522][123582] Updated weights for policy 0, policy_version 64523 (0.0008) [2023-10-10 19:06:00,891][123582] Updated weights for policy 0, policy_version 64533 (0.0007) [2023-10-10 19:06:01,266][123582] Updated weights for policy 0, policy_version 64543 (0.0010) [2023-10-10 19:06:02,515][123614] Updated weights for policy 1, policy_version 64420 (0.0008) [2023-10-10 19:06:02,881][123614] Updated weights for policy 1, policy_version 64430 (0.0009) [2023-10-10 19:06:03,245][123614] Updated weights for policy 1, policy_version 64440 (0.0009) [2023-10-10 19:06:03,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 132087808. Throughput: 0: 1807.7, 1: 1820.0. Samples: 33028068. Policy #0 lag: (min: 25.0, avg: 33.8, max: 57.0) [2023-10-10 19:06:03,789][122664] Avg episode reward: [(0, '64.470'), (1, '75.470')] [2023-10-10 19:06:05,003][123582] Updated weights for policy 0, policy_version 64553 (0.0010) [2023-10-10 19:06:05,367][123582] Updated weights for policy 0, policy_version 64563 (0.0009) [2023-10-10 19:06:05,742][123582] Updated weights for policy 0, policy_version 64573 (0.0008) [2023-10-10 19:06:07,130][123614] Updated weights for policy 1, policy_version 64450 (0.0009) [2023-10-10 19:06:07,501][123614] Updated weights for policy 1, policy_version 64460 (0.0008) [2023-10-10 19:06:07,868][123614] Updated weights for policy 1, policy_version 64470 (0.0008) [2023-10-10 19:06:08,241][123614] Updated weights for policy 1, policy_version 64480 (0.0008) [2023-10-10 19:06:08,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 132153344. Throughput: 0: 1806.2, 1: 1809.7. Samples: 33049444. Policy #0 lag: (min: 29.0, avg: 40.2, max: 61.0) [2023-10-10 19:06:08,789][122664] Avg episode reward: [(0, '64.420'), (1, '75.560')] [2023-10-10 19:06:09,429][123582] Updated weights for policy 0, policy_version 64583 (0.0009) [2023-10-10 19:06:09,795][123582] Updated weights for policy 0, policy_version 64593 (0.0008) [2023-10-10 19:06:10,162][123582] Updated weights for policy 0, policy_version 64603 (0.0009) [2023-10-10 19:06:11,822][123614] Updated weights for policy 1, policy_version 64490 (0.0009) [2023-10-10 19:06:12,194][123614] Updated weights for policy 1, policy_version 64500 (0.0007) [2023-10-10 19:06:12,564][123614] Updated weights for policy 1, policy_version 64510 (0.0010) [2023-10-10 19:06:13,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 132218880. Throughput: 0: 1803.6, 1: 1827.6. Samples: 33060598. Policy #0 lag: (min: 29.0, avg: 40.2, max: 61.0) [2023-10-10 19:06:13,788][122664] Avg episode reward: [(0, '65.620'), (1, '72.050')] [2023-10-10 19:06:13,990][123582] Updated weights for policy 0, policy_version 64613 (0.0007) [2023-10-10 19:06:14,370][123582] Updated weights for policy 0, policy_version 64623 (0.0009) [2023-10-10 19:06:14,737][123582] Updated weights for policy 0, policy_version 64633 (0.0010) [2023-10-10 19:06:16,173][123614] Updated weights for policy 1, policy_version 64520 (0.0010) [2023-10-10 19:06:16,542][123614] Updated weights for policy 1, policy_version 64530 (0.0009) [2023-10-10 19:06:16,913][123614] Updated weights for policy 1, policy_version 64540 (0.0008) [2023-10-10 19:06:18,533][123582] Updated weights for policy 0, policy_version 64643 (0.0010) [2023-10-10 19:06:18,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 132284416. Throughput: 0: 1792.5, 1: 1817.2. Samples: 33081878. Policy #0 lag: (min: 29.0, avg: 40.2, max: 61.0) [2023-10-10 19:06:18,789][122664] Avg episode reward: [(0, '61.710'), (1, '69.550')] [2023-10-10 19:06:18,917][123582] Updated weights for policy 0, policy_version 64653 (0.0007) [2023-10-10 19:06:19,282][123582] Updated weights for policy 0, policy_version 64663 (0.0009) [2023-10-10 19:06:20,564][123614] Updated weights for policy 1, policy_version 64550 (0.0008) [2023-10-10 19:06:20,936][123614] Updated weights for policy 1, policy_version 64560 (0.0010) [2023-10-10 19:06:21,311][123614] Updated weights for policy 1, policy_version 64570 (0.0010) [2023-10-10 19:06:22,994][123582] Updated weights for policy 0, policy_version 64673 (0.0011) [2023-10-10 19:06:23,357][123582] Updated weights for policy 0, policy_version 64683 (0.0007) [2023-10-10 19:06:23,734][123582] Updated weights for policy 0, policy_version 64693 (0.0008) [2023-10-10 19:06:23,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 132349952. Throughput: 0: 1802.8, 1: 1818.4. Samples: 33103820. Policy #0 lag: (min: 29.0, avg: 40.2, max: 61.0) [2023-10-10 19:06:23,789][122664] Avg episode reward: [(0, '62.280'), (1, '68.770')] [2023-10-10 19:06:24,112][123582] Updated weights for policy 0, policy_version 64703 (0.0008) [2023-10-10 19:06:25,203][123614] Updated weights for policy 1, policy_version 64580 (0.0007) [2023-10-10 19:06:25,583][123614] Updated weights for policy 1, policy_version 64590 (0.0010) [2023-10-10 19:06:25,952][123614] Updated weights for policy 1, policy_version 64600 (0.0009) [2023-10-10 19:06:27,745][123582] Updated weights for policy 0, policy_version 64713 (0.0009) [2023-10-10 19:06:28,115][123582] Updated weights for policy 0, policy_version 64723 (0.0009) [2023-10-10 19:06:28,489][123582] Updated weights for policy 0, policy_version 64733 (0.0009) [2023-10-10 19:06:28,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 132448256. Throughput: 0: 1792.0, 1: 1811.4. Samples: 33114142. Policy #0 lag: (min: 29.0, avg: 40.2, max: 61.0) [2023-10-10 19:06:28,789][122664] Avg episode reward: [(0, '62.960'), (1, '62.780')] [2023-10-10 19:06:29,566][123614] Updated weights for policy 1, policy_version 64610 (0.0008) [2023-10-10 19:06:29,936][123614] Updated weights for policy 1, policy_version 64620 (0.0008) [2023-10-10 19:06:30,304][123614] Updated weights for policy 1, policy_version 64630 (0.0008) [2023-10-10 19:06:30,663][123614] Updated weights for policy 1, policy_version 64640 (0.0007) [2023-10-10 19:06:32,146][123582] Updated weights for policy 0, policy_version 64743 (0.0008) [2023-10-10 19:06:32,522][123582] Updated weights for policy 0, policy_version 64753 (0.0009) [2023-10-10 19:06:32,899][123582] Updated weights for policy 0, policy_version 64763 (0.0010) [2023-10-10 19:06:33,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 132513792. Throughput: 0: 1807.3, 1: 1817.6. Samples: 33136468. Policy #0 lag: (min: 29.0, avg: 40.2, max: 61.0) [2023-10-10 19:06:33,789][122664] Avg episode reward: [(0, '63.580'), (1, '62.570')] [2023-10-10 19:06:34,338][123614] Updated weights for policy 1, policy_version 64650 (0.0008) [2023-10-10 19:06:34,704][123614] Updated weights for policy 1, policy_version 64660 (0.0011) [2023-10-10 19:06:35,080][123614] Updated weights for policy 1, policy_version 64670 (0.0009) [2023-10-10 19:06:36,659][123582] Updated weights for policy 0, policy_version 64773 (0.0010) [2023-10-10 19:06:37,032][123582] Updated weights for policy 0, policy_version 64783 (0.0010) [2023-10-10 19:06:37,399][123582] Updated weights for policy 0, policy_version 64793 (0.0009) [2023-10-10 19:06:38,668][123614] Updated weights for policy 1, policy_version 64680 (0.0008) [2023-10-10 19:06:38,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 132579328. Throughput: 0: 1792.9, 1: 1821.9. Samples: 33157962. Policy #0 lag: (min: 29.0, avg: 40.2, max: 61.0) [2023-10-10 19:06:38,789][122664] Avg episode reward: [(0, '62.420'), (1, '66.920')] [2023-10-10 19:06:39,037][123614] Updated weights for policy 1, policy_version 64690 (0.0008) [2023-10-10 19:06:39,410][123614] Updated weights for policy 1, policy_version 64700 (0.0008) [2023-10-10 19:06:41,117][123582] Updated weights for policy 0, policy_version 64803 (0.0009) [2023-10-10 19:06:41,487][123582] Updated weights for policy 0, policy_version 64813 (0.0008) [2023-10-10 19:06:41,864][123582] Updated weights for policy 0, policy_version 64823 (0.0009) [2023-10-10 19:06:43,010][123614] Updated weights for policy 1, policy_version 64710 (0.0010) [2023-10-10 19:06:43,385][123614] Updated weights for policy 1, policy_version 64720 (0.0008) [2023-10-10 19:06:43,754][123614] Updated weights for policy 1, policy_version 64730 (0.0007) [2023-10-10 19:06:43,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 132644864. Throughput: 0: 1808.6, 1: 1818.5. Samples: 33169482. Policy #0 lag: (min: 29.0, avg: 40.2, max: 61.0) [2023-10-10 19:06:43,789][122664] Avg episode reward: [(0, '66.190'), (1, '69.670')] [2023-10-10 19:06:45,584][123582] Updated weights for policy 0, policy_version 64833 (0.0007) [2023-10-10 19:06:45,954][123582] Updated weights for policy 0, policy_version 64843 (0.0008) [2023-10-10 19:06:46,337][123582] Updated weights for policy 0, policy_version 64853 (0.0009) [2023-10-10 19:06:46,708][123582] Updated weights for policy 0, policy_version 64863 (0.0009) [2023-10-10 19:06:47,431][123614] Updated weights for policy 1, policy_version 64740 (0.0008) [2023-10-10 19:06:47,803][123614] Updated weights for policy 1, policy_version 64750 (0.0008) [2023-10-10 19:06:48,166][123614] Updated weights for policy 1, policy_version 64760 (0.0009) [2023-10-10 19:06:48,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 132743168. Throughput: 0: 1798.1, 1: 1817.1. Samples: 33190752. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:06:48,789][122664] Avg episode reward: [(0, '65.620'), (1, '69.040')] [2023-10-10 19:06:50,470][123582] Updated weights for policy 0, policy_version 64873 (0.0010) [2023-10-10 19:06:50,842][123582] Updated weights for policy 0, policy_version 64883 (0.0007) [2023-10-10 19:06:51,212][123582] Updated weights for policy 0, policy_version 64893 (0.0009) [2023-10-10 19:06:51,881][123614] Updated weights for policy 1, policy_version 64770 (0.0008) [2023-10-10 19:06:52,253][123614] Updated weights for policy 1, policy_version 64780 (0.0007) [2023-10-10 19:06:52,627][123614] Updated weights for policy 1, policy_version 64790 (0.0007) [2023-10-10 19:06:52,992][123614] Updated weights for policy 1, policy_version 64800 (0.0010) [2023-10-10 19:06:53,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 132808704. Throughput: 0: 1794.9, 1: 1822.9. Samples: 33212246. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:06:53,789][122664] Avg episode reward: [(0, '65.180'), (1, '71.220')] [2023-10-10 19:06:54,951][123582] Updated weights for policy 0, policy_version 64903 (0.0011) [2023-10-10 19:06:55,320][123582] Updated weights for policy 0, policy_version 64913 (0.0011) [2023-10-10 19:06:55,701][123582] Updated weights for policy 0, policy_version 64923 (0.0007) [2023-10-10 19:06:56,659][123614] Updated weights for policy 1, policy_version 64810 (0.0008) [2023-10-10 19:06:57,028][123614] Updated weights for policy 1, policy_version 64820 (0.0010) [2023-10-10 19:06:57,405][123614] Updated weights for policy 1, policy_version 64830 (0.0010) [2023-10-10 19:06:58,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 132874240. Throughput: 0: 1799.2, 1: 1811.7. Samples: 33223090. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:06:58,789][122664] Avg episode reward: [(0, '58.910'), (1, '68.760')] [2023-10-10 19:06:59,303][123582] Updated weights for policy 0, policy_version 64933 (0.0008) [2023-10-10 19:06:59,672][123582] Updated weights for policy 0, policy_version 64943 (0.0007) [2023-10-10 19:07:00,050][123582] Updated weights for policy 0, policy_version 64953 (0.0008) [2023-10-10 19:07:01,208][123614] Updated weights for policy 1, policy_version 64840 (0.0008) [2023-10-10 19:07:01,581][123614] Updated weights for policy 1, policy_version 64850 (0.0007) [2023-10-10 19:07:01,947][123614] Updated weights for policy 1, policy_version 64860 (0.0008) [2023-10-10 19:07:03,601][123582] Updated weights for policy 0, policy_version 64963 (0.0008) [2023-10-10 19:07:03,788][122664] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 132939776. Throughput: 0: 1809.0, 1: 1816.2. Samples: 33245010. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:07:03,788][122664] Avg episode reward: [(0, '57.720'), (1, '72.830')] [2023-10-10 19:07:03,973][123582] Updated weights for policy 0, policy_version 64973 (0.0008) [2023-10-10 19:07:04,341][123582] Updated weights for policy 0, policy_version 64983 (0.0008) [2023-10-10 19:07:05,772][123614] Updated weights for policy 1, policy_version 64870 (0.0008) [2023-10-10 19:07:06,133][123614] Updated weights for policy 1, policy_version 64880 (0.0009) [2023-10-10 19:07:06,498][123614] Updated weights for policy 1, policy_version 64890 (0.0008) [2023-10-10 19:07:07,960][123582] Updated weights for policy 0, policy_version 64993 (0.0008) [2023-10-10 19:07:08,332][123582] Updated weights for policy 0, policy_version 65003 (0.0008) [2023-10-10 19:07:08,711][123582] Updated weights for policy 0, policy_version 65013 (0.0008) [2023-10-10 19:07:08,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 133005312. Throughput: 0: 1815.7, 1: 1813.3. Samples: 33267124. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:07:08,788][122664] Avg episode reward: [(0, '60.550'), (1, '72.310')] [2023-10-10 19:07:09,083][123582] Updated weights for policy 0, policy_version 65023 (0.0010) [2023-10-10 19:07:10,295][123614] Updated weights for policy 1, policy_version 64900 (0.0010) [2023-10-10 19:07:10,679][123614] Updated weights for policy 1, policy_version 64910 (0.0008) [2023-10-10 19:07:11,041][123614] Updated weights for policy 1, policy_version 64920 (0.0009) [2023-10-10 19:07:12,767][123582] Updated weights for policy 0, policy_version 65033 (0.0009) [2023-10-10 19:07:13,144][123582] Updated weights for policy 0, policy_version 65043 (0.0008) [2023-10-10 19:07:13,508][123582] Updated weights for policy 0, policy_version 65053 (0.0007) [2023-10-10 19:07:13,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 133103616. Throughput: 0: 1815.8, 1: 1819.3. Samples: 33277722. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:07:13,789][122664] Avg episode reward: [(0, '61.680'), (1, '69.500')] [2023-10-10 19:07:14,580][123614] Updated weights for policy 1, policy_version 64930 (0.0008) [2023-10-10 19:07:14,945][123614] Updated weights for policy 1, policy_version 64940 (0.0009) [2023-10-10 19:07:15,319][123614] Updated weights for policy 1, policy_version 64950 (0.0009) [2023-10-10 19:07:15,688][123614] Updated weights for policy 1, policy_version 64960 (0.0010) [2023-10-10 19:07:17,180][123582] Updated weights for policy 0, policy_version 65063 (0.0009) [2023-10-10 19:07:17,552][123582] Updated weights for policy 0, policy_version 65073 (0.0009) [2023-10-10 19:07:17,918][123582] Updated weights for policy 0, policy_version 65083 (0.0008) [2023-10-10 19:07:18,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 133169152. Throughput: 0: 1825.9, 1: 1812.8. Samples: 33300208. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:07:18,789][122664] Avg episode reward: [(0, '63.770'), (1, '67.630')] [2023-10-10 19:07:19,367][123614] Updated weights for policy 1, policy_version 64970 (0.0010) [2023-10-10 19:07:19,736][123614] Updated weights for policy 1, policy_version 64980 (0.0007) [2023-10-10 19:07:20,105][123614] Updated weights for policy 1, policy_version 64990 (0.0010) [2023-10-10 19:07:21,745][123582] Updated weights for policy 0, policy_version 65093 (0.0008) [2023-10-10 19:07:22,138][123582] Updated weights for policy 0, policy_version 65103 (0.0008) [2023-10-10 19:07:22,511][123582] Updated weights for policy 0, policy_version 65113 (0.0009) [2023-10-10 19:07:23,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 133234688. Throughput: 0: 1823.6, 1: 1809.4. Samples: 33321446. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:07:23,789][122664] Avg episode reward: [(0, '63.880'), (1, '66.140')] [2023-10-10 19:07:23,826][123614] Updated weights for policy 1, policy_version 65000 (0.0010) [2023-10-10 19:07:24,200][123614] Updated weights for policy 1, policy_version 65010 (0.0010) [2023-10-10 19:07:24,570][123614] Updated weights for policy 1, policy_version 65020 (0.0009) [2023-10-10 19:07:26,214][123582] Updated weights for policy 0, policy_version 65123 (0.0009) [2023-10-10 19:07:26,601][123582] Updated weights for policy 0, policy_version 65133 (0.0007) [2023-10-10 19:07:26,975][123582] Updated weights for policy 0, policy_version 65143 (0.0010) [2023-10-10 19:07:28,199][123614] Updated weights for policy 1, policy_version 65030 (0.0008) [2023-10-10 19:07:28,569][123614] Updated weights for policy 1, policy_version 65040 (0.0008) [2023-10-10 19:07:28,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 133300224. Throughput: 0: 1823.0, 1: 1802.9. Samples: 33332650. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:07:28,788][122664] Avg episode reward: [(0, '61.110'), (1, '66.480')] [2023-10-10 19:07:28,934][123614] Updated weights for policy 1, policy_version 65050 (0.0008) [2023-10-10 19:07:30,665][123582] Updated weights for policy 0, policy_version 65153 (0.0010) [2023-10-10 19:07:31,026][123582] Updated weights for policy 0, policy_version 65163 (0.0010) [2023-10-10 19:07:31,405][123582] Updated weights for policy 0, policy_version 65173 (0.0007) [2023-10-10 19:07:31,767][123582] Updated weights for policy 0, policy_version 65183 (0.0007) [2023-10-10 19:07:32,564][123614] Updated weights for policy 1, policy_version 65060 (0.0008) [2023-10-10 19:07:32,935][123614] Updated weights for policy 1, policy_version 65070 (0.0009) [2023-10-10 19:07:33,289][123614] Updated weights for policy 1, policy_version 65080 (0.0008) [2023-10-10 19:07:33,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 133398528. Throughput: 0: 1823.6, 1: 1806.8. Samples: 33354120. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:07:33,788][122664] Avg episode reward: [(0, '61.180'), (1, '68.040')] [2023-10-10 19:07:35,353][123582] Updated weights for policy 0, policy_version 65193 (0.0008) [2023-10-10 19:07:35,729][123582] Updated weights for policy 0, policy_version 65203 (0.0009) [2023-10-10 19:07:36,100][123582] Updated weights for policy 0, policy_version 65213 (0.0007) [2023-10-10 19:07:37,115][123614] Updated weights for policy 1, policy_version 65090 (0.0009) [2023-10-10 19:07:37,479][123614] Updated weights for policy 1, policy_version 65100 (0.0009) [2023-10-10 19:07:37,854][123614] Updated weights for policy 1, policy_version 65110 (0.0009) [2023-10-10 19:07:38,223][123614] Updated weights for policy 1, policy_version 65120 (0.0008) [2023-10-10 19:07:38,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 133464064. Throughput: 0: 1835.4, 1: 1804.0. Samples: 33376020. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:07:38,789][122664] Avg episode reward: [(0, '65.330'), (1, '67.930')] [2023-10-10 19:07:39,609][123582] Updated weights for policy 0, policy_version 65223 (0.0009) [2023-10-10 19:07:39,995][123582] Updated weights for policy 0, policy_version 65233 (0.0007) [2023-10-10 19:07:40,364][123582] Updated weights for policy 0, policy_version 65243 (0.0008) [2023-10-10 19:07:41,975][123614] Updated weights for policy 1, policy_version 65130 (0.0009) [2023-10-10 19:07:42,336][123614] Updated weights for policy 1, policy_version 65140 (0.0009) [2023-10-10 19:07:42,712][123614] Updated weights for policy 1, policy_version 65150 (0.0009) [2023-10-10 19:07:43,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 133529600. Throughput: 0: 1831.9, 1: 1814.2. Samples: 33387164. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:07:43,789][122664] Avg episode reward: [(0, '66.450'), (1, '61.880')] [2023-10-10 19:07:44,053][123582] Updated weights for policy 0, policy_version 65253 (0.0008) [2023-10-10 19:07:44,423][123582] Updated weights for policy 0, policy_version 65263 (0.0010) [2023-10-10 19:07:44,810][123582] Updated weights for policy 0, policy_version 65273 (0.0008) [2023-10-10 19:07:46,413][123614] Updated weights for policy 1, policy_version 65160 (0.0009) [2023-10-10 19:07:46,782][123614] Updated weights for policy 1, policy_version 65170 (0.0009) [2023-10-10 19:07:47,159][123614] Updated weights for policy 1, policy_version 65180 (0.0009) [2023-10-10 19:07:48,432][123582] Updated weights for policy 0, policy_version 65283 (0.0008) [2023-10-10 19:07:48,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 133595136. Throughput: 0: 1833.4, 1: 1806.1. Samples: 33408790. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:07:48,788][122664] Avg episode reward: [(0, '62.430'), (1, '62.730')] [2023-10-10 19:07:48,801][123582] Updated weights for policy 0, policy_version 65293 (0.0010) [2023-10-10 19:07:49,177][123582] Updated weights for policy 0, policy_version 65303 (0.0011) [2023-10-10 19:07:50,846][123614] Updated weights for policy 1, policy_version 65190 (0.0008) [2023-10-10 19:07:51,224][123614] Updated weights for policy 1, policy_version 65200 (0.0008) [2023-10-10 19:07:51,590][123614] Updated weights for policy 1, policy_version 65210 (0.0009) [2023-10-10 19:07:52,967][123582] Updated weights for policy 0, policy_version 65313 (0.0009) [2023-10-10 19:07:53,329][123582] Updated weights for policy 0, policy_version 65323 (0.0008) [2023-10-10 19:07:53,712][123582] Updated weights for policy 0, policy_version 65333 (0.0008) [2023-10-10 19:07:53,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 133660672. Throughput: 0: 1829.7, 1: 1812.3. Samples: 33431016. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:07:53,789][122664] Avg episode reward: [(0, '62.710'), (1, '62.670')] [2023-10-10 19:07:53,800][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000065216_66781184.pth... [2023-10-10 19:07:53,833][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000063520_65044480.pth [2023-10-10 19:07:54,079][123582] Updated weights for policy 0, policy_version 65343 (0.0008) [2023-10-10 19:07:54,117][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000065344_66912256.pth... [2023-10-10 19:07:54,146][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000063616_65142784.pth [2023-10-10 19:07:55,364][123614] Updated weights for policy 1, policy_version 65220 (0.0009) [2023-10-10 19:07:55,745][123614] Updated weights for policy 1, policy_version 65230 (0.0010) [2023-10-10 19:07:56,117][123614] Updated weights for policy 1, policy_version 65240 (0.0009) [2023-10-10 19:07:57,713][123582] Updated weights for policy 0, policy_version 65353 (0.0010) [2023-10-10 19:07:58,093][123582] Updated weights for policy 0, policy_version 65363 (0.0008) [2023-10-10 19:07:58,457][123582] Updated weights for policy 0, policy_version 65373 (0.0008) [2023-10-10 19:07:58,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 133758976. Throughput: 0: 1828.0, 1: 1807.4. Samples: 33441318. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:07:58,789][122664] Avg episode reward: [(0, '61.460'), (1, '62.070')] [2023-10-10 19:07:59,779][123614] Updated weights for policy 1, policy_version 65250 (0.0010) [2023-10-10 19:08:00,156][123614] Updated weights for policy 1, policy_version 65260 (0.0009) [2023-10-10 19:08:00,526][123614] Updated weights for policy 1, policy_version 65270 (0.0008) [2023-10-10 19:08:00,886][123614] Updated weights for policy 1, policy_version 65280 (0.0008) [2023-10-10 19:08:02,194][123582] Updated weights for policy 0, policy_version 65383 (0.0007) [2023-10-10 19:08:02,569][123582] Updated weights for policy 0, policy_version 65393 (0.0010) [2023-10-10 19:08:02,949][123582] Updated weights for policy 0, policy_version 65403 (0.0008) [2023-10-10 19:08:03,788][122664] Fps is (10 sec: 16384.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 133824512. Throughput: 0: 1818.0, 1: 1811.2. Samples: 33463518. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:08:03,789][122664] Avg episode reward: [(0, '63.380'), (1, '62.670')] [2023-10-10 19:08:04,627][123614] Updated weights for policy 1, policy_version 65290 (0.0009) [2023-10-10 19:08:04,994][123614] Updated weights for policy 1, policy_version 65300 (0.0009) [2023-10-10 19:08:05,358][123614] Updated weights for policy 1, policy_version 65310 (0.0010) [2023-10-10 19:08:06,670][123582] Updated weights for policy 0, policy_version 65413 (0.0008) [2023-10-10 19:08:07,031][123582] Updated weights for policy 0, policy_version 65423 (0.0009) [2023-10-10 19:08:07,399][123582] Updated weights for policy 0, policy_version 65433 (0.0007) [2023-10-10 19:08:08,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 133890048. Throughput: 0: 1821.9, 1: 1824.1. Samples: 33485518. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:08:08,788][122664] Avg episode reward: [(0, '62.780'), (1, '61.880')] [2023-10-10 19:08:09,006][123614] Updated weights for policy 1, policy_version 65320 (0.0009) [2023-10-10 19:08:09,369][123614] Updated weights for policy 1, policy_version 65330 (0.0009) [2023-10-10 19:08:09,735][123614] Updated weights for policy 1, policy_version 65340 (0.0011) [2023-10-10 19:08:11,115][123582] Updated weights for policy 0, policy_version 65443 (0.0009) [2023-10-10 19:08:11,483][123582] Updated weights for policy 0, policy_version 65453 (0.0007) [2023-10-10 19:08:11,854][123582] Updated weights for policy 0, policy_version 65463 (0.0008) [2023-10-10 19:08:13,403][123614] Updated weights for policy 1, policy_version 65350 (0.0008) [2023-10-10 19:08:13,775][123614] Updated weights for policy 1, policy_version 65360 (0.0010) [2023-10-10 19:08:13,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 133955584. Throughput: 0: 1820.2, 1: 1818.2. Samples: 33496380. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:08:13,788][122664] Avg episode reward: [(0, '65.500'), (1, '62.920')] [2023-10-10 19:08:14,155][123614] Updated weights for policy 1, policy_version 65370 (0.0008) [2023-10-10 19:08:15,620][123582] Updated weights for policy 0, policy_version 65473 (0.0008) [2023-10-10 19:08:16,004][123582] Updated weights for policy 0, policy_version 65483 (0.0008) [2023-10-10 19:08:16,377][123582] Updated weights for policy 0, policy_version 65493 (0.0008) [2023-10-10 19:08:16,748][123582] Updated weights for policy 0, policy_version 65503 (0.0009) [2023-10-10 19:08:17,806][123614] Updated weights for policy 1, policy_version 65380 (0.0007) [2023-10-10 19:08:18,175][123614] Updated weights for policy 1, policy_version 65390 (0.0009) [2023-10-10 19:08:18,552][123614] Updated weights for policy 1, policy_version 65400 (0.0009) [2023-10-10 19:08:18,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 134021120. Throughput: 0: 1812.9, 1: 1822.6. Samples: 33517718. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 19:08:18,788][122664] Avg episode reward: [(0, '66.530'), (1, '67.450')] [2023-10-10 19:08:20,512][123582] Updated weights for policy 0, policy_version 65513 (0.0008) [2023-10-10 19:08:20,886][123582] Updated weights for policy 0, policy_version 65523 (0.0009) [2023-10-10 19:08:21,259][123582] Updated weights for policy 0, policy_version 65533 (0.0010) [2023-10-10 19:08:22,243][123614] Updated weights for policy 1, policy_version 65410 (0.0010) [2023-10-10 19:08:22,603][123614] Updated weights for policy 1, policy_version 65420 (0.0009) [2023-10-10 19:08:22,970][123614] Updated weights for policy 1, policy_version 65430 (0.0008) [2023-10-10 19:08:23,352][123614] Updated weights for policy 1, policy_version 65440 (0.0008) [2023-10-10 19:08:23,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 134119424. Throughput: 0: 1805.7, 1: 1815.7. Samples: 33538982. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 19:08:23,789][122664] Avg episode reward: [(0, '66.320'), (1, '65.410')] [2023-10-10 19:08:25,002][123582] Updated weights for policy 0, policy_version 65543 (0.0008) [2023-10-10 19:08:25,364][123582] Updated weights for policy 0, policy_version 65553 (0.0007) [2023-10-10 19:08:25,744][123582] Updated weights for policy 0, policy_version 65563 (0.0007) [2023-10-10 19:08:26,973][123614] Updated weights for policy 1, policy_version 65450 (0.0008) [2023-10-10 19:08:27,339][123614] Updated weights for policy 1, policy_version 65460 (0.0009) [2023-10-10 19:08:27,704][123614] Updated weights for policy 1, policy_version 65470 (0.0010) [2023-10-10 19:08:28,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 134184960. Throughput: 0: 1805.5, 1: 1818.9. Samples: 33550262. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 19:08:28,789][122664] Avg episode reward: [(0, '64.980'), (1, '60.870')] [2023-10-10 19:08:29,383][123582] Updated weights for policy 0, policy_version 65573 (0.0007) [2023-10-10 19:08:29,752][123582] Updated weights for policy 0, policy_version 65583 (0.0008) [2023-10-10 19:08:30,111][123582] Updated weights for policy 0, policy_version 65593 (0.0007) [2023-10-10 19:08:31,451][123614] Updated weights for policy 1, policy_version 65480 (0.0009) [2023-10-10 19:08:31,823][123614] Updated weights for policy 1, policy_version 65490 (0.0008) [2023-10-10 19:08:32,194][123614] Updated weights for policy 1, policy_version 65500 (0.0008) [2023-10-10 19:08:33,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 134250496. Throughput: 0: 1804.7, 1: 1817.7. Samples: 33571802. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 19:08:33,789][122664] Avg episode reward: [(0, '72.150'), (1, '61.110')] [2023-10-10 19:08:33,841][123582] Updated weights for policy 0, policy_version 65603 (0.0009) [2023-10-10 19:08:34,212][123582] Updated weights for policy 0, policy_version 65613 (0.0008) [2023-10-10 19:08:34,575][123582] Updated weights for policy 0, policy_version 65623 (0.0010) [2023-10-10 19:08:35,915][123614] Updated weights for policy 1, policy_version 65510 (0.0009) [2023-10-10 19:08:36,279][123614] Updated weights for policy 1, policy_version 65520 (0.0010) [2023-10-10 19:08:36,653][123614] Updated weights for policy 1, policy_version 65530 (0.0008) [2023-10-10 19:08:38,252][123582] Updated weights for policy 0, policy_version 65633 (0.0009) [2023-10-10 19:08:38,623][123582] Updated weights for policy 0, policy_version 65643 (0.0009) [2023-10-10 19:08:38,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 134316032. Throughput: 0: 1810.6, 1: 1819.5. Samples: 33594370. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 19:08:38,788][122664] Avg episode reward: [(0, '69.310'), (1, '64.210')] [2023-10-10 19:08:38,992][123582] Updated weights for policy 0, policy_version 65653 (0.0009) [2023-10-10 19:08:39,355][123582] Updated weights for policy 0, policy_version 65663 (0.0008) [2023-10-10 19:08:40,347][123614] Updated weights for policy 1, policy_version 65540 (0.0008) [2023-10-10 19:08:40,744][123614] Updated weights for policy 1, policy_version 65550 (0.0007) [2023-10-10 19:08:41,105][123614] Updated weights for policy 1, policy_version 65560 (0.0007) [2023-10-10 19:08:43,182][123582] Updated weights for policy 0, policy_version 65673 (0.0007) [2023-10-10 19:08:43,562][123582] Updated weights for policy 0, policy_version 65683 (0.0009) [2023-10-10 19:08:43,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 134381568. Throughput: 0: 1799.0, 1: 1825.1. Samples: 33604402. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 19:08:43,788][122664] Avg episode reward: [(0, '68.280'), (1, '60.710')] [2023-10-10 19:08:43,937][123582] Updated weights for policy 0, policy_version 65693 (0.0009) [2023-10-10 19:08:44,857][123614] Updated weights for policy 1, policy_version 65570 (0.0009) [2023-10-10 19:08:45,219][123614] Updated weights for policy 1, policy_version 65580 (0.0008) [2023-10-10 19:08:45,588][123614] Updated weights for policy 1, policy_version 65590 (0.0008) [2023-10-10 19:08:45,956][123614] Updated weights for policy 1, policy_version 65600 (0.0007) [2023-10-10 19:08:47,717][123582] Updated weights for policy 0, policy_version 65703 (0.0011) [2023-10-10 19:08:48,089][123582] Updated weights for policy 0, policy_version 65713 (0.0009) [2023-10-10 19:08:48,467][123582] Updated weights for policy 0, policy_version 65723 (0.0008) [2023-10-10 19:08:48,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 134479872. Throughput: 0: 1815.9, 1: 1823.6. Samples: 33627294. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 19:08:48,788][122664] Avg episode reward: [(0, '66.880'), (1, '67.840')] [2023-10-10 19:08:49,627][123614] Updated weights for policy 1, policy_version 65610 (0.0008) [2023-10-10 19:08:50,008][123614] Updated weights for policy 1, policy_version 65620 (0.0008) [2023-10-10 19:08:50,375][123614] Updated weights for policy 1, policy_version 65630 (0.0007) [2023-10-10 19:08:52,052][123582] Updated weights for policy 0, policy_version 65733 (0.0007) [2023-10-10 19:08:52,426][123582] Updated weights for policy 0, policy_version 65743 (0.0010) [2023-10-10 19:08:52,809][123582] Updated weights for policy 0, policy_version 65753 (0.0009) [2023-10-10 19:08:53,788][122664] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 134545408. Throughput: 0: 1802.2, 1: 1826.8. Samples: 33648822. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 19:08:53,789][122664] Avg episode reward: [(0, '67.350'), (1, '69.320')] [2023-10-10 19:08:53,980][123614] Updated weights for policy 1, policy_version 65640 (0.0009) [2023-10-10 19:08:54,348][123614] Updated weights for policy 1, policy_version 65650 (0.0009) [2023-10-10 19:08:54,722][123614] Updated weights for policy 1, policy_version 65660 (0.0008) [2023-10-10 19:08:56,646][123582] Updated weights for policy 0, policy_version 65763 (0.0008) [2023-10-10 19:08:57,044][123582] Updated weights for policy 0, policy_version 65773 (0.0008) [2023-10-10 19:08:57,415][123582] Updated weights for policy 0, policy_version 65783 (0.0011) [2023-10-10 19:08:58,204][123614] Updated weights for policy 1, policy_version 65670 (0.0008) [2023-10-10 19:08:58,576][123614] Updated weights for policy 1, policy_version 65680 (0.0007) [2023-10-10 19:08:58,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 134610944. Throughput: 0: 1811.1, 1: 1830.0. Samples: 33660232. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 19:08:58,789][122664] Avg episode reward: [(0, '71.010'), (1, '69.720')] [2023-10-10 19:08:58,953][123614] Updated weights for policy 1, policy_version 65690 (0.0007) [2023-10-10 19:09:01,100][123582] Updated weights for policy 0, policy_version 65793 (0.0009) [2023-10-10 19:09:01,464][123582] Updated weights for policy 0, policy_version 65803 (0.0009) [2023-10-10 19:09:01,840][123582] Updated weights for policy 0, policy_version 65813 (0.0010) [2023-10-10 19:09:02,210][123582] Updated weights for policy 0, policy_version 65823 (0.0010) [2023-10-10 19:09:02,684][123614] Updated weights for policy 1, policy_version 65700 (0.0010) [2023-10-10 19:09:03,056][123614] Updated weights for policy 1, policy_version 65710 (0.0010) [2023-10-10 19:09:03,422][123614] Updated weights for policy 1, policy_version 65720 (0.0010) [2023-10-10 19:09:03,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 134709248. Throughput: 0: 1802.9, 1: 1827.6. Samples: 33681088. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:09:03,789][122664] Avg episode reward: [(0, '69.500'), (1, '70.030')] [2023-10-10 19:09:05,944][123582] Updated weights for policy 0, policy_version 65833 (0.0008) [2023-10-10 19:09:06,320][123582] Updated weights for policy 0, policy_version 65843 (0.0008) [2023-10-10 19:09:06,693][123582] Updated weights for policy 0, policy_version 65853 (0.0009) [2023-10-10 19:09:07,212][123614] Updated weights for policy 1, policy_version 65730 (0.0010) [2023-10-10 19:09:07,587][123614] Updated weights for policy 1, policy_version 65740 (0.0011) [2023-10-10 19:09:07,962][123614] Updated weights for policy 1, policy_version 65750 (0.0009) [2023-10-10 19:09:08,322][123614] Updated weights for policy 1, policy_version 65760 (0.0008) [2023-10-10 19:09:08,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 134774784. Throughput: 0: 1808.0, 1: 1825.9. Samples: 33702510. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:09:08,788][122664] Avg episode reward: [(0, '69.780'), (1, '69.560')] [2023-10-10 19:09:10,429][123582] Updated weights for policy 0, policy_version 65863 (0.0008) [2023-10-10 19:09:10,801][123582] Updated weights for policy 0, policy_version 65873 (0.0008) [2023-10-10 19:09:11,173][123582] Updated weights for policy 0, policy_version 65883 (0.0008) [2023-10-10 19:09:12,115][123614] Updated weights for policy 1, policy_version 65770 (0.0008) [2023-10-10 19:09:12,489][123614] Updated weights for policy 1, policy_version 65780 (0.0007) [2023-10-10 19:09:12,850][123614] Updated weights for policy 1, policy_version 65790 (0.0010) [2023-10-10 19:09:13,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 134840320. Throughput: 0: 1807.6, 1: 1826.2. Samples: 33713786. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:09:13,789][122664] Avg episode reward: [(0, '71.790'), (1, '57.730')] [2023-10-10 19:09:15,007][123582] Updated weights for policy 0, policy_version 65893 (0.0010) [2023-10-10 19:09:15,382][123582] Updated weights for policy 0, policy_version 65903 (0.0010) [2023-10-10 19:09:15,762][123582] Updated weights for policy 0, policy_version 65913 (0.0008) [2023-10-10 19:09:16,535][123614] Updated weights for policy 1, policy_version 65800 (0.0008) [2023-10-10 19:09:16,916][123614] Updated weights for policy 1, policy_version 65810 (0.0007) [2023-10-10 19:09:17,281][123614] Updated weights for policy 1, policy_version 65820 (0.0007) [2023-10-10 19:09:18,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 134905856. Throughput: 0: 1799.5, 1: 1827.4. Samples: 33735010. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:09:18,788][122664] Avg episode reward: [(0, '70.460'), (1, '55.270')] [2023-10-10 19:09:19,371][123582] Updated weights for policy 0, policy_version 65923 (0.0009) [2023-10-10 19:09:19,758][123582] Updated weights for policy 0, policy_version 65933 (0.0008) [2023-10-10 19:09:20,129][123582] Updated weights for policy 0, policy_version 65943 (0.0009) [2023-10-10 19:09:20,855][123614] Updated weights for policy 1, policy_version 65830 (0.0008) [2023-10-10 19:09:21,220][123614] Updated weights for policy 1, policy_version 65840 (0.0007) [2023-10-10 19:09:21,588][123614] Updated weights for policy 1, policy_version 65850 (0.0007) [2023-10-10 19:09:23,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 134971392. Throughput: 0: 1807.6, 1: 1828.2. Samples: 33757980. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:09:23,789][122664] Avg episode reward: [(0, '64.910'), (1, '57.530')] [2023-10-10 19:09:23,853][123582] Updated weights for policy 0, policy_version 65953 (0.0009) [2023-10-10 19:09:24,218][123582] Updated weights for policy 0, policy_version 65963 (0.0008) [2023-10-10 19:09:24,591][123582] Updated weights for policy 0, policy_version 65973 (0.0009) [2023-10-10 19:09:24,964][123582] Updated weights for policy 0, policy_version 65983 (0.0008) [2023-10-10 19:09:25,223][123614] Updated weights for policy 1, policy_version 65860 (0.0010) [2023-10-10 19:09:25,617][123614] Updated weights for policy 1, policy_version 65870 (0.0007) [2023-10-10 19:09:25,982][123614] Updated weights for policy 1, policy_version 65880 (0.0007) [2023-10-10 19:09:28,532][123582] Updated weights for policy 0, policy_version 65993 (0.0008) [2023-10-10 19:09:28,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 135036928. Throughput: 0: 1807.7, 1: 1829.8. Samples: 33768088. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:09:28,789][122664] Avg episode reward: [(0, '63.590'), (1, '56.420')] [2023-10-10 19:09:28,911][123582] Updated weights for policy 0, policy_version 66003 (0.0008) [2023-10-10 19:09:29,285][123582] Updated weights for policy 0, policy_version 66013 (0.0008) [2023-10-10 19:09:29,542][123614] Updated weights for policy 1, policy_version 65890 (0.0007) [2023-10-10 19:09:29,909][123614] Updated weights for policy 1, policy_version 65900 (0.0009) [2023-10-10 19:09:30,281][123614] Updated weights for policy 1, policy_version 65910 (0.0007) [2023-10-10 19:09:30,654][123614] Updated weights for policy 1, policy_version 65920 (0.0007) [2023-10-10 19:09:32,922][123582] Updated weights for policy 0, policy_version 66023 (0.0008) [2023-10-10 19:09:33,289][123582] Updated weights for policy 0, policy_version 66033 (0.0012) [2023-10-10 19:09:33,660][123582] Updated weights for policy 0, policy_version 66043 (0.0009) [2023-10-10 19:09:33,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 135102464. Throughput: 0: 1811.8, 1: 1832.4. Samples: 33791284. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:09:33,788][122664] Avg episode reward: [(0, '68.030'), (1, '60.590')] [2023-10-10 19:09:34,262][123614] Updated weights for policy 1, policy_version 65930 (0.0009) [2023-10-10 19:09:34,624][123614] Updated weights for policy 1, policy_version 65940 (0.0010) [2023-10-10 19:09:35,003][123614] Updated weights for policy 1, policy_version 65950 (0.0010) [2023-10-10 19:09:37,369][123582] Updated weights for policy 0, policy_version 66053 (0.0008) [2023-10-10 19:09:37,748][123582] Updated weights for policy 0, policy_version 66063 (0.0007) [2023-10-10 19:09:38,113][123582] Updated weights for policy 0, policy_version 66073 (0.0010) [2023-10-10 19:09:38,756][123614] Updated weights for policy 1, policy_version 65960 (0.0007) [2023-10-10 19:09:38,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 135200768. Throughput: 0: 1808.5, 1: 1819.0. Samples: 33812056. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:09:38,789][122664] Avg episode reward: [(0, '69.610'), (1, '59.100')] [2023-10-10 19:09:39,121][123614] Updated weights for policy 1, policy_version 65970 (0.0007) [2023-10-10 19:09:39,493][123614] Updated weights for policy 1, policy_version 65980 (0.0009) [2023-10-10 19:09:41,889][123582] Updated weights for policy 0, policy_version 66083 (0.0009) [2023-10-10 19:09:42,280][123582] Updated weights for policy 0, policy_version 66093 (0.0007) [2023-10-10 19:09:42,651][123582] Updated weights for policy 0, policy_version 66103 (0.0007) [2023-10-10 19:09:43,116][123614] Updated weights for policy 1, policy_version 65990 (0.0008) [2023-10-10 19:09:43,484][123614] Updated weights for policy 1, policy_version 66000 (0.0011) [2023-10-10 19:09:43,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 135266304. Throughput: 0: 1812.6, 1: 1825.1. Samples: 33823926. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:09:43,788][122664] Avg episode reward: [(0, '74.170'), (1, '61.690')] [2023-10-10 19:09:43,858][123614] Updated weights for policy 1, policy_version 66010 (0.0011) [2023-10-10 19:09:46,254][123582] Updated weights for policy 0, policy_version 66113 (0.0010) [2023-10-10 19:09:46,630][123582] Updated weights for policy 0, policy_version 66123 (0.0007) [2023-10-10 19:09:47,001][123582] Updated weights for policy 0, policy_version 66133 (0.0008) [2023-10-10 19:09:47,374][123582] Updated weights for policy 0, policy_version 66143 (0.0007) [2023-10-10 19:09:47,435][123614] Updated weights for policy 1, policy_version 66020 (0.0010) [2023-10-10 19:09:47,802][123614] Updated weights for policy 1, policy_version 66030 (0.0009) [2023-10-10 19:09:48,160][123614] Updated weights for policy 1, policy_version 66040 (0.0010) [2023-10-10 19:09:48,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 135364608. Throughput: 0: 1817.0, 1: 1823.2. Samples: 33844898. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:09:48,789][122664] Avg episode reward: [(0, '72.990'), (1, '62.540')] [2023-10-10 19:09:51,028][123582] Updated weights for policy 0, policy_version 66153 (0.0007) [2023-10-10 19:09:51,395][123582] Updated weights for policy 0, policy_version 66163 (0.0007) [2023-10-10 19:09:51,778][123582] Updated weights for policy 0, policy_version 66173 (0.0009) [2023-10-10 19:09:51,859][123614] Updated weights for policy 1, policy_version 66050 (0.0007) [2023-10-10 19:09:52,223][123614] Updated weights for policy 1, policy_version 66060 (0.0007) [2023-10-10 19:09:52,597][123614] Updated weights for policy 1, policy_version 66070 (0.0007) [2023-10-10 19:09:52,963][123614] Updated weights for policy 1, policy_version 66080 (0.0007) [2023-10-10 19:09:53,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 135430144. Throughput: 0: 1806.8, 1: 1837.3. Samples: 33866494. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:09:53,788][122664] Avg episode reward: [(0, '71.070'), (1, '63.000')] [2023-10-10 19:09:53,798][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000066080_67665920.pth... [2023-10-10 19:09:53,798][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000066176_67764224.pth... [2023-10-10 19:09:53,838][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000064480_66027520.pth [2023-10-10 19:09:53,838][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000064384_65929216.pth [2023-10-10 19:09:55,546][123582] Updated weights for policy 0, policy_version 66183 (0.0009) [2023-10-10 19:09:55,924][123582] Updated weights for policy 0, policy_version 66193 (0.0010) [2023-10-10 19:09:56,291][123582] Updated weights for policy 0, policy_version 66203 (0.0010) [2023-10-10 19:09:56,526][123614] Updated weights for policy 1, policy_version 66090 (0.0008) [2023-10-10 19:09:56,897][123614] Updated weights for policy 1, policy_version 66100 (0.0007) [2023-10-10 19:09:57,264][123614] Updated weights for policy 1, policy_version 66110 (0.0010) [2023-10-10 19:09:58,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 135495680. Throughput: 0: 1810.0, 1: 1823.3. Samples: 33877288. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:09:58,789][122664] Avg episode reward: [(0, '73.020'), (1, '61.260')] [2023-10-10 19:10:00,146][123582] Updated weights for policy 0, policy_version 66213 (0.0009) [2023-10-10 19:10:00,518][123582] Updated weights for policy 0, policy_version 66223 (0.0007) [2023-10-10 19:10:00,893][123582] Updated weights for policy 0, policy_version 66233 (0.0008) [2023-10-10 19:10:00,903][123614] Updated weights for policy 1, policy_version 66120 (0.0007) [2023-10-10 19:10:01,270][123614] Updated weights for policy 1, policy_version 66130 (0.0010) [2023-10-10 19:10:01,636][123614] Updated weights for policy 1, policy_version 66140 (0.0009) [2023-10-10 19:10:03,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 135561216. Throughput: 0: 1805.2, 1: 1840.6. Samples: 33899074. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:10:03,789][122664] Avg episode reward: [(0, '74.640'), (1, '63.410')] [2023-10-10 19:10:04,579][123582] Updated weights for policy 0, policy_version 66243 (0.0009) [2023-10-10 19:10:04,951][123582] Updated weights for policy 0, policy_version 66253 (0.0010) [2023-10-10 19:10:05,326][123582] Updated weights for policy 0, policy_version 66263 (0.0010) [2023-10-10 19:10:05,511][123614] Updated weights for policy 1, policy_version 66150 (0.0009) [2023-10-10 19:10:05,877][123614] Updated weights for policy 1, policy_version 66160 (0.0008) [2023-10-10 19:10:06,248][123614] Updated weights for policy 1, policy_version 66170 (0.0008) [2023-10-10 19:10:08,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 135626752. Throughput: 0: 1806.5, 1: 1825.7. Samples: 33921432. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:10:08,788][122664] Avg episode reward: [(0, '72.210'), (1, '63.200')] [2023-10-10 19:10:09,004][123582] Updated weights for policy 0, policy_version 66273 (0.0007) [2023-10-10 19:10:09,367][123582] Updated weights for policy 0, policy_version 66283 (0.0008) [2023-10-10 19:10:09,746][123582] Updated weights for policy 0, policy_version 66293 (0.0009) [2023-10-10 19:10:10,097][123614] Updated weights for policy 1, policy_version 66180 (0.0008) [2023-10-10 19:10:10,120][123582] Updated weights for policy 0, policy_version 66303 (0.0010) [2023-10-10 19:10:10,462][123614] Updated weights for policy 1, policy_version 66190 (0.0007) [2023-10-10 19:10:10,831][123614] Updated weights for policy 1, policy_version 66200 (0.0008) [2023-10-10 19:10:13,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 135692288. Throughput: 0: 1805.3, 1: 1825.0. Samples: 33931454. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:10:13,789][122664] Avg episode reward: [(0, '76.810'), (1, '65.060')] [2023-10-10 19:10:13,957][123582] Updated weights for policy 0, policy_version 66313 (0.0008) [2023-10-10 19:10:14,331][123582] Updated weights for policy 0, policy_version 66323 (0.0008) [2023-10-10 19:10:14,526][123614] Updated weights for policy 1, policy_version 66210 (0.0008) [2023-10-10 19:10:14,701][123582] Updated weights for policy 0, policy_version 66333 (0.0009) [2023-10-10 19:10:14,919][123614] Updated weights for policy 1, policy_version 66220 (0.0009) [2023-10-10 19:10:15,286][123614] Updated weights for policy 1, policy_version 66230 (0.0007) [2023-10-10 19:10:15,661][123614] Updated weights for policy 1, policy_version 66240 (0.0007) [2023-10-10 19:10:18,439][123582] Updated weights for policy 0, policy_version 66343 (0.0007) [2023-10-10 19:10:18,788][122664] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 135757824. Throughput: 0: 1795.5, 1: 1821.4. Samples: 33954046. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:10:18,789][122664] Avg episode reward: [(0, '79.640'), (1, '65.780')] [2023-10-10 19:10:18,811][123582] Updated weights for policy 0, policy_version 66353 (0.0009) [2023-10-10 19:10:19,174][123582] Updated weights for policy 0, policy_version 66363 (0.0008) [2023-10-10 19:10:19,214][123614] Updated weights for policy 1, policy_version 66250 (0.0008) [2023-10-10 19:10:19,583][123614] Updated weights for policy 1, policy_version 66260 (0.0008) [2023-10-10 19:10:19,948][123614] Updated weights for policy 1, policy_version 66270 (0.0010) [2023-10-10 19:10:22,901][123582] Updated weights for policy 0, policy_version 66373 (0.0009) [2023-10-10 19:10:23,266][123582] Updated weights for policy 0, policy_version 66383 (0.0008) [2023-10-10 19:10:23,613][123614] Updated weights for policy 1, policy_version 66280 (0.0008) [2023-10-10 19:10:23,632][123582] Updated weights for policy 0, policy_version 66393 (0.0007) [2023-10-10 19:10:23,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 135823360. Throughput: 0: 1811.9, 1: 1818.0. Samples: 33975400. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:10:23,788][122664] Avg episode reward: [(0, '80.260'), (1, '66.910')] [2023-10-10 19:10:23,980][123614] Updated weights for policy 1, policy_version 66290 (0.0008) [2023-10-10 19:10:24,353][123614] Updated weights for policy 1, policy_version 66300 (0.0009) [2023-10-10 19:10:27,514][123582] Updated weights for policy 0, policy_version 66403 (0.0008) [2023-10-10 19:10:27,922][123582] Updated weights for policy 0, policy_version 66413 (0.0009) [2023-10-10 19:10:28,042][123614] Updated weights for policy 1, policy_version 66310 (0.0009) [2023-10-10 19:10:28,290][123582] Updated weights for policy 0, policy_version 66423 (0.0007) [2023-10-10 19:10:28,413][123614] Updated weights for policy 1, policy_version 66320 (0.0009) [2023-10-10 19:10:28,774][123614] Updated weights for policy 1, policy_version 66330 (0.0010) [2023-10-10 19:10:28,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 135921664. Throughput: 0: 1794.4, 1: 1819.8. Samples: 33986564. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:10:28,789][122664] Avg episode reward: [(0, '78.690'), (1, '66.860')] [2023-10-10 19:10:31,965][123582] Updated weights for policy 0, policy_version 66433 (0.0008) [2023-10-10 19:10:32,331][123582] Updated weights for policy 0, policy_version 66443 (0.0007) [2023-10-10 19:10:32,431][123614] Updated weights for policy 1, policy_version 66340 (0.0009) [2023-10-10 19:10:32,706][123582] Updated weights for policy 0, policy_version 66453 (0.0007) [2023-10-10 19:10:32,799][123614] Updated weights for policy 1, policy_version 66350 (0.0008) [2023-10-10 19:10:33,082][123582] Updated weights for policy 0, policy_version 66463 (0.0007) [2023-10-10 19:10:33,159][123614] Updated weights for policy 1, policy_version 66360 (0.0007) [2023-10-10 19:10:33,788][122664] Fps is (10 sec: 19660.3, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 136019968. Throughput: 0: 1808.1, 1: 1819.2. Samples: 34008126. Policy #0 lag: (min: 28.0, avg: 46.6, max: 48.0) [2023-10-10 19:10:33,789][122664] Avg episode reward: [(0, '78.270'), (1, '73.530')] [2023-10-10 19:10:36,733][123582] Updated weights for policy 0, policy_version 66473 (0.0007) [2023-10-10 19:10:36,834][123614] Updated weights for policy 1, policy_version 66370 (0.0008) [2023-10-10 19:10:37,105][123582] Updated weights for policy 0, policy_version 66483 (0.0008) [2023-10-10 19:10:37,188][123614] Updated weights for policy 1, policy_version 66380 (0.0008) [2023-10-10 19:10:37,464][123582] Updated weights for policy 0, policy_version 66493 (0.0008) [2023-10-10 19:10:37,563][123614] Updated weights for policy 1, policy_version 66390 (0.0007) [2023-10-10 19:10:37,926][123614] Updated weights for policy 1, policy_version 66400 (0.0008) [2023-10-10 19:10:38,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 136085504. Throughput: 0: 1788.3, 1: 1819.4. Samples: 34028838. Policy #0 lag: (min: 28.0, avg: 46.6, max: 48.0) [2023-10-10 19:10:38,790][122664] Avg episode reward: [(0, '82.030'), (1, '72.230')] [2023-10-10 19:10:41,216][123582] Updated weights for policy 0, policy_version 66503 (0.0007) [2023-10-10 19:10:41,593][123582] Updated weights for policy 0, policy_version 66513 (0.0008) [2023-10-10 19:10:41,726][123614] Updated weights for policy 1, policy_version 66410 (0.0007) [2023-10-10 19:10:41,964][123582] Updated weights for policy 0, policy_version 66523 (0.0009) [2023-10-10 19:10:42,099][123614] Updated weights for policy 1, policy_version 66420 (0.0008) [2023-10-10 19:10:42,466][123614] Updated weights for policy 1, policy_version 66430 (0.0007) [2023-10-10 19:10:43,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 136151040. Throughput: 0: 1807.2, 1: 1820.3. Samples: 34040524. Policy #0 lag: (min: 28.0, avg: 46.6, max: 48.0) [2023-10-10 19:10:43,789][122664] Avg episode reward: [(0, '84.600'), (1, '70.960')] [2023-10-10 19:10:45,554][123582] Updated weights for policy 0, policy_version 66533 (0.0009) [2023-10-10 19:10:45,923][123582] Updated weights for policy 0, policy_version 66543 (0.0007) [2023-10-10 19:10:46,086][123614] Updated weights for policy 1, policy_version 66440 (0.0009) [2023-10-10 19:10:46,293][123582] Updated weights for policy 0, policy_version 66553 (0.0007) [2023-10-10 19:10:46,449][123614] Updated weights for policy 1, policy_version 66450 (0.0009) [2023-10-10 19:10:46,818][123614] Updated weights for policy 1, policy_version 66460 (0.0010) [2023-10-10 19:10:48,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 136216576. Throughput: 0: 1793.6, 1: 1810.1. Samples: 34061242. Policy #0 lag: (min: 28.0, avg: 46.6, max: 48.0) [2023-10-10 19:10:48,789][122664] Avg episode reward: [(0, '83.200'), (1, '67.650')] [2023-10-10 19:10:49,904][123582] Updated weights for policy 0, policy_version 66563 (0.0008) [2023-10-10 19:10:50,272][123582] Updated weights for policy 0, policy_version 66573 (0.0008) [2023-10-10 19:10:50,599][123614] Updated weights for policy 1, policy_version 66470 (0.0009) [2023-10-10 19:10:50,646][123582] Updated weights for policy 0, policy_version 66583 (0.0008) [2023-10-10 19:10:50,964][123614] Updated weights for policy 1, policy_version 66480 (0.0009) [2023-10-10 19:10:51,331][123614] Updated weights for policy 1, policy_version 66490 (0.0007) [2023-10-10 19:10:53,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 136282112. Throughput: 0: 1798.0, 1: 1817.0. Samples: 34084108. Policy #0 lag: (min: 28.0, avg: 46.6, max: 48.0) [2023-10-10 19:10:53,789][122664] Avg episode reward: [(0, '81.900'), (1, '66.280')] [2023-10-10 19:10:54,392][123582] Updated weights for policy 0, policy_version 66593 (0.0008) [2023-10-10 19:10:54,757][123582] Updated weights for policy 0, policy_version 66603 (0.0010) [2023-10-10 19:10:55,009][123614] Updated weights for policy 1, policy_version 66500 (0.0007) [2023-10-10 19:10:55,126][123582] Updated weights for policy 0, policy_version 66613 (0.0009) [2023-10-10 19:10:55,371][123614] Updated weights for policy 1, policy_version 66510 (0.0008) [2023-10-10 19:10:55,495][123582] Updated weights for policy 0, policy_version 66623 (0.0009) [2023-10-10 19:10:55,749][123614] Updated weights for policy 1, policy_version 66520 (0.0009) [2023-10-10 19:10:58,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 136347648. Throughput: 0: 1794.1, 1: 1812.2. Samples: 34093738. Policy #0 lag: (min: 28.0, avg: 46.6, max: 48.0) [2023-10-10 19:10:58,788][122664] Avg episode reward: [(0, '83.670'), (1, '64.980')] [2023-10-10 19:10:59,219][123582] Updated weights for policy 0, policy_version 66633 (0.0010) [2023-10-10 19:10:59,494][123614] Updated weights for policy 1, policy_version 66530 (0.0010) [2023-10-10 19:10:59,581][123582] Updated weights for policy 0, policy_version 66643 (0.0009) [2023-10-10 19:10:59,861][123614] Updated weights for policy 1, policy_version 66540 (0.0008) [2023-10-10 19:10:59,953][123582] Updated weights for policy 0, policy_version 66653 (0.0008) [2023-10-10 19:11:00,238][123614] Updated weights for policy 1, policy_version 66550 (0.0010) [2023-10-10 19:11:00,619][123614] Updated weights for policy 1, policy_version 66560 (0.0009) [2023-10-10 19:11:03,721][123582] Updated weights for policy 0, policy_version 66663 (0.0007) [2023-10-10 19:11:03,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 136413184. Throughput: 0: 1797.1, 1: 1806.0. Samples: 34116186. Policy #0 lag: (min: 28.0, avg: 46.6, max: 48.0) [2023-10-10 19:11:03,789][122664] Avg episode reward: [(0, '81.000'), (1, '62.840')] [2023-10-10 19:11:04,085][123582] Updated weights for policy 0, policy_version 66673 (0.0008) [2023-10-10 19:11:04,461][123582] Updated weights for policy 0, policy_version 66683 (0.0008) [2023-10-10 19:11:04,530][123614] Updated weights for policy 1, policy_version 66570 (0.0008) [2023-10-10 19:11:04,906][123614] Updated weights for policy 1, policy_version 66580 (0.0008) [2023-10-10 19:11:05,268][123614] Updated weights for policy 1, policy_version 66590 (0.0009) [2023-10-10 19:11:08,196][123582] Updated weights for policy 0, policy_version 66693 (0.0008) [2023-10-10 19:11:08,567][123582] Updated weights for policy 0, policy_version 66703 (0.0008) [2023-10-10 19:11:08,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 136478720. Throughput: 0: 1805.9, 1: 1809.3. Samples: 34138082. Policy #0 lag: (min: 28.0, avg: 46.6, max: 48.0) [2023-10-10 19:11:08,789][122664] Avg episode reward: [(0, '78.990'), (1, '61.810')] [2023-10-10 19:11:08,896][123614] Updated weights for policy 1, policy_version 66600 (0.0008) [2023-10-10 19:11:08,942][123582] Updated weights for policy 0, policy_version 66713 (0.0008) [2023-10-10 19:11:09,269][123614] Updated weights for policy 1, policy_version 66610 (0.0008) [2023-10-10 19:11:09,626][123614] Updated weights for policy 1, policy_version 66620 (0.0008) [2023-10-10 19:11:12,499][123582] Updated weights for policy 0, policy_version 66723 (0.0008) [2023-10-10 19:11:12,890][123582] Updated weights for policy 0, policy_version 66733 (0.0007) [2023-10-10 19:11:13,260][123582] Updated weights for policy 0, policy_version 66743 (0.0007) [2023-10-10 19:11:13,363][123614] Updated weights for policy 1, policy_version 66630 (0.0007) [2023-10-10 19:11:13,721][123614] Updated weights for policy 1, policy_version 66640 (0.0008) [2023-10-10 19:11:13,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 136577024. Throughput: 0: 1805.3, 1: 1802.2. Samples: 34148900. Policy #0 lag: (min: 28.0, avg: 46.6, max: 48.0) [2023-10-10 19:11:13,789][122664] Avg episode reward: [(0, '79.350'), (1, '61.160')] [2023-10-10 19:11:14,087][123614] Updated weights for policy 1, policy_version 66650 (0.0010) [2023-10-10 19:11:17,141][123582] Updated weights for policy 0, policy_version 66753 (0.0008) [2023-10-10 19:11:17,511][123582] Updated weights for policy 0, policy_version 66763 (0.0009) [2023-10-10 19:11:17,665][123614] Updated weights for policy 1, policy_version 66660 (0.0007) [2023-10-10 19:11:17,880][123582] Updated weights for policy 0, policy_version 66773 (0.0009) [2023-10-10 19:11:18,043][123614] Updated weights for policy 1, policy_version 66670 (0.0008) [2023-10-10 19:11:18,248][123582] Updated weights for policy 0, policy_version 66783 (0.0008) [2023-10-10 19:11:18,405][123614] Updated weights for policy 1, policy_version 66680 (0.0008) [2023-10-10 19:11:18,788][122664] Fps is (10 sec: 19660.9, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 136675328. Throughput: 0: 1810.2, 1: 1810.5. Samples: 34171056. Policy #0 lag: (min: 29.0, avg: 31.2, max: 60.0) [2023-10-10 19:11:18,788][122664] Avg episode reward: [(0, '78.750'), (1, '62.760')] [2023-10-10 19:11:22,033][123582] Updated weights for policy 0, policy_version 66793 (0.0008) [2023-10-10 19:11:22,100][123614] Updated weights for policy 1, policy_version 66690 (0.0008) [2023-10-10 19:11:22,400][123582] Updated weights for policy 0, policy_version 66803 (0.0008) [2023-10-10 19:11:22,464][123614] Updated weights for policy 1, policy_version 66700 (0.0007) [2023-10-10 19:11:22,762][123582] Updated weights for policy 0, policy_version 66813 (0.0008) [2023-10-10 19:11:22,829][123614] Updated weights for policy 1, policy_version 66710 (0.0008) [2023-10-10 19:11:23,204][123614] Updated weights for policy 1, policy_version 66720 (0.0007) [2023-10-10 19:11:23,788][122664] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 136740864. Throughput: 0: 1809.6, 1: 1802.4. Samples: 34191374. Policy #0 lag: (min: 29.0, avg: 31.2, max: 60.0) [2023-10-10 19:11:23,789][122664] Avg episode reward: [(0, '79.710'), (1, '65.200')] [2023-10-10 19:11:26,562][123582] Updated weights for policy 0, policy_version 66823 (0.0008) [2023-10-10 19:11:26,932][123582] Updated weights for policy 0, policy_version 66833 (0.0007) [2023-10-10 19:11:26,940][123614] Updated weights for policy 1, policy_version 66730 (0.0008) [2023-10-10 19:11:27,306][123582] Updated weights for policy 0, policy_version 66843 (0.0007) [2023-10-10 19:11:27,314][123614] Updated weights for policy 1, policy_version 66740 (0.0009) [2023-10-10 19:11:27,681][123614] Updated weights for policy 1, policy_version 66750 (0.0009) [2023-10-10 19:11:28,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 136806400. Throughput: 0: 1817.5, 1: 1811.0. Samples: 34203806. Policy #0 lag: (min: 29.0, avg: 31.2, max: 60.0) [2023-10-10 19:11:28,789][122664] Avg episode reward: [(0, '72.730'), (1, '69.390')] [2023-10-10 19:11:31,081][123582] Updated weights for policy 0, policy_version 66853 (0.0010) [2023-10-10 19:11:31,451][123582] Updated weights for policy 0, policy_version 66863 (0.0008) [2023-10-10 19:11:31,529][123614] Updated weights for policy 1, policy_version 66760 (0.0007) [2023-10-10 19:11:31,829][123582] Updated weights for policy 0, policy_version 66873 (0.0009) [2023-10-10 19:11:31,900][123614] Updated weights for policy 1, policy_version 66770 (0.0007) [2023-10-10 19:11:32,273][123614] Updated weights for policy 1, policy_version 66780 (0.0007) [2023-10-10 19:11:33,788][122664] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 136871936. Throughput: 0: 1801.2, 1: 1800.9. Samples: 34223334. Policy #0 lag: (min: 29.0, avg: 31.2, max: 60.0) [2023-10-10 19:11:33,788][122664] Avg episode reward: [(0, '70.500'), (1, '69.450')] [2023-10-10 19:11:35,536][123582] Updated weights for policy 0, policy_version 66883 (0.0008) [2023-10-10 19:11:35,908][123582] Updated weights for policy 0, policy_version 66893 (0.0008) [2023-10-10 19:11:36,062][123614] Updated weights for policy 1, policy_version 66790 (0.0008) [2023-10-10 19:11:36,277][123582] Updated weights for policy 0, policy_version 66903 (0.0009) [2023-10-10 19:11:36,431][123614] Updated weights for policy 1, policy_version 66800 (0.0008) [2023-10-10 19:11:36,797][123614] Updated weights for policy 1, policy_version 66810 (0.0009) [2023-10-10 19:11:38,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 136937472. Throughput: 0: 1794.7, 1: 1804.6. Samples: 34246078. Policy #0 lag: (min: 29.0, avg: 31.2, max: 60.0) [2023-10-10 19:11:38,789][122664] Avg episode reward: [(0, '69.840'), (1, '65.220')] [2023-10-10 19:11:40,133][123582] Updated weights for policy 0, policy_version 66913 (0.0009) [2023-10-10 19:11:40,498][123614] Updated weights for policy 1, policy_version 66820 (0.0008) [2023-10-10 19:11:40,504][123582] Updated weights for policy 0, policy_version 66923 (0.0007) [2023-10-10 19:11:40,872][123614] Updated weights for policy 1, policy_version 66830 (0.0007) [2023-10-10 19:11:40,874][123582] Updated weights for policy 0, policy_version 66933 (0.0010) [2023-10-10 19:11:41,244][123582] Updated weights for policy 0, policy_version 66943 (0.0009) [2023-10-10 19:11:41,249][123614] Updated weights for policy 1, policy_version 66840 (0.0007) [2023-10-10 19:11:43,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 137003008. Throughput: 0: 1795.2, 1: 1808.8. Samples: 34255916. Policy #0 lag: (min: 29.0, avg: 31.2, max: 60.0) [2023-10-10 19:11:43,789][122664] Avg episode reward: [(0, '72.830'), (1, '63.510')] [2023-10-10 19:11:44,827][123614] Updated weights for policy 1, policy_version 66850 (0.0007) [2023-10-10 19:11:44,933][123582] Updated weights for policy 0, policy_version 66953 (0.0009) [2023-10-10 19:11:45,187][123614] Updated weights for policy 1, policy_version 66860 (0.0007) [2023-10-10 19:11:45,297][123582] Updated weights for policy 0, policy_version 66963 (0.0008) [2023-10-10 19:11:45,561][123614] Updated weights for policy 1, policy_version 66870 (0.0009) [2023-10-10 19:11:45,661][123582] Updated weights for policy 0, policy_version 66973 (0.0008) [2023-10-10 19:11:45,923][123614] Updated weights for policy 1, policy_version 66880 (0.0008) [2023-10-10 19:11:48,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 137068544. Throughput: 0: 1795.0, 1: 1818.9. Samples: 34278810. Policy #0 lag: (min: 29.0, avg: 31.2, max: 60.0) [2023-10-10 19:11:48,788][122664] Avg episode reward: [(0, '74.280'), (1, '63.860')] [2023-10-10 19:11:49,508][123582] Updated weights for policy 0, policy_version 66983 (0.0008) [2023-10-10 19:11:49,626][123614] Updated weights for policy 1, policy_version 66890 (0.0008) [2023-10-10 19:11:49,874][123582] Updated weights for policy 0, policy_version 66993 (0.0007) [2023-10-10 19:11:49,990][123614] Updated weights for policy 1, policy_version 66900 (0.0007) [2023-10-10 19:11:50,251][123582] Updated weights for policy 0, policy_version 67003 (0.0007) [2023-10-10 19:11:50,352][123614] Updated weights for policy 1, policy_version 66910 (0.0008) [2023-10-10 19:11:53,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 137134080. Throughput: 0: 1800.1, 1: 1827.6. Samples: 34301328. Policy #0 lag: (min: 29.0, avg: 31.2, max: 60.0) [2023-10-10 19:11:53,789][122664] Avg episode reward: [(0, '82.620'), (1, '62.390')] [2023-10-10 19:11:53,932][123614] Updated weights for policy 1, policy_version 66920 (0.0007) [2023-10-10 19:11:54,003][123582] Updated weights for policy 0, policy_version 67013 (0.0009) [2023-10-10 19:11:54,300][123614] Updated weights for policy 1, policy_version 66930 (0.0008) [2023-10-10 19:11:54,380][123582] Updated weights for policy 0, policy_version 67023 (0.0009) [2023-10-10 19:11:54,659][123614] Updated weights for policy 1, policy_version 66940 (0.0008) [2023-10-10 19:11:54,750][123582] Updated weights for policy 0, policy_version 67033 (0.0007) [2023-10-10 19:11:54,804][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000066944_68550656.pth... [2023-10-10 19:11:54,836][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000065216_66781184.pth [2023-10-10 19:11:55,013][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000067040_68648960.pth... [2023-10-10 19:11:55,053][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000065344_66912256.pth [2023-10-10 19:11:58,325][123614] Updated weights for policy 1, policy_version 66950 (0.0007) [2023-10-10 19:11:58,491][123582] Updated weights for policy 0, policy_version 67043 (0.0007) [2023-10-10 19:11:58,686][123614] Updated weights for policy 1, policy_version 66960 (0.0008) [2023-10-10 19:11:58,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 137199616. Throughput: 0: 1782.2, 1: 1824.4. Samples: 34311200. Policy #0 lag: (min: 29.0, avg: 31.2, max: 60.0) [2023-10-10 19:11:58,788][122664] Avg episode reward: [(0, '82.650'), (1, '63.520')] [2023-10-10 19:11:58,870][123582] Updated weights for policy 0, policy_version 67053 (0.0009) [2023-10-10 19:11:59,049][123614] Updated weights for policy 1, policy_version 66970 (0.0007) [2023-10-10 19:11:59,252][123582] Updated weights for policy 0, policy_version 67063 (0.0010) [2023-10-10 19:12:02,758][123614] Updated weights for policy 1, policy_version 66980 (0.0009) [2023-10-10 19:12:02,943][123582] Updated weights for policy 0, policy_version 67073 (0.0011) [2023-10-10 19:12:03,118][123614] Updated weights for policy 1, policy_version 66990 (0.0007) [2023-10-10 19:12:03,313][123582] Updated weights for policy 0, policy_version 67083 (0.0008) [2023-10-10 19:12:03,492][123614] Updated weights for policy 1, policy_version 67000 (0.0007) [2023-10-10 19:12:03,688][123582] Updated weights for policy 0, policy_version 67093 (0.0007) [2023-10-10 19:12:03,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 137297920. Throughput: 0: 1799.3, 1: 1825.3. Samples: 34334164. Policy #0 lag: (min: 29.0, avg: 31.2, max: 60.0) [2023-10-10 19:12:03,789][122664] Avg episode reward: [(0, '80.300'), (1, '62.350')] [2023-10-10 19:12:04,066][123582] Updated weights for policy 0, policy_version 67103 (0.0010) [2023-10-10 19:12:07,173][123614] Updated weights for policy 1, policy_version 67010 (0.0007) [2023-10-10 19:12:07,549][123614] Updated weights for policy 1, policy_version 67020 (0.0009) [2023-10-10 19:12:07,740][123582] Updated weights for policy 0, policy_version 67113 (0.0008) [2023-10-10 19:12:07,906][123614] Updated weights for policy 1, policy_version 67030 (0.0009) [2023-10-10 19:12:08,112][123582] Updated weights for policy 0, policy_version 67123 (0.0008) [2023-10-10 19:12:08,281][123614] Updated weights for policy 1, policy_version 67040 (0.0008) [2023-10-10 19:12:08,478][123582] Updated weights for policy 0, policy_version 67133 (0.0008) [2023-10-10 19:12:08,788][122664] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 137396224. Throughput: 0: 1797.2, 1: 1827.5. Samples: 34354488. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 19:12:08,789][122664] Avg episode reward: [(0, '81.970'), (1, '61.390')] [2023-10-10 19:12:12,046][123614] Updated weights for policy 1, policy_version 67050 (0.0008) [2023-10-10 19:12:12,095][123582] Updated weights for policy 0, policy_version 67143 (0.0008) [2023-10-10 19:12:12,413][123614] Updated weights for policy 1, policy_version 67060 (0.0008) [2023-10-10 19:12:12,465][123582] Updated weights for policy 0, policy_version 67153 (0.0008) [2023-10-10 19:12:12,784][123614] Updated weights for policy 1, policy_version 67070 (0.0008) [2023-10-10 19:12:12,831][123582] Updated weights for policy 0, policy_version 67163 (0.0008) [2023-10-10 19:12:13,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 137461760. Throughput: 0: 1797.6, 1: 1828.2. Samples: 34366964. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 19:12:13,788][122664] Avg episode reward: [(0, '84.100'), (1, '67.080')] [2023-10-10 19:12:16,474][123582] Updated weights for policy 0, policy_version 67173 (0.0009) [2023-10-10 19:12:16,507][123614] Updated weights for policy 1, policy_version 67080 (0.0010) [2023-10-10 19:12:16,853][123582] Updated weights for policy 0, policy_version 67183 (0.0008) [2023-10-10 19:12:16,881][123614] Updated weights for policy 1, policy_version 67090 (0.0008) [2023-10-10 19:12:17,216][123582] Updated weights for policy 0, policy_version 67193 (0.0008) [2023-10-10 19:12:17,246][123614] Updated weights for policy 1, policy_version 67100 (0.0008) [2023-10-10 19:12:18,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 137527296. Throughput: 0: 1803.7, 1: 1827.4. Samples: 34386734. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 19:12:18,790][122664] Avg episode reward: [(0, '81.310'), (1, '68.730')] [2023-10-10 19:12:20,880][123582] Updated weights for policy 0, policy_version 67203 (0.0008) [2023-10-10 19:12:21,082][123614] Updated weights for policy 1, policy_version 67110 (0.0008) [2023-10-10 19:12:21,253][123582] Updated weights for policy 0, policy_version 67213 (0.0007) [2023-10-10 19:12:21,458][123614] Updated weights for policy 1, policy_version 67120 (0.0007) [2023-10-10 19:12:21,628][123582] Updated weights for policy 0, policy_version 67223 (0.0007) [2023-10-10 19:12:21,830][123614] Updated weights for policy 1, policy_version 67130 (0.0009) [2023-10-10 19:12:23,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 137592832. Throughput: 0: 1802.4, 1: 1823.2. Samples: 34409228. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 19:12:23,789][122664] Avg episode reward: [(0, '82.530'), (1, '72.190')] [2023-10-10 19:12:25,400][123582] Updated weights for policy 0, policy_version 67233 (0.0008) [2023-10-10 19:12:25,498][123614] Updated weights for policy 1, policy_version 67140 (0.0009) [2023-10-10 19:12:25,769][123582] Updated weights for policy 0, policy_version 67243 (0.0009) [2023-10-10 19:12:25,860][123614] Updated weights for policy 1, policy_version 67150 (0.0007) [2023-10-10 19:12:26,131][123582] Updated weights for policy 0, policy_version 67253 (0.0007) [2023-10-10 19:12:26,233][123614] Updated weights for policy 1, policy_version 67160 (0.0010) [2023-10-10 19:12:26,497][123582] Updated weights for policy 0, policy_version 67263 (0.0007) [2023-10-10 19:12:28,788][122664] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 137658368. Throughput: 0: 1806.9, 1: 1824.5. Samples: 34419332. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 19:12:28,788][122664] Avg episode reward: [(0, '81.510'), (1, '69.670')] [2023-10-10 19:12:30,071][123582] Updated weights for policy 0, policy_version 67273 (0.0008) [2023-10-10 19:12:30,098][123614] Updated weights for policy 1, policy_version 67170 (0.0007) [2023-10-10 19:12:30,440][123582] Updated weights for policy 0, policy_version 67283 (0.0009) [2023-10-10 19:12:30,464][123614] Updated weights for policy 1, policy_version 67180 (0.0007) [2023-10-10 19:12:30,816][123582] Updated weights for policy 0, policy_version 67293 (0.0009) [2023-10-10 19:12:30,827][123614] Updated weights for policy 1, policy_version 67190 (0.0007) [2023-10-10 19:12:31,200][123614] Updated weights for policy 1, policy_version 67200 (0.0009) [2023-10-10 19:12:33,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 137723904. Throughput: 0: 1805.8, 1: 1811.5. Samples: 34441588. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 19:12:33,788][122664] Avg episode reward: [(0, '83.520'), (1, '66.130')] [2023-10-10 19:12:34,713][123582] Updated weights for policy 0, policy_version 67303 (0.0008) [2023-10-10 19:12:34,972][123614] Updated weights for policy 1, policy_version 67210 (0.0007) [2023-10-10 19:12:35,093][123582] Updated weights for policy 0, policy_version 67313 (0.0008) [2023-10-10 19:12:35,338][123614] Updated weights for policy 1, policy_version 67220 (0.0008) [2023-10-10 19:12:35,462][123582] Updated weights for policy 0, policy_version 67323 (0.0008) [2023-10-10 19:12:35,704][123614] Updated weights for policy 1, policy_version 67230 (0.0009) [2023-10-10 19:12:38,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 137789440. Throughput: 0: 1813.7, 1: 1806.4. Samples: 34464234. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 19:12:38,788][122664] Avg episode reward: [(0, '83.140'), (1, '65.610')] [2023-10-10 19:12:39,029][123582] Updated weights for policy 0, policy_version 67333 (0.0010) [2023-10-10 19:12:39,230][123614] Updated weights for policy 1, policy_version 67240 (0.0009) [2023-10-10 19:12:39,399][123582] Updated weights for policy 0, policy_version 67343 (0.0008) [2023-10-10 19:12:39,590][123614] Updated weights for policy 1, policy_version 67250 (0.0007) [2023-10-10 19:12:39,767][123582] Updated weights for policy 0, policy_version 67353 (0.0008) [2023-10-10 19:12:39,948][123614] Updated weights for policy 1, policy_version 67260 (0.0009) [2023-10-10 19:12:43,523][123582] Updated weights for policy 0, policy_version 67363 (0.0008) [2023-10-10 19:12:43,633][123614] Updated weights for policy 1, policy_version 67270 (0.0007) [2023-10-10 19:12:43,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 137854976. Throughput: 0: 1813.0, 1: 1802.6. Samples: 34473900. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 19:12:43,788][122664] Avg episode reward: [(0, '85.060'), (1, '63.490')] [2023-10-10 19:12:43,904][123582] Updated weights for policy 0, policy_version 67373 (0.0008) [2023-10-10 19:12:44,004][123614] Updated weights for policy 1, policy_version 67280 (0.0008) [2023-10-10 19:12:44,272][123582] Updated weights for policy 0, policy_version 67383 (0.0008) [2023-10-10 19:12:44,369][123614] Updated weights for policy 1, policy_version 67290 (0.0009) [2023-10-10 19:12:48,058][123582] Updated weights for policy 0, policy_version 67393 (0.0008) [2023-10-10 19:12:48,214][123614] Updated weights for policy 1, policy_version 67300 (0.0009) [2023-10-10 19:12:48,430][123582] Updated weights for policy 0, policy_version 67403 (0.0007) [2023-10-10 19:12:48,589][123614] Updated weights for policy 1, policy_version 67310 (0.0007) [2023-10-10 19:12:48,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 137920512. Throughput: 0: 1804.7, 1: 1799.9. Samples: 34496372. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 19:12:48,789][122664] Avg episode reward: [(0, '83.460'), (1, '63.020')] [2023-10-10 19:12:48,791][123582] Updated weights for policy 0, policy_version 67413 (0.0010) [2023-10-10 19:12:48,960][123614] Updated weights for policy 1, policy_version 67320 (0.0009) [2023-10-10 19:12:49,163][123582] Updated weights for policy 0, policy_version 67423 (0.0007) [2023-10-10 19:12:52,718][123614] Updated weights for policy 1, policy_version 67330 (0.0008) [2023-10-10 19:12:52,938][123582] Updated weights for policy 0, policy_version 67433 (0.0008) [2023-10-10 19:12:53,086][123614] Updated weights for policy 1, policy_version 67340 (0.0008) [2023-10-10 19:12:53,308][123582] Updated weights for policy 0, policy_version 67443 (0.0008) [2023-10-10 19:12:53,452][123614] Updated weights for policy 1, policy_version 67350 (0.0009) [2023-10-10 19:12:53,677][123582] Updated weights for policy 0, policy_version 67453 (0.0009) [2023-10-10 19:12:53,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 138018816. Throughput: 0: 1803.8, 1: 1794.3. Samples: 34516402. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:12:53,789][122664] Avg episode reward: [(0, '83.890'), (1, '65.660')] [2023-10-10 19:12:53,812][123614] Updated weights for policy 1, policy_version 67360 (0.0008) [2023-10-10 19:12:57,387][123582] Updated weights for policy 0, policy_version 67463 (0.0009) [2023-10-10 19:12:57,562][123614] Updated weights for policy 1, policy_version 67370 (0.0008) [2023-10-10 19:12:57,755][123582] Updated weights for policy 0, policy_version 67473 (0.0007) [2023-10-10 19:12:57,931][123614] Updated weights for policy 1, policy_version 67380 (0.0008) [2023-10-10 19:12:58,138][123582] Updated weights for policy 0, policy_version 67483 (0.0009) [2023-10-10 19:12:58,308][123614] Updated weights for policy 1, policy_version 67390 (0.0008) [2023-10-10 19:12:58,788][122664] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 138117120. Throughput: 0: 1801.0, 1: 1789.3. Samples: 34528526. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:12:58,789][122664] Avg episode reward: [(0, '79.320'), (1, '70.110')] [2023-10-10 19:13:01,975][123582] Updated weights for policy 0, policy_version 67493 (0.0009) [2023-10-10 19:13:02,128][123614] Updated weights for policy 1, policy_version 67400 (0.0007) [2023-10-10 19:13:02,347][123582] Updated weights for policy 0, policy_version 67503 (0.0009) [2023-10-10 19:13:02,497][123614] Updated weights for policy 1, policy_version 67410 (0.0008) [2023-10-10 19:13:02,722][123582] Updated weights for policy 0, policy_version 67513 (0.0008) [2023-10-10 19:13:02,879][123614] Updated weights for policy 1, policy_version 67420 (0.0009) [2023-10-10 19:13:03,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 138182656. Throughput: 0: 1808.9, 1: 1798.9. Samples: 34549088. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:13:03,789][122664] Avg episode reward: [(0, '73.880'), (1, '69.010')] [2023-10-10 19:13:06,476][123614] Updated weights for policy 1, policy_version 67430 (0.0009) [2023-10-10 19:13:06,493][123582] Updated weights for policy 0, policy_version 67523 (0.0007) [2023-10-10 19:13:06,852][123614] Updated weights for policy 1, policy_version 67440 (0.0008) [2023-10-10 19:13:06,864][123582] Updated weights for policy 0, policy_version 67533 (0.0009) [2023-10-10 19:13:07,210][123614] Updated weights for policy 1, policy_version 67450 (0.0009) [2023-10-10 19:13:07,245][123582] Updated weights for policy 0, policy_version 67543 (0.0010) [2023-10-10 19:13:08,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 138248192. Throughput: 0: 1791.6, 1: 1793.1. Samples: 34570540. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:13:08,788][122664] Avg episode reward: [(0, '71.770'), (1, '68.660')] [2023-10-10 19:13:10,936][123582] Updated weights for policy 0, policy_version 67553 (0.0009) [2023-10-10 19:13:11,131][123614] Updated weights for policy 1, policy_version 67460 (0.0008) [2023-10-10 19:13:11,306][123582] Updated weights for policy 0, policy_version 67563 (0.0007) [2023-10-10 19:13:11,505][123614] Updated weights for policy 1, policy_version 67470 (0.0007) [2023-10-10 19:13:11,690][123582] Updated weights for policy 0, policy_version 67573 (0.0007) [2023-10-10 19:13:11,874][123614] Updated weights for policy 1, policy_version 67480 (0.0008) [2023-10-10 19:13:12,062][123582] Updated weights for policy 0, policy_version 67583 (0.0007) [2023-10-10 19:13:13,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 138313728. Throughput: 0: 1806.9, 1: 1802.8. Samples: 34581770. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:13:13,789][122664] Avg episode reward: [(0, '71.730'), (1, '72.270')] [2023-10-10 19:13:15,635][123614] Updated weights for policy 1, policy_version 67490 (0.0008) [2023-10-10 19:13:15,705][123582] Updated weights for policy 0, policy_version 67593 (0.0008) [2023-10-10 19:13:16,004][123614] Updated weights for policy 1, policy_version 67500 (0.0007) [2023-10-10 19:13:16,069][123582] Updated weights for policy 0, policy_version 67603 (0.0007) [2023-10-10 19:13:16,374][123614] Updated weights for policy 1, policy_version 67510 (0.0007) [2023-10-10 19:13:16,436][123582] Updated weights for policy 0, policy_version 67613 (0.0007) [2023-10-10 19:13:16,742][123614] Updated weights for policy 1, policy_version 67520 (0.0008) [2023-10-10 19:13:18,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 138379264. Throughput: 0: 1792.2, 1: 1793.3. Samples: 34602934. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:13:18,789][122664] Avg episode reward: [(0, '72.830'), (1, '71.450')] [2023-10-10 19:13:20,177][123582] Updated weights for policy 0, policy_version 67623 (0.0009) [2023-10-10 19:13:20,540][123614] Updated weights for policy 1, policy_version 67530 (0.0009) [2023-10-10 19:13:20,541][123582] Updated weights for policy 0, policy_version 67633 (0.0007) [2023-10-10 19:13:20,904][123614] Updated weights for policy 1, policy_version 67540 (0.0008) [2023-10-10 19:13:20,913][123582] Updated weights for policy 0, policy_version 67643 (0.0009) [2023-10-10 19:13:21,272][123614] Updated weights for policy 1, policy_version 67550 (0.0009) [2023-10-10 19:13:23,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 138444800. Throughput: 0: 1792.5, 1: 1794.6. Samples: 34625656. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:13:23,788][122664] Avg episode reward: [(0, '70.530'), (1, '71.800')] [2023-10-10 19:13:24,593][123582] Updated weights for policy 0, policy_version 67653 (0.0009) [2023-10-10 19:13:24,908][123614] Updated weights for policy 1, policy_version 67560 (0.0007) [2023-10-10 19:13:24,951][123582] Updated weights for policy 0, policy_version 67663 (0.0009) [2023-10-10 19:13:25,282][123614] Updated weights for policy 1, policy_version 67570 (0.0007) [2023-10-10 19:13:25,332][123582] Updated weights for policy 0, policy_version 67673 (0.0008) [2023-10-10 19:13:25,658][123614] Updated weights for policy 1, policy_version 67580 (0.0009) [2023-10-10 19:13:28,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 138510336. Throughput: 0: 1794.0, 1: 1796.4. Samples: 34635470. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:13:28,789][122664] Avg episode reward: [(0, '71.490'), (1, '71.190')] [2023-10-10 19:13:29,154][123582] Updated weights for policy 0, policy_version 67683 (0.0007) [2023-10-10 19:13:29,320][123614] Updated weights for policy 1, policy_version 67590 (0.0009) [2023-10-10 19:13:29,534][123582] Updated weights for policy 0, policy_version 67693 (0.0009) [2023-10-10 19:13:29,683][123614] Updated weights for policy 1, policy_version 67600 (0.0008) [2023-10-10 19:13:29,901][123582] Updated weights for policy 0, policy_version 67703 (0.0009) [2023-10-10 19:13:30,045][123614] Updated weights for policy 1, policy_version 67610 (0.0007) [2023-10-10 19:13:33,562][123582] Updated weights for policy 0, policy_version 67713 (0.0009) [2023-10-10 19:13:33,698][123614] Updated weights for policy 1, policy_version 67620 (0.0007) [2023-10-10 19:13:33,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 138575872. Throughput: 0: 1796.4, 1: 1798.9. Samples: 34658160. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:13:33,789][122664] Avg episode reward: [(0, '72.210'), (1, '67.100')] [2023-10-10 19:13:33,932][123582] Updated weights for policy 0, policy_version 67723 (0.0008) [2023-10-10 19:13:34,060][123614] Updated weights for policy 1, policy_version 67630 (0.0007) [2023-10-10 19:13:34,304][123582] Updated weights for policy 0, policy_version 67733 (0.0009) [2023-10-10 19:13:34,422][123614] Updated weights for policy 1, policy_version 67640 (0.0009) [2023-10-10 19:13:34,684][123582] Updated weights for policy 0, policy_version 67743 (0.0008) [2023-10-10 19:13:38,052][123614] Updated weights for policy 1, policy_version 67650 (0.0009) [2023-10-10 19:13:38,376][123582] Updated weights for policy 0, policy_version 67753 (0.0007) [2023-10-10 19:13:38,419][123614] Updated weights for policy 1, policy_version 67660 (0.0009) [2023-10-10 19:13:38,758][123582] Updated weights for policy 0, policy_version 67763 (0.0008) [2023-10-10 19:13:38,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 138641408. Throughput: 0: 1812.8, 1: 1815.5. Samples: 34679674. Policy #0 lag: (min: 3.0, avg: 12.2, max: 35.0) [2023-10-10 19:13:38,788][122664] Avg episode reward: [(0, '70.320'), (1, '68.400')] [2023-10-10 19:13:38,794][123614] Updated weights for policy 1, policy_version 67670 (0.0008) [2023-10-10 19:13:39,121][123582] Updated weights for policy 0, policy_version 67773 (0.0007) [2023-10-10 19:13:39,156][123614] Updated weights for policy 1, policy_version 67680 (0.0008) [2023-10-10 19:13:42,928][123614] Updated weights for policy 1, policy_version 67690 (0.0009) [2023-10-10 19:13:42,977][123582] Updated weights for policy 0, policy_version 67783 (0.0010) [2023-10-10 19:13:43,300][123614] Updated weights for policy 1, policy_version 67700 (0.0009) [2023-10-10 19:13:43,351][123582] Updated weights for policy 0, policy_version 67793 (0.0011) [2023-10-10 19:13:43,665][123614] Updated weights for policy 1, policy_version 67710 (0.0007) [2023-10-10 19:13:43,722][123582] Updated weights for policy 0, policy_version 67803 (0.0008) [2023-10-10 19:13:43,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 138739712. Throughput: 0: 1796.7, 1: 1810.1. Samples: 34690836. Policy #0 lag: (min: 3.0, avg: 12.2, max: 35.0) [2023-10-10 19:13:43,789][122664] Avg episode reward: [(0, '69.190'), (1, '68.870')] [2023-10-10 19:13:47,355][123614] Updated weights for policy 1, policy_version 67720 (0.0008) [2023-10-10 19:13:47,533][123582] Updated weights for policy 0, policy_version 67813 (0.0008) [2023-10-10 19:13:47,724][123614] Updated weights for policy 1, policy_version 67730 (0.0009) [2023-10-10 19:13:47,907][123582] Updated weights for policy 0, policy_version 67823 (0.0009) [2023-10-10 19:13:48,094][123614] Updated weights for policy 1, policy_version 67740 (0.0009) [2023-10-10 19:13:48,270][123582] Updated weights for policy 0, policy_version 67833 (0.0008) [2023-10-10 19:13:48,788][122664] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 138838016. Throughput: 0: 1813.2, 1: 1816.5. Samples: 34712426. Policy #0 lag: (min: 3.0, avg: 12.2, max: 35.0) [2023-10-10 19:13:48,789][122664] Avg episode reward: [(0, '68.850'), (1, '72.390')] [2023-10-10 19:13:51,685][123614] Updated weights for policy 1, policy_version 67750 (0.0008) [2023-10-10 19:13:52,049][123582] Updated weights for policy 0, policy_version 67843 (0.0008) [2023-10-10 19:13:52,060][123614] Updated weights for policy 1, policy_version 67760 (0.0008) [2023-10-10 19:13:52,418][123614] Updated weights for policy 1, policy_version 67770 (0.0007) [2023-10-10 19:13:52,420][123582] Updated weights for policy 0, policy_version 67853 (0.0007) [2023-10-10 19:13:52,784][123582] Updated weights for policy 0, policy_version 67863 (0.0009) [2023-10-10 19:13:53,788][122664] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 138903552. Throughput: 0: 1797.0, 1: 1811.3. Samples: 34732914. Policy #0 lag: (min: 3.0, avg: 12.2, max: 35.0) [2023-10-10 19:13:53,788][122664] Avg episode reward: [(0, '71.120'), (1, '70.440')] [2023-10-10 19:13:53,796][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000067776_69402624.pth... [2023-10-10 19:13:53,796][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000067872_69500928.pth... [2023-10-10 19:13:53,830][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000066080_67665920.pth [2023-10-10 19:13:53,834][123465] Saving a milestone ./train_atari/atari_demonattack_APPO/checkpoint_p1/milestones/checkpoint_000067776_69402624.pth [2023-10-10 19:13:53,834][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000066176_67764224.pth [2023-10-10 19:13:53,840][123247] Saving a milestone ./train_atari/atari_demonattack_APPO/checkpoint_p0/milestones/checkpoint_000067872_69500928.pth [2023-10-10 19:13:56,248][123614] Updated weights for policy 1, policy_version 67780 (0.0008) [2023-10-10 19:13:56,458][123582] Updated weights for policy 0, policy_version 67873 (0.0009) [2023-10-10 19:13:56,618][123614] Updated weights for policy 1, policy_version 67790 (0.0007) [2023-10-10 19:13:56,830][123582] Updated weights for policy 0, policy_version 67883 (0.0008) [2023-10-10 19:13:56,982][123614] Updated weights for policy 1, policy_version 67800 (0.0008) [2023-10-10 19:13:57,198][123582] Updated weights for policy 0, policy_version 67893 (0.0008) [2023-10-10 19:13:57,568][123582] Updated weights for policy 0, policy_version 67903 (0.0008) [2023-10-10 19:13:58,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 138969088. Throughput: 0: 1819.4, 1: 1813.2. Samples: 34745236. Policy #0 lag: (min: 3.0, avg: 12.2, max: 35.0) [2023-10-10 19:13:58,789][122664] Avg episode reward: [(0, '71.460'), (1, '71.870')] [2023-10-10 19:14:00,653][123614] Updated weights for policy 1, policy_version 67810 (0.0008) [2023-10-10 19:14:01,023][123614] Updated weights for policy 1, policy_version 67820 (0.0008) [2023-10-10 19:14:01,361][123582] Updated weights for policy 0, policy_version 67913 (0.0009) [2023-10-10 19:14:01,388][123614] Updated weights for policy 1, policy_version 67830 (0.0007) [2023-10-10 19:14:01,724][123582] Updated weights for policy 0, policy_version 67923 (0.0009) [2023-10-10 19:14:01,749][123614] Updated weights for policy 1, policy_version 67840 (0.0007) [2023-10-10 19:14:02,092][123582] Updated weights for policy 0, policy_version 67933 (0.0010) [2023-10-10 19:14:03,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 139034624. Throughput: 0: 1800.4, 1: 1813.9. Samples: 34765578. Policy #0 lag: (min: 3.0, avg: 12.2, max: 35.0) [2023-10-10 19:14:03,789][122664] Avg episode reward: [(0, '73.500'), (1, '67.930')] [2023-10-10 19:14:05,555][123614] Updated weights for policy 1, policy_version 67850 (0.0010) [2023-10-10 19:14:05,906][123582] Updated weights for policy 0, policy_version 67943 (0.0008) [2023-10-10 19:14:05,929][123614] Updated weights for policy 1, policy_version 67860 (0.0007) [2023-10-10 19:14:06,264][123582] Updated weights for policy 0, policy_version 67953 (0.0008) [2023-10-10 19:14:06,292][123614] Updated weights for policy 1, policy_version 67870 (0.0008) [2023-10-10 19:14:06,633][123582] Updated weights for policy 0, policy_version 67963 (0.0009) [2023-10-10 19:14:08,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 139100160. Throughput: 0: 1795.5, 1: 1808.1. Samples: 34787818. Policy #0 lag: (min: 3.0, avg: 12.2, max: 35.0) [2023-10-10 19:14:08,789][122664] Avg episode reward: [(0, '71.840'), (1, '71.180')] [2023-10-10 19:14:10,066][123614] Updated weights for policy 1, policy_version 67880 (0.0010) [2023-10-10 19:14:10,295][123582] Updated weights for policy 0, policy_version 67973 (0.0008) [2023-10-10 19:14:10,429][123614] Updated weights for policy 1, policy_version 67890 (0.0007) [2023-10-10 19:14:10,655][123582] Updated weights for policy 0, policy_version 67983 (0.0008) [2023-10-10 19:14:10,797][123614] Updated weights for policy 1, policy_version 67900 (0.0007) [2023-10-10 19:14:11,031][123582] Updated weights for policy 0, policy_version 67993 (0.0010) [2023-10-10 19:14:13,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 139165696. Throughput: 0: 1794.0, 1: 1809.9. Samples: 34797644. Policy #0 lag: (min: 3.0, avg: 12.2, max: 35.0) [2023-10-10 19:14:13,788][122664] Avg episode reward: [(0, '73.130'), (1, '72.250')] [2023-10-10 19:14:14,532][123614] Updated weights for policy 1, policy_version 67910 (0.0007) [2023-10-10 19:14:14,825][123582] Updated weights for policy 0, policy_version 68003 (0.0007) [2023-10-10 19:14:14,896][123614] Updated weights for policy 1, policy_version 67920 (0.0007) [2023-10-10 19:14:15,204][123582] Updated weights for policy 0, policy_version 68013 (0.0008) [2023-10-10 19:14:15,275][123614] Updated weights for policy 1, policy_version 67930 (0.0008) [2023-10-10 19:14:15,579][123582] Updated weights for policy 0, policy_version 68023 (0.0008) [2023-10-10 19:14:18,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 139231232. Throughput: 0: 1792.0, 1: 1808.1. Samples: 34820166. Policy #0 lag: (min: 3.0, avg: 12.2, max: 35.0) [2023-10-10 19:14:18,789][122664] Avg episode reward: [(0, '75.170'), (1, '67.960')] [2023-10-10 19:14:18,876][123614] Updated weights for policy 1, policy_version 67940 (0.0008) [2023-10-10 19:14:19,242][123614] Updated weights for policy 1, policy_version 67950 (0.0009) [2023-10-10 19:14:19,336][123582] Updated weights for policy 0, policy_version 68033 (0.0010) [2023-10-10 19:14:19,606][123614] Updated weights for policy 1, policy_version 67960 (0.0008) [2023-10-10 19:14:19,706][123582] Updated weights for policy 0, policy_version 68043 (0.0008) [2023-10-10 19:14:20,077][123582] Updated weights for policy 0, policy_version 68053 (0.0007) [2023-10-10 19:14:20,455][123582] Updated weights for policy 0, policy_version 68063 (0.0009) [2023-10-10 19:14:23,252][123614] Updated weights for policy 1, policy_version 67970 (0.0007) [2023-10-10 19:14:23,622][123614] Updated weights for policy 1, policy_version 67980 (0.0007) [2023-10-10 19:14:23,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 139296768. Throughput: 0: 1807.3, 1: 1810.4. Samples: 34842472. Policy #0 lag: (min: 16.0, avg: 43.1, max: 48.0) [2023-10-10 19:14:23,789][122664] Avg episode reward: [(0, '79.090'), (1, '68.550')] [2023-10-10 19:14:23,981][123614] Updated weights for policy 1, policy_version 67990 (0.0008) [2023-10-10 19:14:24,096][123582] Updated weights for policy 0, policy_version 68073 (0.0007) [2023-10-10 19:14:24,351][123614] Updated weights for policy 1, policy_version 68000 (0.0009) [2023-10-10 19:14:24,473][123582] Updated weights for policy 0, policy_version 68083 (0.0008) [2023-10-10 19:14:24,836][123582] Updated weights for policy 0, policy_version 68093 (0.0008) [2023-10-10 19:14:28,217][123614] Updated weights for policy 1, policy_version 68010 (0.0007) [2023-10-10 19:14:28,475][123582] Updated weights for policy 0, policy_version 68103 (0.0007) [2023-10-10 19:14:28,577][123614] Updated weights for policy 1, policy_version 68020 (0.0008) [2023-10-10 19:14:28,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 139362304. Throughput: 0: 1800.6, 1: 1802.2. Samples: 34852962. Policy #0 lag: (min: 16.0, avg: 43.1, max: 48.0) [2023-10-10 19:14:28,788][122664] Avg episode reward: [(0, '74.740'), (1, '70.340')] [2023-10-10 19:14:28,855][123582] Updated weights for policy 0, policy_version 68113 (0.0007) [2023-10-10 19:14:28,943][123614] Updated weights for policy 1, policy_version 68030 (0.0009) [2023-10-10 19:14:29,236][123582] Updated weights for policy 0, policy_version 68123 (0.0007) [2023-10-10 19:14:32,646][123614] Updated weights for policy 1, policy_version 68040 (0.0007) [2023-10-10 19:14:32,774][123582] Updated weights for policy 0, policy_version 68133 (0.0007) [2023-10-10 19:14:33,017][123614] Updated weights for policy 1, policy_version 68050 (0.0008) [2023-10-10 19:14:33,158][123582] Updated weights for policy 0, policy_version 68143 (0.0008) [2023-10-10 19:14:33,376][123614] Updated weights for policy 1, policy_version 68060 (0.0007) [2023-10-10 19:14:33,524][123582] Updated weights for policy 0, policy_version 68153 (0.0009) [2023-10-10 19:14:33,788][122664] Fps is (10 sec: 19660.8, 60 sec: 15291.8, 300 sec: 14551.2). Total num frames: 139493376. Throughput: 0: 1816.3, 1: 1812.0. Samples: 34875698. Policy #0 lag: (min: 16.0, avg: 43.1, max: 48.0) [2023-10-10 19:14:33,789][122664] Avg episode reward: [(0, '73.580'), (1, '68.180')] [2023-10-10 19:14:37,008][123582] Updated weights for policy 0, policy_version 68163 (0.0011) [2023-10-10 19:14:37,037][123614] Updated weights for policy 1, policy_version 68070 (0.0008) [2023-10-10 19:14:37,377][123582] Updated weights for policy 0, policy_version 68173 (0.0009) [2023-10-10 19:14:37,402][123614] Updated weights for policy 1, policy_version 68080 (0.0008) [2023-10-10 19:14:37,749][123582] Updated weights for policy 0, policy_version 68183 (0.0009) [2023-10-10 19:14:37,765][123614] Updated weights for policy 1, policy_version 68090 (0.0008) [2023-10-10 19:14:38,788][122664] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 139558912. Throughput: 0: 1821.4, 1: 1797.6. Samples: 34895770. Policy #0 lag: (min: 16.0, avg: 43.1, max: 48.0) [2023-10-10 19:14:38,789][122664] Avg episode reward: [(0, '72.750'), (1, '66.700')] [2023-10-10 19:14:41,376][123582] Updated weights for policy 0, policy_version 68193 (0.0008) [2023-10-10 19:14:41,609][123614] Updated weights for policy 1, policy_version 68100 (0.0008) [2023-10-10 19:14:41,749][123582] Updated weights for policy 0, policy_version 68203 (0.0007) [2023-10-10 19:14:41,968][123614] Updated weights for policy 1, policy_version 68110 (0.0008) [2023-10-10 19:14:42,111][123582] Updated weights for policy 0, policy_version 68213 (0.0009) [2023-10-10 19:14:42,332][123614] Updated weights for policy 1, policy_version 68120 (0.0008) [2023-10-10 19:14:42,487][123582] Updated weights for policy 0, policy_version 68223 (0.0008) [2023-10-10 19:14:43,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 139624448. Throughput: 0: 1811.1, 1: 1810.8. Samples: 34908218. Policy #0 lag: (min: 16.0, avg: 43.1, max: 48.0) [2023-10-10 19:14:43,789][122664] Avg episode reward: [(0, '74.190'), (1, '68.830')] [2023-10-10 19:14:45,912][123614] Updated weights for policy 1, policy_version 68130 (0.0010) [2023-10-10 19:14:46,288][123614] Updated weights for policy 1, policy_version 68140 (0.0008) [2023-10-10 19:14:46,327][123582] Updated weights for policy 0, policy_version 68233 (0.0008) [2023-10-10 19:14:46,653][123614] Updated weights for policy 1, policy_version 68150 (0.0008) [2023-10-10 19:14:46,692][123582] Updated weights for policy 0, policy_version 68243 (0.0009) [2023-10-10 19:14:47,027][123614] Updated weights for policy 1, policy_version 68160 (0.0008) [2023-10-10 19:14:47,065][123582] Updated weights for policy 0, policy_version 68253 (0.0008) [2023-10-10 19:14:48,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 139689984. Throughput: 0: 1814.4, 1: 1802.4. Samples: 34928332. Policy #0 lag: (min: 16.0, avg: 43.1, max: 48.0) [2023-10-10 19:14:48,788][122664] Avg episode reward: [(0, '72.760'), (1, '71.130')] [2023-10-10 19:14:50,667][123582] Updated weights for policy 0, policy_version 68263 (0.0007) [2023-10-10 19:14:50,738][123614] Updated weights for policy 1, policy_version 68170 (0.0008) [2023-10-10 19:14:51,045][123582] Updated weights for policy 0, policy_version 68273 (0.0007) [2023-10-10 19:14:51,107][123614] Updated weights for policy 1, policy_version 68180 (0.0008) [2023-10-10 19:14:51,415][123582] Updated weights for policy 0, policy_version 68283 (0.0007) [2023-10-10 19:14:51,472][123614] Updated weights for policy 1, policy_version 68190 (0.0007) [2023-10-10 19:14:53,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 139755520. Throughput: 0: 1821.3, 1: 1817.9. Samples: 34951580. Policy #0 lag: (min: 16.0, avg: 43.1, max: 48.0) [2023-10-10 19:14:53,789][122664] Avg episode reward: [(0, '73.460'), (1, '74.470')] [2023-10-10 19:14:54,930][123614] Updated weights for policy 1, policy_version 68200 (0.0007) [2023-10-10 19:14:55,194][123582] Updated weights for policy 0, policy_version 68293 (0.0009) [2023-10-10 19:14:55,299][123614] Updated weights for policy 1, policy_version 68210 (0.0008) [2023-10-10 19:14:55,561][123582] Updated weights for policy 0, policy_version 68303 (0.0007) [2023-10-10 19:14:55,657][123614] Updated weights for policy 1, policy_version 68220 (0.0008) [2023-10-10 19:14:55,931][123582] Updated weights for policy 0, policy_version 68313 (0.0008) [2023-10-10 19:14:58,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 139821056. Throughput: 0: 1817.3, 1: 1820.7. Samples: 34961352. Policy #0 lag: (min: 16.0, avg: 43.1, max: 48.0) [2023-10-10 19:14:58,788][122664] Avg episode reward: [(0, '73.900'), (1, '78.990')] [2023-10-10 19:14:59,449][123614] Updated weights for policy 1, policy_version 68230 (0.0009) [2023-10-10 19:14:59,812][123582] Updated weights for policy 0, policy_version 68323 (0.0008) [2023-10-10 19:14:59,815][123614] Updated weights for policy 1, policy_version 68240 (0.0010) [2023-10-10 19:15:00,179][123582] Updated weights for policy 0, policy_version 68333 (0.0008) [2023-10-10 19:15:00,187][123614] Updated weights for policy 1, policy_version 68250 (0.0008) [2023-10-10 19:15:00,548][123582] Updated weights for policy 0, policy_version 68343 (0.0007) [2023-10-10 19:15:03,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 139886592. Throughput: 0: 1818.8, 1: 1822.4. Samples: 34984016. Policy #0 lag: (min: 16.0, avg: 43.1, max: 48.0) [2023-10-10 19:15:03,789][122664] Avg episode reward: [(0, '74.290'), (1, '78.040')] [2023-10-10 19:15:03,799][123614] Updated weights for policy 1, policy_version 68260 (0.0008) [2023-10-10 19:15:04,164][123614] Updated weights for policy 1, policy_version 68270 (0.0007) [2023-10-10 19:15:04,233][123582] Updated weights for policy 0, policy_version 68353 (0.0009) [2023-10-10 19:15:04,544][123614] Updated weights for policy 1, policy_version 68280 (0.0008) [2023-10-10 19:15:04,611][123582] Updated weights for policy 0, policy_version 68363 (0.0008) [2023-10-10 19:15:04,994][123582] Updated weights for policy 0, policy_version 68373 (0.0010) [2023-10-10 19:15:05,362][123582] Updated weights for policy 0, policy_version 68383 (0.0009) [2023-10-10 19:15:08,142][123614] Updated weights for policy 1, policy_version 68290 (0.0010) [2023-10-10 19:15:08,511][123614] Updated weights for policy 1, policy_version 68300 (0.0010) [2023-10-10 19:15:08,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 139952128. Throughput: 0: 1815.7, 1: 1822.4. Samples: 35006190. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-10 19:15:08,789][122664] Avg episode reward: [(0, '74.380'), (1, '80.800')] [2023-10-10 19:15:08,878][123614] Updated weights for policy 1, policy_version 68310 (0.0007) [2023-10-10 19:15:09,099][123582] Updated weights for policy 0, policy_version 68393 (0.0008) [2023-10-10 19:15:09,253][123614] Updated weights for policy 1, policy_version 68320 (0.0010) [2023-10-10 19:15:09,478][123582] Updated weights for policy 0, policy_version 68403 (0.0008) [2023-10-10 19:15:09,851][123582] Updated weights for policy 0, policy_version 68413 (0.0009) [2023-10-10 19:15:13,078][123614] Updated weights for policy 1, policy_version 68330 (0.0010) [2023-10-10 19:15:13,444][123614] Updated weights for policy 1, policy_version 68340 (0.0009) [2023-10-10 19:15:13,586][123582] Updated weights for policy 0, policy_version 68423 (0.0007) [2023-10-10 19:15:13,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 140017664. Throughput: 0: 1813.2, 1: 1824.6. Samples: 35016662. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-10 19:15:13,788][122664] Avg episode reward: [(0, '73.150'), (1, '78.680')] [2023-10-10 19:15:13,810][123614] Updated weights for policy 1, policy_version 68350 (0.0008) [2023-10-10 19:15:13,956][123582] Updated weights for policy 0, policy_version 68433 (0.0010) [2023-10-10 19:15:14,339][123582] Updated weights for policy 0, policy_version 68443 (0.0010) [2023-10-10 19:15:17,567][123614] Updated weights for policy 1, policy_version 68360 (0.0008) [2023-10-10 19:15:17,924][123614] Updated weights for policy 1, policy_version 68370 (0.0007) [2023-10-10 19:15:17,995][123582] Updated weights for policy 0, policy_version 68453 (0.0010) [2023-10-10 19:15:18,297][123614] Updated weights for policy 1, policy_version 68380 (0.0007) [2023-10-10 19:15:18,362][123582] Updated weights for policy 0, policy_version 68463 (0.0009) [2023-10-10 19:15:18,741][123582] Updated weights for policy 0, policy_version 68473 (0.0008) [2023-10-10 19:15:18,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 140115968. Throughput: 0: 1803.0, 1: 1821.3. Samples: 35038792. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-10 19:15:18,789][122664] Avg episode reward: [(0, '69.780'), (1, '79.220')] [2023-10-10 19:15:22,002][123614] Updated weights for policy 1, policy_version 68390 (0.0008) [2023-10-10 19:15:22,375][123582] Updated weights for policy 0, policy_version 68483 (0.0008) [2023-10-10 19:15:22,378][123614] Updated weights for policy 1, policy_version 68400 (0.0007) [2023-10-10 19:15:22,741][123614] Updated weights for policy 1, policy_version 68410 (0.0007) [2023-10-10 19:15:22,756][123582] Updated weights for policy 0, policy_version 68493 (0.0009) [2023-10-10 19:15:23,120][123582] Updated weights for policy 0, policy_version 68503 (0.0008) [2023-10-10 19:15:23,788][122664] Fps is (10 sec: 19660.2, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 140214272. Throughput: 0: 1804.9, 1: 1831.4. Samples: 35059404. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-10 19:15:23,789][122664] Avg episode reward: [(0, '64.380'), (1, '80.010')] [2023-10-10 19:15:26,445][123614] Updated weights for policy 1, policy_version 68420 (0.0007) [2023-10-10 19:15:26,810][123614] Updated weights for policy 1, policy_version 68430 (0.0008) [2023-10-10 19:15:26,842][123582] Updated weights for policy 0, policy_version 68513 (0.0011) [2023-10-10 19:15:27,171][123614] Updated weights for policy 1, policy_version 68440 (0.0010) [2023-10-10 19:15:27,200][123582] Updated weights for policy 0, policy_version 68523 (0.0010) [2023-10-10 19:15:27,576][123582] Updated weights for policy 0, policy_version 68533 (0.0008) [2023-10-10 19:15:27,934][123582] Updated weights for policy 0, policy_version 68543 (0.0008) [2023-10-10 19:15:28,788][122664] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 140279808. Throughput: 0: 1806.5, 1: 1820.4. Samples: 35071432. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-10 19:15:28,789][122664] Avg episode reward: [(0, '64.800'), (1, '81.510')] [2023-10-10 19:15:30,923][123614] Updated weights for policy 1, policy_version 68450 (0.0007) [2023-10-10 19:15:31,292][123614] Updated weights for policy 1, policy_version 68460 (0.0009) [2023-10-10 19:15:31,647][123614] Updated weights for policy 1, policy_version 68470 (0.0007) [2023-10-10 19:15:31,797][123582] Updated weights for policy 0, policy_version 68553 (0.0008) [2023-10-10 19:15:32,014][123614] Updated weights for policy 1, policy_version 68480 (0.0010) [2023-10-10 19:15:32,180][123582] Updated weights for policy 0, policy_version 68563 (0.0008) [2023-10-10 19:15:32,537][123582] Updated weights for policy 0, policy_version 68573 (0.0009) [2023-10-10 19:15:33,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 140345344. Throughput: 0: 1811.7, 1: 1825.0. Samples: 35091982. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-10 19:15:33,789][122664] Avg episode reward: [(0, '67.290'), (1, '75.540')] [2023-10-10 19:15:35,747][123614] Updated weights for policy 1, policy_version 68490 (0.0007) [2023-10-10 19:15:36,115][123614] Updated weights for policy 1, policy_version 68500 (0.0007) [2023-10-10 19:15:36,304][123582] Updated weights for policy 0, policy_version 68583 (0.0007) [2023-10-10 19:15:36,493][123614] Updated weights for policy 1, policy_version 68510 (0.0009) [2023-10-10 19:15:36,680][123582] Updated weights for policy 0, policy_version 68593 (0.0007) [2023-10-10 19:15:37,061][123582] Updated weights for policy 0, policy_version 68603 (0.0008) [2023-10-10 19:15:38,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 140410880. Throughput: 0: 1800.8, 1: 1809.1. Samples: 35114026. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-10 19:15:38,788][122664] Avg episode reward: [(0, '66.660'), (1, '71.970')] [2023-10-10 19:15:40,211][123614] Updated weights for policy 1, policy_version 68520 (0.0008) [2023-10-10 19:15:40,565][123614] Updated weights for policy 1, policy_version 68530 (0.0008) [2023-10-10 19:15:40,667][123582] Updated weights for policy 0, policy_version 68613 (0.0009) [2023-10-10 19:15:40,931][123614] Updated weights for policy 1, policy_version 68540 (0.0009) [2023-10-10 19:15:41,039][123582] Updated weights for policy 0, policy_version 68623 (0.0007) [2023-10-10 19:15:41,403][123582] Updated weights for policy 0, policy_version 68633 (0.0008) [2023-10-10 19:15:43,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 140476416. Throughput: 0: 1812.7, 1: 1808.3. Samples: 35124298. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-10 19:15:43,789][122664] Avg episode reward: [(0, '65.080'), (1, '73.700')] [2023-10-10 19:15:44,526][123614] Updated weights for policy 1, policy_version 68550 (0.0009) [2023-10-10 19:15:44,894][123614] Updated weights for policy 1, policy_version 68560 (0.0008) [2023-10-10 19:15:45,109][123582] Updated weights for policy 0, policy_version 68643 (0.0008) [2023-10-10 19:15:45,252][123614] Updated weights for policy 1, policy_version 68570 (0.0008) [2023-10-10 19:15:45,476][123582] Updated weights for policy 0, policy_version 68653 (0.0010) [2023-10-10 19:15:45,848][123582] Updated weights for policy 0, policy_version 68663 (0.0008) [2023-10-10 19:15:48,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 140541952. Throughput: 0: 1808.5, 1: 1803.6. Samples: 35146558. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-10 19:15:48,789][122664] Avg episode reward: [(0, '63.680'), (1, '72.680')] [2023-10-10 19:15:49,052][123614] Updated weights for policy 1, policy_version 68580 (0.0008) [2023-10-10 19:15:49,414][123614] Updated weights for policy 1, policy_version 68590 (0.0008) [2023-10-10 19:15:49,555][123582] Updated weights for policy 0, policy_version 68673 (0.0011) [2023-10-10 19:15:49,786][123614] Updated weights for policy 1, policy_version 68600 (0.0008) [2023-10-10 19:15:49,978][123582] Updated weights for policy 0, policy_version 68683 (0.0007) [2023-10-10 19:15:50,353][123582] Updated weights for policy 0, policy_version 68693 (0.0009) [2023-10-10 19:15:50,723][123582] Updated weights for policy 0, policy_version 68703 (0.0010) [2023-10-10 19:15:53,437][123614] Updated weights for policy 1, policy_version 68610 (0.0007) [2023-10-10 19:15:53,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 140607488. Throughput: 0: 1800.3, 1: 1808.5. Samples: 35168586. Policy #0 lag: (min: 8.0, avg: 32.5, max: 40.0) [2023-10-10 19:15:53,788][122664] Avg episode reward: [(0, '59.350'), (1, '75.690')] [2023-10-10 19:15:53,799][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000068704_70352896.pth... [2023-10-10 19:15:53,816][123614] Updated weights for policy 1, policy_version 68620 (0.0010) [2023-10-10 19:15:53,834][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000067040_68648960.pth [2023-10-10 19:15:54,179][123614] Updated weights for policy 1, policy_version 68630 (0.0010) [2023-10-10 19:15:54,362][123582] Updated weights for policy 0, policy_version 68713 (0.0008) [2023-10-10 19:15:54,547][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000068640_70287360.pth... [2023-10-10 19:15:54,550][123614] Updated weights for policy 1, policy_version 68640 (0.0008) [2023-10-10 19:15:54,576][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000066944_68550656.pth [2023-10-10 19:15:54,737][123582] Updated weights for policy 0, policy_version 68723 (0.0010) [2023-10-10 19:15:55,114][123582] Updated weights for policy 0, policy_version 68733 (0.0007) [2023-10-10 19:15:58,347][123614] Updated weights for policy 1, policy_version 68650 (0.0007) [2023-10-10 19:15:58,724][123614] Updated weights for policy 1, policy_version 68660 (0.0008) [2023-10-10 19:15:58,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 140673024. Throughput: 0: 1801.7, 1: 1801.8. Samples: 35178822. Policy #0 lag: (min: 8.0, avg: 32.5, max: 40.0) [2023-10-10 19:15:58,789][122664] Avg episode reward: [(0, '59.260'), (1, '73.340')] [2023-10-10 19:15:58,876][123582] Updated weights for policy 0, policy_version 68743 (0.0009) [2023-10-10 19:15:59,087][123614] Updated weights for policy 1, policy_version 68670 (0.0007) [2023-10-10 19:15:59,259][123582] Updated weights for policy 0, policy_version 68753 (0.0008) [2023-10-10 19:15:59,629][123582] Updated weights for policy 0, policy_version 68763 (0.0007) [2023-10-10 19:16:02,834][123614] Updated weights for policy 1, policy_version 68680 (0.0010) [2023-10-10 19:16:03,209][123614] Updated weights for policy 1, policy_version 68690 (0.0008) [2023-10-10 19:16:03,342][123582] Updated weights for policy 0, policy_version 68773 (0.0007) [2023-10-10 19:16:03,575][123614] Updated weights for policy 1, policy_version 68700 (0.0010) [2023-10-10 19:16:03,706][123582] Updated weights for policy 0, policy_version 68783 (0.0008) [2023-10-10 19:16:03,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 140771328. Throughput: 0: 1799.2, 1: 1811.1. Samples: 35201252. Policy #0 lag: (min: 8.0, avg: 32.5, max: 40.0) [2023-10-10 19:16:03,788][122664] Avg episode reward: [(0, '57.310'), (1, '71.340')] [2023-10-10 19:16:04,081][123582] Updated weights for policy 0, policy_version 68793 (0.0009) [2023-10-10 19:16:07,288][123614] Updated weights for policy 1, policy_version 68710 (0.0009) [2023-10-10 19:16:07,668][123614] Updated weights for policy 1, policy_version 68720 (0.0008) [2023-10-10 19:16:07,774][123582] Updated weights for policy 0, policy_version 68803 (0.0009) [2023-10-10 19:16:08,028][123614] Updated weights for policy 1, policy_version 68730 (0.0008) [2023-10-10 19:16:08,144][123582] Updated weights for policy 0, policy_version 68813 (0.0008) [2023-10-10 19:16:08,515][123582] Updated weights for policy 0, policy_version 68823 (0.0009) [2023-10-10 19:16:08,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 140836864. Throughput: 0: 1808.9, 1: 1801.1. Samples: 35221850. Policy #0 lag: (min: 8.0, avg: 32.5, max: 40.0) [2023-10-10 19:16:08,789][122664] Avg episode reward: [(0, '56.440'), (1, '71.800')] [2023-10-10 19:16:11,935][123614] Updated weights for policy 1, policy_version 68740 (0.0007) [2023-10-10 19:16:12,193][123582] Updated weights for policy 0, policy_version 68833 (0.0007) [2023-10-10 19:16:12,304][123614] Updated weights for policy 1, policy_version 68750 (0.0009) [2023-10-10 19:16:12,563][123582] Updated weights for policy 0, policy_version 68843 (0.0008) [2023-10-10 19:16:12,678][123614] Updated weights for policy 1, policy_version 68760 (0.0009) [2023-10-10 19:16:12,932][123582] Updated weights for policy 0, policy_version 68853 (0.0010) [2023-10-10 19:16:13,296][123582] Updated weights for policy 0, policy_version 68863 (0.0010) [2023-10-10 19:16:13,788][122664] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 140935168. Throughput: 0: 1797.3, 1: 1816.8. Samples: 35234068. Policy #0 lag: (min: 8.0, avg: 32.5, max: 40.0) [2023-10-10 19:16:13,789][122664] Avg episode reward: [(0, '57.370'), (1, '67.840')] [2023-10-10 19:16:16,487][123614] Updated weights for policy 1, policy_version 68770 (0.0008) [2023-10-10 19:16:16,855][123614] Updated weights for policy 1, policy_version 68780 (0.0008) [2023-10-10 19:16:17,075][123582] Updated weights for policy 0, policy_version 68873 (0.0010) [2023-10-10 19:16:17,225][123614] Updated weights for policy 1, policy_version 68790 (0.0009) [2023-10-10 19:16:17,450][123582] Updated weights for policy 0, policy_version 68883 (0.0009) [2023-10-10 19:16:17,593][123614] Updated weights for policy 1, policy_version 68800 (0.0007) [2023-10-10 19:16:17,827][123582] Updated weights for policy 0, policy_version 68893 (0.0007) [2023-10-10 19:16:18,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14440.2). Total num frames: 141000704. Throughput: 0: 1805.5, 1: 1803.3. Samples: 35254380. Policy #0 lag: (min: 8.0, avg: 32.5, max: 40.0) [2023-10-10 19:16:18,789][122664] Avg episode reward: [(0, '59.660'), (1, '70.330')] [2023-10-10 19:16:21,434][123614] Updated weights for policy 1, policy_version 68810 (0.0008) [2023-10-10 19:16:21,502][123582] Updated weights for policy 0, policy_version 68903 (0.0008) [2023-10-10 19:16:21,803][123614] Updated weights for policy 1, policy_version 68820 (0.0008) [2023-10-10 19:16:21,870][123582] Updated weights for policy 0, policy_version 68913 (0.0007) [2023-10-10 19:16:22,169][123614] Updated weights for policy 1, policy_version 68830 (0.0008) [2023-10-10 19:16:22,239][123582] Updated weights for policy 0, policy_version 68923 (0.0007) [2023-10-10 19:16:23,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 141066240. Throughput: 0: 1800.8, 1: 1809.7. Samples: 35276500. Policy #0 lag: (min: 8.0, avg: 32.5, max: 40.0) [2023-10-10 19:16:23,788][122664] Avg episode reward: [(0, '64.810'), (1, '73.800')] [2023-10-10 19:16:25,722][123614] Updated weights for policy 1, policy_version 68840 (0.0008) [2023-10-10 19:16:25,996][123582] Updated weights for policy 0, policy_version 68933 (0.0009) [2023-10-10 19:16:26,087][123614] Updated weights for policy 1, policy_version 68850 (0.0007) [2023-10-10 19:16:26,366][123582] Updated weights for policy 0, policy_version 68943 (0.0008) [2023-10-10 19:16:26,452][123614] Updated weights for policy 1, policy_version 68860 (0.0007) [2023-10-10 19:16:26,737][123582] Updated weights for policy 0, policy_version 68953 (0.0008) [2023-10-10 19:16:28,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 141131776. Throughput: 0: 1808.9, 1: 1810.5. Samples: 35287172. Policy #0 lag: (min: 8.0, avg: 32.5, max: 40.0) [2023-10-10 19:16:28,789][122664] Avg episode reward: [(0, '65.670'), (1, '72.700')] [2023-10-10 19:16:30,213][123614] Updated weights for policy 1, policy_version 68870 (0.0008) [2023-10-10 19:16:30,440][123582] Updated weights for policy 0, policy_version 68963 (0.0010) [2023-10-10 19:16:30,589][123614] Updated weights for policy 1, policy_version 68880 (0.0009) [2023-10-10 19:16:30,813][123582] Updated weights for policy 0, policy_version 68973 (0.0009) [2023-10-10 19:16:30,954][123614] Updated weights for policy 1, policy_version 68890 (0.0008) [2023-10-10 19:16:31,184][123582] Updated weights for policy 0, policy_version 68983 (0.0007) [2023-10-10 19:16:33,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 141197312. Throughput: 0: 1803.2, 1: 1806.2. Samples: 35308980. Policy #0 lag: (min: 8.0, avg: 32.5, max: 40.0) [2023-10-10 19:16:33,789][122664] Avg episode reward: [(0, '65.530'), (1, '73.730')] [2023-10-10 19:16:34,678][123614] Updated weights for policy 1, policy_version 68900 (0.0008) [2023-10-10 19:16:34,980][123582] Updated weights for policy 0, policy_version 68993 (0.0007) [2023-10-10 19:16:35,055][123614] Updated weights for policy 1, policy_version 68910 (0.0007) [2023-10-10 19:16:35,391][123582] Updated weights for policy 0, policy_version 69003 (0.0008) [2023-10-10 19:16:35,418][123614] Updated weights for policy 1, policy_version 68920 (0.0009) [2023-10-10 19:16:35,761][123582] Updated weights for policy 0, policy_version 69013 (0.0007) [2023-10-10 19:16:36,125][123582] Updated weights for policy 0, policy_version 69023 (0.0007) [2023-10-10 19:16:38,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 141262848. Throughput: 0: 1808.0, 1: 1815.3. Samples: 35331634. Policy #0 lag: (min: 10.0, avg: 18.0, max: 42.0) [2023-10-10 19:16:38,789][122664] Avg episode reward: [(0, '71.830'), (1, '70.290')] [2023-10-10 19:16:39,084][123614] Updated weights for policy 1, policy_version 68930 (0.0008) [2023-10-10 19:16:39,445][123614] Updated weights for policy 1, policy_version 68940 (0.0009) [2023-10-10 19:16:39,724][123582] Updated weights for policy 0, policy_version 69033 (0.0007) [2023-10-10 19:16:39,811][123614] Updated weights for policy 1, policy_version 68950 (0.0009) [2023-10-10 19:16:40,097][123582] Updated weights for policy 0, policy_version 69043 (0.0008) [2023-10-10 19:16:40,179][123614] Updated weights for policy 1, policy_version 68960 (0.0009) [2023-10-10 19:16:40,474][123582] Updated weights for policy 0, policy_version 69053 (0.0010) [2023-10-10 19:16:43,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 141328384. Throughput: 0: 1803.5, 1: 1808.2. Samples: 35341348. Policy #0 lag: (min: 10.0, avg: 18.0, max: 42.0) [2023-10-10 19:16:43,789][122664] Avg episode reward: [(0, '71.130'), (1, '72.650')] [2023-10-10 19:16:43,812][123614] Updated weights for policy 1, policy_version 68970 (0.0010) [2023-10-10 19:16:44,172][123614] Updated weights for policy 1, policy_version 68980 (0.0009) [2023-10-10 19:16:44,233][123582] Updated weights for policy 0, policy_version 69063 (0.0008) [2023-10-10 19:16:44,533][123614] Updated weights for policy 1, policy_version 68990 (0.0007) [2023-10-10 19:16:44,609][123582] Updated weights for policy 0, policy_version 69073 (0.0009) [2023-10-10 19:16:44,980][123582] Updated weights for policy 0, policy_version 69083 (0.0008) [2023-10-10 19:16:48,305][123614] Updated weights for policy 1, policy_version 69000 (0.0007) [2023-10-10 19:16:48,672][123614] Updated weights for policy 1, policy_version 69010 (0.0009) [2023-10-10 19:16:48,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 141393920. Throughput: 0: 1801.5, 1: 1812.5. Samples: 35363882. Policy #0 lag: (min: 10.0, avg: 18.0, max: 42.0) [2023-10-10 19:16:48,788][122664] Avg episode reward: [(0, '70.340'), (1, '73.910')] [2023-10-10 19:16:48,888][123582] Updated weights for policy 0, policy_version 69093 (0.0007) [2023-10-10 19:16:49,041][123614] Updated weights for policy 1, policy_version 69020 (0.0007) [2023-10-10 19:16:49,254][123582] Updated weights for policy 0, policy_version 69103 (0.0008) [2023-10-10 19:16:49,628][123582] Updated weights for policy 0, policy_version 69113 (0.0008) [2023-10-10 19:16:52,888][123614] Updated weights for policy 1, policy_version 69030 (0.0008) [2023-10-10 19:16:53,248][123614] Updated weights for policy 1, policy_version 69040 (0.0008) [2023-10-10 19:16:53,320][123582] Updated weights for policy 0, policy_version 69123 (0.0008) [2023-10-10 19:16:53,616][123614] Updated weights for policy 1, policy_version 69050 (0.0009) [2023-10-10 19:16:53,690][123582] Updated weights for policy 0, policy_version 69133 (0.0009) [2023-10-10 19:16:53,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 141459456. Throughput: 0: 1811.7, 1: 1807.3. Samples: 35384708. Policy #0 lag: (min: 10.0, avg: 18.0, max: 42.0) [2023-10-10 19:16:53,788][122664] Avg episode reward: [(0, '69.140'), (1, '72.650')] [2023-10-10 19:16:54,071][123582] Updated weights for policy 0, policy_version 69143 (0.0008) [2023-10-10 19:16:57,300][123614] Updated weights for policy 1, policy_version 69060 (0.0008) [2023-10-10 19:16:57,673][123614] Updated weights for policy 1, policy_version 69070 (0.0008) [2023-10-10 19:16:57,749][123582] Updated weights for policy 0, policy_version 69153 (0.0009) [2023-10-10 19:16:58,046][123614] Updated weights for policy 1, policy_version 69080 (0.0009) [2023-10-10 19:16:58,118][123582] Updated weights for policy 0, policy_version 69163 (0.0007) [2023-10-10 19:16:58,494][123582] Updated weights for policy 0, policy_version 69173 (0.0009) [2023-10-10 19:16:58,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 141557760. Throughput: 0: 1797.2, 1: 1803.4. Samples: 35396098. Policy #0 lag: (min: 10.0, avg: 18.0, max: 42.0) [2023-10-10 19:16:58,789][122664] Avg episode reward: [(0, '70.540'), (1, '72.140')] [2023-10-10 19:16:58,858][123582] Updated weights for policy 0, policy_version 69183 (0.0007) [2023-10-10 19:17:01,735][123614] Updated weights for policy 1, policy_version 69090 (0.0008) [2023-10-10 19:17:02,101][123614] Updated weights for policy 1, policy_version 69100 (0.0009) [2023-10-10 19:17:02,413][123582] Updated weights for policy 0, policy_version 69193 (0.0007) [2023-10-10 19:17:02,463][123614] Updated weights for policy 1, policy_version 69110 (0.0007) [2023-10-10 19:17:02,781][123582] Updated weights for policy 0, policy_version 69203 (0.0009) [2023-10-10 19:17:02,831][123614] Updated weights for policy 1, policy_version 69120 (0.0008) [2023-10-10 19:17:03,158][123582] Updated weights for policy 0, policy_version 69213 (0.0008) [2023-10-10 19:17:03,788][122664] Fps is (10 sec: 19660.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 141656064. Throughput: 0: 1810.8, 1: 1808.7. Samples: 35417258. Policy #0 lag: (min: 10.0, avg: 18.0, max: 42.0) [2023-10-10 19:17:03,789][122664] Avg episode reward: [(0, '70.040'), (1, '76.010')] [2023-10-10 19:17:06,621][123614] Updated weights for policy 1, policy_version 69130 (0.0010) [2023-10-10 19:17:06,813][123582] Updated weights for policy 0, policy_version 69223 (0.0009) [2023-10-10 19:17:06,995][123614] Updated weights for policy 1, policy_version 69140 (0.0008) [2023-10-10 19:17:07,180][123582] Updated weights for policy 0, policy_version 69233 (0.0008) [2023-10-10 19:17:07,355][123614] Updated weights for policy 1, policy_version 69150 (0.0007) [2023-10-10 19:17:07,562][123582] Updated weights for policy 0, policy_version 69243 (0.0007) [2023-10-10 19:17:08,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 141721600. Throughput: 0: 1802.0, 1: 1797.9. Samples: 35438494. Policy #0 lag: (min: 10.0, avg: 18.0, max: 42.0) [2023-10-10 19:17:08,789][122664] Avg episode reward: [(0, '72.290'), (1, '75.900')] [2023-10-10 19:17:11,019][123614] Updated weights for policy 1, policy_version 69160 (0.0010) [2023-10-10 19:17:11,381][123614] Updated weights for policy 1, policy_version 69170 (0.0008) [2023-10-10 19:17:11,396][123582] Updated weights for policy 0, policy_version 69253 (0.0008) [2023-10-10 19:17:11,744][123614] Updated weights for policy 1, policy_version 69180 (0.0008) [2023-10-10 19:17:11,771][123582] Updated weights for policy 0, policy_version 69263 (0.0007) [2023-10-10 19:17:12,145][123582] Updated weights for policy 0, policy_version 69273 (0.0008) [2023-10-10 19:17:13,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 141787136. Throughput: 0: 1812.0, 1: 1799.5. Samples: 35449692. Policy #0 lag: (min: 10.0, avg: 18.0, max: 42.0) [2023-10-10 19:17:13,789][122664] Avg episode reward: [(0, '73.540'), (1, '78.740')] [2023-10-10 19:17:15,610][123614] Updated weights for policy 1, policy_version 69190 (0.0008) [2023-10-10 19:17:15,838][123582] Updated weights for policy 0, policy_version 69283 (0.0008) [2023-10-10 19:17:15,976][123614] Updated weights for policy 1, policy_version 69200 (0.0009) [2023-10-10 19:17:16,207][123582] Updated weights for policy 0, policy_version 69293 (0.0008) [2023-10-10 19:17:16,341][123614] Updated weights for policy 1, policy_version 69210 (0.0008) [2023-10-10 19:17:16,567][123582] Updated weights for policy 0, policy_version 69303 (0.0007) [2023-10-10 19:17:18,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 141852672. Throughput: 0: 1803.6, 1: 1796.4. Samples: 35470980. Policy #0 lag: (min: 10.0, avg: 18.0, max: 42.0) [2023-10-10 19:17:18,788][122664] Avg episode reward: [(0, '73.210'), (1, '73.710')] [2023-10-10 19:17:20,091][123614] Updated weights for policy 1, policy_version 69220 (0.0008) [2023-10-10 19:17:20,359][123582] Updated weights for policy 0, policy_version 69313 (0.0009) [2023-10-10 19:17:20,470][123614] Updated weights for policy 1, policy_version 69230 (0.0007) [2023-10-10 19:17:20,786][123582] Updated weights for policy 0, policy_version 69323 (0.0009) [2023-10-10 19:17:20,832][123614] Updated weights for policy 1, policy_version 69240 (0.0007) [2023-10-10 19:17:21,147][123582] Updated weights for policy 0, policy_version 69333 (0.0009) [2023-10-10 19:17:21,530][123582] Updated weights for policy 0, policy_version 69343 (0.0011) [2023-10-10 19:17:23,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 141918208. Throughput: 0: 1798.0, 1: 1800.9. Samples: 35493588. Policy #0 lag: (min: 8.0, avg: 34.4, max: 40.0) [2023-10-10 19:17:23,789][122664] Avg episode reward: [(0, '74.630'), (1, '73.810')] [2023-10-10 19:17:24,322][123614] Updated weights for policy 1, policy_version 69250 (0.0007) [2023-10-10 19:17:24,697][123614] Updated weights for policy 1, policy_version 69260 (0.0010) [2023-10-10 19:17:25,067][123614] Updated weights for policy 1, policy_version 69270 (0.0007) [2023-10-10 19:17:25,217][123582] Updated weights for policy 0, policy_version 69353 (0.0009) [2023-10-10 19:17:25,436][123614] Updated weights for policy 1, policy_version 69280 (0.0007) [2023-10-10 19:17:25,583][123582] Updated weights for policy 0, policy_version 69363 (0.0008) [2023-10-10 19:17:25,955][123582] Updated weights for policy 0, policy_version 69373 (0.0008) [2023-10-10 19:17:28,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 141983744. Throughput: 0: 1804.7, 1: 1797.6. Samples: 35503450. Policy #0 lag: (min: 8.0, avg: 34.4, max: 40.0) [2023-10-10 19:17:28,789][122664] Avg episode reward: [(0, '76.460'), (1, '77.640')] [2023-10-10 19:17:29,196][123614] Updated weights for policy 1, policy_version 69290 (0.0007) [2023-10-10 19:17:29,548][123582] Updated weights for policy 0, policy_version 69383 (0.0008) [2023-10-10 19:17:29,559][123614] Updated weights for policy 1, policy_version 69300 (0.0008) [2023-10-10 19:17:29,924][123582] Updated weights for policy 0, policy_version 69393 (0.0008) [2023-10-10 19:17:29,930][123614] Updated weights for policy 1, policy_version 69310 (0.0008) [2023-10-10 19:17:30,288][123582] Updated weights for policy 0, policy_version 69403 (0.0010) [2023-10-10 19:17:33,737][123614] Updated weights for policy 1, policy_version 69320 (0.0009) [2023-10-10 19:17:33,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 142049280. Throughput: 0: 1808.7, 1: 1793.8. Samples: 35525994. Policy #0 lag: (min: 8.0, avg: 34.4, max: 40.0) [2023-10-10 19:17:33,789][122664] Avg episode reward: [(0, '69.370'), (1, '79.360')] [2023-10-10 19:17:34,084][123582] Updated weights for policy 0, policy_version 69413 (0.0008) [2023-10-10 19:17:34,106][123614] Updated weights for policy 1, policy_version 69330 (0.0009) [2023-10-10 19:17:34,461][123582] Updated weights for policy 0, policy_version 69423 (0.0008) [2023-10-10 19:17:34,469][123614] Updated weights for policy 1, policy_version 69340 (0.0009) [2023-10-10 19:17:34,825][123582] Updated weights for policy 0, policy_version 69433 (0.0007) [2023-10-10 19:17:38,054][123614] Updated weights for policy 1, policy_version 69350 (0.0008) [2023-10-10 19:17:38,419][123614] Updated weights for policy 1, policy_version 69360 (0.0008) [2023-10-10 19:17:38,442][123582] Updated weights for policy 0, policy_version 69443 (0.0009) [2023-10-10 19:17:38,778][123614] Updated weights for policy 1, policy_version 69370 (0.0009) [2023-10-10 19:17:38,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 142114816. Throughput: 0: 1819.3, 1: 1806.3. Samples: 35547862. Policy #0 lag: (min: 8.0, avg: 34.4, max: 40.0) [2023-10-10 19:17:38,789][122664] Avg episode reward: [(0, '66.260'), (1, '76.710')] [2023-10-10 19:17:38,805][123582] Updated weights for policy 0, policy_version 69453 (0.0007) [2023-10-10 19:17:39,179][123582] Updated weights for policy 0, policy_version 69463 (0.0008) [2023-10-10 19:17:42,365][123614] Updated weights for policy 1, policy_version 69380 (0.0008) [2023-10-10 19:17:42,730][123614] Updated weights for policy 1, policy_version 69390 (0.0009) [2023-10-10 19:17:42,936][123582] Updated weights for policy 0, policy_version 69473 (0.0010) [2023-10-10 19:17:43,097][123614] Updated weights for policy 1, policy_version 69400 (0.0009) [2023-10-10 19:17:43,305][123582] Updated weights for policy 0, policy_version 69483 (0.0008) [2023-10-10 19:17:43,671][123582] Updated weights for policy 0, policy_version 69493 (0.0007) [2023-10-10 19:17:43,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 142213120. Throughput: 0: 1813.4, 1: 1808.3. Samples: 35559072. Policy #0 lag: (min: 8.0, avg: 34.4, max: 40.0) [2023-10-10 19:17:43,788][122664] Avg episode reward: [(0, '69.480'), (1, '78.260')] [2023-10-10 19:17:44,045][123582] Updated weights for policy 0, policy_version 69503 (0.0009) [2023-10-10 19:17:46,794][123614] Updated weights for policy 1, policy_version 69410 (0.0007) [2023-10-10 19:17:47,165][123614] Updated weights for policy 1, policy_version 69420 (0.0010) [2023-10-10 19:17:47,541][123614] Updated weights for policy 1, policy_version 69430 (0.0009) [2023-10-10 19:17:47,715][123582] Updated weights for policy 0, policy_version 69513 (0.0007) [2023-10-10 19:17:47,903][123614] Updated weights for policy 1, policy_version 69440 (0.0008) [2023-10-10 19:17:48,087][123582] Updated weights for policy 0, policy_version 69523 (0.0011) [2023-10-10 19:17:48,461][123582] Updated weights for policy 0, policy_version 69533 (0.0009) [2023-10-10 19:17:48,788][122664] Fps is (10 sec: 19661.2, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 142311424. Throughput: 0: 1821.2, 1: 1812.3. Samples: 35580764. Policy #0 lag: (min: 8.0, avg: 34.4, max: 40.0) [2023-10-10 19:17:48,788][122664] Avg episode reward: [(0, '70.330'), (1, '77.750')] [2023-10-10 19:17:51,707][123614] Updated weights for policy 1, policy_version 69450 (0.0009) [2023-10-10 19:17:52,079][123614] Updated weights for policy 1, policy_version 69460 (0.0007) [2023-10-10 19:17:52,202][123582] Updated weights for policy 0, policy_version 69543 (0.0008) [2023-10-10 19:17:52,440][123614] Updated weights for policy 1, policy_version 69470 (0.0008) [2023-10-10 19:17:52,574][123582] Updated weights for policy 0, policy_version 69553 (0.0008) [2023-10-10 19:17:52,956][123582] Updated weights for policy 0, policy_version 69563 (0.0008) [2023-10-10 19:17:53,788][122664] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 142376960. Throughput: 0: 1808.4, 1: 1818.0. Samples: 35601684. Policy #0 lag: (min: 8.0, avg: 34.4, max: 40.0) [2023-10-10 19:17:53,789][122664] Avg episode reward: [(0, '70.020'), (1, '81.820')] [2023-10-10 19:17:53,797][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000069568_71237632.pth... [2023-10-10 19:17:53,797][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000069472_71139328.pth... [2023-10-10 19:17:53,833][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000067776_69402624.pth [2023-10-10 19:17:53,835][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000067872_69500928.pth [2023-10-10 19:17:56,064][123614] Updated weights for policy 1, policy_version 69480 (0.0009) [2023-10-10 19:17:56,441][123614] Updated weights for policy 1, policy_version 69490 (0.0007) [2023-10-10 19:17:56,664][123582] Updated weights for policy 0, policy_version 69573 (0.0010) [2023-10-10 19:17:56,806][123614] Updated weights for policy 1, policy_version 69500 (0.0007) [2023-10-10 19:17:57,032][123582] Updated weights for policy 0, policy_version 69583 (0.0009) [2023-10-10 19:17:57,397][123582] Updated weights for policy 0, policy_version 69593 (0.0010) [2023-10-10 19:17:58,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 142442496. Throughput: 0: 1820.4, 1: 1818.7. Samples: 35613448. Policy #0 lag: (min: 8.0, avg: 34.4, max: 40.0) [2023-10-10 19:17:58,789][122664] Avg episode reward: [(0, '66.820'), (1, '84.600')] [2023-10-10 19:17:58,790][123465] Saving new best policy, reward=84.600! [2023-10-10 19:18:00,421][123614] Updated weights for policy 1, policy_version 69510 (0.0007) [2023-10-10 19:18:00,799][123614] Updated weights for policy 1, policy_version 69520 (0.0010) [2023-10-10 19:18:01,027][123582] Updated weights for policy 0, policy_version 69603 (0.0009) [2023-10-10 19:18:01,157][123614] Updated weights for policy 1, policy_version 69530 (0.0008) [2023-10-10 19:18:01,400][123582] Updated weights for policy 0, policy_version 69613 (0.0008) [2023-10-10 19:18:01,779][123582] Updated weights for policy 0, policy_version 69623 (0.0007) [2023-10-10 19:18:03,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 142508032. Throughput: 0: 1812.0, 1: 1821.0. Samples: 35634464. Policy #0 lag: (min: 8.0, avg: 34.4, max: 40.0) [2023-10-10 19:18:03,789][122664] Avg episode reward: [(0, '70.130'), (1, '84.360')] [2023-10-10 19:18:05,072][123614] Updated weights for policy 1, policy_version 69540 (0.0007) [2023-10-10 19:18:05,430][123614] Updated weights for policy 1, policy_version 69550 (0.0007) [2023-10-10 19:18:05,482][123582] Updated weights for policy 0, policy_version 69633 (0.0010) [2023-10-10 19:18:05,806][123614] Updated weights for policy 1, policy_version 69560 (0.0007) [2023-10-10 19:18:05,873][123582] Updated weights for policy 0, policy_version 69643 (0.0008) [2023-10-10 19:18:06,246][123582] Updated weights for policy 0, policy_version 69653 (0.0010) [2023-10-10 19:18:06,614][123582] Updated weights for policy 0, policy_version 69663 (0.0007) [2023-10-10 19:18:08,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 142573568. Throughput: 0: 1817.1, 1: 1814.1. Samples: 35656994. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:18:08,789][122664] Avg episode reward: [(0, '73.380'), (1, '86.760')] [2023-10-10 19:18:08,801][123465] Saving new best policy, reward=86.760! [2023-10-10 19:18:09,595][123614] Updated weights for policy 1, policy_version 69570 (0.0008) [2023-10-10 19:18:09,965][123614] Updated weights for policy 1, policy_version 69580 (0.0009) [2023-10-10 19:18:10,325][123582] Updated weights for policy 0, policy_version 69673 (0.0007) [2023-10-10 19:18:10,334][123614] Updated weights for policy 1, policy_version 69590 (0.0008) [2023-10-10 19:18:10,703][123582] Updated weights for policy 0, policy_version 69683 (0.0008) [2023-10-10 19:18:10,704][123614] Updated weights for policy 1, policy_version 69600 (0.0008) [2023-10-10 19:18:11,068][123582] Updated weights for policy 0, policy_version 69693 (0.0007) [2023-10-10 19:18:13,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 142639104. Throughput: 0: 1812.6, 1: 1817.5. Samples: 35666806. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:18:13,789][122664] Avg episode reward: [(0, '73.250'), (1, '85.820')] [2023-10-10 19:18:14,319][123614] Updated weights for policy 1, policy_version 69610 (0.0008) [2023-10-10 19:18:14,688][123614] Updated weights for policy 1, policy_version 69620 (0.0008) [2023-10-10 19:18:14,894][123582] Updated weights for policy 0, policy_version 69703 (0.0009) [2023-10-10 19:18:15,055][123614] Updated weights for policy 1, policy_version 69630 (0.0008) [2023-10-10 19:18:15,258][123582] Updated weights for policy 0, policy_version 69713 (0.0009) [2023-10-10 19:18:15,632][123582] Updated weights for policy 0, policy_version 69723 (0.0009) [2023-10-10 19:18:18,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 142704640. Throughput: 0: 1807.1, 1: 1817.8. Samples: 35689116. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:18:18,789][122664] Avg episode reward: [(0, '73.130'), (1, '87.760')] [2023-10-10 19:18:18,846][123614] Updated weights for policy 1, policy_version 69640 (0.0008) [2023-10-10 19:18:19,212][123614] Updated weights for policy 1, policy_version 69650 (0.0007) [2023-10-10 19:18:19,379][123582] Updated weights for policy 0, policy_version 69733 (0.0009) [2023-10-10 19:18:19,580][123614] Updated weights for policy 1, policy_version 69660 (0.0008) [2023-10-10 19:18:19,724][123465] Saving new best policy, reward=87.760! [2023-10-10 19:18:19,737][123582] Updated weights for policy 0, policy_version 69743 (0.0008) [2023-10-10 19:18:20,108][123582] Updated weights for policy 0, policy_version 69753 (0.0011) [2023-10-10 19:18:23,322][123614] Updated weights for policy 1, policy_version 69670 (0.0008) [2023-10-10 19:18:23,689][123614] Updated weights for policy 1, policy_version 69680 (0.0009) [2023-10-10 19:18:23,754][123582] Updated weights for policy 0, policy_version 69763 (0.0007) [2023-10-10 19:18:23,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 142770176. Throughput: 0: 1804.9, 1: 1819.9. Samples: 35710980. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:18:23,789][122664] Avg episode reward: [(0, '73.920'), (1, '87.400')] [2023-10-10 19:18:24,056][123614] Updated weights for policy 1, policy_version 69690 (0.0008) [2023-10-10 19:18:24,124][123582] Updated weights for policy 0, policy_version 69773 (0.0007) [2023-10-10 19:18:24,489][123582] Updated weights for policy 0, policy_version 69783 (0.0007) [2023-10-10 19:18:27,671][123614] Updated weights for policy 1, policy_version 69700 (0.0008) [2023-10-10 19:18:28,033][123614] Updated weights for policy 1, policy_version 69710 (0.0009) [2023-10-10 19:18:28,139][123582] Updated weights for policy 0, policy_version 69793 (0.0009) [2023-10-10 19:18:28,405][123614] Updated weights for policy 1, policy_version 69720 (0.0007) [2023-10-10 19:18:28,504][123582] Updated weights for policy 0, policy_version 69803 (0.0010) [2023-10-10 19:18:28,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 142868480. Throughput: 0: 1803.1, 1: 1809.5. Samples: 35721640. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:18:28,789][122664] Avg episode reward: [(0, '73.500'), (1, '91.660')] [2023-10-10 19:18:28,790][123465] Saving new best policy, reward=91.660! [2023-10-10 19:18:28,886][123582] Updated weights for policy 0, policy_version 69813 (0.0010) [2023-10-10 19:18:29,253][123582] Updated weights for policy 0, policy_version 69823 (0.0008) [2023-10-10 19:18:32,086][123614] Updated weights for policy 1, policy_version 69730 (0.0009) [2023-10-10 19:18:32,452][123614] Updated weights for policy 1, policy_version 69740 (0.0009) [2023-10-10 19:18:32,816][123614] Updated weights for policy 1, policy_version 69750 (0.0009) [2023-10-10 19:18:33,109][123582] Updated weights for policy 0, policy_version 69833 (0.0009) [2023-10-10 19:18:33,187][123614] Updated weights for policy 1, policy_version 69760 (0.0008) [2023-10-10 19:18:33,484][123582] Updated weights for policy 0, policy_version 69843 (0.0008) [2023-10-10 19:18:33,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 142934016. Throughput: 0: 1799.9, 1: 1810.7. Samples: 35743242. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:18:33,789][122664] Avg episode reward: [(0, '70.560'), (1, '83.350')] [2023-10-10 19:18:33,847][123582] Updated weights for policy 0, policy_version 69853 (0.0010) [2023-10-10 19:18:36,881][123614] Updated weights for policy 1, policy_version 69770 (0.0008) [2023-10-10 19:18:37,241][123614] Updated weights for policy 1, policy_version 69780 (0.0008) [2023-10-10 19:18:37,463][123582] Updated weights for policy 0, policy_version 69863 (0.0009) [2023-10-10 19:18:37,614][123614] Updated weights for policy 1, policy_version 69790 (0.0007) [2023-10-10 19:18:37,832][123582] Updated weights for policy 0, policy_version 69873 (0.0009) [2023-10-10 19:18:38,210][123582] Updated weights for policy 0, policy_version 69883 (0.0008) [2023-10-10 19:18:38,788][122664] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14551.2). Total num frames: 143032320. Throughput: 0: 1801.6, 1: 1803.2. Samples: 35763904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:18:38,789][122664] Avg episode reward: [(0, '69.700'), (1, '85.920')] [2023-10-10 19:18:41,261][123614] Updated weights for policy 1, policy_version 69800 (0.0009) [2023-10-10 19:18:41,631][123614] Updated weights for policy 1, policy_version 69810 (0.0010) [2023-10-10 19:18:41,957][123582] Updated weights for policy 0, policy_version 69893 (0.0009) [2023-10-10 19:18:41,995][123614] Updated weights for policy 1, policy_version 69820 (0.0007) [2023-10-10 19:18:42,323][123582] Updated weights for policy 0, policy_version 69903 (0.0007) [2023-10-10 19:18:42,693][123582] Updated weights for policy 0, policy_version 69913 (0.0009) [2023-10-10 19:18:43,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 143097856. Throughput: 0: 1796.3, 1: 1812.2. Samples: 35775828. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:18:43,789][122664] Avg episode reward: [(0, '73.400'), (1, '82.260')] [2023-10-10 19:18:45,657][123614] Updated weights for policy 1, policy_version 69830 (0.0010) [2023-10-10 19:18:46,028][123614] Updated weights for policy 1, policy_version 69840 (0.0008) [2023-10-10 19:18:46,385][123614] Updated weights for policy 1, policy_version 69850 (0.0008) [2023-10-10 19:18:46,413][123582] Updated weights for policy 0, policy_version 69923 (0.0010) [2023-10-10 19:18:46,775][123582] Updated weights for policy 0, policy_version 69933 (0.0009) [2023-10-10 19:18:47,147][123582] Updated weights for policy 0, policy_version 69943 (0.0011) [2023-10-10 19:18:48,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 143163392. Throughput: 0: 1797.4, 1: 1808.6. Samples: 35796734. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:18:48,788][122664] Avg episode reward: [(0, '75.010'), (1, '78.280')] [2023-10-10 19:18:49,927][123614] Updated weights for policy 1, policy_version 69860 (0.0007) [2023-10-10 19:18:50,287][123614] Updated weights for policy 1, policy_version 69870 (0.0009) [2023-10-10 19:18:50,667][123614] Updated weights for policy 1, policy_version 69880 (0.0008) [2023-10-10 19:18:50,947][123582] Updated weights for policy 0, policy_version 69953 (0.0010) [2023-10-10 19:18:51,371][123582] Updated weights for policy 0, policy_version 69963 (0.0009) [2023-10-10 19:18:51,731][123582] Updated weights for policy 0, policy_version 69973 (0.0010) [2023-10-10 19:18:52,097][123582] Updated weights for policy 0, policy_version 69983 (0.0008) [2023-10-10 19:18:53,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 143228928. Throughput: 0: 1800.2, 1: 1812.0. Samples: 35819544. Policy #0 lag: (min: 28.0, avg: 32.0, max: 60.0) [2023-10-10 19:18:53,788][122664] Avg episode reward: [(0, '77.230'), (1, '76.970')] [2023-10-10 19:18:54,447][123614] Updated weights for policy 1, policy_version 69890 (0.0008) [2023-10-10 19:18:54,822][123614] Updated weights for policy 1, policy_version 69900 (0.0009) [2023-10-10 19:18:55,211][123614] Updated weights for policy 1, policy_version 69910 (0.0009) [2023-10-10 19:18:55,574][123614] Updated weights for policy 1, policy_version 69920 (0.0007) [2023-10-10 19:18:55,710][123582] Updated weights for policy 0, policy_version 69993 (0.0008) [2023-10-10 19:18:56,089][123582] Updated weights for policy 0, policy_version 70003 (0.0010) [2023-10-10 19:18:56,459][123582] Updated weights for policy 0, policy_version 70013 (0.0007) [2023-10-10 19:18:58,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 143294464. Throughput: 0: 1806.3, 1: 1810.3. Samples: 35829550. Policy #0 lag: (min: 28.0, avg: 32.0, max: 60.0) [2023-10-10 19:18:58,789][122664] Avg episode reward: [(0, '75.820'), (1, '75.890')] [2023-10-10 19:18:59,354][123614] Updated weights for policy 1, policy_version 69930 (0.0008) [2023-10-10 19:18:59,719][123614] Updated weights for policy 1, policy_version 69940 (0.0008) [2023-10-10 19:19:00,083][123614] Updated weights for policy 1, policy_version 69950 (0.0008) [2023-10-10 19:19:00,108][123582] Updated weights for policy 0, policy_version 70023 (0.0008) [2023-10-10 19:19:00,477][123582] Updated weights for policy 0, policy_version 70033 (0.0009) [2023-10-10 19:19:00,841][123582] Updated weights for policy 0, policy_version 70043 (0.0011) [2023-10-10 19:19:03,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 143360000. Throughput: 0: 1810.1, 1: 1810.4. Samples: 35852040. Policy #0 lag: (min: 28.0, avg: 32.0, max: 60.0) [2023-10-10 19:19:03,789][122664] Avg episode reward: [(0, '73.610'), (1, '85.760')] [2023-10-10 19:19:04,058][123614] Updated weights for policy 1, policy_version 69960 (0.0007) [2023-10-10 19:19:04,402][123582] Updated weights for policy 0, policy_version 70053 (0.0011) [2023-10-10 19:19:04,423][123614] Updated weights for policy 1, policy_version 69970 (0.0008) [2023-10-10 19:19:04,776][123582] Updated weights for policy 0, policy_version 70063 (0.0008) [2023-10-10 19:19:04,792][123614] Updated weights for policy 1, policy_version 69980 (0.0008) [2023-10-10 19:19:05,161][123582] Updated weights for policy 0, policy_version 70073 (0.0008) [2023-10-10 19:19:08,553][123614] Updated weights for policy 1, policy_version 69990 (0.0008) [2023-10-10 19:19:08,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 143425536. Throughput: 0: 1815.1, 1: 1816.0. Samples: 35874380. Policy #0 lag: (min: 28.0, avg: 32.0, max: 60.0) [2023-10-10 19:19:08,789][122664] Avg episode reward: [(0, '74.840'), (1, '84.490')] [2023-10-10 19:19:08,883][123582] Updated weights for policy 0, policy_version 70083 (0.0008) [2023-10-10 19:19:08,922][123614] Updated weights for policy 1, policy_version 70000 (0.0009) [2023-10-10 19:19:09,252][123582] Updated weights for policy 0, policy_version 70093 (0.0007) [2023-10-10 19:19:09,295][123614] Updated weights for policy 1, policy_version 70010 (0.0008) [2023-10-10 19:19:09,625][123582] Updated weights for policy 0, policy_version 70103 (0.0008) [2023-10-10 19:19:12,955][123614] Updated weights for policy 1, policy_version 70020 (0.0008) [2023-10-10 19:19:13,320][123614] Updated weights for policy 1, policy_version 70030 (0.0007) [2023-10-10 19:19:13,448][123582] Updated weights for policy 0, policy_version 70113 (0.0009) [2023-10-10 19:19:13,684][123614] Updated weights for policy 1, policy_version 70040 (0.0007) [2023-10-10 19:19:13,788][122664] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 143491072. Throughput: 0: 1816.0, 1: 1808.0. Samples: 35884716. Policy #0 lag: (min: 28.0, avg: 32.0, max: 60.0) [2023-10-10 19:19:13,788][122664] Avg episode reward: [(0, '77.420'), (1, '78.290')] [2023-10-10 19:19:13,817][123582] Updated weights for policy 0, policy_version 70123 (0.0007) [2023-10-10 19:19:14,182][123582] Updated weights for policy 0, policy_version 70133 (0.0007) [2023-10-10 19:19:14,551][123582] Updated weights for policy 0, policy_version 70143 (0.0008) [2023-10-10 19:19:17,373][123614] Updated weights for policy 1, policy_version 70050 (0.0007) [2023-10-10 19:19:17,744][123614] Updated weights for policy 1, policy_version 70060 (0.0008) [2023-10-10 19:19:18,117][123614] Updated weights for policy 1, policy_version 70070 (0.0007) [2023-10-10 19:19:18,407][123582] Updated weights for policy 0, policy_version 70153 (0.0008) [2023-10-10 19:19:18,480][123614] Updated weights for policy 1, policy_version 70080 (0.0009) [2023-10-10 19:19:18,784][123582] Updated weights for policy 0, policy_version 70163 (0.0009) [2023-10-10 19:19:18,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 143589376. Throughput: 0: 1812.1, 1: 1824.3. Samples: 35906878. Policy #0 lag: (min: 28.0, avg: 32.0, max: 60.0) [2023-10-10 19:19:18,789][122664] Avg episode reward: [(0, '73.110'), (1, '79.200')] [2023-10-10 19:19:19,162][123582] Updated weights for policy 0, policy_version 70173 (0.0010) [2023-10-10 19:19:22,328][123614] Updated weights for policy 1, policy_version 70090 (0.0009) [2023-10-10 19:19:22,692][123614] Updated weights for policy 1, policy_version 70100 (0.0009) [2023-10-10 19:19:22,862][123582] Updated weights for policy 0, policy_version 70183 (0.0010) [2023-10-10 19:19:23,063][123614] Updated weights for policy 1, policy_version 70110 (0.0007) [2023-10-10 19:19:23,239][123582] Updated weights for policy 0, policy_version 70193 (0.0010) [2023-10-10 19:19:23,615][123582] Updated weights for policy 0, policy_version 70203 (0.0009) [2023-10-10 19:19:23,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 143654912. Throughput: 0: 1819.3, 1: 1809.0. Samples: 35927178. Policy #0 lag: (min: 28.0, avg: 32.0, max: 60.0) [2023-10-10 19:19:23,789][122664] Avg episode reward: [(0, '72.620'), (1, '78.330')] [2023-10-10 19:19:26,902][123614] Updated weights for policy 1, policy_version 70120 (0.0008) [2023-10-10 19:19:27,270][123614] Updated weights for policy 1, policy_version 70130 (0.0008) [2023-10-10 19:19:27,375][123582] Updated weights for policy 0, policy_version 70213 (0.0009) [2023-10-10 19:19:27,639][123614] Updated weights for policy 1, policy_version 70140 (0.0008) [2023-10-10 19:19:27,758][123582] Updated weights for policy 0, policy_version 70223 (0.0008) [2023-10-10 19:19:28,122][123582] Updated weights for policy 0, policy_version 70233 (0.0008) [2023-10-10 19:19:28,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 143753216. Throughput: 0: 1808.1, 1: 1822.9. Samples: 35939222. Policy #0 lag: (min: 28.0, avg: 32.0, max: 60.0) [2023-10-10 19:19:28,789][122664] Avg episode reward: [(0, '73.810'), (1, '77.260')] [2023-10-10 19:19:31,273][123614] Updated weights for policy 1, policy_version 70150 (0.0009) [2023-10-10 19:19:31,633][123614] Updated weights for policy 1, policy_version 70160 (0.0009) [2023-10-10 19:19:31,757][123582] Updated weights for policy 0, policy_version 70243 (0.0009) [2023-10-10 19:19:31,994][123614] Updated weights for policy 1, policy_version 70170 (0.0007) [2023-10-10 19:19:32,127][123582] Updated weights for policy 0, policy_version 70253 (0.0007) [2023-10-10 19:19:32,506][123582] Updated weights for policy 0, policy_version 70263 (0.0011) [2023-10-10 19:19:33,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 143818752. Throughput: 0: 1818.5, 1: 1806.3. Samples: 35959852. Policy #0 lag: (min: 28.0, avg: 32.0, max: 60.0) [2023-10-10 19:19:33,789][122664] Avg episode reward: [(0, '80.310'), (1, '75.050')] [2023-10-10 19:19:35,511][123614] Updated weights for policy 1, policy_version 70180 (0.0008) [2023-10-10 19:19:35,883][123614] Updated weights for policy 1, policy_version 70190 (0.0007) [2023-10-10 19:19:36,201][123582] Updated weights for policy 0, policy_version 70273 (0.0009) [2023-10-10 19:19:36,249][123614] Updated weights for policy 1, policy_version 70200 (0.0008) [2023-10-10 19:19:36,609][123582] Updated weights for policy 0, policy_version 70283 (0.0008) [2023-10-10 19:19:36,977][123582] Updated weights for policy 0, policy_version 70293 (0.0008) [2023-10-10 19:19:37,344][123582] Updated weights for policy 0, policy_version 70303 (0.0008) [2023-10-10 19:19:38,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 143884288. Throughput: 0: 1807.2, 1: 1809.8. Samples: 35982308. Policy #0 lag: (min: 31.0, avg: 46.1, max: 63.0) [2023-10-10 19:19:38,789][122664] Avg episode reward: [(0, '76.360'), (1, '69.700')] [2023-10-10 19:19:39,957][123614] Updated weights for policy 1, policy_version 70210 (0.0010) [2023-10-10 19:19:40,321][123614] Updated weights for policy 1, policy_version 70220 (0.0010) [2023-10-10 19:19:40,693][123614] Updated weights for policy 1, policy_version 70230 (0.0011) [2023-10-10 19:19:41,053][123582] Updated weights for policy 0, policy_version 70313 (0.0008) [2023-10-10 19:19:41,061][123614] Updated weights for policy 1, policy_version 70240 (0.0008) [2023-10-10 19:19:41,414][123582] Updated weights for policy 0, policy_version 70323 (0.0011) [2023-10-10 19:19:41,787][123582] Updated weights for policy 0, policy_version 70333 (0.0011) [2023-10-10 19:19:43,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 143949824. Throughput: 0: 1818.3, 1: 1808.7. Samples: 35992764. Policy #0 lag: (min: 31.0, avg: 46.1, max: 63.0) [2023-10-10 19:19:43,789][122664] Avg episode reward: [(0, '76.610'), (1, '67.820')] [2023-10-10 19:19:44,796][123614] Updated weights for policy 1, policy_version 70250 (0.0009) [2023-10-10 19:19:45,168][123614] Updated weights for policy 1, policy_version 70260 (0.0011) [2023-10-10 19:19:45,528][123614] Updated weights for policy 1, policy_version 70270 (0.0008) [2023-10-10 19:19:45,571][123582] Updated weights for policy 0, policy_version 70343 (0.0008) [2023-10-10 19:19:45,936][123582] Updated weights for policy 0, policy_version 70353 (0.0010) [2023-10-10 19:19:46,300][123582] Updated weights for policy 0, policy_version 70363 (0.0008) [2023-10-10 19:19:48,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 144015360. Throughput: 0: 1803.0, 1: 1813.9. Samples: 36014802. Policy #0 lag: (min: 31.0, avg: 46.1, max: 63.0) [2023-10-10 19:19:48,789][122664] Avg episode reward: [(0, '76.350'), (1, '72.010')] [2023-10-10 19:19:49,172][123614] Updated weights for policy 1, policy_version 70280 (0.0009) [2023-10-10 19:19:49,535][123614] Updated weights for policy 1, policy_version 70290 (0.0009) [2023-10-10 19:19:49,900][123614] Updated weights for policy 1, policy_version 70300 (0.0009) [2023-10-10 19:19:49,966][123582] Updated weights for policy 0, policy_version 70373 (0.0008) [2023-10-10 19:19:50,345][123582] Updated weights for policy 0, policy_version 70383 (0.0008) [2023-10-10 19:19:50,717][123582] Updated weights for policy 0, policy_version 70393 (0.0009) [2023-10-10 19:19:53,688][123614] Updated weights for policy 1, policy_version 70310 (0.0009) [2023-10-10 19:19:53,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 144080896. Throughput: 0: 1799.2, 1: 1816.6. Samples: 36037092. Policy #0 lag: (min: 31.0, avg: 46.1, max: 63.0) [2023-10-10 19:19:53,789][122664] Avg episode reward: [(0, '79.180'), (1, '71.160')] [2023-10-10 19:19:53,802][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000070400_72089600.pth... [2023-10-10 19:19:53,835][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000068704_70352896.pth [2023-10-10 19:19:54,056][123614] Updated weights for policy 1, policy_version 70320 (0.0007) [2023-10-10 19:19:54,391][123582] Updated weights for policy 0, policy_version 70403 (0.0010) [2023-10-10 19:19:54,428][123614] Updated weights for policy 1, policy_version 70330 (0.0007) [2023-10-10 19:19:54,641][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000070336_72024064.pth... [2023-10-10 19:19:54,672][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000068640_70287360.pth [2023-10-10 19:19:54,759][123582] Updated weights for policy 0, policy_version 70413 (0.0008) [2023-10-10 19:19:55,127][123582] Updated weights for policy 0, policy_version 70423 (0.0007) [2023-10-10 19:19:58,161][123614] Updated weights for policy 1, policy_version 70340 (0.0008) [2023-10-10 19:19:58,526][123614] Updated weights for policy 1, policy_version 70350 (0.0007) [2023-10-10 19:19:58,774][123582] Updated weights for policy 0, policy_version 70433 (0.0010) [2023-10-10 19:19:58,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 144146432. Throughput: 0: 1802.3, 1: 1814.4. Samples: 36047468. Policy #0 lag: (min: 31.0, avg: 46.1, max: 63.0) [2023-10-10 19:19:58,788][122664] Avg episode reward: [(0, '76.740'), (1, '74.900')] [2023-10-10 19:19:58,892][123614] Updated weights for policy 1, policy_version 70360 (0.0008) [2023-10-10 19:19:59,142][123582] Updated weights for policy 0, policy_version 70443 (0.0008) [2023-10-10 19:19:59,510][123582] Updated weights for policy 0, policy_version 70453 (0.0009) [2023-10-10 19:19:59,880][123582] Updated weights for policy 0, policy_version 70463 (0.0010) [2023-10-10 19:20:02,685][123614] Updated weights for policy 1, policy_version 70370 (0.0007) [2023-10-10 19:20:03,057][123614] Updated weights for policy 1, policy_version 70380 (0.0008) [2023-10-10 19:20:03,427][123614] Updated weights for policy 1, policy_version 70390 (0.0008) [2023-10-10 19:20:03,548][123582] Updated weights for policy 0, policy_version 70473 (0.0007) [2023-10-10 19:20:03,787][123614] Updated weights for policy 1, policy_version 70400 (0.0007) [2023-10-10 19:20:03,788][122664] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 144244736. Throughput: 0: 1807.8, 1: 1816.2. Samples: 36069958. Policy #0 lag: (min: 31.0, avg: 46.1, max: 63.0) [2023-10-10 19:20:03,789][122664] Avg episode reward: [(0, '76.290'), (1, '79.530')] [2023-10-10 19:20:03,923][123582] Updated weights for policy 0, policy_version 70483 (0.0007) [2023-10-10 19:20:04,294][123582] Updated weights for policy 0, policy_version 70493 (0.0009) [2023-10-10 19:20:07,503][123614] Updated weights for policy 1, policy_version 70410 (0.0008) [2023-10-10 19:20:07,866][123614] Updated weights for policy 1, policy_version 70420 (0.0009) [2023-10-10 19:20:08,084][123582] Updated weights for policy 0, policy_version 70503 (0.0010) [2023-10-10 19:20:08,229][123614] Updated weights for policy 1, policy_version 70430 (0.0008) [2023-10-10 19:20:08,453][123582] Updated weights for policy 0, policy_version 70513 (0.0009) [2023-10-10 19:20:08,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 144310272. Throughput: 0: 1815.3, 1: 1812.0. Samples: 36090406. Policy #0 lag: (min: 31.0, avg: 46.1, max: 63.0) [2023-10-10 19:20:08,789][122664] Avg episode reward: [(0, '79.000'), (1, '78.460')] [2023-10-10 19:20:08,832][123582] Updated weights for policy 0, policy_version 70523 (0.0009) [2023-10-10 19:20:11,916][123614] Updated weights for policy 1, policy_version 70440 (0.0009) [2023-10-10 19:20:12,287][123614] Updated weights for policy 1, policy_version 70450 (0.0009) [2023-10-10 19:20:12,516][123582] Updated weights for policy 0, policy_version 70533 (0.0009) [2023-10-10 19:20:12,661][123614] Updated weights for policy 1, policy_version 70460 (0.0009) [2023-10-10 19:20:12,885][123582] Updated weights for policy 0, policy_version 70543 (0.0009) [2023-10-10 19:20:13,255][123582] Updated weights for policy 0, policy_version 70553 (0.0009) [2023-10-10 19:20:13,788][122664] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 144408576. Throughput: 0: 1811.0, 1: 1813.7. Samples: 36102334. Policy #0 lag: (min: 31.0, avg: 46.1, max: 63.0) [2023-10-10 19:20:13,789][122664] Avg episode reward: [(0, '80.220'), (1, '74.210')] [2023-10-10 19:20:16,264][123614] Updated weights for policy 1, policy_version 70470 (0.0008) [2023-10-10 19:20:16,634][123614] Updated weights for policy 1, policy_version 70480 (0.0008) [2023-10-10 19:20:16,958][123582] Updated weights for policy 0, policy_version 70563 (0.0009) [2023-10-10 19:20:16,998][123614] Updated weights for policy 1, policy_version 70490 (0.0008) [2023-10-10 19:20:17,330][123582] Updated weights for policy 0, policy_version 70573 (0.0008) [2023-10-10 19:20:17,696][123582] Updated weights for policy 0, policy_version 70583 (0.0007) [2023-10-10 19:20:18,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 144474112. Throughput: 0: 1817.5, 1: 1813.7. Samples: 36123254. Policy #0 lag: (min: 31.0, avg: 46.1, max: 63.0) [2023-10-10 19:20:18,789][122664] Avg episode reward: [(0, '75.930'), (1, '71.900')] [2023-10-10 19:20:20,695][123614] Updated weights for policy 1, policy_version 70500 (0.0008) [2023-10-10 19:20:21,062][123614] Updated weights for policy 1, policy_version 70510 (0.0011) [2023-10-10 19:20:21,418][123582] Updated weights for policy 0, policy_version 70593 (0.0008) [2023-10-10 19:20:21,426][123614] Updated weights for policy 1, policy_version 70520 (0.0007) [2023-10-10 19:20:21,824][123582] Updated weights for policy 0, policy_version 70603 (0.0009) [2023-10-10 19:20:22,194][123582] Updated weights for policy 0, policy_version 70613 (0.0011) [2023-10-10 19:20:22,562][123582] Updated weights for policy 0, policy_version 70623 (0.0010) [2023-10-10 19:20:23,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 144539648. Throughput: 0: 1809.2, 1: 1810.2. Samples: 36145180. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:20:23,789][122664] Avg episode reward: [(0, '78.890'), (1, '72.250')] [2023-10-10 19:20:25,175][123614] Updated weights for policy 1, policy_version 70530 (0.0007) [2023-10-10 19:20:25,541][123614] Updated weights for policy 1, policy_version 70540 (0.0009) [2023-10-10 19:20:25,915][123614] Updated weights for policy 1, policy_version 70550 (0.0008) [2023-10-10 19:20:26,196][123582] Updated weights for policy 0, policy_version 70633 (0.0008) [2023-10-10 19:20:26,285][123614] Updated weights for policy 1, policy_version 70560 (0.0008) [2023-10-10 19:20:26,582][123582] Updated weights for policy 0, policy_version 70643 (0.0009) [2023-10-10 19:20:26,949][123582] Updated weights for policy 0, policy_version 70653 (0.0011) [2023-10-10 19:20:28,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 144605184. Throughput: 0: 1814.2, 1: 1812.3. Samples: 36155956. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:20:28,789][122664] Avg episode reward: [(0, '79.410'), (1, '73.660')] [2023-10-10 19:20:29,960][123614] Updated weights for policy 1, policy_version 70570 (0.0007) [2023-10-10 19:20:30,329][123614] Updated weights for policy 1, policy_version 70580 (0.0009) [2023-10-10 19:20:30,599][123582] Updated weights for policy 0, policy_version 70663 (0.0007) [2023-10-10 19:20:30,688][123614] Updated weights for policy 1, policy_version 70590 (0.0009) [2023-10-10 19:20:30,979][123582] Updated weights for policy 0, policy_version 70673 (0.0009) [2023-10-10 19:20:31,347][123582] Updated weights for policy 0, policy_version 70683 (0.0007) [2023-10-10 19:20:33,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 144670720. Throughput: 0: 1817.0, 1: 1813.2. Samples: 36178162. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:20:33,789][122664] Avg episode reward: [(0, '76.370'), (1, '75.730')] [2023-10-10 19:20:34,304][123614] Updated weights for policy 1, policy_version 70600 (0.0009) [2023-10-10 19:20:34,676][123614] Updated weights for policy 1, policy_version 70610 (0.0008) [2023-10-10 19:20:34,925][123582] Updated weights for policy 0, policy_version 70693 (0.0011) [2023-10-10 19:20:35,043][123614] Updated weights for policy 1, policy_version 70620 (0.0007) [2023-10-10 19:20:35,284][123582] Updated weights for policy 0, policy_version 70703 (0.0007) [2023-10-10 19:20:35,666][123582] Updated weights for policy 0, policy_version 70713 (0.0008) [2023-10-10 19:20:38,778][123614] Updated weights for policy 1, policy_version 70630 (0.0008) [2023-10-10 19:20:38,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 144736256. Throughput: 0: 1818.7, 1: 1823.3. Samples: 36200984. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:20:38,788][122664] Avg episode reward: [(0, '76.610'), (1, '74.810')] [2023-10-10 19:20:39,146][123614] Updated weights for policy 1, policy_version 70640 (0.0008) [2023-10-10 19:20:39,425][123582] Updated weights for policy 0, policy_version 70723 (0.0007) [2023-10-10 19:20:39,510][123614] Updated weights for policy 1, policy_version 70650 (0.0007) [2023-10-10 19:20:39,794][123582] Updated weights for policy 0, policy_version 70733 (0.0009) [2023-10-10 19:20:40,156][123582] Updated weights for policy 0, policy_version 70743 (0.0009) [2023-10-10 19:20:43,089][123614] Updated weights for policy 1, policy_version 70660 (0.0009) [2023-10-10 19:20:43,455][123614] Updated weights for policy 1, policy_version 70670 (0.0009) [2023-10-10 19:20:43,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 144801792. Throughput: 0: 1816.4, 1: 1819.2. Samples: 36211070. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:20:43,789][122664] Avg episode reward: [(0, '84.410'), (1, '76.970')] [2023-10-10 19:20:43,824][123614] Updated weights for policy 1, policy_version 70680 (0.0007) [2023-10-10 19:20:43,951][123582] Updated weights for policy 0, policy_version 70753 (0.0008) [2023-10-10 19:20:44,320][123582] Updated weights for policy 0, policy_version 70763 (0.0010) [2023-10-10 19:20:44,679][123582] Updated weights for policy 0, policy_version 70773 (0.0009) [2023-10-10 19:20:45,056][123582] Updated weights for policy 0, policy_version 70783 (0.0007) [2023-10-10 19:20:47,534][123614] Updated weights for policy 1, policy_version 70690 (0.0007) [2023-10-10 19:20:47,903][123614] Updated weights for policy 1, policy_version 70700 (0.0008) [2023-10-10 19:20:48,280][123614] Updated weights for policy 1, policy_version 70710 (0.0010) [2023-10-10 19:20:48,648][123614] Updated weights for policy 1, policy_version 70720 (0.0009) [2023-10-10 19:20:48,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 144900096. Throughput: 0: 1814.1, 1: 1816.0. Samples: 36233312. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:20:48,789][122664] Avg episode reward: [(0, '80.290'), (1, '80.710')] [2023-10-10 19:20:48,796][123582] Updated weights for policy 0, policy_version 70793 (0.0007) [2023-10-10 19:20:49,167][123582] Updated weights for policy 0, policy_version 70803 (0.0007) [2023-10-10 19:20:49,533][123582] Updated weights for policy 0, policy_version 70813 (0.0007) [2023-10-10 19:20:52,352][123614] Updated weights for policy 1, policy_version 70730 (0.0007) [2023-10-10 19:20:52,719][123614] Updated weights for policy 1, policy_version 70740 (0.0008) [2023-10-10 19:20:53,087][123614] Updated weights for policy 1, policy_version 70750 (0.0010) [2023-10-10 19:20:53,187][123582] Updated weights for policy 0, policy_version 70823 (0.0007) [2023-10-10 19:20:53,560][123582] Updated weights for policy 0, policy_version 70833 (0.0009) [2023-10-10 19:20:53,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 144965632. Throughput: 0: 1820.9, 1: 1820.9. Samples: 36254286. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:20:53,789][122664] Avg episode reward: [(0, '82.210'), (1, '83.510')] [2023-10-10 19:20:53,936][123582] Updated weights for policy 0, policy_version 70843 (0.0007) [2023-10-10 19:20:56,716][123614] Updated weights for policy 1, policy_version 70760 (0.0007) [2023-10-10 19:20:57,091][123614] Updated weights for policy 1, policy_version 70770 (0.0008) [2023-10-10 19:20:57,458][123614] Updated weights for policy 1, policy_version 70780 (0.0009) [2023-10-10 19:20:57,584][123582] Updated weights for policy 0, policy_version 70853 (0.0009) [2023-10-10 19:20:57,953][123582] Updated weights for policy 0, policy_version 70863 (0.0011) [2023-10-10 19:20:58,331][123582] Updated weights for policy 0, policy_version 70873 (0.0008) [2023-10-10 19:20:58,788][122664] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 145063936. Throughput: 0: 1817.2, 1: 1815.3. Samples: 36265800. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:20:58,789][122664] Avg episode reward: [(0, '74.720'), (1, '83.340')] [2023-10-10 19:21:01,049][123614] Updated weights for policy 1, policy_version 70790 (0.0008) [2023-10-10 19:21:01,417][123614] Updated weights for policy 1, policy_version 70800 (0.0009) [2023-10-10 19:21:01,785][123614] Updated weights for policy 1, policy_version 70810 (0.0008) [2023-10-10 19:21:02,060][123582] Updated weights for policy 0, policy_version 70883 (0.0010) [2023-10-10 19:21:02,441][123582] Updated weights for policy 0, policy_version 70893 (0.0007) [2023-10-10 19:21:02,813][123582] Updated weights for policy 0, policy_version 70903 (0.0009) [2023-10-10 19:21:03,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 145129472. Throughput: 0: 1817.6, 1: 1822.5. Samples: 36287062. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:21:03,789][122664] Avg episode reward: [(0, '74.650'), (1, '83.890')] [2023-10-10 19:21:05,572][123614] Updated weights for policy 1, policy_version 70820 (0.0009) [2023-10-10 19:21:05,930][123614] Updated weights for policy 1, policy_version 70830 (0.0008) [2023-10-10 19:21:06,302][123614] Updated weights for policy 1, policy_version 70840 (0.0007) [2023-10-10 19:21:06,552][123582] Updated weights for policy 0, policy_version 70913 (0.0010) [2023-10-10 19:21:06,964][123582] Updated weights for policy 0, policy_version 70923 (0.0010) [2023-10-10 19:21:07,342][123582] Updated weights for policy 0, policy_version 70933 (0.0008) [2023-10-10 19:21:07,719][123582] Updated weights for policy 0, policy_version 70943 (0.0009) [2023-10-10 19:21:08,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 145195008. Throughput: 0: 1814.4, 1: 1817.2. Samples: 36308602. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:21:08,788][122664] Avg episode reward: [(0, '72.100'), (1, '81.140')] [2023-10-10 19:21:10,069][123614] Updated weights for policy 1, policy_version 70850 (0.0008) [2023-10-10 19:21:10,445][123614] Updated weights for policy 1, policy_version 70860 (0.0008) [2023-10-10 19:21:10,804][123614] Updated weights for policy 1, policy_version 70870 (0.0008) [2023-10-10 19:21:11,175][123614] Updated weights for policy 1, policy_version 70880 (0.0007) [2023-10-10 19:21:11,343][123582] Updated weights for policy 0, policy_version 70953 (0.0009) [2023-10-10 19:21:11,703][123582] Updated weights for policy 0, policy_version 70963 (0.0008) [2023-10-10 19:21:12,068][123582] Updated weights for policy 0, policy_version 70973 (0.0009) [2023-10-10 19:21:13,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 145260544. Throughput: 0: 1817.6, 1: 1815.7. Samples: 36319454. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:21:13,789][122664] Avg episode reward: [(0, '71.400'), (1, '81.970')] [2023-10-10 19:21:14,809][123614] Updated weights for policy 1, policy_version 70890 (0.0008) [2023-10-10 19:21:15,178][123614] Updated weights for policy 1, policy_version 70900 (0.0007) [2023-10-10 19:21:15,547][123614] Updated weights for policy 1, policy_version 70910 (0.0008) [2023-10-10 19:21:15,770][123582] Updated weights for policy 0, policy_version 70983 (0.0008) [2023-10-10 19:21:16,142][123582] Updated weights for policy 0, policy_version 70993 (0.0008) [2023-10-10 19:21:16,523][123582] Updated weights for policy 0, policy_version 71003 (0.0010) [2023-10-10 19:21:18,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 145326080. Throughput: 0: 1808.9, 1: 1815.9. Samples: 36341278. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:21:18,789][122664] Avg episode reward: [(0, '72.310'), (1, '81.370')] [2023-10-10 19:21:19,235][123614] Updated weights for policy 1, policy_version 70920 (0.0008) [2023-10-10 19:21:19,598][123614] Updated weights for policy 1, policy_version 70930 (0.0009) [2023-10-10 19:21:19,965][123614] Updated weights for policy 1, policy_version 70940 (0.0008) [2023-10-10 19:21:20,046][123582] Updated weights for policy 0, policy_version 71013 (0.0008) [2023-10-10 19:21:20,407][123582] Updated weights for policy 0, policy_version 71023 (0.0008) [2023-10-10 19:21:20,782][123582] Updated weights for policy 0, policy_version 71033 (0.0008) [2023-10-10 19:21:23,764][123614] Updated weights for policy 1, policy_version 70950 (0.0009) [2023-10-10 19:21:23,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 145391616. Throughput: 0: 1821.0, 1: 1804.0. Samples: 36364108. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:21:23,788][122664] Avg episode reward: [(0, '72.430'), (1, '81.220')] [2023-10-10 19:21:24,131][123614] Updated weights for policy 1, policy_version 70960 (0.0008) [2023-10-10 19:21:24,303][123582] Updated weights for policy 0, policy_version 71043 (0.0008) [2023-10-10 19:21:24,494][123614] Updated weights for policy 1, policy_version 70970 (0.0007) [2023-10-10 19:21:24,681][123582] Updated weights for policy 0, policy_version 71053 (0.0008) [2023-10-10 19:21:25,060][123582] Updated weights for policy 0, policy_version 71063 (0.0010) [2023-10-10 19:21:28,316][123614] Updated weights for policy 1, policy_version 70980 (0.0007) [2023-10-10 19:21:28,685][123614] Updated weights for policy 1, policy_version 70990 (0.0009) [2023-10-10 19:21:28,717][123582] Updated weights for policy 0, policy_version 71073 (0.0008) [2023-10-10 19:21:28,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 145457152. Throughput: 0: 1823.8, 1: 1804.6. Samples: 36374348. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:21:28,788][122664] Avg episode reward: [(0, '74.110'), (1, '82.380')] [2023-10-10 19:21:29,049][123614] Updated weights for policy 1, policy_version 71000 (0.0008) [2023-10-10 19:21:29,093][123582] Updated weights for policy 0, policy_version 71083 (0.0009) [2023-10-10 19:21:29,467][123582] Updated weights for policy 0, policy_version 71093 (0.0009) [2023-10-10 19:21:29,843][123582] Updated weights for policy 0, policy_version 71103 (0.0008) [2023-10-10 19:21:32,884][123614] Updated weights for policy 1, policy_version 71010 (0.0008) [2023-10-10 19:21:33,245][123614] Updated weights for policy 1, policy_version 71020 (0.0009) [2023-10-10 19:21:33,598][123582] Updated weights for policy 0, policy_version 71113 (0.0007) [2023-10-10 19:21:33,618][123614] Updated weights for policy 1, policy_version 71030 (0.0009) [2023-10-10 19:21:33,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 145522688. Throughput: 0: 1828.4, 1: 1805.2. Samples: 36396824. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:21:33,789][122664] Avg episode reward: [(0, '71.380'), (1, '86.920')] [2023-10-10 19:21:33,980][123614] Updated weights for policy 1, policy_version 71040 (0.0009) [2023-10-10 19:21:33,980][123582] Updated weights for policy 0, policy_version 71123 (0.0007) [2023-10-10 19:21:34,341][123582] Updated weights for policy 0, policy_version 71133 (0.0009) [2023-10-10 19:21:37,787][123614] Updated weights for policy 1, policy_version 71050 (0.0009) [2023-10-10 19:21:38,149][123582] Updated weights for policy 0, policy_version 71143 (0.0009) [2023-10-10 19:21:38,153][123614] Updated weights for policy 1, policy_version 71060 (0.0009) [2023-10-10 19:21:38,525][123614] Updated weights for policy 1, policy_version 71070 (0.0008) [2023-10-10 19:21:38,529][123582] Updated weights for policy 0, policy_version 71153 (0.0008) [2023-10-10 19:21:38,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 145620992. Throughput: 0: 1821.1, 1: 1793.4. Samples: 36416936. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:21:38,789][122664] Avg episode reward: [(0, '69.930'), (1, '89.150')] [2023-10-10 19:21:38,906][123582] Updated weights for policy 0, policy_version 71163 (0.0009) [2023-10-10 19:21:42,306][123614] Updated weights for policy 1, policy_version 71080 (0.0008) [2023-10-10 19:21:42,563][123582] Updated weights for policy 0, policy_version 71173 (0.0007) [2023-10-10 19:21:42,669][123614] Updated weights for policy 1, policy_version 71090 (0.0008) [2023-10-10 19:21:42,937][123582] Updated weights for policy 0, policy_version 71183 (0.0007) [2023-10-10 19:21:43,038][123614] Updated weights for policy 1, policy_version 71100 (0.0008) [2023-10-10 19:21:43,317][123582] Updated weights for policy 0, policy_version 71193 (0.0009) [2023-10-10 19:21:43,788][122664] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 145719296. Throughput: 0: 1825.5, 1: 1801.5. Samples: 36429018. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:21:43,789][122664] Avg episode reward: [(0, '67.690'), (1, '92.140')] [2023-10-10 19:21:43,792][123465] Saving new best policy, reward=92.140! [2023-10-10 19:21:46,700][123614] Updated weights for policy 1, policy_version 71110 (0.0008) [2023-10-10 19:21:46,818][123582] Updated weights for policy 0, policy_version 71203 (0.0009) [2023-10-10 19:21:47,066][123614] Updated weights for policy 1, policy_version 71120 (0.0007) [2023-10-10 19:21:47,194][123582] Updated weights for policy 0, policy_version 71213 (0.0008) [2023-10-10 19:21:47,435][123614] Updated weights for policy 1, policy_version 71130 (0.0007) [2023-10-10 19:21:47,566][123582] Updated weights for policy 0, policy_version 71223 (0.0009) [2023-10-10 19:21:48,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 145784832. Throughput: 0: 1824.7, 1: 1790.4. Samples: 36449742. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:21:48,788][122664] Avg episode reward: [(0, '65.630'), (1, '91.130')] [2023-10-10 19:21:51,164][123582] Updated weights for policy 0, policy_version 71233 (0.0010) [2023-10-10 19:21:51,234][123614] Updated weights for policy 1, policy_version 71140 (0.0007) [2023-10-10 19:21:51,548][123582] Updated weights for policy 0, policy_version 71243 (0.0010) [2023-10-10 19:21:51,608][123614] Updated weights for policy 1, policy_version 71150 (0.0008) [2023-10-10 19:21:51,914][123582] Updated weights for policy 0, policy_version 71253 (0.0007) [2023-10-10 19:21:51,976][123614] Updated weights for policy 1, policy_version 71160 (0.0009) [2023-10-10 19:21:52,282][123582] Updated weights for policy 0, policy_version 71263 (0.0007) [2023-10-10 19:21:53,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 145850368. Throughput: 0: 1835.7, 1: 1792.7. Samples: 36471880. Policy #0 lag: (min: 6.0, avg: 12.9, max: 38.0) [2023-10-10 19:21:53,789][122664] Avg episode reward: [(0, '64.400'), (1, '96.490')] [2023-10-10 19:21:53,801][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000071168_72876032.pth... [2023-10-10 19:21:53,802][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000071264_72974336.pth... [2023-10-10 19:21:53,835][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000069472_71139328.pth [2023-10-10 19:21:53,837][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000069568_71237632.pth [2023-10-10 19:21:53,838][123465] Saving new best policy, reward=96.490! [2023-10-10 19:21:55,704][123614] Updated weights for policy 1, policy_version 71170 (0.0007) [2023-10-10 19:21:56,069][123614] Updated weights for policy 1, policy_version 71180 (0.0008) [2023-10-10 19:21:56,144][123582] Updated weights for policy 0, policy_version 71273 (0.0007) [2023-10-10 19:21:56,437][123614] Updated weights for policy 1, policy_version 71190 (0.0010) [2023-10-10 19:21:56,523][123582] Updated weights for policy 0, policy_version 71283 (0.0007) [2023-10-10 19:21:56,807][123614] Updated weights for policy 1, policy_version 71200 (0.0008) [2023-10-10 19:21:56,885][123582] Updated weights for policy 0, policy_version 71293 (0.0008) [2023-10-10 19:21:58,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 145915904. Throughput: 0: 1824.6, 1: 1796.9. Samples: 36482424. Policy #0 lag: (min: 6.0, avg: 12.9, max: 38.0) [2023-10-10 19:21:58,789][122664] Avg episode reward: [(0, '63.640'), (1, '96.810')] [2023-10-10 19:21:58,790][123465] Saving new best policy, reward=96.810! [2023-10-10 19:22:00,466][123582] Updated weights for policy 0, policy_version 71303 (0.0010) [2023-10-10 19:22:00,777][123614] Updated weights for policy 1, policy_version 71210 (0.0010) [2023-10-10 19:22:00,828][123582] Updated weights for policy 0, policy_version 71313 (0.0008) [2023-10-10 19:22:01,142][123614] Updated weights for policy 1, policy_version 71220 (0.0009) [2023-10-10 19:22:01,201][123582] Updated weights for policy 0, policy_version 71323 (0.0008) [2023-10-10 19:22:01,507][123614] Updated weights for policy 1, policy_version 71230 (0.0007) [2023-10-10 19:22:03,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 145981440. Throughput: 0: 1836.8, 1: 1781.5. Samples: 36504102. Policy #0 lag: (min: 6.0, avg: 12.9, max: 38.0) [2023-10-10 19:22:03,789][122664] Avg episode reward: [(0, '62.790'), (1, '96.360')] [2023-10-10 19:22:04,873][123582] Updated weights for policy 0, policy_version 71333 (0.0008) [2023-10-10 19:22:05,192][123614] Updated weights for policy 1, policy_version 71240 (0.0008) [2023-10-10 19:22:05,254][123582] Updated weights for policy 0, policy_version 71343 (0.0008) [2023-10-10 19:22:05,551][123614] Updated weights for policy 1, policy_version 71250 (0.0007) [2023-10-10 19:22:05,630][123582] Updated weights for policy 0, policy_version 71353 (0.0007) [2023-10-10 19:22:05,921][123614] Updated weights for policy 1, policy_version 71260 (0.0008) [2023-10-10 19:22:08,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 146046976. Throughput: 0: 1823.4, 1: 1792.9. Samples: 36526842. Policy #0 lag: (min: 6.0, avg: 12.9, max: 38.0) [2023-10-10 19:22:08,788][122664] Avg episode reward: [(0, '65.570'), (1, '95.690')] [2023-10-10 19:22:09,357][123582] Updated weights for policy 0, policy_version 71363 (0.0007) [2023-10-10 19:22:09,651][123614] Updated weights for policy 1, policy_version 71270 (0.0008) [2023-10-10 19:22:09,719][123582] Updated weights for policy 0, policy_version 71373 (0.0007) [2023-10-10 19:22:10,026][123614] Updated weights for policy 1, policy_version 71280 (0.0008) [2023-10-10 19:22:10,097][123582] Updated weights for policy 0, policy_version 71383 (0.0009) [2023-10-10 19:22:10,388][123614] Updated weights for policy 1, policy_version 71290 (0.0007) [2023-10-10 19:22:13,725][123582] Updated weights for policy 0, policy_version 71393 (0.0009) [2023-10-10 19:22:13,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 146112512. Throughput: 0: 1818.7, 1: 1787.1. Samples: 36536610. Policy #0 lag: (min: 6.0, avg: 12.9, max: 38.0) [2023-10-10 19:22:13,789][122664] Avg episode reward: [(0, '68.130'), (1, '95.470')] [2023-10-10 19:22:14,085][123614] Updated weights for policy 1, policy_version 71300 (0.0009) [2023-10-10 19:22:14,095][123582] Updated weights for policy 0, policy_version 71403 (0.0007) [2023-10-10 19:22:14,455][123614] Updated weights for policy 1, policy_version 71310 (0.0009) [2023-10-10 19:22:14,469][123582] Updated weights for policy 0, policy_version 71413 (0.0008) [2023-10-10 19:22:14,827][123614] Updated weights for policy 1, policy_version 71320 (0.0008) [2023-10-10 19:22:14,836][123582] Updated weights for policy 0, policy_version 71423 (0.0008) [2023-10-10 19:22:18,569][123582] Updated weights for policy 0, policy_version 71433 (0.0007) [2023-10-10 19:22:18,603][123614] Updated weights for policy 1, policy_version 71330 (0.0010) [2023-10-10 19:22:18,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 146178048. Throughput: 0: 1819.8, 1: 1793.4. Samples: 36559418. Policy #0 lag: (min: 6.0, avg: 12.9, max: 38.0) [2023-10-10 19:22:18,789][122664] Avg episode reward: [(0, '72.470'), (1, '97.710')] [2023-10-10 19:22:18,940][123582] Updated weights for policy 0, policy_version 71443 (0.0009) [2023-10-10 19:22:18,978][123614] Updated weights for policy 1, policy_version 71340 (0.0007) [2023-10-10 19:22:19,311][123582] Updated weights for policy 0, policy_version 71453 (0.0008) [2023-10-10 19:22:19,346][123614] Updated weights for policy 1, policy_version 71350 (0.0008) [2023-10-10 19:22:19,718][123465] Saving new best policy, reward=97.710! [2023-10-10 19:22:19,722][123614] Updated weights for policy 1, policy_version 71360 (0.0011) [2023-10-10 19:22:23,064][123582] Updated weights for policy 0, policy_version 71463 (0.0007) [2023-10-10 19:22:23,443][123582] Updated weights for policy 0, policy_version 71473 (0.0008) [2023-10-10 19:22:23,650][123614] Updated weights for policy 1, policy_version 71370 (0.0007) [2023-10-10 19:22:23,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 146243584. Throughput: 0: 1820.1, 1: 1811.6. Samples: 36580362. Policy #0 lag: (min: 6.0, avg: 12.9, max: 38.0) [2023-10-10 19:22:23,789][122664] Avg episode reward: [(0, '74.020'), (1, '99.720')] [2023-10-10 19:22:23,814][123582] Updated weights for policy 0, policy_version 71483 (0.0008) [2023-10-10 19:22:24,018][123614] Updated weights for policy 1, policy_version 71380 (0.0008) [2023-10-10 19:22:24,381][123614] Updated weights for policy 1, policy_version 71390 (0.0011) [2023-10-10 19:22:24,453][123465] Saving new best policy, reward=99.720! [2023-10-10 19:22:27,612][123582] Updated weights for policy 0, policy_version 71493 (0.0009) [2023-10-10 19:22:27,990][123582] Updated weights for policy 0, policy_version 71503 (0.0008) [2023-10-10 19:22:28,243][123614] Updated weights for policy 1, policy_version 71400 (0.0010) [2023-10-10 19:22:28,362][123582] Updated weights for policy 0, policy_version 71513 (0.0009) [2023-10-10 19:22:28,602][123614] Updated weights for policy 1, policy_version 71410 (0.0007) [2023-10-10 19:22:28,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 146341888. Throughput: 0: 1817.1, 1: 1787.2. Samples: 36591210. Policy #0 lag: (min: 6.0, avg: 12.9, max: 38.0) [2023-10-10 19:22:28,788][122664] Avg episode reward: [(0, '79.310'), (1, '100.570')] [2023-10-10 19:22:28,972][123614] Updated weights for policy 1, policy_version 71420 (0.0007) [2023-10-10 19:22:29,116][123465] Saving new best policy, reward=100.570! [2023-10-10 19:22:32,075][123582] Updated weights for policy 0, policy_version 71523 (0.0008) [2023-10-10 19:22:32,457][123582] Updated weights for policy 0, policy_version 71533 (0.0009) [2023-10-10 19:22:32,784][123614] Updated weights for policy 1, policy_version 71430 (0.0008) [2023-10-10 19:22:32,830][123582] Updated weights for policy 0, policy_version 71543 (0.0011) [2023-10-10 19:22:33,153][123614] Updated weights for policy 1, policy_version 71440 (0.0007) [2023-10-10 19:22:33,519][123614] Updated weights for policy 1, policy_version 71450 (0.0009) [2023-10-10 19:22:33,788][122664] Fps is (10 sec: 19661.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 146440192. Throughput: 0: 1817.3, 1: 1810.1. Samples: 36612978. Policy #0 lag: (min: 6.0, avg: 12.9, max: 38.0) [2023-10-10 19:22:33,789][122664] Avg episode reward: [(0, '80.750'), (1, '99.810')] [2023-10-10 19:22:36,428][123582] Updated weights for policy 0, policy_version 71553 (0.0008) [2023-10-10 19:22:36,795][123582] Updated weights for policy 0, policy_version 71563 (0.0007) [2023-10-10 19:22:37,127][123614] Updated weights for policy 1, policy_version 71460 (0.0007) [2023-10-10 19:22:37,169][123582] Updated weights for policy 0, policy_version 71573 (0.0007) [2023-10-10 19:22:37,489][123614] Updated weights for policy 1, policy_version 71470 (0.0007) [2023-10-10 19:22:37,533][123582] Updated weights for policy 0, policy_version 71583 (0.0007) [2023-10-10 19:22:37,867][123614] Updated weights for policy 1, policy_version 71480 (0.0008) [2023-10-10 19:22:38,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 146505728. Throughput: 0: 1808.9, 1: 1785.5. Samples: 36633626. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 19:22:38,789][122664] Avg episode reward: [(0, '78.780'), (1, '101.120')] [2023-10-10 19:22:38,799][123465] Saving new best policy, reward=101.120! [2023-10-10 19:22:41,232][123582] Updated weights for policy 0, policy_version 71593 (0.0009) [2023-10-10 19:22:41,606][123582] Updated weights for policy 0, policy_version 71603 (0.0008) [2023-10-10 19:22:41,655][123614] Updated weights for policy 1, policy_version 71490 (0.0009) [2023-10-10 19:22:41,970][123582] Updated weights for policy 0, policy_version 71613 (0.0007) [2023-10-10 19:22:42,020][123614] Updated weights for policy 1, policy_version 71500 (0.0007) [2023-10-10 19:22:42,387][123614] Updated weights for policy 1, policy_version 71510 (0.0007) [2023-10-10 19:22:42,759][123614] Updated weights for policy 1, policy_version 71520 (0.0009) [2023-10-10 19:22:43,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 146571264. Throughput: 0: 1815.8, 1: 1807.6. Samples: 36645478. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 19:22:43,789][122664] Avg episode reward: [(0, '75.330'), (1, '100.210')] [2023-10-10 19:22:45,645][123582] Updated weights for policy 0, policy_version 71623 (0.0007) [2023-10-10 19:22:46,022][123582] Updated weights for policy 0, policy_version 71633 (0.0008) [2023-10-10 19:22:46,368][123614] Updated weights for policy 1, policy_version 71530 (0.0007) [2023-10-10 19:22:46,405][123582] Updated weights for policy 0, policy_version 71643 (0.0009) [2023-10-10 19:22:46,737][123614] Updated weights for policy 1, policy_version 71540 (0.0007) [2023-10-10 19:22:47,101][123614] Updated weights for policy 1, policy_version 71550 (0.0009) [2023-10-10 19:22:48,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 146636800. Throughput: 0: 1807.8, 1: 1790.9. Samples: 36666042. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 19:22:48,788][122664] Avg episode reward: [(0, '75.650'), (1, '97.790')] [2023-10-10 19:22:50,256][123582] Updated weights for policy 0, policy_version 71653 (0.0008) [2023-10-10 19:22:50,634][123582] Updated weights for policy 0, policy_version 71663 (0.0009) [2023-10-10 19:22:50,790][123614] Updated weights for policy 1, policy_version 71560 (0.0008) [2023-10-10 19:22:51,002][123582] Updated weights for policy 0, policy_version 71673 (0.0010) [2023-10-10 19:22:51,153][123614] Updated weights for policy 1, policy_version 71570 (0.0007) [2023-10-10 19:22:51,518][123614] Updated weights for policy 1, policy_version 71580 (0.0009) [2023-10-10 19:22:53,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 146702336. Throughput: 0: 1802.3, 1: 1792.5. Samples: 36688610. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 19:22:53,789][122664] Avg episode reward: [(0, '76.330'), (1, '96.140')] [2023-10-10 19:22:54,629][123582] Updated weights for policy 0, policy_version 71683 (0.0007) [2023-10-10 19:22:54,999][123582] Updated weights for policy 0, policy_version 71693 (0.0008) [2023-10-10 19:22:55,149][123614] Updated weights for policy 1, policy_version 71590 (0.0008) [2023-10-10 19:22:55,377][123582] Updated weights for policy 0, policy_version 71703 (0.0009) [2023-10-10 19:22:55,518][123614] Updated weights for policy 1, policy_version 71600 (0.0009) [2023-10-10 19:22:55,897][123614] Updated weights for policy 1, policy_version 71610 (0.0009) [2023-10-10 19:22:58,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 146767872. Throughput: 0: 1803.5, 1: 1793.4. Samples: 36698468. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 19:22:58,788][122664] Avg episode reward: [(0, '75.890'), (1, '93.800')] [2023-10-10 19:22:59,048][123582] Updated weights for policy 0, policy_version 71713 (0.0009) [2023-10-10 19:22:59,408][123582] Updated weights for policy 0, policy_version 71723 (0.0008) [2023-10-10 19:22:59,650][123614] Updated weights for policy 1, policy_version 71620 (0.0007) [2023-10-10 19:22:59,780][123582] Updated weights for policy 0, policy_version 71733 (0.0008) [2023-10-10 19:23:00,021][123614] Updated weights for policy 1, policy_version 71630 (0.0007) [2023-10-10 19:23:00,149][123582] Updated weights for policy 0, policy_version 71743 (0.0009) [2023-10-10 19:23:00,388][123614] Updated weights for policy 1, policy_version 71640 (0.0008) [2023-10-10 19:23:03,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 146833408. Throughput: 0: 1807.4, 1: 1793.3. Samples: 36721450. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 19:23:03,789][122664] Avg episode reward: [(0, '76.160'), (1, '92.370')] [2023-10-10 19:23:03,822][123582] Updated weights for policy 0, policy_version 71753 (0.0009) [2023-10-10 19:23:04,092][123614] Updated weights for policy 1, policy_version 71650 (0.0010) [2023-10-10 19:23:04,201][123582] Updated weights for policy 0, policy_version 71763 (0.0008) [2023-10-10 19:23:04,456][123614] Updated weights for policy 1, policy_version 71660 (0.0009) [2023-10-10 19:23:04,572][123582] Updated weights for policy 0, policy_version 71773 (0.0009) [2023-10-10 19:23:04,822][123614] Updated weights for policy 1, policy_version 71670 (0.0009) [2023-10-10 19:23:05,194][123614] Updated weights for policy 1, policy_version 71680 (0.0009) [2023-10-10 19:23:08,292][123582] Updated weights for policy 0, policy_version 71783 (0.0009) [2023-10-10 19:23:08,669][123582] Updated weights for policy 0, policy_version 71793 (0.0008) [2023-10-10 19:23:08,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 146898944. Throughput: 0: 1811.9, 1: 1811.6. Samples: 36743416. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 19:23:08,789][122664] Avg episode reward: [(0, '74.250'), (1, '89.010')] [2023-10-10 19:23:08,900][123614] Updated weights for policy 1, policy_version 71690 (0.0007) [2023-10-10 19:23:09,035][123582] Updated weights for policy 0, policy_version 71803 (0.0008) [2023-10-10 19:23:09,267][123614] Updated weights for policy 1, policy_version 71700 (0.0008) [2023-10-10 19:23:09,628][123614] Updated weights for policy 1, policy_version 71710 (0.0008) [2023-10-10 19:23:12,653][123582] Updated weights for policy 0, policy_version 71813 (0.0007) [2023-10-10 19:23:13,032][123582] Updated weights for policy 0, policy_version 71823 (0.0009) [2023-10-10 19:23:13,354][123614] Updated weights for policy 1, policy_version 71720 (0.0009) [2023-10-10 19:23:13,403][123582] Updated weights for policy 0, policy_version 71833 (0.0008) [2023-10-10 19:23:13,723][123614] Updated weights for policy 1, policy_version 71730 (0.0008) [2023-10-10 19:23:13,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 146997248. Throughput: 0: 1808.7, 1: 1807.4. Samples: 36753934. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 19:23:13,789][122664] Avg episode reward: [(0, '72.520'), (1, '88.110')] [2023-10-10 19:23:14,096][123614] Updated weights for policy 1, policy_version 71740 (0.0009) [2023-10-10 19:23:17,195][123582] Updated weights for policy 0, policy_version 71843 (0.0008) [2023-10-10 19:23:17,559][123582] Updated weights for policy 0, policy_version 71853 (0.0009) [2023-10-10 19:23:17,721][123614] Updated weights for policy 1, policy_version 71750 (0.0008) [2023-10-10 19:23:17,937][123582] Updated weights for policy 0, policy_version 71863 (0.0009) [2023-10-10 19:23:18,089][123614] Updated weights for policy 1, policy_version 71760 (0.0008) [2023-10-10 19:23:18,453][123614] Updated weights for policy 1, policy_version 71770 (0.0008) [2023-10-10 19:23:18,788][122664] Fps is (10 sec: 19661.3, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 147095552. Throughput: 0: 1813.9, 1: 1813.4. Samples: 36776208. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 19:23:18,788][122664] Avg episode reward: [(0, '75.040'), (1, '87.500')] [2023-10-10 19:23:21,627][123582] Updated weights for policy 0, policy_version 71873 (0.0010) [2023-10-10 19:23:22,003][123582] Updated weights for policy 0, policy_version 71883 (0.0007) [2023-10-10 19:23:22,188][123614] Updated weights for policy 1, policy_version 71780 (0.0007) [2023-10-10 19:23:22,373][123582] Updated weights for policy 0, policy_version 71893 (0.0008) [2023-10-10 19:23:22,553][123614] Updated weights for policy 1, policy_version 71790 (0.0008) [2023-10-10 19:23:22,741][123582] Updated weights for policy 0, policy_version 71903 (0.0008) [2023-10-10 19:23:22,918][123614] Updated weights for policy 1, policy_version 71800 (0.0009) [2023-10-10 19:23:23,788][122664] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14551.2). Total num frames: 147161088. Throughput: 0: 1812.0, 1: 1810.7. Samples: 36796646. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 19:23:23,789][122664] Avg episode reward: [(0, '70.620'), (1, '88.370')] [2023-10-10 19:23:26,550][123582] Updated weights for policy 0, policy_version 71913 (0.0008) [2023-10-10 19:23:26,638][123614] Updated weights for policy 1, policy_version 71810 (0.0007) [2023-10-10 19:23:26,921][123582] Updated weights for policy 0, policy_version 71923 (0.0007) [2023-10-10 19:23:27,004][123614] Updated weights for policy 1, policy_version 71820 (0.0008) [2023-10-10 19:23:27,296][123582] Updated weights for policy 0, policy_version 71933 (0.0009) [2023-10-10 19:23:27,371][123614] Updated weights for policy 1, policy_version 71830 (0.0008) [2023-10-10 19:23:27,746][123614] Updated weights for policy 1, policy_version 71840 (0.0011) [2023-10-10 19:23:28,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 147226624. Throughput: 0: 1819.4, 1: 1815.1. Samples: 36809030. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 19:23:28,788][122664] Avg episode reward: [(0, '67.070'), (1, '87.150')] [2023-10-10 19:23:30,964][123582] Updated weights for policy 0, policy_version 71943 (0.0009) [2023-10-10 19:23:31,332][123582] Updated weights for policy 0, policy_version 71953 (0.0007) [2023-10-10 19:23:31,543][123614] Updated weights for policy 1, policy_version 71850 (0.0007) [2023-10-10 19:23:31,712][123582] Updated weights for policy 0, policy_version 71963 (0.0008) [2023-10-10 19:23:31,916][123614] Updated weights for policy 1, policy_version 71860 (0.0008) [2023-10-10 19:23:32,285][123614] Updated weights for policy 1, policy_version 71870 (0.0010) [2023-10-10 19:23:33,789][122664] Fps is (10 sec: 13106.0, 60 sec: 14199.2, 300 sec: 14440.1). Total num frames: 147292160. Throughput: 0: 1813.6, 1: 1815.5. Samples: 36829354. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 19:23:33,790][122664] Avg episode reward: [(0, '69.450'), (1, '84.860')] [2023-10-10 19:23:35,359][123582] Updated weights for policy 0, policy_version 71973 (0.0008) [2023-10-10 19:23:35,737][123582] Updated weights for policy 0, policy_version 71983 (0.0009) [2023-10-10 19:23:35,925][123614] Updated weights for policy 1, policy_version 71880 (0.0007) [2023-10-10 19:23:36,105][123582] Updated weights for policy 0, policy_version 71993 (0.0010) [2023-10-10 19:23:36,294][123614] Updated weights for policy 1, policy_version 71890 (0.0008) [2023-10-10 19:23:36,661][123614] Updated weights for policy 1, policy_version 71900 (0.0008) [2023-10-10 19:23:38,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 147357696. Throughput: 0: 1821.3, 1: 1812.7. Samples: 36852142. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 19:23:38,788][122664] Avg episode reward: [(0, '66.910'), (1, '82.770')] [2023-10-10 19:23:39,721][123582] Updated weights for policy 0, policy_version 72003 (0.0008) [2023-10-10 19:23:40,095][123582] Updated weights for policy 0, policy_version 72013 (0.0008) [2023-10-10 19:23:40,480][123582] Updated weights for policy 0, policy_version 72023 (0.0009) [2023-10-10 19:23:40,512][123614] Updated weights for policy 1, policy_version 71910 (0.0007) [2023-10-10 19:23:40,876][123614] Updated weights for policy 1, policy_version 71920 (0.0009) [2023-10-10 19:23:41,250][123614] Updated weights for policy 1, policy_version 71930 (0.0010) [2023-10-10 19:23:43,788][122664] Fps is (10 sec: 13108.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 147423232. Throughput: 0: 1825.0, 1: 1811.3. Samples: 36862100. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 19:23:43,788][122664] Avg episode reward: [(0, '66.750'), (1, '83.080')] [2023-10-10 19:23:44,001][123582] Updated weights for policy 0, policy_version 72033 (0.0008) [2023-10-10 19:23:44,374][123582] Updated weights for policy 0, policy_version 72043 (0.0008) [2023-10-10 19:23:44,755][123582] Updated weights for policy 0, policy_version 72053 (0.0008) [2023-10-10 19:23:44,976][123614] Updated weights for policy 1, policy_version 71940 (0.0008) [2023-10-10 19:23:45,115][123582] Updated weights for policy 0, policy_version 72063 (0.0007) [2023-10-10 19:23:45,341][123614] Updated weights for policy 1, policy_version 71950 (0.0007) [2023-10-10 19:23:45,712][123614] Updated weights for policy 1, policy_version 71960 (0.0007) [2023-10-10 19:23:48,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 147488768. Throughput: 0: 1825.4, 1: 1813.9. Samples: 36885218. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 19:23:48,789][122664] Avg episode reward: [(0, '66.200'), (1, '80.940')] [2023-10-10 19:23:48,817][123582] Updated weights for policy 0, policy_version 72073 (0.0008) [2023-10-10 19:23:49,188][123582] Updated weights for policy 0, policy_version 72083 (0.0008) [2023-10-10 19:23:49,346][123614] Updated weights for policy 1, policy_version 71970 (0.0007) [2023-10-10 19:23:49,561][123582] Updated weights for policy 0, policy_version 72093 (0.0007) [2023-10-10 19:23:49,716][123614] Updated weights for policy 1, policy_version 71980 (0.0007) [2023-10-10 19:23:50,078][123614] Updated weights for policy 1, policy_version 71990 (0.0007) [2023-10-10 19:23:50,438][123614] Updated weights for policy 1, policy_version 72000 (0.0007) [2023-10-10 19:23:53,128][123582] Updated weights for policy 0, policy_version 72103 (0.0009) [2023-10-10 19:23:53,494][123582] Updated weights for policy 0, policy_version 72113 (0.0011) [2023-10-10 19:23:53,788][122664] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 147554304. Throughput: 0: 1822.3, 1: 1820.4. Samples: 36907338. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 19:23:53,789][122664] Avg episode reward: [(0, '69.860'), (1, '81.060')] [2023-10-10 19:23:53,865][123582] Updated weights for policy 0, policy_version 72123 (0.0009) [2023-10-10 19:23:54,004][123614] Updated weights for policy 1, policy_version 72010 (0.0008) [2023-10-10 19:23:54,043][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000072128_73859072.pth... [2023-10-10 19:23:54,073][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000070400_72089600.pth [2023-10-10 19:23:54,373][123614] Updated weights for policy 1, policy_version 72020 (0.0010) [2023-10-10 19:23:54,739][123614] Updated weights for policy 1, policy_version 72030 (0.0007) [2023-10-10 19:23:54,811][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000072032_73760768.pth... [2023-10-10 19:23:54,839][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000070336_72024064.pth [2023-10-10 19:23:57,709][123582] Updated weights for policy 0, policy_version 72133 (0.0010) [2023-10-10 19:23:58,085][123582] Updated weights for policy 0, policy_version 72143 (0.0010) [2023-10-10 19:23:58,445][123614] Updated weights for policy 1, policy_version 72040 (0.0007) [2023-10-10 19:23:58,460][123582] Updated weights for policy 0, policy_version 72153 (0.0008) [2023-10-10 19:23:58,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 147652608. Throughput: 0: 1823.4, 1: 1818.1. Samples: 36917802. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 19:23:58,789][122664] Avg episode reward: [(0, '71.150'), (1, '81.510')] [2023-10-10 19:23:58,801][123614] Updated weights for policy 1, policy_version 72050 (0.0007) [2023-10-10 19:23:59,182][123614] Updated weights for policy 1, policy_version 72060 (0.0007) [2023-10-10 19:24:02,262][123582] Updated weights for policy 0, policy_version 72163 (0.0008) [2023-10-10 19:24:02,639][123582] Updated weights for policy 0, policy_version 72173 (0.0009) [2023-10-10 19:24:02,906][123614] Updated weights for policy 1, policy_version 72070 (0.0010) [2023-10-10 19:24:03,006][123582] Updated weights for policy 0, policy_version 72183 (0.0007) [2023-10-10 19:24:03,279][123614] Updated weights for policy 1, policy_version 72080 (0.0009) [2023-10-10 19:24:03,645][123614] Updated weights for policy 1, policy_version 72090 (0.0007) [2023-10-10 19:24:03,788][122664] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 147718144. Throughput: 0: 1821.9, 1: 1819.6. Samples: 36940072. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 19:24:03,789][122664] Avg episode reward: [(0, '70.750'), (1, '79.950')] [2023-10-10 19:24:06,604][123582] Updated weights for policy 0, policy_version 72193 (0.0007) [2023-10-10 19:24:06,972][123582] Updated weights for policy 0, policy_version 72203 (0.0007) [2023-10-10 19:24:07,308][123614] Updated weights for policy 1, policy_version 72100 (0.0008) [2023-10-10 19:24:07,351][123582] Updated weights for policy 0, policy_version 72213 (0.0007) [2023-10-10 19:24:07,689][123614] Updated weights for policy 1, policy_version 72110 (0.0009) [2023-10-10 19:24:07,728][123582] Updated weights for policy 0, policy_version 72223 (0.0007) [2023-10-10 19:24:08,057][123614] Updated weights for policy 1, policy_version 72120 (0.0009) [2023-10-10 19:24:08,788][122664] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 147816448. Throughput: 0: 1821.2, 1: 1817.1. Samples: 36960370. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:24:08,789][122664] Avg episode reward: [(0, '71.890'), (1, '80.810')] [2023-10-10 19:24:11,489][123582] Updated weights for policy 0, policy_version 72233 (0.0009) [2023-10-10 19:24:11,619][123614] Updated weights for policy 1, policy_version 72130 (0.0008) [2023-10-10 19:24:11,850][123582] Updated weights for policy 0, policy_version 72243 (0.0007) [2023-10-10 19:24:11,995][123614] Updated weights for policy 1, policy_version 72140 (0.0009) [2023-10-10 19:24:12,221][123582] Updated weights for policy 0, policy_version 72253 (0.0009) [2023-10-10 19:24:12,364][123614] Updated weights for policy 1, policy_version 72150 (0.0010) [2023-10-10 19:24:12,735][123614] Updated weights for policy 1, policy_version 72160 (0.0009) [2023-10-10 19:24:13,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 147881984. Throughput: 0: 1816.6, 1: 1818.6. Samples: 36972614. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:24:13,789][122664] Avg episode reward: [(0, '73.950'), (1, '86.790')] [2023-10-10 19:24:15,958][123582] Updated weights for policy 0, policy_version 72263 (0.0008) [2023-10-10 19:24:16,336][123582] Updated weights for policy 0, policy_version 72273 (0.0009) [2023-10-10 19:24:16,413][123614] Updated weights for policy 1, policy_version 72170 (0.0009) [2023-10-10 19:24:16,700][123582] Updated weights for policy 0, policy_version 72283 (0.0007) [2023-10-10 19:24:16,780][123614] Updated weights for policy 1, policy_version 72180 (0.0008) [2023-10-10 19:24:17,157][123614] Updated weights for policy 1, policy_version 72190 (0.0010) [2023-10-10 19:24:18,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 147947520. Throughput: 0: 1816.1, 1: 1814.8. Samples: 36992742. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:24:18,789][122664] Avg episode reward: [(0, '71.930'), (1, '89.420')] [2023-10-10 19:24:20,522][123582] Updated weights for policy 0, policy_version 72293 (0.0008) [2023-10-10 19:24:20,895][123582] Updated weights for policy 0, policy_version 72303 (0.0008) [2023-10-10 19:24:20,995][123614] Updated weights for policy 1, policy_version 72200 (0.0008) [2023-10-10 19:24:21,266][123582] Updated weights for policy 0, policy_version 72313 (0.0008) [2023-10-10 19:24:21,362][123614] Updated weights for policy 1, policy_version 72210 (0.0008) [2023-10-10 19:24:21,733][123614] Updated weights for policy 1, policy_version 72220 (0.0009) [2023-10-10 19:24:23,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 148013056. Throughput: 0: 1815.9, 1: 1813.2. Samples: 37015448. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:24:23,788][122664] Avg episode reward: [(0, '69.980'), (1, '90.970')] [2023-10-10 19:24:24,864][123582] Updated weights for policy 0, policy_version 72323 (0.0009) [2023-10-10 19:24:25,239][123582] Updated weights for policy 0, policy_version 72333 (0.0012) [2023-10-10 19:24:25,510][123614] Updated weights for policy 1, policy_version 72230 (0.0007) [2023-10-10 19:24:25,621][123582] Updated weights for policy 0, policy_version 72343 (0.0010) [2023-10-10 19:24:25,882][123614] Updated weights for policy 1, policy_version 72240 (0.0008) [2023-10-10 19:24:26,256][123614] Updated weights for policy 1, policy_version 72250 (0.0007) [2023-10-10 19:24:28,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 148078592. Throughput: 0: 1810.5, 1: 1812.7. Samples: 37025146. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:24:28,789][122664] Avg episode reward: [(0, '70.760'), (1, '96.490')] [2023-10-10 19:24:29,284][123582] Updated weights for policy 0, policy_version 72353 (0.0007) [2023-10-10 19:24:29,671][123582] Updated weights for policy 0, policy_version 72363 (0.0008) [2023-10-10 19:24:29,985][123614] Updated weights for policy 1, policy_version 72260 (0.0007) [2023-10-10 19:24:30,033][123582] Updated weights for policy 0, policy_version 72373 (0.0009) [2023-10-10 19:24:30,352][123614] Updated weights for policy 1, policy_version 72270 (0.0008) [2023-10-10 19:24:30,406][123582] Updated weights for policy 0, policy_version 72383 (0.0007) [2023-10-10 19:24:30,723][123614] Updated weights for policy 1, policy_version 72280 (0.0010) [2023-10-10 19:24:33,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.7, 300 sec: 14440.1). Total num frames: 148144128. Throughput: 0: 1809.6, 1: 1805.3. Samples: 37047888. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:24:33,789][122664] Avg episode reward: [(0, '73.200'), (1, '96.100')] [2023-10-10 19:24:34,017][123582] Updated weights for policy 0, policy_version 72393 (0.0007) [2023-10-10 19:24:34,382][123582] Updated weights for policy 0, policy_version 72403 (0.0008) [2023-10-10 19:24:34,597][123614] Updated weights for policy 1, policy_version 72290 (0.0010) [2023-10-10 19:24:34,750][123582] Updated weights for policy 0, policy_version 72413 (0.0009) [2023-10-10 19:24:34,966][123614] Updated weights for policy 1, policy_version 72300 (0.0008) [2023-10-10 19:24:35,336][123614] Updated weights for policy 1, policy_version 72310 (0.0008) [2023-10-10 19:24:35,705][123614] Updated weights for policy 1, policy_version 72320 (0.0007) [2023-10-10 19:24:38,419][123582] Updated weights for policy 0, policy_version 72423 (0.0009) [2023-10-10 19:24:38,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 148209664. Throughput: 0: 1822.1, 1: 1802.5. Samples: 37070444. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:24:38,788][122664] Avg episode reward: [(0, '72.340'), (1, '98.170')] [2023-10-10 19:24:38,801][123582] Updated weights for policy 0, policy_version 72433 (0.0009) [2023-10-10 19:24:39,178][123582] Updated weights for policy 0, policy_version 72443 (0.0008) [2023-10-10 19:24:39,451][123614] Updated weights for policy 1, policy_version 72330 (0.0009) [2023-10-10 19:24:39,816][123614] Updated weights for policy 1, policy_version 72340 (0.0008) [2023-10-10 19:24:40,190][123614] Updated weights for policy 1, policy_version 72350 (0.0009) [2023-10-10 19:24:42,832][123582] Updated weights for policy 0, policy_version 72453 (0.0009) [2023-10-10 19:24:43,199][123582] Updated weights for policy 0, policy_version 72463 (0.0008) [2023-10-10 19:24:43,570][123582] Updated weights for policy 0, policy_version 72473 (0.0007) [2023-10-10 19:24:43,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 148275200. Throughput: 0: 1816.9, 1: 1798.9. Samples: 37080514. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:24:43,789][122664] Avg episode reward: [(0, '72.510'), (1, '101.740')] [2023-10-10 19:24:43,807][123614] Updated weights for policy 1, policy_version 72360 (0.0008) [2023-10-10 19:24:44,181][123614] Updated weights for policy 1, policy_version 72370 (0.0007) [2023-10-10 19:24:44,549][123614] Updated weights for policy 1, policy_version 72380 (0.0011) [2023-10-10 19:24:44,695][123465] Saving new best policy, reward=101.740! [2023-10-10 19:24:47,309][123582] Updated weights for policy 0, policy_version 72483 (0.0007) [2023-10-10 19:24:47,674][123582] Updated weights for policy 0, policy_version 72493 (0.0008) [2023-10-10 19:24:48,056][123582] Updated weights for policy 0, policy_version 72503 (0.0009) [2023-10-10 19:24:48,353][123614] Updated weights for policy 1, policy_version 72390 (0.0008) [2023-10-10 19:24:48,731][123614] Updated weights for policy 1, policy_version 72400 (0.0009) [2023-10-10 19:24:48,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 148373504. Throughput: 0: 1824.1, 1: 1798.7. Samples: 37103098. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:24:48,789][122664] Avg episode reward: [(0, '69.250'), (1, '104.050')] [2023-10-10 19:24:49,100][123614] Updated weights for policy 1, policy_version 72410 (0.0009) [2023-10-10 19:24:49,317][123465] Saving new best policy, reward=104.050! [2023-10-10 19:24:51,805][123582] Updated weights for policy 0, policy_version 72513 (0.0007) [2023-10-10 19:24:52,178][123582] Updated weights for policy 0, policy_version 72523 (0.0007) [2023-10-10 19:24:52,554][123582] Updated weights for policy 0, policy_version 72533 (0.0007) [2023-10-10 19:24:52,884][123614] Updated weights for policy 1, policy_version 72420 (0.0009) [2023-10-10 19:24:52,939][123582] Updated weights for policy 0, policy_version 72543 (0.0009) [2023-10-10 19:24:53,260][123614] Updated weights for policy 1, policy_version 72430 (0.0008) [2023-10-10 19:24:53,637][123614] Updated weights for policy 1, policy_version 72440 (0.0007) [2023-10-10 19:24:53,788][122664] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 148439040. Throughput: 0: 1821.1, 1: 1798.3. Samples: 37123240. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:24:53,788][122664] Avg episode reward: [(0, '66.920'), (1, '102.960')] [2023-10-10 19:24:56,622][123582] Updated weights for policy 0, policy_version 72553 (0.0011) [2023-10-10 19:24:56,983][123582] Updated weights for policy 0, policy_version 72563 (0.0010) [2023-10-10 19:24:57,357][123582] Updated weights for policy 0, policy_version 72573 (0.0008) [2023-10-10 19:24:57,406][123614] Updated weights for policy 1, policy_version 72450 (0.0007) [2023-10-10 19:24:57,772][123614] Updated weights for policy 1, policy_version 72460 (0.0008) [2023-10-10 19:24:58,137][123614] Updated weights for policy 1, policy_version 72470 (0.0008) [2023-10-10 19:24:58,507][123614] Updated weights for policy 1, policy_version 72480 (0.0007) [2023-10-10 19:24:58,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 148537344. Throughput: 0: 1829.2, 1: 1791.8. Samples: 37135558. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:24:58,789][122664] Avg episode reward: [(0, '66.650'), (1, '103.470')] [2023-10-10 19:25:01,095][123582] Updated weights for policy 0, policy_version 72583 (0.0008) [2023-10-10 19:25:01,469][123582] Updated weights for policy 0, policy_version 72593 (0.0008) [2023-10-10 19:25:01,841][123582] Updated weights for policy 0, policy_version 72603 (0.0010) [2023-10-10 19:25:02,227][123614] Updated weights for policy 1, policy_version 72490 (0.0009) [2023-10-10 19:25:02,598][123614] Updated weights for policy 1, policy_version 72500 (0.0007) [2023-10-10 19:25:02,970][123614] Updated weights for policy 1, policy_version 72510 (0.0008) [2023-10-10 19:25:03,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 148602880. Throughput: 0: 1825.5, 1: 1803.0. Samples: 37156024. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:25:03,789][122664] Avg episode reward: [(0, '69.380'), (1, '104.070')] [2023-10-10 19:25:03,791][123465] Saving new best policy, reward=104.070! [2023-10-10 19:25:05,486][123582] Updated weights for policy 0, policy_version 72613 (0.0009) [2023-10-10 19:25:05,866][123582] Updated weights for policy 0, policy_version 72623 (0.0008) [2023-10-10 19:25:06,235][123582] Updated weights for policy 0, policy_version 72633 (0.0008) [2023-10-10 19:25:06,603][123614] Updated weights for policy 1, policy_version 72520 (0.0007) [2023-10-10 19:25:06,975][123614] Updated weights for policy 1, policy_version 72530 (0.0009) [2023-10-10 19:25:07,349][123614] Updated weights for policy 1, policy_version 72540 (0.0007) [2023-10-10 19:25:08,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 148668416. Throughput: 0: 1823.1, 1: 1793.7. Samples: 37178204. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:25:08,789][122664] Avg episode reward: [(0, '72.360'), (1, '104.900')] [2023-10-10 19:25:08,797][123465] Saving new best policy, reward=104.900! [2023-10-10 19:25:09,914][123582] Updated weights for policy 0, policy_version 72643 (0.0008) [2023-10-10 19:25:10,288][123582] Updated weights for policy 0, policy_version 72653 (0.0009) [2023-10-10 19:25:10,663][123582] Updated weights for policy 0, policy_version 72663 (0.0008) [2023-10-10 19:25:11,114][123614] Updated weights for policy 1, policy_version 72550 (0.0007) [2023-10-10 19:25:11,481][123614] Updated weights for policy 1, policy_version 72560 (0.0007) [2023-10-10 19:25:11,862][123614] Updated weights for policy 1, policy_version 72570 (0.0010) [2023-10-10 19:25:13,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 148733952. Throughput: 0: 1826.3, 1: 1806.1. Samples: 37188604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:25:13,788][122664] Avg episode reward: [(0, '72.750'), (1, '105.300')] [2023-10-10 19:25:13,789][123465] Saving new best policy, reward=105.300! [2023-10-10 19:25:14,348][123582] Updated weights for policy 0, policy_version 72673 (0.0009) [2023-10-10 19:25:14,727][123582] Updated weights for policy 0, policy_version 72683 (0.0010) [2023-10-10 19:25:15,090][123582] Updated weights for policy 0, policy_version 72693 (0.0008) [2023-10-10 19:25:15,452][123614] Updated weights for policy 1, policy_version 72580 (0.0007) [2023-10-10 19:25:15,460][123582] Updated weights for policy 0, policy_version 72703 (0.0007) [2023-10-10 19:25:15,819][123614] Updated weights for policy 1, policy_version 72590 (0.0009) [2023-10-10 19:25:16,178][123614] Updated weights for policy 1, policy_version 72600 (0.0010) [2023-10-10 19:25:18,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 148799488. Throughput: 0: 1821.1, 1: 1800.8. Samples: 37210874. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:25:18,789][122664] Avg episode reward: [(0, '71.170'), (1, '105.670')] [2023-10-10 19:25:18,790][123465] Saving new best policy, reward=105.670! [2023-10-10 19:25:19,100][123582] Updated weights for policy 0, policy_version 72713 (0.0009) [2023-10-10 19:25:19,474][123582] Updated weights for policy 0, policy_version 72723 (0.0011) [2023-10-10 19:25:19,834][123582] Updated weights for policy 0, policy_version 72733 (0.0010) [2023-10-10 19:25:19,988][123614] Updated weights for policy 1, policy_version 72610 (0.0012) [2023-10-10 19:25:20,355][123614] Updated weights for policy 1, policy_version 72620 (0.0009) [2023-10-10 19:25:20,719][123614] Updated weights for policy 1, policy_version 72630 (0.0009) [2023-10-10 19:25:21,093][123614] Updated weights for policy 1, policy_version 72640 (0.0008) [2023-10-10 19:25:23,561][123582] Updated weights for policy 0, policy_version 72743 (0.0009) [2023-10-10 19:25:23,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 148865024. Throughput: 0: 1823.0, 1: 1802.1. Samples: 37233576. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:25:23,789][122664] Avg episode reward: [(0, '73.050'), (1, '105.760')] [2023-10-10 19:25:23,799][123465] Saving new best policy, reward=105.760! [2023-10-10 19:25:23,937][123582] Updated weights for policy 0, policy_version 72753 (0.0008) [2023-10-10 19:25:24,306][123582] Updated weights for policy 0, policy_version 72763 (0.0008) [2023-10-10 19:25:24,805][123614] Updated weights for policy 1, policy_version 72650 (0.0010) [2023-10-10 19:25:25,178][123614] Updated weights for policy 1, policy_version 72660 (0.0008) [2023-10-10 19:25:25,552][123614] Updated weights for policy 1, policy_version 72670 (0.0007) [2023-10-10 19:25:27,926][123582] Updated weights for policy 0, policy_version 72773 (0.0009) [2023-10-10 19:25:28,299][123582] Updated weights for policy 0, policy_version 72783 (0.0008) [2023-10-10 19:25:28,671][123582] Updated weights for policy 0, policy_version 72793 (0.0007) [2023-10-10 19:25:28,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 148930560. Throughput: 0: 1822.5, 1: 1801.5. Samples: 37243592. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:25:28,789][122664] Avg episode reward: [(0, '70.420'), (1, '106.110')] [2023-10-10 19:25:28,790][123465] Saving new best policy, reward=106.110! [2023-10-10 19:25:29,284][123614] Updated weights for policy 1, policy_version 72680 (0.0009) [2023-10-10 19:25:29,659][123614] Updated weights for policy 1, policy_version 72690 (0.0008) [2023-10-10 19:25:30,023][123614] Updated weights for policy 1, policy_version 72700 (0.0011) [2023-10-10 19:25:32,430][123582] Updated weights for policy 0, policy_version 72803 (0.0007) [2023-10-10 19:25:32,797][123582] Updated weights for policy 0, policy_version 72813 (0.0008) [2023-10-10 19:25:33,175][123582] Updated weights for policy 0, policy_version 72823 (0.0007) [2023-10-10 19:25:33,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 149028864. Throughput: 0: 1818.5, 1: 1802.7. Samples: 37266052. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:25:33,789][122664] Avg episode reward: [(0, '75.620'), (1, '111.810')] [2023-10-10 19:25:33,827][123614] Updated weights for policy 1, policy_version 72710 (0.0008) [2023-10-10 19:25:34,204][123614] Updated weights for policy 1, policy_version 72720 (0.0009) [2023-10-10 19:25:34,571][123614] Updated weights for policy 1, policy_version 72730 (0.0011) [2023-10-10 19:25:34,788][123465] Saving new best policy, reward=111.810! [2023-10-10 19:25:36,828][123582] Updated weights for policy 0, policy_version 72833 (0.0007) [2023-10-10 19:25:37,195][123582] Updated weights for policy 0, policy_version 72843 (0.0008) [2023-10-10 19:25:37,563][123582] Updated weights for policy 0, policy_version 72853 (0.0008) [2023-10-10 19:25:37,930][123582] Updated weights for policy 0, policy_version 72863 (0.0010) [2023-10-10 19:25:38,367][123614] Updated weights for policy 1, policy_version 72740 (0.0011) [2023-10-10 19:25:38,729][123614] Updated weights for policy 1, policy_version 72750 (0.0009) [2023-10-10 19:25:38,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 149094400. Throughput: 0: 1815.4, 1: 1818.0. Samples: 37286740. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:25:38,788][122664] Avg episode reward: [(0, '75.270'), (1, '116.700')] [2023-10-10 19:25:39,098][123614] Updated weights for policy 1, policy_version 72760 (0.0008) [2023-10-10 19:25:39,394][123465] Saving new best policy, reward=116.700! [2023-10-10 19:25:41,736][123582] Updated weights for policy 0, policy_version 72873 (0.0009) [2023-10-10 19:25:42,113][123582] Updated weights for policy 0, policy_version 72883 (0.0008) [2023-10-10 19:25:42,483][123582] Updated weights for policy 0, policy_version 72893 (0.0008) [2023-10-10 19:25:42,904][123614] Updated weights for policy 1, policy_version 72770 (0.0008) [2023-10-10 19:25:43,269][123614] Updated weights for policy 1, policy_version 72780 (0.0008) [2023-10-10 19:25:43,637][123614] Updated weights for policy 1, policy_version 72790 (0.0008) [2023-10-10 19:25:43,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 149159936. Throughput: 0: 1814.2, 1: 1805.0. Samples: 37298424. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:25:43,789][122664] Avg episode reward: [(0, '76.310'), (1, '114.470')] [2023-10-10 19:25:44,004][123614] Updated weights for policy 1, policy_version 72800 (0.0009) [2023-10-10 19:25:46,236][123582] Updated weights for policy 0, policy_version 72903 (0.0009) [2023-10-10 19:25:46,609][123582] Updated weights for policy 0, policy_version 72913 (0.0008) [2023-10-10 19:25:46,981][123582] Updated weights for policy 0, policy_version 72923 (0.0007) [2023-10-10 19:25:47,670][123614] Updated weights for policy 1, policy_version 72810 (0.0008) [2023-10-10 19:25:48,039][123614] Updated weights for policy 1, policy_version 72820 (0.0009) [2023-10-10 19:25:48,406][123614] Updated weights for policy 1, policy_version 72830 (0.0008) [2023-10-10 19:25:48,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 149258240. Throughput: 0: 1810.4, 1: 1815.7. Samples: 37319194. Policy #0 lag: (min: 26.0, avg: 27.0, max: 45.0) [2023-10-10 19:25:48,788][122664] Avg episode reward: [(0, '74.600'), (1, '108.710')] [2023-10-10 19:25:50,666][123582] Updated weights for policy 0, policy_version 72933 (0.0007) [2023-10-10 19:25:51,036][123582] Updated weights for policy 0, policy_version 72943 (0.0008) [2023-10-10 19:25:51,403][123582] Updated weights for policy 0, policy_version 72953 (0.0008) [2023-10-10 19:25:52,103][123614] Updated weights for policy 1, policy_version 72840 (0.0010) [2023-10-10 19:25:52,476][123614] Updated weights for policy 1, policy_version 72850 (0.0008) [2023-10-10 19:25:52,847][123614] Updated weights for policy 1, policy_version 72860 (0.0007) [2023-10-10 19:25:53,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 149323776. Throughput: 0: 1812.6, 1: 1806.4. Samples: 37341058. Policy #0 lag: (min: 26.0, avg: 27.0, max: 45.0) [2023-10-10 19:25:53,789][122664] Avg episode reward: [(0, '79.890'), (1, '103.960')] [2023-10-10 19:25:53,800][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000072864_74612736.pth... [2023-10-10 19:25:53,801][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000072960_74711040.pth... [2023-10-10 19:25:53,840][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000071264_72974336.pth [2023-10-10 19:25:53,841][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000071168_72876032.pth [2023-10-10 19:25:55,113][123582] Updated weights for policy 0, policy_version 72963 (0.0008) [2023-10-10 19:25:55,483][123582] Updated weights for policy 0, policy_version 72973 (0.0007) [2023-10-10 19:25:55,865][123582] Updated weights for policy 0, policy_version 72983 (0.0007) [2023-10-10 19:25:56,524][123614] Updated weights for policy 1, policy_version 72870 (0.0009) [2023-10-10 19:25:56,909][123614] Updated weights for policy 1, policy_version 72880 (0.0011) [2023-10-10 19:25:57,279][123614] Updated weights for policy 1, policy_version 72890 (0.0010) [2023-10-10 19:25:58,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 149389312. Throughput: 0: 1809.6, 1: 1818.2. Samples: 37351858. Policy #0 lag: (min: 26.0, avg: 27.0, max: 45.0) [2023-10-10 19:25:58,788][122664] Avg episode reward: [(0, '79.610'), (1, '104.530')] [2023-10-10 19:25:59,346][123582] Updated weights for policy 0, policy_version 72993 (0.0008) [2023-10-10 19:25:59,717][123582] Updated weights for policy 0, policy_version 73003 (0.0009) [2023-10-10 19:26:00,091][123582] Updated weights for policy 0, policy_version 73013 (0.0008) [2023-10-10 19:26:00,467][123582] Updated weights for policy 0, policy_version 73023 (0.0007) [2023-10-10 19:26:01,039][123614] Updated weights for policy 1, policy_version 72900 (0.0009) [2023-10-10 19:26:01,407][123614] Updated weights for policy 1, policy_version 72910 (0.0008) [2023-10-10 19:26:01,788][123614] Updated weights for policy 1, policy_version 72920 (0.0008) [2023-10-10 19:26:03,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 149454848. Throughput: 0: 1818.3, 1: 1809.5. Samples: 37374126. Policy #0 lag: (min: 26.0, avg: 27.0, max: 45.0) [2023-10-10 19:26:03,789][122664] Avg episode reward: [(0, '79.310'), (1, '100.970')] [2023-10-10 19:26:04,053][123582] Updated weights for policy 0, policy_version 73033 (0.0009) [2023-10-10 19:26:04,419][123582] Updated weights for policy 0, policy_version 73043 (0.0010) [2023-10-10 19:26:04,795][123582] Updated weights for policy 0, policy_version 73053 (0.0008) [2023-10-10 19:26:05,403][123614] Updated weights for policy 1, policy_version 72930 (0.0007) [2023-10-10 19:26:05,773][123614] Updated weights for policy 1, policy_version 72940 (0.0009) [2023-10-10 19:26:06,133][123614] Updated weights for policy 1, policy_version 72950 (0.0008) [2023-10-10 19:26:06,499][123614] Updated weights for policy 1, policy_version 72960 (0.0008) [2023-10-10 19:26:08,460][123582] Updated weights for policy 0, policy_version 73063 (0.0009) [2023-10-10 19:26:08,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 149520384. Throughput: 0: 1813.9, 1: 1810.7. Samples: 37396682. Policy #0 lag: (min: 26.0, avg: 27.0, max: 45.0) [2023-10-10 19:26:08,788][122664] Avg episode reward: [(0, '81.880'), (1, '104.910')] [2023-10-10 19:26:08,833][123582] Updated weights for policy 0, policy_version 73073 (0.0008) [2023-10-10 19:26:09,220][123582] Updated weights for policy 0, policy_version 73083 (0.0008) [2023-10-10 19:26:10,267][123614] Updated weights for policy 1, policy_version 72970 (0.0008) [2023-10-10 19:26:10,640][123614] Updated weights for policy 1, policy_version 72980 (0.0008) [2023-10-10 19:26:11,010][123614] Updated weights for policy 1, policy_version 72990 (0.0008) [2023-10-10 19:26:12,909][123582] Updated weights for policy 0, policy_version 73093 (0.0008) [2023-10-10 19:26:13,284][123582] Updated weights for policy 0, policy_version 73103 (0.0007) [2023-10-10 19:26:13,661][123582] Updated weights for policy 0, policy_version 73113 (0.0008) [2023-10-10 19:26:13,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 149585920. Throughput: 0: 1816.3, 1: 1813.1. Samples: 37406914. Policy #0 lag: (min: 26.0, avg: 27.0, max: 45.0) [2023-10-10 19:26:13,789][122664] Avg episode reward: [(0, '84.470'), (1, '98.760')] [2023-10-10 19:26:14,558][123614] Updated weights for policy 1, policy_version 73000 (0.0008) [2023-10-10 19:26:14,923][123614] Updated weights for policy 1, policy_version 73010 (0.0009) [2023-10-10 19:26:15,295][123614] Updated weights for policy 1, policy_version 73020 (0.0008) [2023-10-10 19:26:17,269][123582] Updated weights for policy 0, policy_version 73123 (0.0008) [2023-10-10 19:26:17,643][123582] Updated weights for policy 0, policy_version 73133 (0.0007) [2023-10-10 19:26:18,005][123582] Updated weights for policy 0, policy_version 73143 (0.0010) [2023-10-10 19:26:18,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 149684224. Throughput: 0: 1818.0, 1: 1811.8. Samples: 37429394. Policy #0 lag: (min: 26.0, avg: 27.0, max: 45.0) [2023-10-10 19:26:18,789][122664] Avg episode reward: [(0, '83.330'), (1, '96.470')] [2023-10-10 19:26:19,064][123614] Updated weights for policy 1, policy_version 73030 (0.0008) [2023-10-10 19:26:19,436][123614] Updated weights for policy 1, policy_version 73040 (0.0009) [2023-10-10 19:26:19,806][123614] Updated weights for policy 1, policy_version 73050 (0.0008) [2023-10-10 19:26:21,736][123582] Updated weights for policy 0, policy_version 73153 (0.0010) [2023-10-10 19:26:22,107][123582] Updated weights for policy 0, policy_version 73163 (0.0008) [2023-10-10 19:26:22,476][123582] Updated weights for policy 0, policy_version 73173 (0.0009) [2023-10-10 19:26:22,845][123582] Updated weights for policy 0, policy_version 73183 (0.0010) [2023-10-10 19:26:23,514][123614] Updated weights for policy 1, policy_version 73060 (0.0009) [2023-10-10 19:26:23,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 149749760. Throughput: 0: 1818.3, 1: 1815.6. Samples: 37450268. Policy #0 lag: (min: 26.0, avg: 27.0, max: 45.0) [2023-10-10 19:26:23,789][122664] Avg episode reward: [(0, '80.610'), (1, '93.990')] [2023-10-10 19:26:23,882][123614] Updated weights for policy 1, policy_version 73070 (0.0010) [2023-10-10 19:26:24,260][123614] Updated weights for policy 1, policy_version 73080 (0.0010) [2023-10-10 19:26:26,707][123582] Updated weights for policy 0, policy_version 73193 (0.0008) [2023-10-10 19:26:27,084][123582] Updated weights for policy 0, policy_version 73203 (0.0007) [2023-10-10 19:26:27,444][123582] Updated weights for policy 0, policy_version 73213 (0.0007) [2023-10-10 19:26:27,985][123614] Updated weights for policy 1, policy_version 73090 (0.0007) [2023-10-10 19:26:28,361][123614] Updated weights for policy 1, policy_version 73100 (0.0008) [2023-10-10 19:26:28,733][123614] Updated weights for policy 1, policy_version 73110 (0.0007) [2023-10-10 19:26:28,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 149815296. Throughput: 0: 1817.9, 1: 1811.9. Samples: 37461764. Policy #0 lag: (min: 26.0, avg: 27.0, max: 45.0) [2023-10-10 19:26:28,789][122664] Avg episode reward: [(0, '77.870'), (1, '93.990')] [2023-10-10 19:26:29,097][123614] Updated weights for policy 1, policy_version 73120 (0.0007) [2023-10-10 19:26:31,032][123582] Updated weights for policy 0, policy_version 73223 (0.0009) [2023-10-10 19:26:31,408][123582] Updated weights for policy 0, policy_version 73233 (0.0008) [2023-10-10 19:26:31,772][123582] Updated weights for policy 0, policy_version 73243 (0.0008) [2023-10-10 19:26:32,798][123614] Updated weights for policy 1, policy_version 73130 (0.0011) [2023-10-10 19:26:33,161][123614] Updated weights for policy 1, policy_version 73140 (0.0008) [2023-10-10 19:26:33,529][123614] Updated weights for policy 1, policy_version 73150 (0.0007) [2023-10-10 19:26:33,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 149913600. Throughput: 0: 1818.9, 1: 1821.2. Samples: 37483000. Policy #0 lag: (min: 26.0, avg: 27.0, max: 45.0) [2023-10-10 19:26:33,789][122664] Avg episode reward: [(0, '78.770'), (1, '93.020')] [2023-10-10 19:26:35,512][123582] Updated weights for policy 0, policy_version 73253 (0.0008) [2023-10-10 19:26:35,894][123582] Updated weights for policy 0, policy_version 73263 (0.0007) [2023-10-10 19:26:36,258][123582] Updated weights for policy 0, policy_version 73273 (0.0007) [2023-10-10 19:26:37,174][123614] Updated weights for policy 1, policy_version 73160 (0.0008) [2023-10-10 19:26:37,535][123614] Updated weights for policy 1, policy_version 73170 (0.0007) [2023-10-10 19:26:37,901][123614] Updated weights for policy 1, policy_version 73180 (0.0010) [2023-10-10 19:26:38,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 149979136. Throughput: 0: 1823.1, 1: 1817.0. Samples: 37504862. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 19:26:38,789][122664] Avg episode reward: [(0, '78.640'), (1, '92.010')] [2023-10-10 19:26:39,995][123582] Updated weights for policy 0, policy_version 73283 (0.0007) [2023-10-10 19:26:40,364][123582] Updated weights for policy 0, policy_version 73293 (0.0009) [2023-10-10 19:26:40,735][123582] Updated weights for policy 0, policy_version 73303 (0.0007) [2023-10-10 19:26:41,522][123614] Updated weights for policy 1, policy_version 73190 (0.0009) [2023-10-10 19:26:41,900][123614] Updated weights for policy 1, policy_version 73200 (0.0008) [2023-10-10 19:26:42,268][123614] Updated weights for policy 1, policy_version 73210 (0.0007) [2023-10-10 19:26:43,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 150044672. Throughput: 0: 1820.4, 1: 1820.2. Samples: 37515684. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 19:26:43,789][122664] Avg episode reward: [(0, '76.680'), (1, '92.710')] [2023-10-10 19:26:44,360][123582] Updated weights for policy 0, policy_version 73313 (0.0010) [2023-10-10 19:26:44,739][123582] Updated weights for policy 0, policy_version 73323 (0.0010) [2023-10-10 19:26:45,095][123582] Updated weights for policy 0, policy_version 73333 (0.0009) [2023-10-10 19:26:45,469][123582] Updated weights for policy 0, policy_version 73343 (0.0011) [2023-10-10 19:26:45,753][123614] Updated weights for policy 1, policy_version 73220 (0.0009) [2023-10-10 19:26:46,115][123614] Updated weights for policy 1, policy_version 73230 (0.0007) [2023-10-10 19:26:46,485][123614] Updated weights for policy 1, policy_version 73240 (0.0010) [2023-10-10 19:26:48,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 150110208. Throughput: 0: 1813.5, 1: 1819.6. Samples: 37537612. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 19:26:48,789][122664] Avg episode reward: [(0, '77.840'), (1, '94.570')] [2023-10-10 19:26:49,159][123582] Updated weights for policy 0, policy_version 73353 (0.0008) [2023-10-10 19:26:49,533][123582] Updated weights for policy 0, policy_version 73363 (0.0008) [2023-10-10 19:26:49,895][123582] Updated weights for policy 0, policy_version 73373 (0.0010) [2023-10-10 19:26:50,178][123614] Updated weights for policy 1, policy_version 73250 (0.0009) [2023-10-10 19:26:50,534][123614] Updated weights for policy 1, policy_version 73260 (0.0010) [2023-10-10 19:26:50,905][123614] Updated weights for policy 1, policy_version 73270 (0.0010) [2023-10-10 19:26:51,275][123614] Updated weights for policy 1, policy_version 73280 (0.0009) [2023-10-10 19:26:53,557][123582] Updated weights for policy 0, policy_version 73383 (0.0010) [2023-10-10 19:26:53,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 150175744. Throughput: 0: 1819.0, 1: 1816.4. Samples: 37560276. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 19:26:53,788][122664] Avg episode reward: [(0, '80.800'), (1, '96.120')] [2023-10-10 19:26:53,919][123582] Updated weights for policy 0, policy_version 73393 (0.0009) [2023-10-10 19:26:54,300][123582] Updated weights for policy 0, policy_version 73403 (0.0007) [2023-10-10 19:26:55,197][123614] Updated weights for policy 1, policy_version 73290 (0.0009) [2023-10-10 19:26:55,560][123614] Updated weights for policy 1, policy_version 73300 (0.0011) [2023-10-10 19:26:55,940][123614] Updated weights for policy 1, policy_version 73310 (0.0010) [2023-10-10 19:26:57,968][123582] Updated weights for policy 0, policy_version 73413 (0.0010) [2023-10-10 19:26:58,333][123582] Updated weights for policy 0, policy_version 73423 (0.0009) [2023-10-10 19:26:58,702][123582] Updated weights for policy 0, policy_version 73433 (0.0007) [2023-10-10 19:26:58,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 150241280. Throughput: 0: 1817.5, 1: 1815.0. Samples: 37570374. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 19:26:58,789][122664] Avg episode reward: [(0, '77.330'), (1, '96.660')] [2023-10-10 19:26:59,589][123614] Updated weights for policy 1, policy_version 73320 (0.0008) [2023-10-10 19:26:59,951][123614] Updated weights for policy 1, policy_version 73330 (0.0007) [2023-10-10 19:27:00,322][123614] Updated weights for policy 1, policy_version 73340 (0.0009) [2023-10-10 19:27:02,402][123582] Updated weights for policy 0, policy_version 73443 (0.0007) [2023-10-10 19:27:02,781][123582] Updated weights for policy 0, policy_version 73453 (0.0008) [2023-10-10 19:27:03,141][123582] Updated weights for policy 0, policy_version 73463 (0.0011) [2023-10-10 19:27:03,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 150339584. Throughput: 0: 1820.2, 1: 1815.2. Samples: 37592986. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 19:27:03,788][122664] Avg episode reward: [(0, '73.530'), (1, '93.440')] [2023-10-10 19:27:04,015][123614] Updated weights for policy 1, policy_version 73350 (0.0008) [2023-10-10 19:27:04,387][123614] Updated weights for policy 1, policy_version 73360 (0.0008) [2023-10-10 19:27:04,748][123614] Updated weights for policy 1, policy_version 73370 (0.0007) [2023-10-10 19:27:06,848][123582] Updated weights for policy 0, policy_version 73473 (0.0010) [2023-10-10 19:27:07,216][123582] Updated weights for policy 0, policy_version 73483 (0.0008) [2023-10-10 19:27:07,601][123582] Updated weights for policy 0, policy_version 73493 (0.0010) [2023-10-10 19:27:07,968][123582] Updated weights for policy 0, policy_version 73503 (0.0011) [2023-10-10 19:27:08,421][123614] Updated weights for policy 1, policy_version 73380 (0.0007) [2023-10-10 19:27:08,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 150405120. Throughput: 0: 1821.8, 1: 1821.4. Samples: 37614212. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 19:27:08,788][122664] Avg episode reward: [(0, '70.360'), (1, '92.880')] [2023-10-10 19:27:08,792][123614] Updated weights for policy 1, policy_version 73390 (0.0008) [2023-10-10 19:27:09,163][123614] Updated weights for policy 1, policy_version 73400 (0.0010) [2023-10-10 19:27:11,713][123582] Updated weights for policy 0, policy_version 73513 (0.0008) [2023-10-10 19:27:12,081][123582] Updated weights for policy 0, policy_version 73523 (0.0010) [2023-10-10 19:27:12,450][123582] Updated weights for policy 0, policy_version 73533 (0.0012) [2023-10-10 19:27:12,827][123614] Updated weights for policy 1, policy_version 73410 (0.0008) [2023-10-10 19:27:13,194][123614] Updated weights for policy 1, policy_version 73420 (0.0009) [2023-10-10 19:27:13,559][123614] Updated weights for policy 1, policy_version 73430 (0.0009) [2023-10-10 19:27:13,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 150470656. Throughput: 0: 1822.7, 1: 1825.3. Samples: 37625924. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 19:27:13,788][122664] Avg episode reward: [(0, '71.120'), (1, '86.910')] [2023-10-10 19:27:13,929][123614] Updated weights for policy 1, policy_version 73440 (0.0008) [2023-10-10 19:27:16,162][123582] Updated weights for policy 0, policy_version 73543 (0.0011) [2023-10-10 19:27:16,534][123582] Updated weights for policy 0, policy_version 73553 (0.0009) [2023-10-10 19:27:16,905][123582] Updated weights for policy 0, policy_version 73563 (0.0010) [2023-10-10 19:27:17,751][123614] Updated weights for policy 1, policy_version 73450 (0.0009) [2023-10-10 19:27:18,122][123614] Updated weights for policy 1, policy_version 73460 (0.0010) [2023-10-10 19:27:18,491][123614] Updated weights for policy 1, policy_version 73470 (0.0008) [2023-10-10 19:27:18,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 150568960. Throughput: 0: 1820.9, 1: 1817.7. Samples: 37646734. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 19:27:18,788][122664] Avg episode reward: [(0, '72.550'), (1, '91.360')] [2023-10-10 19:27:20,668][123582] Updated weights for policy 0, policy_version 73573 (0.0007) [2023-10-10 19:27:21,041][123582] Updated weights for policy 0, policy_version 73583 (0.0007) [2023-10-10 19:27:21,413][123582] Updated weights for policy 0, policy_version 73593 (0.0007) [2023-10-10 19:27:22,289][123614] Updated weights for policy 1, policy_version 73480 (0.0008) [2023-10-10 19:27:22,657][123614] Updated weights for policy 1, policy_version 73490 (0.0008) [2023-10-10 19:27:23,035][123614] Updated weights for policy 1, policy_version 73500 (0.0008) [2023-10-10 19:27:23,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 150634496. Throughput: 0: 1819.6, 1: 1821.3. Samples: 37668702. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 19:27:23,788][122664] Avg episode reward: [(0, '72.970'), (1, '93.870')] [2023-10-10 19:27:24,964][123582] Updated weights for policy 0, policy_version 73603 (0.0008) [2023-10-10 19:27:25,335][123582] Updated weights for policy 0, policy_version 73613 (0.0008) [2023-10-10 19:27:25,703][123582] Updated weights for policy 0, policy_version 73623 (0.0007) [2023-10-10 19:27:26,568][123614] Updated weights for policy 1, policy_version 73510 (0.0007) [2023-10-10 19:27:26,936][123614] Updated weights for policy 1, policy_version 73520 (0.0008) [2023-10-10 19:27:27,308][123614] Updated weights for policy 1, policy_version 73530 (0.0011) [2023-10-10 19:27:28,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 150700032. Throughput: 0: 1825.2, 1: 1821.4. Samples: 37679780. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:27:28,789][122664] Avg episode reward: [(0, '75.690'), (1, '95.070')] [2023-10-10 19:27:29,290][123582] Updated weights for policy 0, policy_version 73633 (0.0007) [2023-10-10 19:27:29,652][123582] Updated weights for policy 0, policy_version 73643 (0.0009) [2023-10-10 19:27:30,026][123582] Updated weights for policy 0, policy_version 73653 (0.0009) [2023-10-10 19:27:30,403][123582] Updated weights for policy 0, policy_version 73663 (0.0008) [2023-10-10 19:27:31,104][123614] Updated weights for policy 1, policy_version 73540 (0.0007) [2023-10-10 19:27:31,477][123614] Updated weights for policy 1, policy_version 73550 (0.0007) [2023-10-10 19:27:31,839][123614] Updated weights for policy 1, policy_version 73560 (0.0009) [2023-10-10 19:27:33,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 150765568. Throughput: 0: 1831.9, 1: 1817.9. Samples: 37701854. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:27:33,788][122664] Avg episode reward: [(0, '75.710'), (1, '94.300')] [2023-10-10 19:27:34,082][123582] Updated weights for policy 0, policy_version 73673 (0.0009) [2023-10-10 19:27:34,442][123582] Updated weights for policy 0, policy_version 73683 (0.0008) [2023-10-10 19:27:34,821][123582] Updated weights for policy 0, policy_version 73693 (0.0010) [2023-10-10 19:27:35,424][123614] Updated weights for policy 1, policy_version 73570 (0.0009) [2023-10-10 19:27:35,793][123614] Updated weights for policy 1, policy_version 73580 (0.0009) [2023-10-10 19:27:36,162][123614] Updated weights for policy 1, policy_version 73590 (0.0008) [2023-10-10 19:27:36,525][123614] Updated weights for policy 1, policy_version 73600 (0.0010) [2023-10-10 19:27:38,339][123582] Updated weights for policy 0, policy_version 73703 (0.0008) [2023-10-10 19:27:38,720][123582] Updated weights for policy 0, policy_version 73713 (0.0009) [2023-10-10 19:27:38,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 150831104. Throughput: 0: 1828.3, 1: 1813.1. Samples: 37724136. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:27:38,789][122664] Avg episode reward: [(0, '80.290'), (1, '95.660')] [2023-10-10 19:27:39,087][123582] Updated weights for policy 0, policy_version 73723 (0.0007) [2023-10-10 19:27:40,421][123614] Updated weights for policy 1, policy_version 73610 (0.0007) [2023-10-10 19:27:40,796][123614] Updated weights for policy 1, policy_version 73620 (0.0009) [2023-10-10 19:27:41,164][123614] Updated weights for policy 1, policy_version 73630 (0.0008) [2023-10-10 19:27:42,787][123582] Updated weights for policy 0, policy_version 73733 (0.0007) [2023-10-10 19:27:43,159][123582] Updated weights for policy 0, policy_version 73743 (0.0007) [2023-10-10 19:27:43,527][123582] Updated weights for policy 0, policy_version 73753 (0.0009) [2023-10-10 19:27:43,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 150929408. Throughput: 0: 1833.3, 1: 1811.2. Samples: 37734378. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:27:43,789][122664] Avg episode reward: [(0, '89.320'), (1, '93.840')] [2023-10-10 19:27:44,781][123614] Updated weights for policy 1, policy_version 73640 (0.0007) [2023-10-10 19:27:45,151][123614] Updated weights for policy 1, policy_version 73650 (0.0009) [2023-10-10 19:27:45,514][123614] Updated weights for policy 1, policy_version 73660 (0.0009) [2023-10-10 19:27:46,979][123582] Updated weights for policy 0, policy_version 73763 (0.0008) [2023-10-10 19:27:47,350][123582] Updated weights for policy 0, policy_version 73773 (0.0010) [2023-10-10 19:27:47,731][123582] Updated weights for policy 0, policy_version 73783 (0.0010) [2023-10-10 19:27:48,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 150994944. Throughput: 0: 1827.5, 1: 1818.2. Samples: 37757044. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:27:48,789][122664] Avg episode reward: [(0, '87.890'), (1, '96.130')] [2023-10-10 19:27:49,227][123614] Updated weights for policy 1, policy_version 73670 (0.0008) [2023-10-10 19:27:49,599][123614] Updated weights for policy 1, policy_version 73680 (0.0008) [2023-10-10 19:27:49,971][123614] Updated weights for policy 1, policy_version 73690 (0.0008) [2023-10-10 19:27:51,447][123582] Updated weights for policy 0, policy_version 73793 (0.0010) [2023-10-10 19:27:51,816][123582] Updated weights for policy 0, policy_version 73803 (0.0007) [2023-10-10 19:27:52,184][123582] Updated weights for policy 0, policy_version 73813 (0.0007) [2023-10-10 19:27:52,564][123582] Updated weights for policy 0, policy_version 73823 (0.0009) [2023-10-10 19:27:53,724][123614] Updated weights for policy 1, policy_version 73700 (0.0007) [2023-10-10 19:27:53,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 151060480. Throughput: 0: 1834.0, 1: 1819.6. Samples: 37778628. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:27:53,789][122664] Avg episode reward: [(0, '87.380'), (1, '96.040')] [2023-10-10 19:27:53,797][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000073824_75595776.pth... [2023-10-10 19:27:53,843][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000072128_73859072.pth [2023-10-10 19:27:54,098][123614] Updated weights for policy 1, policy_version 73710 (0.0007) [2023-10-10 19:27:54,463][123614] Updated weights for policy 1, policy_version 73720 (0.0008) [2023-10-10 19:27:54,760][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000073728_75497472.pth... [2023-10-10 19:27:54,797][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000072032_73760768.pth [2023-10-10 19:27:56,159][123582] Updated weights for policy 0, policy_version 73833 (0.0009) [2023-10-10 19:27:56,523][123582] Updated weights for policy 0, policy_version 73843 (0.0007) [2023-10-10 19:27:56,897][123582] Updated weights for policy 0, policy_version 73853 (0.0007) [2023-10-10 19:27:58,138][123614] Updated weights for policy 1, policy_version 73730 (0.0010) [2023-10-10 19:27:58,502][123614] Updated weights for policy 1, policy_version 73740 (0.0008) [2023-10-10 19:27:58,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 151126016. Throughput: 0: 1822.2, 1: 1810.9. Samples: 37789414. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:27:58,789][122664] Avg episode reward: [(0, '87.680'), (1, '94.050')] [2023-10-10 19:27:58,873][123614] Updated weights for policy 1, policy_version 73750 (0.0007) [2023-10-10 19:27:59,242][123614] Updated weights for policy 1, policy_version 73760 (0.0008) [2023-10-10 19:28:00,859][123582] Updated weights for policy 0, policy_version 73863 (0.0008) [2023-10-10 19:28:01,225][123582] Updated weights for policy 0, policy_version 73873 (0.0009) [2023-10-10 19:28:01,589][123582] Updated weights for policy 0, policy_version 73883 (0.0008) [2023-10-10 19:28:02,855][123614] Updated weights for policy 1, policy_version 73770 (0.0007) [2023-10-10 19:28:03,216][123614] Updated weights for policy 1, policy_version 73780 (0.0009) [2023-10-10 19:28:03,585][123614] Updated weights for policy 1, policy_version 73790 (0.0009) [2023-10-10 19:28:03,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 151224320. Throughput: 0: 1833.2, 1: 1818.5. Samples: 37811062. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:28:03,789][122664] Avg episode reward: [(0, '87.300'), (1, '93.210')] [2023-10-10 19:28:05,235][123582] Updated weights for policy 0, policy_version 73893 (0.0008) [2023-10-10 19:28:05,605][123582] Updated weights for policy 0, policy_version 73903 (0.0007) [2023-10-10 19:28:05,972][123582] Updated weights for policy 0, policy_version 73913 (0.0007) [2023-10-10 19:28:07,339][123614] Updated weights for policy 1, policy_version 73800 (0.0007) [2023-10-10 19:28:07,715][123614] Updated weights for policy 1, policy_version 73810 (0.0007) [2023-10-10 19:28:08,075][123614] Updated weights for policy 1, policy_version 73820 (0.0010) [2023-10-10 19:28:08,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 151289856. Throughput: 0: 1828.9, 1: 1810.1. Samples: 37832460. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:28:08,789][122664] Avg episode reward: [(0, '80.380'), (1, '94.250')] [2023-10-10 19:28:09,743][123582] Updated weights for policy 0, policy_version 73923 (0.0008) [2023-10-10 19:28:10,130][123582] Updated weights for policy 0, policy_version 73933 (0.0010) [2023-10-10 19:28:10,503][123582] Updated weights for policy 0, policy_version 73943 (0.0010) [2023-10-10 19:28:11,797][123614] Updated weights for policy 1, policy_version 73830 (0.0009) [2023-10-10 19:28:12,161][123614] Updated weights for policy 1, policy_version 73840 (0.0009) [2023-10-10 19:28:12,526][123614] Updated weights for policy 1, policy_version 73850 (0.0007) [2023-10-10 19:28:13,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 151355392. Throughput: 0: 1829.0, 1: 1812.5. Samples: 37843646. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:28:13,789][122664] Avg episode reward: [(0, '81.580'), (1, '90.010')] [2023-10-10 19:28:14,192][123582] Updated weights for policy 0, policy_version 73953 (0.0009) [2023-10-10 19:28:14,550][123582] Updated weights for policy 0, policy_version 73963 (0.0010) [2023-10-10 19:28:14,923][123582] Updated weights for policy 0, policy_version 73973 (0.0009) [2023-10-10 19:28:15,301][123582] Updated weights for policy 0, policy_version 73983 (0.0010) [2023-10-10 19:28:16,490][123614] Updated weights for policy 1, policy_version 73860 (0.0010) [2023-10-10 19:28:16,863][123614] Updated weights for policy 1, policy_version 73870 (0.0011) [2023-10-10 19:28:17,238][123614] Updated weights for policy 1, policy_version 73880 (0.0008) [2023-10-10 19:28:18,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 151420928. Throughput: 0: 1813.2, 1: 1800.3. Samples: 37864464. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-10 19:28:18,789][122664] Avg episode reward: [(0, '87.140'), (1, '87.660')] [2023-10-10 19:28:19,093][123582] Updated weights for policy 0, policy_version 73993 (0.0010) [2023-10-10 19:28:19,469][123582] Updated weights for policy 0, policy_version 74003 (0.0009) [2023-10-10 19:28:19,838][123582] Updated weights for policy 0, policy_version 74013 (0.0009) [2023-10-10 19:28:20,896][123614] Updated weights for policy 1, policy_version 73890 (0.0007) [2023-10-10 19:28:21,269][123614] Updated weights for policy 1, policy_version 73900 (0.0009) [2023-10-10 19:28:21,639][123614] Updated weights for policy 1, policy_version 73910 (0.0008) [2023-10-10 19:28:22,013][123614] Updated weights for policy 1, policy_version 73920 (0.0007) [2023-10-10 19:28:23,660][123582] Updated weights for policy 0, policy_version 74023 (0.0007) [2023-10-10 19:28:23,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 151486464. Throughput: 0: 1815.2, 1: 1805.8. Samples: 37887080. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-10 19:28:23,789][122664] Avg episode reward: [(0, '87.500'), (1, '88.640')] [2023-10-10 19:28:24,036][123582] Updated weights for policy 0, policy_version 74033 (0.0008) [2023-10-10 19:28:24,408][123582] Updated weights for policy 0, policy_version 74043 (0.0007) [2023-10-10 19:28:25,670][123614] Updated weights for policy 1, policy_version 73930 (0.0008) [2023-10-10 19:28:26,036][123614] Updated weights for policy 1, policy_version 73940 (0.0007) [2023-10-10 19:28:26,403][123614] Updated weights for policy 1, policy_version 73950 (0.0008) [2023-10-10 19:28:27,958][123582] Updated weights for policy 0, policy_version 74053 (0.0009) [2023-10-10 19:28:28,323][123582] Updated weights for policy 0, policy_version 74063 (0.0010) [2023-10-10 19:28:28,693][123582] Updated weights for policy 0, policy_version 74073 (0.0011) [2023-10-10 19:28:28,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 151552000. Throughput: 0: 1808.9, 1: 1809.4. Samples: 37897204. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-10 19:28:28,788][122664] Avg episode reward: [(0, '84.510'), (1, '84.550')] [2023-10-10 19:28:30,183][123614] Updated weights for policy 1, policy_version 73960 (0.0009) [2023-10-10 19:28:30,550][123614] Updated weights for policy 1, policy_version 73970 (0.0009) [2023-10-10 19:28:30,918][123614] Updated weights for policy 1, policy_version 73980 (0.0007) [2023-10-10 19:28:32,327][123582] Updated weights for policy 0, policy_version 74083 (0.0010) [2023-10-10 19:28:32,694][123582] Updated weights for policy 0, policy_version 74093 (0.0008) [2023-10-10 19:28:33,066][123582] Updated weights for policy 0, policy_version 74103 (0.0009) [2023-10-10 19:28:33,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 151650304. Throughput: 0: 1816.4, 1: 1800.2. Samples: 37919794. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-10 19:28:33,788][122664] Avg episode reward: [(0, '86.160'), (1, '85.280')] [2023-10-10 19:28:34,673][123614] Updated weights for policy 1, policy_version 73990 (0.0008) [2023-10-10 19:28:35,042][123614] Updated weights for policy 1, policy_version 74000 (0.0008) [2023-10-10 19:28:35,412][123614] Updated weights for policy 1, policy_version 74010 (0.0010) [2023-10-10 19:28:36,705][123582] Updated weights for policy 0, policy_version 74113 (0.0007) [2023-10-10 19:28:37,075][123582] Updated weights for policy 0, policy_version 74123 (0.0009) [2023-10-10 19:28:37,444][123582] Updated weights for policy 0, policy_version 74133 (0.0010) [2023-10-10 19:28:37,814][123582] Updated weights for policy 0, policy_version 74143 (0.0007) [2023-10-10 19:28:38,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 151715840. Throughput: 0: 1812.0, 1: 1799.6. Samples: 37941152. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-10 19:28:38,788][122664] Avg episode reward: [(0, '79.960'), (1, '79.300')] [2023-10-10 19:28:39,221][123614] Updated weights for policy 1, policy_version 74020 (0.0010) [2023-10-10 19:28:39,590][123614] Updated weights for policy 1, policy_version 74030 (0.0007) [2023-10-10 19:28:39,960][123614] Updated weights for policy 1, policy_version 74040 (0.0007) [2023-10-10 19:28:41,380][123582] Updated weights for policy 0, policy_version 74153 (0.0007) [2023-10-10 19:28:41,754][123582] Updated weights for policy 0, policy_version 74163 (0.0010) [2023-10-10 19:28:42,117][123582] Updated weights for policy 0, policy_version 74173 (0.0007) [2023-10-10 19:28:43,531][123614] Updated weights for policy 1, policy_version 74050 (0.0009) [2023-10-10 19:28:43,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 151781376. Throughput: 0: 1817.5, 1: 1799.5. Samples: 37952178. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-10 19:28:43,788][122664] Avg episode reward: [(0, '77.860'), (1, '85.050')] [2023-10-10 19:28:43,897][123614] Updated weights for policy 1, policy_version 74060 (0.0007) [2023-10-10 19:28:44,273][123614] Updated weights for policy 1, policy_version 74070 (0.0010) [2023-10-10 19:28:44,644][123614] Updated weights for policy 1, policy_version 74080 (0.0008) [2023-10-10 19:28:45,849][123582] Updated weights for policy 0, policy_version 74183 (0.0008) [2023-10-10 19:28:46,231][123582] Updated weights for policy 0, policy_version 74193 (0.0008) [2023-10-10 19:28:46,607][123582] Updated weights for policy 0, policy_version 74203 (0.0009) [2023-10-10 19:28:48,270][123614] Updated weights for policy 1, policy_version 74090 (0.0007) [2023-10-10 19:28:48,642][123614] Updated weights for policy 1, policy_version 74100 (0.0009) [2023-10-10 19:28:48,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 151846912. Throughput: 0: 1813.7, 1: 1810.6. Samples: 37974158. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-10 19:28:48,788][122664] Avg episode reward: [(0, '72.720'), (1, '83.110')] [2023-10-10 19:28:49,016][123614] Updated weights for policy 1, policy_version 74110 (0.0008) [2023-10-10 19:28:50,489][123582] Updated weights for policy 0, policy_version 74213 (0.0008) [2023-10-10 19:28:50,863][123582] Updated weights for policy 0, policy_version 74223 (0.0008) [2023-10-10 19:28:51,223][123582] Updated weights for policy 0, policy_version 74233 (0.0008) [2023-10-10 19:28:52,715][123614] Updated weights for policy 1, policy_version 74120 (0.0010) [2023-10-10 19:28:53,083][123614] Updated weights for policy 1, policy_version 74130 (0.0010) [2023-10-10 19:28:53,462][123614] Updated weights for policy 1, policy_version 74140 (0.0008) [2023-10-10 19:28:53,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 151945216. Throughput: 0: 1808.7, 1: 1800.1. Samples: 37994856. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-10 19:28:53,788][122664] Avg episode reward: [(0, '66.750'), (1, '82.920')] [2023-10-10 19:28:54,913][123582] Updated weights for policy 0, policy_version 74243 (0.0007) [2023-10-10 19:28:55,286][123582] Updated weights for policy 0, policy_version 74253 (0.0011) [2023-10-10 19:28:55,664][123582] Updated weights for policy 0, policy_version 74263 (0.0011) [2023-10-10 19:28:57,187][123614] Updated weights for policy 1, policy_version 74150 (0.0009) [2023-10-10 19:28:57,559][123614] Updated weights for policy 1, policy_version 74160 (0.0007) [2023-10-10 19:28:57,921][123614] Updated weights for policy 1, policy_version 74170 (0.0008) [2023-10-10 19:28:58,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 152010752. Throughput: 0: 1806.9, 1: 1804.6. Samples: 38006164. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-10 19:28:58,789][122664] Avg episode reward: [(0, '67.060'), (1, '82.520')] [2023-10-10 19:28:59,296][123582] Updated weights for policy 0, policy_version 74273 (0.0010) [2023-10-10 19:28:59,674][123582] Updated weights for policy 0, policy_version 74283 (0.0008) [2023-10-10 19:29:00,049][123582] Updated weights for policy 0, policy_version 74293 (0.0011) [2023-10-10 19:29:00,422][123582] Updated weights for policy 0, policy_version 74303 (0.0009) [2023-10-10 19:29:01,597][123614] Updated weights for policy 1, policy_version 74180 (0.0008) [2023-10-10 19:29:01,967][123614] Updated weights for policy 1, policy_version 74190 (0.0009) [2023-10-10 19:29:02,326][123614] Updated weights for policy 1, policy_version 74200 (0.0007) [2023-10-10 19:29:03,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 152076288. Throughput: 0: 1817.7, 1: 1803.6. Samples: 38027424. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-10 19:29:03,788][122664] Avg episode reward: [(0, '74.790'), (1, '81.450')] [2023-10-10 19:29:04,071][123582] Updated weights for policy 0, policy_version 74313 (0.0009) [2023-10-10 19:29:04,438][123582] Updated weights for policy 0, policy_version 74323 (0.0008) [2023-10-10 19:29:04,812][123582] Updated weights for policy 0, policy_version 74333 (0.0010) [2023-10-10 19:29:06,112][123614] Updated weights for policy 1, policy_version 74210 (0.0008) [2023-10-10 19:29:06,478][123614] Updated weights for policy 1, policy_version 74220 (0.0008) [2023-10-10 19:29:06,836][123614] Updated weights for policy 1, policy_version 74230 (0.0008) [2023-10-10 19:29:07,202][123614] Updated weights for policy 1, policy_version 74240 (0.0008) [2023-10-10 19:29:08,521][123582] Updated weights for policy 0, policy_version 74343 (0.0011) [2023-10-10 19:29:08,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 152141824. Throughput: 0: 1817.9, 1: 1804.4. Samples: 38050084. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-10 19:29:08,788][122664] Avg episode reward: [(0, '71.590'), (1, '82.060')] [2023-10-10 19:29:08,891][123582] Updated weights for policy 0, policy_version 74353 (0.0008) [2023-10-10 19:29:09,260][123582] Updated weights for policy 0, policy_version 74363 (0.0009) [2023-10-10 19:29:10,908][123614] Updated weights for policy 1, policy_version 74250 (0.0008) [2023-10-10 19:29:11,273][123614] Updated weights for policy 1, policy_version 74260 (0.0007) [2023-10-10 19:29:11,646][123614] Updated weights for policy 1, policy_version 74270 (0.0007) [2023-10-10 19:29:12,954][123582] Updated weights for policy 0, policy_version 74373 (0.0009) [2023-10-10 19:29:13,335][123582] Updated weights for policy 0, policy_version 74383 (0.0008) [2023-10-10 19:29:13,704][123582] Updated weights for policy 0, policy_version 74393 (0.0007) [2023-10-10 19:29:13,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 152207360. Throughput: 0: 1820.5, 1: 1804.9. Samples: 38060348. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-10 19:29:13,788][122664] Avg episode reward: [(0, '70.970'), (1, '81.600')] [2023-10-10 19:29:15,328][123614] Updated weights for policy 1, policy_version 74280 (0.0009) [2023-10-10 19:29:15,701][123614] Updated weights for policy 1, policy_version 74290 (0.0010) [2023-10-10 19:29:16,078][123614] Updated weights for policy 1, policy_version 74300 (0.0008) [2023-10-10 19:29:17,516][123582] Updated weights for policy 0, policy_version 74403 (0.0009) [2023-10-10 19:29:17,893][123582] Updated weights for policy 0, policy_version 74413 (0.0007) [2023-10-10 19:29:18,265][123582] Updated weights for policy 0, policy_version 74423 (0.0008) [2023-10-10 19:29:18,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 152305664. Throughput: 0: 1818.0, 1: 1806.0. Samples: 38082874. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-10 19:29:18,788][122664] Avg episode reward: [(0, '72.060'), (1, '81.590')] [2023-10-10 19:29:19,778][123614] Updated weights for policy 1, policy_version 74310 (0.0008) [2023-10-10 19:29:20,149][123614] Updated weights for policy 1, policy_version 74320 (0.0007) [2023-10-10 19:29:20,514][123614] Updated weights for policy 1, policy_version 74330 (0.0007) [2023-10-10 19:29:21,953][123582] Updated weights for policy 0, policy_version 74433 (0.0010) [2023-10-10 19:29:22,319][123582] Updated weights for policy 0, policy_version 74443 (0.0010) [2023-10-10 19:29:22,686][123582] Updated weights for policy 0, policy_version 74453 (0.0011) [2023-10-10 19:29:23,051][123582] Updated weights for policy 0, policy_version 74463 (0.0009) [2023-10-10 19:29:23,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 152371200. Throughput: 0: 1811.7, 1: 1813.5. Samples: 38104288. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-10 19:29:23,789][122664] Avg episode reward: [(0, '71.780'), (1, '82.420')] [2023-10-10 19:29:24,082][123614] Updated weights for policy 1, policy_version 74340 (0.0008) [2023-10-10 19:29:24,450][123614] Updated weights for policy 1, policy_version 74350 (0.0009) [2023-10-10 19:29:24,813][123614] Updated weights for policy 1, policy_version 74360 (0.0009) [2023-10-10 19:29:26,727][123582] Updated weights for policy 0, policy_version 74473 (0.0009) [2023-10-10 19:29:27,093][123582] Updated weights for policy 0, policy_version 74483 (0.0008) [2023-10-10 19:29:27,457][123582] Updated weights for policy 0, policy_version 74493 (0.0008) [2023-10-10 19:29:28,467][123614] Updated weights for policy 1, policy_version 74370 (0.0009) [2023-10-10 19:29:28,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 152436736. Throughput: 0: 1821.8, 1: 1809.1. Samples: 38115568. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-10 19:29:28,788][122664] Avg episode reward: [(0, '72.950'), (1, '83.330')] [2023-10-10 19:29:28,841][123614] Updated weights for policy 1, policy_version 74380 (0.0008) [2023-10-10 19:29:29,208][123614] Updated weights for policy 1, policy_version 74390 (0.0008) [2023-10-10 19:29:29,569][123614] Updated weights for policy 1, policy_version 74400 (0.0009) [2023-10-10 19:29:31,002][123582] Updated weights for policy 0, policy_version 74503 (0.0010) [2023-10-10 19:29:31,368][123582] Updated weights for policy 0, policy_version 74513 (0.0008) [2023-10-10 19:29:31,739][123582] Updated weights for policy 0, policy_version 74523 (0.0008) [2023-10-10 19:29:33,206][123614] Updated weights for policy 1, policy_version 74410 (0.0009) [2023-10-10 19:29:33,577][123614] Updated weights for policy 1, policy_version 74420 (0.0007) [2023-10-10 19:29:33,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 152502272. Throughput: 0: 1818.0, 1: 1807.5. Samples: 38137306. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-10 19:29:33,789][122664] Avg episode reward: [(0, '76.700'), (1, '82.860')] [2023-10-10 19:29:33,949][123614] Updated weights for policy 1, policy_version 74430 (0.0009) [2023-10-10 19:29:35,285][123582] Updated weights for policy 0, policy_version 74533 (0.0007) [2023-10-10 19:29:35,661][123582] Updated weights for policy 0, policy_version 74543 (0.0008) [2023-10-10 19:29:36,039][123582] Updated weights for policy 0, policy_version 74553 (0.0008) [2023-10-10 19:29:37,520][123614] Updated weights for policy 1, policy_version 74440 (0.0009) [2023-10-10 19:29:37,895][123614] Updated weights for policy 1, policy_version 74450 (0.0009) [2023-10-10 19:29:38,270][123614] Updated weights for policy 1, policy_version 74460 (0.0009) [2023-10-10 19:29:38,788][122664] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 152600576. Throughput: 0: 1828.7, 1: 1817.0. Samples: 38158910. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-10 19:29:38,789][122664] Avg episode reward: [(0, '77.440'), (1, '82.370')] [2023-10-10 19:29:39,657][123582] Updated weights for policy 0, policy_version 74563 (0.0010) [2023-10-10 19:29:40,027][123582] Updated weights for policy 0, policy_version 74573 (0.0010) [2023-10-10 19:29:40,392][123582] Updated weights for policy 0, policy_version 74583 (0.0008) [2023-10-10 19:29:42,047][123614] Updated weights for policy 1, policy_version 74470 (0.0010) [2023-10-10 19:29:42,421][123614] Updated weights for policy 1, policy_version 74480 (0.0011) [2023-10-10 19:29:42,795][123614] Updated weights for policy 1, policy_version 74490 (0.0009) [2023-10-10 19:29:43,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 152666112. Throughput: 0: 1829.5, 1: 1822.8. Samples: 38170518. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-10 19:29:43,789][122664] Avg episode reward: [(0, '80.560'), (1, '83.500')] [2023-10-10 19:29:44,032][123582] Updated weights for policy 0, policy_version 74593 (0.0009) [2023-10-10 19:29:44,394][123582] Updated weights for policy 0, policy_version 74603 (0.0007) [2023-10-10 19:29:44,771][123582] Updated weights for policy 0, policy_version 74613 (0.0010) [2023-10-10 19:29:45,141][123582] Updated weights for policy 0, policy_version 74623 (0.0010) [2023-10-10 19:29:46,629][123614] Updated weights for policy 1, policy_version 74500 (0.0007) [2023-10-10 19:29:46,993][123614] Updated weights for policy 1, policy_version 74510 (0.0008) [2023-10-10 19:29:47,371][123614] Updated weights for policy 1, policy_version 74520 (0.0008) [2023-10-10 19:29:48,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 152731648. Throughput: 0: 1831.0, 1: 1824.0. Samples: 38191900. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-10 19:29:48,789][122664] Avg episode reward: [(0, '80.250'), (1, '79.780')] [2023-10-10 19:29:48,821][123582] Updated weights for policy 0, policy_version 74633 (0.0008) [2023-10-10 19:29:49,195][123582] Updated weights for policy 0, policy_version 74643 (0.0007) [2023-10-10 19:29:49,576][123582] Updated weights for policy 0, policy_version 74653 (0.0009) [2023-10-10 19:29:51,088][123614] Updated weights for policy 1, policy_version 74530 (0.0010) [2023-10-10 19:29:51,450][123614] Updated weights for policy 1, policy_version 74540 (0.0010) [2023-10-10 19:29:51,819][123614] Updated weights for policy 1, policy_version 74550 (0.0007) [2023-10-10 19:29:52,197][123614] Updated weights for policy 1, policy_version 74560 (0.0007) [2023-10-10 19:29:53,307][123582] Updated weights for policy 0, policy_version 74663 (0.0010) [2023-10-10 19:29:53,676][123582] Updated weights for policy 0, policy_version 74673 (0.0008) [2023-10-10 19:29:53,788][122664] Fps is (10 sec: 13106.7, 60 sec: 14199.3, 300 sec: 14440.1). Total num frames: 152797184. Throughput: 0: 1823.0, 1: 1826.9. Samples: 38214330. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-10 19:29:53,790][122664] Avg episode reward: [(0, '80.600'), (1, '87.980')] [2023-10-10 19:29:53,800][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000074560_76349440.pth... [2023-10-10 19:29:53,832][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000072864_74612736.pth [2023-10-10 19:29:54,054][123582] Updated weights for policy 0, policy_version 74683 (0.0009) [2023-10-10 19:29:54,231][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000074688_76480512.pth... [2023-10-10 19:29:54,270][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000072960_74711040.pth [2023-10-10 19:29:55,985][123614] Updated weights for policy 1, policy_version 74570 (0.0007) [2023-10-10 19:29:56,351][123614] Updated weights for policy 1, policy_version 74580 (0.0010) [2023-10-10 19:29:56,716][123614] Updated weights for policy 1, policy_version 74590 (0.0010) [2023-10-10 19:29:57,657][123582] Updated weights for policy 0, policy_version 74693 (0.0008) [2023-10-10 19:29:58,023][123582] Updated weights for policy 0, policy_version 74703 (0.0010) [2023-10-10 19:29:58,389][123582] Updated weights for policy 0, policy_version 74713 (0.0009) [2023-10-10 19:29:58,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 152895488. Throughput: 0: 1827.0, 1: 1827.6. Samples: 38224804. Policy #0 lag: (min: 14.0, avg: 21.2, max: 46.0) [2023-10-10 19:29:58,789][122664] Avg episode reward: [(0, '83.090'), (1, '87.630')] [2023-10-10 19:30:00,473][123614] Updated weights for policy 1, policy_version 74600 (0.0008) [2023-10-10 19:30:00,843][123614] Updated weights for policy 1, policy_version 74610 (0.0008) [2023-10-10 19:30:01,205][123614] Updated weights for policy 1, policy_version 74620 (0.0008) [2023-10-10 19:30:02,109][123582] Updated weights for policy 0, policy_version 74723 (0.0009) [2023-10-10 19:30:02,480][123582] Updated weights for policy 0, policy_version 74733 (0.0009) [2023-10-10 19:30:02,849][123582] Updated weights for policy 0, policy_version 74743 (0.0008) [2023-10-10 19:30:03,788][122664] Fps is (10 sec: 16385.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 152961024. Throughput: 0: 1823.6, 1: 1824.7. Samples: 38247048. Policy #0 lag: (min: 14.0, avg: 21.2, max: 46.0) [2023-10-10 19:30:03,788][122664] Avg episode reward: [(0, '84.260'), (1, '83.310')] [2023-10-10 19:30:04,881][123614] Updated weights for policy 1, policy_version 74630 (0.0009) [2023-10-10 19:30:05,250][123614] Updated weights for policy 1, policy_version 74640 (0.0009) [2023-10-10 19:30:05,624][123614] Updated weights for policy 1, policy_version 74650 (0.0007) [2023-10-10 19:30:06,420][123582] Updated weights for policy 0, policy_version 74753 (0.0010) [2023-10-10 19:30:06,797][123582] Updated weights for policy 0, policy_version 74763 (0.0011) [2023-10-10 19:30:07,168][123582] Updated weights for policy 0, policy_version 74773 (0.0008) [2023-10-10 19:30:07,528][123582] Updated weights for policy 0, policy_version 74783 (0.0009) [2023-10-10 19:30:08,788][122664] Fps is (10 sec: 13106.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 153026560. Throughput: 0: 1835.1, 1: 1825.0. Samples: 38268994. Policy #0 lag: (min: 14.0, avg: 21.2, max: 46.0) [2023-10-10 19:30:08,790][122664] Avg episode reward: [(0, '86.920'), (1, '82.360')] [2023-10-10 19:30:09,181][123614] Updated weights for policy 1, policy_version 74660 (0.0010) [2023-10-10 19:30:09,547][123614] Updated weights for policy 1, policy_version 74670 (0.0008) [2023-10-10 19:30:09,914][123614] Updated weights for policy 1, policy_version 74680 (0.0010) [2023-10-10 19:30:11,292][123582] Updated weights for policy 0, policy_version 74793 (0.0011) [2023-10-10 19:30:11,669][123582] Updated weights for policy 0, policy_version 74803 (0.0010) [2023-10-10 19:30:12,038][123582] Updated weights for policy 0, policy_version 74813 (0.0008) [2023-10-10 19:30:13,633][123614] Updated weights for policy 1, policy_version 74690 (0.0007) [2023-10-10 19:30:13,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 153092096. Throughput: 0: 1821.2, 1: 1826.1. Samples: 38279694. Policy #0 lag: (min: 14.0, avg: 21.2, max: 46.0) [2023-10-10 19:30:13,788][122664] Avg episode reward: [(0, '93.180'), (1, '86.580')] [2023-10-10 19:30:13,992][123614] Updated weights for policy 1, policy_version 74700 (0.0008) [2023-10-10 19:30:14,354][123614] Updated weights for policy 1, policy_version 74710 (0.0011) [2023-10-10 19:30:14,722][123614] Updated weights for policy 1, policy_version 74720 (0.0009) [2023-10-10 19:30:15,704][123582] Updated weights for policy 0, policy_version 74823 (0.0008) [2023-10-10 19:30:16,074][123582] Updated weights for policy 0, policy_version 74833 (0.0009) [2023-10-10 19:30:16,453][123582] Updated weights for policy 0, policy_version 74843 (0.0009) [2023-10-10 19:30:18,383][123614] Updated weights for policy 1, policy_version 74730 (0.0007) [2023-10-10 19:30:18,750][123614] Updated weights for policy 1, policy_version 74740 (0.0008) [2023-10-10 19:30:18,788][122664] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 153157632. Throughput: 0: 1827.1, 1: 1817.5. Samples: 38301312. Policy #0 lag: (min: 14.0, avg: 21.2, max: 46.0) [2023-10-10 19:30:18,788][122664] Avg episode reward: [(0, '93.470'), (1, '84.810')] [2023-10-10 19:30:19,128][123614] Updated weights for policy 1, policy_version 74750 (0.0010) [2023-10-10 19:30:20,149][123582] Updated weights for policy 0, policy_version 74853 (0.0008) [2023-10-10 19:30:20,517][123582] Updated weights for policy 0, policy_version 74863 (0.0009) [2023-10-10 19:30:20,891][123582] Updated weights for policy 0, policy_version 74873 (0.0010) [2023-10-10 19:30:22,679][123614] Updated weights for policy 1, policy_version 74760 (0.0008) [2023-10-10 19:30:23,050][123614] Updated weights for policy 1, policy_version 74770 (0.0008) [2023-10-10 19:30:23,416][123614] Updated weights for policy 1, policy_version 74780 (0.0008) [2023-10-10 19:30:23,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 153255936. Throughput: 0: 1822.1, 1: 1818.3. Samples: 38322728. Policy #0 lag: (min: 14.0, avg: 21.2, max: 46.0) [2023-10-10 19:30:23,789][122664] Avg episode reward: [(0, '92.320'), (1, '86.540')] [2023-10-10 19:30:24,628][123582] Updated weights for policy 0, policy_version 74883 (0.0009) [2023-10-10 19:30:25,002][123582] Updated weights for policy 0, policy_version 74893 (0.0011) [2023-10-10 19:30:25,369][123582] Updated weights for policy 0, policy_version 74903 (0.0010) [2023-10-10 19:30:27,235][123614] Updated weights for policy 1, policy_version 74790 (0.0008) [2023-10-10 19:30:27,608][123614] Updated weights for policy 1, policy_version 74800 (0.0007) [2023-10-10 19:30:27,978][123614] Updated weights for policy 1, policy_version 74810 (0.0009) [2023-10-10 19:30:28,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 153321472. Throughput: 0: 1824.0, 1: 1813.1. Samples: 38334186. Policy #0 lag: (min: 14.0, avg: 21.2, max: 46.0) [2023-10-10 19:30:28,789][122664] Avg episode reward: [(0, '87.850'), (1, '80.860')] [2023-10-10 19:30:29,028][123582] Updated weights for policy 0, policy_version 74913 (0.0007) [2023-10-10 19:30:29,397][123582] Updated weights for policy 0, policy_version 74923 (0.0007) [2023-10-10 19:30:29,763][123582] Updated weights for policy 0, policy_version 74933 (0.0007) [2023-10-10 19:30:30,147][123582] Updated weights for policy 0, policy_version 74943 (0.0009) [2023-10-10 19:30:31,726][123614] Updated weights for policy 1, policy_version 74820 (0.0011) [2023-10-10 19:30:32,093][123614] Updated weights for policy 1, policy_version 74830 (0.0011) [2023-10-10 19:30:32,462][123614] Updated weights for policy 1, policy_version 74840 (0.0008) [2023-10-10 19:30:33,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 153387008. Throughput: 0: 1820.5, 1: 1820.2. Samples: 38355732. Policy #0 lag: (min: 14.0, avg: 21.2, max: 46.0) [2023-10-10 19:30:33,789][122664] Avg episode reward: [(0, '90.560'), (1, '82.660')] [2023-10-10 19:30:33,825][123582] Updated weights for policy 0, policy_version 74953 (0.0007) [2023-10-10 19:30:34,201][123582] Updated weights for policy 0, policy_version 74963 (0.0009) [2023-10-10 19:30:34,564][123582] Updated weights for policy 0, policy_version 74973 (0.0009) [2023-10-10 19:30:36,227][123614] Updated weights for policy 1, policy_version 74850 (0.0007) [2023-10-10 19:30:36,598][123614] Updated weights for policy 1, policy_version 74860 (0.0008) [2023-10-10 19:30:36,972][123614] Updated weights for policy 1, policy_version 74870 (0.0010) [2023-10-10 19:30:37,351][123614] Updated weights for policy 1, policy_version 74880 (0.0010) [2023-10-10 19:30:38,330][123582] Updated weights for policy 0, policy_version 74983 (0.0008) [2023-10-10 19:30:38,703][123582] Updated weights for policy 0, policy_version 74993 (0.0008) [2023-10-10 19:30:38,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 153452544. Throughput: 0: 1821.3, 1: 1812.3. Samples: 38377842. Policy #0 lag: (min: 14.0, avg: 21.2, max: 46.0) [2023-10-10 19:30:38,788][122664] Avg episode reward: [(0, '89.030'), (1, '81.980')] [2023-10-10 19:30:39,085][123582] Updated weights for policy 0, policy_version 75003 (0.0008) [2023-10-10 19:30:41,108][123614] Updated weights for policy 1, policy_version 74890 (0.0009) [2023-10-10 19:30:41,475][123614] Updated weights for policy 1, policy_version 74900 (0.0010) [2023-10-10 19:30:41,841][123614] Updated weights for policy 1, policy_version 74910 (0.0010) [2023-10-10 19:30:42,775][123582] Updated weights for policy 0, policy_version 75013 (0.0010) [2023-10-10 19:30:43,144][123582] Updated weights for policy 0, policy_version 75023 (0.0011) [2023-10-10 19:30:43,519][123582] Updated weights for policy 0, policy_version 75033 (0.0007) [2023-10-10 19:30:43,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 153550848. Throughput: 0: 1818.1, 1: 1810.4. Samples: 38388086. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-10 19:30:43,788][122664] Avg episode reward: [(0, '88.830'), (1, '80.490')] [2023-10-10 19:30:45,552][123614] Updated weights for policy 1, policy_version 74920 (0.0009) [2023-10-10 19:30:45,928][123614] Updated weights for policy 1, policy_version 74930 (0.0008) [2023-10-10 19:30:46,289][123614] Updated weights for policy 1, policy_version 74940 (0.0008) [2023-10-10 19:30:47,255][123582] Updated weights for policy 0, policy_version 75043 (0.0007) [2023-10-10 19:30:47,635][123582] Updated weights for policy 0, policy_version 75053 (0.0008) [2023-10-10 19:30:48,003][123582] Updated weights for policy 0, policy_version 75063 (0.0008) [2023-10-10 19:30:48,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 153616384. Throughput: 0: 1820.0, 1: 1809.9. Samples: 38410396. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-10 19:30:48,789][122664] Avg episode reward: [(0, '90.030'), (1, '78.800')] [2023-10-10 19:30:49,882][123614] Updated weights for policy 1, policy_version 74950 (0.0008) [2023-10-10 19:30:50,262][123614] Updated weights for policy 1, policy_version 74960 (0.0009) [2023-10-10 19:30:50,623][123614] Updated weights for policy 1, policy_version 74970 (0.0008) [2023-10-10 19:30:51,753][123582] Updated weights for policy 0, policy_version 75073 (0.0008) [2023-10-10 19:30:52,128][123582] Updated weights for policy 0, policy_version 75083 (0.0007) [2023-10-10 19:30:52,495][123582] Updated weights for policy 0, policy_version 75093 (0.0007) [2023-10-10 19:30:52,862][123582] Updated weights for policy 0, policy_version 75103 (0.0007) [2023-10-10 19:30:53,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 153681920. Throughput: 0: 1811.5, 1: 1808.2. Samples: 38431880. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-10 19:30:53,789][122664] Avg episode reward: [(0, '91.690'), (1, '78.730')] [2023-10-10 19:30:54,328][123614] Updated weights for policy 1, policy_version 74980 (0.0010) [2023-10-10 19:30:54,697][123614] Updated weights for policy 1, policy_version 74990 (0.0009) [2023-10-10 19:30:55,070][123614] Updated weights for policy 1, policy_version 75000 (0.0008) [2023-10-10 19:30:56,751][123582] Updated weights for policy 0, policy_version 75113 (0.0010) [2023-10-10 19:30:57,131][123582] Updated weights for policy 0, policy_version 75123 (0.0010) [2023-10-10 19:30:57,496][123582] Updated weights for policy 0, policy_version 75133 (0.0008) [2023-10-10 19:30:58,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 153747456. Throughput: 0: 1821.5, 1: 1803.9. Samples: 38442838. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-10 19:30:58,789][122664] Avg episode reward: [(0, '87.300'), (1, '75.030')] [2023-10-10 19:30:58,909][123614] Updated weights for policy 1, policy_version 75010 (0.0007) [2023-10-10 19:30:59,292][123614] Updated weights for policy 1, policy_version 75020 (0.0010) [2023-10-10 19:30:59,654][123614] Updated weights for policy 1, policy_version 75030 (0.0010) [2023-10-10 19:31:00,030][123614] Updated weights for policy 1, policy_version 75040 (0.0010) [2023-10-10 19:31:01,144][123582] Updated weights for policy 0, policy_version 75143 (0.0008) [2023-10-10 19:31:01,517][123582] Updated weights for policy 0, policy_version 75153 (0.0011) [2023-10-10 19:31:01,881][123582] Updated weights for policy 0, policy_version 75163 (0.0010) [2023-10-10 19:31:03,628][123614] Updated weights for policy 1, policy_version 75050 (0.0010) [2023-10-10 19:31:03,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 153812992. Throughput: 0: 1808.2, 1: 1810.6. Samples: 38464158. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-10 19:31:03,789][122664] Avg episode reward: [(0, '82.790'), (1, '75.860')] [2023-10-10 19:31:03,992][123614] Updated weights for policy 1, policy_version 75060 (0.0010) [2023-10-10 19:31:04,358][123614] Updated weights for policy 1, policy_version 75070 (0.0007) [2023-10-10 19:31:05,589][123582] Updated weights for policy 0, policy_version 75173 (0.0009) [2023-10-10 19:31:05,975][123582] Updated weights for policy 0, policy_version 75183 (0.0008) [2023-10-10 19:31:06,334][123582] Updated weights for policy 0, policy_version 75193 (0.0008) [2023-10-10 19:31:08,095][123614] Updated weights for policy 1, policy_version 75080 (0.0010) [2023-10-10 19:31:08,456][123614] Updated weights for policy 1, policy_version 75090 (0.0007) [2023-10-10 19:31:08,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 153878528. Throughput: 0: 1811.2, 1: 1814.6. Samples: 38485890. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-10 19:31:08,789][122664] Avg episode reward: [(0, '82.170'), (1, '72.580')] [2023-10-10 19:31:08,828][123614] Updated weights for policy 1, policy_version 75100 (0.0008) [2023-10-10 19:31:10,026][123582] Updated weights for policy 0, policy_version 75203 (0.0009) [2023-10-10 19:31:10,389][123582] Updated weights for policy 0, policy_version 75213 (0.0010) [2023-10-10 19:31:10,752][123582] Updated weights for policy 0, policy_version 75223 (0.0007) [2023-10-10 19:31:12,525][123614] Updated weights for policy 1, policy_version 75110 (0.0010) [2023-10-10 19:31:12,897][123614] Updated weights for policy 1, policy_version 75120 (0.0009) [2023-10-10 19:31:13,267][123614] Updated weights for policy 1, policy_version 75130 (0.0009) [2023-10-10 19:31:13,788][122664] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 153976832. Throughput: 0: 1807.8, 1: 1804.6. Samples: 38496742. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-10 19:31:13,788][122664] Avg episode reward: [(0, '84.170'), (1, '69.750')] [2023-10-10 19:31:14,416][123582] Updated weights for policy 0, policy_version 75233 (0.0007) [2023-10-10 19:31:14,788][123582] Updated weights for policy 0, policy_version 75243 (0.0008) [2023-10-10 19:31:15,151][123582] Updated weights for policy 0, policy_version 75253 (0.0009) [2023-10-10 19:31:15,522][123582] Updated weights for policy 0, policy_version 75263 (0.0008) [2023-10-10 19:31:16,880][123614] Updated weights for policy 1, policy_version 75140 (0.0008) [2023-10-10 19:31:17,248][123614] Updated weights for policy 1, policy_version 75150 (0.0010) [2023-10-10 19:31:17,630][123614] Updated weights for policy 1, policy_version 75160 (0.0010) [2023-10-10 19:31:18,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 154042368. Throughput: 0: 1809.8, 1: 1810.9. Samples: 38518664. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-10 19:31:18,788][122664] Avg episode reward: [(0, '85.170'), (1, '68.660')] [2023-10-10 19:31:19,180][123582] Updated weights for policy 0, policy_version 75273 (0.0007) [2023-10-10 19:31:19,551][123582] Updated weights for policy 0, policy_version 75283 (0.0009) [2023-10-10 19:31:19,923][123582] Updated weights for policy 0, policy_version 75293 (0.0010) [2023-10-10 19:31:21,266][123614] Updated weights for policy 1, policy_version 75170 (0.0009) [2023-10-10 19:31:21,633][123614] Updated weights for policy 1, policy_version 75180 (0.0009) [2023-10-10 19:31:22,001][123614] Updated weights for policy 1, policy_version 75190 (0.0007) [2023-10-10 19:31:22,374][123614] Updated weights for policy 1, policy_version 75200 (0.0011) [2023-10-10 19:31:23,741][123582] Updated weights for policy 0, policy_version 75303 (0.0010) [2023-10-10 19:31:23,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 154107904. Throughput: 0: 1820.0, 1: 1806.8. Samples: 38541050. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-10 19:31:23,789][122664] Avg episode reward: [(0, '76.240'), (1, '72.210')] [2023-10-10 19:31:24,106][123582] Updated weights for policy 0, policy_version 75313 (0.0009) [2023-10-10 19:31:24,483][123582] Updated weights for policy 0, policy_version 75323 (0.0007) [2023-10-10 19:31:26,079][123614] Updated weights for policy 1, policy_version 75210 (0.0009) [2023-10-10 19:31:26,456][123614] Updated weights for policy 1, policy_version 75220 (0.0008) [2023-10-10 19:31:26,816][123614] Updated weights for policy 1, policy_version 75230 (0.0008) [2023-10-10 19:31:27,996][123582] Updated weights for policy 0, policy_version 75333 (0.0008) [2023-10-10 19:31:28,360][123582] Updated weights for policy 0, policy_version 75343 (0.0010) [2023-10-10 19:31:28,722][123582] Updated weights for policy 0, policy_version 75353 (0.0009) [2023-10-10 19:31:28,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 154173440. Throughput: 0: 1815.3, 1: 1812.8. Samples: 38551352. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-10 19:31:28,789][122664] Avg episode reward: [(0, '77.330'), (1, '69.240')] [2023-10-10 19:31:30,348][123614] Updated weights for policy 1, policy_version 75240 (0.0007) [2023-10-10 19:31:30,715][123614] Updated weights for policy 1, policy_version 75250 (0.0008) [2023-10-10 19:31:31,088][123614] Updated weights for policy 1, policy_version 75260 (0.0008) [2023-10-10 19:31:32,570][123582] Updated weights for policy 0, policy_version 75363 (0.0008) [2023-10-10 19:31:32,947][123582] Updated weights for policy 0, policy_version 75373 (0.0009) [2023-10-10 19:31:33,320][123582] Updated weights for policy 0, policy_version 75383 (0.0008) [2023-10-10 19:31:33,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 154271744. Throughput: 0: 1818.6, 1: 1816.9. Samples: 38573996. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:31:33,788][122664] Avg episode reward: [(0, '75.880'), (1, '68.880')] [2023-10-10 19:31:34,996][123614] Updated weights for policy 1, policy_version 75270 (0.0008) [2023-10-10 19:31:35,365][123614] Updated weights for policy 1, policy_version 75280 (0.0007) [2023-10-10 19:31:35,731][123614] Updated weights for policy 1, policy_version 75290 (0.0007) [2023-10-10 19:31:36,926][123582] Updated weights for policy 0, policy_version 75393 (0.0007) [2023-10-10 19:31:37,291][123582] Updated weights for policy 0, policy_version 75403 (0.0009) [2023-10-10 19:31:37,661][123582] Updated weights for policy 0, policy_version 75413 (0.0008) [2023-10-10 19:31:38,037][123582] Updated weights for policy 0, policy_version 75423 (0.0008) [2023-10-10 19:31:38,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 154337280. Throughput: 0: 1810.4, 1: 1814.4. Samples: 38594996. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:31:38,789][122664] Avg episode reward: [(0, '74.400'), (1, '65.550')] [2023-10-10 19:31:39,435][123614] Updated weights for policy 1, policy_version 75300 (0.0008) [2023-10-10 19:31:39,800][123614] Updated weights for policy 1, policy_version 75310 (0.0008) [2023-10-10 19:31:40,163][123614] Updated weights for policy 1, policy_version 75320 (0.0009) [2023-10-10 19:31:41,867][123582] Updated weights for policy 0, policy_version 75433 (0.0007) [2023-10-10 19:31:42,245][123582] Updated weights for policy 0, policy_version 75443 (0.0008) [2023-10-10 19:31:42,625][123582] Updated weights for policy 0, policy_version 75453 (0.0009) [2023-10-10 19:31:43,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 154402816. Throughput: 0: 1817.3, 1: 1821.2. Samples: 38606568. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:31:43,788][122664] Avg episode reward: [(0, '77.170'), (1, '67.000')] [2023-10-10 19:31:43,925][123614] Updated weights for policy 1, policy_version 75330 (0.0009) [2023-10-10 19:31:44,304][123614] Updated weights for policy 1, policy_version 75340 (0.0011) [2023-10-10 19:31:44,666][123614] Updated weights for policy 1, policy_version 75350 (0.0010) [2023-10-10 19:31:45,038][123614] Updated weights for policy 1, policy_version 75360 (0.0010) [2023-10-10 19:31:46,406][123582] Updated weights for policy 0, policy_version 75463 (0.0009) [2023-10-10 19:31:46,775][123582] Updated weights for policy 0, policy_version 75473 (0.0009) [2023-10-10 19:31:47,138][123582] Updated weights for policy 0, policy_version 75483 (0.0008) [2023-10-10 19:31:48,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 154468352. Throughput: 0: 1812.3, 1: 1814.4. Samples: 38627356. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:31:48,788][122664] Avg episode reward: [(0, '81.030'), (1, '69.070')] [2023-10-10 19:31:48,819][123614] Updated weights for policy 1, policy_version 75370 (0.0009) [2023-10-10 19:31:49,188][123614] Updated weights for policy 1, policy_version 75380 (0.0009) [2023-10-10 19:31:49,550][123614] Updated weights for policy 1, policy_version 75390 (0.0008) [2023-10-10 19:31:50,805][123582] Updated weights for policy 0, policy_version 75493 (0.0007) [2023-10-10 19:31:51,186][123582] Updated weights for policy 0, policy_version 75503 (0.0007) [2023-10-10 19:31:51,559][123582] Updated weights for policy 0, policy_version 75513 (0.0010) [2023-10-10 19:31:53,326][123614] Updated weights for policy 1, policy_version 75400 (0.0010) [2023-10-10 19:31:53,697][123614] Updated weights for policy 1, policy_version 75410 (0.0010) [2023-10-10 19:31:53,788][122664] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 154533888. Throughput: 0: 1810.4, 1: 1820.4. Samples: 38649272. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:31:53,789][122664] Avg episode reward: [(0, '80.090'), (1, '69.950')] [2023-10-10 19:31:53,798][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000075520_77332480.pth... [2023-10-10 19:31:53,830][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000073824_75595776.pth [2023-10-10 19:31:54,067][123614] Updated weights for policy 1, policy_version 75420 (0.0009) [2023-10-10 19:31:54,216][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000075424_77234176.pth... [2023-10-10 19:31:54,248][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000073728_75497472.pth [2023-10-10 19:31:55,201][123582] Updated weights for policy 0, policy_version 75523 (0.0008) [2023-10-10 19:31:55,564][123582] Updated weights for policy 0, policy_version 75533 (0.0007) [2023-10-10 19:31:55,948][123582] Updated weights for policy 0, policy_version 75543 (0.0007) [2023-10-10 19:31:57,623][123614] Updated weights for policy 1, policy_version 75430 (0.0007) [2023-10-10 19:31:57,998][123614] Updated weights for policy 1, policy_version 75440 (0.0010) [2023-10-10 19:31:58,358][123614] Updated weights for policy 1, policy_version 75450 (0.0008) [2023-10-10 19:31:58,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 154632192. Throughput: 0: 1811.6, 1: 1816.2. Samples: 38659994. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:31:58,789][122664] Avg episode reward: [(0, '80.640'), (1, '70.250')] [2023-10-10 19:31:59,647][123582] Updated weights for policy 0, policy_version 75553 (0.0008) [2023-10-10 19:32:00,017][123582] Updated weights for policy 0, policy_version 75563 (0.0008) [2023-10-10 19:32:00,383][123582] Updated weights for policy 0, policy_version 75573 (0.0011) [2023-10-10 19:32:00,762][123582] Updated weights for policy 0, policy_version 75583 (0.0011) [2023-10-10 19:32:02,221][123614] Updated weights for policy 1, policy_version 75460 (0.0007) [2023-10-10 19:32:02,584][123614] Updated weights for policy 1, policy_version 75470 (0.0007) [2023-10-10 19:32:02,950][123614] Updated weights for policy 1, policy_version 75480 (0.0007) [2023-10-10 19:32:03,788][122664] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 154697728. Throughput: 0: 1807.1, 1: 1818.9. Samples: 38681832. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:32:03,788][122664] Avg episode reward: [(0, '81.110'), (1, '73.080')] [2023-10-10 19:32:04,475][123582] Updated weights for policy 0, policy_version 75593 (0.0008) [2023-10-10 19:32:04,844][123582] Updated weights for policy 0, policy_version 75603 (0.0008) [2023-10-10 19:32:05,221][123582] Updated weights for policy 0, policy_version 75613 (0.0008) [2023-10-10 19:32:06,471][123614] Updated weights for policy 1, policy_version 75490 (0.0008) [2023-10-10 19:32:06,848][123614] Updated weights for policy 1, policy_version 75500 (0.0007) [2023-10-10 19:32:07,210][123614] Updated weights for policy 1, policy_version 75510 (0.0008) [2023-10-10 19:32:07,579][123614] Updated weights for policy 1, policy_version 75520 (0.0010) [2023-10-10 19:32:08,784][123582] Updated weights for policy 0, policy_version 75623 (0.0008) [2023-10-10 19:32:08,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 154763264. Throughput: 0: 1812.8, 1: 1815.2. Samples: 38704312. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:32:08,789][122664] Avg episode reward: [(0, '82.300'), (1, '73.960')] [2023-10-10 19:32:09,160][123582] Updated weights for policy 0, policy_version 75633 (0.0008) [2023-10-10 19:32:09,535][123582] Updated weights for policy 0, policy_version 75643 (0.0007) [2023-10-10 19:32:11,415][123614] Updated weights for policy 1, policy_version 75530 (0.0007) [2023-10-10 19:32:11,783][123614] Updated weights for policy 1, policy_version 75540 (0.0007) [2023-10-10 19:32:12,160][123614] Updated weights for policy 1, policy_version 75550 (0.0008) [2023-10-10 19:32:13,194][123582] Updated weights for policy 0, policy_version 75653 (0.0009) [2023-10-10 19:32:13,572][123582] Updated weights for policy 0, policy_version 75663 (0.0008) [2023-10-10 19:32:13,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 154828800. Throughput: 0: 1805.9, 1: 1823.6. Samples: 38714680. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:32:13,789][122664] Avg episode reward: [(0, '83.850'), (1, '77.000')] [2023-10-10 19:32:13,934][123582] Updated weights for policy 0, policy_version 75673 (0.0007) [2023-10-10 19:32:15,731][123614] Updated weights for policy 1, policy_version 75560 (0.0010) [2023-10-10 19:32:16,109][123614] Updated weights for policy 1, policy_version 75570 (0.0007) [2023-10-10 19:32:16,481][123614] Updated weights for policy 1, policy_version 75580 (0.0008) [2023-10-10 19:32:17,770][123582] Updated weights for policy 0, policy_version 75683 (0.0010) [2023-10-10 19:32:18,141][123582] Updated weights for policy 0, policy_version 75693 (0.0011) [2023-10-10 19:32:18,515][123582] Updated weights for policy 0, policy_version 75703 (0.0009) [2023-10-10 19:32:18,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 154894336. Throughput: 0: 1805.9, 1: 1809.1. Samples: 38736672. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:32:18,789][122664] Avg episode reward: [(0, '91.060'), (1, '81.080')] [2023-10-10 19:32:20,320][123614] Updated weights for policy 1, policy_version 75590 (0.0008) [2023-10-10 19:32:20,690][123614] Updated weights for policy 1, policy_version 75600 (0.0008) [2023-10-10 19:32:21,046][123614] Updated weights for policy 1, policy_version 75610 (0.0008) [2023-10-10 19:32:22,120][123582] Updated weights for policy 0, policy_version 75713 (0.0009) [2023-10-10 19:32:22,488][123582] Updated weights for policy 0, policy_version 75723 (0.0008) [2023-10-10 19:32:22,856][123582] Updated weights for policy 0, policy_version 75733 (0.0008) [2023-10-10 19:32:23,228][123582] Updated weights for policy 0, policy_version 75743 (0.0007) [2023-10-10 19:32:23,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 154992640. Throughput: 0: 1809.9, 1: 1813.9. Samples: 38758064. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:32:23,789][122664] Avg episode reward: [(0, '91.030'), (1, '79.710')] [2023-10-10 19:32:24,725][123614] Updated weights for policy 1, policy_version 75620 (0.0008) [2023-10-10 19:32:25,092][123614] Updated weights for policy 1, policy_version 75630 (0.0007) [2023-10-10 19:32:25,465][123614] Updated weights for policy 1, policy_version 75640 (0.0007) [2023-10-10 19:32:26,905][123582] Updated weights for policy 0, policy_version 75753 (0.0008) [2023-10-10 19:32:27,280][123582] Updated weights for policy 0, policy_version 75763 (0.0010) [2023-10-10 19:32:27,637][123582] Updated weights for policy 0, policy_version 75773 (0.0010) [2023-10-10 19:32:28,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 155058176. Throughput: 0: 1807.6, 1: 1806.1. Samples: 38769186. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:32:28,789][122664] Avg episode reward: [(0, '92.230'), (1, '76.470')] [2023-10-10 19:32:29,226][123614] Updated weights for policy 1, policy_version 75650 (0.0008) [2023-10-10 19:32:29,591][123614] Updated weights for policy 1, policy_version 75660 (0.0008) [2023-10-10 19:32:29,950][123614] Updated weights for policy 1, policy_version 75670 (0.0010) [2023-10-10 19:32:30,323][123614] Updated weights for policy 1, policy_version 75680 (0.0008) [2023-10-10 19:32:31,389][123582] Updated weights for policy 0, policy_version 75783 (0.0008) [2023-10-10 19:32:31,760][123582] Updated weights for policy 0, policy_version 75793 (0.0009) [2023-10-10 19:32:32,134][123582] Updated weights for policy 0, policy_version 75803 (0.0008) [2023-10-10 19:32:33,788][122664] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 155123712. Throughput: 0: 1815.3, 1: 1812.3. Samples: 38790598. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:32:33,789][122664] Avg episode reward: [(0, '90.740'), (1, '77.130')] [2023-10-10 19:32:33,947][123614] Updated weights for policy 1, policy_version 75690 (0.0007) [2023-10-10 19:32:34,319][123614] Updated weights for policy 1, policy_version 75700 (0.0008) [2023-10-10 19:32:34,678][123614] Updated weights for policy 1, policy_version 75710 (0.0011) [2023-10-10 19:32:35,850][123582] Updated weights for policy 0, policy_version 75813 (0.0010) [2023-10-10 19:32:36,222][123582] Updated weights for policy 0, policy_version 75823 (0.0010) [2023-10-10 19:32:36,597][123582] Updated weights for policy 0, policy_version 75833 (0.0011) [2023-10-10 19:32:38,346][123614] Updated weights for policy 1, policy_version 75720 (0.0009) [2023-10-10 19:32:38,717][123614] Updated weights for policy 1, policy_version 75730 (0.0009) [2023-10-10 19:32:38,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 155189248. Throughput: 0: 1815.3, 1: 1814.8. Samples: 38812626. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:32:38,788][122664] Avg episode reward: [(0, '97.380'), (1, '77.980')] [2023-10-10 19:32:39,085][123614] Updated weights for policy 1, policy_version 75740 (0.0007) [2023-10-10 19:32:40,370][123582] Updated weights for policy 0, policy_version 75843 (0.0010) [2023-10-10 19:32:40,742][123582] Updated weights for policy 0, policy_version 75853 (0.0008) [2023-10-10 19:32:41,108][123582] Updated weights for policy 0, policy_version 75863 (0.0009) [2023-10-10 19:32:42,915][123614] Updated weights for policy 1, policy_version 75750 (0.0009) [2023-10-10 19:32:43,292][123614] Updated weights for policy 1, policy_version 75760 (0.0010) [2023-10-10 19:32:43,658][123614] Updated weights for policy 1, policy_version 75770 (0.0010) [2023-10-10 19:32:43,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 155254784. Throughput: 0: 1817.9, 1: 1811.0. Samples: 38823292. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:32:43,789][122664] Avg episode reward: [(0, '94.100'), (1, '81.890')] [2023-10-10 19:32:44,869][123582] Updated weights for policy 0, policy_version 75873 (0.0010) [2023-10-10 19:32:45,239][123582] Updated weights for policy 0, policy_version 75883 (0.0009) [2023-10-10 19:32:45,607][123582] Updated weights for policy 0, policy_version 75893 (0.0007) [2023-10-10 19:32:45,985][123582] Updated weights for policy 0, policy_version 75903 (0.0007) [2023-10-10 19:32:47,401][123614] Updated weights for policy 1, policy_version 75780 (0.0009) [2023-10-10 19:32:47,769][123614] Updated weights for policy 1, policy_version 75790 (0.0008) [2023-10-10 19:32:48,136][123614] Updated weights for policy 1, policy_version 75800 (0.0007) [2023-10-10 19:32:48,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 155353088. Throughput: 0: 1819.0, 1: 1817.7. Samples: 38845484. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:32:48,789][122664] Avg episode reward: [(0, '95.430'), (1, '81.590')] [2023-10-10 19:32:49,476][123582] Updated weights for policy 0, policy_version 75913 (0.0011) [2023-10-10 19:32:49,852][123582] Updated weights for policy 0, policy_version 75923 (0.0010) [2023-10-10 19:32:50,226][123582] Updated weights for policy 0, policy_version 75933 (0.0008) [2023-10-10 19:32:51,748][123614] Updated weights for policy 1, policy_version 75810 (0.0008) [2023-10-10 19:32:52,120][123614] Updated weights for policy 1, policy_version 75820 (0.0009) [2023-10-10 19:32:52,487][123614] Updated weights for policy 1, policy_version 75830 (0.0009) [2023-10-10 19:32:52,857][123614] Updated weights for policy 1, policy_version 75840 (0.0011) [2023-10-10 19:32:53,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 155418624. Throughput: 0: 1821.2, 1: 1807.8. Samples: 38867618. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:32:53,788][122664] Avg episode reward: [(0, '95.450'), (1, '82.730')] [2023-10-10 19:32:53,870][123582] Updated weights for policy 0, policy_version 75943 (0.0008) [2023-10-10 19:32:54,251][123582] Updated weights for policy 0, policy_version 75953 (0.0007) [2023-10-10 19:32:54,623][123582] Updated weights for policy 0, policy_version 75963 (0.0007) [2023-10-10 19:32:56,635][123614] Updated weights for policy 1, policy_version 75850 (0.0010) [2023-10-10 19:32:57,000][123614] Updated weights for policy 1, policy_version 75860 (0.0009) [2023-10-10 19:32:57,368][123614] Updated weights for policy 1, policy_version 75870 (0.0009) [2023-10-10 19:32:58,296][123582] Updated weights for policy 0, policy_version 75973 (0.0008) [2023-10-10 19:32:58,662][123582] Updated weights for policy 0, policy_version 75983 (0.0009) [2023-10-10 19:32:58,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 155484160. Throughput: 0: 1824.8, 1: 1812.4. Samples: 38878352. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:32:58,789][122664] Avg episode reward: [(0, '99.820'), (1, '83.790')] [2023-10-10 19:32:59,043][123582] Updated weights for policy 0, policy_version 75993 (0.0009) [2023-10-10 19:33:01,057][123614] Updated weights for policy 1, policy_version 75880 (0.0007) [2023-10-10 19:33:01,424][123614] Updated weights for policy 1, policy_version 75890 (0.0008) [2023-10-10 19:33:01,802][123614] Updated weights for policy 1, policy_version 75900 (0.0007) [2023-10-10 19:33:02,601][123582] Updated weights for policy 0, policy_version 76003 (0.0009) [2023-10-10 19:33:02,973][123582] Updated weights for policy 0, policy_version 76013 (0.0007) [2023-10-10 19:33:03,342][123582] Updated weights for policy 0, policy_version 76023 (0.0009) [2023-10-10 19:33:03,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 155582464. Throughput: 0: 1828.8, 1: 1806.9. Samples: 38900276. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:33:03,788][122664] Avg episode reward: [(0, '98.290'), (1, '84.900')] [2023-10-10 19:33:05,431][123614] Updated weights for policy 1, policy_version 75910 (0.0008) [2023-10-10 19:33:05,803][123614] Updated weights for policy 1, policy_version 75920 (0.0008) [2023-10-10 19:33:06,179][123614] Updated weights for policy 1, policy_version 75930 (0.0007) [2023-10-10 19:33:06,987][123582] Updated weights for policy 0, policy_version 76033 (0.0008) [2023-10-10 19:33:07,360][123582] Updated weights for policy 0, policy_version 76043 (0.0009) [2023-10-10 19:33:07,731][123582] Updated weights for policy 0, policy_version 76053 (0.0008) [2023-10-10 19:33:08,108][123582] Updated weights for policy 0, policy_version 76063 (0.0007) [2023-10-10 19:33:08,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 155648000. Throughput: 0: 1829.0, 1: 1806.1. Samples: 38921642. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:33:08,789][122664] Avg episode reward: [(0, '99.540'), (1, '86.250')] [2023-10-10 19:33:09,891][123614] Updated weights for policy 1, policy_version 75940 (0.0009) [2023-10-10 19:33:10,263][123614] Updated weights for policy 1, policy_version 75950 (0.0008) [2023-10-10 19:33:10,635][123614] Updated weights for policy 1, policy_version 75960 (0.0009) [2023-10-10 19:33:11,595][123582] Updated weights for policy 0, policy_version 76073 (0.0008) [2023-10-10 19:33:11,961][123582] Updated weights for policy 0, policy_version 76083 (0.0008) [2023-10-10 19:33:12,328][123582] Updated weights for policy 0, policy_version 76093 (0.0007) [2023-10-10 19:33:13,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 155713536. Throughput: 0: 1826.4, 1: 1808.8. Samples: 38932772. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-10 19:33:13,789][122664] Avg episode reward: [(0, '100.250'), (1, '83.190')] [2023-10-10 19:33:14,341][123614] Updated weights for policy 1, policy_version 75970 (0.0009) [2023-10-10 19:33:14,715][123614] Updated weights for policy 1, policy_version 75980 (0.0007) [2023-10-10 19:33:15,076][123614] Updated weights for policy 1, policy_version 75990 (0.0008) [2023-10-10 19:33:15,440][123614] Updated weights for policy 1, policy_version 76000 (0.0007) [2023-10-10 19:33:16,231][123582] Updated weights for policy 0, policy_version 76103 (0.0007) [2023-10-10 19:33:16,624][123582] Updated weights for policy 0, policy_version 76113 (0.0009) [2023-10-10 19:33:16,987][123582] Updated weights for policy 0, policy_version 76123 (0.0007) [2023-10-10 19:33:18,788][122664] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 155779072. Throughput: 0: 1826.4, 1: 1810.8. Samples: 38954272. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-10 19:33:18,788][122664] Avg episode reward: [(0, '101.380'), (1, '85.240')] [2023-10-10 19:33:19,105][123614] Updated weights for policy 1, policy_version 76010 (0.0008) [2023-10-10 19:33:19,469][123614] Updated weights for policy 1, policy_version 76020 (0.0009) [2023-10-10 19:33:19,844][123614] Updated weights for policy 1, policy_version 76030 (0.0009) [2023-10-10 19:33:20,658][123582] Updated weights for policy 0, policy_version 76133 (0.0008) [2023-10-10 19:33:21,045][123582] Updated weights for policy 0, policy_version 76143 (0.0009) [2023-10-10 19:33:21,421][123582] Updated weights for policy 0, policy_version 76153 (0.0011) [2023-10-10 19:33:23,452][123614] Updated weights for policy 1, policy_version 76040 (0.0007) [2023-10-10 19:33:23,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 155844608. Throughput: 0: 1823.9, 1: 1813.6. Samples: 38976312. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-10 19:33:23,789][122664] Avg episode reward: [(0, '97.260'), (1, '87.060')] [2023-10-10 19:33:23,816][123614] Updated weights for policy 1, policy_version 76050 (0.0009) [2023-10-10 19:33:24,187][123614] Updated weights for policy 1, policy_version 76060 (0.0009) [2023-10-10 19:33:25,290][123582] Updated weights for policy 0, policy_version 76163 (0.0009) [2023-10-10 19:33:25,665][123582] Updated weights for policy 0, policy_version 76173 (0.0008) [2023-10-10 19:33:26,031][123582] Updated weights for policy 0, policy_version 76183 (0.0007) [2023-10-10 19:33:27,826][123614] Updated weights for policy 1, policy_version 76070 (0.0009) [2023-10-10 19:33:28,187][123614] Updated weights for policy 1, policy_version 76080 (0.0007) [2023-10-10 19:33:28,552][123614] Updated weights for policy 1, policy_version 76090 (0.0007) [2023-10-10 19:33:28,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 155942912. Throughput: 0: 1819.5, 1: 1816.6. Samples: 38986918. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-10 19:33:28,788][122664] Avg episode reward: [(0, '98.860'), (1, '88.350')] [2023-10-10 19:33:29,689][123582] Updated weights for policy 0, policy_version 76193 (0.0012) [2023-10-10 19:33:30,047][123582] Updated weights for policy 0, policy_version 76203 (0.0012) [2023-10-10 19:33:30,418][123582] Updated weights for policy 0, policy_version 76213 (0.0010) [2023-10-10 19:33:30,786][123582] Updated weights for policy 0, policy_version 76223 (0.0008) [2023-10-10 19:33:32,386][123614] Updated weights for policy 1, policy_version 76100 (0.0009) [2023-10-10 19:33:32,756][123614] Updated weights for policy 1, policy_version 76110 (0.0009) [2023-10-10 19:33:33,122][123614] Updated weights for policy 1, policy_version 76120 (0.0009) [2023-10-10 19:33:33,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 156008448. Throughput: 0: 1816.3, 1: 1811.5. Samples: 39008736. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-10 19:33:33,788][122664] Avg episode reward: [(0, '98.980'), (1, '92.630')] [2023-10-10 19:33:34,296][123582] Updated weights for policy 0, policy_version 76233 (0.0007) [2023-10-10 19:33:34,670][123582] Updated weights for policy 0, policy_version 76243 (0.0009) [2023-10-10 19:33:35,049][123582] Updated weights for policy 0, policy_version 76253 (0.0008) [2023-10-10 19:33:36,890][123614] Updated weights for policy 1, policy_version 76130 (0.0008) [2023-10-10 19:33:37,265][123614] Updated weights for policy 1, policy_version 76140 (0.0007) [2023-10-10 19:33:37,632][123614] Updated weights for policy 1, policy_version 76150 (0.0007) [2023-10-10 19:33:38,001][123614] Updated weights for policy 1, policy_version 76160 (0.0009) [2023-10-10 19:33:38,751][123582] Updated weights for policy 0, policy_version 76263 (0.0009) [2023-10-10 19:33:38,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 156073984. Throughput: 0: 1810.5, 1: 1814.5. Samples: 39030744. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-10 19:33:38,788][122664] Avg episode reward: [(0, '89.030'), (1, '89.850')] [2023-10-10 19:33:39,129][123582] Updated weights for policy 0, policy_version 76273 (0.0010) [2023-10-10 19:33:39,498][123582] Updated weights for policy 0, policy_version 76283 (0.0011) [2023-10-10 19:33:41,764][123614] Updated weights for policy 1, policy_version 76170 (0.0009) [2023-10-10 19:33:42,137][123614] Updated weights for policy 1, policy_version 76180 (0.0008) [2023-10-10 19:33:42,505][123614] Updated weights for policy 1, policy_version 76190 (0.0007) [2023-10-10 19:33:43,298][123582] Updated weights for policy 0, policy_version 76293 (0.0009) [2023-10-10 19:33:43,677][123582] Updated weights for policy 0, policy_version 76303 (0.0009) [2023-10-10 19:33:43,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 156139520. Throughput: 0: 1805.6, 1: 1819.1. Samples: 39041464. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-10 19:33:43,788][122664] Avg episode reward: [(0, '85.790'), (1, '88.470')] [2023-10-10 19:33:44,039][123582] Updated weights for policy 0, policy_version 76313 (0.0009) [2023-10-10 19:33:46,178][123614] Updated weights for policy 1, policy_version 76200 (0.0007) [2023-10-10 19:33:46,540][123614] Updated weights for policy 1, policy_version 76210 (0.0007) [2023-10-10 19:33:46,906][123614] Updated weights for policy 1, policy_version 76220 (0.0007) [2023-10-10 19:33:47,761][123582] Updated weights for policy 0, policy_version 76323 (0.0008) [2023-10-10 19:33:48,132][123582] Updated weights for policy 0, policy_version 76333 (0.0009) [2023-10-10 19:33:48,498][123582] Updated weights for policy 0, policy_version 76343 (0.0008) [2023-10-10 19:33:48,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 156205056. Throughput: 0: 1805.5, 1: 1812.7. Samples: 39063094. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-10 19:33:48,789][122664] Avg episode reward: [(0, '84.120'), (1, '84.290')] [2023-10-10 19:33:50,524][123614] Updated weights for policy 1, policy_version 76230 (0.0007) [2023-10-10 19:33:50,883][123614] Updated weights for policy 1, policy_version 76240 (0.0008) [2023-10-10 19:33:51,257][123614] Updated weights for policy 1, policy_version 76250 (0.0007) [2023-10-10 19:33:52,080][123582] Updated weights for policy 0, policy_version 76353 (0.0008) [2023-10-10 19:33:52,460][123582] Updated weights for policy 0, policy_version 76363 (0.0007) [2023-10-10 19:33:52,833][123582] Updated weights for policy 0, policy_version 76373 (0.0009) [2023-10-10 19:33:53,218][123582] Updated weights for policy 0, policy_version 76383 (0.0008) [2023-10-10 19:33:53,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 156303360. Throughput: 0: 1811.0, 1: 1813.5. Samples: 39084742. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-10 19:33:53,789][122664] Avg episode reward: [(0, '79.670'), (1, '85.930')] [2023-10-10 19:33:53,799][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000076384_78217216.pth... [2023-10-10 19:33:53,799][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000076256_78086144.pth... [2023-10-10 19:33:53,834][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000074688_76480512.pth [2023-10-10 19:33:53,839][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000074560_76349440.pth [2023-10-10 19:33:53,840][123247] Saving a milestone ./train_atari/atari_demonattack_APPO/checkpoint_p0/milestones/checkpoint_000076384_78217216.pth [2023-10-10 19:33:53,843][123465] Saving a milestone ./train_atari/atari_demonattack_APPO/checkpoint_p1/milestones/checkpoint_000076256_78086144.pth [2023-10-10 19:33:55,053][123614] Updated weights for policy 1, policy_version 76260 (0.0008) [2023-10-10 19:33:55,409][123614] Updated weights for policy 1, policy_version 76270 (0.0009) [2023-10-10 19:33:55,780][123614] Updated weights for policy 1, policy_version 76280 (0.0007) [2023-10-10 19:33:57,017][123582] Updated weights for policy 0, policy_version 76393 (0.0011) [2023-10-10 19:33:57,393][123582] Updated weights for policy 0, policy_version 76403 (0.0009) [2023-10-10 19:33:57,762][123582] Updated weights for policy 0, policy_version 76413 (0.0011) [2023-10-10 19:33:58,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 156368896. Throughput: 0: 1809.1, 1: 1814.5. Samples: 39095832. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-10 19:33:58,789][122664] Avg episode reward: [(0, '79.620'), (1, '85.720')] [2023-10-10 19:33:59,573][123614] Updated weights for policy 1, policy_version 76290 (0.0007) [2023-10-10 19:33:59,948][123614] Updated weights for policy 1, policy_version 76300 (0.0008) [2023-10-10 19:34:00,310][123614] Updated weights for policy 1, policy_version 76310 (0.0008) [2023-10-10 19:34:00,683][123614] Updated weights for policy 1, policy_version 76320 (0.0010) [2023-10-10 19:34:01,717][123582] Updated weights for policy 0, policy_version 76423 (0.0009) [2023-10-10 19:34:02,100][123582] Updated weights for policy 0, policy_version 76433 (0.0009) [2023-10-10 19:34:02,471][123582] Updated weights for policy 0, policy_version 76443 (0.0009) [2023-10-10 19:34:03,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 156434432. Throughput: 0: 1809.1, 1: 1812.7. Samples: 39117258. Policy #0 lag: (min: 31.0, avg: 31.6, max: 46.0) [2023-10-10 19:34:03,789][122664] Avg episode reward: [(0, '79.860'), (1, '92.600')] [2023-10-10 19:34:04,327][123614] Updated weights for policy 1, policy_version 76330 (0.0010) [2023-10-10 19:34:04,688][123614] Updated weights for policy 1, policy_version 76340 (0.0011) [2023-10-10 19:34:05,063][123614] Updated weights for policy 1, policy_version 76350 (0.0008) [2023-10-10 19:34:06,077][123582] Updated weights for policy 0, policy_version 76453 (0.0010) [2023-10-10 19:34:06,447][123582] Updated weights for policy 0, policy_version 76463 (0.0010) [2023-10-10 19:34:06,811][123582] Updated weights for policy 0, policy_version 76473 (0.0011) [2023-10-10 19:34:08,711][123614] Updated weights for policy 1, policy_version 76360 (0.0007) [2023-10-10 19:34:08,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 156499968. Throughput: 0: 1801.3, 1: 1823.1. Samples: 39139410. Policy #0 lag: (min: 31.0, avg: 31.6, max: 46.0) [2023-10-10 19:34:08,789][122664] Avg episode reward: [(0, '79.250'), (1, '86.470')] [2023-10-10 19:34:09,081][123614] Updated weights for policy 1, policy_version 76370 (0.0010) [2023-10-10 19:34:09,449][123614] Updated weights for policy 1, policy_version 76380 (0.0009) [2023-10-10 19:34:10,518][123582] Updated weights for policy 0, policy_version 76483 (0.0009) [2023-10-10 19:34:10,892][123582] Updated weights for policy 0, policy_version 76493 (0.0010) [2023-10-10 19:34:11,264][123582] Updated weights for policy 0, policy_version 76503 (0.0009) [2023-10-10 19:34:13,013][123614] Updated weights for policy 1, policy_version 76390 (0.0009) [2023-10-10 19:34:13,380][123614] Updated weights for policy 1, policy_version 76400 (0.0007) [2023-10-10 19:34:13,741][123614] Updated weights for policy 1, policy_version 76410 (0.0008) [2023-10-10 19:34:13,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 156565504. Throughput: 0: 1812.4, 1: 1816.3. Samples: 39150210. Policy #0 lag: (min: 31.0, avg: 31.6, max: 46.0) [2023-10-10 19:34:13,788][122664] Avg episode reward: [(0, '75.140'), (1, '90.060')] [2023-10-10 19:34:15,000][123582] Updated weights for policy 0, policy_version 76513 (0.0007) [2023-10-10 19:34:15,376][123582] Updated weights for policy 0, policy_version 76523 (0.0007) [2023-10-10 19:34:15,745][123582] Updated weights for policy 0, policy_version 76533 (0.0009) [2023-10-10 19:34:16,123][123582] Updated weights for policy 0, policy_version 76543 (0.0008) [2023-10-10 19:34:17,412][123614] Updated weights for policy 1, policy_version 76420 (0.0010) [2023-10-10 19:34:17,773][123614] Updated weights for policy 1, policy_version 76430 (0.0007) [2023-10-10 19:34:18,135][123614] Updated weights for policy 1, policy_version 76440 (0.0007) [2023-10-10 19:34:18,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 156663808. Throughput: 0: 1814.7, 1: 1825.4. Samples: 39172542. Policy #0 lag: (min: 31.0, avg: 31.6, max: 46.0) [2023-10-10 19:34:18,789][122664] Avg episode reward: [(0, '73.330'), (1, '92.710')] [2023-10-10 19:34:19,735][123582] Updated weights for policy 0, policy_version 76553 (0.0008) [2023-10-10 19:34:20,103][123582] Updated weights for policy 0, policy_version 76563 (0.0008) [2023-10-10 19:34:20,478][123582] Updated weights for policy 0, policy_version 76573 (0.0008) [2023-10-10 19:34:21,752][123614] Updated weights for policy 1, policy_version 76450 (0.0007) [2023-10-10 19:34:22,122][123614] Updated weights for policy 1, policy_version 76460 (0.0008) [2023-10-10 19:34:22,485][123614] Updated weights for policy 1, policy_version 76470 (0.0007) [2023-10-10 19:34:22,860][123614] Updated weights for policy 1, policy_version 76480 (0.0009) [2023-10-10 19:34:23,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 156729344. Throughput: 0: 1821.2, 1: 1822.8. Samples: 39194726. Policy #0 lag: (min: 31.0, avg: 31.6, max: 46.0) [2023-10-10 19:34:23,788][122664] Avg episode reward: [(0, '69.510'), (1, '88.810')] [2023-10-10 19:34:24,031][123582] Updated weights for policy 0, policy_version 76583 (0.0010) [2023-10-10 19:34:24,403][123582] Updated weights for policy 0, policy_version 76593 (0.0010) [2023-10-10 19:34:24,765][123582] Updated weights for policy 0, policy_version 76603 (0.0011) [2023-10-10 19:34:26,437][123614] Updated weights for policy 1, policy_version 76490 (0.0007) [2023-10-10 19:34:26,805][123614] Updated weights for policy 1, policy_version 76500 (0.0008) [2023-10-10 19:34:27,180][123614] Updated weights for policy 1, policy_version 76510 (0.0009) [2023-10-10 19:34:28,369][123582] Updated weights for policy 0, policy_version 76613 (0.0009) [2023-10-10 19:34:28,738][123582] Updated weights for policy 0, policy_version 76623 (0.0007) [2023-10-10 19:34:28,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 156794880. Throughput: 0: 1823.8, 1: 1820.6. Samples: 39205460. Policy #0 lag: (min: 31.0, avg: 31.6, max: 46.0) [2023-10-10 19:34:28,788][122664] Avg episode reward: [(0, '64.850'), (1, '91.650')] [2023-10-10 19:34:29,106][123582] Updated weights for policy 0, policy_version 76633 (0.0007) [2023-10-10 19:34:30,749][123614] Updated weights for policy 1, policy_version 76520 (0.0007) [2023-10-10 19:34:31,114][123614] Updated weights for policy 1, policy_version 76530 (0.0008) [2023-10-10 19:34:31,481][123614] Updated weights for policy 1, policy_version 76540 (0.0008) [2023-10-10 19:34:32,833][123582] Updated weights for policy 0, policy_version 76643 (0.0011) [2023-10-10 19:34:33,202][123582] Updated weights for policy 0, policy_version 76653 (0.0010) [2023-10-10 19:34:33,576][123582] Updated weights for policy 0, policy_version 76663 (0.0008) [2023-10-10 19:34:33,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 156860416. Throughput: 0: 1828.0, 1: 1840.9. Samples: 39228194. Policy #0 lag: (min: 31.0, avg: 31.6, max: 46.0) [2023-10-10 19:34:33,789][122664] Avg episode reward: [(0, '65.650'), (1, '89.890')] [2023-10-10 19:34:35,103][123614] Updated weights for policy 1, policy_version 76550 (0.0010) [2023-10-10 19:34:35,466][123614] Updated weights for policy 1, policy_version 76560 (0.0011) [2023-10-10 19:34:35,835][123614] Updated weights for policy 1, policy_version 76570 (0.0008) [2023-10-10 19:34:37,167][123582] Updated weights for policy 0, policy_version 76673 (0.0009) [2023-10-10 19:34:37,540][123582] Updated weights for policy 0, policy_version 76683 (0.0009) [2023-10-10 19:34:37,916][123582] Updated weights for policy 0, policy_version 76693 (0.0010) [2023-10-10 19:34:38,276][123582] Updated weights for policy 0, policy_version 76703 (0.0010) [2023-10-10 19:34:38,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 156958720. Throughput: 0: 1821.7, 1: 1838.9. Samples: 39249472. Policy #0 lag: (min: 31.0, avg: 31.6, max: 46.0) [2023-10-10 19:34:38,788][122664] Avg episode reward: [(0, '65.260'), (1, '88.210')] [2023-10-10 19:34:39,715][123614] Updated weights for policy 1, policy_version 76580 (0.0007) [2023-10-10 19:34:40,077][123614] Updated weights for policy 1, policy_version 76590 (0.0008) [2023-10-10 19:34:40,439][123614] Updated weights for policy 1, policy_version 76600 (0.0011) [2023-10-10 19:34:41,899][123582] Updated weights for policy 0, policy_version 76713 (0.0007) [2023-10-10 19:34:42,278][123582] Updated weights for policy 0, policy_version 76723 (0.0010) [2023-10-10 19:34:42,634][123582] Updated weights for policy 0, policy_version 76733 (0.0010) [2023-10-10 19:34:43,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 157024256. Throughput: 0: 1831.2, 1: 1839.1. Samples: 39260992. Policy #0 lag: (min: 31.0, avg: 31.6, max: 46.0) [2023-10-10 19:34:43,789][122664] Avg episode reward: [(0, '68.190'), (1, '90.350')] [2023-10-10 19:34:44,080][123614] Updated weights for policy 1, policy_version 76610 (0.0010) [2023-10-10 19:34:44,449][123614] Updated weights for policy 1, policy_version 76620 (0.0008) [2023-10-10 19:34:44,821][123614] Updated weights for policy 1, policy_version 76630 (0.0008) [2023-10-10 19:34:45,184][123614] Updated weights for policy 1, policy_version 76640 (0.0007) [2023-10-10 19:34:46,397][123582] Updated weights for policy 0, policy_version 76743 (0.0008) [2023-10-10 19:34:46,771][123582] Updated weights for policy 0, policy_version 76753 (0.0010) [2023-10-10 19:34:47,138][123582] Updated weights for policy 0, policy_version 76763 (0.0007) [2023-10-10 19:34:48,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 157089792. Throughput: 0: 1829.5, 1: 1837.6. Samples: 39282280. Policy #0 lag: (min: 31.0, avg: 31.6, max: 46.0) [2023-10-10 19:34:48,789][122664] Avg episode reward: [(0, '72.060'), (1, '79.520')] [2023-10-10 19:34:48,965][123614] Updated weights for policy 1, policy_version 76650 (0.0009) [2023-10-10 19:34:49,336][123614] Updated weights for policy 1, policy_version 76660 (0.0009) [2023-10-10 19:34:49,715][123614] Updated weights for policy 1, policy_version 76670 (0.0009) [2023-10-10 19:34:50,884][123582] Updated weights for policy 0, policy_version 76773 (0.0008) [2023-10-10 19:34:51,251][123582] Updated weights for policy 0, policy_version 76783 (0.0009) [2023-10-10 19:34:51,612][123582] Updated weights for policy 0, policy_version 76793 (0.0007) [2023-10-10 19:34:53,322][123614] Updated weights for policy 1, policy_version 76680 (0.0008) [2023-10-10 19:34:53,691][123614] Updated weights for policy 1, policy_version 76690 (0.0009) [2023-10-10 19:34:53,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 157155328. Throughput: 0: 1834.1, 1: 1824.8. Samples: 39304062. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:34:53,788][122664] Avg episode reward: [(0, '71.040'), (1, '83.030')] [2023-10-10 19:34:54,057][123614] Updated weights for policy 1, policy_version 76700 (0.0010) [2023-10-10 19:34:55,384][123582] Updated weights for policy 0, policy_version 76803 (0.0008) [2023-10-10 19:34:55,759][123582] Updated weights for policy 0, policy_version 76813 (0.0007) [2023-10-10 19:34:56,133][123582] Updated weights for policy 0, policy_version 76823 (0.0008) [2023-10-10 19:34:57,773][123614] Updated weights for policy 1, policy_version 76710 (0.0008) [2023-10-10 19:34:58,137][123614] Updated weights for policy 1, policy_version 76720 (0.0010) [2023-10-10 19:34:58,507][123614] Updated weights for policy 1, policy_version 76730 (0.0007) [2023-10-10 19:34:58,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 157253632. Throughput: 0: 1828.5, 1: 1834.6. Samples: 39315048. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:34:58,788][122664] Avg episode reward: [(0, '70.010'), (1, '81.310')] [2023-10-10 19:34:59,828][123582] Updated weights for policy 0, policy_version 76833 (0.0008) [2023-10-10 19:35:00,198][123582] Updated weights for policy 0, policy_version 76843 (0.0009) [2023-10-10 19:35:00,563][123582] Updated weights for policy 0, policy_version 76853 (0.0009) [2023-10-10 19:35:00,931][123582] Updated weights for policy 0, policy_version 76863 (0.0009) [2023-10-10 19:35:02,245][123614] Updated weights for policy 1, policy_version 76740 (0.0007) [2023-10-10 19:35:02,619][123614] Updated weights for policy 1, policy_version 76750 (0.0008) [2023-10-10 19:35:02,983][123614] Updated weights for policy 1, policy_version 76760 (0.0010) [2023-10-10 19:35:03,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 157319168. Throughput: 0: 1824.6, 1: 1826.0. Samples: 39336820. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:35:03,789][122664] Avg episode reward: [(0, '76.390'), (1, '81.720')] [2023-10-10 19:35:04,599][123582] Updated weights for policy 0, policy_version 76873 (0.0009) [2023-10-10 19:35:04,976][123582] Updated weights for policy 0, policy_version 76883 (0.0009) [2023-10-10 19:35:05,355][123582] Updated weights for policy 0, policy_version 76893 (0.0009) [2023-10-10 19:35:06,609][123614] Updated weights for policy 1, policy_version 76770 (0.0009) [2023-10-10 19:35:06,975][123614] Updated weights for policy 1, policy_version 76780 (0.0008) [2023-10-10 19:35:07,341][123614] Updated weights for policy 1, policy_version 76790 (0.0011) [2023-10-10 19:35:07,722][123614] Updated weights for policy 1, policy_version 76800 (0.0010) [2023-10-10 19:35:08,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 157384704. Throughput: 0: 1818.7, 1: 1833.7. Samples: 39359084. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:35:08,789][122664] Avg episode reward: [(0, '77.530'), (1, '79.660')] [2023-10-10 19:35:08,897][123582] Updated weights for policy 0, policy_version 76903 (0.0008) [2023-10-10 19:35:09,273][123582] Updated weights for policy 0, policy_version 76913 (0.0009) [2023-10-10 19:35:09,631][123582] Updated weights for policy 0, policy_version 76923 (0.0010) [2023-10-10 19:35:11,364][123614] Updated weights for policy 1, policy_version 76810 (0.0009) [2023-10-10 19:35:11,735][123614] Updated weights for policy 1, policy_version 76820 (0.0008) [2023-10-10 19:35:12,105][123614] Updated weights for policy 1, policy_version 76830 (0.0010) [2023-10-10 19:35:13,125][123582] Updated weights for policy 0, policy_version 76933 (0.0007) [2023-10-10 19:35:13,492][123582] Updated weights for policy 0, policy_version 76943 (0.0007) [2023-10-10 19:35:13,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 157450240. Throughput: 0: 1825.6, 1: 1825.3. Samples: 39369752. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:35:13,788][122664] Avg episode reward: [(0, '80.470'), (1, '82.600')] [2023-10-10 19:35:13,862][123582] Updated weights for policy 0, policy_version 76953 (0.0007) [2023-10-10 19:35:15,710][123614] Updated weights for policy 1, policy_version 76840 (0.0008) [2023-10-10 19:35:16,070][123614] Updated weights for policy 1, policy_version 76850 (0.0009) [2023-10-10 19:35:16,437][123614] Updated weights for policy 1, policy_version 76860 (0.0007) [2023-10-10 19:35:17,640][123582] Updated weights for policy 0, policy_version 76963 (0.0009) [2023-10-10 19:35:18,008][123582] Updated weights for policy 0, policy_version 76973 (0.0008) [2023-10-10 19:35:18,373][123582] Updated weights for policy 0, policy_version 76983 (0.0008) [2023-10-10 19:35:18,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 157548544. Throughput: 0: 1817.8, 1: 1823.9. Samples: 39392070. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:35:18,789][122664] Avg episode reward: [(0, '81.290'), (1, '76.070')] [2023-10-10 19:35:20,100][123614] Updated weights for policy 1, policy_version 76870 (0.0008) [2023-10-10 19:35:20,475][123614] Updated weights for policy 1, policy_version 76880 (0.0007) [2023-10-10 19:35:20,842][123614] Updated weights for policy 1, policy_version 76890 (0.0007) [2023-10-10 19:35:22,093][123582] Updated weights for policy 0, policy_version 76993 (0.0008) [2023-10-10 19:35:22,466][123582] Updated weights for policy 0, policy_version 77003 (0.0007) [2023-10-10 19:35:22,829][123582] Updated weights for policy 0, policy_version 77013 (0.0009) [2023-10-10 19:35:23,209][123582] Updated weights for policy 0, policy_version 77023 (0.0009) [2023-10-10 19:35:23,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 157614080. Throughput: 0: 1814.8, 1: 1830.1. Samples: 39413492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:35:23,788][122664] Avg episode reward: [(0, '83.160'), (1, '74.390')] [2023-10-10 19:35:24,441][123614] Updated weights for policy 1, policy_version 76900 (0.0008) [2023-10-10 19:35:24,818][123614] Updated weights for policy 1, policy_version 76910 (0.0010) [2023-10-10 19:35:25,182][123614] Updated weights for policy 1, policy_version 76920 (0.0011) [2023-10-10 19:35:27,006][123582] Updated weights for policy 0, policy_version 77033 (0.0010) [2023-10-10 19:35:27,373][123582] Updated weights for policy 0, policy_version 77043 (0.0010) [2023-10-10 19:35:27,743][123582] Updated weights for policy 0, policy_version 77053 (0.0009) [2023-10-10 19:35:28,756][123614] Updated weights for policy 1, policy_version 76930 (0.0008) [2023-10-10 19:35:28,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 157679616. Throughput: 0: 1810.2, 1: 1830.0. Samples: 39424802. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:35:28,789][122664] Avg episode reward: [(0, '83.640'), (1, '70.460')] [2023-10-10 19:35:29,134][123614] Updated weights for policy 1, policy_version 76940 (0.0008) [2023-10-10 19:35:29,499][123614] Updated weights for policy 1, policy_version 76950 (0.0010) [2023-10-10 19:35:29,858][123614] Updated weights for policy 1, policy_version 76960 (0.0011) [2023-10-10 19:35:31,451][123582] Updated weights for policy 0, policy_version 77063 (0.0008) [2023-10-10 19:35:31,827][123582] Updated weights for policy 0, policy_version 77073 (0.0009) [2023-10-10 19:35:32,193][123582] Updated weights for policy 0, policy_version 77083 (0.0009) [2023-10-10 19:35:33,471][123614] Updated weights for policy 1, policy_version 76970 (0.0008) [2023-10-10 19:35:33,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 157745152. Throughput: 0: 1809.5, 1: 1833.4. Samples: 39446208. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:35:33,788][122664] Avg episode reward: [(0, '85.910'), (1, '69.950')] [2023-10-10 19:35:33,842][123614] Updated weights for policy 1, policy_version 76980 (0.0008) [2023-10-10 19:35:34,216][123614] Updated weights for policy 1, policy_version 76990 (0.0007) [2023-10-10 19:35:36,043][123582] Updated weights for policy 0, policy_version 77093 (0.0009) [2023-10-10 19:35:36,434][123582] Updated weights for policy 0, policy_version 77103 (0.0008) [2023-10-10 19:35:36,818][123582] Updated weights for policy 0, policy_version 77113 (0.0008) [2023-10-10 19:35:38,016][123614] Updated weights for policy 1, policy_version 77000 (0.0008) [2023-10-10 19:35:38,391][123614] Updated weights for policy 1, policy_version 77010 (0.0009) [2023-10-10 19:35:38,761][123614] Updated weights for policy 1, policy_version 77020 (0.0007) [2023-10-10 19:35:38,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 157810688. Throughput: 0: 1806.9, 1: 1829.9. Samples: 39467720. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:35:38,788][122664] Avg episode reward: [(0, '86.500'), (1, '72.550')] [2023-10-10 19:35:40,635][123582] Updated weights for policy 0, policy_version 77123 (0.0012) [2023-10-10 19:35:41,010][123582] Updated weights for policy 0, policy_version 77133 (0.0009) [2023-10-10 19:35:41,368][123582] Updated weights for policy 0, policy_version 77143 (0.0010) [2023-10-10 19:35:42,503][123614] Updated weights for policy 1, policy_version 77030 (0.0009) [2023-10-10 19:35:42,873][123614] Updated weights for policy 1, policy_version 77040 (0.0008) [2023-10-10 19:35:43,242][123614] Updated weights for policy 1, policy_version 77050 (0.0008) [2023-10-10 19:35:43,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 157908992. Throughput: 0: 1806.4, 1: 1833.3. Samples: 39478836. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-10 19:35:43,789][122664] Avg episode reward: [(0, '84.860'), (1, '76.320')] [2023-10-10 19:35:45,124][123582] Updated weights for policy 0, policy_version 77153 (0.0011) [2023-10-10 19:35:45,499][123582] Updated weights for policy 0, policy_version 77163 (0.0007) [2023-10-10 19:35:45,874][123582] Updated weights for policy 0, policy_version 77173 (0.0011) [2023-10-10 19:35:46,241][123582] Updated weights for policy 0, policy_version 77183 (0.0011) [2023-10-10 19:35:46,882][123614] Updated weights for policy 1, policy_version 77060 (0.0008) [2023-10-10 19:35:47,254][123614] Updated weights for policy 1, policy_version 77070 (0.0008) [2023-10-10 19:35:47,620][123614] Updated weights for policy 1, policy_version 77080 (0.0008) [2023-10-10 19:35:48,788][122664] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 157974528. Throughput: 0: 1798.9, 1: 1828.8. Samples: 39500066. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-10 19:35:48,789][122664] Avg episode reward: [(0, '86.410'), (1, '77.580')] [2023-10-10 19:35:49,991][123582] Updated weights for policy 0, policy_version 77193 (0.0009) [2023-10-10 19:35:50,358][123582] Updated weights for policy 0, policy_version 77203 (0.0009) [2023-10-10 19:35:50,715][123582] Updated weights for policy 0, policy_version 77213 (0.0007) [2023-10-10 19:35:51,306][123614] Updated weights for policy 1, policy_version 77090 (0.0009) [2023-10-10 19:35:51,677][123614] Updated weights for policy 1, policy_version 77100 (0.0008) [2023-10-10 19:35:52,037][123614] Updated weights for policy 1, policy_version 77110 (0.0007) [2023-10-10 19:35:52,406][123614] Updated weights for policy 1, policy_version 77120 (0.0009) [2023-10-10 19:35:53,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 158040064. Throughput: 0: 1801.2, 1: 1833.2. Samples: 39522630. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-10 19:35:53,788][122664] Avg episode reward: [(0, '93.910'), (1, '74.220')] [2023-10-10 19:35:53,800][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000077216_79069184.pth... [2023-10-10 19:35:53,800][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000077120_78970880.pth... [2023-10-10 19:35:53,832][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000075520_77332480.pth [2023-10-10 19:35:53,835][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000075424_77234176.pth [2023-10-10 19:35:54,329][123582] Updated weights for policy 0, policy_version 77223 (0.0011) [2023-10-10 19:35:54,701][123582] Updated weights for policy 0, policy_version 77233 (0.0009) [2023-10-10 19:35:55,070][123582] Updated weights for policy 0, policy_version 77243 (0.0011) [2023-10-10 19:35:56,185][123614] Updated weights for policy 1, policy_version 77130 (0.0008) [2023-10-10 19:35:56,552][123614] Updated weights for policy 1, policy_version 77140 (0.0007) [2023-10-10 19:35:56,931][123614] Updated weights for policy 1, policy_version 77150 (0.0008) [2023-10-10 19:35:58,788][122664] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 158105600. Throughput: 0: 1799.4, 1: 1832.4. Samples: 39533186. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-10 19:35:58,788][122664] Avg episode reward: [(0, '95.010'), (1, '75.880')] [2023-10-10 19:35:58,872][123582] Updated weights for policy 0, policy_version 77253 (0.0010) [2023-10-10 19:35:59,239][123582] Updated weights for policy 0, policy_version 77263 (0.0008) [2023-10-10 19:35:59,608][123582] Updated weights for policy 0, policy_version 77273 (0.0007) [2023-10-10 19:36:00,698][123614] Updated weights for policy 1, policy_version 77160 (0.0010) [2023-10-10 19:36:01,069][123614] Updated weights for policy 1, policy_version 77170 (0.0010) [2023-10-10 19:36:01,441][123614] Updated weights for policy 1, policy_version 77180 (0.0008) [2023-10-10 19:36:03,130][123582] Updated weights for policy 0, policy_version 77283 (0.0009) [2023-10-10 19:36:03,512][123582] Updated weights for policy 0, policy_version 77293 (0.0008) [2023-10-10 19:36:03,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 158171136. Throughput: 0: 1803.1, 1: 1826.5. Samples: 39555400. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-10 19:36:03,788][122664] Avg episode reward: [(0, '96.790'), (1, '75.930')] [2023-10-10 19:36:03,882][123582] Updated weights for policy 0, policy_version 77303 (0.0008) [2023-10-10 19:36:05,194][123614] Updated weights for policy 1, policy_version 77190 (0.0007) [2023-10-10 19:36:05,575][123614] Updated weights for policy 1, policy_version 77200 (0.0009) [2023-10-10 19:36:05,938][123614] Updated weights for policy 1, policy_version 77210 (0.0010) [2023-10-10 19:36:07,563][123582] Updated weights for policy 0, policy_version 77313 (0.0008) [2023-10-10 19:36:07,939][123582] Updated weights for policy 0, policy_version 77323 (0.0011) [2023-10-10 19:36:08,309][123582] Updated weights for policy 0, policy_version 77333 (0.0008) [2023-10-10 19:36:08,679][123582] Updated weights for policy 0, policy_version 77343 (0.0008) [2023-10-10 19:36:08,788][122664] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 158269440. Throughput: 0: 1813.6, 1: 1819.9. Samples: 39577002. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-10 19:36:08,789][122664] Avg episode reward: [(0, '104.430'), (1, '72.040')] [2023-10-10 19:36:09,624][123614] Updated weights for policy 1, policy_version 77220 (0.0008) [2023-10-10 19:36:09,985][123614] Updated weights for policy 1, policy_version 77230 (0.0007) [2023-10-10 19:36:10,355][123614] Updated weights for policy 1, policy_version 77240 (0.0007) [2023-10-10 19:36:12,259][123582] Updated weights for policy 0, policy_version 77353 (0.0010) [2023-10-10 19:36:12,641][123582] Updated weights for policy 0, policy_version 77363 (0.0011) [2023-10-10 19:36:13,002][123582] Updated weights for policy 0, policy_version 77373 (0.0010) [2023-10-10 19:36:13,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 158334976. Throughput: 0: 1810.4, 1: 1818.8. Samples: 39588114. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-10 19:36:13,789][122664] Avg episode reward: [(0, '101.930'), (1, '75.860')] [2023-10-10 19:36:13,924][123614] Updated weights for policy 1, policy_version 77250 (0.0008) [2023-10-10 19:36:14,296][123614] Updated weights for policy 1, policy_version 77260 (0.0009) [2023-10-10 19:36:14,654][123614] Updated weights for policy 1, policy_version 77270 (0.0010) [2023-10-10 19:36:15,024][123614] Updated weights for policy 1, policy_version 77280 (0.0008) [2023-10-10 19:36:16,785][123582] Updated weights for policy 0, policy_version 77383 (0.0008) [2023-10-10 19:36:17,159][123582] Updated weights for policy 0, policy_version 77393 (0.0008) [2023-10-10 19:36:17,530][123582] Updated weights for policy 0, policy_version 77403 (0.0007) [2023-10-10 19:36:18,651][123614] Updated weights for policy 1, policy_version 77290 (0.0008) [2023-10-10 19:36:18,788][122664] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 158400512. Throughput: 0: 1817.6, 1: 1822.7. Samples: 39610022. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-10 19:36:18,788][122664] Avg episode reward: [(0, '101.330'), (1, '76.520')] [2023-10-10 19:36:19,017][123614] Updated weights for policy 1, policy_version 77300 (0.0009) [2023-10-10 19:36:19,381][123614] Updated weights for policy 1, policy_version 77310 (0.0009) [2023-10-10 19:36:21,282][123582] Updated weights for policy 0, policy_version 77413 (0.0008) [2023-10-10 19:36:21,670][123582] Updated weights for policy 0, policy_version 77423 (0.0009) [2023-10-10 19:36:22,045][123582] Updated weights for policy 0, policy_version 77433 (0.0009) [2023-10-10 19:36:22,980][123614] Updated weights for policy 1, policy_version 77320 (0.0009) [2023-10-10 19:36:23,356][123614] Updated weights for policy 1, policy_version 77330 (0.0007) [2023-10-10 19:36:23,733][123614] Updated weights for policy 1, policy_version 77340 (0.0008) [2023-10-10 19:36:23,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 158466048. Throughput: 0: 1808.0, 1: 1819.5. Samples: 39630956. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-10 19:36:23,789][122664] Avg episode reward: [(0, '98.670'), (1, '78.340')] [2023-10-10 19:36:25,769][123582] Updated weights for policy 0, policy_version 77443 (0.0008) [2023-10-10 19:36:26,143][123582] Updated weights for policy 0, policy_version 77453 (0.0009) [2023-10-10 19:36:26,511][123582] Updated weights for policy 0, policy_version 77463 (0.0007) [2023-10-10 19:36:27,289][123614] Updated weights for policy 1, policy_version 77350 (0.0009) [2023-10-10 19:36:27,657][123614] Updated weights for policy 1, policy_version 77360 (0.0009) [2023-10-10 19:36:28,031][123614] Updated weights for policy 1, policy_version 77370 (0.0010) [2023-10-10 19:36:28,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 158564352. Throughput: 0: 1818.2, 1: 1826.6. Samples: 39642852. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-10 19:36:28,789][122664] Avg episode reward: [(0, '100.380'), (1, '78.290')] [2023-10-10 19:36:30,289][123582] Updated weights for policy 0, policy_version 77473 (0.0008) [2023-10-10 19:36:30,654][123582] Updated weights for policy 0, policy_version 77483 (0.0009) [2023-10-10 19:36:31,018][123582] Updated weights for policy 0, policy_version 77493 (0.0008) [2023-10-10 19:36:31,398][123582] Updated weights for policy 0, policy_version 77503 (0.0009) [2023-10-10 19:36:31,834][123614] Updated weights for policy 1, policy_version 77380 (0.0009) [2023-10-10 19:36:32,208][123614] Updated weights for policy 1, policy_version 77390 (0.0007) [2023-10-10 19:36:32,579][123614] Updated weights for policy 1, policy_version 77400 (0.0010) [2023-10-10 19:36:33,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 158629888. Throughput: 0: 1818.8, 1: 1822.0. Samples: 39663900. Policy #0 lag: (min: 16.0, avg: 40.3, max: 48.0) [2023-10-10 19:36:33,789][122664] Avg episode reward: [(0, '104.690'), (1, '76.090')] [2023-10-10 19:36:34,969][123582] Updated weights for policy 0, policy_version 77513 (0.0009) [2023-10-10 19:36:35,343][123582] Updated weights for policy 0, policy_version 77523 (0.0010) [2023-10-10 19:36:35,710][123582] Updated weights for policy 0, policy_version 77533 (0.0008) [2023-10-10 19:36:36,214][123614] Updated weights for policy 1, policy_version 77410 (0.0008) [2023-10-10 19:36:36,587][123614] Updated weights for policy 1, policy_version 77420 (0.0007) [2023-10-10 19:36:36,946][123614] Updated weights for policy 1, policy_version 77430 (0.0008) [2023-10-10 19:36:37,316][123614] Updated weights for policy 1, policy_version 77440 (0.0007) [2023-10-10 19:36:38,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 158695424. Throughput: 0: 1817.2, 1: 1827.1. Samples: 39686624. Policy #0 lag: (min: 16.0, avg: 40.3, max: 48.0) [2023-10-10 19:36:38,789][122664] Avg episode reward: [(0, '98.260'), (1, '74.470')] [2023-10-10 19:36:39,440][123582] Updated weights for policy 0, policy_version 77543 (0.0010) [2023-10-10 19:36:39,811][123582] Updated weights for policy 0, policy_version 77553 (0.0008) [2023-10-10 19:36:40,189][123582] Updated weights for policy 0, policy_version 77563 (0.0008) [2023-10-10 19:36:40,894][123614] Updated weights for policy 1, policy_version 77450 (0.0008) [2023-10-10 19:36:41,262][123614] Updated weights for policy 1, policy_version 77460 (0.0011) [2023-10-10 19:36:41,632][123614] Updated weights for policy 1, policy_version 77470 (0.0007) [2023-10-10 19:36:43,717][123582] Updated weights for policy 0, policy_version 77573 (0.0009) [2023-10-10 19:36:43,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 158760960. Throughput: 0: 1815.9, 1: 1818.6. Samples: 39696738. Policy #0 lag: (min: 16.0, avg: 40.3, max: 48.0) [2023-10-10 19:36:43,788][122664] Avg episode reward: [(0, '98.220'), (1, '74.310')] [2023-10-10 19:36:44,085][123582] Updated weights for policy 0, policy_version 77583 (0.0008) [2023-10-10 19:36:44,458][123582] Updated weights for policy 0, policy_version 77593 (0.0009) [2023-10-10 19:36:45,284][123614] Updated weights for policy 1, policy_version 77480 (0.0008) [2023-10-10 19:36:45,660][123614] Updated weights for policy 1, policy_version 77490 (0.0007) [2023-10-10 19:36:46,029][123614] Updated weights for policy 1, policy_version 77500 (0.0008) [2023-10-10 19:36:48,166][123582] Updated weights for policy 0, policy_version 77603 (0.0007) [2023-10-10 19:36:48,529][123582] Updated weights for policy 0, policy_version 77613 (0.0007) [2023-10-10 19:36:48,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 158826496. Throughput: 0: 1818.4, 1: 1831.4. Samples: 39719640. Policy #0 lag: (min: 16.0, avg: 40.3, max: 48.0) [2023-10-10 19:36:48,788][122664] Avg episode reward: [(0, '96.590'), (1, '74.440')] [2023-10-10 19:36:48,904][123582] Updated weights for policy 0, policy_version 77623 (0.0007) [2023-10-10 19:36:49,843][123614] Updated weights for policy 1, policy_version 77510 (0.0009) [2023-10-10 19:36:50,231][123614] Updated weights for policy 1, policy_version 77520 (0.0010) [2023-10-10 19:36:50,603][123614] Updated weights for policy 1, policy_version 77530 (0.0008) [2023-10-10 19:36:52,554][123582] Updated weights for policy 0, policy_version 77633 (0.0008) [2023-10-10 19:36:52,929][123582] Updated weights for policy 0, policy_version 77643 (0.0010) [2023-10-10 19:36:53,312][123582] Updated weights for policy 0, policy_version 77653 (0.0010) [2023-10-10 19:36:53,672][123582] Updated weights for policy 0, policy_version 77663 (0.0010) [2023-10-10 19:36:53,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 158924800. Throughput: 0: 1816.6, 1: 1832.3. Samples: 39741204. Policy #0 lag: (min: 16.0, avg: 40.3, max: 48.0) [2023-10-10 19:36:53,789][122664] Avg episode reward: [(0, '94.570'), (1, '71.870')] [2023-10-10 19:36:54,156][123614] Updated weights for policy 1, policy_version 77540 (0.0008) [2023-10-10 19:36:54,519][123614] Updated weights for policy 1, policy_version 77550 (0.0009) [2023-10-10 19:36:54,876][123614] Updated weights for policy 1, policy_version 77560 (0.0007) [2023-10-10 19:36:57,478][123582] Updated weights for policy 0, policy_version 77673 (0.0008) [2023-10-10 19:36:57,850][123582] Updated weights for policy 0, policy_version 77683 (0.0007) [2023-10-10 19:36:58,223][123582] Updated weights for policy 0, policy_version 77693 (0.0007) [2023-10-10 19:36:58,780][123614] Updated weights for policy 1, policy_version 77570 (0.0008) [2023-10-10 19:36:58,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 158990336. Throughput: 0: 1806.1, 1: 1839.5. Samples: 39752168. Policy #0 lag: (min: 16.0, avg: 40.3, max: 48.0) [2023-10-10 19:36:58,789][122664] Avg episode reward: [(0, '96.260'), (1, '75.470')] [2023-10-10 19:36:59,145][123614] Updated weights for policy 1, policy_version 77580 (0.0007) [2023-10-10 19:36:59,521][123614] Updated weights for policy 1, policy_version 77590 (0.0010) [2023-10-10 19:36:59,884][123614] Updated weights for policy 1, policy_version 77600 (0.0008) [2023-10-10 19:37:01,958][123582] Updated weights for policy 0, policy_version 77703 (0.0009) [2023-10-10 19:37:02,322][123582] Updated weights for policy 0, policy_version 77713 (0.0008) [2023-10-10 19:37:02,710][123582] Updated weights for policy 0, policy_version 77723 (0.0010) [2023-10-10 19:37:03,498][123614] Updated weights for policy 1, policy_version 77610 (0.0007) [2023-10-10 19:37:03,788][122664] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 159055872. Throughput: 0: 1815.9, 1: 1828.6. Samples: 39774026. Policy #0 lag: (min: 16.0, avg: 40.3, max: 48.0) [2023-10-10 19:37:03,788][122664] Avg episode reward: [(0, '94.700'), (1, '77.120')] [2023-10-10 19:37:03,863][123614] Updated weights for policy 1, policy_version 77620 (0.0009) [2023-10-10 19:37:04,238][123614] Updated weights for policy 1, policy_version 77630 (0.0009) [2023-10-10 19:37:06,300][123582] Updated weights for policy 0, policy_version 77733 (0.0009) [2023-10-10 19:37:06,683][123582] Updated weights for policy 0, policy_version 77743 (0.0010) [2023-10-10 19:37:07,057][123582] Updated weights for policy 0, policy_version 77753 (0.0010) [2023-10-10 19:37:07,901][123614] Updated weights for policy 1, policy_version 77640 (0.0009) [2023-10-10 19:37:08,278][123614] Updated weights for policy 1, policy_version 77650 (0.0008) [2023-10-10 19:37:08,657][123614] Updated weights for policy 1, policy_version 77660 (0.0008) [2023-10-10 19:37:08,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 159121408. Throughput: 0: 1817.3, 1: 1826.0. Samples: 39794906. Policy #0 lag: (min: 16.0, avg: 40.3, max: 48.0) [2023-10-10 19:37:08,788][122664] Avg episode reward: [(0, '91.330'), (1, '77.680')] [2023-10-10 19:37:10,859][123582] Updated weights for policy 0, policy_version 77763 (0.0011) [2023-10-10 19:37:11,231][123582] Updated weights for policy 0, policy_version 77773 (0.0011) [2023-10-10 19:37:11,599][123582] Updated weights for policy 0, policy_version 77783 (0.0010) [2023-10-10 19:37:12,248][123614] Updated weights for policy 1, policy_version 77670 (0.0011) [2023-10-10 19:37:12,620][123614] Updated weights for policy 1, policy_version 77680 (0.0009) [2023-10-10 19:37:12,982][123614] Updated weights for policy 1, policy_version 77690 (0.0011) [2023-10-10 19:37:13,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 159219712. Throughput: 0: 1817.2, 1: 1824.6. Samples: 39806734. Policy #0 lag: (min: 16.0, avg: 40.3, max: 48.0) [2023-10-10 19:37:13,789][122664] Avg episode reward: [(0, '91.470'), (1, '77.870')] [2023-10-10 19:37:15,291][123582] Updated weights for policy 0, policy_version 77793 (0.0009) [2023-10-10 19:37:15,662][123582] Updated weights for policy 0, policy_version 77803 (0.0008) [2023-10-10 19:37:16,026][123582] Updated weights for policy 0, policy_version 77813 (0.0007) [2023-10-10 19:37:16,406][123582] Updated weights for policy 0, policy_version 77823 (0.0008) [2023-10-10 19:37:16,777][123614] Updated weights for policy 1, policy_version 77700 (0.0011) [2023-10-10 19:37:17,149][123614] Updated weights for policy 1, policy_version 77710 (0.0010) [2023-10-10 19:37:17,521][123614] Updated weights for policy 1, policy_version 77720 (0.0010) [2023-10-10 19:37:18,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 159285248. Throughput: 0: 1812.1, 1: 1824.9. Samples: 39827560. Policy #0 lag: (min: 16.0, avg: 40.3, max: 48.0) [2023-10-10 19:37:18,788][122664] Avg episode reward: [(0, '88.090'), (1, '82.340')] [2023-10-10 19:37:20,133][123582] Updated weights for policy 0, policy_version 77833 (0.0009) [2023-10-10 19:37:20,509][123582] Updated weights for policy 0, policy_version 77843 (0.0009) [2023-10-10 19:37:20,882][123582] Updated weights for policy 0, policy_version 77853 (0.0009) [2023-10-10 19:37:21,277][123614] Updated weights for policy 1, policy_version 77730 (0.0010) [2023-10-10 19:37:21,642][123614] Updated weights for policy 1, policy_version 77740 (0.0009) [2023-10-10 19:37:22,006][123614] Updated weights for policy 1, policy_version 77750 (0.0010) [2023-10-10 19:37:22,375][123614] Updated weights for policy 1, policy_version 77760 (0.0010) [2023-10-10 19:37:23,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 159350784. Throughput: 0: 1810.7, 1: 1818.0. Samples: 39849918. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:37:23,789][122664] Avg episode reward: [(0, '90.430'), (1, '83.100')] [2023-10-10 19:37:24,472][123582] Updated weights for policy 0, policy_version 77863 (0.0010) [2023-10-10 19:37:24,843][123582] Updated weights for policy 0, policy_version 77873 (0.0008) [2023-10-10 19:37:25,218][123582] Updated weights for policy 0, policy_version 77883 (0.0008) [2023-10-10 19:37:26,169][123614] Updated weights for policy 1, policy_version 77770 (0.0008) [2023-10-10 19:37:26,543][123614] Updated weights for policy 1, policy_version 77780 (0.0009) [2023-10-10 19:37:26,916][123614] Updated weights for policy 1, policy_version 77790 (0.0008) [2023-10-10 19:37:28,788][122664] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 159416320. Throughput: 0: 1807.4, 1: 1823.7. Samples: 39860138. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:37:28,790][122664] Avg episode reward: [(0, '83.020'), (1, '84.970')] [2023-10-10 19:37:29,010][123582] Updated weights for policy 0, policy_version 77893 (0.0011) [2023-10-10 19:37:29,378][123582] Updated weights for policy 0, policy_version 77903 (0.0008) [2023-10-10 19:37:29,741][123582] Updated weights for policy 0, policy_version 77913 (0.0007) [2023-10-10 19:37:30,701][123614] Updated weights for policy 1, policy_version 77800 (0.0008) [2023-10-10 19:37:31,070][123614] Updated weights for policy 1, policy_version 77810 (0.0009) [2023-10-10 19:37:31,438][123614] Updated weights for policy 1, policy_version 77820 (0.0009) [2023-10-10 19:37:33,267][123582] Updated weights for policy 0, policy_version 77923 (0.0007) [2023-10-10 19:37:33,624][123582] Updated weights for policy 0, policy_version 77933 (0.0007) [2023-10-10 19:37:33,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 159481856. Throughput: 0: 1805.1, 1: 1812.8. Samples: 39882450. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:37:33,789][122664] Avg episode reward: [(0, '86.890'), (1, '89.470')] [2023-10-10 19:37:34,001][123582] Updated weights for policy 0, policy_version 77943 (0.0007) [2023-10-10 19:37:35,262][123614] Updated weights for policy 1, policy_version 77830 (0.0008) [2023-10-10 19:37:35,646][123614] Updated weights for policy 1, policy_version 77840 (0.0010) [2023-10-10 19:37:36,013][123614] Updated weights for policy 1, policy_version 77850 (0.0009) [2023-10-10 19:37:37,762][123582] Updated weights for policy 0, policy_version 77953 (0.0009) [2023-10-10 19:37:38,128][123582] Updated weights for policy 0, policy_version 77963 (0.0008) [2023-10-10 19:37:38,504][123582] Updated weights for policy 0, policy_version 77973 (0.0008) [2023-10-10 19:37:38,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 159547392. Throughput: 0: 1814.0, 1: 1808.1. Samples: 39904198. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:37:38,789][122664] Avg episode reward: [(0, '83.990'), (1, '85.800')] [2023-10-10 19:37:38,889][123582] Updated weights for policy 0, policy_version 77983 (0.0009) [2023-10-10 19:37:39,663][123614] Updated weights for policy 1, policy_version 77860 (0.0008) [2023-10-10 19:37:40,039][123614] Updated weights for policy 1, policy_version 77870 (0.0010) [2023-10-10 19:37:40,403][123614] Updated weights for policy 1, policy_version 77880 (0.0010) [2023-10-10 19:37:42,484][123582] Updated weights for policy 0, policy_version 77993 (0.0010) [2023-10-10 19:37:42,851][123582] Updated weights for policy 0, policy_version 78003 (0.0009) [2023-10-10 19:37:43,229][123582] Updated weights for policy 0, policy_version 78013 (0.0010) [2023-10-10 19:37:43,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 159645696. Throughput: 0: 1817.8, 1: 1801.8. Samples: 39915050. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:37:43,789][122664] Avg episode reward: [(0, '85.040'), (1, '83.710')] [2023-10-10 19:37:44,014][123614] Updated weights for policy 1, policy_version 77890 (0.0010) [2023-10-10 19:37:44,393][123614] Updated weights for policy 1, policy_version 77900 (0.0009) [2023-10-10 19:37:44,754][123614] Updated weights for policy 1, policy_version 77910 (0.0010) [2023-10-10 19:37:45,130][123614] Updated weights for policy 1, policy_version 77920 (0.0011) [2023-10-10 19:37:46,904][123582] Updated weights for policy 0, policy_version 78023 (0.0008) [2023-10-10 19:37:47,276][123582] Updated weights for policy 0, policy_version 78033 (0.0009) [2023-10-10 19:37:47,652][123582] Updated weights for policy 0, policy_version 78043 (0.0010) [2023-10-10 19:37:48,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 159711232. Throughput: 0: 1819.1, 1: 1800.8. Samples: 39936922. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:37:48,789][122664] Avg episode reward: [(0, '76.790'), (1, '83.030')] [2023-10-10 19:37:48,953][123614] Updated weights for policy 1, policy_version 77930 (0.0007) [2023-10-10 19:37:49,313][123614] Updated weights for policy 1, policy_version 77940 (0.0008) [2023-10-10 19:37:49,679][123614] Updated weights for policy 1, policy_version 77950 (0.0009) [2023-10-10 19:37:51,408][123582] Updated weights for policy 0, policy_version 78053 (0.0009) [2023-10-10 19:37:51,793][123582] Updated weights for policy 0, policy_version 78063 (0.0007) [2023-10-10 19:37:52,167][123582] Updated weights for policy 0, policy_version 78073 (0.0011) [2023-10-10 19:37:53,499][123614] Updated weights for policy 1, policy_version 77960 (0.0008) [2023-10-10 19:37:53,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 159776768. Throughput: 0: 1818.2, 1: 1811.1. Samples: 39958226. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:37:53,789][122664] Avg episode reward: [(0, '76.980'), (1, '84.350')] [2023-10-10 19:37:53,796][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000078080_79953920.pth... [2023-10-10 19:37:53,830][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000076384_78217216.pth [2023-10-10 19:37:53,869][123614] Updated weights for policy 1, policy_version 77970 (0.0010) [2023-10-10 19:37:54,234][123614] Updated weights for policy 1, policy_version 77980 (0.0010) [2023-10-10 19:37:54,378][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000077984_79855616.pth... [2023-10-10 19:37:54,409][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000076256_78086144.pth [2023-10-10 19:37:55,731][123582] Updated weights for policy 0, policy_version 78083 (0.0008) [2023-10-10 19:37:56,102][123582] Updated weights for policy 0, policy_version 78093 (0.0007) [2023-10-10 19:37:56,479][123582] Updated weights for policy 0, policy_version 78103 (0.0007) [2023-10-10 19:37:57,786][123614] Updated weights for policy 1, policy_version 77990 (0.0010) [2023-10-10 19:37:58,161][123614] Updated weights for policy 1, policy_version 78000 (0.0008) [2023-10-10 19:37:58,532][123614] Updated weights for policy 1, policy_version 78010 (0.0008) [2023-10-10 19:37:58,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 159875072. Throughput: 0: 1819.2, 1: 1795.5. Samples: 39969398. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:37:58,790][122664] Avg episode reward: [(0, '82.660'), (1, '85.300')] [2023-10-10 19:38:00,237][123582] Updated weights for policy 0, policy_version 78113 (0.0007) [2023-10-10 19:38:00,603][123582] Updated weights for policy 0, policy_version 78123 (0.0008) [2023-10-10 19:38:00,976][123582] Updated weights for policy 0, policy_version 78133 (0.0007) [2023-10-10 19:38:01,349][123582] Updated weights for policy 0, policy_version 78143 (0.0009) [2023-10-10 19:38:02,272][123614] Updated weights for policy 1, policy_version 78020 (0.0009) [2023-10-10 19:38:02,644][123614] Updated weights for policy 1, policy_version 78030 (0.0007) [2023-10-10 19:38:03,008][123614] Updated weights for policy 1, policy_version 78040 (0.0010) [2023-10-10 19:38:03,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 159940608. Throughput: 0: 1826.3, 1: 1808.8. Samples: 39991140. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:38:03,788][122664] Avg episode reward: [(0, '79.330'), (1, '85.070')] [2023-10-10 19:38:04,907][123582] Updated weights for policy 0, policy_version 78153 (0.0008) [2023-10-10 19:38:05,284][123582] Updated weights for policy 0, policy_version 78163 (0.0007) [2023-10-10 19:38:05,660][123582] Updated weights for policy 0, policy_version 78173 (0.0007) [2023-10-10 19:38:06,535][123614] Updated weights for policy 1, policy_version 78050 (0.0007) [2023-10-10 19:38:06,910][123614] Updated weights for policy 1, policy_version 78060 (0.0008) [2023-10-10 19:38:07,275][123614] Updated weights for policy 1, policy_version 78070 (0.0008) [2023-10-10 19:38:07,634][123614] Updated weights for policy 1, policy_version 78080 (0.0007) [2023-10-10 19:38:08,788][122664] Fps is (10 sec: 13107.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 160006144. Throughput: 0: 1833.3, 1: 1805.7. Samples: 40013674. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:38:08,788][122664] Avg episode reward: [(0, '82.830'), (1, '87.800')] [2023-10-10 19:38:09,233][123582] Updated weights for policy 0, policy_version 78183 (0.0009) [2023-10-10 19:38:09,601][123582] Updated weights for policy 0, policy_version 78193 (0.0009) [2023-10-10 19:38:09,969][123582] Updated weights for policy 0, policy_version 78203 (0.0008) [2023-10-10 19:38:11,273][123614] Updated weights for policy 1, policy_version 78090 (0.0008) [2023-10-10 19:38:11,641][123614] Updated weights for policy 1, policy_version 78100 (0.0007) [2023-10-10 19:38:12,004][123614] Updated weights for policy 1, policy_version 78110 (0.0009) [2023-10-10 19:38:13,657][123582] Updated weights for policy 0, policy_version 78213 (0.0009) [2023-10-10 19:38:13,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 160071680. Throughput: 0: 1837.0, 1: 1807.3. Samples: 40024128. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 19:38:13,788][122664] Avg episode reward: [(0, '80.500'), (1, '80.570')] [2023-10-10 19:38:14,035][123582] Updated weights for policy 0, policy_version 78223 (0.0007) [2023-10-10 19:38:14,398][123582] Updated weights for policy 0, policy_version 78233 (0.0008) [2023-10-10 19:38:15,765][123614] Updated weights for policy 1, policy_version 78120 (0.0011) [2023-10-10 19:38:16,127][123614] Updated weights for policy 1, policy_version 78130 (0.0010) [2023-10-10 19:38:16,497][123614] Updated weights for policy 1, policy_version 78140 (0.0009) [2023-10-10 19:38:17,965][123582] Updated weights for policy 0, policy_version 78243 (0.0008) [2023-10-10 19:38:18,332][123582] Updated weights for policy 0, policy_version 78253 (0.0008) [2023-10-10 19:38:18,700][123582] Updated weights for policy 0, policy_version 78263 (0.0007) [2023-10-10 19:38:18,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 160137216. Throughput: 0: 1842.5, 1: 1805.0. Samples: 40046588. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 19:38:18,788][122664] Avg episode reward: [(0, '78.160'), (1, '79.140')] [2023-10-10 19:38:20,343][123614] Updated weights for policy 1, policy_version 78150 (0.0008) [2023-10-10 19:38:20,720][123614] Updated weights for policy 1, policy_version 78160 (0.0007) [2023-10-10 19:38:21,086][123614] Updated weights for policy 1, policy_version 78170 (0.0008) [2023-10-10 19:38:22,381][123582] Updated weights for policy 0, policy_version 78273 (0.0008) [2023-10-10 19:38:22,758][123582] Updated weights for policy 0, policy_version 78283 (0.0008) [2023-10-10 19:38:23,124][123582] Updated weights for policy 0, policy_version 78293 (0.0007) [2023-10-10 19:38:23,487][123582] Updated weights for policy 0, policy_version 78303 (0.0007) [2023-10-10 19:38:23,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 160235520. Throughput: 0: 1834.7, 1: 1808.9. Samples: 40068160. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 19:38:23,788][122664] Avg episode reward: [(0, '78.210'), (1, '77.750')] [2023-10-10 19:38:24,803][123614] Updated weights for policy 1, policy_version 78180 (0.0008) [2023-10-10 19:38:25,175][123614] Updated weights for policy 1, policy_version 78190 (0.0008) [2023-10-10 19:38:25,548][123614] Updated weights for policy 1, policy_version 78200 (0.0008) [2023-10-10 19:38:26,947][123582] Updated weights for policy 0, policy_version 78313 (0.0007) [2023-10-10 19:38:27,326][123582] Updated weights for policy 0, policy_version 78323 (0.0008) [2023-10-10 19:38:27,692][123582] Updated weights for policy 0, policy_version 78333 (0.0008) [2023-10-10 19:38:28,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 160301056. Throughput: 0: 1846.8, 1: 1807.9. Samples: 40079512. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 19:38:28,789][122664] Avg episode reward: [(0, '74.930'), (1, '68.000')] [2023-10-10 19:38:29,128][123614] Updated weights for policy 1, policy_version 78210 (0.0008) [2023-10-10 19:38:29,494][123614] Updated weights for policy 1, policy_version 78220 (0.0009) [2023-10-10 19:38:29,870][123614] Updated weights for policy 1, policy_version 78230 (0.0008) [2023-10-10 19:38:30,237][123614] Updated weights for policy 1, policy_version 78240 (0.0008) [2023-10-10 19:38:31,345][123582] Updated weights for policy 0, policy_version 78343 (0.0009) [2023-10-10 19:38:31,720][123582] Updated weights for policy 0, policy_version 78353 (0.0007) [2023-10-10 19:38:32,103][123582] Updated weights for policy 0, policy_version 78363 (0.0009) [2023-10-10 19:38:33,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 160366592. Throughput: 0: 1834.0, 1: 1818.8. Samples: 40101300. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 19:38:33,789][122664] Avg episode reward: [(0, '71.700'), (1, '62.150')] [2023-10-10 19:38:33,938][123614] Updated weights for policy 1, policy_version 78250 (0.0007) [2023-10-10 19:38:34,310][123614] Updated weights for policy 1, policy_version 78260 (0.0007) [2023-10-10 19:38:34,670][123614] Updated weights for policy 1, policy_version 78270 (0.0008) [2023-10-10 19:38:35,853][123582] Updated weights for policy 0, policy_version 78373 (0.0009) [2023-10-10 19:38:36,234][123582] Updated weights for policy 0, policy_version 78383 (0.0010) [2023-10-10 19:38:36,609][123582] Updated weights for policy 0, policy_version 78393 (0.0007) [2023-10-10 19:38:38,314][123614] Updated weights for policy 1, policy_version 78280 (0.0008) [2023-10-10 19:38:38,691][123614] Updated weights for policy 1, policy_version 78290 (0.0008) [2023-10-10 19:38:38,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 160432128. Throughput: 0: 1848.9, 1: 1817.3. Samples: 40123208. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 19:38:38,789][122664] Avg episode reward: [(0, '71.820'), (1, '62.510')] [2023-10-10 19:38:39,065][123614] Updated weights for policy 1, policy_version 78300 (0.0007) [2023-10-10 19:38:40,336][123582] Updated weights for policy 0, policy_version 78403 (0.0007) [2023-10-10 19:38:40,717][123582] Updated weights for policy 0, policy_version 78413 (0.0009) [2023-10-10 19:38:41,090][123582] Updated weights for policy 0, policy_version 78423 (0.0010) [2023-10-10 19:38:42,828][123614] Updated weights for policy 1, policy_version 78310 (0.0009) [2023-10-10 19:38:43,194][123614] Updated weights for policy 1, policy_version 78320 (0.0010) [2023-10-10 19:38:43,565][123614] Updated weights for policy 1, policy_version 78330 (0.0008) [2023-10-10 19:38:43,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 160530432. Throughput: 0: 1833.7, 1: 1818.0. Samples: 40133726. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 19:38:43,789][122664] Avg episode reward: [(0, '70.950'), (1, '62.890')] [2023-10-10 19:38:44,826][123582] Updated weights for policy 0, policy_version 78433 (0.0008) [2023-10-10 19:38:45,212][123582] Updated weights for policy 0, policy_version 78443 (0.0009) [2023-10-10 19:38:45,587][123582] Updated weights for policy 0, policy_version 78453 (0.0007) [2023-10-10 19:38:45,963][123582] Updated weights for policy 0, policy_version 78463 (0.0008) [2023-10-10 19:38:47,212][123614] Updated weights for policy 1, policy_version 78340 (0.0008) [2023-10-10 19:38:47,582][123614] Updated weights for policy 1, policy_version 78350 (0.0008) [2023-10-10 19:38:47,943][123614] Updated weights for policy 1, policy_version 78360 (0.0009) [2023-10-10 19:38:48,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 160595968. Throughput: 0: 1842.8, 1: 1818.5. Samples: 40155902. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 19:38:48,789][122664] Avg episode reward: [(0, '70.000'), (1, '64.130')] [2023-10-10 19:38:49,724][123582] Updated weights for policy 0, policy_version 78473 (0.0009) [2023-10-10 19:38:50,100][123582] Updated weights for policy 0, policy_version 78483 (0.0012) [2023-10-10 19:38:50,469][123582] Updated weights for policy 0, policy_version 78493 (0.0008) [2023-10-10 19:38:51,587][123614] Updated weights for policy 1, policy_version 78370 (0.0008) [2023-10-10 19:38:51,959][123614] Updated weights for policy 1, policy_version 78380 (0.0011) [2023-10-10 19:38:52,333][123614] Updated weights for policy 1, policy_version 78390 (0.0008) [2023-10-10 19:38:52,703][123614] Updated weights for policy 1, policy_version 78400 (0.0008) [2023-10-10 19:38:53,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 160661504. Throughput: 0: 1829.5, 1: 1815.0. Samples: 40177676. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 19:38:53,789][122664] Avg episode reward: [(0, '73.400'), (1, '70.980')] [2023-10-10 19:38:54,220][123582] Updated weights for policy 0, policy_version 78503 (0.0009) [2023-10-10 19:38:54,601][123582] Updated weights for policy 0, policy_version 78513 (0.0008) [2023-10-10 19:38:54,967][123582] Updated weights for policy 0, policy_version 78523 (0.0008) [2023-10-10 19:38:56,470][123614] Updated weights for policy 1, policy_version 78410 (0.0008) [2023-10-10 19:38:56,844][123614] Updated weights for policy 1, policy_version 78420 (0.0009) [2023-10-10 19:38:57,213][123614] Updated weights for policy 1, policy_version 78430 (0.0010) [2023-10-10 19:38:58,708][123582] Updated weights for policy 0, policy_version 78533 (0.0008) [2023-10-10 19:38:58,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 160727040. Throughput: 0: 1826.7, 1: 1827.1. Samples: 40188548. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 19:38:58,789][122664] Avg episode reward: [(0, '74.080'), (1, '72.880')] [2023-10-10 19:38:59,088][123582] Updated weights for policy 0, policy_version 78543 (0.0008) [2023-10-10 19:38:59,453][123582] Updated weights for policy 0, policy_version 78553 (0.0008) [2023-10-10 19:39:00,846][123614] Updated weights for policy 1, policy_version 78440 (0.0009) [2023-10-10 19:39:01,212][123614] Updated weights for policy 1, policy_version 78450 (0.0009) [2023-10-10 19:39:01,595][123614] Updated weights for policy 1, policy_version 78460 (0.0009) [2023-10-10 19:39:03,084][123582] Updated weights for policy 0, policy_version 78563 (0.0009) [2023-10-10 19:39:03,456][123582] Updated weights for policy 0, policy_version 78573 (0.0008) [2023-10-10 19:39:03,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 160792576. Throughput: 0: 1818.9, 1: 1825.0. Samples: 40210564. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-10 19:39:03,788][122664] Avg episode reward: [(0, '67.950'), (1, '69.730')] [2023-10-10 19:39:03,821][123582] Updated weights for policy 0, policy_version 78583 (0.0009) [2023-10-10 19:39:05,363][123614] Updated weights for policy 1, policy_version 78470 (0.0009) [2023-10-10 19:39:05,740][123614] Updated weights for policy 1, policy_version 78480 (0.0010) [2023-10-10 19:39:06,105][123614] Updated weights for policy 1, policy_version 78490 (0.0007) [2023-10-10 19:39:07,544][123582] Updated weights for policy 0, policy_version 78593 (0.0009) [2023-10-10 19:39:07,910][123582] Updated weights for policy 0, policy_version 78603 (0.0009) [2023-10-10 19:39:08,284][123582] Updated weights for policy 0, policy_version 78613 (0.0009) [2023-10-10 19:39:08,657][123582] Updated weights for policy 0, policy_version 78623 (0.0008) [2023-10-10 19:39:08,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 160890880. Throughput: 0: 1820.5, 1: 1827.0. Samples: 40232300. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-10 19:39:08,789][122664] Avg episode reward: [(0, '71.640'), (1, '69.570')] [2023-10-10 19:39:09,677][123614] Updated weights for policy 1, policy_version 78500 (0.0008) [2023-10-10 19:39:10,040][123614] Updated weights for policy 1, policy_version 78510 (0.0008) [2023-10-10 19:39:10,411][123614] Updated weights for policy 1, policy_version 78520 (0.0009) [2023-10-10 19:39:12,235][123582] Updated weights for policy 0, policy_version 78633 (0.0010) [2023-10-10 19:39:12,606][123582] Updated weights for policy 0, policy_version 78643 (0.0011) [2023-10-10 19:39:12,981][123582] Updated weights for policy 0, policy_version 78653 (0.0008) [2023-10-10 19:39:13,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 160956416. Throughput: 0: 1813.5, 1: 1827.1. Samples: 40243338. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-10 19:39:13,789][122664] Avg episode reward: [(0, '68.370'), (1, '67.010')] [2023-10-10 19:39:14,212][123614] Updated weights for policy 1, policy_version 78530 (0.0008) [2023-10-10 19:39:14,581][123614] Updated weights for policy 1, policy_version 78540 (0.0009) [2023-10-10 19:39:14,944][123614] Updated weights for policy 1, policy_version 78550 (0.0009) [2023-10-10 19:39:15,312][123614] Updated weights for policy 1, policy_version 78560 (0.0010) [2023-10-10 19:39:16,633][123582] Updated weights for policy 0, policy_version 78663 (0.0009) [2023-10-10 19:39:17,003][123582] Updated weights for policy 0, policy_version 78673 (0.0008) [2023-10-10 19:39:17,376][123582] Updated weights for policy 0, policy_version 78683 (0.0009) [2023-10-10 19:39:18,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 161021952. Throughput: 0: 1822.1, 1: 1816.5. Samples: 40265040. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-10 19:39:18,789][122664] Avg episode reward: [(0, '73.360'), (1, '67.290')] [2023-10-10 19:39:18,905][123614] Updated weights for policy 1, policy_version 78570 (0.0009) [2023-10-10 19:39:19,274][123614] Updated weights for policy 1, policy_version 78580 (0.0008) [2023-10-10 19:39:19,637][123614] Updated weights for policy 1, policy_version 78590 (0.0008) [2023-10-10 19:39:20,905][123582] Updated weights for policy 0, policy_version 78693 (0.0008) [2023-10-10 19:39:21,286][123582] Updated weights for policy 0, policy_version 78703 (0.0009) [2023-10-10 19:39:21,665][123582] Updated weights for policy 0, policy_version 78713 (0.0008) [2023-10-10 19:39:23,451][123614] Updated weights for policy 1, policy_version 78600 (0.0008) [2023-10-10 19:39:23,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 161087488. Throughput: 0: 1817.1, 1: 1818.8. Samples: 40286824. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-10 19:39:23,788][122664] Avg episode reward: [(0, '74.320'), (1, '69.740')] [2023-10-10 19:39:23,831][123614] Updated weights for policy 1, policy_version 78610 (0.0008) [2023-10-10 19:39:24,194][123614] Updated weights for policy 1, policy_version 78620 (0.0009) [2023-10-10 19:39:25,381][123582] Updated weights for policy 0, policy_version 78723 (0.0008) [2023-10-10 19:39:25,751][123582] Updated weights for policy 0, policy_version 78733 (0.0008) [2023-10-10 19:39:26,135][123582] Updated weights for policy 0, policy_version 78743 (0.0009) [2023-10-10 19:39:28,041][123614] Updated weights for policy 1, policy_version 78630 (0.0008) [2023-10-10 19:39:28,406][123614] Updated weights for policy 1, policy_version 78640 (0.0007) [2023-10-10 19:39:28,779][123614] Updated weights for policy 1, policy_version 78650 (0.0009) [2023-10-10 19:39:28,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 161153024. Throughput: 0: 1823.0, 1: 1817.0. Samples: 40297524. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-10 19:39:28,788][122664] Avg episode reward: [(0, '78.210'), (1, '67.990')] [2023-10-10 19:39:29,888][123582] Updated weights for policy 0, policy_version 78753 (0.0009) [2023-10-10 19:39:30,254][123582] Updated weights for policy 0, policy_version 78763 (0.0008) [2023-10-10 19:39:30,630][123582] Updated weights for policy 0, policy_version 78773 (0.0009) [2023-10-10 19:39:30,996][123582] Updated weights for policy 0, policy_version 78783 (0.0007) [2023-10-10 19:39:32,345][123614] Updated weights for policy 1, policy_version 78660 (0.0009) [2023-10-10 19:39:32,713][123614] Updated weights for policy 1, policy_version 78670 (0.0008) [2023-10-10 19:39:33,083][123614] Updated weights for policy 1, policy_version 78680 (0.0008) [2023-10-10 19:39:33,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 161251328. Throughput: 0: 1827.9, 1: 1816.1. Samples: 40319882. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-10 19:39:33,788][122664] Avg episode reward: [(0, '79.960'), (1, '72.650')] [2023-10-10 19:39:34,360][123582] Updated weights for policy 0, policy_version 78793 (0.0008) [2023-10-10 19:39:34,723][123582] Updated weights for policy 0, policy_version 78803 (0.0008) [2023-10-10 19:39:35,093][123582] Updated weights for policy 0, policy_version 78813 (0.0007) [2023-10-10 19:39:36,750][123614] Updated weights for policy 1, policy_version 78690 (0.0007) [2023-10-10 19:39:37,119][123614] Updated weights for policy 1, policy_version 78700 (0.0008) [2023-10-10 19:39:37,484][123614] Updated weights for policy 1, policy_version 78710 (0.0011) [2023-10-10 19:39:37,857][123614] Updated weights for policy 1, policy_version 78720 (0.0009) [2023-10-10 19:39:38,724][123582] Updated weights for policy 0, policy_version 78823 (0.0007) [2023-10-10 19:39:38,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 161316864. Throughput: 0: 1844.3, 1: 1814.1. Samples: 40342306. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-10 19:39:38,789][122664] Avg episode reward: [(0, '80.090'), (1, '74.930')] [2023-10-10 19:39:39,094][123582] Updated weights for policy 0, policy_version 78833 (0.0008) [2023-10-10 19:39:39,462][123582] Updated weights for policy 0, policy_version 78843 (0.0010) [2023-10-10 19:39:41,560][123614] Updated weights for policy 1, policy_version 78730 (0.0009) [2023-10-10 19:39:41,932][123614] Updated weights for policy 1, policy_version 78740 (0.0008) [2023-10-10 19:39:42,297][123614] Updated weights for policy 1, policy_version 78750 (0.0008) [2023-10-10 19:39:43,142][123582] Updated weights for policy 0, policy_version 78853 (0.0009) [2023-10-10 19:39:43,522][123582] Updated weights for policy 0, policy_version 78863 (0.0010) [2023-10-10 19:39:43,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 161382400. Throughput: 0: 1843.5, 1: 1809.9. Samples: 40352952. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-10 19:39:43,789][122664] Avg episode reward: [(0, '79.960'), (1, '74.380')] [2023-10-10 19:39:43,897][123582] Updated weights for policy 0, policy_version 78873 (0.0010) [2023-10-10 19:39:46,136][123614] Updated weights for policy 1, policy_version 78760 (0.0008) [2023-10-10 19:39:46,509][123614] Updated weights for policy 1, policy_version 78770 (0.0008) [2023-10-10 19:39:46,872][123614] Updated weights for policy 1, policy_version 78780 (0.0011) [2023-10-10 19:39:47,606][123582] Updated weights for policy 0, policy_version 78883 (0.0011) [2023-10-10 19:39:47,978][123582] Updated weights for policy 0, policy_version 78893 (0.0010) [2023-10-10 19:39:48,363][123582] Updated weights for policy 0, policy_version 78903 (0.0011) [2023-10-10 19:39:48,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 161480704. Throughput: 0: 1844.9, 1: 1805.8. Samples: 40374844. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 19:39:48,789][122664] Avg episode reward: [(0, '83.830'), (1, '73.690')] [2023-10-10 19:39:50,755][123614] Updated weights for policy 1, policy_version 78790 (0.0010) [2023-10-10 19:39:51,130][123614] Updated weights for policy 1, policy_version 78800 (0.0011) [2023-10-10 19:39:51,494][123614] Updated weights for policy 1, policy_version 78810 (0.0008) [2023-10-10 19:39:52,048][123582] Updated weights for policy 0, policy_version 78913 (0.0008) [2023-10-10 19:39:52,424][123582] Updated weights for policy 0, policy_version 78923 (0.0009) [2023-10-10 19:39:52,791][123582] Updated weights for policy 0, policy_version 78933 (0.0008) [2023-10-10 19:39:53,154][123582] Updated weights for policy 0, policy_version 78943 (0.0010) [2023-10-10 19:39:53,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 161546240. Throughput: 0: 1834.6, 1: 1797.1. Samples: 40395726. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 19:39:53,789][122664] Avg episode reward: [(0, '83.850'), (1, '73.690')] [2023-10-10 19:39:53,800][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000078816_80707584.pth... [2023-10-10 19:39:53,801][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000078944_80838656.pth... [2023-10-10 19:39:53,830][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000077120_78970880.pth [2023-10-10 19:39:53,841][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000077216_79069184.pth [2023-10-10 19:39:55,049][123614] Updated weights for policy 1, policy_version 78820 (0.0007) [2023-10-10 19:39:55,415][123614] Updated weights for policy 1, policy_version 78830 (0.0008) [2023-10-10 19:39:55,793][123614] Updated weights for policy 1, policy_version 78840 (0.0009) [2023-10-10 19:39:56,824][123582] Updated weights for policy 0, policy_version 78953 (0.0009) [2023-10-10 19:39:57,190][123582] Updated weights for policy 0, policy_version 78963 (0.0010) [2023-10-10 19:39:57,559][123582] Updated weights for policy 0, policy_version 78973 (0.0010) [2023-10-10 19:39:58,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 161611776. Throughput: 0: 1844.8, 1: 1797.6. Samples: 40407244. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 19:39:58,789][122664] Avg episode reward: [(0, '83.620'), (1, '71.980')] [2023-10-10 19:39:59,526][123614] Updated weights for policy 1, policy_version 78850 (0.0009) [2023-10-10 19:39:59,891][123614] Updated weights for policy 1, policy_version 78860 (0.0009) [2023-10-10 19:40:00,257][123614] Updated weights for policy 1, policy_version 78870 (0.0008) [2023-10-10 19:40:00,626][123614] Updated weights for policy 1, policy_version 78880 (0.0008) [2023-10-10 19:40:01,323][123582] Updated weights for policy 0, policy_version 78983 (0.0010) [2023-10-10 19:40:01,696][123582] Updated weights for policy 0, policy_version 78993 (0.0009) [2023-10-10 19:40:02,068][123582] Updated weights for policy 0, policy_version 79003 (0.0007) [2023-10-10 19:40:03,788][122664] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 161677312. Throughput: 0: 1835.0, 1: 1802.5. Samples: 40428728. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 19:40:03,788][122664] Avg episode reward: [(0, '83.240'), (1, '72.550')] [2023-10-10 19:40:04,407][123614] Updated weights for policy 1, policy_version 78890 (0.0007) [2023-10-10 19:40:04,780][123614] Updated weights for policy 1, policy_version 78900 (0.0007) [2023-10-10 19:40:05,138][123614] Updated weights for policy 1, policy_version 78910 (0.0009) [2023-10-10 19:40:05,652][123582] Updated weights for policy 0, policy_version 79013 (0.0007) [2023-10-10 19:40:06,033][123582] Updated weights for policy 0, policy_version 79023 (0.0007) [2023-10-10 19:40:06,407][123582] Updated weights for policy 0, policy_version 79033 (0.0008) [2023-10-10 19:40:08,738][123614] Updated weights for policy 1, policy_version 78920 (0.0008) [2023-10-10 19:40:08,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 161742848. Throughput: 0: 1839.1, 1: 1817.3. Samples: 40451364. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 19:40:08,789][122664] Avg episode reward: [(0, '79.780'), (1, '72.470')] [2023-10-10 19:40:09,110][123614] Updated weights for policy 1, policy_version 78930 (0.0011) [2023-10-10 19:40:09,472][123614] Updated weights for policy 1, policy_version 78940 (0.0009) [2023-10-10 19:40:10,239][123582] Updated weights for policy 0, policy_version 79043 (0.0008) [2023-10-10 19:40:10,631][123582] Updated weights for policy 0, policy_version 79053 (0.0007) [2023-10-10 19:40:11,012][123582] Updated weights for policy 0, policy_version 79063 (0.0009) [2023-10-10 19:40:13,155][123614] Updated weights for policy 1, policy_version 78950 (0.0010) [2023-10-10 19:40:13,518][123614] Updated weights for policy 1, policy_version 78960 (0.0011) [2023-10-10 19:40:13,788][122664] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 161808384. Throughput: 0: 1831.5, 1: 1813.5. Samples: 40461552. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 19:40:13,789][122664] Avg episode reward: [(0, '79.640'), (1, '71.820')] [2023-10-10 19:40:13,890][123614] Updated weights for policy 1, policy_version 78970 (0.0009) [2023-10-10 19:40:14,738][123582] Updated weights for policy 0, policy_version 79073 (0.0008) [2023-10-10 19:40:15,110][123582] Updated weights for policy 0, policy_version 79083 (0.0010) [2023-10-10 19:40:15,478][123582] Updated weights for policy 0, policy_version 79093 (0.0009) [2023-10-10 19:40:15,840][123582] Updated weights for policy 0, policy_version 79103 (0.0011) [2023-10-10 19:40:17,424][123614] Updated weights for policy 1, policy_version 78980 (0.0009) [2023-10-10 19:40:17,791][123614] Updated weights for policy 1, policy_version 78990 (0.0007) [2023-10-10 19:40:18,164][123614] Updated weights for policy 1, policy_version 79000 (0.0008) [2023-10-10 19:40:18,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 161906688. Throughput: 0: 1820.2, 1: 1813.6. Samples: 40483402. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 19:40:18,789][122664] Avg episode reward: [(0, '78.730'), (1, '75.480')] [2023-10-10 19:40:19,477][123582] Updated weights for policy 0, policy_version 79113 (0.0011) [2023-10-10 19:40:19,848][123582] Updated weights for policy 0, policy_version 79123 (0.0010) [2023-10-10 19:40:20,219][123582] Updated weights for policy 0, policy_version 79133 (0.0009) [2023-10-10 19:40:21,792][123614] Updated weights for policy 1, policy_version 79010 (0.0008) [2023-10-10 19:40:22,159][123614] Updated weights for policy 1, policy_version 79020 (0.0009) [2023-10-10 19:40:22,525][123614] Updated weights for policy 1, policy_version 79030 (0.0008) [2023-10-10 19:40:22,895][123614] Updated weights for policy 1, policy_version 79040 (0.0009) [2023-10-10 19:40:23,788][122664] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 161972224. Throughput: 0: 1812.4, 1: 1813.2. Samples: 40505458. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 19:40:23,788][122664] Avg episode reward: [(0, '78.490'), (1, '76.200')] [2023-10-10 19:40:23,882][123582] Updated weights for policy 0, policy_version 79143 (0.0008) [2023-10-10 19:40:24,256][123582] Updated weights for policy 0, policy_version 79153 (0.0010) [2023-10-10 19:40:24,625][123582] Updated weights for policy 0, policy_version 79163 (0.0012) [2023-10-10 19:40:26,477][123614] Updated weights for policy 1, policy_version 79050 (0.0009) [2023-10-10 19:40:26,843][123614] Updated weights for policy 1, policy_version 79060 (0.0010) [2023-10-10 19:40:27,215][123614] Updated weights for policy 1, policy_version 79070 (0.0009) [2023-10-10 19:40:28,404][123582] Updated weights for policy 0, policy_version 79173 (0.0008) [2023-10-10 19:40:28,777][123582] Updated weights for policy 0, policy_version 79183 (0.0007) [2023-10-10 19:40:28,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 162037760. Throughput: 0: 1810.3, 1: 1815.5. Samples: 40516112. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 19:40:28,789][122664] Avg episode reward: [(0, '79.640'), (1, '75.730')] [2023-10-10 19:40:29,151][123582] Updated weights for policy 0, policy_version 79193 (0.0008) [2023-10-10 19:40:31,030][123614] Updated weights for policy 1, policy_version 79080 (0.0010) [2023-10-10 19:40:31,396][123614] Updated weights for policy 1, policy_version 79090 (0.0009) [2023-10-10 19:40:31,772][123614] Updated weights for policy 1, policy_version 79100 (0.0009) [2023-10-10 19:40:32,705][123582] Updated weights for policy 0, policy_version 79203 (0.0008) [2023-10-10 19:40:33,070][123582] Updated weights for policy 0, policy_version 79213 (0.0009) [2023-10-10 19:40:33,435][123582] Updated weights for policy 0, policy_version 79223 (0.0008) [2023-10-10 19:40:33,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 162136064. Throughput: 0: 1810.1, 1: 1819.4. Samples: 40538174. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 19:40:33,788][122664] Avg episode reward: [(0, '79.570'), (1, '76.270')] [2023-10-10 19:40:35,708][123614] Updated weights for policy 1, policy_version 79110 (0.0008) [2023-10-10 19:40:36,102][123614] Updated weights for policy 1, policy_version 79120 (0.0010) [2023-10-10 19:40:36,471][123614] Updated weights for policy 1, policy_version 79130 (0.0012) [2023-10-10 19:40:37,118][123582] Updated weights for policy 0, policy_version 79233 (0.0008) [2023-10-10 19:40:37,489][123582] Updated weights for policy 0, policy_version 79243 (0.0008) [2023-10-10 19:40:37,867][123582] Updated weights for policy 0, policy_version 79253 (0.0008) [2023-10-10 19:40:38,244][123582] Updated weights for policy 0, policy_version 79263 (0.0011) [2023-10-10 19:40:38,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 162201600. Throughput: 0: 1807.5, 1: 1821.6. Samples: 40559034. Policy #0 lag: (min: 15.0, avg: 20.7, max: 47.0) [2023-10-10 19:40:38,788][122664] Avg episode reward: [(0, '76.430'), (1, '77.910')] [2023-10-10 19:40:40,187][123614] Updated weights for policy 1, policy_version 79140 (0.0011) [2023-10-10 19:40:40,559][123614] Updated weights for policy 1, policy_version 79150 (0.0008) [2023-10-10 19:40:40,919][123614] Updated weights for policy 1, policy_version 79160 (0.0010) [2023-10-10 19:40:42,148][123582] Updated weights for policy 0, policy_version 79273 (0.0010) [2023-10-10 19:40:42,528][123582] Updated weights for policy 0, policy_version 79283 (0.0008) [2023-10-10 19:40:42,899][123582] Updated weights for policy 0, policy_version 79293 (0.0011) [2023-10-10 19:40:43,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 162267136. Throughput: 0: 1800.7, 1: 1822.6. Samples: 40570292. Policy #0 lag: (min: 15.0, avg: 20.7, max: 47.0) [2023-10-10 19:40:43,789][122664] Avg episode reward: [(0, '74.530'), (1, '73.990')] [2023-10-10 19:40:44,635][123614] Updated weights for policy 1, policy_version 79170 (0.0010) [2023-10-10 19:40:45,009][123614] Updated weights for policy 1, policy_version 79180 (0.0009) [2023-10-10 19:40:45,385][123614] Updated weights for policy 1, policy_version 79190 (0.0008) [2023-10-10 19:40:45,751][123614] Updated weights for policy 1, policy_version 79200 (0.0009) [2023-10-10 19:40:46,666][123582] Updated weights for policy 0, policy_version 79303 (0.0008) [2023-10-10 19:40:47,047][123582] Updated weights for policy 0, policy_version 79313 (0.0008) [2023-10-10 19:40:47,425][123582] Updated weights for policy 0, policy_version 79323 (0.0010) [2023-10-10 19:40:48,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 162332672. Throughput: 0: 1798.9, 1: 1820.1. Samples: 40591582. Policy #0 lag: (min: 15.0, avg: 20.7, max: 47.0) [2023-10-10 19:40:48,789][122664] Avg episode reward: [(0, '78.190'), (1, '78.530')] [2023-10-10 19:40:49,442][123614] Updated weights for policy 1, policy_version 79210 (0.0008) [2023-10-10 19:40:49,813][123614] Updated weights for policy 1, policy_version 79220 (0.0008) [2023-10-10 19:40:50,181][123614] Updated weights for policy 1, policy_version 79230 (0.0009) [2023-10-10 19:40:51,353][123582] Updated weights for policy 0, policy_version 79333 (0.0010) [2023-10-10 19:40:51,725][123582] Updated weights for policy 0, policy_version 79343 (0.0007) [2023-10-10 19:40:52,090][123582] Updated weights for policy 0, policy_version 79353 (0.0007) [2023-10-10 19:40:53,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 162398208. Throughput: 0: 1788.4, 1: 1816.4. Samples: 40613578. Policy #0 lag: (min: 15.0, avg: 20.7, max: 47.0) [2023-10-10 19:40:53,788][122664] Avg episode reward: [(0, '79.560'), (1, '76.020')] [2023-10-10 19:40:53,862][123614] Updated weights for policy 1, policy_version 79240 (0.0008) [2023-10-10 19:40:54,229][123614] Updated weights for policy 1, policy_version 79250 (0.0010) [2023-10-10 19:40:54,600][123614] Updated weights for policy 1, policy_version 79260 (0.0010) [2023-10-10 19:40:55,734][123582] Updated weights for policy 0, policy_version 79363 (0.0009) [2023-10-10 19:40:56,137][123582] Updated weights for policy 0, policy_version 79373 (0.0010) [2023-10-10 19:40:56,498][123582] Updated weights for policy 0, policy_version 79383 (0.0009) [2023-10-10 19:40:58,174][123614] Updated weights for policy 1, policy_version 79270 (0.0009) [2023-10-10 19:40:58,544][123614] Updated weights for policy 1, policy_version 79280 (0.0008) [2023-10-10 19:40:58,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 162463744. Throughput: 0: 1800.2, 1: 1811.3. Samples: 40624072. Policy #0 lag: (min: 15.0, avg: 20.7, max: 47.0) [2023-10-10 19:40:58,789][122664] Avg episode reward: [(0, '80.220'), (1, '77.830')] [2023-10-10 19:40:58,909][123614] Updated weights for policy 1, policy_version 79290 (0.0010) [2023-10-10 19:41:00,266][123582] Updated weights for policy 0, policy_version 79393 (0.0009) [2023-10-10 19:41:00,630][123582] Updated weights for policy 0, policy_version 79403 (0.0010) [2023-10-10 19:41:01,004][123582] Updated weights for policy 0, policy_version 79413 (0.0009) [2023-10-10 19:41:01,372][123582] Updated weights for policy 0, policy_version 79423 (0.0008) [2023-10-10 19:41:02,430][123614] Updated weights for policy 1, policy_version 79300 (0.0010) [2023-10-10 19:41:02,806][123614] Updated weights for policy 1, policy_version 79310 (0.0009) [2023-10-10 19:41:03,174][123614] Updated weights for policy 1, policy_version 79320 (0.0009) [2023-10-10 19:41:03,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 162562048. Throughput: 0: 1795.8, 1: 1818.0. Samples: 40646024. Policy #0 lag: (min: 15.0, avg: 20.7, max: 47.0) [2023-10-10 19:41:03,789][122664] Avg episode reward: [(0, '84.060'), (1, '79.930')] [2023-10-10 19:41:05,002][123582] Updated weights for policy 0, policy_version 79433 (0.0011) [2023-10-10 19:41:05,371][123582] Updated weights for policy 0, policy_version 79443 (0.0009) [2023-10-10 19:41:05,748][123582] Updated weights for policy 0, policy_version 79453 (0.0007) [2023-10-10 19:41:06,889][123614] Updated weights for policy 1, policy_version 79330 (0.0007) [2023-10-10 19:41:07,262][123614] Updated weights for policy 1, policy_version 79340 (0.0009) [2023-10-10 19:41:07,645][123614] Updated weights for policy 1, policy_version 79350 (0.0008) [2023-10-10 19:41:08,005][123614] Updated weights for policy 1, policy_version 79360 (0.0008) [2023-10-10 19:41:08,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 162627584. Throughput: 0: 1793.1, 1: 1815.4. Samples: 40667842. Policy #0 lag: (min: 15.0, avg: 20.7, max: 47.0) [2023-10-10 19:41:08,789][122664] Avg episode reward: [(0, '84.150'), (1, '79.580')] [2023-10-10 19:41:09,480][123582] Updated weights for policy 0, policy_version 79463 (0.0008) [2023-10-10 19:41:09,847][123582] Updated weights for policy 0, policy_version 79473 (0.0009) [2023-10-10 19:41:10,228][123582] Updated weights for policy 0, policy_version 79483 (0.0007) [2023-10-10 19:41:11,693][123614] Updated weights for policy 1, policy_version 79370 (0.0007) [2023-10-10 19:41:12,061][123614] Updated weights for policy 1, policy_version 79380 (0.0008) [2023-10-10 19:41:12,429][123614] Updated weights for policy 1, policy_version 79390 (0.0008) [2023-10-10 19:41:13,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 162693120. Throughput: 0: 1795.7, 1: 1820.0. Samples: 40678818. Policy #0 lag: (min: 15.0, avg: 20.7, max: 47.0) [2023-10-10 19:41:13,789][122664] Avg episode reward: [(0, '84.730'), (1, '79.550')] [2023-10-10 19:41:13,894][123582] Updated weights for policy 0, policy_version 79493 (0.0009) [2023-10-10 19:41:14,252][123582] Updated weights for policy 0, policy_version 79503 (0.0009) [2023-10-10 19:41:14,620][123582] Updated weights for policy 0, policy_version 79513 (0.0011) [2023-10-10 19:41:15,929][123614] Updated weights for policy 1, policy_version 79400 (0.0008) [2023-10-10 19:41:16,291][123614] Updated weights for policy 1, policy_version 79410 (0.0007) [2023-10-10 19:41:16,652][123614] Updated weights for policy 1, policy_version 79420 (0.0011) [2023-10-10 19:41:18,386][123582] Updated weights for policy 0, policy_version 79523 (0.0009) [2023-10-10 19:41:18,758][123582] Updated weights for policy 0, policy_version 79533 (0.0008) [2023-10-10 19:41:18,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 162758656. Throughput: 0: 1794.3, 1: 1814.4. Samples: 40700568. Policy #0 lag: (min: 15.0, avg: 20.7, max: 47.0) [2023-10-10 19:41:18,789][122664] Avg episode reward: [(0, '86.300'), (1, '73.180')] [2023-10-10 19:41:19,141][123582] Updated weights for policy 0, policy_version 79543 (0.0007) [2023-10-10 19:41:20,470][123614] Updated weights for policy 1, policy_version 79430 (0.0008) [2023-10-10 19:41:20,856][123614] Updated weights for policy 1, policy_version 79440 (0.0008) [2023-10-10 19:41:21,216][123614] Updated weights for policy 1, policy_version 79450 (0.0008) [2023-10-10 19:41:22,745][123582] Updated weights for policy 0, policy_version 79553 (0.0008) [2023-10-10 19:41:23,122][123582] Updated weights for policy 0, policy_version 79563 (0.0009) [2023-10-10 19:41:23,494][123582] Updated weights for policy 0, policy_version 79573 (0.0008) [2023-10-10 19:41:23,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 162824192. Throughput: 0: 1813.6, 1: 1820.4. Samples: 40722564. Policy #0 lag: (min: 15.0, avg: 20.7, max: 47.0) [2023-10-10 19:41:23,788][122664] Avg episode reward: [(0, '82.430'), (1, '75.140')] [2023-10-10 19:41:23,877][123582] Updated weights for policy 0, policy_version 79583 (0.0009) [2023-10-10 19:41:24,955][123614] Updated weights for policy 1, policy_version 79460 (0.0009) [2023-10-10 19:41:25,322][123614] Updated weights for policy 1, policy_version 79470 (0.0010) [2023-10-10 19:41:25,688][123614] Updated weights for policy 1, policy_version 79480 (0.0010) [2023-10-10 19:41:27,666][123582] Updated weights for policy 0, policy_version 79593 (0.0008) [2023-10-10 19:41:28,036][123582] Updated weights for policy 0, policy_version 79603 (0.0009) [2023-10-10 19:41:28,405][123582] Updated weights for policy 0, policy_version 79613 (0.0010) [2023-10-10 19:41:28,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 162922496. Throughput: 0: 1802.5, 1: 1821.4. Samples: 40733368. Policy #0 lag: (min: 31.0, avg: 43.9, max: 63.0) [2023-10-10 19:41:28,789][122664] Avg episode reward: [(0, '89.620'), (1, '73.430')] [2023-10-10 19:41:29,452][123614] Updated weights for policy 1, policy_version 79490 (0.0010) [2023-10-10 19:41:29,816][123614] Updated weights for policy 1, policy_version 79500 (0.0008) [2023-10-10 19:41:30,187][123614] Updated weights for policy 1, policy_version 79510 (0.0008) [2023-10-10 19:41:30,548][123614] Updated weights for policy 1, policy_version 79520 (0.0008) [2023-10-10 19:41:32,149][123582] Updated weights for policy 0, policy_version 79623 (0.0010) [2023-10-10 19:41:32,520][123582] Updated weights for policy 0, policy_version 79633 (0.0011) [2023-10-10 19:41:32,887][123582] Updated weights for policy 0, policy_version 79643 (0.0010) [2023-10-10 19:41:33,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 162988032. Throughput: 0: 1814.5, 1: 1819.2. Samples: 40755098. Policy #0 lag: (min: 31.0, avg: 43.9, max: 63.0) [2023-10-10 19:41:33,788][122664] Avg episode reward: [(0, '89.780'), (1, '77.270')] [2023-10-10 19:41:34,216][123614] Updated weights for policy 1, policy_version 79530 (0.0009) [2023-10-10 19:41:34,580][123614] Updated weights for policy 1, policy_version 79540 (0.0008) [2023-10-10 19:41:34,950][123614] Updated weights for policy 1, policy_version 79550 (0.0008) [2023-10-10 19:41:36,605][123582] Updated weights for policy 0, policy_version 79653 (0.0009) [2023-10-10 19:41:36,974][123582] Updated weights for policy 0, policy_version 79663 (0.0009) [2023-10-10 19:41:37,351][123582] Updated weights for policy 0, policy_version 79673 (0.0007) [2023-10-10 19:41:38,765][123614] Updated weights for policy 1, policy_version 79560 (0.0008) [2023-10-10 19:41:38,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 163053568. Throughput: 0: 1806.6, 1: 1814.2. Samples: 40776516. Policy #0 lag: (min: 31.0, avg: 43.9, max: 63.0) [2023-10-10 19:41:38,788][122664] Avg episode reward: [(0, '95.880'), (1, '79.880')] [2023-10-10 19:41:39,123][123614] Updated weights for policy 1, policy_version 79570 (0.0009) [2023-10-10 19:41:39,491][123614] Updated weights for policy 1, policy_version 79580 (0.0010) [2023-10-10 19:41:41,225][123582] Updated weights for policy 0, policy_version 79683 (0.0008) [2023-10-10 19:41:41,605][123582] Updated weights for policy 0, policy_version 79693 (0.0008) [2023-10-10 19:41:41,973][123582] Updated weights for policy 0, policy_version 79703 (0.0008) [2023-10-10 19:41:43,087][123614] Updated weights for policy 1, policy_version 79590 (0.0009) [2023-10-10 19:41:43,450][123614] Updated weights for policy 1, policy_version 79600 (0.0007) [2023-10-10 19:41:43,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 163119104. Throughput: 0: 1820.5, 1: 1819.6. Samples: 40787878. Policy #0 lag: (min: 31.0, avg: 43.9, max: 63.0) [2023-10-10 19:41:43,788][122664] Avg episode reward: [(0, '93.330'), (1, '78.280')] [2023-10-10 19:41:43,822][123614] Updated weights for policy 1, policy_version 79610 (0.0007) [2023-10-10 19:41:45,632][123582] Updated weights for policy 0, policy_version 79713 (0.0008) [2023-10-10 19:41:45,995][123582] Updated weights for policy 0, policy_version 79723 (0.0008) [2023-10-10 19:41:46,370][123582] Updated weights for policy 0, policy_version 79733 (0.0007) [2023-10-10 19:41:46,751][123582] Updated weights for policy 0, policy_version 79743 (0.0009) [2023-10-10 19:41:47,620][123614] Updated weights for policy 1, policy_version 79620 (0.0008) [2023-10-10 19:41:47,983][123614] Updated weights for policy 1, policy_version 79630 (0.0008) [2023-10-10 19:41:48,352][123614] Updated weights for policy 1, policy_version 79640 (0.0008) [2023-10-10 19:41:48,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 163217408. Throughput: 0: 1806.6, 1: 1823.6. Samples: 40809384. Policy #0 lag: (min: 31.0, avg: 43.9, max: 63.0) [2023-10-10 19:41:48,789][122664] Avg episode reward: [(0, '94.910'), (1, '73.320')] [2023-10-10 19:41:50,461][123582] Updated weights for policy 0, policy_version 79753 (0.0009) [2023-10-10 19:41:50,840][123582] Updated weights for policy 0, policy_version 79763 (0.0008) [2023-10-10 19:41:51,204][123582] Updated weights for policy 0, policy_version 79773 (0.0008) [2023-10-10 19:41:52,000][123614] Updated weights for policy 1, policy_version 79650 (0.0008) [2023-10-10 19:41:52,353][123614] Updated weights for policy 1, policy_version 79660 (0.0009) [2023-10-10 19:41:52,735][123614] Updated weights for policy 1, policy_version 79670 (0.0008) [2023-10-10 19:41:53,104][123614] Updated weights for policy 1, policy_version 79680 (0.0007) [2023-10-10 19:41:53,788][122664] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 163282944. Throughput: 0: 1809.3, 1: 1820.4. Samples: 40831178. Policy #0 lag: (min: 31.0, avg: 43.9, max: 63.0) [2023-10-10 19:41:53,789][122664] Avg episode reward: [(0, '96.660'), (1, '73.180')] [2023-10-10 19:41:53,801][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000079776_81690624.pth... [2023-10-10 19:41:53,801][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000079680_81592320.pth... [2023-10-10 19:41:53,830][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000078080_79953920.pth [2023-10-10 19:41:53,837][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000077984_79855616.pth [2023-10-10 19:41:54,847][123582] Updated weights for policy 0, policy_version 79783 (0.0009) [2023-10-10 19:41:55,218][123582] Updated weights for policy 0, policy_version 79793 (0.0010) [2023-10-10 19:41:55,589][123582] Updated weights for policy 0, policy_version 79803 (0.0010) [2023-10-10 19:41:56,818][123614] Updated weights for policy 1, policy_version 79690 (0.0007) [2023-10-10 19:41:57,194][123614] Updated weights for policy 1, policy_version 79700 (0.0008) [2023-10-10 19:41:57,559][123614] Updated weights for policy 1, policy_version 79710 (0.0009) [2023-10-10 19:41:58,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 163348480. Throughput: 0: 1804.7, 1: 1823.1. Samples: 40842066. Policy #0 lag: (min: 31.0, avg: 43.9, max: 63.0) [2023-10-10 19:41:58,789][122664] Avg episode reward: [(0, '94.340'), (1, '67.520')] [2023-10-10 19:41:59,366][123582] Updated weights for policy 0, policy_version 79813 (0.0009) [2023-10-10 19:41:59,736][123582] Updated weights for policy 0, policy_version 79823 (0.0009) [2023-10-10 19:42:00,111][123582] Updated weights for policy 0, policy_version 79833 (0.0008) [2023-10-10 19:42:01,096][123614] Updated weights for policy 1, policy_version 79720 (0.0007) [2023-10-10 19:42:01,463][123614] Updated weights for policy 1, policy_version 79730 (0.0008) [2023-10-10 19:42:01,834][123614] Updated weights for policy 1, policy_version 79740 (0.0009) [2023-10-10 19:42:03,556][123582] Updated weights for policy 0, policy_version 79843 (0.0008) [2023-10-10 19:42:03,788][122664] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 163414016. Throughput: 0: 1808.9, 1: 1824.5. Samples: 40864068. Policy #0 lag: (min: 31.0, avg: 43.9, max: 63.0) [2023-10-10 19:42:03,788][122664] Avg episode reward: [(0, '92.080'), (1, '68.630')] [2023-10-10 19:42:03,922][123582] Updated weights for policy 0, policy_version 79853 (0.0010) [2023-10-10 19:42:04,287][123582] Updated weights for policy 0, policy_version 79863 (0.0009) [2023-10-10 19:42:05,664][123614] Updated weights for policy 1, policy_version 79750 (0.0009) [2023-10-10 19:42:06,049][123614] Updated weights for policy 1, policy_version 79760 (0.0007) [2023-10-10 19:42:06,416][123614] Updated weights for policy 1, policy_version 79770 (0.0008) [2023-10-10 19:42:08,118][123582] Updated weights for policy 0, policy_version 79873 (0.0010) [2023-10-10 19:42:08,493][123582] Updated weights for policy 0, policy_version 79883 (0.0008) [2023-10-10 19:42:08,788][122664] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 163479552. Throughput: 0: 1817.2, 1: 1820.3. Samples: 40886252. Policy #0 lag: (min: 31.0, avg: 43.9, max: 63.0) [2023-10-10 19:42:08,788][122664] Avg episode reward: [(0, '87.150'), (1, '65.550')] [2023-10-10 19:42:08,856][123582] Updated weights for policy 0, policy_version 79893 (0.0010) [2023-10-10 19:42:09,236][123582] Updated weights for policy 0, policy_version 79903 (0.0007) [2023-10-10 19:42:10,126][123614] Updated weights for policy 1, policy_version 79780 (0.0008) [2023-10-10 19:42:10,489][123614] Updated weights for policy 1, policy_version 79790 (0.0009) [2023-10-10 19:42:10,854][123614] Updated weights for policy 1, policy_version 79800 (0.0010) [2023-10-10 19:42:12,821][123582] Updated weights for policy 0, policy_version 79913 (0.0009) [2023-10-10 19:42:13,198][123582] Updated weights for policy 0, policy_version 79923 (0.0010) [2023-10-10 19:42:13,568][123582] Updated weights for policy 0, policy_version 79933 (0.0009) [2023-10-10 19:42:13,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 163577856. Throughput: 0: 1804.4, 1: 1819.5. Samples: 40896446. Policy #0 lag: (min: 31.0, avg: 43.9, max: 63.0) [2023-10-10 19:42:13,789][122664] Avg episode reward: [(0, '82.010'), (1, '67.810')] [2023-10-10 19:42:14,534][123614] Updated weights for policy 1, policy_version 79810 (0.0009) [2023-10-10 19:42:14,903][123614] Updated weights for policy 1, policy_version 79820 (0.0011) [2023-10-10 19:42:15,288][123614] Updated weights for policy 1, policy_version 79830 (0.0011) [2023-10-10 19:42:15,648][123614] Updated weights for policy 1, policy_version 79840 (0.0011) [2023-10-10 19:42:17,132][123582] Updated weights for policy 0, policy_version 79943 (0.0011) [2023-10-10 19:42:17,507][123582] Updated weights for policy 0, policy_version 79953 (0.0008) [2023-10-10 19:42:17,878][123582] Updated weights for policy 0, policy_version 79963 (0.0009) [2023-10-10 19:42:18,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 163643392. Throughput: 0: 1813.2, 1: 1820.3. Samples: 40918608. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 19:42:18,788][122664] Avg episode reward: [(0, '72.430'), (1, '69.990')] [2023-10-10 19:42:19,308][123614] Updated weights for policy 1, policy_version 79850 (0.0009) [2023-10-10 19:42:19,690][123614] Updated weights for policy 1, policy_version 79860 (0.0008) [2023-10-10 19:42:20,059][123614] Updated weights for policy 1, policy_version 79870 (0.0007) [2023-10-10 19:42:21,760][123582] Updated weights for policy 0, policy_version 79973 (0.0007) [2023-10-10 19:42:22,135][123582] Updated weights for policy 0, policy_version 79983 (0.0008) [2023-10-10 19:42:22,513][123582] Updated weights for policy 0, policy_version 79993 (0.0007) [2023-10-10 19:42:23,751][123614] Updated weights for policy 1, policy_version 79880 (0.0007) [2023-10-10 19:42:23,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 163708928. Throughput: 0: 1813.4, 1: 1826.0. Samples: 40940290. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 19:42:23,788][122664] Avg episode reward: [(0, '70.660'), (1, '70.310')] [2023-10-10 19:42:24,123][123614] Updated weights for policy 1, policy_version 79890 (0.0007) [2023-10-10 19:42:24,482][123614] Updated weights for policy 1, policy_version 79900 (0.0009) [2023-10-10 19:42:26,255][123582] Updated weights for policy 0, policy_version 80003 (0.0010) [2023-10-10 19:42:26,639][123582] Updated weights for policy 0, policy_version 80013 (0.0008) [2023-10-10 19:42:27,011][123582] Updated weights for policy 0, policy_version 80023 (0.0008) [2023-10-10 19:42:28,262][123614] Updated weights for policy 1, policy_version 79910 (0.0008) [2023-10-10 19:42:28,627][123614] Updated weights for policy 1, policy_version 79920 (0.0009) [2023-10-10 19:42:28,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 163774464. Throughput: 0: 1816.4, 1: 1819.5. Samples: 40951494. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 19:42:28,789][122664] Avg episode reward: [(0, '71.060'), (1, '72.790')] [2023-10-10 19:42:28,988][123614] Updated weights for policy 1, policy_version 79930 (0.0008) [2023-10-10 19:42:30,773][123582] Updated weights for policy 0, policy_version 80033 (0.0008) [2023-10-10 19:42:31,141][123582] Updated weights for policy 0, policy_version 80043 (0.0011) [2023-10-10 19:42:31,509][123582] Updated weights for policy 0, policy_version 80053 (0.0010) [2023-10-10 19:42:31,889][123582] Updated weights for policy 0, policy_version 80063 (0.0009) [2023-10-10 19:42:32,887][123614] Updated weights for policy 1, policy_version 79940 (0.0007) [2023-10-10 19:42:33,265][123614] Updated weights for policy 1, policy_version 79950 (0.0007) [2023-10-10 19:42:33,625][123614] Updated weights for policy 1, policy_version 79960 (0.0008) [2023-10-10 19:42:33,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 163840000. Throughput: 0: 1811.4, 1: 1822.2. Samples: 40972896. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 19:42:33,789][122664] Avg episode reward: [(0, '69.460'), (1, '71.250')] [2023-10-10 19:42:35,433][123582] Updated weights for policy 0, policy_version 80073 (0.0008) [2023-10-10 19:42:35,798][123582] Updated weights for policy 0, policy_version 80083 (0.0007) [2023-10-10 19:42:36,168][123582] Updated weights for policy 0, policy_version 80093 (0.0007) [2023-10-10 19:42:37,421][123614] Updated weights for policy 1, policy_version 79970 (0.0008) [2023-10-10 19:42:37,780][123614] Updated weights for policy 1, policy_version 79980 (0.0007) [2023-10-10 19:42:38,149][123614] Updated weights for policy 1, policy_version 79990 (0.0009) [2023-10-10 19:42:38,522][123614] Updated weights for policy 1, policy_version 80000 (0.0008) [2023-10-10 19:42:38,788][122664] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 163938304. Throughput: 0: 1818.8, 1: 1810.2. Samples: 40994486. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 19:42:38,789][122664] Avg episode reward: [(0, '66.740'), (1, '70.070')] [2023-10-10 19:42:39,756][123582] Updated weights for policy 0, policy_version 80103 (0.0009) [2023-10-10 19:42:40,131][123582] Updated weights for policy 0, policy_version 80113 (0.0010) [2023-10-10 19:42:40,500][123582] Updated weights for policy 0, policy_version 80123 (0.0008) [2023-10-10 19:42:42,177][123614] Updated weights for policy 1, policy_version 80010 (0.0008) [2023-10-10 19:42:42,541][123614] Updated weights for policy 1, policy_version 80020 (0.0008) [2023-10-10 19:42:42,915][123614] Updated weights for policy 1, policy_version 80030 (0.0008) [2023-10-10 19:42:43,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 164003840. Throughput: 0: 1824.5, 1: 1821.1. Samples: 41006118. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 19:42:43,789][122664] Avg episode reward: [(0, '66.810'), (1, '70.020')] [2023-10-10 19:42:44,094][123582] Updated weights for policy 0, policy_version 80133 (0.0009) [2023-10-10 19:42:44,461][123582] Updated weights for policy 0, policy_version 80143 (0.0009) [2023-10-10 19:42:44,832][123582] Updated weights for policy 0, policy_version 80153 (0.0008) [2023-10-10 19:42:46,477][123614] Updated weights for policy 1, policy_version 80040 (0.0009) [2023-10-10 19:42:46,852][123614] Updated weights for policy 1, policy_version 80050 (0.0009) [2023-10-10 19:42:47,223][123614] Updated weights for policy 1, policy_version 80060 (0.0009) [2023-10-10 19:42:48,576][123582] Updated weights for policy 0, policy_version 80163 (0.0009) [2023-10-10 19:42:48,788][122664] Fps is (10 sec: 13107.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 164069376. Throughput: 0: 1825.4, 1: 1811.8. Samples: 41027742. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 19:42:48,788][122664] Avg episode reward: [(0, '68.760'), (1, '66.760')] [2023-10-10 19:42:48,960][123582] Updated weights for policy 0, policy_version 80173 (0.0008) [2023-10-10 19:42:49,325][123582] Updated weights for policy 0, policy_version 80183 (0.0007) [2023-10-10 19:42:50,886][123614] Updated weights for policy 1, policy_version 80070 (0.0007) [2023-10-10 19:42:51,266][123614] Updated weights for policy 1, policy_version 80080 (0.0008) [2023-10-10 19:42:51,631][123614] Updated weights for policy 1, policy_version 80090 (0.0008) [2023-10-10 19:42:52,919][123582] Updated weights for policy 0, policy_version 80193 (0.0008) [2023-10-10 19:42:53,291][123582] Updated weights for policy 0, policy_version 80203 (0.0008) [2023-10-10 19:42:53,667][123582] Updated weights for policy 0, policy_version 80213 (0.0008) [2023-10-10 19:42:53,788][122664] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 164134912. Throughput: 0: 1823.5, 1: 1811.8. Samples: 41049840. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 19:42:53,790][122664] Avg episode reward: [(0, '69.000'), (1, '68.100')] [2023-10-10 19:42:54,042][123582] Updated weights for policy 0, policy_version 80223 (0.0007) [2023-10-10 19:42:55,374][123614] Updated weights for policy 1, policy_version 80100 (0.0008) [2023-10-10 19:42:55,744][123614] Updated weights for policy 1, policy_version 80110 (0.0009) [2023-10-10 19:42:56,122][123614] Updated weights for policy 1, policy_version 80120 (0.0007) [2023-10-10 19:42:57,746][123582] Updated weights for policy 0, policy_version 80233 (0.0009) [2023-10-10 19:42:58,104][123582] Updated weights for policy 0, policy_version 80243 (0.0008) [2023-10-10 19:42:58,476][123582] Updated weights for policy 0, policy_version 80253 (0.0008) [2023-10-10 19:42:58,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 164233216. Throughput: 0: 1831.6, 1: 1814.6. Samples: 41060522. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 19:42:58,789][122664] Avg episode reward: [(0, '71.330'), (1, '67.940')] [2023-10-10 19:42:59,923][123614] Updated weights for policy 1, policy_version 80130 (0.0010) [2023-10-10 19:43:00,293][123614] Updated weights for policy 1, policy_version 80140 (0.0010) [2023-10-10 19:43:00,660][123614] Updated weights for policy 1, policy_version 80150 (0.0009) [2023-10-10 19:43:01,026][123614] Updated weights for policy 1, policy_version 80160 (0.0010) [2023-10-10 19:43:02,163][123582] Updated weights for policy 0, policy_version 80263 (0.0009) [2023-10-10 19:43:02,535][123582] Updated weights for policy 0, policy_version 80273 (0.0011) [2023-10-10 19:43:02,905][123582] Updated weights for policy 0, policy_version 80283 (0.0009) [2023-10-10 19:43:03,788][122664] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 164298752. Throughput: 0: 1832.1, 1: 1810.2. Samples: 41082512. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 19:43:03,789][122664] Avg episode reward: [(0, '66.240'), (1, '68.020')] [2023-10-10 19:43:04,772][123614] Updated weights for policy 1, policy_version 80170 (0.0011) [2023-10-10 19:43:05,141][123614] Updated weights for policy 1, policy_version 80180 (0.0009) [2023-10-10 19:43:05,512][123614] Updated weights for policy 1, policy_version 80190 (0.0007) [2023-10-10 19:43:06,481][123582] Updated weights for policy 0, policy_version 80293 (0.0008) [2023-10-10 19:43:06,851][123582] Updated weights for policy 0, policy_version 80303 (0.0007) [2023-10-10 19:43:07,226][123582] Updated weights for policy 0, policy_version 80313 (0.0010) [2023-10-10 19:43:08,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 164364288. Throughput: 0: 1832.4, 1: 1815.1. Samples: 41104426. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 19:43:08,789][122664] Avg episode reward: [(0, '67.580'), (1, '67.850')] [2023-10-10 19:43:09,131][123614] Updated weights for policy 1, policy_version 80200 (0.0007) [2023-10-10 19:43:09,505][123614] Updated weights for policy 1, policy_version 80210 (0.0008) [2023-10-10 19:43:09,871][123614] Updated weights for policy 1, policy_version 80220 (0.0010) [2023-10-10 19:43:10,926][123582] Updated weights for policy 0, policy_version 80323 (0.0009) [2023-10-10 19:43:11,311][123582] Updated weights for policy 0, policy_version 80333 (0.0010) [2023-10-10 19:43:11,687][123582] Updated weights for policy 0, policy_version 80343 (0.0008) [2023-10-10 19:43:13,501][123614] Updated weights for policy 1, policy_version 80230 (0.0008) [2023-10-10 19:43:13,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 164429824. Throughput: 0: 1824.6, 1: 1813.3. Samples: 41115196. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-10 19:43:13,788][122664] Avg episode reward: [(0, '75.170'), (1, '66.380')] [2023-10-10 19:43:13,874][123614] Updated weights for policy 1, policy_version 80240 (0.0012) [2023-10-10 19:43:14,235][123614] Updated weights for policy 1, policy_version 80250 (0.0010) [2023-10-10 19:43:15,411][123582] Updated weights for policy 0, policy_version 80353 (0.0007) [2023-10-10 19:43:15,782][123582] Updated weights for policy 0, policy_version 80363 (0.0010) [2023-10-10 19:43:16,155][123582] Updated weights for policy 0, policy_version 80373 (0.0008) [2023-10-10 19:43:16,531][123582] Updated weights for policy 0, policy_version 80383 (0.0009) [2023-10-10 19:43:17,937][123614] Updated weights for policy 1, policy_version 80260 (0.0011) [2023-10-10 19:43:18,303][123614] Updated weights for policy 1, policy_version 80270 (0.0010) [2023-10-10 19:43:18,678][123614] Updated weights for policy 1, policy_version 80280 (0.0009) [2023-10-10 19:43:18,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 164495360. Throughput: 0: 1832.2, 1: 1814.4. Samples: 41136990. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-10 19:43:18,789][122664] Avg episode reward: [(0, '76.850'), (1, '67.720')] [2023-10-10 19:43:20,126][123582] Updated weights for policy 0, policy_version 80393 (0.0007) [2023-10-10 19:43:20,505][123582] Updated weights for policy 0, policy_version 80403 (0.0008) [2023-10-10 19:43:20,867][123582] Updated weights for policy 0, policy_version 80413 (0.0010) [2023-10-10 19:43:22,468][123614] Updated weights for policy 1, policy_version 80290 (0.0007) [2023-10-10 19:43:22,835][123614] Updated weights for policy 1, policy_version 80300 (0.0009) [2023-10-10 19:43:23,203][123614] Updated weights for policy 1, policy_version 80310 (0.0009) [2023-10-10 19:43:23,578][123614] Updated weights for policy 1, policy_version 80320 (0.0008) [2023-10-10 19:43:23,788][122664] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 164593664. Throughput: 0: 1825.6, 1: 1816.7. Samples: 41158388. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-10 19:43:23,789][122664] Avg episode reward: [(0, '77.790'), (1, '69.170')] [2023-10-10 19:43:24,503][123582] Updated weights for policy 0, policy_version 80423 (0.0010) [2023-10-10 19:43:24,875][123582] Updated weights for policy 0, policy_version 80433 (0.0007) [2023-10-10 19:43:25,251][123582] Updated weights for policy 0, policy_version 80443 (0.0007) [2023-10-10 19:43:27,222][123614] Updated weights for policy 1, policy_version 80330 (0.0009) [2023-10-10 19:43:27,597][123614] Updated weights for policy 1, policy_version 80340 (0.0008) [2023-10-10 19:43:27,956][123614] Updated weights for policy 1, policy_version 80350 (0.0007) [2023-10-10 19:43:28,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 164659200. Throughput: 0: 1821.9, 1: 1813.0. Samples: 41169690. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-10 19:43:28,789][122664] Avg episode reward: [(0, '78.710'), (1, '67.560')] [2023-10-10 19:43:29,100][123582] Updated weights for policy 0, policy_version 80453 (0.0009) [2023-10-10 19:43:29,475][123582] Updated weights for policy 0, policy_version 80463 (0.0010) [2023-10-10 19:43:29,845][123582] Updated weights for policy 0, policy_version 80473 (0.0009) [2023-10-10 19:43:31,615][123614] Updated weights for policy 1, policy_version 80360 (0.0008) [2023-10-10 19:43:31,981][123614] Updated weights for policy 1, policy_version 80370 (0.0010) [2023-10-10 19:43:32,358][123614] Updated weights for policy 1, policy_version 80380 (0.0010) [2023-10-10 19:43:33,525][123582] Updated weights for policy 0, policy_version 80483 (0.0009) [2023-10-10 19:43:33,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 164724736. Throughput: 0: 1816.2, 1: 1808.0. Samples: 41190832. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-10 19:43:33,789][122664] Avg episode reward: [(0, '78.280'), (1, '73.830')] [2023-10-10 19:43:33,893][123582] Updated weights for policy 0, policy_version 80493 (0.0009) [2023-10-10 19:43:34,266][123582] Updated weights for policy 0, policy_version 80503 (0.0009) [2023-10-10 19:43:36,087][123614] Updated weights for policy 1, policy_version 80390 (0.0008) [2023-10-10 19:43:36,472][123614] Updated weights for policy 1, policy_version 80400 (0.0007) [2023-10-10 19:43:36,837][123614] Updated weights for policy 1, policy_version 80410 (0.0008) [2023-10-10 19:43:38,009][123582] Updated weights for policy 0, policy_version 80513 (0.0009) [2023-10-10 19:43:38,377][123582] Updated weights for policy 0, policy_version 80523 (0.0008) [2023-10-10 19:43:38,750][123582] Updated weights for policy 0, policy_version 80533 (0.0008) [2023-10-10 19:43:38,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.6, 300 sec: 14440.1). Total num frames: 164790272. Throughput: 0: 1814.1, 1: 1816.3. Samples: 41213204. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-10 19:43:38,789][122664] Avg episode reward: [(0, '80.530'), (1, '75.710')] [2023-10-10 19:43:39,122][123582] Updated weights for policy 0, policy_version 80543 (0.0007) [2023-10-10 19:43:40,286][123614] Updated weights for policy 1, policy_version 80420 (0.0009) [2023-10-10 19:43:40,657][123614] Updated weights for policy 1, policy_version 80430 (0.0008) [2023-10-10 19:43:41,013][123614] Updated weights for policy 1, policy_version 80440 (0.0007) [2023-10-10 19:43:42,770][123582] Updated weights for policy 0, policy_version 80553 (0.0007) [2023-10-10 19:43:43,139][123582] Updated weights for policy 0, policy_version 80563 (0.0007) [2023-10-10 19:43:43,507][123582] Updated weights for policy 0, policy_version 80573 (0.0009) [2023-10-10 19:43:43,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 164888576. Throughput: 0: 1812.5, 1: 1816.8. Samples: 41223838. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-10 19:43:43,788][122664] Avg episode reward: [(0, '81.520'), (1, '77.000')] [2023-10-10 19:43:44,718][123614] Updated weights for policy 1, policy_version 80450 (0.0008) [2023-10-10 19:43:45,090][123614] Updated weights for policy 1, policy_version 80460 (0.0007) [2023-10-10 19:43:45,460][123614] Updated weights for policy 1, policy_version 80470 (0.0011) [2023-10-10 19:43:45,833][123614] Updated weights for policy 1, policy_version 80480 (0.0007) [2023-10-10 19:43:46,821][123582] Updated weights for policy 0, policy_version 80583 (0.0009) [2023-10-10 19:43:47,187][123582] Updated weights for policy 0, policy_version 80593 (0.0008) [2023-10-10 19:43:47,563][123582] Updated weights for policy 0, policy_version 80603 (0.0008) [2023-10-10 19:43:48,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 164954112. Throughput: 0: 1811.7, 1: 1827.2. Samples: 41246264. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-10 19:43:48,789][122664] Avg episode reward: [(0, '84.060'), (1, '75.830')] [2023-10-10 19:43:49,290][123614] Updated weights for policy 1, policy_version 80490 (0.0009) [2023-10-10 19:43:49,660][123614] Updated weights for policy 1, policy_version 80500 (0.0008) [2023-10-10 19:43:50,029][123614] Updated weights for policy 1, policy_version 80510 (0.0007) [2023-10-10 19:43:51,280][123582] Updated weights for policy 0, policy_version 80613 (0.0008) [2023-10-10 19:43:51,659][123582] Updated weights for policy 0, policy_version 80623 (0.0010) [2023-10-10 19:43:52,033][123582] Updated weights for policy 0, policy_version 80633 (0.0008) [2023-10-10 19:43:53,630][123614] Updated weights for policy 1, policy_version 80520 (0.0010) [2023-10-10 19:43:53,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 165019648. Throughput: 0: 1818.8, 1: 1821.8. Samples: 41268250. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-10 19:43:53,788][122664] Avg episode reward: [(0, '80.250'), (1, '76.790')] [2023-10-10 19:43:53,797][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000080640_82575360.pth... [2023-10-10 19:43:53,832][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000078944_80838656.pth [2023-10-10 19:43:54,010][123614] Updated weights for policy 1, policy_version 80530 (0.0009) [2023-10-10 19:43:54,375][123614] Updated weights for policy 1, policy_version 80540 (0.0009) [2023-10-10 19:43:54,519][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000080544_82477056.pth... [2023-10-10 19:43:54,559][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000078816_80707584.pth [2023-10-10 19:43:55,846][123582] Updated weights for policy 0, policy_version 80643 (0.0007) [2023-10-10 19:43:56,221][123582] Updated weights for policy 0, policy_version 80653 (0.0007) [2023-10-10 19:43:56,594][123582] Updated weights for policy 0, policy_version 80663 (0.0007) [2023-10-10 19:43:58,160][123614] Updated weights for policy 1, policy_version 80550 (0.0010) [2023-10-10 19:43:58,537][123614] Updated weights for policy 1, policy_version 80560 (0.0010) [2023-10-10 19:43:58,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 165085184. Throughput: 0: 1817.1, 1: 1830.9. Samples: 41279356. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-10 19:43:58,788][122664] Avg episode reward: [(0, '82.650'), (1, '79.120')] [2023-10-10 19:43:58,903][123614] Updated weights for policy 1, policy_version 80570 (0.0009) [2023-10-10 19:44:00,372][123582] Updated weights for policy 0, policy_version 80673 (0.0008) [2023-10-10 19:44:00,738][123582] Updated weights for policy 0, policy_version 80683 (0.0008) [2023-10-10 19:44:01,109][123582] Updated weights for policy 0, policy_version 80693 (0.0009) [2023-10-10 19:44:01,479][123582] Updated weights for policy 0, policy_version 80703 (0.0007) [2023-10-10 19:44:02,604][123614] Updated weights for policy 1, policy_version 80580 (0.0009) [2023-10-10 19:44:02,980][123614] Updated weights for policy 1, policy_version 80590 (0.0010) [2023-10-10 19:44:03,337][123614] Updated weights for policy 1, policy_version 80600 (0.0010) [2023-10-10 19:44:03,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 165183488. Throughput: 0: 1820.9, 1: 1824.6. Samples: 41301038. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-10 19:44:03,789][122664] Avg episode reward: [(0, '85.250'), (1, '79.760')] [2023-10-10 19:44:05,217][123582] Updated weights for policy 0, policy_version 80713 (0.0007) [2023-10-10 19:44:05,581][123582] Updated weights for policy 0, policy_version 80723 (0.0008) [2023-10-10 19:44:05,945][123582] Updated weights for policy 0, policy_version 80733 (0.0007) [2023-10-10 19:44:06,901][123614] Updated weights for policy 1, policy_version 80610 (0.0009) [2023-10-10 19:44:07,270][123614] Updated weights for policy 1, policy_version 80620 (0.0008) [2023-10-10 19:44:07,638][123614] Updated weights for policy 1, policy_version 80630 (0.0011) [2023-10-10 19:44:08,001][123614] Updated weights for policy 1, policy_version 80640 (0.0009) [2023-10-10 19:44:08,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 165249024. Throughput: 0: 1819.4, 1: 1835.9. Samples: 41322874. Policy #0 lag: (min: 1.0, avg: 13.3, max: 33.0) [2023-10-10 19:44:08,788][122664] Avg episode reward: [(0, '85.760'), (1, '78.640')] [2023-10-10 19:44:09,652][123582] Updated weights for policy 0, policy_version 80743 (0.0009) [2023-10-10 19:44:10,021][123582] Updated weights for policy 0, policy_version 80753 (0.0007) [2023-10-10 19:44:10,393][123582] Updated weights for policy 0, policy_version 80763 (0.0008) [2023-10-10 19:44:11,762][123614] Updated weights for policy 1, policy_version 80650 (0.0007) [2023-10-10 19:44:12,122][123614] Updated weights for policy 1, policy_version 80660 (0.0010) [2023-10-10 19:44:12,489][123614] Updated weights for policy 1, policy_version 80670 (0.0009) [2023-10-10 19:44:13,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 165314560. Throughput: 0: 1819.0, 1: 1828.4. Samples: 41333826. Policy #0 lag: (min: 1.0, avg: 13.3, max: 33.0) [2023-10-10 19:44:13,789][122664] Avg episode reward: [(0, '87.490'), (1, '78.700')] [2023-10-10 19:44:14,093][123582] Updated weights for policy 0, policy_version 80773 (0.0009) [2023-10-10 19:44:14,460][123582] Updated weights for policy 0, policy_version 80783 (0.0011) [2023-10-10 19:44:14,836][123582] Updated weights for policy 0, policy_version 80793 (0.0008) [2023-10-10 19:44:16,202][123614] Updated weights for policy 1, policy_version 80680 (0.0010) [2023-10-10 19:44:16,564][123614] Updated weights for policy 1, policy_version 80690 (0.0010) [2023-10-10 19:44:16,927][123614] Updated weights for policy 1, policy_version 80700 (0.0008) [2023-10-10 19:44:18,506][123582] Updated weights for policy 0, policy_version 80803 (0.0008) [2023-10-10 19:44:18,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 165380096. Throughput: 0: 1821.7, 1: 1836.1. Samples: 41355436. Policy #0 lag: (min: 1.0, avg: 13.3, max: 33.0) [2023-10-10 19:44:18,788][122664] Avg episode reward: [(0, '85.510'), (1, '87.830')] [2023-10-10 19:44:18,879][123582] Updated weights for policy 0, policy_version 80813 (0.0008) [2023-10-10 19:44:19,256][123582] Updated weights for policy 0, policy_version 80823 (0.0009) [2023-10-10 19:44:20,757][123614] Updated weights for policy 1, policy_version 80710 (0.0010) [2023-10-10 19:44:21,142][123614] Updated weights for policy 1, policy_version 80720 (0.0009) [2023-10-10 19:44:21,519][123614] Updated weights for policy 1, policy_version 80730 (0.0009) [2023-10-10 19:44:23,003][123582] Updated weights for policy 0, policy_version 80833 (0.0009) [2023-10-10 19:44:23,365][123582] Updated weights for policy 0, policy_version 80843 (0.0008) [2023-10-10 19:44:23,751][123582] Updated weights for policy 0, policy_version 80853 (0.0009) [2023-10-10 19:44:23,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 165445632. Throughput: 0: 1823.6, 1: 1830.9. Samples: 41377656. Policy #0 lag: (min: 1.0, avg: 13.3, max: 33.0) [2023-10-10 19:44:23,789][122664] Avg episode reward: [(0, '80.060'), (1, '86.070')] [2023-10-10 19:44:24,121][123582] Updated weights for policy 0, policy_version 80863 (0.0007) [2023-10-10 19:44:25,064][123614] Updated weights for policy 1, policy_version 80740 (0.0007) [2023-10-10 19:44:25,429][123614] Updated weights for policy 1, policy_version 80750 (0.0008) [2023-10-10 19:44:25,802][123614] Updated weights for policy 1, policy_version 80760 (0.0007) [2023-10-10 19:44:27,989][123582] Updated weights for policy 0, policy_version 80873 (0.0009) [2023-10-10 19:44:28,374][123582] Updated weights for policy 0, policy_version 80883 (0.0010) [2023-10-10 19:44:28,748][123582] Updated weights for policy 0, policy_version 80893 (0.0009) [2023-10-10 19:44:28,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 165511168. Throughput: 0: 1818.7, 1: 1829.1. Samples: 41387988. Policy #0 lag: (min: 1.0, avg: 13.3, max: 33.0) [2023-10-10 19:44:28,789][122664] Avg episode reward: [(0, '81.820'), (1, '85.640')] [2023-10-10 19:44:29,583][123614] Updated weights for policy 1, policy_version 80770 (0.0009) [2023-10-10 19:44:29,957][123614] Updated weights for policy 1, policy_version 80780 (0.0008) [2023-10-10 19:44:30,315][123614] Updated weights for policy 1, policy_version 80790 (0.0008) [2023-10-10 19:44:30,688][123614] Updated weights for policy 1, policy_version 80800 (0.0007) [2023-10-10 19:44:32,414][123582] Updated weights for policy 0, policy_version 80903 (0.0008) [2023-10-10 19:44:32,768][123582] Updated weights for policy 0, policy_version 80913 (0.0008) [2023-10-10 19:44:33,151][123582] Updated weights for policy 0, policy_version 80923 (0.0008) [2023-10-10 19:44:33,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 165609472. Throughput: 0: 1821.9, 1: 1824.8. Samples: 41410364. Policy #0 lag: (min: 1.0, avg: 13.3, max: 33.0) [2023-10-10 19:44:33,789][122664] Avg episode reward: [(0, '78.490'), (1, '88.630')] [2023-10-10 19:44:34,289][123614] Updated weights for policy 1, policy_version 80810 (0.0008) [2023-10-10 19:44:34,646][123614] Updated weights for policy 1, policy_version 80820 (0.0008) [2023-10-10 19:44:35,021][123614] Updated weights for policy 1, policy_version 80830 (0.0008) [2023-10-10 19:44:36,913][123582] Updated weights for policy 0, policy_version 80933 (0.0009) [2023-10-10 19:44:37,293][123582] Updated weights for policy 0, policy_version 80943 (0.0010) [2023-10-10 19:44:37,660][123582] Updated weights for policy 0, policy_version 80953 (0.0008) [2023-10-10 19:44:38,716][123614] Updated weights for policy 1, policy_version 80840 (0.0010) [2023-10-10 19:44:38,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 165675008. Throughput: 0: 1805.4, 1: 1826.9. Samples: 41431704. Policy #0 lag: (min: 1.0, avg: 13.3, max: 33.0) [2023-10-10 19:44:38,789][122664] Avg episode reward: [(0, '83.990'), (1, '87.970')] [2023-10-10 19:44:39,097][123614] Updated weights for policy 1, policy_version 80850 (0.0010) [2023-10-10 19:44:39,462][123614] Updated weights for policy 1, policy_version 80860 (0.0009) [2023-10-10 19:44:41,308][123582] Updated weights for policy 0, policy_version 80963 (0.0008) [2023-10-10 19:44:41,673][123582] Updated weights for policy 0, policy_version 80973 (0.0008) [2023-10-10 19:44:42,048][123582] Updated weights for policy 0, policy_version 80983 (0.0008) [2023-10-10 19:44:43,263][123614] Updated weights for policy 1, policy_version 80870 (0.0008) [2023-10-10 19:44:43,631][123614] Updated weights for policy 1, policy_version 80880 (0.0007) [2023-10-10 19:44:43,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 165740544. Throughput: 0: 1815.5, 1: 1823.7. Samples: 41443120. Policy #0 lag: (min: 1.0, avg: 13.3, max: 33.0) [2023-10-10 19:44:43,788][122664] Avg episode reward: [(0, '81.280'), (1, '88.640')] [2023-10-10 19:44:44,003][123614] Updated weights for policy 1, policy_version 80890 (0.0007) [2023-10-10 19:44:45,716][123582] Updated weights for policy 0, policy_version 80993 (0.0008) [2023-10-10 19:44:46,086][123582] Updated weights for policy 0, policy_version 81003 (0.0007) [2023-10-10 19:44:46,461][123582] Updated weights for policy 0, policy_version 81013 (0.0008) [2023-10-10 19:44:46,835][123582] Updated weights for policy 0, policy_version 81023 (0.0011) [2023-10-10 19:44:47,483][123614] Updated weights for policy 1, policy_version 80900 (0.0008) [2023-10-10 19:44:47,848][123614] Updated weights for policy 1, policy_version 80910 (0.0008) [2023-10-10 19:44:48,213][123614] Updated weights for policy 1, policy_version 80920 (0.0009) [2023-10-10 19:44:48,788][122664] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 165838848. Throughput: 0: 1810.4, 1: 1818.8. Samples: 41464352. Policy #0 lag: (min: 1.0, avg: 13.3, max: 33.0) [2023-10-10 19:44:48,788][122664] Avg episode reward: [(0, '83.930'), (1, '91.340')] [2023-10-10 19:44:50,491][123582] Updated weights for policy 0, policy_version 81033 (0.0008) [2023-10-10 19:44:50,875][123582] Updated weights for policy 0, policy_version 81043 (0.0010) [2023-10-10 19:44:51,238][123582] Updated weights for policy 0, policy_version 81053 (0.0009) [2023-10-10 19:44:51,835][123614] Updated weights for policy 1, policy_version 80930 (0.0007) [2023-10-10 19:44:52,205][123614] Updated weights for policy 1, policy_version 80940 (0.0008) [2023-10-10 19:44:52,575][123614] Updated weights for policy 1, policy_version 80950 (0.0008) [2023-10-10 19:44:52,945][123614] Updated weights for policy 1, policy_version 80960 (0.0010) [2023-10-10 19:44:53,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 165904384. Throughput: 0: 1809.2, 1: 1823.6. Samples: 41486354. Policy #0 lag: (min: 1.0, avg: 13.3, max: 33.0) [2023-10-10 19:44:53,789][122664] Avg episode reward: [(0, '83.530'), (1, '80.690')] [2023-10-10 19:44:54,942][123582] Updated weights for policy 0, policy_version 81063 (0.0010) [2023-10-10 19:44:55,310][123582] Updated weights for policy 0, policy_version 81073 (0.0010) [2023-10-10 19:44:55,679][123582] Updated weights for policy 0, policy_version 81083 (0.0009) [2023-10-10 19:44:56,699][123614] Updated weights for policy 1, policy_version 80970 (0.0009) [2023-10-10 19:44:57,066][123614] Updated weights for policy 1, policy_version 80980 (0.0008) [2023-10-10 19:44:57,429][123614] Updated weights for policy 1, policy_version 80990 (0.0008) [2023-10-10 19:44:58,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 165969920. Throughput: 0: 1808.5, 1: 1819.2. Samples: 41497068. Policy #0 lag: (min: 17.0, avg: 19.1, max: 43.0) [2023-10-10 19:44:58,788][122664] Avg episode reward: [(0, '82.800'), (1, '77.770')] [2023-10-10 19:44:59,281][123582] Updated weights for policy 0, policy_version 81093 (0.0009) [2023-10-10 19:44:59,650][123582] Updated weights for policy 0, policy_version 81103 (0.0009) [2023-10-10 19:45:00,015][123582] Updated weights for policy 0, policy_version 81113 (0.0011) [2023-10-10 19:45:01,058][123614] Updated weights for policy 1, policy_version 81000 (0.0008) [2023-10-10 19:45:01,437][123614] Updated weights for policy 1, policy_version 81010 (0.0008) [2023-10-10 19:45:01,799][123614] Updated weights for policy 1, policy_version 81020 (0.0009) [2023-10-10 19:45:03,580][123582] Updated weights for policy 0, policy_version 81123 (0.0008) [2023-10-10 19:45:03,788][122664] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 166035456. Throughput: 0: 1810.9, 1: 1828.2. Samples: 41519194. Policy #0 lag: (min: 17.0, avg: 19.1, max: 43.0) [2023-10-10 19:45:03,788][122664] Avg episode reward: [(0, '82.100'), (1, '77.320')] [2023-10-10 19:45:03,946][123582] Updated weights for policy 0, policy_version 81133 (0.0008) [2023-10-10 19:45:04,321][123582] Updated weights for policy 0, policy_version 81143 (0.0008) [2023-10-10 19:45:05,677][123614] Updated weights for policy 1, policy_version 81030 (0.0008) [2023-10-10 19:45:06,044][123614] Updated weights for policy 1, policy_version 81040 (0.0010) [2023-10-10 19:45:06,418][123614] Updated weights for policy 1, policy_version 81050 (0.0009) [2023-10-10 19:45:07,992][123582] Updated weights for policy 0, policy_version 81153 (0.0011) [2023-10-10 19:45:08,364][123582] Updated weights for policy 0, policy_version 81163 (0.0008) [2023-10-10 19:45:08,730][123582] Updated weights for policy 0, policy_version 81173 (0.0008) [2023-10-10 19:45:08,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 166100992. Throughput: 0: 1815.9, 1: 1824.8. Samples: 41541486. Policy #0 lag: (min: 17.0, avg: 19.1, max: 43.0) [2023-10-10 19:45:08,789][122664] Avg episode reward: [(0, '82.890'), (1, '74.510')] [2023-10-10 19:45:09,091][123582] Updated weights for policy 0, policy_version 81183 (0.0008) [2023-10-10 19:45:10,060][123614] Updated weights for policy 1, policy_version 81060 (0.0008) [2023-10-10 19:45:10,431][123614] Updated weights for policy 1, policy_version 81070 (0.0007) [2023-10-10 19:45:10,801][123614] Updated weights for policy 1, policy_version 81080 (0.0007) [2023-10-10 19:45:12,787][123582] Updated weights for policy 0, policy_version 81193 (0.0008) [2023-10-10 19:45:13,157][123582] Updated weights for policy 0, policy_version 81203 (0.0009) [2023-10-10 19:45:13,532][123582] Updated weights for policy 0, policy_version 81213 (0.0007) [2023-10-10 19:45:13,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 166199296. Throughput: 0: 1818.0, 1: 1825.8. Samples: 41551960. Policy #0 lag: (min: 17.0, avg: 19.1, max: 43.0) [2023-10-10 19:45:13,789][122664] Avg episode reward: [(0, '79.570'), (1, '76.040')] [2023-10-10 19:45:14,526][123614] Updated weights for policy 1, policy_version 81090 (0.0009) [2023-10-10 19:45:14,896][123614] Updated weights for policy 1, policy_version 81100 (0.0008) [2023-10-10 19:45:15,266][123614] Updated weights for policy 1, policy_version 81110 (0.0008) [2023-10-10 19:45:15,631][123614] Updated weights for policy 1, policy_version 81120 (0.0007) [2023-10-10 19:45:17,200][123582] Updated weights for policy 0, policy_version 81223 (0.0008) [2023-10-10 19:45:17,574][123582] Updated weights for policy 0, policy_version 81233 (0.0009) [2023-10-10 19:45:17,943][123582] Updated weights for policy 0, policy_version 81243 (0.0008) [2023-10-10 19:45:18,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 166264832. Throughput: 0: 1814.0, 1: 1828.6. Samples: 41574282. Policy #0 lag: (min: 17.0, avg: 19.1, max: 43.0) [2023-10-10 19:45:18,789][122664] Avg episode reward: [(0, '82.060'), (1, '74.760')] [2023-10-10 19:45:19,205][123614] Updated weights for policy 1, policy_version 81130 (0.0008) [2023-10-10 19:45:19,570][123614] Updated weights for policy 1, policy_version 81140 (0.0008) [2023-10-10 19:45:19,932][123614] Updated weights for policy 1, policy_version 81150 (0.0012) [2023-10-10 19:45:21,629][123582] Updated weights for policy 0, policy_version 81253 (0.0010) [2023-10-10 19:45:21,995][123582] Updated weights for policy 0, policy_version 81263 (0.0010) [2023-10-10 19:45:22,360][123582] Updated weights for policy 0, policy_version 81273 (0.0007) [2023-10-10 19:45:23,634][123614] Updated weights for policy 1, policy_version 81160 (0.0008) [2023-10-10 19:45:23,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 166330368. Throughput: 0: 1823.7, 1: 1823.1. Samples: 41595810. Policy #0 lag: (min: 17.0, avg: 19.1, max: 43.0) [2023-10-10 19:45:23,789][122664] Avg episode reward: [(0, '78.550'), (1, '75.500')] [2023-10-10 19:45:24,007][123614] Updated weights for policy 1, policy_version 81170 (0.0007) [2023-10-10 19:45:24,369][123614] Updated weights for policy 1, policy_version 81180 (0.0008) [2023-10-10 19:45:26,039][123582] Updated weights for policy 0, policy_version 81283 (0.0009) [2023-10-10 19:45:26,413][123582] Updated weights for policy 0, policy_version 81293 (0.0011) [2023-10-10 19:45:26,793][123582] Updated weights for policy 0, policy_version 81303 (0.0010) [2023-10-10 19:45:27,987][123614] Updated weights for policy 1, policy_version 81190 (0.0009) [2023-10-10 19:45:28,358][123614] Updated weights for policy 1, policy_version 81200 (0.0008) [2023-10-10 19:45:28,720][123614] Updated weights for policy 1, policy_version 81210 (0.0008) [2023-10-10 19:45:28,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 166395904. Throughput: 0: 1823.7, 1: 1826.0. Samples: 41607358. Policy #0 lag: (min: 17.0, avg: 19.1, max: 43.0) [2023-10-10 19:45:28,788][122664] Avg episode reward: [(0, '76.930'), (1, '70.440')] [2023-10-10 19:45:30,550][123582] Updated weights for policy 0, policy_version 81313 (0.0008) [2023-10-10 19:45:30,916][123582] Updated weights for policy 0, policy_version 81323 (0.0009) [2023-10-10 19:45:31,289][123582] Updated weights for policy 0, policy_version 81333 (0.0009) [2023-10-10 19:45:31,667][123582] Updated weights for policy 0, policy_version 81343 (0.0010) [2023-10-10 19:45:32,535][123614] Updated weights for policy 1, policy_version 81220 (0.0008) [2023-10-10 19:45:32,897][123614] Updated weights for policy 1, policy_version 81230 (0.0008) [2023-10-10 19:45:33,272][123614] Updated weights for policy 1, policy_version 81240 (0.0009) [2023-10-10 19:45:33,788][122664] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 166494208. Throughput: 0: 1821.0, 1: 1824.2. Samples: 41628386. Policy #0 lag: (min: 17.0, avg: 19.1, max: 43.0) [2023-10-10 19:45:33,788][122664] Avg episode reward: [(0, '74.830'), (1, '76.780')] [2023-10-10 19:45:35,630][123582] Updated weights for policy 0, policy_version 81353 (0.0009) [2023-10-10 19:45:35,992][123582] Updated weights for policy 0, policy_version 81363 (0.0009) [2023-10-10 19:45:36,365][123582] Updated weights for policy 0, policy_version 81373 (0.0012) [2023-10-10 19:45:36,932][123614] Updated weights for policy 1, policy_version 81250 (0.0007) [2023-10-10 19:45:37,292][123614] Updated weights for policy 1, policy_version 81260 (0.0008) [2023-10-10 19:45:37,659][123614] Updated weights for policy 1, policy_version 81270 (0.0008) [2023-10-10 19:45:38,033][123614] Updated weights for policy 1, policy_version 81280 (0.0009) [2023-10-10 19:45:38,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 166559744. Throughput: 0: 1810.3, 1: 1816.5. Samples: 41649558. Policy #0 lag: (min: 17.0, avg: 19.1, max: 43.0) [2023-10-10 19:45:38,789][122664] Avg episode reward: [(0, '78.220'), (1, '76.380')] [2023-10-10 19:45:40,193][123582] Updated weights for policy 0, policy_version 81383 (0.0010) [2023-10-10 19:45:40,566][123582] Updated weights for policy 0, policy_version 81393 (0.0009) [2023-10-10 19:45:40,943][123582] Updated weights for policy 0, policy_version 81403 (0.0008) [2023-10-10 19:45:41,791][123614] Updated weights for policy 1, policy_version 81290 (0.0009) [2023-10-10 19:45:42,143][123614] Updated weights for policy 1, policy_version 81300 (0.0008) [2023-10-10 19:45:42,520][123614] Updated weights for policy 1, policy_version 81310 (0.0008) [2023-10-10 19:45:43,788][122664] Fps is (10 sec: 13106.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 166625280. Throughput: 0: 1809.4, 1: 1819.2. Samples: 41660356. Policy #0 lag: (min: 17.0, avg: 19.1, max: 43.0) [2023-10-10 19:45:43,789][122664] Avg episode reward: [(0, '79.980'), (1, '77.180')] [2023-10-10 19:45:44,552][123582] Updated weights for policy 0, policy_version 81413 (0.0009) [2023-10-10 19:45:44,935][123582] Updated weights for policy 0, policy_version 81423 (0.0008) [2023-10-10 19:45:45,319][123582] Updated weights for policy 0, policy_version 81433 (0.0009) [2023-10-10 19:45:46,182][123614] Updated weights for policy 1, policy_version 81320 (0.0009) [2023-10-10 19:45:46,549][123614] Updated weights for policy 1, policy_version 81330 (0.0010) [2023-10-10 19:45:46,922][123614] Updated weights for policy 1, policy_version 81340 (0.0011) [2023-10-10 19:45:48,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 166690816. Throughput: 0: 1809.8, 1: 1812.4. Samples: 41682194. Policy #0 lag: (min: 17.0, avg: 19.1, max: 43.0) [2023-10-10 19:45:48,789][122664] Avg episode reward: [(0, '74.330'), (1, '79.930')] [2023-10-10 19:45:49,042][123582] Updated weights for policy 0, policy_version 81443 (0.0010) [2023-10-10 19:45:49,412][123582] Updated weights for policy 0, policy_version 81453 (0.0008) [2023-10-10 19:45:49,783][123582] Updated weights for policy 0, policy_version 81463 (0.0010) [2023-10-10 19:45:50,826][123614] Updated weights for policy 1, policy_version 81350 (0.0008) [2023-10-10 19:45:51,216][123614] Updated weights for policy 1, policy_version 81360 (0.0010) [2023-10-10 19:45:51,571][123614] Updated weights for policy 1, policy_version 81370 (0.0010) [2023-10-10 19:45:53,419][123582] Updated weights for policy 0, policy_version 81473 (0.0007) [2023-10-10 19:45:53,784][123582] Updated weights for policy 0, policy_version 81483 (0.0008) [2023-10-10 19:45:53,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 166756352. Throughput: 0: 1816.0, 1: 1809.1. Samples: 41704616. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:45:53,788][122664] Avg episode reward: [(0, '73.990'), (1, '81.410')] [2023-10-10 19:45:53,798][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000081376_83329024.pth... [2023-10-10 19:45:53,834][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000079680_81592320.pth [2023-10-10 19:45:54,167][123582] Updated weights for policy 0, policy_version 81493 (0.0008) [2023-10-10 19:45:54,524][123582] Updated weights for policy 0, policy_version 81503 (0.0007) [2023-10-10 19:45:54,562][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000081504_83460096.pth... [2023-10-10 19:45:54,593][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000079776_81690624.pth [2023-10-10 19:45:55,278][123614] Updated weights for policy 1, policy_version 81380 (0.0009) [2023-10-10 19:45:55,642][123614] Updated weights for policy 1, policy_version 81390 (0.0007) [2023-10-10 19:45:56,006][123614] Updated weights for policy 1, policy_version 81400 (0.0008) [2023-10-10 19:45:58,166][123582] Updated weights for policy 0, policy_version 81513 (0.0010) [2023-10-10 19:45:58,540][123582] Updated weights for policy 0, policy_version 81523 (0.0008) [2023-10-10 19:45:58,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 166821888. Throughput: 0: 1808.8, 1: 1808.2. Samples: 41714728. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:45:58,789][122664] Avg episode reward: [(0, '77.460'), (1, '81.230')] [2023-10-10 19:45:58,901][123582] Updated weights for policy 0, policy_version 81533 (0.0007) [2023-10-10 19:45:59,805][123614] Updated weights for policy 1, policy_version 81410 (0.0008) [2023-10-10 19:46:00,168][123614] Updated weights for policy 1, policy_version 81420 (0.0008) [2023-10-10 19:46:00,544][123614] Updated weights for policy 1, policy_version 81430 (0.0009) [2023-10-10 19:46:00,910][123614] Updated weights for policy 1, policy_version 81440 (0.0007) [2023-10-10 19:46:02,643][123582] Updated weights for policy 0, policy_version 81543 (0.0008) [2023-10-10 19:46:03,011][123582] Updated weights for policy 0, policy_version 81553 (0.0007) [2023-10-10 19:46:03,378][123582] Updated weights for policy 0, policy_version 81563 (0.0008) [2023-10-10 19:46:03,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 166920192. Throughput: 0: 1823.2, 1: 1803.1. Samples: 41737466. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:46:03,789][122664] Avg episode reward: [(0, '78.480'), (1, '80.400')] [2023-10-10 19:46:04,600][123614] Updated weights for policy 1, policy_version 81450 (0.0011) [2023-10-10 19:46:04,976][123614] Updated weights for policy 1, policy_version 81460 (0.0008) [2023-10-10 19:46:05,338][123614] Updated weights for policy 1, policy_version 81470 (0.0008) [2023-10-10 19:46:07,065][123582] Updated weights for policy 0, policy_version 81573 (0.0009) [2023-10-10 19:46:07,445][123582] Updated weights for policy 0, policy_version 81583 (0.0008) [2023-10-10 19:46:07,821][123582] Updated weights for policy 0, policy_version 81593 (0.0009) [2023-10-10 19:46:08,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 166985728. Throughput: 0: 1812.6, 1: 1812.4. Samples: 41758936. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:46:08,789][122664] Avg episode reward: [(0, '79.980'), (1, '78.400')] [2023-10-10 19:46:08,904][123614] Updated weights for policy 1, policy_version 81480 (0.0011) [2023-10-10 19:46:09,270][123614] Updated weights for policy 1, policy_version 81490 (0.0007) [2023-10-10 19:46:09,648][123614] Updated weights for policy 1, policy_version 81500 (0.0008) [2023-10-10 19:46:11,510][123582] Updated weights for policy 0, policy_version 81603 (0.0008) [2023-10-10 19:46:11,888][123582] Updated weights for policy 0, policy_version 81613 (0.0007) [2023-10-10 19:46:12,255][123582] Updated weights for policy 0, policy_version 81623 (0.0008) [2023-10-10 19:46:13,193][123614] Updated weights for policy 1, policy_version 81510 (0.0009) [2023-10-10 19:46:13,558][123614] Updated weights for policy 1, policy_version 81520 (0.0009) [2023-10-10 19:46:13,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 167051264. Throughput: 0: 1817.1, 1: 1802.9. Samples: 41770258. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:46:13,788][122664] Avg episode reward: [(0, '80.690'), (1, '81.930')] [2023-10-10 19:46:13,917][123614] Updated weights for policy 1, policy_version 81530 (0.0008) [2023-10-10 19:46:15,882][123582] Updated weights for policy 0, policy_version 81633 (0.0008) [2023-10-10 19:46:16,244][123582] Updated weights for policy 0, policy_version 81643 (0.0008) [2023-10-10 19:46:16,620][123582] Updated weights for policy 0, policy_version 81653 (0.0009) [2023-10-10 19:46:16,994][123582] Updated weights for policy 0, policy_version 81663 (0.0011) [2023-10-10 19:46:17,599][123614] Updated weights for policy 1, policy_version 81540 (0.0009) [2023-10-10 19:46:17,964][123614] Updated weights for policy 1, policy_version 81550 (0.0011) [2023-10-10 19:46:18,336][123614] Updated weights for policy 1, policy_version 81560 (0.0009) [2023-10-10 19:46:18,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 167149568. Throughput: 0: 1812.7, 1: 1812.5. Samples: 41791520. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:46:18,789][122664] Avg episode reward: [(0, '81.430'), (1, '85.820')] [2023-10-10 19:46:20,718][123582] Updated weights for policy 0, policy_version 81673 (0.0009) [2023-10-10 19:46:21,075][123582] Updated weights for policy 0, policy_version 81683 (0.0008) [2023-10-10 19:46:21,456][123582] Updated weights for policy 0, policy_version 81693 (0.0007) [2023-10-10 19:46:22,150][123614] Updated weights for policy 1, policy_version 81570 (0.0008) [2023-10-10 19:46:22,522][123614] Updated weights for policy 1, policy_version 81580 (0.0008) [2023-10-10 19:46:22,885][123614] Updated weights for policy 1, policy_version 81590 (0.0009) [2023-10-10 19:46:23,250][123614] Updated weights for policy 1, policy_version 81600 (0.0008) [2023-10-10 19:46:23,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 167215104. Throughput: 0: 1817.2, 1: 1812.0. Samples: 41812876. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:46:23,789][122664] Avg episode reward: [(0, '80.880'), (1, '87.950')] [2023-10-10 19:46:25,285][123582] Updated weights for policy 0, policy_version 81703 (0.0010) [2023-10-10 19:46:25,655][123582] Updated weights for policy 0, policy_version 81713 (0.0010) [2023-10-10 19:46:26,016][123582] Updated weights for policy 0, policy_version 81723 (0.0008) [2023-10-10 19:46:27,093][123614] Updated weights for policy 1, policy_version 81610 (0.0010) [2023-10-10 19:46:27,465][123614] Updated weights for policy 1, policy_version 81620 (0.0007) [2023-10-10 19:46:27,838][123614] Updated weights for policy 1, policy_version 81630 (0.0008) [2023-10-10 19:46:28,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 167280640. Throughput: 0: 1823.2, 1: 1819.2. Samples: 41824262. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:46:28,789][122664] Avg episode reward: [(0, '82.930'), (1, '92.740')] [2023-10-10 19:46:29,762][123582] Updated weights for policy 0, policy_version 81733 (0.0008) [2023-10-10 19:46:30,141][123582] Updated weights for policy 0, policy_version 81743 (0.0008) [2023-10-10 19:46:30,525][123582] Updated weights for policy 0, policy_version 81753 (0.0008) [2023-10-10 19:46:31,624][123614] Updated weights for policy 1, policy_version 81640 (0.0007) [2023-10-10 19:46:32,001][123614] Updated weights for policy 1, policy_version 81650 (0.0007) [2023-10-10 19:46:32,370][123614] Updated weights for policy 1, policy_version 81660 (0.0008) [2023-10-10 19:46:33,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 167346176. Throughput: 0: 1821.7, 1: 1810.1. Samples: 41845628. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:46:33,789][122664] Avg episode reward: [(0, '79.240'), (1, '94.030')] [2023-10-10 19:46:33,989][123582] Updated weights for policy 0, policy_version 81763 (0.0007) [2023-10-10 19:46:34,365][123582] Updated weights for policy 0, policy_version 81773 (0.0009) [2023-10-10 19:46:34,728][123582] Updated weights for policy 0, policy_version 81783 (0.0010) [2023-10-10 19:46:36,068][123614] Updated weights for policy 1, policy_version 81670 (0.0007) [2023-10-10 19:46:36,452][123614] Updated weights for policy 1, policy_version 81680 (0.0008) [2023-10-10 19:46:36,821][123614] Updated weights for policy 1, policy_version 81690 (0.0007) [2023-10-10 19:46:38,428][123582] Updated weights for policy 0, policy_version 81793 (0.0008) [2023-10-10 19:46:38,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 167411712. Throughput: 0: 1822.3, 1: 1821.0. Samples: 41868562. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:46:38,788][122664] Avg episode reward: [(0, '76.370'), (1, '93.220')] [2023-10-10 19:46:38,798][123582] Updated weights for policy 0, policy_version 81803 (0.0007) [2023-10-10 19:46:39,163][123582] Updated weights for policy 0, policy_version 81813 (0.0007) [2023-10-10 19:46:39,533][123582] Updated weights for policy 0, policy_version 81823 (0.0007) [2023-10-10 19:46:40,419][123614] Updated weights for policy 1, policy_version 81700 (0.0008) [2023-10-10 19:46:40,780][123614] Updated weights for policy 1, policy_version 81710 (0.0008) [2023-10-10 19:46:41,155][123614] Updated weights for policy 1, policy_version 81720 (0.0009) [2023-10-10 19:46:43,041][123582] Updated weights for policy 0, policy_version 81833 (0.0009) [2023-10-10 19:46:43,416][123582] Updated weights for policy 0, policy_version 81843 (0.0009) [2023-10-10 19:46:43,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 167477248. Throughput: 0: 1825.3, 1: 1820.5. Samples: 41878788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:46:43,789][122664] Avg episode reward: [(0, '75.260'), (1, '94.130')] [2023-10-10 19:46:43,792][123582] Updated weights for policy 0, policy_version 81853 (0.0008) [2023-10-10 19:46:44,775][123614] Updated weights for policy 1, policy_version 81730 (0.0010) [2023-10-10 19:46:45,134][123614] Updated weights for policy 1, policy_version 81740 (0.0008) [2023-10-10 19:46:45,505][123614] Updated weights for policy 1, policy_version 81750 (0.0008) [2023-10-10 19:46:45,872][123614] Updated weights for policy 1, policy_version 81760 (0.0007) [2023-10-10 19:46:47,476][123582] Updated weights for policy 0, policy_version 81863 (0.0008) [2023-10-10 19:46:47,853][123582] Updated weights for policy 0, policy_version 81873 (0.0008) [2023-10-10 19:46:48,218][123582] Updated weights for policy 0, policy_version 81883 (0.0009) [2023-10-10 19:46:48,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 167575552. Throughput: 0: 1818.9, 1: 1827.9. Samples: 41901572. Policy #0 lag: (min: 1.0, avg: 28.8, max: 32.0) [2023-10-10 19:46:48,789][122664] Avg episode reward: [(0, '79.450'), (1, '99.650')] [2023-10-10 19:46:49,611][123614] Updated weights for policy 1, policy_version 81770 (0.0008) [2023-10-10 19:46:49,974][123614] Updated weights for policy 1, policy_version 81780 (0.0009) [2023-10-10 19:46:50,353][123614] Updated weights for policy 1, policy_version 81790 (0.0007) [2023-10-10 19:46:51,832][123582] Updated weights for policy 0, policy_version 81893 (0.0009) [2023-10-10 19:46:52,200][123582] Updated weights for policy 0, policy_version 81903 (0.0008) [2023-10-10 19:46:52,572][123582] Updated weights for policy 0, policy_version 81913 (0.0008) [2023-10-10 19:46:53,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 167641088. Throughput: 0: 1821.9, 1: 1823.0. Samples: 41922958. Policy #0 lag: (min: 1.0, avg: 28.8, max: 32.0) [2023-10-10 19:46:53,789][122664] Avg episode reward: [(0, '80.710'), (1, '98.970')] [2023-10-10 19:46:53,974][123614] Updated weights for policy 1, policy_version 81800 (0.0008) [2023-10-10 19:46:54,345][123614] Updated weights for policy 1, policy_version 81810 (0.0007) [2023-10-10 19:46:54,716][123614] Updated weights for policy 1, policy_version 81820 (0.0008) [2023-10-10 19:46:56,244][123582] Updated weights for policy 0, policy_version 81923 (0.0008) [2023-10-10 19:46:56,611][123582] Updated weights for policy 0, policy_version 81933 (0.0010) [2023-10-10 19:46:56,977][123582] Updated weights for policy 0, policy_version 81943 (0.0007) [2023-10-10 19:46:58,470][123614] Updated weights for policy 1, policy_version 81830 (0.0008) [2023-10-10 19:46:58,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 167706624. Throughput: 0: 1819.9, 1: 1823.4. Samples: 41934210. Policy #0 lag: (min: 1.0, avg: 28.8, max: 32.0) [2023-10-10 19:46:58,789][122664] Avg episode reward: [(0, '85.030'), (1, '98.950')] [2023-10-10 19:46:58,841][123614] Updated weights for policy 1, policy_version 81840 (0.0007) [2023-10-10 19:46:59,218][123614] Updated weights for policy 1, policy_version 81850 (0.0007) [2023-10-10 19:47:00,640][123582] Updated weights for policy 0, policy_version 81953 (0.0007) [2023-10-10 19:47:01,018][123582] Updated weights for policy 0, policy_version 81963 (0.0011) [2023-10-10 19:47:01,383][123582] Updated weights for policy 0, policy_version 81973 (0.0008) [2023-10-10 19:47:01,752][123582] Updated weights for policy 0, policy_version 81983 (0.0009) [2023-10-10 19:47:02,827][123614] Updated weights for policy 1, policy_version 81860 (0.0009) [2023-10-10 19:47:03,189][123614] Updated weights for policy 1, policy_version 81870 (0.0008) [2023-10-10 19:47:03,555][123614] Updated weights for policy 1, policy_version 81880 (0.0009) [2023-10-10 19:47:03,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 167772160. Throughput: 0: 1823.8, 1: 1827.6. Samples: 41955832. Policy #0 lag: (min: 1.0, avg: 28.8, max: 32.0) [2023-10-10 19:47:03,789][122664] Avg episode reward: [(0, '79.110'), (1, '100.070')] [2023-10-10 19:47:05,663][123582] Updated weights for policy 0, policy_version 81993 (0.0010) [2023-10-10 19:47:06,024][123582] Updated weights for policy 0, policy_version 82003 (0.0009) [2023-10-10 19:47:06,399][123582] Updated weights for policy 0, policy_version 82013 (0.0008) [2023-10-10 19:47:07,194][123614] Updated weights for policy 1, policy_version 81890 (0.0010) [2023-10-10 19:47:07,556][123614] Updated weights for policy 1, policy_version 81900 (0.0009) [2023-10-10 19:47:07,931][123614] Updated weights for policy 1, policy_version 81910 (0.0007) [2023-10-10 19:47:08,296][123614] Updated weights for policy 1, policy_version 81920 (0.0009) [2023-10-10 19:47:08,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 167870464. Throughput: 0: 1824.6, 1: 1820.4. Samples: 41976900. Policy #0 lag: (min: 1.0, avg: 28.8, max: 32.0) [2023-10-10 19:47:08,789][122664] Avg episode reward: [(0, '86.550'), (1, '96.550')] [2023-10-10 19:47:09,907][123582] Updated weights for policy 0, policy_version 82023 (0.0009) [2023-10-10 19:47:10,282][123582] Updated weights for policy 0, policy_version 82033 (0.0010) [2023-10-10 19:47:10,662][123582] Updated weights for policy 0, policy_version 82043 (0.0009) [2023-10-10 19:47:11,908][123614] Updated weights for policy 1, policy_version 81930 (0.0010) [2023-10-10 19:47:12,267][123614] Updated weights for policy 1, policy_version 81940 (0.0009) [2023-10-10 19:47:12,634][123614] Updated weights for policy 1, policy_version 81950 (0.0009) [2023-10-10 19:47:13,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 167936000. Throughput: 0: 1822.8, 1: 1814.1. Samples: 41987924. Policy #0 lag: (min: 1.0, avg: 28.8, max: 32.0) [2023-10-10 19:47:13,788][122664] Avg episode reward: [(0, '83.980'), (1, '98.240')] [2023-10-10 19:47:14,319][123582] Updated weights for policy 0, policy_version 82053 (0.0010) [2023-10-10 19:47:14,693][123582] Updated weights for policy 0, policy_version 82063 (0.0010) [2023-10-10 19:47:15,061][123582] Updated weights for policy 0, policy_version 82073 (0.0008) [2023-10-10 19:47:16,348][123614] Updated weights for policy 1, policy_version 81960 (0.0008) [2023-10-10 19:47:16,731][123614] Updated weights for policy 1, policy_version 81970 (0.0009) [2023-10-10 19:47:17,093][123614] Updated weights for policy 1, policy_version 81980 (0.0010) [2023-10-10 19:47:18,763][123582] Updated weights for policy 0, policy_version 82083 (0.0009) [2023-10-10 19:47:18,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 168001536. Throughput: 0: 1821.8, 1: 1820.8. Samples: 42009548. Policy #0 lag: (min: 1.0, avg: 28.8, max: 32.0) [2023-10-10 19:47:18,789][122664] Avg episode reward: [(0, '85.970'), (1, '96.600')] [2023-10-10 19:47:19,140][123582] Updated weights for policy 0, policy_version 82093 (0.0008) [2023-10-10 19:47:19,519][123582] Updated weights for policy 0, policy_version 82103 (0.0008) [2023-10-10 19:47:20,984][123614] Updated weights for policy 1, policy_version 81990 (0.0011) [2023-10-10 19:47:21,363][123614] Updated weights for policy 1, policy_version 82000 (0.0008) [2023-10-10 19:47:21,730][123614] Updated weights for policy 1, policy_version 82010 (0.0008) [2023-10-10 19:47:23,105][123582] Updated weights for policy 0, policy_version 82113 (0.0008) [2023-10-10 19:47:23,476][123582] Updated weights for policy 0, policy_version 82123 (0.0007) [2023-10-10 19:47:23,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 168067072. Throughput: 0: 1815.9, 1: 1813.7. Samples: 42031896. Policy #0 lag: (min: 1.0, avg: 28.8, max: 32.0) [2023-10-10 19:47:23,789][122664] Avg episode reward: [(0, '85.260'), (1, '97.670')] [2023-10-10 19:47:23,852][123582] Updated weights for policy 0, policy_version 82133 (0.0008) [2023-10-10 19:47:24,220][123582] Updated weights for policy 0, policy_version 82143 (0.0008) [2023-10-10 19:47:25,341][123614] Updated weights for policy 1, policy_version 82020 (0.0008) [2023-10-10 19:47:25,713][123614] Updated weights for policy 1, policy_version 82030 (0.0008) [2023-10-10 19:47:26,075][123614] Updated weights for policy 1, policy_version 82040 (0.0009) [2023-10-10 19:47:27,968][123582] Updated weights for policy 0, policy_version 82153 (0.0007) [2023-10-10 19:47:28,344][123582] Updated weights for policy 0, policy_version 82163 (0.0008) [2023-10-10 19:47:28,715][123582] Updated weights for policy 0, policy_version 82173 (0.0008) [2023-10-10 19:47:28,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 168132608. Throughput: 0: 1816.8, 1: 1814.2. Samples: 42042182. Policy #0 lag: (min: 1.0, avg: 28.8, max: 32.0) [2023-10-10 19:47:28,789][122664] Avg episode reward: [(0, '85.270'), (1, '96.880')] [2023-10-10 19:47:29,859][123614] Updated weights for policy 1, policy_version 82050 (0.0009) [2023-10-10 19:47:30,234][123614] Updated weights for policy 1, policy_version 82060 (0.0010) [2023-10-10 19:47:30,597][123614] Updated weights for policy 1, policy_version 82070 (0.0008) [2023-10-10 19:47:30,970][123614] Updated weights for policy 1, policy_version 82080 (0.0009) [2023-10-10 19:47:32,421][123582] Updated weights for policy 0, policy_version 82183 (0.0007) [2023-10-10 19:47:32,795][123582] Updated weights for policy 0, policy_version 82193 (0.0007) [2023-10-10 19:47:33,173][123582] Updated weights for policy 0, policy_version 82203 (0.0009) [2023-10-10 19:47:33,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 168230912. Throughput: 0: 1815.2, 1: 1806.2. Samples: 42064532. Policy #0 lag: (min: 1.0, avg: 28.8, max: 32.0) [2023-10-10 19:47:33,789][122664] Avg episode reward: [(0, '84.730'), (1, '93.410')] [2023-10-10 19:47:34,624][123614] Updated weights for policy 1, policy_version 82090 (0.0008) [2023-10-10 19:47:34,984][123614] Updated weights for policy 1, policy_version 82100 (0.0009) [2023-10-10 19:47:35,358][123614] Updated weights for policy 1, policy_version 82110 (0.0010) [2023-10-10 19:47:36,843][123582] Updated weights for policy 0, policy_version 82213 (0.0009) [2023-10-10 19:47:37,207][123582] Updated weights for policy 0, policy_version 82223 (0.0010) [2023-10-10 19:47:37,582][123582] Updated weights for policy 0, policy_version 82233 (0.0010) [2023-10-10 19:47:38,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 168296448. Throughput: 0: 1813.1, 1: 1813.8. Samples: 42086168. Policy #0 lag: (min: 29.0, avg: 37.0, max: 61.0) [2023-10-10 19:47:38,788][122664] Avg episode reward: [(0, '86.120'), (1, '85.880')] [2023-10-10 19:47:39,046][123614] Updated weights for policy 1, policy_version 82120 (0.0008) [2023-10-10 19:47:39,425][123614] Updated weights for policy 1, policy_version 82130 (0.0007) [2023-10-10 19:47:39,789][123614] Updated weights for policy 1, policy_version 82140 (0.0009) [2023-10-10 19:47:41,484][123582] Updated weights for policy 0, policy_version 82243 (0.0009) [2023-10-10 19:47:41,849][123582] Updated weights for policy 0, policy_version 82253 (0.0010) [2023-10-10 19:47:42,220][123582] Updated weights for policy 0, policy_version 82263 (0.0008) [2023-10-10 19:47:43,420][123614] Updated weights for policy 1, policy_version 82150 (0.0007) [2023-10-10 19:47:43,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 168361984. Throughput: 0: 1811.7, 1: 1814.3. Samples: 42097380. Policy #0 lag: (min: 29.0, avg: 37.0, max: 61.0) [2023-10-10 19:47:43,788][122664] Avg episode reward: [(0, '88.820'), (1, '85.210')] [2023-10-10 19:47:43,792][123614] Updated weights for policy 1, policy_version 82160 (0.0007) [2023-10-10 19:47:44,162][123614] Updated weights for policy 1, policy_version 82170 (0.0007) [2023-10-10 19:47:46,128][123582] Updated weights for policy 0, policy_version 82273 (0.0007) [2023-10-10 19:47:46,493][123582] Updated weights for policy 0, policy_version 82283 (0.0010) [2023-10-10 19:47:46,876][123582] Updated weights for policy 0, policy_version 82293 (0.0012) [2023-10-10 19:47:47,241][123582] Updated weights for policy 0, policy_version 82303 (0.0010) [2023-10-10 19:47:47,902][123614] Updated weights for policy 1, policy_version 82180 (0.0008) [2023-10-10 19:47:48,266][123614] Updated weights for policy 1, policy_version 82190 (0.0007) [2023-10-10 19:47:48,645][123614] Updated weights for policy 1, policy_version 82200 (0.0009) [2023-10-10 19:47:48,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 168427520. Throughput: 0: 1802.5, 1: 1810.5. Samples: 42118420. Policy #0 lag: (min: 29.0, avg: 37.0, max: 61.0) [2023-10-10 19:47:48,789][122664] Avg episode reward: [(0, '96.400'), (1, '80.390')] [2023-10-10 19:47:51,112][123582] Updated weights for policy 0, policy_version 82313 (0.0009) [2023-10-10 19:47:51,490][123582] Updated weights for policy 0, policy_version 82323 (0.0007) [2023-10-10 19:47:51,852][123582] Updated weights for policy 0, policy_version 82333 (0.0010) [2023-10-10 19:47:52,229][123614] Updated weights for policy 1, policy_version 82210 (0.0007) [2023-10-10 19:47:52,595][123614] Updated weights for policy 1, policy_version 82220 (0.0007) [2023-10-10 19:47:52,964][123614] Updated weights for policy 1, policy_version 82230 (0.0008) [2023-10-10 19:47:53,326][123614] Updated weights for policy 1, policy_version 82240 (0.0007) [2023-10-10 19:47:53,788][122664] Fps is (10 sec: 16383.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 168525824. Throughput: 0: 1800.0, 1: 1816.6. Samples: 42139646. Policy #0 lag: (min: 29.0, avg: 37.0, max: 61.0) [2023-10-10 19:47:53,789][122664] Avg episode reward: [(0, '98.190'), (1, '80.170')] [2023-10-10 19:47:53,802][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000082240_84213760.pth... [2023-10-10 19:47:53,802][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000082336_84312064.pth... [2023-10-10 19:47:53,838][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000080640_82575360.pth [2023-10-10 19:47:53,839][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000080544_82477056.pth [2023-10-10 19:47:55,539][123582] Updated weights for policy 0, policy_version 82343 (0.0007) [2023-10-10 19:47:55,913][123582] Updated weights for policy 0, policy_version 82353 (0.0008) [2023-10-10 19:47:56,300][123582] Updated weights for policy 0, policy_version 82363 (0.0010) [2023-10-10 19:47:56,960][123614] Updated weights for policy 1, policy_version 82250 (0.0008) [2023-10-10 19:47:57,315][123614] Updated weights for policy 1, policy_version 82260 (0.0011) [2023-10-10 19:47:57,683][123614] Updated weights for policy 1, policy_version 82270 (0.0010) [2023-10-10 19:47:58,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 168591360. Throughput: 0: 1804.0, 1: 1825.0. Samples: 42151232. Policy #0 lag: (min: 29.0, avg: 37.0, max: 61.0) [2023-10-10 19:47:58,789][122664] Avg episode reward: [(0, '99.350'), (1, '77.300')] [2023-10-10 19:48:00,000][123582] Updated weights for policy 0, policy_version 82373 (0.0009) [2023-10-10 19:48:00,382][123582] Updated weights for policy 0, policy_version 82383 (0.0010) [2023-10-10 19:48:00,752][123582] Updated weights for policy 0, policy_version 82393 (0.0011) [2023-10-10 19:48:01,471][123614] Updated weights for policy 1, policy_version 82280 (0.0010) [2023-10-10 19:48:01,839][123614] Updated weights for policy 1, policy_version 82290 (0.0008) [2023-10-10 19:48:02,210][123614] Updated weights for policy 1, policy_version 82300 (0.0012) [2023-10-10 19:48:03,788][122664] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 168656896. Throughput: 0: 1798.0, 1: 1818.9. Samples: 42172306. Policy #0 lag: (min: 29.0, avg: 37.0, max: 61.0) [2023-10-10 19:48:03,789][122664] Avg episode reward: [(0, '106.810'), (1, '80.570')] [2023-10-10 19:48:03,790][123247] Saving new best policy, reward=106.810! [2023-10-10 19:48:04,484][123582] Updated weights for policy 0, policy_version 82403 (0.0009) [2023-10-10 19:48:04,851][123582] Updated weights for policy 0, policy_version 82413 (0.0011) [2023-10-10 19:48:05,222][123582] Updated weights for policy 0, policy_version 82423 (0.0009) [2023-10-10 19:48:06,019][123614] Updated weights for policy 1, policy_version 82310 (0.0008) [2023-10-10 19:48:06,393][123614] Updated weights for policy 1, policy_version 82320 (0.0011) [2023-10-10 19:48:06,757][123614] Updated weights for policy 1, policy_version 82330 (0.0008) [2023-10-10 19:48:08,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 168722432. Throughput: 0: 1805.9, 1: 1820.0. Samples: 42195058. Policy #0 lag: (min: 29.0, avg: 37.0, max: 61.0) [2023-10-10 19:48:08,789][122664] Avg episode reward: [(0, '105.220'), (1, '79.890')] [2023-10-10 19:48:08,894][123582] Updated weights for policy 0, policy_version 82433 (0.0010) [2023-10-10 19:48:09,281][123582] Updated weights for policy 0, policy_version 82443 (0.0008) [2023-10-10 19:48:09,656][123582] Updated weights for policy 0, policy_version 82453 (0.0008) [2023-10-10 19:48:10,033][123582] Updated weights for policy 0, policy_version 82463 (0.0008) [2023-10-10 19:48:10,509][123614] Updated weights for policy 1, policy_version 82340 (0.0009) [2023-10-10 19:48:10,879][123614] Updated weights for policy 1, policy_version 82350 (0.0007) [2023-10-10 19:48:11,253][123614] Updated weights for policy 1, policy_version 82360 (0.0007) [2023-10-10 19:48:13,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 168787968. Throughput: 0: 1794.7, 1: 1816.1. Samples: 42204668. Policy #0 lag: (min: 29.0, avg: 37.0, max: 61.0) [2023-10-10 19:48:13,789][122664] Avg episode reward: [(0, '111.040'), (1, '76.110')] [2023-10-10 19:48:13,875][123582] Updated weights for policy 0, policy_version 82473 (0.0009) [2023-10-10 19:48:14,246][123582] Updated weights for policy 0, policy_version 82483 (0.0008) [2023-10-10 19:48:14,617][123582] Updated weights for policy 0, policy_version 82493 (0.0010) [2023-10-10 19:48:14,727][123247] Saving new best policy, reward=111.040! [2023-10-10 19:48:14,915][123614] Updated weights for policy 1, policy_version 82370 (0.0007) [2023-10-10 19:48:15,283][123614] Updated weights for policy 1, policy_version 82380 (0.0007) [2023-10-10 19:48:15,660][123614] Updated weights for policy 1, policy_version 82390 (0.0008) [2023-10-10 19:48:16,031][123614] Updated weights for policy 1, policy_version 82400 (0.0009) [2023-10-10 19:48:18,395][123582] Updated weights for policy 0, policy_version 82503 (0.0008) [2023-10-10 19:48:18,773][123582] Updated weights for policy 0, policy_version 82513 (0.0010) [2023-10-10 19:48:18,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 168853504. Throughput: 0: 1796.8, 1: 1819.0. Samples: 42227242. Policy #0 lag: (min: 29.0, avg: 37.0, max: 61.0) [2023-10-10 19:48:18,788][122664] Avg episode reward: [(0, '110.010'), (1, '79.410')] [2023-10-10 19:48:19,144][123582] Updated weights for policy 0, policy_version 82523 (0.0008) [2023-10-10 19:48:19,599][123614] Updated weights for policy 1, policy_version 82410 (0.0008) [2023-10-10 19:48:19,960][123614] Updated weights for policy 1, policy_version 82420 (0.0009) [2023-10-10 19:48:20,331][123614] Updated weights for policy 1, policy_version 82430 (0.0011) [2023-10-10 19:48:22,677][123582] Updated weights for policy 0, policy_version 82533 (0.0009) [2023-10-10 19:48:23,051][123582] Updated weights for policy 0, policy_version 82543 (0.0010) [2023-10-10 19:48:23,422][123582] Updated weights for policy 0, policy_version 82553 (0.0007) [2023-10-10 19:48:23,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 168951808. Throughput: 0: 1801.9, 1: 1815.6. Samples: 42248954. Policy #0 lag: (min: 29.0, avg: 37.0, max: 61.0) [2023-10-10 19:48:23,789][122664] Avg episode reward: [(0, '104.890'), (1, '81.020')] [2023-10-10 19:48:23,928][123614] Updated weights for policy 1, policy_version 82440 (0.0008) [2023-10-10 19:48:24,294][123614] Updated weights for policy 1, policy_version 82450 (0.0007) [2023-10-10 19:48:24,661][123614] Updated weights for policy 1, policy_version 82460 (0.0008) [2023-10-10 19:48:27,086][123582] Updated weights for policy 0, policy_version 82563 (0.0008) [2023-10-10 19:48:27,461][123582] Updated weights for policy 0, policy_version 82573 (0.0010) [2023-10-10 19:48:27,843][123582] Updated weights for policy 0, policy_version 82583 (0.0011) [2023-10-10 19:48:28,257][123614] Updated weights for policy 1, policy_version 82470 (0.0009) [2023-10-10 19:48:28,617][123614] Updated weights for policy 1, policy_version 82480 (0.0010) [2023-10-10 19:48:28,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 169017344. Throughput: 0: 1803.8, 1: 1817.4. Samples: 42260336. Policy #0 lag: (min: 29.0, avg: 37.0, max: 61.0) [2023-10-10 19:48:28,788][122664] Avg episode reward: [(0, '103.960'), (1, '78.240')] [2023-10-10 19:48:28,987][123614] Updated weights for policy 1, policy_version 82490 (0.0011) [2023-10-10 19:48:31,593][123582] Updated weights for policy 0, policy_version 82593 (0.0008) [2023-10-10 19:48:31,967][123582] Updated weights for policy 0, policy_version 82603 (0.0009) [2023-10-10 19:48:32,344][123582] Updated weights for policy 0, policy_version 82613 (0.0008) [2023-10-10 19:48:32,630][123614] Updated weights for policy 1, policy_version 82500 (0.0010) [2023-10-10 19:48:32,717][123582] Updated weights for policy 0, policy_version 82623 (0.0008) [2023-10-10 19:48:32,996][123614] Updated weights for policy 1, policy_version 82510 (0.0008) [2023-10-10 19:48:33,363][123614] Updated weights for policy 1, policy_version 82520 (0.0009) [2023-10-10 19:48:33,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 169115648. Throughput: 0: 1812.1, 1: 1819.7. Samples: 42281852. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:48:33,788][122664] Avg episode reward: [(0, '102.270'), (1, '78.890')] [2023-10-10 19:48:36,497][123582] Updated weights for policy 0, policy_version 82633 (0.0008) [2023-10-10 19:48:36,867][123582] Updated weights for policy 0, policy_version 82643 (0.0009) [2023-10-10 19:48:36,966][123614] Updated weights for policy 1, policy_version 82530 (0.0008) [2023-10-10 19:48:37,234][123582] Updated weights for policy 0, policy_version 82653 (0.0008) [2023-10-10 19:48:37,326][123614] Updated weights for policy 1, policy_version 82540 (0.0008) [2023-10-10 19:48:37,705][123614] Updated weights for policy 1, policy_version 82550 (0.0009) [2023-10-10 19:48:38,070][123614] Updated weights for policy 1, policy_version 82560 (0.0009) [2023-10-10 19:48:38,788][122664] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 169181184. Throughput: 0: 1802.7, 1: 1824.2. Samples: 42302856. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:48:38,789][122664] Avg episode reward: [(0, '101.620'), (1, '74.320')] [2023-10-10 19:48:40,854][123582] Updated weights for policy 0, policy_version 82663 (0.0010) [2023-10-10 19:48:41,229][123582] Updated weights for policy 0, policy_version 82673 (0.0012) [2023-10-10 19:48:41,607][123582] Updated weights for policy 0, policy_version 82683 (0.0008) [2023-10-10 19:48:41,855][123614] Updated weights for policy 1, policy_version 82570 (0.0007) [2023-10-10 19:48:42,231][123614] Updated weights for policy 1, policy_version 82580 (0.0008) [2023-10-10 19:48:42,607][123614] Updated weights for policy 1, policy_version 82590 (0.0010) [2023-10-10 19:48:43,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 169246720. Throughput: 0: 1809.4, 1: 1815.0. Samples: 42314330. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:48:43,789][122664] Avg episode reward: [(0, '100.570'), (1, '76.450')] [2023-10-10 19:48:45,437][123582] Updated weights for policy 0, policy_version 82693 (0.0010) [2023-10-10 19:48:45,807][123582] Updated weights for policy 0, policy_version 82703 (0.0007) [2023-10-10 19:48:46,191][123582] Updated weights for policy 0, policy_version 82713 (0.0008) [2023-10-10 19:48:46,377][123614] Updated weights for policy 1, policy_version 82600 (0.0007) [2023-10-10 19:48:46,749][123614] Updated weights for policy 1, policy_version 82610 (0.0007) [2023-10-10 19:48:47,120][123614] Updated weights for policy 1, policy_version 82620 (0.0010) [2023-10-10 19:48:48,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 169312256. Throughput: 0: 1798.1, 1: 1819.9. Samples: 42335116. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:48:48,789][122664] Avg episode reward: [(0, '98.730'), (1, '79.160')] [2023-10-10 19:48:49,808][123582] Updated weights for policy 0, policy_version 82723 (0.0008) [2023-10-10 19:48:50,170][123582] Updated weights for policy 0, policy_version 82733 (0.0008) [2023-10-10 19:48:50,534][123582] Updated weights for policy 0, policy_version 82743 (0.0008) [2023-10-10 19:48:50,757][123614] Updated weights for policy 1, policy_version 82630 (0.0009) [2023-10-10 19:48:51,125][123614] Updated weights for policy 1, policy_version 82640 (0.0007) [2023-10-10 19:48:51,503][123614] Updated weights for policy 1, policy_version 82650 (0.0007) [2023-10-10 19:48:53,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 169377792. Throughput: 0: 1795.5, 1: 1826.1. Samples: 42358030. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:48:53,789][122664] Avg episode reward: [(0, '104.440'), (1, '78.810')] [2023-10-10 19:48:54,217][123582] Updated weights for policy 0, policy_version 82753 (0.0008) [2023-10-10 19:48:54,588][123582] Updated weights for policy 0, policy_version 82763 (0.0007) [2023-10-10 19:48:54,966][123582] Updated weights for policy 0, policy_version 82773 (0.0008) [2023-10-10 19:48:55,258][123614] Updated weights for policy 1, policy_version 82660 (0.0007) [2023-10-10 19:48:55,337][123582] Updated weights for policy 0, policy_version 82783 (0.0007) [2023-10-10 19:48:55,627][123614] Updated weights for policy 1, policy_version 82670 (0.0008) [2023-10-10 19:48:55,989][123614] Updated weights for policy 1, policy_version 82680 (0.0008) [2023-10-10 19:48:58,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 169443328. Throughput: 0: 1800.8, 1: 1828.5. Samples: 42367984. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:48:58,789][122664] Avg episode reward: [(0, '104.010'), (1, '79.920')] [2023-10-10 19:48:58,930][123582] Updated weights for policy 0, policy_version 82793 (0.0008) [2023-10-10 19:48:59,296][123582] Updated weights for policy 0, policy_version 82803 (0.0007) [2023-10-10 19:48:59,675][123582] Updated weights for policy 0, policy_version 82813 (0.0008) [2023-10-10 19:48:59,682][123614] Updated weights for policy 1, policy_version 82690 (0.0007) [2023-10-10 19:49:00,058][123614] Updated weights for policy 1, policy_version 82700 (0.0008) [2023-10-10 19:49:00,422][123614] Updated weights for policy 1, policy_version 82710 (0.0007) [2023-10-10 19:49:00,794][123614] Updated weights for policy 1, policy_version 82720 (0.0009) [2023-10-10 19:49:03,247][123582] Updated weights for policy 0, policy_version 82823 (0.0009) [2023-10-10 19:49:03,615][123582] Updated weights for policy 0, policy_version 82833 (0.0007) [2023-10-10 19:49:03,788][122664] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 169508864. Throughput: 0: 1806.7, 1: 1832.8. Samples: 42391020. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:49:03,788][122664] Avg episode reward: [(0, '99.820'), (1, '83.690')] [2023-10-10 19:49:03,992][123582] Updated weights for policy 0, policy_version 82843 (0.0010) [2023-10-10 19:49:04,407][123614] Updated weights for policy 1, policy_version 82730 (0.0009) [2023-10-10 19:49:04,779][123614] Updated weights for policy 1, policy_version 82740 (0.0007) [2023-10-10 19:49:05,153][123614] Updated weights for policy 1, policy_version 82750 (0.0008) [2023-10-10 19:49:07,818][123582] Updated weights for policy 0, policy_version 82853 (0.0008) [2023-10-10 19:49:08,181][123582] Updated weights for policy 0, policy_version 82863 (0.0007) [2023-10-10 19:49:08,558][123582] Updated weights for policy 0, policy_version 82873 (0.0007) [2023-10-10 19:49:08,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 169574400. Throughput: 0: 1809.3, 1: 1826.6. Samples: 42412570. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:49:08,788][122664] Avg episode reward: [(0, '95.770'), (1, '85.390')] [2023-10-10 19:49:08,794][123614] Updated weights for policy 1, policy_version 82760 (0.0007) [2023-10-10 19:49:09,165][123614] Updated weights for policy 1, policy_version 82770 (0.0007) [2023-10-10 19:49:09,540][123614] Updated weights for policy 1, policy_version 82780 (0.0010) [2023-10-10 19:49:12,192][123582] Updated weights for policy 0, policy_version 82883 (0.0008) [2023-10-10 19:49:12,573][123582] Updated weights for policy 0, policy_version 82893 (0.0007) [2023-10-10 19:49:12,949][123582] Updated weights for policy 0, policy_version 82903 (0.0008) [2023-10-10 19:49:13,092][123614] Updated weights for policy 1, policy_version 82790 (0.0008) [2023-10-10 19:49:13,463][123614] Updated weights for policy 1, policy_version 82800 (0.0008) [2023-10-10 19:49:13,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 169672704. Throughput: 0: 1807.0, 1: 1831.6. Samples: 42424072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:49:13,789][122664] Avg episode reward: [(0, '94.050'), (1, '83.040')] [2023-10-10 19:49:13,829][123614] Updated weights for policy 1, policy_version 82810 (0.0008) [2023-10-10 19:49:16,676][123582] Updated weights for policy 0, policy_version 82913 (0.0008) [2023-10-10 19:49:17,057][123582] Updated weights for policy 0, policy_version 82923 (0.0008) [2023-10-10 19:49:17,427][123582] Updated weights for policy 0, policy_version 82933 (0.0007) [2023-10-10 19:49:17,603][123614] Updated weights for policy 1, policy_version 82820 (0.0008) [2023-10-10 19:49:17,794][123582] Updated weights for policy 0, policy_version 82943 (0.0007) [2023-10-10 19:49:17,969][123614] Updated weights for policy 1, policy_version 82830 (0.0009) [2023-10-10 19:49:18,335][123614] Updated weights for policy 1, policy_version 82840 (0.0009) [2023-10-10 19:49:18,788][122664] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 169771008. Throughput: 0: 1811.9, 1: 1824.2. Samples: 42445474. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:49:18,789][122664] Avg episode reward: [(0, '91.140'), (1, '85.840')] [2023-10-10 19:49:21,534][123582] Updated weights for policy 0, policy_version 82953 (0.0009) [2023-10-10 19:49:21,907][123582] Updated weights for policy 0, policy_version 82963 (0.0008) [2023-10-10 19:49:22,070][123614] Updated weights for policy 1, policy_version 82850 (0.0009) [2023-10-10 19:49:22,283][123582] Updated weights for policy 0, policy_version 82973 (0.0008) [2023-10-10 19:49:22,437][123614] Updated weights for policy 1, policy_version 82860 (0.0007) [2023-10-10 19:49:22,807][123614] Updated weights for policy 1, policy_version 82870 (0.0007) [2023-10-10 19:49:23,172][123614] Updated weights for policy 1, policy_version 82880 (0.0007) [2023-10-10 19:49:23,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 169836544. Throughput: 0: 1809.6, 1: 1819.3. Samples: 42466158. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:49:23,789][122664] Avg episode reward: [(0, '80.680'), (1, '81.560')] [2023-10-10 19:49:25,978][123582] Updated weights for policy 0, policy_version 82983 (0.0009) [2023-10-10 19:49:26,347][123582] Updated weights for policy 0, policy_version 82993 (0.0009) [2023-10-10 19:49:26,719][123582] Updated weights for policy 0, policy_version 83003 (0.0007) [2023-10-10 19:49:26,937][123614] Updated weights for policy 1, policy_version 82890 (0.0009) [2023-10-10 19:49:27,309][123614] Updated weights for policy 1, policy_version 82900 (0.0009) [2023-10-10 19:49:27,671][123614] Updated weights for policy 1, policy_version 82910 (0.0009) [2023-10-10 19:49:28,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 169902080. Throughput: 0: 1811.5, 1: 1821.9. Samples: 42477834. Policy #0 lag: (min: 28.0, avg: 31.1, max: 60.0) [2023-10-10 19:49:28,788][122664] Avg episode reward: [(0, '80.710'), (1, '81.730')] [2023-10-10 19:49:30,522][123582] Updated weights for policy 0, policy_version 83013 (0.0007) [2023-10-10 19:49:30,891][123582] Updated weights for policy 0, policy_version 83023 (0.0008) [2023-10-10 19:49:31,273][123582] Updated weights for policy 0, policy_version 83033 (0.0008) [2023-10-10 19:49:31,639][123614] Updated weights for policy 1, policy_version 82920 (0.0009) [2023-10-10 19:49:32,000][123614] Updated weights for policy 1, policy_version 82930 (0.0008) [2023-10-10 19:49:32,361][123614] Updated weights for policy 1, policy_version 82940 (0.0009) [2023-10-10 19:49:33,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 169967616. Throughput: 0: 1817.2, 1: 1811.4. Samples: 42498400. Policy #0 lag: (min: 28.0, avg: 31.1, max: 60.0) [2023-10-10 19:49:33,789][122664] Avg episode reward: [(0, '89.790'), (1, '76.320')] [2023-10-10 19:49:35,036][123582] Updated weights for policy 0, policy_version 83043 (0.0009) [2023-10-10 19:49:35,399][123582] Updated weights for policy 0, policy_version 83053 (0.0007) [2023-10-10 19:49:35,768][123582] Updated weights for policy 0, policy_version 83063 (0.0010) [2023-10-10 19:49:36,118][123614] Updated weights for policy 1, policy_version 82950 (0.0009) [2023-10-10 19:49:36,501][123614] Updated weights for policy 1, policy_version 82960 (0.0007) [2023-10-10 19:49:36,869][123614] Updated weights for policy 1, policy_version 82970 (0.0009) [2023-10-10 19:49:38,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 170033152. Throughput: 0: 1814.0, 1: 1803.2. Samples: 42520806. Policy #0 lag: (min: 28.0, avg: 31.1, max: 60.0) [2023-10-10 19:49:38,789][122664] Avg episode reward: [(0, '91.840'), (1, '72.930')] [2023-10-10 19:49:39,527][123582] Updated weights for policy 0, policy_version 83073 (0.0007) [2023-10-10 19:49:39,893][123582] Updated weights for policy 0, policy_version 83083 (0.0010) [2023-10-10 19:49:40,259][123582] Updated weights for policy 0, policy_version 83093 (0.0010) [2023-10-10 19:49:40,631][123614] Updated weights for policy 1, policy_version 82980 (0.0008) [2023-10-10 19:49:40,633][123582] Updated weights for policy 0, policy_version 83103 (0.0008) [2023-10-10 19:49:40,996][123614] Updated weights for policy 1, policy_version 82990 (0.0009) [2023-10-10 19:49:41,364][123614] Updated weights for policy 1, policy_version 83000 (0.0008) [2023-10-10 19:49:43,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 170098688. Throughput: 0: 1811.2, 1: 1804.9. Samples: 42530708. Policy #0 lag: (min: 28.0, avg: 31.1, max: 60.0) [2023-10-10 19:49:43,789][122664] Avg episode reward: [(0, '89.130'), (1, '74.640')] [2023-10-10 19:49:44,241][123582] Updated weights for policy 0, policy_version 83113 (0.0009) [2023-10-10 19:49:44,609][123582] Updated weights for policy 0, policy_version 83123 (0.0009) [2023-10-10 19:49:44,985][123582] Updated weights for policy 0, policy_version 83133 (0.0008) [2023-10-10 19:49:45,031][123614] Updated weights for policy 1, policy_version 83010 (0.0008) [2023-10-10 19:49:45,395][123614] Updated weights for policy 1, policy_version 83020 (0.0009) [2023-10-10 19:49:45,760][123614] Updated weights for policy 1, policy_version 83030 (0.0010) [2023-10-10 19:49:46,130][123614] Updated weights for policy 1, policy_version 83040 (0.0009) [2023-10-10 19:49:48,671][123582] Updated weights for policy 0, policy_version 83143 (0.0007) [2023-10-10 19:49:48,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 170164224. Throughput: 0: 1819.9, 1: 1794.9. Samples: 42553688. Policy #0 lag: (min: 28.0, avg: 31.1, max: 60.0) [2023-10-10 19:49:48,789][122664] Avg episode reward: [(0, '88.840'), (1, '74.520')] [2023-10-10 19:49:49,048][123582] Updated weights for policy 0, policy_version 83153 (0.0009) [2023-10-10 19:49:49,417][123582] Updated weights for policy 0, policy_version 83163 (0.0008) [2023-10-10 19:49:49,906][123614] Updated weights for policy 1, policy_version 83050 (0.0010) [2023-10-10 19:49:50,265][123614] Updated weights for policy 1, policy_version 83060 (0.0010) [2023-10-10 19:49:50,630][123614] Updated weights for policy 1, policy_version 83070 (0.0010) [2023-10-10 19:49:53,211][123582] Updated weights for policy 0, policy_version 83173 (0.0007) [2023-10-10 19:49:53,590][123582] Updated weights for policy 0, policy_version 83183 (0.0007) [2023-10-10 19:49:53,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 170229760. Throughput: 0: 1826.9, 1: 1803.0. Samples: 42575918. Policy #0 lag: (min: 28.0, avg: 31.1, max: 60.0) [2023-10-10 19:49:53,788][122664] Avg episode reward: [(0, '88.570'), (1, '76.530')] [2023-10-10 19:49:53,796][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000083072_85065728.pth... [2023-10-10 19:49:53,832][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000081376_83329024.pth [2023-10-10 19:49:53,961][123582] Updated weights for policy 0, policy_version 83193 (0.0007) [2023-10-10 19:49:54,212][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000083200_85196800.pth... [2023-10-10 19:49:54,251][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000081504_83460096.pth [2023-10-10 19:49:54,452][123614] Updated weights for policy 1, policy_version 83080 (0.0009) [2023-10-10 19:49:54,819][123614] Updated weights for policy 1, policy_version 83090 (0.0009) [2023-10-10 19:49:55,198][123614] Updated weights for policy 1, policy_version 83100 (0.0008) [2023-10-10 19:49:57,592][123582] Updated weights for policy 0, policy_version 83203 (0.0007) [2023-10-10 19:49:57,961][123582] Updated weights for policy 0, policy_version 83213 (0.0007) [2023-10-10 19:49:58,337][123582] Updated weights for policy 0, policy_version 83223 (0.0008) [2023-10-10 19:49:58,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 170328064. Throughput: 0: 1814.7, 1: 1793.2. Samples: 42586428. Policy #0 lag: (min: 28.0, avg: 31.1, max: 60.0) [2023-10-10 19:49:58,788][122664] Avg episode reward: [(0, '92.440'), (1, '76.120')] [2023-10-10 19:49:58,915][123614] Updated weights for policy 1, policy_version 83110 (0.0007) [2023-10-10 19:49:59,275][123614] Updated weights for policy 1, policy_version 83120 (0.0007) [2023-10-10 19:49:59,661][123614] Updated weights for policy 1, policy_version 83130 (0.0007) [2023-10-10 19:50:01,918][123582] Updated weights for policy 0, policy_version 83233 (0.0008) [2023-10-10 19:50:02,287][123582] Updated weights for policy 0, policy_version 83243 (0.0007) [2023-10-10 19:50:02,659][123582] Updated weights for policy 0, policy_version 83253 (0.0008) [2023-10-10 19:50:03,023][123582] Updated weights for policy 0, policy_version 83263 (0.0009) [2023-10-10 19:50:03,429][123614] Updated weights for policy 1, policy_version 83140 (0.0010) [2023-10-10 19:50:03,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 170393600. Throughput: 0: 1825.5, 1: 1796.7. Samples: 42608472. Policy #0 lag: (min: 28.0, avg: 31.1, max: 60.0) [2023-10-10 19:50:03,788][122664] Avg episode reward: [(0, '91.290'), (1, '75.640')] [2023-10-10 19:50:03,798][123614] Updated weights for policy 1, policy_version 83150 (0.0007) [2023-10-10 19:50:04,173][123614] Updated weights for policy 1, policy_version 83160 (0.0008) [2023-10-10 19:50:06,768][123582] Updated weights for policy 0, policy_version 83273 (0.0009) [2023-10-10 19:50:07,149][123582] Updated weights for policy 0, policy_version 83283 (0.0010) [2023-10-10 19:50:07,518][123582] Updated weights for policy 0, policy_version 83293 (0.0011) [2023-10-10 19:50:07,679][123614] Updated weights for policy 1, policy_version 83170 (0.0009) [2023-10-10 19:50:08,055][123614] Updated weights for policy 1, policy_version 83180 (0.0009) [2023-10-10 19:50:08,411][123614] Updated weights for policy 1, policy_version 83190 (0.0010) [2023-10-10 19:50:08,784][123614] Updated weights for policy 1, policy_version 83200 (0.0009) [2023-10-10 19:50:08,788][122664] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 170491904. Throughput: 0: 1825.3, 1: 1798.6. Samples: 42629234. Policy #0 lag: (min: 28.0, avg: 31.1, max: 60.0) [2023-10-10 19:50:08,788][122664] Avg episode reward: [(0, '94.740'), (1, '75.990')] [2023-10-10 19:50:11,129][123582] Updated weights for policy 0, policy_version 83303 (0.0009) [2023-10-10 19:50:11,508][123582] Updated weights for policy 0, policy_version 83313 (0.0008) [2023-10-10 19:50:11,871][123582] Updated weights for policy 0, policy_version 83323 (0.0007) [2023-10-10 19:50:12,414][123614] Updated weights for policy 1, policy_version 83210 (0.0007) [2023-10-10 19:50:12,775][123614] Updated weights for policy 1, policy_version 83220 (0.0009) [2023-10-10 19:50:13,139][123614] Updated weights for policy 1, policy_version 83230 (0.0008) [2023-10-10 19:50:13,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 170557440. Throughput: 0: 1832.7, 1: 1800.7. Samples: 42641336. Policy #0 lag: (min: 28.0, avg: 31.1, max: 60.0) [2023-10-10 19:50:13,789][122664] Avg episode reward: [(0, '98.050'), (1, '73.150')] [2023-10-10 19:50:15,539][123582] Updated weights for policy 0, policy_version 83333 (0.0008) [2023-10-10 19:50:15,913][123582] Updated weights for policy 0, policy_version 83343 (0.0008) [2023-10-10 19:50:16,284][123582] Updated weights for policy 0, policy_version 83353 (0.0008) [2023-10-10 19:50:16,867][123614] Updated weights for policy 1, policy_version 83240 (0.0007) [2023-10-10 19:50:17,234][123614] Updated weights for policy 1, policy_version 83250 (0.0008) [2023-10-10 19:50:17,608][123614] Updated weights for policy 1, policy_version 83260 (0.0008) [2023-10-10 19:50:18,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 170622976. Throughput: 0: 1828.6, 1: 1807.9. Samples: 42662044. Policy #0 lag: (min: 28.0, avg: 31.1, max: 60.0) [2023-10-10 19:50:18,789][122664] Avg episode reward: [(0, '100.330'), (1, '76.200')] [2023-10-10 19:50:19,741][123582] Updated weights for policy 0, policy_version 83363 (0.0007) [2023-10-10 19:50:20,107][123582] Updated weights for policy 0, policy_version 83373 (0.0007) [2023-10-10 19:50:20,478][123582] Updated weights for policy 0, policy_version 83383 (0.0007) [2023-10-10 19:50:21,478][123614] Updated weights for policy 1, policy_version 83270 (0.0009) [2023-10-10 19:50:21,841][123614] Updated weights for policy 1, policy_version 83280 (0.0010) [2023-10-10 19:50:22,213][123614] Updated weights for policy 1, policy_version 83290 (0.0008) [2023-10-10 19:50:23,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 170688512. Throughput: 0: 1840.0, 1: 1803.6. Samples: 42684768. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:50:23,789][122664] Avg episode reward: [(0, '101.400'), (1, '80.250')] [2023-10-10 19:50:24,039][123582] Updated weights for policy 0, policy_version 83393 (0.0009) [2023-10-10 19:50:24,409][123582] Updated weights for policy 0, policy_version 83403 (0.0008) [2023-10-10 19:50:24,781][123582] Updated weights for policy 0, policy_version 83413 (0.0010) [2023-10-10 19:50:25,143][123582] Updated weights for policy 0, policy_version 83423 (0.0009) [2023-10-10 19:50:26,025][123614] Updated weights for policy 1, policy_version 83300 (0.0008) [2023-10-10 19:50:26,385][123614] Updated weights for policy 1, policy_version 83310 (0.0010) [2023-10-10 19:50:26,754][123614] Updated weights for policy 1, policy_version 83320 (0.0010) [2023-10-10 19:50:28,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 170754048. Throughput: 0: 1844.5, 1: 1813.1. Samples: 42695300. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:50:28,789][122664] Avg episode reward: [(0, '100.060'), (1, '73.780')] [2023-10-10 19:50:28,854][123582] Updated weights for policy 0, policy_version 83433 (0.0010) [2023-10-10 19:50:29,226][123582] Updated weights for policy 0, policy_version 83443 (0.0008) [2023-10-10 19:50:29,582][123582] Updated weights for policy 0, policy_version 83453 (0.0009) [2023-10-10 19:50:30,346][123614] Updated weights for policy 1, policy_version 83330 (0.0009) [2023-10-10 19:50:30,717][123614] Updated weights for policy 1, policy_version 83340 (0.0009) [2023-10-10 19:50:31,083][123614] Updated weights for policy 1, policy_version 83350 (0.0009) [2023-10-10 19:50:31,451][123614] Updated weights for policy 1, policy_version 83360 (0.0009) [2023-10-10 19:50:33,072][123582] Updated weights for policy 0, policy_version 83463 (0.0010) [2023-10-10 19:50:33,442][123582] Updated weights for policy 0, policy_version 83473 (0.0009) [2023-10-10 19:50:33,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 170819584. Throughput: 0: 1835.9, 1: 1803.3. Samples: 42717450. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:50:33,788][122664] Avg episode reward: [(0, '99.010'), (1, '74.330')] [2023-10-10 19:50:33,806][123582] Updated weights for policy 0, policy_version 83483 (0.0007) [2023-10-10 19:50:35,003][123614] Updated weights for policy 1, policy_version 83370 (0.0011) [2023-10-10 19:50:35,365][123614] Updated weights for policy 1, policy_version 83380 (0.0011) [2023-10-10 19:50:35,732][123614] Updated weights for policy 1, policy_version 83390 (0.0011) [2023-10-10 19:50:37,421][123582] Updated weights for policy 0, policy_version 83493 (0.0008) [2023-10-10 19:50:37,801][123582] Updated weights for policy 0, policy_version 83503 (0.0011) [2023-10-10 19:50:38,164][123582] Updated weights for policy 0, policy_version 83513 (0.0011) [2023-10-10 19:50:38,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 170917888. Throughput: 0: 1819.7, 1: 1801.7. Samples: 42738882. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:50:38,788][122664] Avg episode reward: [(0, '99.030'), (1, '74.520')] [2023-10-10 19:50:39,605][123614] Updated weights for policy 1, policy_version 83400 (0.0011) [2023-10-10 19:50:39,978][123614] Updated weights for policy 1, policy_version 83410 (0.0012) [2023-10-10 19:50:40,345][123614] Updated weights for policy 1, policy_version 83420 (0.0011) [2023-10-10 19:50:41,990][123582] Updated weights for policy 0, policy_version 83523 (0.0009) [2023-10-10 19:50:42,359][123582] Updated weights for policy 0, policy_version 83533 (0.0008) [2023-10-10 19:50:42,733][123582] Updated weights for policy 0, policy_version 83543 (0.0009) [2023-10-10 19:50:43,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 170983424. Throughput: 0: 1832.0, 1: 1800.0. Samples: 42749868. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:50:43,788][122664] Avg episode reward: [(0, '102.580'), (1, '73.920')] [2023-10-10 19:50:43,914][123614] Updated weights for policy 1, policy_version 83430 (0.0009) [2023-10-10 19:50:44,285][123614] Updated weights for policy 1, policy_version 83440 (0.0010) [2023-10-10 19:50:44,649][123614] Updated weights for policy 1, policy_version 83450 (0.0008) [2023-10-10 19:50:46,409][123582] Updated weights for policy 0, policy_version 83553 (0.0007) [2023-10-10 19:50:46,782][123582] Updated weights for policy 0, policy_version 83563 (0.0008) [2023-10-10 19:50:47,154][123582] Updated weights for policy 0, policy_version 83573 (0.0007) [2023-10-10 19:50:47,518][123582] Updated weights for policy 0, policy_version 83583 (0.0009) [2023-10-10 19:50:48,565][123614] Updated weights for policy 1, policy_version 83460 (0.0007) [2023-10-10 19:50:48,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 171048960. Throughput: 0: 1819.6, 1: 1805.6. Samples: 42771608. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:50:48,788][122664] Avg episode reward: [(0, '101.890'), (1, '77.880')] [2023-10-10 19:50:48,945][123614] Updated weights for policy 1, policy_version 83470 (0.0007) [2023-10-10 19:50:49,308][123614] Updated weights for policy 1, policy_version 83480 (0.0008) [2023-10-10 19:50:51,245][123582] Updated weights for policy 0, policy_version 83593 (0.0008) [2023-10-10 19:50:51,612][123582] Updated weights for policy 0, policy_version 83603 (0.0008) [2023-10-10 19:50:51,982][123582] Updated weights for policy 0, policy_version 83613 (0.0007) [2023-10-10 19:50:53,115][123614] Updated weights for policy 1, policy_version 83490 (0.0008) [2023-10-10 19:50:53,492][123614] Updated weights for policy 1, policy_version 83500 (0.0007) [2023-10-10 19:50:53,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 171114496. Throughput: 0: 1832.5, 1: 1815.2. Samples: 42793380. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:50:53,789][122664] Avg episode reward: [(0, '100.440'), (1, '79.170')] [2023-10-10 19:50:53,857][123614] Updated weights for policy 1, policy_version 83510 (0.0007) [2023-10-10 19:50:54,230][123614] Updated weights for policy 1, policy_version 83520 (0.0009) [2023-10-10 19:50:55,721][123582] Updated weights for policy 0, policy_version 83623 (0.0007) [2023-10-10 19:50:56,098][123582] Updated weights for policy 0, policy_version 83633 (0.0008) [2023-10-10 19:50:56,468][123582] Updated weights for policy 0, policy_version 83643 (0.0007) [2023-10-10 19:50:58,060][123614] Updated weights for policy 1, policy_version 83530 (0.0008) [2023-10-10 19:50:58,438][123614] Updated weights for policy 1, policy_version 83540 (0.0008) [2023-10-10 19:50:58,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 171180032. Throughput: 0: 1821.9, 1: 1804.3. Samples: 42804514. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:50:58,789][122664] Avg episode reward: [(0, '105.870'), (1, '76.220')] [2023-10-10 19:50:58,801][123614] Updated weights for policy 1, policy_version 83550 (0.0007) [2023-10-10 19:51:00,290][123582] Updated weights for policy 0, policy_version 83653 (0.0010) [2023-10-10 19:51:00,666][123582] Updated weights for policy 0, policy_version 83663 (0.0007) [2023-10-10 19:51:01,044][123582] Updated weights for policy 0, policy_version 83673 (0.0008) [2023-10-10 19:51:02,399][123614] Updated weights for policy 1, policy_version 83560 (0.0009) [2023-10-10 19:51:02,760][123614] Updated weights for policy 1, policy_version 83570 (0.0009) [2023-10-10 19:51:03,121][123614] Updated weights for policy 1, policy_version 83580 (0.0009) [2023-10-10 19:51:03,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 171278336. Throughput: 0: 1821.6, 1: 1820.6. Samples: 42825942. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:51:03,789][122664] Avg episode reward: [(0, '105.340'), (1, '80.950')] [2023-10-10 19:51:04,683][123582] Updated weights for policy 0, policy_version 83683 (0.0007) [2023-10-10 19:51:05,054][123582] Updated weights for policy 0, policy_version 83693 (0.0010) [2023-10-10 19:51:05,419][123582] Updated weights for policy 0, policy_version 83703 (0.0009) [2023-10-10 19:51:06,949][123614] Updated weights for policy 1, policy_version 83590 (0.0009) [2023-10-10 19:51:07,335][123614] Updated weights for policy 1, policy_version 83600 (0.0009) [2023-10-10 19:51:07,712][123614] Updated weights for policy 1, policy_version 83610 (0.0009) [2023-10-10 19:51:08,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 171343872. Throughput: 0: 1823.6, 1: 1805.3. Samples: 42848072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:51:08,788][122664] Avg episode reward: [(0, '103.890'), (1, '81.800')] [2023-10-10 19:51:09,058][123582] Updated weights for policy 0, policy_version 83713 (0.0009) [2023-10-10 19:51:09,432][123582] Updated weights for policy 0, policy_version 83723 (0.0012) [2023-10-10 19:51:09,801][123582] Updated weights for policy 0, policy_version 83733 (0.0008) [2023-10-10 19:51:10,169][123582] Updated weights for policy 0, policy_version 83743 (0.0008) [2023-10-10 19:51:11,508][123614] Updated weights for policy 1, policy_version 83620 (0.0010) [2023-10-10 19:51:11,878][123614] Updated weights for policy 1, policy_version 83630 (0.0008) [2023-10-10 19:51:12,253][123614] Updated weights for policy 1, policy_version 83640 (0.0010) [2023-10-10 19:51:13,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 171409408. Throughput: 0: 1816.8, 1: 1816.5. Samples: 42858796. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:51:13,788][122664] Avg episode reward: [(0, '99.170'), (1, '81.190')] [2023-10-10 19:51:13,844][123582] Updated weights for policy 0, policy_version 83753 (0.0008) [2023-10-10 19:51:14,222][123582] Updated weights for policy 0, policy_version 83763 (0.0008) [2023-10-10 19:51:14,583][123582] Updated weights for policy 0, policy_version 83773 (0.0011) [2023-10-10 19:51:16,106][123614] Updated weights for policy 1, policy_version 83650 (0.0008) [2023-10-10 19:51:16,473][123614] Updated weights for policy 1, policy_version 83660 (0.0008) [2023-10-10 19:51:16,843][123614] Updated weights for policy 1, policy_version 83670 (0.0010) [2023-10-10 19:51:17,216][123614] Updated weights for policy 1, policy_version 83680 (0.0009) [2023-10-10 19:51:18,195][123582] Updated weights for policy 0, policy_version 83783 (0.0011) [2023-10-10 19:51:18,572][123582] Updated weights for policy 0, policy_version 83793 (0.0011) [2023-10-10 19:51:18,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 171474944. Throughput: 0: 1822.1, 1: 1801.1. Samples: 42880496. Policy #0 lag: (min: 3.0, avg: 11.0, max: 35.0) [2023-10-10 19:51:18,789][122664] Avg episode reward: [(0, '103.040'), (1, '80.810')] [2023-10-10 19:51:18,946][123582] Updated weights for policy 0, policy_version 83803 (0.0009) [2023-10-10 19:51:20,869][123614] Updated weights for policy 1, policy_version 83690 (0.0009) [2023-10-10 19:51:21,238][123614] Updated weights for policy 1, policy_version 83700 (0.0009) [2023-10-10 19:51:21,615][123614] Updated weights for policy 1, policy_version 83710 (0.0008) [2023-10-10 19:51:22,562][123582] Updated weights for policy 0, policy_version 83813 (0.0010) [2023-10-10 19:51:22,930][123582] Updated weights for policy 0, policy_version 83823 (0.0009) [2023-10-10 19:51:23,303][123582] Updated weights for policy 0, policy_version 83833 (0.0008) [2023-10-10 19:51:23,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 171573248. Throughput: 0: 1819.8, 1: 1799.9. Samples: 42901768. Policy #0 lag: (min: 3.0, avg: 11.0, max: 35.0) [2023-10-10 19:51:23,789][122664] Avg episode reward: [(0, '105.900'), (1, '79.680')] [2023-10-10 19:51:25,368][123614] Updated weights for policy 1, policy_version 83720 (0.0008) [2023-10-10 19:51:25,735][123614] Updated weights for policy 1, policy_version 83730 (0.0010) [2023-10-10 19:51:26,097][123614] Updated weights for policy 1, policy_version 83740 (0.0007) [2023-10-10 19:51:26,929][123582] Updated weights for policy 0, policy_version 83843 (0.0007) [2023-10-10 19:51:27,304][123582] Updated weights for policy 0, policy_version 83853 (0.0007) [2023-10-10 19:51:27,669][123582] Updated weights for policy 0, policy_version 83863 (0.0007) [2023-10-10 19:51:28,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 171638784. Throughput: 0: 1824.7, 1: 1797.1. Samples: 42912854. Policy #0 lag: (min: 3.0, avg: 11.0, max: 35.0) [2023-10-10 19:51:28,789][122664] Avg episode reward: [(0, '106.690'), (1, '77.980')] [2023-10-10 19:51:29,780][123614] Updated weights for policy 1, policy_version 83750 (0.0008) [2023-10-10 19:51:30,143][123614] Updated weights for policy 1, policy_version 83760 (0.0010) [2023-10-10 19:51:30,522][123614] Updated weights for policy 1, policy_version 83770 (0.0009) [2023-10-10 19:51:31,394][123582] Updated weights for policy 0, policy_version 83873 (0.0008) [2023-10-10 19:51:31,761][123582] Updated weights for policy 0, policy_version 83883 (0.0007) [2023-10-10 19:51:32,127][123582] Updated weights for policy 0, policy_version 83893 (0.0007) [2023-10-10 19:51:32,507][123582] Updated weights for policy 0, policy_version 83903 (0.0008) [2023-10-10 19:51:33,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 171704320. Throughput: 0: 1822.6, 1: 1795.8. Samples: 42934438. Policy #0 lag: (min: 3.0, avg: 11.0, max: 35.0) [2023-10-10 19:51:33,789][122664] Avg episode reward: [(0, '107.040'), (1, '74.510')] [2023-10-10 19:51:34,203][123614] Updated weights for policy 1, policy_version 83780 (0.0008) [2023-10-10 19:51:34,564][123614] Updated weights for policy 1, policy_version 83790 (0.0008) [2023-10-10 19:51:34,931][123614] Updated weights for policy 1, policy_version 83800 (0.0008) [2023-10-10 19:51:36,413][123582] Updated weights for policy 0, policy_version 83913 (0.0008) [2023-10-10 19:51:36,794][123582] Updated weights for policy 0, policy_version 83923 (0.0009) [2023-10-10 19:51:37,165][123582] Updated weights for policy 0, policy_version 83933 (0.0011) [2023-10-10 19:51:38,576][123614] Updated weights for policy 1, policy_version 83810 (0.0007) [2023-10-10 19:51:38,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 171769856. Throughput: 0: 1816.9, 1: 1810.1. Samples: 42956594. Policy #0 lag: (min: 3.0, avg: 11.0, max: 35.0) [2023-10-10 19:51:38,789][122664] Avg episode reward: [(0, '117.880'), (1, '70.990')] [2023-10-10 19:51:38,798][123247] Saving new best policy, reward=117.880! [2023-10-10 19:51:38,931][123614] Updated weights for policy 1, policy_version 83820 (0.0008) [2023-10-10 19:51:39,298][123614] Updated weights for policy 1, policy_version 83830 (0.0007) [2023-10-10 19:51:39,670][123614] Updated weights for policy 1, policy_version 83840 (0.0008) [2023-10-10 19:51:40,780][123582] Updated weights for policy 0, policy_version 83943 (0.0011) [2023-10-10 19:51:41,148][123582] Updated weights for policy 0, policy_version 83953 (0.0011) [2023-10-10 19:51:41,517][123582] Updated weights for policy 0, policy_version 83963 (0.0011) [2023-10-10 19:51:43,273][123614] Updated weights for policy 1, policy_version 83850 (0.0007) [2023-10-10 19:51:43,643][123614] Updated weights for policy 1, policy_version 83860 (0.0007) [2023-10-10 19:51:43,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 171835392. Throughput: 0: 1815.3, 1: 1797.2. Samples: 42967078. Policy #0 lag: (min: 3.0, avg: 11.0, max: 35.0) [2023-10-10 19:51:43,789][122664] Avg episode reward: [(0, '117.170'), (1, '76.120')] [2023-10-10 19:51:44,011][123614] Updated weights for policy 1, policy_version 83870 (0.0010) [2023-10-10 19:51:45,370][123582] Updated weights for policy 0, policy_version 83973 (0.0008) [2023-10-10 19:51:45,746][123582] Updated weights for policy 0, policy_version 83983 (0.0007) [2023-10-10 19:51:46,125][123582] Updated weights for policy 0, policy_version 83993 (0.0007) [2023-10-10 19:51:47,705][123614] Updated weights for policy 1, policy_version 83880 (0.0008) [2023-10-10 19:51:48,082][123614] Updated weights for policy 1, policy_version 83890 (0.0010) [2023-10-10 19:51:48,446][123614] Updated weights for policy 1, policy_version 83900 (0.0009) [2023-10-10 19:51:48,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 171933696. Throughput: 0: 1819.2, 1: 1811.3. Samples: 42989316. Policy #0 lag: (min: 3.0, avg: 11.0, max: 35.0) [2023-10-10 19:51:48,789][122664] Avg episode reward: [(0, '115.180'), (1, '78.650')] [2023-10-10 19:51:49,812][123582] Updated weights for policy 0, policy_version 84003 (0.0008) [2023-10-10 19:51:50,173][123582] Updated weights for policy 0, policy_version 84013 (0.0010) [2023-10-10 19:51:50,545][123582] Updated weights for policy 0, policy_version 84023 (0.0010) [2023-10-10 19:51:52,027][123614] Updated weights for policy 1, policy_version 83910 (0.0007) [2023-10-10 19:51:52,403][123614] Updated weights for policy 1, policy_version 83920 (0.0007) [2023-10-10 19:51:52,764][123614] Updated weights for policy 1, policy_version 83930 (0.0008) [2023-10-10 19:51:53,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 171999232. Throughput: 0: 1811.0, 1: 1810.9. Samples: 43011058. Policy #0 lag: (min: 3.0, avg: 11.0, max: 35.0) [2023-10-10 19:51:53,789][122664] Avg episode reward: [(0, '117.370'), (1, '83.930')] [2023-10-10 19:51:53,796][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000083936_85950464.pth... [2023-10-10 19:51:53,797][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000084032_86048768.pth... [2023-10-10 19:51:53,826][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000082240_84213760.pth [2023-10-10 19:51:53,833][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000082336_84312064.pth [2023-10-10 19:51:54,139][123582] Updated weights for policy 0, policy_version 84033 (0.0010) [2023-10-10 19:51:54,509][123582] Updated weights for policy 0, policy_version 84043 (0.0008) [2023-10-10 19:51:54,882][123582] Updated weights for policy 0, policy_version 84053 (0.0011) [2023-10-10 19:51:55,252][123582] Updated weights for policy 0, policy_version 84063 (0.0009) [2023-10-10 19:51:56,504][123614] Updated weights for policy 1, policy_version 83940 (0.0008) [2023-10-10 19:51:56,869][123614] Updated weights for policy 1, policy_version 83950 (0.0008) [2023-10-10 19:51:57,239][123614] Updated weights for policy 1, policy_version 83960 (0.0007) [2023-10-10 19:51:58,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 172064768. Throughput: 0: 1814.3, 1: 1814.4. Samples: 43022090. Policy #0 lag: (min: 3.0, avg: 11.0, max: 35.0) [2023-10-10 19:51:58,789][122664] Avg episode reward: [(0, '119.100'), (1, '83.670')] [2023-10-10 19:51:58,886][123582] Updated weights for policy 0, policy_version 84073 (0.0008) [2023-10-10 19:51:59,271][123582] Updated weights for policy 0, policy_version 84083 (0.0009) [2023-10-10 19:51:59,637][123582] Updated weights for policy 0, policy_version 84093 (0.0008) [2023-10-10 19:51:59,742][123247] Saving new best policy, reward=119.100! [2023-10-10 19:52:00,941][123614] Updated weights for policy 1, policy_version 83970 (0.0007) [2023-10-10 19:52:01,306][123614] Updated weights for policy 1, policy_version 83980 (0.0007) [2023-10-10 19:52:01,670][123614] Updated weights for policy 1, policy_version 83990 (0.0007) [2023-10-10 19:52:02,035][123614] Updated weights for policy 1, policy_version 84000 (0.0007) [2023-10-10 19:52:03,180][123582] Updated weights for policy 0, policy_version 84103 (0.0010) [2023-10-10 19:52:03,557][123582] Updated weights for policy 0, policy_version 84113 (0.0009) [2023-10-10 19:52:03,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 172130304. Throughput: 0: 1813.9, 1: 1820.4. Samples: 43044038. Policy #0 lag: (min: 3.0, avg: 11.0, max: 35.0) [2023-10-10 19:52:03,788][122664] Avg episode reward: [(0, '124.430'), (1, '85.020')] [2023-10-10 19:52:03,926][123582] Updated weights for policy 0, policy_version 84123 (0.0009) [2023-10-10 19:52:04,115][123247] Saving new best policy, reward=124.430! [2023-10-10 19:52:05,777][123614] Updated weights for policy 1, policy_version 84010 (0.0008) [2023-10-10 19:52:06,143][123614] Updated weights for policy 1, policy_version 84020 (0.0008) [2023-10-10 19:52:06,512][123614] Updated weights for policy 1, policy_version 84030 (0.0007) [2023-10-10 19:52:07,581][123582] Updated weights for policy 0, policy_version 84133 (0.0009) [2023-10-10 19:52:07,950][123582] Updated weights for policy 0, policy_version 84143 (0.0011) [2023-10-10 19:52:08,314][123582] Updated weights for policy 0, policy_version 84153 (0.0011) [2023-10-10 19:52:08,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 172228608. Throughput: 0: 1817.6, 1: 1821.2. Samples: 43065512. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 19:52:08,789][122664] Avg episode reward: [(0, '124.660'), (1, '82.020')] [2023-10-10 19:52:08,802][123247] Saving new best policy, reward=124.660! [2023-10-10 19:52:10,350][123614] Updated weights for policy 1, policy_version 84040 (0.0009) [2023-10-10 19:52:10,714][123614] Updated weights for policy 1, policy_version 84050 (0.0010) [2023-10-10 19:52:11,080][123614] Updated weights for policy 1, policy_version 84060 (0.0009) [2023-10-10 19:52:12,234][123582] Updated weights for policy 0, policy_version 84163 (0.0009) [2023-10-10 19:52:12,608][123582] Updated weights for policy 0, policy_version 84173 (0.0009) [2023-10-10 19:52:12,974][123582] Updated weights for policy 0, policy_version 84183 (0.0008) [2023-10-10 19:52:13,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 172294144. Throughput: 0: 1811.3, 1: 1828.3. Samples: 43076634. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 19:52:13,789][122664] Avg episode reward: [(0, '119.340'), (1, '79.510')] [2023-10-10 19:52:14,837][123614] Updated weights for policy 1, policy_version 84070 (0.0009) [2023-10-10 19:52:15,194][123614] Updated weights for policy 1, policy_version 84080 (0.0007) [2023-10-10 19:52:15,563][123614] Updated weights for policy 1, policy_version 84090 (0.0010) [2023-10-10 19:52:16,734][123582] Updated weights for policy 0, policy_version 84193 (0.0009) [2023-10-10 19:52:17,109][123582] Updated weights for policy 0, policy_version 84203 (0.0009) [2023-10-10 19:52:17,475][123582] Updated weights for policy 0, policy_version 84213 (0.0010) [2023-10-10 19:52:17,842][123582] Updated weights for policy 0, policy_version 84223 (0.0008) [2023-10-10 19:52:18,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 172359680. Throughput: 0: 1814.0, 1: 1832.0. Samples: 43098512. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 19:52:18,788][122664] Avg episode reward: [(0, '115.970'), (1, '80.810')] [2023-10-10 19:52:19,061][123614] Updated weights for policy 1, policy_version 84100 (0.0007) [2023-10-10 19:52:19,424][123614] Updated weights for policy 1, policy_version 84110 (0.0008) [2023-10-10 19:52:19,794][123614] Updated weights for policy 1, policy_version 84120 (0.0010) [2023-10-10 19:52:21,664][123582] Updated weights for policy 0, policy_version 84233 (0.0007) [2023-10-10 19:52:22,036][123582] Updated weights for policy 0, policy_version 84243 (0.0007) [2023-10-10 19:52:22,422][123582] Updated weights for policy 0, policy_version 84253 (0.0009) [2023-10-10 19:52:23,519][123614] Updated weights for policy 1, policy_version 84130 (0.0009) [2023-10-10 19:52:23,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 172425216. Throughput: 0: 1808.7, 1: 1828.2. Samples: 43120256. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 19:52:23,789][122664] Avg episode reward: [(0, '114.860'), (1, '80.390')] [2023-10-10 19:52:23,892][123614] Updated weights for policy 1, policy_version 84140 (0.0008) [2023-10-10 19:52:24,256][123614] Updated weights for policy 1, policy_version 84150 (0.0009) [2023-10-10 19:52:24,621][123614] Updated weights for policy 1, policy_version 84160 (0.0007) [2023-10-10 19:52:26,045][123582] Updated weights for policy 0, policy_version 84263 (0.0008) [2023-10-10 19:52:26,414][123582] Updated weights for policy 0, policy_version 84273 (0.0009) [2023-10-10 19:52:26,794][123582] Updated weights for policy 0, policy_version 84283 (0.0010) [2023-10-10 19:52:28,330][123614] Updated weights for policy 1, policy_version 84170 (0.0011) [2023-10-10 19:52:28,701][123614] Updated weights for policy 1, policy_version 84180 (0.0010) [2023-10-10 19:52:28,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 172490752. Throughput: 0: 1814.2, 1: 1829.7. Samples: 43131052. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 19:52:28,789][122664] Avg episode reward: [(0, '112.710'), (1, '82.460')] [2023-10-10 19:52:29,071][123614] Updated weights for policy 1, policy_version 84190 (0.0009) [2023-10-10 19:52:30,569][123582] Updated weights for policy 0, policy_version 84293 (0.0009) [2023-10-10 19:52:30,944][123582] Updated weights for policy 0, policy_version 84303 (0.0009) [2023-10-10 19:52:31,321][123582] Updated weights for policy 0, policy_version 84313 (0.0009) [2023-10-10 19:52:32,819][123614] Updated weights for policy 1, policy_version 84200 (0.0009) [2023-10-10 19:52:33,183][123614] Updated weights for policy 1, policy_version 84210 (0.0010) [2023-10-10 19:52:33,554][123614] Updated weights for policy 1, policy_version 84220 (0.0010) [2023-10-10 19:52:33,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 172589056. Throughput: 0: 1809.1, 1: 1824.3. Samples: 43152820. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 19:52:33,789][122664] Avg episode reward: [(0, '107.640'), (1, '85.840')] [2023-10-10 19:52:35,018][123582] Updated weights for policy 0, policy_version 84323 (0.0009) [2023-10-10 19:52:35,390][123582] Updated weights for policy 0, policy_version 84333 (0.0008) [2023-10-10 19:52:35,760][123582] Updated weights for policy 0, policy_version 84343 (0.0007) [2023-10-10 19:52:37,384][123614] Updated weights for policy 1, policy_version 84230 (0.0008) [2023-10-10 19:52:37,766][123614] Updated weights for policy 1, policy_version 84240 (0.0007) [2023-10-10 19:52:38,140][123614] Updated weights for policy 1, policy_version 84250 (0.0008) [2023-10-10 19:52:38,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 172654592. Throughput: 0: 1809.9, 1: 1813.4. Samples: 43174106. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 19:52:38,789][122664] Avg episode reward: [(0, '106.140'), (1, '86.720')] [2023-10-10 19:52:39,243][123582] Updated weights for policy 0, policy_version 84353 (0.0007) [2023-10-10 19:52:39,612][123582] Updated weights for policy 0, policy_version 84363 (0.0008) [2023-10-10 19:52:39,992][123582] Updated weights for policy 0, policy_version 84373 (0.0009) [2023-10-10 19:52:40,356][123582] Updated weights for policy 0, policy_version 84383 (0.0008) [2023-10-10 19:52:41,834][123614] Updated weights for policy 1, policy_version 84260 (0.0007) [2023-10-10 19:52:42,202][123614] Updated weights for policy 1, policy_version 84270 (0.0008) [2023-10-10 19:52:42,581][123614] Updated weights for policy 1, policy_version 84280 (0.0012) [2023-10-10 19:52:43,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 172720128. Throughput: 0: 1810.5, 1: 1819.5. Samples: 43185440. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 19:52:43,788][122664] Avg episode reward: [(0, '105.830'), (1, '91.210')] [2023-10-10 19:52:44,185][123582] Updated weights for policy 0, policy_version 84393 (0.0008) [2023-10-10 19:52:44,558][123582] Updated weights for policy 0, policy_version 84403 (0.0010) [2023-10-10 19:52:44,937][123582] Updated weights for policy 0, policy_version 84413 (0.0009) [2023-10-10 19:52:46,199][123614] Updated weights for policy 1, policy_version 84290 (0.0011) [2023-10-10 19:52:46,561][123614] Updated weights for policy 1, policy_version 84300 (0.0011) [2023-10-10 19:52:46,928][123614] Updated weights for policy 1, policy_version 84310 (0.0009) [2023-10-10 19:52:47,293][123614] Updated weights for policy 1, policy_version 84320 (0.0010) [2023-10-10 19:52:48,578][123582] Updated weights for policy 0, policy_version 84423 (0.0008) [2023-10-10 19:52:48,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 172785664. Throughput: 0: 1807.8, 1: 1813.0. Samples: 43206976. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 19:52:48,789][122664] Avg episode reward: [(0, '94.990'), (1, '93.160')] [2023-10-10 19:52:48,956][123582] Updated weights for policy 0, policy_version 84433 (0.0008) [2023-10-10 19:52:49,318][123582] Updated weights for policy 0, policy_version 84443 (0.0011) [2023-10-10 19:52:50,952][123614] Updated weights for policy 1, policy_version 84330 (0.0007) [2023-10-10 19:52:51,318][123614] Updated weights for policy 1, policy_version 84340 (0.0009) [2023-10-10 19:52:51,691][123614] Updated weights for policy 1, policy_version 84350 (0.0011) [2023-10-10 19:52:52,862][123582] Updated weights for policy 0, policy_version 84453 (0.0009) [2023-10-10 19:52:53,236][123582] Updated weights for policy 0, policy_version 84463 (0.0007) [2023-10-10 19:52:53,600][123582] Updated weights for policy 0, policy_version 84473 (0.0008) [2023-10-10 19:52:53,788][122664] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 172851200. Throughput: 0: 1820.3, 1: 1819.6. Samples: 43229310. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 19:52:53,789][122664] Avg episode reward: [(0, '93.530'), (1, '93.240')] [2023-10-10 19:52:55,241][123614] Updated weights for policy 1, policy_version 84360 (0.0008) [2023-10-10 19:52:55,607][123614] Updated weights for policy 1, policy_version 84370 (0.0008) [2023-10-10 19:52:55,969][123614] Updated weights for policy 1, policy_version 84380 (0.0008) [2023-10-10 19:52:57,314][123582] Updated weights for policy 0, policy_version 84483 (0.0011) [2023-10-10 19:52:57,691][123582] Updated weights for policy 0, policy_version 84493 (0.0011) [2023-10-10 19:52:58,059][123582] Updated weights for policy 0, policy_version 84503 (0.0008) [2023-10-10 19:52:58,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 172949504. Throughput: 0: 1815.6, 1: 1817.9. Samples: 43240142. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 19:52:58,789][122664] Avg episode reward: [(0, '82.060'), (1, '96.250')] [2023-10-10 19:52:59,636][123614] Updated weights for policy 1, policy_version 84390 (0.0008) [2023-10-10 19:53:00,009][123614] Updated weights for policy 1, policy_version 84400 (0.0009) [2023-10-10 19:53:00,381][123614] Updated weights for policy 1, policy_version 84410 (0.0008) [2023-10-10 19:53:01,804][123582] Updated weights for policy 0, policy_version 84513 (0.0008) [2023-10-10 19:53:02,188][123582] Updated weights for policy 0, policy_version 84523 (0.0007) [2023-10-10 19:53:02,560][123582] Updated weights for policy 0, policy_version 84533 (0.0008) [2023-10-10 19:53:02,930][123582] Updated weights for policy 0, policy_version 84543 (0.0007) [2023-10-10 19:53:03,788][122664] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 173015040. Throughput: 0: 1816.2, 1: 1812.6. Samples: 43261808. Policy #0 lag: (min: 27.0, avg: 27.9, max: 46.0) [2023-10-10 19:53:03,789][122664] Avg episode reward: [(0, '67.170'), (1, '97.810')] [2023-10-10 19:53:04,184][123614] Updated weights for policy 1, policy_version 84420 (0.0009) [2023-10-10 19:53:04,554][123614] Updated weights for policy 1, policy_version 84430 (0.0008) [2023-10-10 19:53:04,927][123614] Updated weights for policy 1, policy_version 84440 (0.0007) [2023-10-10 19:53:06,705][123582] Updated weights for policy 0, policy_version 84553 (0.0008) [2023-10-10 19:53:07,082][123582] Updated weights for policy 0, policy_version 84563 (0.0007) [2023-10-10 19:53:07,456][123582] Updated weights for policy 0, policy_version 84573 (0.0008) [2023-10-10 19:53:08,661][123614] Updated weights for policy 1, policy_version 84450 (0.0007) [2023-10-10 19:53:08,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 173080576. Throughput: 0: 1811.4, 1: 1812.7. Samples: 43283342. Policy #0 lag: (min: 27.0, avg: 27.9, max: 46.0) [2023-10-10 19:53:08,788][122664] Avg episode reward: [(0, '64.030'), (1, '93.630')] [2023-10-10 19:53:09,025][123614] Updated weights for policy 1, policy_version 84460 (0.0010) [2023-10-10 19:53:09,388][123614] Updated weights for policy 1, policy_version 84470 (0.0010) [2023-10-10 19:53:09,754][123614] Updated weights for policy 1, policy_version 84480 (0.0009) [2023-10-10 19:53:10,965][123582] Updated weights for policy 0, policy_version 84583 (0.0009) [2023-10-10 19:53:11,332][123582] Updated weights for policy 0, policy_version 84593 (0.0008) [2023-10-10 19:53:11,708][123582] Updated weights for policy 0, policy_version 84603 (0.0012) [2023-10-10 19:53:13,405][123614] Updated weights for policy 1, policy_version 84490 (0.0007) [2023-10-10 19:53:13,771][123614] Updated weights for policy 1, policy_version 84500 (0.0010) [2023-10-10 19:53:13,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 173146112. Throughput: 0: 1818.5, 1: 1810.1. Samples: 43294342. Policy #0 lag: (min: 27.0, avg: 27.9, max: 46.0) [2023-10-10 19:53:13,789][122664] Avg episode reward: [(0, '62.450'), (1, '89.510')] [2023-10-10 19:53:14,144][123614] Updated weights for policy 1, policy_version 84510 (0.0007) [2023-10-10 19:53:15,443][123582] Updated weights for policy 0, policy_version 84613 (0.0008) [2023-10-10 19:53:15,812][123582] Updated weights for policy 0, policy_version 84623 (0.0007) [2023-10-10 19:53:16,182][123582] Updated weights for policy 0, policy_version 84633 (0.0008) [2023-10-10 19:53:18,010][123614] Updated weights for policy 1, policy_version 84520 (0.0010) [2023-10-10 19:53:18,378][123614] Updated weights for policy 1, policy_version 84530 (0.0007) [2023-10-10 19:53:18,748][123614] Updated weights for policy 1, policy_version 84540 (0.0008) [2023-10-10 19:53:18,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 173211648. Throughput: 0: 1815.4, 1: 1816.1. Samples: 43316240. Policy #0 lag: (min: 27.0, avg: 27.9, max: 46.0) [2023-10-10 19:53:18,789][122664] Avg episode reward: [(0, '63.930'), (1, '88.350')] [2023-10-10 19:53:19,975][123582] Updated weights for policy 0, policy_version 84643 (0.0008) [2023-10-10 19:53:20,351][123582] Updated weights for policy 0, policy_version 84653 (0.0007) [2023-10-10 19:53:20,728][123582] Updated weights for policy 0, policy_version 84663 (0.0007) [2023-10-10 19:53:22,696][123614] Updated weights for policy 1, policy_version 84550 (0.0008) [2023-10-10 19:53:23,078][123614] Updated weights for policy 1, policy_version 84560 (0.0011) [2023-10-10 19:53:23,446][123614] Updated weights for policy 1, policy_version 84570 (0.0011) [2023-10-10 19:53:23,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 173309952. Throughput: 0: 1820.8, 1: 1808.6. Samples: 43337430. Policy #0 lag: (min: 27.0, avg: 27.9, max: 46.0) [2023-10-10 19:53:23,788][122664] Avg episode reward: [(0, '63.810'), (1, '88.930')] [2023-10-10 19:53:24,214][123582] Updated weights for policy 0, policy_version 84673 (0.0007) [2023-10-10 19:53:24,583][123582] Updated weights for policy 0, policy_version 84683 (0.0009) [2023-10-10 19:53:24,955][123582] Updated weights for policy 0, policy_version 84693 (0.0007) [2023-10-10 19:53:25,323][123582] Updated weights for policy 0, policy_version 84703 (0.0008) [2023-10-10 19:53:27,130][123614] Updated weights for policy 1, policy_version 84580 (0.0009) [2023-10-10 19:53:27,492][123614] Updated weights for policy 1, policy_version 84590 (0.0008) [2023-10-10 19:53:27,867][123614] Updated weights for policy 1, policy_version 84600 (0.0008) [2023-10-10 19:53:28,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 173375488. Throughput: 0: 1820.7, 1: 1811.2. Samples: 43348874. Policy #0 lag: (min: 27.0, avg: 27.9, max: 46.0) [2023-10-10 19:53:28,788][122664] Avg episode reward: [(0, '63.880'), (1, '86.780')] [2023-10-10 19:53:28,956][123582] Updated weights for policy 0, policy_version 84713 (0.0010) [2023-10-10 19:53:29,329][123582] Updated weights for policy 0, policy_version 84723 (0.0007) [2023-10-10 19:53:29,696][123582] Updated weights for policy 0, policy_version 84733 (0.0010) [2023-10-10 19:53:31,542][123614] Updated weights for policy 1, policy_version 84610 (0.0009) [2023-10-10 19:53:31,907][123614] Updated weights for policy 1, policy_version 84620 (0.0007) [2023-10-10 19:53:32,282][123614] Updated weights for policy 1, policy_version 84630 (0.0008) [2023-10-10 19:53:32,653][123614] Updated weights for policy 1, policy_version 84640 (0.0008) [2023-10-10 19:53:33,347][123582] Updated weights for policy 0, policy_version 84743 (0.0008) [2023-10-10 19:53:33,716][123582] Updated weights for policy 0, policy_version 84753 (0.0008) [2023-10-10 19:53:33,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 173441024. Throughput: 0: 1823.9, 1: 1807.4. Samples: 43370384. Policy #0 lag: (min: 27.0, avg: 27.9, max: 46.0) [2023-10-10 19:53:33,789][122664] Avg episode reward: [(0, '67.170'), (1, '91.840')] [2023-10-10 19:53:34,091][123582] Updated weights for policy 0, policy_version 84763 (0.0009) [2023-10-10 19:53:36,355][123614] Updated weights for policy 1, policy_version 84650 (0.0011) [2023-10-10 19:53:36,718][123614] Updated weights for policy 1, policy_version 84660 (0.0011) [2023-10-10 19:53:37,085][123614] Updated weights for policy 1, policy_version 84670 (0.0008) [2023-10-10 19:53:37,779][123582] Updated weights for policy 0, policy_version 84773 (0.0008) [2023-10-10 19:53:38,156][123582] Updated weights for policy 0, policy_version 84783 (0.0009) [2023-10-10 19:53:38,528][123582] Updated weights for policy 0, policy_version 84793 (0.0010) [2023-10-10 19:53:38,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 173539328. Throughput: 0: 1823.3, 1: 1801.6. Samples: 43392432. Policy #0 lag: (min: 27.0, avg: 27.9, max: 46.0) [2023-10-10 19:53:38,789][122664] Avg episode reward: [(0, '69.210'), (1, '94.460')] [2023-10-10 19:53:40,617][123614] Updated weights for policy 1, policy_version 84680 (0.0008) [2023-10-10 19:53:40,990][123614] Updated weights for policy 1, policy_version 84690 (0.0008) [2023-10-10 19:53:41,361][123614] Updated weights for policy 1, policy_version 84700 (0.0008) [2023-10-10 19:53:42,237][123582] Updated weights for policy 0, policy_version 84803 (0.0010) [2023-10-10 19:53:42,597][123582] Updated weights for policy 0, policy_version 84813 (0.0007) [2023-10-10 19:53:42,977][123582] Updated weights for policy 0, policy_version 84823 (0.0009) [2023-10-10 19:53:43,788][122664] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 173604864. Throughput: 0: 1822.6, 1: 1797.5. Samples: 43403046. Policy #0 lag: (min: 27.0, avg: 27.9, max: 46.0) [2023-10-10 19:53:43,788][122664] Avg episode reward: [(0, '66.220'), (1, '91.420')] [2023-10-10 19:53:44,949][123614] Updated weights for policy 1, policy_version 84710 (0.0008) [2023-10-10 19:53:45,311][123614] Updated weights for policy 1, policy_version 84720 (0.0011) [2023-10-10 19:53:45,678][123614] Updated weights for policy 1, policy_version 84730 (0.0010) [2023-10-10 19:53:46,726][123582] Updated weights for policy 0, policy_version 84833 (0.0011) [2023-10-10 19:53:47,101][123582] Updated weights for policy 0, policy_version 84843 (0.0008) [2023-10-10 19:53:47,475][123582] Updated weights for policy 0, policy_version 84853 (0.0008) [2023-10-10 19:53:47,844][123582] Updated weights for policy 0, policy_version 84863 (0.0008) [2023-10-10 19:53:48,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 173670400. Throughput: 0: 1826.0, 1: 1800.4. Samples: 43424996. Policy #0 lag: (min: 27.0, avg: 27.9, max: 46.0) [2023-10-10 19:53:48,789][122664] Avg episode reward: [(0, '69.130'), (1, '88.360')] [2023-10-10 19:53:49,351][123614] Updated weights for policy 1, policy_version 84740 (0.0009) [2023-10-10 19:53:49,733][123614] Updated weights for policy 1, policy_version 84750 (0.0007) [2023-10-10 19:53:50,091][123614] Updated weights for policy 1, policy_version 84760 (0.0008) [2023-10-10 19:53:51,539][123582] Updated weights for policy 0, policy_version 84873 (0.0011) [2023-10-10 19:53:51,919][123582] Updated weights for policy 0, policy_version 84883 (0.0009) [2023-10-10 19:53:52,287][123582] Updated weights for policy 0, policy_version 84893 (0.0011) [2023-10-10 19:53:53,788][122664] Fps is (10 sec: 13106.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 173735936. Throughput: 0: 1823.9, 1: 1814.2. Samples: 43447058. Policy #0 lag: (min: 27.0, avg: 27.9, max: 46.0) [2023-10-10 19:53:53,789][122664] Avg episode reward: [(0, '70.000'), (1, '86.730')] [2023-10-10 19:53:53,799][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000084896_86933504.pth... [2023-10-10 19:53:53,801][123614] Updated weights for policy 1, policy_version 84770 (0.0008) [2023-10-10 19:53:53,835][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000083200_85196800.pth [2023-10-10 19:53:53,839][123247] Saving a milestone ./train_atari/atari_demonattack_APPO/checkpoint_p0/milestones/checkpoint_000084896_86933504.pth [2023-10-10 19:53:54,164][123614] Updated weights for policy 1, policy_version 84780 (0.0009) [2023-10-10 19:53:54,526][123614] Updated weights for policy 1, policy_version 84790 (0.0009) [2023-10-10 19:53:54,892][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000084800_86835200.pth... [2023-10-10 19:53:54,897][123614] Updated weights for policy 1, policy_version 84800 (0.0009) [2023-10-10 19:53:54,932][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000083072_85065728.pth [2023-10-10 19:53:54,935][123465] Saving a milestone ./train_atari/atari_demonattack_APPO/checkpoint_p1/milestones/checkpoint_000084800_86835200.pth [2023-10-10 19:53:56,117][123582] Updated weights for policy 0, policy_version 84903 (0.0009) [2023-10-10 19:53:56,504][123582] Updated weights for policy 0, policy_version 84913 (0.0010) [2023-10-10 19:53:56,869][123582] Updated weights for policy 0, policy_version 84923 (0.0007) [2023-10-10 19:53:58,463][123614] Updated weights for policy 1, policy_version 84810 (0.0008) [2023-10-10 19:53:58,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 173801472. Throughput: 0: 1822.7, 1: 1812.0. Samples: 43457904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:53:58,789][122664] Avg episode reward: [(0, '75.480'), (1, '85.720')] [2023-10-10 19:53:58,832][123614] Updated weights for policy 1, policy_version 84820 (0.0009) [2023-10-10 19:53:59,203][123614] Updated weights for policy 1, policy_version 84830 (0.0009) [2023-10-10 19:54:00,779][123582] Updated weights for policy 0, policy_version 84933 (0.0009) [2023-10-10 19:54:01,150][123582] Updated weights for policy 0, policy_version 84943 (0.0009) [2023-10-10 19:54:01,515][123582] Updated weights for policy 0, policy_version 84953 (0.0008) [2023-10-10 19:54:02,836][123614] Updated weights for policy 1, policy_version 84840 (0.0010) [2023-10-10 19:54:03,216][123614] Updated weights for policy 1, policy_version 84850 (0.0010) [2023-10-10 19:54:03,579][123614] Updated weights for policy 1, policy_version 84860 (0.0009) [2023-10-10 19:54:03,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 173899776. Throughput: 0: 1820.4, 1: 1814.7. Samples: 43479820. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:54:03,789][122664] Avg episode reward: [(0, '72.430'), (1, '85.530')] [2023-10-10 19:54:05,311][123582] Updated weights for policy 0, policy_version 84963 (0.0008) [2023-10-10 19:54:05,693][123582] Updated weights for policy 0, policy_version 84973 (0.0010) [2023-10-10 19:54:06,066][123582] Updated weights for policy 0, policy_version 84983 (0.0011) [2023-10-10 19:54:07,350][123614] Updated weights for policy 1, policy_version 84870 (0.0009) [2023-10-10 19:54:07,720][123614] Updated weights for policy 1, policy_version 84880 (0.0007) [2023-10-10 19:54:08,086][123614] Updated weights for policy 1, policy_version 84890 (0.0011) [2023-10-10 19:54:08,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 173965312. Throughput: 0: 1809.9, 1: 1831.2. Samples: 43501280. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:54:08,789][122664] Avg episode reward: [(0, '70.680'), (1, '87.820')] [2023-10-10 19:54:09,622][123582] Updated weights for policy 0, policy_version 84993 (0.0011) [2023-10-10 19:54:09,992][123582] Updated weights for policy 0, policy_version 85003 (0.0011) [2023-10-10 19:54:10,368][123582] Updated weights for policy 0, policy_version 85013 (0.0010) [2023-10-10 19:54:10,729][123582] Updated weights for policy 0, policy_version 85023 (0.0007) [2023-10-10 19:54:11,896][123614] Updated weights for policy 1, policy_version 84900 (0.0009) [2023-10-10 19:54:12,291][123614] Updated weights for policy 1, policy_version 84910 (0.0008) [2023-10-10 19:54:12,654][123614] Updated weights for policy 1, policy_version 84920 (0.0007) [2023-10-10 19:54:13,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 174030848. Throughput: 0: 1805.2, 1: 1830.8. Samples: 43512496. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:54:13,789][122664] Avg episode reward: [(0, '69.970'), (1, '86.430')] [2023-10-10 19:54:14,473][123582] Updated weights for policy 0, policy_version 85033 (0.0012) [2023-10-10 19:54:14,834][123582] Updated weights for policy 0, policy_version 85043 (0.0010) [2023-10-10 19:54:15,204][123582] Updated weights for policy 0, policy_version 85053 (0.0009) [2023-10-10 19:54:16,321][123614] Updated weights for policy 1, policy_version 84930 (0.0008) [2023-10-10 19:54:16,694][123614] Updated weights for policy 1, policy_version 84940 (0.0008) [2023-10-10 19:54:17,077][123614] Updated weights for policy 1, policy_version 84950 (0.0009) [2023-10-10 19:54:17,435][123614] Updated weights for policy 1, policy_version 84960 (0.0007) [2023-10-10 19:54:18,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 174096384. Throughput: 0: 1800.1, 1: 1824.4. Samples: 43533486. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:54:18,789][122664] Avg episode reward: [(0, '71.010'), (1, '86.360')] [2023-10-10 19:54:18,936][123582] Updated weights for policy 0, policy_version 85063 (0.0007) [2023-10-10 19:54:19,312][123582] Updated weights for policy 0, policy_version 85073 (0.0008) [2023-10-10 19:54:19,676][123582] Updated weights for policy 0, policy_version 85083 (0.0009) [2023-10-10 19:54:21,057][123614] Updated weights for policy 1, policy_version 84970 (0.0007) [2023-10-10 19:54:21,436][123614] Updated weights for policy 1, policy_version 84980 (0.0010) [2023-10-10 19:54:21,805][123614] Updated weights for policy 1, policy_version 84990 (0.0009) [2023-10-10 19:54:23,493][123582] Updated weights for policy 0, policy_version 85093 (0.0008) [2023-10-10 19:54:23,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 174161920. Throughput: 0: 1812.1, 1: 1824.6. Samples: 43556084. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:54:23,789][122664] Avg episode reward: [(0, '74.870'), (1, '88.850')] [2023-10-10 19:54:23,855][123582] Updated weights for policy 0, policy_version 85103 (0.0010) [2023-10-10 19:54:24,225][123582] Updated weights for policy 0, policy_version 85113 (0.0010) [2023-10-10 19:54:25,531][123614] Updated weights for policy 1, policy_version 85000 (0.0008) [2023-10-10 19:54:25,903][123614] Updated weights for policy 1, policy_version 85010 (0.0008) [2023-10-10 19:54:26,269][123614] Updated weights for policy 1, policy_version 85020 (0.0008) [2023-10-10 19:54:27,699][123582] Updated weights for policy 0, policy_version 85123 (0.0009) [2023-10-10 19:54:28,063][123582] Updated weights for policy 0, policy_version 85133 (0.0010) [2023-10-10 19:54:28,429][123582] Updated weights for policy 0, policy_version 85143 (0.0008) [2023-10-10 19:54:28,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 174260224. Throughput: 0: 1800.6, 1: 1830.2. Samples: 43566432. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:54:28,789][122664] Avg episode reward: [(0, '72.420'), (1, '88.420')] [2023-10-10 19:54:29,963][123614] Updated weights for policy 1, policy_version 85030 (0.0010) [2023-10-10 19:54:30,336][123614] Updated weights for policy 1, policy_version 85040 (0.0009) [2023-10-10 19:54:30,707][123614] Updated weights for policy 1, policy_version 85050 (0.0010) [2023-10-10 19:54:32,163][123582] Updated weights for policy 0, policy_version 85153 (0.0009) [2023-10-10 19:54:32,535][123582] Updated weights for policy 0, policy_version 85163 (0.0009) [2023-10-10 19:54:32,914][123582] Updated weights for policy 0, policy_version 85173 (0.0009) [2023-10-10 19:54:33,279][123582] Updated weights for policy 0, policy_version 85183 (0.0012) [2023-10-10 19:54:33,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 174325760. Throughput: 0: 1809.1, 1: 1833.1. Samples: 43588898. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:54:33,789][122664] Avg episode reward: [(0, '77.130'), (1, '83.940')] [2023-10-10 19:54:34,304][123614] Updated weights for policy 1, policy_version 85060 (0.0009) [2023-10-10 19:54:34,675][123614] Updated weights for policy 1, policy_version 85070 (0.0009) [2023-10-10 19:54:35,036][123614] Updated weights for policy 1, policy_version 85080 (0.0009) [2023-10-10 19:54:36,933][123582] Updated weights for policy 0, policy_version 85193 (0.0010) [2023-10-10 19:54:37,312][123582] Updated weights for policy 0, policy_version 85203 (0.0011) [2023-10-10 19:54:37,687][123582] Updated weights for policy 0, policy_version 85213 (0.0011) [2023-10-10 19:54:38,775][123614] Updated weights for policy 1, policy_version 85090 (0.0010) [2023-10-10 19:54:38,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 174391296. Throughput: 0: 1805.4, 1: 1825.8. Samples: 43610460. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:54:38,789][122664] Avg episode reward: [(0, '78.240'), (1, '83.290')] [2023-10-10 19:54:39,147][123614] Updated weights for policy 1, policy_version 85100 (0.0007) [2023-10-10 19:54:39,509][123614] Updated weights for policy 1, policy_version 85110 (0.0010) [2023-10-10 19:54:39,874][123614] Updated weights for policy 1, policy_version 85120 (0.0009) [2023-10-10 19:54:41,291][123582] Updated weights for policy 0, policy_version 85223 (0.0010) [2023-10-10 19:54:41,655][123582] Updated weights for policy 0, policy_version 85233 (0.0010) [2023-10-10 19:54:42,029][123582] Updated weights for policy 0, policy_version 85243 (0.0008) [2023-10-10 19:54:43,578][123614] Updated weights for policy 1, policy_version 85130 (0.0009) [2023-10-10 19:54:43,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 174456832. Throughput: 0: 1814.7, 1: 1822.5. Samples: 43621576. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:54:43,789][122664] Avg episode reward: [(0, '78.730'), (1, '84.970')] [2023-10-10 19:54:43,947][123614] Updated weights for policy 1, policy_version 85140 (0.0009) [2023-10-10 19:54:44,324][123614] Updated weights for policy 1, policy_version 85150 (0.0009) [2023-10-10 19:54:45,712][123582] Updated weights for policy 0, policy_version 85253 (0.0007) [2023-10-10 19:54:46,091][123582] Updated weights for policy 0, policy_version 85263 (0.0009) [2023-10-10 19:54:46,469][123582] Updated weights for policy 0, policy_version 85273 (0.0010) [2023-10-10 19:54:47,984][123614] Updated weights for policy 1, policy_version 85160 (0.0009) [2023-10-10 19:54:48,348][123614] Updated weights for policy 1, policy_version 85170 (0.0007) [2023-10-10 19:54:48,720][123614] Updated weights for policy 1, policy_version 85180 (0.0008) [2023-10-10 19:54:48,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 174522368. Throughput: 0: 1814.4, 1: 1821.6. Samples: 43643442. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:54:48,789][122664] Avg episode reward: [(0, '84.860'), (1, '87.020')] [2023-10-10 19:54:50,202][123582] Updated weights for policy 0, policy_version 85283 (0.0008) [2023-10-10 19:54:50,571][123582] Updated weights for policy 0, policy_version 85293 (0.0008) [2023-10-10 19:54:50,949][123582] Updated weights for policy 0, policy_version 85303 (0.0007) [2023-10-10 19:54:52,335][123614] Updated weights for policy 1, policy_version 85190 (0.0008) [2023-10-10 19:54:52,708][123614] Updated weights for policy 1, policy_version 85200 (0.0009) [2023-10-10 19:54:53,068][123614] Updated weights for policy 1, policy_version 85210 (0.0007) [2023-10-10 19:54:53,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 174620672. Throughput: 0: 1818.0, 1: 1813.5. Samples: 43664698. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:54:53,789][122664] Avg episode reward: [(0, '85.600'), (1, '88.410')] [2023-10-10 19:54:54,629][123582] Updated weights for policy 0, policy_version 85313 (0.0007) [2023-10-10 19:54:54,998][123582] Updated weights for policy 0, policy_version 85323 (0.0007) [2023-10-10 19:54:55,363][123582] Updated weights for policy 0, policy_version 85333 (0.0007) [2023-10-10 19:54:55,734][123582] Updated weights for policy 0, policy_version 85343 (0.0007) [2023-10-10 19:54:56,908][123614] Updated weights for policy 1, policy_version 85220 (0.0008) [2023-10-10 19:54:57,304][123614] Updated weights for policy 1, policy_version 85230 (0.0009) [2023-10-10 19:54:57,672][123614] Updated weights for policy 1, policy_version 85240 (0.0008) [2023-10-10 19:54:58,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 174686208. Throughput: 0: 1819.1, 1: 1811.2. Samples: 43675860. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:54:58,789][122664] Avg episode reward: [(0, '82.000'), (1, '86.310')] [2023-10-10 19:54:59,386][123582] Updated weights for policy 0, policy_version 85353 (0.0008) [2023-10-10 19:54:59,747][123582] Updated weights for policy 0, policy_version 85363 (0.0008) [2023-10-10 19:55:00,126][123582] Updated weights for policy 0, policy_version 85373 (0.0007) [2023-10-10 19:55:01,383][123614] Updated weights for policy 1, policy_version 85250 (0.0008) [2023-10-10 19:55:01,747][123614] Updated weights for policy 1, policy_version 85260 (0.0008) [2023-10-10 19:55:02,117][123614] Updated weights for policy 1, policy_version 85270 (0.0007) [2023-10-10 19:55:02,475][123614] Updated weights for policy 1, policy_version 85280 (0.0007) [2023-10-10 19:55:03,745][123582] Updated weights for policy 0, policy_version 85383 (0.0007) [2023-10-10 19:55:03,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 174751744. Throughput: 0: 1821.5, 1: 1819.9. Samples: 43697352. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:55:03,789][122664] Avg episode reward: [(0, '83.780'), (1, '89.660')] [2023-10-10 19:55:04,121][123582] Updated weights for policy 0, policy_version 85393 (0.0008) [2023-10-10 19:55:04,500][123582] Updated weights for policy 0, policy_version 85403 (0.0008) [2023-10-10 19:55:06,036][123614] Updated weights for policy 1, policy_version 85290 (0.0008) [2023-10-10 19:55:06,413][123614] Updated weights for policy 1, policy_version 85300 (0.0007) [2023-10-10 19:55:06,775][123614] Updated weights for policy 1, policy_version 85310 (0.0007) [2023-10-10 19:55:08,141][123582] Updated weights for policy 0, policy_version 85413 (0.0009) [2023-10-10 19:55:08,511][123582] Updated weights for policy 0, policy_version 85423 (0.0008) [2023-10-10 19:55:08,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 174817280. Throughput: 0: 1815.0, 1: 1818.0. Samples: 43719568. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:55:08,789][122664] Avg episode reward: [(0, '83.600'), (1, '86.290')] [2023-10-10 19:55:08,883][123582] Updated weights for policy 0, policy_version 85433 (0.0010) [2023-10-10 19:55:10,508][123614] Updated weights for policy 1, policy_version 85320 (0.0007) [2023-10-10 19:55:10,879][123614] Updated weights for policy 1, policy_version 85330 (0.0008) [2023-10-10 19:55:11,248][123614] Updated weights for policy 1, policy_version 85340 (0.0008) [2023-10-10 19:55:12,546][123582] Updated weights for policy 0, policy_version 85443 (0.0008) [2023-10-10 19:55:12,920][123582] Updated weights for policy 0, policy_version 85453 (0.0008) [2023-10-10 19:55:13,282][123582] Updated weights for policy 0, policy_version 85463 (0.0007) [2023-10-10 19:55:13,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 174915584. Throughput: 0: 1819.2, 1: 1817.0. Samples: 43730060. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:55:13,789][122664] Avg episode reward: [(0, '80.450'), (1, '84.530')] [2023-10-10 19:55:14,842][123614] Updated weights for policy 1, policy_version 85350 (0.0010) [2023-10-10 19:55:15,218][123614] Updated weights for policy 1, policy_version 85360 (0.0011) [2023-10-10 19:55:15,582][123614] Updated weights for policy 1, policy_version 85370 (0.0011) [2023-10-10 19:55:16,849][123582] Updated weights for policy 0, policy_version 85473 (0.0008) [2023-10-10 19:55:17,221][123582] Updated weights for policy 0, policy_version 85483 (0.0010) [2023-10-10 19:55:17,596][123582] Updated weights for policy 0, policy_version 85493 (0.0009) [2023-10-10 19:55:17,967][123582] Updated weights for policy 0, policy_version 85503 (0.0008) [2023-10-10 19:55:18,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 174981120. Throughput: 0: 1815.7, 1: 1817.5. Samples: 43752392. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:55:18,789][122664] Avg episode reward: [(0, '80.460'), (1, '83.070')] [2023-10-10 19:55:19,246][123614] Updated weights for policy 1, policy_version 85380 (0.0009) [2023-10-10 19:55:19,624][123614] Updated weights for policy 1, policy_version 85390 (0.0008) [2023-10-10 19:55:19,992][123614] Updated weights for policy 1, policy_version 85400 (0.0008) [2023-10-10 19:55:21,792][123582] Updated weights for policy 0, policy_version 85513 (0.0009) [2023-10-10 19:55:22,155][123582] Updated weights for policy 0, policy_version 85523 (0.0009) [2023-10-10 19:55:22,521][123582] Updated weights for policy 0, policy_version 85533 (0.0007) [2023-10-10 19:55:23,641][123614] Updated weights for policy 1, policy_version 85410 (0.0008) [2023-10-10 19:55:23,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 175046656. Throughput: 0: 1824.2, 1: 1815.1. Samples: 43774228. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:55:23,789][122664] Avg episode reward: [(0, '80.740'), (1, '80.020')] [2023-10-10 19:55:24,017][123614] Updated weights for policy 1, policy_version 85420 (0.0012) [2023-10-10 19:55:24,381][123614] Updated weights for policy 1, policy_version 85430 (0.0008) [2023-10-10 19:55:24,746][123614] Updated weights for policy 1, policy_version 85440 (0.0009) [2023-10-10 19:55:26,354][123582] Updated weights for policy 0, policy_version 85543 (0.0007) [2023-10-10 19:55:26,740][123582] Updated weights for policy 0, policy_version 85553 (0.0009) [2023-10-10 19:55:27,107][123582] Updated weights for policy 0, policy_version 85563 (0.0007) [2023-10-10 19:55:28,552][123614] Updated weights for policy 1, policy_version 85450 (0.0007) [2023-10-10 19:55:28,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 175112192. Throughput: 0: 1821.8, 1: 1818.3. Samples: 43785380. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:55:28,788][122664] Avg episode reward: [(0, '79.990'), (1, '75.920')] [2023-10-10 19:55:28,925][123614] Updated weights for policy 1, policy_version 85460 (0.0008) [2023-10-10 19:55:29,285][123614] Updated weights for policy 1, policy_version 85470 (0.0007) [2023-10-10 19:55:30,846][123582] Updated weights for policy 0, policy_version 85573 (0.0009) [2023-10-10 19:55:31,210][123582] Updated weights for policy 0, policy_version 85583 (0.0007) [2023-10-10 19:55:31,589][123582] Updated weights for policy 0, policy_version 85593 (0.0008) [2023-10-10 19:55:32,977][123614] Updated weights for policy 1, policy_version 85480 (0.0007) [2023-10-10 19:55:33,343][123614] Updated weights for policy 1, policy_version 85490 (0.0007) [2023-10-10 19:55:33,716][123614] Updated weights for policy 1, policy_version 85500 (0.0010) [2023-10-10 19:55:33,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 175177728. Throughput: 0: 1816.9, 1: 1820.9. Samples: 43807146. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:55:33,788][122664] Avg episode reward: [(0, '80.050'), (1, '74.800')] [2023-10-10 19:55:35,221][123582] Updated weights for policy 0, policy_version 85603 (0.0011) [2023-10-10 19:55:35,593][123582] Updated weights for policy 0, policy_version 85613 (0.0010) [2023-10-10 19:55:35,965][123582] Updated weights for policy 0, policy_version 85623 (0.0007) [2023-10-10 19:55:37,322][123614] Updated weights for policy 1, policy_version 85510 (0.0008) [2023-10-10 19:55:37,687][123614] Updated weights for policy 1, policy_version 85520 (0.0007) [2023-10-10 19:55:38,067][123614] Updated weights for policy 1, policy_version 85530 (0.0008) [2023-10-10 19:55:38,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 175276032. Throughput: 0: 1820.5, 1: 1825.1. Samples: 43828752. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:55:38,789][122664] Avg episode reward: [(0, '81.220'), (1, '82.210')] [2023-10-10 19:55:39,402][123582] Updated weights for policy 0, policy_version 85633 (0.0009) [2023-10-10 19:55:39,764][123582] Updated weights for policy 0, policy_version 85643 (0.0007) [2023-10-10 19:55:40,142][123582] Updated weights for policy 0, policy_version 85653 (0.0008) [2023-10-10 19:55:40,509][123582] Updated weights for policy 0, policy_version 85663 (0.0007) [2023-10-10 19:55:41,717][123614] Updated weights for policy 1, policy_version 85540 (0.0010) [2023-10-10 19:55:42,091][123614] Updated weights for policy 1, policy_version 85550 (0.0009) [2023-10-10 19:55:42,462][123614] Updated weights for policy 1, policy_version 85560 (0.0009) [2023-10-10 19:55:43,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 175341568. Throughput: 0: 1826.4, 1: 1824.2. Samples: 43840134. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:55:43,789][122664] Avg episode reward: [(0, '86.360'), (1, '81.060')] [2023-10-10 19:55:44,222][123582] Updated weights for policy 0, policy_version 85673 (0.0007) [2023-10-10 19:55:44,595][123582] Updated weights for policy 0, policy_version 85683 (0.0010) [2023-10-10 19:55:44,960][123582] Updated weights for policy 0, policy_version 85693 (0.0010) [2023-10-10 19:55:46,202][123614] Updated weights for policy 1, policy_version 85570 (0.0008) [2023-10-10 19:55:46,572][123614] Updated weights for policy 1, policy_version 85580 (0.0008) [2023-10-10 19:55:46,954][123614] Updated weights for policy 1, policy_version 85590 (0.0008) [2023-10-10 19:55:47,315][123614] Updated weights for policy 1, policy_version 85600 (0.0007) [2023-10-10 19:55:48,535][123582] Updated weights for policy 0, policy_version 85703 (0.0010) [2023-10-10 19:55:48,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 175407104. Throughput: 0: 1826.1, 1: 1822.4. Samples: 43861530. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:55:48,788][122664] Avg episode reward: [(0, '88.830'), (1, '83.130')] [2023-10-10 19:55:48,908][123582] Updated weights for policy 0, policy_version 85713 (0.0010) [2023-10-10 19:55:49,281][123582] Updated weights for policy 0, policy_version 85723 (0.0009) [2023-10-10 19:55:51,067][123614] Updated weights for policy 1, policy_version 85610 (0.0007) [2023-10-10 19:55:51,442][123614] Updated weights for policy 1, policy_version 85620 (0.0008) [2023-10-10 19:55:51,808][123614] Updated weights for policy 1, policy_version 85630 (0.0009) [2023-10-10 19:55:52,949][123582] Updated weights for policy 0, policy_version 85733 (0.0009) [2023-10-10 19:55:53,320][123582] Updated weights for policy 0, policy_version 85743 (0.0010) [2023-10-10 19:55:53,692][123582] Updated weights for policy 0, policy_version 85753 (0.0008) [2023-10-10 19:55:53,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 175472640. Throughput: 0: 1819.3, 1: 1823.3. Samples: 43883480. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:55:53,788][122664] Avg episode reward: [(0, '89.200'), (1, '84.080')] [2023-10-10 19:55:53,800][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000085632_87687168.pth... [2023-10-10 19:55:53,835][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000083936_85950464.pth [2023-10-10 19:55:53,949][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000085760_87818240.pth... [2023-10-10 19:55:53,980][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000084032_86048768.pth [2023-10-10 19:55:55,519][123614] Updated weights for policy 1, policy_version 85640 (0.0009) [2023-10-10 19:55:55,887][123614] Updated weights for policy 1, policy_version 85650 (0.0010) [2023-10-10 19:55:56,260][123614] Updated weights for policy 1, policy_version 85660 (0.0009) [2023-10-10 19:55:57,592][123582] Updated weights for policy 0, policy_version 85763 (0.0008) [2023-10-10 19:55:57,965][123582] Updated weights for policy 0, policy_version 85773 (0.0007) [2023-10-10 19:55:58,339][123582] Updated weights for policy 0, policy_version 85783 (0.0007) [2023-10-10 19:55:58,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 175570944. Throughput: 0: 1825.5, 1: 1820.0. Samples: 43894106. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:55:58,788][122664] Avg episode reward: [(0, '89.930'), (1, '82.600')] [2023-10-10 19:56:00,103][123614] Updated weights for policy 1, policy_version 85670 (0.0008) [2023-10-10 19:56:00,478][123614] Updated weights for policy 1, policy_version 85680 (0.0010) [2023-10-10 19:56:00,842][123614] Updated weights for policy 1, policy_version 85690 (0.0007) [2023-10-10 19:56:02,162][123582] Updated weights for policy 0, policy_version 85793 (0.0008) [2023-10-10 19:56:02,532][123582] Updated weights for policy 0, policy_version 85803 (0.0007) [2023-10-10 19:56:02,914][123582] Updated weights for policy 0, policy_version 85813 (0.0007) [2023-10-10 19:56:03,285][123582] Updated weights for policy 0, policy_version 85823 (0.0007) [2023-10-10 19:56:03,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 175636480. Throughput: 0: 1826.5, 1: 1815.7. Samples: 43916290. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:56:03,788][122664] Avg episode reward: [(0, '83.660'), (1, '77.450')] [2023-10-10 19:56:04,499][123614] Updated weights for policy 1, policy_version 85700 (0.0009) [2023-10-10 19:56:04,868][123614] Updated weights for policy 1, policy_version 85710 (0.0008) [2023-10-10 19:56:05,230][123614] Updated weights for policy 1, policy_version 85720 (0.0007) [2023-10-10 19:56:06,853][123582] Updated weights for policy 0, policy_version 85833 (0.0010) [2023-10-10 19:56:07,222][123582] Updated weights for policy 0, policy_version 85843 (0.0011) [2023-10-10 19:56:07,588][123582] Updated weights for policy 0, policy_version 85853 (0.0010) [2023-10-10 19:56:08,788][123614] Updated weights for policy 1, policy_version 85730 (0.0008) [2023-10-10 19:56:08,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 175702016. Throughput: 0: 1819.2, 1: 1824.4. Samples: 43938194. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:56:08,789][122664] Avg episode reward: [(0, '81.410'), (1, '77.260')] [2023-10-10 19:56:09,158][123614] Updated weights for policy 1, policy_version 85740 (0.0009) [2023-10-10 19:56:09,527][123614] Updated weights for policy 1, policy_version 85750 (0.0011) [2023-10-10 19:56:09,894][123614] Updated weights for policy 1, policy_version 85760 (0.0010) [2023-10-10 19:56:11,491][123582] Updated weights for policy 0, policy_version 85863 (0.0009) [2023-10-10 19:56:11,855][123582] Updated weights for policy 0, policy_version 85873 (0.0008) [2023-10-10 19:56:12,237][123582] Updated weights for policy 0, policy_version 85883 (0.0009) [2023-10-10 19:56:13,580][123614] Updated weights for policy 1, policy_version 85770 (0.0007) [2023-10-10 19:56:13,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 175767552. Throughput: 0: 1819.6, 1: 1822.0. Samples: 43949252. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:56:13,789][122664] Avg episode reward: [(0, '77.840'), (1, '77.890')] [2023-10-10 19:56:13,948][123614] Updated weights for policy 1, policy_version 85780 (0.0007) [2023-10-10 19:56:14,314][123614] Updated weights for policy 1, policy_version 85790 (0.0008) [2023-10-10 19:56:15,995][123582] Updated weights for policy 0, policy_version 85893 (0.0009) [2023-10-10 19:56:16,363][123582] Updated weights for policy 0, policy_version 85903 (0.0008) [2023-10-10 19:56:16,730][123582] Updated weights for policy 0, policy_version 85913 (0.0008) [2023-10-10 19:56:18,001][123614] Updated weights for policy 1, policy_version 85800 (0.0009) [2023-10-10 19:56:18,377][123614] Updated weights for policy 1, policy_version 85810 (0.0010) [2023-10-10 19:56:18,738][123614] Updated weights for policy 1, policy_version 85820 (0.0010) [2023-10-10 19:56:18,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 175833088. Throughput: 0: 1811.9, 1: 1818.5. Samples: 43970514. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:56:18,788][122664] Avg episode reward: [(0, '82.560'), (1, '74.480')] [2023-10-10 19:56:20,547][123582] Updated weights for policy 0, policy_version 85923 (0.0010) [2023-10-10 19:56:20,918][123582] Updated weights for policy 0, policy_version 85933 (0.0010) [2023-10-10 19:56:21,287][123582] Updated weights for policy 0, policy_version 85943 (0.0010) [2023-10-10 19:56:22,503][123614] Updated weights for policy 1, policy_version 85830 (0.0007) [2023-10-10 19:56:22,877][123614] Updated weights for policy 1, policy_version 85840 (0.0010) [2023-10-10 19:56:23,247][123614] Updated weights for policy 1, policy_version 85850 (0.0009) [2023-10-10 19:56:23,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 175931392. Throughput: 0: 1805.7, 1: 1812.9. Samples: 43991588. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:56:23,789][122664] Avg episode reward: [(0, '86.960'), (1, '77.920')] [2023-10-10 19:56:25,042][123582] Updated weights for policy 0, policy_version 85953 (0.0007) [2023-10-10 19:56:25,417][123582] Updated weights for policy 0, policy_version 85963 (0.0009) [2023-10-10 19:56:25,782][123582] Updated weights for policy 0, policy_version 85973 (0.0007) [2023-10-10 19:56:26,161][123582] Updated weights for policy 0, policy_version 85983 (0.0007) [2023-10-10 19:56:26,878][123614] Updated weights for policy 1, policy_version 85860 (0.0009) [2023-10-10 19:56:27,262][123614] Updated weights for policy 1, policy_version 85870 (0.0010) [2023-10-10 19:56:27,635][123614] Updated weights for policy 1, policy_version 85880 (0.0009) [2023-10-10 19:56:28,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 175996928. Throughput: 0: 1797.6, 1: 1821.7. Samples: 44002998. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:56:28,788][122664] Avg episode reward: [(0, '84.300'), (1, '78.530')] [2023-10-10 19:56:29,850][123582] Updated weights for policy 0, policy_version 85993 (0.0007) [2023-10-10 19:56:30,224][123582] Updated weights for policy 0, policy_version 86003 (0.0008) [2023-10-10 19:56:30,585][123582] Updated weights for policy 0, policy_version 86013 (0.0008) [2023-10-10 19:56:31,176][123614] Updated weights for policy 1, policy_version 85890 (0.0008) [2023-10-10 19:56:31,548][123614] Updated weights for policy 1, policy_version 85900 (0.0009) [2023-10-10 19:56:31,916][123614] Updated weights for policy 1, policy_version 85910 (0.0007) [2023-10-10 19:56:32,286][123614] Updated weights for policy 1, policy_version 85920 (0.0009) [2023-10-10 19:56:33,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 176062464. Throughput: 0: 1798.2, 1: 1819.6. Samples: 44024332. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:56:33,789][122664] Avg episode reward: [(0, '80.670'), (1, '80.430')] [2023-10-10 19:56:34,277][123582] Updated weights for policy 0, policy_version 86023 (0.0008) [2023-10-10 19:56:34,650][123582] Updated weights for policy 0, policy_version 86033 (0.0007) [2023-10-10 19:56:35,027][123582] Updated weights for policy 0, policy_version 86043 (0.0007) [2023-10-10 19:56:36,060][123614] Updated weights for policy 1, policy_version 85930 (0.0009) [2023-10-10 19:56:36,427][123614] Updated weights for policy 1, policy_version 85940 (0.0009) [2023-10-10 19:56:36,799][123614] Updated weights for policy 1, policy_version 85950 (0.0007) [2023-10-10 19:56:38,532][123582] Updated weights for policy 0, policy_version 86053 (0.0010) [2023-10-10 19:56:38,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 176128000. Throughput: 0: 1815.7, 1: 1814.2. Samples: 44046828. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:56:38,789][122664] Avg episode reward: [(0, '79.400'), (1, '83.180')] [2023-10-10 19:56:38,902][123582] Updated weights for policy 0, policy_version 86063 (0.0007) [2023-10-10 19:56:39,275][123582] Updated weights for policy 0, policy_version 86073 (0.0007) [2023-10-10 19:56:40,513][123614] Updated weights for policy 1, policy_version 85960 (0.0008) [2023-10-10 19:56:40,891][123614] Updated weights for policy 1, policy_version 85970 (0.0010) [2023-10-10 19:56:41,257][123614] Updated weights for policy 1, policy_version 85980 (0.0007) [2023-10-10 19:56:43,070][123582] Updated weights for policy 0, policy_version 86083 (0.0007) [2023-10-10 19:56:43,454][123582] Updated weights for policy 0, policy_version 86093 (0.0008) [2023-10-10 19:56:43,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 176193536. Throughput: 0: 1798.7, 1: 1818.5. Samples: 44056880. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-10 19:56:43,789][122664] Avg episode reward: [(0, '82.690'), (1, '85.080')] [2023-10-10 19:56:43,832][123582] Updated weights for policy 0, policy_version 86103 (0.0009) [2023-10-10 19:56:44,873][123614] Updated weights for policy 1, policy_version 85990 (0.0008) [2023-10-10 19:56:45,247][123614] Updated weights for policy 1, policy_version 86000 (0.0008) [2023-10-10 19:56:45,613][123614] Updated weights for policy 1, policy_version 86010 (0.0010) [2023-10-10 19:56:47,363][123582] Updated weights for policy 0, policy_version 86113 (0.0009) [2023-10-10 19:56:47,732][123582] Updated weights for policy 0, policy_version 86123 (0.0010) [2023-10-10 19:56:48,108][123582] Updated weights for policy 0, policy_version 86133 (0.0010) [2023-10-10 19:56:48,481][123582] Updated weights for policy 0, policy_version 86143 (0.0010) [2023-10-10 19:56:48,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 176291840. Throughput: 0: 1809.6, 1: 1817.9. Samples: 44079528. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-10 19:56:48,788][122664] Avg episode reward: [(0, '87.440'), (1, '85.750')] [2023-10-10 19:56:49,397][123614] Updated weights for policy 1, policy_version 86020 (0.0010) [2023-10-10 19:56:49,768][123614] Updated weights for policy 1, policy_version 86030 (0.0010) [2023-10-10 19:56:50,136][123614] Updated weights for policy 1, policy_version 86040 (0.0009) [2023-10-10 19:56:52,332][123582] Updated weights for policy 0, policy_version 86153 (0.0008) [2023-10-10 19:56:52,701][123582] Updated weights for policy 0, policy_version 86163 (0.0009) [2023-10-10 19:56:53,072][123582] Updated weights for policy 0, policy_version 86173 (0.0010) [2023-10-10 19:56:53,744][123614] Updated weights for policy 1, policy_version 86050 (0.0007) [2023-10-10 19:56:53,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 176357376. Throughput: 0: 1799.0, 1: 1813.7. Samples: 44100768. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-10 19:56:53,789][122664] Avg episode reward: [(0, '88.120'), (1, '89.220')] [2023-10-10 19:56:54,124][123614] Updated weights for policy 1, policy_version 86060 (0.0007) [2023-10-10 19:56:54,483][123614] Updated weights for policy 1, policy_version 86070 (0.0009) [2023-10-10 19:56:54,849][123614] Updated weights for policy 1, policy_version 86080 (0.0007) [2023-10-10 19:56:56,884][123582] Updated weights for policy 0, policy_version 86183 (0.0007) [2023-10-10 19:56:57,258][123582] Updated weights for policy 0, policy_version 86193 (0.0008) [2023-10-10 19:56:57,622][123582] Updated weights for policy 0, policy_version 86203 (0.0009) [2023-10-10 19:56:58,720][123614] Updated weights for policy 1, policy_version 86090 (0.0007) [2023-10-10 19:56:58,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 176422912. Throughput: 0: 1802.6, 1: 1810.6. Samples: 44111846. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-10 19:56:58,788][122664] Avg episode reward: [(0, '89.780'), (1, '78.820')] [2023-10-10 19:56:59,085][123614] Updated weights for policy 1, policy_version 86100 (0.0007) [2023-10-10 19:56:59,453][123614] Updated weights for policy 1, policy_version 86110 (0.0007) [2023-10-10 19:57:01,312][123582] Updated weights for policy 0, policy_version 86213 (0.0008) [2023-10-10 19:57:01,685][123582] Updated weights for policy 0, policy_version 86223 (0.0009) [2023-10-10 19:57:02,058][123582] Updated weights for policy 0, policy_version 86233 (0.0010) [2023-10-10 19:57:03,294][123614] Updated weights for policy 1, policy_version 86120 (0.0009) [2023-10-10 19:57:03,658][123614] Updated weights for policy 1, policy_version 86130 (0.0009) [2023-10-10 19:57:03,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 176488448. Throughput: 0: 1801.1, 1: 1811.6. Samples: 44133086. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-10 19:57:03,789][122664] Avg episode reward: [(0, '84.470'), (1, '79.750')] [2023-10-10 19:57:04,040][123614] Updated weights for policy 1, policy_version 86140 (0.0009) [2023-10-10 19:57:05,831][123582] Updated weights for policy 0, policy_version 86243 (0.0010) [2023-10-10 19:57:06,195][123582] Updated weights for policy 0, policy_version 86253 (0.0008) [2023-10-10 19:57:06,571][123582] Updated weights for policy 0, policy_version 86263 (0.0011) [2023-10-10 19:57:07,582][123614] Updated weights for policy 1, policy_version 86150 (0.0010) [2023-10-10 19:57:07,953][123614] Updated weights for policy 1, policy_version 86160 (0.0008) [2023-10-10 19:57:08,316][123614] Updated weights for policy 1, policy_version 86170 (0.0008) [2023-10-10 19:57:08,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 176586752. Throughput: 0: 1805.4, 1: 1815.8. Samples: 44154542. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-10 19:57:08,788][122664] Avg episode reward: [(0, '86.840'), (1, '79.430')] [2023-10-10 19:57:10,264][123582] Updated weights for policy 0, policy_version 86273 (0.0010) [2023-10-10 19:57:10,638][123582] Updated weights for policy 0, policy_version 86283 (0.0007) [2023-10-10 19:57:11,013][123582] Updated weights for policy 0, policy_version 86293 (0.0009) [2023-10-10 19:57:11,378][123582] Updated weights for policy 0, policy_version 86303 (0.0008) [2023-10-10 19:57:11,917][123614] Updated weights for policy 1, policy_version 86180 (0.0008) [2023-10-10 19:57:12,278][123614] Updated weights for policy 1, policy_version 86190 (0.0009) [2023-10-10 19:57:12,648][123614] Updated weights for policy 1, policy_version 86200 (0.0009) [2023-10-10 19:57:13,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 176652288. Throughput: 0: 1810.5, 1: 1809.3. Samples: 44165890. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-10 19:57:13,789][122664] Avg episode reward: [(0, '83.150'), (1, '82.570')] [2023-10-10 19:57:14,964][123582] Updated weights for policy 0, policy_version 86313 (0.0009) [2023-10-10 19:57:15,345][123582] Updated weights for policy 0, policy_version 86323 (0.0009) [2023-10-10 19:57:15,716][123582] Updated weights for policy 0, policy_version 86333 (0.0008) [2023-10-10 19:57:16,313][123614] Updated weights for policy 1, policy_version 86210 (0.0009) [2023-10-10 19:57:16,672][123614] Updated weights for policy 1, policy_version 86220 (0.0008) [2023-10-10 19:57:17,045][123614] Updated weights for policy 1, policy_version 86230 (0.0008) [2023-10-10 19:57:17,409][123614] Updated weights for policy 1, policy_version 86240 (0.0008) [2023-10-10 19:57:18,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 176717824. Throughput: 0: 1814.5, 1: 1810.1. Samples: 44187442. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-10 19:57:18,789][122664] Avg episode reward: [(0, '80.310'), (1, '83.900')] [2023-10-10 19:57:19,449][123582] Updated weights for policy 0, policy_version 86343 (0.0007) [2023-10-10 19:57:19,809][123582] Updated weights for policy 0, policy_version 86353 (0.0009) [2023-10-10 19:57:20,179][123582] Updated weights for policy 0, policy_version 86363 (0.0008) [2023-10-10 19:57:21,164][123614] Updated weights for policy 1, policy_version 86250 (0.0011) [2023-10-10 19:57:21,517][123614] Updated weights for policy 1, policy_version 86260 (0.0010) [2023-10-10 19:57:21,884][123614] Updated weights for policy 1, policy_version 86270 (0.0007) [2023-10-10 19:57:23,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 176783360. Throughput: 0: 1809.8, 1: 1816.6. Samples: 44210018. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-10 19:57:23,789][122664] Avg episode reward: [(0, '82.910'), (1, '87.730')] [2023-10-10 19:57:23,969][123582] Updated weights for policy 0, policy_version 86373 (0.0009) [2023-10-10 19:57:24,331][123582] Updated weights for policy 0, policy_version 86383 (0.0008) [2023-10-10 19:57:24,705][123582] Updated weights for policy 0, policy_version 86393 (0.0007) [2023-10-10 19:57:25,566][123614] Updated weights for policy 1, policy_version 86280 (0.0009) [2023-10-10 19:57:25,934][123614] Updated weights for policy 1, policy_version 86290 (0.0009) [2023-10-10 19:57:26,314][123614] Updated weights for policy 1, policy_version 86300 (0.0010) [2023-10-10 19:57:28,217][123582] Updated weights for policy 0, policy_version 86403 (0.0008) [2023-10-10 19:57:28,591][123582] Updated weights for policy 0, policy_version 86413 (0.0011) [2023-10-10 19:57:28,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 176848896. Throughput: 0: 1809.3, 1: 1813.1. Samples: 44219888. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-10 19:57:28,788][122664] Avg episode reward: [(0, '84.720'), (1, '92.390')] [2023-10-10 19:57:28,968][123582] Updated weights for policy 0, policy_version 86423 (0.0008) [2023-10-10 19:57:30,027][123614] Updated weights for policy 1, policy_version 86310 (0.0010) [2023-10-10 19:57:30,395][123614] Updated weights for policy 1, policy_version 86320 (0.0011) [2023-10-10 19:57:30,766][123614] Updated weights for policy 1, policy_version 86330 (0.0010) [2023-10-10 19:57:32,673][123582] Updated weights for policy 0, policy_version 86433 (0.0009) [2023-10-10 19:57:33,042][123582] Updated weights for policy 0, policy_version 86443 (0.0007) [2023-10-10 19:57:33,405][123582] Updated weights for policy 0, policy_version 86453 (0.0008) [2023-10-10 19:57:33,780][123582] Updated weights for policy 0, policy_version 86463 (0.0008) [2023-10-10 19:57:33,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 176914432. Throughput: 0: 1817.3, 1: 1812.6. Samples: 44242872. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-10 19:57:33,789][122664] Avg episode reward: [(0, '89.530'), (1, '90.800')] [2023-10-10 19:57:34,432][123614] Updated weights for policy 1, policy_version 86340 (0.0009) [2023-10-10 19:57:34,793][123614] Updated weights for policy 1, policy_version 86350 (0.0011) [2023-10-10 19:57:35,161][123614] Updated weights for policy 1, policy_version 86360 (0.0007) [2023-10-10 19:57:37,512][123582] Updated weights for policy 0, policy_version 86473 (0.0008) [2023-10-10 19:57:37,878][123582] Updated weights for policy 0, policy_version 86483 (0.0008) [2023-10-10 19:57:38,254][123582] Updated weights for policy 0, policy_version 86493 (0.0008) [2023-10-10 19:57:38,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 177012736. Throughput: 0: 1815.5, 1: 1817.6. Samples: 44264256. Policy #0 lag: (min: 0.0, avg: 22.2, max: 32.0) [2023-10-10 19:57:38,789][122664] Avg episode reward: [(0, '90.990'), (1, '92.270')] [2023-10-10 19:57:38,946][123614] Updated weights for policy 1, policy_version 86370 (0.0008) [2023-10-10 19:57:39,314][123614] Updated weights for policy 1, policy_version 86380 (0.0008) [2023-10-10 19:57:39,687][123614] Updated weights for policy 1, policy_version 86390 (0.0012) [2023-10-10 19:57:40,051][123614] Updated weights for policy 1, policy_version 86400 (0.0011) [2023-10-10 19:57:42,158][123582] Updated weights for policy 0, policy_version 86503 (0.0009) [2023-10-10 19:57:42,548][123582] Updated weights for policy 0, policy_version 86513 (0.0009) [2023-10-10 19:57:42,913][123582] Updated weights for policy 0, policy_version 86523 (0.0007) [2023-10-10 19:57:43,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 177078272. Throughput: 0: 1818.5, 1: 1817.4. Samples: 44275460. Policy #0 lag: (min: 0.0, avg: 22.2, max: 32.0) [2023-10-10 19:57:43,789][122664] Avg episode reward: [(0, '92.150'), (1, '90.200')] [2023-10-10 19:57:43,884][123614] Updated weights for policy 1, policy_version 86410 (0.0007) [2023-10-10 19:57:44,250][123614] Updated weights for policy 1, policy_version 86420 (0.0008) [2023-10-10 19:57:44,620][123614] Updated weights for policy 1, policy_version 86430 (0.0008) [2023-10-10 19:57:46,546][123582] Updated weights for policy 0, policy_version 86533 (0.0008) [2023-10-10 19:57:46,918][123582] Updated weights for policy 0, policy_version 86543 (0.0008) [2023-10-10 19:57:47,288][123582] Updated weights for policy 0, policy_version 86553 (0.0007) [2023-10-10 19:57:48,269][123614] Updated weights for policy 1, policy_version 86440 (0.0007) [2023-10-10 19:57:48,640][123614] Updated weights for policy 1, policy_version 86450 (0.0007) [2023-10-10 19:57:48,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 177143808. Throughput: 0: 1823.1, 1: 1823.7. Samples: 44297190. Policy #0 lag: (min: 0.0, avg: 22.2, max: 32.0) [2023-10-10 19:57:48,788][122664] Avg episode reward: [(0, '92.170'), (1, '88.650')] [2023-10-10 19:57:48,998][123614] Updated weights for policy 1, policy_version 86460 (0.0007) [2023-10-10 19:57:50,923][123582] Updated weights for policy 0, policy_version 86563 (0.0007) [2023-10-10 19:57:51,299][123582] Updated weights for policy 0, policy_version 86573 (0.0007) [2023-10-10 19:57:51,672][123582] Updated weights for policy 0, policy_version 86583 (0.0008) [2023-10-10 19:57:52,612][123614] Updated weights for policy 1, policy_version 86470 (0.0007) [2023-10-10 19:57:52,987][123614] Updated weights for policy 1, policy_version 86480 (0.0008) [2023-10-10 19:57:53,356][123614] Updated weights for policy 1, policy_version 86490 (0.0008) [2023-10-10 19:57:53,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 177242112. Throughput: 0: 1818.7, 1: 1824.4. Samples: 44318484. Policy #0 lag: (min: 0.0, avg: 22.2, max: 32.0) [2023-10-10 19:57:53,789][122664] Avg episode reward: [(0, '90.590'), (1, '89.680')] [2023-10-10 19:57:53,797][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000086592_88670208.pth... [2023-10-10 19:57:53,798][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000086496_88571904.pth... [2023-10-10 19:57:53,834][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000084800_86835200.pth [2023-10-10 19:57:53,835][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000084896_86933504.pth [2023-10-10 19:57:55,479][123582] Updated weights for policy 0, policy_version 86593 (0.0008) [2023-10-10 19:57:55,859][123582] Updated weights for policy 0, policy_version 86603 (0.0008) [2023-10-10 19:57:56,238][123582] Updated weights for policy 0, policy_version 86613 (0.0007) [2023-10-10 19:57:56,610][123582] Updated weights for policy 0, policy_version 86623 (0.0007) [2023-10-10 19:57:57,054][123614] Updated weights for policy 1, policy_version 86500 (0.0010) [2023-10-10 19:57:57,429][123614] Updated weights for policy 1, policy_version 86510 (0.0008) [2023-10-10 19:57:57,815][123614] Updated weights for policy 1, policy_version 86520 (0.0009) [2023-10-10 19:57:58,788][122664] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 177307648. Throughput: 0: 1821.0, 1: 1828.6. Samples: 44330124. Policy #0 lag: (min: 0.0, avg: 22.2, max: 32.0) [2023-10-10 19:57:58,789][122664] Avg episode reward: [(0, '89.590'), (1, '86.890')] [2023-10-10 19:58:00,244][123582] Updated weights for policy 0, policy_version 86633 (0.0008) [2023-10-10 19:58:00,602][123582] Updated weights for policy 0, policy_version 86643 (0.0008) [2023-10-10 19:58:00,964][123582] Updated weights for policy 0, policy_version 86653 (0.0010) [2023-10-10 19:58:01,495][123614] Updated weights for policy 1, policy_version 86530 (0.0009) [2023-10-10 19:58:01,866][123614] Updated weights for policy 1, policy_version 86540 (0.0008) [2023-10-10 19:58:02,245][123614] Updated weights for policy 1, policy_version 86550 (0.0009) [2023-10-10 19:58:02,606][123614] Updated weights for policy 1, policy_version 86560 (0.0007) [2023-10-10 19:58:03,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 177373184. Throughput: 0: 1815.7, 1: 1825.3. Samples: 44351282. Policy #0 lag: (min: 0.0, avg: 22.2, max: 32.0) [2023-10-10 19:58:03,788][122664] Avg episode reward: [(0, '96.190'), (1, '92.780')] [2023-10-10 19:58:04,687][123582] Updated weights for policy 0, policy_version 86663 (0.0008) [2023-10-10 19:58:05,057][123582] Updated weights for policy 0, policy_version 86673 (0.0009) [2023-10-10 19:58:05,428][123582] Updated weights for policy 0, policy_version 86683 (0.0008) [2023-10-10 19:58:06,240][123614] Updated weights for policy 1, policy_version 86570 (0.0008) [2023-10-10 19:58:06,607][123614] Updated weights for policy 1, policy_version 86580 (0.0007) [2023-10-10 19:58:06,974][123614] Updated weights for policy 1, policy_version 86590 (0.0007) [2023-10-10 19:58:08,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 177438720. Throughput: 0: 1822.0, 1: 1828.4. Samples: 44374284. Policy #0 lag: (min: 0.0, avg: 22.2, max: 32.0) [2023-10-10 19:58:08,788][122664] Avg episode reward: [(0, '100.450'), (1, '93.670')] [2023-10-10 19:58:08,998][123582] Updated weights for policy 0, policy_version 86693 (0.0010) [2023-10-10 19:58:09,370][123582] Updated weights for policy 0, policy_version 86703 (0.0009) [2023-10-10 19:58:09,744][123582] Updated weights for policy 0, policy_version 86713 (0.0008) [2023-10-10 19:58:10,580][123614] Updated weights for policy 1, policy_version 86600 (0.0009) [2023-10-10 19:58:10,943][123614] Updated weights for policy 1, policy_version 86610 (0.0010) [2023-10-10 19:58:11,310][123614] Updated weights for policy 1, policy_version 86620 (0.0008) [2023-10-10 19:58:13,412][123582] Updated weights for policy 0, policy_version 86723 (0.0009) [2023-10-10 19:58:13,779][123582] Updated weights for policy 0, policy_version 86733 (0.0008) [2023-10-10 19:58:13,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 177504256. Throughput: 0: 1819.0, 1: 1832.8. Samples: 44384216. Policy #0 lag: (min: 0.0, avg: 22.2, max: 32.0) [2023-10-10 19:58:13,789][122664] Avg episode reward: [(0, '104.740'), (1, '92.570')] [2023-10-10 19:58:14,144][123582] Updated weights for policy 0, policy_version 86743 (0.0007) [2023-10-10 19:58:14,982][123614] Updated weights for policy 1, policy_version 86630 (0.0008) [2023-10-10 19:58:15,345][123614] Updated weights for policy 1, policy_version 86640 (0.0010) [2023-10-10 19:58:15,715][123614] Updated weights for policy 1, policy_version 86650 (0.0011) [2023-10-10 19:58:17,828][123582] Updated weights for policy 0, policy_version 86753 (0.0008) [2023-10-10 19:58:18,205][123582] Updated weights for policy 0, policy_version 86763 (0.0012) [2023-10-10 19:58:18,574][123582] Updated weights for policy 0, policy_version 86773 (0.0010) [2023-10-10 19:58:18,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 177569792. Throughput: 0: 1810.6, 1: 1835.2. Samples: 44406936. Policy #0 lag: (min: 0.0, avg: 22.2, max: 32.0) [2023-10-10 19:58:18,789][122664] Avg episode reward: [(0, '101.660'), (1, '90.320')] [2023-10-10 19:58:18,942][123582] Updated weights for policy 0, policy_version 86783 (0.0009) [2023-10-10 19:58:19,365][123614] Updated weights for policy 1, policy_version 86660 (0.0009) [2023-10-10 19:58:19,734][123614] Updated weights for policy 1, policy_version 86670 (0.0008) [2023-10-10 19:58:20,102][123614] Updated weights for policy 1, policy_version 86680 (0.0008) [2023-10-10 19:58:22,780][123582] Updated weights for policy 0, policy_version 86793 (0.0007) [2023-10-10 19:58:23,153][123582] Updated weights for policy 0, policy_version 86803 (0.0008) [2023-10-10 19:58:23,521][123582] Updated weights for policy 0, policy_version 86813 (0.0008) [2023-10-10 19:58:23,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 177668096. Throughput: 0: 1816.9, 1: 1829.7. Samples: 44428352. Policy #0 lag: (min: 0.0, avg: 22.2, max: 32.0) [2023-10-10 19:58:23,789][122664] Avg episode reward: [(0, '98.600'), (1, '96.150')] [2023-10-10 19:58:23,801][123614] Updated weights for policy 1, policy_version 86690 (0.0007) [2023-10-10 19:58:24,177][123614] Updated weights for policy 1, policy_version 86700 (0.0010) [2023-10-10 19:58:24,546][123614] Updated weights for policy 1, policy_version 86710 (0.0011) [2023-10-10 19:58:24,906][123614] Updated weights for policy 1, policy_version 86720 (0.0011) [2023-10-10 19:58:27,390][123582] Updated weights for policy 0, policy_version 86823 (0.0008) [2023-10-10 19:58:27,765][123582] Updated weights for policy 0, policy_version 86833 (0.0009) [2023-10-10 19:58:28,135][123582] Updated weights for policy 0, policy_version 86843 (0.0009) [2023-10-10 19:58:28,673][123614] Updated weights for policy 1, policy_version 86730 (0.0010) [2023-10-10 19:58:28,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 177733632. Throughput: 0: 1812.2, 1: 1826.3. Samples: 44439192. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-10-10 19:58:28,789][122664] Avg episode reward: [(0, '101.210'), (1, '100.310')] [2023-10-10 19:58:29,042][123614] Updated weights for policy 1, policy_version 86740 (0.0008) [2023-10-10 19:58:29,422][123614] Updated weights for policy 1, policy_version 86750 (0.0008) [2023-10-10 19:58:31,871][123582] Updated weights for policy 0, policy_version 86853 (0.0009) [2023-10-10 19:58:32,236][123582] Updated weights for policy 0, policy_version 86863 (0.0008) [2023-10-10 19:58:32,607][123582] Updated weights for policy 0, policy_version 86873 (0.0008) [2023-10-10 19:58:33,198][123614] Updated weights for policy 1, policy_version 86760 (0.0007) [2023-10-10 19:58:33,566][123614] Updated weights for policy 1, policy_version 86770 (0.0008) [2023-10-10 19:58:33,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 177799168. Throughput: 0: 1817.2, 1: 1816.8. Samples: 44460718. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-10-10 19:58:33,789][122664] Avg episode reward: [(0, '101.700'), (1, '98.880')] [2023-10-10 19:58:33,940][123614] Updated weights for policy 1, policy_version 86780 (0.0010) [2023-10-10 19:58:36,297][123582] Updated weights for policy 0, policy_version 86883 (0.0008) [2023-10-10 19:58:36,662][123582] Updated weights for policy 0, policy_version 86893 (0.0008) [2023-10-10 19:58:37,035][123582] Updated weights for policy 0, policy_version 86903 (0.0008) [2023-10-10 19:58:37,527][123614] Updated weights for policy 1, policy_version 86790 (0.0009) [2023-10-10 19:58:37,898][123614] Updated weights for policy 1, policy_version 86800 (0.0009) [2023-10-10 19:58:38,265][123614] Updated weights for policy 1, policy_version 86810 (0.0007) [2023-10-10 19:58:38,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 177897472. Throughput: 0: 1809.9, 1: 1819.0. Samples: 44481782. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-10-10 19:58:38,789][122664] Avg episode reward: [(0, '100.600'), (1, '105.140')] [2023-10-10 19:58:40,548][123582] Updated weights for policy 0, policy_version 86913 (0.0007) [2023-10-10 19:58:40,921][123582] Updated weights for policy 0, policy_version 86923 (0.0008) [2023-10-10 19:58:41,300][123582] Updated weights for policy 0, policy_version 86933 (0.0008) [2023-10-10 19:58:41,676][123582] Updated weights for policy 0, policy_version 86943 (0.0007) [2023-10-10 19:58:42,083][123614] Updated weights for policy 1, policy_version 86820 (0.0007) [2023-10-10 19:58:42,482][123614] Updated weights for policy 1, policy_version 86830 (0.0007) [2023-10-10 19:58:42,850][123614] Updated weights for policy 1, policy_version 86840 (0.0007) [2023-10-10 19:58:43,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 177963008. Throughput: 0: 1818.2, 1: 1815.6. Samples: 44493646. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-10-10 19:58:43,789][122664] Avg episode reward: [(0, '101.190'), (1, '97.380')] [2023-10-10 19:58:45,316][123582] Updated weights for policy 0, policy_version 86953 (0.0009) [2023-10-10 19:58:45,689][123582] Updated weights for policy 0, policy_version 86963 (0.0009) [2023-10-10 19:58:46,057][123582] Updated weights for policy 0, policy_version 86973 (0.0009) [2023-10-10 19:58:46,467][123614] Updated weights for policy 1, policy_version 86850 (0.0008) [2023-10-10 19:58:46,834][123614] Updated weights for policy 1, policy_version 86860 (0.0009) [2023-10-10 19:58:47,208][123614] Updated weights for policy 1, policy_version 86870 (0.0010) [2023-10-10 19:58:47,576][123614] Updated weights for policy 1, policy_version 86880 (0.0011) [2023-10-10 19:58:48,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 178028544. Throughput: 0: 1807.6, 1: 1816.4. Samples: 44514364. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-10-10 19:58:48,788][122664] Avg episode reward: [(0, '100.210'), (1, '96.080')] [2023-10-10 19:58:49,682][123582] Updated weights for policy 0, policy_version 86983 (0.0010) [2023-10-10 19:58:50,047][123582] Updated weights for policy 0, policy_version 86993 (0.0010) [2023-10-10 19:58:50,422][123582] Updated weights for policy 0, policy_version 87003 (0.0011) [2023-10-10 19:58:51,238][123614] Updated weights for policy 1, policy_version 86890 (0.0008) [2023-10-10 19:58:51,609][123614] Updated weights for policy 1, policy_version 86900 (0.0007) [2023-10-10 19:58:51,979][123614] Updated weights for policy 1, policy_version 86910 (0.0010) [2023-10-10 19:58:53,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 178094080. Throughput: 0: 1806.0, 1: 1813.4. Samples: 44537160. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-10-10 19:58:53,789][122664] Avg episode reward: [(0, '99.300'), (1, '95.240')] [2023-10-10 19:58:54,067][123582] Updated weights for policy 0, policy_version 87013 (0.0009) [2023-10-10 19:58:54,435][123582] Updated weights for policy 0, policy_version 87023 (0.0009) [2023-10-10 19:58:54,806][123582] Updated weights for policy 0, policy_version 87033 (0.0010) [2023-10-10 19:58:55,724][123614] Updated weights for policy 1, policy_version 86920 (0.0010) [2023-10-10 19:58:56,092][123614] Updated weights for policy 1, policy_version 86930 (0.0009) [2023-10-10 19:58:56,459][123614] Updated weights for policy 1, policy_version 86940 (0.0008) [2023-10-10 19:58:58,724][123582] Updated weights for policy 0, policy_version 87043 (0.0008) [2023-10-10 19:58:58,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 178159616. Throughput: 0: 1810.1, 1: 1810.0. Samples: 44547120. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-10-10 19:58:58,789][122664] Avg episode reward: [(0, '97.470'), (1, '97.350')] [2023-10-10 19:58:59,105][123582] Updated weights for policy 0, policy_version 87053 (0.0007) [2023-10-10 19:58:59,470][123582] Updated weights for policy 0, policy_version 87063 (0.0007) [2023-10-10 19:59:00,194][123614] Updated weights for policy 1, policy_version 86950 (0.0009) [2023-10-10 19:59:00,566][123614] Updated weights for policy 1, policy_version 86960 (0.0011) [2023-10-10 19:59:00,941][123614] Updated weights for policy 1, policy_version 86970 (0.0009) [2023-10-10 19:59:03,158][123582] Updated weights for policy 0, policy_version 87073 (0.0010) [2023-10-10 19:59:03,520][123582] Updated weights for policy 0, policy_version 87083 (0.0007) [2023-10-10 19:59:03,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 178225152. Throughput: 0: 1812.4, 1: 1810.1. Samples: 44569952. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-10-10 19:59:03,789][122664] Avg episode reward: [(0, '96.040'), (1, '95.880')] [2023-10-10 19:59:03,900][123582] Updated weights for policy 0, policy_version 87093 (0.0009) [2023-10-10 19:59:04,270][123582] Updated weights for policy 0, policy_version 87103 (0.0008) [2023-10-10 19:59:04,543][123614] Updated weights for policy 1, policy_version 86980 (0.0008) [2023-10-10 19:59:04,916][123614] Updated weights for policy 1, policy_version 86990 (0.0010) [2023-10-10 19:59:05,283][123614] Updated weights for policy 1, policy_version 87000 (0.0009) [2023-10-10 19:59:07,850][123582] Updated weights for policy 0, policy_version 87113 (0.0009) [2023-10-10 19:59:08,229][123582] Updated weights for policy 0, policy_version 87123 (0.0010) [2023-10-10 19:59:08,593][123582] Updated weights for policy 0, policy_version 87133 (0.0008) [2023-10-10 19:59:08,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 178323456. Throughput: 0: 1817.1, 1: 1813.9. Samples: 44591748. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-10-10 19:59:08,789][122664] Avg episode reward: [(0, '92.980'), (1, '93.370')] [2023-10-10 19:59:08,912][123614] Updated weights for policy 1, policy_version 87010 (0.0008) [2023-10-10 19:59:09,281][123614] Updated weights for policy 1, policy_version 87020 (0.0012) [2023-10-10 19:59:09,645][123614] Updated weights for policy 1, policy_version 87030 (0.0008) [2023-10-10 19:59:10,018][123614] Updated weights for policy 1, policy_version 87040 (0.0008) [2023-10-10 19:59:12,235][123582] Updated weights for policy 0, policy_version 87143 (0.0008) [2023-10-10 19:59:12,617][123582] Updated weights for policy 0, policy_version 87153 (0.0008) [2023-10-10 19:59:12,984][123582] Updated weights for policy 0, policy_version 87163 (0.0009) [2023-10-10 19:59:13,556][123614] Updated weights for policy 1, policy_version 87050 (0.0011) [2023-10-10 19:59:13,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 178388992. Throughput: 0: 1819.9, 1: 1822.2. Samples: 44603086. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-10-10 19:59:13,789][122664] Avg episode reward: [(0, '87.060'), (1, '103.020')] [2023-10-10 19:59:13,925][123614] Updated weights for policy 1, policy_version 87060 (0.0011) [2023-10-10 19:59:14,290][123614] Updated weights for policy 1, policy_version 87070 (0.0009) [2023-10-10 19:59:16,568][123582] Updated weights for policy 0, policy_version 87173 (0.0009) [2023-10-10 19:59:16,937][123582] Updated weights for policy 0, policy_version 87183 (0.0011) [2023-10-10 19:59:17,317][123582] Updated weights for policy 0, policy_version 87193 (0.0010) [2023-10-10 19:59:18,120][123614] Updated weights for policy 1, policy_version 87080 (0.0008) [2023-10-10 19:59:18,491][123614] Updated weights for policy 1, policy_version 87090 (0.0009) [2023-10-10 19:59:18,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 178454528. Throughput: 0: 1817.1, 1: 1826.6. Samples: 44624684. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-10-10 19:59:18,789][122664] Avg episode reward: [(0, '85.810'), (1, '100.100')] [2023-10-10 19:59:18,861][123614] Updated weights for policy 1, policy_version 87100 (0.0008) [2023-10-10 19:59:21,064][123582] Updated weights for policy 0, policy_version 87203 (0.0010) [2023-10-10 19:59:21,441][123582] Updated weights for policy 0, policy_version 87213 (0.0011) [2023-10-10 19:59:21,819][123582] Updated weights for policy 0, policy_version 87223 (0.0010) [2023-10-10 19:59:22,583][123614] Updated weights for policy 1, policy_version 87110 (0.0010) [2023-10-10 19:59:22,943][123614] Updated weights for policy 1, policy_version 87120 (0.0008) [2023-10-10 19:59:23,314][123614] Updated weights for policy 1, policy_version 87130 (0.0008) [2023-10-10 19:59:23,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 178552832. Throughput: 0: 1817.6, 1: 1822.4. Samples: 44645582. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:59:23,789][122664] Avg episode reward: [(0, '84.690'), (1, '102.710')] [2023-10-10 19:59:25,532][123582] Updated weights for policy 0, policy_version 87233 (0.0007) [2023-10-10 19:59:25,918][123582] Updated weights for policy 0, policy_version 87243 (0.0007) [2023-10-10 19:59:26,283][123582] Updated weights for policy 0, policy_version 87253 (0.0010) [2023-10-10 19:59:26,656][123582] Updated weights for policy 0, policy_version 87263 (0.0008) [2023-10-10 19:59:26,980][123614] Updated weights for policy 1, policy_version 87140 (0.0007) [2023-10-10 19:59:27,362][123614] Updated weights for policy 1, policy_version 87150 (0.0008) [2023-10-10 19:59:27,741][123614] Updated weights for policy 1, policy_version 87160 (0.0009) [2023-10-10 19:59:28,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 178618368. Throughput: 0: 1814.1, 1: 1829.2. Samples: 44657594. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:59:28,788][122664] Avg episode reward: [(0, '86.290'), (1, '96.250')] [2023-10-10 19:59:30,411][123582] Updated weights for policy 0, policy_version 87273 (0.0009) [2023-10-10 19:59:30,781][123582] Updated weights for policy 0, policy_version 87283 (0.0008) [2023-10-10 19:59:31,154][123582] Updated weights for policy 0, policy_version 87293 (0.0008) [2023-10-10 19:59:31,493][123614] Updated weights for policy 1, policy_version 87170 (0.0009) [2023-10-10 19:59:31,861][123614] Updated weights for policy 1, policy_version 87180 (0.0008) [2023-10-10 19:59:32,231][123614] Updated weights for policy 1, policy_version 87190 (0.0008) [2023-10-10 19:59:32,608][123614] Updated weights for policy 1, policy_version 87200 (0.0007) [2023-10-10 19:59:33,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 178683904. Throughput: 0: 1813.3, 1: 1826.0. Samples: 44678132. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:59:33,789][122664] Avg episode reward: [(0, '80.440'), (1, '93.520')] [2023-10-10 19:59:34,803][123582] Updated weights for policy 0, policy_version 87303 (0.0007) [2023-10-10 19:59:35,179][123582] Updated weights for policy 0, policy_version 87313 (0.0009) [2023-10-10 19:59:35,544][123582] Updated weights for policy 0, policy_version 87323 (0.0008) [2023-10-10 19:59:36,152][123614] Updated weights for policy 1, policy_version 87210 (0.0009) [2023-10-10 19:59:36,512][123614] Updated weights for policy 1, policy_version 87220 (0.0010) [2023-10-10 19:59:36,871][123614] Updated weights for policy 1, policy_version 87230 (0.0007) [2023-10-10 19:59:38,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 178749440. Throughput: 0: 1816.9, 1: 1826.3. Samples: 44701104. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:59:38,789][122664] Avg episode reward: [(0, '85.190'), (1, '96.780')] [2023-10-10 19:59:39,043][123582] Updated weights for policy 0, policy_version 87333 (0.0009) [2023-10-10 19:59:39,408][123582] Updated weights for policy 0, policy_version 87343 (0.0008) [2023-10-10 19:59:39,785][123582] Updated weights for policy 0, policy_version 87353 (0.0010) [2023-10-10 19:59:40,566][123614] Updated weights for policy 1, policy_version 87240 (0.0009) [2023-10-10 19:59:40,930][123614] Updated weights for policy 1, policy_version 87250 (0.0008) [2023-10-10 19:59:41,296][123614] Updated weights for policy 1, policy_version 87260 (0.0008) [2023-10-10 19:59:43,394][123582] Updated weights for policy 0, policy_version 87363 (0.0008) [2023-10-10 19:59:43,769][123582] Updated weights for policy 0, policy_version 87373 (0.0008) [2023-10-10 19:59:43,788][122664] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 178814976. Throughput: 0: 1819.9, 1: 1829.9. Samples: 44711358. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:59:43,788][122664] Avg episode reward: [(0, '85.190'), (1, '98.620')] [2023-10-10 19:59:44,155][123582] Updated weights for policy 0, policy_version 87383 (0.0009) [2023-10-10 19:59:44,891][123614] Updated weights for policy 1, policy_version 87270 (0.0009) [2023-10-10 19:59:45,259][123614] Updated weights for policy 1, policy_version 87280 (0.0010) [2023-10-10 19:59:45,632][123614] Updated weights for policy 1, policy_version 87290 (0.0010) [2023-10-10 19:59:47,956][123582] Updated weights for policy 0, policy_version 87393 (0.0009) [2023-10-10 19:59:48,332][123582] Updated weights for policy 0, policy_version 87403 (0.0008) [2023-10-10 19:59:48,711][123582] Updated weights for policy 0, policy_version 87413 (0.0008) [2023-10-10 19:59:48,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 178880512. Throughput: 0: 1822.8, 1: 1827.6. Samples: 44734224. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:59:48,789][122664] Avg episode reward: [(0, '81.480'), (1, '98.740')] [2023-10-10 19:59:49,083][123582] Updated weights for policy 0, policy_version 87423 (0.0008) [2023-10-10 19:59:49,157][123614] Updated weights for policy 1, policy_version 87300 (0.0009) [2023-10-10 19:59:49,521][123614] Updated weights for policy 1, policy_version 87310 (0.0008) [2023-10-10 19:59:49,888][123614] Updated weights for policy 1, policy_version 87320 (0.0008) [2023-10-10 19:59:52,746][123582] Updated weights for policy 0, policy_version 87433 (0.0009) [2023-10-10 19:59:53,117][123582] Updated weights for policy 0, policy_version 87443 (0.0008) [2023-10-10 19:59:53,494][123582] Updated weights for policy 0, policy_version 87453 (0.0008) [2023-10-10 19:59:53,580][123614] Updated weights for policy 1, policy_version 87330 (0.0007) [2023-10-10 19:59:53,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 178978816. Throughput: 0: 1817.9, 1: 1817.9. Samples: 44755358. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:59:53,788][122664] Avg episode reward: [(0, '83.770'), (1, '92.840')] [2023-10-10 19:59:53,795][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000087456_89554944.pth... [2023-10-10 19:59:53,828][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000085760_87818240.pth [2023-10-10 19:59:53,951][123614] Updated weights for policy 1, policy_version 87340 (0.0009) [2023-10-10 19:59:54,323][123614] Updated weights for policy 1, policy_version 87350 (0.0007) [2023-10-10 19:59:54,686][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000087360_89456640.pth... [2023-10-10 19:59:54,690][123614] Updated weights for policy 1, policy_version 87360 (0.0008) [2023-10-10 19:59:54,716][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000085632_87687168.pth [2023-10-10 19:59:57,138][123582] Updated weights for policy 0, policy_version 87463 (0.0007) [2023-10-10 19:59:57,517][123582] Updated weights for policy 0, policy_version 87473 (0.0007) [2023-10-10 19:59:57,889][123582] Updated weights for policy 0, policy_version 87483 (0.0008) [2023-10-10 19:59:58,534][123614] Updated weights for policy 1, policy_version 87370 (0.0008) [2023-10-10 19:59:58,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 179044352. Throughput: 0: 1822.6, 1: 1817.3. Samples: 44766882. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 19:59:58,789][122664] Avg episode reward: [(0, '80.430'), (1, '96.550')] [2023-10-10 19:59:58,904][123614] Updated weights for policy 1, policy_version 87380 (0.0011) [2023-10-10 19:59:59,275][123614] Updated weights for policy 1, policy_version 87390 (0.0011) [2023-10-10 20:00:01,551][123582] Updated weights for policy 0, policy_version 87493 (0.0010) [2023-10-10 20:00:01,924][123582] Updated weights for policy 0, policy_version 87503 (0.0008) [2023-10-10 20:00:02,292][123582] Updated weights for policy 0, policy_version 87513 (0.0007) [2023-10-10 20:00:03,139][123614] Updated weights for policy 1, policy_version 87400 (0.0008) [2023-10-10 20:00:03,509][123614] Updated weights for policy 1, policy_version 87410 (0.0007) [2023-10-10 20:00:03,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 179109888. Throughput: 0: 1815.7, 1: 1814.4. Samples: 44788038. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:00:03,789][122664] Avg episode reward: [(0, '82.940'), (1, '95.910')] [2023-10-10 20:00:03,870][123614] Updated weights for policy 1, policy_version 87420 (0.0007) [2023-10-10 20:00:06,041][123582] Updated weights for policy 0, policy_version 87523 (0.0007) [2023-10-10 20:00:06,411][123582] Updated weights for policy 0, policy_version 87533 (0.0008) [2023-10-10 20:00:06,782][123582] Updated weights for policy 0, policy_version 87543 (0.0008) [2023-10-10 20:00:07,450][123614] Updated weights for policy 1, policy_version 87430 (0.0009) [2023-10-10 20:00:07,818][123614] Updated weights for policy 1, policy_version 87440 (0.0010) [2023-10-10 20:00:08,179][123614] Updated weights for policy 1, policy_version 87450 (0.0010) [2023-10-10 20:00:08,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 179208192. Throughput: 0: 1822.5, 1: 1814.6. Samples: 44809254. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:00:08,789][122664] Avg episode reward: [(0, '85.510'), (1, '93.590')] [2023-10-10 20:00:10,499][123582] Updated weights for policy 0, policy_version 87553 (0.0008) [2023-10-10 20:00:10,867][123582] Updated weights for policy 0, policy_version 87563 (0.0008) [2023-10-10 20:00:11,249][123582] Updated weights for policy 0, policy_version 87573 (0.0009) [2023-10-10 20:00:11,615][123582] Updated weights for policy 0, policy_version 87583 (0.0008) [2023-10-10 20:00:11,864][123614] Updated weights for policy 1, policy_version 87460 (0.0010) [2023-10-10 20:00:12,253][123614] Updated weights for policy 1, policy_version 87470 (0.0008) [2023-10-10 20:00:12,619][123614] Updated weights for policy 1, policy_version 87480 (0.0011) [2023-10-10 20:00:13,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 179273728. Throughput: 0: 1825.2, 1: 1804.8. Samples: 44820946. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:00:13,789][122664] Avg episode reward: [(0, '89.110'), (1, '89.380')] [2023-10-10 20:00:15,183][123582] Updated weights for policy 0, policy_version 87593 (0.0009) [2023-10-10 20:00:15,554][123582] Updated weights for policy 0, policy_version 87603 (0.0007) [2023-10-10 20:00:15,936][123582] Updated weights for policy 0, policy_version 87613 (0.0008) [2023-10-10 20:00:16,481][123614] Updated weights for policy 1, policy_version 87490 (0.0010) [2023-10-10 20:00:16,844][123614] Updated weights for policy 1, policy_version 87500 (0.0010) [2023-10-10 20:00:17,214][123614] Updated weights for policy 1, policy_version 87510 (0.0008) [2023-10-10 20:00:17,585][123614] Updated weights for policy 1, policy_version 87520 (0.0007) [2023-10-10 20:00:18,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 179339264. Throughput: 0: 1830.8, 1: 1806.8. Samples: 44841824. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:00:18,789][122664] Avg episode reward: [(0, '91.700'), (1, '92.780')] [2023-10-10 20:00:19,716][123582] Updated weights for policy 0, policy_version 87623 (0.0010) [2023-10-10 20:00:20,085][123582] Updated weights for policy 0, policy_version 87633 (0.0009) [2023-10-10 20:00:20,456][123582] Updated weights for policy 0, policy_version 87643 (0.0008) [2023-10-10 20:00:21,274][123614] Updated weights for policy 1, policy_version 87530 (0.0007) [2023-10-10 20:00:21,644][123614] Updated weights for policy 1, policy_version 87540 (0.0007) [2023-10-10 20:00:22,017][123614] Updated weights for policy 1, policy_version 87550 (0.0007) [2023-10-10 20:00:23,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 179404800. Throughput: 0: 1825.6, 1: 1807.2. Samples: 44864582. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-10 20:00:23,789][122664] Avg episode reward: [(0, '88.340'), (1, '91.000')] [2023-10-10 20:00:24,123][123582] Updated weights for policy 0, policy_version 87653 (0.0010) [2023-10-10 20:00:24,504][123582] Updated weights for policy 0, policy_version 87663 (0.0008) [2023-10-10 20:00:24,875][123582] Updated weights for policy 0, policy_version 87673 (0.0009) [2023-10-10 20:00:25,701][123614] Updated weights for policy 1, policy_version 87560 (0.0007) [2023-10-10 20:00:26,070][123614] Updated weights for policy 1, policy_version 87570 (0.0009) [2023-10-10 20:00:26,431][123614] Updated weights for policy 1, policy_version 87580 (0.0009) [2023-10-10 20:00:28,611][123582] Updated weights for policy 0, policy_version 87683 (0.0008) [2023-10-10 20:00:28,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 179470336. Throughput: 0: 1823.1, 1: 1804.6. Samples: 44874602. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-10 20:00:28,789][122664] Avg episode reward: [(0, '86.150'), (1, '92.760')] [2023-10-10 20:00:28,986][123582] Updated weights for policy 0, policy_version 87693 (0.0008) [2023-10-10 20:00:29,354][123582] Updated weights for policy 0, policy_version 87703 (0.0008) [2023-10-10 20:00:30,241][123614] Updated weights for policy 1, policy_version 87590 (0.0009) [2023-10-10 20:00:30,615][123614] Updated weights for policy 1, policy_version 87600 (0.0007) [2023-10-10 20:00:30,979][123614] Updated weights for policy 1, policy_version 87610 (0.0007) [2023-10-10 20:00:32,933][123582] Updated weights for policy 0, policy_version 87713 (0.0008) [2023-10-10 20:00:33,307][123582] Updated weights for policy 0, policy_version 87723 (0.0008) [2023-10-10 20:00:33,673][123582] Updated weights for policy 0, policy_version 87733 (0.0008) [2023-10-10 20:00:33,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 179535872. Throughput: 0: 1822.9, 1: 1806.4. Samples: 44897544. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-10 20:00:33,789][122664] Avg episode reward: [(0, '89.090'), (1, '96.660')] [2023-10-10 20:00:34,040][123582] Updated weights for policy 0, policy_version 87743 (0.0008) [2023-10-10 20:00:34,668][123614] Updated weights for policy 1, policy_version 87620 (0.0008) [2023-10-10 20:00:35,032][123614] Updated weights for policy 1, policy_version 87630 (0.0008) [2023-10-10 20:00:35,395][123614] Updated weights for policy 1, policy_version 87640 (0.0007) [2023-10-10 20:00:37,856][123582] Updated weights for policy 0, policy_version 87753 (0.0008) [2023-10-10 20:00:38,228][123582] Updated weights for policy 0, policy_version 87763 (0.0009) [2023-10-10 20:00:38,605][123582] Updated weights for policy 0, policy_version 87773 (0.0009) [2023-10-10 20:00:38,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 179634176. Throughput: 0: 1824.7, 1: 1818.7. Samples: 44919310. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-10 20:00:38,789][122664] Avg episode reward: [(0, '90.360'), (1, '97.570')] [2023-10-10 20:00:39,095][123614] Updated weights for policy 1, policy_version 87650 (0.0008) [2023-10-10 20:00:39,460][123614] Updated weights for policy 1, policy_version 87660 (0.0007) [2023-10-10 20:00:39,828][123614] Updated weights for policy 1, policy_version 87670 (0.0009) [2023-10-10 20:00:40,186][123614] Updated weights for policy 1, policy_version 87680 (0.0009) [2023-10-10 20:00:42,427][123582] Updated weights for policy 0, policy_version 87783 (0.0009) [2023-10-10 20:00:42,789][123582] Updated weights for policy 0, policy_version 87793 (0.0009) [2023-10-10 20:00:43,159][123582] Updated weights for policy 0, policy_version 87803 (0.0010) [2023-10-10 20:00:43,719][123614] Updated weights for policy 1, policy_version 87690 (0.0007) [2023-10-10 20:00:43,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 179699712. Throughput: 0: 1813.8, 1: 1818.9. Samples: 44930354. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-10 20:00:43,788][122664] Avg episode reward: [(0, '94.840'), (1, '96.670')] [2023-10-10 20:00:44,095][123614] Updated weights for policy 1, policy_version 87700 (0.0009) [2023-10-10 20:00:44,459][123614] Updated weights for policy 1, policy_version 87710 (0.0009) [2023-10-10 20:00:46,852][123582] Updated weights for policy 0, policy_version 87813 (0.0009) [2023-10-10 20:00:47,227][123582] Updated weights for policy 0, policy_version 87823 (0.0007) [2023-10-10 20:00:47,593][123582] Updated weights for policy 0, policy_version 87833 (0.0008) [2023-10-10 20:00:48,135][123614] Updated weights for policy 1, policy_version 87720 (0.0010) [2023-10-10 20:00:48,511][123614] Updated weights for policy 1, policy_version 87730 (0.0008) [2023-10-10 20:00:48,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 179765248. Throughput: 0: 1825.2, 1: 1820.6. Samples: 44952096. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-10 20:00:48,788][122664] Avg episode reward: [(0, '90.180'), (1, '94.610')] [2023-10-10 20:00:48,870][123614] Updated weights for policy 1, policy_version 87740 (0.0007) [2023-10-10 20:00:51,285][123582] Updated weights for policy 0, policy_version 87843 (0.0008) [2023-10-10 20:00:51,653][123582] Updated weights for policy 0, policy_version 87853 (0.0008) [2023-10-10 20:00:52,029][123582] Updated weights for policy 0, policy_version 87863 (0.0008) [2023-10-10 20:00:52,595][123614] Updated weights for policy 1, policy_version 87750 (0.0009) [2023-10-10 20:00:52,960][123614] Updated weights for policy 1, policy_version 87760 (0.0010) [2023-10-10 20:00:53,330][123614] Updated weights for policy 1, policy_version 87770 (0.0009) [2023-10-10 20:00:53,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 179863552. Throughput: 0: 1816.8, 1: 1817.3. Samples: 44972788. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-10 20:00:53,789][122664] Avg episode reward: [(0, '90.540'), (1, '95.050')] [2023-10-10 20:00:55,659][123582] Updated weights for policy 0, policy_version 87873 (0.0011) [2023-10-10 20:00:56,040][123582] Updated weights for policy 0, policy_version 87883 (0.0010) [2023-10-10 20:00:56,411][123582] Updated weights for policy 0, policy_version 87893 (0.0009) [2023-10-10 20:00:56,779][123582] Updated weights for policy 0, policy_version 87903 (0.0008) [2023-10-10 20:00:57,054][123614] Updated weights for policy 1, policy_version 87780 (0.0008) [2023-10-10 20:00:57,423][123614] Updated weights for policy 1, policy_version 87790 (0.0010) [2023-10-10 20:00:57,789][123614] Updated weights for policy 1, policy_version 87800 (0.0007) [2023-10-10 20:00:58,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 179929088. Throughput: 0: 1816.1, 1: 1822.0. Samples: 44984662. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-10 20:00:58,788][122664] Avg episode reward: [(0, '89.650'), (1, '97.070')] [2023-10-10 20:01:00,508][123582] Updated weights for policy 0, policy_version 87913 (0.0008) [2023-10-10 20:01:00,884][123582] Updated weights for policy 0, policy_version 87923 (0.0007) [2023-10-10 20:01:01,259][123582] Updated weights for policy 0, policy_version 87933 (0.0007) [2023-10-10 20:01:01,596][123614] Updated weights for policy 1, policy_version 87810 (0.0010) [2023-10-10 20:01:01,964][123614] Updated weights for policy 1, policy_version 87820 (0.0007) [2023-10-10 20:01:02,331][123614] Updated weights for policy 1, policy_version 87830 (0.0007) [2023-10-10 20:01:02,702][123614] Updated weights for policy 1, policy_version 87840 (0.0007) [2023-10-10 20:01:03,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 179994624. Throughput: 0: 1809.7, 1: 1825.8. Samples: 45005422. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-10 20:01:03,789][122664] Avg episode reward: [(0, '89.500'), (1, '96.630')] [2023-10-10 20:01:04,870][123582] Updated weights for policy 0, policy_version 87943 (0.0008) [2023-10-10 20:01:05,244][123582] Updated weights for policy 0, policy_version 87953 (0.0008) [2023-10-10 20:01:05,625][123582] Updated weights for policy 0, policy_version 87963 (0.0011) [2023-10-10 20:01:06,454][123614] Updated weights for policy 1, policy_version 87850 (0.0008) [2023-10-10 20:01:06,824][123614] Updated weights for policy 1, policy_version 87860 (0.0008) [2023-10-10 20:01:07,185][123614] Updated weights for policy 1, policy_version 87870 (0.0010) [2023-10-10 20:01:08,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 180060160. Throughput: 0: 1813.6, 1: 1822.0. Samples: 45028182. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-10 20:01:08,789][122664] Avg episode reward: [(0, '89.210'), (1, '100.560')] [2023-10-10 20:01:09,329][123582] Updated weights for policy 0, policy_version 87973 (0.0009) [2023-10-10 20:01:09,697][123582] Updated weights for policy 0, policy_version 87983 (0.0008) [2023-10-10 20:01:10,064][123582] Updated weights for policy 0, policy_version 87993 (0.0008) [2023-10-10 20:01:10,852][123614] Updated weights for policy 1, policy_version 87880 (0.0009) [2023-10-10 20:01:11,226][123614] Updated weights for policy 1, policy_version 87890 (0.0007) [2023-10-10 20:01:11,591][123614] Updated weights for policy 1, policy_version 87900 (0.0007) [2023-10-10 20:01:13,730][123582] Updated weights for policy 0, policy_version 88003 (0.0008) [2023-10-10 20:01:13,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 180125696. Throughput: 0: 1811.4, 1: 1825.4. Samples: 45038260. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-10 20:01:13,788][122664] Avg episode reward: [(0, '88.350'), (1, '101.400')] [2023-10-10 20:01:14,103][123582] Updated weights for policy 0, policy_version 88013 (0.0008) [2023-10-10 20:01:14,467][123582] Updated weights for policy 0, policy_version 88023 (0.0010) [2023-10-10 20:01:15,299][123614] Updated weights for policy 1, policy_version 87910 (0.0007) [2023-10-10 20:01:15,664][123614] Updated weights for policy 1, policy_version 87920 (0.0010) [2023-10-10 20:01:16,034][123614] Updated weights for policy 1, policy_version 87930 (0.0010) [2023-10-10 20:01:18,101][123582] Updated weights for policy 0, policy_version 88033 (0.0009) [2023-10-10 20:01:18,484][123582] Updated weights for policy 0, policy_version 88043 (0.0010) [2023-10-10 20:01:18,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 180191232. Throughput: 0: 1814.1, 1: 1823.4. Samples: 45061232. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-10 20:01:18,789][122664] Avg episode reward: [(0, '91.780'), (1, '93.980')] [2023-10-10 20:01:18,844][123582] Updated weights for policy 0, policy_version 88053 (0.0007) [2023-10-10 20:01:19,222][123582] Updated weights for policy 0, policy_version 88063 (0.0009) [2023-10-10 20:01:19,620][123614] Updated weights for policy 1, policy_version 87940 (0.0008) [2023-10-10 20:01:19,991][123614] Updated weights for policy 1, policy_version 87950 (0.0011) [2023-10-10 20:01:20,352][123614] Updated weights for policy 1, policy_version 87960 (0.0007) [2023-10-10 20:01:22,814][123582] Updated weights for policy 0, policy_version 88073 (0.0008) [2023-10-10 20:01:23,191][123582] Updated weights for policy 0, policy_version 88083 (0.0010) [2023-10-10 20:01:23,565][123582] Updated weights for policy 0, policy_version 88093 (0.0011) [2023-10-10 20:01:23,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 180289536. Throughput: 0: 1821.3, 1: 1816.7. Samples: 45083020. Policy #0 lag: (min: 31.0, avg: 33.0, max: 61.0) [2023-10-10 20:01:23,789][122664] Avg episode reward: [(0, '93.350'), (1, '92.210')] [2023-10-10 20:01:24,185][123614] Updated weights for policy 1, policy_version 87970 (0.0008) [2023-10-10 20:01:24,548][123614] Updated weights for policy 1, policy_version 87980 (0.0011) [2023-10-10 20:01:24,912][123614] Updated weights for policy 1, policy_version 87990 (0.0007) [2023-10-10 20:01:25,278][123614] Updated weights for policy 1, policy_version 88000 (0.0010) [2023-10-10 20:01:27,194][123582] Updated weights for policy 0, policy_version 88103 (0.0007) [2023-10-10 20:01:27,583][123582] Updated weights for policy 0, policy_version 88113 (0.0007) [2023-10-10 20:01:27,953][123582] Updated weights for policy 0, policy_version 88123 (0.0009) [2023-10-10 20:01:28,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 180355072. Throughput: 0: 1827.3, 1: 1811.2. Samples: 45094086. Policy #0 lag: (min: 31.0, avg: 33.0, max: 61.0) [2023-10-10 20:01:28,789][122664] Avg episode reward: [(0, '101.170'), (1, '89.220')] [2023-10-10 20:01:29,005][123614] Updated weights for policy 1, policy_version 88010 (0.0009) [2023-10-10 20:01:29,373][123614] Updated weights for policy 1, policy_version 88020 (0.0007) [2023-10-10 20:01:29,737][123614] Updated weights for policy 1, policy_version 88030 (0.0008) [2023-10-10 20:01:31,686][123582] Updated weights for policy 0, policy_version 88133 (0.0009) [2023-10-10 20:01:32,063][123582] Updated weights for policy 0, policy_version 88143 (0.0009) [2023-10-10 20:01:32,447][123582] Updated weights for policy 0, policy_version 88153 (0.0009) [2023-10-10 20:01:33,328][123614] Updated weights for policy 1, policy_version 88040 (0.0008) [2023-10-10 20:01:33,709][123614] Updated weights for policy 1, policy_version 88050 (0.0008) [2023-10-10 20:01:33,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 180420608. Throughput: 0: 1821.3, 1: 1810.7. Samples: 45115534. Policy #0 lag: (min: 31.0, avg: 33.0, max: 61.0) [2023-10-10 20:01:33,788][122664] Avg episode reward: [(0, '98.220'), (1, '88.440')] [2023-10-10 20:01:34,078][123614] Updated weights for policy 1, policy_version 88060 (0.0010) [2023-10-10 20:01:36,044][123582] Updated weights for policy 0, policy_version 88163 (0.0010) [2023-10-10 20:01:36,420][123582] Updated weights for policy 0, policy_version 88173 (0.0008) [2023-10-10 20:01:36,795][123582] Updated weights for policy 0, policy_version 88183 (0.0007) [2023-10-10 20:01:37,801][123614] Updated weights for policy 1, policy_version 88070 (0.0011) [2023-10-10 20:01:38,181][123614] Updated weights for policy 1, policy_version 88080 (0.0007) [2023-10-10 20:01:38,552][123614] Updated weights for policy 1, policy_version 88090 (0.0008) [2023-10-10 20:01:38,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 180518912. Throughput: 0: 1830.8, 1: 1814.8. Samples: 45136842. Policy #0 lag: (min: 31.0, avg: 33.0, max: 61.0) [2023-10-10 20:01:38,789][122664] Avg episode reward: [(0, '95.580'), (1, '84.010')] [2023-10-10 20:01:40,290][123582] Updated weights for policy 0, policy_version 88193 (0.0009) [2023-10-10 20:01:40,659][123582] Updated weights for policy 0, policy_version 88203 (0.0008) [2023-10-10 20:01:41,030][123582] Updated weights for policy 0, policy_version 88213 (0.0009) [2023-10-10 20:01:41,397][123582] Updated weights for policy 0, policy_version 88223 (0.0007) [2023-10-10 20:01:42,393][123614] Updated weights for policy 1, policy_version 88100 (0.0009) [2023-10-10 20:01:42,791][123614] Updated weights for policy 1, policy_version 88110 (0.0009) [2023-10-10 20:01:43,157][123614] Updated weights for policy 1, policy_version 88120 (0.0008) [2023-10-10 20:01:43,788][122664] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 180584448. Throughput: 0: 1825.7, 1: 1807.2. Samples: 45148142. Policy #0 lag: (min: 31.0, avg: 33.0, max: 61.0) [2023-10-10 20:01:43,789][122664] Avg episode reward: [(0, '95.690'), (1, '84.500')] [2023-10-10 20:01:45,030][123582] Updated weights for policy 0, policy_version 88233 (0.0011) [2023-10-10 20:01:45,395][123582] Updated weights for policy 0, policy_version 88243 (0.0009) [2023-10-10 20:01:45,770][123582] Updated weights for policy 0, policy_version 88253 (0.0008) [2023-10-10 20:01:46,971][123614] Updated weights for policy 1, policy_version 88130 (0.0009) [2023-10-10 20:01:47,344][123614] Updated weights for policy 1, policy_version 88140 (0.0012) [2023-10-10 20:01:47,712][123614] Updated weights for policy 1, policy_version 88150 (0.0009) [2023-10-10 20:01:48,078][123614] Updated weights for policy 1, policy_version 88160 (0.0009) [2023-10-10 20:01:48,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 180649984. Throughput: 0: 1839.0, 1: 1810.2. Samples: 45169634. Policy #0 lag: (min: 31.0, avg: 33.0, max: 61.0) [2023-10-10 20:01:48,789][122664] Avg episode reward: [(0, '94.140'), (1, '85.920')] [2023-10-10 20:01:49,534][123582] Updated weights for policy 0, policy_version 88263 (0.0007) [2023-10-10 20:01:49,903][123582] Updated weights for policy 0, policy_version 88273 (0.0007) [2023-10-10 20:01:50,273][123582] Updated weights for policy 0, policy_version 88283 (0.0008) [2023-10-10 20:01:51,539][123614] Updated weights for policy 1, policy_version 88170 (0.0010) [2023-10-10 20:01:51,911][123614] Updated weights for policy 1, policy_version 88180 (0.0009) [2023-10-10 20:01:52,284][123614] Updated weights for policy 1, policy_version 88190 (0.0009) [2023-10-10 20:01:53,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 180715520. Throughput: 0: 1833.4, 1: 1806.7. Samples: 45191988. Policy #0 lag: (min: 31.0, avg: 33.0, max: 61.0) [2023-10-10 20:01:53,789][122664] Avg episode reward: [(0, '94.570'), (1, '80.690')] [2023-10-10 20:01:53,799][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000088192_90308608.pth... [2023-10-10 20:01:53,837][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000086496_88571904.pth [2023-10-10 20:01:54,039][123582] Updated weights for policy 0, policy_version 88293 (0.0009) [2023-10-10 20:01:54,412][123582] Updated weights for policy 0, policy_version 88303 (0.0009) [2023-10-10 20:01:54,780][123582] Updated weights for policy 0, policy_version 88313 (0.0011) [2023-10-10 20:01:55,040][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000088320_90439680.pth... [2023-10-10 20:01:55,078][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000086592_88670208.pth [2023-10-10 20:01:56,044][123614] Updated weights for policy 1, policy_version 88200 (0.0009) [2023-10-10 20:01:56,419][123614] Updated weights for policy 1, policy_version 88210 (0.0012) [2023-10-10 20:01:56,792][123614] Updated weights for policy 1, policy_version 88220 (0.0008) [2023-10-10 20:01:58,517][123582] Updated weights for policy 0, policy_version 88323 (0.0009) [2023-10-10 20:01:58,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 180781056. Throughput: 0: 1830.0, 1: 1814.2. Samples: 45202248. Policy #0 lag: (min: 31.0, avg: 33.0, max: 61.0) [2023-10-10 20:01:58,789][122664] Avg episode reward: [(0, '100.720'), (1, '82.020')] [2023-10-10 20:01:58,895][123582] Updated weights for policy 0, policy_version 88333 (0.0009) [2023-10-10 20:01:59,273][123582] Updated weights for policy 0, policy_version 88343 (0.0008) [2023-10-10 20:02:00,520][123614] Updated weights for policy 1, policy_version 88230 (0.0007) [2023-10-10 20:02:00,879][123614] Updated weights for policy 1, policy_version 88240 (0.0009) [2023-10-10 20:02:01,259][123614] Updated weights for policy 1, policy_version 88250 (0.0011) [2023-10-10 20:02:03,058][123582] Updated weights for policy 0, policy_version 88353 (0.0008) [2023-10-10 20:02:03,430][123582] Updated weights for policy 0, policy_version 88363 (0.0011) [2023-10-10 20:02:03,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 180846592. Throughput: 0: 1822.5, 1: 1805.1. Samples: 45224472. Policy #0 lag: (min: 31.0, avg: 33.0, max: 61.0) [2023-10-10 20:02:03,789][122664] Avg episode reward: [(0, '93.920'), (1, '82.800')] [2023-10-10 20:02:03,796][123582] Updated weights for policy 0, policy_version 88373 (0.0009) [2023-10-10 20:02:04,178][123582] Updated weights for policy 0, policy_version 88383 (0.0010) [2023-10-10 20:02:05,086][123614] Updated weights for policy 1, policy_version 88260 (0.0010) [2023-10-10 20:02:05,457][123614] Updated weights for policy 1, policy_version 88270 (0.0010) [2023-10-10 20:02:05,834][123614] Updated weights for policy 1, policy_version 88280 (0.0009) [2023-10-10 20:02:07,913][123582] Updated weights for policy 0, policy_version 88393 (0.0011) [2023-10-10 20:02:08,278][123582] Updated weights for policy 0, policy_version 88403 (0.0009) [2023-10-10 20:02:08,658][123582] Updated weights for policy 0, policy_version 88413 (0.0009) [2023-10-10 20:02:08,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 180944896. Throughput: 0: 1817.0, 1: 1802.2. Samples: 45245882. Policy #0 lag: (min: 31.0, avg: 33.0, max: 61.0) [2023-10-10 20:02:08,789][122664] Avg episode reward: [(0, '90.140'), (1, '84.290')] [2023-10-10 20:02:09,505][123614] Updated weights for policy 1, policy_version 88290 (0.0009) [2023-10-10 20:02:09,874][123614] Updated weights for policy 1, policy_version 88300 (0.0007) [2023-10-10 20:02:10,240][123614] Updated weights for policy 1, policy_version 88310 (0.0007) [2023-10-10 20:02:10,607][123614] Updated weights for policy 1, policy_version 88320 (0.0007) [2023-10-10 20:02:12,354][123582] Updated weights for policy 0, policy_version 88423 (0.0008) [2023-10-10 20:02:12,729][123582] Updated weights for policy 0, policy_version 88433 (0.0007) [2023-10-10 20:02:13,110][123582] Updated weights for policy 0, policy_version 88443 (0.0007) [2023-10-10 20:02:13,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 181010432. Throughput: 0: 1814.1, 1: 1805.2. Samples: 45256956. Policy #0 lag: (min: 31.0, avg: 33.0, max: 61.0) [2023-10-10 20:02:13,789][122664] Avg episode reward: [(0, '93.080'), (1, '82.210')] [2023-10-10 20:02:14,259][123614] Updated weights for policy 1, policy_version 88330 (0.0008) [2023-10-10 20:02:14,626][123614] Updated weights for policy 1, policy_version 88340 (0.0008) [2023-10-10 20:02:14,985][123614] Updated weights for policy 1, policy_version 88350 (0.0007) [2023-10-10 20:02:16,783][123582] Updated weights for policy 0, policy_version 88453 (0.0008) [2023-10-10 20:02:17,150][123582] Updated weights for policy 0, policy_version 88463 (0.0008) [2023-10-10 20:02:17,522][123582] Updated weights for policy 0, policy_version 88473 (0.0009) [2023-10-10 20:02:18,561][123614] Updated weights for policy 1, policy_version 88360 (0.0009) [2023-10-10 20:02:18,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 181075968. Throughput: 0: 1815.3, 1: 1811.4. Samples: 45278738. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:02:18,788][122664] Avg episode reward: [(0, '92.790'), (1, '93.570')] [2023-10-10 20:02:18,929][123614] Updated weights for policy 1, policy_version 88370 (0.0007) [2023-10-10 20:02:19,298][123614] Updated weights for policy 1, policy_version 88380 (0.0009) [2023-10-10 20:02:21,306][123582] Updated weights for policy 0, policy_version 88483 (0.0010) [2023-10-10 20:02:21,681][123582] Updated weights for policy 0, policy_version 88493 (0.0008) [2023-10-10 20:02:22,064][123582] Updated weights for policy 0, policy_version 88503 (0.0007) [2023-10-10 20:02:23,052][123614] Updated weights for policy 1, policy_version 88390 (0.0009) [2023-10-10 20:02:23,423][123614] Updated weights for policy 1, policy_version 88400 (0.0009) [2023-10-10 20:02:23,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 181141504. Throughput: 0: 1801.6, 1: 1817.1. Samples: 45299684. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:02:23,789][122664] Avg episode reward: [(0, '91.980'), (1, '96.070')] [2023-10-10 20:02:23,800][123614] Updated weights for policy 1, policy_version 88410 (0.0009) [2023-10-10 20:02:25,774][123582] Updated weights for policy 0, policy_version 88513 (0.0008) [2023-10-10 20:02:26,144][123582] Updated weights for policy 0, policy_version 88523 (0.0007) [2023-10-10 20:02:26,517][123582] Updated weights for policy 0, policy_version 88533 (0.0007) [2023-10-10 20:02:26,896][123582] Updated weights for policy 0, policy_version 88543 (0.0008) [2023-10-10 20:02:27,583][123614] Updated weights for policy 1, policy_version 88420 (0.0008) [2023-10-10 20:02:27,966][123614] Updated weights for policy 1, policy_version 88430 (0.0010) [2023-10-10 20:02:28,337][123614] Updated weights for policy 1, policy_version 88440 (0.0009) [2023-10-10 20:02:28,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 181239808. Throughput: 0: 1815.2, 1: 1813.1. Samples: 45311416. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:02:28,789][122664] Avg episode reward: [(0, '93.610'), (1, '98.120')] [2023-10-10 20:02:30,693][123582] Updated weights for policy 0, policy_version 88553 (0.0007) [2023-10-10 20:02:31,067][123582] Updated weights for policy 0, policy_version 88563 (0.0008) [2023-10-10 20:02:31,449][123582] Updated weights for policy 0, policy_version 88573 (0.0007) [2023-10-10 20:02:32,022][123614] Updated weights for policy 1, policy_version 88450 (0.0008) [2023-10-10 20:02:32,379][123614] Updated weights for policy 1, policy_version 88460 (0.0011) [2023-10-10 20:02:32,746][123614] Updated weights for policy 1, policy_version 88470 (0.0008) [2023-10-10 20:02:33,111][123614] Updated weights for policy 1, policy_version 88480 (0.0009) [2023-10-10 20:02:33,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 181305344. Throughput: 0: 1798.4, 1: 1817.5. Samples: 45332346. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:02:33,789][122664] Avg episode reward: [(0, '89.740'), (1, '100.090')] [2023-10-10 20:02:35,076][123582] Updated weights for policy 0, policy_version 88583 (0.0007) [2023-10-10 20:02:35,453][123582] Updated weights for policy 0, policy_version 88593 (0.0007) [2023-10-10 20:02:35,829][123582] Updated weights for policy 0, policy_version 88603 (0.0007) [2023-10-10 20:02:36,746][123614] Updated weights for policy 1, policy_version 88490 (0.0008) [2023-10-10 20:02:37,122][123614] Updated weights for policy 1, policy_version 88500 (0.0008) [2023-10-10 20:02:37,487][123614] Updated weights for policy 1, policy_version 88510 (0.0010) [2023-10-10 20:02:38,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 181370880. Throughput: 0: 1802.7, 1: 1810.2. Samples: 45354566. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:02:38,788][122664] Avg episode reward: [(0, '87.240'), (1, '100.590')] [2023-10-10 20:02:39,548][123582] Updated weights for policy 0, policy_version 88613 (0.0007) [2023-10-10 20:02:39,915][123582] Updated weights for policy 0, policy_version 88623 (0.0007) [2023-10-10 20:02:40,290][123582] Updated weights for policy 0, policy_version 88633 (0.0009) [2023-10-10 20:02:41,242][123614] Updated weights for policy 1, policy_version 88520 (0.0010) [2023-10-10 20:02:41,614][123614] Updated weights for policy 1, policy_version 88530 (0.0010) [2023-10-10 20:02:41,975][123614] Updated weights for policy 1, policy_version 88540 (0.0010) [2023-10-10 20:02:43,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 181436416. Throughput: 0: 1803.7, 1: 1810.2. Samples: 45364872. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:02:43,789][122664] Avg episode reward: [(0, '83.690'), (1, '95.850')] [2023-10-10 20:02:43,855][123582] Updated weights for policy 0, policy_version 88643 (0.0009) [2023-10-10 20:02:44,228][123582] Updated weights for policy 0, policy_version 88653 (0.0007) [2023-10-10 20:02:44,599][123582] Updated weights for policy 0, policy_version 88663 (0.0009) [2023-10-10 20:02:45,734][123614] Updated weights for policy 1, policy_version 88550 (0.0010) [2023-10-10 20:02:46,098][123614] Updated weights for policy 1, policy_version 88560 (0.0009) [2023-10-10 20:02:46,470][123614] Updated weights for policy 1, policy_version 88570 (0.0009) [2023-10-10 20:02:48,358][123582] Updated weights for policy 0, policy_version 88673 (0.0008) [2023-10-10 20:02:48,740][123582] Updated weights for policy 0, policy_version 88683 (0.0007) [2023-10-10 20:02:48,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 181501952. Throughput: 0: 1812.5, 1: 1807.2. Samples: 45387360. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:02:48,788][122664] Avg episode reward: [(0, '80.750'), (1, '92.250')] [2023-10-10 20:02:49,111][123582] Updated weights for policy 0, policy_version 88693 (0.0008) [2023-10-10 20:02:49,485][123582] Updated weights for policy 0, policy_version 88703 (0.0010) [2023-10-10 20:02:50,246][123614] Updated weights for policy 1, policy_version 88580 (0.0009) [2023-10-10 20:02:50,624][123614] Updated weights for policy 1, policy_version 88590 (0.0009) [2023-10-10 20:02:50,996][123614] Updated weights for policy 1, policy_version 88600 (0.0010) [2023-10-10 20:02:53,081][123582] Updated weights for policy 0, policy_version 88713 (0.0008) [2023-10-10 20:02:53,466][123582] Updated weights for policy 0, policy_version 88723 (0.0008) [2023-10-10 20:02:53,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 181567488. Throughput: 0: 1822.9, 1: 1806.7. Samples: 45409214. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:02:53,789][122664] Avg episode reward: [(0, '81.520'), (1, '93.840')] [2023-10-10 20:02:53,828][123582] Updated weights for policy 0, policy_version 88733 (0.0008) [2023-10-10 20:02:54,795][123614] Updated weights for policy 1, policy_version 88610 (0.0009) [2023-10-10 20:02:55,168][123614] Updated weights for policy 1, policy_version 88620 (0.0008) [2023-10-10 20:02:55,539][123614] Updated weights for policy 1, policy_version 88630 (0.0009) [2023-10-10 20:02:55,913][123614] Updated weights for policy 1, policy_version 88640 (0.0010) [2023-10-10 20:02:57,549][123582] Updated weights for policy 0, policy_version 88743 (0.0009) [2023-10-10 20:02:57,932][123582] Updated weights for policy 0, policy_version 88753 (0.0008) [2023-10-10 20:02:58,293][123582] Updated weights for policy 0, policy_version 88763 (0.0008) [2023-10-10 20:02:58,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 181665792. Throughput: 0: 1814.0, 1: 1809.2. Samples: 45419996. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:02:58,788][122664] Avg episode reward: [(0, '82.120'), (1, '98.170')] [2023-10-10 20:02:59,561][123614] Updated weights for policy 1, policy_version 88650 (0.0009) [2023-10-10 20:02:59,930][123614] Updated weights for policy 1, policy_version 88660 (0.0008) [2023-10-10 20:03:00,297][123614] Updated weights for policy 1, policy_version 88670 (0.0008) [2023-10-10 20:03:02,084][123582] Updated weights for policy 0, policy_version 88773 (0.0009) [2023-10-10 20:03:02,454][123582] Updated weights for policy 0, policy_version 88783 (0.0008) [2023-10-10 20:03:02,832][123582] Updated weights for policy 0, policy_version 88793 (0.0008) [2023-10-10 20:03:03,788][122664] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 181731328. Throughput: 0: 1821.1, 1: 1803.5. Samples: 45441846. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:03:03,789][122664] Avg episode reward: [(0, '79.800'), (1, '98.300')] [2023-10-10 20:03:03,929][123614] Updated weights for policy 1, policy_version 88680 (0.0008) [2023-10-10 20:03:04,294][123614] Updated weights for policy 1, policy_version 88690 (0.0007) [2023-10-10 20:03:04,657][123614] Updated weights for policy 1, policy_version 88700 (0.0011) [2023-10-10 20:03:06,399][123582] Updated weights for policy 0, policy_version 88803 (0.0009) [2023-10-10 20:03:06,775][123582] Updated weights for policy 0, policy_version 88813 (0.0011) [2023-10-10 20:03:07,147][123582] Updated weights for policy 0, policy_version 88823 (0.0010) [2023-10-10 20:03:08,418][123614] Updated weights for policy 1, policy_version 88710 (0.0007) [2023-10-10 20:03:08,786][123614] Updated weights for policy 1, policy_version 88720 (0.0010) [2023-10-10 20:03:08,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 181796864. Throughput: 0: 1813.1, 1: 1818.8. Samples: 45463120. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:03:08,788][122664] Avg episode reward: [(0, '76.910'), (1, '103.050')] [2023-10-10 20:03:09,151][123614] Updated weights for policy 1, policy_version 88730 (0.0008) [2023-10-10 20:03:10,867][123582] Updated weights for policy 0, policy_version 88833 (0.0010) [2023-10-10 20:03:11,242][123582] Updated weights for policy 0, policy_version 88843 (0.0009) [2023-10-10 20:03:11,628][123582] Updated weights for policy 0, policy_version 88853 (0.0009) [2023-10-10 20:03:11,993][123582] Updated weights for policy 0, policy_version 88863 (0.0009) [2023-10-10 20:03:13,011][123614] Updated weights for policy 1, policy_version 88740 (0.0009) [2023-10-10 20:03:13,409][123614] Updated weights for policy 1, policy_version 88750 (0.0009) [2023-10-10 20:03:13,778][123614] Updated weights for policy 1, policy_version 88760 (0.0008) [2023-10-10 20:03:13,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 181862400. Throughput: 0: 1812.7, 1: 1805.6. Samples: 45474238. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:03:13,789][122664] Avg episode reward: [(0, '69.230'), (1, '103.940')] [2023-10-10 20:03:15,635][123582] Updated weights for policy 0, policy_version 88873 (0.0008) [2023-10-10 20:03:16,004][123582] Updated weights for policy 0, policy_version 88883 (0.0008) [2023-10-10 20:03:16,377][123582] Updated weights for policy 0, policy_version 88893 (0.0009) [2023-10-10 20:03:17,305][123614] Updated weights for policy 1, policy_version 88770 (0.0007) [2023-10-10 20:03:17,667][123614] Updated weights for policy 1, policy_version 88780 (0.0007) [2023-10-10 20:03:18,037][123614] Updated weights for policy 1, policy_version 88790 (0.0007) [2023-10-10 20:03:18,405][123614] Updated weights for policy 1, policy_version 88800 (0.0007) [2023-10-10 20:03:18,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 181960704. Throughput: 0: 1812.7, 1: 1821.8. Samples: 45495896. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-10 20:03:18,788][122664] Avg episode reward: [(0, '79.110'), (1, '110.940')] [2023-10-10 20:03:20,093][123582] Updated weights for policy 0, policy_version 88903 (0.0010) [2023-10-10 20:03:20,460][123582] Updated weights for policy 0, policy_version 88913 (0.0007) [2023-10-10 20:03:20,842][123582] Updated weights for policy 0, policy_version 88923 (0.0010) [2023-10-10 20:03:22,100][123614] Updated weights for policy 1, policy_version 88810 (0.0009) [2023-10-10 20:03:22,469][123614] Updated weights for policy 1, policy_version 88820 (0.0011) [2023-10-10 20:03:22,836][123614] Updated weights for policy 1, policy_version 88830 (0.0009) [2023-10-10 20:03:23,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 182026240. Throughput: 0: 1813.3, 1: 1809.4. Samples: 45517586. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-10 20:03:23,789][122664] Avg episode reward: [(0, '79.940'), (1, '117.930')] [2023-10-10 20:03:23,800][123465] Saving new best policy, reward=117.930! [2023-10-10 20:03:24,526][123582] Updated weights for policy 0, policy_version 88933 (0.0008) [2023-10-10 20:03:24,904][123582] Updated weights for policy 0, policy_version 88943 (0.0007) [2023-10-10 20:03:25,276][123582] Updated weights for policy 0, policy_version 88953 (0.0007) [2023-10-10 20:03:26,517][123614] Updated weights for policy 1, policy_version 88840 (0.0008) [2023-10-10 20:03:26,883][123614] Updated weights for policy 1, policy_version 88850 (0.0008) [2023-10-10 20:03:27,246][123614] Updated weights for policy 1, policy_version 88860 (0.0009) [2023-10-10 20:03:28,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 182091776. Throughput: 0: 1817.7, 1: 1818.8. Samples: 45528514. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-10 20:03:28,788][122664] Avg episode reward: [(0, '81.050'), (1, '115.970')] [2023-10-10 20:03:28,870][123582] Updated weights for policy 0, policy_version 88963 (0.0010) [2023-10-10 20:03:29,247][123582] Updated weights for policy 0, policy_version 88973 (0.0011) [2023-10-10 20:03:29,617][123582] Updated weights for policy 0, policy_version 88983 (0.0009) [2023-10-10 20:03:31,063][123614] Updated weights for policy 1, policy_version 88870 (0.0009) [2023-10-10 20:03:31,437][123614] Updated weights for policy 1, policy_version 88880 (0.0007) [2023-10-10 20:03:31,798][123614] Updated weights for policy 1, policy_version 88890 (0.0007) [2023-10-10 20:03:33,376][123582] Updated weights for policy 0, policy_version 88993 (0.0008) [2023-10-10 20:03:33,751][123582] Updated weights for policy 0, policy_version 89003 (0.0007) [2023-10-10 20:03:33,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 182157312. Throughput: 0: 1814.0, 1: 1807.3. Samples: 45550322. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-10 20:03:33,789][122664] Avg episode reward: [(0, '81.700'), (1, '113.030')] [2023-10-10 20:03:34,124][123582] Updated weights for policy 0, policy_version 89013 (0.0007) [2023-10-10 20:03:34,498][123582] Updated weights for policy 0, policy_version 89023 (0.0008) [2023-10-10 20:03:35,420][123614] Updated weights for policy 1, policy_version 88900 (0.0008) [2023-10-10 20:03:35,790][123614] Updated weights for policy 1, policy_version 88910 (0.0007) [2023-10-10 20:03:36,154][123614] Updated weights for policy 1, policy_version 88920 (0.0007) [2023-10-10 20:03:38,169][123582] Updated weights for policy 0, policy_version 89033 (0.0008) [2023-10-10 20:03:38,543][123582] Updated weights for policy 0, policy_version 89043 (0.0008) [2023-10-10 20:03:38,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 182222848. Throughput: 0: 1817.0, 1: 1811.6. Samples: 45572500. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-10 20:03:38,789][122664] Avg episode reward: [(0, '89.750'), (1, '112.320')] [2023-10-10 20:03:38,912][123582] Updated weights for policy 0, policy_version 89053 (0.0007) [2023-10-10 20:03:39,832][123614] Updated weights for policy 1, policy_version 88930 (0.0008) [2023-10-10 20:03:40,200][123614] Updated weights for policy 1, policy_version 88940 (0.0009) [2023-10-10 20:03:40,568][123614] Updated weights for policy 1, policy_version 88950 (0.0008) [2023-10-10 20:03:40,937][123614] Updated weights for policy 1, policy_version 88960 (0.0008) [2023-10-10 20:03:42,705][123582] Updated weights for policy 0, policy_version 89063 (0.0008) [2023-10-10 20:03:43,080][123582] Updated weights for policy 0, policy_version 89073 (0.0010) [2023-10-10 20:03:43,440][123582] Updated weights for policy 0, policy_version 89083 (0.0010) [2023-10-10 20:03:43,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 182321152. Throughput: 0: 1813.2, 1: 1811.9. Samples: 45583124. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-10 20:03:43,789][122664] Avg episode reward: [(0, '92.330'), (1, '110.490')] [2023-10-10 20:03:44,489][123614] Updated weights for policy 1, policy_version 88970 (0.0010) [2023-10-10 20:03:44,862][123614] Updated weights for policy 1, policy_version 88980 (0.0008) [2023-10-10 20:03:45,221][123614] Updated weights for policy 1, policy_version 88990 (0.0007) [2023-10-10 20:03:47,071][123582] Updated weights for policy 0, policy_version 89093 (0.0008) [2023-10-10 20:03:47,443][123582] Updated weights for policy 0, policy_version 89103 (0.0008) [2023-10-10 20:03:47,820][123582] Updated weights for policy 0, policy_version 89113 (0.0007) [2023-10-10 20:03:48,781][123614] Updated weights for policy 1, policy_version 89000 (0.0007) [2023-10-10 20:03:48,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 182386688. Throughput: 0: 1816.5, 1: 1824.0. Samples: 45605668. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-10 20:03:48,789][122664] Avg episode reward: [(0, '91.210'), (1, '107.880')] [2023-10-10 20:03:49,149][123614] Updated weights for policy 1, policy_version 89010 (0.0008) [2023-10-10 20:03:49,514][123614] Updated weights for policy 1, policy_version 89020 (0.0008) [2023-10-10 20:03:51,535][123582] Updated weights for policy 0, policy_version 89123 (0.0008) [2023-10-10 20:03:51,914][123582] Updated weights for policy 0, policy_version 89133 (0.0008) [2023-10-10 20:03:52,278][123582] Updated weights for policy 0, policy_version 89143 (0.0007) [2023-10-10 20:03:53,213][123614] Updated weights for policy 1, policy_version 89030 (0.0008) [2023-10-10 20:03:53,580][123614] Updated weights for policy 1, policy_version 89040 (0.0007) [2023-10-10 20:03:53,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 182452224. Throughput: 0: 1817.9, 1: 1814.0. Samples: 45626556. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-10 20:03:53,788][122664] Avg episode reward: [(0, '92.580'), (1, '106.760')] [2023-10-10 20:03:53,797][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000089152_91291648.pth... [2023-10-10 20:03:53,826][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000087456_89554944.pth [2023-10-10 20:03:53,945][123614] Updated weights for policy 1, policy_version 89050 (0.0007) [2023-10-10 20:03:54,161][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000089056_91193344.pth... [2023-10-10 20:03:54,205][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000087360_89456640.pth [2023-10-10 20:03:55,902][123582] Updated weights for policy 0, policy_version 89153 (0.0007) [2023-10-10 20:03:56,273][123582] Updated weights for policy 0, policy_version 89163 (0.0007) [2023-10-10 20:03:56,639][123582] Updated weights for policy 0, policy_version 89173 (0.0010) [2023-10-10 20:03:57,014][123582] Updated weights for policy 0, policy_version 89183 (0.0007) [2023-10-10 20:03:57,578][123614] Updated weights for policy 1, policy_version 89060 (0.0007) [2023-10-10 20:03:57,964][123614] Updated weights for policy 1, policy_version 89070 (0.0007) [2023-10-10 20:03:58,331][123614] Updated weights for policy 1, policy_version 89080 (0.0008) [2023-10-10 20:03:58,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 182550528. Throughput: 0: 1821.4, 1: 1830.8. Samples: 45638590. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-10 20:03:58,789][122664] Avg episode reward: [(0, '96.820'), (1, '104.860')] [2023-10-10 20:04:00,866][123582] Updated weights for policy 0, policy_version 89193 (0.0009) [2023-10-10 20:04:01,231][123582] Updated weights for policy 0, policy_version 89203 (0.0007) [2023-10-10 20:04:01,611][123582] Updated weights for policy 0, policy_version 89213 (0.0008) [2023-10-10 20:04:01,946][123614] Updated weights for policy 1, policy_version 89090 (0.0009) [2023-10-10 20:04:02,295][123614] Updated weights for policy 1, policy_version 89100 (0.0008) [2023-10-10 20:04:02,664][123614] Updated weights for policy 1, policy_version 89110 (0.0008) [2023-10-10 20:04:03,033][123614] Updated weights for policy 1, policy_version 89120 (0.0009) [2023-10-10 20:04:03,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 182616064. Throughput: 0: 1813.4, 1: 1818.7. Samples: 45659342. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-10 20:04:03,788][122664] Avg episode reward: [(0, '98.430'), (1, '103.420')] [2023-10-10 20:04:05,306][123582] Updated weights for policy 0, policy_version 89223 (0.0011) [2023-10-10 20:04:05,678][123582] Updated weights for policy 0, policy_version 89233 (0.0010) [2023-10-10 20:04:06,059][123582] Updated weights for policy 0, policy_version 89243 (0.0009) [2023-10-10 20:04:06,850][123614] Updated weights for policy 1, policy_version 89130 (0.0009) [2023-10-10 20:04:07,217][123614] Updated weights for policy 1, policy_version 89140 (0.0009) [2023-10-10 20:04:07,586][123614] Updated weights for policy 1, policy_version 89150 (0.0007) [2023-10-10 20:04:08,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 182681600. Throughput: 0: 1815.5, 1: 1828.8. Samples: 45681576. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-10 20:04:08,789][122664] Avg episode reward: [(0, '103.810'), (1, '103.300')] [2023-10-10 20:04:09,648][123582] Updated weights for policy 0, policy_version 89253 (0.0008) [2023-10-10 20:04:10,029][123582] Updated weights for policy 0, policy_version 89263 (0.0010) [2023-10-10 20:04:10,393][123582] Updated weights for policy 0, policy_version 89273 (0.0010) [2023-10-10 20:04:11,289][123614] Updated weights for policy 1, policy_version 89160 (0.0009) [2023-10-10 20:04:11,653][123614] Updated weights for policy 1, policy_version 89170 (0.0010) [2023-10-10 20:04:12,026][123614] Updated weights for policy 1, policy_version 89180 (0.0007) [2023-10-10 20:04:13,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 182747136. Throughput: 0: 1812.6, 1: 1820.5. Samples: 45692004. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-10 20:04:13,788][122664] Avg episode reward: [(0, '104.260'), (1, '99.110')] [2023-10-10 20:04:13,919][123582] Updated weights for policy 0, policy_version 89283 (0.0008) [2023-10-10 20:04:14,294][123582] Updated weights for policy 0, policy_version 89293 (0.0008) [2023-10-10 20:04:14,674][123582] Updated weights for policy 0, policy_version 89303 (0.0008) [2023-10-10 20:04:15,802][123614] Updated weights for policy 1, policy_version 89190 (0.0009) [2023-10-10 20:04:16,171][123614] Updated weights for policy 1, policy_version 89200 (0.0007) [2023-10-10 20:04:16,539][123614] Updated weights for policy 1, policy_version 89210 (0.0007) [2023-10-10 20:04:18,521][123582] Updated weights for policy 0, policy_version 89313 (0.0011) [2023-10-10 20:04:18,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 182812672. Throughput: 0: 1813.9, 1: 1826.0. Samples: 45714116. Policy #0 lag: (min: 0.0, avg: 21.7, max: 32.0) [2023-10-10 20:04:18,788][122664] Avg episode reward: [(0, '109.120'), (1, '99.500')] [2023-10-10 20:04:18,886][123582] Updated weights for policy 0, policy_version 89323 (0.0010) [2023-10-10 20:04:19,255][123582] Updated weights for policy 0, policy_version 89333 (0.0010) [2023-10-10 20:04:19,621][123582] Updated weights for policy 0, policy_version 89343 (0.0009) [2023-10-10 20:04:20,150][123614] Updated weights for policy 1, policy_version 89220 (0.0007) [2023-10-10 20:04:20,521][123614] Updated weights for policy 1, policy_version 89230 (0.0008) [2023-10-10 20:04:20,887][123614] Updated weights for policy 1, policy_version 89240 (0.0007) [2023-10-10 20:04:23,283][123582] Updated weights for policy 0, policy_version 89353 (0.0011) [2023-10-10 20:04:23,660][123582] Updated weights for policy 0, policy_version 89363 (0.0010) [2023-10-10 20:04:23,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 182878208. Throughput: 0: 1811.9, 1: 1827.6. Samples: 45736276. Policy #0 lag: (min: 0.0, avg: 21.7, max: 32.0) [2023-10-10 20:04:23,788][122664] Avg episode reward: [(0, '103.580'), (1, '99.920')] [2023-10-10 20:04:24,030][123582] Updated weights for policy 0, policy_version 89373 (0.0009) [2023-10-10 20:04:24,784][123614] Updated weights for policy 1, policy_version 89250 (0.0011) [2023-10-10 20:04:25,159][123614] Updated weights for policy 1, policy_version 89260 (0.0010) [2023-10-10 20:04:25,532][123614] Updated weights for policy 1, policy_version 89270 (0.0010) [2023-10-10 20:04:25,899][123614] Updated weights for policy 1, policy_version 89280 (0.0007) [2023-10-10 20:04:27,918][123582] Updated weights for policy 0, policy_version 89383 (0.0009) [2023-10-10 20:04:28,286][123582] Updated weights for policy 0, policy_version 89393 (0.0007) [2023-10-10 20:04:28,676][123582] Updated weights for policy 0, policy_version 89403 (0.0009) [2023-10-10 20:04:28,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 182943744. Throughput: 0: 1811.7, 1: 1824.6. Samples: 45746758. Policy #0 lag: (min: 0.0, avg: 21.7, max: 32.0) [2023-10-10 20:04:28,788][122664] Avg episode reward: [(0, '109.400'), (1, '100.050')] [2023-10-10 20:04:29,626][123614] Updated weights for policy 1, policy_version 89290 (0.0009) [2023-10-10 20:04:29,996][123614] Updated weights for policy 1, policy_version 89300 (0.0010) [2023-10-10 20:04:30,369][123614] Updated weights for policy 1, policy_version 89310 (0.0010) [2023-10-10 20:04:32,224][123582] Updated weights for policy 0, policy_version 89413 (0.0009) [2023-10-10 20:04:32,599][123582] Updated weights for policy 0, policy_version 89423 (0.0010) [2023-10-10 20:04:32,965][123582] Updated weights for policy 0, policy_version 89433 (0.0009) [2023-10-10 20:04:33,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 183042048. Throughput: 0: 1816.3, 1: 1807.6. Samples: 45768744. Policy #0 lag: (min: 0.0, avg: 21.7, max: 32.0) [2023-10-10 20:04:33,788][122664] Avg episode reward: [(0, '110.030'), (1, '109.510')] [2023-10-10 20:04:34,100][123614] Updated weights for policy 1, policy_version 89320 (0.0009) [2023-10-10 20:04:34,466][123614] Updated weights for policy 1, policy_version 89330 (0.0008) [2023-10-10 20:04:34,833][123614] Updated weights for policy 1, policy_version 89340 (0.0008) [2023-10-10 20:04:36,643][123582] Updated weights for policy 0, policy_version 89443 (0.0011) [2023-10-10 20:04:37,021][123582] Updated weights for policy 0, policy_version 89453 (0.0010) [2023-10-10 20:04:37,377][123582] Updated weights for policy 0, policy_version 89463 (0.0011) [2023-10-10 20:04:38,482][123614] Updated weights for policy 1, policy_version 89350 (0.0007) [2023-10-10 20:04:38,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 183107584. Throughput: 0: 1816.0, 1: 1820.2. Samples: 45790184. Policy #0 lag: (min: 0.0, avg: 21.7, max: 32.0) [2023-10-10 20:04:38,788][122664] Avg episode reward: [(0, '107.300'), (1, '117.990')] [2023-10-10 20:04:38,859][123614] Updated weights for policy 1, policy_version 89360 (0.0009) [2023-10-10 20:04:39,225][123614] Updated weights for policy 1, policy_version 89370 (0.0008) [2023-10-10 20:04:39,441][123465] Saving new best policy, reward=117.990! [2023-10-10 20:04:41,080][123582] Updated weights for policy 0, policy_version 89473 (0.0008) [2023-10-10 20:04:41,455][123582] Updated weights for policy 0, policy_version 89483 (0.0010) [2023-10-10 20:04:41,820][123582] Updated weights for policy 0, policy_version 89493 (0.0010) [2023-10-10 20:04:42,200][123582] Updated weights for policy 0, policy_version 89503 (0.0009) [2023-10-10 20:04:43,110][123614] Updated weights for policy 1, policy_version 89380 (0.0008) [2023-10-10 20:04:43,493][123614] Updated weights for policy 1, policy_version 89390 (0.0007) [2023-10-10 20:04:43,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 183173120. Throughput: 0: 1818.9, 1: 1803.2. Samples: 45801584. Policy #0 lag: (min: 0.0, avg: 21.7, max: 32.0) [2023-10-10 20:04:43,788][122664] Avg episode reward: [(0, '108.660'), (1, '122.580')] [2023-10-10 20:04:43,858][123614] Updated weights for policy 1, policy_version 89400 (0.0007) [2023-10-10 20:04:44,149][123465] Saving new best policy, reward=122.580! [2023-10-10 20:04:45,726][123582] Updated weights for policy 0, policy_version 89513 (0.0008) [2023-10-10 20:04:46,109][123582] Updated weights for policy 0, policy_version 89523 (0.0007) [2023-10-10 20:04:46,477][123582] Updated weights for policy 0, policy_version 89533 (0.0010) [2023-10-10 20:04:47,690][123614] Updated weights for policy 1, policy_version 89410 (0.0008) [2023-10-10 20:04:48,067][123614] Updated weights for policy 1, policy_version 89420 (0.0010) [2023-10-10 20:04:48,440][123614] Updated weights for policy 1, policy_version 89430 (0.0008) [2023-10-10 20:04:48,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 183238656. Throughput: 0: 1820.7, 1: 1817.9. Samples: 45823076. Policy #0 lag: (min: 0.0, avg: 21.7, max: 32.0) [2023-10-10 20:04:48,789][122664] Avg episode reward: [(0, '101.740'), (1, '122.530')] [2023-10-10 20:04:48,804][123614] Updated weights for policy 1, policy_version 89440 (0.0009) [2023-10-10 20:04:50,195][123582] Updated weights for policy 0, policy_version 89543 (0.0008) [2023-10-10 20:04:50,569][123582] Updated weights for policy 0, policy_version 89553 (0.0010) [2023-10-10 20:04:50,942][123582] Updated weights for policy 0, policy_version 89563 (0.0009) [2023-10-10 20:04:52,480][123614] Updated weights for policy 1, policy_version 89450 (0.0010) [2023-10-10 20:04:52,846][123614] Updated weights for policy 1, policy_version 89460 (0.0010) [2023-10-10 20:04:53,207][123614] Updated weights for policy 1, policy_version 89470 (0.0009) [2023-10-10 20:04:53,788][122664] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 183336960. Throughput: 0: 1820.7, 1: 1799.3. Samples: 45844478. Policy #0 lag: (min: 0.0, avg: 21.7, max: 32.0) [2023-10-10 20:04:53,789][122664] Avg episode reward: [(0, '99.440'), (1, '121.790')] [2023-10-10 20:04:54,585][123582] Updated weights for policy 0, policy_version 89573 (0.0010) [2023-10-10 20:04:54,955][123582] Updated weights for policy 0, policy_version 89583 (0.0009) [2023-10-10 20:04:55,332][123582] Updated weights for policy 0, policy_version 89593 (0.0011) [2023-10-10 20:04:56,855][123614] Updated weights for policy 1, policy_version 89480 (0.0009) [2023-10-10 20:04:57,220][123614] Updated weights for policy 1, policy_version 89490 (0.0008) [2023-10-10 20:04:57,595][123614] Updated weights for policy 1, policy_version 89500 (0.0007) [2023-10-10 20:04:58,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 183402496. Throughput: 0: 1821.6, 1: 1819.7. Samples: 45855864. Policy #0 lag: (min: 0.0, avg: 21.7, max: 32.0) [2023-10-10 20:04:58,789][122664] Avg episode reward: [(0, '92.940'), (1, '123.000')] [2023-10-10 20:04:58,790][123465] Saving new best policy, reward=123.000! [2023-10-10 20:04:58,959][123582] Updated weights for policy 0, policy_version 89603 (0.0009) [2023-10-10 20:04:59,336][123582] Updated weights for policy 0, policy_version 89613 (0.0009) [2023-10-10 20:04:59,719][123582] Updated weights for policy 0, policy_version 89623 (0.0008) [2023-10-10 20:05:01,254][123614] Updated weights for policy 1, policy_version 89510 (0.0009) [2023-10-10 20:05:01,628][123614] Updated weights for policy 1, policy_version 89520 (0.0009) [2023-10-10 20:05:01,995][123614] Updated weights for policy 1, policy_version 89530 (0.0010) [2023-10-10 20:05:03,383][123582] Updated weights for policy 0, policy_version 89633 (0.0011) [2023-10-10 20:05:03,757][123582] Updated weights for policy 0, policy_version 89643 (0.0008) [2023-10-10 20:05:03,788][122664] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 183468032. Throughput: 0: 1819.6, 1: 1808.5. Samples: 45877380. Policy #0 lag: (min: 0.0, avg: 21.7, max: 32.0) [2023-10-10 20:05:03,788][122664] Avg episode reward: [(0, '92.180'), (1, '117.850')] [2023-10-10 20:05:04,138][123582] Updated weights for policy 0, policy_version 89653 (0.0007) [2023-10-10 20:05:04,501][123582] Updated weights for policy 0, policy_version 89663 (0.0008) [2023-10-10 20:05:05,774][123614] Updated weights for policy 1, policy_version 89540 (0.0010) [2023-10-10 20:05:06,144][123614] Updated weights for policy 1, policy_version 89550 (0.0009) [2023-10-10 20:05:06,523][123614] Updated weights for policy 1, policy_version 89560 (0.0007) [2023-10-10 20:05:08,151][123582] Updated weights for policy 0, policy_version 89673 (0.0007) [2023-10-10 20:05:08,521][123582] Updated weights for policy 0, policy_version 89683 (0.0007) [2023-10-10 20:05:08,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 183533568. Throughput: 0: 1820.8, 1: 1806.4. Samples: 45899502. Policy #0 lag: (min: 0.0, avg: 21.7, max: 32.0) [2023-10-10 20:05:08,789][122664] Avg episode reward: [(0, '89.840'), (1, '112.800')] [2023-10-10 20:05:08,896][123582] Updated weights for policy 0, policy_version 89693 (0.0008) [2023-10-10 20:05:10,157][123614] Updated weights for policy 1, policy_version 89570 (0.0007) [2023-10-10 20:05:10,528][123614] Updated weights for policy 1, policy_version 89580 (0.0010) [2023-10-10 20:05:10,898][123614] Updated weights for policy 1, policy_version 89590 (0.0008) [2023-10-10 20:05:11,273][123614] Updated weights for policy 1, policy_version 89600 (0.0007) [2023-10-10 20:05:12,739][123582] Updated weights for policy 0, policy_version 89703 (0.0009) [2023-10-10 20:05:13,112][123582] Updated weights for policy 0, policy_version 89713 (0.0009) [2023-10-10 20:05:13,473][123582] Updated weights for policy 0, policy_version 89723 (0.0008) [2023-10-10 20:05:13,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 183631872. Throughput: 0: 1824.0, 1: 1808.1. Samples: 45910202. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:05:13,788][122664] Avg episode reward: [(0, '92.420'), (1, '113.070')] [2023-10-10 20:05:15,062][123614] Updated weights for policy 1, policy_version 89610 (0.0010) [2023-10-10 20:05:15,423][123614] Updated weights for policy 1, policy_version 89620 (0.0010) [2023-10-10 20:05:15,791][123614] Updated weights for policy 1, policy_version 89630 (0.0010) [2023-10-10 20:05:17,126][123582] Updated weights for policy 0, policy_version 89733 (0.0009) [2023-10-10 20:05:17,503][123582] Updated weights for policy 0, policy_version 89743 (0.0010) [2023-10-10 20:05:17,882][123582] Updated weights for policy 0, policy_version 89753 (0.0011) [2023-10-10 20:05:18,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 183697408. Throughput: 0: 1822.7, 1: 1806.8. Samples: 45932070. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:05:18,789][122664] Avg episode reward: [(0, '87.290'), (1, '110.180')] [2023-10-10 20:05:19,556][123614] Updated weights for policy 1, policy_version 89640 (0.0009) [2023-10-10 20:05:19,922][123614] Updated weights for policy 1, policy_version 89650 (0.0007) [2023-10-10 20:05:20,290][123614] Updated weights for policy 1, policy_version 89660 (0.0007) [2023-10-10 20:05:21,455][123582] Updated weights for policy 0, policy_version 89763 (0.0010) [2023-10-10 20:05:21,825][123582] Updated weights for policy 0, policy_version 89773 (0.0008) [2023-10-10 20:05:22,189][123582] Updated weights for policy 0, policy_version 89783 (0.0008) [2023-10-10 20:05:23,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 183762944. Throughput: 0: 1822.8, 1: 1813.9. Samples: 45953836. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:05:23,789][122664] Avg episode reward: [(0, '85.040'), (1, '97.770')] [2023-10-10 20:05:24,043][123614] Updated weights for policy 1, policy_version 89670 (0.0007) [2023-10-10 20:05:24,402][123614] Updated weights for policy 1, policy_version 89680 (0.0009) [2023-10-10 20:05:24,766][123614] Updated weights for policy 1, policy_version 89690 (0.0009) [2023-10-10 20:05:25,889][123582] Updated weights for policy 0, policy_version 89793 (0.0011) [2023-10-10 20:05:26,268][123582] Updated weights for policy 0, policy_version 89803 (0.0011) [2023-10-10 20:05:26,645][123582] Updated weights for policy 0, policy_version 89813 (0.0008) [2023-10-10 20:05:27,011][123582] Updated weights for policy 0, policy_version 89823 (0.0007) [2023-10-10 20:05:28,450][123614] Updated weights for policy 1, policy_version 89700 (0.0008) [2023-10-10 20:05:28,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 183828480. Throughput: 0: 1818.8, 1: 1809.5. Samples: 45964854. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:05:28,788][122664] Avg episode reward: [(0, '85.900'), (1, '92.270')] [2023-10-10 20:05:28,848][123614] Updated weights for policy 1, policy_version 89710 (0.0008) [2023-10-10 20:05:29,209][123614] Updated weights for policy 1, policy_version 89720 (0.0008) [2023-10-10 20:05:30,715][123582] Updated weights for policy 0, policy_version 89833 (0.0008) [2023-10-10 20:05:31,077][123582] Updated weights for policy 0, policy_version 89843 (0.0008) [2023-10-10 20:05:31,452][123582] Updated weights for policy 0, policy_version 89853 (0.0008) [2023-10-10 20:05:32,778][123614] Updated weights for policy 1, policy_version 89730 (0.0011) [2023-10-10 20:05:33,143][123614] Updated weights for policy 1, policy_version 89740 (0.0007) [2023-10-10 20:05:33,509][123614] Updated weights for policy 1, policy_version 89750 (0.0007) [2023-10-10 20:05:33,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 183894016. Throughput: 0: 1820.4, 1: 1813.4. Samples: 45986598. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:05:33,789][122664] Avg episode reward: [(0, '88.850'), (1, '91.340')] [2023-10-10 20:05:33,881][123614] Updated weights for policy 1, policy_version 89760 (0.0010) [2023-10-10 20:05:35,117][123582] Updated weights for policy 0, policy_version 89863 (0.0010) [2023-10-10 20:05:35,492][123582] Updated weights for policy 0, policy_version 89873 (0.0009) [2023-10-10 20:05:35,871][123582] Updated weights for policy 0, policy_version 89883 (0.0009) [2023-10-10 20:05:37,637][123614] Updated weights for policy 1, policy_version 89770 (0.0008) [2023-10-10 20:05:38,004][123614] Updated weights for policy 1, policy_version 89780 (0.0009) [2023-10-10 20:05:38,372][123614] Updated weights for policy 1, policy_version 89790 (0.0009) [2023-10-10 20:05:38,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 183992320. Throughput: 0: 1819.2, 1: 1811.7. Samples: 46007866. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:05:38,789][122664] Avg episode reward: [(0, '85.880'), (1, '89.430')] [2023-10-10 20:05:39,722][123582] Updated weights for policy 0, policy_version 89893 (0.0008) [2023-10-10 20:05:40,086][123582] Updated weights for policy 0, policy_version 89903 (0.0011) [2023-10-10 20:05:40,449][123582] Updated weights for policy 0, policy_version 89913 (0.0010) [2023-10-10 20:05:42,068][123614] Updated weights for policy 1, policy_version 89800 (0.0007) [2023-10-10 20:05:42,436][123614] Updated weights for policy 1, policy_version 89810 (0.0008) [2023-10-10 20:05:42,801][123614] Updated weights for policy 1, policy_version 89820 (0.0007) [2023-10-10 20:05:43,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 184057856. Throughput: 0: 1813.1, 1: 1810.9. Samples: 46018944. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:05:43,789][122664] Avg episode reward: [(0, '88.770'), (1, '81.170')] [2023-10-10 20:05:44,088][123582] Updated weights for policy 0, policy_version 89923 (0.0011) [2023-10-10 20:05:44,465][123582] Updated weights for policy 0, policy_version 89933 (0.0008) [2023-10-10 20:05:44,833][123582] Updated weights for policy 0, policy_version 89943 (0.0009) [2023-10-10 20:05:46,500][123614] Updated weights for policy 1, policy_version 89830 (0.0007) [2023-10-10 20:05:46,866][123614] Updated weights for policy 1, policy_version 89840 (0.0009) [2023-10-10 20:05:47,240][123614] Updated weights for policy 1, policy_version 89850 (0.0008) [2023-10-10 20:05:48,504][123582] Updated weights for policy 0, policy_version 89953 (0.0009) [2023-10-10 20:05:48,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 184123392. Throughput: 0: 1818.8, 1: 1806.9. Samples: 46040536. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:05:48,789][122664] Avg episode reward: [(0, '80.500'), (1, '73.250')] [2023-10-10 20:05:48,864][123582] Updated weights for policy 0, policy_version 89963 (0.0007) [2023-10-10 20:05:49,232][123582] Updated weights for policy 0, policy_version 89973 (0.0007) [2023-10-10 20:05:49,597][123582] Updated weights for policy 0, policy_version 89983 (0.0009) [2023-10-10 20:05:50,883][123614] Updated weights for policy 1, policy_version 89860 (0.0007) [2023-10-10 20:05:51,257][123614] Updated weights for policy 1, policy_version 89870 (0.0007) [2023-10-10 20:05:51,624][123614] Updated weights for policy 1, policy_version 89880 (0.0008) [2023-10-10 20:05:53,397][123582] Updated weights for policy 0, policy_version 89993 (0.0009) [2023-10-10 20:05:53,779][123582] Updated weights for policy 0, policy_version 90003 (0.0009) [2023-10-10 20:05:53,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 184188928. Throughput: 0: 1819.4, 1: 1808.5. Samples: 46062756. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:05:53,789][122664] Avg episode reward: [(0, '83.470'), (1, '68.210')] [2023-10-10 20:05:53,799][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000089888_92045312.pth... [2023-10-10 20:05:53,833][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000088192_90308608.pth [2023-10-10 20:05:54,149][123582] Updated weights for policy 0, policy_version 90013 (0.0007) [2023-10-10 20:05:54,256][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000090016_92176384.pth... [2023-10-10 20:05:54,288][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000088320_90439680.pth [2023-10-10 20:05:55,352][123614] Updated weights for policy 1, policy_version 89890 (0.0007) [2023-10-10 20:05:55,727][123614] Updated weights for policy 1, policy_version 89900 (0.0009) [2023-10-10 20:05:56,097][123614] Updated weights for policy 1, policy_version 89910 (0.0007) [2023-10-10 20:05:56,465][123614] Updated weights for policy 1, policy_version 89920 (0.0008) [2023-10-10 20:05:57,961][123582] Updated weights for policy 0, policy_version 90023 (0.0008) [2023-10-10 20:05:58,339][123582] Updated weights for policy 0, policy_version 90033 (0.0007) [2023-10-10 20:05:58,713][123582] Updated weights for policy 0, policy_version 90043 (0.0009) [2023-10-10 20:05:58,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 184254464. Throughput: 0: 1812.7, 1: 1807.5. Samples: 46073108. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:05:58,789][122664] Avg episode reward: [(0, '83.470'), (1, '69.450')] [2023-10-10 20:06:00,201][123614] Updated weights for policy 1, policy_version 89930 (0.0010) [2023-10-10 20:06:00,560][123614] Updated weights for policy 1, policy_version 89940 (0.0009) [2023-10-10 20:06:00,938][123614] Updated weights for policy 1, policy_version 89950 (0.0009) [2023-10-10 20:06:02,439][123582] Updated weights for policy 0, policy_version 90053 (0.0008) [2023-10-10 20:06:02,804][123582] Updated weights for policy 0, policy_version 90063 (0.0011) [2023-10-10 20:06:03,180][123582] Updated weights for policy 0, policy_version 90073 (0.0008) [2023-10-10 20:06:03,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 184352768. Throughput: 0: 1819.9, 1: 1812.9. Samples: 46095544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:06:03,789][122664] Avg episode reward: [(0, '83.030'), (1, '72.550')] [2023-10-10 20:06:04,630][123614] Updated weights for policy 1, policy_version 89960 (0.0010) [2023-10-10 20:06:04,990][123614] Updated weights for policy 1, policy_version 89970 (0.0008) [2023-10-10 20:06:05,362][123614] Updated weights for policy 1, policy_version 89980 (0.0007) [2023-10-10 20:06:06,798][123582] Updated weights for policy 0, policy_version 90083 (0.0007) [2023-10-10 20:06:07,171][123582] Updated weights for policy 0, policy_version 90093 (0.0008) [2023-10-10 20:06:07,553][123582] Updated weights for policy 0, policy_version 90103 (0.0007) [2023-10-10 20:06:08,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 184418304. Throughput: 0: 1815.1, 1: 1817.1. Samples: 46117286. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:06:08,788][122664] Avg episode reward: [(0, '86.730'), (1, '73.100')] [2023-10-10 20:06:09,038][123614] Updated weights for policy 1, policy_version 89990 (0.0009) [2023-10-10 20:06:09,398][123614] Updated weights for policy 1, policy_version 90000 (0.0010) [2023-10-10 20:06:09,774][123614] Updated weights for policy 1, policy_version 90010 (0.0008) [2023-10-10 20:06:11,116][123582] Updated weights for policy 0, policy_version 90113 (0.0009) [2023-10-10 20:06:11,488][123582] Updated weights for policy 0, policy_version 90123 (0.0008) [2023-10-10 20:06:11,863][123582] Updated weights for policy 0, policy_version 90133 (0.0008) [2023-10-10 20:06:12,232][123582] Updated weights for policy 0, policy_version 90143 (0.0007) [2023-10-10 20:06:13,433][123614] Updated weights for policy 1, policy_version 90020 (0.0009) [2023-10-10 20:06:13,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 184483840. Throughput: 0: 1818.7, 1: 1813.3. Samples: 46128292. Policy #0 lag: (min: 16.0, avg: 44.9, max: 48.0) [2023-10-10 20:06:13,788][122664] Avg episode reward: [(0, '90.530'), (1, '73.870')] [2023-10-10 20:06:13,820][123614] Updated weights for policy 1, policy_version 90030 (0.0010) [2023-10-10 20:06:14,183][123614] Updated weights for policy 1, policy_version 90040 (0.0009) [2023-10-10 20:06:15,781][123582] Updated weights for policy 0, policy_version 90153 (0.0011) [2023-10-10 20:06:16,147][123582] Updated weights for policy 0, policy_version 90163 (0.0008) [2023-10-10 20:06:16,519][123582] Updated weights for policy 0, policy_version 90173 (0.0009) [2023-10-10 20:06:17,858][123614] Updated weights for policy 1, policy_version 90050 (0.0009) [2023-10-10 20:06:18,229][123614] Updated weights for policy 1, policy_version 90060 (0.0008) [2023-10-10 20:06:18,610][123614] Updated weights for policy 1, policy_version 90070 (0.0010) [2023-10-10 20:06:18,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 184549376. Throughput: 0: 1817.4, 1: 1813.1. Samples: 46149970. Policy #0 lag: (min: 16.0, avg: 44.9, max: 48.0) [2023-10-10 20:06:18,789][122664] Avg episode reward: [(0, '91.900'), (1, '70.900')] [2023-10-10 20:06:18,979][123614] Updated weights for policy 1, policy_version 90080 (0.0009) [2023-10-10 20:06:20,268][123582] Updated weights for policy 0, policy_version 90183 (0.0008) [2023-10-10 20:06:20,640][123582] Updated weights for policy 0, policy_version 90193 (0.0009) [2023-10-10 20:06:21,015][123582] Updated weights for policy 0, policy_version 90203 (0.0007) [2023-10-10 20:06:22,749][123614] Updated weights for policy 1, policy_version 90090 (0.0008) [2023-10-10 20:06:23,113][123614] Updated weights for policy 1, policy_version 90100 (0.0008) [2023-10-10 20:06:23,490][123614] Updated weights for policy 1, policy_version 90110 (0.0009) [2023-10-10 20:06:23,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 184647680. Throughput: 0: 1824.4, 1: 1812.5. Samples: 46171528. Policy #0 lag: (min: 16.0, avg: 44.9, max: 48.0) [2023-10-10 20:06:23,788][122664] Avg episode reward: [(0, '95.140'), (1, '78.220')] [2023-10-10 20:06:24,535][123582] Updated weights for policy 0, policy_version 90213 (0.0008) [2023-10-10 20:06:24,911][123582] Updated weights for policy 0, policy_version 90223 (0.0008) [2023-10-10 20:06:25,285][123582] Updated weights for policy 0, policy_version 90233 (0.0009) [2023-10-10 20:06:27,237][123614] Updated weights for policy 1, policy_version 90120 (0.0009) [2023-10-10 20:06:27,603][123614] Updated weights for policy 1, policy_version 90130 (0.0008) [2023-10-10 20:06:27,973][123614] Updated weights for policy 1, policy_version 90140 (0.0008) [2023-10-10 20:06:28,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 184713216. Throughput: 0: 1829.0, 1: 1815.4. Samples: 46182944. Policy #0 lag: (min: 16.0, avg: 44.9, max: 48.0) [2023-10-10 20:06:28,788][122664] Avg episode reward: [(0, '95.900'), (1, '80.030')] [2023-10-10 20:06:28,970][123582] Updated weights for policy 0, policy_version 90243 (0.0007) [2023-10-10 20:06:29,343][123582] Updated weights for policy 0, policy_version 90253 (0.0009) [2023-10-10 20:06:29,718][123582] Updated weights for policy 0, policy_version 90263 (0.0008) [2023-10-10 20:06:31,487][123614] Updated weights for policy 1, policy_version 90150 (0.0007) [2023-10-10 20:06:31,863][123614] Updated weights for policy 1, policy_version 90160 (0.0008) [2023-10-10 20:06:32,225][123614] Updated weights for policy 1, policy_version 90170 (0.0008) [2023-10-10 20:06:33,387][123582] Updated weights for policy 0, policy_version 90273 (0.0010) [2023-10-10 20:06:33,758][123582] Updated weights for policy 0, policy_version 90283 (0.0011) [2023-10-10 20:06:33,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 184778752. Throughput: 0: 1822.1, 1: 1816.8. Samples: 46204284. Policy #0 lag: (min: 16.0, avg: 44.9, max: 48.0) [2023-10-10 20:06:33,788][122664] Avg episode reward: [(0, '96.400'), (1, '80.400')] [2023-10-10 20:06:34,123][123582] Updated weights for policy 0, policy_version 90293 (0.0008) [2023-10-10 20:06:34,486][123582] Updated weights for policy 0, policy_version 90303 (0.0009) [2023-10-10 20:06:35,942][123614] Updated weights for policy 1, policy_version 90180 (0.0010) [2023-10-10 20:06:36,313][123614] Updated weights for policy 1, policy_version 90190 (0.0010) [2023-10-10 20:06:36,686][123614] Updated weights for policy 1, policy_version 90200 (0.0009) [2023-10-10 20:06:38,332][123582] Updated weights for policy 0, policy_version 90313 (0.0007) [2023-10-10 20:06:38,704][123582] Updated weights for policy 0, policy_version 90323 (0.0007) [2023-10-10 20:06:38,788][122664] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 184844288. Throughput: 0: 1824.7, 1: 1814.8. Samples: 46226536. Policy #0 lag: (min: 16.0, avg: 44.9, max: 48.0) [2023-10-10 20:06:38,789][122664] Avg episode reward: [(0, '96.750'), (1, '84.820')] [2023-10-10 20:06:39,073][123582] Updated weights for policy 0, policy_version 90333 (0.0008) [2023-10-10 20:06:40,308][123614] Updated weights for policy 1, policy_version 90210 (0.0008) [2023-10-10 20:06:40,669][123614] Updated weights for policy 1, policy_version 90220 (0.0011) [2023-10-10 20:06:41,046][123614] Updated weights for policy 1, policy_version 90230 (0.0010) [2023-10-10 20:06:41,414][123614] Updated weights for policy 1, policy_version 90240 (0.0010) [2023-10-10 20:06:42,838][123582] Updated weights for policy 0, policy_version 90343 (0.0010) [2023-10-10 20:06:43,216][123582] Updated weights for policy 0, policy_version 90353 (0.0007) [2023-10-10 20:06:43,591][123582] Updated weights for policy 0, policy_version 90363 (0.0008) [2023-10-10 20:06:43,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 184942592. Throughput: 0: 1828.0, 1: 1815.2. Samples: 46237054. Policy #0 lag: (min: 16.0, avg: 44.9, max: 48.0) [2023-10-10 20:06:43,789][122664] Avg episode reward: [(0, '104.810'), (1, '87.310')] [2023-10-10 20:06:45,229][123614] Updated weights for policy 1, policy_version 90250 (0.0007) [2023-10-10 20:06:45,592][123614] Updated weights for policy 1, policy_version 90260 (0.0007) [2023-10-10 20:06:45,959][123614] Updated weights for policy 1, policy_version 90270 (0.0007) [2023-10-10 20:06:47,226][123582] Updated weights for policy 0, policy_version 90373 (0.0010) [2023-10-10 20:06:47,596][123582] Updated weights for policy 0, policy_version 90383 (0.0010) [2023-10-10 20:06:47,977][123582] Updated weights for policy 0, policy_version 90393 (0.0009) [2023-10-10 20:06:48,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 185008128. Throughput: 0: 1826.8, 1: 1816.6. Samples: 46259500. Policy #0 lag: (min: 16.0, avg: 44.9, max: 48.0) [2023-10-10 20:06:48,789][122664] Avg episode reward: [(0, '101.500'), (1, '89.890')] [2023-10-10 20:06:49,659][123614] Updated weights for policy 1, policy_version 90280 (0.0007) [2023-10-10 20:06:50,032][123614] Updated weights for policy 1, policy_version 90290 (0.0007) [2023-10-10 20:06:50,394][123614] Updated weights for policy 1, policy_version 90300 (0.0007) [2023-10-10 20:06:51,571][123582] Updated weights for policy 0, policy_version 90403 (0.0008) [2023-10-10 20:06:51,936][123582] Updated weights for policy 0, policy_version 90413 (0.0008) [2023-10-10 20:06:52,311][123582] Updated weights for policy 0, policy_version 90423 (0.0007) [2023-10-10 20:06:53,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 185073664. Throughput: 0: 1832.4, 1: 1813.9. Samples: 46281370. Policy #0 lag: (min: 16.0, avg: 44.9, max: 48.0) [2023-10-10 20:06:53,788][122664] Avg episode reward: [(0, '102.790'), (1, '90.330')] [2023-10-10 20:06:54,175][123614] Updated weights for policy 1, policy_version 90310 (0.0008) [2023-10-10 20:06:54,551][123614] Updated weights for policy 1, policy_version 90320 (0.0009) [2023-10-10 20:06:54,920][123614] Updated weights for policy 1, policy_version 90330 (0.0008) [2023-10-10 20:06:55,937][123582] Updated weights for policy 0, policy_version 90433 (0.0007) [2023-10-10 20:06:56,309][123582] Updated weights for policy 0, policy_version 90443 (0.0008) [2023-10-10 20:06:56,684][123582] Updated weights for policy 0, policy_version 90453 (0.0009) [2023-10-10 20:06:57,050][123582] Updated weights for policy 0, policy_version 90463 (0.0011) [2023-10-10 20:06:58,641][123614] Updated weights for policy 1, policy_version 90340 (0.0009) [2023-10-10 20:06:58,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 185139200. Throughput: 0: 1831.0, 1: 1811.4. Samples: 46292200. Policy #0 lag: (min: 16.0, avg: 44.9, max: 48.0) [2023-10-10 20:06:58,789][122664] Avg episode reward: [(0, '106.350'), (1, '95.560')] [2023-10-10 20:06:59,035][123614] Updated weights for policy 1, policy_version 90350 (0.0008) [2023-10-10 20:06:59,391][123614] Updated weights for policy 1, policy_version 90360 (0.0008) [2023-10-10 20:07:00,722][123582] Updated weights for policy 0, policy_version 90473 (0.0008) [2023-10-10 20:07:01,092][123582] Updated weights for policy 0, policy_version 90483 (0.0009) [2023-10-10 20:07:01,466][123582] Updated weights for policy 0, policy_version 90493 (0.0009) [2023-10-10 20:07:02,925][123614] Updated weights for policy 1, policy_version 90370 (0.0010) [2023-10-10 20:07:03,288][123614] Updated weights for policy 1, policy_version 90380 (0.0008) [2023-10-10 20:07:03,661][123614] Updated weights for policy 1, policy_version 90390 (0.0008) [2023-10-10 20:07:03,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 185204736. Throughput: 0: 1833.3, 1: 1817.9. Samples: 46314274. Policy #0 lag: (min: 16.0, avg: 44.9, max: 48.0) [2023-10-10 20:07:03,788][122664] Avg episode reward: [(0, '105.800'), (1, '95.510')] [2023-10-10 20:07:04,034][123614] Updated weights for policy 1, policy_version 90400 (0.0009) [2023-10-10 20:07:05,159][123582] Updated weights for policy 0, policy_version 90503 (0.0008) [2023-10-10 20:07:05,530][123582] Updated weights for policy 0, policy_version 90513 (0.0008) [2023-10-10 20:07:05,905][123582] Updated weights for policy 0, policy_version 90523 (0.0009) [2023-10-10 20:07:07,745][123614] Updated weights for policy 1, policy_version 90410 (0.0009) [2023-10-10 20:07:08,119][123614] Updated weights for policy 1, policy_version 90420 (0.0008) [2023-10-10 20:07:08,491][123614] Updated weights for policy 1, policy_version 90430 (0.0008) [2023-10-10 20:07:08,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 185303040. Throughput: 0: 1828.8, 1: 1817.8. Samples: 46335626. Policy #0 lag: (min: 16.0, avg: 44.9, max: 48.0) [2023-10-10 20:07:08,789][122664] Avg episode reward: [(0, '103.820'), (1, '96.890')] [2023-10-10 20:07:09,431][123582] Updated weights for policy 0, policy_version 90533 (0.0009) [2023-10-10 20:07:09,811][123582] Updated weights for policy 0, policy_version 90543 (0.0008) [2023-10-10 20:07:10,181][123582] Updated weights for policy 0, policy_version 90553 (0.0010) [2023-10-10 20:07:12,151][123614] Updated weights for policy 1, policy_version 90440 (0.0009) [2023-10-10 20:07:12,522][123614] Updated weights for policy 1, policy_version 90450 (0.0010) [2023-10-10 20:07:12,886][123614] Updated weights for policy 1, policy_version 90460 (0.0010) [2023-10-10 20:07:13,736][123582] Updated weights for policy 0, policy_version 90563 (0.0009) [2023-10-10 20:07:13,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 185368576. Throughput: 0: 1830.5, 1: 1820.1. Samples: 46347222. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 20:07:13,789][122664] Avg episode reward: [(0, '99.840'), (1, '95.560')] [2023-10-10 20:07:14,101][123582] Updated weights for policy 0, policy_version 90573 (0.0009) [2023-10-10 20:07:14,475][123582] Updated weights for policy 0, policy_version 90583 (0.0009) [2023-10-10 20:07:16,672][123614] Updated weights for policy 1, policy_version 90470 (0.0010) [2023-10-10 20:07:17,044][123614] Updated weights for policy 1, policy_version 90480 (0.0010) [2023-10-10 20:07:17,419][123614] Updated weights for policy 1, policy_version 90490 (0.0009) [2023-10-10 20:07:18,188][123582] Updated weights for policy 0, policy_version 90593 (0.0010) [2023-10-10 20:07:18,556][123582] Updated weights for policy 0, policy_version 90603 (0.0010) [2023-10-10 20:07:18,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 185434112. Throughput: 0: 1834.8, 1: 1815.8. Samples: 46368560. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 20:07:18,789][122664] Avg episode reward: [(0, '101.080'), (1, '97.610')] [2023-10-10 20:07:18,915][123582] Updated weights for policy 0, policy_version 90613 (0.0009) [2023-10-10 20:07:19,287][123582] Updated weights for policy 0, policy_version 90623 (0.0009) [2023-10-10 20:07:21,361][123614] Updated weights for policy 1, policy_version 90500 (0.0008) [2023-10-10 20:07:21,740][123614] Updated weights for policy 1, policy_version 90510 (0.0007) [2023-10-10 20:07:22,113][123614] Updated weights for policy 1, policy_version 90520 (0.0008) [2023-10-10 20:07:22,975][123582] Updated weights for policy 0, policy_version 90633 (0.0009) [2023-10-10 20:07:23,349][123582] Updated weights for policy 0, policy_version 90643 (0.0010) [2023-10-10 20:07:23,712][123582] Updated weights for policy 0, policy_version 90653 (0.0008) [2023-10-10 20:07:23,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 185499648. Throughput: 0: 1828.0, 1: 1812.1. Samples: 46390338. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 20:07:23,789][122664] Avg episode reward: [(0, '107.650'), (1, '103.030')] [2023-10-10 20:07:25,802][123614] Updated weights for policy 1, policy_version 90530 (0.0009) [2023-10-10 20:07:26,170][123614] Updated weights for policy 1, policy_version 90540 (0.0008) [2023-10-10 20:07:26,549][123614] Updated weights for policy 1, policy_version 90550 (0.0009) [2023-10-10 20:07:26,919][123614] Updated weights for policy 1, policy_version 90560 (0.0010) [2023-10-10 20:07:27,362][123582] Updated weights for policy 0, policy_version 90663 (0.0008) [2023-10-10 20:07:27,746][123582] Updated weights for policy 0, policy_version 90673 (0.0007) [2023-10-10 20:07:28,124][123582] Updated weights for policy 0, policy_version 90683 (0.0009) [2023-10-10 20:07:28,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 185597952. Throughput: 0: 1838.2, 1: 1816.9. Samples: 46401534. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 20:07:28,789][122664] Avg episode reward: [(0, '106.490'), (1, '101.880')] [2023-10-10 20:07:30,583][123614] Updated weights for policy 1, policy_version 90570 (0.0009) [2023-10-10 20:07:30,958][123614] Updated weights for policy 1, policy_version 90580 (0.0010) [2023-10-10 20:07:31,322][123614] Updated weights for policy 1, policy_version 90590 (0.0009) [2023-10-10 20:07:31,826][123582] Updated weights for policy 0, policy_version 90693 (0.0008) [2023-10-10 20:07:32,193][123582] Updated weights for policy 0, policy_version 90703 (0.0007) [2023-10-10 20:07:32,564][123582] Updated weights for policy 0, policy_version 90713 (0.0007) [2023-10-10 20:07:33,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 185663488. Throughput: 0: 1821.1, 1: 1811.7. Samples: 46422976. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 20:07:33,789][122664] Avg episode reward: [(0, '108.280'), (1, '97.710')] [2023-10-10 20:07:34,961][123614] Updated weights for policy 1, policy_version 90600 (0.0009) [2023-10-10 20:07:35,327][123614] Updated weights for policy 1, policy_version 90610 (0.0008) [2023-10-10 20:07:35,700][123614] Updated weights for policy 1, policy_version 90620 (0.0008) [2023-10-10 20:07:36,377][123582] Updated weights for policy 0, policy_version 90723 (0.0008) [2023-10-10 20:07:36,736][123582] Updated weights for policy 0, policy_version 90733 (0.0010) [2023-10-10 20:07:37,104][123582] Updated weights for policy 0, policy_version 90743 (0.0011) [2023-10-10 20:07:38,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 185729024. Throughput: 0: 1823.1, 1: 1819.4. Samples: 46445280. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 20:07:38,789][122664] Avg episode reward: [(0, '110.930'), (1, '91.180')] [2023-10-10 20:07:39,192][123614] Updated weights for policy 1, policy_version 90630 (0.0010) [2023-10-10 20:07:39,551][123614] Updated weights for policy 1, policy_version 90640 (0.0010) [2023-10-10 20:07:39,924][123614] Updated weights for policy 1, policy_version 90650 (0.0010) [2023-10-10 20:07:40,865][123582] Updated weights for policy 0, policy_version 90753 (0.0010) [2023-10-10 20:07:41,240][123582] Updated weights for policy 0, policy_version 90763 (0.0009) [2023-10-10 20:07:41,609][123582] Updated weights for policy 0, policy_version 90773 (0.0010) [2023-10-10 20:07:41,980][123582] Updated weights for policy 0, policy_version 90783 (0.0007) [2023-10-10 20:07:43,431][123614] Updated weights for policy 1, policy_version 90660 (0.0007) [2023-10-10 20:07:43,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 185794560. Throughput: 0: 1818.8, 1: 1824.6. Samples: 46456152. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 20:07:43,788][122664] Avg episode reward: [(0, '113.100'), (1, '84.240')] [2023-10-10 20:07:43,809][123614] Updated weights for policy 1, policy_version 90670 (0.0007) [2023-10-10 20:07:44,180][123614] Updated weights for policy 1, policy_version 90680 (0.0009) [2023-10-10 20:07:45,697][123582] Updated weights for policy 0, policy_version 90793 (0.0008) [2023-10-10 20:07:46,062][123582] Updated weights for policy 0, policy_version 90803 (0.0008) [2023-10-10 20:07:46,442][123582] Updated weights for policy 0, policy_version 90813 (0.0007) [2023-10-10 20:07:47,955][123614] Updated weights for policy 1, policy_version 90690 (0.0008) [2023-10-10 20:07:48,325][123614] Updated weights for policy 1, policy_version 90700 (0.0007) [2023-10-10 20:07:48,701][123614] Updated weights for policy 1, policy_version 90710 (0.0007) [2023-10-10 20:07:48,788][122664] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 185860096. Throughput: 0: 1820.6, 1: 1820.3. Samples: 46478114. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 20:07:48,788][122664] Avg episode reward: [(0, '114.080'), (1, '83.930')] [2023-10-10 20:07:49,060][123614] Updated weights for policy 1, policy_version 90720 (0.0008) [2023-10-10 20:07:50,000][123582] Updated weights for policy 0, policy_version 90823 (0.0009) [2023-10-10 20:07:50,368][123582] Updated weights for policy 0, policy_version 90833 (0.0009) [2023-10-10 20:07:50,734][123582] Updated weights for policy 0, policy_version 90843 (0.0008) [2023-10-10 20:07:52,727][123614] Updated weights for policy 1, policy_version 90730 (0.0008) [2023-10-10 20:07:53,107][123614] Updated weights for policy 1, policy_version 90740 (0.0011) [2023-10-10 20:07:53,468][123614] Updated weights for policy 1, policy_version 90750 (0.0009) [2023-10-10 20:07:53,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 185958400. Throughput: 0: 1818.6, 1: 1822.5. Samples: 46499476. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 20:07:53,788][122664] Avg episode reward: [(0, '116.750'), (1, '85.480')] [2023-10-10 20:07:53,799][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000090848_93028352.pth... [2023-10-10 20:07:53,799][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000090752_92930048.pth... [2023-10-10 20:07:53,832][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000089152_91291648.pth [2023-10-10 20:07:53,834][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000089056_91193344.pth [2023-10-10 20:07:54,389][123582] Updated weights for policy 0, policy_version 90853 (0.0008) [2023-10-10 20:07:54,769][123582] Updated weights for policy 0, policy_version 90863 (0.0010) [2023-10-10 20:07:55,144][123582] Updated weights for policy 0, policy_version 90873 (0.0011) [2023-10-10 20:07:57,167][123614] Updated weights for policy 1, policy_version 90760 (0.0009) [2023-10-10 20:07:57,537][123614] Updated weights for policy 1, policy_version 90770 (0.0007) [2023-10-10 20:07:57,913][123614] Updated weights for policy 1, policy_version 90780 (0.0008) [2023-10-10 20:07:58,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 186023936. Throughput: 0: 1817.1, 1: 1819.9. Samples: 46510886. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 20:07:58,789][122664] Avg episode reward: [(0, '113.480'), (1, '82.900')] [2023-10-10 20:07:58,818][123582] Updated weights for policy 0, policy_version 90883 (0.0010) [2023-10-10 20:07:59,199][123582] Updated weights for policy 0, policy_version 90893 (0.0009) [2023-10-10 20:07:59,566][123582] Updated weights for policy 0, policy_version 90903 (0.0008) [2023-10-10 20:08:01,700][123614] Updated weights for policy 1, policy_version 90790 (0.0007) [2023-10-10 20:08:02,074][123614] Updated weights for policy 1, policy_version 90800 (0.0008) [2023-10-10 20:08:02,443][123614] Updated weights for policy 1, policy_version 90810 (0.0008) [2023-10-10 20:08:03,096][123582] Updated weights for policy 0, policy_version 90913 (0.0008) [2023-10-10 20:08:03,462][123582] Updated weights for policy 0, policy_version 90923 (0.0008) [2023-10-10 20:08:03,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 186089472. Throughput: 0: 1819.0, 1: 1819.7. Samples: 46532302. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 20:08:03,788][122664] Avg episode reward: [(0, '113.030'), (1, '80.020')] [2023-10-10 20:08:03,830][123582] Updated weights for policy 0, policy_version 90933 (0.0008) [2023-10-10 20:08:04,203][123582] Updated weights for policy 0, policy_version 90943 (0.0010) [2023-10-10 20:08:06,185][123614] Updated weights for policy 1, policy_version 90820 (0.0009) [2023-10-10 20:08:06,556][123614] Updated weights for policy 1, policy_version 90830 (0.0008) [2023-10-10 20:08:06,929][123614] Updated weights for policy 1, policy_version 90840 (0.0009) [2023-10-10 20:08:08,042][123582] Updated weights for policy 0, policy_version 90953 (0.0008) [2023-10-10 20:08:08,405][123582] Updated weights for policy 0, policy_version 90963 (0.0010) [2023-10-10 20:08:08,778][123582] Updated weights for policy 0, policy_version 90973 (0.0008) [2023-10-10 20:08:08,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 186155008. Throughput: 0: 1817.1, 1: 1822.2. Samples: 46554106. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 20:08:08,788][122664] Avg episode reward: [(0, '120.130'), (1, '83.190')] [2023-10-10 20:08:10,634][123614] Updated weights for policy 1, policy_version 90850 (0.0010) [2023-10-10 20:08:11,000][123614] Updated weights for policy 1, policy_version 90860 (0.0008) [2023-10-10 20:08:11,373][123614] Updated weights for policy 1, policy_version 90870 (0.0009) [2023-10-10 20:08:11,744][123614] Updated weights for policy 1, policy_version 90880 (0.0011) [2023-10-10 20:08:12,432][123582] Updated weights for policy 0, policy_version 90983 (0.0008) [2023-10-10 20:08:12,812][123582] Updated weights for policy 0, policy_version 90993 (0.0009) [2023-10-10 20:08:13,173][123582] Updated weights for policy 0, policy_version 91003 (0.0007) [2023-10-10 20:08:13,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 186253312. Throughput: 0: 1812.8, 1: 1818.7. Samples: 46564948. Policy #0 lag: (min: 29.0, avg: 34.0, max: 61.0) [2023-10-10 20:08:13,789][122664] Avg episode reward: [(0, '117.330'), (1, '84.960')] [2023-10-10 20:08:15,451][123614] Updated weights for policy 1, policy_version 90890 (0.0008) [2023-10-10 20:08:15,827][123614] Updated weights for policy 1, policy_version 90900 (0.0008) [2023-10-10 20:08:16,188][123614] Updated weights for policy 1, policy_version 90910 (0.0009) [2023-10-10 20:08:16,882][123582] Updated weights for policy 0, policy_version 91013 (0.0009) [2023-10-10 20:08:17,255][123582] Updated weights for policy 0, policy_version 91023 (0.0007) [2023-10-10 20:08:17,633][123582] Updated weights for policy 0, policy_version 91033 (0.0008) [2023-10-10 20:08:18,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 186318848. Throughput: 0: 1816.1, 1: 1820.1. Samples: 46586606. Policy #0 lag: (min: 29.0, avg: 34.0, max: 61.0) [2023-10-10 20:08:18,788][122664] Avg episode reward: [(0, '116.230'), (1, '84.420')] [2023-10-10 20:08:19,975][123614] Updated weights for policy 1, policy_version 90920 (0.0009) [2023-10-10 20:08:20,350][123614] Updated weights for policy 1, policy_version 90930 (0.0008) [2023-10-10 20:08:20,721][123614] Updated weights for policy 1, policy_version 90940 (0.0007) [2023-10-10 20:08:21,313][123582] Updated weights for policy 0, policy_version 91043 (0.0008) [2023-10-10 20:08:21,680][123582] Updated weights for policy 0, policy_version 91053 (0.0009) [2023-10-10 20:08:22,055][123582] Updated weights for policy 0, policy_version 91063 (0.0009) [2023-10-10 20:08:23,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 186384384. Throughput: 0: 1817.7, 1: 1810.3. Samples: 46608542. Policy #0 lag: (min: 29.0, avg: 34.0, max: 61.0) [2023-10-10 20:08:23,789][122664] Avg episode reward: [(0, '109.230'), (1, '85.360')] [2023-10-10 20:08:24,424][123614] Updated weights for policy 1, policy_version 90950 (0.0007) [2023-10-10 20:08:24,784][123614] Updated weights for policy 1, policy_version 90960 (0.0007) [2023-10-10 20:08:25,147][123614] Updated weights for policy 1, policy_version 90970 (0.0007) [2023-10-10 20:08:25,843][123582] Updated weights for policy 0, policy_version 91073 (0.0012) [2023-10-10 20:08:26,215][123582] Updated weights for policy 0, policy_version 91083 (0.0008) [2023-10-10 20:08:26,596][123582] Updated weights for policy 0, policy_version 91093 (0.0007) [2023-10-10 20:08:26,977][123582] Updated weights for policy 0, policy_version 91103 (0.0008) [2023-10-10 20:08:28,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 186449920. Throughput: 0: 1815.8, 1: 1807.4. Samples: 46619196. Policy #0 lag: (min: 29.0, avg: 34.0, max: 61.0) [2023-10-10 20:08:28,788][122664] Avg episode reward: [(0, '110.480'), (1, '87.500')] [2023-10-10 20:08:28,906][123614] Updated weights for policy 1, policy_version 90980 (0.0009) [2023-10-10 20:08:29,280][123614] Updated weights for policy 1, policy_version 90990 (0.0009) [2023-10-10 20:08:29,641][123614] Updated weights for policy 1, policy_version 91000 (0.0008) [2023-10-10 20:08:30,562][123582] Updated weights for policy 0, policy_version 91113 (0.0009) [2023-10-10 20:08:30,929][123582] Updated weights for policy 0, policy_version 91123 (0.0008) [2023-10-10 20:08:31,297][123582] Updated weights for policy 0, policy_version 91133 (0.0007) [2023-10-10 20:08:33,289][123614] Updated weights for policy 1, policy_version 91010 (0.0010) [2023-10-10 20:08:33,660][123614] Updated weights for policy 1, policy_version 91020 (0.0007) [2023-10-10 20:08:33,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 186515456. Throughput: 0: 1819.6, 1: 1811.1. Samples: 46641494. Policy #0 lag: (min: 29.0, avg: 34.0, max: 61.0) [2023-10-10 20:08:33,788][122664] Avg episode reward: [(0, '108.070'), (1, '89.440')] [2023-10-10 20:08:34,033][123614] Updated weights for policy 1, policy_version 91030 (0.0007) [2023-10-10 20:08:34,405][123614] Updated weights for policy 1, policy_version 91040 (0.0007) [2023-10-10 20:08:35,041][123582] Updated weights for policy 0, policy_version 91143 (0.0007) [2023-10-10 20:08:35,408][123582] Updated weights for policy 0, policy_version 91153 (0.0008) [2023-10-10 20:08:35,786][123582] Updated weights for policy 0, policy_version 91163 (0.0009) [2023-10-10 20:08:38,062][123614] Updated weights for policy 1, policy_version 91050 (0.0010) [2023-10-10 20:08:38,421][123614] Updated weights for policy 1, policy_version 91060 (0.0008) [2023-10-10 20:08:38,785][123614] Updated weights for policy 1, policy_version 91070 (0.0007) [2023-10-10 20:08:38,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 186580992. Throughput: 0: 1818.7, 1: 1814.8. Samples: 46662986. Policy #0 lag: (min: 29.0, avg: 34.0, max: 61.0) [2023-10-10 20:08:38,788][122664] Avg episode reward: [(0, '109.480'), (1, '84.050')] [2023-10-10 20:08:39,384][123582] Updated weights for policy 0, policy_version 91173 (0.0009) [2023-10-10 20:08:39,747][123582] Updated weights for policy 0, policy_version 91183 (0.0009) [2023-10-10 20:08:40,108][123582] Updated weights for policy 0, policy_version 91193 (0.0007) [2023-10-10 20:08:42,364][123614] Updated weights for policy 1, policy_version 91080 (0.0008) [2023-10-10 20:08:42,728][123614] Updated weights for policy 1, policy_version 91090 (0.0007) [2023-10-10 20:08:43,096][123614] Updated weights for policy 1, policy_version 91100 (0.0008) [2023-10-10 20:08:43,687][123582] Updated weights for policy 0, policy_version 91203 (0.0007) [2023-10-10 20:08:43,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 186679296. Throughput: 0: 1820.9, 1: 1811.6. Samples: 46674348. Policy #0 lag: (min: 29.0, avg: 34.0, max: 61.0) [2023-10-10 20:08:43,788][122664] Avg episode reward: [(0, '107.050'), (1, '83.030')] [2023-10-10 20:08:44,059][123582] Updated weights for policy 0, policy_version 91213 (0.0009) [2023-10-10 20:08:44,439][123582] Updated weights for policy 0, policy_version 91223 (0.0008) [2023-10-10 20:08:46,783][123614] Updated weights for policy 1, policy_version 91110 (0.0009) [2023-10-10 20:08:47,148][123614] Updated weights for policy 1, policy_version 91120 (0.0009) [2023-10-10 20:08:47,525][123614] Updated weights for policy 1, policy_version 91130 (0.0008) [2023-10-10 20:08:48,093][123582] Updated weights for policy 0, policy_version 91233 (0.0008) [2023-10-10 20:08:48,464][123582] Updated weights for policy 0, policy_version 91243 (0.0007) [2023-10-10 20:08:48,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 186744832. Throughput: 0: 1819.5, 1: 1821.7. Samples: 46696156. Policy #0 lag: (min: 29.0, avg: 34.0, max: 61.0) [2023-10-10 20:08:48,789][122664] Avg episode reward: [(0, '107.720'), (1, '82.140')] [2023-10-10 20:08:48,837][123582] Updated weights for policy 0, policy_version 91253 (0.0009) [2023-10-10 20:08:49,216][123582] Updated weights for policy 0, policy_version 91263 (0.0008) [2023-10-10 20:08:51,129][123614] Updated weights for policy 1, policy_version 91140 (0.0011) [2023-10-10 20:08:51,496][123614] Updated weights for policy 1, policy_version 91150 (0.0008) [2023-10-10 20:08:51,874][123614] Updated weights for policy 1, policy_version 91160 (0.0010) [2023-10-10 20:08:52,943][123582] Updated weights for policy 0, policy_version 91273 (0.0007) [2023-10-10 20:08:53,309][123582] Updated weights for policy 0, policy_version 91283 (0.0008) [2023-10-10 20:08:53,674][123582] Updated weights for policy 0, policy_version 91293 (0.0011) [2023-10-10 20:08:53,788][122664] Fps is (10 sec: 16383.4, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 186843136. Throughput: 0: 1814.7, 1: 1823.4. Samples: 46717820. Policy #0 lag: (min: 29.0, avg: 34.0, max: 61.0) [2023-10-10 20:08:53,789][122664] Avg episode reward: [(0, '101.470'), (1, '85.140')] [2023-10-10 20:08:55,585][123614] Updated weights for policy 1, policy_version 91170 (0.0007) [2023-10-10 20:08:55,951][123614] Updated weights for policy 1, policy_version 91180 (0.0011) [2023-10-10 20:08:56,322][123614] Updated weights for policy 1, policy_version 91190 (0.0008) [2023-10-10 20:08:56,689][123614] Updated weights for policy 1, policy_version 91200 (0.0008) [2023-10-10 20:08:57,473][123582] Updated weights for policy 0, policy_version 91303 (0.0008) [2023-10-10 20:08:57,842][123582] Updated weights for policy 0, policy_version 91313 (0.0009) [2023-10-10 20:08:58,208][123582] Updated weights for policy 0, policy_version 91323 (0.0009) [2023-10-10 20:08:58,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 186908672. Throughput: 0: 1820.0, 1: 1821.3. Samples: 46728806. Policy #0 lag: (min: 29.0, avg: 34.0, max: 61.0) [2023-10-10 20:08:58,789][122664] Avg episode reward: [(0, '98.630'), (1, '86.310')] [2023-10-10 20:09:00,265][123614] Updated weights for policy 1, policy_version 91210 (0.0009) [2023-10-10 20:09:00,635][123614] Updated weights for policy 1, policy_version 91220 (0.0009) [2023-10-10 20:09:01,004][123614] Updated weights for policy 1, policy_version 91230 (0.0007) [2023-10-10 20:09:01,902][123582] Updated weights for policy 0, policy_version 91333 (0.0010) [2023-10-10 20:09:02,275][123582] Updated weights for policy 0, policy_version 91343 (0.0008) [2023-10-10 20:09:02,652][123582] Updated weights for policy 0, policy_version 91353 (0.0007) [2023-10-10 20:09:03,788][122664] Fps is (10 sec: 13107.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 186974208. Throughput: 0: 1819.6, 1: 1826.5. Samples: 46750678. Policy #0 lag: (min: 29.0, avg: 34.0, max: 61.0) [2023-10-10 20:09:03,788][122664] Avg episode reward: [(0, '102.570'), (1, '90.680')] [2023-10-10 20:09:04,649][123614] Updated weights for policy 1, policy_version 91240 (0.0007) [2023-10-10 20:09:05,012][123614] Updated weights for policy 1, policy_version 91250 (0.0007) [2023-10-10 20:09:05,377][123614] Updated weights for policy 1, policy_version 91260 (0.0008) [2023-10-10 20:09:06,515][123582] Updated weights for policy 0, policy_version 91363 (0.0010) [2023-10-10 20:09:06,873][123582] Updated weights for policy 0, policy_version 91373 (0.0008) [2023-10-10 20:09:07,237][123582] Updated weights for policy 0, policy_version 91383 (0.0010) [2023-10-10 20:09:08,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 187039744. Throughput: 0: 1817.9, 1: 1830.8. Samples: 46772734. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:09:08,788][122664] Avg episode reward: [(0, '100.490'), (1, '95.460')] [2023-10-10 20:09:09,037][123614] Updated weights for policy 1, policy_version 91270 (0.0008) [2023-10-10 20:09:09,408][123614] Updated weights for policy 1, policy_version 91280 (0.0010) [2023-10-10 20:09:09,779][123614] Updated weights for policy 1, policy_version 91290 (0.0009) [2023-10-10 20:09:10,994][123582] Updated weights for policy 0, policy_version 91393 (0.0008) [2023-10-10 20:09:11,361][123582] Updated weights for policy 0, policy_version 91403 (0.0011) [2023-10-10 20:09:11,727][123582] Updated weights for policy 0, policy_version 91413 (0.0010) [2023-10-10 20:09:12,107][123582] Updated weights for policy 0, policy_version 91423 (0.0010) [2023-10-10 20:09:13,610][123614] Updated weights for policy 1, policy_version 91300 (0.0008) [2023-10-10 20:09:13,788][122664] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 187105280. Throughput: 0: 1818.1, 1: 1828.8. Samples: 46783310. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:09:13,789][122664] Avg episode reward: [(0, '101.890'), (1, '97.740')] [2023-10-10 20:09:13,992][123614] Updated weights for policy 1, policy_version 91310 (0.0009) [2023-10-10 20:09:14,356][123614] Updated weights for policy 1, policy_version 91320 (0.0009) [2023-10-10 20:09:15,867][123582] Updated weights for policy 0, policy_version 91433 (0.0007) [2023-10-10 20:09:16,246][123582] Updated weights for policy 0, policy_version 91443 (0.0007) [2023-10-10 20:09:16,612][123582] Updated weights for policy 0, policy_version 91453 (0.0007) [2023-10-10 20:09:17,938][123614] Updated weights for policy 1, policy_version 91330 (0.0011) [2023-10-10 20:09:18,300][123614] Updated weights for policy 1, policy_version 91340 (0.0009) [2023-10-10 20:09:18,668][123614] Updated weights for policy 1, policy_version 91350 (0.0007) [2023-10-10 20:09:18,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 187170816. Throughput: 0: 1810.8, 1: 1824.5. Samples: 46805084. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:09:18,788][122664] Avg episode reward: [(0, '102.850'), (1, '95.880')] [2023-10-10 20:09:19,035][123614] Updated weights for policy 1, policy_version 91360 (0.0007) [2023-10-10 20:09:20,258][123582] Updated weights for policy 0, policy_version 91463 (0.0010) [2023-10-10 20:09:20,635][123582] Updated weights for policy 0, policy_version 91473 (0.0010) [2023-10-10 20:09:21,009][123582] Updated weights for policy 0, policy_version 91483 (0.0009) [2023-10-10 20:09:22,768][123614] Updated weights for policy 1, policy_version 91370 (0.0007) [2023-10-10 20:09:23,139][123614] Updated weights for policy 1, policy_version 91380 (0.0009) [2023-10-10 20:09:23,518][123614] Updated weights for policy 1, policy_version 91390 (0.0008) [2023-10-10 20:09:23,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 187269120. Throughput: 0: 1812.8, 1: 1815.0. Samples: 46826240. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:09:23,788][122664] Avg episode reward: [(0, '102.220'), (1, '101.310')] [2023-10-10 20:09:24,694][123582] Updated weights for policy 0, policy_version 91493 (0.0008) [2023-10-10 20:09:25,074][123582] Updated weights for policy 0, policy_version 91503 (0.0008) [2023-10-10 20:09:25,439][123582] Updated weights for policy 0, policy_version 91513 (0.0008) [2023-10-10 20:09:27,166][123614] Updated weights for policy 1, policy_version 91400 (0.0010) [2023-10-10 20:09:27,533][123614] Updated weights for policy 1, policy_version 91410 (0.0009) [2023-10-10 20:09:27,897][123614] Updated weights for policy 1, policy_version 91420 (0.0008) [2023-10-10 20:09:28,788][122664] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 187334656. Throughput: 0: 1810.4, 1: 1819.7. Samples: 46837704. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:09:28,789][122664] Avg episode reward: [(0, '102.140'), (1, '101.050')] [2023-10-10 20:09:29,083][123582] Updated weights for policy 0, policy_version 91523 (0.0009) [2023-10-10 20:09:29,457][123582] Updated weights for policy 0, policy_version 91533 (0.0011) [2023-10-10 20:09:29,830][123582] Updated weights for policy 0, policy_version 91543 (0.0012) [2023-10-10 20:09:31,734][123614] Updated weights for policy 1, policy_version 91430 (0.0009) [2023-10-10 20:09:32,104][123614] Updated weights for policy 1, policy_version 91440 (0.0007) [2023-10-10 20:09:32,469][123614] Updated weights for policy 1, policy_version 91450 (0.0008) [2023-10-10 20:09:33,575][123582] Updated weights for policy 0, policy_version 91553 (0.0010) [2023-10-10 20:09:33,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 187400192. Throughput: 0: 1806.5, 1: 1810.6. Samples: 46858928. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:09:33,789][122664] Avg episode reward: [(0, '101.400'), (1, '101.050')] [2023-10-10 20:09:33,941][123582] Updated weights for policy 0, policy_version 91563 (0.0008) [2023-10-10 20:09:34,309][123582] Updated weights for policy 0, policy_version 91573 (0.0008) [2023-10-10 20:09:34,677][123582] Updated weights for policy 0, policy_version 91583 (0.0009) [2023-10-10 20:09:36,065][123614] Updated weights for policy 1, policy_version 91460 (0.0007) [2023-10-10 20:09:36,432][123614] Updated weights for policy 1, policy_version 91470 (0.0008) [2023-10-10 20:09:36,799][123614] Updated weights for policy 1, policy_version 91480 (0.0008) [2023-10-10 20:09:38,321][123582] Updated weights for policy 0, policy_version 91593 (0.0009) [2023-10-10 20:09:38,691][123582] Updated weights for policy 0, policy_version 91603 (0.0008) [2023-10-10 20:09:38,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 187465728. Throughput: 0: 1820.7, 1: 1813.2. Samples: 46881344. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:09:38,789][122664] Avg episode reward: [(0, '102.500'), (1, '95.880')] [2023-10-10 20:09:39,075][123582] Updated weights for policy 0, policy_version 91613 (0.0009) [2023-10-10 20:09:40,570][123614] Updated weights for policy 1, policy_version 91490 (0.0008) [2023-10-10 20:09:40,939][123614] Updated weights for policy 1, policy_version 91500 (0.0008) [2023-10-10 20:09:41,320][123614] Updated weights for policy 1, policy_version 91510 (0.0009) [2023-10-10 20:09:41,687][123614] Updated weights for policy 1, policy_version 91520 (0.0008) [2023-10-10 20:09:42,750][123582] Updated weights for policy 0, policy_version 91623 (0.0010) [2023-10-10 20:09:43,116][123582] Updated weights for policy 0, policy_version 91633 (0.0010) [2023-10-10 20:09:43,488][123582] Updated weights for policy 0, policy_version 91643 (0.0009) [2023-10-10 20:09:43,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 187564032. Throughput: 0: 1808.8, 1: 1813.5. Samples: 46891808. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:09:43,788][122664] Avg episode reward: [(0, '104.030'), (1, '98.680')] [2023-10-10 20:09:45,510][123614] Updated weights for policy 1, policy_version 91530 (0.0009) [2023-10-10 20:09:45,884][123614] Updated weights for policy 1, policy_version 91540 (0.0008) [2023-10-10 20:09:46,260][123614] Updated weights for policy 1, policy_version 91550 (0.0008) [2023-10-10 20:09:47,100][123582] Updated weights for policy 0, policy_version 91653 (0.0008) [2023-10-10 20:09:47,476][123582] Updated weights for policy 0, policy_version 91663 (0.0008) [2023-10-10 20:09:47,837][123582] Updated weights for policy 0, policy_version 91673 (0.0011) [2023-10-10 20:09:48,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 187629568. Throughput: 0: 1815.7, 1: 1805.2. Samples: 46913620. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:09:48,789][122664] Avg episode reward: [(0, '100.500'), (1, '99.770')] [2023-10-10 20:09:50,071][123614] Updated weights for policy 1, policy_version 91560 (0.0007) [2023-10-10 20:09:50,431][123614] Updated weights for policy 1, policy_version 91570 (0.0008) [2023-10-10 20:09:50,796][123614] Updated weights for policy 1, policy_version 91580 (0.0011) [2023-10-10 20:09:51,463][123582] Updated weights for policy 0, policy_version 91683 (0.0008) [2023-10-10 20:09:51,824][123582] Updated weights for policy 0, policy_version 91693 (0.0008) [2023-10-10 20:09:52,205][123582] Updated weights for policy 0, policy_version 91703 (0.0009) [2023-10-10 20:09:53,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 187695104. Throughput: 0: 1814.3, 1: 1800.8. Samples: 46935410. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:09:53,789][122664] Avg episode reward: [(0, '99.830'), (1, '97.840')] [2023-10-10 20:09:53,799][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000091584_93782016.pth... [2023-10-10 20:09:53,799][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000091712_93913088.pth... [2023-10-10 20:09:53,838][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000089888_92045312.pth [2023-10-10 20:09:53,839][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000090016_92176384.pth [2023-10-10 20:09:54,559][123614] Updated weights for policy 1, policy_version 91590 (0.0009) [2023-10-10 20:09:54,923][123614] Updated weights for policy 1, policy_version 91600 (0.0009) [2023-10-10 20:09:55,294][123614] Updated weights for policy 1, policy_version 91610 (0.0011) [2023-10-10 20:09:55,919][123582] Updated weights for policy 0, policy_version 91713 (0.0007) [2023-10-10 20:09:56,291][123582] Updated weights for policy 0, policy_version 91723 (0.0010) [2023-10-10 20:09:56,672][123582] Updated weights for policy 0, policy_version 91733 (0.0011) [2023-10-10 20:09:57,039][123582] Updated weights for policy 0, policy_version 91743 (0.0009) [2023-10-10 20:09:58,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 187760640. Throughput: 0: 1818.2, 1: 1801.8. Samples: 46946208. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:09:58,789][122664] Avg episode reward: [(0, '102.530'), (1, '98.340')] [2023-10-10 20:09:59,112][123614] Updated weights for policy 1, policy_version 91620 (0.0007) [2023-10-10 20:09:59,487][123614] Updated weights for policy 1, policy_version 91630 (0.0007) [2023-10-10 20:09:59,847][123614] Updated weights for policy 1, policy_version 91640 (0.0009) [2023-10-10 20:10:00,818][123582] Updated weights for policy 0, policy_version 91753 (0.0008) [2023-10-10 20:10:01,202][123582] Updated weights for policy 0, policy_version 91763 (0.0010) [2023-10-10 20:10:01,583][123582] Updated weights for policy 0, policy_version 91773 (0.0010) [2023-10-10 20:10:03,584][123614] Updated weights for policy 1, policy_version 91650 (0.0008) [2023-10-10 20:10:03,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 187826176. Throughput: 0: 1818.4, 1: 1799.8. Samples: 46967906. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:10:03,788][122664] Avg episode reward: [(0, '98.120'), (1, '99.300')] [2023-10-10 20:10:03,959][123614] Updated weights for policy 1, policy_version 91660 (0.0010) [2023-10-10 20:10:04,317][123614] Updated weights for policy 1, policy_version 91670 (0.0009) [2023-10-10 20:10:04,680][123614] Updated weights for policy 1, policy_version 91680 (0.0011) [2023-10-10 20:10:05,127][123582] Updated weights for policy 0, policy_version 91783 (0.0009) [2023-10-10 20:10:05,505][123582] Updated weights for policy 0, policy_version 91793 (0.0008) [2023-10-10 20:10:05,883][123582] Updated weights for policy 0, policy_version 91803 (0.0008) [2023-10-10 20:10:08,477][123614] Updated weights for policy 1, policy_version 91690 (0.0007) [2023-10-10 20:10:08,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 187891712. Throughput: 0: 1820.6, 1: 1819.9. Samples: 46990062. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:10:08,789][122664] Avg episode reward: [(0, '94.950'), (1, '100.880')] [2023-10-10 20:10:08,835][123614] Updated weights for policy 1, policy_version 91700 (0.0007) [2023-10-10 20:10:09,206][123614] Updated weights for policy 1, policy_version 91710 (0.0010) [2023-10-10 20:10:09,500][123582] Updated weights for policy 0, policy_version 91813 (0.0009) [2023-10-10 20:10:09,877][123582] Updated weights for policy 0, policy_version 91823 (0.0007) [2023-10-10 20:10:10,256][123582] Updated weights for policy 0, policy_version 91833 (0.0008) [2023-10-10 20:10:13,061][123614] Updated weights for policy 1, policy_version 91720 (0.0010) [2023-10-10 20:10:13,432][123614] Updated weights for policy 1, policy_version 91730 (0.0009) [2023-10-10 20:10:13,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 187957248. Throughput: 0: 1820.9, 1: 1798.5. Samples: 47000574. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:10:13,788][122664] Avg episode reward: [(0, '99.430'), (1, '106.580')] [2023-10-10 20:10:13,799][123614] Updated weights for policy 1, policy_version 91740 (0.0009) [2023-10-10 20:10:13,966][123582] Updated weights for policy 0, policy_version 91843 (0.0009) [2023-10-10 20:10:14,348][123582] Updated weights for policy 0, policy_version 91853 (0.0010) [2023-10-10 20:10:14,714][123582] Updated weights for policy 0, policy_version 91863 (0.0007) [2023-10-10 20:10:17,389][123614] Updated weights for policy 1, policy_version 91750 (0.0008) [2023-10-10 20:10:17,753][123614] Updated weights for policy 1, policy_version 91760 (0.0008) [2023-10-10 20:10:18,127][123614] Updated weights for policy 1, policy_version 91770 (0.0008) [2023-10-10 20:10:18,524][123582] Updated weights for policy 0, policy_version 91873 (0.0008) [2023-10-10 20:10:18,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 188055552. Throughput: 0: 1820.8, 1: 1815.8. Samples: 47022576. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:10:18,789][122664] Avg episode reward: [(0, '101.480'), (1, '103.700')] [2023-10-10 20:10:18,885][123582] Updated weights for policy 0, policy_version 91883 (0.0008) [2023-10-10 20:10:19,262][123582] Updated weights for policy 0, policy_version 91893 (0.0008) [2023-10-10 20:10:19,636][123582] Updated weights for policy 0, policy_version 91903 (0.0009) [2023-10-10 20:10:21,774][123614] Updated weights for policy 1, policy_version 91780 (0.0008) [2023-10-10 20:10:22,156][123614] Updated weights for policy 1, policy_version 91790 (0.0009) [2023-10-10 20:10:22,519][123614] Updated weights for policy 1, policy_version 91800 (0.0011) [2023-10-10 20:10:23,330][123582] Updated weights for policy 0, policy_version 91913 (0.0008) [2023-10-10 20:10:23,700][123582] Updated weights for policy 0, policy_version 91923 (0.0008) [2023-10-10 20:10:23,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 188121088. Throughput: 0: 1820.1, 1: 1790.1. Samples: 47043806. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:10:23,789][122664] Avg episode reward: [(0, '100.740'), (1, '100.920')] [2023-10-10 20:10:24,081][123582] Updated weights for policy 0, policy_version 91933 (0.0009) [2023-10-10 20:10:26,236][123614] Updated weights for policy 1, policy_version 91810 (0.0008) [2023-10-10 20:10:26,601][123614] Updated weights for policy 1, policy_version 91820 (0.0010) [2023-10-10 20:10:26,974][123614] Updated weights for policy 1, policy_version 91830 (0.0009) [2023-10-10 20:10:27,333][123614] Updated weights for policy 1, policy_version 91840 (0.0010) [2023-10-10 20:10:27,843][123582] Updated weights for policy 0, policy_version 91943 (0.0008) [2023-10-10 20:10:28,227][123582] Updated weights for policy 0, policy_version 91953 (0.0007) [2023-10-10 20:10:28,599][123582] Updated weights for policy 0, policy_version 91963 (0.0008) [2023-10-10 20:10:28,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 188219392. Throughput: 0: 1818.2, 1: 1811.8. Samples: 47055156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:10:28,788][122664] Avg episode reward: [(0, '102.420'), (1, '101.640')] [2023-10-10 20:10:31,070][123614] Updated weights for policy 1, policy_version 91850 (0.0007) [2023-10-10 20:10:31,443][123614] Updated weights for policy 1, policy_version 91860 (0.0007) [2023-10-10 20:10:31,810][123614] Updated weights for policy 1, policy_version 91870 (0.0007) [2023-10-10 20:10:32,255][123582] Updated weights for policy 0, policy_version 91973 (0.0008) [2023-10-10 20:10:32,626][123582] Updated weights for policy 0, policy_version 91983 (0.0008) [2023-10-10 20:10:33,002][123582] Updated weights for policy 0, policy_version 91993 (0.0008) [2023-10-10 20:10:33,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 188284928. Throughput: 0: 1822.1, 1: 1800.6. Samples: 47076640. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:10:33,789][122664] Avg episode reward: [(0, '105.680'), (1, '106.230')] [2023-10-10 20:10:35,566][123614] Updated weights for policy 1, policy_version 91880 (0.0007) [2023-10-10 20:10:35,933][123614] Updated weights for policy 1, policy_version 91890 (0.0007) [2023-10-10 20:10:36,301][123614] Updated weights for policy 1, policy_version 91900 (0.0008) [2023-10-10 20:10:36,478][123582] Updated weights for policy 0, policy_version 92003 (0.0009) [2023-10-10 20:10:36,848][123582] Updated weights for policy 0, policy_version 92013 (0.0007) [2023-10-10 20:10:37,224][123582] Updated weights for policy 0, policy_version 92023 (0.0007) [2023-10-10 20:10:38,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 188350464. Throughput: 0: 1823.5, 1: 1800.0. Samples: 47098466. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:10:38,788][122664] Avg episode reward: [(0, '109.310'), (1, '103.240')] [2023-10-10 20:10:39,983][123614] Updated weights for policy 1, policy_version 91910 (0.0008) [2023-10-10 20:10:40,357][123614] Updated weights for policy 1, policy_version 91920 (0.0009) [2023-10-10 20:10:40,733][123614] Updated weights for policy 1, policy_version 91930 (0.0009) [2023-10-10 20:10:40,955][123582] Updated weights for policy 0, policy_version 92033 (0.0008) [2023-10-10 20:10:41,333][123582] Updated weights for policy 0, policy_version 92043 (0.0007) [2023-10-10 20:10:41,712][123582] Updated weights for policy 0, policy_version 92053 (0.0007) [2023-10-10 20:10:42,084][123582] Updated weights for policy 0, policy_version 92063 (0.0007) [2023-10-10 20:10:43,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 188416000. Throughput: 0: 1825.6, 1: 1796.5. Samples: 47109204. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:10:43,789][122664] Avg episode reward: [(0, '107.570'), (1, '98.290')] [2023-10-10 20:10:44,534][123614] Updated weights for policy 1, policy_version 91940 (0.0008) [2023-10-10 20:10:44,925][123614] Updated weights for policy 1, policy_version 91950 (0.0007) [2023-10-10 20:10:45,283][123614] Updated weights for policy 1, policy_version 91960 (0.0007) [2023-10-10 20:10:45,646][123582] Updated weights for policy 0, policy_version 92073 (0.0008) [2023-10-10 20:10:46,019][123582] Updated weights for policy 0, policy_version 92083 (0.0009) [2023-10-10 20:10:46,387][123582] Updated weights for policy 0, policy_version 92093 (0.0007) [2023-10-10 20:10:48,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 188481536. Throughput: 0: 1825.9, 1: 1799.5. Samples: 47131050. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:10:48,789][122664] Avg episode reward: [(0, '103.570'), (1, '100.950')] [2023-10-10 20:10:48,935][123614] Updated weights for policy 1, policy_version 91970 (0.0008) [2023-10-10 20:10:49,300][123614] Updated weights for policy 1, policy_version 91980 (0.0010) [2023-10-10 20:10:49,664][123614] Updated weights for policy 1, policy_version 91990 (0.0009) [2023-10-10 20:10:50,038][123614] Updated weights for policy 1, policy_version 92000 (0.0009) [2023-10-10 20:10:50,228][123582] Updated weights for policy 0, policy_version 92103 (0.0008) [2023-10-10 20:10:50,595][123582] Updated weights for policy 0, policy_version 92113 (0.0008) [2023-10-10 20:10:50,971][123582] Updated weights for policy 0, policy_version 92123 (0.0007) [2023-10-10 20:10:53,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 188547072. Throughput: 0: 1818.4, 1: 1809.4. Samples: 47153310. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:10:53,789][122664] Avg episode reward: [(0, '100.730'), (1, '101.540')] [2023-10-10 20:10:53,862][123614] Updated weights for policy 1, policy_version 92010 (0.0008) [2023-10-10 20:10:54,237][123614] Updated weights for policy 1, policy_version 92020 (0.0009) [2023-10-10 20:10:54,611][123614] Updated weights for policy 1, policy_version 92030 (0.0007) [2023-10-10 20:10:54,773][123582] Updated weights for policy 0, policy_version 92133 (0.0008) [2023-10-10 20:10:55,149][123582] Updated weights for policy 0, policy_version 92143 (0.0009) [2023-10-10 20:10:55,521][123582] Updated weights for policy 0, policy_version 92153 (0.0008) [2023-10-10 20:10:58,225][123614] Updated weights for policy 1, policy_version 92040 (0.0010) [2023-10-10 20:10:58,591][123614] Updated weights for policy 1, policy_version 92050 (0.0009) [2023-10-10 20:10:58,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 188612608. Throughput: 0: 1818.3, 1: 1803.9. Samples: 47163572. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:10:58,789][122664] Avg episode reward: [(0, '105.220'), (1, '100.400')] [2023-10-10 20:10:58,960][123614] Updated weights for policy 1, policy_version 92060 (0.0008) [2023-10-10 20:10:59,009][123582] Updated weights for policy 0, policy_version 92163 (0.0007) [2023-10-10 20:10:59,375][123582] Updated weights for policy 0, policy_version 92173 (0.0009) [2023-10-10 20:10:59,739][123582] Updated weights for policy 0, policy_version 92183 (0.0012) [2023-10-10 20:11:02,644][123614] Updated weights for policy 1, policy_version 92070 (0.0009) [2023-10-10 20:11:03,009][123614] Updated weights for policy 1, policy_version 92080 (0.0009) [2023-10-10 20:11:03,384][123614] Updated weights for policy 1, policy_version 92090 (0.0008) [2023-10-10 20:11:03,551][123582] Updated weights for policy 0, policy_version 92193 (0.0009) [2023-10-10 20:11:03,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 188710912. Throughput: 0: 1821.5, 1: 1812.1. Samples: 47186088. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:11:03,789][122664] Avg episode reward: [(0, '102.940'), (1, '99.620')] [2023-10-10 20:11:03,919][123582] Updated weights for policy 0, policy_version 92203 (0.0009) [2023-10-10 20:11:04,283][123582] Updated weights for policy 0, policy_version 92213 (0.0009) [2023-10-10 20:11:04,657][123582] Updated weights for policy 0, policy_version 92223 (0.0008) [2023-10-10 20:11:07,133][123614] Updated weights for policy 1, policy_version 92100 (0.0010) [2023-10-10 20:11:07,504][123614] Updated weights for policy 1, policy_version 92110 (0.0008) [2023-10-10 20:11:07,868][123614] Updated weights for policy 1, policy_version 92120 (0.0009) [2023-10-10 20:11:08,289][123582] Updated weights for policy 0, policy_version 92233 (0.0009) [2023-10-10 20:11:08,663][123582] Updated weights for policy 0, policy_version 92243 (0.0007) [2023-10-10 20:11:08,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 188776448. Throughput: 0: 1820.9, 1: 1808.4. Samples: 47207126. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-10 20:11:08,789][122664] Avg episode reward: [(0, '94.630'), (1, '105.960')] [2023-10-10 20:11:09,038][123582] Updated weights for policy 0, policy_version 92253 (0.0007) [2023-10-10 20:11:11,639][123614] Updated weights for policy 1, policy_version 92130 (0.0008) [2023-10-10 20:11:12,009][123614] Updated weights for policy 1, policy_version 92140 (0.0009) [2023-10-10 20:11:12,385][123614] Updated weights for policy 1, policy_version 92150 (0.0008) [2023-10-10 20:11:12,681][123582] Updated weights for policy 0, policy_version 92263 (0.0009) [2023-10-10 20:11:12,750][123614] Updated weights for policy 1, policy_version 92160 (0.0007) [2023-10-10 20:11:13,062][123582] Updated weights for policy 0, policy_version 92273 (0.0007) [2023-10-10 20:11:13,433][123582] Updated weights for policy 0, policy_version 92283 (0.0007) [2023-10-10 20:11:13,788][122664] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 188874752. Throughput: 0: 1825.2, 1: 1811.9. Samples: 47218828. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-10 20:11:13,789][122664] Avg episode reward: [(0, '94.710'), (1, '105.010')] [2023-10-10 20:11:16,355][123614] Updated weights for policy 1, policy_version 92170 (0.0007) [2023-10-10 20:11:16,714][123614] Updated weights for policy 1, policy_version 92180 (0.0008) [2023-10-10 20:11:17,044][123582] Updated weights for policy 0, policy_version 92293 (0.0009) [2023-10-10 20:11:17,084][123614] Updated weights for policy 1, policy_version 92190 (0.0008) [2023-10-10 20:11:17,427][123582] Updated weights for policy 0, policy_version 92303 (0.0008) [2023-10-10 20:11:17,801][123582] Updated weights for policy 0, policy_version 92313 (0.0008) [2023-10-10 20:11:18,788][122664] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 188940288. Throughput: 0: 1818.4, 1: 1802.3. Samples: 47239570. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-10 20:11:18,788][122664] Avg episode reward: [(0, '97.980'), (1, '102.310')] [2023-10-10 20:11:20,759][123614] Updated weights for policy 1, policy_version 92200 (0.0009) [2023-10-10 20:11:21,140][123614] Updated weights for policy 1, policy_version 92210 (0.0009) [2023-10-10 20:11:21,412][123582] Updated weights for policy 0, policy_version 92323 (0.0007) [2023-10-10 20:11:21,511][123614] Updated weights for policy 1, policy_version 92220 (0.0007) [2023-10-10 20:11:21,784][123582] Updated weights for policy 0, policy_version 92333 (0.0008) [2023-10-10 20:11:22,153][123582] Updated weights for policy 0, policy_version 92343 (0.0010) [2023-10-10 20:11:23,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 189005824. Throughput: 0: 1821.1, 1: 1806.3. Samples: 47261702. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-10 20:11:23,789][122664] Avg episode reward: [(0, '94.380'), (1, '100.930')] [2023-10-10 20:11:25,176][123614] Updated weights for policy 1, policy_version 92230 (0.0008) [2023-10-10 20:11:25,541][123614] Updated weights for policy 1, policy_version 92240 (0.0008) [2023-10-10 20:11:25,905][123614] Updated weights for policy 1, policy_version 92250 (0.0007) [2023-10-10 20:11:25,925][123582] Updated weights for policy 0, policy_version 92353 (0.0010) [2023-10-10 20:11:26,300][123582] Updated weights for policy 0, policy_version 92363 (0.0008) [2023-10-10 20:11:26,670][123582] Updated weights for policy 0, policy_version 92373 (0.0008) [2023-10-10 20:11:27,043][123582] Updated weights for policy 0, policy_version 92383 (0.0007) [2023-10-10 20:11:28,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 189071360. Throughput: 0: 1816.8, 1: 1808.6. Samples: 47272346. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-10 20:11:28,788][122664] Avg episode reward: [(0, '96.750'), (1, '98.140')] [2023-10-10 20:11:29,659][123614] Updated weights for policy 1, policy_version 92260 (0.0007) [2023-10-10 20:11:30,030][123614] Updated weights for policy 1, policy_version 92270 (0.0008) [2023-10-10 20:11:30,389][123614] Updated weights for policy 1, policy_version 92280 (0.0008) [2023-10-10 20:11:30,612][123582] Updated weights for policy 0, policy_version 92393 (0.0007) [2023-10-10 20:11:30,981][123582] Updated weights for policy 0, policy_version 92403 (0.0008) [2023-10-10 20:11:31,355][123582] Updated weights for policy 0, policy_version 92413 (0.0008) [2023-10-10 20:11:33,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 189136896. Throughput: 0: 1820.1, 1: 1812.5. Samples: 47294516. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-10 20:11:33,789][122664] Avg episode reward: [(0, '96.800'), (1, '97.260')] [2023-10-10 20:11:34,116][123614] Updated weights for policy 1, policy_version 92290 (0.0008) [2023-10-10 20:11:34,481][123614] Updated weights for policy 1, policy_version 92300 (0.0008) [2023-10-10 20:11:34,839][123582] Updated weights for policy 0, policy_version 92423 (0.0008) [2023-10-10 20:11:34,849][123614] Updated weights for policy 1, policy_version 92310 (0.0010) [2023-10-10 20:11:35,216][123614] Updated weights for policy 1, policy_version 92320 (0.0008) [2023-10-10 20:11:35,220][123582] Updated weights for policy 0, policy_version 92433 (0.0009) [2023-10-10 20:11:35,589][123582] Updated weights for policy 0, policy_version 92443 (0.0009) [2023-10-10 20:11:38,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 189202432. Throughput: 0: 1824.3, 1: 1817.5. Samples: 47317192. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-10 20:11:38,789][122664] Avg episode reward: [(0, '92.620'), (1, '102.040')] [2023-10-10 20:11:38,959][123614] Updated weights for policy 1, policy_version 92330 (0.0007) [2023-10-10 20:11:39,319][123614] Updated weights for policy 1, policy_version 92340 (0.0008) [2023-10-10 20:11:39,414][123582] Updated weights for policy 0, policy_version 92453 (0.0008) [2023-10-10 20:11:39,687][123614] Updated weights for policy 1, policy_version 92350 (0.0007) [2023-10-10 20:11:39,790][123582] Updated weights for policy 0, policy_version 92463 (0.0008) [2023-10-10 20:11:40,159][123582] Updated weights for policy 0, policy_version 92473 (0.0010) [2023-10-10 20:11:43,204][123614] Updated weights for policy 1, policy_version 92360 (0.0008) [2023-10-10 20:11:43,576][123614] Updated weights for policy 1, policy_version 92370 (0.0007) [2023-10-10 20:11:43,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 189267968. Throughput: 0: 1824.5, 1: 1813.1. Samples: 47327266. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-10 20:11:43,788][122664] Avg episode reward: [(0, '92.130'), (1, '97.140')] [2023-10-10 20:11:43,936][123614] Updated weights for policy 1, policy_version 92380 (0.0007) [2023-10-10 20:11:43,958][123582] Updated weights for policy 0, policy_version 92483 (0.0007) [2023-10-10 20:11:44,330][123582] Updated weights for policy 0, policy_version 92493 (0.0008) [2023-10-10 20:11:44,707][123582] Updated weights for policy 0, policy_version 92503 (0.0009) [2023-10-10 20:11:47,445][123614] Updated weights for policy 1, policy_version 92390 (0.0009) [2023-10-10 20:11:47,812][123614] Updated weights for policy 1, policy_version 92400 (0.0007) [2023-10-10 20:11:48,180][123614] Updated weights for policy 1, policy_version 92410 (0.0008) [2023-10-10 20:11:48,507][123582] Updated weights for policy 0, policy_version 92513 (0.0007) [2023-10-10 20:11:48,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 189366272. Throughput: 0: 1812.9, 1: 1818.9. Samples: 47349516. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-10 20:11:48,789][122664] Avg episode reward: [(0, '90.740'), (1, '99.930')] [2023-10-10 20:11:48,880][123582] Updated weights for policy 0, policy_version 92523 (0.0008) [2023-10-10 20:11:49,251][123582] Updated weights for policy 0, policy_version 92533 (0.0008) [2023-10-10 20:11:49,623][123582] Updated weights for policy 0, policy_version 92543 (0.0007) [2023-10-10 20:11:51,927][123614] Updated weights for policy 1, policy_version 92420 (0.0007) [2023-10-10 20:11:52,294][123614] Updated weights for policy 1, policy_version 92430 (0.0010) [2023-10-10 20:11:52,661][123614] Updated weights for policy 1, policy_version 92440 (0.0009) [2023-10-10 20:11:53,450][123582] Updated weights for policy 0, policy_version 92553 (0.0009) [2023-10-10 20:11:53,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 189431808. Throughput: 0: 1816.2, 1: 1828.8. Samples: 47371150. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-10 20:11:53,788][122664] Avg episode reward: [(0, '86.830'), (1, '98.510')] [2023-10-10 20:11:53,796][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000092448_94666752.pth... [2023-10-10 20:11:53,824][123582] Updated weights for policy 0, policy_version 92563 (0.0009) [2023-10-10 20:11:53,825][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000090752_92930048.pth [2023-10-10 20:11:54,191][123582] Updated weights for policy 0, policy_version 92573 (0.0008) [2023-10-10 20:11:54,299][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000092576_94797824.pth... [2023-10-10 20:11:54,337][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000090848_93028352.pth [2023-10-10 20:11:56,274][123614] Updated weights for policy 1, policy_version 92450 (0.0010) [2023-10-10 20:11:56,645][123614] Updated weights for policy 1, policy_version 92460 (0.0007) [2023-10-10 20:11:57,014][123614] Updated weights for policy 1, policy_version 92470 (0.0008) [2023-10-10 20:11:57,375][123614] Updated weights for policy 1, policy_version 92480 (0.0008) [2023-10-10 20:11:57,839][123582] Updated weights for policy 0, policy_version 92583 (0.0008) [2023-10-10 20:11:58,227][123582] Updated weights for policy 0, policy_version 92593 (0.0009) [2023-10-10 20:11:58,603][123582] Updated weights for policy 0, policy_version 92603 (0.0009) [2023-10-10 20:11:58,788][122664] Fps is (10 sec: 16384.2, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 189530112. Throughput: 0: 1808.6, 1: 1824.8. Samples: 47382330. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-10 20:11:58,788][122664] Avg episode reward: [(0, '85.820'), (1, '98.910')] [2023-10-10 20:12:01,101][123614] Updated weights for policy 1, policy_version 92490 (0.0010) [2023-10-10 20:12:01,477][123614] Updated weights for policy 1, policy_version 92500 (0.0010) [2023-10-10 20:12:01,855][123614] Updated weights for policy 1, policy_version 92510 (0.0009) [2023-10-10 20:12:02,378][123582] Updated weights for policy 0, policy_version 92613 (0.0009) [2023-10-10 20:12:02,748][123582] Updated weights for policy 0, policy_version 92623 (0.0008) [2023-10-10 20:12:03,122][123582] Updated weights for policy 0, policy_version 92633 (0.0007) [2023-10-10 20:12:03,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 189595648. Throughput: 0: 1820.6, 1: 1833.3. Samples: 47403996. Policy #0 lag: (min: 29.0, avg: 29.6, max: 43.0) [2023-10-10 20:12:03,789][122664] Avg episode reward: [(0, '86.430'), (1, '94.820')] [2023-10-10 20:12:05,494][123614] Updated weights for policy 1, policy_version 92520 (0.0009) [2023-10-10 20:12:05,857][123614] Updated weights for policy 1, policy_version 92530 (0.0009) [2023-10-10 20:12:06,228][123614] Updated weights for policy 1, policy_version 92540 (0.0009) [2023-10-10 20:12:06,732][123582] Updated weights for policy 0, policy_version 92643 (0.0007) [2023-10-10 20:12:07,103][123582] Updated weights for policy 0, policy_version 92653 (0.0008) [2023-10-10 20:12:07,474][123582] Updated weights for policy 0, policy_version 92663 (0.0008) [2023-10-10 20:12:08,788][122664] Fps is (10 sec: 13106.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 189661184. Throughput: 0: 1811.8, 1: 1831.9. Samples: 47425666. Policy #0 lag: (min: 29.0, avg: 29.6, max: 43.0) [2023-10-10 20:12:08,789][122664] Avg episode reward: [(0, '87.360'), (1, '94.500')] [2023-10-10 20:12:10,056][123614] Updated weights for policy 1, policy_version 92550 (0.0009) [2023-10-10 20:12:10,431][123614] Updated weights for policy 1, policy_version 92560 (0.0008) [2023-10-10 20:12:10,797][123614] Updated weights for policy 1, policy_version 92570 (0.0007) [2023-10-10 20:12:11,132][123582] Updated weights for policy 0, policy_version 92673 (0.0008) [2023-10-10 20:12:11,499][123582] Updated weights for policy 0, policy_version 92683 (0.0010) [2023-10-10 20:12:11,867][123582] Updated weights for policy 0, policy_version 92693 (0.0012) [2023-10-10 20:12:12,237][123582] Updated weights for policy 0, policy_version 92703 (0.0011) [2023-10-10 20:12:13,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 189726720. Throughput: 0: 1822.5, 1: 1832.3. Samples: 47436808. Policy #0 lag: (min: 29.0, avg: 29.6, max: 43.0) [2023-10-10 20:12:13,788][122664] Avg episode reward: [(0, '81.760'), (1, '92.370')] [2023-10-10 20:12:14,454][123614] Updated weights for policy 1, policy_version 92580 (0.0007) [2023-10-10 20:12:14,830][123614] Updated weights for policy 1, policy_version 92590 (0.0008) [2023-10-10 20:12:15,197][123614] Updated weights for policy 1, policy_version 92600 (0.0010) [2023-10-10 20:12:15,917][123582] Updated weights for policy 0, policy_version 92713 (0.0009) [2023-10-10 20:12:16,295][123582] Updated weights for policy 0, policy_version 92723 (0.0008) [2023-10-10 20:12:16,671][123582] Updated weights for policy 0, policy_version 92733 (0.0008) [2023-10-10 20:12:18,788][122664] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 189792256. Throughput: 0: 1809.9, 1: 1833.7. Samples: 47458476. Policy #0 lag: (min: 29.0, avg: 29.6, max: 43.0) [2023-10-10 20:12:18,788][122664] Avg episode reward: [(0, '81.200'), (1, '90.950')] [2023-10-10 20:12:18,882][123614] Updated weights for policy 1, policy_version 92610 (0.0011) [2023-10-10 20:12:19,246][123614] Updated weights for policy 1, policy_version 92620 (0.0009) [2023-10-10 20:12:19,621][123614] Updated weights for policy 1, policy_version 92630 (0.0008) [2023-10-10 20:12:19,992][123614] Updated weights for policy 1, policy_version 92640 (0.0008) [2023-10-10 20:12:20,376][123582] Updated weights for policy 0, policy_version 92743 (0.0008) [2023-10-10 20:12:20,743][123582] Updated weights for policy 0, policy_version 92753 (0.0008) [2023-10-10 20:12:21,110][123582] Updated weights for policy 0, policy_version 92763 (0.0009) [2023-10-10 20:12:23,621][123614] Updated weights for policy 1, policy_version 92650 (0.0007) [2023-10-10 20:12:23,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 189857792. Throughput: 0: 1808.5, 1: 1830.0. Samples: 47480922. Policy #0 lag: (min: 29.0, avg: 29.6, max: 43.0) [2023-10-10 20:12:23,788][122664] Avg episode reward: [(0, '84.170'), (1, '92.800')] [2023-10-10 20:12:23,995][123614] Updated weights for policy 1, policy_version 92660 (0.0012) [2023-10-10 20:12:24,367][123614] Updated weights for policy 1, policy_version 92670 (0.0011) [2023-10-10 20:12:24,845][123582] Updated weights for policy 0, policy_version 92773 (0.0009) [2023-10-10 20:12:25,226][123582] Updated weights for policy 0, policy_version 92783 (0.0007) [2023-10-10 20:12:25,592][123582] Updated weights for policy 0, policy_version 92793 (0.0007) [2023-10-10 20:12:28,138][123614] Updated weights for policy 1, policy_version 92680 (0.0009) [2023-10-10 20:12:28,516][123614] Updated weights for policy 1, policy_version 92690 (0.0008) [2023-10-10 20:12:28,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 189923328. Throughput: 0: 1810.9, 1: 1837.1. Samples: 47491424. Policy #0 lag: (min: 29.0, avg: 29.6, max: 43.0) [2023-10-10 20:12:28,788][122664] Avg episode reward: [(0, '84.160'), (1, '89.350')] [2023-10-10 20:12:28,886][123614] Updated weights for policy 1, policy_version 92700 (0.0008) [2023-10-10 20:12:29,102][123582] Updated weights for policy 0, policy_version 92803 (0.0007) [2023-10-10 20:12:29,466][123582] Updated weights for policy 0, policy_version 92813 (0.0007) [2023-10-10 20:12:29,845][123582] Updated weights for policy 0, policy_version 92823 (0.0010) [2023-10-10 20:12:32,655][123614] Updated weights for policy 1, policy_version 92710 (0.0008) [2023-10-10 20:12:33,024][123614] Updated weights for policy 1, policy_version 92720 (0.0008) [2023-10-10 20:12:33,387][123614] Updated weights for policy 1, policy_version 92730 (0.0009) [2023-10-10 20:12:33,535][123582] Updated weights for policy 0, policy_version 92833 (0.0010) [2023-10-10 20:12:33,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 190021632. Throughput: 0: 1818.1, 1: 1829.3. Samples: 47513650. Policy #0 lag: (min: 29.0, avg: 29.6, max: 43.0) [2023-10-10 20:12:33,788][122664] Avg episode reward: [(0, '78.670'), (1, '86.320')] [2023-10-10 20:12:33,908][123582] Updated weights for policy 0, policy_version 92843 (0.0011) [2023-10-10 20:12:34,285][123582] Updated weights for policy 0, policy_version 92853 (0.0011) [2023-10-10 20:12:34,646][123582] Updated weights for policy 0, policy_version 92863 (0.0009) [2023-10-10 20:12:37,049][123614] Updated weights for policy 1, policy_version 92740 (0.0011) [2023-10-10 20:12:37,423][123614] Updated weights for policy 1, policy_version 92750 (0.0009) [2023-10-10 20:12:37,784][123614] Updated weights for policy 1, policy_version 92760 (0.0008) [2023-10-10 20:12:38,281][123582] Updated weights for policy 0, policy_version 92873 (0.0008) [2023-10-10 20:12:38,652][123582] Updated weights for policy 0, policy_version 92883 (0.0010) [2023-10-10 20:12:38,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 190087168. Throughput: 0: 1823.5, 1: 1819.0. Samples: 47535064. Policy #0 lag: (min: 29.0, avg: 29.6, max: 43.0) [2023-10-10 20:12:38,789][122664] Avg episode reward: [(0, '75.790'), (1, '84.900')] [2023-10-10 20:12:39,016][123582] Updated weights for policy 0, policy_version 92893 (0.0007) [2023-10-10 20:12:41,505][123614] Updated weights for policy 1, policy_version 92770 (0.0009) [2023-10-10 20:12:41,878][123614] Updated weights for policy 1, policy_version 92780 (0.0008) [2023-10-10 20:12:42,246][123614] Updated weights for policy 1, policy_version 92790 (0.0008) [2023-10-10 20:12:42,608][123614] Updated weights for policy 1, policy_version 92800 (0.0007) [2023-10-10 20:12:42,848][123582] Updated weights for policy 0, policy_version 92903 (0.0008) [2023-10-10 20:12:43,204][123582] Updated weights for policy 0, policy_version 92913 (0.0011) [2023-10-10 20:12:43,573][123582] Updated weights for policy 0, policy_version 92923 (0.0010) [2023-10-10 20:12:43,788][122664] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 190185472. Throughput: 0: 1824.3, 1: 1822.5. Samples: 47546440. Policy #0 lag: (min: 29.0, avg: 29.6, max: 43.0) [2023-10-10 20:12:43,789][122664] Avg episode reward: [(0, '76.400'), (1, '90.130')] [2023-10-10 20:12:46,285][123614] Updated weights for policy 1, policy_version 92810 (0.0009) [2023-10-10 20:12:46,661][123614] Updated weights for policy 1, policy_version 92820 (0.0009) [2023-10-10 20:12:47,025][123614] Updated weights for policy 1, policy_version 92830 (0.0008) [2023-10-10 20:12:47,292][123582] Updated weights for policy 0, policy_version 92933 (0.0010) [2023-10-10 20:12:47,655][123582] Updated weights for policy 0, policy_version 92943 (0.0010) [2023-10-10 20:12:48,029][123582] Updated weights for policy 0, policy_version 92953 (0.0009) [2023-10-10 20:12:48,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 190251008. Throughput: 0: 1819.3, 1: 1818.9. Samples: 47567718. Policy #0 lag: (min: 29.0, avg: 29.6, max: 43.0) [2023-10-10 20:12:48,789][122664] Avg episode reward: [(0, '76.840'), (1, '90.890')] [2023-10-10 20:12:50,881][123614] Updated weights for policy 1, policy_version 92840 (0.0008) [2023-10-10 20:12:51,252][123614] Updated weights for policy 1, policy_version 92850 (0.0007) [2023-10-10 20:12:51,623][123614] Updated weights for policy 1, policy_version 92860 (0.0008) [2023-10-10 20:12:51,738][123582] Updated weights for policy 0, policy_version 92963 (0.0009) [2023-10-10 20:12:52,121][123582] Updated weights for policy 0, policy_version 92973 (0.0009) [2023-10-10 20:12:52,493][123582] Updated weights for policy 0, policy_version 92983 (0.0008) [2023-10-10 20:12:53,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 190316544. Throughput: 0: 1819.2, 1: 1820.1. Samples: 47589434. Policy #0 lag: (min: 29.0, avg: 29.6, max: 43.0) [2023-10-10 20:12:53,788][122664] Avg episode reward: [(0, '75.070'), (1, '99.530')] [2023-10-10 20:12:55,336][123614] Updated weights for policy 1, policy_version 92870 (0.0010) [2023-10-10 20:12:55,701][123614] Updated weights for policy 1, policy_version 92880 (0.0010) [2023-10-10 20:12:56,069][123614] Updated weights for policy 1, policy_version 92890 (0.0009) [2023-10-10 20:12:56,188][123582] Updated weights for policy 0, policy_version 92993 (0.0008) [2023-10-10 20:12:56,562][123582] Updated weights for policy 0, policy_version 93003 (0.0007) [2023-10-10 20:12:56,942][123582] Updated weights for policy 0, policy_version 93013 (0.0008) [2023-10-10 20:12:57,309][123582] Updated weights for policy 0, policy_version 93023 (0.0007) [2023-10-10 20:12:58,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 190382080. Throughput: 0: 1813.7, 1: 1816.2. Samples: 47600154. Policy #0 lag: (min: 29.0, avg: 29.6, max: 43.0) [2023-10-10 20:12:58,789][122664] Avg episode reward: [(0, '78.020'), (1, '96.960')] [2023-10-10 20:12:59,896][123614] Updated weights for policy 1, policy_version 92900 (0.0009) [2023-10-10 20:13:00,265][123614] Updated weights for policy 1, policy_version 92910 (0.0007) [2023-10-10 20:13:00,642][123614] Updated weights for policy 1, policy_version 92920 (0.0008) [2023-10-10 20:13:01,050][123582] Updated weights for policy 0, policy_version 93033 (0.0009) [2023-10-10 20:13:01,421][123582] Updated weights for policy 0, policy_version 93043 (0.0011) [2023-10-10 20:13:01,805][123582] Updated weights for policy 0, policy_version 93053 (0.0010) [2023-10-10 20:13:03,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 190447616. Throughput: 0: 1814.9, 1: 1809.9. Samples: 47621592. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 20:13:03,789][122664] Avg episode reward: [(0, '77.660'), (1, '93.110')] [2023-10-10 20:13:04,272][123614] Updated weights for policy 1, policy_version 92930 (0.0008) [2023-10-10 20:13:04,662][123614] Updated weights for policy 1, policy_version 92940 (0.0007) [2023-10-10 20:13:05,036][123614] Updated weights for policy 1, policy_version 92950 (0.0009) [2023-10-10 20:13:05,399][123614] Updated weights for policy 1, policy_version 92960 (0.0008) [2023-10-10 20:13:05,554][123582] Updated weights for policy 0, policy_version 93063 (0.0008) [2023-10-10 20:13:05,938][123582] Updated weights for policy 0, policy_version 93073 (0.0008) [2023-10-10 20:13:06,304][123582] Updated weights for policy 0, policy_version 93083 (0.0009) [2023-10-10 20:13:08,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 190513152. Throughput: 0: 1811.6, 1: 1813.2. Samples: 47644038. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 20:13:08,789][122664] Avg episode reward: [(0, '81.330'), (1, '91.820')] [2023-10-10 20:13:08,892][123614] Updated weights for policy 1, policy_version 92970 (0.0007) [2023-10-10 20:13:09,253][123614] Updated weights for policy 1, policy_version 92980 (0.0008) [2023-10-10 20:13:09,620][123614] Updated weights for policy 1, policy_version 92990 (0.0010) [2023-10-10 20:13:10,015][123582] Updated weights for policy 0, policy_version 93093 (0.0011) [2023-10-10 20:13:10,374][123582] Updated weights for policy 0, policy_version 93103 (0.0009) [2023-10-10 20:13:10,739][123582] Updated weights for policy 0, policy_version 93113 (0.0009) [2023-10-10 20:13:13,143][123614] Updated weights for policy 1, policy_version 93000 (0.0010) [2023-10-10 20:13:13,508][123614] Updated weights for policy 1, policy_version 93010 (0.0008) [2023-10-10 20:13:13,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 190578688. Throughput: 0: 1809.6, 1: 1805.7. Samples: 47654112. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 20:13:13,789][122664] Avg episode reward: [(0, '80.950'), (1, '90.960')] [2023-10-10 20:13:13,866][123614] Updated weights for policy 1, policy_version 93020 (0.0007) [2023-10-10 20:13:14,470][123582] Updated weights for policy 0, policy_version 93123 (0.0010) [2023-10-10 20:13:14,845][123582] Updated weights for policy 0, policy_version 93133 (0.0009) [2023-10-10 20:13:15,222][123582] Updated weights for policy 0, policy_version 93143 (0.0009) [2023-10-10 20:13:17,611][123614] Updated weights for policy 1, policy_version 93030 (0.0008) [2023-10-10 20:13:17,980][123614] Updated weights for policy 1, policy_version 93040 (0.0008) [2023-10-10 20:13:18,344][123614] Updated weights for policy 1, policy_version 93050 (0.0008) [2023-10-10 20:13:18,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 190676992. Throughput: 0: 1807.4, 1: 1808.7. Samples: 47676374. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 20:13:18,789][122664] Avg episode reward: [(0, '82.640'), (1, '87.210')] [2023-10-10 20:13:18,918][123582] Updated weights for policy 0, policy_version 93153 (0.0010) [2023-10-10 20:13:19,290][123582] Updated weights for policy 0, policy_version 93163 (0.0008) [2023-10-10 20:13:19,661][123582] Updated weights for policy 0, policy_version 93173 (0.0010) [2023-10-10 20:13:20,035][123582] Updated weights for policy 0, policy_version 93183 (0.0009) [2023-10-10 20:13:22,046][123614] Updated weights for policy 1, policy_version 93060 (0.0009) [2023-10-10 20:13:22,424][123614] Updated weights for policy 1, policy_version 93070 (0.0008) [2023-10-10 20:13:22,796][123614] Updated weights for policy 1, policy_version 93080 (0.0008) [2023-10-10 20:13:23,708][123582] Updated weights for policy 0, policy_version 93193 (0.0009) [2023-10-10 20:13:23,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 190742528. Throughput: 0: 1811.8, 1: 1808.9. Samples: 47697994. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 20:13:23,789][122664] Avg episode reward: [(0, '83.440'), (1, '86.790')] [2023-10-10 20:13:24,084][123582] Updated weights for policy 0, policy_version 93203 (0.0010) [2023-10-10 20:13:24,459][123582] Updated weights for policy 0, policy_version 93213 (0.0008) [2023-10-10 20:13:26,420][123614] Updated weights for policy 1, policy_version 93090 (0.0007) [2023-10-10 20:13:26,800][123614] Updated weights for policy 1, policy_version 93100 (0.0008) [2023-10-10 20:13:27,166][123614] Updated weights for policy 1, policy_version 93110 (0.0010) [2023-10-10 20:13:27,539][123614] Updated weights for policy 1, policy_version 93120 (0.0010) [2023-10-10 20:13:28,239][123582] Updated weights for policy 0, policy_version 93223 (0.0008) [2023-10-10 20:13:28,608][123582] Updated weights for policy 0, policy_version 93233 (0.0007) [2023-10-10 20:13:28,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 190808064. Throughput: 0: 1805.9, 1: 1808.4. Samples: 47709086. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 20:13:28,789][122664] Avg episode reward: [(0, '84.700'), (1, '80.210')] [2023-10-10 20:13:28,972][123582] Updated weights for policy 0, policy_version 93243 (0.0007) [2023-10-10 20:13:31,299][123614] Updated weights for policy 1, policy_version 93130 (0.0010) [2023-10-10 20:13:31,660][123614] Updated weights for policy 1, policy_version 93140 (0.0009) [2023-10-10 20:13:32,030][123614] Updated weights for policy 1, policy_version 93150 (0.0008) [2023-10-10 20:13:32,686][123582] Updated weights for policy 0, policy_version 93253 (0.0009) [2023-10-10 20:13:33,052][123582] Updated weights for policy 0, policy_version 93263 (0.0011) [2023-10-10 20:13:33,417][123582] Updated weights for policy 0, policy_version 93273 (0.0010) [2023-10-10 20:13:33,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 190906368. Throughput: 0: 1815.8, 1: 1805.6. Samples: 47730680. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 20:13:33,789][122664] Avg episode reward: [(0, '83.110'), (1, '79.390')] [2023-10-10 20:13:35,624][123614] Updated weights for policy 1, policy_version 93160 (0.0009) [2023-10-10 20:13:36,000][123614] Updated weights for policy 1, policy_version 93170 (0.0008) [2023-10-10 20:13:36,367][123614] Updated weights for policy 1, policy_version 93180 (0.0007) [2023-10-10 20:13:37,158][123582] Updated weights for policy 0, policy_version 93283 (0.0010) [2023-10-10 20:13:37,529][123582] Updated weights for policy 0, policy_version 93293 (0.0009) [2023-10-10 20:13:37,893][123582] Updated weights for policy 0, policy_version 93303 (0.0008) [2023-10-10 20:13:38,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 190971904. Throughput: 0: 1803.3, 1: 1809.8. Samples: 47752024. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 20:13:38,789][122664] Avg episode reward: [(0, '86.270'), (1, '81.050')] [2023-10-10 20:13:40,158][123614] Updated weights for policy 1, policy_version 93190 (0.0008) [2023-10-10 20:13:40,527][123614] Updated weights for policy 1, policy_version 93200 (0.0008) [2023-10-10 20:13:40,889][123614] Updated weights for policy 1, policy_version 93210 (0.0011) [2023-10-10 20:13:41,464][123582] Updated weights for policy 0, policy_version 93313 (0.0007) [2023-10-10 20:13:41,837][123582] Updated weights for policy 0, policy_version 93323 (0.0008) [2023-10-10 20:13:42,207][123582] Updated weights for policy 0, policy_version 93333 (0.0007) [2023-10-10 20:13:42,585][123582] Updated weights for policy 0, policy_version 93343 (0.0008) [2023-10-10 20:13:43,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 191037440. Throughput: 0: 1815.4, 1: 1811.6. Samples: 47763366. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 20:13:43,789][122664] Avg episode reward: [(0, '90.530'), (1, '79.360')] [2023-10-10 20:13:44,804][123614] Updated weights for policy 1, policy_version 93220 (0.0010) [2023-10-10 20:13:45,182][123614] Updated weights for policy 1, policy_version 93230 (0.0009) [2023-10-10 20:13:45,546][123614] Updated weights for policy 1, policy_version 93240 (0.0008) [2023-10-10 20:13:46,396][123582] Updated weights for policy 0, policy_version 93353 (0.0008) [2023-10-10 20:13:46,764][123582] Updated weights for policy 0, policy_version 93363 (0.0011) [2023-10-10 20:13:47,151][123582] Updated weights for policy 0, policy_version 93373 (0.0010) [2023-10-10 20:13:48,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 191102976. Throughput: 0: 1803.7, 1: 1817.5. Samples: 47784546. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 20:13:48,789][122664] Avg episode reward: [(0, '88.920'), (1, '83.550')] [2023-10-10 20:13:49,282][123614] Updated weights for policy 1, policy_version 93250 (0.0008) [2023-10-10 20:13:49,670][123614] Updated weights for policy 1, policy_version 93260 (0.0009) [2023-10-10 20:13:50,030][123614] Updated weights for policy 1, policy_version 93270 (0.0010) [2023-10-10 20:13:50,404][123614] Updated weights for policy 1, policy_version 93280 (0.0009) [2023-10-10 20:13:50,937][123582] Updated weights for policy 0, policy_version 93383 (0.0010) [2023-10-10 20:13:51,309][123582] Updated weights for policy 0, policy_version 93393 (0.0011) [2023-10-10 20:13:51,686][123582] Updated weights for policy 0, policy_version 93403 (0.0012) [2023-10-10 20:13:53,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 191168512. Throughput: 0: 1805.1, 1: 1815.1. Samples: 47806948. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 20:13:53,789][122664] Avg episode reward: [(0, '85.020'), (1, '84.010')] [2023-10-10 20:13:53,799][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000093408_95649792.pth... [2023-10-10 20:13:53,835][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000091712_93913088.pth [2023-10-10 20:13:53,839][123247] Saving a milestone ./train_atari/atari_demonattack_APPO/checkpoint_p0/milestones/checkpoint_000093408_95649792.pth [2023-10-10 20:13:54,101][123614] Updated weights for policy 1, policy_version 93290 (0.0010) [2023-10-10 20:13:54,475][123614] Updated weights for policy 1, policy_version 93300 (0.0010) [2023-10-10 20:13:54,830][123614] Updated weights for policy 1, policy_version 93310 (0.0010) [2023-10-10 20:13:54,903][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000093312_95551488.pth... [2023-10-10 20:13:54,932][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000091584_93782016.pth [2023-10-10 20:13:54,935][123465] Saving a milestone ./train_atari/atari_demonattack_APPO/checkpoint_p1/milestones/checkpoint_000093312_95551488.pth [2023-10-10 20:13:55,429][123582] Updated weights for policy 0, policy_version 93413 (0.0009) [2023-10-10 20:13:55,804][123582] Updated weights for policy 0, policy_version 93423 (0.0007) [2023-10-10 20:13:56,169][123582] Updated weights for policy 0, policy_version 93433 (0.0008) [2023-10-10 20:13:58,541][123614] Updated weights for policy 1, policy_version 93320 (0.0011) [2023-10-10 20:13:58,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 191234048. Throughput: 0: 1804.1, 1: 1812.8. Samples: 47816872. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 20:13:58,788][122664] Avg episode reward: [(0, '86.560'), (1, '84.850')] [2023-10-10 20:13:58,914][123614] Updated weights for policy 1, policy_version 93330 (0.0010) [2023-10-10 20:13:59,274][123614] Updated weights for policy 1, policy_version 93340 (0.0010) [2023-10-10 20:13:59,851][123582] Updated weights for policy 0, policy_version 93443 (0.0009) [2023-10-10 20:14:00,216][123582] Updated weights for policy 0, policy_version 93453 (0.0009) [2023-10-10 20:14:00,588][123582] Updated weights for policy 0, policy_version 93463 (0.0010) [2023-10-10 20:14:02,934][123614] Updated weights for policy 1, policy_version 93350 (0.0009) [2023-10-10 20:14:03,302][123614] Updated weights for policy 1, policy_version 93360 (0.0009) [2023-10-10 20:14:03,683][123614] Updated weights for policy 1, policy_version 93370 (0.0010) [2023-10-10 20:14:03,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 191299584. Throughput: 0: 1803.7, 1: 1819.2. Samples: 47839404. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:14:03,789][122664] Avg episode reward: [(0, '89.380'), (1, '84.400')] [2023-10-10 20:14:04,313][123582] Updated weights for policy 0, policy_version 93473 (0.0010) [2023-10-10 20:14:04,687][123582] Updated weights for policy 0, policy_version 93483 (0.0008) [2023-10-10 20:14:05,058][123582] Updated weights for policy 0, policy_version 93493 (0.0008) [2023-10-10 20:14:05,426][123582] Updated weights for policy 0, policy_version 93503 (0.0009) [2023-10-10 20:14:07,218][123614] Updated weights for policy 1, policy_version 93380 (0.0010) [2023-10-10 20:14:07,582][123614] Updated weights for policy 1, policy_version 93390 (0.0007) [2023-10-10 20:14:07,942][123614] Updated weights for policy 1, policy_version 93400 (0.0008) [2023-10-10 20:14:08,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 191397888. Throughput: 0: 1809.2, 1: 1814.3. Samples: 47861050. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:14:08,789][122664] Avg episode reward: [(0, '86.630'), (1, '74.570')] [2023-10-10 20:14:09,012][123582] Updated weights for policy 0, policy_version 93513 (0.0007) [2023-10-10 20:14:09,374][123582] Updated weights for policy 0, policy_version 93523 (0.0007) [2023-10-10 20:14:09,745][123582] Updated weights for policy 0, policy_version 93533 (0.0009) [2023-10-10 20:14:11,608][123614] Updated weights for policy 1, policy_version 93410 (0.0007) [2023-10-10 20:14:11,984][123614] Updated weights for policy 1, policy_version 93420 (0.0007) [2023-10-10 20:14:12,352][123614] Updated weights for policy 1, policy_version 93430 (0.0010) [2023-10-10 20:14:12,721][123614] Updated weights for policy 1, policy_version 93440 (0.0008) [2023-10-10 20:14:13,536][123582] Updated weights for policy 0, policy_version 93543 (0.0007) [2023-10-10 20:14:13,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 191463424. Throughput: 0: 1803.1, 1: 1819.1. Samples: 47872084. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:14:13,789][122664] Avg episode reward: [(0, '89.830'), (1, '79.070')] [2023-10-10 20:14:13,908][123582] Updated weights for policy 0, policy_version 93553 (0.0007) [2023-10-10 20:14:14,272][123582] Updated weights for policy 0, policy_version 93563 (0.0007) [2023-10-10 20:14:16,523][123614] Updated weights for policy 1, policy_version 93450 (0.0008) [2023-10-10 20:14:16,885][123614] Updated weights for policy 1, policy_version 93460 (0.0009) [2023-10-10 20:14:17,260][123614] Updated weights for policy 1, policy_version 93470 (0.0010) [2023-10-10 20:14:17,980][123582] Updated weights for policy 0, policy_version 93573 (0.0010) [2023-10-10 20:14:18,360][123582] Updated weights for policy 0, policy_version 93583 (0.0011) [2023-10-10 20:14:18,741][123582] Updated weights for policy 0, policy_version 93593 (0.0010) [2023-10-10 20:14:18,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 191528960. Throughput: 0: 1805.7, 1: 1815.3. Samples: 47893620. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:14:18,788][122664] Avg episode reward: [(0, '88.840'), (1, '77.910')] [2023-10-10 20:14:20,978][123614] Updated weights for policy 1, policy_version 93480 (0.0009) [2023-10-10 20:14:21,341][123614] Updated weights for policy 1, policy_version 93490 (0.0008) [2023-10-10 20:14:21,707][123614] Updated weights for policy 1, policy_version 93500 (0.0008) [2023-10-10 20:14:22,500][123582] Updated weights for policy 0, policy_version 93603 (0.0009) [2023-10-10 20:14:22,867][123582] Updated weights for policy 0, policy_version 93613 (0.0008) [2023-10-10 20:14:23,229][123582] Updated weights for policy 0, policy_version 93623 (0.0007) [2023-10-10 20:14:23,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 191627264. Throughput: 0: 1808.1, 1: 1812.9. Samples: 47914968. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:14:23,789][122664] Avg episode reward: [(0, '88.050'), (1, '75.190')] [2023-10-10 20:14:25,295][123614] Updated weights for policy 1, policy_version 93510 (0.0007) [2023-10-10 20:14:25,661][123614] Updated weights for policy 1, policy_version 93520 (0.0007) [2023-10-10 20:14:26,030][123614] Updated weights for policy 1, policy_version 93530 (0.0007) [2023-10-10 20:14:26,977][123582] Updated weights for policy 0, policy_version 93633 (0.0007) [2023-10-10 20:14:27,351][123582] Updated weights for policy 0, policy_version 93643 (0.0009) [2023-10-10 20:14:27,722][123582] Updated weights for policy 0, policy_version 93653 (0.0007) [2023-10-10 20:14:28,087][123582] Updated weights for policy 0, policy_version 93663 (0.0008) [2023-10-10 20:14:28,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 191692800. Throughput: 0: 1800.0, 1: 1816.4. Samples: 47926106. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:14:28,788][122664] Avg episode reward: [(0, '97.880'), (1, '80.770')] [2023-10-10 20:14:29,820][123614] Updated weights for policy 1, policy_version 93540 (0.0008) [2023-10-10 20:14:30,186][123614] Updated weights for policy 1, policy_version 93550 (0.0007) [2023-10-10 20:14:30,559][123614] Updated weights for policy 1, policy_version 93560 (0.0008) [2023-10-10 20:14:31,873][123582] Updated weights for policy 0, policy_version 93673 (0.0008) [2023-10-10 20:14:32,250][123582] Updated weights for policy 0, policy_version 93683 (0.0009) [2023-10-10 20:14:32,619][123582] Updated weights for policy 0, policy_version 93693 (0.0009) [2023-10-10 20:14:33,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 191758336. Throughput: 0: 1813.5, 1: 1813.3. Samples: 47947750. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:14:33,788][122664] Avg episode reward: [(0, '98.770'), (1, '79.920')] [2023-10-10 20:14:34,257][123614] Updated weights for policy 1, policy_version 93570 (0.0008) [2023-10-10 20:14:34,663][123614] Updated weights for policy 1, policy_version 93580 (0.0009) [2023-10-10 20:14:35,040][123614] Updated weights for policy 1, policy_version 93590 (0.0007) [2023-10-10 20:14:35,413][123614] Updated weights for policy 1, policy_version 93600 (0.0008) [2023-10-10 20:14:36,232][123582] Updated weights for policy 0, policy_version 93703 (0.0008) [2023-10-10 20:14:36,597][123582] Updated weights for policy 0, policy_version 93713 (0.0008) [2023-10-10 20:14:36,977][123582] Updated weights for policy 0, policy_version 93723 (0.0008) [2023-10-10 20:14:38,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 191823872. Throughput: 0: 1805.7, 1: 1818.6. Samples: 47970042. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:14:38,789][122664] Avg episode reward: [(0, '98.240'), (1, '83.020')] [2023-10-10 20:14:39,148][123614] Updated weights for policy 1, policy_version 93610 (0.0009) [2023-10-10 20:14:39,522][123614] Updated weights for policy 1, policy_version 93620 (0.0009) [2023-10-10 20:14:39,903][123614] Updated weights for policy 1, policy_version 93630 (0.0009) [2023-10-10 20:14:40,682][123582] Updated weights for policy 0, policy_version 93733 (0.0009) [2023-10-10 20:14:41,059][123582] Updated weights for policy 0, policy_version 93743 (0.0011) [2023-10-10 20:14:41,420][123582] Updated weights for policy 0, policy_version 93753 (0.0010) [2023-10-10 20:14:43,507][123614] Updated weights for policy 1, policy_version 93640 (0.0008) [2023-10-10 20:14:43,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 191889408. Throughput: 0: 1816.1, 1: 1817.1. Samples: 47980366. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:14:43,788][122664] Avg episode reward: [(0, '98.190'), (1, '84.210')] [2023-10-10 20:14:43,869][123614] Updated weights for policy 1, policy_version 93650 (0.0007) [2023-10-10 20:14:44,239][123614] Updated weights for policy 1, policy_version 93660 (0.0007) [2023-10-10 20:14:45,075][123582] Updated weights for policy 0, policy_version 93763 (0.0009) [2023-10-10 20:14:45,458][123582] Updated weights for policy 0, policy_version 93773 (0.0008) [2023-10-10 20:14:45,828][123582] Updated weights for policy 0, policy_version 93783 (0.0007) [2023-10-10 20:14:47,989][123614] Updated weights for policy 1, policy_version 93670 (0.0007) [2023-10-10 20:14:48,346][123614] Updated weights for policy 1, policy_version 93680 (0.0007) [2023-10-10 20:14:48,715][123614] Updated weights for policy 1, policy_version 93690 (0.0008) [2023-10-10 20:14:48,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 191954944. Throughput: 0: 1810.7, 1: 1819.1. Samples: 48002744. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:14:48,788][122664] Avg episode reward: [(0, '100.650'), (1, '87.650')] [2023-10-10 20:14:49,631][123582] Updated weights for policy 0, policy_version 93793 (0.0007) [2023-10-10 20:14:49,995][123582] Updated weights for policy 0, policy_version 93803 (0.0009) [2023-10-10 20:14:50,362][123582] Updated weights for policy 0, policy_version 93813 (0.0007) [2023-10-10 20:14:50,735][123582] Updated weights for policy 0, policy_version 93823 (0.0007) [2023-10-10 20:14:52,371][123614] Updated weights for policy 1, policy_version 93700 (0.0009) [2023-10-10 20:14:52,741][123614] Updated weights for policy 1, policy_version 93710 (0.0009) [2023-10-10 20:14:53,111][123614] Updated weights for policy 1, policy_version 93720 (0.0008) [2023-10-10 20:14:53,788][122664] Fps is (10 sec: 16383.0, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 192053248. Throughput: 0: 1807.8, 1: 1818.6. Samples: 48024238. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:14:53,790][122664] Avg episode reward: [(0, '105.350'), (1, '90.410')] [2023-10-10 20:14:54,325][123582] Updated weights for policy 0, policy_version 93833 (0.0007) [2023-10-10 20:14:54,706][123582] Updated weights for policy 0, policy_version 93843 (0.0008) [2023-10-10 20:14:55,082][123582] Updated weights for policy 0, policy_version 93853 (0.0009) [2023-10-10 20:14:56,846][123614] Updated weights for policy 1, policy_version 93730 (0.0007) [2023-10-10 20:14:57,209][123614] Updated weights for policy 1, policy_version 93740 (0.0009) [2023-10-10 20:14:57,575][123614] Updated weights for policy 1, policy_version 93750 (0.0008) [2023-10-10 20:14:57,946][123614] Updated weights for policy 1, policy_version 93760 (0.0008) [2023-10-10 20:14:58,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 192118784. Throughput: 0: 1812.5, 1: 1823.2. Samples: 48035692. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:14:58,789][122664] Avg episode reward: [(0, '104.170'), (1, '87.810')] [2023-10-10 20:14:58,850][123582] Updated weights for policy 0, policy_version 93863 (0.0007) [2023-10-10 20:14:59,226][123582] Updated weights for policy 0, policy_version 93873 (0.0009) [2023-10-10 20:14:59,598][123582] Updated weights for policy 0, policy_version 93883 (0.0008) [2023-10-10 20:15:01,670][123614] Updated weights for policy 1, policy_version 93770 (0.0009) [2023-10-10 20:15:02,043][123614] Updated weights for policy 1, policy_version 93780 (0.0008) [2023-10-10 20:15:02,410][123614] Updated weights for policy 1, policy_version 93790 (0.0007) [2023-10-10 20:15:03,235][123582] Updated weights for policy 0, policy_version 93893 (0.0010) [2023-10-10 20:15:03,609][123582] Updated weights for policy 0, policy_version 93903 (0.0008) [2023-10-10 20:15:03,788][122664] Fps is (10 sec: 13107.9, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 192184320. Throughput: 0: 1816.0, 1: 1817.2. Samples: 48057112. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:15:03,789][122664] Avg episode reward: [(0, '105.710'), (1, '80.890')] [2023-10-10 20:15:03,980][123582] Updated weights for policy 0, policy_version 93913 (0.0007) [2023-10-10 20:15:05,990][123614] Updated weights for policy 1, policy_version 93800 (0.0008) [2023-10-10 20:15:06,359][123614] Updated weights for policy 1, policy_version 93810 (0.0007) [2023-10-10 20:15:06,729][123614] Updated weights for policy 1, policy_version 93820 (0.0008) [2023-10-10 20:15:07,553][123582] Updated weights for policy 0, policy_version 93923 (0.0008) [2023-10-10 20:15:07,920][123582] Updated weights for policy 0, policy_version 93933 (0.0007) [2023-10-10 20:15:08,291][123582] Updated weights for policy 0, policy_version 93943 (0.0008) [2023-10-10 20:15:08,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 192282624. Throughput: 0: 1822.5, 1: 1818.2. Samples: 48078802. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:15:08,789][122664] Avg episode reward: [(0, '106.190'), (1, '79.750')] [2023-10-10 20:15:10,347][123614] Updated weights for policy 1, policy_version 93830 (0.0008) [2023-10-10 20:15:10,715][123614] Updated weights for policy 1, policy_version 93840 (0.0008) [2023-10-10 20:15:11,081][123614] Updated weights for policy 1, policy_version 93850 (0.0009) [2023-10-10 20:15:11,915][123582] Updated weights for policy 0, policy_version 93953 (0.0008) [2023-10-10 20:15:12,294][123582] Updated weights for policy 0, policy_version 93963 (0.0009) [2023-10-10 20:15:12,677][123582] Updated weights for policy 0, policy_version 93973 (0.0009) [2023-10-10 20:15:13,044][123582] Updated weights for policy 0, policy_version 93983 (0.0007) [2023-10-10 20:15:13,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 192348160. Throughput: 0: 1826.4, 1: 1817.8. Samples: 48090092. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:15:13,789][122664] Avg episode reward: [(0, '106.490'), (1, '79.900')] [2023-10-10 20:15:14,709][123614] Updated weights for policy 1, policy_version 93860 (0.0009) [2023-10-10 20:15:15,085][123614] Updated weights for policy 1, policy_version 93870 (0.0008) [2023-10-10 20:15:15,458][123614] Updated weights for policy 1, policy_version 93880 (0.0008) [2023-10-10 20:15:16,594][123582] Updated weights for policy 0, policy_version 93993 (0.0008) [2023-10-10 20:15:16,968][123582] Updated weights for policy 0, policy_version 94003 (0.0009) [2023-10-10 20:15:17,343][123582] Updated weights for policy 0, policy_version 94013 (0.0008) [2023-10-10 20:15:18,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 192413696. Throughput: 0: 1818.1, 1: 1817.0. Samples: 48111328. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:15:18,789][122664] Avg episode reward: [(0, '103.320'), (1, '79.390')] [2023-10-10 20:15:19,332][123614] Updated weights for policy 1, policy_version 93890 (0.0008) [2023-10-10 20:15:19,738][123614] Updated weights for policy 1, policy_version 93900 (0.0009) [2023-10-10 20:15:20,090][123614] Updated weights for policy 1, policy_version 93910 (0.0008) [2023-10-10 20:15:20,451][123614] Updated weights for policy 1, policy_version 93920 (0.0008) [2023-10-10 20:15:21,033][123582] Updated weights for policy 0, policy_version 94023 (0.0008) [2023-10-10 20:15:21,406][123582] Updated weights for policy 0, policy_version 94033 (0.0008) [2023-10-10 20:15:21,783][123582] Updated weights for policy 0, policy_version 94043 (0.0007) [2023-10-10 20:15:23,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 192479232. Throughput: 0: 1826.0, 1: 1817.6. Samples: 48134002. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:15:23,788][122664] Avg episode reward: [(0, '103.130'), (1, '82.030')] [2023-10-10 20:15:24,024][123614] Updated weights for policy 1, policy_version 93930 (0.0009) [2023-10-10 20:15:24,388][123614] Updated weights for policy 1, policy_version 93940 (0.0007) [2023-10-10 20:15:24,758][123614] Updated weights for policy 1, policy_version 93950 (0.0008) [2023-10-10 20:15:25,514][123582] Updated weights for policy 0, policy_version 94053 (0.0007) [2023-10-10 20:15:25,884][123582] Updated weights for policy 0, policy_version 94063 (0.0008) [2023-10-10 20:15:26,253][123582] Updated weights for policy 0, policy_version 94073 (0.0010) [2023-10-10 20:15:28,479][123614] Updated weights for policy 1, policy_version 93960 (0.0007) [2023-10-10 20:15:28,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 192544768. Throughput: 0: 1817.3, 1: 1822.8. Samples: 48144170. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:15:28,789][122664] Avg episode reward: [(0, '101.330'), (1, '80.940')] [2023-10-10 20:15:28,837][123614] Updated weights for policy 1, policy_version 93970 (0.0007) [2023-10-10 20:15:29,206][123614] Updated weights for policy 1, policy_version 93980 (0.0008) [2023-10-10 20:15:29,980][123582] Updated weights for policy 0, policy_version 94083 (0.0008) [2023-10-10 20:15:30,352][123582] Updated weights for policy 0, policy_version 94093 (0.0007) [2023-10-10 20:15:30,727][123582] Updated weights for policy 0, policy_version 94103 (0.0007) [2023-10-10 20:15:32,883][123614] Updated weights for policy 1, policy_version 93990 (0.0008) [2023-10-10 20:15:33,258][123614] Updated weights for policy 1, policy_version 94000 (0.0009) [2023-10-10 20:15:33,623][123614] Updated weights for policy 1, policy_version 94010 (0.0007) [2023-10-10 20:15:33,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 192610304. Throughput: 0: 1823.9, 1: 1821.6. Samples: 48166790. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:15:33,788][122664] Avg episode reward: [(0, '103.430'), (1, '85.170')] [2023-10-10 20:15:34,342][123582] Updated weights for policy 0, policy_version 94113 (0.0008) [2023-10-10 20:15:34,711][123582] Updated weights for policy 0, policy_version 94123 (0.0008) [2023-10-10 20:15:35,086][123582] Updated weights for policy 0, policy_version 94133 (0.0010) [2023-10-10 20:15:35,465][123582] Updated weights for policy 0, policy_version 94143 (0.0010) [2023-10-10 20:15:37,282][123614] Updated weights for policy 1, policy_version 94020 (0.0007) [2023-10-10 20:15:37,645][123614] Updated weights for policy 1, policy_version 94030 (0.0007) [2023-10-10 20:15:38,010][123614] Updated weights for policy 1, policy_version 94040 (0.0008) [2023-10-10 20:15:38,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 192708608. Throughput: 0: 1829.0, 1: 1820.1. Samples: 48188446. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:15:38,789][122664] Avg episode reward: [(0, '117.640'), (1, '86.140')] [2023-10-10 20:15:39,011][123582] Updated weights for policy 0, policy_version 94153 (0.0011) [2023-10-10 20:15:39,385][123582] Updated weights for policy 0, policy_version 94163 (0.0007) [2023-10-10 20:15:39,758][123582] Updated weights for policy 0, policy_version 94173 (0.0008) [2023-10-10 20:15:41,588][123614] Updated weights for policy 1, policy_version 94050 (0.0009) [2023-10-10 20:15:41,960][123614] Updated weights for policy 1, policy_version 94060 (0.0008) [2023-10-10 20:15:42,322][123614] Updated weights for policy 1, policy_version 94070 (0.0007) [2023-10-10 20:15:42,695][123614] Updated weights for policy 1, policy_version 94080 (0.0008) [2023-10-10 20:15:43,434][123582] Updated weights for policy 0, policy_version 94183 (0.0010) [2023-10-10 20:15:43,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 192774144. Throughput: 0: 1828.0, 1: 1817.3. Samples: 48199730. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:15:43,789][122664] Avg episode reward: [(0, '118.650'), (1, '78.540')] [2023-10-10 20:15:43,818][123582] Updated weights for policy 0, policy_version 94193 (0.0010) [2023-10-10 20:15:44,193][123582] Updated weights for policy 0, policy_version 94203 (0.0011) [2023-10-10 20:15:46,400][123614] Updated weights for policy 1, policy_version 94090 (0.0007) [2023-10-10 20:15:46,760][123614] Updated weights for policy 1, policy_version 94100 (0.0007) [2023-10-10 20:15:47,130][123614] Updated weights for policy 1, policy_version 94110 (0.0008) [2023-10-10 20:15:47,961][123582] Updated weights for policy 0, policy_version 94213 (0.0008) [2023-10-10 20:15:48,339][123582] Updated weights for policy 0, policy_version 94223 (0.0009) [2023-10-10 20:15:48,698][123582] Updated weights for policy 0, policy_version 94233 (0.0010) [2023-10-10 20:15:48,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 192839680. Throughput: 0: 1822.9, 1: 1829.4. Samples: 48221464. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:15:48,790][122664] Avg episode reward: [(0, '118.060'), (1, '81.200')] [2023-10-10 20:15:50,889][123614] Updated weights for policy 1, policy_version 94120 (0.0007) [2023-10-10 20:15:51,265][123614] Updated weights for policy 1, policy_version 94130 (0.0007) [2023-10-10 20:15:51,642][123614] Updated weights for policy 1, policy_version 94140 (0.0009) [2023-10-10 20:15:52,404][123582] Updated weights for policy 0, policy_version 94243 (0.0009) [2023-10-10 20:15:52,778][123582] Updated weights for policy 0, policy_version 94253 (0.0009) [2023-10-10 20:15:53,154][123582] Updated weights for policy 0, policy_version 94263 (0.0010) [2023-10-10 20:15:53,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 192937984. Throughput: 0: 1822.2, 1: 1827.8. Samples: 48243054. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:15:53,789][122664] Avg episode reward: [(0, '116.830'), (1, '81.980')] [2023-10-10 20:15:53,797][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000094144_96403456.pth... [2023-10-10 20:15:53,798][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000094272_96534528.pth... [2023-10-10 20:15:53,838][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000092448_94666752.pth [2023-10-10 20:15:53,839][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000092576_94797824.pth [2023-10-10 20:15:55,269][123614] Updated weights for policy 1, policy_version 94150 (0.0009) [2023-10-10 20:15:55,633][123614] Updated weights for policy 1, policy_version 94160 (0.0009) [2023-10-10 20:15:56,000][123614] Updated weights for policy 1, policy_version 94170 (0.0007) [2023-10-10 20:15:56,916][123582] Updated weights for policy 0, policy_version 94273 (0.0010) [2023-10-10 20:15:57,287][123582] Updated weights for policy 0, policy_version 94283 (0.0007) [2023-10-10 20:15:57,664][123582] Updated weights for policy 0, policy_version 94293 (0.0007) [2023-10-10 20:15:58,042][123582] Updated weights for policy 0, policy_version 94303 (0.0011) [2023-10-10 20:15:58,788][122664] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 193003520. Throughput: 0: 1823.2, 1: 1825.9. Samples: 48254298. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-10 20:15:58,788][122664] Avg episode reward: [(0, '120.390'), (1, '77.910')] [2023-10-10 20:15:59,797][123614] Updated weights for policy 1, policy_version 94180 (0.0008) [2023-10-10 20:16:00,176][123614] Updated weights for policy 1, policy_version 94190 (0.0012) [2023-10-10 20:16:00,541][123614] Updated weights for policy 1, policy_version 94200 (0.0009) [2023-10-10 20:16:01,693][123582] Updated weights for policy 0, policy_version 94313 (0.0007) [2023-10-10 20:16:02,067][123582] Updated weights for policy 0, policy_version 94323 (0.0008) [2023-10-10 20:16:02,437][123582] Updated weights for policy 0, policy_version 94333 (0.0010) [2023-10-10 20:16:03,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 193069056. Throughput: 0: 1828.4, 1: 1826.2. Samples: 48275784. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-10 20:16:03,789][122664] Avg episode reward: [(0, '118.460'), (1, '76.340')] [2023-10-10 20:16:04,242][123614] Updated weights for policy 1, policy_version 94210 (0.0008) [2023-10-10 20:16:04,646][123614] Updated weights for policy 1, policy_version 94220 (0.0007) [2023-10-10 20:16:05,013][123614] Updated weights for policy 1, policy_version 94230 (0.0007) [2023-10-10 20:16:05,382][123614] Updated weights for policy 1, policy_version 94240 (0.0009) [2023-10-10 20:16:06,116][123582] Updated weights for policy 0, policy_version 94343 (0.0009) [2023-10-10 20:16:06,496][123582] Updated weights for policy 0, policy_version 94353 (0.0009) [2023-10-10 20:16:06,866][123582] Updated weights for policy 0, policy_version 94363 (0.0010) [2023-10-10 20:16:08,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 193134592. Throughput: 0: 1825.9, 1: 1822.8. Samples: 48298192. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-10 20:16:08,788][122664] Avg episode reward: [(0, '108.690'), (1, '78.220')] [2023-10-10 20:16:09,153][123614] Updated weights for policy 1, policy_version 94250 (0.0007) [2023-10-10 20:16:09,519][123614] Updated weights for policy 1, policy_version 94260 (0.0009) [2023-10-10 20:16:09,889][123614] Updated weights for policy 1, policy_version 94270 (0.0008) [2023-10-10 20:16:10,572][123582] Updated weights for policy 0, policy_version 94373 (0.0009) [2023-10-10 20:16:10,949][123582] Updated weights for policy 0, policy_version 94383 (0.0009) [2023-10-10 20:16:11,325][123582] Updated weights for policy 0, policy_version 94393 (0.0007) [2023-10-10 20:16:13,526][123614] Updated weights for policy 1, policy_version 94280 (0.0008) [2023-10-10 20:16:13,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 193200128. Throughput: 0: 1832.6, 1: 1818.6. Samples: 48308472. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-10 20:16:13,788][122664] Avg episode reward: [(0, '109.130'), (1, '79.020')] [2023-10-10 20:16:13,904][123614] Updated weights for policy 1, policy_version 94290 (0.0008) [2023-10-10 20:16:14,270][123614] Updated weights for policy 1, policy_version 94300 (0.0010) [2023-10-10 20:16:14,887][123582] Updated weights for policy 0, policy_version 94403 (0.0009) [2023-10-10 20:16:15,257][123582] Updated weights for policy 0, policy_version 94413 (0.0009) [2023-10-10 20:16:15,631][123582] Updated weights for policy 0, policy_version 94423 (0.0011) [2023-10-10 20:16:17,874][123614] Updated weights for policy 1, policy_version 94310 (0.0008) [2023-10-10 20:16:18,239][123614] Updated weights for policy 1, policy_version 94320 (0.0008) [2023-10-10 20:16:18,610][123614] Updated weights for policy 1, policy_version 94330 (0.0008) [2023-10-10 20:16:18,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 193265664. Throughput: 0: 1830.0, 1: 1823.0. Samples: 48331178. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-10 20:16:18,789][122664] Avg episode reward: [(0, '104.550'), (1, '78.480')] [2023-10-10 20:16:19,469][123582] Updated weights for policy 0, policy_version 94433 (0.0010) [2023-10-10 20:16:19,846][123582] Updated weights for policy 0, policy_version 94443 (0.0010) [2023-10-10 20:16:20,210][123582] Updated weights for policy 0, policy_version 94453 (0.0009) [2023-10-10 20:16:20,579][123582] Updated weights for policy 0, policy_version 94463 (0.0009) [2023-10-10 20:16:22,154][123614] Updated weights for policy 1, policy_version 94340 (0.0008) [2023-10-10 20:16:22,514][123614] Updated weights for policy 1, policy_version 94350 (0.0011) [2023-10-10 20:16:22,886][123614] Updated weights for policy 1, policy_version 94360 (0.0010) [2023-10-10 20:16:23,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 193363968. Throughput: 0: 1823.1, 1: 1823.4. Samples: 48352538. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-10 20:16:23,789][122664] Avg episode reward: [(0, '103.920'), (1, '77.220')] [2023-10-10 20:16:24,101][123582] Updated weights for policy 0, policy_version 94473 (0.0010) [2023-10-10 20:16:24,468][123582] Updated weights for policy 0, policy_version 94483 (0.0009) [2023-10-10 20:16:24,845][123582] Updated weights for policy 0, policy_version 94493 (0.0008) [2023-10-10 20:16:26,566][123614] Updated weights for policy 1, policy_version 94370 (0.0008) [2023-10-10 20:16:26,929][123614] Updated weights for policy 1, policy_version 94380 (0.0008) [2023-10-10 20:16:27,294][123614] Updated weights for policy 1, policy_version 94390 (0.0008) [2023-10-10 20:16:27,660][123614] Updated weights for policy 1, policy_version 94400 (0.0008) [2023-10-10 20:16:28,625][123582] Updated weights for policy 0, policy_version 94503 (0.0008) [2023-10-10 20:16:28,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 193429504. Throughput: 0: 1824.3, 1: 1824.8. Samples: 48363938. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-10 20:16:28,789][122664] Avg episode reward: [(0, '99.230'), (1, '80.630')] [2023-10-10 20:16:29,011][123582] Updated weights for policy 0, policy_version 94513 (0.0007) [2023-10-10 20:16:29,378][123582] Updated weights for policy 0, policy_version 94523 (0.0010) [2023-10-10 20:16:31,180][123614] Updated weights for policy 1, policy_version 94410 (0.0008) [2023-10-10 20:16:31,553][123614] Updated weights for policy 1, policy_version 94420 (0.0009) [2023-10-10 20:16:31,914][123614] Updated weights for policy 1, policy_version 94430 (0.0008) [2023-10-10 20:16:32,976][123582] Updated weights for policy 0, policy_version 94533 (0.0011) [2023-10-10 20:16:33,356][123582] Updated weights for policy 0, policy_version 94543 (0.0007) [2023-10-10 20:16:33,727][123582] Updated weights for policy 0, policy_version 94553 (0.0007) [2023-10-10 20:16:33,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 193495040. Throughput: 0: 1824.5, 1: 1826.8. Samples: 48385768. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-10 20:16:33,788][122664] Avg episode reward: [(0, '92.370'), (1, '79.490')] [2023-10-10 20:16:35,695][123614] Updated weights for policy 1, policy_version 94440 (0.0007) [2023-10-10 20:16:36,063][123614] Updated weights for policy 1, policy_version 94450 (0.0008) [2023-10-10 20:16:36,425][123614] Updated weights for policy 1, policy_version 94460 (0.0008) [2023-10-10 20:16:37,386][123582] Updated weights for policy 0, policy_version 94563 (0.0009) [2023-10-10 20:16:37,757][123582] Updated weights for policy 0, policy_version 94573 (0.0011) [2023-10-10 20:16:38,126][123582] Updated weights for policy 0, policy_version 94583 (0.0011) [2023-10-10 20:16:38,788][122664] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 193593344. Throughput: 0: 1825.9, 1: 1825.1. Samples: 48407348. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-10 20:16:38,789][122664] Avg episode reward: [(0, '93.980'), (1, '79.290')] [2023-10-10 20:16:40,141][123614] Updated weights for policy 1, policy_version 94470 (0.0010) [2023-10-10 20:16:40,509][123614] Updated weights for policy 1, policy_version 94480 (0.0009) [2023-10-10 20:16:40,869][123614] Updated weights for policy 1, policy_version 94490 (0.0009) [2023-10-10 20:16:41,725][123582] Updated weights for policy 0, policy_version 94593 (0.0010) [2023-10-10 20:16:42,093][123582] Updated weights for policy 0, policy_version 94603 (0.0008) [2023-10-10 20:16:42,470][123582] Updated weights for policy 0, policy_version 94613 (0.0010) [2023-10-10 20:16:42,841][123582] Updated weights for policy 0, policy_version 94623 (0.0007) [2023-10-10 20:16:43,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 193658880. Throughput: 0: 1826.1, 1: 1824.4. Samples: 48418570. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-10 20:16:43,789][122664] Avg episode reward: [(0, '93.980'), (1, '74.020')] [2023-10-10 20:16:44,619][123614] Updated weights for policy 1, policy_version 94500 (0.0010) [2023-10-10 20:16:44,991][123614] Updated weights for policy 1, policy_version 94510 (0.0008) [2023-10-10 20:16:45,355][123614] Updated weights for policy 1, policy_version 94520 (0.0008) [2023-10-10 20:16:46,424][123582] Updated weights for policy 0, policy_version 94633 (0.0008) [2023-10-10 20:16:46,794][123582] Updated weights for policy 0, policy_version 94643 (0.0007) [2023-10-10 20:16:47,156][123582] Updated weights for policy 0, policy_version 94653 (0.0008) [2023-10-10 20:16:48,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 193724416. Throughput: 0: 1822.1, 1: 1831.6. Samples: 48440198. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-10 20:16:48,789][122664] Avg episode reward: [(0, '95.180'), (1, '77.630')] [2023-10-10 20:16:49,260][123614] Updated weights for policy 1, policy_version 94530 (0.0009) [2023-10-10 20:16:49,675][123614] Updated weights for policy 1, policy_version 94540 (0.0011) [2023-10-10 20:16:50,047][123614] Updated weights for policy 1, policy_version 94550 (0.0010) [2023-10-10 20:16:50,411][123614] Updated weights for policy 1, policy_version 94560 (0.0007) [2023-10-10 20:16:50,904][123582] Updated weights for policy 0, policy_version 94663 (0.0008) [2023-10-10 20:16:51,276][123582] Updated weights for policy 0, policy_version 94673 (0.0007) [2023-10-10 20:16:51,646][123582] Updated weights for policy 0, policy_version 94683 (0.0010) [2023-10-10 20:16:53,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 193789952. Throughput: 0: 1822.2, 1: 1824.7. Samples: 48462304. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-10 20:16:53,788][122664] Avg episode reward: [(0, '88.790'), (1, '79.910')] [2023-10-10 20:16:54,091][123614] Updated weights for policy 1, policy_version 94570 (0.0008) [2023-10-10 20:16:54,469][123614] Updated weights for policy 1, policy_version 94580 (0.0009) [2023-10-10 20:16:54,846][123614] Updated weights for policy 1, policy_version 94590 (0.0007) [2023-10-10 20:16:55,429][123582] Updated weights for policy 0, policy_version 94693 (0.0011) [2023-10-10 20:16:55,806][123582] Updated weights for policy 0, policy_version 94703 (0.0009) [2023-10-10 20:16:56,178][123582] Updated weights for policy 0, policy_version 94713 (0.0008) [2023-10-10 20:16:58,622][123614] Updated weights for policy 1, policy_version 94600 (0.0007) [2023-10-10 20:16:58,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 193855488. Throughput: 0: 1814.2, 1: 1823.7. Samples: 48472178. Policy #0 lag: (min: 9.0, avg: 36.8, max: 40.0) [2023-10-10 20:16:58,789][122664] Avg episode reward: [(0, '81.160'), (1, '73.870')] [2023-10-10 20:16:58,989][123614] Updated weights for policy 1, policy_version 94610 (0.0007) [2023-10-10 20:16:59,362][123614] Updated weights for policy 1, policy_version 94620 (0.0008) [2023-10-10 20:16:59,929][123582] Updated weights for policy 0, policy_version 94723 (0.0009) [2023-10-10 20:17:00,304][123582] Updated weights for policy 0, policy_version 94733 (0.0010) [2023-10-10 20:17:00,672][123582] Updated weights for policy 0, policy_version 94743 (0.0007) [2023-10-10 20:17:02,962][123614] Updated weights for policy 1, policy_version 94630 (0.0008) [2023-10-10 20:17:03,324][123614] Updated weights for policy 1, policy_version 94640 (0.0008) [2023-10-10 20:17:03,689][123614] Updated weights for policy 1, policy_version 94650 (0.0008) [2023-10-10 20:17:03,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 193921024. Throughput: 0: 1813.3, 1: 1818.1. Samples: 48494588. Policy #0 lag: (min: 9.0, avg: 36.8, max: 40.0) [2023-10-10 20:17:03,788][122664] Avg episode reward: [(0, '82.010'), (1, '72.680')] [2023-10-10 20:17:04,367][123582] Updated weights for policy 0, policy_version 94753 (0.0008) [2023-10-10 20:17:04,736][123582] Updated weights for policy 0, policy_version 94763 (0.0009) [2023-10-10 20:17:05,100][123582] Updated weights for policy 0, policy_version 94773 (0.0011) [2023-10-10 20:17:05,473][123582] Updated weights for policy 0, policy_version 94783 (0.0008) [2023-10-10 20:17:07,307][123614] Updated weights for policy 1, policy_version 94660 (0.0011) [2023-10-10 20:17:07,667][123614] Updated weights for policy 1, policy_version 94670 (0.0011) [2023-10-10 20:17:08,049][123614] Updated weights for policy 1, policy_version 94680 (0.0012) [2023-10-10 20:17:08,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 194019328. Throughput: 0: 1814.0, 1: 1820.4. Samples: 48516086. Policy #0 lag: (min: 9.0, avg: 36.8, max: 40.0) [2023-10-10 20:17:08,789][122664] Avg episode reward: [(0, '79.290'), (1, '73.530')] [2023-10-10 20:17:09,107][123582] Updated weights for policy 0, policy_version 94793 (0.0009) [2023-10-10 20:17:09,471][123582] Updated weights for policy 0, policy_version 94803 (0.0008) [2023-10-10 20:17:09,843][123582] Updated weights for policy 0, policy_version 94813 (0.0009) [2023-10-10 20:17:11,854][123614] Updated weights for policy 1, policy_version 94690 (0.0009) [2023-10-10 20:17:12,227][123614] Updated weights for policy 1, policy_version 94700 (0.0008) [2023-10-10 20:17:12,591][123614] Updated weights for policy 1, policy_version 94710 (0.0009) [2023-10-10 20:17:12,961][123614] Updated weights for policy 1, policy_version 94720 (0.0008) [2023-10-10 20:17:13,696][123582] Updated weights for policy 0, policy_version 94823 (0.0008) [2023-10-10 20:17:13,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 194084864. Throughput: 0: 1811.9, 1: 1818.9. Samples: 48527326. Policy #0 lag: (min: 9.0, avg: 36.8, max: 40.0) [2023-10-10 20:17:13,788][122664] Avg episode reward: [(0, '81.140'), (1, '72.280')] [2023-10-10 20:17:14,069][123582] Updated weights for policy 0, policy_version 94833 (0.0009) [2023-10-10 20:17:14,450][123582] Updated weights for policy 0, policy_version 94843 (0.0010) [2023-10-10 20:17:16,607][123614] Updated weights for policy 1, policy_version 94730 (0.0008) [2023-10-10 20:17:16,973][123614] Updated weights for policy 1, policy_version 94740 (0.0008) [2023-10-10 20:17:17,334][123614] Updated weights for policy 1, policy_version 94750 (0.0007) [2023-10-10 20:17:18,096][123582] Updated weights for policy 0, policy_version 94853 (0.0007) [2023-10-10 20:17:18,465][123582] Updated weights for policy 0, policy_version 94863 (0.0007) [2023-10-10 20:17:18,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 194150400. Throughput: 0: 1811.1, 1: 1808.5. Samples: 48548652. Policy #0 lag: (min: 9.0, avg: 36.8, max: 40.0) [2023-10-10 20:17:18,788][122664] Avg episode reward: [(0, '78.910'), (1, '73.640')] [2023-10-10 20:17:18,837][123582] Updated weights for policy 0, policy_version 94873 (0.0008) [2023-10-10 20:17:21,015][123614] Updated weights for policy 1, policy_version 94760 (0.0009) [2023-10-10 20:17:21,377][123614] Updated weights for policy 1, policy_version 94770 (0.0009) [2023-10-10 20:17:21,741][123614] Updated weights for policy 1, policy_version 94780 (0.0009) [2023-10-10 20:17:22,573][123582] Updated weights for policy 0, policy_version 94883 (0.0008) [2023-10-10 20:17:22,946][123582] Updated weights for policy 0, policy_version 94893 (0.0012) [2023-10-10 20:17:23,317][123582] Updated weights for policy 0, policy_version 94903 (0.0010) [2023-10-10 20:17:23,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 194248704. Throughput: 0: 1808.6, 1: 1804.4. Samples: 48569934. Policy #0 lag: (min: 9.0, avg: 36.8, max: 40.0) [2023-10-10 20:17:23,789][122664] Avg episode reward: [(0, '75.720'), (1, '76.890')] [2023-10-10 20:17:25,531][123614] Updated weights for policy 1, policy_version 94790 (0.0009) [2023-10-10 20:17:25,895][123614] Updated weights for policy 1, policy_version 94800 (0.0010) [2023-10-10 20:17:26,261][123614] Updated weights for policy 1, policy_version 94810 (0.0007) [2023-10-10 20:17:26,855][123582] Updated weights for policy 0, policy_version 94913 (0.0010) [2023-10-10 20:17:27,226][123582] Updated weights for policy 0, policy_version 94923 (0.0010) [2023-10-10 20:17:27,607][123582] Updated weights for policy 0, policy_version 94933 (0.0009) [2023-10-10 20:17:27,972][123582] Updated weights for policy 0, policy_version 94943 (0.0010) [2023-10-10 20:17:28,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 194314240. Throughput: 0: 1809.5, 1: 1803.1. Samples: 48581136. Policy #0 lag: (min: 9.0, avg: 36.8, max: 40.0) [2023-10-10 20:17:28,789][122664] Avg episode reward: [(0, '76.990'), (1, '79.160')] [2023-10-10 20:17:29,889][123614] Updated weights for policy 1, policy_version 94820 (0.0008) [2023-10-10 20:17:30,256][123614] Updated weights for policy 1, policy_version 94830 (0.0010) [2023-10-10 20:17:30,620][123614] Updated weights for policy 1, policy_version 94840 (0.0008) [2023-10-10 20:17:31,766][123582] Updated weights for policy 0, policy_version 94953 (0.0008) [2023-10-10 20:17:32,134][123582] Updated weights for policy 0, policy_version 94963 (0.0007) [2023-10-10 20:17:32,502][123582] Updated weights for policy 0, policy_version 94973 (0.0009) [2023-10-10 20:17:33,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 194379776. Throughput: 0: 1815.3, 1: 1797.6. Samples: 48602780. Policy #0 lag: (min: 9.0, avg: 36.8, max: 40.0) [2023-10-10 20:17:33,788][122664] Avg episode reward: [(0, '72.490'), (1, '77.410')] [2023-10-10 20:17:34,355][123614] Updated weights for policy 1, policy_version 94850 (0.0009) [2023-10-10 20:17:34,738][123614] Updated weights for policy 1, policy_version 94860 (0.0010) [2023-10-10 20:17:35,102][123614] Updated weights for policy 1, policy_version 94870 (0.0007) [2023-10-10 20:17:35,465][123614] Updated weights for policy 1, policy_version 94880 (0.0010) [2023-10-10 20:17:36,102][123582] Updated weights for policy 0, policy_version 94983 (0.0008) [2023-10-10 20:17:36,472][123582] Updated weights for policy 0, policy_version 94993 (0.0008) [2023-10-10 20:17:36,846][123582] Updated weights for policy 0, policy_version 95003 (0.0008) [2023-10-10 20:17:38,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 194445312. Throughput: 0: 1813.9, 1: 1806.6. Samples: 48625228. Policy #0 lag: (min: 9.0, avg: 36.8, max: 40.0) [2023-10-10 20:17:38,789][122664] Avg episode reward: [(0, '71.120'), (1, '76.540')] [2023-10-10 20:17:39,208][123614] Updated weights for policy 1, policy_version 94890 (0.0007) [2023-10-10 20:17:39,574][123614] Updated weights for policy 1, policy_version 94900 (0.0008) [2023-10-10 20:17:39,938][123614] Updated weights for policy 1, policy_version 94910 (0.0010) [2023-10-10 20:17:40,439][123582] Updated weights for policy 0, policy_version 95013 (0.0008) [2023-10-10 20:17:40,818][123582] Updated weights for policy 0, policy_version 95023 (0.0008) [2023-10-10 20:17:41,189][123582] Updated weights for policy 0, policy_version 95033 (0.0007) [2023-10-10 20:17:43,734][123614] Updated weights for policy 1, policy_version 94920 (0.0009) [2023-10-10 20:17:43,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 194510848. Throughput: 0: 1820.7, 1: 1808.3. Samples: 48635480. Policy #0 lag: (min: 9.0, avg: 36.8, max: 40.0) [2023-10-10 20:17:43,789][122664] Avg episode reward: [(0, '70.920'), (1, '73.700')] [2023-10-10 20:17:44,109][123614] Updated weights for policy 1, policy_version 94930 (0.0008) [2023-10-10 20:17:44,472][123614] Updated weights for policy 1, policy_version 94940 (0.0008) [2023-10-10 20:17:44,954][123582] Updated weights for policy 0, policy_version 95043 (0.0007) [2023-10-10 20:17:45,321][123582] Updated weights for policy 0, policy_version 95053 (0.0008) [2023-10-10 20:17:45,697][123582] Updated weights for policy 0, policy_version 95063 (0.0009) [2023-10-10 20:17:48,250][123614] Updated weights for policy 1, policy_version 94950 (0.0010) [2023-10-10 20:17:48,621][123614] Updated weights for policy 1, policy_version 94960 (0.0007) [2023-10-10 20:17:48,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 194576384. Throughput: 0: 1825.1, 1: 1814.5. Samples: 48658370. Policy #0 lag: (min: 9.0, avg: 36.8, max: 40.0) [2023-10-10 20:17:48,789][122664] Avg episode reward: [(0, '78.780'), (1, '74.630')] [2023-10-10 20:17:48,995][123614] Updated weights for policy 1, policy_version 94970 (0.0008) [2023-10-10 20:17:49,230][123582] Updated weights for policy 0, policy_version 95073 (0.0007) [2023-10-10 20:17:49,597][123582] Updated weights for policy 0, policy_version 95083 (0.0009) [2023-10-10 20:17:49,977][123582] Updated weights for policy 0, policy_version 95093 (0.0009) [2023-10-10 20:17:50,349][123582] Updated weights for policy 0, policy_version 95103 (0.0008) [2023-10-10 20:17:52,687][123614] Updated weights for policy 1, policy_version 94980 (0.0008) [2023-10-10 20:17:53,063][123614] Updated weights for policy 1, policy_version 94990 (0.0009) [2023-10-10 20:17:53,438][123614] Updated weights for policy 1, policy_version 95000 (0.0008) [2023-10-10 20:17:53,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 194674688. Throughput: 0: 1824.5, 1: 1810.1. Samples: 48679646. Policy #0 lag: (min: 9.0, avg: 36.8, max: 40.0) [2023-10-10 20:17:53,789][122664] Avg episode reward: [(0, '78.400'), (1, '74.850')] [2023-10-10 20:17:53,800][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000095008_97288192.pth... [2023-10-10 20:17:53,835][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000093312_95551488.pth [2023-10-10 20:17:54,084][123582] Updated weights for policy 0, policy_version 95113 (0.0010) [2023-10-10 20:17:54,447][123582] Updated weights for policy 0, policy_version 95123 (0.0011) [2023-10-10 20:17:54,819][123582] Updated weights for policy 0, policy_version 95133 (0.0007) [2023-10-10 20:17:54,934][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000095136_97419264.pth... [2023-10-10 20:17:54,975][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000093408_95649792.pth [2023-10-10 20:17:57,288][123614] Updated weights for policy 1, policy_version 95010 (0.0008) [2023-10-10 20:17:57,657][123614] Updated weights for policy 1, policy_version 95020 (0.0010) [2023-10-10 20:17:58,023][123614] Updated weights for policy 1, policy_version 95030 (0.0009) [2023-10-10 20:17:58,397][123614] Updated weights for policy 1, policy_version 95040 (0.0008) [2023-10-10 20:17:58,631][123582] Updated weights for policy 0, policy_version 95143 (0.0009) [2023-10-10 20:17:58,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 194740224. Throughput: 0: 1819.8, 1: 1807.8. Samples: 48690570. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 20:17:58,789][122664] Avg episode reward: [(0, '79.880'), (1, '75.170')] [2023-10-10 20:17:59,013][123582] Updated weights for policy 0, policy_version 95153 (0.0011) [2023-10-10 20:17:59,374][123582] Updated weights for policy 0, policy_version 95163 (0.0010) [2023-10-10 20:18:02,241][123614] Updated weights for policy 1, policy_version 95050 (0.0009) [2023-10-10 20:18:02,598][123614] Updated weights for policy 1, policy_version 95060 (0.0009) [2023-10-10 20:18:02,959][123614] Updated weights for policy 1, policy_version 95070 (0.0010) [2023-10-10 20:18:03,168][123582] Updated weights for policy 0, policy_version 95173 (0.0009) [2023-10-10 20:18:03,546][123582] Updated weights for policy 0, policy_version 95183 (0.0008) [2023-10-10 20:18:03,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 194805760. Throughput: 0: 1813.8, 1: 1811.9. Samples: 48711808. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 20:18:03,788][122664] Avg episode reward: [(0, '80.670'), (1, '77.320')] [2023-10-10 20:18:03,913][123582] Updated weights for policy 0, policy_version 95193 (0.0007) [2023-10-10 20:18:06,511][123614] Updated weights for policy 1, policy_version 95080 (0.0010) [2023-10-10 20:18:06,881][123614] Updated weights for policy 1, policy_version 95090 (0.0010) [2023-10-10 20:18:07,250][123614] Updated weights for policy 1, policy_version 95100 (0.0009) [2023-10-10 20:18:07,522][123582] Updated weights for policy 0, policy_version 95203 (0.0008) [2023-10-10 20:18:07,891][123582] Updated weights for policy 0, policy_version 95213 (0.0010) [2023-10-10 20:18:08,261][123582] Updated weights for policy 0, policy_version 95223 (0.0010) [2023-10-10 20:18:08,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 194904064. Throughput: 0: 1821.7, 1: 1807.7. Samples: 48733256. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 20:18:08,789][122664] Avg episode reward: [(0, '76.260'), (1, '77.160')] [2023-10-10 20:18:10,721][123614] Updated weights for policy 1, policy_version 95110 (0.0007) [2023-10-10 20:18:11,095][123614] Updated weights for policy 1, policy_version 95120 (0.0010) [2023-10-10 20:18:11,459][123614] Updated weights for policy 1, policy_version 95130 (0.0007) [2023-10-10 20:18:11,918][123582] Updated weights for policy 0, policy_version 95233 (0.0009) [2023-10-10 20:18:12,284][123582] Updated weights for policy 0, policy_version 95243 (0.0007) [2023-10-10 20:18:12,644][123582] Updated weights for policy 0, policy_version 95253 (0.0008) [2023-10-10 20:18:13,015][123582] Updated weights for policy 0, policy_version 95263 (0.0009) [2023-10-10 20:18:13,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 194969600. Throughput: 0: 1814.6, 1: 1811.8. Samples: 48744324. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 20:18:13,789][122664] Avg episode reward: [(0, '76.280'), (1, '78.580')] [2023-10-10 20:18:15,003][123614] Updated weights for policy 1, policy_version 95140 (0.0008) [2023-10-10 20:18:15,373][123614] Updated weights for policy 1, policy_version 95150 (0.0008) [2023-10-10 20:18:15,742][123614] Updated weights for policy 1, policy_version 95160 (0.0011) [2023-10-10 20:18:16,721][123582] Updated weights for policy 0, policy_version 95273 (0.0008) [2023-10-10 20:18:17,085][123582] Updated weights for policy 0, policy_version 95283 (0.0009) [2023-10-10 20:18:17,461][123582] Updated weights for policy 0, policy_version 95293 (0.0007) [2023-10-10 20:18:18,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 195035136. Throughput: 0: 1814.6, 1: 1813.5. Samples: 48766044. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 20:18:18,789][122664] Avg episode reward: [(0, '74.520'), (1, '80.240')] [2023-10-10 20:18:19,553][123614] Updated weights for policy 1, policy_version 95170 (0.0009) [2023-10-10 20:18:19,953][123614] Updated weights for policy 1, policy_version 95180 (0.0007) [2023-10-10 20:18:20,312][123614] Updated weights for policy 1, policy_version 95190 (0.0007) [2023-10-10 20:18:20,677][123614] Updated weights for policy 1, policy_version 95200 (0.0008) [2023-10-10 20:18:21,193][123582] Updated weights for policy 0, policy_version 95303 (0.0010) [2023-10-10 20:18:21,568][123582] Updated weights for policy 0, policy_version 95313 (0.0008) [2023-10-10 20:18:21,941][123582] Updated weights for policy 0, policy_version 95323 (0.0007) [2023-10-10 20:18:23,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 195100672. Throughput: 0: 1812.0, 1: 1811.9. Samples: 48788302. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 20:18:23,788][122664] Avg episode reward: [(0, '71.900'), (1, '80.110')] [2023-10-10 20:18:24,420][123614] Updated weights for policy 1, policy_version 95210 (0.0011) [2023-10-10 20:18:24,791][123614] Updated weights for policy 1, policy_version 95220 (0.0011) [2023-10-10 20:18:25,156][123614] Updated weights for policy 1, policy_version 95230 (0.0007) [2023-10-10 20:18:25,698][123582] Updated weights for policy 0, policy_version 95333 (0.0007) [2023-10-10 20:18:26,078][123582] Updated weights for policy 0, policy_version 95343 (0.0008) [2023-10-10 20:18:26,448][123582] Updated weights for policy 0, policy_version 95353 (0.0009) [2023-10-10 20:18:28,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 195166208. Throughput: 0: 1814.6, 1: 1810.9. Samples: 48798626. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 20:18:28,789][122664] Avg episode reward: [(0, '76.400'), (1, '80.420')] [2023-10-10 20:18:28,853][123614] Updated weights for policy 1, policy_version 95240 (0.0009) [2023-10-10 20:18:29,218][123614] Updated weights for policy 1, policy_version 95250 (0.0009) [2023-10-10 20:18:29,574][123614] Updated weights for policy 1, policy_version 95260 (0.0007) [2023-10-10 20:18:30,159][123582] Updated weights for policy 0, policy_version 95363 (0.0008) [2023-10-10 20:18:30,529][123582] Updated weights for policy 0, policy_version 95373 (0.0011) [2023-10-10 20:18:30,911][123582] Updated weights for policy 0, policy_version 95383 (0.0010) [2023-10-10 20:18:33,142][123614] Updated weights for policy 1, policy_version 95270 (0.0009) [2023-10-10 20:18:33,509][123614] Updated weights for policy 1, policy_version 95280 (0.0007) [2023-10-10 20:18:33,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 195231744. Throughput: 0: 1807.7, 1: 1809.7. Samples: 48821156. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 20:18:33,788][122664] Avg episode reward: [(0, '74.810'), (1, '79.440')] [2023-10-10 20:18:33,871][123614] Updated weights for policy 1, policy_version 95290 (0.0007) [2023-10-10 20:18:34,583][123582] Updated weights for policy 0, policy_version 95393 (0.0008) [2023-10-10 20:18:34,947][123582] Updated weights for policy 0, policy_version 95403 (0.0008) [2023-10-10 20:18:35,315][123582] Updated weights for policy 0, policy_version 95413 (0.0009) [2023-10-10 20:18:35,685][123582] Updated weights for policy 0, policy_version 95423 (0.0011) [2023-10-10 20:18:37,741][123614] Updated weights for policy 1, policy_version 95300 (0.0007) [2023-10-10 20:18:38,120][123614] Updated weights for policy 1, policy_version 95310 (0.0007) [2023-10-10 20:18:38,482][123614] Updated weights for policy 1, policy_version 95320 (0.0008) [2023-10-10 20:18:38,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 195330048. Throughput: 0: 1811.1, 1: 1815.5. Samples: 48842840. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 20:18:38,788][122664] Avg episode reward: [(0, '74.450'), (1, '80.640')] [2023-10-10 20:18:39,299][123582] Updated weights for policy 0, policy_version 95433 (0.0009) [2023-10-10 20:18:39,669][123582] Updated weights for policy 0, policy_version 95443 (0.0008) [2023-10-10 20:18:40,040][123582] Updated weights for policy 0, policy_version 95453 (0.0009) [2023-10-10 20:18:42,228][123614] Updated weights for policy 1, policy_version 95330 (0.0007) [2023-10-10 20:18:42,593][123614] Updated weights for policy 1, policy_version 95340 (0.0009) [2023-10-10 20:18:42,956][123614] Updated weights for policy 1, policy_version 95350 (0.0008) [2023-10-10 20:18:43,323][123614] Updated weights for policy 1, policy_version 95360 (0.0008) [2023-10-10 20:18:43,721][123582] Updated weights for policy 0, policy_version 95463 (0.0007) [2023-10-10 20:18:43,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 195395584. Throughput: 0: 1820.2, 1: 1816.2. Samples: 48854208. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 20:18:43,789][122664] Avg episode reward: [(0, '75.700'), (1, '78.390')] [2023-10-10 20:18:44,090][123582] Updated weights for policy 0, policy_version 95473 (0.0007) [2023-10-10 20:18:44,465][123582] Updated weights for policy 0, policy_version 95483 (0.0008) [2023-10-10 20:18:46,990][123614] Updated weights for policy 1, policy_version 95370 (0.0007) [2023-10-10 20:18:47,359][123614] Updated weights for policy 1, policy_version 95380 (0.0008) [2023-10-10 20:18:47,720][123614] Updated weights for policy 1, policy_version 95390 (0.0010) [2023-10-10 20:18:48,158][123582] Updated weights for policy 0, policy_version 95493 (0.0008) [2023-10-10 20:18:48,532][123582] Updated weights for policy 0, policy_version 95503 (0.0007) [2023-10-10 20:18:48,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 195461120. Throughput: 0: 1825.9, 1: 1816.7. Samples: 48875724. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 20:18:48,789][122664] Avg episode reward: [(0, '84.160'), (1, '75.580')] [2023-10-10 20:18:48,896][123582] Updated weights for policy 0, policy_version 95513 (0.0008) [2023-10-10 20:18:51,386][123614] Updated weights for policy 1, policy_version 95400 (0.0010) [2023-10-10 20:18:51,755][123614] Updated weights for policy 1, policy_version 95410 (0.0010) [2023-10-10 20:18:52,123][123614] Updated weights for policy 1, policy_version 95420 (0.0007) [2023-10-10 20:18:52,693][123582] Updated weights for policy 0, policy_version 95523 (0.0010) [2023-10-10 20:18:53,057][123582] Updated weights for policy 0, policy_version 95533 (0.0007) [2023-10-10 20:18:53,426][123582] Updated weights for policy 0, policy_version 95543 (0.0007) [2023-10-10 20:18:53,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 195559424. Throughput: 0: 1822.8, 1: 1821.0. Samples: 48897226. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-10 20:18:53,788][122664] Avg episode reward: [(0, '91.420'), (1, '78.130')] [2023-10-10 20:18:55,804][123614] Updated weights for policy 1, policy_version 95430 (0.0009) [2023-10-10 20:18:56,170][123614] Updated weights for policy 1, policy_version 95440 (0.0010) [2023-10-10 20:18:56,534][123614] Updated weights for policy 1, policy_version 95450 (0.0010) [2023-10-10 20:18:57,057][123582] Updated weights for policy 0, policy_version 95553 (0.0009) [2023-10-10 20:18:57,429][123582] Updated weights for policy 0, policy_version 95563 (0.0007) [2023-10-10 20:18:57,792][123582] Updated weights for policy 0, policy_version 95573 (0.0007) [2023-10-10 20:18:58,163][123582] Updated weights for policy 0, policy_version 95583 (0.0008) [2023-10-10 20:18:58,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 195624960. Throughput: 0: 1821.1, 1: 1820.1. Samples: 48908178. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-10 20:18:58,789][122664] Avg episode reward: [(0, '92.200'), (1, '79.300')] [2023-10-10 20:19:00,234][123614] Updated weights for policy 1, policy_version 95460 (0.0008) [2023-10-10 20:19:00,600][123614] Updated weights for policy 1, policy_version 95470 (0.0008) [2023-10-10 20:19:00,968][123614] Updated weights for policy 1, policy_version 95480 (0.0012) [2023-10-10 20:19:01,919][123582] Updated weights for policy 0, policy_version 95593 (0.0008) [2023-10-10 20:19:02,290][123582] Updated weights for policy 0, policy_version 95603 (0.0008) [2023-10-10 20:19:02,655][123582] Updated weights for policy 0, policy_version 95613 (0.0009) [2023-10-10 20:19:03,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 195690496. Throughput: 0: 1820.5, 1: 1817.9. Samples: 48929770. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-10 20:19:03,789][122664] Avg episode reward: [(0, '93.960'), (1, '80.220')] [2023-10-10 20:19:04,724][123614] Updated weights for policy 1, policy_version 95490 (0.0009) [2023-10-10 20:19:05,122][123614] Updated weights for policy 1, policy_version 95500 (0.0011) [2023-10-10 20:19:05,487][123614] Updated weights for policy 1, policy_version 95510 (0.0010) [2023-10-10 20:19:05,857][123614] Updated weights for policy 1, policy_version 95520 (0.0009) [2023-10-10 20:19:06,162][123582] Updated weights for policy 0, policy_version 95623 (0.0009) [2023-10-10 20:19:06,533][123582] Updated weights for policy 0, policy_version 95633 (0.0007) [2023-10-10 20:19:06,908][123582] Updated weights for policy 0, policy_version 95643 (0.0009) [2023-10-10 20:19:08,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 195756032. Throughput: 0: 1821.5, 1: 1820.0. Samples: 48952172. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-10 20:19:08,789][122664] Avg episode reward: [(0, '95.170'), (1, '81.360')] [2023-10-10 20:19:09,571][123614] Updated weights for policy 1, policy_version 95530 (0.0009) [2023-10-10 20:19:09,939][123614] Updated weights for policy 1, policy_version 95540 (0.0008) [2023-10-10 20:19:10,305][123614] Updated weights for policy 1, policy_version 95550 (0.0011) [2023-10-10 20:19:10,713][123582] Updated weights for policy 0, policy_version 95653 (0.0009) [2023-10-10 20:19:11,092][123582] Updated weights for policy 0, policy_version 95663 (0.0008) [2023-10-10 20:19:11,452][123582] Updated weights for policy 0, policy_version 95673 (0.0007) [2023-10-10 20:19:13,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 195821568. Throughput: 0: 1820.7, 1: 1819.6. Samples: 48962438. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-10 20:19:13,789][122664] Avg episode reward: [(0, '90.000'), (1, '83.880')] [2023-10-10 20:19:14,001][123614] Updated weights for policy 1, policy_version 95560 (0.0009) [2023-10-10 20:19:14,369][123614] Updated weights for policy 1, policy_version 95570 (0.0008) [2023-10-10 20:19:14,739][123614] Updated weights for policy 1, policy_version 95580 (0.0011) [2023-10-10 20:19:15,072][123582] Updated weights for policy 0, policy_version 95683 (0.0008) [2023-10-10 20:19:15,445][123582] Updated weights for policy 0, policy_version 95693 (0.0007) [2023-10-10 20:19:15,808][123582] Updated weights for policy 0, policy_version 95703 (0.0010) [2023-10-10 20:19:18,446][123614] Updated weights for policy 1, policy_version 95590 (0.0010) [2023-10-10 20:19:18,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 195887104. Throughput: 0: 1823.0, 1: 1820.2. Samples: 48985100. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-10 20:19:18,789][122664] Avg episode reward: [(0, '91.410'), (1, '81.400')] [2023-10-10 20:19:18,818][123614] Updated weights for policy 1, policy_version 95600 (0.0007) [2023-10-10 20:19:19,193][123614] Updated weights for policy 1, policy_version 95610 (0.0007) [2023-10-10 20:19:19,515][123582] Updated weights for policy 0, policy_version 95713 (0.0007) [2023-10-10 20:19:19,882][123582] Updated weights for policy 0, policy_version 95723 (0.0010) [2023-10-10 20:19:20,262][123582] Updated weights for policy 0, policy_version 95733 (0.0007) [2023-10-10 20:19:20,632][123582] Updated weights for policy 0, policy_version 95743 (0.0009) [2023-10-10 20:19:22,840][123614] Updated weights for policy 1, policy_version 95620 (0.0010) [2023-10-10 20:19:23,213][123614] Updated weights for policy 1, policy_version 95630 (0.0009) [2023-10-10 20:19:23,577][123614] Updated weights for policy 1, policy_version 95640 (0.0007) [2023-10-10 20:19:23,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 195952640. Throughput: 0: 1812.8, 1: 1819.6. Samples: 49006298. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-10 20:19:23,788][122664] Avg episode reward: [(0, '93.620'), (1, '85.770')] [2023-10-10 20:19:24,392][123582] Updated weights for policy 0, policy_version 95753 (0.0010) [2023-10-10 20:19:24,768][123582] Updated weights for policy 0, policy_version 95763 (0.0008) [2023-10-10 20:19:25,145][123582] Updated weights for policy 0, policy_version 95773 (0.0008) [2023-10-10 20:19:27,184][123614] Updated weights for policy 1, policy_version 95650 (0.0009) [2023-10-10 20:19:27,568][123614] Updated weights for policy 1, policy_version 95660 (0.0008) [2023-10-10 20:19:27,937][123614] Updated weights for policy 1, policy_version 95670 (0.0008) [2023-10-10 20:19:28,303][123614] Updated weights for policy 1, policy_version 95680 (0.0008) [2023-10-10 20:19:28,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 196050944. Throughput: 0: 1806.2, 1: 1824.4. Samples: 49017584. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-10 20:19:28,788][122664] Avg episode reward: [(0, '90.710'), (1, '80.920')] [2023-10-10 20:19:28,997][123582] Updated weights for policy 0, policy_version 95783 (0.0010) [2023-10-10 20:19:29,366][123582] Updated weights for policy 0, policy_version 95793 (0.0009) [2023-10-10 20:19:29,734][123582] Updated weights for policy 0, policy_version 95803 (0.0008) [2023-10-10 20:19:32,086][123614] Updated weights for policy 1, policy_version 95690 (0.0011) [2023-10-10 20:19:32,449][123614] Updated weights for policy 1, policy_version 95700 (0.0010) [2023-10-10 20:19:32,816][123614] Updated weights for policy 1, policy_version 95710 (0.0010) [2023-10-10 20:19:33,348][123582] Updated weights for policy 0, policy_version 95813 (0.0009) [2023-10-10 20:19:33,712][123582] Updated weights for policy 0, policy_version 95823 (0.0008) [2023-10-10 20:19:33,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 196116480. Throughput: 0: 1807.3, 1: 1820.7. Samples: 49038986. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-10 20:19:33,789][122664] Avg episode reward: [(0, '91.200'), (1, '83.630')] [2023-10-10 20:19:34,079][123582] Updated weights for policy 0, policy_version 95833 (0.0008) [2023-10-10 20:19:36,534][123614] Updated weights for policy 1, policy_version 95720 (0.0009) [2023-10-10 20:19:36,908][123614] Updated weights for policy 1, policy_version 95730 (0.0009) [2023-10-10 20:19:37,270][123614] Updated weights for policy 1, policy_version 95740 (0.0009) [2023-10-10 20:19:37,673][123582] Updated weights for policy 0, policy_version 95843 (0.0009) [2023-10-10 20:19:38,050][123582] Updated weights for policy 0, policy_version 95853 (0.0011) [2023-10-10 20:19:38,418][123582] Updated weights for policy 0, policy_version 95863 (0.0012) [2023-10-10 20:19:38,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 196214784. Throughput: 0: 1815.3, 1: 1821.0. Samples: 49060858. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-10 20:19:38,788][122664] Avg episode reward: [(0, '99.990'), (1, '83.890')] [2023-10-10 20:19:40,961][123614] Updated weights for policy 1, policy_version 95750 (0.0009) [2023-10-10 20:19:41,324][123614] Updated weights for policy 1, policy_version 95760 (0.0007) [2023-10-10 20:19:41,698][123614] Updated weights for policy 1, policy_version 95770 (0.0009) [2023-10-10 20:19:42,043][123582] Updated weights for policy 0, policy_version 95873 (0.0010) [2023-10-10 20:19:42,417][123582] Updated weights for policy 0, policy_version 95883 (0.0008) [2023-10-10 20:19:42,792][123582] Updated weights for policy 0, policy_version 95893 (0.0009) [2023-10-10 20:19:43,166][123582] Updated weights for policy 0, policy_version 95903 (0.0009) [2023-10-10 20:19:43,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 196280320. Throughput: 0: 1815.5, 1: 1823.8. Samples: 49071944. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-10 20:19:43,789][122664] Avg episode reward: [(0, '104.420'), (1, '82.410')] [2023-10-10 20:19:45,278][123614] Updated weights for policy 1, policy_version 95780 (0.0008) [2023-10-10 20:19:45,657][123614] Updated weights for policy 1, policy_version 95790 (0.0008) [2023-10-10 20:19:46,020][123614] Updated weights for policy 1, policy_version 95800 (0.0007) [2023-10-10 20:19:46,887][123582] Updated weights for policy 0, policy_version 95913 (0.0007) [2023-10-10 20:19:47,254][123582] Updated weights for policy 0, policy_version 95923 (0.0008) [2023-10-10 20:19:47,633][123582] Updated weights for policy 0, policy_version 95933 (0.0008) [2023-10-10 20:19:48,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 196345856. Throughput: 0: 1822.2, 1: 1826.6. Samples: 49093968. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-10 20:19:48,788][122664] Avg episode reward: [(0, '105.930'), (1, '83.120')] [2023-10-10 20:19:49,854][123614] Updated weights for policy 1, policy_version 95810 (0.0007) [2023-10-10 20:19:50,225][123614] Updated weights for policy 1, policy_version 95820 (0.0007) [2023-10-10 20:19:50,597][123614] Updated weights for policy 1, policy_version 95830 (0.0007) [2023-10-10 20:19:50,959][123614] Updated weights for policy 1, policy_version 95840 (0.0008) [2023-10-10 20:19:51,278][123582] Updated weights for policy 0, policy_version 95943 (0.0008) [2023-10-10 20:19:51,648][123582] Updated weights for policy 0, policy_version 95953 (0.0010) [2023-10-10 20:19:52,029][123582] Updated weights for policy 0, policy_version 95963 (0.0008) [2023-10-10 20:19:53,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 196411392. Throughput: 0: 1813.5, 1: 1828.2. Samples: 49116050. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-10 20:19:53,790][122664] Avg episode reward: [(0, '110.200'), (1, '88.150')] [2023-10-10 20:19:53,800][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000095840_98140160.pth... [2023-10-10 20:19:53,801][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000095968_98271232.pth... [2023-10-10 20:19:53,831][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000094144_96403456.pth [2023-10-10 20:19:53,839][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000094272_96534528.pth [2023-10-10 20:19:54,321][123614] Updated weights for policy 1, policy_version 95850 (0.0008) [2023-10-10 20:19:54,696][123614] Updated weights for policy 1, policy_version 95860 (0.0009) [2023-10-10 20:19:55,056][123614] Updated weights for policy 1, policy_version 95870 (0.0008) [2023-10-10 20:19:55,808][123582] Updated weights for policy 0, policy_version 95973 (0.0010) [2023-10-10 20:19:56,188][123582] Updated weights for policy 0, policy_version 95983 (0.0009) [2023-10-10 20:19:56,561][123582] Updated weights for policy 0, policy_version 95993 (0.0008) [2023-10-10 20:19:58,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 196476928. Throughput: 0: 1817.8, 1: 1825.8. Samples: 49126400. Policy #0 lag: (min: 25.0, avg: 29.3, max: 57.0) [2023-10-10 20:19:58,788][122664] Avg episode reward: [(0, '109.470'), (1, '91.960')] [2023-10-10 20:19:58,910][123614] Updated weights for policy 1, policy_version 95880 (0.0008) [2023-10-10 20:19:59,283][123614] Updated weights for policy 1, policy_version 95890 (0.0008) [2023-10-10 20:19:59,644][123614] Updated weights for policy 1, policy_version 95900 (0.0008) [2023-10-10 20:20:00,333][123582] Updated weights for policy 0, policy_version 96003 (0.0009) [2023-10-10 20:20:00,704][123582] Updated weights for policy 0, policy_version 96013 (0.0011) [2023-10-10 20:20:01,078][123582] Updated weights for policy 0, policy_version 96023 (0.0010) [2023-10-10 20:20:03,168][123614] Updated weights for policy 1, policy_version 95910 (0.0009) [2023-10-10 20:20:03,536][123614] Updated weights for policy 1, policy_version 95920 (0.0010) [2023-10-10 20:20:03,788][122664] Fps is (10 sec: 13107.8, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 196542464. Throughput: 0: 1813.4, 1: 1820.1. Samples: 49148604. Policy #0 lag: (min: 25.0, avg: 29.3, max: 57.0) [2023-10-10 20:20:03,788][122664] Avg episode reward: [(0, '111.340'), (1, '87.000')] [2023-10-10 20:20:03,900][123614] Updated weights for policy 1, policy_version 95930 (0.0008) [2023-10-10 20:20:04,599][123582] Updated weights for policy 0, policy_version 96033 (0.0007) [2023-10-10 20:20:04,972][123582] Updated weights for policy 0, policy_version 96043 (0.0007) [2023-10-10 20:20:05,335][123582] Updated weights for policy 0, policy_version 96053 (0.0007) [2023-10-10 20:20:05,702][123582] Updated weights for policy 0, policy_version 96063 (0.0011) [2023-10-10 20:20:07,690][123614] Updated weights for policy 1, policy_version 95940 (0.0009) [2023-10-10 20:20:08,055][123614] Updated weights for policy 1, policy_version 95950 (0.0008) [2023-10-10 20:20:08,424][123614] Updated weights for policy 1, policy_version 95960 (0.0008) [2023-10-10 20:20:08,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 196640768. Throughput: 0: 1825.9, 1: 1814.0. Samples: 49170096. Policy #0 lag: (min: 25.0, avg: 29.3, max: 57.0) [2023-10-10 20:20:08,788][122664] Avg episode reward: [(0, '110.190'), (1, '88.300')] [2023-10-10 20:20:09,513][123582] Updated weights for policy 0, policy_version 96073 (0.0009) [2023-10-10 20:20:09,879][123582] Updated weights for policy 0, policy_version 96083 (0.0008) [2023-10-10 20:20:10,259][123582] Updated weights for policy 0, policy_version 96093 (0.0009) [2023-10-10 20:20:12,217][123614] Updated weights for policy 1, policy_version 95970 (0.0007) [2023-10-10 20:20:12,585][123614] Updated weights for policy 1, policy_version 95980 (0.0008) [2023-10-10 20:20:12,953][123614] Updated weights for policy 1, policy_version 95990 (0.0010) [2023-10-10 20:20:13,325][123614] Updated weights for policy 1, policy_version 96000 (0.0009) [2023-10-10 20:20:13,788][122664] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 196706304. Throughput: 0: 1826.1, 1: 1810.3. Samples: 49181220. Policy #0 lag: (min: 25.0, avg: 29.3, max: 57.0) [2023-10-10 20:20:13,789][122664] Avg episode reward: [(0, '110.140'), (1, '90.540')] [2023-10-10 20:20:14,048][123582] Updated weights for policy 0, policy_version 96103 (0.0007) [2023-10-10 20:20:14,419][123582] Updated weights for policy 0, policy_version 96113 (0.0008) [2023-10-10 20:20:14,796][123582] Updated weights for policy 0, policy_version 96123 (0.0007) [2023-10-10 20:20:17,016][123614] Updated weights for policy 1, policy_version 96010 (0.0009) [2023-10-10 20:20:17,381][123614] Updated weights for policy 1, policy_version 96020 (0.0008) [2023-10-10 20:20:17,746][123614] Updated weights for policy 1, policy_version 96030 (0.0009) [2023-10-10 20:20:18,532][123582] Updated weights for policy 0, policy_version 96133 (0.0009) [2023-10-10 20:20:18,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 196771840. Throughput: 0: 1825.7, 1: 1812.4. Samples: 49202700. Policy #0 lag: (min: 25.0, avg: 29.3, max: 57.0) [2023-10-10 20:20:18,789][122664] Avg episode reward: [(0, '107.350'), (1, '92.450')] [2023-10-10 20:20:18,906][123582] Updated weights for policy 0, policy_version 96143 (0.0010) [2023-10-10 20:20:19,276][123582] Updated weights for policy 0, policy_version 96153 (0.0010) [2023-10-10 20:20:21,422][123614] Updated weights for policy 1, policy_version 96040 (0.0008) [2023-10-10 20:20:21,786][123614] Updated weights for policy 1, policy_version 96050 (0.0008) [2023-10-10 20:20:22,154][123614] Updated weights for policy 1, policy_version 96060 (0.0009) [2023-10-10 20:20:22,941][123582] Updated weights for policy 0, policy_version 96163 (0.0010) [2023-10-10 20:20:23,310][123582] Updated weights for policy 0, policy_version 96173 (0.0008) [2023-10-10 20:20:23,675][123582] Updated weights for policy 0, policy_version 96183 (0.0009) [2023-10-10 20:20:23,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 196837376. Throughput: 0: 1822.3, 1: 1808.4. Samples: 49224238. Policy #0 lag: (min: 25.0, avg: 29.3, max: 57.0) [2023-10-10 20:20:23,788][122664] Avg episode reward: [(0, '100.620'), (1, '95.550')] [2023-10-10 20:20:25,889][123614] Updated weights for policy 1, policy_version 96070 (0.0007) [2023-10-10 20:20:26,266][123614] Updated weights for policy 1, policy_version 96080 (0.0008) [2023-10-10 20:20:26,645][123614] Updated weights for policy 1, policy_version 96090 (0.0009) [2023-10-10 20:20:27,189][123582] Updated weights for policy 0, policy_version 96193 (0.0011) [2023-10-10 20:20:27,564][123582] Updated weights for policy 0, policy_version 96203 (0.0009) [2023-10-10 20:20:27,934][123582] Updated weights for policy 0, policy_version 96213 (0.0008) [2023-10-10 20:20:28,301][123582] Updated weights for policy 0, policy_version 96223 (0.0007) [2023-10-10 20:20:28,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 196935680. Throughput: 0: 1814.0, 1: 1810.6. Samples: 49235048. Policy #0 lag: (min: 25.0, avg: 29.3, max: 57.0) [2023-10-10 20:20:28,789][122664] Avg episode reward: [(0, '99.540'), (1, '96.120')] [2023-10-10 20:20:30,424][123614] Updated weights for policy 1, policy_version 96100 (0.0010) [2023-10-10 20:20:30,792][123614] Updated weights for policy 1, policy_version 96110 (0.0010) [2023-10-10 20:20:31,155][123614] Updated weights for policy 1, policy_version 96120 (0.0008) [2023-10-10 20:20:31,915][123582] Updated weights for policy 0, policy_version 96233 (0.0009) [2023-10-10 20:20:32,274][123582] Updated weights for policy 0, policy_version 96243 (0.0008) [2023-10-10 20:20:32,657][123582] Updated weights for policy 0, policy_version 96253 (0.0008) [2023-10-10 20:20:33,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 197001216. Throughput: 0: 1813.4, 1: 1800.1. Samples: 49256576. Policy #0 lag: (min: 25.0, avg: 29.3, max: 57.0) [2023-10-10 20:20:33,789][122664] Avg episode reward: [(0, '98.870'), (1, '95.720')] [2023-10-10 20:20:34,828][123614] Updated weights for policy 1, policy_version 96130 (0.0008) [2023-10-10 20:20:35,223][123614] Updated weights for policy 1, policy_version 96140 (0.0008) [2023-10-10 20:20:35,597][123614] Updated weights for policy 1, policy_version 96150 (0.0007) [2023-10-10 20:20:35,960][123614] Updated weights for policy 1, policy_version 96160 (0.0008) [2023-10-10 20:20:36,462][123582] Updated weights for policy 0, policy_version 96263 (0.0007) [2023-10-10 20:20:36,837][123582] Updated weights for policy 0, policy_version 96273 (0.0007) [2023-10-10 20:20:37,214][123582] Updated weights for policy 0, policy_version 96283 (0.0010) [2023-10-10 20:20:38,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 197066752. Throughput: 0: 1821.1, 1: 1798.5. Samples: 49278932. Policy #0 lag: (min: 25.0, avg: 29.3, max: 57.0) [2023-10-10 20:20:38,789][122664] Avg episode reward: [(0, '95.820'), (1, '95.950')] [2023-10-10 20:20:39,725][123614] Updated weights for policy 1, policy_version 96170 (0.0009) [2023-10-10 20:20:40,090][123614] Updated weights for policy 1, policy_version 96180 (0.0009) [2023-10-10 20:20:40,458][123614] Updated weights for policy 1, policy_version 96190 (0.0007) [2023-10-10 20:20:40,862][123582] Updated weights for policy 0, policy_version 96293 (0.0007) [2023-10-10 20:20:41,229][123582] Updated weights for policy 0, policy_version 96303 (0.0009) [2023-10-10 20:20:41,605][123582] Updated weights for policy 0, policy_version 96313 (0.0008) [2023-10-10 20:20:43,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 197132288. Throughput: 0: 1821.8, 1: 1800.1. Samples: 49289388. Policy #0 lag: (min: 25.0, avg: 29.3, max: 57.0) [2023-10-10 20:20:43,788][122664] Avg episode reward: [(0, '93.300'), (1, '92.850')] [2023-10-10 20:20:44,128][123614] Updated weights for policy 1, policy_version 96200 (0.0010) [2023-10-10 20:20:44,491][123614] Updated weights for policy 1, policy_version 96210 (0.0008) [2023-10-10 20:20:44,856][123614] Updated weights for policy 1, policy_version 96220 (0.0007) [2023-10-10 20:20:45,266][123582] Updated weights for policy 0, policy_version 96323 (0.0009) [2023-10-10 20:20:45,635][123582] Updated weights for policy 0, policy_version 96333 (0.0007) [2023-10-10 20:20:45,996][123582] Updated weights for policy 0, policy_version 96343 (0.0007) [2023-10-10 20:20:48,573][123614] Updated weights for policy 1, policy_version 96230 (0.0008) [2023-10-10 20:20:48,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 197197824. Throughput: 0: 1822.8, 1: 1803.3. Samples: 49311780. Policy #0 lag: (min: 25.0, avg: 29.3, max: 57.0) [2023-10-10 20:20:48,789][122664] Avg episode reward: [(0, '93.300'), (1, '94.190')] [2023-10-10 20:20:48,952][123614] Updated weights for policy 1, policy_version 96240 (0.0008) [2023-10-10 20:20:49,316][123614] Updated weights for policy 1, policy_version 96250 (0.0008) [2023-10-10 20:20:49,674][123582] Updated weights for policy 0, policy_version 96353 (0.0007) [2023-10-10 20:20:50,049][123582] Updated weights for policy 0, policy_version 96363 (0.0008) [2023-10-10 20:20:50,418][123582] Updated weights for policy 0, policy_version 96373 (0.0007) [2023-10-10 20:20:50,783][123582] Updated weights for policy 0, policy_version 96383 (0.0009) [2023-10-10 20:20:53,211][123614] Updated weights for policy 1, policy_version 96260 (0.0009) [2023-10-10 20:20:53,587][123614] Updated weights for policy 1, policy_version 96270 (0.0009) [2023-10-10 20:20:53,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.6, 300 sec: 14440.1). Total num frames: 197263360. Throughput: 0: 1812.0, 1: 1816.2. Samples: 49333368. Policy #0 lag: (min: 25.0, avg: 29.3, max: 57.0) [2023-10-10 20:20:53,789][122664] Avg episode reward: [(0, '92.050'), (1, '89.820')] [2023-10-10 20:20:53,955][123614] Updated weights for policy 1, policy_version 96280 (0.0008) [2023-10-10 20:20:54,557][123582] Updated weights for policy 0, policy_version 96393 (0.0009) [2023-10-10 20:20:54,921][123582] Updated weights for policy 0, policy_version 96403 (0.0007) [2023-10-10 20:20:55,292][123582] Updated weights for policy 0, policy_version 96413 (0.0007) [2023-10-10 20:20:57,649][123614] Updated weights for policy 1, policy_version 96290 (0.0008) [2023-10-10 20:20:58,009][123614] Updated weights for policy 1, policy_version 96300 (0.0007) [2023-10-10 20:20:58,380][123614] Updated weights for policy 1, policy_version 96310 (0.0008) [2023-10-10 20:20:58,751][123614] Updated weights for policy 1, policy_version 96320 (0.0007) [2023-10-10 20:20:58,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 197361664. Throughput: 0: 1818.4, 1: 1802.0. Samples: 49344138. Policy #0 lag: (min: 25.0, avg: 29.3, max: 57.0) [2023-10-10 20:20:58,788][122664] Avg episode reward: [(0, '95.090'), (1, '88.000')] [2023-10-10 20:20:58,916][123582] Updated weights for policy 0, policy_version 96423 (0.0007) [2023-10-10 20:20:59,299][123582] Updated weights for policy 0, policy_version 96433 (0.0007) [2023-10-10 20:20:59,666][123582] Updated weights for policy 0, policy_version 96443 (0.0007) [2023-10-10 20:21:02,419][123614] Updated weights for policy 1, policy_version 96330 (0.0008) [2023-10-10 20:21:02,784][123614] Updated weights for policy 1, policy_version 96340 (0.0008) [2023-10-10 20:21:03,149][123614] Updated weights for policy 1, policy_version 96350 (0.0008) [2023-10-10 20:21:03,306][123582] Updated weights for policy 0, policy_version 96453 (0.0008) [2023-10-10 20:21:03,674][123582] Updated weights for policy 0, policy_version 96463 (0.0008) [2023-10-10 20:21:03,788][122664] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 197427200. Throughput: 0: 1819.2, 1: 1814.5. Samples: 49366214. Policy #0 lag: (min: 31.0, avg: 33.2, max: 60.0) [2023-10-10 20:21:03,789][122664] Avg episode reward: [(0, '88.490'), (1, '92.150')] [2023-10-10 20:21:04,048][123582] Updated weights for policy 0, policy_version 96473 (0.0008) [2023-10-10 20:21:06,891][123614] Updated weights for policy 1, policy_version 96360 (0.0007) [2023-10-10 20:21:07,265][123614] Updated weights for policy 1, policy_version 96370 (0.0008) [2023-10-10 20:21:07,628][123614] Updated weights for policy 1, policy_version 96380 (0.0008) [2023-10-10 20:21:07,704][123582] Updated weights for policy 0, policy_version 96483 (0.0010) [2023-10-10 20:21:08,079][123582] Updated weights for policy 0, policy_version 96493 (0.0007) [2023-10-10 20:21:08,445][123582] Updated weights for policy 0, policy_version 96503 (0.0009) [2023-10-10 20:21:08,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 197525504. Throughput: 0: 1821.4, 1: 1804.6. Samples: 49387408. Policy #0 lag: (min: 31.0, avg: 33.2, max: 60.0) [2023-10-10 20:21:08,788][122664] Avg episode reward: [(0, '88.680'), (1, '89.930')] [2023-10-10 20:21:11,228][123614] Updated weights for policy 1, policy_version 96390 (0.0008) [2023-10-10 20:21:11,594][123614] Updated weights for policy 1, policy_version 96400 (0.0009) [2023-10-10 20:21:11,968][123614] Updated weights for policy 1, policy_version 96410 (0.0008) [2023-10-10 20:21:12,143][123582] Updated weights for policy 0, policy_version 96513 (0.0008) [2023-10-10 20:21:12,520][123582] Updated weights for policy 0, policy_version 96523 (0.0007) [2023-10-10 20:21:12,885][123582] Updated weights for policy 0, policy_version 96533 (0.0007) [2023-10-10 20:21:13,268][123582] Updated weights for policy 0, policy_version 96543 (0.0009) [2023-10-10 20:21:13,788][122664] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 197591040. Throughput: 0: 1825.2, 1: 1814.2. Samples: 49398818. Policy #0 lag: (min: 31.0, avg: 33.2, max: 60.0) [2023-10-10 20:21:13,788][122664] Avg episode reward: [(0, '89.050'), (1, '80.510')] [2023-10-10 20:21:15,637][123614] Updated weights for policy 1, policy_version 96420 (0.0007) [2023-10-10 20:21:16,006][123614] Updated weights for policy 1, policy_version 96430 (0.0010) [2023-10-10 20:21:16,375][123614] Updated weights for policy 1, policy_version 96440 (0.0007) [2023-10-10 20:21:16,898][123582] Updated weights for policy 0, policy_version 96553 (0.0008) [2023-10-10 20:21:17,260][123582] Updated weights for policy 0, policy_version 96563 (0.0011) [2023-10-10 20:21:17,637][123582] Updated weights for policy 0, policy_version 96573 (0.0010) [2023-10-10 20:21:18,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 197656576. Throughput: 0: 1824.8, 1: 1805.9. Samples: 49419958. Policy #0 lag: (min: 31.0, avg: 33.2, max: 60.0) [2023-10-10 20:21:18,789][122664] Avg episode reward: [(0, '89.300'), (1, '82.280')] [2023-10-10 20:21:20,298][123614] Updated weights for policy 1, policy_version 96450 (0.0008) [2023-10-10 20:21:20,705][123614] Updated weights for policy 1, policy_version 96460 (0.0008) [2023-10-10 20:21:21,063][123614] Updated weights for policy 1, policy_version 96470 (0.0007) [2023-10-10 20:21:21,285][123582] Updated weights for policy 0, policy_version 96583 (0.0008) [2023-10-10 20:21:21,434][123614] Updated weights for policy 1, policy_version 96480 (0.0007) [2023-10-10 20:21:21,661][123582] Updated weights for policy 0, policy_version 96593 (0.0009) [2023-10-10 20:21:22,030][123582] Updated weights for policy 0, policy_version 96603 (0.0007) [2023-10-10 20:21:23,788][122664] Fps is (10 sec: 13106.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 197722112. Throughput: 0: 1824.8, 1: 1801.1. Samples: 49442098. Policy #0 lag: (min: 31.0, avg: 33.2, max: 60.0) [2023-10-10 20:21:23,789][122664] Avg episode reward: [(0, '93.540'), (1, '80.390')] [2023-10-10 20:21:25,129][123614] Updated weights for policy 1, policy_version 96490 (0.0008) [2023-10-10 20:21:25,498][123614] Updated weights for policy 1, policy_version 96500 (0.0007) [2023-10-10 20:21:25,692][123582] Updated weights for policy 0, policy_version 96613 (0.0008) [2023-10-10 20:21:25,864][123614] Updated weights for policy 1, policy_version 96510 (0.0008) [2023-10-10 20:21:26,066][123582] Updated weights for policy 0, policy_version 96623 (0.0007) [2023-10-10 20:21:26,438][123582] Updated weights for policy 0, policy_version 96633 (0.0007) [2023-10-10 20:21:28,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 197787648. Throughput: 0: 1820.0, 1: 1804.3. Samples: 49452482. Policy #0 lag: (min: 31.0, avg: 33.2, max: 60.0) [2023-10-10 20:21:28,789][122664] Avg episode reward: [(0, '93.650'), (1, '76.350')] [2023-10-10 20:21:29,508][123614] Updated weights for policy 1, policy_version 96520 (0.0009) [2023-10-10 20:21:29,878][123614] Updated weights for policy 1, policy_version 96530 (0.0010) [2023-10-10 20:21:30,239][123582] Updated weights for policy 0, policy_version 96643 (0.0008) [2023-10-10 20:21:30,246][123614] Updated weights for policy 1, policy_version 96540 (0.0009) [2023-10-10 20:21:30,619][123582] Updated weights for policy 0, policy_version 96653 (0.0008) [2023-10-10 20:21:30,983][123582] Updated weights for policy 0, policy_version 96663 (0.0008) [2023-10-10 20:21:33,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 197853184. Throughput: 0: 1817.8, 1: 1807.0. Samples: 49474894. Policy #0 lag: (min: 31.0, avg: 33.2, max: 60.0) [2023-10-10 20:21:33,789][122664] Avg episode reward: [(0, '92.900'), (1, '73.770')] [2023-10-10 20:21:33,912][123614] Updated weights for policy 1, policy_version 96550 (0.0007) [2023-10-10 20:21:34,283][123614] Updated weights for policy 1, policy_version 96560 (0.0007) [2023-10-10 20:21:34,644][123614] Updated weights for policy 1, policy_version 96570 (0.0007) [2023-10-10 20:21:34,725][123582] Updated weights for policy 0, policy_version 96673 (0.0008) [2023-10-10 20:21:35,100][123582] Updated weights for policy 0, policy_version 96683 (0.0008) [2023-10-10 20:21:35,469][123582] Updated weights for policy 0, policy_version 96693 (0.0008) [2023-10-10 20:21:35,842][123582] Updated weights for policy 0, policy_version 96703 (0.0008) [2023-10-10 20:21:38,385][123614] Updated weights for policy 1, policy_version 96580 (0.0009) [2023-10-10 20:21:38,758][123614] Updated weights for policy 1, policy_version 96590 (0.0011) [2023-10-10 20:21:38,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 197918720. Throughput: 0: 1819.5, 1: 1815.9. Samples: 49496962. Policy #0 lag: (min: 31.0, avg: 33.2, max: 60.0) [2023-10-10 20:21:38,788][122664] Avg episode reward: [(0, '91.130'), (1, '69.840')] [2023-10-10 20:21:39,125][123614] Updated weights for policy 1, policy_version 96600 (0.0008) [2023-10-10 20:21:39,451][123582] Updated weights for policy 0, policy_version 96713 (0.0008) [2023-10-10 20:21:39,816][123582] Updated weights for policy 0, policy_version 96723 (0.0007) [2023-10-10 20:21:40,187][123582] Updated weights for policy 0, policy_version 96733 (0.0009) [2023-10-10 20:21:42,828][123614] Updated weights for policy 1, policy_version 96610 (0.0007) [2023-10-10 20:21:43,202][123614] Updated weights for policy 1, policy_version 96620 (0.0010) [2023-10-10 20:21:43,563][123614] Updated weights for policy 1, policy_version 96630 (0.0010) [2023-10-10 20:21:43,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 197984256. Throughput: 0: 1819.2, 1: 1812.0. Samples: 49507544. Policy #0 lag: (min: 31.0, avg: 33.2, max: 60.0) [2023-10-10 20:21:43,788][122664] Avg episode reward: [(0, '96.680'), (1, '73.590')] [2023-10-10 20:21:43,800][123582] Updated weights for policy 0, policy_version 96743 (0.0009) [2023-10-10 20:21:43,928][123614] Updated weights for policy 1, policy_version 96640 (0.0008) [2023-10-10 20:21:44,182][123582] Updated weights for policy 0, policy_version 96753 (0.0009) [2023-10-10 20:21:44,558][123582] Updated weights for policy 0, policy_version 96763 (0.0008) [2023-10-10 20:21:47,598][123614] Updated weights for policy 1, policy_version 96650 (0.0010) [2023-10-10 20:21:47,975][123614] Updated weights for policy 1, policy_version 96660 (0.0009) [2023-10-10 20:21:48,186][123582] Updated weights for policy 0, policy_version 96773 (0.0008) [2023-10-10 20:21:48,341][123614] Updated weights for policy 1, policy_version 96670 (0.0007) [2023-10-10 20:21:48,552][123582] Updated weights for policy 0, policy_version 96783 (0.0007) [2023-10-10 20:21:48,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 198082560. Throughput: 0: 1824.2, 1: 1816.0. Samples: 49530020. Policy #0 lag: (min: 31.0, avg: 33.2, max: 60.0) [2023-10-10 20:21:48,788][122664] Avg episode reward: [(0, '98.580'), (1, '74.600')] [2023-10-10 20:21:48,928][123582] Updated weights for policy 0, policy_version 96793 (0.0007) [2023-10-10 20:21:52,010][123614] Updated weights for policy 1, policy_version 96680 (0.0009) [2023-10-10 20:21:52,387][123614] Updated weights for policy 1, policy_version 96690 (0.0010) [2023-10-10 20:21:52,616][123582] Updated weights for policy 0, policy_version 96803 (0.0008) [2023-10-10 20:21:52,741][123614] Updated weights for policy 1, policy_version 96700 (0.0008) [2023-10-10 20:21:52,983][123582] Updated weights for policy 0, policy_version 96813 (0.0010) [2023-10-10 20:21:53,359][123582] Updated weights for policy 0, policy_version 96823 (0.0011) [2023-10-10 20:21:53,788][122664] Fps is (10 sec: 19660.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 198180864. Throughput: 0: 1816.7, 1: 1815.2. Samples: 49550844. Policy #0 lag: (min: 31.0, avg: 33.2, max: 60.0) [2023-10-10 20:21:53,790][122664] Avg episode reward: [(0, '99.430'), (1, '77.100')] [2023-10-10 20:21:53,799][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000096704_99024896.pth... [2023-10-10 20:21:53,800][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000096832_99155968.pth... [2023-10-10 20:21:53,830][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000095008_97288192.pth [2023-10-10 20:21:53,836][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000095136_97419264.pth [2023-10-10 20:21:56,408][123614] Updated weights for policy 1, policy_version 96710 (0.0008) [2023-10-10 20:21:56,777][123614] Updated weights for policy 1, policy_version 96720 (0.0009) [2023-10-10 20:21:57,151][123614] Updated weights for policy 1, policy_version 96730 (0.0008) [2023-10-10 20:21:57,179][123582] Updated weights for policy 0, policy_version 96833 (0.0008) [2023-10-10 20:21:57,550][123582] Updated weights for policy 0, policy_version 96843 (0.0008) [2023-10-10 20:21:57,914][123582] Updated weights for policy 0, policy_version 96853 (0.0009) [2023-10-10 20:21:58,294][123582] Updated weights for policy 0, policy_version 96863 (0.0010) [2023-10-10 20:21:58,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 198246400. Throughput: 0: 1815.8, 1: 1822.6. Samples: 49562548. Policy #0 lag: (min: 31.0, avg: 33.2, max: 60.0) [2023-10-10 20:21:58,788][122664] Avg episode reward: [(0, '99.420'), (1, '76.110')] [2023-10-10 20:22:00,804][123614] Updated weights for policy 1, policy_version 96740 (0.0010) [2023-10-10 20:22:01,175][123614] Updated weights for policy 1, policy_version 96750 (0.0010) [2023-10-10 20:22:01,553][123614] Updated weights for policy 1, policy_version 96760 (0.0007) [2023-10-10 20:22:01,994][123582] Updated weights for policy 0, policy_version 96873 (0.0008) [2023-10-10 20:22:02,365][123582] Updated weights for policy 0, policy_version 96883 (0.0009) [2023-10-10 20:22:02,731][123582] Updated weights for policy 0, policy_version 96893 (0.0008) [2023-10-10 20:22:03,788][122664] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 198311936. Throughput: 0: 1813.9, 1: 1818.1. Samples: 49583400. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:22:03,789][122664] Avg episode reward: [(0, '101.470'), (1, '76.640')] [2023-10-10 20:22:05,195][123614] Updated weights for policy 1, policy_version 96770 (0.0007) [2023-10-10 20:22:05,586][123614] Updated weights for policy 1, policy_version 96780 (0.0007) [2023-10-10 20:22:05,955][123614] Updated weights for policy 1, policy_version 96790 (0.0007) [2023-10-10 20:22:06,318][123614] Updated weights for policy 1, policy_version 96800 (0.0008) [2023-10-10 20:22:06,526][123582] Updated weights for policy 0, policy_version 96903 (0.0008) [2023-10-10 20:22:06,895][123582] Updated weights for policy 0, policy_version 96913 (0.0009) [2023-10-10 20:22:07,266][123582] Updated weights for policy 0, policy_version 96923 (0.0011) [2023-10-10 20:22:08,788][122664] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 198377472. Throughput: 0: 1805.5, 1: 1830.6. Samples: 49605722. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:22:08,789][122664] Avg episode reward: [(0, '103.990'), (1, '77.050')] [2023-10-10 20:22:09,887][123614] Updated weights for policy 1, policy_version 96810 (0.0010) [2023-10-10 20:22:10,264][123614] Updated weights for policy 1, policy_version 96820 (0.0008) [2023-10-10 20:22:10,624][123614] Updated weights for policy 1, policy_version 96830 (0.0008) [2023-10-10 20:22:10,936][123582] Updated weights for policy 0, policy_version 96933 (0.0008) [2023-10-10 20:22:11,313][123582] Updated weights for policy 0, policy_version 96943 (0.0008) [2023-10-10 20:22:11,674][123582] Updated weights for policy 0, policy_version 96953 (0.0010) [2023-10-10 20:22:13,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 198443008. Throughput: 0: 1813.0, 1: 1826.2. Samples: 49616248. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:22:13,788][122664] Avg episode reward: [(0, '105.450'), (1, '79.340')] [2023-10-10 20:22:14,485][123614] Updated weights for policy 1, policy_version 96840 (0.0009) [2023-10-10 20:22:14,859][123614] Updated weights for policy 1, policy_version 96850 (0.0008) [2023-10-10 20:22:15,227][123614] Updated weights for policy 1, policy_version 96860 (0.0009) [2023-10-10 20:22:15,429][123582] Updated weights for policy 0, policy_version 96963 (0.0008) [2023-10-10 20:22:15,801][123582] Updated weights for policy 0, policy_version 96973 (0.0007) [2023-10-10 20:22:16,164][123582] Updated weights for policy 0, policy_version 96983 (0.0007) [2023-10-10 20:22:18,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 198508544. Throughput: 0: 1808.3, 1: 1823.1. Samples: 49638306. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:22:18,788][122664] Avg episode reward: [(0, '105.800'), (1, '80.140')] [2023-10-10 20:22:18,981][123614] Updated weights for policy 1, policy_version 96870 (0.0009) [2023-10-10 20:22:19,342][123614] Updated weights for policy 1, policy_version 96880 (0.0009) [2023-10-10 20:22:19,721][123614] Updated weights for policy 1, policy_version 96890 (0.0007) [2023-10-10 20:22:19,822][123582] Updated weights for policy 0, policy_version 96993 (0.0008) [2023-10-10 20:22:20,187][123582] Updated weights for policy 0, policy_version 97003 (0.0009) [2023-10-10 20:22:20,563][123582] Updated weights for policy 0, policy_version 97013 (0.0009) [2023-10-10 20:22:20,927][123582] Updated weights for policy 0, policy_version 97023 (0.0009) [2023-10-10 20:22:23,328][123614] Updated weights for policy 1, policy_version 96900 (0.0009) [2023-10-10 20:22:23,697][123614] Updated weights for policy 1, policy_version 96910 (0.0008) [2023-10-10 20:22:23,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 198574080. Throughput: 0: 1809.1, 1: 1825.2. Samples: 49660504. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:22:23,789][122664] Avg episode reward: [(0, '116.110'), (1, '83.490')] [2023-10-10 20:22:24,066][123614] Updated weights for policy 1, policy_version 96920 (0.0008) [2023-10-10 20:22:24,740][123582] Updated weights for policy 0, policy_version 97033 (0.0010) [2023-10-10 20:22:25,113][123582] Updated weights for policy 0, policy_version 97043 (0.0007) [2023-10-10 20:22:25,489][123582] Updated weights for policy 0, policy_version 97053 (0.0008) [2023-10-10 20:22:27,866][123614] Updated weights for policy 1, policy_version 96930 (0.0008) [2023-10-10 20:22:28,230][123614] Updated weights for policy 1, policy_version 96940 (0.0010) [2023-10-10 20:22:28,600][123614] Updated weights for policy 1, policy_version 96950 (0.0007) [2023-10-10 20:22:28,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 198639616. Throughput: 0: 1805.6, 1: 1826.8. Samples: 49671006. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:22:28,788][122664] Avg episode reward: [(0, '115.020'), (1, '87.210')] [2023-10-10 20:22:28,966][123614] Updated weights for policy 1, policy_version 96960 (0.0008) [2023-10-10 20:22:29,272][123582] Updated weights for policy 0, policy_version 97063 (0.0007) [2023-10-10 20:22:29,652][123582] Updated weights for policy 0, policy_version 97073 (0.0009) [2023-10-10 20:22:30,026][123582] Updated weights for policy 0, policy_version 97083 (0.0009) [2023-10-10 20:22:32,814][123614] Updated weights for policy 1, policy_version 96970 (0.0008) [2023-10-10 20:22:33,186][123614] Updated weights for policy 1, policy_version 96980 (0.0008) [2023-10-10 20:22:33,556][123614] Updated weights for policy 1, policy_version 96990 (0.0009) [2023-10-10 20:22:33,652][123582] Updated weights for policy 0, policy_version 97093 (0.0008) [2023-10-10 20:22:33,788][122664] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 198737920. Throughput: 0: 1797.9, 1: 1824.8. Samples: 49693042. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:22:33,789][122664] Avg episode reward: [(0, '116.390'), (1, '84.830')] [2023-10-10 20:22:34,025][123582] Updated weights for policy 0, policy_version 97103 (0.0008) [2023-10-10 20:22:34,396][123582] Updated weights for policy 0, policy_version 97113 (0.0009) [2023-10-10 20:22:37,015][123614] Updated weights for policy 1, policy_version 97000 (0.0008) [2023-10-10 20:22:37,383][123614] Updated weights for policy 1, policy_version 97010 (0.0008) [2023-10-10 20:22:37,750][123614] Updated weights for policy 1, policy_version 97020 (0.0007) [2023-10-10 20:22:38,001][123582] Updated weights for policy 0, policy_version 97123 (0.0011) [2023-10-10 20:22:38,377][123582] Updated weights for policy 0, policy_version 97133 (0.0010) [2023-10-10 20:22:38,754][123582] Updated weights for policy 0, policy_version 97143 (0.0011) [2023-10-10 20:22:38,788][122664] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 198803456. Throughput: 0: 1812.6, 1: 1823.7. Samples: 49714476. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:22:38,789][122664] Avg episode reward: [(0, '117.870'), (1, '91.900')] [2023-10-10 20:22:41,567][123614] Updated weights for policy 1, policy_version 97030 (0.0009) [2023-10-10 20:22:41,926][123614] Updated weights for policy 1, policy_version 97040 (0.0009) [2023-10-10 20:22:42,295][123614] Updated weights for policy 1, policy_version 97050 (0.0010) [2023-10-10 20:22:42,405][123582] Updated weights for policy 0, policy_version 97153 (0.0010) [2023-10-10 20:22:42,772][123582] Updated weights for policy 0, policy_version 97163 (0.0007) [2023-10-10 20:22:43,149][123582] Updated weights for policy 0, policy_version 97173 (0.0007) [2023-10-10 20:22:43,514][123582] Updated weights for policy 0, policy_version 97183 (0.0007) [2023-10-10 20:22:43,788][122664] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 198901760. Throughput: 0: 1809.7, 1: 1821.2. Samples: 49725936. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:22:43,788][122664] Avg episode reward: [(0, '117.820'), (1, '93.060')] [2023-10-10 20:22:46,026][123614] Updated weights for policy 1, policy_version 97060 (0.0008) [2023-10-10 20:22:46,404][123614] Updated weights for policy 1, policy_version 97070 (0.0008) [2023-10-10 20:22:46,775][123614] Updated weights for policy 1, policy_version 97080 (0.0008) [2023-10-10 20:22:47,198][123582] Updated weights for policy 0, policy_version 97193 (0.0009) [2023-10-10 20:22:47,562][123582] Updated weights for policy 0, policy_version 97203 (0.0009) [2023-10-10 20:22:47,947][123582] Updated weights for policy 0, policy_version 97213 (0.0010) [2023-10-10 20:22:48,788][122664] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 198967296. Throughput: 0: 1822.7, 1: 1819.6. Samples: 49747300. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:22:48,788][122664] Avg episode reward: [(0, '116.510'), (1, '96.650')] [2023-10-10 20:22:50,384][123614] Updated weights for policy 1, policy_version 97090 (0.0008) [2023-10-10 20:22:50,792][123614] Updated weights for policy 1, policy_version 97100 (0.0010) [2023-10-10 20:22:51,156][123614] Updated weights for policy 1, policy_version 97110 (0.0009) [2023-10-10 20:22:51,518][123614] Updated weights for policy 1, policy_version 97120 (0.0007) [2023-10-10 20:22:51,691][123582] Updated weights for policy 0, policy_version 97223 (0.0009) [2023-10-10 20:22:52,070][123582] Updated weights for policy 0, policy_version 97233 (0.0011) [2023-10-10 20:22:52,436][123582] Updated weights for policy 0, policy_version 97243 (0.0010) [2023-10-10 20:22:53,788][122664] Fps is (10 sec: 13106.7, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 199032832. Throughput: 0: 1818.0, 1: 1816.8. Samples: 49769290. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:22:53,789][122664] Avg episode reward: [(0, '119.450'), (1, '96.120')] [2023-10-10 20:22:55,035][123614] Updated weights for policy 1, policy_version 97130 (0.0007) [2023-10-10 20:22:55,404][123614] Updated weights for policy 1, policy_version 97140 (0.0009) [2023-10-10 20:22:55,772][123614] Updated weights for policy 1, policy_version 97150 (0.0007) [2023-10-10 20:22:56,133][123582] Updated weights for policy 0, policy_version 97253 (0.0009) [2023-10-10 20:22:56,503][123582] Updated weights for policy 0, policy_version 97263 (0.0009) [2023-10-10 20:22:56,860][123582] Updated weights for policy 0, policy_version 97273 (0.0007) [2023-10-10 20:22:58,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 199098368. Throughput: 0: 1823.7, 1: 1815.9. Samples: 49780028. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:22:58,789][122664] Avg episode reward: [(0, '120.730'), (1, '97.540')] [2023-10-10 20:22:59,511][123614] Updated weights for policy 1, policy_version 97160 (0.0008) [2023-10-10 20:22:59,888][123614] Updated weights for policy 1, policy_version 97170 (0.0008) [2023-10-10 20:23:00,257][123614] Updated weights for policy 1, policy_version 97180 (0.0009) [2023-10-10 20:23:00,440][123582] Updated weights for policy 0, policy_version 97283 (0.0008) [2023-10-10 20:23:00,819][123582] Updated weights for policy 0, policy_version 97293 (0.0009) [2023-10-10 20:23:01,200][123582] Updated weights for policy 0, policy_version 97303 (0.0010) [2023-10-10 20:23:03,788][122664] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 199163904. Throughput: 0: 1818.4, 1: 1814.8. Samples: 49801802. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 20:23:03,789][122664] Avg episode reward: [(0, '116.020'), (1, '94.960')] [2023-10-10 20:23:03,985][123614] Updated weights for policy 1, policy_version 97190 (0.0009) [2023-10-10 20:23:04,354][123614] Updated weights for policy 1, policy_version 97200 (0.0009) [2023-10-10 20:23:04,713][123614] Updated weights for policy 1, policy_version 97210 (0.0012) [2023-10-10 20:23:05,009][123582] Updated weights for policy 0, policy_version 97313 (0.0007) [2023-10-10 20:23:05,382][123582] Updated weights for policy 0, policy_version 97323 (0.0009) [2023-10-10 20:23:05,756][123582] Updated weights for policy 0, policy_version 97333 (0.0007) [2023-10-10 20:23:06,138][123582] Updated weights for policy 0, policy_version 97343 (0.0008) [2023-10-10 20:23:08,261][123614] Updated weights for policy 1, policy_version 97220 (0.0009) [2023-10-10 20:23:08,628][123614] Updated weights for policy 1, policy_version 97230 (0.0007) [2023-10-10 20:23:08,788][122664] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 199229440. Throughput: 0: 1820.4, 1: 1814.9. Samples: 49824096. Policy #0 lag: (min: 1.0, avg: 16.7, max: 33.0) [2023-10-10 20:23:08,789][122664] Avg episode reward: [(0, '113.700'), (1, '90.820')] [2023-10-10 20:23:08,993][123614] Updated weights for policy 1, policy_version 97240 (0.0008) [2023-10-10 20:23:09,712][123582] Updated weights for policy 0, policy_version 97353 (0.0009) [2023-10-10 20:23:10,096][123582] Updated weights for policy 0, policy_version 97363 (0.0007) [2023-10-10 20:23:10,465][123582] Updated weights for policy 0, policy_version 97373 (0.0008) [2023-10-10 20:23:12,687][123614] Updated weights for policy 1, policy_version 97250 (0.0008) [2023-10-10 20:23:13,059][123614] Updated weights for policy 1, policy_version 97260 (0.0007) [2023-10-10 20:23:13,421][123614] Updated weights for policy 1, policy_version 97270 (0.0007) [2023-10-10 20:23:13,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 199294976. Throughput: 0: 1820.9, 1: 1817.8. Samples: 49834748. Policy #0 lag: (min: 1.0, avg: 16.7, max: 33.0) [2023-10-10 20:23:13,788][122664] Avg episode reward: [(0, '112.260'), (1, '89.680')] [2023-10-10 20:23:13,795][123614] Updated weights for policy 1, policy_version 97280 (0.0008) [2023-10-10 20:23:14,144][123582] Updated weights for policy 0, policy_version 97383 (0.0009) [2023-10-10 20:23:14,523][123582] Updated weights for policy 0, policy_version 97393 (0.0008) [2023-10-10 20:23:14,890][123582] Updated weights for policy 0, policy_version 97403 (0.0010) [2023-10-10 20:23:17,507][123614] Updated weights for policy 1, policy_version 97290 (0.0007) [2023-10-10 20:23:17,873][123614] Updated weights for policy 1, policy_version 97300 (0.0008) [2023-10-10 20:23:18,241][123614] Updated weights for policy 1, policy_version 97310 (0.0010) [2023-10-10 20:23:18,696][123582] Updated weights for policy 0, policy_version 97413 (0.0009) [2023-10-10 20:23:18,788][122664] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 199393280. Throughput: 0: 1820.8, 1: 1816.2. Samples: 49856708. Policy #0 lag: (min: 1.0, avg: 16.7, max: 33.0) [2023-10-10 20:23:18,788][122664] Avg episode reward: [(0, '110.610'), (1, '87.100')] [2023-10-10 20:23:19,069][123582] Updated weights for policy 0, policy_version 97423 (0.0007) [2023-10-10 20:23:19,447][123582] Updated weights for policy 0, policy_version 97433 (0.0007) [2023-10-10 20:23:21,993][123614] Updated weights for policy 1, policy_version 97320 (0.0008) [2023-10-10 20:23:22,364][123614] Updated weights for policy 1, policy_version 97330 (0.0009) [2023-10-10 20:23:22,742][123614] Updated weights for policy 1, policy_version 97340 (0.0007) [2023-10-10 20:23:23,002][123582] Updated weights for policy 0, policy_version 97443 (0.0008) [2023-10-10 20:23:23,376][123582] Updated weights for policy 0, policy_version 97453 (0.0010) [2023-10-10 20:23:23,753][123582] Updated weights for policy 0, policy_version 97463 (0.0009) [2023-10-10 20:23:23,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 199458816. Throughput: 0: 1818.3, 1: 1817.7. Samples: 49878098. Policy #0 lag: (min: 1.0, avg: 16.7, max: 33.0) [2023-10-10 20:23:23,788][122664] Avg episode reward: [(0, '115.440'), (1, '85.340')] [2023-10-10 20:23:26,457][123614] Updated weights for policy 1, policy_version 97350 (0.0010) [2023-10-10 20:23:26,829][123614] Updated weights for policy 1, policy_version 97360 (0.0009) [2023-10-10 20:23:27,196][123614] Updated weights for policy 1, policy_version 97370 (0.0007) [2023-10-10 20:23:27,472][123582] Updated weights for policy 0, policy_version 97473 (0.0008) [2023-10-10 20:23:27,844][123582] Updated weights for policy 0, policy_version 97483 (0.0009) [2023-10-10 20:23:28,213][123582] Updated weights for policy 0, policy_version 97493 (0.0007) [2023-10-10 20:23:28,580][123582] Updated weights for policy 0, policy_version 97503 (0.0008) [2023-10-10 20:23:28,788][122664] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 199557120. Throughput: 0: 1819.0, 1: 1817.1. Samples: 49889564. Policy #0 lag: (min: 1.0, avg: 16.7, max: 33.0) [2023-10-10 20:23:28,789][122664] Avg episode reward: [(0, '115.390'), (1, '79.910')] [2023-10-10 20:23:30,864][123614] Updated weights for policy 1, policy_version 97380 (0.0009) [2023-10-10 20:23:31,237][123614] Updated weights for policy 1, policy_version 97390 (0.0009) [2023-10-10 20:23:31,601][123614] Updated weights for policy 1, policy_version 97400 (0.0010) [2023-10-10 20:23:32,283][123582] Updated weights for policy 0, policy_version 97513 (0.0010) [2023-10-10 20:23:32,660][123582] Updated weights for policy 0, policy_version 97523 (0.0008) [2023-10-10 20:23:33,030][123582] Updated weights for policy 0, policy_version 97533 (0.0012) [2023-10-10 20:23:33,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 199622656. Throughput: 0: 1817.8, 1: 1816.4. Samples: 49910842. Policy #0 lag: (min: 1.0, avg: 16.7, max: 33.0) [2023-10-10 20:23:33,788][122664] Avg episode reward: [(0, '113.380'), (1, '83.580')] [2023-10-10 20:23:35,359][123614] Updated weights for policy 1, policy_version 97410 (0.0010) [2023-10-10 20:23:35,721][123614] Updated weights for policy 1, policy_version 97420 (0.0007) [2023-10-10 20:23:36,084][123614] Updated weights for policy 1, policy_version 97430 (0.0007) [2023-10-10 20:23:36,447][123614] Updated weights for policy 1, policy_version 97440 (0.0007) [2023-10-10 20:23:36,707][123582] Updated weights for policy 0, policy_version 97543 (0.0009) [2023-10-10 20:23:37,079][123582] Updated weights for policy 0, policy_version 97553 (0.0011) [2023-10-10 20:23:37,448][123582] Updated weights for policy 0, policy_version 97563 (0.0008) [2023-10-10 20:23:38,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 199688192. Throughput: 0: 1814.4, 1: 1809.3. Samples: 49932356. Policy #0 lag: (min: 1.0, avg: 16.7, max: 33.0) [2023-10-10 20:23:38,788][122664] Avg episode reward: [(0, '118.080'), (1, '80.850')] [2023-10-10 20:23:40,230][123614] Updated weights for policy 1, policy_version 97450 (0.0008) [2023-10-10 20:23:40,605][123614] Updated weights for policy 1, policy_version 97460 (0.0010) [2023-10-10 20:23:40,975][123614] Updated weights for policy 1, policy_version 97470 (0.0008) [2023-10-10 20:23:41,173][123582] Updated weights for policy 0, policy_version 97573 (0.0012) [2023-10-10 20:23:41,548][123582] Updated weights for policy 0, policy_version 97583 (0.0008) [2023-10-10 20:23:41,921][123582] Updated weights for policy 0, policy_version 97593 (0.0007) [2023-10-10 20:23:43,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 199753728. Throughput: 0: 1817.6, 1: 1812.0. Samples: 49943360. Policy #0 lag: (min: 1.0, avg: 16.7, max: 33.0) [2023-10-10 20:23:43,789][122664] Avg episode reward: [(0, '115.850'), (1, '84.010')] [2023-10-10 20:23:44,624][123614] Updated weights for policy 1, policy_version 97480 (0.0010) [2023-10-10 20:23:44,990][123614] Updated weights for policy 1, policy_version 97490 (0.0008) [2023-10-10 20:23:45,369][123614] Updated weights for policy 1, policy_version 97500 (0.0008) [2023-10-10 20:23:45,531][123582] Updated weights for policy 0, policy_version 97603 (0.0009) [2023-10-10 20:23:45,907][123582] Updated weights for policy 0, policy_version 97613 (0.0010) [2023-10-10 20:23:46,279][123582] Updated weights for policy 0, policy_version 97623 (0.0008) [2023-10-10 20:23:48,788][122664] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 199819264. Throughput: 0: 1820.1, 1: 1814.1. Samples: 49965342. Policy #0 lag: (min: 1.0, avg: 16.7, max: 33.0) [2023-10-10 20:23:48,789][122664] Avg episode reward: [(0, '111.350'), (1, '78.250')] [2023-10-10 20:23:48,978][123614] Updated weights for policy 1, policy_version 97510 (0.0009) [2023-10-10 20:23:49,343][123614] Updated weights for policy 1, policy_version 97520 (0.0009) [2023-10-10 20:23:49,712][123614] Updated weights for policy 1, policy_version 97530 (0.0008) [2023-10-10 20:23:50,081][123582] Updated weights for policy 0, policy_version 97633 (0.0008) [2023-10-10 20:23:50,445][123582] Updated weights for policy 0, policy_version 97643 (0.0009) [2023-10-10 20:23:50,829][123582] Updated weights for policy 0, policy_version 97653 (0.0009) [2023-10-10 20:23:51,194][123582] Updated weights for policy 0, policy_version 97663 (0.0009) [2023-10-10 20:23:53,459][123614] Updated weights for policy 1, policy_version 97540 (0.0010) [2023-10-10 20:23:53,788][122664] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 199884800. Throughput: 0: 1817.7, 1: 1816.2. Samples: 49987622. Policy #0 lag: (min: 1.0, avg: 16.7, max: 33.0) [2023-10-10 20:23:53,788][122664] Avg episode reward: [(0, '106.760'), (1, '78.390')] [2023-10-10 20:23:53,795][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000097664_100007936.pth... [2023-10-10 20:23:53,824][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000095968_98271232.pth [2023-10-10 20:23:53,825][123614] Updated weights for policy 1, policy_version 97550 (0.0007) [2023-10-10 20:23:54,195][123614] Updated weights for policy 1, policy_version 97560 (0.0008) [2023-10-10 20:23:54,491][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000097568_99909632.pth... [2023-10-10 20:23:54,522][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000095840_98140160.pth [2023-10-10 20:23:54,756][123582] Updated weights for policy 0, policy_version 97673 (0.0008) [2023-10-10 20:23:55,134][123582] Updated weights for policy 0, policy_version 97683 (0.0012) [2023-10-10 20:23:55,503][123582] Updated weights for policy 0, policy_version 97693 (0.0009) [2023-10-10 20:23:57,963][123614] Updated weights for policy 1, policy_version 97570 (0.0009) [2023-10-10 20:23:58,337][123614] Updated weights for policy 1, policy_version 97580 (0.0008) [2023-10-10 20:23:58,704][123614] Updated weights for policy 1, policy_version 97590 (0.0007) [2023-10-10 20:23:58,788][122664] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 199950336. Throughput: 0: 1818.1, 1: 1810.2. Samples: 49998022. Policy #0 lag: (min: 1.0, avg: 16.7, max: 33.0) [2023-10-10 20:23:58,788][122664] Avg episode reward: [(0, '104.060'), (1, '79.550')] [2023-10-10 20:23:59,075][123614] Updated weights for policy 1, policy_version 97600 (0.0008) [2023-10-10 20:23:59,273][123582] Updated weights for policy 0, policy_version 97703 (0.0008) [2023-10-10 20:23:59,643][123582] Updated weights for policy 0, policy_version 97713 (0.0008) [2023-10-10 20:24:00,006][123582] Updated weights for policy 0, policy_version 97723 (0.0009) [2023-10-10 20:24:03,000][123614] Updated weights for policy 1, policy_version 97610 (0.0011) [2023-10-10 20:24:03,372][123614] Updated weights for policy 1, policy_version 97620 (0.0010) [2023-10-10 20:24:03,722][123582] Updated weights for policy 0, policy_version 97733 (0.0009) [2023-10-10 20:24:03,734][123614] Updated weights for policy 1, policy_version 97630 (0.0007) [2023-10-10 20:24:03,788][122664] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 200015872. Throughput: 0: 1815.9, 1: 1822.5. Samples: 50020438. Policy #0 lag: (min: 1.0, avg: 16.7, max: 33.0) [2023-10-10 20:24:03,789][122664] Avg episode reward: [(0, '98.720'), (1, '73.730')] [2023-10-10 20:24:04,094][123582] Updated weights for policy 0, policy_version 97743 (0.0007) [2023-10-10 20:24:04,464][123582] Updated weights for policy 0, policy_version 97753 (0.0007) [2023-10-10 20:24:07,278][123614] Updated weights for policy 1, policy_version 97640 (0.0009) [2023-10-10 20:24:07,637][123614] Updated weights for policy 1, policy_version 97650 (0.0010) [2023-10-10 20:24:08,002][123614] Updated weights for policy 1, policy_version 97660 (0.0010) [2023-10-10 20:24:08,159][123582] Updated weights for policy 0, policy_version 97763 (0.0008) [2023-10-10 20:24:08,527][123582] Updated weights for policy 0, policy_version 97773 (0.0008) [2023-10-10 20:24:08,788][122664] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 200114176. Throughput: 0: 1823.5, 1: 1810.3. Samples: 50041620. Policy #0 lag: (min: 1.0, avg: 16.7, max: 33.0) [2023-10-10 20:24:08,788][122664] Avg episode reward: [(0, '98.410'), (1, '75.090')] [2023-10-10 20:24:08,909][123582] Updated weights for policy 0, policy_version 97783 (0.0009) [2023-10-10 20:24:09,244][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000097792_100139008.pth... [2023-10-10 20:24:09,245][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000097664_100007936.pth... [2023-10-10 20:24:09,245][123626] Stopping RolloutWorker_w10... [2023-10-10 20:24:09,245][123626] Loop rollout_proc10_evt_loop terminating... [2023-10-10 20:24:09,245][123621] Stopping RolloutWorker_w3... [2023-10-10 20:24:09,246][123621] Loop rollout_proc3_evt_loop terminating... [2023-10-10 20:24:09,246][123615] Stopping RolloutWorker_w0... [2023-10-10 20:24:09,246][123631] Stopping RolloutWorker_w13... [2023-10-10 20:24:09,246][123615] Loop rollout_proc0_evt_loop terminating... [2023-10-10 20:24:09,246][123623] Stopping RolloutWorker_w5... [2023-10-10 20:24:09,246][123631] Loop rollout_proc13_evt_loop terminating... [2023-10-10 20:24:09,246][124221] Stopping RolloutWorker_w15... [2023-10-10 20:24:09,246][123623] Loop rollout_proc5_evt_loop terminating... [2023-10-10 20:24:09,247][124221] Loop rollout_proc15_evt_loop terminating... [2023-10-10 20:24:09,247][123630] Stopping RolloutWorker_w12... [2023-10-10 20:24:09,247][123622] Stopping RolloutWorker_w4... [2023-10-10 20:24:09,247][123622] Loop rollout_proc4_evt_loop terminating... [2023-10-10 20:24:09,247][123630] Loop rollout_proc12_evt_loop terminating... [2023-10-10 20:24:09,247][124189] Stopping RolloutWorker_w14... [2023-10-10 20:24:09,247][123624] Stopping RolloutWorker_w6... [2023-10-10 20:24:09,248][123624] Loop rollout_proc6_evt_loop terminating... [2023-10-10 20:24:09,248][124189] Loop rollout_proc14_evt_loop terminating... [2023-10-10 20:24:09,248][123620] Stopping RolloutWorker_w2... [2023-10-10 20:24:09,248][123620] Loop rollout_proc2_evt_loop terminating... [2023-10-10 20:24:09,249][123625] Stopping RolloutWorker_w7... [2023-10-10 20:24:09,249][123629] Stopping RolloutWorker_w11... [2023-10-10 20:24:09,249][123619] Stopping RolloutWorker_w1... [2023-10-10 20:24:09,249][123625] Loop rollout_proc7_evt_loop terminating... [2023-10-10 20:24:09,250][123629] Loop rollout_proc11_evt_loop terminating... [2023-10-10 20:24:09,250][123619] Loop rollout_proc1_evt_loop terminating... [2023-10-10 20:24:09,250][123628] Stopping RolloutWorker_w8... [2023-10-10 20:24:09,250][123627] Stopping RolloutWorker_w9... [2023-10-10 20:24:09,250][123628] Loop rollout_proc8_evt_loop terminating... [2023-10-10 20:24:09,250][123627] Loop rollout_proc9_evt_loop terminating... [2023-10-10 20:24:09,256][122664] Component RolloutWorker_w10 stopped! [2023-10-10 20:24:09,251][123465] Stopping Batcher_1... [2023-10-10 20:24:09,257][122664] Component RolloutWorker_w3 stopped! [2023-10-10 20:24:09,258][122664] Component RolloutWorker_w13 stopped! [2023-10-10 20:24:09,259][122664] Component RolloutWorker_w0 stopped! [2023-10-10 20:24:09,259][122664] Component RolloutWorker_w5 stopped! [2023-10-10 20:24:09,259][122664] Component RolloutWorker_w15 stopped! [2023-10-10 20:24:09,260][122664] Component RolloutWorker_w4 stopped! [2023-10-10 20:24:09,260][122664] Component RolloutWorker_w12 stopped! [2023-10-10 20:24:09,260][122664] Component RolloutWorker_w6 stopped! [2023-10-10 20:24:09,261][122664] Component RolloutWorker_w14 stopped! [2023-10-10 20:24:09,261][122664] Component RolloutWorker_w2 stopped! [2023-10-10 20:24:09,261][122664] Component RolloutWorker_w7 stopped! [2023-10-10 20:24:09,262][122664] Component RolloutWorker_w11 stopped! [2023-10-10 20:24:09,262][122664] Component RolloutWorker_w1 stopped! [2023-10-10 20:24:09,262][122664] Component RolloutWorker_w8 stopped! [2023-10-10 20:24:09,263][122664] Component RolloutWorker_w9 stopped! [2023-10-10 20:24:09,263][122664] Component Batcher_1 stopped! [2023-10-10 20:24:09,263][122664] Component Batcher_0 stopped! [2023-10-10 20:24:09,260][123247] Stopping Batcher_0... [2023-10-10 20:24:09,273][123614] Weights refcount: 2 0 [2023-10-10 20:24:09,275][123614] Stopping InferenceWorker_p1-w0... [2023-10-10 20:24:09,275][122664] Component InferenceWorker_p1-w0 stopped! [2023-10-10 20:24:09,276][123614] Loop inference_proc1-0_evt_loop terminating... [2023-10-10 20:24:09,267][123465] Loop batcher_evt_loop terminating... [2023-10-10 20:24:09,277][123582] Weights refcount: 2 0 [2023-10-10 20:24:09,279][123582] Stopping InferenceWorker_p0-w0... [2023-10-10 20:24:09,279][123582] Loop inference_proc0-0_evt_loop terminating... [2023-10-10 20:24:09,279][122664] Component InferenceWorker_p0-w0 stopped! [2023-10-10 20:24:09,282][123247] Loop batcher_evt_loop terminating... [2023-10-10 20:24:09,284][123247] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000096832_99155968.pth [2023-10-10 20:24:09,287][123465] Removing ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000096704_99024896.pth [2023-10-10 20:24:09,288][123247] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p0/checkpoint_000097792_100139008.pth... [2023-10-10 20:24:09,292][123465] Saving ./train_atari/atari_demonattack_APPO/checkpoint_p1/checkpoint_000097664_100007936.pth... [2023-10-10 20:24:09,327][123247] Stopping LearnerWorker_p0... [2023-10-10 20:24:09,327][123247] Loop learner_proc0_evt_loop terminating... [2023-10-10 20:24:09,327][122664] Component LearnerWorker_p0 stopped! [2023-10-10 20:24:09,331][123465] Stopping LearnerWorker_p1... [2023-10-10 20:24:09,331][123465] Loop learner_proc1_evt_loop terminating... [2023-10-10 20:24:09,331][122664] Component LearnerWorker_p1 stopped! [2023-10-10 20:24:09,332][122664] Waiting for process learner_proc0 to stop... [2023-10-10 20:24:10,174][122664] Waiting for process learner_proc1 to stop... [2023-10-10 20:24:10,175][122664] Waiting for process inference_proc0-0 to join... [2023-10-10 20:24:10,273][122664] Waiting for process inference_proc1-0 to join... [2023-10-10 20:24:10,274][122664] Waiting for process rollout_proc0 to join... [2023-10-10 20:24:10,275][122664] Waiting for process rollout_proc1 to join... [2023-10-10 20:24:10,275][122664] Waiting for process rollout_proc2 to join... [2023-10-10 20:24:10,276][122664] Waiting for process rollout_proc3 to join... [2023-10-10 20:24:10,277][122664] Waiting for process rollout_proc4 to join... [2023-10-10 20:24:10,277][122664] Waiting for process rollout_proc5 to join... [2023-10-10 20:24:10,278][122664] Waiting for process rollout_proc6 to join... [2023-10-10 20:24:10,279][122664] Waiting for process rollout_proc7 to join... [2023-10-10 20:24:10,279][122664] Waiting for process rollout_proc8 to join... [2023-10-10 20:24:10,280][122664] Waiting for process rollout_proc9 to join... [2023-10-10 20:24:10,280][122664] Waiting for process rollout_proc10 to join... [2023-10-10 20:24:10,281][122664] Waiting for process rollout_proc11 to join... [2023-10-10 20:24:10,282][122664] Waiting for process rollout_proc12 to join... [2023-10-10 20:24:10,282][122664] Waiting for process rollout_proc13 to join... [2023-10-10 20:24:10,283][122664] Waiting for process rollout_proc14 to join... [2023-10-10 20:24:10,284][122664] Waiting for process rollout_proc15 to join... [2023-10-10 20:24:10,284][122664] Batcher 0 profile tree view: batching: 170.9893, releasing_batches: 0.0897 [2023-10-10 20:24:10,285][122664] Batcher 1 profile tree view: batching: 170.8556, releasing_batches: 0.0895 [2023-10-10 20:24:10,286][122664] InferenceWorker_p0-w0 profile tree view: wait_policy: 0.0001 wait_policy_total: 1748.8900 update_model: 199.6336 weight_update: 0.0010 one_step: 0.0034 handle_policy_step: 11184.4289 deserialize: 62.1760, stack: 193.1575, obs_to_device_normalize: 2496.8057, forward: 5048.0045, prepare_outputs: 2446.0654, send_messages: 455.6494 [2023-10-10 20:24:10,286][122664] InferenceWorker_p1-w0 profile tree view: wait_policy: 0.0001 wait_policy_total: 1774.2973 update_model: 198.8056 weight_update: 0.0009 one_step: 0.0024 handle_policy_step: 11156.8133 deserialize: 63.3641, stack: 189.6431, obs_to_device_normalize: 2496.4523, forward: 5015.8154, prepare_outputs: 2450.7877, send_messages: 462.9721 [2023-10-10 20:24:10,287][122664] Learner 0 profile tree view: misc: 0.0180, prepare_batch: 262.3940 train: 3651.6544 epoch_init: 0.1969, minibatch_init: 12.7285, losses_postprocess: 900.8035, kl_divergence: 31.4534, update: 388.6203, after_optimizer: 2137.1608 calculate_losses: 164.0933 losses_init: 0.3560, forward_head: 55.4326, bptt_initial: 1.3997, bptt: 1.7702, tail: 37.6337, advantages_returns: 11.0041, losses: 43.1662 [2023-10-10 20:24:10,287][122664] Learner 1 profile tree view: misc: 0.0183, prepare_batch: 261.5321 train: 3615.1920 epoch_init: 0.1877, minibatch_init: 13.1338, losses_postprocess: 892.5370, kl_divergence: 30.8263, update: 386.9173, after_optimizer: 2105.6091 calculate_losses: 169.2643 losses_init: 0.4576, forward_head: 59.1640, bptt_initial: 1.4301, bptt: 1.9433, tail: 37.6904, advantages_returns: 10.9748, losses: 43.9987 [2023-10-10 20:24:10,287][122664] RolloutWorker_w0 profile tree view: wait_for_trajectories: 1.2373, enqueue_policy_requests: 408.9998, process_policy_outputs: 189.8310, env_step: 6519.6660, finalize_trajectories: 3.3369, complete_rollouts: 2.9629 post_env_step: 374.9026 process_env_step: 83.3732 [2023-10-10 20:24:10,287][122664] RolloutWorker_w15 profile tree view: wait_for_trajectories: 1.2238, enqueue_policy_requests: 405.7348, process_policy_outputs: 193.1953, env_step: 6404.0420, finalize_trajectories: 3.4198, complete_rollouts: 2.9672 post_env_step: 380.3826 process_env_step: 84.8490 [2023-10-10 20:24:10,288][122664] Loop Runner_EvtLoop terminating... [2023-10-10 20:24:10,288][122664] Runner profile tree view: main_loop: 13811.8958 [2023-10-10 20:24:10,288][122664] Collected {0: 100139008, 1: 100007936}, FPS: 14490.9