[2023-10-14 17:33:47,313][60425] Saving configuration to ./train_atari/atari_roadrunner_APPO/config.json... [2023-10-14 17:33:47,630][60425] Rollout worker 0 uses device cpu [2023-10-14 17:33:47,631][60425] Rollout worker 1 uses device cpu [2023-10-14 17:33:47,632][60425] Rollout worker 2 uses device cpu [2023-10-14 17:33:47,632][60425] Rollout worker 3 uses device cpu [2023-10-14 17:33:47,633][60425] Rollout worker 4 uses device cpu [2023-10-14 17:33:47,633][60425] Rollout worker 5 uses device cpu [2023-10-14 17:33:47,634][60425] Rollout worker 6 uses device cpu [2023-10-14 17:33:47,634][60425] Rollout worker 7 uses device cpu [2023-10-14 17:33:47,635][60425] Rollout worker 8 uses device cpu [2023-10-14 17:33:47,635][60425] Rollout worker 9 uses device cpu [2023-10-14 17:33:47,635][60425] Rollout worker 10 uses device cpu [2023-10-14 17:33:47,636][60425] Rollout worker 11 uses device cpu [2023-10-14 17:33:47,636][60425] Rollout worker 12 uses device cpu [2023-10-14 17:33:47,637][60425] Rollout worker 13 uses device cpu [2023-10-14 17:33:47,637][60425] Rollout worker 14 uses device cpu [2023-10-14 17:33:47,638][60425] Rollout worker 15 uses device cpu [2023-10-14 17:33:47,925][60425] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-10-14 17:33:47,925][60425] InferenceWorker_p0-w0: min num requests: 2 [2023-10-14 17:33:47,928][60425] Using GPUs [1] for process 1 (actually maps to GPUs [1]) [2023-10-14 17:33:47,929][60425] InferenceWorker_p1-w0: min num requests: 2 [2023-10-14 17:33:47,975][60425] Starting all processes... [2023-10-14 17:33:47,975][60425] Starting process learner_proc0 [2023-10-14 17:33:49,663][60425] Starting process learner_proc1 [2023-10-14 17:33:49,667][61172] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-10-14 17:33:49,667][61172] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 [2023-10-14 17:33:49,685][61172] Num visible devices: 1 [2023-10-14 17:33:49,707][61172] Setting fixed seed 1234 [2023-10-14 17:33:49,708][61172] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-10-14 17:33:49,708][61172] Initializing actor-critic model on device cuda:0 [2023-10-14 17:33:49,709][61172] RunningMeanStd input shape: (4, 84, 84) [2023-10-14 17:33:49,709][61172] RunningMeanStd input shape: (1,) [2023-10-14 17:33:49,720][61172] ConvEncoder: input_channels=4 [2023-10-14 17:33:49,899][61172] Conv encoder output size: 512 [2023-10-14 17:33:49,901][61172] Created Actor Critic model with architecture: [2023-10-14 17:33:49,901][61172] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): MultiInputEncoder( (encoders): ModuleDict( (obs): ConvEncoder( (enc): RecursiveScriptModule( original_name=ConvEncoderImpl (conv_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Conv2d) (1): RecursiveScriptModule(original_name=ReLU) (2): RecursiveScriptModule(original_name=Conv2d) (3): RecursiveScriptModule(original_name=ReLU) (4): RecursiveScriptModule(original_name=Conv2d) (5): RecursiveScriptModule(original_name=ReLU) ) (mlp_layers): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=ReLU) ) ) ) ) ) (core): ModelCoreIdentity() (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=512, out_features=1, bias=True) (action_parameterization): ActionParameterizationDefault( (distribution_linear): Linear(in_features=512, out_features=18, bias=True) ) ) [2023-10-14 17:33:50,474][61172] Using optimizer [2023-10-14 17:33:50,475][61172] No checkpoints found [2023-10-14 17:33:50,475][61172] Did not load from checkpoint, starting from scratch! [2023-10-14 17:33:50,475][61172] Initialized policy 0 weights for model version 0 [2023-10-14 17:33:50,477][61172] LearnerWorker_p0 finished initialization! [2023-10-14 17:33:50,477][61172] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-10-14 17:33:51,405][60425] Starting all processes... [2023-10-14 17:33:51,408][61248] Using GPUs [1] for process 1 (actually maps to GPUs [1]) [2023-10-14 17:33:51,409][61248] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [1]) for learning process 1 [2023-10-14 17:33:51,414][60425] Starting process inference_proc0-0 [2023-10-14 17:33:51,415][60425] Starting process inference_proc1-0 [2023-10-14 17:33:51,415][60425] Starting process rollout_proc0 [2023-10-14 17:33:51,415][60425] Starting process rollout_proc1 [2023-10-14 17:33:51,427][61248] Num visible devices: 1 [2023-10-14 17:33:51,415][60425] Starting process rollout_proc2 [2023-10-14 17:33:51,443][61248] Setting fixed seed 1234 [2023-10-14 17:33:51,416][60425] Starting process rollout_proc3 [2023-10-14 17:33:51,445][61248] Using GPUs [0] for process 1 (actually maps to GPUs [1]) [2023-10-14 17:33:51,445][61248] Initializing actor-critic model on device cuda:0 [2023-10-14 17:33:51,445][61248] RunningMeanStd input shape: (4, 84, 84) [2023-10-14 17:33:51,446][61248] RunningMeanStd input shape: (1,) [2023-10-14 17:33:51,416][60425] Starting process rollout_proc4 [2023-10-14 17:33:51,416][60425] Starting process rollout_proc5 [2023-10-14 17:33:51,417][60425] Starting process rollout_proc6 [2023-10-14 17:33:51,417][60425] Starting process rollout_proc7 [2023-10-14 17:33:51,459][61248] ConvEncoder: input_channels=4 [2023-10-14 17:33:51,417][60425] Starting process rollout_proc8 [2023-10-14 17:33:51,421][60425] Starting process rollout_proc9 [2023-10-14 17:33:51,422][60425] Starting process rollout_proc10 [2023-10-14 17:33:51,422][60425] Starting process rollout_proc11 [2023-10-14 17:33:51,422][60425] Starting process rollout_proc12 [2023-10-14 17:33:51,423][60425] Starting process rollout_proc13 [2023-10-14 17:33:51,880][61248] Conv encoder output size: 512 [2023-10-14 17:33:51,883][61248] Created Actor Critic model with architecture: [2023-10-14 17:33:51,884][61248] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): MultiInputEncoder( (encoders): ModuleDict( (obs): ConvEncoder( (enc): RecursiveScriptModule( original_name=ConvEncoderImpl (conv_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Conv2d) (1): RecursiveScriptModule(original_name=ReLU) (2): RecursiveScriptModule(original_name=Conv2d) (3): RecursiveScriptModule(original_name=ReLU) (4): RecursiveScriptModule(original_name=Conv2d) (5): RecursiveScriptModule(original_name=ReLU) ) (mlp_layers): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=ReLU) ) ) ) ) ) (core): ModelCoreIdentity() (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=512, out_features=1, bias=True) (action_parameterization): ActionParameterizationDefault( (distribution_linear): Linear(in_features=512, out_features=18, bias=True) ) ) [2023-10-14 17:33:52,570][61248] Using optimizer [2023-10-14 17:33:52,571][61248] No checkpoints found [2023-10-14 17:33:52,571][61248] Did not load from checkpoint, starting from scratch! [2023-10-14 17:33:52,571][61248] Initialized policy 1 weights for model version 0 [2023-10-14 17:33:52,573][61248] LearnerWorker_p1 finished initialization! [2023-10-14 17:33:52,573][61248] Using GPUs [0] for process 1 (actually maps to GPUs [1]) [2023-10-14 17:33:53,617][60425] Starting process rollout_proc14 [2023-10-14 17:33:53,623][61592] Worker 6 uses CPU cores [12, 13] [2023-10-14 17:33:53,627][60425] Starting process rollout_proc15 [2023-10-14 17:33:53,633][61589] Worker 3 uses CPU cores [6, 7] [2023-10-14 17:33:53,635][61584] Worker 0 uses CPU cores [0, 1] [2023-10-14 17:33:53,649][61596] Worker 9 uses CPU cores [18, 19] [2023-10-14 17:33:53,653][61585] Using GPUs [1] for process 1 (actually maps to GPUs [1]) [2023-10-14 17:33:53,653][61585] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [1]) for inference process 1 [2023-10-14 17:33:53,671][61585] Num visible devices: 1 [2023-10-14 17:33:53,855][61552] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-10-14 17:33:53,856][61552] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 [2023-10-14 17:33:53,874][61552] Num visible devices: 1 [2023-10-14 17:33:53,993][61590] Worker 5 uses CPU cores [10, 11] [2023-10-14 17:33:53,999][61593] Worker 8 uses CPU cores [16, 17] [2023-10-14 17:33:54,009][61599] Worker 13 uses CPU cores [26, 27] [2023-10-14 17:33:54,039][61588] Worker 2 uses CPU cores [4, 5] [2023-10-14 17:33:54,075][61594] Worker 7 uses CPU cores [14, 15] [2023-10-14 17:33:54,101][61595] Worker 10 uses CPU cores [20, 21] [2023-10-14 17:33:54,120][61587] Worker 1 uses CPU cores [2, 3] [2023-10-14 17:33:54,201][61598] Worker 12 uses CPU cores [24, 25] [2023-10-14 17:33:54,229][61597] Worker 11 uses CPU cores [22, 23] [2023-10-14 17:33:54,246][61591] Worker 4 uses CPU cores [8, 9] [2023-10-14 17:33:54,514][61552] RunningMeanStd input shape: (4, 84, 84) [2023-10-14 17:33:54,514][61552] RunningMeanStd input shape: (1,) [2023-10-14 17:33:54,526][61552] ConvEncoder: input_channels=4 [2023-10-14 17:33:54,557][61585] RunningMeanStd input shape: (4, 84, 84) [2023-10-14 17:33:54,558][61585] RunningMeanStd input shape: (1,) [2023-10-14 17:33:54,570][61585] ConvEncoder: input_channels=4 [2023-10-14 17:33:54,628][61552] Conv encoder output size: 512 [2023-10-14 17:33:54,672][61585] Conv encoder output size: 512 [2023-10-14 17:33:55,592][62147] Worker 14 uses CPU cores [28, 29] [2023-10-14 17:33:55,624][60425] Inference worker 0-0 is ready! [2023-10-14 17:33:55,625][60425] Inference worker 1-0 is ready! [2023-10-14 17:33:55,625][62179] Worker 15 uses CPU cores [30, 31] [2023-10-14 17:33:55,626][60425] All inference workers are ready! Signal rollout workers to start! [2023-10-14 17:33:55,627][61592] EnvRunner 6-0 uses policy 0 [2023-10-14 17:33:55,627][61588] EnvRunner 2-0 uses policy 0 [2023-10-14 17:33:55,627][61593] EnvRunner 8-0 uses policy 0 [2023-10-14 17:33:55,627][61587] EnvRunner 1-0 uses policy 1 [2023-10-14 17:33:55,627][61584] EnvRunner 0-0 uses policy 0 [2023-10-14 17:33:55,627][61598] EnvRunner 12-0 uses policy 0 [2023-10-14 17:33:55,627][61599] EnvRunner 13-0 uses policy 1 [2023-10-14 17:33:55,627][61597] EnvRunner 11-0 uses policy 1 [2023-10-14 17:33:55,628][61591] EnvRunner 4-0 uses policy 0 [2023-10-14 17:33:55,628][61594] EnvRunner 7-0 uses policy 1 [2023-10-14 17:33:55,628][61595] EnvRunner 10-0 uses policy 0 [2023-10-14 17:33:55,628][61596] EnvRunner 9-0 uses policy 1 [2023-10-14 17:33:55,628][61590] EnvRunner 5-0 uses policy 1 [2023-10-14 17:33:55,628][61589] EnvRunner 3-0 uses policy 1 [2023-10-14 17:33:55,628][60425] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan, 1: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-10-14 17:33:55,806][62147] EnvRunner 14-0 uses policy 0 [2023-10-14 17:33:55,826][62179] EnvRunner 15-0 uses policy 1 [2023-10-14 17:33:57,912][60425] Heartbeat connected on Batcher_0 [2023-10-14 17:33:57,915][60425] Heartbeat connected on LearnerWorker_p0 [2023-10-14 17:33:57,918][60425] Heartbeat connected on Batcher_1 [2023-10-14 17:33:57,921][60425] Heartbeat connected on LearnerWorker_p1 [2023-10-14 17:33:57,929][60425] Heartbeat connected on InferenceWorker_p0-w0 [2023-10-14 17:33:57,931][60425] Heartbeat connected on InferenceWorker_p1-w0 [2023-10-14 17:33:57,932][60425] Heartbeat connected on RolloutWorker_w0 [2023-10-14 17:33:57,939][60425] Heartbeat connected on RolloutWorker_w1 [2023-10-14 17:33:57,940][60425] Heartbeat connected on RolloutWorker_w3 [2023-10-14 17:33:57,943][60425] Heartbeat connected on RolloutWorker_w2 [2023-10-14 17:33:57,948][60425] Heartbeat connected on RolloutWorker_w5 [2023-10-14 17:33:57,948][60425] Heartbeat connected on RolloutWorker_w4 [2023-10-14 17:33:57,948][60425] Heartbeat connected on RolloutWorker_w6 [2023-10-14 17:33:57,951][60425] Heartbeat connected on RolloutWorker_w7 [2023-10-14 17:33:57,954][60425] Heartbeat connected on RolloutWorker_w8 [2023-10-14 17:33:57,960][60425] Heartbeat connected on RolloutWorker_w10 [2023-10-14 17:33:57,961][60425] Heartbeat connected on RolloutWorker_w9 [2023-10-14 17:33:57,963][60425] Heartbeat connected on RolloutWorker_w11 [2023-10-14 17:33:57,968][60425] Heartbeat connected on RolloutWorker_w12 [2023-10-14 17:33:57,973][60425] Heartbeat connected on RolloutWorker_w13 [2023-10-14 17:33:57,974][60425] Heartbeat connected on RolloutWorker_w15 [2023-10-14 17:33:57,974][60425] Heartbeat connected on RolloutWorker_w14 [2023-10-14 17:33:58,343][60425] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 402.1, 1: 313.7. Samples: 1944. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-10-14 17:33:58,344][60425] Avg episode reward: [(0, '0.000'), (1, '0.000')] [2023-10-14 17:34:03,343][60425] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 849.2, 1: 814.4. Samples: 12836. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-10-14 17:34:03,344][60425] Avg episode reward: [(0, '0.010'), (1, '0.010')] [2023-10-14 17:34:05,736][61552] Updated weights for policy 0, policy_version 10 (0.0007) [2023-10-14 17:34:06,104][61552] Updated weights for policy 0, policy_version 20 (0.0008) [2023-10-14 17:34:06,456][61585] Updated weights for policy 1, policy_version 10 (0.0008) [2023-10-14 17:34:06,477][61552] Updated weights for policy 0, policy_version 30 (0.0009) [2023-10-14 17:34:06,814][61585] Updated weights for policy 1, policy_version 20 (0.0010) [2023-10-14 17:34:07,186][61585] Updated weights for policy 1, policy_version 30 (0.0008) [2023-10-14 17:34:08,343][60425] Fps is (10 sec: 6553.5, 60 sec: 5153.9, 300 sec: 5153.9). Total num frames: 65536. Throughput: 0: 1125.8, 1: 1161.1. Samples: 29080. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-14 17:34:08,344][60425] Avg episode reward: [(0, '0.050'), (1, '0.060')] [2023-10-14 17:34:09,189][61552] Updated weights for policy 0, policy_version 40 (0.0007) [2023-10-14 17:34:09,558][61552] Updated weights for policy 0, policy_version 50 (0.0007) [2023-10-14 17:34:09,578][61585] Updated weights for policy 1, policy_version 40 (0.0007) [2023-10-14 17:34:09,920][61552] Updated weights for policy 0, policy_version 60 (0.0009) [2023-10-14 17:34:09,939][61585] Updated weights for policy 1, policy_version 50 (0.0008) [2023-10-14 17:34:10,293][61585] Updated weights for policy 1, policy_version 60 (0.0010) [2023-10-14 17:34:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 7398.6, 300 sec: 7398.6). Total num frames: 131072. Throughput: 0: 1375.2, 1: 1342.8. Samples: 48150. Policy #0 lag: (min: 33.0, avg: 33.0, max: 33.0) [2023-10-14 17:34:13,344][60425] Avg episode reward: [(0, '0.090'), (1, '0.030')] [2023-10-14 17:34:13,741][61552] Updated weights for policy 0, policy_version 70 (0.0007) [2023-10-14 17:34:14,116][61552] Updated weights for policy 0, policy_version 80 (0.0007) [2023-10-14 17:34:14,246][61585] Updated weights for policy 1, policy_version 70 (0.0008) [2023-10-14 17:34:14,487][61552] Updated weights for policy 0, policy_version 90 (0.0007) [2023-10-14 17:34:14,612][61585] Updated weights for policy 1, policy_version 80 (0.0007) [2023-10-14 17:34:14,976][61585] Updated weights for policy 1, policy_version 90 (0.0007) [2023-10-14 17:34:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 8655.1, 300 sec: 8655.1). Total num frames: 196608. Throughput: 0: 1267.8, 1: 1242.9. Samples: 57032. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-14 17:34:18,344][60425] Avg episode reward: [(0, '0.060'), (1, '0.090')] [2023-10-14 17:34:18,604][61552] Updated weights for policy 0, policy_version 100 (0.0007) [2023-10-14 17:34:18,978][61585] Updated weights for policy 1, policy_version 100 (0.0008) [2023-10-14 17:34:18,984][61552] Updated weights for policy 0, policy_version 110 (0.0009) [2023-10-14 17:34:19,337][61585] Updated weights for policy 1, policy_version 110 (0.0008) [2023-10-14 17:34:19,360][61552] Updated weights for policy 0, policy_version 120 (0.0008) [2023-10-14 17:34:19,698][61585] Updated weights for policy 1, policy_version 120 (0.0008) [2023-10-14 17:34:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 9458.3, 300 sec: 9458.3). Total num frames: 262144. Throughput: 0: 1389.8, 1: 1368.9. Samples: 76460. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-14 17:34:23,344][60425] Avg episode reward: [(0, '0.160'), (1, '0.060')] [2023-10-14 17:34:23,346][61172] Saving new best policy, reward=0.160! [2023-10-14 17:34:23,346][61248] Saving new best policy, reward=0.060! [2023-10-14 17:34:23,659][61552] Updated weights for policy 0, policy_version 130 (0.0008) [2023-10-14 17:34:24,022][61552] Updated weights for policy 0, policy_version 140 (0.0009) [2023-10-14 17:34:24,049][61585] Updated weights for policy 1, policy_version 130 (0.0009) [2023-10-14 17:34:24,385][61552] Updated weights for policy 0, policy_version 150 (0.0008) [2023-10-14 17:34:24,414][61585] Updated weights for policy 1, policy_version 140 (0.0008) [2023-10-14 17:34:24,758][61552] Updated weights for policy 0, policy_version 160 (0.0008) [2023-10-14 17:34:24,781][61585] Updated weights for policy 1, policy_version 150 (0.0009) [2023-10-14 17:34:25,154][61585] Updated weights for policy 1, policy_version 160 (0.0008) [2023-10-14 17:34:28,343][60425] Fps is (10 sec: 13107.1, 60 sec: 10015.9, 300 sec: 10015.9). Total num frames: 327680. Throughput: 0: 1481.2, 1: 1460.9. Samples: 96254. Policy #0 lag: (min: 31.0, avg: 43.3, max: 63.0) [2023-10-14 17:34:28,344][60425] Avg episode reward: [(0, '0.150'), (1, '0.090')] [2023-10-14 17:34:28,356][61248] Saving new best policy, reward=0.090! [2023-10-14 17:34:29,151][61552] Updated weights for policy 0, policy_version 170 (0.0007) [2023-10-14 17:34:29,519][61552] Updated weights for policy 0, policy_version 180 (0.0007) [2023-10-14 17:34:29,591][61585] Updated weights for policy 1, policy_version 170 (0.0008) [2023-10-14 17:34:29,885][61552] Updated weights for policy 0, policy_version 190 (0.0007) [2023-10-14 17:34:29,956][61585] Updated weights for policy 1, policy_version 180 (0.0008) [2023-10-14 17:34:30,322][61585] Updated weights for policy 1, policy_version 190 (0.0008) [2023-10-14 17:34:33,343][60425] Fps is (10 sec: 13107.3, 60 sec: 10425.8, 300 sec: 10425.8). Total num frames: 393216. Throughput: 0: 1399.0, 1: 1382.4. Samples: 104904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:34:33,344][60425] Avg episode reward: [(0, '0.080'), (1, '0.180')] [2023-10-14 17:34:33,345][61248] Saving new best policy, reward=0.180! [2023-10-14 17:34:34,221][61552] Updated weights for policy 0, policy_version 200 (0.0010) [2023-10-14 17:34:34,476][61585] Updated weights for policy 1, policy_version 200 (0.0010) [2023-10-14 17:34:34,589][61552] Updated weights for policy 0, policy_version 210 (0.0008) [2023-10-14 17:34:34,846][61585] Updated weights for policy 1, policy_version 210 (0.0008) [2023-10-14 17:34:34,959][61552] Updated weights for policy 0, policy_version 220 (0.0007) [2023-10-14 17:34:35,217][61585] Updated weights for policy 1, policy_version 220 (0.0007) [2023-10-14 17:34:38,343][60425] Fps is (10 sec: 13107.5, 60 sec: 10739.7, 300 sec: 10739.7). Total num frames: 458752. Throughput: 0: 1465.3, 1: 1456.4. Samples: 124802. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-14 17:34:38,344][60425] Avg episode reward: [(0, '0.260'), (1, '0.420')] [2023-10-14 17:34:38,344][61172] Saving new best policy, reward=0.260! [2023-10-14 17:34:38,344][61248] Saving new best policy, reward=0.420! [2023-10-14 17:34:39,295][61552] Updated weights for policy 0, policy_version 230 (0.0007) [2023-10-14 17:34:39,419][61585] Updated weights for policy 1, policy_version 230 (0.0008) [2023-10-14 17:34:39,662][61552] Updated weights for policy 0, policy_version 240 (0.0007) [2023-10-14 17:34:39,775][61585] Updated weights for policy 1, policy_version 240 (0.0008) [2023-10-14 17:34:40,030][61552] Updated weights for policy 0, policy_version 250 (0.0008) [2023-10-14 17:34:40,148][61585] Updated weights for policy 1, policy_version 250 (0.0010) [2023-10-14 17:34:43,343][60425] Fps is (10 sec: 13107.2, 60 sec: 10987.7, 300 sec: 10987.7). Total num frames: 524288. Throughput: 0: 1592.9, 1: 1584.7. Samples: 144936. Policy #0 lag: (min: 4.0, avg: 12.9, max: 36.0) [2023-10-14 17:34:43,344][60425] Avg episode reward: [(0, '0.310'), (1, '0.270')] [2023-10-14 17:34:43,352][61172] Saving new best policy, reward=0.310! [2023-10-14 17:34:44,111][61552] Updated weights for policy 0, policy_version 260 (0.0008) [2023-10-14 17:34:44,339][61585] Updated weights for policy 1, policy_version 260 (0.0009) [2023-10-14 17:34:44,471][61552] Updated weights for policy 0, policy_version 270 (0.0009) [2023-10-14 17:34:44,691][61585] Updated weights for policy 1, policy_version 270 (0.0008) [2023-10-14 17:34:44,844][61552] Updated weights for policy 0, policy_version 280 (0.0009) [2023-10-14 17:34:45,059][61585] Updated weights for policy 1, policy_version 280 (0.0007) [2023-10-14 17:34:48,343][60425] Fps is (10 sec: 13107.1, 60 sec: 11188.8, 300 sec: 11188.8). Total num frames: 589824. Throughput: 0: 1570.3, 1: 1563.1. Samples: 153838. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-10-14 17:34:48,344][60425] Avg episode reward: [(0, '0.540'), (1, '0.250')] [2023-10-14 17:34:48,345][61172] Saving new best policy, reward=0.540! [2023-10-14 17:34:48,934][61552] Updated weights for policy 0, policy_version 290 (0.0008) [2023-10-14 17:34:49,178][61585] Updated weights for policy 1, policy_version 290 (0.0009) [2023-10-14 17:34:49,307][61552] Updated weights for policy 0, policy_version 300 (0.0008) [2023-10-14 17:34:49,551][61585] Updated weights for policy 1, policy_version 300 (0.0009) [2023-10-14 17:34:49,680][61552] Updated weights for policy 0, policy_version 310 (0.0008) [2023-10-14 17:34:49,906][61585] Updated weights for policy 1, policy_version 310 (0.0009) [2023-10-14 17:34:50,048][61552] Updated weights for policy 0, policy_version 320 (0.0009) [2023-10-14 17:34:50,271][61585] Updated weights for policy 1, policy_version 320 (0.0009) [2023-10-14 17:34:53,343][60425] Fps is (10 sec: 13107.4, 60 sec: 11355.0, 300 sec: 11355.0). Total num frames: 655360. Throughput: 0: 1617.9, 1: 1597.0. Samples: 173750. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:34:53,344][60425] Avg episode reward: [(0, '0.720'), (1, '0.560')] [2023-10-14 17:34:53,344][61172] Saving new best policy, reward=0.720! [2023-10-14 17:34:53,344][61248] Saving new best policy, reward=0.560! [2023-10-14 17:34:54,186][61552] Updated weights for policy 0, policy_version 330 (0.0009) [2023-10-14 17:34:54,536][61585] Updated weights for policy 1, policy_version 330 (0.0009) [2023-10-14 17:34:54,545][61552] Updated weights for policy 0, policy_version 340 (0.0008) [2023-10-14 17:34:54,902][61585] Updated weights for policy 1, policy_version 340 (0.0008) [2023-10-14 17:34:54,915][61552] Updated weights for policy 0, policy_version 350 (0.0008) [2023-10-14 17:34:55,270][61585] Updated weights for policy 1, policy_version 350 (0.0007) [2023-10-14 17:34:58,344][60425] Fps is (10 sec: 13106.9, 60 sec: 12014.9, 300 sec: 11494.6). Total num frames: 720896. Throughput: 0: 1621.3, 1: 1620.8. Samples: 194042. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-14 17:34:58,345][60425] Avg episode reward: [(0, '0.810'), (1, '0.890')] [2023-10-14 17:34:58,354][61172] Saving new best policy, reward=0.810! [2023-10-14 17:34:58,355][61248] Saving new best policy, reward=0.890! [2023-10-14 17:34:59,104][61552] Updated weights for policy 0, policy_version 360 (0.0007) [2023-10-14 17:34:59,449][61585] Updated weights for policy 1, policy_version 360 (0.0008) [2023-10-14 17:34:59,479][61552] Updated weights for policy 0, policy_version 370 (0.0007) [2023-10-14 17:34:59,808][61585] Updated weights for policy 1, policy_version 370 (0.0009) [2023-10-14 17:34:59,847][61552] Updated weights for policy 0, policy_version 380 (0.0008) [2023-10-14 17:35:00,172][61585] Updated weights for policy 1, policy_version 380 (0.0007) [2023-10-14 17:35:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 11613.7). Total num frames: 786432. Throughput: 0: 1623.7, 1: 1620.1. Samples: 203004. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-14 17:35:03,344][60425] Avg episode reward: [(0, '0.900'), (1, '1.030')] [2023-10-14 17:35:03,344][61248] Saving new best policy, reward=1.030! [2023-10-14 17:35:03,344][61172] Saving new best policy, reward=0.900! [2023-10-14 17:35:04,073][61552] Updated weights for policy 0, policy_version 390 (0.0008) [2023-10-14 17:35:04,313][61585] Updated weights for policy 1, policy_version 390 (0.0009) [2023-10-14 17:35:04,442][61552] Updated weights for policy 0, policy_version 400 (0.0009) [2023-10-14 17:35:04,682][61585] Updated weights for policy 1, policy_version 400 (0.0008) [2023-10-14 17:35:04,810][61552] Updated weights for policy 0, policy_version 410 (0.0008) [2023-10-14 17:35:05,043][61585] Updated weights for policy 1, policy_version 410 (0.0010) [2023-10-14 17:35:08,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 11716.4). Total num frames: 851968. Throughput: 0: 1632.1, 1: 1626.8. Samples: 223112. Policy #0 lag: (min: 22.0, avg: 22.9, max: 40.0) [2023-10-14 17:35:08,344][60425] Avg episode reward: [(0, '1.160'), (1, '1.320')] [2023-10-14 17:35:08,345][61248] Saving new best policy, reward=1.320! [2023-10-14 17:35:08,344][61172] Saving new best policy, reward=1.160! [2023-10-14 17:35:08,978][61552] Updated weights for policy 0, policy_version 420 (0.0007) [2023-10-14 17:35:09,219][61585] Updated weights for policy 1, policy_version 420 (0.0008) [2023-10-14 17:35:09,348][61552] Updated weights for policy 0, policy_version 430 (0.0007) [2023-10-14 17:35:09,579][61585] Updated weights for policy 1, policy_version 430 (0.0008) [2023-10-14 17:35:09,725][61552] Updated weights for policy 0, policy_version 440 (0.0009) [2023-10-14 17:35:09,939][61585] Updated weights for policy 1, policy_version 440 (0.0007) [2023-10-14 17:35:13,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 11805.9). Total num frames: 917504. Throughput: 0: 1631.7, 1: 1641.6. Samples: 243556. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:35:13,344][60425] Avg episode reward: [(0, '1.550'), (1, '1.610')] [2023-10-14 17:35:13,354][61172] Saving new best policy, reward=1.550! [2023-10-14 17:35:13,354][61248] Saving new best policy, reward=1.610! [2023-10-14 17:35:13,876][61552] Updated weights for policy 0, policy_version 450 (0.0008) [2023-10-14 17:35:14,240][61585] Updated weights for policy 1, policy_version 450 (0.0008) [2023-10-14 17:35:14,253][61552] Updated weights for policy 0, policy_version 460 (0.0009) [2023-10-14 17:35:14,605][61585] Updated weights for policy 1, policy_version 460 (0.0008) [2023-10-14 17:35:14,620][61552] Updated weights for policy 0, policy_version 470 (0.0008) [2023-10-14 17:35:14,974][61585] Updated weights for policy 1, policy_version 470 (0.0009) [2023-10-14 17:35:14,984][61552] Updated weights for policy 0, policy_version 480 (0.0010) [2023-10-14 17:35:15,337][61585] Updated weights for policy 1, policy_version 480 (0.0009) [2023-10-14 17:35:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 11884.6). Total num frames: 983040. Throughput: 0: 1634.9, 1: 1641.9. Samples: 252360. Policy #0 lag: (min: 1.0, avg: 7.6, max: 33.0) [2023-10-14 17:35:18,344][60425] Avg episode reward: [(0, '1.710'), (1, '1.520')] [2023-10-14 17:35:18,344][61172] Saving new best policy, reward=1.710! [2023-10-14 17:35:19,089][61552] Updated weights for policy 0, policy_version 490 (0.0010) [2023-10-14 17:35:19,431][61585] Updated weights for policy 1, policy_version 490 (0.0008) [2023-10-14 17:35:19,464][61552] Updated weights for policy 0, policy_version 500 (0.0008) [2023-10-14 17:35:19,792][61585] Updated weights for policy 1, policy_version 500 (0.0008) [2023-10-14 17:35:19,830][61552] Updated weights for policy 0, policy_version 510 (0.0007) [2023-10-14 17:35:20,164][61585] Updated weights for policy 1, policy_version 510 (0.0007) [2023-10-14 17:35:23,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 11954.2). Total num frames: 1048576. Throughput: 0: 1641.2, 1: 1646.6. Samples: 272756. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-14 17:35:23,345][60425] Avg episode reward: [(0, '1.680'), (1, '1.400')] [2023-10-14 17:35:24,088][61552] Updated weights for policy 0, policy_version 520 (0.0008) [2023-10-14 17:35:24,287][61585] Updated weights for policy 1, policy_version 520 (0.0008) [2023-10-14 17:35:24,455][61552] Updated weights for policy 0, policy_version 530 (0.0007) [2023-10-14 17:35:24,659][61585] Updated weights for policy 1, policy_version 530 (0.0008) [2023-10-14 17:35:24,827][61552] Updated weights for policy 0, policy_version 540 (0.0009) [2023-10-14 17:35:25,029][61585] Updated weights for policy 1, policy_version 540 (0.0008) [2023-10-14 17:35:28,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 12016.4). Total num frames: 1114112. Throughput: 0: 1640.5, 1: 1649.9. Samples: 293002. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) [2023-10-14 17:35:28,344][60425] Avg episode reward: [(0, '1.830'), (1, '1.520')] [2023-10-14 17:35:28,355][61172] Saving new best policy, reward=1.830! [2023-10-14 17:35:28,946][61552] Updated weights for policy 0, policy_version 550 (0.0008) [2023-10-14 17:35:29,208][61585] Updated weights for policy 1, policy_version 550 (0.0007) [2023-10-14 17:35:29,313][61552] Updated weights for policy 0, policy_version 560 (0.0007) [2023-10-14 17:35:29,568][61585] Updated weights for policy 1, policy_version 560 (0.0009) [2023-10-14 17:35:29,680][61552] Updated weights for policy 0, policy_version 570 (0.0007) [2023-10-14 17:35:29,932][61585] Updated weights for policy 1, policy_version 570 (0.0008) [2023-10-14 17:35:33,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 12072.2). Total num frames: 1179648. Throughput: 0: 1643.3, 1: 1651.5. Samples: 302102. Policy #0 lag: (min: 31.0, avg: 32.7, max: 60.0) [2023-10-14 17:35:33,344][60425] Avg episode reward: [(0, '2.260'), (1, '1.720')] [2023-10-14 17:35:33,345][61248] Saving new best policy, reward=1.720! [2023-10-14 17:35:33,592][61552] Updated weights for policy 0, policy_version 580 (0.0008) [2023-10-14 17:35:33,972][61552] Updated weights for policy 0, policy_version 590 (0.0009) [2023-10-14 17:35:33,987][61585] Updated weights for policy 1, policy_version 580 (0.0008) [2023-10-14 17:35:34,338][61552] Updated weights for policy 0, policy_version 600 (0.0007) [2023-10-14 17:35:34,356][61585] Updated weights for policy 1, policy_version 590 (0.0010) [2023-10-14 17:35:34,640][61172] Saving new best policy, reward=2.260! [2023-10-14 17:35:34,718][61585] Updated weights for policy 1, policy_version 600 (0.0009) [2023-10-14 17:35:38,344][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 12122.6). Total num frames: 1245184. Throughput: 0: 1647.2, 1: 1657.1. Samples: 322446. Policy #0 lag: (min: 17.0, avg: 30.5, max: 49.0) [2023-10-14 17:35:38,345][60425] Avg episode reward: [(0, '2.810'), (1, '2.190')] [2023-10-14 17:35:38,346][61248] Saving new best policy, reward=2.190! [2023-10-14 17:35:38,472][61552] Updated weights for policy 0, policy_version 610 (0.0009) [2023-10-14 17:35:38,837][61552] Updated weights for policy 0, policy_version 620 (0.0007) [2023-10-14 17:35:38,872][61585] Updated weights for policy 1, policy_version 610 (0.0008) [2023-10-14 17:35:39,219][61552] Updated weights for policy 0, policy_version 630 (0.0009) [2023-10-14 17:35:39,283][61585] Updated weights for policy 1, policy_version 620 (0.0008) [2023-10-14 17:35:39,585][61172] Saving new best policy, reward=2.810! [2023-10-14 17:35:39,588][61552] Updated weights for policy 0, policy_version 640 (0.0008) [2023-10-14 17:35:39,648][61585] Updated weights for policy 1, policy_version 630 (0.0009) [2023-10-14 17:35:40,004][61585] Updated weights for policy 1, policy_version 640 (0.0009) [2023-10-14 17:35:43,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 12168.3). Total num frames: 1310720. Throughput: 0: 1648.5, 1: 1654.3. Samples: 342666. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-14 17:35:43,345][60425] Avg episode reward: [(0, '3.370'), (1, '2.420')] [2023-10-14 17:35:43,358][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000000640_655360.pth... [2023-10-14 17:35:43,358][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000000640_655360.pth... [2023-10-14 17:35:43,387][61172] Saving new best policy, reward=3.370! [2023-10-14 17:35:43,393][61248] Saving new best policy, reward=2.420! [2023-10-14 17:35:43,896][61552] Updated weights for policy 0, policy_version 650 (0.0007) [2023-10-14 17:35:44,180][61585] Updated weights for policy 1, policy_version 650 (0.0008) [2023-10-14 17:35:44,255][61552] Updated weights for policy 0, policy_version 660 (0.0008) [2023-10-14 17:35:44,537][61585] Updated weights for policy 1, policy_version 660 (0.0008) [2023-10-14 17:35:44,618][61552] Updated weights for policy 0, policy_version 670 (0.0009) [2023-10-14 17:35:44,896][61585] Updated weights for policy 1, policy_version 670 (0.0009) [2023-10-14 17:35:48,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 12210.0). Total num frames: 1376256. Throughput: 0: 1646.0, 1: 1654.6. Samples: 351532. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:35:48,344][60425] Avg episode reward: [(0, '3.290'), (1, '2.870')] [2023-10-14 17:35:48,345][61248] Saving new best policy, reward=2.870! [2023-10-14 17:35:48,728][61552] Updated weights for policy 0, policy_version 680 (0.0009) [2023-10-14 17:35:49,101][61552] Updated weights for policy 0, policy_version 690 (0.0008) [2023-10-14 17:35:49,225][61585] Updated weights for policy 1, policy_version 680 (0.0007) [2023-10-14 17:35:49,469][61552] Updated weights for policy 0, policy_version 700 (0.0007) [2023-10-14 17:35:49,596][61585] Updated weights for policy 1, policy_version 690 (0.0008) [2023-10-14 17:35:49,961][61585] Updated weights for policy 1, policy_version 700 (0.0009) [2023-10-14 17:35:53,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 12248.1). Total num frames: 1441792. Throughput: 0: 1644.2, 1: 1653.4. Samples: 371502. Policy #0 lag: (min: 15.0, avg: 15.1, max: 20.0) [2023-10-14 17:35:53,344][60425] Avg episode reward: [(0, '3.560'), (1, '3.230')] [2023-10-14 17:35:53,345][61248] Saving new best policy, reward=3.230! [2023-10-14 17:35:53,649][61552] Updated weights for policy 0, policy_version 710 (0.0009) [2023-10-14 17:35:53,981][61585] Updated weights for policy 1, policy_version 710 (0.0010) [2023-10-14 17:35:54,009][61552] Updated weights for policy 0, policy_version 720 (0.0008) [2023-10-14 17:35:54,344][61585] Updated weights for policy 1, policy_version 720 (0.0007) [2023-10-14 17:35:54,380][61552] Updated weights for policy 0, policy_version 730 (0.0008) [2023-10-14 17:35:54,601][61172] Saving new best policy, reward=3.560! [2023-10-14 17:35:54,714][61585] Updated weights for policy 1, policy_version 730 (0.0008) [2023-10-14 17:35:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12283.1). Total num frames: 1507328. Throughput: 0: 1653.1, 1: 1647.0. Samples: 392060. Policy #0 lag: (min: 4.0, avg: 8.0, max: 36.0) [2023-10-14 17:35:58,344][60425] Avg episode reward: [(0, '3.440'), (1, '3.310')] [2023-10-14 17:35:58,354][61248] Saving new best policy, reward=3.310! [2023-10-14 17:35:58,538][61552] Updated weights for policy 0, policy_version 740 (0.0007) [2023-10-14 17:35:58,804][61585] Updated weights for policy 1, policy_version 740 (0.0009) [2023-10-14 17:35:58,911][61552] Updated weights for policy 0, policy_version 750 (0.0010) [2023-10-14 17:35:59,169][61585] Updated weights for policy 1, policy_version 750 (0.0008) [2023-10-14 17:35:59,286][61552] Updated weights for policy 0, policy_version 760 (0.0007) [2023-10-14 17:35:59,529][61585] Updated weights for policy 1, policy_version 760 (0.0007) [2023-10-14 17:36:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12315.4). Total num frames: 1572864. Throughput: 0: 1654.7, 1: 1650.1. Samples: 401076. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:36:03,344][60425] Avg episode reward: [(0, '3.560'), (1, '3.640')] [2023-10-14 17:36:03,344][61248] Saving new best policy, reward=3.640! [2023-10-14 17:36:03,412][61552] Updated weights for policy 0, policy_version 770 (0.0009) [2023-10-14 17:36:03,644][61585] Updated weights for policy 1, policy_version 770 (0.0007) [2023-10-14 17:36:03,769][61552] Updated weights for policy 0, policy_version 780 (0.0008) [2023-10-14 17:36:04,005][61585] Updated weights for policy 1, policy_version 780 (0.0008) [2023-10-14 17:36:04,152][61552] Updated weights for policy 0, policy_version 790 (0.0008) [2023-10-14 17:36:04,364][61585] Updated weights for policy 1, policy_version 790 (0.0009) [2023-10-14 17:36:04,516][61552] Updated weights for policy 0, policy_version 800 (0.0008) [2023-10-14 17:36:04,735][61585] Updated weights for policy 1, policy_version 800 (0.0009) [2023-10-14 17:36:08,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12345.2). Total num frames: 1638400. Throughput: 0: 1654.5, 1: 1647.4. Samples: 421340. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-14 17:36:08,344][60425] Avg episode reward: [(0, '3.710'), (1, '3.180')] [2023-10-14 17:36:08,766][61552] Updated weights for policy 0, policy_version 810 (0.0010) [2023-10-14 17:36:08,908][61585] Updated weights for policy 1, policy_version 810 (0.0007) [2023-10-14 17:36:09,131][61552] Updated weights for policy 0, policy_version 820 (0.0009) [2023-10-14 17:36:09,273][61585] Updated weights for policy 1, policy_version 820 (0.0007) [2023-10-14 17:36:09,496][61552] Updated weights for policy 0, policy_version 830 (0.0009) [2023-10-14 17:36:09,562][61172] Saving new best policy, reward=3.710! [2023-10-14 17:36:09,644][61585] Updated weights for policy 1, policy_version 830 (0.0007) [2023-10-14 17:36:13,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 12372.8). Total num frames: 1703936. Throughput: 0: 1652.4, 1: 1649.9. Samples: 441608. Policy #0 lag: (min: 12.0, avg: 18.1, max: 44.0) [2023-10-14 17:36:13,344][60425] Avg episode reward: [(0, '3.800'), (1, '3.430')] [2023-10-14 17:36:13,639][61552] Updated weights for policy 0, policy_version 840 (0.0008) [2023-10-14 17:36:13,939][61585] Updated weights for policy 1, policy_version 840 (0.0008) [2023-10-14 17:36:14,013][61552] Updated weights for policy 0, policy_version 850 (0.0008) [2023-10-14 17:36:14,316][61585] Updated weights for policy 1, policy_version 850 (0.0009) [2023-10-14 17:36:14,382][61552] Updated weights for policy 0, policy_version 860 (0.0009) [2023-10-14 17:36:14,528][61172] Saving new best policy, reward=3.800! [2023-10-14 17:36:14,679][61585] Updated weights for policy 1, policy_version 860 (0.0007) [2023-10-14 17:36:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12398.6). Total num frames: 1769472. Throughput: 0: 1651.0, 1: 1649.9. Samples: 450642. Policy #0 lag: (min: 8.0, avg: 31.8, max: 40.0) [2023-10-14 17:36:18,344][60425] Avg episode reward: [(0, '4.000'), (1, '3.250')] [2023-10-14 17:36:18,634][61552] Updated weights for policy 0, policy_version 870 (0.0008) [2023-10-14 17:36:18,700][61585] Updated weights for policy 1, policy_version 870 (0.0008) [2023-10-14 17:36:19,011][61552] Updated weights for policy 0, policy_version 880 (0.0007) [2023-10-14 17:36:19,053][61585] Updated weights for policy 1, policy_version 880 (0.0008) [2023-10-14 17:36:19,369][61552] Updated weights for policy 0, policy_version 890 (0.0008) [2023-10-14 17:36:19,416][61585] Updated weights for policy 1, policy_version 890 (0.0007) [2023-10-14 17:36:19,586][61172] Saving new best policy, reward=4.000! [2023-10-14 17:36:23,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 12422.6). Total num frames: 1835008. Throughput: 0: 1650.1, 1: 1649.3. Samples: 470918. Policy #0 lag: (min: 16.0, avg: 43.3, max: 48.0) [2023-10-14 17:36:23,344][60425] Avg episode reward: [(0, '4.220'), (1, '3.340')] [2023-10-14 17:36:23,598][61552] Updated weights for policy 0, policy_version 900 (0.0008) [2023-10-14 17:36:23,695][61585] Updated weights for policy 1, policy_version 900 (0.0009) [2023-10-14 17:36:23,963][61552] Updated weights for policy 0, policy_version 910 (0.0008) [2023-10-14 17:36:24,052][61585] Updated weights for policy 1, policy_version 910 (0.0009) [2023-10-14 17:36:24,334][61552] Updated weights for policy 0, policy_version 920 (0.0009) [2023-10-14 17:36:24,422][61585] Updated weights for policy 1, policy_version 920 (0.0007) [2023-10-14 17:36:24,628][61172] Saving new best policy, reward=4.220! [2023-10-14 17:36:28,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 12445.0). Total num frames: 1900544. Throughput: 0: 1644.5, 1: 1653.5. Samples: 491076. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-14 17:36:28,344][60425] Avg episode reward: [(0, '4.650'), (1, '3.180')] [2023-10-14 17:36:28,350][61552] Updated weights for policy 0, policy_version 930 (0.0008) [2023-10-14 17:36:28,605][61585] Updated weights for policy 1, policy_version 930 (0.0009) [2023-10-14 17:36:28,722][61552] Updated weights for policy 0, policy_version 940 (0.0007) [2023-10-14 17:36:28,980][61585] Updated weights for policy 1, policy_version 940 (0.0009) [2023-10-14 17:36:29,096][61552] Updated weights for policy 0, policy_version 950 (0.0009) [2023-10-14 17:36:29,350][61585] Updated weights for policy 1, policy_version 950 (0.0008) [2023-10-14 17:36:29,460][61172] Saving new best policy, reward=4.650! [2023-10-14 17:36:29,461][61552] Updated weights for policy 0, policy_version 960 (0.0007) [2023-10-14 17:36:29,715][61585] Updated weights for policy 1, policy_version 960 (0.0010) [2023-10-14 17:36:33,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 12466.0). Total num frames: 1966080. Throughput: 0: 1646.4, 1: 1655.0. Samples: 500098. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:36:33,344][60425] Avg episode reward: [(0, '4.510'), (1, '3.280')] [2023-10-14 17:36:33,680][61552] Updated weights for policy 0, policy_version 970 (0.0007) [2023-10-14 17:36:33,825][61585] Updated weights for policy 1, policy_version 970 (0.0008) [2023-10-14 17:36:34,039][61552] Updated weights for policy 0, policy_version 980 (0.0008) [2023-10-14 17:36:34,195][61585] Updated weights for policy 1, policy_version 980 (0.0007) [2023-10-14 17:36:34,415][61552] Updated weights for policy 0, policy_version 990 (0.0007) [2023-10-14 17:36:34,566][61585] Updated weights for policy 1, policy_version 990 (0.0007) [2023-10-14 17:36:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 12485.7). Total num frames: 2031616. Throughput: 0: 1651.2, 1: 1661.6. Samples: 520582. Policy #0 lag: (min: 21.0, avg: 25.4, max: 53.0) [2023-10-14 17:36:38,344][60425] Avg episode reward: [(0, '3.760'), (1, '3.440')] [2023-10-14 17:36:38,635][61552] Updated weights for policy 0, policy_version 1000 (0.0007) [2023-10-14 17:36:38,722][61585] Updated weights for policy 1, policy_version 1000 (0.0009) [2023-10-14 17:36:39,022][61552] Updated weights for policy 0, policy_version 1010 (0.0009) [2023-10-14 17:36:39,098][61585] Updated weights for policy 1, policy_version 1010 (0.0007) [2023-10-14 17:36:39,389][61552] Updated weights for policy 0, policy_version 1020 (0.0010) [2023-10-14 17:36:39,460][61585] Updated weights for policy 1, policy_version 1020 (0.0009) [2023-10-14 17:36:43,344][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 12504.2). Total num frames: 2097152. Throughput: 0: 1649.4, 1: 1657.5. Samples: 540872. Policy #0 lag: (min: 13.0, avg: 20.5, max: 45.0) [2023-10-14 17:36:43,344][60425] Avg episode reward: [(0, '3.830'), (1, '4.020')] [2023-10-14 17:36:43,600][61552] Updated weights for policy 0, policy_version 1030 (0.0008) [2023-10-14 17:36:43,607][61585] Updated weights for policy 1, policy_version 1030 (0.0007) [2023-10-14 17:36:43,972][61585] Updated weights for policy 1, policy_version 1040 (0.0008) [2023-10-14 17:36:43,975][61552] Updated weights for policy 0, policy_version 1040 (0.0009) [2023-10-14 17:36:44,344][61585] Updated weights for policy 1, policy_version 1050 (0.0007) [2023-10-14 17:36:44,349][61552] Updated weights for policy 0, policy_version 1050 (0.0009) [2023-10-14 17:36:44,568][61248] Saving new best policy, reward=4.020! [2023-10-14 17:36:48,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12521.7). Total num frames: 2162688. Throughput: 0: 1647.0, 1: 1658.8. Samples: 549838. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-14 17:36:48,344][60425] Avg episode reward: [(0, '3.790'), (1, '4.080')] [2023-10-14 17:36:48,535][61552] Updated weights for policy 0, policy_version 1060 (0.0008) [2023-10-14 17:36:48,542][61585] Updated weights for policy 1, policy_version 1060 (0.0009) [2023-10-14 17:36:48,899][61552] Updated weights for policy 0, policy_version 1070 (0.0007) [2023-10-14 17:36:48,905][61585] Updated weights for policy 1, policy_version 1070 (0.0008) [2023-10-14 17:36:49,266][61585] Updated weights for policy 1, policy_version 1080 (0.0008) [2023-10-14 17:36:49,272][61552] Updated weights for policy 0, policy_version 1080 (0.0007) [2023-10-14 17:36:49,564][61248] Saving new best policy, reward=4.080! [2023-10-14 17:36:53,326][61552] Updated weights for policy 0, policy_version 1090 (0.0009) [2023-10-14 17:36:53,343][60425] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 12538.1). Total num frames: 2228224. Throughput: 0: 1645.0, 1: 1657.5. Samples: 569954. Policy #0 lag: (min: 31.0, avg: 43.6, max: 63.0) [2023-10-14 17:36:53,344][60425] Avg episode reward: [(0, '3.980'), (1, '4.030')] [2023-10-14 17:36:53,357][61585] Updated weights for policy 1, policy_version 1090 (0.0007) [2023-10-14 17:36:53,694][61552] Updated weights for policy 0, policy_version 1100 (0.0009) [2023-10-14 17:36:53,716][61585] Updated weights for policy 1, policy_version 1100 (0.0007) [2023-10-14 17:36:54,060][61552] Updated weights for policy 0, policy_version 1110 (0.0010) [2023-10-14 17:36:54,075][61585] Updated weights for policy 1, policy_version 1110 (0.0010) [2023-10-14 17:36:54,429][61552] Updated weights for policy 0, policy_version 1120 (0.0009) [2023-10-14 17:36:54,446][61585] Updated weights for policy 1, policy_version 1120 (0.0009) [2023-10-14 17:36:58,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 12553.7). Total num frames: 2293760. Throughput: 0: 1645.5, 1: 1656.9. Samples: 590218. Policy #0 lag: (min: 6.0, avg: 10.2, max: 38.0) [2023-10-14 17:36:58,345][60425] Avg episode reward: [(0, '4.020'), (1, '3.680')] [2023-10-14 17:36:58,661][61585] Updated weights for policy 1, policy_version 1130 (0.0008) [2023-10-14 17:36:58,715][61552] Updated weights for policy 0, policy_version 1130 (0.0010) [2023-10-14 17:36:59,036][61585] Updated weights for policy 1, policy_version 1140 (0.0007) [2023-10-14 17:36:59,077][61552] Updated weights for policy 0, policy_version 1140 (0.0009) [2023-10-14 17:36:59,402][61585] Updated weights for policy 1, policy_version 1150 (0.0007) [2023-10-14 17:36:59,450][61552] Updated weights for policy 0, policy_version 1150 (0.0007) [2023-10-14 17:37:03,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12568.5). Total num frames: 2359296. Throughput: 0: 1645.2, 1: 1653.2. Samples: 599066. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:37:03,344][60425] Avg episode reward: [(0, '4.280'), (1, '3.890')] [2023-10-14 17:37:03,463][61552] Updated weights for policy 0, policy_version 1160 (0.0009) [2023-10-14 17:37:03,501][61585] Updated weights for policy 1, policy_version 1160 (0.0007) [2023-10-14 17:37:03,831][61552] Updated weights for policy 0, policy_version 1170 (0.0007) [2023-10-14 17:37:03,864][61585] Updated weights for policy 1, policy_version 1170 (0.0008) [2023-10-14 17:37:04,197][61552] Updated weights for policy 0, policy_version 1180 (0.0009) [2023-10-14 17:37:04,231][61585] Updated weights for policy 1, policy_version 1180 (0.0008) [2023-10-14 17:37:08,260][61552] Updated weights for policy 0, policy_version 1190 (0.0009) [2023-10-14 17:37:08,290][61585] Updated weights for policy 1, policy_version 1190 (0.0008) [2023-10-14 17:37:08,343][60425] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 12582.4). Total num frames: 2424832. Throughput: 0: 1651.0, 1: 1651.1. Samples: 619514. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-14 17:37:08,344][60425] Avg episode reward: [(0, '4.150'), (1, '3.780')] [2023-10-14 17:37:08,639][61552] Updated weights for policy 0, policy_version 1200 (0.0007) [2023-10-14 17:37:08,658][61585] Updated weights for policy 1, policy_version 1200 (0.0007) [2023-10-14 17:37:09,012][61552] Updated weights for policy 0, policy_version 1210 (0.0009) [2023-10-14 17:37:09,014][61585] Updated weights for policy 1, policy_version 1210 (0.0008) [2023-10-14 17:37:13,208][61585] Updated weights for policy 1, policy_version 1220 (0.0008) [2023-10-14 17:37:13,303][61552] Updated weights for policy 0, policy_version 1220 (0.0010) [2023-10-14 17:37:13,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12595.7). Total num frames: 2490368. Throughput: 0: 1648.1, 1: 1646.9. Samples: 639352. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:37:13,344][60425] Avg episode reward: [(0, '4.090'), (1, '4.080')] [2023-10-14 17:37:13,594][61585] Updated weights for policy 1, policy_version 1230 (0.0008) [2023-10-14 17:37:13,671][61552] Updated weights for policy 0, policy_version 1230 (0.0008) [2023-10-14 17:37:13,962][61585] Updated weights for policy 1, policy_version 1240 (0.0008) [2023-10-14 17:37:14,039][61552] Updated weights for policy 0, policy_version 1240 (0.0009) [2023-10-14 17:37:18,059][61585] Updated weights for policy 1, policy_version 1250 (0.0007) [2023-10-14 17:37:18,189][61552] Updated weights for policy 0, policy_version 1250 (0.0011) [2023-10-14 17:37:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12608.3). Total num frames: 2555904. Throughput: 0: 1648.9, 1: 1645.7. Samples: 648352. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:37:18,344][60425] Avg episode reward: [(0, '4.190'), (1, '3.710')] [2023-10-14 17:37:18,427][61585] Updated weights for policy 1, policy_version 1260 (0.0008) [2023-10-14 17:37:18,559][61552] Updated weights for policy 0, policy_version 1260 (0.0008) [2023-10-14 17:37:18,790][61585] Updated weights for policy 1, policy_version 1270 (0.0009) [2023-10-14 17:37:18,926][61552] Updated weights for policy 0, policy_version 1270 (0.0007) [2023-10-14 17:37:19,167][61585] Updated weights for policy 1, policy_version 1280 (0.0009) [2023-10-14 17:37:19,302][61552] Updated weights for policy 0, policy_version 1280 (0.0007) [2023-10-14 17:37:23,105][61585] Updated weights for policy 1, policy_version 1290 (0.0008) [2023-10-14 17:37:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12620.3). Total num frames: 2621440. Throughput: 0: 1643.9, 1: 1650.3. Samples: 668820. Policy #0 lag: (min: 31.0, avg: 31.4, max: 42.0) [2023-10-14 17:37:23,344][60425] Avg episode reward: [(0, '4.570'), (1, '3.780')] [2023-10-14 17:37:23,345][61552] Updated weights for policy 0, policy_version 1290 (0.0008) [2023-10-14 17:37:23,462][61585] Updated weights for policy 1, policy_version 1300 (0.0008) [2023-10-14 17:37:23,708][61552] Updated weights for policy 0, policy_version 1300 (0.0008) [2023-10-14 17:37:23,828][61585] Updated weights for policy 1, policy_version 1310 (0.0009) [2023-10-14 17:37:24,080][61552] Updated weights for policy 0, policy_version 1310 (0.0007) [2023-10-14 17:37:28,007][61585] Updated weights for policy 1, policy_version 1320 (0.0008) [2023-10-14 17:37:28,293][61552] Updated weights for policy 0, policy_version 1320 (0.0008) [2023-10-14 17:37:28,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12631.8). Total num frames: 2686976. Throughput: 0: 1639.6, 1: 1656.6. Samples: 689200. Policy #0 lag: (min: 26.0, avg: 29.7, max: 58.0) [2023-10-14 17:37:28,344][60425] Avg episode reward: [(0, '5.060'), (1, '3.880')] [2023-10-14 17:37:28,371][61585] Updated weights for policy 1, policy_version 1330 (0.0008) [2023-10-14 17:37:28,663][61552] Updated weights for policy 0, policy_version 1330 (0.0008) [2023-10-14 17:37:28,724][61585] Updated weights for policy 1, policy_version 1340 (0.0010) [2023-10-14 17:37:29,032][61552] Updated weights for policy 0, policy_version 1340 (0.0007) [2023-10-14 17:37:29,177][61172] Saving new best policy, reward=5.060! [2023-10-14 17:37:32,879][61585] Updated weights for policy 1, policy_version 1350 (0.0008) [2023-10-14 17:37:33,008][61552] Updated weights for policy 0, policy_version 1350 (0.0007) [2023-10-14 17:37:33,250][61585] Updated weights for policy 1, policy_version 1360 (0.0009) [2023-10-14 17:37:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12642.7). Total num frames: 2752512. Throughput: 0: 1647.1, 1: 1657.3. Samples: 698536. Policy #0 lag: (min: 28.0, avg: 32.9, max: 60.0) [2023-10-14 17:37:33,344][60425] Avg episode reward: [(0, '5.240'), (1, '4.150')] [2023-10-14 17:37:33,378][61552] Updated weights for policy 0, policy_version 1360 (0.0010) [2023-10-14 17:37:33,609][61585] Updated weights for policy 1, policy_version 1370 (0.0009) [2023-10-14 17:37:33,747][61552] Updated weights for policy 0, policy_version 1370 (0.0009) [2023-10-14 17:37:33,829][61248] Saving new best policy, reward=4.150! [2023-10-14 17:37:33,968][61172] Saving new best policy, reward=5.240! [2023-10-14 17:37:37,798][61585] Updated weights for policy 1, policy_version 1380 (0.0007) [2023-10-14 17:37:37,832][61552] Updated weights for policy 0, policy_version 1380 (0.0007) [2023-10-14 17:37:38,171][61585] Updated weights for policy 1, policy_version 1390 (0.0007) [2023-10-14 17:37:38,208][61552] Updated weights for policy 0, policy_version 1390 (0.0008) [2023-10-14 17:37:38,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 12653.1). Total num frames: 2818048. Throughput: 0: 1652.2, 1: 1658.9. Samples: 718952. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-14 17:37:38,344][60425] Avg episode reward: [(0, '5.680'), (1, '4.300')] [2023-10-14 17:37:38,541][61585] Updated weights for policy 1, policy_version 1400 (0.0008) [2023-10-14 17:37:38,574][61552] Updated weights for policy 0, policy_version 1400 (0.0009) [2023-10-14 17:37:38,830][61248] Saving new best policy, reward=4.300! [2023-10-14 17:37:38,866][61172] Saving new best policy, reward=5.680! [2023-10-14 17:37:42,690][61585] Updated weights for policy 1, policy_version 1410 (0.0009) [2023-10-14 17:37:42,751][61552] Updated weights for policy 0, policy_version 1410 (0.0008) [2023-10-14 17:37:43,056][61585] Updated weights for policy 1, policy_version 1420 (0.0009) [2023-10-14 17:37:43,128][61552] Updated weights for policy 0, policy_version 1420 (0.0007) [2023-10-14 17:37:43,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 12663.1). Total num frames: 2883584. Throughput: 0: 1650.2, 1: 1654.5. Samples: 738930. Policy #0 lag: (min: 10.0, avg: 11.9, max: 39.0) [2023-10-14 17:37:43,344][60425] Avg episode reward: [(0, '5.580'), (1, '4.240')] [2023-10-14 17:37:43,429][61585] Updated weights for policy 1, policy_version 1430 (0.0008) [2023-10-14 17:37:43,499][61552] Updated weights for policy 0, policy_version 1430 (0.0007) [2023-10-14 17:37:43,794][61585] Updated weights for policy 1, policy_version 1440 (0.0008) [2023-10-14 17:37:43,795][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000001440_1474560.pth... [2023-10-14 17:37:43,859][61552] Updated weights for policy 0, policy_version 1440 (0.0008) [2023-10-14 17:37:43,859][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000001440_1474560.pth... [2023-10-14 17:37:48,067][61585] Updated weights for policy 1, policy_version 1450 (0.0010) [2023-10-14 17:37:48,140][61552] Updated weights for policy 0, policy_version 1450 (0.0010) [2023-10-14 17:37:48,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 12672.6). Total num frames: 2949120. Throughput: 0: 1651.4, 1: 1657.6. Samples: 747974. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-14 17:37:48,344][60425] Avg episode reward: [(0, '5.580'), (1, '4.380')] [2023-10-14 17:37:48,436][61585] Updated weights for policy 1, policy_version 1460 (0.0010) [2023-10-14 17:37:48,502][61552] Updated weights for policy 0, policy_version 1460 (0.0009) [2023-10-14 17:37:48,800][61585] Updated weights for policy 1, policy_version 1470 (0.0007) [2023-10-14 17:37:48,873][61248] Saving new best policy, reward=4.380! [2023-10-14 17:37:48,876][61552] Updated weights for policy 0, policy_version 1470 (0.0008) [2023-10-14 17:37:53,114][61552] Updated weights for policy 0, policy_version 1480 (0.0008) [2023-10-14 17:37:53,267][61585] Updated weights for policy 1, policy_version 1480 (0.0007) [2023-10-14 17:37:53,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 12681.8). Total num frames: 3014656. Throughput: 0: 1646.0, 1: 1654.8. Samples: 768048. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-14 17:37:53,344][60425] Avg episode reward: [(0, '4.980'), (1, '4.700')] [2023-10-14 17:37:53,486][61552] Updated weights for policy 0, policy_version 1490 (0.0009) [2023-10-14 17:37:53,629][61585] Updated weights for policy 1, policy_version 1490 (0.0009) [2023-10-14 17:37:53,852][61552] Updated weights for policy 0, policy_version 1500 (0.0008) [2023-10-14 17:37:53,999][61585] Updated weights for policy 1, policy_version 1500 (0.0009) [2023-10-14 17:37:54,137][61248] Saving new best policy, reward=4.700! [2023-10-14 17:37:58,010][61552] Updated weights for policy 0, policy_version 1510 (0.0008) [2023-10-14 17:37:58,214][61585] Updated weights for policy 1, policy_version 1510 (0.0007) [2023-10-14 17:37:58,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 12690.5). Total num frames: 3080192. Throughput: 0: 1649.5, 1: 1658.2. Samples: 788198. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:37:58,344][60425] Avg episode reward: [(0, '5.040'), (1, '4.480')] [2023-10-14 17:37:58,382][61552] Updated weights for policy 0, policy_version 1520 (0.0008) [2023-10-14 17:37:58,589][61585] Updated weights for policy 1, policy_version 1520 (0.0007) [2023-10-14 17:37:58,753][61552] Updated weights for policy 0, policy_version 1530 (0.0009) [2023-10-14 17:37:58,958][61585] Updated weights for policy 1, policy_version 1530 (0.0007) [2023-10-14 17:38:03,029][61552] Updated weights for policy 0, policy_version 1540 (0.0008) [2023-10-14 17:38:03,055][61585] Updated weights for policy 1, policy_version 1540 (0.0007) [2023-10-14 17:38:03,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 12698.9). Total num frames: 3145728. Throughput: 0: 1647.3, 1: 1659.0. Samples: 797136. Policy #0 lag: (min: 5.0, avg: 6.3, max: 30.0) [2023-10-14 17:38:03,344][60425] Avg episode reward: [(0, '5.390'), (1, '4.470')] [2023-10-14 17:38:03,394][61552] Updated weights for policy 0, policy_version 1550 (0.0009) [2023-10-14 17:38:03,422][61585] Updated weights for policy 1, policy_version 1550 (0.0008) [2023-10-14 17:38:03,757][61552] Updated weights for policy 0, policy_version 1560 (0.0008) [2023-10-14 17:38:03,782][61585] Updated weights for policy 1, policy_version 1560 (0.0007) [2023-10-14 17:38:07,819][61552] Updated weights for policy 0, policy_version 1570 (0.0008) [2023-10-14 17:38:07,916][61585] Updated weights for policy 1, policy_version 1570 (0.0008) [2023-10-14 17:38:08,187][61552] Updated weights for policy 0, policy_version 1580 (0.0008) [2023-10-14 17:38:08,275][61585] Updated weights for policy 1, policy_version 1580 (0.0009) [2023-10-14 17:38:08,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 12707.0). Total num frames: 3211264. Throughput: 0: 1657.6, 1: 1654.4. Samples: 817860. Policy #0 lag: (min: 17.0, avg: 20.0, max: 49.0) [2023-10-14 17:38:08,344][60425] Avg episode reward: [(0, '6.060'), (1, '4.120')] [2023-10-14 17:38:08,560][61552] Updated weights for policy 0, policy_version 1590 (0.0008) [2023-10-14 17:38:08,651][61585] Updated weights for policy 1, policy_version 1590 (0.0009) [2023-10-14 17:38:08,931][61172] Saving new best policy, reward=6.060! [2023-10-14 17:38:08,932][61552] Updated weights for policy 0, policy_version 1600 (0.0007) [2023-10-14 17:38:09,011][61585] Updated weights for policy 1, policy_version 1600 (0.0010) [2023-10-14 17:38:13,097][61552] Updated weights for policy 0, policy_version 1610 (0.0007) [2023-10-14 17:38:13,155][61585] Updated weights for policy 1, policy_version 1610 (0.0008) [2023-10-14 17:38:13,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 12714.8). Total num frames: 3276800. Throughput: 0: 1656.1, 1: 1645.5. Samples: 837776. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-14 17:38:13,344][60425] Avg episode reward: [(0, '6.360'), (1, '4.070')] [2023-10-14 17:38:13,465][61552] Updated weights for policy 0, policy_version 1620 (0.0007) [2023-10-14 17:38:13,524][61585] Updated weights for policy 1, policy_version 1620 (0.0008) [2023-10-14 17:38:13,836][61552] Updated weights for policy 0, policy_version 1630 (0.0008) [2023-10-14 17:38:13,887][61585] Updated weights for policy 1, policy_version 1630 (0.0007) [2023-10-14 17:38:13,900][61172] Saving new best policy, reward=6.360! [2023-10-14 17:38:17,769][61585] Updated weights for policy 1, policy_version 1640 (0.0009) [2023-10-14 17:38:17,836][61552] Updated weights for policy 0, policy_version 1640 (0.0008) [2023-10-14 17:38:18,138][61585] Updated weights for policy 1, policy_version 1650 (0.0008) [2023-10-14 17:38:18,209][61552] Updated weights for policy 0, policy_version 1650 (0.0009) [2023-10-14 17:38:18,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12722.3). Total num frames: 3342336. Throughput: 0: 1651.4, 1: 1642.7. Samples: 846768. Policy #0 lag: (min: 13.0, avg: 13.2, max: 22.0) [2023-10-14 17:38:18,344][60425] Avg episode reward: [(0, '6.250'), (1, '4.140')] [2023-10-14 17:38:18,503][61585] Updated weights for policy 1, policy_version 1660 (0.0007) [2023-10-14 17:38:18,583][61552] Updated weights for policy 0, policy_version 1660 (0.0008) [2023-10-14 17:38:22,732][61585] Updated weights for policy 1, policy_version 1670 (0.0008) [2023-10-14 17:38:22,860][61552] Updated weights for policy 0, policy_version 1670 (0.0008) [2023-10-14 17:38:23,090][61585] Updated weights for policy 1, policy_version 1680 (0.0007) [2023-10-14 17:38:23,231][61552] Updated weights for policy 0, policy_version 1680 (0.0008) [2023-10-14 17:38:23,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 12729.4). Total num frames: 3407872. Throughput: 0: 1650.2, 1: 1641.8. Samples: 867094. Policy #0 lag: (min: 4.0, avg: 5.4, max: 29.0) [2023-10-14 17:38:23,344][60425] Avg episode reward: [(0, '5.970'), (1, '4.320')] [2023-10-14 17:38:23,458][61585] Updated weights for policy 1, policy_version 1690 (0.0008) [2023-10-14 17:38:23,591][61552] Updated weights for policy 0, policy_version 1690 (0.0008) [2023-10-14 17:38:27,780][61552] Updated weights for policy 0, policy_version 1700 (0.0010) [2023-10-14 17:38:27,930][61585] Updated weights for policy 1, policy_version 1700 (0.0009) [2023-10-14 17:38:28,148][61552] Updated weights for policy 0, policy_version 1710 (0.0007) [2023-10-14 17:38:28,295][61585] Updated weights for policy 1, policy_version 1710 (0.0009) [2023-10-14 17:38:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12736.4). Total num frames: 3473408. Throughput: 0: 1649.1, 1: 1642.2. Samples: 887038. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-10-14 17:38:28,344][60425] Avg episode reward: [(0, '5.930'), (1, '4.300')] [2023-10-14 17:38:28,518][61552] Updated weights for policy 0, policy_version 1720 (0.0008) [2023-10-14 17:38:28,661][61585] Updated weights for policy 1, policy_version 1720 (0.0008) [2023-10-14 17:38:32,664][61552] Updated weights for policy 0, policy_version 1730 (0.0009) [2023-10-14 17:38:32,787][61585] Updated weights for policy 1, policy_version 1730 (0.0010) [2023-10-14 17:38:33,044][61552] Updated weights for policy 0, policy_version 1740 (0.0010) [2023-10-14 17:38:33,151][61585] Updated weights for policy 1, policy_version 1740 (0.0008) [2023-10-14 17:38:33,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 12743.1). Total num frames: 3538944. Throughput: 0: 1647.1, 1: 1643.8. Samples: 896064. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-14 17:38:33,344][60425] Avg episode reward: [(0, '6.410'), (1, '4.890')] [2023-10-14 17:38:33,414][61552] Updated weights for policy 0, policy_version 1750 (0.0008) [2023-10-14 17:38:33,519][61585] Updated weights for policy 1, policy_version 1750 (0.0008) [2023-10-14 17:38:33,790][61172] Saving new best policy, reward=6.410! [2023-10-14 17:38:33,793][61552] Updated weights for policy 0, policy_version 1760 (0.0008) [2023-10-14 17:38:33,882][61248] Saving new best policy, reward=4.890! [2023-10-14 17:38:33,883][61585] Updated weights for policy 1, policy_version 1760 (0.0008) [2023-10-14 17:38:37,964][61585] Updated weights for policy 1, policy_version 1770 (0.0008) [2023-10-14 17:38:38,106][61552] Updated weights for policy 0, policy_version 1770 (0.0007) [2023-10-14 17:38:38,329][61585] Updated weights for policy 1, policy_version 1780 (0.0007) [2023-10-14 17:38:38,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 12749.5). Total num frames: 3604480. Throughput: 0: 1651.6, 1: 1643.9. Samples: 916346. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-14 17:38:38,345][60425] Avg episode reward: [(0, '6.720'), (1, '5.550')] [2023-10-14 17:38:38,474][61552] Updated weights for policy 0, policy_version 1780 (0.0007) [2023-10-14 17:38:38,684][61585] Updated weights for policy 1, policy_version 1790 (0.0007) [2023-10-14 17:38:38,756][61248] Saving new best policy, reward=5.550! [2023-10-14 17:38:38,842][61552] Updated weights for policy 0, policy_version 1790 (0.0009) [2023-10-14 17:38:38,915][61172] Saving new best policy, reward=6.720! [2023-10-14 17:38:42,754][61585] Updated weights for policy 1, policy_version 1800 (0.0008) [2023-10-14 17:38:42,977][61552] Updated weights for policy 0, policy_version 1800 (0.0009) [2023-10-14 17:38:43,116][61585] Updated weights for policy 1, policy_version 1810 (0.0009) [2023-10-14 17:38:43,340][61552] Updated weights for policy 0, policy_version 1810 (0.0007) [2023-10-14 17:38:43,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 12755.7). Total num frames: 3670016. Throughput: 0: 1652.2, 1: 1638.3. Samples: 936270. Policy #0 lag: (min: 8.0, avg: 36.1, max: 40.0) [2023-10-14 17:38:43,344][60425] Avg episode reward: [(0, '7.340'), (1, '5.730')] [2023-10-14 17:38:43,484][61585] Updated weights for policy 1, policy_version 1820 (0.0009) [2023-10-14 17:38:43,628][61248] Saving new best policy, reward=5.730! [2023-10-14 17:38:43,710][61552] Updated weights for policy 0, policy_version 1820 (0.0007) [2023-10-14 17:38:43,860][61172] Saving new best policy, reward=7.340! [2023-10-14 17:38:47,798][61585] Updated weights for policy 1, policy_version 1830 (0.0009) [2023-10-14 17:38:47,802][61552] Updated weights for policy 0, policy_version 1830 (0.0010) [2023-10-14 17:38:48,175][61552] Updated weights for policy 0, policy_version 1840 (0.0010) [2023-10-14 17:38:48,179][61585] Updated weights for policy 1, policy_version 1840 (0.0008) [2023-10-14 17:38:48,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 12761.7). Total num frames: 3735552. Throughput: 0: 1653.6, 1: 1644.7. Samples: 945558. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:38:48,344][60425] Avg episode reward: [(0, '7.460'), (1, '5.840')] [2023-10-14 17:38:48,537][61552] Updated weights for policy 0, policy_version 1850 (0.0009) [2023-10-14 17:38:48,546][61585] Updated weights for policy 1, policy_version 1850 (0.0008) [2023-10-14 17:38:48,759][61172] Saving new best policy, reward=7.460! [2023-10-14 17:38:48,760][61248] Saving new best policy, reward=5.840! [2023-10-14 17:38:52,453][61552] Updated weights for policy 0, policy_version 1860 (0.0007) [2023-10-14 17:38:52,726][61585] Updated weights for policy 1, policy_version 1860 (0.0009) [2023-10-14 17:38:52,831][61552] Updated weights for policy 0, policy_version 1870 (0.0009) [2023-10-14 17:38:53,094][61585] Updated weights for policy 1, policy_version 1870 (0.0007) [2023-10-14 17:38:53,200][61552] Updated weights for policy 0, policy_version 1880 (0.0008) [2023-10-14 17:38:53,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 12885.0). Total num frames: 3801088. Throughput: 0: 1652.3, 1: 1639.8. Samples: 966006. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:38:53,344][60425] Avg episode reward: [(0, '8.720'), (1, '5.670')] [2023-10-14 17:38:53,455][61585] Updated weights for policy 1, policy_version 1880 (0.0007) [2023-10-14 17:38:53,497][61172] Saving new best policy, reward=8.720! [2023-10-14 17:38:57,324][61552] Updated weights for policy 0, policy_version 1890 (0.0009) [2023-10-14 17:38:57,638][61585] Updated weights for policy 1, policy_version 1890 (0.0008) [2023-10-14 17:38:57,695][61552] Updated weights for policy 0, policy_version 1900 (0.0009) [2023-10-14 17:38:58,010][61585] Updated weights for policy 1, policy_version 1900 (0.0008) [2023-10-14 17:38:58,057][61552] Updated weights for policy 0, policy_version 1910 (0.0007) [2023-10-14 17:38:58,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 3866624. Throughput: 0: 1646.5, 1: 1636.3. Samples: 985502. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-14 17:38:58,344][60425] Avg episode reward: [(0, '8.200'), (1, '6.100')] [2023-10-14 17:38:58,383][61585] Updated weights for policy 1, policy_version 1910 (0.0008) [2023-10-14 17:38:58,426][61552] Updated weights for policy 0, policy_version 1920 (0.0009) [2023-10-14 17:38:58,753][61585] Updated weights for policy 1, policy_version 1920 (0.0009) [2023-10-14 17:38:58,753][61248] Saving new best policy, reward=6.100! [2023-10-14 17:39:02,658][61552] Updated weights for policy 0, policy_version 1930 (0.0008) [2023-10-14 17:39:02,844][61585] Updated weights for policy 1, policy_version 1930 (0.0009) [2023-10-14 17:39:03,030][61552] Updated weights for policy 0, policy_version 1940 (0.0010) [2023-10-14 17:39:03,217][61585] Updated weights for policy 1, policy_version 1940 (0.0008) [2023-10-14 17:39:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 3932160. Throughput: 0: 1654.4, 1: 1640.7. Samples: 995050. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-14 17:39:03,344][60425] Avg episode reward: [(0, '8.060'), (1, '6.320')] [2023-10-14 17:39:03,411][61552] Updated weights for policy 0, policy_version 1950 (0.0007) [2023-10-14 17:39:03,579][61585] Updated weights for policy 1, policy_version 1950 (0.0008) [2023-10-14 17:39:03,651][61248] Saving new best policy, reward=6.320! [2023-10-14 17:39:07,580][61552] Updated weights for policy 0, policy_version 1960 (0.0007) [2023-10-14 17:39:07,801][61585] Updated weights for policy 1, policy_version 1960 (0.0008) [2023-10-14 17:39:07,954][61552] Updated weights for policy 0, policy_version 1970 (0.0009) [2023-10-14 17:39:08,159][61585] Updated weights for policy 1, policy_version 1970 (0.0007) [2023-10-14 17:39:08,324][61552] Updated weights for policy 0, policy_version 1980 (0.0007) [2023-10-14 17:39:08,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 3997696. Throughput: 0: 1650.8, 1: 1638.3. Samples: 1015102. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-14 17:39:08,344][60425] Avg episode reward: [(0, '8.170'), (1, '5.840')] [2023-10-14 17:39:08,526][61585] Updated weights for policy 1, policy_version 1980 (0.0009) [2023-10-14 17:39:12,516][61552] Updated weights for policy 0, policy_version 1990 (0.0010) [2023-10-14 17:39:12,809][61585] Updated weights for policy 1, policy_version 1990 (0.0008) [2023-10-14 17:39:12,890][61552] Updated weights for policy 0, policy_version 2000 (0.0009) [2023-10-14 17:39:13,187][61585] Updated weights for policy 1, policy_version 2000 (0.0007) [2023-10-14 17:39:13,254][61552] Updated weights for policy 0, policy_version 2010 (0.0009) [2023-10-14 17:39:13,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 4063232. Throughput: 0: 1643.9, 1: 1635.7. Samples: 1034622. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:39:13,344][60425] Avg episode reward: [(0, '8.290'), (1, '6.150')] [2023-10-14 17:39:13,552][61585] Updated weights for policy 1, policy_version 2010 (0.0007) [2023-10-14 17:39:17,515][61552] Updated weights for policy 0, policy_version 2020 (0.0009) [2023-10-14 17:39:17,731][61585] Updated weights for policy 1, policy_version 2020 (0.0007) [2023-10-14 17:39:17,894][61552] Updated weights for policy 0, policy_version 2030 (0.0009) [2023-10-14 17:39:18,105][61585] Updated weights for policy 1, policy_version 2030 (0.0008) [2023-10-14 17:39:18,257][61552] Updated weights for policy 0, policy_version 2040 (0.0008) [2023-10-14 17:39:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 4128768. Throughput: 0: 1651.5, 1: 1638.7. Samples: 1044122. Policy #0 lag: (min: 1.0, avg: 8.7, max: 33.0) [2023-10-14 17:39:18,344][60425] Avg episode reward: [(0, '7.340'), (1, '5.980')] [2023-10-14 17:39:18,479][61585] Updated weights for policy 1, policy_version 2040 (0.0008) [2023-10-14 17:39:22,458][61552] Updated weights for policy 0, policy_version 2050 (0.0009) [2023-10-14 17:39:22,714][61585] Updated weights for policy 1, policy_version 2050 (0.0009) [2023-10-14 17:39:22,832][61552] Updated weights for policy 0, policy_version 2060 (0.0010) [2023-10-14 17:39:23,080][61585] Updated weights for policy 1, policy_version 2060 (0.0008) [2023-10-14 17:39:23,192][61552] Updated weights for policy 0, policy_version 2070 (0.0008) [2023-10-14 17:39:23,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 4194304. Throughput: 0: 1645.1, 1: 1638.8. Samples: 1064120. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:39:23,344][60425] Avg episode reward: [(0, '7.290'), (1, '6.030')] [2023-10-14 17:39:23,449][61585] Updated weights for policy 1, policy_version 2070 (0.0010) [2023-10-14 17:39:23,569][61552] Updated weights for policy 0, policy_version 2080 (0.0008) [2023-10-14 17:39:23,817][61585] Updated weights for policy 1, policy_version 2080 (0.0009) [2023-10-14 17:39:27,792][61585] Updated weights for policy 1, policy_version 2090 (0.0009) [2023-10-14 17:39:27,823][61552] Updated weights for policy 0, policy_version 2090 (0.0008) [2023-10-14 17:39:28,156][61585] Updated weights for policy 1, policy_version 2100 (0.0009) [2023-10-14 17:39:28,191][61552] Updated weights for policy 0, policy_version 2100 (0.0009) [2023-10-14 17:39:28,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 4259840. Throughput: 0: 1640.1, 1: 1641.9. Samples: 1083960. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:39:28,344][60425] Avg episode reward: [(0, '7.090'), (1, '6.050')] [2023-10-14 17:39:28,524][61585] Updated weights for policy 1, policy_version 2110 (0.0007) [2023-10-14 17:39:28,566][61552] Updated weights for policy 0, policy_version 2110 (0.0007) [2023-10-14 17:39:32,511][61585] Updated weights for policy 1, policy_version 2120 (0.0007) [2023-10-14 17:39:32,829][61552] Updated weights for policy 0, policy_version 2120 (0.0009) [2023-10-14 17:39:32,890][61585] Updated weights for policy 1, policy_version 2130 (0.0009) [2023-10-14 17:39:33,195][61552] Updated weights for policy 0, policy_version 2130 (0.0007) [2023-10-14 17:39:33,251][61585] Updated weights for policy 1, policy_version 2140 (0.0008) [2023-10-14 17:39:33,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 4325376. Throughput: 0: 1642.6, 1: 1645.6. Samples: 1093526. Policy #0 lag: (min: 26.0, avg: 29.2, max: 58.0) [2023-10-14 17:39:33,344][60425] Avg episode reward: [(0, '8.180'), (1, '6.270')] [2023-10-14 17:39:33,560][61552] Updated weights for policy 0, policy_version 2140 (0.0009) [2023-10-14 17:39:37,359][61585] Updated weights for policy 1, policy_version 2150 (0.0009) [2023-10-14 17:39:37,728][61585] Updated weights for policy 1, policy_version 2160 (0.0009) [2023-10-14 17:39:37,886][61552] Updated weights for policy 0, policy_version 2150 (0.0009) [2023-10-14 17:39:38,093][61585] Updated weights for policy 1, policy_version 2170 (0.0008) [2023-10-14 17:39:38,257][61552] Updated weights for policy 0, policy_version 2160 (0.0008) [2023-10-14 17:39:38,343][60425] Fps is (10 sec: 16384.2, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 4423680. Throughput: 0: 1635.3, 1: 1648.1. Samples: 1113760. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) [2023-10-14 17:39:38,344][60425] Avg episode reward: [(0, '9.290'), (1, '6.510')] [2023-10-14 17:39:38,345][61248] Saving new best policy, reward=6.510! [2023-10-14 17:39:38,633][61552] Updated weights for policy 0, policy_version 2170 (0.0008) [2023-10-14 17:39:38,856][61172] Saving new best policy, reward=9.290! [2023-10-14 17:39:42,135][61585] Updated weights for policy 1, policy_version 2180 (0.0010) [2023-10-14 17:39:42,504][61585] Updated weights for policy 1, policy_version 2190 (0.0008) [2023-10-14 17:39:42,772][61552] Updated weights for policy 0, policy_version 2180 (0.0008) [2023-10-14 17:39:42,874][61585] Updated weights for policy 1, policy_version 2200 (0.0008) [2023-10-14 17:39:43,147][61552] Updated weights for policy 0, policy_version 2190 (0.0009) [2023-10-14 17:39:43,343][60425] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 4489216. Throughput: 0: 1639.6, 1: 1643.1. Samples: 1133224. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-14 17:39:43,344][60425] Avg episode reward: [(0, '10.000'), (1, '6.610')] [2023-10-14 17:39:43,352][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000002208_2260992.pth... [2023-10-14 17:39:43,392][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000000640_655360.pth [2023-10-14 17:39:43,396][61248] Saving new best policy, reward=6.610! [2023-10-14 17:39:43,514][61552] Updated weights for policy 0, policy_version 2200 (0.0008) [2023-10-14 17:39:43,812][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000002208_2260992.pth... [2023-10-14 17:39:43,851][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000000640_655360.pth [2023-10-14 17:39:43,856][61172] Saving new best policy, reward=10.000! [2023-10-14 17:39:47,162][61585] Updated weights for policy 1, policy_version 2210 (0.0007) [2023-10-14 17:39:47,528][61585] Updated weights for policy 1, policy_version 2220 (0.0008) [2023-10-14 17:39:47,606][61552] Updated weights for policy 0, policy_version 2210 (0.0007) [2023-10-14 17:39:47,885][61585] Updated weights for policy 1, policy_version 2230 (0.0007) [2023-10-14 17:39:47,983][61552] Updated weights for policy 0, policy_version 2220 (0.0009) [2023-10-14 17:39:48,250][61585] Updated weights for policy 1, policy_version 2240 (0.0007) [2023-10-14 17:39:48,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 4554752. Throughput: 0: 1631.6, 1: 1658.5. Samples: 1143104. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:39:48,344][60425] Avg episode reward: [(0, '9.390'), (1, '6.390')] [2023-10-14 17:39:48,347][61552] Updated weights for policy 0, policy_version 2230 (0.0009) [2023-10-14 17:39:48,722][61552] Updated weights for policy 0, policy_version 2240 (0.0007) [2023-10-14 17:39:52,472][61585] Updated weights for policy 1, policy_version 2250 (0.0008) [2023-10-14 17:39:52,795][61552] Updated weights for policy 0, policy_version 2250 (0.0008) [2023-10-14 17:39:52,834][61585] Updated weights for policy 1, policy_version 2260 (0.0008) [2023-10-14 17:39:53,165][61552] Updated weights for policy 0, policy_version 2260 (0.0007) [2023-10-14 17:39:53,205][61585] Updated weights for policy 1, policy_version 2270 (0.0007) [2023-10-14 17:39:53,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 4620288. Throughput: 0: 1635.1, 1: 1661.2. Samples: 1163434. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:39:53,344][60425] Avg episode reward: [(0, '9.970'), (1, '6.250')] [2023-10-14 17:39:53,534][61552] Updated weights for policy 0, policy_version 2270 (0.0009) [2023-10-14 17:39:57,361][61585] Updated weights for policy 1, policy_version 2280 (0.0008) [2023-10-14 17:39:57,568][61552] Updated weights for policy 0, policy_version 2280 (0.0010) [2023-10-14 17:39:57,717][61585] Updated weights for policy 1, policy_version 2290 (0.0007) [2023-10-14 17:39:57,939][61552] Updated weights for policy 0, policy_version 2290 (0.0010) [2023-10-14 17:39:58,087][61585] Updated weights for policy 1, policy_version 2300 (0.0007) [2023-10-14 17:39:58,305][61552] Updated weights for policy 0, policy_version 2300 (0.0010) [2023-10-14 17:39:58,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 4685824. Throughput: 0: 1643.7, 1: 1650.4. Samples: 1182858. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-10-14 17:39:58,344][60425] Avg episode reward: [(0, '9.150'), (1, '6.460')] [2023-10-14 17:40:02,284][61585] Updated weights for policy 1, policy_version 2310 (0.0008) [2023-10-14 17:40:02,370][61552] Updated weights for policy 0, policy_version 2310 (0.0007) [2023-10-14 17:40:02,647][61585] Updated weights for policy 1, policy_version 2320 (0.0007) [2023-10-14 17:40:02,739][61552] Updated weights for policy 0, policy_version 2320 (0.0008) [2023-10-14 17:40:03,015][61585] Updated weights for policy 1, policy_version 2330 (0.0008) [2023-10-14 17:40:03,111][61552] Updated weights for policy 0, policy_version 2330 (0.0009) [2023-10-14 17:40:03,343][60425] Fps is (10 sec: 16384.3, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 4784128. Throughput: 0: 1645.9, 1: 1663.8. Samples: 1193060. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:40:03,344][60425] Avg episode reward: [(0, '10.140'), (1, '6.490')] [2023-10-14 17:40:03,344][61172] Saving new best policy, reward=10.140! [2023-10-14 17:40:07,217][61552] Updated weights for policy 0, policy_version 2340 (0.0008) [2023-10-14 17:40:07,254][61585] Updated weights for policy 1, policy_version 2340 (0.0007) [2023-10-14 17:40:07,575][61552] Updated weights for policy 0, policy_version 2350 (0.0008) [2023-10-14 17:40:07,623][61585] Updated weights for policy 1, policy_version 2350 (0.0010) [2023-10-14 17:40:07,942][61552] Updated weights for policy 0, policy_version 2360 (0.0007) [2023-10-14 17:40:07,990][61585] Updated weights for policy 1, policy_version 2360 (0.0009) [2023-10-14 17:40:08,343][60425] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 4849664. Throughput: 0: 1652.0, 1: 1669.5. Samples: 1213584. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-14 17:40:08,344][60425] Avg episode reward: [(0, '10.220'), (1, '6.550')] [2023-10-14 17:40:08,344][61172] Saving new best policy, reward=10.220! [2023-10-14 17:40:12,053][61585] Updated weights for policy 1, policy_version 2370 (0.0008) [2023-10-14 17:40:12,181][61552] Updated weights for policy 0, policy_version 2370 (0.0007) [2023-10-14 17:40:12,422][61585] Updated weights for policy 1, policy_version 2380 (0.0010) [2023-10-14 17:40:12,582][61552] Updated weights for policy 0, policy_version 2380 (0.0009) [2023-10-14 17:40:12,789][61585] Updated weights for policy 1, policy_version 2390 (0.0008) [2023-10-14 17:40:12,954][61552] Updated weights for policy 0, policy_version 2390 (0.0009) [2023-10-14 17:40:13,160][61585] Updated weights for policy 1, policy_version 2400 (0.0007) [2023-10-14 17:40:13,318][61552] Updated weights for policy 0, policy_version 2400 (0.0010) [2023-10-14 17:40:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 13329.3). Total num frames: 4915200. Throughput: 0: 1646.1, 1: 1656.8. Samples: 1232590. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:40:13,344][60425] Avg episode reward: [(0, '10.450'), (1, '7.460')] [2023-10-14 17:40:13,355][61172] Saving new best policy, reward=10.450! [2023-10-14 17:40:13,355][61248] Saving new best policy, reward=7.460! [2023-10-14 17:40:17,282][61552] Updated weights for policy 0, policy_version 2410 (0.0008) [2023-10-14 17:40:17,341][61585] Updated weights for policy 1, policy_version 2410 (0.0008) [2023-10-14 17:40:17,655][61552] Updated weights for policy 0, policy_version 2420 (0.0010) [2023-10-14 17:40:17,701][61585] Updated weights for policy 1, policy_version 2420 (0.0010) [2023-10-14 17:40:18,025][61552] Updated weights for policy 0, policy_version 2430 (0.0007) [2023-10-14 17:40:18,072][61585] Updated weights for policy 1, policy_version 2430 (0.0009) [2023-10-14 17:40:18,343][60425] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 4980736. Throughput: 0: 1659.4, 1: 1666.0. Samples: 1243166. Policy #0 lag: (min: 4.0, avg: 12.0, max: 36.0) [2023-10-14 17:40:18,344][60425] Avg episode reward: [(0, '10.420'), (1, '7.950')] [2023-10-14 17:40:18,344][61248] Saving new best policy, reward=7.950! [2023-10-14 17:40:22,149][61585] Updated weights for policy 1, policy_version 2440 (0.0007) [2023-10-14 17:40:22,154][61552] Updated weights for policy 0, policy_version 2440 (0.0009) [2023-10-14 17:40:22,517][61585] Updated weights for policy 1, policy_version 2450 (0.0008) [2023-10-14 17:40:22,527][61552] Updated weights for policy 0, policy_version 2450 (0.0009) [2023-10-14 17:40:22,887][61585] Updated weights for policy 1, policy_version 2460 (0.0007) [2023-10-14 17:40:22,895][61552] Updated weights for policy 0, policy_version 2460 (0.0010) [2023-10-14 17:40:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 5046272. Throughput: 0: 1660.4, 1: 1657.9. Samples: 1263082. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) [2023-10-14 17:40:23,344][60425] Avg episode reward: [(0, '11.010'), (1, '7.790')] [2023-10-14 17:40:23,345][61172] Saving new best policy, reward=11.010! [2023-10-14 17:40:27,004][61585] Updated weights for policy 1, policy_version 2470 (0.0009) [2023-10-14 17:40:27,060][61552] Updated weights for policy 0, policy_version 2470 (0.0008) [2023-10-14 17:40:27,374][61585] Updated weights for policy 1, policy_version 2480 (0.0010) [2023-10-14 17:40:27,431][61552] Updated weights for policy 0, policy_version 2480 (0.0008) [2023-10-14 17:40:27,736][61585] Updated weights for policy 1, policy_version 2490 (0.0008) [2023-10-14 17:40:27,794][61552] Updated weights for policy 0, policy_version 2490 (0.0007) [2023-10-14 17:40:28,343][60425] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 5111808. Throughput: 0: 1643.9, 1: 1649.9. Samples: 1281442. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-14 17:40:28,344][60425] Avg episode reward: [(0, '11.920'), (1, '7.000')] [2023-10-14 17:40:28,355][61172] Saving new best policy, reward=11.920! [2023-10-14 17:40:31,942][61585] Updated weights for policy 1, policy_version 2500 (0.0007) [2023-10-14 17:40:31,964][61552] Updated weights for policy 0, policy_version 2500 (0.0007) [2023-10-14 17:40:32,298][61585] Updated weights for policy 1, policy_version 2510 (0.0007) [2023-10-14 17:40:32,342][61552] Updated weights for policy 0, policy_version 2510 (0.0008) [2023-10-14 17:40:32,673][61585] Updated weights for policy 1, policy_version 2520 (0.0007) [2023-10-14 17:40:32,701][61552] Updated weights for policy 0, policy_version 2520 (0.0008) [2023-10-14 17:40:33,343][60425] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 5177344. Throughput: 0: 1661.8, 1: 1651.7. Samples: 1292212. Policy #0 lag: (min: 18.0, avg: 22.2, max: 50.0) [2023-10-14 17:40:33,344][60425] Avg episode reward: [(0, '12.380'), (1, '7.240')] [2023-10-14 17:40:33,344][61172] Saving new best policy, reward=12.380! [2023-10-14 17:40:36,746][61585] Updated weights for policy 1, policy_version 2530 (0.0008) [2023-10-14 17:40:36,799][61552] Updated weights for policy 0, policy_version 2530 (0.0009) [2023-10-14 17:40:37,116][61585] Updated weights for policy 1, policy_version 2540 (0.0008) [2023-10-14 17:40:37,168][61552] Updated weights for policy 0, policy_version 2540 (0.0009) [2023-10-14 17:40:37,491][61585] Updated weights for policy 1, policy_version 2550 (0.0009) [2023-10-14 17:40:37,532][61552] Updated weights for policy 0, policy_version 2550 (0.0007) [2023-10-14 17:40:37,859][61585] Updated weights for policy 1, policy_version 2560 (0.0009) [2023-10-14 17:40:37,913][61552] Updated weights for policy 0, policy_version 2560 (0.0009) [2023-10-14 17:40:38,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 5242880. Throughput: 0: 1661.4, 1: 1652.6. Samples: 1312566. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-10-14 17:40:38,344][60425] Avg episode reward: [(0, '13.260'), (1, '7.230')] [2023-10-14 17:40:38,344][61172] Saving new best policy, reward=13.260! [2023-10-14 17:40:41,986][61585] Updated weights for policy 1, policy_version 2570 (0.0009) [2023-10-14 17:40:42,115][61552] Updated weights for policy 0, policy_version 2570 (0.0008) [2023-10-14 17:40:42,359][61585] Updated weights for policy 1, policy_version 2580 (0.0009) [2023-10-14 17:40:42,495][61552] Updated weights for policy 0, policy_version 2580 (0.0010) [2023-10-14 17:40:42,716][61585] Updated weights for policy 1, policy_version 2590 (0.0009) [2023-10-14 17:40:42,855][61552] Updated weights for policy 0, policy_version 2590 (0.0008) [2023-10-14 17:40:43,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 5308416. Throughput: 0: 1644.7, 1: 1646.7. Samples: 1330970. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-14 17:40:43,345][60425] Avg episode reward: [(0, '13.310'), (1, '7.800')] [2023-10-14 17:40:43,355][61172] Saving new best policy, reward=13.310! [2023-10-14 17:40:46,927][61552] Updated weights for policy 0, policy_version 2600 (0.0007) [2023-10-14 17:40:47,029][61585] Updated weights for policy 1, policy_version 2600 (0.0008) [2023-10-14 17:40:47,299][61552] Updated weights for policy 0, policy_version 2610 (0.0007) [2023-10-14 17:40:47,405][61585] Updated weights for policy 1, policy_version 2610 (0.0008) [2023-10-14 17:40:47,675][61552] Updated weights for policy 0, policy_version 2620 (0.0008) [2023-10-14 17:40:47,771][61585] Updated weights for policy 1, policy_version 2620 (0.0008) [2023-10-14 17:40:48,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 5373952. Throughput: 0: 1658.6, 1: 1651.2. Samples: 1341998. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-14 17:40:48,344][60425] Avg episode reward: [(0, '13.750'), (1, '8.180')] [2023-10-14 17:40:48,345][61172] Saving new best policy, reward=13.750! [2023-10-14 17:40:48,345][61248] Saving new best policy, reward=8.180! [2023-10-14 17:40:51,777][61552] Updated weights for policy 0, policy_version 2630 (0.0008) [2023-10-14 17:40:51,821][61585] Updated weights for policy 1, policy_version 2630 (0.0008) [2023-10-14 17:40:52,153][61552] Updated weights for policy 0, policy_version 2640 (0.0008) [2023-10-14 17:40:52,196][61585] Updated weights for policy 1, policy_version 2640 (0.0007) [2023-10-14 17:40:52,529][61552] Updated weights for policy 0, policy_version 2650 (0.0009) [2023-10-14 17:40:52,569][61585] Updated weights for policy 1, policy_version 2650 (0.0007) [2023-10-14 17:40:53,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 5439488. Throughput: 0: 1656.8, 1: 1646.4. Samples: 1362230. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:40:53,344][60425] Avg episode reward: [(0, '14.020'), (1, '8.010')] [2023-10-14 17:40:53,345][61172] Saving new best policy, reward=14.020! [2023-10-14 17:40:56,559][61585] Updated weights for policy 1, policy_version 2660 (0.0010) [2023-10-14 17:40:56,735][61552] Updated weights for policy 0, policy_version 2660 (0.0009) [2023-10-14 17:40:56,928][61585] Updated weights for policy 1, policy_version 2670 (0.0009) [2023-10-14 17:40:57,141][61552] Updated weights for policy 0, policy_version 2670 (0.0007) [2023-10-14 17:40:57,285][61585] Updated weights for policy 1, policy_version 2680 (0.0008) [2023-10-14 17:40:57,510][61552] Updated weights for policy 0, policy_version 2680 (0.0007) [2023-10-14 17:40:58,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 5505024. Throughput: 0: 1644.5, 1: 1645.2. Samples: 1380628. Policy #0 lag: (min: 31.0, avg: 39.6, max: 63.0) [2023-10-14 17:40:58,344][60425] Avg episode reward: [(0, '13.080'), (1, '8.260')] [2023-10-14 17:40:58,356][61248] Saving new best policy, reward=8.260! [2023-10-14 17:41:01,448][61585] Updated weights for policy 1, policy_version 2690 (0.0009) [2023-10-14 17:41:01,564][61552] Updated weights for policy 0, policy_version 2690 (0.0009) [2023-10-14 17:41:01,818][61585] Updated weights for policy 1, policy_version 2700 (0.0009) [2023-10-14 17:41:01,933][61552] Updated weights for policy 0, policy_version 2700 (0.0010) [2023-10-14 17:41:02,186][61585] Updated weights for policy 1, policy_version 2710 (0.0009) [2023-10-14 17:41:02,305][61552] Updated weights for policy 0, policy_version 2710 (0.0007) [2023-10-14 17:41:02,544][61585] Updated weights for policy 1, policy_version 2720 (0.0008) [2023-10-14 17:41:02,670][61552] Updated weights for policy 0, policy_version 2720 (0.0007) [2023-10-14 17:41:03,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 5570560. Throughput: 0: 1651.0, 1: 1651.3. Samples: 1391770. Policy #0 lag: (min: 14.0, avg: 18.2, max: 46.0) [2023-10-14 17:41:03,344][60425] Avg episode reward: [(0, '14.420'), (1, '7.860')] [2023-10-14 17:41:03,345][61172] Saving new best policy, reward=14.420! [2023-10-14 17:41:06,793][61552] Updated weights for policy 0, policy_version 2730 (0.0010) [2023-10-14 17:41:06,907][61585] Updated weights for policy 1, policy_version 2730 (0.0008) [2023-10-14 17:41:07,165][61552] Updated weights for policy 0, policy_version 2740 (0.0007) [2023-10-14 17:41:07,274][61585] Updated weights for policy 1, policy_version 2740 (0.0007) [2023-10-14 17:41:07,534][61552] Updated weights for policy 0, policy_version 2750 (0.0008) [2023-10-14 17:41:07,644][61585] Updated weights for policy 1, policy_version 2750 (0.0008) [2023-10-14 17:41:08,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 5636096. Throughput: 0: 1647.2, 1: 1649.7. Samples: 1411444. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-14 17:41:08,344][60425] Avg episode reward: [(0, '14.010'), (1, '8.140')] [2023-10-14 17:41:11,686][61552] Updated weights for policy 0, policy_version 2760 (0.0008) [2023-10-14 17:41:11,867][61585] Updated weights for policy 1, policy_version 2760 (0.0009) [2023-10-14 17:41:12,054][61552] Updated weights for policy 0, policy_version 2770 (0.0009) [2023-10-14 17:41:12,236][61585] Updated weights for policy 1, policy_version 2770 (0.0008) [2023-10-14 17:41:12,420][61552] Updated weights for policy 0, policy_version 2780 (0.0008) [2023-10-14 17:41:12,599][61585] Updated weights for policy 1, policy_version 2780 (0.0008) [2023-10-14 17:41:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 5701632. Throughput: 0: 1648.9, 1: 1645.3. Samples: 1429684. Policy #0 lag: (min: 25.0, avg: 29.5, max: 57.0) [2023-10-14 17:41:13,344][60425] Avg episode reward: [(0, '13.570'), (1, '7.990')] [2023-10-14 17:41:16,555][61552] Updated weights for policy 0, policy_version 2790 (0.0010) [2023-10-14 17:41:16,842][61585] Updated weights for policy 1, policy_version 2790 (0.0009) [2023-10-14 17:41:16,927][61552] Updated weights for policy 0, policy_version 2800 (0.0010) [2023-10-14 17:41:17,210][61585] Updated weights for policy 1, policy_version 2800 (0.0009) [2023-10-14 17:41:17,294][61552] Updated weights for policy 0, policy_version 2810 (0.0008) [2023-10-14 17:41:17,574][61585] Updated weights for policy 1, policy_version 2810 (0.0010) [2023-10-14 17:41:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 5767168. Throughput: 0: 1659.3, 1: 1647.7. Samples: 1441028. Policy #0 lag: (min: 19.0, avg: 22.7, max: 51.0) [2023-10-14 17:41:18,344][60425] Avg episode reward: [(0, '15.680'), (1, '8.600')] [2023-10-14 17:41:18,344][61172] Saving new best policy, reward=15.680! [2023-10-14 17:41:18,345][61248] Saving new best policy, reward=8.600! [2023-10-14 17:41:21,428][61552] Updated weights for policy 0, policy_version 2820 (0.0008) [2023-10-14 17:41:21,797][61552] Updated weights for policy 0, policy_version 2830 (0.0007) [2023-10-14 17:41:21,801][61585] Updated weights for policy 1, policy_version 2820 (0.0008) [2023-10-14 17:41:22,163][61585] Updated weights for policy 1, policy_version 2830 (0.0009) [2023-10-14 17:41:22,175][61552] Updated weights for policy 0, policy_version 2840 (0.0008) [2023-10-14 17:41:22,543][61585] Updated weights for policy 1, policy_version 2840 (0.0009) [2023-10-14 17:41:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 5832704. Throughput: 0: 1649.9, 1: 1642.0. Samples: 1460702. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:41:23,345][60425] Avg episode reward: [(0, '15.080'), (1, '9.470')] [2023-10-14 17:41:23,346][61248] Saving new best policy, reward=9.470! [2023-10-14 17:41:26,258][61552] Updated weights for policy 0, policy_version 2850 (0.0008) [2023-10-14 17:41:26,629][61552] Updated weights for policy 0, policy_version 2860 (0.0009) [2023-10-14 17:41:26,683][61585] Updated weights for policy 1, policy_version 2850 (0.0009) [2023-10-14 17:41:27,002][61552] Updated weights for policy 0, policy_version 2870 (0.0008) [2023-10-14 17:41:27,057][61585] Updated weights for policy 1, policy_version 2860 (0.0010) [2023-10-14 17:41:27,380][61552] Updated weights for policy 0, policy_version 2880 (0.0010) [2023-10-14 17:41:27,423][61585] Updated weights for policy 1, policy_version 2870 (0.0009) [2023-10-14 17:41:27,790][61585] Updated weights for policy 1, policy_version 2880 (0.0009) [2023-10-14 17:41:28,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 5898240. Throughput: 0: 1651.5, 1: 1642.4. Samples: 1479194. Policy #0 lag: (min: 16.0, avg: 42.8, max: 48.0) [2023-10-14 17:41:28,344][60425] Avg episode reward: [(0, '15.640'), (1, '11.280')] [2023-10-14 17:41:28,358][61248] Saving new best policy, reward=11.280! [2023-10-14 17:41:31,581][61552] Updated weights for policy 0, policy_version 2890 (0.0009) [2023-10-14 17:41:31,956][61552] Updated weights for policy 0, policy_version 2900 (0.0007) [2023-10-14 17:41:31,982][61585] Updated weights for policy 1, policy_version 2890 (0.0009) [2023-10-14 17:41:32,335][61552] Updated weights for policy 0, policy_version 2910 (0.0008) [2023-10-14 17:41:32,360][61585] Updated weights for policy 1, policy_version 2900 (0.0008) [2023-10-14 17:41:32,722][61585] Updated weights for policy 1, policy_version 2910 (0.0010) [2023-10-14 17:41:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 5963776. Throughput: 0: 1656.7, 1: 1641.6. Samples: 1490422. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-14 17:41:33,344][60425] Avg episode reward: [(0, '17.550'), (1, '10.920')] [2023-10-14 17:41:33,345][61172] Saving new best policy, reward=17.550! [2023-10-14 17:41:36,480][61552] Updated weights for policy 0, policy_version 2920 (0.0009) [2023-10-14 17:41:36,602][61585] Updated weights for policy 1, policy_version 2920 (0.0008) [2023-10-14 17:41:36,861][61552] Updated weights for policy 0, policy_version 2930 (0.0009) [2023-10-14 17:41:36,973][61585] Updated weights for policy 1, policy_version 2930 (0.0008) [2023-10-14 17:41:37,229][61552] Updated weights for policy 0, policy_version 2940 (0.0007) [2023-10-14 17:41:37,342][61585] Updated weights for policy 1, policy_version 2940 (0.0010) [2023-10-14 17:41:38,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 6029312. Throughput: 0: 1647.3, 1: 1638.6. Samples: 1510094. Policy #0 lag: (min: 3.0, avg: 11.0, max: 35.0) [2023-10-14 17:41:38,344][60425] Avg episode reward: [(0, '16.320'), (1, '11.150')] [2023-10-14 17:41:41,388][61552] Updated weights for policy 0, policy_version 2950 (0.0010) [2023-10-14 17:41:41,391][61585] Updated weights for policy 1, policy_version 2950 (0.0008) [2023-10-14 17:41:41,770][61552] Updated weights for policy 0, policy_version 2960 (0.0008) [2023-10-14 17:41:41,771][61585] Updated weights for policy 1, policy_version 2960 (0.0008) [2023-10-14 17:41:42,135][61585] Updated weights for policy 1, policy_version 2970 (0.0007) [2023-10-14 17:41:42,145][61552] Updated weights for policy 0, policy_version 2970 (0.0007) [2023-10-14 17:41:43,344][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 6094848. Throughput: 0: 1657.0, 1: 1643.0. Samples: 1529128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:41:43,345][60425] Avg episode reward: [(0, '16.870'), (1, '10.560')] [2023-10-14 17:41:43,354][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000002976_3047424.pth... [2023-10-14 17:41:43,354][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000002976_3047424.pth... [2023-10-14 17:41:43,392][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000001440_1474560.pth [2023-10-14 17:41:43,396][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000001440_1474560.pth [2023-10-14 17:41:46,229][61585] Updated weights for policy 1, policy_version 2980 (0.0008) [2023-10-14 17:41:46,354][61552] Updated weights for policy 0, policy_version 2980 (0.0008) [2023-10-14 17:41:46,596][61585] Updated weights for policy 1, policy_version 2990 (0.0007) [2023-10-14 17:41:46,740][61552] Updated weights for policy 0, policy_version 2990 (0.0010) [2023-10-14 17:41:46,963][61585] Updated weights for policy 1, policy_version 3000 (0.0008) [2023-10-14 17:41:47,116][61552] Updated weights for policy 0, policy_version 3000 (0.0007) [2023-10-14 17:41:48,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 6160384. Throughput: 0: 1661.9, 1: 1644.7. Samples: 1540568. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:41:48,344][60425] Avg episode reward: [(0, '17.810'), (1, '10.690')] [2023-10-14 17:41:48,345][61172] Saving new best policy, reward=17.810! [2023-10-14 17:41:51,189][61552] Updated weights for policy 0, policy_version 3010 (0.0007) [2023-10-14 17:41:51,257][61585] Updated weights for policy 1, policy_version 3010 (0.0007) [2023-10-14 17:41:51,556][61552] Updated weights for policy 0, policy_version 3020 (0.0009) [2023-10-14 17:41:51,659][61585] Updated weights for policy 1, policy_version 3020 (0.0009) [2023-10-14 17:41:51,928][61552] Updated weights for policy 0, policy_version 3030 (0.0009) [2023-10-14 17:41:52,029][61585] Updated weights for policy 1, policy_version 3030 (0.0007) [2023-10-14 17:41:52,302][61552] Updated weights for policy 0, policy_version 3040 (0.0008) [2023-10-14 17:41:52,397][61585] Updated weights for policy 1, policy_version 3040 (0.0007) [2023-10-14 17:41:53,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 6225920. Throughput: 0: 1650.0, 1: 1640.0. Samples: 1559490. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:41:53,344][60425] Avg episode reward: [(0, '17.250'), (1, '11.170')] [2023-10-14 17:41:56,476][61552] Updated weights for policy 0, policy_version 3050 (0.0010) [2023-10-14 17:41:56,552][61585] Updated weights for policy 1, policy_version 3050 (0.0007) [2023-10-14 17:41:56,845][61552] Updated weights for policy 0, policy_version 3060 (0.0007) [2023-10-14 17:41:56,908][61585] Updated weights for policy 1, policy_version 3060 (0.0007) [2023-10-14 17:41:57,211][61552] Updated weights for policy 0, policy_version 3070 (0.0008) [2023-10-14 17:41:57,276][61585] Updated weights for policy 1, policy_version 3070 (0.0009) [2023-10-14 17:41:58,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 6291456. Throughput: 0: 1656.9, 1: 1649.2. Samples: 1578456. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) [2023-10-14 17:41:58,344][60425] Avg episode reward: [(0, '17.580'), (1, '11.990')] [2023-10-14 17:41:58,352][61248] Saving new best policy, reward=11.990! [2023-10-14 17:42:01,279][61552] Updated weights for policy 0, policy_version 3080 (0.0008) [2023-10-14 17:42:01,650][61552] Updated weights for policy 0, policy_version 3090 (0.0007) [2023-10-14 17:42:01,698][61585] Updated weights for policy 1, policy_version 3080 (0.0007) [2023-10-14 17:42:02,016][61552] Updated weights for policy 0, policy_version 3100 (0.0007) [2023-10-14 17:42:02,051][61585] Updated weights for policy 1, policy_version 3090 (0.0007) [2023-10-14 17:42:02,417][61585] Updated weights for policy 1, policy_version 3100 (0.0008) [2023-10-14 17:42:03,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 6356992. Throughput: 0: 1658.4, 1: 1647.4. Samples: 1589788. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) [2023-10-14 17:42:03,344][60425] Avg episode reward: [(0, '17.840'), (1, '12.330')] [2023-10-14 17:42:03,345][61172] Saving new best policy, reward=17.840! [2023-10-14 17:42:03,345][61248] Saving new best policy, reward=12.330! [2023-10-14 17:42:06,132][61552] Updated weights for policy 0, policy_version 3110 (0.0008) [2023-10-14 17:42:06,489][61585] Updated weights for policy 1, policy_version 3110 (0.0008) [2023-10-14 17:42:06,501][61552] Updated weights for policy 0, policy_version 3120 (0.0007) [2023-10-14 17:42:06,850][61585] Updated weights for policy 1, policy_version 3120 (0.0008) [2023-10-14 17:42:06,861][61552] Updated weights for policy 0, policy_version 3130 (0.0007) [2023-10-14 17:42:07,218][61585] Updated weights for policy 1, policy_version 3130 (0.0008) [2023-10-14 17:42:08,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 6422528. Throughput: 0: 1651.4, 1: 1646.6. Samples: 1609110. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:42:08,344][60425] Avg episode reward: [(0, '18.740'), (1, '13.220')] [2023-10-14 17:42:08,346][61172] Saving new best policy, reward=18.740! [2023-10-14 17:42:08,346][61248] Saving new best policy, reward=13.220! [2023-10-14 17:42:10,928][61552] Updated weights for policy 0, policy_version 3140 (0.0007) [2023-10-14 17:42:11,290][61552] Updated weights for policy 0, policy_version 3150 (0.0009) [2023-10-14 17:42:11,292][61585] Updated weights for policy 1, policy_version 3140 (0.0008) [2023-10-14 17:42:11,656][61552] Updated weights for policy 0, policy_version 3160 (0.0008) [2023-10-14 17:42:11,662][61585] Updated weights for policy 1, policy_version 3150 (0.0009) [2023-10-14 17:42:12,030][61585] Updated weights for policy 1, policy_version 3160 (0.0008) [2023-10-14 17:42:13,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 6488064. Throughput: 0: 1667.6, 1: 1648.8. Samples: 1628430. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:42:13,344][60425] Avg episode reward: [(0, '18.830'), (1, '13.060')] [2023-10-14 17:42:13,353][61172] Saving new best policy, reward=18.830! [2023-10-14 17:42:15,771][61552] Updated weights for policy 0, policy_version 3170 (0.0008) [2023-10-14 17:42:16,150][61552] Updated weights for policy 0, policy_version 3180 (0.0008) [2023-10-14 17:42:16,183][61585] Updated weights for policy 1, policy_version 3170 (0.0010) [2023-10-14 17:42:16,523][61552] Updated weights for policy 0, policy_version 3190 (0.0008) [2023-10-14 17:42:16,550][61585] Updated weights for policy 1, policy_version 3180 (0.0009) [2023-10-14 17:42:16,906][61552] Updated weights for policy 0, policy_version 3200 (0.0009) [2023-10-14 17:42:16,912][61585] Updated weights for policy 1, policy_version 3190 (0.0009) [2023-10-14 17:42:17,277][61585] Updated weights for policy 1, policy_version 3200 (0.0007) [2023-10-14 17:42:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 6553600. Throughput: 0: 1664.9, 1: 1652.4. Samples: 1639700. Policy #0 lag: (min: 29.0, avg: 31.5, max: 61.0) [2023-10-14 17:42:18,344][60425] Avg episode reward: [(0, '18.840'), (1, '13.790')] [2023-10-14 17:42:18,345][61172] Saving new best policy, reward=18.840! [2023-10-14 17:42:18,345][61248] Saving new best policy, reward=13.790! [2023-10-14 17:42:21,027][61552] Updated weights for policy 0, policy_version 3210 (0.0007) [2023-10-14 17:42:21,385][61552] Updated weights for policy 0, policy_version 3220 (0.0010) [2023-10-14 17:42:21,425][61585] Updated weights for policy 1, policy_version 3210 (0.0010) [2023-10-14 17:42:21,758][61552] Updated weights for policy 0, policy_version 3230 (0.0009) [2023-10-14 17:42:21,790][61585] Updated weights for policy 1, policy_version 3220 (0.0010) [2023-10-14 17:42:22,157][61585] Updated weights for policy 1, policy_version 3230 (0.0009) [2023-10-14 17:42:23,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 6619136. Throughput: 0: 1652.0, 1: 1644.1. Samples: 1658420. Policy #0 lag: (min: 29.0, avg: 31.5, max: 61.0) [2023-10-14 17:42:23,345][60425] Avg episode reward: [(0, '17.650'), (1, '12.270')] [2023-10-14 17:42:25,716][61552] Updated weights for policy 0, policy_version 3240 (0.0010) [2023-10-14 17:42:26,091][61552] Updated weights for policy 0, policy_version 3250 (0.0008) [2023-10-14 17:42:26,169][61585] Updated weights for policy 1, policy_version 3240 (0.0010) [2023-10-14 17:42:26,467][61552] Updated weights for policy 0, policy_version 3260 (0.0008) [2023-10-14 17:42:26,539][61585] Updated weights for policy 1, policy_version 3250 (0.0009) [2023-10-14 17:42:26,894][61585] Updated weights for policy 1, policy_version 3260 (0.0007) [2023-10-14 17:42:28,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 6684672. Throughput: 0: 1666.8, 1: 1649.6. Samples: 1678362. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-14 17:42:28,344][60425] Avg episode reward: [(0, '17.920'), (1, '10.980')] [2023-10-14 17:42:30,519][61552] Updated weights for policy 0, policy_version 3270 (0.0008) [2023-10-14 17:42:30,882][61552] Updated weights for policy 0, policy_version 3280 (0.0009) [2023-10-14 17:42:30,985][61585] Updated weights for policy 1, policy_version 3270 (0.0008) [2023-10-14 17:42:31,257][61552] Updated weights for policy 0, policy_version 3290 (0.0009) [2023-10-14 17:42:31,354][61585] Updated weights for policy 1, policy_version 3280 (0.0009) [2023-10-14 17:42:31,725][61585] Updated weights for policy 1, policy_version 3290 (0.0008) [2023-10-14 17:42:33,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 6750208. Throughput: 0: 1664.2, 1: 1649.1. Samples: 1689664. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-14 17:42:33,344][60425] Avg episode reward: [(0, '17.780'), (1, '11.260')] [2023-10-14 17:42:35,382][61552] Updated weights for policy 0, policy_version 3300 (0.0007) [2023-10-14 17:42:35,774][61552] Updated weights for policy 0, policy_version 3310 (0.0007) [2023-10-14 17:42:35,893][61585] Updated weights for policy 1, policy_version 3300 (0.0009) [2023-10-14 17:42:36,154][61552] Updated weights for policy 0, policy_version 3320 (0.0009) [2023-10-14 17:42:36,289][61585] Updated weights for policy 1, policy_version 3310 (0.0008) [2023-10-14 17:42:36,647][61585] Updated weights for policy 1, policy_version 3320 (0.0008) [2023-10-14 17:42:38,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 6815744. Throughput: 0: 1657.6, 1: 1641.4. Samples: 1707944. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-14 17:42:38,344][60425] Avg episode reward: [(0, '17.520'), (1, '11.840')] [2023-10-14 17:42:40,179][61552] Updated weights for policy 0, policy_version 3330 (0.0009) [2023-10-14 17:42:40,547][61552] Updated weights for policy 0, policy_version 3340 (0.0008) [2023-10-14 17:42:40,844][61585] Updated weights for policy 1, policy_version 3330 (0.0008) [2023-10-14 17:42:40,919][61552] Updated weights for policy 0, policy_version 3350 (0.0009) [2023-10-14 17:42:41,216][61585] Updated weights for policy 1, policy_version 3340 (0.0009) [2023-10-14 17:42:41,280][61552] Updated weights for policy 0, policy_version 3360 (0.0007) [2023-10-14 17:42:41,575][61585] Updated weights for policy 1, policy_version 3350 (0.0008) [2023-10-14 17:42:41,943][61585] Updated weights for policy 1, policy_version 3360 (0.0007) [2023-10-14 17:42:43,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 6881280. Throughput: 0: 1675.5, 1: 1651.6. Samples: 1728174. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-10-14 17:42:43,344][60425] Avg episode reward: [(0, '18.820'), (1, '12.750')] [2023-10-14 17:42:45,388][61552] Updated weights for policy 0, policy_version 3370 (0.0010) [2023-10-14 17:42:45,762][61552] Updated weights for policy 0, policy_version 3380 (0.0009) [2023-10-14 17:42:46,131][61552] Updated weights for policy 0, policy_version 3390 (0.0008) [2023-10-14 17:42:46,136][61585] Updated weights for policy 1, policy_version 3370 (0.0007) [2023-10-14 17:42:46,495][61585] Updated weights for policy 1, policy_version 3380 (0.0010) [2023-10-14 17:42:46,858][61585] Updated weights for policy 1, policy_version 3390 (0.0009) [2023-10-14 17:42:48,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 6946816. Throughput: 0: 1658.5, 1: 1659.4. Samples: 1739094. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-10-14 17:42:48,344][60425] Avg episode reward: [(0, '18.610'), (1, '14.020')] [2023-10-14 17:42:48,345][61248] Saving new best policy, reward=14.020! [2023-10-14 17:42:50,315][61552] Updated weights for policy 0, policy_version 3400 (0.0009) [2023-10-14 17:42:50,691][61552] Updated weights for policy 0, policy_version 3410 (0.0009) [2023-10-14 17:42:51,027][61585] Updated weights for policy 1, policy_version 3400 (0.0009) [2023-10-14 17:42:51,055][61552] Updated weights for policy 0, policy_version 3420 (0.0008) [2023-10-14 17:42:51,395][61585] Updated weights for policy 1, policy_version 3410 (0.0008) [2023-10-14 17:42:51,759][61585] Updated weights for policy 1, policy_version 3420 (0.0008) [2023-10-14 17:42:53,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 7012352. Throughput: 0: 1662.2, 1: 1644.6. Samples: 1757916. Policy #0 lag: (min: 31.0, avg: 31.6, max: 49.0) [2023-10-14 17:42:53,344][60425] Avg episode reward: [(0, '19.810'), (1, '14.740')] [2023-10-14 17:42:53,344][61248] Saving new best policy, reward=14.740! [2023-10-14 17:42:53,344][61172] Saving new best policy, reward=19.810! [2023-10-14 17:42:55,224][61552] Updated weights for policy 0, policy_version 3430 (0.0009) [2023-10-14 17:42:55,604][61552] Updated weights for policy 0, policy_version 3440 (0.0007) [2023-10-14 17:42:55,815][61585] Updated weights for policy 1, policy_version 3430 (0.0010) [2023-10-14 17:42:55,965][61552] Updated weights for policy 0, policy_version 3450 (0.0007) [2023-10-14 17:42:56,186][61585] Updated weights for policy 1, policy_version 3440 (0.0008) [2023-10-14 17:42:56,544][61585] Updated weights for policy 1, policy_version 3450 (0.0009) [2023-10-14 17:42:58,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 7077888. Throughput: 0: 1665.8, 1: 1661.9. Samples: 1778176. Policy #0 lag: (min: 31.0, avg: 31.6, max: 49.0) [2023-10-14 17:42:58,344][60425] Avg episode reward: [(0, '19.690'), (1, '15.170')] [2023-10-14 17:42:58,355][61248] Saving new best policy, reward=15.170! [2023-10-14 17:43:00,031][61552] Updated weights for policy 0, policy_version 3460 (0.0009) [2023-10-14 17:43:00,409][61552] Updated weights for policy 0, policy_version 3470 (0.0009) [2023-10-14 17:43:00,712][61585] Updated weights for policy 1, policy_version 3460 (0.0009) [2023-10-14 17:43:00,788][61552] Updated weights for policy 0, policy_version 3480 (0.0007) [2023-10-14 17:43:01,074][61585] Updated weights for policy 1, policy_version 3470 (0.0009) [2023-10-14 17:43:01,434][61585] Updated weights for policy 1, policy_version 3480 (0.0011) [2023-10-14 17:43:03,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 7143424. Throughput: 0: 1649.1, 1: 1659.7. Samples: 1788598. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-14 17:43:03,344][60425] Avg episode reward: [(0, '19.450'), (1, '16.090')] [2023-10-14 17:43:03,346][61248] Saving new best policy, reward=16.090! [2023-10-14 17:43:04,873][61552] Updated weights for policy 0, policy_version 3490 (0.0008) [2023-10-14 17:43:05,252][61552] Updated weights for policy 0, policy_version 3500 (0.0009) [2023-10-14 17:43:05,622][61552] Updated weights for policy 0, policy_version 3510 (0.0008) [2023-10-14 17:43:05,662][61585] Updated weights for policy 1, policy_version 3490 (0.0010) [2023-10-14 17:43:05,988][61552] Updated weights for policy 0, policy_version 3520 (0.0009) [2023-10-14 17:43:06,031][61585] Updated weights for policy 1, policy_version 3500 (0.0010) [2023-10-14 17:43:06,402][61585] Updated weights for policy 1, policy_version 3510 (0.0008) [2023-10-14 17:43:06,767][61585] Updated weights for policy 1, policy_version 3520 (0.0009) [2023-10-14 17:43:08,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 7208960. Throughput: 0: 1664.8, 1: 1650.3. Samples: 1807598. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:43:08,344][60425] Avg episode reward: [(0, '19.620'), (1, '16.320')] [2023-10-14 17:43:08,345][61248] Saving new best policy, reward=16.320! [2023-10-14 17:43:10,155][61552] Updated weights for policy 0, policy_version 3530 (0.0008) [2023-10-14 17:43:10,520][61552] Updated weights for policy 0, policy_version 3540 (0.0009) [2023-10-14 17:43:10,889][61552] Updated weights for policy 0, policy_version 3550 (0.0008) [2023-10-14 17:43:10,897][61585] Updated weights for policy 1, policy_version 3530 (0.0007) [2023-10-14 17:43:11,273][61585] Updated weights for policy 1, policy_version 3540 (0.0009) [2023-10-14 17:43:11,628][61585] Updated weights for policy 1, policy_version 3550 (0.0008) [2023-10-14 17:43:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 7274496. Throughput: 0: 1665.9, 1: 1662.3. Samples: 1828130. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:43:13,344][60425] Avg episode reward: [(0, '18.330'), (1, '15.240')] [2023-10-14 17:43:15,207][61552] Updated weights for policy 0, policy_version 3560 (0.0009) [2023-10-14 17:43:15,582][61552] Updated weights for policy 0, policy_version 3570 (0.0007) [2023-10-14 17:43:15,826][61585] Updated weights for policy 1, policy_version 3560 (0.0008) [2023-10-14 17:43:15,948][61552] Updated weights for policy 0, policy_version 3580 (0.0009) [2023-10-14 17:43:16,190][61585] Updated weights for policy 1, policy_version 3570 (0.0008) [2023-10-14 17:43:16,554][61585] Updated weights for policy 1, policy_version 3580 (0.0008) [2023-10-14 17:43:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 7340032. Throughput: 0: 1649.2, 1: 1659.8. Samples: 1838572. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:43:18,344][60425] Avg episode reward: [(0, '20.300'), (1, '16.280')] [2023-10-14 17:43:18,345][61172] Saving new best policy, reward=20.300! [2023-10-14 17:43:19,953][61552] Updated weights for policy 0, policy_version 3590 (0.0010) [2023-10-14 17:43:20,332][61552] Updated weights for policy 0, policy_version 3600 (0.0008) [2023-10-14 17:43:20,702][61552] Updated weights for policy 0, policy_version 3610 (0.0007) [2023-10-14 17:43:20,737][61585] Updated weights for policy 1, policy_version 3590 (0.0008) [2023-10-14 17:43:21,100][61585] Updated weights for policy 1, policy_version 3600 (0.0010) [2023-10-14 17:43:21,463][61585] Updated weights for policy 1, policy_version 3610 (0.0009) [2023-10-14 17:43:23,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 7405568. Throughput: 0: 1665.9, 1: 1661.5. Samples: 1857674. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:43:23,344][60425] Avg episode reward: [(0, '18.790'), (1, '15.920')] [2023-10-14 17:43:24,821][61552] Updated weights for policy 0, policy_version 3620 (0.0007) [2023-10-14 17:43:25,214][61552] Updated weights for policy 0, policy_version 3630 (0.0008) [2023-10-14 17:43:25,587][61585] Updated weights for policy 1, policy_version 3620 (0.0009) [2023-10-14 17:43:25,588][61552] Updated weights for policy 0, policy_version 3640 (0.0009) [2023-10-14 17:43:25,984][61585] Updated weights for policy 1, policy_version 3630 (0.0008) [2023-10-14 17:43:26,348][61585] Updated weights for policy 1, policy_version 3640 (0.0009) [2023-10-14 17:43:28,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 7471104. Throughput: 0: 1663.7, 1: 1665.2. Samples: 1877978. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:43:28,344][60425] Avg episode reward: [(0, '20.330'), (1, '17.630')] [2023-10-14 17:43:28,355][61172] Saving new best policy, reward=20.330! [2023-10-14 17:43:28,355][61248] Saving new best policy, reward=17.630! [2023-10-14 17:43:29,649][61552] Updated weights for policy 0, policy_version 3650 (0.0007) [2023-10-14 17:43:30,018][61552] Updated weights for policy 0, policy_version 3660 (0.0010) [2023-10-14 17:43:30,393][61552] Updated weights for policy 0, policy_version 3670 (0.0009) [2023-10-14 17:43:30,440][61585] Updated weights for policy 1, policy_version 3650 (0.0008) [2023-10-14 17:43:30,763][61552] Updated weights for policy 0, policy_version 3680 (0.0007) [2023-10-14 17:43:30,798][61585] Updated weights for policy 1, policy_version 3660 (0.0008) [2023-10-14 17:43:31,170][61585] Updated weights for policy 1, policy_version 3670 (0.0010) [2023-10-14 17:43:31,539][61585] Updated weights for policy 1, policy_version 3680 (0.0009) [2023-10-14 17:43:33,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 7536640. Throughput: 0: 1649.4, 1: 1654.0. Samples: 1887748. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:43:33,344][60425] Avg episode reward: [(0, '19.950'), (1, '17.370')] [2023-10-14 17:43:34,856][61552] Updated weights for policy 0, policy_version 3690 (0.0008) [2023-10-14 17:43:35,228][61552] Updated weights for policy 0, policy_version 3700 (0.0007) [2023-10-14 17:43:35,585][61585] Updated weights for policy 1, policy_version 3690 (0.0007) [2023-10-14 17:43:35,593][61552] Updated weights for policy 0, policy_version 3710 (0.0008) [2023-10-14 17:43:35,965][61585] Updated weights for policy 1, policy_version 3700 (0.0007) [2023-10-14 17:43:36,323][61585] Updated weights for policy 1, policy_version 3710 (0.0010) [2023-10-14 17:43:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 7602176. Throughput: 0: 1662.8, 1: 1659.7. Samples: 1907428. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:43:38,345][60425] Avg episode reward: [(0, '21.400'), (1, '18.650')] [2023-10-14 17:43:38,346][61172] Saving new best policy, reward=21.400! [2023-10-14 17:43:38,346][61248] Saving new best policy, reward=18.650! [2023-10-14 17:43:39,702][61552] Updated weights for policy 0, policy_version 3720 (0.0007) [2023-10-14 17:43:40,060][61552] Updated weights for policy 0, policy_version 3730 (0.0009) [2023-10-14 17:43:40,341][61585] Updated weights for policy 1, policy_version 3720 (0.0007) [2023-10-14 17:43:40,429][61552] Updated weights for policy 0, policy_version 3740 (0.0009) [2023-10-14 17:43:40,716][61585] Updated weights for policy 1, policy_version 3730 (0.0008) [2023-10-14 17:43:41,075][61585] Updated weights for policy 1, policy_version 3740 (0.0008) [2023-10-14 17:43:43,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 7667712. Throughput: 0: 1663.2, 1: 1671.3. Samples: 1928230. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:43:43,345][60425] Avg episode reward: [(0, '20.830'), (1, '19.090')] [2023-10-14 17:43:43,353][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000003744_3833856.pth... [2023-10-14 17:43:43,353][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000003744_3833856.pth... [2023-10-14 17:43:43,392][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000002208_2260992.pth [2023-10-14 17:43:43,392][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000002208_2260992.pth [2023-10-14 17:43:43,398][61248] Saving new best policy, reward=19.090! [2023-10-14 17:43:44,559][61552] Updated weights for policy 0, policy_version 3750 (0.0007) [2023-10-14 17:43:44,934][61552] Updated weights for policy 0, policy_version 3760 (0.0009) [2023-10-14 17:43:45,088][61585] Updated weights for policy 1, policy_version 3750 (0.0007) [2023-10-14 17:43:45,300][61552] Updated weights for policy 0, policy_version 3770 (0.0007) [2023-10-14 17:43:45,449][61585] Updated weights for policy 1, policy_version 3760 (0.0008) [2023-10-14 17:43:45,815][61585] Updated weights for policy 1, policy_version 3770 (0.0010) [2023-10-14 17:43:48,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 7733248. Throughput: 0: 1652.1, 1: 1656.9. Samples: 1937502. Policy #0 lag: (min: 17.0, avg: 44.7, max: 48.0) [2023-10-14 17:43:48,344][60425] Avg episode reward: [(0, '21.020'), (1, '19.390')] [2023-10-14 17:43:48,344][61248] Saving new best policy, reward=19.390! [2023-10-14 17:43:49,387][61552] Updated weights for policy 0, policy_version 3780 (0.0008) [2023-10-14 17:43:49,761][61552] Updated weights for policy 0, policy_version 3790 (0.0010) [2023-10-14 17:43:49,959][61585] Updated weights for policy 1, policy_version 3780 (0.0008) [2023-10-14 17:43:50,122][61552] Updated weights for policy 0, policy_version 3800 (0.0007) [2023-10-14 17:43:50,325][61585] Updated weights for policy 1, policy_version 3790 (0.0010) [2023-10-14 17:43:50,686][61585] Updated weights for policy 1, policy_version 3800 (0.0009) [2023-10-14 17:43:53,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 7798784. Throughput: 0: 1658.4, 1: 1671.8. Samples: 1957456. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-14 17:43:53,344][60425] Avg episode reward: [(0, '20.300'), (1, '18.840')] [2023-10-14 17:43:54,259][61552] Updated weights for policy 0, policy_version 3810 (0.0008) [2023-10-14 17:43:54,614][61552] Updated weights for policy 0, policy_version 3820 (0.0010) [2023-10-14 17:43:54,728][61585] Updated weights for policy 1, policy_version 3810 (0.0008) [2023-10-14 17:43:54,983][61552] Updated weights for policy 0, policy_version 3830 (0.0009) [2023-10-14 17:43:55,099][61585] Updated weights for policy 1, policy_version 3820 (0.0007) [2023-10-14 17:43:55,352][61552] Updated weights for policy 0, policy_version 3840 (0.0008) [2023-10-14 17:43:55,460][61585] Updated weights for policy 1, policy_version 3830 (0.0007) [2023-10-14 17:43:55,831][61585] Updated weights for policy 1, policy_version 3840 (0.0008) [2023-10-14 17:43:58,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 7864320. Throughput: 0: 1659.2, 1: 1671.7. Samples: 1978022. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-14 17:43:58,344][60425] Avg episode reward: [(0, '20.790'), (1, '19.790')] [2023-10-14 17:43:58,354][61248] Saving new best policy, reward=19.790! [2023-10-14 17:43:59,590][61552] Updated weights for policy 0, policy_version 3850 (0.0009) [2023-10-14 17:43:59,967][61552] Updated weights for policy 0, policy_version 3860 (0.0008) [2023-10-14 17:44:00,049][61585] Updated weights for policy 1, policy_version 3850 (0.0009) [2023-10-14 17:44:00,330][61552] Updated weights for policy 0, policy_version 3870 (0.0009) [2023-10-14 17:44:00,424][61585] Updated weights for policy 1, policy_version 3860 (0.0008) [2023-10-14 17:44:00,801][61585] Updated weights for policy 1, policy_version 3870 (0.0009) [2023-10-14 17:44:03,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 7929856. Throughput: 0: 1650.4, 1: 1648.1. Samples: 1987006. Policy #0 lag: (min: 17.0, avg: 27.0, max: 49.0) [2023-10-14 17:44:03,344][60425] Avg episode reward: [(0, '20.870'), (1, '19.110')] [2023-10-14 17:44:04,579][61552] Updated weights for policy 0, policy_version 3880 (0.0011) [2023-10-14 17:44:04,922][61585] Updated weights for policy 1, policy_version 3880 (0.0008) [2023-10-14 17:44:04,958][61552] Updated weights for policy 0, policy_version 3890 (0.0009) [2023-10-14 17:44:05,278][61585] Updated weights for policy 1, policy_version 3890 (0.0009) [2023-10-14 17:44:05,317][61552] Updated weights for policy 0, policy_version 3900 (0.0009) [2023-10-14 17:44:05,646][61585] Updated weights for policy 1, policy_version 3900 (0.0008) [2023-10-14 17:44:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 7995392. Throughput: 0: 1655.2, 1: 1666.5. Samples: 2007154. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:44:08,344][60425] Avg episode reward: [(0, '20.640'), (1, '18.970')] [2023-10-14 17:44:09,378][61552] Updated weights for policy 0, policy_version 3910 (0.0008) [2023-10-14 17:44:09,674][61585] Updated weights for policy 1, policy_version 3910 (0.0008) [2023-10-14 17:44:09,755][61552] Updated weights for policy 0, policy_version 3920 (0.0009) [2023-10-14 17:44:10,044][61585] Updated weights for policy 1, policy_version 3920 (0.0010) [2023-10-14 17:44:10,125][61552] Updated weights for policy 0, policy_version 3930 (0.0010) [2023-10-14 17:44:10,412][61585] Updated weights for policy 1, policy_version 3930 (0.0009) [2023-10-14 17:44:13,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 8060928. Throughput: 0: 1654.1, 1: 1670.3. Samples: 2027576. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:44:13,344][60425] Avg episode reward: [(0, '20.920'), (1, '18.950')] [2023-10-14 17:44:14,405][61552] Updated weights for policy 0, policy_version 3940 (0.0009) [2023-10-14 17:44:14,625][61585] Updated weights for policy 1, policy_version 3940 (0.0009) [2023-10-14 17:44:14,786][61552] Updated weights for policy 0, policy_version 3950 (0.0009) [2023-10-14 17:44:15,019][61585] Updated weights for policy 1, policy_version 3950 (0.0008) [2023-10-14 17:44:15,154][61552] Updated weights for policy 0, policy_version 3960 (0.0008) [2023-10-14 17:44:15,383][61585] Updated weights for policy 1, policy_version 3960 (0.0007) [2023-10-14 17:44:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 8126464. Throughput: 0: 1651.5, 1: 1649.7. Samples: 2036304. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-14 17:44:18,344][60425] Avg episode reward: [(0, '21.040'), (1, '17.410')] [2023-10-14 17:44:19,143][61552] Updated weights for policy 0, policy_version 3970 (0.0008) [2023-10-14 17:44:19,468][61585] Updated weights for policy 1, policy_version 3970 (0.0007) [2023-10-14 17:44:19,502][61552] Updated weights for policy 0, policy_version 3980 (0.0009) [2023-10-14 17:44:19,841][61585] Updated weights for policy 1, policy_version 3980 (0.0008) [2023-10-14 17:44:19,868][61552] Updated weights for policy 0, policy_version 3990 (0.0009) [2023-10-14 17:44:20,206][61585] Updated weights for policy 1, policy_version 3990 (0.0008) [2023-10-14 17:44:20,241][61552] Updated weights for policy 0, policy_version 4000 (0.0007) [2023-10-14 17:44:20,574][61585] Updated weights for policy 1, policy_version 4000 (0.0009) [2023-10-14 17:44:23,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 8192000. Throughput: 0: 1651.9, 1: 1664.8. Samples: 2056678. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-14 17:44:23,344][60425] Avg episode reward: [(0, '20.620'), (1, '18.700')] [2023-10-14 17:44:24,363][61552] Updated weights for policy 0, policy_version 4010 (0.0007) [2023-10-14 17:44:24,735][61552] Updated weights for policy 0, policy_version 4020 (0.0008) [2023-10-14 17:44:24,788][61585] Updated weights for policy 1, policy_version 4010 (0.0008) [2023-10-14 17:44:25,102][61552] Updated weights for policy 0, policy_version 4030 (0.0007) [2023-10-14 17:44:25,159][61585] Updated weights for policy 1, policy_version 4020 (0.0009) [2023-10-14 17:44:25,520][61585] Updated weights for policy 1, policy_version 4030 (0.0007) [2023-10-14 17:44:28,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 8257536. Throughput: 0: 1651.1, 1: 1656.1. Samples: 2077052. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:44:28,344][60425] Avg episode reward: [(0, '19.610'), (1, '19.390')] [2023-10-14 17:44:29,197][61552] Updated weights for policy 0, policy_version 4040 (0.0009) [2023-10-14 17:44:29,560][61552] Updated weights for policy 0, policy_version 4050 (0.0008) [2023-10-14 17:44:29,637][61585] Updated weights for policy 1, policy_version 4040 (0.0008) [2023-10-14 17:44:29,938][61552] Updated weights for policy 0, policy_version 4060 (0.0007) [2023-10-14 17:44:30,001][61585] Updated weights for policy 1, policy_version 4050 (0.0007) [2023-10-14 17:44:30,365][61585] Updated weights for policy 1, policy_version 4060 (0.0007) [2023-10-14 17:44:33,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 8323072. Throughput: 0: 1655.1, 1: 1648.4. Samples: 2086160. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:44:33,344][60425] Avg episode reward: [(0, '20.510'), (1, '20.290')] [2023-10-14 17:44:33,344][61248] Saving new best policy, reward=20.290! [2023-10-14 17:44:34,182][61552] Updated weights for policy 0, policy_version 4070 (0.0010) [2023-10-14 17:44:34,545][61552] Updated weights for policy 0, policy_version 4080 (0.0008) [2023-10-14 17:44:34,548][61585] Updated weights for policy 1, policy_version 4070 (0.0007) [2023-10-14 17:44:34,907][61585] Updated weights for policy 1, policy_version 4080 (0.0008) [2023-10-14 17:44:34,911][61552] Updated weights for policy 0, policy_version 4090 (0.0008) [2023-10-14 17:44:35,276][61585] Updated weights for policy 1, policy_version 4090 (0.0007) [2023-10-14 17:44:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 8388608. Throughput: 0: 1656.1, 1: 1658.2. Samples: 2106598. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-14 17:44:38,344][60425] Avg episode reward: [(0, '20.010'), (1, '20.630')] [2023-10-14 17:44:38,344][61248] Saving new best policy, reward=20.630! [2023-10-14 17:44:38,975][61552] Updated weights for policy 0, policy_version 4100 (0.0008) [2023-10-14 17:44:39,353][61552] Updated weights for policy 0, policy_version 4110 (0.0007) [2023-10-14 17:44:39,453][61585] Updated weights for policy 1, policy_version 4100 (0.0008) [2023-10-14 17:44:39,734][61552] Updated weights for policy 0, policy_version 4120 (0.0007) [2023-10-14 17:44:39,819][61585] Updated weights for policy 1, policy_version 4110 (0.0009) [2023-10-14 17:44:40,187][61585] Updated weights for policy 1, policy_version 4120 (0.0009) [2023-10-14 17:44:43,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 8454144. Throughput: 0: 1656.1, 1: 1653.6. Samples: 2126958. Policy #0 lag: (min: 7.0, avg: 14.2, max: 39.0) [2023-10-14 17:44:43,344][60425] Avg episode reward: [(0, '18.180'), (1, '22.140')] [2023-10-14 17:44:43,352][61248] Saving new best policy, reward=22.140! [2023-10-14 17:44:43,892][61552] Updated weights for policy 0, policy_version 4130 (0.0007) [2023-10-14 17:44:44,263][61552] Updated weights for policy 0, policy_version 4140 (0.0007) [2023-10-14 17:44:44,288][61585] Updated weights for policy 1, policy_version 4130 (0.0008) [2023-10-14 17:44:44,634][61552] Updated weights for policy 0, policy_version 4150 (0.0010) [2023-10-14 17:44:44,665][61585] Updated weights for policy 1, policy_version 4140 (0.0008) [2023-10-14 17:44:44,997][61552] Updated weights for policy 0, policy_version 4160 (0.0008) [2023-10-14 17:44:45,027][61585] Updated weights for policy 1, policy_version 4150 (0.0008) [2023-10-14 17:44:45,402][61585] Updated weights for policy 1, policy_version 4160 (0.0007) [2023-10-14 17:44:48,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 8519680. Throughput: 0: 1657.2, 1: 1653.1. Samples: 2135970. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:44:48,344][60425] Avg episode reward: [(0, '19.730'), (1, '21.250')] [2023-10-14 17:44:49,099][61552] Updated weights for policy 0, policy_version 4170 (0.0008) [2023-10-14 17:44:49,464][61552] Updated weights for policy 0, policy_version 4180 (0.0009) [2023-10-14 17:44:49,686][61585] Updated weights for policy 1, policy_version 4170 (0.0008) [2023-10-14 17:44:49,831][61552] Updated weights for policy 0, policy_version 4190 (0.0007) [2023-10-14 17:44:50,052][61585] Updated weights for policy 1, policy_version 4180 (0.0007) [2023-10-14 17:44:50,414][61585] Updated weights for policy 1, policy_version 4190 (0.0009) [2023-10-14 17:44:53,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 8585216. Throughput: 0: 1659.5, 1: 1655.6. Samples: 2156334. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:44:53,344][60425] Avg episode reward: [(0, '20.210'), (1, '22.510')] [2023-10-14 17:44:53,345][61248] Saving new best policy, reward=22.510! [2023-10-14 17:44:53,978][61552] Updated weights for policy 0, policy_version 4200 (0.0008) [2023-10-14 17:44:54,356][61552] Updated weights for policy 0, policy_version 4210 (0.0007) [2023-10-14 17:44:54,550][61585] Updated weights for policy 1, policy_version 4200 (0.0008) [2023-10-14 17:44:54,714][61552] Updated weights for policy 0, policy_version 4220 (0.0009) [2023-10-14 17:44:54,918][61585] Updated weights for policy 1, policy_version 4210 (0.0009) [2023-10-14 17:44:55,283][61585] Updated weights for policy 1, policy_version 4220 (0.0008) [2023-10-14 17:44:58,344][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 8650752. Throughput: 0: 1662.4, 1: 1656.0. Samples: 2176906. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:44:58,345][60425] Avg episode reward: [(0, '20.980'), (1, '24.280')] [2023-10-14 17:44:58,353][61248] Saving new best policy, reward=24.280! [2023-10-14 17:44:59,011][61552] Updated weights for policy 0, policy_version 4230 (0.0008) [2023-10-14 17:44:59,389][61552] Updated weights for policy 0, policy_version 4240 (0.0007) [2023-10-14 17:44:59,564][61585] Updated weights for policy 1, policy_version 4230 (0.0007) [2023-10-14 17:44:59,766][61552] Updated weights for policy 0, policy_version 4250 (0.0007) [2023-10-14 17:44:59,951][61585] Updated weights for policy 1, policy_version 4240 (0.0008) [2023-10-14 17:45:00,319][61585] Updated weights for policy 1, policy_version 4250 (0.0008) [2023-10-14 17:45:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 8716288. Throughput: 0: 1663.7, 1: 1652.5. Samples: 2185536. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:45:03,344][60425] Avg episode reward: [(0, '21.250'), (1, '24.710')] [2023-10-14 17:45:03,346][61248] Saving new best policy, reward=24.710! [2023-10-14 17:45:03,699][61552] Updated weights for policy 0, policy_version 4260 (0.0008) [2023-10-14 17:45:04,075][61552] Updated weights for policy 0, policy_version 4270 (0.0008) [2023-10-14 17:45:04,438][61552] Updated weights for policy 0, policy_version 4280 (0.0009) [2023-10-14 17:45:04,522][61585] Updated weights for policy 1, policy_version 4260 (0.0007) [2023-10-14 17:45:04,884][61585] Updated weights for policy 1, policy_version 4270 (0.0009) [2023-10-14 17:45:05,262][61585] Updated weights for policy 1, policy_version 4280 (0.0008) [2023-10-14 17:45:08,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 8781824. Throughput: 0: 1669.6, 1: 1649.6. Samples: 2206042. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-14 17:45:08,344][60425] Avg episode reward: [(0, '21.890'), (1, '22.670')] [2023-10-14 17:45:08,380][61552] Updated weights for policy 0, policy_version 4290 (0.0008) [2023-10-14 17:45:08,751][61552] Updated weights for policy 0, policy_version 4300 (0.0008) [2023-10-14 17:45:09,120][61552] Updated weights for policy 0, policy_version 4310 (0.0010) [2023-10-14 17:45:09,209][61585] Updated weights for policy 1, policy_version 4290 (0.0008) [2023-10-14 17:45:09,478][61172] Saving new best policy, reward=21.890! [2023-10-14 17:45:09,478][61552] Updated weights for policy 0, policy_version 4320 (0.0008) [2023-10-14 17:45:09,571][61585] Updated weights for policy 1, policy_version 4300 (0.0008) [2023-10-14 17:45:09,943][61585] Updated weights for policy 1, policy_version 4310 (0.0009) [2023-10-14 17:45:10,315][61585] Updated weights for policy 1, policy_version 4320 (0.0007) [2023-10-14 17:45:13,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 8847360. Throughput: 0: 1672.0, 1: 1653.2. Samples: 2226684. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-14 17:45:13,344][60425] Avg episode reward: [(0, '21.080'), (1, '23.180')] [2023-10-14 17:45:13,579][61552] Updated weights for policy 0, policy_version 4330 (0.0010) [2023-10-14 17:45:13,949][61552] Updated weights for policy 0, policy_version 4340 (0.0008) [2023-10-14 17:45:14,310][61552] Updated weights for policy 0, policy_version 4350 (0.0007) [2023-10-14 17:45:14,310][61585] Updated weights for policy 1, policy_version 4330 (0.0008) [2023-10-14 17:45:14,675][61585] Updated weights for policy 1, policy_version 4340 (0.0007) [2023-10-14 17:45:15,047][61585] Updated weights for policy 1, policy_version 4350 (0.0008) [2023-10-14 17:45:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 8912896. Throughput: 0: 1671.6, 1: 1656.9. Samples: 2235942. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:45:18,344][60425] Avg episode reward: [(0, '21.940'), (1, '22.540')] [2023-10-14 17:45:18,618][61552] Updated weights for policy 0, policy_version 4360 (0.0008) [2023-10-14 17:45:18,986][61552] Updated weights for policy 0, policy_version 4370 (0.0007) [2023-10-14 17:45:19,185][61585] Updated weights for policy 1, policy_version 4360 (0.0009) [2023-10-14 17:45:19,365][61552] Updated weights for policy 0, policy_version 4380 (0.0009) [2023-10-14 17:45:19,502][61172] Saving new best policy, reward=21.940! [2023-10-14 17:45:19,550][61585] Updated weights for policy 1, policy_version 4370 (0.0008) [2023-10-14 17:45:19,909][61585] Updated weights for policy 1, policy_version 4380 (0.0009) [2023-10-14 17:45:23,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 8978432. Throughput: 0: 1667.9, 1: 1658.7. Samples: 2256296. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:45:23,345][60425] Avg episode reward: [(0, '21.550'), (1, '22.700')] [2023-10-14 17:45:23,436][61552] Updated weights for policy 0, policy_version 4390 (0.0009) [2023-10-14 17:45:23,820][61552] Updated weights for policy 0, policy_version 4400 (0.0008) [2023-10-14 17:45:23,979][61585] Updated weights for policy 1, policy_version 4390 (0.0007) [2023-10-14 17:45:24,187][61552] Updated weights for policy 0, policy_version 4410 (0.0009) [2023-10-14 17:45:24,341][61585] Updated weights for policy 1, policy_version 4400 (0.0007) [2023-10-14 17:45:24,713][61585] Updated weights for policy 1, policy_version 4410 (0.0011) [2023-10-14 17:45:28,185][61552] Updated weights for policy 0, policy_version 4420 (0.0008) [2023-10-14 17:45:28,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 9043968. Throughput: 0: 1667.6, 1: 1664.1. Samples: 2276884. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:45:28,344][60425] Avg episode reward: [(0, '21.410'), (1, '24.600')] [2023-10-14 17:45:28,553][61552] Updated weights for policy 0, policy_version 4430 (0.0010) [2023-10-14 17:45:28,841][61585] Updated weights for policy 1, policy_version 4420 (0.0009) [2023-10-14 17:45:28,926][61552] Updated weights for policy 0, policy_version 4440 (0.0008) [2023-10-14 17:45:29,204][61585] Updated weights for policy 1, policy_version 4430 (0.0007) [2023-10-14 17:45:29,583][61585] Updated weights for policy 1, policy_version 4440 (0.0008) [2023-10-14 17:45:33,064][61552] Updated weights for policy 0, policy_version 4450 (0.0008) [2023-10-14 17:45:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13107.2). Total num frames: 9109504. Throughput: 0: 1670.4, 1: 1660.7. Samples: 2285868. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:45:33,345][60425] Avg episode reward: [(0, '23.110'), (1, '24.610')] [2023-10-14 17:45:33,434][61552] Updated weights for policy 0, policy_version 4460 (0.0009) [2023-10-14 17:45:33,702][61585] Updated weights for policy 1, policy_version 4450 (0.0007) [2023-10-14 17:45:33,793][61552] Updated weights for policy 0, policy_version 4470 (0.0009) [2023-10-14 17:45:34,066][61585] Updated weights for policy 1, policy_version 4460 (0.0009) [2023-10-14 17:45:34,160][61172] Saving new best policy, reward=23.110! [2023-10-14 17:45:34,160][61552] Updated weights for policy 0, policy_version 4480 (0.0009) [2023-10-14 17:45:34,428][61585] Updated weights for policy 1, policy_version 4470 (0.0010) [2023-10-14 17:45:34,802][61585] Updated weights for policy 1, policy_version 4480 (0.0011) [2023-10-14 17:45:38,128][61552] Updated weights for policy 0, policy_version 4490 (0.0007) [2023-10-14 17:45:38,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 9175040. Throughput: 0: 1669.8, 1: 1655.8. Samples: 2305986. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:45:38,344][60425] Avg episode reward: [(0, '21.570'), (1, '24.360')] [2023-10-14 17:45:38,496][61552] Updated weights for policy 0, policy_version 4500 (0.0009) [2023-10-14 17:45:38,861][61552] Updated weights for policy 0, policy_version 4510 (0.0008) [2023-10-14 17:45:39,085][61585] Updated weights for policy 1, policy_version 4490 (0.0008) [2023-10-14 17:45:39,452][61585] Updated weights for policy 1, policy_version 4500 (0.0008) [2023-10-14 17:45:39,818][61585] Updated weights for policy 1, policy_version 4510 (0.0009) [2023-10-14 17:45:43,073][61552] Updated weights for policy 0, policy_version 4520 (0.0009) [2023-10-14 17:45:43,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 9240576. Throughput: 0: 1673.4, 1: 1659.5. Samples: 2326884. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:45:43,344][60425] Avg episode reward: [(0, '21.420'), (1, '24.570')] [2023-10-14 17:45:43,352][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000004512_4620288.pth... [2023-10-14 17:45:43,396][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000002976_3047424.pth [2023-10-14 17:45:43,435][61552] Updated weights for policy 0, policy_version 4530 (0.0007) [2023-10-14 17:45:43,804][61552] Updated weights for policy 0, policy_version 4540 (0.0007) [2023-10-14 17:45:43,892][61585] Updated weights for policy 1, policy_version 4520 (0.0009) [2023-10-14 17:45:43,953][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000004544_4653056.pth... [2023-10-14 17:45:43,992][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000002976_3047424.pth [2023-10-14 17:45:44,252][61585] Updated weights for policy 1, policy_version 4530 (0.0010) [2023-10-14 17:45:44,619][61585] Updated weights for policy 1, policy_version 4540 (0.0007) [2023-10-14 17:45:47,946][61552] Updated weights for policy 0, policy_version 4550 (0.0010) [2023-10-14 17:45:48,313][61552] Updated weights for policy 0, policy_version 4560 (0.0009) [2023-10-14 17:45:48,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 9306112. Throughput: 0: 1673.3, 1: 1669.2. Samples: 2335946. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:45:48,344][60425] Avg episode reward: [(0, '22.290'), (1, '25.020')] [2023-10-14 17:45:48,344][61248] Saving new best policy, reward=25.020! [2023-10-14 17:45:48,682][61552] Updated weights for policy 0, policy_version 4570 (0.0011) [2023-10-14 17:45:48,842][61585] Updated weights for policy 1, policy_version 4550 (0.0008) [2023-10-14 17:45:49,207][61585] Updated weights for policy 1, policy_version 4560 (0.0008) [2023-10-14 17:45:49,577][61585] Updated weights for policy 1, policy_version 4570 (0.0007) [2023-10-14 17:45:52,725][61552] Updated weights for policy 0, policy_version 4580 (0.0008) [2023-10-14 17:45:53,105][61552] Updated weights for policy 0, policy_version 4590 (0.0008) [2023-10-14 17:45:53,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 9371648. Throughput: 0: 1668.7, 1: 1671.3. Samples: 2356340. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:45:53,344][60425] Avg episode reward: [(0, '21.880'), (1, '25.770')] [2023-10-14 17:45:53,346][61248] Saving new best policy, reward=25.770! [2023-10-14 17:45:53,474][61552] Updated weights for policy 0, policy_version 4600 (0.0008) [2023-10-14 17:45:53,619][61585] Updated weights for policy 1, policy_version 4580 (0.0007) [2023-10-14 17:45:53,982][61585] Updated weights for policy 1, policy_version 4590 (0.0008) [2023-10-14 17:45:54,352][61585] Updated weights for policy 1, policy_version 4600 (0.0009) [2023-10-14 17:45:57,630][61552] Updated weights for policy 0, policy_version 4610 (0.0009) [2023-10-14 17:45:57,998][61552] Updated weights for policy 0, policy_version 4620 (0.0008) [2023-10-14 17:45:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 9437184. Throughput: 0: 1659.7, 1: 1671.9. Samples: 2376608. Policy #0 lag: (min: 10.0, avg: 16.1, max: 42.0) [2023-10-14 17:45:58,344][60425] Avg episode reward: [(0, '22.400'), (1, '25.340')] [2023-10-14 17:45:58,356][61552] Updated weights for policy 0, policy_version 4630 (0.0009) [2023-10-14 17:45:58,364][61585] Updated weights for policy 1, policy_version 4610 (0.0009) [2023-10-14 17:45:58,727][61552] Updated weights for policy 0, policy_version 4640 (0.0008) [2023-10-14 17:45:58,733][61585] Updated weights for policy 1, policy_version 4620 (0.0007) [2023-10-14 17:45:59,096][61585] Updated weights for policy 1, policy_version 4630 (0.0009) [2023-10-14 17:45:59,461][61585] Updated weights for policy 1, policy_version 4640 (0.0008) [2023-10-14 17:46:02,728][61552] Updated weights for policy 0, policy_version 4650 (0.0011) [2023-10-14 17:46:03,104][61552] Updated weights for policy 0, policy_version 4660 (0.0011) [2023-10-14 17:46:03,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 9502720. Throughput: 0: 1663.3, 1: 1669.7. Samples: 2385928. Policy #0 lag: (min: 10.0, avg: 16.1, max: 42.0) [2023-10-14 17:46:03,344][60425] Avg episode reward: [(0, '22.560'), (1, '24.940')] [2023-10-14 17:46:03,487][61552] Updated weights for policy 0, policy_version 4670 (0.0008) [2023-10-14 17:46:03,655][61585] Updated weights for policy 1, policy_version 4650 (0.0009) [2023-10-14 17:46:04,020][61585] Updated weights for policy 1, policy_version 4660 (0.0009) [2023-10-14 17:46:04,390][61585] Updated weights for policy 1, policy_version 4670 (0.0007) [2023-10-14 17:46:07,703][61552] Updated weights for policy 0, policy_version 4680 (0.0009) [2023-10-14 17:46:08,080][61552] Updated weights for policy 0, policy_version 4690 (0.0009) [2023-10-14 17:46:08,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 9568256. Throughput: 0: 1668.2, 1: 1664.8. Samples: 2406278. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:46:08,344][60425] Avg episode reward: [(0, '22.140'), (1, '25.810')] [2023-10-14 17:46:08,456][61552] Updated weights for policy 0, policy_version 4700 (0.0007) [2023-10-14 17:46:08,620][61585] Updated weights for policy 1, policy_version 4680 (0.0007) [2023-10-14 17:46:08,980][61585] Updated weights for policy 1, policy_version 4690 (0.0008) [2023-10-14 17:46:09,351][61585] Updated weights for policy 1, policy_version 4700 (0.0009) [2023-10-14 17:46:09,491][61248] Saving new best policy, reward=25.810! [2023-10-14 17:46:12,465][61552] Updated weights for policy 0, policy_version 4710 (0.0009) [2023-10-14 17:46:12,824][61552] Updated weights for policy 0, policy_version 4720 (0.0010) [2023-10-14 17:46:13,195][61552] Updated weights for policy 0, policy_version 4730 (0.0009) [2023-10-14 17:46:13,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 9633792. Throughput: 0: 1660.8, 1: 1664.0. Samples: 2426502. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:46:13,344][60425] Avg episode reward: [(0, '22.970'), (1, '26.380')] [2023-10-14 17:46:13,374][61585] Updated weights for policy 1, policy_version 4710 (0.0009) [2023-10-14 17:46:13,736][61585] Updated weights for policy 1, policy_version 4720 (0.0009) [2023-10-14 17:46:14,096][61585] Updated weights for policy 1, policy_version 4730 (0.0009) [2023-10-14 17:46:14,318][61248] Saving new best policy, reward=26.380! [2023-10-14 17:46:17,331][61552] Updated weights for policy 0, policy_version 4740 (0.0008) [2023-10-14 17:46:17,698][61552] Updated weights for policy 0, policy_version 4750 (0.0009) [2023-10-14 17:46:18,074][61552] Updated weights for policy 0, policy_version 4760 (0.0010) [2023-10-14 17:46:18,257][61585] Updated weights for policy 1, policy_version 4740 (0.0008) [2023-10-14 17:46:18,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 9699328. Throughput: 0: 1668.5, 1: 1668.5. Samples: 2436034. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:46:18,344][60425] Avg episode reward: [(0, '21.910'), (1, '25.670')] [2023-10-14 17:46:18,622][61585] Updated weights for policy 1, policy_version 4750 (0.0009) [2023-10-14 17:46:18,994][61585] Updated weights for policy 1, policy_version 4760 (0.0012) [2023-10-14 17:46:22,310][61552] Updated weights for policy 0, policy_version 4770 (0.0008) [2023-10-14 17:46:22,676][61552] Updated weights for policy 0, policy_version 4780 (0.0008) [2023-10-14 17:46:22,932][61585] Updated weights for policy 1, policy_version 4770 (0.0008) [2023-10-14 17:46:23,042][61552] Updated weights for policy 0, policy_version 4790 (0.0007) [2023-10-14 17:46:23,300][61585] Updated weights for policy 1, policy_version 4780 (0.0007) [2023-10-14 17:46:23,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 9764864. Throughput: 0: 1661.0, 1: 1676.9. Samples: 2456192. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:46:23,344][60425] Avg episode reward: [(0, '21.680'), (1, '26.980')] [2023-10-14 17:46:23,418][61552] Updated weights for policy 0, policy_version 4800 (0.0008) [2023-10-14 17:46:23,676][61585] Updated weights for policy 1, policy_version 4790 (0.0011) [2023-10-14 17:46:24,033][61248] Saving new best policy, reward=26.980! [2023-10-14 17:46:24,040][61585] Updated weights for policy 1, policy_version 4800 (0.0009) [2023-10-14 17:46:27,857][61552] Updated weights for policy 0, policy_version 4810 (0.0009) [2023-10-14 17:46:28,058][61585] Updated weights for policy 1, policy_version 4810 (0.0007) [2023-10-14 17:46:28,226][61552] Updated weights for policy 0, policy_version 4820 (0.0008) [2023-10-14 17:46:28,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 9830400. Throughput: 0: 1648.0, 1: 1673.3. Samples: 2476342. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-14 17:46:28,344][60425] Avg episode reward: [(0, '21.810'), (1, '27.810')] [2023-10-14 17:46:28,430][61585] Updated weights for policy 1, policy_version 4820 (0.0008) [2023-10-14 17:46:28,596][61552] Updated weights for policy 0, policy_version 4830 (0.0009) [2023-10-14 17:46:28,793][61585] Updated weights for policy 1, policy_version 4830 (0.0007) [2023-10-14 17:46:28,861][61248] Saving new best policy, reward=27.810! [2023-10-14 17:46:32,874][61552] Updated weights for policy 0, policy_version 4840 (0.0009) [2023-10-14 17:46:32,916][61585] Updated weights for policy 1, policy_version 4840 (0.0009) [2023-10-14 17:46:33,249][61552] Updated weights for policy 0, policy_version 4850 (0.0009) [2023-10-14 17:46:33,281][61585] Updated weights for policy 1, policy_version 4850 (0.0008) [2023-10-14 17:46:33,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 9895936. Throughput: 0: 1654.7, 1: 1671.7. Samples: 2485634. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-14 17:46:33,344][60425] Avg episode reward: [(0, '22.570'), (1, '26.690')] [2023-10-14 17:46:33,615][61552] Updated weights for policy 0, policy_version 4860 (0.0009) [2023-10-14 17:46:33,640][61585] Updated weights for policy 1, policy_version 4860 (0.0009) [2023-10-14 17:46:37,673][61552] Updated weights for policy 0, policy_version 4870 (0.0009) [2023-10-14 17:46:37,865][61585] Updated weights for policy 1, policy_version 4870 (0.0009) [2023-10-14 17:46:38,040][61552] Updated weights for policy 0, policy_version 4880 (0.0007) [2023-10-14 17:46:38,228][61585] Updated weights for policy 1, policy_version 4880 (0.0007) [2023-10-14 17:46:38,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 9961472. Throughput: 0: 1652.0, 1: 1670.1. Samples: 2505832. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-14 17:46:38,344][60425] Avg episode reward: [(0, '20.820'), (1, '25.360')] [2023-10-14 17:46:38,408][61552] Updated weights for policy 0, policy_version 4890 (0.0008) [2023-10-14 17:46:38,600][61585] Updated weights for policy 1, policy_version 4890 (0.0008) [2023-10-14 17:46:42,478][61552] Updated weights for policy 0, policy_version 4900 (0.0009) [2023-10-14 17:46:42,781][61585] Updated weights for policy 1, policy_version 4900 (0.0007) [2023-10-14 17:46:42,841][61552] Updated weights for policy 0, policy_version 4910 (0.0008) [2023-10-14 17:46:43,152][61585] Updated weights for policy 1, policy_version 4910 (0.0007) [2023-10-14 17:46:43,207][61552] Updated weights for policy 0, policy_version 4920 (0.0009) [2023-10-14 17:46:43,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 10027008. Throughput: 0: 1651.4, 1: 1663.0. Samples: 2525758. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:46:43,344][60425] Avg episode reward: [(0, '22.590'), (1, '25.140')] [2023-10-14 17:46:43,512][61585] Updated weights for policy 1, policy_version 4920 (0.0007) [2023-10-14 17:46:47,366][61552] Updated weights for policy 0, policy_version 4930 (0.0008) [2023-10-14 17:46:47,694][61585] Updated weights for policy 1, policy_version 4930 (0.0008) [2023-10-14 17:46:47,740][61552] Updated weights for policy 0, policy_version 4940 (0.0009) [2023-10-14 17:46:48,057][61585] Updated weights for policy 1, policy_version 4940 (0.0010) [2023-10-14 17:46:48,109][61552] Updated weights for policy 0, policy_version 4950 (0.0009) [2023-10-14 17:46:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 10092544. Throughput: 0: 1654.8, 1: 1662.8. Samples: 2535222. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:46:48,344][60425] Avg episode reward: [(0, '22.150'), (1, '26.530')] [2023-10-14 17:46:48,421][61585] Updated weights for policy 1, policy_version 4950 (0.0008) [2023-10-14 17:46:48,475][61552] Updated weights for policy 0, policy_version 4960 (0.0008) [2023-10-14 17:46:48,785][61585] Updated weights for policy 1, policy_version 4960 (0.0008) [2023-10-14 17:46:52,526][61552] Updated weights for policy 0, policy_version 4970 (0.0007) [2023-10-14 17:46:52,839][61585] Updated weights for policy 1, policy_version 4970 (0.0009) [2023-10-14 17:46:52,888][61552] Updated weights for policy 0, policy_version 4980 (0.0008) [2023-10-14 17:46:53,204][61585] Updated weights for policy 1, policy_version 4980 (0.0010) [2023-10-14 17:46:53,260][61552] Updated weights for policy 0, policy_version 4990 (0.0007) [2023-10-14 17:46:53,343][60425] Fps is (10 sec: 16384.0, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 10190848. Throughput: 0: 1654.4, 1: 1668.0. Samples: 2555784. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-14 17:46:53,344][60425] Avg episode reward: [(0, '22.310'), (1, '25.250')] [2023-10-14 17:46:53,559][61585] Updated weights for policy 1, policy_version 4990 (0.0010) [2023-10-14 17:46:57,331][61552] Updated weights for policy 0, policy_version 5000 (0.0009) [2023-10-14 17:46:57,701][61552] Updated weights for policy 0, policy_version 5010 (0.0008) [2023-10-14 17:46:57,721][61585] Updated weights for policy 1, policy_version 5000 (0.0008) [2023-10-14 17:46:58,073][61552] Updated weights for policy 0, policy_version 5020 (0.0008) [2023-10-14 17:46:58,086][61585] Updated weights for policy 1, policy_version 5010 (0.0008) [2023-10-14 17:46:58,343][60425] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 10256384. Throughput: 0: 1648.7, 1: 1659.9. Samples: 2575388. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-14 17:46:58,344][60425] Avg episode reward: [(0, '20.370'), (1, '27.030')] [2023-10-14 17:46:58,456][61585] Updated weights for policy 1, policy_version 5020 (0.0009) [2023-10-14 17:47:02,229][61552] Updated weights for policy 0, policy_version 5030 (0.0009) [2023-10-14 17:47:02,484][61585] Updated weights for policy 1, policy_version 5030 (0.0009) [2023-10-14 17:47:02,602][61552] Updated weights for policy 0, policy_version 5040 (0.0007) [2023-10-14 17:47:02,836][61585] Updated weights for policy 1, policy_version 5040 (0.0008) [2023-10-14 17:47:02,976][61552] Updated weights for policy 0, policy_version 5050 (0.0007) [2023-10-14 17:47:03,209][61585] Updated weights for policy 1, policy_version 5050 (0.0008) [2023-10-14 17:47:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 10321920. Throughput: 0: 1652.0, 1: 1664.4. Samples: 2585274. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-14 17:47:03,344][60425] Avg episode reward: [(0, '21.380'), (1, '27.250')] [2023-10-14 17:47:06,779][61552] Updated weights for policy 0, policy_version 5060 (0.0010) [2023-10-14 17:47:07,138][61552] Updated weights for policy 0, policy_version 5070 (0.0009) [2023-10-14 17:47:07,428][61585] Updated weights for policy 1, policy_version 5060 (0.0008) [2023-10-14 17:47:07,508][61552] Updated weights for policy 0, policy_version 5080 (0.0008) [2023-10-14 17:47:07,792][61585] Updated weights for policy 1, policy_version 5070 (0.0007) [2023-10-14 17:47:08,170][61585] Updated weights for policy 1, policy_version 5080 (0.0009) [2023-10-14 17:47:08,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 10387456. Throughput: 0: 1661.8, 1: 1655.6. Samples: 2605472. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:47:08,344][60425] Avg episode reward: [(0, '23.010'), (1, '28.450')] [2023-10-14 17:47:08,454][61248] Saving new best policy, reward=28.450! [2023-10-14 17:47:11,620][61552] Updated weights for policy 0, policy_version 5090 (0.0008) [2023-10-14 17:47:11,992][61552] Updated weights for policy 0, policy_version 5100 (0.0007) [2023-10-14 17:47:12,155][61585] Updated weights for policy 1, policy_version 5090 (0.0009) [2023-10-14 17:47:12,351][61552] Updated weights for policy 0, policy_version 5110 (0.0009) [2023-10-14 17:47:12,521][61585] Updated weights for policy 1, policy_version 5100 (0.0008) [2023-10-14 17:47:12,729][61552] Updated weights for policy 0, policy_version 5120 (0.0008) [2023-10-14 17:47:12,900][61585] Updated weights for policy 1, policy_version 5110 (0.0009) [2023-10-14 17:47:13,270][61585] Updated weights for policy 1, policy_version 5120 (0.0009) [2023-10-14 17:47:13,343][60425] Fps is (10 sec: 16384.0, 60 sec: 14199.4, 300 sec: 13329.4). Total num frames: 10485760. Throughput: 0: 1645.1, 1: 1643.0. Samples: 2624304. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:47:13,344][60425] Avg episode reward: [(0, '21.670'), (1, '29.580')] [2023-10-14 17:47:13,352][61248] Saving new best policy, reward=29.580! [2023-10-14 17:47:17,095][61552] Updated weights for policy 0, policy_version 5130 (0.0007) [2023-10-14 17:47:17,466][61552] Updated weights for policy 0, policy_version 5140 (0.0008) [2023-10-14 17:47:17,548][61585] Updated weights for policy 1, policy_version 5130 (0.0007) [2023-10-14 17:47:17,826][61552] Updated weights for policy 0, policy_version 5150 (0.0008) [2023-10-14 17:47:17,918][61585] Updated weights for policy 1, policy_version 5140 (0.0009) [2023-10-14 17:47:18,277][61585] Updated weights for policy 1, policy_version 5150 (0.0008) [2023-10-14 17:47:18,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 10518528. Throughput: 0: 1668.3, 1: 1655.3. Samples: 2635196. Policy #0 lag: (min: 0.0, avg: 24.7, max: 32.0) [2023-10-14 17:47:18,344][60425] Avg episode reward: [(0, '21.830'), (1, '28.610')] [2023-10-14 17:47:22,044][61552] Updated weights for policy 0, policy_version 5160 (0.0008) [2023-10-14 17:47:22,416][61552] Updated weights for policy 0, policy_version 5170 (0.0009) [2023-10-14 17:47:22,579][61585] Updated weights for policy 1, policy_version 5160 (0.0008) [2023-10-14 17:47:22,779][61552] Updated weights for policy 0, policy_version 5180 (0.0008) [2023-10-14 17:47:22,945][61585] Updated weights for policy 1, policy_version 5170 (0.0008) [2023-10-14 17:47:23,315][61585] Updated weights for policy 1, policy_version 5180 (0.0009) [2023-10-14 17:47:23,343][60425] Fps is (10 sec: 9830.4, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 10584064. Throughput: 0: 1669.4, 1: 1660.2. Samples: 2655664. Policy #0 lag: (min: 0.0, avg: 24.7, max: 32.0) [2023-10-14 17:47:23,344][60425] Avg episode reward: [(0, '23.630'), (1, '26.510')] [2023-10-14 17:47:23,344][61172] Saving new best policy, reward=23.630! [2023-10-14 17:47:26,970][61552] Updated weights for policy 0, policy_version 5190 (0.0008) [2023-10-14 17:47:27,325][61552] Updated weights for policy 0, policy_version 5200 (0.0009) [2023-10-14 17:47:27,453][61585] Updated weights for policy 1, policy_version 5190 (0.0011) [2023-10-14 17:47:27,706][61552] Updated weights for policy 0, policy_version 5210 (0.0008) [2023-10-14 17:47:27,825][61585] Updated weights for policy 1, policy_version 5200 (0.0009) [2023-10-14 17:47:28,207][61585] Updated weights for policy 1, policy_version 5210 (0.0010) [2023-10-14 17:47:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 10649600. Throughput: 0: 1653.8, 1: 1653.0. Samples: 2674562. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:47:28,344][60425] Avg episode reward: [(0, '23.030'), (1, '24.600')] [2023-10-14 17:47:31,714][61552] Updated weights for policy 0, policy_version 5220 (0.0007) [2023-10-14 17:47:32,092][61552] Updated weights for policy 0, policy_version 5230 (0.0008) [2023-10-14 17:47:32,294][61585] Updated weights for policy 1, policy_version 5220 (0.0008) [2023-10-14 17:47:32,454][61552] Updated weights for policy 0, policy_version 5240 (0.0008) [2023-10-14 17:47:32,666][61585] Updated weights for policy 1, policy_version 5230 (0.0007) [2023-10-14 17:47:33,037][61585] Updated weights for policy 1, policy_version 5240 (0.0010) [2023-10-14 17:47:33,343][60425] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 10747904. Throughput: 0: 1670.0, 1: 1663.0. Samples: 2685210. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:47:33,344][60425] Avg episode reward: [(0, '24.070'), (1, '26.870')] [2023-10-14 17:47:33,344][61172] Saving new best policy, reward=24.070! [2023-10-14 17:47:36,642][61552] Updated weights for policy 0, policy_version 5250 (0.0008) [2023-10-14 17:47:37,013][61552] Updated weights for policy 0, policy_version 5260 (0.0009) [2023-10-14 17:47:37,094][61585] Updated weights for policy 1, policy_version 5250 (0.0011) [2023-10-14 17:47:37,382][61552] Updated weights for policy 0, policy_version 5270 (0.0009) [2023-10-14 17:47:37,458][61585] Updated weights for policy 1, policy_version 5260 (0.0007) [2023-10-14 17:47:37,750][61552] Updated weights for policy 0, policy_version 5280 (0.0009) [2023-10-14 17:47:37,831][61585] Updated weights for policy 1, policy_version 5270 (0.0009) [2023-10-14 17:47:38,198][61585] Updated weights for policy 1, policy_version 5280 (0.0007) [2023-10-14 17:47:38,343][60425] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 10813440. Throughput: 0: 1665.1, 1: 1661.5. Samples: 2705478. Policy #0 lag: (min: 5.0, avg: 5.1, max: 11.0) [2023-10-14 17:47:38,344][60425] Avg episode reward: [(0, '24.210'), (1, '28.910')] [2023-10-14 17:47:38,345][61172] Saving new best policy, reward=24.210! [2023-10-14 17:47:41,875][61552] Updated weights for policy 0, policy_version 5290 (0.0010) [2023-10-14 17:47:42,212][61585] Updated weights for policy 1, policy_version 5290 (0.0009) [2023-10-14 17:47:42,254][61552] Updated weights for policy 0, policy_version 5300 (0.0008) [2023-10-14 17:47:42,571][61585] Updated weights for policy 1, policy_version 5300 (0.0008) [2023-10-14 17:47:42,621][61552] Updated weights for policy 0, policy_version 5310 (0.0008) [2023-10-14 17:47:42,941][61585] Updated weights for policy 1, policy_version 5310 (0.0007) [2023-10-14 17:47:43,344][60425] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 13329.3). Total num frames: 10878976. Throughput: 0: 1652.0, 1: 1650.7. Samples: 2724008. Policy #0 lag: (min: 5.0, avg: 5.1, max: 11.0) [2023-10-14 17:47:43,345][60425] Avg episode reward: [(0, '23.700'), (1, '31.710')] [2023-10-14 17:47:43,355][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000005312_5439488.pth... [2023-10-14 17:47:43,356][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000005312_5439488.pth... [2023-10-14 17:47:43,392][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000003744_3833856.pth [2023-10-14 17:47:43,394][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000003744_3833856.pth [2023-10-14 17:47:43,398][61248] Saving new best policy, reward=31.710! [2023-10-14 17:47:46,538][61552] Updated weights for policy 0, policy_version 5320 (0.0008) [2023-10-14 17:47:46,906][61552] Updated weights for policy 0, policy_version 5330 (0.0008) [2023-10-14 17:47:46,956][61585] Updated weights for policy 1, policy_version 5320 (0.0008) [2023-10-14 17:47:47,273][61552] Updated weights for policy 0, policy_version 5340 (0.0007) [2023-10-14 17:47:47,318][61585] Updated weights for policy 1, policy_version 5330 (0.0009) [2023-10-14 17:47:47,686][61585] Updated weights for policy 1, policy_version 5340 (0.0007) [2023-10-14 17:47:48,343][60425] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 10944512. Throughput: 0: 1670.3, 1: 1668.4. Samples: 2735516. Policy #0 lag: (min: 19.0, avg: 20.7, max: 47.0) [2023-10-14 17:47:48,344][60425] Avg episode reward: [(0, '22.730'), (1, '30.560')] [2023-10-14 17:47:51,360][61552] Updated weights for policy 0, policy_version 5350 (0.0008) [2023-10-14 17:47:51,725][61552] Updated weights for policy 0, policy_version 5360 (0.0007) [2023-10-14 17:47:51,901][61585] Updated weights for policy 1, policy_version 5350 (0.0010) [2023-10-14 17:47:52,091][61552] Updated weights for policy 0, policy_version 5370 (0.0009) [2023-10-14 17:47:52,269][61585] Updated weights for policy 1, policy_version 5360 (0.0008) [2023-10-14 17:47:52,633][61585] Updated weights for policy 1, policy_version 5370 (0.0009) [2023-10-14 17:47:53,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 11010048. Throughput: 0: 1659.8, 1: 1669.8. Samples: 2755302. Policy #0 lag: (min: 19.0, avg: 20.7, max: 47.0) [2023-10-14 17:47:53,344][60425] Avg episode reward: [(0, '23.080'), (1, '31.640')] [2023-10-14 17:47:56,290][61552] Updated weights for policy 0, policy_version 5380 (0.0008) [2023-10-14 17:47:56,663][61552] Updated weights for policy 0, policy_version 5390 (0.0008) [2023-10-14 17:47:56,768][61585] Updated weights for policy 1, policy_version 5380 (0.0008) [2023-10-14 17:47:57,031][61552] Updated weights for policy 0, policy_version 5400 (0.0009) [2023-10-14 17:47:57,141][61585] Updated weights for policy 1, policy_version 5390 (0.0010) [2023-10-14 17:47:57,517][61585] Updated weights for policy 1, policy_version 5400 (0.0009) [2023-10-14 17:47:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 11075584. Throughput: 0: 1668.0, 1: 1658.2. Samples: 2773982. Policy #0 lag: (min: 3.0, avg: 10.2, max: 35.0) [2023-10-14 17:47:58,344][60425] Avg episode reward: [(0, '22.280'), (1, '32.620')] [2023-10-14 17:47:58,351][61248] Saving new best policy, reward=32.620! [2023-10-14 17:48:01,049][61552] Updated weights for policy 0, policy_version 5410 (0.0007) [2023-10-14 17:48:01,415][61552] Updated weights for policy 0, policy_version 5420 (0.0007) [2023-10-14 17:48:01,682][61585] Updated weights for policy 1, policy_version 5410 (0.0009) [2023-10-14 17:48:01,776][61552] Updated weights for policy 0, policy_version 5430 (0.0008) [2023-10-14 17:48:02,043][61585] Updated weights for policy 1, policy_version 5420 (0.0009) [2023-10-14 17:48:02,150][61552] Updated weights for policy 0, policy_version 5440 (0.0007) [2023-10-14 17:48:02,415][61585] Updated weights for policy 1, policy_version 5430 (0.0009) [2023-10-14 17:48:02,782][61585] Updated weights for policy 1, policy_version 5440 (0.0007) [2023-10-14 17:48:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 11141120. Throughput: 0: 1672.5, 1: 1669.0. Samples: 2785564. Policy #0 lag: (min: 3.0, avg: 10.2, max: 35.0) [2023-10-14 17:48:03,344][60425] Avg episode reward: [(0, '22.880'), (1, '30.820')] [2023-10-14 17:48:05,998][61552] Updated weights for policy 0, policy_version 5450 (0.0007) [2023-10-14 17:48:06,372][61552] Updated weights for policy 0, policy_version 5460 (0.0010) [2023-10-14 17:48:06,738][61552] Updated weights for policy 0, policy_version 5470 (0.0008) [2023-10-14 17:48:06,880][61585] Updated weights for policy 1, policy_version 5450 (0.0007) [2023-10-14 17:48:07,235][61585] Updated weights for policy 1, policy_version 5460 (0.0008) [2023-10-14 17:48:07,612][61585] Updated weights for policy 1, policy_version 5470 (0.0009) [2023-10-14 17:48:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 11206656. Throughput: 0: 1654.4, 1: 1664.1. Samples: 2804994. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:48:08,344][60425] Avg episode reward: [(0, '21.750'), (1, '30.430')] [2023-10-14 17:48:10,792][61552] Updated weights for policy 0, policy_version 5480 (0.0008) [2023-10-14 17:48:11,166][61552] Updated weights for policy 0, policy_version 5490 (0.0009) [2023-10-14 17:48:11,537][61552] Updated weights for policy 0, policy_version 5500 (0.0009) [2023-10-14 17:48:11,798][61585] Updated weights for policy 1, policy_version 5480 (0.0007) [2023-10-14 17:48:12,175][61585] Updated weights for policy 1, policy_version 5490 (0.0007) [2023-10-14 17:48:12,542][61585] Updated weights for policy 1, policy_version 5500 (0.0007) [2023-10-14 17:48:13,344][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 11272192. Throughput: 0: 1671.7, 1: 1654.2. Samples: 2824232. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:48:13,345][60425] Avg episode reward: [(0, '22.300'), (1, '31.890')] [2023-10-14 17:48:15,597][61552] Updated weights for policy 0, policy_version 5510 (0.0009) [2023-10-14 17:48:15,963][61552] Updated weights for policy 0, policy_version 5520 (0.0010) [2023-10-14 17:48:16,330][61552] Updated weights for policy 0, policy_version 5530 (0.0009) [2023-10-14 17:48:16,620][61585] Updated weights for policy 1, policy_version 5510 (0.0007) [2023-10-14 17:48:17,000][61585] Updated weights for policy 1, policy_version 5520 (0.0007) [2023-10-14 17:48:17,363][61585] Updated weights for policy 1, policy_version 5530 (0.0011) [2023-10-14 17:48:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 11337728. Throughput: 0: 1671.0, 1: 1668.7. Samples: 2835498. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:48:18,344][60425] Avg episode reward: [(0, '24.360'), (1, '32.720')] [2023-10-14 17:48:18,344][61248] Saving new best policy, reward=32.720! [2023-10-14 17:48:18,344][61172] Saving new best policy, reward=24.360! [2023-10-14 17:48:20,407][61552] Updated weights for policy 0, policy_version 5540 (0.0008) [2023-10-14 17:48:20,782][61552] Updated weights for policy 0, policy_version 5550 (0.0007) [2023-10-14 17:48:21,147][61552] Updated weights for policy 0, policy_version 5560 (0.0008) [2023-10-14 17:48:21,422][61585] Updated weights for policy 1, policy_version 5540 (0.0011) [2023-10-14 17:48:21,789][61585] Updated weights for policy 1, policy_version 5550 (0.0009) [2023-10-14 17:48:22,157][61585] Updated weights for policy 1, policy_version 5560 (0.0007) [2023-10-14 17:48:23,343][60425] Fps is (10 sec: 13107.7, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 11403264. Throughput: 0: 1656.2, 1: 1656.8. Samples: 2854566. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:48:23,344][60425] Avg episode reward: [(0, '23.990'), (1, '31.250')] [2023-10-14 17:48:25,232][61552] Updated weights for policy 0, policy_version 5570 (0.0009) [2023-10-14 17:48:25,599][61552] Updated weights for policy 0, policy_version 5580 (0.0007) [2023-10-14 17:48:25,975][61552] Updated weights for policy 0, policy_version 5590 (0.0008) [2023-10-14 17:48:26,258][61585] Updated weights for policy 1, policy_version 5570 (0.0007) [2023-10-14 17:48:26,342][61552] Updated weights for policy 0, policy_version 5600 (0.0008) [2023-10-14 17:48:26,626][61585] Updated weights for policy 1, policy_version 5580 (0.0010) [2023-10-14 17:48:27,000][61585] Updated weights for policy 1, policy_version 5590 (0.0007) [2023-10-14 17:48:27,367][61585] Updated weights for policy 1, policy_version 5600 (0.0009) [2023-10-14 17:48:28,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 11468800. Throughput: 0: 1685.9, 1: 1658.6. Samples: 2874510. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:48:28,344][60425] Avg episode reward: [(0, '23.790'), (1, '30.950')] [2023-10-14 17:48:30,518][61552] Updated weights for policy 0, policy_version 5610 (0.0008) [2023-10-14 17:48:30,905][61552] Updated weights for policy 0, policy_version 5620 (0.0010) [2023-10-14 17:48:31,277][61552] Updated weights for policy 0, policy_version 5630 (0.0010) [2023-10-14 17:48:31,451][61585] Updated weights for policy 1, policy_version 5610 (0.0009) [2023-10-14 17:48:31,819][61585] Updated weights for policy 1, policy_version 5620 (0.0009) [2023-10-14 17:48:32,185][61585] Updated weights for policy 1, policy_version 5630 (0.0008) [2023-10-14 17:48:33,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 11534336. Throughput: 0: 1668.8, 1: 1663.8. Samples: 2885482. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:48:33,344][60425] Avg episode reward: [(0, '24.680'), (1, '28.860')] [2023-10-14 17:48:33,344][61172] Saving new best policy, reward=24.680! [2023-10-14 17:48:35,337][61552] Updated weights for policy 0, policy_version 5640 (0.0010) [2023-10-14 17:48:35,709][61552] Updated weights for policy 0, policy_version 5650 (0.0007) [2023-10-14 17:48:36,083][61552] Updated weights for policy 0, policy_version 5660 (0.0008) [2023-10-14 17:48:36,184][61585] Updated weights for policy 1, policy_version 5640 (0.0008) [2023-10-14 17:48:36,548][61585] Updated weights for policy 1, policy_version 5650 (0.0008) [2023-10-14 17:48:36,911][61585] Updated weights for policy 1, policy_version 5660 (0.0010) [2023-10-14 17:48:38,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 11599872. Throughput: 0: 1663.3, 1: 1653.3. Samples: 2904550. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:48:38,344][60425] Avg episode reward: [(0, '23.680'), (1, '29.660')] [2023-10-14 17:48:40,157][61552] Updated weights for policy 0, policy_version 5670 (0.0007) [2023-10-14 17:48:40,522][61552] Updated weights for policy 0, policy_version 5680 (0.0007) [2023-10-14 17:48:40,895][61552] Updated weights for policy 0, policy_version 5690 (0.0008) [2023-10-14 17:48:41,017][61585] Updated weights for policy 1, policy_version 5670 (0.0008) [2023-10-14 17:48:41,378][61585] Updated weights for policy 1, policy_version 5680 (0.0009) [2023-10-14 17:48:41,744][61585] Updated weights for policy 1, policy_version 5690 (0.0011) [2023-10-14 17:48:43,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 11665408. Throughput: 0: 1682.4, 1: 1666.4. Samples: 2924678. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:48:43,344][60425] Avg episode reward: [(0, '22.170'), (1, '29.300')] [2023-10-14 17:48:44,733][61552] Updated weights for policy 0, policy_version 5700 (0.0008) [2023-10-14 17:48:45,103][61552] Updated weights for policy 0, policy_version 5710 (0.0009) [2023-10-14 17:48:45,469][61552] Updated weights for policy 0, policy_version 5720 (0.0008) [2023-10-14 17:48:45,898][61585] Updated weights for policy 1, policy_version 5700 (0.0008) [2023-10-14 17:48:46,263][61585] Updated weights for policy 1, policy_version 5710 (0.0009) [2023-10-14 17:48:46,626][61585] Updated weights for policy 1, policy_version 5720 (0.0009) [2023-10-14 17:48:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 11730944. Throughput: 0: 1658.7, 1: 1670.4. Samples: 2935372. Policy #0 lag: (min: 1.0, avg: 3.2, max: 28.0) [2023-10-14 17:48:48,344][60425] Avg episode reward: [(0, '23.490'), (1, '29.060')] [2023-10-14 17:48:49,830][61552] Updated weights for policy 0, policy_version 5730 (0.0008) [2023-10-14 17:48:50,208][61552] Updated weights for policy 0, policy_version 5740 (0.0009) [2023-10-14 17:48:50,572][61552] Updated weights for policy 0, policy_version 5750 (0.0008) [2023-10-14 17:48:50,882][61585] Updated weights for policy 1, policy_version 5730 (0.0009) [2023-10-14 17:48:50,935][61552] Updated weights for policy 0, policy_version 5760 (0.0009) [2023-10-14 17:48:51,252][61585] Updated weights for policy 1, policy_version 5740 (0.0008) [2023-10-14 17:48:51,623][61585] Updated weights for policy 1, policy_version 5750 (0.0009) [2023-10-14 17:48:51,993][61585] Updated weights for policy 1, policy_version 5760 (0.0007) [2023-10-14 17:48:53,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 11796480. Throughput: 0: 1673.6, 1: 1653.5. Samples: 2954712. Policy #0 lag: (min: 1.0, avg: 3.2, max: 28.0) [2023-10-14 17:48:53,344][60425] Avg episode reward: [(0, '24.470'), (1, '30.150')] [2023-10-14 17:48:55,153][61552] Updated weights for policy 0, policy_version 5770 (0.0010) [2023-10-14 17:48:55,522][61552] Updated weights for policy 0, policy_version 5780 (0.0009) [2023-10-14 17:48:55,894][61552] Updated weights for policy 0, policy_version 5790 (0.0008) [2023-10-14 17:48:56,143][61585] Updated weights for policy 1, policy_version 5770 (0.0009) [2023-10-14 17:48:56,514][61585] Updated weights for policy 1, policy_version 5780 (0.0007) [2023-10-14 17:48:56,881][61585] Updated weights for policy 1, policy_version 5790 (0.0007) [2023-10-14 17:48:58,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 11862016. Throughput: 0: 1680.0, 1: 1666.3. Samples: 2974812. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:48:58,344][60425] Avg episode reward: [(0, '25.580'), (1, '32.260')] [2023-10-14 17:48:58,355][61172] Saving new best policy, reward=25.580! [2023-10-14 17:48:59,994][61552] Updated weights for policy 0, policy_version 5800 (0.0009) [2023-10-14 17:49:00,366][61552] Updated weights for policy 0, policy_version 5810 (0.0008) [2023-10-14 17:49:00,741][61552] Updated weights for policy 0, policy_version 5820 (0.0008) [2023-10-14 17:49:00,837][61585] Updated weights for policy 1, policy_version 5800 (0.0009) [2023-10-14 17:49:01,209][61585] Updated weights for policy 1, policy_version 5810 (0.0010) [2023-10-14 17:49:01,575][61585] Updated weights for policy 1, policy_version 5820 (0.0008) [2023-10-14 17:49:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 11927552. Throughput: 0: 1657.3, 1: 1660.2. Samples: 2984788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:49:03,344][60425] Avg episode reward: [(0, '24.030'), (1, '33.210')] [2023-10-14 17:49:03,345][61248] Saving new best policy, reward=33.210! [2023-10-14 17:49:04,763][61552] Updated weights for policy 0, policy_version 5830 (0.0008) [2023-10-14 17:49:05,143][61552] Updated weights for policy 0, policy_version 5840 (0.0008) [2023-10-14 17:49:05,512][61552] Updated weights for policy 0, policy_version 5850 (0.0007) [2023-10-14 17:49:05,757][61585] Updated weights for policy 1, policy_version 5830 (0.0009) [2023-10-14 17:49:06,115][61585] Updated weights for policy 1, policy_version 5840 (0.0008) [2023-10-14 17:49:06,475][61585] Updated weights for policy 1, policy_version 5850 (0.0007) [2023-10-14 17:49:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 11993088. Throughput: 0: 1673.7, 1: 1649.5. Samples: 3004110. Policy #0 lag: (min: 17.0, avg: 29.5, max: 49.0) [2023-10-14 17:49:08,344][60425] Avg episode reward: [(0, '25.320'), (1, '33.410')] [2023-10-14 17:49:08,346][61248] Saving new best policy, reward=33.410! [2023-10-14 17:49:09,616][61552] Updated weights for policy 0, policy_version 5860 (0.0008) [2023-10-14 17:49:09,980][61552] Updated weights for policy 0, policy_version 5870 (0.0009) [2023-10-14 17:49:10,353][61552] Updated weights for policy 0, policy_version 5880 (0.0008) [2023-10-14 17:49:10,408][61585] Updated weights for policy 1, policy_version 5860 (0.0009) [2023-10-14 17:49:10,782][61585] Updated weights for policy 1, policy_version 5870 (0.0008) [2023-10-14 17:49:11,147][61585] Updated weights for policy 1, policy_version 5880 (0.0008) [2023-10-14 17:49:13,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 12058624. Throughput: 0: 1669.2, 1: 1667.5. Samples: 3024658. Policy #0 lag: (min: 17.0, avg: 29.5, max: 49.0) [2023-10-14 17:49:13,344][60425] Avg episode reward: [(0, '24.390'), (1, '35.300')] [2023-10-14 17:49:13,354][61248] Saving new best policy, reward=35.300! [2023-10-14 17:49:14,439][61552] Updated weights for policy 0, policy_version 5890 (0.0008) [2023-10-14 17:49:14,813][61552] Updated weights for policy 0, policy_version 5900 (0.0008) [2023-10-14 17:49:15,175][61585] Updated weights for policy 1, policy_version 5890 (0.0009) [2023-10-14 17:49:15,180][61552] Updated weights for policy 0, policy_version 5910 (0.0009) [2023-10-14 17:49:15,541][61585] Updated weights for policy 1, policy_version 5900 (0.0007) [2023-10-14 17:49:15,547][61552] Updated weights for policy 0, policy_version 5920 (0.0008) [2023-10-14 17:49:15,916][61585] Updated weights for policy 1, policy_version 5910 (0.0008) [2023-10-14 17:49:16,283][61585] Updated weights for policy 1, policy_version 5920 (0.0011) [2023-10-14 17:49:18,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 12124160. Throughput: 0: 1652.3, 1: 1655.9. Samples: 3034352. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-14 17:49:18,344][60425] Avg episode reward: [(0, '24.920'), (1, '34.920')] [2023-10-14 17:49:19,532][61552] Updated weights for policy 0, policy_version 5930 (0.0010) [2023-10-14 17:49:19,911][61552] Updated weights for policy 0, policy_version 5940 (0.0009) [2023-10-14 17:49:20,278][61552] Updated weights for policy 0, policy_version 5950 (0.0009) [2023-10-14 17:49:20,400][61585] Updated weights for policy 1, policy_version 5930 (0.0008) [2023-10-14 17:49:20,771][61585] Updated weights for policy 1, policy_version 5940 (0.0008) [2023-10-14 17:49:21,142][61585] Updated weights for policy 1, policy_version 5950 (0.0007) [2023-10-14 17:49:23,343][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 12189696. Throughput: 0: 1671.4, 1: 1657.8. Samples: 3054366. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-14 17:49:23,345][60425] Avg episode reward: [(0, '25.820'), (1, '33.820')] [2023-10-14 17:49:23,346][61172] Saving new best policy, reward=25.820! [2023-10-14 17:49:24,269][61552] Updated weights for policy 0, policy_version 5960 (0.0009) [2023-10-14 17:49:24,632][61552] Updated weights for policy 0, policy_version 5970 (0.0010) [2023-10-14 17:49:25,009][61552] Updated weights for policy 0, policy_version 5980 (0.0010) [2023-10-14 17:49:25,218][61585] Updated weights for policy 1, policy_version 5960 (0.0008) [2023-10-14 17:49:25,589][61585] Updated weights for policy 1, policy_version 5970 (0.0007) [2023-10-14 17:49:25,961][61585] Updated weights for policy 1, policy_version 5980 (0.0008) [2023-10-14 17:49:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 12255232. Throughput: 0: 1666.6, 1: 1673.0. Samples: 3074960. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:49:28,344][60425] Avg episode reward: [(0, '26.090'), (1, '34.650')] [2023-10-14 17:49:28,354][61172] Saving new best policy, reward=26.090! [2023-10-14 17:49:29,249][61552] Updated weights for policy 0, policy_version 5990 (0.0009) [2023-10-14 17:49:29,632][61552] Updated weights for policy 0, policy_version 6000 (0.0008) [2023-10-14 17:49:29,989][61552] Updated weights for policy 0, policy_version 6010 (0.0009) [2023-10-14 17:49:30,161][61585] Updated weights for policy 1, policy_version 5990 (0.0009) [2023-10-14 17:49:30,543][61585] Updated weights for policy 1, policy_version 6000 (0.0008) [2023-10-14 17:49:30,910][61585] Updated weights for policy 1, policy_version 6010 (0.0008) [2023-10-14 17:49:33,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 12320768. Throughput: 0: 1658.2, 1: 1655.1. Samples: 3084468. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:49:33,344][60425] Avg episode reward: [(0, '25.460'), (1, '33.350')] [2023-10-14 17:49:34,158][61552] Updated weights for policy 0, policy_version 6020 (0.0008) [2023-10-14 17:49:34,532][61552] Updated weights for policy 0, policy_version 6030 (0.0008) [2023-10-14 17:49:34,902][61552] Updated weights for policy 0, policy_version 6040 (0.0009) [2023-10-14 17:49:35,196][61585] Updated weights for policy 1, policy_version 6020 (0.0009) [2023-10-14 17:49:35,552][61585] Updated weights for policy 1, policy_version 6030 (0.0009) [2023-10-14 17:49:35,935][61585] Updated weights for policy 1, policy_version 6040 (0.0008) [2023-10-14 17:49:38,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 12386304. Throughput: 0: 1666.6, 1: 1663.1. Samples: 3104548. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:49:38,344][60425] Avg episode reward: [(0, '23.320'), (1, '33.530')] [2023-10-14 17:49:39,062][61552] Updated weights for policy 0, policy_version 6050 (0.0009) [2023-10-14 17:49:39,432][61552] Updated weights for policy 0, policy_version 6060 (0.0007) [2023-10-14 17:49:39,811][61552] Updated weights for policy 0, policy_version 6070 (0.0009) [2023-10-14 17:49:40,185][61552] Updated weights for policy 0, policy_version 6080 (0.0008) [2023-10-14 17:49:40,213][61585] Updated weights for policy 1, policy_version 6050 (0.0008) [2023-10-14 17:49:40,589][61585] Updated weights for policy 1, policy_version 6060 (0.0008) [2023-10-14 17:49:40,962][61585] Updated weights for policy 1, policy_version 6070 (0.0009) [2023-10-14 17:49:41,323][61585] Updated weights for policy 1, policy_version 6080 (0.0011) [2023-10-14 17:49:43,344][60425] Fps is (10 sec: 13106.7, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 12451840. Throughput: 0: 1666.9, 1: 1669.2. Samples: 3124938. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:49:43,344][60425] Avg episode reward: [(0, '24.300'), (1, '32.670')] [2023-10-14 17:49:43,351][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000006080_6225920.pth... [2023-10-14 17:49:43,351][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000006080_6225920.pth... [2023-10-14 17:49:43,381][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000004544_4653056.pth [2023-10-14 17:49:43,392][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000004512_4620288.pth [2023-10-14 17:49:44,272][61552] Updated weights for policy 0, policy_version 6090 (0.0008) [2023-10-14 17:49:44,654][61552] Updated weights for policy 0, policy_version 6100 (0.0010) [2023-10-14 17:49:45,028][61552] Updated weights for policy 0, policy_version 6110 (0.0008) [2023-10-14 17:49:45,673][61585] Updated weights for policy 1, policy_version 6090 (0.0007) [2023-10-14 17:49:46,045][61585] Updated weights for policy 1, policy_version 6100 (0.0008) [2023-10-14 17:49:46,403][61585] Updated weights for policy 1, policy_version 6110 (0.0007) [2023-10-14 17:49:48,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 12517376. Throughput: 0: 1666.1, 1: 1663.5. Samples: 3134620. Policy #0 lag: (min: 27.0, avg: 51.2, max: 56.0) [2023-10-14 17:49:48,344][60425] Avg episode reward: [(0, '23.270'), (1, '34.490')] [2023-10-14 17:49:49,202][61552] Updated weights for policy 0, policy_version 6120 (0.0008) [2023-10-14 17:49:49,567][61552] Updated weights for policy 0, policy_version 6130 (0.0010) [2023-10-14 17:49:49,939][61552] Updated weights for policy 0, policy_version 6140 (0.0008) [2023-10-14 17:49:50,230][61585] Updated weights for policy 1, policy_version 6120 (0.0008) [2023-10-14 17:49:50,593][61585] Updated weights for policy 1, policy_version 6130 (0.0007) [2023-10-14 17:49:50,970][61585] Updated weights for policy 1, policy_version 6140 (0.0008) [2023-10-14 17:49:53,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 12582912. Throughput: 0: 1669.6, 1: 1666.7. Samples: 3154244. Policy #0 lag: (min: 27.0, avg: 51.2, max: 56.0) [2023-10-14 17:49:53,345][60425] Avg episode reward: [(0, '24.630'), (1, '37.550')] [2023-10-14 17:49:53,346][61248] Saving new best policy, reward=37.550! [2023-10-14 17:49:54,075][61552] Updated weights for policy 0, policy_version 6150 (0.0009) [2023-10-14 17:49:54,445][61552] Updated weights for policy 0, policy_version 6160 (0.0011) [2023-10-14 17:49:54,808][61552] Updated weights for policy 0, policy_version 6170 (0.0010) [2023-10-14 17:49:55,130][61585] Updated weights for policy 1, policy_version 6150 (0.0008) [2023-10-14 17:49:55,501][61585] Updated weights for policy 1, policy_version 6160 (0.0009) [2023-10-14 17:49:55,875][61585] Updated weights for policy 1, policy_version 6170 (0.0011) [2023-10-14 17:49:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 12648448. Throughput: 0: 1670.5, 1: 1668.4. Samples: 3174908. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:49:58,344][60425] Avg episode reward: [(0, '24.780'), (1, '37.920')] [2023-10-14 17:49:58,350][61248] Saving new best policy, reward=37.920! [2023-10-14 17:49:58,841][61552] Updated weights for policy 0, policy_version 6180 (0.0009) [2023-10-14 17:49:59,204][61552] Updated weights for policy 0, policy_version 6190 (0.0009) [2023-10-14 17:49:59,582][61552] Updated weights for policy 0, policy_version 6200 (0.0010) [2023-10-14 17:49:59,810][61585] Updated weights for policy 1, policy_version 6180 (0.0007) [2023-10-14 17:50:00,182][61585] Updated weights for policy 1, policy_version 6190 (0.0009) [2023-10-14 17:50:00,549][61585] Updated weights for policy 1, policy_version 6200 (0.0008) [2023-10-14 17:50:03,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 12713984. Throughput: 0: 1673.0, 1: 1658.2. Samples: 3184258. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:50:03,344][60425] Avg episode reward: [(0, '25.040'), (1, '37.660')] [2023-10-14 17:50:03,673][61552] Updated weights for policy 0, policy_version 6210 (0.0008) [2023-10-14 17:50:04,045][61552] Updated weights for policy 0, policy_version 6220 (0.0007) [2023-10-14 17:50:04,423][61552] Updated weights for policy 0, policy_version 6230 (0.0008) [2023-10-14 17:50:04,686][61585] Updated weights for policy 1, policy_version 6210 (0.0008) [2023-10-14 17:50:04,791][61552] Updated weights for policy 0, policy_version 6240 (0.0008) [2023-10-14 17:50:05,043][61585] Updated weights for policy 1, policy_version 6220 (0.0011) [2023-10-14 17:50:05,417][61585] Updated weights for policy 1, policy_version 6230 (0.0009) [2023-10-14 17:50:05,776][61585] Updated weights for policy 1, policy_version 6240 (0.0010) [2023-10-14 17:50:08,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 12779520. Throughput: 0: 1669.5, 1: 1668.5. Samples: 3204574. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-14 17:50:08,344][60425] Avg episode reward: [(0, '24.990'), (1, '38.410')] [2023-10-14 17:50:08,345][61248] Saving new best policy, reward=38.410! [2023-10-14 17:50:08,823][61552] Updated weights for policy 0, policy_version 6250 (0.0009) [2023-10-14 17:50:09,194][61552] Updated weights for policy 0, policy_version 6260 (0.0008) [2023-10-14 17:50:09,570][61552] Updated weights for policy 0, policy_version 6270 (0.0007) [2023-10-14 17:50:09,763][61585] Updated weights for policy 1, policy_version 6250 (0.0009) [2023-10-14 17:50:10,135][61585] Updated weights for policy 1, policy_version 6260 (0.0009) [2023-10-14 17:50:10,495][61585] Updated weights for policy 1, policy_version 6270 (0.0008) [2023-10-14 17:50:13,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 12845056. Throughput: 0: 1674.9, 1: 1667.2. Samples: 3225356. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-14 17:50:13,344][60425] Avg episode reward: [(0, '22.690'), (1, '35.070')] [2023-10-14 17:50:13,771][61552] Updated weights for policy 0, policy_version 6280 (0.0008) [2023-10-14 17:50:14,136][61552] Updated weights for policy 0, policy_version 6290 (0.0009) [2023-10-14 17:50:14,515][61552] Updated weights for policy 0, policy_version 6300 (0.0009) [2023-10-14 17:50:14,627][61585] Updated weights for policy 1, policy_version 6280 (0.0007) [2023-10-14 17:50:15,004][61585] Updated weights for policy 1, policy_version 6290 (0.0009) [2023-10-14 17:50:15,366][61585] Updated weights for policy 1, policy_version 6300 (0.0009) [2023-10-14 17:50:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 12910592. Throughput: 0: 1677.8, 1: 1653.9. Samples: 3234392. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:50:18,344][60425] Avg episode reward: [(0, '25.700'), (1, '35.070')] [2023-10-14 17:50:18,681][61552] Updated weights for policy 0, policy_version 6310 (0.0008) [2023-10-14 17:50:19,043][61552] Updated weights for policy 0, policy_version 6320 (0.0009) [2023-10-14 17:50:19,410][61552] Updated weights for policy 0, policy_version 6330 (0.0009) [2023-10-14 17:50:19,586][61585] Updated weights for policy 1, policy_version 6310 (0.0008) [2023-10-14 17:50:19,942][61585] Updated weights for policy 1, policy_version 6320 (0.0008) [2023-10-14 17:50:20,309][61585] Updated weights for policy 1, policy_version 6330 (0.0009) [2023-10-14 17:50:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 12976128. Throughput: 0: 1673.7, 1: 1665.7. Samples: 3254824. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:50:23,344][60425] Avg episode reward: [(0, '23.700'), (1, '33.550')] [2023-10-14 17:50:23,483][61552] Updated weights for policy 0, policy_version 6340 (0.0008) [2023-10-14 17:50:23,848][61552] Updated weights for policy 0, policy_version 6350 (0.0009) [2023-10-14 17:50:24,219][61552] Updated weights for policy 0, policy_version 6360 (0.0010) [2023-10-14 17:50:24,496][61585] Updated weights for policy 1, policy_version 6340 (0.0008) [2023-10-14 17:50:24,861][61585] Updated weights for policy 1, policy_version 6350 (0.0010) [2023-10-14 17:50:25,233][61585] Updated weights for policy 1, policy_version 6360 (0.0008) [2023-10-14 17:50:28,201][61552] Updated weights for policy 0, policy_version 6370 (0.0009) [2023-10-14 17:50:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 13041664. Throughput: 0: 1672.2, 1: 1665.8. Samples: 3275148. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-14 17:50:28,344][60425] Avg episode reward: [(0, '25.770'), (1, '34.680')] [2023-10-14 17:50:28,576][61552] Updated weights for policy 0, policy_version 6380 (0.0009) [2023-10-14 17:50:28,957][61552] Updated weights for policy 0, policy_version 6390 (0.0009) [2023-10-14 17:50:29,325][61552] Updated weights for policy 0, policy_version 6400 (0.0009) [2023-10-14 17:50:29,438][61585] Updated weights for policy 1, policy_version 6370 (0.0009) [2023-10-14 17:50:29,808][61585] Updated weights for policy 1, policy_version 6380 (0.0009) [2023-10-14 17:50:30,182][61585] Updated weights for policy 1, policy_version 6390 (0.0008) [2023-10-14 17:50:30,555][61585] Updated weights for policy 1, policy_version 6400 (0.0009) [2023-10-14 17:50:33,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 13107200. Throughput: 0: 1674.6, 1: 1649.4. Samples: 3284202. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-14 17:50:33,344][60425] Avg episode reward: [(0, '25.200'), (1, '34.370')] [2023-10-14 17:50:33,498][61552] Updated weights for policy 0, policy_version 6410 (0.0010) [2023-10-14 17:50:33,867][61552] Updated weights for policy 0, policy_version 6420 (0.0009) [2023-10-14 17:50:34,232][61552] Updated weights for policy 0, policy_version 6430 (0.0010) [2023-10-14 17:50:34,802][61585] Updated weights for policy 1, policy_version 6410 (0.0007) [2023-10-14 17:50:35,169][61585] Updated weights for policy 1, policy_version 6420 (0.0008) [2023-10-14 17:50:35,533][61585] Updated weights for policy 1, policy_version 6430 (0.0008) [2023-10-14 17:50:38,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 13172736. Throughput: 0: 1672.0, 1: 1662.7. Samples: 3304306. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:50:38,344][60425] Avg episode reward: [(0, '26.280'), (1, '36.020')] [2023-10-14 17:50:38,466][61552] Updated weights for policy 0, policy_version 6440 (0.0008) [2023-10-14 17:50:38,831][61552] Updated weights for policy 0, policy_version 6450 (0.0007) [2023-10-14 17:50:39,203][61552] Updated weights for policy 0, policy_version 6460 (0.0011) [2023-10-14 17:50:39,348][61172] Saving new best policy, reward=26.280! [2023-10-14 17:50:39,594][61585] Updated weights for policy 1, policy_version 6440 (0.0010) [2023-10-14 17:50:39,956][61585] Updated weights for policy 1, policy_version 6450 (0.0010) [2023-10-14 17:50:40,320][61585] Updated weights for policy 1, policy_version 6460 (0.0009) [2023-10-14 17:50:43,275][61552] Updated weights for policy 0, policy_version 6470 (0.0008) [2023-10-14 17:50:43,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 13238272. Throughput: 0: 1668.9, 1: 1657.4. Samples: 3324592. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:50:43,344][60425] Avg episode reward: [(0, '24.820'), (1, '37.670')] [2023-10-14 17:50:43,652][61552] Updated weights for policy 0, policy_version 6480 (0.0008) [2023-10-14 17:50:44,022][61552] Updated weights for policy 0, policy_version 6490 (0.0008) [2023-10-14 17:50:44,445][61585] Updated weights for policy 1, policy_version 6470 (0.0009) [2023-10-14 17:50:44,812][61585] Updated weights for policy 1, policy_version 6480 (0.0008) [2023-10-14 17:50:45,187][61585] Updated weights for policy 1, policy_version 6490 (0.0008) [2023-10-14 17:50:48,326][61552] Updated weights for policy 0, policy_version 6500 (0.0008) [2023-10-14 17:50:48,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 13303808. Throughput: 0: 1663.5, 1: 1651.0. Samples: 3333412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:50:48,344][60425] Avg episode reward: [(0, '23.350'), (1, '38.550')] [2023-10-14 17:50:48,345][61248] Saving new best policy, reward=38.550! [2023-10-14 17:50:48,692][61552] Updated weights for policy 0, policy_version 6510 (0.0008) [2023-10-14 17:50:49,063][61552] Updated weights for policy 0, policy_version 6520 (0.0008) [2023-10-14 17:50:49,404][61585] Updated weights for policy 1, policy_version 6500 (0.0008) [2023-10-14 17:50:49,777][61585] Updated weights for policy 1, policy_version 6510 (0.0007) [2023-10-14 17:50:50,136][61585] Updated weights for policy 1, policy_version 6520 (0.0008) [2023-10-14 17:50:53,326][61552] Updated weights for policy 0, policy_version 6530 (0.0007) [2023-10-14 17:50:53,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 13369344. Throughput: 0: 1660.5, 1: 1652.7. Samples: 3353668. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:50:53,345][60425] Avg episode reward: [(0, '23.180'), (1, '38.100')] [2023-10-14 17:50:53,707][61552] Updated weights for policy 0, policy_version 6540 (0.0007) [2023-10-14 17:50:54,077][61552] Updated weights for policy 0, policy_version 6550 (0.0007) [2023-10-14 17:50:54,253][61585] Updated weights for policy 1, policy_version 6530 (0.0007) [2023-10-14 17:50:54,448][61552] Updated weights for policy 0, policy_version 6560 (0.0008) [2023-10-14 17:50:54,615][61585] Updated weights for policy 1, policy_version 6540 (0.0009) [2023-10-14 17:50:54,972][61585] Updated weights for policy 1, policy_version 6550 (0.0010) [2023-10-14 17:50:55,337][61585] Updated weights for policy 1, policy_version 6560 (0.0012) [2023-10-14 17:50:58,339][61552] Updated weights for policy 0, policy_version 6570 (0.0009) [2023-10-14 17:50:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 13434880. Throughput: 0: 1662.0, 1: 1652.8. Samples: 3374522. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:50:58,344][60425] Avg episode reward: [(0, '24.330'), (1, '36.770')] [2023-10-14 17:50:58,718][61552] Updated weights for policy 0, policy_version 6580 (0.0007) [2023-10-14 17:50:59,085][61552] Updated weights for policy 0, policy_version 6590 (0.0007) [2023-10-14 17:50:59,384][61585] Updated weights for policy 1, policy_version 6570 (0.0008) [2023-10-14 17:50:59,756][61585] Updated weights for policy 1, policy_version 6580 (0.0011) [2023-10-14 17:51:00,121][61585] Updated weights for policy 1, policy_version 6590 (0.0011) [2023-10-14 17:51:03,094][61552] Updated weights for policy 0, policy_version 6600 (0.0010) [2023-10-14 17:51:03,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 13500416. Throughput: 0: 1660.8, 1: 1657.6. Samples: 3383724. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:51:03,344][60425] Avg episode reward: [(0, '25.460'), (1, '37.330')] [2023-10-14 17:51:03,467][61552] Updated weights for policy 0, policy_version 6610 (0.0009) [2023-10-14 17:51:03,832][61552] Updated weights for policy 0, policy_version 6620 (0.0009) [2023-10-14 17:51:04,155][61585] Updated weights for policy 1, policy_version 6600 (0.0009) [2023-10-14 17:51:04,518][61585] Updated weights for policy 1, policy_version 6610 (0.0008) [2023-10-14 17:51:04,895][61585] Updated weights for policy 1, policy_version 6620 (0.0010) [2023-10-14 17:51:07,932][61552] Updated weights for policy 0, policy_version 6630 (0.0009) [2023-10-14 17:51:08,310][61552] Updated weights for policy 0, policy_version 6640 (0.0008) [2023-10-14 17:51:08,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 13565952. Throughput: 0: 1662.0, 1: 1657.4. Samples: 3404200. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-14 17:51:08,344][60425] Avg episode reward: [(0, '26.840'), (1, '35.750')] [2023-10-14 17:51:08,682][61552] Updated weights for policy 0, policy_version 6650 (0.0009) [2023-10-14 17:51:08,907][61172] Saving new best policy, reward=26.840! [2023-10-14 17:51:09,042][61585] Updated weights for policy 1, policy_version 6630 (0.0011) [2023-10-14 17:51:09,407][61585] Updated weights for policy 1, policy_version 6640 (0.0011) [2023-10-14 17:51:09,776][61585] Updated weights for policy 1, policy_version 6650 (0.0009) [2023-10-14 17:51:12,813][61552] Updated weights for policy 0, policy_version 6660 (0.0010) [2023-10-14 17:51:13,185][61552] Updated weights for policy 0, policy_version 6670 (0.0008) [2023-10-14 17:51:13,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 13631488. Throughput: 0: 1661.1, 1: 1662.4. Samples: 3424704. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-14 17:51:13,344][60425] Avg episode reward: [(0, '26.470'), (1, '35.250')] [2023-10-14 17:51:13,560][61552] Updated weights for policy 0, policy_version 6680 (0.0008) [2023-10-14 17:51:13,844][61585] Updated weights for policy 1, policy_version 6660 (0.0008) [2023-10-14 17:51:14,205][61585] Updated weights for policy 1, policy_version 6670 (0.0009) [2023-10-14 17:51:14,579][61585] Updated weights for policy 1, policy_version 6680 (0.0008) [2023-10-14 17:51:17,479][61552] Updated weights for policy 0, policy_version 6690 (0.0009) [2023-10-14 17:51:17,846][61552] Updated weights for policy 0, policy_version 6700 (0.0007) [2023-10-14 17:51:18,223][61552] Updated weights for policy 0, policy_version 6710 (0.0008) [2023-10-14 17:51:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 13697024. Throughput: 0: 1660.3, 1: 1664.3. Samples: 3433808. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-14 17:51:18,344][60425] Avg episode reward: [(0, '25.790'), (1, '34.370')] [2023-10-14 17:51:18,589][61552] Updated weights for policy 0, policy_version 6720 (0.0007) [2023-10-14 17:51:18,704][61585] Updated weights for policy 1, policy_version 6690 (0.0009) [2023-10-14 17:51:19,073][61585] Updated weights for policy 1, policy_version 6700 (0.0008) [2023-10-14 17:51:19,440][61585] Updated weights for policy 1, policy_version 6710 (0.0009) [2023-10-14 17:51:19,799][61585] Updated weights for policy 1, policy_version 6720 (0.0007) [2023-10-14 17:51:22,597][61552] Updated weights for policy 0, policy_version 6730 (0.0010) [2023-10-14 17:51:22,959][61552] Updated weights for policy 0, policy_version 6740 (0.0009) [2023-10-14 17:51:23,334][61552] Updated weights for policy 0, policy_version 6750 (0.0008) [2023-10-14 17:51:23,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 13762560. Throughput: 0: 1664.7, 1: 1668.4. Samples: 3454296. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-14 17:51:23,344][60425] Avg episode reward: [(0, '26.110'), (1, '36.710')] [2023-10-14 17:51:24,108][61585] Updated weights for policy 1, policy_version 6730 (0.0009) [2023-10-14 17:51:24,486][61585] Updated weights for policy 1, policy_version 6740 (0.0007) [2023-10-14 17:51:24,849][61585] Updated weights for policy 1, policy_version 6750 (0.0007) [2023-10-14 17:51:27,513][61552] Updated weights for policy 0, policy_version 6760 (0.0008) [2023-10-14 17:51:27,889][61552] Updated weights for policy 0, policy_version 6770 (0.0009) [2023-10-14 17:51:28,252][61552] Updated weights for policy 0, policy_version 6780 (0.0007) [2023-10-14 17:51:28,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 13828096. Throughput: 0: 1659.7, 1: 1667.6. Samples: 3474322. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-14 17:51:28,344][60425] Avg episode reward: [(0, '24.830'), (1, '36.910')] [2023-10-14 17:51:28,846][61585] Updated weights for policy 1, policy_version 6760 (0.0008) [2023-10-14 17:51:29,217][61585] Updated weights for policy 1, policy_version 6770 (0.0007) [2023-10-14 17:51:29,581][61585] Updated weights for policy 1, policy_version 6780 (0.0009) [2023-10-14 17:51:32,216][61552] Updated weights for policy 0, policy_version 6790 (0.0009) [2023-10-14 17:51:32,590][61552] Updated weights for policy 0, policy_version 6800 (0.0010) [2023-10-14 17:51:32,953][61552] Updated weights for policy 0, policy_version 6810 (0.0009) [2023-10-14 17:51:33,343][60425] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 13926400. Throughput: 0: 1679.6, 1: 1668.4. Samples: 3484068. Policy #0 lag: (min: 31.0, avg: 41.5, max: 63.0) [2023-10-14 17:51:33,344][60425] Avg episode reward: [(0, '26.860'), (1, '38.890')] [2023-10-14 17:51:33,345][61172] Saving new best policy, reward=26.860! [2023-10-14 17:51:33,345][61248] Saving new best policy, reward=38.890! [2023-10-14 17:51:33,713][61585] Updated weights for policy 1, policy_version 6790 (0.0007) [2023-10-14 17:51:34,081][61585] Updated weights for policy 1, policy_version 6800 (0.0007) [2023-10-14 17:51:34,448][61585] Updated weights for policy 1, policy_version 6810 (0.0009) [2023-10-14 17:51:37,057][61552] Updated weights for policy 0, policy_version 6820 (0.0008) [2023-10-14 17:51:37,435][61552] Updated weights for policy 0, policy_version 6830 (0.0008) [2023-10-14 17:51:37,796][61552] Updated weights for policy 0, policy_version 6840 (0.0008) [2023-10-14 17:51:38,343][60425] Fps is (10 sec: 16384.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 13991936. Throughput: 0: 1682.5, 1: 1675.3. Samples: 3504768. Policy #0 lag: (min: 31.0, avg: 41.5, max: 63.0) [2023-10-14 17:51:38,344][60425] Avg episode reward: [(0, '25.620'), (1, '40.890')] [2023-10-14 17:51:38,523][61585] Updated weights for policy 1, policy_version 6820 (0.0010) [2023-10-14 17:51:38,885][61585] Updated weights for policy 1, policy_version 6830 (0.0009) [2023-10-14 17:51:39,245][61585] Updated weights for policy 1, policy_version 6840 (0.0008) [2023-10-14 17:51:39,536][61248] Saving new best policy, reward=40.890! [2023-10-14 17:51:41,988][61552] Updated weights for policy 0, policy_version 6850 (0.0009) [2023-10-14 17:51:42,350][61552] Updated weights for policy 0, policy_version 6860 (0.0010) [2023-10-14 17:51:42,720][61552] Updated weights for policy 0, policy_version 6870 (0.0010) [2023-10-14 17:51:43,088][61552] Updated weights for policy 0, policy_version 6880 (0.0007) [2023-10-14 17:51:43,200][61585] Updated weights for policy 1, policy_version 6850 (0.0011) [2023-10-14 17:51:43,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 14057472. Throughput: 0: 1657.5, 1: 1677.6. Samples: 3524600. Policy #0 lag: (min: 10.0, avg: 34.6, max: 40.0) [2023-10-14 17:51:43,344][60425] Avg episode reward: [(0, '25.530'), (1, '39.570')] [2023-10-14 17:51:43,354][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000006880_7045120.pth... [2023-10-14 17:51:43,393][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000005312_5439488.pth [2023-10-14 17:51:43,565][61585] Updated weights for policy 1, policy_version 6860 (0.0008) [2023-10-14 17:51:43,930][61585] Updated weights for policy 1, policy_version 6870 (0.0009) [2023-10-14 17:51:44,301][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000006880_7045120.pth... [2023-10-14 17:51:44,301][61585] Updated weights for policy 1, policy_version 6880 (0.0009) [2023-10-14 17:51:44,338][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000005312_5439488.pth [2023-10-14 17:51:47,170][61552] Updated weights for policy 0, policy_version 6890 (0.0008) [2023-10-14 17:51:47,539][61552] Updated weights for policy 0, policy_version 6900 (0.0007) [2023-10-14 17:51:47,902][61552] Updated weights for policy 0, policy_version 6910 (0.0007) [2023-10-14 17:51:48,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 14123008. Throughput: 0: 1675.1, 1: 1675.0. Samples: 3534478. Policy #0 lag: (min: 10.0, avg: 34.6, max: 40.0) [2023-10-14 17:51:48,344][60425] Avg episode reward: [(0, '28.080'), (1, '37.950')] [2023-10-14 17:51:48,345][61172] Saving new best policy, reward=28.080! [2023-10-14 17:51:48,579][61585] Updated weights for policy 1, policy_version 6890 (0.0009) [2023-10-14 17:51:48,957][61585] Updated weights for policy 1, policy_version 6900 (0.0009) [2023-10-14 17:51:49,327][61585] Updated weights for policy 1, policy_version 6910 (0.0008) [2023-10-14 17:51:52,131][61552] Updated weights for policy 0, policy_version 6920 (0.0009) [2023-10-14 17:51:52,494][61552] Updated weights for policy 0, policy_version 6930 (0.0010) [2023-10-14 17:51:52,876][61552] Updated weights for policy 0, policy_version 6940 (0.0009) [2023-10-14 17:51:53,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 14188544. Throughput: 0: 1670.9, 1: 1670.4. Samples: 3554558. Policy #0 lag: (min: 11.0, avg: 18.8, max: 43.0) [2023-10-14 17:51:53,344][60425] Avg episode reward: [(0, '25.750'), (1, '38.110')] [2023-10-14 17:51:53,394][61585] Updated weights for policy 1, policy_version 6920 (0.0010) [2023-10-14 17:51:53,769][61585] Updated weights for policy 1, policy_version 6930 (0.0007) [2023-10-14 17:51:54,136][61585] Updated weights for policy 1, policy_version 6940 (0.0010) [2023-10-14 17:51:56,795][61552] Updated weights for policy 0, policy_version 6950 (0.0009) [2023-10-14 17:51:57,168][61552] Updated weights for policy 0, policy_version 6960 (0.0008) [2023-10-14 17:51:57,548][61552] Updated weights for policy 0, policy_version 6970 (0.0009) [2023-10-14 17:51:58,139][61585] Updated weights for policy 1, policy_version 6950 (0.0009) [2023-10-14 17:51:58,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 14254080. Throughput: 0: 1648.9, 1: 1672.2. Samples: 3574154. Policy #0 lag: (min: 11.0, avg: 18.8, max: 43.0) [2023-10-14 17:51:58,344][60425] Avg episode reward: [(0, '24.810'), (1, '37.380')] [2023-10-14 17:51:58,509][61585] Updated weights for policy 1, policy_version 6960 (0.0008) [2023-10-14 17:51:58,877][61585] Updated weights for policy 1, policy_version 6970 (0.0009) [2023-10-14 17:52:01,651][61552] Updated weights for policy 0, policy_version 6980 (0.0009) [2023-10-14 17:52:02,014][61552] Updated weights for policy 0, policy_version 6990 (0.0009) [2023-10-14 17:52:02,384][61552] Updated weights for policy 0, policy_version 7000 (0.0009) [2023-10-14 17:52:02,933][61585] Updated weights for policy 1, policy_version 6980 (0.0008) [2023-10-14 17:52:03,292][61585] Updated weights for policy 1, policy_version 6990 (0.0008) [2023-10-14 17:52:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 14319616. Throughput: 0: 1676.2, 1: 1670.1. Samples: 3584392. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:52:03,344][60425] Avg episode reward: [(0, '25.190'), (1, '36.790')] [2023-10-14 17:52:03,668][61585] Updated weights for policy 1, policy_version 7000 (0.0008) [2023-10-14 17:52:06,410][61552] Updated weights for policy 0, policy_version 7010 (0.0007) [2023-10-14 17:52:06,776][61552] Updated weights for policy 0, policy_version 7020 (0.0010) [2023-10-14 17:52:07,155][61552] Updated weights for policy 0, policy_version 7030 (0.0009) [2023-10-14 17:52:07,522][61552] Updated weights for policy 0, policy_version 7040 (0.0009) [2023-10-14 17:52:07,716][61585] Updated weights for policy 1, policy_version 7010 (0.0010) [2023-10-14 17:52:08,101][61585] Updated weights for policy 1, policy_version 7020 (0.0008) [2023-10-14 17:52:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 14385152. Throughput: 0: 1666.7, 1: 1671.2. Samples: 3604500. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:52:08,344][60425] Avg episode reward: [(0, '25.280'), (1, '37.390')] [2023-10-14 17:52:08,461][61585] Updated weights for policy 1, policy_version 7030 (0.0010) [2023-10-14 17:52:08,838][61585] Updated weights for policy 1, policy_version 7040 (0.0008) [2023-10-14 17:52:11,743][61552] Updated weights for policy 0, policy_version 7050 (0.0010) [2023-10-14 17:52:12,115][61552] Updated weights for policy 0, policy_version 7060 (0.0010) [2023-10-14 17:52:12,485][61552] Updated weights for policy 0, policy_version 7070 (0.0008) [2023-10-14 17:52:12,953][61585] Updated weights for policy 1, policy_version 7050 (0.0009) [2023-10-14 17:52:13,330][61585] Updated weights for policy 1, policy_version 7060 (0.0009) [2023-10-14 17:52:13,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 14450688. Throughput: 0: 1655.8, 1: 1665.9. Samples: 3623800. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:52:13,344][60425] Avg episode reward: [(0, '24.440'), (1, '39.200')] [2023-10-14 17:52:13,701][61585] Updated weights for policy 1, policy_version 7070 (0.0007) [2023-10-14 17:52:16,760][61552] Updated weights for policy 0, policy_version 7080 (0.0008) [2023-10-14 17:52:17,136][61552] Updated weights for policy 0, policy_version 7090 (0.0010) [2023-10-14 17:52:17,495][61552] Updated weights for policy 0, policy_version 7100 (0.0010) [2023-10-14 17:52:17,778][61585] Updated weights for policy 1, policy_version 7080 (0.0008) [2023-10-14 17:52:18,145][61585] Updated weights for policy 1, policy_version 7090 (0.0008) [2023-10-14 17:52:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 14516224. Throughput: 0: 1669.2, 1: 1667.9. Samples: 3634236. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:52:18,344][60425] Avg episode reward: [(0, '25.930'), (1, '36.900')] [2023-10-14 17:52:18,517][61585] Updated weights for policy 1, policy_version 7100 (0.0009) [2023-10-14 17:52:21,411][61552] Updated weights for policy 0, policy_version 7110 (0.0008) [2023-10-14 17:52:21,780][61552] Updated weights for policy 0, policy_version 7120 (0.0007) [2023-10-14 17:52:22,157][61552] Updated weights for policy 0, policy_version 7130 (0.0008) [2023-10-14 17:52:22,892][61585] Updated weights for policy 1, policy_version 7110 (0.0009) [2023-10-14 17:52:23,255][61585] Updated weights for policy 1, policy_version 7120 (0.0010) [2023-10-14 17:52:23,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 14581760. Throughput: 0: 1657.1, 1: 1660.1. Samples: 3654044. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-14 17:52:23,344][60425] Avg episode reward: [(0, '26.740'), (1, '41.720')] [2023-10-14 17:52:23,623][61585] Updated weights for policy 1, policy_version 7130 (0.0011) [2023-10-14 17:52:23,840][61248] Saving new best policy, reward=41.720! [2023-10-14 17:52:26,206][61552] Updated weights for policy 0, policy_version 7140 (0.0008) [2023-10-14 17:52:26,573][61552] Updated weights for policy 0, policy_version 7150 (0.0010) [2023-10-14 17:52:26,955][61552] Updated weights for policy 0, policy_version 7160 (0.0010) [2023-10-14 17:52:27,783][61585] Updated weights for policy 1, policy_version 7140 (0.0010) [2023-10-14 17:52:28,154][61585] Updated weights for policy 1, policy_version 7150 (0.0009) [2023-10-14 17:52:28,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 14647296. Throughput: 0: 1660.0, 1: 1649.9. Samples: 3673542. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-14 17:52:28,344][60425] Avg episode reward: [(0, '24.040'), (1, '41.550')] [2023-10-14 17:52:28,512][61585] Updated weights for policy 1, policy_version 7160 (0.0008) [2023-10-14 17:52:30,991][61552] Updated weights for policy 0, policy_version 7170 (0.0011) [2023-10-14 17:52:31,369][61552] Updated weights for policy 0, policy_version 7180 (0.0010) [2023-10-14 17:52:31,737][61552] Updated weights for policy 0, policy_version 7190 (0.0008) [2023-10-14 17:52:32,107][61552] Updated weights for policy 0, policy_version 7200 (0.0008) [2023-10-14 17:52:32,609][61585] Updated weights for policy 1, policy_version 7170 (0.0009) [2023-10-14 17:52:32,983][61585] Updated weights for policy 1, policy_version 7180 (0.0009) [2023-10-14 17:52:33,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 14712832. Throughput: 0: 1671.8, 1: 1652.9. Samples: 3684088. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:52:33,344][60425] Avg episode reward: [(0, '23.890'), (1, '42.510')] [2023-10-14 17:52:33,356][61585] Updated weights for policy 1, policy_version 7190 (0.0008) [2023-10-14 17:52:33,720][61248] Saving new best policy, reward=42.510! [2023-10-14 17:52:33,724][61585] Updated weights for policy 1, policy_version 7200 (0.0007) [2023-10-14 17:52:36,249][61552] Updated weights for policy 0, policy_version 7210 (0.0011) [2023-10-14 17:52:36,618][61552] Updated weights for policy 0, policy_version 7220 (0.0009) [2023-10-14 17:52:36,993][61552] Updated weights for policy 0, policy_version 7230 (0.0008) [2023-10-14 17:52:37,866][61585] Updated weights for policy 1, policy_version 7210 (0.0011) [2023-10-14 17:52:38,236][61585] Updated weights for policy 1, policy_version 7220 (0.0008) [2023-10-14 17:52:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 14778368. Throughput: 0: 1654.4, 1: 1660.3. Samples: 3703716. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:52:38,344][60425] Avg episode reward: [(0, '24.970'), (1, '44.950')] [2023-10-14 17:52:38,603][61585] Updated weights for policy 1, policy_version 7230 (0.0009) [2023-10-14 17:52:38,674][61248] Saving new best policy, reward=44.950! [2023-10-14 17:52:41,111][61552] Updated weights for policy 0, policy_version 7240 (0.0008) [2023-10-14 17:52:41,486][61552] Updated weights for policy 0, policy_version 7250 (0.0008) [2023-10-14 17:52:41,863][61552] Updated weights for policy 0, policy_version 7260 (0.0009) [2023-10-14 17:52:42,766][61585] Updated weights for policy 1, policy_version 7240 (0.0007) [2023-10-14 17:52:43,137][61585] Updated weights for policy 1, policy_version 7250 (0.0007) [2023-10-14 17:52:43,344][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 14843904. Throughput: 0: 1668.7, 1: 1648.6. Samples: 3723430. Policy #0 lag: (min: 9.0, avg: 17.5, max: 41.0) [2023-10-14 17:52:43,345][60425] Avg episode reward: [(0, '24.850'), (1, '46.140')] [2023-10-14 17:52:43,498][61585] Updated weights for policy 1, policy_version 7260 (0.0007) [2023-10-14 17:52:43,645][61248] Saving new best policy, reward=46.140! [2023-10-14 17:52:46,025][61552] Updated weights for policy 0, policy_version 7270 (0.0009) [2023-10-14 17:52:46,402][61552] Updated weights for policy 0, policy_version 7280 (0.0007) [2023-10-14 17:52:46,767][61552] Updated weights for policy 0, policy_version 7290 (0.0009) [2023-10-14 17:52:47,676][61585] Updated weights for policy 1, policy_version 7270 (0.0009) [2023-10-14 17:52:48,045][61585] Updated weights for policy 1, policy_version 7280 (0.0008) [2023-10-14 17:52:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 14909440. Throughput: 0: 1665.8, 1: 1655.8. Samples: 3733864. Policy #0 lag: (min: 9.0, avg: 17.5, max: 41.0) [2023-10-14 17:52:48,344][60425] Avg episode reward: [(0, '24.140'), (1, '45.940')] [2023-10-14 17:52:48,421][61585] Updated weights for policy 1, policy_version 7290 (0.0008) [2023-10-14 17:52:50,984][61552] Updated weights for policy 0, policy_version 7300 (0.0010) [2023-10-14 17:52:51,340][61552] Updated weights for policy 0, policy_version 7310 (0.0009) [2023-10-14 17:52:51,711][61552] Updated weights for policy 0, policy_version 7320 (0.0010) [2023-10-14 17:52:52,622][61585] Updated weights for policy 1, policy_version 7300 (0.0009) [2023-10-14 17:52:52,990][61585] Updated weights for policy 1, policy_version 7310 (0.0011) [2023-10-14 17:52:53,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 14974976. Throughput: 0: 1652.8, 1: 1653.1. Samples: 3753262. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-14 17:52:53,344][60425] Avg episode reward: [(0, '25.040'), (1, '43.930')] [2023-10-14 17:52:53,355][61585] Updated weights for policy 1, policy_version 7320 (0.0009) [2023-10-14 17:52:55,778][61552] Updated weights for policy 0, policy_version 7330 (0.0010) [2023-10-14 17:52:56,154][61552] Updated weights for policy 0, policy_version 7340 (0.0007) [2023-10-14 17:52:56,525][61552] Updated weights for policy 0, policy_version 7350 (0.0007) [2023-10-14 17:52:56,896][61552] Updated weights for policy 0, policy_version 7360 (0.0007) [2023-10-14 17:52:57,550][61585] Updated weights for policy 1, policy_version 7330 (0.0008) [2023-10-14 17:52:57,967][61585] Updated weights for policy 1, policy_version 7340 (0.0009) [2023-10-14 17:52:58,337][61585] Updated weights for policy 1, policy_version 7350 (0.0008) [2023-10-14 17:52:58,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 15040512. Throughput: 0: 1672.2, 1: 1651.9. Samples: 3773384. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-14 17:52:58,345][60425] Avg episode reward: [(0, '26.590'), (1, '44.370')] [2023-10-14 17:52:58,708][61585] Updated weights for policy 1, policy_version 7360 (0.0008) [2023-10-14 17:53:01,093][61552] Updated weights for policy 0, policy_version 7370 (0.0007) [2023-10-14 17:53:01,462][61552] Updated weights for policy 0, policy_version 7380 (0.0009) [2023-10-14 17:53:01,828][61552] Updated weights for policy 0, policy_version 7390 (0.0011) [2023-10-14 17:53:02,816][61585] Updated weights for policy 1, policy_version 7370 (0.0007) [2023-10-14 17:53:03,200][61585] Updated weights for policy 1, policy_version 7380 (0.0007) [2023-10-14 17:53:03,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 15106048. Throughput: 0: 1671.2, 1: 1653.9. Samples: 3783868. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-14 17:53:03,344][60425] Avg episode reward: [(0, '24.740'), (1, '42.310')] [2023-10-14 17:53:03,570][61585] Updated weights for policy 1, policy_version 7390 (0.0008) [2023-10-14 17:53:05,775][61552] Updated weights for policy 0, policy_version 7400 (0.0009) [2023-10-14 17:53:06,149][61552] Updated weights for policy 0, policy_version 7410 (0.0008) [2023-10-14 17:53:06,518][61552] Updated weights for policy 0, policy_version 7420 (0.0008) [2023-10-14 17:53:07,764][61585] Updated weights for policy 1, policy_version 7400 (0.0008) [2023-10-14 17:53:08,128][61585] Updated weights for policy 1, policy_version 7410 (0.0007) [2023-10-14 17:53:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 15171584. Throughput: 0: 1655.9, 1: 1656.3. Samples: 3803090. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-14 17:53:08,344][60425] Avg episode reward: [(0, '25.280'), (1, '43.690')] [2023-10-14 17:53:08,501][61585] Updated weights for policy 1, policy_version 7420 (0.0009) [2023-10-14 17:53:10,592][61552] Updated weights for policy 0, policy_version 7430 (0.0009) [2023-10-14 17:53:10,968][61552] Updated weights for policy 0, policy_version 7440 (0.0008) [2023-10-14 17:53:11,342][61552] Updated weights for policy 0, policy_version 7450 (0.0008) [2023-10-14 17:53:12,452][61585] Updated weights for policy 1, policy_version 7430 (0.0009) [2023-10-14 17:53:12,809][61585] Updated weights for policy 1, policy_version 7440 (0.0011) [2023-10-14 17:53:13,180][61585] Updated weights for policy 1, policy_version 7450 (0.0007) [2023-10-14 17:53:13,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 15237120. Throughput: 0: 1679.9, 1: 1652.7. Samples: 3823508. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-14 17:53:13,344][60425] Avg episode reward: [(0, '24.850'), (1, '45.360')] [2023-10-14 17:53:15,054][61552] Updated weights for policy 0, policy_version 7460 (0.0008) [2023-10-14 17:53:15,421][61552] Updated weights for policy 0, policy_version 7470 (0.0007) [2023-10-14 17:53:15,779][61552] Updated weights for policy 0, policy_version 7480 (0.0009) [2023-10-14 17:53:17,437][61585] Updated weights for policy 1, policy_version 7460 (0.0008) [2023-10-14 17:53:17,809][61585] Updated weights for policy 1, policy_version 7470 (0.0008) [2023-10-14 17:53:18,170][61585] Updated weights for policy 1, policy_version 7480 (0.0008) [2023-10-14 17:53:18,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 15302656. Throughput: 0: 1665.1, 1: 1660.4. Samples: 3833734. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-14 17:53:18,344][60425] Avg episode reward: [(0, '27.440'), (1, '43.570')] [2023-10-14 17:53:19,784][61552] Updated weights for policy 0, policy_version 7490 (0.0008) [2023-10-14 17:53:20,157][61552] Updated weights for policy 0, policy_version 7500 (0.0007) [2023-10-14 17:53:20,535][61552] Updated weights for policy 0, policy_version 7510 (0.0008) [2023-10-14 17:53:20,905][61552] Updated weights for policy 0, policy_version 7520 (0.0009) [2023-10-14 17:53:22,348][61585] Updated weights for policy 1, policy_version 7490 (0.0008) [2023-10-14 17:53:22,716][61585] Updated weights for policy 1, policy_version 7500 (0.0008) [2023-10-14 17:53:23,092][61585] Updated weights for policy 1, policy_version 7510 (0.0007) [2023-10-14 17:53:23,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 15368192. Throughput: 0: 1677.8, 1: 1654.7. Samples: 3853678. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:53:23,344][60425] Avg episode reward: [(0, '26.910'), (1, '43.880')] [2023-10-14 17:53:23,451][61585] Updated weights for policy 1, policy_version 7520 (0.0007) [2023-10-14 17:53:25,042][61552] Updated weights for policy 0, policy_version 7530 (0.0008) [2023-10-14 17:53:25,403][61552] Updated weights for policy 0, policy_version 7540 (0.0008) [2023-10-14 17:53:25,775][61552] Updated weights for policy 0, policy_version 7550 (0.0009) [2023-10-14 17:53:27,464][61585] Updated weights for policy 1, policy_version 7530 (0.0007) [2023-10-14 17:53:27,837][61585] Updated weights for policy 1, policy_version 7540 (0.0009) [2023-10-14 17:53:28,195][61585] Updated weights for policy 1, policy_version 7550 (0.0010) [2023-10-14 17:53:28,343][60425] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 15466496. Throughput: 0: 1686.4, 1: 1652.5. Samples: 3873680. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:53:28,344][60425] Avg episode reward: [(0, '26.410'), (1, '42.190')] [2023-10-14 17:53:30,194][61552] Updated weights for policy 0, policy_version 7560 (0.0008) [2023-10-14 17:53:30,563][61552] Updated weights for policy 0, policy_version 7570 (0.0009) [2023-10-14 17:53:30,942][61552] Updated weights for policy 0, policy_version 7580 (0.0007) [2023-10-14 17:53:32,311][61585] Updated weights for policy 1, policy_version 7560 (0.0007) [2023-10-14 17:53:32,682][61585] Updated weights for policy 1, policy_version 7570 (0.0007) [2023-10-14 17:53:33,050][61585] Updated weights for policy 1, policy_version 7580 (0.0009) [2023-10-14 17:53:33,343][60425] Fps is (10 sec: 16384.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 15532032. Throughput: 0: 1670.6, 1: 1658.9. Samples: 3883692. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:53:33,344][60425] Avg episode reward: [(0, '27.290'), (1, '42.370')] [2023-10-14 17:53:34,923][61552] Updated weights for policy 0, policy_version 7590 (0.0008) [2023-10-14 17:53:35,296][61552] Updated weights for policy 0, policy_version 7600 (0.0007) [2023-10-14 17:53:35,665][61552] Updated weights for policy 0, policy_version 7610 (0.0008) [2023-10-14 17:53:36,981][61585] Updated weights for policy 1, policy_version 7590 (0.0007) [2023-10-14 17:53:37,343][61585] Updated weights for policy 1, policy_version 7600 (0.0009) [2023-10-14 17:53:37,706][61585] Updated weights for policy 1, policy_version 7610 (0.0009) [2023-10-14 17:53:38,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 15597568. Throughput: 0: 1680.6, 1: 1666.3. Samples: 3903872. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:53:38,344][60425] Avg episode reward: [(0, '24.560'), (1, '42.370')] [2023-10-14 17:53:39,739][61552] Updated weights for policy 0, policy_version 7620 (0.0008) [2023-10-14 17:53:40,120][61552] Updated weights for policy 0, policy_version 7630 (0.0009) [2023-10-14 17:53:40,498][61552] Updated weights for policy 0, policy_version 7640 (0.0010) [2023-10-14 17:53:41,900][61585] Updated weights for policy 1, policy_version 7620 (0.0008) [2023-10-14 17:53:42,274][61585] Updated weights for policy 1, policy_version 7630 (0.0007) [2023-10-14 17:53:42,635][61585] Updated weights for policy 1, policy_version 7640 (0.0007) [2023-10-14 17:53:43,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 15663104. Throughput: 0: 1684.2, 1: 1651.4. Samples: 3923486. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:53:43,344][60425] Avg episode reward: [(0, '27.100'), (1, '42.520')] [2023-10-14 17:53:43,353][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000007648_7831552.pth... [2023-10-14 17:53:43,354][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000007648_7831552.pth... [2023-10-14 17:53:43,390][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000006080_6225920.pth [2023-10-14 17:53:43,394][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000006080_6225920.pth [2023-10-14 17:53:43,396][61172] Saving a milestone ./train_atari/atari_roadrunner_APPO/checkpoint_p0/milestones/checkpoint_000007648_7831552.pth [2023-10-14 17:53:43,399][61248] Saving a milestone ./train_atari/atari_roadrunner_APPO/checkpoint_p1/milestones/checkpoint_000007648_7831552.pth [2023-10-14 17:53:44,397][61552] Updated weights for policy 0, policy_version 7650 (0.0007) [2023-10-14 17:53:44,768][61552] Updated weights for policy 0, policy_version 7660 (0.0007) [2023-10-14 17:53:45,135][61552] Updated weights for policy 0, policy_version 7670 (0.0007) [2023-10-14 17:53:45,505][61552] Updated weights for policy 0, policy_version 7680 (0.0007) [2023-10-14 17:53:46,932][61585] Updated weights for policy 1, policy_version 7650 (0.0008) [2023-10-14 17:53:47,342][61585] Updated weights for policy 1, policy_version 7660 (0.0010) [2023-10-14 17:53:47,698][61585] Updated weights for policy 1, policy_version 7670 (0.0010) [2023-10-14 17:53:48,068][61585] Updated weights for policy 1, policy_version 7680 (0.0009) [2023-10-14 17:53:48,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 15728640. Throughput: 0: 1658.0, 1: 1668.7. Samples: 3933570. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:53:48,344][60425] Avg episode reward: [(0, '27.130'), (1, '43.600')] [2023-10-14 17:53:49,530][61552] Updated weights for policy 0, policy_version 7690 (0.0009) [2023-10-14 17:53:49,907][61552] Updated weights for policy 0, policy_version 7700 (0.0009) [2023-10-14 17:53:50,271][61552] Updated weights for policy 0, policy_version 7710 (0.0007) [2023-10-14 17:53:52,185][61585] Updated weights for policy 1, policy_version 7690 (0.0007) [2023-10-14 17:53:52,547][61585] Updated weights for policy 1, policy_version 7700 (0.0009) [2023-10-14 17:53:52,915][61585] Updated weights for policy 1, policy_version 7710 (0.0007) [2023-10-14 17:53:53,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 15794176. Throughput: 0: 1689.8, 1: 1661.9. Samples: 3953916. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-14 17:53:53,344][60425] Avg episode reward: [(0, '27.430'), (1, '41.550')] [2023-10-14 17:53:54,483][61552] Updated weights for policy 0, policy_version 7720 (0.0008) [2023-10-14 17:53:54,849][61552] Updated weights for policy 0, policy_version 7730 (0.0009) [2023-10-14 17:53:55,219][61552] Updated weights for policy 0, policy_version 7740 (0.0010) [2023-10-14 17:53:56,847][61585] Updated weights for policy 1, policy_version 7720 (0.0009) [2023-10-14 17:53:57,221][61585] Updated weights for policy 1, policy_version 7730 (0.0007) [2023-10-14 17:53:57,590][61585] Updated weights for policy 1, policy_version 7740 (0.0007) [2023-10-14 17:53:58,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 15859712. Throughput: 0: 1682.6, 1: 1646.2. Samples: 3973304. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-14 17:53:58,345][60425] Avg episode reward: [(0, '27.440'), (1, '41.290')] [2023-10-14 17:53:59,111][61552] Updated weights for policy 0, policy_version 7750 (0.0008) [2023-10-14 17:53:59,480][61552] Updated weights for policy 0, policy_version 7760 (0.0007) [2023-10-14 17:53:59,851][61552] Updated weights for policy 0, policy_version 7770 (0.0010) [2023-10-14 17:54:01,741][61585] Updated weights for policy 1, policy_version 7750 (0.0007) [2023-10-14 17:54:02,110][61585] Updated weights for policy 1, policy_version 7760 (0.0007) [2023-10-14 17:54:02,474][61585] Updated weights for policy 1, policy_version 7770 (0.0010) [2023-10-14 17:54:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 15925248. Throughput: 0: 1667.4, 1: 1662.7. Samples: 3983586. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:54:03,344][60425] Avg episode reward: [(0, '27.850'), (1, '40.270')] [2023-10-14 17:54:03,856][61552] Updated weights for policy 0, policy_version 7780 (0.0007) [2023-10-14 17:54:04,223][61552] Updated weights for policy 0, policy_version 7790 (0.0007) [2023-10-14 17:54:04,602][61552] Updated weights for policy 0, policy_version 7800 (0.0008) [2023-10-14 17:54:06,585][61585] Updated weights for policy 1, policy_version 7780 (0.0007) [2023-10-14 17:54:06,947][61585] Updated weights for policy 1, policy_version 7790 (0.0008) [2023-10-14 17:54:07,315][61585] Updated weights for policy 1, policy_version 7800 (0.0011) [2023-10-14 17:54:08,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 15990784. Throughput: 0: 1684.4, 1: 1657.8. Samples: 4004080. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:54:08,344][60425] Avg episode reward: [(0, '27.180'), (1, '41.790')] [2023-10-14 17:54:08,706][61552] Updated weights for policy 0, policy_version 7810 (0.0009) [2023-10-14 17:54:09,075][61552] Updated weights for policy 0, policy_version 7820 (0.0008) [2023-10-14 17:54:09,449][61552] Updated weights for policy 0, policy_version 7830 (0.0010) [2023-10-14 17:54:09,817][61552] Updated weights for policy 0, policy_version 7840 (0.0008) [2023-10-14 17:54:11,423][61585] Updated weights for policy 1, policy_version 7810 (0.0010) [2023-10-14 17:54:11,794][61585] Updated weights for policy 1, policy_version 7820 (0.0007) [2023-10-14 17:54:12,165][61585] Updated weights for policy 1, policy_version 7830 (0.0008) [2023-10-14 17:54:12,522][61585] Updated weights for policy 1, policy_version 7840 (0.0007) [2023-10-14 17:54:13,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 16056320. Throughput: 0: 1687.2, 1: 1647.5. Samples: 4023742. Policy #0 lag: (min: 23.0, avg: 29.3, max: 55.0) [2023-10-14 17:54:13,344][60425] Avg episode reward: [(0, '24.060'), (1, '39.780')] [2023-10-14 17:54:13,956][61552] Updated weights for policy 0, policy_version 7850 (0.0007) [2023-10-14 17:54:14,327][61552] Updated weights for policy 0, policy_version 7860 (0.0009) [2023-10-14 17:54:14,705][61552] Updated weights for policy 0, policy_version 7870 (0.0008) [2023-10-14 17:54:16,733][61585] Updated weights for policy 1, policy_version 7850 (0.0008) [2023-10-14 17:54:17,088][61585] Updated weights for policy 1, policy_version 7860 (0.0009) [2023-10-14 17:54:17,461][61585] Updated weights for policy 1, policy_version 7870 (0.0008) [2023-10-14 17:54:18,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 16121856. Throughput: 0: 1675.6, 1: 1664.6. Samples: 4034002. Policy #0 lag: (min: 23.0, avg: 29.3, max: 55.0) [2023-10-14 17:54:18,344][60425] Avg episode reward: [(0, '25.390'), (1, '40.400')] [2023-10-14 17:54:18,852][61552] Updated weights for policy 0, policy_version 7880 (0.0008) [2023-10-14 17:54:19,211][61552] Updated weights for policy 0, policy_version 7890 (0.0008) [2023-10-14 17:54:19,587][61552] Updated weights for policy 0, policy_version 7900 (0.0009) [2023-10-14 17:54:21,523][61585] Updated weights for policy 1, policy_version 7880 (0.0010) [2023-10-14 17:54:21,887][61585] Updated weights for policy 1, policy_version 7890 (0.0007) [2023-10-14 17:54:22,250][61585] Updated weights for policy 1, policy_version 7900 (0.0009) [2023-10-14 17:54:23,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 16187392. Throughput: 0: 1686.6, 1: 1646.3. Samples: 4053852. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:54:23,344][60425] Avg episode reward: [(0, '26.000'), (1, '42.820')] [2023-10-14 17:54:23,753][61552] Updated weights for policy 0, policy_version 7910 (0.0010) [2023-10-14 17:54:24,120][61552] Updated weights for policy 0, policy_version 7920 (0.0009) [2023-10-14 17:54:24,483][61552] Updated weights for policy 0, policy_version 7930 (0.0011) [2023-10-14 17:54:26,426][61585] Updated weights for policy 1, policy_version 7910 (0.0009) [2023-10-14 17:54:26,788][61585] Updated weights for policy 1, policy_version 7920 (0.0008) [2023-10-14 17:54:27,149][61585] Updated weights for policy 1, policy_version 7930 (0.0009) [2023-10-14 17:54:28,343][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 16252928. Throughput: 0: 1690.2, 1: 1652.2. Samples: 4073894. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:54:28,344][60425] Avg episode reward: [(0, '23.810'), (1, '40.640')] [2023-10-14 17:54:28,520][61552] Updated weights for policy 0, policy_version 7940 (0.0009) [2023-10-14 17:54:28,896][61552] Updated weights for policy 0, policy_version 7950 (0.0009) [2023-10-14 17:54:29,273][61552] Updated weights for policy 0, policy_version 7960 (0.0010) [2023-10-14 17:54:31,277][61585] Updated weights for policy 1, policy_version 7940 (0.0009) [2023-10-14 17:54:31,647][61585] Updated weights for policy 1, policy_version 7950 (0.0008) [2023-10-14 17:54:32,023][61585] Updated weights for policy 1, policy_version 7960 (0.0008) [2023-10-14 17:54:33,285][61552] Updated weights for policy 0, policy_version 7970 (0.0008) [2023-10-14 17:54:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 16318464. Throughput: 0: 1689.2, 1: 1660.4. Samples: 4084302. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-14 17:54:33,344][60425] Avg episode reward: [(0, '25.040'), (1, '44.230')] [2023-10-14 17:54:33,653][61552] Updated weights for policy 0, policy_version 7980 (0.0009) [2023-10-14 17:54:34,030][61552] Updated weights for policy 0, policy_version 7990 (0.0007) [2023-10-14 17:54:34,393][61552] Updated weights for policy 0, policy_version 8000 (0.0008) [2023-10-14 17:54:36,061][61585] Updated weights for policy 1, policy_version 7970 (0.0008) [2023-10-14 17:54:36,420][61585] Updated weights for policy 1, policy_version 7980 (0.0008) [2023-10-14 17:54:36,783][61585] Updated weights for policy 1, policy_version 7990 (0.0010) [2023-10-14 17:54:37,158][61585] Updated weights for policy 1, policy_version 8000 (0.0008) [2023-10-14 17:54:38,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 16384000. Throughput: 0: 1687.4, 1: 1652.8. Samples: 4104228. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-14 17:54:38,345][60425] Avg episode reward: [(0, '26.180'), (1, '44.140')] [2023-10-14 17:54:38,509][61552] Updated weights for policy 0, policy_version 8010 (0.0008) [2023-10-14 17:54:38,873][61552] Updated weights for policy 0, policy_version 8020 (0.0007) [2023-10-14 17:54:39,248][61552] Updated weights for policy 0, policy_version 8030 (0.0009) [2023-10-14 17:54:41,314][61585] Updated weights for policy 1, policy_version 8010 (0.0009) [2023-10-14 17:54:41,693][61585] Updated weights for policy 1, policy_version 8020 (0.0011) [2023-10-14 17:54:42,053][61585] Updated weights for policy 1, policy_version 8030 (0.0010) [2023-10-14 17:54:43,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 16449536. Throughput: 0: 1684.9, 1: 1669.2. Samples: 4124240. Policy #0 lag: (min: 27.0, avg: 33.1, max: 59.0) [2023-10-14 17:54:43,344][60425] Avg episode reward: [(0, '25.300'), (1, '40.630')] [2023-10-14 17:54:43,401][61552] Updated weights for policy 0, policy_version 8040 (0.0011) [2023-10-14 17:54:43,773][61552] Updated weights for policy 0, policy_version 8050 (0.0010) [2023-10-14 17:54:44,146][61552] Updated weights for policy 0, policy_version 8060 (0.0008) [2023-10-14 17:54:46,042][61585] Updated weights for policy 1, policy_version 8040 (0.0008) [2023-10-14 17:54:46,409][61585] Updated weights for policy 1, policy_version 8050 (0.0009) [2023-10-14 17:54:46,788][61585] Updated weights for policy 1, policy_version 8060 (0.0009) [2023-10-14 17:54:48,337][61552] Updated weights for policy 0, policy_version 8070 (0.0008) [2023-10-14 17:54:48,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 16515072. Throughput: 0: 1681.0, 1: 1674.5. Samples: 4134582. Policy #0 lag: (min: 27.0, avg: 33.1, max: 59.0) [2023-10-14 17:54:48,344][60425] Avg episode reward: [(0, '24.980'), (1, '43.880')] [2023-10-14 17:54:48,711][61552] Updated weights for policy 0, policy_version 8080 (0.0010) [2023-10-14 17:54:49,093][61552] Updated weights for policy 0, policy_version 8090 (0.0009) [2023-10-14 17:54:50,843][61585] Updated weights for policy 1, policy_version 8070 (0.0009) [2023-10-14 17:54:51,213][61585] Updated weights for policy 1, policy_version 8080 (0.0009) [2023-10-14 17:54:51,577][61585] Updated weights for policy 1, policy_version 8090 (0.0009) [2023-10-14 17:54:53,159][61552] Updated weights for policy 0, policy_version 8100 (0.0009) [2023-10-14 17:54:53,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 16580608. Throughput: 0: 1669.1, 1: 1656.8. Samples: 4153744. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 17:54:53,344][60425] Avg episode reward: [(0, '26.550'), (1, '42.530')] [2023-10-14 17:54:53,530][61552] Updated weights for policy 0, policy_version 8110 (0.0010) [2023-10-14 17:54:53,912][61552] Updated weights for policy 0, policy_version 8120 (0.0011) [2023-10-14 17:54:55,714][61585] Updated weights for policy 1, policy_version 8100 (0.0009) [2023-10-14 17:54:56,080][61585] Updated weights for policy 1, policy_version 8110 (0.0009) [2023-10-14 17:54:56,448][61585] Updated weights for policy 1, policy_version 8120 (0.0009) [2023-10-14 17:54:57,936][61552] Updated weights for policy 0, policy_version 8130 (0.0008) [2023-10-14 17:54:58,309][61552] Updated weights for policy 0, policy_version 8140 (0.0009) [2023-10-14 17:54:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 16646144. Throughput: 0: 1671.1, 1: 1672.3. Samples: 4174192. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 17:54:58,344][60425] Avg episode reward: [(0, '26.810'), (1, '42.620')] [2023-10-14 17:54:58,684][61552] Updated weights for policy 0, policy_version 8150 (0.0009) [2023-10-14 17:54:59,042][61552] Updated weights for policy 0, policy_version 8160 (0.0008) [2023-10-14 17:55:00,426][61585] Updated weights for policy 1, policy_version 8130 (0.0009) [2023-10-14 17:55:00,802][61585] Updated weights for policy 1, policy_version 8140 (0.0008) [2023-10-14 17:55:01,165][61585] Updated weights for policy 1, policy_version 8150 (0.0008) [2023-10-14 17:55:01,530][61585] Updated weights for policy 1, policy_version 8160 (0.0008) [2023-10-14 17:55:03,126][61552] Updated weights for policy 0, policy_version 8170 (0.0010) [2023-10-14 17:55:03,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 16711680. Throughput: 0: 1670.9, 1: 1663.7. Samples: 4184062. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:55:03,344][60425] Avg episode reward: [(0, '27.370'), (1, '42.830')] [2023-10-14 17:55:03,511][61552] Updated weights for policy 0, policy_version 8180 (0.0009) [2023-10-14 17:55:03,878][61552] Updated weights for policy 0, policy_version 8190 (0.0008) [2023-10-14 17:55:05,698][61585] Updated weights for policy 1, policy_version 8170 (0.0007) [2023-10-14 17:55:06,068][61585] Updated weights for policy 1, policy_version 8180 (0.0008) [2023-10-14 17:55:06,431][61585] Updated weights for policy 1, policy_version 8190 (0.0009) [2023-10-14 17:55:08,041][61552] Updated weights for policy 0, policy_version 8200 (0.0009) [2023-10-14 17:55:08,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 16777216. Throughput: 0: 1672.9, 1: 1659.6. Samples: 4203814. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:55:08,344][60425] Avg episode reward: [(0, '28.010'), (1, '43.970')] [2023-10-14 17:55:08,422][61552] Updated weights for policy 0, policy_version 8210 (0.0009) [2023-10-14 17:55:08,790][61552] Updated weights for policy 0, policy_version 8220 (0.0008) [2023-10-14 17:55:10,532][61585] Updated weights for policy 1, policy_version 8200 (0.0008) [2023-10-14 17:55:10,899][61585] Updated weights for policy 1, policy_version 8210 (0.0007) [2023-10-14 17:55:11,261][61585] Updated weights for policy 1, policy_version 8220 (0.0008) [2023-10-14 17:55:12,872][61552] Updated weights for policy 0, policy_version 8230 (0.0010) [2023-10-14 17:55:13,242][61552] Updated weights for policy 0, policy_version 8240 (0.0009) [2023-10-14 17:55:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 16842752. Throughput: 0: 1663.5, 1: 1681.7. Samples: 4224428. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:55:13,344][60425] Avg episode reward: [(0, '27.080'), (1, '46.350')] [2023-10-14 17:55:13,357][61248] Saving new best policy, reward=46.350! [2023-10-14 17:55:13,614][61552] Updated weights for policy 0, policy_version 8250 (0.0007) [2023-10-14 17:55:15,267][61585] Updated weights for policy 1, policy_version 8230 (0.0008) [2023-10-14 17:55:15,629][61585] Updated weights for policy 1, policy_version 8240 (0.0008) [2023-10-14 17:55:15,989][61585] Updated weights for policy 1, policy_version 8250 (0.0010) [2023-10-14 17:55:17,767][61552] Updated weights for policy 0, policy_version 8260 (0.0010) [2023-10-14 17:55:18,135][61552] Updated weights for policy 0, policy_version 8270 (0.0008) [2023-10-14 17:55:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 16908288. Throughput: 0: 1662.5, 1: 1663.6. Samples: 4233976. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:55:18,344][60425] Avg episode reward: [(0, '26.120'), (1, '44.110')] [2023-10-14 17:55:18,506][61552] Updated weights for policy 0, policy_version 8280 (0.0007) [2023-10-14 17:55:20,413][61585] Updated weights for policy 1, policy_version 8260 (0.0008) [2023-10-14 17:55:20,780][61585] Updated weights for policy 1, policy_version 8270 (0.0009) [2023-10-14 17:55:21,148][61585] Updated weights for policy 1, policy_version 8280 (0.0007) [2023-10-14 17:55:22,602][61552] Updated weights for policy 0, policy_version 8290 (0.0009) [2023-10-14 17:55:22,965][61552] Updated weights for policy 0, policy_version 8300 (0.0009) [2023-10-14 17:55:23,332][61552] Updated weights for policy 0, policy_version 8310 (0.0007) [2023-10-14 17:55:23,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 16973824. Throughput: 0: 1659.5, 1: 1664.2. Samples: 4253792. Policy #0 lag: (min: 11.0, avg: 35.6, max: 40.0) [2023-10-14 17:55:23,344][60425] Avg episode reward: [(0, '27.860'), (1, '50.140')] [2023-10-14 17:55:23,345][61248] Saving new best policy, reward=50.140! [2023-10-14 17:55:23,705][61552] Updated weights for policy 0, policy_version 8320 (0.0010) [2023-10-14 17:55:25,014][61585] Updated weights for policy 1, policy_version 8290 (0.0009) [2023-10-14 17:55:25,429][61585] Updated weights for policy 1, policy_version 8300 (0.0008) [2023-10-14 17:55:25,803][61585] Updated weights for policy 1, policy_version 8310 (0.0008) [2023-10-14 17:55:26,161][61585] Updated weights for policy 1, policy_version 8320 (0.0010) [2023-10-14 17:55:27,836][61552] Updated weights for policy 0, policy_version 8330 (0.0008) [2023-10-14 17:55:28,200][61552] Updated weights for policy 0, policy_version 8340 (0.0008) [2023-10-14 17:55:28,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 17039360. Throughput: 0: 1654.0, 1: 1674.8. Samples: 4274036. Policy #0 lag: (min: 11.0, avg: 35.6, max: 40.0) [2023-10-14 17:55:28,344][60425] Avg episode reward: [(0, '29.730'), (1, '45.260')] [2023-10-14 17:55:28,577][61552] Updated weights for policy 0, policy_version 8350 (0.0007) [2023-10-14 17:55:28,646][61172] Saving new best policy, reward=29.730! [2023-10-14 17:55:30,200][61585] Updated weights for policy 1, policy_version 8330 (0.0009) [2023-10-14 17:55:30,569][61585] Updated weights for policy 1, policy_version 8340 (0.0009) [2023-10-14 17:55:30,936][61585] Updated weights for policy 1, policy_version 8350 (0.0010) [2023-10-14 17:55:32,622][61552] Updated weights for policy 0, policy_version 8360 (0.0007) [2023-10-14 17:55:33,004][61552] Updated weights for policy 0, policy_version 8370 (0.0009) [2023-10-14 17:55:33,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 17104896. Throughput: 0: 1664.9, 1: 1649.6. Samples: 4283736. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:55:33,345][60425] Avg episode reward: [(0, '27.960'), (1, '44.250')] [2023-10-14 17:55:33,361][61552] Updated weights for policy 0, policy_version 8380 (0.0010) [2023-10-14 17:55:35,011][61585] Updated weights for policy 1, policy_version 8360 (0.0010) [2023-10-14 17:55:35,379][61585] Updated weights for policy 1, policy_version 8370 (0.0011) [2023-10-14 17:55:35,750][61585] Updated weights for policy 1, policy_version 8380 (0.0010) [2023-10-14 17:55:37,563][61552] Updated weights for policy 0, policy_version 8390 (0.0009) [2023-10-14 17:55:37,930][61552] Updated weights for policy 0, policy_version 8400 (0.0008) [2023-10-14 17:55:38,300][61552] Updated weights for policy 0, policy_version 8410 (0.0007) [2023-10-14 17:55:38,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 17170432. Throughput: 0: 1667.1, 1: 1670.5. Samples: 4303936. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:55:38,344][60425] Avg episode reward: [(0, '28.400'), (1, '46.880')] [2023-10-14 17:55:39,803][61585] Updated weights for policy 1, policy_version 8390 (0.0010) [2023-10-14 17:55:40,171][61585] Updated weights for policy 1, policy_version 8400 (0.0009) [2023-10-14 17:55:40,533][61585] Updated weights for policy 1, policy_version 8410 (0.0007) [2023-10-14 17:55:42,382][61552] Updated weights for policy 0, policy_version 8420 (0.0007) [2023-10-14 17:55:42,755][61552] Updated weights for policy 0, policy_version 8430 (0.0009) [2023-10-14 17:55:43,138][61552] Updated weights for policy 0, policy_version 8440 (0.0010) [2023-10-14 17:55:43,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 17235968. Throughput: 0: 1652.6, 1: 1675.2. Samples: 4323944. Policy #0 lag: (min: 0.0, avg: 25.0, max: 32.0) [2023-10-14 17:55:43,344][60425] Avg episode reward: [(0, '28.440'), (1, '45.640')] [2023-10-14 17:55:43,352][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000008416_8617984.pth... [2023-10-14 17:55:43,387][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000006880_7045120.pth [2023-10-14 17:55:43,432][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000008448_8650752.pth... [2023-10-14 17:55:43,470][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000006880_7045120.pth [2023-10-14 17:55:44,863][61585] Updated weights for policy 1, policy_version 8420 (0.0009) [2023-10-14 17:55:45,237][61585] Updated weights for policy 1, policy_version 8430 (0.0008) [2023-10-14 17:55:45,607][61585] Updated weights for policy 1, policy_version 8440 (0.0010) [2023-10-14 17:55:47,472][61552] Updated weights for policy 0, policy_version 8450 (0.0010) [2023-10-14 17:55:47,845][61552] Updated weights for policy 0, policy_version 8460 (0.0009) [2023-10-14 17:55:48,226][61552] Updated weights for policy 0, policy_version 8470 (0.0010) [2023-10-14 17:55:48,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 17301504. Throughput: 0: 1664.5, 1: 1658.0. Samples: 4333572. Policy #0 lag: (min: 0.0, avg: 25.0, max: 32.0) [2023-10-14 17:55:48,344][60425] Avg episode reward: [(0, '28.820'), (1, '45.830')] [2023-10-14 17:55:48,595][61552] Updated weights for policy 0, policy_version 8480 (0.0009) [2023-10-14 17:55:49,737][61585] Updated weights for policy 1, policy_version 8450 (0.0010) [2023-10-14 17:55:50,101][61585] Updated weights for policy 1, policy_version 8460 (0.0010) [2023-10-14 17:55:50,476][61585] Updated weights for policy 1, policy_version 8470 (0.0010) [2023-10-14 17:55:50,837][61585] Updated weights for policy 1, policy_version 8480 (0.0008) [2023-10-14 17:55:52,839][61552] Updated weights for policy 0, policy_version 8490 (0.0007) [2023-10-14 17:55:53,207][61552] Updated weights for policy 0, policy_version 8500 (0.0008) [2023-10-14 17:55:53,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 17367040. Throughput: 0: 1657.7, 1: 1669.7. Samples: 4353548. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-14 17:55:53,344][60425] Avg episode reward: [(0, '25.000'), (1, '44.750')] [2023-10-14 17:55:53,583][61552] Updated weights for policy 0, policy_version 8510 (0.0007) [2023-10-14 17:55:54,870][61585] Updated weights for policy 1, policy_version 8490 (0.0008) [2023-10-14 17:55:55,224][61585] Updated weights for policy 1, policy_version 8500 (0.0009) [2023-10-14 17:55:55,589][61585] Updated weights for policy 1, policy_version 8510 (0.0009) [2023-10-14 17:55:57,604][61552] Updated weights for policy 0, policy_version 8520 (0.0010) [2023-10-14 17:55:57,975][61552] Updated weights for policy 0, policy_version 8530 (0.0008) [2023-10-14 17:55:58,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 17432576. Throughput: 0: 1652.8, 1: 1664.4. Samples: 4373698. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-14 17:55:58,345][60425] Avg episode reward: [(0, '28.380'), (1, '45.990')] [2023-10-14 17:55:58,352][61552] Updated weights for policy 0, policy_version 8540 (0.0009) [2023-10-14 17:55:59,857][61585] Updated weights for policy 1, policy_version 8520 (0.0008) [2023-10-14 17:56:00,231][61585] Updated weights for policy 1, policy_version 8530 (0.0008) [2023-10-14 17:56:00,596][61585] Updated weights for policy 1, policy_version 8540 (0.0009) [2023-10-14 17:56:02,207][61552] Updated weights for policy 0, policy_version 8550 (0.0007) [2023-10-14 17:56:02,582][61552] Updated weights for policy 0, policy_version 8560 (0.0010) [2023-10-14 17:56:02,949][61552] Updated weights for policy 0, policy_version 8570 (0.0010) [2023-10-14 17:56:03,343][60425] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 17530880. Throughput: 0: 1665.5, 1: 1652.5. Samples: 4383288. Policy #0 lag: (min: 6.0, avg: 6.6, max: 23.0) [2023-10-14 17:56:03,345][60425] Avg episode reward: [(0, '27.600'), (1, '44.430')] [2023-10-14 17:56:04,730][61585] Updated weights for policy 1, policy_version 8550 (0.0008) [2023-10-14 17:56:05,102][61585] Updated weights for policy 1, policy_version 8560 (0.0008) [2023-10-14 17:56:05,471][61585] Updated weights for policy 1, policy_version 8570 (0.0008) [2023-10-14 17:56:07,113][61552] Updated weights for policy 0, policy_version 8580 (0.0008) [2023-10-14 17:56:07,491][61552] Updated weights for policy 0, policy_version 8590 (0.0007) [2023-10-14 17:56:07,859][61552] Updated weights for policy 0, policy_version 8600 (0.0010) [2023-10-14 17:56:08,343][60425] Fps is (10 sec: 16384.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 17596416. Throughput: 0: 1669.2, 1: 1668.0. Samples: 4403962. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:56:08,344][60425] Avg episode reward: [(0, '27.450'), (1, '47.110')] [2023-10-14 17:56:09,530][61585] Updated weights for policy 1, policy_version 8580 (0.0010) [2023-10-14 17:56:09,895][61585] Updated weights for policy 1, policy_version 8590 (0.0010) [2023-10-14 17:56:10,271][61585] Updated weights for policy 1, policy_version 8600 (0.0010) [2023-10-14 17:56:11,873][61552] Updated weights for policy 0, policy_version 8610 (0.0009) [2023-10-14 17:56:12,235][61552] Updated weights for policy 0, policy_version 8620 (0.0008) [2023-10-14 17:56:12,611][61552] Updated weights for policy 0, policy_version 8630 (0.0008) [2023-10-14 17:56:12,972][61552] Updated weights for policy 0, policy_version 8640 (0.0007) [2023-10-14 17:56:13,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 17661952. Throughput: 0: 1658.6, 1: 1668.3. Samples: 4423746. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:56:13,344][60425] Avg episode reward: [(0, '27.980'), (1, '45.400')] [2023-10-14 17:56:14,413][61585] Updated weights for policy 1, policy_version 8610 (0.0009) [2023-10-14 17:56:14,824][61585] Updated weights for policy 1, policy_version 8620 (0.0008) [2023-10-14 17:56:15,204][61585] Updated weights for policy 1, policy_version 8630 (0.0008) [2023-10-14 17:56:15,569][61585] Updated weights for policy 1, policy_version 8640 (0.0010) [2023-10-14 17:56:17,125][61552] Updated weights for policy 0, policy_version 8650 (0.0010) [2023-10-14 17:56:17,485][61552] Updated weights for policy 0, policy_version 8660 (0.0011) [2023-10-14 17:56:17,862][61552] Updated weights for policy 0, policy_version 8670 (0.0010) [2023-10-14 17:56:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 17727488. Throughput: 0: 1674.9, 1: 1657.9. Samples: 4433714. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-14 17:56:18,344][60425] Avg episode reward: [(0, '28.950'), (1, '49.200')] [2023-10-14 17:56:19,612][61585] Updated weights for policy 1, policy_version 8650 (0.0011) [2023-10-14 17:56:19,970][61585] Updated weights for policy 1, policy_version 8660 (0.0008) [2023-10-14 17:56:20,343][61585] Updated weights for policy 1, policy_version 8670 (0.0008) [2023-10-14 17:56:22,013][61552] Updated weights for policy 0, policy_version 8680 (0.0011) [2023-10-14 17:56:22,379][61552] Updated weights for policy 0, policy_version 8690 (0.0007) [2023-10-14 17:56:22,747][61552] Updated weights for policy 0, policy_version 8700 (0.0007) [2023-10-14 17:56:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 17793024. Throughput: 0: 1672.6, 1: 1660.0. Samples: 4453904. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-14 17:56:23,344][60425] Avg episode reward: [(0, '29.860'), (1, '49.370')] [2023-10-14 17:56:23,344][61172] Saving new best policy, reward=29.860! [2023-10-14 17:56:24,523][61585] Updated weights for policy 1, policy_version 8680 (0.0009) [2023-10-14 17:56:24,895][61585] Updated weights for policy 1, policy_version 8690 (0.0009) [2023-10-14 17:56:25,260][61585] Updated weights for policy 1, policy_version 8700 (0.0009) [2023-10-14 17:56:26,761][61552] Updated weights for policy 0, policy_version 8710 (0.0008) [2023-10-14 17:56:27,133][61552] Updated weights for policy 0, policy_version 8720 (0.0008) [2023-10-14 17:56:27,498][61552] Updated weights for policy 0, policy_version 8730 (0.0007) [2023-10-14 17:56:28,344][60425] Fps is (10 sec: 13106.7, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 17858560. Throughput: 0: 1660.6, 1: 1658.6. Samples: 4473308. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-10-14 17:56:28,344][60425] Avg episode reward: [(0, '29.980'), (1, '46.910')] [2023-10-14 17:56:28,353][61172] Saving new best policy, reward=29.980! [2023-10-14 17:56:29,593][61585] Updated weights for policy 1, policy_version 8710 (0.0008) [2023-10-14 17:56:29,964][61585] Updated weights for policy 1, policy_version 8720 (0.0009) [2023-10-14 17:56:30,342][61585] Updated weights for policy 1, policy_version 8730 (0.0008) [2023-10-14 17:56:31,507][61552] Updated weights for policy 0, policy_version 8740 (0.0009) [2023-10-14 17:56:31,875][61552] Updated weights for policy 0, policy_version 8750 (0.0009) [2023-10-14 17:56:32,247][61552] Updated weights for policy 0, policy_version 8760 (0.0007) [2023-10-14 17:56:33,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 17924096. Throughput: 0: 1677.1, 1: 1653.0. Samples: 4483424. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-10-14 17:56:33,344][60425] Avg episode reward: [(0, '29.060'), (1, '50.290')] [2023-10-14 17:56:33,345][61248] Saving new best policy, reward=50.290! [2023-10-14 17:56:34,574][61585] Updated weights for policy 1, policy_version 8740 (0.0009) [2023-10-14 17:56:34,928][61585] Updated weights for policy 1, policy_version 8750 (0.0008) [2023-10-14 17:56:35,293][61585] Updated weights for policy 1, policy_version 8760 (0.0007) [2023-10-14 17:56:36,190][61552] Updated weights for policy 0, policy_version 8770 (0.0008) [2023-10-14 17:56:36,564][61552] Updated weights for policy 0, policy_version 8780 (0.0010) [2023-10-14 17:56:36,936][61552] Updated weights for policy 0, policy_version 8790 (0.0007) [2023-10-14 17:56:37,307][61552] Updated weights for policy 0, policy_version 8800 (0.0010) [2023-10-14 17:56:38,343][60425] Fps is (10 sec: 13107.7, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 17989632. Throughput: 0: 1671.2, 1: 1658.4. Samples: 4503378. Policy #0 lag: (min: 10.0, avg: 18.0, max: 42.0) [2023-10-14 17:56:38,344][60425] Avg episode reward: [(0, '29.840'), (1, '44.890')] [2023-10-14 17:56:39,386][61585] Updated weights for policy 1, policy_version 8770 (0.0008) [2023-10-14 17:56:39,746][61585] Updated weights for policy 1, policy_version 8780 (0.0008) [2023-10-14 17:56:40,109][61585] Updated weights for policy 1, policy_version 8790 (0.0007) [2023-10-14 17:56:40,475][61585] Updated weights for policy 1, policy_version 8800 (0.0007) [2023-10-14 17:56:41,460][61552] Updated weights for policy 0, policy_version 8810 (0.0008) [2023-10-14 17:56:41,834][61552] Updated weights for policy 0, policy_version 8820 (0.0009) [2023-10-14 17:56:42,214][61552] Updated weights for policy 0, policy_version 8830 (0.0008) [2023-10-14 17:56:43,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 18055168. Throughput: 0: 1662.1, 1: 1664.7. Samples: 4523400. Policy #0 lag: (min: 10.0, avg: 18.0, max: 42.0) [2023-10-14 17:56:43,344][60425] Avg episode reward: [(0, '28.250'), (1, '48.960')] [2023-10-14 17:56:44,363][61585] Updated weights for policy 1, policy_version 8810 (0.0007) [2023-10-14 17:56:44,732][61585] Updated weights for policy 1, policy_version 8820 (0.0009) [2023-10-14 17:56:45,099][61585] Updated weights for policy 1, policy_version 8830 (0.0010) [2023-10-14 17:56:46,169][61552] Updated weights for policy 0, policy_version 8840 (0.0008) [2023-10-14 17:56:46,530][61552] Updated weights for policy 0, policy_version 8850 (0.0010) [2023-10-14 17:56:46,897][61552] Updated weights for policy 0, policy_version 8860 (0.0010) [2023-10-14 17:56:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 18120704. Throughput: 0: 1682.4, 1: 1663.8. Samples: 4533866. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:56:48,344][60425] Avg episode reward: [(0, '29.020'), (1, '44.830')] [2023-10-14 17:56:49,218][61585] Updated weights for policy 1, policy_version 8840 (0.0008) [2023-10-14 17:56:49,589][61585] Updated weights for policy 1, policy_version 8850 (0.0009) [2023-10-14 17:56:49,948][61585] Updated weights for policy 1, policy_version 8860 (0.0008) [2023-10-14 17:56:51,041][61552] Updated weights for policy 0, policy_version 8870 (0.0009) [2023-10-14 17:56:51,404][61552] Updated weights for policy 0, policy_version 8880 (0.0009) [2023-10-14 17:56:51,778][61552] Updated weights for policy 0, policy_version 8890 (0.0010) [2023-10-14 17:56:53,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 18186240. Throughput: 0: 1655.8, 1: 1662.9. Samples: 4553304. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:56:53,344][60425] Avg episode reward: [(0, '28.540'), (1, '44.300')] [2023-10-14 17:56:54,190][61585] Updated weights for policy 1, policy_version 8870 (0.0008) [2023-10-14 17:56:54,553][61585] Updated weights for policy 1, policy_version 8880 (0.0007) [2023-10-14 17:56:54,919][61585] Updated weights for policy 1, policy_version 8890 (0.0008) [2023-10-14 17:56:55,875][61552] Updated weights for policy 0, policy_version 8900 (0.0008) [2023-10-14 17:56:56,246][61552] Updated weights for policy 0, policy_version 8910 (0.0009) [2023-10-14 17:56:56,612][61552] Updated weights for policy 0, policy_version 8920 (0.0007) [2023-10-14 17:56:58,344][60425] Fps is (10 sec: 13106.6, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 18251776. Throughput: 0: 1666.3, 1: 1659.9. Samples: 4573422. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:56:58,345][60425] Avg episode reward: [(0, '28.090'), (1, '44.940')] [2023-10-14 17:56:59,049][61585] Updated weights for policy 1, policy_version 8900 (0.0009) [2023-10-14 17:56:59,446][61585] Updated weights for policy 1, policy_version 8910 (0.0007) [2023-10-14 17:56:59,819][61585] Updated weights for policy 1, policy_version 8920 (0.0008) [2023-10-14 17:57:00,794][61552] Updated weights for policy 0, policy_version 8930 (0.0008) [2023-10-14 17:57:01,166][61552] Updated weights for policy 0, policy_version 8940 (0.0008) [2023-10-14 17:57:01,530][61552] Updated weights for policy 0, policy_version 8950 (0.0009) [2023-10-14 17:57:01,900][61552] Updated weights for policy 0, policy_version 8960 (0.0007) [2023-10-14 17:57:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 18317312. Throughput: 0: 1670.4, 1: 1660.5. Samples: 4583602. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:57:03,344][60425] Avg episode reward: [(0, '28.680'), (1, '46.150')] [2023-10-14 17:57:03,831][61585] Updated weights for policy 1, policy_version 8930 (0.0009) [2023-10-14 17:57:04,195][61585] Updated weights for policy 1, policy_version 8940 (0.0010) [2023-10-14 17:57:04,554][61585] Updated weights for policy 1, policy_version 8950 (0.0008) [2023-10-14 17:57:04,920][61585] Updated weights for policy 1, policy_version 8960 (0.0010) [2023-10-14 17:57:06,253][61552] Updated weights for policy 0, policy_version 8970 (0.0008) [2023-10-14 17:57:06,628][61552] Updated weights for policy 0, policy_version 8980 (0.0011) [2023-10-14 17:57:07,001][61552] Updated weights for policy 0, policy_version 8990 (0.0007) [2023-10-14 17:57:08,343][60425] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 18382848. Throughput: 0: 1654.5, 1: 1666.4. Samples: 4603344. Policy #0 lag: (min: 12.0, avg: 17.9, max: 44.0) [2023-10-14 17:57:08,344][60425] Avg episode reward: [(0, '30.410'), (1, '47.530')] [2023-10-14 17:57:08,345][61172] Saving new best policy, reward=30.410! [2023-10-14 17:57:09,042][61585] Updated weights for policy 1, policy_version 8970 (0.0007) [2023-10-14 17:57:09,403][61585] Updated weights for policy 1, policy_version 8980 (0.0007) [2023-10-14 17:57:09,773][61585] Updated weights for policy 1, policy_version 8990 (0.0010) [2023-10-14 17:57:10,937][61552] Updated weights for policy 0, policy_version 9000 (0.0008) [2023-10-14 17:57:11,316][61552] Updated weights for policy 0, policy_version 9010 (0.0009) [2023-10-14 17:57:11,674][61552] Updated weights for policy 0, policy_version 9020 (0.0010) [2023-10-14 17:57:13,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 18448384. Throughput: 0: 1673.3, 1: 1667.4. Samples: 4623642. Policy #0 lag: (min: 12.0, avg: 17.9, max: 44.0) [2023-10-14 17:57:13,345][60425] Avg episode reward: [(0, '30.970'), (1, '49.000')] [2023-10-14 17:57:13,354][61172] Saving new best policy, reward=30.970! [2023-10-14 17:57:13,890][61585] Updated weights for policy 1, policy_version 9000 (0.0007) [2023-10-14 17:57:14,248][61585] Updated weights for policy 1, policy_version 9010 (0.0009) [2023-10-14 17:57:14,617][61585] Updated weights for policy 1, policy_version 9020 (0.0007) [2023-10-14 17:57:15,564][61552] Updated weights for policy 0, policy_version 9030 (0.0009) [2023-10-14 17:57:15,939][61552] Updated weights for policy 0, policy_version 9040 (0.0007) [2023-10-14 17:57:16,313][61552] Updated weights for policy 0, policy_version 9050 (0.0009) [2023-10-14 17:57:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 18513920. Throughput: 0: 1671.4, 1: 1672.1. Samples: 4633884. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:57:18,344][60425] Avg episode reward: [(0, '29.380'), (1, '49.020')] [2023-10-14 17:57:18,670][61585] Updated weights for policy 1, policy_version 9030 (0.0007) [2023-10-14 17:57:19,038][61585] Updated weights for policy 1, policy_version 9040 (0.0007) [2023-10-14 17:57:19,401][61585] Updated weights for policy 1, policy_version 9050 (0.0007) [2023-10-14 17:57:20,599][61552] Updated weights for policy 0, policy_version 9060 (0.0008) [2023-10-14 17:57:20,970][61552] Updated weights for policy 0, policy_version 9070 (0.0007) [2023-10-14 17:57:21,335][61552] Updated weights for policy 0, policy_version 9080 (0.0009) [2023-10-14 17:57:23,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 18579456. Throughput: 0: 1659.5, 1: 1674.6. Samples: 4653410. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:57:23,344][60425] Avg episode reward: [(0, '29.930'), (1, '47.700')] [2023-10-14 17:57:23,461][61585] Updated weights for policy 1, policy_version 9060 (0.0008) [2023-10-14 17:57:23,828][61585] Updated weights for policy 1, policy_version 9070 (0.0009) [2023-10-14 17:57:24,195][61585] Updated weights for policy 1, policy_version 9080 (0.0011) [2023-10-14 17:57:25,458][61552] Updated weights for policy 0, policy_version 9090 (0.0010) [2023-10-14 17:57:25,826][61552] Updated weights for policy 0, policy_version 9100 (0.0007) [2023-10-14 17:57:26,209][61552] Updated weights for policy 0, policy_version 9110 (0.0008) [2023-10-14 17:57:26,572][61552] Updated weights for policy 0, policy_version 9120 (0.0008) [2023-10-14 17:57:28,227][61585] Updated weights for policy 1, policy_version 9090 (0.0007) [2023-10-14 17:57:28,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 18644992. Throughput: 0: 1675.1, 1: 1672.2. Samples: 4674030. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-14 17:57:28,345][60425] Avg episode reward: [(0, '29.170'), (1, '47.780')] [2023-10-14 17:57:28,585][61585] Updated weights for policy 1, policy_version 9100 (0.0008) [2023-10-14 17:57:28,950][61585] Updated weights for policy 1, policy_version 9110 (0.0008) [2023-10-14 17:57:29,319][61585] Updated weights for policy 1, policy_version 9120 (0.0008) [2023-10-14 17:57:30,614][61552] Updated weights for policy 0, policy_version 9130 (0.0007) [2023-10-14 17:57:30,973][61552] Updated weights for policy 0, policy_version 9140 (0.0007) [2023-10-14 17:57:31,344][61552] Updated weights for policy 0, policy_version 9150 (0.0009) [2023-10-14 17:57:33,322][61585] Updated weights for policy 1, policy_version 9130 (0.0008) [2023-10-14 17:57:33,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 18710528. Throughput: 0: 1662.0, 1: 1672.4. Samples: 4683912. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-14 17:57:33,344][60425] Avg episode reward: [(0, '28.030'), (1, '48.420')] [2023-10-14 17:57:33,687][61585] Updated weights for policy 1, policy_version 9140 (0.0009) [2023-10-14 17:57:34,052][61585] Updated weights for policy 1, policy_version 9150 (0.0007) [2023-10-14 17:57:35,442][61552] Updated weights for policy 0, policy_version 9160 (0.0009) [2023-10-14 17:57:35,824][61552] Updated weights for policy 0, policy_version 9170 (0.0010) [2023-10-14 17:57:36,187][61552] Updated weights for policy 0, policy_version 9180 (0.0010) [2023-10-14 17:57:38,165][61585] Updated weights for policy 1, policy_version 9160 (0.0009) [2023-10-14 17:57:38,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 18776064. Throughput: 0: 1668.4, 1: 1672.8. Samples: 4703662. Policy #0 lag: (min: 17.0, avg: 19.6, max: 49.0) [2023-10-14 17:57:38,344][60425] Avg episode reward: [(0, '28.820'), (1, '48.830')] [2023-10-14 17:57:38,530][61585] Updated weights for policy 1, policy_version 9170 (0.0010) [2023-10-14 17:57:38,898][61585] Updated weights for policy 1, policy_version 9180 (0.0010) [2023-10-14 17:57:40,315][61552] Updated weights for policy 0, policy_version 9190 (0.0010) [2023-10-14 17:57:40,686][61552] Updated weights for policy 0, policy_version 9200 (0.0009) [2023-10-14 17:57:41,064][61552] Updated weights for policy 0, policy_version 9210 (0.0007) [2023-10-14 17:57:43,112][61585] Updated weights for policy 1, policy_version 9190 (0.0010) [2023-10-14 17:57:43,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 18841600. Throughput: 0: 1680.0, 1: 1673.6. Samples: 4724330. Policy #0 lag: (min: 17.0, avg: 19.6, max: 49.0) [2023-10-14 17:57:43,345][60425] Avg episode reward: [(0, '30.680'), (1, '48.410')] [2023-10-14 17:57:43,356][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000009216_9437184.pth... [2023-10-14 17:57:43,395][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000007648_7831552.pth [2023-10-14 17:57:43,480][61585] Updated weights for policy 1, policy_version 9200 (0.0010) [2023-10-14 17:57:43,848][61585] Updated weights for policy 1, policy_version 9210 (0.0008) [2023-10-14 17:57:44,068][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000009216_9437184.pth... [2023-10-14 17:57:44,109][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000007648_7831552.pth [2023-10-14 17:57:44,827][61552] Updated weights for policy 0, policy_version 9220 (0.0010) [2023-10-14 17:57:45,199][61552] Updated weights for policy 0, policy_version 9230 (0.0010) [2023-10-14 17:57:45,571][61552] Updated weights for policy 0, policy_version 9240 (0.0009) [2023-10-14 17:57:48,073][61585] Updated weights for policy 1, policy_version 9220 (0.0010) [2023-10-14 17:57:48,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 18907136. Throughput: 0: 1663.0, 1: 1677.1. Samples: 4733906. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:57:48,344][60425] Avg episode reward: [(0, '30.680'), (1, '48.950')] [2023-10-14 17:57:48,472][61585] Updated weights for policy 1, policy_version 9230 (0.0009) [2023-10-14 17:57:48,845][61585] Updated weights for policy 1, policy_version 9240 (0.0008) [2023-10-14 17:57:49,761][61552] Updated weights for policy 0, policy_version 9250 (0.0008) [2023-10-14 17:57:50,125][61552] Updated weights for policy 0, policy_version 9260 (0.0008) [2023-10-14 17:57:50,493][61552] Updated weights for policy 0, policy_version 9270 (0.0009) [2023-10-14 17:57:50,864][61552] Updated weights for policy 0, policy_version 9280 (0.0008) [2023-10-14 17:57:52,846][61585] Updated weights for policy 1, policy_version 9250 (0.0007) [2023-10-14 17:57:53,208][61585] Updated weights for policy 1, policy_version 9260 (0.0009) [2023-10-14 17:57:53,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 18972672. Throughput: 0: 1673.9, 1: 1672.6. Samples: 4753936. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:57:53,344][60425] Avg episode reward: [(0, '29.000'), (1, '48.080')] [2023-10-14 17:57:53,575][61585] Updated weights for policy 1, policy_version 9270 (0.0008) [2023-10-14 17:57:53,941][61585] Updated weights for policy 1, policy_version 9280 (0.0008) [2023-10-14 17:57:54,956][61552] Updated weights for policy 0, policy_version 9290 (0.0009) [2023-10-14 17:57:55,325][61552] Updated weights for policy 0, policy_version 9300 (0.0009) [2023-10-14 17:57:55,701][61552] Updated weights for policy 0, policy_version 9310 (0.0008) [2023-10-14 17:57:58,105][61585] Updated weights for policy 1, policy_version 9290 (0.0008) [2023-10-14 17:57:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 19038208. Throughput: 0: 1679.2, 1: 1673.8. Samples: 4774528. Policy #0 lag: (min: 11.0, avg: 12.8, max: 40.0) [2023-10-14 17:57:58,344][60425] Avg episode reward: [(0, '29.870'), (1, '47.650')] [2023-10-14 17:57:58,481][61585] Updated weights for policy 1, policy_version 9300 (0.0007) [2023-10-14 17:57:58,852][61585] Updated weights for policy 1, policy_version 9310 (0.0007) [2023-10-14 17:57:59,696][61552] Updated weights for policy 0, policy_version 9320 (0.0010) [2023-10-14 17:58:00,057][61552] Updated weights for policy 0, policy_version 9330 (0.0008) [2023-10-14 17:58:00,428][61552] Updated weights for policy 0, policy_version 9340 (0.0010) [2023-10-14 17:58:02,943][61585] Updated weights for policy 1, policy_version 9320 (0.0007) [2023-10-14 17:58:03,313][61585] Updated weights for policy 1, policy_version 9330 (0.0008) [2023-10-14 17:58:03,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 19103744. Throughput: 0: 1657.6, 1: 1673.2. Samples: 4783774. Policy #0 lag: (min: 11.0, avg: 12.8, max: 40.0) [2023-10-14 17:58:03,344][60425] Avg episode reward: [(0, '31.460'), (1, '46.750')] [2023-10-14 17:58:03,345][61172] Saving new best policy, reward=31.460! [2023-10-14 17:58:03,675][61585] Updated weights for policy 1, policy_version 9340 (0.0007) [2023-10-14 17:58:04,330][61552] Updated weights for policy 0, policy_version 9350 (0.0009) [2023-10-14 17:58:04,688][61552] Updated weights for policy 0, policy_version 9360 (0.0009) [2023-10-14 17:58:05,063][61552] Updated weights for policy 0, policy_version 9370 (0.0007) [2023-10-14 17:58:07,897][61585] Updated weights for policy 1, policy_version 9350 (0.0007) [2023-10-14 17:58:08,262][61585] Updated weights for policy 1, policy_version 9360 (0.0008) [2023-10-14 17:58:08,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 19169280. Throughput: 0: 1687.0, 1: 1669.5. Samples: 4804454. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-14 17:58:08,344][60425] Avg episode reward: [(0, '30.220'), (1, '47.630')] [2023-10-14 17:58:08,622][61585] Updated weights for policy 1, policy_version 9370 (0.0007) [2023-10-14 17:58:09,214][61552] Updated weights for policy 0, policy_version 9380 (0.0008) [2023-10-14 17:58:09,578][61552] Updated weights for policy 0, policy_version 9390 (0.0008) [2023-10-14 17:58:09,952][61552] Updated weights for policy 0, policy_version 9400 (0.0009) [2023-10-14 17:58:12,696][61585] Updated weights for policy 1, policy_version 9380 (0.0008) [2023-10-14 17:58:13,069][61585] Updated weights for policy 1, policy_version 9390 (0.0008) [2023-10-14 17:58:13,344][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 19234816. Throughput: 0: 1688.0, 1: 1663.0. Samples: 4824826. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-14 17:58:13,345][60425] Avg episode reward: [(0, '30.930'), (1, '49.960')] [2023-10-14 17:58:13,446][61585] Updated weights for policy 1, policy_version 9400 (0.0008) [2023-10-14 17:58:14,028][61552] Updated weights for policy 0, policy_version 9410 (0.0008) [2023-10-14 17:58:14,390][61552] Updated weights for policy 0, policy_version 9420 (0.0008) [2023-10-14 17:58:14,764][61552] Updated weights for policy 0, policy_version 9430 (0.0009) [2023-10-14 17:58:15,143][61552] Updated weights for policy 0, policy_version 9440 (0.0009) [2023-10-14 17:58:17,465][61585] Updated weights for policy 1, policy_version 9410 (0.0008) [2023-10-14 17:58:17,834][61585] Updated weights for policy 1, policy_version 9420 (0.0008) [2023-10-14 17:58:18,207][61585] Updated weights for policy 1, policy_version 9430 (0.0009) [2023-10-14 17:58:18,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 19300352. Throughput: 0: 1669.1, 1: 1670.0. Samples: 4834174. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-14 17:58:18,344][60425] Avg episode reward: [(0, '31.070'), (1, '48.770')] [2023-10-14 17:58:18,566][61585] Updated weights for policy 1, policy_version 9440 (0.0008) [2023-10-14 17:58:19,123][61552] Updated weights for policy 0, policy_version 9450 (0.0009) [2023-10-14 17:58:19,495][61552] Updated weights for policy 0, policy_version 9460 (0.0007) [2023-10-14 17:58:19,866][61552] Updated weights for policy 0, policy_version 9470 (0.0011) [2023-10-14 17:58:22,827][61585] Updated weights for policy 1, policy_version 9450 (0.0011) [2023-10-14 17:58:23,196][61585] Updated weights for policy 1, policy_version 9460 (0.0007) [2023-10-14 17:58:23,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 19365888. Throughput: 0: 1686.9, 1: 1670.3. Samples: 4854734. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-14 17:58:23,344][60425] Avg episode reward: [(0, '31.500'), (1, '50.150')] [2023-10-14 17:58:23,346][61172] Saving new best policy, reward=31.500! [2023-10-14 17:58:23,557][61585] Updated weights for policy 1, policy_version 9470 (0.0009) [2023-10-14 17:58:24,057][61552] Updated weights for policy 0, policy_version 9480 (0.0009) [2023-10-14 17:58:24,422][61552] Updated weights for policy 0, policy_version 9490 (0.0008) [2023-10-14 17:58:24,803][61552] Updated weights for policy 0, policy_version 9500 (0.0009) [2023-10-14 17:58:27,670][61585] Updated weights for policy 1, policy_version 9480 (0.0010) [2023-10-14 17:58:28,033][61585] Updated weights for policy 1, policy_version 9490 (0.0011) [2023-10-14 17:58:28,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 19431424. Throughput: 0: 1682.9, 1: 1662.8. Samples: 4874886. Policy #0 lag: (min: 21.0, avg: 27.5, max: 53.0) [2023-10-14 17:58:28,344][60425] Avg episode reward: [(0, '32.390'), (1, '49.290')] [2023-10-14 17:58:28,353][61172] Saving new best policy, reward=32.390! [2023-10-14 17:58:28,403][61585] Updated weights for policy 1, policy_version 9500 (0.0009) [2023-10-14 17:58:29,042][61552] Updated weights for policy 0, policy_version 9510 (0.0008) [2023-10-14 17:58:29,413][61552] Updated weights for policy 0, policy_version 9520 (0.0008) [2023-10-14 17:58:29,780][61552] Updated weights for policy 0, policy_version 9530 (0.0009) [2023-10-14 17:58:32,678][61585] Updated weights for policy 1, policy_version 9510 (0.0008) [2023-10-14 17:58:33,082][61585] Updated weights for policy 1, policy_version 9520 (0.0009) [2023-10-14 17:58:33,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 19496960. Throughput: 0: 1673.9, 1: 1672.5. Samples: 4884492. Policy #0 lag: (min: 21.0, avg: 27.5, max: 53.0) [2023-10-14 17:58:33,344][60425] Avg episode reward: [(0, '31.810'), (1, '48.830')] [2023-10-14 17:58:33,434][61585] Updated weights for policy 1, policy_version 9530 (0.0011) [2023-10-14 17:58:33,801][61552] Updated weights for policy 0, policy_version 9540 (0.0009) [2023-10-14 17:58:34,172][61552] Updated weights for policy 0, policy_version 9550 (0.0010) [2023-10-14 17:58:34,539][61552] Updated weights for policy 0, policy_version 9560 (0.0010) [2023-10-14 17:58:37,287][61585] Updated weights for policy 1, policy_version 9540 (0.0008) [2023-10-14 17:58:37,644][61585] Updated weights for policy 1, policy_version 9550 (0.0008) [2023-10-14 17:58:38,018][61585] Updated weights for policy 1, policy_version 9560 (0.0010) [2023-10-14 17:58:38,343][60425] Fps is (10 sec: 16384.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 19595264. Throughput: 0: 1684.2, 1: 1669.2. Samples: 4904840. Policy #0 lag: (min: 18.0, avg: 20.5, max: 50.0) [2023-10-14 17:58:38,344][60425] Avg episode reward: [(0, '28.720'), (1, '48.790')] [2023-10-14 17:58:38,626][61552] Updated weights for policy 0, policy_version 9570 (0.0010) [2023-10-14 17:58:38,994][61552] Updated weights for policy 0, policy_version 9580 (0.0007) [2023-10-14 17:58:39,360][61552] Updated weights for policy 0, policy_version 9590 (0.0007) [2023-10-14 17:58:39,727][61552] Updated weights for policy 0, policy_version 9600 (0.0008) [2023-10-14 17:58:42,155][61585] Updated weights for policy 1, policy_version 9570 (0.0009) [2023-10-14 17:58:42,521][61585] Updated weights for policy 1, policy_version 9580 (0.0007) [2023-10-14 17:58:42,888][61585] Updated weights for policy 1, policy_version 9590 (0.0008) [2023-10-14 17:58:43,267][61585] Updated weights for policy 1, policy_version 9600 (0.0009) [2023-10-14 17:58:43,343][60425] Fps is (10 sec: 16383.9, 60 sec: 13653.4, 300 sec: 13329.3). Total num frames: 19660800. Throughput: 0: 1684.8, 1: 1651.1. Samples: 4924646. Policy #0 lag: (min: 18.0, avg: 20.5, max: 50.0) [2023-10-14 17:58:43,344][60425] Avg episode reward: [(0, '31.520'), (1, '49.110')] [2023-10-14 17:58:43,813][61552] Updated weights for policy 0, policy_version 9610 (0.0007) [2023-10-14 17:58:44,187][61552] Updated weights for policy 0, policy_version 9620 (0.0008) [2023-10-14 17:58:44,554][61552] Updated weights for policy 0, policy_version 9630 (0.0010) [2023-10-14 17:58:47,566][61585] Updated weights for policy 1, policy_version 9610 (0.0009) [2023-10-14 17:58:47,926][61585] Updated weights for policy 1, policy_version 9620 (0.0010) [2023-10-14 17:58:48,295][61585] Updated weights for policy 1, policy_version 9630 (0.0010) [2023-10-14 17:58:48,343][60425] Fps is (10 sec: 9830.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 19693568. Throughput: 0: 1681.7, 1: 1664.9. Samples: 4934372. Policy #0 lag: (min: 14.0, avg: 22.0, max: 46.0) [2023-10-14 17:58:48,344][60425] Avg episode reward: [(0, '30.890'), (1, '49.650')] [2023-10-14 17:58:48,575][61552] Updated weights for policy 0, policy_version 9640 (0.0009) [2023-10-14 17:58:48,956][61552] Updated weights for policy 0, policy_version 9650 (0.0011) [2023-10-14 17:58:49,328][61552] Updated weights for policy 0, policy_version 9660 (0.0011) [2023-10-14 17:58:52,282][61585] Updated weights for policy 1, policy_version 9640 (0.0008) [2023-10-14 17:58:52,648][61585] Updated weights for policy 1, policy_version 9650 (0.0007) [2023-10-14 17:58:53,014][61585] Updated weights for policy 1, policy_version 9660 (0.0008) [2023-10-14 17:58:53,291][61552] Updated weights for policy 0, policy_version 9670 (0.0009) [2023-10-14 17:58:53,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 19791872. Throughput: 0: 1676.1, 1: 1667.2. Samples: 4954900. Policy #0 lag: (min: 14.0, avg: 22.0, max: 46.0) [2023-10-14 17:58:53,344][60425] Avg episode reward: [(0, '32.220'), (1, '48.930')] [2023-10-14 17:58:53,659][61552] Updated weights for policy 0, policy_version 9680 (0.0008) [2023-10-14 17:58:54,038][61552] Updated weights for policy 0, policy_version 9690 (0.0008) [2023-10-14 17:58:57,153][61585] Updated weights for policy 1, policy_version 9670 (0.0008) [2023-10-14 17:58:57,524][61585] Updated weights for policy 1, policy_version 9680 (0.0008) [2023-10-14 17:58:57,893][61585] Updated weights for policy 1, policy_version 9690 (0.0009) [2023-10-14 17:58:57,998][61552] Updated weights for policy 0, policy_version 9700 (0.0009) [2023-10-14 17:58:58,343][60425] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 19857408. Throughput: 0: 1680.9, 1: 1652.3. Samples: 4974818. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:58:58,344][60425] Avg episode reward: [(0, '31.740'), (1, '50.480')] [2023-10-14 17:58:58,356][61248] Saving new best policy, reward=50.480! [2023-10-14 17:58:58,435][61552] Updated weights for policy 0, policy_version 9712 (0.0008) [2023-10-14 17:58:58,812][61552] Updated weights for policy 0, policy_version 9722 (0.0008) [2023-10-14 17:59:01,917][61585] Updated weights for policy 1, policy_version 9700 (0.0008) [2023-10-14 17:59:02,283][61585] Updated weights for policy 1, policy_version 9710 (0.0008) [2023-10-14 17:59:02,652][61585] Updated weights for policy 1, policy_version 9720 (0.0010) [2023-10-14 17:59:02,987][61552] Updated weights for policy 0, policy_version 9732 (0.0009) [2023-10-14 17:59:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 19922944. Throughput: 0: 1678.8, 1: 1668.0. Samples: 4984782. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:59:03,344][60425] Avg episode reward: [(0, '30.780'), (1, '47.670')] [2023-10-14 17:59:03,364][61552] Updated weights for policy 0, policy_version 9742 (0.0008) [2023-10-14 17:59:03,730][61552] Updated weights for policy 0, policy_version 9752 (0.0007) [2023-10-14 17:59:06,939][61585] Updated weights for policy 1, policy_version 9730 (0.0009) [2023-10-14 17:59:07,307][61585] Updated weights for policy 1, policy_version 9740 (0.0007) [2023-10-14 17:59:07,670][61585] Updated weights for policy 1, policy_version 9750 (0.0009) [2023-10-14 17:59:07,919][61552] Updated weights for policy 0, policy_version 9762 (0.0009) [2023-10-14 17:59:08,037][61585] Updated weights for policy 1, policy_version 9760 (0.0008) [2023-10-14 17:59:08,298][61552] Updated weights for policy 0, policy_version 9772 (0.0011) [2023-10-14 17:59:08,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 19988480. Throughput: 0: 1673.2, 1: 1667.1. Samples: 5005046. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:59:08,344][60425] Avg episode reward: [(0, '30.720'), (1, '48.490')] [2023-10-14 17:59:08,668][61552] Updated weights for policy 0, policy_version 9782 (0.0009) [2023-10-14 17:59:09,043][61552] Updated weights for policy 0, policy_version 9792 (0.0009) [2023-10-14 17:59:11,882][61585] Updated weights for policy 1, policy_version 9770 (0.0007) [2023-10-14 17:59:12,257][61585] Updated weights for policy 1, policy_version 9780 (0.0008) [2023-10-14 17:59:12,620][61585] Updated weights for policy 1, policy_version 9790 (0.0008) [2023-10-14 17:59:13,113][61552] Updated weights for policy 0, policy_version 9802 (0.0007) [2023-10-14 17:59:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 20054016. Throughput: 0: 1675.6, 1: 1649.9. Samples: 5024536. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:59:13,344][60425] Avg episode reward: [(0, '28.870'), (1, '50.060')] [2023-10-14 17:59:13,478][61552] Updated weights for policy 0, policy_version 9812 (0.0008) [2023-10-14 17:59:13,855][61552] Updated weights for policy 0, policy_version 9822 (0.0009) [2023-10-14 17:59:16,726][61585] Updated weights for policy 1, policy_version 9800 (0.0008) [2023-10-14 17:59:17,088][61585] Updated weights for policy 1, policy_version 9810 (0.0007) [2023-10-14 17:59:17,457][61585] Updated weights for policy 1, policy_version 9820 (0.0007) [2023-10-14 17:59:18,090][61552] Updated weights for policy 0, policy_version 9832 (0.0008) [2023-10-14 17:59:18,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 20119552. Throughput: 0: 1672.6, 1: 1665.7. Samples: 5034714. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-14 17:59:18,344][60425] Avg episode reward: [(0, '30.830'), (1, '45.920')] [2023-10-14 17:59:18,460][61552] Updated weights for policy 0, policy_version 9842 (0.0010) [2023-10-14 17:59:18,832][61552] Updated weights for policy 0, policy_version 9852 (0.0012) [2023-10-14 17:59:21,704][61585] Updated weights for policy 1, policy_version 9830 (0.0009) [2023-10-14 17:59:22,083][61585] Updated weights for policy 1, policy_version 9840 (0.0010) [2023-10-14 17:59:22,445][61585] Updated weights for policy 1, policy_version 9850 (0.0008) [2023-10-14 17:59:23,061][61552] Updated weights for policy 0, policy_version 9862 (0.0009) [2023-10-14 17:59:23,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 20185088. Throughput: 0: 1665.4, 1: 1663.5. Samples: 5054644. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-14 17:59:23,344][60425] Avg episode reward: [(0, '30.780'), (1, '48.870')] [2023-10-14 17:59:23,436][61552] Updated weights for policy 0, policy_version 9872 (0.0007) [2023-10-14 17:59:23,803][61552] Updated weights for policy 0, policy_version 9882 (0.0009) [2023-10-14 17:59:26,625][61585] Updated weights for policy 1, policy_version 9860 (0.0011) [2023-10-14 17:59:26,995][61585] Updated weights for policy 1, policy_version 9870 (0.0010) [2023-10-14 17:59:27,358][61585] Updated weights for policy 1, policy_version 9880 (0.0007) [2023-10-14 17:59:27,884][61552] Updated weights for policy 0, policy_version 9892 (0.0009) [2023-10-14 17:59:28,265][61552] Updated weights for policy 0, policy_version 9902 (0.0011) [2023-10-14 17:59:28,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 20250624. Throughput: 0: 1663.9, 1: 1656.3. Samples: 5074052. Policy #0 lag: (min: 26.0, avg: 33.7, max: 58.0) [2023-10-14 17:59:28,344][60425] Avg episode reward: [(0, '31.190'), (1, '49.170')] [2023-10-14 17:59:28,631][61552] Updated weights for policy 0, policy_version 9912 (0.0011) [2023-10-14 17:59:31,411][61585] Updated weights for policy 1, policy_version 9890 (0.0008) [2023-10-14 17:59:31,785][61585] Updated weights for policy 1, policy_version 9900 (0.0009) [2023-10-14 17:59:32,157][61585] Updated weights for policy 1, policy_version 9910 (0.0008) [2023-10-14 17:59:32,528][61585] Updated weights for policy 1, policy_version 9920 (0.0007) [2023-10-14 17:59:32,742][61552] Updated weights for policy 0, policy_version 9922 (0.0009) [2023-10-14 17:59:33,111][61552] Updated weights for policy 0, policy_version 9932 (0.0009) [2023-10-14 17:59:33,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 20316160. Throughput: 0: 1661.9, 1: 1672.5. Samples: 5084422. Policy #0 lag: (min: 26.0, avg: 33.7, max: 58.0) [2023-10-14 17:59:33,344][60425] Avg episode reward: [(0, '31.320'), (1, '50.040')] [2023-10-14 17:59:33,484][61552] Updated weights for policy 0, policy_version 9942 (0.0009) [2023-10-14 17:59:33,858][61552] Updated weights for policy 0, policy_version 9952 (0.0009) [2023-10-14 17:59:36,435][61585] Updated weights for policy 1, policy_version 9930 (0.0007) [2023-10-14 17:59:36,804][61585] Updated weights for policy 1, policy_version 9940 (0.0011) [2023-10-14 17:59:37,172][61585] Updated weights for policy 1, policy_version 9950 (0.0010) [2023-10-14 17:59:38,085][61552] Updated weights for policy 0, policy_version 9962 (0.0008) [2023-10-14 17:59:38,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 20381696. Throughput: 0: 1662.3, 1: 1662.9. Samples: 5104534. Policy #0 lag: (min: 31.0, avg: 33.1, max: 61.0) [2023-10-14 17:59:38,344][60425] Avg episode reward: [(0, '29.460'), (1, '48.110')] [2023-10-14 17:59:38,453][61552] Updated weights for policy 0, policy_version 9972 (0.0009) [2023-10-14 17:59:38,821][61552] Updated weights for policy 0, policy_version 9982 (0.0009) [2023-10-14 17:59:41,293][61585] Updated weights for policy 1, policy_version 9960 (0.0008) [2023-10-14 17:59:41,668][61585] Updated weights for policy 1, policy_version 9970 (0.0009) [2023-10-14 17:59:42,036][61585] Updated weights for policy 1, policy_version 9980 (0.0008) [2023-10-14 17:59:42,920][61552] Updated weights for policy 0, policy_version 9992 (0.0007) [2023-10-14 17:59:43,298][61552] Updated weights for policy 0, policy_version 10002 (0.0007) [2023-10-14 17:59:43,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 20447232. Throughput: 0: 1657.8, 1: 1665.5. Samples: 5124366. Policy #0 lag: (min: 31.0, avg: 33.1, max: 61.0) [2023-10-14 17:59:43,344][60425] Avg episode reward: [(0, '30.880'), (1, '49.910')] [2023-10-14 17:59:43,355][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000009984_10223616.pth... [2023-10-14 17:59:43,390][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000008416_8617984.pth [2023-10-14 17:59:43,671][61552] Updated weights for policy 0, policy_version 10012 (0.0007) [2023-10-14 17:59:43,821][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000010016_10256384.pth... [2023-10-14 17:59:43,859][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000008448_8650752.pth [2023-10-14 17:59:46,008][61585] Updated weights for policy 1, policy_version 9990 (0.0008) [2023-10-14 17:59:46,380][61585] Updated weights for policy 1, policy_version 10000 (0.0008) [2023-10-14 17:59:46,754][61585] Updated weights for policy 1, policy_version 10010 (0.0008) [2023-10-14 17:59:47,888][61552] Updated weights for policy 0, policy_version 10022 (0.0009) [2023-10-14 17:59:48,253][61552] Updated weights for policy 0, policy_version 10032 (0.0009) [2023-10-14 17:59:48,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 20512768. Throughput: 0: 1659.4, 1: 1673.4. Samples: 5134758. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:59:48,344][60425] Avg episode reward: [(0, '30.870'), (1, '49.260')] [2023-10-14 17:59:48,627][61552] Updated weights for policy 0, policy_version 10042 (0.0010) [2023-10-14 17:59:50,944][61585] Updated weights for policy 1, policy_version 10020 (0.0008) [2023-10-14 17:59:51,312][61585] Updated weights for policy 1, policy_version 10030 (0.0009) [2023-10-14 17:59:51,669][61585] Updated weights for policy 1, policy_version 10040 (0.0007) [2023-10-14 17:59:52,666][61552] Updated weights for policy 0, policy_version 10052 (0.0008) [2023-10-14 17:59:53,037][61552] Updated weights for policy 0, policy_version 10062 (0.0007) [2023-10-14 17:59:53,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 20578304. Throughput: 0: 1665.0, 1: 1650.4. Samples: 5154238. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 17:59:53,344][60425] Avg episode reward: [(0, '31.760'), (1, '51.780')] [2023-10-14 17:59:53,345][61248] Saving new best policy, reward=51.780! [2023-10-14 17:59:53,397][61552] Updated weights for policy 0, policy_version 10072 (0.0009) [2023-10-14 17:59:55,606][61585] Updated weights for policy 1, policy_version 10050 (0.0007) [2023-10-14 17:59:55,971][61585] Updated weights for policy 1, policy_version 10060 (0.0009) [2023-10-14 17:59:56,340][61585] Updated weights for policy 1, policy_version 10070 (0.0009) [2023-10-14 17:59:56,707][61585] Updated weights for policy 1, policy_version 10080 (0.0007) [2023-10-14 17:59:57,439][61552] Updated weights for policy 0, policy_version 10082 (0.0008) [2023-10-14 17:59:57,801][61552] Updated weights for policy 0, policy_version 10092 (0.0008) [2023-10-14 17:59:58,169][61552] Updated weights for policy 0, policy_version 10102 (0.0009) [2023-10-14 17:59:58,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 20643840. Throughput: 0: 1659.6, 1: 1671.5. Samples: 5174434. Policy #0 lag: (min: 4.0, avg: 15.1, max: 36.0) [2023-10-14 17:59:58,344][60425] Avg episode reward: [(0, '32.630'), (1, '50.960')] [2023-10-14 17:59:58,540][61172] Saving new best policy, reward=32.630! [2023-10-14 17:59:58,540][61552] Updated weights for policy 0, policy_version 10112 (0.0010) [2023-10-14 18:00:00,744][61585] Updated weights for policy 1, policy_version 10090 (0.0009) [2023-10-14 18:00:01,108][61585] Updated weights for policy 1, policy_version 10100 (0.0008) [2023-10-14 18:00:01,471][61585] Updated weights for policy 1, policy_version 10110 (0.0010) [2023-10-14 18:00:02,600][61552] Updated weights for policy 0, policy_version 10122 (0.0010) [2023-10-14 18:00:02,980][61552] Updated weights for policy 0, policy_version 10132 (0.0008) [2023-10-14 18:00:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 20709376. Throughput: 0: 1667.6, 1: 1667.8. Samples: 5184808. Policy #0 lag: (min: 4.0, avg: 15.1, max: 36.0) [2023-10-14 18:00:03,344][60425] Avg episode reward: [(0, '29.720'), (1, '49.620')] [2023-10-14 18:00:03,346][61552] Updated weights for policy 0, policy_version 10142 (0.0008) [2023-10-14 18:00:05,652][61585] Updated weights for policy 1, policy_version 10120 (0.0010) [2023-10-14 18:00:06,031][61585] Updated weights for policy 1, policy_version 10130 (0.0009) [2023-10-14 18:00:06,397][61585] Updated weights for policy 1, policy_version 10140 (0.0008) [2023-10-14 18:00:07,350][61552] Updated weights for policy 0, policy_version 10152 (0.0007) [2023-10-14 18:00:07,712][61552] Updated weights for policy 0, policy_version 10162 (0.0009) [2023-10-14 18:00:08,079][61552] Updated weights for policy 0, policy_version 10172 (0.0008) [2023-10-14 18:00:08,343][60425] Fps is (10 sec: 16384.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 20807680. Throughput: 0: 1677.8, 1: 1653.8. Samples: 5204566. Policy #0 lag: (min: 9.0, avg: 17.2, max: 41.0) [2023-10-14 18:00:08,344][60425] Avg episode reward: [(0, '30.000'), (1, '50.860')] [2023-10-14 18:00:10,761][61585] Updated weights for policy 1, policy_version 10150 (0.0009) [2023-10-14 18:00:11,155][61585] Updated weights for policy 1, policy_version 10160 (0.0008) [2023-10-14 18:00:11,514][61585] Updated weights for policy 1, policy_version 10170 (0.0009) [2023-10-14 18:00:12,237][61552] Updated weights for policy 0, policy_version 10182 (0.0009) [2023-10-14 18:00:12,610][61552] Updated weights for policy 0, policy_version 10192 (0.0010) [2023-10-14 18:00:12,986][61552] Updated weights for policy 0, policy_version 10202 (0.0010) [2023-10-14 18:00:13,343][60425] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 20873216. Throughput: 0: 1660.2, 1: 1675.2. Samples: 5224148. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-14 18:00:13,344][60425] Avg episode reward: [(0, '30.390'), (1, '48.600')] [2023-10-14 18:00:15,539][61585] Updated weights for policy 1, policy_version 10180 (0.0008) [2023-10-14 18:00:15,905][61585] Updated weights for policy 1, policy_version 10190 (0.0010) [2023-10-14 18:00:16,276][61585] Updated weights for policy 1, policy_version 10200 (0.0011) [2023-10-14 18:00:16,890][61552] Updated weights for policy 0, policy_version 10212 (0.0008) [2023-10-14 18:00:17,261][61552] Updated weights for policy 0, policy_version 10222 (0.0007) [2023-10-14 18:00:17,644][61552] Updated weights for policy 0, policy_version 10232 (0.0008) [2023-10-14 18:00:18,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 20938752. Throughput: 0: 1676.4, 1: 1664.3. Samples: 5234752. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-14 18:00:18,344][60425] Avg episode reward: [(0, '30.150'), (1, '49.780')] [2023-10-14 18:00:20,530][61585] Updated weights for policy 1, policy_version 10210 (0.0009) [2023-10-14 18:00:20,910][61585] Updated weights for policy 1, policy_version 10220 (0.0009) [2023-10-14 18:00:21,268][61585] Updated weights for policy 1, policy_version 10230 (0.0008) [2023-10-14 18:00:21,640][61585] Updated weights for policy 1, policy_version 10240 (0.0008) [2023-10-14 18:00:21,862][61552] Updated weights for policy 0, policy_version 10242 (0.0007) [2023-10-14 18:00:22,236][61552] Updated weights for policy 0, policy_version 10252 (0.0009) [2023-10-14 18:00:22,615][61552] Updated weights for policy 0, policy_version 10262 (0.0010) [2023-10-14 18:00:22,974][61552] Updated weights for policy 0, policy_version 10272 (0.0011) [2023-10-14 18:00:23,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 21004288. Throughput: 0: 1675.0, 1: 1657.3. Samples: 5254488. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-14 18:00:23,344][60425] Avg episode reward: [(0, '30.780'), (1, '47.080')] [2023-10-14 18:00:25,723][61585] Updated weights for policy 1, policy_version 10250 (0.0008) [2023-10-14 18:00:26,087][61585] Updated weights for policy 1, policy_version 10260 (0.0007) [2023-10-14 18:00:26,450][61585] Updated weights for policy 1, policy_version 10270 (0.0008) [2023-10-14 18:00:27,164][61552] Updated weights for policy 0, policy_version 10282 (0.0008) [2023-10-14 18:00:27,536][61552] Updated weights for policy 0, policy_version 10292 (0.0008) [2023-10-14 18:00:27,916][61552] Updated weights for policy 0, policy_version 10302 (0.0009) [2023-10-14 18:00:28,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 21069824. Throughput: 0: 1652.8, 1: 1670.2. Samples: 5273900. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-14 18:00:28,344][60425] Avg episode reward: [(0, '30.830'), (1, '48.430')] [2023-10-14 18:00:30,471][61585] Updated weights for policy 1, policy_version 10280 (0.0011) [2023-10-14 18:00:30,831][61585] Updated weights for policy 1, policy_version 10290 (0.0010) [2023-10-14 18:00:31,189][61585] Updated weights for policy 1, policy_version 10300 (0.0010) [2023-10-14 18:00:32,000][61552] Updated weights for policy 0, policy_version 10312 (0.0012) [2023-10-14 18:00:32,372][61552] Updated weights for policy 0, policy_version 10322 (0.0010) [2023-10-14 18:00:32,737][61552] Updated weights for policy 0, policy_version 10332 (0.0007) [2023-10-14 18:00:33,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 21135360. Throughput: 0: 1675.2, 1: 1656.8. Samples: 5284700. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:00:33,344][60425] Avg episode reward: [(0, '32.260'), (1, '46.430')] [2023-10-14 18:00:35,255][61585] Updated weights for policy 1, policy_version 10310 (0.0010) [2023-10-14 18:00:35,629][61585] Updated weights for policy 1, policy_version 10320 (0.0008) [2023-10-14 18:00:36,000][61585] Updated weights for policy 1, policy_version 10330 (0.0010) [2023-10-14 18:00:36,911][61552] Updated weights for policy 0, policy_version 10342 (0.0010) [2023-10-14 18:00:37,287][61552] Updated weights for policy 0, policy_version 10352 (0.0009) [2023-10-14 18:00:37,651][61552] Updated weights for policy 0, policy_version 10362 (0.0008) [2023-10-14 18:00:38,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 21200896. Throughput: 0: 1675.5, 1: 1662.7. Samples: 5304462. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:00:38,345][60425] Avg episode reward: [(0, '30.120'), (1, '51.550')] [2023-10-14 18:00:40,078][61585] Updated weights for policy 1, policy_version 10340 (0.0009) [2023-10-14 18:00:40,434][61585] Updated weights for policy 1, policy_version 10350 (0.0007) [2023-10-14 18:00:40,798][61585] Updated weights for policy 1, policy_version 10360 (0.0009) [2023-10-14 18:00:41,690][61552] Updated weights for policy 0, policy_version 10372 (0.0009) [2023-10-14 18:00:42,064][61552] Updated weights for policy 0, policy_version 10382 (0.0010) [2023-10-14 18:00:42,444][61552] Updated weights for policy 0, policy_version 10392 (0.0007) [2023-10-14 18:00:43,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 21266432. Throughput: 0: 1651.3, 1: 1667.0. Samples: 5323760. Policy #0 lag: (min: 26.0, avg: 30.1, max: 58.0) [2023-10-14 18:00:43,344][60425] Avg episode reward: [(0, '30.220'), (1, '49.040')] [2023-10-14 18:00:45,012][61585] Updated weights for policy 1, policy_version 10370 (0.0009) [2023-10-14 18:00:45,379][61585] Updated weights for policy 1, policy_version 10380 (0.0009) [2023-10-14 18:00:45,746][61585] Updated weights for policy 1, policy_version 10390 (0.0009) [2023-10-14 18:00:46,112][61585] Updated weights for policy 1, policy_version 10400 (0.0011) [2023-10-14 18:00:46,563][61552] Updated weights for policy 0, policy_version 10402 (0.0008) [2023-10-14 18:00:46,942][61552] Updated weights for policy 0, policy_version 10412 (0.0011) [2023-10-14 18:00:47,311][61552] Updated weights for policy 0, policy_version 10422 (0.0011) [2023-10-14 18:00:47,687][61552] Updated weights for policy 0, policy_version 10432 (0.0007) [2023-10-14 18:00:48,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 21331968. Throughput: 0: 1671.5, 1: 1653.5. Samples: 5334432. Policy #0 lag: (min: 26.0, avg: 30.1, max: 58.0) [2023-10-14 18:00:48,344][60425] Avg episode reward: [(0, '32.120'), (1, '47.090')] [2023-10-14 18:00:50,180][61585] Updated weights for policy 1, policy_version 10410 (0.0007) [2023-10-14 18:00:50,546][61585] Updated weights for policy 1, policy_version 10420 (0.0008) [2023-10-14 18:00:50,912][61585] Updated weights for policy 1, policy_version 10430 (0.0010) [2023-10-14 18:00:51,787][61552] Updated weights for policy 0, policy_version 10442 (0.0010) [2023-10-14 18:00:52,161][61552] Updated weights for policy 0, policy_version 10452 (0.0009) [2023-10-14 18:00:52,536][61552] Updated weights for policy 0, policy_version 10462 (0.0007) [2023-10-14 18:00:53,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 21397504. Throughput: 0: 1657.8, 1: 1666.0. Samples: 5354140. Policy #0 lag: (min: 20.0, avg: 23.2, max: 52.0) [2023-10-14 18:00:53,344][60425] Avg episode reward: [(0, '29.390'), (1, '49.010')] [2023-10-14 18:00:54,923][61585] Updated weights for policy 1, policy_version 10440 (0.0007) [2023-10-14 18:00:55,292][61585] Updated weights for policy 1, policy_version 10450 (0.0007) [2023-10-14 18:00:55,662][61585] Updated weights for policy 1, policy_version 10460 (0.0010) [2023-10-14 18:00:56,548][61552] Updated weights for policy 0, policy_version 10472 (0.0010) [2023-10-14 18:00:56,916][61552] Updated weights for policy 0, policy_version 10482 (0.0011) [2023-10-14 18:00:57,295][61552] Updated weights for policy 0, policy_version 10492 (0.0011) [2023-10-14 18:00:58,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 21463040. Throughput: 0: 1656.8, 1: 1674.1. Samples: 5374040. Policy #0 lag: (min: 20.0, avg: 23.2, max: 52.0) [2023-10-14 18:00:58,344][60425] Avg episode reward: [(0, '29.900'), (1, '46.990')] [2023-10-14 18:00:59,811][61585] Updated weights for policy 1, policy_version 10470 (0.0010) [2023-10-14 18:01:00,200][61585] Updated weights for policy 1, policy_version 10480 (0.0008) [2023-10-14 18:01:00,561][61585] Updated weights for policy 1, policy_version 10490 (0.0008) [2023-10-14 18:01:01,461][61552] Updated weights for policy 0, policy_version 10502 (0.0010) [2023-10-14 18:01:01,831][61552] Updated weights for policy 0, policy_version 10512 (0.0007) [2023-10-14 18:01:02,210][61552] Updated weights for policy 0, policy_version 10522 (0.0008) [2023-10-14 18:01:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 21528576. Throughput: 0: 1671.3, 1: 1653.5. Samples: 5384366. Policy #0 lag: (min: 11.0, avg: 19.0, max: 43.0) [2023-10-14 18:01:03,344][60425] Avg episode reward: [(0, '30.740'), (1, '49.760')] [2023-10-14 18:01:04,562][61585] Updated weights for policy 1, policy_version 10500 (0.0007) [2023-10-14 18:01:04,934][61585] Updated weights for policy 1, policy_version 10510 (0.0011) [2023-10-14 18:01:05,303][61585] Updated weights for policy 1, policy_version 10520 (0.0009) [2023-10-14 18:01:06,325][61552] Updated weights for policy 0, policy_version 10532 (0.0009) [2023-10-14 18:01:06,698][61552] Updated weights for policy 0, policy_version 10542 (0.0007) [2023-10-14 18:01:07,066][61552] Updated weights for policy 0, policy_version 10552 (0.0007) [2023-10-14 18:01:08,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 21594112. Throughput: 0: 1659.5, 1: 1667.0. Samples: 5404178. Policy #0 lag: (min: 11.0, avg: 19.0, max: 43.0) [2023-10-14 18:01:08,344][60425] Avg episode reward: [(0, '30.500'), (1, '48.540')] [2023-10-14 18:01:09,491][61585] Updated weights for policy 1, policy_version 10530 (0.0010) [2023-10-14 18:01:09,862][61585] Updated weights for policy 1, policy_version 10540 (0.0008) [2023-10-14 18:01:10,221][61585] Updated weights for policy 1, policy_version 10550 (0.0009) [2023-10-14 18:01:10,588][61585] Updated weights for policy 1, policy_version 10560 (0.0007) [2023-10-14 18:01:11,099][61552] Updated weights for policy 0, policy_version 10562 (0.0008) [2023-10-14 18:01:11,464][61552] Updated weights for policy 0, policy_version 10572 (0.0011) [2023-10-14 18:01:11,846][61552] Updated weights for policy 0, policy_version 10582 (0.0009) [2023-10-14 18:01:12,226][61552] Updated weights for policy 0, policy_version 10592 (0.0008) [2023-10-14 18:01:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 21659648. Throughput: 0: 1667.6, 1: 1672.4. Samples: 5424200. Policy #0 lag: (min: 25.0, avg: 38.9, max: 57.0) [2023-10-14 18:01:13,344][60425] Avg episode reward: [(0, '31.100'), (1, '47.420')] [2023-10-14 18:01:14,760][61585] Updated weights for policy 1, policy_version 10570 (0.0008) [2023-10-14 18:01:15,123][61585] Updated weights for policy 1, policy_version 10580 (0.0007) [2023-10-14 18:01:15,491][61585] Updated weights for policy 1, policy_version 10590 (0.0009) [2023-10-14 18:01:16,265][61552] Updated weights for policy 0, policy_version 10602 (0.0009) [2023-10-14 18:01:16,636][61552] Updated weights for policy 0, policy_version 10612 (0.0008) [2023-10-14 18:01:17,004][61552] Updated weights for policy 0, policy_version 10622 (0.0009) [2023-10-14 18:01:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 21725184. Throughput: 0: 1675.9, 1: 1651.7. Samples: 5434442. Policy #0 lag: (min: 25.0, avg: 38.9, max: 57.0) [2023-10-14 18:01:18,344][60425] Avg episode reward: [(0, '30.320'), (1, '48.060')] [2023-10-14 18:01:19,604][61585] Updated weights for policy 1, policy_version 10600 (0.0008) [2023-10-14 18:01:19,971][61585] Updated weights for policy 1, policy_version 10610 (0.0011) [2023-10-14 18:01:20,336][61585] Updated weights for policy 1, policy_version 10620 (0.0007) [2023-10-14 18:01:21,071][61552] Updated weights for policy 0, policy_version 10632 (0.0009) [2023-10-14 18:01:21,432][61552] Updated weights for policy 0, policy_version 10642 (0.0009) [2023-10-14 18:01:21,808][61552] Updated weights for policy 0, policy_version 10652 (0.0008) [2023-10-14 18:01:23,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 21790720. Throughput: 0: 1654.3, 1: 1672.6. Samples: 5454170. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-14 18:01:23,344][60425] Avg episode reward: [(0, '30.380'), (1, '50.440')] [2023-10-14 18:01:24,521][61585] Updated weights for policy 1, policy_version 10630 (0.0008) [2023-10-14 18:01:24,888][61585] Updated weights for policy 1, policy_version 10640 (0.0010) [2023-10-14 18:01:25,249][61585] Updated weights for policy 1, policy_version 10650 (0.0008) [2023-10-14 18:01:26,059][61552] Updated weights for policy 0, policy_version 10662 (0.0009) [2023-10-14 18:01:26,424][61552] Updated weights for policy 0, policy_version 10672 (0.0010) [2023-10-14 18:01:26,791][61552] Updated weights for policy 0, policy_version 10682 (0.0008) [2023-10-14 18:01:28,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 21856256. Throughput: 0: 1671.6, 1: 1672.2. Samples: 5474232. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-14 18:01:28,344][60425] Avg episode reward: [(0, '30.330'), (1, '49.640')] [2023-10-14 18:01:29,506][61585] Updated weights for policy 1, policy_version 10660 (0.0009) [2023-10-14 18:01:29,876][61585] Updated weights for policy 1, policy_version 10670 (0.0009) [2023-10-14 18:01:30,251][61585] Updated weights for policy 1, policy_version 10680 (0.0009) [2023-10-14 18:01:30,768][61552] Updated weights for policy 0, policy_version 10692 (0.0008) [2023-10-14 18:01:31,137][61552] Updated weights for policy 0, policy_version 10702 (0.0009) [2023-10-14 18:01:31,509][61552] Updated weights for policy 0, policy_version 10712 (0.0010) [2023-10-14 18:01:33,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 21921792. Throughput: 0: 1674.4, 1: 1660.5. Samples: 5484502. Policy #0 lag: (min: 18.0, avg: 45.3, max: 48.0) [2023-10-14 18:01:33,344][60425] Avg episode reward: [(0, '29.410'), (1, '51.760')] [2023-10-14 18:01:34,319][61585] Updated weights for policy 1, policy_version 10690 (0.0010) [2023-10-14 18:01:34,689][61585] Updated weights for policy 1, policy_version 10700 (0.0010) [2023-10-14 18:01:35,058][61585] Updated weights for policy 1, policy_version 10710 (0.0009) [2023-10-14 18:01:35,415][61585] Updated weights for policy 1, policy_version 10720 (0.0007) [2023-10-14 18:01:35,508][61552] Updated weights for policy 0, policy_version 10722 (0.0008) [2023-10-14 18:01:35,880][61552] Updated weights for policy 0, policy_version 10732 (0.0008) [2023-10-14 18:01:36,258][61552] Updated weights for policy 0, policy_version 10742 (0.0010) [2023-10-14 18:01:36,629][61552] Updated weights for policy 0, policy_version 10752 (0.0010) [2023-10-14 18:01:38,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 21987328. Throughput: 0: 1658.0, 1: 1673.3. Samples: 5504046. Policy #0 lag: (min: 18.0, avg: 45.3, max: 48.0) [2023-10-14 18:01:38,344][60425] Avg episode reward: [(0, '29.790'), (1, '51.300')] [2023-10-14 18:01:39,468][61585] Updated weights for policy 1, policy_version 10730 (0.0008) [2023-10-14 18:01:39,833][61585] Updated weights for policy 1, policy_version 10740 (0.0008) [2023-10-14 18:01:40,205][61585] Updated weights for policy 1, policy_version 10750 (0.0011) [2023-10-14 18:01:40,631][61552] Updated weights for policy 0, policy_version 10762 (0.0009) [2023-10-14 18:01:41,002][61552] Updated weights for policy 0, policy_version 10772 (0.0009) [2023-10-14 18:01:41,381][61552] Updated weights for policy 0, policy_version 10782 (0.0009) [2023-10-14 18:01:43,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 22052864. Throughput: 0: 1677.2, 1: 1666.5. Samples: 5524502. Policy #0 lag: (min: 5.0, avg: 9.5, max: 37.0) [2023-10-14 18:01:43,344][60425] Avg episode reward: [(0, '32.090'), (1, '50.860')] [2023-10-14 18:01:43,354][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000010752_11010048.pth... [2023-10-14 18:01:43,354][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000010784_11042816.pth... [2023-10-14 18:01:43,384][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000009216_9437184.pth [2023-10-14 18:01:43,391][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000009216_9437184.pth [2023-10-14 18:01:44,276][61585] Updated weights for policy 1, policy_version 10760 (0.0010) [2023-10-14 18:01:44,636][61585] Updated weights for policy 1, policy_version 10770 (0.0008) [2023-10-14 18:01:45,001][61585] Updated weights for policy 1, policy_version 10780 (0.0011) [2023-10-14 18:01:45,530][61552] Updated weights for policy 0, policy_version 10792 (0.0008) [2023-10-14 18:01:45,911][61552] Updated weights for policy 0, policy_version 10802 (0.0007) [2023-10-14 18:01:46,279][61552] Updated weights for policy 0, policy_version 10812 (0.0007) [2023-10-14 18:01:48,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 22118400. Throughput: 0: 1663.8, 1: 1668.6. Samples: 5534324. Policy #0 lag: (min: 5.0, avg: 9.5, max: 37.0) [2023-10-14 18:01:48,344][60425] Avg episode reward: [(0, '32.100'), (1, '47.120')] [2023-10-14 18:01:49,106][61585] Updated weights for policy 1, policy_version 10790 (0.0007) [2023-10-14 18:01:49,467][61585] Updated weights for policy 1, policy_version 10800 (0.0007) [2023-10-14 18:01:49,835][61585] Updated weights for policy 1, policy_version 10810 (0.0007) [2023-10-14 18:01:50,493][61552] Updated weights for policy 0, policy_version 10822 (0.0007) [2023-10-14 18:01:50,857][61552] Updated weights for policy 0, policy_version 10832 (0.0008) [2023-10-14 18:01:51,219][61552] Updated weights for policy 0, policy_version 10842 (0.0008) [2023-10-14 18:01:53,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 22183936. Throughput: 0: 1655.2, 1: 1677.2. Samples: 5554136. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-14 18:01:53,344][60425] Avg episode reward: [(0, '31.980'), (1, '48.030')] [2023-10-14 18:01:53,663][61585] Updated weights for policy 1, policy_version 10820 (0.0008) [2023-10-14 18:01:54,042][61585] Updated weights for policy 1, policy_version 10830 (0.0010) [2023-10-14 18:01:54,413][61585] Updated weights for policy 1, policy_version 10840 (0.0008) [2023-10-14 18:01:55,280][61552] Updated weights for policy 0, policy_version 10852 (0.0011) [2023-10-14 18:01:55,677][61552] Updated weights for policy 0, policy_version 10862 (0.0010) [2023-10-14 18:01:56,050][61552] Updated weights for policy 0, policy_version 10872 (0.0010) [2023-10-14 18:01:58,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 22249472. Throughput: 0: 1668.3, 1: 1675.5. Samples: 5574672. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-14 18:01:58,345][60425] Avg episode reward: [(0, '31.480'), (1, '49.460')] [2023-10-14 18:01:58,435][61585] Updated weights for policy 1, policy_version 10850 (0.0008) [2023-10-14 18:01:58,807][61585] Updated weights for policy 1, policy_version 10860 (0.0009) [2023-10-14 18:01:59,168][61585] Updated weights for policy 1, policy_version 10870 (0.0009) [2023-10-14 18:01:59,535][61585] Updated weights for policy 1, policy_version 10880 (0.0009) [2023-10-14 18:02:00,131][61552] Updated weights for policy 0, policy_version 10882 (0.0010) [2023-10-14 18:02:00,501][61552] Updated weights for policy 0, policy_version 10892 (0.0009) [2023-10-14 18:02:00,872][61552] Updated weights for policy 0, policy_version 10902 (0.0008) [2023-10-14 18:02:01,240][61552] Updated weights for policy 0, policy_version 10912 (0.0008) [2023-10-14 18:02:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 22315008. Throughput: 0: 1651.5, 1: 1681.2. Samples: 5584414. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-10-14 18:02:03,344][60425] Avg episode reward: [(0, '31.670'), (1, '48.650')] [2023-10-14 18:02:03,750][61585] Updated weights for policy 1, policy_version 10890 (0.0009) [2023-10-14 18:02:04,118][61585] Updated weights for policy 1, policy_version 10900 (0.0010) [2023-10-14 18:02:04,487][61585] Updated weights for policy 1, policy_version 10910 (0.0008) [2023-10-14 18:02:05,389][61552] Updated weights for policy 0, policy_version 10922 (0.0008) [2023-10-14 18:02:05,750][61552] Updated weights for policy 0, policy_version 10932 (0.0009) [2023-10-14 18:02:06,125][61552] Updated weights for policy 0, policy_version 10942 (0.0009) [2023-10-14 18:02:08,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 22380544. Throughput: 0: 1661.5, 1: 1678.4. Samples: 5604466. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-10-14 18:02:08,344][60425] Avg episode reward: [(0, '31.260'), (1, '50.720')] [2023-10-14 18:02:08,592][61585] Updated weights for policy 1, policy_version 10920 (0.0011) [2023-10-14 18:02:08,963][61585] Updated weights for policy 1, policy_version 10930 (0.0008) [2023-10-14 18:02:09,338][61585] Updated weights for policy 1, policy_version 10940 (0.0007) [2023-10-14 18:02:10,158][61552] Updated weights for policy 0, policy_version 10952 (0.0008) [2023-10-14 18:02:10,530][61552] Updated weights for policy 0, policy_version 10962 (0.0008) [2023-10-14 18:02:10,899][61552] Updated weights for policy 0, policy_version 10972 (0.0008) [2023-10-14 18:02:13,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 22446080. Throughput: 0: 1675.8, 1: 1677.5. Samples: 5625130. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:02:13,344][60425] Avg episode reward: [(0, '30.610'), (1, '50.450')] [2023-10-14 18:02:13,416][61585] Updated weights for policy 1, policy_version 10950 (0.0010) [2023-10-14 18:02:13,778][61585] Updated weights for policy 1, policy_version 10960 (0.0008) [2023-10-14 18:02:14,145][61585] Updated weights for policy 1, policy_version 10970 (0.0008) [2023-10-14 18:02:14,838][61552] Updated weights for policy 0, policy_version 10982 (0.0008) [2023-10-14 18:02:15,204][61552] Updated weights for policy 0, policy_version 10992 (0.0010) [2023-10-14 18:02:15,582][61552] Updated weights for policy 0, policy_version 11002 (0.0008) [2023-10-14 18:02:18,197][61585] Updated weights for policy 1, policy_version 10980 (0.0008) [2023-10-14 18:02:18,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 22511616. Throughput: 0: 1652.2, 1: 1683.2. Samples: 5634592. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:02:18,344][60425] Avg episode reward: [(0, '32.350'), (1, '51.690')] [2023-10-14 18:02:18,564][61585] Updated weights for policy 1, policy_version 10990 (0.0008) [2023-10-14 18:02:18,935][61585] Updated weights for policy 1, policy_version 11000 (0.0008) [2023-10-14 18:02:19,732][61552] Updated weights for policy 0, policy_version 11012 (0.0010) [2023-10-14 18:02:20,110][61552] Updated weights for policy 0, policy_version 11022 (0.0010) [2023-10-14 18:02:20,480][61552] Updated weights for policy 0, policy_version 11032 (0.0010) [2023-10-14 18:02:23,152][61585] Updated weights for policy 1, policy_version 11010 (0.0008) [2023-10-14 18:02:23,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 22577152. Throughput: 0: 1668.6, 1: 1681.6. Samples: 5654808. Policy #0 lag: (min: 26.0, avg: 26.6, max: 44.0) [2023-10-14 18:02:23,344][60425] Avg episode reward: [(0, '31.360'), (1, '50.960')] [2023-10-14 18:02:23,523][61585] Updated weights for policy 1, policy_version 11020 (0.0009) [2023-10-14 18:02:23,890][61585] Updated weights for policy 1, policy_version 11030 (0.0010) [2023-10-14 18:02:24,263][61585] Updated weights for policy 1, policy_version 11040 (0.0009) [2023-10-14 18:02:24,566][61552] Updated weights for policy 0, policy_version 11042 (0.0009) [2023-10-14 18:02:24,936][61552] Updated weights for policy 0, policy_version 11052 (0.0008) [2023-10-14 18:02:25,310][61552] Updated weights for policy 0, policy_version 11062 (0.0007) [2023-10-14 18:02:25,667][61552] Updated weights for policy 0, policy_version 11072 (0.0007) [2023-10-14 18:02:28,341][61585] Updated weights for policy 1, policy_version 11050 (0.0008) [2023-10-14 18:02:28,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 22642688. Throughput: 0: 1672.3, 1: 1681.8. Samples: 5675436. Policy #0 lag: (min: 26.0, avg: 26.6, max: 44.0) [2023-10-14 18:02:28,344][60425] Avg episode reward: [(0, '31.270'), (1, '50.920')] [2023-10-14 18:02:28,718][61585] Updated weights for policy 1, policy_version 11060 (0.0010) [2023-10-14 18:02:29,080][61585] Updated weights for policy 1, policy_version 11070 (0.0009) [2023-10-14 18:02:29,654][61552] Updated weights for policy 0, policy_version 11082 (0.0009) [2023-10-14 18:02:30,028][61552] Updated weights for policy 0, policy_version 11092 (0.0009) [2023-10-14 18:02:30,400][61552] Updated weights for policy 0, policy_version 11102 (0.0011) [2023-10-14 18:02:33,172][61585] Updated weights for policy 1, policy_version 11080 (0.0010) [2023-10-14 18:02:33,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 22708224. Throughput: 0: 1657.5, 1: 1679.3. Samples: 5684482. Policy #0 lag: (min: 0.0, avg: 27.4, max: 32.0) [2023-10-14 18:02:33,344][60425] Avg episode reward: [(0, '31.730'), (1, '48.620')] [2023-10-14 18:02:33,541][61585] Updated weights for policy 1, policy_version 11090 (0.0008) [2023-10-14 18:02:33,900][61585] Updated weights for policy 1, policy_version 11100 (0.0008) [2023-10-14 18:02:34,529][61552] Updated weights for policy 0, policy_version 11112 (0.0009) [2023-10-14 18:02:34,890][61552] Updated weights for policy 0, policy_version 11122 (0.0008) [2023-10-14 18:02:35,266][61552] Updated weights for policy 0, policy_version 11132 (0.0008) [2023-10-14 18:02:38,054][61585] Updated weights for policy 1, policy_version 11110 (0.0008) [2023-10-14 18:02:38,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 22773760. Throughput: 0: 1682.4, 1: 1674.9. Samples: 5705216. Policy #0 lag: (min: 0.0, avg: 27.4, max: 32.0) [2023-10-14 18:02:38,344][60425] Avg episode reward: [(0, '30.490'), (1, '49.450')] [2023-10-14 18:02:38,414][61585] Updated weights for policy 1, policy_version 11120 (0.0008) [2023-10-14 18:02:38,782][61585] Updated weights for policy 1, policy_version 11130 (0.0007) [2023-10-14 18:02:39,161][61552] Updated weights for policy 0, policy_version 11142 (0.0009) [2023-10-14 18:02:39,531][61552] Updated weights for policy 0, policy_version 11152 (0.0009) [2023-10-14 18:02:39,909][61552] Updated weights for policy 0, policy_version 11162 (0.0008) [2023-10-14 18:02:42,691][61585] Updated weights for policy 1, policy_version 11140 (0.0007) [2023-10-14 18:02:43,056][61585] Updated weights for policy 1, policy_version 11150 (0.0007) [2023-10-14 18:02:43,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 22839296. Throughput: 0: 1686.9, 1: 1671.0. Samples: 5725778. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 18:02:43,344][60425] Avg episode reward: [(0, '29.770'), (1, '50.020')] [2023-10-14 18:02:43,424][61585] Updated weights for policy 1, policy_version 11160 (0.0010) [2023-10-14 18:02:44,064][61552] Updated weights for policy 0, policy_version 11172 (0.0008) [2023-10-14 18:02:44,448][61552] Updated weights for policy 0, policy_version 11182 (0.0007) [2023-10-14 18:02:44,824][61552] Updated weights for policy 0, policy_version 11192 (0.0007) [2023-10-14 18:02:47,676][61585] Updated weights for policy 1, policy_version 11170 (0.0008) [2023-10-14 18:02:48,042][61585] Updated weights for policy 1, policy_version 11180 (0.0010) [2023-10-14 18:02:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 22904832. Throughput: 0: 1674.0, 1: 1669.3. Samples: 5734864. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 18:02:48,344][60425] Avg episode reward: [(0, '31.810'), (1, '52.460')] [2023-10-14 18:02:48,411][61585] Updated weights for policy 1, policy_version 11190 (0.0009) [2023-10-14 18:02:48,772][61248] Saving new best policy, reward=52.460! [2023-10-14 18:02:48,775][61585] Updated weights for policy 1, policy_version 11200 (0.0009) [2023-10-14 18:02:48,869][61552] Updated weights for policy 0, policy_version 11202 (0.0008) [2023-10-14 18:02:49,235][61552] Updated weights for policy 0, policy_version 11212 (0.0009) [2023-10-14 18:02:49,613][61552] Updated weights for policy 0, policy_version 11222 (0.0007) [2023-10-14 18:02:49,977][61552] Updated weights for policy 0, policy_version 11232 (0.0007) [2023-10-14 18:02:52,905][61585] Updated weights for policy 1, policy_version 11210 (0.0008) [2023-10-14 18:02:53,273][61585] Updated weights for policy 1, policy_version 11220 (0.0009) [2023-10-14 18:02:53,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 22970368. Throughput: 0: 1686.9, 1: 1666.8. Samples: 5755386. Policy #0 lag: (min: 17.0, avg: 22.9, max: 49.0) [2023-10-14 18:02:53,344][60425] Avg episode reward: [(0, '32.490'), (1, '53.630')] [2023-10-14 18:02:53,640][61585] Updated weights for policy 1, policy_version 11230 (0.0008) [2023-10-14 18:02:53,705][61248] Saving new best policy, reward=53.630! [2023-10-14 18:02:54,031][61552] Updated weights for policy 0, policy_version 11242 (0.0008) [2023-10-14 18:02:54,396][61552] Updated weights for policy 0, policy_version 11252 (0.0009) [2023-10-14 18:02:54,772][61552] Updated weights for policy 0, policy_version 11262 (0.0010) [2023-10-14 18:02:57,765][61585] Updated weights for policy 1, policy_version 11240 (0.0009) [2023-10-14 18:02:58,134][61585] Updated weights for policy 1, policy_version 11250 (0.0007) [2023-10-14 18:02:58,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 23035904. Throughput: 0: 1683.1, 1: 1661.8. Samples: 5775650. Policy #0 lag: (min: 17.0, avg: 22.9, max: 49.0) [2023-10-14 18:02:58,344][60425] Avg episode reward: [(0, '30.440'), (1, '53.960')] [2023-10-14 18:02:58,507][61585] Updated weights for policy 1, policy_version 11260 (0.0010) [2023-10-14 18:02:58,657][61248] Saving new best policy, reward=53.960! [2023-10-14 18:02:59,012][61552] Updated weights for policy 0, policy_version 11272 (0.0009) [2023-10-14 18:02:59,383][61552] Updated weights for policy 0, policy_version 11282 (0.0008) [2023-10-14 18:02:59,760][61552] Updated weights for policy 0, policy_version 11292 (0.0008) [2023-10-14 18:03:02,701][61585] Updated weights for policy 1, policy_version 11270 (0.0008) [2023-10-14 18:03:03,075][61585] Updated weights for policy 1, policy_version 11280 (0.0010) [2023-10-14 18:03:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 23101440. Throughput: 0: 1676.9, 1: 1664.9. Samples: 5784976. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:03:03,344][60425] Avg episode reward: [(0, '29.500'), (1, '51.840')] [2023-10-14 18:03:03,434][61585] Updated weights for policy 1, policy_version 11290 (0.0010) [2023-10-14 18:03:03,753][61552] Updated weights for policy 0, policy_version 11302 (0.0008) [2023-10-14 18:03:04,127][61552] Updated weights for policy 0, policy_version 11312 (0.0008) [2023-10-14 18:03:04,501][61552] Updated weights for policy 0, policy_version 11322 (0.0010) [2023-10-14 18:03:07,542][61585] Updated weights for policy 1, policy_version 11300 (0.0007) [2023-10-14 18:03:07,902][61585] Updated weights for policy 1, policy_version 11310 (0.0007) [2023-10-14 18:03:08,274][61585] Updated weights for policy 1, policy_version 11320 (0.0007) [2023-10-14 18:03:08,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 23166976. Throughput: 0: 1684.7, 1: 1663.9. Samples: 5805492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:03:08,344][60425] Avg episode reward: [(0, '33.070'), (1, '51.290')] [2023-10-14 18:03:08,344][61172] Saving new best policy, reward=33.070! [2023-10-14 18:03:08,669][61552] Updated weights for policy 0, policy_version 11332 (0.0008) [2023-10-14 18:03:09,046][61552] Updated weights for policy 0, policy_version 11342 (0.0007) [2023-10-14 18:03:09,414][61552] Updated weights for policy 0, policy_version 11352 (0.0007) [2023-10-14 18:03:12,277][61585] Updated weights for policy 1, policy_version 11330 (0.0008) [2023-10-14 18:03:12,645][61585] Updated weights for policy 1, policy_version 11340 (0.0008) [2023-10-14 18:03:13,011][61585] Updated weights for policy 1, policy_version 11350 (0.0009) [2023-10-14 18:03:13,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 23232512. Throughput: 0: 1681.0, 1: 1656.4. Samples: 5825620. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:03:13,344][60425] Avg episode reward: [(0, '34.590'), (1, '51.180')] [2023-10-14 18:03:13,352][61172] Saving new best policy, reward=34.590! [2023-10-14 18:03:13,367][61585] Updated weights for policy 1, policy_version 11360 (0.0009) [2023-10-14 18:03:13,581][61552] Updated weights for policy 0, policy_version 11362 (0.0009) [2023-10-14 18:03:13,952][61552] Updated weights for policy 0, policy_version 11372 (0.0009) [2023-10-14 18:03:14,318][61552] Updated weights for policy 0, policy_version 11382 (0.0010) [2023-10-14 18:03:14,682][61552] Updated weights for policy 0, policy_version 11392 (0.0008) [2023-10-14 18:03:17,380][61585] Updated weights for policy 1, policy_version 11370 (0.0010) [2023-10-14 18:03:17,748][61585] Updated weights for policy 1, policy_version 11380 (0.0008) [2023-10-14 18:03:18,124][61585] Updated weights for policy 1, policy_version 11390 (0.0008) [2023-10-14 18:03:18,343][60425] Fps is (10 sec: 16384.0, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 23330816. Throughput: 0: 1679.7, 1: 1673.7. Samples: 5835380. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:03:18,344][60425] Avg episode reward: [(0, '32.830'), (1, '48.960')] [2023-10-14 18:03:18,773][61552] Updated weights for policy 0, policy_version 11402 (0.0007) [2023-10-14 18:03:19,150][61552] Updated weights for policy 0, policy_version 11412 (0.0010) [2023-10-14 18:03:19,511][61552] Updated weights for policy 0, policy_version 11422 (0.0007) [2023-10-14 18:03:22,403][61585] Updated weights for policy 1, policy_version 11400 (0.0007) [2023-10-14 18:03:22,781][61585] Updated weights for policy 1, policy_version 11410 (0.0007) [2023-10-14 18:03:23,149][61585] Updated weights for policy 1, policy_version 11420 (0.0008) [2023-10-14 18:03:23,343][60425] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 23396352. Throughput: 0: 1678.8, 1: 1674.0. Samples: 5856092. Policy #0 lag: (min: 24.0, avg: 53.7, max: 56.0) [2023-10-14 18:03:23,344][60425] Avg episode reward: [(0, '34.060'), (1, '51.190')] [2023-10-14 18:03:23,506][61552] Updated weights for policy 0, policy_version 11432 (0.0009) [2023-10-14 18:03:23,878][61552] Updated weights for policy 0, policy_version 11442 (0.0008) [2023-10-14 18:03:24,244][61552] Updated weights for policy 0, policy_version 11452 (0.0008) [2023-10-14 18:03:27,209][61585] Updated weights for policy 1, policy_version 11430 (0.0008) [2023-10-14 18:03:27,575][61585] Updated weights for policy 1, policy_version 11440 (0.0008) [2023-10-14 18:03:27,945][61585] Updated weights for policy 1, policy_version 11450 (0.0008) [2023-10-14 18:03:28,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 23461888. Throughput: 0: 1677.6, 1: 1655.6. Samples: 5875776. Policy #0 lag: (min: 24.0, avg: 53.7, max: 56.0) [2023-10-14 18:03:28,344][60425] Avg episode reward: [(0, '35.950'), (1, '51.080')] [2023-10-14 18:03:28,346][61552] Updated weights for policy 0, policy_version 11462 (0.0008) [2023-10-14 18:03:28,722][61552] Updated weights for policy 0, policy_version 11472 (0.0009) [2023-10-14 18:03:29,089][61552] Updated weights for policy 0, policy_version 11482 (0.0008) [2023-10-14 18:03:29,300][61172] Saving new best policy, reward=35.950! [2023-10-14 18:03:31,867][61585] Updated weights for policy 1, policy_version 11460 (0.0008) [2023-10-14 18:03:32,225][61585] Updated weights for policy 1, policy_version 11470 (0.0010) [2023-10-14 18:03:32,588][61585] Updated weights for policy 1, policy_version 11480 (0.0010) [2023-10-14 18:03:33,284][61552] Updated weights for policy 0, policy_version 11492 (0.0008) [2023-10-14 18:03:33,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 23527424. Throughput: 0: 1675.4, 1: 1676.0. Samples: 5885676. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:03:33,344][60425] Avg episode reward: [(0, '34.500'), (1, '51.510')] [2023-10-14 18:03:33,675][61552] Updated weights for policy 0, policy_version 11502 (0.0007) [2023-10-14 18:03:34,044][61552] Updated weights for policy 0, policy_version 11512 (0.0009) [2023-10-14 18:03:36,722][61585] Updated weights for policy 1, policy_version 11490 (0.0010) [2023-10-14 18:03:37,092][61585] Updated weights for policy 1, policy_version 11500 (0.0009) [2023-10-14 18:03:37,462][61585] Updated weights for policy 1, policy_version 11510 (0.0007) [2023-10-14 18:03:37,819][61585] Updated weights for policy 1, policy_version 11520 (0.0008) [2023-10-14 18:03:38,154][61552] Updated weights for policy 0, policy_version 11522 (0.0008) [2023-10-14 18:03:38,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 23592960. Throughput: 0: 1670.7, 1: 1676.2. Samples: 5905998. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:03:38,344][60425] Avg episode reward: [(0, '35.630'), (1, '51.580')] [2023-10-14 18:03:38,531][61552] Updated weights for policy 0, policy_version 11532 (0.0007) [2023-10-14 18:03:38,903][61552] Updated weights for policy 0, policy_version 11542 (0.0007) [2023-10-14 18:03:39,276][61552] Updated weights for policy 0, policy_version 11552 (0.0007) [2023-10-14 18:03:42,036][61585] Updated weights for policy 1, policy_version 11530 (0.0008) [2023-10-14 18:03:42,398][61585] Updated weights for policy 1, policy_version 11540 (0.0011) [2023-10-14 18:03:42,767][61585] Updated weights for policy 1, policy_version 11550 (0.0010) [2023-10-14 18:03:43,264][61552] Updated weights for policy 0, policy_version 11562 (0.0007) [2023-10-14 18:03:43,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 23658496. Throughput: 0: 1678.0, 1: 1656.4. Samples: 5925700. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-10-14 18:03:43,344][60425] Avg episode reward: [(0, '35.340'), (1, '50.920')] [2023-10-14 18:03:43,352][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000011552_11829248.pth... [2023-10-14 18:03:43,385][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000009984_10223616.pth [2023-10-14 18:03:43,630][61552] Updated weights for policy 0, policy_version 11572 (0.0008) [2023-10-14 18:03:44,007][61552] Updated weights for policy 0, policy_version 11582 (0.0008) [2023-10-14 18:03:44,071][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000011584_11862016.pth... [2023-10-14 18:03:44,100][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000010016_10256384.pth [2023-10-14 18:03:46,747][61585] Updated weights for policy 1, policy_version 11560 (0.0009) [2023-10-14 18:03:47,124][61585] Updated weights for policy 1, policy_version 11570 (0.0008) [2023-10-14 18:03:47,486][61585] Updated weights for policy 1, policy_version 11580 (0.0009) [2023-10-14 18:03:48,100][61552] Updated weights for policy 0, policy_version 11592 (0.0009) [2023-10-14 18:03:48,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 23724032. Throughput: 0: 1677.6, 1: 1673.7. Samples: 5935784. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-10-14 18:03:48,344][60425] Avg episode reward: [(0, '33.890'), (1, '49.780')] [2023-10-14 18:03:48,477][61552] Updated weights for policy 0, policy_version 11602 (0.0009) [2023-10-14 18:03:48,835][61552] Updated weights for policy 0, policy_version 11612 (0.0011) [2023-10-14 18:03:51,582][61585] Updated weights for policy 1, policy_version 11590 (0.0009) [2023-10-14 18:03:51,953][61585] Updated weights for policy 1, policy_version 11600 (0.0008) [2023-10-14 18:03:52,312][61585] Updated weights for policy 1, policy_version 11610 (0.0010) [2023-10-14 18:03:52,945][61552] Updated weights for policy 0, policy_version 11622 (0.0008) [2023-10-14 18:03:53,313][61552] Updated weights for policy 0, policy_version 11632 (0.0010) [2023-10-14 18:03:53,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 23789568. Throughput: 0: 1676.1, 1: 1664.0. Samples: 5955796. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-10-14 18:03:53,344][60425] Avg episode reward: [(0, '35.240'), (1, '49.550')] [2023-10-14 18:03:53,682][61552] Updated weights for policy 0, policy_version 11642 (0.0012) [2023-10-14 18:03:56,421][61585] Updated weights for policy 1, policy_version 11620 (0.0009) [2023-10-14 18:03:56,785][61585] Updated weights for policy 1, policy_version 11630 (0.0009) [2023-10-14 18:03:57,150][61585] Updated weights for policy 1, policy_version 11640 (0.0010) [2023-10-14 18:03:57,628][61552] Updated weights for policy 0, policy_version 11652 (0.0008) [2023-10-14 18:03:57,994][61552] Updated weights for policy 0, policy_version 11662 (0.0008) [2023-10-14 18:03:58,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 23855104. Throughput: 0: 1675.2, 1: 1656.5. Samples: 5975546. Policy #0 lag: (min: 31.0, avg: 43.1, max: 63.0) [2023-10-14 18:03:58,344][60425] Avg episode reward: [(0, '36.690'), (1, '51.820')] [2023-10-14 18:03:58,372][61552] Updated weights for policy 0, policy_version 11672 (0.0009) [2023-10-14 18:03:58,662][61172] Saving new best policy, reward=36.690! [2023-10-14 18:04:01,367][61585] Updated weights for policy 1, policy_version 11650 (0.0009) [2023-10-14 18:04:01,731][61585] Updated weights for policy 1, policy_version 11660 (0.0010) [2023-10-14 18:04:02,103][61585] Updated weights for policy 1, policy_version 11670 (0.0011) [2023-10-14 18:04:02,465][61585] Updated weights for policy 1, policy_version 11680 (0.0009) [2023-10-14 18:04:02,527][61552] Updated weights for policy 0, policy_version 11682 (0.0009) [2023-10-14 18:04:02,892][61552] Updated weights for policy 0, policy_version 11692 (0.0009) [2023-10-14 18:04:03,264][61552] Updated weights for policy 0, policy_version 11702 (0.0008) [2023-10-14 18:04:03,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 23920640. Throughput: 0: 1680.3, 1: 1665.6. Samples: 5985944. Policy #0 lag: (min: 31.0, avg: 43.1, max: 63.0) [2023-10-14 18:04:03,344][60425] Avg episode reward: [(0, '34.870'), (1, '46.970')] [2023-10-14 18:04:03,638][61552] Updated weights for policy 0, policy_version 11712 (0.0010) [2023-10-14 18:04:06,594][61585] Updated weights for policy 1, policy_version 11690 (0.0008) [2023-10-14 18:04:06,969][61585] Updated weights for policy 1, policy_version 11700 (0.0010) [2023-10-14 18:04:07,333][61585] Updated weights for policy 1, policy_version 11710 (0.0010) [2023-10-14 18:04:07,746][61552] Updated weights for policy 0, policy_version 11722 (0.0008) [2023-10-14 18:04:08,121][61552] Updated weights for policy 0, policy_version 11732 (0.0009) [2023-10-14 18:04:08,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 23986176. Throughput: 0: 1678.9, 1: 1649.0. Samples: 6005848. Policy #0 lag: (min: 31.0, avg: 43.1, max: 63.0) [2023-10-14 18:04:08,344][60425] Avg episode reward: [(0, '34.090'), (1, '50.090')] [2023-10-14 18:04:08,490][61552] Updated weights for policy 0, policy_version 11742 (0.0009) [2023-10-14 18:04:11,442][61585] Updated weights for policy 1, policy_version 11720 (0.0008) [2023-10-14 18:04:11,812][61585] Updated weights for policy 1, policy_version 11730 (0.0008) [2023-10-14 18:04:12,173][61585] Updated weights for policy 1, policy_version 11740 (0.0007) [2023-10-14 18:04:12,628][61552] Updated weights for policy 0, policy_version 11752 (0.0009) [2023-10-14 18:04:13,009][61552] Updated weights for policy 0, policy_version 11762 (0.0008) [2023-10-14 18:04:13,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 24051712. Throughput: 0: 1668.9, 1: 1653.2. Samples: 6025272. Policy #0 lag: (min: 31.0, avg: 32.3, max: 55.0) [2023-10-14 18:04:13,344][60425] Avg episode reward: [(0, '34.120'), (1, '51.660')] [2023-10-14 18:04:13,378][61552] Updated weights for policy 0, policy_version 11772 (0.0008) [2023-10-14 18:04:16,499][61585] Updated weights for policy 1, policy_version 11750 (0.0009) [2023-10-14 18:04:16,875][61585] Updated weights for policy 1, policy_version 11760 (0.0008) [2023-10-14 18:04:17,244][61585] Updated weights for policy 1, policy_version 11770 (0.0008) [2023-10-14 18:04:17,447][61552] Updated weights for policy 0, policy_version 11782 (0.0009) [2023-10-14 18:04:17,824][61552] Updated weights for policy 0, policy_version 11792 (0.0010) [2023-10-14 18:04:18,208][61552] Updated weights for policy 0, policy_version 11802 (0.0008) [2023-10-14 18:04:18,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 24117248. Throughput: 0: 1679.9, 1: 1658.0. Samples: 6035884. Policy #0 lag: (min: 31.0, avg: 32.3, max: 55.0) [2023-10-14 18:04:18,344][60425] Avg episode reward: [(0, '32.650'), (1, '49.460')] [2023-10-14 18:04:21,372][61585] Updated weights for policy 1, policy_version 11780 (0.0009) [2023-10-14 18:04:21,742][61585] Updated weights for policy 1, policy_version 11790 (0.0010) [2023-10-14 18:04:22,108][61585] Updated weights for policy 1, policy_version 11800 (0.0007) [2023-10-14 18:04:22,185][61552] Updated weights for policy 0, policy_version 11812 (0.0007) [2023-10-14 18:04:22,550][61552] Updated weights for policy 0, policy_version 11822 (0.0011) [2023-10-14 18:04:22,928][61552] Updated weights for policy 0, policy_version 11832 (0.0010) [2023-10-14 18:04:23,343][60425] Fps is (10 sec: 16384.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 24215552. Throughput: 0: 1684.9, 1: 1646.2. Samples: 6055896. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:04:23,344][60425] Avg episode reward: [(0, '35.050'), (1, '47.350')] [2023-10-14 18:04:26,160][61585] Updated weights for policy 1, policy_version 11810 (0.0007) [2023-10-14 18:04:26,527][61585] Updated weights for policy 1, policy_version 11820 (0.0009) [2023-10-14 18:04:26,889][61585] Updated weights for policy 1, policy_version 11830 (0.0009) [2023-10-14 18:04:26,959][61552] Updated weights for policy 0, policy_version 11842 (0.0007) [2023-10-14 18:04:27,257][61585] Updated weights for policy 1, policy_version 11840 (0.0008) [2023-10-14 18:04:27,341][61552] Updated weights for policy 0, policy_version 11852 (0.0008) [2023-10-14 18:04:27,699][61552] Updated weights for policy 0, policy_version 11862 (0.0010) [2023-10-14 18:04:28,071][61552] Updated weights for policy 0, policy_version 11872 (0.0008) [2023-10-14 18:04:28,343][60425] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 24281088. Throughput: 0: 1657.0, 1: 1657.9. Samples: 6074870. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:04:28,344][60425] Avg episode reward: [(0, '32.160'), (1, '47.120')] [2023-10-14 18:04:31,390][61585] Updated weights for policy 1, policy_version 11850 (0.0010) [2023-10-14 18:04:31,757][61585] Updated weights for policy 1, policy_version 11860 (0.0009) [2023-10-14 18:04:32,117][61585] Updated weights for policy 1, policy_version 11870 (0.0008) [2023-10-14 18:04:32,169][61552] Updated weights for policy 0, policy_version 11882 (0.0008) [2023-10-14 18:04:32,538][61552] Updated weights for policy 0, policy_version 11892 (0.0008) [2023-10-14 18:04:32,919][61552] Updated weights for policy 0, policy_version 11902 (0.0009) [2023-10-14 18:04:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 24346624. Throughput: 0: 1676.8, 1: 1660.3. Samples: 6085952. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:04:33,344][60425] Avg episode reward: [(0, '31.610'), (1, '47.650')] [2023-10-14 18:04:36,230][61585] Updated weights for policy 1, policy_version 11880 (0.0010) [2023-10-14 18:04:36,610][61585] Updated weights for policy 1, policy_version 11890 (0.0010) [2023-10-14 18:04:36,977][61585] Updated weights for policy 1, policy_version 11900 (0.0009) [2023-10-14 18:04:37,013][61552] Updated weights for policy 0, policy_version 11912 (0.0010) [2023-10-14 18:04:37,384][61552] Updated weights for policy 0, policy_version 11922 (0.0011) [2023-10-14 18:04:37,754][61552] Updated weights for policy 0, policy_version 11932 (0.0008) [2023-10-14 18:04:38,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 24412160. Throughput: 0: 1679.6, 1: 1653.2. Samples: 6105776. Policy #0 lag: (min: 1.0, avg: 8.5, max: 33.0) [2023-10-14 18:04:38,344][60425] Avg episode reward: [(0, '35.250'), (1, '47.820')] [2023-10-14 18:04:41,034][61585] Updated weights for policy 1, policy_version 11910 (0.0009) [2023-10-14 18:04:41,396][61585] Updated weights for policy 1, policy_version 11920 (0.0009) [2023-10-14 18:04:41,759][61585] Updated weights for policy 1, policy_version 11930 (0.0009) [2023-10-14 18:04:41,849][61552] Updated weights for policy 0, policy_version 11942 (0.0007) [2023-10-14 18:04:42,220][61552] Updated weights for policy 0, policy_version 11952 (0.0007) [2023-10-14 18:04:42,587][61552] Updated weights for policy 0, policy_version 11962 (0.0008) [2023-10-14 18:04:43,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 24477696. Throughput: 0: 1659.6, 1: 1661.4. Samples: 6124994. Policy #0 lag: (min: 1.0, avg: 8.5, max: 33.0) [2023-10-14 18:04:43,344][60425] Avg episode reward: [(0, '33.540'), (1, '46.500')] [2023-10-14 18:04:45,925][61585] Updated weights for policy 1, policy_version 11940 (0.0009) [2023-10-14 18:04:46,287][61585] Updated weights for policy 1, policy_version 11950 (0.0008) [2023-10-14 18:04:46,497][61552] Updated weights for policy 0, policy_version 11972 (0.0009) [2023-10-14 18:04:46,656][61585] Updated weights for policy 1, policy_version 11960 (0.0009) [2023-10-14 18:04:46,866][61552] Updated weights for policy 0, policy_version 11982 (0.0007) [2023-10-14 18:04:47,233][61552] Updated weights for policy 0, policy_version 11992 (0.0007) [2023-10-14 18:04:48,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 24543232. Throughput: 0: 1684.5, 1: 1663.4. Samples: 6136600. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:04:48,344][60425] Avg episode reward: [(0, '34.410'), (1, '48.890')] [2023-10-14 18:04:51,025][61585] Updated weights for policy 1, policy_version 11970 (0.0007) [2023-10-14 18:04:51,305][61552] Updated weights for policy 0, policy_version 12002 (0.0009) [2023-10-14 18:04:51,388][61585] Updated weights for policy 1, policy_version 11980 (0.0010) [2023-10-14 18:04:51,675][61552] Updated weights for policy 0, policy_version 12012 (0.0007) [2023-10-14 18:04:51,752][61585] Updated weights for policy 1, policy_version 11990 (0.0008) [2023-10-14 18:04:52,037][61552] Updated weights for policy 0, policy_version 12022 (0.0008) [2023-10-14 18:04:52,118][61585] Updated weights for policy 1, policy_version 12000 (0.0008) [2023-10-14 18:04:52,412][61552] Updated weights for policy 0, policy_version 12032 (0.0009) [2023-10-14 18:04:53,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 24608768. Throughput: 0: 1669.2, 1: 1657.3. Samples: 6155540. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:04:53,344][60425] Avg episode reward: [(0, '35.040'), (1, '48.410')] [2023-10-14 18:04:56,163][61585] Updated weights for policy 1, policy_version 12010 (0.0008) [2023-10-14 18:04:56,509][61552] Updated weights for policy 0, policy_version 12042 (0.0009) [2023-10-14 18:04:56,538][61585] Updated weights for policy 1, policy_version 12020 (0.0008) [2023-10-14 18:04:56,890][61552] Updated weights for policy 0, policy_version 12052 (0.0007) [2023-10-14 18:04:56,907][61585] Updated weights for policy 1, policy_version 12030 (0.0007) [2023-10-14 18:04:57,256][61552] Updated weights for policy 0, policy_version 12062 (0.0008) [2023-10-14 18:04:58,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 24674304. Throughput: 0: 1661.5, 1: 1661.8. Samples: 6174818. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:04:58,344][60425] Avg episode reward: [(0, '34.920'), (1, '51.500')] [2023-10-14 18:05:00,867][61585] Updated weights for policy 1, policy_version 12040 (0.0007) [2023-10-14 18:05:01,236][61585] Updated weights for policy 1, policy_version 12050 (0.0009) [2023-10-14 18:05:01,435][61552] Updated weights for policy 0, policy_version 12072 (0.0010) [2023-10-14 18:05:01,603][61585] Updated weights for policy 1, policy_version 12060 (0.0008) [2023-10-14 18:05:01,801][61552] Updated weights for policy 0, policy_version 12082 (0.0009) [2023-10-14 18:05:02,168][61552] Updated weights for policy 0, policy_version 12092 (0.0007) [2023-10-14 18:05:03,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 24739840. Throughput: 0: 1678.1, 1: 1659.7. Samples: 6186086. Policy #0 lag: (min: 22.0, avg: 29.4, max: 54.0) [2023-10-14 18:05:03,344][60425] Avg episode reward: [(0, '36.300'), (1, '50.700')] [2023-10-14 18:05:05,675][61585] Updated weights for policy 1, policy_version 12070 (0.0008) [2023-10-14 18:05:06,043][61585] Updated weights for policy 1, policy_version 12080 (0.0010) [2023-10-14 18:05:06,218][61552] Updated weights for policy 0, policy_version 12102 (0.0009) [2023-10-14 18:05:06,409][61585] Updated weights for policy 1, policy_version 12090 (0.0009) [2023-10-14 18:05:06,580][61552] Updated weights for policy 0, policy_version 12112 (0.0008) [2023-10-14 18:05:06,944][61552] Updated weights for policy 0, policy_version 12122 (0.0011) [2023-10-14 18:05:08,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 24805376. Throughput: 0: 1663.1, 1: 1647.9. Samples: 6204892. Policy #0 lag: (min: 22.0, avg: 29.4, max: 54.0) [2023-10-14 18:05:08,345][60425] Avg episode reward: [(0, '35.640'), (1, '51.480')] [2023-10-14 18:05:10,468][61585] Updated weights for policy 1, policy_version 12100 (0.0008) [2023-10-14 18:05:10,836][61585] Updated weights for policy 1, policy_version 12110 (0.0009) [2023-10-14 18:05:11,197][61552] Updated weights for policy 0, policy_version 12132 (0.0010) [2023-10-14 18:05:11,207][61585] Updated weights for policy 1, policy_version 12120 (0.0008) [2023-10-14 18:05:11,592][61552] Updated weights for policy 0, policy_version 12142 (0.0008) [2023-10-14 18:05:11,958][61552] Updated weights for policy 0, policy_version 12152 (0.0010) [2023-10-14 18:05:13,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 24870912. Throughput: 0: 1668.7, 1: 1666.5. Samples: 6224952. Policy #0 lag: (min: 22.0, avg: 31.0, max: 54.0) [2023-10-14 18:05:13,345][60425] Avg episode reward: [(0, '35.340'), (1, '52.470')] [2023-10-14 18:05:15,352][61585] Updated weights for policy 1, policy_version 12130 (0.0008) [2023-10-14 18:05:15,715][61585] Updated weights for policy 1, policy_version 12140 (0.0007) [2023-10-14 18:05:15,961][61552] Updated weights for policy 0, policy_version 12162 (0.0010) [2023-10-14 18:05:16,080][61585] Updated weights for policy 1, policy_version 12150 (0.0010) [2023-10-14 18:05:16,334][61552] Updated weights for policy 0, policy_version 12172 (0.0008) [2023-10-14 18:05:16,447][61585] Updated weights for policy 1, policy_version 12160 (0.0010) [2023-10-14 18:05:16,704][61552] Updated weights for policy 0, policy_version 12182 (0.0007) [2023-10-14 18:05:17,073][61552] Updated weights for policy 0, policy_version 12192 (0.0007) [2023-10-14 18:05:18,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 24936448. Throughput: 0: 1678.8, 1: 1658.0. Samples: 6236106. Policy #0 lag: (min: 22.0, avg: 31.0, max: 54.0) [2023-10-14 18:05:18,344][60425] Avg episode reward: [(0, '35.480'), (1, '50.680')] [2023-10-14 18:05:20,646][61585] Updated weights for policy 1, policy_version 12170 (0.0009) [2023-10-14 18:05:21,018][61585] Updated weights for policy 1, policy_version 12180 (0.0007) [2023-10-14 18:05:21,183][61552] Updated weights for policy 0, policy_version 12202 (0.0008) [2023-10-14 18:05:21,381][61585] Updated weights for policy 1, policy_version 12190 (0.0007) [2023-10-14 18:05:21,545][61552] Updated weights for policy 0, policy_version 12212 (0.0009) [2023-10-14 18:05:21,916][61552] Updated weights for policy 0, policy_version 12222 (0.0010) [2023-10-14 18:05:23,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 25001984. Throughput: 0: 1657.5, 1: 1654.7. Samples: 6254822. Policy #0 lag: (min: 22.0, avg: 31.0, max: 54.0) [2023-10-14 18:05:23,344][60425] Avg episode reward: [(0, '36.620'), (1, '51.900')] [2023-10-14 18:05:25,572][61585] Updated weights for policy 1, policy_version 12200 (0.0007) [2023-10-14 18:05:25,945][61585] Updated weights for policy 1, policy_version 12210 (0.0008) [2023-10-14 18:05:26,040][61552] Updated weights for policy 0, policy_version 12232 (0.0008) [2023-10-14 18:05:26,304][61585] Updated weights for policy 1, policy_version 12220 (0.0008) [2023-10-14 18:05:26,410][61552] Updated weights for policy 0, policy_version 12242 (0.0009) [2023-10-14 18:05:26,775][61552] Updated weights for policy 0, policy_version 12252 (0.0009) [2023-10-14 18:05:28,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 25067520. Throughput: 0: 1667.9, 1: 1661.6. Samples: 6274824. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) [2023-10-14 18:05:28,344][60425] Avg episode reward: [(0, '38.990'), (1, '49.970')] [2023-10-14 18:05:28,356][61172] Saving new best policy, reward=38.990! [2023-10-14 18:05:30,522][61585] Updated weights for policy 1, policy_version 12230 (0.0008) [2023-10-14 18:05:30,877][61585] Updated weights for policy 1, policy_version 12240 (0.0009) [2023-10-14 18:05:31,018][61552] Updated weights for policy 0, policy_version 12262 (0.0009) [2023-10-14 18:05:31,253][61585] Updated weights for policy 1, policy_version 12250 (0.0009) [2023-10-14 18:05:31,390][61552] Updated weights for policy 0, policy_version 12272 (0.0008) [2023-10-14 18:05:31,752][61552] Updated weights for policy 0, policy_version 12282 (0.0008) [2023-10-14 18:05:33,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 25133056. Throughput: 0: 1661.6, 1: 1649.9. Samples: 6285618. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) [2023-10-14 18:05:33,344][60425] Avg episode reward: [(0, '37.390'), (1, '52.300')] [2023-10-14 18:05:35,299][61585] Updated weights for policy 1, policy_version 12260 (0.0009) [2023-10-14 18:05:35,666][61585] Updated weights for policy 1, policy_version 12270 (0.0012) [2023-10-14 18:05:35,898][61552] Updated weights for policy 0, policy_version 12292 (0.0008) [2023-10-14 18:05:36,036][61585] Updated weights for policy 1, policy_version 12280 (0.0007) [2023-10-14 18:05:36,262][61552] Updated weights for policy 0, policy_version 12302 (0.0009) [2023-10-14 18:05:36,631][61552] Updated weights for policy 0, policy_version 12312 (0.0009) [2023-10-14 18:05:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 25198592. Throughput: 0: 1651.1, 1: 1654.4. Samples: 6304286. Policy #0 lag: (min: 27.0, avg: 32.1, max: 59.0) [2023-10-14 18:05:38,344][60425] Avg episode reward: [(0, '36.620'), (1, '51.310')] [2023-10-14 18:05:40,154][61585] Updated weights for policy 1, policy_version 12290 (0.0007) [2023-10-14 18:05:40,534][61585] Updated weights for policy 1, policy_version 12300 (0.0009) [2023-10-14 18:05:40,759][61552] Updated weights for policy 0, policy_version 12322 (0.0008) [2023-10-14 18:05:40,890][61585] Updated weights for policy 1, policy_version 12310 (0.0008) [2023-10-14 18:05:41,132][61552] Updated weights for policy 0, policy_version 12332 (0.0009) [2023-10-14 18:05:41,259][61585] Updated weights for policy 1, policy_version 12320 (0.0009) [2023-10-14 18:05:41,491][61552] Updated weights for policy 0, policy_version 12342 (0.0009) [2023-10-14 18:05:41,860][61552] Updated weights for policy 0, policy_version 12352 (0.0008) [2023-10-14 18:05:43,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 25264128. Throughput: 0: 1664.5, 1: 1666.2. Samples: 6324702. Policy #0 lag: (min: 27.0, avg: 32.1, max: 59.0) [2023-10-14 18:05:43,345][60425] Avg episode reward: [(0, '40.670'), (1, '52.240')] [2023-10-14 18:05:43,356][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000012352_12648448.pth... [2023-10-14 18:05:43,356][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000012320_12615680.pth... [2023-10-14 18:05:43,387][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000010784_11042816.pth [2023-10-14 18:05:43,391][61172] Saving new best policy, reward=40.670! [2023-10-14 18:05:43,392][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000010752_11010048.pth [2023-10-14 18:05:45,620][61585] Updated weights for policy 1, policy_version 12330 (0.0007) [2023-10-14 18:05:45,722][61552] Updated weights for policy 0, policy_version 12362 (0.0009) [2023-10-14 18:05:45,985][61585] Updated weights for policy 1, policy_version 12340 (0.0007) [2023-10-14 18:05:46,087][61552] Updated weights for policy 0, policy_version 12372 (0.0010) [2023-10-14 18:05:46,349][61585] Updated weights for policy 1, policy_version 12350 (0.0010) [2023-10-14 18:05:46,463][61552] Updated weights for policy 0, policy_version 12382 (0.0009) [2023-10-14 18:05:48,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 25329664. Throughput: 0: 1661.8, 1: 1654.4. Samples: 6335314. Policy #0 lag: (min: 27.0, avg: 32.1, max: 59.0) [2023-10-14 18:05:48,344][60425] Avg episode reward: [(0, '36.420'), (1, '51.210')] [2023-10-14 18:05:50,356][61585] Updated weights for policy 1, policy_version 12360 (0.0008) [2023-10-14 18:05:50,725][61585] Updated weights for policy 1, policy_version 12370 (0.0008) [2023-10-14 18:05:50,798][61552] Updated weights for policy 0, policy_version 12392 (0.0007) [2023-10-14 18:05:51,096][61585] Updated weights for policy 1, policy_version 12380 (0.0009) [2023-10-14 18:05:51,163][61552] Updated weights for policy 0, policy_version 12402 (0.0008) [2023-10-14 18:05:51,536][61552] Updated weights for policy 0, policy_version 12412 (0.0009) [2023-10-14 18:05:53,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 25395200. Throughput: 0: 1652.2, 1: 1663.7. Samples: 6354108. Policy #0 lag: (min: 25.0, avg: 27.9, max: 47.0) [2023-10-14 18:05:53,344][60425] Avg episode reward: [(0, '37.520'), (1, '50.880')] [2023-10-14 18:05:54,986][61585] Updated weights for policy 1, policy_version 12390 (0.0009) [2023-10-14 18:05:55,355][61585] Updated weights for policy 1, policy_version 12400 (0.0007) [2023-10-14 18:05:55,528][61552] Updated weights for policy 0, policy_version 12422 (0.0008) [2023-10-14 18:05:55,721][61585] Updated weights for policy 1, policy_version 12410 (0.0007) [2023-10-14 18:05:55,902][61552] Updated weights for policy 0, policy_version 12432 (0.0007) [2023-10-14 18:05:56,268][61552] Updated weights for policy 0, policy_version 12442 (0.0008) [2023-10-14 18:05:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 25460736. Throughput: 0: 1669.3, 1: 1663.7. Samples: 6374936. Policy #0 lag: (min: 25.0, avg: 27.9, max: 47.0) [2023-10-14 18:05:58,344][60425] Avg episode reward: [(0, '36.540'), (1, '53.190')] [2023-10-14 18:05:59,899][61585] Updated weights for policy 1, policy_version 12420 (0.0008) [2023-10-14 18:06:00,265][61585] Updated weights for policy 1, policy_version 12430 (0.0009) [2023-10-14 18:06:00,398][61552] Updated weights for policy 0, policy_version 12452 (0.0009) [2023-10-14 18:06:00,635][61585] Updated weights for policy 1, policy_version 12440 (0.0007) [2023-10-14 18:06:00,795][61552] Updated weights for policy 0, policy_version 12462 (0.0008) [2023-10-14 18:06:01,171][61552] Updated weights for policy 0, policy_version 12472 (0.0008) [2023-10-14 18:06:03,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 25526272. Throughput: 0: 1653.2, 1: 1652.5. Samples: 6384866. Policy #0 lag: (min: 25.0, avg: 27.9, max: 47.0) [2023-10-14 18:06:03,344][60425] Avg episode reward: [(0, '34.400'), (1, '51.910')] [2023-10-14 18:06:04,858][61585] Updated weights for policy 1, policy_version 12450 (0.0007) [2023-10-14 18:06:05,230][61585] Updated weights for policy 1, policy_version 12460 (0.0009) [2023-10-14 18:06:05,258][61552] Updated weights for policy 0, policy_version 12482 (0.0007) [2023-10-14 18:06:05,587][61585] Updated weights for policy 1, policy_version 12470 (0.0008) [2023-10-14 18:06:05,628][61552] Updated weights for policy 0, policy_version 12492 (0.0008) [2023-10-14 18:06:05,951][61585] Updated weights for policy 1, policy_version 12480 (0.0009) [2023-10-14 18:06:05,988][61552] Updated weights for policy 0, policy_version 12502 (0.0009) [2023-10-14 18:06:06,365][61552] Updated weights for policy 0, policy_version 12512 (0.0009) [2023-10-14 18:06:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 25591808. Throughput: 0: 1654.4, 1: 1661.8. Samples: 6404052. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-14 18:06:08,344][60425] Avg episode reward: [(0, '36.030'), (1, '52.590')] [2023-10-14 18:06:10,130][61585] Updated weights for policy 1, policy_version 12490 (0.0007) [2023-10-14 18:06:10,453][61552] Updated weights for policy 0, policy_version 12522 (0.0009) [2023-10-14 18:06:10,499][61585] Updated weights for policy 1, policy_version 12500 (0.0008) [2023-10-14 18:06:10,811][61552] Updated weights for policy 0, policy_version 12532 (0.0007) [2023-10-14 18:06:10,868][61585] Updated weights for policy 1, policy_version 12510 (0.0007) [2023-10-14 18:06:11,180][61552] Updated weights for policy 0, policy_version 12542 (0.0009) [2023-10-14 18:06:13,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 25657344. Throughput: 0: 1667.5, 1: 1661.9. Samples: 6424644. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-14 18:06:13,344][60425] Avg episode reward: [(0, '40.150'), (1, '52.060')] [2023-10-14 18:06:14,879][61585] Updated weights for policy 1, policy_version 12520 (0.0010) [2023-10-14 18:06:15,246][61585] Updated weights for policy 1, policy_version 12530 (0.0008) [2023-10-14 18:06:15,331][61552] Updated weights for policy 0, policy_version 12552 (0.0008) [2023-10-14 18:06:15,615][61585] Updated weights for policy 1, policy_version 12540 (0.0010) [2023-10-14 18:06:15,689][61552] Updated weights for policy 0, policy_version 12562 (0.0010) [2023-10-14 18:06:16,069][61552] Updated weights for policy 0, policy_version 12572 (0.0009) [2023-10-14 18:06:18,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 25722880. Throughput: 0: 1657.0, 1: 1648.3. Samples: 6434356. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-14 18:06:18,344][60425] Avg episode reward: [(0, '36.010'), (1, '51.420')] [2023-10-14 18:06:19,652][61585] Updated weights for policy 1, policy_version 12550 (0.0009) [2023-10-14 18:06:20,025][61585] Updated weights for policy 1, policy_version 12560 (0.0010) [2023-10-14 18:06:20,162][61552] Updated weights for policy 0, policy_version 12582 (0.0009) [2023-10-14 18:06:20,385][61585] Updated weights for policy 1, policy_version 12570 (0.0008) [2023-10-14 18:06:20,545][61552] Updated weights for policy 0, policy_version 12592 (0.0009) [2023-10-14 18:06:20,908][61552] Updated weights for policy 0, policy_version 12602 (0.0007) [2023-10-14 18:06:23,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 25788416. Throughput: 0: 1666.6, 1: 1664.1. Samples: 6454168. Policy #0 lag: (min: 9.0, avg: 11.8, max: 40.0) [2023-10-14 18:06:23,344][60425] Avg episode reward: [(0, '37.400'), (1, '52.620')] [2023-10-14 18:06:24,585][61585] Updated weights for policy 1, policy_version 12580 (0.0010) [2023-10-14 18:06:24,943][61585] Updated weights for policy 1, policy_version 12590 (0.0010) [2023-10-14 18:06:24,980][61552] Updated weights for policy 0, policy_version 12612 (0.0009) [2023-10-14 18:06:25,313][61585] Updated weights for policy 1, policy_version 12600 (0.0009) [2023-10-14 18:06:25,345][61552] Updated weights for policy 0, policy_version 12622 (0.0007) [2023-10-14 18:06:25,721][61552] Updated weights for policy 0, policy_version 12632 (0.0010) [2023-10-14 18:06:28,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 25853952. Throughput: 0: 1669.0, 1: 1661.9. Samples: 6474590. Policy #0 lag: (min: 9.0, avg: 11.8, max: 40.0) [2023-10-14 18:06:28,344][60425] Avg episode reward: [(0, '39.480'), (1, '50.090')] [2023-10-14 18:06:29,502][61585] Updated weights for policy 1, policy_version 12610 (0.0008) [2023-10-14 18:06:29,817][61552] Updated weights for policy 0, policy_version 12642 (0.0007) [2023-10-14 18:06:29,867][61585] Updated weights for policy 1, policy_version 12620 (0.0007) [2023-10-14 18:06:30,192][61552] Updated weights for policy 0, policy_version 12652 (0.0008) [2023-10-14 18:06:30,235][61585] Updated weights for policy 1, policy_version 12630 (0.0007) [2023-10-14 18:06:30,561][61552] Updated weights for policy 0, policy_version 12662 (0.0009) [2023-10-14 18:06:30,605][61585] Updated weights for policy 1, policy_version 12640 (0.0008) [2023-10-14 18:06:30,936][61552] Updated weights for policy 0, policy_version 12672 (0.0008) [2023-10-14 18:06:33,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 25919488. Throughput: 0: 1652.8, 1: 1649.9. Samples: 6483934. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:06:33,344][60425] Avg episode reward: [(0, '37.500'), (1, '50.540')] [2023-10-14 18:06:34,834][61585] Updated weights for policy 1, policy_version 12650 (0.0007) [2023-10-14 18:06:35,123][61552] Updated weights for policy 0, policy_version 12682 (0.0007) [2023-10-14 18:06:35,207][61585] Updated weights for policy 1, policy_version 12660 (0.0009) [2023-10-14 18:06:35,486][61552] Updated weights for policy 0, policy_version 12692 (0.0007) [2023-10-14 18:06:35,563][61585] Updated weights for policy 1, policy_version 12670 (0.0008) [2023-10-14 18:06:35,868][61552] Updated weights for policy 0, policy_version 12702 (0.0008) [2023-10-14 18:06:38,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 25985024. Throughput: 0: 1663.5, 1: 1659.5. Samples: 6503642. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:06:38,344][60425] Avg episode reward: [(0, '38.740'), (1, '50.540')] [2023-10-14 18:06:39,702][61585] Updated weights for policy 1, policy_version 12680 (0.0009) [2023-10-14 18:06:39,950][61552] Updated weights for policy 0, policy_version 12712 (0.0008) [2023-10-14 18:06:40,070][61585] Updated weights for policy 1, policy_version 12690 (0.0008) [2023-10-14 18:06:40,318][61552] Updated weights for policy 0, policy_version 12722 (0.0008) [2023-10-14 18:06:40,434][61585] Updated weights for policy 1, policy_version 12700 (0.0010) [2023-10-14 18:06:40,689][61552] Updated weights for policy 0, policy_version 12732 (0.0009) [2023-10-14 18:06:43,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 26050560. Throughput: 0: 1661.2, 1: 1655.7. Samples: 6524198. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:06:43,345][60425] Avg episode reward: [(0, '37.930'), (1, '50.730')] [2023-10-14 18:06:44,473][61585] Updated weights for policy 1, policy_version 12710 (0.0009) [2023-10-14 18:06:44,844][61585] Updated weights for policy 1, policy_version 12720 (0.0008) [2023-10-14 18:06:44,886][61552] Updated weights for policy 0, policy_version 12742 (0.0008) [2023-10-14 18:06:45,219][61585] Updated weights for policy 1, policy_version 12730 (0.0008) [2023-10-14 18:06:45,267][61552] Updated weights for policy 0, policy_version 12752 (0.0007) [2023-10-14 18:06:45,642][61552] Updated weights for policy 0, policy_version 12762 (0.0008) [2023-10-14 18:06:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 26116096. Throughput: 0: 1651.0, 1: 1649.8. Samples: 6533402. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:06:48,344][60425] Avg episode reward: [(0, '36.480'), (1, '51.020')] [2023-10-14 18:06:49,391][61585] Updated weights for policy 1, policy_version 12740 (0.0008) [2023-10-14 18:06:49,747][61585] Updated weights for policy 1, policy_version 12750 (0.0008) [2023-10-14 18:06:49,761][61552] Updated weights for policy 0, policy_version 12772 (0.0008) [2023-10-14 18:06:50,118][61552] Updated weights for policy 0, policy_version 12782 (0.0009) [2023-10-14 18:06:50,119][61585] Updated weights for policy 1, policy_version 12760 (0.0008) [2023-10-14 18:06:50,487][61552] Updated weights for policy 0, policy_version 12792 (0.0008) [2023-10-14 18:06:53,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 26181632. Throughput: 0: 1669.1, 1: 1659.0. Samples: 6553814. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:06:53,344][60425] Avg episode reward: [(0, '39.380'), (1, '50.390')] [2023-10-14 18:06:54,246][61585] Updated weights for policy 1, policy_version 12770 (0.0009) [2023-10-14 18:06:54,357][61552] Updated weights for policy 0, policy_version 12802 (0.0008) [2023-10-14 18:06:54,616][61585] Updated weights for policy 1, policy_version 12780 (0.0009) [2023-10-14 18:06:54,720][61552] Updated weights for policy 0, policy_version 12812 (0.0008) [2023-10-14 18:06:54,978][61585] Updated weights for policy 1, policy_version 12790 (0.0009) [2023-10-14 18:06:55,091][61552] Updated weights for policy 0, policy_version 12822 (0.0008) [2023-10-14 18:06:55,340][61585] Updated weights for policy 1, policy_version 12800 (0.0007) [2023-10-14 18:06:55,459][61552] Updated weights for policy 0, policy_version 12832 (0.0007) [2023-10-14 18:06:58,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 26247168. Throughput: 0: 1668.9, 1: 1663.6. Samples: 6574608. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:06:58,344][60425] Avg episode reward: [(0, '38.100'), (1, '49.170')] [2023-10-14 18:06:59,595][61552] Updated weights for policy 0, policy_version 12842 (0.0009) [2023-10-14 18:06:59,628][61585] Updated weights for policy 1, policy_version 12810 (0.0010) [2023-10-14 18:06:59,968][61552] Updated weights for policy 0, policy_version 12852 (0.0007) [2023-10-14 18:06:59,991][61585] Updated weights for policy 1, policy_version 12820 (0.0008) [2023-10-14 18:07:00,341][61552] Updated weights for policy 0, policy_version 12862 (0.0009) [2023-10-14 18:07:00,371][61585] Updated weights for policy 1, policy_version 12830 (0.0007) [2023-10-14 18:07:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 26312704. Throughput: 0: 1656.7, 1: 1658.8. Samples: 6583554. Policy #0 lag: (min: 24.0, avg: 50.1, max: 56.0) [2023-10-14 18:07:03,344][60425] Avg episode reward: [(0, '37.780'), (1, '49.520')] [2023-10-14 18:07:04,370][61585] Updated weights for policy 1, policy_version 12840 (0.0009) [2023-10-14 18:07:04,404][61552] Updated weights for policy 0, policy_version 12872 (0.0008) [2023-10-14 18:07:04,731][61585] Updated weights for policy 1, policy_version 12850 (0.0008) [2023-10-14 18:07:04,784][61552] Updated weights for policy 0, policy_version 12882 (0.0009) [2023-10-14 18:07:05,092][61585] Updated weights for policy 1, policy_version 12860 (0.0008) [2023-10-14 18:07:05,158][61552] Updated weights for policy 0, policy_version 12892 (0.0009) [2023-10-14 18:07:08,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 26378240. Throughput: 0: 1674.7, 1: 1659.2. Samples: 6604190. Policy #0 lag: (min: 24.0, avg: 50.1, max: 56.0) [2023-10-14 18:07:08,344][60425] Avg episode reward: [(0, '36.450'), (1, '51.460')] [2023-10-14 18:07:09,233][61585] Updated weights for policy 1, policy_version 12870 (0.0008) [2023-10-14 18:07:09,382][61552] Updated weights for policy 0, policy_version 12902 (0.0009) [2023-10-14 18:07:09,605][61585] Updated weights for policy 1, policy_version 12880 (0.0007) [2023-10-14 18:07:09,751][61552] Updated weights for policy 0, policy_version 12912 (0.0008) [2023-10-14 18:07:09,974][61585] Updated weights for policy 1, policy_version 12890 (0.0007) [2023-10-14 18:07:10,122][61552] Updated weights for policy 0, policy_version 12922 (0.0008) [2023-10-14 18:07:13,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 26443776. Throughput: 0: 1674.7, 1: 1661.3. Samples: 6624710. Policy #0 lag: (min: 24.0, avg: 50.1, max: 56.0) [2023-10-14 18:07:13,344][60425] Avg episode reward: [(0, '35.540'), (1, '51.720')] [2023-10-14 18:07:13,970][61552] Updated weights for policy 0, policy_version 12932 (0.0008) [2023-10-14 18:07:14,152][61585] Updated weights for policy 1, policy_version 12900 (0.0009) [2023-10-14 18:07:14,351][61552] Updated weights for policy 0, policy_version 12942 (0.0008) [2023-10-14 18:07:14,530][61585] Updated weights for policy 1, policy_version 12910 (0.0009) [2023-10-14 18:07:14,723][61552] Updated weights for policy 0, policy_version 12952 (0.0008) [2023-10-14 18:07:14,892][61585] Updated weights for policy 1, policy_version 12920 (0.0008) [2023-10-14 18:07:18,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 26509312. Throughput: 0: 1670.3, 1: 1659.4. Samples: 6633772. Policy #0 lag: (min: 1.0, avg: 7.0, max: 33.0) [2023-10-14 18:07:18,344][60425] Avg episode reward: [(0, '37.320'), (1, '51.940')] [2023-10-14 18:07:18,749][61552] Updated weights for policy 0, policy_version 12962 (0.0009) [2023-10-14 18:07:19,125][61552] Updated weights for policy 0, policy_version 12972 (0.0008) [2023-10-14 18:07:19,262][61585] Updated weights for policy 1, policy_version 12930 (0.0009) [2023-10-14 18:07:19,498][61552] Updated weights for policy 0, policy_version 12982 (0.0007) [2023-10-14 18:07:19,659][61585] Updated weights for policy 1, policy_version 12940 (0.0008) [2023-10-14 18:07:19,861][61552] Updated weights for policy 0, policy_version 12992 (0.0009) [2023-10-14 18:07:20,038][61585] Updated weights for policy 1, policy_version 12950 (0.0009) [2023-10-14 18:07:20,400][61585] Updated weights for policy 1, policy_version 12960 (0.0009) [2023-10-14 18:07:23,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 26574848. Throughput: 0: 1682.0, 1: 1660.4. Samples: 6654052. Policy #0 lag: (min: 1.0, avg: 7.0, max: 33.0) [2023-10-14 18:07:23,344][60425] Avg episode reward: [(0, '37.950'), (1, '55.180')] [2023-10-14 18:07:23,345][61248] Saving new best policy, reward=55.180! [2023-10-14 18:07:23,931][61552] Updated weights for policy 0, policy_version 13002 (0.0009) [2023-10-14 18:07:24,305][61552] Updated weights for policy 0, policy_version 13012 (0.0009) [2023-10-14 18:07:24,545][61585] Updated weights for policy 1, policy_version 12970 (0.0008) [2023-10-14 18:07:24,666][61552] Updated weights for policy 0, policy_version 13022 (0.0007) [2023-10-14 18:07:24,905][61585] Updated weights for policy 1, policy_version 12980 (0.0008) [2023-10-14 18:07:25,280][61585] Updated weights for policy 1, policy_version 12990 (0.0008) [2023-10-14 18:07:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 26640384. Throughput: 0: 1684.7, 1: 1661.1. Samples: 6674758. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-14 18:07:28,344][60425] Avg episode reward: [(0, '36.320'), (1, '51.160')] [2023-10-14 18:07:28,584][61552] Updated weights for policy 0, policy_version 13032 (0.0009) [2023-10-14 18:07:28,967][61552] Updated weights for policy 0, policy_version 13042 (0.0009) [2023-10-14 18:07:29,339][61552] Updated weights for policy 0, policy_version 13052 (0.0009) [2023-10-14 18:07:29,417][61585] Updated weights for policy 1, policy_version 13000 (0.0008) [2023-10-14 18:07:29,783][61585] Updated weights for policy 1, policy_version 13010 (0.0008) [2023-10-14 18:07:30,149][61585] Updated weights for policy 1, policy_version 13020 (0.0008) [2023-10-14 18:07:33,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 26705920. Throughput: 0: 1684.3, 1: 1662.4. Samples: 6684004. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-14 18:07:33,344][60425] Avg episode reward: [(0, '38.440'), (1, '54.640')] [2023-10-14 18:07:33,497][61552] Updated weights for policy 0, policy_version 13062 (0.0008) [2023-10-14 18:07:33,883][61552] Updated weights for policy 0, policy_version 13072 (0.0007) [2023-10-14 18:07:34,243][61552] Updated weights for policy 0, policy_version 13082 (0.0007) [2023-10-14 18:07:34,287][61585] Updated weights for policy 1, policy_version 13030 (0.0008) [2023-10-14 18:07:34,649][61585] Updated weights for policy 1, policy_version 13040 (0.0009) [2023-10-14 18:07:35,017][61585] Updated weights for policy 1, policy_version 13050 (0.0011) [2023-10-14 18:07:38,344][60425] Fps is (10 sec: 13106.7, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 26771456. Throughput: 0: 1683.4, 1: 1656.3. Samples: 6704098. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-14 18:07:38,345][60425] Avg episode reward: [(0, '39.040'), (1, '50.610')] [2023-10-14 18:07:38,378][61552] Updated weights for policy 0, policy_version 13092 (0.0007) [2023-10-14 18:07:38,744][61552] Updated weights for policy 0, policy_version 13102 (0.0007) [2023-10-14 18:07:39,108][61585] Updated weights for policy 1, policy_version 13060 (0.0008) [2023-10-14 18:07:39,126][61552] Updated weights for policy 0, policy_version 13112 (0.0009) [2023-10-14 18:07:39,481][61585] Updated weights for policy 1, policy_version 13070 (0.0007) [2023-10-14 18:07:39,838][61585] Updated weights for policy 1, policy_version 13080 (0.0008) [2023-10-14 18:07:43,317][61552] Updated weights for policy 0, policy_version 13122 (0.0007) [2023-10-14 18:07:43,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 26836992. Throughput: 0: 1679.3, 1: 1653.7. Samples: 6724594. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:07:43,344][60425] Avg episode reward: [(0, '39.480'), (1, '51.840')] [2023-10-14 18:07:43,353][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000013088_13402112.pth... [2023-10-14 18:07:43,388][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000011552_11829248.pth [2023-10-14 18:07:43,679][61552] Updated weights for policy 0, policy_version 13132 (0.0007) [2023-10-14 18:07:43,959][61585] Updated weights for policy 1, policy_version 13090 (0.0009) [2023-10-14 18:07:44,049][61552] Updated weights for policy 0, policy_version 13142 (0.0008) [2023-10-14 18:07:44,323][61585] Updated weights for policy 1, policy_version 13100 (0.0009) [2023-10-14 18:07:44,421][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000013152_13467648.pth... [2023-10-14 18:07:44,425][61552] Updated weights for policy 0, policy_version 13152 (0.0007) [2023-10-14 18:07:44,450][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000011584_11862016.pth [2023-10-14 18:07:44,695][61585] Updated weights for policy 1, policy_version 13110 (0.0008) [2023-10-14 18:07:45,069][61585] Updated weights for policy 1, policy_version 13120 (0.0007) [2023-10-14 18:07:48,343][60425] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 26902528. Throughput: 0: 1678.4, 1: 1654.4. Samples: 6733534. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:07:48,344][60425] Avg episode reward: [(0, '40.300'), (1, '50.920')] [2023-10-14 18:07:48,627][61552] Updated weights for policy 0, policy_version 13162 (0.0009) [2023-10-14 18:07:48,989][61552] Updated weights for policy 0, policy_version 13172 (0.0008) [2023-10-14 18:07:49,167][61585] Updated weights for policy 1, policy_version 13130 (0.0008) [2023-10-14 18:07:49,360][61552] Updated weights for policy 0, policy_version 13182 (0.0009) [2023-10-14 18:07:49,544][61585] Updated weights for policy 1, policy_version 13140 (0.0008) [2023-10-14 18:07:49,913][61585] Updated weights for policy 1, policy_version 13150 (0.0010) [2023-10-14 18:07:53,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 26968064. Throughput: 0: 1670.5, 1: 1652.1. Samples: 6753708. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:07:53,345][60425] Avg episode reward: [(0, '38.290'), (1, '50.120')] [2023-10-14 18:07:53,491][61552] Updated weights for policy 0, policy_version 13192 (0.0010) [2023-10-14 18:07:53,858][61552] Updated weights for policy 0, policy_version 13202 (0.0010) [2023-10-14 18:07:54,025][61585] Updated weights for policy 1, policy_version 13160 (0.0009) [2023-10-14 18:07:54,228][61552] Updated weights for policy 0, policy_version 13212 (0.0008) [2023-10-14 18:07:54,399][61585] Updated weights for policy 1, policy_version 13170 (0.0009) [2023-10-14 18:07:54,764][61585] Updated weights for policy 1, policy_version 13180 (0.0011) [2023-10-14 18:07:58,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 27033600. Throughput: 0: 1668.7, 1: 1652.5. Samples: 6774164. Policy #0 lag: (min: 11.0, avg: 11.0, max: 16.0) [2023-10-14 18:07:58,344][60425] Avg episode reward: [(0, '37.570'), (1, '52.020')] [2023-10-14 18:07:58,505][61552] Updated weights for policy 0, policy_version 13222 (0.0008) [2023-10-14 18:07:58,812][61585] Updated weights for policy 1, policy_version 13190 (0.0010) [2023-10-14 18:07:58,877][61552] Updated weights for policy 0, policy_version 13232 (0.0007) [2023-10-14 18:07:59,186][61585] Updated weights for policy 1, policy_version 13200 (0.0009) [2023-10-14 18:07:59,242][61552] Updated weights for policy 0, policy_version 13242 (0.0008) [2023-10-14 18:07:59,551][61585] Updated weights for policy 1, policy_version 13210 (0.0010) [2023-10-14 18:08:03,338][61552] Updated weights for policy 0, policy_version 13252 (0.0007) [2023-10-14 18:08:03,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 27099136. Throughput: 0: 1668.4, 1: 1653.3. Samples: 6783248. Policy #0 lag: (min: 11.0, avg: 11.0, max: 16.0) [2023-10-14 18:08:03,344][60425] Avg episode reward: [(0, '38.200'), (1, '52.620')] [2023-10-14 18:08:03,705][61585] Updated weights for policy 1, policy_version 13220 (0.0010) [2023-10-14 18:08:03,713][61552] Updated weights for policy 0, policy_version 13262 (0.0008) [2023-10-14 18:08:04,069][61585] Updated weights for policy 1, policy_version 13230 (0.0009) [2023-10-14 18:08:04,086][61552] Updated weights for policy 0, policy_version 13272 (0.0008) [2023-10-14 18:08:04,434][61585] Updated weights for policy 1, policy_version 13240 (0.0008) [2023-10-14 18:08:08,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 27164672. Throughput: 0: 1663.8, 1: 1658.0. Samples: 6803534. Policy #0 lag: (min: 11.0, avg: 11.0, max: 16.0) [2023-10-14 18:08:08,345][60425] Avg episode reward: [(0, '39.000'), (1, '55.470')] [2023-10-14 18:08:08,346][61248] Saving new best policy, reward=55.470! [2023-10-14 18:08:08,363][61552] Updated weights for policy 0, policy_version 13282 (0.0009) [2023-10-14 18:08:08,634][61585] Updated weights for policy 1, policy_version 13250 (0.0009) [2023-10-14 18:08:08,735][61552] Updated weights for policy 0, policy_version 13292 (0.0007) [2023-10-14 18:08:09,047][61585] Updated weights for policy 1, policy_version 13260 (0.0008) [2023-10-14 18:08:09,099][61552] Updated weights for policy 0, policy_version 13302 (0.0007) [2023-10-14 18:08:09,417][61585] Updated weights for policy 1, policy_version 13270 (0.0008) [2023-10-14 18:08:09,472][61552] Updated weights for policy 0, policy_version 13312 (0.0010) [2023-10-14 18:08:09,778][61585] Updated weights for policy 1, policy_version 13280 (0.0010) [2023-10-14 18:08:13,343][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 27230208. Throughput: 0: 1660.8, 1: 1652.4. Samples: 6823854. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:08:13,345][60425] Avg episode reward: [(0, '37.960'), (1, '51.450')] [2023-10-14 18:08:13,635][61552] Updated weights for policy 0, policy_version 13322 (0.0008) [2023-10-14 18:08:13,836][61585] Updated weights for policy 1, policy_version 13290 (0.0007) [2023-10-14 18:08:14,001][61552] Updated weights for policy 0, policy_version 13332 (0.0010) [2023-10-14 18:08:14,213][61585] Updated weights for policy 1, policy_version 13300 (0.0008) [2023-10-14 18:08:14,372][61552] Updated weights for policy 0, policy_version 13342 (0.0008) [2023-10-14 18:08:14,567][61585] Updated weights for policy 1, policy_version 13310 (0.0009) [2023-10-14 18:08:18,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 27295744. Throughput: 0: 1654.2, 1: 1649.3. Samples: 6832662. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:08:18,344][60425] Avg episode reward: [(0, '37.460'), (1, '54.680')] [2023-10-14 18:08:18,558][61552] Updated weights for policy 0, policy_version 13352 (0.0007) [2023-10-14 18:08:18,749][61585] Updated weights for policy 1, policy_version 13320 (0.0009) [2023-10-14 18:08:18,928][61552] Updated weights for policy 0, policy_version 13362 (0.0007) [2023-10-14 18:08:19,110][61585] Updated weights for policy 1, policy_version 13330 (0.0008) [2023-10-14 18:08:19,297][61552] Updated weights for policy 0, policy_version 13372 (0.0010) [2023-10-14 18:08:19,470][61585] Updated weights for policy 1, policy_version 13340 (0.0009) [2023-10-14 18:08:23,287][61552] Updated weights for policy 0, policy_version 13382 (0.0008) [2023-10-14 18:08:23,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 27361280. Throughput: 0: 1655.3, 1: 1658.1. Samples: 6853200. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:08:23,344][60425] Avg episode reward: [(0, '38.130'), (1, '53.070')] [2023-10-14 18:08:23,505][61585] Updated weights for policy 1, policy_version 13350 (0.0009) [2023-10-14 18:08:23,655][61552] Updated weights for policy 0, policy_version 13392 (0.0009) [2023-10-14 18:08:23,865][61585] Updated weights for policy 1, policy_version 13360 (0.0008) [2023-10-14 18:08:24,028][61552] Updated weights for policy 0, policy_version 13402 (0.0008) [2023-10-14 18:08:24,235][61585] Updated weights for policy 1, policy_version 13370 (0.0008) [2023-10-14 18:08:28,179][61552] Updated weights for policy 0, policy_version 13412 (0.0009) [2023-10-14 18:08:28,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 27426816. Throughput: 0: 1656.0, 1: 1652.7. Samples: 6873486. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-14 18:08:28,344][60425] Avg episode reward: [(0, '40.060'), (1, '53.990')] [2023-10-14 18:08:28,479][61585] Updated weights for policy 1, policy_version 13380 (0.0009) [2023-10-14 18:08:28,544][61552] Updated weights for policy 0, policy_version 13422 (0.0009) [2023-10-14 18:08:28,840][61585] Updated weights for policy 1, policy_version 13390 (0.0007) [2023-10-14 18:08:28,909][61552] Updated weights for policy 0, policy_version 13432 (0.0008) [2023-10-14 18:08:29,217][61585] Updated weights for policy 1, policy_version 13400 (0.0009) [2023-10-14 18:08:32,974][61552] Updated weights for policy 0, policy_version 13442 (0.0008) [2023-10-14 18:08:33,335][61552] Updated weights for policy 0, policy_version 13452 (0.0008) [2023-10-14 18:08:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 27492352. Throughput: 0: 1656.8, 1: 1654.5. Samples: 6882540. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-14 18:08:33,344][60425] Avg episode reward: [(0, '37.870'), (1, '55.210')] [2023-10-14 18:08:33,419][61585] Updated weights for policy 1, policy_version 13410 (0.0008) [2023-10-14 18:08:33,704][61552] Updated weights for policy 0, policy_version 13462 (0.0008) [2023-10-14 18:08:33,778][61585] Updated weights for policy 1, policy_version 13420 (0.0009) [2023-10-14 18:08:34,072][61552] Updated weights for policy 0, policy_version 13472 (0.0007) [2023-10-14 18:08:34,144][61585] Updated weights for policy 1, policy_version 13430 (0.0008) [2023-10-14 18:08:34,512][61585] Updated weights for policy 1, policy_version 13440 (0.0009) [2023-10-14 18:08:38,074][61552] Updated weights for policy 0, policy_version 13482 (0.0007) [2023-10-14 18:08:38,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 27557888. Throughput: 0: 1661.9, 1: 1655.0. Samples: 6902968. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-14 18:08:38,344][60425] Avg episode reward: [(0, '38.410'), (1, '55.800')] [2023-10-14 18:08:38,446][61552] Updated weights for policy 0, policy_version 13492 (0.0008) [2023-10-14 18:08:38,601][61585] Updated weights for policy 1, policy_version 13450 (0.0008) [2023-10-14 18:08:38,810][61552] Updated weights for policy 0, policy_version 13502 (0.0007) [2023-10-14 18:08:38,964][61585] Updated weights for policy 1, policy_version 13460 (0.0009) [2023-10-14 18:08:39,324][61585] Updated weights for policy 1, policy_version 13470 (0.0012) [2023-10-14 18:08:39,398][61248] Saving new best policy, reward=55.800! [2023-10-14 18:08:42,935][61552] Updated weights for policy 0, policy_version 13512 (0.0008) [2023-10-14 18:08:43,309][61552] Updated weights for policy 0, policy_version 13522 (0.0009) [2023-10-14 18:08:43,344][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 27623424. Throughput: 0: 1661.7, 1: 1647.8. Samples: 6923090. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:08:43,345][60425] Avg episode reward: [(0, '38.300'), (1, '53.070')] [2023-10-14 18:08:43,645][61585] Updated weights for policy 1, policy_version 13480 (0.0008) [2023-10-14 18:08:43,685][61552] Updated weights for policy 0, policy_version 13532 (0.0009) [2023-10-14 18:08:44,010][61585] Updated weights for policy 1, policy_version 13490 (0.0007) [2023-10-14 18:08:44,378][61585] Updated weights for policy 1, policy_version 13500 (0.0007) [2023-10-14 18:08:47,737][61552] Updated weights for policy 0, policy_version 13542 (0.0009) [2023-10-14 18:08:48,103][61552] Updated weights for policy 0, policy_version 13552 (0.0008) [2023-10-14 18:08:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 27688960. Throughput: 0: 1661.4, 1: 1648.3. Samples: 6932186. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:08:48,344][60425] Avg episode reward: [(0, '39.660'), (1, '51.750')] [2023-10-14 18:08:48,466][61552] Updated weights for policy 0, policy_version 13562 (0.0009) [2023-10-14 18:08:48,558][61585] Updated weights for policy 1, policy_version 13510 (0.0009) [2023-10-14 18:08:48,917][61585] Updated weights for policy 1, policy_version 13520 (0.0010) [2023-10-14 18:08:49,285][61585] Updated weights for policy 1, policy_version 13530 (0.0008) [2023-10-14 18:08:52,598][61552] Updated weights for policy 0, policy_version 13572 (0.0008) [2023-10-14 18:08:52,964][61552] Updated weights for policy 0, policy_version 13582 (0.0007) [2023-10-14 18:08:53,332][61552] Updated weights for policy 0, policy_version 13592 (0.0009) [2023-10-14 18:08:53,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 27754496. Throughput: 0: 1665.0, 1: 1646.8. Samples: 6952564. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:08:53,344][60425] Avg episode reward: [(0, '38.280'), (1, '53.120')] [2023-10-14 18:08:53,549][61585] Updated weights for policy 1, policy_version 13540 (0.0010) [2023-10-14 18:08:53,942][61585] Updated weights for policy 1, policy_version 13550 (0.0010) [2023-10-14 18:08:54,304][61585] Updated weights for policy 1, policy_version 13560 (0.0008) [2023-10-14 18:08:57,325][61552] Updated weights for policy 0, policy_version 13602 (0.0008) [2023-10-14 18:08:57,705][61552] Updated weights for policy 0, policy_version 13612 (0.0008) [2023-10-14 18:08:58,073][61552] Updated weights for policy 0, policy_version 13622 (0.0007) [2023-10-14 18:08:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 27820032. Throughput: 0: 1654.2, 1: 1648.0. Samples: 6972452. Policy #0 lag: (min: 31.0, avg: 32.3, max: 56.0) [2023-10-14 18:08:58,344][60425] Avg episode reward: [(0, '38.720'), (1, '52.120')] [2023-10-14 18:08:58,431][61552] Updated weights for policy 0, policy_version 13632 (0.0007) [2023-10-14 18:08:58,484][61585] Updated weights for policy 1, policy_version 13570 (0.0008) [2023-10-14 18:08:58,853][61585] Updated weights for policy 1, policy_version 13580 (0.0008) [2023-10-14 18:08:59,209][61585] Updated weights for policy 1, policy_version 13590 (0.0009) [2023-10-14 18:08:59,580][61585] Updated weights for policy 1, policy_version 13600 (0.0008) [2023-10-14 18:09:02,352][61552] Updated weights for policy 0, policy_version 13642 (0.0009) [2023-10-14 18:09:02,721][61552] Updated weights for policy 0, policy_version 13652 (0.0008) [2023-10-14 18:09:03,093][61552] Updated weights for policy 0, policy_version 13662 (0.0008) [2023-10-14 18:09:03,343][60425] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 27918336. Throughput: 0: 1672.7, 1: 1651.8. Samples: 6982264. Policy #0 lag: (min: 31.0, avg: 32.3, max: 56.0) [2023-10-14 18:09:03,344][60425] Avg episode reward: [(0, '38.340'), (1, '54.220')] [2023-10-14 18:09:03,725][61585] Updated weights for policy 1, policy_version 13610 (0.0008) [2023-10-14 18:09:04,093][61585] Updated weights for policy 1, policy_version 13620 (0.0010) [2023-10-14 18:09:04,467][61585] Updated weights for policy 1, policy_version 13630 (0.0008) [2023-10-14 18:09:07,246][61552] Updated weights for policy 0, policy_version 13672 (0.0011) [2023-10-14 18:09:07,615][61552] Updated weights for policy 0, policy_version 13682 (0.0009) [2023-10-14 18:09:07,984][61552] Updated weights for policy 0, policy_version 13692 (0.0007) [2023-10-14 18:09:08,343][60425] Fps is (10 sec: 16383.8, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 27983872. Throughput: 0: 1676.7, 1: 1650.4. Samples: 7002918. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-14 18:09:08,344][60425] Avg episode reward: [(0, '39.120'), (1, '52.180')] [2023-10-14 18:09:08,468][61585] Updated weights for policy 1, policy_version 13640 (0.0008) [2023-10-14 18:09:08,834][61585] Updated weights for policy 1, policy_version 13650 (0.0007) [2023-10-14 18:09:09,196][61585] Updated weights for policy 1, policy_version 13660 (0.0007) [2023-10-14 18:09:12,237][61552] Updated weights for policy 0, policy_version 13702 (0.0008) [2023-10-14 18:09:12,615][61552] Updated weights for policy 0, policy_version 13712 (0.0009) [2023-10-14 18:09:12,985][61552] Updated weights for policy 0, policy_version 13722 (0.0007) [2023-10-14 18:09:13,284][61585] Updated weights for policy 1, policy_version 13670 (0.0008) [2023-10-14 18:09:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 28049408. Throughput: 0: 1657.6, 1: 1653.2. Samples: 7022468. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-14 18:09:13,344][60425] Avg episode reward: [(0, '38.630'), (1, '52.670')] [2023-10-14 18:09:13,650][61585] Updated weights for policy 1, policy_version 13680 (0.0010) [2023-10-14 18:09:14,019][61585] Updated weights for policy 1, policy_version 13690 (0.0008) [2023-10-14 18:09:17,092][61552] Updated weights for policy 0, policy_version 13732 (0.0009) [2023-10-14 18:09:17,467][61552] Updated weights for policy 0, policy_version 13742 (0.0009) [2023-10-14 18:09:17,829][61552] Updated weights for policy 0, policy_version 13752 (0.0007) [2023-10-14 18:09:18,019][61585] Updated weights for policy 1, policy_version 13700 (0.0009) [2023-10-14 18:09:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 28114944. Throughput: 0: 1677.2, 1: 1651.3. Samples: 7032322. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-14 18:09:18,344][60425] Avg episode reward: [(0, '38.980'), (1, '56.260')] [2023-10-14 18:09:18,394][61585] Updated weights for policy 1, policy_version 13710 (0.0009) [2023-10-14 18:09:18,749][61585] Updated weights for policy 1, policy_version 13720 (0.0008) [2023-10-14 18:09:19,038][61248] Saving new best policy, reward=56.260! [2023-10-14 18:09:21,921][61552] Updated weights for policy 0, policy_version 13762 (0.0009) [2023-10-14 18:09:22,288][61552] Updated weights for policy 0, policy_version 13772 (0.0011) [2023-10-14 18:09:22,671][61552] Updated weights for policy 0, policy_version 13782 (0.0008) [2023-10-14 18:09:23,017][61585] Updated weights for policy 1, policy_version 13730 (0.0009) [2023-10-14 18:09:23,038][61552] Updated weights for policy 0, policy_version 13792 (0.0007) [2023-10-14 18:09:23,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 28180480. Throughput: 0: 1673.3, 1: 1653.1. Samples: 7052660. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:09:23,344][60425] Avg episode reward: [(0, '34.660'), (1, '53.160')] [2023-10-14 18:09:23,383][61585] Updated weights for policy 1, policy_version 13740 (0.0007) [2023-10-14 18:09:23,747][61585] Updated weights for policy 1, policy_version 13750 (0.0008) [2023-10-14 18:09:24,119][61585] Updated weights for policy 1, policy_version 13760 (0.0010) [2023-10-14 18:09:27,213][61552] Updated weights for policy 0, policy_version 13802 (0.0008) [2023-10-14 18:09:27,580][61552] Updated weights for policy 0, policy_version 13812 (0.0007) [2023-10-14 18:09:27,952][61552] Updated weights for policy 0, policy_version 13822 (0.0010) [2023-10-14 18:09:28,066][61585] Updated weights for policy 1, policy_version 13770 (0.0007) [2023-10-14 18:09:28,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 28246016. Throughput: 0: 1655.7, 1: 1663.6. Samples: 7072454. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:09:28,344][60425] Avg episode reward: [(0, '37.610'), (1, '54.410')] [2023-10-14 18:09:28,427][61585] Updated weights for policy 1, policy_version 13780 (0.0007) [2023-10-14 18:09:28,800][61585] Updated weights for policy 1, policy_version 13790 (0.0010) [2023-10-14 18:09:31,891][61552] Updated weights for policy 0, policy_version 13832 (0.0008) [2023-10-14 18:09:32,260][61552] Updated weights for policy 0, policy_version 13842 (0.0007) [2023-10-14 18:09:32,631][61552] Updated weights for policy 0, policy_version 13852 (0.0008) [2023-10-14 18:09:32,818][61585] Updated weights for policy 1, policy_version 13800 (0.0009) [2023-10-14 18:09:33,183][61585] Updated weights for policy 1, policy_version 13810 (0.0008) [2023-10-14 18:09:33,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 28311552. Throughput: 0: 1672.3, 1: 1666.6. Samples: 7082438. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:09:33,344][60425] Avg episode reward: [(0, '39.400'), (1, '52.760')] [2023-10-14 18:09:33,544][61585] Updated weights for policy 1, policy_version 13820 (0.0008) [2023-10-14 18:09:36,844][61552] Updated weights for policy 0, policy_version 13862 (0.0009) [2023-10-14 18:09:37,222][61552] Updated weights for policy 0, policy_version 13872 (0.0008) [2023-10-14 18:09:37,588][61552] Updated weights for policy 0, policy_version 13882 (0.0007) [2023-10-14 18:09:37,657][61585] Updated weights for policy 1, policy_version 13830 (0.0009) [2023-10-14 18:09:38,028][61585] Updated weights for policy 1, policy_version 13840 (0.0007) [2023-10-14 18:09:38,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 28377088. Throughput: 0: 1670.9, 1: 1670.6. Samples: 7102932. Policy #0 lag: (min: 18.0, avg: 23.8, max: 50.0) [2023-10-14 18:09:38,344][60425] Avg episode reward: [(0, '40.960'), (1, '51.780')] [2023-10-14 18:09:38,344][61172] Saving new best policy, reward=40.960! [2023-10-14 18:09:38,387][61585] Updated weights for policy 1, policy_version 13850 (0.0007) [2023-10-14 18:09:41,478][61552] Updated weights for policy 0, policy_version 13892 (0.0010) [2023-10-14 18:09:41,844][61552] Updated weights for policy 0, policy_version 13902 (0.0010) [2023-10-14 18:09:42,214][61552] Updated weights for policy 0, policy_version 13912 (0.0010) [2023-10-14 18:09:42,597][61585] Updated weights for policy 1, policy_version 13860 (0.0009) [2023-10-14 18:09:42,989][61585] Updated weights for policy 1, policy_version 13870 (0.0009) [2023-10-14 18:09:43,343][61585] Updated weights for policy 1, policy_version 13880 (0.0009) [2023-10-14 18:09:43,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 28442624. Throughput: 0: 1660.3, 1: 1665.2. Samples: 7122104. Policy #0 lag: (min: 18.0, avg: 23.8, max: 50.0) [2023-10-14 18:09:43,345][60425] Avg episode reward: [(0, '39.240'), (1, '54.600')] [2023-10-14 18:09:43,354][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000013920_14254080.pth... [2023-10-14 18:09:43,388][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000012352_12648448.pth [2023-10-14 18:09:43,635][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000013888_14221312.pth... [2023-10-14 18:09:43,675][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000012320_12615680.pth [2023-10-14 18:09:46,410][61552] Updated weights for policy 0, policy_version 13922 (0.0007) [2023-10-14 18:09:46,775][61552] Updated weights for policy 0, policy_version 13932 (0.0007) [2023-10-14 18:09:47,141][61552] Updated weights for policy 0, policy_version 13942 (0.0008) [2023-10-14 18:09:47,440][61585] Updated weights for policy 1, policy_version 13890 (0.0008) [2023-10-14 18:09:47,506][61552] Updated weights for policy 0, policy_version 13952 (0.0007) [2023-10-14 18:09:47,807][61585] Updated weights for policy 1, policy_version 13900 (0.0010) [2023-10-14 18:09:48,174][61585] Updated weights for policy 1, policy_version 13910 (0.0009) [2023-10-14 18:09:48,344][60425] Fps is (10 sec: 13106.5, 60 sec: 13653.2, 300 sec: 13218.3). Total num frames: 28508160. Throughput: 0: 1673.6, 1: 1671.4. Samples: 7132788. Policy #0 lag: (min: 18.0, avg: 23.8, max: 50.0) [2023-10-14 18:09:48,344][60425] Avg episode reward: [(0, '40.180'), (1, '55.400')] [2023-10-14 18:09:48,543][61585] Updated weights for policy 1, policy_version 13920 (0.0009) [2023-10-14 18:09:51,558][61552] Updated weights for policy 0, policy_version 13962 (0.0007) [2023-10-14 18:09:51,924][61552] Updated weights for policy 0, policy_version 13972 (0.0008) [2023-10-14 18:09:52,290][61552] Updated weights for policy 0, policy_version 13982 (0.0008) [2023-10-14 18:09:52,829][61585] Updated weights for policy 1, policy_version 13930 (0.0007) [2023-10-14 18:09:53,202][61585] Updated weights for policy 1, policy_version 13940 (0.0009) [2023-10-14 18:09:53,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 28573696. Throughput: 0: 1658.3, 1: 1670.1. Samples: 7152694. Policy #0 lag: (min: 28.0, avg: 52.5, max: 56.0) [2023-10-14 18:09:53,344][60425] Avg episode reward: [(0, '39.860'), (1, '56.850')] [2023-10-14 18:09:53,559][61585] Updated weights for policy 1, policy_version 13950 (0.0008) [2023-10-14 18:09:53,627][61248] Saving new best policy, reward=56.850! [2023-10-14 18:09:56,623][61552] Updated weights for policy 0, policy_version 13992 (0.0009) [2023-10-14 18:09:56,996][61552] Updated weights for policy 0, policy_version 14002 (0.0008) [2023-10-14 18:09:57,363][61552] Updated weights for policy 0, policy_version 14012 (0.0007) [2023-10-14 18:09:57,658][61585] Updated weights for policy 1, policy_version 13960 (0.0009) [2023-10-14 18:09:58,018][61585] Updated weights for policy 1, policy_version 13970 (0.0009) [2023-10-14 18:09:58,343][60425] Fps is (10 sec: 13107.9, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 28639232. Throughput: 0: 1663.0, 1: 1665.2. Samples: 7172240. Policy #0 lag: (min: 28.0, avg: 52.5, max: 56.0) [2023-10-14 18:09:58,344][60425] Avg episode reward: [(0, '39.280'), (1, '56.460')] [2023-10-14 18:09:58,386][61585] Updated weights for policy 1, policy_version 13980 (0.0009) [2023-10-14 18:10:01,341][61552] Updated weights for policy 0, policy_version 14022 (0.0007) [2023-10-14 18:10:01,712][61552] Updated weights for policy 0, policy_version 14032 (0.0008) [2023-10-14 18:10:02,076][61552] Updated weights for policy 0, policy_version 14042 (0.0008) [2023-10-14 18:10:02,443][61585] Updated weights for policy 1, policy_version 13990 (0.0008) [2023-10-14 18:10:02,812][61585] Updated weights for policy 1, policy_version 14000 (0.0011) [2023-10-14 18:10:03,185][61585] Updated weights for policy 1, policy_version 14010 (0.0008) [2023-10-14 18:10:03,344][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 28704768. Throughput: 0: 1674.7, 1: 1675.5. Samples: 7183080. Policy #0 lag: (min: 28.0, avg: 52.5, max: 56.0) [2023-10-14 18:10:03,345][60425] Avg episode reward: [(0, '38.750'), (1, '54.640')] [2023-10-14 18:10:06,109][61552] Updated weights for policy 0, policy_version 14052 (0.0008) [2023-10-14 18:10:06,487][61552] Updated weights for policy 0, policy_version 14062 (0.0010) [2023-10-14 18:10:06,858][61552] Updated weights for policy 0, policy_version 14072 (0.0008) [2023-10-14 18:10:07,293][61585] Updated weights for policy 1, policy_version 14020 (0.0008) [2023-10-14 18:10:07,652][61585] Updated weights for policy 1, policy_version 14030 (0.0010) [2023-10-14 18:10:08,014][61585] Updated weights for policy 1, policy_version 14040 (0.0009) [2023-10-14 18:10:08,343][60425] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 28803072. Throughput: 0: 1662.3, 1: 1679.2. Samples: 7203026. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-14 18:10:08,344][60425] Avg episode reward: [(0, '38.310'), (1, '53.940')] [2023-10-14 18:10:11,020][61552] Updated weights for policy 0, policy_version 14082 (0.0009) [2023-10-14 18:10:11,399][61552] Updated weights for policy 0, policy_version 14092 (0.0008) [2023-10-14 18:10:11,760][61552] Updated weights for policy 0, policy_version 14102 (0.0009) [2023-10-14 18:10:11,975][61585] Updated weights for policy 1, policy_version 14050 (0.0007) [2023-10-14 18:10:12,134][61552] Updated weights for policy 0, policy_version 14112 (0.0009) [2023-10-14 18:10:12,340][61585] Updated weights for policy 1, policy_version 14060 (0.0010) [2023-10-14 18:10:12,711][61585] Updated weights for policy 1, policy_version 14070 (0.0011) [2023-10-14 18:10:13,068][61585] Updated weights for policy 1, policy_version 14080 (0.0011) [2023-10-14 18:10:13,343][60425] Fps is (10 sec: 16384.6, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 28868608. Throughput: 0: 1672.7, 1: 1659.6. Samples: 7222408. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-14 18:10:13,344][60425] Avg episode reward: [(0, '36.630'), (1, '50.970')] [2023-10-14 18:10:15,966][61552] Updated weights for policy 0, policy_version 14122 (0.0008) [2023-10-14 18:10:16,334][61552] Updated weights for policy 0, policy_version 14132 (0.0010) [2023-10-14 18:10:16,700][61552] Updated weights for policy 0, policy_version 14142 (0.0008) [2023-10-14 18:10:17,238][61585] Updated weights for policy 1, policy_version 14090 (0.0010) [2023-10-14 18:10:17,600][61585] Updated weights for policy 1, policy_version 14100 (0.0008) [2023-10-14 18:10:17,966][61585] Updated weights for policy 1, policy_version 14110 (0.0008) [2023-10-14 18:10:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 28934144. Throughput: 0: 1682.9, 1: 1672.3. Samples: 7233424. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-14 18:10:18,344][60425] Avg episode reward: [(0, '36.330'), (1, '50.910')] [2023-10-14 18:10:20,991][61552] Updated weights for policy 0, policy_version 14152 (0.0010) [2023-10-14 18:10:21,354][61552] Updated weights for policy 0, policy_version 14162 (0.0011) [2023-10-14 18:10:21,731][61552] Updated weights for policy 0, policy_version 14172 (0.0010) [2023-10-14 18:10:22,268][61585] Updated weights for policy 1, policy_version 14120 (0.0009) [2023-10-14 18:10:22,642][61585] Updated weights for policy 1, policy_version 14130 (0.0008) [2023-10-14 18:10:23,009][61585] Updated weights for policy 1, policy_version 14140 (0.0007) [2023-10-14 18:10:23,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 28999680. Throughput: 0: 1663.4, 1: 1670.7. Samples: 7252966. Policy #0 lag: (min: 30.0, avg: 33.0, max: 62.0) [2023-10-14 18:10:23,344][60425] Avg episode reward: [(0, '37.060'), (1, '53.520')] [2023-10-14 18:10:25,713][61552] Updated weights for policy 0, policy_version 14182 (0.0009) [2023-10-14 18:10:26,091][61552] Updated weights for policy 0, policy_version 14192 (0.0009) [2023-10-14 18:10:26,459][61552] Updated weights for policy 0, policy_version 14202 (0.0010) [2023-10-14 18:10:27,128][61585] Updated weights for policy 1, policy_version 14150 (0.0008) [2023-10-14 18:10:27,522][61585] Updated weights for policy 1, policy_version 14160 (0.0009) [2023-10-14 18:10:27,896][61585] Updated weights for policy 1, policy_version 14170 (0.0010) [2023-10-14 18:10:28,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 29065216. Throughput: 0: 1687.0, 1: 1656.1. Samples: 7272544. Policy #0 lag: (min: 30.0, avg: 33.0, max: 62.0) [2023-10-14 18:10:28,344][60425] Avg episode reward: [(0, '38.950'), (1, '53.380')] [2023-10-14 18:10:30,502][61552] Updated weights for policy 0, policy_version 14212 (0.0007) [2023-10-14 18:10:30,866][61552] Updated weights for policy 0, policy_version 14222 (0.0008) [2023-10-14 18:10:31,245][61552] Updated weights for policy 0, policy_version 14232 (0.0010) [2023-10-14 18:10:31,923][61585] Updated weights for policy 1, policy_version 14180 (0.0010) [2023-10-14 18:10:32,280][61585] Updated weights for policy 1, policy_version 14190 (0.0008) [2023-10-14 18:10:32,652][61585] Updated weights for policy 1, policy_version 14200 (0.0008) [2023-10-14 18:10:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 29130752. Throughput: 0: 1676.8, 1: 1668.4. Samples: 7283320. Policy #0 lag: (min: 30.0, avg: 33.0, max: 62.0) [2023-10-14 18:10:33,344][60425] Avg episode reward: [(0, '38.780'), (1, '53.970')] [2023-10-14 18:10:35,286][61552] Updated weights for policy 0, policy_version 14242 (0.0009) [2023-10-14 18:10:35,660][61552] Updated weights for policy 0, policy_version 14252 (0.0007) [2023-10-14 18:10:36,024][61552] Updated weights for policy 0, policy_version 14262 (0.0009) [2023-10-14 18:10:36,390][61552] Updated weights for policy 0, policy_version 14272 (0.0007) [2023-10-14 18:10:36,813][61585] Updated weights for policy 1, policy_version 14210 (0.0009) [2023-10-14 18:10:37,186][61585] Updated weights for policy 1, policy_version 14220 (0.0009) [2023-10-14 18:10:37,550][61585] Updated weights for policy 1, policy_version 14230 (0.0007) [2023-10-14 18:10:37,910][61585] Updated weights for policy 1, policy_version 14240 (0.0009) [2023-10-14 18:10:38,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 29196288. Throughput: 0: 1673.4, 1: 1667.6. Samples: 7303038. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:10:38,344][60425] Avg episode reward: [(0, '38.940'), (1, '54.320')] [2023-10-14 18:10:40,634][61552] Updated weights for policy 0, policy_version 14282 (0.0008) [2023-10-14 18:10:41,020][61552] Updated weights for policy 0, policy_version 14292 (0.0007) [2023-10-14 18:10:41,385][61552] Updated weights for policy 0, policy_version 14302 (0.0008) [2023-10-14 18:10:41,929][61585] Updated weights for policy 1, policy_version 14250 (0.0009) [2023-10-14 18:10:42,300][61585] Updated weights for policy 1, policy_version 14260 (0.0008) [2023-10-14 18:10:42,657][61585] Updated weights for policy 1, policy_version 14270 (0.0009) [2023-10-14 18:10:43,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 29261824. Throughput: 0: 1684.9, 1: 1646.9. Samples: 7322170. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:10:43,344][60425] Avg episode reward: [(0, '40.380'), (1, '56.650')] [2023-10-14 18:10:45,425][61552] Updated weights for policy 0, policy_version 14312 (0.0008) [2023-10-14 18:10:45,802][61552] Updated weights for policy 0, policy_version 14322 (0.0009) [2023-10-14 18:10:46,166][61552] Updated weights for policy 0, policy_version 14332 (0.0010) [2023-10-14 18:10:46,816][61585] Updated weights for policy 1, policy_version 14280 (0.0011) [2023-10-14 18:10:47,178][61585] Updated weights for policy 1, policy_version 14290 (0.0010) [2023-10-14 18:10:47,547][61585] Updated weights for policy 1, policy_version 14300 (0.0010) [2023-10-14 18:10:48,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 29327360. Throughput: 0: 1668.1, 1: 1664.4. Samples: 7333042. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:10:48,344][60425] Avg episode reward: [(0, '40.740'), (1, '55.820')] [2023-10-14 18:10:50,236][61552] Updated weights for policy 0, policy_version 14342 (0.0010) [2023-10-14 18:10:50,601][61552] Updated weights for policy 0, policy_version 14352 (0.0008) [2023-10-14 18:10:50,969][61552] Updated weights for policy 0, policy_version 14362 (0.0008) [2023-10-14 18:10:51,753][61585] Updated weights for policy 1, policy_version 14310 (0.0007) [2023-10-14 18:10:52,119][61585] Updated weights for policy 1, policy_version 14320 (0.0008) [2023-10-14 18:10:52,485][61585] Updated weights for policy 1, policy_version 14330 (0.0009) [2023-10-14 18:10:53,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 29392896. Throughput: 0: 1671.3, 1: 1654.5. Samples: 7352686. Policy #0 lag: (min: 27.0, avg: 28.1, max: 49.0) [2023-10-14 18:10:53,345][60425] Avg episode reward: [(0, '38.740'), (1, '56.700')] [2023-10-14 18:10:54,965][61552] Updated weights for policy 0, policy_version 14372 (0.0008) [2023-10-14 18:10:55,344][61552] Updated weights for policy 0, policy_version 14382 (0.0007) [2023-10-14 18:10:55,718][61552] Updated weights for policy 0, policy_version 14392 (0.0010) [2023-10-14 18:10:56,799][61585] Updated weights for policy 1, policy_version 14340 (0.0008) [2023-10-14 18:10:57,162][61585] Updated weights for policy 1, policy_version 14350 (0.0007) [2023-10-14 18:10:57,529][61585] Updated weights for policy 1, policy_version 14360 (0.0007) [2023-10-14 18:10:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 29458432. Throughput: 0: 1685.1, 1: 1648.6. Samples: 7372426. Policy #0 lag: (min: 27.0, avg: 28.1, max: 49.0) [2023-10-14 18:10:58,344][60425] Avg episode reward: [(0, '39.150'), (1, '54.240')] [2023-10-14 18:10:59,807][61552] Updated weights for policy 0, policy_version 14402 (0.0008) [2023-10-14 18:11:00,170][61552] Updated weights for policy 0, policy_version 14412 (0.0009) [2023-10-14 18:11:00,541][61552] Updated weights for policy 0, policy_version 14422 (0.0007) [2023-10-14 18:11:00,917][61552] Updated weights for policy 0, policy_version 14432 (0.0008) [2023-10-14 18:11:01,548][61585] Updated weights for policy 1, policy_version 14370 (0.0009) [2023-10-14 18:11:01,913][61585] Updated weights for policy 1, policy_version 14380 (0.0008) [2023-10-14 18:11:02,275][61585] Updated weights for policy 1, policy_version 14390 (0.0009) [2023-10-14 18:11:02,645][61585] Updated weights for policy 1, policy_version 14400 (0.0007) [2023-10-14 18:11:03,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 29523968. Throughput: 0: 1659.5, 1: 1659.3. Samples: 7382772. Policy #0 lag: (min: 27.0, avg: 28.1, max: 49.0) [2023-10-14 18:11:03,344][60425] Avg episode reward: [(0, '41.000'), (1, '54.100')] [2023-10-14 18:11:03,345][61172] Saving new best policy, reward=41.000! [2023-10-14 18:11:04,952][61552] Updated weights for policy 0, policy_version 14442 (0.0010) [2023-10-14 18:11:05,322][61552] Updated weights for policy 0, policy_version 14452 (0.0007) [2023-10-14 18:11:05,688][61552] Updated weights for policy 0, policy_version 14462 (0.0007) [2023-10-14 18:11:06,606][61585] Updated weights for policy 1, policy_version 14410 (0.0010) [2023-10-14 18:11:06,968][61585] Updated weights for policy 1, policy_version 14420 (0.0008) [2023-10-14 18:11:07,334][61585] Updated weights for policy 1, policy_version 14430 (0.0011) [2023-10-14 18:11:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 29589504. Throughput: 0: 1675.9, 1: 1651.1. Samples: 7402680. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:11:08,344][60425] Avg episode reward: [(0, '40.360'), (1, '52.740')] [2023-10-14 18:11:09,874][61552] Updated weights for policy 0, policy_version 14472 (0.0007) [2023-10-14 18:11:10,252][61552] Updated weights for policy 0, policy_version 14482 (0.0007) [2023-10-14 18:11:10,612][61552] Updated weights for policy 0, policy_version 14492 (0.0007) [2023-10-14 18:11:11,539][61585] Updated weights for policy 1, policy_version 14440 (0.0010) [2023-10-14 18:11:11,909][61585] Updated weights for policy 1, policy_version 14450 (0.0009) [2023-10-14 18:11:12,288][61585] Updated weights for policy 1, policy_version 14460 (0.0008) [2023-10-14 18:11:13,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 29655040. Throughput: 0: 1677.9, 1: 1653.0. Samples: 7422432. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:11:13,344][60425] Avg episode reward: [(0, '40.240'), (1, '54.240')] [2023-10-14 18:11:14,579][61552] Updated weights for policy 0, policy_version 14502 (0.0009) [2023-10-14 18:11:14,947][61552] Updated weights for policy 0, policy_version 14512 (0.0012) [2023-10-14 18:11:15,314][61552] Updated weights for policy 0, policy_version 14522 (0.0011) [2023-10-14 18:11:16,455][61585] Updated weights for policy 1, policy_version 14470 (0.0008) [2023-10-14 18:11:16,824][61585] Updated weights for policy 1, policy_version 14480 (0.0007) [2023-10-14 18:11:17,187][61585] Updated weights for policy 1, policy_version 14490 (0.0008) [2023-10-14 18:11:18,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 29720576. Throughput: 0: 1660.6, 1: 1661.1. Samples: 7432798. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:11:18,344][60425] Avg episode reward: [(0, '41.060'), (1, '53.460')] [2023-10-14 18:11:18,345][61172] Saving new best policy, reward=41.060! [2023-10-14 18:11:19,535][61552] Updated weights for policy 0, policy_version 14532 (0.0010) [2023-10-14 18:11:19,906][61552] Updated weights for policy 0, policy_version 14542 (0.0008) [2023-10-14 18:11:20,264][61552] Updated weights for policy 0, policy_version 14552 (0.0008) [2023-10-14 18:11:21,266][61585] Updated weights for policy 1, policy_version 14500 (0.0009) [2023-10-14 18:11:21,632][61585] Updated weights for policy 1, policy_version 14510 (0.0009) [2023-10-14 18:11:22,006][61585] Updated weights for policy 1, policy_version 14520 (0.0010) [2023-10-14 18:11:23,344][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 29786112. Throughput: 0: 1674.4, 1: 1646.3. Samples: 7452472. Policy #0 lag: (min: 31.0, avg: 41.3, max: 63.0) [2023-10-14 18:11:23,345][60425] Avg episode reward: [(0, '42.240'), (1, '51.920')] [2023-10-14 18:11:23,346][61172] Saving new best policy, reward=42.240! [2023-10-14 18:11:24,352][61552] Updated weights for policy 0, policy_version 14562 (0.0009) [2023-10-14 18:11:24,727][61552] Updated weights for policy 0, policy_version 14572 (0.0009) [2023-10-14 18:11:25,090][61552] Updated weights for policy 0, policy_version 14582 (0.0009) [2023-10-14 18:11:25,463][61552] Updated weights for policy 0, policy_version 14592 (0.0008) [2023-10-14 18:11:26,185][61585] Updated weights for policy 1, policy_version 14530 (0.0010) [2023-10-14 18:11:26,551][61585] Updated weights for policy 1, policy_version 14540 (0.0009) [2023-10-14 18:11:26,910][61585] Updated weights for policy 1, policy_version 14550 (0.0009) [2023-10-14 18:11:27,276][61585] Updated weights for policy 1, policy_version 14560 (0.0007) [2023-10-14 18:11:28,344][60425] Fps is (10 sec: 13106.7, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 29851648. Throughput: 0: 1677.6, 1: 1657.6. Samples: 7472256. Policy #0 lag: (min: 31.0, avg: 41.3, max: 63.0) [2023-10-14 18:11:28,345][60425] Avg episode reward: [(0, '40.710'), (1, '54.110')] [2023-10-14 18:11:29,748][61552] Updated weights for policy 0, policy_version 14602 (0.0007) [2023-10-14 18:11:30,117][61552] Updated weights for policy 0, policy_version 14612 (0.0008) [2023-10-14 18:11:30,484][61552] Updated weights for policy 0, policy_version 14622 (0.0007) [2023-10-14 18:11:31,311][61585] Updated weights for policy 1, policy_version 14570 (0.0010) [2023-10-14 18:11:31,682][61585] Updated weights for policy 1, policy_version 14580 (0.0008) [2023-10-14 18:11:32,054][61585] Updated weights for policy 1, policy_version 14590 (0.0008) [2023-10-14 18:11:33,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 29917184. Throughput: 0: 1660.3, 1: 1661.2. Samples: 7482510. Policy #0 lag: (min: 31.0, avg: 41.3, max: 63.0) [2023-10-14 18:11:33,344][60425] Avg episode reward: [(0, '42.530'), (1, '53.880')] [2023-10-14 18:11:33,345][61172] Saving new best policy, reward=42.530! [2023-10-14 18:11:34,623][61552] Updated weights for policy 0, policy_version 14632 (0.0010) [2023-10-14 18:11:34,994][61552] Updated weights for policy 0, policy_version 14642 (0.0007) [2023-10-14 18:11:35,359][61552] Updated weights for policy 0, policy_version 14652 (0.0007) [2023-10-14 18:11:36,075][61585] Updated weights for policy 1, policy_version 14600 (0.0010) [2023-10-14 18:11:36,437][61585] Updated weights for policy 1, policy_version 14610 (0.0010) [2023-10-14 18:11:36,808][61585] Updated weights for policy 1, policy_version 14620 (0.0009) [2023-10-14 18:11:38,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 29982720. Throughput: 0: 1669.2, 1: 1645.7. Samples: 7501854. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:11:38,344][60425] Avg episode reward: [(0, '40.310'), (1, '53.190')] [2023-10-14 18:11:39,360][61552] Updated weights for policy 0, policy_version 14662 (0.0009) [2023-10-14 18:11:39,733][61552] Updated weights for policy 0, policy_version 14672 (0.0011) [2023-10-14 18:11:40,110][61552] Updated weights for policy 0, policy_version 14682 (0.0009) [2023-10-14 18:11:40,986][61585] Updated weights for policy 1, policy_version 14630 (0.0009) [2023-10-14 18:11:41,344][61585] Updated weights for policy 1, policy_version 14640 (0.0009) [2023-10-14 18:11:41,710][61585] Updated weights for policy 1, policy_version 14650 (0.0011) [2023-10-14 18:11:43,344][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 30048256. Throughput: 0: 1667.1, 1: 1658.5. Samples: 7522082. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:11:43,345][60425] Avg episode reward: [(0, '40.730'), (1, '54.820')] [2023-10-14 18:11:43,357][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000014688_15040512.pth... [2023-10-14 18:11:43,357][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000014656_15007744.pth... [2023-10-14 18:11:43,407][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000013088_13402112.pth [2023-10-14 18:11:43,408][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000013152_13467648.pth [2023-10-14 18:11:44,231][61552] Updated weights for policy 0, policy_version 14692 (0.0009) [2023-10-14 18:11:44,608][61552] Updated weights for policy 0, policy_version 14702 (0.0009) [2023-10-14 18:11:44,981][61552] Updated weights for policy 0, policy_version 14712 (0.0009) [2023-10-14 18:11:45,955][61585] Updated weights for policy 1, policy_version 14660 (0.0009) [2023-10-14 18:11:46,328][61585] Updated weights for policy 1, policy_version 14670 (0.0010) [2023-10-14 18:11:46,698][61585] Updated weights for policy 1, policy_version 14680 (0.0009) [2023-10-14 18:11:48,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 30113792. Throughput: 0: 1663.3, 1: 1662.8. Samples: 7532444. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:11:48,344][60425] Avg episode reward: [(0, '41.760'), (1, '52.070')] [2023-10-14 18:11:48,962][61552] Updated weights for policy 0, policy_version 14722 (0.0008) [2023-10-14 18:11:49,343][61552] Updated weights for policy 0, policy_version 14732 (0.0007) [2023-10-14 18:11:49,712][61552] Updated weights for policy 0, policy_version 14742 (0.0008) [2023-10-14 18:11:50,085][61552] Updated weights for policy 0, policy_version 14752 (0.0008) [2023-10-14 18:11:50,916][61585] Updated weights for policy 1, policy_version 14690 (0.0009) [2023-10-14 18:11:51,293][61585] Updated weights for policy 1, policy_version 14700 (0.0010) [2023-10-14 18:11:51,657][61585] Updated weights for policy 1, policy_version 14710 (0.0009) [2023-10-14 18:11:52,022][61585] Updated weights for policy 1, policy_version 14720 (0.0007) [2023-10-14 18:11:53,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 30179328. Throughput: 0: 1671.2, 1: 1648.2. Samples: 7552054. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-14 18:11:53,344][60425] Avg episode reward: [(0, '43.410'), (1, '52.170')] [2023-10-14 18:11:53,345][61172] Saving new best policy, reward=43.410! [2023-10-14 18:11:54,055][61552] Updated weights for policy 0, policy_version 14762 (0.0007) [2023-10-14 18:11:54,425][61552] Updated weights for policy 0, policy_version 14772 (0.0008) [2023-10-14 18:11:54,792][61552] Updated weights for policy 0, policy_version 14782 (0.0008) [2023-10-14 18:11:56,009][61585] Updated weights for policy 1, policy_version 14730 (0.0008) [2023-10-14 18:11:56,379][61585] Updated weights for policy 1, policy_version 14740 (0.0008) [2023-10-14 18:11:56,735][61585] Updated weights for policy 1, policy_version 14750 (0.0009) [2023-10-14 18:11:58,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 30244864. Throughput: 0: 1670.4, 1: 1660.1. Samples: 7572308. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-14 18:11:58,345][60425] Avg episode reward: [(0, '42.570'), (1, '56.530')] [2023-10-14 18:11:58,912][61552] Updated weights for policy 0, policy_version 14792 (0.0007) [2023-10-14 18:11:59,283][61552] Updated weights for policy 0, policy_version 14802 (0.0009) [2023-10-14 18:11:59,656][61552] Updated weights for policy 0, policy_version 14812 (0.0007) [2023-10-14 18:12:00,959][61585] Updated weights for policy 1, policy_version 14760 (0.0009) [2023-10-14 18:12:01,311][61585] Updated weights for policy 1, policy_version 14770 (0.0011) [2023-10-14 18:12:01,685][61585] Updated weights for policy 1, policy_version 14780 (0.0010) [2023-10-14 18:12:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 30310400. Throughput: 0: 1673.4, 1: 1656.0. Samples: 7582624. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-14 18:12:03,344][60425] Avg episode reward: [(0, '42.630'), (1, '56.010')] [2023-10-14 18:12:03,554][61552] Updated weights for policy 0, policy_version 14822 (0.0008) [2023-10-14 18:12:03,923][61552] Updated weights for policy 0, policy_version 14832 (0.0007) [2023-10-14 18:12:04,295][61552] Updated weights for policy 0, policy_version 14842 (0.0007) [2023-10-14 18:12:06,033][61585] Updated weights for policy 1, policy_version 14790 (0.0009) [2023-10-14 18:12:06,428][61585] Updated weights for policy 1, policy_version 14800 (0.0007) [2023-10-14 18:12:06,791][61585] Updated weights for policy 1, policy_version 14810 (0.0009) [2023-10-14 18:12:08,344][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 30375936. Throughput: 0: 1678.5, 1: 1650.1. Samples: 7602260. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-10-14 18:12:08,345][60425] Avg episode reward: [(0, '42.250'), (1, '55.310')] [2023-10-14 18:12:08,480][61552] Updated weights for policy 0, policy_version 14852 (0.0007) [2023-10-14 18:12:08,846][61552] Updated weights for policy 0, policy_version 14862 (0.0008) [2023-10-14 18:12:09,218][61552] Updated weights for policy 0, policy_version 14872 (0.0007) [2023-10-14 18:12:10,854][61585] Updated weights for policy 1, policy_version 14820 (0.0011) [2023-10-14 18:12:11,218][61585] Updated weights for policy 1, policy_version 14830 (0.0011) [2023-10-14 18:12:11,590][61585] Updated weights for policy 1, policy_version 14840 (0.0009) [2023-10-14 18:12:13,289][61552] Updated weights for policy 0, policy_version 14882 (0.0007) [2023-10-14 18:12:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 30441472. Throughput: 0: 1679.9, 1: 1659.9. Samples: 7622544. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-10-14 18:12:13,344][60425] Avg episode reward: [(0, '42.340'), (1, '53.110')] [2023-10-14 18:12:13,649][61552] Updated weights for policy 0, policy_version 14892 (0.0007) [2023-10-14 18:12:14,018][61552] Updated weights for policy 0, policy_version 14902 (0.0010) [2023-10-14 18:12:14,392][61552] Updated weights for policy 0, policy_version 14912 (0.0008) [2023-10-14 18:12:15,762][61585] Updated weights for policy 1, policy_version 14850 (0.0010) [2023-10-14 18:12:16,133][61585] Updated weights for policy 1, policy_version 14860 (0.0008) [2023-10-14 18:12:16,499][61585] Updated weights for policy 1, policy_version 14870 (0.0007) [2023-10-14 18:12:16,862][61585] Updated weights for policy 1, policy_version 14880 (0.0009) [2023-10-14 18:12:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 30507008. Throughput: 0: 1683.3, 1: 1655.9. Samples: 7632776. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-10-14 18:12:18,344][60425] Avg episode reward: [(0, '41.920'), (1, '56.030')] [2023-10-14 18:12:18,568][61552] Updated weights for policy 0, policy_version 14922 (0.0009) [2023-10-14 18:12:18,946][61552] Updated weights for policy 0, policy_version 14932 (0.0008) [2023-10-14 18:12:19,318][61552] Updated weights for policy 0, policy_version 14942 (0.0010) [2023-10-14 18:12:20,876][61585] Updated weights for policy 1, policy_version 14890 (0.0009) [2023-10-14 18:12:21,240][61585] Updated weights for policy 1, policy_version 14900 (0.0010) [2023-10-14 18:12:21,609][61585] Updated weights for policy 1, policy_version 14910 (0.0007) [2023-10-14 18:12:23,318][61552] Updated weights for policy 0, policy_version 14952 (0.0007) [2023-10-14 18:12:23,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 30572544. Throughput: 0: 1687.4, 1: 1654.7. Samples: 7652250. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-14 18:12:23,344][60425] Avg episode reward: [(0, '40.940'), (1, '51.710')] [2023-10-14 18:12:23,696][61552] Updated weights for policy 0, policy_version 14962 (0.0007) [2023-10-14 18:12:24,060][61552] Updated weights for policy 0, policy_version 14972 (0.0009) [2023-10-14 18:12:25,726][61585] Updated weights for policy 1, policy_version 14920 (0.0008) [2023-10-14 18:12:26,080][61585] Updated weights for policy 1, policy_version 14930 (0.0008) [2023-10-14 18:12:26,445][61585] Updated weights for policy 1, policy_version 14940 (0.0009) [2023-10-14 18:12:28,063][61552] Updated weights for policy 0, policy_version 14982 (0.0008) [2023-10-14 18:12:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 30638080. Throughput: 0: 1685.0, 1: 1664.0. Samples: 7672790. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-14 18:12:28,344][60425] Avg episode reward: [(0, '40.840'), (1, '54.890')] [2023-10-14 18:12:28,431][61552] Updated weights for policy 0, policy_version 14992 (0.0007) [2023-10-14 18:12:28,790][61552] Updated weights for policy 0, policy_version 15002 (0.0008) [2023-10-14 18:12:30,508][61585] Updated weights for policy 1, policy_version 14950 (0.0008) [2023-10-14 18:12:30,872][61585] Updated weights for policy 1, policy_version 14960 (0.0010) [2023-10-14 18:12:31,232][61585] Updated weights for policy 1, policy_version 14970 (0.0008) [2023-10-14 18:12:32,697][61552] Updated weights for policy 0, policy_version 15012 (0.0007) [2023-10-14 18:12:33,062][61552] Updated weights for policy 0, policy_version 15022 (0.0008) [2023-10-14 18:12:33,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 30703616. Throughput: 0: 1687.5, 1: 1654.0. Samples: 7682810. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-14 18:12:33,344][60425] Avg episode reward: [(0, '41.420'), (1, '52.520')] [2023-10-14 18:12:33,440][61552] Updated weights for policy 0, policy_version 15032 (0.0011) [2023-10-14 18:12:35,257][61585] Updated weights for policy 1, policy_version 14980 (0.0008) [2023-10-14 18:12:35,625][61585] Updated weights for policy 1, policy_version 14990 (0.0007) [2023-10-14 18:12:35,988][61585] Updated weights for policy 1, policy_version 15000 (0.0008) [2023-10-14 18:12:37,677][61552] Updated weights for policy 0, policy_version 15042 (0.0010) [2023-10-14 18:12:38,051][61552] Updated weights for policy 0, policy_version 15052 (0.0008) [2023-10-14 18:12:38,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 30769152. Throughput: 0: 1686.7, 1: 1660.6. Samples: 7702684. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-14 18:12:38,344][60425] Avg episode reward: [(0, '40.760'), (1, '50.010')] [2023-10-14 18:12:38,425][61552] Updated weights for policy 0, policy_version 15062 (0.0010) [2023-10-14 18:12:38,793][61552] Updated weights for policy 0, policy_version 15072 (0.0009) [2023-10-14 18:12:40,179][61585] Updated weights for policy 1, policy_version 15010 (0.0008) [2023-10-14 18:12:40,553][61585] Updated weights for policy 1, policy_version 15020 (0.0008) [2023-10-14 18:12:40,926][61585] Updated weights for policy 1, policy_version 15030 (0.0007) [2023-10-14 18:12:41,287][61585] Updated weights for policy 1, policy_version 15040 (0.0008) [2023-10-14 18:12:42,927][61552] Updated weights for policy 0, policy_version 15082 (0.0010) [2023-10-14 18:12:43,299][61552] Updated weights for policy 0, policy_version 15092 (0.0011) [2023-10-14 18:12:43,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 30834688. Throughput: 0: 1680.0, 1: 1667.5. Samples: 7722942. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-14 18:12:43,345][60425] Avg episode reward: [(0, '41.870'), (1, '54.050')] [2023-10-14 18:12:43,671][61552] Updated weights for policy 0, policy_version 15102 (0.0010) [2023-10-14 18:12:45,422][61585] Updated weights for policy 1, policy_version 15050 (0.0008) [2023-10-14 18:12:45,783][61585] Updated weights for policy 1, policy_version 15060 (0.0008) [2023-10-14 18:12:46,154][61585] Updated weights for policy 1, policy_version 15070 (0.0008) [2023-10-14 18:12:47,789][61552] Updated weights for policy 0, policy_version 15112 (0.0008) [2023-10-14 18:12:48,151][61552] Updated weights for policy 0, policy_version 15122 (0.0009) [2023-10-14 18:12:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 30900224. Throughput: 0: 1680.5, 1: 1656.9. Samples: 7732808. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-14 18:12:48,344][60425] Avg episode reward: [(0, '41.240'), (1, '53.520')] [2023-10-14 18:12:48,522][61552] Updated weights for policy 0, policy_version 15132 (0.0008) [2023-10-14 18:12:50,210][61585] Updated weights for policy 1, policy_version 15080 (0.0007) [2023-10-14 18:12:50,583][61585] Updated weights for policy 1, policy_version 15090 (0.0009) [2023-10-14 18:12:50,953][61585] Updated weights for policy 1, policy_version 15100 (0.0008) [2023-10-14 18:12:52,619][61552] Updated weights for policy 0, policy_version 15142 (0.0007) [2023-10-14 18:12:52,989][61552] Updated weights for policy 0, policy_version 15152 (0.0007) [2023-10-14 18:12:53,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 30965760. Throughput: 0: 1677.6, 1: 1667.7. Samples: 7752800. Policy #0 lag: (min: 24.0, avg: 32.6, max: 56.0) [2023-10-14 18:12:53,344][60425] Avg episode reward: [(0, '41.720'), (1, '55.190')] [2023-10-14 18:12:53,365][61552] Updated weights for policy 0, policy_version 15162 (0.0007) [2023-10-14 18:12:55,135][61585] Updated weights for policy 1, policy_version 15110 (0.0008) [2023-10-14 18:12:55,517][61585] Updated weights for policy 1, policy_version 15120 (0.0008) [2023-10-14 18:12:55,884][61585] Updated weights for policy 1, policy_version 15130 (0.0008) [2023-10-14 18:12:57,418][61552] Updated weights for policy 0, policy_version 15172 (0.0008) [2023-10-14 18:12:57,788][61552] Updated weights for policy 0, policy_version 15182 (0.0008) [2023-10-14 18:12:58,152][61552] Updated weights for policy 0, policy_version 15192 (0.0009) [2023-10-14 18:12:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 31031296. Throughput: 0: 1670.8, 1: 1674.5. Samples: 7773080. Policy #0 lag: (min: 24.0, avg: 32.6, max: 56.0) [2023-10-14 18:12:58,344][60425] Avg episode reward: [(0, '42.410'), (1, '55.620')] [2023-10-14 18:12:59,758][61585] Updated weights for policy 1, policy_version 15140 (0.0009) [2023-10-14 18:13:00,132][61585] Updated weights for policy 1, policy_version 15150 (0.0009) [2023-10-14 18:13:00,498][61585] Updated weights for policy 1, policy_version 15160 (0.0009) [2023-10-14 18:13:02,248][61552] Updated weights for policy 0, policy_version 15202 (0.0010) [2023-10-14 18:13:02,610][61552] Updated weights for policy 0, policy_version 15212 (0.0007) [2023-10-14 18:13:02,986][61552] Updated weights for policy 0, policy_version 15222 (0.0007) [2023-10-14 18:13:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 31096832. Throughput: 0: 1679.4, 1: 1653.3. Samples: 7782748. Policy #0 lag: (min: 24.0, avg: 32.6, max: 56.0) [2023-10-14 18:13:03,344][60425] Avg episode reward: [(0, '43.280'), (1, '55.240')] [2023-10-14 18:13:03,356][61552] Updated weights for policy 0, policy_version 15232 (0.0007) [2023-10-14 18:13:04,526][61585] Updated weights for policy 1, policy_version 15170 (0.0010) [2023-10-14 18:13:04,891][61585] Updated weights for policy 1, policy_version 15180 (0.0008) [2023-10-14 18:13:05,250][61585] Updated weights for policy 1, policy_version 15190 (0.0007) [2023-10-14 18:13:05,622][61585] Updated weights for policy 1, policy_version 15200 (0.0010) [2023-10-14 18:13:07,505][61552] Updated weights for policy 0, policy_version 15242 (0.0009) [2023-10-14 18:13:07,886][61552] Updated weights for policy 0, policy_version 15252 (0.0010) [2023-10-14 18:13:08,256][61552] Updated weights for policy 0, policy_version 15262 (0.0010) [2023-10-14 18:13:08,343][60425] Fps is (10 sec: 16383.8, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 31195136. Throughput: 0: 1676.5, 1: 1674.3. Samples: 7803038. Policy #0 lag: (min: 31.0, avg: 31.6, max: 49.0) [2023-10-14 18:13:08,345][60425] Avg episode reward: [(0, '44.120'), (1, '53.710')] [2023-10-14 18:13:08,346][61172] Saving new best policy, reward=44.120! [2023-10-14 18:13:09,827][61585] Updated weights for policy 1, policy_version 15210 (0.0009) [2023-10-14 18:13:10,200][61585] Updated weights for policy 1, policy_version 15220 (0.0009) [2023-10-14 18:13:10,565][61585] Updated weights for policy 1, policy_version 15230 (0.0007) [2023-10-14 18:13:12,374][61552] Updated weights for policy 0, policy_version 15272 (0.0010) [2023-10-14 18:13:12,748][61552] Updated weights for policy 0, policy_version 15282 (0.0009) [2023-10-14 18:13:13,122][61552] Updated weights for policy 0, policy_version 15292 (0.0007) [2023-10-14 18:13:13,343][60425] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 31260672. Throughput: 0: 1660.4, 1: 1672.4. Samples: 7822770. Policy #0 lag: (min: 31.0, avg: 31.6, max: 49.0) [2023-10-14 18:13:13,344][60425] Avg episode reward: [(0, '43.650'), (1, '55.140')] [2023-10-14 18:13:14,641][61585] Updated weights for policy 1, policy_version 15240 (0.0007) [2023-10-14 18:13:15,003][61585] Updated weights for policy 1, policy_version 15250 (0.0011) [2023-10-14 18:13:15,367][61585] Updated weights for policy 1, policy_version 15260 (0.0008) [2023-10-14 18:13:17,105][61552] Updated weights for policy 0, policy_version 15302 (0.0007) [2023-10-14 18:13:17,475][61552] Updated weights for policy 0, policy_version 15312 (0.0010) [2023-10-14 18:13:17,853][61552] Updated weights for policy 0, policy_version 15322 (0.0009) [2023-10-14 18:13:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 31326208. Throughput: 0: 1670.7, 1: 1654.3. Samples: 7832436. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:13:18,344][60425] Avg episode reward: [(0, '41.770'), (1, '53.990')] [2023-10-14 18:13:19,655][61585] Updated weights for policy 1, policy_version 15270 (0.0009) [2023-10-14 18:13:20,013][61585] Updated weights for policy 1, policy_version 15280 (0.0007) [2023-10-14 18:13:20,382][61585] Updated weights for policy 1, policy_version 15290 (0.0007) [2023-10-14 18:13:22,008][61552] Updated weights for policy 0, policy_version 15332 (0.0009) [2023-10-14 18:13:22,383][61552] Updated weights for policy 0, policy_version 15342 (0.0009) [2023-10-14 18:13:22,755][61552] Updated weights for policy 0, policy_version 15352 (0.0009) [2023-10-14 18:13:23,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 31391744. Throughput: 0: 1669.5, 1: 1665.1. Samples: 7852742. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:13:23,344][60425] Avg episode reward: [(0, '44.170'), (1, '53.770')] [2023-10-14 18:13:23,346][61172] Saving new best policy, reward=44.170! [2023-10-14 18:13:24,548][61585] Updated weights for policy 1, policy_version 15300 (0.0009) [2023-10-14 18:13:24,908][61585] Updated weights for policy 1, policy_version 15310 (0.0010) [2023-10-14 18:13:25,275][61585] Updated weights for policy 1, policy_version 15320 (0.0009) [2023-10-14 18:13:26,858][61552] Updated weights for policy 0, policy_version 15362 (0.0009) [2023-10-14 18:13:27,234][61552] Updated weights for policy 0, policy_version 15372 (0.0011) [2023-10-14 18:13:27,599][61552] Updated weights for policy 0, policy_version 15382 (0.0010) [2023-10-14 18:13:27,965][61552] Updated weights for policy 0, policy_version 15392 (0.0011) [2023-10-14 18:13:28,344][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 31457280. Throughput: 0: 1649.7, 1: 1670.1. Samples: 7872334. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:13:28,345][60425] Avg episode reward: [(0, '44.010'), (1, '55.030')] [2023-10-14 18:13:29,238][61585] Updated weights for policy 1, policy_version 15330 (0.0009) [2023-10-14 18:13:29,603][61585] Updated weights for policy 1, policy_version 15340 (0.0008) [2023-10-14 18:13:29,969][61585] Updated weights for policy 1, policy_version 15350 (0.0008) [2023-10-14 18:13:30,326][61585] Updated weights for policy 1, policy_version 15360 (0.0008) [2023-10-14 18:13:32,116][61552] Updated weights for policy 0, policy_version 15402 (0.0008) [2023-10-14 18:13:32,487][61552] Updated weights for policy 0, policy_version 15412 (0.0008) [2023-10-14 18:13:32,860][61552] Updated weights for policy 0, policy_version 15422 (0.0009) [2023-10-14 18:13:33,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 31522816. Throughput: 0: 1669.6, 1: 1657.0. Samples: 7882508. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:13:33,344][60425] Avg episode reward: [(0, '45.010'), (1, '54.440')] [2023-10-14 18:13:33,345][61172] Saving new best policy, reward=45.010! [2023-10-14 18:13:34,406][61585] Updated weights for policy 1, policy_version 15370 (0.0010) [2023-10-14 18:13:34,779][61585] Updated weights for policy 1, policy_version 15380 (0.0007) [2023-10-14 18:13:35,144][61585] Updated weights for policy 1, policy_version 15390 (0.0007) [2023-10-14 18:13:36,873][61552] Updated weights for policy 0, policy_version 15432 (0.0008) [2023-10-14 18:13:37,235][61552] Updated weights for policy 0, policy_version 15442 (0.0007) [2023-10-14 18:13:37,600][61552] Updated weights for policy 0, policy_version 15452 (0.0007) [2023-10-14 18:13:38,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 31588352. Throughput: 0: 1667.8, 1: 1671.5. Samples: 7903070. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:13:38,344][60425] Avg episode reward: [(0, '45.320'), (1, '55.270')] [2023-10-14 18:13:38,346][61172] Saving new best policy, reward=45.320! [2023-10-14 18:13:39,350][61585] Updated weights for policy 1, policy_version 15400 (0.0008) [2023-10-14 18:13:39,711][61585] Updated weights for policy 1, policy_version 15410 (0.0009) [2023-10-14 18:13:40,083][61585] Updated weights for policy 1, policy_version 15420 (0.0007) [2023-10-14 18:13:41,617][61552] Updated weights for policy 0, policy_version 15462 (0.0009) [2023-10-14 18:13:41,979][61552] Updated weights for policy 0, policy_version 15472 (0.0011) [2023-10-14 18:13:42,356][61552] Updated weights for policy 0, policy_version 15482 (0.0010) [2023-10-14 18:13:43,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 31653888. Throughput: 0: 1652.1, 1: 1673.1. Samples: 7922716. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:13:43,344][60425] Avg episode reward: [(0, '43.660'), (1, '53.320')] [2023-10-14 18:13:43,352][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000015488_15859712.pth... [2023-10-14 18:13:43,352][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000015424_15794176.pth... [2023-10-14 18:13:43,386][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000013888_14221312.pth [2023-10-14 18:13:43,390][61248] Saving a milestone ./train_atari/atari_roadrunner_APPO/checkpoint_p1/milestones/checkpoint_000015424_15794176.pth [2023-10-14 18:13:43,393][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000013920_14254080.pth [2023-10-14 18:13:43,398][61172] Saving a milestone ./train_atari/atari_roadrunner_APPO/checkpoint_p0/milestones/checkpoint_000015488_15859712.pth [2023-10-14 18:13:44,174][61585] Updated weights for policy 1, policy_version 15430 (0.0010) [2023-10-14 18:13:44,546][61585] Updated weights for policy 1, policy_version 15440 (0.0008) [2023-10-14 18:13:44,915][61585] Updated weights for policy 1, policy_version 15450 (0.0007) [2023-10-14 18:13:46,609][61552] Updated weights for policy 0, policy_version 15492 (0.0008) [2023-10-14 18:13:46,980][61552] Updated weights for policy 0, policy_version 15502 (0.0007) [2023-10-14 18:13:47,350][61552] Updated weights for policy 0, policy_version 15512 (0.0008) [2023-10-14 18:13:48,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 31719424. Throughput: 0: 1667.8, 1: 1665.4. Samples: 7932742. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:13:48,344][60425] Avg episode reward: [(0, '42.620'), (1, '54.510')] [2023-10-14 18:13:49,121][61585] Updated weights for policy 1, policy_version 15460 (0.0008) [2023-10-14 18:13:49,497][61585] Updated weights for policy 1, policy_version 15470 (0.0008) [2023-10-14 18:13:49,861][61585] Updated weights for policy 1, policy_version 15480 (0.0009) [2023-10-14 18:13:51,400][61552] Updated weights for policy 0, policy_version 15522 (0.0008) [2023-10-14 18:13:51,770][61552] Updated weights for policy 0, policy_version 15532 (0.0008) [2023-10-14 18:13:52,135][61552] Updated weights for policy 0, policy_version 15542 (0.0007) [2023-10-14 18:13:52,503][61552] Updated weights for policy 0, policy_version 15552 (0.0008) [2023-10-14 18:13:53,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 31784960. Throughput: 0: 1664.3, 1: 1663.8. Samples: 7952800. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:13:53,344][60425] Avg episode reward: [(0, '43.700'), (1, '56.610')] [2023-10-14 18:13:53,964][61585] Updated weights for policy 1, policy_version 15490 (0.0009) [2023-10-14 18:13:54,336][61585] Updated weights for policy 1, policy_version 15500 (0.0009) [2023-10-14 18:13:54,715][61585] Updated weights for policy 1, policy_version 15510 (0.0010) [2023-10-14 18:13:55,081][61585] Updated weights for policy 1, policy_version 15520 (0.0010) [2023-10-14 18:13:56,438][61552] Updated weights for policy 0, policy_version 15562 (0.0008) [2023-10-14 18:13:56,803][61552] Updated weights for policy 0, policy_version 15572 (0.0009) [2023-10-14 18:13:57,168][61552] Updated weights for policy 0, policy_version 15582 (0.0007) [2023-10-14 18:13:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 31850496. Throughput: 0: 1660.4, 1: 1666.8. Samples: 7972490. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:13:58,344][60425] Avg episode reward: [(0, '41.300'), (1, '54.780')] [2023-10-14 18:13:59,186][61585] Updated weights for policy 1, policy_version 15530 (0.0007) [2023-10-14 18:13:59,554][61585] Updated weights for policy 1, policy_version 15540 (0.0007) [2023-10-14 18:13:59,920][61585] Updated weights for policy 1, policy_version 15550 (0.0007) [2023-10-14 18:14:01,406][61552] Updated weights for policy 0, policy_version 15592 (0.0008) [2023-10-14 18:14:01,785][61552] Updated weights for policy 0, policy_version 15602 (0.0010) [2023-10-14 18:14:02,166][61552] Updated weights for policy 0, policy_version 15612 (0.0010) [2023-10-14 18:14:03,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 31916032. Throughput: 0: 1676.6, 1: 1667.9. Samples: 7982938. Policy #0 lag: (min: 21.0, avg: 23.8, max: 53.0) [2023-10-14 18:14:03,344][60425] Avg episode reward: [(0, '44.390'), (1, '56.620')] [2023-10-14 18:14:03,942][61585] Updated weights for policy 1, policy_version 15560 (0.0008) [2023-10-14 18:14:04,295][61585] Updated weights for policy 1, policy_version 15570 (0.0008) [2023-10-14 18:14:04,671][61585] Updated weights for policy 1, policy_version 15580 (0.0009) [2023-10-14 18:14:06,094][61552] Updated weights for policy 0, policy_version 15622 (0.0008) [2023-10-14 18:14:06,472][61552] Updated weights for policy 0, policy_version 15632 (0.0007) [2023-10-14 18:14:06,840][61552] Updated weights for policy 0, policy_version 15642 (0.0009) [2023-10-14 18:14:08,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 31981568. Throughput: 0: 1656.4, 1: 1674.6. Samples: 8002640. Policy #0 lag: (min: 21.0, avg: 23.8, max: 53.0) [2023-10-14 18:14:08,344][60425] Avg episode reward: [(0, '43.210'), (1, '55.380')] [2023-10-14 18:14:08,653][61585] Updated weights for policy 1, policy_version 15590 (0.0011) [2023-10-14 18:14:09,023][61585] Updated weights for policy 1, policy_version 15600 (0.0011) [2023-10-14 18:14:09,384][61585] Updated weights for policy 1, policy_version 15610 (0.0011) [2023-10-14 18:14:10,633][61552] Updated weights for policy 0, policy_version 15652 (0.0008) [2023-10-14 18:14:11,000][61552] Updated weights for policy 0, policy_version 15662 (0.0007) [2023-10-14 18:14:11,366][61552] Updated weights for policy 0, policy_version 15672 (0.0008) [2023-10-14 18:14:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 32047104. Throughput: 0: 1680.4, 1: 1676.8. Samples: 8023404. Policy #0 lag: (min: 21.0, avg: 23.8, max: 53.0) [2023-10-14 18:14:13,344][60425] Avg episode reward: [(0, '47.670'), (1, '57.180')] [2023-10-14 18:14:13,351][61172] Saving new best policy, reward=47.670! [2023-10-14 18:14:13,550][61585] Updated weights for policy 1, policy_version 15620 (0.0009) [2023-10-14 18:14:13,920][61585] Updated weights for policy 1, policy_version 15630 (0.0010) [2023-10-14 18:14:14,278][61585] Updated weights for policy 1, policy_version 15640 (0.0008) [2023-10-14 18:14:14,561][61248] Saving new best policy, reward=57.180! [2023-10-14 18:14:15,482][61552] Updated weights for policy 0, policy_version 15682 (0.0007) [2023-10-14 18:14:15,855][61552] Updated weights for policy 0, policy_version 15692 (0.0011) [2023-10-14 18:14:16,239][61552] Updated weights for policy 0, policy_version 15702 (0.0008) [2023-10-14 18:14:16,608][61552] Updated weights for policy 0, policy_version 15712 (0.0007) [2023-10-14 18:14:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 32112640. Throughput: 0: 1679.2, 1: 1673.3. Samples: 8033374. Policy #0 lag: (min: 1.0, avg: 14.4, max: 33.0) [2023-10-14 18:14:18,344][60425] Avg episode reward: [(0, '44.090'), (1, '56.460')] [2023-10-14 18:14:18,603][61585] Updated weights for policy 1, policy_version 15650 (0.0007) [2023-10-14 18:14:18,979][61585] Updated weights for policy 1, policy_version 15660 (0.0012) [2023-10-14 18:14:19,340][61585] Updated weights for policy 1, policy_version 15670 (0.0007) [2023-10-14 18:14:19,713][61585] Updated weights for policy 1, policy_version 15680 (0.0009) [2023-10-14 18:14:20,768][61552] Updated weights for policy 0, policy_version 15722 (0.0008) [2023-10-14 18:14:21,131][61552] Updated weights for policy 0, policy_version 15732 (0.0008) [2023-10-14 18:14:21,506][61552] Updated weights for policy 0, policy_version 15742 (0.0010) [2023-10-14 18:14:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 32178176. Throughput: 0: 1658.5, 1: 1667.7. Samples: 8052750. Policy #0 lag: (min: 1.0, avg: 14.4, max: 33.0) [2023-10-14 18:14:23,344][60425] Avg episode reward: [(0, '43.390'), (1, '56.680')] [2023-10-14 18:14:23,764][61585] Updated weights for policy 1, policy_version 15690 (0.0007) [2023-10-14 18:14:24,125][61585] Updated weights for policy 1, policy_version 15700 (0.0010) [2023-10-14 18:14:24,490][61585] Updated weights for policy 1, policy_version 15710 (0.0009) [2023-10-14 18:14:25,610][61552] Updated weights for policy 0, policy_version 15752 (0.0007) [2023-10-14 18:14:25,983][61552] Updated weights for policy 0, policy_version 15762 (0.0008) [2023-10-14 18:14:26,345][61552] Updated weights for policy 0, policy_version 15772 (0.0009) [2023-10-14 18:14:28,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 32243712. Throughput: 0: 1678.3, 1: 1665.9. Samples: 8073206. Policy #0 lag: (min: 1.0, avg: 14.4, max: 33.0) [2023-10-14 18:14:28,344][60425] Avg episode reward: [(0, '41.170'), (1, '52.370')] [2023-10-14 18:14:28,543][61585] Updated weights for policy 1, policy_version 15720 (0.0008) [2023-10-14 18:14:28,919][61585] Updated weights for policy 1, policy_version 15730 (0.0009) [2023-10-14 18:14:29,287][61585] Updated weights for policy 1, policy_version 15740 (0.0008) [2023-10-14 18:14:30,513][61552] Updated weights for policy 0, policy_version 15782 (0.0010) [2023-10-14 18:14:30,880][61552] Updated weights for policy 0, policy_version 15792 (0.0008) [2023-10-14 18:14:31,248][61552] Updated weights for policy 0, policy_version 15802 (0.0010) [2023-10-14 18:14:33,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 32309248. Throughput: 0: 1671.5, 1: 1669.5. Samples: 8083084. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-14 18:14:33,344][60425] Avg episode reward: [(0, '44.630'), (1, '53.020')] [2023-10-14 18:14:33,382][61585] Updated weights for policy 1, policy_version 15750 (0.0008) [2023-10-14 18:14:33,771][61585] Updated weights for policy 1, policy_version 15760 (0.0008) [2023-10-14 18:14:34,137][61585] Updated weights for policy 1, policy_version 15770 (0.0009) [2023-10-14 18:14:35,305][61552] Updated weights for policy 0, policy_version 15812 (0.0010) [2023-10-14 18:14:35,682][61552] Updated weights for policy 0, policy_version 15822 (0.0009) [2023-10-14 18:14:36,055][61552] Updated weights for policy 0, policy_version 15832 (0.0007) [2023-10-14 18:14:38,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 32374784. Throughput: 0: 1660.7, 1: 1668.6. Samples: 8102618. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-14 18:14:38,344][60425] Avg episode reward: [(0, '43.800'), (1, '54.340')] [2023-10-14 18:14:38,395][61585] Updated weights for policy 1, policy_version 15780 (0.0009) [2023-10-14 18:14:38,764][61585] Updated weights for policy 1, policy_version 15790 (0.0007) [2023-10-14 18:14:39,129][61585] Updated weights for policy 1, policy_version 15800 (0.0008) [2023-10-14 18:14:40,300][61552] Updated weights for policy 0, policy_version 15842 (0.0008) [2023-10-14 18:14:40,678][61552] Updated weights for policy 0, policy_version 15852 (0.0009) [2023-10-14 18:14:41,052][61552] Updated weights for policy 0, policy_version 15862 (0.0008) [2023-10-14 18:14:41,415][61552] Updated weights for policy 0, policy_version 15872 (0.0009) [2023-10-14 18:14:43,160][61585] Updated weights for policy 1, policy_version 15810 (0.0008) [2023-10-14 18:14:43,344][60425] Fps is (10 sec: 13106.7, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 32440320. Throughput: 0: 1680.2, 1: 1670.2. Samples: 8123260. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-14 18:14:43,345][60425] Avg episode reward: [(0, '44.830'), (1, '54.330')] [2023-10-14 18:14:43,535][61585] Updated weights for policy 1, policy_version 15820 (0.0007) [2023-10-14 18:14:43,894][61585] Updated weights for policy 1, policy_version 15830 (0.0008) [2023-10-14 18:14:44,267][61585] Updated weights for policy 1, policy_version 15840 (0.0010) [2023-10-14 18:14:45,506][61552] Updated weights for policy 0, policy_version 15882 (0.0008) [2023-10-14 18:14:45,885][61552] Updated weights for policy 0, policy_version 15892 (0.0007) [2023-10-14 18:14:46,249][61552] Updated weights for policy 0, policy_version 15902 (0.0009) [2023-10-14 18:14:48,326][61585] Updated weights for policy 1, policy_version 15850 (0.0008) [2023-10-14 18:14:48,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 32505856. Throughput: 0: 1667.5, 1: 1668.3. Samples: 8133050. Policy #0 lag: (min: 7.0, avg: 30.0, max: 32.0) [2023-10-14 18:14:48,344][60425] Avg episode reward: [(0, '45.450'), (1, '54.200')] [2023-10-14 18:14:48,682][61585] Updated weights for policy 1, policy_version 15860 (0.0009) [2023-10-14 18:14:49,053][61585] Updated weights for policy 1, policy_version 15870 (0.0008) [2023-10-14 18:14:50,455][61552] Updated weights for policy 0, policy_version 15912 (0.0008) [2023-10-14 18:14:50,826][61552] Updated weights for policy 0, policy_version 15922 (0.0008) [2023-10-14 18:14:51,192][61552] Updated weights for policy 0, policy_version 15932 (0.0009) [2023-10-14 18:14:53,085][61585] Updated weights for policy 1, policy_version 15880 (0.0007) [2023-10-14 18:14:53,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 32571392. Throughput: 0: 1670.6, 1: 1674.3. Samples: 8153164. Policy #0 lag: (min: 7.0, avg: 30.0, max: 32.0) [2023-10-14 18:14:53,345][60425] Avg episode reward: [(0, '45.640'), (1, '52.510')] [2023-10-14 18:14:53,453][61585] Updated weights for policy 1, policy_version 15890 (0.0008) [2023-10-14 18:14:53,824][61585] Updated weights for policy 1, policy_version 15900 (0.0008) [2023-10-14 18:14:55,498][61552] Updated weights for policy 0, policy_version 15942 (0.0010) [2023-10-14 18:14:55,875][61552] Updated weights for policy 0, policy_version 15952 (0.0008) [2023-10-14 18:14:56,239][61552] Updated weights for policy 0, policy_version 15962 (0.0010) [2023-10-14 18:14:57,793][61585] Updated weights for policy 1, policy_version 15910 (0.0008) [2023-10-14 18:14:58,155][61585] Updated weights for policy 1, policy_version 15920 (0.0007) [2023-10-14 18:14:58,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 32636928. Throughput: 0: 1664.8, 1: 1671.2. Samples: 8173522. Policy #0 lag: (min: 7.0, avg: 30.0, max: 32.0) [2023-10-14 18:14:58,344][60425] Avg episode reward: [(0, '45.180'), (1, '54.530')] [2023-10-14 18:14:58,515][61585] Updated weights for policy 1, policy_version 15930 (0.0007) [2023-10-14 18:15:00,331][61552] Updated weights for policy 0, policy_version 15972 (0.0010) [2023-10-14 18:15:00,700][61552] Updated weights for policy 0, policy_version 15982 (0.0009) [2023-10-14 18:15:01,081][61552] Updated weights for policy 0, policy_version 15992 (0.0011) [2023-10-14 18:15:02,519][61585] Updated weights for policy 1, policy_version 15940 (0.0008) [2023-10-14 18:15:02,884][61585] Updated weights for policy 1, policy_version 15950 (0.0009) [2023-10-14 18:15:03,247][61585] Updated weights for policy 1, policy_version 15960 (0.0008) [2023-10-14 18:15:03,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 32702464. Throughput: 0: 1658.4, 1: 1680.5. Samples: 8183628. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:15:03,344][60425] Avg episode reward: [(0, '47.680'), (1, '54.180')] [2023-10-14 18:15:03,345][61172] Saving new best policy, reward=47.680! [2023-10-14 18:15:04,943][61552] Updated weights for policy 0, policy_version 16002 (0.0010) [2023-10-14 18:15:05,311][61552] Updated weights for policy 0, policy_version 16012 (0.0008) [2023-10-14 18:15:05,676][61552] Updated weights for policy 0, policy_version 16022 (0.0007) [2023-10-14 18:15:06,042][61552] Updated weights for policy 0, policy_version 16032 (0.0008) [2023-10-14 18:15:07,296][61585] Updated weights for policy 1, policy_version 15970 (0.0010) [2023-10-14 18:15:07,673][61585] Updated weights for policy 1, policy_version 15980 (0.0008) [2023-10-14 18:15:08,045][61585] Updated weights for policy 1, policy_version 15990 (0.0008) [2023-10-14 18:15:08,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 32768000. Throughput: 0: 1665.7, 1: 1681.6. Samples: 8203380. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:15:08,344][60425] Avg episode reward: [(0, '45.920'), (1, '52.630')] [2023-10-14 18:15:08,409][61585] Updated weights for policy 1, policy_version 16000 (0.0009) [2023-10-14 18:15:10,103][61552] Updated weights for policy 0, policy_version 16042 (0.0007) [2023-10-14 18:15:10,476][61552] Updated weights for policy 0, policy_version 16052 (0.0007) [2023-10-14 18:15:10,844][61552] Updated weights for policy 0, policy_version 16062 (0.0008) [2023-10-14 18:15:12,476][61585] Updated weights for policy 1, policy_version 16010 (0.0008) [2023-10-14 18:15:12,849][61585] Updated weights for policy 1, policy_version 16020 (0.0008) [2023-10-14 18:15:13,220][61585] Updated weights for policy 1, policy_version 16030 (0.0007) [2023-10-14 18:15:13,343][60425] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 32866304. Throughput: 0: 1668.9, 1: 1667.6. Samples: 8223348. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:15:13,344][60425] Avg episode reward: [(0, '45.630'), (1, '52.910')] [2023-10-14 18:15:15,051][61552] Updated weights for policy 0, policy_version 16072 (0.0009) [2023-10-14 18:15:15,417][61552] Updated weights for policy 0, policy_version 16082 (0.0007) [2023-10-14 18:15:15,795][61552] Updated weights for policy 0, policy_version 16092 (0.0008) [2023-10-14 18:15:17,274][61585] Updated weights for policy 1, policy_version 16040 (0.0008) [2023-10-14 18:15:17,647][61585] Updated weights for policy 1, policy_version 16050 (0.0009) [2023-10-14 18:15:18,006][61585] Updated weights for policy 1, policy_version 16060 (0.0007) [2023-10-14 18:15:18,343][60425] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 32931840. Throughput: 0: 1659.1, 1: 1682.5. Samples: 8233456. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) [2023-10-14 18:15:18,344][60425] Avg episode reward: [(0, '46.890'), (1, '50.790')] [2023-10-14 18:15:19,863][61552] Updated weights for policy 0, policy_version 16102 (0.0009) [2023-10-14 18:15:20,237][61552] Updated weights for policy 0, policy_version 16112 (0.0010) [2023-10-14 18:15:20,606][61552] Updated weights for policy 0, policy_version 16122 (0.0010) [2023-10-14 18:15:22,254][61585] Updated weights for policy 1, policy_version 16070 (0.0007) [2023-10-14 18:15:22,647][61585] Updated weights for policy 1, policy_version 16080 (0.0009) [2023-10-14 18:15:23,016][61585] Updated weights for policy 1, policy_version 16090 (0.0009) [2023-10-14 18:15:23,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 32997376. Throughput: 0: 1666.0, 1: 1685.4. Samples: 8253432. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) [2023-10-14 18:15:23,344][60425] Avg episode reward: [(0, '46.540'), (1, '51.640')] [2023-10-14 18:15:24,768][61552] Updated weights for policy 0, policy_version 16132 (0.0010) [2023-10-14 18:15:25,143][61552] Updated weights for policy 0, policy_version 16142 (0.0007) [2023-10-14 18:15:25,503][61552] Updated weights for policy 0, policy_version 16152 (0.0008) [2023-10-14 18:15:27,149][61585] Updated weights for policy 1, policy_version 16100 (0.0009) [2023-10-14 18:15:27,509][61585] Updated weights for policy 1, policy_version 16110 (0.0009) [2023-10-14 18:15:27,881][61585] Updated weights for policy 1, policy_version 16120 (0.0010) [2023-10-14 18:15:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 33062912. Throughput: 0: 1665.9, 1: 1659.8. Samples: 8272916. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) [2023-10-14 18:15:28,344][60425] Avg episode reward: [(0, '46.720'), (1, '53.490')] [2023-10-14 18:15:29,482][61552] Updated weights for policy 0, policy_version 16162 (0.0010) [2023-10-14 18:15:29,853][61552] Updated weights for policy 0, policy_version 16172 (0.0010) [2023-10-14 18:15:30,224][61552] Updated weights for policy 0, policy_version 16182 (0.0007) [2023-10-14 18:15:30,583][61552] Updated weights for policy 0, policy_version 16192 (0.0007) [2023-10-14 18:15:32,154][61585] Updated weights for policy 1, policy_version 16130 (0.0008) [2023-10-14 18:15:32,521][61585] Updated weights for policy 1, policy_version 16140 (0.0010) [2023-10-14 18:15:32,890][61585] Updated weights for policy 1, policy_version 16150 (0.0009) [2023-10-14 18:15:33,261][61585] Updated weights for policy 1, policy_version 16160 (0.0008) [2023-10-14 18:15:33,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 33128448. Throughput: 0: 1652.1, 1: 1673.3. Samples: 8282692. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:15:33,344][60425] Avg episode reward: [(0, '46.520'), (1, '51.380')] [2023-10-14 18:15:34,389][61552] Updated weights for policy 0, policy_version 16202 (0.0008) [2023-10-14 18:15:34,748][61552] Updated weights for policy 0, policy_version 16212 (0.0010) [2023-10-14 18:15:35,115][61552] Updated weights for policy 0, policy_version 16222 (0.0009) [2023-10-14 18:15:37,412][61585] Updated weights for policy 1, policy_version 16170 (0.0011) [2023-10-14 18:15:37,774][61585] Updated weights for policy 1, policy_version 16180 (0.0011) [2023-10-14 18:15:38,150][61585] Updated weights for policy 1, policy_version 16190 (0.0011) [2023-10-14 18:15:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 33193984. Throughput: 0: 1672.9, 1: 1661.7. Samples: 8303224. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:15:38,344][60425] Avg episode reward: [(0, '48.060'), (1, '51.510')] [2023-10-14 18:15:38,345][61172] Saving new best policy, reward=48.060! [2023-10-14 18:15:39,313][61552] Updated weights for policy 0, policy_version 16232 (0.0008) [2023-10-14 18:15:39,678][61552] Updated weights for policy 0, policy_version 16242 (0.0010) [2023-10-14 18:15:40,048][61552] Updated weights for policy 0, policy_version 16252 (0.0010) [2023-10-14 18:15:42,307][61585] Updated weights for policy 1, policy_version 16200 (0.0008) [2023-10-14 18:15:42,673][61585] Updated weights for policy 1, policy_version 16210 (0.0010) [2023-10-14 18:15:43,034][61585] Updated weights for policy 1, policy_version 16220 (0.0010) [2023-10-14 18:15:43,344][60425] Fps is (10 sec: 13106.7, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 33259520. Throughput: 0: 1677.5, 1: 1644.1. Samples: 8322996. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:15:43,345][60425] Avg episode reward: [(0, '50.530'), (1, '53.370')] [2023-10-14 18:15:43,357][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000016224_16613376.pth... [2023-10-14 18:15:43,357][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000016256_16646144.pth... [2023-10-14 18:15:43,386][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000014656_15007744.pth [2023-10-14 18:15:43,391][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000014688_15040512.pth [2023-10-14 18:15:43,394][61172] Saving new best policy, reward=50.530! [2023-10-14 18:15:44,288][61552] Updated weights for policy 0, policy_version 16262 (0.0009) [2023-10-14 18:15:44,679][61552] Updated weights for policy 0, policy_version 16272 (0.0009) [2023-10-14 18:15:45,049][61552] Updated weights for policy 0, policy_version 16282 (0.0007) [2023-10-14 18:15:47,268][61585] Updated weights for policy 1, policy_version 16230 (0.0008) [2023-10-14 18:15:47,641][61585] Updated weights for policy 1, policy_version 16240 (0.0008) [2023-10-14 18:15:48,006][61585] Updated weights for policy 1, policy_version 16250 (0.0009) [2023-10-14 18:15:48,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 33325056. Throughput: 0: 1657.6, 1: 1650.8. Samples: 8332504. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-14 18:15:48,344][60425] Avg episode reward: [(0, '49.590'), (1, '53.620')] [2023-10-14 18:15:49,243][61552] Updated weights for policy 0, policy_version 16292 (0.0008) [2023-10-14 18:15:49,612][61552] Updated weights for policy 0, policy_version 16302 (0.0007) [2023-10-14 18:15:49,981][61552] Updated weights for policy 0, policy_version 16312 (0.0007) [2023-10-14 18:15:52,202][61585] Updated weights for policy 1, policy_version 16260 (0.0008) [2023-10-14 18:15:52,566][61585] Updated weights for policy 1, policy_version 16270 (0.0007) [2023-10-14 18:15:52,924][61585] Updated weights for policy 1, policy_version 16280 (0.0007) [2023-10-14 18:15:53,343][60425] Fps is (10 sec: 13107.7, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 33390592. Throughput: 0: 1668.0, 1: 1649.5. Samples: 8352666. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-14 18:15:53,344][60425] Avg episode reward: [(0, '46.780'), (1, '55.560')] [2023-10-14 18:15:54,060][61552] Updated weights for policy 0, policy_version 16322 (0.0010) [2023-10-14 18:15:54,430][61552] Updated weights for policy 0, policy_version 16332 (0.0007) [2023-10-14 18:15:54,796][61552] Updated weights for policy 0, policy_version 16342 (0.0010) [2023-10-14 18:15:55,167][61552] Updated weights for policy 0, policy_version 16352 (0.0009) [2023-10-14 18:15:57,058][61585] Updated weights for policy 1, policy_version 16290 (0.0009) [2023-10-14 18:15:57,418][61585] Updated weights for policy 1, policy_version 16300 (0.0007) [2023-10-14 18:15:57,779][61585] Updated weights for policy 1, policy_version 16310 (0.0009) [2023-10-14 18:15:58,140][61585] Updated weights for policy 1, policy_version 16320 (0.0011) [2023-10-14 18:15:58,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 33456128. Throughput: 0: 1667.7, 1: 1644.3. Samples: 8372386. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-14 18:15:58,344][60425] Avg episode reward: [(0, '46.550'), (1, '56.280')] [2023-10-14 18:15:59,362][61552] Updated weights for policy 0, policy_version 16362 (0.0011) [2023-10-14 18:15:59,730][61552] Updated weights for policy 0, policy_version 16372 (0.0009) [2023-10-14 18:16:00,107][61552] Updated weights for policy 0, policy_version 16382 (0.0007) [2023-10-14 18:16:02,426][61585] Updated weights for policy 1, policy_version 16330 (0.0011) [2023-10-14 18:16:02,798][61585] Updated weights for policy 1, policy_version 16340 (0.0007) [2023-10-14 18:16:03,160][61585] Updated weights for policy 1, policy_version 16350 (0.0008) [2023-10-14 18:16:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 33521664. Throughput: 0: 1658.9, 1: 1643.6. Samples: 8382070. Policy #0 lag: (min: 8.0, avg: 30.0, max: 40.0) [2023-10-14 18:16:03,344][60425] Avg episode reward: [(0, '46.080'), (1, '54.640')] [2023-10-14 18:16:04,122][61552] Updated weights for policy 0, policy_version 16392 (0.0009) [2023-10-14 18:16:04,486][61552] Updated weights for policy 0, policy_version 16402 (0.0007) [2023-10-14 18:16:04,839][61552] Updated weights for policy 0, policy_version 16412 (0.0007) [2023-10-14 18:16:07,367][61585] Updated weights for policy 1, policy_version 16360 (0.0010) [2023-10-14 18:16:07,741][61585] Updated weights for policy 1, policy_version 16370 (0.0007) [2023-10-14 18:16:08,106][61585] Updated weights for policy 1, policy_version 16380 (0.0007) [2023-10-14 18:16:08,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 33587200. Throughput: 0: 1671.8, 1: 1646.0. Samples: 8402734. Policy #0 lag: (min: 8.0, avg: 30.0, max: 40.0) [2023-10-14 18:16:08,344][60425] Avg episode reward: [(0, '48.880'), (1, '54.450')] [2023-10-14 18:16:08,811][61552] Updated weights for policy 0, policy_version 16422 (0.0007) [2023-10-14 18:16:09,176][61552] Updated weights for policy 0, policy_version 16432 (0.0009) [2023-10-14 18:16:09,552][61552] Updated weights for policy 0, policy_version 16442 (0.0009) [2023-10-14 18:16:12,185][61585] Updated weights for policy 1, policy_version 16390 (0.0008) [2023-10-14 18:16:12,546][61585] Updated weights for policy 1, policy_version 16400 (0.0010) [2023-10-14 18:16:12,910][61585] Updated weights for policy 1, policy_version 16410 (0.0007) [2023-10-14 18:16:13,343][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 33652736. Throughput: 0: 1675.9, 1: 1648.1. Samples: 8422500. Policy #0 lag: (min: 8.0, avg: 30.0, max: 40.0) [2023-10-14 18:16:13,345][60425] Avg episode reward: [(0, '45.170'), (1, '55.770')] [2023-10-14 18:16:13,565][61552] Updated weights for policy 0, policy_version 16452 (0.0008) [2023-10-14 18:16:13,942][61552] Updated weights for policy 0, policy_version 16462 (0.0009) [2023-10-14 18:16:14,312][61552] Updated weights for policy 0, policy_version 16472 (0.0010) [2023-10-14 18:16:17,050][61585] Updated weights for policy 1, policy_version 16420 (0.0008) [2023-10-14 18:16:17,418][61585] Updated weights for policy 1, policy_version 16430 (0.0009) [2023-10-14 18:16:17,770][61585] Updated weights for policy 1, policy_version 16440 (0.0008) [2023-10-14 18:16:18,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 33718272. Throughput: 0: 1672.8, 1: 1649.8. Samples: 8432206. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-14 18:16:18,344][60425] Avg episode reward: [(0, '45.920'), (1, '55.620')] [2023-10-14 18:16:18,426][61552] Updated weights for policy 0, policy_version 16482 (0.0010) [2023-10-14 18:16:18,790][61552] Updated weights for policy 0, policy_version 16492 (0.0007) [2023-10-14 18:16:19,158][61552] Updated weights for policy 0, policy_version 16502 (0.0007) [2023-10-14 18:16:19,528][61552] Updated weights for policy 0, policy_version 16512 (0.0010) [2023-10-14 18:16:21,915][61585] Updated weights for policy 1, policy_version 16450 (0.0007) [2023-10-14 18:16:22,282][61585] Updated weights for policy 1, policy_version 16460 (0.0009) [2023-10-14 18:16:22,648][61585] Updated weights for policy 1, policy_version 16470 (0.0009) [2023-10-14 18:16:23,020][61585] Updated weights for policy 1, policy_version 16480 (0.0008) [2023-10-14 18:16:23,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 33783808. Throughput: 0: 1664.9, 1: 1653.2. Samples: 8452542. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-14 18:16:23,344][60425] Avg episode reward: [(0, '46.770'), (1, '55.590')] [2023-10-14 18:16:23,768][61552] Updated weights for policy 0, policy_version 16522 (0.0007) [2023-10-14 18:16:24,129][61552] Updated weights for policy 0, policy_version 16532 (0.0008) [2023-10-14 18:16:24,503][61552] Updated weights for policy 0, policy_version 16542 (0.0009) [2023-10-14 18:16:27,103][61585] Updated weights for policy 1, policy_version 16490 (0.0009) [2023-10-14 18:16:27,475][61585] Updated weights for policy 1, policy_version 16500 (0.0007) [2023-10-14 18:16:27,838][61585] Updated weights for policy 1, policy_version 16510 (0.0009) [2023-10-14 18:16:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 33849344. Throughput: 0: 1670.7, 1: 1644.5. Samples: 8472178. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-14 18:16:28,344][60425] Avg episode reward: [(0, '48.130'), (1, '54.590')] [2023-10-14 18:16:28,524][61552] Updated weights for policy 0, policy_version 16552 (0.0010) [2023-10-14 18:16:28,901][61552] Updated weights for policy 0, policy_version 16562 (0.0008) [2023-10-14 18:16:29,279][61552] Updated weights for policy 0, policy_version 16572 (0.0009) [2023-10-14 18:16:31,943][61585] Updated weights for policy 1, policy_version 16520 (0.0007) [2023-10-14 18:16:32,314][61585] Updated weights for policy 1, policy_version 16530 (0.0008) [2023-10-14 18:16:32,673][61585] Updated weights for policy 1, policy_version 16540 (0.0010) [2023-10-14 18:16:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 33914880. Throughput: 0: 1673.5, 1: 1655.4. Samples: 8482306. Policy #0 lag: (min: 9.0, avg: 17.1, max: 41.0) [2023-10-14 18:16:33,344][60425] Avg episode reward: [(0, '51.620'), (1, '56.470')] [2023-10-14 18:16:33,597][61552] Updated weights for policy 0, policy_version 16582 (0.0008) [2023-10-14 18:16:33,974][61552] Updated weights for policy 0, policy_version 16592 (0.0007) [2023-10-14 18:16:34,340][61552] Updated weights for policy 0, policy_version 16602 (0.0007) [2023-10-14 18:16:34,557][61172] Saving new best policy, reward=51.620! [2023-10-14 18:16:36,713][61585] Updated weights for policy 1, policy_version 16550 (0.0009) [2023-10-14 18:16:37,073][61585] Updated weights for policy 1, policy_version 16560 (0.0009) [2023-10-14 18:16:37,439][61585] Updated weights for policy 1, policy_version 16570 (0.0009) [2023-10-14 18:16:38,281][61552] Updated weights for policy 0, policy_version 16612 (0.0008) [2023-10-14 18:16:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 33980416. Throughput: 0: 1680.7, 1: 1653.1. Samples: 8502684. Policy #0 lag: (min: 9.0, avg: 17.1, max: 41.0) [2023-10-14 18:16:38,344][60425] Avg episode reward: [(0, '50.980'), (1, '54.640')] [2023-10-14 18:16:38,657][61552] Updated weights for policy 0, policy_version 16622 (0.0008) [2023-10-14 18:16:39,020][61552] Updated weights for policy 0, policy_version 16632 (0.0008) [2023-10-14 18:16:41,490][61585] Updated weights for policy 1, policy_version 16580 (0.0010) [2023-10-14 18:16:41,851][61585] Updated weights for policy 1, policy_version 16590 (0.0007) [2023-10-14 18:16:42,215][61585] Updated weights for policy 1, policy_version 16600 (0.0007) [2023-10-14 18:16:43,235][61552] Updated weights for policy 0, policy_version 16642 (0.0008) [2023-10-14 18:16:43,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 34045952. Throughput: 0: 1678.9, 1: 1649.6. Samples: 8522168. Policy #0 lag: (min: 9.0, avg: 17.1, max: 41.0) [2023-10-14 18:16:43,344][60425] Avg episode reward: [(0, '48.830'), (1, '53.220')] [2023-10-14 18:16:43,605][61552] Updated weights for policy 0, policy_version 16652 (0.0007) [2023-10-14 18:16:43,975][61552] Updated weights for policy 0, policy_version 16662 (0.0009) [2023-10-14 18:16:44,351][61552] Updated weights for policy 0, policy_version 16672 (0.0008) [2023-10-14 18:16:46,395][61585] Updated weights for policy 1, policy_version 16610 (0.0008) [2023-10-14 18:16:46,755][61585] Updated weights for policy 1, policy_version 16620 (0.0009) [2023-10-14 18:16:47,130][61585] Updated weights for policy 1, policy_version 16630 (0.0008) [2023-10-14 18:16:47,502][61585] Updated weights for policy 1, policy_version 16640 (0.0009) [2023-10-14 18:16:48,313][61552] Updated weights for policy 0, policy_version 16682 (0.0008) [2023-10-14 18:16:48,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 34111488. Throughput: 0: 1680.3, 1: 1662.9. Samples: 8532514. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-14 18:16:48,344][60425] Avg episode reward: [(0, '50.170'), (1, '53.240')] [2023-10-14 18:16:48,684][61552] Updated weights for policy 0, policy_version 16692 (0.0007) [2023-10-14 18:16:49,062][61552] Updated weights for policy 0, policy_version 16702 (0.0007) [2023-10-14 18:16:51,588][61585] Updated weights for policy 1, policy_version 16650 (0.0011) [2023-10-14 18:16:51,959][61585] Updated weights for policy 1, policy_version 16660 (0.0009) [2023-10-14 18:16:52,321][61585] Updated weights for policy 1, policy_version 16670 (0.0009) [2023-10-14 18:16:53,089][61552] Updated weights for policy 0, policy_version 16712 (0.0010) [2023-10-14 18:16:53,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 34177024. Throughput: 0: 1678.2, 1: 1644.2. Samples: 8552242. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-14 18:16:53,344][60425] Avg episode reward: [(0, '49.730'), (1, '53.120')] [2023-10-14 18:16:53,468][61552] Updated weights for policy 0, policy_version 16722 (0.0009) [2023-10-14 18:16:53,832][61552] Updated weights for policy 0, policy_version 16732 (0.0008) [2023-10-14 18:16:56,667][61585] Updated weights for policy 1, policy_version 16680 (0.0009) [2023-10-14 18:16:57,038][61585] Updated weights for policy 1, policy_version 16690 (0.0007) [2023-10-14 18:16:57,404][61585] Updated weights for policy 1, policy_version 16700 (0.0008) [2023-10-14 18:16:57,841][61552] Updated weights for policy 0, policy_version 16742 (0.0008) [2023-10-14 18:16:58,213][61552] Updated weights for policy 0, policy_version 16752 (0.0010) [2023-10-14 18:16:58,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 34242560. Throughput: 0: 1677.4, 1: 1645.9. Samples: 8572048. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-14 18:16:58,344][60425] Avg episode reward: [(0, '50.900'), (1, '56.410')] [2023-10-14 18:16:58,571][61552] Updated weights for policy 0, policy_version 16762 (0.0010) [2023-10-14 18:17:01,734][61585] Updated weights for policy 1, policy_version 16710 (0.0010) [2023-10-14 18:17:02,100][61585] Updated weights for policy 1, policy_version 16720 (0.0009) [2023-10-14 18:17:02,463][61585] Updated weights for policy 1, policy_version 16730 (0.0009) [2023-10-14 18:17:02,644][61552] Updated weights for policy 0, policy_version 16772 (0.0011) [2023-10-14 18:17:03,011][61552] Updated weights for policy 0, policy_version 16782 (0.0008) [2023-10-14 18:17:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 34308096. Throughput: 0: 1678.3, 1: 1655.8. Samples: 8582240. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-14 18:17:03,344][60425] Avg episode reward: [(0, '51.930'), (1, '52.020')] [2023-10-14 18:17:03,384][61552] Updated weights for policy 0, policy_version 16792 (0.0010) [2023-10-14 18:17:03,671][61172] Saving new best policy, reward=51.930! [2023-10-14 18:17:06,601][61585] Updated weights for policy 1, policy_version 16740 (0.0008) [2023-10-14 18:17:06,963][61585] Updated weights for policy 1, policy_version 16750 (0.0009) [2023-10-14 18:17:07,333][61585] Updated weights for policy 1, policy_version 16760 (0.0008) [2023-10-14 18:17:07,469][61552] Updated weights for policy 0, policy_version 16802 (0.0009) [2023-10-14 18:17:07,830][61552] Updated weights for policy 0, policy_version 16812 (0.0007) [2023-10-14 18:17:08,207][61552] Updated weights for policy 0, policy_version 16822 (0.0007) [2023-10-14 18:17:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 34373632. Throughput: 0: 1684.1, 1: 1650.6. Samples: 8602606. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-14 18:17:08,344][60425] Avg episode reward: [(0, '51.680'), (1, '53.980')] [2023-10-14 18:17:08,585][61552] Updated weights for policy 0, policy_version 16832 (0.0009) [2023-10-14 18:17:11,354][61585] Updated weights for policy 1, policy_version 16770 (0.0009) [2023-10-14 18:17:11,721][61585] Updated weights for policy 1, policy_version 16780 (0.0008) [2023-10-14 18:17:12,094][61585] Updated weights for policy 1, policy_version 16790 (0.0008) [2023-10-14 18:17:12,463][61585] Updated weights for policy 1, policy_version 16800 (0.0008) [2023-10-14 18:17:12,791][61552] Updated weights for policy 0, policy_version 16842 (0.0010) [2023-10-14 18:17:13,161][61552] Updated weights for policy 0, policy_version 16852 (0.0011) [2023-10-14 18:17:13,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 34439168. Throughput: 0: 1673.1, 1: 1657.4. Samples: 8622054. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-14 18:17:13,344][60425] Avg episode reward: [(0, '47.040'), (1, '56.100')] [2023-10-14 18:17:13,526][61552] Updated weights for policy 0, policy_version 16862 (0.0010) [2023-10-14 18:17:16,526][61585] Updated weights for policy 1, policy_version 16810 (0.0010) [2023-10-14 18:17:16,897][61585] Updated weights for policy 1, policy_version 16820 (0.0011) [2023-10-14 18:17:17,265][61585] Updated weights for policy 1, policy_version 16830 (0.0010) [2023-10-14 18:17:17,623][61552] Updated weights for policy 0, policy_version 16872 (0.0009) [2023-10-14 18:17:17,999][61552] Updated weights for policy 0, policy_version 16882 (0.0007) [2023-10-14 18:17:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 34504704. Throughput: 0: 1676.6, 1: 1664.4. Samples: 8632650. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:17:18,344][60425] Avg episode reward: [(0, '50.160'), (1, '52.680')] [2023-10-14 18:17:18,359][61552] Updated weights for policy 0, policy_version 16892 (0.0009) [2023-10-14 18:17:21,248][61585] Updated weights for policy 1, policy_version 16840 (0.0008) [2023-10-14 18:17:21,609][61585] Updated weights for policy 1, policy_version 16850 (0.0009) [2023-10-14 18:17:21,980][61585] Updated weights for policy 1, policy_version 16860 (0.0008) [2023-10-14 18:17:22,495][61552] Updated weights for policy 0, policy_version 16902 (0.0008) [2023-10-14 18:17:22,874][61552] Updated weights for policy 0, policy_version 16912 (0.0010) [2023-10-14 18:17:23,246][61552] Updated weights for policy 0, policy_version 16922 (0.0009) [2023-10-14 18:17:23,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 34570240. Throughput: 0: 1676.8, 1: 1653.2. Samples: 8652532. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:17:23,344][60425] Avg episode reward: [(0, '53.120'), (1, '55.350')] [2023-10-14 18:17:23,470][61172] Saving new best policy, reward=53.120! [2023-10-14 18:17:26,097][61585] Updated weights for policy 1, policy_version 16870 (0.0009) [2023-10-14 18:17:26,465][61585] Updated weights for policy 1, policy_version 16880 (0.0009) [2023-10-14 18:17:26,828][61585] Updated weights for policy 1, policy_version 16890 (0.0008) [2023-10-14 18:17:27,274][61552] Updated weights for policy 0, policy_version 16932 (0.0008) [2023-10-14 18:17:27,652][61552] Updated weights for policy 0, policy_version 16942 (0.0007) [2023-10-14 18:17:28,016][61552] Updated weights for policy 0, policy_version 16952 (0.0008) [2023-10-14 18:17:28,343][60425] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 34668544. Throughput: 0: 1663.5, 1: 1667.3. Samples: 8672054. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-14 18:17:28,344][60425] Avg episode reward: [(0, '49.140'), (1, '56.650')] [2023-10-14 18:17:30,812][61585] Updated weights for policy 1, policy_version 16900 (0.0008) [2023-10-14 18:17:31,180][61585] Updated weights for policy 1, policy_version 16910 (0.0009) [2023-10-14 18:17:31,540][61585] Updated weights for policy 1, policy_version 16920 (0.0008) [2023-10-14 18:17:32,249][61552] Updated weights for policy 0, policy_version 16962 (0.0008) [2023-10-14 18:17:32,613][61552] Updated weights for policy 0, policy_version 16972 (0.0009) [2023-10-14 18:17:32,984][61552] Updated weights for policy 0, policy_version 16982 (0.0007) [2023-10-14 18:17:33,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 34701312. Throughput: 0: 1674.2, 1: 1668.4. Samples: 8682934. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-14 18:17:33,344][60425] Avg episode reward: [(0, '51.180'), (1, '55.050')] [2023-10-14 18:17:33,354][61552] Updated weights for policy 0, policy_version 16992 (0.0009) [2023-10-14 18:17:35,421][61585] Updated weights for policy 1, policy_version 16930 (0.0009) [2023-10-14 18:17:35,791][61585] Updated weights for policy 1, policy_version 16940 (0.0009) [2023-10-14 18:17:36,160][61585] Updated weights for policy 1, policy_version 16950 (0.0009) [2023-10-14 18:17:36,520][61585] Updated weights for policy 1, policy_version 16960 (0.0009) [2023-10-14 18:17:37,419][61552] Updated weights for policy 0, policy_version 17002 (0.0008) [2023-10-14 18:17:37,792][61552] Updated weights for policy 0, policy_version 17012 (0.0010) [2023-10-14 18:17:38,162][61552] Updated weights for policy 0, policy_version 17022 (0.0008) [2023-10-14 18:17:38,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 34799616. Throughput: 0: 1676.0, 1: 1663.6. Samples: 8702526. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-14 18:17:38,344][60425] Avg episode reward: [(0, '51.020'), (1, '54.960')] [2023-10-14 18:17:40,659][61585] Updated weights for policy 1, policy_version 16970 (0.0009) [2023-10-14 18:17:41,019][61585] Updated weights for policy 1, policy_version 16980 (0.0008) [2023-10-14 18:17:41,391][61585] Updated weights for policy 1, policy_version 16990 (0.0009) [2023-10-14 18:17:42,235][61552] Updated weights for policy 0, policy_version 17032 (0.0007) [2023-10-14 18:17:42,595][61552] Updated weights for policy 0, policy_version 17042 (0.0009) [2023-10-14 18:17:42,964][61552] Updated weights for policy 0, policy_version 17052 (0.0010) [2023-10-14 18:17:43,343][60425] Fps is (10 sec: 16384.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 34865152. Throughput: 0: 1653.8, 1: 1682.3. Samples: 8722170. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-14 18:17:43,344][60425] Avg episode reward: [(0, '50.530'), (1, '56.850')] [2023-10-14 18:17:43,353][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000016992_17399808.pth... [2023-10-14 18:17:43,353][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000017056_17465344.pth... [2023-10-14 18:17:43,393][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000015424_15794176.pth [2023-10-14 18:17:43,393][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000015488_15859712.pth [2023-10-14 18:17:45,546][61585] Updated weights for policy 1, policy_version 17000 (0.0008) [2023-10-14 18:17:45,929][61585] Updated weights for policy 1, policy_version 17010 (0.0008) [2023-10-14 18:17:46,286][61585] Updated weights for policy 1, policy_version 17020 (0.0008) [2023-10-14 18:17:46,989][61552] Updated weights for policy 0, policy_version 17062 (0.0008) [2023-10-14 18:17:47,360][61552] Updated weights for policy 0, policy_version 17072 (0.0007) [2023-10-14 18:17:47,738][61552] Updated weights for policy 0, policy_version 17082 (0.0009) [2023-10-14 18:17:48,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 34930688. Throughput: 0: 1671.2, 1: 1669.1. Samples: 8732554. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-14 18:17:48,344][60425] Avg episode reward: [(0, '53.280'), (1, '53.910')] [2023-10-14 18:17:48,344][61172] Saving new best policy, reward=53.280! [2023-10-14 18:17:50,517][61585] Updated weights for policy 1, policy_version 17030 (0.0008) [2023-10-14 18:17:50,876][61585] Updated weights for policy 1, policy_version 17040 (0.0009) [2023-10-14 18:17:51,244][61585] Updated weights for policy 1, policy_version 17050 (0.0008) [2023-10-14 18:17:51,875][61552] Updated weights for policy 0, policy_version 17092 (0.0011) [2023-10-14 18:17:52,251][61552] Updated weights for policy 0, policy_version 17102 (0.0010) [2023-10-14 18:17:52,617][61552] Updated weights for policy 0, policy_version 17112 (0.0009) [2023-10-14 18:17:53,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 34996224. Throughput: 0: 1673.2, 1: 1652.4. Samples: 8752254. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-14 18:17:53,344][60425] Avg episode reward: [(0, '50.640'), (1, '56.330')] [2023-10-14 18:17:55,402][61585] Updated weights for policy 1, policy_version 17060 (0.0009) [2023-10-14 18:17:55,765][61585] Updated weights for policy 1, policy_version 17070 (0.0007) [2023-10-14 18:17:56,135][61585] Updated weights for policy 1, policy_version 17080 (0.0008) [2023-10-14 18:17:56,597][61552] Updated weights for policy 0, policy_version 17122 (0.0011) [2023-10-14 18:17:56,973][61552] Updated weights for policy 0, policy_version 17132 (0.0009) [2023-10-14 18:17:57,349][61552] Updated weights for policy 0, policy_version 17142 (0.0007) [2023-10-14 18:17:57,722][61552] Updated weights for policy 0, policy_version 17152 (0.0010) [2023-10-14 18:17:58,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 35061760. Throughput: 0: 1656.5, 1: 1676.6. Samples: 8772044. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-14 18:17:58,344][60425] Avg episode reward: [(0, '54.210'), (1, '55.420')] [2023-10-14 18:17:58,354][61172] Saving new best policy, reward=54.210! [2023-10-14 18:18:00,229][61585] Updated weights for policy 1, policy_version 17090 (0.0008) [2023-10-14 18:18:00,611][61585] Updated weights for policy 1, policy_version 17100 (0.0008) [2023-10-14 18:18:00,975][61585] Updated weights for policy 1, policy_version 17110 (0.0009) [2023-10-14 18:18:01,336][61585] Updated weights for policy 1, policy_version 17120 (0.0009) [2023-10-14 18:18:01,864][61552] Updated weights for policy 0, policy_version 17162 (0.0011) [2023-10-14 18:18:02,233][61552] Updated weights for policy 0, policy_version 17172 (0.0010) [2023-10-14 18:18:02,601][61552] Updated weights for policy 0, policy_version 17182 (0.0010) [2023-10-14 18:18:03,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 35127296. Throughput: 0: 1678.0, 1: 1657.1. Samples: 8782728. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-14 18:18:03,344][60425] Avg episode reward: [(0, '54.780'), (1, '55.270')] [2023-10-14 18:18:03,344][61172] Saving new best policy, reward=54.780! [2023-10-14 18:18:05,656][61585] Updated weights for policy 1, policy_version 17130 (0.0009) [2023-10-14 18:18:06,010][61585] Updated weights for policy 1, policy_version 17140 (0.0008) [2023-10-14 18:18:06,386][61585] Updated weights for policy 1, policy_version 17150 (0.0010) [2023-10-14 18:18:06,618][61552] Updated weights for policy 0, policy_version 17192 (0.0007) [2023-10-14 18:18:06,985][61552] Updated weights for policy 0, policy_version 17202 (0.0009) [2023-10-14 18:18:07,354][61552] Updated weights for policy 0, policy_version 17212 (0.0009) [2023-10-14 18:18:08,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 35192832. Throughput: 0: 1669.2, 1: 1655.6. Samples: 8802150. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-14 18:18:08,344][60425] Avg episode reward: [(0, '49.720'), (1, '56.420')] [2023-10-14 18:18:10,497][61585] Updated weights for policy 1, policy_version 17160 (0.0008) [2023-10-14 18:18:10,862][61585] Updated weights for policy 1, policy_version 17170 (0.0009) [2023-10-14 18:18:11,228][61585] Updated weights for policy 1, policy_version 17180 (0.0008) [2023-10-14 18:18:11,512][61552] Updated weights for policy 0, policy_version 17222 (0.0009) [2023-10-14 18:18:11,893][61552] Updated weights for policy 0, policy_version 17232 (0.0010) [2023-10-14 18:18:12,271][61552] Updated weights for policy 0, policy_version 17242 (0.0009) [2023-10-14 18:18:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 35258368. Throughput: 0: 1662.0, 1: 1670.0. Samples: 8821998. Policy #0 lag: (min: 26.0, avg: 26.6, max: 39.0) [2023-10-14 18:18:13,344][60425] Avg episode reward: [(0, '50.760'), (1, '54.830')] [2023-10-14 18:18:15,439][61585] Updated weights for policy 1, policy_version 17190 (0.0008) [2023-10-14 18:18:15,799][61585] Updated weights for policy 1, policy_version 17200 (0.0007) [2023-10-14 18:18:16,174][61585] Updated weights for policy 1, policy_version 17210 (0.0009) [2023-10-14 18:18:16,355][61552] Updated weights for policy 0, policy_version 17252 (0.0008) [2023-10-14 18:18:16,727][61552] Updated weights for policy 0, policy_version 17262 (0.0008) [2023-10-14 18:18:17,100][61552] Updated weights for policy 0, policy_version 17272 (0.0007) [2023-10-14 18:18:18,344][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 35323904. Throughput: 0: 1678.7, 1: 1656.5. Samples: 8833018. Policy #0 lag: (min: 26.0, avg: 26.6, max: 39.0) [2023-10-14 18:18:18,345][60425] Avg episode reward: [(0, '53.160'), (1, '54.110')] [2023-10-14 18:18:20,153][61585] Updated weights for policy 1, policy_version 17220 (0.0010) [2023-10-14 18:18:20,524][61585] Updated weights for policy 1, policy_version 17230 (0.0010) [2023-10-14 18:18:20,892][61585] Updated weights for policy 1, policy_version 17240 (0.0010) [2023-10-14 18:18:21,279][61552] Updated weights for policy 0, policy_version 17282 (0.0008) [2023-10-14 18:18:21,659][61552] Updated weights for policy 0, policy_version 17292 (0.0010) [2023-10-14 18:18:22,019][61552] Updated weights for policy 0, policy_version 17302 (0.0011) [2023-10-14 18:18:22,391][61552] Updated weights for policy 0, policy_version 17312 (0.0008) [2023-10-14 18:18:23,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 35389440. Throughput: 0: 1663.5, 1: 1664.3. Samples: 8852276. Policy #0 lag: (min: 26.0, avg: 26.6, max: 39.0) [2023-10-14 18:18:23,344][60425] Avg episode reward: [(0, '52.770'), (1, '54.300')] [2023-10-14 18:18:24,941][61585] Updated weights for policy 1, policy_version 17250 (0.0008) [2023-10-14 18:18:25,313][61585] Updated weights for policy 1, policy_version 17260 (0.0008) [2023-10-14 18:18:25,680][61585] Updated weights for policy 1, policy_version 17270 (0.0007) [2023-10-14 18:18:26,044][61585] Updated weights for policy 1, policy_version 17280 (0.0008) [2023-10-14 18:18:26,442][61552] Updated weights for policy 0, policy_version 17322 (0.0012) [2023-10-14 18:18:26,817][61552] Updated weights for policy 0, policy_version 17332 (0.0008) [2023-10-14 18:18:27,191][61552] Updated weights for policy 0, policy_version 17342 (0.0007) [2023-10-14 18:18:28,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 35454976. Throughput: 0: 1664.4, 1: 1663.3. Samples: 8871916. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-14 18:18:28,344][60425] Avg episode reward: [(0, '52.630'), (1, '57.010')] [2023-10-14 18:18:30,128][61585] Updated weights for policy 1, policy_version 17290 (0.0007) [2023-10-14 18:18:30,491][61585] Updated weights for policy 1, policy_version 17300 (0.0007) [2023-10-14 18:18:30,850][61585] Updated weights for policy 1, policy_version 17310 (0.0007) [2023-10-14 18:18:31,303][61552] Updated weights for policy 0, policy_version 17352 (0.0010) [2023-10-14 18:18:31,672][61552] Updated weights for policy 0, policy_version 17362 (0.0009) [2023-10-14 18:18:32,032][61552] Updated weights for policy 0, policy_version 17372 (0.0010) [2023-10-14 18:18:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 35520512. Throughput: 0: 1672.0, 1: 1656.7. Samples: 8882348. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-14 18:18:33,345][60425] Avg episode reward: [(0, '51.250'), (1, '53.780')] [2023-10-14 18:18:35,095][61585] Updated weights for policy 1, policy_version 17320 (0.0007) [2023-10-14 18:18:35,470][61585] Updated weights for policy 1, policy_version 17330 (0.0008) [2023-10-14 18:18:35,834][61585] Updated weights for policy 1, policy_version 17340 (0.0008) [2023-10-14 18:18:36,132][61552] Updated weights for policy 0, policy_version 17382 (0.0009) [2023-10-14 18:18:36,493][61552] Updated weights for policy 0, policy_version 17392 (0.0008) [2023-10-14 18:18:36,862][61552] Updated weights for policy 0, policy_version 17402 (0.0007) [2023-10-14 18:18:38,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 35586048. Throughput: 0: 1646.8, 1: 1671.4. Samples: 8901574. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-14 18:18:38,344][60425] Avg episode reward: [(0, '53.300'), (1, '57.850')] [2023-10-14 18:18:38,345][61248] Saving new best policy, reward=57.850! [2023-10-14 18:18:39,915][61585] Updated weights for policy 1, policy_version 17350 (0.0009) [2023-10-14 18:18:40,277][61585] Updated weights for policy 1, policy_version 17360 (0.0012) [2023-10-14 18:18:40,647][61585] Updated weights for policy 1, policy_version 17370 (0.0009) [2023-10-14 18:18:40,987][61552] Updated weights for policy 0, policy_version 17412 (0.0009) [2023-10-14 18:18:41,356][61552] Updated weights for policy 0, policy_version 17422 (0.0009) [2023-10-14 18:18:41,733][61552] Updated weights for policy 0, policy_version 17432 (0.0008) [2023-10-14 18:18:43,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 35651584. Throughput: 0: 1657.5, 1: 1663.4. Samples: 8921486. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-14 18:18:43,344][60425] Avg episode reward: [(0, '54.970'), (1, '56.690')] [2023-10-14 18:18:43,354][61172] Saving new best policy, reward=54.970! [2023-10-14 18:18:44,884][61585] Updated weights for policy 1, policy_version 17380 (0.0008) [2023-10-14 18:18:45,257][61585] Updated weights for policy 1, policy_version 17390 (0.0009) [2023-10-14 18:18:45,620][61585] Updated weights for policy 1, policy_version 17400 (0.0008) [2023-10-14 18:18:45,903][61552] Updated weights for policy 0, policy_version 17442 (0.0009) [2023-10-14 18:18:46,271][61552] Updated weights for policy 0, policy_version 17452 (0.0010) [2023-10-14 18:18:46,632][61552] Updated weights for policy 0, policy_version 17462 (0.0007) [2023-10-14 18:18:47,000][61552] Updated weights for policy 0, policy_version 17472 (0.0007) [2023-10-14 18:18:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 35717120. Throughput: 0: 1662.4, 1: 1652.7. Samples: 8931910. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-14 18:18:48,344][60425] Avg episode reward: [(0, '54.070'), (1, '58.730')] [2023-10-14 18:18:48,345][61248] Saving new best policy, reward=58.730! [2023-10-14 18:18:49,775][61585] Updated weights for policy 1, policy_version 17410 (0.0008) [2023-10-14 18:18:50,150][61585] Updated weights for policy 1, policy_version 17420 (0.0008) [2023-10-14 18:18:50,523][61585] Updated weights for policy 1, policy_version 17430 (0.0009) [2023-10-14 18:18:50,886][61585] Updated weights for policy 1, policy_version 17440 (0.0009) [2023-10-14 18:18:51,176][61552] Updated weights for policy 0, policy_version 17482 (0.0008) [2023-10-14 18:18:51,541][61552] Updated weights for policy 0, policy_version 17492 (0.0008) [2023-10-14 18:18:51,898][61552] Updated weights for policy 0, policy_version 17502 (0.0009) [2023-10-14 18:18:53,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 35782656. Throughput: 0: 1648.5, 1: 1660.1. Samples: 8951038. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-14 18:18:53,344][60425] Avg episode reward: [(0, '54.930'), (1, '57.650')] [2023-10-14 18:18:54,972][61585] Updated weights for policy 1, policy_version 17450 (0.0007) [2023-10-14 18:18:55,340][61585] Updated weights for policy 1, policy_version 17460 (0.0007) [2023-10-14 18:18:55,708][61585] Updated weights for policy 1, policy_version 17470 (0.0008) [2023-10-14 18:18:56,244][61552] Updated weights for policy 0, policy_version 17512 (0.0009) [2023-10-14 18:18:56,625][61552] Updated weights for policy 0, policy_version 17522 (0.0009) [2023-10-14 18:18:56,988][61552] Updated weights for policy 0, policy_version 17532 (0.0011) [2023-10-14 18:18:58,344][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 35848192. Throughput: 0: 1654.8, 1: 1656.4. Samples: 8971006. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:18:58,345][60425] Avg episode reward: [(0, '54.020'), (1, '55.290')] [2023-10-14 18:18:59,895][61585] Updated weights for policy 1, policy_version 17480 (0.0008) [2023-10-14 18:19:00,257][61585] Updated weights for policy 1, policy_version 17490 (0.0008) [2023-10-14 18:19:00,628][61585] Updated weights for policy 1, policy_version 17500 (0.0008) [2023-10-14 18:19:01,080][61552] Updated weights for policy 0, policy_version 17542 (0.0010) [2023-10-14 18:19:01,453][61552] Updated weights for policy 0, policy_version 17552 (0.0010) [2023-10-14 18:19:01,825][61552] Updated weights for policy 0, policy_version 17562 (0.0008) [2023-10-14 18:19:03,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 35913728. Throughput: 0: 1655.7, 1: 1642.5. Samples: 8981438. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:19:03,344][60425] Avg episode reward: [(0, '51.920'), (1, '55.900')] [2023-10-14 18:19:04,757][61585] Updated weights for policy 1, policy_version 17510 (0.0008) [2023-10-14 18:19:05,124][61585] Updated weights for policy 1, policy_version 17520 (0.0008) [2023-10-14 18:19:05,493][61585] Updated weights for policy 1, policy_version 17530 (0.0009) [2023-10-14 18:19:05,756][61552] Updated weights for policy 0, policy_version 17572 (0.0008) [2023-10-14 18:19:06,127][61552] Updated weights for policy 0, policy_version 17582 (0.0010) [2023-10-14 18:19:06,502][61552] Updated weights for policy 0, policy_version 17592 (0.0009) [2023-10-14 18:19:08,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 35979264. Throughput: 0: 1649.3, 1: 1656.8. Samples: 9001048. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:19:08,344][60425] Avg episode reward: [(0, '55.400'), (1, '57.230')] [2023-10-14 18:19:08,345][61172] Saving new best policy, reward=55.400! [2023-10-14 18:19:09,727][61585] Updated weights for policy 1, policy_version 17540 (0.0009) [2023-10-14 18:19:10,103][61585] Updated weights for policy 1, policy_version 17550 (0.0010) [2023-10-14 18:19:10,459][61552] Updated weights for policy 0, policy_version 17602 (0.0007) [2023-10-14 18:19:10,462][61585] Updated weights for policy 1, policy_version 17560 (0.0008) [2023-10-14 18:19:10,827][61552] Updated weights for policy 0, policy_version 17612 (0.0009) [2023-10-14 18:19:11,203][61552] Updated weights for policy 0, policy_version 17622 (0.0010) [2023-10-14 18:19:11,570][61552] Updated weights for policy 0, policy_version 17632 (0.0009) [2023-10-14 18:19:13,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 36044800. Throughput: 0: 1664.8, 1: 1662.4. Samples: 9021636. Policy #0 lag: (min: 26.0, avg: 53.3, max: 56.0) [2023-10-14 18:19:13,344][60425] Avg episode reward: [(0, '51.010'), (1, '54.960')] [2023-10-14 18:19:14,421][61585] Updated weights for policy 1, policy_version 17570 (0.0008) [2023-10-14 18:19:14,787][61585] Updated weights for policy 1, policy_version 17580 (0.0008) [2023-10-14 18:19:15,153][61585] Updated weights for policy 1, policy_version 17590 (0.0008) [2023-10-14 18:19:15,521][61585] Updated weights for policy 1, policy_version 17600 (0.0009) [2023-10-14 18:19:15,864][61552] Updated weights for policy 0, policy_version 17642 (0.0009) [2023-10-14 18:19:16,228][61552] Updated weights for policy 0, policy_version 17652 (0.0009) [2023-10-14 18:19:16,605][61552] Updated weights for policy 0, policy_version 17662 (0.0011) [2023-10-14 18:19:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 36110336. Throughput: 0: 1659.4, 1: 1656.1. Samples: 9031548. Policy #0 lag: (min: 26.0, avg: 53.3, max: 56.0) [2023-10-14 18:19:18,344][60425] Avg episode reward: [(0, '50.960'), (1, '58.920')] [2023-10-14 18:19:18,345][61248] Saving new best policy, reward=58.920! [2023-10-14 18:19:19,544][61585] Updated weights for policy 1, policy_version 17610 (0.0011) [2023-10-14 18:19:19,903][61585] Updated weights for policy 1, policy_version 17620 (0.0007) [2023-10-14 18:19:20,269][61585] Updated weights for policy 1, policy_version 17630 (0.0007) [2023-10-14 18:19:20,649][61552] Updated weights for policy 0, policy_version 17672 (0.0010) [2023-10-14 18:19:21,026][61552] Updated weights for policy 0, policy_version 17682 (0.0009) [2023-10-14 18:19:21,391][61552] Updated weights for policy 0, policy_version 17692 (0.0008) [2023-10-14 18:19:23,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 36175872. Throughput: 0: 1658.1, 1: 1666.4. Samples: 9051176. Policy #0 lag: (min: 26.0, avg: 53.3, max: 56.0) [2023-10-14 18:19:23,344][60425] Avg episode reward: [(0, '53.250'), (1, '58.430')] [2023-10-14 18:19:24,402][61585] Updated weights for policy 1, policy_version 17640 (0.0008) [2023-10-14 18:19:24,782][61585] Updated weights for policy 1, policy_version 17650 (0.0009) [2023-10-14 18:19:25,143][61585] Updated weights for policy 1, policy_version 17660 (0.0009) [2023-10-14 18:19:25,484][61552] Updated weights for policy 0, policy_version 17702 (0.0009) [2023-10-14 18:19:25,862][61552] Updated weights for policy 0, policy_version 17712 (0.0009) [2023-10-14 18:19:26,233][61552] Updated weights for policy 0, policy_version 17722 (0.0009) [2023-10-14 18:19:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 36241408. Throughput: 0: 1671.2, 1: 1667.2. Samples: 9071716. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-14 18:19:28,344][60425] Avg episode reward: [(0, '52.740'), (1, '58.210')] [2023-10-14 18:19:29,275][61585] Updated weights for policy 1, policy_version 17670 (0.0008) [2023-10-14 18:19:29,643][61585] Updated weights for policy 1, policy_version 17680 (0.0007) [2023-10-14 18:19:30,003][61585] Updated weights for policy 1, policy_version 17690 (0.0008) [2023-10-14 18:19:30,299][61552] Updated weights for policy 0, policy_version 17732 (0.0009) [2023-10-14 18:19:30,670][61552] Updated weights for policy 0, policy_version 17742 (0.0008) [2023-10-14 18:19:31,040][61552] Updated weights for policy 0, policy_version 17752 (0.0009) [2023-10-14 18:19:33,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 36306944. Throughput: 0: 1658.7, 1: 1666.4. Samples: 9081540. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-14 18:19:33,344][60425] Avg episode reward: [(0, '52.530'), (1, '59.920')] [2023-10-14 18:19:33,344][61248] Saving new best policy, reward=59.920! [2023-10-14 18:19:34,072][61585] Updated weights for policy 1, policy_version 17700 (0.0008) [2023-10-14 18:19:34,441][61585] Updated weights for policy 1, policy_version 17710 (0.0010) [2023-10-14 18:19:34,806][61585] Updated weights for policy 1, policy_version 17720 (0.0010) [2023-10-14 18:19:35,041][61552] Updated weights for policy 0, policy_version 17762 (0.0010) [2023-10-14 18:19:35,409][61552] Updated weights for policy 0, policy_version 17772 (0.0009) [2023-10-14 18:19:35,790][61552] Updated weights for policy 0, policy_version 17782 (0.0010) [2023-10-14 18:19:36,157][61552] Updated weights for policy 0, policy_version 17792 (0.0009) [2023-10-14 18:19:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 36372480. Throughput: 0: 1665.5, 1: 1677.1. Samples: 9101456. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-14 18:19:38,344][60425] Avg episode reward: [(0, '52.040'), (1, '56.430')] [2023-10-14 18:19:38,871][61585] Updated weights for policy 1, policy_version 17730 (0.0008) [2023-10-14 18:19:39,244][61585] Updated weights for policy 1, policy_version 17740 (0.0010) [2023-10-14 18:19:39,608][61585] Updated weights for policy 1, policy_version 17750 (0.0008) [2023-10-14 18:19:39,967][61585] Updated weights for policy 1, policy_version 17760 (0.0007) [2023-10-14 18:19:40,349][61552] Updated weights for policy 0, policy_version 17802 (0.0007) [2023-10-14 18:19:40,724][61552] Updated weights for policy 0, policy_version 17812 (0.0010) [2023-10-14 18:19:41,087][61552] Updated weights for policy 0, policy_version 17822 (0.0007) [2023-10-14 18:19:43,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 36438016. Throughput: 0: 1678.8, 1: 1677.9. Samples: 9122058. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-14 18:19:43,344][60425] Avg episode reward: [(0, '53.650'), (1, '59.110')] [2023-10-14 18:19:43,351][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000017760_18186240.pth... [2023-10-14 18:19:43,352][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000017824_18251776.pth... [2023-10-14 18:19:43,390][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000016256_16646144.pth [2023-10-14 18:19:43,391][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000016224_16613376.pth [2023-10-14 18:19:44,129][61585] Updated weights for policy 1, policy_version 17770 (0.0008) [2023-10-14 18:19:44,504][61585] Updated weights for policy 1, policy_version 17780 (0.0008) [2023-10-14 18:19:44,861][61585] Updated weights for policy 1, policy_version 17790 (0.0011) [2023-10-14 18:19:45,348][61552] Updated weights for policy 0, policy_version 17832 (0.0007) [2023-10-14 18:19:45,731][61552] Updated weights for policy 0, policy_version 17842 (0.0008) [2023-10-14 18:19:46,097][61552] Updated weights for policy 0, policy_version 17852 (0.0009) [2023-10-14 18:19:48,345][60425] Fps is (10 sec: 13104.6, 60 sec: 13106.8, 300 sec: 13329.3). Total num frames: 36503552. Throughput: 0: 1659.3, 1: 1677.4. Samples: 9131594. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-14 18:19:48,346][60425] Avg episode reward: [(0, '54.560'), (1, '56.480')] [2023-10-14 18:19:48,993][61585] Updated weights for policy 1, policy_version 17800 (0.0007) [2023-10-14 18:19:49,362][61585] Updated weights for policy 1, policy_version 17810 (0.0008) [2023-10-14 18:19:49,722][61585] Updated weights for policy 1, policy_version 17820 (0.0007) [2023-10-14 18:19:50,179][61552] Updated weights for policy 0, policy_version 17862 (0.0008) [2023-10-14 18:19:50,546][61552] Updated weights for policy 0, policy_version 17872 (0.0007) [2023-10-14 18:19:50,921][61552] Updated weights for policy 0, policy_version 17882 (0.0007) [2023-10-14 18:19:53,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 36569088. Throughput: 0: 1665.2, 1: 1677.1. Samples: 9151452. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-14 18:19:53,344][60425] Avg episode reward: [(0, '50.300'), (1, '55.050')] [2023-10-14 18:19:53,733][61585] Updated weights for policy 1, policy_version 17830 (0.0008) [2023-10-14 18:19:54,100][61585] Updated weights for policy 1, policy_version 17840 (0.0008) [2023-10-14 18:19:54,463][61585] Updated weights for policy 1, policy_version 17850 (0.0009) [2023-10-14 18:19:54,707][61552] Updated weights for policy 0, policy_version 17892 (0.0008) [2023-10-14 18:19:55,076][61552] Updated weights for policy 0, policy_version 17902 (0.0009) [2023-10-14 18:19:55,437][61552] Updated weights for policy 0, policy_version 17912 (0.0008) [2023-10-14 18:19:58,343][60425] Fps is (10 sec: 13109.5, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 36634624. Throughput: 0: 1674.5, 1: 1673.7. Samples: 9172308. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-14 18:19:58,344][60425] Avg episode reward: [(0, '50.230'), (1, '55.290')] [2023-10-14 18:19:58,637][61585] Updated weights for policy 1, policy_version 17860 (0.0008) [2023-10-14 18:19:59,008][61585] Updated weights for policy 1, policy_version 17870 (0.0007) [2023-10-14 18:19:59,384][61585] Updated weights for policy 1, policy_version 17880 (0.0010) [2023-10-14 18:19:59,640][61552] Updated weights for policy 0, policy_version 17922 (0.0008) [2023-10-14 18:20:00,012][61552] Updated weights for policy 0, policy_version 17932 (0.0010) [2023-10-14 18:20:00,376][61552] Updated weights for policy 0, policy_version 17942 (0.0011) [2023-10-14 18:20:00,746][61552] Updated weights for policy 0, policy_version 17952 (0.0009) [2023-10-14 18:20:03,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 36700160. Throughput: 0: 1657.0, 1: 1673.4. Samples: 9181416. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-14 18:20:03,344][60425] Avg episode reward: [(0, '53.420'), (1, '54.740')] [2023-10-14 18:20:03,545][61585] Updated weights for policy 1, policy_version 17890 (0.0010) [2023-10-14 18:20:03,915][61585] Updated weights for policy 1, policy_version 17900 (0.0008) [2023-10-14 18:20:04,284][61585] Updated weights for policy 1, policy_version 17910 (0.0007) [2023-10-14 18:20:04,649][61585] Updated weights for policy 1, policy_version 17920 (0.0008) [2023-10-14 18:20:04,902][61552] Updated weights for policy 0, policy_version 17962 (0.0011) [2023-10-14 18:20:05,275][61552] Updated weights for policy 0, policy_version 17972 (0.0008) [2023-10-14 18:20:05,638][61552] Updated weights for policy 0, policy_version 17982 (0.0010) [2023-10-14 18:20:08,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 36765696. Throughput: 0: 1678.4, 1: 1670.8. Samples: 9201886. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-14 18:20:08,344][60425] Avg episode reward: [(0, '50.820'), (1, '54.510')] [2023-10-14 18:20:08,620][61585] Updated weights for policy 1, policy_version 17930 (0.0008) [2023-10-14 18:20:08,987][61585] Updated weights for policy 1, policy_version 17940 (0.0009) [2023-10-14 18:20:09,354][61585] Updated weights for policy 1, policy_version 17950 (0.0008) [2023-10-14 18:20:09,716][61552] Updated weights for policy 0, policy_version 17992 (0.0008) [2023-10-14 18:20:10,087][61552] Updated weights for policy 0, policy_version 18002 (0.0007) [2023-10-14 18:20:10,456][61552] Updated weights for policy 0, policy_version 18012 (0.0011) [2023-10-14 18:20:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 36831232. Throughput: 0: 1676.5, 1: 1673.6. Samples: 9222472. Policy #0 lag: (min: 31.0, avg: 54.6, max: 56.0) [2023-10-14 18:20:13,344][60425] Avg episode reward: [(0, '51.830'), (1, '55.120')] [2023-10-14 18:20:13,467][61585] Updated weights for policy 1, policy_version 17960 (0.0007) [2023-10-14 18:20:13,844][61585] Updated weights for policy 1, policy_version 17970 (0.0008) [2023-10-14 18:20:14,226][61585] Updated weights for policy 1, policy_version 17980 (0.0008) [2023-10-14 18:20:14,568][61552] Updated weights for policy 0, policy_version 18022 (0.0008) [2023-10-14 18:20:14,932][61552] Updated weights for policy 0, policy_version 18032 (0.0011) [2023-10-14 18:20:15,313][61552] Updated weights for policy 0, policy_version 18042 (0.0011) [2023-10-14 18:20:18,344][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 36896768. Throughput: 0: 1661.3, 1: 1667.5. Samples: 9231340. Policy #0 lag: (min: 31.0, avg: 54.6, max: 56.0) [2023-10-14 18:20:18,345][60425] Avg episode reward: [(0, '52.910'), (1, '57.120')] [2023-10-14 18:20:18,441][61585] Updated weights for policy 1, policy_version 17990 (0.0008) [2023-10-14 18:20:18,817][61585] Updated weights for policy 1, policy_version 18000 (0.0009) [2023-10-14 18:20:19,191][61585] Updated weights for policy 1, policy_version 18010 (0.0010) [2023-10-14 18:20:19,426][61552] Updated weights for policy 0, policy_version 18052 (0.0008) [2023-10-14 18:20:19,797][61552] Updated weights for policy 0, policy_version 18062 (0.0009) [2023-10-14 18:20:20,160][61552] Updated weights for policy 0, policy_version 18072 (0.0009) [2023-10-14 18:20:23,191][61585] Updated weights for policy 1, policy_version 18020 (0.0008) [2023-10-14 18:20:23,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 36962304. Throughput: 0: 1672.7, 1: 1669.2. Samples: 9251846. Policy #0 lag: (min: 31.0, avg: 54.6, max: 56.0) [2023-10-14 18:20:23,345][60425] Avg episode reward: [(0, '55.240'), (1, '55.300')] [2023-10-14 18:20:23,545][61585] Updated weights for policy 1, policy_version 18030 (0.0008) [2023-10-14 18:20:23,908][61585] Updated weights for policy 1, policy_version 18040 (0.0010) [2023-10-14 18:20:24,291][61552] Updated weights for policy 0, policy_version 18082 (0.0008) [2023-10-14 18:20:24,657][61552] Updated weights for policy 0, policy_version 18092 (0.0008) [2023-10-14 18:20:25,029][61552] Updated weights for policy 0, policy_version 18102 (0.0008) [2023-10-14 18:20:25,398][61552] Updated weights for policy 0, policy_version 18112 (0.0009) [2023-10-14 18:20:27,982][61585] Updated weights for policy 1, policy_version 18050 (0.0009) [2023-10-14 18:20:28,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 37027840. Throughput: 0: 1674.4, 1: 1667.8. Samples: 9272456. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:20:28,344][60425] Avg episode reward: [(0, '53.910'), (1, '58.910')] [2023-10-14 18:20:28,353][61585] Updated weights for policy 1, policy_version 18060 (0.0007) [2023-10-14 18:20:28,730][61585] Updated weights for policy 1, policy_version 18070 (0.0010) [2023-10-14 18:20:29,096][61585] Updated weights for policy 1, policy_version 18080 (0.0010) [2023-10-14 18:20:29,578][61552] Updated weights for policy 0, policy_version 18122 (0.0010) [2023-10-14 18:20:29,941][61552] Updated weights for policy 0, policy_version 18132 (0.0009) [2023-10-14 18:20:30,318][61552] Updated weights for policy 0, policy_version 18142 (0.0008) [2023-10-14 18:20:33,249][61585] Updated weights for policy 1, policy_version 18090 (0.0010) [2023-10-14 18:20:33,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 37093376. Throughput: 0: 1663.9, 1: 1667.6. Samples: 9281506. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:20:33,344][60425] Avg episode reward: [(0, '54.850'), (1, '56.330')] [2023-10-14 18:20:33,616][61585] Updated weights for policy 1, policy_version 18100 (0.0008) [2023-10-14 18:20:33,977][61585] Updated weights for policy 1, policy_version 18110 (0.0009) [2023-10-14 18:20:34,313][61552] Updated weights for policy 0, policy_version 18152 (0.0009) [2023-10-14 18:20:34,686][61552] Updated weights for policy 0, policy_version 18162 (0.0009) [2023-10-14 18:20:35,068][61552] Updated weights for policy 0, policy_version 18172 (0.0008) [2023-10-14 18:20:37,938][61585] Updated weights for policy 1, policy_version 18120 (0.0008) [2023-10-14 18:20:38,306][61585] Updated weights for policy 1, policy_version 18130 (0.0008) [2023-10-14 18:20:38,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 37158912. Throughput: 0: 1681.6, 1: 1669.2. Samples: 9302234. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:20:38,344][60425] Avg episode reward: [(0, '53.940'), (1, '58.950')] [2023-10-14 18:20:38,665][61585] Updated weights for policy 1, policy_version 18140 (0.0008) [2023-10-14 18:20:38,960][61552] Updated weights for policy 0, policy_version 18182 (0.0010) [2023-10-14 18:20:39,327][61552] Updated weights for policy 0, policy_version 18192 (0.0010) [2023-10-14 18:20:39,701][61552] Updated weights for policy 0, policy_version 18202 (0.0008) [2023-10-14 18:20:42,822][61585] Updated weights for policy 1, policy_version 18150 (0.0007) [2023-10-14 18:20:43,187][61585] Updated weights for policy 1, policy_version 18160 (0.0007) [2023-10-14 18:20:43,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 37224448. Throughput: 0: 1673.6, 1: 1661.2. Samples: 9322374. Policy #0 lag: (min: 33.0, avg: 47.1, max: 48.0) [2023-10-14 18:20:43,345][60425] Avg episode reward: [(0, '51.890'), (1, '56.060')] [2023-10-14 18:20:43,558][61585] Updated weights for policy 1, policy_version 18170 (0.0008) [2023-10-14 18:20:43,779][61552] Updated weights for policy 0, policy_version 18212 (0.0009) [2023-10-14 18:20:44,150][61552] Updated weights for policy 0, policy_version 18222 (0.0010) [2023-10-14 18:20:44,517][61552] Updated weights for policy 0, policy_version 18232 (0.0007) [2023-10-14 18:20:47,680][61585] Updated weights for policy 1, policy_version 18180 (0.0009) [2023-10-14 18:20:48,043][61585] Updated weights for policy 1, policy_version 18190 (0.0008) [2023-10-14 18:20:48,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.6, 300 sec: 13218.3). Total num frames: 37289984. Throughput: 0: 1673.1, 1: 1665.4. Samples: 9331648. Policy #0 lag: (min: 33.0, avg: 47.1, max: 48.0) [2023-10-14 18:20:48,344][60425] Avg episode reward: [(0, '54.850'), (1, '57.140')] [2023-10-14 18:20:48,406][61585] Updated weights for policy 1, policy_version 18200 (0.0009) [2023-10-14 18:20:48,700][61552] Updated weights for policy 0, policy_version 18242 (0.0010) [2023-10-14 18:20:49,064][61552] Updated weights for policy 0, policy_version 18252 (0.0008) [2023-10-14 18:20:49,434][61552] Updated weights for policy 0, policy_version 18262 (0.0009) [2023-10-14 18:20:49,808][61552] Updated weights for policy 0, policy_version 18272 (0.0007) [2023-10-14 18:20:52,662][61585] Updated weights for policy 1, policy_version 18210 (0.0009) [2023-10-14 18:20:53,026][61585] Updated weights for policy 1, policy_version 18220 (0.0011) [2023-10-14 18:20:53,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 37355520. Throughput: 0: 1674.9, 1: 1662.0. Samples: 9352048. Policy #0 lag: (min: 33.0, avg: 47.1, max: 48.0) [2023-10-14 18:20:53,344][60425] Avg episode reward: [(0, '52.210'), (1, '51.910')] [2023-10-14 18:20:53,394][61585] Updated weights for policy 1, policy_version 18230 (0.0007) [2023-10-14 18:20:53,760][61585] Updated weights for policy 1, policy_version 18240 (0.0009) [2023-10-14 18:20:53,884][61552] Updated weights for policy 0, policy_version 18282 (0.0008) [2023-10-14 18:20:54,254][61552] Updated weights for policy 0, policy_version 18292 (0.0008) [2023-10-14 18:20:54,616][61552] Updated weights for policy 0, policy_version 18302 (0.0008) [2023-10-14 18:20:57,869][61585] Updated weights for policy 1, policy_version 18250 (0.0008) [2023-10-14 18:20:58,240][61585] Updated weights for policy 1, policy_version 18260 (0.0010) [2023-10-14 18:20:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 37421056. Throughput: 0: 1681.1, 1: 1652.0. Samples: 9372462. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:20:58,344][60425] Avg episode reward: [(0, '54.290'), (1, '53.560')] [2023-10-14 18:20:58,614][61585] Updated weights for policy 1, policy_version 18270 (0.0008) [2023-10-14 18:20:58,621][61552] Updated weights for policy 0, policy_version 18312 (0.0008) [2023-10-14 18:20:58,985][61552] Updated weights for policy 0, policy_version 18322 (0.0009) [2023-10-14 18:20:59,351][61552] Updated weights for policy 0, policy_version 18332 (0.0009) [2023-10-14 18:21:02,859][61585] Updated weights for policy 1, policy_version 18280 (0.0010) [2023-10-14 18:21:03,234][61585] Updated weights for policy 1, policy_version 18290 (0.0011) [2023-10-14 18:21:03,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 37486592. Throughput: 0: 1678.8, 1: 1662.4. Samples: 9381694. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:21:03,344][60425] Avg episode reward: [(0, '51.700'), (1, '53.400')] [2023-10-14 18:21:03,471][61552] Updated weights for policy 0, policy_version 18342 (0.0009) [2023-10-14 18:21:03,587][61585] Updated weights for policy 1, policy_version 18300 (0.0010) [2023-10-14 18:21:03,833][61552] Updated weights for policy 0, policy_version 18352 (0.0008) [2023-10-14 18:21:04,195][61552] Updated weights for policy 0, policy_version 18362 (0.0010) [2023-10-14 18:21:07,824][61585] Updated weights for policy 1, policy_version 18310 (0.0010) [2023-10-14 18:21:08,110][61552] Updated weights for policy 0, policy_version 18372 (0.0010) [2023-10-14 18:21:08,191][61585] Updated weights for policy 1, policy_version 18320 (0.0008) [2023-10-14 18:21:08,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 37552128. Throughput: 0: 1680.0, 1: 1658.9. Samples: 9402094. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:21:08,344][60425] Avg episode reward: [(0, '50.410'), (1, '56.290')] [2023-10-14 18:21:08,477][61552] Updated weights for policy 0, policy_version 18382 (0.0007) [2023-10-14 18:21:08,552][61585] Updated weights for policy 1, policy_version 18330 (0.0008) [2023-10-14 18:21:08,843][61552] Updated weights for policy 0, policy_version 18392 (0.0008) [2023-10-14 18:21:12,648][61585] Updated weights for policy 1, policy_version 18340 (0.0009) [2023-10-14 18:21:12,895][61552] Updated weights for policy 0, policy_version 18402 (0.0007) [2023-10-14 18:21:13,026][61585] Updated weights for policy 1, policy_version 18350 (0.0009) [2023-10-14 18:21:13,261][61552] Updated weights for policy 0, policy_version 18412 (0.0009) [2023-10-14 18:21:13,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 37617664. Throughput: 0: 1683.9, 1: 1652.7. Samples: 9422600. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:21:13,344][60425] Avg episode reward: [(0, '51.220'), (1, '56.580')] [2023-10-14 18:21:13,382][61585] Updated weights for policy 1, policy_version 18360 (0.0009) [2023-10-14 18:21:13,632][61552] Updated weights for policy 0, policy_version 18422 (0.0009) [2023-10-14 18:21:14,001][61552] Updated weights for policy 0, policy_version 18432 (0.0009) [2023-10-14 18:21:17,445][61585] Updated weights for policy 1, policy_version 18370 (0.0009) [2023-10-14 18:21:17,812][61585] Updated weights for policy 1, policy_version 18380 (0.0008) [2023-10-14 18:21:18,178][61585] Updated weights for policy 1, policy_version 18390 (0.0007) [2023-10-14 18:21:18,244][61552] Updated weights for policy 0, policy_version 18442 (0.0008) [2023-10-14 18:21:18,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 37683200. Throughput: 0: 1683.2, 1: 1656.4. Samples: 9431788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:21:18,344][60425] Avg episode reward: [(0, '52.000'), (1, '56.600')] [2023-10-14 18:21:18,548][61585] Updated weights for policy 1, policy_version 18400 (0.0008) [2023-10-14 18:21:18,611][61552] Updated weights for policy 0, policy_version 18452 (0.0008) [2023-10-14 18:21:18,988][61552] Updated weights for policy 0, policy_version 18462 (0.0009) [2023-10-14 18:21:22,679][61585] Updated weights for policy 1, policy_version 18410 (0.0009) [2023-10-14 18:21:22,995][61552] Updated weights for policy 0, policy_version 18472 (0.0007) [2023-10-14 18:21:23,045][61585] Updated weights for policy 1, policy_version 18420 (0.0008) [2023-10-14 18:21:23,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 37748736. Throughput: 0: 1680.4, 1: 1657.4. Samples: 9452436. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:21:23,344][60425] Avg episode reward: [(0, '48.600'), (1, '56.100')] [2023-10-14 18:21:23,370][61552] Updated weights for policy 0, policy_version 18482 (0.0009) [2023-10-14 18:21:23,413][61585] Updated weights for policy 1, policy_version 18430 (0.0008) [2023-10-14 18:21:23,730][61552] Updated weights for policy 0, policy_version 18492 (0.0009) [2023-10-14 18:21:27,477][61585] Updated weights for policy 1, policy_version 18440 (0.0009) [2023-10-14 18:21:27,837][61585] Updated weights for policy 1, policy_version 18450 (0.0008) [2023-10-14 18:21:27,879][61552] Updated weights for policy 0, policy_version 18502 (0.0008) [2023-10-14 18:21:28,195][61585] Updated weights for policy 1, policy_version 18460 (0.0008) [2023-10-14 18:21:28,244][61552] Updated weights for policy 0, policy_version 18512 (0.0008) [2023-10-14 18:21:28,343][60425] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 37847040. Throughput: 0: 1681.0, 1: 1654.7. Samples: 9472480. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:21:28,344][60425] Avg episode reward: [(0, '49.100'), (1, '58.130')] [2023-10-14 18:21:28,607][61552] Updated weights for policy 0, policy_version 18522 (0.0009) [2023-10-14 18:21:32,445][61585] Updated weights for policy 1, policy_version 18470 (0.0008) [2023-10-14 18:21:32,813][61585] Updated weights for policy 1, policy_version 18480 (0.0009) [2023-10-14 18:21:32,837][61552] Updated weights for policy 0, policy_version 18532 (0.0009) [2023-10-14 18:21:33,185][61585] Updated weights for policy 1, policy_version 18490 (0.0009) [2023-10-14 18:21:33,209][61552] Updated weights for policy 0, policy_version 18542 (0.0008) [2023-10-14 18:21:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 37879808. Throughput: 0: 1679.8, 1: 1661.6. Samples: 9482012. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:21:33,344][60425] Avg episode reward: [(0, '50.030'), (1, '56.010')] [2023-10-14 18:21:33,578][61552] Updated weights for policy 0, policy_version 18552 (0.0007) [2023-10-14 18:21:37,340][61585] Updated weights for policy 1, policy_version 18500 (0.0008) [2023-10-14 18:21:37,706][61585] Updated weights for policy 1, policy_version 18510 (0.0009) [2023-10-14 18:21:37,847][61552] Updated weights for policy 0, policy_version 18562 (0.0009) [2023-10-14 18:21:38,071][61585] Updated weights for policy 1, policy_version 18520 (0.0007) [2023-10-14 18:21:38,221][61552] Updated weights for policy 0, policy_version 18572 (0.0009) [2023-10-14 18:21:38,343][60425] Fps is (10 sec: 9830.6, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 37945344. Throughput: 0: 1677.6, 1: 1665.0. Samples: 9502466. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:21:38,344][60425] Avg episode reward: [(0, '49.950'), (1, '57.150')] [2023-10-14 18:21:38,587][61552] Updated weights for policy 0, policy_version 18582 (0.0008) [2023-10-14 18:21:38,967][61552] Updated weights for policy 0, policy_version 18592 (0.0007) [2023-10-14 18:21:42,210][61585] Updated weights for policy 1, policy_version 18530 (0.0007) [2023-10-14 18:21:42,575][61585] Updated weights for policy 1, policy_version 18540 (0.0010) [2023-10-14 18:21:42,936][61585] Updated weights for policy 1, policy_version 18550 (0.0009) [2023-10-14 18:21:42,977][61552] Updated weights for policy 0, policy_version 18602 (0.0008) [2023-10-14 18:21:43,306][61585] Updated weights for policy 1, policy_version 18560 (0.0010) [2023-10-14 18:21:43,344][60425] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 38043648. Throughput: 0: 1666.8, 1: 1661.8. Samples: 9522250. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-14 18:21:43,345][60425] Avg episode reward: [(0, '48.950'), (1, '54.710')] [2023-10-14 18:21:43,347][61552] Updated weights for policy 0, policy_version 18612 (0.0010) [2023-10-14 18:21:43,353][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000018560_19005440.pth... [2023-10-14 18:21:43,392][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000016992_17399808.pth [2023-10-14 18:21:43,722][61552] Updated weights for policy 0, policy_version 18622 (0.0008) [2023-10-14 18:21:43,795][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000018624_19070976.pth... [2023-10-14 18:21:43,832][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000017056_17465344.pth [2023-10-14 18:21:47,515][61585] Updated weights for policy 1, policy_version 18570 (0.0008) [2023-10-14 18:21:47,851][61552] Updated weights for policy 0, policy_version 18632 (0.0010) [2023-10-14 18:21:47,875][61585] Updated weights for policy 1, policy_version 18580 (0.0009) [2023-10-14 18:21:48,215][61552] Updated weights for policy 0, policy_version 18642 (0.0009) [2023-10-14 18:21:48,237][61585] Updated weights for policy 1, policy_version 18590 (0.0008) [2023-10-14 18:21:48,343][60425] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 38109184. Throughput: 0: 1668.1, 1: 1671.1. Samples: 9531956. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-14 18:21:48,344][60425] Avg episode reward: [(0, '51.250'), (1, '54.740')] [2023-10-14 18:21:48,586][61552] Updated weights for policy 0, policy_version 18652 (0.0010) [2023-10-14 18:21:52,226][61585] Updated weights for policy 1, policy_version 18600 (0.0009) [2023-10-14 18:21:52,590][61585] Updated weights for policy 1, policy_version 18610 (0.0009) [2023-10-14 18:21:52,778][61552] Updated weights for policy 0, policy_version 18662 (0.0007) [2023-10-14 18:21:52,952][61585] Updated weights for policy 1, policy_version 18620 (0.0011) [2023-10-14 18:21:53,143][61552] Updated weights for policy 0, policy_version 18672 (0.0007) [2023-10-14 18:21:53,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 38174720. Throughput: 0: 1662.8, 1: 1671.7. Samples: 9552146. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-14 18:21:53,344][60425] Avg episode reward: [(0, '50.110'), (1, '53.710')] [2023-10-14 18:21:53,515][61552] Updated weights for policy 0, policy_version 18682 (0.0007) [2023-10-14 18:21:57,106][61585] Updated weights for policy 1, policy_version 18630 (0.0009) [2023-10-14 18:21:57,430][61552] Updated weights for policy 0, policy_version 18692 (0.0008) [2023-10-14 18:21:57,475][61585] Updated weights for policy 1, policy_version 18640 (0.0009) [2023-10-14 18:21:57,797][61552] Updated weights for policy 0, policy_version 18702 (0.0008) [2023-10-14 18:21:57,831][61585] Updated weights for policy 1, policy_version 18650 (0.0008) [2023-10-14 18:21:58,157][61552] Updated weights for policy 0, policy_version 18712 (0.0008) [2023-10-14 18:21:58,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 38240256. Throughput: 0: 1651.9, 1: 1659.5. Samples: 9571616. Policy #0 lag: (min: 11.0, avg: 17.6, max: 43.0) [2023-10-14 18:21:58,344][60425] Avg episode reward: [(0, '53.120'), (1, '55.780')] [2023-10-14 18:22:01,917][61585] Updated weights for policy 1, policy_version 18660 (0.0007) [2023-10-14 18:22:02,284][61585] Updated weights for policy 1, policy_version 18670 (0.0008) [2023-10-14 18:22:02,368][61552] Updated weights for policy 0, policy_version 18722 (0.0010) [2023-10-14 18:22:02,646][61585] Updated weights for policy 1, policy_version 18680 (0.0010) [2023-10-14 18:22:02,733][61552] Updated weights for policy 0, policy_version 18732 (0.0008) [2023-10-14 18:22:03,108][61552] Updated weights for policy 0, policy_version 18742 (0.0008) [2023-10-14 18:22:03,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 38305792. Throughput: 0: 1662.2, 1: 1674.0. Samples: 9581918. Policy #0 lag: (min: 11.0, avg: 17.6, max: 43.0) [2023-10-14 18:22:03,344][60425] Avg episode reward: [(0, '52.530'), (1, '56.890')] [2023-10-14 18:22:03,483][61552] Updated weights for policy 0, policy_version 18752 (0.0008) [2023-10-14 18:22:06,792][61585] Updated weights for policy 1, policy_version 18690 (0.0010) [2023-10-14 18:22:07,171][61585] Updated weights for policy 1, policy_version 18700 (0.0009) [2023-10-14 18:22:07,544][61585] Updated weights for policy 1, policy_version 18710 (0.0007) [2023-10-14 18:22:07,629][61552] Updated weights for policy 0, policy_version 18762 (0.0010) [2023-10-14 18:22:07,902][61585] Updated weights for policy 1, policy_version 18720 (0.0007) [2023-10-14 18:22:07,996][61552] Updated weights for policy 0, policy_version 18772 (0.0009) [2023-10-14 18:22:08,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 38371328. Throughput: 0: 1661.0, 1: 1669.7. Samples: 9602318. Policy #0 lag: (min: 11.0, avg: 17.6, max: 43.0) [2023-10-14 18:22:08,344][60425] Avg episode reward: [(0, '54.920'), (1, '54.900')] [2023-10-14 18:22:08,369][61552] Updated weights for policy 0, policy_version 18782 (0.0008) [2023-10-14 18:22:12,066][61585] Updated weights for policy 1, policy_version 18730 (0.0008) [2023-10-14 18:22:12,362][61552] Updated weights for policy 0, policy_version 18792 (0.0009) [2023-10-14 18:22:12,427][61585] Updated weights for policy 1, policy_version 18740 (0.0007) [2023-10-14 18:22:12,727][61552] Updated weights for policy 0, policy_version 18802 (0.0008) [2023-10-14 18:22:12,789][61585] Updated weights for policy 1, policy_version 18750 (0.0007) [2023-10-14 18:22:13,104][61552] Updated weights for policy 0, policy_version 18812 (0.0008) [2023-10-14 18:22:13,343][60425] Fps is (10 sec: 16384.0, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 38469632. Throughput: 0: 1648.5, 1: 1654.5. Samples: 9621118. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:22:13,344][60425] Avg episode reward: [(0, '53.250'), (1, '55.140')] [2023-10-14 18:22:16,674][61585] Updated weights for policy 1, policy_version 18760 (0.0008) [2023-10-14 18:22:17,037][61585] Updated weights for policy 1, policy_version 18770 (0.0007) [2023-10-14 18:22:17,039][61552] Updated weights for policy 0, policy_version 18822 (0.0010) [2023-10-14 18:22:17,399][61585] Updated weights for policy 1, policy_version 18780 (0.0008) [2023-10-14 18:22:17,414][61552] Updated weights for policy 0, policy_version 18832 (0.0008) [2023-10-14 18:22:17,788][61552] Updated weights for policy 0, policy_version 18842 (0.0008) [2023-10-14 18:22:18,343][60425] Fps is (10 sec: 16383.7, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 38535168. Throughput: 0: 1665.3, 1: 1672.0. Samples: 9632190. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:22:18,344][60425] Avg episode reward: [(0, '55.340'), (1, '54.910')] [2023-10-14 18:22:21,559][61585] Updated weights for policy 1, policy_version 18790 (0.0008) [2023-10-14 18:22:21,920][61585] Updated weights for policy 1, policy_version 18800 (0.0007) [2023-10-14 18:22:22,071][61552] Updated weights for policy 0, policy_version 18852 (0.0007) [2023-10-14 18:22:22,278][61585] Updated weights for policy 1, policy_version 18810 (0.0007) [2023-10-14 18:22:22,437][61552] Updated weights for policy 0, policy_version 18862 (0.0007) [2023-10-14 18:22:22,809][61552] Updated weights for policy 0, policy_version 18872 (0.0009) [2023-10-14 18:22:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 38600704. Throughput: 0: 1663.6, 1: 1660.9. Samples: 9652068. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-14 18:22:23,344][60425] Avg episode reward: [(0, '54.770'), (1, '56.700')] [2023-10-14 18:22:26,192][61585] Updated weights for policy 1, policy_version 18820 (0.0007) [2023-10-14 18:22:26,561][61585] Updated weights for policy 1, policy_version 18830 (0.0008) [2023-10-14 18:22:26,797][61552] Updated weights for policy 0, policy_version 18882 (0.0008) [2023-10-14 18:22:26,927][61585] Updated weights for policy 1, policy_version 18840 (0.0007) [2023-10-14 18:22:27,160][61552] Updated weights for policy 0, policy_version 18892 (0.0008) [2023-10-14 18:22:27,526][61552] Updated weights for policy 0, policy_version 18902 (0.0007) [2023-10-14 18:22:27,898][61552] Updated weights for policy 0, policy_version 18912 (0.0008) [2023-10-14 18:22:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 38666240. Throughput: 0: 1646.7, 1: 1656.1. Samples: 9670874. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-14 18:22:28,344][60425] Avg episode reward: [(0, '55.890'), (1, '54.440')] [2023-10-14 18:22:28,355][61172] Saving new best policy, reward=55.890! [2023-10-14 18:22:31,145][61585] Updated weights for policy 1, policy_version 18850 (0.0008) [2023-10-14 18:22:31,503][61585] Updated weights for policy 1, policy_version 18860 (0.0008) [2023-10-14 18:22:31,865][61585] Updated weights for policy 1, policy_version 18870 (0.0008) [2023-10-14 18:22:32,127][61552] Updated weights for policy 0, policy_version 18922 (0.0010) [2023-10-14 18:22:32,230][61585] Updated weights for policy 1, policy_version 18880 (0.0007) [2023-10-14 18:22:32,502][61552] Updated weights for policy 0, policy_version 18932 (0.0010) [2023-10-14 18:22:32,868][61552] Updated weights for policy 0, policy_version 18942 (0.0010) [2023-10-14 18:22:33,343][60425] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 38731776. Throughput: 0: 1665.5, 1: 1672.5. Samples: 9682168. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-14 18:22:33,344][60425] Avg episode reward: [(0, '52.930'), (1, '57.890')] [2023-10-14 18:22:36,386][61585] Updated weights for policy 1, policy_version 18890 (0.0009) [2023-10-14 18:22:36,748][61585] Updated weights for policy 1, policy_version 18900 (0.0009) [2023-10-14 18:22:36,981][61552] Updated weights for policy 0, policy_version 18952 (0.0008) [2023-10-14 18:22:37,116][61585] Updated weights for policy 1, policy_version 18910 (0.0007) [2023-10-14 18:22:37,345][61552] Updated weights for policy 0, policy_version 18962 (0.0007) [2023-10-14 18:22:37,710][61552] Updated weights for policy 0, policy_version 18972 (0.0007) [2023-10-14 18:22:38,343][60425] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 38797312. Throughput: 0: 1668.9, 1: 1655.9. Samples: 9701760. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-14 18:22:38,344][60425] Avg episode reward: [(0, '52.970'), (1, '55.080')] [2023-10-14 18:22:41,281][61585] Updated weights for policy 1, policy_version 18920 (0.0009) [2023-10-14 18:22:41,651][61585] Updated weights for policy 1, policy_version 18930 (0.0008) [2023-10-14 18:22:41,874][61552] Updated weights for policy 0, policy_version 18982 (0.0007) [2023-10-14 18:22:42,004][61585] Updated weights for policy 1, policy_version 18940 (0.0008) [2023-10-14 18:22:42,247][61552] Updated weights for policy 0, policy_version 18992 (0.0009) [2023-10-14 18:22:42,622][61552] Updated weights for policy 0, policy_version 19002 (0.0008) [2023-10-14 18:22:43,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 38862848. Throughput: 0: 1653.2, 1: 1660.3. Samples: 9720720. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-14 18:22:43,345][60425] Avg episode reward: [(0, '54.780'), (1, '57.310')] [2023-10-14 18:22:46,026][61585] Updated weights for policy 1, policy_version 18950 (0.0010) [2023-10-14 18:22:46,392][61585] Updated weights for policy 1, policy_version 18960 (0.0007) [2023-10-14 18:22:46,550][61552] Updated weights for policy 0, policy_version 19012 (0.0008) [2023-10-14 18:22:46,758][61585] Updated weights for policy 1, policy_version 18970 (0.0010) [2023-10-14 18:22:46,925][61552] Updated weights for policy 0, policy_version 19022 (0.0008) [2023-10-14 18:22:47,289][61552] Updated weights for policy 0, policy_version 19032 (0.0011) [2023-10-14 18:22:48,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 38928384. Throughput: 0: 1667.5, 1: 1670.9. Samples: 9732146. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-14 18:22:48,344][60425] Avg episode reward: [(0, '55.800'), (1, '55.930')] [2023-10-14 18:22:50,892][61585] Updated weights for policy 1, policy_version 18980 (0.0009) [2023-10-14 18:22:51,252][61585] Updated weights for policy 1, policy_version 18990 (0.0008) [2023-10-14 18:22:51,621][61585] Updated weights for policy 1, policy_version 19000 (0.0008) [2023-10-14 18:22:51,631][61552] Updated weights for policy 0, policy_version 19042 (0.0011) [2023-10-14 18:22:51,999][61552] Updated weights for policy 0, policy_version 19052 (0.0007) [2023-10-14 18:22:52,371][61552] Updated weights for policy 0, policy_version 19062 (0.0008) [2023-10-14 18:22:52,733][61552] Updated weights for policy 0, policy_version 19072 (0.0008) [2023-10-14 18:22:53,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 38993920. Throughput: 0: 1659.2, 1: 1648.7. Samples: 9751172. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:22:53,344][60425] Avg episode reward: [(0, '52.820'), (1, '53.120')] [2023-10-14 18:22:55,752][61585] Updated weights for policy 1, policy_version 19010 (0.0007) [2023-10-14 18:22:56,114][61585] Updated weights for policy 1, policy_version 19020 (0.0007) [2023-10-14 18:22:56,478][61585] Updated weights for policy 1, policy_version 19030 (0.0007) [2023-10-14 18:22:56,833][61585] Updated weights for policy 1, policy_version 19040 (0.0008) [2023-10-14 18:22:56,839][61552] Updated weights for policy 0, policy_version 19082 (0.0009) [2023-10-14 18:22:57,212][61552] Updated weights for policy 0, policy_version 19092 (0.0007) [2023-10-14 18:22:57,581][61552] Updated weights for policy 0, policy_version 19102 (0.0008) [2023-10-14 18:22:58,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 39059456. Throughput: 0: 1650.1, 1: 1667.8. Samples: 9770426. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:22:58,344][60425] Avg episode reward: [(0, '54.200'), (1, '55.850')] [2023-10-14 18:23:00,985][61585] Updated weights for policy 1, policy_version 19050 (0.0008) [2023-10-14 18:23:01,362][61585] Updated weights for policy 1, policy_version 19060 (0.0009) [2023-10-14 18:23:01,711][61552] Updated weights for policy 0, policy_version 19112 (0.0008) [2023-10-14 18:23:01,720][61585] Updated weights for policy 1, policy_version 19070 (0.0008) [2023-10-14 18:23:02,083][61552] Updated weights for policy 0, policy_version 19122 (0.0010) [2023-10-14 18:23:02,457][61552] Updated weights for policy 0, policy_version 19132 (0.0010) [2023-10-14 18:23:03,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 39124992. Throughput: 0: 1664.2, 1: 1663.9. Samples: 9781952. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:23:03,344][60425] Avg episode reward: [(0, '53.740'), (1, '57.490')] [2023-10-14 18:23:05,946][61585] Updated weights for policy 1, policy_version 19080 (0.0009) [2023-10-14 18:23:06,310][61585] Updated weights for policy 1, policy_version 19090 (0.0009) [2023-10-14 18:23:06,541][61552] Updated weights for policy 0, policy_version 19142 (0.0009) [2023-10-14 18:23:06,671][61585] Updated weights for policy 1, policy_version 19100 (0.0007) [2023-10-14 18:23:06,902][61552] Updated weights for policy 0, policy_version 19152 (0.0007) [2023-10-14 18:23:07,274][61552] Updated weights for policy 0, policy_version 19162 (0.0009) [2023-10-14 18:23:08,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 39190528. Throughput: 0: 1664.0, 1: 1647.8. Samples: 9801102. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-14 18:23:08,344][60425] Avg episode reward: [(0, '50.760'), (1, '56.900')] [2023-10-14 18:23:10,728][61585] Updated weights for policy 1, policy_version 19110 (0.0008) [2023-10-14 18:23:11,089][61585] Updated weights for policy 1, policy_version 19120 (0.0008) [2023-10-14 18:23:11,344][61552] Updated weights for policy 0, policy_version 19172 (0.0010) [2023-10-14 18:23:11,455][61585] Updated weights for policy 1, policy_version 19130 (0.0009) [2023-10-14 18:23:11,707][61552] Updated weights for policy 0, policy_version 19182 (0.0009) [2023-10-14 18:23:12,090][61552] Updated weights for policy 0, policy_version 19192 (0.0008) [2023-10-14 18:23:13,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 39256064. Throughput: 0: 1667.5, 1: 1664.1. Samples: 9820796. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-14 18:23:13,344][60425] Avg episode reward: [(0, '55.760'), (1, '55.290')] [2023-10-14 18:23:15,661][61585] Updated weights for policy 1, policy_version 19140 (0.0008) [2023-10-14 18:23:15,999][61552] Updated weights for policy 0, policy_version 19202 (0.0008) [2023-10-14 18:23:16,029][61585] Updated weights for policy 1, policy_version 19150 (0.0007) [2023-10-14 18:23:16,363][61552] Updated weights for policy 0, policy_version 19212 (0.0008) [2023-10-14 18:23:16,390][61585] Updated weights for policy 1, policy_version 19160 (0.0007) [2023-10-14 18:23:16,739][61552] Updated weights for policy 0, policy_version 19222 (0.0008) [2023-10-14 18:23:17,110][61552] Updated weights for policy 0, policy_version 19232 (0.0009) [2023-10-14 18:23:18,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 39321600. Throughput: 0: 1679.8, 1: 1655.9. Samples: 9832274. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-14 18:23:18,344][60425] Avg episode reward: [(0, '54.780'), (1, '54.570')] [2023-10-14 18:23:20,643][61585] Updated weights for policy 1, policy_version 19170 (0.0008) [2023-10-14 18:23:21,004][61585] Updated weights for policy 1, policy_version 19180 (0.0008) [2023-10-14 18:23:21,344][61552] Updated weights for policy 0, policy_version 19242 (0.0009) [2023-10-14 18:23:21,364][61585] Updated weights for policy 1, policy_version 19190 (0.0008) [2023-10-14 18:23:21,713][61552] Updated weights for policy 0, policy_version 19252 (0.0010) [2023-10-14 18:23:21,739][61585] Updated weights for policy 1, policy_version 19200 (0.0007) [2023-10-14 18:23:22,075][61552] Updated weights for policy 0, policy_version 19262 (0.0008) [2023-10-14 18:23:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 39387136. Throughput: 0: 1660.4, 1: 1654.4. Samples: 9850930. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-14 18:23:23,344][60425] Avg episode reward: [(0, '52.680'), (1, '54.820')] [2023-10-14 18:23:25,854][61585] Updated weights for policy 1, policy_version 19210 (0.0009) [2023-10-14 18:23:26,217][61552] Updated weights for policy 0, policy_version 19272 (0.0009) [2023-10-14 18:23:26,231][61585] Updated weights for policy 1, policy_version 19220 (0.0010) [2023-10-14 18:23:26,584][61552] Updated weights for policy 0, policy_version 19282 (0.0008) [2023-10-14 18:23:26,593][61585] Updated weights for policy 1, policy_version 19230 (0.0008) [2023-10-14 18:23:26,951][61552] Updated weights for policy 0, policy_version 19292 (0.0008) [2023-10-14 18:23:28,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 39452672. Throughput: 0: 1668.5, 1: 1664.6. Samples: 9870712. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-14 18:23:28,344][60425] Avg episode reward: [(0, '56.570'), (1, '52.540')] [2023-10-14 18:23:28,354][61172] Saving new best policy, reward=56.570! [2023-10-14 18:23:30,704][61585] Updated weights for policy 1, policy_version 19240 (0.0008) [2023-10-14 18:23:31,068][61585] Updated weights for policy 1, policy_version 19250 (0.0010) [2023-10-14 18:23:31,147][61552] Updated weights for policy 0, policy_version 19302 (0.0009) [2023-10-14 18:23:31,440][61585] Updated weights for policy 1, policy_version 19260 (0.0007) [2023-10-14 18:23:31,516][61552] Updated weights for policy 0, policy_version 19312 (0.0010) [2023-10-14 18:23:31,892][61552] Updated weights for policy 0, policy_version 19322 (0.0008) [2023-10-14 18:23:33,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 39518208. Throughput: 0: 1673.2, 1: 1657.1. Samples: 9882012. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-14 18:23:33,344][60425] Avg episode reward: [(0, '52.480'), (1, '57.640')] [2023-10-14 18:23:35,606][61585] Updated weights for policy 1, policy_version 19270 (0.0009) [2023-10-14 18:23:35,972][61585] Updated weights for policy 1, policy_version 19280 (0.0009) [2023-10-14 18:23:36,129][61552] Updated weights for policy 0, policy_version 19332 (0.0009) [2023-10-14 18:23:36,334][61585] Updated weights for policy 1, policy_version 19290 (0.0007) [2023-10-14 18:23:36,495][61552] Updated weights for policy 0, policy_version 19342 (0.0007) [2023-10-14 18:23:36,869][61552] Updated weights for policy 0, policy_version 19352 (0.0008) [2023-10-14 18:23:38,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 39583744. Throughput: 0: 1662.7, 1: 1663.4. Samples: 9900848. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-14 18:23:38,344][60425] Avg episode reward: [(0, '54.770'), (1, '59.250')] [2023-10-14 18:23:40,273][61585] Updated weights for policy 1, policy_version 19300 (0.0007) [2023-10-14 18:23:40,636][61585] Updated weights for policy 1, policy_version 19310 (0.0009) [2023-10-14 18:23:40,995][61585] Updated weights for policy 1, policy_version 19320 (0.0009) [2023-10-14 18:23:40,996][61552] Updated weights for policy 0, policy_version 19362 (0.0008) [2023-10-14 18:23:41,363][61552] Updated weights for policy 0, policy_version 19372 (0.0010) [2023-10-14 18:23:41,740][61552] Updated weights for policy 0, policy_version 19382 (0.0007) [2023-10-14 18:23:42,106][61552] Updated weights for policy 0, policy_version 19392 (0.0007) [2023-10-14 18:23:43,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 39649280. Throughput: 0: 1670.3, 1: 1671.6. Samples: 9920814. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-14 18:23:43,344][60425] Avg episode reward: [(0, '52.450'), (1, '57.010')] [2023-10-14 18:23:43,354][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000019328_19791872.pth... [2023-10-14 18:23:43,354][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000019392_19857408.pth... [2023-10-14 18:23:43,384][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000017824_18251776.pth [2023-10-14 18:23:43,394][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000017760_18186240.pth [2023-10-14 18:23:45,178][61585] Updated weights for policy 1, policy_version 19330 (0.0008) [2023-10-14 18:23:45,540][61585] Updated weights for policy 1, policy_version 19340 (0.0010) [2023-10-14 18:23:45,916][61585] Updated weights for policy 1, policy_version 19350 (0.0008) [2023-10-14 18:23:46,140][61552] Updated weights for policy 0, policy_version 19402 (0.0009) [2023-10-14 18:23:46,276][61585] Updated weights for policy 1, policy_version 19360 (0.0007) [2023-10-14 18:23:46,520][61552] Updated weights for policy 0, policy_version 19412 (0.0010) [2023-10-14 18:23:46,887][61552] Updated weights for policy 0, policy_version 19422 (0.0009) [2023-10-14 18:23:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 39714816. Throughput: 0: 1669.1, 1: 1658.8. Samples: 9931706. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-14 18:23:48,344][60425] Avg episode reward: [(0, '54.290'), (1, '56.750')] [2023-10-14 18:23:50,355][61585] Updated weights for policy 1, policy_version 19370 (0.0008) [2023-10-14 18:23:50,727][61585] Updated weights for policy 1, policy_version 19380 (0.0008) [2023-10-14 18:23:50,987][61552] Updated weights for policy 0, policy_version 19432 (0.0008) [2023-10-14 18:23:51,082][61585] Updated weights for policy 1, policy_version 19390 (0.0009) [2023-10-14 18:23:51,356][61552] Updated weights for policy 0, policy_version 19442 (0.0008) [2023-10-14 18:23:51,717][61552] Updated weights for policy 0, policy_version 19452 (0.0008) [2023-10-14 18:23:53,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 39780352. Throughput: 0: 1651.2, 1: 1671.6. Samples: 9950630. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:23:53,344][60425] Avg episode reward: [(0, '54.110'), (1, '56.410')] [2023-10-14 18:23:55,155][61585] Updated weights for policy 1, policy_version 19400 (0.0009) [2023-10-14 18:23:55,527][61585] Updated weights for policy 1, policy_version 19410 (0.0008) [2023-10-14 18:23:55,696][61552] Updated weights for policy 0, policy_version 19462 (0.0007) [2023-10-14 18:23:55,891][61585] Updated weights for policy 1, policy_version 19420 (0.0007) [2023-10-14 18:23:56,077][61552] Updated weights for policy 0, policy_version 19472 (0.0009) [2023-10-14 18:23:56,447][61552] Updated weights for policy 0, policy_version 19482 (0.0008) [2023-10-14 18:23:58,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 39845888. Throughput: 0: 1663.5, 1: 1672.2. Samples: 9970906. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:23:58,344][60425] Avg episode reward: [(0, '53.590'), (1, '54.050')] [2023-10-14 18:23:59,916][61585] Updated weights for policy 1, policy_version 19430 (0.0008) [2023-10-14 18:24:00,277][61585] Updated weights for policy 1, policy_version 19440 (0.0007) [2023-10-14 18:24:00,535][61552] Updated weights for policy 0, policy_version 19492 (0.0009) [2023-10-14 18:24:00,638][61585] Updated weights for policy 1, policy_version 19450 (0.0008) [2023-10-14 18:24:00,909][61552] Updated weights for policy 0, policy_version 19502 (0.0008) [2023-10-14 18:24:01,272][61552] Updated weights for policy 0, policy_version 19512 (0.0010) [2023-10-14 18:24:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 39911424. Throughput: 0: 1653.5, 1: 1652.7. Samples: 9981052. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:24:03,344][60425] Avg episode reward: [(0, '55.450'), (1, '53.380')] [2023-10-14 18:24:04,769][61585] Updated weights for policy 1, policy_version 19460 (0.0008) [2023-10-14 18:24:05,135][61585] Updated weights for policy 1, policy_version 19470 (0.0008) [2023-10-14 18:24:05,411][61552] Updated weights for policy 0, policy_version 19522 (0.0008) [2023-10-14 18:24:05,497][61585] Updated weights for policy 1, policy_version 19480 (0.0009) [2023-10-14 18:24:05,779][61552] Updated weights for policy 0, policy_version 19532 (0.0008) [2023-10-14 18:24:06,149][61552] Updated weights for policy 0, policy_version 19542 (0.0009) [2023-10-14 18:24:06,515][61552] Updated weights for policy 0, policy_version 19552 (0.0010) [2023-10-14 18:24:08,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 39976960. Throughput: 0: 1653.6, 1: 1670.7. Samples: 10000524. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:24:08,344][60425] Avg episode reward: [(0, '53.970'), (1, '53.970')] [2023-10-14 18:24:09,562][61585] Updated weights for policy 1, policy_version 19490 (0.0007) [2023-10-14 18:24:09,931][61585] Updated weights for policy 1, policy_version 19500 (0.0010) [2023-10-14 18:24:10,306][61585] Updated weights for policy 1, policy_version 19510 (0.0009) [2023-10-14 18:24:10,592][61552] Updated weights for policy 0, policy_version 19562 (0.0008) [2023-10-14 18:24:10,669][61585] Updated weights for policy 1, policy_version 19520 (0.0009) [2023-10-14 18:24:10,962][61552] Updated weights for policy 0, policy_version 19572 (0.0008) [2023-10-14 18:24:11,331][61552] Updated weights for policy 0, policy_version 19582 (0.0009) [2023-10-14 18:24:13,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 40042496. Throughput: 0: 1673.6, 1: 1676.9. Samples: 10021484. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:24:13,344][60425] Avg episode reward: [(0, '55.140'), (1, '51.340')] [2023-10-14 18:24:14,804][61585] Updated weights for policy 1, policy_version 19530 (0.0010) [2023-10-14 18:24:15,178][61585] Updated weights for policy 1, policy_version 19540 (0.0008) [2023-10-14 18:24:15,414][61552] Updated weights for policy 0, policy_version 19592 (0.0008) [2023-10-14 18:24:15,545][61585] Updated weights for policy 1, policy_version 19550 (0.0008) [2023-10-14 18:24:15,792][61552] Updated weights for policy 0, policy_version 19602 (0.0008) [2023-10-14 18:24:16,157][61552] Updated weights for policy 0, policy_version 19612 (0.0007) [2023-10-14 18:24:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 40108032. Throughput: 0: 1661.7, 1: 1650.8. Samples: 10031076. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:24:18,344][60425] Avg episode reward: [(0, '55.950'), (1, '55.460')] [2023-10-14 18:24:19,638][61585] Updated weights for policy 1, policy_version 19560 (0.0007) [2023-10-14 18:24:20,005][61585] Updated weights for policy 1, policy_version 19570 (0.0007) [2023-10-14 18:24:20,239][61552] Updated weights for policy 0, policy_version 19622 (0.0009) [2023-10-14 18:24:20,371][61585] Updated weights for policy 1, policy_version 19580 (0.0008) [2023-10-14 18:24:20,595][61552] Updated weights for policy 0, policy_version 19632 (0.0009) [2023-10-14 18:24:20,965][61552] Updated weights for policy 0, policy_version 19642 (0.0007) [2023-10-14 18:24:23,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 40173568. Throughput: 0: 1664.5, 1: 1668.9. Samples: 10050852. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) [2023-10-14 18:24:23,344][60425] Avg episode reward: [(0, '55.310'), (1, '55.560')] [2023-10-14 18:24:24,485][61585] Updated weights for policy 1, policy_version 19590 (0.0007) [2023-10-14 18:24:24,853][61585] Updated weights for policy 1, policy_version 19600 (0.0009) [2023-10-14 18:24:25,015][61552] Updated weights for policy 0, policy_version 19652 (0.0009) [2023-10-14 18:24:25,218][61585] Updated weights for policy 1, policy_version 19610 (0.0007) [2023-10-14 18:24:25,382][61552] Updated weights for policy 0, policy_version 19662 (0.0008) [2023-10-14 18:24:25,757][61552] Updated weights for policy 0, policy_version 19672 (0.0010) [2023-10-14 18:24:28,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 40239104. Throughput: 0: 1680.9, 1: 1669.2. Samples: 10071566. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) [2023-10-14 18:24:28,345][60425] Avg episode reward: [(0, '55.300'), (1, '56.770')] [2023-10-14 18:24:29,210][61585] Updated weights for policy 1, policy_version 19620 (0.0009) [2023-10-14 18:24:29,582][61585] Updated weights for policy 1, policy_version 19630 (0.0009) [2023-10-14 18:24:29,857][61552] Updated weights for policy 0, policy_version 19682 (0.0008) [2023-10-14 18:24:29,936][61585] Updated weights for policy 1, policy_version 19640 (0.0009) [2023-10-14 18:24:30,230][61552] Updated weights for policy 0, policy_version 19692 (0.0008) [2023-10-14 18:24:30,605][61552] Updated weights for policy 0, policy_version 19702 (0.0007) [2023-10-14 18:24:30,962][61552] Updated weights for policy 0, policy_version 19712 (0.0007) [2023-10-14 18:24:33,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 40304640. Throughput: 0: 1659.1, 1: 1658.0. Samples: 10080976. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) [2023-10-14 18:24:33,344][60425] Avg episode reward: [(0, '51.930'), (1, '56.490')] [2023-10-14 18:24:34,234][61585] Updated weights for policy 1, policy_version 19650 (0.0008) [2023-10-14 18:24:34,614][61585] Updated weights for policy 1, policy_version 19660 (0.0008) [2023-10-14 18:24:34,953][61552] Updated weights for policy 0, policy_version 19722 (0.0009) [2023-10-14 18:24:34,979][61585] Updated weights for policy 1, policy_version 19670 (0.0008) [2023-10-14 18:24:35,330][61552] Updated weights for policy 0, policy_version 19732 (0.0009) [2023-10-14 18:24:35,349][61585] Updated weights for policy 1, policy_version 19680 (0.0009) [2023-10-14 18:24:35,705][61552] Updated weights for policy 0, policy_version 19742 (0.0011) [2023-10-14 18:24:38,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 40370176. Throughput: 0: 1671.0, 1: 1669.8. Samples: 10100968. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) [2023-10-14 18:24:38,344][60425] Avg episode reward: [(0, '53.270'), (1, '54.250')] [2023-10-14 18:24:39,399][61585] Updated weights for policy 1, policy_version 19690 (0.0007) [2023-10-14 18:24:39,759][61585] Updated weights for policy 1, policy_version 19700 (0.0009) [2023-10-14 18:24:39,795][61552] Updated weights for policy 0, policy_version 19752 (0.0009) [2023-10-14 18:24:40,136][61585] Updated weights for policy 1, policy_version 19710 (0.0008) [2023-10-14 18:24:40,165][61552] Updated weights for policy 0, policy_version 19762 (0.0008) [2023-10-14 18:24:40,534][61552] Updated weights for policy 0, policy_version 19772 (0.0008) [2023-10-14 18:24:43,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 40435712. Throughput: 0: 1681.4, 1: 1671.2. Samples: 10121772. Policy #0 lag: (min: 11.0, avg: 18.9, max: 43.0) [2023-10-14 18:24:43,344][60425] Avg episode reward: [(0, '53.330'), (1, '54.740')] [2023-10-14 18:24:44,126][61585] Updated weights for policy 1, policy_version 19720 (0.0007) [2023-10-14 18:24:44,500][61585] Updated weights for policy 1, policy_version 19730 (0.0009) [2023-10-14 18:24:44,603][61552] Updated weights for policy 0, policy_version 19782 (0.0008) [2023-10-14 18:24:44,856][61585] Updated weights for policy 1, policy_version 19740 (0.0008) [2023-10-14 18:24:44,970][61552] Updated weights for policy 0, policy_version 19792 (0.0008) [2023-10-14 18:24:45,335][61552] Updated weights for policy 0, policy_version 19802 (0.0009) [2023-10-14 18:24:48,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 40501248. Throughput: 0: 1657.0, 1: 1669.3. Samples: 10130736. Policy #0 lag: (min: 11.0, avg: 18.9, max: 43.0) [2023-10-14 18:24:48,344][60425] Avg episode reward: [(0, '55.240'), (1, '53.810')] [2023-10-14 18:24:49,045][61585] Updated weights for policy 1, policy_version 19750 (0.0009) [2023-10-14 18:24:49,417][61585] Updated weights for policy 1, policy_version 19760 (0.0009) [2023-10-14 18:24:49,489][61552] Updated weights for policy 0, policy_version 19812 (0.0009) [2023-10-14 18:24:49,777][61585] Updated weights for policy 1, policy_version 19770 (0.0010) [2023-10-14 18:24:49,859][61552] Updated weights for policy 0, policy_version 19822 (0.0008) [2023-10-14 18:24:50,231][61552] Updated weights for policy 0, policy_version 19832 (0.0007) [2023-10-14 18:24:53,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 40566784. Throughput: 0: 1678.1, 1: 1668.5. Samples: 10151124. Policy #0 lag: (min: 11.0, avg: 18.9, max: 43.0) [2023-10-14 18:24:53,344][60425] Avg episode reward: [(0, '49.420'), (1, '53.080')] [2023-10-14 18:24:53,867][61585] Updated weights for policy 1, policy_version 19780 (0.0007) [2023-10-14 18:24:54,238][61585] Updated weights for policy 1, policy_version 19790 (0.0008) [2023-10-14 18:24:54,424][61552] Updated weights for policy 0, policy_version 19842 (0.0009) [2023-10-14 18:24:54,605][61585] Updated weights for policy 1, policy_version 19800 (0.0009) [2023-10-14 18:24:54,795][61552] Updated weights for policy 0, policy_version 19852 (0.0009) [2023-10-14 18:24:55,177][61552] Updated weights for policy 0, policy_version 19862 (0.0009) [2023-10-14 18:24:55,554][61552] Updated weights for policy 0, policy_version 19872 (0.0008) [2023-10-14 18:24:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 40632320. Throughput: 0: 1672.6, 1: 1668.0. Samples: 10171814. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 18:24:58,344][60425] Avg episode reward: [(0, '50.630'), (1, '54.450')] [2023-10-14 18:24:58,791][61585] Updated weights for policy 1, policy_version 19810 (0.0008) [2023-10-14 18:24:59,162][61585] Updated weights for policy 1, policy_version 19820 (0.0008) [2023-10-14 18:24:59,525][61585] Updated weights for policy 1, policy_version 19830 (0.0008) [2023-10-14 18:24:59,603][61552] Updated weights for policy 0, policy_version 19882 (0.0008) [2023-10-14 18:24:59,896][61585] Updated weights for policy 1, policy_version 19840 (0.0008) [2023-10-14 18:24:59,963][61552] Updated weights for policy 0, policy_version 19892 (0.0008) [2023-10-14 18:25:00,341][61552] Updated weights for policy 0, policy_version 19902 (0.0011) [2023-10-14 18:25:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 40697856. Throughput: 0: 1657.3, 1: 1669.4. Samples: 10180776. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 18:25:03,344][60425] Avg episode reward: [(0, '51.940'), (1, '56.480')] [2023-10-14 18:25:04,180][61585] Updated weights for policy 1, policy_version 19850 (0.0007) [2023-10-14 18:25:04,342][61552] Updated weights for policy 0, policy_version 19912 (0.0008) [2023-10-14 18:25:04,550][61585] Updated weights for policy 1, policy_version 19860 (0.0007) [2023-10-14 18:25:04,705][61552] Updated weights for policy 0, policy_version 19922 (0.0009) [2023-10-14 18:25:04,910][61585] Updated weights for policy 1, policy_version 19870 (0.0007) [2023-10-14 18:25:05,081][61552] Updated weights for policy 0, policy_version 19932 (0.0008) [2023-10-14 18:25:08,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 40763392. Throughput: 0: 1673.3, 1: 1666.3. Samples: 10201136. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 18:25:08,344][60425] Avg episode reward: [(0, '53.370'), (1, '53.250')] [2023-10-14 18:25:08,988][61585] Updated weights for policy 1, policy_version 19880 (0.0010) [2023-10-14 18:25:09,290][61552] Updated weights for policy 0, policy_version 19942 (0.0007) [2023-10-14 18:25:09,352][61585] Updated weights for policy 1, policy_version 19890 (0.0007) [2023-10-14 18:25:09,660][61552] Updated weights for policy 0, policy_version 19952 (0.0008) [2023-10-14 18:25:09,721][61585] Updated weights for policy 1, policy_version 19900 (0.0007) [2023-10-14 18:25:10,034][61552] Updated weights for policy 0, policy_version 19962 (0.0007) [2023-10-14 18:25:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 40828928. Throughput: 0: 1674.1, 1: 1661.0. Samples: 10221642. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 18:25:13,344][60425] Avg episode reward: [(0, '51.250'), (1, '53.680')] [2023-10-14 18:25:13,933][61585] Updated weights for policy 1, policy_version 19910 (0.0009) [2023-10-14 18:25:14,065][61552] Updated weights for policy 0, policy_version 19972 (0.0007) [2023-10-14 18:25:14,299][61585] Updated weights for policy 1, policy_version 19920 (0.0008) [2023-10-14 18:25:14,442][61552] Updated weights for policy 0, policy_version 19982 (0.0008) [2023-10-14 18:25:14,663][61585] Updated weights for policy 1, policy_version 19930 (0.0008) [2023-10-14 18:25:14,812][61552] Updated weights for policy 0, policy_version 19992 (0.0009) [2023-10-14 18:25:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 40894464. Throughput: 0: 1662.7, 1: 1660.2. Samples: 10230504. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:25:18,344][60425] Avg episode reward: [(0, '52.070'), (1, '57.260')] [2023-10-14 18:25:18,876][61585] Updated weights for policy 1, policy_version 19940 (0.0008) [2023-10-14 18:25:18,909][61552] Updated weights for policy 0, policy_version 20002 (0.0010) [2023-10-14 18:25:19,240][61585] Updated weights for policy 1, policy_version 19950 (0.0007) [2023-10-14 18:25:19,290][61552] Updated weights for policy 0, policy_version 20012 (0.0007) [2023-10-14 18:25:19,603][61585] Updated weights for policy 1, policy_version 19960 (0.0008) [2023-10-14 18:25:19,661][61552] Updated weights for policy 0, policy_version 20022 (0.0008) [2023-10-14 18:25:20,034][61552] Updated weights for policy 0, policy_version 20032 (0.0008) [2023-10-14 18:25:23,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 40960000. Throughput: 0: 1670.1, 1: 1662.9. Samples: 10250950. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:25:23,344][60425] Avg episode reward: [(0, '50.720'), (1, '56.450')] [2023-10-14 18:25:23,730][61585] Updated weights for policy 1, policy_version 19970 (0.0008) [2023-10-14 18:25:24,037][61552] Updated weights for policy 0, policy_version 20042 (0.0007) [2023-10-14 18:25:24,099][61585] Updated weights for policy 1, policy_version 19980 (0.0007) [2023-10-14 18:25:24,415][61552] Updated weights for policy 0, policy_version 20052 (0.0007) [2023-10-14 18:25:24,465][61585] Updated weights for policy 1, policy_version 19990 (0.0010) [2023-10-14 18:25:24,777][61552] Updated weights for policy 0, policy_version 20062 (0.0007) [2023-10-14 18:25:24,818][61585] Updated weights for policy 1, policy_version 20000 (0.0007) [2023-10-14 18:25:28,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 41025536. Throughput: 0: 1669.9, 1: 1662.0. Samples: 10271706. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:25:28,344][60425] Avg episode reward: [(0, '52.390'), (1, '55.620')] [2023-10-14 18:25:28,845][61552] Updated weights for policy 0, policy_version 20072 (0.0007) [2023-10-14 18:25:28,948][61585] Updated weights for policy 1, policy_version 20010 (0.0007) [2023-10-14 18:25:29,209][61552] Updated weights for policy 0, policy_version 20082 (0.0008) [2023-10-14 18:25:29,321][61585] Updated weights for policy 1, policy_version 20020 (0.0007) [2023-10-14 18:25:29,569][61552] Updated weights for policy 0, policy_version 20092 (0.0007) [2023-10-14 18:25:29,688][61585] Updated weights for policy 1, policy_version 20030 (0.0007) [2023-10-14 18:25:33,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 41091072. Throughput: 0: 1675.7, 1: 1662.1. Samples: 10280936. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:25:33,344][60425] Avg episode reward: [(0, '54.800'), (1, '55.290')] [2023-10-14 18:25:33,624][61552] Updated weights for policy 0, policy_version 20102 (0.0008) [2023-10-14 18:25:33,930][61585] Updated weights for policy 1, policy_version 20040 (0.0008) [2023-10-14 18:25:34,006][61552] Updated weights for policy 0, policy_version 20112 (0.0008) [2023-10-14 18:25:34,296][61585] Updated weights for policy 1, policy_version 20050 (0.0008) [2023-10-14 18:25:34,373][61552] Updated weights for policy 0, policy_version 20122 (0.0008) [2023-10-14 18:25:34,663][61585] Updated weights for policy 1, policy_version 20060 (0.0008) [2023-10-14 18:25:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 41156608. Throughput: 0: 1679.2, 1: 1658.5. Samples: 10301322. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:25:38,344][60425] Avg episode reward: [(0, '51.720'), (1, '55.470')] [2023-10-14 18:25:38,364][61552] Updated weights for policy 0, policy_version 20132 (0.0007) [2023-10-14 18:25:38,671][61585] Updated weights for policy 1, policy_version 20070 (0.0008) [2023-10-14 18:25:38,730][61552] Updated weights for policy 0, policy_version 20142 (0.0007) [2023-10-14 18:25:39,035][61585] Updated weights for policy 1, policy_version 20080 (0.0007) [2023-10-14 18:25:39,090][61552] Updated weights for policy 0, policy_version 20152 (0.0008) [2023-10-14 18:25:39,394][61585] Updated weights for policy 1, policy_version 20090 (0.0007) [2023-10-14 18:25:43,206][61552] Updated weights for policy 0, policy_version 20162 (0.0008) [2023-10-14 18:25:43,344][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 41222144. Throughput: 0: 1681.6, 1: 1651.6. Samples: 10321812. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:25:43,345][60425] Avg episode reward: [(0, '55.980'), (1, '52.990')] [2023-10-14 18:25:43,573][61585] Updated weights for policy 1, policy_version 20100 (0.0009) [2023-10-14 18:25:43,584][61552] Updated weights for policy 0, policy_version 20172 (0.0010) [2023-10-14 18:25:43,934][61585] Updated weights for policy 1, policy_version 20110 (0.0007) [2023-10-14 18:25:43,950][61552] Updated weights for policy 0, policy_version 20182 (0.0009) [2023-10-14 18:25:44,311][61585] Updated weights for policy 1, policy_version 20120 (0.0007) [2023-10-14 18:25:44,314][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000020192_20676608.pth... [2023-10-14 18:25:44,314][61552] Updated weights for policy 0, policy_version 20192 (0.0007) [2023-10-14 18:25:44,347][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000018624_19070976.pth [2023-10-14 18:25:44,604][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000020128_20611072.pth... [2023-10-14 18:25:44,632][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000018560_19005440.pth [2023-10-14 18:25:48,287][61585] Updated weights for policy 1, policy_version 20130 (0.0008) [2023-10-14 18:25:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 41287680. Throughput: 0: 1680.1, 1: 1652.6. Samples: 10330746. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:25:48,344][60425] Avg episode reward: [(0, '57.360'), (1, '57.090')] [2023-10-14 18:25:48,687][61585] Updated weights for policy 1, policy_version 20140 (0.0009) [2023-10-14 18:25:48,724][61552] Updated weights for policy 0, policy_version 20202 (0.0008) [2023-10-14 18:25:49,058][61585] Updated weights for policy 1, policy_version 20150 (0.0008) [2023-10-14 18:25:49,084][61552] Updated weights for policy 0, policy_version 20212 (0.0009) [2023-10-14 18:25:49,418][61585] Updated weights for policy 1, policy_version 20160 (0.0009) [2023-10-14 18:25:49,452][61552] Updated weights for policy 0, policy_version 20222 (0.0007) [2023-10-14 18:25:49,525][61172] Saving new best policy, reward=57.360! [2023-10-14 18:25:53,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 41353216. Throughput: 0: 1669.8, 1: 1656.7. Samples: 10350828. Policy #0 lag: (min: 9.0, avg: 15.4, max: 41.0) [2023-10-14 18:25:53,344][60425] Avg episode reward: [(0, '53.520'), (1, '54.570')] [2023-10-14 18:25:53,474][61552] Updated weights for policy 0, policy_version 20232 (0.0009) [2023-10-14 18:25:53,504][61585] Updated weights for policy 1, policy_version 20170 (0.0009) [2023-10-14 18:25:53,842][61552] Updated weights for policy 0, policy_version 20242 (0.0009) [2023-10-14 18:25:53,871][61585] Updated weights for policy 1, policy_version 20180 (0.0010) [2023-10-14 18:25:54,200][61552] Updated weights for policy 0, policy_version 20252 (0.0009) [2023-10-14 18:25:54,240][61585] Updated weights for policy 1, policy_version 20190 (0.0009) [2023-10-14 18:25:58,326][61552] Updated weights for policy 0, policy_version 20262 (0.0009) [2023-10-14 18:25:58,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 41418752. Throughput: 0: 1664.9, 1: 1656.5. Samples: 10371108. Policy #0 lag: (min: 9.0, avg: 15.4, max: 41.0) [2023-10-14 18:25:58,344][60425] Avg episode reward: [(0, '55.770'), (1, '58.640')] [2023-10-14 18:25:58,522][61585] Updated weights for policy 1, policy_version 20200 (0.0008) [2023-10-14 18:25:58,690][61552] Updated weights for policy 0, policy_version 20272 (0.0008) [2023-10-14 18:25:58,888][61585] Updated weights for policy 1, policy_version 20210 (0.0008) [2023-10-14 18:25:59,066][61552] Updated weights for policy 0, policy_version 20282 (0.0007) [2023-10-14 18:25:59,246][61585] Updated weights for policy 1, policy_version 20220 (0.0008) [2023-10-14 18:26:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 41484288. Throughput: 0: 1665.7, 1: 1657.4. Samples: 10380042. Policy #0 lag: (min: 9.0, avg: 15.4, max: 41.0) [2023-10-14 18:26:03,344][60425] Avg episode reward: [(0, '55.820'), (1, '57.890')] [2023-10-14 18:26:03,382][61552] Updated weights for policy 0, policy_version 20292 (0.0007) [2023-10-14 18:26:03,540][61585] Updated weights for policy 1, policy_version 20230 (0.0008) [2023-10-14 18:26:03,744][61552] Updated weights for policy 0, policy_version 20302 (0.0007) [2023-10-14 18:26:03,907][61585] Updated weights for policy 1, policy_version 20240 (0.0007) [2023-10-14 18:26:04,110][61552] Updated weights for policy 0, policy_version 20312 (0.0007) [2023-10-14 18:26:04,263][61585] Updated weights for policy 1, policy_version 20250 (0.0010) [2023-10-14 18:26:08,276][61552] Updated weights for policy 0, policy_version 20322 (0.0008) [2023-10-14 18:26:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 41549824. Throughput: 0: 1667.1, 1: 1654.2. Samples: 10400408. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-14 18:26:08,344][60425] Avg episode reward: [(0, '56.680'), (1, '56.520')] [2023-10-14 18:26:08,346][61585] Updated weights for policy 1, policy_version 20260 (0.0008) [2023-10-14 18:26:08,700][61552] Updated weights for policy 0, policy_version 20332 (0.0008) [2023-10-14 18:26:08,716][61585] Updated weights for policy 1, policy_version 20270 (0.0009) [2023-10-14 18:26:09,067][61552] Updated weights for policy 0, policy_version 20342 (0.0009) [2023-10-14 18:26:09,078][61585] Updated weights for policy 1, policy_version 20280 (0.0007) [2023-10-14 18:26:09,440][61552] Updated weights for policy 0, policy_version 20352 (0.0008) [2023-10-14 18:26:13,257][61585] Updated weights for policy 1, policy_version 20290 (0.0007) [2023-10-14 18:26:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 41615360. Throughput: 0: 1660.8, 1: 1650.0. Samples: 10420692. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-14 18:26:13,344][60425] Avg episode reward: [(0, '55.970'), (1, '56.520')] [2023-10-14 18:26:13,508][61552] Updated weights for policy 0, policy_version 20362 (0.0007) [2023-10-14 18:26:13,625][61585] Updated weights for policy 1, policy_version 20300 (0.0008) [2023-10-14 18:26:13,873][61552] Updated weights for policy 0, policy_version 20372 (0.0008) [2023-10-14 18:26:13,982][61585] Updated weights for policy 1, policy_version 20310 (0.0008) [2023-10-14 18:26:14,258][61552] Updated weights for policy 0, policy_version 20382 (0.0010) [2023-10-14 18:26:14,359][61585] Updated weights for policy 1, policy_version 20320 (0.0008) [2023-10-14 18:26:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 41680896. Throughput: 0: 1658.6, 1: 1648.8. Samples: 10429772. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-14 18:26:18,344][60425] Avg episode reward: [(0, '58.480'), (1, '53.300')] [2023-10-14 18:26:18,382][61552] Updated weights for policy 0, policy_version 20392 (0.0007) [2023-10-14 18:26:18,632][61585] Updated weights for policy 1, policy_version 20330 (0.0010) [2023-10-14 18:26:18,751][61552] Updated weights for policy 0, policy_version 20402 (0.0007) [2023-10-14 18:26:19,002][61585] Updated weights for policy 1, policy_version 20340 (0.0007) [2023-10-14 18:26:19,112][61552] Updated weights for policy 0, policy_version 20412 (0.0008) [2023-10-14 18:26:19,253][61172] Saving new best policy, reward=58.480! [2023-10-14 18:26:19,375][61585] Updated weights for policy 1, policy_version 20350 (0.0008) [2023-10-14 18:26:23,219][61552] Updated weights for policy 0, policy_version 20422 (0.0008) [2023-10-14 18:26:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 41746432. Throughput: 0: 1650.7, 1: 1651.5. Samples: 10449918. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-14 18:26:23,344][60425] Avg episode reward: [(0, '55.740'), (1, '58.230')] [2023-10-14 18:26:23,443][61585] Updated weights for policy 1, policy_version 20360 (0.0007) [2023-10-14 18:26:23,588][61552] Updated weights for policy 0, policy_version 20432 (0.0008) [2023-10-14 18:26:23,804][61585] Updated weights for policy 1, policy_version 20370 (0.0008) [2023-10-14 18:26:23,950][61552] Updated weights for policy 0, policy_version 20442 (0.0009) [2023-10-14 18:26:24,169][61585] Updated weights for policy 1, policy_version 20380 (0.0008) [2023-10-14 18:26:28,170][61552] Updated weights for policy 0, policy_version 20452 (0.0009) [2023-10-14 18:26:28,271][61585] Updated weights for policy 1, policy_version 20390 (0.0010) [2023-10-14 18:26:28,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 41811968. Throughput: 0: 1648.3, 1: 1651.7. Samples: 10470314. Policy #0 lag: (min: 0.0, avg: 25.9, max: 32.0) [2023-10-14 18:26:28,344][60425] Avg episode reward: [(0, '52.400'), (1, '54.750')] [2023-10-14 18:26:28,531][61552] Updated weights for policy 0, policy_version 20462 (0.0009) [2023-10-14 18:26:28,634][61585] Updated weights for policy 1, policy_version 20400 (0.0009) [2023-10-14 18:26:28,903][61552] Updated weights for policy 0, policy_version 20472 (0.0008) [2023-10-14 18:26:28,996][61585] Updated weights for policy 1, policy_version 20410 (0.0007) [2023-10-14 18:26:32,924][61552] Updated weights for policy 0, policy_version 20482 (0.0008) [2023-10-14 18:26:33,112][61585] Updated weights for policy 1, policy_version 20420 (0.0009) [2023-10-14 18:26:33,299][61552] Updated weights for policy 0, policy_version 20492 (0.0008) [2023-10-14 18:26:33,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 41877504. Throughput: 0: 1647.0, 1: 1654.1. Samples: 10479294. Policy #0 lag: (min: 0.0, avg: 25.9, max: 32.0) [2023-10-14 18:26:33,344][60425] Avg episode reward: [(0, '53.630'), (1, '56.160')] [2023-10-14 18:26:33,518][61585] Updated weights for policy 1, policy_version 20430 (0.0008) [2023-10-14 18:26:33,665][61552] Updated weights for policy 0, policy_version 20502 (0.0007) [2023-10-14 18:26:33,880][61585] Updated weights for policy 1, policy_version 20440 (0.0007) [2023-10-14 18:26:34,036][61552] Updated weights for policy 0, policy_version 20512 (0.0009) [2023-10-14 18:26:37,897][61585] Updated weights for policy 1, policy_version 20450 (0.0008) [2023-10-14 18:26:38,167][61552] Updated weights for policy 0, policy_version 20522 (0.0008) [2023-10-14 18:26:38,264][61585] Updated weights for policy 1, policy_version 20460 (0.0008) [2023-10-14 18:26:38,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 41943040. Throughput: 0: 1659.2, 1: 1652.7. Samples: 10499864. Policy #0 lag: (min: 0.0, avg: 25.9, max: 32.0) [2023-10-14 18:26:38,344][60425] Avg episode reward: [(0, '54.640'), (1, '57.400')] [2023-10-14 18:26:38,529][61552] Updated weights for policy 0, policy_version 20532 (0.0008) [2023-10-14 18:26:38,637][61585] Updated weights for policy 1, policy_version 20470 (0.0008) [2023-10-14 18:26:38,897][61552] Updated weights for policy 0, policy_version 20542 (0.0007) [2023-10-14 18:26:38,996][61585] Updated weights for policy 1, policy_version 20480 (0.0008) [2023-10-14 18:26:43,161][61585] Updated weights for policy 1, policy_version 20490 (0.0007) [2023-10-14 18:26:43,187][61552] Updated weights for policy 0, policy_version 20552 (0.0009) [2023-10-14 18:26:43,344][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 42008576. Throughput: 0: 1655.2, 1: 1654.2. Samples: 10520030. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:26:43,345][60425] Avg episode reward: [(0, '53.620'), (1, '54.030')] [2023-10-14 18:26:43,534][61585] Updated weights for policy 1, policy_version 20500 (0.0008) [2023-10-14 18:26:43,564][61552] Updated weights for policy 0, policy_version 20562 (0.0007) [2023-10-14 18:26:43,893][61585] Updated weights for policy 1, policy_version 20510 (0.0009) [2023-10-14 18:26:43,935][61552] Updated weights for policy 0, policy_version 20572 (0.0007) [2023-10-14 18:26:47,955][61552] Updated weights for policy 0, policy_version 20582 (0.0007) [2023-10-14 18:26:48,166][61585] Updated weights for policy 1, policy_version 20520 (0.0007) [2023-10-14 18:26:48,332][61552] Updated weights for policy 0, policy_version 20592 (0.0008) [2023-10-14 18:26:48,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 42074112. Throughput: 0: 1659.3, 1: 1655.7. Samples: 10529216. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:26:48,344][60425] Avg episode reward: [(0, '51.450'), (1, '56.120')] [2023-10-14 18:26:48,539][61585] Updated weights for policy 1, policy_version 20530 (0.0010) [2023-10-14 18:26:48,710][61552] Updated weights for policy 0, policy_version 20602 (0.0009) [2023-10-14 18:26:48,908][61585] Updated weights for policy 1, policy_version 20540 (0.0008) [2023-10-14 18:26:52,896][61552] Updated weights for policy 0, policy_version 20612 (0.0009) [2023-10-14 18:26:53,035][61585] Updated weights for policy 1, policy_version 20550 (0.0008) [2023-10-14 18:26:53,288][61552] Updated weights for policy 0, policy_version 20622 (0.0007) [2023-10-14 18:26:53,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 42139648. Throughput: 0: 1661.8, 1: 1658.6. Samples: 10549826. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:26:53,344][60425] Avg episode reward: [(0, '51.750'), (1, '57.190')] [2023-10-14 18:26:53,409][61585] Updated weights for policy 1, policy_version 20560 (0.0009) [2023-10-14 18:26:53,655][61552] Updated weights for policy 0, policy_version 20632 (0.0008) [2023-10-14 18:26:53,785][61585] Updated weights for policy 1, policy_version 20570 (0.0010) [2023-10-14 18:26:57,890][61552] Updated weights for policy 0, policy_version 20642 (0.0009) [2023-10-14 18:26:58,007][61585] Updated weights for policy 1, policy_version 20580 (0.0007) [2023-10-14 18:26:58,262][61552] Updated weights for policy 0, policy_version 20652 (0.0007) [2023-10-14 18:26:58,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 42205184. Throughput: 0: 1658.8, 1: 1656.6. Samples: 10569886. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:26:58,345][60425] Avg episode reward: [(0, '50.470'), (1, '54.790')] [2023-10-14 18:26:58,381][61585] Updated weights for policy 1, policy_version 20590 (0.0008) [2023-10-14 18:26:58,621][61552] Updated weights for policy 0, policy_version 20662 (0.0007) [2023-10-14 18:26:58,742][61585] Updated weights for policy 1, policy_version 20600 (0.0008) [2023-10-14 18:26:58,992][61552] Updated weights for policy 0, policy_version 20672 (0.0007) [2023-10-14 18:27:02,841][61552] Updated weights for policy 0, policy_version 20682 (0.0009) [2023-10-14 18:27:02,982][61585] Updated weights for policy 1, policy_version 20610 (0.0009) [2023-10-14 18:27:03,209][61552] Updated weights for policy 0, policy_version 20692 (0.0009) [2023-10-14 18:27:03,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 42270720. Throughput: 0: 1658.8, 1: 1655.0. Samples: 10578892. Policy #0 lag: (min: 27.0, avg: 28.4, max: 52.0) [2023-10-14 18:27:03,344][60425] Avg episode reward: [(0, '51.890'), (1, '56.220')] [2023-10-14 18:27:03,346][61585] Updated weights for policy 1, policy_version 20620 (0.0008) [2023-10-14 18:27:03,582][61552] Updated weights for policy 0, policy_version 20702 (0.0008) [2023-10-14 18:27:03,710][61585] Updated weights for policy 1, policy_version 20630 (0.0008) [2023-10-14 18:27:04,081][61585] Updated weights for policy 1, policy_version 20640 (0.0011) [2023-10-14 18:27:07,708][61552] Updated weights for policy 0, policy_version 20712 (0.0010) [2023-10-14 18:27:08,070][61552] Updated weights for policy 0, policy_version 20722 (0.0010) [2023-10-14 18:27:08,283][61585] Updated weights for policy 1, policy_version 20650 (0.0009) [2023-10-14 18:27:08,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 42336256. Throughput: 0: 1663.3, 1: 1651.1. Samples: 10599070. Policy #0 lag: (min: 27.0, avg: 28.4, max: 52.0) [2023-10-14 18:27:08,345][60425] Avg episode reward: [(0, '51.990'), (1, '57.300')] [2023-10-14 18:27:08,442][61552] Updated weights for policy 0, policy_version 20732 (0.0008) [2023-10-14 18:27:08,645][61585] Updated weights for policy 1, policy_version 20660 (0.0009) [2023-10-14 18:27:09,017][61585] Updated weights for policy 1, policy_version 20670 (0.0009) [2023-10-14 18:27:12,585][61552] Updated weights for policy 0, policy_version 20742 (0.0007) [2023-10-14 18:27:12,956][61552] Updated weights for policy 0, policy_version 20752 (0.0007) [2023-10-14 18:27:13,028][61585] Updated weights for policy 1, policy_version 20680 (0.0009) [2023-10-14 18:27:13,329][61552] Updated weights for policy 0, policy_version 20762 (0.0009) [2023-10-14 18:27:13,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 42401792. Throughput: 0: 1654.8, 1: 1658.1. Samples: 10619398. Policy #0 lag: (min: 27.0, avg: 28.4, max: 52.0) [2023-10-14 18:27:13,344][60425] Avg episode reward: [(0, '49.800'), (1, '58.740')] [2023-10-14 18:27:13,384][61585] Updated weights for policy 1, policy_version 20690 (0.0008) [2023-10-14 18:27:13,752][61585] Updated weights for policy 1, policy_version 20700 (0.0008) [2023-10-14 18:27:17,588][61552] Updated weights for policy 0, policy_version 20772 (0.0008) [2023-10-14 18:27:17,896][61585] Updated weights for policy 1, policy_version 20710 (0.0009) [2023-10-14 18:27:17,944][61552] Updated weights for policy 0, policy_version 20782 (0.0008) [2023-10-14 18:27:18,256][61585] Updated weights for policy 1, policy_version 20720 (0.0007) [2023-10-14 18:27:18,312][61552] Updated weights for policy 0, policy_version 20792 (0.0009) [2023-10-14 18:27:18,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 42467328. Throughput: 0: 1664.4, 1: 1658.3. Samples: 10628814. Policy #0 lag: (min: 27.0, avg: 28.4, max: 52.0) [2023-10-14 18:27:18,344][60425] Avg episode reward: [(0, '49.620'), (1, '56.090')] [2023-10-14 18:27:18,622][61585] Updated weights for policy 1, policy_version 20730 (0.0007) [2023-10-14 18:27:22,491][61552] Updated weights for policy 0, policy_version 20802 (0.0009) [2023-10-14 18:27:22,853][61585] Updated weights for policy 1, policy_version 20740 (0.0007) [2023-10-14 18:27:22,859][61552] Updated weights for policy 0, policy_version 20812 (0.0007) [2023-10-14 18:27:23,225][61552] Updated weights for policy 0, policy_version 20822 (0.0008) [2023-10-14 18:27:23,254][61585] Updated weights for policy 1, policy_version 20750 (0.0007) [2023-10-14 18:27:23,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 42532864. Throughput: 0: 1660.6, 1: 1656.8. Samples: 10649144. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:27:23,344][60425] Avg episode reward: [(0, '52.960'), (1, '55.550')] [2023-10-14 18:27:23,596][61552] Updated weights for policy 0, policy_version 20832 (0.0007) [2023-10-14 18:27:23,613][61585] Updated weights for policy 1, policy_version 20760 (0.0007) [2023-10-14 18:27:27,562][61552] Updated weights for policy 0, policy_version 20842 (0.0009) [2023-10-14 18:27:27,711][61585] Updated weights for policy 1, policy_version 20770 (0.0009) [2023-10-14 18:27:27,936][61552] Updated weights for policy 0, policy_version 20852 (0.0008) [2023-10-14 18:27:28,073][61585] Updated weights for policy 1, policy_version 20780 (0.0009) [2023-10-14 18:27:28,305][61552] Updated weights for policy 0, policy_version 20862 (0.0007) [2023-10-14 18:27:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 42598400. Throughput: 0: 1656.5, 1: 1649.9. Samples: 10668818. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:27:28,344][60425] Avg episode reward: [(0, '52.690'), (1, '54.670')] [2023-10-14 18:27:28,438][61585] Updated weights for policy 1, policy_version 20790 (0.0009) [2023-10-14 18:27:28,804][61585] Updated weights for policy 1, policy_version 20800 (0.0010) [2023-10-14 18:27:32,422][61552] Updated weights for policy 0, policy_version 20872 (0.0008) [2023-10-14 18:27:32,793][61552] Updated weights for policy 0, policy_version 20882 (0.0009) [2023-10-14 18:27:32,872][61585] Updated weights for policy 1, policy_version 20810 (0.0008) [2023-10-14 18:27:33,162][61552] Updated weights for policy 0, policy_version 20892 (0.0008) [2023-10-14 18:27:33,236][61585] Updated weights for policy 1, policy_version 20820 (0.0009) [2023-10-14 18:27:33,343][60425] Fps is (10 sec: 16384.0, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 42696704. Throughput: 0: 1665.1, 1: 1650.2. Samples: 10678404. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:27:33,344][60425] Avg episode reward: [(0, '50.630'), (1, '55.670')] [2023-10-14 18:27:33,610][61585] Updated weights for policy 1, policy_version 20830 (0.0008) [2023-10-14 18:27:37,287][61552] Updated weights for policy 0, policy_version 20902 (0.0007) [2023-10-14 18:27:37,657][61552] Updated weights for policy 0, policy_version 20912 (0.0008) [2023-10-14 18:27:37,702][61585] Updated weights for policy 1, policy_version 20840 (0.0009) [2023-10-14 18:27:38,026][61552] Updated weights for policy 0, policy_version 20922 (0.0008) [2023-10-14 18:27:38,070][61585] Updated weights for policy 1, policy_version 20850 (0.0008) [2023-10-14 18:27:38,343][60425] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 42762240. Throughput: 0: 1662.9, 1: 1649.3. Samples: 10698876. Policy #0 lag: (min: 9.0, avg: 24.2, max: 41.0) [2023-10-14 18:27:38,344][60425] Avg episode reward: [(0, '55.500'), (1, '56.390')] [2023-10-14 18:27:38,445][61585] Updated weights for policy 1, policy_version 20860 (0.0008) [2023-10-14 18:27:42,341][61552] Updated weights for policy 0, policy_version 20932 (0.0007) [2023-10-14 18:27:42,667][61585] Updated weights for policy 1, policy_version 20870 (0.0010) [2023-10-14 18:27:42,728][61552] Updated weights for policy 0, policy_version 20942 (0.0007) [2023-10-14 18:27:43,036][61585] Updated weights for policy 1, policy_version 20880 (0.0009) [2023-10-14 18:27:43,109][61552] Updated weights for policy 0, policy_version 20952 (0.0009) [2023-10-14 18:27:43,343][60425] Fps is (10 sec: 9830.3, 60 sec: 13107.3, 300 sec: 13107.2). Total num frames: 42795008. Throughput: 0: 1651.3, 1: 1648.3. Samples: 10718370. Policy #0 lag: (min: 9.0, avg: 24.2, max: 41.0) [2023-10-14 18:27:43,344][60425] Avg episode reward: [(0, '54.360'), (1, '58.540')] [2023-10-14 18:27:43,393][61585] Updated weights for policy 1, policy_version 20890 (0.0009) [2023-10-14 18:27:43,395][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000020960_21463040.pth... [2023-10-14 18:27:43,424][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000019392_19857408.pth [2023-10-14 18:27:43,612][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000020896_21397504.pth... [2023-10-14 18:27:43,653][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000019328_19791872.pth [2023-10-14 18:27:47,086][61552] Updated weights for policy 0, policy_version 20962 (0.0007) [2023-10-14 18:27:47,461][61552] Updated weights for policy 0, policy_version 20972 (0.0008) [2023-10-14 18:27:47,646][61585] Updated weights for policy 1, policy_version 20900 (0.0008) [2023-10-14 18:27:47,826][61552] Updated weights for policy 0, policy_version 20982 (0.0009) [2023-10-14 18:27:48,016][61585] Updated weights for policy 1, policy_version 20910 (0.0010) [2023-10-14 18:27:48,195][61552] Updated weights for policy 0, policy_version 20992 (0.0008) [2023-10-14 18:27:48,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 42893312. Throughput: 0: 1662.5, 1: 1656.6. Samples: 10728250. Policy #0 lag: (min: 9.0, avg: 24.2, max: 41.0) [2023-10-14 18:27:48,344][60425] Avg episode reward: [(0, '55.000'), (1, '56.210')] [2023-10-14 18:27:48,385][61585] Updated weights for policy 1, policy_version 20920 (0.0010) [2023-10-14 18:27:52,236][61552] Updated weights for policy 0, policy_version 21002 (0.0008) [2023-10-14 18:27:52,467][61585] Updated weights for policy 1, policy_version 20930 (0.0008) [2023-10-14 18:27:52,608][61552] Updated weights for policy 0, policy_version 21012 (0.0009) [2023-10-14 18:27:52,832][61585] Updated weights for policy 1, policy_version 20940 (0.0008) [2023-10-14 18:27:52,976][61552] Updated weights for policy 0, policy_version 21022 (0.0007) [2023-10-14 18:27:53,203][61585] Updated weights for policy 1, policy_version 20950 (0.0007) [2023-10-14 18:27:53,343][60425] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 42958848. Throughput: 0: 1663.4, 1: 1663.2. Samples: 10748764. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-10-14 18:27:53,344][60425] Avg episode reward: [(0, '52.150'), (1, '56.360')] [2023-10-14 18:27:53,575][61585] Updated weights for policy 1, policy_version 20960 (0.0007) [2023-10-14 18:27:57,095][61552] Updated weights for policy 0, policy_version 21032 (0.0007) [2023-10-14 18:27:57,458][61552] Updated weights for policy 0, policy_version 21042 (0.0007) [2023-10-14 18:27:57,537][61585] Updated weights for policy 1, policy_version 20970 (0.0008) [2023-10-14 18:27:57,826][61552] Updated weights for policy 0, policy_version 21052 (0.0009) [2023-10-14 18:27:57,902][61585] Updated weights for policy 1, policy_version 20980 (0.0009) [2023-10-14 18:27:58,269][61585] Updated weights for policy 1, policy_version 20990 (0.0010) [2023-10-14 18:27:58,343][60425] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 43057152. Throughput: 0: 1652.8, 1: 1643.8. Samples: 10767742. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-10-14 18:27:58,344][60425] Avg episode reward: [(0, '56.040'), (1, '58.620')] [2023-10-14 18:28:01,892][61552] Updated weights for policy 0, policy_version 21062 (0.0009) [2023-10-14 18:28:02,255][61552] Updated weights for policy 0, policy_version 21072 (0.0009) [2023-10-14 18:28:02,485][61585] Updated weights for policy 1, policy_version 21000 (0.0008) [2023-10-14 18:28:02,619][61552] Updated weights for policy 0, policy_version 21082 (0.0009) [2023-10-14 18:28:02,857][61585] Updated weights for policy 1, policy_version 21010 (0.0008) [2023-10-14 18:28:03,219][61585] Updated weights for policy 1, policy_version 21020 (0.0010) [2023-10-14 18:28:03,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 43089920. Throughput: 0: 1668.1, 1: 1657.2. Samples: 10778450. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-10-14 18:28:03,344][60425] Avg episode reward: [(0, '53.790'), (1, '55.950')] [2023-10-14 18:28:06,776][61552] Updated weights for policy 0, policy_version 21092 (0.0009) [2023-10-14 18:28:07,148][61552] Updated weights for policy 0, policy_version 21102 (0.0008) [2023-10-14 18:28:07,474][61585] Updated weights for policy 1, policy_version 21030 (0.0008) [2023-10-14 18:28:07,513][61552] Updated weights for policy 0, policy_version 21112 (0.0008) [2023-10-14 18:28:07,857][61585] Updated weights for policy 1, policy_version 21040 (0.0007) [2023-10-14 18:28:08,227][61585] Updated weights for policy 1, policy_version 21050 (0.0008) [2023-10-14 18:28:08,343][60425] Fps is (10 sec: 9830.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 43155456. Throughput: 0: 1664.0, 1: 1659.1. Samples: 10798684. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-10-14 18:28:08,344][60425] Avg episode reward: [(0, '56.010'), (1, '53.760')] [2023-10-14 18:28:11,672][61552] Updated weights for policy 0, policy_version 21122 (0.0007) [2023-10-14 18:28:12,038][61552] Updated weights for policy 0, policy_version 21132 (0.0011) [2023-10-14 18:28:12,286][61585] Updated weights for policy 1, policy_version 21060 (0.0008) [2023-10-14 18:28:12,412][61552] Updated weights for policy 0, policy_version 21142 (0.0008) [2023-10-14 18:28:12,641][61585] Updated weights for policy 1, policy_version 21070 (0.0010) [2023-10-14 18:28:12,781][61552] Updated weights for policy 0, policy_version 21152 (0.0008) [2023-10-14 18:28:13,015][61585] Updated weights for policy 1, policy_version 21080 (0.0009) [2023-10-14 18:28:13,343][60425] Fps is (10 sec: 16383.8, 60 sec: 14199.4, 300 sec: 13329.4). Total num frames: 43253760. Throughput: 0: 1650.8, 1: 1653.2. Samples: 10817498. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:28:13,344][60425] Avg episode reward: [(0, '53.630'), (1, '58.090')] [2023-10-14 18:28:16,825][61552] Updated weights for policy 0, policy_version 21162 (0.0011) [2023-10-14 18:28:17,091][61585] Updated weights for policy 1, policy_version 21090 (0.0008) [2023-10-14 18:28:17,187][61552] Updated weights for policy 0, policy_version 21172 (0.0009) [2023-10-14 18:28:17,459][61585] Updated weights for policy 1, policy_version 21100 (0.0008) [2023-10-14 18:28:17,552][61552] Updated weights for policy 0, policy_version 21182 (0.0009) [2023-10-14 18:28:17,824][61585] Updated weights for policy 1, policy_version 21110 (0.0007) [2023-10-14 18:28:18,200][61585] Updated weights for policy 1, policy_version 21120 (0.0008) [2023-10-14 18:28:18,343][60425] Fps is (10 sec: 16384.3, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 43319296. Throughput: 0: 1663.6, 1: 1664.1. Samples: 10828150. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:28:18,344][60425] Avg episode reward: [(0, '58.310'), (1, '56.090')] [2023-10-14 18:28:21,759][61552] Updated weights for policy 0, policy_version 21192 (0.0008) [2023-10-14 18:28:22,130][61552] Updated weights for policy 0, policy_version 21202 (0.0009) [2023-10-14 18:28:22,190][61585] Updated weights for policy 1, policy_version 21130 (0.0008) [2023-10-14 18:28:22,495][61552] Updated weights for policy 0, policy_version 21212 (0.0007) [2023-10-14 18:28:22,557][61585] Updated weights for policy 1, policy_version 21140 (0.0009) [2023-10-14 18:28:22,922][61585] Updated weights for policy 1, policy_version 21150 (0.0008) [2023-10-14 18:28:23,343][60425] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 43384832. Throughput: 0: 1655.6, 1: 1663.2. Samples: 10848218. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:28:23,344][60425] Avg episode reward: [(0, '55.860'), (1, '58.060')] [2023-10-14 18:28:26,630][61552] Updated weights for policy 0, policy_version 21222 (0.0007) [2023-10-14 18:28:26,984][61585] Updated weights for policy 1, policy_version 21160 (0.0008) [2023-10-14 18:28:26,991][61552] Updated weights for policy 0, policy_version 21232 (0.0008) [2023-10-14 18:28:27,350][61585] Updated weights for policy 1, policy_version 21170 (0.0007) [2023-10-14 18:28:27,359][61552] Updated weights for policy 0, policy_version 21242 (0.0007) [2023-10-14 18:28:27,704][61585] Updated weights for policy 1, policy_version 21180 (0.0009) [2023-10-14 18:28:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 43450368. Throughput: 0: 1649.0, 1: 1644.0. Samples: 10866556. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:28:28,344][60425] Avg episode reward: [(0, '56.670'), (1, '60.090')] [2023-10-14 18:28:28,352][61248] Saving new best policy, reward=60.090! [2023-10-14 18:28:31,535][61552] Updated weights for policy 0, policy_version 21252 (0.0008) [2023-10-14 18:28:31,762][61585] Updated weights for policy 1, policy_version 21190 (0.0007) [2023-10-14 18:28:31,937][61552] Updated weights for policy 0, policy_version 21262 (0.0008) [2023-10-14 18:28:32,131][61585] Updated weights for policy 1, policy_version 21200 (0.0007) [2023-10-14 18:28:32,303][61552] Updated weights for policy 0, policy_version 21272 (0.0008) [2023-10-14 18:28:32,498][61585] Updated weights for policy 1, policy_version 21210 (0.0007) [2023-10-14 18:28:33,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 43515904. Throughput: 0: 1664.8, 1: 1658.9. Samples: 10877820. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-10-14 18:28:33,344][60425] Avg episode reward: [(0, '58.990'), (1, '55.300')] [2023-10-14 18:28:33,344][61172] Saving new best policy, reward=58.990! [2023-10-14 18:28:36,391][61552] Updated weights for policy 0, policy_version 21282 (0.0008) [2023-10-14 18:28:36,574][61585] Updated weights for policy 1, policy_version 21220 (0.0009) [2023-10-14 18:28:36,759][61552] Updated weights for policy 0, policy_version 21292 (0.0007) [2023-10-14 18:28:36,949][61585] Updated weights for policy 1, policy_version 21230 (0.0008) [2023-10-14 18:28:37,129][61552] Updated weights for policy 0, policy_version 21302 (0.0009) [2023-10-14 18:28:37,321][61585] Updated weights for policy 1, policy_version 21240 (0.0009) [2023-10-14 18:28:37,500][61552] Updated weights for policy 0, policy_version 21312 (0.0007) [2023-10-14 18:28:38,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 43581440. Throughput: 0: 1652.1, 1: 1652.8. Samples: 10897488. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-10-14 18:28:38,344][60425] Avg episode reward: [(0, '58.450'), (1, '54.530')] [2023-10-14 18:28:41,625][61552] Updated weights for policy 0, policy_version 21322 (0.0009) [2023-10-14 18:28:41,751][61585] Updated weights for policy 1, policy_version 21250 (0.0009) [2023-10-14 18:28:41,999][61552] Updated weights for policy 0, policy_version 21332 (0.0008) [2023-10-14 18:28:42,119][61585] Updated weights for policy 1, policy_version 21260 (0.0007) [2023-10-14 18:28:42,365][61552] Updated weights for policy 0, policy_version 21342 (0.0008) [2023-10-14 18:28:42,485][61585] Updated weights for policy 1, policy_version 21270 (0.0007) [2023-10-14 18:28:42,851][61585] Updated weights for policy 1, policy_version 21280 (0.0007) [2023-10-14 18:28:43,343][60425] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 43646976. Throughput: 0: 1653.3, 1: 1645.4. Samples: 10916184. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-10-14 18:28:43,344][60425] Avg episode reward: [(0, '52.720'), (1, '54.210')] [2023-10-14 18:28:46,556][61552] Updated weights for policy 0, policy_version 21352 (0.0007) [2023-10-14 18:28:46,922][61552] Updated weights for policy 0, policy_version 21362 (0.0008) [2023-10-14 18:28:47,051][61585] Updated weights for policy 1, policy_version 21290 (0.0007) [2023-10-14 18:28:47,287][61552] Updated weights for policy 0, policy_version 21372 (0.0007) [2023-10-14 18:28:47,415][61585] Updated weights for policy 1, policy_version 21300 (0.0008) [2023-10-14 18:28:47,782][61585] Updated weights for policy 1, policy_version 21310 (0.0008) [2023-10-14 18:28:48,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 43712512. Throughput: 0: 1657.5, 1: 1656.8. Samples: 10927592. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-10-14 18:28:48,344][60425] Avg episode reward: [(0, '55.250'), (1, '55.700')] [2023-10-14 18:28:51,457][61552] Updated weights for policy 0, policy_version 21382 (0.0008) [2023-10-14 18:28:51,815][61552] Updated weights for policy 0, policy_version 21392 (0.0009) [2023-10-14 18:28:52,135][61585] Updated weights for policy 1, policy_version 21320 (0.0009) [2023-10-14 18:28:52,187][61552] Updated weights for policy 0, policy_version 21402 (0.0008) [2023-10-14 18:28:52,490][61585] Updated weights for policy 1, policy_version 21330 (0.0009) [2023-10-14 18:28:52,863][61585] Updated weights for policy 1, policy_version 21340 (0.0010) [2023-10-14 18:28:53,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 43778048. Throughput: 0: 1651.2, 1: 1654.3. Samples: 10947428. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-14 18:28:53,344][60425] Avg episode reward: [(0, '58.010'), (1, '52.480')] [2023-10-14 18:28:56,127][61552] Updated weights for policy 0, policy_version 21412 (0.0010) [2023-10-14 18:28:56,494][61552] Updated weights for policy 0, policy_version 21422 (0.0011) [2023-10-14 18:28:56,755][61585] Updated weights for policy 1, policy_version 21350 (0.0009) [2023-10-14 18:28:56,862][61552] Updated weights for policy 0, policy_version 21432 (0.0008) [2023-10-14 18:28:57,136][61585] Updated weights for policy 1, policy_version 21360 (0.0009) [2023-10-14 18:28:57,492][61585] Updated weights for policy 1, policy_version 21370 (0.0008) [2023-10-14 18:28:58,344][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 43843584. Throughput: 0: 1658.6, 1: 1647.1. Samples: 10966256. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-14 18:28:58,345][60425] Avg episode reward: [(0, '54.710'), (1, '56.420')] [2023-10-14 18:29:00,945][61552] Updated weights for policy 0, policy_version 21442 (0.0007) [2023-10-14 18:29:01,319][61552] Updated weights for policy 0, policy_version 21452 (0.0008) [2023-10-14 18:29:01,529][61585] Updated weights for policy 1, policy_version 21380 (0.0007) [2023-10-14 18:29:01,689][61552] Updated weights for policy 0, policy_version 21462 (0.0008) [2023-10-14 18:29:01,895][61585] Updated weights for policy 1, policy_version 21390 (0.0008) [2023-10-14 18:29:02,053][61552] Updated weights for policy 0, policy_version 21472 (0.0008) [2023-10-14 18:29:02,263][61585] Updated weights for policy 1, policy_version 21400 (0.0007) [2023-10-14 18:29:03,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 43909120. Throughput: 0: 1665.9, 1: 1659.6. Samples: 10977794. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-14 18:29:03,344][60425] Avg episode reward: [(0, '55.130'), (1, '53.990')] [2023-10-14 18:29:06,093][61552] Updated weights for policy 0, policy_version 21482 (0.0008) [2023-10-14 18:29:06,280][61585] Updated weights for policy 1, policy_version 21410 (0.0009) [2023-10-14 18:29:06,467][61552] Updated weights for policy 0, policy_version 21492 (0.0010) [2023-10-14 18:29:06,643][61585] Updated weights for policy 1, policy_version 21420 (0.0009) [2023-10-14 18:29:06,826][61552] Updated weights for policy 0, policy_version 21502 (0.0010) [2023-10-14 18:29:07,009][61585] Updated weights for policy 1, policy_version 21430 (0.0008) [2023-10-14 18:29:07,369][61585] Updated weights for policy 1, policy_version 21440 (0.0009) [2023-10-14 18:29:08,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 43974656. Throughput: 0: 1653.2, 1: 1651.3. Samples: 10996920. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-14 18:29:08,344][60425] Avg episode reward: [(0, '52.500'), (1, '54.710')] [2023-10-14 18:29:10,698][61552] Updated weights for policy 0, policy_version 21512 (0.0007) [2023-10-14 18:29:11,070][61552] Updated weights for policy 0, policy_version 21522 (0.0009) [2023-10-14 18:29:11,437][61552] Updated weights for policy 0, policy_version 21532 (0.0008) [2023-10-14 18:29:11,501][61585] Updated weights for policy 1, policy_version 21450 (0.0009) [2023-10-14 18:29:11,872][61585] Updated weights for policy 1, policy_version 21460 (0.0008) [2023-10-14 18:29:12,232][61585] Updated weights for policy 1, policy_version 21470 (0.0007) [2023-10-14 18:29:13,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 44040192. Throughput: 0: 1677.2, 1: 1654.6. Samples: 11016488. Policy #0 lag: (min: 10.0, avg: 16.9, max: 42.0) [2023-10-14 18:29:13,344][60425] Avg episode reward: [(0, '57.540'), (1, '58.030')] [2023-10-14 18:29:15,597][61552] Updated weights for policy 0, policy_version 21542 (0.0008) [2023-10-14 18:29:15,968][61552] Updated weights for policy 0, policy_version 21552 (0.0010) [2023-10-14 18:29:16,336][61552] Updated weights for policy 0, policy_version 21562 (0.0008) [2023-10-14 18:29:16,398][61585] Updated weights for policy 1, policy_version 21480 (0.0007) [2023-10-14 18:29:16,760][61585] Updated weights for policy 1, policy_version 21490 (0.0008) [2023-10-14 18:29:17,126][61585] Updated weights for policy 1, policy_version 21500 (0.0011) [2023-10-14 18:29:18,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 44105728. Throughput: 0: 1672.4, 1: 1663.3. Samples: 11027930. Policy #0 lag: (min: 10.0, avg: 16.9, max: 42.0) [2023-10-14 18:29:18,345][60425] Avg episode reward: [(0, '52.930'), (1, '56.630')] [2023-10-14 18:29:20,314][61552] Updated weights for policy 0, policy_version 21572 (0.0008) [2023-10-14 18:29:20,686][61552] Updated weights for policy 0, policy_version 21582 (0.0010) [2023-10-14 18:29:21,055][61552] Updated weights for policy 0, policy_version 21592 (0.0007) [2023-10-14 18:29:21,548][61585] Updated weights for policy 1, policy_version 21510 (0.0010) [2023-10-14 18:29:21,914][61585] Updated weights for policy 1, policy_version 21520 (0.0010) [2023-10-14 18:29:22,285][61585] Updated weights for policy 1, policy_version 21530 (0.0010) [2023-10-14 18:29:23,344][60425] Fps is (10 sec: 13106.7, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 44171264. Throughput: 0: 1668.4, 1: 1651.7. Samples: 11046894. Policy #0 lag: (min: 10.0, avg: 16.9, max: 42.0) [2023-10-14 18:29:23,345][60425] Avg episode reward: [(0, '54.430'), (1, '58.840')] [2023-10-14 18:29:25,260][61552] Updated weights for policy 0, policy_version 21602 (0.0007) [2023-10-14 18:29:25,657][61552] Updated weights for policy 0, policy_version 21612 (0.0007) [2023-10-14 18:29:26,022][61552] Updated weights for policy 0, policy_version 21622 (0.0010) [2023-10-14 18:29:26,233][61585] Updated weights for policy 1, policy_version 21540 (0.0008) [2023-10-14 18:29:26,395][61552] Updated weights for policy 0, policy_version 21632 (0.0008) [2023-10-14 18:29:26,596][61585] Updated weights for policy 1, policy_version 21550 (0.0009) [2023-10-14 18:29:26,975][61585] Updated weights for policy 1, policy_version 21560 (0.0008) [2023-10-14 18:29:28,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 44236800. Throughput: 0: 1686.1, 1: 1660.2. Samples: 11066768. Policy #0 lag: (min: 10.0, avg: 16.9, max: 42.0) [2023-10-14 18:29:28,344][60425] Avg episode reward: [(0, '55.650'), (1, '57.450')] [2023-10-14 18:29:30,487][61552] Updated weights for policy 0, policy_version 21642 (0.0009) [2023-10-14 18:29:30,864][61552] Updated weights for policy 0, policy_version 21652 (0.0009) [2023-10-14 18:29:30,998][61585] Updated weights for policy 1, policy_version 21570 (0.0008) [2023-10-14 18:29:31,230][61552] Updated weights for policy 0, policy_version 21662 (0.0007) [2023-10-14 18:29:31,359][61585] Updated weights for policy 1, policy_version 21580 (0.0009) [2023-10-14 18:29:31,717][61585] Updated weights for policy 1, policy_version 21590 (0.0010) [2023-10-14 18:29:32,083][61585] Updated weights for policy 1, policy_version 21600 (0.0010) [2023-10-14 18:29:33,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 44302336. Throughput: 0: 1673.9, 1: 1665.0. Samples: 11077840. Policy #0 lag: (min: 28.0, avg: 29.4, max: 54.0) [2023-10-14 18:29:33,344][60425] Avg episode reward: [(0, '51.270'), (1, '56.380')] [2023-10-14 18:29:35,120][61552] Updated weights for policy 0, policy_version 21672 (0.0007) [2023-10-14 18:29:35,484][61552] Updated weights for policy 0, policy_version 21682 (0.0010) [2023-10-14 18:29:35,856][61552] Updated weights for policy 0, policy_version 21692 (0.0008) [2023-10-14 18:29:36,208][61585] Updated weights for policy 1, policy_version 21610 (0.0009) [2023-10-14 18:29:36,576][61585] Updated weights for policy 1, policy_version 21620 (0.0008) [2023-10-14 18:29:36,943][61585] Updated weights for policy 1, policy_version 21630 (0.0008) [2023-10-14 18:29:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 44367872. Throughput: 0: 1673.6, 1: 1648.0. Samples: 11096898. Policy #0 lag: (min: 28.0, avg: 29.4, max: 54.0) [2023-10-14 18:29:38,344][60425] Avg episode reward: [(0, '54.330'), (1, '54.200')] [2023-10-14 18:29:40,150][61552] Updated weights for policy 0, policy_version 21702 (0.0009) [2023-10-14 18:29:40,518][61552] Updated weights for policy 0, policy_version 21712 (0.0009) [2023-10-14 18:29:40,882][61552] Updated weights for policy 0, policy_version 21722 (0.0008) [2023-10-14 18:29:41,095][61585] Updated weights for policy 1, policy_version 21640 (0.0009) [2023-10-14 18:29:41,459][61585] Updated weights for policy 1, policy_version 21650 (0.0008) [2023-10-14 18:29:41,828][61585] Updated weights for policy 1, policy_version 21660 (0.0010) [2023-10-14 18:29:43,344][60425] Fps is (10 sec: 13106.7, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 44433408. Throughput: 0: 1687.3, 1: 1659.5. Samples: 11116862. Policy #0 lag: (min: 28.0, avg: 29.4, max: 54.0) [2023-10-14 18:29:43,345][60425] Avg episode reward: [(0, '54.310'), (1, '54.720')] [2023-10-14 18:29:43,356][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000021664_22183936.pth... [2023-10-14 18:29:43,356][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000021728_22249472.pth... [2023-10-14 18:29:43,385][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000020128_20611072.pth [2023-10-14 18:29:43,388][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000020192_20676608.pth [2023-10-14 18:29:44,930][61552] Updated weights for policy 0, policy_version 21732 (0.0008) [2023-10-14 18:29:45,294][61552] Updated weights for policy 0, policy_version 21742 (0.0007) [2023-10-14 18:29:45,663][61552] Updated weights for policy 0, policy_version 21752 (0.0008) [2023-10-14 18:29:46,264][61585] Updated weights for policy 1, policy_version 21670 (0.0009) [2023-10-14 18:29:46,641][61585] Updated weights for policy 1, policy_version 21680 (0.0010) [2023-10-14 18:29:47,011][61585] Updated weights for policy 1, policy_version 21690 (0.0010) [2023-10-14 18:29:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 44498944. Throughput: 0: 1663.8, 1: 1660.8. Samples: 11127402. Policy #0 lag: (min: 28.0, avg: 29.4, max: 54.0) [2023-10-14 18:29:48,344][60425] Avg episode reward: [(0, '55.110'), (1, '54.690')] [2023-10-14 18:29:49,663][61552] Updated weights for policy 0, policy_version 21762 (0.0009) [2023-10-14 18:29:50,028][61552] Updated weights for policy 0, policy_version 21772 (0.0007) [2023-10-14 18:29:50,408][61552] Updated weights for policy 0, policy_version 21782 (0.0009) [2023-10-14 18:29:50,782][61552] Updated weights for policy 0, policy_version 21792 (0.0010) [2023-10-14 18:29:51,113][61585] Updated weights for policy 1, policy_version 21700 (0.0010) [2023-10-14 18:29:51,471][61585] Updated weights for policy 1, policy_version 21710 (0.0009) [2023-10-14 18:29:51,835][61585] Updated weights for policy 1, policy_version 21720 (0.0008) [2023-10-14 18:29:53,343][60425] Fps is (10 sec: 13107.8, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 44564480. Throughput: 0: 1678.9, 1: 1648.3. Samples: 11146644. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:29:53,344][60425] Avg episode reward: [(0, '53.010'), (1, '54.450')] [2023-10-14 18:29:54,782][61552] Updated weights for policy 0, policy_version 21802 (0.0008) [2023-10-14 18:29:55,162][61552] Updated weights for policy 0, policy_version 21812 (0.0010) [2023-10-14 18:29:55,523][61552] Updated weights for policy 0, policy_version 21822 (0.0010) [2023-10-14 18:29:56,002][61585] Updated weights for policy 1, policy_version 21730 (0.0009) [2023-10-14 18:29:56,367][61585] Updated weights for policy 1, policy_version 21740 (0.0008) [2023-10-14 18:29:56,733][61585] Updated weights for policy 1, policy_version 21750 (0.0009) [2023-10-14 18:29:57,101][61585] Updated weights for policy 1, policy_version 21760 (0.0008) [2023-10-14 18:29:58,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 44630016. Throughput: 0: 1683.1, 1: 1661.2. Samples: 11166982. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:29:58,344][60425] Avg episode reward: [(0, '54.530'), (1, '56.310')] [2023-10-14 18:29:59,495][61552] Updated weights for policy 0, policy_version 21832 (0.0010) [2023-10-14 18:29:59,866][61552] Updated weights for policy 0, policy_version 21842 (0.0008) [2023-10-14 18:30:00,236][61552] Updated weights for policy 0, policy_version 21852 (0.0007) [2023-10-14 18:30:01,267][61585] Updated weights for policy 1, policy_version 21770 (0.0007) [2023-10-14 18:30:01,642][61585] Updated weights for policy 1, policy_version 21780 (0.0008) [2023-10-14 18:30:01,994][61585] Updated weights for policy 1, policy_version 21790 (0.0009) [2023-10-14 18:30:03,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 44695552. Throughput: 0: 1663.8, 1: 1659.3. Samples: 11177466. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:30:03,344][60425] Avg episode reward: [(0, '56.280'), (1, '59.270')] [2023-10-14 18:30:04,420][61552] Updated weights for policy 0, policy_version 21862 (0.0011) [2023-10-14 18:30:04,783][61552] Updated weights for policy 0, policy_version 21872 (0.0007) [2023-10-14 18:30:05,154][61552] Updated weights for policy 0, policy_version 21882 (0.0008) [2023-10-14 18:30:06,042][61585] Updated weights for policy 1, policy_version 21800 (0.0008) [2023-10-14 18:30:06,409][61585] Updated weights for policy 1, policy_version 21810 (0.0009) [2023-10-14 18:30:06,783][61585] Updated weights for policy 1, policy_version 21820 (0.0009) [2023-10-14 18:30:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 44761088. Throughput: 0: 1681.9, 1: 1656.8. Samples: 11197134. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:30:08,344][60425] Avg episode reward: [(0, '56.920'), (1, '57.150')] [2023-10-14 18:30:09,159][61552] Updated weights for policy 0, policy_version 21892 (0.0008) [2023-10-14 18:30:09,540][61552] Updated weights for policy 0, policy_version 21902 (0.0007) [2023-10-14 18:30:09,911][61552] Updated weights for policy 0, policy_version 21912 (0.0009) [2023-10-14 18:30:10,894][61585] Updated weights for policy 1, policy_version 21830 (0.0008) [2023-10-14 18:30:11,261][61585] Updated weights for policy 1, policy_version 21840 (0.0008) [2023-10-14 18:30:11,626][61585] Updated weights for policy 1, policy_version 21850 (0.0008) [2023-10-14 18:30:13,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 44826624. Throughput: 0: 1683.6, 1: 1668.4. Samples: 11217612. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-14 18:30:13,344][60425] Avg episode reward: [(0, '57.850'), (1, '57.590')] [2023-10-14 18:30:14,183][61552] Updated weights for policy 0, policy_version 21922 (0.0009) [2023-10-14 18:30:14,577][61552] Updated weights for policy 0, policy_version 21932 (0.0008) [2023-10-14 18:30:14,942][61552] Updated weights for policy 0, policy_version 21942 (0.0010) [2023-10-14 18:30:15,317][61552] Updated weights for policy 0, policy_version 21952 (0.0009) [2023-10-14 18:30:15,774][61585] Updated weights for policy 1, policy_version 21860 (0.0008) [2023-10-14 18:30:16,139][61585] Updated weights for policy 1, policy_version 21870 (0.0008) [2023-10-14 18:30:16,495][61585] Updated weights for policy 1, policy_version 21880 (0.0009) [2023-10-14 18:30:18,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 44892160. Throughput: 0: 1663.6, 1: 1657.5. Samples: 11227292. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-14 18:30:18,344][60425] Avg episode reward: [(0, '58.720'), (1, '55.570')] [2023-10-14 18:30:19,242][61552] Updated weights for policy 0, policy_version 21962 (0.0010) [2023-10-14 18:30:19,618][61552] Updated weights for policy 0, policy_version 21972 (0.0008) [2023-10-14 18:30:19,992][61552] Updated weights for policy 0, policy_version 21982 (0.0009) [2023-10-14 18:30:20,602][61585] Updated weights for policy 1, policy_version 21890 (0.0010) [2023-10-14 18:30:20,972][61585] Updated weights for policy 1, policy_version 21900 (0.0008) [2023-10-14 18:30:21,335][61585] Updated weights for policy 1, policy_version 21910 (0.0008) [2023-10-14 18:30:21,703][61585] Updated weights for policy 1, policy_version 21920 (0.0008) [2023-10-14 18:30:23,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 44957696. Throughput: 0: 1678.1, 1: 1655.4. Samples: 11246904. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-14 18:30:23,344][60425] Avg episode reward: [(0, '59.040'), (1, '56.180')] [2023-10-14 18:30:23,345][61172] Saving new best policy, reward=59.040! [2023-10-14 18:30:24,286][61552] Updated weights for policy 0, policy_version 21992 (0.0008) [2023-10-14 18:30:24,655][61552] Updated weights for policy 0, policy_version 22002 (0.0010) [2023-10-14 18:30:25,036][61552] Updated weights for policy 0, policy_version 22012 (0.0008) [2023-10-14 18:30:25,820][61585] Updated weights for policy 1, policy_version 21930 (0.0010) [2023-10-14 18:30:26,195][61585] Updated weights for policy 1, policy_version 21940 (0.0010) [2023-10-14 18:30:26,557][61585] Updated weights for policy 1, policy_version 21950 (0.0010) [2023-10-14 18:30:28,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 45023232. Throughput: 0: 1677.5, 1: 1665.0. Samples: 11267276. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-14 18:30:28,344][60425] Avg episode reward: [(0, '57.090'), (1, '56.110')] [2023-10-14 18:30:29,190][61552] Updated weights for policy 0, policy_version 22022 (0.0008) [2023-10-14 18:30:29,550][61552] Updated weights for policy 0, policy_version 22032 (0.0008) [2023-10-14 18:30:29,930][61552] Updated weights for policy 0, policy_version 22042 (0.0008) [2023-10-14 18:30:30,691][61585] Updated weights for policy 1, policy_version 21960 (0.0008) [2023-10-14 18:30:31,073][61585] Updated weights for policy 1, policy_version 21970 (0.0008) [2023-10-14 18:30:31,443][61585] Updated weights for policy 1, policy_version 21980 (0.0009) [2023-10-14 18:30:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 45088768. Throughput: 0: 1674.1, 1: 1657.8. Samples: 11277338. Policy #0 lag: (min: 21.0, avg: 31.0, max: 53.0) [2023-10-14 18:30:33,344][60425] Avg episode reward: [(0, '58.200'), (1, '58.300')] [2023-10-14 18:30:33,823][61552] Updated weights for policy 0, policy_version 22052 (0.0007) [2023-10-14 18:30:34,198][61552] Updated weights for policy 0, policy_version 22062 (0.0008) [2023-10-14 18:30:34,568][61552] Updated weights for policy 0, policy_version 22072 (0.0007) [2023-10-14 18:30:35,380][61585] Updated weights for policy 1, policy_version 21990 (0.0008) [2023-10-14 18:30:35,739][61585] Updated weights for policy 1, policy_version 22000 (0.0007) [2023-10-14 18:30:36,104][61585] Updated weights for policy 1, policy_version 22010 (0.0009) [2023-10-14 18:30:38,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 45154304. Throughput: 0: 1676.4, 1: 1663.4. Samples: 11296938. Policy #0 lag: (min: 21.0, avg: 31.0, max: 53.0) [2023-10-14 18:30:38,344][60425] Avg episode reward: [(0, '56.720'), (1, '57.720')] [2023-10-14 18:30:38,713][61552] Updated weights for policy 0, policy_version 22082 (0.0009) [2023-10-14 18:30:39,098][61552] Updated weights for policy 0, policy_version 22092 (0.0009) [2023-10-14 18:30:39,461][61552] Updated weights for policy 0, policy_version 22102 (0.0008) [2023-10-14 18:30:39,823][61552] Updated weights for policy 0, policy_version 22112 (0.0010) [2023-10-14 18:30:40,172][61585] Updated weights for policy 1, policy_version 22020 (0.0009) [2023-10-14 18:30:40,540][61585] Updated weights for policy 1, policy_version 22030 (0.0007) [2023-10-14 18:30:40,923][61585] Updated weights for policy 1, policy_version 22040 (0.0009) [2023-10-14 18:30:43,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 45219840. Throughput: 0: 1668.5, 1: 1674.9. Samples: 11317436. Policy #0 lag: (min: 21.0, avg: 31.0, max: 53.0) [2023-10-14 18:30:43,344][60425] Avg episode reward: [(0, '59.060'), (1, '56.400')] [2023-10-14 18:30:43,354][61172] Saving new best policy, reward=59.060! [2023-10-14 18:30:43,943][61552] Updated weights for policy 0, policy_version 22122 (0.0011) [2023-10-14 18:30:44,305][61552] Updated weights for policy 0, policy_version 22132 (0.0007) [2023-10-14 18:30:44,673][61552] Updated weights for policy 0, policy_version 22142 (0.0008) [2023-10-14 18:30:45,054][61585] Updated weights for policy 1, policy_version 22050 (0.0011) [2023-10-14 18:30:45,433][61585] Updated weights for policy 1, policy_version 22060 (0.0010) [2023-10-14 18:30:45,793][61585] Updated weights for policy 1, policy_version 22070 (0.0010) [2023-10-14 18:30:46,156][61585] Updated weights for policy 1, policy_version 22080 (0.0011) [2023-10-14 18:30:48,344][60425] Fps is (10 sec: 13106.0, 60 sec: 13107.0, 300 sec: 13329.3). Total num frames: 45285376. Throughput: 0: 1665.7, 1: 1658.9. Samples: 11327076. Policy #0 lag: (min: 21.0, avg: 31.0, max: 53.0) [2023-10-14 18:30:48,345][60425] Avg episode reward: [(0, '61.190'), (1, '54.570')] [2023-10-14 18:30:48,346][61172] Saving new best policy, reward=61.190! [2023-10-14 18:30:48,779][61552] Updated weights for policy 0, policy_version 22152 (0.0010) [2023-10-14 18:30:49,162][61552] Updated weights for policy 0, policy_version 22162 (0.0010) [2023-10-14 18:30:49,531][61552] Updated weights for policy 0, policy_version 22172 (0.0011) [2023-10-14 18:30:50,192][61585] Updated weights for policy 1, policy_version 22090 (0.0010) [2023-10-14 18:30:50,567][61585] Updated weights for policy 1, policy_version 22100 (0.0007) [2023-10-14 18:30:50,937][61585] Updated weights for policy 1, policy_version 22110 (0.0007) [2023-10-14 18:30:53,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 45350912. Throughput: 0: 1660.8, 1: 1666.5. Samples: 11346864. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:30:53,345][60425] Avg episode reward: [(0, '58.310'), (1, '52.400')] [2023-10-14 18:30:53,806][61552] Updated weights for policy 0, policy_version 22182 (0.0009) [2023-10-14 18:30:54,167][61552] Updated weights for policy 0, policy_version 22192 (0.0009) [2023-10-14 18:30:54,535][61552] Updated weights for policy 0, policy_version 22202 (0.0010) [2023-10-14 18:30:54,941][61585] Updated weights for policy 1, policy_version 22120 (0.0010) [2023-10-14 18:30:55,301][61585] Updated weights for policy 1, policy_version 22130 (0.0011) [2023-10-14 18:30:55,671][61585] Updated weights for policy 1, policy_version 22140 (0.0011) [2023-10-14 18:30:58,343][60425] Fps is (10 sec: 13108.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 45416448. Throughput: 0: 1659.2, 1: 1670.8. Samples: 11367458. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:30:58,344][60425] Avg episode reward: [(0, '56.920'), (1, '53.270')] [2023-10-14 18:30:58,699][61552] Updated weights for policy 0, policy_version 22212 (0.0008) [2023-10-14 18:30:59,068][61552] Updated weights for policy 0, policy_version 22222 (0.0009) [2023-10-14 18:30:59,433][61552] Updated weights for policy 0, policy_version 22232 (0.0010) [2023-10-14 18:30:59,835][61585] Updated weights for policy 1, policy_version 22150 (0.0007) [2023-10-14 18:31:00,203][61585] Updated weights for policy 1, policy_version 22160 (0.0009) [2023-10-14 18:31:00,575][61585] Updated weights for policy 1, policy_version 22170 (0.0009) [2023-10-14 18:31:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 45481984. Throughput: 0: 1660.9, 1: 1652.6. Samples: 11376402. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:31:03,344][60425] Avg episode reward: [(0, '57.730'), (1, '55.460')] [2023-10-14 18:31:03,412][61552] Updated weights for policy 0, policy_version 22242 (0.0008) [2023-10-14 18:31:03,809][61552] Updated weights for policy 0, policy_version 22252 (0.0008) [2023-10-14 18:31:04,169][61552] Updated weights for policy 0, policy_version 22262 (0.0010) [2023-10-14 18:31:04,539][61552] Updated weights for policy 0, policy_version 22272 (0.0007) [2023-10-14 18:31:04,699][61585] Updated weights for policy 1, policy_version 22180 (0.0007) [2023-10-14 18:31:05,064][61585] Updated weights for policy 1, policy_version 22190 (0.0007) [2023-10-14 18:31:05,421][61585] Updated weights for policy 1, policy_version 22200 (0.0010) [2023-10-14 18:31:08,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 45547520. Throughput: 0: 1662.0, 1: 1671.8. Samples: 11396926. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:31:08,344][60425] Avg episode reward: [(0, '57.420'), (1, '56.470')] [2023-10-14 18:31:08,538][61552] Updated weights for policy 0, policy_version 22282 (0.0010) [2023-10-14 18:31:08,918][61552] Updated weights for policy 0, policy_version 22292 (0.0010) [2023-10-14 18:31:09,282][61552] Updated weights for policy 0, policy_version 22302 (0.0010) [2023-10-14 18:31:09,567][61585] Updated weights for policy 1, policy_version 22210 (0.0009) [2023-10-14 18:31:09,946][61585] Updated weights for policy 1, policy_version 22220 (0.0009) [2023-10-14 18:31:10,313][61585] Updated weights for policy 1, policy_version 22230 (0.0009) [2023-10-14 18:31:10,672][61585] Updated weights for policy 1, policy_version 22240 (0.0010) [2023-10-14 18:31:13,340][61552] Updated weights for policy 0, policy_version 22312 (0.0009) [2023-10-14 18:31:13,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 45613056. Throughput: 0: 1665.4, 1: 1674.3. Samples: 11417564. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-14 18:31:13,344][60425] Avg episode reward: [(0, '55.420'), (1, '58.890')] [2023-10-14 18:31:13,705][61552] Updated weights for policy 0, policy_version 22322 (0.0008) [2023-10-14 18:31:14,074][61552] Updated weights for policy 0, policy_version 22332 (0.0009) [2023-10-14 18:31:14,774][61585] Updated weights for policy 1, policy_version 22250 (0.0007) [2023-10-14 18:31:15,142][61585] Updated weights for policy 1, policy_version 22260 (0.0009) [2023-10-14 18:31:15,515][61585] Updated weights for policy 1, policy_version 22270 (0.0008) [2023-10-14 18:31:18,116][61552] Updated weights for policy 0, policy_version 22342 (0.0010) [2023-10-14 18:31:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 45678592. Throughput: 0: 1663.5, 1: 1655.0. Samples: 11426670. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-14 18:31:18,344][60425] Avg episode reward: [(0, '58.270'), (1, '57.250')] [2023-10-14 18:31:18,487][61552] Updated weights for policy 0, policy_version 22352 (0.0010) [2023-10-14 18:31:18,863][61552] Updated weights for policy 0, policy_version 22362 (0.0007) [2023-10-14 18:31:19,538][61585] Updated weights for policy 1, policy_version 22280 (0.0009) [2023-10-14 18:31:19,904][61585] Updated weights for policy 1, policy_version 22290 (0.0008) [2023-10-14 18:31:20,270][61585] Updated weights for policy 1, policy_version 22300 (0.0007) [2023-10-14 18:31:22,903][61552] Updated weights for policy 0, policy_version 22372 (0.0008) [2023-10-14 18:31:23,273][61552] Updated weights for policy 0, policy_version 22382 (0.0009) [2023-10-14 18:31:23,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 45744128. Throughput: 0: 1671.2, 1: 1672.8. Samples: 11447420. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-14 18:31:23,344][60425] Avg episode reward: [(0, '53.970'), (1, '56.090')] [2023-10-14 18:31:23,644][61552] Updated weights for policy 0, policy_version 22392 (0.0008) [2023-10-14 18:31:24,389][61585] Updated weights for policy 1, policy_version 22310 (0.0008) [2023-10-14 18:31:24,774][61585] Updated weights for policy 1, policy_version 22320 (0.0011) [2023-10-14 18:31:25,151][61585] Updated weights for policy 1, policy_version 22330 (0.0010) [2023-10-14 18:31:27,828][61552] Updated weights for policy 0, policy_version 22402 (0.0009) [2023-10-14 18:31:28,195][61552] Updated weights for policy 0, policy_version 22412 (0.0009) [2023-10-14 18:31:28,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 45809664. Throughput: 0: 1678.6, 1: 1667.7. Samples: 11468020. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-14 18:31:28,344][60425] Avg episode reward: [(0, '54.020'), (1, '58.170')] [2023-10-14 18:31:28,570][61552] Updated weights for policy 0, policy_version 22422 (0.0007) [2023-10-14 18:31:28,936][61552] Updated weights for policy 0, policy_version 22432 (0.0009) [2023-10-14 18:31:29,052][61585] Updated weights for policy 1, policy_version 22340 (0.0009) [2023-10-14 18:31:29,418][61585] Updated weights for policy 1, policy_version 22350 (0.0007) [2023-10-14 18:31:29,778][61585] Updated weights for policy 1, policy_version 22360 (0.0009) [2023-10-14 18:31:33,154][61552] Updated weights for policy 0, policy_version 22442 (0.0007) [2023-10-14 18:31:33,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 45875200. Throughput: 0: 1678.3, 1: 1657.5. Samples: 11477184. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-14 18:31:33,344][60425] Avg episode reward: [(0, '53.860'), (1, '55.560')] [2023-10-14 18:31:33,531][61552] Updated weights for policy 0, policy_version 22452 (0.0009) [2023-10-14 18:31:33,903][61552] Updated weights for policy 0, policy_version 22462 (0.0008) [2023-10-14 18:31:33,949][61585] Updated weights for policy 1, policy_version 22370 (0.0007) [2023-10-14 18:31:34,315][61585] Updated weights for policy 1, policy_version 22380 (0.0008) [2023-10-14 18:31:34,685][61585] Updated weights for policy 1, policy_version 22390 (0.0008) [2023-10-14 18:31:35,048][61585] Updated weights for policy 1, policy_version 22400 (0.0009) [2023-10-14 18:31:37,892][61552] Updated weights for policy 0, policy_version 22472 (0.0010) [2023-10-14 18:31:38,269][61552] Updated weights for policy 0, policy_version 22482 (0.0007) [2023-10-14 18:31:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 45940736. Throughput: 0: 1687.8, 1: 1673.1. Samples: 11498106. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-14 18:31:38,344][60425] Avg episode reward: [(0, '55.500'), (1, '55.540')] [2023-10-14 18:31:38,646][61552] Updated weights for policy 0, policy_version 22492 (0.0009) [2023-10-14 18:31:39,175][61585] Updated weights for policy 1, policy_version 22410 (0.0009) [2023-10-14 18:31:39,540][61585] Updated weights for policy 1, policy_version 22420 (0.0008) [2023-10-14 18:31:39,903][61585] Updated weights for policy 1, policy_version 22430 (0.0009) [2023-10-14 18:31:42,711][61552] Updated weights for policy 0, policy_version 22502 (0.0009) [2023-10-14 18:31:43,071][61552] Updated weights for policy 0, policy_version 22512 (0.0007) [2023-10-14 18:31:43,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 46006272. Throughput: 0: 1680.6, 1: 1672.0. Samples: 11518322. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-14 18:31:43,344][60425] Avg episode reward: [(0, '55.320'), (1, '57.180')] [2023-10-14 18:31:43,350][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000022432_22970368.pth... [2023-10-14 18:31:43,383][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000020896_21397504.pth [2023-10-14 18:31:43,437][61552] Updated weights for policy 0, policy_version 22522 (0.0007) [2023-10-14 18:31:43,656][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000022528_23068672.pth... [2023-10-14 18:31:43,694][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000020960_21463040.pth [2023-10-14 18:31:44,063][61585] Updated weights for policy 1, policy_version 22440 (0.0010) [2023-10-14 18:31:44,426][61585] Updated weights for policy 1, policy_version 22450 (0.0010) [2023-10-14 18:31:44,791][61585] Updated weights for policy 1, policy_version 22460 (0.0009) [2023-10-14 18:31:47,583][61552] Updated weights for policy 0, policy_version 22532 (0.0008) [2023-10-14 18:31:47,948][61552] Updated weights for policy 0, policy_version 22542 (0.0010) [2023-10-14 18:31:48,317][61552] Updated weights for policy 0, policy_version 22552 (0.0008) [2023-10-14 18:31:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.4, 300 sec: 13329.4). Total num frames: 46071808. Throughput: 0: 1689.6, 1: 1671.0. Samples: 11527628. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-14 18:31:48,344][60425] Avg episode reward: [(0, '55.660'), (1, '58.560')] [2023-10-14 18:31:48,831][61585] Updated weights for policy 1, policy_version 22470 (0.0008) [2023-10-14 18:31:49,202][61585] Updated weights for policy 1, policy_version 22480 (0.0010) [2023-10-14 18:31:49,578][61585] Updated weights for policy 1, policy_version 22490 (0.0010) [2023-10-14 18:31:52,444][61552] Updated weights for policy 0, policy_version 22562 (0.0010) [2023-10-14 18:31:52,851][61552] Updated weights for policy 0, policy_version 22572 (0.0009) [2023-10-14 18:31:53,215][61552] Updated weights for policy 0, policy_version 22582 (0.0008) [2023-10-14 18:31:53,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 46137344. Throughput: 0: 1681.1, 1: 1672.7. Samples: 11547846. Policy #0 lag: (min: 31.0, avg: 32.6, max: 58.0) [2023-10-14 18:31:53,344][60425] Avg episode reward: [(0, '57.440'), (1, '53.600')] [2023-10-14 18:31:53,571][61552] Updated weights for policy 0, policy_version 22592 (0.0008) [2023-10-14 18:31:53,768][61585] Updated weights for policy 1, policy_version 22500 (0.0010) [2023-10-14 18:31:54,128][61585] Updated weights for policy 1, policy_version 22510 (0.0008) [2023-10-14 18:31:54,499][61585] Updated weights for policy 1, policy_version 22520 (0.0009) [2023-10-14 18:31:57,571][61552] Updated weights for policy 0, policy_version 22602 (0.0009) [2023-10-14 18:31:57,938][61552] Updated weights for policy 0, policy_version 22612 (0.0008) [2023-10-14 18:31:58,305][61552] Updated weights for policy 0, policy_version 22622 (0.0010) [2023-10-14 18:31:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 46202880. Throughput: 0: 1669.1, 1: 1670.5. Samples: 11567848. Policy #0 lag: (min: 31.0, avg: 32.6, max: 58.0) [2023-10-14 18:31:58,344][60425] Avg episode reward: [(0, '55.620'), (1, '55.380')] [2023-10-14 18:31:58,583][61585] Updated weights for policy 1, policy_version 22530 (0.0008) [2023-10-14 18:31:58,952][61585] Updated weights for policy 1, policy_version 22540 (0.0009) [2023-10-14 18:31:59,311][61585] Updated weights for policy 1, policy_version 22550 (0.0010) [2023-10-14 18:31:59,678][61585] Updated weights for policy 1, policy_version 22560 (0.0009) [2023-10-14 18:32:02,435][61552] Updated weights for policy 0, policy_version 22632 (0.0008) [2023-10-14 18:32:02,808][61552] Updated weights for policy 0, policy_version 22642 (0.0007) [2023-10-14 18:32:03,167][61552] Updated weights for policy 0, policy_version 22652 (0.0008) [2023-10-14 18:32:03,343][60425] Fps is (10 sec: 16384.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 46301184. Throughput: 0: 1681.9, 1: 1668.5. Samples: 11577438. Policy #0 lag: (min: 31.0, avg: 32.6, max: 58.0) [2023-10-14 18:32:03,344][60425] Avg episode reward: [(0, '56.880'), (1, '54.410')] [2023-10-14 18:32:03,689][61585] Updated weights for policy 1, policy_version 22570 (0.0008) [2023-10-14 18:32:04,053][61585] Updated weights for policy 1, policy_version 22580 (0.0007) [2023-10-14 18:32:04,416][61585] Updated weights for policy 1, policy_version 22590 (0.0007) [2023-10-14 18:32:07,292][61552] Updated weights for policy 0, policy_version 22662 (0.0008) [2023-10-14 18:32:07,657][61552] Updated weights for policy 0, policy_version 22672 (0.0010) [2023-10-14 18:32:08,022][61552] Updated weights for policy 0, policy_version 22682 (0.0009) [2023-10-14 18:32:08,343][60425] Fps is (10 sec: 16384.0, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 46366720. Throughput: 0: 1676.8, 1: 1673.2. Samples: 11598168. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-14 18:32:08,344][60425] Avg episode reward: [(0, '55.200'), (1, '55.180')] [2023-10-14 18:32:08,574][61585] Updated weights for policy 1, policy_version 22600 (0.0008) [2023-10-14 18:32:08,945][61585] Updated weights for policy 1, policy_version 22610 (0.0009) [2023-10-14 18:32:09,312][61585] Updated weights for policy 1, policy_version 22620 (0.0008) [2023-10-14 18:32:12,024][61552] Updated weights for policy 0, policy_version 22692 (0.0010) [2023-10-14 18:32:12,402][61552] Updated weights for policy 0, policy_version 22702 (0.0010) [2023-10-14 18:32:12,767][61552] Updated weights for policy 0, policy_version 22712 (0.0008) [2023-10-14 18:32:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 46432256. Throughput: 0: 1657.7, 1: 1675.7. Samples: 11618026. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-14 18:32:13,344][60425] Avg episode reward: [(0, '55.650'), (1, '56.490')] [2023-10-14 18:32:13,458][61585] Updated weights for policy 1, policy_version 22630 (0.0007) [2023-10-14 18:32:13,836][61585] Updated weights for policy 1, policy_version 22640 (0.0009) [2023-10-14 18:32:14,205][61585] Updated weights for policy 1, policy_version 22650 (0.0009) [2023-10-14 18:32:16,637][61552] Updated weights for policy 0, policy_version 22722 (0.0009) [2023-10-14 18:32:17,006][61552] Updated weights for policy 0, policy_version 22732 (0.0007) [2023-10-14 18:32:17,366][61552] Updated weights for policy 0, policy_version 22742 (0.0007) [2023-10-14 18:32:17,734][61552] Updated weights for policy 0, policy_version 22752 (0.0008) [2023-10-14 18:32:18,337][61585] Updated weights for policy 1, policy_version 22660 (0.0009) [2023-10-14 18:32:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 46497792. Throughput: 0: 1678.5, 1: 1670.4. Samples: 11627886. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-14 18:32:18,344][60425] Avg episode reward: [(0, '54.650'), (1, '55.360')] [2023-10-14 18:32:18,706][61585] Updated weights for policy 1, policy_version 22670 (0.0008) [2023-10-14 18:32:19,075][61585] Updated weights for policy 1, policy_version 22680 (0.0007) [2023-10-14 18:32:21,825][61552] Updated weights for policy 0, policy_version 22762 (0.0009) [2023-10-14 18:32:22,194][61552] Updated weights for policy 0, policy_version 22772 (0.0011) [2023-10-14 18:32:22,567][61552] Updated weights for policy 0, policy_version 22782 (0.0009) [2023-10-14 18:32:23,118][61585] Updated weights for policy 1, policy_version 22690 (0.0008) [2023-10-14 18:32:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 46563328. Throughput: 0: 1667.8, 1: 1666.0. Samples: 11648124. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-14 18:32:23,344][60425] Avg episode reward: [(0, '56.790'), (1, '56.080')] [2023-10-14 18:32:23,480][61585] Updated weights for policy 1, policy_version 22700 (0.0011) [2023-10-14 18:32:23,854][61585] Updated weights for policy 1, policy_version 22710 (0.0011) [2023-10-14 18:32:24,236][61585] Updated weights for policy 1, policy_version 22720 (0.0011) [2023-10-14 18:32:26,646][61552] Updated weights for policy 0, policy_version 22792 (0.0008) [2023-10-14 18:32:27,018][61552] Updated weights for policy 0, policy_version 22802 (0.0008) [2023-10-14 18:32:27,385][61552] Updated weights for policy 0, policy_version 22812 (0.0007) [2023-10-14 18:32:28,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 46628864. Throughput: 0: 1656.1, 1: 1667.8. Samples: 11667898. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-14 18:32:28,344][60425] Avg episode reward: [(0, '54.950'), (1, '53.360')] [2023-10-14 18:32:28,388][61585] Updated weights for policy 1, policy_version 22730 (0.0009) [2023-10-14 18:32:28,746][61585] Updated weights for policy 1, policy_version 22740 (0.0010) [2023-10-14 18:32:29,112][61585] Updated weights for policy 1, policy_version 22750 (0.0009) [2023-10-14 18:32:31,391][61552] Updated weights for policy 0, policy_version 22822 (0.0008) [2023-10-14 18:32:31,761][61552] Updated weights for policy 0, policy_version 22832 (0.0010) [2023-10-14 18:32:32,130][61552] Updated weights for policy 0, policy_version 22842 (0.0008) [2023-10-14 18:32:33,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 46694400. Throughput: 0: 1682.9, 1: 1663.7. Samples: 11678226. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-14 18:32:33,344][60425] Avg episode reward: [(0, '57.360'), (1, '55.750')] [2023-10-14 18:32:33,451][61585] Updated weights for policy 1, policy_version 22760 (0.0008) [2023-10-14 18:32:33,819][61585] Updated weights for policy 1, policy_version 22770 (0.0008) [2023-10-14 18:32:34,183][61585] Updated weights for policy 1, policy_version 22780 (0.0007) [2023-10-14 18:32:36,211][61552] Updated weights for policy 0, policy_version 22852 (0.0008) [2023-10-14 18:32:36,573][61552] Updated weights for policy 0, policy_version 22862 (0.0009) [2023-10-14 18:32:36,945][61552] Updated weights for policy 0, policy_version 22872 (0.0007) [2023-10-14 18:32:38,277][61585] Updated weights for policy 1, policy_version 22790 (0.0008) [2023-10-14 18:32:38,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 46759936. Throughput: 0: 1674.3, 1: 1664.6. Samples: 11698096. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-14 18:32:38,344][60425] Avg episode reward: [(0, '55.380'), (1, '57.580')] [2023-10-14 18:32:38,655][61585] Updated weights for policy 1, policy_version 22800 (0.0009) [2023-10-14 18:32:39,020][61585] Updated weights for policy 1, policy_version 22810 (0.0008) [2023-10-14 18:32:41,267][61552] Updated weights for policy 0, policy_version 22882 (0.0009) [2023-10-14 18:32:41,679][61552] Updated weights for policy 0, policy_version 22892 (0.0007) [2023-10-14 18:32:42,039][61552] Updated weights for policy 0, policy_version 22902 (0.0010) [2023-10-14 18:32:42,408][61552] Updated weights for policy 0, policy_version 22912 (0.0010) [2023-10-14 18:32:43,038][61585] Updated weights for policy 1, policy_version 22820 (0.0007) [2023-10-14 18:32:43,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 46825472. Throughput: 0: 1668.5, 1: 1665.6. Samples: 11717886. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-14 18:32:43,344][60425] Avg episode reward: [(0, '55.990'), (1, '56.050')] [2023-10-14 18:32:43,396][61585] Updated weights for policy 1, policy_version 22830 (0.0008) [2023-10-14 18:32:43,769][61585] Updated weights for policy 1, policy_version 22840 (0.0009) [2023-10-14 18:32:46,273][61552] Updated weights for policy 0, policy_version 22922 (0.0011) [2023-10-14 18:32:46,642][61552] Updated weights for policy 0, policy_version 22932 (0.0009) [2023-10-14 18:32:47,013][61552] Updated weights for policy 0, policy_version 22942 (0.0010) [2023-10-14 18:32:47,925][61585] Updated weights for policy 1, policy_version 22850 (0.0009) [2023-10-14 18:32:48,291][61585] Updated weights for policy 1, policy_version 22860 (0.0008) [2023-10-14 18:32:48,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 46891008. Throughput: 0: 1687.1, 1: 1667.2. Samples: 11728384. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-14 18:32:48,344][60425] Avg episode reward: [(0, '52.410'), (1, '55.740')] [2023-10-14 18:32:48,657][61585] Updated weights for policy 1, policy_version 22870 (0.0008) [2023-10-14 18:32:49,017][61585] Updated weights for policy 1, policy_version 22880 (0.0007) [2023-10-14 18:32:51,253][61552] Updated weights for policy 0, policy_version 22952 (0.0011) [2023-10-14 18:32:51,618][61552] Updated weights for policy 0, policy_version 22962 (0.0011) [2023-10-14 18:32:51,996][61552] Updated weights for policy 0, policy_version 22972 (0.0008) [2023-10-14 18:32:53,213][61585] Updated weights for policy 1, policy_version 22890 (0.0007) [2023-10-14 18:32:53,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 46956544. Throughput: 0: 1666.4, 1: 1660.3. Samples: 11747870. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-14 18:32:53,345][60425] Avg episode reward: [(0, '58.040'), (1, '58.360')] [2023-10-14 18:32:53,580][61585] Updated weights for policy 1, policy_version 22900 (0.0008) [2023-10-14 18:32:53,943][61585] Updated weights for policy 1, policy_version 22910 (0.0008) [2023-10-14 18:32:56,064][61552] Updated weights for policy 0, policy_version 22982 (0.0008) [2023-10-14 18:32:56,441][61552] Updated weights for policy 0, policy_version 22992 (0.0010) [2023-10-14 18:32:56,801][61552] Updated weights for policy 0, policy_version 23002 (0.0009) [2023-10-14 18:32:58,195][61585] Updated weights for policy 1, policy_version 22920 (0.0008) [2023-10-14 18:32:58,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 47022080. Throughput: 0: 1669.5, 1: 1656.7. Samples: 11767706. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-14 18:32:58,344][60425] Avg episode reward: [(0, '55.400'), (1, '56.500')] [2023-10-14 18:32:58,565][61585] Updated weights for policy 1, policy_version 22930 (0.0007) [2023-10-14 18:32:58,929][61585] Updated weights for policy 1, policy_version 22940 (0.0007) [2023-10-14 18:33:00,973][61552] Updated weights for policy 0, policy_version 23012 (0.0010) [2023-10-14 18:33:01,346][61552] Updated weights for policy 0, policy_version 23022 (0.0010) [2023-10-14 18:33:01,715][61552] Updated weights for policy 0, policy_version 23032 (0.0010) [2023-10-14 18:33:03,068][61585] Updated weights for policy 1, policy_version 22950 (0.0009) [2023-10-14 18:33:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 47087616. Throughput: 0: 1674.6, 1: 1660.3. Samples: 11777954. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-14 18:33:03,344][60425] Avg episode reward: [(0, '53.900'), (1, '57.250')] [2023-10-14 18:33:03,445][61585] Updated weights for policy 1, policy_version 22960 (0.0010) [2023-10-14 18:33:03,806][61585] Updated weights for policy 1, policy_version 22970 (0.0009) [2023-10-14 18:33:05,680][61552] Updated weights for policy 0, policy_version 23042 (0.0011) [2023-10-14 18:33:06,059][61552] Updated weights for policy 0, policy_version 23052 (0.0009) [2023-10-14 18:33:06,424][61552] Updated weights for policy 0, policy_version 23062 (0.0009) [2023-10-14 18:33:06,794][61552] Updated weights for policy 0, policy_version 23072 (0.0009) [2023-10-14 18:33:07,945][61585] Updated weights for policy 1, policy_version 22980 (0.0008) [2023-10-14 18:33:08,306][61585] Updated weights for policy 1, policy_version 22990 (0.0007) [2023-10-14 18:33:08,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 47153152. Throughput: 0: 1654.9, 1: 1664.2. Samples: 11797482. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:33:08,345][60425] Avg episode reward: [(0, '55.630'), (1, '57.420')] [2023-10-14 18:33:08,677][61585] Updated weights for policy 1, policy_version 23000 (0.0008) [2023-10-14 18:33:10,879][61552] Updated weights for policy 0, policy_version 23082 (0.0010) [2023-10-14 18:33:11,252][61552] Updated weights for policy 0, policy_version 23092 (0.0009) [2023-10-14 18:33:11,625][61552] Updated weights for policy 0, policy_version 23102 (0.0009) [2023-10-14 18:33:12,798][61585] Updated weights for policy 1, policy_version 23010 (0.0007) [2023-10-14 18:33:13,167][61585] Updated weights for policy 1, policy_version 23020 (0.0011) [2023-10-14 18:33:13,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 47218688. Throughput: 0: 1668.6, 1: 1660.2. Samples: 11817694. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:33:13,344][60425] Avg episode reward: [(0, '56.680'), (1, '53.240')] [2023-10-14 18:33:13,536][61585] Updated weights for policy 1, policy_version 23030 (0.0010) [2023-10-14 18:33:13,899][61585] Updated weights for policy 1, policy_version 23040 (0.0009) [2023-10-14 18:33:15,657][61552] Updated weights for policy 0, policy_version 23112 (0.0009) [2023-10-14 18:33:16,030][61552] Updated weights for policy 0, policy_version 23122 (0.0008) [2023-10-14 18:33:16,396][61552] Updated weights for policy 0, policy_version 23132 (0.0010) [2023-10-14 18:33:18,000][61585] Updated weights for policy 1, policy_version 23050 (0.0010) [2023-10-14 18:33:18,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 47284224. Throughput: 0: 1661.1, 1: 1662.2. Samples: 11827774. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:33:18,344][60425] Avg episode reward: [(0, '56.040'), (1, '57.260')] [2023-10-14 18:33:18,367][61585] Updated weights for policy 1, policy_version 23060 (0.0008) [2023-10-14 18:33:18,724][61585] Updated weights for policy 1, policy_version 23070 (0.0011) [2023-10-14 18:33:20,621][61552] Updated weights for policy 0, policy_version 23142 (0.0010) [2023-10-14 18:33:20,985][61552] Updated weights for policy 0, policy_version 23152 (0.0010) [2023-10-14 18:33:21,348][61552] Updated weights for policy 0, policy_version 23162 (0.0010) [2023-10-14 18:33:22,797][61585] Updated weights for policy 1, policy_version 23080 (0.0008) [2023-10-14 18:33:23,174][61585] Updated weights for policy 1, policy_version 23090 (0.0008) [2023-10-14 18:33:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 47349760. Throughput: 0: 1655.7, 1: 1662.5. Samples: 11847412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:33:23,344][60425] Avg episode reward: [(0, '60.280'), (1, '58.400')] [2023-10-14 18:33:23,540][61585] Updated weights for policy 1, policy_version 23100 (0.0009) [2023-10-14 18:33:25,476][61552] Updated weights for policy 0, policy_version 23172 (0.0008) [2023-10-14 18:33:25,859][61552] Updated weights for policy 0, policy_version 23182 (0.0009) [2023-10-14 18:33:26,226][61552] Updated weights for policy 0, policy_version 23192 (0.0007) [2023-10-14 18:33:27,811][61585] Updated weights for policy 1, policy_version 23110 (0.0010) [2023-10-14 18:33:28,178][61585] Updated weights for policy 1, policy_version 23120 (0.0010) [2023-10-14 18:33:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 47415296. Throughput: 0: 1672.4, 1: 1656.8. Samples: 11867696. Policy #0 lag: (min: 27.0, avg: 27.0, max: 29.0) [2023-10-14 18:33:28,344][60425] Avg episode reward: [(0, '58.150'), (1, '58.810')] [2023-10-14 18:33:28,541][61585] Updated weights for policy 1, policy_version 23130 (0.0010) [2023-10-14 18:33:30,219][61552] Updated weights for policy 0, policy_version 23202 (0.0009) [2023-10-14 18:33:30,605][61552] Updated weights for policy 0, policy_version 23212 (0.0007) [2023-10-14 18:33:30,971][61552] Updated weights for policy 0, policy_version 23222 (0.0007) [2023-10-14 18:33:31,347][61552] Updated weights for policy 0, policy_version 23232 (0.0010) [2023-10-14 18:33:32,522][61585] Updated weights for policy 1, policy_version 23140 (0.0009) [2023-10-14 18:33:32,882][61585] Updated weights for policy 1, policy_version 23150 (0.0008) [2023-10-14 18:33:33,252][61585] Updated weights for policy 1, policy_version 23160 (0.0007) [2023-10-14 18:33:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 47480832. Throughput: 0: 1656.3, 1: 1661.4. Samples: 11877678. Policy #0 lag: (min: 27.0, avg: 27.0, max: 29.0) [2023-10-14 18:33:33,344][60425] Avg episode reward: [(0, '55.510'), (1, '58.390')] [2023-10-14 18:33:35,271][61552] Updated weights for policy 0, policy_version 23242 (0.0010) [2023-10-14 18:33:35,641][61552] Updated weights for policy 0, policy_version 23252 (0.0011) [2023-10-14 18:33:36,026][61552] Updated weights for policy 0, policy_version 23262 (0.0009) [2023-10-14 18:33:37,298][61585] Updated weights for policy 1, policy_version 23170 (0.0008) [2023-10-14 18:33:37,671][61585] Updated weights for policy 1, policy_version 23180 (0.0008) [2023-10-14 18:33:38,033][61585] Updated weights for policy 1, policy_version 23190 (0.0007) [2023-10-14 18:33:38,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 47546368. Throughput: 0: 1666.0, 1: 1666.8. Samples: 11897844. Policy #0 lag: (min: 27.0, avg: 27.0, max: 29.0) [2023-10-14 18:33:38,344][60425] Avg episode reward: [(0, '60.710'), (1, '57.950')] [2023-10-14 18:33:38,404][61585] Updated weights for policy 1, policy_version 23200 (0.0010) [2023-10-14 18:33:40,288][61552] Updated weights for policy 0, policy_version 23272 (0.0011) [2023-10-14 18:33:40,653][61552] Updated weights for policy 0, policy_version 23282 (0.0008) [2023-10-14 18:33:41,019][61552] Updated weights for policy 0, policy_version 23292 (0.0011) [2023-10-14 18:33:42,481][61585] Updated weights for policy 1, policy_version 23210 (0.0008) [2023-10-14 18:33:42,856][61585] Updated weights for policy 1, policy_version 23220 (0.0007) [2023-10-14 18:33:43,228][61585] Updated weights for policy 1, policy_version 23230 (0.0007) [2023-10-14 18:33:43,344][60425] Fps is (10 sec: 16382.9, 60 sec: 13653.2, 300 sec: 13329.3). Total num frames: 47644672. Throughput: 0: 1674.3, 1: 1656.1. Samples: 11917576. Policy #0 lag: (min: 27.0, avg: 27.0, max: 29.0) [2023-10-14 18:33:43,345][60425] Avg episode reward: [(0, '56.690'), (1, '56.880')] [2023-10-14 18:33:43,356][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000023232_23789568.pth... [2023-10-14 18:33:43,357][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000023296_23855104.pth... [2023-10-14 18:33:43,386][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000021664_22183936.pth [2023-10-14 18:33:43,390][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000021728_22249472.pth [2023-10-14 18:33:43,391][61248] Saving a milestone ./train_atari/atari_roadrunner_APPO/checkpoint_p1/milestones/checkpoint_000023232_23789568.pth [2023-10-14 18:33:43,393][61172] Saving a milestone ./train_atari/atari_roadrunner_APPO/checkpoint_p0/milestones/checkpoint_000023296_23855104.pth [2023-10-14 18:33:45,192][61552] Updated weights for policy 0, policy_version 23302 (0.0011) [2023-10-14 18:33:45,575][61552] Updated weights for policy 0, policy_version 23312 (0.0010) [2023-10-14 18:33:45,954][61552] Updated weights for policy 0, policy_version 23322 (0.0008) [2023-10-14 18:33:47,247][61585] Updated weights for policy 1, policy_version 23240 (0.0008) [2023-10-14 18:33:47,623][61585] Updated weights for policy 1, policy_version 23250 (0.0008) [2023-10-14 18:33:47,991][61585] Updated weights for policy 1, policy_version 23260 (0.0009) [2023-10-14 18:33:48,343][60425] Fps is (10 sec: 16384.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 47710208. Throughput: 0: 1661.9, 1: 1672.0. Samples: 11927980. Policy #0 lag: (min: 6.0, avg: 10.2, max: 38.0) [2023-10-14 18:33:48,344][60425] Avg episode reward: [(0, '58.290'), (1, '57.820')] [2023-10-14 18:33:49,974][61552] Updated weights for policy 0, policy_version 23332 (0.0008) [2023-10-14 18:33:50,350][61552] Updated weights for policy 0, policy_version 23342 (0.0009) [2023-10-14 18:33:50,713][61552] Updated weights for policy 0, policy_version 23352 (0.0010) [2023-10-14 18:33:52,186][61585] Updated weights for policy 1, policy_version 23270 (0.0010) [2023-10-14 18:33:52,552][61585] Updated weights for policy 1, policy_version 23280 (0.0007) [2023-10-14 18:33:52,920][61585] Updated weights for policy 1, policy_version 23290 (0.0008) [2023-10-14 18:33:53,344][60425] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 47775744. Throughput: 0: 1676.0, 1: 1661.5. Samples: 11947670. Policy #0 lag: (min: 6.0, avg: 10.2, max: 38.0) [2023-10-14 18:33:53,345][60425] Avg episode reward: [(0, '55.700'), (1, '54.440')] [2023-10-14 18:33:54,843][61552] Updated weights for policy 0, policy_version 23362 (0.0010) [2023-10-14 18:33:55,216][61552] Updated weights for policy 0, policy_version 23372 (0.0009) [2023-10-14 18:33:55,585][61552] Updated weights for policy 0, policy_version 23382 (0.0009) [2023-10-14 18:33:55,948][61552] Updated weights for policy 0, policy_version 23392 (0.0008) [2023-10-14 18:33:57,179][61585] Updated weights for policy 1, policy_version 23300 (0.0009) [2023-10-14 18:33:57,542][61585] Updated weights for policy 1, policy_version 23310 (0.0009) [2023-10-14 18:33:57,925][61585] Updated weights for policy 1, policy_version 23320 (0.0008) [2023-10-14 18:33:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 47841280. Throughput: 0: 1685.5, 1: 1644.1. Samples: 11967528. Policy #0 lag: (min: 6.0, avg: 10.2, max: 38.0) [2023-10-14 18:33:58,344][60425] Avg episode reward: [(0, '54.530'), (1, '57.180')] [2023-10-14 18:33:59,915][61552] Updated weights for policy 0, policy_version 23402 (0.0009) [2023-10-14 18:34:00,287][61552] Updated weights for policy 0, policy_version 23412 (0.0010) [2023-10-14 18:34:00,663][61552] Updated weights for policy 0, policy_version 23422 (0.0008) [2023-10-14 18:34:02,060][61585] Updated weights for policy 1, policy_version 23330 (0.0008) [2023-10-14 18:34:02,433][61585] Updated weights for policy 1, policy_version 23340 (0.0007) [2023-10-14 18:34:02,794][61585] Updated weights for policy 1, policy_version 23350 (0.0007) [2023-10-14 18:34:03,155][61585] Updated weights for policy 1, policy_version 23360 (0.0007) [2023-10-14 18:34:03,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 47906816. Throughput: 0: 1665.4, 1: 1663.5. Samples: 11977574. Policy #0 lag: (min: 6.0, avg: 10.2, max: 38.0) [2023-10-14 18:34:03,344][60425] Avg episode reward: [(0, '54.930'), (1, '57.050')] [2023-10-14 18:34:04,700][61552] Updated weights for policy 0, policy_version 23432 (0.0009) [2023-10-14 18:34:05,073][61552] Updated weights for policy 0, policy_version 23442 (0.0011) [2023-10-14 18:34:05,441][61552] Updated weights for policy 0, policy_version 23452 (0.0008) [2023-10-14 18:34:07,185][61585] Updated weights for policy 1, policy_version 23370 (0.0009) [2023-10-14 18:34:07,546][61585] Updated weights for policy 1, policy_version 23380 (0.0010) [2023-10-14 18:34:07,913][61585] Updated weights for policy 1, policy_version 23390 (0.0011) [2023-10-14 18:34:08,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 47972352. Throughput: 0: 1683.9, 1: 1666.3. Samples: 11998170. Policy #0 lag: (min: 20.0, avg: 29.1, max: 52.0) [2023-10-14 18:34:08,344][60425] Avg episode reward: [(0, '58.690'), (1, '55.520')] [2023-10-14 18:34:09,602][61552] Updated weights for policy 0, policy_version 23462 (0.0007) [2023-10-14 18:34:09,974][61552] Updated weights for policy 0, policy_version 23472 (0.0007) [2023-10-14 18:34:10,334][61552] Updated weights for policy 0, policy_version 23482 (0.0009) [2023-10-14 18:34:12,249][61585] Updated weights for policy 1, policy_version 23400 (0.0008) [2023-10-14 18:34:12,611][61585] Updated weights for policy 1, policy_version 23410 (0.0007) [2023-10-14 18:34:12,968][61585] Updated weights for policy 1, policy_version 23420 (0.0007) [2023-10-14 18:34:13,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 48037888. Throughput: 0: 1687.4, 1: 1651.1. Samples: 12017926. Policy #0 lag: (min: 20.0, avg: 29.1, max: 52.0) [2023-10-14 18:34:13,344][60425] Avg episode reward: [(0, '56.120'), (1, '57.960')] [2023-10-14 18:34:14,357][61552] Updated weights for policy 0, policy_version 23492 (0.0010) [2023-10-14 18:34:14,724][61552] Updated weights for policy 0, policy_version 23502 (0.0008) [2023-10-14 18:34:15,082][61552] Updated weights for policy 0, policy_version 23512 (0.0007) [2023-10-14 18:34:16,984][61585] Updated weights for policy 1, policy_version 23430 (0.0010) [2023-10-14 18:34:17,350][61585] Updated weights for policy 1, policy_version 23440 (0.0008) [2023-10-14 18:34:17,716][61585] Updated weights for policy 1, policy_version 23450 (0.0011) [2023-10-14 18:34:18,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 48103424. Throughput: 0: 1672.7, 1: 1665.2. Samples: 12027880. Policy #0 lag: (min: 20.0, avg: 29.1, max: 52.0) [2023-10-14 18:34:18,344][60425] Avg episode reward: [(0, '54.700'), (1, '55.980')] [2023-10-14 18:34:19,238][61552] Updated weights for policy 0, policy_version 23522 (0.0009) [2023-10-14 18:34:19,607][61552] Updated weights for policy 0, policy_version 23532 (0.0010) [2023-10-14 18:34:19,980][61552] Updated weights for policy 0, policy_version 23542 (0.0011) [2023-10-14 18:34:20,345][61552] Updated weights for policy 0, policy_version 23552 (0.0007) [2023-10-14 18:34:21,803][61585] Updated weights for policy 1, policy_version 23460 (0.0009) [2023-10-14 18:34:22,167][61585] Updated weights for policy 1, policy_version 23470 (0.0008) [2023-10-14 18:34:22,539][61585] Updated weights for policy 1, policy_version 23480 (0.0008) [2023-10-14 18:34:23,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 48168960. Throughput: 0: 1684.8, 1: 1657.5. Samples: 12048246. Policy #0 lag: (min: 20.0, avg: 29.1, max: 52.0) [2023-10-14 18:34:23,344][60425] Avg episode reward: [(0, '58.560'), (1, '56.230')] [2023-10-14 18:34:24,363][61552] Updated weights for policy 0, policy_version 23562 (0.0008) [2023-10-14 18:34:24,735][61552] Updated weights for policy 0, policy_version 23572 (0.0009) [2023-10-14 18:34:25,106][61552] Updated weights for policy 0, policy_version 23582 (0.0008) [2023-10-14 18:34:26,693][61585] Updated weights for policy 1, policy_version 23490 (0.0009) [2023-10-14 18:34:27,061][61585] Updated weights for policy 1, policy_version 23500 (0.0009) [2023-10-14 18:34:27,440][61585] Updated weights for policy 1, policy_version 23510 (0.0010) [2023-10-14 18:34:27,802][61585] Updated weights for policy 1, policy_version 23520 (0.0011) [2023-10-14 18:34:28,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 48234496. Throughput: 0: 1693.2, 1: 1644.6. Samples: 12067774. Policy #0 lag: (min: 26.0, avg: 28.0, max: 50.0) [2023-10-14 18:34:28,345][60425] Avg episode reward: [(0, '55.150'), (1, '58.260')] [2023-10-14 18:34:29,022][61552] Updated weights for policy 0, policy_version 23592 (0.0007) [2023-10-14 18:34:29,397][61552] Updated weights for policy 0, policy_version 23602 (0.0007) [2023-10-14 18:34:29,764][61552] Updated weights for policy 0, policy_version 23612 (0.0007) [2023-10-14 18:34:31,898][61585] Updated weights for policy 1, policy_version 23530 (0.0010) [2023-10-14 18:34:32,278][61585] Updated weights for policy 1, policy_version 23540 (0.0011) [2023-10-14 18:34:32,644][61585] Updated weights for policy 1, policy_version 23550 (0.0009) [2023-10-14 18:34:33,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 48300032. Throughput: 0: 1679.7, 1: 1653.0. Samples: 12077950. Policy #0 lag: (min: 26.0, avg: 28.0, max: 50.0) [2023-10-14 18:34:33,344][60425] Avg episode reward: [(0, '59.640'), (1, '55.640')] [2023-10-14 18:34:33,842][61552] Updated weights for policy 0, policy_version 23622 (0.0009) [2023-10-14 18:34:34,204][61552] Updated weights for policy 0, policy_version 23632 (0.0010) [2023-10-14 18:34:34,572][61552] Updated weights for policy 0, policy_version 23642 (0.0010) [2023-10-14 18:34:36,859][61585] Updated weights for policy 1, policy_version 23560 (0.0009) [2023-10-14 18:34:37,234][61585] Updated weights for policy 1, policy_version 23570 (0.0008) [2023-10-14 18:34:37,595][61585] Updated weights for policy 1, policy_version 23580 (0.0008) [2023-10-14 18:34:38,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 48365568. Throughput: 0: 1689.8, 1: 1656.7. Samples: 12098264. Policy #0 lag: (min: 26.0, avg: 28.0, max: 50.0) [2023-10-14 18:34:38,344][60425] Avg episode reward: [(0, '55.830'), (1, '54.370')] [2023-10-14 18:34:38,545][61552] Updated weights for policy 0, policy_version 23652 (0.0009) [2023-10-14 18:34:38,909][61552] Updated weights for policy 0, policy_version 23662 (0.0007) [2023-10-14 18:34:39,288][61552] Updated weights for policy 0, policy_version 23672 (0.0010) [2023-10-14 18:34:41,776][61585] Updated weights for policy 1, policy_version 23590 (0.0008) [2023-10-14 18:34:42,137][61585] Updated weights for policy 1, policy_version 23600 (0.0009) [2023-10-14 18:34:42,510][61585] Updated weights for policy 1, policy_version 23610 (0.0010) [2023-10-14 18:34:43,338][61552] Updated weights for policy 0, policy_version 23682 (0.0010) [2023-10-14 18:34:43,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.3, 300 sec: 13329.3). Total num frames: 48431104. Throughput: 0: 1685.3, 1: 1653.3. Samples: 12117764. Policy #0 lag: (min: 26.0, avg: 28.0, max: 50.0) [2023-10-14 18:34:43,345][60425] Avg episode reward: [(0, '59.500'), (1, '55.510')] [2023-10-14 18:34:43,712][61552] Updated weights for policy 0, policy_version 23692 (0.0008) [2023-10-14 18:34:44,078][61552] Updated weights for policy 0, policy_version 23702 (0.0009) [2023-10-14 18:34:44,451][61552] Updated weights for policy 0, policy_version 23712 (0.0009) [2023-10-14 18:34:46,706][61585] Updated weights for policy 1, policy_version 23620 (0.0009) [2023-10-14 18:34:47,070][61585] Updated weights for policy 1, policy_version 23630 (0.0008) [2023-10-14 18:34:47,427][61585] Updated weights for policy 1, policy_version 23640 (0.0010) [2023-10-14 18:34:48,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 48496640. Throughput: 0: 1679.1, 1: 1658.9. Samples: 12127786. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-10-14 18:34:48,344][60425] Avg episode reward: [(0, '55.500'), (1, '56.740')] [2023-10-14 18:34:48,680][61552] Updated weights for policy 0, policy_version 23722 (0.0009) [2023-10-14 18:34:49,054][61552] Updated weights for policy 0, policy_version 23732 (0.0007) [2023-10-14 18:34:49,424][61552] Updated weights for policy 0, policy_version 23742 (0.0007) [2023-10-14 18:34:51,559][61585] Updated weights for policy 1, policy_version 23650 (0.0011) [2023-10-14 18:34:51,934][61585] Updated weights for policy 1, policy_version 23660 (0.0008) [2023-10-14 18:34:52,306][61585] Updated weights for policy 1, policy_version 23670 (0.0009) [2023-10-14 18:34:52,673][61585] Updated weights for policy 1, policy_version 23680 (0.0008) [2023-10-14 18:34:53,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 48562176. Throughput: 0: 1679.0, 1: 1651.3. Samples: 12148036. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-10-14 18:34:53,344][60425] Avg episode reward: [(0, '55.010'), (1, '56.640')] [2023-10-14 18:34:53,452][61552] Updated weights for policy 0, policy_version 23752 (0.0009) [2023-10-14 18:34:53,817][61552] Updated weights for policy 0, policy_version 23762 (0.0011) [2023-10-14 18:34:54,192][61552] Updated weights for policy 0, policy_version 23772 (0.0010) [2023-10-14 18:34:56,838][61585] Updated weights for policy 1, policy_version 23690 (0.0007) [2023-10-14 18:34:57,197][61585] Updated weights for policy 1, policy_version 23700 (0.0009) [2023-10-14 18:34:57,562][61585] Updated weights for policy 1, policy_version 23710 (0.0008) [2023-10-14 18:34:58,224][61552] Updated weights for policy 0, policy_version 23782 (0.0009) [2023-10-14 18:34:58,344][60425] Fps is (10 sec: 13106.6, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 48627712. Throughput: 0: 1682.7, 1: 1647.6. Samples: 12167790. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-10-14 18:34:58,345][60425] Avg episode reward: [(0, '57.640'), (1, '55.760')] [2023-10-14 18:34:58,595][61552] Updated weights for policy 0, policy_version 23792 (0.0009) [2023-10-14 18:34:58,972][61552] Updated weights for policy 0, policy_version 23802 (0.0008) [2023-10-14 18:35:01,568][61585] Updated weights for policy 1, policy_version 23720 (0.0008) [2023-10-14 18:35:01,929][61585] Updated weights for policy 1, policy_version 23730 (0.0009) [2023-10-14 18:35:02,300][61585] Updated weights for policy 1, policy_version 23740 (0.0008) [2023-10-14 18:35:03,017][61552] Updated weights for policy 0, policy_version 23812 (0.0007) [2023-10-14 18:35:03,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 48693248. Throughput: 0: 1682.4, 1: 1658.7. Samples: 12178226. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-10-14 18:35:03,344][60425] Avg episode reward: [(0, '53.110'), (1, '55.520')] [2023-10-14 18:35:03,399][61552] Updated weights for policy 0, policy_version 23822 (0.0008) [2023-10-14 18:35:03,771][61552] Updated weights for policy 0, policy_version 23832 (0.0007) [2023-10-14 18:35:06,471][61585] Updated weights for policy 1, policy_version 23750 (0.0009) [2023-10-14 18:35:06,834][61585] Updated weights for policy 1, policy_version 23760 (0.0010) [2023-10-14 18:35:07,196][61585] Updated weights for policy 1, policy_version 23770 (0.0009) [2023-10-14 18:35:07,957][61552] Updated weights for policy 0, policy_version 23842 (0.0011) [2023-10-14 18:35:08,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 48758784. Throughput: 0: 1683.3, 1: 1651.4. Samples: 12198310. Policy #0 lag: (min: 31.0, avg: 32.4, max: 56.0) [2023-10-14 18:35:08,344][60425] Avg episode reward: [(0, '52.620'), (1, '53.700')] [2023-10-14 18:35:08,362][61552] Updated weights for policy 0, policy_version 23852 (0.0010) [2023-10-14 18:35:08,728][61552] Updated weights for policy 0, policy_version 23862 (0.0008) [2023-10-14 18:35:09,103][61552] Updated weights for policy 0, policy_version 23872 (0.0009) [2023-10-14 18:35:11,244][61585] Updated weights for policy 1, policy_version 23780 (0.0008) [2023-10-14 18:35:11,608][61585] Updated weights for policy 1, policy_version 23790 (0.0008) [2023-10-14 18:35:11,979][61585] Updated weights for policy 1, policy_version 23800 (0.0008) [2023-10-14 18:35:13,167][61552] Updated weights for policy 0, policy_version 23882 (0.0007) [2023-10-14 18:35:13,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 48824320. Throughput: 0: 1676.7, 1: 1659.2. Samples: 12217890. Policy #0 lag: (min: 31.0, avg: 32.4, max: 56.0) [2023-10-14 18:35:13,344][60425] Avg episode reward: [(0, '51.590'), (1, '53.900')] [2023-10-14 18:35:13,537][61552] Updated weights for policy 0, policy_version 23892 (0.0008) [2023-10-14 18:35:13,908][61552] Updated weights for policy 0, policy_version 23902 (0.0008) [2023-10-14 18:35:16,261][61585] Updated weights for policy 1, policy_version 23810 (0.0009) [2023-10-14 18:35:16,628][61585] Updated weights for policy 1, policy_version 23820 (0.0010) [2023-10-14 18:35:17,000][61585] Updated weights for policy 1, policy_version 23830 (0.0010) [2023-10-14 18:35:17,361][61585] Updated weights for policy 1, policy_version 23840 (0.0007) [2023-10-14 18:35:17,924][61552] Updated weights for policy 0, policy_version 23912 (0.0010) [2023-10-14 18:35:18,285][61552] Updated weights for policy 0, policy_version 23922 (0.0009) [2023-10-14 18:35:18,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 48889856. Throughput: 0: 1679.9, 1: 1661.6. Samples: 12228316. Policy #0 lag: (min: 31.0, avg: 32.4, max: 56.0) [2023-10-14 18:35:18,344][60425] Avg episode reward: [(0, '58.020'), (1, '54.560')] [2023-10-14 18:35:18,660][61552] Updated weights for policy 0, policy_version 23932 (0.0007) [2023-10-14 18:35:21,617][61585] Updated weights for policy 1, policy_version 23850 (0.0009) [2023-10-14 18:35:21,979][61585] Updated weights for policy 1, policy_version 23860 (0.0007) [2023-10-14 18:35:22,352][61585] Updated weights for policy 1, policy_version 23870 (0.0007) [2023-10-14 18:35:22,720][61552] Updated weights for policy 0, policy_version 23942 (0.0010) [2023-10-14 18:35:23,092][61552] Updated weights for policy 0, policy_version 23952 (0.0010) [2023-10-14 18:35:23,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 48955392. Throughput: 0: 1681.3, 1: 1646.8. Samples: 12248030. Policy #0 lag: (min: 31.0, avg: 32.4, max: 56.0) [2023-10-14 18:35:23,344][60425] Avg episode reward: [(0, '53.450'), (1, '53.670')] [2023-10-14 18:35:23,458][61552] Updated weights for policy 0, policy_version 23962 (0.0008) [2023-10-14 18:35:26,470][61585] Updated weights for policy 1, policy_version 23880 (0.0007) [2023-10-14 18:35:26,836][61585] Updated weights for policy 1, policy_version 23890 (0.0007) [2023-10-14 18:35:27,204][61585] Updated weights for policy 1, policy_version 23900 (0.0009) [2023-10-14 18:35:27,572][61552] Updated weights for policy 0, policy_version 23972 (0.0009) [2023-10-14 18:35:27,925][61552] Updated weights for policy 0, policy_version 23982 (0.0009) [2023-10-14 18:35:28,289][61552] Updated weights for policy 0, policy_version 23992 (0.0008) [2023-10-14 18:35:28,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 49020928. Throughput: 0: 1677.0, 1: 1650.1. Samples: 12267484. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-14 18:35:28,344][60425] Avg episode reward: [(0, '56.470'), (1, '54.640')] [2023-10-14 18:35:31,169][61585] Updated weights for policy 1, policy_version 23910 (0.0010) [2023-10-14 18:35:31,535][61585] Updated weights for policy 1, policy_version 23920 (0.0011) [2023-10-14 18:35:31,900][61585] Updated weights for policy 1, policy_version 23930 (0.0008) [2023-10-14 18:35:32,305][61552] Updated weights for policy 0, policy_version 24002 (0.0009) [2023-10-14 18:35:32,672][61552] Updated weights for policy 0, policy_version 24012 (0.0007) [2023-10-14 18:35:33,050][61552] Updated weights for policy 0, policy_version 24022 (0.0008) [2023-10-14 18:35:33,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 49086464. Throughput: 0: 1684.8, 1: 1657.7. Samples: 12278200. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-14 18:35:33,344][60425] Avg episode reward: [(0, '56.930'), (1, '54.610')] [2023-10-14 18:35:33,413][61552] Updated weights for policy 0, policy_version 24032 (0.0009) [2023-10-14 18:35:36,049][61585] Updated weights for policy 1, policy_version 23940 (0.0009) [2023-10-14 18:35:36,409][61585] Updated weights for policy 1, policy_version 23950 (0.0008) [2023-10-14 18:35:36,773][61585] Updated weights for policy 1, policy_version 23960 (0.0009) [2023-10-14 18:35:37,441][61552] Updated weights for policy 0, policy_version 24042 (0.0009) [2023-10-14 18:35:37,808][61552] Updated weights for policy 0, policy_version 24052 (0.0011) [2023-10-14 18:35:38,177][61552] Updated weights for policy 0, policy_version 24062 (0.0009) [2023-10-14 18:35:38,343][60425] Fps is (10 sec: 16384.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 49184768. Throughput: 0: 1690.3, 1: 1644.8. Samples: 12298116. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-14 18:35:38,344][60425] Avg episode reward: [(0, '55.420'), (1, '56.750')] [2023-10-14 18:35:40,985][61585] Updated weights for policy 1, policy_version 23970 (0.0010) [2023-10-14 18:35:41,358][61585] Updated weights for policy 1, policy_version 23980 (0.0008) [2023-10-14 18:35:41,720][61585] Updated weights for policy 1, policy_version 23990 (0.0007) [2023-10-14 18:35:42,079][61585] Updated weights for policy 1, policy_version 24000 (0.0007) [2023-10-14 18:35:42,330][61552] Updated weights for policy 0, policy_version 24072 (0.0008) [2023-10-14 18:35:42,706][61552] Updated weights for policy 0, policy_version 24082 (0.0008) [2023-10-14 18:35:43,074][61552] Updated weights for policy 0, policy_version 24092 (0.0009) [2023-10-14 18:35:43,343][60425] Fps is (10 sec: 16383.7, 60 sec: 13653.4, 300 sec: 13440.5). Total num frames: 49250304. Throughput: 0: 1669.1, 1: 1660.3. Samples: 12317612. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:35:43,344][60425] Avg episode reward: [(0, '58.130'), (1, '60.570')] [2023-10-14 18:35:43,355][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000024096_24674304.pth... [2023-10-14 18:35:43,356][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000024000_24576000.pth... [2023-10-14 18:35:43,395][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000022528_23068672.pth [2023-10-14 18:35:43,401][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000022432_22970368.pth [2023-10-14 18:35:43,404][61248] Saving new best policy, reward=60.570! [2023-10-14 18:35:46,115][61585] Updated weights for policy 1, policy_version 24010 (0.0007) [2023-10-14 18:35:46,477][61585] Updated weights for policy 1, policy_version 24020 (0.0008) [2023-10-14 18:35:46,844][61585] Updated weights for policy 1, policy_version 24030 (0.0009) [2023-10-14 18:35:47,113][61552] Updated weights for policy 0, policy_version 24102 (0.0009) [2023-10-14 18:35:47,485][61552] Updated weights for policy 0, policy_version 24112 (0.0011) [2023-10-14 18:35:47,860][61552] Updated weights for policy 0, policy_version 24122 (0.0009) [2023-10-14 18:35:48,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 49315840. Throughput: 0: 1682.6, 1: 1658.5. Samples: 12328578. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:35:48,344][60425] Avg episode reward: [(0, '59.190'), (1, '59.240')] [2023-10-14 18:35:50,976][61585] Updated weights for policy 1, policy_version 24040 (0.0008) [2023-10-14 18:35:51,336][61585] Updated weights for policy 1, policy_version 24050 (0.0008) [2023-10-14 18:35:51,701][61585] Updated weights for policy 1, policy_version 24060 (0.0007) [2023-10-14 18:35:52,005][61552] Updated weights for policy 0, policy_version 24132 (0.0007) [2023-10-14 18:35:52,375][61552] Updated weights for policy 0, policy_version 24142 (0.0007) [2023-10-14 18:35:52,744][61552] Updated weights for policy 0, policy_version 24152 (0.0007) [2023-10-14 18:35:53,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 49381376. Throughput: 0: 1680.8, 1: 1647.3. Samples: 12348072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:35:53,344][60425] Avg episode reward: [(0, '56.840'), (1, '57.450')] [2023-10-14 18:35:55,940][61585] Updated weights for policy 1, policy_version 24070 (0.0007) [2023-10-14 18:35:56,303][61585] Updated weights for policy 1, policy_version 24080 (0.0008) [2023-10-14 18:35:56,665][61585] Updated weights for policy 1, policy_version 24090 (0.0009) [2023-10-14 18:35:56,851][61552] Updated weights for policy 0, policy_version 24162 (0.0007) [2023-10-14 18:35:57,254][61552] Updated weights for policy 0, policy_version 24172 (0.0008) [2023-10-14 18:35:57,620][61552] Updated weights for policy 0, policy_version 24182 (0.0009) [2023-10-14 18:35:57,986][61552] Updated weights for policy 0, policy_version 24192 (0.0008) [2023-10-14 18:35:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 49446912. Throughput: 0: 1665.4, 1: 1661.1. Samples: 12367580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:35:58,344][60425] Avg episode reward: [(0, '59.280'), (1, '57.460')] [2023-10-14 18:36:00,680][61585] Updated weights for policy 1, policy_version 24100 (0.0008) [2023-10-14 18:36:01,053][61585] Updated weights for policy 1, policy_version 24110 (0.0009) [2023-10-14 18:36:01,413][61585] Updated weights for policy 1, policy_version 24120 (0.0010) [2023-10-14 18:36:02,203][61552] Updated weights for policy 0, policy_version 24202 (0.0009) [2023-10-14 18:36:02,572][61552] Updated weights for policy 0, policy_version 24212 (0.0008) [2023-10-14 18:36:02,935][61552] Updated weights for policy 0, policy_version 24222 (0.0007) [2023-10-14 18:36:03,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 49512448. Throughput: 0: 1682.5, 1: 1657.2. Samples: 12378604. Policy #0 lag: (min: 0.0, avg: 18.4, max: 32.0) [2023-10-14 18:36:03,344][60425] Avg episode reward: [(0, '52.020'), (1, '55.710')] [2023-10-14 18:36:05,645][61585] Updated weights for policy 1, policy_version 24130 (0.0009) [2023-10-14 18:36:06,010][61585] Updated weights for policy 1, policy_version 24140 (0.0007) [2023-10-14 18:36:06,374][61585] Updated weights for policy 1, policy_version 24150 (0.0007) [2023-10-14 18:36:06,739][61585] Updated weights for policy 1, policy_version 24160 (0.0007) [2023-10-14 18:36:06,914][61552] Updated weights for policy 0, policy_version 24232 (0.0007) [2023-10-14 18:36:07,281][61552] Updated weights for policy 0, policy_version 24242 (0.0007) [2023-10-14 18:36:07,652][61552] Updated weights for policy 0, policy_version 24252 (0.0008) [2023-10-14 18:36:08,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 49577984. Throughput: 0: 1679.7, 1: 1656.2. Samples: 12398146. Policy #0 lag: (min: 0.0, avg: 18.4, max: 32.0) [2023-10-14 18:36:08,344][60425] Avg episode reward: [(0, '55.670'), (1, '55.460')] [2023-10-14 18:36:10,903][61585] Updated weights for policy 1, policy_version 24170 (0.0009) [2023-10-14 18:36:11,271][61585] Updated weights for policy 1, policy_version 24180 (0.0009) [2023-10-14 18:36:11,629][61585] Updated weights for policy 1, policy_version 24190 (0.0009) [2023-10-14 18:36:11,786][61552] Updated weights for policy 0, policy_version 24262 (0.0008) [2023-10-14 18:36:12,166][61552] Updated weights for policy 0, policy_version 24272 (0.0009) [2023-10-14 18:36:12,539][61552] Updated weights for policy 0, policy_version 24282 (0.0009) [2023-10-14 18:36:13,343][60425] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 49643520. Throughput: 0: 1658.6, 1: 1677.5. Samples: 12417608. Policy #0 lag: (min: 0.0, avg: 18.4, max: 32.0) [2023-10-14 18:36:13,344][60425] Avg episode reward: [(0, '50.580'), (1, '57.160')] [2023-10-14 18:36:15,618][61585] Updated weights for policy 1, policy_version 24200 (0.0009) [2023-10-14 18:36:15,974][61585] Updated weights for policy 1, policy_version 24210 (0.0008) [2023-10-14 18:36:16,338][61585] Updated weights for policy 1, policy_version 24220 (0.0009) [2023-10-14 18:36:16,649][61552] Updated weights for policy 0, policy_version 24292 (0.0008) [2023-10-14 18:36:17,021][61552] Updated weights for policy 0, policy_version 24302 (0.0008) [2023-10-14 18:36:17,388][61552] Updated weights for policy 0, policy_version 24312 (0.0007) [2023-10-14 18:36:18,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 49709056. Throughput: 0: 1678.5, 1: 1664.3. Samples: 12428628. Policy #0 lag: (min: 0.0, avg: 18.4, max: 32.0) [2023-10-14 18:36:18,344][60425] Avg episode reward: [(0, '57.410'), (1, '57.780')] [2023-10-14 18:36:20,397][61585] Updated weights for policy 1, policy_version 24230 (0.0007) [2023-10-14 18:36:20,763][61585] Updated weights for policy 1, policy_version 24240 (0.0007) [2023-10-14 18:36:21,131][61585] Updated weights for policy 1, policy_version 24250 (0.0009) [2023-10-14 18:36:21,335][61552] Updated weights for policy 0, policy_version 24322 (0.0008) [2023-10-14 18:36:21,708][61552] Updated weights for policy 0, policy_version 24332 (0.0007) [2023-10-14 18:36:22,081][61552] Updated weights for policy 0, policy_version 24342 (0.0009) [2023-10-14 18:36:22,450][61552] Updated weights for policy 0, policy_version 24352 (0.0009) [2023-10-14 18:36:23,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 49774592. Throughput: 0: 1667.2, 1: 1664.4. Samples: 12448038. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:36:23,344][60425] Avg episode reward: [(0, '52.290'), (1, '57.750')] [2023-10-14 18:36:25,138][61585] Updated weights for policy 1, policy_version 24260 (0.0008) [2023-10-14 18:36:25,495][61585] Updated weights for policy 1, policy_version 24270 (0.0007) [2023-10-14 18:36:25,862][61585] Updated weights for policy 1, policy_version 24280 (0.0008) [2023-10-14 18:36:26,486][61552] Updated weights for policy 0, policy_version 24362 (0.0009) [2023-10-14 18:36:26,853][61552] Updated weights for policy 0, policy_version 24372 (0.0010) [2023-10-14 18:36:27,230][61552] Updated weights for policy 0, policy_version 24382 (0.0009) [2023-10-14 18:36:28,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 49840128. Throughput: 0: 1666.5, 1: 1674.3. Samples: 12467946. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:36:28,344][60425] Avg episode reward: [(0, '53.860'), (1, '57.620')] [2023-10-14 18:36:30,006][61585] Updated weights for policy 1, policy_version 24290 (0.0009) [2023-10-14 18:36:30,377][61585] Updated weights for policy 1, policy_version 24300 (0.0009) [2023-10-14 18:36:30,747][61585] Updated weights for policy 1, policy_version 24310 (0.0009) [2023-10-14 18:36:31,104][61585] Updated weights for policy 1, policy_version 24320 (0.0009) [2023-10-14 18:36:31,236][61552] Updated weights for policy 0, policy_version 24392 (0.0007) [2023-10-14 18:36:31,613][61552] Updated weights for policy 0, policy_version 24402 (0.0009) [2023-10-14 18:36:31,979][61552] Updated weights for policy 0, policy_version 24412 (0.0008) [2023-10-14 18:36:33,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 49905664. Throughput: 0: 1687.1, 1: 1653.9. Samples: 12478920. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:36:33,344][60425] Avg episode reward: [(0, '52.710'), (1, '59.090')] [2023-10-14 18:36:35,205][61585] Updated weights for policy 1, policy_version 24330 (0.0010) [2023-10-14 18:36:35,565][61585] Updated weights for policy 1, policy_version 24340 (0.0009) [2023-10-14 18:36:35,934][61585] Updated weights for policy 1, policy_version 24350 (0.0009) [2023-10-14 18:36:36,043][61552] Updated weights for policy 0, policy_version 24422 (0.0008) [2023-10-14 18:36:36,413][61552] Updated weights for policy 0, policy_version 24432 (0.0008) [2023-10-14 18:36:36,774][61552] Updated weights for policy 0, policy_version 24442 (0.0008) [2023-10-14 18:36:38,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 49971200. Throughput: 0: 1666.3, 1: 1671.4. Samples: 12498268. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:36:38,344][60425] Avg episode reward: [(0, '53.770'), (1, '57.130')] [2023-10-14 18:36:40,202][61585] Updated weights for policy 1, policy_version 24360 (0.0011) [2023-10-14 18:36:40,568][61585] Updated weights for policy 1, policy_version 24370 (0.0010) [2023-10-14 18:36:40,874][61552] Updated weights for policy 0, policy_version 24452 (0.0007) [2023-10-14 18:36:40,943][61585] Updated weights for policy 1, policy_version 24380 (0.0009) [2023-10-14 18:36:41,239][61552] Updated weights for policy 0, policy_version 24462 (0.0008) [2023-10-14 18:36:41,618][61552] Updated weights for policy 0, policy_version 24472 (0.0009) [2023-10-14 18:36:43,344][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 50036736. Throughput: 0: 1673.8, 1: 1672.4. Samples: 12518162. Policy #0 lag: (min: 31.0, avg: 32.3, max: 56.0) [2023-10-14 18:36:43,345][60425] Avg episode reward: [(0, '55.660'), (1, '57.110')] [2023-10-14 18:36:45,127][61585] Updated weights for policy 1, policy_version 24390 (0.0008) [2023-10-14 18:36:45,492][61585] Updated weights for policy 1, policy_version 24400 (0.0007) [2023-10-14 18:36:45,695][61552] Updated weights for policy 0, policy_version 24482 (0.0008) [2023-10-14 18:36:45,856][61585] Updated weights for policy 1, policy_version 24410 (0.0007) [2023-10-14 18:36:46,111][61552] Updated weights for policy 0, policy_version 24492 (0.0009) [2023-10-14 18:36:46,482][61552] Updated weights for policy 0, policy_version 24502 (0.0008) [2023-10-14 18:36:46,848][61552] Updated weights for policy 0, policy_version 24512 (0.0007) [2023-10-14 18:36:48,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 50102272. Throughput: 0: 1680.8, 1: 1656.2. Samples: 12528770. Policy #0 lag: (min: 31.0, avg: 32.3, max: 56.0) [2023-10-14 18:36:48,344][60425] Avg episode reward: [(0, '53.440'), (1, '61.560')] [2023-10-14 18:36:48,346][61248] Saving new best policy, reward=61.560! [2023-10-14 18:36:50,170][61585] Updated weights for policy 1, policy_version 24420 (0.0008) [2023-10-14 18:36:50,538][61585] Updated weights for policy 1, policy_version 24430 (0.0007) [2023-10-14 18:36:50,731][61552] Updated weights for policy 0, policy_version 24522 (0.0008) [2023-10-14 18:36:50,899][61585] Updated weights for policy 1, policy_version 24440 (0.0007) [2023-10-14 18:36:51,109][61552] Updated weights for policy 0, policy_version 24532 (0.0009) [2023-10-14 18:36:51,481][61552] Updated weights for policy 0, policy_version 24542 (0.0008) [2023-10-14 18:36:53,343][60425] Fps is (10 sec: 13107.8, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 50167808. Throughput: 0: 1658.2, 1: 1668.5. Samples: 12547848. Policy #0 lag: (min: 31.0, avg: 32.3, max: 56.0) [2023-10-14 18:36:53,344][60425] Avg episode reward: [(0, '50.490'), (1, '57.440')] [2023-10-14 18:36:55,037][61585] Updated weights for policy 1, policy_version 24450 (0.0007) [2023-10-14 18:36:55,456][61585] Updated weights for policy 1, policy_version 24460 (0.0007) [2023-10-14 18:36:55,633][61552] Updated weights for policy 0, policy_version 24552 (0.0009) [2023-10-14 18:36:55,814][61585] Updated weights for policy 1, policy_version 24470 (0.0007) [2023-10-14 18:36:56,006][61552] Updated weights for policy 0, policy_version 24562 (0.0009) [2023-10-14 18:36:56,186][61585] Updated weights for policy 1, policy_version 24480 (0.0008) [2023-10-14 18:36:56,374][61552] Updated weights for policy 0, policy_version 24572 (0.0007) [2023-10-14 18:36:58,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 50233344. Throughput: 0: 1685.4, 1: 1658.9. Samples: 12568104. Policy #0 lag: (min: 31.0, avg: 32.3, max: 56.0) [2023-10-14 18:36:58,344][60425] Avg episode reward: [(0, '52.620'), (1, '59.320')] [2023-10-14 18:37:00,248][61585] Updated weights for policy 1, policy_version 24490 (0.0010) [2023-10-14 18:37:00,354][61552] Updated weights for policy 0, policy_version 24582 (0.0007) [2023-10-14 18:37:00,614][61585] Updated weights for policy 1, policy_version 24500 (0.0007) [2023-10-14 18:37:00,718][61552] Updated weights for policy 0, policy_version 24592 (0.0007) [2023-10-14 18:37:00,984][61585] Updated weights for policy 1, policy_version 24510 (0.0008) [2023-10-14 18:37:01,082][61552] Updated weights for policy 0, policy_version 24602 (0.0007) [2023-10-14 18:37:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 50298880. Throughput: 0: 1676.0, 1: 1648.0. Samples: 12578208. Policy #0 lag: (min: 19.0, avg: 25.8, max: 51.0) [2023-10-14 18:37:03,344][60425] Avg episode reward: [(0, '52.760'), (1, '57.290')] [2023-10-14 18:37:05,011][61585] Updated weights for policy 1, policy_version 24520 (0.0009) [2023-10-14 18:37:05,180][61552] Updated weights for policy 0, policy_version 24612 (0.0008) [2023-10-14 18:37:05,370][61585] Updated weights for policy 1, policy_version 24530 (0.0009) [2023-10-14 18:37:05,554][61552] Updated weights for policy 0, policy_version 24622 (0.0009) [2023-10-14 18:37:05,738][61585] Updated weights for policy 1, policy_version 24540 (0.0008) [2023-10-14 18:37:05,915][61552] Updated weights for policy 0, policy_version 24632 (0.0008) [2023-10-14 18:37:08,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 50364416. Throughput: 0: 1668.1, 1: 1657.6. Samples: 12597696. Policy #0 lag: (min: 19.0, avg: 25.8, max: 51.0) [2023-10-14 18:37:08,344][60425] Avg episode reward: [(0, '54.990'), (1, '58.320')] [2023-10-14 18:37:09,909][61585] Updated weights for policy 1, policy_version 24550 (0.0008) [2023-10-14 18:37:10,138][61552] Updated weights for policy 0, policy_version 24642 (0.0010) [2023-10-14 18:37:10,266][61585] Updated weights for policy 1, policy_version 24560 (0.0008) [2023-10-14 18:37:10,509][61552] Updated weights for policy 0, policy_version 24652 (0.0008) [2023-10-14 18:37:10,630][61585] Updated weights for policy 1, policy_version 24570 (0.0008) [2023-10-14 18:37:10,869][61552] Updated weights for policy 0, policy_version 24662 (0.0007) [2023-10-14 18:37:11,239][61552] Updated weights for policy 0, policy_version 24672 (0.0008) [2023-10-14 18:37:13,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 50429952. Throughput: 0: 1683.2, 1: 1652.5. Samples: 12618054. Policy #0 lag: (min: 19.0, avg: 25.8, max: 51.0) [2023-10-14 18:37:13,344][60425] Avg episode reward: [(0, '54.000'), (1, '56.480')] [2023-10-14 18:37:14,845][61585] Updated weights for policy 1, policy_version 24580 (0.0009) [2023-10-14 18:37:15,207][61585] Updated weights for policy 1, policy_version 24590 (0.0009) [2023-10-14 18:37:15,325][61552] Updated weights for policy 0, policy_version 24682 (0.0009) [2023-10-14 18:37:15,574][61585] Updated weights for policy 1, policy_version 24600 (0.0008) [2023-10-14 18:37:15,679][61552] Updated weights for policy 0, policy_version 24692 (0.0009) [2023-10-14 18:37:16,046][61552] Updated weights for policy 0, policy_version 24702 (0.0008) [2023-10-14 18:37:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 50495488. Throughput: 0: 1652.8, 1: 1647.4. Samples: 12627432. Policy #0 lag: (min: 19.0, avg: 25.8, max: 51.0) [2023-10-14 18:37:18,344][60425] Avg episode reward: [(0, '54.690'), (1, '56.790')] [2023-10-14 18:37:19,727][61585] Updated weights for policy 1, policy_version 24610 (0.0009) [2023-10-14 18:37:20,097][61585] Updated weights for policy 1, policy_version 24620 (0.0011) [2023-10-14 18:37:20,314][61552] Updated weights for policy 0, policy_version 24712 (0.0009) [2023-10-14 18:37:20,459][61585] Updated weights for policy 1, policy_version 24630 (0.0007) [2023-10-14 18:37:20,677][61552] Updated weights for policy 0, policy_version 24722 (0.0009) [2023-10-14 18:37:20,830][61585] Updated weights for policy 1, policy_version 24640 (0.0007) [2023-10-14 18:37:21,049][61552] Updated weights for policy 0, policy_version 24732 (0.0007) [2023-10-14 18:37:23,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 50561024. Throughput: 0: 1661.5, 1: 1645.8. Samples: 12647094. Policy #0 lag: (min: 3.0, avg: 3.4, max: 17.0) [2023-10-14 18:37:23,344][60425] Avg episode reward: [(0, '53.720'), (1, '57.710')] [2023-10-14 18:37:24,930][61585] Updated weights for policy 1, policy_version 24650 (0.0008) [2023-10-14 18:37:25,074][61552] Updated weights for policy 0, policy_version 24742 (0.0007) [2023-10-14 18:37:25,299][61585] Updated weights for policy 1, policy_version 24660 (0.0009) [2023-10-14 18:37:25,447][61552] Updated weights for policy 0, policy_version 24752 (0.0008) [2023-10-14 18:37:25,655][61585] Updated weights for policy 1, policy_version 24670 (0.0010) [2023-10-14 18:37:25,809][61552] Updated weights for policy 0, policy_version 24762 (0.0008) [2023-10-14 18:37:28,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 50626560. Throughput: 0: 1676.1, 1: 1648.3. Samples: 12667762. Policy #0 lag: (min: 3.0, avg: 3.4, max: 17.0) [2023-10-14 18:37:28,344][60425] Avg episode reward: [(0, '51.980'), (1, '57.930')] [2023-10-14 18:37:29,744][61585] Updated weights for policy 1, policy_version 24680 (0.0008) [2023-10-14 18:37:29,989][61552] Updated weights for policy 0, policy_version 24772 (0.0008) [2023-10-14 18:37:30,104][61585] Updated weights for policy 1, policy_version 24690 (0.0009) [2023-10-14 18:37:30,385][61552] Updated weights for policy 0, policy_version 24782 (0.0007) [2023-10-14 18:37:30,468][61585] Updated weights for policy 1, policy_version 24700 (0.0009) [2023-10-14 18:37:30,747][61552] Updated weights for policy 0, policy_version 24792 (0.0009) [2023-10-14 18:37:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 50692096. Throughput: 0: 1655.6, 1: 1641.7. Samples: 12677148. Policy #0 lag: (min: 3.0, avg: 3.4, max: 17.0) [2023-10-14 18:37:33,344][60425] Avg episode reward: [(0, '54.200'), (1, '60.450')] [2023-10-14 18:37:34,527][61585] Updated weights for policy 1, policy_version 24710 (0.0010) [2023-10-14 18:37:34,839][61552] Updated weights for policy 0, policy_version 24802 (0.0009) [2023-10-14 18:37:34,892][61585] Updated weights for policy 1, policy_version 24720 (0.0008) [2023-10-14 18:37:35,204][61552] Updated weights for policy 0, policy_version 24812 (0.0008) [2023-10-14 18:37:35,256][61585] Updated weights for policy 1, policy_version 24730 (0.0008) [2023-10-14 18:37:35,576][61552] Updated weights for policy 0, policy_version 24822 (0.0009) [2023-10-14 18:37:35,939][61552] Updated weights for policy 0, policy_version 24832 (0.0010) [2023-10-14 18:37:38,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 50757632. Throughput: 0: 1666.5, 1: 1651.4. Samples: 12697154. Policy #0 lag: (min: 3.0, avg: 3.4, max: 17.0) [2023-10-14 18:37:38,344][60425] Avg episode reward: [(0, '55.190'), (1, '56.730')] [2023-10-14 18:37:39,383][61585] Updated weights for policy 1, policy_version 24740 (0.0009) [2023-10-14 18:37:39,744][61585] Updated weights for policy 1, policy_version 24750 (0.0008) [2023-10-14 18:37:39,948][61552] Updated weights for policy 0, policy_version 24842 (0.0008) [2023-10-14 18:37:40,098][61585] Updated weights for policy 1, policy_version 24760 (0.0007) [2023-10-14 18:37:40,317][61552] Updated weights for policy 0, policy_version 24852 (0.0009) [2023-10-14 18:37:40,678][61552] Updated weights for policy 0, policy_version 24862 (0.0008) [2023-10-14 18:37:43,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 50823168. Throughput: 0: 1674.8, 1: 1660.2. Samples: 12718176. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:37:43,344][60425] Avg episode reward: [(0, '61.520'), (1, '58.950')] [2023-10-14 18:37:43,353][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000024864_25460736.pth... [2023-10-14 18:37:43,353][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000024768_25362432.pth... [2023-10-14 18:37:43,385][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000023232_23789568.pth [2023-10-14 18:37:43,393][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000023296_23855104.pth [2023-10-14 18:37:43,397][61172] Saving new best policy, reward=61.520! [2023-10-14 18:37:44,189][61585] Updated weights for policy 1, policy_version 24770 (0.0009) [2023-10-14 18:37:44,593][61552] Updated weights for policy 0, policy_version 24872 (0.0009) [2023-10-14 18:37:44,608][61585] Updated weights for policy 1, policy_version 24780 (0.0009) [2023-10-14 18:37:44,957][61552] Updated weights for policy 0, policy_version 24882 (0.0008) [2023-10-14 18:37:44,984][61585] Updated weights for policy 1, policy_version 24790 (0.0007) [2023-10-14 18:37:45,333][61552] Updated weights for policy 0, policy_version 24892 (0.0007) [2023-10-14 18:37:45,350][61585] Updated weights for policy 1, policy_version 24800 (0.0007) [2023-10-14 18:37:48,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 50888704. Throughput: 0: 1656.9, 1: 1652.0. Samples: 12727108. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:37:48,344][60425] Avg episode reward: [(0, '55.130'), (1, '55.270')] [2023-10-14 18:37:49,473][61552] Updated weights for policy 0, policy_version 24902 (0.0008) [2023-10-14 18:37:49,629][61585] Updated weights for policy 1, policy_version 24810 (0.0007) [2023-10-14 18:37:49,830][61552] Updated weights for policy 0, policy_version 24912 (0.0008) [2023-10-14 18:37:49,996][61585] Updated weights for policy 1, policy_version 24820 (0.0009) [2023-10-14 18:37:50,198][61552] Updated weights for policy 0, policy_version 24922 (0.0009) [2023-10-14 18:37:50,359][61585] Updated weights for policy 1, policy_version 24830 (0.0008) [2023-10-14 18:37:53,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 50954240. Throughput: 0: 1669.0, 1: 1652.4. Samples: 12747162. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:37:53,344][60425] Avg episode reward: [(0, '55.090'), (1, '56.220')] [2023-10-14 18:37:54,359][61552] Updated weights for policy 0, policy_version 24932 (0.0008) [2023-10-14 18:37:54,679][61585] Updated weights for policy 1, policy_version 24840 (0.0008) [2023-10-14 18:37:54,724][61552] Updated weights for policy 0, policy_version 24942 (0.0008) [2023-10-14 18:37:55,048][61585] Updated weights for policy 1, policy_version 24850 (0.0007) [2023-10-14 18:37:55,083][61552] Updated weights for policy 0, policy_version 24952 (0.0009) [2023-10-14 18:37:55,417][61585] Updated weights for policy 1, policy_version 24860 (0.0009) [2023-10-14 18:37:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 51019776. Throughput: 0: 1673.6, 1: 1652.4. Samples: 12767724. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:37:58,344][60425] Avg episode reward: [(0, '59.360'), (1, '53.330')] [2023-10-14 18:37:59,278][61552] Updated weights for policy 0, policy_version 24962 (0.0009) [2023-10-14 18:37:59,562][61585] Updated weights for policy 1, policy_version 24870 (0.0008) [2023-10-14 18:37:59,633][61552] Updated weights for policy 0, policy_version 24972 (0.0008) [2023-10-14 18:37:59,929][61585] Updated weights for policy 1, policy_version 24880 (0.0009) [2023-10-14 18:38:00,003][61552] Updated weights for policy 0, policy_version 24982 (0.0009) [2023-10-14 18:38:00,286][61585] Updated weights for policy 1, policy_version 24890 (0.0007) [2023-10-14 18:38:00,369][61552] Updated weights for policy 0, policy_version 24992 (0.0008) [2023-10-14 18:38:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 51085312. Throughput: 0: 1666.9, 1: 1651.0. Samples: 12776738. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-14 18:38:03,344][60425] Avg episode reward: [(0, '61.340'), (1, '54.310')] [2023-10-14 18:38:04,438][61585] Updated weights for policy 1, policy_version 24900 (0.0010) [2023-10-14 18:38:04,490][61552] Updated weights for policy 0, policy_version 25002 (0.0007) [2023-10-14 18:38:04,795][61585] Updated weights for policy 1, policy_version 24910 (0.0008) [2023-10-14 18:38:04,858][61552] Updated weights for policy 0, policy_version 25012 (0.0009) [2023-10-14 18:38:05,163][61585] Updated weights for policy 1, policy_version 24920 (0.0008) [2023-10-14 18:38:05,225][61552] Updated weights for policy 0, policy_version 25022 (0.0007) [2023-10-14 18:38:08,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 51150848. Throughput: 0: 1679.6, 1: 1655.0. Samples: 12797150. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-14 18:38:08,344][60425] Avg episode reward: [(0, '57.760'), (1, '54.650')] [2023-10-14 18:38:09,363][61585] Updated weights for policy 1, policy_version 24930 (0.0007) [2023-10-14 18:38:09,400][61552] Updated weights for policy 0, policy_version 25032 (0.0008) [2023-10-14 18:38:09,728][61585] Updated weights for policy 1, policy_version 24940 (0.0010) [2023-10-14 18:38:09,761][61552] Updated weights for policy 0, policy_version 25042 (0.0007) [2023-10-14 18:38:10,099][61585] Updated weights for policy 1, policy_version 24950 (0.0008) [2023-10-14 18:38:10,133][61552] Updated weights for policy 0, policy_version 25052 (0.0008) [2023-10-14 18:38:10,462][61585] Updated weights for policy 1, policy_version 24960 (0.0007) [2023-10-14 18:38:13,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 51216384. Throughput: 0: 1678.4, 1: 1652.7. Samples: 12817664. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-14 18:38:13,344][60425] Avg episode reward: [(0, '57.250'), (1, '55.330')] [2023-10-14 18:38:14,174][61552] Updated weights for policy 0, policy_version 25062 (0.0008) [2023-10-14 18:38:14,541][61552] Updated weights for policy 0, policy_version 25072 (0.0009) [2023-10-14 18:38:14,542][61585] Updated weights for policy 1, policy_version 24970 (0.0007) [2023-10-14 18:38:14,902][61585] Updated weights for policy 1, policy_version 24980 (0.0007) [2023-10-14 18:38:14,914][61552] Updated weights for policy 0, policy_version 25082 (0.0007) [2023-10-14 18:38:15,259][61585] Updated weights for policy 1, policy_version 24990 (0.0008) [2023-10-14 18:38:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 51281920. Throughput: 0: 1670.3, 1: 1654.9. Samples: 12826782. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-14 18:38:18,344][60425] Avg episode reward: [(0, '59.120'), (1, '56.240')] [2023-10-14 18:38:19,002][61552] Updated weights for policy 0, policy_version 25092 (0.0008) [2023-10-14 18:38:19,379][61552] Updated weights for policy 0, policy_version 25102 (0.0009) [2023-10-14 18:38:19,470][61585] Updated weights for policy 1, policy_version 25000 (0.0009) [2023-10-14 18:38:19,749][61552] Updated weights for policy 0, policy_version 25112 (0.0008) [2023-10-14 18:38:19,827][61585] Updated weights for policy 1, policy_version 25010 (0.0008) [2023-10-14 18:38:20,189][61585] Updated weights for policy 1, policy_version 25020 (0.0008) [2023-10-14 18:38:23,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 51347456. Throughput: 0: 1681.9, 1: 1652.1. Samples: 12847186. Policy #0 lag: (min: 10.0, avg: 14.7, max: 42.0) [2023-10-14 18:38:23,345][60425] Avg episode reward: [(0, '57.900'), (1, '59.950')] [2023-10-14 18:38:23,875][61552] Updated weights for policy 0, policy_version 25122 (0.0008) [2023-10-14 18:38:24,248][61552] Updated weights for policy 0, policy_version 25132 (0.0010) [2023-10-14 18:38:24,323][61585] Updated weights for policy 1, policy_version 25030 (0.0010) [2023-10-14 18:38:24,613][61552] Updated weights for policy 0, policy_version 25142 (0.0009) [2023-10-14 18:38:24,697][61585] Updated weights for policy 1, policy_version 25040 (0.0008) [2023-10-14 18:38:24,976][61552] Updated weights for policy 0, policy_version 25152 (0.0008) [2023-10-14 18:38:25,060][61585] Updated weights for policy 1, policy_version 25050 (0.0009) [2023-10-14 18:38:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 51412992. Throughput: 0: 1673.6, 1: 1648.5. Samples: 12867672. Policy #0 lag: (min: 10.0, avg: 14.7, max: 42.0) [2023-10-14 18:38:28,344][60425] Avg episode reward: [(0, '56.160'), (1, '58.250')] [2023-10-14 18:38:29,079][61552] Updated weights for policy 0, policy_version 25162 (0.0007) [2023-10-14 18:38:29,249][61585] Updated weights for policy 1, policy_version 25060 (0.0008) [2023-10-14 18:38:29,451][61552] Updated weights for policy 0, policy_version 25172 (0.0007) [2023-10-14 18:38:29,632][61585] Updated weights for policy 1, policy_version 25070 (0.0008) [2023-10-14 18:38:29,820][61552] Updated weights for policy 0, policy_version 25182 (0.0010) [2023-10-14 18:38:29,995][61585] Updated weights for policy 1, policy_version 25080 (0.0009) [2023-10-14 18:38:33,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 51478528. Throughput: 0: 1675.7, 1: 1647.5. Samples: 12876650. Policy #0 lag: (min: 10.0, avg: 14.7, max: 42.0) [2023-10-14 18:38:33,344][60425] Avg episode reward: [(0, '56.510'), (1, '54.950')] [2023-10-14 18:38:33,830][61552] Updated weights for policy 0, policy_version 25192 (0.0010) [2023-10-14 18:38:34,201][61552] Updated weights for policy 0, policy_version 25202 (0.0009) [2023-10-14 18:38:34,217][61585] Updated weights for policy 1, policy_version 25090 (0.0008) [2023-10-14 18:38:34,564][61552] Updated weights for policy 0, policy_version 25212 (0.0009) [2023-10-14 18:38:34,587][61585] Updated weights for policy 1, policy_version 25100 (0.0008) [2023-10-14 18:38:34,956][61585] Updated weights for policy 1, policy_version 25110 (0.0010) [2023-10-14 18:38:35,322][61585] Updated weights for policy 1, policy_version 25120 (0.0011) [2023-10-14 18:38:38,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 51544064. Throughput: 0: 1682.6, 1: 1650.5. Samples: 12897152. Policy #0 lag: (min: 10.0, avg: 14.7, max: 42.0) [2023-10-14 18:38:38,344][60425] Avg episode reward: [(0, '57.460'), (1, '60.020')] [2023-10-14 18:38:38,550][61552] Updated weights for policy 0, policy_version 25222 (0.0009) [2023-10-14 18:38:38,926][61552] Updated weights for policy 0, policy_version 25232 (0.0011) [2023-10-14 18:38:39,290][61552] Updated weights for policy 0, policy_version 25242 (0.0009) [2023-10-14 18:38:39,431][61585] Updated weights for policy 1, policy_version 25130 (0.0009) [2023-10-14 18:38:39,790][61585] Updated weights for policy 1, policy_version 25140 (0.0010) [2023-10-14 18:38:40,158][61585] Updated weights for policy 1, policy_version 25150 (0.0009) [2023-10-14 18:38:43,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 51609600. Throughput: 0: 1677.4, 1: 1654.4. Samples: 12917654. Policy #0 lag: (min: 25.0, avg: 33.0, max: 57.0) [2023-10-14 18:38:43,344][60425] Avg episode reward: [(0, '55.470'), (1, '55.700')] [2023-10-14 18:38:43,420][61552] Updated weights for policy 0, policy_version 25252 (0.0008) [2023-10-14 18:38:43,795][61552] Updated weights for policy 0, policy_version 25262 (0.0008) [2023-10-14 18:38:44,171][61552] Updated weights for policy 0, policy_version 25272 (0.0009) [2023-10-14 18:38:44,299][61585] Updated weights for policy 1, policy_version 25160 (0.0007) [2023-10-14 18:38:44,666][61585] Updated weights for policy 1, policy_version 25170 (0.0010) [2023-10-14 18:38:45,029][61585] Updated weights for policy 1, policy_version 25180 (0.0010) [2023-10-14 18:38:48,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 51675136. Throughput: 0: 1678.3, 1: 1651.9. Samples: 12926594. Policy #0 lag: (min: 25.0, avg: 33.0, max: 57.0) [2023-10-14 18:38:48,344][60425] Avg episode reward: [(0, '56.140'), (1, '57.020')] [2023-10-14 18:38:48,357][61552] Updated weights for policy 0, policy_version 25282 (0.0009) [2023-10-14 18:38:48,734][61552] Updated weights for policy 0, policy_version 25292 (0.0008) [2023-10-14 18:38:49,099][61552] Updated weights for policy 0, policy_version 25302 (0.0008) [2023-10-14 18:38:49,276][61585] Updated weights for policy 1, policy_version 25190 (0.0009) [2023-10-14 18:38:49,462][61552] Updated weights for policy 0, policy_version 25312 (0.0008) [2023-10-14 18:38:49,640][61585] Updated weights for policy 1, policy_version 25200 (0.0008) [2023-10-14 18:38:50,011][61585] Updated weights for policy 1, policy_version 25210 (0.0008) [2023-10-14 18:38:53,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 51740672. Throughput: 0: 1676.8, 1: 1650.2. Samples: 12946864. Policy #0 lag: (min: 25.0, avg: 33.0, max: 57.0) [2023-10-14 18:38:53,344][60425] Avg episode reward: [(0, '57.950'), (1, '58.580')] [2023-10-14 18:38:53,346][61552] Updated weights for policy 0, policy_version 25322 (0.0007) [2023-10-14 18:38:53,715][61552] Updated weights for policy 0, policy_version 25332 (0.0007) [2023-10-14 18:38:54,087][61552] Updated weights for policy 0, policy_version 25342 (0.0008) [2023-10-14 18:38:54,314][61585] Updated weights for policy 1, policy_version 25220 (0.0010) [2023-10-14 18:38:54,680][61585] Updated weights for policy 1, policy_version 25230 (0.0009) [2023-10-14 18:38:55,046][61585] Updated weights for policy 1, policy_version 25240 (0.0010) [2023-10-14 18:38:58,005][61552] Updated weights for policy 0, policy_version 25352 (0.0009) [2023-10-14 18:38:58,344][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 51806208. Throughput: 0: 1678.7, 1: 1649.2. Samples: 12967422. Policy #0 lag: (min: 25.0, avg: 33.0, max: 57.0) [2023-10-14 18:38:58,345][60425] Avg episode reward: [(0, '55.260'), (1, '58.360')] [2023-10-14 18:38:58,384][61552] Updated weights for policy 0, policy_version 25362 (0.0010) [2023-10-14 18:38:58,764][61552] Updated weights for policy 0, policy_version 25372 (0.0008) [2023-10-14 18:38:59,160][61585] Updated weights for policy 1, policy_version 25250 (0.0010) [2023-10-14 18:38:59,521][61585] Updated weights for policy 1, policy_version 25260 (0.0009) [2023-10-14 18:38:59,896][61585] Updated weights for policy 1, policy_version 25270 (0.0010) [2023-10-14 18:39:00,258][61585] Updated weights for policy 1, policy_version 25280 (0.0010) [2023-10-14 18:39:03,041][61552] Updated weights for policy 0, policy_version 25382 (0.0009) [2023-10-14 18:39:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 51871744. Throughput: 0: 1675.8, 1: 1647.6. Samples: 12976336. Policy #0 lag: (min: 18.0, avg: 20.0, max: 40.0) [2023-10-14 18:39:03,344][60425] Avg episode reward: [(0, '55.430'), (1, '58.320')] [2023-10-14 18:39:03,410][61552] Updated weights for policy 0, policy_version 25392 (0.0010) [2023-10-14 18:39:03,779][61552] Updated weights for policy 0, policy_version 25402 (0.0010) [2023-10-14 18:39:04,113][61585] Updated weights for policy 1, policy_version 25290 (0.0010) [2023-10-14 18:39:04,482][61585] Updated weights for policy 1, policy_version 25300 (0.0008) [2023-10-14 18:39:04,844][61585] Updated weights for policy 1, policy_version 25310 (0.0008) [2023-10-14 18:39:07,926][61552] Updated weights for policy 0, policy_version 25412 (0.0009) [2023-10-14 18:39:08,322][61552] Updated weights for policy 0, policy_version 25422 (0.0007) [2023-10-14 18:39:08,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 51937280. Throughput: 0: 1677.9, 1: 1647.3. Samples: 12996816. Policy #0 lag: (min: 18.0, avg: 20.0, max: 40.0) [2023-10-14 18:39:08,344][60425] Avg episode reward: [(0, '55.610'), (1, '60.350')] [2023-10-14 18:39:08,686][61552] Updated weights for policy 0, policy_version 25432 (0.0009) [2023-10-14 18:39:09,112][61585] Updated weights for policy 1, policy_version 25320 (0.0008) [2023-10-14 18:39:09,482][61585] Updated weights for policy 1, policy_version 25330 (0.0007) [2023-10-14 18:39:09,838][61585] Updated weights for policy 1, policy_version 25340 (0.0009) [2023-10-14 18:39:12,805][61552] Updated weights for policy 0, policy_version 25442 (0.0010) [2023-10-14 18:39:13,179][61552] Updated weights for policy 0, policy_version 25452 (0.0008) [2023-10-14 18:39:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 52002816. Throughput: 0: 1673.0, 1: 1651.1. Samples: 13017258. Policy #0 lag: (min: 18.0, avg: 20.0, max: 40.0) [2023-10-14 18:39:13,344][60425] Avg episode reward: [(0, '57.220'), (1, '56.770')] [2023-10-14 18:39:13,542][61552] Updated weights for policy 0, policy_version 25462 (0.0009) [2023-10-14 18:39:13,789][61585] Updated weights for policy 1, policy_version 25350 (0.0010) [2023-10-14 18:39:13,910][61552] Updated weights for policy 0, policy_version 25472 (0.0008) [2023-10-14 18:39:14,159][61585] Updated weights for policy 1, policy_version 25360 (0.0008) [2023-10-14 18:39:14,534][61585] Updated weights for policy 1, policy_version 25370 (0.0007) [2023-10-14 18:39:17,962][61552] Updated weights for policy 0, policy_version 25482 (0.0007) [2023-10-14 18:39:18,332][61552] Updated weights for policy 0, policy_version 25492 (0.0008) [2023-10-14 18:39:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 52068352. Throughput: 0: 1673.9, 1: 1653.5. Samples: 13026382. Policy #0 lag: (min: 18.0, avg: 20.0, max: 40.0) [2023-10-14 18:39:18,344][60425] Avg episode reward: [(0, '54.990'), (1, '57.510')] [2023-10-14 18:39:18,691][61552] Updated weights for policy 0, policy_version 25502 (0.0007) [2023-10-14 18:39:18,770][61585] Updated weights for policy 1, policy_version 25380 (0.0009) [2023-10-14 18:39:19,145][61585] Updated weights for policy 1, policy_version 25390 (0.0010) [2023-10-14 18:39:19,502][61585] Updated weights for policy 1, policy_version 25400 (0.0007) [2023-10-14 18:39:22,667][61552] Updated weights for policy 0, policy_version 25512 (0.0009) [2023-10-14 18:39:23,034][61552] Updated weights for policy 0, policy_version 25522 (0.0010) [2023-10-14 18:39:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 52133888. Throughput: 0: 1670.5, 1: 1658.2. Samples: 13046940. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 18:39:23,344][60425] Avg episode reward: [(0, '56.710'), (1, '55.380')] [2023-10-14 18:39:23,411][61552] Updated weights for policy 0, policy_version 25532 (0.0010) [2023-10-14 18:39:23,567][61585] Updated weights for policy 1, policy_version 25410 (0.0007) [2023-10-14 18:39:23,939][61585] Updated weights for policy 1, policy_version 25420 (0.0009) [2023-10-14 18:39:24,298][61585] Updated weights for policy 1, policy_version 25430 (0.0008) [2023-10-14 18:39:24,661][61585] Updated weights for policy 1, policy_version 25440 (0.0007) [2023-10-14 18:39:27,633][61552] Updated weights for policy 0, policy_version 25542 (0.0008) [2023-10-14 18:39:28,007][61552] Updated weights for policy 0, policy_version 25552 (0.0009) [2023-10-14 18:39:28,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 52199424. Throughput: 0: 1667.3, 1: 1659.8. Samples: 13067374. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 18:39:28,344][60425] Avg episode reward: [(0, '54.020'), (1, '56.860')] [2023-10-14 18:39:28,381][61552] Updated weights for policy 0, policy_version 25562 (0.0009) [2023-10-14 18:39:28,843][61585] Updated weights for policy 1, policy_version 25450 (0.0008) [2023-10-14 18:39:29,206][61585] Updated weights for policy 1, policy_version 25460 (0.0011) [2023-10-14 18:39:29,572][61585] Updated weights for policy 1, policy_version 25470 (0.0010) [2023-10-14 18:39:32,197][61552] Updated weights for policy 0, policy_version 25572 (0.0009) [2023-10-14 18:39:32,573][61552] Updated weights for policy 0, policy_version 25582 (0.0007) [2023-10-14 18:39:32,935][61552] Updated weights for policy 0, policy_version 25592 (0.0009) [2023-10-14 18:39:33,343][60425] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 52297728. Throughput: 0: 1678.7, 1: 1662.0. Samples: 13076928. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 18:39:33,344][60425] Avg episode reward: [(0, '61.060'), (1, '54.680')] [2023-10-14 18:39:33,667][61585] Updated weights for policy 1, policy_version 25480 (0.0008) [2023-10-14 18:39:34,028][61585] Updated weights for policy 1, policy_version 25490 (0.0009) [2023-10-14 18:39:34,389][61585] Updated weights for policy 1, policy_version 25500 (0.0010) [2023-10-14 18:39:37,047][61552] Updated weights for policy 0, policy_version 25602 (0.0009) [2023-10-14 18:39:37,408][61552] Updated weights for policy 0, policy_version 25612 (0.0008) [2023-10-14 18:39:37,784][61552] Updated weights for policy 0, policy_version 25622 (0.0010) [2023-10-14 18:39:38,145][61552] Updated weights for policy 0, policy_version 25632 (0.0009) [2023-10-14 18:39:38,343][60425] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 52363264. Throughput: 0: 1682.5, 1: 1665.3. Samples: 13097514. Policy #0 lag: (min: 17.0, avg: 29.2, max: 49.0) [2023-10-14 18:39:38,344][60425] Avg episode reward: [(0, '61.930'), (1, '57.480')] [2023-10-14 18:39:38,346][61172] Saving new best policy, reward=61.930! [2023-10-14 18:39:38,462][61585] Updated weights for policy 1, policy_version 25510 (0.0007) [2023-10-14 18:39:38,827][61585] Updated weights for policy 1, policy_version 25520 (0.0008) [2023-10-14 18:39:39,189][61585] Updated weights for policy 1, policy_version 25530 (0.0007) [2023-10-14 18:39:42,076][61552] Updated weights for policy 0, policy_version 25642 (0.0009) [2023-10-14 18:39:42,444][61552] Updated weights for policy 0, policy_version 25652 (0.0007) [2023-10-14 18:39:42,817][61552] Updated weights for policy 0, policy_version 25662 (0.0007) [2023-10-14 18:39:43,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 52428800. Throughput: 0: 1654.2, 1: 1668.2. Samples: 13116932. Policy #0 lag: (min: 17.0, avg: 29.2, max: 49.0) [2023-10-14 18:39:43,344][60425] Avg episode reward: [(0, '59.180'), (1, '54.260')] [2023-10-14 18:39:43,352][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000025664_26279936.pth... [2023-10-14 18:39:43,380][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000024096_24674304.pth [2023-10-14 18:39:43,417][61585] Updated weights for policy 1, policy_version 25540 (0.0008) [2023-10-14 18:39:43,791][61585] Updated weights for policy 1, policy_version 25550 (0.0009) [2023-10-14 18:39:44,156][61585] Updated weights for policy 1, policy_version 25560 (0.0008) [2023-10-14 18:39:44,442][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000025568_26181632.pth... [2023-10-14 18:39:44,471][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000024000_24576000.pth [2023-10-14 18:39:47,085][61552] Updated weights for policy 0, policy_version 25672 (0.0009) [2023-10-14 18:39:47,457][61552] Updated weights for policy 0, policy_version 25682 (0.0010) [2023-10-14 18:39:47,819][61552] Updated weights for policy 0, policy_version 25692 (0.0010) [2023-10-14 18:39:48,300][61585] Updated weights for policy 1, policy_version 25570 (0.0008) [2023-10-14 18:39:48,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 52494336. Throughput: 0: 1679.1, 1: 1665.5. Samples: 13126844. Policy #0 lag: (min: 17.0, avg: 29.2, max: 49.0) [2023-10-14 18:39:48,344][60425] Avg episode reward: [(0, '59.790'), (1, '56.620')] [2023-10-14 18:39:48,672][61585] Updated weights for policy 1, policy_version 25580 (0.0011) [2023-10-14 18:39:49,028][61585] Updated weights for policy 1, policy_version 25590 (0.0011) [2023-10-14 18:39:49,394][61585] Updated weights for policy 1, policy_version 25600 (0.0008) [2023-10-14 18:39:52,023][61552] Updated weights for policy 0, policy_version 25702 (0.0011) [2023-10-14 18:39:52,409][61552] Updated weights for policy 0, policy_version 25712 (0.0011) [2023-10-14 18:39:52,772][61552] Updated weights for policy 0, policy_version 25722 (0.0011) [2023-10-14 18:39:53,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 52559872. Throughput: 0: 1676.2, 1: 1666.3. Samples: 13147230. Policy #0 lag: (min: 17.0, avg: 29.2, max: 49.0) [2023-10-14 18:39:53,344][60425] Avg episode reward: [(0, '59.780'), (1, '55.040')] [2023-10-14 18:39:53,433][61585] Updated weights for policy 1, policy_version 25610 (0.0007) [2023-10-14 18:39:53,801][61585] Updated weights for policy 1, policy_version 25620 (0.0008) [2023-10-14 18:39:54,181][61585] Updated weights for policy 1, policy_version 25630 (0.0008) [2023-10-14 18:39:56,969][61552] Updated weights for policy 0, policy_version 25732 (0.0007) [2023-10-14 18:39:57,329][61552] Updated weights for policy 0, policy_version 25742 (0.0008) [2023-10-14 18:39:57,708][61552] Updated weights for policy 0, policy_version 25752 (0.0009) [2023-10-14 18:39:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 52625408. Throughput: 0: 1655.1, 1: 1666.7. Samples: 13166738. Policy #0 lag: (min: 17.0, avg: 23.8, max: 49.0) [2023-10-14 18:39:58,344][60425] Avg episode reward: [(0, '61.020'), (1, '55.590')] [2023-10-14 18:39:58,352][61585] Updated weights for policy 1, policy_version 25640 (0.0010) [2023-10-14 18:39:58,718][61585] Updated weights for policy 1, policy_version 25650 (0.0008) [2023-10-14 18:39:59,093][61585] Updated weights for policy 1, policy_version 25660 (0.0008) [2023-10-14 18:40:01,870][61552] Updated weights for policy 0, policy_version 25762 (0.0009) [2023-10-14 18:40:02,240][61552] Updated weights for policy 0, policy_version 25772 (0.0007) [2023-10-14 18:40:02,614][61552] Updated weights for policy 0, policy_version 25782 (0.0009) [2023-10-14 18:40:02,976][61552] Updated weights for policy 0, policy_version 25792 (0.0009) [2023-10-14 18:40:03,230][61585] Updated weights for policy 1, policy_version 25670 (0.0010) [2023-10-14 18:40:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 52690944. Throughput: 0: 1672.8, 1: 1664.4. Samples: 13176552. Policy #0 lag: (min: 17.0, avg: 23.8, max: 49.0) [2023-10-14 18:40:03,344][60425] Avg episode reward: [(0, '61.120'), (1, '57.820')] [2023-10-14 18:40:03,596][61585] Updated weights for policy 1, policy_version 25680 (0.0010) [2023-10-14 18:40:03,964][61585] Updated weights for policy 1, policy_version 25690 (0.0009) [2023-10-14 18:40:07,146][61552] Updated weights for policy 0, policy_version 25802 (0.0009) [2023-10-14 18:40:07,521][61552] Updated weights for policy 0, policy_version 25812 (0.0009) [2023-10-14 18:40:07,887][61552] Updated weights for policy 0, policy_version 25822 (0.0007) [2023-10-14 18:40:08,207][61585] Updated weights for policy 1, policy_version 25700 (0.0008) [2023-10-14 18:40:08,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 52756480. Throughput: 0: 1666.0, 1: 1669.7. Samples: 13197046. Policy #0 lag: (min: 17.0, avg: 23.8, max: 49.0) [2023-10-14 18:40:08,344][60425] Avg episode reward: [(0, '56.050'), (1, '56.550')] [2023-10-14 18:40:08,596][61585] Updated weights for policy 1, policy_version 25710 (0.0010) [2023-10-14 18:40:08,969][61585] Updated weights for policy 1, policy_version 25720 (0.0007) [2023-10-14 18:40:11,934][61552] Updated weights for policy 0, policy_version 25832 (0.0007) [2023-10-14 18:40:12,305][61552] Updated weights for policy 0, policy_version 25842 (0.0008) [2023-10-14 18:40:12,684][61552] Updated weights for policy 0, policy_version 25852 (0.0008) [2023-10-14 18:40:13,116][61585] Updated weights for policy 1, policy_version 25730 (0.0008) [2023-10-14 18:40:13,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 52822016. Throughput: 0: 1645.4, 1: 1664.6. Samples: 13216326. Policy #0 lag: (min: 17.0, avg: 23.8, max: 49.0) [2023-10-14 18:40:13,344][60425] Avg episode reward: [(0, '58.090'), (1, '57.500')] [2023-10-14 18:40:13,485][61585] Updated weights for policy 1, policy_version 25740 (0.0008) [2023-10-14 18:40:13,854][61585] Updated weights for policy 1, policy_version 25750 (0.0007) [2023-10-14 18:40:14,217][61585] Updated weights for policy 1, policy_version 25760 (0.0008) [2023-10-14 18:40:16,943][61552] Updated weights for policy 0, policy_version 25862 (0.0009) [2023-10-14 18:40:17,318][61552] Updated weights for policy 0, policy_version 25872 (0.0009) [2023-10-14 18:40:17,691][61552] Updated weights for policy 0, policy_version 25882 (0.0009) [2023-10-14 18:40:18,245][61585] Updated weights for policy 1, policy_version 25770 (0.0009) [2023-10-14 18:40:18,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 52887552. Throughput: 0: 1659.2, 1: 1663.2. Samples: 13226440. Policy #0 lag: (min: 20.0, avg: 24.7, max: 52.0) [2023-10-14 18:40:18,344][60425] Avg episode reward: [(0, '56.330'), (1, '56.550')] [2023-10-14 18:40:18,620][61585] Updated weights for policy 1, policy_version 25780 (0.0008) [2023-10-14 18:40:18,984][61585] Updated weights for policy 1, policy_version 25790 (0.0010) [2023-10-14 18:40:21,875][61552] Updated weights for policy 0, policy_version 25892 (0.0009) [2023-10-14 18:40:22,245][61552] Updated weights for policy 0, policy_version 25902 (0.0009) [2023-10-14 18:40:22,622][61552] Updated weights for policy 0, policy_version 25912 (0.0011) [2023-10-14 18:40:23,026][61585] Updated weights for policy 1, policy_version 25800 (0.0009) [2023-10-14 18:40:23,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 52953088. Throughput: 0: 1652.0, 1: 1664.5. Samples: 13246758. Policy #0 lag: (min: 20.0, avg: 24.7, max: 52.0) [2023-10-14 18:40:23,344][60425] Avg episode reward: [(0, '56.500'), (1, '56.820')] [2023-10-14 18:40:23,398][61585] Updated weights for policy 1, policy_version 25810 (0.0009) [2023-10-14 18:40:23,764][61585] Updated weights for policy 1, policy_version 25820 (0.0007) [2023-10-14 18:40:26,745][61552] Updated weights for policy 0, policy_version 25922 (0.0008) [2023-10-14 18:40:27,121][61552] Updated weights for policy 0, policy_version 25932 (0.0008) [2023-10-14 18:40:27,493][61552] Updated weights for policy 0, policy_version 25942 (0.0008) [2023-10-14 18:40:27,861][61552] Updated weights for policy 0, policy_version 25952 (0.0007) [2023-10-14 18:40:27,865][61585] Updated weights for policy 1, policy_version 25830 (0.0009) [2023-10-14 18:40:28,230][61585] Updated weights for policy 1, policy_version 25840 (0.0009) [2023-10-14 18:40:28,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 53018624. Throughput: 0: 1650.1, 1: 1664.9. Samples: 13266106. Policy #0 lag: (min: 20.0, avg: 24.7, max: 52.0) [2023-10-14 18:40:28,344][60425] Avg episode reward: [(0, '59.120'), (1, '58.930')] [2023-10-14 18:40:28,589][61585] Updated weights for policy 1, policy_version 25850 (0.0008) [2023-10-14 18:40:31,969][61552] Updated weights for policy 0, policy_version 25962 (0.0008) [2023-10-14 18:40:32,339][61552] Updated weights for policy 0, policy_version 25972 (0.0008) [2023-10-14 18:40:32,709][61552] Updated weights for policy 0, policy_version 25982 (0.0009) [2023-10-14 18:40:32,752][61585] Updated weights for policy 1, policy_version 25860 (0.0009) [2023-10-14 18:40:33,118][61585] Updated weights for policy 1, policy_version 25870 (0.0008) [2023-10-14 18:40:33,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 53084160. Throughput: 0: 1651.3, 1: 1665.3. Samples: 13276090. Policy #0 lag: (min: 20.0, avg: 24.7, max: 52.0) [2023-10-14 18:40:33,344][60425] Avg episode reward: [(0, '60.030'), (1, '56.800')] [2023-10-14 18:40:33,481][61585] Updated weights for policy 1, policy_version 25880 (0.0008) [2023-10-14 18:40:36,797][61552] Updated weights for policy 0, policy_version 25992 (0.0010) [2023-10-14 18:40:37,169][61552] Updated weights for policy 0, policy_version 26002 (0.0009) [2023-10-14 18:40:37,548][61552] Updated weights for policy 0, policy_version 26012 (0.0009) [2023-10-14 18:40:37,637][61585] Updated weights for policy 1, policy_version 25890 (0.0008) [2023-10-14 18:40:38,004][61585] Updated weights for policy 1, policy_version 25900 (0.0009) [2023-10-14 18:40:38,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 53149696. Throughput: 0: 1646.6, 1: 1666.7. Samples: 13296328. Policy #0 lag: (min: 1.0, avg: 12.8, max: 33.0) [2023-10-14 18:40:38,345][60425] Avg episode reward: [(0, '56.990'), (1, '59.700')] [2023-10-14 18:40:38,366][61585] Updated weights for policy 1, policy_version 25910 (0.0009) [2023-10-14 18:40:38,732][61585] Updated weights for policy 1, policy_version 25920 (0.0008) [2023-10-14 18:40:41,755][61552] Updated weights for policy 0, policy_version 26022 (0.0009) [2023-10-14 18:40:42,141][61552] Updated weights for policy 0, policy_version 26032 (0.0007) [2023-10-14 18:40:42,505][61552] Updated weights for policy 0, policy_version 26042 (0.0007) [2023-10-14 18:40:42,783][61585] Updated weights for policy 1, policy_version 25930 (0.0007) [2023-10-14 18:40:43,149][61585] Updated weights for policy 1, policy_version 25940 (0.0008) [2023-10-14 18:40:43,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 53215232. Throughput: 0: 1646.8, 1: 1658.4. Samples: 13315470. Policy #0 lag: (min: 1.0, avg: 12.8, max: 33.0) [2023-10-14 18:40:43,344][60425] Avg episode reward: [(0, '57.450'), (1, '57.760')] [2023-10-14 18:40:43,519][61585] Updated weights for policy 1, policy_version 25950 (0.0008) [2023-10-14 18:40:46,562][61552] Updated weights for policy 0, policy_version 26052 (0.0008) [2023-10-14 18:40:46,932][61552] Updated weights for policy 0, policy_version 26062 (0.0009) [2023-10-14 18:40:47,300][61552] Updated weights for policy 0, policy_version 26072 (0.0010) [2023-10-14 18:40:47,603][61585] Updated weights for policy 1, policy_version 25960 (0.0010) [2023-10-14 18:40:47,962][61585] Updated weights for policy 1, policy_version 25970 (0.0009) [2023-10-14 18:40:48,325][61585] Updated weights for policy 1, policy_version 25980 (0.0007) [2023-10-14 18:40:48,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 53280768. Throughput: 0: 1654.6, 1: 1668.6. Samples: 13326094. Policy #0 lag: (min: 1.0, avg: 12.8, max: 33.0) [2023-10-14 18:40:48,344][60425] Avg episode reward: [(0, '53.680'), (1, '57.260')] [2023-10-14 18:40:51,483][61552] Updated weights for policy 0, policy_version 26082 (0.0008) [2023-10-14 18:40:51,850][61552] Updated weights for policy 0, policy_version 26092 (0.0009) [2023-10-14 18:40:52,228][61552] Updated weights for policy 0, policy_version 26102 (0.0008) [2023-10-14 18:40:52,444][61585] Updated weights for policy 1, policy_version 25990 (0.0009) [2023-10-14 18:40:52,606][61552] Updated weights for policy 0, policy_version 26112 (0.0007) [2023-10-14 18:40:52,813][61585] Updated weights for policy 1, policy_version 26000 (0.0008) [2023-10-14 18:40:53,183][61585] Updated weights for policy 1, policy_version 26010 (0.0009) [2023-10-14 18:40:53,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 53346304. Throughput: 0: 1652.2, 1: 1661.5. Samples: 13346162. Policy #0 lag: (min: 1.0, avg: 12.8, max: 33.0) [2023-10-14 18:40:53,344][60425] Avg episode reward: [(0, '52.320'), (1, '56.500')] [2023-10-14 18:40:56,632][61552] Updated weights for policy 0, policy_version 26122 (0.0007) [2023-10-14 18:40:56,999][61552] Updated weights for policy 0, policy_version 26132 (0.0010) [2023-10-14 18:40:57,252][61585] Updated weights for policy 1, policy_version 26020 (0.0008) [2023-10-14 18:40:57,365][61552] Updated weights for policy 0, policy_version 26142 (0.0007) [2023-10-14 18:40:57,637][61585] Updated weights for policy 1, policy_version 26030 (0.0009) [2023-10-14 18:40:58,006][61585] Updated weights for policy 1, policy_version 26040 (0.0008) [2023-10-14 18:40:58,343][60425] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 53444608. Throughput: 0: 1656.7, 1: 1649.7. Samples: 13365112. Policy #0 lag: (min: 28.0, avg: 30.8, max: 60.0) [2023-10-14 18:40:58,344][60425] Avg episode reward: [(0, '56.200'), (1, '54.900')] [2023-10-14 18:41:01,563][61552] Updated weights for policy 0, policy_version 26152 (0.0011) [2023-10-14 18:41:01,924][61552] Updated weights for policy 0, policy_version 26162 (0.0009) [2023-10-14 18:41:02,155][61585] Updated weights for policy 1, policy_version 26050 (0.0008) [2023-10-14 18:41:02,289][61552] Updated weights for policy 0, policy_version 26172 (0.0007) [2023-10-14 18:41:02,526][61585] Updated weights for policy 1, policy_version 26060 (0.0007) [2023-10-14 18:41:02,892][61585] Updated weights for policy 1, policy_version 26070 (0.0009) [2023-10-14 18:41:03,263][61585] Updated weights for policy 1, policy_version 26080 (0.0009) [2023-10-14 18:41:03,344][60425] Fps is (10 sec: 16383.5, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 53510144. Throughput: 0: 1661.2, 1: 1663.5. Samples: 13376054. Policy #0 lag: (min: 28.0, avg: 30.8, max: 60.0) [2023-10-14 18:41:03,345][60425] Avg episode reward: [(0, '59.010'), (1, '59.500')] [2023-10-14 18:41:06,341][61552] Updated weights for policy 0, policy_version 26182 (0.0009) [2023-10-14 18:41:06,707][61552] Updated weights for policy 0, policy_version 26192 (0.0008) [2023-10-14 18:41:07,077][61552] Updated weights for policy 0, policy_version 26202 (0.0008) [2023-10-14 18:41:07,486][61585] Updated weights for policy 1, policy_version 26090 (0.0008) [2023-10-14 18:41:07,848][61585] Updated weights for policy 1, policy_version 26100 (0.0008) [2023-10-14 18:41:08,222][61585] Updated weights for policy 1, policy_version 26110 (0.0008) [2023-10-14 18:41:08,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 53575680. Throughput: 0: 1653.5, 1: 1662.1. Samples: 13395960. Policy #0 lag: (min: 28.0, avg: 30.8, max: 60.0) [2023-10-14 18:41:08,344][60425] Avg episode reward: [(0, '56.150'), (1, '58.770')] [2023-10-14 18:41:11,051][61552] Updated weights for policy 0, policy_version 26212 (0.0008) [2023-10-14 18:41:11,418][61552] Updated weights for policy 0, policy_version 26222 (0.0009) [2023-10-14 18:41:11,798][61552] Updated weights for policy 0, policy_version 26232 (0.0011) [2023-10-14 18:41:12,308][61585] Updated weights for policy 1, policy_version 26120 (0.0008) [2023-10-14 18:41:12,664][61585] Updated weights for policy 1, policy_version 26130 (0.0010) [2023-10-14 18:41:13,044][61585] Updated weights for policy 1, policy_version 26140 (0.0010) [2023-10-14 18:41:13,344][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.2, 300 sec: 13329.3). Total num frames: 53641216. Throughput: 0: 1666.3, 1: 1647.1. Samples: 13415208. Policy #0 lag: (min: 28.0, avg: 30.8, max: 60.0) [2023-10-14 18:41:13,345][60425] Avg episode reward: [(0, '56.710'), (1, '58.000')] [2023-10-14 18:41:15,835][61552] Updated weights for policy 0, policy_version 26242 (0.0009) [2023-10-14 18:41:16,198][61552] Updated weights for policy 0, policy_version 26252 (0.0008) [2023-10-14 18:41:16,566][61552] Updated weights for policy 0, policy_version 26262 (0.0009) [2023-10-14 18:41:16,933][61552] Updated weights for policy 0, policy_version 26272 (0.0009) [2023-10-14 18:41:17,033][61585] Updated weights for policy 1, policy_version 26150 (0.0010) [2023-10-14 18:41:17,401][61585] Updated weights for policy 1, policy_version 26160 (0.0009) [2023-10-14 18:41:17,764][61585] Updated weights for policy 1, policy_version 26170 (0.0008) [2023-10-14 18:41:18,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 53706752. Throughput: 0: 1673.2, 1: 1664.1. Samples: 13426270. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-14 18:41:18,344][60425] Avg episode reward: [(0, '58.090'), (1, '59.400')] [2023-10-14 18:41:21,144][61552] Updated weights for policy 0, policy_version 26282 (0.0009) [2023-10-14 18:41:21,516][61552] Updated weights for policy 0, policy_version 26292 (0.0011) [2023-10-14 18:41:21,793][61585] Updated weights for policy 1, policy_version 26180 (0.0008) [2023-10-14 18:41:21,886][61552] Updated weights for policy 0, policy_version 26302 (0.0008) [2023-10-14 18:41:22,161][61585] Updated weights for policy 1, policy_version 26190 (0.0008) [2023-10-14 18:41:22,528][61585] Updated weights for policy 1, policy_version 26200 (0.0010) [2023-10-14 18:41:23,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 53772288. Throughput: 0: 1658.8, 1: 1663.2. Samples: 13445816. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-14 18:41:23,344][60425] Avg episode reward: [(0, '57.090'), (1, '55.420')] [2023-10-14 18:41:26,029][61552] Updated weights for policy 0, policy_version 26312 (0.0008) [2023-10-14 18:41:26,387][61552] Updated weights for policy 0, policy_version 26322 (0.0010) [2023-10-14 18:41:26,650][61585] Updated weights for policy 1, policy_version 26210 (0.0008) [2023-10-14 18:41:26,768][61552] Updated weights for policy 0, policy_version 26332 (0.0009) [2023-10-14 18:41:27,020][61585] Updated weights for policy 1, policy_version 26220 (0.0008) [2023-10-14 18:41:27,388][61585] Updated weights for policy 1, policy_version 26230 (0.0007) [2023-10-14 18:41:27,745][61585] Updated weights for policy 1, policy_version 26240 (0.0008) [2023-10-14 18:41:28,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 53837824. Throughput: 0: 1675.0, 1: 1646.3. Samples: 13464928. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-14 18:41:28,344][60425] Avg episode reward: [(0, '58.790'), (1, '55.010')] [2023-10-14 18:41:30,920][61552] Updated weights for policy 0, policy_version 26342 (0.0007) [2023-10-14 18:41:31,311][61552] Updated weights for policy 0, policy_version 26352 (0.0008) [2023-10-14 18:41:31,672][61552] Updated weights for policy 0, policy_version 26362 (0.0009) [2023-10-14 18:41:31,974][61585] Updated weights for policy 1, policy_version 26250 (0.0009) [2023-10-14 18:41:32,334][61585] Updated weights for policy 1, policy_version 26260 (0.0011) [2023-10-14 18:41:32,702][61585] Updated weights for policy 1, policy_version 26270 (0.0009) [2023-10-14 18:41:33,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 53903360. Throughput: 0: 1673.7, 1: 1660.9. Samples: 13476152. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-14 18:41:33,344][60425] Avg episode reward: [(0, '59.430'), (1, '58.950')] [2023-10-14 18:41:35,685][61552] Updated weights for policy 0, policy_version 26372 (0.0009) [2023-10-14 18:41:36,056][61552] Updated weights for policy 0, policy_version 26382 (0.0010) [2023-10-14 18:41:36,427][61552] Updated weights for policy 0, policy_version 26392 (0.0008) [2023-10-14 18:41:37,152][61585] Updated weights for policy 1, policy_version 26280 (0.0010) [2023-10-14 18:41:37,523][61585] Updated weights for policy 1, policy_version 26290 (0.0009) [2023-10-14 18:41:37,891][61585] Updated weights for policy 1, policy_version 26300 (0.0008) [2023-10-14 18:41:38,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 53968896. Throughput: 0: 1655.9, 1: 1658.5. Samples: 13495308. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:41:38,344][60425] Avg episode reward: [(0, '58.550'), (1, '55.510')] [2023-10-14 18:41:40,340][61552] Updated weights for policy 0, policy_version 26402 (0.0008) [2023-10-14 18:41:40,703][61552] Updated weights for policy 0, policy_version 26412 (0.0009) [2023-10-14 18:41:41,078][61552] Updated weights for policy 0, policy_version 26422 (0.0008) [2023-10-14 18:41:41,442][61552] Updated weights for policy 0, policy_version 26432 (0.0009) [2023-10-14 18:41:42,212][61585] Updated weights for policy 1, policy_version 26310 (0.0009) [2023-10-14 18:41:42,574][61585] Updated weights for policy 1, policy_version 26320 (0.0009) [2023-10-14 18:41:42,937][61585] Updated weights for policy 1, policy_version 26330 (0.0009) [2023-10-14 18:41:43,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 54034432. Throughput: 0: 1678.2, 1: 1653.7. Samples: 13515048. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:41:43,345][60425] Avg episode reward: [(0, '55.870'), (1, '55.590')] [2023-10-14 18:41:43,355][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000026336_26968064.pth... [2023-10-14 18:41:43,356][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000026432_27066368.pth... [2023-10-14 18:41:43,392][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000024864_25460736.pth [2023-10-14 18:41:43,395][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000024768_25362432.pth [2023-10-14 18:41:45,500][61552] Updated weights for policy 0, policy_version 26442 (0.0008) [2023-10-14 18:41:45,862][61552] Updated weights for policy 0, policy_version 26452 (0.0007) [2023-10-14 18:41:46,234][61552] Updated weights for policy 0, policy_version 26462 (0.0009) [2023-10-14 18:41:47,097][61585] Updated weights for policy 1, policy_version 26340 (0.0010) [2023-10-14 18:41:47,500][61585] Updated weights for policy 1, policy_version 26350 (0.0010) [2023-10-14 18:41:47,863][61585] Updated weights for policy 1, policy_version 26360 (0.0009) [2023-10-14 18:41:48,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 54099968. Throughput: 0: 1666.2, 1: 1657.9. Samples: 13525638. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:41:48,344][60425] Avg episode reward: [(0, '55.420'), (1, '56.570')] [2023-10-14 18:41:50,345][61552] Updated weights for policy 0, policy_version 26472 (0.0009) [2023-10-14 18:41:50,708][61552] Updated weights for policy 0, policy_version 26482 (0.0010) [2023-10-14 18:41:51,087][61552] Updated weights for policy 0, policy_version 26492 (0.0008) [2023-10-14 18:41:51,813][61585] Updated weights for policy 1, policy_version 26370 (0.0009) [2023-10-14 18:41:52,188][61585] Updated weights for policy 1, policy_version 26380 (0.0010) [2023-10-14 18:41:52,566][61585] Updated weights for policy 1, policy_version 26390 (0.0009) [2023-10-14 18:41:52,933][61585] Updated weights for policy 1, policy_version 26400 (0.0007) [2023-10-14 18:41:53,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 54165504. Throughput: 0: 1658.4, 1: 1659.6. Samples: 13545270. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:41:53,345][60425] Avg episode reward: [(0, '54.800'), (1, '57.150')] [2023-10-14 18:41:55,257][61552] Updated weights for policy 0, policy_version 26502 (0.0009) [2023-10-14 18:41:55,628][61552] Updated weights for policy 0, policy_version 26512 (0.0007) [2023-10-14 18:41:55,995][61552] Updated weights for policy 0, policy_version 26522 (0.0009) [2023-10-14 18:41:56,980][61585] Updated weights for policy 1, policy_version 26410 (0.0008) [2023-10-14 18:41:57,334][61585] Updated weights for policy 1, policy_version 26420 (0.0009) [2023-10-14 18:41:57,701][61585] Updated weights for policy 1, policy_version 26430 (0.0008) [2023-10-14 18:41:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 54231040. Throughput: 0: 1676.7, 1: 1647.7. Samples: 13564806. Policy #0 lag: (min: 31.0, avg: 31.7, max: 47.0) [2023-10-14 18:41:58,344][60425] Avg episode reward: [(0, '56.880'), (1, '57.880')] [2023-10-14 18:41:59,912][61552] Updated weights for policy 0, policy_version 26532 (0.0007) [2023-10-14 18:42:00,279][61552] Updated weights for policy 0, policy_version 26542 (0.0008) [2023-10-14 18:42:00,656][61552] Updated weights for policy 0, policy_version 26552 (0.0009) [2023-10-14 18:42:01,700][61585] Updated weights for policy 1, policy_version 26440 (0.0010) [2023-10-14 18:42:02,068][61585] Updated weights for policy 1, policy_version 26450 (0.0008) [2023-10-14 18:42:02,445][61585] Updated weights for policy 1, policy_version 26460 (0.0008) [2023-10-14 18:42:03,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 54296576. Throughput: 0: 1652.4, 1: 1657.7. Samples: 13575226. Policy #0 lag: (min: 31.0, avg: 31.7, max: 47.0) [2023-10-14 18:42:03,344][60425] Avg episode reward: [(0, '57.430'), (1, '57.250')] [2023-10-14 18:42:04,808][61552] Updated weights for policy 0, policy_version 26562 (0.0008) [2023-10-14 18:42:05,170][61552] Updated weights for policy 0, policy_version 26572 (0.0007) [2023-10-14 18:42:05,543][61552] Updated weights for policy 0, policy_version 26582 (0.0008) [2023-10-14 18:42:05,912][61552] Updated weights for policy 0, policy_version 26592 (0.0010) [2023-10-14 18:42:06,526][61585] Updated weights for policy 1, policy_version 26470 (0.0007) [2023-10-14 18:42:06,893][61585] Updated weights for policy 1, policy_version 26480 (0.0007) [2023-10-14 18:42:07,256][61585] Updated weights for policy 1, policy_version 26490 (0.0007) [2023-10-14 18:42:08,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 54362112. Throughput: 0: 1665.6, 1: 1644.8. Samples: 13594780. Policy #0 lag: (min: 31.0, avg: 31.7, max: 47.0) [2023-10-14 18:42:08,344][60425] Avg episode reward: [(0, '59.600'), (1, '57.610')] [2023-10-14 18:42:10,078][61552] Updated weights for policy 0, policy_version 26602 (0.0009) [2023-10-14 18:42:10,447][61552] Updated weights for policy 0, policy_version 26612 (0.0009) [2023-10-14 18:42:10,818][61552] Updated weights for policy 0, policy_version 26622 (0.0009) [2023-10-14 18:42:11,536][61585] Updated weights for policy 1, policy_version 26500 (0.0008) [2023-10-14 18:42:11,899][61585] Updated weights for policy 1, policy_version 26510 (0.0007) [2023-10-14 18:42:12,261][61585] Updated weights for policy 1, policy_version 26520 (0.0009) [2023-10-14 18:42:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 54427648. Throughput: 0: 1678.0, 1: 1652.0. Samples: 13614780. Policy #0 lag: (min: 31.0, avg: 31.7, max: 47.0) [2023-10-14 18:42:13,344][60425] Avg episode reward: [(0, '55.850'), (1, '58.360')] [2023-10-14 18:42:14,767][61552] Updated weights for policy 0, policy_version 26632 (0.0009) [2023-10-14 18:42:15,144][61552] Updated weights for policy 0, policy_version 26642 (0.0009) [2023-10-14 18:42:15,511][61552] Updated weights for policy 0, policy_version 26652 (0.0010) [2023-10-14 18:42:16,435][61585] Updated weights for policy 1, policy_version 26530 (0.0009) [2023-10-14 18:42:16,792][61585] Updated weights for policy 1, policy_version 26540 (0.0010) [2023-10-14 18:42:17,153][61585] Updated weights for policy 1, policy_version 26550 (0.0009) [2023-10-14 18:42:17,515][61585] Updated weights for policy 1, policy_version 26560 (0.0010) [2023-10-14 18:42:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 54493184. Throughput: 0: 1651.5, 1: 1655.7. Samples: 13624978. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 18:42:18,344][60425] Avg episode reward: [(0, '56.910'), (1, '55.620')] [2023-10-14 18:42:19,580][61552] Updated weights for policy 0, policy_version 26662 (0.0009) [2023-10-14 18:42:19,937][61552] Updated weights for policy 0, policy_version 26672 (0.0009) [2023-10-14 18:42:20,312][61552] Updated weights for policy 0, policy_version 26682 (0.0007) [2023-10-14 18:42:21,485][61585] Updated weights for policy 1, policy_version 26570 (0.0008) [2023-10-14 18:42:21,846][61585] Updated weights for policy 1, policy_version 26580 (0.0009) [2023-10-14 18:42:22,218][61585] Updated weights for policy 1, policy_version 26590 (0.0008) [2023-10-14 18:42:23,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 54558720. Throughput: 0: 1679.6, 1: 1649.6. Samples: 13645126. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 18:42:23,344][60425] Avg episode reward: [(0, '56.020'), (1, '60.610')] [2023-10-14 18:42:24,470][61552] Updated weights for policy 0, policy_version 26692 (0.0008) [2023-10-14 18:42:24,854][61552] Updated weights for policy 0, policy_version 26702 (0.0009) [2023-10-14 18:42:25,233][61552] Updated weights for policy 0, policy_version 26712 (0.0008) [2023-10-14 18:42:26,315][61585] Updated weights for policy 1, policy_version 26600 (0.0007) [2023-10-14 18:42:26,679][61585] Updated weights for policy 1, policy_version 26610 (0.0008) [2023-10-14 18:42:27,057][61585] Updated weights for policy 1, policy_version 26620 (0.0010) [2023-10-14 18:42:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 54624256. Throughput: 0: 1679.3, 1: 1657.8. Samples: 13665220. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 18:42:28,344][60425] Avg episode reward: [(0, '58.070'), (1, '54.230')] [2023-10-14 18:42:29,272][61552] Updated weights for policy 0, policy_version 26722 (0.0007) [2023-10-14 18:42:29,642][61552] Updated weights for policy 0, policy_version 26732 (0.0008) [2023-10-14 18:42:30,022][61552] Updated weights for policy 0, policy_version 26742 (0.0008) [2023-10-14 18:42:30,389][61552] Updated weights for policy 0, policy_version 26752 (0.0008) [2023-10-14 18:42:31,288][61585] Updated weights for policy 1, policy_version 26630 (0.0009) [2023-10-14 18:42:31,649][61585] Updated weights for policy 1, policy_version 26640 (0.0009) [2023-10-14 18:42:32,027][61585] Updated weights for policy 1, policy_version 26650 (0.0010) [2023-10-14 18:42:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 54689792. Throughput: 0: 1661.3, 1: 1666.0. Samples: 13675370. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 18:42:33,344][60425] Avg episode reward: [(0, '60.140'), (1, '58.520')] [2023-10-14 18:42:34,455][61552] Updated weights for policy 0, policy_version 26762 (0.0007) [2023-10-14 18:42:34,831][61552] Updated weights for policy 0, policy_version 26772 (0.0009) [2023-10-14 18:42:35,196][61552] Updated weights for policy 0, policy_version 26782 (0.0008) [2023-10-14 18:42:36,257][61585] Updated weights for policy 1, policy_version 26660 (0.0009) [2023-10-14 18:42:36,662][61585] Updated weights for policy 1, policy_version 26670 (0.0009) [2023-10-14 18:42:37,038][61585] Updated weights for policy 1, policy_version 26680 (0.0008) [2023-10-14 18:42:38,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 54755328. Throughput: 0: 1682.8, 1: 1646.9. Samples: 13695102. Policy #0 lag: (min: 28.0, avg: 32.2, max: 60.0) [2023-10-14 18:42:38,344][60425] Avg episode reward: [(0, '58.180'), (1, '54.110')] [2023-10-14 18:42:39,265][61552] Updated weights for policy 0, policy_version 26792 (0.0009) [2023-10-14 18:42:39,640][61552] Updated weights for policy 0, policy_version 26802 (0.0009) [2023-10-14 18:42:40,008][61552] Updated weights for policy 0, policy_version 26812 (0.0008) [2023-10-14 18:42:41,334][61585] Updated weights for policy 1, policy_version 26690 (0.0009) [2023-10-14 18:42:41,701][61585] Updated weights for policy 1, policy_version 26700 (0.0007) [2023-10-14 18:42:42,063][61585] Updated weights for policy 1, policy_version 26710 (0.0007) [2023-10-14 18:42:42,428][61585] Updated weights for policy 1, policy_version 26720 (0.0009) [2023-10-14 18:42:43,344][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 54820864. Throughput: 0: 1681.9, 1: 1654.0. Samples: 13714922. Policy #0 lag: (min: 28.0, avg: 32.2, max: 60.0) [2023-10-14 18:42:43,345][60425] Avg episode reward: [(0, '59.610'), (1, '54.590')] [2023-10-14 18:42:44,173][61552] Updated weights for policy 0, policy_version 26822 (0.0008) [2023-10-14 18:42:44,542][61552] Updated weights for policy 0, policy_version 26832 (0.0009) [2023-10-14 18:42:44,913][61552] Updated weights for policy 0, policy_version 26842 (0.0008) [2023-10-14 18:42:46,411][61585] Updated weights for policy 1, policy_version 26730 (0.0010) [2023-10-14 18:42:46,776][61585] Updated weights for policy 1, policy_version 26740 (0.0008) [2023-10-14 18:42:47,144][61585] Updated weights for policy 1, policy_version 26750 (0.0008) [2023-10-14 18:42:48,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 54886400. Throughput: 0: 1676.3, 1: 1657.0. Samples: 13725224. Policy #0 lag: (min: 28.0, avg: 32.2, max: 60.0) [2023-10-14 18:42:48,344][60425] Avg episode reward: [(0, '56.270'), (1, '57.460')] [2023-10-14 18:42:49,106][61552] Updated weights for policy 0, policy_version 26852 (0.0008) [2023-10-14 18:42:49,478][61552] Updated weights for policy 0, policy_version 26862 (0.0008) [2023-10-14 18:42:49,854][61552] Updated weights for policy 0, policy_version 26872 (0.0008) [2023-10-14 18:42:51,304][61585] Updated weights for policy 1, policy_version 26760 (0.0009) [2023-10-14 18:42:51,666][61585] Updated weights for policy 1, policy_version 26770 (0.0008) [2023-10-14 18:42:52,043][61585] Updated weights for policy 1, policy_version 26780 (0.0009) [2023-10-14 18:42:53,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 54951936. Throughput: 0: 1684.2, 1: 1654.0. Samples: 13745002. Policy #0 lag: (min: 28.0, avg: 32.2, max: 60.0) [2023-10-14 18:42:53,344][60425] Avg episode reward: [(0, '58.270'), (1, '53.800')] [2023-10-14 18:42:54,146][61552] Updated weights for policy 0, policy_version 26882 (0.0008) [2023-10-14 18:42:54,531][61552] Updated weights for policy 0, policy_version 26892 (0.0011) [2023-10-14 18:42:54,897][61552] Updated weights for policy 0, policy_version 26902 (0.0007) [2023-10-14 18:42:55,273][61552] Updated weights for policy 0, policy_version 26912 (0.0009) [2023-10-14 18:42:56,192][61585] Updated weights for policy 1, policy_version 26790 (0.0009) [2023-10-14 18:42:56,557][61585] Updated weights for policy 1, policy_version 26800 (0.0009) [2023-10-14 18:42:56,925][61585] Updated weights for policy 1, policy_version 26810 (0.0008) [2023-10-14 18:42:58,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 55017472. Throughput: 0: 1676.4, 1: 1658.0. Samples: 13764826. Policy #0 lag: (min: 37.0, avg: 54.8, max: 56.0) [2023-10-14 18:42:58,344][60425] Avg episode reward: [(0, '57.570'), (1, '58.380')] [2023-10-14 18:42:59,260][61552] Updated weights for policy 0, policy_version 26922 (0.0008) [2023-10-14 18:42:59,626][61552] Updated weights for policy 0, policy_version 26932 (0.0008) [2023-10-14 18:42:59,994][61552] Updated weights for policy 0, policy_version 26942 (0.0008) [2023-10-14 18:43:01,009][61585] Updated weights for policy 1, policy_version 26820 (0.0007) [2023-10-14 18:43:01,369][61585] Updated weights for policy 1, policy_version 26830 (0.0008) [2023-10-14 18:43:01,741][61585] Updated weights for policy 1, policy_version 26840 (0.0007) [2023-10-14 18:43:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 55083008. Throughput: 0: 1676.5, 1: 1660.9. Samples: 13775164. Policy #0 lag: (min: 37.0, avg: 54.8, max: 56.0) [2023-10-14 18:43:03,344][60425] Avg episode reward: [(0, '61.460'), (1, '57.900')] [2023-10-14 18:43:04,048][61552] Updated weights for policy 0, policy_version 26952 (0.0009) [2023-10-14 18:43:04,417][61552] Updated weights for policy 0, policy_version 26962 (0.0008) [2023-10-14 18:43:04,791][61552] Updated weights for policy 0, policy_version 26972 (0.0008) [2023-10-14 18:43:05,829][61585] Updated weights for policy 1, policy_version 26850 (0.0009) [2023-10-14 18:43:06,200][61585] Updated weights for policy 1, policy_version 26860 (0.0007) [2023-10-14 18:43:06,570][61585] Updated weights for policy 1, policy_version 26870 (0.0009) [2023-10-14 18:43:06,938][61585] Updated weights for policy 1, policy_version 26880 (0.0009) [2023-10-14 18:43:08,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 55148544. Throughput: 0: 1674.6, 1: 1649.1. Samples: 13794694. Policy #0 lag: (min: 37.0, avg: 54.8, max: 56.0) [2023-10-14 18:43:08,344][60425] Avg episode reward: [(0, '55.430'), (1, '56.680')] [2023-10-14 18:43:08,816][61552] Updated weights for policy 0, policy_version 26982 (0.0010) [2023-10-14 18:43:09,194][61552] Updated weights for policy 0, policy_version 26992 (0.0009) [2023-10-14 18:43:09,568][61552] Updated weights for policy 0, policy_version 27002 (0.0010) [2023-10-14 18:43:11,016][61585] Updated weights for policy 1, policy_version 26890 (0.0009) [2023-10-14 18:43:11,389][61585] Updated weights for policy 1, policy_version 26900 (0.0009) [2023-10-14 18:43:11,749][61585] Updated weights for policy 1, policy_version 26910 (0.0007) [2023-10-14 18:43:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 55214080. Throughput: 0: 1674.5, 1: 1654.4. Samples: 13815020. Policy #0 lag: (min: 37.0, avg: 54.8, max: 56.0) [2023-10-14 18:43:13,344][60425] Avg episode reward: [(0, '57.740'), (1, '60.810')] [2023-10-14 18:43:13,669][61552] Updated weights for policy 0, policy_version 27012 (0.0010) [2023-10-14 18:43:14,067][61552] Updated weights for policy 0, policy_version 27022 (0.0010) [2023-10-14 18:43:14,429][61552] Updated weights for policy 0, policy_version 27032 (0.0008) [2023-10-14 18:43:16,034][61585] Updated weights for policy 1, policy_version 26920 (0.0007) [2023-10-14 18:43:16,405][61585] Updated weights for policy 1, policy_version 26930 (0.0007) [2023-10-14 18:43:16,768][61585] Updated weights for policy 1, policy_version 26940 (0.0009) [2023-10-14 18:43:18,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 55279616. Throughput: 0: 1673.3, 1: 1655.1. Samples: 13825148. Policy #0 lag: (min: 31.0, avg: 31.4, max: 46.0) [2023-10-14 18:43:18,344][60425] Avg episode reward: [(0, '57.420'), (1, '56.860')] [2023-10-14 18:43:18,532][61552] Updated weights for policy 0, policy_version 27042 (0.0008) [2023-10-14 18:43:18,900][61552] Updated weights for policy 0, policy_version 27052 (0.0009) [2023-10-14 18:43:19,278][61552] Updated weights for policy 0, policy_version 27062 (0.0008) [2023-10-14 18:43:19,650][61552] Updated weights for policy 0, policy_version 27072 (0.0009) [2023-10-14 18:43:21,059][61585] Updated weights for policy 1, policy_version 26950 (0.0010) [2023-10-14 18:43:21,428][61585] Updated weights for policy 1, policy_version 26960 (0.0009) [2023-10-14 18:43:21,788][61585] Updated weights for policy 1, policy_version 26970 (0.0009) [2023-10-14 18:43:23,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 55345152. Throughput: 0: 1668.7, 1: 1648.7. Samples: 13844384. Policy #0 lag: (min: 31.0, avg: 31.4, max: 46.0) [2023-10-14 18:43:23,344][60425] Avg episode reward: [(0, '58.720'), (1, '57.490')] [2023-10-14 18:43:23,730][61552] Updated weights for policy 0, policy_version 27082 (0.0008) [2023-10-14 18:43:24,111][61552] Updated weights for policy 0, policy_version 27092 (0.0010) [2023-10-14 18:43:24,488][61552] Updated weights for policy 0, policy_version 27102 (0.0008) [2023-10-14 18:43:25,770][61585] Updated weights for policy 1, policy_version 26980 (0.0009) [2023-10-14 18:43:26,133][61585] Updated weights for policy 1, policy_version 26990 (0.0009) [2023-10-14 18:43:26,501][61585] Updated weights for policy 1, policy_version 27000 (0.0009) [2023-10-14 18:43:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 55410688. Throughput: 0: 1661.4, 1: 1664.6. Samples: 13864592. Policy #0 lag: (min: 31.0, avg: 31.4, max: 46.0) [2023-10-14 18:43:28,344][60425] Avg episode reward: [(0, '54.630'), (1, '54.790')] [2023-10-14 18:43:28,633][61552] Updated weights for policy 0, policy_version 27112 (0.0007) [2023-10-14 18:43:28,999][61552] Updated weights for policy 0, policy_version 27122 (0.0007) [2023-10-14 18:43:29,374][61552] Updated weights for policy 0, policy_version 27132 (0.0007) [2023-10-14 18:43:30,790][61585] Updated weights for policy 1, policy_version 27010 (0.0009) [2023-10-14 18:43:31,159][61585] Updated weights for policy 1, policy_version 27020 (0.0008) [2023-10-14 18:43:31,518][61585] Updated weights for policy 1, policy_version 27030 (0.0007) [2023-10-14 18:43:31,891][61585] Updated weights for policy 1, policy_version 27040 (0.0008) [2023-10-14 18:43:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 55476224. Throughput: 0: 1661.7, 1: 1659.5. Samples: 13874678. Policy #0 lag: (min: 31.0, avg: 31.4, max: 46.0) [2023-10-14 18:43:33,344][60425] Avg episode reward: [(0, '54.980'), (1, '59.060')] [2023-10-14 18:43:33,460][61552] Updated weights for policy 0, policy_version 27142 (0.0010) [2023-10-14 18:43:33,816][61552] Updated weights for policy 0, policy_version 27152 (0.0009) [2023-10-14 18:43:34,191][61552] Updated weights for policy 0, policy_version 27162 (0.0007) [2023-10-14 18:43:35,937][61585] Updated weights for policy 1, policy_version 27050 (0.0007) [2023-10-14 18:43:36,304][61585] Updated weights for policy 1, policy_version 27060 (0.0008) [2023-10-14 18:43:36,668][61585] Updated weights for policy 1, policy_version 27070 (0.0009) [2023-10-14 18:43:38,172][61552] Updated weights for policy 0, policy_version 27172 (0.0008) [2023-10-14 18:43:38,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 55541760. Throughput: 0: 1662.6, 1: 1652.9. Samples: 13894200. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:43:38,344][60425] Avg episode reward: [(0, '54.240'), (1, '55.290')] [2023-10-14 18:43:38,551][61552] Updated weights for policy 0, policy_version 27182 (0.0009) [2023-10-14 18:43:38,920][61552] Updated weights for policy 0, policy_version 27192 (0.0009) [2023-10-14 18:43:40,751][61585] Updated weights for policy 1, policy_version 27080 (0.0008) [2023-10-14 18:43:41,116][61585] Updated weights for policy 1, policy_version 27090 (0.0009) [2023-10-14 18:43:41,474][61585] Updated weights for policy 1, policy_version 27100 (0.0010) [2023-10-14 18:43:43,002][61552] Updated weights for policy 0, policy_version 27202 (0.0009) [2023-10-14 18:43:43,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 55607296. Throughput: 0: 1668.7, 1: 1668.3. Samples: 13914990. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:43:43,344][60425] Avg episode reward: [(0, '57.510'), (1, '58.610')] [2023-10-14 18:43:43,355][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000027104_27754496.pth... [2023-10-14 18:43:43,375][61552] Updated weights for policy 0, policy_version 27212 (0.0008) [2023-10-14 18:43:43,393][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000025568_26181632.pth [2023-10-14 18:43:43,744][61552] Updated weights for policy 0, policy_version 27222 (0.0010) [2023-10-14 18:43:44,105][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000027232_27885568.pth... [2023-10-14 18:43:44,106][61552] Updated weights for policy 0, policy_version 27232 (0.0010) [2023-10-14 18:43:44,142][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000025664_26279936.pth [2023-10-14 18:43:45,406][61585] Updated weights for policy 1, policy_version 27110 (0.0008) [2023-10-14 18:43:45,778][61585] Updated weights for policy 1, policy_version 27120 (0.0009) [2023-10-14 18:43:46,140][61585] Updated weights for policy 1, policy_version 27130 (0.0010) [2023-10-14 18:43:48,184][61552] Updated weights for policy 0, policy_version 27242 (0.0009) [2023-10-14 18:43:48,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 55672832. Throughput: 0: 1667.3, 1: 1654.0. Samples: 13924624. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:43:48,344][60425] Avg episode reward: [(0, '54.120'), (1, '57.510')] [2023-10-14 18:43:48,555][61552] Updated weights for policy 0, policy_version 27252 (0.0010) [2023-10-14 18:43:48,921][61552] Updated weights for policy 0, policy_version 27262 (0.0007) [2023-10-14 18:43:50,372][61585] Updated weights for policy 1, policy_version 27140 (0.0010) [2023-10-14 18:43:50,738][61585] Updated weights for policy 1, policy_version 27150 (0.0009) [2023-10-14 18:43:51,108][61585] Updated weights for policy 1, policy_version 27160 (0.0010) [2023-10-14 18:43:53,013][61552] Updated weights for policy 0, policy_version 27272 (0.0009) [2023-10-14 18:43:53,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 55738368. Throughput: 0: 1671.2, 1: 1655.6. Samples: 13944404. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:43:53,344][60425] Avg episode reward: [(0, '56.560'), (1, '57.190')] [2023-10-14 18:43:53,375][61552] Updated weights for policy 0, policy_version 27282 (0.0010) [2023-10-14 18:43:53,738][61552] Updated weights for policy 0, policy_version 27292 (0.0010) [2023-10-14 18:43:55,012][61585] Updated weights for policy 1, policy_version 27170 (0.0010) [2023-10-14 18:43:55,378][61585] Updated weights for policy 1, policy_version 27180 (0.0007) [2023-10-14 18:43:55,743][61585] Updated weights for policy 1, policy_version 27190 (0.0007) [2023-10-14 18:43:56,112][61585] Updated weights for policy 1, policy_version 27200 (0.0010) [2023-10-14 18:43:57,886][61552] Updated weights for policy 0, policy_version 27302 (0.0009) [2023-10-14 18:43:58,260][61552] Updated weights for policy 0, policy_version 27312 (0.0009) [2023-10-14 18:43:58,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 55803904. Throughput: 0: 1670.0, 1: 1668.0. Samples: 13965232. Policy #0 lag: (min: 3.0, avg: 5.8, max: 35.0) [2023-10-14 18:43:58,344][60425] Avg episode reward: [(0, '56.880'), (1, '56.100')] [2023-10-14 18:43:58,631][61552] Updated weights for policy 0, policy_version 27322 (0.0010) [2023-10-14 18:44:00,190][61585] Updated weights for policy 1, policy_version 27210 (0.0007) [2023-10-14 18:44:00,558][61585] Updated weights for policy 1, policy_version 27220 (0.0008) [2023-10-14 18:44:00,914][61585] Updated weights for policy 1, policy_version 27230 (0.0012) [2023-10-14 18:44:02,929][61552] Updated weights for policy 0, policy_version 27332 (0.0011) [2023-10-14 18:44:03,317][61552] Updated weights for policy 0, policy_version 27342 (0.0010) [2023-10-14 18:44:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 55869440. Throughput: 0: 1670.8, 1: 1651.5. Samples: 13974650. Policy #0 lag: (min: 3.0, avg: 5.8, max: 35.0) [2023-10-14 18:44:03,344][60425] Avg episode reward: [(0, '57.540'), (1, '54.550')] [2023-10-14 18:44:03,681][61552] Updated weights for policy 0, policy_version 27352 (0.0009) [2023-10-14 18:44:05,006][61585] Updated weights for policy 1, policy_version 27240 (0.0008) [2023-10-14 18:44:05,378][61585] Updated weights for policy 1, policy_version 27250 (0.0012) [2023-10-14 18:44:05,753][61585] Updated weights for policy 1, policy_version 27260 (0.0008) [2023-10-14 18:44:07,641][61552] Updated weights for policy 0, policy_version 27362 (0.0011) [2023-10-14 18:44:08,019][61552] Updated weights for policy 0, policy_version 27372 (0.0009) [2023-10-14 18:44:08,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 55934976. Throughput: 0: 1671.0, 1: 1668.9. Samples: 13994682. Policy #0 lag: (min: 3.0, avg: 5.8, max: 35.0) [2023-10-14 18:44:08,344][60425] Avg episode reward: [(0, '54.920'), (1, '55.390')] [2023-10-14 18:44:08,382][61552] Updated weights for policy 0, policy_version 27382 (0.0008) [2023-10-14 18:44:08,755][61552] Updated weights for policy 0, policy_version 27392 (0.0008) [2023-10-14 18:44:09,955][61585] Updated weights for policy 1, policy_version 27270 (0.0009) [2023-10-14 18:44:10,324][61585] Updated weights for policy 1, policy_version 27280 (0.0008) [2023-10-14 18:44:10,679][61585] Updated weights for policy 1, policy_version 27290 (0.0009) [2023-10-14 18:44:12,784][61552] Updated weights for policy 0, policy_version 27402 (0.0008) [2023-10-14 18:44:13,162][61552] Updated weights for policy 0, policy_version 27412 (0.0010) [2023-10-14 18:44:13,344][60425] Fps is (10 sec: 13106.7, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 56000512. Throughput: 0: 1675.6, 1: 1673.1. Samples: 14015286. Policy #0 lag: (min: 3.0, avg: 5.8, max: 35.0) [2023-10-14 18:44:13,345][60425] Avg episode reward: [(0, '58.230'), (1, '56.100')] [2023-10-14 18:44:13,526][61552] Updated weights for policy 0, policy_version 27422 (0.0009) [2023-10-14 18:44:14,722][61585] Updated weights for policy 1, policy_version 27300 (0.0009) [2023-10-14 18:44:15,082][61585] Updated weights for policy 1, policy_version 27310 (0.0011) [2023-10-14 18:44:15,453][61585] Updated weights for policy 1, policy_version 27320 (0.0007) [2023-10-14 18:44:17,610][61552] Updated weights for policy 0, policy_version 27432 (0.0010) [2023-10-14 18:44:17,983][61552] Updated weights for policy 0, policy_version 27442 (0.0010) [2023-10-14 18:44:18,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 56066048. Throughput: 0: 1681.8, 1: 1650.9. Samples: 14024646. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:44:18,344][60425] Avg episode reward: [(0, '59.030'), (1, '56.110')] [2023-10-14 18:44:18,351][61552] Updated weights for policy 0, policy_version 27452 (0.0010) [2023-10-14 18:44:19,772][61585] Updated weights for policy 1, policy_version 27330 (0.0010) [2023-10-14 18:44:20,143][61585] Updated weights for policy 1, policy_version 27340 (0.0010) [2023-10-14 18:44:20,499][61585] Updated weights for policy 1, policy_version 27350 (0.0008) [2023-10-14 18:44:20,863][61585] Updated weights for policy 1, policy_version 27360 (0.0008) [2023-10-14 18:44:22,493][61552] Updated weights for policy 0, policy_version 27462 (0.0008) [2023-10-14 18:44:22,868][61552] Updated weights for policy 0, policy_version 27472 (0.0007) [2023-10-14 18:44:23,237][61552] Updated weights for policy 0, policy_version 27482 (0.0008) [2023-10-14 18:44:23,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 56131584. Throughput: 0: 1678.6, 1: 1668.1. Samples: 14044804. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:44:23,344][60425] Avg episode reward: [(0, '56.070'), (1, '57.140')] [2023-10-14 18:44:24,996][61585] Updated weights for policy 1, policy_version 27370 (0.0008) [2023-10-14 18:44:25,365][61585] Updated weights for policy 1, policy_version 27380 (0.0008) [2023-10-14 18:44:25,738][61585] Updated weights for policy 1, policy_version 27390 (0.0009) [2023-10-14 18:44:27,293][61552] Updated weights for policy 0, policy_version 27492 (0.0007) [2023-10-14 18:44:27,661][61552] Updated weights for policy 0, policy_version 27502 (0.0009) [2023-10-14 18:44:28,029][61552] Updated weights for policy 0, policy_version 27512 (0.0011) [2023-10-14 18:44:28,343][60425] Fps is (10 sec: 16383.9, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 56229888. Throughput: 0: 1663.4, 1: 1664.0. Samples: 14064722. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:44:28,344][60425] Avg episode reward: [(0, '58.800'), (1, '55.390')] [2023-10-14 18:44:29,849][61585] Updated weights for policy 1, policy_version 27400 (0.0008) [2023-10-14 18:44:30,220][61585] Updated weights for policy 1, policy_version 27410 (0.0008) [2023-10-14 18:44:30,581][61585] Updated weights for policy 1, policy_version 27420 (0.0009) [2023-10-14 18:44:32,137][61552] Updated weights for policy 0, policy_version 27522 (0.0008) [2023-10-14 18:44:32,498][61552] Updated weights for policy 0, policy_version 27532 (0.0008) [2023-10-14 18:44:32,875][61552] Updated weights for policy 0, policy_version 27542 (0.0009) [2023-10-14 18:44:33,246][61552] Updated weights for policy 0, policy_version 27552 (0.0008) [2023-10-14 18:44:33,343][60425] Fps is (10 sec: 16384.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 56295424. Throughput: 0: 1678.4, 1: 1650.5. Samples: 14074428. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:44:33,344][60425] Avg episode reward: [(0, '61.290'), (1, '52.640')] [2023-10-14 18:44:34,635][61585] Updated weights for policy 1, policy_version 27430 (0.0009) [2023-10-14 18:44:35,011][61585] Updated weights for policy 1, policy_version 27440 (0.0009) [2023-10-14 18:44:35,378][61585] Updated weights for policy 1, policy_version 27450 (0.0009) [2023-10-14 18:44:37,338][61552] Updated weights for policy 0, policy_version 27562 (0.0008) [2023-10-14 18:44:37,710][61552] Updated weights for policy 0, policy_version 27572 (0.0008) [2023-10-14 18:44:38,081][61552] Updated weights for policy 0, policy_version 27582 (0.0008) [2023-10-14 18:44:38,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.2, 300 sec: 13329.3). Total num frames: 56360960. Throughput: 0: 1673.1, 1: 1664.5. Samples: 14094594. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:44:38,345][60425] Avg episode reward: [(0, '58.970'), (1, '57.050')] [2023-10-14 18:44:39,522][61585] Updated weights for policy 1, policy_version 27460 (0.0008) [2023-10-14 18:44:39,891][61585] Updated weights for policy 1, policy_version 27470 (0.0009) [2023-10-14 18:44:40,266][61585] Updated weights for policy 1, policy_version 27480 (0.0009) [2023-10-14 18:44:42,078][61552] Updated weights for policy 0, policy_version 27592 (0.0010) [2023-10-14 18:44:42,449][61552] Updated weights for policy 0, policy_version 27602 (0.0009) [2023-10-14 18:44:42,822][61552] Updated weights for policy 0, policy_version 27612 (0.0009) [2023-10-14 18:44:43,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 56426496. Throughput: 0: 1653.5, 1: 1660.8. Samples: 14114378. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:44:43,345][60425] Avg episode reward: [(0, '58.900'), (1, '55.680')] [2023-10-14 18:44:44,360][61585] Updated weights for policy 1, policy_version 27490 (0.0008) [2023-10-14 18:44:44,729][61585] Updated weights for policy 1, policy_version 27500 (0.0011) [2023-10-14 18:44:45,093][61585] Updated weights for policy 1, policy_version 27510 (0.0011) [2023-10-14 18:44:45,463][61585] Updated weights for policy 1, policy_version 27520 (0.0010) [2023-10-14 18:44:47,034][61552] Updated weights for policy 0, policy_version 27622 (0.0007) [2023-10-14 18:44:47,412][61552] Updated weights for policy 0, policy_version 27632 (0.0007) [2023-10-14 18:44:47,791][61552] Updated weights for policy 0, policy_version 27642 (0.0008) [2023-10-14 18:44:48,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 56492032. Throughput: 0: 1676.4, 1: 1649.7. Samples: 14124324. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:44:48,344][60425] Avg episode reward: [(0, '61.680'), (1, '55.260')] [2023-10-14 18:44:49,699][61585] Updated weights for policy 1, policy_version 27530 (0.0008) [2023-10-14 18:44:50,063][61585] Updated weights for policy 1, policy_version 27540 (0.0010) [2023-10-14 18:44:50,425][61585] Updated weights for policy 1, policy_version 27550 (0.0008) [2023-10-14 18:44:51,987][61552] Updated weights for policy 0, policy_version 27652 (0.0009) [2023-10-14 18:44:52,371][61552] Updated weights for policy 0, policy_version 27662 (0.0007) [2023-10-14 18:44:52,738][61552] Updated weights for policy 0, policy_version 27672 (0.0007) [2023-10-14 18:44:53,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 56557568. Throughput: 0: 1677.7, 1: 1656.6. Samples: 14144726. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-10-14 18:44:53,344][60425] Avg episode reward: [(0, '56.670'), (1, '55.460')] [2023-10-14 18:44:54,770][61585] Updated weights for policy 1, policy_version 27560 (0.0008) [2023-10-14 18:44:55,139][61585] Updated weights for policy 1, policy_version 27570 (0.0008) [2023-10-14 18:44:55,502][61585] Updated weights for policy 1, policy_version 27580 (0.0009) [2023-10-14 18:44:56,581][61552] Updated weights for policy 0, policy_version 27682 (0.0008) [2023-10-14 18:44:56,960][61552] Updated weights for policy 0, policy_version 27692 (0.0007) [2023-10-14 18:44:57,328][61552] Updated weights for policy 0, policy_version 27702 (0.0010) [2023-10-14 18:44:57,700][61552] Updated weights for policy 0, policy_version 27712 (0.0010) [2023-10-14 18:44:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 56623104. Throughput: 0: 1650.0, 1: 1652.1. Samples: 14163882. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-10-14 18:44:58,344][60425] Avg episode reward: [(0, '56.830'), (1, '56.990')] [2023-10-14 18:44:59,711][61585] Updated weights for policy 1, policy_version 27590 (0.0010) [2023-10-14 18:45:00,076][61585] Updated weights for policy 1, policy_version 27600 (0.0009) [2023-10-14 18:45:00,442][61585] Updated weights for policy 1, policy_version 27610 (0.0008) [2023-10-14 18:45:01,984][61552] Updated weights for policy 0, policy_version 27722 (0.0009) [2023-10-14 18:45:02,351][61552] Updated weights for policy 0, policy_version 27732 (0.0008) [2023-10-14 18:45:02,711][61552] Updated weights for policy 0, policy_version 27742 (0.0008) [2023-10-14 18:45:03,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 56688640. Throughput: 0: 1668.3, 1: 1649.2. Samples: 14173932. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-10-14 18:45:03,344][60425] Avg episode reward: [(0, '59.860'), (1, '58.560')] [2023-10-14 18:45:04,689][61585] Updated weights for policy 1, policy_version 27620 (0.0010) [2023-10-14 18:45:05,070][61585] Updated weights for policy 1, policy_version 27630 (0.0009) [2023-10-14 18:45:05,425][61585] Updated weights for policy 1, policy_version 27640 (0.0009) [2023-10-14 18:45:06,740][61552] Updated weights for policy 0, policy_version 27752 (0.0010) [2023-10-14 18:45:07,117][61552] Updated weights for policy 0, policy_version 27762 (0.0008) [2023-10-14 18:45:07,499][61552] Updated weights for policy 0, policy_version 27772 (0.0010) [2023-10-14 18:45:08,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 56754176. Throughput: 0: 1669.7, 1: 1653.6. Samples: 14194354. Policy #0 lag: (min: 20.0, avg: 28.0, max: 52.0) [2023-10-14 18:45:08,345][60425] Avg episode reward: [(0, '58.250'), (1, '56.690')] [2023-10-14 18:45:09,641][61585] Updated weights for policy 1, policy_version 27650 (0.0008) [2023-10-14 18:45:10,000][61585] Updated weights for policy 1, policy_version 27660 (0.0007) [2023-10-14 18:45:10,373][61585] Updated weights for policy 1, policy_version 27670 (0.0007) [2023-10-14 18:45:10,734][61585] Updated weights for policy 1, policy_version 27680 (0.0007) [2023-10-14 18:45:11,591][61552] Updated weights for policy 0, policy_version 27782 (0.0010) [2023-10-14 18:45:11,969][61552] Updated weights for policy 0, policy_version 27792 (0.0008) [2023-10-14 18:45:12,333][61552] Updated weights for policy 0, policy_version 27802 (0.0009) [2023-10-14 18:45:13,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 56819712. Throughput: 0: 1659.0, 1: 1657.1. Samples: 14213948. Policy #0 lag: (min: 9.0, avg: 25.2, max: 41.0) [2023-10-14 18:45:13,344][60425] Avg episode reward: [(0, '53.840'), (1, '54.860')] [2023-10-14 18:45:14,706][61585] Updated weights for policy 1, policy_version 27690 (0.0010) [2023-10-14 18:45:15,072][61585] Updated weights for policy 1, policy_version 27700 (0.0010) [2023-10-14 18:45:15,432][61585] Updated weights for policy 1, policy_version 27710 (0.0011) [2023-10-14 18:45:16,328][61552] Updated weights for policy 0, policy_version 27812 (0.0008) [2023-10-14 18:45:16,699][61552] Updated weights for policy 0, policy_version 27822 (0.0008) [2023-10-14 18:45:17,070][61552] Updated weights for policy 0, policy_version 27832 (0.0008) [2023-10-14 18:45:18,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 56885248. Throughput: 0: 1674.5, 1: 1654.0. Samples: 14224212. Policy #0 lag: (min: 9.0, avg: 25.2, max: 41.0) [2023-10-14 18:45:18,344][60425] Avg episode reward: [(0, '57.570'), (1, '56.950')] [2023-10-14 18:45:19,564][61585] Updated weights for policy 1, policy_version 27720 (0.0010) [2023-10-14 18:45:19,927][61585] Updated weights for policy 1, policy_version 27730 (0.0009) [2023-10-14 18:45:20,298][61585] Updated weights for policy 1, policy_version 27740 (0.0010) [2023-10-14 18:45:21,156][61552] Updated weights for policy 0, policy_version 27842 (0.0007) [2023-10-14 18:45:21,525][61552] Updated weights for policy 0, policy_version 27852 (0.0009) [2023-10-14 18:45:21,892][61552] Updated weights for policy 0, policy_version 27862 (0.0008) [2023-10-14 18:45:22,264][61552] Updated weights for policy 0, policy_version 27872 (0.0009) [2023-10-14 18:45:23,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 56950784. Throughput: 0: 1659.8, 1: 1656.9. Samples: 14243846. Policy #0 lag: (min: 9.0, avg: 25.2, max: 41.0) [2023-10-14 18:45:23,344][60425] Avg episode reward: [(0, '56.430'), (1, '53.430')] [2023-10-14 18:45:24,473][61585] Updated weights for policy 1, policy_version 27750 (0.0008) [2023-10-14 18:45:24,831][61585] Updated weights for policy 1, policy_version 27760 (0.0008) [2023-10-14 18:45:25,195][61585] Updated weights for policy 1, policy_version 27770 (0.0007) [2023-10-14 18:45:26,189][61552] Updated weights for policy 0, policy_version 27882 (0.0008) [2023-10-14 18:45:26,562][61552] Updated weights for policy 0, policy_version 27892 (0.0008) [2023-10-14 18:45:26,937][61552] Updated weights for policy 0, policy_version 27902 (0.0009) [2023-10-14 18:45:28,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 57016320. Throughput: 0: 1670.1, 1: 1654.4. Samples: 14263984. Policy #0 lag: (min: 9.0, avg: 25.2, max: 41.0) [2023-10-14 18:45:28,344][60425] Avg episode reward: [(0, '57.640'), (1, '55.070')] [2023-10-14 18:45:29,288][61585] Updated weights for policy 1, policy_version 27780 (0.0008) [2023-10-14 18:45:29,663][61585] Updated weights for policy 1, policy_version 27790 (0.0010) [2023-10-14 18:45:30,016][61585] Updated weights for policy 1, policy_version 27800 (0.0010) [2023-10-14 18:45:30,917][61552] Updated weights for policy 0, policy_version 27912 (0.0009) [2023-10-14 18:45:31,285][61552] Updated weights for policy 0, policy_version 27922 (0.0008) [2023-10-14 18:45:31,654][61552] Updated weights for policy 0, policy_version 27932 (0.0009) [2023-10-14 18:45:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 57081856. Throughput: 0: 1676.4, 1: 1654.8. Samples: 14274230. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-14 18:45:33,344][60425] Avg episode reward: [(0, '59.330'), (1, '54.960')] [2023-10-14 18:45:34,226][61585] Updated weights for policy 1, policy_version 27810 (0.0009) [2023-10-14 18:45:34,593][61585] Updated weights for policy 1, policy_version 27820 (0.0012) [2023-10-14 18:45:34,950][61585] Updated weights for policy 1, policy_version 27830 (0.0009) [2023-10-14 18:45:35,317][61585] Updated weights for policy 1, policy_version 27840 (0.0008) [2023-10-14 18:45:35,885][61552] Updated weights for policy 0, policy_version 27942 (0.0009) [2023-10-14 18:45:36,258][61552] Updated weights for policy 0, policy_version 27952 (0.0008) [2023-10-14 18:45:36,625][61552] Updated weights for policy 0, policy_version 27962 (0.0008) [2023-10-14 18:45:38,344][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 57147392. Throughput: 0: 1653.4, 1: 1653.9. Samples: 14293552. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-14 18:45:38,345][60425] Avg episode reward: [(0, '57.410'), (1, '54.370')] [2023-10-14 18:45:39,563][61585] Updated weights for policy 1, policy_version 27850 (0.0008) [2023-10-14 18:45:39,923][61585] Updated weights for policy 1, policy_version 27860 (0.0010) [2023-10-14 18:45:40,287][61585] Updated weights for policy 1, policy_version 27870 (0.0007) [2023-10-14 18:45:40,931][61552] Updated weights for policy 0, policy_version 27972 (0.0008) [2023-10-14 18:45:41,315][61552] Updated weights for policy 0, policy_version 27982 (0.0008) [2023-10-14 18:45:41,681][61552] Updated weights for policy 0, policy_version 27992 (0.0010) [2023-10-14 18:45:43,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 57212928. Throughput: 0: 1673.2, 1: 1661.0. Samples: 14313920. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-14 18:45:43,344][60425] Avg episode reward: [(0, '59.280'), (1, '58.180')] [2023-10-14 18:45:43,353][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000027872_28540928.pth... [2023-10-14 18:45:43,353][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000028000_28672000.pth... [2023-10-14 18:45:43,388][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000026336_26968064.pth [2023-10-14 18:45:43,390][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000026432_27066368.pth [2023-10-14 18:45:44,243][61585] Updated weights for policy 1, policy_version 27880 (0.0008) [2023-10-14 18:45:44,611][61585] Updated weights for policy 1, policy_version 27890 (0.0010) [2023-10-14 18:45:44,979][61585] Updated weights for policy 1, policy_version 27900 (0.0007) [2023-10-14 18:45:45,950][61552] Updated weights for policy 0, policy_version 28002 (0.0010) [2023-10-14 18:45:46,320][61552] Updated weights for policy 0, policy_version 28012 (0.0007) [2023-10-14 18:45:46,690][61552] Updated weights for policy 0, policy_version 28022 (0.0008) [2023-10-14 18:45:47,061][61552] Updated weights for policy 0, policy_version 28032 (0.0009) [2023-10-14 18:45:48,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 57278464. Throughput: 0: 1677.7, 1: 1662.8. Samples: 14324252. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-14 18:45:48,344][60425] Avg episode reward: [(0, '58.720'), (1, '55.880')] [2023-10-14 18:45:49,063][61585] Updated weights for policy 1, policy_version 27910 (0.0008) [2023-10-14 18:45:49,432][61585] Updated weights for policy 1, policy_version 27920 (0.0009) [2023-10-14 18:45:49,798][61585] Updated weights for policy 1, policy_version 27930 (0.0009) [2023-10-14 18:45:50,903][61552] Updated weights for policy 0, policy_version 28042 (0.0010) [2023-10-14 18:45:51,264][61552] Updated weights for policy 0, policy_version 28052 (0.0010) [2023-10-14 18:45:51,631][61552] Updated weights for policy 0, policy_version 28062 (0.0010) [2023-10-14 18:45:53,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 57344000. Throughput: 0: 1650.9, 1: 1663.3. Samples: 14343492. Policy #0 lag: (min: 14.0, avg: 21.7, max: 46.0) [2023-10-14 18:45:53,344][60425] Avg episode reward: [(0, '58.010'), (1, '58.320')] [2023-10-14 18:45:53,782][61585] Updated weights for policy 1, policy_version 27940 (0.0010) [2023-10-14 18:45:54,158][61585] Updated weights for policy 1, policy_version 27950 (0.0009) [2023-10-14 18:45:54,530][61585] Updated weights for policy 1, policy_version 27960 (0.0009) [2023-10-14 18:45:55,619][61552] Updated weights for policy 0, policy_version 28072 (0.0007) [2023-10-14 18:45:55,983][61552] Updated weights for policy 0, policy_version 28082 (0.0007) [2023-10-14 18:45:56,350][61552] Updated weights for policy 0, policy_version 28092 (0.0009) [2023-10-14 18:45:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 57409536. Throughput: 0: 1676.0, 1: 1663.6. Samples: 14364230. Policy #0 lag: (min: 14.0, avg: 21.7, max: 46.0) [2023-10-14 18:45:58,344][60425] Avg episode reward: [(0, '59.680'), (1, '56.340')] [2023-10-14 18:45:58,646][61585] Updated weights for policy 1, policy_version 27970 (0.0007) [2023-10-14 18:45:59,015][61585] Updated weights for policy 1, policy_version 27980 (0.0009) [2023-10-14 18:45:59,379][61585] Updated weights for policy 1, policy_version 27990 (0.0009) [2023-10-14 18:45:59,744][61585] Updated weights for policy 1, policy_version 28000 (0.0008) [2023-10-14 18:46:00,353][61552] Updated weights for policy 0, policy_version 28102 (0.0008) [2023-10-14 18:46:00,730][61552] Updated weights for policy 0, policy_version 28112 (0.0008) [2023-10-14 18:46:01,103][61552] Updated weights for policy 0, policy_version 28122 (0.0010) [2023-10-14 18:46:03,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 57475072. Throughput: 0: 1664.1, 1: 1665.5. Samples: 14374044. Policy #0 lag: (min: 14.0, avg: 21.7, max: 46.0) [2023-10-14 18:46:03,344][60425] Avg episode reward: [(0, '59.230'), (1, '58.430')] [2023-10-14 18:46:03,867][61585] Updated weights for policy 1, policy_version 28010 (0.0011) [2023-10-14 18:46:04,232][61585] Updated weights for policy 1, policy_version 28020 (0.0009) [2023-10-14 18:46:04,590][61585] Updated weights for policy 1, policy_version 28030 (0.0008) [2023-10-14 18:46:05,186][61552] Updated weights for policy 0, policy_version 28132 (0.0009) [2023-10-14 18:46:05,558][61552] Updated weights for policy 0, policy_version 28142 (0.0010) [2023-10-14 18:46:05,928][61552] Updated weights for policy 0, policy_version 28152 (0.0007) [2023-10-14 18:46:08,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 57540608. Throughput: 0: 1665.9, 1: 1667.5. Samples: 14393848. Policy #0 lag: (min: 14.0, avg: 21.7, max: 46.0) [2023-10-14 18:46:08,344][60425] Avg episode reward: [(0, '59.830'), (1, '59.160')] [2023-10-14 18:46:08,732][61585] Updated weights for policy 1, policy_version 28040 (0.0012) [2023-10-14 18:46:09,096][61585] Updated weights for policy 1, policy_version 28050 (0.0010) [2023-10-14 18:46:09,468][61585] Updated weights for policy 1, policy_version 28060 (0.0008) [2023-10-14 18:46:09,991][61552] Updated weights for policy 0, policy_version 28162 (0.0009) [2023-10-14 18:46:10,361][61552] Updated weights for policy 0, policy_version 28172 (0.0009) [2023-10-14 18:46:10,725][61552] Updated weights for policy 0, policy_version 28182 (0.0009) [2023-10-14 18:46:11,096][61552] Updated weights for policy 0, policy_version 28192 (0.0011) [2023-10-14 18:46:13,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 57606144. Throughput: 0: 1674.5, 1: 1670.5. Samples: 14414512. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-14 18:46:13,344][60425] Avg episode reward: [(0, '60.700'), (1, '54.700')] [2023-10-14 18:46:13,580][61585] Updated weights for policy 1, policy_version 28070 (0.0009) [2023-10-14 18:46:13,941][61585] Updated weights for policy 1, policy_version 28080 (0.0009) [2023-10-14 18:46:14,304][61585] Updated weights for policy 1, policy_version 28090 (0.0008) [2023-10-14 18:46:15,235][61552] Updated weights for policy 0, policy_version 28202 (0.0007) [2023-10-14 18:46:15,608][61552] Updated weights for policy 0, policy_version 28212 (0.0010) [2023-10-14 18:46:15,977][61552] Updated weights for policy 0, policy_version 28222 (0.0008) [2023-10-14 18:46:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 57671680. Throughput: 0: 1657.2, 1: 1668.6. Samples: 14423890. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-14 18:46:18,344][60425] Avg episode reward: [(0, '60.090'), (1, '57.460')] [2023-10-14 18:46:18,587][61585] Updated weights for policy 1, policy_version 28100 (0.0009) [2023-10-14 18:46:18,950][61585] Updated weights for policy 1, policy_version 28110 (0.0007) [2023-10-14 18:46:19,318][61585] Updated weights for policy 1, policy_version 28120 (0.0007) [2023-10-14 18:46:20,176][61552] Updated weights for policy 0, policy_version 28232 (0.0009) [2023-10-14 18:46:20,549][61552] Updated weights for policy 0, policy_version 28242 (0.0010) [2023-10-14 18:46:20,902][61552] Updated weights for policy 0, policy_version 28252 (0.0009) [2023-10-14 18:46:23,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 57737216. Throughput: 0: 1671.0, 1: 1667.8. Samples: 14443798. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-14 18:46:23,344][60425] Avg episode reward: [(0, '59.390'), (1, '55.080')] [2023-10-14 18:46:23,423][61585] Updated weights for policy 1, policy_version 28130 (0.0007) [2023-10-14 18:46:23,799][61585] Updated weights for policy 1, policy_version 28140 (0.0009) [2023-10-14 18:46:24,161][61585] Updated weights for policy 1, policy_version 28150 (0.0008) [2023-10-14 18:46:24,530][61585] Updated weights for policy 1, policy_version 28160 (0.0007) [2023-10-14 18:46:25,011][61552] Updated weights for policy 0, policy_version 28262 (0.0008) [2023-10-14 18:46:25,379][61552] Updated weights for policy 0, policy_version 28272 (0.0007) [2023-10-14 18:46:25,751][61552] Updated weights for policy 0, policy_version 28282 (0.0009) [2023-10-14 18:46:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 57802752. Throughput: 0: 1680.4, 1: 1661.5. Samples: 14464302. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-14 18:46:28,344][60425] Avg episode reward: [(0, '58.070'), (1, '57.030')] [2023-10-14 18:46:28,783][61585] Updated weights for policy 1, policy_version 28170 (0.0009) [2023-10-14 18:46:29,152][61585] Updated weights for policy 1, policy_version 28180 (0.0008) [2023-10-14 18:46:29,510][61585] Updated weights for policy 1, policy_version 28190 (0.0010) [2023-10-14 18:46:29,942][61552] Updated weights for policy 0, policy_version 28292 (0.0008) [2023-10-14 18:46:30,335][61552] Updated weights for policy 0, policy_version 28302 (0.0008) [2023-10-14 18:46:30,711][61552] Updated weights for policy 0, policy_version 28312 (0.0010) [2023-10-14 18:46:33,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 57868288. Throughput: 0: 1658.3, 1: 1657.3. Samples: 14473456. Policy #0 lag: (min: 7.0, avg: 11.8, max: 39.0) [2023-10-14 18:46:33,344][60425] Avg episode reward: [(0, '62.760'), (1, '58.970')] [2023-10-14 18:46:33,344][61172] Saving new best policy, reward=62.760! [2023-10-14 18:46:33,522][61585] Updated weights for policy 1, policy_version 28200 (0.0008) [2023-10-14 18:46:33,885][61585] Updated weights for policy 1, policy_version 28210 (0.0007) [2023-10-14 18:46:34,249][61585] Updated weights for policy 1, policy_version 28220 (0.0007) [2023-10-14 18:46:34,870][61552] Updated weights for policy 0, policy_version 28322 (0.0009) [2023-10-14 18:46:35,247][61552] Updated weights for policy 0, policy_version 28332 (0.0009) [2023-10-14 18:46:35,618][61552] Updated weights for policy 0, policy_version 28342 (0.0009) [2023-10-14 18:46:35,992][61552] Updated weights for policy 0, policy_version 28352 (0.0008) [2023-10-14 18:46:38,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 57933824. Throughput: 0: 1676.2, 1: 1659.9. Samples: 14493618. Policy #0 lag: (min: 7.0, avg: 11.8, max: 39.0) [2023-10-14 18:46:38,344][60425] Avg episode reward: [(0, '62.000'), (1, '58.920')] [2023-10-14 18:46:38,603][61585] Updated weights for policy 1, policy_version 28230 (0.0008) [2023-10-14 18:46:38,966][61585] Updated weights for policy 1, policy_version 28240 (0.0010) [2023-10-14 18:46:39,328][61585] Updated weights for policy 1, policy_version 28250 (0.0010) [2023-10-14 18:46:40,091][61552] Updated weights for policy 0, policy_version 28362 (0.0009) [2023-10-14 18:46:40,470][61552] Updated weights for policy 0, policy_version 28372 (0.0011) [2023-10-14 18:46:40,849][61552] Updated weights for policy 0, policy_version 28382 (0.0010) [2023-10-14 18:46:43,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 57999360. Throughput: 0: 1675.8, 1: 1654.2. Samples: 14514082. Policy #0 lag: (min: 7.0, avg: 11.8, max: 39.0) [2023-10-14 18:46:43,344][60425] Avg episode reward: [(0, '62.820'), (1, '60.160')] [2023-10-14 18:46:43,351][61172] Saving new best policy, reward=62.820! [2023-10-14 18:46:43,416][61585] Updated weights for policy 1, policy_version 28260 (0.0009) [2023-10-14 18:46:43,780][61585] Updated weights for policy 1, policy_version 28270 (0.0007) [2023-10-14 18:46:44,140][61585] Updated weights for policy 1, policy_version 28280 (0.0010) [2023-10-14 18:46:44,697][61552] Updated weights for policy 0, policy_version 28392 (0.0010) [2023-10-14 18:46:45,063][61552] Updated weights for policy 0, policy_version 28402 (0.0011) [2023-10-14 18:46:45,426][61552] Updated weights for policy 0, policy_version 28412 (0.0011) [2023-10-14 18:46:48,336][61585] Updated weights for policy 1, policy_version 28290 (0.0008) [2023-10-14 18:46:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 58064896. Throughput: 0: 1658.8, 1: 1652.9. Samples: 14523072. Policy #0 lag: (min: 7.0, avg: 11.8, max: 39.0) [2023-10-14 18:46:48,344][60425] Avg episode reward: [(0, '62.650'), (1, '59.910')] [2023-10-14 18:46:48,705][61585] Updated weights for policy 1, policy_version 28300 (0.0012) [2023-10-14 18:46:49,084][61585] Updated weights for policy 1, policy_version 28310 (0.0008) [2023-10-14 18:46:49,445][61585] Updated weights for policy 1, policy_version 28320 (0.0010) [2023-10-14 18:46:49,495][61552] Updated weights for policy 0, policy_version 28422 (0.0009) [2023-10-14 18:46:49,858][61552] Updated weights for policy 0, policy_version 28432 (0.0008) [2023-10-14 18:46:50,236][61552] Updated weights for policy 0, policy_version 28442 (0.0008) [2023-10-14 18:46:53,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 58130432. Throughput: 0: 1676.4, 1: 1652.5. Samples: 14543650. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-14 18:46:53,344][60425] Avg episode reward: [(0, '60.890'), (1, '58.070')] [2023-10-14 18:46:53,593][61585] Updated weights for policy 1, policy_version 28330 (0.0011) [2023-10-14 18:46:53,958][61585] Updated weights for policy 1, policy_version 28340 (0.0010) [2023-10-14 18:46:54,310][61585] Updated weights for policy 1, policy_version 28350 (0.0007) [2023-10-14 18:46:54,385][61552] Updated weights for policy 0, policy_version 28452 (0.0010) [2023-10-14 18:46:54,751][61552] Updated weights for policy 0, policy_version 28462 (0.0009) [2023-10-14 18:46:55,118][61552] Updated weights for policy 0, policy_version 28472 (0.0009) [2023-10-14 18:46:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 58195968. Throughput: 0: 1677.3, 1: 1646.8. Samples: 14564098. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-14 18:46:58,344][60425] Avg episode reward: [(0, '62.700'), (1, '63.150')] [2023-10-14 18:46:58,492][61585] Updated weights for policy 1, policy_version 28360 (0.0007) [2023-10-14 18:46:58,857][61585] Updated weights for policy 1, policy_version 28370 (0.0009) [2023-10-14 18:46:59,061][61552] Updated weights for policy 0, policy_version 28482 (0.0008) [2023-10-14 18:46:59,241][61585] Updated weights for policy 1, policy_version 28380 (0.0009) [2023-10-14 18:46:59,378][61248] Saving new best policy, reward=63.150! [2023-10-14 18:46:59,430][61552] Updated weights for policy 0, policy_version 28492 (0.0007) [2023-10-14 18:46:59,786][61552] Updated weights for policy 0, policy_version 28502 (0.0010) [2023-10-14 18:47:00,153][61552] Updated weights for policy 0, policy_version 28512 (0.0009) [2023-10-14 18:47:03,292][61585] Updated weights for policy 1, policy_version 28390 (0.0008) [2023-10-14 18:47:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 58261504. Throughput: 0: 1668.1, 1: 1647.1. Samples: 14573072. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-14 18:47:03,344][60425] Avg episode reward: [(0, '58.960'), (1, '59.160')] [2023-10-14 18:47:03,652][61585] Updated weights for policy 1, policy_version 28400 (0.0008) [2023-10-14 18:47:04,016][61585] Updated weights for policy 1, policy_version 28410 (0.0008) [2023-10-14 18:47:04,220][61552] Updated weights for policy 0, policy_version 28522 (0.0008) [2023-10-14 18:47:04,583][61552] Updated weights for policy 0, policy_version 28532 (0.0008) [2023-10-14 18:47:04,956][61552] Updated weights for policy 0, policy_version 28542 (0.0008) [2023-10-14 18:47:08,268][61585] Updated weights for policy 1, policy_version 28420 (0.0009) [2023-10-14 18:47:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 58327040. Throughput: 0: 1682.5, 1: 1645.9. Samples: 14593574. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-14 18:47:08,344][60425] Avg episode reward: [(0, '61.230'), (1, '61.360')] [2023-10-14 18:47:08,632][61585] Updated weights for policy 1, policy_version 28430 (0.0009) [2023-10-14 18:47:08,897][61552] Updated weights for policy 0, policy_version 28552 (0.0009) [2023-10-14 18:47:08,995][61585] Updated weights for policy 1, policy_version 28440 (0.0010) [2023-10-14 18:47:09,259][61552] Updated weights for policy 0, policy_version 28562 (0.0008) [2023-10-14 18:47:09,625][61552] Updated weights for policy 0, policy_version 28572 (0.0010) [2023-10-14 18:47:13,184][61585] Updated weights for policy 1, policy_version 28450 (0.0009) [2023-10-14 18:47:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 58392576. Throughput: 0: 1680.2, 1: 1651.6. Samples: 14614232. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-14 18:47:13,344][60425] Avg episode reward: [(0, '59.020'), (1, '58.240')] [2023-10-14 18:47:13,584][61585] Updated weights for policy 1, policy_version 28460 (0.0008) [2023-10-14 18:47:13,753][61552] Updated weights for policy 0, policy_version 28582 (0.0009) [2023-10-14 18:47:13,934][61585] Updated weights for policy 1, policy_version 28470 (0.0008) [2023-10-14 18:47:14,117][61552] Updated weights for policy 0, policy_version 28592 (0.0008) [2023-10-14 18:47:14,302][61585] Updated weights for policy 1, policy_version 28480 (0.0009) [2023-10-14 18:47:14,491][61552] Updated weights for policy 0, policy_version 28602 (0.0010) [2023-10-14 18:47:18,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 58458112. Throughput: 0: 1675.5, 1: 1652.8. Samples: 14623230. Policy #0 lag: (min: 17.0, avg: 25.4, max: 49.0) [2023-10-14 18:47:18,345][60425] Avg episode reward: [(0, '60.340'), (1, '58.080')] [2023-10-14 18:47:18,473][61585] Updated weights for policy 1, policy_version 28490 (0.0008) [2023-10-14 18:47:18,796][61552] Updated weights for policy 0, policy_version 28612 (0.0010) [2023-10-14 18:47:18,834][61585] Updated weights for policy 1, policy_version 28500 (0.0008) [2023-10-14 18:47:19,175][61552] Updated weights for policy 0, policy_version 28622 (0.0008) [2023-10-14 18:47:19,197][61585] Updated weights for policy 1, policy_version 28510 (0.0007) [2023-10-14 18:47:19,532][61552] Updated weights for policy 0, policy_version 28632 (0.0010) [2023-10-14 18:47:23,282][61585] Updated weights for policy 1, policy_version 28520 (0.0007) [2023-10-14 18:47:23,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 58523648. Throughput: 0: 1680.7, 1: 1650.9. Samples: 14643538. Policy #0 lag: (min: 17.0, avg: 25.4, max: 49.0) [2023-10-14 18:47:23,344][60425] Avg episode reward: [(0, '60.880'), (1, '61.240')] [2023-10-14 18:47:23,650][61585] Updated weights for policy 1, policy_version 28530 (0.0008) [2023-10-14 18:47:23,723][61552] Updated weights for policy 0, policy_version 28642 (0.0009) [2023-10-14 18:47:24,007][61585] Updated weights for policy 1, policy_version 28540 (0.0012) [2023-10-14 18:47:24,080][61552] Updated weights for policy 0, policy_version 28652 (0.0009) [2023-10-14 18:47:24,453][61552] Updated weights for policy 0, policy_version 28662 (0.0010) [2023-10-14 18:47:24,823][61552] Updated weights for policy 0, policy_version 28672 (0.0009) [2023-10-14 18:47:27,981][61585] Updated weights for policy 1, policy_version 28550 (0.0008) [2023-10-14 18:47:28,339][61585] Updated weights for policy 1, policy_version 28560 (0.0010) [2023-10-14 18:47:28,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 58589184. Throughput: 0: 1682.7, 1: 1656.0. Samples: 14664324. Policy #0 lag: (min: 17.0, avg: 25.4, max: 49.0) [2023-10-14 18:47:28,344][60425] Avg episode reward: [(0, '63.340'), (1, '58.810')] [2023-10-14 18:47:28,351][61172] Saving new best policy, reward=63.340! [2023-10-14 18:47:28,711][61585] Updated weights for policy 1, policy_version 28570 (0.0008) [2023-10-14 18:47:28,846][61552] Updated weights for policy 0, policy_version 28682 (0.0007) [2023-10-14 18:47:29,208][61552] Updated weights for policy 0, policy_version 28692 (0.0009) [2023-10-14 18:47:29,580][61552] Updated weights for policy 0, policy_version 28702 (0.0010) [2023-10-14 18:47:32,912][61585] Updated weights for policy 1, policy_version 28580 (0.0008) [2023-10-14 18:47:33,271][61585] Updated weights for policy 1, policy_version 28590 (0.0007) [2023-10-14 18:47:33,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 58654720. Throughput: 0: 1687.7, 1: 1654.6. Samples: 14673476. Policy #0 lag: (min: 17.0, avg: 25.4, max: 49.0) [2023-10-14 18:47:33,344][60425] Avg episode reward: [(0, '61.220'), (1, '59.110')] [2023-10-14 18:47:33,651][61585] Updated weights for policy 1, policy_version 28600 (0.0008) [2023-10-14 18:47:33,651][61552] Updated weights for policy 0, policy_version 28712 (0.0009) [2023-10-14 18:47:34,019][61552] Updated weights for policy 0, policy_version 28722 (0.0008) [2023-10-14 18:47:34,390][61552] Updated weights for policy 0, policy_version 28732 (0.0008) [2023-10-14 18:47:37,868][61585] Updated weights for policy 1, policy_version 28610 (0.0007) [2023-10-14 18:47:38,239][61585] Updated weights for policy 1, policy_version 28620 (0.0008) [2023-10-14 18:47:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 58720256. Throughput: 0: 1685.3, 1: 1655.7. Samples: 14693996. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 18:47:38,344][60425] Avg episode reward: [(0, '58.990'), (1, '61.410')] [2023-10-14 18:47:38,477][61552] Updated weights for policy 0, policy_version 28742 (0.0007) [2023-10-14 18:47:38,598][61585] Updated weights for policy 1, policy_version 28630 (0.0008) [2023-10-14 18:47:38,846][61552] Updated weights for policy 0, policy_version 28752 (0.0007) [2023-10-14 18:47:38,963][61585] Updated weights for policy 1, policy_version 28640 (0.0007) [2023-10-14 18:47:39,223][61552] Updated weights for policy 0, policy_version 28762 (0.0007) [2023-10-14 18:47:43,056][61585] Updated weights for policy 1, policy_version 28650 (0.0009) [2023-10-14 18:47:43,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 58785792. Throughput: 0: 1682.0, 1: 1656.3. Samples: 14714318. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 18:47:43,344][60425] Avg episode reward: [(0, '60.250'), (1, '59.110')] [2023-10-14 18:47:43,427][61552] Updated weights for policy 0, policy_version 28772 (0.0007) [2023-10-14 18:47:43,434][61585] Updated weights for policy 1, policy_version 28660 (0.0008) [2023-10-14 18:47:43,788][61552] Updated weights for policy 0, policy_version 28782 (0.0008) [2023-10-14 18:47:43,794][61585] Updated weights for policy 1, policy_version 28670 (0.0008) [2023-10-14 18:47:43,864][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000028672_29360128.pth... [2023-10-14 18:47:43,895][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000027104_27754496.pth [2023-10-14 18:47:44,159][61552] Updated weights for policy 0, policy_version 28792 (0.0007) [2023-10-14 18:47:44,448][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000028800_29491200.pth... [2023-10-14 18:47:44,478][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000027232_27885568.pth [2023-10-14 18:47:47,899][61585] Updated weights for policy 1, policy_version 28680 (0.0010) [2023-10-14 18:47:48,190][61552] Updated weights for policy 0, policy_version 28802 (0.0008) [2023-10-14 18:47:48,267][61585] Updated weights for policy 1, policy_version 28690 (0.0007) [2023-10-14 18:47:48,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 58851328. Throughput: 0: 1681.1, 1: 1659.4. Samples: 14723394. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 18:47:48,344][60425] Avg episode reward: [(0, '59.600'), (1, '56.410')] [2023-10-14 18:47:48,557][61552] Updated weights for policy 0, policy_version 28812 (0.0009) [2023-10-14 18:47:48,627][61585] Updated weights for policy 1, policy_version 28700 (0.0009) [2023-10-14 18:47:48,931][61552] Updated weights for policy 0, policy_version 28822 (0.0009) [2023-10-14 18:47:49,307][61552] Updated weights for policy 0, policy_version 28832 (0.0010) [2023-10-14 18:47:52,743][61585] Updated weights for policy 1, policy_version 28710 (0.0009) [2023-10-14 18:47:53,108][61585] Updated weights for policy 1, policy_version 28720 (0.0010) [2023-10-14 18:47:53,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 58916864. Throughput: 0: 1678.5, 1: 1663.5. Samples: 14743966. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 18:47:53,344][60425] Avg episode reward: [(0, '62.280'), (1, '58.960')] [2023-10-14 18:47:53,361][61552] Updated weights for policy 0, policy_version 28842 (0.0007) [2023-10-14 18:47:53,473][61585] Updated weights for policy 1, policy_version 28730 (0.0008) [2023-10-14 18:47:53,737][61552] Updated weights for policy 0, policy_version 28852 (0.0008) [2023-10-14 18:47:54,095][61552] Updated weights for policy 0, policy_version 28862 (0.0010) [2023-10-14 18:47:57,571][61585] Updated weights for policy 1, policy_version 28740 (0.0008) [2023-10-14 18:47:57,930][61585] Updated weights for policy 1, policy_version 28750 (0.0009) [2023-10-14 18:47:58,234][61552] Updated weights for policy 0, policy_version 28872 (0.0009) [2023-10-14 18:47:58,302][61585] Updated weights for policy 1, policy_version 28760 (0.0009) [2023-10-14 18:47:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 58982400. Throughput: 0: 1678.4, 1: 1657.2. Samples: 14764332. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 18:47:58,344][60425] Avg episode reward: [(0, '60.410'), (1, '56.150')] [2023-10-14 18:47:58,590][61552] Updated weights for policy 0, policy_version 28882 (0.0009) [2023-10-14 18:47:58,959][61552] Updated weights for policy 0, policy_version 28892 (0.0010) [2023-10-14 18:48:02,490][61585] Updated weights for policy 1, policy_version 28770 (0.0010) [2023-10-14 18:48:02,895][61552] Updated weights for policy 0, policy_version 28902 (0.0008) [2023-10-14 18:48:02,904][61585] Updated weights for policy 1, policy_version 28780 (0.0009) [2023-10-14 18:48:03,276][61585] Updated weights for policy 1, policy_version 28790 (0.0008) [2023-10-14 18:48:03,280][61552] Updated weights for policy 0, policy_version 28912 (0.0008) [2023-10-14 18:48:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 59047936. Throughput: 0: 1678.4, 1: 1668.5. Samples: 14773842. Policy #0 lag: (min: 30.0, avg: 32.7, max: 62.0) [2023-10-14 18:48:03,344][60425] Avg episode reward: [(0, '61.670'), (1, '60.070')] [2023-10-14 18:48:03,635][61585] Updated weights for policy 1, policy_version 28800 (0.0008) [2023-10-14 18:48:03,647][61552] Updated weights for policy 0, policy_version 28922 (0.0007) [2023-10-14 18:48:07,665][61552] Updated weights for policy 0, policy_version 28932 (0.0009) [2023-10-14 18:48:07,760][61585] Updated weights for policy 1, policy_version 28810 (0.0009) [2023-10-14 18:48:08,027][61552] Updated weights for policy 0, policy_version 28942 (0.0009) [2023-10-14 18:48:08,133][61585] Updated weights for policy 1, policy_version 28820 (0.0008) [2023-10-14 18:48:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 59113472. Throughput: 0: 1685.4, 1: 1659.4. Samples: 14794054. Policy #0 lag: (min: 30.0, avg: 32.7, max: 62.0) [2023-10-14 18:48:08,344][60425] Avg episode reward: [(0, '60.880'), (1, '56.650')] [2023-10-14 18:48:08,399][61552] Updated weights for policy 0, policy_version 28952 (0.0008) [2023-10-14 18:48:08,499][61585] Updated weights for policy 1, policy_version 28830 (0.0008) [2023-10-14 18:48:12,488][61585] Updated weights for policy 1, policy_version 28840 (0.0010) [2023-10-14 18:48:12,585][61552] Updated weights for policy 0, policy_version 28962 (0.0010) [2023-10-14 18:48:12,853][61585] Updated weights for policy 1, policy_version 28850 (0.0008) [2023-10-14 18:48:12,962][61552] Updated weights for policy 0, policy_version 28972 (0.0009) [2023-10-14 18:48:13,220][61585] Updated weights for policy 1, policy_version 28860 (0.0008) [2023-10-14 18:48:13,331][61552] Updated weights for policy 0, policy_version 28982 (0.0008) [2023-10-14 18:48:13,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 59179008. Throughput: 0: 1674.6, 1: 1646.6. Samples: 14813778. Policy #0 lag: (min: 30.0, avg: 32.7, max: 62.0) [2023-10-14 18:48:13,344][60425] Avg episode reward: [(0, '63.230'), (1, '56.450')] [2023-10-14 18:48:13,692][61552] Updated weights for policy 0, policy_version 28992 (0.0007) [2023-10-14 18:48:17,436][61585] Updated weights for policy 1, policy_version 28870 (0.0008) [2023-10-14 18:48:17,670][61552] Updated weights for policy 0, policy_version 29002 (0.0009) [2023-10-14 18:48:17,807][61585] Updated weights for policy 1, policy_version 28880 (0.0009) [2023-10-14 18:48:18,032][61552] Updated weights for policy 0, policy_version 29012 (0.0008) [2023-10-14 18:48:18,162][61585] Updated weights for policy 1, policy_version 28890 (0.0008) [2023-10-14 18:48:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 59244544. Throughput: 0: 1676.1, 1: 1660.7. Samples: 14823634. Policy #0 lag: (min: 30.0, avg: 32.7, max: 62.0) [2023-10-14 18:48:18,344][60425] Avg episode reward: [(0, '61.850'), (1, '57.150')] [2023-10-14 18:48:18,410][61552] Updated weights for policy 0, policy_version 29022 (0.0007) [2023-10-14 18:48:22,321][61585] Updated weights for policy 1, policy_version 28900 (0.0008) [2023-10-14 18:48:22,492][61552] Updated weights for policy 0, policy_version 29032 (0.0009) [2023-10-14 18:48:22,687][61585] Updated weights for policy 1, policy_version 28910 (0.0007) [2023-10-14 18:48:22,864][61552] Updated weights for policy 0, policy_version 29042 (0.0009) [2023-10-14 18:48:23,060][61585] Updated weights for policy 1, policy_version 28920 (0.0007) [2023-10-14 18:48:23,242][61552] Updated weights for policy 0, policy_version 29052 (0.0008) [2023-10-14 18:48:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 59310080. Throughput: 0: 1675.2, 1: 1658.4. Samples: 14844008. Policy #0 lag: (min: 30.0, avg: 32.7, max: 62.0) [2023-10-14 18:48:23,344][60425] Avg episode reward: [(0, '62.760'), (1, '56.140')] [2023-10-14 18:48:27,132][61585] Updated weights for policy 1, policy_version 28930 (0.0009) [2023-10-14 18:48:27,436][61552] Updated weights for policy 0, policy_version 29062 (0.0008) [2023-10-14 18:48:27,496][61585] Updated weights for policy 1, policy_version 28940 (0.0008) [2023-10-14 18:48:27,816][61552] Updated weights for policy 0, policy_version 29072 (0.0009) [2023-10-14 18:48:27,868][61585] Updated weights for policy 1, policy_version 28950 (0.0009) [2023-10-14 18:48:28,187][61552] Updated weights for policy 0, policy_version 29082 (0.0008) [2023-10-14 18:48:28,225][61585] Updated weights for policy 1, policy_version 28960 (0.0008) [2023-10-14 18:48:28,343][60425] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 59408384. Throughput: 0: 1667.4, 1: 1643.3. Samples: 14863300. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-14 18:48:28,345][60425] Avg episode reward: [(0, '65.640'), (1, '55.080')] [2023-10-14 18:48:28,410][61172] Saving new best policy, reward=65.640! [2023-10-14 18:48:32,162][61552] Updated weights for policy 0, policy_version 29092 (0.0007) [2023-10-14 18:48:32,360][61585] Updated weights for policy 1, policy_version 28970 (0.0009) [2023-10-14 18:48:32,539][61552] Updated weights for policy 0, policy_version 29102 (0.0008) [2023-10-14 18:48:32,725][61585] Updated weights for policy 1, policy_version 28980 (0.0008) [2023-10-14 18:48:32,911][61552] Updated weights for policy 0, policy_version 29112 (0.0009) [2023-10-14 18:48:33,085][61585] Updated weights for policy 1, policy_version 28990 (0.0008) [2023-10-14 18:48:33,343][60425] Fps is (10 sec: 19660.6, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 59506688. Throughput: 0: 1678.9, 1: 1658.7. Samples: 14873588. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-14 18:48:33,344][60425] Avg episode reward: [(0, '61.440'), (1, '56.540')] [2023-10-14 18:48:37,064][61552] Updated weights for policy 0, policy_version 29122 (0.0008) [2023-10-14 18:48:37,179][61585] Updated weights for policy 1, policy_version 29000 (0.0009) [2023-10-14 18:48:37,435][61552] Updated weights for policy 0, policy_version 29132 (0.0007) [2023-10-14 18:48:37,542][61585] Updated weights for policy 1, policy_version 29010 (0.0008) [2023-10-14 18:48:37,794][61552] Updated weights for policy 0, policy_version 29142 (0.0009) [2023-10-14 18:48:37,912][61585] Updated weights for policy 1, policy_version 29020 (0.0007) [2023-10-14 18:48:38,164][61552] Updated weights for policy 0, policy_version 29152 (0.0007) [2023-10-14 18:48:38,343][60425] Fps is (10 sec: 16384.5, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 59572224. Throughput: 0: 1680.6, 1: 1659.7. Samples: 14894280. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-14 18:48:38,344][60425] Avg episode reward: [(0, '58.970'), (1, '58.030')] [2023-10-14 18:48:42,250][61552] Updated weights for policy 0, policy_version 29162 (0.0008) [2023-10-14 18:48:42,264][61585] Updated weights for policy 1, policy_version 29030 (0.0007) [2023-10-14 18:48:42,626][61552] Updated weights for policy 0, policy_version 29172 (0.0007) [2023-10-14 18:48:42,627][61585] Updated weights for policy 1, policy_version 29040 (0.0008) [2023-10-14 18:48:42,986][61585] Updated weights for policy 1, policy_version 29050 (0.0008) [2023-10-14 18:48:42,990][61552] Updated weights for policy 0, policy_version 29182 (0.0008) [2023-10-14 18:48:43,343][60425] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 59637760. Throughput: 0: 1661.2, 1: 1647.3. Samples: 14913214. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-14 18:48:43,344][60425] Avg episode reward: [(0, '55.720'), (1, '58.840')] [2023-10-14 18:48:47,072][61552] Updated weights for policy 0, policy_version 29192 (0.0008) [2023-10-14 18:48:47,143][61585] Updated weights for policy 1, policy_version 29060 (0.0008) [2023-10-14 18:48:47,436][61552] Updated weights for policy 0, policy_version 29202 (0.0009) [2023-10-14 18:48:47,504][61585] Updated weights for policy 1, policy_version 29070 (0.0009) [2023-10-14 18:48:47,801][61552] Updated weights for policy 0, policy_version 29212 (0.0008) [2023-10-14 18:48:47,869][61585] Updated weights for policy 1, policy_version 29080 (0.0008) [2023-10-14 18:48:48,343][60425] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 59703296. Throughput: 0: 1679.1, 1: 1652.6. Samples: 14923766. Policy #0 lag: (min: 21.0, avg: 22.5, max: 47.0) [2023-10-14 18:48:48,344][60425] Avg episode reward: [(0, '56.360'), (1, '56.740')] [2023-10-14 18:48:52,041][61552] Updated weights for policy 0, policy_version 29222 (0.0009) [2023-10-14 18:48:52,109][61585] Updated weights for policy 1, policy_version 29090 (0.0008) [2023-10-14 18:48:52,406][61552] Updated weights for policy 0, policy_version 29232 (0.0009) [2023-10-14 18:48:52,502][61585] Updated weights for policy 1, policy_version 29100 (0.0008) [2023-10-14 18:48:52,766][61552] Updated weights for policy 0, policy_version 29242 (0.0007) [2023-10-14 18:48:52,867][61585] Updated weights for policy 1, policy_version 29110 (0.0008) [2023-10-14 18:48:53,232][61585] Updated weights for policy 1, policy_version 29120 (0.0007) [2023-10-14 18:48:53,343][60425] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 59768832. Throughput: 0: 1680.5, 1: 1656.5. Samples: 14944222. Policy #0 lag: (min: 21.0, avg: 22.5, max: 47.0) [2023-10-14 18:48:53,344][60425] Avg episode reward: [(0, '58.730'), (1, '56.690')] [2023-10-14 18:48:56,807][61552] Updated weights for policy 0, policy_version 29252 (0.0010) [2023-10-14 18:48:57,178][61552] Updated weights for policy 0, policy_version 29262 (0.0007) [2023-10-14 18:48:57,264][61585] Updated weights for policy 1, policy_version 29130 (0.0009) [2023-10-14 18:48:57,551][61552] Updated weights for policy 0, policy_version 29272 (0.0008) [2023-10-14 18:48:57,636][61585] Updated weights for policy 1, policy_version 29140 (0.0008) [2023-10-14 18:48:58,005][61585] Updated weights for policy 1, policy_version 29150 (0.0007) [2023-10-14 18:48:58,343][60425] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 59834368. Throughput: 0: 1659.5, 1: 1642.3. Samples: 14962358. Policy #0 lag: (min: 21.0, avg: 22.5, max: 47.0) [2023-10-14 18:48:58,344][60425] Avg episode reward: [(0, '57.440'), (1, '57.290')] [2023-10-14 18:49:01,768][61552] Updated weights for policy 0, policy_version 29282 (0.0008) [2023-10-14 18:49:02,112][61585] Updated weights for policy 1, policy_version 29160 (0.0008) [2023-10-14 18:49:02,135][61552] Updated weights for policy 0, policy_version 29292 (0.0007) [2023-10-14 18:49:02,480][61585] Updated weights for policy 1, policy_version 29170 (0.0009) [2023-10-14 18:49:02,500][61552] Updated weights for policy 0, policy_version 29302 (0.0008) [2023-10-14 18:49:02,838][61585] Updated weights for policy 1, policy_version 29180 (0.0007) [2023-10-14 18:49:02,874][61552] Updated weights for policy 0, policy_version 29312 (0.0009) [2023-10-14 18:49:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 59899904. Throughput: 0: 1673.7, 1: 1649.7. Samples: 14973188. Policy #0 lag: (min: 21.0, avg: 22.5, max: 47.0) [2023-10-14 18:49:03,344][60425] Avg episode reward: [(0, '57.630'), (1, '58.030')] [2023-10-14 18:49:07,064][61552] Updated weights for policy 0, policy_version 29322 (0.0009) [2023-10-14 18:49:07,191][61585] Updated weights for policy 1, policy_version 29190 (0.0009) [2023-10-14 18:49:07,429][61552] Updated weights for policy 0, policy_version 29332 (0.0009) [2023-10-14 18:49:07,542][61585] Updated weights for policy 1, policy_version 29200 (0.0008) [2023-10-14 18:49:07,795][61552] Updated weights for policy 0, policy_version 29342 (0.0007) [2023-10-14 18:49:07,903][61585] Updated weights for policy 1, policy_version 29210 (0.0008) [2023-10-14 18:49:08,343][60425] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 59965440. Throughput: 0: 1668.9, 1: 1651.6. Samples: 14993430. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:49:08,344][60425] Avg episode reward: [(0, '59.780'), (1, '60.120')] [2023-10-14 18:49:11,770][61552] Updated weights for policy 0, policy_version 29352 (0.0008) [2023-10-14 18:49:11,931][61585] Updated weights for policy 1, policy_version 29220 (0.0009) [2023-10-14 18:49:12,146][61552] Updated weights for policy 0, policy_version 29362 (0.0007) [2023-10-14 18:49:12,287][61585] Updated weights for policy 1, policy_version 29230 (0.0008) [2023-10-14 18:49:12,507][61552] Updated weights for policy 0, policy_version 29372 (0.0008) [2023-10-14 18:49:12,652][61585] Updated weights for policy 1, policy_version 29240 (0.0010) [2023-10-14 18:49:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 60030976. Throughput: 0: 1653.8, 1: 1648.0. Samples: 15011878. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:49:13,344][60425] Avg episode reward: [(0, '62.030'), (1, '58.980')] [2023-10-14 18:49:16,479][61552] Updated weights for policy 0, policy_version 29382 (0.0009) [2023-10-14 18:49:16,852][61552] Updated weights for policy 0, policy_version 29392 (0.0010) [2023-10-14 18:49:16,909][61585] Updated weights for policy 1, policy_version 29250 (0.0010) [2023-10-14 18:49:17,213][61552] Updated weights for policy 0, policy_version 29402 (0.0009) [2023-10-14 18:49:17,277][61585] Updated weights for policy 1, policy_version 29260 (0.0007) [2023-10-14 18:49:17,640][61585] Updated weights for policy 1, policy_version 29270 (0.0007) [2023-10-14 18:49:18,006][61585] Updated weights for policy 1, policy_version 29280 (0.0008) [2023-10-14 18:49:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 60096512. Throughput: 0: 1672.1, 1: 1649.9. Samples: 15023080. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:49:18,344][60425] Avg episode reward: [(0, '59.930'), (1, '56.930')] [2023-10-14 18:49:21,294][61552] Updated weights for policy 0, policy_version 29412 (0.0010) [2023-10-14 18:49:21,663][61552] Updated weights for policy 0, policy_version 29422 (0.0007) [2023-10-14 18:49:22,029][61552] Updated weights for policy 0, policy_version 29432 (0.0007) [2023-10-14 18:49:22,261][61585] Updated weights for policy 1, policy_version 29290 (0.0008) [2023-10-14 18:49:22,637][61585] Updated weights for policy 1, policy_version 29300 (0.0009) [2023-10-14 18:49:23,004][61585] Updated weights for policy 1, policy_version 29310 (0.0008) [2023-10-14 18:49:23,343][60425] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 13329.4). Total num frames: 60162048. Throughput: 0: 1657.3, 1: 1643.7. Samples: 15042828. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:49:23,344][60425] Avg episode reward: [(0, '62.590'), (1, '59.030')] [2023-10-14 18:49:25,941][61552] Updated weights for policy 0, policy_version 29442 (0.0007) [2023-10-14 18:49:26,314][61552] Updated weights for policy 0, policy_version 29452 (0.0009) [2023-10-14 18:49:26,681][61552] Updated weights for policy 0, policy_version 29462 (0.0007) [2023-10-14 18:49:27,045][61552] Updated weights for policy 0, policy_version 29472 (0.0007) [2023-10-14 18:49:27,227][61585] Updated weights for policy 1, policy_version 29320 (0.0010) [2023-10-14 18:49:27,592][61585] Updated weights for policy 1, policy_version 29330 (0.0009) [2023-10-14 18:49:27,954][61585] Updated weights for policy 1, policy_version 29340 (0.0008) [2023-10-14 18:49:28,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 60227584. Throughput: 0: 1663.9, 1: 1640.3. Samples: 15061904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:49:28,345][60425] Avg episode reward: [(0, '57.990'), (1, '59.010')] [2023-10-14 18:49:31,141][61552] Updated weights for policy 0, policy_version 29482 (0.0009) [2023-10-14 18:49:31,500][61552] Updated weights for policy 0, policy_version 29492 (0.0010) [2023-10-14 18:49:31,872][61552] Updated weights for policy 0, policy_version 29502 (0.0009) [2023-10-14 18:49:32,012][61585] Updated weights for policy 1, policy_version 29350 (0.0008) [2023-10-14 18:49:32,376][61585] Updated weights for policy 1, policy_version 29360 (0.0010) [2023-10-14 18:49:32,749][61585] Updated weights for policy 1, policy_version 29370 (0.0010) [2023-10-14 18:49:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 60293120. Throughput: 0: 1677.6, 1: 1644.8. Samples: 15073276. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-14 18:49:33,344][60425] Avg episode reward: [(0, '59.370'), (1, '57.500')] [2023-10-14 18:49:35,954][61552] Updated weights for policy 0, policy_version 29512 (0.0010) [2023-10-14 18:49:36,314][61552] Updated weights for policy 0, policy_version 29522 (0.0009) [2023-10-14 18:49:36,683][61552] Updated weights for policy 0, policy_version 29532 (0.0010) [2023-10-14 18:49:37,051][61585] Updated weights for policy 1, policy_version 29380 (0.0009) [2023-10-14 18:49:37,446][61585] Updated weights for policy 1, policy_version 29390 (0.0009) [2023-10-14 18:49:37,805][61585] Updated weights for policy 1, policy_version 29400 (0.0011) [2023-10-14 18:49:38,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 60358656. Throughput: 0: 1649.9, 1: 1649.1. Samples: 15092674. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-14 18:49:38,344][60425] Avg episode reward: [(0, '58.070'), (1, '60.470')] [2023-10-14 18:49:40,800][61552] Updated weights for policy 0, policy_version 29542 (0.0008) [2023-10-14 18:49:41,184][61552] Updated weights for policy 0, policy_version 29552 (0.0007) [2023-10-14 18:49:41,550][61552] Updated weights for policy 0, policy_version 29562 (0.0008) [2023-10-14 18:49:41,989][61585] Updated weights for policy 1, policy_version 29410 (0.0011) [2023-10-14 18:49:42,366][61585] Updated weights for policy 1, policy_version 29420 (0.0009) [2023-10-14 18:49:42,723][61585] Updated weights for policy 1, policy_version 29430 (0.0007) [2023-10-14 18:49:43,086][61585] Updated weights for policy 1, policy_version 29440 (0.0007) [2023-10-14 18:49:43,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 60424192. Throughput: 0: 1674.5, 1: 1647.7. Samples: 15111860. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-14 18:49:43,344][60425] Avg episode reward: [(0, '57.690'), (1, '55.260')] [2023-10-14 18:49:43,352][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000029568_30277632.pth... [2023-10-14 18:49:43,352][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000029440_30146560.pth... [2023-10-14 18:49:43,382][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000028000_28672000.pth [2023-10-14 18:49:43,389][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000027872_28540928.pth [2023-10-14 18:49:45,573][61552] Updated weights for policy 0, policy_version 29572 (0.0008) [2023-10-14 18:49:45,958][61552] Updated weights for policy 0, policy_version 29582 (0.0009) [2023-10-14 18:49:46,321][61552] Updated weights for policy 0, policy_version 29592 (0.0009) [2023-10-14 18:49:47,202][61585] Updated weights for policy 1, policy_version 29450 (0.0008) [2023-10-14 18:49:47,561][61585] Updated weights for policy 1, policy_version 29460 (0.0010) [2023-10-14 18:49:47,924][61585] Updated weights for policy 1, policy_version 29470 (0.0009) [2023-10-14 18:49:48,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 60489728. Throughput: 0: 1676.0, 1: 1647.1. Samples: 15122730. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-14 18:49:48,344][60425] Avg episode reward: [(0, '59.990'), (1, '55.660')] [2023-10-14 18:49:50,412][61552] Updated weights for policy 0, policy_version 29602 (0.0008) [2023-10-14 18:49:50,778][61552] Updated weights for policy 0, policy_version 29612 (0.0008) [2023-10-14 18:49:51,143][61552] Updated weights for policy 0, policy_version 29622 (0.0009) [2023-10-14 18:49:51,514][61552] Updated weights for policy 0, policy_version 29632 (0.0008) [2023-10-14 18:49:52,115][61585] Updated weights for policy 1, policy_version 29480 (0.0010) [2023-10-14 18:49:52,486][61585] Updated weights for policy 1, policy_version 29490 (0.0010) [2023-10-14 18:49:52,853][61585] Updated weights for policy 1, policy_version 29500 (0.0011) [2023-10-14 18:49:53,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 60555264. Throughput: 0: 1658.6, 1: 1649.8. Samples: 15142306. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-14 18:49:53,344][60425] Avg episode reward: [(0, '59.610'), (1, '57.400')] [2023-10-14 18:49:55,599][61552] Updated weights for policy 0, policy_version 29642 (0.0009) [2023-10-14 18:49:55,967][61552] Updated weights for policy 0, policy_version 29652 (0.0007) [2023-10-14 18:49:56,327][61552] Updated weights for policy 0, policy_version 29662 (0.0009) [2023-10-14 18:49:56,899][61585] Updated weights for policy 1, policy_version 29510 (0.0010) [2023-10-14 18:49:57,270][61585] Updated weights for policy 1, policy_version 29520 (0.0008) [2023-10-14 18:49:57,641][61585] Updated weights for policy 1, policy_version 29530 (0.0008) [2023-10-14 18:49:58,344][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 60620800. Throughput: 0: 1685.6, 1: 1648.1. Samples: 15161898. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:49:58,345][60425] Avg episode reward: [(0, '61.890'), (1, '57.100')] [2023-10-14 18:50:00,379][61552] Updated weights for policy 0, policy_version 29672 (0.0008) [2023-10-14 18:50:00,758][61552] Updated weights for policy 0, policy_version 29682 (0.0007) [2023-10-14 18:50:01,124][61552] Updated weights for policy 0, policy_version 29692 (0.0008) [2023-10-14 18:50:01,672][61585] Updated weights for policy 1, policy_version 29540 (0.0010) [2023-10-14 18:50:02,041][61585] Updated weights for policy 1, policy_version 29550 (0.0011) [2023-10-14 18:50:02,400][61585] Updated weights for policy 1, policy_version 29560 (0.0008) [2023-10-14 18:50:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 60686336. Throughput: 0: 1670.5, 1: 1657.1. Samples: 15172822. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:50:03,344][60425] Avg episode reward: [(0, '63.130'), (1, '57.550')] [2023-10-14 18:50:05,216][61552] Updated weights for policy 0, policy_version 29702 (0.0008) [2023-10-14 18:50:05,596][61552] Updated weights for policy 0, policy_version 29712 (0.0007) [2023-10-14 18:50:05,970][61552] Updated weights for policy 0, policy_version 29722 (0.0007) [2023-10-14 18:50:06,505][61585] Updated weights for policy 1, policy_version 29570 (0.0010) [2023-10-14 18:50:06,862][61585] Updated weights for policy 1, policy_version 29580 (0.0010) [2023-10-14 18:50:07,230][61585] Updated weights for policy 1, policy_version 29590 (0.0009) [2023-10-14 18:50:07,589][61585] Updated weights for policy 1, policy_version 29600 (0.0007) [2023-10-14 18:50:08,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 60751872. Throughput: 0: 1671.4, 1: 1652.7. Samples: 15192412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:50:08,344][60425] Avg episode reward: [(0, '58.820'), (1, '58.870')] [2023-10-14 18:50:10,061][61552] Updated weights for policy 0, policy_version 29732 (0.0009) [2023-10-14 18:50:10,439][61552] Updated weights for policy 0, policy_version 29742 (0.0008) [2023-10-14 18:50:10,806][61552] Updated weights for policy 0, policy_version 29752 (0.0009) [2023-10-14 18:50:11,741][61585] Updated weights for policy 1, policy_version 29610 (0.0008) [2023-10-14 18:50:12,099][61585] Updated weights for policy 1, policy_version 29620 (0.0010) [2023-10-14 18:50:12,472][61585] Updated weights for policy 1, policy_version 29630 (0.0011) [2023-10-14 18:50:13,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 60817408. Throughput: 0: 1688.2, 1: 1651.4. Samples: 15212186. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:50:13,344][60425] Avg episode reward: [(0, '63.670'), (1, '55.440')] [2023-10-14 18:50:15,061][61552] Updated weights for policy 0, policy_version 29762 (0.0008) [2023-10-14 18:50:15,429][61552] Updated weights for policy 0, policy_version 29772 (0.0007) [2023-10-14 18:50:15,801][61552] Updated weights for policy 0, policy_version 29782 (0.0008) [2023-10-14 18:50:16,167][61552] Updated weights for policy 0, policy_version 29792 (0.0010) [2023-10-14 18:50:16,759][61585] Updated weights for policy 1, policy_version 29640 (0.0008) [2023-10-14 18:50:17,119][61585] Updated weights for policy 1, policy_version 29650 (0.0010) [2023-10-14 18:50:17,487][61585] Updated weights for policy 1, policy_version 29660 (0.0008) [2023-10-14 18:50:18,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 60882944. Throughput: 0: 1665.3, 1: 1660.5. Samples: 15222934. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:50:18,344][60425] Avg episode reward: [(0, '62.880'), (1, '58.610')] [2023-10-14 18:50:20,045][61552] Updated weights for policy 0, policy_version 29802 (0.0011) [2023-10-14 18:50:20,416][61552] Updated weights for policy 0, policy_version 29812 (0.0007) [2023-10-14 18:50:20,784][61552] Updated weights for policy 0, policy_version 29822 (0.0008) [2023-10-14 18:50:21,471][61585] Updated weights for policy 1, policy_version 29670 (0.0008) [2023-10-14 18:50:21,863][61585] Updated weights for policy 1, policy_version 29680 (0.0007) [2023-10-14 18:50:22,223][61585] Updated weights for policy 1, policy_version 29690 (0.0008) [2023-10-14 18:50:23,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 60948480. Throughput: 0: 1682.0, 1: 1649.8. Samples: 15242608. Policy #0 lag: (min: 1.0, avg: 14.2, max: 33.0) [2023-10-14 18:50:23,344][60425] Avg episode reward: [(0, '60.110'), (1, '58.830')] [2023-10-14 18:50:24,834][61552] Updated weights for policy 0, policy_version 29832 (0.0010) [2023-10-14 18:50:25,212][61552] Updated weights for policy 0, policy_version 29842 (0.0010) [2023-10-14 18:50:25,580][61552] Updated weights for policy 0, policy_version 29852 (0.0008) [2023-10-14 18:50:26,152][61585] Updated weights for policy 1, policy_version 29700 (0.0010) [2023-10-14 18:50:26,516][61585] Updated weights for policy 1, policy_version 29710 (0.0008) [2023-10-14 18:50:26,886][61585] Updated weights for policy 1, policy_version 29720 (0.0009) [2023-10-14 18:50:28,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 61014016. Throughput: 0: 1693.1, 1: 1661.7. Samples: 15262828. Policy #0 lag: (min: 1.0, avg: 14.2, max: 33.0) [2023-10-14 18:50:28,345][60425] Avg episode reward: [(0, '61.230'), (1, '59.870')] [2023-10-14 18:50:29,594][61552] Updated weights for policy 0, policy_version 29862 (0.0009) [2023-10-14 18:50:29,979][61552] Updated weights for policy 0, policy_version 29872 (0.0008) [2023-10-14 18:50:30,349][61552] Updated weights for policy 0, policy_version 29882 (0.0007) [2023-10-14 18:50:30,957][61585] Updated weights for policy 1, policy_version 29730 (0.0010) [2023-10-14 18:50:31,326][61585] Updated weights for policy 1, policy_version 29740 (0.0010) [2023-10-14 18:50:31,686][61585] Updated weights for policy 1, policy_version 29750 (0.0009) [2023-10-14 18:50:32,049][61585] Updated weights for policy 1, policy_version 29760 (0.0008) [2023-10-14 18:50:33,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 61079552. Throughput: 0: 1669.5, 1: 1675.1. Samples: 15273238. Policy #0 lag: (min: 1.0, avg: 14.2, max: 33.0) [2023-10-14 18:50:33,344][60425] Avg episode reward: [(0, '60.980'), (1, '60.590')] [2023-10-14 18:50:34,237][61552] Updated weights for policy 0, policy_version 29892 (0.0009) [2023-10-14 18:50:34,610][61552] Updated weights for policy 0, policy_version 29902 (0.0008) [2023-10-14 18:50:34,979][61552] Updated weights for policy 0, policy_version 29912 (0.0009) [2023-10-14 18:50:36,259][61585] Updated weights for policy 1, policy_version 29770 (0.0008) [2023-10-14 18:50:36,626][61585] Updated weights for policy 1, policy_version 29780 (0.0008) [2023-10-14 18:50:36,986][61585] Updated weights for policy 1, policy_version 29790 (0.0008) [2023-10-14 18:50:38,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 61145088. Throughput: 0: 1697.7, 1: 1651.0. Samples: 15292998. Policy #0 lag: (min: 1.0, avg: 14.2, max: 33.0) [2023-10-14 18:50:38,344][60425] Avg episode reward: [(0, '61.050'), (1, '60.450')] [2023-10-14 18:50:39,041][61552] Updated weights for policy 0, policy_version 29922 (0.0008) [2023-10-14 18:50:39,410][61552] Updated weights for policy 0, policy_version 29932 (0.0011) [2023-10-14 18:50:39,783][61552] Updated weights for policy 0, policy_version 29942 (0.0009) [2023-10-14 18:50:40,146][61552] Updated weights for policy 0, policy_version 29952 (0.0009) [2023-10-14 18:50:41,096][61585] Updated weights for policy 1, policy_version 29800 (0.0009) [2023-10-14 18:50:41,466][61585] Updated weights for policy 1, policy_version 29810 (0.0009) [2023-10-14 18:50:41,818][61585] Updated weights for policy 1, policy_version 29820 (0.0008) [2023-10-14 18:50:43,344][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 61210624. Throughput: 0: 1697.5, 1: 1665.4. Samples: 15313228. Policy #0 lag: (min: 1.0, avg: 14.2, max: 33.0) [2023-10-14 18:50:43,345][60425] Avg episode reward: [(0, '61.930'), (1, '60.460')] [2023-10-14 18:50:44,213][61552] Updated weights for policy 0, policy_version 29962 (0.0009) [2023-10-14 18:50:44,583][61552] Updated weights for policy 0, policy_version 29972 (0.0008) [2023-10-14 18:50:44,949][61552] Updated weights for policy 0, policy_version 29982 (0.0009) [2023-10-14 18:50:45,944][61585] Updated weights for policy 1, policy_version 29830 (0.0009) [2023-10-14 18:50:46,314][61585] Updated weights for policy 1, policy_version 29840 (0.0008) [2023-10-14 18:50:46,675][61585] Updated weights for policy 1, policy_version 29850 (0.0009) [2023-10-14 18:50:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 61276160. Throughput: 0: 1679.8, 1: 1667.2. Samples: 15323438. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-14 18:50:48,344][60425] Avg episode reward: [(0, '63.390'), (1, '58.830')] [2023-10-14 18:50:49,169][61552] Updated weights for policy 0, policy_version 29992 (0.0008) [2023-10-14 18:50:49,548][61552] Updated weights for policy 0, policy_version 30002 (0.0008) [2023-10-14 18:50:49,909][61552] Updated weights for policy 0, policy_version 30012 (0.0008) [2023-10-14 18:50:50,737][61585] Updated weights for policy 1, policy_version 29860 (0.0009) [2023-10-14 18:50:51,096][61585] Updated weights for policy 1, policy_version 29870 (0.0008) [2023-10-14 18:50:51,466][61585] Updated weights for policy 1, policy_version 29880 (0.0008) [2023-10-14 18:50:53,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 61341696. Throughput: 0: 1687.2, 1: 1650.1. Samples: 15342590. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-14 18:50:53,344][60425] Avg episode reward: [(0, '61.080'), (1, '59.090')] [2023-10-14 18:50:53,978][61552] Updated weights for policy 0, policy_version 30022 (0.0010) [2023-10-14 18:50:54,344][61552] Updated weights for policy 0, policy_version 30032 (0.0008) [2023-10-14 18:50:54,719][61552] Updated weights for policy 0, policy_version 30042 (0.0009) [2023-10-14 18:50:55,517][61585] Updated weights for policy 1, policy_version 29890 (0.0008) [2023-10-14 18:50:55,891][61585] Updated weights for policy 1, policy_version 29900 (0.0007) [2023-10-14 18:50:56,250][61585] Updated weights for policy 1, policy_version 29910 (0.0007) [2023-10-14 18:50:56,622][61585] Updated weights for policy 1, policy_version 29920 (0.0008) [2023-10-14 18:50:58,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 61407232. Throughput: 0: 1687.4, 1: 1674.3. Samples: 15363462. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-14 18:50:58,345][60425] Avg episode reward: [(0, '65.210'), (1, '59.980')] [2023-10-14 18:50:58,733][61552] Updated weights for policy 0, policy_version 30052 (0.0008) [2023-10-14 18:50:59,104][61552] Updated weights for policy 0, policy_version 30062 (0.0007) [2023-10-14 18:50:59,472][61552] Updated weights for policy 0, policy_version 30072 (0.0007) [2023-10-14 18:51:00,560][61585] Updated weights for policy 1, policy_version 29930 (0.0010) [2023-10-14 18:51:00,931][61585] Updated weights for policy 1, policy_version 29940 (0.0011) [2023-10-14 18:51:01,302][61585] Updated weights for policy 1, policy_version 29950 (0.0009) [2023-10-14 18:51:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 61472768. Throughput: 0: 1678.8, 1: 1665.5. Samples: 15373430. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-14 18:51:03,344][60425] Avg episode reward: [(0, '62.250'), (1, '52.640')] [2023-10-14 18:51:03,543][61552] Updated weights for policy 0, policy_version 30082 (0.0009) [2023-10-14 18:51:03,923][61552] Updated weights for policy 0, policy_version 30092 (0.0008) [2023-10-14 18:51:04,291][61552] Updated weights for policy 0, policy_version 30102 (0.0007) [2023-10-14 18:51:04,661][61552] Updated weights for policy 0, policy_version 30112 (0.0010) [2023-10-14 18:51:05,503][61585] Updated weights for policy 1, policy_version 29960 (0.0009) [2023-10-14 18:51:05,874][61585] Updated weights for policy 1, policy_version 29970 (0.0007) [2023-10-14 18:51:06,233][61585] Updated weights for policy 1, policy_version 29980 (0.0009) [2023-10-14 18:51:08,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 61538304. Throughput: 0: 1686.2, 1: 1662.0. Samples: 15393280. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-14 18:51:08,344][60425] Avg episode reward: [(0, '62.220'), (1, '53.410')] [2023-10-14 18:51:08,651][61552] Updated weights for policy 0, policy_version 30122 (0.0008) [2023-10-14 18:51:09,015][61552] Updated weights for policy 0, policy_version 30132 (0.0007) [2023-10-14 18:51:09,387][61552] Updated weights for policy 0, policy_version 30142 (0.0009) [2023-10-14 18:51:10,450][61585] Updated weights for policy 1, policy_version 29990 (0.0007) [2023-10-14 18:51:10,823][61585] Updated weights for policy 1, policy_version 30000 (0.0007) [2023-10-14 18:51:11,203][61585] Updated weights for policy 1, policy_version 30010 (0.0007) [2023-10-14 18:51:13,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 61603840. Throughput: 0: 1682.3, 1: 1674.5. Samples: 15413886. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-14 18:51:13,344][60425] Avg episode reward: [(0, '63.170'), (1, '59.030')] [2023-10-14 18:51:13,378][61552] Updated weights for policy 0, policy_version 30152 (0.0008) [2023-10-14 18:51:13,746][61552] Updated weights for policy 0, policy_version 30162 (0.0008) [2023-10-14 18:51:14,116][61552] Updated weights for policy 0, policy_version 30172 (0.0008) [2023-10-14 18:51:15,135][61585] Updated weights for policy 1, policy_version 30020 (0.0009) [2023-10-14 18:51:15,503][61585] Updated weights for policy 1, policy_version 30030 (0.0009) [2023-10-14 18:51:15,875][61585] Updated weights for policy 1, policy_version 30040 (0.0010) [2023-10-14 18:51:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 61669376. Throughput: 0: 1683.1, 1: 1651.5. Samples: 15423294. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-14 18:51:18,344][60425] Avg episode reward: [(0, '60.540'), (1, '55.690')] [2023-10-14 18:51:18,411][61552] Updated weights for policy 0, policy_version 30182 (0.0010) [2023-10-14 18:51:18,787][61552] Updated weights for policy 0, policy_version 30192 (0.0011) [2023-10-14 18:51:19,160][61552] Updated weights for policy 0, policy_version 30202 (0.0007) [2023-10-14 18:51:19,936][61585] Updated weights for policy 1, policy_version 30050 (0.0009) [2023-10-14 18:51:20,302][61585] Updated weights for policy 1, policy_version 30060 (0.0011) [2023-10-14 18:51:20,671][61585] Updated weights for policy 1, policy_version 30070 (0.0008) [2023-10-14 18:51:21,030][61585] Updated weights for policy 1, policy_version 30080 (0.0007) [2023-10-14 18:51:23,087][61552] Updated weights for policy 0, policy_version 30212 (0.0007) [2023-10-14 18:51:23,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 61734912. Throughput: 0: 1676.7, 1: 1667.0. Samples: 15443462. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-14 18:51:23,344][60425] Avg episode reward: [(0, '61.850'), (1, '58.530')] [2023-10-14 18:51:23,455][61552] Updated weights for policy 0, policy_version 30222 (0.0010) [2023-10-14 18:51:23,817][61552] Updated weights for policy 0, policy_version 30232 (0.0010) [2023-10-14 18:51:25,149][61585] Updated weights for policy 1, policy_version 30090 (0.0007) [2023-10-14 18:51:25,520][61585] Updated weights for policy 1, policy_version 30100 (0.0009) [2023-10-14 18:51:25,888][61585] Updated weights for policy 1, policy_version 30110 (0.0009) [2023-10-14 18:51:28,103][61552] Updated weights for policy 0, policy_version 30242 (0.0008) [2023-10-14 18:51:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 61800448. Throughput: 0: 1675.7, 1: 1675.3. Samples: 15464022. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-14 18:51:28,344][60425] Avg episode reward: [(0, '61.860'), (1, '58.440')] [2023-10-14 18:51:28,472][61552] Updated weights for policy 0, policy_version 30252 (0.0007) [2023-10-14 18:51:28,852][61552] Updated weights for policy 0, policy_version 30262 (0.0009) [2023-10-14 18:51:29,218][61552] Updated weights for policy 0, policy_version 30272 (0.0008) [2023-10-14 18:51:30,119][61585] Updated weights for policy 1, policy_version 30120 (0.0007) [2023-10-14 18:51:30,493][61585] Updated weights for policy 1, policy_version 30130 (0.0007) [2023-10-14 18:51:30,854][61585] Updated weights for policy 1, policy_version 30140 (0.0008) [2023-10-14 18:51:33,329][61552] Updated weights for policy 0, policy_version 30282 (0.0008) [2023-10-14 18:51:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 61865984. Throughput: 0: 1675.0, 1: 1656.8. Samples: 15473366. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-14 18:51:33,344][60425] Avg episode reward: [(0, '60.440'), (1, '56.300')] [2023-10-14 18:51:33,700][61552] Updated weights for policy 0, policy_version 30292 (0.0009) [2023-10-14 18:51:34,065][61552] Updated weights for policy 0, policy_version 30302 (0.0009) [2023-10-14 18:51:34,930][61585] Updated weights for policy 1, policy_version 30150 (0.0007) [2023-10-14 18:51:35,292][61585] Updated weights for policy 1, policy_version 30160 (0.0008) [2023-10-14 18:51:35,670][61585] Updated weights for policy 1, policy_version 30170 (0.0008) [2023-10-14 18:51:38,050][61552] Updated weights for policy 0, policy_version 30312 (0.0009) [2023-10-14 18:51:38,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 61931520. Throughput: 0: 1682.0, 1: 1673.7. Samples: 15493598. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-14 18:51:38,344][60425] Avg episode reward: [(0, '59.450'), (1, '57.190')] [2023-10-14 18:51:38,414][61552] Updated weights for policy 0, policy_version 30322 (0.0007) [2023-10-14 18:51:38,792][61552] Updated weights for policy 0, policy_version 30332 (0.0008) [2023-10-14 18:51:39,722][61585] Updated weights for policy 1, policy_version 30180 (0.0008) [2023-10-14 18:51:40,085][61585] Updated weights for policy 1, policy_version 30190 (0.0008) [2023-10-14 18:51:40,456][61585] Updated weights for policy 1, policy_version 30200 (0.0008) [2023-10-14 18:51:42,714][61552] Updated weights for policy 0, policy_version 30342 (0.0008) [2023-10-14 18:51:43,084][61552] Updated weights for policy 0, policy_version 30352 (0.0008) [2023-10-14 18:51:43,344][60425] Fps is (10 sec: 13106.7, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 61997056. Throughput: 0: 1674.3, 1: 1671.6. Samples: 15514026. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-14 18:51:43,345][60425] Avg episode reward: [(0, '62.430'), (1, '54.720')] [2023-10-14 18:51:43,354][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000030208_30932992.pth... [2023-10-14 18:51:43,384][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000028672_29360128.pth [2023-10-14 18:51:43,448][61552] Updated weights for policy 0, policy_version 30362 (0.0008) [2023-10-14 18:51:43,670][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000030368_31096832.pth... [2023-10-14 18:51:43,699][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000028800_29491200.pth [2023-10-14 18:51:44,617][61585] Updated weights for policy 1, policy_version 30210 (0.0009) [2023-10-14 18:51:44,976][61585] Updated weights for policy 1, policy_version 30220 (0.0008) [2023-10-14 18:51:45,346][61585] Updated weights for policy 1, policy_version 30230 (0.0009) [2023-10-14 18:51:45,703][61585] Updated weights for policy 1, policy_version 30240 (0.0008) [2023-10-14 18:51:47,625][61552] Updated weights for policy 0, policy_version 30372 (0.0008) [2023-10-14 18:51:47,986][61552] Updated weights for policy 0, policy_version 30382 (0.0008) [2023-10-14 18:51:48,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 62062592. Throughput: 0: 1676.2, 1: 1654.9. Samples: 15523332. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-14 18:51:48,344][60425] Avg episode reward: [(0, '59.990'), (1, '56.860')] [2023-10-14 18:51:48,353][61552] Updated weights for policy 0, policy_version 30392 (0.0009) [2023-10-14 18:51:49,853][61585] Updated weights for policy 1, policy_version 30250 (0.0008) [2023-10-14 18:51:50,223][61585] Updated weights for policy 1, policy_version 30260 (0.0007) [2023-10-14 18:51:50,601][61585] Updated weights for policy 1, policy_version 30270 (0.0010) [2023-10-14 18:51:52,271][61552] Updated weights for policy 0, policy_version 30402 (0.0011) [2023-10-14 18:51:52,653][61552] Updated weights for policy 0, policy_version 30412 (0.0010) [2023-10-14 18:51:53,019][61552] Updated weights for policy 0, policy_version 30422 (0.0007) [2023-10-14 18:51:53,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 62128128. Throughput: 0: 1678.8, 1: 1664.5. Samples: 15543732. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-14 18:51:53,344][60425] Avg episode reward: [(0, '57.030'), (1, '53.730')] [2023-10-14 18:51:53,387][61552] Updated weights for policy 0, policy_version 30432 (0.0008) [2023-10-14 18:51:54,846][61585] Updated weights for policy 1, policy_version 30280 (0.0008) [2023-10-14 18:51:55,210][61585] Updated weights for policy 1, policy_version 30290 (0.0009) [2023-10-14 18:51:55,569][61585] Updated weights for policy 1, policy_version 30300 (0.0009) [2023-10-14 18:51:57,495][61552] Updated weights for policy 0, policy_version 30442 (0.0008) [2023-10-14 18:51:57,864][61552] Updated weights for policy 0, policy_version 30452 (0.0007) [2023-10-14 18:51:58,236][61552] Updated weights for policy 0, policy_version 30462 (0.0009) [2023-10-14 18:51:58,343][60425] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 62226432. Throughput: 0: 1662.3, 1: 1667.9. Samples: 15563744. Policy #0 lag: (min: 12.0, avg: 15.8, max: 44.0) [2023-10-14 18:51:58,344][60425] Avg episode reward: [(0, '60.600'), (1, '54.990')] [2023-10-14 18:51:59,836][61585] Updated weights for policy 1, policy_version 30310 (0.0010) [2023-10-14 18:52:00,233][61585] Updated weights for policy 1, policy_version 30320 (0.0007) [2023-10-14 18:52:00,597][61585] Updated weights for policy 1, policy_version 30330 (0.0007) [2023-10-14 18:52:02,361][61552] Updated weights for policy 0, policy_version 30472 (0.0007) [2023-10-14 18:52:02,726][61552] Updated weights for policy 0, policy_version 30482 (0.0010) [2023-10-14 18:52:03,095][61552] Updated weights for policy 0, policy_version 30492 (0.0007) [2023-10-14 18:52:03,343][60425] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 62291968. Throughput: 0: 1676.5, 1: 1655.8. Samples: 15573250. Policy #0 lag: (min: 12.0, avg: 15.8, max: 44.0) [2023-10-14 18:52:03,344][60425] Avg episode reward: [(0, '61.710'), (1, '59.460')] [2023-10-14 18:52:04,713][61585] Updated weights for policy 1, policy_version 30340 (0.0008) [2023-10-14 18:52:05,081][61585] Updated weights for policy 1, policy_version 30350 (0.0008) [2023-10-14 18:52:05,446][61585] Updated weights for policy 1, policy_version 30360 (0.0009) [2023-10-14 18:52:07,396][61552] Updated weights for policy 0, policy_version 30502 (0.0008) [2023-10-14 18:52:07,770][61552] Updated weights for policy 0, policy_version 30512 (0.0008) [2023-10-14 18:52:08,128][61552] Updated weights for policy 0, policy_version 30522 (0.0007) [2023-10-14 18:52:08,343][60425] Fps is (10 sec: 9830.7, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 62324736. Throughput: 0: 1675.5, 1: 1658.5. Samples: 15593492. Policy #0 lag: (min: 12.0, avg: 15.8, max: 44.0) [2023-10-14 18:52:08,344][60425] Avg episode reward: [(0, '63.010'), (1, '54.000')] [2023-10-14 18:52:09,542][61585] Updated weights for policy 1, policy_version 30370 (0.0009) [2023-10-14 18:52:09,915][61585] Updated weights for policy 1, policy_version 30380 (0.0011) [2023-10-14 18:52:10,283][61585] Updated weights for policy 1, policy_version 30390 (0.0008) [2023-10-14 18:52:10,643][61585] Updated weights for policy 1, policy_version 30400 (0.0009) [2023-10-14 18:52:12,354][61552] Updated weights for policy 0, policy_version 30532 (0.0008) [2023-10-14 18:52:12,717][61552] Updated weights for policy 0, policy_version 30542 (0.0009) [2023-10-14 18:52:13,089][61552] Updated weights for policy 0, policy_version 30552 (0.0007) [2023-10-14 18:52:13,343][60425] Fps is (10 sec: 9830.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 62390272. Throughput: 0: 1655.6, 1: 1660.8. Samples: 15613264. Policy #0 lag: (min: 12.0, avg: 15.8, max: 44.0) [2023-10-14 18:52:13,344][60425] Avg episode reward: [(0, '60.400'), (1, '57.760')] [2023-10-14 18:52:14,808][61585] Updated weights for policy 1, policy_version 30410 (0.0007) [2023-10-14 18:52:15,178][61585] Updated weights for policy 1, policy_version 30420 (0.0007) [2023-10-14 18:52:15,545][61585] Updated weights for policy 1, policy_version 30430 (0.0008) [2023-10-14 18:52:17,274][61552] Updated weights for policy 0, policy_version 30562 (0.0009) [2023-10-14 18:52:17,640][61552] Updated weights for policy 0, policy_version 30572 (0.0011) [2023-10-14 18:52:18,022][61552] Updated weights for policy 0, policy_version 30582 (0.0008) [2023-10-14 18:52:18,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 62455808. Throughput: 0: 1668.6, 1: 1648.6. Samples: 15622642. Policy #0 lag: (min: 12.0, avg: 15.8, max: 44.0) [2023-10-14 18:52:18,344][60425] Avg episode reward: [(0, '60.910'), (1, '56.390')] [2023-10-14 18:52:18,388][61552] Updated weights for policy 0, policy_version 30592 (0.0008) [2023-10-14 18:52:19,693][61585] Updated weights for policy 1, policy_version 30440 (0.0008) [2023-10-14 18:52:20,058][61585] Updated weights for policy 1, policy_version 30450 (0.0009) [2023-10-14 18:52:20,423][61585] Updated weights for policy 1, policy_version 30460 (0.0008) [2023-10-14 18:52:22,450][61552] Updated weights for policy 0, policy_version 30602 (0.0008) [2023-10-14 18:52:22,816][61552] Updated weights for policy 0, policy_version 30612 (0.0007) [2023-10-14 18:52:23,175][61552] Updated weights for policy 0, policy_version 30622 (0.0009) [2023-10-14 18:52:23,343][60425] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 62554112. Throughput: 0: 1666.1, 1: 1652.0. Samples: 15642914. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:52:23,344][60425] Avg episode reward: [(0, '61.360'), (1, '57.400')] [2023-10-14 18:52:24,585][61585] Updated weights for policy 1, policy_version 30470 (0.0009) [2023-10-14 18:52:24,948][61585] Updated weights for policy 1, policy_version 30480 (0.0009) [2023-10-14 18:52:25,318][61585] Updated weights for policy 1, policy_version 30490 (0.0008) [2023-10-14 18:52:27,322][61552] Updated weights for policy 0, policy_version 30632 (0.0009) [2023-10-14 18:52:27,688][61552] Updated weights for policy 0, policy_version 30642 (0.0009) [2023-10-14 18:52:28,058][61552] Updated weights for policy 0, policy_version 30652 (0.0008) [2023-10-14 18:52:28,343][60425] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 62619648. Throughput: 0: 1655.7, 1: 1650.8. Samples: 15662818. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:52:28,344][60425] Avg episode reward: [(0, '57.390'), (1, '59.540')] [2023-10-14 18:52:29,533][61585] Updated weights for policy 1, policy_version 30500 (0.0010) [2023-10-14 18:52:29,894][61585] Updated weights for policy 1, policy_version 30510 (0.0010) [2023-10-14 18:52:30,251][61585] Updated weights for policy 1, policy_version 30520 (0.0011) [2023-10-14 18:52:32,195][61552] Updated weights for policy 0, policy_version 30662 (0.0008) [2023-10-14 18:52:32,571][61552] Updated weights for policy 0, policy_version 30672 (0.0008) [2023-10-14 18:52:32,939][61552] Updated weights for policy 0, policy_version 30682 (0.0007) [2023-10-14 18:52:33,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 62685184. Throughput: 0: 1668.4, 1: 1647.9. Samples: 15672562. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:52:33,344][60425] Avg episode reward: [(0, '60.350'), (1, '59.190')] [2023-10-14 18:52:34,427][61585] Updated weights for policy 1, policy_version 30530 (0.0010) [2023-10-14 18:52:34,787][61585] Updated weights for policy 1, policy_version 30540 (0.0010) [2023-10-14 18:52:35,148][61585] Updated weights for policy 1, policy_version 30550 (0.0007) [2023-10-14 18:52:35,515][61585] Updated weights for policy 1, policy_version 30560 (0.0007) [2023-10-14 18:52:36,747][61552] Updated weights for policy 0, policy_version 30692 (0.0007) [2023-10-14 18:52:37,123][61552] Updated weights for policy 0, policy_version 30702 (0.0008) [2023-10-14 18:52:37,494][61552] Updated weights for policy 0, policy_version 30712 (0.0008) [2023-10-14 18:52:38,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 62750720. Throughput: 0: 1667.4, 1: 1650.4. Samples: 15693034. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:52:38,345][60425] Avg episode reward: [(0, '60.370'), (1, '59.540')] [2023-10-14 18:52:39,674][61585] Updated weights for policy 1, policy_version 30570 (0.0008) [2023-10-14 18:52:40,040][61585] Updated weights for policy 1, policy_version 30580 (0.0008) [2023-10-14 18:52:40,406][61585] Updated weights for policy 1, policy_version 30590 (0.0008) [2023-10-14 18:52:41,682][61552] Updated weights for policy 0, policy_version 30722 (0.0009) [2023-10-14 18:52:42,056][61552] Updated weights for policy 0, policy_version 30732 (0.0011) [2023-10-14 18:52:42,431][61552] Updated weights for policy 0, policy_version 30742 (0.0011) [2023-10-14 18:52:42,799][61552] Updated weights for policy 0, policy_version 30752 (0.0009) [2023-10-14 18:52:43,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 62816256. Throughput: 0: 1658.0, 1: 1648.8. Samples: 15712552. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:52:43,344][60425] Avg episode reward: [(0, '61.340'), (1, '55.250')] [2023-10-14 18:52:44,561][61585] Updated weights for policy 1, policy_version 30600 (0.0007) [2023-10-14 18:52:44,924][61585] Updated weights for policy 1, policy_version 30610 (0.0007) [2023-10-14 18:52:45,279][61585] Updated weights for policy 1, policy_version 30620 (0.0007) [2023-10-14 18:52:46,871][61552] Updated weights for policy 0, policy_version 30762 (0.0008) [2023-10-14 18:52:47,237][61552] Updated weights for policy 0, policy_version 30772 (0.0008) [2023-10-14 18:52:47,604][61552] Updated weights for policy 0, policy_version 30782 (0.0009) [2023-10-14 18:52:48,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 62881792. Throughput: 0: 1672.3, 1: 1650.9. Samples: 15722790. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:52:48,344][60425] Avg episode reward: [(0, '59.230'), (1, '58.970')] [2023-10-14 18:52:49,598][61585] Updated weights for policy 1, policy_version 30630 (0.0009) [2023-10-14 18:52:49,967][61585] Updated weights for policy 1, policy_version 30640 (0.0009) [2023-10-14 18:52:50,324][61585] Updated weights for policy 1, policy_version 30650 (0.0010) [2023-10-14 18:52:51,738][61552] Updated weights for policy 0, policy_version 30792 (0.0009) [2023-10-14 18:52:52,107][61552] Updated weights for policy 0, policy_version 30802 (0.0009) [2023-10-14 18:52:52,477][61552] Updated weights for policy 0, policy_version 30812 (0.0009) [2023-10-14 18:52:53,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 62947328. Throughput: 0: 1662.7, 1: 1655.1. Samples: 15742792. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:52:53,344][60425] Avg episode reward: [(0, '60.620'), (1, '56.610')] [2023-10-14 18:52:54,411][61585] Updated weights for policy 1, policy_version 30660 (0.0008) [2023-10-14 18:52:54,779][61585] Updated weights for policy 1, policy_version 30670 (0.0008) [2023-10-14 18:52:55,140][61585] Updated weights for policy 1, policy_version 30680 (0.0008) [2023-10-14 18:52:56,426][61552] Updated weights for policy 0, policy_version 30822 (0.0007) [2023-10-14 18:52:56,800][61552] Updated weights for policy 0, policy_version 30832 (0.0008) [2023-10-14 18:52:57,179][61552] Updated weights for policy 0, policy_version 30842 (0.0008) [2023-10-14 18:52:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 63012864. Throughput: 0: 1664.4, 1: 1656.2. Samples: 15762690. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:52:58,344][60425] Avg episode reward: [(0, '60.990'), (1, '54.540')] [2023-10-14 18:52:59,186][61585] Updated weights for policy 1, policy_version 30690 (0.0009) [2023-10-14 18:52:59,549][61585] Updated weights for policy 1, policy_version 30700 (0.0008) [2023-10-14 18:52:59,914][61585] Updated weights for policy 1, policy_version 30710 (0.0007) [2023-10-14 18:53:00,277][61585] Updated weights for policy 1, policy_version 30720 (0.0009) [2023-10-14 18:53:01,099][61552] Updated weights for policy 0, policy_version 30852 (0.0009) [2023-10-14 18:53:01,461][61552] Updated weights for policy 0, policy_version 30862 (0.0008) [2023-10-14 18:53:01,846][61552] Updated weights for policy 0, policy_version 30872 (0.0009) [2023-10-14 18:53:03,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 63078400. Throughput: 0: 1689.2, 1: 1657.4. Samples: 15773240. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:53:03,344][60425] Avg episode reward: [(0, '61.190'), (1, '57.980')] [2023-10-14 18:53:04,344][61585] Updated weights for policy 1, policy_version 30730 (0.0008) [2023-10-14 18:53:04,710][61585] Updated weights for policy 1, policy_version 30740 (0.0007) [2023-10-14 18:53:05,074][61585] Updated weights for policy 1, policy_version 30750 (0.0007) [2023-10-14 18:53:05,902][61552] Updated weights for policy 0, policy_version 30882 (0.0008) [2023-10-14 18:53:06,272][61552] Updated weights for policy 0, policy_version 30892 (0.0011) [2023-10-14 18:53:06,651][61552] Updated weights for policy 0, policy_version 30902 (0.0008) [2023-10-14 18:53:07,016][61552] Updated weights for policy 0, policy_version 30912 (0.0008) [2023-10-14 18:53:08,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 63143936. Throughput: 0: 1670.6, 1: 1665.6. Samples: 15793044. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:53:08,345][60425] Avg episode reward: [(0, '62.540'), (1, '57.240')] [2023-10-14 18:53:09,195][61585] Updated weights for policy 1, policy_version 30760 (0.0009) [2023-10-14 18:53:09,561][61585] Updated weights for policy 1, policy_version 30770 (0.0008) [2023-10-14 18:53:09,930][61585] Updated weights for policy 1, policy_version 30780 (0.0008) [2023-10-14 18:53:11,222][61552] Updated weights for policy 0, policy_version 30922 (0.0010) [2023-10-14 18:53:11,594][61552] Updated weights for policy 0, policy_version 30932 (0.0008) [2023-10-14 18:53:11,957][61552] Updated weights for policy 0, policy_version 30942 (0.0008) [2023-10-14 18:53:13,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 63209472. Throughput: 0: 1672.1, 1: 1664.7. Samples: 15812974. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-14 18:53:13,344][60425] Avg episode reward: [(0, '63.940'), (1, '59.530')] [2023-10-14 18:53:14,130][61585] Updated weights for policy 1, policy_version 30790 (0.0010) [2023-10-14 18:53:14,497][61585] Updated weights for policy 1, policy_version 30800 (0.0007) [2023-10-14 18:53:14,857][61585] Updated weights for policy 1, policy_version 30810 (0.0009) [2023-10-14 18:53:16,009][61552] Updated weights for policy 0, policy_version 30952 (0.0007) [2023-10-14 18:53:16,386][61552] Updated weights for policy 0, policy_version 30962 (0.0007) [2023-10-14 18:53:16,756][61552] Updated weights for policy 0, policy_version 30972 (0.0009) [2023-10-14 18:53:18,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 63275008. Throughput: 0: 1687.7, 1: 1663.7. Samples: 15823376. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-14 18:53:18,344][60425] Avg episode reward: [(0, '65.630'), (1, '59.330')] [2023-10-14 18:53:18,920][61585] Updated weights for policy 1, policy_version 30820 (0.0008) [2023-10-14 18:53:19,293][61585] Updated weights for policy 1, policy_version 30830 (0.0008) [2023-10-14 18:53:19,661][61585] Updated weights for policy 1, policy_version 30840 (0.0009) [2023-10-14 18:53:20,985][61552] Updated weights for policy 0, policy_version 30982 (0.0007) [2023-10-14 18:53:21,346][61552] Updated weights for policy 0, policy_version 30992 (0.0009) [2023-10-14 18:53:21,714][61552] Updated weights for policy 0, policy_version 31002 (0.0008) [2023-10-14 18:53:23,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 63340544. Throughput: 0: 1662.9, 1: 1668.5. Samples: 15842942. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-14 18:53:23,344][60425] Avg episode reward: [(0, '61.930'), (1, '61.960')] [2023-10-14 18:53:23,685][61585] Updated weights for policy 1, policy_version 30850 (0.0008) [2023-10-14 18:53:24,059][61585] Updated weights for policy 1, policy_version 30860 (0.0008) [2023-10-14 18:53:24,414][61585] Updated weights for policy 1, policy_version 30870 (0.0009) [2023-10-14 18:53:24,786][61585] Updated weights for policy 1, policy_version 30880 (0.0010) [2023-10-14 18:53:25,749][61552] Updated weights for policy 0, policy_version 31012 (0.0009) [2023-10-14 18:53:26,117][61552] Updated weights for policy 0, policy_version 31022 (0.0008) [2023-10-14 18:53:26,492][61552] Updated weights for policy 0, policy_version 31032 (0.0008) [2023-10-14 18:53:28,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 63406080. Throughput: 0: 1679.1, 1: 1668.7. Samples: 15863200. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-14 18:53:28,345][60425] Avg episode reward: [(0, '61.700'), (1, '56.350')] [2023-10-14 18:53:28,898][61585] Updated weights for policy 1, policy_version 30890 (0.0008) [2023-10-14 18:53:29,266][61585] Updated weights for policy 1, policy_version 30900 (0.0010) [2023-10-14 18:53:29,629][61585] Updated weights for policy 1, policy_version 30910 (0.0009) [2023-10-14 18:53:30,665][61552] Updated weights for policy 0, policy_version 31042 (0.0008) [2023-10-14 18:53:31,032][61552] Updated weights for policy 0, policy_version 31052 (0.0007) [2023-10-14 18:53:31,400][61552] Updated weights for policy 0, policy_version 31062 (0.0010) [2023-10-14 18:53:31,772][61552] Updated weights for policy 0, policy_version 31072 (0.0007) [2023-10-14 18:53:33,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 63471616. Throughput: 0: 1675.2, 1: 1669.0. Samples: 15873276. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-14 18:53:33,344][60425] Avg episode reward: [(0, '63.520'), (1, '60.090')] [2023-10-14 18:53:33,977][61585] Updated weights for policy 1, policy_version 30920 (0.0009) [2023-10-14 18:53:34,357][61585] Updated weights for policy 1, policy_version 30930 (0.0007) [2023-10-14 18:53:34,720][61585] Updated weights for policy 1, policy_version 30940 (0.0007) [2023-10-14 18:53:35,931][61552] Updated weights for policy 0, policy_version 31082 (0.0008) [2023-10-14 18:53:36,297][61552] Updated weights for policy 0, policy_version 31092 (0.0009) [2023-10-14 18:53:36,665][61552] Updated weights for policy 0, policy_version 31102 (0.0009) [2023-10-14 18:53:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 63537152. Throughput: 0: 1665.6, 1: 1664.6. Samples: 15892650. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:53:38,345][60425] Avg episode reward: [(0, '64.480'), (1, '59.680')] [2023-10-14 18:53:38,780][61585] Updated weights for policy 1, policy_version 30950 (0.0009) [2023-10-14 18:53:39,135][61585] Updated weights for policy 1, policy_version 30960 (0.0009) [2023-10-14 18:53:39,500][61585] Updated weights for policy 1, policy_version 30970 (0.0008) [2023-10-14 18:53:40,485][61552] Updated weights for policy 0, policy_version 31112 (0.0009) [2023-10-14 18:53:40,860][61552] Updated weights for policy 0, policy_version 31122 (0.0008) [2023-10-14 18:53:41,220][61552] Updated weights for policy 0, policy_version 31132 (0.0011) [2023-10-14 18:53:43,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 63602688. Throughput: 0: 1691.0, 1: 1664.9. Samples: 15913706. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:53:43,344][60425] Avg episode reward: [(0, '61.600'), (1, '58.080')] [2023-10-14 18:53:43,354][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000031136_31883264.pth... [2023-10-14 18:53:43,392][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000029568_30277632.pth [2023-10-14 18:53:43,398][61172] Saving a milestone ./train_atari/atari_roadrunner_APPO/checkpoint_p0/milestones/checkpoint_000031136_31883264.pth [2023-10-14 18:53:43,437][61585] Updated weights for policy 1, policy_version 30980 (0.0008) [2023-10-14 18:53:43,813][61585] Updated weights for policy 1, policy_version 30990 (0.0008) [2023-10-14 18:53:44,177][61585] Updated weights for policy 1, policy_version 31000 (0.0010) [2023-10-14 18:53:44,471][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000031008_31752192.pth... [2023-10-14 18:53:44,509][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000029440_30146560.pth [2023-10-14 18:53:44,515][61248] Saving a milestone ./train_atari/atari_roadrunner_APPO/checkpoint_p1/milestones/checkpoint_000031008_31752192.pth [2023-10-14 18:53:45,211][61552] Updated weights for policy 0, policy_version 31142 (0.0008) [2023-10-14 18:53:45,579][61552] Updated weights for policy 0, policy_version 31152 (0.0007) [2023-10-14 18:53:45,948][61552] Updated weights for policy 0, policy_version 31162 (0.0009) [2023-10-14 18:53:48,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 63668224. Throughput: 0: 1664.7, 1: 1666.5. Samples: 15923142. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:53:48,344][60425] Avg episode reward: [(0, '63.640'), (1, '57.030')] [2023-10-14 18:53:48,357][61585] Updated weights for policy 1, policy_version 31010 (0.0011) [2023-10-14 18:53:48,720][61585] Updated weights for policy 1, policy_version 31020 (0.0010) [2023-10-14 18:53:49,097][61585] Updated weights for policy 1, policy_version 31030 (0.0010) [2023-10-14 18:53:49,462][61585] Updated weights for policy 1, policy_version 31040 (0.0009) [2023-10-14 18:53:50,088][61552] Updated weights for policy 0, policy_version 31172 (0.0010) [2023-10-14 18:53:50,454][61552] Updated weights for policy 0, policy_version 31182 (0.0007) [2023-10-14 18:53:50,822][61552] Updated weights for policy 0, policy_version 31192 (0.0009) [2023-10-14 18:53:53,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 63733760. Throughput: 0: 1669.4, 1: 1660.5. Samples: 15942892. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:53:53,344][60425] Avg episode reward: [(0, '59.870'), (1, '57.090')] [2023-10-14 18:53:53,608][61585] Updated weights for policy 1, policy_version 31050 (0.0009) [2023-10-14 18:53:53,965][61585] Updated weights for policy 1, policy_version 31060 (0.0011) [2023-10-14 18:53:54,336][61585] Updated weights for policy 1, policy_version 31070 (0.0008) [2023-10-14 18:53:54,996][61552] Updated weights for policy 0, policy_version 31202 (0.0009) [2023-10-14 18:53:55,363][61552] Updated weights for policy 0, policy_version 31212 (0.0008) [2023-10-14 18:53:55,733][61552] Updated weights for policy 0, policy_version 31222 (0.0008) [2023-10-14 18:53:56,104][61552] Updated weights for policy 0, policy_version 31232 (0.0010) [2023-10-14 18:53:58,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 63799296. Throughput: 0: 1678.6, 1: 1664.0. Samples: 15963392. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:53:58,345][60425] Avg episode reward: [(0, '60.970'), (1, '56.460')] [2023-10-14 18:53:58,454][61585] Updated weights for policy 1, policy_version 31080 (0.0009) [2023-10-14 18:53:58,815][61585] Updated weights for policy 1, policy_version 31090 (0.0009) [2023-10-14 18:53:59,185][61585] Updated weights for policy 1, policy_version 31100 (0.0007) [2023-10-14 18:54:00,164][61552] Updated weights for policy 0, policy_version 31242 (0.0008) [2023-10-14 18:54:00,548][61552] Updated weights for policy 0, policy_version 31252 (0.0008) [2023-10-14 18:54:00,922][61552] Updated weights for policy 0, policy_version 31262 (0.0008) [2023-10-14 18:54:03,198][61585] Updated weights for policy 1, policy_version 31110 (0.0008) [2023-10-14 18:54:03,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 63864832. Throughput: 0: 1654.0, 1: 1666.9. Samples: 15972816. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) [2023-10-14 18:54:03,344][60425] Avg episode reward: [(0, '65.920'), (1, '55.780')] [2023-10-14 18:54:03,345][61172] Saving new best policy, reward=65.920! [2023-10-14 18:54:03,565][61585] Updated weights for policy 1, policy_version 31120 (0.0011) [2023-10-14 18:54:03,927][61585] Updated weights for policy 1, policy_version 31130 (0.0008) [2023-10-14 18:54:05,059][61552] Updated weights for policy 0, policy_version 31272 (0.0008) [2023-10-14 18:54:05,433][61552] Updated weights for policy 0, policy_version 31282 (0.0007) [2023-10-14 18:54:05,800][61552] Updated weights for policy 0, policy_version 31292 (0.0008) [2023-10-14 18:54:08,030][61585] Updated weights for policy 1, policy_version 31140 (0.0007) [2023-10-14 18:54:08,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 63930368. Throughput: 0: 1668.0, 1: 1665.9. Samples: 15992970. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) [2023-10-14 18:54:08,345][60425] Avg episode reward: [(0, '65.280'), (1, '58.230')] [2023-10-14 18:54:08,401][61585] Updated weights for policy 1, policy_version 31150 (0.0008) [2023-10-14 18:54:08,772][61585] Updated weights for policy 1, policy_version 31160 (0.0009) [2023-10-14 18:54:10,046][61552] Updated weights for policy 0, policy_version 31302 (0.0007) [2023-10-14 18:54:10,416][61552] Updated weights for policy 0, policy_version 31312 (0.0007) [2023-10-14 18:54:10,791][61552] Updated weights for policy 0, policy_version 31322 (0.0007) [2023-10-14 18:54:12,807][61585] Updated weights for policy 1, policy_version 31170 (0.0010) [2023-10-14 18:54:13,163][61585] Updated weights for policy 1, policy_version 31180 (0.0008) [2023-10-14 18:54:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 63995904. Throughput: 0: 1671.7, 1: 1667.2. Samples: 16013450. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) [2023-10-14 18:54:13,344][60425] Avg episode reward: [(0, '64.010'), (1, '56.580')] [2023-10-14 18:54:13,532][61585] Updated weights for policy 1, policy_version 31190 (0.0009) [2023-10-14 18:54:13,893][61585] Updated weights for policy 1, policy_version 31200 (0.0008) [2023-10-14 18:54:14,980][61552] Updated weights for policy 0, policy_version 31332 (0.0008) [2023-10-14 18:54:15,348][61552] Updated weights for policy 0, policy_version 31342 (0.0007) [2023-10-14 18:54:15,713][61552] Updated weights for policy 0, policy_version 31352 (0.0007) [2023-10-14 18:54:17,927][61585] Updated weights for policy 1, policy_version 31210 (0.0009) [2023-10-14 18:54:18,296][61585] Updated weights for policy 1, policy_version 31220 (0.0009) [2023-10-14 18:54:18,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 64061440. Throughput: 0: 1658.5, 1: 1667.7. Samples: 16022954. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) [2023-10-14 18:54:18,344][60425] Avg episode reward: [(0, '61.850'), (1, '56.100')] [2023-10-14 18:54:18,670][61585] Updated weights for policy 1, policy_version 31230 (0.0009) [2023-10-14 18:54:19,920][61552] Updated weights for policy 0, policy_version 31362 (0.0008) [2023-10-14 18:54:20,287][61552] Updated weights for policy 0, policy_version 31372 (0.0007) [2023-10-14 18:54:20,651][61552] Updated weights for policy 0, policy_version 31382 (0.0009) [2023-10-14 18:54:21,017][61552] Updated weights for policy 0, policy_version 31392 (0.0009) [2023-10-14 18:54:22,906][61585] Updated weights for policy 1, policy_version 31240 (0.0009) [2023-10-14 18:54:23,280][61585] Updated weights for policy 1, policy_version 31250 (0.0009) [2023-10-14 18:54:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 64126976. Throughput: 0: 1667.7, 1: 1673.2. Samples: 16042990. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) [2023-10-14 18:54:23,344][60425] Avg episode reward: [(0, '63.760'), (1, '58.890')] [2023-10-14 18:54:23,650][61585] Updated weights for policy 1, policy_version 31260 (0.0008) [2023-10-14 18:54:25,124][61552] Updated weights for policy 0, policy_version 31402 (0.0008) [2023-10-14 18:54:25,503][61552] Updated weights for policy 0, policy_version 31412 (0.0008) [2023-10-14 18:54:25,875][61552] Updated weights for policy 0, policy_version 31422 (0.0010) [2023-10-14 18:54:27,616][61585] Updated weights for policy 1, policy_version 31270 (0.0009) [2023-10-14 18:54:27,982][61585] Updated weights for policy 1, policy_version 31280 (0.0008) [2023-10-14 18:54:28,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 64192512. Throughput: 0: 1661.1, 1: 1662.6. Samples: 16063270. Policy #0 lag: (min: 12.0, avg: 20.0, max: 44.0) [2023-10-14 18:54:28,344][60425] Avg episode reward: [(0, '62.130'), (1, '56.500')] [2023-10-14 18:54:28,347][61585] Updated weights for policy 1, policy_version 31290 (0.0008) [2023-10-14 18:54:29,825][61552] Updated weights for policy 0, policy_version 31432 (0.0009) [2023-10-14 18:54:30,201][61552] Updated weights for policy 0, policy_version 31442 (0.0008) [2023-10-14 18:54:30,569][61552] Updated weights for policy 0, policy_version 31452 (0.0009) [2023-10-14 18:54:32,355][61585] Updated weights for policy 1, policy_version 31300 (0.0008) [2023-10-14 18:54:32,725][61585] Updated weights for policy 1, policy_version 31310 (0.0007) [2023-10-14 18:54:33,089][61585] Updated weights for policy 1, policy_version 31320 (0.0007) [2023-10-14 18:54:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 64258048. Throughput: 0: 1653.9, 1: 1672.2. Samples: 16072816. Policy #0 lag: (min: 12.0, avg: 20.0, max: 44.0) [2023-10-14 18:54:33,344][60425] Avg episode reward: [(0, '60.500'), (1, '54.880')] [2023-10-14 18:54:34,703][61552] Updated weights for policy 0, policy_version 31462 (0.0009) [2023-10-14 18:54:35,076][61552] Updated weights for policy 0, policy_version 31472 (0.0009) [2023-10-14 18:54:35,441][61552] Updated weights for policy 0, policy_version 31482 (0.0008) [2023-10-14 18:54:37,425][61585] Updated weights for policy 1, policy_version 31330 (0.0008) [2023-10-14 18:54:37,785][61585] Updated weights for policy 1, policy_version 31340 (0.0009) [2023-10-14 18:54:38,155][61585] Updated weights for policy 1, policy_version 31350 (0.0008) [2023-10-14 18:54:38,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 64323584. Throughput: 0: 1660.1, 1: 1672.4. Samples: 16092858. Policy #0 lag: (min: 12.0, avg: 20.0, max: 44.0) [2023-10-14 18:54:38,344][60425] Avg episode reward: [(0, '62.820'), (1, '57.280')] [2023-10-14 18:54:38,515][61585] Updated weights for policy 1, policy_version 31360 (0.0008) [2023-10-14 18:54:39,508][61552] Updated weights for policy 0, policy_version 31492 (0.0007) [2023-10-14 18:54:39,874][61552] Updated weights for policy 0, policy_version 31502 (0.0008) [2023-10-14 18:54:40,244][61552] Updated weights for policy 0, policy_version 31512 (0.0007) [2023-10-14 18:54:42,725][61585] Updated weights for policy 1, policy_version 31370 (0.0010) [2023-10-14 18:54:43,087][61585] Updated weights for policy 1, policy_version 31380 (0.0011) [2023-10-14 18:54:43,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13218.3). Total num frames: 64389120. Throughput: 0: 1669.3, 1: 1661.6. Samples: 16113282. Policy #0 lag: (min: 12.0, avg: 20.0, max: 44.0) [2023-10-14 18:54:43,345][60425] Avg episode reward: [(0, '60.720'), (1, '55.690')] [2023-10-14 18:54:43,456][61585] Updated weights for policy 1, policy_version 31390 (0.0009) [2023-10-14 18:54:44,314][61552] Updated weights for policy 0, policy_version 31522 (0.0008) [2023-10-14 18:54:44,686][61552] Updated weights for policy 0, policy_version 31532 (0.0010) [2023-10-14 18:54:45,061][61552] Updated weights for policy 0, policy_version 31542 (0.0011) [2023-10-14 18:54:45,434][61552] Updated weights for policy 0, policy_version 31552 (0.0007) [2023-10-14 18:54:47,528][61585] Updated weights for policy 1, policy_version 31400 (0.0009) [2023-10-14 18:54:47,903][61585] Updated weights for policy 1, policy_version 31410 (0.0009) [2023-10-14 18:54:48,270][61585] Updated weights for policy 1, policy_version 31420 (0.0008) [2023-10-14 18:54:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 64454656. Throughput: 0: 1661.4, 1: 1667.7. Samples: 16122624. Policy #0 lag: (min: 12.0, avg: 20.0, max: 44.0) [2023-10-14 18:54:48,344][60425] Avg episode reward: [(0, '60.200'), (1, '56.450')] [2023-10-14 18:54:49,566][61552] Updated weights for policy 0, policy_version 31562 (0.0008) [2023-10-14 18:54:49,945][61552] Updated weights for policy 0, policy_version 31572 (0.0010) [2023-10-14 18:54:50,318][61552] Updated weights for policy 0, policy_version 31582 (0.0010) [2023-10-14 18:54:52,378][61585] Updated weights for policy 1, policy_version 31430 (0.0009) [2023-10-14 18:54:52,748][61585] Updated weights for policy 1, policy_version 31440 (0.0009) [2023-10-14 18:54:53,105][61585] Updated weights for policy 1, policy_version 31450 (0.0009) [2023-10-14 18:54:53,343][60425] Fps is (10 sec: 16384.6, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 64552960. Throughput: 0: 1666.2, 1: 1667.2. Samples: 16142970. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:54:53,344][60425] Avg episode reward: [(0, '63.840'), (1, '57.440')] [2023-10-14 18:54:54,282][61552] Updated weights for policy 0, policy_version 31592 (0.0009) [2023-10-14 18:54:54,651][61552] Updated weights for policy 0, policy_version 31602 (0.0007) [2023-10-14 18:54:55,011][61552] Updated weights for policy 0, policy_version 31612 (0.0011) [2023-10-14 18:54:57,291][61585] Updated weights for policy 1, policy_version 31460 (0.0008) [2023-10-14 18:54:57,662][61585] Updated weights for policy 1, policy_version 31470 (0.0011) [2023-10-14 18:54:58,040][61585] Updated weights for policy 1, policy_version 31480 (0.0010) [2023-10-14 18:54:58,343][60425] Fps is (10 sec: 16383.8, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 64618496. Throughput: 0: 1673.2, 1: 1649.4. Samples: 16162968. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:54:58,344][60425] Avg episode reward: [(0, '60.250'), (1, '57.100')] [2023-10-14 18:54:59,218][61552] Updated weights for policy 0, policy_version 31622 (0.0010) [2023-10-14 18:54:59,596][61552] Updated weights for policy 0, policy_version 31632 (0.0007) [2023-10-14 18:54:59,964][61552] Updated weights for policy 0, policy_version 31642 (0.0007) [2023-10-14 18:55:02,149][61585] Updated weights for policy 1, policy_version 31490 (0.0008) [2023-10-14 18:55:02,521][61585] Updated weights for policy 1, policy_version 31500 (0.0010) [2023-10-14 18:55:02,889][61585] Updated weights for policy 1, policy_version 31510 (0.0010) [2023-10-14 18:55:03,256][61585] Updated weights for policy 1, policy_version 31520 (0.0007) [2023-10-14 18:55:03,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 64684032. Throughput: 0: 1659.8, 1: 1667.1. Samples: 16172662. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:55:03,344][60425] Avg episode reward: [(0, '61.100'), (1, '58.380')] [2023-10-14 18:55:04,156][61552] Updated weights for policy 0, policy_version 31652 (0.0008) [2023-10-14 18:55:04,525][61552] Updated weights for policy 0, policy_version 31662 (0.0008) [2023-10-14 18:55:04,890][61552] Updated weights for policy 0, policy_version 31672 (0.0009) [2023-10-14 18:55:07,633][61585] Updated weights for policy 1, policy_version 31530 (0.0011) [2023-10-14 18:55:07,994][61585] Updated weights for policy 1, policy_version 31540 (0.0011) [2023-10-14 18:55:08,343][60425] Fps is (10 sec: 9830.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 64716800. Throughput: 0: 1670.0, 1: 1662.0. Samples: 16192930. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:55:08,344][60425] Avg episode reward: [(0, '60.780'), (1, '58.430')] [2023-10-14 18:55:08,368][61585] Updated weights for policy 1, policy_version 31550 (0.0010) [2023-10-14 18:55:08,854][61552] Updated weights for policy 0, policy_version 31682 (0.0008) [2023-10-14 18:55:09,219][61552] Updated weights for policy 0, policy_version 31692 (0.0010) [2023-10-14 18:55:09,591][61552] Updated weights for policy 0, policy_version 31702 (0.0009) [2023-10-14 18:55:09,951][61552] Updated weights for policy 0, policy_version 31712 (0.0008) [2023-10-14 18:55:12,621][61585] Updated weights for policy 1, policy_version 31560 (0.0009) [2023-10-14 18:55:13,005][61585] Updated weights for policy 1, policy_version 31570 (0.0008) [2023-10-14 18:55:13,343][60425] Fps is (10 sec: 9830.6, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 64782336. Throughput: 0: 1674.5, 1: 1654.5. Samples: 16213078. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:55:13,344][60425] Avg episode reward: [(0, '59.340'), (1, '59.780')] [2023-10-14 18:55:13,367][61585] Updated weights for policy 1, policy_version 31580 (0.0007) [2023-10-14 18:55:14,038][61552] Updated weights for policy 0, policy_version 31722 (0.0009) [2023-10-14 18:55:14,401][61552] Updated weights for policy 0, policy_version 31732 (0.0009) [2023-10-14 18:55:14,776][61552] Updated weights for policy 0, policy_version 31742 (0.0008) [2023-10-14 18:55:17,341][61585] Updated weights for policy 1, policy_version 31590 (0.0007) [2023-10-14 18:55:17,704][61585] Updated weights for policy 1, policy_version 31600 (0.0008) [2023-10-14 18:55:18,065][61585] Updated weights for policy 1, policy_version 31610 (0.0008) [2023-10-14 18:55:18,343][60425] Fps is (10 sec: 16384.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 64880640. Throughput: 0: 1676.9, 1: 1654.3. Samples: 16222722. Policy #0 lag: (min: 31.0, avg: 46.7, max: 63.0) [2023-10-14 18:55:18,344][60425] Avg episode reward: [(0, '59.180'), (1, '56.870')] [2023-10-14 18:55:18,754][61552] Updated weights for policy 0, policy_version 31752 (0.0007) [2023-10-14 18:55:19,130][61552] Updated weights for policy 0, policy_version 31762 (0.0011) [2023-10-14 18:55:19,504][61552] Updated weights for policy 0, policy_version 31772 (0.0007) [2023-10-14 18:55:22,077][61585] Updated weights for policy 1, policy_version 31620 (0.0010) [2023-10-14 18:55:22,446][61585] Updated weights for policy 1, policy_version 31630 (0.0009) [2023-10-14 18:55:22,811][61585] Updated weights for policy 1, policy_version 31640 (0.0010) [2023-10-14 18:55:23,343][60425] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 64946176. Throughput: 0: 1680.0, 1: 1659.5. Samples: 16243136. Policy #0 lag: (min: 31.0, avg: 46.7, max: 63.0) [2023-10-14 18:55:23,344][60425] Avg episode reward: [(0, '56.750'), (1, '56.970')] [2023-10-14 18:55:23,643][61552] Updated weights for policy 0, policy_version 31782 (0.0009) [2023-10-14 18:55:24,009][61552] Updated weights for policy 0, policy_version 31792 (0.0009) [2023-10-14 18:55:24,379][61552] Updated weights for policy 0, policy_version 31802 (0.0009) [2023-10-14 18:55:26,791][61585] Updated weights for policy 1, policy_version 31650 (0.0010) [2023-10-14 18:55:27,148][61585] Updated weights for policy 1, policy_version 31660 (0.0007) [2023-10-14 18:55:27,509][61585] Updated weights for policy 1, policy_version 31670 (0.0007) [2023-10-14 18:55:27,875][61585] Updated weights for policy 1, policy_version 31680 (0.0007) [2023-10-14 18:55:28,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 65011712. Throughput: 0: 1668.5, 1: 1652.9. Samples: 16262748. Policy #0 lag: (min: 31.0, avg: 46.7, max: 63.0) [2023-10-14 18:55:28,345][60425] Avg episode reward: [(0, '57.290'), (1, '58.630')] [2023-10-14 18:55:28,549][61552] Updated weights for policy 0, policy_version 31812 (0.0008) [2023-10-14 18:55:28,922][61552] Updated weights for policy 0, policy_version 31822 (0.0008) [2023-10-14 18:55:29,293][61552] Updated weights for policy 0, policy_version 31832 (0.0007) [2023-10-14 18:55:31,987][61585] Updated weights for policy 1, policy_version 31690 (0.0008) [2023-10-14 18:55:32,348][61585] Updated weights for policy 1, policy_version 31700 (0.0009) [2023-10-14 18:55:32,709][61585] Updated weights for policy 1, policy_version 31710 (0.0008) [2023-10-14 18:55:33,331][61552] Updated weights for policy 0, policy_version 31842 (0.0009) [2023-10-14 18:55:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 65077248. Throughput: 0: 1669.6, 1: 1671.9. Samples: 16272994. Policy #0 lag: (min: 31.0, avg: 46.7, max: 63.0) [2023-10-14 18:55:33,344][60425] Avg episode reward: [(0, '58.310'), (1, '57.320')] [2023-10-14 18:55:33,702][61552] Updated weights for policy 0, policy_version 31852 (0.0009) [2023-10-14 18:55:34,068][61552] Updated weights for policy 0, policy_version 31862 (0.0007) [2023-10-14 18:55:34,432][61552] Updated weights for policy 0, policy_version 31872 (0.0007) [2023-10-14 18:55:36,708][61585] Updated weights for policy 1, policy_version 31720 (0.0009) [2023-10-14 18:55:37,070][61585] Updated weights for policy 1, policy_version 31730 (0.0008) [2023-10-14 18:55:37,442][61585] Updated weights for policy 1, policy_version 31740 (0.0008) [2023-10-14 18:55:38,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 65142784. Throughput: 0: 1671.9, 1: 1667.2. Samples: 16293234. Policy #0 lag: (min: 31.0, avg: 46.7, max: 63.0) [2023-10-14 18:55:38,344][60425] Avg episode reward: [(0, '62.060'), (1, '59.170')] [2023-10-14 18:55:38,583][61552] Updated weights for policy 0, policy_version 31882 (0.0008) [2023-10-14 18:55:38,952][61552] Updated weights for policy 0, policy_version 31892 (0.0007) [2023-10-14 18:55:39,316][61552] Updated weights for policy 0, policy_version 31902 (0.0007) [2023-10-14 18:55:41,561][61585] Updated weights for policy 1, policy_version 31750 (0.0010) [2023-10-14 18:55:41,933][61585] Updated weights for policy 1, policy_version 31760 (0.0008) [2023-10-14 18:55:42,302][61585] Updated weights for policy 1, policy_version 31770 (0.0007) [2023-10-14 18:55:43,248][61552] Updated weights for policy 0, policy_version 31912 (0.0007) [2023-10-14 18:55:43,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 65208320. Throughput: 0: 1674.1, 1: 1661.4. Samples: 16313066. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-14 18:55:43,344][60425] Avg episode reward: [(0, '57.420'), (1, '57.440')] [2023-10-14 18:55:43,353][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000031776_32538624.pth... [2023-10-14 18:55:43,389][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000030208_30932992.pth [2023-10-14 18:55:43,614][61552] Updated weights for policy 0, policy_version 31922 (0.0009) [2023-10-14 18:55:43,991][61552] Updated weights for policy 0, policy_version 31932 (0.0008) [2023-10-14 18:55:44,131][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000031936_32702464.pth... [2023-10-14 18:55:44,168][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000030368_31096832.pth [2023-10-14 18:55:46,424][61585] Updated weights for policy 1, policy_version 31780 (0.0008) [2023-10-14 18:55:46,787][61585] Updated weights for policy 1, policy_version 31790 (0.0008) [2023-10-14 18:55:47,156][61585] Updated weights for policy 1, policy_version 31800 (0.0008) [2023-10-14 18:55:48,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 65273856. Throughput: 0: 1672.7, 1: 1671.3. Samples: 16323144. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-14 18:55:48,344][60425] Avg episode reward: [(0, '59.590'), (1, '59.230')] [2023-10-14 18:55:48,367][61552] Updated weights for policy 0, policy_version 31942 (0.0008) [2023-10-14 18:55:48,736][61552] Updated weights for policy 0, policy_version 31952 (0.0009) [2023-10-14 18:55:49,109][61552] Updated weights for policy 0, policy_version 31962 (0.0007) [2023-10-14 18:55:51,281][61585] Updated weights for policy 1, policy_version 31810 (0.0008) [2023-10-14 18:55:51,649][61585] Updated weights for policy 1, policy_version 31820 (0.0007) [2023-10-14 18:55:52,014][61585] Updated weights for policy 1, policy_version 31830 (0.0008) [2023-10-14 18:55:52,386][61585] Updated weights for policy 1, policy_version 31840 (0.0009) [2023-10-14 18:55:53,211][61552] Updated weights for policy 0, policy_version 31972 (0.0008) [2023-10-14 18:55:53,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 65339392. Throughput: 0: 1675.7, 1: 1662.7. Samples: 16343158. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-14 18:55:53,344][60425] Avg episode reward: [(0, '62.300'), (1, '54.510')] [2023-10-14 18:55:53,575][61552] Updated weights for policy 0, policy_version 31982 (0.0007) [2023-10-14 18:55:53,941][61552] Updated weights for policy 0, policy_version 31992 (0.0010) [2023-10-14 18:55:56,444][61585] Updated weights for policy 1, policy_version 31850 (0.0009) [2023-10-14 18:55:56,810][61585] Updated weights for policy 1, policy_version 31860 (0.0008) [2023-10-14 18:55:57,178][61585] Updated weights for policy 1, policy_version 31870 (0.0008) [2023-10-14 18:55:58,242][61552] Updated weights for policy 0, policy_version 32002 (0.0010) [2023-10-14 18:55:58,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 65404928. Throughput: 0: 1668.6, 1: 1664.6. Samples: 16363070. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-14 18:55:58,345][60425] Avg episode reward: [(0, '60.430'), (1, '58.780')] [2023-10-14 18:55:58,613][61552] Updated weights for policy 0, policy_version 32012 (0.0009) [2023-10-14 18:55:58,987][61552] Updated weights for policy 0, policy_version 32022 (0.0010) [2023-10-14 18:55:59,358][61552] Updated weights for policy 0, policy_version 32032 (0.0008) [2023-10-14 18:56:01,225][61585] Updated weights for policy 1, policy_version 31880 (0.0009) [2023-10-14 18:56:01,600][61585] Updated weights for policy 1, policy_version 31890 (0.0010) [2023-10-14 18:56:01,962][61585] Updated weights for policy 1, policy_version 31900 (0.0009) [2023-10-14 18:56:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 65470464. Throughput: 0: 1662.7, 1: 1685.2. Samples: 16373376. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-14 18:56:03,344][60425] Avg episode reward: [(0, '62.210'), (1, '58.670')] [2023-10-14 18:56:03,462][61552] Updated weights for policy 0, policy_version 32042 (0.0010) [2023-10-14 18:56:03,818][61552] Updated weights for policy 0, policy_version 32052 (0.0009) [2023-10-14 18:56:04,197][61552] Updated weights for policy 0, policy_version 32062 (0.0008) [2023-10-14 18:56:06,251][61585] Updated weights for policy 1, policy_version 31910 (0.0008) [2023-10-14 18:56:06,618][61585] Updated weights for policy 1, policy_version 31920 (0.0008) [2023-10-14 18:56:06,980][61585] Updated weights for policy 1, policy_version 31930 (0.0010) [2023-10-14 18:56:08,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 65536000. Throughput: 0: 1670.1, 1: 1660.3. Samples: 16393002. Policy #0 lag: (min: 25.0, avg: 46.2, max: 57.0) [2023-10-14 18:56:08,344][60425] Avg episode reward: [(0, '64.180'), (1, '59.700')] [2023-10-14 18:56:08,366][61552] Updated weights for policy 0, policy_version 32072 (0.0009) [2023-10-14 18:56:08,730][61552] Updated weights for policy 0, policy_version 32082 (0.0008) [2023-10-14 18:56:09,105][61552] Updated weights for policy 0, policy_version 32092 (0.0009) [2023-10-14 18:56:11,187][61585] Updated weights for policy 1, policy_version 31940 (0.0007) [2023-10-14 18:56:11,557][61585] Updated weights for policy 1, policy_version 31950 (0.0011) [2023-10-14 18:56:11,916][61585] Updated weights for policy 1, policy_version 31960 (0.0009) [2023-10-14 18:56:13,176][61552] Updated weights for policy 0, policy_version 32102 (0.0009) [2023-10-14 18:56:13,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 65601536. Throughput: 0: 1671.3, 1: 1668.1. Samples: 16413022. Policy #0 lag: (min: 25.0, avg: 46.2, max: 57.0) [2023-10-14 18:56:13,344][60425] Avg episode reward: [(0, '62.810'), (1, '57.380')] [2023-10-14 18:56:13,545][61552] Updated weights for policy 0, policy_version 32112 (0.0011) [2023-10-14 18:56:13,915][61552] Updated weights for policy 0, policy_version 32122 (0.0010) [2023-10-14 18:56:15,998][61585] Updated weights for policy 1, policy_version 31970 (0.0009) [2023-10-14 18:56:16,368][61585] Updated weights for policy 1, policy_version 31980 (0.0008) [2023-10-14 18:56:16,727][61585] Updated weights for policy 1, policy_version 31990 (0.0010) [2023-10-14 18:56:17,094][61585] Updated weights for policy 1, policy_version 32000 (0.0010) [2023-10-14 18:56:18,068][61552] Updated weights for policy 0, policy_version 32132 (0.0009) [2023-10-14 18:56:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 65667072. Throughput: 0: 1666.7, 1: 1669.5. Samples: 16423124. Policy #0 lag: (min: 25.0, avg: 46.2, max: 57.0) [2023-10-14 18:56:18,344][60425] Avg episode reward: [(0, '58.690'), (1, '59.160')] [2023-10-14 18:56:18,435][61552] Updated weights for policy 0, policy_version 32142 (0.0007) [2023-10-14 18:56:18,808][61552] Updated weights for policy 0, policy_version 32152 (0.0010) [2023-10-14 18:56:21,094][61585] Updated weights for policy 1, policy_version 32010 (0.0008) [2023-10-14 18:56:21,449][61585] Updated weights for policy 1, policy_version 32020 (0.0010) [2023-10-14 18:56:21,824][61585] Updated weights for policy 1, policy_version 32030 (0.0008) [2023-10-14 18:56:22,857][61552] Updated weights for policy 0, policy_version 32162 (0.0009) [2023-10-14 18:56:23,221][61552] Updated weights for policy 0, policy_version 32172 (0.0009) [2023-10-14 18:56:23,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 65732608. Throughput: 0: 1673.4, 1: 1649.2. Samples: 16442750. Policy #0 lag: (min: 25.0, avg: 46.2, max: 57.0) [2023-10-14 18:56:23,345][60425] Avg episode reward: [(0, '59.720'), (1, '57.090')] [2023-10-14 18:56:23,581][61552] Updated weights for policy 0, policy_version 32182 (0.0008) [2023-10-14 18:56:23,952][61552] Updated weights for policy 0, policy_version 32192 (0.0009) [2023-10-14 18:56:25,941][61585] Updated weights for policy 1, policy_version 32040 (0.0008) [2023-10-14 18:56:26,296][61585] Updated weights for policy 1, policy_version 32050 (0.0008) [2023-10-14 18:56:26,663][61585] Updated weights for policy 1, policy_version 32060 (0.0009) [2023-10-14 18:56:27,932][61552] Updated weights for policy 0, policy_version 32202 (0.0008) [2023-10-14 18:56:28,300][61552] Updated weights for policy 0, policy_version 32212 (0.0007) [2023-10-14 18:56:28,344][60425] Fps is (10 sec: 13106.7, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 65798144. Throughput: 0: 1665.0, 1: 1667.7. Samples: 16463038. Policy #0 lag: (min: 25.0, avg: 46.2, max: 57.0) [2023-10-14 18:56:28,345][60425] Avg episode reward: [(0, '63.490'), (1, '59.660')] [2023-10-14 18:56:28,659][61552] Updated weights for policy 0, policy_version 32222 (0.0008) [2023-10-14 18:56:30,775][61585] Updated weights for policy 1, policy_version 32070 (0.0007) [2023-10-14 18:56:31,148][61585] Updated weights for policy 1, policy_version 32080 (0.0009) [2023-10-14 18:56:31,511][61585] Updated weights for policy 1, policy_version 32090 (0.0008) [2023-10-14 18:56:32,815][61552] Updated weights for policy 0, policy_version 32232 (0.0007) [2023-10-14 18:56:33,180][61552] Updated weights for policy 0, policy_version 32242 (0.0007) [2023-10-14 18:56:33,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 65863680. Throughput: 0: 1672.3, 1: 1664.6. Samples: 16473302. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 18:56:33,344][60425] Avg episode reward: [(0, '62.870'), (1, '57.450')] [2023-10-14 18:56:33,553][61552] Updated weights for policy 0, policy_version 32252 (0.0008) [2023-10-14 18:56:35,595][61585] Updated weights for policy 1, policy_version 32100 (0.0009) [2023-10-14 18:56:35,969][61585] Updated weights for policy 1, policy_version 32110 (0.0009) [2023-10-14 18:56:36,337][61585] Updated weights for policy 1, policy_version 32120 (0.0009) [2023-10-14 18:56:37,721][61552] Updated weights for policy 0, policy_version 32262 (0.0008) [2023-10-14 18:56:38,096][61552] Updated weights for policy 0, policy_version 32272 (0.0008) [2023-10-14 18:56:38,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 65929216. Throughput: 0: 1668.2, 1: 1656.7. Samples: 16492776. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 18:56:38,344][60425] Avg episode reward: [(0, '60.000'), (1, '60.320')] [2023-10-14 18:56:38,469][61552] Updated weights for policy 0, policy_version 32282 (0.0011) [2023-10-14 18:56:40,434][61585] Updated weights for policy 1, policy_version 32130 (0.0008) [2023-10-14 18:56:40,803][61585] Updated weights for policy 1, policy_version 32140 (0.0007) [2023-10-14 18:56:41,168][61585] Updated weights for policy 1, policy_version 32150 (0.0008) [2023-10-14 18:56:41,531][61585] Updated weights for policy 1, policy_version 32160 (0.0009) [2023-10-14 18:56:42,566][61552] Updated weights for policy 0, policy_version 32292 (0.0008) [2023-10-14 18:56:42,931][61552] Updated weights for policy 0, policy_version 32302 (0.0007) [2023-10-14 18:56:43,300][61552] Updated weights for policy 0, policy_version 32312 (0.0007) [2023-10-14 18:56:43,344][60425] Fps is (10 sec: 13106.7, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 65994752. Throughput: 0: 1664.6, 1: 1666.8. Samples: 16512984. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 18:56:43,345][60425] Avg episode reward: [(0, '61.980'), (1, '58.420')] [2023-10-14 18:56:45,823][61585] Updated weights for policy 1, policy_version 32170 (0.0009) [2023-10-14 18:56:46,190][61585] Updated weights for policy 1, policy_version 32180 (0.0008) [2023-10-14 18:56:46,559][61585] Updated weights for policy 1, policy_version 32190 (0.0010) [2023-10-14 18:56:47,214][61552] Updated weights for policy 0, policy_version 32322 (0.0009) [2023-10-14 18:56:47,581][61552] Updated weights for policy 0, policy_version 32332 (0.0007) [2023-10-14 18:56:47,956][61552] Updated weights for policy 0, policy_version 32342 (0.0009) [2023-10-14 18:56:48,331][61552] Updated weights for policy 0, policy_version 32352 (0.0009) [2023-10-14 18:56:48,343][60425] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 66093056. Throughput: 0: 1673.6, 1: 1651.6. Samples: 16523008. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 18:56:48,344][60425] Avg episode reward: [(0, '61.600'), (1, '57.530')] [2023-10-14 18:56:50,515][61585] Updated weights for policy 1, policy_version 32200 (0.0007) [2023-10-14 18:56:50,886][61585] Updated weights for policy 1, policy_version 32210 (0.0008) [2023-10-14 18:56:51,246][61585] Updated weights for policy 1, policy_version 32220 (0.0009) [2023-10-14 18:56:52,313][61552] Updated weights for policy 0, policy_version 32362 (0.0009) [2023-10-14 18:56:52,679][61552] Updated weights for policy 0, policy_version 32372 (0.0011) [2023-10-14 18:56:53,059][61552] Updated weights for policy 0, policy_version 32382 (0.0008) [2023-10-14 18:56:53,343][60425] Fps is (10 sec: 16384.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 66158592. Throughput: 0: 1672.2, 1: 1652.4. Samples: 16542610. Policy #0 lag: (min: 27.0, avg: 35.0, max: 59.0) [2023-10-14 18:56:53,344][60425] Avg episode reward: [(0, '62.270'), (1, '58.260')] [2023-10-14 18:56:55,343][61585] Updated weights for policy 1, policy_version 32230 (0.0009) [2023-10-14 18:56:55,711][61585] Updated weights for policy 1, policy_version 32240 (0.0009) [2023-10-14 18:56:56,079][61585] Updated weights for policy 1, policy_version 32250 (0.0010) [2023-10-14 18:56:57,279][61552] Updated weights for policy 0, policy_version 32392 (0.0007) [2023-10-14 18:56:57,645][61552] Updated weights for policy 0, policy_version 32402 (0.0008) [2023-10-14 18:56:58,014][61552] Updated weights for policy 0, policy_version 32412 (0.0008) [2023-10-14 18:56:58,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 66224128. Throughput: 0: 1651.0, 1: 1668.9. Samples: 16562418. Policy #0 lag: (min: 27.0, avg: 35.0, max: 59.0) [2023-10-14 18:56:58,344][60425] Avg episode reward: [(0, '62.450'), (1, '55.010')] [2023-10-14 18:57:00,134][61585] Updated weights for policy 1, policy_version 32260 (0.0009) [2023-10-14 18:57:00,497][61585] Updated weights for policy 1, policy_version 32270 (0.0010) [2023-10-14 18:57:00,863][61585] Updated weights for policy 1, policy_version 32280 (0.0007) [2023-10-14 18:57:02,147][61552] Updated weights for policy 0, policy_version 32422 (0.0008) [2023-10-14 18:57:02,521][61552] Updated weights for policy 0, policy_version 32432 (0.0009) [2023-10-14 18:57:02,908][61552] Updated weights for policy 0, policy_version 32442 (0.0009) [2023-10-14 18:57:03,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 66289664. Throughput: 0: 1670.2, 1: 1653.3. Samples: 16572680. Policy #0 lag: (min: 27.0, avg: 35.0, max: 59.0) [2023-10-14 18:57:03,344][60425] Avg episode reward: [(0, '62.320'), (1, '55.790')] [2023-10-14 18:57:05,080][61585] Updated weights for policy 1, policy_version 32290 (0.0009) [2023-10-14 18:57:05,464][61585] Updated weights for policy 1, policy_version 32300 (0.0009) [2023-10-14 18:57:05,834][61585] Updated weights for policy 1, policy_version 32310 (0.0008) [2023-10-14 18:57:06,194][61585] Updated weights for policy 1, policy_version 32320 (0.0007) [2023-10-14 18:57:06,859][61552] Updated weights for policy 0, policy_version 32452 (0.0010) [2023-10-14 18:57:07,229][61552] Updated weights for policy 0, policy_version 32462 (0.0009) [2023-10-14 18:57:07,597][61552] Updated weights for policy 0, policy_version 32472 (0.0008) [2023-10-14 18:57:08,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 66355200. Throughput: 0: 1666.4, 1: 1667.1. Samples: 16592758. Policy #0 lag: (min: 27.0, avg: 35.0, max: 59.0) [2023-10-14 18:57:08,344][60425] Avg episode reward: [(0, '60.230'), (1, '58.190')] [2023-10-14 18:57:10,293][61585] Updated weights for policy 1, policy_version 32330 (0.0011) [2023-10-14 18:57:10,667][61585] Updated weights for policy 1, policy_version 32340 (0.0008) [2023-10-14 18:57:11,042][61585] Updated weights for policy 1, policy_version 32350 (0.0008) [2023-10-14 18:57:11,763][61552] Updated weights for policy 0, policy_version 32482 (0.0008) [2023-10-14 18:57:12,130][61552] Updated weights for policy 0, policy_version 32492 (0.0009) [2023-10-14 18:57:12,501][61552] Updated weights for policy 0, policy_version 32502 (0.0008) [2023-10-14 18:57:12,870][61552] Updated weights for policy 0, policy_version 32512 (0.0009) [2023-10-14 18:57:13,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 66420736. Throughput: 0: 1646.0, 1: 1671.4. Samples: 16612320. Policy #0 lag: (min: 27.0, avg: 35.0, max: 59.0) [2023-10-14 18:57:13,344][60425] Avg episode reward: [(0, '60.410'), (1, '56.490')] [2023-10-14 18:57:15,026][61585] Updated weights for policy 1, policy_version 32360 (0.0009) [2023-10-14 18:57:15,396][61585] Updated weights for policy 1, policy_version 32370 (0.0010) [2023-10-14 18:57:15,767][61585] Updated weights for policy 1, policy_version 32380 (0.0008) [2023-10-14 18:57:16,775][61552] Updated weights for policy 0, policy_version 32522 (0.0010) [2023-10-14 18:57:17,130][61552] Updated weights for policy 0, policy_version 32532 (0.0009) [2023-10-14 18:57:17,498][61552] Updated weights for policy 0, policy_version 32542 (0.0008) [2023-10-14 18:57:18,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 66486272. Throughput: 0: 1671.5, 1: 1654.6. Samples: 16622974. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-10-14 18:57:18,345][60425] Avg episode reward: [(0, '62.940'), (1, '59.630')] [2023-10-14 18:57:19,874][61585] Updated weights for policy 1, policy_version 32390 (0.0010) [2023-10-14 18:57:20,244][61585] Updated weights for policy 1, policy_version 32400 (0.0007) [2023-10-14 18:57:20,613][61585] Updated weights for policy 1, policy_version 32410 (0.0008) [2023-10-14 18:57:21,634][61552] Updated weights for policy 0, policy_version 32552 (0.0011) [2023-10-14 18:57:21,998][61552] Updated weights for policy 0, policy_version 32562 (0.0010) [2023-10-14 18:57:22,364][61552] Updated weights for policy 0, policy_version 32572 (0.0009) [2023-10-14 18:57:23,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 66551808. Throughput: 0: 1665.0, 1: 1668.9. Samples: 16642804. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-10-14 18:57:23,344][60425] Avg episode reward: [(0, '63.240'), (1, '57.970')] [2023-10-14 18:57:24,785][61585] Updated weights for policy 1, policy_version 32420 (0.0009) [2023-10-14 18:57:25,145][61585] Updated weights for policy 1, policy_version 32430 (0.0009) [2023-10-14 18:57:25,512][61585] Updated weights for policy 1, policy_version 32440 (0.0009) [2023-10-14 18:57:26,509][61552] Updated weights for policy 0, policy_version 32582 (0.0008) [2023-10-14 18:57:26,878][61552] Updated weights for policy 0, policy_version 32592 (0.0007) [2023-10-14 18:57:27,246][61552] Updated weights for policy 0, policy_version 32602 (0.0007) [2023-10-14 18:57:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.3). Total num frames: 66617344. Throughput: 0: 1651.8, 1: 1673.0. Samples: 16662600. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-10-14 18:57:28,344][60425] Avg episode reward: [(0, '61.530'), (1, '56.500')] [2023-10-14 18:57:29,681][61585] Updated weights for policy 1, policy_version 32450 (0.0008) [2023-10-14 18:57:30,046][61585] Updated weights for policy 1, policy_version 32460 (0.0010) [2023-10-14 18:57:30,416][61585] Updated weights for policy 1, policy_version 32470 (0.0007) [2023-10-14 18:57:30,791][61585] Updated weights for policy 1, policy_version 32480 (0.0008) [2023-10-14 18:57:31,319][61552] Updated weights for policy 0, policy_version 32612 (0.0008) [2023-10-14 18:57:31,687][61552] Updated weights for policy 0, policy_version 32622 (0.0008) [2023-10-14 18:57:32,066][61552] Updated weights for policy 0, policy_version 32632 (0.0010) [2023-10-14 18:57:33,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 66682880. Throughput: 0: 1672.8, 1: 1657.1. Samples: 16672854. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-10-14 18:57:33,344][60425] Avg episode reward: [(0, '60.640'), (1, '56.080')] [2023-10-14 18:57:35,039][61585] Updated weights for policy 1, policy_version 32490 (0.0011) [2023-10-14 18:57:35,415][61585] Updated weights for policy 1, policy_version 32500 (0.0009) [2023-10-14 18:57:35,782][61585] Updated weights for policy 1, policy_version 32510 (0.0008) [2023-10-14 18:57:35,968][61552] Updated weights for policy 0, policy_version 32642 (0.0008) [2023-10-14 18:57:36,337][61552] Updated weights for policy 0, policy_version 32652 (0.0008) [2023-10-14 18:57:36,703][61552] Updated weights for policy 0, policy_version 32662 (0.0008) [2023-10-14 18:57:37,070][61552] Updated weights for policy 0, policy_version 32672 (0.0007) [2023-10-14 18:57:38,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 66748416. Throughput: 0: 1657.5, 1: 1670.5. Samples: 16692370. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-10-14 18:57:38,344][60425] Avg episode reward: [(0, '62.760'), (1, '58.440')] [2023-10-14 18:57:40,003][61585] Updated weights for policy 1, policy_version 32520 (0.0007) [2023-10-14 18:57:40,358][61585] Updated weights for policy 1, policy_version 32530 (0.0010) [2023-10-14 18:57:40,723][61585] Updated weights for policy 1, policy_version 32540 (0.0008) [2023-10-14 18:57:41,207][61552] Updated weights for policy 0, policy_version 32682 (0.0009) [2023-10-14 18:57:41,578][61552] Updated weights for policy 0, policy_version 32692 (0.0007) [2023-10-14 18:57:41,954][61552] Updated weights for policy 0, policy_version 32702 (0.0007) [2023-10-14 18:57:43,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 66813952. Throughput: 0: 1670.2, 1: 1663.5. Samples: 16712432. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-10-14 18:57:43,344][60425] Avg episode reward: [(0, '62.940'), (1, '53.960')] [2023-10-14 18:57:43,355][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000032704_33488896.pth... [2023-10-14 18:57:43,355][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000032544_33325056.pth... [2023-10-14 18:57:43,389][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000031008_31752192.pth [2023-10-14 18:57:43,395][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000031136_31883264.pth [2023-10-14 18:57:44,876][61585] Updated weights for policy 1, policy_version 32550 (0.0008) [2023-10-14 18:57:45,242][61585] Updated weights for policy 1, policy_version 32560 (0.0009) [2023-10-14 18:57:45,620][61585] Updated weights for policy 1, policy_version 32570 (0.0009) [2023-10-14 18:57:46,157][61552] Updated weights for policy 0, policy_version 32712 (0.0009) [2023-10-14 18:57:46,539][61552] Updated weights for policy 0, policy_version 32722 (0.0011) [2023-10-14 18:57:46,895][61552] Updated weights for policy 0, policy_version 32732 (0.0011) [2023-10-14 18:57:48,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 66879488. Throughput: 0: 1683.2, 1: 1654.9. Samples: 16722894. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-10-14 18:57:48,344][60425] Avg episode reward: [(0, '62.960'), (1, '60.840')] [2023-10-14 18:57:49,676][61585] Updated weights for policy 1, policy_version 32580 (0.0008) [2023-10-14 18:57:50,045][61585] Updated weights for policy 1, policy_version 32590 (0.0007) [2023-10-14 18:57:50,421][61585] Updated weights for policy 1, policy_version 32600 (0.0010) [2023-10-14 18:57:50,886][61552] Updated weights for policy 0, policy_version 32742 (0.0009) [2023-10-14 18:57:51,263][61552] Updated weights for policy 0, policy_version 32752 (0.0009) [2023-10-14 18:57:51,630][61552] Updated weights for policy 0, policy_version 32762 (0.0007) [2023-10-14 18:57:53,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 66945024. Throughput: 0: 1655.1, 1: 1665.0. Samples: 16742164. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-10-14 18:57:53,344][60425] Avg episode reward: [(0, '64.360'), (1, '56.340')] [2023-10-14 18:57:54,549][61585] Updated weights for policy 1, policy_version 32610 (0.0009) [2023-10-14 18:57:54,910][61585] Updated weights for policy 1, policy_version 32620 (0.0010) [2023-10-14 18:57:55,287][61585] Updated weights for policy 1, policy_version 32630 (0.0009) [2023-10-14 18:57:55,653][61585] Updated weights for policy 1, policy_version 32640 (0.0008) [2023-10-14 18:57:55,890][61552] Updated weights for policy 0, policy_version 32772 (0.0008) [2023-10-14 18:57:56,254][61552] Updated weights for policy 0, policy_version 32782 (0.0009) [2023-10-14 18:57:56,618][61552] Updated weights for policy 0, policy_version 32792 (0.0008) [2023-10-14 18:57:58,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 67010560. Throughput: 0: 1669.4, 1: 1662.7. Samples: 16762264. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-10-14 18:57:58,344][60425] Avg episode reward: [(0, '64.010'), (1, '57.100')] [2023-10-14 18:57:59,764][61585] Updated weights for policy 1, policy_version 32650 (0.0008) [2023-10-14 18:58:00,135][61585] Updated weights for policy 1, policy_version 32660 (0.0009) [2023-10-14 18:58:00,500][61585] Updated weights for policy 1, policy_version 32670 (0.0008) [2023-10-14 18:58:00,782][61552] Updated weights for policy 0, policy_version 32802 (0.0008) [2023-10-14 18:58:01,149][61552] Updated weights for policy 0, policy_version 32812 (0.0010) [2023-10-14 18:58:01,513][61552] Updated weights for policy 0, policy_version 32822 (0.0011) [2023-10-14 18:58:01,888][61552] Updated weights for policy 0, policy_version 32832 (0.0011) [2023-10-14 18:58:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 67076096. Throughput: 0: 1673.4, 1: 1656.2. Samples: 16772808. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-10-14 18:58:03,344][60425] Avg episode reward: [(0, '63.130'), (1, '58.320')] [2023-10-14 18:58:04,299][61585] Updated weights for policy 1, policy_version 32680 (0.0010) [2023-10-14 18:58:04,667][61585] Updated weights for policy 1, policy_version 32690 (0.0009) [2023-10-14 18:58:05,035][61585] Updated weights for policy 1, policy_version 32700 (0.0008) [2023-10-14 18:58:05,846][61552] Updated weights for policy 0, policy_version 32842 (0.0008) [2023-10-14 18:58:06,207][61552] Updated weights for policy 0, policy_version 32852 (0.0009) [2023-10-14 18:58:06,589][61552] Updated weights for policy 0, policy_version 32862 (0.0009) [2023-10-14 18:58:08,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 67141632. Throughput: 0: 1661.0, 1: 1665.4. Samples: 16792490. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-14 18:58:08,344][60425] Avg episode reward: [(0, '63.300'), (1, '60.040')] [2023-10-14 18:58:09,143][61585] Updated weights for policy 1, policy_version 32710 (0.0008) [2023-10-14 18:58:09,503][61585] Updated weights for policy 1, policy_version 32720 (0.0009) [2023-10-14 18:58:09,864][61585] Updated weights for policy 1, policy_version 32730 (0.0008) [2023-10-14 18:58:10,538][61552] Updated weights for policy 0, policy_version 32872 (0.0007) [2023-10-14 18:58:10,893][61552] Updated weights for policy 0, policy_version 32882 (0.0009) [2023-10-14 18:58:11,268][61552] Updated weights for policy 0, policy_version 32892 (0.0009) [2023-10-14 18:58:13,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 67207168. Throughput: 0: 1689.7, 1: 1669.3. Samples: 16813756. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-14 18:58:13,345][60425] Avg episode reward: [(0, '60.530'), (1, '61.130')] [2023-10-14 18:58:13,692][61585] Updated weights for policy 1, policy_version 32740 (0.0007) [2023-10-14 18:58:14,061][61585] Updated weights for policy 1, policy_version 32750 (0.0008) [2023-10-14 18:58:14,420][61585] Updated weights for policy 1, policy_version 32760 (0.0008) [2023-10-14 18:58:15,294][61552] Updated weights for policy 0, policy_version 32902 (0.0010) [2023-10-14 18:58:15,660][61552] Updated weights for policy 0, policy_version 32912 (0.0011) [2023-10-14 18:58:16,030][61552] Updated weights for policy 0, policy_version 32922 (0.0008) [2023-10-14 18:58:18,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 67272704. Throughput: 0: 1673.6, 1: 1674.9. Samples: 16823538. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-14 18:58:18,344][60425] Avg episode reward: [(0, '61.790'), (1, '55.650')] [2023-10-14 18:58:18,478][61585] Updated weights for policy 1, policy_version 32770 (0.0008) [2023-10-14 18:58:18,835][61585] Updated weights for policy 1, policy_version 32780 (0.0010) [2023-10-14 18:58:19,196][61585] Updated weights for policy 1, policy_version 32790 (0.0007) [2023-10-14 18:58:19,563][61585] Updated weights for policy 1, policy_version 32800 (0.0008) [2023-10-14 18:58:20,172][61552] Updated weights for policy 0, policy_version 32932 (0.0008) [2023-10-14 18:58:20,544][61552] Updated weights for policy 0, policy_version 32942 (0.0007) [2023-10-14 18:58:20,912][61552] Updated weights for policy 0, policy_version 32952 (0.0009) [2023-10-14 18:58:23,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 67338240. Throughput: 0: 1674.7, 1: 1685.3. Samples: 16843566. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-14 18:58:23,344][60425] Avg episode reward: [(0, '62.350'), (1, '55.900')] [2023-10-14 18:58:23,753][61585] Updated weights for policy 1, policy_version 32810 (0.0009) [2023-10-14 18:58:24,131][61585] Updated weights for policy 1, policy_version 32820 (0.0010) [2023-10-14 18:58:24,495][61585] Updated weights for policy 1, policy_version 32830 (0.0009) [2023-10-14 18:58:24,944][61552] Updated weights for policy 0, policy_version 32962 (0.0009) [2023-10-14 18:58:25,323][61552] Updated weights for policy 0, policy_version 32972 (0.0008) [2023-10-14 18:58:25,699][61552] Updated weights for policy 0, policy_version 32982 (0.0011) [2023-10-14 18:58:26,067][61552] Updated weights for policy 0, policy_version 32992 (0.0009) [2023-10-14 18:58:28,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 67403776. Throughput: 0: 1692.8, 1: 1682.7. Samples: 16864328. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-14 18:58:28,344][60425] Avg episode reward: [(0, '62.160'), (1, '54.690')] [2023-10-14 18:58:28,633][61585] Updated weights for policy 1, policy_version 32840 (0.0010) [2023-10-14 18:58:29,006][61585] Updated weights for policy 1, policy_version 32850 (0.0009) [2023-10-14 18:58:29,378][61585] Updated weights for policy 1, policy_version 32860 (0.0008) [2023-10-14 18:58:29,938][61552] Updated weights for policy 0, policy_version 33002 (0.0008) [2023-10-14 18:58:30,313][61552] Updated weights for policy 0, policy_version 33012 (0.0007) [2023-10-14 18:58:30,686][61552] Updated weights for policy 0, policy_version 33022 (0.0010) [2023-10-14 18:58:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 67469312. Throughput: 0: 1668.2, 1: 1680.2. Samples: 16873572. Policy #0 lag: (min: 13.0, avg: 20.7, max: 45.0) [2023-10-14 18:58:33,344][60425] Avg episode reward: [(0, '61.460'), (1, '55.600')] [2023-10-14 18:58:33,519][61585] Updated weights for policy 1, policy_version 32870 (0.0010) [2023-10-14 18:58:33,872][61585] Updated weights for policy 1, policy_version 32880 (0.0010) [2023-10-14 18:58:34,249][61585] Updated weights for policy 1, policy_version 32890 (0.0008) [2023-10-14 18:58:34,676][61552] Updated weights for policy 0, policy_version 33032 (0.0008) [2023-10-14 18:58:35,036][61552] Updated weights for policy 0, policy_version 33042 (0.0009) [2023-10-14 18:58:35,409][61552] Updated weights for policy 0, policy_version 33052 (0.0008) [2023-10-14 18:58:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 67534848. Throughput: 0: 1694.3, 1: 1680.8. Samples: 16894042. Policy #0 lag: (min: 13.0, avg: 20.7, max: 45.0) [2023-10-14 18:58:38,344][60425] Avg episode reward: [(0, '60.070'), (1, '58.000')] [2023-10-14 18:58:38,367][61585] Updated weights for policy 1, policy_version 32900 (0.0008) [2023-10-14 18:58:38,741][61585] Updated weights for policy 1, policy_version 32910 (0.0008) [2023-10-14 18:58:39,109][61585] Updated weights for policy 1, policy_version 32920 (0.0007) [2023-10-14 18:58:39,659][61552] Updated weights for policy 0, policy_version 33062 (0.0009) [2023-10-14 18:58:40,031][61552] Updated weights for policy 0, policy_version 33072 (0.0011) [2023-10-14 18:58:40,411][61552] Updated weights for policy 0, policy_version 33082 (0.0008) [2023-10-14 18:58:43,239][61585] Updated weights for policy 1, policy_version 32930 (0.0007) [2023-10-14 18:58:43,344][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 67600384. Throughput: 0: 1703.7, 1: 1683.6. Samples: 16914690. Policy #0 lag: (min: 13.0, avg: 20.7, max: 45.0) [2023-10-14 18:58:43,345][60425] Avg episode reward: [(0, '62.640'), (1, '54.640')] [2023-10-14 18:58:43,594][61585] Updated weights for policy 1, policy_version 32940 (0.0009) [2023-10-14 18:58:43,965][61585] Updated weights for policy 1, policy_version 32950 (0.0008) [2023-10-14 18:58:44,324][61585] Updated weights for policy 1, policy_version 32960 (0.0009) [2023-10-14 18:58:44,406][61552] Updated weights for policy 0, policy_version 33092 (0.0008) [2023-10-14 18:58:44,772][61552] Updated weights for policy 0, policy_version 33102 (0.0007) [2023-10-14 18:58:45,136][61552] Updated weights for policy 0, policy_version 33112 (0.0007) [2023-10-14 18:58:48,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 67665920. Throughput: 0: 1672.8, 1: 1683.5. Samples: 16923840. Policy #0 lag: (min: 13.0, avg: 20.7, max: 45.0) [2023-10-14 18:58:48,344][60425] Avg episode reward: [(0, '64.490'), (1, '57.840')] [2023-10-14 18:58:48,548][61585] Updated weights for policy 1, policy_version 32970 (0.0008) [2023-10-14 18:58:48,922][61585] Updated weights for policy 1, policy_version 32980 (0.0007) [2023-10-14 18:58:49,257][61552] Updated weights for policy 0, policy_version 33122 (0.0008) [2023-10-14 18:58:49,287][61585] Updated weights for policy 1, policy_version 32990 (0.0009) [2023-10-14 18:58:49,630][61552] Updated weights for policy 0, policy_version 33132 (0.0010) [2023-10-14 18:58:50,006][61552] Updated weights for policy 0, policy_version 33142 (0.0008) [2023-10-14 18:58:50,372][61552] Updated weights for policy 0, policy_version 33152 (0.0008) [2023-10-14 18:58:53,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 67731456. Throughput: 0: 1693.7, 1: 1676.3. Samples: 16944142. Policy #0 lag: (min: 13.0, avg: 20.7, max: 45.0) [2023-10-14 18:58:53,344][60425] Avg episode reward: [(0, '65.520'), (1, '57.640')] [2023-10-14 18:58:53,425][61585] Updated weights for policy 1, policy_version 33000 (0.0009) [2023-10-14 18:58:53,797][61585] Updated weights for policy 1, policy_version 33010 (0.0008) [2023-10-14 18:58:54,155][61585] Updated weights for policy 1, policy_version 33020 (0.0008) [2023-10-14 18:58:54,334][61552] Updated weights for policy 0, policy_version 33162 (0.0007) [2023-10-14 18:58:54,706][61552] Updated weights for policy 0, policy_version 33172 (0.0009) [2023-10-14 18:58:55,075][61552] Updated weights for policy 0, policy_version 33182 (0.0008) [2023-10-14 18:58:58,249][61585] Updated weights for policy 1, policy_version 33030 (0.0010) [2023-10-14 18:58:58,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 67796992. Throughput: 0: 1686.7, 1: 1672.4. Samples: 16964912. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-14 18:58:58,344][60425] Avg episode reward: [(0, '64.710'), (1, '55.750')] [2023-10-14 18:58:58,622][61585] Updated weights for policy 1, policy_version 33040 (0.0008) [2023-10-14 18:58:58,981][61585] Updated weights for policy 1, policy_version 33050 (0.0009) [2023-10-14 18:58:59,125][61552] Updated weights for policy 0, policy_version 33192 (0.0009) [2023-10-14 18:58:59,492][61552] Updated weights for policy 0, policy_version 33202 (0.0008) [2023-10-14 18:58:59,865][61552] Updated weights for policy 0, policy_version 33212 (0.0007) [2023-10-14 18:59:03,099][61585] Updated weights for policy 1, policy_version 33060 (0.0008) [2023-10-14 18:59:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 67862528. Throughput: 0: 1674.3, 1: 1668.9. Samples: 16973980. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-14 18:59:03,344][60425] Avg episode reward: [(0, '68.430'), (1, '57.520')] [2023-10-14 18:59:03,344][61172] Saving new best policy, reward=68.430! [2023-10-14 18:59:03,454][61585] Updated weights for policy 1, policy_version 33070 (0.0008) [2023-10-14 18:59:03,822][61585] Updated weights for policy 1, policy_version 33080 (0.0009) [2023-10-14 18:59:03,980][61552] Updated weights for policy 0, policy_version 33222 (0.0008) [2023-10-14 18:59:04,340][61552] Updated weights for policy 0, policy_version 33232 (0.0009) [2023-10-14 18:59:04,708][61552] Updated weights for policy 0, policy_version 33242 (0.0010) [2023-10-14 18:59:08,050][61585] Updated weights for policy 1, policy_version 33090 (0.0008) [2023-10-14 18:59:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 67928064. Throughput: 0: 1690.3, 1: 1669.1. Samples: 16994736. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-14 18:59:08,344][60425] Avg episode reward: [(0, '64.800'), (1, '54.800')] [2023-10-14 18:59:08,419][61585] Updated weights for policy 1, policy_version 33100 (0.0011) [2023-10-14 18:59:08,777][61585] Updated weights for policy 1, policy_version 33110 (0.0008) [2023-10-14 18:59:08,868][61552] Updated weights for policy 0, policy_version 33252 (0.0009) [2023-10-14 18:59:09,150][61585] Updated weights for policy 1, policy_version 33120 (0.0008) [2023-10-14 18:59:09,231][61552] Updated weights for policy 0, policy_version 33262 (0.0007) [2023-10-14 18:59:09,599][61552] Updated weights for policy 0, policy_version 33272 (0.0008) [2023-10-14 18:59:13,326][61585] Updated weights for policy 1, policy_version 33130 (0.0009) [2023-10-14 18:59:13,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 67993600. Throughput: 0: 1683.3, 1: 1673.5. Samples: 17015386. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-14 18:59:13,344][60425] Avg episode reward: [(0, '68.420'), (1, '58.740')] [2023-10-14 18:59:13,691][61552] Updated weights for policy 0, policy_version 33282 (0.0008) [2023-10-14 18:59:13,691][61585] Updated weights for policy 1, policy_version 33140 (0.0007) [2023-10-14 18:59:14,048][61552] Updated weights for policy 0, policy_version 33292 (0.0009) [2023-10-14 18:59:14,057][61585] Updated weights for policy 1, policy_version 33150 (0.0008) [2023-10-14 18:59:14,423][61552] Updated weights for policy 0, policy_version 33302 (0.0011) [2023-10-14 18:59:14,795][61552] Updated weights for policy 0, policy_version 33312 (0.0011) [2023-10-14 18:59:18,028][61585] Updated weights for policy 1, policy_version 33160 (0.0008) [2023-10-14 18:59:18,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 68059136. Throughput: 0: 1677.9, 1: 1671.2. Samples: 17024280. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-14 18:59:18,344][60425] Avg episode reward: [(0, '66.110'), (1, '55.770')] [2023-10-14 18:59:18,405][61585] Updated weights for policy 1, policy_version 33170 (0.0009) [2023-10-14 18:59:18,773][61585] Updated weights for policy 1, policy_version 33180 (0.0007) [2023-10-14 18:59:18,925][61552] Updated weights for policy 0, policy_version 33322 (0.0011) [2023-10-14 18:59:19,288][61552] Updated weights for policy 0, policy_version 33332 (0.0009) [2023-10-14 18:59:19,651][61552] Updated weights for policy 0, policy_version 33342 (0.0009) [2023-10-14 18:59:22,833][61585] Updated weights for policy 1, policy_version 33190 (0.0008) [2023-10-14 18:59:23,194][61585] Updated weights for policy 1, policy_version 33200 (0.0008) [2023-10-14 18:59:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 68124672. Throughput: 0: 1682.5, 1: 1672.0. Samples: 17044996. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:59:23,345][60425] Avg episode reward: [(0, '63.670'), (1, '57.580')] [2023-10-14 18:59:23,560][61585] Updated weights for policy 1, policy_version 33210 (0.0008) [2023-10-14 18:59:23,740][61552] Updated weights for policy 0, policy_version 33352 (0.0008) [2023-10-14 18:59:24,107][61552] Updated weights for policy 0, policy_version 33362 (0.0011) [2023-10-14 18:59:24,476][61552] Updated weights for policy 0, policy_version 33372 (0.0010) [2023-10-14 18:59:27,558][61585] Updated weights for policy 1, policy_version 33220 (0.0009) [2023-10-14 18:59:27,925][61585] Updated weights for policy 1, policy_version 33230 (0.0009) [2023-10-14 18:59:28,286][61585] Updated weights for policy 1, policy_version 33240 (0.0007) [2023-10-14 18:59:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 68190208. Throughput: 0: 1672.8, 1: 1666.8. Samples: 17064970. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:59:28,344][60425] Avg episode reward: [(0, '61.100'), (1, '58.550')] [2023-10-14 18:59:28,723][61552] Updated weights for policy 0, policy_version 33382 (0.0008) [2023-10-14 18:59:29,088][61552] Updated weights for policy 0, policy_version 33392 (0.0009) [2023-10-14 18:59:29,472][61552] Updated weights for policy 0, policy_version 33402 (0.0009) [2023-10-14 18:59:32,385][61585] Updated weights for policy 1, policy_version 33250 (0.0008) [2023-10-14 18:59:32,748][61585] Updated weights for policy 1, policy_version 33260 (0.0007) [2023-10-14 18:59:33,111][61585] Updated weights for policy 1, policy_version 33270 (0.0007) [2023-10-14 18:59:33,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 68255744. Throughput: 0: 1673.8, 1: 1673.6. Samples: 17074474. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:59:33,344][60425] Avg episode reward: [(0, '61.730'), (1, '58.950')] [2023-10-14 18:59:33,443][61552] Updated weights for policy 0, policy_version 33412 (0.0007) [2023-10-14 18:59:33,472][61585] Updated weights for policy 1, policy_version 33280 (0.0007) [2023-10-14 18:59:33,815][61552] Updated weights for policy 0, policy_version 33422 (0.0007) [2023-10-14 18:59:34,188][61552] Updated weights for policy 0, policy_version 33432 (0.0009) [2023-10-14 18:59:37,612][61585] Updated weights for policy 1, policy_version 33290 (0.0008) [2023-10-14 18:59:37,986][61585] Updated weights for policy 1, policy_version 33300 (0.0008) [2023-10-14 18:59:38,265][61552] Updated weights for policy 0, policy_version 33442 (0.0009) [2023-10-14 18:59:38,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 68321280. Throughput: 0: 1674.3, 1: 1679.9. Samples: 17095078. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:59:38,344][60425] Avg episode reward: [(0, '62.340'), (1, '64.950')] [2023-10-14 18:59:38,348][61585] Updated weights for policy 1, policy_version 33310 (0.0009) [2023-10-14 18:59:38,419][61248] Saving new best policy, reward=64.950! [2023-10-14 18:59:38,641][61552] Updated weights for policy 0, policy_version 33452 (0.0008) [2023-10-14 18:59:39,003][61552] Updated weights for policy 0, policy_version 33462 (0.0007) [2023-10-14 18:59:39,376][61552] Updated weights for policy 0, policy_version 33472 (0.0007) [2023-10-14 18:59:42,302][61585] Updated weights for policy 1, policy_version 33320 (0.0009) [2023-10-14 18:59:42,668][61585] Updated weights for policy 1, policy_version 33330 (0.0009) [2023-10-14 18:59:43,035][61585] Updated weights for policy 1, policy_version 33340 (0.0010) [2023-10-14 18:59:43,344][60425] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 68419584. Throughput: 0: 1675.1, 1: 1665.3. Samples: 17115232. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 18:59:43,345][60425] Avg episode reward: [(0, '60.730'), (1, '62.280')] [2023-10-14 18:59:43,356][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000033344_34144256.pth... [2023-10-14 18:59:43,388][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000031776_32538624.pth [2023-10-14 18:59:43,423][61552] Updated weights for policy 0, policy_version 33482 (0.0008) [2023-10-14 18:59:43,788][61552] Updated weights for policy 0, policy_version 33492 (0.0010) [2023-10-14 18:59:44,162][61552] Updated weights for policy 0, policy_version 33502 (0.0009) [2023-10-14 18:59:44,237][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000033504_34308096.pth... [2023-10-14 18:59:44,270][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000031936_32702464.pth [2023-10-14 18:59:47,219][61585] Updated weights for policy 1, policy_version 33350 (0.0008) [2023-10-14 18:59:47,581][61585] Updated weights for policy 1, policy_version 33360 (0.0009) [2023-10-14 18:59:47,946][61585] Updated weights for policy 1, policy_version 33370 (0.0008) [2023-10-14 18:59:48,203][61552] Updated weights for policy 0, policy_version 33512 (0.0008) [2023-10-14 18:59:48,343][60425] Fps is (10 sec: 16384.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 68485120. Throughput: 0: 1676.6, 1: 1678.4. Samples: 17124956. Policy #0 lag: (min: 17.0, avg: 33.1, max: 49.0) [2023-10-14 18:59:48,344][60425] Avg episode reward: [(0, '60.840'), (1, '60.360')] [2023-10-14 18:59:48,575][61552] Updated weights for policy 0, policy_version 33522 (0.0008) [2023-10-14 18:59:48,944][61552] Updated weights for policy 0, policy_version 33532 (0.0008) [2023-10-14 18:59:52,072][61585] Updated weights for policy 1, policy_version 33380 (0.0007) [2023-10-14 18:59:52,426][61585] Updated weights for policy 1, policy_version 33390 (0.0009) [2023-10-14 18:59:52,793][61585] Updated weights for policy 1, policy_version 33400 (0.0008) [2023-10-14 18:59:53,081][61552] Updated weights for policy 0, policy_version 33542 (0.0009) [2023-10-14 18:59:53,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 68550656. Throughput: 0: 1671.1, 1: 1675.9. Samples: 17145348. Policy #0 lag: (min: 17.0, avg: 33.1, max: 49.0) [2023-10-14 18:59:53,344][60425] Avg episode reward: [(0, '61.910'), (1, '59.160')] [2023-10-14 18:59:53,452][61552] Updated weights for policy 0, policy_version 33552 (0.0009) [2023-10-14 18:59:53,830][61552] Updated weights for policy 0, policy_version 33562 (0.0010) [2023-10-14 18:59:56,911][61585] Updated weights for policy 1, policy_version 33410 (0.0009) [2023-10-14 18:59:57,281][61585] Updated weights for policy 1, policy_version 33420 (0.0011) [2023-10-14 18:59:57,653][61585] Updated weights for policy 1, policy_version 33430 (0.0011) [2023-10-14 18:59:57,979][61552] Updated weights for policy 0, policy_version 33572 (0.0008) [2023-10-14 18:59:58,016][61585] Updated weights for policy 1, policy_version 33440 (0.0009) [2023-10-14 18:59:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 68616192. Throughput: 0: 1673.1, 1: 1650.8. Samples: 17164958. Policy #0 lag: (min: 17.0, avg: 33.1, max: 49.0) [2023-10-14 18:59:58,344][60425] Avg episode reward: [(0, '64.130'), (1, '57.750')] [2023-10-14 18:59:58,355][61552] Updated weights for policy 0, policy_version 33582 (0.0010) [2023-10-14 18:59:58,720][61552] Updated weights for policy 0, policy_version 33592 (0.0009) [2023-10-14 19:00:02,177][61585] Updated weights for policy 1, policy_version 33450 (0.0009) [2023-10-14 19:00:02,546][61585] Updated weights for policy 1, policy_version 33460 (0.0009) [2023-10-14 19:00:02,758][61552] Updated weights for policy 0, policy_version 33602 (0.0009) [2023-10-14 19:00:02,903][61585] Updated weights for policy 1, policy_version 33470 (0.0008) [2023-10-14 19:00:03,121][61552] Updated weights for policy 0, policy_version 33612 (0.0009) [2023-10-14 19:00:03,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 68681728. Throughput: 0: 1675.8, 1: 1672.3. Samples: 17174946. Policy #0 lag: (min: 17.0, avg: 33.1, max: 49.0) [2023-10-14 19:00:03,344][60425] Avg episode reward: [(0, '61.490'), (1, '59.840')] [2023-10-14 19:00:03,492][61552] Updated weights for policy 0, policy_version 33622 (0.0007) [2023-10-14 19:00:03,864][61552] Updated weights for policy 0, policy_version 33632 (0.0009) [2023-10-14 19:00:07,086][61585] Updated weights for policy 1, policy_version 33480 (0.0008) [2023-10-14 19:00:07,453][61585] Updated weights for policy 1, policy_version 33490 (0.0008) [2023-10-14 19:00:07,820][61585] Updated weights for policy 1, policy_version 33500 (0.0009) [2023-10-14 19:00:07,920][61552] Updated weights for policy 0, policy_version 33642 (0.0008) [2023-10-14 19:00:08,284][61552] Updated weights for policy 0, policy_version 33652 (0.0007) [2023-10-14 19:00:08,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 68747264. Throughput: 0: 1670.6, 1: 1671.6. Samples: 17195396. Policy #0 lag: (min: 17.0, avg: 33.1, max: 49.0) [2023-10-14 19:00:08,344][60425] Avg episode reward: [(0, '64.880'), (1, '60.570')] [2023-10-14 19:00:08,648][61552] Updated weights for policy 0, policy_version 33662 (0.0008) [2023-10-14 19:00:11,829][61585] Updated weights for policy 1, policy_version 33510 (0.0010) [2023-10-14 19:00:12,191][61585] Updated weights for policy 1, policy_version 33520 (0.0011) [2023-10-14 19:00:12,550][61585] Updated weights for policy 1, policy_version 33530 (0.0007) [2023-10-14 19:00:12,733][61552] Updated weights for policy 0, policy_version 33672 (0.0007) [2023-10-14 19:00:13,114][61552] Updated weights for policy 0, policy_version 33682 (0.0011) [2023-10-14 19:00:13,344][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 68812800. Throughput: 0: 1675.5, 1: 1652.6. Samples: 17214736. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 19:00:13,345][60425] Avg episode reward: [(0, '63.380'), (1, '58.090')] [2023-10-14 19:00:13,481][61552] Updated weights for policy 0, policy_version 33692 (0.0010) [2023-10-14 19:00:16,627][61585] Updated weights for policy 1, policy_version 33540 (0.0007) [2023-10-14 19:00:16,986][61585] Updated weights for policy 1, policy_version 33550 (0.0007) [2023-10-14 19:00:17,354][61585] Updated weights for policy 1, policy_version 33560 (0.0008) [2023-10-14 19:00:17,644][61552] Updated weights for policy 0, policy_version 33702 (0.0007) [2023-10-14 19:00:18,014][61552] Updated weights for policy 0, policy_version 33712 (0.0007) [2023-10-14 19:00:18,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 68878336. Throughput: 0: 1676.9, 1: 1670.3. Samples: 17225096. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 19:00:18,344][60425] Avg episode reward: [(0, '63.580'), (1, '58.980')] [2023-10-14 19:00:18,386][61552] Updated weights for policy 0, policy_version 33722 (0.0009) [2023-10-14 19:00:21,409][61585] Updated weights for policy 1, policy_version 33570 (0.0009) [2023-10-14 19:00:21,770][61585] Updated weights for policy 1, policy_version 33580 (0.0009) [2023-10-14 19:00:22,133][61585] Updated weights for policy 1, policy_version 33590 (0.0011) [2023-10-14 19:00:22,487][61585] Updated weights for policy 1, policy_version 33600 (0.0008) [2023-10-14 19:00:22,543][61552] Updated weights for policy 0, policy_version 33732 (0.0007) [2023-10-14 19:00:22,913][61552] Updated weights for policy 0, policy_version 33742 (0.0009) [2023-10-14 19:00:23,275][61552] Updated weights for policy 0, policy_version 33752 (0.0009) [2023-10-14 19:00:23,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 68943872. Throughput: 0: 1677.0, 1: 1662.0. Samples: 17245330. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 19:00:23,344][60425] Avg episode reward: [(0, '65.010'), (1, '59.320')] [2023-10-14 19:00:26,665][61585] Updated weights for policy 1, policy_version 33610 (0.0008) [2023-10-14 19:00:27,033][61585] Updated weights for policy 1, policy_version 33620 (0.0009) [2023-10-14 19:00:27,393][61585] Updated weights for policy 1, policy_version 33630 (0.0009) [2023-10-14 19:00:27,423][61552] Updated weights for policy 0, policy_version 33762 (0.0009) [2023-10-14 19:00:27,790][61552] Updated weights for policy 0, policy_version 33772 (0.0010) [2023-10-14 19:00:28,164][61552] Updated weights for policy 0, policy_version 33782 (0.0008) [2023-10-14 19:00:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 69009408. Throughput: 0: 1662.7, 1: 1660.1. Samples: 17264758. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 19:00:28,344][60425] Avg episode reward: [(0, '64.840'), (1, '55.200')] [2023-10-14 19:00:28,527][61552] Updated weights for policy 0, policy_version 33792 (0.0010) [2023-10-14 19:00:31,597][61585] Updated weights for policy 1, policy_version 33640 (0.0009) [2023-10-14 19:00:31,960][61585] Updated weights for policy 1, policy_version 33650 (0.0010) [2023-10-14 19:00:32,330][61585] Updated weights for policy 1, policy_version 33660 (0.0007) [2023-10-14 19:00:32,447][61552] Updated weights for policy 0, policy_version 33802 (0.0008) [2023-10-14 19:00:32,824][61552] Updated weights for policy 0, policy_version 33812 (0.0007) [2023-10-14 19:00:33,193][61552] Updated weights for policy 0, policy_version 33822 (0.0008) [2023-10-14 19:00:33,343][60425] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 69107712. Throughput: 0: 1666.1, 1: 1669.2. Samples: 17275042. Policy #0 lag: (min: 23.0, avg: 46.6, max: 48.0) [2023-10-14 19:00:33,344][60425] Avg episode reward: [(0, '61.250'), (1, '59.330')] [2023-10-14 19:00:36,527][61585] Updated weights for policy 1, policy_version 33670 (0.0008) [2023-10-14 19:00:36,887][61585] Updated weights for policy 1, policy_version 33680 (0.0009) [2023-10-14 19:00:37,260][61585] Updated weights for policy 1, policy_version 33690 (0.0009) [2023-10-14 19:00:37,286][61552] Updated weights for policy 0, policy_version 33832 (0.0008) [2023-10-14 19:00:37,653][61552] Updated weights for policy 0, policy_version 33842 (0.0009) [2023-10-14 19:00:38,028][61552] Updated weights for policy 0, policy_version 33852 (0.0007) [2023-10-14 19:00:38,343][60425] Fps is (10 sec: 16383.7, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 69173248. Throughput: 0: 1670.7, 1: 1657.2. Samples: 17295102. Policy #0 lag: (min: 23.0, avg: 46.6, max: 48.0) [2023-10-14 19:00:38,344][60425] Avg episode reward: [(0, '63.560'), (1, '57.960')] [2023-10-14 19:00:41,372][61585] Updated weights for policy 1, policy_version 33700 (0.0008) [2023-10-14 19:00:41,739][61585] Updated weights for policy 1, policy_version 33710 (0.0009) [2023-10-14 19:00:41,962][61552] Updated weights for policy 0, policy_version 33862 (0.0007) [2023-10-14 19:00:42,111][61585] Updated weights for policy 1, policy_version 33720 (0.0009) [2023-10-14 19:00:42,325][61552] Updated weights for policy 0, policy_version 33872 (0.0007) [2023-10-14 19:00:42,692][61552] Updated weights for policy 0, policy_version 33882 (0.0007) [2023-10-14 19:00:43,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 69238784. Throughput: 0: 1651.2, 1: 1661.2. Samples: 17314016. Policy #0 lag: (min: 23.0, avg: 46.6, max: 48.0) [2023-10-14 19:00:43,344][60425] Avg episode reward: [(0, '61.580'), (1, '57.300')] [2023-10-14 19:00:46,069][61585] Updated weights for policy 1, policy_version 33730 (0.0009) [2023-10-14 19:00:46,471][61585] Updated weights for policy 1, policy_version 33740 (0.0009) [2023-10-14 19:00:46,827][61552] Updated weights for policy 0, policy_version 33892 (0.0009) [2023-10-14 19:00:46,830][61585] Updated weights for policy 1, policy_version 33750 (0.0010) [2023-10-14 19:00:47,195][61585] Updated weights for policy 1, policy_version 33760 (0.0008) [2023-10-14 19:00:47,203][61552] Updated weights for policy 0, policy_version 33902 (0.0009) [2023-10-14 19:00:47,568][61552] Updated weights for policy 0, policy_version 33912 (0.0009) [2023-10-14 19:00:48,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 69304320. Throughput: 0: 1672.0, 1: 1670.2. Samples: 17325346. Policy #0 lag: (min: 23.0, avg: 46.6, max: 48.0) [2023-10-14 19:00:48,344][60425] Avg episode reward: [(0, '62.790'), (1, '58.750')] [2023-10-14 19:00:51,393][61585] Updated weights for policy 1, policy_version 33770 (0.0010) [2023-10-14 19:00:51,728][61552] Updated weights for policy 0, policy_version 33922 (0.0007) [2023-10-14 19:00:51,756][61585] Updated weights for policy 1, policy_version 33780 (0.0008) [2023-10-14 19:00:52,103][61552] Updated weights for policy 0, policy_version 33932 (0.0009) [2023-10-14 19:00:52,131][61585] Updated weights for policy 1, policy_version 33790 (0.0007) [2023-10-14 19:00:52,457][61552] Updated weights for policy 0, policy_version 33942 (0.0009) [2023-10-14 19:00:52,819][61552] Updated weights for policy 0, policy_version 33952 (0.0007) [2023-10-14 19:00:53,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 69369856. Throughput: 0: 1674.8, 1: 1648.4. Samples: 17344940. Policy #0 lag: (min: 23.0, avg: 46.6, max: 48.0) [2023-10-14 19:00:53,344][60425] Avg episode reward: [(0, '61.610'), (1, '57.560')] [2023-10-14 19:00:56,336][61585] Updated weights for policy 1, policy_version 33800 (0.0008) [2023-10-14 19:00:56,702][61585] Updated weights for policy 1, policy_version 33810 (0.0009) [2023-10-14 19:00:56,917][61552] Updated weights for policy 0, policy_version 33962 (0.0007) [2023-10-14 19:00:57,055][61585] Updated weights for policy 1, policy_version 33820 (0.0008) [2023-10-14 19:00:57,274][61552] Updated weights for policy 0, policy_version 33972 (0.0007) [2023-10-14 19:00:57,648][61552] Updated weights for policy 0, policy_version 33982 (0.0010) [2023-10-14 19:00:58,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 69435392. Throughput: 0: 1655.5, 1: 1659.8. Samples: 17363922. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 19:00:58,344][60425] Avg episode reward: [(0, '59.640'), (1, '57.520')] [2023-10-14 19:01:01,193][61585] Updated weights for policy 1, policy_version 33830 (0.0008) [2023-10-14 19:01:01,565][61585] Updated weights for policy 1, policy_version 33840 (0.0007) [2023-10-14 19:01:01,926][61585] Updated weights for policy 1, policy_version 33850 (0.0009) [2023-10-14 19:01:01,927][61552] Updated weights for policy 0, policy_version 33992 (0.0008) [2023-10-14 19:01:02,303][61552] Updated weights for policy 0, policy_version 34002 (0.0007) [2023-10-14 19:01:02,668][61552] Updated weights for policy 0, policy_version 34012 (0.0007) [2023-10-14 19:01:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 69500928. Throughput: 0: 1677.5, 1: 1661.1. Samples: 17375330. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 19:01:03,344][60425] Avg episode reward: [(0, '61.760'), (1, '53.490')] [2023-10-14 19:01:06,051][61585] Updated weights for policy 1, policy_version 33860 (0.0009) [2023-10-14 19:01:06,417][61585] Updated weights for policy 1, policy_version 33870 (0.0009) [2023-10-14 19:01:06,762][61552] Updated weights for policy 0, policy_version 34022 (0.0009) [2023-10-14 19:01:06,773][61585] Updated weights for policy 1, policy_version 33880 (0.0008) [2023-10-14 19:01:07,135][61552] Updated weights for policy 0, policy_version 34032 (0.0007) [2023-10-14 19:01:07,497][61552] Updated weights for policy 0, policy_version 34042 (0.0007) [2023-10-14 19:01:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 69566464. Throughput: 0: 1670.1, 1: 1652.2. Samples: 17394834. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 19:01:08,344][60425] Avg episode reward: [(0, '62.800'), (1, '55.540')] [2023-10-14 19:01:11,012][61585] Updated weights for policy 1, policy_version 33890 (0.0010) [2023-10-14 19:01:11,379][61585] Updated weights for policy 1, policy_version 33900 (0.0007) [2023-10-14 19:01:11,415][61552] Updated weights for policy 0, policy_version 34052 (0.0009) [2023-10-14 19:01:11,730][61585] Updated weights for policy 1, policy_version 33910 (0.0008) [2023-10-14 19:01:11,788][61552] Updated weights for policy 0, policy_version 34062 (0.0009) [2023-10-14 19:01:12,091][61585] Updated weights for policy 1, policy_version 33920 (0.0009) [2023-10-14 19:01:12,158][61552] Updated weights for policy 0, policy_version 34072 (0.0008) [2023-10-14 19:01:13,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 69632000. Throughput: 0: 1659.8, 1: 1657.1. Samples: 17414018. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 19:01:13,344][60425] Avg episode reward: [(0, '63.930'), (1, '53.380')] [2023-10-14 19:01:16,118][61552] Updated weights for policy 0, policy_version 34082 (0.0009) [2023-10-14 19:01:16,185][61585] Updated weights for policy 1, policy_version 33930 (0.0010) [2023-10-14 19:01:16,481][61552] Updated weights for policy 0, policy_version 34092 (0.0007) [2023-10-14 19:01:16,548][61585] Updated weights for policy 1, policy_version 33940 (0.0010) [2023-10-14 19:01:16,859][61552] Updated weights for policy 0, policy_version 34102 (0.0009) [2023-10-14 19:01:16,908][61585] Updated weights for policy 1, policy_version 33950 (0.0008) [2023-10-14 19:01:17,225][61552] Updated weights for policy 0, policy_version 34112 (0.0008) [2023-10-14 19:01:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 69697536. Throughput: 0: 1686.8, 1: 1665.4. Samples: 17425892. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 19:01:18,344][60425] Avg episode reward: [(0, '57.510'), (1, '58.660')] [2023-10-14 19:01:20,897][61585] Updated weights for policy 1, policy_version 33960 (0.0008) [2023-10-14 19:01:21,258][61585] Updated weights for policy 1, policy_version 33970 (0.0008) [2023-10-14 19:01:21,354][61552] Updated weights for policy 0, policy_version 34122 (0.0007) [2023-10-14 19:01:21,624][61585] Updated weights for policy 1, policy_version 33980 (0.0007) [2023-10-14 19:01:21,711][61552] Updated weights for policy 0, policy_version 34132 (0.0009) [2023-10-14 19:01:22,078][61552] Updated weights for policy 0, policy_version 34142 (0.0008) [2023-10-14 19:01:23,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 69763072. Throughput: 0: 1666.6, 1: 1656.3. Samples: 17444630. Policy #0 lag: (min: 1.0, avg: 17.6, max: 33.0) [2023-10-14 19:01:23,344][60425] Avg episode reward: [(0, '58.870'), (1, '57.780')] [2023-10-14 19:01:25,749][61585] Updated weights for policy 1, policy_version 33990 (0.0009) [2023-10-14 19:01:26,109][61585] Updated weights for policy 1, policy_version 34000 (0.0007) [2023-10-14 19:01:26,173][61552] Updated weights for policy 0, policy_version 34152 (0.0008) [2023-10-14 19:01:26,473][61585] Updated weights for policy 1, policy_version 34010 (0.0008) [2023-10-14 19:01:26,542][61552] Updated weights for policy 0, policy_version 34162 (0.0009) [2023-10-14 19:01:26,917][61552] Updated weights for policy 0, policy_version 34172 (0.0010) [2023-10-14 19:01:28,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 69828608. Throughput: 0: 1676.8, 1: 1674.6. Samples: 17464828. Policy #0 lag: (min: 1.0, avg: 17.6, max: 33.0) [2023-10-14 19:01:28,344][60425] Avg episode reward: [(0, '60.720'), (1, '58.050')] [2023-10-14 19:01:30,577][61585] Updated weights for policy 1, policy_version 34020 (0.0010) [2023-10-14 19:01:30,733][61552] Updated weights for policy 0, policy_version 34182 (0.0007) [2023-10-14 19:01:30,939][61585] Updated weights for policy 1, policy_version 34030 (0.0008) [2023-10-14 19:01:31,101][61552] Updated weights for policy 0, policy_version 34192 (0.0010) [2023-10-14 19:01:31,308][61585] Updated weights for policy 1, policy_version 34040 (0.0010) [2023-10-14 19:01:31,473][61552] Updated weights for policy 0, policy_version 34202 (0.0008) [2023-10-14 19:01:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 69894144. Throughput: 0: 1686.8, 1: 1663.2. Samples: 17476098. Policy #0 lag: (min: 1.0, avg: 17.6, max: 33.0) [2023-10-14 19:01:33,345][60425] Avg episode reward: [(0, '63.670'), (1, '60.350')] [2023-10-14 19:01:35,442][61585] Updated weights for policy 1, policy_version 34050 (0.0008) [2023-10-14 19:01:35,550][61552] Updated weights for policy 0, policy_version 34212 (0.0008) [2023-10-14 19:01:35,838][61585] Updated weights for policy 1, policy_version 34060 (0.0008) [2023-10-14 19:01:35,907][61552] Updated weights for policy 0, policy_version 34222 (0.0008) [2023-10-14 19:01:36,214][61585] Updated weights for policy 1, policy_version 34070 (0.0010) [2023-10-14 19:01:36,287][61552] Updated weights for policy 0, policy_version 34232 (0.0010) [2023-10-14 19:01:36,570][61585] Updated weights for policy 1, policy_version 34080 (0.0008) [2023-10-14 19:01:38,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 69959680. Throughput: 0: 1664.4, 1: 1665.1. Samples: 17494768. Policy #0 lag: (min: 1.0, avg: 17.6, max: 33.0) [2023-10-14 19:01:38,345][60425] Avg episode reward: [(0, '60.440'), (1, '59.150')] [2023-10-14 19:01:40,456][61552] Updated weights for policy 0, policy_version 34242 (0.0008) [2023-10-14 19:01:40,633][61585] Updated weights for policy 1, policy_version 34090 (0.0008) [2023-10-14 19:01:40,821][61552] Updated weights for policy 0, policy_version 34252 (0.0007) [2023-10-14 19:01:40,998][61585] Updated weights for policy 1, policy_version 34100 (0.0007) [2023-10-14 19:01:41,190][61552] Updated weights for policy 0, policy_version 34262 (0.0007) [2023-10-14 19:01:41,358][61585] Updated weights for policy 1, policy_version 34110 (0.0009) [2023-10-14 19:01:41,553][61552] Updated weights for policy 0, policy_version 34272 (0.0009) [2023-10-14 19:01:43,344][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 70025216. Throughput: 0: 1685.5, 1: 1680.5. Samples: 17515394. Policy #0 lag: (min: 1.0, avg: 17.6, max: 33.0) [2023-10-14 19:01:43,345][60425] Avg episode reward: [(0, '63.360'), (1, '59.220')] [2023-10-14 19:01:43,358][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000034272_35094528.pth... [2023-10-14 19:01:43,358][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000034112_34930688.pth... [2023-10-14 19:01:43,406][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000032704_33488896.pth [2023-10-14 19:01:43,406][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000032544_33325056.pth [2023-10-14 19:01:45,341][61585] Updated weights for policy 1, policy_version 34120 (0.0010) [2023-10-14 19:01:45,704][61585] Updated weights for policy 1, policy_version 34130 (0.0007) [2023-10-14 19:01:45,762][61552] Updated weights for policy 0, policy_version 34282 (0.0008) [2023-10-14 19:01:46,067][61585] Updated weights for policy 1, policy_version 34140 (0.0008) [2023-10-14 19:01:46,121][61552] Updated weights for policy 0, policy_version 34292 (0.0009) [2023-10-14 19:01:46,494][61552] Updated weights for policy 0, policy_version 34302 (0.0010) [2023-10-14 19:01:48,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 70090752. Throughput: 0: 1680.2, 1: 1667.5. Samples: 17525976. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-14 19:01:48,344][60425] Avg episode reward: [(0, '62.440'), (1, '60.640')] [2023-10-14 19:01:50,198][61585] Updated weights for policy 1, policy_version 34150 (0.0008) [2023-10-14 19:01:50,549][61585] Updated weights for policy 1, policy_version 34160 (0.0007) [2023-10-14 19:01:50,564][61552] Updated weights for policy 0, policy_version 34312 (0.0009) [2023-10-14 19:01:50,920][61585] Updated weights for policy 1, policy_version 34170 (0.0010) [2023-10-14 19:01:50,940][61552] Updated weights for policy 0, policy_version 34322 (0.0010) [2023-10-14 19:01:51,323][61552] Updated weights for policy 0, policy_version 34332 (0.0008) [2023-10-14 19:01:53,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 70156288. Throughput: 0: 1667.4, 1: 1666.5. Samples: 17544860. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-14 19:01:53,344][60425] Avg episode reward: [(0, '60.380'), (1, '58.920')] [2023-10-14 19:01:55,012][61585] Updated weights for policy 1, policy_version 34180 (0.0008) [2023-10-14 19:01:55,376][61585] Updated weights for policy 1, policy_version 34190 (0.0008) [2023-10-14 19:01:55,443][61552] Updated weights for policy 0, policy_version 34342 (0.0007) [2023-10-14 19:01:55,748][61585] Updated weights for policy 1, policy_version 34200 (0.0008) [2023-10-14 19:01:55,828][61552] Updated weights for policy 0, policy_version 34352 (0.0008) [2023-10-14 19:01:56,190][61552] Updated weights for policy 0, policy_version 34362 (0.0008) [2023-10-14 19:01:58,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 70221824. Throughput: 0: 1685.0, 1: 1678.9. Samples: 17565392. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-14 19:01:58,344][60425] Avg episode reward: [(0, '64.310'), (1, '56.960')] [2023-10-14 19:01:59,839][61585] Updated weights for policy 1, policy_version 34210 (0.0009) [2023-10-14 19:02:00,210][61585] Updated weights for policy 1, policy_version 34220 (0.0008) [2023-10-14 19:02:00,232][61552] Updated weights for policy 0, policy_version 34372 (0.0008) [2023-10-14 19:02:00,579][61585] Updated weights for policy 1, policy_version 34230 (0.0008) [2023-10-14 19:02:00,608][61552] Updated weights for policy 0, policy_version 34382 (0.0008) [2023-10-14 19:02:00,946][61585] Updated weights for policy 1, policy_version 34240 (0.0009) [2023-10-14 19:02:00,975][61552] Updated weights for policy 0, policy_version 34392 (0.0007) [2023-10-14 19:02:03,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 70287360. Throughput: 0: 1665.9, 1: 1653.1. Samples: 17575246. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-14 19:02:03,344][60425] Avg episode reward: [(0, '63.570'), (1, '57.480')] [2023-10-14 19:02:05,013][61585] Updated weights for policy 1, policy_version 34250 (0.0009) [2023-10-14 19:02:05,252][61552] Updated weights for policy 0, policy_version 34402 (0.0008) [2023-10-14 19:02:05,374][61585] Updated weights for policy 1, policy_version 34260 (0.0008) [2023-10-14 19:02:05,613][61552] Updated weights for policy 0, policy_version 34412 (0.0007) [2023-10-14 19:02:05,746][61585] Updated weights for policy 1, policy_version 34270 (0.0007) [2023-10-14 19:02:05,980][61552] Updated weights for policy 0, policy_version 34422 (0.0009) [2023-10-14 19:02:06,346][61552] Updated weights for policy 0, policy_version 34432 (0.0009) [2023-10-14 19:02:08,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 70352896. Throughput: 0: 1666.4, 1: 1667.4. Samples: 17594650. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-14 19:02:08,345][60425] Avg episode reward: [(0, '65.960'), (1, '58.190')] [2023-10-14 19:02:09,918][61585] Updated weights for policy 1, policy_version 34280 (0.0008) [2023-10-14 19:02:10,275][61585] Updated weights for policy 1, policy_version 34290 (0.0009) [2023-10-14 19:02:10,502][61552] Updated weights for policy 0, policy_version 34442 (0.0008) [2023-10-14 19:02:10,644][61585] Updated weights for policy 1, policy_version 34300 (0.0008) [2023-10-14 19:02:10,873][61552] Updated weights for policy 0, policy_version 34452 (0.0009) [2023-10-14 19:02:11,250][61552] Updated weights for policy 0, policy_version 34462 (0.0009) [2023-10-14 19:02:13,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 70418432. Throughput: 0: 1671.4, 1: 1662.6. Samples: 17614858. Policy #0 lag: (min: 20.0, avg: 20.3, max: 31.0) [2023-10-14 19:02:13,344][60425] Avg episode reward: [(0, '66.460'), (1, '57.780')] [2023-10-14 19:02:14,693][61585] Updated weights for policy 1, policy_version 34310 (0.0008) [2023-10-14 19:02:15,059][61585] Updated weights for policy 1, policy_version 34320 (0.0008) [2023-10-14 19:02:15,423][61585] Updated weights for policy 1, policy_version 34330 (0.0009) [2023-10-14 19:02:15,499][61552] Updated weights for policy 0, policy_version 34472 (0.0010) [2023-10-14 19:02:15,872][61552] Updated weights for policy 0, policy_version 34482 (0.0008) [2023-10-14 19:02:16,241][61552] Updated weights for policy 0, policy_version 34492 (0.0007) [2023-10-14 19:02:18,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 70483968. Throughput: 0: 1655.0, 1: 1646.6. Samples: 17624670. Policy #0 lag: (min: 20.0, avg: 20.3, max: 31.0) [2023-10-14 19:02:18,344][60425] Avg episode reward: [(0, '66.230'), (1, '55.030')] [2023-10-14 19:02:19,643][61585] Updated weights for policy 1, policy_version 34340 (0.0009) [2023-10-14 19:02:20,001][61585] Updated weights for policy 1, policy_version 34350 (0.0009) [2023-10-14 19:02:20,176][61552] Updated weights for policy 0, policy_version 34502 (0.0007) [2023-10-14 19:02:20,360][61585] Updated weights for policy 1, policy_version 34360 (0.0009) [2023-10-14 19:02:20,551][61552] Updated weights for policy 0, policy_version 34512 (0.0008) [2023-10-14 19:02:20,922][61552] Updated weights for policy 0, policy_version 34522 (0.0009) [2023-10-14 19:02:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 70549504. Throughput: 0: 1661.3, 1: 1667.5. Samples: 17644566. Policy #0 lag: (min: 20.0, avg: 20.3, max: 31.0) [2023-10-14 19:02:23,344][60425] Avg episode reward: [(0, '67.500'), (1, '61.150')] [2023-10-14 19:02:24,592][61585] Updated weights for policy 1, policy_version 34370 (0.0007) [2023-10-14 19:02:24,936][61552] Updated weights for policy 0, policy_version 34532 (0.0010) [2023-10-14 19:02:25,009][61585] Updated weights for policy 1, policy_version 34380 (0.0009) [2023-10-14 19:02:25,302][61552] Updated weights for policy 0, policy_version 34542 (0.0010) [2023-10-14 19:02:25,381][61585] Updated weights for policy 1, policy_version 34390 (0.0007) [2023-10-14 19:02:25,665][61552] Updated weights for policy 0, policy_version 34552 (0.0008) [2023-10-14 19:02:25,736][61585] Updated weights for policy 1, policy_version 34400 (0.0007) [2023-10-14 19:02:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 70615040. Throughput: 0: 1658.8, 1: 1662.9. Samples: 17664868. Policy #0 lag: (min: 20.0, avg: 20.3, max: 31.0) [2023-10-14 19:02:28,344][60425] Avg episode reward: [(0, '67.100'), (1, '63.500')] [2023-10-14 19:02:29,619][61585] Updated weights for policy 1, policy_version 34410 (0.0008) [2023-10-14 19:02:29,894][61552] Updated weights for policy 0, policy_version 34562 (0.0007) [2023-10-14 19:02:29,980][61585] Updated weights for policy 1, policy_version 34420 (0.0008) [2023-10-14 19:02:30,259][61552] Updated weights for policy 0, policy_version 34572 (0.0008) [2023-10-14 19:02:30,346][61585] Updated weights for policy 1, policy_version 34430 (0.0008) [2023-10-14 19:02:30,623][61552] Updated weights for policy 0, policy_version 34582 (0.0009) [2023-10-14 19:02:30,989][61552] Updated weights for policy 0, policy_version 34592 (0.0010) [2023-10-14 19:02:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 70680576. Throughput: 0: 1651.3, 1: 1651.4. Samples: 17674596. Policy #0 lag: (min: 20.0, avg: 20.3, max: 31.0) [2023-10-14 19:02:33,344][60425] Avg episode reward: [(0, '66.130'), (1, '58.840')] [2023-10-14 19:02:34,517][61585] Updated weights for policy 1, policy_version 34440 (0.0009) [2023-10-14 19:02:34,883][61585] Updated weights for policy 1, policy_version 34450 (0.0009) [2023-10-14 19:02:35,137][61552] Updated weights for policy 0, policy_version 34602 (0.0008) [2023-10-14 19:02:35,249][61585] Updated weights for policy 1, policy_version 34460 (0.0009) [2023-10-14 19:02:35,512][61552] Updated weights for policy 0, policy_version 34612 (0.0007) [2023-10-14 19:02:35,886][61552] Updated weights for policy 0, policy_version 34622 (0.0008) [2023-10-14 19:02:38,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 70746112. Throughput: 0: 1657.6, 1: 1667.0. Samples: 17694466. Policy #0 lag: (min: 18.0, avg: 26.0, max: 50.0) [2023-10-14 19:02:38,344][60425] Avg episode reward: [(0, '66.220'), (1, '60.100')] [2023-10-14 19:02:39,439][61585] Updated weights for policy 1, policy_version 34470 (0.0008) [2023-10-14 19:02:39,804][61585] Updated weights for policy 1, policy_version 34480 (0.0008) [2023-10-14 19:02:39,905][61552] Updated weights for policy 0, policy_version 34632 (0.0009) [2023-10-14 19:02:40,165][61585] Updated weights for policy 1, policy_version 34490 (0.0008) [2023-10-14 19:02:40,282][61552] Updated weights for policy 0, policy_version 34642 (0.0008) [2023-10-14 19:02:40,642][61552] Updated weights for policy 0, policy_version 34652 (0.0010) [2023-10-14 19:02:43,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 70811648. Throughput: 0: 1666.1, 1: 1661.8. Samples: 17715150. Policy #0 lag: (min: 18.0, avg: 26.0, max: 50.0) [2023-10-14 19:02:43,344][60425] Avg episode reward: [(0, '67.310'), (1, '61.150')] [2023-10-14 19:02:44,364][61585] Updated weights for policy 1, policy_version 34500 (0.0007) [2023-10-14 19:02:44,733][61585] Updated weights for policy 1, policy_version 34510 (0.0008) [2023-10-14 19:02:44,765][61552] Updated weights for policy 0, policy_version 34662 (0.0010) [2023-10-14 19:02:45,090][61585] Updated weights for policy 1, policy_version 34520 (0.0008) [2023-10-14 19:02:45,146][61552] Updated weights for policy 0, policy_version 34672 (0.0009) [2023-10-14 19:02:45,513][61552] Updated weights for policy 0, policy_version 34682 (0.0007) [2023-10-14 19:02:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 70877184. Throughput: 0: 1654.6, 1: 1653.8. Samples: 17724124. Policy #0 lag: (min: 18.0, avg: 26.0, max: 50.0) [2023-10-14 19:02:48,344][60425] Avg episode reward: [(0, '68.740'), (1, '63.590')] [2023-10-14 19:02:48,344][61172] Saving new best policy, reward=68.740! [2023-10-14 19:02:49,240][61585] Updated weights for policy 1, policy_version 34530 (0.0007) [2023-10-14 19:02:49,613][61585] Updated weights for policy 1, policy_version 34540 (0.0008) [2023-10-14 19:02:49,655][61552] Updated weights for policy 0, policy_version 34692 (0.0008) [2023-10-14 19:02:49,983][61585] Updated weights for policy 1, policy_version 34550 (0.0007) [2023-10-14 19:02:50,021][61552] Updated weights for policy 0, policy_version 34702 (0.0008) [2023-10-14 19:02:50,350][61585] Updated weights for policy 1, policy_version 34560 (0.0007) [2023-10-14 19:02:50,388][61552] Updated weights for policy 0, policy_version 34712 (0.0008) [2023-10-14 19:02:53,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 70942720. Throughput: 0: 1663.3, 1: 1663.7. Samples: 17744362. Policy #0 lag: (min: 18.0, avg: 26.0, max: 50.0) [2023-10-14 19:02:53,344][60425] Avg episode reward: [(0, '63.380'), (1, '54.400')] [2023-10-14 19:02:54,493][61552] Updated weights for policy 0, policy_version 34722 (0.0007) [2023-10-14 19:02:54,543][61585] Updated weights for policy 1, policy_version 34570 (0.0007) [2023-10-14 19:02:54,858][61552] Updated weights for policy 0, policy_version 34732 (0.0009) [2023-10-14 19:02:54,909][61585] Updated weights for policy 1, policy_version 34580 (0.0008) [2023-10-14 19:02:55,226][61552] Updated weights for policy 0, policy_version 34742 (0.0007) [2023-10-14 19:02:55,267][61585] Updated weights for policy 1, policy_version 34590 (0.0008) [2023-10-14 19:02:55,598][61552] Updated weights for policy 0, policy_version 34752 (0.0010) [2023-10-14 19:02:58,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 71008256. Throughput: 0: 1667.1, 1: 1665.4. Samples: 17764820. Policy #0 lag: (min: 18.0, avg: 26.0, max: 50.0) [2023-10-14 19:02:58,344][60425] Avg episode reward: [(0, '62.840'), (1, '61.810')] [2023-10-14 19:02:59,462][61585] Updated weights for policy 1, policy_version 34600 (0.0010) [2023-10-14 19:02:59,595][61552] Updated weights for policy 0, policy_version 34762 (0.0010) [2023-10-14 19:02:59,817][61585] Updated weights for policy 1, policy_version 34610 (0.0010) [2023-10-14 19:02:59,963][61552] Updated weights for policy 0, policy_version 34772 (0.0010) [2023-10-14 19:03:00,186][61585] Updated weights for policy 1, policy_version 34620 (0.0007) [2023-10-14 19:03:00,332][61552] Updated weights for policy 0, policy_version 34782 (0.0007) [2023-10-14 19:03:03,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 71073792. Throughput: 0: 1652.4, 1: 1663.5. Samples: 17773886. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-14 19:03:03,345][60425] Avg episode reward: [(0, '63.220'), (1, '59.190')] [2023-10-14 19:03:04,350][61585] Updated weights for policy 1, policy_version 34630 (0.0008) [2023-10-14 19:03:04,517][61552] Updated weights for policy 0, policy_version 34792 (0.0009) [2023-10-14 19:03:04,714][61585] Updated weights for policy 1, policy_version 34640 (0.0009) [2023-10-14 19:03:04,887][61552] Updated weights for policy 0, policy_version 34802 (0.0009) [2023-10-14 19:03:05,090][61585] Updated weights for policy 1, policy_version 34650 (0.0007) [2023-10-14 19:03:05,263][61552] Updated weights for policy 0, policy_version 34812 (0.0007) [2023-10-14 19:03:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 71139328. Throughput: 0: 1663.3, 1: 1660.8. Samples: 17794152. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-14 19:03:08,344][60425] Avg episode reward: [(0, '58.140'), (1, '58.650')] [2023-10-14 19:03:09,113][61585] Updated weights for policy 1, policy_version 34660 (0.0007) [2023-10-14 19:03:09,475][61585] Updated weights for policy 1, policy_version 34670 (0.0008) [2023-10-14 19:03:09,524][61552] Updated weights for policy 0, policy_version 34822 (0.0007) [2023-10-14 19:03:09,844][61585] Updated weights for policy 1, policy_version 34680 (0.0007) [2023-10-14 19:03:09,898][61552] Updated weights for policy 0, policy_version 34832 (0.0007) [2023-10-14 19:03:10,276][61552] Updated weights for policy 0, policy_version 34842 (0.0010) [2023-10-14 19:03:13,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 71204864. Throughput: 0: 1661.8, 1: 1661.2. Samples: 17814402. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-14 19:03:13,344][60425] Avg episode reward: [(0, '59.020'), (1, '56.250')] [2023-10-14 19:03:14,079][61585] Updated weights for policy 1, policy_version 34690 (0.0008) [2023-10-14 19:03:14,478][61585] Updated weights for policy 1, policy_version 34700 (0.0009) [2023-10-14 19:03:14,574][61552] Updated weights for policy 0, policy_version 34852 (0.0010) [2023-10-14 19:03:14,846][61585] Updated weights for policy 1, policy_version 34710 (0.0007) [2023-10-14 19:03:14,941][61552] Updated weights for policy 0, policy_version 34862 (0.0009) [2023-10-14 19:03:15,212][61585] Updated weights for policy 1, policy_version 34720 (0.0011) [2023-10-14 19:03:15,314][61552] Updated weights for policy 0, policy_version 34872 (0.0007) [2023-10-14 19:03:18,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 71270400. Throughput: 0: 1646.0, 1: 1659.1. Samples: 17823322. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-14 19:03:18,344][60425] Avg episode reward: [(0, '63.340'), (1, '57.120')] [2023-10-14 19:03:19,342][61585] Updated weights for policy 1, policy_version 34730 (0.0007) [2023-10-14 19:03:19,381][61552] Updated weights for policy 0, policy_version 34882 (0.0008) [2023-10-14 19:03:19,708][61585] Updated weights for policy 1, policy_version 34740 (0.0008) [2023-10-14 19:03:19,753][61552] Updated weights for policy 0, policy_version 34892 (0.0009) [2023-10-14 19:03:20,074][61585] Updated weights for policy 1, policy_version 34750 (0.0008) [2023-10-14 19:03:20,124][61552] Updated weights for policy 0, policy_version 34902 (0.0009) [2023-10-14 19:03:20,488][61552] Updated weights for policy 0, policy_version 34912 (0.0010) [2023-10-14 19:03:23,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 71335936. Throughput: 0: 1661.3, 1: 1660.5. Samples: 17843948. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-14 19:03:23,344][60425] Avg episode reward: [(0, '62.910'), (1, '55.150')] [2023-10-14 19:03:24,156][61585] Updated weights for policy 1, policy_version 34760 (0.0009) [2023-10-14 19:03:24,522][61585] Updated weights for policy 1, policy_version 34770 (0.0008) [2023-10-14 19:03:24,664][61552] Updated weights for policy 0, policy_version 34922 (0.0008) [2023-10-14 19:03:24,887][61585] Updated weights for policy 1, policy_version 34780 (0.0009) [2023-10-14 19:03:25,026][61552] Updated weights for policy 0, policy_version 34932 (0.0007) [2023-10-14 19:03:25,397][61552] Updated weights for policy 0, policy_version 34942 (0.0010) [2023-10-14 19:03:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 71401472. Throughput: 0: 1651.1, 1: 1665.8. Samples: 17864410. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:03:28,344][60425] Avg episode reward: [(0, '61.420'), (1, '57.040')] [2023-10-14 19:03:28,890][61585] Updated weights for policy 1, policy_version 34790 (0.0010) [2023-10-14 19:03:29,260][61585] Updated weights for policy 1, policy_version 34800 (0.0008) [2023-10-14 19:03:29,576][61552] Updated weights for policy 0, policy_version 34952 (0.0008) [2023-10-14 19:03:29,623][61585] Updated weights for policy 1, policy_version 34810 (0.0008) [2023-10-14 19:03:29,936][61552] Updated weights for policy 0, policy_version 34962 (0.0007) [2023-10-14 19:03:30,306][61552] Updated weights for policy 0, policy_version 34972 (0.0008) [2023-10-14 19:03:33,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 71467008. Throughput: 0: 1648.3, 1: 1669.9. Samples: 17873442. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:03:33,344][60425] Avg episode reward: [(0, '62.390'), (1, '56.460')] [2023-10-14 19:03:33,703][61585] Updated weights for policy 1, policy_version 34820 (0.0010) [2023-10-14 19:03:34,062][61585] Updated weights for policy 1, policy_version 34830 (0.0009) [2023-10-14 19:03:34,351][61552] Updated weights for policy 0, policy_version 34982 (0.0008) [2023-10-14 19:03:34,432][61585] Updated weights for policy 1, policy_version 34840 (0.0008) [2023-10-14 19:03:34,707][61552] Updated weights for policy 0, policy_version 34992 (0.0008) [2023-10-14 19:03:35,076][61552] Updated weights for policy 0, policy_version 35002 (0.0008) [2023-10-14 19:03:38,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 71532544. Throughput: 0: 1649.6, 1: 1666.7. Samples: 17893598. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:03:38,344][60425] Avg episode reward: [(0, '63.910'), (1, '58.320')] [2023-10-14 19:03:38,476][61585] Updated weights for policy 1, policy_version 34850 (0.0009) [2023-10-14 19:03:38,849][61585] Updated weights for policy 1, policy_version 34860 (0.0009) [2023-10-14 19:03:39,214][61585] Updated weights for policy 1, policy_version 34870 (0.0010) [2023-10-14 19:03:39,393][61552] Updated weights for policy 0, policy_version 35012 (0.0009) [2023-10-14 19:03:39,587][61585] Updated weights for policy 1, policy_version 34880 (0.0009) [2023-10-14 19:03:39,763][61552] Updated weights for policy 0, policy_version 35022 (0.0008) [2023-10-14 19:03:40,125][61552] Updated weights for policy 0, policy_version 35032 (0.0009) [2023-10-14 19:03:43,344][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 71598080. Throughput: 0: 1647.5, 1: 1667.5. Samples: 17913998. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:03:43,345][60425] Avg episode reward: [(0, '62.860'), (1, '58.510')] [2023-10-14 19:03:43,357][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000034880_35717120.pth... [2023-10-14 19:03:43,357][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000035040_35880960.pth... [2023-10-14 19:03:43,394][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000033504_34308096.pth [2023-10-14 19:03:43,396][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000033344_34144256.pth [2023-10-14 19:03:43,861][61585] Updated weights for policy 1, policy_version 34890 (0.0011) [2023-10-14 19:03:44,115][61552] Updated weights for policy 0, policy_version 35042 (0.0008) [2023-10-14 19:03:44,230][61585] Updated weights for policy 1, policy_version 34900 (0.0008) [2023-10-14 19:03:44,477][61552] Updated weights for policy 0, policy_version 35052 (0.0009) [2023-10-14 19:03:44,591][61585] Updated weights for policy 1, policy_version 34910 (0.0009) [2023-10-14 19:03:44,854][61552] Updated weights for policy 0, policy_version 35062 (0.0009) [2023-10-14 19:03:45,228][61552] Updated weights for policy 0, policy_version 35072 (0.0011) [2023-10-14 19:03:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 71663616. Throughput: 0: 1647.5, 1: 1666.8. Samples: 17923030. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:03:48,344][60425] Avg episode reward: [(0, '62.840'), (1, '56.550')] [2023-10-14 19:03:48,689][61585] Updated weights for policy 1, policy_version 34920 (0.0008) [2023-10-14 19:03:49,046][61585] Updated weights for policy 1, policy_version 34930 (0.0008) [2023-10-14 19:03:49,372][61552] Updated weights for policy 0, policy_version 35082 (0.0009) [2023-10-14 19:03:49,419][61585] Updated weights for policy 1, policy_version 34940 (0.0007) [2023-10-14 19:03:49,743][61552] Updated weights for policy 0, policy_version 35092 (0.0008) [2023-10-14 19:03:50,118][61552] Updated weights for policy 0, policy_version 35102 (0.0010) [2023-10-14 19:03:53,303][61585] Updated weights for policy 1, policy_version 34950 (0.0007) [2023-10-14 19:03:53,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 71729152. Throughput: 0: 1646.6, 1: 1668.4. Samples: 17943324. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:03:53,344][60425] Avg episode reward: [(0, '63.170'), (1, '57.840')] [2023-10-14 19:03:53,676][61585] Updated weights for policy 1, policy_version 34960 (0.0009) [2023-10-14 19:03:54,036][61585] Updated weights for policy 1, policy_version 34970 (0.0009) [2023-10-14 19:03:54,233][61552] Updated weights for policy 0, policy_version 35112 (0.0010) [2023-10-14 19:03:54,606][61552] Updated weights for policy 0, policy_version 35122 (0.0008) [2023-10-14 19:03:54,984][61552] Updated weights for policy 0, policy_version 35132 (0.0010) [2023-10-14 19:03:58,292][61585] Updated weights for policy 1, policy_version 34980 (0.0010) [2023-10-14 19:03:58,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 71794688. Throughput: 0: 1654.5, 1: 1665.4. Samples: 17963800. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:03:58,344][60425] Avg episode reward: [(0, '65.210'), (1, '53.950')] [2023-10-14 19:03:58,658][61585] Updated weights for policy 1, policy_version 34990 (0.0009) [2023-10-14 19:03:59,030][61585] Updated weights for policy 1, policy_version 35000 (0.0009) [2023-10-14 19:03:59,105][61552] Updated weights for policy 0, policy_version 35142 (0.0008) [2023-10-14 19:03:59,468][61552] Updated weights for policy 0, policy_version 35152 (0.0007) [2023-10-14 19:03:59,843][61552] Updated weights for policy 0, policy_version 35162 (0.0008) [2023-10-14 19:04:03,001][61585] Updated weights for policy 1, policy_version 35010 (0.0007) [2023-10-14 19:04:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 71860224. Throughput: 0: 1659.1, 1: 1666.0. Samples: 17972952. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:04:03,344][60425] Avg episode reward: [(0, '61.630'), (1, '56.450')] [2023-10-14 19:04:03,373][61585] Updated weights for policy 1, policy_version 35020 (0.0007) [2023-10-14 19:04:03,729][61585] Updated weights for policy 1, policy_version 35030 (0.0008) [2023-10-14 19:04:03,955][61552] Updated weights for policy 0, policy_version 35172 (0.0007) [2023-10-14 19:04:04,098][61585] Updated weights for policy 1, policy_version 35040 (0.0007) [2023-10-14 19:04:04,325][61552] Updated weights for policy 0, policy_version 35182 (0.0007) [2023-10-14 19:04:04,690][61552] Updated weights for policy 0, policy_version 35192 (0.0008) [2023-10-14 19:04:08,237][61585] Updated weights for policy 1, policy_version 35050 (0.0009) [2023-10-14 19:04:08,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 71925760. Throughput: 0: 1657.9, 1: 1664.5. Samples: 17993456. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:04:08,344][60425] Avg episode reward: [(0, '62.720'), (1, '57.440')] [2023-10-14 19:04:08,604][61585] Updated weights for policy 1, policy_version 35060 (0.0011) [2023-10-14 19:04:08,750][61552] Updated weights for policy 0, policy_version 35202 (0.0007) [2023-10-14 19:04:08,979][61585] Updated weights for policy 1, policy_version 35070 (0.0007) [2023-10-14 19:04:09,113][61552] Updated weights for policy 0, policy_version 35212 (0.0009) [2023-10-14 19:04:09,478][61552] Updated weights for policy 0, policy_version 35222 (0.0009) [2023-10-14 19:04:09,845][61552] Updated weights for policy 0, policy_version 35232 (0.0008) [2023-10-14 19:04:13,030][61585] Updated weights for policy 1, policy_version 35080 (0.0008) [2023-10-14 19:04:13,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 71991296. Throughput: 0: 1665.5, 1: 1665.3. Samples: 18014298. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:04:13,344][60425] Avg episode reward: [(0, '63.550'), (1, '61.970')] [2023-10-14 19:04:13,392][61585] Updated weights for policy 1, policy_version 35090 (0.0007) [2023-10-14 19:04:13,755][61585] Updated weights for policy 1, policy_version 35100 (0.0009) [2023-10-14 19:04:13,792][61552] Updated weights for policy 0, policy_version 35242 (0.0011) [2023-10-14 19:04:14,166][61552] Updated weights for policy 0, policy_version 35252 (0.0008) [2023-10-14 19:04:14,540][61552] Updated weights for policy 0, policy_version 35262 (0.0008) [2023-10-14 19:04:17,948][61585] Updated weights for policy 1, policy_version 35110 (0.0008) [2023-10-14 19:04:18,320][61585] Updated weights for policy 1, policy_version 35120 (0.0009) [2023-10-14 19:04:18,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 72056832. Throughput: 0: 1669.7, 1: 1665.5. Samples: 18023524. Policy #0 lag: (min: 31.0, avg: 39.7, max: 63.0) [2023-10-14 19:04:18,344][60425] Avg episode reward: [(0, '61.560'), (1, '58.080')] [2023-10-14 19:04:18,690][61585] Updated weights for policy 1, policy_version 35130 (0.0007) [2023-10-14 19:04:18,745][61552] Updated weights for policy 0, policy_version 35272 (0.0007) [2023-10-14 19:04:19,124][61552] Updated weights for policy 0, policy_version 35282 (0.0009) [2023-10-14 19:04:19,490][61552] Updated weights for policy 0, policy_version 35292 (0.0008) [2023-10-14 19:04:22,672][61585] Updated weights for policy 1, policy_version 35140 (0.0007) [2023-10-14 19:04:23,050][61585] Updated weights for policy 1, policy_version 35150 (0.0009) [2023-10-14 19:04:23,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 72122368. Throughput: 0: 1677.0, 1: 1669.3. Samples: 18044184. Policy #0 lag: (min: 31.0, avg: 39.7, max: 63.0) [2023-10-14 19:04:23,344][60425] Avg episode reward: [(0, '65.600'), (1, '61.870')] [2023-10-14 19:04:23,353][61552] Updated weights for policy 0, policy_version 35302 (0.0008) [2023-10-14 19:04:23,419][61585] Updated weights for policy 1, policy_version 35160 (0.0008) [2023-10-14 19:04:23,720][61552] Updated weights for policy 0, policy_version 35312 (0.0011) [2023-10-14 19:04:24,084][61552] Updated weights for policy 0, policy_version 35322 (0.0007) [2023-10-14 19:04:27,614][61585] Updated weights for policy 1, policy_version 35170 (0.0008) [2023-10-14 19:04:27,984][61585] Updated weights for policy 1, policy_version 35180 (0.0007) [2023-10-14 19:04:28,131][61552] Updated weights for policy 0, policy_version 35332 (0.0009) [2023-10-14 19:04:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 72187904. Throughput: 0: 1685.3, 1: 1668.9. Samples: 18064936. Policy #0 lag: (min: 31.0, avg: 39.7, max: 63.0) [2023-10-14 19:04:28,344][60425] Avg episode reward: [(0, '61.270'), (1, '58.210')] [2023-10-14 19:04:28,348][61585] Updated weights for policy 1, policy_version 35190 (0.0008) [2023-10-14 19:04:28,496][61552] Updated weights for policy 0, policy_version 35342 (0.0008) [2023-10-14 19:04:28,714][61585] Updated weights for policy 1, policy_version 35200 (0.0009) [2023-10-14 19:04:28,859][61552] Updated weights for policy 0, policy_version 35352 (0.0010) [2023-10-14 19:04:32,712][61585] Updated weights for policy 1, policy_version 35210 (0.0007) [2023-10-14 19:04:33,067][61585] Updated weights for policy 1, policy_version 35220 (0.0009) [2023-10-14 19:04:33,126][61552] Updated weights for policy 0, policy_version 35362 (0.0010) [2023-10-14 19:04:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 72253440. Throughput: 0: 1681.4, 1: 1676.9. Samples: 18074156. Policy #0 lag: (min: 31.0, avg: 39.7, max: 63.0) [2023-10-14 19:04:33,344][60425] Avg episode reward: [(0, '63.660'), (1, '60.530')] [2023-10-14 19:04:33,431][61585] Updated weights for policy 1, policy_version 35230 (0.0008) [2023-10-14 19:04:33,502][61552] Updated weights for policy 0, policy_version 35372 (0.0009) [2023-10-14 19:04:33,874][61552] Updated weights for policy 0, policy_version 35382 (0.0010) [2023-10-14 19:04:34,243][61552] Updated weights for policy 0, policy_version 35392 (0.0008) [2023-10-14 19:04:37,406][61585] Updated weights for policy 1, policy_version 35240 (0.0007) [2023-10-14 19:04:37,782][61585] Updated weights for policy 1, policy_version 35250 (0.0007) [2023-10-14 19:04:38,150][61585] Updated weights for policy 1, policy_version 35260 (0.0007) [2023-10-14 19:04:38,267][61552] Updated weights for policy 0, policy_version 35402 (0.0008) [2023-10-14 19:04:38,343][60425] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 72351744. Throughput: 0: 1687.2, 1: 1680.2. Samples: 18094856. Policy #0 lag: (min: 31.0, avg: 39.7, max: 63.0) [2023-10-14 19:04:38,344][60425] Avg episode reward: [(0, '63.130'), (1, '55.870')] [2023-10-14 19:04:38,630][61552] Updated weights for policy 0, policy_version 35412 (0.0008) [2023-10-14 19:04:39,002][61552] Updated weights for policy 0, policy_version 35422 (0.0009) [2023-10-14 19:04:42,319][61585] Updated weights for policy 1, policy_version 35270 (0.0009) [2023-10-14 19:04:42,686][61585] Updated weights for policy 1, policy_version 35280 (0.0008) [2023-10-14 19:04:42,896][61552] Updated weights for policy 0, policy_version 35432 (0.0008) [2023-10-14 19:04:43,052][61585] Updated weights for policy 1, policy_version 35290 (0.0009) [2023-10-14 19:04:43,273][61552] Updated weights for policy 0, policy_version 35442 (0.0009) [2023-10-14 19:04:43,343][60425] Fps is (10 sec: 16383.9, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 72417280. Throughput: 0: 1684.2, 1: 1673.0. Samples: 18114872. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:04:43,344][60425] Avg episode reward: [(0, '62.560'), (1, '59.970')] [2023-10-14 19:04:43,633][61552] Updated weights for policy 0, policy_version 35452 (0.0011) [2023-10-14 19:04:47,107][61585] Updated weights for policy 1, policy_version 35300 (0.0009) [2023-10-14 19:04:47,468][61585] Updated weights for policy 1, policy_version 35310 (0.0008) [2023-10-14 19:04:47,710][61552] Updated weights for policy 0, policy_version 35462 (0.0009) [2023-10-14 19:04:47,837][61585] Updated weights for policy 1, policy_version 35320 (0.0008) [2023-10-14 19:04:48,082][61552] Updated weights for policy 0, policy_version 35472 (0.0007) [2023-10-14 19:04:48,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 72482816. Throughput: 0: 1684.2, 1: 1686.9. Samples: 18124652. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:04:48,344][60425] Avg episode reward: [(0, '63.400'), (1, '54.660')] [2023-10-14 19:04:48,442][61552] Updated weights for policy 0, policy_version 35482 (0.0009) [2023-10-14 19:04:52,100][61585] Updated weights for policy 1, policy_version 35330 (0.0007) [2023-10-14 19:04:52,461][61585] Updated weights for policy 1, policy_version 35340 (0.0007) [2023-10-14 19:04:52,684][61552] Updated weights for policy 0, policy_version 35492 (0.0008) [2023-10-14 19:04:52,833][61585] Updated weights for policy 1, policy_version 35350 (0.0008) [2023-10-14 19:04:53,052][61552] Updated weights for policy 0, policy_version 35502 (0.0008) [2023-10-14 19:04:53,195][61585] Updated weights for policy 1, policy_version 35360 (0.0009) [2023-10-14 19:04:53,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 72548352. Throughput: 0: 1684.3, 1: 1687.4. Samples: 18145184. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:04:53,344][60425] Avg episode reward: [(0, '64.160'), (1, '61.120')] [2023-10-14 19:04:53,434][61552] Updated weights for policy 0, policy_version 35512 (0.0009) [2023-10-14 19:04:57,382][61585] Updated weights for policy 1, policy_version 35370 (0.0008) [2023-10-14 19:04:57,524][61552] Updated weights for policy 0, policy_version 35522 (0.0008) [2023-10-14 19:04:57,750][61585] Updated weights for policy 1, policy_version 35380 (0.0009) [2023-10-14 19:04:57,897][61552] Updated weights for policy 0, policy_version 35532 (0.0008) [2023-10-14 19:04:58,127][61585] Updated weights for policy 1, policy_version 35390 (0.0008) [2023-10-14 19:04:58,265][61552] Updated weights for policy 0, policy_version 35542 (0.0007) [2023-10-14 19:04:58,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 72613888. Throughput: 0: 1681.1, 1: 1664.2. Samples: 18164836. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:04:58,344][60425] Avg episode reward: [(0, '64.540'), (1, '57.790')] [2023-10-14 19:04:58,629][61552] Updated weights for policy 0, policy_version 35552 (0.0010) [2023-10-14 19:05:02,206][61585] Updated weights for policy 1, policy_version 35400 (0.0007) [2023-10-14 19:05:02,572][61585] Updated weights for policy 1, policy_version 35410 (0.0007) [2023-10-14 19:05:02,804][61552] Updated weights for policy 0, policy_version 35562 (0.0007) [2023-10-14 19:05:02,935][61585] Updated weights for policy 1, policy_version 35420 (0.0007) [2023-10-14 19:05:03,165][61552] Updated weights for policy 0, policy_version 35572 (0.0009) [2023-10-14 19:05:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 72679424. Throughput: 0: 1680.8, 1: 1678.0. Samples: 18174672. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:05:03,344][60425] Avg episode reward: [(0, '59.790'), (1, '59.310')] [2023-10-14 19:05:03,539][61552] Updated weights for policy 0, policy_version 35582 (0.0008) [2023-10-14 19:05:07,057][61585] Updated weights for policy 1, policy_version 35430 (0.0008) [2023-10-14 19:05:07,425][61585] Updated weights for policy 1, policy_version 35440 (0.0008) [2023-10-14 19:05:07,788][61585] Updated weights for policy 1, policy_version 35450 (0.0007) [2023-10-14 19:05:07,788][61552] Updated weights for policy 0, policy_version 35592 (0.0007) [2023-10-14 19:05:08,159][61552] Updated weights for policy 0, policy_version 35602 (0.0009) [2023-10-14 19:05:08,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 72744960. Throughput: 0: 1677.1, 1: 1674.7. Samples: 18195014. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:05:08,344][60425] Avg episode reward: [(0, '62.750'), (1, '61.690')] [2023-10-14 19:05:08,523][61552] Updated weights for policy 0, policy_version 35612 (0.0007) [2023-10-14 19:05:12,011][61585] Updated weights for policy 1, policy_version 35460 (0.0008) [2023-10-14 19:05:12,367][61585] Updated weights for policy 1, policy_version 35470 (0.0008) [2023-10-14 19:05:12,626][61552] Updated weights for policy 0, policy_version 35622 (0.0008) [2023-10-14 19:05:12,733][61585] Updated weights for policy 1, policy_version 35480 (0.0008) [2023-10-14 19:05:12,997][61552] Updated weights for policy 0, policy_version 35632 (0.0007) [2023-10-14 19:05:13,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 72810496. Throughput: 0: 1661.7, 1: 1651.0. Samples: 18214006. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:05:13,344][60425] Avg episode reward: [(0, '62.630'), (1, '57.340')] [2023-10-14 19:05:13,362][61552] Updated weights for policy 0, policy_version 35642 (0.0007) [2023-10-14 19:05:16,853][61585] Updated weights for policy 1, policy_version 35490 (0.0009) [2023-10-14 19:05:17,216][61585] Updated weights for policy 1, policy_version 35500 (0.0007) [2023-10-14 19:05:17,409][61552] Updated weights for policy 0, policy_version 35652 (0.0009) [2023-10-14 19:05:17,585][61585] Updated weights for policy 1, policy_version 35510 (0.0008) [2023-10-14 19:05:17,782][61552] Updated weights for policy 0, policy_version 35662 (0.0008) [2023-10-14 19:05:17,944][61585] Updated weights for policy 1, policy_version 35520 (0.0007) [2023-10-14 19:05:18,148][61552] Updated weights for policy 0, policy_version 35672 (0.0008) [2023-10-14 19:05:18,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 72876032. Throughput: 0: 1670.6, 1: 1665.8. Samples: 18224294. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:05:18,344][60425] Avg episode reward: [(0, '62.200'), (1, '61.330')] [2023-10-14 19:05:22,004][61585] Updated weights for policy 1, policy_version 35530 (0.0010) [2023-10-14 19:05:22,146][61552] Updated weights for policy 0, policy_version 35682 (0.0010) [2023-10-14 19:05:22,368][61585] Updated weights for policy 1, policy_version 35540 (0.0007) [2023-10-14 19:05:22,513][61552] Updated weights for policy 0, policy_version 35692 (0.0007) [2023-10-14 19:05:22,745][61585] Updated weights for policy 1, policy_version 35550 (0.0008) [2023-10-14 19:05:22,872][61552] Updated weights for policy 0, policy_version 35702 (0.0009) [2023-10-14 19:05:23,245][61552] Updated weights for policy 0, policy_version 35712 (0.0008) [2023-10-14 19:05:23,343][60425] Fps is (10 sec: 16383.6, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 72974336. Throughput: 0: 1670.0, 1: 1661.7. Samples: 18244780. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:05:23,344][60425] Avg episode reward: [(0, '64.470'), (1, '54.690')] [2023-10-14 19:05:26,953][61585] Updated weights for policy 1, policy_version 35560 (0.0008) [2023-10-14 19:05:27,163][61552] Updated weights for policy 0, policy_version 35722 (0.0009) [2023-10-14 19:05:27,311][61585] Updated weights for policy 1, policy_version 35570 (0.0008) [2023-10-14 19:05:27,531][61552] Updated weights for policy 0, policy_version 35732 (0.0008) [2023-10-14 19:05:27,674][61585] Updated weights for policy 1, policy_version 35580 (0.0007) [2023-10-14 19:05:27,906][61552] Updated weights for policy 0, policy_version 35742 (0.0007) [2023-10-14 19:05:28,343][60425] Fps is (10 sec: 16383.8, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 73039872. Throughput: 0: 1654.1, 1: 1649.4. Samples: 18263530. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 19:05:28,344][60425] Avg episode reward: [(0, '62.320'), (1, '62.550')] [2023-10-14 19:05:31,569][61585] Updated weights for policy 1, policy_version 35590 (0.0008) [2023-10-14 19:05:31,928][61585] Updated weights for policy 1, policy_version 35600 (0.0008) [2023-10-14 19:05:32,055][61552] Updated weights for policy 0, policy_version 35752 (0.0008) [2023-10-14 19:05:32,295][61585] Updated weights for policy 1, policy_version 35610 (0.0007) [2023-10-14 19:05:32,411][61552] Updated weights for policy 0, policy_version 35762 (0.0008) [2023-10-14 19:05:32,780][61552] Updated weights for policy 0, policy_version 35772 (0.0008) [2023-10-14 19:05:33,343][60425] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 13329.4). Total num frames: 73105408. Throughput: 0: 1670.9, 1: 1667.3. Samples: 18274874. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 19:05:33,344][60425] Avg episode reward: [(0, '60.890'), (1, '58.580')] [2023-10-14 19:05:36,428][61585] Updated weights for policy 1, policy_version 35620 (0.0007) [2023-10-14 19:05:36,816][61552] Updated weights for policy 0, policy_version 35782 (0.0008) [2023-10-14 19:05:36,821][61585] Updated weights for policy 1, policy_version 35630 (0.0008) [2023-10-14 19:05:37,178][61585] Updated weights for policy 1, policy_version 35640 (0.0008) [2023-10-14 19:05:37,182][61552] Updated weights for policy 0, policy_version 35792 (0.0009) [2023-10-14 19:05:37,554][61552] Updated weights for policy 0, policy_version 35802 (0.0010) [2023-10-14 19:05:38,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 73170944. Throughput: 0: 1669.9, 1: 1657.0. Samples: 18294896. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 19:05:38,344][60425] Avg episode reward: [(0, '63.580'), (1, '60.600')] [2023-10-14 19:05:41,270][61585] Updated weights for policy 1, policy_version 35650 (0.0008) [2023-10-14 19:05:41,634][61585] Updated weights for policy 1, policy_version 35660 (0.0009) [2023-10-14 19:05:41,695][61552] Updated weights for policy 0, policy_version 35812 (0.0009) [2023-10-14 19:05:42,004][61585] Updated weights for policy 1, policy_version 35670 (0.0007) [2023-10-14 19:05:42,060][61552] Updated weights for policy 0, policy_version 35822 (0.0008) [2023-10-14 19:05:42,372][61585] Updated weights for policy 1, policy_version 35680 (0.0007) [2023-10-14 19:05:42,437][61552] Updated weights for policy 0, policy_version 35832 (0.0007) [2023-10-14 19:05:43,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 73236480. Throughput: 0: 1648.1, 1: 1656.9. Samples: 18313560. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 19:05:43,344][60425] Avg episode reward: [(0, '59.710'), (1, '56.300')] [2023-10-14 19:05:43,356][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000035680_36536320.pth... [2023-10-14 19:05:43,356][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000035840_36700160.pth... [2023-10-14 19:05:43,390][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000034272_35094528.pth [2023-10-14 19:05:43,395][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000034112_34930688.pth [2023-10-14 19:05:46,433][61585] Updated weights for policy 1, policy_version 35690 (0.0009) [2023-10-14 19:05:46,455][61552] Updated weights for policy 0, policy_version 35842 (0.0009) [2023-10-14 19:05:46,795][61585] Updated weights for policy 1, policy_version 35700 (0.0008) [2023-10-14 19:05:46,823][61552] Updated weights for policy 0, policy_version 35852 (0.0008) [2023-10-14 19:05:47,161][61585] Updated weights for policy 1, policy_version 35710 (0.0007) [2023-10-14 19:05:47,190][61552] Updated weights for policy 0, policy_version 35862 (0.0010) [2023-10-14 19:05:47,553][61552] Updated weights for policy 0, policy_version 35872 (0.0008) [2023-10-14 19:05:48,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 73302016. Throughput: 0: 1676.7, 1: 1669.1. Samples: 18325232. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 19:05:48,344][60425] Avg episode reward: [(0, '64.410'), (1, '58.810')] [2023-10-14 19:05:51,369][61585] Updated weights for policy 1, policy_version 35720 (0.0009) [2023-10-14 19:05:51,681][61552] Updated weights for policy 0, policy_version 35882 (0.0007) [2023-10-14 19:05:51,734][61585] Updated weights for policy 1, policy_version 35730 (0.0008) [2023-10-14 19:05:52,054][61552] Updated weights for policy 0, policy_version 35892 (0.0007) [2023-10-14 19:05:52,095][61585] Updated weights for policy 1, policy_version 35740 (0.0007) [2023-10-14 19:05:52,418][61552] Updated weights for policy 0, policy_version 35902 (0.0009) [2023-10-14 19:05:53,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 73367552. Throughput: 0: 1672.8, 1: 1652.0. Samples: 18344630. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-14 19:05:53,344][60425] Avg episode reward: [(0, '63.510'), (1, '55.490')] [2023-10-14 19:05:56,079][61585] Updated weights for policy 1, policy_version 35750 (0.0008) [2023-10-14 19:05:56,445][61585] Updated weights for policy 1, policy_version 35760 (0.0009) [2023-10-14 19:05:56,740][61552] Updated weights for policy 0, policy_version 35912 (0.0008) [2023-10-14 19:05:56,812][61585] Updated weights for policy 1, policy_version 35770 (0.0007) [2023-10-14 19:05:57,132][61552] Updated weights for policy 0, policy_version 35922 (0.0008) [2023-10-14 19:05:57,494][61552] Updated weights for policy 0, policy_version 35932 (0.0008) [2023-10-14 19:05:58,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 73433088. Throughput: 0: 1653.9, 1: 1671.7. Samples: 18363656. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-14 19:05:58,344][60425] Avg episode reward: [(0, '61.360'), (1, '62.080')] [2023-10-14 19:06:00,891][61585] Updated weights for policy 1, policy_version 35780 (0.0007) [2023-10-14 19:06:01,255][61585] Updated weights for policy 1, policy_version 35790 (0.0010) [2023-10-14 19:06:01,552][61552] Updated weights for policy 0, policy_version 35942 (0.0008) [2023-10-14 19:06:01,614][61585] Updated weights for policy 1, policy_version 35800 (0.0009) [2023-10-14 19:06:01,911][61552] Updated weights for policy 0, policy_version 35952 (0.0010) [2023-10-14 19:06:02,281][61552] Updated weights for policy 0, policy_version 35962 (0.0008) [2023-10-14 19:06:03,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 73498624. Throughput: 0: 1679.2, 1: 1676.5. Samples: 18375298. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-14 19:06:03,344][60425] Avg episode reward: [(0, '64.580'), (1, '59.580')] [2023-10-14 19:06:05,773][61585] Updated weights for policy 1, policy_version 35810 (0.0007) [2023-10-14 19:06:06,138][61585] Updated weights for policy 1, policy_version 35820 (0.0008) [2023-10-14 19:06:06,342][61552] Updated weights for policy 0, policy_version 35972 (0.0010) [2023-10-14 19:06:06,507][61585] Updated weights for policy 1, policy_version 35830 (0.0009) [2023-10-14 19:06:06,716][61552] Updated weights for policy 0, policy_version 35982 (0.0009) [2023-10-14 19:06:06,869][61585] Updated weights for policy 1, policy_version 35840 (0.0009) [2023-10-14 19:06:07,085][61552] Updated weights for policy 0, policy_version 35992 (0.0009) [2023-10-14 19:06:08,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 73564160. Throughput: 0: 1667.3, 1: 1655.9. Samples: 18394322. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-14 19:06:08,344][60425] Avg episode reward: [(0, '67.200'), (1, '57.010')] [2023-10-14 19:06:10,931][61585] Updated weights for policy 1, policy_version 35850 (0.0010) [2023-10-14 19:06:11,059][61552] Updated weights for policy 0, policy_version 36002 (0.0010) [2023-10-14 19:06:11,296][61585] Updated weights for policy 1, policy_version 35860 (0.0009) [2023-10-14 19:06:11,418][61552] Updated weights for policy 0, policy_version 36012 (0.0007) [2023-10-14 19:06:11,664][61585] Updated weights for policy 1, policy_version 35870 (0.0008) [2023-10-14 19:06:11,777][61552] Updated weights for policy 0, policy_version 36022 (0.0008) [2023-10-14 19:06:12,146][61552] Updated weights for policy 0, policy_version 36032 (0.0008) [2023-10-14 19:06:13,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 73629696. Throughput: 0: 1666.4, 1: 1674.4. Samples: 18413866. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-14 19:06:13,344][60425] Avg episode reward: [(0, '67.310'), (1, '58.530')] [2023-10-14 19:06:15,858][61585] Updated weights for policy 1, policy_version 35880 (0.0007) [2023-10-14 19:06:16,221][61585] Updated weights for policy 1, policy_version 35890 (0.0008) [2023-10-14 19:06:16,300][61552] Updated weights for policy 0, policy_version 36042 (0.0007) [2023-10-14 19:06:16,591][61585] Updated weights for policy 1, policy_version 35900 (0.0008) [2023-10-14 19:06:16,669][61552] Updated weights for policy 0, policy_version 36052 (0.0009) [2023-10-14 19:06:17,035][61552] Updated weights for policy 0, policy_version 36062 (0.0007) [2023-10-14 19:06:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 73695232. Throughput: 0: 1672.7, 1: 1659.3. Samples: 18424816. Policy #0 lag: (min: 23.0, avg: 23.1, max: 28.0) [2023-10-14 19:06:18,344][60425] Avg episode reward: [(0, '65.650'), (1, '57.700')] [2023-10-14 19:06:20,832][61585] Updated weights for policy 1, policy_version 35910 (0.0009) [2023-10-14 19:06:21,199][61585] Updated weights for policy 1, policy_version 35920 (0.0009) [2023-10-14 19:06:21,279][61552] Updated weights for policy 0, policy_version 36072 (0.0009) [2023-10-14 19:06:21,562][61585] Updated weights for policy 1, policy_version 35930 (0.0008) [2023-10-14 19:06:21,644][61552] Updated weights for policy 0, policy_version 36082 (0.0009) [2023-10-14 19:06:22,021][61552] Updated weights for policy 0, policy_version 36092 (0.0010) [2023-10-14 19:06:23,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 73760768. Throughput: 0: 1657.4, 1: 1649.6. Samples: 18443712. Policy #0 lag: (min: 23.0, avg: 23.1, max: 28.0) [2023-10-14 19:06:23,344][60425] Avg episode reward: [(0, '64.680'), (1, '59.610')] [2023-10-14 19:06:25,601][61585] Updated weights for policy 1, policy_version 35940 (0.0008) [2023-10-14 19:06:25,996][61585] Updated weights for policy 1, policy_version 35950 (0.0010) [2023-10-14 19:06:26,156][61552] Updated weights for policy 0, policy_version 36102 (0.0009) [2023-10-14 19:06:26,359][61585] Updated weights for policy 1, policy_version 35960 (0.0007) [2023-10-14 19:06:26,522][61552] Updated weights for policy 0, policy_version 36112 (0.0010) [2023-10-14 19:06:26,889][61552] Updated weights for policy 0, policy_version 36122 (0.0007) [2023-10-14 19:06:28,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 73826304. Throughput: 0: 1669.0, 1: 1668.5. Samples: 18463750. Policy #0 lag: (min: 23.0, avg: 23.1, max: 28.0) [2023-10-14 19:06:28,344][60425] Avg episode reward: [(0, '69.140'), (1, '57.010')] [2023-10-14 19:06:28,354][61172] Saving new best policy, reward=69.140! [2023-10-14 19:06:30,465][61585] Updated weights for policy 1, policy_version 35970 (0.0008) [2023-10-14 19:06:30,834][61585] Updated weights for policy 1, policy_version 35980 (0.0009) [2023-10-14 19:06:31,065][61552] Updated weights for policy 0, policy_version 36132 (0.0008) [2023-10-14 19:06:31,204][61585] Updated weights for policy 1, policy_version 35990 (0.0008) [2023-10-14 19:06:31,429][61552] Updated weights for policy 0, policy_version 36142 (0.0008) [2023-10-14 19:06:31,560][61585] Updated weights for policy 1, policy_version 36000 (0.0008) [2023-10-14 19:06:31,792][61552] Updated weights for policy 0, policy_version 36152 (0.0007) [2023-10-14 19:06:33,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 73891840. Throughput: 0: 1667.3, 1: 1660.5. Samples: 18474984. Policy #0 lag: (min: 23.0, avg: 23.1, max: 28.0) [2023-10-14 19:06:33,344][60425] Avg episode reward: [(0, '66.710'), (1, '61.690')] [2023-10-14 19:06:35,584][61585] Updated weights for policy 1, policy_version 36010 (0.0007) [2023-10-14 19:06:35,881][61552] Updated weights for policy 0, policy_version 36162 (0.0008) [2023-10-14 19:06:35,958][61585] Updated weights for policy 1, policy_version 36020 (0.0007) [2023-10-14 19:06:36,250][61552] Updated weights for policy 0, policy_version 36172 (0.0010) [2023-10-14 19:06:36,321][61585] Updated weights for policy 1, policy_version 36030 (0.0007) [2023-10-14 19:06:36,619][61552] Updated weights for policy 0, policy_version 36182 (0.0007) [2023-10-14 19:06:36,987][61552] Updated weights for policy 0, policy_version 36192 (0.0007) [2023-10-14 19:06:38,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 73957376. Throughput: 0: 1652.9, 1: 1658.4. Samples: 18493640. Policy #0 lag: (min: 23.0, avg: 23.1, max: 28.0) [2023-10-14 19:06:38,344][60425] Avg episode reward: [(0, '67.930'), (1, '59.230')] [2023-10-14 19:06:40,766][61585] Updated weights for policy 1, policy_version 36040 (0.0009) [2023-10-14 19:06:41,037][61552] Updated weights for policy 0, policy_version 36202 (0.0009) [2023-10-14 19:06:41,139][61585] Updated weights for policy 1, policy_version 36050 (0.0009) [2023-10-14 19:06:41,400][61552] Updated weights for policy 0, policy_version 36212 (0.0009) [2023-10-14 19:06:41,497][61585] Updated weights for policy 1, policy_version 36060 (0.0007) [2023-10-14 19:06:41,769][61552] Updated weights for policy 0, policy_version 36222 (0.0007) [2023-10-14 19:06:43,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 74022912. Throughput: 0: 1673.7, 1: 1662.5. Samples: 18513782. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-14 19:06:43,344][60425] Avg episode reward: [(0, '62.690'), (1, '63.040')] [2023-10-14 19:06:45,670][61585] Updated weights for policy 1, policy_version 36070 (0.0008) [2023-10-14 19:06:45,832][61552] Updated weights for policy 0, policy_version 36232 (0.0007) [2023-10-14 19:06:46,036][61585] Updated weights for policy 1, policy_version 36080 (0.0009) [2023-10-14 19:06:46,199][61552] Updated weights for policy 0, policy_version 36242 (0.0008) [2023-10-14 19:06:46,394][61585] Updated weights for policy 1, policy_version 36090 (0.0008) [2023-10-14 19:06:46,568][61552] Updated weights for policy 0, policy_version 36252 (0.0009) [2023-10-14 19:06:48,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 74088448. Throughput: 0: 1666.0, 1: 1655.1. Samples: 18524748. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-14 19:06:48,344][60425] Avg episode reward: [(0, '61.860'), (1, '58.380')] [2023-10-14 19:06:50,521][61585] Updated weights for policy 1, policy_version 36100 (0.0008) [2023-10-14 19:06:50,704][61552] Updated weights for policy 0, policy_version 36262 (0.0008) [2023-10-14 19:06:50,883][61585] Updated weights for policy 1, policy_version 36110 (0.0007) [2023-10-14 19:06:51,064][61552] Updated weights for policy 0, policy_version 36272 (0.0007) [2023-10-14 19:06:51,255][61585] Updated weights for policy 1, policy_version 36120 (0.0008) [2023-10-14 19:06:51,433][61552] Updated weights for policy 0, policy_version 36282 (0.0009) [2023-10-14 19:06:53,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 74153984. Throughput: 0: 1651.4, 1: 1655.6. Samples: 18543138. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-14 19:06:53,344][60425] Avg episode reward: [(0, '62.540'), (1, '61.770')] [2023-10-14 19:06:55,161][61585] Updated weights for policy 1, policy_version 36130 (0.0008) [2023-10-14 19:06:55,534][61585] Updated weights for policy 1, policy_version 36140 (0.0010) [2023-10-14 19:06:55,611][61552] Updated weights for policy 0, policy_version 36292 (0.0008) [2023-10-14 19:06:55,899][61585] Updated weights for policy 1, policy_version 36150 (0.0008) [2023-10-14 19:06:55,987][61552] Updated weights for policy 0, policy_version 36302 (0.0009) [2023-10-14 19:06:56,265][61585] Updated weights for policy 1, policy_version 36160 (0.0008) [2023-10-14 19:06:56,346][61552] Updated weights for policy 0, policy_version 36312 (0.0010) [2023-10-14 19:06:58,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 74219520. Throughput: 0: 1663.6, 1: 1659.8. Samples: 18563418. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-14 19:06:58,344][60425] Avg episode reward: [(0, '63.910'), (1, '59.450')] [2023-10-14 19:07:00,445][61552] Updated weights for policy 0, policy_version 36322 (0.0009) [2023-10-14 19:07:00,541][61585] Updated weights for policy 1, policy_version 36170 (0.0009) [2023-10-14 19:07:00,812][61552] Updated weights for policy 0, policy_version 36332 (0.0008) [2023-10-14 19:07:00,899][61585] Updated weights for policy 1, policy_version 36180 (0.0009) [2023-10-14 19:07:01,184][61552] Updated weights for policy 0, policy_version 36342 (0.0008) [2023-10-14 19:07:01,259][61585] Updated weights for policy 1, policy_version 36190 (0.0009) [2023-10-14 19:07:01,551][61552] Updated weights for policy 0, policy_version 36352 (0.0008) [2023-10-14 19:07:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 74285056. Throughput: 0: 1659.8, 1: 1654.9. Samples: 18573976. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-14 19:07:03,344][60425] Avg episode reward: [(0, '64.810'), (1, '59.390')] [2023-10-14 19:07:05,482][61585] Updated weights for policy 1, policy_version 36200 (0.0008) [2023-10-14 19:07:05,727][61552] Updated weights for policy 0, policy_version 36362 (0.0007) [2023-10-14 19:07:05,843][61585] Updated weights for policy 1, policy_version 36210 (0.0008) [2023-10-14 19:07:06,090][61552] Updated weights for policy 0, policy_version 36372 (0.0009) [2023-10-14 19:07:06,207][61585] Updated weights for policy 1, policy_version 36220 (0.0009) [2023-10-14 19:07:06,456][61552] Updated weights for policy 0, policy_version 36382 (0.0010) [2023-10-14 19:07:08,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 74350592. Throughput: 0: 1656.9, 1: 1660.9. Samples: 18593014. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:07:08,344][60425] Avg episode reward: [(0, '64.380'), (1, '56.770')] [2023-10-14 19:07:10,515][61585] Updated weights for policy 1, policy_version 36230 (0.0007) [2023-10-14 19:07:10,611][61552] Updated weights for policy 0, policy_version 36392 (0.0008) [2023-10-14 19:07:10,906][61585] Updated weights for policy 1, policy_version 36240 (0.0007) [2023-10-14 19:07:10,970][61552] Updated weights for policy 0, policy_version 36402 (0.0007) [2023-10-14 19:07:11,263][61585] Updated weights for policy 1, policy_version 36250 (0.0009) [2023-10-14 19:07:11,334][61552] Updated weights for policy 0, policy_version 36412 (0.0007) [2023-10-14 19:07:13,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 74416128. Throughput: 0: 1669.4, 1: 1655.6. Samples: 18613372. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:07:13,344][60425] Avg episode reward: [(0, '62.840'), (1, '58.130')] [2023-10-14 19:07:15,266][61585] Updated weights for policy 1, policy_version 36260 (0.0008) [2023-10-14 19:07:15,391][61552] Updated weights for policy 0, policy_version 36422 (0.0009) [2023-10-14 19:07:15,630][61585] Updated weights for policy 1, policy_version 36270 (0.0007) [2023-10-14 19:07:15,760][61552] Updated weights for policy 0, policy_version 36432 (0.0008) [2023-10-14 19:07:15,995][61585] Updated weights for policy 1, policy_version 36280 (0.0007) [2023-10-14 19:07:16,129][61552] Updated weights for policy 0, policy_version 36442 (0.0008) [2023-10-14 19:07:18,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 74481664. Throughput: 0: 1660.8, 1: 1647.4. Samples: 18623854. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:07:18,344][60425] Avg episode reward: [(0, '60.480'), (1, '60.990')] [2023-10-14 19:07:20,051][61552] Updated weights for policy 0, policy_version 36452 (0.0009) [2023-10-14 19:07:20,110][61585] Updated weights for policy 1, policy_version 36290 (0.0007) [2023-10-14 19:07:20,423][61552] Updated weights for policy 0, policy_version 36462 (0.0008) [2023-10-14 19:07:20,470][61585] Updated weights for policy 1, policy_version 36300 (0.0007) [2023-10-14 19:07:20,807][61552] Updated weights for policy 0, policy_version 36472 (0.0008) [2023-10-14 19:07:20,839][61585] Updated weights for policy 1, policy_version 36310 (0.0007) [2023-10-14 19:07:21,202][61585] Updated weights for policy 1, policy_version 36320 (0.0009) [2023-10-14 19:07:23,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 74547200. Throughput: 0: 1666.3, 1: 1657.4. Samples: 18643204. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:07:23,345][60425] Avg episode reward: [(0, '63.810'), (1, '58.330')] [2023-10-14 19:07:24,858][61552] Updated weights for policy 0, policy_version 36482 (0.0010) [2023-10-14 19:07:25,228][61552] Updated weights for policy 0, policy_version 36492 (0.0008) [2023-10-14 19:07:25,475][61585] Updated weights for policy 1, policy_version 36330 (0.0010) [2023-10-14 19:07:25,599][61552] Updated weights for policy 0, policy_version 36502 (0.0010) [2023-10-14 19:07:25,841][61585] Updated weights for policy 1, policy_version 36340 (0.0008) [2023-10-14 19:07:25,964][61552] Updated weights for policy 0, policy_version 36512 (0.0009) [2023-10-14 19:07:26,201][61585] Updated weights for policy 1, policy_version 36350 (0.0008) [2023-10-14 19:07:28,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 74612736. Throughput: 0: 1668.1, 1: 1660.1. Samples: 18663552. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:07:28,344][60425] Avg episode reward: [(0, '63.470'), (1, '59.750')] [2023-10-14 19:07:30,209][61552] Updated weights for policy 0, policy_version 36522 (0.0007) [2023-10-14 19:07:30,358][61585] Updated weights for policy 1, policy_version 36360 (0.0009) [2023-10-14 19:07:30,582][61552] Updated weights for policy 0, policy_version 36532 (0.0008) [2023-10-14 19:07:30,737][61585] Updated weights for policy 1, policy_version 36370 (0.0008) [2023-10-14 19:07:30,951][61552] Updated weights for policy 0, policy_version 36542 (0.0007) [2023-10-14 19:07:31,096][61585] Updated weights for policy 1, policy_version 36380 (0.0009) [2023-10-14 19:07:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 74678272. Throughput: 0: 1651.9, 1: 1653.0. Samples: 18673468. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-14 19:07:33,344][60425] Avg episode reward: [(0, '61.970'), (1, '64.750')] [2023-10-14 19:07:35,190][61552] Updated weights for policy 0, policy_version 36552 (0.0009) [2023-10-14 19:07:35,302][61585] Updated weights for policy 1, policy_version 36390 (0.0009) [2023-10-14 19:07:35,559][61552] Updated weights for policy 0, policy_version 36562 (0.0008) [2023-10-14 19:07:35,667][61585] Updated weights for policy 1, policy_version 36400 (0.0009) [2023-10-14 19:07:35,941][61552] Updated weights for policy 0, policy_version 36572 (0.0008) [2023-10-14 19:07:36,037][61585] Updated weights for policy 1, policy_version 36410 (0.0008) [2023-10-14 19:07:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 74743808. Throughput: 0: 1663.2, 1: 1657.2. Samples: 18692554. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-14 19:07:38,344][60425] Avg episode reward: [(0, '60.470'), (1, '60.060')] [2023-10-14 19:07:40,066][61552] Updated weights for policy 0, policy_version 36582 (0.0007) [2023-10-14 19:07:40,250][61585] Updated weights for policy 1, policy_version 36420 (0.0008) [2023-10-14 19:07:40,439][61552] Updated weights for policy 0, policy_version 36592 (0.0008) [2023-10-14 19:07:40,611][61585] Updated weights for policy 1, policy_version 36430 (0.0009) [2023-10-14 19:07:40,798][61552] Updated weights for policy 0, policy_version 36602 (0.0008) [2023-10-14 19:07:40,984][61585] Updated weights for policy 1, policy_version 36440 (0.0008) [2023-10-14 19:07:43,344][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 74809344. Throughput: 0: 1670.7, 1: 1653.3. Samples: 18713000. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-14 19:07:43,345][60425] Avg episode reward: [(0, '61.490'), (1, '61.200')] [2023-10-14 19:07:43,355][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000036608_37486592.pth... [2023-10-14 19:07:43,356][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000036448_37322752.pth... [2023-10-14 19:07:43,392][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000034880_35717120.pth [2023-10-14 19:07:43,395][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000035040_35880960.pth [2023-10-14 19:07:45,052][61552] Updated weights for policy 0, policy_version 36612 (0.0008) [2023-10-14 19:07:45,160][61585] Updated weights for policy 1, policy_version 36450 (0.0009) [2023-10-14 19:07:45,424][61552] Updated weights for policy 0, policy_version 36622 (0.0008) [2023-10-14 19:07:45,521][61585] Updated weights for policy 1, policy_version 36460 (0.0007) [2023-10-14 19:07:45,784][61552] Updated weights for policy 0, policy_version 36632 (0.0008) [2023-10-14 19:07:45,879][61585] Updated weights for policy 1, policy_version 36470 (0.0009) [2023-10-14 19:07:46,246][61585] Updated weights for policy 1, policy_version 36480 (0.0008) [2023-10-14 19:07:48,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 74874880. Throughput: 0: 1656.9, 1: 1650.7. Samples: 18722816. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-14 19:07:48,344][60425] Avg episode reward: [(0, '58.640'), (1, '63.380')] [2023-10-14 19:07:50,084][61552] Updated weights for policy 0, policy_version 36642 (0.0009) [2023-10-14 19:07:50,422][61585] Updated weights for policy 1, policy_version 36490 (0.0008) [2023-10-14 19:07:50,453][61552] Updated weights for policy 0, policy_version 36652 (0.0009) [2023-10-14 19:07:50,787][61585] Updated weights for policy 1, policy_version 36500 (0.0008) [2023-10-14 19:07:50,819][61552] Updated weights for policy 0, policy_version 36662 (0.0007) [2023-10-14 19:07:51,143][61585] Updated weights for policy 1, policy_version 36510 (0.0008) [2023-10-14 19:07:51,181][61552] Updated weights for policy 0, policy_version 36672 (0.0008) [2023-10-14 19:07:53,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 74940416. Throughput: 0: 1658.9, 1: 1651.3. Samples: 18741972. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-14 19:07:53,344][60425] Avg episode reward: [(0, '59.240'), (1, '59.350')] [2023-10-14 19:07:55,037][61585] Updated weights for policy 1, policy_version 36520 (0.0007) [2023-10-14 19:07:55,341][61552] Updated weights for policy 0, policy_version 36682 (0.0009) [2023-10-14 19:07:55,416][61585] Updated weights for policy 1, policy_version 36530 (0.0008) [2023-10-14 19:07:55,711][61552] Updated weights for policy 0, policy_version 36692 (0.0007) [2023-10-14 19:07:55,774][61585] Updated weights for policy 1, policy_version 36540 (0.0008) [2023-10-14 19:07:56,074][61552] Updated weights for policy 0, policy_version 36702 (0.0009) [2023-10-14 19:07:58,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 75005952. Throughput: 0: 1655.0, 1: 1658.2. Samples: 18762466. Policy #0 lag: (min: 19.0, avg: 26.7, max: 51.0) [2023-10-14 19:07:58,344][60425] Avg episode reward: [(0, '60.050'), (1, '65.010')] [2023-10-14 19:07:58,357][61248] Saving new best policy, reward=65.010! [2023-10-14 19:07:59,810][61585] Updated weights for policy 1, policy_version 36550 (0.0009) [2023-10-14 19:08:00,139][61552] Updated weights for policy 0, policy_version 36712 (0.0007) [2023-10-14 19:08:00,192][61585] Updated weights for policy 1, policy_version 36560 (0.0009) [2023-10-14 19:08:00,510][61552] Updated weights for policy 0, policy_version 36722 (0.0008) [2023-10-14 19:08:00,550][61585] Updated weights for policy 1, policy_version 36570 (0.0008) [2023-10-14 19:08:00,877][61552] Updated weights for policy 0, policy_version 36732 (0.0007) [2023-10-14 19:08:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 75071488. Throughput: 0: 1642.8, 1: 1647.0. Samples: 18771894. Policy #0 lag: (min: 19.0, avg: 26.7, max: 51.0) [2023-10-14 19:08:03,344][60425] Avg episode reward: [(0, '62.300'), (1, '61.710')] [2023-10-14 19:08:04,707][61585] Updated weights for policy 1, policy_version 36580 (0.0009) [2023-10-14 19:08:04,970][61552] Updated weights for policy 0, policy_version 36742 (0.0009) [2023-10-14 19:08:05,079][61585] Updated weights for policy 1, policy_version 36590 (0.0008) [2023-10-14 19:08:05,344][61552] Updated weights for policy 0, policy_version 36752 (0.0007) [2023-10-14 19:08:05,453][61585] Updated weights for policy 1, policy_version 36600 (0.0008) [2023-10-14 19:08:05,711][61552] Updated weights for policy 0, policy_version 36762 (0.0008) [2023-10-14 19:08:08,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 75137024. Throughput: 0: 1650.9, 1: 1656.0. Samples: 18792012. Policy #0 lag: (min: 19.0, avg: 26.7, max: 51.0) [2023-10-14 19:08:08,344][60425] Avg episode reward: [(0, '57.470'), (1, '62.220')] [2023-10-14 19:08:09,506][61585] Updated weights for policy 1, policy_version 36610 (0.0007) [2023-10-14 19:08:09,857][61552] Updated weights for policy 0, policy_version 36772 (0.0010) [2023-10-14 19:08:09,880][61585] Updated weights for policy 1, policy_version 36620 (0.0008) [2023-10-14 19:08:10,224][61552] Updated weights for policy 0, policy_version 36782 (0.0008) [2023-10-14 19:08:10,233][61585] Updated weights for policy 1, policy_version 36630 (0.0008) [2023-10-14 19:08:10,600][61585] Updated weights for policy 1, policy_version 36640 (0.0008) [2023-10-14 19:08:10,600][61552] Updated weights for policy 0, policy_version 36792 (0.0008) [2023-10-14 19:08:13,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 75202560. Throughput: 0: 1654.2, 1: 1656.8. Samples: 18812544. Policy #0 lag: (min: 19.0, avg: 26.7, max: 51.0) [2023-10-14 19:08:13,344][60425] Avg episode reward: [(0, '59.150'), (1, '62.640')] [2023-10-14 19:08:14,681][61552] Updated weights for policy 0, policy_version 36802 (0.0009) [2023-10-14 19:08:14,707][61585] Updated weights for policy 1, policy_version 36650 (0.0008) [2023-10-14 19:08:15,054][61552] Updated weights for policy 0, policy_version 36812 (0.0009) [2023-10-14 19:08:15,072][61585] Updated weights for policy 1, policy_version 36660 (0.0009) [2023-10-14 19:08:15,419][61552] Updated weights for policy 0, policy_version 36822 (0.0007) [2023-10-14 19:08:15,438][61585] Updated weights for policy 1, policy_version 36670 (0.0007) [2023-10-14 19:08:15,784][61552] Updated weights for policy 0, policy_version 36832 (0.0011) [2023-10-14 19:08:18,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 75268096. Throughput: 0: 1648.2, 1: 1648.1. Samples: 18821802. Policy #0 lag: (min: 19.0, avg: 26.7, max: 51.0) [2023-10-14 19:08:18,344][60425] Avg episode reward: [(0, '65.130'), (1, '61.520')] [2023-10-14 19:08:19,461][61585] Updated weights for policy 1, policy_version 36680 (0.0007) [2023-10-14 19:08:19,816][61585] Updated weights for policy 1, policy_version 36690 (0.0007) [2023-10-14 19:08:19,997][61552] Updated weights for policy 0, policy_version 36842 (0.0009) [2023-10-14 19:08:20,184][61585] Updated weights for policy 1, policy_version 36700 (0.0007) [2023-10-14 19:08:20,366][61552] Updated weights for policy 0, policy_version 36852 (0.0007) [2023-10-14 19:08:20,741][61552] Updated weights for policy 0, policy_version 36862 (0.0008) [2023-10-14 19:08:23,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 75333632. Throughput: 0: 1655.8, 1: 1665.8. Samples: 18842028. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-14 19:08:23,344][60425] Avg episode reward: [(0, '61.860'), (1, '58.240')] [2023-10-14 19:08:24,399][61585] Updated weights for policy 1, policy_version 36710 (0.0007) [2023-10-14 19:08:24,768][61585] Updated weights for policy 1, policy_version 36720 (0.0007) [2023-10-14 19:08:24,941][61552] Updated weights for policy 0, policy_version 36872 (0.0008) [2023-10-14 19:08:25,139][61585] Updated weights for policy 1, policy_version 36730 (0.0008) [2023-10-14 19:08:25,315][61552] Updated weights for policy 0, policy_version 36882 (0.0007) [2023-10-14 19:08:25,687][61552] Updated weights for policy 0, policy_version 36892 (0.0009) [2023-10-14 19:08:28,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 75399168. Throughput: 0: 1649.7, 1: 1669.4. Samples: 18862356. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-14 19:08:28,344][60425] Avg episode reward: [(0, '59.620'), (1, '60.140')] [2023-10-14 19:08:29,054][61585] Updated weights for policy 1, policy_version 36740 (0.0008) [2023-10-14 19:08:29,423][61585] Updated weights for policy 1, policy_version 36750 (0.0007) [2023-10-14 19:08:29,607][61552] Updated weights for policy 0, policy_version 36902 (0.0008) [2023-10-14 19:08:29,790][61585] Updated weights for policy 1, policy_version 36760 (0.0009) [2023-10-14 19:08:29,983][61552] Updated weights for policy 0, policy_version 36912 (0.0008) [2023-10-14 19:08:30,359][61552] Updated weights for policy 0, policy_version 36922 (0.0008) [2023-10-14 19:08:33,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 75464704. Throughput: 0: 1641.3, 1: 1663.6. Samples: 18871534. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-14 19:08:33,344][60425] Avg episode reward: [(0, '61.680'), (1, '57.720')] [2023-10-14 19:08:33,810][61585] Updated weights for policy 1, policy_version 36770 (0.0009) [2023-10-14 19:08:34,185][61585] Updated weights for policy 1, policy_version 36780 (0.0010) [2023-10-14 19:08:34,429][61552] Updated weights for policy 0, policy_version 36932 (0.0008) [2023-10-14 19:08:34,548][61585] Updated weights for policy 1, policy_version 36790 (0.0009) [2023-10-14 19:08:34,793][61552] Updated weights for policy 0, policy_version 36942 (0.0008) [2023-10-14 19:08:34,912][61585] Updated weights for policy 1, policy_version 36800 (0.0009) [2023-10-14 19:08:35,163][61552] Updated weights for policy 0, policy_version 36952 (0.0009) [2023-10-14 19:08:38,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 75530240. Throughput: 0: 1657.6, 1: 1674.9. Samples: 18891932. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-14 19:08:38,345][60425] Avg episode reward: [(0, '68.930'), (1, '59.910')] [2023-10-14 19:08:39,058][61585] Updated weights for policy 1, policy_version 36810 (0.0007) [2023-10-14 19:08:39,382][61552] Updated weights for policy 0, policy_version 36962 (0.0007) [2023-10-14 19:08:39,426][61585] Updated weights for policy 1, policy_version 36820 (0.0008) [2023-10-14 19:08:39,760][61552] Updated weights for policy 0, policy_version 36972 (0.0007) [2023-10-14 19:08:39,787][61585] Updated weights for policy 1, policy_version 36830 (0.0007) [2023-10-14 19:08:40,124][61552] Updated weights for policy 0, policy_version 36982 (0.0010) [2023-10-14 19:08:40,498][61552] Updated weights for policy 0, policy_version 36992 (0.0009) [2023-10-14 19:08:43,344][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 75595776. Throughput: 0: 1656.0, 1: 1676.3. Samples: 18912416. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-14 19:08:43,344][60425] Avg episode reward: [(0, '66.460'), (1, '58.020')] [2023-10-14 19:08:43,796][61585] Updated weights for policy 1, policy_version 36840 (0.0009) [2023-10-14 19:08:44,163][61585] Updated weights for policy 1, policy_version 36850 (0.0008) [2023-10-14 19:08:44,540][61585] Updated weights for policy 1, policy_version 36860 (0.0007) [2023-10-14 19:08:44,740][61552] Updated weights for policy 0, policy_version 37002 (0.0007) [2023-10-14 19:08:45,106][61552] Updated weights for policy 0, policy_version 37012 (0.0011) [2023-10-14 19:08:45,473][61552] Updated weights for policy 0, policy_version 37022 (0.0008) [2023-10-14 19:08:48,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 75661312. Throughput: 0: 1644.3, 1: 1674.0. Samples: 18921218. Policy #0 lag: (min: 2.0, avg: 2.0, max: 6.0) [2023-10-14 19:08:48,344][60425] Avg episode reward: [(0, '64.710'), (1, '58.260')] [2023-10-14 19:08:48,781][61585] Updated weights for policy 1, policy_version 36870 (0.0008) [2023-10-14 19:08:49,165][61585] Updated weights for policy 1, policy_version 36880 (0.0009) [2023-10-14 19:08:49,531][61585] Updated weights for policy 1, policy_version 36890 (0.0009) [2023-10-14 19:08:49,589][61552] Updated weights for policy 0, policy_version 37032 (0.0009) [2023-10-14 19:08:49,959][61552] Updated weights for policy 0, policy_version 37042 (0.0007) [2023-10-14 19:08:50,327][61552] Updated weights for policy 0, policy_version 37052 (0.0008) [2023-10-14 19:08:53,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 75726848. Throughput: 0: 1654.8, 1: 1669.1. Samples: 18941584. Policy #0 lag: (min: 2.0, avg: 2.0, max: 6.0) [2023-10-14 19:08:53,344][60425] Avg episode reward: [(0, '69.320'), (1, '58.150')] [2023-10-14 19:08:53,345][61172] Saving new best policy, reward=69.320! [2023-10-14 19:08:53,812][61585] Updated weights for policy 1, policy_version 36900 (0.0008) [2023-10-14 19:08:54,174][61585] Updated weights for policy 1, policy_version 36910 (0.0008) [2023-10-14 19:08:54,387][61552] Updated weights for policy 0, policy_version 37062 (0.0008) [2023-10-14 19:08:54,548][61585] Updated weights for policy 1, policy_version 36920 (0.0007) [2023-10-14 19:08:54,758][61552] Updated weights for policy 0, policy_version 37072 (0.0007) [2023-10-14 19:08:55,132][61552] Updated weights for policy 0, policy_version 37082 (0.0008) [2023-10-14 19:08:58,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 75792384. Throughput: 0: 1661.1, 1: 1669.1. Samples: 18962400. Policy #0 lag: (min: 2.0, avg: 2.0, max: 6.0) [2023-10-14 19:08:58,344][60425] Avg episode reward: [(0, '65.880'), (1, '57.590')] [2023-10-14 19:08:58,747][61585] Updated weights for policy 1, policy_version 36930 (0.0008) [2023-10-14 19:08:59,113][61585] Updated weights for policy 1, policy_version 36940 (0.0008) [2023-10-14 19:08:59,139][61552] Updated weights for policy 0, policy_version 37092 (0.0009) [2023-10-14 19:08:59,476][61585] Updated weights for policy 1, policy_version 36950 (0.0007) [2023-10-14 19:08:59,505][61552] Updated weights for policy 0, policy_version 37102 (0.0007) [2023-10-14 19:08:59,836][61585] Updated weights for policy 1, policy_version 36960 (0.0011) [2023-10-14 19:08:59,873][61552] Updated weights for policy 0, policy_version 37112 (0.0009) [2023-10-14 19:09:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 75857920. Throughput: 0: 1657.3, 1: 1665.6. Samples: 18971332. Policy #0 lag: (min: 2.0, avg: 2.0, max: 6.0) [2023-10-14 19:09:03,344][60425] Avg episode reward: [(0, '67.880'), (1, '57.380')] [2023-10-14 19:09:03,999][61585] Updated weights for policy 1, policy_version 36970 (0.0009) [2023-10-14 19:09:04,014][61552] Updated weights for policy 0, policy_version 37122 (0.0011) [2023-10-14 19:09:04,357][61585] Updated weights for policy 1, policy_version 36980 (0.0008) [2023-10-14 19:09:04,390][61552] Updated weights for policy 0, policy_version 37132 (0.0008) [2023-10-14 19:09:04,734][61585] Updated weights for policy 1, policy_version 36990 (0.0009) [2023-10-14 19:09:04,761][61552] Updated weights for policy 0, policy_version 37142 (0.0009) [2023-10-14 19:09:05,124][61552] Updated weights for policy 0, policy_version 37152 (0.0007) [2023-10-14 19:09:08,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 75923456. Throughput: 0: 1660.1, 1: 1662.4. Samples: 18991542. Policy #0 lag: (min: 2.0, avg: 2.0, max: 6.0) [2023-10-14 19:09:08,344][60425] Avg episode reward: [(0, '66.950'), (1, '56.570')] [2023-10-14 19:09:09,048][61585] Updated weights for policy 1, policy_version 37000 (0.0008) [2023-10-14 19:09:09,325][61552] Updated weights for policy 0, policy_version 37162 (0.0007) [2023-10-14 19:09:09,411][61585] Updated weights for policy 1, policy_version 37010 (0.0008) [2023-10-14 19:09:09,693][61552] Updated weights for policy 0, policy_version 37172 (0.0010) [2023-10-14 19:09:09,783][61585] Updated weights for policy 1, policy_version 37020 (0.0010) [2023-10-14 19:09:10,059][61552] Updated weights for policy 0, policy_version 37182 (0.0009) [2023-10-14 19:09:13,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 75988992. Throughput: 0: 1664.6, 1: 1661.8. Samples: 19012044. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:09:13,345][60425] Avg episode reward: [(0, '69.380'), (1, '58.510')] [2023-10-14 19:09:13,356][61172] Saving new best policy, reward=69.380! [2023-10-14 19:09:13,929][61585] Updated weights for policy 1, policy_version 37030 (0.0009) [2023-10-14 19:09:14,293][61585] Updated weights for policy 1, policy_version 37040 (0.0008) [2023-10-14 19:09:14,304][61552] Updated weights for policy 0, policy_version 37192 (0.0008) [2023-10-14 19:09:14,657][61585] Updated weights for policy 1, policy_version 37050 (0.0009) [2023-10-14 19:09:14,679][61552] Updated weights for policy 0, policy_version 37202 (0.0009) [2023-10-14 19:09:15,045][61552] Updated weights for policy 0, policy_version 37212 (0.0008) [2023-10-14 19:09:18,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 76054528. Throughput: 0: 1664.0, 1: 1656.4. Samples: 19020954. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:09:18,344][60425] Avg episode reward: [(0, '61.670'), (1, '54.540')] [2023-10-14 19:09:18,867][61585] Updated weights for policy 1, policy_version 37060 (0.0010) [2023-10-14 19:09:19,060][61552] Updated weights for policy 0, policy_version 37222 (0.0008) [2023-10-14 19:09:19,227][61585] Updated weights for policy 1, policy_version 37070 (0.0008) [2023-10-14 19:09:19,434][61552] Updated weights for policy 0, policy_version 37232 (0.0007) [2023-10-14 19:09:19,594][61585] Updated weights for policy 1, policy_version 37080 (0.0007) [2023-10-14 19:09:19,801][61552] Updated weights for policy 0, policy_version 37242 (0.0008) [2023-10-14 19:09:23,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 76120064. Throughput: 0: 1663.3, 1: 1654.8. Samples: 19041242. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:09:23,344][60425] Avg episode reward: [(0, '63.460'), (1, '60.590')] [2023-10-14 19:09:23,707][61585] Updated weights for policy 1, policy_version 37090 (0.0009) [2023-10-14 19:09:23,826][61552] Updated weights for policy 0, policy_version 37252 (0.0010) [2023-10-14 19:09:24,071][61585] Updated weights for policy 1, policy_version 37100 (0.0007) [2023-10-14 19:09:24,191][61552] Updated weights for policy 0, policy_version 37262 (0.0009) [2023-10-14 19:09:24,441][61585] Updated weights for policy 1, policy_version 37110 (0.0007) [2023-10-14 19:09:24,562][61552] Updated weights for policy 0, policy_version 37272 (0.0009) [2023-10-14 19:09:24,803][61585] Updated weights for policy 1, policy_version 37120 (0.0007) [2023-10-14 19:09:28,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 76185600. Throughput: 0: 1671.8, 1: 1652.4. Samples: 19062002. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:09:28,344][60425] Avg episode reward: [(0, '68.210'), (1, '55.200')] [2023-10-14 19:09:28,513][61552] Updated weights for policy 0, policy_version 37282 (0.0009) [2023-10-14 19:09:28,879][61552] Updated weights for policy 0, policy_version 37292 (0.0009) [2023-10-14 19:09:28,974][61585] Updated weights for policy 1, policy_version 37130 (0.0007) [2023-10-14 19:09:29,242][61552] Updated weights for policy 0, policy_version 37302 (0.0009) [2023-10-14 19:09:29,333][61585] Updated weights for policy 1, policy_version 37140 (0.0008) [2023-10-14 19:09:29,607][61552] Updated weights for policy 0, policy_version 37312 (0.0009) [2023-10-14 19:09:29,696][61585] Updated weights for policy 1, policy_version 37150 (0.0009) [2023-10-14 19:09:33,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 76251136. Throughput: 0: 1674.3, 1: 1653.6. Samples: 19070972. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:09:33,344][60425] Avg episode reward: [(0, '66.340'), (1, '58.900')] [2023-10-14 19:09:33,734][61585] Updated weights for policy 1, policy_version 37160 (0.0008) [2023-10-14 19:09:33,760][61552] Updated weights for policy 0, policy_version 37322 (0.0007) [2023-10-14 19:09:34,103][61585] Updated weights for policy 1, policy_version 37170 (0.0008) [2023-10-14 19:09:34,118][61552] Updated weights for policy 0, policy_version 37332 (0.0009) [2023-10-14 19:09:34,465][61585] Updated weights for policy 1, policy_version 37180 (0.0007) [2023-10-14 19:09:34,490][61552] Updated weights for policy 0, policy_version 37342 (0.0007) [2023-10-14 19:09:38,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 76316672. Throughput: 0: 1674.2, 1: 1660.5. Samples: 19091648. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:09:38,344][60425] Avg episode reward: [(0, '65.170'), (1, '59.580')] [2023-10-14 19:09:38,463][61552] Updated weights for policy 0, policy_version 37352 (0.0009) [2023-10-14 19:09:38,531][61585] Updated weights for policy 1, policy_version 37190 (0.0008) [2023-10-14 19:09:38,835][61552] Updated weights for policy 0, policy_version 37362 (0.0009) [2023-10-14 19:09:38,898][61585] Updated weights for policy 1, policy_version 37200 (0.0007) [2023-10-14 19:09:39,208][61552] Updated weights for policy 0, policy_version 37372 (0.0009) [2023-10-14 19:09:39,255][61585] Updated weights for policy 1, policy_version 37210 (0.0008) [2023-10-14 19:09:43,258][61585] Updated weights for policy 1, policy_version 37220 (0.0007) [2023-10-14 19:09:43,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 76382208. Throughput: 0: 1666.6, 1: 1660.7. Samples: 19112126. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-14 19:09:43,344][60425] Avg episode reward: [(0, '67.050'), (1, '60.290')] [2023-10-14 19:09:43,420][61552] Updated weights for policy 0, policy_version 37382 (0.0009) [2023-10-14 19:09:43,622][61585] Updated weights for policy 1, policy_version 37230 (0.0008) [2023-10-14 19:09:43,786][61552] Updated weights for policy 0, policy_version 37392 (0.0009) [2023-10-14 19:09:43,986][61585] Updated weights for policy 1, policy_version 37240 (0.0009) [2023-10-14 19:09:44,145][61552] Updated weights for policy 0, policy_version 37402 (0.0009) [2023-10-14 19:09:44,279][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000037248_38141952.pth... [2023-10-14 19:09:44,311][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000035680_36536320.pth [2023-10-14 19:09:44,365][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000037408_38305792.pth... [2023-10-14 19:09:44,394][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000035840_36700160.pth [2023-10-14 19:09:48,080][61585] Updated weights for policy 1, policy_version 37250 (0.0008) [2023-10-14 19:09:48,252][61552] Updated weights for policy 0, policy_version 37412 (0.0008) [2023-10-14 19:09:48,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 76447744. Throughput: 0: 1668.8, 1: 1661.8. Samples: 19121210. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-14 19:09:48,344][60425] Avg episode reward: [(0, '64.230'), (1, '59.120')] [2023-10-14 19:09:48,449][61585] Updated weights for policy 1, policy_version 37260 (0.0009) [2023-10-14 19:09:48,620][61552] Updated weights for policy 0, policy_version 37422 (0.0008) [2023-10-14 19:09:48,814][61585] Updated weights for policy 1, policy_version 37270 (0.0007) [2023-10-14 19:09:48,995][61552] Updated weights for policy 0, policy_version 37432 (0.0010) [2023-10-14 19:09:49,175][61585] Updated weights for policy 1, policy_version 37280 (0.0007) [2023-10-14 19:09:52,967][61552] Updated weights for policy 0, policy_version 37442 (0.0007) [2023-10-14 19:09:53,251][61585] Updated weights for policy 1, policy_version 37290 (0.0008) [2023-10-14 19:09:53,338][61552] Updated weights for policy 0, policy_version 37452 (0.0007) [2023-10-14 19:09:53,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 76513280. Throughput: 0: 1671.9, 1: 1663.1. Samples: 19141618. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-14 19:09:53,344][60425] Avg episode reward: [(0, '65.380'), (1, '58.600')] [2023-10-14 19:09:53,619][61585] Updated weights for policy 1, policy_version 37300 (0.0008) [2023-10-14 19:09:53,696][61552] Updated weights for policy 0, policy_version 37462 (0.0008) [2023-10-14 19:09:53,986][61585] Updated weights for policy 1, policy_version 37310 (0.0008) [2023-10-14 19:09:54,059][61552] Updated weights for policy 0, policy_version 37472 (0.0007) [2023-10-14 19:09:58,119][61585] Updated weights for policy 1, policy_version 37320 (0.0008) [2023-10-14 19:09:58,308][61552] Updated weights for policy 0, policy_version 37482 (0.0009) [2023-10-14 19:09:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 76578816. Throughput: 0: 1673.2, 1: 1665.3. Samples: 19162276. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-14 19:09:58,344][60425] Avg episode reward: [(0, '70.490'), (1, '59.890')] [2023-10-14 19:09:58,474][61585] Updated weights for policy 1, policy_version 37330 (0.0008) [2023-10-14 19:09:58,669][61552] Updated weights for policy 0, policy_version 37492 (0.0009) [2023-10-14 19:09:58,836][61585] Updated weights for policy 1, policy_version 37340 (0.0008) [2023-10-14 19:09:59,039][61552] Updated weights for policy 0, policy_version 37502 (0.0007) [2023-10-14 19:09:59,109][61172] Saving new best policy, reward=70.490! [2023-10-14 19:10:02,948][61585] Updated weights for policy 1, policy_version 37350 (0.0008) [2023-10-14 19:10:03,254][61552] Updated weights for policy 0, policy_version 37512 (0.0010) [2023-10-14 19:10:03,316][61585] Updated weights for policy 1, policy_version 37360 (0.0009) [2023-10-14 19:10:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 76644352. Throughput: 0: 1675.9, 1: 1665.5. Samples: 19171318. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-14 19:10:03,344][60425] Avg episode reward: [(0, '67.320'), (1, '58.140')] [2023-10-14 19:10:03,620][61552] Updated weights for policy 0, policy_version 37522 (0.0008) [2023-10-14 19:10:03,686][61585] Updated weights for policy 1, policy_version 37370 (0.0008) [2023-10-14 19:10:03,991][61552] Updated weights for policy 0, policy_version 37532 (0.0007) [2023-10-14 19:10:07,762][61585] Updated weights for policy 1, policy_version 37380 (0.0009) [2023-10-14 19:10:08,124][61585] Updated weights for policy 1, policy_version 37390 (0.0008) [2023-10-14 19:10:08,270][61552] Updated weights for policy 0, policy_version 37542 (0.0011) [2023-10-14 19:10:08,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 76709888. Throughput: 0: 1672.9, 1: 1671.7. Samples: 19191750. Policy #0 lag: (min: 24.0, avg: 45.0, max: 56.0) [2023-10-14 19:10:08,344][60425] Avg episode reward: [(0, '63.030'), (1, '61.460')] [2023-10-14 19:10:08,483][61585] Updated weights for policy 1, policy_version 37400 (0.0007) [2023-10-14 19:10:08,637][61552] Updated weights for policy 0, policy_version 37552 (0.0007) [2023-10-14 19:10:09,014][61552] Updated weights for policy 0, policy_version 37562 (0.0009) [2023-10-14 19:10:12,600][61585] Updated weights for policy 1, policy_version 37410 (0.0009) [2023-10-14 19:10:12,967][61585] Updated weights for policy 1, policy_version 37420 (0.0008) [2023-10-14 19:10:13,124][61552] Updated weights for policy 0, policy_version 37572 (0.0010) [2023-10-14 19:10:13,338][61585] Updated weights for policy 1, policy_version 37430 (0.0008) [2023-10-14 19:10:13,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 76775424. Throughput: 0: 1666.8, 1: 1667.1. Samples: 19212024. Policy #0 lag: (min: 24.0, avg: 45.0, max: 56.0) [2023-10-14 19:10:13,344][60425] Avg episode reward: [(0, '64.500'), (1, '55.910')] [2023-10-14 19:10:13,495][61552] Updated weights for policy 0, policy_version 37582 (0.0007) [2023-10-14 19:10:13,700][61585] Updated weights for policy 1, policy_version 37440 (0.0007) [2023-10-14 19:10:13,865][61552] Updated weights for policy 0, policy_version 37592 (0.0008) [2023-10-14 19:10:17,831][61585] Updated weights for policy 1, policy_version 37450 (0.0007) [2023-10-14 19:10:17,929][61552] Updated weights for policy 0, policy_version 37602 (0.0008) [2023-10-14 19:10:18,195][61585] Updated weights for policy 1, policy_version 37460 (0.0008) [2023-10-14 19:10:18,295][61552] Updated weights for policy 0, policy_version 37612 (0.0008) [2023-10-14 19:10:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 76840960. Throughput: 0: 1665.6, 1: 1672.2. Samples: 19221174. Policy #0 lag: (min: 24.0, avg: 45.0, max: 56.0) [2023-10-14 19:10:18,344][60425] Avg episode reward: [(0, '69.280'), (1, '59.500')] [2023-10-14 19:10:18,567][61585] Updated weights for policy 1, policy_version 37470 (0.0008) [2023-10-14 19:10:18,657][61552] Updated weights for policy 0, policy_version 37622 (0.0008) [2023-10-14 19:10:19,022][61552] Updated weights for policy 0, policy_version 37632 (0.0008) [2023-10-14 19:10:22,702][61585] Updated weights for policy 1, policy_version 37480 (0.0009) [2023-10-14 19:10:23,069][61585] Updated weights for policy 1, policy_version 37490 (0.0008) [2023-10-14 19:10:23,122][61552] Updated weights for policy 0, policy_version 37642 (0.0009) [2023-10-14 19:10:23,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 76906496. Throughput: 0: 1664.1, 1: 1671.5. Samples: 19241746. Policy #0 lag: (min: 24.0, avg: 45.0, max: 56.0) [2023-10-14 19:10:23,344][60425] Avg episode reward: [(0, '64.580'), (1, '57.920')] [2023-10-14 19:10:23,430][61585] Updated weights for policy 1, policy_version 37500 (0.0007) [2023-10-14 19:10:23,482][61552] Updated weights for policy 0, policy_version 37652 (0.0007) [2023-10-14 19:10:23,848][61552] Updated weights for policy 0, policy_version 37662 (0.0008) [2023-10-14 19:10:27,592][61585] Updated weights for policy 1, policy_version 37510 (0.0009) [2023-10-14 19:10:27,798][61552] Updated weights for policy 0, policy_version 37672 (0.0009) [2023-10-14 19:10:27,961][61585] Updated weights for policy 1, policy_version 37520 (0.0007) [2023-10-14 19:10:28,154][61552] Updated weights for policy 0, policy_version 37682 (0.0009) [2023-10-14 19:10:28,333][61585] Updated weights for policy 1, policy_version 37530 (0.0008) [2023-10-14 19:10:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 76972032. Throughput: 0: 1665.6, 1: 1656.7. Samples: 19261628. Policy #0 lag: (min: 24.0, avg: 45.0, max: 56.0) [2023-10-14 19:10:28,344][60425] Avg episode reward: [(0, '64.190'), (1, '61.130')] [2023-10-14 19:10:28,532][61552] Updated weights for policy 0, policy_version 37692 (0.0009) [2023-10-14 19:10:32,323][61585] Updated weights for policy 1, policy_version 37540 (0.0007) [2023-10-14 19:10:32,666][61552] Updated weights for policy 0, policy_version 37702 (0.0008) [2023-10-14 19:10:32,692][61585] Updated weights for policy 1, policy_version 37550 (0.0007) [2023-10-14 19:10:33,025][61552] Updated weights for policy 0, policy_version 37712 (0.0009) [2023-10-14 19:10:33,048][61585] Updated weights for policy 1, policy_version 37560 (0.0008) [2023-10-14 19:10:33,343][60425] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 77070336. Throughput: 0: 1671.2, 1: 1668.7. Samples: 19271508. Policy #0 lag: (min: 24.0, avg: 45.0, max: 56.0) [2023-10-14 19:10:33,344][60425] Avg episode reward: [(0, '66.410'), (1, '58.030')] [2023-10-14 19:10:33,396][61552] Updated weights for policy 0, policy_version 37722 (0.0007) [2023-10-14 19:10:37,408][61585] Updated weights for policy 1, policy_version 37570 (0.0007) [2023-10-14 19:10:37,612][61552] Updated weights for policy 0, policy_version 37732 (0.0009) [2023-10-14 19:10:37,773][61585] Updated weights for policy 1, policy_version 37580 (0.0008) [2023-10-14 19:10:37,975][61552] Updated weights for policy 0, policy_version 37742 (0.0009) [2023-10-14 19:10:38,146][61585] Updated weights for policy 1, policy_version 37590 (0.0008) [2023-10-14 19:10:38,342][61552] Updated weights for policy 0, policy_version 37752 (0.0008) [2023-10-14 19:10:38,344][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 77103104. Throughput: 0: 1671.5, 1: 1668.2. Samples: 19291902. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-14 19:10:38,345][60425] Avg episode reward: [(0, '63.830'), (1, '59.350')] [2023-10-14 19:10:38,506][61585] Updated weights for policy 1, policy_version 37600 (0.0008) [2023-10-14 19:10:42,496][61552] Updated weights for policy 0, policy_version 37762 (0.0010) [2023-10-14 19:10:42,570][61585] Updated weights for policy 1, policy_version 37610 (0.0008) [2023-10-14 19:10:42,866][61552] Updated weights for policy 0, policy_version 37772 (0.0008) [2023-10-14 19:10:42,927][61585] Updated weights for policy 1, policy_version 37620 (0.0008) [2023-10-14 19:10:43,232][61552] Updated weights for policy 0, policy_version 37782 (0.0007) [2023-10-14 19:10:43,290][61585] Updated weights for policy 1, policy_version 37630 (0.0007) [2023-10-14 19:10:43,343][60425] Fps is (10 sec: 9830.3, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 77168640. Throughput: 0: 1662.8, 1: 1655.9. Samples: 19311614. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-14 19:10:43,344][60425] Avg episode reward: [(0, '62.480'), (1, '58.460')] [2023-10-14 19:10:43,598][61552] Updated weights for policy 0, policy_version 37792 (0.0009) [2023-10-14 19:10:47,484][61585] Updated weights for policy 1, policy_version 37640 (0.0008) [2023-10-14 19:10:47,710][61552] Updated weights for policy 0, policy_version 37802 (0.0007) [2023-10-14 19:10:47,852][61585] Updated weights for policy 1, policy_version 37650 (0.0009) [2023-10-14 19:10:48,082][61552] Updated weights for policy 0, policy_version 37812 (0.0008) [2023-10-14 19:10:48,217][61585] Updated weights for policy 1, policy_version 37660 (0.0008) [2023-10-14 19:10:48,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 77234176. Throughput: 0: 1668.4, 1: 1667.9. Samples: 19321450. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-14 19:10:48,344][60425] Avg episode reward: [(0, '67.120'), (1, '56.450')] [2023-10-14 19:10:48,441][61552] Updated weights for policy 0, policy_version 37822 (0.0008) [2023-10-14 19:10:52,258][61585] Updated weights for policy 1, policy_version 37670 (0.0008) [2023-10-14 19:10:52,503][61552] Updated weights for policy 0, policy_version 37832 (0.0008) [2023-10-14 19:10:52,621][61585] Updated weights for policy 1, policy_version 37680 (0.0008) [2023-10-14 19:10:52,872][61552] Updated weights for policy 0, policy_version 37842 (0.0008) [2023-10-14 19:10:52,986][61585] Updated weights for policy 1, policy_version 37690 (0.0008) [2023-10-14 19:10:53,246][61552] Updated weights for policy 0, policy_version 37852 (0.0009) [2023-10-14 19:10:53,343][60425] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 77332480. Throughput: 0: 1675.0, 1: 1663.8. Samples: 19341996. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-14 19:10:53,344][60425] Avg episode reward: [(0, '62.370'), (1, '54.640')] [2023-10-14 19:10:57,211][61585] Updated weights for policy 1, policy_version 37700 (0.0007) [2023-10-14 19:10:57,387][61552] Updated weights for policy 0, policy_version 37862 (0.0009) [2023-10-14 19:10:57,571][61585] Updated weights for policy 1, policy_version 37710 (0.0008) [2023-10-14 19:10:57,749][61552] Updated weights for policy 0, policy_version 37872 (0.0009) [2023-10-14 19:10:57,936][61585] Updated weights for policy 1, policy_version 37720 (0.0008) [2023-10-14 19:10:58,125][61552] Updated weights for policy 0, policy_version 37882 (0.0009) [2023-10-14 19:10:58,343][60425] Fps is (10 sec: 19660.5, 60 sec: 14199.4, 300 sec: 13329.4). Total num frames: 77430784. Throughput: 0: 1661.7, 1: 1651.2. Samples: 19361104. Policy #0 lag: (min: 25.0, avg: 33.1, max: 57.0) [2023-10-14 19:10:58,344][60425] Avg episode reward: [(0, '63.200'), (1, '55.330')] [2023-10-14 19:11:01,934][61585] Updated weights for policy 1, policy_version 37730 (0.0009) [2023-10-14 19:11:02,299][61552] Updated weights for policy 0, policy_version 37892 (0.0008) [2023-10-14 19:11:02,300][61585] Updated weights for policy 1, policy_version 37740 (0.0007) [2023-10-14 19:11:02,660][61552] Updated weights for policy 0, policy_version 37902 (0.0008) [2023-10-14 19:11:02,674][61585] Updated weights for policy 1, policy_version 37750 (0.0009) [2023-10-14 19:11:03,024][61552] Updated weights for policy 0, policy_version 37912 (0.0007) [2023-10-14 19:11:03,035][61585] Updated weights for policy 1, policy_version 37760 (0.0008) [2023-10-14 19:11:03,343][60425] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 77496320. Throughput: 0: 1676.1, 1: 1662.7. Samples: 19371422. Policy #0 lag: (min: 25.0, avg: 33.1, max: 57.0) [2023-10-14 19:11:03,344][60425] Avg episode reward: [(0, '64.300'), (1, '58.710')] [2023-10-14 19:11:07,175][61585] Updated weights for policy 1, policy_version 37770 (0.0009) [2023-10-14 19:11:07,187][61552] Updated weights for policy 0, policy_version 37922 (0.0011) [2023-10-14 19:11:07,541][61585] Updated weights for policy 1, policy_version 37780 (0.0008) [2023-10-14 19:11:07,560][61552] Updated weights for policy 0, policy_version 37932 (0.0008) [2023-10-14 19:11:07,908][61585] Updated weights for policy 1, policy_version 37790 (0.0007) [2023-10-14 19:11:07,917][61552] Updated weights for policy 0, policy_version 37942 (0.0008) [2023-10-14 19:11:08,293][61552] Updated weights for policy 0, policy_version 37952 (0.0009) [2023-10-14 19:11:08,343][60425] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 77561856. Throughput: 0: 1671.9, 1: 1664.0. Samples: 19391858. Policy #0 lag: (min: 25.0, avg: 33.1, max: 57.0) [2023-10-14 19:11:08,344][60425] Avg episode reward: [(0, '66.420'), (1, '57.160')] [2023-10-14 19:11:12,092][61585] Updated weights for policy 1, policy_version 37800 (0.0007) [2023-10-14 19:11:12,389][61552] Updated weights for policy 0, policy_version 37962 (0.0008) [2023-10-14 19:11:12,451][61585] Updated weights for policy 1, policy_version 37810 (0.0008) [2023-10-14 19:11:12,759][61552] Updated weights for policy 0, policy_version 37972 (0.0008) [2023-10-14 19:11:12,813][61585] Updated weights for policy 1, policy_version 37820 (0.0009) [2023-10-14 19:11:13,124][61552] Updated weights for policy 0, policy_version 37982 (0.0009) [2023-10-14 19:11:13,343][60425] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 13329.3). Total num frames: 77627392. Throughput: 0: 1657.4, 1: 1656.8. Samples: 19410768. Policy #0 lag: (min: 25.0, avg: 33.1, max: 57.0) [2023-10-14 19:11:13,344][60425] Avg episode reward: [(0, '64.270'), (1, '63.910')] [2023-10-14 19:11:16,860][61585] Updated weights for policy 1, policy_version 37830 (0.0007) [2023-10-14 19:11:17,213][61552] Updated weights for policy 0, policy_version 37992 (0.0007) [2023-10-14 19:11:17,224][61585] Updated weights for policy 1, policy_version 37840 (0.0007) [2023-10-14 19:11:17,587][61585] Updated weights for policy 1, policy_version 37850 (0.0007) [2023-10-14 19:11:17,589][61552] Updated weights for policy 0, policy_version 38002 (0.0008) [2023-10-14 19:11:17,950][61552] Updated weights for policy 0, policy_version 38012 (0.0009) [2023-10-14 19:11:18,343][60425] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 77692928. Throughput: 0: 1665.6, 1: 1665.1. Samples: 19421388. Policy #0 lag: (min: 25.0, avg: 33.1, max: 57.0) [2023-10-14 19:11:18,344][60425] Avg episode reward: [(0, '63.320'), (1, '58.520')] [2023-10-14 19:11:21,757][61585] Updated weights for policy 1, policy_version 37860 (0.0010) [2023-10-14 19:11:22,048][61552] Updated weights for policy 0, policy_version 38022 (0.0009) [2023-10-14 19:11:22,123][61585] Updated weights for policy 1, policy_version 37870 (0.0010) [2023-10-14 19:11:22,426][61552] Updated weights for policy 0, policy_version 38032 (0.0007) [2023-10-14 19:11:22,493][61585] Updated weights for policy 1, policy_version 37880 (0.0007) [2023-10-14 19:11:22,788][61552] Updated weights for policy 0, policy_version 38042 (0.0009) [2023-10-14 19:11:23,343][60425] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 77758464. Throughput: 0: 1667.1, 1: 1663.7. Samples: 19441788. Policy #0 lag: (min: 25.0, avg: 33.1, max: 57.0) [2023-10-14 19:11:23,344][60425] Avg episode reward: [(0, '66.190'), (1, '61.600')] [2023-10-14 19:11:26,581][61585] Updated weights for policy 1, policy_version 37890 (0.0009) [2023-10-14 19:11:26,788][61552] Updated weights for policy 0, policy_version 38052 (0.0008) [2023-10-14 19:11:26,955][61585] Updated weights for policy 1, policy_version 37900 (0.0008) [2023-10-14 19:11:27,163][61552] Updated weights for policy 0, policy_version 38062 (0.0007) [2023-10-14 19:11:27,312][61585] Updated weights for policy 1, policy_version 37910 (0.0008) [2023-10-14 19:11:27,530][61552] Updated weights for policy 0, policy_version 38072 (0.0009) [2023-10-14 19:11:27,675][61585] Updated weights for policy 1, policy_version 37920 (0.0008) [2023-10-14 19:11:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 77824000. Throughput: 0: 1650.8, 1: 1656.2. Samples: 19460430. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-14 19:11:28,344][60425] Avg episode reward: [(0, '67.120'), (1, '56.440')] [2023-10-14 19:11:31,657][61552] Updated weights for policy 0, policy_version 38082 (0.0009) [2023-10-14 19:11:31,781][61585] Updated weights for policy 1, policy_version 37930 (0.0007) [2023-10-14 19:11:32,016][61552] Updated weights for policy 0, policy_version 38092 (0.0007) [2023-10-14 19:11:32,151][61585] Updated weights for policy 1, policy_version 37940 (0.0007) [2023-10-14 19:11:32,391][61552] Updated weights for policy 0, policy_version 38102 (0.0007) [2023-10-14 19:11:32,513][61585] Updated weights for policy 1, policy_version 37950 (0.0007) [2023-10-14 19:11:32,753][61552] Updated weights for policy 0, policy_version 38112 (0.0007) [2023-10-14 19:11:33,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 77889536. Throughput: 0: 1667.5, 1: 1674.3. Samples: 19471830. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-14 19:11:33,344][60425] Avg episode reward: [(0, '68.420'), (1, '58.020')] [2023-10-14 19:11:36,650][61585] Updated weights for policy 1, policy_version 37960 (0.0008) [2023-10-14 19:11:36,897][61552] Updated weights for policy 0, policy_version 38122 (0.0009) [2023-10-14 19:11:37,013][61585] Updated weights for policy 1, policy_version 37970 (0.0007) [2023-10-14 19:11:37,262][61552] Updated weights for policy 0, policy_version 38132 (0.0009) [2023-10-14 19:11:37,380][61585] Updated weights for policy 1, policy_version 37980 (0.0009) [2023-10-14 19:11:37,627][61552] Updated weights for policy 0, policy_version 38142 (0.0009) [2023-10-14 19:11:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 77955072. Throughput: 0: 1656.5, 1: 1666.8. Samples: 19491544. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-14 19:11:38,344][60425] Avg episode reward: [(0, '67.410'), (1, '56.980')] [2023-10-14 19:11:41,546][61585] Updated weights for policy 1, policy_version 37990 (0.0008) [2023-10-14 19:11:41,657][61552] Updated weights for policy 0, policy_version 38152 (0.0009) [2023-10-14 19:11:41,900][61585] Updated weights for policy 1, policy_version 38000 (0.0009) [2023-10-14 19:11:42,041][61552] Updated weights for policy 0, policy_version 38162 (0.0011) [2023-10-14 19:11:42,267][61585] Updated weights for policy 1, policy_version 38010 (0.0009) [2023-10-14 19:11:42,407][61552] Updated weights for policy 0, policy_version 38172 (0.0010) [2023-10-14 19:11:43,344][60425] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13329.3). Total num frames: 78020608. Throughput: 0: 1648.3, 1: 1666.8. Samples: 19510286. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-14 19:11:43,345][60425] Avg episode reward: [(0, '68.580'), (1, '56.100')] [2023-10-14 19:11:43,356][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000038176_39092224.pth... [2023-10-14 19:11:43,356][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000038016_38928384.pth... [2023-10-14 19:11:43,393][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000036608_37486592.pth [2023-10-14 19:11:43,394][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000036448_37322752.pth [2023-10-14 19:11:46,313][61585] Updated weights for policy 1, policy_version 38020 (0.0009) [2023-10-14 19:11:46,571][61552] Updated weights for policy 0, policy_version 38182 (0.0008) [2023-10-14 19:11:46,679][61585] Updated weights for policy 1, policy_version 38030 (0.0008) [2023-10-14 19:11:46,947][61552] Updated weights for policy 0, policy_version 38192 (0.0009) [2023-10-14 19:11:47,037][61585] Updated weights for policy 1, policy_version 38040 (0.0007) [2023-10-14 19:11:47,316][61552] Updated weights for policy 0, policy_version 38202 (0.0009) [2023-10-14 19:11:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 78086144. Throughput: 0: 1663.2, 1: 1677.3. Samples: 19521748. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-14 19:11:48,344][60425] Avg episode reward: [(0, '65.740'), (1, '56.420')] [2023-10-14 19:11:51,275][61585] Updated weights for policy 1, policy_version 38050 (0.0008) [2023-10-14 19:11:51,518][61552] Updated weights for policy 0, policy_version 38212 (0.0007) [2023-10-14 19:11:51,639][61585] Updated weights for policy 1, policy_version 38060 (0.0007) [2023-10-14 19:11:51,883][61552] Updated weights for policy 0, policy_version 38222 (0.0007) [2023-10-14 19:11:52,006][61585] Updated weights for policy 1, policy_version 38070 (0.0007) [2023-10-14 19:11:52,241][61552] Updated weights for policy 0, policy_version 38232 (0.0007) [2023-10-14 19:11:52,375][61585] Updated weights for policy 1, policy_version 38080 (0.0008) [2023-10-14 19:11:53,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 78151680. Throughput: 0: 1657.4, 1: 1660.9. Samples: 19541184. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-14 19:11:53,344][60425] Avg episode reward: [(0, '69.100'), (1, '55.600')] [2023-10-14 19:11:56,227][61552] Updated weights for policy 0, policy_version 38242 (0.0008) [2023-10-14 19:11:56,459][61585] Updated weights for policy 1, policy_version 38090 (0.0008) [2023-10-14 19:11:56,600][61552] Updated weights for policy 0, policy_version 38252 (0.0008) [2023-10-14 19:11:56,814][61585] Updated weights for policy 1, policy_version 38100 (0.0009) [2023-10-14 19:11:56,968][61552] Updated weights for policy 0, policy_version 38262 (0.0008) [2023-10-14 19:11:57,187][61585] Updated weights for policy 1, policy_version 38110 (0.0008) [2023-10-14 19:11:57,333][61552] Updated weights for policy 0, policy_version 38272 (0.0007) [2023-10-14 19:11:58,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 78217216. Throughput: 0: 1657.0, 1: 1665.6. Samples: 19560284. Policy #0 lag: (min: 10.0, avg: 12.8, max: 42.0) [2023-10-14 19:11:58,344][60425] Avg episode reward: [(0, '68.410'), (1, '56.770')] [2023-10-14 19:12:01,247][61585] Updated weights for policy 1, policy_version 38120 (0.0008) [2023-10-14 19:12:01,589][61552] Updated weights for policy 0, policy_version 38282 (0.0008) [2023-10-14 19:12:01,622][61585] Updated weights for policy 1, policy_version 38130 (0.0009) [2023-10-14 19:12:01,953][61552] Updated weights for policy 0, policy_version 38292 (0.0009) [2023-10-14 19:12:01,991][61585] Updated weights for policy 1, policy_version 38140 (0.0008) [2023-10-14 19:12:02,312][61552] Updated weights for policy 0, policy_version 38302 (0.0009) [2023-10-14 19:12:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 78282752. Throughput: 0: 1671.7, 1: 1673.3. Samples: 19571916. Policy #0 lag: (min: 10.0, avg: 12.8, max: 42.0) [2023-10-14 19:12:03,344][60425] Avg episode reward: [(0, '67.390'), (1, '58.550')] [2023-10-14 19:12:06,041][61585] Updated weights for policy 1, policy_version 38150 (0.0009) [2023-10-14 19:12:06,283][61552] Updated weights for policy 0, policy_version 38312 (0.0009) [2023-10-14 19:12:06,403][61585] Updated weights for policy 1, policy_version 38160 (0.0007) [2023-10-14 19:12:06,644][61552] Updated weights for policy 0, policy_version 38322 (0.0008) [2023-10-14 19:12:06,771][61585] Updated weights for policy 1, policy_version 38170 (0.0008) [2023-10-14 19:12:07,010][61552] Updated weights for policy 0, policy_version 38332 (0.0008) [2023-10-14 19:12:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 78348288. Throughput: 0: 1659.8, 1: 1650.2. Samples: 19590738. Policy #0 lag: (min: 10.0, avg: 12.8, max: 42.0) [2023-10-14 19:12:08,344][60425] Avg episode reward: [(0, '66.110'), (1, '55.640')] [2023-10-14 19:12:10,832][61585] Updated weights for policy 1, policy_version 38180 (0.0009) [2023-10-14 19:12:11,200][61585] Updated weights for policy 1, policy_version 38190 (0.0010) [2023-10-14 19:12:11,252][61552] Updated weights for policy 0, policy_version 38342 (0.0010) [2023-10-14 19:12:11,572][61585] Updated weights for policy 1, policy_version 38200 (0.0009) [2023-10-14 19:12:11,622][61552] Updated weights for policy 0, policy_version 38352 (0.0009) [2023-10-14 19:12:11,979][61552] Updated weights for policy 0, policy_version 38362 (0.0009) [2023-10-14 19:12:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 78413824. Throughput: 0: 1668.2, 1: 1664.0. Samples: 19610382. Policy #0 lag: (min: 10.0, avg: 12.8, max: 42.0) [2023-10-14 19:12:13,344][60425] Avg episode reward: [(0, '70.170'), (1, '57.360')] [2023-10-14 19:12:15,798][61585] Updated weights for policy 1, policy_version 38210 (0.0007) [2023-10-14 19:12:15,963][61552] Updated weights for policy 0, policy_version 38372 (0.0010) [2023-10-14 19:12:16,150][61585] Updated weights for policy 1, policy_version 38220 (0.0011) [2023-10-14 19:12:16,332][61552] Updated weights for policy 0, policy_version 38382 (0.0008) [2023-10-14 19:12:16,517][61585] Updated weights for policy 1, policy_version 38230 (0.0010) [2023-10-14 19:12:16,695][61552] Updated weights for policy 0, policy_version 38392 (0.0008) [2023-10-14 19:12:16,875][61585] Updated weights for policy 1, policy_version 38240 (0.0009) [2023-10-14 19:12:18,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 78479360. Throughput: 0: 1671.3, 1: 1657.3. Samples: 19621618. Policy #0 lag: (min: 10.0, avg: 12.8, max: 42.0) [2023-10-14 19:12:18,344][60425] Avg episode reward: [(0, '64.990'), (1, '59.140')] [2023-10-14 19:12:21,041][61585] Updated weights for policy 1, policy_version 38250 (0.0009) [2023-10-14 19:12:21,041][61552] Updated weights for policy 0, policy_version 38402 (0.0009) [2023-10-14 19:12:21,410][61585] Updated weights for policy 1, policy_version 38260 (0.0008) [2023-10-14 19:12:21,459][61552] Updated weights for policy 0, policy_version 38412 (0.0010) [2023-10-14 19:12:21,777][61585] Updated weights for policy 1, policy_version 38270 (0.0008) [2023-10-14 19:12:21,832][61552] Updated weights for policy 0, policy_version 38422 (0.0009) [2023-10-14 19:12:22,193][61552] Updated weights for policy 0, policy_version 38432 (0.0010) [2023-10-14 19:12:23,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 78544896. Throughput: 0: 1658.0, 1: 1642.9. Samples: 19640088. Policy #0 lag: (min: 20.0, avg: 26.0, max: 52.0) [2023-10-14 19:12:23,344][60425] Avg episode reward: [(0, '63.390'), (1, '58.740')] [2023-10-14 19:12:25,966][61585] Updated weights for policy 1, policy_version 38280 (0.0010) [2023-10-14 19:12:26,179][61552] Updated weights for policy 0, policy_version 38442 (0.0008) [2023-10-14 19:12:26,332][61585] Updated weights for policy 1, policy_version 38290 (0.0008) [2023-10-14 19:12:26,543][61552] Updated weights for policy 0, policy_version 38452 (0.0010) [2023-10-14 19:12:26,693][61585] Updated weights for policy 1, policy_version 38300 (0.0008) [2023-10-14 19:12:26,908][61552] Updated weights for policy 0, policy_version 38462 (0.0009) [2023-10-14 19:12:28,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 78610432. Throughput: 0: 1671.8, 1: 1654.9. Samples: 19659988. Policy #0 lag: (min: 20.0, avg: 26.0, max: 52.0) [2023-10-14 19:12:28,344][60425] Avg episode reward: [(0, '65.120'), (1, '52.960')] [2023-10-14 19:12:30,884][61585] Updated weights for policy 1, policy_version 38310 (0.0007) [2023-10-14 19:12:31,012][61552] Updated weights for policy 0, policy_version 38472 (0.0007) [2023-10-14 19:12:31,256][61585] Updated weights for policy 1, policy_version 38320 (0.0008) [2023-10-14 19:12:31,374][61552] Updated weights for policy 0, policy_version 38482 (0.0007) [2023-10-14 19:12:31,619][61585] Updated weights for policy 1, policy_version 38330 (0.0008) [2023-10-14 19:12:31,748][61552] Updated weights for policy 0, policy_version 38492 (0.0007) [2023-10-14 19:12:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 78675968. Throughput: 0: 1673.9, 1: 1649.6. Samples: 19671306. Policy #0 lag: (min: 20.0, avg: 26.0, max: 52.0) [2023-10-14 19:12:33,344][60425] Avg episode reward: [(0, '66.190'), (1, '62.560')] [2023-10-14 19:12:35,803][61585] Updated weights for policy 1, policy_version 38340 (0.0008) [2023-10-14 19:12:35,965][61552] Updated weights for policy 0, policy_version 38502 (0.0009) [2023-10-14 19:12:36,165][61585] Updated weights for policy 1, policy_version 38350 (0.0009) [2023-10-14 19:12:36,339][61552] Updated weights for policy 0, policy_version 38512 (0.0007) [2023-10-14 19:12:36,521][61585] Updated weights for policy 1, policy_version 38360 (0.0009) [2023-10-14 19:12:36,704][61552] Updated weights for policy 0, policy_version 38522 (0.0009) [2023-10-14 19:12:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 78741504. Throughput: 0: 1657.1, 1: 1641.5. Samples: 19689620. Policy #0 lag: (min: 20.0, avg: 26.0, max: 52.0) [2023-10-14 19:12:38,344][60425] Avg episode reward: [(0, '61.070'), (1, '56.190')] [2023-10-14 19:12:40,706][61585] Updated weights for policy 1, policy_version 38370 (0.0009) [2023-10-14 19:12:40,795][61552] Updated weights for policy 0, policy_version 38532 (0.0008) [2023-10-14 19:12:41,069][61585] Updated weights for policy 1, policy_version 38380 (0.0008) [2023-10-14 19:12:41,164][61552] Updated weights for policy 0, policy_version 38542 (0.0009) [2023-10-14 19:12:41,440][61585] Updated weights for policy 1, policy_version 38390 (0.0008) [2023-10-14 19:12:41,527][61552] Updated weights for policy 0, policy_version 38552 (0.0009) [2023-10-14 19:12:41,804][61585] Updated weights for policy 1, policy_version 38400 (0.0007) [2023-10-14 19:12:43,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 78807040. Throughput: 0: 1666.9, 1: 1653.5. Samples: 19709702. Policy #0 lag: (min: 20.0, avg: 26.0, max: 52.0) [2023-10-14 19:12:43,345][60425] Avg episode reward: [(0, '61.990'), (1, '61.310')] [2023-10-14 19:12:45,616][61552] Updated weights for policy 0, policy_version 38562 (0.0010) [2023-10-14 19:12:45,987][61552] Updated weights for policy 0, policy_version 38572 (0.0008) [2023-10-14 19:12:46,007][61585] Updated weights for policy 1, policy_version 38410 (0.0010) [2023-10-14 19:12:46,352][61552] Updated weights for policy 0, policy_version 38582 (0.0009) [2023-10-14 19:12:46,363][61585] Updated weights for policy 1, policy_version 38420 (0.0007) [2023-10-14 19:12:46,713][61552] Updated weights for policy 0, policy_version 38592 (0.0009) [2023-10-14 19:12:46,731][61585] Updated weights for policy 1, policy_version 38430 (0.0009) [2023-10-14 19:12:48,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 78872576. Throughput: 0: 1664.4, 1: 1646.0. Samples: 19720886. Policy #0 lag: (min: 20.0, avg: 26.0, max: 52.0) [2023-10-14 19:12:48,344][60425] Avg episode reward: [(0, '65.580'), (1, '56.100')] [2023-10-14 19:12:50,785][61552] Updated weights for policy 0, policy_version 38602 (0.0008) [2023-10-14 19:12:50,862][61585] Updated weights for policy 1, policy_version 38440 (0.0009) [2023-10-14 19:12:51,150][61552] Updated weights for policy 0, policy_version 38612 (0.0009) [2023-10-14 19:12:51,216][61585] Updated weights for policy 1, policy_version 38450 (0.0008) [2023-10-14 19:12:51,520][61552] Updated weights for policy 0, policy_version 38622 (0.0008) [2023-10-14 19:12:51,587][61585] Updated weights for policy 1, policy_version 38460 (0.0007) [2023-10-14 19:12:53,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 78938112. Throughput: 0: 1650.0, 1: 1645.4. Samples: 19739032. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-14 19:12:53,344][60425] Avg episode reward: [(0, '65.400'), (1, '57.010')] [2023-10-14 19:12:55,465][61585] Updated weights for policy 1, policy_version 38470 (0.0009) [2023-10-14 19:12:55,672][61552] Updated weights for policy 0, policy_version 38632 (0.0009) [2023-10-14 19:12:55,827][61585] Updated weights for policy 1, policy_version 38480 (0.0008) [2023-10-14 19:12:56,031][61552] Updated weights for policy 0, policy_version 38642 (0.0007) [2023-10-14 19:12:56,186][61585] Updated weights for policy 1, policy_version 38490 (0.0007) [2023-10-14 19:12:56,409][61552] Updated weights for policy 0, policy_version 38652 (0.0010) [2023-10-14 19:12:58,344][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 79003648. Throughput: 0: 1661.2, 1: 1658.3. Samples: 19759760. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-14 19:12:58,345][60425] Avg episode reward: [(0, '69.030'), (1, '57.030')] [2023-10-14 19:13:00,458][61585] Updated weights for policy 1, policy_version 38500 (0.0007) [2023-10-14 19:13:00,499][61552] Updated weights for policy 0, policy_version 38662 (0.0007) [2023-10-14 19:13:00,827][61585] Updated weights for policy 1, policy_version 38510 (0.0007) [2023-10-14 19:13:00,874][61552] Updated weights for policy 0, policy_version 38672 (0.0008) [2023-10-14 19:13:01,183][61585] Updated weights for policy 1, policy_version 38520 (0.0008) [2023-10-14 19:13:01,245][61552] Updated weights for policy 0, policy_version 38682 (0.0010) [2023-10-14 19:13:03,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 79069184. Throughput: 0: 1650.2, 1: 1651.0. Samples: 19770172. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-14 19:13:03,344][60425] Avg episode reward: [(0, '62.930'), (1, '57.030')] [2023-10-14 19:13:05,161][61585] Updated weights for policy 1, policy_version 38530 (0.0008) [2023-10-14 19:13:05,352][61552] Updated weights for policy 0, policy_version 38692 (0.0009) [2023-10-14 19:13:05,527][61585] Updated weights for policy 1, policy_version 38540 (0.0007) [2023-10-14 19:13:05,725][61552] Updated weights for policy 0, policy_version 38702 (0.0008) [2023-10-14 19:13:05,888][61585] Updated weights for policy 1, policy_version 38550 (0.0008) [2023-10-14 19:13:06,082][61552] Updated weights for policy 0, policy_version 38712 (0.0008) [2023-10-14 19:13:06,248][61585] Updated weights for policy 1, policy_version 38560 (0.0007) [2023-10-14 19:13:08,343][60425] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 79134720. Throughput: 0: 1653.0, 1: 1657.4. Samples: 19789056. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-14 19:13:08,344][60425] Avg episode reward: [(0, '67.310'), (1, '55.390')] [2023-10-14 19:13:10,194][61552] Updated weights for policy 0, policy_version 38722 (0.0009) [2023-10-14 19:13:10,438][61585] Updated weights for policy 1, policy_version 38570 (0.0007) [2023-10-14 19:13:10,564][61552] Updated weights for policy 0, policy_version 38732 (0.0011) [2023-10-14 19:13:10,801][61585] Updated weights for policy 1, policy_version 38580 (0.0008) [2023-10-14 19:13:10,936][61552] Updated weights for policy 0, policy_version 38742 (0.0007) [2023-10-14 19:13:11,162][61585] Updated weights for policy 1, policy_version 38590 (0.0007) [2023-10-14 19:13:11,296][61552] Updated weights for policy 0, policy_version 38752 (0.0008) [2023-10-14 19:13:13,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 79200256. Throughput: 0: 1656.8, 1: 1660.2. Samples: 19809256. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-14 19:13:13,344][60425] Avg episode reward: [(0, '63.180'), (1, '63.590')] [2023-10-14 19:13:15,398][61585] Updated weights for policy 1, policy_version 38600 (0.0007) [2023-10-14 19:13:15,478][61552] Updated weights for policy 0, policy_version 38762 (0.0007) [2023-10-14 19:13:15,761][61585] Updated weights for policy 1, policy_version 38610 (0.0007) [2023-10-14 19:13:15,850][61552] Updated weights for policy 0, policy_version 38772 (0.0009) [2023-10-14 19:13:16,131][61585] Updated weights for policy 1, policy_version 38620 (0.0008) [2023-10-14 19:13:16,217][61552] Updated weights for policy 0, policy_version 38782 (0.0009) [2023-10-14 19:13:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 79265792. Throughput: 0: 1640.8, 1: 1652.5. Samples: 19819508. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-14 19:13:18,344][60425] Avg episode reward: [(0, '67.510'), (1, '62.110')] [2023-10-14 19:13:20,211][61585] Updated weights for policy 1, policy_version 38630 (0.0009) [2023-10-14 19:13:20,440][61552] Updated weights for policy 0, policy_version 38792 (0.0007) [2023-10-14 19:13:20,572][61585] Updated weights for policy 1, policy_version 38640 (0.0007) [2023-10-14 19:13:20,816][61552] Updated weights for policy 0, policy_version 38802 (0.0009) [2023-10-14 19:13:20,933][61585] Updated weights for policy 1, policy_version 38650 (0.0009) [2023-10-14 19:13:21,179][61552] Updated weights for policy 0, policy_version 38812 (0.0008) [2023-10-14 19:13:23,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 79331328. Throughput: 0: 1650.1, 1: 1661.2. Samples: 19838628. Policy #0 lag: (min: 3.0, avg: 13.5, max: 35.0) [2023-10-14 19:13:23,345][60425] Avg episode reward: [(0, '66.080'), (1, '59.220')] [2023-10-14 19:13:24,936][61585] Updated weights for policy 1, policy_version 38660 (0.0008) [2023-10-14 19:13:25,299][61552] Updated weights for policy 0, policy_version 38822 (0.0010) [2023-10-14 19:13:25,304][61585] Updated weights for policy 1, policy_version 38670 (0.0007) [2023-10-14 19:13:25,666][61585] Updated weights for policy 1, policy_version 38680 (0.0008) [2023-10-14 19:13:25,669][61552] Updated weights for policy 0, policy_version 38832 (0.0009) [2023-10-14 19:13:26,040][61552] Updated weights for policy 0, policy_version 38842 (0.0009) [2023-10-14 19:13:28,343][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 79396864. Throughput: 0: 1655.0, 1: 1669.1. Samples: 19859288. Policy #0 lag: (min: 3.0, avg: 13.5, max: 35.0) [2023-10-14 19:13:28,344][60425] Avg episode reward: [(0, '68.870'), (1, '61.810')] [2023-10-14 19:13:29,953][61585] Updated weights for policy 1, policy_version 38690 (0.0008) [2023-10-14 19:13:30,153][61552] Updated weights for policy 0, policy_version 38852 (0.0011) [2023-10-14 19:13:30,325][61585] Updated weights for policy 1, policy_version 38700 (0.0007) [2023-10-14 19:13:30,525][61552] Updated weights for policy 0, policy_version 38862 (0.0009) [2023-10-14 19:13:30,693][61585] Updated weights for policy 1, policy_version 38710 (0.0008) [2023-10-14 19:13:30,897][61552] Updated weights for policy 0, policy_version 38872 (0.0010) [2023-10-14 19:13:31,059][61585] Updated weights for policy 1, policy_version 38720 (0.0009) [2023-10-14 19:13:33,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 79462400. Throughput: 0: 1641.2, 1: 1656.9. Samples: 19869298. Policy #0 lag: (min: 3.0, avg: 13.5, max: 35.0) [2023-10-14 19:13:33,344][60425] Avg episode reward: [(0, '61.980'), (1, '61.640')] [2023-10-14 19:13:35,065][61552] Updated weights for policy 0, policy_version 38882 (0.0010) [2023-10-14 19:13:35,231][61585] Updated weights for policy 1, policy_version 38730 (0.0007) [2023-10-14 19:13:35,434][61552] Updated weights for policy 0, policy_version 38892 (0.0007) [2023-10-14 19:13:35,617][61585] Updated weights for policy 1, policy_version 38740 (0.0008) [2023-10-14 19:13:35,809][61552] Updated weights for policy 0, policy_version 38902 (0.0007) [2023-10-14 19:13:35,980][61585] Updated weights for policy 1, policy_version 38750 (0.0007) [2023-10-14 19:13:36,179][61552] Updated weights for policy 0, policy_version 38912 (0.0008) [2023-10-14 19:13:38,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 79527936. Throughput: 0: 1653.1, 1: 1675.8. Samples: 19888832. Policy #0 lag: (min: 3.0, avg: 13.5, max: 35.0) [2023-10-14 19:13:38,344][60425] Avg episode reward: [(0, '62.560'), (1, '56.810')] [2023-10-14 19:13:39,912][61585] Updated weights for policy 1, policy_version 38760 (0.0011) [2023-10-14 19:13:40,255][61552] Updated weights for policy 0, policy_version 38922 (0.0007) [2023-10-14 19:13:40,280][61585] Updated weights for policy 1, policy_version 38770 (0.0007) [2023-10-14 19:13:40,623][61552] Updated weights for policy 0, policy_version 38932 (0.0009) [2023-10-14 19:13:40,643][61585] Updated weights for policy 1, policy_version 38780 (0.0008) [2023-10-14 19:13:40,986][61552] Updated weights for policy 0, policy_version 38942 (0.0010) [2023-10-14 19:13:43,344][60425] Fps is (10 sec: 13106.7, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 79593472. Throughput: 0: 1653.5, 1: 1671.6. Samples: 19909392. Policy #0 lag: (min: 3.0, avg: 13.5, max: 35.0) [2023-10-14 19:13:43,345][60425] Avg episode reward: [(0, '66.580'), (1, '56.070')] [2023-10-14 19:13:43,355][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000038784_39714816.pth... [2023-10-14 19:13:43,356][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000038944_39878656.pth... [2023-10-14 19:13:43,387][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000037248_38141952.pth [2023-10-14 19:13:43,390][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000037408_38305792.pth [2023-10-14 19:13:43,391][61248] Saving a milestone ./train_atari/atari_roadrunner_APPO/checkpoint_p1/milestones/checkpoint_000038784_39714816.pth [2023-10-14 19:13:43,394][61172] Saving a milestone ./train_atari/atari_roadrunner_APPO/checkpoint_p0/milestones/checkpoint_000038944_39878656.pth [2023-10-14 19:13:44,724][61585] Updated weights for policy 1, policy_version 38790 (0.0009) [2023-10-14 19:13:45,083][61585] Updated weights for policy 1, policy_version 38800 (0.0009) [2023-10-14 19:13:45,262][61552] Updated weights for policy 0, policy_version 38952 (0.0010) [2023-10-14 19:13:45,448][61585] Updated weights for policy 1, policy_version 38810 (0.0008) [2023-10-14 19:13:45,628][61552] Updated weights for policy 0, policy_version 38962 (0.0008) [2023-10-14 19:13:45,997][61552] Updated weights for policy 0, policy_version 38972 (0.0010) [2023-10-14 19:13:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 79659008. Throughput: 0: 1646.4, 1: 1656.5. Samples: 19918804. Policy #0 lag: (min: 16.0, avg: 39.4, max: 48.0) [2023-10-14 19:13:48,344][60425] Avg episode reward: [(0, '68.180'), (1, '58.260')] [2023-10-14 19:13:49,513][61585] Updated weights for policy 1, policy_version 38820 (0.0008) [2023-10-14 19:13:49,875][61585] Updated weights for policy 1, policy_version 38830 (0.0008) [2023-10-14 19:13:50,032][61552] Updated weights for policy 0, policy_version 38982 (0.0007) [2023-10-14 19:13:50,243][61585] Updated weights for policy 1, policy_version 38840 (0.0009) [2023-10-14 19:13:50,391][61552] Updated weights for policy 0, policy_version 38992 (0.0008) [2023-10-14 19:13:50,764][61552] Updated weights for policy 0, policy_version 39002 (0.0007) [2023-10-14 19:13:53,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 79724544. Throughput: 0: 1653.7, 1: 1676.1. Samples: 19938898. Policy #0 lag: (min: 16.0, avg: 39.4, max: 48.0) [2023-10-14 19:13:53,344][60425] Avg episode reward: [(0, '65.830'), (1, '58.280')] [2023-10-14 19:13:54,344][61585] Updated weights for policy 1, policy_version 38850 (0.0008) [2023-10-14 19:13:54,713][61585] Updated weights for policy 1, policy_version 38860 (0.0007) [2023-10-14 19:13:54,937][61552] Updated weights for policy 0, policy_version 39012 (0.0007) [2023-10-14 19:13:55,074][61585] Updated weights for policy 1, policy_version 38870 (0.0008) [2023-10-14 19:13:55,312][61552] Updated weights for policy 0, policy_version 39022 (0.0008) [2023-10-14 19:13:55,445][61585] Updated weights for policy 1, policy_version 38880 (0.0007) [2023-10-14 19:13:55,680][61552] Updated weights for policy 0, policy_version 39032 (0.0009) [2023-10-14 19:13:58,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 79790080. Throughput: 0: 1660.3, 1: 1674.3. Samples: 19959312. Policy #0 lag: (min: 16.0, avg: 39.4, max: 48.0) [2023-10-14 19:13:58,344][60425] Avg episode reward: [(0, '64.690'), (1, '56.500')] [2023-10-14 19:13:59,617][61585] Updated weights for policy 1, policy_version 38890 (0.0008) [2023-10-14 19:13:59,716][61552] Updated weights for policy 0, policy_version 39042 (0.0009) [2023-10-14 19:13:59,979][61585] Updated weights for policy 1, policy_version 38900 (0.0008) [2023-10-14 19:14:00,084][61552] Updated weights for policy 0, policy_version 39052 (0.0008) [2023-10-14 19:14:00,339][61585] Updated weights for policy 1, policy_version 38910 (0.0009) [2023-10-14 19:14:00,454][61552] Updated weights for policy 0, policy_version 39062 (0.0007) [2023-10-14 19:14:00,819][61552] Updated weights for policy 0, policy_version 39072 (0.0008) [2023-10-14 19:14:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 79855616. Throughput: 0: 1651.8, 1: 1663.6. Samples: 19968700. Policy #0 lag: (min: 16.0, avg: 39.4, max: 48.0) [2023-10-14 19:14:03,344][60425] Avg episode reward: [(0, '70.490'), (1, '57.970')] [2023-10-14 19:14:04,411][61585] Updated weights for policy 1, policy_version 38920 (0.0011) [2023-10-14 19:14:04,773][61585] Updated weights for policy 1, policy_version 38930 (0.0009) [2023-10-14 19:14:04,957][61552] Updated weights for policy 0, policy_version 39082 (0.0007) [2023-10-14 19:14:05,144][61585] Updated weights for policy 1, policy_version 38940 (0.0009) [2023-10-14 19:14:05,316][61552] Updated weights for policy 0, policy_version 39092 (0.0007) [2023-10-14 19:14:05,695][61552] Updated weights for policy 0, policy_version 39102 (0.0009) [2023-10-14 19:14:08,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 79921152. Throughput: 0: 1664.7, 1: 1677.1. Samples: 19989008. Policy #0 lag: (min: 16.0, avg: 39.4, max: 48.0) [2023-10-14 19:14:08,344][60425] Avg episode reward: [(0, '64.690'), (1, '57.150')] [2023-10-14 19:14:09,471][61585] Updated weights for policy 1, policy_version 38950 (0.0010) [2023-10-14 19:14:09,799][61552] Updated weights for policy 0, policy_version 39112 (0.0008) [2023-10-14 19:14:09,841][61585] Updated weights for policy 1, policy_version 38960 (0.0010) [2023-10-14 19:14:10,169][61552] Updated weights for policy 0, policy_version 39122 (0.0008) [2023-10-14 19:14:10,208][61585] Updated weights for policy 1, policy_version 38970 (0.0007) [2023-10-14 19:14:10,536][61552] Updated weights for policy 0, policy_version 39132 (0.0009) [2023-10-14 19:14:13,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 79986688. Throughput: 0: 1669.2, 1: 1670.5. Samples: 20009570. Policy #0 lag: (min: 16.0, avg: 39.4, max: 48.0) [2023-10-14 19:14:13,344][60425] Avg episode reward: [(0, '63.460'), (1, '57.060')] [2023-10-14 19:14:14,295][61585] Updated weights for policy 1, policy_version 38980 (0.0007) [2023-10-14 19:14:14,550][61552] Updated weights for policy 0, policy_version 39142 (0.0008) [2023-10-14 19:14:14,658][61585] Updated weights for policy 1, policy_version 38990 (0.0010) [2023-10-14 19:14:14,914][61552] Updated weights for policy 0, policy_version 39152 (0.0007) [2023-10-14 19:14:15,022][61585] Updated weights for policy 1, policy_version 39000 (0.0009) [2023-10-14 19:14:15,291][61552] Updated weights for policy 0, policy_version 39162 (0.0007) [2023-10-14 19:14:18,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 80052224. Throughput: 0: 1657.9, 1: 1660.1. Samples: 20018610. Policy #0 lag: (min: 16.0, avg: 32.3, max: 48.0) [2023-10-14 19:14:18,344][60425] Avg episode reward: [(0, '65.680'), (1, '58.460')] [2023-10-14 19:14:19,194][61585] Updated weights for policy 1, policy_version 39010 (0.0008) [2023-10-14 19:14:19,484][61552] Updated weights for policy 0, policy_version 39172 (0.0008) [2023-10-14 19:14:19,557][61585] Updated weights for policy 1, policy_version 39020 (0.0008) [2023-10-14 19:14:19,850][61552] Updated weights for policy 0, policy_version 39182 (0.0008) [2023-10-14 19:14:19,926][61585] Updated weights for policy 1, policy_version 39030 (0.0010) [2023-10-14 19:14:20,215][61552] Updated weights for policy 0, policy_version 39192 (0.0008) [2023-10-14 19:14:20,290][61585] Updated weights for policy 1, policy_version 39040 (0.0010) [2023-10-14 19:14:23,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 80117760. Throughput: 0: 1671.1, 1: 1665.2. Samples: 20038966. Policy #0 lag: (min: 16.0, avg: 32.3, max: 48.0) [2023-10-14 19:14:23,344][60425] Avg episode reward: [(0, '66.080'), (1, '60.910')] [2023-10-14 19:14:24,159][61552] Updated weights for policy 0, policy_version 39202 (0.0009) [2023-10-14 19:14:24,447][61585] Updated weights for policy 1, policy_version 39050 (0.0007) [2023-10-14 19:14:24,529][61552] Updated weights for policy 0, policy_version 39212 (0.0008) [2023-10-14 19:14:24,819][61585] Updated weights for policy 1, policy_version 39060 (0.0009) [2023-10-14 19:14:24,898][61552] Updated weights for policy 0, policy_version 39222 (0.0009) [2023-10-14 19:14:25,194][61585] Updated weights for policy 1, policy_version 39070 (0.0007) [2023-10-14 19:14:25,258][61552] Updated weights for policy 0, policy_version 39232 (0.0009) [2023-10-14 19:14:28,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 80183296. Throughput: 0: 1672.9, 1: 1659.9. Samples: 20059364. Policy #0 lag: (min: 16.0, avg: 32.3, max: 48.0) [2023-10-14 19:14:28,344][60425] Avg episode reward: [(0, '64.880'), (1, '54.230')] [2023-10-14 19:14:29,132][61585] Updated weights for policy 1, policy_version 39080 (0.0008) [2023-10-14 19:14:29,354][61552] Updated weights for policy 0, policy_version 39242 (0.0008) [2023-10-14 19:14:29,492][61585] Updated weights for policy 1, policy_version 39090 (0.0010) [2023-10-14 19:14:29,728][61552] Updated weights for policy 0, policy_version 39252 (0.0008) [2023-10-14 19:14:29,863][61585] Updated weights for policy 1, policy_version 39100 (0.0009) [2023-10-14 19:14:30,098][61552] Updated weights for policy 0, policy_version 39262 (0.0008) [2023-10-14 19:14:33,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 80248832. Throughput: 0: 1666.1, 1: 1659.8. Samples: 20068468. Policy #0 lag: (min: 16.0, avg: 32.3, max: 48.0) [2023-10-14 19:14:33,344][60425] Avg episode reward: [(0, '64.980'), (1, '57.510')] [2023-10-14 19:14:33,933][61585] Updated weights for policy 1, policy_version 39110 (0.0009) [2023-10-14 19:14:34,066][61552] Updated weights for policy 0, policy_version 39272 (0.0008) [2023-10-14 19:14:34,292][61585] Updated weights for policy 1, policy_version 39120 (0.0007) [2023-10-14 19:14:34,431][61552] Updated weights for policy 0, policy_version 39282 (0.0008) [2023-10-14 19:14:34,662][61585] Updated weights for policy 1, policy_version 39130 (0.0007) [2023-10-14 19:14:34,806][61552] Updated weights for policy 0, policy_version 39292 (0.0009) [2023-10-14 19:14:38,344][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 80314368. Throughput: 0: 1681.2, 1: 1658.8. Samples: 20089200. Policy #0 lag: (min: 16.0, avg: 32.3, max: 48.0) [2023-10-14 19:14:38,345][60425] Avg episode reward: [(0, '65.960'), (1, '61.160')] [2023-10-14 19:14:38,915][61552] Updated weights for policy 0, policy_version 39302 (0.0007) [2023-10-14 19:14:38,923][61585] Updated weights for policy 1, policy_version 39140 (0.0009) [2023-10-14 19:14:39,281][61585] Updated weights for policy 1, policy_version 39150 (0.0009) [2023-10-14 19:14:39,285][61552] Updated weights for policy 0, policy_version 39312 (0.0008) [2023-10-14 19:14:39,641][61585] Updated weights for policy 1, policy_version 39160 (0.0010) [2023-10-14 19:14:39,646][61552] Updated weights for policy 0, policy_version 39322 (0.0008) [2023-10-14 19:14:43,343][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 80379904. Throughput: 0: 1679.9, 1: 1661.9. Samples: 20109692. Policy #0 lag: (min: 16.0, avg: 32.3, max: 48.0) [2023-10-14 19:14:43,344][60425] Avg episode reward: [(0, '63.150'), (1, '62.440')] [2023-10-14 19:14:43,788][61585] Updated weights for policy 1, policy_version 39170 (0.0008) [2023-10-14 19:14:43,823][61552] Updated weights for policy 0, policy_version 39332 (0.0009) [2023-10-14 19:14:44,148][61585] Updated weights for policy 1, policy_version 39180 (0.0010) [2023-10-14 19:14:44,186][61552] Updated weights for policy 0, policy_version 39342 (0.0010) [2023-10-14 19:14:44,518][61585] Updated weights for policy 1, policy_version 39190 (0.0007) [2023-10-14 19:14:44,545][61552] Updated weights for policy 0, policy_version 39352 (0.0008) [2023-10-14 19:14:44,880][61585] Updated weights for policy 1, policy_version 39200 (0.0010) [2023-10-14 19:14:48,344][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 80445440. Throughput: 0: 1679.6, 1: 1657.3. Samples: 20118862. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:14:48,344][60425] Avg episode reward: [(0, '66.040'), (1, '58.750')] [2023-10-14 19:14:48,720][61552] Updated weights for policy 0, policy_version 39362 (0.0008) [2023-10-14 19:14:48,964][61585] Updated weights for policy 1, policy_version 39210 (0.0007) [2023-10-14 19:14:49,113][61552] Updated weights for policy 0, policy_version 39372 (0.0007) [2023-10-14 19:14:49,332][61585] Updated weights for policy 1, policy_version 39220 (0.0009) [2023-10-14 19:14:49,477][61552] Updated weights for policy 0, policy_version 39382 (0.0010) [2023-10-14 19:14:49,701][61585] Updated weights for policy 1, policy_version 39230 (0.0008) [2023-10-14 19:14:49,843][61552] Updated weights for policy 0, policy_version 39392 (0.0008) [2023-10-14 19:14:53,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 80510976. Throughput: 0: 1678.9, 1: 1655.9. Samples: 20139076. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:14:53,344][60425] Avg episode reward: [(0, '67.920'), (1, '58.580')] [2023-10-14 19:14:53,890][61585] Updated weights for policy 1, policy_version 39240 (0.0009) [2023-10-14 19:14:53,911][61552] Updated weights for policy 0, policy_version 39402 (0.0008) [2023-10-14 19:14:54,247][61585] Updated weights for policy 1, policy_version 39250 (0.0009) [2023-10-14 19:14:54,282][61552] Updated weights for policy 0, policy_version 39412 (0.0007) [2023-10-14 19:14:54,614][61585] Updated weights for policy 1, policy_version 39260 (0.0007) [2023-10-14 19:14:54,645][61552] Updated weights for policy 0, policy_version 39422 (0.0009) [2023-10-14 19:14:58,343][60425] Fps is (10 sec: 13107.8, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 80576512. Throughput: 0: 1677.6, 1: 1659.4. Samples: 20159736. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:14:58,344][60425] Avg episode reward: [(0, '70.100'), (1, '59.230')] [2023-10-14 19:14:58,614][61585] Updated weights for policy 1, policy_version 39270 (0.0008) [2023-10-14 19:14:58,676][61552] Updated weights for policy 0, policy_version 39432 (0.0007) [2023-10-14 19:14:58,985][61585] Updated weights for policy 1, policy_version 39280 (0.0008) [2023-10-14 19:14:59,045][61552] Updated weights for policy 0, policy_version 39442 (0.0009) [2023-10-14 19:14:59,345][61585] Updated weights for policy 1, policy_version 39290 (0.0008) [2023-10-14 19:14:59,413][61552] Updated weights for policy 0, policy_version 39452 (0.0009) [2023-10-14 19:15:03,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 80642048. Throughput: 0: 1674.5, 1: 1662.1. Samples: 20168756. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:15:03,344][60425] Avg episode reward: [(0, '65.240'), (1, '60.400')] [2023-10-14 19:15:03,496][61585] Updated weights for policy 1, policy_version 39300 (0.0007) [2023-10-14 19:15:03,557][61552] Updated weights for policy 0, policy_version 39462 (0.0009) [2023-10-14 19:15:03,865][61585] Updated weights for policy 1, policy_version 39310 (0.0008) [2023-10-14 19:15:03,926][61552] Updated weights for policy 0, policy_version 39472 (0.0008) [2023-10-14 19:15:04,234][61585] Updated weights for policy 1, policy_version 39320 (0.0009) [2023-10-14 19:15:04,289][61552] Updated weights for policy 0, policy_version 39482 (0.0008) [2023-10-14 19:15:08,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 80707584. Throughput: 0: 1673.3, 1: 1660.8. Samples: 20188998. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:15:08,344][60425] Avg episode reward: [(0, '63.920'), (1, '56.660')] [2023-10-14 19:15:08,500][61552] Updated weights for policy 0, policy_version 39492 (0.0009) [2023-10-14 19:15:08,507][61585] Updated weights for policy 1, policy_version 39330 (0.0009) [2023-10-14 19:15:08,866][61552] Updated weights for policy 0, policy_version 39502 (0.0008) [2023-10-14 19:15:08,927][61585] Updated weights for policy 1, policy_version 39340 (0.0009) [2023-10-14 19:15:09,236][61552] Updated weights for policy 0, policy_version 39512 (0.0008) [2023-10-14 19:15:09,279][61585] Updated weights for policy 1, policy_version 39350 (0.0010) [2023-10-14 19:15:09,646][61585] Updated weights for policy 1, policy_version 39360 (0.0008) [2023-10-14 19:15:13,225][61552] Updated weights for policy 0, policy_version 39522 (0.0008) [2023-10-14 19:15:13,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 80773120. Throughput: 0: 1677.0, 1: 1659.3. Samples: 20209494. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:15:13,344][60425] Avg episode reward: [(0, '64.370'), (1, '60.230')] [2023-10-14 19:15:13,595][61552] Updated weights for policy 0, policy_version 39532 (0.0007) [2023-10-14 19:15:13,823][61585] Updated weights for policy 1, policy_version 39370 (0.0007) [2023-10-14 19:15:13,968][61552] Updated weights for policy 0, policy_version 39542 (0.0007) [2023-10-14 19:15:14,183][61585] Updated weights for policy 1, policy_version 39380 (0.0008) [2023-10-14 19:15:14,328][61552] Updated weights for policy 0, policy_version 39552 (0.0009) [2023-10-14 19:15:14,549][61585] Updated weights for policy 1, policy_version 39390 (0.0007) [2023-10-14 19:15:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 80838656. Throughput: 0: 1675.1, 1: 1657.6. Samples: 20218438. Policy #0 lag: (min: 14.0, avg: 19.8, max: 46.0) [2023-10-14 19:15:18,344][60425] Avg episode reward: [(0, '67.220'), (1, '58.670')] [2023-10-14 19:15:18,348][61552] Updated weights for policy 0, policy_version 39562 (0.0010) [2023-10-14 19:15:18,711][61552] Updated weights for policy 0, policy_version 39572 (0.0007) [2023-10-14 19:15:18,736][61585] Updated weights for policy 1, policy_version 39400 (0.0008) [2023-10-14 19:15:19,075][61552] Updated weights for policy 0, policy_version 39582 (0.0007) [2023-10-14 19:15:19,113][61585] Updated weights for policy 1, policy_version 39410 (0.0008) [2023-10-14 19:15:19,471][61585] Updated weights for policy 1, policy_version 39420 (0.0010) [2023-10-14 19:15:23,272][61552] Updated weights for policy 0, policy_version 39592 (0.0008) [2023-10-14 19:15:23,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 80904192. Throughput: 0: 1669.4, 1: 1654.4. Samples: 20238770. Policy #0 lag: (min: 14.0, avg: 19.8, max: 46.0) [2023-10-14 19:15:23,344][60425] Avg episode reward: [(0, '64.080'), (1, '57.630')] [2023-10-14 19:15:23,644][61552] Updated weights for policy 0, policy_version 39602 (0.0009) [2023-10-14 19:15:23,703][61585] Updated weights for policy 1, policy_version 39430 (0.0009) [2023-10-14 19:15:24,013][61552] Updated weights for policy 0, policy_version 39612 (0.0010) [2023-10-14 19:15:24,059][61585] Updated weights for policy 1, policy_version 39440 (0.0008) [2023-10-14 19:15:24,428][61585] Updated weights for policy 1, policy_version 39450 (0.0007) [2023-10-14 19:15:28,026][61552] Updated weights for policy 0, policy_version 39622 (0.0007) [2023-10-14 19:15:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 80969728. Throughput: 0: 1669.7, 1: 1651.4. Samples: 20259140. Policy #0 lag: (min: 14.0, avg: 19.8, max: 46.0) [2023-10-14 19:15:28,344][60425] Avg episode reward: [(0, '66.510'), (1, '56.890')] [2023-10-14 19:15:28,407][61552] Updated weights for policy 0, policy_version 39632 (0.0007) [2023-10-14 19:15:28,582][61585] Updated weights for policy 1, policy_version 39460 (0.0008) [2023-10-14 19:15:28,774][61552] Updated weights for policy 0, policy_version 39642 (0.0007) [2023-10-14 19:15:28,951][61585] Updated weights for policy 1, policy_version 39470 (0.0007) [2023-10-14 19:15:29,319][61585] Updated weights for policy 1, policy_version 39480 (0.0008) [2023-10-14 19:15:32,945][61552] Updated weights for policy 0, policy_version 39652 (0.0009) [2023-10-14 19:15:33,319][61552] Updated weights for policy 0, policy_version 39662 (0.0011) [2023-10-14 19:15:33,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 81035264. Throughput: 0: 1665.1, 1: 1653.7. Samples: 20268206. Policy #0 lag: (min: 14.0, avg: 19.8, max: 46.0) [2023-10-14 19:15:33,344][60425] Avg episode reward: [(0, '67.760'), (1, '60.810')] [2023-10-14 19:15:33,387][61585] Updated weights for policy 1, policy_version 39490 (0.0010) [2023-10-14 19:15:33,682][61552] Updated weights for policy 0, policy_version 39672 (0.0009) [2023-10-14 19:15:33,753][61585] Updated weights for policy 1, policy_version 39500 (0.0008) [2023-10-14 19:15:34,113][61585] Updated weights for policy 1, policy_version 39510 (0.0009) [2023-10-14 19:15:34,468][61585] Updated weights for policy 1, policy_version 39520 (0.0010) [2023-10-14 19:15:37,797][61552] Updated weights for policy 0, policy_version 39682 (0.0007) [2023-10-14 19:15:38,207][61552] Updated weights for policy 0, policy_version 39692 (0.0007) [2023-10-14 19:15:38,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 81100800. Throughput: 0: 1674.0, 1: 1657.9. Samples: 20289010. Policy #0 lag: (min: 14.0, avg: 19.8, max: 46.0) [2023-10-14 19:15:38,344][60425] Avg episode reward: [(0, '65.390'), (1, '60.430')] [2023-10-14 19:15:38,575][61552] Updated weights for policy 0, policy_version 39702 (0.0008) [2023-10-14 19:15:38,620][61585] Updated weights for policy 1, policy_version 39530 (0.0008) [2023-10-14 19:15:38,943][61552] Updated weights for policy 0, policy_version 39712 (0.0007) [2023-10-14 19:15:38,995][61585] Updated weights for policy 1, policy_version 39540 (0.0007) [2023-10-14 19:15:39,351][61585] Updated weights for policy 1, policy_version 39550 (0.0008) [2023-10-14 19:15:43,003][61552] Updated weights for policy 0, policy_version 39722 (0.0011) [2023-10-14 19:15:43,343][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 81166336. Throughput: 0: 1667.6, 1: 1656.2. Samples: 20309308. Policy #0 lag: (min: 14.0, avg: 19.8, max: 46.0) [2023-10-14 19:15:43,344][60425] Avg episode reward: [(0, '67.970'), (1, '60.930')] [2023-10-14 19:15:43,369][61552] Updated weights for policy 0, policy_version 39732 (0.0008) [2023-10-14 19:15:43,517][61585] Updated weights for policy 1, policy_version 39560 (0.0008) [2023-10-14 19:15:43,738][61552] Updated weights for policy 0, policy_version 39742 (0.0007) [2023-10-14 19:15:43,806][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000039744_40697856.pth... [2023-10-14 19:15:43,844][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000038176_39092224.pth [2023-10-14 19:15:43,883][61585] Updated weights for policy 1, policy_version 39570 (0.0008) [2023-10-14 19:15:44,249][61585] Updated weights for policy 1, policy_version 39580 (0.0011) [2023-10-14 19:15:44,389][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000039584_40534016.pth... [2023-10-14 19:15:44,418][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000038016_38928384.pth [2023-10-14 19:15:47,645][61552] Updated weights for policy 0, policy_version 39752 (0.0008) [2023-10-14 19:15:48,009][61552] Updated weights for policy 0, policy_version 39762 (0.0009) [2023-10-14 19:15:48,218][61585] Updated weights for policy 1, policy_version 39590 (0.0008) [2023-10-14 19:15:48,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13218.3). Total num frames: 81231872. Throughput: 0: 1673.2, 1: 1653.1. Samples: 20318442. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-14 19:15:48,344][60425] Avg episode reward: [(0, '67.930'), (1, '58.870')] [2023-10-14 19:15:48,374][61552] Updated weights for policy 0, policy_version 39772 (0.0007) [2023-10-14 19:15:48,588][61585] Updated weights for policy 1, policy_version 39600 (0.0009) [2023-10-14 19:15:48,953][61585] Updated weights for policy 1, policy_version 39610 (0.0008) [2023-10-14 19:15:52,638][61552] Updated weights for policy 0, policy_version 39782 (0.0009) [2023-10-14 19:15:53,002][61552] Updated weights for policy 0, policy_version 39792 (0.0010) [2023-10-14 19:15:53,256][61585] Updated weights for policy 1, policy_version 39620 (0.0007) [2023-10-14 19:15:53,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 81297408. Throughput: 0: 1674.2, 1: 1656.4. Samples: 20338874. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-14 19:15:53,344][60425] Avg episode reward: [(0, '70.810'), (1, '61.500')] [2023-10-14 19:15:53,381][61552] Updated weights for policy 0, policy_version 39802 (0.0008) [2023-10-14 19:15:53,600][61172] Saving new best policy, reward=70.810! [2023-10-14 19:15:53,620][61585] Updated weights for policy 1, policy_version 39630 (0.0007) [2023-10-14 19:15:53,981][61585] Updated weights for policy 1, policy_version 39640 (0.0011) [2023-10-14 19:15:57,390][61552] Updated weights for policy 0, policy_version 39812 (0.0007) [2023-10-14 19:15:57,749][61552] Updated weights for policy 0, policy_version 39822 (0.0009) [2023-10-14 19:15:58,106][61585] Updated weights for policy 1, policy_version 39650 (0.0008) [2023-10-14 19:15:58,119][61552] Updated weights for policy 0, policy_version 39832 (0.0009) [2023-10-14 19:15:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 81362944. Throughput: 0: 1663.9, 1: 1658.0. Samples: 20358982. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-14 19:15:58,344][60425] Avg episode reward: [(0, '69.150'), (1, '60.350')] [2023-10-14 19:15:58,483][61585] Updated weights for policy 1, policy_version 39660 (0.0008) [2023-10-14 19:15:58,841][61585] Updated weights for policy 1, policy_version 39670 (0.0007) [2023-10-14 19:15:59,210][61585] Updated weights for policy 1, policy_version 39680 (0.0008) [2023-10-14 19:16:02,224][61552] Updated weights for policy 0, policy_version 39842 (0.0008) [2023-10-14 19:16:02,590][61552] Updated weights for policy 0, policy_version 39852 (0.0011) [2023-10-14 19:16:02,964][61552] Updated weights for policy 0, policy_version 39862 (0.0008) [2023-10-14 19:16:03,259][61585] Updated weights for policy 1, policy_version 39690 (0.0007) [2023-10-14 19:16:03,332][61552] Updated weights for policy 0, policy_version 39872 (0.0010) [2023-10-14 19:16:03,343][60425] Fps is (10 sec: 16384.2, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 81461248. Throughput: 0: 1678.2, 1: 1659.9. Samples: 20368650. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-14 19:16:03,344][60425] Avg episode reward: [(0, '68.990'), (1, '58.060')] [2023-10-14 19:16:03,638][61585] Updated weights for policy 1, policy_version 39700 (0.0008) [2023-10-14 19:16:04,000][61585] Updated weights for policy 1, policy_version 39710 (0.0008) [2023-10-14 19:16:07,466][61552] Updated weights for policy 0, policy_version 39882 (0.0008) [2023-10-14 19:16:07,828][61552] Updated weights for policy 0, policy_version 39892 (0.0009) [2023-10-14 19:16:08,065][61585] Updated weights for policy 1, policy_version 39720 (0.0009) [2023-10-14 19:16:08,201][61552] Updated weights for policy 0, policy_version 39902 (0.0008) [2023-10-14 19:16:08,343][60425] Fps is (10 sec: 16384.2, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 81526784. Throughput: 0: 1683.1, 1: 1663.4. Samples: 20389364. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-14 19:16:08,344][60425] Avg episode reward: [(0, '67.870'), (1, '57.680')] [2023-10-14 19:16:08,429][61585] Updated weights for policy 1, policy_version 39730 (0.0009) [2023-10-14 19:16:08,791][61585] Updated weights for policy 1, policy_version 39740 (0.0009) [2023-10-14 19:16:12,395][61552] Updated weights for policy 0, policy_version 39912 (0.0008) [2023-10-14 19:16:12,772][61552] Updated weights for policy 0, policy_version 39922 (0.0008) [2023-10-14 19:16:12,902][61585] Updated weights for policy 1, policy_version 39750 (0.0009) [2023-10-14 19:16:13,129][61552] Updated weights for policy 0, policy_version 39932 (0.0009) [2023-10-14 19:16:13,266][61585] Updated weights for policy 1, policy_version 39760 (0.0009) [2023-10-14 19:16:13,343][60425] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 81592320. Throughput: 0: 1667.7, 1: 1668.2. Samples: 20409254. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-14 19:16:13,345][60425] Avg episode reward: [(0, '71.420'), (1, '59.420')] [2023-10-14 19:16:13,354][61172] Saving new best policy, reward=71.420! [2023-10-14 19:16:13,633][61585] Updated weights for policy 1, policy_version 39770 (0.0012) [2023-10-14 19:16:17,100][61552] Updated weights for policy 0, policy_version 39942 (0.0010) [2023-10-14 19:16:17,472][61552] Updated weights for policy 0, policy_version 39952 (0.0010) [2023-10-14 19:16:17,674][61585] Updated weights for policy 1, policy_version 39780 (0.0009) [2023-10-14 19:16:17,836][61552] Updated weights for policy 0, policy_version 39962 (0.0007) [2023-10-14 19:16:18,047][61585] Updated weights for policy 1, policy_version 39790 (0.0007) [2023-10-14 19:16:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 81657856. Throughput: 0: 1681.9, 1: 1668.1. Samples: 20418954. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-14 19:16:18,344][60425] Avg episode reward: [(0, '68.350'), (1, '59.530')] [2023-10-14 19:16:18,407][61585] Updated weights for policy 1, policy_version 39800 (0.0007) [2023-10-14 19:16:21,999][61552] Updated weights for policy 0, policy_version 39972 (0.0009) [2023-10-14 19:16:22,377][61552] Updated weights for policy 0, policy_version 39982 (0.0010) [2023-10-14 19:16:22,591][61585] Updated weights for policy 1, policy_version 39810 (0.0009) [2023-10-14 19:16:22,753][61552] Updated weights for policy 0, policy_version 39992 (0.0009) [2023-10-14 19:16:22,955][61585] Updated weights for policy 1, policy_version 39820 (0.0008) [2023-10-14 19:16:23,317][61585] Updated weights for policy 1, policy_version 39830 (0.0007) [2023-10-14 19:16:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 81723392. Throughput: 0: 1677.3, 1: 1666.3. Samples: 20439474. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-14 19:16:23,344][60425] Avg episode reward: [(0, '66.350'), (1, '63.340')] [2023-10-14 19:16:23,699][61585] Updated weights for policy 1, policy_version 39840 (0.0010) [2023-10-14 19:16:26,917][61552] Updated weights for policy 0, policy_version 40002 (0.0009) [2023-10-14 19:16:27,337][61552] Updated weights for policy 0, policy_version 40012 (0.0009) [2023-10-14 19:16:27,702][61552] Updated weights for policy 0, policy_version 40022 (0.0009) [2023-10-14 19:16:27,805][61585] Updated weights for policy 1, policy_version 39850 (0.0010) [2023-10-14 19:16:28,078][61552] Updated weights for policy 0, policy_version 40032 (0.0008) [2023-10-14 19:16:28,176][61585] Updated weights for policy 1, policy_version 39860 (0.0008) [2023-10-14 19:16:28,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 81788928. Throughput: 0: 1657.3, 1: 1662.1. Samples: 20458684. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-14 19:16:28,344][60425] Avg episode reward: [(0, '65.250'), (1, '63.240')] [2023-10-14 19:16:28,533][61585] Updated weights for policy 1, policy_version 39870 (0.0008) [2023-10-14 19:16:32,153][61552] Updated weights for policy 0, policy_version 40042 (0.0010) [2023-10-14 19:16:32,520][61552] Updated weights for policy 0, policy_version 40052 (0.0008) [2023-10-14 19:16:32,575][61585] Updated weights for policy 1, policy_version 39880 (0.0007) [2023-10-14 19:16:32,887][61552] Updated weights for policy 0, policy_version 40062 (0.0008) [2023-10-14 19:16:32,945][61585] Updated weights for policy 1, policy_version 39890 (0.0007) [2023-10-14 19:16:33,304][61585] Updated weights for policy 1, policy_version 39900 (0.0007) [2023-10-14 19:16:33,344][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 81854464. Throughput: 0: 1672.3, 1: 1669.7. Samples: 20468832. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-14 19:16:33,345][60425] Avg episode reward: [(0, '68.950'), (1, '59.470')] [2023-10-14 19:16:36,880][61552] Updated weights for policy 0, policy_version 40072 (0.0008) [2023-10-14 19:16:37,250][61552] Updated weights for policy 0, policy_version 40082 (0.0007) [2023-10-14 19:16:37,454][61585] Updated weights for policy 1, policy_version 39910 (0.0008) [2023-10-14 19:16:37,623][61552] Updated weights for policy 0, policy_version 40092 (0.0009) [2023-10-14 19:16:37,822][61585] Updated weights for policy 1, policy_version 39920 (0.0008) [2023-10-14 19:16:38,197][61585] Updated weights for policy 1, policy_version 39930 (0.0009) [2023-10-14 19:16:38,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 81920000. Throughput: 0: 1672.0, 1: 1670.8. Samples: 20489296. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-14 19:16:38,344][60425] Avg episode reward: [(0, '65.910'), (1, '57.420')] [2023-10-14 19:16:41,667][61552] Updated weights for policy 0, policy_version 40102 (0.0008) [2023-10-14 19:16:42,035][61552] Updated weights for policy 0, policy_version 40112 (0.0008) [2023-10-14 19:16:42,351][61585] Updated weights for policy 1, policy_version 39940 (0.0008) [2023-10-14 19:16:42,405][61552] Updated weights for policy 0, policy_version 40122 (0.0007) [2023-10-14 19:16:42,745][61585] Updated weights for policy 1, policy_version 39950 (0.0009) [2023-10-14 19:16:43,108][61585] Updated weights for policy 1, policy_version 39960 (0.0010) [2023-10-14 19:16:43,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 81985536. Throughput: 0: 1656.5, 1: 1660.7. Samples: 20508256. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-14 19:16:43,344][60425] Avg episode reward: [(0, '67.200'), (1, '60.080')] [2023-10-14 19:16:46,457][61552] Updated weights for policy 0, policy_version 40132 (0.0009) [2023-10-14 19:16:46,830][61552] Updated weights for policy 0, policy_version 40142 (0.0008) [2023-10-14 19:16:47,201][61552] Updated weights for policy 0, policy_version 40152 (0.0009) [2023-10-14 19:16:47,236][61585] Updated weights for policy 1, policy_version 39970 (0.0010) [2023-10-14 19:16:47,595][61585] Updated weights for policy 1, policy_version 39980 (0.0009) [2023-10-14 19:16:47,962][61585] Updated weights for policy 1, policy_version 39990 (0.0010) [2023-10-14 19:16:48,331][61585] Updated weights for policy 1, policy_version 40000 (0.0010) [2023-10-14 19:16:48,343][60425] Fps is (10 sec: 16384.2, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 82083840. Throughput: 0: 1673.6, 1: 1668.2. Samples: 20519034. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-14 19:16:48,344][60425] Avg episode reward: [(0, '68.290'), (1, '58.350')] [2023-10-14 19:16:51,199][61552] Updated weights for policy 0, policy_version 40162 (0.0008) [2023-10-14 19:16:51,566][61552] Updated weights for policy 0, policy_version 40172 (0.0008) [2023-10-14 19:16:51,932][61552] Updated weights for policy 0, policy_version 40182 (0.0007) [2023-10-14 19:16:52,298][61552] Updated weights for policy 0, policy_version 40192 (0.0008) [2023-10-14 19:16:52,547][61585] Updated weights for policy 1, policy_version 40010 (0.0009) [2023-10-14 19:16:52,911][61585] Updated weights for policy 1, policy_version 40020 (0.0009) [2023-10-14 19:16:53,286][61585] Updated weights for policy 1, policy_version 40030 (0.0008) [2023-10-14 19:16:53,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 82116608. Throughput: 0: 1655.8, 1: 1666.0. Samples: 20538848. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-14 19:16:53,344][60425] Avg episode reward: [(0, '66.080'), (1, '56.210')] [2023-10-14 19:16:56,414][61552] Updated weights for policy 0, policy_version 40202 (0.0008) [2023-10-14 19:16:56,780][61552] Updated weights for policy 0, policy_version 40212 (0.0009) [2023-10-14 19:16:57,139][61552] Updated weights for policy 0, policy_version 40222 (0.0008) [2023-10-14 19:16:57,415][61585] Updated weights for policy 1, policy_version 40040 (0.0009) [2023-10-14 19:16:57,776][61585] Updated weights for policy 1, policy_version 40050 (0.0009) [2023-10-14 19:16:58,136][61585] Updated weights for policy 1, policy_version 40060 (0.0010) [2023-10-14 19:16:58,343][60425] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13329.3). Total num frames: 82214912. Throughput: 0: 1658.4, 1: 1651.6. Samples: 20558202. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-14 19:16:58,344][60425] Avg episode reward: [(0, '65.630'), (1, '54.880')] [2023-10-14 19:17:01,272][61552] Updated weights for policy 0, policy_version 40232 (0.0009) [2023-10-14 19:17:01,633][61552] Updated weights for policy 0, policy_version 40242 (0.0008) [2023-10-14 19:17:02,002][61552] Updated weights for policy 0, policy_version 40252 (0.0008) [2023-10-14 19:17:02,211][61585] Updated weights for policy 1, policy_version 40070 (0.0008) [2023-10-14 19:17:02,574][61585] Updated weights for policy 1, policy_version 40080 (0.0008) [2023-10-14 19:17:02,942][61585] Updated weights for policy 1, policy_version 40090 (0.0007) [2023-10-14 19:17:03,343][60425] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 82280448. Throughput: 0: 1674.1, 1: 1664.6. Samples: 20569196. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-14 19:17:03,344][60425] Avg episode reward: [(0, '69.800'), (1, '57.740')] [2023-10-14 19:17:06,097][61552] Updated weights for policy 0, policy_version 40262 (0.0010) [2023-10-14 19:17:06,474][61552] Updated weights for policy 0, policy_version 40272 (0.0008) [2023-10-14 19:17:06,846][61552] Updated weights for policy 0, policy_version 40282 (0.0009) [2023-10-14 19:17:07,129][61585] Updated weights for policy 1, policy_version 40100 (0.0008) [2023-10-14 19:17:07,485][61585] Updated weights for policy 1, policy_version 40110 (0.0009) [2023-10-14 19:17:07,859][61585] Updated weights for policy 1, policy_version 40120 (0.0008) [2023-10-14 19:17:08,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 82345984. Throughput: 0: 1654.7, 1: 1664.1. Samples: 20588822. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-14 19:17:08,344][60425] Avg episode reward: [(0, '64.790'), (1, '56.030')] [2023-10-14 19:17:10,937][61552] Updated weights for policy 0, policy_version 40292 (0.0007) [2023-10-14 19:17:11,304][61552] Updated weights for policy 0, policy_version 40302 (0.0009) [2023-10-14 19:17:11,675][61552] Updated weights for policy 0, policy_version 40312 (0.0009) [2023-10-14 19:17:11,926][61585] Updated weights for policy 1, policy_version 40130 (0.0009) [2023-10-14 19:17:12,304][61585] Updated weights for policy 1, policy_version 40140 (0.0011) [2023-10-14 19:17:12,663][61585] Updated weights for policy 1, policy_version 40150 (0.0010) [2023-10-14 19:17:13,031][61585] Updated weights for policy 1, policy_version 40160 (0.0007) [2023-10-14 19:17:13,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 82411520. Throughput: 0: 1670.8, 1: 1648.8. Samples: 20608068. Policy #0 lag: (min: 0.0, avg: 27.1, max: 32.0) [2023-10-14 19:17:13,344][60425] Avg episode reward: [(0, '66.270'), (1, '57.880')] [2023-10-14 19:17:15,844][61552] Updated weights for policy 0, policy_version 40322 (0.0009) [2023-10-14 19:17:16,230][61552] Updated weights for policy 0, policy_version 40332 (0.0010) [2023-10-14 19:17:16,598][61552] Updated weights for policy 0, policy_version 40342 (0.0009) [2023-10-14 19:17:16,965][61552] Updated weights for policy 0, policy_version 40352 (0.0007) [2023-10-14 19:17:17,025][61585] Updated weights for policy 1, policy_version 40170 (0.0008) [2023-10-14 19:17:17,386][61585] Updated weights for policy 1, policy_version 40180 (0.0009) [2023-10-14 19:17:17,758][61585] Updated weights for policy 1, policy_version 40190 (0.0009) [2023-10-14 19:17:18,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 82477056. Throughput: 0: 1683.6, 1: 1662.4. Samples: 20619406. Policy #0 lag: (min: 0.0, avg: 27.1, max: 32.0) [2023-10-14 19:17:18,344][60425] Avg episode reward: [(0, '66.850'), (1, '58.790')] [2023-10-14 19:17:21,100][61552] Updated weights for policy 0, policy_version 40362 (0.0007) [2023-10-14 19:17:21,465][61552] Updated weights for policy 0, policy_version 40372 (0.0009) [2023-10-14 19:17:21,836][61552] Updated weights for policy 0, policy_version 40382 (0.0007) [2023-10-14 19:17:21,889][61585] Updated weights for policy 1, policy_version 40200 (0.0010) [2023-10-14 19:17:22,246][61585] Updated weights for policy 1, policy_version 40210 (0.0011) [2023-10-14 19:17:22,605][61585] Updated weights for policy 1, policy_version 40220 (0.0009) [2023-10-14 19:17:23,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 82542592. Throughput: 0: 1664.6, 1: 1662.9. Samples: 20639034. Policy #0 lag: (min: 0.0, avg: 27.1, max: 32.0) [2023-10-14 19:17:23,344][60425] Avg episode reward: [(0, '65.560'), (1, '55.090')] [2023-10-14 19:17:25,759][61552] Updated weights for policy 0, policy_version 40392 (0.0009) [2023-10-14 19:17:26,129][61552] Updated weights for policy 0, policy_version 40402 (0.0010) [2023-10-14 19:17:26,495][61552] Updated weights for policy 0, policy_version 40412 (0.0011) [2023-10-14 19:17:26,815][61585] Updated weights for policy 1, policy_version 40230 (0.0008) [2023-10-14 19:17:27,186][61585] Updated weights for policy 1, policy_version 40240 (0.0008) [2023-10-14 19:17:27,545][61585] Updated weights for policy 1, policy_version 40250 (0.0008) [2023-10-14 19:17:28,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 82608128. Throughput: 0: 1687.6, 1: 1648.0. Samples: 20658362. Policy #0 lag: (min: 0.0, avg: 27.1, max: 32.0) [2023-10-14 19:17:28,345][60425] Avg episode reward: [(0, '66.020'), (1, '55.860')] [2023-10-14 19:17:30,624][61552] Updated weights for policy 0, policy_version 40422 (0.0009) [2023-10-14 19:17:30,992][61552] Updated weights for policy 0, policy_version 40432 (0.0007) [2023-10-14 19:17:31,361][61552] Updated weights for policy 0, policy_version 40442 (0.0007) [2023-10-14 19:17:31,671][61585] Updated weights for policy 1, policy_version 40260 (0.0008) [2023-10-14 19:17:32,075][61585] Updated weights for policy 1, policy_version 40270 (0.0007) [2023-10-14 19:17:32,440][61585] Updated weights for policy 1, policy_version 40280 (0.0010) [2023-10-14 19:17:33,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 82673664. Throughput: 0: 1678.1, 1: 1665.6. Samples: 20669500. Policy #0 lag: (min: 0.0, avg: 27.1, max: 32.0) [2023-10-14 19:17:33,344][60425] Avg episode reward: [(0, '71.950'), (1, '57.450')] [2023-10-14 19:17:33,344][61172] Saving new best policy, reward=71.950! [2023-10-14 19:17:35,564][61552] Updated weights for policy 0, policy_version 40452 (0.0009) [2023-10-14 19:17:35,930][61552] Updated weights for policy 0, policy_version 40462 (0.0008) [2023-10-14 19:17:36,296][61552] Updated weights for policy 0, policy_version 40472 (0.0008) [2023-10-14 19:17:36,443][61585] Updated weights for policy 1, policy_version 40290 (0.0007) [2023-10-14 19:17:36,798][61585] Updated weights for policy 1, policy_version 40300 (0.0008) [2023-10-14 19:17:37,163][61585] Updated weights for policy 1, policy_version 40310 (0.0008) [2023-10-14 19:17:37,532][61585] Updated weights for policy 1, policy_version 40320 (0.0009) [2023-10-14 19:17:38,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 82739200. Throughput: 0: 1667.7, 1: 1657.2. Samples: 20688466. Policy #0 lag: (min: 0.0, avg: 27.1, max: 32.0) [2023-10-14 19:17:38,344][60425] Avg episode reward: [(0, '68.190'), (1, '54.140')] [2023-10-14 19:17:40,286][61552] Updated weights for policy 0, policy_version 40482 (0.0010) [2023-10-14 19:17:40,666][61552] Updated weights for policy 0, policy_version 40492 (0.0011) [2023-10-14 19:17:41,038][61552] Updated weights for policy 0, policy_version 40502 (0.0009) [2023-10-14 19:17:41,406][61552] Updated weights for policy 0, policy_version 40512 (0.0009) [2023-10-14 19:17:41,580][61585] Updated weights for policy 1, policy_version 40330 (0.0008) [2023-10-14 19:17:41,949][61585] Updated weights for policy 1, policy_version 40340 (0.0007) [2023-10-14 19:17:42,315][61585] Updated weights for policy 1, policy_version 40350 (0.0008) [2023-10-14 19:17:43,344][60425] Fps is (10 sec: 13106.7, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 82804736. Throughput: 0: 1682.0, 1: 1653.8. Samples: 20708310. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:17:43,345][60425] Avg episode reward: [(0, '67.190'), (1, '60.910')] [2023-10-14 19:17:43,357][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000040352_41320448.pth... [2023-10-14 19:17:43,357][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000040512_41484288.pth... [2023-10-14 19:17:43,392][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000038944_39878656.pth [2023-10-14 19:17:43,396][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000038784_39714816.pth [2023-10-14 19:17:45,535][61552] Updated weights for policy 0, policy_version 40522 (0.0010) [2023-10-14 19:17:45,905][61552] Updated weights for policy 0, policy_version 40532 (0.0010) [2023-10-14 19:17:46,277][61552] Updated weights for policy 0, policy_version 40542 (0.0009) [2023-10-14 19:17:46,530][61585] Updated weights for policy 1, policy_version 40360 (0.0007) [2023-10-14 19:17:46,899][61585] Updated weights for policy 1, policy_version 40370 (0.0010) [2023-10-14 19:17:47,271][61585] Updated weights for policy 1, policy_version 40380 (0.0009) [2023-10-14 19:17:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 82870272. Throughput: 0: 1664.5, 1: 1668.2. Samples: 20719166. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:17:48,344][60425] Avg episode reward: [(0, '68.840'), (1, '58.190')] [2023-10-14 19:17:50,415][61552] Updated weights for policy 0, policy_version 40552 (0.0009) [2023-10-14 19:17:50,788][61552] Updated weights for policy 0, policy_version 40562 (0.0011) [2023-10-14 19:17:51,155][61552] Updated weights for policy 0, policy_version 40572 (0.0010) [2023-10-14 19:17:51,179][61585] Updated weights for policy 1, policy_version 40390 (0.0009) [2023-10-14 19:17:51,540][61585] Updated weights for policy 1, policy_version 40400 (0.0008) [2023-10-14 19:17:51,912][61585] Updated weights for policy 1, policy_version 40410 (0.0010) [2023-10-14 19:17:53,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 82935808. Throughput: 0: 1662.4, 1: 1650.6. Samples: 20737908. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:17:53,345][60425] Avg episode reward: [(0, '69.780'), (1, '57.950')] [2023-10-14 19:17:55,268][61552] Updated weights for policy 0, policy_version 40582 (0.0007) [2023-10-14 19:17:55,636][61552] Updated weights for policy 0, policy_version 40592 (0.0008) [2023-10-14 19:17:56,006][61552] Updated weights for policy 0, policy_version 40602 (0.0009) [2023-10-14 19:17:56,166][61585] Updated weights for policy 1, policy_version 40420 (0.0007) [2023-10-14 19:17:56,528][61585] Updated weights for policy 1, policy_version 40430 (0.0011) [2023-10-14 19:17:56,899][61585] Updated weights for policy 1, policy_version 40440 (0.0011) [2023-10-14 19:17:58,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 83001344. Throughput: 0: 1674.0, 1: 1657.7. Samples: 20757998. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:17:58,344][60425] Avg episode reward: [(0, '68.430'), (1, '61.050')] [2023-10-14 19:18:00,085][61552] Updated weights for policy 0, policy_version 40612 (0.0007) [2023-10-14 19:18:00,458][61552] Updated weights for policy 0, policy_version 40622 (0.0010) [2023-10-14 19:18:00,826][61552] Updated weights for policy 0, policy_version 40632 (0.0010) [2023-10-14 19:18:00,999][61585] Updated weights for policy 1, policy_version 40450 (0.0010) [2023-10-14 19:18:01,360][61585] Updated weights for policy 1, policy_version 40460 (0.0009) [2023-10-14 19:18:01,729][61585] Updated weights for policy 1, policy_version 40470 (0.0008) [2023-10-14 19:18:02,090][61585] Updated weights for policy 1, policy_version 40480 (0.0009) [2023-10-14 19:18:03,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 83066880. Throughput: 0: 1654.7, 1: 1667.8. Samples: 20768916. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:18:03,344][60425] Avg episode reward: [(0, '67.790'), (1, '58.660')] [2023-10-14 19:18:05,015][61552] Updated weights for policy 0, policy_version 40642 (0.0010) [2023-10-14 19:18:05,423][61552] Updated weights for policy 0, policy_version 40652 (0.0009) [2023-10-14 19:18:05,788][61552] Updated weights for policy 0, policy_version 40662 (0.0009) [2023-10-14 19:18:06,032][61585] Updated weights for policy 1, policy_version 40490 (0.0007) [2023-10-14 19:18:06,146][61552] Updated weights for policy 0, policy_version 40672 (0.0009) [2023-10-14 19:18:06,404][61585] Updated weights for policy 1, policy_version 40500 (0.0009) [2023-10-14 19:18:06,770][61585] Updated weights for policy 1, policy_version 40510 (0.0010) [2023-10-14 19:18:08,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 83132416. Throughput: 0: 1661.9, 1: 1648.3. Samples: 20787992. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:18:08,344][60425] Avg episode reward: [(0, '70.450'), (1, '57.170')] [2023-10-14 19:18:10,330][61552] Updated weights for policy 0, policy_version 40682 (0.0008) [2023-10-14 19:18:10,695][61552] Updated weights for policy 0, policy_version 40692 (0.0009) [2023-10-14 19:18:10,898][61585] Updated weights for policy 1, policy_version 40520 (0.0008) [2023-10-14 19:18:11,065][61552] Updated weights for policy 0, policy_version 40702 (0.0007) [2023-10-14 19:18:11,265][61585] Updated weights for policy 1, policy_version 40530 (0.0008) [2023-10-14 19:18:11,638][61585] Updated weights for policy 1, policy_version 40540 (0.0007) [2023-10-14 19:18:13,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 83197952. Throughput: 0: 1664.7, 1: 1672.8. Samples: 20808548. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:18:13,344][60425] Avg episode reward: [(0, '70.000'), (1, '59.170')] [2023-10-14 19:18:15,105][61552] Updated weights for policy 0, policy_version 40712 (0.0009) [2023-10-14 19:18:15,477][61552] Updated weights for policy 0, policy_version 40722 (0.0009) [2023-10-14 19:18:15,740][61585] Updated weights for policy 1, policy_version 40550 (0.0007) [2023-10-14 19:18:15,845][61552] Updated weights for policy 0, policy_version 40732 (0.0008) [2023-10-14 19:18:16,099][61585] Updated weights for policy 1, policy_version 40560 (0.0009) [2023-10-14 19:18:16,466][61585] Updated weights for policy 1, policy_version 40570 (0.0008) [2023-10-14 19:18:18,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 83263488. Throughput: 0: 1650.4, 1: 1669.9. Samples: 20818912. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:18:18,344][60425] Avg episode reward: [(0, '69.660'), (1, '57.250')] [2023-10-14 19:18:19,815][61552] Updated weights for policy 0, policy_version 40742 (0.0008) [2023-10-14 19:18:20,178][61552] Updated weights for policy 0, policy_version 40752 (0.0008) [2023-10-14 19:18:20,551][61552] Updated weights for policy 0, policy_version 40762 (0.0008) [2023-10-14 19:18:20,723][61585] Updated weights for policy 1, policy_version 40580 (0.0008) [2023-10-14 19:18:21,093][61585] Updated weights for policy 1, policy_version 40590 (0.0008) [2023-10-14 19:18:21,454][61585] Updated weights for policy 1, policy_version 40600 (0.0009) [2023-10-14 19:18:23,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 83329024. Throughput: 0: 1672.3, 1: 1657.9. Samples: 20838328. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:18:23,344][60425] Avg episode reward: [(0, '71.250'), (1, '59.550')] [2023-10-14 19:18:24,614][61552] Updated weights for policy 0, policy_version 40772 (0.0008) [2023-10-14 19:18:24,976][61552] Updated weights for policy 0, policy_version 40782 (0.0009) [2023-10-14 19:18:25,359][61552] Updated weights for policy 0, policy_version 40792 (0.0010) [2023-10-14 19:18:25,635][61585] Updated weights for policy 1, policy_version 40610 (0.0009) [2023-10-14 19:18:26,056][61585] Updated weights for policy 1, policy_version 40620 (0.0008) [2023-10-14 19:18:26,416][61585] Updated weights for policy 1, policy_version 40630 (0.0009) [2023-10-14 19:18:26,785][61585] Updated weights for policy 1, policy_version 40640 (0.0010) [2023-10-14 19:18:28,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 83394560. Throughput: 0: 1673.0, 1: 1666.5. Samples: 20858588. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:18:28,344][60425] Avg episode reward: [(0, '70.170'), (1, '57.880')] [2023-10-14 19:18:29,310][61552] Updated weights for policy 0, policy_version 40802 (0.0010) [2023-10-14 19:18:29,676][61552] Updated weights for policy 0, policy_version 40812 (0.0009) [2023-10-14 19:18:30,045][61552] Updated weights for policy 0, policy_version 40822 (0.0008) [2023-10-14 19:18:30,406][61552] Updated weights for policy 0, policy_version 40832 (0.0007) [2023-10-14 19:18:30,892][61585] Updated weights for policy 1, policy_version 40650 (0.0010) [2023-10-14 19:18:31,264][61585] Updated weights for policy 1, policy_version 40660 (0.0009) [2023-10-14 19:18:31,635][61585] Updated weights for policy 1, policy_version 40670 (0.0012) [2023-10-14 19:18:33,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 83460096. Throughput: 0: 1662.6, 1: 1662.4. Samples: 20868792. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:18:33,344][60425] Avg episode reward: [(0, '71.810'), (1, '57.470')] [2023-10-14 19:18:34,686][61552] Updated weights for policy 0, policy_version 40842 (0.0008) [2023-10-14 19:18:35,057][61552] Updated weights for policy 0, policy_version 40852 (0.0008) [2023-10-14 19:18:35,429][61552] Updated weights for policy 0, policy_version 40862 (0.0009) [2023-10-14 19:18:35,814][61585] Updated weights for policy 1, policy_version 40680 (0.0008) [2023-10-14 19:18:36,179][61585] Updated weights for policy 1, policy_version 40690 (0.0009) [2023-10-14 19:18:36,536][61585] Updated weights for policy 1, policy_version 40700 (0.0010) [2023-10-14 19:18:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 83525632. Throughput: 0: 1685.8, 1: 1658.5. Samples: 20888400. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:18:38,344][60425] Avg episode reward: [(0, '67.950'), (1, '60.300')] [2023-10-14 19:18:39,343][61552] Updated weights for policy 0, policy_version 40872 (0.0009) [2023-10-14 19:18:39,710][61552] Updated weights for policy 0, policy_version 40882 (0.0010) [2023-10-14 19:18:40,075][61552] Updated weights for policy 0, policy_version 40892 (0.0010) [2023-10-14 19:18:40,628][61585] Updated weights for policy 1, policy_version 40710 (0.0011) [2023-10-14 19:18:41,000][61585] Updated weights for policy 1, policy_version 40720 (0.0011) [2023-10-14 19:18:41,364][61585] Updated weights for policy 1, policy_version 40730 (0.0009) [2023-10-14 19:18:43,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 83591168. Throughput: 0: 1689.3, 1: 1676.0. Samples: 20909438. Policy #0 lag: (min: 1.0, avg: 12.9, max: 33.0) [2023-10-14 19:18:43,345][60425] Avg episode reward: [(0, '66.050'), (1, '58.910')] [2023-10-14 19:18:44,168][61552] Updated weights for policy 0, policy_version 40902 (0.0010) [2023-10-14 19:18:44,537][61552] Updated weights for policy 0, policy_version 40912 (0.0007) [2023-10-14 19:18:44,908][61552] Updated weights for policy 0, policy_version 40922 (0.0012) [2023-10-14 19:18:45,578][61585] Updated weights for policy 1, policy_version 40740 (0.0008) [2023-10-14 19:18:45,948][61585] Updated weights for policy 1, policy_version 40750 (0.0008) [2023-10-14 19:18:46,312][61585] Updated weights for policy 1, policy_version 40760 (0.0009) [2023-10-14 19:18:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 83656704. Throughput: 0: 1677.5, 1: 1667.1. Samples: 20919420. Policy #0 lag: (min: 1.0, avg: 12.9, max: 33.0) [2023-10-14 19:18:48,344][60425] Avg episode reward: [(0, '69.290'), (1, '54.470')] [2023-10-14 19:18:49,111][61552] Updated weights for policy 0, policy_version 40932 (0.0010) [2023-10-14 19:18:49,485][61552] Updated weights for policy 0, policy_version 40942 (0.0009) [2023-10-14 19:18:49,867][61552] Updated weights for policy 0, policy_version 40952 (0.0010) [2023-10-14 19:18:50,279][61585] Updated weights for policy 1, policy_version 40770 (0.0009) [2023-10-14 19:18:50,645][61585] Updated weights for policy 1, policy_version 40780 (0.0007) [2023-10-14 19:18:51,007][61585] Updated weights for policy 1, policy_version 40790 (0.0007) [2023-10-14 19:18:51,383][61585] Updated weights for policy 1, policy_version 40800 (0.0009) [2023-10-14 19:18:53,343][60425] Fps is (10 sec: 13107.7, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 83722240. Throughput: 0: 1687.1, 1: 1665.2. Samples: 20938844. Policy #0 lag: (min: 1.0, avg: 12.9, max: 33.0) [2023-10-14 19:18:53,344][60425] Avg episode reward: [(0, '65.900'), (1, '62.680')] [2023-10-14 19:18:53,850][61552] Updated weights for policy 0, policy_version 40962 (0.0008) [2023-10-14 19:18:54,222][61552] Updated weights for policy 0, policy_version 40972 (0.0010) [2023-10-14 19:18:54,594][61552] Updated weights for policy 0, policy_version 40982 (0.0009) [2023-10-14 19:18:54,964][61552] Updated weights for policy 0, policy_version 40992 (0.0008) [2023-10-14 19:18:55,483][61585] Updated weights for policy 1, policy_version 40810 (0.0009) [2023-10-14 19:18:55,851][61585] Updated weights for policy 1, policy_version 40820 (0.0008) [2023-10-14 19:18:56,210][61585] Updated weights for policy 1, policy_version 40830 (0.0009) [2023-10-14 19:18:58,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 83787776. Throughput: 0: 1684.8, 1: 1668.1. Samples: 20959426. Policy #0 lag: (min: 1.0, avg: 12.9, max: 33.0) [2023-10-14 19:18:58,344][60425] Avg episode reward: [(0, '68.670'), (1, '58.190')] [2023-10-14 19:18:58,999][61552] Updated weights for policy 0, policy_version 41002 (0.0010) [2023-10-14 19:18:59,369][61552] Updated weights for policy 0, policy_version 41012 (0.0011) [2023-10-14 19:18:59,745][61552] Updated weights for policy 0, policy_version 41022 (0.0008) [2023-10-14 19:19:00,289][61585] Updated weights for policy 1, policy_version 40840 (0.0008) [2023-10-14 19:19:00,652][61585] Updated weights for policy 1, policy_version 40850 (0.0008) [2023-10-14 19:19:01,025][61585] Updated weights for policy 1, policy_version 40860 (0.0007) [2023-10-14 19:19:03,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 83853312. Throughput: 0: 1677.2, 1: 1658.5. Samples: 20969016. Policy #0 lag: (min: 1.0, avg: 12.9, max: 33.0) [2023-10-14 19:19:03,344][60425] Avg episode reward: [(0, '66.690'), (1, '61.650')] [2023-10-14 19:19:03,809][61552] Updated weights for policy 0, policy_version 41032 (0.0008) [2023-10-14 19:19:04,181][61552] Updated weights for policy 0, policy_version 41042 (0.0010) [2023-10-14 19:19:04,552][61552] Updated weights for policy 0, policy_version 41052 (0.0009) [2023-10-14 19:19:05,105][61585] Updated weights for policy 1, policy_version 40870 (0.0007) [2023-10-14 19:19:05,458][61585] Updated weights for policy 1, policy_version 40880 (0.0007) [2023-10-14 19:19:05,828][61585] Updated weights for policy 1, policy_version 40890 (0.0010) [2023-10-14 19:19:08,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 83918848. Throughput: 0: 1681.6, 1: 1667.8. Samples: 20989048. Policy #0 lag: (min: 1.0, avg: 12.9, max: 33.0) [2023-10-14 19:19:08,344][60425] Avg episode reward: [(0, '69.300'), (1, '61.390')] [2023-10-14 19:19:08,553][61552] Updated weights for policy 0, policy_version 41062 (0.0007) [2023-10-14 19:19:08,923][61552] Updated weights for policy 0, policy_version 41072 (0.0009) [2023-10-14 19:19:09,291][61552] Updated weights for policy 0, policy_version 41082 (0.0010) [2023-10-14 19:19:10,026][61585] Updated weights for policy 1, policy_version 40900 (0.0009) [2023-10-14 19:19:10,387][61585] Updated weights for policy 1, policy_version 40910 (0.0009) [2023-10-14 19:19:10,750][61585] Updated weights for policy 1, policy_version 40920 (0.0007) [2023-10-14 19:19:13,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 83984384. Throughput: 0: 1679.1, 1: 1675.2. Samples: 21009530. Policy #0 lag: (min: 28.0, avg: 28.8, max: 48.0) [2023-10-14 19:19:13,344][60425] Avg episode reward: [(0, '70.280'), (1, '60.020')] [2023-10-14 19:19:13,469][61552] Updated weights for policy 0, policy_version 41092 (0.0008) [2023-10-14 19:19:13,844][61552] Updated weights for policy 0, policy_version 41102 (0.0007) [2023-10-14 19:19:14,211][61552] Updated weights for policy 0, policy_version 41112 (0.0007) [2023-10-14 19:19:14,858][61585] Updated weights for policy 1, policy_version 40930 (0.0008) [2023-10-14 19:19:15,278][61585] Updated weights for policy 1, policy_version 40940 (0.0008) [2023-10-14 19:19:15,654][61585] Updated weights for policy 1, policy_version 40950 (0.0008) [2023-10-14 19:19:16,018][61585] Updated weights for policy 1, policy_version 40960 (0.0009) [2023-10-14 19:19:18,300][61552] Updated weights for policy 0, policy_version 41122 (0.0008) [2023-10-14 19:19:18,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 84049920. Throughput: 0: 1678.1, 1: 1652.0. Samples: 21018648. Policy #0 lag: (min: 28.0, avg: 28.8, max: 48.0) [2023-10-14 19:19:18,344][60425] Avg episode reward: [(0, '67.270'), (1, '56.620')] [2023-10-14 19:19:18,659][61552] Updated weights for policy 0, policy_version 41132 (0.0009) [2023-10-14 19:19:19,033][61552] Updated weights for policy 0, policy_version 41142 (0.0008) [2023-10-14 19:19:19,398][61552] Updated weights for policy 0, policy_version 41152 (0.0007) [2023-10-14 19:19:20,086][61585] Updated weights for policy 1, policy_version 40970 (0.0009) [2023-10-14 19:19:20,445][61585] Updated weights for policy 1, policy_version 40980 (0.0009) [2023-10-14 19:19:20,822][61585] Updated weights for policy 1, policy_version 40990 (0.0011) [2023-10-14 19:19:23,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 84115456. Throughput: 0: 1674.1, 1: 1666.3. Samples: 21038718. Policy #0 lag: (min: 28.0, avg: 28.8, max: 48.0) [2023-10-14 19:19:23,344][60425] Avg episode reward: [(0, '67.800'), (1, '61.970')] [2023-10-14 19:19:23,542][61552] Updated weights for policy 0, policy_version 41162 (0.0009) [2023-10-14 19:19:23,905][61552] Updated weights for policy 0, policy_version 41172 (0.0008) [2023-10-14 19:19:24,273][61552] Updated weights for policy 0, policy_version 41182 (0.0010) [2023-10-14 19:19:24,974][61585] Updated weights for policy 1, policy_version 41000 (0.0009) [2023-10-14 19:19:25,337][61585] Updated weights for policy 1, policy_version 41010 (0.0011) [2023-10-14 19:19:25,713][61585] Updated weights for policy 1, policy_version 41020 (0.0010) [2023-10-14 19:19:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 84180992. Throughput: 0: 1668.4, 1: 1663.3. Samples: 21059364. Policy #0 lag: (min: 28.0, avg: 28.8, max: 48.0) [2023-10-14 19:19:28,344][60425] Avg episode reward: [(0, '66.470'), (1, '59.690')] [2023-10-14 19:19:28,360][61552] Updated weights for policy 0, policy_version 41192 (0.0009) [2023-10-14 19:19:28,724][61552] Updated weights for policy 0, policy_version 41202 (0.0009) [2023-10-14 19:19:29,103][61552] Updated weights for policy 0, policy_version 41212 (0.0010) [2023-10-14 19:19:29,993][61585] Updated weights for policy 1, policy_version 41030 (0.0008) [2023-10-14 19:19:30,348][61585] Updated weights for policy 1, policy_version 41040 (0.0009) [2023-10-14 19:19:30,718][61585] Updated weights for policy 1, policy_version 41050 (0.0008) [2023-10-14 19:19:33,127][61552] Updated weights for policy 0, policy_version 41222 (0.0009) [2023-10-14 19:19:33,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 84246528. Throughput: 0: 1669.2, 1: 1648.5. Samples: 21068720. Policy #0 lag: (min: 28.0, avg: 28.8, max: 48.0) [2023-10-14 19:19:33,344][60425] Avg episode reward: [(0, '69.440'), (1, '55.780')] [2023-10-14 19:19:33,490][61552] Updated weights for policy 0, policy_version 41232 (0.0009) [2023-10-14 19:19:33,861][61552] Updated weights for policy 0, policy_version 41242 (0.0007) [2023-10-14 19:19:34,938][61585] Updated weights for policy 1, policy_version 41060 (0.0008) [2023-10-14 19:19:35,308][61585] Updated weights for policy 1, policy_version 41070 (0.0008) [2023-10-14 19:19:35,670][61585] Updated weights for policy 1, policy_version 41080 (0.0011) [2023-10-14 19:19:37,919][61552] Updated weights for policy 0, policy_version 41252 (0.0010) [2023-10-14 19:19:38,284][61552] Updated weights for policy 0, policy_version 41262 (0.0007) [2023-10-14 19:19:38,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 84312064. Throughput: 0: 1678.7, 1: 1659.4. Samples: 21089058. Policy #0 lag: (min: 28.0, avg: 28.8, max: 48.0) [2023-10-14 19:19:38,344][60425] Avg episode reward: [(0, '66.310'), (1, '59.320')] [2023-10-14 19:19:38,654][61552] Updated weights for policy 0, policy_version 41272 (0.0008) [2023-10-14 19:19:39,700][61585] Updated weights for policy 1, policy_version 41090 (0.0009) [2023-10-14 19:19:40,066][61585] Updated weights for policy 1, policy_version 41100 (0.0009) [2023-10-14 19:19:40,427][61585] Updated weights for policy 1, policy_version 41110 (0.0008) [2023-10-14 19:19:40,789][61585] Updated weights for policy 1, policy_version 41120 (0.0008) [2023-10-14 19:19:42,783][61552] Updated weights for policy 0, policy_version 41282 (0.0010) [2023-10-14 19:19:43,198][61552] Updated weights for policy 0, policy_version 41292 (0.0010) [2023-10-14 19:19:43,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 84377600. Throughput: 0: 1677.1, 1: 1662.0. Samples: 21109688. Policy #0 lag: (min: 10.0, avg: 10.0, max: 14.0) [2023-10-14 19:19:43,344][60425] Avg episode reward: [(0, '64.930'), (1, '57.150')] [2023-10-14 19:19:43,360][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000041120_42106880.pth... [2023-10-14 19:19:43,391][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000039584_40534016.pth [2023-10-14 19:19:43,550][61552] Updated weights for policy 0, policy_version 41302 (0.0009) [2023-10-14 19:19:43,910][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000041312_42303488.pth... [2023-10-14 19:19:43,913][61552] Updated weights for policy 0, policy_version 41312 (0.0008) [2023-10-14 19:19:43,941][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000039744_40697856.pth [2023-10-14 19:19:44,945][61585] Updated weights for policy 1, policy_version 41130 (0.0010) [2023-10-14 19:19:45,307][61585] Updated weights for policy 1, policy_version 41140 (0.0008) [2023-10-14 19:19:45,679][61585] Updated weights for policy 1, policy_version 41150 (0.0009) [2023-10-14 19:19:48,021][61552] Updated weights for policy 0, policy_version 41322 (0.0008) [2023-10-14 19:19:48,343][60425] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 84443136. Throughput: 0: 1674.5, 1: 1653.2. Samples: 21118764. Policy #0 lag: (min: 10.0, avg: 10.0, max: 14.0) [2023-10-14 19:19:48,344][60425] Avg episode reward: [(0, '68.090'), (1, '55.590')] [2023-10-14 19:19:48,392][61552] Updated weights for policy 0, policy_version 41332 (0.0009) [2023-10-14 19:19:48,766][61552] Updated weights for policy 0, policy_version 41342 (0.0009) [2023-10-14 19:19:49,609][61585] Updated weights for policy 1, policy_version 41160 (0.0010) [2023-10-14 19:19:49,980][61585] Updated weights for policy 1, policy_version 41170 (0.0009) [2023-10-14 19:19:50,354][61585] Updated weights for policy 1, policy_version 41180 (0.0009) [2023-10-14 19:19:52,790][61552] Updated weights for policy 0, policy_version 41352 (0.0011) [2023-10-14 19:19:53,156][61552] Updated weights for policy 0, policy_version 41362 (0.0010) [2023-10-14 19:19:53,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 84508672. Throughput: 0: 1673.7, 1: 1666.2. Samples: 21139344. Policy #0 lag: (min: 10.0, avg: 10.0, max: 14.0) [2023-10-14 19:19:53,344][60425] Avg episode reward: [(0, '67.380'), (1, '55.050')] [2023-10-14 19:19:53,533][61552] Updated weights for policy 0, policy_version 41372 (0.0009) [2023-10-14 19:19:54,566][61585] Updated weights for policy 1, policy_version 41190 (0.0010) [2023-10-14 19:19:54,927][61585] Updated weights for policy 1, policy_version 41200 (0.0008) [2023-10-14 19:19:55,301][61585] Updated weights for policy 1, policy_version 41210 (0.0010) [2023-10-14 19:19:57,674][61552] Updated weights for policy 0, policy_version 41382 (0.0008) [2023-10-14 19:19:58,043][61552] Updated weights for policy 0, policy_version 41392 (0.0008) [2023-10-14 19:19:58,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 84574208. Throughput: 0: 1669.0, 1: 1666.6. Samples: 21159630. Policy #0 lag: (min: 10.0, avg: 10.0, max: 14.0) [2023-10-14 19:19:58,344][60425] Avg episode reward: [(0, '68.870'), (1, '62.700')] [2023-10-14 19:19:58,407][61552] Updated weights for policy 0, policy_version 41402 (0.0010) [2023-10-14 19:19:59,286][61585] Updated weights for policy 1, policy_version 41220 (0.0010) [2023-10-14 19:19:59,656][61585] Updated weights for policy 1, policy_version 41230 (0.0009) [2023-10-14 19:20:00,022][61585] Updated weights for policy 1, policy_version 41240 (0.0009) [2023-10-14 19:20:02,460][61552] Updated weights for policy 0, policy_version 41412 (0.0010) [2023-10-14 19:20:02,826][61552] Updated weights for policy 0, policy_version 41422 (0.0009) [2023-10-14 19:20:03,204][61552] Updated weights for policy 0, policy_version 41432 (0.0010) [2023-10-14 19:20:03,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 84639744. Throughput: 0: 1674.6, 1: 1666.3. Samples: 21168990. Policy #0 lag: (min: 10.0, avg: 10.0, max: 14.0) [2023-10-14 19:20:03,344][60425] Avg episode reward: [(0, '68.650'), (1, '56.660')] [2023-10-14 19:20:04,034][61585] Updated weights for policy 1, policy_version 41250 (0.0008) [2023-10-14 19:20:04,402][61585] Updated weights for policy 1, policy_version 41260 (0.0008) [2023-10-14 19:20:04,767][61585] Updated weights for policy 1, policy_version 41270 (0.0007) [2023-10-14 19:20:05,124][61585] Updated weights for policy 1, policy_version 41280 (0.0007) [2023-10-14 19:20:07,270][61552] Updated weights for policy 0, policy_version 41442 (0.0009) [2023-10-14 19:20:07,644][61552] Updated weights for policy 0, policy_version 41452 (0.0008) [2023-10-14 19:20:08,020][61552] Updated weights for policy 0, policy_version 41462 (0.0008) [2023-10-14 19:20:08,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 84705280. Throughput: 0: 1679.1, 1: 1671.9. Samples: 21189512. Policy #0 lag: (min: 10.0, avg: 10.0, max: 14.0) [2023-10-14 19:20:08,344][60425] Avg episode reward: [(0, '70.180'), (1, '60.740')] [2023-10-14 19:20:08,398][61552] Updated weights for policy 0, policy_version 41472 (0.0008) [2023-10-14 19:20:09,317][61585] Updated weights for policy 1, policy_version 41290 (0.0009) [2023-10-14 19:20:09,680][61585] Updated weights for policy 1, policy_version 41300 (0.0008) [2023-10-14 19:20:10,043][61585] Updated weights for policy 1, policy_version 41310 (0.0008) [2023-10-14 19:20:12,396][61552] Updated weights for policy 0, policy_version 41482 (0.0007) [2023-10-14 19:20:12,763][61552] Updated weights for policy 0, policy_version 41492 (0.0007) [2023-10-14 19:20:13,133][61552] Updated weights for policy 0, policy_version 41502 (0.0007) [2023-10-14 19:20:13,344][60425] Fps is (10 sec: 16383.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 84803584. Throughput: 0: 1666.3, 1: 1668.7. Samples: 21209440. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:20:13,345][60425] Avg episode reward: [(0, '70.930'), (1, '61.970')] [2023-10-14 19:20:14,244][61585] Updated weights for policy 1, policy_version 41320 (0.0009) [2023-10-14 19:20:14,598][61585] Updated weights for policy 1, policy_version 41330 (0.0008) [2023-10-14 19:20:14,968][61585] Updated weights for policy 1, policy_version 41340 (0.0009) [2023-10-14 19:20:17,199][61552] Updated weights for policy 0, policy_version 41512 (0.0009) [2023-10-14 19:20:17,571][61552] Updated weights for policy 0, policy_version 41522 (0.0008) [2023-10-14 19:20:17,939][61552] Updated weights for policy 0, policy_version 41532 (0.0008) [2023-10-14 19:20:18,343][60425] Fps is (10 sec: 16384.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 84869120. Throughput: 0: 1682.1, 1: 1665.1. Samples: 21219342. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:20:18,344][60425] Avg episode reward: [(0, '70.220'), (1, '59.620')] [2023-10-14 19:20:19,010][61585] Updated weights for policy 1, policy_version 41350 (0.0007) [2023-10-14 19:20:19,381][61585] Updated weights for policy 1, policy_version 41360 (0.0008) [2023-10-14 19:20:19,755][61585] Updated weights for policy 1, policy_version 41370 (0.0010) [2023-10-14 19:20:22,210][61552] Updated weights for policy 0, policy_version 41542 (0.0010) [2023-10-14 19:20:22,577][61552] Updated weights for policy 0, policy_version 41552 (0.0007) [2023-10-14 19:20:22,951][61552] Updated weights for policy 0, policy_version 41562 (0.0009) [2023-10-14 19:20:23,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 84934656. Throughput: 0: 1670.5, 1: 1679.4. Samples: 21239802. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:20:23,344][60425] Avg episode reward: [(0, '71.570'), (1, '61.640')] [2023-10-14 19:20:23,788][61585] Updated weights for policy 1, policy_version 41380 (0.0007) [2023-10-14 19:20:24,154][61585] Updated weights for policy 1, policy_version 41390 (0.0008) [2023-10-14 19:20:24,524][61585] Updated weights for policy 1, policy_version 41400 (0.0011) [2023-10-14 19:20:26,948][61552] Updated weights for policy 0, policy_version 41572 (0.0009) [2023-10-14 19:20:27,320][61552] Updated weights for policy 0, policy_version 41582 (0.0009) [2023-10-14 19:20:27,688][61552] Updated weights for policy 0, policy_version 41592 (0.0009) [2023-10-14 19:20:28,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 85000192. Throughput: 0: 1651.9, 1: 1675.4. Samples: 21259416. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:20:28,344][60425] Avg episode reward: [(0, '67.530'), (1, '63.620')] [2023-10-14 19:20:28,633][61585] Updated weights for policy 1, policy_version 41410 (0.0011) [2023-10-14 19:20:28,999][61585] Updated weights for policy 1, policy_version 41420 (0.0011) [2023-10-14 19:20:29,363][61585] Updated weights for policy 1, policy_version 41430 (0.0008) [2023-10-14 19:20:29,733][61585] Updated weights for policy 1, policy_version 41440 (0.0007) [2023-10-14 19:20:31,835][61552] Updated weights for policy 0, policy_version 41602 (0.0009) [2023-10-14 19:20:32,254][61552] Updated weights for policy 0, policy_version 41612 (0.0007) [2023-10-14 19:20:32,612][61552] Updated weights for policy 0, policy_version 41622 (0.0008) [2023-10-14 19:20:32,978][61552] Updated weights for policy 0, policy_version 41632 (0.0009) [2023-10-14 19:20:33,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 85065728. Throughput: 0: 1675.2, 1: 1671.1. Samples: 21269344. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:20:33,344][60425] Avg episode reward: [(0, '67.900'), (1, '56.670')] [2023-10-14 19:20:33,945][61585] Updated weights for policy 1, policy_version 41450 (0.0007) [2023-10-14 19:20:34,304][61585] Updated weights for policy 1, policy_version 41460 (0.0008) [2023-10-14 19:20:34,667][61585] Updated weights for policy 1, policy_version 41470 (0.0007) [2023-10-14 19:20:37,086][61552] Updated weights for policy 0, policy_version 41642 (0.0008) [2023-10-14 19:20:37,452][61552] Updated weights for policy 0, policy_version 41652 (0.0009) [2023-10-14 19:20:37,822][61552] Updated weights for policy 0, policy_version 41662 (0.0009) [2023-10-14 19:20:38,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 85131264. Throughput: 0: 1669.6, 1: 1670.7. Samples: 21289658. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:20:38,344][60425] Avg episode reward: [(0, '69.400'), (1, '59.980')] [2023-10-14 19:20:38,814][61585] Updated weights for policy 1, policy_version 41480 (0.0009) [2023-10-14 19:20:39,184][61585] Updated weights for policy 1, policy_version 41490 (0.0009) [2023-10-14 19:20:39,539][61585] Updated weights for policy 1, policy_version 41500 (0.0010) [2023-10-14 19:20:41,889][61552] Updated weights for policy 0, policy_version 41672 (0.0008) [2023-10-14 19:20:42,254][61552] Updated weights for policy 0, policy_version 41682 (0.0007) [2023-10-14 19:20:42,619][61552] Updated weights for policy 0, policy_version 41692 (0.0010) [2023-10-14 19:20:43,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 85196800. Throughput: 0: 1649.4, 1: 1678.7. Samples: 21309394. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:20:43,344][60425] Avg episode reward: [(0, '70.570'), (1, '60.950')] [2023-10-14 19:20:43,409][61585] Updated weights for policy 1, policy_version 41510 (0.0009) [2023-10-14 19:20:43,775][61585] Updated weights for policy 1, policy_version 41520 (0.0008) [2023-10-14 19:20:44,148][61585] Updated weights for policy 1, policy_version 41530 (0.0008) [2023-10-14 19:20:46,737][61552] Updated weights for policy 0, policy_version 41702 (0.0011) [2023-10-14 19:20:47,102][61552] Updated weights for policy 0, policy_version 41712 (0.0011) [2023-10-14 19:20:47,465][61552] Updated weights for policy 0, policy_version 41722 (0.0011) [2023-10-14 19:20:48,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 85262336. Throughput: 0: 1666.9, 1: 1680.2. Samples: 21319612. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:20:48,344][60425] Avg episode reward: [(0, '71.890'), (1, '61.280')] [2023-10-14 19:20:48,374][61585] Updated weights for policy 1, policy_version 41540 (0.0008) [2023-10-14 19:20:48,749][61585] Updated weights for policy 1, policy_version 41550 (0.0009) [2023-10-14 19:20:49,108][61585] Updated weights for policy 1, policy_version 41560 (0.0010) [2023-10-14 19:20:51,605][61552] Updated weights for policy 0, policy_version 41732 (0.0009) [2023-10-14 19:20:51,982][61552] Updated weights for policy 0, policy_version 41742 (0.0008) [2023-10-14 19:20:52,350][61552] Updated weights for policy 0, policy_version 41752 (0.0007) [2023-10-14 19:20:53,056][61585] Updated weights for policy 1, policy_version 41570 (0.0007) [2023-10-14 19:20:53,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 85327872. Throughput: 0: 1658.4, 1: 1679.7. Samples: 21339728. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:20:53,344][60425] Avg episode reward: [(0, '67.790'), (1, '57.860')] [2023-10-14 19:20:53,417][61585] Updated weights for policy 1, policy_version 41580 (0.0007) [2023-10-14 19:20:53,775][61585] Updated weights for policy 1, policy_version 41590 (0.0007) [2023-10-14 19:20:54,132][61585] Updated weights for policy 1, policy_version 41600 (0.0009) [2023-10-14 19:20:56,322][61552] Updated weights for policy 0, policy_version 41762 (0.0008) [2023-10-14 19:20:56,691][61552] Updated weights for policy 0, policy_version 41772 (0.0011) [2023-10-14 19:20:57,064][61552] Updated weights for policy 0, policy_version 41782 (0.0008) [2023-10-14 19:20:57,423][61552] Updated weights for policy 0, policy_version 41792 (0.0009) [2023-10-14 19:20:58,193][61585] Updated weights for policy 1, policy_version 41610 (0.0010) [2023-10-14 19:20:58,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 85393408. Throughput: 0: 1648.0, 1: 1685.1. Samples: 21359426. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:20:58,344][60425] Avg episode reward: [(0, '67.470'), (1, '58.330')] [2023-10-14 19:20:58,558][61585] Updated weights for policy 1, policy_version 41620 (0.0009) [2023-10-14 19:20:58,921][61585] Updated weights for policy 1, policy_version 41630 (0.0009) [2023-10-14 19:21:01,522][61552] Updated weights for policy 0, policy_version 41802 (0.0011) [2023-10-14 19:21:01,902][61552] Updated weights for policy 0, policy_version 41812 (0.0010) [2023-10-14 19:21:02,273][61552] Updated weights for policy 0, policy_version 41822 (0.0008) [2023-10-14 19:21:02,980][61585] Updated weights for policy 1, policy_version 41640 (0.0008) [2023-10-14 19:21:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 85458944. Throughput: 0: 1662.7, 1: 1682.0. Samples: 21369854. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:21:03,344][60425] Avg episode reward: [(0, '69.970'), (1, '63.030')] [2023-10-14 19:21:03,346][61585] Updated weights for policy 1, policy_version 41650 (0.0009) [2023-10-14 19:21:03,714][61585] Updated weights for policy 1, policy_version 41660 (0.0009) [2023-10-14 19:21:06,401][61552] Updated weights for policy 0, policy_version 41832 (0.0008) [2023-10-14 19:21:06,760][61552] Updated weights for policy 0, policy_version 41842 (0.0010) [2023-10-14 19:21:07,130][61552] Updated weights for policy 0, policy_version 41852 (0.0007) [2023-10-14 19:21:07,809][61585] Updated weights for policy 1, policy_version 41670 (0.0009) [2023-10-14 19:21:08,174][61585] Updated weights for policy 1, policy_version 41680 (0.0007) [2023-10-14 19:21:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 85524480. Throughput: 0: 1654.0, 1: 1681.7. Samples: 21389906. Policy #0 lag: (min: 31.0, avg: 42.5, max: 63.0) [2023-10-14 19:21:08,344][60425] Avg episode reward: [(0, '67.510'), (1, '61.080')] [2023-10-14 19:21:08,546][61585] Updated weights for policy 1, policy_version 41690 (0.0007) [2023-10-14 19:21:11,258][61552] Updated weights for policy 0, policy_version 41862 (0.0009) [2023-10-14 19:21:11,636][61552] Updated weights for policy 0, policy_version 41872 (0.0007) [2023-10-14 19:21:12,003][61552] Updated weights for policy 0, policy_version 41882 (0.0007) [2023-10-14 19:21:12,727][61585] Updated weights for policy 1, policy_version 41700 (0.0007) [2023-10-14 19:21:13,097][61585] Updated weights for policy 1, policy_version 41710 (0.0009) [2023-10-14 19:21:13,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 85590016. Throughput: 0: 1660.7, 1: 1678.4. Samples: 21409676. Policy #0 lag: (min: 31.0, avg: 42.5, max: 63.0) [2023-10-14 19:21:13,344][60425] Avg episode reward: [(0, '67.640'), (1, '62.230')] [2023-10-14 19:21:13,456][61585] Updated weights for policy 1, policy_version 41720 (0.0009) [2023-10-14 19:21:16,100][61552] Updated weights for policy 0, policy_version 41892 (0.0009) [2023-10-14 19:21:16,473][61552] Updated weights for policy 0, policy_version 41902 (0.0011) [2023-10-14 19:21:16,836][61552] Updated weights for policy 0, policy_version 41912 (0.0010) [2023-10-14 19:21:17,637][61585] Updated weights for policy 1, policy_version 41730 (0.0008) [2023-10-14 19:21:18,004][61585] Updated weights for policy 1, policy_version 41740 (0.0009) [2023-10-14 19:21:18,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 85655552. Throughput: 0: 1667.8, 1: 1684.8. Samples: 21420208. Policy #0 lag: (min: 31.0, avg: 42.5, max: 63.0) [2023-10-14 19:21:18,344][60425] Avg episode reward: [(0, '71.100'), (1, '64.170')] [2023-10-14 19:21:18,358][61585] Updated weights for policy 1, policy_version 41750 (0.0010) [2023-10-14 19:21:18,722][61585] Updated weights for policy 1, policy_version 41760 (0.0008) [2023-10-14 19:21:20,977][61552] Updated weights for policy 0, policy_version 41922 (0.0009) [2023-10-14 19:21:21,362][61552] Updated weights for policy 0, policy_version 41932 (0.0010) [2023-10-14 19:21:21,740][61552] Updated weights for policy 0, policy_version 41942 (0.0010) [2023-10-14 19:21:22,106][61552] Updated weights for policy 0, policy_version 41952 (0.0008) [2023-10-14 19:21:22,750][61585] Updated weights for policy 1, policy_version 41770 (0.0011) [2023-10-14 19:21:23,123][61585] Updated weights for policy 1, policy_version 41780 (0.0010) [2023-10-14 19:21:23,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 85721088. Throughput: 0: 1650.6, 1: 1687.8. Samples: 21439888. Policy #0 lag: (min: 31.0, avg: 42.5, max: 63.0) [2023-10-14 19:21:23,344][60425] Avg episode reward: [(0, '63.380'), (1, '59.050')] [2023-10-14 19:21:23,491][61585] Updated weights for policy 1, policy_version 41790 (0.0007) [2023-10-14 19:21:26,185][61552] Updated weights for policy 0, policy_version 41962 (0.0010) [2023-10-14 19:21:26,557][61552] Updated weights for policy 0, policy_version 41972 (0.0010) [2023-10-14 19:21:26,922][61552] Updated weights for policy 0, policy_version 41982 (0.0010) [2023-10-14 19:21:27,592][61585] Updated weights for policy 1, policy_version 41800 (0.0009) [2023-10-14 19:21:27,962][61585] Updated weights for policy 1, policy_version 41810 (0.0008) [2023-10-14 19:21:28,336][61585] Updated weights for policy 1, policy_version 41820 (0.0007) [2023-10-14 19:21:28,344][60425] Fps is (10 sec: 13106.7, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 85786624. Throughput: 0: 1665.1, 1: 1669.5. Samples: 21459454. Policy #0 lag: (min: 31.0, avg: 42.5, max: 63.0) [2023-10-14 19:21:28,345][60425] Avg episode reward: [(0, '68.310'), (1, '65.630')] [2023-10-14 19:21:28,481][61248] Saving new best policy, reward=65.630! [2023-10-14 19:21:31,219][61552] Updated weights for policy 0, policy_version 41992 (0.0010) [2023-10-14 19:21:31,587][61552] Updated weights for policy 0, policy_version 42002 (0.0008) [2023-10-14 19:21:31,952][61552] Updated weights for policy 0, policy_version 42012 (0.0010) [2023-10-14 19:21:32,494][61585] Updated weights for policy 1, policy_version 41830 (0.0007) [2023-10-14 19:21:32,884][61585] Updated weights for policy 1, policy_version 41840 (0.0007) [2023-10-14 19:21:33,247][61585] Updated weights for policy 1, policy_version 41850 (0.0009) [2023-10-14 19:21:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 85852160. Throughput: 0: 1667.6, 1: 1680.8. Samples: 21470292. Policy #0 lag: (min: 31.0, avg: 42.5, max: 63.0) [2023-10-14 19:21:33,344][60425] Avg episode reward: [(0, '65.950'), (1, '63.180')] [2023-10-14 19:21:36,027][61552] Updated weights for policy 0, policy_version 42022 (0.0008) [2023-10-14 19:21:36,396][61552] Updated weights for policy 0, policy_version 42032 (0.0007) [2023-10-14 19:21:36,774][61552] Updated weights for policy 0, policy_version 42042 (0.0009) [2023-10-14 19:21:37,261][61585] Updated weights for policy 1, policy_version 41860 (0.0008) [2023-10-14 19:21:37,628][61585] Updated weights for policy 1, policy_version 41870 (0.0007) [2023-10-14 19:21:37,984][61585] Updated weights for policy 1, policy_version 41880 (0.0007) [2023-10-14 19:21:38,343][60425] Fps is (10 sec: 16384.7, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 85950464. Throughput: 0: 1655.0, 1: 1681.3. Samples: 21489862. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:21:38,344][60425] Avg episode reward: [(0, '70.210'), (1, '59.800')] [2023-10-14 19:21:40,913][61552] Updated weights for policy 0, policy_version 42052 (0.0010) [2023-10-14 19:21:41,281][61552] Updated weights for policy 0, policy_version 42062 (0.0008) [2023-10-14 19:21:41,655][61552] Updated weights for policy 0, policy_version 42072 (0.0007) [2023-10-14 19:21:42,008][61585] Updated weights for policy 1, policy_version 41890 (0.0008) [2023-10-14 19:21:42,374][61585] Updated weights for policy 1, policy_version 41900 (0.0007) [2023-10-14 19:21:42,745][61585] Updated weights for policy 1, policy_version 41910 (0.0008) [2023-10-14 19:21:43,106][61585] Updated weights for policy 1, policy_version 41920 (0.0007) [2023-10-14 19:21:43,343][60425] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 86016000. Throughput: 0: 1671.8, 1: 1660.8. Samples: 21509392. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:21:43,344][60425] Avg episode reward: [(0, '66.850'), (1, '61.570')] [2023-10-14 19:21:43,354][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000041920_42926080.pth... [2023-10-14 19:21:43,354][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000042080_43089920.pth... [2023-10-14 19:21:43,389][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000040512_41484288.pth [2023-10-14 19:21:43,392][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000040352_41320448.pth [2023-10-14 19:21:45,803][61552] Updated weights for policy 0, policy_version 42082 (0.0009) [2023-10-14 19:21:46,168][61552] Updated weights for policy 0, policy_version 42092 (0.0011) [2023-10-14 19:21:46,537][61552] Updated weights for policy 0, policy_version 42102 (0.0012) [2023-10-14 19:21:46,916][61552] Updated weights for policy 0, policy_version 42112 (0.0010) [2023-10-14 19:21:47,152][61585] Updated weights for policy 1, policy_version 41930 (0.0007) [2023-10-14 19:21:47,507][61585] Updated weights for policy 1, policy_version 41940 (0.0008) [2023-10-14 19:21:47,872][61585] Updated weights for policy 1, policy_version 41950 (0.0010) [2023-10-14 19:21:48,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 86081536. Throughput: 0: 1670.5, 1: 1679.2. Samples: 21520590. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:21:48,344][60425] Avg episode reward: [(0, '70.120'), (1, '60.670')] [2023-10-14 19:21:50,872][61552] Updated weights for policy 0, policy_version 42122 (0.0007) [2023-10-14 19:21:51,243][61552] Updated weights for policy 0, policy_version 42132 (0.0008) [2023-10-14 19:21:51,606][61552] Updated weights for policy 0, policy_version 42142 (0.0010) [2023-10-14 19:21:52,153][61585] Updated weights for policy 1, policy_version 41960 (0.0008) [2023-10-14 19:21:52,527][61585] Updated weights for policy 1, policy_version 41970 (0.0008) [2023-10-14 19:21:52,884][61585] Updated weights for policy 1, policy_version 41980 (0.0007) [2023-10-14 19:21:53,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 86147072. Throughput: 0: 1660.8, 1: 1674.0. Samples: 21539972. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:21:53,344][60425] Avg episode reward: [(0, '66.700'), (1, '63.110')] [2023-10-14 19:21:55,500][61552] Updated weights for policy 0, policy_version 42152 (0.0010) [2023-10-14 19:21:55,863][61552] Updated weights for policy 0, policy_version 42162 (0.0010) [2023-10-14 19:21:56,237][61552] Updated weights for policy 0, policy_version 42172 (0.0009) [2023-10-14 19:21:57,055][61585] Updated weights for policy 1, policy_version 41990 (0.0008) [2023-10-14 19:21:57,426][61585] Updated weights for policy 1, policy_version 42000 (0.0008) [2023-10-14 19:21:57,798][61585] Updated weights for policy 1, policy_version 42010 (0.0007) [2023-10-14 19:21:58,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 86212608. Throughput: 0: 1681.5, 1: 1654.8. Samples: 21559812. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:21:58,345][60425] Avg episode reward: [(0, '70.920'), (1, '59.220')] [2023-10-14 19:22:00,264][61552] Updated weights for policy 0, policy_version 42182 (0.0009) [2023-10-14 19:22:00,638][61552] Updated weights for policy 0, policy_version 42192 (0.0009) [2023-10-14 19:22:01,009][61552] Updated weights for policy 0, policy_version 42202 (0.0008) [2023-10-14 19:22:01,809][61585] Updated weights for policy 1, policy_version 42020 (0.0009) [2023-10-14 19:22:02,183][61585] Updated weights for policy 1, policy_version 42030 (0.0010) [2023-10-14 19:22:02,548][61585] Updated weights for policy 1, policy_version 42040 (0.0009) [2023-10-14 19:22:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 86278144. Throughput: 0: 1668.7, 1: 1669.0. Samples: 21570406. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:22:03,344][60425] Avg episode reward: [(0, '72.020'), (1, '61.880')] [2023-10-14 19:22:03,345][61172] Saving new best policy, reward=72.020! [2023-10-14 19:22:05,043][61552] Updated weights for policy 0, policy_version 42212 (0.0007) [2023-10-14 19:22:05,409][61552] Updated weights for policy 0, policy_version 42222 (0.0008) [2023-10-14 19:22:05,780][61552] Updated weights for policy 0, policy_version 42232 (0.0009) [2023-10-14 19:22:06,638][61585] Updated weights for policy 1, policy_version 42050 (0.0008) [2023-10-14 19:22:07,004][61585] Updated weights for policy 1, policy_version 42060 (0.0007) [2023-10-14 19:22:07,367][61585] Updated weights for policy 1, policy_version 42070 (0.0010) [2023-10-14 19:22:07,731][61585] Updated weights for policy 1, policy_version 42080 (0.0012) [2023-10-14 19:22:08,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 86343680. Throughput: 0: 1678.5, 1: 1660.2. Samples: 21590132. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-10-14 19:22:08,344][60425] Avg episode reward: [(0, '72.580'), (1, '63.070')] [2023-10-14 19:22:08,345][61172] Saving new best policy, reward=72.580! [2023-10-14 19:22:09,918][61552] Updated weights for policy 0, policy_version 42242 (0.0007) [2023-10-14 19:22:10,335][61552] Updated weights for policy 0, policy_version 42252 (0.0007) [2023-10-14 19:22:10,715][61552] Updated weights for policy 0, policy_version 42262 (0.0009) [2023-10-14 19:22:11,078][61552] Updated weights for policy 0, policy_version 42272 (0.0009) [2023-10-14 19:22:11,801][61585] Updated weights for policy 1, policy_version 42090 (0.0009) [2023-10-14 19:22:12,167][61585] Updated weights for policy 1, policy_version 42100 (0.0008) [2023-10-14 19:22:12,532][61585] Updated weights for policy 1, policy_version 42110 (0.0009) [2023-10-14 19:22:13,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 86409216. Throughput: 0: 1690.6, 1: 1651.3. Samples: 21609836. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-10-14 19:22:13,344][60425] Avg episode reward: [(0, '70.270'), (1, '60.270')] [2023-10-14 19:22:15,104][61552] Updated weights for policy 0, policy_version 42282 (0.0009) [2023-10-14 19:22:15,472][61552] Updated weights for policy 0, policy_version 42292 (0.0010) [2023-10-14 19:22:15,828][61552] Updated weights for policy 0, policy_version 42302 (0.0009) [2023-10-14 19:22:16,754][61585] Updated weights for policy 1, policy_version 42120 (0.0011) [2023-10-14 19:22:17,132][61585] Updated weights for policy 1, policy_version 42130 (0.0010) [2023-10-14 19:22:17,484][61585] Updated weights for policy 1, policy_version 42140 (0.0010) [2023-10-14 19:22:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 86474752. Throughput: 0: 1673.7, 1: 1662.9. Samples: 21620436. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-10-14 19:22:18,344][60425] Avg episode reward: [(0, '68.810'), (1, '68.350')] [2023-10-14 19:22:18,344][61248] Saving new best policy, reward=68.350! [2023-10-14 19:22:19,883][61552] Updated weights for policy 0, policy_version 42312 (0.0010) [2023-10-14 19:22:20,256][61552] Updated weights for policy 0, policy_version 42322 (0.0007) [2023-10-14 19:22:20,627][61552] Updated weights for policy 0, policy_version 42332 (0.0007) [2023-10-14 19:22:21,700][61585] Updated weights for policy 1, policy_version 42150 (0.0010) [2023-10-14 19:22:22,088][61585] Updated weights for policy 1, policy_version 42160 (0.0009) [2023-10-14 19:22:22,453][61585] Updated weights for policy 1, policy_version 42170 (0.0010) [2023-10-14 19:22:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 86540288. Throughput: 0: 1689.2, 1: 1655.7. Samples: 21640384. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-10-14 19:22:23,344][60425] Avg episode reward: [(0, '68.270'), (1, '63.620')] [2023-10-14 19:22:24,572][61552] Updated weights for policy 0, policy_version 42342 (0.0007) [2023-10-14 19:22:24,942][61552] Updated weights for policy 0, policy_version 42352 (0.0009) [2023-10-14 19:22:25,297][61552] Updated weights for policy 0, policy_version 42362 (0.0007) [2023-10-14 19:22:26,486][61585] Updated weights for policy 1, policy_version 42180 (0.0008) [2023-10-14 19:22:26,849][61585] Updated weights for policy 1, policy_version 42190 (0.0009) [2023-10-14 19:22:27,221][61585] Updated weights for policy 1, policy_version 42200 (0.0007) [2023-10-14 19:22:28,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13329.3). Total num frames: 86605824. Throughput: 0: 1698.0, 1: 1650.7. Samples: 21660080. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-10-14 19:22:28,345][60425] Avg episode reward: [(0, '69.310'), (1, '65.020')] [2023-10-14 19:22:29,404][61552] Updated weights for policy 0, policy_version 42372 (0.0009) [2023-10-14 19:22:29,780][61552] Updated weights for policy 0, policy_version 42382 (0.0011) [2023-10-14 19:22:30,132][61552] Updated weights for policy 0, policy_version 42392 (0.0010) [2023-10-14 19:22:31,409][61585] Updated weights for policy 1, policy_version 42210 (0.0009) [2023-10-14 19:22:31,769][61585] Updated weights for policy 1, policy_version 42220 (0.0007) [2023-10-14 19:22:32,132][61585] Updated weights for policy 1, policy_version 42230 (0.0007) [2023-10-14 19:22:32,507][61585] Updated weights for policy 1, policy_version 42240 (0.0007) [2023-10-14 19:22:33,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 86671360. Throughput: 0: 1671.5, 1: 1659.9. Samples: 21670504. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-10-14 19:22:33,344][60425] Avg episode reward: [(0, '73.660'), (1, '65.980')] [2023-10-14 19:22:33,345][61172] Saving new best policy, reward=73.660! [2023-10-14 19:22:34,208][61552] Updated weights for policy 0, policy_version 42402 (0.0009) [2023-10-14 19:22:34,580][61552] Updated weights for policy 0, policy_version 42412 (0.0007) [2023-10-14 19:22:34,943][61552] Updated weights for policy 0, policy_version 42422 (0.0009) [2023-10-14 19:22:35,315][61552] Updated weights for policy 0, policy_version 42432 (0.0009) [2023-10-14 19:22:36,623][61585] Updated weights for policy 1, policy_version 42250 (0.0008) [2023-10-14 19:22:36,995][61585] Updated weights for policy 1, policy_version 42260 (0.0009) [2023-10-14 19:22:37,353][61585] Updated weights for policy 1, policy_version 42270 (0.0009) [2023-10-14 19:22:38,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 86736896. Throughput: 0: 1693.7, 1: 1652.1. Samples: 21690532. Policy #0 lag: (min: 34.0, avg: 54.9, max: 56.0) [2023-10-14 19:22:38,344][60425] Avg episode reward: [(0, '69.120'), (1, '63.270')] [2023-10-14 19:22:39,402][61552] Updated weights for policy 0, policy_version 42442 (0.0007) [2023-10-14 19:22:39,764][61552] Updated weights for policy 0, policy_version 42452 (0.0010) [2023-10-14 19:22:40,130][61552] Updated weights for policy 0, policy_version 42462 (0.0007) [2023-10-14 19:22:41,348][61585] Updated weights for policy 1, policy_version 42280 (0.0008) [2023-10-14 19:22:41,713][61585] Updated weights for policy 1, policy_version 42290 (0.0007) [2023-10-14 19:22:42,075][61585] Updated weights for policy 1, policy_version 42300 (0.0009) [2023-10-14 19:22:43,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 86802432. Throughput: 0: 1688.6, 1: 1661.2. Samples: 21710550. Policy #0 lag: (min: 34.0, avg: 54.9, max: 56.0) [2023-10-14 19:22:43,345][60425] Avg episode reward: [(0, '68.260'), (1, '66.000')] [2023-10-14 19:22:44,260][61552] Updated weights for policy 0, policy_version 42472 (0.0008) [2023-10-14 19:22:44,627][61552] Updated weights for policy 0, policy_version 42482 (0.0008) [2023-10-14 19:22:45,006][61552] Updated weights for policy 0, policy_version 42492 (0.0009) [2023-10-14 19:22:46,150][61585] Updated weights for policy 1, policy_version 42310 (0.0010) [2023-10-14 19:22:46,505][61585] Updated weights for policy 1, policy_version 42320 (0.0009) [2023-10-14 19:22:46,868][61585] Updated weights for policy 1, policy_version 42330 (0.0009) [2023-10-14 19:22:48,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 86867968. Throughput: 0: 1675.5, 1: 1670.6. Samples: 21720980. Policy #0 lag: (min: 34.0, avg: 54.9, max: 56.0) [2023-10-14 19:22:48,344][60425] Avg episode reward: [(0, '69.180'), (1, '62.450')] [2023-10-14 19:22:49,176][61552] Updated weights for policy 0, policy_version 42502 (0.0008) [2023-10-14 19:22:49,546][61552] Updated weights for policy 0, policy_version 42512 (0.0009) [2023-10-14 19:22:49,927][61552] Updated weights for policy 0, policy_version 42522 (0.0010) [2023-10-14 19:22:50,971][61585] Updated weights for policy 1, policy_version 42340 (0.0009) [2023-10-14 19:22:51,330][61585] Updated weights for policy 1, policy_version 42350 (0.0010) [2023-10-14 19:22:51,696][61585] Updated weights for policy 1, policy_version 42360 (0.0009) [2023-10-14 19:22:53,344][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 86933504. Throughput: 0: 1687.3, 1: 1655.9. Samples: 21740576. Policy #0 lag: (min: 34.0, avg: 54.9, max: 56.0) [2023-10-14 19:22:53,345][60425] Avg episode reward: [(0, '70.310'), (1, '58.630')] [2023-10-14 19:22:53,985][61552] Updated weights for policy 0, policy_version 42532 (0.0009) [2023-10-14 19:22:54,360][61552] Updated weights for policy 0, policy_version 42542 (0.0009) [2023-10-14 19:22:54,725][61552] Updated weights for policy 0, policy_version 42552 (0.0009) [2023-10-14 19:22:55,574][61585] Updated weights for policy 1, policy_version 42370 (0.0007) [2023-10-14 19:22:55,936][61585] Updated weights for policy 1, policy_version 42380 (0.0008) [2023-10-14 19:22:56,311][61585] Updated weights for policy 1, policy_version 42390 (0.0008) [2023-10-14 19:22:56,673][61585] Updated weights for policy 1, policy_version 42400 (0.0008) [2023-10-14 19:22:58,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 86999040. Throughput: 0: 1688.6, 1: 1672.4. Samples: 21761080. Policy #0 lag: (min: 34.0, avg: 54.9, max: 56.0) [2023-10-14 19:22:58,344][60425] Avg episode reward: [(0, '68.370'), (1, '62.880')] [2023-10-14 19:22:58,793][61552] Updated weights for policy 0, policy_version 42562 (0.0009) [2023-10-14 19:22:59,208][61552] Updated weights for policy 0, policy_version 42572 (0.0007) [2023-10-14 19:22:59,573][61552] Updated weights for policy 0, policy_version 42582 (0.0008) [2023-10-14 19:22:59,943][61552] Updated weights for policy 0, policy_version 42592 (0.0007) [2023-10-14 19:23:00,861][61585] Updated weights for policy 1, policy_version 42410 (0.0009) [2023-10-14 19:23:01,228][61585] Updated weights for policy 1, policy_version 42420 (0.0007) [2023-10-14 19:23:01,594][61585] Updated weights for policy 1, policy_version 42430 (0.0009) [2023-10-14 19:23:03,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 87064576. Throughput: 0: 1675.2, 1: 1668.2. Samples: 21770886. Policy #0 lag: (min: 34.0, avg: 54.9, max: 56.0) [2023-10-14 19:23:03,344][60425] Avg episode reward: [(0, '69.550'), (1, '62.080')] [2023-10-14 19:23:03,911][61552] Updated weights for policy 0, policy_version 42602 (0.0008) [2023-10-14 19:23:04,276][61552] Updated weights for policy 0, policy_version 42612 (0.0008) [2023-10-14 19:23:04,640][61552] Updated weights for policy 0, policy_version 42622 (0.0009) [2023-10-14 19:23:05,760][61585] Updated weights for policy 1, policy_version 42440 (0.0008) [2023-10-14 19:23:06,132][61585] Updated weights for policy 1, policy_version 42450 (0.0007) [2023-10-14 19:23:06,493][61585] Updated weights for policy 1, policy_version 42460 (0.0007) [2023-10-14 19:23:08,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 87130112. Throughput: 0: 1681.8, 1: 1656.4. Samples: 21790600. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:23:08,344][60425] Avg episode reward: [(0, '65.400'), (1, '64.000')] [2023-10-14 19:23:08,687][61552] Updated weights for policy 0, policy_version 42632 (0.0008) [2023-10-14 19:23:09,051][61552] Updated weights for policy 0, policy_version 42642 (0.0007) [2023-10-14 19:23:09,423][61552] Updated weights for policy 0, policy_version 42652 (0.0007) [2023-10-14 19:23:10,618][61585] Updated weights for policy 1, policy_version 42470 (0.0007) [2023-10-14 19:23:11,002][61585] Updated weights for policy 1, policy_version 42480 (0.0011) [2023-10-14 19:23:11,367][61585] Updated weights for policy 1, policy_version 42490 (0.0012) [2023-10-14 19:23:13,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 87195648. Throughput: 0: 1676.8, 1: 1678.0. Samples: 21811046. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:23:13,344][60425] Avg episode reward: [(0, '67.580'), (1, '65.570')] [2023-10-14 19:23:13,516][61552] Updated weights for policy 0, policy_version 42662 (0.0008) [2023-10-14 19:23:13,884][61552] Updated weights for policy 0, policy_version 42672 (0.0009) [2023-10-14 19:23:14,253][61552] Updated weights for policy 0, policy_version 42682 (0.0010) [2023-10-14 19:23:15,444][61585] Updated weights for policy 1, policy_version 42500 (0.0011) [2023-10-14 19:23:15,814][61585] Updated weights for policy 1, policy_version 42510 (0.0009) [2023-10-14 19:23:16,183][61585] Updated weights for policy 1, policy_version 42520 (0.0009) [2023-10-14 19:23:18,085][61552] Updated weights for policy 0, policy_version 42692 (0.0008) [2023-10-14 19:23:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 87261184. Throughput: 0: 1673.7, 1: 1668.6. Samples: 21820908. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:23:18,344][60425] Avg episode reward: [(0, '67.600'), (1, '63.740')] [2023-10-14 19:23:18,451][61552] Updated weights for policy 0, policy_version 42702 (0.0010) [2023-10-14 19:23:18,825][61552] Updated weights for policy 0, policy_version 42712 (0.0007) [2023-10-14 19:23:20,326][61585] Updated weights for policy 1, policy_version 42530 (0.0008) [2023-10-14 19:23:20,696][61585] Updated weights for policy 1, policy_version 42540 (0.0009) [2023-10-14 19:23:21,065][61585] Updated weights for policy 1, policy_version 42550 (0.0010) [2023-10-14 19:23:21,428][61585] Updated weights for policy 1, policy_version 42560 (0.0009) [2023-10-14 19:23:22,985][61552] Updated weights for policy 0, policy_version 42722 (0.0010) [2023-10-14 19:23:23,344][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 87326720. Throughput: 0: 1679.7, 1: 1658.2. Samples: 21840738. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:23:23,345][60425] Avg episode reward: [(0, '68.030'), (1, '61.990')] [2023-10-14 19:23:23,357][61552] Updated weights for policy 0, policy_version 42732 (0.0010) [2023-10-14 19:23:23,714][61552] Updated weights for policy 0, policy_version 42742 (0.0009) [2023-10-14 19:23:24,077][61552] Updated weights for policy 0, policy_version 42752 (0.0010) [2023-10-14 19:23:25,718][61585] Updated weights for policy 1, policy_version 42570 (0.0008) [2023-10-14 19:23:26,093][61585] Updated weights for policy 1, policy_version 42580 (0.0007) [2023-10-14 19:23:26,449][61585] Updated weights for policy 1, policy_version 42590 (0.0007) [2023-10-14 19:23:28,200][61552] Updated weights for policy 0, policy_version 42762 (0.0007) [2023-10-14 19:23:28,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 87392256. Throughput: 0: 1677.8, 1: 1665.3. Samples: 21860992. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:23:28,344][60425] Avg episode reward: [(0, '67.250'), (1, '65.590')] [2023-10-14 19:23:28,566][61552] Updated weights for policy 0, policy_version 42772 (0.0007) [2023-10-14 19:23:28,931][61552] Updated weights for policy 0, policy_version 42782 (0.0009) [2023-10-14 19:23:30,607][61585] Updated weights for policy 1, policy_version 42600 (0.0008) [2023-10-14 19:23:30,962][61585] Updated weights for policy 1, policy_version 42610 (0.0009) [2023-10-14 19:23:31,333][61585] Updated weights for policy 1, policy_version 42620 (0.0009) [2023-10-14 19:23:32,914][61552] Updated weights for policy 0, policy_version 42792 (0.0008) [2023-10-14 19:23:33,288][61552] Updated weights for policy 0, policy_version 42802 (0.0008) [2023-10-14 19:23:33,343][60425] Fps is (10 sec: 13107.7, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 87457792. Throughput: 0: 1678.2, 1: 1653.9. Samples: 21870926. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:23:33,344][60425] Avg episode reward: [(0, '68.450'), (1, '61.520')] [2023-10-14 19:23:33,656][61552] Updated weights for policy 0, policy_version 42812 (0.0009) [2023-10-14 19:23:35,355][61585] Updated weights for policy 1, policy_version 42630 (0.0008) [2023-10-14 19:23:35,722][61585] Updated weights for policy 1, policy_version 42640 (0.0007) [2023-10-14 19:23:36,075][61585] Updated weights for policy 1, policy_version 42650 (0.0007) [2023-10-14 19:23:37,817][61552] Updated weights for policy 0, policy_version 42822 (0.0008) [2023-10-14 19:23:38,182][61552] Updated weights for policy 0, policy_version 42832 (0.0007) [2023-10-14 19:23:38,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 87523328. Throughput: 0: 1688.3, 1: 1658.1. Samples: 21891166. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:23:38,344][60425] Avg episode reward: [(0, '68.940'), (1, '64.800')] [2023-10-14 19:23:38,553][61552] Updated weights for policy 0, policy_version 42842 (0.0007) [2023-10-14 19:23:40,271][61585] Updated weights for policy 1, policy_version 42660 (0.0009) [2023-10-14 19:23:40,637][61585] Updated weights for policy 1, policy_version 42670 (0.0011) [2023-10-14 19:23:41,010][61585] Updated weights for policy 1, policy_version 42680 (0.0011) [2023-10-14 19:23:42,662][61552] Updated weights for policy 0, policy_version 42852 (0.0007) [2023-10-14 19:23:43,040][61552] Updated weights for policy 0, policy_version 42862 (0.0007) [2023-10-14 19:23:43,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 87588864. Throughput: 0: 1679.9, 1: 1662.8. Samples: 21911500. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:23:43,344][60425] Avg episode reward: [(0, '68.810'), (1, '66.690')] [2023-10-14 19:23:43,351][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000042688_43712512.pth... [2023-10-14 19:23:43,392][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000041120_42106880.pth [2023-10-14 19:23:43,408][61552] Updated weights for policy 0, policy_version 42872 (0.0008) [2023-10-14 19:23:43,700][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000042880_43909120.pth... [2023-10-14 19:23:43,728][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000041312_42303488.pth [2023-10-14 19:23:45,111][61585] Updated weights for policy 1, policy_version 42690 (0.0009) [2023-10-14 19:23:45,481][61585] Updated weights for policy 1, policy_version 42700 (0.0007) [2023-10-14 19:23:45,838][61585] Updated weights for policy 1, policy_version 42710 (0.0008) [2023-10-14 19:23:46,203][61585] Updated weights for policy 1, policy_version 42720 (0.0009) [2023-10-14 19:23:47,708][61552] Updated weights for policy 0, policy_version 42882 (0.0008) [2023-10-14 19:23:48,120][61552] Updated weights for policy 0, policy_version 42892 (0.0007) [2023-10-14 19:23:48,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 87654400. Throughput: 0: 1687.7, 1: 1653.8. Samples: 21921254. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:23:48,344][60425] Avg episode reward: [(0, '68.550'), (1, '61.780')] [2023-10-14 19:23:48,492][61552] Updated weights for policy 0, policy_version 42902 (0.0008) [2023-10-14 19:23:48,864][61552] Updated weights for policy 0, policy_version 42912 (0.0008) [2023-10-14 19:23:50,357][61585] Updated weights for policy 1, policy_version 42730 (0.0007) [2023-10-14 19:23:50,732][61585] Updated weights for policy 1, policy_version 42740 (0.0007) [2023-10-14 19:23:51,099][61585] Updated weights for policy 1, policy_version 42750 (0.0008) [2023-10-14 19:23:52,847][61552] Updated weights for policy 0, policy_version 42922 (0.0008) [2023-10-14 19:23:53,224][61552] Updated weights for policy 0, policy_version 42932 (0.0008) [2023-10-14 19:23:53,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 87719936. Throughput: 0: 1680.7, 1: 1663.9. Samples: 21941106. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:23:53,344][60425] Avg episode reward: [(0, '67.140'), (1, '64.740')] [2023-10-14 19:23:53,594][61552] Updated weights for policy 0, policy_version 42942 (0.0009) [2023-10-14 19:23:55,108][61585] Updated weights for policy 1, policy_version 42760 (0.0010) [2023-10-14 19:23:55,475][61585] Updated weights for policy 1, policy_version 42770 (0.0008) [2023-10-14 19:23:55,845][61585] Updated weights for policy 1, policy_version 42780 (0.0008) [2023-10-14 19:23:57,716][61552] Updated weights for policy 0, policy_version 42952 (0.0009) [2023-10-14 19:23:58,098][61552] Updated weights for policy 0, policy_version 42962 (0.0009) [2023-10-14 19:23:58,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 87785472. Throughput: 0: 1675.8, 1: 1670.1. Samples: 21961610. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:23:58,344][60425] Avg episode reward: [(0, '67.650'), (1, '65.290')] [2023-10-14 19:23:58,467][61552] Updated weights for policy 0, policy_version 42972 (0.0010) [2023-10-14 19:23:59,913][61585] Updated weights for policy 1, policy_version 42790 (0.0007) [2023-10-14 19:24:00,290][61585] Updated weights for policy 1, policy_version 42800 (0.0007) [2023-10-14 19:24:00,657][61585] Updated weights for policy 1, policy_version 42810 (0.0008) [2023-10-14 19:24:02,540][61552] Updated weights for policy 0, policy_version 42982 (0.0010) [2023-10-14 19:24:02,910][61552] Updated weights for policy 0, policy_version 42992 (0.0010) [2023-10-14 19:24:03,281][61552] Updated weights for policy 0, policy_version 43002 (0.0010) [2023-10-14 19:24:03,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 87851008. Throughput: 0: 1682.9, 1: 1658.7. Samples: 21971278. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:24:03,344][60425] Avg episode reward: [(0, '68.380'), (1, '62.140')] [2023-10-14 19:24:04,628][61585] Updated weights for policy 1, policy_version 42820 (0.0009) [2023-10-14 19:24:05,008][61585] Updated weights for policy 1, policy_version 42830 (0.0009) [2023-10-14 19:24:05,372][61585] Updated weights for policy 1, policy_version 42840 (0.0009) [2023-10-14 19:24:07,155][61552] Updated weights for policy 0, policy_version 43012 (0.0008) [2023-10-14 19:24:07,529][61552] Updated weights for policy 0, policy_version 43022 (0.0008) [2023-10-14 19:24:07,900][61552] Updated weights for policy 0, policy_version 43032 (0.0007) [2023-10-14 19:24:08,343][60425] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 87949312. Throughput: 0: 1679.4, 1: 1673.6. Samples: 21991620. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 19:24:08,344][60425] Avg episode reward: [(0, '64.670'), (1, '63.980')] [2023-10-14 19:24:09,455][61585] Updated weights for policy 1, policy_version 42850 (0.0008) [2023-10-14 19:24:09,815][61585] Updated weights for policy 1, policy_version 42860 (0.0008) [2023-10-14 19:24:10,189][61585] Updated weights for policy 1, policy_version 42870 (0.0010) [2023-10-14 19:24:10,560][61585] Updated weights for policy 1, policy_version 42880 (0.0008) [2023-10-14 19:24:11,951][61552] Updated weights for policy 0, policy_version 43042 (0.0008) [2023-10-14 19:24:12,317][61552] Updated weights for policy 0, policy_version 43052 (0.0010) [2023-10-14 19:24:12,688][61552] Updated weights for policy 0, policy_version 43062 (0.0008) [2023-10-14 19:24:13,062][61552] Updated weights for policy 0, policy_version 43072 (0.0008) [2023-10-14 19:24:13,343][60425] Fps is (10 sec: 16383.8, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 88014848. Throughput: 0: 1658.0, 1: 1685.3. Samples: 22011442. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 19:24:13,344][60425] Avg episode reward: [(0, '65.840'), (1, '60.240')] [2023-10-14 19:24:14,541][61585] Updated weights for policy 1, policy_version 42890 (0.0009) [2023-10-14 19:24:14,905][61585] Updated weights for policy 1, policy_version 42900 (0.0008) [2023-10-14 19:24:15,275][61585] Updated weights for policy 1, policy_version 42910 (0.0009) [2023-10-14 19:24:17,231][61552] Updated weights for policy 0, policy_version 43082 (0.0011) [2023-10-14 19:24:17,588][61552] Updated weights for policy 0, policy_version 43092 (0.0008) [2023-10-14 19:24:17,960][61552] Updated weights for policy 0, policy_version 43102 (0.0009) [2023-10-14 19:24:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 88080384. Throughput: 0: 1675.5, 1: 1667.5. Samples: 22021360. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 19:24:18,344][60425] Avg episode reward: [(0, '66.040'), (1, '66.260')] [2023-10-14 19:24:19,254][61585] Updated weights for policy 1, policy_version 42920 (0.0008) [2023-10-14 19:24:19,627][61585] Updated weights for policy 1, policy_version 42930 (0.0009) [2023-10-14 19:24:19,986][61585] Updated weights for policy 1, policy_version 42940 (0.0007) [2023-10-14 19:24:22,109][61552] Updated weights for policy 0, policy_version 43112 (0.0008) [2023-10-14 19:24:22,477][61552] Updated weights for policy 0, policy_version 43122 (0.0008) [2023-10-14 19:24:22,847][61552] Updated weights for policy 0, policy_version 43132 (0.0007) [2023-10-14 19:24:23,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 88145920. Throughput: 0: 1662.5, 1: 1687.5. Samples: 22041914. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 19:24:23,344][60425] Avg episode reward: [(0, '67.010'), (1, '60.170')] [2023-10-14 19:24:24,056][61585] Updated weights for policy 1, policy_version 42950 (0.0008) [2023-10-14 19:24:24,411][61585] Updated weights for policy 1, policy_version 42960 (0.0011) [2023-10-14 19:24:24,781][61585] Updated weights for policy 1, policy_version 42970 (0.0010) [2023-10-14 19:24:27,111][61552] Updated weights for policy 0, policy_version 43142 (0.0008) [2023-10-14 19:24:27,481][61552] Updated weights for policy 0, policy_version 43152 (0.0008) [2023-10-14 19:24:27,840][61552] Updated weights for policy 0, policy_version 43162 (0.0009) [2023-10-14 19:24:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 88211456. Throughput: 0: 1654.2, 1: 1687.6. Samples: 22061880. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 19:24:28,345][60425] Avg episode reward: [(0, '66.250'), (1, '63.850')] [2023-10-14 19:24:29,035][61585] Updated weights for policy 1, policy_version 42980 (0.0007) [2023-10-14 19:24:29,394][61585] Updated weights for policy 1, policy_version 42990 (0.0007) [2023-10-14 19:24:29,759][61585] Updated weights for policy 1, policy_version 43000 (0.0009) [2023-10-14 19:24:31,730][61552] Updated weights for policy 0, policy_version 43172 (0.0008) [2023-10-14 19:24:32,100][61552] Updated weights for policy 0, policy_version 43182 (0.0010) [2023-10-14 19:24:32,468][61552] Updated weights for policy 0, policy_version 43192 (0.0007) [2023-10-14 19:24:33,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 88276992. Throughput: 0: 1672.6, 1: 1675.8. Samples: 22071932. Policy #0 lag: (min: 31.0, avg: 32.9, max: 61.0) [2023-10-14 19:24:33,345][60425] Avg episode reward: [(0, '69.620'), (1, '69.120')] [2023-10-14 19:24:33,346][61248] Saving new best policy, reward=69.120! [2023-10-14 19:24:33,873][61585] Updated weights for policy 1, policy_version 43010 (0.0008) [2023-10-14 19:24:34,242][61585] Updated weights for policy 1, policy_version 43020 (0.0009) [2023-10-14 19:24:34,597][61585] Updated weights for policy 1, policy_version 43030 (0.0009) [2023-10-14 19:24:34,962][61585] Updated weights for policy 1, policy_version 43040 (0.0010) [2023-10-14 19:24:36,630][61552] Updated weights for policy 0, policy_version 43202 (0.0010) [2023-10-14 19:24:37,024][61552] Updated weights for policy 0, policy_version 43212 (0.0008) [2023-10-14 19:24:37,381][61552] Updated weights for policy 0, policy_version 43222 (0.0008) [2023-10-14 19:24:37,747][61552] Updated weights for policy 0, policy_version 43232 (0.0007) [2023-10-14 19:24:38,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 88342528. Throughput: 0: 1673.2, 1: 1683.6. Samples: 22092162. Policy #0 lag: (min: 31.0, avg: 32.9, max: 61.0) [2023-10-14 19:24:38,344][60425] Avg episode reward: [(0, '69.660'), (1, '65.400')] [2023-10-14 19:24:39,139][61585] Updated weights for policy 1, policy_version 43050 (0.0007) [2023-10-14 19:24:39,510][61585] Updated weights for policy 1, policy_version 43060 (0.0008) [2023-10-14 19:24:39,866][61585] Updated weights for policy 1, policy_version 43070 (0.0007) [2023-10-14 19:24:41,918][61552] Updated weights for policy 0, policy_version 43242 (0.0011) [2023-10-14 19:24:42,295][61552] Updated weights for policy 0, policy_version 43252 (0.0010) [2023-10-14 19:24:42,649][61552] Updated weights for policy 0, policy_version 43262 (0.0011) [2023-10-14 19:24:43,344][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 88408064. Throughput: 0: 1653.2, 1: 1687.5. Samples: 22111942. Policy #0 lag: (min: 31.0, avg: 32.9, max: 61.0) [2023-10-14 19:24:43,345][60425] Avg episode reward: [(0, '62.580'), (1, '61.820')] [2023-10-14 19:24:43,763][61585] Updated weights for policy 1, policy_version 43080 (0.0007) [2023-10-14 19:24:44,131][61585] Updated weights for policy 1, policy_version 43090 (0.0008) [2023-10-14 19:24:44,493][61585] Updated weights for policy 1, policy_version 43100 (0.0008) [2023-10-14 19:24:46,811][61552] Updated weights for policy 0, policy_version 43272 (0.0008) [2023-10-14 19:24:47,178][61552] Updated weights for policy 0, policy_version 43282 (0.0008) [2023-10-14 19:24:47,556][61552] Updated weights for policy 0, policy_version 43292 (0.0009) [2023-10-14 19:24:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 88473600. Throughput: 0: 1673.2, 1: 1681.7. Samples: 22122250. Policy #0 lag: (min: 31.0, avg: 32.9, max: 61.0) [2023-10-14 19:24:48,344][60425] Avg episode reward: [(0, '66.030'), (1, '65.000')] [2023-10-14 19:24:48,767][61585] Updated weights for policy 1, policy_version 43110 (0.0008) [2023-10-14 19:24:49,133][61585] Updated weights for policy 1, policy_version 43120 (0.0009) [2023-10-14 19:24:49,503][61585] Updated weights for policy 1, policy_version 43130 (0.0008) [2023-10-14 19:24:51,463][61552] Updated weights for policy 0, policy_version 43302 (0.0009) [2023-10-14 19:24:51,822][61552] Updated weights for policy 0, policy_version 43312 (0.0010) [2023-10-14 19:24:52,205][61552] Updated weights for policy 0, policy_version 43322 (0.0009) [2023-10-14 19:24:53,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 88539136. Throughput: 0: 1661.2, 1: 1686.8. Samples: 22142282. Policy #0 lag: (min: 31.0, avg: 32.9, max: 61.0) [2023-10-14 19:24:53,344][60425] Avg episode reward: [(0, '70.720'), (1, '61.810')] [2023-10-14 19:24:53,466][61585] Updated weights for policy 1, policy_version 43140 (0.0009) [2023-10-14 19:24:53,820][61585] Updated weights for policy 1, policy_version 43150 (0.0008) [2023-10-14 19:24:54,187][61585] Updated weights for policy 1, policy_version 43160 (0.0008) [2023-10-14 19:24:56,293][61552] Updated weights for policy 0, policy_version 43332 (0.0008) [2023-10-14 19:24:56,667][61552] Updated weights for policy 0, policy_version 43342 (0.0009) [2023-10-14 19:24:57,026][61552] Updated weights for policy 0, policy_version 43352 (0.0009) [2023-10-14 19:24:58,139][61585] Updated weights for policy 1, policy_version 43170 (0.0009) [2023-10-14 19:24:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 88604672. Throughput: 0: 1663.6, 1: 1686.0. Samples: 22162176. Policy #0 lag: (min: 31.0, avg: 32.9, max: 61.0) [2023-10-14 19:24:58,344][60425] Avg episode reward: [(0, '69.040'), (1, '63.020')] [2023-10-14 19:24:58,503][61585] Updated weights for policy 1, policy_version 43180 (0.0008) [2023-10-14 19:24:58,867][61585] Updated weights for policy 1, policy_version 43190 (0.0008) [2023-10-14 19:24:59,236][61585] Updated weights for policy 1, policy_version 43200 (0.0008) [2023-10-14 19:25:01,061][61552] Updated weights for policy 0, policy_version 43362 (0.0008) [2023-10-14 19:25:01,429][61552] Updated weights for policy 0, policy_version 43372 (0.0008) [2023-10-14 19:25:01,788][61552] Updated weights for policy 0, policy_version 43382 (0.0009) [2023-10-14 19:25:02,161][61552] Updated weights for policy 0, policy_version 43392 (0.0008) [2023-10-14 19:25:03,237][61585] Updated weights for policy 1, policy_version 43210 (0.0007) [2023-10-14 19:25:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 88670208. Throughput: 0: 1673.0, 1: 1683.8. Samples: 22172416. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:25:03,344][60425] Avg episode reward: [(0, '68.560'), (1, '65.560')] [2023-10-14 19:25:03,604][61585] Updated weights for policy 1, policy_version 43220 (0.0007) [2023-10-14 19:25:03,967][61585] Updated weights for policy 1, policy_version 43230 (0.0010) [2023-10-14 19:25:06,136][61552] Updated weights for policy 0, policy_version 43402 (0.0010) [2023-10-14 19:25:06,495][61552] Updated weights for policy 0, policy_version 43412 (0.0010) [2023-10-14 19:25:06,860][61552] Updated weights for policy 0, policy_version 43422 (0.0011) [2023-10-14 19:25:08,260][61585] Updated weights for policy 1, policy_version 43240 (0.0008) [2023-10-14 19:25:08,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 88735744. Throughput: 0: 1655.9, 1: 1686.7. Samples: 22192330. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:25:08,344][60425] Avg episode reward: [(0, '71.400'), (1, '61.450')] [2023-10-14 19:25:08,630][61585] Updated weights for policy 1, policy_version 43250 (0.0007) [2023-10-14 19:25:08,997][61585] Updated weights for policy 1, policy_version 43260 (0.0007) [2023-10-14 19:25:11,061][61552] Updated weights for policy 0, policy_version 43432 (0.0010) [2023-10-14 19:25:11,421][61552] Updated weights for policy 0, policy_version 43442 (0.0008) [2023-10-14 19:25:11,786][61552] Updated weights for policy 0, policy_version 43452 (0.0009) [2023-10-14 19:25:13,069][61585] Updated weights for policy 1, policy_version 43270 (0.0007) [2023-10-14 19:25:13,344][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 88801280. Throughput: 0: 1663.5, 1: 1684.2. Samples: 22212524. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:25:13,345][60425] Avg episode reward: [(0, '68.940'), (1, '67.030')] [2023-10-14 19:25:13,438][61585] Updated weights for policy 1, policy_version 43280 (0.0008) [2023-10-14 19:25:13,811][61585] Updated weights for policy 1, policy_version 43290 (0.0008) [2023-10-14 19:25:15,901][61552] Updated weights for policy 0, policy_version 43462 (0.0009) [2023-10-14 19:25:16,277][61552] Updated weights for policy 0, policy_version 43472 (0.0007) [2023-10-14 19:25:16,650][61552] Updated weights for policy 0, policy_version 43482 (0.0010) [2023-10-14 19:25:17,837][61585] Updated weights for policy 1, policy_version 43300 (0.0010) [2023-10-14 19:25:18,204][61585] Updated weights for policy 1, policy_version 43310 (0.0009) [2023-10-14 19:25:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 88866816. Throughput: 0: 1671.2, 1: 1682.7. Samples: 22222854. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:25:18,344][60425] Avg episode reward: [(0, '69.890'), (1, '65.070')] [2023-10-14 19:25:18,567][61585] Updated weights for policy 1, policy_version 43320 (0.0010) [2023-10-14 19:25:20,791][61552] Updated weights for policy 0, policy_version 43492 (0.0007) [2023-10-14 19:25:21,156][61552] Updated weights for policy 0, policy_version 43502 (0.0009) [2023-10-14 19:25:21,521][61552] Updated weights for policy 0, policy_version 43512 (0.0009) [2023-10-14 19:25:22,682][61585] Updated weights for policy 1, policy_version 43330 (0.0010) [2023-10-14 19:25:23,047][61585] Updated weights for policy 1, policy_version 43340 (0.0010) [2023-10-14 19:25:23,343][60425] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 88932352. Throughput: 0: 1650.3, 1: 1689.3. Samples: 22242444. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:25:23,344][60425] Avg episode reward: [(0, '71.140'), (1, '64.790')] [2023-10-14 19:25:23,420][61585] Updated weights for policy 1, policy_version 43350 (0.0011) [2023-10-14 19:25:23,790][61585] Updated weights for policy 1, policy_version 43360 (0.0009) [2023-10-14 19:25:25,828][61552] Updated weights for policy 0, policy_version 43522 (0.0009) [2023-10-14 19:25:26,242][61552] Updated weights for policy 0, policy_version 43532 (0.0007) [2023-10-14 19:25:26,605][61552] Updated weights for policy 0, policy_version 43542 (0.0008) [2023-10-14 19:25:26,972][61552] Updated weights for policy 0, policy_version 43552 (0.0009) [2023-10-14 19:25:27,687][61585] Updated weights for policy 1, policy_version 43370 (0.0007) [2023-10-14 19:25:28,049][61585] Updated weights for policy 1, policy_version 43380 (0.0007) [2023-10-14 19:25:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 88997888. Throughput: 0: 1674.1, 1: 1677.3. Samples: 22262754. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:25:28,344][60425] Avg episode reward: [(0, '69.270'), (1, '70.790')] [2023-10-14 19:25:28,421][61585] Updated weights for policy 1, policy_version 43390 (0.0009) [2023-10-14 19:25:28,493][61248] Saving new best policy, reward=70.790! [2023-10-14 19:25:30,854][61552] Updated weights for policy 0, policy_version 43562 (0.0007) [2023-10-14 19:25:31,226][61552] Updated weights for policy 0, policy_version 43572 (0.0008) [2023-10-14 19:25:31,598][61552] Updated weights for policy 0, policy_version 43582 (0.0010) [2023-10-14 19:25:32,475][61585] Updated weights for policy 1, policy_version 43400 (0.0009) [2023-10-14 19:25:32,842][61585] Updated weights for policy 1, policy_version 43410 (0.0007) [2023-10-14 19:25:33,206][61585] Updated weights for policy 1, policy_version 43420 (0.0007) [2023-10-14 19:25:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 89063424. Throughput: 0: 1673.9, 1: 1687.8. Samples: 22273528. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) [2023-10-14 19:25:33,344][60425] Avg episode reward: [(0, '69.790'), (1, '65.640')] [2023-10-14 19:25:35,667][61552] Updated weights for policy 0, policy_version 43592 (0.0008) [2023-10-14 19:25:36,026][61552] Updated weights for policy 0, policy_version 43602 (0.0008) [2023-10-14 19:25:36,404][61552] Updated weights for policy 0, policy_version 43612 (0.0011) [2023-10-14 19:25:37,475][61585] Updated weights for policy 1, policy_version 43430 (0.0009) [2023-10-14 19:25:37,858][61585] Updated weights for policy 1, policy_version 43440 (0.0011) [2023-10-14 19:25:38,218][61585] Updated weights for policy 1, policy_version 43450 (0.0011) [2023-10-14 19:25:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 89128960. Throughput: 0: 1661.8, 1: 1690.9. Samples: 22293156. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) [2023-10-14 19:25:38,344][60425] Avg episode reward: [(0, '71.370'), (1, '63.800')] [2023-10-14 19:25:40,704][61552] Updated weights for policy 0, policy_version 43622 (0.0008) [2023-10-14 19:25:41,068][61552] Updated weights for policy 0, policy_version 43632 (0.0009) [2023-10-14 19:25:41,437][61552] Updated weights for policy 0, policy_version 43642 (0.0009) [2023-10-14 19:25:42,363][61585] Updated weights for policy 1, policy_version 43460 (0.0008) [2023-10-14 19:25:42,728][61585] Updated weights for policy 1, policy_version 43470 (0.0009) [2023-10-14 19:25:43,096][61585] Updated weights for policy 1, policy_version 43480 (0.0008) [2023-10-14 19:25:43,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 89194496. Throughput: 0: 1685.0, 1: 1669.7. Samples: 22313140. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) [2023-10-14 19:25:43,344][60425] Avg episode reward: [(0, '70.540'), (1, '65.990')] [2023-10-14 19:25:43,353][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000043648_44695552.pth... [2023-10-14 19:25:43,387][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000042080_43089920.pth [2023-10-14 19:25:43,388][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000043488_44531712.pth... [2023-10-14 19:25:43,426][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000041920_42926080.pth [2023-10-14 19:25:45,387][61552] Updated weights for policy 0, policy_version 43652 (0.0007) [2023-10-14 19:25:45,763][61552] Updated weights for policy 0, policy_version 43662 (0.0008) [2023-10-14 19:25:46,135][61552] Updated weights for policy 0, policy_version 43672 (0.0010) [2023-10-14 19:25:47,285][61585] Updated weights for policy 1, policy_version 43490 (0.0007) [2023-10-14 19:25:47,642][61585] Updated weights for policy 1, policy_version 43500 (0.0010) [2023-10-14 19:25:48,013][61585] Updated weights for policy 1, policy_version 43510 (0.0009) [2023-10-14 19:25:48,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 89260032. Throughput: 0: 1676.7, 1: 1683.3. Samples: 22323618. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) [2023-10-14 19:25:48,344][60425] Avg episode reward: [(0, '68.250'), (1, '62.780')] [2023-10-14 19:25:48,376][61585] Updated weights for policy 1, policy_version 43520 (0.0009) [2023-10-14 19:25:50,161][61552] Updated weights for policy 0, policy_version 43682 (0.0011) [2023-10-14 19:25:50,528][61552] Updated weights for policy 0, policy_version 43692 (0.0007) [2023-10-14 19:25:50,892][61552] Updated weights for policy 0, policy_version 43702 (0.0008) [2023-10-14 19:25:51,261][61552] Updated weights for policy 0, policy_version 43712 (0.0010) [2023-10-14 19:25:52,568][61585] Updated weights for policy 1, policy_version 43530 (0.0009) [2023-10-14 19:25:52,938][61585] Updated weights for policy 1, policy_version 43540 (0.0007) [2023-10-14 19:25:53,300][61585] Updated weights for policy 1, policy_version 43550 (0.0008) [2023-10-14 19:25:53,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 89325568. Throughput: 0: 1679.6, 1: 1676.1. Samples: 22343336. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) [2023-10-14 19:25:53,344][60425] Avg episode reward: [(0, '66.630'), (1, '65.690')] [2023-10-14 19:25:55,349][61552] Updated weights for policy 0, policy_version 43722 (0.0010) [2023-10-14 19:25:55,719][61552] Updated weights for policy 0, policy_version 43732 (0.0009) [2023-10-14 19:25:56,085][61552] Updated weights for policy 0, policy_version 43742 (0.0007) [2023-10-14 19:25:57,245][61585] Updated weights for policy 1, policy_version 43560 (0.0010) [2023-10-14 19:25:57,611][61585] Updated weights for policy 1, policy_version 43570 (0.0011) [2023-10-14 19:25:57,972][61585] Updated weights for policy 1, policy_version 43580 (0.0010) [2023-10-14 19:25:58,343][60425] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 89423872. Throughput: 0: 1689.1, 1: 1659.9. Samples: 22363228. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) [2023-10-14 19:25:58,344][60425] Avg episode reward: [(0, '69.920'), (1, '65.010')] [2023-10-14 19:26:00,109][61552] Updated weights for policy 0, policy_version 43752 (0.0010) [2023-10-14 19:26:00,477][61552] Updated weights for policy 0, policy_version 43762 (0.0011) [2023-10-14 19:26:00,849][61552] Updated weights for policy 0, policy_version 43772 (0.0010) [2023-10-14 19:26:02,098][61585] Updated weights for policy 1, policy_version 43590 (0.0009) [2023-10-14 19:26:02,469][61585] Updated weights for policy 1, policy_version 43600 (0.0007) [2023-10-14 19:26:02,833][61585] Updated weights for policy 1, policy_version 43610 (0.0008) [2023-10-14 19:26:03,343][60425] Fps is (10 sec: 16383.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 89489408. Throughput: 0: 1667.7, 1: 1684.0. Samples: 22373680. Policy #0 lag: (min: 11.0, avg: 13.0, max: 42.0) [2023-10-14 19:26:03,345][60425] Avg episode reward: [(0, '67.360'), (1, '68.040')] [2023-10-14 19:26:04,909][61552] Updated weights for policy 0, policy_version 43782 (0.0009) [2023-10-14 19:26:05,280][61552] Updated weights for policy 0, policy_version 43792 (0.0009) [2023-10-14 19:26:05,650][61552] Updated weights for policy 0, policy_version 43802 (0.0008) [2023-10-14 19:26:06,886][61585] Updated weights for policy 1, policy_version 43620 (0.0010) [2023-10-14 19:26:07,241][61585] Updated weights for policy 1, policy_version 43630 (0.0009) [2023-10-14 19:26:07,611][61585] Updated weights for policy 1, policy_version 43640 (0.0008) [2023-10-14 19:26:08,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 89554944. Throughput: 0: 1686.4, 1: 1679.3. Samples: 22393902. Policy #0 lag: (min: 11.0, avg: 13.0, max: 42.0) [2023-10-14 19:26:08,344][60425] Avg episode reward: [(0, '71.380'), (1, '64.830')] [2023-10-14 19:26:09,665][61552] Updated weights for policy 0, policy_version 43812 (0.0009) [2023-10-14 19:26:10,037][61552] Updated weights for policy 0, policy_version 43822 (0.0010) [2023-10-14 19:26:10,407][61552] Updated weights for policy 0, policy_version 43832 (0.0010) [2023-10-14 19:26:11,771][61585] Updated weights for policy 1, policy_version 43650 (0.0007) [2023-10-14 19:26:12,130][61585] Updated weights for policy 1, policy_version 43660 (0.0009) [2023-10-14 19:26:12,495][61585] Updated weights for policy 1, policy_version 43670 (0.0009) [2023-10-14 19:26:12,858][61585] Updated weights for policy 1, policy_version 43680 (0.0011) [2023-10-14 19:26:13,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 89620480. Throughput: 0: 1691.0, 1: 1657.0. Samples: 22413414. Policy #0 lag: (min: 11.0, avg: 13.0, max: 42.0) [2023-10-14 19:26:13,344][60425] Avg episode reward: [(0, '69.730'), (1, '64.390')] [2023-10-14 19:26:14,490][61552] Updated weights for policy 0, policy_version 43842 (0.0008) [2023-10-14 19:26:14,894][61552] Updated weights for policy 0, policy_version 43852 (0.0010) [2023-10-14 19:26:15,263][61552] Updated weights for policy 0, policy_version 43862 (0.0011) [2023-10-14 19:26:15,634][61552] Updated weights for policy 0, policy_version 43872 (0.0009) [2023-10-14 19:26:16,751][61585] Updated weights for policy 1, policy_version 43690 (0.0007) [2023-10-14 19:26:17,122][61585] Updated weights for policy 1, policy_version 43700 (0.0009) [2023-10-14 19:26:17,480][61585] Updated weights for policy 1, policy_version 43710 (0.0008) [2023-10-14 19:26:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 89686016. Throughput: 0: 1660.7, 1: 1672.7. Samples: 22423532. Policy #0 lag: (min: 11.0, avg: 13.0, max: 42.0) [2023-10-14 19:26:18,344][60425] Avg episode reward: [(0, '66.130'), (1, '64.640')] [2023-10-14 19:26:19,711][61552] Updated weights for policy 0, policy_version 43882 (0.0009) [2023-10-14 19:26:20,080][61552] Updated weights for policy 0, policy_version 43892 (0.0009) [2023-10-14 19:26:20,447][61552] Updated weights for policy 0, policy_version 43902 (0.0007) [2023-10-14 19:26:21,758][61585] Updated weights for policy 1, policy_version 43720 (0.0008) [2023-10-14 19:26:22,133][61585] Updated weights for policy 1, policy_version 43730 (0.0010) [2023-10-14 19:26:22,491][61585] Updated weights for policy 1, policy_version 43740 (0.0011) [2023-10-14 19:26:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.5). Total num frames: 89751552. Throughput: 0: 1683.6, 1: 1657.7. Samples: 22443516. Policy #0 lag: (min: 11.0, avg: 13.0, max: 42.0) [2023-10-14 19:26:23,344][60425] Avg episode reward: [(0, '70.600'), (1, '68.620')] [2023-10-14 19:26:24,460][61552] Updated weights for policy 0, policy_version 43912 (0.0007) [2023-10-14 19:26:24,830][61552] Updated weights for policy 0, policy_version 43922 (0.0008) [2023-10-14 19:26:25,200][61552] Updated weights for policy 0, policy_version 43932 (0.0007) [2023-10-14 19:26:26,602][61585] Updated weights for policy 1, policy_version 43750 (0.0010) [2023-10-14 19:26:26,968][61585] Updated weights for policy 1, policy_version 43760 (0.0009) [2023-10-14 19:26:27,336][61585] Updated weights for policy 1, policy_version 43770 (0.0009) [2023-10-14 19:26:28,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 89817088. Throughput: 0: 1684.8, 1: 1653.8. Samples: 22463378. Policy #0 lag: (min: 11.0, avg: 13.0, max: 42.0) [2023-10-14 19:26:28,345][60425] Avg episode reward: [(0, '69.300'), (1, '65.970')] [2023-10-14 19:26:29,283][61552] Updated weights for policy 0, policy_version 43942 (0.0007) [2023-10-14 19:26:29,653][61552] Updated weights for policy 0, policy_version 43952 (0.0007) [2023-10-14 19:26:30,025][61552] Updated weights for policy 0, policy_version 43962 (0.0008) [2023-10-14 19:26:31,343][61585] Updated weights for policy 1, policy_version 43780 (0.0008) [2023-10-14 19:26:31,697][61585] Updated weights for policy 1, policy_version 43790 (0.0008) [2023-10-14 19:26:32,061][61585] Updated weights for policy 1, policy_version 43800 (0.0007) [2023-10-14 19:26:33,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 89882624. Throughput: 0: 1662.9, 1: 1671.7. Samples: 22473676. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 19:26:33,345][60425] Avg episode reward: [(0, '72.180'), (1, '67.340')] [2023-10-14 19:26:33,851][61552] Updated weights for policy 0, policy_version 43972 (0.0011) [2023-10-14 19:26:34,218][61552] Updated weights for policy 0, policy_version 43982 (0.0009) [2023-10-14 19:26:34,591][61552] Updated weights for policy 0, policy_version 43992 (0.0012) [2023-10-14 19:26:36,139][61585] Updated weights for policy 1, policy_version 43810 (0.0008) [2023-10-14 19:26:36,516][61585] Updated weights for policy 1, policy_version 43820 (0.0007) [2023-10-14 19:26:36,877][61585] Updated weights for policy 1, policy_version 43830 (0.0008) [2023-10-14 19:26:37,250][61585] Updated weights for policy 1, policy_version 43840 (0.0007) [2023-10-14 19:26:38,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 89948160. Throughput: 0: 1685.6, 1: 1660.4. Samples: 22493908. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 19:26:38,344][60425] Avg episode reward: [(0, '70.270'), (1, '70.780')] [2023-10-14 19:26:38,653][61552] Updated weights for policy 0, policy_version 44002 (0.0008) [2023-10-14 19:26:39,012][61552] Updated weights for policy 0, policy_version 44012 (0.0009) [2023-10-14 19:26:39,381][61552] Updated weights for policy 0, policy_version 44022 (0.0009) [2023-10-14 19:26:39,755][61552] Updated weights for policy 0, policy_version 44032 (0.0010) [2023-10-14 19:26:41,264][61585] Updated weights for policy 1, policy_version 43850 (0.0009) [2023-10-14 19:26:41,624][61585] Updated weights for policy 1, policy_version 43860 (0.0008) [2023-10-14 19:26:41,995][61585] Updated weights for policy 1, policy_version 43870 (0.0007) [2023-10-14 19:26:43,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 90013696. Throughput: 0: 1685.6, 1: 1668.7. Samples: 22514174. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 19:26:43,344][60425] Avg episode reward: [(0, '73.730'), (1, '69.310')] [2023-10-14 19:26:43,353][61172] Saving new best policy, reward=73.730! [2023-10-14 19:26:43,890][61552] Updated weights for policy 0, policy_version 44042 (0.0010) [2023-10-14 19:26:44,261][61552] Updated weights for policy 0, policy_version 44052 (0.0009) [2023-10-14 19:26:44,626][61552] Updated weights for policy 0, policy_version 44062 (0.0009) [2023-10-14 19:26:46,062][61585] Updated weights for policy 1, policy_version 43880 (0.0007) [2023-10-14 19:26:46,424][61585] Updated weights for policy 1, policy_version 43890 (0.0009) [2023-10-14 19:26:46,789][61585] Updated weights for policy 1, policy_version 43900 (0.0010) [2023-10-14 19:26:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 90079232. Throughput: 0: 1676.2, 1: 1673.1. Samples: 22524398. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 19:26:48,344][60425] Avg episode reward: [(0, '74.570'), (1, '66.840')] [2023-10-14 19:26:48,671][61552] Updated weights for policy 0, policy_version 44072 (0.0009) [2023-10-14 19:26:49,048][61552] Updated weights for policy 0, policy_version 44082 (0.0009) [2023-10-14 19:26:49,417][61552] Updated weights for policy 0, policy_version 44092 (0.0010) [2023-10-14 19:26:49,558][61172] Saving new best policy, reward=74.570! [2023-10-14 19:26:50,979][61585] Updated weights for policy 1, policy_version 43910 (0.0009) [2023-10-14 19:26:51,350][61585] Updated weights for policy 1, policy_version 43920 (0.0008) [2023-10-14 19:26:51,714][61585] Updated weights for policy 1, policy_version 43930 (0.0007) [2023-10-14 19:26:53,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 90144768. Throughput: 0: 1680.8, 1: 1648.3. Samples: 22543710. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 19:26:53,344][60425] Avg episode reward: [(0, '70.650'), (1, '70.040')] [2023-10-14 19:26:53,524][61552] Updated weights for policy 0, policy_version 44102 (0.0007) [2023-10-14 19:26:53,898][61552] Updated weights for policy 0, policy_version 44112 (0.0007) [2023-10-14 19:26:54,263][61552] Updated weights for policy 0, policy_version 44122 (0.0007) [2023-10-14 19:26:55,902][61585] Updated weights for policy 1, policy_version 43940 (0.0007) [2023-10-14 19:26:56,259][61585] Updated weights for policy 1, policy_version 43950 (0.0009) [2023-10-14 19:26:56,628][61585] Updated weights for policy 1, policy_version 43960 (0.0010) [2023-10-14 19:26:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 90210304. Throughput: 0: 1679.6, 1: 1666.6. Samples: 22563996. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 19:26:58,344][60425] Avg episode reward: [(0, '72.690'), (1, '69.960')] [2023-10-14 19:26:58,494][61552] Updated weights for policy 0, policy_version 44132 (0.0007) [2023-10-14 19:26:58,865][61552] Updated weights for policy 0, policy_version 44142 (0.0007) [2023-10-14 19:26:59,225][61552] Updated weights for policy 0, policy_version 44152 (0.0009) [2023-10-14 19:27:00,602][61585] Updated weights for policy 1, policy_version 43970 (0.0007) [2023-10-14 19:27:00,977][61585] Updated weights for policy 1, policy_version 43980 (0.0007) [2023-10-14 19:27:01,334][61585] Updated weights for policy 1, policy_version 43990 (0.0010) [2023-10-14 19:27:01,699][61585] Updated weights for policy 1, policy_version 44000 (0.0009) [2023-10-14 19:27:03,338][61552] Updated weights for policy 0, policy_version 44162 (0.0010) [2023-10-14 19:27:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 90275840. Throughput: 0: 1683.9, 1: 1664.9. Samples: 22574228. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:27:03,344][60425] Avg episode reward: [(0, '73.910'), (1, '69.350')] [2023-10-14 19:27:03,734][61552] Updated weights for policy 0, policy_version 44172 (0.0008) [2023-10-14 19:27:04,103][61552] Updated weights for policy 0, policy_version 44182 (0.0009) [2023-10-14 19:27:04,468][61552] Updated weights for policy 0, policy_version 44192 (0.0007) [2023-10-14 19:27:05,984][61585] Updated weights for policy 1, policy_version 44010 (0.0009) [2023-10-14 19:27:06,349][61585] Updated weights for policy 1, policy_version 44020 (0.0008) [2023-10-14 19:27:06,713][61585] Updated weights for policy 1, policy_version 44030 (0.0010) [2023-10-14 19:27:08,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 90341376. Throughput: 0: 1686.2, 1: 1659.4. Samples: 22594068. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:27:08,344][60425] Avg episode reward: [(0, '72.380'), (1, '71.940')] [2023-10-14 19:27:08,345][61248] Saving new best policy, reward=71.940! [2023-10-14 19:27:08,465][61552] Updated weights for policy 0, policy_version 44202 (0.0009) [2023-10-14 19:27:08,833][61552] Updated weights for policy 0, policy_version 44212 (0.0007) [2023-10-14 19:27:09,191][61552] Updated weights for policy 0, policy_version 44222 (0.0007) [2023-10-14 19:27:10,761][61585] Updated weights for policy 1, policy_version 44040 (0.0009) [2023-10-14 19:27:11,127][61585] Updated weights for policy 1, policy_version 44050 (0.0009) [2023-10-14 19:27:11,499][61585] Updated weights for policy 1, policy_version 44060 (0.0008) [2023-10-14 19:27:13,281][61552] Updated weights for policy 0, policy_version 44232 (0.0008) [2023-10-14 19:27:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 90406912. Throughput: 0: 1681.3, 1: 1676.2. Samples: 22614464. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:27:13,344][60425] Avg episode reward: [(0, '68.400'), (1, '68.090')] [2023-10-14 19:27:13,655][61552] Updated weights for policy 0, policy_version 44242 (0.0007) [2023-10-14 19:27:14,019][61552] Updated weights for policy 0, policy_version 44252 (0.0008) [2023-10-14 19:27:15,439][61585] Updated weights for policy 1, policy_version 44070 (0.0009) [2023-10-14 19:27:15,805][61585] Updated weights for policy 1, policy_version 44080 (0.0009) [2023-10-14 19:27:16,176][61585] Updated weights for policy 1, policy_version 44090 (0.0009) [2023-10-14 19:27:18,176][61552] Updated weights for policy 0, policy_version 44262 (0.0008) [2023-10-14 19:27:18,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 90472448. Throughput: 0: 1682.6, 1: 1667.2. Samples: 22624414. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:27:18,344][60425] Avg episode reward: [(0, '71.090'), (1, '70.240')] [2023-10-14 19:27:18,536][61552] Updated weights for policy 0, policy_version 44272 (0.0008) [2023-10-14 19:27:18,914][61552] Updated weights for policy 0, policy_version 44282 (0.0008) [2023-10-14 19:27:20,294][61585] Updated weights for policy 1, policy_version 44100 (0.0009) [2023-10-14 19:27:20,661][61585] Updated weights for policy 1, policy_version 44110 (0.0007) [2023-10-14 19:27:21,020][61585] Updated weights for policy 1, policy_version 44120 (0.0010) [2023-10-14 19:27:23,157][61552] Updated weights for policy 0, policy_version 44292 (0.0007) [2023-10-14 19:27:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 90537984. Throughput: 0: 1675.7, 1: 1666.8. Samples: 22644322. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:27:23,344][60425] Avg episode reward: [(0, '69.750'), (1, '68.560')] [2023-10-14 19:27:23,533][61552] Updated weights for policy 0, policy_version 44302 (0.0008) [2023-10-14 19:27:23,902][61552] Updated weights for policy 0, policy_version 44312 (0.0007) [2023-10-14 19:27:25,269][61585] Updated weights for policy 1, policy_version 44130 (0.0009) [2023-10-14 19:27:25,643][61585] Updated weights for policy 1, policy_version 44140 (0.0010) [2023-10-14 19:27:26,000][61585] Updated weights for policy 1, policy_version 44150 (0.0007) [2023-10-14 19:27:26,371][61585] Updated weights for policy 1, policy_version 44160 (0.0009) [2023-10-14 19:27:27,945][61552] Updated weights for policy 0, policy_version 44322 (0.0008) [2023-10-14 19:27:28,321][61552] Updated weights for policy 0, policy_version 44332 (0.0007) [2023-10-14 19:27:28,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 90603520. Throughput: 0: 1675.0, 1: 1675.8. Samples: 22664960. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:27:28,344][60425] Avg episode reward: [(0, '66.820'), (1, '71.410')] [2023-10-14 19:27:28,690][61552] Updated weights for policy 0, policy_version 44342 (0.0007) [2023-10-14 19:27:29,066][61552] Updated weights for policy 0, policy_version 44352 (0.0008) [2023-10-14 19:27:30,338][61585] Updated weights for policy 1, policy_version 44170 (0.0011) [2023-10-14 19:27:30,703][61585] Updated weights for policy 1, policy_version 44180 (0.0011) [2023-10-14 19:27:31,071][61585] Updated weights for policy 1, policy_version 44190 (0.0009) [2023-10-14 19:27:33,057][61552] Updated weights for policy 0, policy_version 44362 (0.0007) [2023-10-14 19:27:33,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 90669056. Throughput: 0: 1678.0, 1: 1659.8. Samples: 22674600. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 19:27:33,344][60425] Avg episode reward: [(0, '70.570'), (1, '65.460')] [2023-10-14 19:27:33,427][61552] Updated weights for policy 0, policy_version 44372 (0.0008) [2023-10-14 19:27:33,786][61552] Updated weights for policy 0, policy_version 44382 (0.0007) [2023-10-14 19:27:35,211][61585] Updated weights for policy 1, policy_version 44200 (0.0009) [2023-10-14 19:27:35,584][61585] Updated weights for policy 1, policy_version 44210 (0.0009) [2023-10-14 19:27:35,948][61585] Updated weights for policy 1, policy_version 44220 (0.0008) [2023-10-14 19:27:37,702][61552] Updated weights for policy 0, policy_version 44392 (0.0009) [2023-10-14 19:27:38,061][61552] Updated weights for policy 0, policy_version 44402 (0.0008) [2023-10-14 19:27:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 90734592. Throughput: 0: 1688.2, 1: 1672.5. Samples: 22694942. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 19:27:38,345][60425] Avg episode reward: [(0, '73.380'), (1, '67.550')] [2023-10-14 19:27:38,441][61552] Updated weights for policy 0, policy_version 44412 (0.0009) [2023-10-14 19:27:40,094][61585] Updated weights for policy 1, policy_version 44230 (0.0008) [2023-10-14 19:27:40,454][61585] Updated weights for policy 1, policy_version 44240 (0.0010) [2023-10-14 19:27:40,820][61585] Updated weights for policy 1, policy_version 44250 (0.0010) [2023-10-14 19:27:42,505][61552] Updated weights for policy 0, policy_version 44422 (0.0009) [2023-10-14 19:27:42,871][61552] Updated weights for policy 0, policy_version 44432 (0.0011) [2023-10-14 19:27:43,245][61552] Updated weights for policy 0, policy_version 44442 (0.0009) [2023-10-14 19:27:43,344][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 90800128. Throughput: 0: 1677.3, 1: 1683.6. Samples: 22715238. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 19:27:43,345][60425] Avg episode reward: [(0, '70.260'), (1, '71.280')] [2023-10-14 19:27:43,358][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000044256_45318144.pth... [2023-10-14 19:27:43,392][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000042688_43712512.pth [2023-10-14 19:27:43,466][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000044448_45514752.pth... [2023-10-14 19:27:43,502][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000042880_43909120.pth [2023-10-14 19:27:44,997][61585] Updated weights for policy 1, policy_version 44260 (0.0009) [2023-10-14 19:27:45,359][61585] Updated weights for policy 1, policy_version 44270 (0.0007) [2023-10-14 19:27:45,731][61585] Updated weights for policy 1, policy_version 44280 (0.0007) [2023-10-14 19:27:47,286][61552] Updated weights for policy 0, policy_version 44452 (0.0009) [2023-10-14 19:27:47,652][61552] Updated weights for policy 0, policy_version 44462 (0.0008) [2023-10-14 19:27:48,030][61552] Updated weights for policy 0, policy_version 44472 (0.0008) [2023-10-14 19:27:48,343][60425] Fps is (10 sec: 16384.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 90898432. Throughput: 0: 1683.0, 1: 1669.8. Samples: 22725104. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 19:27:48,344][60425] Avg episode reward: [(0, '71.300'), (1, '69.810')] [2023-10-14 19:27:49,716][61585] Updated weights for policy 1, policy_version 44290 (0.0009) [2023-10-14 19:27:50,080][61585] Updated weights for policy 1, policy_version 44300 (0.0007) [2023-10-14 19:27:50,450][61585] Updated weights for policy 1, policy_version 44310 (0.0007) [2023-10-14 19:27:50,816][61585] Updated weights for policy 1, policy_version 44320 (0.0009) [2023-10-14 19:27:52,206][61552] Updated weights for policy 0, policy_version 44482 (0.0009) [2023-10-14 19:27:52,602][61552] Updated weights for policy 0, policy_version 44492 (0.0009) [2023-10-14 19:27:52,979][61552] Updated weights for policy 0, policy_version 44502 (0.0007) [2023-10-14 19:27:53,343][60425] Fps is (10 sec: 16384.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 90963968. Throughput: 0: 1683.1, 1: 1678.5. Samples: 22745340. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 19:27:53,344][60425] Avg episode reward: [(0, '70.830'), (1, '70.680')] [2023-10-14 19:27:53,345][61552] Updated weights for policy 0, policy_version 44512 (0.0008) [2023-10-14 19:27:54,880][61585] Updated weights for policy 1, policy_version 44330 (0.0007) [2023-10-14 19:27:55,240][61585] Updated weights for policy 1, policy_version 44340 (0.0007) [2023-10-14 19:27:55,608][61585] Updated weights for policy 1, policy_version 44350 (0.0010) [2023-10-14 19:27:57,367][61552] Updated weights for policy 0, policy_version 44522 (0.0008) [2023-10-14 19:27:57,736][61552] Updated weights for policy 0, policy_version 44532 (0.0007) [2023-10-14 19:27:58,121][61552] Updated weights for policy 0, policy_version 44542 (0.0009) [2023-10-14 19:27:58,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 91029504. Throughput: 0: 1664.8, 1: 1685.6. Samples: 22765232. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:27:58,344][60425] Avg episode reward: [(0, '72.550'), (1, '67.020')] [2023-10-14 19:27:59,502][61585] Updated weights for policy 1, policy_version 44360 (0.0009) [2023-10-14 19:27:59,867][61585] Updated weights for policy 1, policy_version 44370 (0.0010) [2023-10-14 19:28:00,237][61585] Updated weights for policy 1, policy_version 44380 (0.0012) [2023-10-14 19:28:02,112][61552] Updated weights for policy 0, policy_version 44552 (0.0008) [2023-10-14 19:28:02,491][61552] Updated weights for policy 0, policy_version 44562 (0.0011) [2023-10-14 19:28:02,853][61552] Updated weights for policy 0, policy_version 44572 (0.0007) [2023-10-14 19:28:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 91095040. Throughput: 0: 1685.0, 1: 1666.4. Samples: 22775226. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:28:03,344][60425] Avg episode reward: [(0, '68.970'), (1, '67.560')] [2023-10-14 19:28:04,330][61585] Updated weights for policy 1, policy_version 44390 (0.0010) [2023-10-14 19:28:04,691][61585] Updated weights for policy 1, policy_version 44400 (0.0011) [2023-10-14 19:28:05,061][61585] Updated weights for policy 1, policy_version 44410 (0.0007) [2023-10-14 19:28:07,000][61552] Updated weights for policy 0, policy_version 44582 (0.0007) [2023-10-14 19:28:07,369][61552] Updated weights for policy 0, policy_version 44592 (0.0009) [2023-10-14 19:28:07,748][61552] Updated weights for policy 0, policy_version 44602 (0.0009) [2023-10-14 19:28:08,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 91160576. Throughput: 0: 1688.6, 1: 1681.4. Samples: 22795972. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:28:08,344][60425] Avg episode reward: [(0, '76.410'), (1, '68.690')] [2023-10-14 19:28:08,345][61172] Saving new best policy, reward=76.410! [2023-10-14 19:28:09,222][61585] Updated weights for policy 1, policy_version 44420 (0.0007) [2023-10-14 19:28:09,576][61585] Updated weights for policy 1, policy_version 44430 (0.0009) [2023-10-14 19:28:09,949][61585] Updated weights for policy 1, policy_version 44440 (0.0009) [2023-10-14 19:28:11,783][61552] Updated weights for policy 0, policy_version 44612 (0.0009) [2023-10-14 19:28:12,163][61552] Updated weights for policy 0, policy_version 44622 (0.0011) [2023-10-14 19:28:12,518][61552] Updated weights for policy 0, policy_version 44632 (0.0011) [2023-10-14 19:28:13,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 91226112. Throughput: 0: 1662.5, 1: 1686.3. Samples: 22815654. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:28:13,344][60425] Avg episode reward: [(0, '69.810'), (1, '68.270')] [2023-10-14 19:28:13,806][61585] Updated weights for policy 1, policy_version 44450 (0.0008) [2023-10-14 19:28:14,168][61585] Updated weights for policy 1, policy_version 44460 (0.0009) [2023-10-14 19:28:14,536][61585] Updated weights for policy 1, policy_version 44470 (0.0009) [2023-10-14 19:28:14,901][61585] Updated weights for policy 1, policy_version 44480 (0.0011) [2023-10-14 19:28:16,713][61552] Updated weights for policy 0, policy_version 44642 (0.0011) [2023-10-14 19:28:17,076][61552] Updated weights for policy 0, policy_version 44652 (0.0009) [2023-10-14 19:28:17,439][61552] Updated weights for policy 0, policy_version 44662 (0.0007) [2023-10-14 19:28:17,809][61552] Updated weights for policy 0, policy_version 44672 (0.0007) [2023-10-14 19:28:18,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 91291648. Throughput: 0: 1682.3, 1: 1679.1. Samples: 22825862. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:28:18,344][60425] Avg episode reward: [(0, '71.390'), (1, '64.210')] [2023-10-14 19:28:19,049][61585] Updated weights for policy 1, policy_version 44490 (0.0007) [2023-10-14 19:28:19,410][61585] Updated weights for policy 1, policy_version 44500 (0.0007) [2023-10-14 19:28:19,775][61585] Updated weights for policy 1, policy_version 44510 (0.0010) [2023-10-14 19:28:21,823][61552] Updated weights for policy 0, policy_version 44682 (0.0010) [2023-10-14 19:28:22,189][61552] Updated weights for policy 0, policy_version 44692 (0.0008) [2023-10-14 19:28:22,550][61552] Updated weights for policy 0, policy_version 44702 (0.0009) [2023-10-14 19:28:23,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 91357184. Throughput: 0: 1665.9, 1: 1695.7. Samples: 22846214. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:28:23,344][60425] Avg episode reward: [(0, '70.020'), (1, '66.530')] [2023-10-14 19:28:23,693][61585] Updated weights for policy 1, policy_version 44520 (0.0010) [2023-10-14 19:28:24,061][61585] Updated weights for policy 1, policy_version 44530 (0.0009) [2023-10-14 19:28:24,419][61585] Updated weights for policy 1, policy_version 44540 (0.0009) [2023-10-14 19:28:26,622][61552] Updated weights for policy 0, policy_version 44712 (0.0008) [2023-10-14 19:28:26,984][61552] Updated weights for policy 0, policy_version 44722 (0.0008) [2023-10-14 19:28:27,344][61552] Updated weights for policy 0, policy_version 44732 (0.0009) [2023-10-14 19:28:28,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 91422720. Throughput: 0: 1654.6, 1: 1698.5. Samples: 22866126. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-14 19:28:28,344][60425] Avg episode reward: [(0, '66.860'), (1, '68.370')] [2023-10-14 19:28:28,366][61585] Updated weights for policy 1, policy_version 44550 (0.0008) [2023-10-14 19:28:28,734][61585] Updated weights for policy 1, policy_version 44560 (0.0008) [2023-10-14 19:28:29,100][61585] Updated weights for policy 1, policy_version 44570 (0.0009) [2023-10-14 19:28:31,331][61552] Updated weights for policy 0, policy_version 44742 (0.0009) [2023-10-14 19:28:31,706][61552] Updated weights for policy 0, policy_version 44752 (0.0010) [2023-10-14 19:28:32,069][61552] Updated weights for policy 0, policy_version 44762 (0.0008) [2023-10-14 19:28:33,265][61585] Updated weights for policy 1, policy_version 44580 (0.0010) [2023-10-14 19:28:33,343][60425] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 91488256. Throughput: 0: 1680.3, 1: 1687.4. Samples: 22876654. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-14 19:28:33,345][60425] Avg episode reward: [(0, '71.990'), (1, '68.500')] [2023-10-14 19:28:33,626][61585] Updated weights for policy 1, policy_version 44590 (0.0007) [2023-10-14 19:28:33,995][61585] Updated weights for policy 1, policy_version 44600 (0.0007) [2023-10-14 19:28:36,073][61552] Updated weights for policy 0, policy_version 44772 (0.0008) [2023-10-14 19:28:36,446][61552] Updated weights for policy 0, policy_version 44782 (0.0008) [2023-10-14 19:28:36,805][61552] Updated weights for policy 0, policy_version 44792 (0.0008) [2023-10-14 19:28:37,946][61585] Updated weights for policy 1, policy_version 44610 (0.0008) [2023-10-14 19:28:38,306][61585] Updated weights for policy 1, policy_version 44620 (0.0007) [2023-10-14 19:28:38,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 91553792. Throughput: 0: 1663.9, 1: 1698.2. Samples: 22896632. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-14 19:28:38,344][60425] Avg episode reward: [(0, '70.810'), (1, '66.530')] [2023-10-14 19:28:38,671][61585] Updated weights for policy 1, policy_version 44630 (0.0007) [2023-10-14 19:28:39,037][61585] Updated weights for policy 1, policy_version 44640 (0.0007) [2023-10-14 19:28:40,845][61552] Updated weights for policy 0, policy_version 44802 (0.0009) [2023-10-14 19:28:41,250][61552] Updated weights for policy 0, policy_version 44812 (0.0009) [2023-10-14 19:28:41,620][61552] Updated weights for policy 0, policy_version 44822 (0.0007) [2023-10-14 19:28:41,991][61552] Updated weights for policy 0, policy_version 44832 (0.0007) [2023-10-14 19:28:43,131][61585] Updated weights for policy 1, policy_version 44650 (0.0007) [2023-10-14 19:28:43,344][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 91619328. Throughput: 0: 1672.2, 1: 1698.7. Samples: 22916926. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-14 19:28:43,345][60425] Avg episode reward: [(0, '70.970'), (1, '70.790')] [2023-10-14 19:28:43,497][61585] Updated weights for policy 1, policy_version 44660 (0.0007) [2023-10-14 19:28:43,866][61585] Updated weights for policy 1, policy_version 44670 (0.0009) [2023-10-14 19:28:45,990][61552] Updated weights for policy 0, policy_version 44842 (0.0008) [2023-10-14 19:28:46,350][61552] Updated weights for policy 0, policy_version 44852 (0.0008) [2023-10-14 19:28:46,716][61552] Updated weights for policy 0, policy_version 44862 (0.0010) [2023-10-14 19:28:48,171][61585] Updated weights for policy 1, policy_version 44680 (0.0009) [2023-10-14 19:28:48,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 91684864. Throughput: 0: 1679.1, 1: 1697.3. Samples: 22927162. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-14 19:28:48,344][60425] Avg episode reward: [(0, '70.690'), (1, '70.460')] [2023-10-14 19:28:48,548][61585] Updated weights for policy 1, policy_version 44690 (0.0008) [2023-10-14 19:28:48,912][61585] Updated weights for policy 1, policy_version 44700 (0.0007) [2023-10-14 19:28:50,869][61552] Updated weights for policy 0, policy_version 44872 (0.0008) [2023-10-14 19:28:51,245][61552] Updated weights for policy 0, policy_version 44882 (0.0009) [2023-10-14 19:28:51,614][61552] Updated weights for policy 0, policy_version 44892 (0.0010) [2023-10-14 19:28:53,034][61585] Updated weights for policy 1, policy_version 44710 (0.0009) [2023-10-14 19:28:53,343][60425] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 91750400. Throughput: 0: 1657.5, 1: 1688.4. Samples: 22946538. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-14 19:28:53,344][60425] Avg episode reward: [(0, '71.200'), (1, '71.560')] [2023-10-14 19:28:53,397][61585] Updated weights for policy 1, policy_version 44720 (0.0008) [2023-10-14 19:28:53,763][61585] Updated weights for policy 1, policy_version 44730 (0.0009) [2023-10-14 19:28:55,816][61552] Updated weights for policy 0, policy_version 44902 (0.0008) [2023-10-14 19:28:56,175][61552] Updated weights for policy 0, policy_version 44912 (0.0011) [2023-10-14 19:28:56,546][61552] Updated weights for policy 0, policy_version 44922 (0.0010) [2023-10-14 19:28:57,607][61585] Updated weights for policy 1, policy_version 44740 (0.0009) [2023-10-14 19:28:57,963][61585] Updated weights for policy 1, policy_version 44750 (0.0011) [2023-10-14 19:28:58,334][61585] Updated weights for policy 1, policy_version 44760 (0.0010) [2023-10-14 19:28:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 91815936. Throughput: 0: 1680.6, 1: 1682.9. Samples: 22967010. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-14 19:28:58,344][60425] Avg episode reward: [(0, '69.080'), (1, '72.150')] [2023-10-14 19:28:58,625][61248] Saving new best policy, reward=72.150! [2023-10-14 19:29:00,653][61552] Updated weights for policy 0, policy_version 44932 (0.0009) [2023-10-14 19:29:01,017][61552] Updated weights for policy 0, policy_version 44942 (0.0008) [2023-10-14 19:29:01,374][61552] Updated weights for policy 0, policy_version 44952 (0.0007) [2023-10-14 19:29:02,419][61585] Updated weights for policy 1, policy_version 44770 (0.0009) [2023-10-14 19:29:02,789][61585] Updated weights for policy 1, policy_version 44780 (0.0007) [2023-10-14 19:29:03,159][61585] Updated weights for policy 1, policy_version 44790 (0.0007) [2023-10-14 19:29:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 91881472. Throughput: 0: 1683.4, 1: 1685.1. Samples: 22977444. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-14 19:29:03,344][60425] Avg episode reward: [(0, '66.910'), (1, '71.040')] [2023-10-14 19:29:03,521][61585] Updated weights for policy 1, policy_version 44800 (0.0008) [2023-10-14 19:29:05,456][61552] Updated weights for policy 0, policy_version 44962 (0.0007) [2023-10-14 19:29:05,832][61552] Updated weights for policy 0, policy_version 44972 (0.0007) [2023-10-14 19:29:06,201][61552] Updated weights for policy 0, policy_version 44982 (0.0008) [2023-10-14 19:29:06,574][61552] Updated weights for policy 0, policy_version 44992 (0.0009) [2023-10-14 19:29:07,875][61585] Updated weights for policy 1, policy_version 44810 (0.0009) [2023-10-14 19:29:08,246][61585] Updated weights for policy 1, policy_version 44820 (0.0008) [2023-10-14 19:29:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 91947008. Throughput: 0: 1669.8, 1: 1678.7. Samples: 22996896. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-14 19:29:08,344][60425] Avg episode reward: [(0, '69.900'), (1, '72.150')] [2023-10-14 19:29:08,611][61585] Updated weights for policy 1, policy_version 44830 (0.0007) [2023-10-14 19:29:10,554][61552] Updated weights for policy 0, policy_version 45002 (0.0008) [2023-10-14 19:29:10,930][61552] Updated weights for policy 0, policy_version 45012 (0.0011) [2023-10-14 19:29:11,291][61552] Updated weights for policy 0, policy_version 45022 (0.0010) [2023-10-14 19:29:12,713][61585] Updated weights for policy 1, policy_version 44840 (0.0007) [2023-10-14 19:29:13,082][61585] Updated weights for policy 1, policy_version 44850 (0.0011) [2023-10-14 19:29:13,343][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 92012544. Throughput: 0: 1693.3, 1: 1663.9. Samples: 23017200. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-14 19:29:13,345][60425] Avg episode reward: [(0, '70.450'), (1, '69.880')] [2023-10-14 19:29:13,443][61585] Updated weights for policy 1, policy_version 44860 (0.0011) [2023-10-14 19:29:15,278][61552] Updated weights for policy 0, policy_version 45032 (0.0008) [2023-10-14 19:29:15,659][61552] Updated weights for policy 0, policy_version 45042 (0.0007) [2023-10-14 19:29:16,027][61552] Updated weights for policy 0, policy_version 45052 (0.0008) [2023-10-14 19:29:17,440][61585] Updated weights for policy 1, policy_version 44870 (0.0010) [2023-10-14 19:29:17,799][61585] Updated weights for policy 1, policy_version 44880 (0.0011) [2023-10-14 19:29:18,168][61585] Updated weights for policy 1, policy_version 44890 (0.0009) [2023-10-14 19:29:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 92078080. Throughput: 0: 1672.8, 1: 1671.4. Samples: 23027142. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-14 19:29:18,344][60425] Avg episode reward: [(0, '71.670'), (1, '72.400')] [2023-10-14 19:29:18,374][61248] Saving new best policy, reward=72.400! [2023-10-14 19:29:20,076][61552] Updated weights for policy 0, policy_version 45062 (0.0009) [2023-10-14 19:29:20,451][61552] Updated weights for policy 0, policy_version 45072 (0.0007) [2023-10-14 19:29:20,812][61552] Updated weights for policy 0, policy_version 45082 (0.0008) [2023-10-14 19:29:22,318][61585] Updated weights for policy 1, policy_version 44900 (0.0009) [2023-10-14 19:29:22,678][61585] Updated weights for policy 1, policy_version 44910 (0.0008) [2023-10-14 19:29:23,045][61585] Updated weights for policy 1, policy_version 44920 (0.0009) [2023-10-14 19:29:23,343][60425] Fps is (10 sec: 16384.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 92176384. Throughput: 0: 1677.4, 1: 1669.0. Samples: 23047220. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-14 19:29:23,344][60425] Avg episode reward: [(0, '73.940'), (1, '70.940')] [2023-10-14 19:29:25,029][61552] Updated weights for policy 0, policy_version 45092 (0.0008) [2023-10-14 19:29:25,398][61552] Updated weights for policy 0, policy_version 45102 (0.0007) [2023-10-14 19:29:25,758][61552] Updated weights for policy 0, policy_version 45112 (0.0007) [2023-10-14 19:29:27,216][61585] Updated weights for policy 1, policy_version 44930 (0.0009) [2023-10-14 19:29:27,581][61585] Updated weights for policy 1, policy_version 44940 (0.0007) [2023-10-14 19:29:27,952][61585] Updated weights for policy 1, policy_version 44950 (0.0007) [2023-10-14 19:29:28,327][61585] Updated weights for policy 1, policy_version 44960 (0.0010) [2023-10-14 19:29:28,344][60425] Fps is (10 sec: 16383.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 92241920. Throughput: 0: 1685.3, 1: 1651.5. Samples: 23067080. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-14 19:29:28,345][60425] Avg episode reward: [(0, '69.090'), (1, '70.080')] [2023-10-14 19:29:29,830][61552] Updated weights for policy 0, policy_version 45122 (0.0009) [2023-10-14 19:29:30,225][61552] Updated weights for policy 0, policy_version 45132 (0.0009) [2023-10-14 19:29:30,587][61552] Updated weights for policy 0, policy_version 45142 (0.0008) [2023-10-14 19:29:30,956][61552] Updated weights for policy 0, policy_version 45152 (0.0010) [2023-10-14 19:29:32,346][61585] Updated weights for policy 1, policy_version 44970 (0.0009) [2023-10-14 19:29:32,720][61585] Updated weights for policy 1, policy_version 44980 (0.0009) [2023-10-14 19:29:33,100][61585] Updated weights for policy 1, policy_version 44990 (0.0008) [2023-10-14 19:29:33,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 92307456. Throughput: 0: 1661.5, 1: 1669.1. Samples: 23077040. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:29:33,344][60425] Avg episode reward: [(0, '71.390'), (1, '70.120')] [2023-10-14 19:29:35,147][61552] Updated weights for policy 0, policy_version 45162 (0.0011) [2023-10-14 19:29:35,502][61552] Updated weights for policy 0, policy_version 45172 (0.0011) [2023-10-14 19:29:35,873][61552] Updated weights for policy 0, policy_version 45182 (0.0010) [2023-10-14 19:29:37,206][61585] Updated weights for policy 1, policy_version 45000 (0.0009) [2023-10-14 19:29:37,584][61585] Updated weights for policy 1, policy_version 45010 (0.0008) [2023-10-14 19:29:37,936][61585] Updated weights for policy 1, policy_version 45020 (0.0008) [2023-10-14 19:29:38,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 92372992. Throughput: 0: 1670.9, 1: 1676.1. Samples: 23097154. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:29:38,344][60425] Avg episode reward: [(0, '72.320'), (1, '69.460')] [2023-10-14 19:29:40,238][61552] Updated weights for policy 0, policy_version 45192 (0.0009) [2023-10-14 19:29:40,602][61552] Updated weights for policy 0, policy_version 45202 (0.0007) [2023-10-14 19:29:40,969][61552] Updated weights for policy 0, policy_version 45212 (0.0008) [2023-10-14 19:29:42,045][61585] Updated weights for policy 1, policy_version 45030 (0.0007) [2023-10-14 19:29:42,410][61585] Updated weights for policy 1, policy_version 45040 (0.0007) [2023-10-14 19:29:42,774][61585] Updated weights for policy 1, policy_version 45050 (0.0008) [2023-10-14 19:29:43,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 92438528. Throughput: 0: 1671.4, 1: 1652.9. Samples: 23116604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:29:43,344][60425] Avg episode reward: [(0, '70.910'), (1, '69.490')] [2023-10-14 19:29:43,352][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000045056_46137344.pth... [2023-10-14 19:29:43,352][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000045216_46301184.pth... [2023-10-14 19:29:43,387][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000043648_44695552.pth [2023-10-14 19:29:43,392][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000043488_44531712.pth [2023-10-14 19:29:44,938][61552] Updated weights for policy 0, policy_version 45222 (0.0010) [2023-10-14 19:29:45,312][61552] Updated weights for policy 0, policy_version 45232 (0.0008) [2023-10-14 19:29:45,677][61552] Updated weights for policy 0, policy_version 45242 (0.0010) [2023-10-14 19:29:46,812][61585] Updated weights for policy 1, policy_version 45060 (0.0009) [2023-10-14 19:29:47,181][61585] Updated weights for policy 1, policy_version 45070 (0.0007) [2023-10-14 19:29:47,546][61585] Updated weights for policy 1, policy_version 45080 (0.0007) [2023-10-14 19:29:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 92504064. Throughput: 0: 1649.6, 1: 1670.2. Samples: 23126838. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:29:48,344][60425] Avg episode reward: [(0, '72.440'), (1, '71.350')] [2023-10-14 19:29:49,654][61552] Updated weights for policy 0, policy_version 45252 (0.0009) [2023-10-14 19:29:50,027][61552] Updated weights for policy 0, policy_version 45262 (0.0009) [2023-10-14 19:29:50,395][61552] Updated weights for policy 0, policy_version 45272 (0.0009) [2023-10-14 19:29:51,641][61585] Updated weights for policy 1, policy_version 45090 (0.0007) [2023-10-14 19:29:52,002][61585] Updated weights for policy 1, policy_version 45100 (0.0007) [2023-10-14 19:29:52,376][61585] Updated weights for policy 1, policy_version 45110 (0.0008) [2023-10-14 19:29:52,739][61585] Updated weights for policy 1, policy_version 45120 (0.0007) [2023-10-14 19:29:53,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 92569600. Throughput: 0: 1666.4, 1: 1674.0. Samples: 23147218. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:29:53,344][60425] Avg episode reward: [(0, '76.030'), (1, '72.960')] [2023-10-14 19:29:53,345][61248] Saving new best policy, reward=72.960! [2023-10-14 19:29:54,500][61552] Updated weights for policy 0, policy_version 45282 (0.0010) [2023-10-14 19:29:54,874][61552] Updated weights for policy 0, policy_version 45292 (0.0011) [2023-10-14 19:29:55,243][61552] Updated weights for policy 0, policy_version 45302 (0.0009) [2023-10-14 19:29:55,616][61552] Updated weights for policy 0, policy_version 45312 (0.0007) [2023-10-14 19:29:56,862][61585] Updated weights for policy 1, policy_version 45130 (0.0009) [2023-10-14 19:29:57,214][61585] Updated weights for policy 1, policy_version 45140 (0.0007) [2023-10-14 19:29:57,589][61585] Updated weights for policy 1, policy_version 45150 (0.0008) [2023-10-14 19:29:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 92635136. Throughput: 0: 1668.4, 1: 1656.4. Samples: 23166812. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:29:58,344][60425] Avg episode reward: [(0, '72.610'), (1, '69.720')] [2023-10-14 19:29:59,598][61552] Updated weights for policy 0, policy_version 45322 (0.0007) [2023-10-14 19:29:59,970][61552] Updated weights for policy 0, policy_version 45332 (0.0011) [2023-10-14 19:30:00,339][61552] Updated weights for policy 0, policy_version 45342 (0.0008) [2023-10-14 19:30:01,601][61585] Updated weights for policy 1, policy_version 45160 (0.0010) [2023-10-14 19:30:01,969][61585] Updated weights for policy 1, policy_version 45170 (0.0010) [2023-10-14 19:30:02,342][61585] Updated weights for policy 1, policy_version 45180 (0.0009) [2023-10-14 19:30:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 92700672. Throughput: 0: 1656.0, 1: 1678.9. Samples: 23177210. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:30:03,344][60425] Avg episode reward: [(0, '68.680'), (1, '66.660')] [2023-10-14 19:30:04,565][61552] Updated weights for policy 0, policy_version 45352 (0.0009) [2023-10-14 19:30:04,937][61552] Updated weights for policy 0, policy_version 45362 (0.0008) [2023-10-14 19:30:05,303][61552] Updated weights for policy 0, policy_version 45372 (0.0009) [2023-10-14 19:30:06,633][61585] Updated weights for policy 1, policy_version 45190 (0.0008) [2023-10-14 19:30:06,996][61585] Updated weights for policy 1, policy_version 45200 (0.0009) [2023-10-14 19:30:07,365][61585] Updated weights for policy 1, policy_version 45210 (0.0009) [2023-10-14 19:30:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 92766208. Throughput: 0: 1664.4, 1: 1671.0. Samples: 23197314. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:30:08,344][60425] Avg episode reward: [(0, '70.850'), (1, '68.040')] [2023-10-14 19:30:09,352][61552] Updated weights for policy 0, policy_version 45382 (0.0008) [2023-10-14 19:30:09,713][61552] Updated weights for policy 0, policy_version 45392 (0.0009) [2023-10-14 19:30:10,087][61552] Updated weights for policy 0, policy_version 45402 (0.0007) [2023-10-14 19:30:11,341][61585] Updated weights for policy 1, policy_version 45220 (0.0009) [2023-10-14 19:30:11,712][61585] Updated weights for policy 1, policy_version 45230 (0.0010) [2023-10-14 19:30:12,079][61585] Updated weights for policy 1, policy_version 45240 (0.0009) [2023-10-14 19:30:13,343][60425] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 92831744. Throughput: 0: 1666.3, 1: 1669.9. Samples: 23217206. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:30:13,344][60425] Avg episode reward: [(0, '69.910'), (1, '69.540')] [2023-10-14 19:30:14,266][61552] Updated weights for policy 0, policy_version 45412 (0.0009) [2023-10-14 19:30:14,631][61552] Updated weights for policy 0, policy_version 45422 (0.0008) [2023-10-14 19:30:15,001][61552] Updated weights for policy 0, policy_version 45432 (0.0010) [2023-10-14 19:30:16,100][61585] Updated weights for policy 1, policy_version 45250 (0.0008) [2023-10-14 19:30:16,467][61585] Updated weights for policy 1, policy_version 45260 (0.0008) [2023-10-14 19:30:16,834][61585] Updated weights for policy 1, policy_version 45270 (0.0008) [2023-10-14 19:30:17,194][61585] Updated weights for policy 1, policy_version 45280 (0.0011) [2023-10-14 19:30:18,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 92897280. Throughput: 0: 1661.5, 1: 1682.2. Samples: 23227508. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:30:18,344][60425] Avg episode reward: [(0, '73.750'), (1, '68.610')] [2023-10-14 19:30:19,093][61552] Updated weights for policy 0, policy_version 45442 (0.0009) [2023-10-14 19:30:19,464][61552] Updated weights for policy 0, policy_version 45452 (0.0009) [2023-10-14 19:30:19,848][61552] Updated weights for policy 0, policy_version 45462 (0.0011) [2023-10-14 19:30:20,218][61552] Updated weights for policy 0, policy_version 45472 (0.0010) [2023-10-14 19:30:21,329][61585] Updated weights for policy 1, policy_version 45290 (0.0008) [2023-10-14 19:30:21,700][61585] Updated weights for policy 1, policy_version 45300 (0.0008) [2023-10-14 19:30:22,061][61585] Updated weights for policy 1, policy_version 45310 (0.0007) [2023-10-14 19:30:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 92962816. Throughput: 0: 1670.1, 1: 1661.5. Samples: 23247076. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:30:23,345][60425] Avg episode reward: [(0, '72.260'), (1, '67.310')] [2023-10-14 19:30:24,311][61552] Updated weights for policy 0, policy_version 45482 (0.0010) [2023-10-14 19:30:24,670][61552] Updated weights for policy 0, policy_version 45492 (0.0008) [2023-10-14 19:30:25,038][61552] Updated weights for policy 0, policy_version 45502 (0.0010) [2023-10-14 19:30:26,126][61585] Updated weights for policy 1, policy_version 45320 (0.0008) [2023-10-14 19:30:26,504][61585] Updated weights for policy 1, policy_version 45330 (0.0010) [2023-10-14 19:30:26,862][61585] Updated weights for policy 1, policy_version 45340 (0.0009) [2023-10-14 19:30:28,343][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 93028352. Throughput: 0: 1666.5, 1: 1678.3. Samples: 23267118. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:30:28,344][60425] Avg episode reward: [(0, '71.310'), (1, '67.750')] [2023-10-14 19:30:29,166][61552] Updated weights for policy 0, policy_version 45512 (0.0007) [2023-10-14 19:30:29,530][61552] Updated weights for policy 0, policy_version 45522 (0.0007) [2023-10-14 19:30:29,896][61552] Updated weights for policy 0, policy_version 45532 (0.0009) [2023-10-14 19:30:31,096][61585] Updated weights for policy 1, policy_version 45350 (0.0010) [2023-10-14 19:30:31,464][61585] Updated weights for policy 1, policy_version 45360 (0.0010) [2023-10-14 19:30:31,823][61585] Updated weights for policy 1, policy_version 45370 (0.0011) [2023-10-14 19:30:33,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 93093888. Throughput: 0: 1665.1, 1: 1683.8. Samples: 23277540. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:30:33,344][60425] Avg episode reward: [(0, '67.920'), (1, '68.000')] [2023-10-14 19:30:34,059][61552] Updated weights for policy 0, policy_version 45542 (0.0009) [2023-10-14 19:30:34,442][61552] Updated weights for policy 0, policy_version 45552 (0.0010) [2023-10-14 19:30:34,801][61552] Updated weights for policy 0, policy_version 45562 (0.0011) [2023-10-14 19:30:35,933][61585] Updated weights for policy 1, policy_version 45380 (0.0009) [2023-10-14 19:30:36,299][61585] Updated weights for policy 1, policy_version 45390 (0.0010) [2023-10-14 19:30:36,660][61585] Updated weights for policy 1, policy_version 45400 (0.0011) [2023-10-14 19:30:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 93159424. Throughput: 0: 1670.4, 1: 1659.8. Samples: 23297078. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:30:38,344][60425] Avg episode reward: [(0, '71.250'), (1, '68.250')] [2023-10-14 19:30:38,842][61552] Updated weights for policy 0, policy_version 45572 (0.0009) [2023-10-14 19:30:39,215][61552] Updated weights for policy 0, policy_version 45582 (0.0008) [2023-10-14 19:30:39,590][61552] Updated weights for policy 0, policy_version 45592 (0.0009) [2023-10-14 19:30:40,729][61585] Updated weights for policy 1, policy_version 45410 (0.0010) [2023-10-14 19:30:41,089][61585] Updated weights for policy 1, policy_version 45420 (0.0008) [2023-10-14 19:30:41,457][61585] Updated weights for policy 1, policy_version 45430 (0.0009) [2023-10-14 19:30:41,820][61585] Updated weights for policy 1, policy_version 45440 (0.0009) [2023-10-14 19:30:43,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 93224960. Throughput: 0: 1671.9, 1: 1677.1. Samples: 23317516. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:30:43,344][60425] Avg episode reward: [(0, '74.390'), (1, '63.990')] [2023-10-14 19:30:43,590][61552] Updated weights for policy 0, policy_version 45602 (0.0007) [2023-10-14 19:30:43,961][61552] Updated weights for policy 0, policy_version 45612 (0.0009) [2023-10-14 19:30:44,321][61552] Updated weights for policy 0, policy_version 45622 (0.0008) [2023-10-14 19:30:44,688][61552] Updated weights for policy 0, policy_version 45632 (0.0008) [2023-10-14 19:30:45,974][61585] Updated weights for policy 1, policy_version 45450 (0.0007) [2023-10-14 19:30:46,350][61585] Updated weights for policy 1, policy_version 45460 (0.0010) [2023-10-14 19:30:46,715][61585] Updated weights for policy 1, policy_version 45470 (0.0007) [2023-10-14 19:30:48,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 93290496. Throughput: 0: 1670.3, 1: 1669.5. Samples: 23327502. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:30:48,344][60425] Avg episode reward: [(0, '67.420'), (1, '67.410')] [2023-10-14 19:30:48,824][61552] Updated weights for policy 0, policy_version 45642 (0.0011) [2023-10-14 19:30:49,191][61552] Updated weights for policy 0, policy_version 45652 (0.0009) [2023-10-14 19:30:49,556][61552] Updated weights for policy 0, policy_version 45662 (0.0011) [2023-10-14 19:30:50,820][61585] Updated weights for policy 1, policy_version 45480 (0.0008) [2023-10-14 19:30:51,199][61585] Updated weights for policy 1, policy_version 45490 (0.0010) [2023-10-14 19:30:51,562][61585] Updated weights for policy 1, policy_version 45500 (0.0010) [2023-10-14 19:30:53,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 93356032. Throughput: 0: 1671.8, 1: 1651.8. Samples: 23346876. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:30:53,344][60425] Avg episode reward: [(0, '67.700'), (1, '66.230')] [2023-10-14 19:30:53,846][61552] Updated weights for policy 0, policy_version 45672 (0.0008) [2023-10-14 19:30:54,218][61552] Updated weights for policy 0, policy_version 45682 (0.0007) [2023-10-14 19:30:54,574][61552] Updated weights for policy 0, policy_version 45692 (0.0009) [2023-10-14 19:30:55,644][61585] Updated weights for policy 1, policy_version 45510 (0.0009) [2023-10-14 19:30:56,001][61585] Updated weights for policy 1, policy_version 45520 (0.0012) [2023-10-14 19:30:56,364][61585] Updated weights for policy 1, policy_version 45530 (0.0010) [2023-10-14 19:30:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 93421568. Throughput: 0: 1673.4, 1: 1668.8. Samples: 23367602. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:30:58,344][60425] Avg episode reward: [(0, '63.760'), (1, '68.030')] [2023-10-14 19:30:58,661][61552] Updated weights for policy 0, policy_version 45702 (0.0008) [2023-10-14 19:30:59,017][61552] Updated weights for policy 0, policy_version 45712 (0.0010) [2023-10-14 19:30:59,388][61552] Updated weights for policy 0, policy_version 45722 (0.0008) [2023-10-14 19:31:00,492][61585] Updated weights for policy 1, policy_version 45540 (0.0009) [2023-10-14 19:31:00,861][61585] Updated weights for policy 1, policy_version 45550 (0.0012) [2023-10-14 19:31:01,219][61585] Updated weights for policy 1, policy_version 45560 (0.0010) [2023-10-14 19:31:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 93487104. Throughput: 0: 1673.4, 1: 1660.6. Samples: 23377540. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:31:03,344][60425] Avg episode reward: [(0, '68.590'), (1, '68.670')] [2023-10-14 19:31:03,419][61552] Updated weights for policy 0, policy_version 45732 (0.0008) [2023-10-14 19:31:03,783][61552] Updated weights for policy 0, policy_version 45742 (0.0008) [2023-10-14 19:31:04,149][61552] Updated weights for policy 0, policy_version 45752 (0.0008) [2023-10-14 19:31:05,316][61585] Updated weights for policy 1, policy_version 45570 (0.0010) [2023-10-14 19:31:05,682][61585] Updated weights for policy 1, policy_version 45580 (0.0007) [2023-10-14 19:31:06,054][61585] Updated weights for policy 1, policy_version 45590 (0.0008) [2023-10-14 19:31:06,420][61585] Updated weights for policy 1, policy_version 45600 (0.0009) [2023-10-14 19:31:08,119][61552] Updated weights for policy 0, policy_version 45762 (0.0008) [2023-10-14 19:31:08,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 93552640. Throughput: 0: 1680.0, 1: 1665.6. Samples: 23397628. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:31:08,344][60425] Avg episode reward: [(0, '69.210'), (1, '68.760')] [2023-10-14 19:31:08,510][61552] Updated weights for policy 0, policy_version 45772 (0.0010) [2023-10-14 19:31:08,888][61552] Updated weights for policy 0, policy_version 45782 (0.0010) [2023-10-14 19:31:09,252][61552] Updated weights for policy 0, policy_version 45792 (0.0009) [2023-10-14 19:31:10,558][61585] Updated weights for policy 1, policy_version 45610 (0.0010) [2023-10-14 19:31:10,938][61585] Updated weights for policy 1, policy_version 45620 (0.0009) [2023-10-14 19:31:11,312][61585] Updated weights for policy 1, policy_version 45630 (0.0009) [2023-10-14 19:31:13,344][60425] Fps is (10 sec: 13106.7, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 93618176. Throughput: 0: 1684.1, 1: 1671.5. Samples: 23418120. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:31:13,345][60425] Avg episode reward: [(0, '68.320'), (1, '69.660')] [2023-10-14 19:31:13,463][61552] Updated weights for policy 0, policy_version 45802 (0.0008) [2023-10-14 19:31:13,824][61552] Updated weights for policy 0, policy_version 45812 (0.0010) [2023-10-14 19:31:14,190][61552] Updated weights for policy 0, policy_version 45822 (0.0010) [2023-10-14 19:31:15,383][61585] Updated weights for policy 1, policy_version 45640 (0.0007) [2023-10-14 19:31:15,751][61585] Updated weights for policy 1, policy_version 45650 (0.0009) [2023-10-14 19:31:16,112][61585] Updated weights for policy 1, policy_version 45660 (0.0007) [2023-10-14 19:31:18,252][61552] Updated weights for policy 0, policy_version 45832 (0.0009) [2023-10-14 19:31:18,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 93683712. Throughput: 0: 1684.8, 1: 1655.7. Samples: 23427860. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:31:18,344][60425] Avg episode reward: [(0, '71.170'), (1, '71.140')] [2023-10-14 19:31:18,625][61552] Updated weights for policy 0, policy_version 45842 (0.0008) [2023-10-14 19:31:18,991][61552] Updated weights for policy 0, policy_version 45852 (0.0009) [2023-10-14 19:31:20,216][61585] Updated weights for policy 1, policy_version 45670 (0.0008) [2023-10-14 19:31:20,586][61585] Updated weights for policy 1, policy_version 45680 (0.0009) [2023-10-14 19:31:20,956][61585] Updated weights for policy 1, policy_version 45690 (0.0010) [2023-10-14 19:31:23,059][61552] Updated weights for policy 0, policy_version 45862 (0.0009) [2023-10-14 19:31:23,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 93749248. Throughput: 0: 1682.8, 1: 1666.7. Samples: 23447806. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:31:23,344][60425] Avg episode reward: [(0, '69.020'), (1, '72.440')] [2023-10-14 19:31:23,418][61552] Updated weights for policy 0, policy_version 45872 (0.0008) [2023-10-14 19:31:23,790][61552] Updated weights for policy 0, policy_version 45882 (0.0010) [2023-10-14 19:31:25,038][61585] Updated weights for policy 1, policy_version 45700 (0.0010) [2023-10-14 19:31:25,404][61585] Updated weights for policy 1, policy_version 45710 (0.0008) [2023-10-14 19:31:25,770][61585] Updated weights for policy 1, policy_version 45720 (0.0009) [2023-10-14 19:31:27,766][61552] Updated weights for policy 0, policy_version 45892 (0.0009) [2023-10-14 19:31:28,125][61552] Updated weights for policy 0, policy_version 45902 (0.0008) [2023-10-14 19:31:28,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 93814784. Throughput: 0: 1675.7, 1: 1675.3. Samples: 23468314. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:31:28,344][60425] Avg episode reward: [(0, '71.370'), (1, '68.750')] [2023-10-14 19:31:28,496][61552] Updated weights for policy 0, policy_version 45912 (0.0009) [2023-10-14 19:31:29,817][61585] Updated weights for policy 1, policy_version 45730 (0.0009) [2023-10-14 19:31:30,183][61585] Updated weights for policy 1, policy_version 45740 (0.0009) [2023-10-14 19:31:30,545][61585] Updated weights for policy 1, policy_version 45750 (0.0007) [2023-10-14 19:31:30,919][61585] Updated weights for policy 1, policy_version 45760 (0.0010) [2023-10-14 19:31:32,541][61552] Updated weights for policy 0, policy_version 45922 (0.0008) [2023-10-14 19:31:32,911][61552] Updated weights for policy 0, policy_version 45932 (0.0009) [2023-10-14 19:31:33,284][61552] Updated weights for policy 0, policy_version 45942 (0.0009) [2023-10-14 19:31:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 93880320. Throughput: 0: 1682.9, 1: 1657.8. Samples: 23477834. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:31:33,344][60425] Avg episode reward: [(0, '73.490'), (1, '66.580')] [2023-10-14 19:31:33,647][61552] Updated weights for policy 0, policy_version 45952 (0.0008) [2023-10-14 19:31:34,896][61585] Updated weights for policy 1, policy_version 45770 (0.0007) [2023-10-14 19:31:35,269][61585] Updated weights for policy 1, policy_version 45780 (0.0007) [2023-10-14 19:31:35,635][61585] Updated weights for policy 1, policy_version 45790 (0.0008) [2023-10-14 19:31:37,665][61552] Updated weights for policy 0, policy_version 45962 (0.0008) [2023-10-14 19:31:38,029][61552] Updated weights for policy 0, policy_version 45972 (0.0007) [2023-10-14 19:31:38,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 93945856. Throughput: 0: 1686.4, 1: 1675.5. Samples: 23498162. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:31:38,344][60425] Avg episode reward: [(0, '75.710'), (1, '70.170')] [2023-10-14 19:31:38,395][61552] Updated weights for policy 0, policy_version 45982 (0.0009) [2023-10-14 19:31:39,749][61585] Updated weights for policy 1, policy_version 45800 (0.0008) [2023-10-14 19:31:40,113][61585] Updated weights for policy 1, policy_version 45810 (0.0007) [2023-10-14 19:31:40,477][61585] Updated weights for policy 1, policy_version 45820 (0.0008) [2023-10-14 19:31:42,570][61552] Updated weights for policy 0, policy_version 45992 (0.0008) [2023-10-14 19:31:42,942][61552] Updated weights for policy 0, policy_version 46002 (0.0010) [2023-10-14 19:31:43,309][61552] Updated weights for policy 0, policy_version 46012 (0.0010) [2023-10-14 19:31:43,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 94011392. Throughput: 0: 1676.1, 1: 1677.2. Samples: 23518502. Policy #0 lag: (min: 1.0, avg: 7.3, max: 33.0) [2023-10-14 19:31:43,344][60425] Avg episode reward: [(0, '70.140'), (1, '68.200')] [2023-10-14 19:31:43,355][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000045824_46923776.pth... [2023-10-14 19:31:43,392][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000044256_45318144.pth [2023-10-14 19:31:43,457][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000046016_47120384.pth... [2023-10-14 19:31:43,485][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000044448_45514752.pth [2023-10-14 19:31:44,574][61585] Updated weights for policy 1, policy_version 45830 (0.0007) [2023-10-14 19:31:44,943][61585] Updated weights for policy 1, policy_version 45840 (0.0007) [2023-10-14 19:31:45,315][61585] Updated weights for policy 1, policy_version 45850 (0.0007) [2023-10-14 19:31:47,291][61552] Updated weights for policy 0, policy_version 46022 (0.0009) [2023-10-14 19:31:47,654][61552] Updated weights for policy 0, policy_version 46032 (0.0009) [2023-10-14 19:31:48,019][61552] Updated weights for policy 0, policy_version 46042 (0.0010) [2023-10-14 19:31:48,343][60425] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 94109696. Throughput: 0: 1689.6, 1: 1655.7. Samples: 23528078. Policy #0 lag: (min: 1.0, avg: 7.3, max: 33.0) [2023-10-14 19:31:48,344][60425] Avg episode reward: [(0, '71.910'), (1, '73.120')] [2023-10-14 19:31:48,345][61248] Saving new best policy, reward=73.120! [2023-10-14 19:31:49,519][61585] Updated weights for policy 1, policy_version 45860 (0.0007) [2023-10-14 19:31:49,878][61585] Updated weights for policy 1, policy_version 45870 (0.0007) [2023-10-14 19:31:50,245][61585] Updated weights for policy 1, policy_version 45880 (0.0007) [2023-10-14 19:31:52,152][61552] Updated weights for policy 0, policy_version 46052 (0.0009) [2023-10-14 19:31:52,523][61552] Updated weights for policy 0, policy_version 46062 (0.0009) [2023-10-14 19:31:52,891][61552] Updated weights for policy 0, policy_version 46072 (0.0008) [2023-10-14 19:31:53,343][60425] Fps is (10 sec: 16384.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 94175232. Throughput: 0: 1681.9, 1: 1669.8. Samples: 23548454. Policy #0 lag: (min: 1.0, avg: 7.3, max: 33.0) [2023-10-14 19:31:53,344][60425] Avg episode reward: [(0, '69.430'), (1, '72.100')] [2023-10-14 19:31:54,456][61585] Updated weights for policy 1, policy_version 45890 (0.0008) [2023-10-14 19:31:54,858][61585] Updated weights for policy 1, policy_version 45900 (0.0010) [2023-10-14 19:31:55,214][61585] Updated weights for policy 1, policy_version 45910 (0.0009) [2023-10-14 19:31:55,573][61585] Updated weights for policy 1, policy_version 45920 (0.0009) [2023-10-14 19:31:56,954][61552] Updated weights for policy 0, policy_version 46082 (0.0009) [2023-10-14 19:31:57,353][61552] Updated weights for policy 0, policy_version 46092 (0.0010) [2023-10-14 19:31:57,727][61552] Updated weights for policy 0, policy_version 46102 (0.0009) [2023-10-14 19:31:58,088][61552] Updated weights for policy 0, policy_version 46112 (0.0008) [2023-10-14 19:31:58,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 94240768. Throughput: 0: 1663.4, 1: 1670.5. Samples: 23568146. Policy #0 lag: (min: 1.0, avg: 7.3, max: 33.0) [2023-10-14 19:31:58,344][60425] Avg episode reward: [(0, '72.560'), (1, '71.020')] [2023-10-14 19:31:59,530][61585] Updated weights for policy 1, policy_version 45930 (0.0008) [2023-10-14 19:31:59,896][61585] Updated weights for policy 1, policy_version 45940 (0.0009) [2023-10-14 19:32:00,271][61585] Updated weights for policy 1, policy_version 45950 (0.0009) [2023-10-14 19:32:02,150][61552] Updated weights for policy 0, policy_version 46122 (0.0011) [2023-10-14 19:32:02,508][61552] Updated weights for policy 0, policy_version 46132 (0.0010) [2023-10-14 19:32:02,894][61552] Updated weights for policy 0, policy_version 46142 (0.0010) [2023-10-14 19:32:03,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 94306304. Throughput: 0: 1684.6, 1: 1658.8. Samples: 23578314. Policy #0 lag: (min: 1.0, avg: 7.3, max: 33.0) [2023-10-14 19:32:03,344][60425] Avg episode reward: [(0, '73.120'), (1, '67.900')] [2023-10-14 19:32:04,535][61585] Updated weights for policy 1, policy_version 45960 (0.0008) [2023-10-14 19:32:04,887][61585] Updated weights for policy 1, policy_version 45970 (0.0010) [2023-10-14 19:32:05,247][61585] Updated weights for policy 1, policy_version 45980 (0.0009) [2023-10-14 19:32:07,062][61552] Updated weights for policy 0, policy_version 46152 (0.0009) [2023-10-14 19:32:07,425][61552] Updated weights for policy 0, policy_version 46162 (0.0008) [2023-10-14 19:32:07,791][61552] Updated weights for policy 0, policy_version 46172 (0.0008) [2023-10-14 19:32:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 94371840. Throughput: 0: 1685.5, 1: 1666.6. Samples: 23598650. Policy #0 lag: (min: 0.0, avg: 21.5, max: 32.0) [2023-10-14 19:32:08,344][60425] Avg episode reward: [(0, '72.340'), (1, '75.520')] [2023-10-14 19:32:08,344][61248] Saving new best policy, reward=75.520! [2023-10-14 19:32:09,405][61585] Updated weights for policy 1, policy_version 45990 (0.0009) [2023-10-14 19:32:09,769][61585] Updated weights for policy 1, policy_version 46000 (0.0009) [2023-10-14 19:32:10,132][61585] Updated weights for policy 1, policy_version 46010 (0.0009) [2023-10-14 19:32:11,838][61552] Updated weights for policy 0, policy_version 46182 (0.0007) [2023-10-14 19:32:12,204][61552] Updated weights for policy 0, policy_version 46192 (0.0008) [2023-10-14 19:32:12,575][61552] Updated weights for policy 0, policy_version 46202 (0.0009) [2023-10-14 19:32:13,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 94437376. Throughput: 0: 1664.8, 1: 1666.9. Samples: 23618242. Policy #0 lag: (min: 0.0, avg: 21.5, max: 32.0) [2023-10-14 19:32:13,344][60425] Avg episode reward: [(0, '66.120'), (1, '73.380')] [2023-10-14 19:32:14,271][61585] Updated weights for policy 1, policy_version 46020 (0.0008) [2023-10-14 19:32:14,629][61585] Updated weights for policy 1, policy_version 46030 (0.0009) [2023-10-14 19:32:14,991][61585] Updated weights for policy 1, policy_version 46040 (0.0010) [2023-10-14 19:32:16,498][61552] Updated weights for policy 0, policy_version 46212 (0.0008) [2023-10-14 19:32:16,862][61552] Updated weights for policy 0, policy_version 46222 (0.0011) [2023-10-14 19:32:17,232][61552] Updated weights for policy 0, policy_version 46232 (0.0011) [2023-10-14 19:32:18,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 94502912. Throughput: 0: 1686.9, 1: 1661.4. Samples: 23628508. Policy #0 lag: (min: 0.0, avg: 21.5, max: 32.0) [2023-10-14 19:32:18,344][60425] Avg episode reward: [(0, '70.270'), (1, '70.030')] [2023-10-14 19:32:19,104][61585] Updated weights for policy 1, policy_version 46050 (0.0009) [2023-10-14 19:32:19,469][61585] Updated weights for policy 1, policy_version 46060 (0.0008) [2023-10-14 19:32:19,832][61585] Updated weights for policy 1, policy_version 46070 (0.0008) [2023-10-14 19:32:20,196][61585] Updated weights for policy 1, policy_version 46080 (0.0008) [2023-10-14 19:32:21,119][61552] Updated weights for policy 0, policy_version 46242 (0.0010) [2023-10-14 19:32:21,484][61552] Updated weights for policy 0, policy_version 46252 (0.0008) [2023-10-14 19:32:21,848][61552] Updated weights for policy 0, policy_version 46262 (0.0007) [2023-10-14 19:32:22,224][61552] Updated weights for policy 0, policy_version 46272 (0.0007) [2023-10-14 19:32:23,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 94568448. Throughput: 0: 1673.0, 1: 1675.8. Samples: 23648856. Policy #0 lag: (min: 0.0, avg: 21.5, max: 32.0) [2023-10-14 19:32:23,344][60425] Avg episode reward: [(0, '72.690'), (1, '72.610')] [2023-10-14 19:32:24,026][61585] Updated weights for policy 1, policy_version 46090 (0.0008) [2023-10-14 19:32:24,390][61585] Updated weights for policy 1, policy_version 46100 (0.0007) [2023-10-14 19:32:24,760][61585] Updated weights for policy 1, policy_version 46110 (0.0007) [2023-10-14 19:32:26,314][61552] Updated weights for policy 0, policy_version 46282 (0.0010) [2023-10-14 19:32:26,680][61552] Updated weights for policy 0, policy_version 46292 (0.0010) [2023-10-14 19:32:27,048][61552] Updated weights for policy 0, policy_version 46302 (0.0008) [2023-10-14 19:32:28,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 94633984. Throughput: 0: 1664.8, 1: 1681.2. Samples: 23669072. Policy #0 lag: (min: 0.0, avg: 21.5, max: 32.0) [2023-10-14 19:32:28,345][60425] Avg episode reward: [(0, '67.550'), (1, '69.650')] [2023-10-14 19:32:28,713][61585] Updated weights for policy 1, policy_version 46120 (0.0007) [2023-10-14 19:32:29,069][61585] Updated weights for policy 1, policy_version 46130 (0.0009) [2023-10-14 19:32:29,432][61585] Updated weights for policy 1, policy_version 46140 (0.0009) [2023-10-14 19:32:31,372][61552] Updated weights for policy 0, policy_version 46312 (0.0008) [2023-10-14 19:32:31,742][61552] Updated weights for policy 0, policy_version 46322 (0.0008) [2023-10-14 19:32:32,120][61552] Updated weights for policy 0, policy_version 46332 (0.0009) [2023-10-14 19:32:33,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 94699520. Throughput: 0: 1680.0, 1: 1683.9. Samples: 23679450. Policy #0 lag: (min: 0.0, avg: 21.5, max: 32.0) [2023-10-14 19:32:33,344][60425] Avg episode reward: [(0, '70.900'), (1, '65.240')] [2023-10-14 19:32:33,520][61585] Updated weights for policy 1, policy_version 46150 (0.0007) [2023-10-14 19:32:33,888][61585] Updated weights for policy 1, policy_version 46160 (0.0008) [2023-10-14 19:32:34,253][61585] Updated weights for policy 1, policy_version 46170 (0.0008) [2023-10-14 19:32:36,276][61552] Updated weights for policy 0, policy_version 46342 (0.0009) [2023-10-14 19:32:36,638][61552] Updated weights for policy 0, policy_version 46352 (0.0009) [2023-10-14 19:32:37,010][61552] Updated weights for policy 0, policy_version 46362 (0.0010) [2023-10-14 19:32:38,276][61585] Updated weights for policy 1, policy_version 46180 (0.0008) [2023-10-14 19:32:38,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 94765056. Throughput: 0: 1667.6, 1: 1681.7. Samples: 23699172. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:32:38,344][60425] Avg episode reward: [(0, '69.530'), (1, '69.430')] [2023-10-14 19:32:38,646][61585] Updated weights for policy 1, policy_version 46190 (0.0009) [2023-10-14 19:32:39,002][61585] Updated weights for policy 1, policy_version 46200 (0.0009) [2023-10-14 19:32:40,936][61552] Updated weights for policy 0, policy_version 46372 (0.0008) [2023-10-14 19:32:41,307][61552] Updated weights for policy 0, policy_version 46382 (0.0009) [2023-10-14 19:32:41,680][61552] Updated weights for policy 0, policy_version 46392 (0.0008) [2023-10-14 19:32:43,105][61585] Updated weights for policy 1, policy_version 46210 (0.0008) [2023-10-14 19:32:43,343][60425] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 94830592. Throughput: 0: 1673.5, 1: 1688.3. Samples: 23719426. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:32:43,345][60425] Avg episode reward: [(0, '71.860'), (1, '72.640')] [2023-10-14 19:32:43,521][61585] Updated weights for policy 1, policy_version 46220 (0.0008) [2023-10-14 19:32:43,881][61585] Updated weights for policy 1, policy_version 46230 (0.0008) [2023-10-14 19:32:44,250][61585] Updated weights for policy 1, policy_version 46240 (0.0009) [2023-10-14 19:32:45,712][61552] Updated weights for policy 0, policy_version 46402 (0.0007) [2023-10-14 19:32:46,113][61552] Updated weights for policy 0, policy_version 46412 (0.0008) [2023-10-14 19:32:46,480][61552] Updated weights for policy 0, policy_version 46422 (0.0007) [2023-10-14 19:32:46,848][61552] Updated weights for policy 0, policy_version 46432 (0.0007) [2023-10-14 19:32:48,342][61585] Updated weights for policy 1, policy_version 46250 (0.0009) [2023-10-14 19:32:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 94896128. Throughput: 0: 1676.8, 1: 1684.4. Samples: 23729566. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:32:48,344][60425] Avg episode reward: [(0, '71.830'), (1, '70.160')] [2023-10-14 19:32:48,705][61585] Updated weights for policy 1, policy_version 46260 (0.0010) [2023-10-14 19:32:49,078][61585] Updated weights for policy 1, policy_version 46270 (0.0011) [2023-10-14 19:32:50,867][61552] Updated weights for policy 0, policy_version 46442 (0.0011) [2023-10-14 19:32:51,238][61552] Updated weights for policy 0, policy_version 46452 (0.0010) [2023-10-14 19:32:51,598][61552] Updated weights for policy 0, policy_version 46462 (0.0007) [2023-10-14 19:32:53,239][61585] Updated weights for policy 1, policy_version 46280 (0.0010) [2023-10-14 19:32:53,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 94961664. Throughput: 0: 1651.6, 1: 1686.8. Samples: 23748876. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:32:53,344][60425] Avg episode reward: [(0, '70.000'), (1, '72.440')] [2023-10-14 19:32:53,602][61585] Updated weights for policy 1, policy_version 46290 (0.0009) [2023-10-14 19:32:53,965][61585] Updated weights for policy 1, policy_version 46300 (0.0010) [2023-10-14 19:32:55,768][61552] Updated weights for policy 0, policy_version 46472 (0.0007) [2023-10-14 19:32:56,132][61552] Updated weights for policy 0, policy_version 46482 (0.0009) [2023-10-14 19:32:56,498][61552] Updated weights for policy 0, policy_version 46492 (0.0007) [2023-10-14 19:32:58,028][61585] Updated weights for policy 1, policy_version 46310 (0.0009) [2023-10-14 19:32:58,344][60425] Fps is (10 sec: 13106.7, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 95027200. Throughput: 0: 1679.2, 1: 1689.3. Samples: 23769826. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:32:58,345][60425] Avg episode reward: [(0, '70.170'), (1, '73.440')] [2023-10-14 19:32:58,390][61585] Updated weights for policy 1, policy_version 46320 (0.0008) [2023-10-14 19:32:58,765][61585] Updated weights for policy 1, policy_version 46330 (0.0010) [2023-10-14 19:33:00,544][61552] Updated weights for policy 0, policy_version 46502 (0.0009) [2023-10-14 19:33:00,913][61552] Updated weights for policy 0, policy_version 46512 (0.0008) [2023-10-14 19:33:01,280][61552] Updated weights for policy 0, policy_version 46522 (0.0009) [2023-10-14 19:33:02,756][61585] Updated weights for policy 1, policy_version 46340 (0.0009) [2023-10-14 19:33:03,110][61585] Updated weights for policy 1, policy_version 46350 (0.0007) [2023-10-14 19:33:03,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 95092736. Throughput: 0: 1671.7, 1: 1688.8. Samples: 23779734. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:33:03,345][60425] Avg episode reward: [(0, '73.230'), (1, '74.420')] [2023-10-14 19:33:03,474][61585] Updated weights for policy 1, policy_version 46360 (0.0008) [2023-10-14 19:33:05,373][61552] Updated weights for policy 0, policy_version 46532 (0.0008) [2023-10-14 19:33:05,750][61552] Updated weights for policy 0, policy_version 46542 (0.0007) [2023-10-14 19:33:06,122][61552] Updated weights for policy 0, policy_version 46552 (0.0007) [2023-10-14 19:33:07,695][61585] Updated weights for policy 1, policy_version 46370 (0.0011) [2023-10-14 19:33:08,055][61585] Updated weights for policy 1, policy_version 46380 (0.0009) [2023-10-14 19:33:08,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 95158272. Throughput: 0: 1659.1, 1: 1685.5. Samples: 23799364. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:33:08,345][60425] Avg episode reward: [(0, '70.050'), (1, '68.710')] [2023-10-14 19:33:08,419][61585] Updated weights for policy 1, policy_version 46390 (0.0009) [2023-10-14 19:33:08,783][61585] Updated weights for policy 1, policy_version 46400 (0.0009) [2023-10-14 19:33:10,177][61552] Updated weights for policy 0, policy_version 46562 (0.0008) [2023-10-14 19:33:10,545][61552] Updated weights for policy 0, policy_version 46572 (0.0008) [2023-10-14 19:33:10,922][61552] Updated weights for policy 0, policy_version 46582 (0.0009) [2023-10-14 19:33:11,289][61552] Updated weights for policy 0, policy_version 46592 (0.0008) [2023-10-14 19:33:12,849][61585] Updated weights for policy 1, policy_version 46410 (0.0010) [2023-10-14 19:33:13,217][61585] Updated weights for policy 1, policy_version 46420 (0.0011) [2023-10-14 19:33:13,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 95223808. Throughput: 0: 1675.8, 1: 1669.3. Samples: 23819600. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:33:13,344][60425] Avg episode reward: [(0, '70.910'), (1, '67.480')] [2023-10-14 19:33:13,587][61585] Updated weights for policy 1, policy_version 46430 (0.0010) [2023-10-14 19:33:15,544][61552] Updated weights for policy 0, policy_version 46602 (0.0009) [2023-10-14 19:33:15,906][61552] Updated weights for policy 0, policy_version 46612 (0.0009) [2023-10-14 19:33:16,280][61552] Updated weights for policy 0, policy_version 46622 (0.0007) [2023-10-14 19:33:17,726][61585] Updated weights for policy 1, policy_version 46440 (0.0009) [2023-10-14 19:33:18,083][61585] Updated weights for policy 1, policy_version 46450 (0.0009) [2023-10-14 19:33:18,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 95289344. Throughput: 0: 1663.7, 1: 1675.2. Samples: 23829698. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:33:18,344][60425] Avg episode reward: [(0, '70.180'), (1, '69.860')] [2023-10-14 19:33:18,452][61585] Updated weights for policy 1, policy_version 46460 (0.0009) [2023-10-14 19:33:20,329][61552] Updated weights for policy 0, policy_version 46632 (0.0010) [2023-10-14 19:33:20,691][61552] Updated weights for policy 0, policy_version 46642 (0.0010) [2023-10-14 19:33:21,058][61552] Updated weights for policy 0, policy_version 46652 (0.0009) [2023-10-14 19:33:22,472][61585] Updated weights for policy 1, policy_version 46470 (0.0009) [2023-10-14 19:33:22,844][61585] Updated weights for policy 1, policy_version 46480 (0.0010) [2023-10-14 19:33:23,210][61585] Updated weights for policy 1, policy_version 46490 (0.0009) [2023-10-14 19:33:23,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 95354880. Throughput: 0: 1660.8, 1: 1680.4. Samples: 23849526. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:33:23,344][60425] Avg episode reward: [(0, '68.480'), (1, '72.950')] [2023-10-14 19:33:25,307][61552] Updated weights for policy 0, policy_version 46662 (0.0008) [2023-10-14 19:33:25,688][61552] Updated weights for policy 0, policy_version 46672 (0.0008) [2023-10-14 19:33:26,065][61552] Updated weights for policy 0, policy_version 46682 (0.0008) [2023-10-14 19:33:27,629][61585] Updated weights for policy 1, policy_version 46500 (0.0008) [2023-10-14 19:33:28,013][61585] Updated weights for policy 1, policy_version 46510 (0.0007) [2023-10-14 19:33:28,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 95420416. Throughput: 0: 1672.9, 1: 1666.9. Samples: 23869712. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:33:28,344][60425] Avg episode reward: [(0, '73.670'), (1, '71.060')] [2023-10-14 19:33:28,382][61585] Updated weights for policy 1, policy_version 46520 (0.0008) [2023-10-14 19:33:30,134][61552] Updated weights for policy 0, policy_version 46692 (0.0008) [2023-10-14 19:33:30,522][61552] Updated weights for policy 0, policy_version 46702 (0.0010) [2023-10-14 19:33:30,880][61552] Updated weights for policy 0, policy_version 46712 (0.0008) [2023-10-14 19:33:32,314][61585] Updated weights for policy 1, policy_version 46530 (0.0007) [2023-10-14 19:33:32,688][61585] Updated weights for policy 1, policy_version 46540 (0.0010) [2023-10-14 19:33:33,059][61585] Updated weights for policy 1, policy_version 46550 (0.0010) [2023-10-14 19:33:33,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 95485952. Throughput: 0: 1656.4, 1: 1676.7. Samples: 23879558. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:33:33,344][60425] Avg episode reward: [(0, '77.040'), (1, '70.350')] [2023-10-14 19:33:33,345][61172] Saving new best policy, reward=77.040! [2023-10-14 19:33:33,425][61585] Updated weights for policy 1, policy_version 46560 (0.0009) [2023-10-14 19:33:35,012][61552] Updated weights for policy 0, policy_version 46722 (0.0009) [2023-10-14 19:33:35,377][61552] Updated weights for policy 0, policy_version 46732 (0.0008) [2023-10-14 19:33:35,745][61552] Updated weights for policy 0, policy_version 46742 (0.0011) [2023-10-14 19:33:36,114][61552] Updated weights for policy 0, policy_version 46752 (0.0008) [2023-10-14 19:33:37,338][61585] Updated weights for policy 1, policy_version 46570 (0.0008) [2023-10-14 19:33:37,694][61585] Updated weights for policy 1, policy_version 46580 (0.0007) [2023-10-14 19:33:38,056][61585] Updated weights for policy 1, policy_version 46590 (0.0008) [2023-10-14 19:33:38,343][60425] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 95584256. Throughput: 0: 1669.1, 1: 1683.1. Samples: 23899724. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:33:38,344][60425] Avg episode reward: [(0, '76.980'), (1, '72.030')] [2023-10-14 19:33:40,235][61552] Updated weights for policy 0, policy_version 46762 (0.0010) [2023-10-14 19:33:40,611][61552] Updated weights for policy 0, policy_version 46772 (0.0011) [2023-10-14 19:33:40,987][61552] Updated weights for policy 0, policy_version 46782 (0.0008) [2023-10-14 19:33:41,984][61585] Updated weights for policy 1, policy_version 46600 (0.0008) [2023-10-14 19:33:42,343][61585] Updated weights for policy 1, policy_version 46610 (0.0009) [2023-10-14 19:33:42,716][61585] Updated weights for policy 1, policy_version 46620 (0.0007) [2023-10-14 19:33:43,344][60425] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 95649792. Throughput: 0: 1664.6, 1: 1660.8. Samples: 23919468. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:33:43,345][60425] Avg episode reward: [(0, '73.170'), (1, '70.460')] [2023-10-14 19:33:43,356][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000046784_47906816.pth... [2023-10-14 19:33:43,356][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000046624_47742976.pth... [2023-10-14 19:33:43,396][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000045056_46137344.pth [2023-10-14 19:33:43,397][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000045216_46301184.pth [2023-10-14 19:33:43,401][61248] Saving a milestone ./train_atari/atari_roadrunner_APPO/checkpoint_p1/milestones/checkpoint_000046624_47742976.pth [2023-10-14 19:33:43,401][61172] Saving a milestone ./train_atari/atari_roadrunner_APPO/checkpoint_p0/milestones/checkpoint_000046784_47906816.pth [2023-10-14 19:33:45,017][61552] Updated weights for policy 0, policy_version 46792 (0.0009) [2023-10-14 19:33:45,387][61552] Updated weights for policy 0, policy_version 46802 (0.0008) [2023-10-14 19:33:45,754][61552] Updated weights for policy 0, policy_version 46812 (0.0008) [2023-10-14 19:33:46,983][61585] Updated weights for policy 1, policy_version 46630 (0.0009) [2023-10-14 19:33:47,346][61585] Updated weights for policy 1, policy_version 46640 (0.0008) [2023-10-14 19:33:47,710][61585] Updated weights for policy 1, policy_version 46650 (0.0008) [2023-10-14 19:33:48,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 95715328. Throughput: 0: 1650.2, 1: 1682.4. Samples: 23929700. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:33:48,344][60425] Avg episode reward: [(0, '72.450'), (1, '72.360')] [2023-10-14 19:33:49,871][61552] Updated weights for policy 0, policy_version 46822 (0.0007) [2023-10-14 19:33:50,234][61552] Updated weights for policy 0, policy_version 46832 (0.0007) [2023-10-14 19:33:50,600][61552] Updated weights for policy 0, policy_version 46842 (0.0007) [2023-10-14 19:33:51,794][61585] Updated weights for policy 1, policy_version 46660 (0.0009) [2023-10-14 19:33:52,154][61585] Updated weights for policy 1, policy_version 46670 (0.0008) [2023-10-14 19:33:52,525][61585] Updated weights for policy 1, policy_version 46680 (0.0009) [2023-10-14 19:33:53,343][60425] Fps is (10 sec: 13107.8, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 95780864. Throughput: 0: 1668.3, 1: 1673.9. Samples: 23949764. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:33:53,344][60425] Avg episode reward: [(0, '73.140'), (1, '75.550')] [2023-10-14 19:33:53,344][61248] Saving new best policy, reward=75.550! [2023-10-14 19:33:54,824][61552] Updated weights for policy 0, policy_version 46852 (0.0010) [2023-10-14 19:33:55,193][61552] Updated weights for policy 0, policy_version 46862 (0.0009) [2023-10-14 19:33:55,563][61552] Updated weights for policy 0, policy_version 46872 (0.0009) [2023-10-14 19:33:56,708][61585] Updated weights for policy 1, policy_version 46690 (0.0010) [2023-10-14 19:33:57,075][61585] Updated weights for policy 1, policy_version 46700 (0.0010) [2023-10-14 19:33:57,437][61585] Updated weights for policy 1, policy_version 46710 (0.0011) [2023-10-14 19:33:57,808][61585] Updated weights for policy 1, policy_version 46720 (0.0010) [2023-10-14 19:33:58,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 95846400. Throughput: 0: 1665.2, 1: 1658.8. Samples: 23969180. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:33:58,344][60425] Avg episode reward: [(0, '75.920'), (1, '64.820')] [2023-10-14 19:33:59,738][61552] Updated weights for policy 0, policy_version 46882 (0.0009) [2023-10-14 19:34:00,108][61552] Updated weights for policy 0, policy_version 46892 (0.0011) [2023-10-14 19:34:00,477][61552] Updated weights for policy 0, policy_version 46902 (0.0007) [2023-10-14 19:34:00,837][61552] Updated weights for policy 0, policy_version 46912 (0.0012) [2023-10-14 19:34:01,769][61585] Updated weights for policy 1, policy_version 46730 (0.0009) [2023-10-14 19:34:02,131][61585] Updated weights for policy 1, policy_version 46740 (0.0008) [2023-10-14 19:34:02,495][61585] Updated weights for policy 1, policy_version 46750 (0.0009) [2023-10-14 19:34:03,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 95911936. Throughput: 0: 1651.7, 1: 1681.4. Samples: 23979688. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:34:03,344][60425] Avg episode reward: [(0, '71.910'), (1, '69.100')] [2023-10-14 19:34:04,740][61552] Updated weights for policy 0, policy_version 46922 (0.0008) [2023-10-14 19:34:05,103][61552] Updated weights for policy 0, policy_version 46932 (0.0008) [2023-10-14 19:34:05,475][61552] Updated weights for policy 0, policy_version 46942 (0.0008) [2023-10-14 19:34:06,636][61585] Updated weights for policy 1, policy_version 46760 (0.0010) [2023-10-14 19:34:07,007][61585] Updated weights for policy 1, policy_version 46770 (0.0009) [2023-10-14 19:34:07,378][61585] Updated weights for policy 1, policy_version 46780 (0.0007) [2023-10-14 19:34:08,344][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 95977472. Throughput: 0: 1661.9, 1: 1670.4. Samples: 23999484. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:34:08,345][60425] Avg episode reward: [(0, '74.050'), (1, '74.170')] [2023-10-14 19:34:09,386][61552] Updated weights for policy 0, policy_version 46952 (0.0008) [2023-10-14 19:34:09,750][61552] Updated weights for policy 0, policy_version 46962 (0.0009) [2023-10-14 19:34:10,115][61552] Updated weights for policy 0, policy_version 46972 (0.0009) [2023-10-14 19:34:11,526][61585] Updated weights for policy 1, policy_version 46790 (0.0008) [2023-10-14 19:34:11,906][61585] Updated weights for policy 1, policy_version 46800 (0.0009) [2023-10-14 19:34:12,273][61585] Updated weights for policy 1, policy_version 46810 (0.0008) [2023-10-14 19:34:13,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 96043008. Throughput: 0: 1666.6, 1: 1657.9. Samples: 24019316. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:34:13,345][60425] Avg episode reward: [(0, '71.110'), (1, '72.600')] [2023-10-14 19:34:14,323][61552] Updated weights for policy 0, policy_version 46982 (0.0009) [2023-10-14 19:34:14,694][61552] Updated weights for policy 0, policy_version 46992 (0.0010) [2023-10-14 19:34:15,056][61552] Updated weights for policy 0, policy_version 47002 (0.0009) [2023-10-14 19:34:16,300][61585] Updated weights for policy 1, policy_version 46820 (0.0009) [2023-10-14 19:34:16,695][61585] Updated weights for policy 1, policy_version 46830 (0.0008) [2023-10-14 19:34:17,059][61585] Updated weights for policy 1, policy_version 46840 (0.0007) [2023-10-14 19:34:18,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 96108544. Throughput: 0: 1652.6, 1: 1683.3. Samples: 24029674. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:34:18,344][60425] Avg episode reward: [(0, '72.090'), (1, '70.990')] [2023-10-14 19:34:19,479][61552] Updated weights for policy 0, policy_version 47012 (0.0009) [2023-10-14 19:34:19,856][61552] Updated weights for policy 0, policy_version 47022 (0.0009) [2023-10-14 19:34:20,226][61552] Updated weights for policy 0, policy_version 47032 (0.0011) [2023-10-14 19:34:20,993][61585] Updated weights for policy 1, policy_version 46850 (0.0009) [2023-10-14 19:34:21,353][61585] Updated weights for policy 1, policy_version 46860 (0.0010) [2023-10-14 19:34:21,723][61585] Updated weights for policy 1, policy_version 46870 (0.0010) [2023-10-14 19:34:22,085][61585] Updated weights for policy 1, policy_version 46880 (0.0009) [2023-10-14 19:34:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 96174080. Throughput: 0: 1659.1, 1: 1662.7. Samples: 24049204. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:34:23,344][60425] Avg episode reward: [(0, '75.050'), (1, '73.040')] [2023-10-14 19:34:24,317][61552] Updated weights for policy 0, policy_version 47042 (0.0007) [2023-10-14 19:34:24,690][61552] Updated weights for policy 0, policy_version 47052 (0.0008) [2023-10-14 19:34:25,059][61552] Updated weights for policy 0, policy_version 47062 (0.0007) [2023-10-14 19:34:25,423][61552] Updated weights for policy 0, policy_version 47072 (0.0008) [2023-10-14 19:34:26,164][61585] Updated weights for policy 1, policy_version 46890 (0.0012) [2023-10-14 19:34:26,523][61585] Updated weights for policy 1, policy_version 46900 (0.0009) [2023-10-14 19:34:26,894][61585] Updated weights for policy 1, policy_version 46910 (0.0008) [2023-10-14 19:34:28,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 96239616. Throughput: 0: 1658.8, 1: 1677.1. Samples: 24069582. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:34:28,345][60425] Avg episode reward: [(0, '69.610'), (1, '67.590')] [2023-10-14 19:34:29,616][61552] Updated weights for policy 0, policy_version 47082 (0.0011) [2023-10-14 19:34:29,988][61552] Updated weights for policy 0, policy_version 47092 (0.0009) [2023-10-14 19:34:30,370][61552] Updated weights for policy 0, policy_version 47102 (0.0008) [2023-10-14 19:34:30,954][61585] Updated weights for policy 1, policy_version 46920 (0.0008) [2023-10-14 19:34:31,316][61585] Updated weights for policy 1, policy_version 46930 (0.0009) [2023-10-14 19:34:31,686][61585] Updated weights for policy 1, policy_version 46940 (0.0009) [2023-10-14 19:34:33,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 96305152. Throughput: 0: 1652.9, 1: 1681.8. Samples: 24079762. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:34:33,344][60425] Avg episode reward: [(0, '72.520'), (1, '75.370')] [2023-10-14 19:34:34,546][61552] Updated weights for policy 0, policy_version 47112 (0.0010) [2023-10-14 19:34:34,921][61552] Updated weights for policy 0, policy_version 47122 (0.0007) [2023-10-14 19:34:35,299][61552] Updated weights for policy 0, policy_version 47132 (0.0008) [2023-10-14 19:34:35,659][61585] Updated weights for policy 1, policy_version 46950 (0.0010) [2023-10-14 19:34:36,037][61585] Updated weights for policy 1, policy_version 46960 (0.0009) [2023-10-14 19:34:36,395][61585] Updated weights for policy 1, policy_version 46970 (0.0010) [2023-10-14 19:34:38,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 96370688. Throughput: 0: 1665.9, 1: 1666.1. Samples: 24099706. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:34:38,344][60425] Avg episode reward: [(0, '73.170'), (1, '68.220')] [2023-10-14 19:34:39,339][61552] Updated weights for policy 0, policy_version 47142 (0.0009) [2023-10-14 19:34:39,717][61552] Updated weights for policy 0, policy_version 47152 (0.0008) [2023-10-14 19:34:40,081][61552] Updated weights for policy 0, policy_version 47162 (0.0009) [2023-10-14 19:34:40,378][61585] Updated weights for policy 1, policy_version 46980 (0.0012) [2023-10-14 19:34:40,748][61585] Updated weights for policy 1, policy_version 46990 (0.0011) [2023-10-14 19:34:41,112][61585] Updated weights for policy 1, policy_version 47000 (0.0010) [2023-10-14 19:34:43,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 96436224. Throughput: 0: 1663.8, 1: 1696.2. Samples: 24120378. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:34:43,344][60425] Avg episode reward: [(0, '71.830'), (1, '66.210')] [2023-10-14 19:34:44,300][61552] Updated weights for policy 0, policy_version 47172 (0.0007) [2023-10-14 19:34:44,672][61552] Updated weights for policy 0, policy_version 47182 (0.0007) [2023-10-14 19:34:45,035][61552] Updated weights for policy 0, policy_version 47192 (0.0009) [2023-10-14 19:34:45,146][61585] Updated weights for policy 1, policy_version 47010 (0.0008) [2023-10-14 19:34:45,515][61585] Updated weights for policy 1, policy_version 47020 (0.0008) [2023-10-14 19:34:45,881][61585] Updated weights for policy 1, policy_version 47030 (0.0008) [2023-10-14 19:34:46,250][61585] Updated weights for policy 1, policy_version 47040 (0.0008) [2023-10-14 19:34:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 96501760. Throughput: 0: 1662.7, 1: 1676.2. Samples: 24129940. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:34:48,344][60425] Avg episode reward: [(0, '71.000'), (1, '71.350')] [2023-10-14 19:34:49,167][61552] Updated weights for policy 0, policy_version 47202 (0.0007) [2023-10-14 19:34:49,534][61552] Updated weights for policy 0, policy_version 47212 (0.0007) [2023-10-14 19:34:49,895][61552] Updated weights for policy 0, policy_version 47222 (0.0008) [2023-10-14 19:34:50,263][61552] Updated weights for policy 0, policy_version 47232 (0.0007) [2023-10-14 19:34:50,449][61585] Updated weights for policy 1, policy_version 47050 (0.0009) [2023-10-14 19:34:50,814][61585] Updated weights for policy 1, policy_version 47060 (0.0007) [2023-10-14 19:34:51,178][61585] Updated weights for policy 1, policy_version 47070 (0.0008) [2023-10-14 19:34:53,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 96567296. Throughput: 0: 1671.5, 1: 1671.4. Samples: 24149914. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:34:53,344][60425] Avg episode reward: [(0, '71.340'), (1, '73.160')] [2023-10-14 19:34:54,157][61552] Updated weights for policy 0, policy_version 47242 (0.0008) [2023-10-14 19:34:54,526][61552] Updated weights for policy 0, policy_version 47252 (0.0009) [2023-10-14 19:34:54,892][61552] Updated weights for policy 0, policy_version 47262 (0.0009) [2023-10-14 19:34:55,252][61585] Updated weights for policy 1, policy_version 47080 (0.0010) [2023-10-14 19:34:55,611][61585] Updated weights for policy 1, policy_version 47090 (0.0010) [2023-10-14 19:34:55,980][61585] Updated weights for policy 1, policy_version 47100 (0.0008) [2023-10-14 19:34:58,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 96632832. Throughput: 0: 1673.1, 1: 1690.1. Samples: 24170658. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-10-14 19:34:58,345][60425] Avg episode reward: [(0, '69.900'), (1, '69.670')] [2023-10-14 19:34:58,787][61552] Updated weights for policy 0, policy_version 47272 (0.0009) [2023-10-14 19:34:59,150][61552] Updated weights for policy 0, policy_version 47282 (0.0010) [2023-10-14 19:34:59,512][61552] Updated weights for policy 0, policy_version 47292 (0.0007) [2023-10-14 19:35:00,098][61585] Updated weights for policy 1, policy_version 47110 (0.0009) [2023-10-14 19:35:00,468][61585] Updated weights for policy 1, policy_version 47120 (0.0008) [2023-10-14 19:35:00,822][61585] Updated weights for policy 1, policy_version 47130 (0.0007) [2023-10-14 19:35:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 96698368. Throughput: 0: 1678.6, 1: 1668.9. Samples: 24180312. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-10-14 19:35:03,344][60425] Avg episode reward: [(0, '72.120'), (1, '69.440')] [2023-10-14 19:35:03,555][61552] Updated weights for policy 0, policy_version 47302 (0.0009) [2023-10-14 19:35:03,935][61552] Updated weights for policy 0, policy_version 47312 (0.0007) [2023-10-14 19:35:04,302][61552] Updated weights for policy 0, policy_version 47322 (0.0011) [2023-10-14 19:35:04,990][61585] Updated weights for policy 1, policy_version 47140 (0.0009) [2023-10-14 19:35:05,400][61585] Updated weights for policy 1, policy_version 47150 (0.0010) [2023-10-14 19:35:05,761][61585] Updated weights for policy 1, policy_version 47160 (0.0009) [2023-10-14 19:35:08,254][61552] Updated weights for policy 0, policy_version 47332 (0.0011) [2023-10-14 19:35:08,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 96763904. Throughput: 0: 1684.4, 1: 1671.6. Samples: 24200226. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-10-14 19:35:08,344][60425] Avg episode reward: [(0, '74.900'), (1, '67.730')] [2023-10-14 19:35:08,631][61552] Updated weights for policy 0, policy_version 47342 (0.0009) [2023-10-14 19:35:08,991][61552] Updated weights for policy 0, policy_version 47352 (0.0007) [2023-10-14 19:35:09,818][61585] Updated weights for policy 1, policy_version 47170 (0.0010) [2023-10-14 19:35:10,176][61585] Updated weights for policy 1, policy_version 47180 (0.0011) [2023-10-14 19:35:10,543][61585] Updated weights for policy 1, policy_version 47190 (0.0008) [2023-10-14 19:35:10,909][61585] Updated weights for policy 1, policy_version 47200 (0.0012) [2023-10-14 19:35:13,076][61552] Updated weights for policy 0, policy_version 47362 (0.0008) [2023-10-14 19:35:13,344][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 96829440. Throughput: 0: 1688.2, 1: 1678.3. Samples: 24221076. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-10-14 19:35:13,344][60425] Avg episode reward: [(0, '69.100'), (1, '67.890')] [2023-10-14 19:35:13,451][61552] Updated weights for policy 0, policy_version 47372 (0.0009) [2023-10-14 19:35:13,822][61552] Updated weights for policy 0, policy_version 47382 (0.0009) [2023-10-14 19:35:14,183][61552] Updated weights for policy 0, policy_version 47392 (0.0011) [2023-10-14 19:35:15,084][61585] Updated weights for policy 1, policy_version 47210 (0.0009) [2023-10-14 19:35:15,455][61585] Updated weights for policy 1, policy_version 47220 (0.0007) [2023-10-14 19:35:15,823][61585] Updated weights for policy 1, policy_version 47230 (0.0007) [2023-10-14 19:35:18,340][61552] Updated weights for policy 0, policy_version 47402 (0.0009) [2023-10-14 19:35:18,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 96894976. Throughput: 0: 1686.2, 1: 1659.3. Samples: 24230312. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-10-14 19:35:18,344][60425] Avg episode reward: [(0, '65.900'), (1, '68.790')] [2023-10-14 19:35:18,704][61552] Updated weights for policy 0, policy_version 47412 (0.0008) [2023-10-14 19:35:19,079][61552] Updated weights for policy 0, policy_version 47422 (0.0007) [2023-10-14 19:35:20,037][61585] Updated weights for policy 1, policy_version 47240 (0.0010) [2023-10-14 19:35:20,398][61585] Updated weights for policy 1, policy_version 47250 (0.0007) [2023-10-14 19:35:20,768][61585] Updated weights for policy 1, policy_version 47260 (0.0009) [2023-10-14 19:35:23,288][61552] Updated weights for policy 0, policy_version 47432 (0.0010) [2023-10-14 19:35:23,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 96960512. Throughput: 0: 1675.8, 1: 1671.5. Samples: 24250334. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-10-14 19:35:23,344][60425] Avg episode reward: [(0, '68.580'), (1, '68.900')] [2023-10-14 19:35:23,659][61552] Updated weights for policy 0, policy_version 47442 (0.0008) [2023-10-14 19:35:24,025][61552] Updated weights for policy 0, policy_version 47452 (0.0009) [2023-10-14 19:35:24,958][61585] Updated weights for policy 1, policy_version 47270 (0.0008) [2023-10-14 19:35:25,315][61585] Updated weights for policy 1, policy_version 47280 (0.0010) [2023-10-14 19:35:25,684][61585] Updated weights for policy 1, policy_version 47290 (0.0007) [2023-10-14 19:35:28,233][61552] Updated weights for policy 0, policy_version 47462 (0.0008) [2023-10-14 19:35:28,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 97026048. Throughput: 0: 1676.3, 1: 1664.4. Samples: 24270710. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-10-14 19:35:28,344][60425] Avg episode reward: [(0, '72.700'), (1, '67.710')] [2023-10-14 19:35:28,605][61552] Updated weights for policy 0, policy_version 47472 (0.0010) [2023-10-14 19:35:28,967][61552] Updated weights for policy 0, policy_version 47482 (0.0010) [2023-10-14 19:35:29,545][61585] Updated weights for policy 1, policy_version 47300 (0.0007) [2023-10-14 19:35:29,916][61585] Updated weights for policy 1, policy_version 47310 (0.0007) [2023-10-14 19:35:30,290][61585] Updated weights for policy 1, policy_version 47320 (0.0008) [2023-10-14 19:35:33,199][61552] Updated weights for policy 0, policy_version 47492 (0.0010) [2023-10-14 19:35:33,343][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 97091584. Throughput: 0: 1677.6, 1: 1654.7. Samples: 24279894. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-14 19:35:33,344][60425] Avg episode reward: [(0, '69.910'), (1, '71.120')] [2023-10-14 19:35:33,580][61552] Updated weights for policy 0, policy_version 47502 (0.0008) [2023-10-14 19:35:33,944][61552] Updated weights for policy 0, policy_version 47512 (0.0009) [2023-10-14 19:35:34,443][61585] Updated weights for policy 1, policy_version 47330 (0.0009) [2023-10-14 19:35:34,812][61585] Updated weights for policy 1, policy_version 47340 (0.0008) [2023-10-14 19:35:35,182][61585] Updated weights for policy 1, policy_version 47350 (0.0008) [2023-10-14 19:35:35,545][61585] Updated weights for policy 1, policy_version 47360 (0.0009) [2023-10-14 19:35:38,098][61552] Updated weights for policy 0, policy_version 47522 (0.0009) [2023-10-14 19:35:38,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 97157120. Throughput: 0: 1669.8, 1: 1668.2. Samples: 24300124. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-14 19:35:38,344][60425] Avg episode reward: [(0, '70.260'), (1, '70.060')] [2023-10-14 19:35:38,474][61552] Updated weights for policy 0, policy_version 47532 (0.0008) [2023-10-14 19:35:38,856][61552] Updated weights for policy 0, policy_version 47542 (0.0009) [2023-10-14 19:35:39,228][61552] Updated weights for policy 0, policy_version 47552 (0.0009) [2023-10-14 19:35:39,575][61585] Updated weights for policy 1, policy_version 47370 (0.0009) [2023-10-14 19:35:39,947][61585] Updated weights for policy 1, policy_version 47380 (0.0008) [2023-10-14 19:35:40,316][61585] Updated weights for policy 1, policy_version 47390 (0.0009) [2023-10-14 19:35:43,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 97222656. Throughput: 0: 1661.9, 1: 1671.6. Samples: 24320664. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-14 19:35:43,344][60425] Avg episode reward: [(0, '69.090'), (1, '67.320')] [2023-10-14 19:35:43,351][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000047392_48529408.pth... [2023-10-14 19:35:43,369][61552] Updated weights for policy 0, policy_version 47562 (0.0009) [2023-10-14 19:35:43,390][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000045824_46923776.pth [2023-10-14 19:35:43,737][61552] Updated weights for policy 0, policy_version 47572 (0.0007) [2023-10-14 19:35:44,107][61552] Updated weights for policy 0, policy_version 47582 (0.0007) [2023-10-14 19:35:44,179][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000047584_48726016.pth... [2023-10-14 19:35:44,222][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000046016_47120384.pth [2023-10-14 19:35:44,435][61585] Updated weights for policy 1, policy_version 47400 (0.0010) [2023-10-14 19:35:44,805][61585] Updated weights for policy 1, policy_version 47410 (0.0007) [2023-10-14 19:35:45,166][61585] Updated weights for policy 1, policy_version 47420 (0.0007) [2023-10-14 19:35:48,211][61552] Updated weights for policy 0, policy_version 47592 (0.0008) [2023-10-14 19:35:48,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 97288192. Throughput: 0: 1662.5, 1: 1661.3. Samples: 24329882. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-14 19:35:48,344][60425] Avg episode reward: [(0, '71.920'), (1, '71.830')] [2023-10-14 19:35:48,586][61552] Updated weights for policy 0, policy_version 47602 (0.0010) [2023-10-14 19:35:48,953][61552] Updated weights for policy 0, policy_version 47612 (0.0008) [2023-10-14 19:35:49,432][61585] Updated weights for policy 1, policy_version 47430 (0.0008) [2023-10-14 19:35:49,794][61585] Updated weights for policy 1, policy_version 47440 (0.0009) [2023-10-14 19:35:50,156][61585] Updated weights for policy 1, policy_version 47450 (0.0008) [2023-10-14 19:35:52,969][61552] Updated weights for policy 0, policy_version 47622 (0.0010) [2023-10-14 19:35:53,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 97353728. Throughput: 0: 1667.0, 1: 1670.8. Samples: 24350428. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-14 19:35:53,344][60425] Avg episode reward: [(0, '73.820'), (1, '72.710')] [2023-10-14 19:35:53,361][61552] Updated weights for policy 0, policy_version 47632 (0.0007) [2023-10-14 19:35:53,736][61552] Updated weights for policy 0, policy_version 47642 (0.0007) [2023-10-14 19:35:54,284][61585] Updated weights for policy 1, policy_version 47460 (0.0007) [2023-10-14 19:35:54,669][61585] Updated weights for policy 1, policy_version 47470 (0.0008) [2023-10-14 19:35:55,034][61585] Updated weights for policy 1, policy_version 47480 (0.0009) [2023-10-14 19:35:57,865][61552] Updated weights for policy 0, policy_version 47652 (0.0009) [2023-10-14 19:35:58,227][61552] Updated weights for policy 0, policy_version 47662 (0.0007) [2023-10-14 19:35:58,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 97419264. Throughput: 0: 1660.1, 1: 1664.4. Samples: 24370678. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-14 19:35:58,344][60425] Avg episode reward: [(0, '71.230'), (1, '67.160')] [2023-10-14 19:35:58,594][61552] Updated weights for policy 0, policy_version 47672 (0.0008) [2023-10-14 19:35:59,167][61585] Updated weights for policy 1, policy_version 47490 (0.0008) [2023-10-14 19:35:59,520][61585] Updated weights for policy 1, policy_version 47500 (0.0009) [2023-10-14 19:35:59,888][61585] Updated weights for policy 1, policy_version 47510 (0.0008) [2023-10-14 19:36:00,250][61585] Updated weights for policy 1, policy_version 47520 (0.0009) [2023-10-14 19:36:02,619][61552] Updated weights for policy 0, policy_version 47682 (0.0008) [2023-10-14 19:36:02,984][61552] Updated weights for policy 0, policy_version 47692 (0.0008) [2023-10-14 19:36:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 97484800. Throughput: 0: 1666.5, 1: 1661.5. Samples: 24380070. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-14 19:36:03,344][60425] Avg episode reward: [(0, '70.940'), (1, '66.930')] [2023-10-14 19:36:03,352][61552] Updated weights for policy 0, policy_version 47702 (0.0010) [2023-10-14 19:36:03,720][61552] Updated weights for policy 0, policy_version 47712 (0.0007) [2023-10-14 19:36:04,410][61585] Updated weights for policy 1, policy_version 47530 (0.0008) [2023-10-14 19:36:04,780][61585] Updated weights for policy 1, policy_version 47540 (0.0008) [2023-10-14 19:36:05,142][61585] Updated weights for policy 1, policy_version 47550 (0.0009) [2023-10-14 19:36:07,803][61552] Updated weights for policy 0, policy_version 47722 (0.0008) [2023-10-14 19:36:08,171][61552] Updated weights for policy 0, policy_version 47732 (0.0009) [2023-10-14 19:36:08,344][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 97550336. Throughput: 0: 1671.7, 1: 1674.8. Samples: 24400928. Policy #0 lag: (min: 9.0, avg: 22.8, max: 41.0) [2023-10-14 19:36:08,345][60425] Avg episode reward: [(0, '66.740'), (1, '63.570')] [2023-10-14 19:36:08,541][61552] Updated weights for policy 0, policy_version 47742 (0.0007) [2023-10-14 19:36:09,039][61585] Updated weights for policy 1, policy_version 47560 (0.0009) [2023-10-14 19:36:09,418][61585] Updated weights for policy 1, policy_version 47570 (0.0009) [2023-10-14 19:36:09,786][61585] Updated weights for policy 1, policy_version 47580 (0.0008) [2023-10-14 19:36:12,635][61552] Updated weights for policy 0, policy_version 47752 (0.0008) [2023-10-14 19:36:13,006][61552] Updated weights for policy 0, policy_version 47762 (0.0009) [2023-10-14 19:36:13,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 97615872. Throughput: 0: 1666.8, 1: 1679.7. Samples: 24421302. Policy #0 lag: (min: 9.0, avg: 22.8, max: 41.0) [2023-10-14 19:36:13,344][60425] Avg episode reward: [(0, '68.560'), (1, '65.180')] [2023-10-14 19:36:13,373][61552] Updated weights for policy 0, policy_version 47772 (0.0009) [2023-10-14 19:36:13,868][61585] Updated weights for policy 1, policy_version 47590 (0.0008) [2023-10-14 19:36:14,227][61585] Updated weights for policy 1, policy_version 47600 (0.0009) [2023-10-14 19:36:14,599][61585] Updated weights for policy 1, policy_version 47610 (0.0009) [2023-10-14 19:36:17,624][61552] Updated weights for policy 0, policy_version 47782 (0.0009) [2023-10-14 19:36:17,980][61552] Updated weights for policy 0, policy_version 47792 (0.0008) [2023-10-14 19:36:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 97681408. Throughput: 0: 1670.2, 1: 1680.0. Samples: 24430652. Policy #0 lag: (min: 9.0, avg: 22.8, max: 41.0) [2023-10-14 19:36:18,344][60425] Avg episode reward: [(0, '66.670'), (1, '67.890')] [2023-10-14 19:36:18,350][61552] Updated weights for policy 0, policy_version 47802 (0.0008) [2023-10-14 19:36:18,663][61585] Updated weights for policy 1, policy_version 47620 (0.0009) [2023-10-14 19:36:19,024][61585] Updated weights for policy 1, policy_version 47630 (0.0010) [2023-10-14 19:36:19,396][61585] Updated weights for policy 1, policy_version 47640 (0.0010) [2023-10-14 19:36:22,335][61552] Updated weights for policy 0, policy_version 47812 (0.0007) [2023-10-14 19:36:22,714][61552] Updated weights for policy 0, policy_version 47822 (0.0007) [2023-10-14 19:36:23,080][61552] Updated weights for policy 0, policy_version 47832 (0.0009) [2023-10-14 19:36:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 97746944. Throughput: 0: 1673.6, 1: 1681.8. Samples: 24451118. Policy #0 lag: (min: 9.0, avg: 22.8, max: 41.0) [2023-10-14 19:36:23,344][60425] Avg episode reward: [(0, '66.310'), (1, '69.070')] [2023-10-14 19:36:23,531][61585] Updated weights for policy 1, policy_version 47650 (0.0007) [2023-10-14 19:36:23,887][61585] Updated weights for policy 1, policy_version 47660 (0.0009) [2023-10-14 19:36:24,260][61585] Updated weights for policy 1, policy_version 47670 (0.0009) [2023-10-14 19:36:24,621][61585] Updated weights for policy 1, policy_version 47680 (0.0009) [2023-10-14 19:36:27,163][61552] Updated weights for policy 0, policy_version 47842 (0.0010) [2023-10-14 19:36:27,537][61552] Updated weights for policy 0, policy_version 47852 (0.0007) [2023-10-14 19:36:27,896][61552] Updated weights for policy 0, policy_version 47862 (0.0007) [2023-10-14 19:36:28,266][61552] Updated weights for policy 0, policy_version 47872 (0.0009) [2023-10-14 19:36:28,343][60425] Fps is (10 sec: 16384.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 97845248. Throughput: 0: 1662.8, 1: 1678.3. Samples: 24471016. Policy #0 lag: (min: 9.0, avg: 22.8, max: 41.0) [2023-10-14 19:36:28,344][60425] Avg episode reward: [(0, '66.340'), (1, '64.640')] [2023-10-14 19:36:28,667][61585] Updated weights for policy 1, policy_version 47690 (0.0007) [2023-10-14 19:36:29,033][61585] Updated weights for policy 1, policy_version 47700 (0.0007) [2023-10-14 19:36:29,397][61585] Updated weights for policy 1, policy_version 47710 (0.0009) [2023-10-14 19:36:32,326][61552] Updated weights for policy 0, policy_version 47882 (0.0007) [2023-10-14 19:36:32,693][61552] Updated weights for policy 0, policy_version 47892 (0.0007) [2023-10-14 19:36:33,063][61552] Updated weights for policy 0, policy_version 47902 (0.0007) [2023-10-14 19:36:33,343][60425] Fps is (10 sec: 16384.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 97910784. Throughput: 0: 1672.1, 1: 1676.6. Samples: 24480574. Policy #0 lag: (min: 9.0, avg: 22.8, max: 41.0) [2023-10-14 19:36:33,344][60425] Avg episode reward: [(0, '70.810'), (1, '65.250')] [2023-10-14 19:36:33,633][61585] Updated weights for policy 1, policy_version 47720 (0.0008) [2023-10-14 19:36:34,004][61585] Updated weights for policy 1, policy_version 47730 (0.0009) [2023-10-14 19:36:34,370][61585] Updated weights for policy 1, policy_version 47740 (0.0009) [2023-10-14 19:36:37,218][61552] Updated weights for policy 0, policy_version 47912 (0.0008) [2023-10-14 19:36:37,591][61552] Updated weights for policy 0, policy_version 47922 (0.0008) [2023-10-14 19:36:37,954][61552] Updated weights for policy 0, policy_version 47932 (0.0008) [2023-10-14 19:36:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 97976320. Throughput: 0: 1670.0, 1: 1684.4. Samples: 24501376. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:36:38,344][60425] Avg episode reward: [(0, '69.340'), (1, '67.870')] [2023-10-14 19:36:38,495][61585] Updated weights for policy 1, policy_version 47750 (0.0007) [2023-10-14 19:36:38,886][61585] Updated weights for policy 1, policy_version 47760 (0.0007) [2023-10-14 19:36:39,261][61585] Updated weights for policy 1, policy_version 47770 (0.0007) [2023-10-14 19:36:42,147][61552] Updated weights for policy 0, policy_version 47942 (0.0008) [2023-10-14 19:36:42,516][61552] Updated weights for policy 0, policy_version 47952 (0.0009) [2023-10-14 19:36:42,896][61552] Updated weights for policy 0, policy_version 47962 (0.0009) [2023-10-14 19:36:43,249][61585] Updated weights for policy 1, policy_version 47780 (0.0008) [2023-10-14 19:36:43,343][60425] Fps is (10 sec: 13106.7, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 98041856. Throughput: 0: 1654.8, 1: 1686.2. Samples: 24521026. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:36:43,345][60425] Avg episode reward: [(0, '73.460'), (1, '74.370')] [2023-10-14 19:36:43,614][61585] Updated weights for policy 1, policy_version 47790 (0.0009) [2023-10-14 19:36:43,981][61585] Updated weights for policy 1, policy_version 47800 (0.0008) [2023-10-14 19:36:46,854][61552] Updated weights for policy 0, policy_version 47972 (0.0008) [2023-10-14 19:36:47,234][61552] Updated weights for policy 0, policy_version 47982 (0.0008) [2023-10-14 19:36:47,595][61552] Updated weights for policy 0, policy_version 47992 (0.0008) [2023-10-14 19:36:47,909][61585] Updated weights for policy 1, policy_version 47810 (0.0008) [2023-10-14 19:36:48,272][61585] Updated weights for policy 1, policy_version 47820 (0.0008) [2023-10-14 19:36:48,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 98107392. Throughput: 0: 1670.1, 1: 1685.7. Samples: 24531080. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:36:48,344][60425] Avg episode reward: [(0, '71.920'), (1, '69.710')] [2023-10-14 19:36:48,637][61585] Updated weights for policy 1, policy_version 47830 (0.0008) [2023-10-14 19:36:49,003][61585] Updated weights for policy 1, policy_version 47840 (0.0008) [2023-10-14 19:36:51,629][61552] Updated weights for policy 0, policy_version 48002 (0.0009) [2023-10-14 19:36:52,006][61552] Updated weights for policy 0, policy_version 48012 (0.0008) [2023-10-14 19:36:52,373][61552] Updated weights for policy 0, policy_version 48022 (0.0010) [2023-10-14 19:36:52,733][61552] Updated weights for policy 0, policy_version 48032 (0.0010) [2023-10-14 19:36:53,161][61585] Updated weights for policy 1, policy_version 47850 (0.0010) [2023-10-14 19:36:53,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 98172928. Throughput: 0: 1665.3, 1: 1686.4. Samples: 24551756. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:36:53,344][60425] Avg episode reward: [(0, '68.900'), (1, '69.600')] [2023-10-14 19:36:53,533][61585] Updated weights for policy 1, policy_version 47860 (0.0008) [2023-10-14 19:36:53,896][61585] Updated weights for policy 1, policy_version 47870 (0.0008) [2023-10-14 19:36:56,769][61552] Updated weights for policy 0, policy_version 48042 (0.0008) [2023-10-14 19:36:57,132][61552] Updated weights for policy 0, policy_version 48052 (0.0008) [2023-10-14 19:36:57,507][61552] Updated weights for policy 0, policy_version 48062 (0.0008) [2023-10-14 19:36:57,961][61585] Updated weights for policy 1, policy_version 47880 (0.0007) [2023-10-14 19:36:58,324][61585] Updated weights for policy 1, policy_version 47890 (0.0010) [2023-10-14 19:36:58,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 98238464. Throughput: 0: 1653.7, 1: 1679.9. Samples: 24571314. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:36:58,344][60425] Avg episode reward: [(0, '69.390'), (1, '73.960')] [2023-10-14 19:36:58,686][61585] Updated weights for policy 1, policy_version 47900 (0.0007) [2023-10-14 19:37:01,500][61552] Updated weights for policy 0, policy_version 48072 (0.0007) [2023-10-14 19:37:01,868][61552] Updated weights for policy 0, policy_version 48082 (0.0009) [2023-10-14 19:37:02,229][61552] Updated weights for policy 0, policy_version 48092 (0.0008) [2023-10-14 19:37:02,777][61585] Updated weights for policy 1, policy_version 47910 (0.0008) [2023-10-14 19:37:03,146][61585] Updated weights for policy 1, policy_version 47920 (0.0007) [2023-10-14 19:37:03,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 98304000. Throughput: 0: 1682.3, 1: 1678.9. Samples: 24581908. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:37:03,344][60425] Avg episode reward: [(0, '73.760'), (1, '71.610')] [2023-10-14 19:37:03,506][61585] Updated weights for policy 1, policy_version 47930 (0.0008) [2023-10-14 19:37:06,249][61552] Updated weights for policy 0, policy_version 48102 (0.0007) [2023-10-14 19:37:06,615][61552] Updated weights for policy 0, policy_version 48112 (0.0007) [2023-10-14 19:37:06,983][61552] Updated weights for policy 0, policy_version 48122 (0.0007) [2023-10-14 19:37:07,564][61585] Updated weights for policy 1, policy_version 47940 (0.0008) [2023-10-14 19:37:07,935][61585] Updated weights for policy 1, policy_version 47950 (0.0009) [2023-10-14 19:37:08,307][61585] Updated weights for policy 1, policy_version 47960 (0.0009) [2023-10-14 19:37:08,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 98369536. Throughput: 0: 1669.8, 1: 1676.3. Samples: 24601696. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:37:08,344][60425] Avg episode reward: [(0, '75.280'), (1, '74.110')] [2023-10-14 19:37:11,238][61552] Updated weights for policy 0, policy_version 48132 (0.0010) [2023-10-14 19:37:11,610][61552] Updated weights for policy 0, policy_version 48142 (0.0010) [2023-10-14 19:37:11,970][61552] Updated weights for policy 0, policy_version 48152 (0.0010) [2023-10-14 19:37:12,405][61585] Updated weights for policy 1, policy_version 47970 (0.0008) [2023-10-14 19:37:12,769][61585] Updated weights for policy 1, policy_version 47980 (0.0007) [2023-10-14 19:37:13,144][61585] Updated weights for policy 1, policy_version 47990 (0.0007) [2023-10-14 19:37:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 98435072. Throughput: 0: 1666.8, 1: 1669.2. Samples: 24621136. Policy #0 lag: (min: 10.0, avg: 13.7, max: 41.0) [2023-10-14 19:37:13,344][60425] Avg episode reward: [(0, '71.070'), (1, '72.590')] [2023-10-14 19:37:13,502][61585] Updated weights for policy 1, policy_version 48000 (0.0009) [2023-10-14 19:37:16,055][61552] Updated weights for policy 0, policy_version 48162 (0.0010) [2023-10-14 19:37:16,424][61552] Updated weights for policy 0, policy_version 48172 (0.0008) [2023-10-14 19:37:16,788][61552] Updated weights for policy 0, policy_version 48182 (0.0009) [2023-10-14 19:37:17,165][61552] Updated weights for policy 0, policy_version 48192 (0.0009) [2023-10-14 19:37:17,714][61585] Updated weights for policy 1, policy_version 48010 (0.0008) [2023-10-14 19:37:18,094][61585] Updated weights for policy 1, policy_version 48020 (0.0008) [2023-10-14 19:37:18,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 98500608. Throughput: 0: 1682.9, 1: 1676.3. Samples: 24631738. Policy #0 lag: (min: 10.0, avg: 13.7, max: 41.0) [2023-10-14 19:37:18,344][60425] Avg episode reward: [(0, '67.940'), (1, '73.800')] [2023-10-14 19:37:18,455][61585] Updated weights for policy 1, policy_version 48030 (0.0009) [2023-10-14 19:37:21,236][61552] Updated weights for policy 0, policy_version 48202 (0.0010) [2023-10-14 19:37:21,601][61552] Updated weights for policy 0, policy_version 48212 (0.0011) [2023-10-14 19:37:21,976][61552] Updated weights for policy 0, policy_version 48222 (0.0008) [2023-10-14 19:37:22,442][61585] Updated weights for policy 1, policy_version 48040 (0.0007) [2023-10-14 19:37:22,814][61585] Updated weights for policy 1, policy_version 48050 (0.0009) [2023-10-14 19:37:23,174][61585] Updated weights for policy 1, policy_version 48060 (0.0011) [2023-10-14 19:37:23,343][60425] Fps is (10 sec: 16384.0, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 98598912. Throughput: 0: 1664.3, 1: 1677.1. Samples: 24651740. Policy #0 lag: (min: 10.0, avg: 13.7, max: 41.0) [2023-10-14 19:37:23,344][60425] Avg episode reward: [(0, '67.060'), (1, '68.960')] [2023-10-14 19:37:26,038][61552] Updated weights for policy 0, policy_version 48232 (0.0008) [2023-10-14 19:37:26,399][61552] Updated weights for policy 0, policy_version 48242 (0.0010) [2023-10-14 19:37:26,760][61552] Updated weights for policy 0, policy_version 48252 (0.0007) [2023-10-14 19:37:27,323][61585] Updated weights for policy 1, policy_version 48070 (0.0008) [2023-10-14 19:37:27,690][61585] Updated weights for policy 1, policy_version 48080 (0.0009) [2023-10-14 19:37:28,055][61585] Updated weights for policy 1, policy_version 48090 (0.0010) [2023-10-14 19:37:28,343][60425] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 98664448. Throughput: 0: 1675.4, 1: 1662.3. Samples: 24671222. Policy #0 lag: (min: 10.0, avg: 13.7, max: 41.0) [2023-10-14 19:37:28,344][60425] Avg episode reward: [(0, '72.330'), (1, '72.780')] [2023-10-14 19:37:30,808][61552] Updated weights for policy 0, policy_version 48262 (0.0009) [2023-10-14 19:37:31,183][61552] Updated weights for policy 0, policy_version 48272 (0.0011) [2023-10-14 19:37:31,547][61552] Updated weights for policy 0, policy_version 48282 (0.0011) [2023-10-14 19:37:32,079][61585] Updated weights for policy 1, policy_version 48100 (0.0011) [2023-10-14 19:37:32,444][61585] Updated weights for policy 1, policy_version 48110 (0.0008) [2023-10-14 19:37:32,805][61585] Updated weights for policy 1, policy_version 48120 (0.0008) [2023-10-14 19:37:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 98729984. Throughput: 0: 1685.2, 1: 1674.8. Samples: 24682282. Policy #0 lag: (min: 10.0, avg: 13.7, max: 41.0) [2023-10-14 19:37:33,344][60425] Avg episode reward: [(0, '74.110'), (1, '75.670')] [2023-10-14 19:37:33,345][61248] Saving new best policy, reward=75.670! [2023-10-14 19:37:35,563][61552] Updated weights for policy 0, policy_version 48292 (0.0008) [2023-10-14 19:37:35,932][61552] Updated weights for policy 0, policy_version 48302 (0.0007) [2023-10-14 19:37:36,295][61552] Updated weights for policy 0, policy_version 48312 (0.0007) [2023-10-14 19:37:36,877][61585] Updated weights for policy 1, policy_version 48130 (0.0010) [2023-10-14 19:37:37,254][61585] Updated weights for policy 1, policy_version 48140 (0.0009) [2023-10-14 19:37:37,617][61585] Updated weights for policy 1, policy_version 48150 (0.0007) [2023-10-14 19:37:37,981][61585] Updated weights for policy 1, policy_version 48160 (0.0008) [2023-10-14 19:37:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 98795520. Throughput: 0: 1664.1, 1: 1673.1. Samples: 24701928. Policy #0 lag: (min: 10.0, avg: 13.7, max: 41.0) [2023-10-14 19:37:38,344][60425] Avg episode reward: [(0, '71.280'), (1, '70.890')] [2023-10-14 19:37:40,291][61552] Updated weights for policy 0, policy_version 48322 (0.0008) [2023-10-14 19:37:40,657][61552] Updated weights for policy 0, policy_version 48332 (0.0007) [2023-10-14 19:37:41,031][61552] Updated weights for policy 0, policy_version 48342 (0.0007) [2023-10-14 19:37:41,395][61552] Updated weights for policy 0, policy_version 48352 (0.0009) [2023-10-14 19:37:42,161][61585] Updated weights for policy 1, policy_version 48170 (0.0007) [2023-10-14 19:37:42,523][61585] Updated weights for policy 1, policy_version 48180 (0.0007) [2023-10-14 19:37:42,891][61585] Updated weights for policy 1, policy_version 48190 (0.0009) [2023-10-14 19:37:43,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 98861056. Throughput: 0: 1690.2, 1: 1654.7. Samples: 24721834. Policy #0 lag: (min: 10.0, avg: 13.7, max: 41.0) [2023-10-14 19:37:43,345][60425] Avg episode reward: [(0, '69.680'), (1, '68.890')] [2023-10-14 19:37:43,357][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000048192_49348608.pth... [2023-10-14 19:37:43,358][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000048352_49512448.pth... [2023-10-14 19:37:43,393][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000046784_47906816.pth [2023-10-14 19:37:43,397][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000046624_47742976.pth [2023-10-14 19:37:45,561][61552] Updated weights for policy 0, policy_version 48362 (0.0008) [2023-10-14 19:37:45,935][61552] Updated weights for policy 0, policy_version 48372 (0.0009) [2023-10-14 19:37:46,296][61552] Updated weights for policy 0, policy_version 48382 (0.0008) [2023-10-14 19:37:46,886][61585] Updated weights for policy 1, policy_version 48200 (0.0009) [2023-10-14 19:37:47,254][61585] Updated weights for policy 1, policy_version 48210 (0.0008) [2023-10-14 19:37:47,630][61585] Updated weights for policy 1, policy_version 48220 (0.0008) [2023-10-14 19:37:48,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 98926592. Throughput: 0: 1670.0, 1: 1676.5. Samples: 24732502. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:37:48,344][60425] Avg episode reward: [(0, '70.010'), (1, '71.350')] [2023-10-14 19:37:50,336][61552] Updated weights for policy 0, policy_version 48392 (0.0008) [2023-10-14 19:37:50,711][61552] Updated weights for policy 0, policy_version 48402 (0.0009) [2023-10-14 19:37:51,079][61552] Updated weights for policy 0, policy_version 48412 (0.0007) [2023-10-14 19:37:51,663][61585] Updated weights for policy 1, policy_version 48230 (0.0010) [2023-10-14 19:37:52,027][61585] Updated weights for policy 1, policy_version 48240 (0.0007) [2023-10-14 19:37:52,393][61585] Updated weights for policy 1, policy_version 48250 (0.0009) [2023-10-14 19:37:53,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 98992128. Throughput: 0: 1668.0, 1: 1672.3. Samples: 24752006. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:37:53,344][60425] Avg episode reward: [(0, '72.900'), (1, '75.920')] [2023-10-14 19:37:53,345][61248] Saving new best policy, reward=75.920! [2023-10-14 19:37:55,187][61552] Updated weights for policy 0, policy_version 48422 (0.0008) [2023-10-14 19:37:55,554][61552] Updated weights for policy 0, policy_version 48432 (0.0007) [2023-10-14 19:37:55,928][61552] Updated weights for policy 0, policy_version 48442 (0.0009) [2023-10-14 19:37:56,556][61585] Updated weights for policy 1, policy_version 48260 (0.0009) [2023-10-14 19:37:56,919][61585] Updated weights for policy 1, policy_version 48270 (0.0011) [2023-10-14 19:37:57,282][61585] Updated weights for policy 1, policy_version 48280 (0.0011) [2023-10-14 19:37:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 99057664. Throughput: 0: 1685.6, 1: 1660.7. Samples: 24771718. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:37:58,344][60425] Avg episode reward: [(0, '73.710'), (1, '73.960')] [2023-10-14 19:37:59,975][61552] Updated weights for policy 0, policy_version 48452 (0.0008) [2023-10-14 19:38:00,338][61552] Updated weights for policy 0, policy_version 48462 (0.0009) [2023-10-14 19:38:00,718][61552] Updated weights for policy 0, policy_version 48472 (0.0010) [2023-10-14 19:38:01,364][61585] Updated weights for policy 1, policy_version 48290 (0.0009) [2023-10-14 19:38:01,726][61585] Updated weights for policy 1, policy_version 48300 (0.0010) [2023-10-14 19:38:02,079][61585] Updated weights for policy 1, policy_version 48310 (0.0008) [2023-10-14 19:38:02,444][61585] Updated weights for policy 1, policy_version 48320 (0.0010) [2023-10-14 19:38:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 99123200. Throughput: 0: 1665.6, 1: 1686.8. Samples: 24782598. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:38:03,344][60425] Avg episode reward: [(0, '76.800'), (1, '70.930')] [2023-10-14 19:38:04,714][61552] Updated weights for policy 0, policy_version 48482 (0.0009) [2023-10-14 19:38:05,072][61552] Updated weights for policy 0, policy_version 48492 (0.0007) [2023-10-14 19:38:05,452][61552] Updated weights for policy 0, policy_version 48502 (0.0007) [2023-10-14 19:38:05,817][61552] Updated weights for policy 0, policy_version 48512 (0.0008) [2023-10-14 19:38:06,541][61585] Updated weights for policy 1, policy_version 48330 (0.0009) [2023-10-14 19:38:06,897][61585] Updated weights for policy 1, policy_version 48340 (0.0010) [2023-10-14 19:38:07,263][61585] Updated weights for policy 1, policy_version 48350 (0.0007) [2023-10-14 19:38:08,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 99188736. Throughput: 0: 1673.0, 1: 1672.7. Samples: 24802294. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:38:08,344][60425] Avg episode reward: [(0, '76.440'), (1, '73.950')] [2023-10-14 19:38:10,053][61552] Updated weights for policy 0, policy_version 48522 (0.0009) [2023-10-14 19:38:10,430][61552] Updated weights for policy 0, policy_version 48532 (0.0008) [2023-10-14 19:38:10,792][61552] Updated weights for policy 0, policy_version 48542 (0.0007) [2023-10-14 19:38:11,339][61585] Updated weights for policy 1, policy_version 48360 (0.0008) [2023-10-14 19:38:11,709][61585] Updated weights for policy 1, policy_version 48370 (0.0008) [2023-10-14 19:38:12,087][61585] Updated weights for policy 1, policy_version 48380 (0.0007) [2023-10-14 19:38:13,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 99254272. Throughput: 0: 1677.6, 1: 1678.7. Samples: 24822256. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:38:13,344][60425] Avg episode reward: [(0, '71.240'), (1, '81.300')] [2023-10-14 19:38:13,357][61248] Saving new best policy, reward=81.300! [2023-10-14 19:38:14,982][61552] Updated weights for policy 0, policy_version 48552 (0.0010) [2023-10-14 19:38:15,350][61552] Updated weights for policy 0, policy_version 48562 (0.0009) [2023-10-14 19:38:15,730][61552] Updated weights for policy 0, policy_version 48572 (0.0007) [2023-10-14 19:38:16,358][61585] Updated weights for policy 1, policy_version 48390 (0.0008) [2023-10-14 19:38:16,729][61585] Updated weights for policy 1, policy_version 48400 (0.0009) [2023-10-14 19:38:17,087][61585] Updated weights for policy 1, policy_version 48410 (0.0008) [2023-10-14 19:38:18,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 99319808. Throughput: 0: 1653.0, 1: 1690.0. Samples: 24832716. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:38:18,344][60425] Avg episode reward: [(0, '72.830'), (1, '74.180')] [2023-10-14 19:38:19,668][61552] Updated weights for policy 0, policy_version 48582 (0.0008) [2023-10-14 19:38:20,033][61552] Updated weights for policy 0, policy_version 48592 (0.0008) [2023-10-14 19:38:20,400][61552] Updated weights for policy 0, policy_version 48602 (0.0007) [2023-10-14 19:38:21,210][61585] Updated weights for policy 1, policy_version 48420 (0.0008) [2023-10-14 19:38:21,575][61585] Updated weights for policy 1, policy_version 48430 (0.0008) [2023-10-14 19:38:21,934][61585] Updated weights for policy 1, policy_version 48440 (0.0010) [2023-10-14 19:38:23,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 99385344. Throughput: 0: 1675.9, 1: 1669.4. Samples: 24852464. Policy #0 lag: (min: 26.0, avg: 28.7, max: 58.0) [2023-10-14 19:38:23,345][60425] Avg episode reward: [(0, '71.350'), (1, '68.210')] [2023-10-14 19:38:24,556][61552] Updated weights for policy 0, policy_version 48612 (0.0008) [2023-10-14 19:38:24,952][61552] Updated weights for policy 0, policy_version 48622 (0.0009) [2023-10-14 19:38:25,321][61552] Updated weights for policy 0, policy_version 48632 (0.0009) [2023-10-14 19:38:25,895][61585] Updated weights for policy 1, policy_version 48450 (0.0010) [2023-10-14 19:38:26,259][61585] Updated weights for policy 1, policy_version 48460 (0.0010) [2023-10-14 19:38:26,629][61585] Updated weights for policy 1, policy_version 48470 (0.0007) [2023-10-14 19:38:26,988][61585] Updated weights for policy 1, policy_version 48480 (0.0007) [2023-10-14 19:38:28,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 99450880. Throughput: 0: 1669.8, 1: 1678.6. Samples: 24872512. Policy #0 lag: (min: 26.0, avg: 28.7, max: 58.0) [2023-10-14 19:38:28,344][60425] Avg episode reward: [(0, '68.910'), (1, '76.770')] [2023-10-14 19:38:29,419][61552] Updated weights for policy 0, policy_version 48642 (0.0008) [2023-10-14 19:38:29,783][61552] Updated weights for policy 0, policy_version 48652 (0.0010) [2023-10-14 19:38:30,147][61552] Updated weights for policy 0, policy_version 48662 (0.0011) [2023-10-14 19:38:30,521][61552] Updated weights for policy 0, policy_version 48672 (0.0009) [2023-10-14 19:38:30,966][61585] Updated weights for policy 1, policy_version 48490 (0.0008) [2023-10-14 19:38:31,329][61585] Updated weights for policy 1, policy_version 48500 (0.0009) [2023-10-14 19:38:31,705][61585] Updated weights for policy 1, policy_version 48510 (0.0009) [2023-10-14 19:38:33,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 99516416. Throughput: 0: 1656.9, 1: 1683.8. Samples: 24882834. Policy #0 lag: (min: 26.0, avg: 28.7, max: 58.0) [2023-10-14 19:38:33,344][60425] Avg episode reward: [(0, '70.600'), (1, '67.270')] [2023-10-14 19:38:34,542][61552] Updated weights for policy 0, policy_version 48682 (0.0009) [2023-10-14 19:38:34,914][61552] Updated weights for policy 0, policy_version 48692 (0.0008) [2023-10-14 19:38:35,282][61552] Updated weights for policy 0, policy_version 48702 (0.0007) [2023-10-14 19:38:35,908][61585] Updated weights for policy 1, policy_version 48520 (0.0008) [2023-10-14 19:38:36,280][61585] Updated weights for policy 1, policy_version 48530 (0.0010) [2023-10-14 19:38:36,656][61585] Updated weights for policy 1, policy_version 48540 (0.0009) [2023-10-14 19:38:38,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 99581952. Throughput: 0: 1682.0, 1: 1667.7. Samples: 24902740. Policy #0 lag: (min: 26.0, avg: 28.7, max: 58.0) [2023-10-14 19:38:38,344][60425] Avg episode reward: [(0, '69.950'), (1, '65.690')] [2023-10-14 19:38:39,440][61552] Updated weights for policy 0, policy_version 48712 (0.0008) [2023-10-14 19:38:39,818][61552] Updated weights for policy 0, policy_version 48722 (0.0011) [2023-10-14 19:38:40,190][61552] Updated weights for policy 0, policy_version 48732 (0.0008) [2023-10-14 19:38:40,681][61585] Updated weights for policy 1, policy_version 48550 (0.0009) [2023-10-14 19:38:41,045][61585] Updated weights for policy 1, policy_version 48560 (0.0009) [2023-10-14 19:38:41,404][61585] Updated weights for policy 1, policy_version 48570 (0.0009) [2023-10-14 19:38:43,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 99647488. Throughput: 0: 1677.1, 1: 1687.9. Samples: 24923146. Policy #0 lag: (min: 26.0, avg: 28.7, max: 58.0) [2023-10-14 19:38:43,345][60425] Avg episode reward: [(0, '72.560'), (1, '70.870')] [2023-10-14 19:38:44,357][61552] Updated weights for policy 0, policy_version 48742 (0.0009) [2023-10-14 19:38:44,731][61552] Updated weights for policy 0, policy_version 48752 (0.0011) [2023-10-14 19:38:45,111][61552] Updated weights for policy 0, policy_version 48762 (0.0009) [2023-10-14 19:38:45,401][61585] Updated weights for policy 1, policy_version 48580 (0.0009) [2023-10-14 19:38:45,773][61585] Updated weights for policy 1, policy_version 48590 (0.0007) [2023-10-14 19:38:46,136][61585] Updated weights for policy 1, policy_version 48600 (0.0008) [2023-10-14 19:38:48,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 99713024. Throughput: 0: 1666.8, 1: 1674.3. Samples: 24932948. Policy #0 lag: (min: 26.0, avg: 28.7, max: 58.0) [2023-10-14 19:38:48,344][60425] Avg episode reward: [(0, '71.060'), (1, '68.000')] [2023-10-14 19:38:49,174][61552] Updated weights for policy 0, policy_version 48772 (0.0009) [2023-10-14 19:38:49,540][61552] Updated weights for policy 0, policy_version 48782 (0.0010) [2023-10-14 19:38:49,912][61552] Updated weights for policy 0, policy_version 48792 (0.0009) [2023-10-14 19:38:50,226][61585] Updated weights for policy 1, policy_version 48610 (0.0008) [2023-10-14 19:38:50,586][61585] Updated weights for policy 1, policy_version 48620 (0.0008) [2023-10-14 19:38:50,947][61585] Updated weights for policy 1, policy_version 48630 (0.0010) [2023-10-14 19:38:51,312][61585] Updated weights for policy 1, policy_version 48640 (0.0011) [2023-10-14 19:38:53,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 99778560. Throughput: 0: 1679.5, 1: 1667.5. Samples: 24952906. Policy #0 lag: (min: 26.0, avg: 28.7, max: 58.0) [2023-10-14 19:38:53,344][60425] Avg episode reward: [(0, '74.000'), (1, '66.180')] [2023-10-14 19:38:53,995][61552] Updated weights for policy 0, policy_version 48802 (0.0008) [2023-10-14 19:38:54,363][61552] Updated weights for policy 0, policy_version 48812 (0.0008) [2023-10-14 19:38:54,733][61552] Updated weights for policy 0, policy_version 48822 (0.0008) [2023-10-14 19:38:55,108][61552] Updated weights for policy 0, policy_version 48832 (0.0008) [2023-10-14 19:38:55,528][61585] Updated weights for policy 1, policy_version 48650 (0.0009) [2023-10-14 19:38:55,892][61585] Updated weights for policy 1, policy_version 48660 (0.0007) [2023-10-14 19:38:56,264][61585] Updated weights for policy 1, policy_version 48670 (0.0007) [2023-10-14 19:38:58,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 99844096. Throughput: 0: 1683.2, 1: 1675.9. Samples: 24973416. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:38:58,344][60425] Avg episode reward: [(0, '72.570'), (1, '72.000')] [2023-10-14 19:38:59,225][61552] Updated weights for policy 0, policy_version 48842 (0.0009) [2023-10-14 19:38:59,588][61552] Updated weights for policy 0, policy_version 48852 (0.0008) [2023-10-14 19:38:59,961][61552] Updated weights for policy 0, policy_version 48862 (0.0009) [2023-10-14 19:39:00,428][61585] Updated weights for policy 1, policy_version 48680 (0.0009) [2023-10-14 19:39:00,795][61585] Updated weights for policy 1, policy_version 48690 (0.0008) [2023-10-14 19:39:01,151][61585] Updated weights for policy 1, policy_version 48700 (0.0008) [2023-10-14 19:39:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 99909632. Throughput: 0: 1677.2, 1: 1663.9. Samples: 24983066. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:39:03,344][60425] Avg episode reward: [(0, '68.430'), (1, '73.500')] [2023-10-14 19:39:03,963][61552] Updated weights for policy 0, policy_version 48872 (0.0010) [2023-10-14 19:39:04,338][61552] Updated weights for policy 0, policy_version 48882 (0.0009) [2023-10-14 19:39:04,702][61552] Updated weights for policy 0, policy_version 48892 (0.0009) [2023-10-14 19:39:05,159][61585] Updated weights for policy 1, policy_version 48710 (0.0008) [2023-10-14 19:39:05,542][61585] Updated weights for policy 1, policy_version 48720 (0.0007) [2023-10-14 19:39:05,910][61585] Updated weights for policy 1, policy_version 48730 (0.0007) [2023-10-14 19:39:08,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 99975168. Throughput: 0: 1681.3, 1: 1668.8. Samples: 25003216. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:39:08,344][60425] Avg episode reward: [(0, '75.310'), (1, '72.890')] [2023-10-14 19:39:08,769][61552] Updated weights for policy 0, policy_version 48902 (0.0009) [2023-10-14 19:39:09,132][61552] Updated weights for policy 0, policy_version 48912 (0.0007) [2023-10-14 19:39:09,493][61552] Updated weights for policy 0, policy_version 48922 (0.0008) [2023-10-14 19:39:09,890][61585] Updated weights for policy 1, policy_version 48740 (0.0008) [2023-10-14 19:39:10,252][61585] Updated weights for policy 1, policy_version 48750 (0.0008) [2023-10-14 19:39:10,619][61585] Updated weights for policy 1, policy_version 48760 (0.0010) [2023-10-14 19:39:13,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 100040704. Throughput: 0: 1683.6, 1: 1683.2. Samples: 25024016. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:39:13,344][60425] Avg episode reward: [(0, '75.230'), (1, '77.150')] [2023-10-14 19:39:13,628][61552] Updated weights for policy 0, policy_version 48932 (0.0007) [2023-10-14 19:39:14,026][61552] Updated weights for policy 0, policy_version 48942 (0.0009) [2023-10-14 19:39:14,384][61552] Updated weights for policy 0, policy_version 48952 (0.0011) [2023-10-14 19:39:14,618][61585] Updated weights for policy 1, policy_version 48770 (0.0011) [2023-10-14 19:39:14,987][61585] Updated weights for policy 1, policy_version 48780 (0.0008) [2023-10-14 19:39:15,344][61585] Updated weights for policy 1, policy_version 48790 (0.0008) [2023-10-14 19:39:15,714][61585] Updated weights for policy 1, policy_version 48800 (0.0008) [2023-10-14 19:39:18,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 100106240. Throughput: 0: 1681.2, 1: 1658.8. Samples: 25033132. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:39:18,344][60425] Avg episode reward: [(0, '75.870'), (1, '75.910')] [2023-10-14 19:39:18,490][61552] Updated weights for policy 0, policy_version 48962 (0.0008) [2023-10-14 19:39:18,857][61552] Updated weights for policy 0, policy_version 48972 (0.0008) [2023-10-14 19:39:19,231][61552] Updated weights for policy 0, policy_version 48982 (0.0008) [2023-10-14 19:39:19,603][61552] Updated weights for policy 0, policy_version 48992 (0.0009) [2023-10-14 19:39:19,946][61585] Updated weights for policy 1, policy_version 48810 (0.0007) [2023-10-14 19:39:20,316][61585] Updated weights for policy 1, policy_version 48820 (0.0007) [2023-10-14 19:39:20,685][61585] Updated weights for policy 1, policy_version 48830 (0.0007) [2023-10-14 19:39:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 100171776. Throughput: 0: 1674.8, 1: 1675.8. Samples: 25053516. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:39:23,344][60425] Avg episode reward: [(0, '72.110'), (1, '77.320')] [2023-10-14 19:39:23,542][61552] Updated weights for policy 0, policy_version 49002 (0.0008) [2023-10-14 19:39:23,916][61552] Updated weights for policy 0, policy_version 49012 (0.0009) [2023-10-14 19:39:24,288][61552] Updated weights for policy 0, policy_version 49022 (0.0009) [2023-10-14 19:39:24,836][61585] Updated weights for policy 1, policy_version 48840 (0.0009) [2023-10-14 19:39:25,193][61585] Updated weights for policy 1, policy_version 48850 (0.0008) [2023-10-14 19:39:25,566][61585] Updated weights for policy 1, policy_version 48860 (0.0009) [2023-10-14 19:39:28,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 100237312. Throughput: 0: 1679.8, 1: 1677.8. Samples: 25074240. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:39:28,344][60425] Avg episode reward: [(0, '70.440'), (1, '75.920')] [2023-10-14 19:39:28,447][61552] Updated weights for policy 0, policy_version 49032 (0.0009) [2023-10-14 19:39:28,808][61552] Updated weights for policy 0, policy_version 49042 (0.0007) [2023-10-14 19:39:29,171][61552] Updated weights for policy 0, policy_version 49052 (0.0007) [2023-10-14 19:39:29,556][61585] Updated weights for policy 1, policy_version 48870 (0.0008) [2023-10-14 19:39:29,926][61585] Updated weights for policy 1, policy_version 48880 (0.0007) [2023-10-14 19:39:30,294][61585] Updated weights for policy 1, policy_version 48890 (0.0007) [2023-10-14 19:39:33,088][61552] Updated weights for policy 0, policy_version 49062 (0.0008) [2023-10-14 19:39:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 100302848. Throughput: 0: 1683.2, 1: 1657.5. Samples: 25083278. Policy #0 lag: (min: 27.0, avg: 27.5, max: 42.0) [2023-10-14 19:39:33,344][60425] Avg episode reward: [(0, '71.720'), (1, '67.950')] [2023-10-14 19:39:33,465][61552] Updated weights for policy 0, policy_version 49072 (0.0009) [2023-10-14 19:39:33,830][61552] Updated weights for policy 0, policy_version 49082 (0.0009) [2023-10-14 19:39:34,212][61585] Updated weights for policy 1, policy_version 48900 (0.0008) [2023-10-14 19:39:34,584][61585] Updated weights for policy 1, policy_version 48910 (0.0007) [2023-10-14 19:39:34,945][61585] Updated weights for policy 1, policy_version 48920 (0.0008) [2023-10-14 19:39:38,008][61552] Updated weights for policy 0, policy_version 49092 (0.0009) [2023-10-14 19:39:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 100368384. Throughput: 0: 1682.5, 1: 1679.7. Samples: 25104206. Policy #0 lag: (min: 27.0, avg: 27.5, max: 42.0) [2023-10-14 19:39:38,344][60425] Avg episode reward: [(0, '74.680'), (1, '70.070')] [2023-10-14 19:39:38,369][61552] Updated weights for policy 0, policy_version 49102 (0.0007) [2023-10-14 19:39:38,735][61552] Updated weights for policy 0, policy_version 49112 (0.0008) [2023-10-14 19:39:38,929][61585] Updated weights for policy 1, policy_version 48930 (0.0008) [2023-10-14 19:39:39,295][61585] Updated weights for policy 1, policy_version 48940 (0.0008) [2023-10-14 19:39:39,651][61585] Updated weights for policy 1, policy_version 48950 (0.0008) [2023-10-14 19:39:40,021][61585] Updated weights for policy 1, policy_version 48960 (0.0007) [2023-10-14 19:39:42,819][61552] Updated weights for policy 0, policy_version 49122 (0.0008) [2023-10-14 19:39:43,199][61552] Updated weights for policy 0, policy_version 49132 (0.0011) [2023-10-14 19:39:43,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 100433920. Throughput: 0: 1682.1, 1: 1682.7. Samples: 25124830. Policy #0 lag: (min: 27.0, avg: 27.5, max: 42.0) [2023-10-14 19:39:43,345][60425] Avg episode reward: [(0, '73.750'), (1, '65.630')] [2023-10-14 19:39:43,354][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000048960_50135040.pth... [2023-10-14 19:39:43,389][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000047392_48529408.pth [2023-10-14 19:39:43,567][61552] Updated weights for policy 0, policy_version 49142 (0.0008) [2023-10-14 19:39:43,924][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000049152_50331648.pth... [2023-10-14 19:39:43,924][61552] Updated weights for policy 0, policy_version 49152 (0.0009) [2023-10-14 19:39:43,958][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000047584_48726016.pth [2023-10-14 19:39:44,263][61585] Updated weights for policy 1, policy_version 48970 (0.0011) [2023-10-14 19:39:44,622][61585] Updated weights for policy 1, policy_version 48980 (0.0010) [2023-10-14 19:39:44,987][61585] Updated weights for policy 1, policy_version 48990 (0.0008) [2023-10-14 19:39:48,007][61552] Updated weights for policy 0, policy_version 49162 (0.0009) [2023-10-14 19:39:48,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 100499456. Throughput: 0: 1683.2, 1: 1666.5. Samples: 25133804. Policy #0 lag: (min: 27.0, avg: 27.5, max: 42.0) [2023-10-14 19:39:48,344][60425] Avg episode reward: [(0, '67.590'), (1, '67.500')] [2023-10-14 19:39:48,378][61552] Updated weights for policy 0, policy_version 49172 (0.0009) [2023-10-14 19:39:48,754][61552] Updated weights for policy 0, policy_version 49182 (0.0008) [2023-10-14 19:39:49,200][61585] Updated weights for policy 1, policy_version 49000 (0.0010) [2023-10-14 19:39:49,560][61585] Updated weights for policy 1, policy_version 49010 (0.0010) [2023-10-14 19:39:49,934][61585] Updated weights for policy 1, policy_version 49020 (0.0007) [2023-10-14 19:39:52,857][61552] Updated weights for policy 0, policy_version 49192 (0.0010) [2023-10-14 19:39:53,226][61552] Updated weights for policy 0, policy_version 49202 (0.0008) [2023-10-14 19:39:53,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 100564992. Throughput: 0: 1679.3, 1: 1675.0. Samples: 25154160. Policy #0 lag: (min: 27.0, avg: 27.5, max: 42.0) [2023-10-14 19:39:53,344][60425] Avg episode reward: [(0, '71.080'), (1, '66.620')] [2023-10-14 19:39:53,598][61552] Updated weights for policy 0, policy_version 49212 (0.0010) [2023-10-14 19:39:54,223][61585] Updated weights for policy 1, policy_version 49030 (0.0008) [2023-10-14 19:39:54,614][61585] Updated weights for policy 1, policy_version 49040 (0.0010) [2023-10-14 19:39:54,974][61585] Updated weights for policy 1, policy_version 49050 (0.0007) [2023-10-14 19:39:57,653][61552] Updated weights for policy 0, policy_version 49222 (0.0009) [2023-10-14 19:39:58,032][61552] Updated weights for policy 0, policy_version 49232 (0.0008) [2023-10-14 19:39:58,343][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 100630528. Throughput: 0: 1669.3, 1: 1671.7. Samples: 25174364. Policy #0 lag: (min: 27.0, avg: 27.5, max: 42.0) [2023-10-14 19:39:58,345][60425] Avg episode reward: [(0, '75.650'), (1, '69.690')] [2023-10-14 19:39:58,406][61552] Updated weights for policy 0, policy_version 49242 (0.0007) [2023-10-14 19:39:58,907][61585] Updated weights for policy 1, policy_version 49060 (0.0007) [2023-10-14 19:39:59,274][61585] Updated weights for policy 1, policy_version 49070 (0.0008) [2023-10-14 19:39:59,644][61585] Updated weights for policy 1, policy_version 49080 (0.0008) [2023-10-14 19:40:02,640][61552] Updated weights for policy 0, policy_version 49252 (0.0008) [2023-10-14 19:40:03,048][61552] Updated weights for policy 0, policy_version 49262 (0.0007) [2023-10-14 19:40:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 100696064. Throughput: 0: 1677.4, 1: 1669.6. Samples: 25183750. Policy #0 lag: (min: 27.0, avg: 27.5, max: 42.0) [2023-10-14 19:40:03,344][60425] Avg episode reward: [(0, '75.150'), (1, '68.070')] [2023-10-14 19:40:03,424][61552] Updated weights for policy 0, policy_version 49272 (0.0007) [2023-10-14 19:40:03,757][61585] Updated weights for policy 1, policy_version 49090 (0.0010) [2023-10-14 19:40:04,119][61585] Updated weights for policy 1, policy_version 49100 (0.0008) [2023-10-14 19:40:04,478][61585] Updated weights for policy 1, policy_version 49110 (0.0009) [2023-10-14 19:40:04,843][61585] Updated weights for policy 1, policy_version 49120 (0.0011) [2023-10-14 19:40:07,366][61552] Updated weights for policy 0, policy_version 49282 (0.0008) [2023-10-14 19:40:07,720][61552] Updated weights for policy 0, policy_version 49292 (0.0009) [2023-10-14 19:40:08,103][61552] Updated weights for policy 0, policy_version 49302 (0.0009) [2023-10-14 19:40:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 100761600. Throughput: 0: 1674.7, 1: 1673.2. Samples: 25204172. Policy #0 lag: (min: 21.0, avg: 28.8, max: 53.0) [2023-10-14 19:40:08,345][60425] Avg episode reward: [(0, '72.490'), (1, '70.620')] [2023-10-14 19:40:08,476][61552] Updated weights for policy 0, policy_version 49312 (0.0008) [2023-10-14 19:40:08,805][61585] Updated weights for policy 1, policy_version 49130 (0.0009) [2023-10-14 19:40:09,167][61585] Updated weights for policy 1, policy_version 49140 (0.0008) [2023-10-14 19:40:09,531][61585] Updated weights for policy 1, policy_version 49150 (0.0008) [2023-10-14 19:40:12,609][61552] Updated weights for policy 0, policy_version 49322 (0.0008) [2023-10-14 19:40:12,976][61552] Updated weights for policy 0, policy_version 49332 (0.0007) [2023-10-14 19:40:13,341][61552] Updated weights for policy 0, policy_version 49342 (0.0008) [2023-10-14 19:40:13,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 100827136. Throughput: 0: 1663.6, 1: 1677.2. Samples: 25224572. Policy #0 lag: (min: 21.0, avg: 28.8, max: 53.0) [2023-10-14 19:40:13,344][60425] Avg episode reward: [(0, '74.840'), (1, '71.680')] [2023-10-14 19:40:13,491][61585] Updated weights for policy 1, policy_version 49160 (0.0008) [2023-10-14 19:40:13,868][61585] Updated weights for policy 1, policy_version 49170 (0.0008) [2023-10-14 19:40:14,224][61585] Updated weights for policy 1, policy_version 49180 (0.0008) [2023-10-14 19:40:17,271][61552] Updated weights for policy 0, policy_version 49352 (0.0009) [2023-10-14 19:40:17,637][61552] Updated weights for policy 0, policy_version 49362 (0.0009) [2023-10-14 19:40:18,007][61552] Updated weights for policy 0, policy_version 49372 (0.0007) [2023-10-14 19:40:18,220][61585] Updated weights for policy 1, policy_version 49190 (0.0008) [2023-10-14 19:40:18,343][60425] Fps is (10 sec: 16384.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 100925440. Throughput: 0: 1675.6, 1: 1680.3. Samples: 25234294. Policy #0 lag: (min: 21.0, avg: 28.8, max: 53.0) [2023-10-14 19:40:18,344][60425] Avg episode reward: [(0, '75.160'), (1, '71.700')] [2023-10-14 19:40:18,586][61585] Updated weights for policy 1, policy_version 49200 (0.0010) [2023-10-14 19:40:18,954][61585] Updated weights for policy 1, policy_version 49210 (0.0009) [2023-10-14 19:40:22,121][61552] Updated weights for policy 0, policy_version 49382 (0.0008) [2023-10-14 19:40:22,491][61552] Updated weights for policy 0, policy_version 49392 (0.0009) [2023-10-14 19:40:22,853][61552] Updated weights for policy 0, policy_version 49402 (0.0009) [2023-10-14 19:40:23,085][61585] Updated weights for policy 1, policy_version 49220 (0.0009) [2023-10-14 19:40:23,343][60425] Fps is (10 sec: 16384.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 100990976. Throughput: 0: 1671.8, 1: 1675.2. Samples: 25254822. Policy #0 lag: (min: 21.0, avg: 28.8, max: 53.0) [2023-10-14 19:40:23,344][60425] Avg episode reward: [(0, '76.790'), (1, '75.240')] [2023-10-14 19:40:23,446][61585] Updated weights for policy 1, policy_version 49230 (0.0008) [2023-10-14 19:40:23,811][61585] Updated weights for policy 1, policy_version 49240 (0.0009) [2023-10-14 19:40:26,903][61552] Updated weights for policy 0, policy_version 49412 (0.0009) [2023-10-14 19:40:27,267][61552] Updated weights for policy 0, policy_version 49422 (0.0011) [2023-10-14 19:40:27,632][61552] Updated weights for policy 0, policy_version 49432 (0.0011) [2023-10-14 19:40:28,018][61585] Updated weights for policy 1, policy_version 49250 (0.0010) [2023-10-14 19:40:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 101056512. Throughput: 0: 1648.4, 1: 1676.2. Samples: 25274438. Policy #0 lag: (min: 21.0, avg: 28.8, max: 53.0) [2023-10-14 19:40:28,344][60425] Avg episode reward: [(0, '73.770'), (1, '75.140')] [2023-10-14 19:40:28,392][61585] Updated weights for policy 1, policy_version 49260 (0.0009) [2023-10-14 19:40:28,760][61585] Updated weights for policy 1, policy_version 49270 (0.0007) [2023-10-14 19:40:29,126][61585] Updated weights for policy 1, policy_version 49280 (0.0009) [2023-10-14 19:40:31,852][61552] Updated weights for policy 0, policy_version 49442 (0.0009) [2023-10-14 19:40:32,218][61552] Updated weights for policy 0, policy_version 49452 (0.0008) [2023-10-14 19:40:32,591][61552] Updated weights for policy 0, policy_version 49462 (0.0007) [2023-10-14 19:40:32,963][61552] Updated weights for policy 0, policy_version 49472 (0.0007) [2023-10-14 19:40:33,135][61585] Updated weights for policy 1, policy_version 49290 (0.0008) [2023-10-14 19:40:33,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 101122048. Throughput: 0: 1674.7, 1: 1678.7. Samples: 25284710. Policy #0 lag: (min: 21.0, avg: 28.8, max: 53.0) [2023-10-14 19:40:33,344][60425] Avg episode reward: [(0, '78.690'), (1, '72.920')] [2023-10-14 19:40:33,346][61172] Saving new best policy, reward=78.690! [2023-10-14 19:40:33,500][61585] Updated weights for policy 1, policy_version 49300 (0.0008) [2023-10-14 19:40:33,866][61585] Updated weights for policy 1, policy_version 49310 (0.0008) [2023-10-14 19:40:37,047][61552] Updated weights for policy 0, policy_version 49482 (0.0011) [2023-10-14 19:40:37,413][61552] Updated weights for policy 0, policy_version 49492 (0.0010) [2023-10-14 19:40:37,786][61552] Updated weights for policy 0, policy_version 49502 (0.0008) [2023-10-14 19:40:37,893][61585] Updated weights for policy 1, policy_version 49320 (0.0009) [2023-10-14 19:40:38,258][61585] Updated weights for policy 1, policy_version 49330 (0.0010) [2023-10-14 19:40:38,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 101187584. Throughput: 0: 1673.3, 1: 1684.4. Samples: 25305256. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-10-14 19:40:38,345][60425] Avg episode reward: [(0, '75.410'), (1, '74.990')] [2023-10-14 19:40:38,626][61585] Updated weights for policy 1, policy_version 49340 (0.0009) [2023-10-14 19:40:41,877][61552] Updated weights for policy 0, policy_version 49512 (0.0009) [2023-10-14 19:40:42,249][61552] Updated weights for policy 0, policy_version 49522 (0.0009) [2023-10-14 19:40:42,610][61552] Updated weights for policy 0, policy_version 49532 (0.0008) [2023-10-14 19:40:42,933][61585] Updated weights for policy 1, policy_version 49350 (0.0010) [2023-10-14 19:40:43,325][61585] Updated weights for policy 1, policy_version 49360 (0.0009) [2023-10-14 19:40:43,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 101253120. Throughput: 0: 1658.5, 1: 1682.8. Samples: 25324724. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-10-14 19:40:43,344][60425] Avg episode reward: [(0, '77.010'), (1, '72.260')] [2023-10-14 19:40:43,698][61585] Updated weights for policy 1, policy_version 49370 (0.0008) [2023-10-14 19:40:46,683][61552] Updated weights for policy 0, policy_version 49542 (0.0009) [2023-10-14 19:40:47,042][61552] Updated weights for policy 0, policy_version 49552 (0.0011) [2023-10-14 19:40:47,414][61552] Updated weights for policy 0, policy_version 49562 (0.0010) [2023-10-14 19:40:47,578][61585] Updated weights for policy 1, policy_version 49380 (0.0008) [2023-10-14 19:40:47,947][61585] Updated weights for policy 1, policy_version 49390 (0.0009) [2023-10-14 19:40:48,317][61585] Updated weights for policy 1, policy_version 49400 (0.0011) [2023-10-14 19:40:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 101318656. Throughput: 0: 1678.6, 1: 1680.7. Samples: 25334920. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-10-14 19:40:48,344][60425] Avg episode reward: [(0, '77.930'), (1, '71.370')] [2023-10-14 19:40:51,597][61552] Updated weights for policy 0, policy_version 49572 (0.0008) [2023-10-14 19:40:51,987][61552] Updated weights for policy 0, policy_version 49582 (0.0007) [2023-10-14 19:40:52,343][61585] Updated weights for policy 1, policy_version 49410 (0.0008) [2023-10-14 19:40:52,350][61552] Updated weights for policy 0, policy_version 49592 (0.0007) [2023-10-14 19:40:52,707][61585] Updated weights for policy 1, policy_version 49420 (0.0007) [2023-10-14 19:40:53,062][61585] Updated weights for policy 1, policy_version 49430 (0.0008) [2023-10-14 19:40:53,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 101384192. Throughput: 0: 1673.2, 1: 1689.2. Samples: 25355476. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-10-14 19:40:53,344][60425] Avg episode reward: [(0, '75.010'), (1, '74.340')] [2023-10-14 19:40:53,421][61585] Updated weights for policy 1, policy_version 49440 (0.0009) [2023-10-14 19:40:56,532][61552] Updated weights for policy 0, policy_version 49602 (0.0008) [2023-10-14 19:40:56,895][61552] Updated weights for policy 0, policy_version 49612 (0.0009) [2023-10-14 19:40:57,270][61552] Updated weights for policy 0, policy_version 49622 (0.0009) [2023-10-14 19:40:57,605][61585] Updated weights for policy 1, policy_version 49450 (0.0009) [2023-10-14 19:40:57,640][61552] Updated weights for policy 0, policy_version 49632 (0.0008) [2023-10-14 19:40:57,973][61585] Updated weights for policy 1, policy_version 49460 (0.0009) [2023-10-14 19:40:58,337][61585] Updated weights for policy 1, policy_version 49470 (0.0007) [2023-10-14 19:40:58,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 101449728. Throughput: 0: 1659.7, 1: 1673.2. Samples: 25374554. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-10-14 19:40:58,344][60425] Avg episode reward: [(0, '75.380'), (1, '69.830')] [2023-10-14 19:41:01,667][61552] Updated weights for policy 0, policy_version 49642 (0.0009) [2023-10-14 19:41:02,030][61552] Updated weights for policy 0, policy_version 49652 (0.0010) [2023-10-14 19:41:02,403][61552] Updated weights for policy 0, policy_version 49662 (0.0008) [2023-10-14 19:41:02,602][61585] Updated weights for policy 1, policy_version 49480 (0.0008) [2023-10-14 19:41:02,962][61585] Updated weights for policy 1, policy_version 49490 (0.0010) [2023-10-14 19:41:03,331][61585] Updated weights for policy 1, policy_version 49500 (0.0007) [2023-10-14 19:41:03,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 101515264. Throughput: 0: 1674.0, 1: 1679.7. Samples: 25385210. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-10-14 19:41:03,344][60425] Avg episode reward: [(0, '74.330'), (1, '70.530')] [2023-10-14 19:41:06,652][61552] Updated weights for policy 0, policy_version 49672 (0.0007) [2023-10-14 19:41:07,029][61552] Updated weights for policy 0, policy_version 49682 (0.0008) [2023-10-14 19:41:07,317][61585] Updated weights for policy 1, policy_version 49510 (0.0007) [2023-10-14 19:41:07,392][61552] Updated weights for policy 0, policy_version 49692 (0.0009) [2023-10-14 19:41:07,676][61585] Updated weights for policy 1, policy_version 49520 (0.0008) [2023-10-14 19:41:08,058][61585] Updated weights for policy 1, policy_version 49530 (0.0011) [2023-10-14 19:41:08,343][60425] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 101613568. Throughput: 0: 1667.9, 1: 1682.8. Samples: 25405604. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-10-14 19:41:08,344][60425] Avg episode reward: [(0, '73.870'), (1, '73.360')] [2023-10-14 19:41:11,380][61552] Updated weights for policy 0, policy_version 49702 (0.0008) [2023-10-14 19:41:11,746][61552] Updated weights for policy 0, policy_version 49712 (0.0007) [2023-10-14 19:41:12,113][61552] Updated weights for policy 0, policy_version 49722 (0.0007) [2023-10-14 19:41:12,148][61585] Updated weights for policy 1, policy_version 49540 (0.0008) [2023-10-14 19:41:12,516][61585] Updated weights for policy 1, policy_version 49550 (0.0008) [2023-10-14 19:41:12,878][61585] Updated weights for policy 1, policy_version 49560 (0.0011) [2023-10-14 19:41:13,344][60425] Fps is (10 sec: 16383.5, 60 sec: 14199.4, 300 sec: 13551.5). Total num frames: 101679104. Throughput: 0: 1672.3, 1: 1669.1. Samples: 25424802. Policy #0 lag: (min: 14.0, avg: 14.9, max: 35.0) [2023-10-14 19:41:13,345][60425] Avg episode reward: [(0, '73.850'), (1, '69.840')] [2023-10-14 19:41:16,166][61552] Updated weights for policy 0, policy_version 49732 (0.0009) [2023-10-14 19:41:16,545][61552] Updated weights for policy 0, policy_version 49742 (0.0008) [2023-10-14 19:41:16,905][61552] Updated weights for policy 0, policy_version 49752 (0.0009) [2023-10-14 19:41:16,922][61585] Updated weights for policy 1, policy_version 49570 (0.0009) [2023-10-14 19:41:17,294][61585] Updated weights for policy 1, policy_version 49580 (0.0007) [2023-10-14 19:41:17,658][61585] Updated weights for policy 1, policy_version 49590 (0.0007) [2023-10-14 19:41:18,031][61585] Updated weights for policy 1, policy_version 49600 (0.0010) [2023-10-14 19:41:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 101744640. Throughput: 0: 1672.3, 1: 1685.1. Samples: 25435794. Policy #0 lag: (min: 14.0, avg: 14.9, max: 35.0) [2023-10-14 19:41:18,344][60425] Avg episode reward: [(0, '75.640'), (1, '68.580')] [2023-10-14 19:41:21,054][61552] Updated weights for policy 0, policy_version 49762 (0.0007) [2023-10-14 19:41:21,420][61552] Updated weights for policy 0, policy_version 49772 (0.0008) [2023-10-14 19:41:21,791][61552] Updated weights for policy 0, policy_version 49782 (0.0008) [2023-10-14 19:41:22,155][61552] Updated weights for policy 0, policy_version 49792 (0.0007) [2023-10-14 19:41:22,279][61585] Updated weights for policy 1, policy_version 49610 (0.0008) [2023-10-14 19:41:22,644][61585] Updated weights for policy 1, policy_version 49620 (0.0008) [2023-10-14 19:41:23,010][61585] Updated weights for policy 1, policy_version 49630 (0.0007) [2023-10-14 19:41:23,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 101810176. Throughput: 0: 1655.2, 1: 1685.0. Samples: 25455566. Policy #0 lag: (min: 14.0, avg: 14.9, max: 35.0) [2023-10-14 19:41:23,344][60425] Avg episode reward: [(0, '72.340'), (1, '64.870')] [2023-10-14 19:41:26,274][61552] Updated weights for policy 0, policy_version 49802 (0.0008) [2023-10-14 19:41:26,662][61552] Updated weights for policy 0, policy_version 49812 (0.0009) [2023-10-14 19:41:27,023][61552] Updated weights for policy 0, policy_version 49822 (0.0008) [2023-10-14 19:41:27,139][61585] Updated weights for policy 1, policy_version 49640 (0.0009) [2023-10-14 19:41:27,501][61585] Updated weights for policy 1, policy_version 49650 (0.0010) [2023-10-14 19:41:27,866][61585] Updated weights for policy 1, policy_version 49660 (0.0009) [2023-10-14 19:41:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 101875712. Throughput: 0: 1662.7, 1: 1663.5. Samples: 25474404. Policy #0 lag: (min: 14.0, avg: 14.9, max: 35.0) [2023-10-14 19:41:28,344][60425] Avg episode reward: [(0, '73.600'), (1, '71.520')] [2023-10-14 19:41:31,060][61552] Updated weights for policy 0, policy_version 49832 (0.0008) [2023-10-14 19:41:31,420][61552] Updated weights for policy 0, policy_version 49842 (0.0009) [2023-10-14 19:41:31,794][61552] Updated weights for policy 0, policy_version 49852 (0.0009) [2023-10-14 19:41:32,030][61585] Updated weights for policy 1, policy_version 49670 (0.0008) [2023-10-14 19:41:32,393][61585] Updated weights for policy 1, policy_version 49680 (0.0010) [2023-10-14 19:41:32,760][61585] Updated weights for policy 1, policy_version 49690 (0.0011) [2023-10-14 19:41:33,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 101941248. Throughput: 0: 1666.4, 1: 1682.5. Samples: 25485620. Policy #0 lag: (min: 14.0, avg: 14.9, max: 35.0) [2023-10-14 19:41:33,344][60425] Avg episode reward: [(0, '71.070'), (1, '72.910')] [2023-10-14 19:41:35,767][61552] Updated weights for policy 0, policy_version 49862 (0.0009) [2023-10-14 19:41:36,132][61552] Updated weights for policy 0, policy_version 49872 (0.0009) [2023-10-14 19:41:36,499][61552] Updated weights for policy 0, policy_version 49882 (0.0008) [2023-10-14 19:41:36,878][61585] Updated weights for policy 1, policy_version 49700 (0.0009) [2023-10-14 19:41:37,235][61585] Updated weights for policy 1, policy_version 49710 (0.0008) [2023-10-14 19:41:37,598][61585] Updated weights for policy 1, policy_version 49720 (0.0009) [2023-10-14 19:41:38,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 102006784. Throughput: 0: 1655.8, 1: 1669.2. Samples: 25505102. Policy #0 lag: (min: 14.0, avg: 14.9, max: 35.0) [2023-10-14 19:41:38,344][60425] Avg episode reward: [(0, '73.740'), (1, '71.630')] [2023-10-14 19:41:40,626][61552] Updated weights for policy 0, policy_version 49892 (0.0009) [2023-10-14 19:41:41,016][61552] Updated weights for policy 0, policy_version 49902 (0.0008) [2023-10-14 19:41:41,385][61552] Updated weights for policy 0, policy_version 49912 (0.0008) [2023-10-14 19:41:41,601][61585] Updated weights for policy 1, policy_version 49730 (0.0008) [2023-10-14 19:41:41,968][61585] Updated weights for policy 1, policy_version 49740 (0.0011) [2023-10-14 19:41:42,343][61585] Updated weights for policy 1, policy_version 49750 (0.0011) [2023-10-14 19:41:42,705][61585] Updated weights for policy 1, policy_version 49760 (0.0009) [2023-10-14 19:41:43,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 102072320. Throughput: 0: 1677.1, 1: 1653.9. Samples: 25524446. Policy #0 lag: (min: 14.0, avg: 14.9, max: 35.0) [2023-10-14 19:41:43,345][60425] Avg episode reward: [(0, '72.550'), (1, '73.310')] [2023-10-14 19:41:43,356][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000049920_51118080.pth... [2023-10-14 19:41:43,356][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000049760_50954240.pth... [2023-10-14 19:41:43,386][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000048352_49512448.pth [2023-10-14 19:41:43,396][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000048192_49348608.pth [2023-10-14 19:41:45,342][61552] Updated weights for policy 0, policy_version 49922 (0.0008) [2023-10-14 19:41:45,709][61552] Updated weights for policy 0, policy_version 49932 (0.0008) [2023-10-14 19:41:46,081][61552] Updated weights for policy 0, policy_version 49942 (0.0008) [2023-10-14 19:41:46,456][61552] Updated weights for policy 0, policy_version 49952 (0.0007) [2023-10-14 19:41:46,836][61585] Updated weights for policy 1, policy_version 49770 (0.0011) [2023-10-14 19:41:47,197][61585] Updated weights for policy 1, policy_version 49780 (0.0008) [2023-10-14 19:41:47,559][61585] Updated weights for policy 1, policy_version 49790 (0.0009) [2023-10-14 19:41:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 102137856. Throughput: 0: 1665.6, 1: 1671.8. Samples: 25535390. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:41:48,344][60425] Avg episode reward: [(0, '74.660'), (1, '69.900')] [2023-10-14 19:41:50,640][61552] Updated weights for policy 0, policy_version 49962 (0.0008) [2023-10-14 19:41:51,007][61552] Updated weights for policy 0, policy_version 49972 (0.0010) [2023-10-14 19:41:51,372][61552] Updated weights for policy 0, policy_version 49982 (0.0009) [2023-10-14 19:41:51,901][61585] Updated weights for policy 1, policy_version 49800 (0.0009) [2023-10-14 19:41:52,274][61585] Updated weights for policy 1, policy_version 49810 (0.0008) [2023-10-14 19:41:52,642][61585] Updated weights for policy 1, policy_version 49820 (0.0008) [2023-10-14 19:41:53,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 102203392. Throughput: 0: 1653.4, 1: 1658.8. Samples: 25554654. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:41:53,344][60425] Avg episode reward: [(0, '69.590'), (1, '72.510')] [2023-10-14 19:41:55,187][61552] Updated weights for policy 0, policy_version 49992 (0.0008) [2023-10-14 19:41:55,566][61552] Updated weights for policy 0, policy_version 50002 (0.0008) [2023-10-14 19:41:55,929][61552] Updated weights for policy 0, policy_version 50012 (0.0011) [2023-10-14 19:41:56,691][61585] Updated weights for policy 1, policy_version 49830 (0.0008) [2023-10-14 19:41:57,053][61585] Updated weights for policy 1, policy_version 49840 (0.0007) [2023-10-14 19:41:57,414][61585] Updated weights for policy 1, policy_version 49850 (0.0008) [2023-10-14 19:41:58,344][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 102268928. Throughput: 0: 1675.0, 1: 1647.8. Samples: 25574328. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:41:58,345][60425] Avg episode reward: [(0, '70.490'), (1, '71.050')] [2023-10-14 19:42:00,022][61552] Updated weights for policy 0, policy_version 50022 (0.0010) [2023-10-14 19:42:00,397][61552] Updated weights for policy 0, policy_version 50032 (0.0010) [2023-10-14 19:42:00,753][61552] Updated weights for policy 0, policy_version 50042 (0.0009) [2023-10-14 19:42:01,553][61585] Updated weights for policy 1, policy_version 49860 (0.0007) [2023-10-14 19:42:01,919][61585] Updated weights for policy 1, policy_version 49870 (0.0007) [2023-10-14 19:42:02,282][61585] Updated weights for policy 1, policy_version 49880 (0.0007) [2023-10-14 19:42:03,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 102334464. Throughput: 0: 1657.2, 1: 1656.9. Samples: 25584930. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:42:03,344][60425] Avg episode reward: [(0, '70.110'), (1, '77.030')] [2023-10-14 19:42:04,818][61552] Updated weights for policy 0, policy_version 50052 (0.0009) [2023-10-14 19:42:05,188][61552] Updated weights for policy 0, policy_version 50062 (0.0011) [2023-10-14 19:42:05,561][61552] Updated weights for policy 0, policy_version 50072 (0.0008) [2023-10-14 19:42:06,348][61585] Updated weights for policy 1, policy_version 49890 (0.0007) [2023-10-14 19:42:06,705][61585] Updated weights for policy 1, policy_version 49900 (0.0008) [2023-10-14 19:42:07,064][61585] Updated weights for policy 1, policy_version 49910 (0.0010) [2023-10-14 19:42:07,428][61585] Updated weights for policy 1, policy_version 49920 (0.0008) [2023-10-14 19:42:08,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 102400000. Throughput: 0: 1667.1, 1: 1644.0. Samples: 25604566. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:42:08,344][60425] Avg episode reward: [(0, '72.850'), (1, '73.680')] [2023-10-14 19:42:09,838][61552] Updated weights for policy 0, policy_version 50082 (0.0008) [2023-10-14 19:42:10,215][61552] Updated weights for policy 0, policy_version 50092 (0.0010) [2023-10-14 19:42:10,582][61552] Updated weights for policy 0, policy_version 50102 (0.0009) [2023-10-14 19:42:10,960][61552] Updated weights for policy 0, policy_version 50112 (0.0010) [2023-10-14 19:42:11,608][61585] Updated weights for policy 1, policy_version 49930 (0.0007) [2023-10-14 19:42:11,983][61585] Updated weights for policy 1, policy_version 49940 (0.0008) [2023-10-14 19:42:12,335][61585] Updated weights for policy 1, policy_version 49950 (0.0007) [2023-10-14 19:42:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 102465536. Throughput: 0: 1680.9, 1: 1651.0. Samples: 25624338. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:42:13,344][60425] Avg episode reward: [(0, '71.150'), (1, '72.540')] [2023-10-14 19:42:14,948][61552] Updated weights for policy 0, policy_version 50122 (0.0007) [2023-10-14 19:42:15,312][61552] Updated weights for policy 0, policy_version 50132 (0.0008) [2023-10-14 19:42:15,684][61552] Updated weights for policy 0, policy_version 50142 (0.0008) [2023-10-14 19:42:16,359][61585] Updated weights for policy 1, policy_version 49960 (0.0008) [2023-10-14 19:42:16,722][61585] Updated weights for policy 1, policy_version 49970 (0.0007) [2023-10-14 19:42:17,090][61585] Updated weights for policy 1, policy_version 49980 (0.0007) [2023-10-14 19:42:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 102531072. Throughput: 0: 1657.5, 1: 1663.6. Samples: 25635066. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:42:18,344][60425] Avg episode reward: [(0, '73.620'), (1, '72.560')] [2023-10-14 19:42:19,748][61552] Updated weights for policy 0, policy_version 50152 (0.0009) [2023-10-14 19:42:20,108][61552] Updated weights for policy 0, policy_version 50162 (0.0010) [2023-10-14 19:42:20,485][61552] Updated weights for policy 0, policy_version 50172 (0.0009) [2023-10-14 19:42:21,202][61585] Updated weights for policy 1, policy_version 49990 (0.0009) [2023-10-14 19:42:21,593][61585] Updated weights for policy 1, policy_version 50000 (0.0009) [2023-10-14 19:42:21,969][61585] Updated weights for policy 1, policy_version 50010 (0.0009) [2023-10-14 19:42:23,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 102596608. Throughput: 0: 1672.4, 1: 1651.8. Samples: 25654692. Policy #0 lag: (min: 26.0, avg: 26.0, max: 27.0) [2023-10-14 19:42:23,344][60425] Avg episode reward: [(0, '71.520'), (1, '75.380')] [2023-10-14 19:42:24,685][61552] Updated weights for policy 0, policy_version 50182 (0.0008) [2023-10-14 19:42:25,056][61552] Updated weights for policy 0, policy_version 50192 (0.0009) [2023-10-14 19:42:25,425][61552] Updated weights for policy 0, policy_version 50202 (0.0009) [2023-10-14 19:42:25,985][61585] Updated weights for policy 1, policy_version 50020 (0.0009) [2023-10-14 19:42:26,359][61585] Updated weights for policy 1, policy_version 50030 (0.0010) [2023-10-14 19:42:26,724][61585] Updated weights for policy 1, policy_version 50040 (0.0010) [2023-10-14 19:42:28,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 102662144. Throughput: 0: 1678.1, 1: 1665.9. Samples: 25674926. Policy #0 lag: (min: 26.0, avg: 26.0, max: 27.0) [2023-10-14 19:42:28,344][60425] Avg episode reward: [(0, '73.810'), (1, '76.950')] [2023-10-14 19:42:29,566][61552] Updated weights for policy 0, policy_version 50212 (0.0008) [2023-10-14 19:42:29,941][61552] Updated weights for policy 0, policy_version 50222 (0.0010) [2023-10-14 19:42:30,314][61552] Updated weights for policy 0, policy_version 50232 (0.0010) [2023-10-14 19:42:30,691][61585] Updated weights for policy 1, policy_version 50050 (0.0008) [2023-10-14 19:42:31,059][61585] Updated weights for policy 1, policy_version 50060 (0.0008) [2023-10-14 19:42:31,426][61585] Updated weights for policy 1, policy_version 50070 (0.0008) [2023-10-14 19:42:31,787][61585] Updated weights for policy 1, policy_version 50080 (0.0007) [2023-10-14 19:42:33,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 102727680. Throughput: 0: 1664.5, 1: 1664.0. Samples: 25685170. Policy #0 lag: (min: 26.0, avg: 26.0, max: 27.0) [2023-10-14 19:42:33,344][60425] Avg episode reward: [(0, '72.810'), (1, '74.520')] [2023-10-14 19:42:34,376][61552] Updated weights for policy 0, policy_version 50242 (0.0009) [2023-10-14 19:42:34,738][61552] Updated weights for policy 0, policy_version 50252 (0.0009) [2023-10-14 19:42:35,108][61552] Updated weights for policy 0, policy_version 50262 (0.0008) [2023-10-14 19:42:35,474][61552] Updated weights for policy 0, policy_version 50272 (0.0007) [2023-10-14 19:42:35,884][61585] Updated weights for policy 1, policy_version 50090 (0.0009) [2023-10-14 19:42:36,258][61585] Updated weights for policy 1, policy_version 50100 (0.0009) [2023-10-14 19:42:36,624][61585] Updated weights for policy 1, policy_version 50110 (0.0010) [2023-10-14 19:42:38,344][60425] Fps is (10 sec: 13106.7, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 102793216. Throughput: 0: 1682.6, 1: 1649.3. Samples: 25704592. Policy #0 lag: (min: 26.0, avg: 26.0, max: 27.0) [2023-10-14 19:42:38,345][60425] Avg episode reward: [(0, '75.820'), (1, '77.450')] [2023-10-14 19:42:39,554][61552] Updated weights for policy 0, policy_version 50282 (0.0009) [2023-10-14 19:42:39,915][61552] Updated weights for policy 0, policy_version 50292 (0.0009) [2023-10-14 19:42:40,281][61552] Updated weights for policy 0, policy_version 50302 (0.0007) [2023-10-14 19:42:40,902][61585] Updated weights for policy 1, policy_version 50120 (0.0008) [2023-10-14 19:42:41,265][61585] Updated weights for policy 1, policy_version 50130 (0.0009) [2023-10-14 19:42:41,639][61585] Updated weights for policy 1, policy_version 50140 (0.0011) [2023-10-14 19:42:43,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 102858752. Throughput: 0: 1680.7, 1: 1676.4. Samples: 25725398. Policy #0 lag: (min: 26.0, avg: 26.0, max: 27.0) [2023-10-14 19:42:43,344][60425] Avg episode reward: [(0, '75.400'), (1, '76.080')] [2023-10-14 19:42:44,319][61552] Updated weights for policy 0, policy_version 50312 (0.0010) [2023-10-14 19:42:44,685][61552] Updated weights for policy 0, policy_version 50322 (0.0010) [2023-10-14 19:42:45,050][61552] Updated weights for policy 0, policy_version 50332 (0.0012) [2023-10-14 19:42:45,704][61585] Updated weights for policy 1, policy_version 50150 (0.0009) [2023-10-14 19:42:46,074][61585] Updated weights for policy 1, policy_version 50160 (0.0009) [2023-10-14 19:42:46,440][61585] Updated weights for policy 1, policy_version 50170 (0.0010) [2023-10-14 19:42:48,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 102924288. Throughput: 0: 1670.9, 1: 1671.4. Samples: 25735336. Policy #0 lag: (min: 26.0, avg: 26.0, max: 27.0) [2023-10-14 19:42:48,344][60425] Avg episode reward: [(0, '75.260'), (1, '71.100')] [2023-10-14 19:42:49,126][61552] Updated weights for policy 0, policy_version 50342 (0.0009) [2023-10-14 19:42:49,486][61552] Updated weights for policy 0, policy_version 50352 (0.0008) [2023-10-14 19:42:49,858][61552] Updated weights for policy 0, policy_version 50362 (0.0008) [2023-10-14 19:42:50,678][61585] Updated weights for policy 1, policy_version 50180 (0.0009) [2023-10-14 19:42:51,044][61585] Updated weights for policy 1, policy_version 50190 (0.0009) [2023-10-14 19:42:51,402][61585] Updated weights for policy 1, policy_version 50200 (0.0007) [2023-10-14 19:42:53,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 102989824. Throughput: 0: 1682.3, 1: 1659.9. Samples: 25754964. Policy #0 lag: (min: 26.0, avg: 26.0, max: 27.0) [2023-10-14 19:42:53,344][60425] Avg episode reward: [(0, '75.550'), (1, '74.990')] [2023-10-14 19:42:53,828][61552] Updated weights for policy 0, policy_version 50372 (0.0008) [2023-10-14 19:42:54,200][61552] Updated weights for policy 0, policy_version 50382 (0.0007) [2023-10-14 19:42:54,559][61552] Updated weights for policy 0, policy_version 50392 (0.0007) [2023-10-14 19:42:55,389][61585] Updated weights for policy 1, policy_version 50210 (0.0009) [2023-10-14 19:42:55,749][61585] Updated weights for policy 1, policy_version 50220 (0.0008) [2023-10-14 19:42:56,120][61585] Updated weights for policy 1, policy_version 50230 (0.0009) [2023-10-14 19:42:56,490][61585] Updated weights for policy 1, policy_version 50240 (0.0010) [2023-10-14 19:42:58,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 103055360. Throughput: 0: 1687.6, 1: 1677.7. Samples: 25775776. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-14 19:42:58,344][60425] Avg episode reward: [(0, '72.130'), (1, '71.700')] [2023-10-14 19:42:58,647][61552] Updated weights for policy 0, policy_version 50402 (0.0008) [2023-10-14 19:42:59,023][61552] Updated weights for policy 0, policy_version 50412 (0.0009) [2023-10-14 19:42:59,390][61552] Updated weights for policy 0, policy_version 50422 (0.0010) [2023-10-14 19:42:59,753][61552] Updated weights for policy 0, policy_version 50432 (0.0011) [2023-10-14 19:43:00,593][61585] Updated weights for policy 1, policy_version 50250 (0.0007) [2023-10-14 19:43:00,953][61585] Updated weights for policy 1, policy_version 50260 (0.0007) [2023-10-14 19:43:01,326][61585] Updated weights for policy 1, policy_version 50270 (0.0010) [2023-10-14 19:43:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 103120896. Throughput: 0: 1677.4, 1: 1667.0. Samples: 25785562. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-14 19:43:03,344][60425] Avg episode reward: [(0, '75.470'), (1, '69.990')] [2023-10-14 19:43:04,098][61552] Updated weights for policy 0, policy_version 50442 (0.0008) [2023-10-14 19:43:04,459][61552] Updated weights for policy 0, policy_version 50452 (0.0007) [2023-10-14 19:43:04,835][61552] Updated weights for policy 0, policy_version 50462 (0.0010) [2023-10-14 19:43:05,339][61585] Updated weights for policy 1, policy_version 50280 (0.0009) [2023-10-14 19:43:05,709][61585] Updated weights for policy 1, policy_version 50290 (0.0008) [2023-10-14 19:43:06,067][61585] Updated weights for policy 1, policy_version 50300 (0.0008) [2023-10-14 19:43:08,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 103186432. Throughput: 0: 1679.0, 1: 1669.7. Samples: 25805384. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-14 19:43:08,344][60425] Avg episode reward: [(0, '73.870'), (1, '72.010')] [2023-10-14 19:43:08,768][61552] Updated weights for policy 0, policy_version 50472 (0.0009) [2023-10-14 19:43:09,127][61552] Updated weights for policy 0, policy_version 50482 (0.0009) [2023-10-14 19:43:09,499][61552] Updated weights for policy 0, policy_version 50492 (0.0011) [2023-10-14 19:43:10,303][61585] Updated weights for policy 1, policy_version 50310 (0.0008) [2023-10-14 19:43:10,681][61585] Updated weights for policy 1, policy_version 50320 (0.0007) [2023-10-14 19:43:11,048][61585] Updated weights for policy 1, policy_version 50330 (0.0011) [2023-10-14 19:43:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 103251968. Throughput: 0: 1677.6, 1: 1675.8. Samples: 25825832. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-14 19:43:13,344][60425] Avg episode reward: [(0, '68.090'), (1, '69.370')] [2023-10-14 19:43:13,610][61552] Updated weights for policy 0, policy_version 50502 (0.0008) [2023-10-14 19:43:13,988][61552] Updated weights for policy 0, policy_version 50512 (0.0009) [2023-10-14 19:43:14,360][61552] Updated weights for policy 0, policy_version 50522 (0.0009) [2023-10-14 19:43:15,052][61585] Updated weights for policy 1, policy_version 50340 (0.0010) [2023-10-14 19:43:15,420][61585] Updated weights for policy 1, policy_version 50350 (0.0008) [2023-10-14 19:43:15,797][61585] Updated weights for policy 1, policy_version 50360 (0.0007) [2023-10-14 19:43:18,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 103317504. Throughput: 0: 1675.6, 1: 1664.8. Samples: 25835488. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-14 19:43:18,344][60425] Avg episode reward: [(0, '68.930'), (1, '73.670')] [2023-10-14 19:43:18,460][61552] Updated weights for policy 0, policy_version 50532 (0.0009) [2023-10-14 19:43:18,840][61552] Updated weights for policy 0, policy_version 50542 (0.0010) [2023-10-14 19:43:19,216][61552] Updated weights for policy 0, policy_version 50552 (0.0009) [2023-10-14 19:43:19,878][61585] Updated weights for policy 1, policy_version 50370 (0.0010) [2023-10-14 19:43:20,242][61585] Updated weights for policy 1, policy_version 50380 (0.0009) [2023-10-14 19:43:20,619][61585] Updated weights for policy 1, policy_version 50390 (0.0010) [2023-10-14 19:43:20,987][61585] Updated weights for policy 1, policy_version 50400 (0.0010) [2023-10-14 19:43:23,178][61552] Updated weights for policy 0, policy_version 50562 (0.0008) [2023-10-14 19:43:23,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 103383040. Throughput: 0: 1676.1, 1: 1677.7. Samples: 25855512. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-14 19:43:23,344][60425] Avg episode reward: [(0, '71.050'), (1, '72.730')] [2023-10-14 19:43:23,553][61552] Updated weights for policy 0, policy_version 50572 (0.0007) [2023-10-14 19:43:23,922][61552] Updated weights for policy 0, policy_version 50582 (0.0008) [2023-10-14 19:43:24,287][61552] Updated weights for policy 0, policy_version 50592 (0.0009) [2023-10-14 19:43:25,034][61585] Updated weights for policy 1, policy_version 50410 (0.0010) [2023-10-14 19:43:25,414][61585] Updated weights for policy 1, policy_version 50420 (0.0007) [2023-10-14 19:43:25,788][61585] Updated weights for policy 1, policy_version 50430 (0.0010) [2023-10-14 19:43:28,335][61552] Updated weights for policy 0, policy_version 50602 (0.0007) [2023-10-14 19:43:28,344][60425] Fps is (10 sec: 13106.7, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 103448576. Throughput: 0: 1679.8, 1: 1672.1. Samples: 25876236. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-14 19:43:28,345][60425] Avg episode reward: [(0, '72.130'), (1, '70.360')] [2023-10-14 19:43:28,711][61552] Updated weights for policy 0, policy_version 50612 (0.0007) [2023-10-14 19:43:29,087][61552] Updated weights for policy 0, policy_version 50622 (0.0008) [2023-10-14 19:43:29,989][61585] Updated weights for policy 1, policy_version 50440 (0.0010) [2023-10-14 19:43:30,357][61585] Updated weights for policy 1, policy_version 50450 (0.0010) [2023-10-14 19:43:30,729][61585] Updated weights for policy 1, policy_version 50460 (0.0009) [2023-10-14 19:43:33,221][61552] Updated weights for policy 0, policy_version 50632 (0.0008) [2023-10-14 19:43:33,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 103514112. Throughput: 0: 1684.4, 1: 1655.5. Samples: 25885630. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:43:33,344][60425] Avg episode reward: [(0, '71.670'), (1, '68.810')] [2023-10-14 19:43:33,603][61552] Updated weights for policy 0, policy_version 50642 (0.0009) [2023-10-14 19:43:33,968][61552] Updated weights for policy 0, policy_version 50652 (0.0010) [2023-10-14 19:43:34,944][61585] Updated weights for policy 1, policy_version 50470 (0.0008) [2023-10-14 19:43:35,303][61585] Updated weights for policy 1, policy_version 50480 (0.0009) [2023-10-14 19:43:35,669][61585] Updated weights for policy 1, policy_version 50490 (0.0007) [2023-10-14 19:43:38,098][61552] Updated weights for policy 0, policy_version 50662 (0.0008) [2023-10-14 19:43:38,343][60425] Fps is (10 sec: 13107.7, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 103579648. Throughput: 0: 1680.5, 1: 1670.7. Samples: 25905766. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:43:38,344][60425] Avg episode reward: [(0, '73.450'), (1, '71.600')] [2023-10-14 19:43:38,463][61552] Updated weights for policy 0, policy_version 50672 (0.0007) [2023-10-14 19:43:38,828][61552] Updated weights for policy 0, policy_version 50682 (0.0007) [2023-10-14 19:43:39,739][61585] Updated weights for policy 1, policy_version 50500 (0.0009) [2023-10-14 19:43:40,109][61585] Updated weights for policy 1, policy_version 50510 (0.0008) [2023-10-14 19:43:40,468][61585] Updated weights for policy 1, policy_version 50520 (0.0009) [2023-10-14 19:43:42,887][61552] Updated weights for policy 0, policy_version 50692 (0.0009) [2023-10-14 19:43:43,263][61552] Updated weights for policy 0, policy_version 50702 (0.0008) [2023-10-14 19:43:43,344][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 103645184. Throughput: 0: 1676.8, 1: 1671.5. Samples: 25926450. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:43:43,345][60425] Avg episode reward: [(0, '75.600'), (1, '69.710')] [2023-10-14 19:43:43,355][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000050528_51740672.pth... [2023-10-14 19:43:43,388][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000048960_50135040.pth [2023-10-14 19:43:43,625][61552] Updated weights for policy 0, policy_version 50712 (0.0007) [2023-10-14 19:43:43,914][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000050720_51937280.pth... [2023-10-14 19:43:43,943][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000049152_50331648.pth [2023-10-14 19:43:44,504][61585] Updated weights for policy 1, policy_version 50530 (0.0007) [2023-10-14 19:43:44,870][61585] Updated weights for policy 1, policy_version 50540 (0.0007) [2023-10-14 19:43:45,240][61585] Updated weights for policy 1, policy_version 50550 (0.0008) [2023-10-14 19:43:45,605][61585] Updated weights for policy 1, policy_version 50560 (0.0011) [2023-10-14 19:43:47,767][61552] Updated weights for policy 0, policy_version 50722 (0.0011) [2023-10-14 19:43:48,133][61552] Updated weights for policy 0, policy_version 50732 (0.0011) [2023-10-14 19:43:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 103710720. Throughput: 0: 1681.5, 1: 1651.1. Samples: 25935528. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:43:48,344][60425] Avg episode reward: [(0, '74.150'), (1, '72.360')] [2023-10-14 19:43:48,504][61552] Updated weights for policy 0, policy_version 50742 (0.0010) [2023-10-14 19:43:48,875][61552] Updated weights for policy 0, policy_version 50752 (0.0008) [2023-10-14 19:43:49,806][61585] Updated weights for policy 1, policy_version 50570 (0.0008) [2023-10-14 19:43:50,178][61585] Updated weights for policy 1, policy_version 50580 (0.0009) [2023-10-14 19:43:50,538][61585] Updated weights for policy 1, policy_version 50590 (0.0008) [2023-10-14 19:43:53,006][61552] Updated weights for policy 0, policy_version 50762 (0.0007) [2023-10-14 19:43:53,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 103776256. Throughput: 0: 1678.7, 1: 1661.1. Samples: 25955672. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:43:53,344][60425] Avg episode reward: [(0, '77.010'), (1, '69.850')] [2023-10-14 19:43:53,378][61552] Updated weights for policy 0, policy_version 50772 (0.0008) [2023-10-14 19:43:53,763][61552] Updated weights for policy 0, policy_version 50782 (0.0010) [2023-10-14 19:43:54,649][61585] Updated weights for policy 1, policy_version 50600 (0.0008) [2023-10-14 19:43:55,027][61585] Updated weights for policy 1, policy_version 50610 (0.0008) [2023-10-14 19:43:55,385][61585] Updated weights for policy 1, policy_version 50620 (0.0010) [2023-10-14 19:43:57,916][61552] Updated weights for policy 0, policy_version 50792 (0.0010) [2023-10-14 19:43:58,286][61552] Updated weights for policy 0, policy_version 50802 (0.0007) [2023-10-14 19:43:58,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 103841792. Throughput: 0: 1675.3, 1: 1667.2. Samples: 25976242. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:43:58,344][60425] Avg episode reward: [(0, '77.220'), (1, '71.510')] [2023-10-14 19:43:58,650][61552] Updated weights for policy 0, policy_version 50812 (0.0007) [2023-10-14 19:43:59,417][61585] Updated weights for policy 1, policy_version 50630 (0.0009) [2023-10-14 19:43:59,786][61585] Updated weights for policy 1, policy_version 50640 (0.0008) [2023-10-14 19:44:00,149][61585] Updated weights for policy 1, policy_version 50650 (0.0011) [2023-10-14 19:44:02,551][61552] Updated weights for policy 0, policy_version 50822 (0.0008) [2023-10-14 19:44:02,922][61552] Updated weights for policy 0, policy_version 50832 (0.0008) [2023-10-14 19:44:03,291][61552] Updated weights for policy 0, policy_version 50842 (0.0009) [2023-10-14 19:44:03,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 103907328. Throughput: 0: 1678.8, 1: 1652.8. Samples: 25985410. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:44:03,344][60425] Avg episode reward: [(0, '77.330'), (1, '70.120')] [2023-10-14 19:44:04,213][61585] Updated weights for policy 1, policy_version 50660 (0.0008) [2023-10-14 19:44:04,574][61585] Updated weights for policy 1, policy_version 50670 (0.0007) [2023-10-14 19:44:04,944][61585] Updated weights for policy 1, policy_version 50680 (0.0009) [2023-10-14 19:44:07,459][61552] Updated weights for policy 0, policy_version 50852 (0.0008) [2023-10-14 19:44:07,846][61552] Updated weights for policy 0, policy_version 50862 (0.0007) [2023-10-14 19:44:08,222][61552] Updated weights for policy 0, policy_version 50872 (0.0009) [2023-10-14 19:44:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 103972864. Throughput: 0: 1677.3, 1: 1664.8. Samples: 26005908. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:44:08,344][60425] Avg episode reward: [(0, '72.930'), (1, '75.490')] [2023-10-14 19:44:08,987][61585] Updated weights for policy 1, policy_version 50690 (0.0011) [2023-10-14 19:44:09,358][61585] Updated weights for policy 1, policy_version 50700 (0.0008) [2023-10-14 19:44:09,730][61585] Updated weights for policy 1, policy_version 50710 (0.0007) [2023-10-14 19:44:10,096][61585] Updated weights for policy 1, policy_version 50720 (0.0008) [2023-10-14 19:44:12,390][61552] Updated weights for policy 0, policy_version 50882 (0.0009) [2023-10-14 19:44:12,763][61552] Updated weights for policy 0, policy_version 50892 (0.0007) [2023-10-14 19:44:13,129][61552] Updated weights for policy 0, policy_version 50902 (0.0007) [2023-10-14 19:44:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 104038400. Throughput: 0: 1658.3, 1: 1667.7. Samples: 26025906. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:44:13,344][60425] Avg episode reward: [(0, '75.780'), (1, '71.670')] [2023-10-14 19:44:13,500][61552] Updated weights for policy 0, policy_version 50912 (0.0007) [2023-10-14 19:44:14,110][61585] Updated weights for policy 1, policy_version 50730 (0.0008) [2023-10-14 19:44:14,472][61585] Updated weights for policy 1, policy_version 50740 (0.0008) [2023-10-14 19:44:14,840][61585] Updated weights for policy 1, policy_version 50750 (0.0010) [2023-10-14 19:44:17,486][61552] Updated weights for policy 0, policy_version 50922 (0.0007) [2023-10-14 19:44:17,850][61552] Updated weights for policy 0, policy_version 50932 (0.0007) [2023-10-14 19:44:18,224][61552] Updated weights for policy 0, policy_version 50942 (0.0007) [2023-10-14 19:44:18,343][60425] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 104136704. Throughput: 0: 1665.2, 1: 1665.1. Samples: 26035492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:44:18,344][60425] Avg episode reward: [(0, '72.610'), (1, '74.140')] [2023-10-14 19:44:18,952][61585] Updated weights for policy 1, policy_version 50760 (0.0007) [2023-10-14 19:44:19,318][61585] Updated weights for policy 1, policy_version 50770 (0.0007) [2023-10-14 19:44:19,683][61585] Updated weights for policy 1, policy_version 50780 (0.0008) [2023-10-14 19:44:22,388][61552] Updated weights for policy 0, policy_version 50952 (0.0008) [2023-10-14 19:44:22,766][61552] Updated weights for policy 0, policy_version 50962 (0.0007) [2023-10-14 19:44:23,127][61552] Updated weights for policy 0, policy_version 50972 (0.0008) [2023-10-14 19:44:23,343][60425] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 104202240. Throughput: 0: 1665.9, 1: 1673.1. Samples: 26056024. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:44:23,344][60425] Avg episode reward: [(0, '76.440'), (1, '74.190')] [2023-10-14 19:44:23,771][61585] Updated weights for policy 1, policy_version 50790 (0.0008) [2023-10-14 19:44:24,126][61585] Updated weights for policy 1, policy_version 50800 (0.0008) [2023-10-14 19:44:24,488][61585] Updated weights for policy 1, policy_version 50810 (0.0008) [2023-10-14 19:44:27,254][61552] Updated weights for policy 0, policy_version 50982 (0.0007) [2023-10-14 19:44:27,619][61552] Updated weights for policy 0, policy_version 50992 (0.0007) [2023-10-14 19:44:27,987][61552] Updated weights for policy 0, policy_version 51002 (0.0010) [2023-10-14 19:44:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 104267776. Throughput: 0: 1648.5, 1: 1671.3. Samples: 26075836. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:44:28,344][60425] Avg episode reward: [(0, '73.900'), (1, '68.010')] [2023-10-14 19:44:28,759][61585] Updated weights for policy 1, policy_version 50820 (0.0008) [2023-10-14 19:44:29,130][61585] Updated weights for policy 1, policy_version 50830 (0.0009) [2023-10-14 19:44:29,506][61585] Updated weights for policy 1, policy_version 50840 (0.0009) [2023-10-14 19:44:32,042][61552] Updated weights for policy 0, policy_version 51012 (0.0010) [2023-10-14 19:44:32,405][61552] Updated weights for policy 0, policy_version 51022 (0.0010) [2023-10-14 19:44:32,780][61552] Updated weights for policy 0, policy_version 51032 (0.0009) [2023-10-14 19:44:33,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 104333312. Throughput: 0: 1666.4, 1: 1670.9. Samples: 26085706. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:44:33,344][60425] Avg episode reward: [(0, '72.260'), (1, '72.980')] [2023-10-14 19:44:33,604][61585] Updated weights for policy 1, policy_version 50850 (0.0007) [2023-10-14 19:44:33,970][61585] Updated weights for policy 1, policy_version 50860 (0.0008) [2023-10-14 19:44:34,338][61585] Updated weights for policy 1, policy_version 50870 (0.0008) [2023-10-14 19:44:34,704][61585] Updated weights for policy 1, policy_version 50880 (0.0009) [2023-10-14 19:44:36,903][61552] Updated weights for policy 0, policy_version 51042 (0.0007) [2023-10-14 19:44:37,275][61552] Updated weights for policy 0, policy_version 51052 (0.0007) [2023-10-14 19:44:37,645][61552] Updated weights for policy 0, policy_version 51062 (0.0007) [2023-10-14 19:44:38,024][61552] Updated weights for policy 0, policy_version 51072 (0.0007) [2023-10-14 19:44:38,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 104398848. Throughput: 0: 1670.7, 1: 1678.1. Samples: 26106366. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-14 19:44:38,344][60425] Avg episode reward: [(0, '69.260'), (1, '73.640')] [2023-10-14 19:44:38,873][61585] Updated weights for policy 1, policy_version 50890 (0.0008) [2023-10-14 19:44:39,238][61585] Updated weights for policy 1, policy_version 50900 (0.0008) [2023-10-14 19:44:39,603][61585] Updated weights for policy 1, policy_version 50910 (0.0007) [2023-10-14 19:44:42,213][61552] Updated weights for policy 0, policy_version 51082 (0.0007) [2023-10-14 19:44:42,574][61552] Updated weights for policy 0, policy_version 51092 (0.0007) [2023-10-14 19:44:42,942][61552] Updated weights for policy 0, policy_version 51102 (0.0008) [2023-10-14 19:44:43,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 104464384. Throughput: 0: 1648.3, 1: 1678.3. Samples: 26125938. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-14 19:44:43,344][60425] Avg episode reward: [(0, '75.430'), (1, '71.810')] [2023-10-14 19:44:43,712][61585] Updated weights for policy 1, policy_version 50920 (0.0008) [2023-10-14 19:44:44,092][61585] Updated weights for policy 1, policy_version 50930 (0.0008) [2023-10-14 19:44:44,462][61585] Updated weights for policy 1, policy_version 50940 (0.0007) [2023-10-14 19:44:47,137][61552] Updated weights for policy 0, policy_version 51112 (0.0008) [2023-10-14 19:44:47,500][61552] Updated weights for policy 0, policy_version 51122 (0.0007) [2023-10-14 19:44:47,876][61552] Updated weights for policy 0, policy_version 51132 (0.0008) [2023-10-14 19:44:48,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 104529920. Throughput: 0: 1665.2, 1: 1677.0. Samples: 26135808. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-14 19:44:48,344][60425] Avg episode reward: [(0, '74.380'), (1, '71.190')] [2023-10-14 19:44:48,628][61585] Updated weights for policy 1, policy_version 50950 (0.0010) [2023-10-14 19:44:48,991][61585] Updated weights for policy 1, policy_version 50960 (0.0007) [2023-10-14 19:44:49,357][61585] Updated weights for policy 1, policy_version 50970 (0.0008) [2023-10-14 19:44:51,867][61552] Updated weights for policy 0, policy_version 51142 (0.0007) [2023-10-14 19:44:52,230][61552] Updated weights for policy 0, policy_version 51152 (0.0007) [2023-10-14 19:44:52,602][61552] Updated weights for policy 0, policy_version 51162 (0.0007) [2023-10-14 19:44:53,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 104595456. Throughput: 0: 1665.5, 1: 1674.8. Samples: 26156224. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-14 19:44:53,344][60425] Avg episode reward: [(0, '73.230'), (1, '72.000')] [2023-10-14 19:44:53,482][61585] Updated weights for policy 1, policy_version 50980 (0.0010) [2023-10-14 19:44:53,854][61585] Updated weights for policy 1, policy_version 50990 (0.0008) [2023-10-14 19:44:54,221][61585] Updated weights for policy 1, policy_version 51000 (0.0007) [2023-10-14 19:44:56,759][61552] Updated weights for policy 0, policy_version 51172 (0.0009) [2023-10-14 19:44:57,159][61552] Updated weights for policy 0, policy_version 51182 (0.0009) [2023-10-14 19:44:57,519][61552] Updated weights for policy 0, policy_version 51192 (0.0008) [2023-10-14 19:44:58,242][61585] Updated weights for policy 1, policy_version 51010 (0.0008) [2023-10-14 19:44:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 104660992. Throughput: 0: 1651.9, 1: 1679.2. Samples: 26175806. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-14 19:44:58,344][60425] Avg episode reward: [(0, '76.110'), (1, '73.790')] [2023-10-14 19:44:58,622][61585] Updated weights for policy 1, policy_version 51020 (0.0007) [2023-10-14 19:44:58,984][61585] Updated weights for policy 1, policy_version 51030 (0.0007) [2023-10-14 19:44:59,348][61585] Updated weights for policy 1, policy_version 51040 (0.0008) [2023-10-14 19:45:01,520][61552] Updated weights for policy 0, policy_version 51202 (0.0007) [2023-10-14 19:45:01,895][61552] Updated weights for policy 0, policy_version 51212 (0.0007) [2023-10-14 19:45:02,258][61552] Updated weights for policy 0, policy_version 51222 (0.0007) [2023-10-14 19:45:02,619][61552] Updated weights for policy 0, policy_version 51232 (0.0007) [2023-10-14 19:45:03,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 104726528. Throughput: 0: 1670.9, 1: 1676.5. Samples: 26186124. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-14 19:45:03,344][60425] Avg episode reward: [(0, '69.880'), (1, '75.460')] [2023-10-14 19:45:03,345][61585] Updated weights for policy 1, policy_version 51050 (0.0008) [2023-10-14 19:45:03,714][61585] Updated weights for policy 1, policy_version 51060 (0.0009) [2023-10-14 19:45:04,085][61585] Updated weights for policy 1, policy_version 51070 (0.0011) [2023-10-14 19:45:06,759][61552] Updated weights for policy 0, policy_version 51242 (0.0009) [2023-10-14 19:45:07,139][61552] Updated weights for policy 0, policy_version 51252 (0.0009) [2023-10-14 19:45:07,509][61552] Updated weights for policy 0, policy_version 51262 (0.0008) [2023-10-14 19:45:08,177][61585] Updated weights for policy 1, policy_version 51080 (0.0007) [2023-10-14 19:45:08,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 104792064. Throughput: 0: 1668.8, 1: 1674.8. Samples: 26206482. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-14 19:45:08,344][60425] Avg episode reward: [(0, '73.790'), (1, '77.380')] [2023-10-14 19:45:08,553][61585] Updated weights for policy 1, policy_version 51090 (0.0008) [2023-10-14 19:45:08,921][61585] Updated weights for policy 1, policy_version 51100 (0.0011) [2023-10-14 19:45:11,545][61552] Updated weights for policy 0, policy_version 51272 (0.0010) [2023-10-14 19:45:11,916][61552] Updated weights for policy 0, policy_version 51282 (0.0009) [2023-10-14 19:45:12,270][61552] Updated weights for policy 0, policy_version 51292 (0.0008) [2023-10-14 19:45:12,914][61585] Updated weights for policy 1, policy_version 51110 (0.0010) [2023-10-14 19:45:13,277][61585] Updated weights for policy 1, policy_version 51120 (0.0010) [2023-10-14 19:45:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 104857600. Throughput: 0: 1666.6, 1: 1670.4. Samples: 26226000. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:45:13,344][60425] Avg episode reward: [(0, '75.980'), (1, '72.700')] [2023-10-14 19:45:13,647][61585] Updated weights for policy 1, policy_version 51130 (0.0010) [2023-10-14 19:45:16,289][61552] Updated weights for policy 0, policy_version 51302 (0.0011) [2023-10-14 19:45:16,649][61552] Updated weights for policy 0, policy_version 51312 (0.0010) [2023-10-14 19:45:17,020][61552] Updated weights for policy 0, policy_version 51322 (0.0008) [2023-10-14 19:45:17,847][61585] Updated weights for policy 1, policy_version 51140 (0.0010) [2023-10-14 19:45:18,203][61585] Updated weights for policy 1, policy_version 51150 (0.0010) [2023-10-14 19:45:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 104923136. Throughput: 0: 1676.4, 1: 1669.7. Samples: 26236284. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:45:18,344][60425] Avg episode reward: [(0, '76.130'), (1, '72.330')] [2023-10-14 19:45:18,567][61585] Updated weights for policy 1, policy_version 51160 (0.0011) [2023-10-14 19:45:21,102][61552] Updated weights for policy 0, policy_version 51332 (0.0009) [2023-10-14 19:45:21,475][61552] Updated weights for policy 0, policy_version 51342 (0.0007) [2023-10-14 19:45:21,851][61552] Updated weights for policy 0, policy_version 51352 (0.0008) [2023-10-14 19:45:22,677][61585] Updated weights for policy 1, policy_version 51170 (0.0011) [2023-10-14 19:45:23,046][61585] Updated weights for policy 1, policy_version 51180 (0.0007) [2023-10-14 19:45:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 104988672. Throughput: 0: 1660.9, 1: 1666.1. Samples: 26256076. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:45:23,344][60425] Avg episode reward: [(0, '76.480'), (1, '74.530')] [2023-10-14 19:45:23,410][61585] Updated weights for policy 1, policy_version 51190 (0.0009) [2023-10-14 19:45:23,768][61585] Updated weights for policy 1, policy_version 51200 (0.0008) [2023-10-14 19:45:25,958][61552] Updated weights for policy 0, policy_version 51362 (0.0007) [2023-10-14 19:45:26,337][61552] Updated weights for policy 0, policy_version 51372 (0.0008) [2023-10-14 19:45:26,695][61552] Updated weights for policy 0, policy_version 51382 (0.0009) [2023-10-14 19:45:27,067][61552] Updated weights for policy 0, policy_version 51392 (0.0007) [2023-10-14 19:45:27,976][61585] Updated weights for policy 1, policy_version 51210 (0.0009) [2023-10-14 19:45:28,339][61585] Updated weights for policy 1, policy_version 51220 (0.0007) [2023-10-14 19:45:28,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 105054208. Throughput: 0: 1673.1, 1: 1658.8. Samples: 26275874. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:45:28,344][60425] Avg episode reward: [(0, '76.560'), (1, '70.750')] [2023-10-14 19:45:28,716][61585] Updated weights for policy 1, policy_version 51230 (0.0007) [2023-10-14 19:45:31,041][61552] Updated weights for policy 0, policy_version 51402 (0.0008) [2023-10-14 19:45:31,403][61552] Updated weights for policy 0, policy_version 51412 (0.0008) [2023-10-14 19:45:31,769][61552] Updated weights for policy 0, policy_version 51422 (0.0008) [2023-10-14 19:45:32,887][61585] Updated weights for policy 1, policy_version 51240 (0.0010) [2023-10-14 19:45:33,261][61585] Updated weights for policy 1, policy_version 51250 (0.0010) [2023-10-14 19:45:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 105119744. Throughput: 0: 1684.6, 1: 1663.1. Samples: 26286454. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:45:33,344][60425] Avg episode reward: [(0, '72.090'), (1, '73.860')] [2023-10-14 19:45:33,626][61585] Updated weights for policy 1, policy_version 51260 (0.0008) [2023-10-14 19:45:35,817][61552] Updated weights for policy 0, policy_version 51432 (0.0010) [2023-10-14 19:45:36,199][61552] Updated weights for policy 0, policy_version 51442 (0.0011) [2023-10-14 19:45:36,567][61552] Updated weights for policy 0, policy_version 51452 (0.0010) [2023-10-14 19:45:37,789][61585] Updated weights for policy 1, policy_version 51270 (0.0008) [2023-10-14 19:45:38,153][61585] Updated weights for policy 1, policy_version 51280 (0.0009) [2023-10-14 19:45:38,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 105185280. Throughput: 0: 1663.7, 1: 1665.1. Samples: 26306016. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:45:38,344][60425] Avg episode reward: [(0, '77.190'), (1, '68.160')] [2023-10-14 19:45:38,518][61585] Updated weights for policy 1, policy_version 51290 (0.0007) [2023-10-14 19:45:40,649][61552] Updated weights for policy 0, policy_version 51462 (0.0009) [2023-10-14 19:45:41,013][61552] Updated weights for policy 0, policy_version 51472 (0.0008) [2023-10-14 19:45:41,382][61552] Updated weights for policy 0, policy_version 51482 (0.0008) [2023-10-14 19:45:42,622][61585] Updated weights for policy 1, policy_version 51300 (0.0008) [2023-10-14 19:45:42,990][61585] Updated weights for policy 1, policy_version 51310 (0.0008) [2023-10-14 19:45:43,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 105250816. Throughput: 0: 1692.2, 1: 1656.2. Samples: 26326482. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:45:43,344][60425] Avg episode reward: [(0, '79.520'), (1, '69.570')] [2023-10-14 19:45:43,350][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000051488_52723712.pth... [2023-10-14 19:45:43,364][61585] Updated weights for policy 1, policy_version 51320 (0.0010) [2023-10-14 19:45:43,384][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000049920_51118080.pth [2023-10-14 19:45:43,387][61172] Saving new best policy, reward=79.520! [2023-10-14 19:45:43,653][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000051328_52559872.pth... [2023-10-14 19:45:43,692][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000049760_50954240.pth [2023-10-14 19:45:45,576][61552] Updated weights for policy 0, policy_version 51492 (0.0008) [2023-10-14 19:45:45,965][61552] Updated weights for policy 0, policy_version 51502 (0.0009) [2023-10-14 19:45:46,339][61552] Updated weights for policy 0, policy_version 51512 (0.0009) [2023-10-14 19:45:47,286][61585] Updated weights for policy 1, policy_version 51330 (0.0011) [2023-10-14 19:45:47,661][61585] Updated weights for policy 1, policy_version 51340 (0.0009) [2023-10-14 19:45:48,022][61585] Updated weights for policy 1, policy_version 51350 (0.0010) [2023-10-14 19:45:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 105316352. Throughput: 0: 1682.3, 1: 1663.4. Samples: 26336682. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-14 19:45:48,344][60425] Avg episode reward: [(0, '77.460'), (1, '68.940')] [2023-10-14 19:45:48,382][61585] Updated weights for policy 1, policy_version 51360 (0.0010) [2023-10-14 19:45:50,463][61552] Updated weights for policy 0, policy_version 51522 (0.0008) [2023-10-14 19:45:50,831][61552] Updated weights for policy 0, policy_version 51532 (0.0008) [2023-10-14 19:45:51,197][61552] Updated weights for policy 0, policy_version 51542 (0.0007) [2023-10-14 19:45:51,563][61552] Updated weights for policy 0, policy_version 51552 (0.0008) [2023-10-14 19:45:52,549][61585] Updated weights for policy 1, policy_version 51370 (0.0009) [2023-10-14 19:45:52,913][61585] Updated weights for policy 1, policy_version 51380 (0.0008) [2023-10-14 19:45:53,284][61585] Updated weights for policy 1, policy_version 51390 (0.0007) [2023-10-14 19:45:53,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 105381888. Throughput: 0: 1661.0, 1: 1664.2. Samples: 26356114. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-14 19:45:53,344][60425] Avg episode reward: [(0, '74.920'), (1, '73.900')] [2023-10-14 19:45:55,471][61552] Updated weights for policy 0, policy_version 51562 (0.0011) [2023-10-14 19:45:55,845][61552] Updated weights for policy 0, policy_version 51572 (0.0007) [2023-10-14 19:45:56,215][61552] Updated weights for policy 0, policy_version 51582 (0.0007) [2023-10-14 19:45:57,466][61585] Updated weights for policy 1, policy_version 51400 (0.0008) [2023-10-14 19:45:57,827][61585] Updated weights for policy 1, policy_version 51410 (0.0007) [2023-10-14 19:45:58,195][61585] Updated weights for policy 1, policy_version 51420 (0.0007) [2023-10-14 19:45:58,343][60425] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 105480192. Throughput: 0: 1688.3, 1: 1655.5. Samples: 26376474. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-14 19:45:58,345][60425] Avg episode reward: [(0, '77.970'), (1, '75.940')] [2023-10-14 19:46:00,220][61552] Updated weights for policy 0, policy_version 51592 (0.0010) [2023-10-14 19:46:00,579][61552] Updated weights for policy 0, policy_version 51602 (0.0010) [2023-10-14 19:46:00,960][61552] Updated weights for policy 0, policy_version 51612 (0.0010) [2023-10-14 19:46:02,279][61585] Updated weights for policy 1, policy_version 51430 (0.0009) [2023-10-14 19:46:02,640][61585] Updated weights for policy 1, policy_version 51440 (0.0008) [2023-10-14 19:46:03,009][61585] Updated weights for policy 1, policy_version 51450 (0.0007) [2023-10-14 19:46:03,343][60425] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 105545728. Throughput: 0: 1669.9, 1: 1672.3. Samples: 26386682. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-14 19:46:03,345][60425] Avg episode reward: [(0, '74.300'), (1, '74.980')] [2023-10-14 19:46:05,038][61552] Updated weights for policy 0, policy_version 51622 (0.0009) [2023-10-14 19:46:05,409][61552] Updated weights for policy 0, policy_version 51632 (0.0008) [2023-10-14 19:46:05,776][61552] Updated weights for policy 0, policy_version 51642 (0.0009) [2023-10-14 19:46:07,217][61585] Updated weights for policy 1, policy_version 51460 (0.0007) [2023-10-14 19:46:07,588][61585] Updated weights for policy 1, policy_version 51470 (0.0007) [2023-10-14 19:46:07,950][61585] Updated weights for policy 1, policy_version 51480 (0.0008) [2023-10-14 19:46:08,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 105611264. Throughput: 0: 1675.5, 1: 1674.6. Samples: 26406828. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-14 19:46:08,344][60425] Avg episode reward: [(0, '74.100'), (1, '77.980')] [2023-10-14 19:46:09,788][61552] Updated weights for policy 0, policy_version 51652 (0.0010) [2023-10-14 19:46:10,165][61552] Updated weights for policy 0, policy_version 51662 (0.0011) [2023-10-14 19:46:10,525][61552] Updated weights for policy 0, policy_version 51672 (0.0010) [2023-10-14 19:46:11,972][61585] Updated weights for policy 1, policy_version 51490 (0.0009) [2023-10-14 19:46:12,342][61585] Updated weights for policy 1, policy_version 51500 (0.0008) [2023-10-14 19:46:12,711][61585] Updated weights for policy 1, policy_version 51510 (0.0008) [2023-10-14 19:46:13,087][61585] Updated weights for policy 1, policy_version 51520 (0.0008) [2023-10-14 19:46:13,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 105676800. Throughput: 0: 1689.3, 1: 1665.3. Samples: 26426832. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-14 19:46:13,344][60425] Avg episode reward: [(0, '76.690'), (1, '72.590')] [2023-10-14 19:46:14,643][61552] Updated weights for policy 0, policy_version 51682 (0.0009) [2023-10-14 19:46:15,003][61552] Updated weights for policy 0, policy_version 51692 (0.0007) [2023-10-14 19:46:15,376][61552] Updated weights for policy 0, policy_version 51702 (0.0009) [2023-10-14 19:46:15,749][61552] Updated weights for policy 0, policy_version 51712 (0.0008) [2023-10-14 19:46:17,215][61585] Updated weights for policy 1, policy_version 51530 (0.0009) [2023-10-14 19:46:17,589][61585] Updated weights for policy 1, policy_version 51540 (0.0008) [2023-10-14 19:46:17,947][61585] Updated weights for policy 1, policy_version 51550 (0.0007) [2023-10-14 19:46:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 105742336. Throughput: 0: 1663.0, 1: 1682.7. Samples: 26437012. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-14 19:46:18,344][60425] Avg episode reward: [(0, '76.750'), (1, '75.990')] [2023-10-14 19:46:19,751][61552] Updated weights for policy 0, policy_version 51722 (0.0009) [2023-10-14 19:46:20,118][61552] Updated weights for policy 0, policy_version 51732 (0.0011) [2023-10-14 19:46:20,491][61552] Updated weights for policy 0, policy_version 51742 (0.0010) [2023-10-14 19:46:22,045][61585] Updated weights for policy 1, policy_version 51560 (0.0008) [2023-10-14 19:46:22,427][61585] Updated weights for policy 1, policy_version 51570 (0.0007) [2023-10-14 19:46:22,791][61585] Updated weights for policy 1, policy_version 51580 (0.0008) [2023-10-14 19:46:23,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 105807872. Throughput: 0: 1681.4, 1: 1682.0. Samples: 26457366. Policy #0 lag: (min: 1.0, avg: 12.8, max: 33.0) [2023-10-14 19:46:23,344][60425] Avg episode reward: [(0, '74.250'), (1, '77.290')] [2023-10-14 19:46:24,630][61552] Updated weights for policy 0, policy_version 51752 (0.0010) [2023-10-14 19:46:25,004][61552] Updated weights for policy 0, policy_version 51762 (0.0009) [2023-10-14 19:46:25,376][61552] Updated weights for policy 0, policy_version 51772 (0.0008) [2023-10-14 19:46:26,565][61585] Updated weights for policy 1, policy_version 51590 (0.0010) [2023-10-14 19:46:26,927][61585] Updated weights for policy 1, policy_version 51600 (0.0010) [2023-10-14 19:46:27,288][61585] Updated weights for policy 1, policy_version 51610 (0.0007) [2023-10-14 19:46:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 105873408. Throughput: 0: 1684.5, 1: 1661.0. Samples: 26477030. Policy #0 lag: (min: 1.0, avg: 12.8, max: 33.0) [2023-10-14 19:46:28,344][60425] Avg episode reward: [(0, '75.240'), (1, '75.560')] [2023-10-14 19:46:29,420][61552] Updated weights for policy 0, policy_version 51782 (0.0009) [2023-10-14 19:46:29,792][61552] Updated weights for policy 0, policy_version 51792 (0.0009) [2023-10-14 19:46:30,161][61552] Updated weights for policy 0, policy_version 51802 (0.0008) [2023-10-14 19:46:31,357][61585] Updated weights for policy 1, policy_version 51620 (0.0010) [2023-10-14 19:46:31,724][61585] Updated weights for policy 1, policy_version 51630 (0.0010) [2023-10-14 19:46:32,086][61585] Updated weights for policy 1, policy_version 51640 (0.0009) [2023-10-14 19:46:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 105938944. Throughput: 0: 1667.4, 1: 1683.0. Samples: 26487452. Policy #0 lag: (min: 1.0, avg: 12.8, max: 33.0) [2023-10-14 19:46:33,344][60425] Avg episode reward: [(0, '69.040'), (1, '73.210')] [2023-10-14 19:46:34,306][61552] Updated weights for policy 0, policy_version 51812 (0.0009) [2023-10-14 19:46:34,699][61552] Updated weights for policy 0, policy_version 51822 (0.0008) [2023-10-14 19:46:35,061][61552] Updated weights for policy 0, policy_version 51832 (0.0010) [2023-10-14 19:46:36,100][61585] Updated weights for policy 1, policy_version 51650 (0.0009) [2023-10-14 19:46:36,462][61585] Updated weights for policy 1, policy_version 51660 (0.0008) [2023-10-14 19:46:36,827][61585] Updated weights for policy 1, policy_version 51670 (0.0010) [2023-10-14 19:46:37,193][61585] Updated weights for policy 1, policy_version 51680 (0.0009) [2023-10-14 19:46:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 106004480. Throughput: 0: 1689.6, 1: 1672.8. Samples: 26507420. Policy #0 lag: (min: 1.0, avg: 12.8, max: 33.0) [2023-10-14 19:46:38,344][60425] Avg episode reward: [(0, '71.740'), (1, '72.080')] [2023-10-14 19:46:39,246][61552] Updated weights for policy 0, policy_version 51842 (0.0010) [2023-10-14 19:46:39,614][61552] Updated weights for policy 0, policy_version 51852 (0.0008) [2023-10-14 19:46:39,979][61552] Updated weights for policy 0, policy_version 51862 (0.0008) [2023-10-14 19:46:40,351][61552] Updated weights for policy 0, policy_version 51872 (0.0007) [2023-10-14 19:46:41,260][61585] Updated weights for policy 1, policy_version 51690 (0.0009) [2023-10-14 19:46:41,623][61585] Updated weights for policy 1, policy_version 51700 (0.0007) [2023-10-14 19:46:41,990][61585] Updated weights for policy 1, policy_version 51710 (0.0007) [2023-10-14 19:46:43,344][60425] Fps is (10 sec: 13106.5, 60 sec: 13653.2, 300 sec: 13329.3). Total num frames: 106070016. Throughput: 0: 1683.6, 1: 1673.1. Samples: 26527530. Policy #0 lag: (min: 1.0, avg: 12.8, max: 33.0) [2023-10-14 19:46:43,345][60425] Avg episode reward: [(0, '71.780'), (1, '70.710')] [2023-10-14 19:46:44,392][61552] Updated weights for policy 0, policy_version 51882 (0.0009) [2023-10-14 19:46:44,765][61552] Updated weights for policy 0, policy_version 51892 (0.0007) [2023-10-14 19:46:45,137][61552] Updated weights for policy 0, policy_version 51902 (0.0007) [2023-10-14 19:46:46,046][61585] Updated weights for policy 1, policy_version 51720 (0.0009) [2023-10-14 19:46:46,419][61585] Updated weights for policy 1, policy_version 51730 (0.0009) [2023-10-14 19:46:46,782][61585] Updated weights for policy 1, policy_version 51740 (0.0011) [2023-10-14 19:46:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 106135552. Throughput: 0: 1674.9, 1: 1685.7. Samples: 26537910. Policy #0 lag: (min: 1.0, avg: 12.8, max: 33.0) [2023-10-14 19:46:48,344][60425] Avg episode reward: [(0, '71.070'), (1, '72.930')] [2023-10-14 19:46:49,049][61552] Updated weights for policy 0, policy_version 51912 (0.0007) [2023-10-14 19:46:49,412][61552] Updated weights for policy 0, policy_version 51922 (0.0009) [2023-10-14 19:46:49,785][61552] Updated weights for policy 0, policy_version 51932 (0.0008) [2023-10-14 19:46:50,909][61585] Updated weights for policy 1, policy_version 51750 (0.0010) [2023-10-14 19:46:51,278][61585] Updated weights for policy 1, policy_version 51760 (0.0010) [2023-10-14 19:46:51,650][61585] Updated weights for policy 1, policy_version 51770 (0.0008) [2023-10-14 19:46:53,343][60425] Fps is (10 sec: 13107.7, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 106201088. Throughput: 0: 1687.8, 1: 1661.3. Samples: 26557540. Policy #0 lag: (min: 1.0, avg: 12.8, max: 33.0) [2023-10-14 19:46:53,344][60425] Avg episode reward: [(0, '71.610'), (1, '77.880')] [2023-10-14 19:46:53,866][61552] Updated weights for policy 0, policy_version 51942 (0.0009) [2023-10-14 19:46:54,244][61552] Updated weights for policy 0, policy_version 51952 (0.0008) [2023-10-14 19:46:54,610][61552] Updated weights for policy 0, policy_version 51962 (0.0008) [2023-10-14 19:46:55,724][61585] Updated weights for policy 1, policy_version 51780 (0.0008) [2023-10-14 19:46:56,087][61585] Updated weights for policy 1, policy_version 51790 (0.0009) [2023-10-14 19:46:56,446][61585] Updated weights for policy 1, policy_version 51800 (0.0009) [2023-10-14 19:46:58,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 106266624. Throughput: 0: 1683.5, 1: 1675.0. Samples: 26577964. Policy #0 lag: (min: 20.0, avg: 38.6, max: 40.0) [2023-10-14 19:46:58,344][60425] Avg episode reward: [(0, '73.060'), (1, '73.380')] [2023-10-14 19:46:58,698][61552] Updated weights for policy 0, policy_version 51972 (0.0008) [2023-10-14 19:46:59,071][61552] Updated weights for policy 0, policy_version 51982 (0.0009) [2023-10-14 19:46:59,435][61552] Updated weights for policy 0, policy_version 51992 (0.0008) [2023-10-14 19:47:00,630][61585] Updated weights for policy 1, policy_version 51810 (0.0009) [2023-10-14 19:47:01,006][61585] Updated weights for policy 1, policy_version 51820 (0.0009) [2023-10-14 19:47:01,374][61585] Updated weights for policy 1, policy_version 51830 (0.0009) [2023-10-14 19:47:01,733][61585] Updated weights for policy 1, policy_version 51840 (0.0009) [2023-10-14 19:47:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 106332160. Throughput: 0: 1675.2, 1: 1680.1. Samples: 26588000. Policy #0 lag: (min: 20.0, avg: 38.6, max: 40.0) [2023-10-14 19:47:03,344][60425] Avg episode reward: [(0, '69.900'), (1, '74.230')] [2023-10-14 19:47:03,487][61552] Updated weights for policy 0, policy_version 52002 (0.0008) [2023-10-14 19:47:03,851][61552] Updated weights for policy 0, policy_version 52012 (0.0008) [2023-10-14 19:47:04,216][61552] Updated weights for policy 0, policy_version 52022 (0.0008) [2023-10-14 19:47:04,593][61552] Updated weights for policy 0, policy_version 52032 (0.0009) [2023-10-14 19:47:05,835][61585] Updated weights for policy 1, policy_version 51850 (0.0008) [2023-10-14 19:47:06,191][61585] Updated weights for policy 1, policy_version 51860 (0.0008) [2023-10-14 19:47:06,566][61585] Updated weights for policy 1, policy_version 51870 (0.0008) [2023-10-14 19:47:08,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 106397696. Throughput: 0: 1685.0, 1: 1656.4. Samples: 26607732. Policy #0 lag: (min: 20.0, avg: 38.6, max: 40.0) [2023-10-14 19:47:08,344][60425] Avg episode reward: [(0, '70.790'), (1, '77.840')] [2023-10-14 19:47:08,661][61552] Updated weights for policy 0, policy_version 52042 (0.0010) [2023-10-14 19:47:09,035][61552] Updated weights for policy 0, policy_version 52052 (0.0009) [2023-10-14 19:47:09,400][61552] Updated weights for policy 0, policy_version 52062 (0.0007) [2023-10-14 19:47:10,796][61585] Updated weights for policy 1, policy_version 51880 (0.0007) [2023-10-14 19:47:11,165][61585] Updated weights for policy 1, policy_version 51890 (0.0010) [2023-10-14 19:47:11,527][61585] Updated weights for policy 1, policy_version 51900 (0.0009) [2023-10-14 19:47:13,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 106463232. Throughput: 0: 1680.0, 1: 1683.9. Samples: 26628406. Policy #0 lag: (min: 20.0, avg: 38.6, max: 40.0) [2023-10-14 19:47:13,344][60425] Avg episode reward: [(0, '72.960'), (1, '75.490')] [2023-10-14 19:47:13,474][61552] Updated weights for policy 0, policy_version 52072 (0.0008) [2023-10-14 19:47:13,831][61552] Updated weights for policy 0, policy_version 52082 (0.0007) [2023-10-14 19:47:14,202][61552] Updated weights for policy 0, policy_version 52092 (0.0007) [2023-10-14 19:47:15,647][61585] Updated weights for policy 1, policy_version 51910 (0.0008) [2023-10-14 19:47:16,016][61585] Updated weights for policy 1, policy_version 51920 (0.0009) [2023-10-14 19:47:16,394][61585] Updated weights for policy 1, policy_version 51930 (0.0009) [2023-10-14 19:47:18,316][61552] Updated weights for policy 0, policy_version 52102 (0.0008) [2023-10-14 19:47:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 106528768. Throughput: 0: 1677.5, 1: 1676.0. Samples: 26638358. Policy #0 lag: (min: 20.0, avg: 38.6, max: 40.0) [2023-10-14 19:47:18,344][60425] Avg episode reward: [(0, '72.510'), (1, '71.050')] [2023-10-14 19:47:18,684][61552] Updated weights for policy 0, policy_version 52112 (0.0007) [2023-10-14 19:47:19,055][61552] Updated weights for policy 0, policy_version 52122 (0.0007) [2023-10-14 19:47:20,384][61585] Updated weights for policy 1, policy_version 51940 (0.0009) [2023-10-14 19:47:20,745][61585] Updated weights for policy 1, policy_version 51950 (0.0009) [2023-10-14 19:47:21,116][61585] Updated weights for policy 1, policy_version 51960 (0.0008) [2023-10-14 19:47:23,305][61552] Updated weights for policy 0, policy_version 52132 (0.0007) [2023-10-14 19:47:23,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 106594304. Throughput: 0: 1676.8, 1: 1664.7. Samples: 26657784. Policy #0 lag: (min: 20.0, avg: 38.6, max: 40.0) [2023-10-14 19:47:23,344][60425] Avg episode reward: [(0, '72.940'), (1, '74.390')] [2023-10-14 19:47:23,703][61552] Updated weights for policy 0, policy_version 52142 (0.0007) [2023-10-14 19:47:24,067][61552] Updated weights for policy 0, policy_version 52152 (0.0009) [2023-10-14 19:47:25,244][61585] Updated weights for policy 1, policy_version 51970 (0.0009) [2023-10-14 19:47:25,620][61585] Updated weights for policy 1, policy_version 51980 (0.0009) [2023-10-14 19:47:25,974][61585] Updated weights for policy 1, policy_version 51990 (0.0009) [2023-10-14 19:47:26,339][61585] Updated weights for policy 1, policy_version 52000 (0.0009) [2023-10-14 19:47:27,911][61552] Updated weights for policy 0, policy_version 52162 (0.0007) [2023-10-14 19:47:28,289][61552] Updated weights for policy 0, policy_version 52172 (0.0007) [2023-10-14 19:47:28,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 106659840. Throughput: 0: 1681.4, 1: 1678.9. Samples: 26678746. Policy #0 lag: (min: 20.0, avg: 38.6, max: 40.0) [2023-10-14 19:47:28,345][60425] Avg episode reward: [(0, '77.500'), (1, '73.290')] [2023-10-14 19:47:28,656][61552] Updated weights for policy 0, policy_version 52182 (0.0007) [2023-10-14 19:47:29,019][61552] Updated weights for policy 0, policy_version 52192 (0.0008) [2023-10-14 19:47:30,421][61585] Updated weights for policy 1, policy_version 52010 (0.0007) [2023-10-14 19:47:30,785][61585] Updated weights for policy 1, policy_version 52020 (0.0008) [2023-10-14 19:47:31,155][61585] Updated weights for policy 1, policy_version 52030 (0.0008) [2023-10-14 19:47:33,043][61552] Updated weights for policy 0, policy_version 52202 (0.0008) [2023-10-14 19:47:33,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 106725376. Throughput: 0: 1681.9, 1: 1665.2. Samples: 26688532. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:47:33,344][60425] Avg episode reward: [(0, '79.420'), (1, '74.590')] [2023-10-14 19:47:33,401][61552] Updated weights for policy 0, policy_version 52212 (0.0007) [2023-10-14 19:47:33,776][61552] Updated weights for policy 0, policy_version 52222 (0.0009) [2023-10-14 19:47:35,463][61585] Updated weights for policy 1, policy_version 52040 (0.0007) [2023-10-14 19:47:35,826][61585] Updated weights for policy 1, policy_version 52050 (0.0009) [2023-10-14 19:47:36,184][61585] Updated weights for policy 1, policy_version 52060 (0.0008) [2023-10-14 19:47:37,929][61552] Updated weights for policy 0, policy_version 52232 (0.0008) [2023-10-14 19:47:38,302][61552] Updated weights for policy 0, policy_version 52242 (0.0007) [2023-10-14 19:47:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 106790912. Throughput: 0: 1679.1, 1: 1674.7. Samples: 26708458. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:47:38,344][60425] Avg episode reward: [(0, '78.230'), (1, '73.460')] [2023-10-14 19:47:38,680][61552] Updated weights for policy 0, policy_version 52252 (0.0009) [2023-10-14 19:47:40,217][61585] Updated weights for policy 1, policy_version 52070 (0.0008) [2023-10-14 19:47:40,585][61585] Updated weights for policy 1, policy_version 52080 (0.0008) [2023-10-14 19:47:40,951][61585] Updated weights for policy 1, policy_version 52090 (0.0009) [2023-10-14 19:47:42,657][61552] Updated weights for policy 0, policy_version 52262 (0.0008) [2023-10-14 19:47:43,030][61552] Updated weights for policy 0, policy_version 52272 (0.0007) [2023-10-14 19:47:43,344][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 106856448. Throughput: 0: 1676.1, 1: 1678.1. Samples: 26728902. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:47:43,345][60425] Avg episode reward: [(0, '76.190'), (1, '73.070')] [2023-10-14 19:47:43,356][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000052096_53346304.pth... [2023-10-14 19:47:43,392][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000050528_51740672.pth [2023-10-14 19:47:43,398][61552] Updated weights for policy 0, policy_version 52282 (0.0007) [2023-10-14 19:47:43,618][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000052288_53542912.pth... [2023-10-14 19:47:43,646][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000050720_51937280.pth [2023-10-14 19:47:44,927][61585] Updated weights for policy 1, policy_version 52100 (0.0008) [2023-10-14 19:47:45,292][61585] Updated weights for policy 1, policy_version 52110 (0.0010) [2023-10-14 19:47:45,655][61585] Updated weights for policy 1, policy_version 52120 (0.0010) [2023-10-14 19:47:47,622][61552] Updated weights for policy 0, policy_version 52292 (0.0008) [2023-10-14 19:47:47,996][61552] Updated weights for policy 0, policy_version 52302 (0.0009) [2023-10-14 19:47:48,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 106921984. Throughput: 0: 1685.2, 1: 1660.0. Samples: 26738538. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:47:48,344][60425] Avg episode reward: [(0, '74.580'), (1, '78.080')] [2023-10-14 19:47:48,379][61552] Updated weights for policy 0, policy_version 52312 (0.0008) [2023-10-14 19:47:49,790][61585] Updated weights for policy 1, policy_version 52130 (0.0011) [2023-10-14 19:47:50,153][61585] Updated weights for policy 1, policy_version 52140 (0.0008) [2023-10-14 19:47:50,522][61585] Updated weights for policy 1, policy_version 52150 (0.0007) [2023-10-14 19:47:50,878][61585] Updated weights for policy 1, policy_version 52160 (0.0010) [2023-10-14 19:47:52,591][61552] Updated weights for policy 0, policy_version 52322 (0.0009) [2023-10-14 19:47:52,965][61552] Updated weights for policy 0, policy_version 52332 (0.0008) [2023-10-14 19:47:53,326][61552] Updated weights for policy 0, policy_version 52342 (0.0009) [2023-10-14 19:47:53,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 106987520. Throughput: 0: 1679.2, 1: 1677.6. Samples: 26758786. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:47:53,344][60425] Avg episode reward: [(0, '81.200'), (1, '75.840')] [2023-10-14 19:47:53,688][61172] Saving new best policy, reward=81.200! [2023-10-14 19:47:53,695][61552] Updated weights for policy 0, policy_version 52352 (0.0009) [2023-10-14 19:47:54,923][61585] Updated weights for policy 1, policy_version 52170 (0.0007) [2023-10-14 19:47:55,287][61585] Updated weights for policy 1, policy_version 52180 (0.0008) [2023-10-14 19:47:55,657][61585] Updated weights for policy 1, policy_version 52190 (0.0007) [2023-10-14 19:47:57,842][61552] Updated weights for policy 0, policy_version 52362 (0.0008) [2023-10-14 19:47:58,202][61552] Updated weights for policy 0, policy_version 52372 (0.0009) [2023-10-14 19:47:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 107053056. Throughput: 0: 1674.4, 1: 1674.3. Samples: 26779096. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:47:58,344][60425] Avg episode reward: [(0, '79.270'), (1, '69.940')] [2023-10-14 19:47:58,574][61552] Updated weights for policy 0, policy_version 52382 (0.0009) [2023-10-14 19:47:59,675][61585] Updated weights for policy 1, policy_version 52200 (0.0009) [2023-10-14 19:48:00,059][61585] Updated weights for policy 1, policy_version 52210 (0.0007) [2023-10-14 19:48:00,428][61585] Updated weights for policy 1, policy_version 52220 (0.0007) [2023-10-14 19:48:02,622][61552] Updated weights for policy 0, policy_version 52392 (0.0008) [2023-10-14 19:48:02,991][61552] Updated weights for policy 0, policy_version 52402 (0.0007) [2023-10-14 19:48:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 107118592. Throughput: 0: 1680.4, 1: 1655.2. Samples: 26788460. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:48:03,344][60425] Avg episode reward: [(0, '76.630'), (1, '70.950')] [2023-10-14 19:48:03,357][61552] Updated weights for policy 0, policy_version 52412 (0.0009) [2023-10-14 19:48:04,621][61585] Updated weights for policy 1, policy_version 52230 (0.0008) [2023-10-14 19:48:04,982][61585] Updated weights for policy 1, policy_version 52240 (0.0009) [2023-10-14 19:48:05,351][61585] Updated weights for policy 1, policy_version 52250 (0.0007) [2023-10-14 19:48:07,435][61552] Updated weights for policy 0, policy_version 52422 (0.0008) [2023-10-14 19:48:07,811][61552] Updated weights for policy 0, policy_version 52432 (0.0010) [2023-10-14 19:48:08,182][61552] Updated weights for policy 0, policy_version 52442 (0.0010) [2023-10-14 19:48:08,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 107184128. Throughput: 0: 1682.7, 1: 1678.2. Samples: 26809024. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-14 19:48:08,344][60425] Avg episode reward: [(0, '79.200'), (1, '73.460')] [2023-10-14 19:48:09,357][61585] Updated weights for policy 1, policy_version 52260 (0.0007) [2023-10-14 19:48:09,719][61585] Updated weights for policy 1, policy_version 52270 (0.0010) [2023-10-14 19:48:10,080][61585] Updated weights for policy 1, policy_version 52280 (0.0007) [2023-10-14 19:48:12,228][61552] Updated weights for policy 0, policy_version 52452 (0.0009) [2023-10-14 19:48:12,603][61552] Updated weights for policy 0, policy_version 52462 (0.0008) [2023-10-14 19:48:12,960][61552] Updated weights for policy 0, policy_version 52472 (0.0008) [2023-10-14 19:48:13,343][60425] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 107282432. Throughput: 0: 1663.7, 1: 1674.7. Samples: 26828976. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-14 19:48:13,345][60425] Avg episode reward: [(0, '78.290'), (1, '68.040')] [2023-10-14 19:48:14,158][61585] Updated weights for policy 1, policy_version 52290 (0.0008) [2023-10-14 19:48:14,531][61585] Updated weights for policy 1, policy_version 52300 (0.0009) [2023-10-14 19:48:14,899][61585] Updated weights for policy 1, policy_version 52310 (0.0009) [2023-10-14 19:48:15,249][61585] Updated weights for policy 1, policy_version 52320 (0.0008) [2023-10-14 19:48:16,968][61552] Updated weights for policy 0, policy_version 52482 (0.0009) [2023-10-14 19:48:17,346][61552] Updated weights for policy 0, policy_version 52492 (0.0010) [2023-10-14 19:48:17,709][61552] Updated weights for policy 0, policy_version 52502 (0.0009) [2023-10-14 19:48:18,075][61552] Updated weights for policy 0, policy_version 52512 (0.0010) [2023-10-14 19:48:18,343][60425] Fps is (10 sec: 16384.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 107347968. Throughput: 0: 1674.7, 1: 1661.2. Samples: 26838646. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-14 19:48:18,344][60425] Avg episode reward: [(0, '76.190'), (1, '70.580')] [2023-10-14 19:48:19,334][61585] Updated weights for policy 1, policy_version 52330 (0.0010) [2023-10-14 19:48:19,691][61585] Updated weights for policy 1, policy_version 52340 (0.0009) [2023-10-14 19:48:20,055][61585] Updated weights for policy 1, policy_version 52350 (0.0009) [2023-10-14 19:48:22,186][61552] Updated weights for policy 0, policy_version 52522 (0.0010) [2023-10-14 19:48:22,551][61552] Updated weights for policy 0, policy_version 52532 (0.0008) [2023-10-14 19:48:22,919][61552] Updated weights for policy 0, policy_version 52542 (0.0008) [2023-10-14 19:48:23,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 107413504. Throughput: 0: 1671.0, 1: 1675.3. Samples: 26859042. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-14 19:48:23,344][60425] Avg episode reward: [(0, '76.500'), (1, '71.660')] [2023-10-14 19:48:24,130][61585] Updated weights for policy 1, policy_version 52360 (0.0008) [2023-10-14 19:48:24,497][61585] Updated weights for policy 1, policy_version 52370 (0.0009) [2023-10-14 19:48:24,874][61585] Updated weights for policy 1, policy_version 52380 (0.0009) [2023-10-14 19:48:27,193][61552] Updated weights for policy 0, policy_version 52552 (0.0008) [2023-10-14 19:48:27,561][61552] Updated weights for policy 0, policy_version 52562 (0.0008) [2023-10-14 19:48:27,933][61552] Updated weights for policy 0, policy_version 52572 (0.0008) [2023-10-14 19:48:28,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 107479040. Throughput: 0: 1655.8, 1: 1677.7. Samples: 26878910. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-14 19:48:28,344][60425] Avg episode reward: [(0, '79.420'), (1, '63.700')] [2023-10-14 19:48:28,864][61585] Updated weights for policy 1, policy_version 52390 (0.0008) [2023-10-14 19:48:29,231][61585] Updated weights for policy 1, policy_version 52400 (0.0009) [2023-10-14 19:48:29,599][61585] Updated weights for policy 1, policy_version 52410 (0.0011) [2023-10-14 19:48:31,777][61552] Updated weights for policy 0, policy_version 52582 (0.0007) [2023-10-14 19:48:32,147][61552] Updated weights for policy 0, policy_version 52592 (0.0008) [2023-10-14 19:48:32,522][61552] Updated weights for policy 0, policy_version 52602 (0.0008) [2023-10-14 19:48:33,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 107544576. Throughput: 0: 1669.1, 1: 1671.6. Samples: 26888868. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-14 19:48:33,344][60425] Avg episode reward: [(0, '81.500'), (1, '68.560')] [2023-10-14 19:48:33,345][61172] Saving new best policy, reward=81.500! [2023-10-14 19:48:33,809][61585] Updated weights for policy 1, policy_version 52420 (0.0010) [2023-10-14 19:48:34,187][61585] Updated weights for policy 1, policy_version 52430 (0.0008) [2023-10-14 19:48:34,553][61585] Updated weights for policy 1, policy_version 52440 (0.0009) [2023-10-14 19:48:36,614][61552] Updated weights for policy 0, policy_version 52612 (0.0008) [2023-10-14 19:48:36,983][61552] Updated weights for policy 0, policy_version 52622 (0.0007) [2023-10-14 19:48:37,354][61552] Updated weights for policy 0, policy_version 52632 (0.0007) [2023-10-14 19:48:38,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 107610112. Throughput: 0: 1662.5, 1: 1678.6. Samples: 26909134. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-10-14 19:48:38,344][60425] Avg episode reward: [(0, '78.740'), (1, '71.140')] [2023-10-14 19:48:38,646][61585] Updated weights for policy 1, policy_version 52450 (0.0007) [2023-10-14 19:48:39,015][61585] Updated weights for policy 1, policy_version 52460 (0.0007) [2023-10-14 19:48:39,376][61585] Updated weights for policy 1, policy_version 52470 (0.0007) [2023-10-14 19:48:39,749][61585] Updated weights for policy 1, policy_version 52480 (0.0008) [2023-10-14 19:48:41,641][61552] Updated weights for policy 0, policy_version 52642 (0.0009) [2023-10-14 19:48:42,009][61552] Updated weights for policy 0, policy_version 52652 (0.0007) [2023-10-14 19:48:42,381][61552] Updated weights for policy 0, policy_version 52662 (0.0010) [2023-10-14 19:48:42,744][61552] Updated weights for policy 0, policy_version 52672 (0.0007) [2023-10-14 19:48:43,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 107675648. Throughput: 0: 1646.3, 1: 1679.5. Samples: 26928758. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-10-14 19:48:43,344][60425] Avg episode reward: [(0, '79.790'), (1, '73.020')] [2023-10-14 19:48:43,857][61585] Updated weights for policy 1, policy_version 52490 (0.0008) [2023-10-14 19:48:44,225][61585] Updated weights for policy 1, policy_version 52500 (0.0007) [2023-10-14 19:48:44,591][61585] Updated weights for policy 1, policy_version 52510 (0.0009) [2023-10-14 19:48:46,619][61552] Updated weights for policy 0, policy_version 52682 (0.0009) [2023-10-14 19:48:46,990][61552] Updated weights for policy 0, policy_version 52692 (0.0007) [2023-10-14 19:48:47,345][61552] Updated weights for policy 0, policy_version 52702 (0.0008) [2023-10-14 19:48:48,346][60425] Fps is (10 sec: 13103.6, 60 sec: 13652.7, 300 sec: 13440.3). Total num frames: 107741184. Throughput: 0: 1669.9, 1: 1674.0. Samples: 26938942. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-10-14 19:48:48,347][60425] Avg episode reward: [(0, '76.740'), (1, '69.360')] [2023-10-14 19:48:48,824][61585] Updated weights for policy 1, policy_version 52520 (0.0008) [2023-10-14 19:48:49,187][61585] Updated weights for policy 1, policy_version 52530 (0.0009) [2023-10-14 19:48:49,556][61585] Updated weights for policy 1, policy_version 52540 (0.0008) [2023-10-14 19:48:51,518][61552] Updated weights for policy 0, policy_version 52712 (0.0010) [2023-10-14 19:48:51,893][61552] Updated weights for policy 0, policy_version 52722 (0.0008) [2023-10-14 19:48:52,264][61552] Updated weights for policy 0, policy_version 52732 (0.0010) [2023-10-14 19:48:53,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 107806720. Throughput: 0: 1661.8, 1: 1666.0. Samples: 26958774. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-10-14 19:48:53,344][60425] Avg episode reward: [(0, '73.140'), (1, '73.790')] [2023-10-14 19:48:53,724][61585] Updated weights for policy 1, policy_version 52550 (0.0008) [2023-10-14 19:48:54,084][61585] Updated weights for policy 1, policy_version 52560 (0.0009) [2023-10-14 19:48:54,446][61585] Updated weights for policy 1, policy_version 52570 (0.0009) [2023-10-14 19:48:56,462][61552] Updated weights for policy 0, policy_version 52742 (0.0009) [2023-10-14 19:48:56,842][61552] Updated weights for policy 0, policy_version 52752 (0.0012) [2023-10-14 19:48:57,203][61552] Updated weights for policy 0, policy_version 52762 (0.0009) [2023-10-14 19:48:58,343][60425] Fps is (10 sec: 13110.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 107872256. Throughput: 0: 1654.8, 1: 1675.0. Samples: 26978816. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-10-14 19:48:58,344][60425] Avg episode reward: [(0, '76.940'), (1, '76.240')] [2023-10-14 19:48:58,563][61585] Updated weights for policy 1, policy_version 52580 (0.0011) [2023-10-14 19:48:58,940][61585] Updated weights for policy 1, policy_version 52590 (0.0010) [2023-10-14 19:48:59,315][61585] Updated weights for policy 1, policy_version 52600 (0.0011) [2023-10-14 19:49:01,221][61552] Updated weights for policy 0, policy_version 52772 (0.0009) [2023-10-14 19:49:01,590][61552] Updated weights for policy 0, policy_version 52782 (0.0008) [2023-10-14 19:49:01,954][61552] Updated weights for policy 0, policy_version 52792 (0.0010) [2023-10-14 19:49:03,339][61585] Updated weights for policy 1, policy_version 52610 (0.0011) [2023-10-14 19:49:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 107937792. Throughput: 0: 1673.9, 1: 1675.1. Samples: 26989352. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-10-14 19:49:03,344][60425] Avg episode reward: [(0, '77.300'), (1, '76.650')] [2023-10-14 19:49:03,707][61585] Updated weights for policy 1, policy_version 52620 (0.0010) [2023-10-14 19:49:04,068][61585] Updated weights for policy 1, policy_version 52630 (0.0008) [2023-10-14 19:49:04,435][61585] Updated weights for policy 1, policy_version 52640 (0.0008) [2023-10-14 19:49:06,028][61552] Updated weights for policy 0, policy_version 52802 (0.0008) [2023-10-14 19:49:06,388][61552] Updated weights for policy 0, policy_version 52812 (0.0008) [2023-10-14 19:49:06,763][61552] Updated weights for policy 0, policy_version 52822 (0.0008) [2023-10-14 19:49:07,131][61552] Updated weights for policy 0, policy_version 52832 (0.0008) [2023-10-14 19:49:08,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 108003328. Throughput: 0: 1660.5, 1: 1672.9. Samples: 27009046. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-10-14 19:49:08,344][60425] Avg episode reward: [(0, '72.470'), (1, '74.870')] [2023-10-14 19:49:08,550][61585] Updated weights for policy 1, policy_version 52650 (0.0007) [2023-10-14 19:49:08,926][61585] Updated weights for policy 1, policy_version 52660 (0.0009) [2023-10-14 19:49:09,282][61585] Updated weights for policy 1, policy_version 52670 (0.0008) [2023-10-14 19:49:11,196][61552] Updated weights for policy 0, policy_version 52842 (0.0007) [2023-10-14 19:49:11,563][61552] Updated weights for policy 0, policy_version 52852 (0.0009) [2023-10-14 19:49:11,921][61552] Updated weights for policy 0, policy_version 52862 (0.0007) [2023-10-14 19:49:13,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 108068864. Throughput: 0: 1671.5, 1: 1672.4. Samples: 27029384. Policy #0 lag: (min: 21.0, avg: 32.3, max: 53.0) [2023-10-14 19:49:13,344][60425] Avg episode reward: [(0, '74.350'), (1, '69.790')] [2023-10-14 19:49:13,361][61585] Updated weights for policy 1, policy_version 52680 (0.0009) [2023-10-14 19:49:13,734][61585] Updated weights for policy 1, policy_version 52690 (0.0009) [2023-10-14 19:49:14,104][61585] Updated weights for policy 1, policy_version 52700 (0.0007) [2023-10-14 19:49:15,989][61552] Updated weights for policy 0, policy_version 52872 (0.0008) [2023-10-14 19:49:16,357][61552] Updated weights for policy 0, policy_version 52882 (0.0011) [2023-10-14 19:49:16,725][61552] Updated weights for policy 0, policy_version 52892 (0.0009) [2023-10-14 19:49:18,332][61585] Updated weights for policy 1, policy_version 52710 (0.0009) [2023-10-14 19:49:18,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 108134400. Throughput: 0: 1680.7, 1: 1668.7. Samples: 27039588. Policy #0 lag: (min: 21.0, avg: 32.3, max: 53.0) [2023-10-14 19:49:18,344][60425] Avg episode reward: [(0, '68.980'), (1, '73.180')] [2023-10-14 19:49:18,699][61585] Updated weights for policy 1, policy_version 52720 (0.0008) [2023-10-14 19:49:19,063][61585] Updated weights for policy 1, policy_version 52730 (0.0009) [2023-10-14 19:49:20,895][61552] Updated weights for policy 0, policy_version 52902 (0.0008) [2023-10-14 19:49:21,259][61552] Updated weights for policy 0, policy_version 52912 (0.0009) [2023-10-14 19:49:21,622][61552] Updated weights for policy 0, policy_version 52922 (0.0011) [2023-10-14 19:49:23,220][61585] Updated weights for policy 1, policy_version 52740 (0.0007) [2023-10-14 19:49:23,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 108199936. Throughput: 0: 1661.2, 1: 1663.7. Samples: 27058756. Policy #0 lag: (min: 21.0, avg: 32.3, max: 53.0) [2023-10-14 19:49:23,344][60425] Avg episode reward: [(0, '74.240'), (1, '70.870')] [2023-10-14 19:49:23,585][61585] Updated weights for policy 1, policy_version 52750 (0.0007) [2023-10-14 19:49:23,957][61585] Updated weights for policy 1, policy_version 52760 (0.0008) [2023-10-14 19:49:25,647][61552] Updated weights for policy 0, policy_version 52932 (0.0009) [2023-10-14 19:49:26,007][61552] Updated weights for policy 0, policy_version 52942 (0.0008) [2023-10-14 19:49:26,375][61552] Updated weights for policy 0, policy_version 52952 (0.0008) [2023-10-14 19:49:28,159][61585] Updated weights for policy 1, policy_version 52770 (0.0007) [2023-10-14 19:49:28,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 108265472. Throughput: 0: 1685.1, 1: 1660.4. Samples: 27079304. Policy #0 lag: (min: 21.0, avg: 32.3, max: 53.0) [2023-10-14 19:49:28,344][60425] Avg episode reward: [(0, '71.740'), (1, '73.850')] [2023-10-14 19:49:28,523][61585] Updated weights for policy 1, policy_version 52780 (0.0007) [2023-10-14 19:49:28,898][61585] Updated weights for policy 1, policy_version 52790 (0.0007) [2023-10-14 19:49:29,260][61585] Updated weights for policy 1, policy_version 52800 (0.0008) [2023-10-14 19:49:30,461][61552] Updated weights for policy 0, policy_version 52962 (0.0009) [2023-10-14 19:49:30,825][61552] Updated weights for policy 0, policy_version 52972 (0.0010) [2023-10-14 19:49:31,196][61552] Updated weights for policy 0, policy_version 52982 (0.0010) [2023-10-14 19:49:31,565][61552] Updated weights for policy 0, policy_version 52992 (0.0008) [2023-10-14 19:49:33,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 108331008. Throughput: 0: 1677.7, 1: 1664.5. Samples: 27089332. Policy #0 lag: (min: 21.0, avg: 32.3, max: 53.0) [2023-10-14 19:49:33,344][60425] Avg episode reward: [(0, '74.600'), (1, '70.310')] [2023-10-14 19:49:33,426][61585] Updated weights for policy 1, policy_version 52810 (0.0008) [2023-10-14 19:49:33,790][61585] Updated weights for policy 1, policy_version 52820 (0.0007) [2023-10-14 19:49:34,151][61585] Updated weights for policy 1, policy_version 52830 (0.0007) [2023-10-14 19:49:35,663][61552] Updated weights for policy 0, policy_version 53002 (0.0010) [2023-10-14 19:49:36,046][61552] Updated weights for policy 0, policy_version 53012 (0.0009) [2023-10-14 19:49:36,399][61552] Updated weights for policy 0, policy_version 53022 (0.0010) [2023-10-14 19:49:38,167][61585] Updated weights for policy 1, policy_version 52840 (0.0008) [2023-10-14 19:49:38,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 108396544. Throughput: 0: 1665.9, 1: 1675.0. Samples: 27109114. Policy #0 lag: (min: 21.0, avg: 32.3, max: 53.0) [2023-10-14 19:49:38,344][60425] Avg episode reward: [(0, '74.470'), (1, '73.060')] [2023-10-14 19:49:38,536][61585] Updated weights for policy 1, policy_version 52850 (0.0009) [2023-10-14 19:49:38,900][61585] Updated weights for policy 1, policy_version 52860 (0.0009) [2023-10-14 19:49:40,434][61552] Updated weights for policy 0, policy_version 53032 (0.0008) [2023-10-14 19:49:40,803][61552] Updated weights for policy 0, policy_version 53042 (0.0008) [2023-10-14 19:49:41,174][61552] Updated weights for policy 0, policy_version 53052 (0.0009) [2023-10-14 19:49:42,905][61585] Updated weights for policy 1, policy_version 52870 (0.0008) [2023-10-14 19:49:43,260][61585] Updated weights for policy 1, policy_version 52880 (0.0008) [2023-10-14 19:49:43,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 108462080. Throughput: 0: 1688.8, 1: 1665.8. Samples: 27129774. Policy #0 lag: (min: 21.0, avg: 32.3, max: 53.0) [2023-10-14 19:49:43,344][60425] Avg episode reward: [(0, '79.830'), (1, '72.160')] [2023-10-14 19:49:43,353][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000053056_54329344.pth... [2023-10-14 19:49:43,389][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000051488_52723712.pth [2023-10-14 19:49:43,625][61585] Updated weights for policy 1, policy_version 52890 (0.0008) [2023-10-14 19:49:43,835][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000052896_54165504.pth... [2023-10-14 19:49:43,873][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000051328_52559872.pth [2023-10-14 19:49:45,390][61552] Updated weights for policy 0, policy_version 53062 (0.0010) [2023-10-14 19:49:45,764][61552] Updated weights for policy 0, policy_version 53072 (0.0010) [2023-10-14 19:49:46,131][61552] Updated weights for policy 0, policy_version 53082 (0.0010) [2023-10-14 19:49:47,628][61585] Updated weights for policy 1, policy_version 52900 (0.0008) [2023-10-14 19:49:47,990][61585] Updated weights for policy 1, policy_version 52910 (0.0009) [2023-10-14 19:49:48,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.8, 300 sec: 13329.4). Total num frames: 108527616. Throughput: 0: 1668.5, 1: 1667.4. Samples: 27139466. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-14 19:49:48,344][60425] Avg episode reward: [(0, '78.270'), (1, '74.210')] [2023-10-14 19:49:48,352][61585] Updated weights for policy 1, policy_version 52920 (0.0009) [2023-10-14 19:49:50,352][61552] Updated weights for policy 0, policy_version 53092 (0.0009) [2023-10-14 19:49:50,723][61552] Updated weights for policy 0, policy_version 53102 (0.0008) [2023-10-14 19:49:51,090][61552] Updated weights for policy 0, policy_version 53112 (0.0008) [2023-10-14 19:49:52,502][61585] Updated weights for policy 1, policy_version 52930 (0.0009) [2023-10-14 19:49:52,876][61585] Updated weights for policy 1, policy_version 52940 (0.0008) [2023-10-14 19:49:53,248][61585] Updated weights for policy 1, policy_version 52950 (0.0011) [2023-10-14 19:49:53,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 108593152. Throughput: 0: 1666.6, 1: 1670.3. Samples: 27159206. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-14 19:49:53,344][60425] Avg episode reward: [(0, '74.800'), (1, '73.070')] [2023-10-14 19:49:53,618][61585] Updated weights for policy 1, policy_version 52960 (0.0010) [2023-10-14 19:49:55,060][61552] Updated weights for policy 0, policy_version 53122 (0.0009) [2023-10-14 19:49:55,439][61552] Updated weights for policy 0, policy_version 53132 (0.0009) [2023-10-14 19:49:55,814][61552] Updated weights for policy 0, policy_version 53142 (0.0008) [2023-10-14 19:49:56,180][61552] Updated weights for policy 0, policy_version 53152 (0.0008) [2023-10-14 19:49:57,733][61585] Updated weights for policy 1, policy_version 52970 (0.0009) [2023-10-14 19:49:58,107][61585] Updated weights for policy 1, policy_version 52980 (0.0009) [2023-10-14 19:49:58,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 108658688. Throughput: 0: 1673.6, 1: 1659.6. Samples: 27179382. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-14 19:49:58,344][60425] Avg episode reward: [(0, '80.780'), (1, '69.290')] [2023-10-14 19:49:58,477][61585] Updated weights for policy 1, policy_version 52990 (0.0007) [2023-10-14 19:50:00,185][61552] Updated weights for policy 0, policy_version 53162 (0.0009) [2023-10-14 19:50:00,552][61552] Updated weights for policy 0, policy_version 53172 (0.0009) [2023-10-14 19:50:00,910][61552] Updated weights for policy 0, policy_version 53182 (0.0007) [2023-10-14 19:50:02,474][61585] Updated weights for policy 1, policy_version 53000 (0.0008) [2023-10-14 19:50:02,842][61585] Updated weights for policy 1, policy_version 53010 (0.0007) [2023-10-14 19:50:03,213][61585] Updated weights for policy 1, policy_version 53020 (0.0008) [2023-10-14 19:50:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 108724224. Throughput: 0: 1652.8, 1: 1671.4. Samples: 27189176. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-14 19:50:03,344][60425] Avg episode reward: [(0, '78.470'), (1, '71.510')] [2023-10-14 19:50:05,117][61552] Updated weights for policy 0, policy_version 53192 (0.0007) [2023-10-14 19:50:05,489][61552] Updated weights for policy 0, policy_version 53202 (0.0009) [2023-10-14 19:50:05,842][61552] Updated weights for policy 0, policy_version 53212 (0.0009) [2023-10-14 19:50:07,216][61585] Updated weights for policy 1, policy_version 53030 (0.0008) [2023-10-14 19:50:07,591][61585] Updated weights for policy 1, policy_version 53040 (0.0008) [2023-10-14 19:50:07,955][61585] Updated weights for policy 1, policy_version 53050 (0.0008) [2023-10-14 19:50:08,343][60425] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 108822528. Throughput: 0: 1672.1, 1: 1680.0. Samples: 27209596. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-14 19:50:08,344][60425] Avg episode reward: [(0, '74.990'), (1, '72.410')] [2023-10-14 19:50:09,887][61552] Updated weights for policy 0, policy_version 53222 (0.0007) [2023-10-14 19:50:10,255][61552] Updated weights for policy 0, policy_version 53232 (0.0008) [2023-10-14 19:50:10,621][61552] Updated weights for policy 0, policy_version 53242 (0.0011) [2023-10-14 19:50:12,051][61585] Updated weights for policy 1, policy_version 53060 (0.0007) [2023-10-14 19:50:12,415][61585] Updated weights for policy 1, policy_version 53070 (0.0008) [2023-10-14 19:50:12,770][61585] Updated weights for policy 1, policy_version 53080 (0.0008) [2023-10-14 19:50:13,343][60425] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 108888064. Throughput: 0: 1670.4, 1: 1664.3. Samples: 27229366. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-14 19:50:13,344][60425] Avg episode reward: [(0, '76.600'), (1, '77.580')] [2023-10-14 19:50:14,774][61552] Updated weights for policy 0, policy_version 53252 (0.0011) [2023-10-14 19:50:15,141][61552] Updated weights for policy 0, policy_version 53262 (0.0007) [2023-10-14 19:50:15,502][61552] Updated weights for policy 0, policy_version 53272 (0.0007) [2023-10-14 19:50:16,993][61585] Updated weights for policy 1, policy_version 53090 (0.0009) [2023-10-14 19:50:17,357][61585] Updated weights for policy 1, policy_version 53100 (0.0007) [2023-10-14 19:50:17,732][61585] Updated weights for policy 1, policy_version 53110 (0.0007) [2023-10-14 19:50:18,090][61585] Updated weights for policy 1, policy_version 53120 (0.0009) [2023-10-14 19:50:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 108953600. Throughput: 0: 1652.0, 1: 1682.7. Samples: 27239390. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-14 19:50:18,344][60425] Avg episode reward: [(0, '76.930'), (1, '74.040')] [2023-10-14 19:50:19,540][61552] Updated weights for policy 0, policy_version 53282 (0.0011) [2023-10-14 19:50:19,919][61552] Updated weights for policy 0, policy_version 53292 (0.0008) [2023-10-14 19:50:20,287][61552] Updated weights for policy 0, policy_version 53302 (0.0009) [2023-10-14 19:50:20,658][61552] Updated weights for policy 0, policy_version 53312 (0.0007) [2023-10-14 19:50:22,064][61585] Updated weights for policy 1, policy_version 53130 (0.0008) [2023-10-14 19:50:22,435][61585] Updated weights for policy 1, policy_version 53140 (0.0010) [2023-10-14 19:50:22,800][61585] Updated weights for policy 1, policy_version 53150 (0.0008) [2023-10-14 19:50:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 109019136. Throughput: 0: 1667.5, 1: 1675.6. Samples: 27259552. Policy #0 lag: (min: 10.0, avg: 19.7, max: 42.0) [2023-10-14 19:50:23,344][60425] Avg episode reward: [(0, '74.940'), (1, '80.480')] [2023-10-14 19:50:24,803][61552] Updated weights for policy 0, policy_version 53322 (0.0009) [2023-10-14 19:50:25,170][61552] Updated weights for policy 0, policy_version 53332 (0.0007) [2023-10-14 19:50:25,540][61552] Updated weights for policy 0, policy_version 53342 (0.0008) [2023-10-14 19:50:27,109][61585] Updated weights for policy 1, policy_version 53160 (0.0007) [2023-10-14 19:50:27,485][61585] Updated weights for policy 1, policy_version 53170 (0.0009) [2023-10-14 19:50:27,866][61585] Updated weights for policy 1, policy_version 53180 (0.0011) [2023-10-14 19:50:28,343][60425] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 109084672. Throughput: 0: 1666.6, 1: 1648.2. Samples: 27278942. Policy #0 lag: (min: 10.0, avg: 19.7, max: 42.0) [2023-10-14 19:50:28,345][60425] Avg episode reward: [(0, '73.530'), (1, '75.860')] [2023-10-14 19:50:29,650][61552] Updated weights for policy 0, policy_version 53352 (0.0008) [2023-10-14 19:50:30,023][61552] Updated weights for policy 0, policy_version 53362 (0.0008) [2023-10-14 19:50:30,381][61552] Updated weights for policy 0, policy_version 53372 (0.0010) [2023-10-14 19:50:31,977][61585] Updated weights for policy 1, policy_version 53190 (0.0008) [2023-10-14 19:50:32,342][61585] Updated weights for policy 1, policy_version 53200 (0.0008) [2023-10-14 19:50:32,711][61585] Updated weights for policy 1, policy_version 53210 (0.0008) [2023-10-14 19:50:33,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 109150208. Throughput: 0: 1653.1, 1: 1671.8. Samples: 27289086. Policy #0 lag: (min: 10.0, avg: 19.7, max: 42.0) [2023-10-14 19:50:33,344][60425] Avg episode reward: [(0, '74.530'), (1, '77.490')] [2023-10-14 19:50:34,432][61552] Updated weights for policy 0, policy_version 53382 (0.0008) [2023-10-14 19:50:34,787][61552] Updated weights for policy 0, policy_version 53392 (0.0009) [2023-10-14 19:50:35,155][61552] Updated weights for policy 0, policy_version 53402 (0.0008) [2023-10-14 19:50:36,931][61585] Updated weights for policy 1, policy_version 53220 (0.0008) [2023-10-14 19:50:37,294][61585] Updated weights for policy 1, policy_version 53230 (0.0011) [2023-10-14 19:50:37,664][61585] Updated weights for policy 1, policy_version 53240 (0.0010) [2023-10-14 19:50:38,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 109215744. Throughput: 0: 1674.6, 1: 1668.9. Samples: 27309662. Policy #0 lag: (min: 10.0, avg: 19.7, max: 42.0) [2023-10-14 19:50:38,344][60425] Avg episode reward: [(0, '73.700'), (1, '75.840')] [2023-10-14 19:50:39,108][61552] Updated weights for policy 0, policy_version 53412 (0.0009) [2023-10-14 19:50:39,474][61552] Updated weights for policy 0, policy_version 53422 (0.0009) [2023-10-14 19:50:39,839][61552] Updated weights for policy 0, policy_version 53432 (0.0008) [2023-10-14 19:50:41,512][61585] Updated weights for policy 1, policy_version 53250 (0.0007) [2023-10-14 19:50:41,874][61585] Updated weights for policy 1, policy_version 53260 (0.0008) [2023-10-14 19:50:42,241][61585] Updated weights for policy 1, policy_version 53270 (0.0009) [2023-10-14 19:50:42,616][61585] Updated weights for policy 1, policy_version 53280 (0.0007) [2023-10-14 19:50:43,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 109281280. Throughput: 0: 1677.6, 1: 1653.4. Samples: 27329276. Policy #0 lag: (min: 10.0, avg: 19.7, max: 42.0) [2023-10-14 19:50:43,344][60425] Avg episode reward: [(0, '76.760'), (1, '75.390')] [2023-10-14 19:50:43,970][61552] Updated weights for policy 0, policy_version 53442 (0.0010) [2023-10-14 19:50:44,337][61552] Updated weights for policy 0, policy_version 53452 (0.0009) [2023-10-14 19:50:44,709][61552] Updated weights for policy 0, policy_version 53462 (0.0008) [2023-10-14 19:50:45,066][61552] Updated weights for policy 0, policy_version 53472 (0.0010) [2023-10-14 19:50:46,723][61585] Updated weights for policy 1, policy_version 53290 (0.0008) [2023-10-14 19:50:47,090][61585] Updated weights for policy 1, policy_version 53300 (0.0008) [2023-10-14 19:50:47,455][61585] Updated weights for policy 1, policy_version 53310 (0.0008) [2023-10-14 19:50:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 109346816. Throughput: 0: 1667.5, 1: 1671.6. Samples: 27339434. Policy #0 lag: (min: 10.0, avg: 19.7, max: 42.0) [2023-10-14 19:50:48,344][60425] Avg episode reward: [(0, '75.220'), (1, '78.580')] [2023-10-14 19:50:49,240][61552] Updated weights for policy 0, policy_version 53482 (0.0008) [2023-10-14 19:50:49,604][61552] Updated weights for policy 0, policy_version 53492 (0.0010) [2023-10-14 19:50:49,970][61552] Updated weights for policy 0, policy_version 53502 (0.0010) [2023-10-14 19:50:51,701][61585] Updated weights for policy 1, policy_version 53320 (0.0010) [2023-10-14 19:50:52,064][61585] Updated weights for policy 1, policy_version 53330 (0.0008) [2023-10-14 19:50:52,433][61585] Updated weights for policy 1, policy_version 53340 (0.0008) [2023-10-14 19:50:53,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 109412352. Throughput: 0: 1673.9, 1: 1655.9. Samples: 27359438. Policy #0 lag: (min: 10.0, avg: 19.7, max: 42.0) [2023-10-14 19:50:53,344][60425] Avg episode reward: [(0, '76.260'), (1, '76.500')] [2023-10-14 19:50:54,142][61552] Updated weights for policy 0, policy_version 53512 (0.0007) [2023-10-14 19:50:54,504][61552] Updated weights for policy 0, policy_version 53522 (0.0009) [2023-10-14 19:50:54,877][61552] Updated weights for policy 0, policy_version 53532 (0.0008) [2023-10-14 19:50:56,535][61585] Updated weights for policy 1, policy_version 53350 (0.0008) [2023-10-14 19:50:56,899][61585] Updated weights for policy 1, policy_version 53360 (0.0007) [2023-10-14 19:50:57,260][61585] Updated weights for policy 1, policy_version 53370 (0.0009) [2023-10-14 19:50:58,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 109477888. Throughput: 0: 1677.8, 1: 1656.2. Samples: 27379394. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-10-14 19:50:58,344][60425] Avg episode reward: [(0, '75.600'), (1, '71.000')] [2023-10-14 19:50:58,895][61552] Updated weights for policy 0, policy_version 53542 (0.0011) [2023-10-14 19:50:59,273][61552] Updated weights for policy 0, policy_version 53552 (0.0008) [2023-10-14 19:50:59,638][61552] Updated weights for policy 0, policy_version 53562 (0.0009) [2023-10-14 19:51:01,321][61585] Updated weights for policy 1, policy_version 53380 (0.0007) [2023-10-14 19:51:01,685][61585] Updated weights for policy 1, policy_version 53390 (0.0009) [2023-10-14 19:51:02,054][61585] Updated weights for policy 1, policy_version 53400 (0.0009) [2023-10-14 19:51:03,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 109543424. Throughput: 0: 1675.6, 1: 1668.4. Samples: 27389870. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-10-14 19:51:03,344][60425] Avg episode reward: [(0, '76.330'), (1, '70.070')] [2023-10-14 19:51:03,648][61552] Updated weights for policy 0, policy_version 53572 (0.0008) [2023-10-14 19:51:04,019][61552] Updated weights for policy 0, policy_version 53582 (0.0009) [2023-10-14 19:51:04,385][61552] Updated weights for policy 0, policy_version 53592 (0.0008) [2023-10-14 19:51:06,106][61585] Updated weights for policy 1, policy_version 53410 (0.0007) [2023-10-14 19:51:06,466][61585] Updated weights for policy 1, policy_version 53420 (0.0008) [2023-10-14 19:51:06,834][61585] Updated weights for policy 1, policy_version 53430 (0.0010) [2023-10-14 19:51:07,196][61585] Updated weights for policy 1, policy_version 53440 (0.0010) [2023-10-14 19:51:08,344][60425] Fps is (10 sec: 13106.7, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 109608960. Throughput: 0: 1686.2, 1: 1654.8. Samples: 27409902. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-10-14 19:51:08,345][60425] Avg episode reward: [(0, '74.440'), (1, '78.630')] [2023-10-14 19:51:08,565][61552] Updated weights for policy 0, policy_version 53602 (0.0008) [2023-10-14 19:51:08,932][61552] Updated weights for policy 0, policy_version 53612 (0.0008) [2023-10-14 19:51:09,306][61552] Updated weights for policy 0, policy_version 53622 (0.0009) [2023-10-14 19:51:09,668][61552] Updated weights for policy 0, policy_version 53632 (0.0008) [2023-10-14 19:51:11,223][61585] Updated weights for policy 1, policy_version 53450 (0.0008) [2023-10-14 19:51:11,587][61585] Updated weights for policy 1, policy_version 53460 (0.0009) [2023-10-14 19:51:11,969][61585] Updated weights for policy 1, policy_version 53470 (0.0009) [2023-10-14 19:51:13,343][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 109674496. Throughput: 0: 1683.6, 1: 1675.8. Samples: 27430116. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-10-14 19:51:13,344][60425] Avg episode reward: [(0, '74.470'), (1, '78.650')] [2023-10-14 19:51:13,728][61552] Updated weights for policy 0, policy_version 53642 (0.0010) [2023-10-14 19:51:14,098][61552] Updated weights for policy 0, policy_version 53652 (0.0010) [2023-10-14 19:51:14,461][61552] Updated weights for policy 0, policy_version 53662 (0.0010) [2023-10-14 19:51:15,976][61585] Updated weights for policy 1, policy_version 53480 (0.0010) [2023-10-14 19:51:16,346][61585] Updated weights for policy 1, policy_version 53490 (0.0011) [2023-10-14 19:51:16,716][61585] Updated weights for policy 1, policy_version 53500 (0.0011) [2023-10-14 19:51:18,343][60425] Fps is (10 sec: 13108.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 109740032. Throughput: 0: 1683.4, 1: 1676.8. Samples: 27440292. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-10-14 19:51:18,344][60425] Avg episode reward: [(0, '75.240'), (1, '75.100')] [2023-10-14 19:51:18,767][61552] Updated weights for policy 0, policy_version 53672 (0.0008) [2023-10-14 19:51:19,138][61552] Updated weights for policy 0, policy_version 53682 (0.0009) [2023-10-14 19:51:19,509][61552] Updated weights for policy 0, policy_version 53692 (0.0008) [2023-10-14 19:51:20,752][61585] Updated weights for policy 1, policy_version 53510 (0.0008) [2023-10-14 19:51:21,120][61585] Updated weights for policy 1, policy_version 53520 (0.0008) [2023-10-14 19:51:21,490][61585] Updated weights for policy 1, policy_version 53530 (0.0008) [2023-10-14 19:51:23,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 109805568. Throughput: 0: 1676.0, 1: 1656.7. Samples: 27459630. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-10-14 19:51:23,344][60425] Avg episode reward: [(0, '71.020'), (1, '76.220')] [2023-10-14 19:51:23,513][61552] Updated weights for policy 0, policy_version 53702 (0.0008) [2023-10-14 19:51:23,882][61552] Updated weights for policy 0, policy_version 53712 (0.0007) [2023-10-14 19:51:24,245][61552] Updated weights for policy 0, policy_version 53722 (0.0008) [2023-10-14 19:51:25,615][61585] Updated weights for policy 1, policy_version 53540 (0.0009) [2023-10-14 19:51:25,972][61585] Updated weights for policy 1, policy_version 53550 (0.0010) [2023-10-14 19:51:26,344][61585] Updated weights for policy 1, policy_version 53560 (0.0010) [2023-10-14 19:51:28,344][60425] Fps is (10 sec: 13106.6, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 109871104. Throughput: 0: 1673.8, 1: 1679.2. Samples: 27480162. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-10-14 19:51:28,345][60425] Avg episode reward: [(0, '73.780'), (1, '77.260')] [2023-10-14 19:51:28,375][61552] Updated weights for policy 0, policy_version 53732 (0.0008) [2023-10-14 19:51:28,750][61552] Updated weights for policy 0, policy_version 53742 (0.0008) [2023-10-14 19:51:29,119][61552] Updated weights for policy 0, policy_version 53752 (0.0009) [2023-10-14 19:51:30,441][61585] Updated weights for policy 1, policy_version 53570 (0.0008) [2023-10-14 19:51:30,808][61585] Updated weights for policy 1, policy_version 53580 (0.0007) [2023-10-14 19:51:31,174][61585] Updated weights for policy 1, policy_version 53590 (0.0010) [2023-10-14 19:51:31,527][61585] Updated weights for policy 1, policy_version 53600 (0.0009) [2023-10-14 19:51:33,159][61552] Updated weights for policy 0, policy_version 53762 (0.0010) [2023-10-14 19:51:33,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 109936640. Throughput: 0: 1672.2, 1: 1673.0. Samples: 27489966. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-14 19:51:33,344][60425] Avg episode reward: [(0, '71.450'), (1, '75.860')] [2023-10-14 19:51:33,521][61552] Updated weights for policy 0, policy_version 53772 (0.0008) [2023-10-14 19:51:33,892][61552] Updated weights for policy 0, policy_version 53782 (0.0008) [2023-10-14 19:51:34,263][61552] Updated weights for policy 0, policy_version 53792 (0.0010) [2023-10-14 19:51:35,586][61585] Updated weights for policy 1, policy_version 53610 (0.0008) [2023-10-14 19:51:35,958][61585] Updated weights for policy 1, policy_version 53620 (0.0007) [2023-10-14 19:51:36,324][61585] Updated weights for policy 1, policy_version 53630 (0.0007) [2023-10-14 19:51:38,294][61552] Updated weights for policy 0, policy_version 53802 (0.0008) [2023-10-14 19:51:38,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 110002176. Throughput: 0: 1677.5, 1: 1663.9. Samples: 27509798. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-14 19:51:38,344][60425] Avg episode reward: [(0, '72.490'), (1, '76.150')] [2023-10-14 19:51:38,660][61552] Updated weights for policy 0, policy_version 53812 (0.0007) [2023-10-14 19:51:39,034][61552] Updated weights for policy 0, policy_version 53822 (0.0007) [2023-10-14 19:51:40,418][61585] Updated weights for policy 1, policy_version 53640 (0.0010) [2023-10-14 19:51:40,784][61585] Updated weights for policy 1, policy_version 53650 (0.0011) [2023-10-14 19:51:41,152][61585] Updated weights for policy 1, policy_version 53660 (0.0008) [2023-10-14 19:51:43,164][61552] Updated weights for policy 0, policy_version 53832 (0.0008) [2023-10-14 19:51:43,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 110067712. Throughput: 0: 1675.2, 1: 1683.9. Samples: 27530550. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-14 19:51:43,344][60425] Avg episode reward: [(0, '72.410'), (1, '77.070')] [2023-10-14 19:51:43,352][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000053664_54951936.pth... [2023-10-14 19:51:43,387][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000052096_53346304.pth [2023-10-14 19:51:43,531][61552] Updated weights for policy 0, policy_version 53842 (0.0008) [2023-10-14 19:51:43,897][61552] Updated weights for policy 0, policy_version 53852 (0.0008) [2023-10-14 19:51:44,040][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000053856_55148544.pth... [2023-10-14 19:51:44,069][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000052288_53542912.pth [2023-10-14 19:51:45,313][61585] Updated weights for policy 1, policy_version 53670 (0.0009) [2023-10-14 19:51:45,673][61585] Updated weights for policy 1, policy_version 53680 (0.0008) [2023-10-14 19:51:46,033][61585] Updated weights for policy 1, policy_version 53690 (0.0008) [2023-10-14 19:51:47,875][61552] Updated weights for policy 0, policy_version 53862 (0.0010) [2023-10-14 19:51:48,240][61552] Updated weights for policy 0, policy_version 53872 (0.0008) [2023-10-14 19:51:48,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 110133248. Throughput: 0: 1676.1, 1: 1667.6. Samples: 27540340. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-14 19:51:48,344][60425] Avg episode reward: [(0, '77.040'), (1, '77.670')] [2023-10-14 19:51:48,617][61552] Updated weights for policy 0, policy_version 53882 (0.0009) [2023-10-14 19:51:50,165][61585] Updated weights for policy 1, policy_version 53700 (0.0008) [2023-10-14 19:51:50,528][61585] Updated weights for policy 1, policy_version 53710 (0.0007) [2023-10-14 19:51:50,900][61585] Updated weights for policy 1, policy_version 53720 (0.0008) [2023-10-14 19:51:52,757][61552] Updated weights for policy 0, policy_version 53892 (0.0009) [2023-10-14 19:51:53,127][61552] Updated weights for policy 0, policy_version 53902 (0.0007) [2023-10-14 19:51:53,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 110198784. Throughput: 0: 1671.6, 1: 1671.2. Samples: 27560326. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-14 19:51:53,344][60425] Avg episode reward: [(0, '76.210'), (1, '74.160')] [2023-10-14 19:51:53,494][61552] Updated weights for policy 0, policy_version 53912 (0.0008) [2023-10-14 19:51:54,791][61585] Updated weights for policy 1, policy_version 53730 (0.0009) [2023-10-14 19:51:55,150][61585] Updated weights for policy 1, policy_version 53740 (0.0007) [2023-10-14 19:51:55,507][61585] Updated weights for policy 1, policy_version 53750 (0.0007) [2023-10-14 19:51:55,868][61585] Updated weights for policy 1, policy_version 53760 (0.0007) [2023-10-14 19:51:57,568][61552] Updated weights for policy 0, policy_version 53922 (0.0008) [2023-10-14 19:51:57,947][61552] Updated weights for policy 0, policy_version 53932 (0.0009) [2023-10-14 19:51:58,323][61552] Updated weights for policy 0, policy_version 53942 (0.0008) [2023-10-14 19:51:58,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 110264320. Throughput: 0: 1666.0, 1: 1683.1. Samples: 27580822. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-14 19:51:58,344][60425] Avg episode reward: [(0, '75.300'), (1, '77.670')] [2023-10-14 19:51:58,686][61552] Updated weights for policy 0, policy_version 53952 (0.0008) [2023-10-14 19:52:00,001][61585] Updated weights for policy 1, policy_version 53770 (0.0007) [2023-10-14 19:52:00,365][61585] Updated weights for policy 1, policy_version 53780 (0.0007) [2023-10-14 19:52:00,734][61585] Updated weights for policy 1, policy_version 53790 (0.0007) [2023-10-14 19:52:02,702][61552] Updated weights for policy 0, policy_version 53962 (0.0009) [2023-10-14 19:52:03,068][61552] Updated weights for policy 0, policy_version 53972 (0.0009) [2023-10-14 19:52:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 110329856. Throughput: 0: 1674.9, 1: 1660.4. Samples: 27590380. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-14 19:52:03,344][60425] Avg episode reward: [(0, '76.390'), (1, '71.420')] [2023-10-14 19:52:03,442][61552] Updated weights for policy 0, policy_version 53982 (0.0007) [2023-10-14 19:52:04,839][61585] Updated weights for policy 1, policy_version 53800 (0.0010) [2023-10-14 19:52:05,212][61585] Updated weights for policy 1, policy_version 53810 (0.0010) [2023-10-14 19:52:05,582][61585] Updated weights for policy 1, policy_version 53820 (0.0010) [2023-10-14 19:52:07,426][61552] Updated weights for policy 0, policy_version 53992 (0.0009) [2023-10-14 19:52:07,787][61552] Updated weights for policy 0, policy_version 54002 (0.0008) [2023-10-14 19:52:08,163][61552] Updated weights for policy 0, policy_version 54012 (0.0010) [2023-10-14 19:52:08,343][60425] Fps is (10 sec: 16384.0, 60 sec: 13653.5, 300 sec: 13440.4). Total num frames: 110428160. Throughput: 0: 1681.5, 1: 1675.8. Samples: 27610708. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:52:08,344][60425] Avg episode reward: [(0, '78.320'), (1, '76.140')] [2023-10-14 19:52:09,676][61585] Updated weights for policy 1, policy_version 53830 (0.0011) [2023-10-14 19:52:10,040][61585] Updated weights for policy 1, policy_version 53840 (0.0007) [2023-10-14 19:52:10,415][61585] Updated weights for policy 1, policy_version 53850 (0.0009) [2023-10-14 19:52:12,298][61552] Updated weights for policy 0, policy_version 54022 (0.0008) [2023-10-14 19:52:12,662][61552] Updated weights for policy 0, policy_version 54032 (0.0007) [2023-10-14 19:52:13,028][61552] Updated weights for policy 0, policy_version 54042 (0.0007) [2023-10-14 19:52:13,343][60425] Fps is (10 sec: 16383.8, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 110493696. Throughput: 0: 1668.1, 1: 1680.6. Samples: 27630850. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:52:13,344][60425] Avg episode reward: [(0, '76.770'), (1, '77.000')] [2023-10-14 19:52:14,572][61585] Updated weights for policy 1, policy_version 53860 (0.0008) [2023-10-14 19:52:14,933][61585] Updated weights for policy 1, policy_version 53870 (0.0010) [2023-10-14 19:52:15,293][61585] Updated weights for policy 1, policy_version 53880 (0.0008) [2023-10-14 19:52:17,034][61552] Updated weights for policy 0, policy_version 54052 (0.0007) [2023-10-14 19:52:17,400][61552] Updated weights for policy 0, policy_version 54062 (0.0007) [2023-10-14 19:52:17,775][61552] Updated weights for policy 0, policy_version 54072 (0.0008) [2023-10-14 19:52:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 110559232. Throughput: 0: 1685.8, 1: 1659.0. Samples: 27640482. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:52:18,344][60425] Avg episode reward: [(0, '77.980'), (1, '78.610')] [2023-10-14 19:52:19,502][61585] Updated weights for policy 1, policy_version 53890 (0.0010) [2023-10-14 19:52:19,872][61585] Updated weights for policy 1, policy_version 53900 (0.0008) [2023-10-14 19:52:20,232][61585] Updated weights for policy 1, policy_version 53910 (0.0009) [2023-10-14 19:52:20,608][61585] Updated weights for policy 1, policy_version 53920 (0.0008) [2023-10-14 19:52:21,907][61552] Updated weights for policy 0, policy_version 54082 (0.0010) [2023-10-14 19:52:22,260][61552] Updated weights for policy 0, policy_version 54092 (0.0009) [2023-10-14 19:52:22,627][61552] Updated weights for policy 0, policy_version 54102 (0.0011) [2023-10-14 19:52:22,998][61552] Updated weights for policy 0, policy_version 54112 (0.0009) [2023-10-14 19:52:23,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 110624768. Throughput: 0: 1683.0, 1: 1675.2. Samples: 27660914. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:52:23,344][60425] Avg episode reward: [(0, '76.790'), (1, '74.030')] [2023-10-14 19:52:24,720][61585] Updated weights for policy 1, policy_version 53930 (0.0007) [2023-10-14 19:52:25,081][61585] Updated weights for policy 1, policy_version 53940 (0.0007) [2023-10-14 19:52:25,455][61585] Updated weights for policy 1, policy_version 53950 (0.0007) [2023-10-14 19:52:27,071][61552] Updated weights for policy 0, policy_version 54122 (0.0007) [2023-10-14 19:52:27,432][61552] Updated weights for policy 0, policy_version 54132 (0.0010) [2023-10-14 19:52:27,798][61552] Updated weights for policy 0, policy_version 54142 (0.0009) [2023-10-14 19:52:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 110690304. Throughput: 0: 1655.7, 1: 1677.0. Samples: 27680522. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:52:28,344][60425] Avg episode reward: [(0, '73.660'), (1, '72.710')] [2023-10-14 19:52:29,549][61585] Updated weights for policy 1, policy_version 53960 (0.0009) [2023-10-14 19:52:29,918][61585] Updated weights for policy 1, policy_version 53970 (0.0009) [2023-10-14 19:52:30,280][61585] Updated weights for policy 1, policy_version 53980 (0.0008) [2023-10-14 19:52:32,088][61552] Updated weights for policy 0, policy_version 54152 (0.0009) [2023-10-14 19:52:32,455][61552] Updated weights for policy 0, policy_version 54162 (0.0010) [2023-10-14 19:52:32,821][61552] Updated weights for policy 0, policy_version 54172 (0.0008) [2023-10-14 19:52:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 110755840. Throughput: 0: 1675.1, 1: 1663.6. Samples: 27690578. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:52:33,344][60425] Avg episode reward: [(0, '77.070'), (1, '75.860')] [2023-10-14 19:52:34,401][61585] Updated weights for policy 1, policy_version 53990 (0.0007) [2023-10-14 19:52:34,764][61585] Updated weights for policy 1, policy_version 54000 (0.0010) [2023-10-14 19:52:35,132][61585] Updated weights for policy 1, policy_version 54010 (0.0008) [2023-10-14 19:52:36,909][61552] Updated weights for policy 0, policy_version 54182 (0.0008) [2023-10-14 19:52:37,278][61552] Updated weights for policy 0, policy_version 54192 (0.0007) [2023-10-14 19:52:37,640][61552] Updated weights for policy 0, policy_version 54202 (0.0009) [2023-10-14 19:52:38,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 110821376. Throughput: 0: 1672.4, 1: 1678.4. Samples: 27711114. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:52:38,344][60425] Avg episode reward: [(0, '76.430'), (1, '76.560')] [2023-10-14 19:52:39,091][61585] Updated weights for policy 1, policy_version 54020 (0.0009) [2023-10-14 19:52:39,468][61585] Updated weights for policy 1, policy_version 54030 (0.0007) [2023-10-14 19:52:39,836][61585] Updated weights for policy 1, policy_version 54040 (0.0008) [2023-10-14 19:52:41,835][61552] Updated weights for policy 0, policy_version 54212 (0.0011) [2023-10-14 19:52:42,205][61552] Updated weights for policy 0, policy_version 54222 (0.0008) [2023-10-14 19:52:42,568][61552] Updated weights for policy 0, policy_version 54232 (0.0007) [2023-10-14 19:52:43,344][60425] Fps is (10 sec: 13106.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 110886912. Throughput: 0: 1655.8, 1: 1680.1. Samples: 27730938. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:52:43,345][60425] Avg episode reward: [(0, '76.780'), (1, '74.920')] [2023-10-14 19:52:43,903][61585] Updated weights for policy 1, policy_version 54050 (0.0009) [2023-10-14 19:52:44,261][61585] Updated weights for policy 1, policy_version 54060 (0.0008) [2023-10-14 19:52:44,637][61585] Updated weights for policy 1, policy_version 54070 (0.0008) [2023-10-14 19:52:44,997][61585] Updated weights for policy 1, policy_version 54080 (0.0007) [2023-10-14 19:52:46,655][61552] Updated weights for policy 0, policy_version 54242 (0.0008) [2023-10-14 19:52:47,025][61552] Updated weights for policy 0, policy_version 54252 (0.0009) [2023-10-14 19:52:47,392][61552] Updated weights for policy 0, policy_version 54262 (0.0009) [2023-10-14 19:52:47,762][61552] Updated weights for policy 0, policy_version 54272 (0.0010) [2023-10-14 19:52:48,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 110952448. Throughput: 0: 1673.6, 1: 1676.1. Samples: 27741120. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:52:48,344][60425] Avg episode reward: [(0, '74.010'), (1, '77.230')] [2023-10-14 19:52:48,868][61585] Updated weights for policy 1, policy_version 54090 (0.0008) [2023-10-14 19:52:49,230][61585] Updated weights for policy 1, policy_version 54100 (0.0008) [2023-10-14 19:52:49,599][61585] Updated weights for policy 1, policy_version 54110 (0.0009) [2023-10-14 19:52:51,883][61552] Updated weights for policy 0, policy_version 54282 (0.0009) [2023-10-14 19:52:52,257][61552] Updated weights for policy 0, policy_version 54292 (0.0008) [2023-10-14 19:52:52,638][61552] Updated weights for policy 0, policy_version 54302 (0.0009) [2023-10-14 19:52:53,343][60425] Fps is (10 sec: 13107.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 111017984. Throughput: 0: 1664.3, 1: 1682.7. Samples: 27761320. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:52:53,344][60425] Avg episode reward: [(0, '75.920'), (1, '73.290')] [2023-10-14 19:52:53,655][61585] Updated weights for policy 1, policy_version 54120 (0.0008) [2023-10-14 19:52:54,033][61585] Updated weights for policy 1, policy_version 54130 (0.0009) [2023-10-14 19:52:54,397][61585] Updated weights for policy 1, policy_version 54140 (0.0011) [2023-10-14 19:52:56,819][61552] Updated weights for policy 0, policy_version 54312 (0.0009) [2023-10-14 19:52:57,192][61552] Updated weights for policy 0, policy_version 54322 (0.0010) [2023-10-14 19:52:57,559][61552] Updated weights for policy 0, policy_version 54332 (0.0008) [2023-10-14 19:52:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 111083520. Throughput: 0: 1652.0, 1: 1686.4. Samples: 27781078. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:52:58,344][60425] Avg episode reward: [(0, '75.680'), (1, '74.810')] [2023-10-14 19:52:58,521][61585] Updated weights for policy 1, policy_version 54150 (0.0008) [2023-10-14 19:52:58,897][61585] Updated weights for policy 1, policy_version 54160 (0.0007) [2023-10-14 19:52:59,255][61585] Updated weights for policy 1, policy_version 54170 (0.0008) [2023-10-14 19:53:01,483][61552] Updated weights for policy 0, policy_version 54342 (0.0008) [2023-10-14 19:53:01,845][61552] Updated weights for policy 0, policy_version 54352 (0.0008) [2023-10-14 19:53:02,207][61552] Updated weights for policy 0, policy_version 54362 (0.0010) [2023-10-14 19:53:03,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 111149056. Throughput: 0: 1668.8, 1: 1687.3. Samples: 27791506. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:53:03,344][60425] Avg episode reward: [(0, '75.980'), (1, '75.670')] [2023-10-14 19:53:03,345][61585] Updated weights for policy 1, policy_version 54180 (0.0008) [2023-10-14 19:53:03,714][61585] Updated weights for policy 1, policy_version 54190 (0.0007) [2023-10-14 19:53:04,081][61585] Updated weights for policy 1, policy_version 54200 (0.0011) [2023-10-14 19:53:06,347][61552] Updated weights for policy 0, policy_version 54372 (0.0008) [2023-10-14 19:53:06,721][61552] Updated weights for policy 0, policy_version 54382 (0.0009) [2023-10-14 19:53:07,078][61552] Updated weights for policy 0, policy_version 54392 (0.0008) [2023-10-14 19:53:08,194][61585] Updated weights for policy 1, policy_version 54210 (0.0008) [2023-10-14 19:53:08,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 111214592. Throughput: 0: 1659.2, 1: 1692.0. Samples: 27811716. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:53:08,344][60425] Avg episode reward: [(0, '79.110'), (1, '74.050')] [2023-10-14 19:53:08,559][61585] Updated weights for policy 1, policy_version 54220 (0.0007) [2023-10-14 19:53:08,924][61585] Updated weights for policy 1, policy_version 54230 (0.0010) [2023-10-14 19:53:09,288][61585] Updated weights for policy 1, policy_version 54240 (0.0010) [2023-10-14 19:53:11,074][61552] Updated weights for policy 0, policy_version 54402 (0.0008) [2023-10-14 19:53:11,438][61552] Updated weights for policy 0, policy_version 54412 (0.0010) [2023-10-14 19:53:11,812][61552] Updated weights for policy 0, policy_version 54422 (0.0007) [2023-10-14 19:53:12,175][61552] Updated weights for policy 0, policy_version 54432 (0.0008) [2023-10-14 19:53:13,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 111280128. Throughput: 0: 1675.4, 1: 1688.0. Samples: 27831876. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:53:13,344][60425] Avg episode reward: [(0, '81.940'), (1, '79.900')] [2023-10-14 19:53:13,353][61172] Saving new best policy, reward=81.940! [2023-10-14 19:53:13,549][61585] Updated weights for policy 1, policy_version 54250 (0.0008) [2023-10-14 19:53:13,905][61585] Updated weights for policy 1, policy_version 54260 (0.0009) [2023-10-14 19:53:14,276][61585] Updated weights for policy 1, policy_version 54270 (0.0009) [2023-10-14 19:53:16,175][61552] Updated weights for policy 0, policy_version 54442 (0.0009) [2023-10-14 19:53:16,537][61552] Updated weights for policy 0, policy_version 54452 (0.0008) [2023-10-14 19:53:16,908][61552] Updated weights for policy 0, policy_version 54462 (0.0008) [2023-10-14 19:53:18,289][61585] Updated weights for policy 1, policy_version 54280 (0.0008) [2023-10-14 19:53:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 111345664. Throughput: 0: 1685.4, 1: 1685.6. Samples: 27842270. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:53:18,344][60425] Avg episode reward: [(0, '77.670'), (1, '76.990')] [2023-10-14 19:53:18,656][61585] Updated weights for policy 1, policy_version 54290 (0.0007) [2023-10-14 19:53:19,017][61585] Updated weights for policy 1, policy_version 54300 (0.0008) [2023-10-14 19:53:20,908][61552] Updated weights for policy 0, policy_version 54472 (0.0009) [2023-10-14 19:53:21,274][61552] Updated weights for policy 0, policy_version 54482 (0.0010) [2023-10-14 19:53:21,650][61552] Updated weights for policy 0, policy_version 54492 (0.0009) [2023-10-14 19:53:23,033][61585] Updated weights for policy 1, policy_version 54310 (0.0009) [2023-10-14 19:53:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 111411200. Throughput: 0: 1663.7, 1: 1686.0. Samples: 27861850. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:53:23,344][60425] Avg episode reward: [(0, '78.750'), (1, '77.140')] [2023-10-14 19:53:23,399][61585] Updated weights for policy 1, policy_version 54320 (0.0008) [2023-10-14 19:53:23,757][61585] Updated weights for policy 1, policy_version 54330 (0.0009) [2023-10-14 19:53:25,740][61552] Updated weights for policy 0, policy_version 54502 (0.0007) [2023-10-14 19:53:26,116][61552] Updated weights for policy 0, policy_version 54512 (0.0007) [2023-10-14 19:53:26,483][61552] Updated weights for policy 0, policy_version 54522 (0.0009) [2023-10-14 19:53:28,038][61585] Updated weights for policy 1, policy_version 54340 (0.0007) [2023-10-14 19:53:28,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 111476736. Throughput: 0: 1685.1, 1: 1680.8. Samples: 27882404. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-14 19:53:28,344][60425] Avg episode reward: [(0, '79.740'), (1, '79.090')] [2023-10-14 19:53:28,408][61585] Updated weights for policy 1, policy_version 54350 (0.0009) [2023-10-14 19:53:28,760][61585] Updated weights for policy 1, policy_version 54360 (0.0010) [2023-10-14 19:53:30,556][61552] Updated weights for policy 0, policy_version 54532 (0.0008) [2023-10-14 19:53:30,925][61552] Updated weights for policy 0, policy_version 54542 (0.0009) [2023-10-14 19:53:31,287][61552] Updated weights for policy 0, policy_version 54552 (0.0009) [2023-10-14 19:53:32,770][61585] Updated weights for policy 1, policy_version 54370 (0.0009) [2023-10-14 19:53:33,134][61585] Updated weights for policy 1, policy_version 54380 (0.0007) [2023-10-14 19:53:33,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 111542272. Throughput: 0: 1681.3, 1: 1681.1. Samples: 27892428. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-14 19:53:33,344][60425] Avg episode reward: [(0, '80.710'), (1, '78.410')] [2023-10-14 19:53:33,500][61585] Updated weights for policy 1, policy_version 54390 (0.0009) [2023-10-14 19:53:33,857][61585] Updated weights for policy 1, policy_version 54400 (0.0010) [2023-10-14 19:53:35,494][61552] Updated weights for policy 0, policy_version 54562 (0.0010) [2023-10-14 19:53:35,862][61552] Updated weights for policy 0, policy_version 54572 (0.0009) [2023-10-14 19:53:36,230][61552] Updated weights for policy 0, policy_version 54582 (0.0007) [2023-10-14 19:53:36,601][61552] Updated weights for policy 0, policy_version 54592 (0.0007) [2023-10-14 19:53:38,005][61585] Updated weights for policy 1, policy_version 54410 (0.0011) [2023-10-14 19:53:38,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 111607808. Throughput: 0: 1667.7, 1: 1682.5. Samples: 27912080. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-14 19:53:38,344][60425] Avg episode reward: [(0, '78.570'), (1, '69.660')] [2023-10-14 19:53:38,374][61585] Updated weights for policy 1, policy_version 54420 (0.0009) [2023-10-14 19:53:38,747][61585] Updated weights for policy 1, policy_version 54430 (0.0009) [2023-10-14 19:53:40,658][61552] Updated weights for policy 0, policy_version 54602 (0.0007) [2023-10-14 19:53:41,023][61552] Updated weights for policy 0, policy_version 54612 (0.0010) [2023-10-14 19:53:41,386][61552] Updated weights for policy 0, policy_version 54622 (0.0010) [2023-10-14 19:53:42,744][61585] Updated weights for policy 1, policy_version 54440 (0.0009) [2023-10-14 19:53:43,106][61585] Updated weights for policy 1, policy_version 54450 (0.0007) [2023-10-14 19:53:43,344][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.5). Total num frames: 111673344. Throughput: 0: 1693.6, 1: 1667.9. Samples: 27932346. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-14 19:53:43,345][60425] Avg episode reward: [(0, '75.680'), (1, '73.660')] [2023-10-14 19:53:43,356][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000054624_55934976.pth... [2023-10-14 19:53:43,391][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000053056_54329344.pth [2023-10-14 19:53:43,397][61172] Saving a milestone ./train_atari/atari_roadrunner_APPO/checkpoint_p0/milestones/checkpoint_000054624_55934976.pth [2023-10-14 19:53:43,473][61585] Updated weights for policy 1, policy_version 54460 (0.0008) [2023-10-14 19:53:43,615][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000054464_55771136.pth... [2023-10-14 19:53:43,644][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000052896_54165504.pth [2023-10-14 19:53:43,647][61248] Saving a milestone ./train_atari/atari_roadrunner_APPO/checkpoint_p1/milestones/checkpoint_000054464_55771136.pth [2023-10-14 19:53:45,518][61552] Updated weights for policy 0, policy_version 54632 (0.0009) [2023-10-14 19:53:45,891][61552] Updated weights for policy 0, policy_version 54642 (0.0009) [2023-10-14 19:53:46,259][61552] Updated weights for policy 0, policy_version 54652 (0.0010) [2023-10-14 19:53:47,467][61585] Updated weights for policy 1, policy_version 54470 (0.0008) [2023-10-14 19:53:47,832][61585] Updated weights for policy 1, policy_version 54480 (0.0010) [2023-10-14 19:53:48,203][61585] Updated weights for policy 1, policy_version 54490 (0.0008) [2023-10-14 19:53:48,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 111738880. Throughput: 0: 1674.7, 1: 1676.3. Samples: 27942298. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-14 19:53:48,344][60425] Avg episode reward: [(0, '77.720'), (1, '73.490')] [2023-10-14 19:53:50,335][61552] Updated weights for policy 0, policy_version 54662 (0.0009) [2023-10-14 19:53:50,704][61552] Updated weights for policy 0, policy_version 54672 (0.0008) [2023-10-14 19:53:51,073][61552] Updated weights for policy 0, policy_version 54682 (0.0008) [2023-10-14 19:53:52,440][61585] Updated weights for policy 1, policy_version 54500 (0.0010) [2023-10-14 19:53:52,806][61585] Updated weights for policy 1, policy_version 54510 (0.0009) [2023-10-14 19:53:53,169][61585] Updated weights for policy 1, policy_version 54520 (0.0011) [2023-10-14 19:53:53,343][60425] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 111804416. Throughput: 0: 1667.2, 1: 1671.6. Samples: 27961960. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-14 19:53:53,344][60425] Avg episode reward: [(0, '78.990'), (1, '73.080')] [2023-10-14 19:53:55,173][61552] Updated weights for policy 0, policy_version 54692 (0.0009) [2023-10-14 19:53:55,536][61552] Updated weights for policy 0, policy_version 54702 (0.0011) [2023-10-14 19:53:55,910][61552] Updated weights for policy 0, policy_version 54712 (0.0010) [2023-10-14 19:53:57,151][61585] Updated weights for policy 1, policy_version 54530 (0.0009) [2023-10-14 19:53:57,523][61585] Updated weights for policy 1, policy_version 54540 (0.0007) [2023-10-14 19:53:57,890][61585] Updated weights for policy 1, policy_version 54550 (0.0008) [2023-10-14 19:53:58,250][61585] Updated weights for policy 1, policy_version 54560 (0.0008) [2023-10-14 19:53:58,344][60425] Fps is (10 sec: 16383.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 111902720. Throughput: 0: 1678.0, 1: 1661.6. Samples: 27982160. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-14 19:53:58,344][60425] Avg episode reward: [(0, '76.710'), (1, '74.940')] [2023-10-14 19:53:59,825][61552] Updated weights for policy 0, policy_version 54722 (0.0007) [2023-10-14 19:54:00,207][61552] Updated weights for policy 0, policy_version 54732 (0.0009) [2023-10-14 19:54:00,561][61552] Updated weights for policy 0, policy_version 54742 (0.0011) [2023-10-14 19:54:00,924][61552] Updated weights for policy 0, policy_version 54752 (0.0008) [2023-10-14 19:54:02,559][61585] Updated weights for policy 1, policy_version 54570 (0.0010) [2023-10-14 19:54:02,924][61585] Updated weights for policy 1, policy_version 54580 (0.0011) [2023-10-14 19:54:03,286][61585] Updated weights for policy 1, policy_version 54590 (0.0009) [2023-10-14 19:54:03,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 111935488. Throughput: 0: 1653.5, 1: 1675.1. Samples: 27992056. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-14 19:54:03,345][60425] Avg episode reward: [(0, '77.320'), (1, '72.350')] [2023-10-14 19:54:04,944][61552] Updated weights for policy 0, policy_version 54762 (0.0011) [2023-10-14 19:54:05,323][61552] Updated weights for policy 0, policy_version 54772 (0.0010) [2023-10-14 19:54:05,698][61552] Updated weights for policy 0, policy_version 54782 (0.0011) [2023-10-14 19:54:07,410][61585] Updated weights for policy 1, policy_version 54600 (0.0009) [2023-10-14 19:54:07,772][61585] Updated weights for policy 1, policy_version 54610 (0.0010) [2023-10-14 19:54:08,138][61585] Updated weights for policy 1, policy_version 54620 (0.0009) [2023-10-14 19:54:08,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 112033792. Throughput: 0: 1674.7, 1: 1671.1. Samples: 28012414. Policy #0 lag: (min: 25.0, avg: 29.1, max: 57.0) [2023-10-14 19:54:08,344][60425] Avg episode reward: [(0, '77.900'), (1, '77.000')] [2023-10-14 19:54:09,717][61552] Updated weights for policy 0, policy_version 54792 (0.0010) [2023-10-14 19:54:10,076][61552] Updated weights for policy 0, policy_version 54802 (0.0010) [2023-10-14 19:54:10,444][61552] Updated weights for policy 0, policy_version 54812 (0.0011) [2023-10-14 19:54:12,280][61585] Updated weights for policy 1, policy_version 54630 (0.0009) [2023-10-14 19:54:12,655][61585] Updated weights for policy 1, policy_version 54640 (0.0008) [2023-10-14 19:54:13,007][61585] Updated weights for policy 1, policy_version 54650 (0.0007) [2023-10-14 19:54:13,343][60425] Fps is (10 sec: 16384.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 112099328. Throughput: 0: 1676.1, 1: 1655.7. Samples: 28032336. Policy #0 lag: (min: 25.0, avg: 29.1, max: 57.0) [2023-10-14 19:54:13,344][60425] Avg episode reward: [(0, '73.190'), (1, '68.240')] [2023-10-14 19:54:14,556][61552] Updated weights for policy 0, policy_version 54822 (0.0011) [2023-10-14 19:54:14,922][61552] Updated weights for policy 0, policy_version 54832 (0.0010) [2023-10-14 19:54:15,292][61552] Updated weights for policy 0, policy_version 54842 (0.0011) [2023-10-14 19:54:17,156][61585] Updated weights for policy 1, policy_version 54660 (0.0007) [2023-10-14 19:54:17,524][61585] Updated weights for policy 1, policy_version 54670 (0.0008) [2023-10-14 19:54:17,897][61585] Updated weights for policy 1, policy_version 54680 (0.0007) [2023-10-14 19:54:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 112164864. Throughput: 0: 1655.7, 1: 1670.5. Samples: 28042108. Policy #0 lag: (min: 25.0, avg: 29.1, max: 57.0) [2023-10-14 19:54:18,344][60425] Avg episode reward: [(0, '72.900'), (1, '71.360')] [2023-10-14 19:54:19,399][61552] Updated weights for policy 0, policy_version 54852 (0.0009) [2023-10-14 19:54:19,765][61552] Updated weights for policy 0, policy_version 54862 (0.0009) [2023-10-14 19:54:20,133][61552] Updated weights for policy 0, policy_version 54872 (0.0009) [2023-10-14 19:54:21,955][61585] Updated weights for policy 1, policy_version 54690 (0.0008) [2023-10-14 19:54:22,323][61585] Updated weights for policy 1, policy_version 54700 (0.0009) [2023-10-14 19:54:22,683][61585] Updated weights for policy 1, policy_version 54710 (0.0008) [2023-10-14 19:54:23,041][61585] Updated weights for policy 1, policy_version 54720 (0.0009) [2023-10-14 19:54:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 112230400. Throughput: 0: 1673.5, 1: 1672.6. Samples: 28062652. Policy #0 lag: (min: 25.0, avg: 29.1, max: 57.0) [2023-10-14 19:54:23,344][60425] Avg episode reward: [(0, '73.590'), (1, '73.290')] [2023-10-14 19:54:24,176][61552] Updated weights for policy 0, policy_version 54882 (0.0010) [2023-10-14 19:54:24,551][61552] Updated weights for policy 0, policy_version 54892 (0.0010) [2023-10-14 19:54:24,919][61552] Updated weights for policy 0, policy_version 54902 (0.0008) [2023-10-14 19:54:25,299][61552] Updated weights for policy 0, policy_version 54912 (0.0007) [2023-10-14 19:54:27,083][61585] Updated weights for policy 1, policy_version 54730 (0.0007) [2023-10-14 19:54:27,451][61585] Updated weights for policy 1, policy_version 54740 (0.0009) [2023-10-14 19:54:27,828][61585] Updated weights for policy 1, policy_version 54750 (0.0010) [2023-10-14 19:54:28,344][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 112295936. Throughput: 0: 1678.0, 1: 1655.7. Samples: 28082364. Policy #0 lag: (min: 25.0, avg: 29.1, max: 57.0) [2023-10-14 19:54:28,345][60425] Avg episode reward: [(0, '70.290'), (1, '70.930')] [2023-10-14 19:54:29,433][61552] Updated weights for policy 0, policy_version 54922 (0.0008) [2023-10-14 19:54:29,804][61552] Updated weights for policy 0, policy_version 54932 (0.0008) [2023-10-14 19:54:30,177][61552] Updated weights for policy 0, policy_version 54942 (0.0009) [2023-10-14 19:54:31,871][61585] Updated weights for policy 1, policy_version 54760 (0.0008) [2023-10-14 19:54:32,250][61585] Updated weights for policy 1, policy_version 54770 (0.0009) [2023-10-14 19:54:32,617][61585] Updated weights for policy 1, policy_version 54780 (0.0008) [2023-10-14 19:54:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 112361472. Throughput: 0: 1666.3, 1: 1675.0. Samples: 28092656. Policy #0 lag: (min: 25.0, avg: 29.1, max: 57.0) [2023-10-14 19:54:33,344][60425] Avg episode reward: [(0, '69.930'), (1, '74.190')] [2023-10-14 19:54:34,337][61552] Updated weights for policy 0, policy_version 54952 (0.0007) [2023-10-14 19:54:34,723][61552] Updated weights for policy 0, policy_version 54962 (0.0008) [2023-10-14 19:54:35,087][61552] Updated weights for policy 0, policy_version 54972 (0.0008) [2023-10-14 19:54:36,584][61585] Updated weights for policy 1, policy_version 54790 (0.0008) [2023-10-14 19:54:36,953][61585] Updated weights for policy 1, policy_version 54800 (0.0009) [2023-10-14 19:54:37,326][61585] Updated weights for policy 1, policy_version 54810 (0.0009) [2023-10-14 19:54:38,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 112427008. Throughput: 0: 1680.8, 1: 1668.8. Samples: 28112694. Policy #0 lag: (min: 25.0, avg: 29.1, max: 57.0) [2023-10-14 19:54:38,344][60425] Avg episode reward: [(0, '69.690'), (1, '77.620')] [2023-10-14 19:54:39,118][61552] Updated weights for policy 0, policy_version 54982 (0.0010) [2023-10-14 19:54:39,484][61552] Updated weights for policy 0, policy_version 54992 (0.0007) [2023-10-14 19:54:39,857][61552] Updated weights for policy 0, policy_version 55002 (0.0007) [2023-10-14 19:54:41,493][61585] Updated weights for policy 1, policy_version 54820 (0.0008) [2023-10-14 19:54:41,863][61585] Updated weights for policy 1, policy_version 54830 (0.0007) [2023-10-14 19:54:42,226][61585] Updated weights for policy 1, policy_version 54840 (0.0007) [2023-10-14 19:54:43,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 112492544. Throughput: 0: 1685.2, 1: 1660.1. Samples: 28132694. Policy #0 lag: (min: 25.0, avg: 29.1, max: 57.0) [2023-10-14 19:54:43,344][60425] Avg episode reward: [(0, '75.670'), (1, '73.220')] [2023-10-14 19:54:44,046][61552] Updated weights for policy 0, policy_version 55012 (0.0009) [2023-10-14 19:54:44,405][61552] Updated weights for policy 0, policy_version 55022 (0.0010) [2023-10-14 19:54:44,767][61552] Updated weights for policy 0, policy_version 55032 (0.0009) [2023-10-14 19:54:46,092][61585] Updated weights for policy 1, policy_version 54850 (0.0008) [2023-10-14 19:54:46,455][61585] Updated weights for policy 1, policy_version 54860 (0.0008) [2023-10-14 19:54:46,824][61585] Updated weights for policy 1, policy_version 54870 (0.0009) [2023-10-14 19:54:47,200][61585] Updated weights for policy 1, policy_version 54880 (0.0009) [2023-10-14 19:54:48,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 112558080. Throughput: 0: 1676.0, 1: 1681.2. Samples: 28143128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:54:48,344][60425] Avg episode reward: [(0, '80.670'), (1, '74.800')] [2023-10-14 19:54:48,897][61552] Updated weights for policy 0, policy_version 55042 (0.0008) [2023-10-14 19:54:49,261][61552] Updated weights for policy 0, policy_version 55052 (0.0007) [2023-10-14 19:54:49,631][61552] Updated weights for policy 0, policy_version 55062 (0.0010) [2023-10-14 19:54:50,003][61552] Updated weights for policy 0, policy_version 55072 (0.0010) [2023-10-14 19:54:51,229][61585] Updated weights for policy 1, policy_version 54890 (0.0009) [2023-10-14 19:54:51,589][61585] Updated weights for policy 1, policy_version 54900 (0.0009) [2023-10-14 19:54:51,948][61585] Updated weights for policy 1, policy_version 54910 (0.0011) [2023-10-14 19:54:53,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 112623616. Throughput: 0: 1680.3, 1: 1664.0. Samples: 28162904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:54:53,344][60425] Avg episode reward: [(0, '75.960'), (1, '79.050')] [2023-10-14 19:54:54,116][61552] Updated weights for policy 0, policy_version 55082 (0.0009) [2023-10-14 19:54:54,487][61552] Updated weights for policy 0, policy_version 55092 (0.0007) [2023-10-14 19:54:54,860][61552] Updated weights for policy 0, policy_version 55102 (0.0011) [2023-10-14 19:54:56,234][61585] Updated weights for policy 1, policy_version 54920 (0.0010) [2023-10-14 19:54:56,596][61585] Updated weights for policy 1, policy_version 54930 (0.0008) [2023-10-14 19:54:56,962][61585] Updated weights for policy 1, policy_version 54940 (0.0008) [2023-10-14 19:54:58,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 112689152. Throughput: 0: 1679.0, 1: 1671.1. Samples: 28183090. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:54:58,345][60425] Avg episode reward: [(0, '77.120'), (1, '77.710')] [2023-10-14 19:54:58,972][61552] Updated weights for policy 0, policy_version 55112 (0.0009) [2023-10-14 19:54:59,345][61552] Updated weights for policy 0, policy_version 55122 (0.0008) [2023-10-14 19:54:59,709][61552] Updated weights for policy 0, policy_version 55132 (0.0008) [2023-10-14 19:55:00,948][61585] Updated weights for policy 1, policy_version 54950 (0.0008) [2023-10-14 19:55:01,315][61585] Updated weights for policy 1, policy_version 54960 (0.0008) [2023-10-14 19:55:01,681][61585] Updated weights for policy 1, policy_version 54970 (0.0008) [2023-10-14 19:55:03,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 112754688. Throughput: 0: 1678.2, 1: 1685.4. Samples: 28193470. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:55:03,344][60425] Avg episode reward: [(0, '74.450'), (1, '77.220')] [2023-10-14 19:55:03,786][61552] Updated weights for policy 0, policy_version 55142 (0.0008) [2023-10-14 19:55:04,146][61552] Updated weights for policy 0, policy_version 55152 (0.0009) [2023-10-14 19:55:04,522][61552] Updated weights for policy 0, policy_version 55162 (0.0009) [2023-10-14 19:55:05,725][61585] Updated weights for policy 1, policy_version 54980 (0.0008) [2023-10-14 19:55:06,092][61585] Updated weights for policy 1, policy_version 54990 (0.0008) [2023-10-14 19:55:06,447][61585] Updated weights for policy 1, policy_version 55000 (0.0008) [2023-10-14 19:55:08,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 112820224. Throughput: 0: 1686.2, 1: 1656.8. Samples: 28213088. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:55:08,344][60425] Avg episode reward: [(0, '75.560'), (1, '76.400')] [2023-10-14 19:55:08,480][61552] Updated weights for policy 0, policy_version 55172 (0.0007) [2023-10-14 19:55:08,848][61552] Updated weights for policy 0, policy_version 55182 (0.0007) [2023-10-14 19:55:09,212][61552] Updated weights for policy 0, policy_version 55192 (0.0010) [2023-10-14 19:55:10,440][61585] Updated weights for policy 1, policy_version 55010 (0.0009) [2023-10-14 19:55:10,814][61585] Updated weights for policy 1, policy_version 55020 (0.0009) [2023-10-14 19:55:11,176][61585] Updated weights for policy 1, policy_version 55030 (0.0009) [2023-10-14 19:55:11,544][61585] Updated weights for policy 1, policy_version 55040 (0.0010) [2023-10-14 19:55:13,181][61552] Updated weights for policy 0, policy_version 55202 (0.0010) [2023-10-14 19:55:13,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 112885760. Throughput: 0: 1683.5, 1: 1679.3. Samples: 28233690. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:55:13,344][60425] Avg episode reward: [(0, '76.180'), (1, '71.700')] [2023-10-14 19:55:13,553][61552] Updated weights for policy 0, policy_version 55212 (0.0007) [2023-10-14 19:55:13,927][61552] Updated weights for policy 0, policy_version 55222 (0.0007) [2023-10-14 19:55:14,293][61552] Updated weights for policy 0, policy_version 55232 (0.0007) [2023-10-14 19:55:15,650][61585] Updated weights for policy 1, policy_version 55050 (0.0009) [2023-10-14 19:55:16,014][61585] Updated weights for policy 1, policy_version 55060 (0.0007) [2023-10-14 19:55:16,369][61585] Updated weights for policy 1, policy_version 55070 (0.0007) [2023-10-14 19:55:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 112951296. Throughput: 0: 1685.9, 1: 1674.1. Samples: 28243856. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:55:18,344][60425] Avg episode reward: [(0, '79.160'), (1, '73.760')] [2023-10-14 19:55:18,470][61552] Updated weights for policy 0, policy_version 55242 (0.0007) [2023-10-14 19:55:18,842][61552] Updated weights for policy 0, policy_version 55252 (0.0007) [2023-10-14 19:55:19,212][61552] Updated weights for policy 0, policy_version 55262 (0.0007) [2023-10-14 19:55:20,523][61585] Updated weights for policy 1, policy_version 55080 (0.0009) [2023-10-14 19:55:20,887][61585] Updated weights for policy 1, policy_version 55090 (0.0007) [2023-10-14 19:55:21,247][61585] Updated weights for policy 1, policy_version 55100 (0.0008) [2023-10-14 19:55:23,339][61552] Updated weights for policy 0, policy_version 55272 (0.0007) [2023-10-14 19:55:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 113016832. Throughput: 0: 1689.7, 1: 1669.3. Samples: 28263846. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:55:23,344][60425] Avg episode reward: [(0, '78.320'), (1, '77.750')] [2023-10-14 19:55:23,706][61552] Updated weights for policy 0, policy_version 55282 (0.0008) [2023-10-14 19:55:24,071][61552] Updated weights for policy 0, policy_version 55292 (0.0009) [2023-10-14 19:55:25,528][61585] Updated weights for policy 1, policy_version 55110 (0.0009) [2023-10-14 19:55:25,919][61585] Updated weights for policy 1, policy_version 55120 (0.0007) [2023-10-14 19:55:26,293][61585] Updated weights for policy 1, policy_version 55130 (0.0012) [2023-10-14 19:55:28,138][61552] Updated weights for policy 0, policy_version 55302 (0.0008) [2023-10-14 19:55:28,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 113082368. Throughput: 0: 1681.1, 1: 1685.3. Samples: 28284180. Policy #0 lag: (min: 20.0, avg: 20.0, max: 22.0) [2023-10-14 19:55:28,344][60425] Avg episode reward: [(0, '78.520'), (1, '73.930')] [2023-10-14 19:55:28,502][61552] Updated weights for policy 0, policy_version 55312 (0.0009) [2023-10-14 19:55:28,871][61552] Updated weights for policy 0, policy_version 55322 (0.0007) [2023-10-14 19:55:30,376][61585] Updated weights for policy 1, policy_version 55140 (0.0009) [2023-10-14 19:55:30,744][61585] Updated weights for policy 1, policy_version 55150 (0.0008) [2023-10-14 19:55:31,105][61585] Updated weights for policy 1, policy_version 55160 (0.0008) [2023-10-14 19:55:32,882][61552] Updated weights for policy 0, policy_version 55332 (0.0007) [2023-10-14 19:55:33,260][61552] Updated weights for policy 0, policy_version 55342 (0.0009) [2023-10-14 19:55:33,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 113147904. Throughput: 0: 1685.5, 1: 1669.7. Samples: 28294112. Policy #0 lag: (min: 20.0, avg: 20.0, max: 22.0) [2023-10-14 19:55:33,344][60425] Avg episode reward: [(0, '72.560'), (1, '73.040')] [2023-10-14 19:55:33,621][61552] Updated weights for policy 0, policy_version 55352 (0.0009) [2023-10-14 19:55:35,326][61585] Updated weights for policy 1, policy_version 55170 (0.0008) [2023-10-14 19:55:35,693][61585] Updated weights for policy 1, policy_version 55180 (0.0010) [2023-10-14 19:55:36,056][61585] Updated weights for policy 1, policy_version 55190 (0.0008) [2023-10-14 19:55:36,415][61585] Updated weights for policy 1, policy_version 55200 (0.0008) [2023-10-14 19:55:37,839][61552] Updated weights for policy 0, policy_version 55362 (0.0008) [2023-10-14 19:55:38,208][61552] Updated weights for policy 0, policy_version 55372 (0.0008) [2023-10-14 19:55:38,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 113213440. Throughput: 0: 1680.4, 1: 1670.9. Samples: 28313714. Policy #0 lag: (min: 20.0, avg: 20.0, max: 22.0) [2023-10-14 19:55:38,344][60425] Avg episode reward: [(0, '80.170'), (1, '73.910')] [2023-10-14 19:55:38,580][61552] Updated weights for policy 0, policy_version 55382 (0.0007) [2023-10-14 19:55:38,942][61552] Updated weights for policy 0, policy_version 55392 (0.0010) [2023-10-14 19:55:40,366][61585] Updated weights for policy 1, policy_version 55210 (0.0009) [2023-10-14 19:55:40,731][61585] Updated weights for policy 1, policy_version 55220 (0.0007) [2023-10-14 19:55:41,087][61585] Updated weights for policy 1, policy_version 55230 (0.0009) [2023-10-14 19:55:43,046][61552] Updated weights for policy 0, policy_version 55402 (0.0012) [2023-10-14 19:55:43,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 113278976. Throughput: 0: 1678.1, 1: 1681.7. Samples: 28334282. Policy #0 lag: (min: 20.0, avg: 20.0, max: 22.0) [2023-10-14 19:55:43,344][60425] Avg episode reward: [(0, '77.290'), (1, '72.060')] [2023-10-14 19:55:43,352][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000055232_56557568.pth... [2023-10-14 19:55:43,386][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000053664_54951936.pth [2023-10-14 19:55:43,422][61552] Updated weights for policy 0, policy_version 55412 (0.0008) [2023-10-14 19:55:43,788][61552] Updated weights for policy 0, policy_version 55422 (0.0009) [2023-10-14 19:55:43,862][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000055424_56754176.pth... [2023-10-14 19:55:43,902][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000053856_55148544.pth [2023-10-14 19:55:45,226][61585] Updated weights for policy 1, policy_version 55240 (0.0009) [2023-10-14 19:55:45,584][61585] Updated weights for policy 1, policy_version 55250 (0.0008) [2023-10-14 19:55:45,940][61585] Updated weights for policy 1, policy_version 55260 (0.0009) [2023-10-14 19:55:47,926][61552] Updated weights for policy 0, policy_version 55432 (0.0008) [2023-10-14 19:55:48,296][61552] Updated weights for policy 0, policy_version 55442 (0.0009) [2023-10-14 19:55:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 113344512. Throughput: 0: 1680.8, 1: 1663.5. Samples: 28343962. Policy #0 lag: (min: 20.0, avg: 20.0, max: 22.0) [2023-10-14 19:55:48,344][60425] Avg episode reward: [(0, '76.210'), (1, '77.370')] [2023-10-14 19:55:48,661][61552] Updated weights for policy 0, policy_version 55452 (0.0009) [2023-10-14 19:55:50,047][61585] Updated weights for policy 1, policy_version 55270 (0.0009) [2023-10-14 19:55:50,415][61585] Updated weights for policy 1, policy_version 55280 (0.0011) [2023-10-14 19:55:50,774][61585] Updated weights for policy 1, policy_version 55290 (0.0010) [2023-10-14 19:55:52,686][61552] Updated weights for policy 0, policy_version 55462 (0.0009) [2023-10-14 19:55:53,053][61552] Updated weights for policy 0, policy_version 55472 (0.0009) [2023-10-14 19:55:53,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 113410048. Throughput: 0: 1674.9, 1: 1680.8. Samples: 28364092. Policy #0 lag: (min: 20.0, avg: 20.0, max: 22.0) [2023-10-14 19:55:53,344][60425] Avg episode reward: [(0, '74.650'), (1, '76.980')] [2023-10-14 19:55:53,425][61552] Updated weights for policy 0, policy_version 55482 (0.0007) [2023-10-14 19:55:54,875][61585] Updated weights for policy 1, policy_version 55300 (0.0009) [2023-10-14 19:55:55,241][61585] Updated weights for policy 1, policy_version 55310 (0.0010) [2023-10-14 19:55:55,601][61585] Updated weights for policy 1, policy_version 55320 (0.0007) [2023-10-14 19:55:57,439][61552] Updated weights for policy 0, policy_version 55492 (0.0007) [2023-10-14 19:55:57,810][61552] Updated weights for policy 0, policy_version 55502 (0.0008) [2023-10-14 19:55:58,174][61552] Updated weights for policy 0, policy_version 55512 (0.0008) [2023-10-14 19:55:58,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 113475584. Throughput: 0: 1664.8, 1: 1680.5. Samples: 28384228. Policy #0 lag: (min: 20.0, avg: 20.0, max: 22.0) [2023-10-14 19:55:58,345][60425] Avg episode reward: [(0, '77.420'), (1, '72.470')] [2023-10-14 19:55:59,595][61585] Updated weights for policy 1, policy_version 55330 (0.0008) [2023-10-14 19:55:59,955][61585] Updated weights for policy 1, policy_version 55340 (0.0008) [2023-10-14 19:56:00,319][61585] Updated weights for policy 1, policy_version 55350 (0.0007) [2023-10-14 19:56:00,682][61585] Updated weights for policy 1, policy_version 55360 (0.0009) [2023-10-14 19:56:02,400][61552] Updated weights for policy 0, policy_version 55522 (0.0010) [2023-10-14 19:56:02,759][61552] Updated weights for policy 0, policy_version 55532 (0.0009) [2023-10-14 19:56:03,140][61552] Updated weights for policy 0, policy_version 55542 (0.0011) [2023-10-14 19:56:03,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 113541120. Throughput: 0: 1672.6, 1: 1659.6. Samples: 28393804. Policy #0 lag: (min: 20.0, avg: 20.0, max: 22.0) [2023-10-14 19:56:03,344][60425] Avg episode reward: [(0, '74.200'), (1, '73.880')] [2023-10-14 19:56:03,503][61552] Updated weights for policy 0, policy_version 55552 (0.0010) [2023-10-14 19:56:04,779][61585] Updated weights for policy 1, policy_version 55370 (0.0010) [2023-10-14 19:56:05,151][61585] Updated weights for policy 1, policy_version 55380 (0.0008) [2023-10-14 19:56:05,517][61585] Updated weights for policy 1, policy_version 55390 (0.0009) [2023-10-14 19:56:07,717][61552] Updated weights for policy 0, policy_version 55562 (0.0008) [2023-10-14 19:56:08,075][61552] Updated weights for policy 0, policy_version 55572 (0.0010) [2023-10-14 19:56:08,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 113606656. Throughput: 0: 1669.9, 1: 1673.5. Samples: 28414300. Policy #0 lag: (min: 6.0, avg: 7.6, max: 34.0) [2023-10-14 19:56:08,344][60425] Avg episode reward: [(0, '75.230'), (1, '76.520')] [2023-10-14 19:56:08,460][61552] Updated weights for policy 0, policy_version 55582 (0.0010) [2023-10-14 19:56:09,617][61585] Updated weights for policy 1, policy_version 55400 (0.0009) [2023-10-14 19:56:09,984][61585] Updated weights for policy 1, policy_version 55410 (0.0008) [2023-10-14 19:56:10,348][61585] Updated weights for policy 1, policy_version 55420 (0.0007) [2023-10-14 19:56:12,512][61552] Updated weights for policy 0, policy_version 55592 (0.0008) [2023-10-14 19:56:12,882][61552] Updated weights for policy 0, policy_version 55602 (0.0007) [2023-10-14 19:56:13,256][61552] Updated weights for policy 0, policy_version 55612 (0.0007) [2023-10-14 19:56:13,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 113672192. Throughput: 0: 1658.9, 1: 1681.4. Samples: 28434494. Policy #0 lag: (min: 6.0, avg: 7.6, max: 34.0) [2023-10-14 19:56:13,344][60425] Avg episode reward: [(0, '74.430'), (1, '78.450')] [2023-10-14 19:56:14,566][61585] Updated weights for policy 1, policy_version 55430 (0.0009) [2023-10-14 19:56:14,953][61585] Updated weights for policy 1, policy_version 55440 (0.0009) [2023-10-14 19:56:15,312][61585] Updated weights for policy 1, policy_version 55450 (0.0008) [2023-10-14 19:56:17,281][61552] Updated weights for policy 0, policy_version 55622 (0.0007) [2023-10-14 19:56:17,649][61552] Updated weights for policy 0, policy_version 55632 (0.0008) [2023-10-14 19:56:18,016][61552] Updated weights for policy 0, policy_version 55642 (0.0008) [2023-10-14 19:56:18,343][60425] Fps is (10 sec: 16384.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 113770496. Throughput: 0: 1669.5, 1: 1662.5. Samples: 28444054. Policy #0 lag: (min: 6.0, avg: 7.6, max: 34.0) [2023-10-14 19:56:18,344][60425] Avg episode reward: [(0, '74.590'), (1, '75.280')] [2023-10-14 19:56:19,251][61585] Updated weights for policy 1, policy_version 55460 (0.0008) [2023-10-14 19:56:19,617][61585] Updated weights for policy 1, policy_version 55470 (0.0008) [2023-10-14 19:56:19,983][61585] Updated weights for policy 1, policy_version 55480 (0.0010) [2023-10-14 19:56:22,033][61552] Updated weights for policy 0, policy_version 55652 (0.0010) [2023-10-14 19:56:22,403][61552] Updated weights for policy 0, policy_version 55662 (0.0008) [2023-10-14 19:56:22,765][61552] Updated weights for policy 0, policy_version 55672 (0.0009) [2023-10-14 19:56:23,343][60425] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13440.5). Total num frames: 113836032. Throughput: 0: 1671.5, 1: 1682.9. Samples: 28464664. Policy #0 lag: (min: 6.0, avg: 7.6, max: 34.0) [2023-10-14 19:56:23,344][60425] Avg episode reward: [(0, '74.780'), (1, '72.670')] [2023-10-14 19:56:24,015][61585] Updated weights for policy 1, policy_version 55490 (0.0010) [2023-10-14 19:56:24,370][61585] Updated weights for policy 1, policy_version 55500 (0.0008) [2023-10-14 19:56:24,746][61585] Updated weights for policy 1, policy_version 55510 (0.0009) [2023-10-14 19:56:25,108][61585] Updated weights for policy 1, policy_version 55520 (0.0008) [2023-10-14 19:56:26,782][61552] Updated weights for policy 0, policy_version 55682 (0.0009) [2023-10-14 19:56:27,144][61552] Updated weights for policy 0, policy_version 55692 (0.0008) [2023-10-14 19:56:27,509][61552] Updated weights for policy 0, policy_version 55702 (0.0009) [2023-10-14 19:56:27,878][61552] Updated weights for policy 0, policy_version 55712 (0.0008) [2023-10-14 19:56:28,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 113901568. Throughput: 0: 1651.1, 1: 1684.0. Samples: 28484360. Policy #0 lag: (min: 6.0, avg: 7.6, max: 34.0) [2023-10-14 19:56:28,344][60425] Avg episode reward: [(0, '71.960'), (1, '76.160')] [2023-10-14 19:56:29,230][61585] Updated weights for policy 1, policy_version 55530 (0.0010) [2023-10-14 19:56:29,593][61585] Updated weights for policy 1, policy_version 55540 (0.0008) [2023-10-14 19:56:29,955][61585] Updated weights for policy 1, policy_version 55550 (0.0007) [2023-10-14 19:56:32,147][61552] Updated weights for policy 0, policy_version 55722 (0.0007) [2023-10-14 19:56:32,522][61552] Updated weights for policy 0, policy_version 55732 (0.0009) [2023-10-14 19:56:32,888][61552] Updated weights for policy 0, policy_version 55742 (0.0008) [2023-10-14 19:56:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 113967104. Throughput: 0: 1670.7, 1: 1678.9. Samples: 28494692. Policy #0 lag: (min: 6.0, avg: 7.6, max: 34.0) [2023-10-14 19:56:33,344][60425] Avg episode reward: [(0, '74.590'), (1, '73.480')] [2023-10-14 19:56:33,911][61585] Updated weights for policy 1, policy_version 55560 (0.0007) [2023-10-14 19:56:34,280][61585] Updated weights for policy 1, policy_version 55570 (0.0007) [2023-10-14 19:56:34,640][61585] Updated weights for policy 1, policy_version 55580 (0.0008) [2023-10-14 19:56:36,874][61552] Updated weights for policy 0, policy_version 55752 (0.0009) [2023-10-14 19:56:37,247][61552] Updated weights for policy 0, policy_version 55762 (0.0007) [2023-10-14 19:56:37,616][61552] Updated weights for policy 0, policy_version 55772 (0.0010) [2023-10-14 19:56:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 114032640. Throughput: 0: 1672.8, 1: 1689.5. Samples: 28515398. Policy #0 lag: (min: 6.0, avg: 7.6, max: 34.0) [2023-10-14 19:56:38,344][60425] Avg episode reward: [(0, '74.530'), (1, '74.370')] [2023-10-14 19:56:38,539][61585] Updated weights for policy 1, policy_version 55590 (0.0009) [2023-10-14 19:56:38,913][61585] Updated weights for policy 1, policy_version 55600 (0.0008) [2023-10-14 19:56:39,278][61585] Updated weights for policy 1, policy_version 55610 (0.0008) [2023-10-14 19:56:41,715][61552] Updated weights for policy 0, policy_version 55782 (0.0008) [2023-10-14 19:56:42,077][61552] Updated weights for policy 0, policy_version 55792 (0.0008) [2023-10-14 19:56:42,449][61552] Updated weights for policy 0, policy_version 55802 (0.0008) [2023-10-14 19:56:43,233][61585] Updated weights for policy 1, policy_version 55620 (0.0010) [2023-10-14 19:56:43,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 114098176. Throughput: 0: 1659.6, 1: 1700.2. Samples: 28535420. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-14 19:56:43,344][60425] Avg episode reward: [(0, '74.900'), (1, '74.220')] [2023-10-14 19:56:43,604][61585] Updated weights for policy 1, policy_version 55630 (0.0009) [2023-10-14 19:56:43,966][61585] Updated weights for policy 1, policy_version 55640 (0.0008) [2023-10-14 19:56:46,417][61552] Updated weights for policy 0, policy_version 55812 (0.0008) [2023-10-14 19:56:46,785][61552] Updated weights for policy 0, policy_version 55822 (0.0010) [2023-10-14 19:56:47,157][61552] Updated weights for policy 0, policy_version 55832 (0.0011) [2023-10-14 19:56:48,173][61585] Updated weights for policy 1, policy_version 55650 (0.0009) [2023-10-14 19:56:48,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 114163712. Throughput: 0: 1681.3, 1: 1698.6. Samples: 28545900. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-14 19:56:48,344][60425] Avg episode reward: [(0, '72.960'), (1, '78.030')] [2023-10-14 19:56:48,532][61585] Updated weights for policy 1, policy_version 55660 (0.0010) [2023-10-14 19:56:48,905][61585] Updated weights for policy 1, policy_version 55670 (0.0008) [2023-10-14 19:56:49,276][61585] Updated weights for policy 1, policy_version 55680 (0.0008) [2023-10-14 19:56:51,199][61552] Updated weights for policy 0, policy_version 55842 (0.0008) [2023-10-14 19:56:51,562][61552] Updated weights for policy 0, policy_version 55852 (0.0008) [2023-10-14 19:56:51,938][61552] Updated weights for policy 0, policy_version 55862 (0.0008) [2023-10-14 19:56:52,306][61552] Updated weights for policy 0, policy_version 55872 (0.0009) [2023-10-14 19:56:53,317][61585] Updated weights for policy 1, policy_version 55690 (0.0007) [2023-10-14 19:56:53,344][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 114229248. Throughput: 0: 1667.6, 1: 1697.5. Samples: 28565728. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-14 19:56:53,345][60425] Avg episode reward: [(0, '78.230'), (1, '77.550')] [2023-10-14 19:56:53,696][61585] Updated weights for policy 1, policy_version 55700 (0.0008) [2023-10-14 19:56:54,060][61585] Updated weights for policy 1, policy_version 55710 (0.0008) [2023-10-14 19:56:56,439][61552] Updated weights for policy 0, policy_version 55882 (0.0007) [2023-10-14 19:56:56,803][61552] Updated weights for policy 0, policy_version 55892 (0.0009) [2023-10-14 19:56:57,168][61552] Updated weights for policy 0, policy_version 55902 (0.0009) [2023-10-14 19:56:58,048][61585] Updated weights for policy 1, policy_version 55720 (0.0008) [2023-10-14 19:56:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 114294784. Throughput: 0: 1664.8, 1: 1698.8. Samples: 28585854. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-14 19:56:58,344][60425] Avg episode reward: [(0, '74.380'), (1, '78.740')] [2023-10-14 19:56:58,413][61585] Updated weights for policy 1, policy_version 55730 (0.0007) [2023-10-14 19:56:58,779][61585] Updated weights for policy 1, policy_version 55740 (0.0008) [2023-10-14 19:57:01,275][61552] Updated weights for policy 0, policy_version 55912 (0.0009) [2023-10-14 19:57:01,646][61552] Updated weights for policy 0, policy_version 55922 (0.0010) [2023-10-14 19:57:02,017][61552] Updated weights for policy 0, policy_version 55932 (0.0010) [2023-10-14 19:57:02,923][61585] Updated weights for policy 1, policy_version 55750 (0.0008) [2023-10-14 19:57:03,310][61585] Updated weights for policy 1, policy_version 55760 (0.0007) [2023-10-14 19:57:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 114360320. Throughput: 0: 1680.5, 1: 1701.8. Samples: 28596256. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-14 19:57:03,345][60425] Avg episode reward: [(0, '72.860'), (1, '73.900')] [2023-10-14 19:57:03,680][61585] Updated weights for policy 1, policy_version 55770 (0.0007) [2023-10-14 19:57:06,151][61552] Updated weights for policy 0, policy_version 55942 (0.0009) [2023-10-14 19:57:06,520][61552] Updated weights for policy 0, policy_version 55952 (0.0009) [2023-10-14 19:57:06,892][61552] Updated weights for policy 0, policy_version 55962 (0.0010) [2023-10-14 19:57:07,737][61585] Updated weights for policy 1, policy_version 55780 (0.0008) [2023-10-14 19:57:08,095][61585] Updated weights for policy 1, policy_version 55790 (0.0007) [2023-10-14 19:57:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 114425856. Throughput: 0: 1664.4, 1: 1697.4. Samples: 28615948. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-14 19:57:08,344][60425] Avg episode reward: [(0, '77.150'), (1, '75.930')] [2023-10-14 19:57:08,446][61585] Updated weights for policy 1, policy_version 55800 (0.0007) [2023-10-14 19:57:10,766][61552] Updated weights for policy 0, policy_version 55972 (0.0010) [2023-10-14 19:57:11,141][61552] Updated weights for policy 0, policy_version 55982 (0.0007) [2023-10-14 19:57:11,504][61552] Updated weights for policy 0, policy_version 55992 (0.0008) [2023-10-14 19:57:12,508][61585] Updated weights for policy 1, policy_version 55810 (0.0008) [2023-10-14 19:57:12,875][61585] Updated weights for policy 1, policy_version 55820 (0.0010) [2023-10-14 19:57:13,233][61585] Updated weights for policy 1, policy_version 55830 (0.0008) [2023-10-14 19:57:13,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 114491392. Throughput: 0: 1677.3, 1: 1689.3. Samples: 28635852. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-14 19:57:13,344][60425] Avg episode reward: [(0, '75.140'), (1, '76.890')] [2023-10-14 19:57:13,593][61585] Updated weights for policy 1, policy_version 55840 (0.0010) [2023-10-14 19:57:15,695][61552] Updated weights for policy 0, policy_version 56002 (0.0008) [2023-10-14 19:57:16,054][61552] Updated weights for policy 0, policy_version 56012 (0.0008) [2023-10-14 19:57:16,416][61552] Updated weights for policy 0, policy_version 56022 (0.0007) [2023-10-14 19:57:16,787][61552] Updated weights for policy 0, policy_version 56032 (0.0007) [2023-10-14 19:57:17,611][61585] Updated weights for policy 1, policy_version 55850 (0.0010) [2023-10-14 19:57:17,978][61585] Updated weights for policy 1, policy_version 55860 (0.0007) [2023-10-14 19:57:18,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 114556928. Throughput: 0: 1681.9, 1: 1691.7. Samples: 28646504. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-14 19:57:18,344][60425] Avg episode reward: [(0, '75.160'), (1, '75.310')] [2023-10-14 19:57:18,347][61585] Updated weights for policy 1, policy_version 55870 (0.0008) [2023-10-14 19:57:20,871][61552] Updated weights for policy 0, policy_version 56042 (0.0007) [2023-10-14 19:57:21,242][61552] Updated weights for policy 0, policy_version 56052 (0.0009) [2023-10-14 19:57:21,615][61552] Updated weights for policy 0, policy_version 56062 (0.0009) [2023-10-14 19:57:22,622][61585] Updated weights for policy 1, policy_version 55880 (0.0008) [2023-10-14 19:57:22,987][61585] Updated weights for policy 1, policy_version 55890 (0.0010) [2023-10-14 19:57:23,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 114622464. Throughput: 0: 1656.0, 1: 1688.1. Samples: 28665884. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-10-14 19:57:23,344][60425] Avg episode reward: [(0, '73.380'), (1, '76.600')] [2023-10-14 19:57:23,359][61585] Updated weights for policy 1, policy_version 55900 (0.0011) [2023-10-14 19:57:25,836][61552] Updated weights for policy 0, policy_version 56072 (0.0010) [2023-10-14 19:57:26,209][61552] Updated weights for policy 0, policy_version 56082 (0.0010) [2023-10-14 19:57:26,568][61552] Updated weights for policy 0, policy_version 56092 (0.0010) [2023-10-14 19:57:27,490][61585] Updated weights for policy 1, policy_version 55910 (0.0011) [2023-10-14 19:57:27,860][61585] Updated weights for policy 1, policy_version 55920 (0.0008) [2023-10-14 19:57:28,228][61585] Updated weights for policy 1, policy_version 55930 (0.0008) [2023-10-14 19:57:28,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 114688000. Throughput: 0: 1677.2, 1: 1668.6. Samples: 28685980. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-10-14 19:57:28,344][60425] Avg episode reward: [(0, '75.010'), (1, '71.680')] [2023-10-14 19:57:30,503][61552] Updated weights for policy 0, policy_version 56102 (0.0009) [2023-10-14 19:57:30,881][61552] Updated weights for policy 0, policy_version 56112 (0.0010) [2023-10-14 19:57:31,254][61552] Updated weights for policy 0, policy_version 56122 (0.0010) [2023-10-14 19:57:32,254][61585] Updated weights for policy 1, policy_version 55940 (0.0009) [2023-10-14 19:57:32,625][61585] Updated weights for policy 1, policy_version 55950 (0.0008) [2023-10-14 19:57:32,992][61585] Updated weights for policy 1, policy_version 55960 (0.0007) [2023-10-14 19:57:33,343][60425] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 114786304. Throughput: 0: 1664.3, 1: 1679.8. Samples: 28696382. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-10-14 19:57:33,344][60425] Avg episode reward: [(0, '75.570'), (1, '74.970')] [2023-10-14 19:57:35,482][61552] Updated weights for policy 0, policy_version 56132 (0.0008) [2023-10-14 19:57:35,853][61552] Updated weights for policy 0, policy_version 56142 (0.0007) [2023-10-14 19:57:36,219][61552] Updated weights for policy 0, policy_version 56152 (0.0011) [2023-10-14 19:57:37,112][61585] Updated weights for policy 1, policy_version 55970 (0.0009) [2023-10-14 19:57:37,477][61585] Updated weights for policy 1, policy_version 55980 (0.0011) [2023-10-14 19:57:37,847][61585] Updated weights for policy 1, policy_version 55990 (0.0009) [2023-10-14 19:57:38,209][61585] Updated weights for policy 1, policy_version 56000 (0.0007) [2023-10-14 19:57:38,343][60425] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 114851840. Throughput: 0: 1661.6, 1: 1682.4. Samples: 28716208. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-10-14 19:57:38,344][60425] Avg episode reward: [(0, '78.380'), (1, '72.310')] [2023-10-14 19:57:40,243][61552] Updated weights for policy 0, policy_version 56162 (0.0008) [2023-10-14 19:57:40,648][61552] Updated weights for policy 0, policy_version 56172 (0.0007) [2023-10-14 19:57:41,010][61552] Updated weights for policy 0, policy_version 56182 (0.0008) [2023-10-14 19:57:41,377][61552] Updated weights for policy 0, policy_version 56192 (0.0010) [2023-10-14 19:57:42,230][61585] Updated weights for policy 1, policy_version 56010 (0.0008) [2023-10-14 19:57:42,600][61585] Updated weights for policy 1, policy_version 56020 (0.0009) [2023-10-14 19:57:42,963][61585] Updated weights for policy 1, policy_version 56030 (0.0009) [2023-10-14 19:57:43,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 114917376. Throughput: 0: 1677.6, 1: 1659.5. Samples: 28736022. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-10-14 19:57:43,344][60425] Avg episode reward: [(0, '75.840'), (1, '76.960')] [2023-10-14 19:57:43,356][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000056192_57540608.pth... [2023-10-14 19:57:43,356][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000056032_57376768.pth... [2023-10-14 19:57:43,392][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000054624_55934976.pth [2023-10-14 19:57:43,392][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000054464_55771136.pth [2023-10-14 19:57:45,410][61552] Updated weights for policy 0, policy_version 56202 (0.0007) [2023-10-14 19:57:45,788][61552] Updated weights for policy 0, policy_version 56212 (0.0007) [2023-10-14 19:57:46,154][61552] Updated weights for policy 0, policy_version 56222 (0.0008) [2023-10-14 19:57:47,111][61585] Updated weights for policy 1, policy_version 56040 (0.0007) [2023-10-14 19:57:47,467][61585] Updated weights for policy 1, policy_version 56050 (0.0010) [2023-10-14 19:57:47,827][61585] Updated weights for policy 1, policy_version 56060 (0.0009) [2023-10-14 19:57:48,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13653.2, 300 sec: 13440.4). Total num frames: 114982912. Throughput: 0: 1662.1, 1: 1682.2. Samples: 28746752. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-10-14 19:57:48,344][60425] Avg episode reward: [(0, '75.450'), (1, '76.100')] [2023-10-14 19:57:50,121][61552] Updated weights for policy 0, policy_version 56232 (0.0008) [2023-10-14 19:57:50,494][61552] Updated weights for policy 0, policy_version 56242 (0.0008) [2023-10-14 19:57:50,858][61552] Updated weights for policy 0, policy_version 56252 (0.0008) [2023-10-14 19:57:52,086][61585] Updated weights for policy 1, policy_version 56070 (0.0009) [2023-10-14 19:57:52,475][61585] Updated weights for policy 1, policy_version 56080 (0.0009) [2023-10-14 19:57:52,848][61585] Updated weights for policy 1, policy_version 56090 (0.0012) [2023-10-14 19:57:53,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 115048448. Throughput: 0: 1665.6, 1: 1684.4. Samples: 28766696. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-10-14 19:57:53,344][60425] Avg episode reward: [(0, '73.470'), (1, '74.250')] [2023-10-14 19:57:54,937][61552] Updated weights for policy 0, policy_version 56262 (0.0009) [2023-10-14 19:57:55,312][61552] Updated weights for policy 0, policy_version 56272 (0.0008) [2023-10-14 19:57:55,685][61552] Updated weights for policy 0, policy_version 56282 (0.0009) [2023-10-14 19:57:56,847][61585] Updated weights for policy 1, policy_version 56100 (0.0007) [2023-10-14 19:57:57,226][61585] Updated weights for policy 1, policy_version 56110 (0.0008) [2023-10-14 19:57:57,586][61585] Updated weights for policy 1, policy_version 56120 (0.0008) [2023-10-14 19:57:58,343][60425] Fps is (10 sec: 13107.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 115113984. Throughput: 0: 1681.6, 1: 1659.1. Samples: 28786180. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-10-14 19:57:58,344][60425] Avg episode reward: [(0, '76.840'), (1, '75.420')] [2023-10-14 19:57:59,652][61552] Updated weights for policy 0, policy_version 56292 (0.0008) [2023-10-14 19:58:00,036][61552] Updated weights for policy 0, policy_version 56302 (0.0009) [2023-10-14 19:58:00,414][61552] Updated weights for policy 0, policy_version 56312 (0.0009) [2023-10-14 19:58:01,672][61585] Updated weights for policy 1, policy_version 56130 (0.0008) [2023-10-14 19:58:02,043][61585] Updated weights for policy 1, policy_version 56140 (0.0011) [2023-10-14 19:58:02,404][61585] Updated weights for policy 1, policy_version 56150 (0.0010) [2023-10-14 19:58:02,769][61585] Updated weights for policy 1, policy_version 56160 (0.0010) [2023-10-14 19:58:03,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 115179520. Throughput: 0: 1659.0, 1: 1675.9. Samples: 28796574. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:58:03,344][60425] Avg episode reward: [(0, '76.510'), (1, '76.070')] [2023-10-14 19:58:04,370][61552] Updated weights for policy 0, policy_version 56322 (0.0008) [2023-10-14 19:58:04,742][61552] Updated weights for policy 0, policy_version 56332 (0.0008) [2023-10-14 19:58:05,118][61552] Updated weights for policy 0, policy_version 56342 (0.0009) [2023-10-14 19:58:05,479][61552] Updated weights for policy 0, policy_version 56352 (0.0007) [2023-10-14 19:58:06,798][61585] Updated weights for policy 1, policy_version 56170 (0.0008) [2023-10-14 19:58:07,163][61585] Updated weights for policy 1, policy_version 56180 (0.0008) [2023-10-14 19:58:07,530][61585] Updated weights for policy 1, policy_version 56190 (0.0008) [2023-10-14 19:58:08,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 115245056. Throughput: 0: 1689.0, 1: 1669.8. Samples: 28817032. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:58:08,345][60425] Avg episode reward: [(0, '78.430'), (1, '76.410')] [2023-10-14 19:58:09,530][61552] Updated weights for policy 0, policy_version 56362 (0.0008) [2023-10-14 19:58:09,896][61552] Updated weights for policy 0, policy_version 56372 (0.0010) [2023-10-14 19:58:10,257][61552] Updated weights for policy 0, policy_version 56382 (0.0007) [2023-10-14 19:58:11,435][61585] Updated weights for policy 1, policy_version 56200 (0.0009) [2023-10-14 19:58:11,804][61585] Updated weights for policy 1, policy_version 56210 (0.0008) [2023-10-14 19:58:12,160][61585] Updated weights for policy 1, policy_version 56220 (0.0007) [2023-10-14 19:58:13,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 115310592. Throughput: 0: 1690.2, 1: 1661.5. Samples: 28836806. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:58:13,344][60425] Avg episode reward: [(0, '74.800'), (1, '76.460')] [2023-10-14 19:58:14,387][61552] Updated weights for policy 0, policy_version 56392 (0.0010) [2023-10-14 19:58:14,756][61552] Updated weights for policy 0, policy_version 56402 (0.0009) [2023-10-14 19:58:15,112][61552] Updated weights for policy 0, policy_version 56412 (0.0007) [2023-10-14 19:58:16,230][61585] Updated weights for policy 1, policy_version 56230 (0.0009) [2023-10-14 19:58:16,595][61585] Updated weights for policy 1, policy_version 56240 (0.0007) [2023-10-14 19:58:16,958][61585] Updated weights for policy 1, policy_version 56250 (0.0007) [2023-10-14 19:58:18,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 115376128. Throughput: 0: 1668.8, 1: 1680.7. Samples: 28847110. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:58:18,344][60425] Avg episode reward: [(0, '72.430'), (1, '75.670')] [2023-10-14 19:58:19,290][61552] Updated weights for policy 0, policy_version 56422 (0.0008) [2023-10-14 19:58:19,665][61552] Updated weights for policy 0, policy_version 56432 (0.0008) [2023-10-14 19:58:20,039][61552] Updated weights for policy 0, policy_version 56442 (0.0009) [2023-10-14 19:58:21,005][61585] Updated weights for policy 1, policy_version 56260 (0.0008) [2023-10-14 19:58:21,367][61585] Updated weights for policy 1, policy_version 56270 (0.0009) [2023-10-14 19:58:21,728][61585] Updated weights for policy 1, policy_version 56280 (0.0011) [2023-10-14 19:58:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 115441664. Throughput: 0: 1681.8, 1: 1661.9. Samples: 28866676. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:58:23,344][60425] Avg episode reward: [(0, '73.990'), (1, '75.310')] [2023-10-14 19:58:24,287][61552] Updated weights for policy 0, policy_version 56452 (0.0011) [2023-10-14 19:58:24,656][61552] Updated weights for policy 0, policy_version 56462 (0.0008) [2023-10-14 19:58:25,039][61552] Updated weights for policy 0, policy_version 56472 (0.0009) [2023-10-14 19:58:25,839][61585] Updated weights for policy 1, policy_version 56290 (0.0010) [2023-10-14 19:58:26,206][61585] Updated weights for policy 1, policy_version 56300 (0.0008) [2023-10-14 19:58:26,579][61585] Updated weights for policy 1, policy_version 56310 (0.0009) [2023-10-14 19:58:26,939][61585] Updated weights for policy 1, policy_version 56320 (0.0010) [2023-10-14 19:58:28,344][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 115507200. Throughput: 0: 1682.6, 1: 1669.8. Samples: 28886880. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:58:28,345][60425] Avg episode reward: [(0, '77.250'), (1, '75.700')] [2023-10-14 19:58:29,304][61552] Updated weights for policy 0, policy_version 56482 (0.0009) [2023-10-14 19:58:29,711][61552] Updated weights for policy 0, policy_version 56492 (0.0010) [2023-10-14 19:58:30,086][61552] Updated weights for policy 0, policy_version 56502 (0.0011) [2023-10-14 19:58:30,457][61552] Updated weights for policy 0, policy_version 56512 (0.0009) [2023-10-14 19:58:31,028][61585] Updated weights for policy 1, policy_version 56330 (0.0010) [2023-10-14 19:58:31,395][61585] Updated weights for policy 1, policy_version 56340 (0.0009) [2023-10-14 19:58:31,759][61585] Updated weights for policy 1, policy_version 56350 (0.0011) [2023-10-14 19:58:33,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 115572736. Throughput: 0: 1664.9, 1: 1672.0. Samples: 28896912. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:58:33,344][60425] Avg episode reward: [(0, '75.880'), (1, '75.050')] [2023-10-14 19:58:34,475][61552] Updated weights for policy 0, policy_version 56522 (0.0008) [2023-10-14 19:58:34,843][61552] Updated weights for policy 0, policy_version 56532 (0.0007) [2023-10-14 19:58:35,206][61552] Updated weights for policy 0, policy_version 56542 (0.0007) [2023-10-14 19:58:35,931][61585] Updated weights for policy 1, policy_version 56360 (0.0008) [2023-10-14 19:58:36,300][61585] Updated weights for policy 1, policy_version 56370 (0.0008) [2023-10-14 19:58:36,662][61585] Updated weights for policy 1, policy_version 56380 (0.0009) [2023-10-14 19:58:38,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 115638272. Throughput: 0: 1680.9, 1: 1650.2. Samples: 28916596. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 19:58:38,344][60425] Avg episode reward: [(0, '72.790'), (1, '73.740')] [2023-10-14 19:58:39,244][61552] Updated weights for policy 0, policy_version 56552 (0.0007) [2023-10-14 19:58:39,623][61552] Updated weights for policy 0, policy_version 56562 (0.0007) [2023-10-14 19:58:39,983][61552] Updated weights for policy 0, policy_version 56572 (0.0008) [2023-10-14 19:58:40,857][61585] Updated weights for policy 1, policy_version 56390 (0.0007) [2023-10-14 19:58:41,237][61585] Updated weights for policy 1, policy_version 56400 (0.0010) [2023-10-14 19:58:41,605][61585] Updated weights for policy 1, policy_version 56410 (0.0009) [2023-10-14 19:58:43,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 115703808. Throughput: 0: 1678.6, 1: 1672.2. Samples: 28936966. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-10-14 19:58:43,344][60425] Avg episode reward: [(0, '74.920'), (1, '73.920')] [2023-10-14 19:58:43,918][61552] Updated weights for policy 0, policy_version 56582 (0.0007) [2023-10-14 19:58:44,288][61552] Updated weights for policy 0, policy_version 56592 (0.0008) [2023-10-14 19:58:44,665][61552] Updated weights for policy 0, policy_version 56602 (0.0009) [2023-10-14 19:58:45,724][61585] Updated weights for policy 1, policy_version 56420 (0.0010) [2023-10-14 19:58:46,093][61585] Updated weights for policy 1, policy_version 56430 (0.0010) [2023-10-14 19:58:46,462][61585] Updated weights for policy 1, policy_version 56440 (0.0012) [2023-10-14 19:58:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 115769344. Throughput: 0: 1676.6, 1: 1668.9. Samples: 28947122. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-10-14 19:58:48,344][60425] Avg episode reward: [(0, '70.400'), (1, '73.200')] [2023-10-14 19:58:48,901][61552] Updated weights for policy 0, policy_version 56612 (0.0011) [2023-10-14 19:58:49,278][61552] Updated weights for policy 0, policy_version 56622 (0.0009) [2023-10-14 19:58:49,656][61552] Updated weights for policy 0, policy_version 56632 (0.0011) [2023-10-14 19:58:50,525][61585] Updated weights for policy 1, policy_version 56450 (0.0009) [2023-10-14 19:58:50,891][61585] Updated weights for policy 1, policy_version 56460 (0.0008) [2023-10-14 19:58:51,252][61585] Updated weights for policy 1, policy_version 56470 (0.0010) [2023-10-14 19:58:51,614][61585] Updated weights for policy 1, policy_version 56480 (0.0010) [2023-10-14 19:58:53,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 115834880. Throughput: 0: 1666.6, 1: 1647.7. Samples: 28966176. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-10-14 19:58:53,344][60425] Avg episode reward: [(0, '76.790'), (1, '73.870')] [2023-10-14 19:58:53,808][61552] Updated weights for policy 0, policy_version 56642 (0.0008) [2023-10-14 19:58:54,187][61552] Updated weights for policy 0, policy_version 56652 (0.0007) [2023-10-14 19:58:54,557][61552] Updated weights for policy 0, policy_version 56662 (0.0009) [2023-10-14 19:58:54,926][61552] Updated weights for policy 0, policy_version 56672 (0.0007) [2023-10-14 19:58:55,787][61585] Updated weights for policy 1, policy_version 56490 (0.0008) [2023-10-14 19:58:56,158][61585] Updated weights for policy 1, policy_version 56500 (0.0010) [2023-10-14 19:58:56,535][61585] Updated weights for policy 1, policy_version 56510 (0.0010) [2023-10-14 19:58:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 115900416. Throughput: 0: 1669.6, 1: 1661.4. Samples: 28986698. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-10-14 19:58:58,344][60425] Avg episode reward: [(0, '73.430'), (1, '77.390')] [2023-10-14 19:58:58,952][61552] Updated weights for policy 0, policy_version 56682 (0.0008) [2023-10-14 19:58:59,324][61552] Updated weights for policy 0, policy_version 56692 (0.0008) [2023-10-14 19:58:59,689][61552] Updated weights for policy 0, policy_version 56702 (0.0010) [2023-10-14 19:59:00,656][61585] Updated weights for policy 1, policy_version 56520 (0.0012) [2023-10-14 19:59:01,019][61585] Updated weights for policy 1, policy_version 56530 (0.0007) [2023-10-14 19:59:01,384][61585] Updated weights for policy 1, policy_version 56540 (0.0009) [2023-10-14 19:59:03,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 115965952. Throughput: 0: 1670.9, 1: 1652.7. Samples: 28996672. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-10-14 19:59:03,344][60425] Avg episode reward: [(0, '75.210'), (1, '78.650')] [2023-10-14 19:59:03,729][61552] Updated weights for policy 0, policy_version 56712 (0.0008) [2023-10-14 19:59:04,092][61552] Updated weights for policy 0, policy_version 56722 (0.0011) [2023-10-14 19:59:04,456][61552] Updated weights for policy 0, policy_version 56732 (0.0011) [2023-10-14 19:59:05,310][61585] Updated weights for policy 1, policy_version 56550 (0.0008) [2023-10-14 19:59:05,678][61585] Updated weights for policy 1, policy_version 56560 (0.0007) [2023-10-14 19:59:06,037][61585] Updated weights for policy 1, policy_version 56570 (0.0009) [2023-10-14 19:59:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 116031488. Throughput: 0: 1677.2, 1: 1660.5. Samples: 29016874. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-10-14 19:59:08,344][60425] Avg episode reward: [(0, '78.720'), (1, '76.910')] [2023-10-14 19:59:08,651][61552] Updated weights for policy 0, policy_version 56742 (0.0010) [2023-10-14 19:59:09,025][61552] Updated weights for policy 0, policy_version 56752 (0.0010) [2023-10-14 19:59:09,389][61552] Updated weights for policy 0, policy_version 56762 (0.0011) [2023-10-14 19:59:10,151][61585] Updated weights for policy 1, policy_version 56580 (0.0010) [2023-10-14 19:59:10,518][61585] Updated weights for policy 1, policy_version 56590 (0.0009) [2023-10-14 19:59:10,883][61585] Updated weights for policy 1, policy_version 56600 (0.0008) [2023-10-14 19:59:13,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 116097024. Throughput: 0: 1675.9, 1: 1675.6. Samples: 29037696. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-10-14 19:59:13,344][60425] Avg episode reward: [(0, '73.760'), (1, '78.440')] [2023-10-14 19:59:13,446][61552] Updated weights for policy 0, policy_version 56772 (0.0011) [2023-10-14 19:59:13,846][61552] Updated weights for policy 0, policy_version 56782 (0.0008) [2023-10-14 19:59:14,208][61552] Updated weights for policy 0, policy_version 56792 (0.0009) [2023-10-14 19:59:14,697][61585] Updated weights for policy 1, policy_version 56610 (0.0009) [2023-10-14 19:59:15,056][61585] Updated weights for policy 1, policy_version 56620 (0.0009) [2023-10-14 19:59:15,423][61585] Updated weights for policy 1, policy_version 56630 (0.0011) [2023-10-14 19:59:15,786][61585] Updated weights for policy 1, policy_version 56640 (0.0010) [2023-10-14 19:59:18,180][61552] Updated weights for policy 0, policy_version 56802 (0.0009) [2023-10-14 19:59:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 116162560. Throughput: 0: 1680.5, 1: 1654.8. Samples: 29047002. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-10-14 19:59:18,344][60425] Avg episode reward: [(0, '76.990'), (1, '79.870')] [2023-10-14 19:59:18,545][61552] Updated weights for policy 0, policy_version 56812 (0.0008) [2023-10-14 19:59:18,916][61552] Updated weights for policy 0, policy_version 56822 (0.0008) [2023-10-14 19:59:19,291][61552] Updated weights for policy 0, policy_version 56832 (0.0009) [2023-10-14 19:59:19,927][61585] Updated weights for policy 1, policy_version 56650 (0.0009) [2023-10-14 19:59:20,296][61585] Updated weights for policy 1, policy_version 56660 (0.0007) [2023-10-14 19:59:20,659][61585] Updated weights for policy 1, policy_version 56670 (0.0007) [2023-10-14 19:59:23,314][61552] Updated weights for policy 0, policy_version 56842 (0.0008) [2023-10-14 19:59:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 116228096. Throughput: 0: 1677.6, 1: 1671.6. Samples: 29067306. Policy #0 lag: (min: 26.0, avg: 26.1, max: 32.0) [2023-10-14 19:59:23,344][60425] Avg episode reward: [(0, '72.910'), (1, '81.530')] [2023-10-14 19:59:23,345][61248] Saving new best policy, reward=81.530! [2023-10-14 19:59:23,691][61552] Updated weights for policy 0, policy_version 56852 (0.0009) [2023-10-14 19:59:24,053][61552] Updated weights for policy 0, policy_version 56862 (0.0010) [2023-10-14 19:59:24,927][61585] Updated weights for policy 1, policy_version 56680 (0.0011) [2023-10-14 19:59:25,288][61585] Updated weights for policy 1, policy_version 56690 (0.0012) [2023-10-14 19:59:25,658][61585] Updated weights for policy 1, policy_version 56700 (0.0009) [2023-10-14 19:59:28,165][61552] Updated weights for policy 0, policy_version 56872 (0.0008) [2023-10-14 19:59:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 116293632. Throughput: 0: 1675.2, 1: 1676.7. Samples: 29087802. Policy #0 lag: (min: 26.0, avg: 26.1, max: 32.0) [2023-10-14 19:59:28,344][60425] Avg episode reward: [(0, '78.060'), (1, '82.290')] [2023-10-14 19:59:28,354][61248] Saving new best policy, reward=82.290! [2023-10-14 19:59:28,526][61552] Updated weights for policy 0, policy_version 56882 (0.0011) [2023-10-14 19:59:28,900][61552] Updated weights for policy 0, policy_version 56892 (0.0007) [2023-10-14 19:59:29,845][61585] Updated weights for policy 1, policy_version 56710 (0.0009) [2023-10-14 19:59:30,203][61585] Updated weights for policy 1, policy_version 56720 (0.0008) [2023-10-14 19:59:30,575][61585] Updated weights for policy 1, policy_version 56730 (0.0007) [2023-10-14 19:59:33,003][61552] Updated weights for policy 0, policy_version 56902 (0.0007) [2023-10-14 19:59:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 116359168. Throughput: 0: 1671.7, 1: 1657.7. Samples: 29096948. Policy #0 lag: (min: 26.0, avg: 26.1, max: 32.0) [2023-10-14 19:59:33,344][60425] Avg episode reward: [(0, '70.420'), (1, '79.320')] [2023-10-14 19:59:33,385][61552] Updated weights for policy 0, policy_version 56912 (0.0008) [2023-10-14 19:59:33,753][61552] Updated weights for policy 0, policy_version 56922 (0.0009) [2023-10-14 19:59:34,740][61585] Updated weights for policy 1, policy_version 56740 (0.0007) [2023-10-14 19:59:35,112][61585] Updated weights for policy 1, policy_version 56750 (0.0008) [2023-10-14 19:59:35,471][61585] Updated weights for policy 1, policy_version 56760 (0.0008) [2023-10-14 19:59:37,876][61552] Updated weights for policy 0, policy_version 56932 (0.0007) [2023-10-14 19:59:38,249][61552] Updated weights for policy 0, policy_version 56942 (0.0008) [2023-10-14 19:59:38,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 116424704. Throughput: 0: 1677.9, 1: 1684.4. Samples: 29117482. Policy #0 lag: (min: 26.0, avg: 26.1, max: 32.0) [2023-10-14 19:59:38,344][60425] Avg episode reward: [(0, '75.540'), (1, '78.210')] [2023-10-14 19:59:38,620][61552] Updated weights for policy 0, policy_version 56952 (0.0008) [2023-10-14 19:59:39,723][61585] Updated weights for policy 1, policy_version 56770 (0.0011) [2023-10-14 19:59:40,081][61585] Updated weights for policy 1, policy_version 56780 (0.0008) [2023-10-14 19:59:40,445][61585] Updated weights for policy 1, policy_version 56790 (0.0007) [2023-10-14 19:59:40,816][61585] Updated weights for policy 1, policy_version 56800 (0.0007) [2023-10-14 19:59:42,768][61552] Updated weights for policy 0, policy_version 56962 (0.0009) [2023-10-14 19:59:43,130][61552] Updated weights for policy 0, policy_version 56972 (0.0010) [2023-10-14 19:59:43,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 116490240. Throughput: 0: 1672.9, 1: 1687.5. Samples: 29137918. Policy #0 lag: (min: 26.0, avg: 26.1, max: 32.0) [2023-10-14 19:59:43,344][60425] Avg episode reward: [(0, '74.960'), (1, '76.880')] [2023-10-14 19:59:43,351][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000056800_58163200.pth... [2023-10-14 19:59:43,381][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000055232_56557568.pth [2023-10-14 19:59:43,504][61552] Updated weights for policy 0, policy_version 56982 (0.0009) [2023-10-14 19:59:43,869][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000056992_58359808.pth... [2023-10-14 19:59:43,870][61552] Updated weights for policy 0, policy_version 56992 (0.0009) [2023-10-14 19:59:43,907][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000055424_56754176.pth [2023-10-14 19:59:44,910][61585] Updated weights for policy 1, policy_version 56810 (0.0007) [2023-10-14 19:59:45,271][61585] Updated weights for policy 1, policy_version 56820 (0.0007) [2023-10-14 19:59:45,633][61585] Updated weights for policy 1, policy_version 56830 (0.0007) [2023-10-14 19:59:48,129][61552] Updated weights for policy 0, policy_version 57002 (0.0008) [2023-10-14 19:59:48,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 116555776. Throughput: 0: 1673.5, 1: 1668.0. Samples: 29147040. Policy #0 lag: (min: 26.0, avg: 26.1, max: 32.0) [2023-10-14 19:59:48,344][60425] Avg episode reward: [(0, '74.660'), (1, '75.810')] [2023-10-14 19:59:48,496][61552] Updated weights for policy 0, policy_version 57012 (0.0009) [2023-10-14 19:59:48,870][61552] Updated weights for policy 0, policy_version 57022 (0.0009) [2023-10-14 19:59:49,583][61585] Updated weights for policy 1, policy_version 56840 (0.0010) [2023-10-14 19:59:49,946][61585] Updated weights for policy 1, policy_version 56850 (0.0009) [2023-10-14 19:59:50,313][61585] Updated weights for policy 1, policy_version 56860 (0.0011) [2023-10-14 19:59:52,886][61552] Updated weights for policy 0, policy_version 57032 (0.0009) [2023-10-14 19:59:53,250][61552] Updated weights for policy 0, policy_version 57042 (0.0010) [2023-10-14 19:59:53,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 116621312. Throughput: 0: 1672.8, 1: 1677.3. Samples: 29167630. Policy #0 lag: (min: 26.0, avg: 26.1, max: 32.0) [2023-10-14 19:59:53,344][60425] Avg episode reward: [(0, '76.560'), (1, '74.620')] [2023-10-14 19:59:53,617][61552] Updated weights for policy 0, policy_version 57052 (0.0010) [2023-10-14 19:59:54,350][61585] Updated weights for policy 1, policy_version 56870 (0.0011) [2023-10-14 19:59:54,722][61585] Updated weights for policy 1, policy_version 56880 (0.0010) [2023-10-14 19:59:55,089][61585] Updated weights for policy 1, policy_version 56890 (0.0009) [2023-10-14 19:59:57,747][61552] Updated weights for policy 0, policy_version 57062 (0.0008) [2023-10-14 19:59:58,125][61552] Updated weights for policy 0, policy_version 57072 (0.0007) [2023-10-14 19:59:58,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 116686848. Throughput: 0: 1668.8, 1: 1674.2. Samples: 29188132. Policy #0 lag: (min: 26.0, avg: 26.1, max: 32.0) [2023-10-14 19:59:58,344][60425] Avg episode reward: [(0, '72.960'), (1, '74.790')] [2023-10-14 19:59:58,499][61552] Updated weights for policy 0, policy_version 57082 (0.0008) [2023-10-14 19:59:59,189][61585] Updated weights for policy 1, policy_version 56900 (0.0009) [2023-10-14 19:59:59,554][61585] Updated weights for policy 1, policy_version 56910 (0.0008) [2023-10-14 19:59:59,925][61585] Updated weights for policy 1, policy_version 56920 (0.0008) [2023-10-14 20:00:02,601][61552] Updated weights for policy 0, policy_version 57092 (0.0009) [2023-10-14 20:00:02,970][61552] Updated weights for policy 0, policy_version 57102 (0.0012) [2023-10-14 20:00:03,343][61552] Updated weights for policy 0, policy_version 57112 (0.0007) [2023-10-14 20:00:03,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 116752384. Throughput: 0: 1672.5, 1: 1675.2. Samples: 29197650. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:00:03,344][60425] Avg episode reward: [(0, '72.950'), (1, '75.020')] [2023-10-14 20:00:03,956][61585] Updated weights for policy 1, policy_version 56930 (0.0008) [2023-10-14 20:00:04,320][61585] Updated weights for policy 1, policy_version 56940 (0.0009) [2023-10-14 20:00:04,684][61585] Updated weights for policy 1, policy_version 56950 (0.0009) [2023-10-14 20:00:05,044][61585] Updated weights for policy 1, policy_version 56960 (0.0009) [2023-10-14 20:00:07,291][61552] Updated weights for policy 0, policy_version 57122 (0.0007) [2023-10-14 20:00:07,656][61552] Updated weights for policy 0, policy_version 57132 (0.0009) [2023-10-14 20:00:08,031][61552] Updated weights for policy 0, policy_version 57142 (0.0009) [2023-10-14 20:00:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 116817920. Throughput: 0: 1672.5, 1: 1681.4. Samples: 29218234. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:00:08,344][60425] Avg episode reward: [(0, '72.770'), (1, '73.280')] [2023-10-14 20:00:08,396][61552] Updated weights for policy 0, policy_version 57152 (0.0010) [2023-10-14 20:00:09,130][61585] Updated weights for policy 1, policy_version 56970 (0.0010) [2023-10-14 20:00:09,493][61585] Updated weights for policy 1, policy_version 56980 (0.0011) [2023-10-14 20:00:09,853][61585] Updated weights for policy 1, policy_version 56990 (0.0009) [2023-10-14 20:00:12,576][61552] Updated weights for policy 0, policy_version 57162 (0.0007) [2023-10-14 20:00:12,944][61552] Updated weights for policy 0, policy_version 57172 (0.0009) [2023-10-14 20:00:13,314][61552] Updated weights for policy 0, policy_version 57182 (0.0008) [2023-10-14 20:00:13,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 116883456. Throughput: 0: 1659.6, 1: 1686.7. Samples: 29238382. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:00:13,344][60425] Avg episode reward: [(0, '71.870'), (1, '75.370')] [2023-10-14 20:00:13,740][61585] Updated weights for policy 1, policy_version 57000 (0.0008) [2023-10-14 20:00:14,096][61585] Updated weights for policy 1, policy_version 57010 (0.0008) [2023-10-14 20:00:14,467][61585] Updated weights for policy 1, policy_version 57020 (0.0007) [2023-10-14 20:00:17,500][61552] Updated weights for policy 0, policy_version 57192 (0.0007) [2023-10-14 20:00:17,889][61552] Updated weights for policy 0, policy_version 57202 (0.0008) [2023-10-14 20:00:18,258][61552] Updated weights for policy 0, policy_version 57212 (0.0011) [2023-10-14 20:00:18,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 116948992. Throughput: 0: 1671.9, 1: 1682.6. Samples: 29247900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:00:18,344][60425] Avg episode reward: [(0, '70.370'), (1, '71.600')] [2023-10-14 20:00:18,787][61585] Updated weights for policy 1, policy_version 57030 (0.0008) [2023-10-14 20:00:19,162][61585] Updated weights for policy 1, policy_version 57040 (0.0009) [2023-10-14 20:00:19,518][61585] Updated weights for policy 1, policy_version 57050 (0.0008) [2023-10-14 20:00:22,173][61552] Updated weights for policy 0, policy_version 57222 (0.0008) [2023-10-14 20:00:22,543][61552] Updated weights for policy 0, policy_version 57232 (0.0009) [2023-10-14 20:00:22,917][61552] Updated weights for policy 0, policy_version 57242 (0.0008) [2023-10-14 20:00:23,343][60425] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 117047296. Throughput: 0: 1677.4, 1: 1683.3. Samples: 29268712. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:00:23,344][60425] Avg episode reward: [(0, '72.580'), (1, '68.910')] [2023-10-14 20:00:23,528][61585] Updated weights for policy 1, policy_version 57060 (0.0007) [2023-10-14 20:00:23,892][61585] Updated weights for policy 1, policy_version 57070 (0.0009) [2023-10-14 20:00:24,260][61585] Updated weights for policy 1, policy_version 57080 (0.0009) [2023-10-14 20:00:27,072][61552] Updated weights for policy 0, policy_version 57252 (0.0009) [2023-10-14 20:00:27,437][61552] Updated weights for policy 0, policy_version 57262 (0.0009) [2023-10-14 20:00:27,798][61552] Updated weights for policy 0, policy_version 57272 (0.0009) [2023-10-14 20:00:28,344][60425] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 117112832. Throughput: 0: 1660.7, 1: 1684.5. Samples: 29288452. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:00:28,345][60425] Avg episode reward: [(0, '69.840'), (1, '75.700')] [2023-10-14 20:00:28,509][61585] Updated weights for policy 1, policy_version 57090 (0.0009) [2023-10-14 20:00:28,874][61585] Updated weights for policy 1, policy_version 57100 (0.0007) [2023-10-14 20:00:29,245][61585] Updated weights for policy 1, policy_version 57110 (0.0009) [2023-10-14 20:00:29,604][61585] Updated weights for policy 1, policy_version 57120 (0.0008) [2023-10-14 20:00:31,927][61552] Updated weights for policy 0, policy_version 57282 (0.0008) [2023-10-14 20:00:32,304][61552] Updated weights for policy 0, policy_version 57292 (0.0010) [2023-10-14 20:00:32,676][61552] Updated weights for policy 0, policy_version 57302 (0.0008) [2023-10-14 20:00:33,053][61552] Updated weights for policy 0, policy_version 57312 (0.0008) [2023-10-14 20:00:33,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 117178368. Throughput: 0: 1676.0, 1: 1680.3. Samples: 29298070. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:00:33,344][60425] Avg episode reward: [(0, '73.530'), (1, '77.590')] [2023-10-14 20:00:33,857][61585] Updated weights for policy 1, policy_version 57130 (0.0008) [2023-10-14 20:00:34,231][61585] Updated weights for policy 1, policy_version 57140 (0.0009) [2023-10-14 20:00:34,591][61585] Updated weights for policy 1, policy_version 57150 (0.0010) [2023-10-14 20:00:37,172][61552] Updated weights for policy 0, policy_version 57322 (0.0007) [2023-10-14 20:00:37,539][61552] Updated weights for policy 0, policy_version 57332 (0.0008) [2023-10-14 20:00:37,910][61552] Updated weights for policy 0, policy_version 57342 (0.0009) [2023-10-14 20:00:38,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 117243904. Throughput: 0: 1672.8, 1: 1678.0. Samples: 29318418. Policy #0 lag: (min: 11.0, avg: 17.6, max: 43.0) [2023-10-14 20:00:38,344][60425] Avg episode reward: [(0, '74.030'), (1, '74.010')] [2023-10-14 20:00:38,815][61585] Updated weights for policy 1, policy_version 57160 (0.0009) [2023-10-14 20:00:39,195][61585] Updated weights for policy 1, policy_version 57170 (0.0008) [2023-10-14 20:00:39,553][61585] Updated weights for policy 1, policy_version 57180 (0.0009) [2023-10-14 20:00:41,811][61552] Updated weights for policy 0, policy_version 57352 (0.0008) [2023-10-14 20:00:42,186][61552] Updated weights for policy 0, policy_version 57362 (0.0008) [2023-10-14 20:00:42,556][61552] Updated weights for policy 0, policy_version 57372 (0.0008) [2023-10-14 20:00:43,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 117309440. Throughput: 0: 1649.9, 1: 1676.1. Samples: 29337800. Policy #0 lag: (min: 11.0, avg: 17.6, max: 43.0) [2023-10-14 20:00:43,345][60425] Avg episode reward: [(0, '73.190'), (1, '73.110')] [2023-10-14 20:00:43,412][61585] Updated weights for policy 1, policy_version 57190 (0.0009) [2023-10-14 20:00:43,770][61585] Updated weights for policy 1, policy_version 57200 (0.0007) [2023-10-14 20:00:44,136][61585] Updated weights for policy 1, policy_version 57210 (0.0008) [2023-10-14 20:00:46,413][61552] Updated weights for policy 0, policy_version 57382 (0.0008) [2023-10-14 20:00:46,781][61552] Updated weights for policy 0, policy_version 57392 (0.0008) [2023-10-14 20:00:47,160][61552] Updated weights for policy 0, policy_version 57402 (0.0008) [2023-10-14 20:00:48,244][61585] Updated weights for policy 1, policy_version 57220 (0.0008) [2023-10-14 20:00:48,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 117374976. Throughput: 0: 1673.5, 1: 1669.8. Samples: 29348096. Policy #0 lag: (min: 11.0, avg: 17.6, max: 43.0) [2023-10-14 20:00:48,344][60425] Avg episode reward: [(0, '76.400'), (1, '73.270')] [2023-10-14 20:00:48,609][61585] Updated weights for policy 1, policy_version 57230 (0.0008) [2023-10-14 20:00:48,983][61585] Updated weights for policy 1, policy_version 57240 (0.0007) [2023-10-14 20:00:51,257][61552] Updated weights for policy 0, policy_version 57412 (0.0010) [2023-10-14 20:00:51,650][61552] Updated weights for policy 0, policy_version 57422 (0.0008) [2023-10-14 20:00:52,014][61552] Updated weights for policy 0, policy_version 57432 (0.0009) [2023-10-14 20:00:53,071][61585] Updated weights for policy 1, policy_version 57250 (0.0008) [2023-10-14 20:00:53,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 117440512. Throughput: 0: 1661.6, 1: 1669.8. Samples: 29368148. Policy #0 lag: (min: 11.0, avg: 17.6, max: 43.0) [2023-10-14 20:00:53,344][60425] Avg episode reward: [(0, '74.990'), (1, '72.480')] [2023-10-14 20:00:53,445][61585] Updated weights for policy 1, policy_version 57260 (0.0007) [2023-10-14 20:00:53,807][61585] Updated weights for policy 1, policy_version 57270 (0.0009) [2023-10-14 20:00:54,171][61585] Updated weights for policy 1, policy_version 57280 (0.0008) [2023-10-14 20:00:56,077][61552] Updated weights for policy 0, policy_version 57442 (0.0010) [2023-10-14 20:00:56,443][61552] Updated weights for policy 0, policy_version 57452 (0.0010) [2023-10-14 20:00:56,813][61552] Updated weights for policy 0, policy_version 57462 (0.0009) [2023-10-14 20:00:57,181][61552] Updated weights for policy 0, policy_version 57472 (0.0010) [2023-10-14 20:00:58,195][61585] Updated weights for policy 1, policy_version 57290 (0.0007) [2023-10-14 20:00:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 117506048. Throughput: 0: 1664.8, 1: 1667.7. Samples: 29388346. Policy #0 lag: (min: 11.0, avg: 17.6, max: 43.0) [2023-10-14 20:00:58,344][60425] Avg episode reward: [(0, '73.250'), (1, '77.920')] [2023-10-14 20:00:58,569][61585] Updated weights for policy 1, policy_version 57300 (0.0007) [2023-10-14 20:00:58,929][61585] Updated weights for policy 1, policy_version 57310 (0.0007) [2023-10-14 20:01:01,340][61552] Updated weights for policy 0, policy_version 57482 (0.0009) [2023-10-14 20:01:01,716][61552] Updated weights for policy 0, policy_version 57492 (0.0009) [2023-10-14 20:01:02,086][61552] Updated weights for policy 0, policy_version 57502 (0.0007) [2023-10-14 20:01:03,111][61585] Updated weights for policy 1, policy_version 57320 (0.0008) [2023-10-14 20:01:03,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 117571584. Throughput: 0: 1686.6, 1: 1668.6. Samples: 29398884. Policy #0 lag: (min: 11.0, avg: 17.6, max: 43.0) [2023-10-14 20:01:03,344][60425] Avg episode reward: [(0, '71.700'), (1, '71.980')] [2023-10-14 20:01:03,502][61585] Updated weights for policy 1, policy_version 57330 (0.0009) [2023-10-14 20:01:03,861][61585] Updated weights for policy 1, policy_version 57340 (0.0009) [2023-10-14 20:01:06,069][61552] Updated weights for policy 0, policy_version 57512 (0.0008) [2023-10-14 20:01:06,436][61552] Updated weights for policy 0, policy_version 57522 (0.0007) [2023-10-14 20:01:06,805][61552] Updated weights for policy 0, policy_version 57532 (0.0009) [2023-10-14 20:01:08,079][61585] Updated weights for policy 1, policy_version 57350 (0.0009) [2023-10-14 20:01:08,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 117637120. Throughput: 0: 1664.3, 1: 1660.8. Samples: 29418342. Policy #0 lag: (min: 11.0, avg: 17.6, max: 43.0) [2023-10-14 20:01:08,344][60425] Avg episode reward: [(0, '76.030'), (1, '77.040')] [2023-10-14 20:01:08,446][61585] Updated weights for policy 1, policy_version 57360 (0.0009) [2023-10-14 20:01:08,824][61585] Updated weights for policy 1, policy_version 57370 (0.0008) [2023-10-14 20:01:10,820][61552] Updated weights for policy 0, policy_version 57542 (0.0009) [2023-10-14 20:01:11,190][61552] Updated weights for policy 0, policy_version 57552 (0.0008) [2023-10-14 20:01:11,557][61552] Updated weights for policy 0, policy_version 57562 (0.0007) [2023-10-14 20:01:13,016][61585] Updated weights for policy 1, policy_version 57380 (0.0008) [2023-10-14 20:01:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 117702656. Throughput: 0: 1675.7, 1: 1664.6. Samples: 29438764. Policy #0 lag: (min: 11.0, avg: 17.6, max: 43.0) [2023-10-14 20:01:13,344][60425] Avg episode reward: [(0, '72.960'), (1, '76.330')] [2023-10-14 20:01:13,394][61585] Updated weights for policy 1, policy_version 57390 (0.0009) [2023-10-14 20:01:13,755][61585] Updated weights for policy 1, policy_version 57400 (0.0009) [2023-10-14 20:01:15,615][61552] Updated weights for policy 0, policy_version 57572 (0.0008) [2023-10-14 20:01:15,982][61552] Updated weights for policy 0, policy_version 57582 (0.0008) [2023-10-14 20:01:16,360][61552] Updated weights for policy 0, policy_version 57592 (0.0009) [2023-10-14 20:01:17,782][61585] Updated weights for policy 1, policy_version 57410 (0.0009) [2023-10-14 20:01:18,154][61585] Updated weights for policy 1, policy_version 57420 (0.0007) [2023-10-14 20:01:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 117768192. Throughput: 0: 1683.8, 1: 1666.5. Samples: 29448834. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) [2023-10-14 20:01:18,344][60425] Avg episode reward: [(0, '75.960'), (1, '74.690')] [2023-10-14 20:01:18,509][61585] Updated weights for policy 1, policy_version 57430 (0.0010) [2023-10-14 20:01:18,877][61585] Updated weights for policy 1, policy_version 57440 (0.0012) [2023-10-14 20:01:20,479][61552] Updated weights for policy 0, policy_version 57602 (0.0007) [2023-10-14 20:01:20,849][61552] Updated weights for policy 0, policy_version 57612 (0.0008) [2023-10-14 20:01:21,216][61552] Updated weights for policy 0, policy_version 57622 (0.0008) [2023-10-14 20:01:21,587][61552] Updated weights for policy 0, policy_version 57632 (0.0010) [2023-10-14 20:01:23,202][61585] Updated weights for policy 1, policy_version 57450 (0.0008) [2023-10-14 20:01:23,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 117833728. Throughput: 0: 1660.5, 1: 1668.1. Samples: 29468206. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) [2023-10-14 20:01:23,344][60425] Avg episode reward: [(0, '73.660'), (1, '72.110')] [2023-10-14 20:01:23,567][61585] Updated weights for policy 1, policy_version 57460 (0.0008) [2023-10-14 20:01:23,932][61585] Updated weights for policy 1, policy_version 57470 (0.0008) [2023-10-14 20:01:25,711][61552] Updated weights for policy 0, policy_version 57642 (0.0009) [2023-10-14 20:01:26,074][61552] Updated weights for policy 0, policy_version 57652 (0.0010) [2023-10-14 20:01:26,447][61552] Updated weights for policy 0, policy_version 57662 (0.0009) [2023-10-14 20:01:27,850][61585] Updated weights for policy 1, policy_version 57480 (0.0011) [2023-10-14 20:01:28,224][61585] Updated weights for policy 1, policy_version 57490 (0.0010) [2023-10-14 20:01:28,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 117899264. Throughput: 0: 1689.7, 1: 1668.1. Samples: 29488902. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) [2023-10-14 20:01:28,344][60425] Avg episode reward: [(0, '71.960'), (1, '75.820')] [2023-10-14 20:01:28,590][61585] Updated weights for policy 1, policy_version 57500 (0.0010) [2023-10-14 20:01:30,504][61552] Updated weights for policy 0, policy_version 57672 (0.0007) [2023-10-14 20:01:30,877][61552] Updated weights for policy 0, policy_version 57682 (0.0007) [2023-10-14 20:01:31,246][61552] Updated weights for policy 0, policy_version 57692 (0.0008) [2023-10-14 20:01:32,574][61585] Updated weights for policy 1, policy_version 57510 (0.0011) [2023-10-14 20:01:32,928][61585] Updated weights for policy 1, policy_version 57520 (0.0009) [2023-10-14 20:01:33,295][61585] Updated weights for policy 1, policy_version 57530 (0.0008) [2023-10-14 20:01:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 117964800. Throughput: 0: 1678.0, 1: 1671.4. Samples: 29498818. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) [2023-10-14 20:01:33,344][60425] Avg episode reward: [(0, '72.940'), (1, '71.670')] [2023-10-14 20:01:35,355][61552] Updated weights for policy 0, policy_version 57702 (0.0008) [2023-10-14 20:01:35,723][61552] Updated weights for policy 0, policy_version 57712 (0.0008) [2023-10-14 20:01:36,101][61552] Updated weights for policy 0, policy_version 57722 (0.0008) [2023-10-14 20:01:37,275][61585] Updated weights for policy 1, policy_version 57540 (0.0008) [2023-10-14 20:01:37,649][61585] Updated weights for policy 1, policy_version 57550 (0.0007) [2023-10-14 20:01:38,003][61585] Updated weights for policy 1, policy_version 57560 (0.0007) [2023-10-14 20:01:38,343][60425] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 118063104. Throughput: 0: 1675.8, 1: 1671.6. Samples: 29518782. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) [2023-10-14 20:01:38,344][60425] Avg episode reward: [(0, '79.030'), (1, '72.450')] [2023-10-14 20:01:40,323][61552] Updated weights for policy 0, policy_version 57732 (0.0008) [2023-10-14 20:01:40,713][61552] Updated weights for policy 0, policy_version 57742 (0.0010) [2023-10-14 20:01:41,082][61552] Updated weights for policy 0, policy_version 57752 (0.0008) [2023-10-14 20:01:42,069][61585] Updated weights for policy 1, policy_version 57570 (0.0009) [2023-10-14 20:01:42,436][61585] Updated weights for policy 1, policy_version 57580 (0.0008) [2023-10-14 20:01:42,804][61585] Updated weights for policy 1, policy_version 57590 (0.0008) [2023-10-14 20:01:43,169][61585] Updated weights for policy 1, policy_version 57600 (0.0009) [2023-10-14 20:01:43,344][60425] Fps is (10 sec: 16383.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 118128640. Throughput: 0: 1681.4, 1: 1655.9. Samples: 29538526. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) [2023-10-14 20:01:43,345][60425] Avg episode reward: [(0, '76.960'), (1, '73.000')] [2023-10-14 20:01:43,357][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000057600_58982400.pth... [2023-10-14 20:01:43,357][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000057760_59146240.pth... [2023-10-14 20:01:43,404][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000056032_57376768.pth [2023-10-14 20:01:43,404][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000056192_57540608.pth [2023-10-14 20:01:45,225][61552] Updated weights for policy 0, policy_version 57762 (0.0009) [2023-10-14 20:01:45,594][61552] Updated weights for policy 0, policy_version 57772 (0.0009) [2023-10-14 20:01:45,956][61552] Updated weights for policy 0, policy_version 57782 (0.0007) [2023-10-14 20:01:46,335][61552] Updated weights for policy 0, policy_version 57792 (0.0007) [2023-10-14 20:01:47,221][61585] Updated weights for policy 1, policy_version 57610 (0.0007) [2023-10-14 20:01:47,586][61585] Updated weights for policy 1, policy_version 57620 (0.0007) [2023-10-14 20:01:47,949][61585] Updated weights for policy 1, policy_version 57630 (0.0007) [2023-10-14 20:01:48,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 118194176. Throughput: 0: 1661.3, 1: 1675.7. Samples: 29549048. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) [2023-10-14 20:01:48,344][60425] Avg episode reward: [(0, '76.220'), (1, '69.340')] [2023-10-14 20:01:50,379][61552] Updated weights for policy 0, policy_version 57802 (0.0007) [2023-10-14 20:01:50,743][61552] Updated weights for policy 0, policy_version 57812 (0.0009) [2023-10-14 20:01:51,115][61552] Updated weights for policy 0, policy_version 57822 (0.0008) [2023-10-14 20:01:52,160][61585] Updated weights for policy 1, policy_version 57640 (0.0008) [2023-10-14 20:01:52,543][61585] Updated weights for policy 1, policy_version 57650 (0.0008) [2023-10-14 20:01:52,900][61585] Updated weights for policy 1, policy_version 57660 (0.0007) [2023-10-14 20:01:53,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 118259712. Throughput: 0: 1663.1, 1: 1685.3. Samples: 29569022. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) [2023-10-14 20:01:53,344][60425] Avg episode reward: [(0, '75.500'), (1, '70.860')] [2023-10-14 20:01:55,190][61552] Updated weights for policy 0, policy_version 57832 (0.0010) [2023-10-14 20:01:55,558][61552] Updated weights for policy 0, policy_version 57842 (0.0009) [2023-10-14 20:01:55,927][61552] Updated weights for policy 0, policy_version 57852 (0.0008) [2023-10-14 20:01:56,894][61585] Updated weights for policy 1, policy_version 57670 (0.0010) [2023-10-14 20:01:57,253][61585] Updated weights for policy 1, policy_version 57680 (0.0011) [2023-10-14 20:01:57,616][61585] Updated weights for policy 1, policy_version 57690 (0.0008) [2023-10-14 20:01:58,344][60425] Fps is (10 sec: 13106.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 118325248. Throughput: 0: 1673.7, 1: 1656.2. Samples: 29588608. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:01:58,345][60425] Avg episode reward: [(0, '75.250'), (1, '71.890')] [2023-10-14 20:01:59,802][61552] Updated weights for policy 0, policy_version 57862 (0.0008) [2023-10-14 20:02:00,161][61552] Updated weights for policy 0, policy_version 57872 (0.0009) [2023-10-14 20:02:00,534][61552] Updated weights for policy 0, policy_version 57882 (0.0007) [2023-10-14 20:02:01,842][61585] Updated weights for policy 1, policy_version 57700 (0.0011) [2023-10-14 20:02:02,207][61585] Updated weights for policy 1, policy_version 57710 (0.0007) [2023-10-14 20:02:02,565][61585] Updated weights for policy 1, policy_version 57720 (0.0008) [2023-10-14 20:02:03,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 118390784. Throughput: 0: 1659.5, 1: 1679.7. Samples: 29599098. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:02:03,344][60425] Avg episode reward: [(0, '76.980'), (1, '70.800')] [2023-10-14 20:02:04,709][61552] Updated weights for policy 0, policy_version 57892 (0.0009) [2023-10-14 20:02:05,067][61552] Updated weights for policy 0, policy_version 57902 (0.0008) [2023-10-14 20:02:05,438][61552] Updated weights for policy 0, policy_version 57912 (0.0007) [2023-10-14 20:02:06,614][61585] Updated weights for policy 1, policy_version 57730 (0.0008) [2023-10-14 20:02:06,978][61585] Updated weights for policy 1, policy_version 57740 (0.0009) [2023-10-14 20:02:07,337][61585] Updated weights for policy 1, policy_version 57750 (0.0009) [2023-10-14 20:02:07,695][61585] Updated weights for policy 1, policy_version 57760 (0.0011) [2023-10-14 20:02:08,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 118456320. Throughput: 0: 1678.6, 1: 1677.9. Samples: 29619248. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:02:08,344][60425] Avg episode reward: [(0, '77.030'), (1, '69.970')] [2023-10-14 20:02:09,376][61552] Updated weights for policy 0, policy_version 57922 (0.0008) [2023-10-14 20:02:09,745][61552] Updated weights for policy 0, policy_version 57932 (0.0008) [2023-10-14 20:02:10,105][61552] Updated weights for policy 0, policy_version 57942 (0.0009) [2023-10-14 20:02:10,475][61552] Updated weights for policy 0, policy_version 57952 (0.0007) [2023-10-14 20:02:11,836][61585] Updated weights for policy 1, policy_version 57770 (0.0008) [2023-10-14 20:02:12,216][61585] Updated weights for policy 1, policy_version 57780 (0.0009) [2023-10-14 20:02:12,583][61585] Updated weights for policy 1, policy_version 57790 (0.0009) [2023-10-14 20:02:13,343][60425] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 118521856. Throughput: 0: 1682.2, 1: 1654.1. Samples: 29639038. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:02:13,344][60425] Avg episode reward: [(0, '76.370'), (1, '69.810')] [2023-10-14 20:02:14,566][61552] Updated weights for policy 0, policy_version 57962 (0.0010) [2023-10-14 20:02:14,930][61552] Updated weights for policy 0, policy_version 57972 (0.0007) [2023-10-14 20:02:15,306][61552] Updated weights for policy 0, policy_version 57982 (0.0007) [2023-10-14 20:02:16,602][61585] Updated weights for policy 1, policy_version 57800 (0.0008) [2023-10-14 20:02:16,963][61585] Updated weights for policy 1, policy_version 57810 (0.0011) [2023-10-14 20:02:17,324][61585] Updated weights for policy 1, policy_version 57820 (0.0009) [2023-10-14 20:02:18,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 118587392. Throughput: 0: 1668.3, 1: 1679.1. Samples: 29649450. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:02:18,344][60425] Avg episode reward: [(0, '72.490'), (1, '72.360')] [2023-10-14 20:02:19,343][61552] Updated weights for policy 0, policy_version 57992 (0.0010) [2023-10-14 20:02:19,701][61552] Updated weights for policy 0, policy_version 58002 (0.0011) [2023-10-14 20:02:20,068][61552] Updated weights for policy 0, policy_version 58012 (0.0011) [2023-10-14 20:02:21,467][61585] Updated weights for policy 1, policy_version 57830 (0.0010) [2023-10-14 20:02:21,831][61585] Updated weights for policy 1, policy_version 57840 (0.0009) [2023-10-14 20:02:22,195][61585] Updated weights for policy 1, policy_version 57850 (0.0009) [2023-10-14 20:02:23,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 118652928. Throughput: 0: 1680.0, 1: 1667.0. Samples: 29669398. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:02:23,344][60425] Avg episode reward: [(0, '74.910'), (1, '70.670')] [2023-10-14 20:02:24,279][61552] Updated weights for policy 0, policy_version 58022 (0.0008) [2023-10-14 20:02:24,643][61552] Updated weights for policy 0, policy_version 58032 (0.0008) [2023-10-14 20:02:25,014][61552] Updated weights for policy 0, policy_version 58042 (0.0007) [2023-10-14 20:02:26,257][61585] Updated weights for policy 1, policy_version 57860 (0.0008) [2023-10-14 20:02:26,622][61585] Updated weights for policy 1, policy_version 57870 (0.0009) [2023-10-14 20:02:26,995][61585] Updated weights for policy 1, policy_version 57880 (0.0008) [2023-10-14 20:02:28,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 118718464. Throughput: 0: 1684.3, 1: 1668.6. Samples: 29689406. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:02:28,345][60425] Avg episode reward: [(0, '72.190'), (1, '73.300')] [2023-10-14 20:02:29,051][61552] Updated weights for policy 0, policy_version 58052 (0.0007) [2023-10-14 20:02:29,439][61552] Updated weights for policy 0, policy_version 58062 (0.0009) [2023-10-14 20:02:29,802][61552] Updated weights for policy 0, policy_version 58072 (0.0009) [2023-10-14 20:02:30,962][61585] Updated weights for policy 1, policy_version 57890 (0.0010) [2023-10-14 20:02:31,329][61585] Updated weights for policy 1, policy_version 57900 (0.0008) [2023-10-14 20:02:31,696][61585] Updated weights for policy 1, policy_version 57910 (0.0008) [2023-10-14 20:02:32,062][61585] Updated weights for policy 1, policy_version 57920 (0.0009) [2023-10-14 20:02:33,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 118784000. Throughput: 0: 1671.2, 1: 1683.4. Samples: 29700006. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:02:33,344][60425] Avg episode reward: [(0, '76.300'), (1, '68.670')] [2023-10-14 20:02:33,897][61552] Updated weights for policy 0, policy_version 58082 (0.0007) [2023-10-14 20:02:34,269][61552] Updated weights for policy 0, policy_version 58092 (0.0007) [2023-10-14 20:02:34,629][61552] Updated weights for policy 0, policy_version 58102 (0.0009) [2023-10-14 20:02:35,000][61552] Updated weights for policy 0, policy_version 58112 (0.0010) [2023-10-14 20:02:36,129][61585] Updated weights for policy 1, policy_version 57930 (0.0007) [2023-10-14 20:02:36,500][61585] Updated weights for policy 1, policy_version 57940 (0.0009) [2023-10-14 20:02:36,867][61585] Updated weights for policy 1, policy_version 57950 (0.0010) [2023-10-14 20:02:38,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 118849536. Throughput: 0: 1684.0, 1: 1660.9. Samples: 29719540. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-14 20:02:38,344][60425] Avg episode reward: [(0, '72.510'), (1, '75.010')] [2023-10-14 20:02:39,198][61552] Updated weights for policy 0, policy_version 58122 (0.0010) [2023-10-14 20:02:39,570][61552] Updated weights for policy 0, policy_version 58132 (0.0008) [2023-10-14 20:02:39,938][61552] Updated weights for policy 0, policy_version 58142 (0.0008) [2023-10-14 20:02:41,183][61585] Updated weights for policy 1, policy_version 57960 (0.0009) [2023-10-14 20:02:41,547][61585] Updated weights for policy 1, policy_version 57970 (0.0010) [2023-10-14 20:02:41,908][61585] Updated weights for policy 1, policy_version 57980 (0.0010) [2023-10-14 20:02:43,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 118915072. Throughput: 0: 1680.1, 1: 1678.0. Samples: 29739720. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-14 20:02:43,344][60425] Avg episode reward: [(0, '72.540'), (1, '77.570')] [2023-10-14 20:02:43,978][61552] Updated weights for policy 0, policy_version 58152 (0.0011) [2023-10-14 20:02:44,338][61552] Updated weights for policy 0, policy_version 58162 (0.0009) [2023-10-14 20:02:44,717][61552] Updated weights for policy 0, policy_version 58172 (0.0009) [2023-10-14 20:02:45,783][61585] Updated weights for policy 1, policy_version 57990 (0.0011) [2023-10-14 20:02:46,151][61585] Updated weights for policy 1, policy_version 58000 (0.0010) [2023-10-14 20:02:46,519][61585] Updated weights for policy 1, policy_version 58010 (0.0010) [2023-10-14 20:02:48,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 118980608. Throughput: 0: 1671.2, 1: 1681.3. Samples: 29749958. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-14 20:02:48,344][60425] Avg episode reward: [(0, '72.200'), (1, '73.160')] [2023-10-14 20:02:48,933][61552] Updated weights for policy 0, policy_version 58182 (0.0007) [2023-10-14 20:02:49,308][61552] Updated weights for policy 0, policy_version 58192 (0.0007) [2023-10-14 20:02:49,685][61552] Updated weights for policy 0, policy_version 58202 (0.0009) [2023-10-14 20:02:50,613][61585] Updated weights for policy 1, policy_version 58020 (0.0009) [2023-10-14 20:02:50,975][61585] Updated weights for policy 1, policy_version 58030 (0.0009) [2023-10-14 20:02:51,346][61585] Updated weights for policy 1, policy_version 58040 (0.0009) [2023-10-14 20:02:53,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 119046144. Throughput: 0: 1678.1, 1: 1661.2. Samples: 29769516. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-14 20:02:53,346][60425] Avg episode reward: [(0, '77.210'), (1, '74.870')] [2023-10-14 20:02:53,716][61552] Updated weights for policy 0, policy_version 58212 (0.0010) [2023-10-14 20:02:54,078][61552] Updated weights for policy 0, policy_version 58222 (0.0009) [2023-10-14 20:02:54,455][61552] Updated weights for policy 0, policy_version 58232 (0.0008) [2023-10-14 20:02:55,333][61585] Updated weights for policy 1, policy_version 58050 (0.0008) [2023-10-14 20:02:55,697][61585] Updated weights for policy 1, policy_version 58060 (0.0007) [2023-10-14 20:02:56,060][61585] Updated weights for policy 1, policy_version 58070 (0.0007) [2023-10-14 20:02:56,418][61585] Updated weights for policy 1, policy_version 58080 (0.0010) [2023-10-14 20:02:58,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 119111680. Throughput: 0: 1672.8, 1: 1690.0. Samples: 29790364. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-14 20:02:58,344][60425] Avg episode reward: [(0, '71.850'), (1, '75.920')] [2023-10-14 20:02:58,429][61552] Updated weights for policy 0, policy_version 58242 (0.0009) [2023-10-14 20:02:58,799][61552] Updated weights for policy 0, policy_version 58252 (0.0010) [2023-10-14 20:02:59,175][61552] Updated weights for policy 0, policy_version 58262 (0.0009) [2023-10-14 20:02:59,535][61552] Updated weights for policy 0, policy_version 58272 (0.0008) [2023-10-14 20:03:00,448][61585] Updated weights for policy 1, policy_version 58090 (0.0008) [2023-10-14 20:03:00,814][61585] Updated weights for policy 1, policy_version 58100 (0.0009) [2023-10-14 20:03:01,187][61585] Updated weights for policy 1, policy_version 58110 (0.0010) [2023-10-14 20:03:03,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 119177216. Throughput: 0: 1668.3, 1: 1678.9. Samples: 29800078. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-14 20:03:03,344][60425] Avg episode reward: [(0, '75.670'), (1, '73.000')] [2023-10-14 20:03:03,757][61552] Updated weights for policy 0, policy_version 58282 (0.0009) [2023-10-14 20:03:04,129][61552] Updated weights for policy 0, policy_version 58292 (0.0010) [2023-10-14 20:03:04,503][61552] Updated weights for policy 0, policy_version 58302 (0.0007) [2023-10-14 20:03:05,311][61585] Updated weights for policy 1, policy_version 58120 (0.0007) [2023-10-14 20:03:05,675][61585] Updated weights for policy 1, policy_version 58130 (0.0007) [2023-10-14 20:03:06,041][61585] Updated weights for policy 1, policy_version 58140 (0.0007) [2023-10-14 20:03:08,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 119242752. Throughput: 0: 1670.4, 1: 1674.6. Samples: 29819922. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-14 20:03:08,344][60425] Avg episode reward: [(0, '75.200'), (1, '70.510')] [2023-10-14 20:03:08,577][61552] Updated weights for policy 0, policy_version 58312 (0.0009) [2023-10-14 20:03:08,950][61552] Updated weights for policy 0, policy_version 58322 (0.0007) [2023-10-14 20:03:09,315][61552] Updated weights for policy 0, policy_version 58332 (0.0010) [2023-10-14 20:03:10,120][61585] Updated weights for policy 1, policy_version 58150 (0.0007) [2023-10-14 20:03:10,482][61585] Updated weights for policy 1, policy_version 58160 (0.0010) [2023-10-14 20:03:10,849][61585] Updated weights for policy 1, policy_version 58170 (0.0009) [2023-10-14 20:03:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 119308288. Throughput: 0: 1667.3, 1: 1688.8. Samples: 29840428. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-14 20:03:13,345][60425] Avg episode reward: [(0, '73.490'), (1, '74.340')] [2023-10-14 20:03:13,418][61552] Updated weights for policy 0, policy_version 58342 (0.0008) [2023-10-14 20:03:13,797][61552] Updated weights for policy 0, policy_version 58352 (0.0007) [2023-10-14 20:03:14,167][61552] Updated weights for policy 0, policy_version 58362 (0.0007) [2023-10-14 20:03:14,922][61585] Updated weights for policy 1, policy_version 58180 (0.0007) [2023-10-14 20:03:15,285][61585] Updated weights for policy 1, policy_version 58190 (0.0008) [2023-10-14 20:03:15,653][61585] Updated weights for policy 1, policy_version 58200 (0.0009) [2023-10-14 20:03:18,154][61552] Updated weights for policy 0, policy_version 58372 (0.0007) [2023-10-14 20:03:18,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 119373824. Throughput: 0: 1671.2, 1: 1659.6. Samples: 29849894. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:03:18,344][60425] Avg episode reward: [(0, '73.500'), (1, '72.760')] [2023-10-14 20:03:18,527][61552] Updated weights for policy 0, policy_version 58382 (0.0008) [2023-10-14 20:03:18,890][61552] Updated weights for policy 0, policy_version 58392 (0.0007) [2023-10-14 20:03:19,786][61585] Updated weights for policy 1, policy_version 58210 (0.0007) [2023-10-14 20:03:20,151][61585] Updated weights for policy 1, policy_version 58220 (0.0009) [2023-10-14 20:03:20,511][61585] Updated weights for policy 1, policy_version 58230 (0.0009) [2023-10-14 20:03:20,874][61585] Updated weights for policy 1, policy_version 58240 (0.0009) [2023-10-14 20:03:23,203][61552] Updated weights for policy 0, policy_version 58402 (0.0008) [2023-10-14 20:03:23,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 119439360. Throughput: 0: 1668.6, 1: 1677.4. Samples: 29870112. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:03:23,344][60425] Avg episode reward: [(0, '76.250'), (1, '72.820')] [2023-10-14 20:03:23,571][61552] Updated weights for policy 0, policy_version 58412 (0.0007) [2023-10-14 20:03:23,941][61552] Updated weights for policy 0, policy_version 58422 (0.0008) [2023-10-14 20:03:24,305][61552] Updated weights for policy 0, policy_version 58432 (0.0010) [2023-10-14 20:03:24,985][61585] Updated weights for policy 1, policy_version 58250 (0.0009) [2023-10-14 20:03:25,353][61585] Updated weights for policy 1, policy_version 58260 (0.0010) [2023-10-14 20:03:25,732][61585] Updated weights for policy 1, policy_version 58270 (0.0010) [2023-10-14 20:03:28,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 119504896. Throughput: 0: 1668.5, 1: 1689.0. Samples: 29890810. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:03:28,344][60425] Avg episode reward: [(0, '73.050'), (1, '75.070')] [2023-10-14 20:03:28,368][61552] Updated weights for policy 0, policy_version 58442 (0.0010) [2023-10-14 20:03:28,731][61552] Updated weights for policy 0, policy_version 58452 (0.0009) [2023-10-14 20:03:29,105][61552] Updated weights for policy 0, policy_version 58462 (0.0008) [2023-10-14 20:03:29,737][61585] Updated weights for policy 1, policy_version 58280 (0.0009) [2023-10-14 20:03:30,104][61585] Updated weights for policy 1, policy_version 58290 (0.0008) [2023-10-14 20:03:30,466][61585] Updated weights for policy 1, policy_version 58300 (0.0007) [2023-10-14 20:03:33,115][61552] Updated weights for policy 0, policy_version 58472 (0.0008) [2023-10-14 20:03:33,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 119570432. Throughput: 0: 1670.3, 1: 1666.2. Samples: 29900100. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:03:33,344][60425] Avg episode reward: [(0, '74.760'), (1, '72.380')] [2023-10-14 20:03:33,492][61552] Updated weights for policy 0, policy_version 58482 (0.0007) [2023-10-14 20:03:33,849][61552] Updated weights for policy 0, policy_version 58492 (0.0008) [2023-10-14 20:03:34,373][61585] Updated weights for policy 1, policy_version 58310 (0.0011) [2023-10-14 20:03:34,739][61585] Updated weights for policy 1, policy_version 58320 (0.0008) [2023-10-14 20:03:35,111][61585] Updated weights for policy 1, policy_version 58330 (0.0007) [2023-10-14 20:03:38,101][61552] Updated weights for policy 0, policy_version 58502 (0.0007) [2023-10-14 20:03:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 119635968. Throughput: 0: 1673.9, 1: 1697.5. Samples: 29921228. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:03:38,344][60425] Avg episode reward: [(0, '75.120'), (1, '73.950')] [2023-10-14 20:03:38,470][61552] Updated weights for policy 0, policy_version 58512 (0.0007) [2023-10-14 20:03:38,834][61552] Updated weights for policy 0, policy_version 58522 (0.0009) [2023-10-14 20:03:39,164][61585] Updated weights for policy 1, policy_version 58340 (0.0008) [2023-10-14 20:03:39,536][61585] Updated weights for policy 1, policy_version 58350 (0.0009) [2023-10-14 20:03:39,901][61585] Updated weights for policy 1, policy_version 58360 (0.0008) [2023-10-14 20:03:42,837][61552] Updated weights for policy 0, policy_version 58532 (0.0008) [2023-10-14 20:03:43,209][61552] Updated weights for policy 0, policy_version 58542 (0.0007) [2023-10-14 20:03:43,344][60425] Fps is (10 sec: 13106.7, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 119701504. Throughput: 0: 1671.9, 1: 1690.7. Samples: 29941686. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:03:43,345][60425] Avg episode reward: [(0, '72.800'), (1, '73.610')] [2023-10-14 20:03:43,354][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000058368_59768832.pth... [2023-10-14 20:03:43,393][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000056800_58163200.pth [2023-10-14 20:03:43,574][61552] Updated weights for policy 0, policy_version 58552 (0.0007) [2023-10-14 20:03:43,867][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000058560_59965440.pth... [2023-10-14 20:03:43,906][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000056992_58359808.pth [2023-10-14 20:03:43,971][61585] Updated weights for policy 1, policy_version 58370 (0.0010) [2023-10-14 20:03:44,330][61585] Updated weights for policy 1, policy_version 58380 (0.0009) [2023-10-14 20:03:44,698][61585] Updated weights for policy 1, policy_version 58390 (0.0009) [2023-10-14 20:03:45,062][61585] Updated weights for policy 1, policy_version 58400 (0.0009) [2023-10-14 20:03:47,592][61552] Updated weights for policy 0, policy_version 58562 (0.0010) [2023-10-14 20:03:47,959][61552] Updated weights for policy 0, policy_version 58572 (0.0007) [2023-10-14 20:03:48,322][61552] Updated weights for policy 0, policy_version 58582 (0.0010) [2023-10-14 20:03:48,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 119767040. Throughput: 0: 1676.0, 1: 1676.8. Samples: 29950952. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:03:48,344][60425] Avg episode reward: [(0, '76.140'), (1, '73.060')] [2023-10-14 20:03:48,692][61552] Updated weights for policy 0, policy_version 58592 (0.0008) [2023-10-14 20:03:49,057][61585] Updated weights for policy 1, policy_version 58410 (0.0011) [2023-10-14 20:03:49,425][61585] Updated weights for policy 1, policy_version 58420 (0.0007) [2023-10-14 20:03:49,796][61585] Updated weights for policy 1, policy_version 58430 (0.0008) [2023-10-14 20:03:52,745][61552] Updated weights for policy 0, policy_version 58602 (0.0007) [2023-10-14 20:03:53,112][61552] Updated weights for policy 0, policy_version 58612 (0.0010) [2023-10-14 20:03:53,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 119832576. Throughput: 0: 1678.9, 1: 1691.5. Samples: 29971590. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:03:53,344][60425] Avg episode reward: [(0, '74.790'), (1, '70.060')] [2023-10-14 20:03:53,486][61552] Updated weights for policy 0, policy_version 58622 (0.0008) [2023-10-14 20:03:53,888][61585] Updated weights for policy 1, policy_version 58440 (0.0008) [2023-10-14 20:03:54,248][61585] Updated weights for policy 1, policy_version 58450 (0.0010) [2023-10-14 20:03:54,627][61585] Updated weights for policy 1, policy_version 58460 (0.0009) [2023-10-14 20:03:57,539][61552] Updated weights for policy 0, policy_version 58632 (0.0009) [2023-10-14 20:03:57,902][61552] Updated weights for policy 0, policy_version 58642 (0.0009) [2023-10-14 20:03:58,263][61552] Updated weights for policy 0, policy_version 58652 (0.0007) [2023-10-14 20:03:58,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 119898112. Throughput: 0: 1669.9, 1: 1693.0. Samples: 29991760. Policy #0 lag: (min: 31.0, avg: 40.6, max: 63.0) [2023-10-14 20:03:58,344][60425] Avg episode reward: [(0, '76.470'), (1, '73.800')] [2023-10-14 20:03:58,687][61585] Updated weights for policy 1, policy_version 58470 (0.0009) [2023-10-14 20:03:59,054][61585] Updated weights for policy 1, policy_version 58480 (0.0008) [2023-10-14 20:03:59,414][61585] Updated weights for policy 1, policy_version 58490 (0.0008) [2023-10-14 20:04:02,257][61552] Updated weights for policy 0, policy_version 58662 (0.0008) [2023-10-14 20:04:02,631][61552] Updated weights for policy 0, policy_version 58672 (0.0008) [2023-10-14 20:04:02,997][61552] Updated weights for policy 0, policy_version 58682 (0.0007) [2023-10-14 20:04:03,343][60425] Fps is (10 sec: 16384.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 119996416. Throughput: 0: 1681.6, 1: 1687.6. Samples: 30001508. Policy #0 lag: (min: 31.0, avg: 40.6, max: 63.0) [2023-10-14 20:04:03,344][60425] Avg episode reward: [(0, '74.510'), (1, '73.740')] [2023-10-14 20:04:03,613][61585] Updated weights for policy 1, policy_version 58500 (0.0010) [2023-10-14 20:04:03,976][61585] Updated weights for policy 1, policy_version 58510 (0.0009) [2023-10-14 20:04:04,347][61585] Updated weights for policy 1, policy_version 58520 (0.0008) [2023-10-14 20:04:07,087][61552] Updated weights for policy 0, policy_version 58692 (0.0008) [2023-10-14 20:04:07,489][61552] Updated weights for policy 0, policy_version 58702 (0.0011) [2023-10-14 20:04:07,866][61552] Updated weights for policy 0, policy_version 58712 (0.0009) [2023-10-14 20:04:08,343][60425] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 120061952. Throughput: 0: 1684.5, 1: 1686.8. Samples: 30021820. Policy #0 lag: (min: 31.0, avg: 40.6, max: 63.0) [2023-10-14 20:04:08,344][60425] Avg episode reward: [(0, '75.070'), (1, '72.980')] [2023-10-14 20:04:08,409][61585] Updated weights for policy 1, policy_version 58530 (0.0009) [2023-10-14 20:04:08,786][61585] Updated weights for policy 1, policy_version 58540 (0.0008) [2023-10-14 20:04:09,164][61585] Updated weights for policy 1, policy_version 58550 (0.0007) [2023-10-14 20:04:09,529][61585] Updated weights for policy 1, policy_version 58560 (0.0009) [2023-10-14 20:04:11,958][61552] Updated weights for policy 0, policy_version 58722 (0.0009) [2023-10-14 20:04:12,330][61552] Updated weights for policy 0, policy_version 58732 (0.0008) [2023-10-14 20:04:12,693][61552] Updated weights for policy 0, policy_version 58742 (0.0009) [2023-10-14 20:04:13,068][61552] Updated weights for policy 0, policy_version 58752 (0.0008) [2023-10-14 20:04:13,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 120127488. Throughput: 0: 1661.4, 1: 1689.8. Samples: 30041612. Policy #0 lag: (min: 31.0, avg: 40.6, max: 63.0) [2023-10-14 20:04:13,344][60425] Avg episode reward: [(0, '76.910'), (1, '74.190')] [2023-10-14 20:04:13,461][61585] Updated weights for policy 1, policy_version 58570 (0.0010) [2023-10-14 20:04:13,831][61585] Updated weights for policy 1, policy_version 58580 (0.0009) [2023-10-14 20:04:14,195][61585] Updated weights for policy 1, policy_version 58590 (0.0007) [2023-10-14 20:04:17,083][61552] Updated weights for policy 0, policy_version 58762 (0.0008) [2023-10-14 20:04:17,445][61552] Updated weights for policy 0, policy_version 58772 (0.0009) [2023-10-14 20:04:17,815][61552] Updated weights for policy 0, policy_version 58782 (0.0009) [2023-10-14 20:04:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 120193024. Throughput: 0: 1679.5, 1: 1685.2. Samples: 30051510. Policy #0 lag: (min: 31.0, avg: 40.6, max: 63.0) [2023-10-14 20:04:18,344][60425] Avg episode reward: [(0, '74.660'), (1, '72.260')] [2023-10-14 20:04:18,375][61585] Updated weights for policy 1, policy_version 58600 (0.0009) [2023-10-14 20:04:18,747][61585] Updated weights for policy 1, policy_version 58610 (0.0008) [2023-10-14 20:04:19,124][61585] Updated weights for policy 1, policy_version 58620 (0.0011) [2023-10-14 20:04:21,813][61552] Updated weights for policy 0, policy_version 58792 (0.0009) [2023-10-14 20:04:22,179][61552] Updated weights for policy 0, policy_version 58802 (0.0007) [2023-10-14 20:04:22,541][61552] Updated weights for policy 0, policy_version 58812 (0.0009) [2023-10-14 20:04:23,204][61585] Updated weights for policy 1, policy_version 58630 (0.0010) [2023-10-14 20:04:23,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 120258560. Throughput: 0: 1674.7, 1: 1671.8. Samples: 30071818. Policy #0 lag: (min: 31.0, avg: 40.6, max: 63.0) [2023-10-14 20:04:23,344][60425] Avg episode reward: [(0, '76.590'), (1, '72.950')] [2023-10-14 20:04:23,574][61585] Updated weights for policy 1, policy_version 58640 (0.0009) [2023-10-14 20:04:23,933][61585] Updated weights for policy 1, policy_version 58650 (0.0011) [2023-10-14 20:04:26,721][61552] Updated weights for policy 0, policy_version 58822 (0.0008) [2023-10-14 20:04:27,084][61552] Updated weights for policy 0, policy_version 58832 (0.0008) [2023-10-14 20:04:27,463][61552] Updated weights for policy 0, policy_version 58842 (0.0010) [2023-10-14 20:04:28,181][61585] Updated weights for policy 1, policy_version 58660 (0.0009) [2023-10-14 20:04:28,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 120324096. Throughput: 0: 1652.9, 1: 1674.4. Samples: 30091414. Policy #0 lag: (min: 31.0, avg: 40.6, max: 63.0) [2023-10-14 20:04:28,344][60425] Avg episode reward: [(0, '74.710'), (1, '71.760')] [2023-10-14 20:04:28,548][61585] Updated weights for policy 1, policy_version 58670 (0.0008) [2023-10-14 20:04:28,906][61585] Updated weights for policy 1, policy_version 58680 (0.0008) [2023-10-14 20:04:31,476][61552] Updated weights for policy 0, policy_version 58852 (0.0010) [2023-10-14 20:04:31,857][61552] Updated weights for policy 0, policy_version 58862 (0.0010) [2023-10-14 20:04:32,223][61552] Updated weights for policy 0, policy_version 58872 (0.0010) [2023-10-14 20:04:32,915][61585] Updated weights for policy 1, policy_version 58690 (0.0007) [2023-10-14 20:04:33,270][61585] Updated weights for policy 1, policy_version 58700 (0.0009) [2023-10-14 20:04:33,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 120389632. Throughput: 0: 1679.7, 1: 1671.4. Samples: 30101752. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:04:33,344][60425] Avg episode reward: [(0, '77.890'), (1, '69.660')] [2023-10-14 20:04:33,644][61585] Updated weights for policy 1, policy_version 58710 (0.0008) [2023-10-14 20:04:34,009][61585] Updated weights for policy 1, policy_version 58720 (0.0008) [2023-10-14 20:04:36,456][61552] Updated weights for policy 0, policy_version 58882 (0.0010) [2023-10-14 20:04:36,826][61552] Updated weights for policy 0, policy_version 58892 (0.0009) [2023-10-14 20:04:37,203][61552] Updated weights for policy 0, policy_version 58902 (0.0009) [2023-10-14 20:04:37,567][61552] Updated weights for policy 0, policy_version 58912 (0.0008) [2023-10-14 20:04:38,096][61585] Updated weights for policy 1, policy_version 58730 (0.0010) [2023-10-14 20:04:38,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 120455168. Throughput: 0: 1666.1, 1: 1676.8. Samples: 30122016. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:04:38,344][60425] Avg episode reward: [(0, '78.120'), (1, '68.620')] [2023-10-14 20:04:38,464][61585] Updated weights for policy 1, policy_version 58740 (0.0010) [2023-10-14 20:04:38,830][61585] Updated weights for policy 1, policy_version 58750 (0.0009) [2023-10-14 20:04:41,646][61552] Updated weights for policy 0, policy_version 58922 (0.0008) [2023-10-14 20:04:42,016][61552] Updated weights for policy 0, policy_version 58932 (0.0007) [2023-10-14 20:04:42,392][61552] Updated weights for policy 0, policy_version 58942 (0.0007) [2023-10-14 20:04:43,006][61585] Updated weights for policy 1, policy_version 58760 (0.0008) [2023-10-14 20:04:43,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 120520704. Throughput: 0: 1661.6, 1: 1672.8. Samples: 30141806. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:04:43,344][60425] Avg episode reward: [(0, '77.300'), (1, '72.870')] [2023-10-14 20:04:43,368][61585] Updated weights for policy 1, policy_version 58770 (0.0009) [2023-10-14 20:04:43,741][61585] Updated weights for policy 1, policy_version 58780 (0.0008) [2023-10-14 20:04:46,554][61552] Updated weights for policy 0, policy_version 58952 (0.0009) [2023-10-14 20:04:46,918][61552] Updated weights for policy 0, policy_version 58962 (0.0011) [2023-10-14 20:04:47,291][61552] Updated weights for policy 0, policy_version 58972 (0.0009) [2023-10-14 20:04:47,797][61585] Updated weights for policy 1, policy_version 58790 (0.0008) [2023-10-14 20:04:48,174][61585] Updated weights for policy 1, policy_version 58800 (0.0008) [2023-10-14 20:04:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 120586240. Throughput: 0: 1674.4, 1: 1674.5. Samples: 30152212. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:04:48,344][60425] Avg episode reward: [(0, '78.280'), (1, '69.530')] [2023-10-14 20:04:48,531][61585] Updated weights for policy 1, policy_version 58810 (0.0009) [2023-10-14 20:04:51,302][61552] Updated weights for policy 0, policy_version 58982 (0.0008) [2023-10-14 20:04:51,658][61552] Updated weights for policy 0, policy_version 58992 (0.0009) [2023-10-14 20:04:52,024][61552] Updated weights for policy 0, policy_version 59002 (0.0009) [2023-10-14 20:04:52,721][61585] Updated weights for policy 1, policy_version 58820 (0.0008) [2023-10-14 20:04:53,090][61585] Updated weights for policy 1, policy_version 58830 (0.0008) [2023-10-14 20:04:53,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 120651776. Throughput: 0: 1660.2, 1: 1683.1. Samples: 30172268. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:04:53,344][60425] Avg episode reward: [(0, '78.430'), (1, '71.190')] [2023-10-14 20:04:53,462][61585] Updated weights for policy 1, policy_version 58840 (0.0008) [2023-10-14 20:04:56,177][61552] Updated weights for policy 0, policy_version 59012 (0.0010) [2023-10-14 20:04:56,561][61552] Updated weights for policy 0, policy_version 59022 (0.0011) [2023-10-14 20:04:56,926][61552] Updated weights for policy 0, policy_version 59032 (0.0011) [2023-10-14 20:04:57,574][61585] Updated weights for policy 1, policy_version 58850 (0.0009) [2023-10-14 20:04:57,942][61585] Updated weights for policy 1, policy_version 58860 (0.0007) [2023-10-14 20:04:58,318][61585] Updated weights for policy 1, policy_version 58870 (0.0008) [2023-10-14 20:04:58,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 120717312. Throughput: 0: 1669.2, 1: 1668.6. Samples: 30191812. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:04:58,345][60425] Avg episode reward: [(0, '77.780'), (1, '73.030')] [2023-10-14 20:04:58,686][61585] Updated weights for policy 1, policy_version 58880 (0.0011) [2023-10-14 20:05:01,163][61552] Updated weights for policy 0, policy_version 59042 (0.0008) [2023-10-14 20:05:01,536][61552] Updated weights for policy 0, policy_version 59052 (0.0008) [2023-10-14 20:05:01,895][61552] Updated weights for policy 0, policy_version 59062 (0.0007) [2023-10-14 20:05:02,264][61552] Updated weights for policy 0, policy_version 59072 (0.0011) [2023-10-14 20:05:02,863][61585] Updated weights for policy 1, policy_version 58890 (0.0010) [2023-10-14 20:05:03,224][61585] Updated weights for policy 1, policy_version 58900 (0.0011) [2023-10-14 20:05:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 120782848. Throughput: 0: 1675.4, 1: 1676.0. Samples: 30202326. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:05:03,344][60425] Avg episode reward: [(0, '76.730'), (1, '71.280')] [2023-10-14 20:05:03,593][61585] Updated weights for policy 1, policy_version 58910 (0.0008) [2023-10-14 20:05:06,380][61552] Updated weights for policy 0, policy_version 59082 (0.0008) [2023-10-14 20:05:06,748][61552] Updated weights for policy 0, policy_version 59092 (0.0008) [2023-10-14 20:05:07,118][61552] Updated weights for policy 0, policy_version 59102 (0.0008) [2023-10-14 20:05:07,809][61585] Updated weights for policy 1, policy_version 58920 (0.0009) [2023-10-14 20:05:08,165][61585] Updated weights for policy 1, policy_version 58930 (0.0010) [2023-10-14 20:05:08,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 120848384. Throughput: 0: 1659.7, 1: 1682.1. Samples: 30222198. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:05:08,344][60425] Avg episode reward: [(0, '76.900'), (1, '73.430')] [2023-10-14 20:05:08,526][61585] Updated weights for policy 1, policy_version 58940 (0.0011) [2023-10-14 20:05:11,060][61552] Updated weights for policy 0, policy_version 59112 (0.0009) [2023-10-14 20:05:11,422][61552] Updated weights for policy 0, policy_version 59122 (0.0008) [2023-10-14 20:05:11,796][61552] Updated weights for policy 0, policy_version 59132 (0.0009) [2023-10-14 20:05:12,672][61585] Updated weights for policy 1, policy_version 58950 (0.0010) [2023-10-14 20:05:13,038][61585] Updated weights for policy 1, policy_version 58960 (0.0008) [2023-10-14 20:05:13,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 120913920. Throughput: 0: 1673.5, 1: 1672.0. Samples: 30241960. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:05:13,344][60425] Avg episode reward: [(0, '77.740'), (1, '77.430')] [2023-10-14 20:05:13,397][61585] Updated weights for policy 1, policy_version 58970 (0.0008) [2023-10-14 20:05:15,886][61552] Updated weights for policy 0, policy_version 59142 (0.0011) [2023-10-14 20:05:16,251][61552] Updated weights for policy 0, policy_version 59152 (0.0009) [2023-10-14 20:05:16,626][61552] Updated weights for policy 0, policy_version 59162 (0.0007) [2023-10-14 20:05:17,246][61585] Updated weights for policy 1, policy_version 58980 (0.0007) [2023-10-14 20:05:17,611][61585] Updated weights for policy 1, policy_version 58990 (0.0007) [2023-10-14 20:05:17,979][61585] Updated weights for policy 1, policy_version 59000 (0.0010) [2023-10-14 20:05:18,343][60425] Fps is (10 sec: 16383.9, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 121012224. Throughput: 0: 1672.9, 1: 1679.7. Samples: 30252618. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:05:18,344][60425] Avg episode reward: [(0, '73.740'), (1, '75.280')] [2023-10-14 20:05:20,659][61552] Updated weights for policy 0, policy_version 59172 (0.0009) [2023-10-14 20:05:21,025][61552] Updated weights for policy 0, policy_version 59182 (0.0009) [2023-10-14 20:05:21,388][61552] Updated weights for policy 0, policy_version 59192 (0.0008) [2023-10-14 20:05:22,233][61585] Updated weights for policy 1, policy_version 59010 (0.0007) [2023-10-14 20:05:22,593][61585] Updated weights for policy 1, policy_version 59020 (0.0008) [2023-10-14 20:05:22,965][61585] Updated weights for policy 1, policy_version 59030 (0.0007) [2023-10-14 20:05:23,328][61585] Updated weights for policy 1, policy_version 59040 (0.0008) [2023-10-14 20:05:23,343][60425] Fps is (10 sec: 16384.0, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 121077760. Throughput: 0: 1661.2, 1: 1676.1. Samples: 30272192. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:05:23,344][60425] Avg episode reward: [(0, '73.460'), (1, '73.430')] [2023-10-14 20:05:25,390][61552] Updated weights for policy 0, policy_version 59202 (0.0008) [2023-10-14 20:05:25,764][61552] Updated weights for policy 0, policy_version 59212 (0.0010) [2023-10-14 20:05:26,127][61552] Updated weights for policy 0, policy_version 59222 (0.0008) [2023-10-14 20:05:26,495][61552] Updated weights for policy 0, policy_version 59232 (0.0009) [2023-10-14 20:05:27,415][61585] Updated weights for policy 1, policy_version 59050 (0.0009) [2023-10-14 20:05:27,782][61585] Updated weights for policy 1, policy_version 59060 (0.0009) [2023-10-14 20:05:28,145][61585] Updated weights for policy 1, policy_version 59070 (0.0009) [2023-10-14 20:05:28,344][60425] Fps is (10 sec: 13106.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 121143296. Throughput: 0: 1679.3, 1: 1658.3. Samples: 30291998. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:05:28,345][60425] Avg episode reward: [(0, '73.050'), (1, '75.920')] [2023-10-14 20:05:30,492][61552] Updated weights for policy 0, policy_version 59242 (0.0009) [2023-10-14 20:05:30,868][61552] Updated weights for policy 0, policy_version 59252 (0.0009) [2023-10-14 20:05:31,228][61552] Updated weights for policy 0, policy_version 59262 (0.0008) [2023-10-14 20:05:32,150][61585] Updated weights for policy 1, policy_version 59080 (0.0011) [2023-10-14 20:05:32,523][61585] Updated weights for policy 1, policy_version 59090 (0.0010) [2023-10-14 20:05:32,883][61585] Updated weights for policy 1, policy_version 59100 (0.0007) [2023-10-14 20:05:33,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 121208832. Throughput: 0: 1667.9, 1: 1670.3. Samples: 30302428. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:05:33,344][60425] Avg episode reward: [(0, '75.820'), (1, '74.660')] [2023-10-14 20:05:35,392][61552] Updated weights for policy 0, policy_version 59272 (0.0008) [2023-10-14 20:05:35,768][61552] Updated weights for policy 0, policy_version 59282 (0.0009) [2023-10-14 20:05:36,134][61552] Updated weights for policy 0, policy_version 59292 (0.0008) [2023-10-14 20:05:36,962][61585] Updated weights for policy 1, policy_version 59110 (0.0008) [2023-10-14 20:05:37,326][61585] Updated weights for policy 1, policy_version 59120 (0.0008) [2023-10-14 20:05:37,683][61585] Updated weights for policy 1, policy_version 59130 (0.0010) [2023-10-14 20:05:38,343][60425] Fps is (10 sec: 13107.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 121274368. Throughput: 0: 1668.9, 1: 1665.3. Samples: 30322306. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:05:38,344][60425] Avg episode reward: [(0, '73.740'), (1, '76.960')] [2023-10-14 20:05:40,218][61552] Updated weights for policy 0, policy_version 59302 (0.0008) [2023-10-14 20:05:40,592][61552] Updated weights for policy 0, policy_version 59312 (0.0009) [2023-10-14 20:05:40,955][61552] Updated weights for policy 0, policy_version 59322 (0.0009) [2023-10-14 20:05:41,814][61585] Updated weights for policy 1, policy_version 59140 (0.0009) [2023-10-14 20:05:42,175][61585] Updated weights for policy 1, policy_version 59150 (0.0009) [2023-10-14 20:05:42,536][61585] Updated weights for policy 1, policy_version 59160 (0.0007) [2023-10-14 20:05:43,344][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 121339904. Throughput: 0: 1687.3, 1: 1655.0. Samples: 30342214. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:05:43,345][60425] Avg episode reward: [(0, '74.810'), (1, '75.550')] [2023-10-14 20:05:43,357][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000059328_60751872.pth... [2023-10-14 20:05:43,358][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000059168_60588032.pth... [2023-10-14 20:05:43,398][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000057760_59146240.pth [2023-10-14 20:05:43,406][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000057600_58982400.pth [2023-10-14 20:05:45,013][61552] Updated weights for policy 0, policy_version 59332 (0.0009) [2023-10-14 20:05:45,410][61552] Updated weights for policy 0, policy_version 59342 (0.0010) [2023-10-14 20:05:45,778][61552] Updated weights for policy 0, policy_version 59352 (0.0010) [2023-10-14 20:05:46,645][61585] Updated weights for policy 1, policy_version 59170 (0.0008) [2023-10-14 20:05:47,009][61585] Updated weights for policy 1, policy_version 59180 (0.0009) [2023-10-14 20:05:47,373][61585] Updated weights for policy 1, policy_version 59190 (0.0007) [2023-10-14 20:05:47,733][61585] Updated weights for policy 1, policy_version 59200 (0.0009) [2023-10-14 20:05:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 121405440. Throughput: 0: 1668.8, 1: 1670.3. Samples: 30352586. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:05:48,344][60425] Avg episode reward: [(0, '73.460'), (1, '74.590')] [2023-10-14 20:05:49,934][61552] Updated weights for policy 0, policy_version 59362 (0.0010) [2023-10-14 20:05:50,306][61552] Updated weights for policy 0, policy_version 59372 (0.0009) [2023-10-14 20:05:50,670][61552] Updated weights for policy 0, policy_version 59382 (0.0010) [2023-10-14 20:05:51,044][61552] Updated weights for policy 0, policy_version 59392 (0.0008) [2023-10-14 20:05:51,905][61585] Updated weights for policy 1, policy_version 59210 (0.0009) [2023-10-14 20:05:52,272][61585] Updated weights for policy 1, policy_version 59220 (0.0011) [2023-10-14 20:05:52,637][61585] Updated weights for policy 1, policy_version 59230 (0.0011) [2023-10-14 20:05:53,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 121470976. Throughput: 0: 1668.8, 1: 1662.2. Samples: 30372094. Policy #0 lag: (min: 29.0, avg: 34.9, max: 61.0) [2023-10-14 20:05:53,344][60425] Avg episode reward: [(0, '77.970'), (1, '70.240')] [2023-10-14 20:05:55,024][61552] Updated weights for policy 0, policy_version 59402 (0.0010) [2023-10-14 20:05:55,391][61552] Updated weights for policy 0, policy_version 59412 (0.0007) [2023-10-14 20:05:55,767][61552] Updated weights for policy 0, policy_version 59422 (0.0008) [2023-10-14 20:05:56,814][61585] Updated weights for policy 1, policy_version 59240 (0.0009) [2023-10-14 20:05:57,188][61585] Updated weights for policy 1, policy_version 59250 (0.0007) [2023-10-14 20:05:57,551][61585] Updated weights for policy 1, policy_version 59260 (0.0008) [2023-10-14 20:05:58,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 121536512. Throughput: 0: 1680.4, 1: 1644.4. Samples: 30391576. Policy #0 lag: (min: 29.0, avg: 34.9, max: 61.0) [2023-10-14 20:05:58,344][60425] Avg episode reward: [(0, '75.800'), (1, '73.450')] [2023-10-14 20:05:59,941][61552] Updated weights for policy 0, policy_version 59432 (0.0011) [2023-10-14 20:06:00,315][61552] Updated weights for policy 0, policy_version 59442 (0.0010) [2023-10-14 20:06:00,686][61552] Updated weights for policy 0, policy_version 59452 (0.0010) [2023-10-14 20:06:01,627][61585] Updated weights for policy 1, policy_version 59270 (0.0009) [2023-10-14 20:06:01,996][61585] Updated weights for policy 1, policy_version 59280 (0.0007) [2023-10-14 20:06:02,363][61585] Updated weights for policy 1, policy_version 59290 (0.0007) [2023-10-14 20:06:03,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 121602048. Throughput: 0: 1653.8, 1: 1662.9. Samples: 30401870. Policy #0 lag: (min: 29.0, avg: 34.9, max: 61.0) [2023-10-14 20:06:03,345][60425] Avg episode reward: [(0, '74.410'), (1, '73.100')] [2023-10-14 20:06:04,910][61552] Updated weights for policy 0, policy_version 59462 (0.0009) [2023-10-14 20:06:05,287][61552] Updated weights for policy 0, policy_version 59472 (0.0008) [2023-10-14 20:06:05,645][61552] Updated weights for policy 0, policy_version 59482 (0.0007) [2023-10-14 20:06:06,558][61585] Updated weights for policy 1, policy_version 59300 (0.0009) [2023-10-14 20:06:06,923][61585] Updated weights for policy 1, policy_version 59310 (0.0009) [2023-10-14 20:06:07,288][61585] Updated weights for policy 1, policy_version 59320 (0.0007) [2023-10-14 20:06:08,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 121667584. Throughput: 0: 1672.1, 1: 1653.5. Samples: 30421842. Policy #0 lag: (min: 29.0, avg: 34.9, max: 61.0) [2023-10-14 20:06:08,344][60425] Avg episode reward: [(0, '74.890'), (1, '74.740')] [2023-10-14 20:06:09,903][61552] Updated weights for policy 0, policy_version 59492 (0.0008) [2023-10-14 20:06:10,268][61552] Updated weights for policy 0, policy_version 59502 (0.0007) [2023-10-14 20:06:10,643][61552] Updated weights for policy 0, policy_version 59512 (0.0007) [2023-10-14 20:06:11,203][61585] Updated weights for policy 1, policy_version 59330 (0.0009) [2023-10-14 20:06:11,566][61585] Updated weights for policy 1, policy_version 59340 (0.0008) [2023-10-14 20:06:11,928][61585] Updated weights for policy 1, policy_version 59350 (0.0009) [2023-10-14 20:06:12,295][61585] Updated weights for policy 1, policy_version 59360 (0.0008) [2023-10-14 20:06:13,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 121733120. Throughput: 0: 1671.1, 1: 1658.1. Samples: 30441808. Policy #0 lag: (min: 29.0, avg: 34.9, max: 61.0) [2023-10-14 20:06:13,344][60425] Avg episode reward: [(0, '74.330'), (1, '74.640')] [2023-10-14 20:06:14,763][61552] Updated weights for policy 0, policy_version 59522 (0.0007) [2023-10-14 20:06:15,133][61552] Updated weights for policy 0, policy_version 59532 (0.0008) [2023-10-14 20:06:15,506][61552] Updated weights for policy 0, policy_version 59542 (0.0009) [2023-10-14 20:06:15,874][61552] Updated weights for policy 0, policy_version 59552 (0.0009) [2023-10-14 20:06:16,267][61585] Updated weights for policy 1, policy_version 59370 (0.0009) [2023-10-14 20:06:16,641][61585] Updated weights for policy 1, policy_version 59380 (0.0009) [2023-10-14 20:06:17,014][61585] Updated weights for policy 1, policy_version 59390 (0.0009) [2023-10-14 20:06:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 121798656. Throughput: 0: 1658.3, 1: 1679.6. Samples: 30452632. Policy #0 lag: (min: 29.0, avg: 34.9, max: 61.0) [2023-10-14 20:06:18,344][60425] Avg episode reward: [(0, '76.460'), (1, '72.200')] [2023-10-14 20:06:20,058][61552] Updated weights for policy 0, policy_version 59562 (0.0009) [2023-10-14 20:06:20,430][61552] Updated weights for policy 0, policy_version 59572 (0.0009) [2023-10-14 20:06:20,798][61552] Updated weights for policy 0, policy_version 59582 (0.0008) [2023-10-14 20:06:21,085][61585] Updated weights for policy 1, policy_version 59400 (0.0009) [2023-10-14 20:06:21,452][61585] Updated weights for policy 1, policy_version 59410 (0.0010) [2023-10-14 20:06:21,814][61585] Updated weights for policy 1, policy_version 59420 (0.0007) [2023-10-14 20:06:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 121864192. Throughput: 0: 1665.7, 1: 1660.1. Samples: 30471968. Policy #0 lag: (min: 29.0, avg: 34.9, max: 61.0) [2023-10-14 20:06:23,344][60425] Avg episode reward: [(0, '73.730'), (1, '68.730')] [2023-10-14 20:06:24,840][61552] Updated weights for policy 0, policy_version 59592 (0.0008) [2023-10-14 20:06:25,214][61552] Updated weights for policy 0, policy_version 59602 (0.0007) [2023-10-14 20:06:25,581][61552] Updated weights for policy 0, policy_version 59612 (0.0008) [2023-10-14 20:06:25,944][61585] Updated weights for policy 1, policy_version 59430 (0.0009) [2023-10-14 20:06:26,306][61585] Updated weights for policy 1, policy_version 59440 (0.0011) [2023-10-14 20:06:26,675][61585] Updated weights for policy 1, policy_version 59450 (0.0007) [2023-10-14 20:06:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 121929728. Throughput: 0: 1660.7, 1: 1674.8. Samples: 30492312. Policy #0 lag: (min: 29.0, avg: 34.9, max: 61.0) [2023-10-14 20:06:28,344][60425] Avg episode reward: [(0, '74.080'), (1, '72.660')] [2023-10-14 20:06:29,730][61552] Updated weights for policy 0, policy_version 59622 (0.0009) [2023-10-14 20:06:30,099][61552] Updated weights for policy 0, policy_version 59632 (0.0007) [2023-10-14 20:06:30,473][61552] Updated weights for policy 0, policy_version 59642 (0.0007) [2023-10-14 20:06:30,657][61585] Updated weights for policy 1, policy_version 59460 (0.0007) [2023-10-14 20:06:31,014][61585] Updated weights for policy 1, policy_version 59470 (0.0008) [2023-10-14 20:06:31,379][61585] Updated weights for policy 1, policy_version 59480 (0.0008) [2023-10-14 20:06:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 121995264. Throughput: 0: 1652.4, 1: 1679.2. Samples: 30502508. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-14 20:06:33,344][60425] Avg episode reward: [(0, '71.170'), (1, '74.320')] [2023-10-14 20:06:34,388][61552] Updated weights for policy 0, policy_version 59652 (0.0007) [2023-10-14 20:06:34,757][61552] Updated weights for policy 0, policy_version 59662 (0.0010) [2023-10-14 20:06:35,123][61552] Updated weights for policy 0, policy_version 59672 (0.0008) [2023-10-14 20:06:35,494][61585] Updated weights for policy 1, policy_version 59490 (0.0007) [2023-10-14 20:06:35,861][61585] Updated weights for policy 1, policy_version 59500 (0.0007) [2023-10-14 20:06:36,225][61585] Updated weights for policy 1, policy_version 59510 (0.0009) [2023-10-14 20:06:36,592][61585] Updated weights for policy 1, policy_version 59520 (0.0009) [2023-10-14 20:06:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 122060800. Throughput: 0: 1671.1, 1: 1666.4. Samples: 30522280. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-14 20:06:38,344][60425] Avg episode reward: [(0, '74.600'), (1, '70.300')] [2023-10-14 20:06:39,345][61552] Updated weights for policy 0, policy_version 59682 (0.0007) [2023-10-14 20:06:39,718][61552] Updated weights for policy 0, policy_version 59692 (0.0009) [2023-10-14 20:06:40,086][61552] Updated weights for policy 0, policy_version 59702 (0.0007) [2023-10-14 20:06:40,458][61552] Updated weights for policy 0, policy_version 59712 (0.0007) [2023-10-14 20:06:40,750][61585] Updated weights for policy 1, policy_version 59530 (0.0010) [2023-10-14 20:06:41,111][61585] Updated weights for policy 1, policy_version 59540 (0.0010) [2023-10-14 20:06:41,476][61585] Updated weights for policy 1, policy_version 59550 (0.0011) [2023-10-14 20:06:43,344][60425] Fps is (10 sec: 13106.7, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 122126336. Throughput: 0: 1667.5, 1: 1691.1. Samples: 30542716. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-14 20:06:43,345][60425] Avg episode reward: [(0, '75.000'), (1, '73.620')] [2023-10-14 20:06:44,599][61552] Updated weights for policy 0, policy_version 59722 (0.0009) [2023-10-14 20:06:44,966][61552] Updated weights for policy 0, policy_version 59732 (0.0007) [2023-10-14 20:06:45,340][61552] Updated weights for policy 0, policy_version 59742 (0.0008) [2023-10-14 20:06:45,728][61585] Updated weights for policy 1, policy_version 59560 (0.0010) [2023-10-14 20:06:46,107][61585] Updated weights for policy 1, policy_version 59570 (0.0010) [2023-10-14 20:06:46,466][61585] Updated weights for policy 1, policy_version 59580 (0.0009) [2023-10-14 20:06:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 122191872. Throughput: 0: 1666.0, 1: 1681.1. Samples: 30552486. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-14 20:06:48,344][60425] Avg episode reward: [(0, '73.130'), (1, '71.380')] [2023-10-14 20:06:49,277][61552] Updated weights for policy 0, policy_version 59752 (0.0008) [2023-10-14 20:06:49,634][61552] Updated weights for policy 0, policy_version 59762 (0.0008) [2023-10-14 20:06:50,002][61552] Updated weights for policy 0, policy_version 59772 (0.0009) [2023-10-14 20:06:50,438][61585] Updated weights for policy 1, policy_version 59590 (0.0009) [2023-10-14 20:06:50,813][61585] Updated weights for policy 1, policy_version 59600 (0.0008) [2023-10-14 20:06:51,174][61585] Updated weights for policy 1, policy_version 59610 (0.0010) [2023-10-14 20:06:53,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 122257408. Throughput: 0: 1674.1, 1: 1670.4. Samples: 30572344. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-14 20:06:53,344][60425] Avg episode reward: [(0, '75.290'), (1, '71.430')] [2023-10-14 20:06:54,147][61552] Updated weights for policy 0, policy_version 59782 (0.0009) [2023-10-14 20:06:54,516][61552] Updated weights for policy 0, policy_version 59792 (0.0008) [2023-10-14 20:06:54,883][61552] Updated weights for policy 0, policy_version 59802 (0.0007) [2023-10-14 20:06:55,288][61585] Updated weights for policy 1, policy_version 59620 (0.0008) [2023-10-14 20:06:55,653][61585] Updated weights for policy 1, policy_version 59630 (0.0009) [2023-10-14 20:06:56,023][61585] Updated weights for policy 1, policy_version 59640 (0.0008) [2023-10-14 20:06:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 122322944. Throughput: 0: 1675.6, 1: 1684.7. Samples: 30593020. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-14 20:06:58,344][60425] Avg episode reward: [(0, '74.180'), (1, '71.970')] [2023-10-14 20:06:59,000][61552] Updated weights for policy 0, policy_version 59812 (0.0007) [2023-10-14 20:06:59,369][61552] Updated weights for policy 0, policy_version 59822 (0.0009) [2023-10-14 20:06:59,733][61552] Updated weights for policy 0, policy_version 59832 (0.0010) [2023-10-14 20:07:00,167][61585] Updated weights for policy 1, policy_version 59650 (0.0007) [2023-10-14 20:07:00,547][61585] Updated weights for policy 1, policy_version 59660 (0.0007) [2023-10-14 20:07:00,911][61585] Updated weights for policy 1, policy_version 59670 (0.0008) [2023-10-14 20:07:01,267][61585] Updated weights for policy 1, policy_version 59680 (0.0010) [2023-10-14 20:07:03,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 122388480. Throughput: 0: 1670.2, 1: 1663.8. Samples: 30602662. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-14 20:07:03,344][60425] Avg episode reward: [(0, '72.590'), (1, '76.510')] [2023-10-14 20:07:03,692][61552] Updated weights for policy 0, policy_version 59842 (0.0009) [2023-10-14 20:07:04,058][61552] Updated weights for policy 0, policy_version 59852 (0.0009) [2023-10-14 20:07:04,425][61552] Updated weights for policy 0, policy_version 59862 (0.0008) [2023-10-14 20:07:04,802][61552] Updated weights for policy 0, policy_version 59872 (0.0010) [2023-10-14 20:07:05,404][61585] Updated weights for policy 1, policy_version 59690 (0.0009) [2023-10-14 20:07:05,766][61585] Updated weights for policy 1, policy_version 59700 (0.0008) [2023-10-14 20:07:06,114][61585] Updated weights for policy 1, policy_version 59710 (0.0009) [2023-10-14 20:07:08,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 122454016. Throughput: 0: 1682.4, 1: 1667.0. Samples: 30622694. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-14 20:07:08,344][60425] Avg episode reward: [(0, '74.520'), (1, '76.670')] [2023-10-14 20:07:08,594][61552] Updated weights for policy 0, policy_version 59882 (0.0009) [2023-10-14 20:07:08,955][61552] Updated weights for policy 0, policy_version 59892 (0.0009) [2023-10-14 20:07:09,329][61552] Updated weights for policy 0, policy_version 59902 (0.0008) [2023-10-14 20:07:10,170][61585] Updated weights for policy 1, policy_version 59720 (0.0009) [2023-10-14 20:07:10,533][61585] Updated weights for policy 1, policy_version 59730 (0.0008) [2023-10-14 20:07:10,903][61585] Updated weights for policy 1, policy_version 59740 (0.0009) [2023-10-14 20:07:13,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 122519552. Throughput: 0: 1685.8, 1: 1673.7. Samples: 30643490. Policy #0 lag: (min: 31.0, avg: 41.3, max: 63.0) [2023-10-14 20:07:13,344][60425] Avg episode reward: [(0, '77.690'), (1, '73.870')] [2023-10-14 20:07:13,395][61552] Updated weights for policy 0, policy_version 59912 (0.0008) [2023-10-14 20:07:13,766][61552] Updated weights for policy 0, policy_version 59922 (0.0008) [2023-10-14 20:07:14,134][61552] Updated weights for policy 0, policy_version 59932 (0.0010) [2023-10-14 20:07:15,043][61585] Updated weights for policy 1, policy_version 59750 (0.0010) [2023-10-14 20:07:15,410][61585] Updated weights for policy 1, policy_version 59760 (0.0007) [2023-10-14 20:07:15,763][61585] Updated weights for policy 1, policy_version 59770 (0.0007) [2023-10-14 20:07:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 122585088. Throughput: 0: 1685.6, 1: 1655.3. Samples: 30652848. Policy #0 lag: (min: 31.0, avg: 41.3, max: 63.0) [2023-10-14 20:07:18,344][60425] Avg episode reward: [(0, '76.050'), (1, '72.480')] [2023-10-14 20:07:18,419][61552] Updated weights for policy 0, policy_version 59942 (0.0009) [2023-10-14 20:07:18,806][61552] Updated weights for policy 0, policy_version 59952 (0.0007) [2023-10-14 20:07:19,169][61552] Updated weights for policy 0, policy_version 59962 (0.0008) [2023-10-14 20:07:19,871][61585] Updated weights for policy 1, policy_version 59780 (0.0009) [2023-10-14 20:07:20,235][61585] Updated weights for policy 1, policy_version 59790 (0.0009) [2023-10-14 20:07:20,601][61585] Updated weights for policy 1, policy_version 59800 (0.0007) [2023-10-14 20:07:23,287][61552] Updated weights for policy 0, policy_version 59972 (0.0008) [2023-10-14 20:07:23,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 122650624. Throughput: 0: 1681.2, 1: 1668.7. Samples: 30673026. Policy #0 lag: (min: 31.0, avg: 41.3, max: 63.0) [2023-10-14 20:07:23,344][60425] Avg episode reward: [(0, '74.450'), (1, '74.230')] [2023-10-14 20:07:23,650][61552] Updated weights for policy 0, policy_version 59982 (0.0009) [2023-10-14 20:07:24,031][61552] Updated weights for policy 0, policy_version 59992 (0.0007) [2023-10-14 20:07:24,690][61585] Updated weights for policy 1, policy_version 59810 (0.0007) [2023-10-14 20:07:25,053][61585] Updated weights for policy 1, policy_version 59820 (0.0007) [2023-10-14 20:07:25,420][61585] Updated weights for policy 1, policy_version 59830 (0.0010) [2023-10-14 20:07:25,790][61585] Updated weights for policy 1, policy_version 59840 (0.0009) [2023-10-14 20:07:27,920][61552] Updated weights for policy 0, policy_version 60002 (0.0007) [2023-10-14 20:07:28,299][61552] Updated weights for policy 0, policy_version 60012 (0.0010) [2023-10-14 20:07:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 122716160. Throughput: 0: 1683.5, 1: 1667.3. Samples: 30693504. Policy #0 lag: (min: 31.0, avg: 41.3, max: 63.0) [2023-10-14 20:07:28,344][60425] Avg episode reward: [(0, '78.550'), (1, '76.090')] [2023-10-14 20:07:28,673][61552] Updated weights for policy 0, policy_version 60022 (0.0010) [2023-10-14 20:07:29,040][61552] Updated weights for policy 0, policy_version 60032 (0.0011) [2023-10-14 20:07:29,952][61585] Updated weights for policy 1, policy_version 59850 (0.0010) [2023-10-14 20:07:30,317][61585] Updated weights for policy 1, policy_version 59860 (0.0010) [2023-10-14 20:07:30,680][61585] Updated weights for policy 1, policy_version 59870 (0.0010) [2023-10-14 20:07:33,140][61552] Updated weights for policy 0, policy_version 60042 (0.0009) [2023-10-14 20:07:33,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 122781696. Throughput: 0: 1685.5, 1: 1651.9. Samples: 30702668. Policy #0 lag: (min: 31.0, avg: 41.3, max: 63.0) [2023-10-14 20:07:33,344][60425] Avg episode reward: [(0, '78.920'), (1, '73.730')] [2023-10-14 20:07:33,506][61552] Updated weights for policy 0, policy_version 60052 (0.0010) [2023-10-14 20:07:33,880][61552] Updated weights for policy 0, policy_version 60062 (0.0009) [2023-10-14 20:07:34,726][61585] Updated weights for policy 1, policy_version 59880 (0.0010) [2023-10-14 20:07:35,086][61585] Updated weights for policy 1, policy_version 59890 (0.0010) [2023-10-14 20:07:35,447][61585] Updated weights for policy 1, policy_version 59900 (0.0010) [2023-10-14 20:07:37,824][61552] Updated weights for policy 0, policy_version 60072 (0.0009) [2023-10-14 20:07:38,198][61552] Updated weights for policy 0, policy_version 60082 (0.0010) [2023-10-14 20:07:38,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 122847232. Throughput: 0: 1685.7, 1: 1668.1. Samples: 30723266. Policy #0 lag: (min: 31.0, avg: 41.3, max: 63.0) [2023-10-14 20:07:38,344][60425] Avg episode reward: [(0, '77.680'), (1, '74.630')] [2023-10-14 20:07:38,559][61552] Updated weights for policy 0, policy_version 60092 (0.0010) [2023-10-14 20:07:39,558][61585] Updated weights for policy 1, policy_version 59910 (0.0007) [2023-10-14 20:07:39,924][61585] Updated weights for policy 1, policy_version 59920 (0.0008) [2023-10-14 20:07:40,288][61585] Updated weights for policy 1, policy_version 59930 (0.0007) [2023-10-14 20:07:42,814][61552] Updated weights for policy 0, policy_version 60102 (0.0008) [2023-10-14 20:07:43,192][61552] Updated weights for policy 0, policy_version 60112 (0.0009) [2023-10-14 20:07:43,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 122912768. Throughput: 0: 1678.8, 1: 1675.9. Samples: 30743980. Policy #0 lag: (min: 31.0, avg: 41.3, max: 63.0) [2023-10-14 20:07:43,344][60425] Avg episode reward: [(0, '77.140'), (1, '73.800')] [2023-10-14 20:07:43,353][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000059936_61374464.pth... [2023-10-14 20:07:43,385][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000058368_59768832.pth [2023-10-14 20:07:43,568][61552] Updated weights for policy 0, policy_version 60122 (0.0010) [2023-10-14 20:07:43,773][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000060128_61571072.pth... [2023-10-14 20:07:43,808][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000058560_59965440.pth [2023-10-14 20:07:44,325][61585] Updated weights for policy 1, policy_version 59940 (0.0009) [2023-10-14 20:07:44,688][61585] Updated weights for policy 1, policy_version 59950 (0.0009) [2023-10-14 20:07:45,052][61585] Updated weights for policy 1, policy_version 59960 (0.0010) [2023-10-14 20:07:47,635][61552] Updated weights for policy 0, policy_version 60132 (0.0008) [2023-10-14 20:07:48,009][61552] Updated weights for policy 0, policy_version 60142 (0.0010) [2023-10-14 20:07:48,346][60425] Fps is (10 sec: 13103.9, 60 sec: 13106.6, 300 sec: 13329.2). Total num frames: 122978304. Throughput: 0: 1681.3, 1: 1661.5. Samples: 30753096. Policy #0 lag: (min: 31.0, avg: 41.3, max: 63.0) [2023-10-14 20:07:48,346][60425] Avg episode reward: [(0, '77.540'), (1, '71.170')] [2023-10-14 20:07:48,378][61552] Updated weights for policy 0, policy_version 60152 (0.0008) [2023-10-14 20:07:49,092][61585] Updated weights for policy 1, policy_version 59970 (0.0010) [2023-10-14 20:07:49,454][61585] Updated weights for policy 1, policy_version 59980 (0.0007) [2023-10-14 20:07:49,807][61585] Updated weights for policy 1, policy_version 59990 (0.0010) [2023-10-14 20:07:50,168][61585] Updated weights for policy 1, policy_version 60000 (0.0008) [2023-10-14 20:07:52,499][61552] Updated weights for policy 0, policy_version 60162 (0.0009) [2023-10-14 20:07:52,864][61552] Updated weights for policy 0, policy_version 60172 (0.0009) [2023-10-14 20:07:53,238][61552] Updated weights for policy 0, policy_version 60182 (0.0010) [2023-10-14 20:07:53,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 123043840. Throughput: 0: 1677.1, 1: 1682.3. Samples: 30773870. Policy #0 lag: (min: 8.0, avg: 31.1, max: 40.0) [2023-10-14 20:07:53,345][60425] Avg episode reward: [(0, '74.240'), (1, '72.200')] [2023-10-14 20:07:53,605][61552] Updated weights for policy 0, policy_version 60192 (0.0011) [2023-10-14 20:07:54,255][61585] Updated weights for policy 1, policy_version 60010 (0.0008) [2023-10-14 20:07:54,612][61585] Updated weights for policy 1, policy_version 60020 (0.0008) [2023-10-14 20:07:54,987][61585] Updated weights for policy 1, policy_version 60030 (0.0009) [2023-10-14 20:07:57,643][61552] Updated weights for policy 0, policy_version 60202 (0.0009) [2023-10-14 20:07:58,017][61552] Updated weights for policy 0, policy_version 60212 (0.0007) [2023-10-14 20:07:58,343][60425] Fps is (10 sec: 13110.2, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 123109376. Throughput: 0: 1665.5, 1: 1680.3. Samples: 30794052. Policy #0 lag: (min: 8.0, avg: 31.1, max: 40.0) [2023-10-14 20:07:58,344][60425] Avg episode reward: [(0, '74.220'), (1, '72.750')] [2023-10-14 20:07:58,387][61552] Updated weights for policy 0, policy_version 60222 (0.0009) [2023-10-14 20:07:59,027][61585] Updated weights for policy 1, policy_version 60040 (0.0007) [2023-10-14 20:07:59,392][61585] Updated weights for policy 1, policy_version 60050 (0.0008) [2023-10-14 20:07:59,760][61585] Updated weights for policy 1, policy_version 60060 (0.0009) [2023-10-14 20:08:02,522][61552] Updated weights for policy 0, policy_version 60232 (0.0008) [2023-10-14 20:08:02,892][61552] Updated weights for policy 0, policy_version 60242 (0.0007) [2023-10-14 20:08:03,252][61552] Updated weights for policy 0, policy_version 60252 (0.0007) [2023-10-14 20:08:03,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 123174912. Throughput: 0: 1677.1, 1: 1673.4. Samples: 30803620. Policy #0 lag: (min: 8.0, avg: 31.1, max: 40.0) [2023-10-14 20:08:03,344][60425] Avg episode reward: [(0, '77.000'), (1, '77.790')] [2023-10-14 20:08:03,842][61585] Updated weights for policy 1, policy_version 60070 (0.0010) [2023-10-14 20:08:04,209][61585] Updated weights for policy 1, policy_version 60080 (0.0008) [2023-10-14 20:08:04,566][61585] Updated weights for policy 1, policy_version 60090 (0.0008) [2023-10-14 20:08:07,475][61552] Updated weights for policy 0, policy_version 60262 (0.0008) [2023-10-14 20:08:07,851][61552] Updated weights for policy 0, policy_version 60272 (0.0008) [2023-10-14 20:08:08,221][61552] Updated weights for policy 0, policy_version 60282 (0.0008) [2023-10-14 20:08:08,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 123240448. Throughput: 0: 1680.2, 1: 1682.7. Samples: 30824358. Policy #0 lag: (min: 8.0, avg: 31.1, max: 40.0) [2023-10-14 20:08:08,344][60425] Avg episode reward: [(0, '75.420'), (1, '71.930')] [2023-10-14 20:08:08,714][61585] Updated weights for policy 1, policy_version 60100 (0.0007) [2023-10-14 20:08:09,087][61585] Updated weights for policy 1, policy_version 60110 (0.0009) [2023-10-14 20:08:09,448][61585] Updated weights for policy 1, policy_version 60120 (0.0009) [2023-10-14 20:08:12,253][61552] Updated weights for policy 0, policy_version 60292 (0.0009) [2023-10-14 20:08:12,617][61552] Updated weights for policy 0, policy_version 60302 (0.0008) [2023-10-14 20:08:12,987][61552] Updated weights for policy 0, policy_version 60312 (0.0007) [2023-10-14 20:08:13,343][60425] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 123338752. Throughput: 0: 1664.2, 1: 1686.6. Samples: 30844290. Policy #0 lag: (min: 8.0, avg: 31.1, max: 40.0) [2023-10-14 20:08:13,345][60425] Avg episode reward: [(0, '73.780'), (1, '74.720')] [2023-10-14 20:08:13,475][61585] Updated weights for policy 1, policy_version 60130 (0.0008) [2023-10-14 20:08:13,838][61585] Updated weights for policy 1, policy_version 60140 (0.0010) [2023-10-14 20:08:14,209][61585] Updated weights for policy 1, policy_version 60150 (0.0008) [2023-10-14 20:08:14,581][61585] Updated weights for policy 1, policy_version 60160 (0.0009) [2023-10-14 20:08:17,142][61552] Updated weights for policy 0, policy_version 60322 (0.0009) [2023-10-14 20:08:17,512][61552] Updated weights for policy 0, policy_version 60332 (0.0008) [2023-10-14 20:08:17,879][61552] Updated weights for policy 0, policy_version 60342 (0.0009) [2023-10-14 20:08:18,237][61552] Updated weights for policy 0, policy_version 60352 (0.0010) [2023-10-14 20:08:18,343][60425] Fps is (10 sec: 16384.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 123404288. Throughput: 0: 1676.1, 1: 1685.2. Samples: 30853930. Policy #0 lag: (min: 8.0, avg: 31.1, max: 40.0) [2023-10-14 20:08:18,344][60425] Avg episode reward: [(0, '74.790'), (1, '73.210')] [2023-10-14 20:08:18,769][61585] Updated weights for policy 1, policy_version 60170 (0.0011) [2023-10-14 20:08:19,128][61585] Updated weights for policy 1, policy_version 60180 (0.0010) [2023-10-14 20:08:19,488][61585] Updated weights for policy 1, policy_version 60190 (0.0010) [2023-10-14 20:08:22,246][61552] Updated weights for policy 0, policy_version 60362 (0.0008) [2023-10-14 20:08:22,604][61552] Updated weights for policy 0, policy_version 60372 (0.0008) [2023-10-14 20:08:22,973][61552] Updated weights for policy 0, policy_version 60382 (0.0009) [2023-10-14 20:08:23,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 123469824. Throughput: 0: 1676.4, 1: 1682.2. Samples: 30874406. Policy #0 lag: (min: 8.0, avg: 31.1, max: 40.0) [2023-10-14 20:08:23,344][60425] Avg episode reward: [(0, '77.640'), (1, '79.030')] [2023-10-14 20:08:23,545][61585] Updated weights for policy 1, policy_version 60200 (0.0011) [2023-10-14 20:08:23,916][61585] Updated weights for policy 1, policy_version 60210 (0.0012) [2023-10-14 20:08:24,286][61585] Updated weights for policy 1, policy_version 60220 (0.0011) [2023-10-14 20:08:27,003][61552] Updated weights for policy 0, policy_version 60392 (0.0009) [2023-10-14 20:08:27,379][61552] Updated weights for policy 0, policy_version 60402 (0.0010) [2023-10-14 20:08:27,738][61552] Updated weights for policy 0, policy_version 60412 (0.0009) [2023-10-14 20:08:28,270][61585] Updated weights for policy 1, policy_version 60230 (0.0010) [2023-10-14 20:08:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 123535360. Throughput: 0: 1659.5, 1: 1675.9. Samples: 30894072. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-14 20:08:28,344][60425] Avg episode reward: [(0, '77.470'), (1, '73.690')] [2023-10-14 20:08:28,641][61585] Updated weights for policy 1, policy_version 60240 (0.0008) [2023-10-14 20:08:29,002][61585] Updated weights for policy 1, policy_version 60250 (0.0007) [2023-10-14 20:08:31,766][61552] Updated weights for policy 0, policy_version 60422 (0.0009) [2023-10-14 20:08:32,145][61552] Updated weights for policy 0, policy_version 60432 (0.0008) [2023-10-14 20:08:32,508][61552] Updated weights for policy 0, policy_version 60442 (0.0007) [2023-10-14 20:08:33,155][61585] Updated weights for policy 1, policy_version 60260 (0.0007) [2023-10-14 20:08:33,344][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 123600896. Throughput: 0: 1683.5, 1: 1674.6. Samples: 30904204. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-14 20:08:33,345][60425] Avg episode reward: [(0, '79.180'), (1, '75.110')] [2023-10-14 20:08:33,519][61585] Updated weights for policy 1, policy_version 60270 (0.0007) [2023-10-14 20:08:33,882][61585] Updated weights for policy 1, policy_version 60280 (0.0007) [2023-10-14 20:08:36,541][61552] Updated weights for policy 0, policy_version 60452 (0.0008) [2023-10-14 20:08:36,922][61552] Updated weights for policy 0, policy_version 60462 (0.0012) [2023-10-14 20:08:37,286][61552] Updated weights for policy 0, policy_version 60472 (0.0009) [2023-10-14 20:08:38,136][61585] Updated weights for policy 1, policy_version 60290 (0.0008) [2023-10-14 20:08:38,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.5). Total num frames: 123666432. Throughput: 0: 1673.1, 1: 1672.1. Samples: 30924404. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-14 20:08:38,344][60425] Avg episode reward: [(0, '81.810'), (1, '75.490')] [2023-10-14 20:08:38,505][61585] Updated weights for policy 1, policy_version 60300 (0.0007) [2023-10-14 20:08:38,868][61585] Updated weights for policy 1, policy_version 60310 (0.0009) [2023-10-14 20:08:39,240][61585] Updated weights for policy 1, policy_version 60320 (0.0009) [2023-10-14 20:08:41,415][61552] Updated weights for policy 0, policy_version 60482 (0.0011) [2023-10-14 20:08:41,779][61552] Updated weights for policy 0, policy_version 60492 (0.0009) [2023-10-14 20:08:42,156][61552] Updated weights for policy 0, policy_version 60502 (0.0009) [2023-10-14 20:08:42,519][61552] Updated weights for policy 0, policy_version 60512 (0.0010) [2023-10-14 20:08:43,290][61585] Updated weights for policy 1, policy_version 60330 (0.0007) [2023-10-14 20:08:43,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 123731968. Throughput: 0: 1659.2, 1: 1670.0. Samples: 30943864. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-14 20:08:43,344][60425] Avg episode reward: [(0, '78.290'), (1, '77.690')] [2023-10-14 20:08:43,661][61585] Updated weights for policy 1, policy_version 60340 (0.0008) [2023-10-14 20:08:44,035][61585] Updated weights for policy 1, policy_version 60350 (0.0009) [2023-10-14 20:08:46,732][61552] Updated weights for policy 0, policy_version 60522 (0.0008) [2023-10-14 20:08:47,100][61552] Updated weights for policy 0, policy_version 60532 (0.0008) [2023-10-14 20:08:47,462][61552] Updated weights for policy 0, policy_version 60542 (0.0007) [2023-10-14 20:08:48,201][61585] Updated weights for policy 1, policy_version 60360 (0.0007) [2023-10-14 20:08:48,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.9, 300 sec: 13440.4). Total num frames: 123797504. Throughput: 0: 1677.1, 1: 1668.9. Samples: 30954188. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-14 20:08:48,344][60425] Avg episode reward: [(0, '75.980'), (1, '74.700')] [2023-10-14 20:08:48,567][61585] Updated weights for policy 1, policy_version 60370 (0.0010) [2023-10-14 20:08:48,933][61585] Updated weights for policy 1, policy_version 60380 (0.0008) [2023-10-14 20:08:51,764][61552] Updated weights for policy 0, policy_version 60552 (0.0008) [2023-10-14 20:08:52,130][61552] Updated weights for policy 0, policy_version 60562 (0.0008) [2023-10-14 20:08:52,493][61552] Updated weights for policy 0, policy_version 60572 (0.0009) [2023-10-14 20:08:52,918][61585] Updated weights for policy 1, policy_version 60390 (0.0007) [2023-10-14 20:08:53,289][61585] Updated weights for policy 1, policy_version 60400 (0.0009) [2023-10-14 20:08:53,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 123863040. Throughput: 0: 1666.4, 1: 1669.1. Samples: 30974458. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-14 20:08:53,345][60425] Avg episode reward: [(0, '75.290'), (1, '72.530')] [2023-10-14 20:08:53,652][61585] Updated weights for policy 1, policy_version 60410 (0.0008) [2023-10-14 20:08:56,617][61552] Updated weights for policy 0, policy_version 60582 (0.0008) [2023-10-14 20:08:56,984][61552] Updated weights for policy 0, policy_version 60592 (0.0008) [2023-10-14 20:08:57,355][61552] Updated weights for policy 0, policy_version 60602 (0.0010) [2023-10-14 20:08:57,823][61585] Updated weights for policy 1, policy_version 60420 (0.0008) [2023-10-14 20:08:58,193][61585] Updated weights for policy 1, policy_version 60430 (0.0008) [2023-10-14 20:08:58,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 123928576. Throughput: 0: 1656.5, 1: 1667.0. Samples: 30993844. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-14 20:08:58,344][60425] Avg episode reward: [(0, '76.210'), (1, '75.510')] [2023-10-14 20:08:58,573][61585] Updated weights for policy 1, policy_version 60440 (0.0008) [2023-10-14 20:09:01,425][61552] Updated weights for policy 0, policy_version 60612 (0.0010) [2023-10-14 20:09:01,797][61552] Updated weights for policy 0, policy_version 60622 (0.0009) [2023-10-14 20:09:02,167][61552] Updated weights for policy 0, policy_version 60632 (0.0008) [2023-10-14 20:09:02,632][61585] Updated weights for policy 1, policy_version 60450 (0.0008) [2023-10-14 20:09:03,046][61585] Updated weights for policy 1, policy_version 60460 (0.0008) [2023-10-14 20:09:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 123994112. Throughput: 0: 1671.7, 1: 1669.8. Samples: 31004298. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-14 20:09:03,344][60425] Avg episode reward: [(0, '77.270'), (1, '73.710')] [2023-10-14 20:09:03,424][61585] Updated weights for policy 1, policy_version 60470 (0.0009) [2023-10-14 20:09:03,780][61585] Updated weights for policy 1, policy_version 60480 (0.0008) [2023-10-14 20:09:06,192][61552] Updated weights for policy 0, policy_version 60642 (0.0007) [2023-10-14 20:09:06,560][61552] Updated weights for policy 0, policy_version 60652 (0.0009) [2023-10-14 20:09:06,940][61552] Updated weights for policy 0, policy_version 60662 (0.0009) [2023-10-14 20:09:07,304][61552] Updated weights for policy 0, policy_version 60672 (0.0008) [2023-10-14 20:09:07,726][61585] Updated weights for policy 1, policy_version 60490 (0.0009) [2023-10-14 20:09:08,092][61585] Updated weights for policy 1, policy_version 60500 (0.0007) [2023-10-14 20:09:08,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 124059648. Throughput: 0: 1653.6, 1: 1675.8. Samples: 31024232. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 20:09:08,344][60425] Avg episode reward: [(0, '79.230'), (1, '76.970')] [2023-10-14 20:09:08,461][61585] Updated weights for policy 1, policy_version 60510 (0.0008) [2023-10-14 20:09:11,605][61552] Updated weights for policy 0, policy_version 60682 (0.0008) [2023-10-14 20:09:11,970][61552] Updated weights for policy 0, policy_version 60692 (0.0010) [2023-10-14 20:09:12,334][61552] Updated weights for policy 0, policy_version 60702 (0.0007) [2023-10-14 20:09:12,729][61585] Updated weights for policy 1, policy_version 60520 (0.0007) [2023-10-14 20:09:13,104][61585] Updated weights for policy 1, policy_version 60530 (0.0009) [2023-10-14 20:09:13,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 124125184. Throughput: 0: 1657.3, 1: 1670.1. Samples: 31043806. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 20:09:13,344][60425] Avg episode reward: [(0, '74.210'), (1, '71.760')] [2023-10-14 20:09:13,475][61585] Updated weights for policy 1, policy_version 60540 (0.0009) [2023-10-14 20:09:16,714][61552] Updated weights for policy 0, policy_version 60712 (0.0008) [2023-10-14 20:09:17,081][61552] Updated weights for policy 0, policy_version 60722 (0.0007) [2023-10-14 20:09:17,450][61552] Updated weights for policy 0, policy_version 60732 (0.0009) [2023-10-14 20:09:17,540][61585] Updated weights for policy 1, policy_version 60550 (0.0008) [2023-10-14 20:09:17,914][61585] Updated weights for policy 1, policy_version 60560 (0.0009) [2023-10-14 20:09:18,294][61585] Updated weights for policy 1, policy_version 60570 (0.0008) [2023-10-14 20:09:18,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 124190720. Throughput: 0: 1655.1, 1: 1679.6. Samples: 31054262. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 20:09:18,344][60425] Avg episode reward: [(0, '74.910'), (1, '75.350')] [2023-10-14 20:09:21,529][61552] Updated weights for policy 0, policy_version 60742 (0.0010) [2023-10-14 20:09:21,898][61552] Updated weights for policy 0, policy_version 60752 (0.0009) [2023-10-14 20:09:22,268][61552] Updated weights for policy 0, policy_version 60762 (0.0008) [2023-10-14 20:09:22,302][61585] Updated weights for policy 1, policy_version 60580 (0.0007) [2023-10-14 20:09:22,679][61585] Updated weights for policy 1, policy_version 60590 (0.0009) [2023-10-14 20:09:23,043][61585] Updated weights for policy 1, policy_version 60600 (0.0011) [2023-10-14 20:09:23,343][60425] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 124289024. Throughput: 0: 1651.8, 1: 1682.4. Samples: 31074444. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 20:09:23,344][60425] Avg episode reward: [(0, '76.280'), (1, '71.900')] [2023-10-14 20:09:26,229][61552] Updated weights for policy 0, policy_version 60772 (0.0009) [2023-10-14 20:09:26,598][61552] Updated weights for policy 0, policy_version 60782 (0.0011) [2023-10-14 20:09:26,965][61552] Updated weights for policy 0, policy_version 60792 (0.0008) [2023-10-14 20:09:27,036][61585] Updated weights for policy 1, policy_version 60610 (0.0008) [2023-10-14 20:09:27,393][61585] Updated weights for policy 1, policy_version 60620 (0.0007) [2023-10-14 20:09:27,758][61585] Updated weights for policy 1, policy_version 60630 (0.0008) [2023-10-14 20:09:28,113][61585] Updated weights for policy 1, policy_version 60640 (0.0009) [2023-10-14 20:09:28,343][60425] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 124354560. Throughput: 0: 1660.0, 1: 1669.7. Samples: 31093702. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 20:09:28,344][60425] Avg episode reward: [(0, '78.890'), (1, '74.560')] [2023-10-14 20:09:31,032][61552] Updated weights for policy 0, policy_version 60802 (0.0009) [2023-10-14 20:09:31,401][61552] Updated weights for policy 0, policy_version 60812 (0.0011) [2023-10-14 20:09:31,769][61552] Updated weights for policy 0, policy_version 60822 (0.0008) [2023-10-14 20:09:32,139][61552] Updated weights for policy 0, policy_version 60832 (0.0008) [2023-10-14 20:09:32,167][61585] Updated weights for policy 1, policy_version 60650 (0.0008) [2023-10-14 20:09:32,530][61585] Updated weights for policy 1, policy_version 60660 (0.0009) [2023-10-14 20:09:32,889][61585] Updated weights for policy 1, policy_version 60670 (0.0009) [2023-10-14 20:09:33,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 124420096. Throughput: 0: 1663.2, 1: 1689.1. Samples: 31105044. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 20:09:33,344][60425] Avg episode reward: [(0, '77.580'), (1, '76.240')] [2023-10-14 20:09:36,143][61552] Updated weights for policy 0, policy_version 60842 (0.0008) [2023-10-14 20:09:36,517][61552] Updated weights for policy 0, policy_version 60852 (0.0008) [2023-10-14 20:09:36,857][61585] Updated weights for policy 1, policy_version 60680 (0.0010) [2023-10-14 20:09:36,883][61552] Updated weights for policy 0, policy_version 60862 (0.0008) [2023-10-14 20:09:37,223][61585] Updated weights for policy 1, policy_version 60690 (0.0010) [2023-10-14 20:09:37,586][61585] Updated weights for policy 1, policy_version 60700 (0.0011) [2023-10-14 20:09:38,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 124485632. Throughput: 0: 1649.6, 1: 1686.0. Samples: 31124560. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 20:09:38,344][60425] Avg episode reward: [(0, '75.880'), (1, '76.430')] [2023-10-14 20:09:41,119][61552] Updated weights for policy 0, policy_version 60872 (0.0009) [2023-10-14 20:09:41,488][61552] Updated weights for policy 0, policy_version 60882 (0.0010) [2023-10-14 20:09:41,769][61585] Updated weights for policy 1, policy_version 60710 (0.0008) [2023-10-14 20:09:41,850][61552] Updated weights for policy 0, policy_version 60892 (0.0007) [2023-10-14 20:09:42,134][61585] Updated weights for policy 1, policy_version 60720 (0.0008) [2023-10-14 20:09:42,504][61585] Updated weights for policy 1, policy_version 60730 (0.0007) [2023-10-14 20:09:43,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 124551168. Throughput: 0: 1662.8, 1: 1666.7. Samples: 31143670. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 20:09:43,344][60425] Avg episode reward: [(0, '75.780'), (1, '76.380')] [2023-10-14 20:09:43,351][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000060896_62357504.pth... [2023-10-14 20:09:43,351][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000060736_62193664.pth... [2023-10-14 20:09:43,387][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000059328_60751872.pth [2023-10-14 20:09:43,398][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000059168_60588032.pth [2023-10-14 20:09:45,985][61552] Updated weights for policy 0, policy_version 60902 (0.0007) [2023-10-14 20:09:46,361][61552] Updated weights for policy 0, policy_version 60912 (0.0008) [2023-10-14 20:09:46,575][61585] Updated weights for policy 1, policy_version 60740 (0.0008) [2023-10-14 20:09:46,724][61552] Updated weights for policy 0, policy_version 60922 (0.0008) [2023-10-14 20:09:46,936][61585] Updated weights for policy 1, policy_version 60750 (0.0007) [2023-10-14 20:09:47,304][61585] Updated weights for policy 1, policy_version 60760 (0.0007) [2023-10-14 20:09:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 124616704. Throughput: 0: 1659.5, 1: 1694.3. Samples: 31155218. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-14 20:09:48,344][60425] Avg episode reward: [(0, '76.020'), (1, '79.950')] [2023-10-14 20:09:50,874][61552] Updated weights for policy 0, policy_version 60932 (0.0008) [2023-10-14 20:09:51,243][61552] Updated weights for policy 0, policy_version 60942 (0.0009) [2023-10-14 20:09:51,517][61585] Updated weights for policy 1, policy_version 60770 (0.0008) [2023-10-14 20:09:51,616][61552] Updated weights for policy 0, policy_version 60952 (0.0008) [2023-10-14 20:09:51,938][61585] Updated weights for policy 1, policy_version 60780 (0.0009) [2023-10-14 20:09:52,303][61585] Updated weights for policy 1, policy_version 60790 (0.0008) [2023-10-14 20:09:52,665][61585] Updated weights for policy 1, policy_version 60800 (0.0009) [2023-10-14 20:09:53,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 124682240. Throughput: 0: 1651.1, 1: 1686.2. Samples: 31174410. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-14 20:09:53,344][60425] Avg episode reward: [(0, '78.490'), (1, '80.580')] [2023-10-14 20:09:55,527][61552] Updated weights for policy 0, policy_version 60962 (0.0009) [2023-10-14 20:09:55,903][61552] Updated weights for policy 0, policy_version 60972 (0.0008) [2023-10-14 20:09:56,262][61552] Updated weights for policy 0, policy_version 60982 (0.0010) [2023-10-14 20:09:56,625][61552] Updated weights for policy 0, policy_version 60992 (0.0009) [2023-10-14 20:09:56,779][61585] Updated weights for policy 1, policy_version 60810 (0.0009) [2023-10-14 20:09:57,153][61585] Updated weights for policy 1, policy_version 60820 (0.0009) [2023-10-14 20:09:57,516][61585] Updated weights for policy 1, policy_version 60830 (0.0008) [2023-10-14 20:09:58,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 124747776. Throughput: 0: 1671.0, 1: 1667.4. Samples: 31194034. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-14 20:09:58,344][60425] Avg episode reward: [(0, '76.300'), (1, '77.770')] [2023-10-14 20:10:00,648][61552] Updated weights for policy 0, policy_version 61002 (0.0007) [2023-10-14 20:10:01,010][61552] Updated weights for policy 0, policy_version 61012 (0.0007) [2023-10-14 20:10:01,372][61552] Updated weights for policy 0, policy_version 61022 (0.0008) [2023-10-14 20:10:01,558][61585] Updated weights for policy 1, policy_version 60840 (0.0008) [2023-10-14 20:10:01,927][61585] Updated weights for policy 1, policy_version 60850 (0.0007) [2023-10-14 20:10:02,282][61585] Updated weights for policy 1, policy_version 60860 (0.0007) [2023-10-14 20:10:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 124813312. Throughput: 0: 1669.1, 1: 1685.4. Samples: 31205218. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-14 20:10:03,344][60425] Avg episode reward: [(0, '75.160'), (1, '76.330')] [2023-10-14 20:10:05,407][61552] Updated weights for policy 0, policy_version 61032 (0.0008) [2023-10-14 20:10:05,782][61552] Updated weights for policy 0, policy_version 61042 (0.0007) [2023-10-14 20:10:06,151][61552] Updated weights for policy 0, policy_version 61052 (0.0009) [2023-10-14 20:10:06,415][61585] Updated weights for policy 1, policy_version 60870 (0.0007) [2023-10-14 20:10:06,775][61585] Updated weights for policy 1, policy_version 60880 (0.0008) [2023-10-14 20:10:07,145][61585] Updated weights for policy 1, policy_version 60890 (0.0008) [2023-10-14 20:10:08,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 124878848. Throughput: 0: 1667.0, 1: 1672.6. Samples: 31224726. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-14 20:10:08,344][60425] Avg episode reward: [(0, '76.620'), (1, '78.000')] [2023-10-14 20:10:10,379][61552] Updated weights for policy 0, policy_version 61062 (0.0007) [2023-10-14 20:10:10,743][61552] Updated weights for policy 0, policy_version 61072 (0.0008) [2023-10-14 20:10:11,108][61552] Updated weights for policy 0, policy_version 61082 (0.0010) [2023-10-14 20:10:11,205][61585] Updated weights for policy 1, policy_version 60900 (0.0008) [2023-10-14 20:10:11,579][61585] Updated weights for policy 1, policy_version 60910 (0.0009) [2023-10-14 20:10:11,953][61585] Updated weights for policy 1, policy_version 60920 (0.0008) [2023-10-14 20:10:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 124944384. Throughput: 0: 1680.9, 1: 1674.7. Samples: 31244702. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-14 20:10:13,345][60425] Avg episode reward: [(0, '77.410'), (1, '72.820')] [2023-10-14 20:10:15,146][61552] Updated weights for policy 0, policy_version 61092 (0.0010) [2023-10-14 20:10:15,518][61552] Updated weights for policy 0, policy_version 61102 (0.0009) [2023-10-14 20:10:15,887][61552] Updated weights for policy 0, policy_version 61112 (0.0007) [2023-10-14 20:10:15,988][61585] Updated weights for policy 1, policy_version 60930 (0.0008) [2023-10-14 20:10:16,348][61585] Updated weights for policy 1, policy_version 60940 (0.0009) [2023-10-14 20:10:16,723][61585] Updated weights for policy 1, policy_version 60950 (0.0008) [2023-10-14 20:10:17,076][61585] Updated weights for policy 1, policy_version 60960 (0.0009) [2023-10-14 20:10:18,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 125009920. Throughput: 0: 1660.8, 1: 1683.3. Samples: 31255528. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-14 20:10:18,344][60425] Avg episode reward: [(0, '76.990'), (1, '75.240')] [2023-10-14 20:10:19,932][61552] Updated weights for policy 0, policy_version 61122 (0.0008) [2023-10-14 20:10:20,309][61552] Updated weights for policy 0, policy_version 61132 (0.0007) [2023-10-14 20:10:20,677][61552] Updated weights for policy 0, policy_version 61142 (0.0007) [2023-10-14 20:10:21,042][61552] Updated weights for policy 0, policy_version 61152 (0.0007) [2023-10-14 20:10:21,089][61585] Updated weights for policy 1, policy_version 60970 (0.0008) [2023-10-14 20:10:21,460][61585] Updated weights for policy 1, policy_version 60980 (0.0008) [2023-10-14 20:10:21,828][61585] Updated weights for policy 1, policy_version 60990 (0.0010) [2023-10-14 20:10:23,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 125075456. Throughput: 0: 1671.0, 1: 1663.4. Samples: 31274610. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-14 20:10:23,344][60425] Avg episode reward: [(0, '79.310'), (1, '76.180')] [2023-10-14 20:10:25,013][61552] Updated weights for policy 0, policy_version 61162 (0.0009) [2023-10-14 20:10:25,372][61552] Updated weights for policy 0, policy_version 61172 (0.0011) [2023-10-14 20:10:25,721][61585] Updated weights for policy 1, policy_version 61000 (0.0009) [2023-10-14 20:10:25,746][61552] Updated weights for policy 0, policy_version 61182 (0.0009) [2023-10-14 20:10:26,080][61585] Updated weights for policy 1, policy_version 61010 (0.0011) [2023-10-14 20:10:26,448][61585] Updated weights for policy 1, policy_version 61020 (0.0010) [2023-10-14 20:10:28,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 125140992. Throughput: 0: 1686.3, 1: 1681.2. Samples: 31295208. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-14 20:10:28,345][60425] Avg episode reward: [(0, '77.650'), (1, '75.200')] [2023-10-14 20:10:29,932][61552] Updated weights for policy 0, policy_version 61192 (0.0009) [2023-10-14 20:10:30,308][61552] Updated weights for policy 0, policy_version 61202 (0.0010) [2023-10-14 20:10:30,509][61585] Updated weights for policy 1, policy_version 61030 (0.0008) [2023-10-14 20:10:30,682][61552] Updated weights for policy 0, policy_version 61212 (0.0009) [2023-10-14 20:10:30,872][61585] Updated weights for policy 1, policy_version 61040 (0.0007) [2023-10-14 20:10:31,232][61585] Updated weights for policy 1, policy_version 61050 (0.0008) [2023-10-14 20:10:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 125206528. Throughput: 0: 1661.8, 1: 1670.3. Samples: 31305164. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-14 20:10:33,344][60425] Avg episode reward: [(0, '71.490'), (1, '73.750')] [2023-10-14 20:10:34,714][61552] Updated weights for policy 0, policy_version 61222 (0.0008) [2023-10-14 20:10:35,079][61552] Updated weights for policy 0, policy_version 61232 (0.0008) [2023-10-14 20:10:35,368][61585] Updated weights for policy 1, policy_version 61060 (0.0009) [2023-10-14 20:10:35,454][61552] Updated weights for policy 0, policy_version 61242 (0.0009) [2023-10-14 20:10:35,735][61585] Updated weights for policy 1, policy_version 61070 (0.0008) [2023-10-14 20:10:36,093][61585] Updated weights for policy 1, policy_version 61080 (0.0009) [2023-10-14 20:10:38,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 125272064. Throughput: 0: 1681.4, 1: 1658.3. Samples: 31324698. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-14 20:10:38,344][60425] Avg episode reward: [(0, '78.740'), (1, '73.940')] [2023-10-14 20:10:39,611][61552] Updated weights for policy 0, policy_version 61252 (0.0010) [2023-10-14 20:10:39,981][61552] Updated weights for policy 0, policy_version 61262 (0.0011) [2023-10-14 20:10:40,342][61585] Updated weights for policy 1, policy_version 61090 (0.0007) [2023-10-14 20:10:40,353][61552] Updated weights for policy 0, policy_version 61272 (0.0009) [2023-10-14 20:10:40,725][61585] Updated weights for policy 1, policy_version 61100 (0.0007) [2023-10-14 20:10:41,090][61585] Updated weights for policy 1, policy_version 61110 (0.0010) [2023-10-14 20:10:41,455][61585] Updated weights for policy 1, policy_version 61120 (0.0010) [2023-10-14 20:10:43,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 125337600. Throughput: 0: 1677.2, 1: 1679.9. Samples: 31345102. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-14 20:10:43,345][60425] Avg episode reward: [(0, '73.730'), (1, '71.500')] [2023-10-14 20:10:44,419][61552] Updated weights for policy 0, policy_version 61282 (0.0007) [2023-10-14 20:10:44,789][61552] Updated weights for policy 0, policy_version 61292 (0.0009) [2023-10-14 20:10:45,159][61552] Updated weights for policy 0, policy_version 61302 (0.0009) [2023-10-14 20:10:45,521][61552] Updated weights for policy 0, policy_version 61312 (0.0007) [2023-10-14 20:10:45,530][61585] Updated weights for policy 1, policy_version 61130 (0.0007) [2023-10-14 20:10:45,898][61585] Updated weights for policy 1, policy_version 61140 (0.0007) [2023-10-14 20:10:46,266][61585] Updated weights for policy 1, policy_version 61150 (0.0008) [2023-10-14 20:10:48,345][60425] Fps is (10 sec: 13105.4, 60 sec: 13106.9, 300 sec: 13329.3). Total num frames: 125403136. Throughput: 0: 1656.2, 1: 1667.6. Samples: 31354794. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-14 20:10:48,345][60425] Avg episode reward: [(0, '67.590'), (1, '73.110')] [2023-10-14 20:10:49,680][61552] Updated weights for policy 0, policy_version 61322 (0.0009) [2023-10-14 20:10:50,060][61552] Updated weights for policy 0, policy_version 61332 (0.0009) [2023-10-14 20:10:50,430][61552] Updated weights for policy 0, policy_version 61342 (0.0008) [2023-10-14 20:10:50,513][61585] Updated weights for policy 1, policy_version 61160 (0.0009) [2023-10-14 20:10:50,868][61585] Updated weights for policy 1, policy_version 61170 (0.0008) [2023-10-14 20:10:51,239][61585] Updated weights for policy 1, policy_version 61180 (0.0009) [2023-10-14 20:10:53,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 125468672. Throughput: 0: 1667.8, 1: 1659.3. Samples: 31374446. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-14 20:10:53,344][60425] Avg episode reward: [(0, '72.300'), (1, '71.740')] [2023-10-14 20:10:54,536][61552] Updated weights for policy 0, policy_version 61352 (0.0009) [2023-10-14 20:10:54,908][61552] Updated weights for policy 0, policy_version 61362 (0.0010) [2023-10-14 20:10:55,278][61552] Updated weights for policy 0, policy_version 61372 (0.0008) [2023-10-14 20:10:55,317][61585] Updated weights for policy 1, policy_version 61190 (0.0009) [2023-10-14 20:10:55,669][61585] Updated weights for policy 1, policy_version 61200 (0.0008) [2023-10-14 20:10:56,036][61585] Updated weights for policy 1, policy_version 61210 (0.0007) [2023-10-14 20:10:58,343][60425] Fps is (10 sec: 13108.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 125534208. Throughput: 0: 1669.3, 1: 1677.1. Samples: 31395290. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-14 20:10:58,345][60425] Avg episode reward: [(0, '74.240'), (1, '74.820')] [2023-10-14 20:10:59,339][61552] Updated weights for policy 0, policy_version 61382 (0.0008) [2023-10-14 20:10:59,698][61552] Updated weights for policy 0, policy_version 61392 (0.0009) [2023-10-14 20:11:00,034][61585] Updated weights for policy 1, policy_version 61220 (0.0007) [2023-10-14 20:11:00,076][61552] Updated weights for policy 0, policy_version 61402 (0.0009) [2023-10-14 20:11:00,402][61585] Updated weights for policy 1, policy_version 61230 (0.0007) [2023-10-14 20:11:00,778][61585] Updated weights for policy 1, policy_version 61240 (0.0008) [2023-10-14 20:11:03,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 125599744. Throughput: 0: 1656.5, 1: 1658.2. Samples: 31404692. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-14 20:11:03,344][60425] Avg episode reward: [(0, '72.570'), (1, '70.250')] [2023-10-14 20:11:04,119][61552] Updated weights for policy 0, policy_version 61412 (0.0009) [2023-10-14 20:11:04,483][61552] Updated weights for policy 0, policy_version 61422 (0.0010) [2023-10-14 20:11:04,744][61585] Updated weights for policy 1, policy_version 61250 (0.0009) [2023-10-14 20:11:04,851][61552] Updated weights for policy 0, policy_version 61432 (0.0008) [2023-10-14 20:11:05,107][61585] Updated weights for policy 1, policy_version 61260 (0.0008) [2023-10-14 20:11:05,469][61585] Updated weights for policy 1, policy_version 61270 (0.0008) [2023-10-14 20:11:05,844][61585] Updated weights for policy 1, policy_version 61280 (0.0009) [2023-10-14 20:11:08,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 125665280. Throughput: 0: 1665.8, 1: 1671.3. Samples: 31424780. Policy #0 lag: (min: 12.0, avg: 20.9, max: 44.0) [2023-10-14 20:11:08,344][60425] Avg episode reward: [(0, '70.590'), (1, '73.310')] [2023-10-14 20:11:09,050][61552] Updated weights for policy 0, policy_version 61442 (0.0007) [2023-10-14 20:11:09,421][61552] Updated weights for policy 0, policy_version 61452 (0.0007) [2023-10-14 20:11:09,784][61552] Updated weights for policy 0, policy_version 61462 (0.0008) [2023-10-14 20:11:09,902][61585] Updated weights for policy 1, policy_version 61290 (0.0010) [2023-10-14 20:11:10,156][61552] Updated weights for policy 0, policy_version 61472 (0.0008) [2023-10-14 20:11:10,264][61585] Updated weights for policy 1, policy_version 61300 (0.0009) [2023-10-14 20:11:10,623][61585] Updated weights for policy 1, policy_version 61310 (0.0008) [2023-10-14 20:11:13,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 125730816. Throughput: 0: 1661.9, 1: 1677.1. Samples: 31445460. Policy #0 lag: (min: 12.0, avg: 20.9, max: 44.0) [2023-10-14 20:11:13,344][60425] Avg episode reward: [(0, '73.560'), (1, '71.770')] [2023-10-14 20:11:14,422][61552] Updated weights for policy 0, policy_version 61482 (0.0007) [2023-10-14 20:11:14,660][61585] Updated weights for policy 1, policy_version 61320 (0.0009) [2023-10-14 20:11:14,789][61552] Updated weights for policy 0, policy_version 61492 (0.0008) [2023-10-14 20:11:15,023][61585] Updated weights for policy 1, policy_version 61330 (0.0008) [2023-10-14 20:11:15,156][61552] Updated weights for policy 0, policy_version 61502 (0.0007) [2023-10-14 20:11:15,387][61585] Updated weights for policy 1, policy_version 61340 (0.0007) [2023-10-14 20:11:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 125796352. Throughput: 0: 1658.0, 1: 1658.5. Samples: 31454404. Policy #0 lag: (min: 12.0, avg: 20.9, max: 44.0) [2023-10-14 20:11:18,344][60425] Avg episode reward: [(0, '69.750'), (1, '72.170')] [2023-10-14 20:11:19,111][61552] Updated weights for policy 0, policy_version 61512 (0.0010) [2023-10-14 20:11:19,390][61585] Updated weights for policy 1, policy_version 61350 (0.0008) [2023-10-14 20:11:19,492][61552] Updated weights for policy 0, policy_version 61522 (0.0008) [2023-10-14 20:11:19,746][61585] Updated weights for policy 1, policy_version 61360 (0.0012) [2023-10-14 20:11:19,855][61552] Updated weights for policy 0, policy_version 61532 (0.0008) [2023-10-14 20:11:20,107][61585] Updated weights for policy 1, policy_version 61370 (0.0008) [2023-10-14 20:11:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 125861888. Throughput: 0: 1665.2, 1: 1679.6. Samples: 31475214. Policy #0 lag: (min: 12.0, avg: 20.9, max: 44.0) [2023-10-14 20:11:23,344][60425] Avg episode reward: [(0, '71.380'), (1, '70.010')] [2023-10-14 20:11:23,823][61552] Updated weights for policy 0, policy_version 61542 (0.0009) [2023-10-14 20:11:24,201][61552] Updated weights for policy 0, policy_version 61552 (0.0009) [2023-10-14 20:11:24,273][61585] Updated weights for policy 1, policy_version 61380 (0.0010) [2023-10-14 20:11:24,563][61552] Updated weights for policy 0, policy_version 61562 (0.0008) [2023-10-14 20:11:24,630][61585] Updated weights for policy 1, policy_version 61390 (0.0010) [2023-10-14 20:11:24,991][61585] Updated weights for policy 1, policy_version 61400 (0.0010) [2023-10-14 20:11:28,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 125927424. Throughput: 0: 1665.9, 1: 1682.3. Samples: 31495772. Policy #0 lag: (min: 12.0, avg: 20.9, max: 44.0) [2023-10-14 20:11:28,344][60425] Avg episode reward: [(0, '71.060'), (1, '75.060')] [2023-10-14 20:11:28,727][61552] Updated weights for policy 0, policy_version 61572 (0.0010) [2023-10-14 20:11:29,094][61552] Updated weights for policy 0, policy_version 61582 (0.0008) [2023-10-14 20:11:29,209][61585] Updated weights for policy 1, policy_version 61410 (0.0008) [2023-10-14 20:11:29,463][61552] Updated weights for policy 0, policy_version 61592 (0.0007) [2023-10-14 20:11:29,613][61585] Updated weights for policy 1, policy_version 61420 (0.0008) [2023-10-14 20:11:29,986][61585] Updated weights for policy 1, policy_version 61430 (0.0009) [2023-10-14 20:11:30,341][61585] Updated weights for policy 1, policy_version 61440 (0.0009) [2023-10-14 20:11:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 125992960. Throughput: 0: 1665.6, 1: 1663.8. Samples: 31504614. Policy #0 lag: (min: 12.0, avg: 20.9, max: 44.0) [2023-10-14 20:11:33,344][60425] Avg episode reward: [(0, '71.340'), (1, '71.320')] [2023-10-14 20:11:33,582][61552] Updated weights for policy 0, policy_version 61602 (0.0009) [2023-10-14 20:11:33,947][61552] Updated weights for policy 0, policy_version 61612 (0.0008) [2023-10-14 20:11:34,320][61552] Updated weights for policy 0, policy_version 61622 (0.0009) [2023-10-14 20:11:34,373][61585] Updated weights for policy 1, policy_version 61450 (0.0009) [2023-10-14 20:11:34,682][61552] Updated weights for policy 0, policy_version 61632 (0.0007) [2023-10-14 20:11:34,744][61585] Updated weights for policy 1, policy_version 61460 (0.0008) [2023-10-14 20:11:35,104][61585] Updated weights for policy 1, policy_version 61470 (0.0010) [2023-10-14 20:11:38,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 126058496. Throughput: 0: 1670.7, 1: 1680.0. Samples: 31525226. Policy #0 lag: (min: 12.0, avg: 20.9, max: 44.0) [2023-10-14 20:11:38,344][60425] Avg episode reward: [(0, '73.300'), (1, '77.460')] [2023-10-14 20:11:38,708][61552] Updated weights for policy 0, policy_version 61642 (0.0007) [2023-10-14 20:11:39,084][61552] Updated weights for policy 0, policy_version 61652 (0.0009) [2023-10-14 20:11:39,258][61585] Updated weights for policy 1, policy_version 61480 (0.0007) [2023-10-14 20:11:39,441][61552] Updated weights for policy 0, policy_version 61662 (0.0008) [2023-10-14 20:11:39,622][61585] Updated weights for policy 1, policy_version 61490 (0.0009) [2023-10-14 20:11:39,979][61585] Updated weights for policy 1, policy_version 61500 (0.0010) [2023-10-14 20:11:43,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 126124032. Throughput: 0: 1673.4, 1: 1674.4. Samples: 31545940. Policy #0 lag: (min: 12.0, avg: 20.9, max: 44.0) [2023-10-14 20:11:43,344][60425] Avg episode reward: [(0, '73.270'), (1, '73.710')] [2023-10-14 20:11:43,354][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000061504_62980096.pth... [2023-10-14 20:11:43,388][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000059936_61374464.pth [2023-10-14 20:11:43,597][61552] Updated weights for policy 0, policy_version 61672 (0.0008) [2023-10-14 20:11:43,958][61552] Updated weights for policy 0, policy_version 61682 (0.0009) [2023-10-14 20:11:44,025][61585] Updated weights for policy 1, policy_version 61510 (0.0009) [2023-10-14 20:11:44,329][61552] Updated weights for policy 0, policy_version 61692 (0.0010) [2023-10-14 20:11:44,394][61585] Updated weights for policy 1, policy_version 61520 (0.0008) [2023-10-14 20:11:44,477][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000061696_63176704.pth... [2023-10-14 20:11:44,514][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000060128_61571072.pth [2023-10-14 20:11:44,753][61585] Updated weights for policy 1, policy_version 61530 (0.0010) [2023-10-14 20:11:48,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.5, 300 sec: 13329.4). Total num frames: 126189568. Throughput: 0: 1674.0, 1: 1668.7. Samples: 31555112. Policy #0 lag: (min: 22.0, avg: 25.3, max: 54.0) [2023-10-14 20:11:48,344][60425] Avg episode reward: [(0, '72.490'), (1, '74.950')] [2023-10-14 20:11:48,524][61552] Updated weights for policy 0, policy_version 61702 (0.0010) [2023-10-14 20:11:48,872][61585] Updated weights for policy 1, policy_version 61540 (0.0010) [2023-10-14 20:11:48,896][61552] Updated weights for policy 0, policy_version 61712 (0.0008) [2023-10-14 20:11:49,240][61585] Updated weights for policy 1, policy_version 61550 (0.0009) [2023-10-14 20:11:49,271][61552] Updated weights for policy 0, policy_version 61722 (0.0007) [2023-10-14 20:11:49,599][61585] Updated weights for policy 1, policy_version 61560 (0.0008) [2023-10-14 20:11:53,286][61552] Updated weights for policy 0, policy_version 61732 (0.0008) [2023-10-14 20:11:53,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 126255104. Throughput: 0: 1675.4, 1: 1675.9. Samples: 31575590. Policy #0 lag: (min: 22.0, avg: 25.3, max: 54.0) [2023-10-14 20:11:53,344][60425] Avg episode reward: [(0, '72.060'), (1, '76.010')] [2023-10-14 20:11:53,639][61585] Updated weights for policy 1, policy_version 61570 (0.0011) [2023-10-14 20:11:53,652][61552] Updated weights for policy 0, policy_version 61742 (0.0008) [2023-10-14 20:11:54,005][61585] Updated weights for policy 1, policy_version 61580 (0.0009) [2023-10-14 20:11:54,017][61552] Updated weights for policy 0, policy_version 61752 (0.0007) [2023-10-14 20:11:54,373][61585] Updated weights for policy 1, policy_version 61590 (0.0009) [2023-10-14 20:11:54,724][61585] Updated weights for policy 1, policy_version 61600 (0.0009) [2023-10-14 20:11:58,049][61552] Updated weights for policy 0, policy_version 61762 (0.0008) [2023-10-14 20:11:58,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 126320640. Throughput: 0: 1682.1, 1: 1674.9. Samples: 31596526. Policy #0 lag: (min: 22.0, avg: 25.3, max: 54.0) [2023-10-14 20:11:58,344][60425] Avg episode reward: [(0, '76.670'), (1, '77.780')] [2023-10-14 20:11:58,428][61552] Updated weights for policy 0, policy_version 61772 (0.0010) [2023-10-14 20:11:58,718][61585] Updated weights for policy 1, policy_version 61610 (0.0007) [2023-10-14 20:11:58,804][61552] Updated weights for policy 0, policy_version 61782 (0.0008) [2023-10-14 20:11:59,086][61585] Updated weights for policy 1, policy_version 61620 (0.0009) [2023-10-14 20:11:59,174][61552] Updated weights for policy 0, policy_version 61792 (0.0009) [2023-10-14 20:11:59,448][61585] Updated weights for policy 1, policy_version 61630 (0.0011) [2023-10-14 20:12:03,075][61552] Updated weights for policy 0, policy_version 61802 (0.0008) [2023-10-14 20:12:03,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 126386176. Throughput: 0: 1686.1, 1: 1672.7. Samples: 31605548. Policy #0 lag: (min: 22.0, avg: 25.3, max: 54.0) [2023-10-14 20:12:03,344][60425] Avg episode reward: [(0, '70.650'), (1, '69.290')] [2023-10-14 20:12:03,437][61552] Updated weights for policy 0, policy_version 61812 (0.0008) [2023-10-14 20:12:03,606][61585] Updated weights for policy 1, policy_version 61640 (0.0010) [2023-10-14 20:12:03,811][61552] Updated weights for policy 0, policy_version 61822 (0.0008) [2023-10-14 20:12:03,974][61585] Updated weights for policy 1, policy_version 61650 (0.0008) [2023-10-14 20:12:04,339][61585] Updated weights for policy 1, policy_version 61660 (0.0008) [2023-10-14 20:12:08,017][61552] Updated weights for policy 0, policy_version 61832 (0.0009) [2023-10-14 20:12:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 126451712. Throughput: 0: 1682.7, 1: 1666.1. Samples: 31625912. Policy #0 lag: (min: 22.0, avg: 25.3, max: 54.0) [2023-10-14 20:12:08,344][60425] Avg episode reward: [(0, '73.800'), (1, '76.390')] [2023-10-14 20:12:08,396][61552] Updated weights for policy 0, policy_version 61842 (0.0010) [2023-10-14 20:12:08,521][61585] Updated weights for policy 1, policy_version 61670 (0.0009) [2023-10-14 20:12:08,760][61552] Updated weights for policy 0, policy_version 61852 (0.0007) [2023-10-14 20:12:08,887][61585] Updated weights for policy 1, policy_version 61680 (0.0009) [2023-10-14 20:12:09,253][61585] Updated weights for policy 1, policy_version 61690 (0.0009) [2023-10-14 20:12:12,970][61552] Updated weights for policy 0, policy_version 61862 (0.0008) [2023-10-14 20:12:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 126517248. Throughput: 0: 1683.4, 1: 1665.0. Samples: 31646450. Policy #0 lag: (min: 22.0, avg: 25.3, max: 54.0) [2023-10-14 20:12:13,344][60425] Avg episode reward: [(0, '74.870'), (1, '76.510')] [2023-10-14 20:12:13,356][61552] Updated weights for policy 0, policy_version 61872 (0.0009) [2023-10-14 20:12:13,384][61585] Updated weights for policy 1, policy_version 61700 (0.0010) [2023-10-14 20:12:13,725][61552] Updated weights for policy 0, policy_version 61882 (0.0008) [2023-10-14 20:12:13,751][61585] Updated weights for policy 1, policy_version 61710 (0.0008) [2023-10-14 20:12:14,114][61585] Updated weights for policy 1, policy_version 61720 (0.0009) [2023-10-14 20:12:17,885][61552] Updated weights for policy 0, policy_version 61892 (0.0009) [2023-10-14 20:12:18,254][61552] Updated weights for policy 0, policy_version 61902 (0.0007) [2023-10-14 20:12:18,311][61585] Updated weights for policy 1, policy_version 61730 (0.0008) [2023-10-14 20:12:18,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 126582784. Throughput: 0: 1682.8, 1: 1669.7. Samples: 31655478. Policy #0 lag: (min: 22.0, avg: 25.3, max: 54.0) [2023-10-14 20:12:18,344][60425] Avg episode reward: [(0, '72.580'), (1, '76.950')] [2023-10-14 20:12:18,617][61552] Updated weights for policy 0, policy_version 61912 (0.0007) [2023-10-14 20:12:18,708][61585] Updated weights for policy 1, policy_version 61740 (0.0009) [2023-10-14 20:12:19,062][61585] Updated weights for policy 1, policy_version 61750 (0.0008) [2023-10-14 20:12:19,425][61585] Updated weights for policy 1, policy_version 61760 (0.0010) [2023-10-14 20:12:22,760][61552] Updated weights for policy 0, policy_version 61922 (0.0009) [2023-10-14 20:12:23,134][61552] Updated weights for policy 0, policy_version 61932 (0.0007) [2023-10-14 20:12:23,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 126648320. Throughput: 0: 1676.8, 1: 1668.5. Samples: 31675762. Policy #0 lag: (min: 22.0, avg: 25.3, max: 54.0) [2023-10-14 20:12:23,344][60425] Avg episode reward: [(0, '71.750'), (1, '72.510')] [2023-10-14 20:12:23,499][61552] Updated weights for policy 0, policy_version 61942 (0.0007) [2023-10-14 20:12:23,677][61585] Updated weights for policy 1, policy_version 61770 (0.0007) [2023-10-14 20:12:23,864][61552] Updated weights for policy 0, policy_version 61952 (0.0007) [2023-10-14 20:12:24,042][61585] Updated weights for policy 1, policy_version 61780 (0.0010) [2023-10-14 20:12:24,412][61585] Updated weights for policy 1, policy_version 61790 (0.0009) [2023-10-14 20:12:28,049][61552] Updated weights for policy 0, policy_version 61962 (0.0007) [2023-10-14 20:12:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 126713856. Throughput: 0: 1672.4, 1: 1668.9. Samples: 31696296. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:12:28,344][60425] Avg episode reward: [(0, '72.920'), (1, '74.950')] [2023-10-14 20:12:28,408][61552] Updated weights for policy 0, policy_version 61972 (0.0009) [2023-10-14 20:12:28,443][61585] Updated weights for policy 1, policy_version 61800 (0.0009) [2023-10-14 20:12:28,783][61552] Updated weights for policy 0, policy_version 61982 (0.0008) [2023-10-14 20:12:28,807][61585] Updated weights for policy 1, policy_version 61810 (0.0009) [2023-10-14 20:12:29,168][61585] Updated weights for policy 1, policy_version 61820 (0.0010) [2023-10-14 20:12:32,953][61552] Updated weights for policy 0, policy_version 61992 (0.0008) [2023-10-14 20:12:33,194][61585] Updated weights for policy 1, policy_version 61830 (0.0008) [2023-10-14 20:12:33,319][61552] Updated weights for policy 0, policy_version 62002 (0.0010) [2023-10-14 20:12:33,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 126779392. Throughput: 0: 1672.2, 1: 1667.6. Samples: 31705402. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:12:33,344][60425] Avg episode reward: [(0, '72.860'), (1, '79.040')] [2023-10-14 20:12:33,560][61585] Updated weights for policy 1, policy_version 61840 (0.0007) [2023-10-14 20:12:33,686][61552] Updated weights for policy 0, policy_version 62012 (0.0008) [2023-10-14 20:12:33,916][61585] Updated weights for policy 1, policy_version 61850 (0.0010) [2023-10-14 20:12:37,875][61552] Updated weights for policy 0, policy_version 62022 (0.0010) [2023-10-14 20:12:38,200][61585] Updated weights for policy 1, policy_version 61860 (0.0009) [2023-10-14 20:12:38,243][61552] Updated weights for policy 0, policy_version 62032 (0.0008) [2023-10-14 20:12:38,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 126844928. Throughput: 0: 1668.1, 1: 1675.3. Samples: 31726042. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:12:38,344][60425] Avg episode reward: [(0, '73.740'), (1, '73.110')] [2023-10-14 20:12:38,575][61585] Updated weights for policy 1, policy_version 61870 (0.0009) [2023-10-14 20:12:38,613][61552] Updated weights for policy 0, policy_version 62042 (0.0008) [2023-10-14 20:12:38,929][61585] Updated weights for policy 1, policy_version 61880 (0.0008) [2023-10-14 20:12:42,903][61552] Updated weights for policy 0, policy_version 62052 (0.0008) [2023-10-14 20:12:43,098][61585] Updated weights for policy 1, policy_version 61890 (0.0008) [2023-10-14 20:12:43,276][61552] Updated weights for policy 0, policy_version 62062 (0.0007) [2023-10-14 20:12:43,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.5). Total num frames: 126910464. Throughput: 0: 1661.9, 1: 1671.3. Samples: 31746518. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:12:43,344][60425] Avg episode reward: [(0, '74.190'), (1, '74.320')] [2023-10-14 20:12:43,463][61585] Updated weights for policy 1, policy_version 61900 (0.0007) [2023-10-14 20:12:43,641][61552] Updated weights for policy 0, policy_version 62072 (0.0008) [2023-10-14 20:12:43,819][61585] Updated weights for policy 1, policy_version 61910 (0.0009) [2023-10-14 20:12:44,183][61585] Updated weights for policy 1, policy_version 61920 (0.0009) [2023-10-14 20:12:47,664][61552] Updated weights for policy 0, policy_version 62082 (0.0010) [2023-10-14 20:12:48,031][61552] Updated weights for policy 0, policy_version 62092 (0.0007) [2023-10-14 20:12:48,123][61585] Updated weights for policy 1, policy_version 61930 (0.0009) [2023-10-14 20:12:48,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 126976000. Throughput: 0: 1663.3, 1: 1673.0. Samples: 31755682. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:12:48,344][60425] Avg episode reward: [(0, '72.170'), (1, '73.570')] [2023-10-14 20:12:48,394][61552] Updated weights for policy 0, policy_version 62102 (0.0010) [2023-10-14 20:12:48,488][61585] Updated weights for policy 1, policy_version 61940 (0.0008) [2023-10-14 20:12:48,759][61552] Updated weights for policy 0, policy_version 62112 (0.0008) [2023-10-14 20:12:48,847][61585] Updated weights for policy 1, policy_version 61950 (0.0009) [2023-10-14 20:12:52,758][61552] Updated weights for policy 0, policy_version 62122 (0.0009) [2023-10-14 20:12:52,831][61585] Updated weights for policy 1, policy_version 61960 (0.0007) [2023-10-14 20:12:53,131][61552] Updated weights for policy 0, policy_version 62132 (0.0008) [2023-10-14 20:12:53,190][61585] Updated weights for policy 1, policy_version 61970 (0.0009) [2023-10-14 20:12:53,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 127041536. Throughput: 0: 1661.5, 1: 1684.4. Samples: 31776474. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:12:53,344][60425] Avg episode reward: [(0, '75.880'), (1, '72.700')] [2023-10-14 20:12:53,489][61552] Updated weights for policy 0, policy_version 62142 (0.0008) [2023-10-14 20:12:53,551][61585] Updated weights for policy 1, policy_version 61980 (0.0008) [2023-10-14 20:12:57,654][61585] Updated weights for policy 1, policy_version 61990 (0.0009) [2023-10-14 20:12:57,669][61552] Updated weights for policy 0, policy_version 62152 (0.0008) [2023-10-14 20:12:58,013][61585] Updated weights for policy 1, policy_version 62000 (0.0010) [2023-10-14 20:12:58,044][61552] Updated weights for policy 0, policy_version 62162 (0.0009) [2023-10-14 20:12:58,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 127107072. Throughput: 0: 1656.1, 1: 1675.6. Samples: 31796374. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:12:58,344][60425] Avg episode reward: [(0, '73.690'), (1, '76.780')] [2023-10-14 20:12:58,375][61585] Updated weights for policy 1, policy_version 62010 (0.0009) [2023-10-14 20:12:58,413][61552] Updated weights for policy 0, policy_version 62172 (0.0009) [2023-10-14 20:13:02,362][61552] Updated weights for policy 0, policy_version 62182 (0.0008) [2023-10-14 20:13:02,387][61585] Updated weights for policy 1, policy_version 62020 (0.0008) [2023-10-14 20:13:02,728][61552] Updated weights for policy 0, policy_version 62192 (0.0008) [2023-10-14 20:13:02,755][61585] Updated weights for policy 1, policy_version 62030 (0.0010) [2023-10-14 20:13:03,095][61552] Updated weights for policy 0, policy_version 62202 (0.0011) [2023-10-14 20:13:03,121][61585] Updated weights for policy 1, policy_version 62040 (0.0008) [2023-10-14 20:13:03,343][60425] Fps is (10 sec: 16384.0, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 127205376. Throughput: 0: 1664.9, 1: 1684.5. Samples: 31806202. Policy #0 lag: (min: 21.0, avg: 36.9, max: 53.0) [2023-10-14 20:13:03,344][60425] Avg episode reward: [(0, '75.380'), (1, '76.620')] [2023-10-14 20:13:07,159][61552] Updated weights for policy 0, policy_version 62212 (0.0008) [2023-10-14 20:13:07,314][61585] Updated weights for policy 1, policy_version 62050 (0.0008) [2023-10-14 20:13:07,536][61552] Updated weights for policy 0, policy_version 62222 (0.0007) [2023-10-14 20:13:07,707][61585] Updated weights for policy 1, policy_version 62060 (0.0010) [2023-10-14 20:13:07,892][61552] Updated weights for policy 0, policy_version 62232 (0.0009) [2023-10-14 20:13:08,074][61585] Updated weights for policy 1, policy_version 62070 (0.0009) [2023-10-14 20:13:08,343][60425] Fps is (10 sec: 16384.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 127270912. Throughput: 0: 1673.8, 1: 1685.4. Samples: 31826926. Policy #0 lag: (min: 21.0, avg: 36.9, max: 53.0) [2023-10-14 20:13:08,344][60425] Avg episode reward: [(0, '73.120'), (1, '77.030')] [2023-10-14 20:13:08,445][61585] Updated weights for policy 1, policy_version 62080 (0.0009) [2023-10-14 20:13:11,898][61552] Updated weights for policy 0, policy_version 62242 (0.0009) [2023-10-14 20:13:12,259][61552] Updated weights for policy 0, policy_version 62252 (0.0009) [2023-10-14 20:13:12,372][61585] Updated weights for policy 1, policy_version 62090 (0.0009) [2023-10-14 20:13:12,627][61552] Updated weights for policy 0, policy_version 62262 (0.0007) [2023-10-14 20:13:12,733][61585] Updated weights for policy 1, policy_version 62100 (0.0008) [2023-10-14 20:13:12,996][61552] Updated weights for policy 0, policy_version 62272 (0.0007) [2023-10-14 20:13:13,090][61585] Updated weights for policy 1, policy_version 62110 (0.0007) [2023-10-14 20:13:13,343][60425] Fps is (10 sec: 16383.8, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 127369216. Throughput: 0: 1655.2, 1: 1668.6. Samples: 31845868. Policy #0 lag: (min: 21.0, avg: 36.9, max: 53.0) [2023-10-14 20:13:13,344][60425] Avg episode reward: [(0, '75.260'), (1, '73.250')] [2023-10-14 20:13:16,987][61552] Updated weights for policy 0, policy_version 62282 (0.0008) [2023-10-14 20:13:17,273][61585] Updated weights for policy 1, policy_version 62120 (0.0007) [2023-10-14 20:13:17,362][61552] Updated weights for policy 0, policy_version 62292 (0.0008) [2023-10-14 20:13:17,632][61585] Updated weights for policy 1, policy_version 62130 (0.0007) [2023-10-14 20:13:17,721][61552] Updated weights for policy 0, policy_version 62302 (0.0008) [2023-10-14 20:13:17,995][61585] Updated weights for policy 1, policy_version 62140 (0.0008) [2023-10-14 20:13:18,343][60425] Fps is (10 sec: 16383.7, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 127434752. Throughput: 0: 1674.1, 1: 1682.9. Samples: 31856468. Policy #0 lag: (min: 21.0, avg: 36.9, max: 53.0) [2023-10-14 20:13:18,344][60425] Avg episode reward: [(0, '72.700'), (1, '73.940')] [2023-10-14 20:13:21,812][61552] Updated weights for policy 0, policy_version 62312 (0.0008) [2023-10-14 20:13:22,091][61585] Updated weights for policy 1, policy_version 62150 (0.0009) [2023-10-14 20:13:22,176][61552] Updated weights for policy 0, policy_version 62322 (0.0007) [2023-10-14 20:13:22,467][61585] Updated weights for policy 1, policy_version 62160 (0.0008) [2023-10-14 20:13:22,537][61552] Updated weights for policy 0, policy_version 62332 (0.0009) [2023-10-14 20:13:22,834][61585] Updated weights for policy 1, policy_version 62170 (0.0007) [2023-10-14 20:13:23,343][60425] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 127500288. Throughput: 0: 1674.8, 1: 1679.8. Samples: 31877000. Policy #0 lag: (min: 21.0, avg: 36.9, max: 53.0) [2023-10-14 20:13:23,344][60425] Avg episode reward: [(0, '76.330'), (1, '77.270')] [2023-10-14 20:13:26,606][61552] Updated weights for policy 0, policy_version 62342 (0.0009) [2023-10-14 20:13:26,920][61585] Updated weights for policy 1, policy_version 62180 (0.0007) [2023-10-14 20:13:26,972][61552] Updated weights for policy 0, policy_version 62352 (0.0009) [2023-10-14 20:13:27,287][61585] Updated weights for policy 1, policy_version 62190 (0.0008) [2023-10-14 20:13:27,338][61552] Updated weights for policy 0, policy_version 62362 (0.0010) [2023-10-14 20:13:27,659][61585] Updated weights for policy 1, policy_version 62200 (0.0008) [2023-10-14 20:13:28,343][60425] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 127565824. Throughput: 0: 1650.4, 1: 1660.8. Samples: 31895524. Policy #0 lag: (min: 21.0, avg: 36.9, max: 53.0) [2023-10-14 20:13:28,344][60425] Avg episode reward: [(0, '72.830'), (1, '72.210')] [2023-10-14 20:13:31,242][61552] Updated weights for policy 0, policy_version 62372 (0.0009) [2023-10-14 20:13:31,617][61552] Updated weights for policy 0, policy_version 62382 (0.0009) [2023-10-14 20:13:31,827][61585] Updated weights for policy 1, policy_version 62210 (0.0009) [2023-10-14 20:13:31,976][61552] Updated weights for policy 0, policy_version 62392 (0.0008) [2023-10-14 20:13:32,199][61585] Updated weights for policy 1, policy_version 62220 (0.0009) [2023-10-14 20:13:32,560][61585] Updated weights for policy 1, policy_version 62230 (0.0008) [2023-10-14 20:13:32,920][61585] Updated weights for policy 1, policy_version 62240 (0.0007) [2023-10-14 20:13:33,343][60425] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 127631360. Throughput: 0: 1683.2, 1: 1680.6. Samples: 31907054. Policy #0 lag: (min: 21.0, avg: 36.9, max: 53.0) [2023-10-14 20:13:33,344][60425] Avg episode reward: [(0, '71.360'), (1, '69.600')] [2023-10-14 20:13:36,060][61552] Updated weights for policy 0, policy_version 62402 (0.0008) [2023-10-14 20:13:36,430][61552] Updated weights for policy 0, policy_version 62412 (0.0009) [2023-10-14 20:13:36,786][61552] Updated weights for policy 0, policy_version 62422 (0.0009) [2023-10-14 20:13:36,930][61585] Updated weights for policy 1, policy_version 62250 (0.0008) [2023-10-14 20:13:37,156][61552] Updated weights for policy 0, policy_version 62432 (0.0008) [2023-10-14 20:13:37,299][61585] Updated weights for policy 1, policy_version 62260 (0.0008) [2023-10-14 20:13:37,658][61585] Updated weights for policy 1, policy_version 62270 (0.0010) [2023-10-14 20:13:38,343][60425] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 127696896. Throughput: 0: 1667.5, 1: 1671.0. Samples: 31926704. Policy #0 lag: (min: 21.0, avg: 36.9, max: 53.0) [2023-10-14 20:13:38,344][60425] Avg episode reward: [(0, '74.840'), (1, '74.300')] [2023-10-14 20:13:41,268][61552] Updated weights for policy 0, policy_version 62442 (0.0009) [2023-10-14 20:13:41,539][61585] Updated weights for policy 1, policy_version 62280 (0.0009) [2023-10-14 20:13:41,642][61552] Updated weights for policy 0, policy_version 62452 (0.0008) [2023-10-14 20:13:41,909][61585] Updated weights for policy 1, policy_version 62290 (0.0009) [2023-10-14 20:13:42,007][61552] Updated weights for policy 0, policy_version 62462 (0.0007) [2023-10-14 20:13:42,274][61585] Updated weights for policy 1, policy_version 62300 (0.0007) [2023-10-14 20:13:43,344][60425] Fps is (10 sec: 13106.6, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 127762432. Throughput: 0: 1667.6, 1: 1662.8. Samples: 31946246. Policy #0 lag: (min: 21.0, avg: 36.9, max: 53.0) [2023-10-14 20:13:43,345][60425] Avg episode reward: [(0, '75.030'), (1, '75.570')] [2023-10-14 20:13:43,357][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000062304_63799296.pth... [2023-10-14 20:13:43,357][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000062464_63963136.pth... [2023-10-14 20:13:43,389][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000060896_62357504.pth [2023-10-14 20:13:43,393][61172] Saving a milestone ./train_atari/atari_roadrunner_APPO/checkpoint_p0/milestones/checkpoint_000062464_63963136.pth [2023-10-14 20:13:43,397][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000060736_62193664.pth [2023-10-14 20:13:43,403][61248] Saving a milestone ./train_atari/atari_roadrunner_APPO/checkpoint_p1/milestones/checkpoint_000062304_63799296.pth [2023-10-14 20:13:46,157][61552] Updated weights for policy 0, policy_version 62472 (0.0008) [2023-10-14 20:13:46,407][61585] Updated weights for policy 1, policy_version 62310 (0.0008) [2023-10-14 20:13:46,534][61552] Updated weights for policy 0, policy_version 62482 (0.0007) [2023-10-14 20:13:46,779][61585] Updated weights for policy 1, policy_version 62320 (0.0009) [2023-10-14 20:13:46,907][61552] Updated weights for policy 0, policy_version 62492 (0.0007) [2023-10-14 20:13:47,136][61585] Updated weights for policy 1, policy_version 62330 (0.0010) [2023-10-14 20:13:48,343][60425] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 127827968. Throughput: 0: 1686.3, 1: 1682.0. Samples: 31957772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:13:48,344][60425] Avg episode reward: [(0, '74.770'), (1, '71.440')] [2023-10-14 20:13:51,016][61552] Updated weights for policy 0, policy_version 62502 (0.0008) [2023-10-14 20:13:51,377][61585] Updated weights for policy 1, policy_version 62340 (0.0009) [2023-10-14 20:13:51,380][61552] Updated weights for policy 0, policy_version 62512 (0.0009) [2023-10-14 20:13:51,750][61552] Updated weights for policy 0, policy_version 62522 (0.0008) [2023-10-14 20:13:51,779][61585] Updated weights for policy 1, policy_version 62350 (0.0009) [2023-10-14 20:13:52,141][61585] Updated weights for policy 1, policy_version 62360 (0.0009) [2023-10-14 20:13:53,343][60425] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 127893504. Throughput: 0: 1656.3, 1: 1670.6. Samples: 31976634. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:13:53,344][60425] Avg episode reward: [(0, '74.900'), (1, '72.960')] [2023-10-14 20:13:55,824][61552] Updated weights for policy 0, policy_version 62532 (0.0007) [2023-10-14 20:13:56,190][61585] Updated weights for policy 1, policy_version 62370 (0.0010) [2023-10-14 20:13:56,200][61552] Updated weights for policy 0, policy_version 62542 (0.0007) [2023-10-14 20:13:56,554][61585] Updated weights for policy 1, policy_version 62380 (0.0009) [2023-10-14 20:13:56,560][61552] Updated weights for policy 0, policy_version 62552 (0.0007) [2023-10-14 20:13:56,916][61585] Updated weights for policy 1, policy_version 62390 (0.0008) [2023-10-14 20:13:57,282][61585] Updated weights for policy 1, policy_version 62400 (0.0008) [2023-10-14 20:13:58,343][60425] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 127959040. Throughput: 0: 1672.2, 1: 1668.8. Samples: 31996214. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:13:58,344][60425] Avg episode reward: [(0, '78.880'), (1, '76.430')] [2023-10-14 20:14:00,787][61552] Updated weights for policy 0, policy_version 62562 (0.0010) [2023-10-14 20:14:00,985][61585] Updated weights for policy 1, policy_version 62410 (0.0007) [2023-10-14 20:14:01,162][61552] Updated weights for policy 0, policy_version 62572 (0.0009) [2023-10-14 20:14:01,348][61585] Updated weights for policy 1, policy_version 62420 (0.0008) [2023-10-14 20:14:01,525][61552] Updated weights for policy 0, policy_version 62582 (0.0010) [2023-10-14 20:14:01,707][61585] Updated weights for policy 1, policy_version 62430 (0.0007) [2023-10-14 20:14:01,901][61552] Updated weights for policy 0, policy_version 62592 (0.0009) [2023-10-14 20:14:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 128024576. Throughput: 0: 1678.9, 1: 1684.7. Samples: 32007832. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:14:03,344][60425] Avg episode reward: [(0, '75.980'), (1, '75.580')] [2023-10-14 20:14:05,891][61585] Updated weights for policy 1, policy_version 62440 (0.0009) [2023-10-14 20:14:06,037][61552] Updated weights for policy 0, policy_version 62602 (0.0008) [2023-10-14 20:14:06,264][61585] Updated weights for policy 1, policy_version 62450 (0.0009) [2023-10-14 20:14:06,399][61552] Updated weights for policy 0, policy_version 62612 (0.0009) [2023-10-14 20:14:06,615][61585] Updated weights for policy 1, policy_version 62460 (0.0008) [2023-10-14 20:14:06,761][61552] Updated weights for policy 0, policy_version 62622 (0.0009) [2023-10-14 20:14:08,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 128090112. Throughput: 0: 1659.5, 1: 1660.0. Samples: 32026374. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:14:08,344][60425] Avg episode reward: [(0, '76.400'), (1, '75.780')] [2023-10-14 20:14:10,639][61585] Updated weights for policy 1, policy_version 62470 (0.0010) [2023-10-14 20:14:11,004][61585] Updated weights for policy 1, policy_version 62480 (0.0007) [2023-10-14 20:14:11,029][61552] Updated weights for policy 0, policy_version 62632 (0.0009) [2023-10-14 20:14:11,356][61585] Updated weights for policy 1, policy_version 62490 (0.0009) [2023-10-14 20:14:11,386][61552] Updated weights for policy 0, policy_version 62642 (0.0007) [2023-10-14 20:14:11,760][61552] Updated weights for policy 0, policy_version 62652 (0.0010) [2023-10-14 20:14:13,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 128155648. Throughput: 0: 1679.4, 1: 1681.2. Samples: 32046750. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:14:13,344][60425] Avg episode reward: [(0, '74.690'), (1, '75.860')] [2023-10-14 20:14:15,525][61585] Updated weights for policy 1, policy_version 62500 (0.0008) [2023-10-14 20:14:15,790][61552] Updated weights for policy 0, policy_version 62662 (0.0008) [2023-10-14 20:14:15,887][61585] Updated weights for policy 1, policy_version 62510 (0.0007) [2023-10-14 20:14:16,153][61552] Updated weights for policy 0, policy_version 62672 (0.0008) [2023-10-14 20:14:16,249][61585] Updated weights for policy 1, policy_version 62520 (0.0008) [2023-10-14 20:14:16,524][61552] Updated weights for policy 0, policy_version 62682 (0.0008) [2023-10-14 20:14:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 128221184. Throughput: 0: 1670.4, 1: 1680.4. Samples: 32057844. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:14:18,344][60425] Avg episode reward: [(0, '75.040'), (1, '74.710')] [2023-10-14 20:14:20,303][61585] Updated weights for policy 1, policy_version 62530 (0.0008) [2023-10-14 20:14:20,577][61552] Updated weights for policy 0, policy_version 62692 (0.0009) [2023-10-14 20:14:20,667][61585] Updated weights for policy 1, policy_version 62540 (0.0008) [2023-10-14 20:14:20,931][61552] Updated weights for policy 0, policy_version 62702 (0.0009) [2023-10-14 20:14:21,035][61585] Updated weights for policy 1, policy_version 62550 (0.0008) [2023-10-14 20:14:21,303][61552] Updated weights for policy 0, policy_version 62712 (0.0007) [2023-10-14 20:14:21,401][61585] Updated weights for policy 1, policy_version 62560 (0.0008) [2023-10-14 20:14:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 128286720. Throughput: 0: 1660.7, 1: 1667.3. Samples: 32076462. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:14:23,344][60425] Avg episode reward: [(0, '73.510'), (1, '69.660')] [2023-10-14 20:14:25,246][61552] Updated weights for policy 0, policy_version 62722 (0.0010) [2023-10-14 20:14:25,626][61552] Updated weights for policy 0, policy_version 62732 (0.0008) [2023-10-14 20:14:25,689][61585] Updated weights for policy 1, policy_version 62570 (0.0008) [2023-10-14 20:14:25,992][61552] Updated weights for policy 0, policy_version 62742 (0.0008) [2023-10-14 20:14:26,040][61585] Updated weights for policy 1, policy_version 62580 (0.0008) [2023-10-14 20:14:26,351][61552] Updated weights for policy 0, policy_version 62752 (0.0009) [2023-10-14 20:14:26,411][61585] Updated weights for policy 1, policy_version 62590 (0.0008) [2023-10-14 20:14:28,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 128352256. Throughput: 0: 1663.1, 1: 1682.0. Samples: 32096776. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:14:28,345][60425] Avg episode reward: [(0, '74.740'), (1, '75.850')] [2023-10-14 20:14:30,481][61585] Updated weights for policy 1, policy_version 62600 (0.0010) [2023-10-14 20:14:30,667][61552] Updated weights for policy 0, policy_version 62762 (0.0009) [2023-10-14 20:14:30,841][61585] Updated weights for policy 1, policy_version 62610 (0.0007) [2023-10-14 20:14:31,032][61552] Updated weights for policy 0, policy_version 62772 (0.0009) [2023-10-14 20:14:31,201][61585] Updated weights for policy 1, policy_version 62620 (0.0007) [2023-10-14 20:14:31,394][61552] Updated weights for policy 0, policy_version 62782 (0.0010) [2023-10-14 20:14:33,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 128417792. Throughput: 0: 1653.5, 1: 1668.5. Samples: 32107260. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) [2023-10-14 20:14:33,344][60425] Avg episode reward: [(0, '69.460'), (1, '75.250')] [2023-10-14 20:14:35,396][61585] Updated weights for policy 1, policy_version 62630 (0.0010) [2023-10-14 20:14:35,768][61585] Updated weights for policy 1, policy_version 62640 (0.0008) [2023-10-14 20:14:35,780][61552] Updated weights for policy 0, policy_version 62792 (0.0009) [2023-10-14 20:14:36,131][61585] Updated weights for policy 1, policy_version 62650 (0.0008) [2023-10-14 20:14:36,139][61552] Updated weights for policy 0, policy_version 62802 (0.0008) [2023-10-14 20:14:36,509][61552] Updated weights for policy 0, policy_version 62812 (0.0010) [2023-10-14 20:14:38,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 128483328. Throughput: 0: 1652.9, 1: 1666.5. Samples: 32126008. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) [2023-10-14 20:14:38,344][60425] Avg episode reward: [(0, '67.970'), (1, '77.650')] [2023-10-14 20:14:40,241][61585] Updated weights for policy 1, policy_version 62660 (0.0008) [2023-10-14 20:14:40,517][61552] Updated weights for policy 0, policy_version 62822 (0.0009) [2023-10-14 20:14:40,641][61585] Updated weights for policy 1, policy_version 62670 (0.0007) [2023-10-14 20:14:40,879][61552] Updated weights for policy 0, policy_version 62832 (0.0007) [2023-10-14 20:14:41,008][61585] Updated weights for policy 1, policy_version 62680 (0.0007) [2023-10-14 20:14:41,240][61552] Updated weights for policy 0, policy_version 62842 (0.0007) [2023-10-14 20:14:43,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 128548864. Throughput: 0: 1656.1, 1: 1681.7. Samples: 32146416. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) [2023-10-14 20:14:43,344][60425] Avg episode reward: [(0, '71.310'), (1, '75.100')] [2023-10-14 20:14:45,191][61552] Updated weights for policy 0, policy_version 62852 (0.0008) [2023-10-14 20:14:45,211][61585] Updated weights for policy 1, policy_version 62690 (0.0008) [2023-10-14 20:14:45,559][61552] Updated weights for policy 0, policy_version 62862 (0.0007) [2023-10-14 20:14:45,573][61585] Updated weights for policy 1, policy_version 62700 (0.0008) [2023-10-14 20:14:45,935][61552] Updated weights for policy 0, policy_version 62872 (0.0007) [2023-10-14 20:14:45,936][61585] Updated weights for policy 1, policy_version 62710 (0.0008) [2023-10-14 20:14:46,303][61585] Updated weights for policy 1, policy_version 62720 (0.0009) [2023-10-14 20:14:48,344][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 128614400. Throughput: 0: 1646.9, 1: 1659.7. Samples: 32156630. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) [2023-10-14 20:14:48,344][60425] Avg episode reward: [(0, '71.070'), (1, '79.490')] [2023-10-14 20:14:50,210][61552] Updated weights for policy 0, policy_version 62882 (0.0008) [2023-10-14 20:14:50,583][61552] Updated weights for policy 0, policy_version 62892 (0.0008) [2023-10-14 20:14:50,610][61585] Updated weights for policy 1, policy_version 62730 (0.0009) [2023-10-14 20:14:50,950][61552] Updated weights for policy 0, policy_version 62902 (0.0007) [2023-10-14 20:14:50,966][61585] Updated weights for policy 1, policy_version 62740 (0.0008) [2023-10-14 20:14:51,308][61552] Updated weights for policy 0, policy_version 62912 (0.0009) [2023-10-14 20:14:51,328][61585] Updated weights for policy 1, policy_version 62750 (0.0008) [2023-10-14 20:14:53,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 128679936. Throughput: 0: 1653.4, 1: 1664.0. Samples: 32175656. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) [2023-10-14 20:14:53,344][60425] Avg episode reward: [(0, '72.030'), (1, '76.730')] [2023-10-14 20:14:55,385][61585] Updated weights for policy 1, policy_version 62760 (0.0008) [2023-10-14 20:14:55,445][61552] Updated weights for policy 0, policy_version 62922 (0.0008) [2023-10-14 20:14:55,754][61585] Updated weights for policy 1, policy_version 62770 (0.0010) [2023-10-14 20:14:55,814][61552] Updated weights for policy 0, policy_version 62932 (0.0007) [2023-10-14 20:14:56,118][61585] Updated weights for policy 1, policy_version 62780 (0.0008) [2023-10-14 20:14:56,180][61552] Updated weights for policy 0, policy_version 62942 (0.0007) [2023-10-14 20:14:58,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 128745472. Throughput: 0: 1659.5, 1: 1662.2. Samples: 32196228. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) [2023-10-14 20:14:58,345][60425] Avg episode reward: [(0, '73.750'), (1, '76.090')] [2023-10-14 20:15:00,225][61585] Updated weights for policy 1, policy_version 62790 (0.0008) [2023-10-14 20:15:00,306][61552] Updated weights for policy 0, policy_version 62952 (0.0007) [2023-10-14 20:15:00,586][61585] Updated weights for policy 1, policy_version 62800 (0.0009) [2023-10-14 20:15:00,669][61552] Updated weights for policy 0, policy_version 62962 (0.0007) [2023-10-14 20:15:00,946][61585] Updated weights for policy 1, policy_version 62810 (0.0008) [2023-10-14 20:15:01,036][61552] Updated weights for policy 0, policy_version 62972 (0.0010) [2023-10-14 20:15:03,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 128811008. Throughput: 0: 1644.3, 1: 1655.6. Samples: 32206342. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) [2023-10-14 20:15:03,344][60425] Avg episode reward: [(0, '74.430'), (1, '78.430')] [2023-10-14 20:15:05,108][61552] Updated weights for policy 0, policy_version 62982 (0.0010) [2023-10-14 20:15:05,134][61585] Updated weights for policy 1, policy_version 62820 (0.0008) [2023-10-14 20:15:05,473][61552] Updated weights for policy 0, policy_version 62992 (0.0007) [2023-10-14 20:15:05,500][61585] Updated weights for policy 1, policy_version 62830 (0.0008) [2023-10-14 20:15:05,836][61552] Updated weights for policy 0, policy_version 63002 (0.0008) [2023-10-14 20:15:05,869][61585] Updated weights for policy 1, policy_version 62840 (0.0008) [2023-10-14 20:15:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 128876544. Throughput: 0: 1657.3, 1: 1658.5. Samples: 32225676. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) [2023-10-14 20:15:08,344][60425] Avg episode reward: [(0, '75.640'), (1, '74.250')] [2023-10-14 20:15:10,093][61552] Updated weights for policy 0, policy_version 63012 (0.0008) [2023-10-14 20:15:10,215][61585] Updated weights for policy 1, policy_version 62850 (0.0008) [2023-10-14 20:15:10,470][61552] Updated weights for policy 0, policy_version 63022 (0.0007) [2023-10-14 20:15:10,582][61585] Updated weights for policy 1, policy_version 62860 (0.0008) [2023-10-14 20:15:10,840][61552] Updated weights for policy 0, policy_version 63032 (0.0007) [2023-10-14 20:15:10,948][61585] Updated weights for policy 1, policy_version 62870 (0.0010) [2023-10-14 20:15:11,308][61585] Updated weights for policy 1, policy_version 62880 (0.0008) [2023-10-14 20:15:13,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 128942080. Throughput: 0: 1660.5, 1: 1658.8. Samples: 32246144. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) [2023-10-14 20:15:13,344][60425] Avg episode reward: [(0, '78.890'), (1, '75.450')] [2023-10-14 20:15:15,089][61552] Updated weights for policy 0, policy_version 63042 (0.0009) [2023-10-14 20:15:15,394][61585] Updated weights for policy 1, policy_version 62890 (0.0008) [2023-10-14 20:15:15,454][61552] Updated weights for policy 0, policy_version 63052 (0.0009) [2023-10-14 20:15:15,761][61585] Updated weights for policy 1, policy_version 62900 (0.0010) [2023-10-14 20:15:15,822][61552] Updated weights for policy 0, policy_version 63062 (0.0009) [2023-10-14 20:15:16,118][61585] Updated weights for policy 1, policy_version 62910 (0.0008) [2023-10-14 20:15:16,193][61552] Updated weights for policy 0, policy_version 63072 (0.0008) [2023-10-14 20:15:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 129007616. Throughput: 0: 1653.3, 1: 1654.0. Samples: 32256090. Policy #0 lag: (min: 28.0, avg: 30.2, max: 60.0) [2023-10-14 20:15:18,344][60425] Avg episode reward: [(0, '76.220'), (1, '79.300')] [2023-10-14 20:15:20,204][61585] Updated weights for policy 1, policy_version 62920 (0.0008) [2023-10-14 20:15:20,408][61552] Updated weights for policy 0, policy_version 63082 (0.0007) [2023-10-14 20:15:20,568][61585] Updated weights for policy 1, policy_version 62930 (0.0008) [2023-10-14 20:15:20,776][61552] Updated weights for policy 0, policy_version 63092 (0.0008) [2023-10-14 20:15:20,937][61585] Updated weights for policy 1, policy_version 62940 (0.0010) [2023-10-14 20:15:21,130][61552] Updated weights for policy 0, policy_version 63102 (0.0008) [2023-10-14 20:15:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 129073152. Throughput: 0: 1657.7, 1: 1656.2. Samples: 32275132. Policy #0 lag: (min: 28.0, avg: 30.2, max: 60.0) [2023-10-14 20:15:23,344][60425] Avg episode reward: [(0, '73.660'), (1, '77.300')] [2023-10-14 20:15:25,078][61585] Updated weights for policy 1, policy_version 62950 (0.0008) [2023-10-14 20:15:25,406][61552] Updated weights for policy 0, policy_version 63112 (0.0008) [2023-10-14 20:15:25,444][61585] Updated weights for policy 1, policy_version 62960 (0.0008) [2023-10-14 20:15:25,785][61552] Updated weights for policy 0, policy_version 63122 (0.0008) [2023-10-14 20:15:25,802][61585] Updated weights for policy 1, policy_version 62970 (0.0009) [2023-10-14 20:15:26,161][61552] Updated weights for policy 0, policy_version 63132 (0.0010) [2023-10-14 20:15:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 129138688. Throughput: 0: 1650.8, 1: 1660.7. Samples: 32295438. Policy #0 lag: (min: 28.0, avg: 30.2, max: 60.0) [2023-10-14 20:15:28,344][60425] Avg episode reward: [(0, '76.090'), (1, '73.060')] [2023-10-14 20:15:29,924][61585] Updated weights for policy 1, policy_version 62980 (0.0008) [2023-10-14 20:15:30,294][61585] Updated weights for policy 1, policy_version 62990 (0.0007) [2023-10-14 20:15:30,314][61552] Updated weights for policy 0, policy_version 63142 (0.0008) [2023-10-14 20:15:30,655][61585] Updated weights for policy 1, policy_version 63000 (0.0007) [2023-10-14 20:15:30,683][61552] Updated weights for policy 0, policy_version 63152 (0.0008) [2023-10-14 20:15:31,049][61552] Updated weights for policy 0, policy_version 63162 (0.0010) [2023-10-14 20:15:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 129204224. Throughput: 0: 1650.1, 1: 1653.7. Samples: 32305304. Policy #0 lag: (min: 28.0, avg: 30.2, max: 60.0) [2023-10-14 20:15:33,344][60425] Avg episode reward: [(0, '72.920'), (1, '75.120')] [2023-10-14 20:15:34,853][61585] Updated weights for policy 1, policy_version 63010 (0.0007) [2023-10-14 20:15:35,050][61552] Updated weights for policy 0, policy_version 63172 (0.0010) [2023-10-14 20:15:35,211][61585] Updated weights for policy 1, policy_version 63020 (0.0008) [2023-10-14 20:15:35,416][61552] Updated weights for policy 0, policy_version 63182 (0.0009) [2023-10-14 20:15:35,572][61585] Updated weights for policy 1, policy_version 63030 (0.0010) [2023-10-14 20:15:35,781][61552] Updated weights for policy 0, policy_version 63192 (0.0009) [2023-10-14 20:15:35,936][61585] Updated weights for policy 1, policy_version 63040 (0.0007) [2023-10-14 20:15:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 129269760. Throughput: 0: 1654.2, 1: 1661.6. Samples: 32324866. Policy #0 lag: (min: 28.0, avg: 30.2, max: 60.0) [2023-10-14 20:15:38,344][60425] Avg episode reward: [(0, '79.360'), (1, '74.660')] [2023-10-14 20:15:39,820][61552] Updated weights for policy 0, policy_version 63202 (0.0008) [2023-10-14 20:15:40,077][61585] Updated weights for policy 1, policy_version 63050 (0.0009) [2023-10-14 20:15:40,190][61552] Updated weights for policy 0, policy_version 63212 (0.0007) [2023-10-14 20:15:40,436][61585] Updated weights for policy 1, policy_version 63060 (0.0007) [2023-10-14 20:15:40,552][61552] Updated weights for policy 0, policy_version 63222 (0.0009) [2023-10-14 20:15:40,796][61585] Updated weights for policy 1, policy_version 63070 (0.0007) [2023-10-14 20:15:40,929][61552] Updated weights for policy 0, policy_version 63232 (0.0008) [2023-10-14 20:15:43,344][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 129335296. Throughput: 0: 1650.4, 1: 1663.5. Samples: 32345354. Policy #0 lag: (min: 28.0, avg: 30.2, max: 60.0) [2023-10-14 20:15:43,345][60425] Avg episode reward: [(0, '72.470'), (1, '76.550')] [2023-10-14 20:15:43,354][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000063232_64749568.pth... [2023-10-14 20:15:43,354][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000063072_64585728.pth... [2023-10-14 20:15:43,388][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000061504_62980096.pth [2023-10-14 20:15:43,391][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000061696_63176704.pth [2023-10-14 20:15:44,927][61585] Updated weights for policy 1, policy_version 63080 (0.0009) [2023-10-14 20:15:45,056][61552] Updated weights for policy 0, policy_version 63242 (0.0008) [2023-10-14 20:15:45,279][61585] Updated weights for policy 1, policy_version 63090 (0.0007) [2023-10-14 20:15:45,435][61552] Updated weights for policy 0, policy_version 63252 (0.0008) [2023-10-14 20:15:45,651][61585] Updated weights for policy 1, policy_version 63100 (0.0007) [2023-10-14 20:15:45,807][61552] Updated weights for policy 0, policy_version 63262 (0.0007) [2023-10-14 20:15:48,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 129400832. Throughput: 0: 1644.8, 1: 1654.1. Samples: 32354792. Policy #0 lag: (min: 28.0, avg: 30.2, max: 60.0) [2023-10-14 20:15:48,344][60425] Avg episode reward: [(0, '74.390'), (1, '74.710')] [2023-10-14 20:15:49,668][61585] Updated weights for policy 1, policy_version 63110 (0.0008) [2023-10-14 20:15:49,860][61552] Updated weights for policy 0, policy_version 63272 (0.0009) [2023-10-14 20:15:50,033][61585] Updated weights for policy 1, policy_version 63120 (0.0010) [2023-10-14 20:15:50,240][61552] Updated weights for policy 0, policy_version 63282 (0.0008) [2023-10-14 20:15:50,401][61585] Updated weights for policy 1, policy_version 63130 (0.0008) [2023-10-14 20:15:50,604][61552] Updated weights for policy 0, policy_version 63292 (0.0007) [2023-10-14 20:15:53,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 129466368. Throughput: 0: 1652.4, 1: 1662.3. Samples: 32374838. Policy #0 lag: (min: 28.0, avg: 30.2, max: 60.0) [2023-10-14 20:15:53,344][60425] Avg episode reward: [(0, '73.720'), (1, '75.930')] [2023-10-14 20:15:54,435][61552] Updated weights for policy 0, policy_version 63302 (0.0009) [2023-10-14 20:15:54,602][61585] Updated weights for policy 1, policy_version 63140 (0.0009) [2023-10-14 20:15:54,800][61552] Updated weights for policy 0, policy_version 63312 (0.0010) [2023-10-14 20:15:54,965][61585] Updated weights for policy 1, policy_version 63150 (0.0010) [2023-10-14 20:15:55,164][61552] Updated weights for policy 0, policy_version 63322 (0.0010) [2023-10-14 20:15:55,327][61585] Updated weights for policy 1, policy_version 63160 (0.0008) [2023-10-14 20:15:58,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 129531904. Throughput: 0: 1659.4, 1: 1661.2. Samples: 32395570. Policy #0 lag: (min: 28.0, avg: 30.2, max: 60.0) [2023-10-14 20:15:58,345][60425] Avg episode reward: [(0, '75.660'), (1, '74.260')] [2023-10-14 20:15:59,335][61585] Updated weights for policy 1, policy_version 63170 (0.0008) [2023-10-14 20:15:59,464][61552] Updated weights for policy 0, policy_version 63332 (0.0008) [2023-10-14 20:15:59,707][61585] Updated weights for policy 1, policy_version 63180 (0.0008) [2023-10-14 20:15:59,833][61552] Updated weights for policy 0, policy_version 63342 (0.0008) [2023-10-14 20:16:00,072][61585] Updated weights for policy 1, policy_version 63190 (0.0007) [2023-10-14 20:16:00,197][61552] Updated weights for policy 0, policy_version 63352 (0.0008) [2023-10-14 20:16:00,430][61585] Updated weights for policy 1, policy_version 63200 (0.0009) [2023-10-14 20:16:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 129597440. Throughput: 0: 1648.1, 1: 1651.8. Samples: 32404584. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-14 20:16:03,344][60425] Avg episode reward: [(0, '76.970'), (1, '73.770')] [2023-10-14 20:16:04,426][61552] Updated weights for policy 0, policy_version 63362 (0.0007) [2023-10-14 20:16:04,499][61585] Updated weights for policy 1, policy_version 63210 (0.0008) [2023-10-14 20:16:04,789][61552] Updated weights for policy 0, policy_version 63372 (0.0008) [2023-10-14 20:16:04,862][61585] Updated weights for policy 1, policy_version 63220 (0.0008) [2023-10-14 20:16:05,163][61552] Updated weights for policy 0, policy_version 63382 (0.0009) [2023-10-14 20:16:05,229][61585] Updated weights for policy 1, policy_version 63230 (0.0008) [2023-10-14 20:16:05,543][61552] Updated weights for policy 0, policy_version 63392 (0.0009) [2023-10-14 20:16:08,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 129662976. Throughput: 0: 1665.1, 1: 1664.9. Samples: 32424978. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-14 20:16:08,344][60425] Avg episode reward: [(0, '78.780'), (1, '78.010')] [2023-10-14 20:16:09,330][61585] Updated weights for policy 1, policy_version 63240 (0.0008) [2023-10-14 20:16:09,492][61552] Updated weights for policy 0, policy_version 63402 (0.0008) [2023-10-14 20:16:09,683][61585] Updated weights for policy 1, policy_version 63250 (0.0008) [2023-10-14 20:16:09,860][61552] Updated weights for policy 0, policy_version 63412 (0.0010) [2023-10-14 20:16:10,048][61585] Updated weights for policy 1, policy_version 63260 (0.0010) [2023-10-14 20:16:10,226][61552] Updated weights for policy 0, policy_version 63422 (0.0009) [2023-10-14 20:16:13,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 129728512. Throughput: 0: 1676.6, 1: 1668.3. Samples: 32445956. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-14 20:16:13,344][60425] Avg episode reward: [(0, '78.970'), (1, '77.470')] [2023-10-14 20:16:14,114][61585] Updated weights for policy 1, policy_version 63270 (0.0010) [2023-10-14 20:16:14,457][61552] Updated weights for policy 0, policy_version 63432 (0.0007) [2023-10-14 20:16:14,495][61585] Updated weights for policy 1, policy_version 63280 (0.0009) [2023-10-14 20:16:14,833][61552] Updated weights for policy 0, policy_version 63442 (0.0007) [2023-10-14 20:16:14,847][61585] Updated weights for policy 1, policy_version 63290 (0.0007) [2023-10-14 20:16:15,189][61552] Updated weights for policy 0, policy_version 63452 (0.0009) [2023-10-14 20:16:18,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 129794048. Throughput: 0: 1659.2, 1: 1665.2. Samples: 32454904. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-14 20:16:18,344][60425] Avg episode reward: [(0, '73.770'), (1, '75.690')] [2023-10-14 20:16:18,858][61585] Updated weights for policy 1, policy_version 63300 (0.0010) [2023-10-14 20:16:19,173][61552] Updated weights for policy 0, policy_version 63462 (0.0009) [2023-10-14 20:16:19,216][61585] Updated weights for policy 1, policy_version 63310 (0.0010) [2023-10-14 20:16:19,539][61552] Updated weights for policy 0, policy_version 63472 (0.0008) [2023-10-14 20:16:19,578][61585] Updated weights for policy 1, policy_version 63320 (0.0008) [2023-10-14 20:16:19,907][61552] Updated weights for policy 0, policy_version 63482 (0.0008) [2023-10-14 20:16:23,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 129859584. Throughput: 0: 1673.4, 1: 1673.1. Samples: 32475456. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-14 20:16:23,344][60425] Avg episode reward: [(0, '78.810'), (1, '75.800')] [2023-10-14 20:16:23,747][61585] Updated weights for policy 1, policy_version 63330 (0.0008) [2023-10-14 20:16:24,102][61585] Updated weights for policy 1, policy_version 63340 (0.0008) [2023-10-14 20:16:24,154][61552] Updated weights for policy 0, policy_version 63492 (0.0008) [2023-10-14 20:16:24,465][61585] Updated weights for policy 1, policy_version 63350 (0.0008) [2023-10-14 20:16:24,529][61552] Updated weights for policy 0, policy_version 63502 (0.0009) [2023-10-14 20:16:24,829][61585] Updated weights for policy 1, policy_version 63360 (0.0008) [2023-10-14 20:16:24,888][61552] Updated weights for policy 0, policy_version 63512 (0.0008) [2023-10-14 20:16:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 129925120. Throughput: 0: 1675.9, 1: 1670.9. Samples: 32495960. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-14 20:16:28,344][60425] Avg episode reward: [(0, '77.070'), (1, '74.480')] [2023-10-14 20:16:28,828][61585] Updated weights for policy 1, policy_version 63370 (0.0007) [2023-10-14 20:16:28,985][61552] Updated weights for policy 0, policy_version 63522 (0.0007) [2023-10-14 20:16:29,197][61585] Updated weights for policy 1, policy_version 63380 (0.0007) [2023-10-14 20:16:29,353][61552] Updated weights for policy 0, policy_version 63532 (0.0009) [2023-10-14 20:16:29,569][61585] Updated weights for policy 1, policy_version 63390 (0.0009) [2023-10-14 20:16:29,720][61552] Updated weights for policy 0, policy_version 63542 (0.0010) [2023-10-14 20:16:30,076][61552] Updated weights for policy 0, policy_version 63552 (0.0008) [2023-10-14 20:16:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 129990656. Throughput: 0: 1672.2, 1: 1670.1. Samples: 32505198. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-14 20:16:33,344][60425] Avg episode reward: [(0, '78.540'), (1, '72.990')] [2023-10-14 20:16:33,669][61585] Updated weights for policy 1, policy_version 63400 (0.0008) [2023-10-14 20:16:33,942][61552] Updated weights for policy 0, policy_version 63562 (0.0007) [2023-10-14 20:16:34,033][61585] Updated weights for policy 1, policy_version 63410 (0.0008) [2023-10-14 20:16:34,296][61552] Updated weights for policy 0, policy_version 63572 (0.0009) [2023-10-14 20:16:34,389][61585] Updated weights for policy 1, policy_version 63420 (0.0007) [2023-10-14 20:16:34,666][61552] Updated weights for policy 0, policy_version 63582 (0.0008) [2023-10-14 20:16:38,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 130056192. Throughput: 0: 1678.7, 1: 1679.7. Samples: 32525966. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-14 20:16:38,344][60425] Avg episode reward: [(0, '76.000'), (1, '73.280')] [2023-10-14 20:16:38,345][61585] Updated weights for policy 1, policy_version 63430 (0.0009) [2023-10-14 20:16:38,703][61585] Updated weights for policy 1, policy_version 63440 (0.0009) [2023-10-14 20:16:38,789][61552] Updated weights for policy 0, policy_version 63592 (0.0007) [2023-10-14 20:16:39,078][61585] Updated weights for policy 1, policy_version 63450 (0.0009) [2023-10-14 20:16:39,158][61552] Updated weights for policy 0, policy_version 63602 (0.0009) [2023-10-14 20:16:39,523][61552] Updated weights for policy 0, policy_version 63612 (0.0011) [2023-10-14 20:16:43,330][61585] Updated weights for policy 1, policy_version 63460 (0.0009) [2023-10-14 20:16:43,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 130121728. Throughput: 0: 1676.1, 1: 1680.8. Samples: 32546628. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-14 20:16:43,344][60425] Avg episode reward: [(0, '77.240'), (1, '77.810')] [2023-10-14 20:16:43,654][61552] Updated weights for policy 0, policy_version 63622 (0.0008) [2023-10-14 20:16:43,691][61585] Updated weights for policy 1, policy_version 63470 (0.0008) [2023-10-14 20:16:44,023][61552] Updated weights for policy 0, policy_version 63632 (0.0008) [2023-10-14 20:16:44,060][61585] Updated weights for policy 1, policy_version 63480 (0.0009) [2023-10-14 20:16:44,393][61552] Updated weights for policy 0, policy_version 63642 (0.0008) [2023-10-14 20:16:48,192][61585] Updated weights for policy 1, policy_version 63490 (0.0011) [2023-10-14 20:16:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 130187264. Throughput: 0: 1678.6, 1: 1679.2. Samples: 32555686. Policy #0 lag: (min: 2.0, avg: 4.2, max: 26.0) [2023-10-14 20:16:48,344][60425] Avg episode reward: [(0, '75.810'), (1, '79.300')] [2023-10-14 20:16:48,514][61552] Updated weights for policy 0, policy_version 63652 (0.0008) [2023-10-14 20:16:48,555][61585] Updated weights for policy 1, policy_version 63500 (0.0008) [2023-10-14 20:16:48,877][61552] Updated weights for policy 0, policy_version 63662 (0.0008) [2023-10-14 20:16:48,928][61585] Updated weights for policy 1, policy_version 63510 (0.0008) [2023-10-14 20:16:49,243][61552] Updated weights for policy 0, policy_version 63672 (0.0008) [2023-10-14 20:16:49,292][61585] Updated weights for policy 1, policy_version 63520 (0.0008) [2023-10-14 20:16:53,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 130252800. Throughput: 0: 1680.2, 1: 1673.8. Samples: 32575908. Policy #0 lag: (min: 2.0, avg: 4.2, max: 26.0) [2023-10-14 20:16:53,344][60425] Avg episode reward: [(0, '79.050'), (1, '79.010')] [2023-10-14 20:16:53,415][61552] Updated weights for policy 0, policy_version 63682 (0.0007) [2023-10-14 20:16:53,485][61585] Updated weights for policy 1, policy_version 63530 (0.0008) [2023-10-14 20:16:53,790][61552] Updated weights for policy 0, policy_version 63692 (0.0008) [2023-10-14 20:16:53,847][61585] Updated weights for policy 1, policy_version 63540 (0.0007) [2023-10-14 20:16:54,153][61552] Updated weights for policy 0, policy_version 63702 (0.0009) [2023-10-14 20:16:54,221][61585] Updated weights for policy 1, policy_version 63550 (0.0009) [2023-10-14 20:16:54,513][61552] Updated weights for policy 0, policy_version 63712 (0.0011) [2023-10-14 20:16:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 130318336. Throughput: 0: 1674.9, 1: 1669.4. Samples: 32596448. Policy #0 lag: (min: 2.0, avg: 4.2, max: 26.0) [2023-10-14 20:16:58,344][60425] Avg episode reward: [(0, '75.400'), (1, '77.080')] [2023-10-14 20:16:58,483][61585] Updated weights for policy 1, policy_version 63560 (0.0007) [2023-10-14 20:16:58,655][61552] Updated weights for policy 0, policy_version 63722 (0.0007) [2023-10-14 20:16:58,846][61585] Updated weights for policy 1, policy_version 63570 (0.0007) [2023-10-14 20:16:59,025][61552] Updated weights for policy 0, policy_version 63732 (0.0007) [2023-10-14 20:16:59,216][61585] Updated weights for policy 1, policy_version 63580 (0.0007) [2023-10-14 20:16:59,391][61552] Updated weights for policy 0, policy_version 63742 (0.0007) [2023-10-14 20:17:03,325][61585] Updated weights for policy 1, policy_version 63590 (0.0008) [2023-10-14 20:17:03,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 130383872. Throughput: 0: 1676.8, 1: 1669.4. Samples: 32605484. Policy #0 lag: (min: 2.0, avg: 4.2, max: 26.0) [2023-10-14 20:17:03,345][60425] Avg episode reward: [(0, '79.200'), (1, '77.460')] [2023-10-14 20:17:03,565][61552] Updated weights for policy 0, policy_version 63752 (0.0008) [2023-10-14 20:17:03,703][61585] Updated weights for policy 1, policy_version 63600 (0.0009) [2023-10-14 20:17:03,921][61552] Updated weights for policy 0, policy_version 63762 (0.0010) [2023-10-14 20:17:04,071][61585] Updated weights for policy 1, policy_version 63610 (0.0007) [2023-10-14 20:17:04,286][61552] Updated weights for policy 0, policy_version 63772 (0.0009) [2023-10-14 20:17:08,203][61585] Updated weights for policy 1, policy_version 63620 (0.0008) [2023-10-14 20:17:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 130449408. Throughput: 0: 1674.7, 1: 1670.3. Samples: 32625980. Policy #0 lag: (min: 2.0, avg: 4.2, max: 26.0) [2023-10-14 20:17:08,344][60425] Avg episode reward: [(0, '78.080'), (1, '78.720')] [2023-10-14 20:17:08,379][61552] Updated weights for policy 0, policy_version 63782 (0.0010) [2023-10-14 20:17:08,559][61585] Updated weights for policy 1, policy_version 63630 (0.0007) [2023-10-14 20:17:08,742][61552] Updated weights for policy 0, policy_version 63792 (0.0009) [2023-10-14 20:17:08,927][61585] Updated weights for policy 1, policy_version 63640 (0.0008) [2023-10-14 20:17:09,105][61552] Updated weights for policy 0, policy_version 63802 (0.0009) [2023-10-14 20:17:13,078][61585] Updated weights for policy 1, policy_version 63650 (0.0008) [2023-10-14 20:17:13,089][61552] Updated weights for policy 0, policy_version 63812 (0.0007) [2023-10-14 20:17:13,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 130514944. Throughput: 0: 1674.5, 1: 1672.2. Samples: 32646564. Policy #0 lag: (min: 2.0, avg: 4.2, max: 26.0) [2023-10-14 20:17:13,344][60425] Avg episode reward: [(0, '75.550'), (1, '75.410')] [2023-10-14 20:17:13,446][61585] Updated weights for policy 1, policy_version 63660 (0.0008) [2023-10-14 20:17:13,451][61552] Updated weights for policy 0, policy_version 63822 (0.0007) [2023-10-14 20:17:13,801][61585] Updated weights for policy 1, policy_version 63670 (0.0007) [2023-10-14 20:17:13,822][61552] Updated weights for policy 0, policy_version 63832 (0.0007) [2023-10-14 20:17:14,162][61585] Updated weights for policy 1, policy_version 63680 (0.0009) [2023-10-14 20:17:18,051][61552] Updated weights for policy 0, policy_version 63842 (0.0009) [2023-10-14 20:17:18,325][61585] Updated weights for policy 1, policy_version 63690 (0.0009) [2023-10-14 20:17:18,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 130580480. Throughput: 0: 1667.9, 1: 1671.3. Samples: 32655462. Policy #0 lag: (min: 2.0, avg: 4.2, max: 26.0) [2023-10-14 20:17:18,344][60425] Avg episode reward: [(0, '75.110'), (1, '72.900')] [2023-10-14 20:17:18,426][61552] Updated weights for policy 0, policy_version 63852 (0.0007) [2023-10-14 20:17:18,693][61585] Updated weights for policy 1, policy_version 63700 (0.0008) [2023-10-14 20:17:18,796][61552] Updated weights for policy 0, policy_version 63862 (0.0007) [2023-10-14 20:17:19,050][61585] Updated weights for policy 1, policy_version 63710 (0.0007) [2023-10-14 20:17:19,157][61552] Updated weights for policy 0, policy_version 63872 (0.0008) [2023-10-14 20:17:23,059][61552] Updated weights for policy 0, policy_version 63882 (0.0008) [2023-10-14 20:17:23,261][61585] Updated weights for policy 1, policy_version 63720 (0.0007) [2023-10-14 20:17:23,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 130646016. Throughput: 0: 1670.0, 1: 1661.1. Samples: 32675866. Policy #0 lag: (min: 2.0, avg: 4.2, max: 26.0) [2023-10-14 20:17:23,344][60425] Avg episode reward: [(0, '76.880'), (1, '73.860')] [2023-10-14 20:17:23,439][61552] Updated weights for policy 0, policy_version 63892 (0.0008) [2023-10-14 20:17:23,625][61585] Updated weights for policy 1, policy_version 63730 (0.0007) [2023-10-14 20:17:23,815][61552] Updated weights for policy 0, policy_version 63902 (0.0007) [2023-10-14 20:17:23,988][61585] Updated weights for policy 1, policy_version 63740 (0.0008) [2023-10-14 20:17:27,995][61552] Updated weights for policy 0, policy_version 63912 (0.0010) [2023-10-14 20:17:28,019][61585] Updated weights for policy 1, policy_version 63750 (0.0007) [2023-10-14 20:17:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 130711552. Throughput: 0: 1665.5, 1: 1661.8. Samples: 32696354. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:17:28,344][60425] Avg episode reward: [(0, '77.050'), (1, '74.040')] [2023-10-14 20:17:28,361][61552] Updated weights for policy 0, policy_version 63922 (0.0009) [2023-10-14 20:17:28,383][61585] Updated weights for policy 1, policy_version 63760 (0.0009) [2023-10-14 20:17:28,734][61552] Updated weights for policy 0, policy_version 63932 (0.0007) [2023-10-14 20:17:28,743][61585] Updated weights for policy 1, policy_version 63770 (0.0009) [2023-10-14 20:17:32,798][61585] Updated weights for policy 1, policy_version 63780 (0.0007) [2023-10-14 20:17:32,905][61552] Updated weights for policy 0, policy_version 63942 (0.0007) [2023-10-14 20:17:33,159][61585] Updated weights for policy 1, policy_version 63790 (0.0010) [2023-10-14 20:17:33,275][61552] Updated weights for policy 0, policy_version 63952 (0.0011) [2023-10-14 20:17:33,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 130777088. Throughput: 0: 1662.7, 1: 1665.3. Samples: 32705446. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:17:33,344][60425] Avg episode reward: [(0, '75.880'), (1, '74.210')] [2023-10-14 20:17:33,519][61585] Updated weights for policy 1, policy_version 63800 (0.0007) [2023-10-14 20:17:33,638][61552] Updated weights for policy 0, policy_version 63962 (0.0008) [2023-10-14 20:17:37,612][61585] Updated weights for policy 1, policy_version 63810 (0.0007) [2023-10-14 20:17:37,771][61552] Updated weights for policy 0, policy_version 63972 (0.0008) [2023-10-14 20:17:37,977][61585] Updated weights for policy 1, policy_version 63820 (0.0008) [2023-10-14 20:17:38,144][61552] Updated weights for policy 0, policy_version 63982 (0.0008) [2023-10-14 20:17:38,332][61585] Updated weights for policy 1, policy_version 63830 (0.0009) [2023-10-14 20:17:38,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 130842624. Throughput: 0: 1664.0, 1: 1673.4. Samples: 32726088. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:17:38,344][60425] Avg episode reward: [(0, '73.210'), (1, '75.300')] [2023-10-14 20:17:38,506][61552] Updated weights for policy 0, policy_version 63992 (0.0007) [2023-10-14 20:17:38,700][61585] Updated weights for policy 1, policy_version 63840 (0.0008) [2023-10-14 20:17:42,294][61552] Updated weights for policy 0, policy_version 64002 (0.0007) [2023-10-14 20:17:42,657][61552] Updated weights for policy 0, policy_version 64012 (0.0009) [2023-10-14 20:17:42,872][61585] Updated weights for policy 1, policy_version 63850 (0.0007) [2023-10-14 20:17:43,034][61552] Updated weights for policy 0, policy_version 64022 (0.0007) [2023-10-14 20:17:43,244][61585] Updated weights for policy 1, policy_version 63860 (0.0007) [2023-10-14 20:17:43,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 130908160. Throughput: 0: 1658.8, 1: 1663.9. Samples: 32745970. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:17:43,344][60425] Avg episode reward: [(0, '74.780'), (1, '73.360')] [2023-10-14 20:17:43,390][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000064032_65568768.pth... [2023-10-14 20:17:43,394][61552] Updated weights for policy 0, policy_version 64032 (0.0007) [2023-10-14 20:17:43,419][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000062464_63963136.pth [2023-10-14 20:17:43,611][61585] Updated weights for policy 1, policy_version 63870 (0.0009) [2023-10-14 20:17:43,678][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000063872_65404928.pth... [2023-10-14 20:17:43,714][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000062304_63799296.pth [2023-10-14 20:17:47,554][61552] Updated weights for policy 0, policy_version 64042 (0.0007) [2023-10-14 20:17:47,703][61585] Updated weights for policy 1, policy_version 63880 (0.0008) [2023-10-14 20:17:47,922][61552] Updated weights for policy 0, policy_version 64052 (0.0009) [2023-10-14 20:17:48,067][61585] Updated weights for policy 1, policy_version 63890 (0.0009) [2023-10-14 20:17:48,286][61552] Updated weights for policy 0, policy_version 64062 (0.0010) [2023-10-14 20:17:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 130973696. Throughput: 0: 1670.0, 1: 1668.0. Samples: 32755692. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:17:48,344][60425] Avg episode reward: [(0, '76.980'), (1, '75.170')] [2023-10-14 20:17:48,429][61585] Updated weights for policy 1, policy_version 63900 (0.0009) [2023-10-14 20:17:52,266][61552] Updated weights for policy 0, policy_version 64072 (0.0008) [2023-10-14 20:17:52,496][61585] Updated weights for policy 1, policy_version 63910 (0.0007) [2023-10-14 20:17:52,639][61552] Updated weights for policy 0, policy_version 64082 (0.0007) [2023-10-14 20:17:52,887][61585] Updated weights for policy 1, policy_version 63920 (0.0007) [2023-10-14 20:17:53,004][61552] Updated weights for policy 0, policy_version 64092 (0.0007) [2023-10-14 20:17:53,260][61585] Updated weights for policy 1, policy_version 63930 (0.0009) [2023-10-14 20:17:53,343][60425] Fps is (10 sec: 16384.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 131072000. Throughput: 0: 1674.3, 1: 1667.7. Samples: 32776370. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:17:53,344][60425] Avg episode reward: [(0, '74.190'), (1, '74.720')] [2023-10-14 20:17:57,116][61552] Updated weights for policy 0, policy_version 64102 (0.0010) [2023-10-14 20:17:57,337][61585] Updated weights for policy 1, policy_version 63940 (0.0008) [2023-10-14 20:17:57,490][61552] Updated weights for policy 0, policy_version 64112 (0.0009) [2023-10-14 20:17:57,694][61585] Updated weights for policy 1, policy_version 63950 (0.0009) [2023-10-14 20:17:57,859][61552] Updated weights for policy 0, policy_version 64122 (0.0008) [2023-10-14 20:17:58,064][61585] Updated weights for policy 1, policy_version 63960 (0.0008) [2023-10-14 20:17:58,343][60425] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 131137536. Throughput: 0: 1651.8, 1: 1655.1. Samples: 32795374. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:17:58,344][60425] Avg episode reward: [(0, '74.770'), (1, '74.360')] [2023-10-14 20:18:02,030][61552] Updated weights for policy 0, policy_version 64132 (0.0008) [2023-10-14 20:18:02,339][61585] Updated weights for policy 1, policy_version 63970 (0.0010) [2023-10-14 20:18:02,401][61552] Updated weights for policy 0, policy_version 64142 (0.0009) [2023-10-14 20:18:02,702][61585] Updated weights for policy 1, policy_version 63980 (0.0008) [2023-10-14 20:18:02,767][61552] Updated weights for policy 0, policy_version 64152 (0.0008) [2023-10-14 20:18:03,067][61585] Updated weights for policy 1, policy_version 63990 (0.0008) [2023-10-14 20:18:03,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 131203072. Throughput: 0: 1674.0, 1: 1664.4. Samples: 32805692. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:18:03,344][60425] Avg episode reward: [(0, '74.020'), (1, '77.630')] [2023-10-14 20:18:03,436][61585] Updated weights for policy 1, policy_version 64000 (0.0009) [2023-10-14 20:18:06,887][61552] Updated weights for policy 0, policy_version 64162 (0.0008) [2023-10-14 20:18:07,264][61552] Updated weights for policy 0, policy_version 64172 (0.0010) [2023-10-14 20:18:07,530][61585] Updated weights for policy 1, policy_version 64010 (0.0008) [2023-10-14 20:18:07,622][61552] Updated weights for policy 0, policy_version 64182 (0.0010) [2023-10-14 20:18:07,899][61585] Updated weights for policy 1, policy_version 64020 (0.0007) [2023-10-14 20:18:07,995][61552] Updated weights for policy 0, policy_version 64192 (0.0007) [2023-10-14 20:18:08,262][61585] Updated weights for policy 1, policy_version 64030 (0.0007) [2023-10-14 20:18:08,344][60425] Fps is (10 sec: 16383.5, 60 sec: 14199.4, 300 sec: 13329.3). Total num frames: 131301376. Throughput: 0: 1674.8, 1: 1670.7. Samples: 32826412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:18:08,345][60425] Avg episode reward: [(0, '78.960'), (1, '75.660')] [2023-10-14 20:18:12,150][61552] Updated weights for policy 0, policy_version 64202 (0.0007) [2023-10-14 20:18:12,152][61585] Updated weights for policy 1, policy_version 64040 (0.0007) [2023-10-14 20:18:12,519][61552] Updated weights for policy 0, policy_version 64212 (0.0007) [2023-10-14 20:18:12,524][61585] Updated weights for policy 1, policy_version 64050 (0.0008) [2023-10-14 20:18:12,883][61585] Updated weights for policy 1, policy_version 64060 (0.0009) [2023-10-14 20:18:12,887][61552] Updated weights for policy 0, policy_version 64222 (0.0009) [2023-10-14 20:18:13,343][60425] Fps is (10 sec: 16383.7, 60 sec: 14199.4, 300 sec: 13329.4). Total num frames: 131366912. Throughput: 0: 1651.3, 1: 1655.9. Samples: 32845178. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:18:13,344][60425] Avg episode reward: [(0, '71.400'), (1, '73.730')] [2023-10-14 20:18:17,054][61585] Updated weights for policy 1, policy_version 64070 (0.0007) [2023-10-14 20:18:17,156][61552] Updated weights for policy 0, policy_version 64232 (0.0008) [2023-10-14 20:18:17,412][61585] Updated weights for policy 1, policy_version 64080 (0.0010) [2023-10-14 20:18:17,524][61552] Updated weights for policy 0, policy_version 64242 (0.0009) [2023-10-14 20:18:17,772][61585] Updated weights for policy 1, policy_version 64090 (0.0010) [2023-10-14 20:18:17,893][61552] Updated weights for policy 0, policy_version 64252 (0.0008) [2023-10-14 20:18:18,343][60425] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 131432448. Throughput: 0: 1673.7, 1: 1672.1. Samples: 32856010. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:18:18,345][60425] Avg episode reward: [(0, '76.460'), (1, '72.380')] [2023-10-14 20:18:21,966][61585] Updated weights for policy 1, policy_version 64100 (0.0009) [2023-10-14 20:18:22,031][61552] Updated weights for policy 0, policy_version 64262 (0.0007) [2023-10-14 20:18:22,326][61585] Updated weights for policy 1, policy_version 64110 (0.0007) [2023-10-14 20:18:22,397][61552] Updated weights for policy 0, policy_version 64272 (0.0009) [2023-10-14 20:18:22,696][61585] Updated weights for policy 1, policy_version 64120 (0.0008) [2023-10-14 20:18:22,768][61552] Updated weights for policy 0, policy_version 64282 (0.0009) [2023-10-14 20:18:23,343][60425] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 131497984. Throughput: 0: 1676.8, 1: 1662.4. Samples: 32876352. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:18:23,344][60425] Avg episode reward: [(0, '75.290'), (1, '73.910')] [2023-10-14 20:18:26,858][61552] Updated weights for policy 0, policy_version 64292 (0.0009) [2023-10-14 20:18:26,914][61585] Updated weights for policy 1, policy_version 64130 (0.0010) [2023-10-14 20:18:27,229][61552] Updated weights for policy 0, policy_version 64302 (0.0010) [2023-10-14 20:18:27,285][61585] Updated weights for policy 1, policy_version 64140 (0.0010) [2023-10-14 20:18:27,600][61552] Updated weights for policy 0, policy_version 64312 (0.0008) [2023-10-14 20:18:27,649][61585] Updated weights for policy 1, policy_version 64150 (0.0008) [2023-10-14 20:18:28,003][61585] Updated weights for policy 1, policy_version 64160 (0.0009) [2023-10-14 20:18:28,343][60425] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 131563520. Throughput: 0: 1656.6, 1: 1649.6. Samples: 32894750. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:18:28,344][60425] Avg episode reward: [(0, '76.670'), (1, '76.460')] [2023-10-14 20:18:31,891][61552] Updated weights for policy 0, policy_version 64322 (0.0008) [2023-10-14 20:18:32,234][61585] Updated weights for policy 1, policy_version 64170 (0.0008) [2023-10-14 20:18:32,282][61552] Updated weights for policy 0, policy_version 64332 (0.0007) [2023-10-14 20:18:32,592][61585] Updated weights for policy 1, policy_version 64180 (0.0007) [2023-10-14 20:18:32,652][61552] Updated weights for policy 0, policy_version 64342 (0.0008) [2023-10-14 20:18:32,952][61585] Updated weights for policy 1, policy_version 64190 (0.0008) [2023-10-14 20:18:33,015][61552] Updated weights for policy 0, policy_version 64352 (0.0007) [2023-10-14 20:18:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 131629056. Throughput: 0: 1664.8, 1: 1664.3. Samples: 32905504. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:18:33,344][60425] Avg episode reward: [(0, '74.060'), (1, '80.150')] [2023-10-14 20:18:36,992][61552] Updated weights for policy 0, policy_version 64362 (0.0009) [2023-10-14 20:18:37,028][61585] Updated weights for policy 1, policy_version 64200 (0.0008) [2023-10-14 20:18:37,348][61552] Updated weights for policy 0, policy_version 64372 (0.0008) [2023-10-14 20:18:37,398][61585] Updated weights for policy 1, policy_version 64210 (0.0008) [2023-10-14 20:18:37,726][61552] Updated weights for policy 0, policy_version 64382 (0.0007) [2023-10-14 20:18:37,762][61585] Updated weights for policy 1, policy_version 64220 (0.0009) [2023-10-14 20:18:38,343][60425] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13329.4). Total num frames: 131694592. Throughput: 0: 1654.8, 1: 1663.3. Samples: 32925688. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:18:38,344][60425] Avg episode reward: [(0, '76.180'), (1, '75.440')] [2023-10-14 20:18:41,850][61585] Updated weights for policy 1, policy_version 64230 (0.0008) [2023-10-14 20:18:41,917][61552] Updated weights for policy 0, policy_version 64392 (0.0012) [2023-10-14 20:18:42,205][61585] Updated weights for policy 1, policy_version 64240 (0.0007) [2023-10-14 20:18:42,296][61552] Updated weights for policy 0, policy_version 64402 (0.0009) [2023-10-14 20:18:42,568][61585] Updated weights for policy 1, policy_version 64250 (0.0009) [2023-10-14 20:18:42,660][61552] Updated weights for policy 0, policy_version 64412 (0.0009) [2023-10-14 20:18:43,344][60425] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 13329.3). Total num frames: 131760128. Throughput: 0: 1654.2, 1: 1652.0. Samples: 32944154. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:18:43,345][60425] Avg episode reward: [(0, '76.670'), (1, '75.250')] [2023-10-14 20:18:46,650][61552] Updated weights for policy 0, policy_version 64422 (0.0007) [2023-10-14 20:18:46,676][61585] Updated weights for policy 1, policy_version 64260 (0.0009) [2023-10-14 20:18:47,012][61552] Updated weights for policy 0, policy_version 64432 (0.0008) [2023-10-14 20:18:47,036][61585] Updated weights for policy 1, policy_version 64270 (0.0009) [2023-10-14 20:18:47,376][61552] Updated weights for policy 0, policy_version 64442 (0.0009) [2023-10-14 20:18:47,397][61585] Updated weights for policy 1, policy_version 64280 (0.0007) [2023-10-14 20:18:48,343][60425] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 131825664. Throughput: 0: 1662.3, 1: 1665.7. Samples: 32955450. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:18:48,344][60425] Avg episode reward: [(0, '77.580'), (1, '75.370')] [2023-10-14 20:18:51,569][61552] Updated weights for policy 0, policy_version 64452 (0.0009) [2023-10-14 20:18:51,600][61585] Updated weights for policy 1, policy_version 64290 (0.0008) [2023-10-14 20:18:51,938][61552] Updated weights for policy 0, policy_version 64462 (0.0007) [2023-10-14 20:18:51,961][61585] Updated weights for policy 1, policy_version 64300 (0.0007) [2023-10-14 20:18:52,296][61552] Updated weights for policy 0, policy_version 64472 (0.0008) [2023-10-14 20:18:52,324][61585] Updated weights for policy 1, policy_version 64310 (0.0008) [2023-10-14 20:18:52,687][61585] Updated weights for policy 1, policy_version 64320 (0.0008) [2023-10-14 20:18:53,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 131891200. Throughput: 0: 1650.9, 1: 1655.4. Samples: 32975198. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:18:53,344][60425] Avg episode reward: [(0, '76.080'), (1, '78.800')] [2023-10-14 20:18:56,450][61552] Updated weights for policy 0, policy_version 64482 (0.0007) [2023-10-14 20:18:56,731][61585] Updated weights for policy 1, policy_version 64330 (0.0007) [2023-10-14 20:18:56,818][61552] Updated weights for policy 0, policy_version 64492 (0.0009) [2023-10-14 20:18:57,090][61585] Updated weights for policy 1, policy_version 64340 (0.0007) [2023-10-14 20:18:57,184][61552] Updated weights for policy 0, policy_version 64502 (0.0009) [2023-10-14 20:18:57,448][61585] Updated weights for policy 1, policy_version 64350 (0.0008) [2023-10-14 20:18:57,545][61552] Updated weights for policy 0, policy_version 64512 (0.0008) [2023-10-14 20:18:58,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 131956736. Throughput: 0: 1650.7, 1: 1650.8. Samples: 32993744. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:18:58,345][60425] Avg episode reward: [(0, '78.810'), (1, '75.620')] [2023-10-14 20:19:01,655][61585] Updated weights for policy 1, policy_version 64360 (0.0010) [2023-10-14 20:19:01,872][61552] Updated weights for policy 0, policy_version 64522 (0.0007) [2023-10-14 20:19:02,017][61585] Updated weights for policy 1, policy_version 64370 (0.0009) [2023-10-14 20:19:02,236][61552] Updated weights for policy 0, policy_version 64532 (0.0008) [2023-10-14 20:19:02,383][61585] Updated weights for policy 1, policy_version 64380 (0.0008) [2023-10-14 20:19:02,603][61552] Updated weights for policy 0, policy_version 64542 (0.0009) [2023-10-14 20:19:03,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 132022272. Throughput: 0: 1657.7, 1: 1659.5. Samples: 33005286. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:19:03,344][60425] Avg episode reward: [(0, '82.730'), (1, '75.800')] [2023-10-14 20:19:03,345][61172] Saving new best policy, reward=82.730! [2023-10-14 20:19:06,525][61585] Updated weights for policy 1, policy_version 64390 (0.0009) [2023-10-14 20:19:06,713][61552] Updated weights for policy 0, policy_version 64552 (0.0009) [2023-10-14 20:19:06,897][61585] Updated weights for policy 1, policy_version 64400 (0.0008) [2023-10-14 20:19:07,083][61552] Updated weights for policy 0, policy_version 64562 (0.0008) [2023-10-14 20:19:07,261][61585] Updated weights for policy 1, policy_version 64410 (0.0008) [2023-10-14 20:19:07,440][61552] Updated weights for policy 0, policy_version 64572 (0.0008) [2023-10-14 20:19:08,344][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 132087808. Throughput: 0: 1643.9, 1: 1654.6. Samples: 33024786. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:19:08,345][60425] Avg episode reward: [(0, '74.870'), (1, '74.770')] [2023-10-14 20:19:11,439][61585] Updated weights for policy 1, policy_version 64420 (0.0008) [2023-10-14 20:19:11,717][61552] Updated weights for policy 0, policy_version 64582 (0.0007) [2023-10-14 20:19:11,806][61585] Updated weights for policy 1, policy_version 64430 (0.0009) [2023-10-14 20:19:12,083][61552] Updated weights for policy 0, policy_version 64592 (0.0010) [2023-10-14 20:19:12,164][61585] Updated weights for policy 1, policy_version 64440 (0.0008) [2023-10-14 20:19:12,463][61552] Updated weights for policy 0, policy_version 64602 (0.0008) [2023-10-14 20:19:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 132153344. Throughput: 0: 1646.5, 1: 1658.8. Samples: 33043488. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:19:13,345][60425] Avg episode reward: [(0, '80.460'), (1, '72.610')] [2023-10-14 20:19:16,278][61585] Updated weights for policy 1, policy_version 64450 (0.0008) [2023-10-14 20:19:16,629][61552] Updated weights for policy 0, policy_version 64612 (0.0009) [2023-10-14 20:19:16,640][61585] Updated weights for policy 1, policy_version 64460 (0.0009) [2023-10-14 20:19:16,998][61585] Updated weights for policy 1, policy_version 64470 (0.0007) [2023-10-14 20:19:17,019][61552] Updated weights for policy 0, policy_version 64622 (0.0007) [2023-10-14 20:19:17,364][61585] Updated weights for policy 1, policy_version 64480 (0.0007) [2023-10-14 20:19:17,394][61552] Updated weights for policy 0, policy_version 64632 (0.0008) [2023-10-14 20:19:18,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 132218880. Throughput: 0: 1655.0, 1: 1669.0. Samples: 33055084. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:19:18,344][60425] Avg episode reward: [(0, '78.410'), (1, '79.690')] [2023-10-14 20:19:21,431][61552] Updated weights for policy 0, policy_version 64642 (0.0008) [2023-10-14 20:19:21,492][61585] Updated weights for policy 1, policy_version 64490 (0.0009) [2023-10-14 20:19:21,796][61552] Updated weights for policy 0, policy_version 64652 (0.0008) [2023-10-14 20:19:21,851][61585] Updated weights for policy 1, policy_version 64500 (0.0009) [2023-10-14 20:19:22,160][61552] Updated weights for policy 0, policy_version 64662 (0.0008) [2023-10-14 20:19:22,215][61585] Updated weights for policy 1, policy_version 64510 (0.0008) [2023-10-14 20:19:22,523][61552] Updated weights for policy 0, policy_version 64672 (0.0009) [2023-10-14 20:19:23,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 132284416. Throughput: 0: 1651.2, 1: 1654.7. Samples: 33074454. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:19:23,344][60425] Avg episode reward: [(0, '76.890'), (1, '76.040')] [2023-10-14 20:19:26,238][61585] Updated weights for policy 1, policy_version 64520 (0.0008) [2023-10-14 20:19:26,590][61585] Updated weights for policy 1, policy_version 64530 (0.0010) [2023-10-14 20:19:26,882][61552] Updated weights for policy 0, policy_version 64682 (0.0009) [2023-10-14 20:19:26,952][61585] Updated weights for policy 1, policy_version 64540 (0.0007) [2023-10-14 20:19:27,252][61552] Updated weights for policy 0, policy_version 64692 (0.0010) [2023-10-14 20:19:27,612][61552] Updated weights for policy 0, policy_version 64702 (0.0010) [2023-10-14 20:19:28,344][60425] Fps is (10 sec: 13106.6, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 132349952. Throughput: 0: 1646.8, 1: 1669.2. Samples: 33093372. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:19:28,345][60425] Avg episode reward: [(0, '74.290'), (1, '77.220')] [2023-10-14 20:19:31,040][61585] Updated weights for policy 1, policy_version 64550 (0.0010) [2023-10-14 20:19:31,405][61585] Updated weights for policy 1, policy_version 64560 (0.0009) [2023-10-14 20:19:31,755][61552] Updated weights for policy 0, policy_version 64712 (0.0008) [2023-10-14 20:19:31,769][61585] Updated weights for policy 1, policy_version 64570 (0.0007) [2023-10-14 20:19:32,118][61552] Updated weights for policy 0, policy_version 64722 (0.0009) [2023-10-14 20:19:32,484][61552] Updated weights for policy 0, policy_version 64732 (0.0008) [2023-10-14 20:19:33,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 132415488. Throughput: 0: 1647.5, 1: 1670.8. Samples: 33104770. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:19:33,344][60425] Avg episode reward: [(0, '78.210'), (1, '73.980')] [2023-10-14 20:19:35,765][61585] Updated weights for policy 1, policy_version 64580 (0.0009) [2023-10-14 20:19:36,129][61585] Updated weights for policy 1, policy_version 64590 (0.0010) [2023-10-14 20:19:36,503][61585] Updated weights for policy 1, policy_version 64600 (0.0009) [2023-10-14 20:19:36,613][61552] Updated weights for policy 0, policy_version 64742 (0.0008) [2023-10-14 20:19:36,979][61552] Updated weights for policy 0, policy_version 64752 (0.0009) [2023-10-14 20:19:37,350][61552] Updated weights for policy 0, policy_version 64762 (0.0010) [2023-10-14 20:19:38,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 132481024. Throughput: 0: 1649.2, 1: 1655.0. Samples: 33123890. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:19:38,344][60425] Avg episode reward: [(0, '77.020'), (1, '75.680')] [2023-10-14 20:19:40,536][61585] Updated weights for policy 1, policy_version 64610 (0.0008) [2023-10-14 20:19:40,906][61585] Updated weights for policy 1, policy_version 64620 (0.0008) [2023-10-14 20:19:41,269][61585] Updated weights for policy 1, policy_version 64630 (0.0010) [2023-10-14 20:19:41,541][61552] Updated weights for policy 0, policy_version 64772 (0.0009) [2023-10-14 20:19:41,634][61585] Updated weights for policy 1, policy_version 64640 (0.0009) [2023-10-14 20:19:41,903][61552] Updated weights for policy 0, policy_version 64782 (0.0010) [2023-10-14 20:19:42,267][61552] Updated weights for policy 0, policy_version 64792 (0.0010) [2023-10-14 20:19:43,344][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 132546560. Throughput: 0: 1652.3, 1: 1677.8. Samples: 33143598. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:19:43,345][60425] Avg episode reward: [(0, '82.480'), (1, '74.970')] [2023-10-14 20:19:43,355][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000064640_66191360.pth... [2023-10-14 20:19:43,355][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000064800_66355200.pth... [2023-10-14 20:19:43,391][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000063232_64749568.pth [2023-10-14 20:19:43,393][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000063072_64585728.pth [2023-10-14 20:19:45,714][61585] Updated weights for policy 1, policy_version 64650 (0.0008) [2023-10-14 20:19:46,065][61585] Updated weights for policy 1, policy_version 64660 (0.0010) [2023-10-14 20:19:46,282][61552] Updated weights for policy 0, policy_version 64802 (0.0008) [2023-10-14 20:19:46,427][61585] Updated weights for policy 1, policy_version 64670 (0.0009) [2023-10-14 20:19:46,654][61552] Updated weights for policy 0, policy_version 64812 (0.0008) [2023-10-14 20:19:47,026][61552] Updated weights for policy 0, policy_version 64822 (0.0009) [2023-10-14 20:19:47,394][61552] Updated weights for policy 0, policy_version 64832 (0.0007) [2023-10-14 20:19:48,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 132612096. Throughput: 0: 1651.5, 1: 1667.2. Samples: 33154630. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:19:48,344][60425] Avg episode reward: [(0, '73.160'), (1, '76.960')] [2023-10-14 20:19:50,578][61585] Updated weights for policy 1, policy_version 64680 (0.0008) [2023-10-14 20:19:50,941][61585] Updated weights for policy 1, policy_version 64690 (0.0007) [2023-10-14 20:19:51,311][61585] Updated weights for policy 1, policy_version 64700 (0.0008) [2023-10-14 20:19:51,455][61552] Updated weights for policy 0, policy_version 64842 (0.0008) [2023-10-14 20:19:51,814][61552] Updated weights for policy 0, policy_version 64852 (0.0007) [2023-10-14 20:19:52,181][61552] Updated weights for policy 0, policy_version 64862 (0.0008) [2023-10-14 20:19:53,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 132677632. Throughput: 0: 1651.0, 1: 1658.4. Samples: 33173710. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:19:53,345][60425] Avg episode reward: [(0, '73.900'), (1, '75.760')] [2023-10-14 20:19:55,262][61585] Updated weights for policy 1, policy_version 64710 (0.0008) [2023-10-14 20:19:55,618][61585] Updated weights for policy 1, policy_version 64720 (0.0008) [2023-10-14 20:19:55,995][61585] Updated weights for policy 1, policy_version 64730 (0.0009) [2023-10-14 20:19:56,203][61552] Updated weights for policy 0, policy_version 64872 (0.0008) [2023-10-14 20:19:56,573][61552] Updated weights for policy 0, policy_version 64882 (0.0008) [2023-10-14 20:19:56,943][61552] Updated weights for policy 0, policy_version 64892 (0.0010) [2023-10-14 20:19:58,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 132743168. Throughput: 0: 1660.6, 1: 1682.9. Samples: 33193946. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:19:58,344][60425] Avg episode reward: [(0, '74.100'), (1, '73.170')] [2023-10-14 20:20:00,090][61585] Updated weights for policy 1, policy_version 64740 (0.0009) [2023-10-14 20:20:00,450][61585] Updated weights for policy 1, policy_version 64750 (0.0010) [2023-10-14 20:20:00,819][61585] Updated weights for policy 1, policy_version 64760 (0.0008) [2023-10-14 20:20:01,014][61552] Updated weights for policy 0, policy_version 64902 (0.0008) [2023-10-14 20:20:01,381][61552] Updated weights for policy 0, policy_version 64912 (0.0010) [2023-10-14 20:20:01,749][61552] Updated weights for policy 0, policy_version 64922 (0.0009) [2023-10-14 20:20:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 132808704. Throughput: 0: 1661.9, 1: 1662.3. Samples: 33204674. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:20:03,344][60425] Avg episode reward: [(0, '77.620'), (1, '78.050')] [2023-10-14 20:20:05,027][61585] Updated weights for policy 1, policy_version 64770 (0.0008) [2023-10-14 20:20:05,399][61585] Updated weights for policy 1, policy_version 64780 (0.0008) [2023-10-14 20:20:05,772][61585] Updated weights for policy 1, policy_version 64790 (0.0009) [2023-10-14 20:20:05,974][61552] Updated weights for policy 0, policy_version 64932 (0.0009) [2023-10-14 20:20:06,141][61585] Updated weights for policy 1, policy_version 64800 (0.0009) [2023-10-14 20:20:06,368][61552] Updated weights for policy 0, policy_version 64942 (0.0010) [2023-10-14 20:20:06,740][61552] Updated weights for policy 0, policy_version 64952 (0.0010) [2023-10-14 20:20:08,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 132874240. Throughput: 0: 1646.8, 1: 1669.3. Samples: 33223682. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:20:08,345][60425] Avg episode reward: [(0, '76.890'), (1, '74.410')] [2023-10-14 20:20:10,406][61585] Updated weights for policy 1, policy_version 64810 (0.0009) [2023-10-14 20:20:10,713][61552] Updated weights for policy 0, policy_version 64962 (0.0007) [2023-10-14 20:20:10,773][61585] Updated weights for policy 1, policy_version 64820 (0.0008) [2023-10-14 20:20:11,074][61552] Updated weights for policy 0, policy_version 64972 (0.0008) [2023-10-14 20:20:11,136][61585] Updated weights for policy 1, policy_version 64830 (0.0009) [2023-10-14 20:20:11,445][61552] Updated weights for policy 0, policy_version 64982 (0.0007) [2023-10-14 20:20:11,805][61552] Updated weights for policy 0, policy_version 64992 (0.0007) [2023-10-14 20:20:13,344][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 132939776. Throughput: 0: 1669.8, 1: 1676.3. Samples: 33243948. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:20:13,345][60425] Avg episode reward: [(0, '71.420'), (1, '73.230')] [2023-10-14 20:20:15,024][61585] Updated weights for policy 1, policy_version 64840 (0.0007) [2023-10-14 20:20:15,395][61585] Updated weights for policy 1, policy_version 64850 (0.0008) [2023-10-14 20:20:15,768][61585] Updated weights for policy 1, policy_version 64860 (0.0008) [2023-10-14 20:20:15,881][61552] Updated weights for policy 0, policy_version 65002 (0.0008) [2023-10-14 20:20:16,243][61552] Updated weights for policy 0, policy_version 65012 (0.0007) [2023-10-14 20:20:16,622][61552] Updated weights for policy 0, policy_version 65022 (0.0008) [2023-10-14 20:20:18,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 133005312. Throughput: 0: 1666.7, 1: 1657.4. Samples: 33254356. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:20:18,344][60425] Avg episode reward: [(0, '73.890'), (1, '72.010')] [2023-10-14 20:20:19,849][61585] Updated weights for policy 1, policy_version 64870 (0.0009) [2023-10-14 20:20:20,208][61585] Updated weights for policy 1, policy_version 64880 (0.0010) [2023-10-14 20:20:20,577][61585] Updated weights for policy 1, policy_version 64890 (0.0009) [2023-10-14 20:20:20,765][61552] Updated weights for policy 0, policy_version 65032 (0.0008) [2023-10-14 20:20:21,132][61552] Updated weights for policy 0, policy_version 65042 (0.0010) [2023-10-14 20:20:21,497][61552] Updated weights for policy 0, policy_version 65052 (0.0009) [2023-10-14 20:20:23,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 133070848. Throughput: 0: 1646.7, 1: 1679.7. Samples: 33273580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:20:23,344][60425] Avg episode reward: [(0, '73.790'), (1, '73.250')] [2023-10-14 20:20:24,659][61585] Updated weights for policy 1, policy_version 64900 (0.0009) [2023-10-14 20:20:25,023][61585] Updated weights for policy 1, policy_version 64910 (0.0010) [2023-10-14 20:20:25,391][61585] Updated weights for policy 1, policy_version 64920 (0.0008) [2023-10-14 20:20:25,554][61552] Updated weights for policy 0, policy_version 65062 (0.0009) [2023-10-14 20:20:25,927][61552] Updated weights for policy 0, policy_version 65072 (0.0009) [2023-10-14 20:20:26,280][61552] Updated weights for policy 0, policy_version 65082 (0.0010) [2023-10-14 20:20:28,343][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 133136384. Throughput: 0: 1669.1, 1: 1677.5. Samples: 33294196. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:20:28,345][60425] Avg episode reward: [(0, '72.930'), (1, '74.020')] [2023-10-14 20:20:29,642][61585] Updated weights for policy 1, policy_version 64930 (0.0008) [2023-10-14 20:20:30,006][61585] Updated weights for policy 1, policy_version 64940 (0.0010) [2023-10-14 20:20:30,364][61552] Updated weights for policy 0, policy_version 65092 (0.0007) [2023-10-14 20:20:30,367][61585] Updated weights for policy 1, policy_version 64950 (0.0008) [2023-10-14 20:20:30,726][61585] Updated weights for policy 1, policy_version 64960 (0.0009) [2023-10-14 20:20:30,730][61552] Updated weights for policy 0, policy_version 65102 (0.0008) [2023-10-14 20:20:31,095][61552] Updated weights for policy 0, policy_version 65112 (0.0007) [2023-10-14 20:20:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 133201920. Throughput: 0: 1660.5, 1: 1663.8. Samples: 33304224. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:20:33,344][60425] Avg episode reward: [(0, '73.460'), (1, '74.990')] [2023-10-14 20:20:34,677][61585] Updated weights for policy 1, policy_version 64970 (0.0007) [2023-10-14 20:20:35,041][61585] Updated weights for policy 1, policy_version 64980 (0.0009) [2023-10-14 20:20:35,165][61552] Updated weights for policy 0, policy_version 65122 (0.0007) [2023-10-14 20:20:35,394][61585] Updated weights for policy 1, policy_version 64990 (0.0008) [2023-10-14 20:20:35,532][61552] Updated weights for policy 0, policy_version 65132 (0.0009) [2023-10-14 20:20:35,903][61552] Updated weights for policy 0, policy_version 65142 (0.0007) [2023-10-14 20:20:36,268][61552] Updated weights for policy 0, policy_version 65152 (0.0011) [2023-10-14 20:20:38,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 133267456. Throughput: 0: 1655.1, 1: 1689.2. Samples: 33324202. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:20:38,344][60425] Avg episode reward: [(0, '75.670'), (1, '74.980')] [2023-10-14 20:20:39,476][61585] Updated weights for policy 1, policy_version 65000 (0.0007) [2023-10-14 20:20:39,843][61585] Updated weights for policy 1, policy_version 65010 (0.0008) [2023-10-14 20:20:40,204][61585] Updated weights for policy 1, policy_version 65020 (0.0007) [2023-10-14 20:20:40,374][61552] Updated weights for policy 0, policy_version 65162 (0.0010) [2023-10-14 20:20:40,747][61552] Updated weights for policy 0, policy_version 65172 (0.0008) [2023-10-14 20:20:41,116][61552] Updated weights for policy 0, policy_version 65182 (0.0009) [2023-10-14 20:20:43,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 133332992. Throughput: 0: 1669.6, 1: 1683.6. Samples: 33344840. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:20:43,345][60425] Avg episode reward: [(0, '75.650'), (1, '70.620')] [2023-10-14 20:20:44,200][61585] Updated weights for policy 1, policy_version 65030 (0.0009) [2023-10-14 20:20:44,556][61585] Updated weights for policy 1, policy_version 65040 (0.0010) [2023-10-14 20:20:44,926][61585] Updated weights for policy 1, policy_version 65050 (0.0011) [2023-10-14 20:20:45,256][61552] Updated weights for policy 0, policy_version 65192 (0.0008) [2023-10-14 20:20:45,631][61552] Updated weights for policy 0, policy_version 65202 (0.0010) [2023-10-14 20:20:46,002][61552] Updated weights for policy 0, policy_version 65212 (0.0008) [2023-10-14 20:20:48,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 133398528. Throughput: 0: 1653.6, 1: 1675.2. Samples: 33354470. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:20:48,344][60425] Avg episode reward: [(0, '73.620'), (1, '76.720')] [2023-10-14 20:20:49,107][61585] Updated weights for policy 1, policy_version 65060 (0.0008) [2023-10-14 20:20:49,485][61585] Updated weights for policy 1, policy_version 65070 (0.0009) [2023-10-14 20:20:49,849][61585] Updated weights for policy 1, policy_version 65080 (0.0009) [2023-10-14 20:20:50,124][61552] Updated weights for policy 0, policy_version 65222 (0.0009) [2023-10-14 20:20:50,492][61552] Updated weights for policy 0, policy_version 65232 (0.0009) [2023-10-14 20:20:50,865][61552] Updated weights for policy 0, policy_version 65242 (0.0009) [2023-10-14 20:20:53,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 133464064. Throughput: 0: 1668.7, 1: 1682.3. Samples: 33374478. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:20:53,344][60425] Avg episode reward: [(0, '76.250'), (1, '77.770')] [2023-10-14 20:20:54,012][61585] Updated weights for policy 1, policy_version 65090 (0.0008) [2023-10-14 20:20:54,392][61585] Updated weights for policy 1, policy_version 65100 (0.0010) [2023-10-14 20:20:54,748][61585] Updated weights for policy 1, policy_version 65110 (0.0010) [2023-10-14 20:20:55,100][61552] Updated weights for policy 0, policy_version 65252 (0.0008) [2023-10-14 20:20:55,110][61585] Updated weights for policy 1, policy_version 65120 (0.0008) [2023-10-14 20:20:55,467][61552] Updated weights for policy 0, policy_version 65262 (0.0009) [2023-10-14 20:20:55,838][61552] Updated weights for policy 0, policy_version 65272 (0.0008) [2023-10-14 20:20:58,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 133529600. Throughput: 0: 1673.7, 1: 1684.9. Samples: 33395082. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:20:58,344][60425] Avg episode reward: [(0, '77.580'), (1, '74.170')] [2023-10-14 20:20:59,217][61585] Updated weights for policy 1, policy_version 65130 (0.0009) [2023-10-14 20:20:59,588][61585] Updated weights for policy 1, policy_version 65140 (0.0009) [2023-10-14 20:20:59,910][61552] Updated weights for policy 0, policy_version 65282 (0.0007) [2023-10-14 20:20:59,959][61585] Updated weights for policy 1, policy_version 65150 (0.0009) [2023-10-14 20:21:00,282][61552] Updated weights for policy 0, policy_version 65292 (0.0008) [2023-10-14 20:21:00,652][61552] Updated weights for policy 0, policy_version 65302 (0.0009) [2023-10-14 20:21:01,010][61552] Updated weights for policy 0, policy_version 65312 (0.0009) [2023-10-14 20:21:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 133595136. Throughput: 0: 1659.4, 1: 1677.4. Samples: 33404512. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:21:03,344][60425] Avg episode reward: [(0, '74.100'), (1, '73.950')] [2023-10-14 20:21:03,953][61585] Updated weights for policy 1, policy_version 65160 (0.0009) [2023-10-14 20:21:04,314][61585] Updated weights for policy 1, policy_version 65170 (0.0008) [2023-10-14 20:21:04,679][61585] Updated weights for policy 1, policy_version 65180 (0.0008) [2023-10-14 20:21:05,056][61552] Updated weights for policy 0, policy_version 65322 (0.0008) [2023-10-14 20:21:05,430][61552] Updated weights for policy 0, policy_version 65332 (0.0010) [2023-10-14 20:21:05,796][61552] Updated weights for policy 0, policy_version 65342 (0.0010) [2023-10-14 20:21:08,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 133660672. Throughput: 0: 1676.4, 1: 1683.7. Samples: 33424782. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:21:08,344][60425] Avg episode reward: [(0, '77.990'), (1, '76.300')] [2023-10-14 20:21:08,663][61585] Updated weights for policy 1, policy_version 65190 (0.0008) [2023-10-14 20:21:09,024][61585] Updated weights for policy 1, policy_version 65200 (0.0007) [2023-10-14 20:21:09,392][61585] Updated weights for policy 1, policy_version 65210 (0.0008) [2023-10-14 20:21:09,862][61552] Updated weights for policy 0, policy_version 65352 (0.0011) [2023-10-14 20:21:10,223][61552] Updated weights for policy 0, policy_version 65362 (0.0008) [2023-10-14 20:21:10,600][61552] Updated weights for policy 0, policy_version 65372 (0.0009) [2023-10-14 20:21:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 133726208. Throughput: 0: 1678.9, 1: 1685.6. Samples: 33445600. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:21:13,344][60425] Avg episode reward: [(0, '77.740'), (1, '77.590')] [2023-10-14 20:21:13,368][61585] Updated weights for policy 1, policy_version 65220 (0.0009) [2023-10-14 20:21:13,724][61585] Updated weights for policy 1, policy_version 65230 (0.0007) [2023-10-14 20:21:14,104][61585] Updated weights for policy 1, policy_version 65240 (0.0008) [2023-10-14 20:21:14,649][61552] Updated weights for policy 0, policy_version 65382 (0.0009) [2023-10-14 20:21:15,012][61552] Updated weights for policy 0, policy_version 65392 (0.0007) [2023-10-14 20:21:15,379][61552] Updated weights for policy 0, policy_version 65402 (0.0010) [2023-10-14 20:21:18,183][61585] Updated weights for policy 1, policy_version 65250 (0.0007) [2023-10-14 20:21:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 133791744. Throughput: 0: 1660.4, 1: 1685.2. Samples: 33454780. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:21:18,344][60425] Avg episode reward: [(0, '73.900'), (1, '76.540')] [2023-10-14 20:21:18,540][61585] Updated weights for policy 1, policy_version 65260 (0.0008) [2023-10-14 20:21:18,909][61585] Updated weights for policy 1, policy_version 65270 (0.0011) [2023-10-14 20:21:19,270][61585] Updated weights for policy 1, policy_version 65280 (0.0009) [2023-10-14 20:21:19,622][61552] Updated weights for policy 0, policy_version 65412 (0.0009) [2023-10-14 20:21:19,993][61552] Updated weights for policy 0, policy_version 65422 (0.0008) [2023-10-14 20:21:20,357][61552] Updated weights for policy 0, policy_version 65432 (0.0007) [2023-10-14 20:21:23,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 133857280. Throughput: 0: 1675.2, 1: 1681.9. Samples: 33475270. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:21:23,344][60425] Avg episode reward: [(0, '74.350'), (1, '74.670')] [2023-10-14 20:21:23,556][61585] Updated weights for policy 1, policy_version 65290 (0.0008) [2023-10-14 20:21:23,920][61585] Updated weights for policy 1, policy_version 65300 (0.0009) [2023-10-14 20:21:24,291][61585] Updated weights for policy 1, policy_version 65310 (0.0011) [2023-10-14 20:21:24,421][61552] Updated weights for policy 0, policy_version 65442 (0.0007) [2023-10-14 20:21:24,794][61552] Updated weights for policy 0, policy_version 65452 (0.0009) [2023-10-14 20:21:25,162][61552] Updated weights for policy 0, policy_version 65462 (0.0009) [2023-10-14 20:21:25,524][61552] Updated weights for policy 0, policy_version 65472 (0.0009) [2023-10-14 20:21:28,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 133922816. Throughput: 0: 1677.2, 1: 1679.8. Samples: 33495904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:21:28,344][60425] Avg episode reward: [(0, '75.830'), (1, '71.420')] [2023-10-14 20:21:28,453][61585] Updated weights for policy 1, policy_version 65320 (0.0009) [2023-10-14 20:21:28,815][61585] Updated weights for policy 1, policy_version 65330 (0.0009) [2023-10-14 20:21:29,184][61585] Updated weights for policy 1, policy_version 65340 (0.0008) [2023-10-14 20:21:29,559][61552] Updated weights for policy 0, policy_version 65482 (0.0010) [2023-10-14 20:21:29,932][61552] Updated weights for policy 0, policy_version 65492 (0.0007) [2023-10-14 20:21:30,303][61552] Updated weights for policy 0, policy_version 65502 (0.0009) [2023-10-14 20:21:33,311][61585] Updated weights for policy 1, policy_version 65350 (0.0010) [2023-10-14 20:21:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 133988352. Throughput: 0: 1668.2, 1: 1678.4. Samples: 33505068. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:21:33,344][60425] Avg episode reward: [(0, '72.510'), (1, '72.910')] [2023-10-14 20:21:33,677][61585] Updated weights for policy 1, policy_version 65360 (0.0008) [2023-10-14 20:21:34,046][61585] Updated weights for policy 1, policy_version 65370 (0.0009) [2023-10-14 20:21:34,484][61552] Updated weights for policy 0, policy_version 65512 (0.0009) [2023-10-14 20:21:34,859][61552] Updated weights for policy 0, policy_version 65522 (0.0008) [2023-10-14 20:21:35,218][61552] Updated weights for policy 0, policy_version 65532 (0.0008) [2023-10-14 20:21:38,097][61585] Updated weights for policy 1, policy_version 65380 (0.0007) [2023-10-14 20:21:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 134053888. Throughput: 0: 1679.8, 1: 1679.5. Samples: 33525646. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:21:38,344][60425] Avg episode reward: [(0, '76.120'), (1, '74.520')] [2023-10-14 20:21:38,474][61585] Updated weights for policy 1, policy_version 65390 (0.0008) [2023-10-14 20:21:38,846][61585] Updated weights for policy 1, policy_version 65400 (0.0008) [2023-10-14 20:21:39,104][61552] Updated weights for policy 0, policy_version 65542 (0.0009) [2023-10-14 20:21:39,460][61552] Updated weights for policy 0, policy_version 65552 (0.0010) [2023-10-14 20:21:39,826][61552] Updated weights for policy 0, policy_version 65562 (0.0009) [2023-10-14 20:21:42,884][61585] Updated weights for policy 1, policy_version 65410 (0.0008) [2023-10-14 20:21:43,251][61585] Updated weights for policy 1, policy_version 65420 (0.0010) [2023-10-14 20:21:43,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 134119424. Throughput: 0: 1679.0, 1: 1681.0. Samples: 33546280. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:21:43,345][60425] Avg episode reward: [(0, '76.800'), (1, '77.130')] [2023-10-14 20:21:43,354][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000065568_67141632.pth... [2023-10-14 20:21:43,391][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000064032_65568768.pth [2023-10-14 20:21:43,624][61585] Updated weights for policy 1, policy_version 65430 (0.0010) [2023-10-14 20:21:43,976][61552] Updated weights for policy 0, policy_version 65572 (0.0009) [2023-10-14 20:21:43,989][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000065440_67010560.pth... [2023-10-14 20:21:43,990][61585] Updated weights for policy 1, policy_version 65440 (0.0009) [2023-10-14 20:21:44,018][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000063872_65404928.pth [2023-10-14 20:21:44,375][61552] Updated weights for policy 0, policy_version 65582 (0.0009) [2023-10-14 20:21:44,736][61552] Updated weights for policy 0, policy_version 65592 (0.0007) [2023-10-14 20:21:48,191][61585] Updated weights for policy 1, policy_version 65450 (0.0009) [2023-10-14 20:21:48,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 134184960. Throughput: 0: 1669.4, 1: 1680.6. Samples: 33555262. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:21:48,344][60425] Avg episode reward: [(0, '73.550'), (1, '76.690')] [2023-10-14 20:21:48,555][61585] Updated weights for policy 1, policy_version 65460 (0.0010) [2023-10-14 20:21:48,822][61552] Updated weights for policy 0, policy_version 65602 (0.0009) [2023-10-14 20:21:48,923][61585] Updated weights for policy 1, policy_version 65470 (0.0008) [2023-10-14 20:21:49,180][61552] Updated weights for policy 0, policy_version 65612 (0.0009) [2023-10-14 20:21:49,544][61552] Updated weights for policy 0, policy_version 65622 (0.0011) [2023-10-14 20:21:49,908][61552] Updated weights for policy 0, policy_version 65632 (0.0008) [2023-10-14 20:21:53,044][61585] Updated weights for policy 1, policy_version 65480 (0.0010) [2023-10-14 20:21:53,343][60425] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 134250496. Throughput: 0: 1676.5, 1: 1672.9. Samples: 33575504. Policy #0 lag: (min: 24.0, avg: 42.1, max: 56.0) [2023-10-14 20:21:53,344][60425] Avg episode reward: [(0, '80.050'), (1, '71.890')] [2023-10-14 20:21:53,408][61585] Updated weights for policy 1, policy_version 65490 (0.0011) [2023-10-14 20:21:53,766][61585] Updated weights for policy 1, policy_version 65500 (0.0009) [2023-10-14 20:21:53,871][61552] Updated weights for policy 0, policy_version 65642 (0.0009) [2023-10-14 20:21:54,249][61552] Updated weights for policy 0, policy_version 65652 (0.0008) [2023-10-14 20:21:54,616][61552] Updated weights for policy 0, policy_version 65662 (0.0007) [2023-10-14 20:21:57,806][61585] Updated weights for policy 1, policy_version 65510 (0.0009) [2023-10-14 20:21:58,171][61585] Updated weights for policy 1, policy_version 65520 (0.0011) [2023-10-14 20:21:58,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 134316032. Throughput: 0: 1681.5, 1: 1665.7. Samples: 33596226. Policy #0 lag: (min: 24.0, avg: 42.1, max: 56.0) [2023-10-14 20:21:58,345][60425] Avg episode reward: [(0, '80.670'), (1, '73.830')] [2023-10-14 20:21:58,541][61585] Updated weights for policy 1, policy_version 65530 (0.0008) [2023-10-14 20:21:58,671][61552] Updated weights for policy 0, policy_version 65672 (0.0007) [2023-10-14 20:21:59,043][61552] Updated weights for policy 0, policy_version 65682 (0.0010) [2023-10-14 20:21:59,409][61552] Updated weights for policy 0, policy_version 65692 (0.0011) [2023-10-14 20:22:02,740][61585] Updated weights for policy 1, policy_version 65540 (0.0008) [2023-10-14 20:22:03,114][61585] Updated weights for policy 1, policy_version 65550 (0.0007) [2023-10-14 20:22:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 134381568. Throughput: 0: 1682.8, 1: 1667.0. Samples: 33605518. Policy #0 lag: (min: 24.0, avg: 42.1, max: 56.0) [2023-10-14 20:22:03,344][60425] Avg episode reward: [(0, '81.650'), (1, '73.080')] [2023-10-14 20:22:03,487][61585] Updated weights for policy 1, policy_version 65560 (0.0007) [2023-10-14 20:22:03,583][61552] Updated weights for policy 0, policy_version 65702 (0.0009) [2023-10-14 20:22:03,940][61552] Updated weights for policy 0, policy_version 65712 (0.0008) [2023-10-14 20:22:04,306][61552] Updated weights for policy 0, policy_version 65722 (0.0007) [2023-10-14 20:22:07,648][61585] Updated weights for policy 1, policy_version 65570 (0.0008) [2023-10-14 20:22:08,017][61585] Updated weights for policy 1, policy_version 65580 (0.0007) [2023-10-14 20:22:08,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 134447104. Throughput: 0: 1682.7, 1: 1664.8. Samples: 33625906. Policy #0 lag: (min: 24.0, avg: 42.1, max: 56.0) [2023-10-14 20:22:08,344][60425] Avg episode reward: [(0, '76.250'), (1, '70.300')] [2023-10-14 20:22:08,401][61552] Updated weights for policy 0, policy_version 65732 (0.0007) [2023-10-14 20:22:08,401][61585] Updated weights for policy 1, policy_version 65590 (0.0009) [2023-10-14 20:22:08,759][61552] Updated weights for policy 0, policy_version 65742 (0.0007) [2023-10-14 20:22:08,767][61585] Updated weights for policy 1, policy_version 65600 (0.0008) [2023-10-14 20:22:09,130][61552] Updated weights for policy 0, policy_version 65752 (0.0008) [2023-10-14 20:22:12,823][61585] Updated weights for policy 1, policy_version 65610 (0.0008) [2023-10-14 20:22:13,193][61585] Updated weights for policy 1, policy_version 65620 (0.0010) [2023-10-14 20:22:13,231][61552] Updated weights for policy 0, policy_version 65762 (0.0008) [2023-10-14 20:22:13,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 134512640. Throughput: 0: 1680.0, 1: 1658.5. Samples: 33646136. Policy #0 lag: (min: 24.0, avg: 42.1, max: 56.0) [2023-10-14 20:22:13,344][60425] Avg episode reward: [(0, '83.290'), (1, '70.530')] [2023-10-14 20:22:13,562][61585] Updated weights for policy 1, policy_version 65630 (0.0009) [2023-10-14 20:22:13,596][61552] Updated weights for policy 0, policy_version 65772 (0.0009) [2023-10-14 20:22:13,961][61552] Updated weights for policy 0, policy_version 65782 (0.0010) [2023-10-14 20:22:14,328][61172] Saving new best policy, reward=83.290! [2023-10-14 20:22:14,333][61552] Updated weights for policy 0, policy_version 65792 (0.0009) [2023-10-14 20:22:17,740][61585] Updated weights for policy 1, policy_version 65640 (0.0010) [2023-10-14 20:22:18,101][61585] Updated weights for policy 1, policy_version 65650 (0.0009) [2023-10-14 20:22:18,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 134578176. Throughput: 0: 1676.2, 1: 1667.8. Samples: 33655548. Policy #0 lag: (min: 24.0, avg: 42.1, max: 56.0) [2023-10-14 20:22:18,344][60425] Avg episode reward: [(0, '79.560'), (1, '70.540')] [2023-10-14 20:22:18,468][61552] Updated weights for policy 0, policy_version 65802 (0.0009) [2023-10-14 20:22:18,471][61585] Updated weights for policy 1, policy_version 65660 (0.0008) [2023-10-14 20:22:18,833][61552] Updated weights for policy 0, policy_version 65812 (0.0007) [2023-10-14 20:22:19,211][61552] Updated weights for policy 0, policy_version 65822 (0.0008) [2023-10-14 20:22:22,536][61585] Updated weights for policy 1, policy_version 65670 (0.0007) [2023-10-14 20:22:22,909][61585] Updated weights for policy 1, policy_version 65680 (0.0009) [2023-10-14 20:22:23,084][61552] Updated weights for policy 0, policy_version 65832 (0.0009) [2023-10-14 20:22:23,266][61585] Updated weights for policy 1, policy_version 65690 (0.0008) [2023-10-14 20:22:23,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 134643712. Throughput: 0: 1679.5, 1: 1668.3. Samples: 33676296. Policy #0 lag: (min: 24.0, avg: 42.1, max: 56.0) [2023-10-14 20:22:23,344][60425] Avg episode reward: [(0, '79.910'), (1, '75.120')] [2023-10-14 20:22:23,448][61552] Updated weights for policy 0, policy_version 65842 (0.0009) [2023-10-14 20:22:23,819][61552] Updated weights for policy 0, policy_version 65852 (0.0009) [2023-10-14 20:22:27,391][61585] Updated weights for policy 1, policy_version 65700 (0.0009) [2023-10-14 20:22:27,761][61585] Updated weights for policy 1, policy_version 65710 (0.0011) [2023-10-14 20:22:27,926][61552] Updated weights for policy 0, policy_version 65862 (0.0010) [2023-10-14 20:22:28,124][61585] Updated weights for policy 1, policy_version 65720 (0.0009) [2023-10-14 20:22:28,291][61552] Updated weights for policy 0, policy_version 65872 (0.0007) [2023-10-14 20:22:28,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 134709248. Throughput: 0: 1683.1, 1: 1655.1. Samples: 33696498. Policy #0 lag: (min: 24.0, avg: 42.1, max: 56.0) [2023-10-14 20:22:28,344][60425] Avg episode reward: [(0, '77.960'), (1, '74.090')] [2023-10-14 20:22:28,652][61552] Updated weights for policy 0, policy_version 65882 (0.0010) [2023-10-14 20:22:32,226][61585] Updated weights for policy 1, policy_version 65730 (0.0010) [2023-10-14 20:22:32,549][61552] Updated weights for policy 0, policy_version 65892 (0.0009) [2023-10-14 20:22:32,599][61585] Updated weights for policy 1, policy_version 65740 (0.0008) [2023-10-14 20:22:32,923][61552] Updated weights for policy 0, policy_version 65902 (0.0007) [2023-10-14 20:22:32,963][61585] Updated weights for policy 1, policy_version 65750 (0.0007) [2023-10-14 20:22:33,283][61552] Updated weights for policy 0, policy_version 65912 (0.0008) [2023-10-14 20:22:33,326][61585] Updated weights for policy 1, policy_version 65760 (0.0008) [2023-10-14 20:22:33,343][60425] Fps is (10 sec: 16384.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 134807552. Throughput: 0: 1687.6, 1: 1667.3. Samples: 33706234. Policy #0 lag: (min: 24.0, avg: 42.1, max: 56.0) [2023-10-14 20:22:33,344][60425] Avg episode reward: [(0, '79.480'), (1, '77.240')] [2023-10-14 20:22:37,443][61552] Updated weights for policy 0, policy_version 65922 (0.0008) [2023-10-14 20:22:37,607][61585] Updated weights for policy 1, policy_version 65770 (0.0010) [2023-10-14 20:22:37,809][61552] Updated weights for policy 0, policy_version 65932 (0.0008) [2023-10-14 20:22:37,965][61585] Updated weights for policy 1, policy_version 65780 (0.0010) [2023-10-14 20:22:38,186][61552] Updated weights for policy 0, policy_version 65942 (0.0007) [2023-10-14 20:22:38,338][61585] Updated weights for policy 1, policy_version 65790 (0.0009) [2023-10-14 20:22:38,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 134840320. Throughput: 0: 1693.2, 1: 1668.8. Samples: 33726798. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-14 20:22:38,344][60425] Avg episode reward: [(0, '80.180'), (1, '72.220')] [2023-10-14 20:22:38,550][61552] Updated weights for policy 0, policy_version 65952 (0.0007) [2023-10-14 20:22:42,315][61585] Updated weights for policy 1, policy_version 65800 (0.0007) [2023-10-14 20:22:42,670][61585] Updated weights for policy 1, policy_version 65810 (0.0009) [2023-10-14 20:22:42,708][61552] Updated weights for policy 0, policy_version 65962 (0.0007) [2023-10-14 20:22:43,033][61585] Updated weights for policy 1, policy_version 65820 (0.0009) [2023-10-14 20:22:43,084][61552] Updated weights for policy 0, policy_version 65972 (0.0009) [2023-10-14 20:22:43,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 134938624. Throughput: 0: 1675.2, 1: 1657.1. Samples: 33746178. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-14 20:22:43,344][60425] Avg episode reward: [(0, '79.340'), (1, '74.440')] [2023-10-14 20:22:43,447][61552] Updated weights for policy 0, policy_version 65982 (0.0009) [2023-10-14 20:22:47,069][61585] Updated weights for policy 1, policy_version 65830 (0.0008) [2023-10-14 20:22:47,434][61585] Updated weights for policy 1, policy_version 65840 (0.0007) [2023-10-14 20:22:47,579][61552] Updated weights for policy 0, policy_version 65992 (0.0008) [2023-10-14 20:22:47,803][61585] Updated weights for policy 1, policy_version 65850 (0.0009) [2023-10-14 20:22:47,946][61552] Updated weights for policy 0, policy_version 66002 (0.0007) [2023-10-14 20:22:48,319][61552] Updated weights for policy 0, policy_version 66012 (0.0009) [2023-10-14 20:22:48,343][60425] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 135004160. Throughput: 0: 1681.5, 1: 1672.3. Samples: 33756438. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-14 20:22:48,344][60425] Avg episode reward: [(0, '78.640'), (1, '75.780')] [2023-10-14 20:22:51,935][61585] Updated weights for policy 1, policy_version 65860 (0.0008) [2023-10-14 20:22:52,305][61585] Updated weights for policy 1, policy_version 65870 (0.0008) [2023-10-14 20:22:52,399][61552] Updated weights for policy 0, policy_version 66022 (0.0007) [2023-10-14 20:22:52,667][61585] Updated weights for policy 1, policy_version 65880 (0.0009) [2023-10-14 20:22:52,770][61552] Updated weights for policy 0, policy_version 66032 (0.0008) [2023-10-14 20:22:53,136][61552] Updated weights for policy 0, policy_version 66042 (0.0007) [2023-10-14 20:22:53,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 135069696. Throughput: 0: 1686.3, 1: 1671.7. Samples: 33777016. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-14 20:22:53,344][60425] Avg episode reward: [(0, '77.850'), (1, '75.500')] [2023-10-14 20:22:56,806][61585] Updated weights for policy 1, policy_version 65890 (0.0009) [2023-10-14 20:22:57,167][61585] Updated weights for policy 1, policy_version 65900 (0.0009) [2023-10-14 20:22:57,347][61552] Updated weights for policy 0, policy_version 66052 (0.0009) [2023-10-14 20:22:57,521][61585] Updated weights for policy 1, policy_version 65910 (0.0009) [2023-10-14 20:22:57,719][61552] Updated weights for policy 0, policy_version 66062 (0.0008) [2023-10-14 20:22:57,890][61585] Updated weights for policy 1, policy_version 65920 (0.0009) [2023-10-14 20:22:58,079][61552] Updated weights for policy 0, policy_version 66072 (0.0009) [2023-10-14 20:22:58,344][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 135135232. Throughput: 0: 1670.9, 1: 1654.2. Samples: 33795766. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-14 20:22:58,345][60425] Avg episode reward: [(0, '76.550'), (1, '74.610')] [2023-10-14 20:23:02,067][61585] Updated weights for policy 1, policy_version 65930 (0.0009) [2023-10-14 20:23:02,132][61552] Updated weights for policy 0, policy_version 66082 (0.0009) [2023-10-14 20:23:02,438][61585] Updated weights for policy 1, policy_version 65940 (0.0008) [2023-10-14 20:23:02,497][61552] Updated weights for policy 0, policy_version 66092 (0.0009) [2023-10-14 20:23:02,796][61585] Updated weights for policy 1, policy_version 65950 (0.0008) [2023-10-14 20:23:02,879][61552] Updated weights for policy 0, policy_version 66102 (0.0008) [2023-10-14 20:23:03,248][61552] Updated weights for policy 0, policy_version 66112 (0.0008) [2023-10-14 20:23:03,343][60425] Fps is (10 sec: 16383.9, 60 sec: 14199.4, 300 sec: 13329.4). Total num frames: 135233536. Throughput: 0: 1685.0, 1: 1665.9. Samples: 33806336. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-14 20:23:03,344][60425] Avg episode reward: [(0, '77.950'), (1, '73.740')] [2023-10-14 20:23:06,945][61585] Updated weights for policy 1, policy_version 65960 (0.0009) [2023-10-14 20:23:07,305][61585] Updated weights for policy 1, policy_version 65970 (0.0009) [2023-10-14 20:23:07,339][61552] Updated weights for policy 0, policy_version 66122 (0.0009) [2023-10-14 20:23:07,677][61585] Updated weights for policy 1, policy_version 65980 (0.0009) [2023-10-14 20:23:07,708][61552] Updated weights for policy 0, policy_version 66132 (0.0009) [2023-10-14 20:23:08,079][61552] Updated weights for policy 0, policy_version 66142 (0.0008) [2023-10-14 20:23:08,343][60425] Fps is (10 sec: 16384.5, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 135299072. Throughput: 0: 1680.8, 1: 1660.2. Samples: 33826638. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-14 20:23:08,344][60425] Avg episode reward: [(0, '79.660'), (1, '77.090')] [2023-10-14 20:23:11,642][61585] Updated weights for policy 1, policy_version 65990 (0.0009) [2023-10-14 20:23:12,009][61585] Updated weights for policy 1, policy_version 66000 (0.0009) [2023-10-14 20:23:12,179][61552] Updated weights for policy 0, policy_version 66152 (0.0007) [2023-10-14 20:23:12,374][61585] Updated weights for policy 1, policy_version 66010 (0.0008) [2023-10-14 20:23:12,540][61552] Updated weights for policy 0, policy_version 66162 (0.0007) [2023-10-14 20:23:12,908][61552] Updated weights for policy 0, policy_version 66172 (0.0007) [2023-10-14 20:23:13,343][60425] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13329.4). Total num frames: 135364608. Throughput: 0: 1659.9, 1: 1646.3. Samples: 33845278. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-14 20:23:13,345][60425] Avg episode reward: [(0, '76.900'), (1, '76.820')] [2023-10-14 20:23:16,528][61585] Updated weights for policy 1, policy_version 66020 (0.0009) [2023-10-14 20:23:16,895][61585] Updated weights for policy 1, policy_version 66030 (0.0010) [2023-10-14 20:23:16,961][61552] Updated weights for policy 0, policy_version 66182 (0.0009) [2023-10-14 20:23:17,251][61585] Updated weights for policy 1, policy_version 66040 (0.0007) [2023-10-14 20:23:17,322][61552] Updated weights for policy 0, policy_version 66192 (0.0008) [2023-10-14 20:23:17,680][61552] Updated weights for policy 0, policy_version 66202 (0.0009) [2023-10-14 20:23:18,343][60425] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 13329.3). Total num frames: 135430144. Throughput: 0: 1673.5, 1: 1662.8. Samples: 33856372. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-14 20:23:18,344][60425] Avg episode reward: [(0, '83.790'), (1, '74.370')] [2023-10-14 20:23:18,345][61172] Saving new best policy, reward=83.790! [2023-10-14 20:23:21,415][61585] Updated weights for policy 1, policy_version 66050 (0.0008) [2023-10-14 20:23:21,845][61585] Updated weights for policy 1, policy_version 66060 (0.0009) [2023-10-14 20:23:21,960][61552] Updated weights for policy 0, policy_version 66212 (0.0010) [2023-10-14 20:23:22,217][61585] Updated weights for policy 1, policy_version 66070 (0.0008) [2023-10-14 20:23:22,345][61552] Updated weights for policy 0, policy_version 66222 (0.0007) [2023-10-14 20:23:22,588][61585] Updated weights for policy 1, policy_version 66080 (0.0007) [2023-10-14 20:23:22,710][61552] Updated weights for policy 0, policy_version 66232 (0.0009) [2023-10-14 20:23:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 135495680. Throughput: 0: 1665.6, 1: 1658.6. Samples: 33876386. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-14 20:23:23,344][60425] Avg episode reward: [(0, '74.260'), (1, '75.840')] [2023-10-14 20:23:26,692][61585] Updated weights for policy 1, policy_version 66090 (0.0008) [2023-10-14 20:23:26,782][61552] Updated weights for policy 0, policy_version 66242 (0.0011) [2023-10-14 20:23:27,067][61585] Updated weights for policy 1, policy_version 66100 (0.0008) [2023-10-14 20:23:27,151][61552] Updated weights for policy 0, policy_version 66252 (0.0009) [2023-10-14 20:23:27,437][61585] Updated weights for policy 1, policy_version 66110 (0.0009) [2023-10-14 20:23:27,515][61552] Updated weights for policy 0, policy_version 66262 (0.0007) [2023-10-14 20:23:27,884][61552] Updated weights for policy 0, policy_version 66272 (0.0009) [2023-10-14 20:23:28,343][60425] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 13329.3). Total num frames: 135561216. Throughput: 0: 1652.4, 1: 1653.2. Samples: 33894932. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-14 20:23:28,345][60425] Avg episode reward: [(0, '79.590'), (1, '75.080')] [2023-10-14 20:23:31,647][61585] Updated weights for policy 1, policy_version 66120 (0.0007) [2023-10-14 20:23:31,946][61552] Updated weights for policy 0, policy_version 66282 (0.0007) [2023-10-14 20:23:32,012][61585] Updated weights for policy 1, policy_version 66130 (0.0009) [2023-10-14 20:23:32,309][61552] Updated weights for policy 0, policy_version 66292 (0.0010) [2023-10-14 20:23:32,381][61585] Updated weights for policy 1, policy_version 66140 (0.0010) [2023-10-14 20:23:32,680][61552] Updated weights for policy 0, policy_version 66302 (0.0009) [2023-10-14 20:23:33,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 135626752. Throughput: 0: 1670.3, 1: 1660.4. Samples: 33906320. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-14 20:23:33,344][60425] Avg episode reward: [(0, '75.360'), (1, '76.760')] [2023-10-14 20:23:36,484][61585] Updated weights for policy 1, policy_version 66150 (0.0009) [2023-10-14 20:23:36,603][61552] Updated weights for policy 0, policy_version 66312 (0.0008) [2023-10-14 20:23:36,842][61585] Updated weights for policy 1, policy_version 66160 (0.0007) [2023-10-14 20:23:36,973][61552] Updated weights for policy 0, policy_version 66322 (0.0008) [2023-10-14 20:23:37,210][61585] Updated weights for policy 1, policy_version 66170 (0.0008) [2023-10-14 20:23:37,335][61552] Updated weights for policy 0, policy_version 66332 (0.0010) [2023-10-14 20:23:38,343][60425] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 135692288. Throughput: 0: 1661.8, 1: 1647.2. Samples: 33925920. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-14 20:23:38,344][60425] Avg episode reward: [(0, '74.590'), (1, '75.470')] [2023-10-14 20:23:41,172][61585] Updated weights for policy 1, policy_version 66180 (0.0008) [2023-10-14 20:23:41,450][61552] Updated weights for policy 0, policy_version 66342 (0.0010) [2023-10-14 20:23:41,536][61585] Updated weights for policy 1, policy_version 66190 (0.0009) [2023-10-14 20:23:41,821][61552] Updated weights for policy 0, policy_version 66352 (0.0007) [2023-10-14 20:23:41,900][61585] Updated weights for policy 1, policy_version 66200 (0.0008) [2023-10-14 20:23:42,191][61552] Updated weights for policy 0, policy_version 66362 (0.0009) [2023-10-14 20:23:43,344][60425] Fps is (10 sec: 13106.6, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 135757824. Throughput: 0: 1657.7, 1: 1658.6. Samples: 33944998. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-14 20:23:43,345][60425] Avg episode reward: [(0, '79.570'), (1, '71.850')] [2023-10-14 20:23:43,354][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000066368_67960832.pth... [2023-10-14 20:23:43,355][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000066208_67796992.pth... [2023-10-14 20:23:43,391][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000064640_66191360.pth [2023-10-14 20:23:43,395][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000064800_66355200.pth [2023-10-14 20:23:45,892][61585] Updated weights for policy 1, policy_version 66210 (0.0008) [2023-10-14 20:23:46,151][61552] Updated weights for policy 0, policy_version 66372 (0.0009) [2023-10-14 20:23:46,248][61585] Updated weights for policy 1, policy_version 66220 (0.0007) [2023-10-14 20:23:46,528][61552] Updated weights for policy 0, policy_version 66382 (0.0009) [2023-10-14 20:23:46,619][61585] Updated weights for policy 1, policy_version 66230 (0.0008) [2023-10-14 20:23:46,912][61552] Updated weights for policy 0, policy_version 66392 (0.0008) [2023-10-14 20:23:46,981][61585] Updated weights for policy 1, policy_version 66240 (0.0009) [2023-10-14 20:23:48,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 135823360. Throughput: 0: 1674.7, 1: 1669.2. Samples: 33956814. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-14 20:23:48,344][60425] Avg episode reward: [(0, '79.390'), (1, '70.780')] [2023-10-14 20:23:50,978][61552] Updated weights for policy 0, policy_version 66402 (0.0009) [2023-10-14 20:23:51,073][61585] Updated weights for policy 1, policy_version 66250 (0.0008) [2023-10-14 20:23:51,336][61552] Updated weights for policy 0, policy_version 66412 (0.0010) [2023-10-14 20:23:51,436][61585] Updated weights for policy 1, policy_version 66260 (0.0010) [2023-10-14 20:23:51,703][61552] Updated weights for policy 0, policy_version 66422 (0.0010) [2023-10-14 20:23:51,795][61585] Updated weights for policy 1, policy_version 66270 (0.0007) [2023-10-14 20:23:52,075][61552] Updated weights for policy 0, policy_version 66432 (0.0008) [2023-10-14 20:23:53,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 135888896. Throughput: 0: 1656.4, 1: 1652.8. Samples: 33975554. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-14 20:23:53,345][60425] Avg episode reward: [(0, '80.290'), (1, '69.640')] [2023-10-14 20:23:55,867][61585] Updated weights for policy 1, policy_version 66280 (0.0008) [2023-10-14 20:23:56,120][61552] Updated weights for policy 0, policy_version 66442 (0.0008) [2023-10-14 20:23:56,232][61585] Updated weights for policy 1, policy_version 66290 (0.0007) [2023-10-14 20:23:56,478][61552] Updated weights for policy 0, policy_version 66452 (0.0010) [2023-10-14 20:23:56,589][61585] Updated weights for policy 1, policy_version 66300 (0.0008) [2023-10-14 20:23:56,844][61552] Updated weights for policy 0, policy_version 66462 (0.0009) [2023-10-14 20:23:58,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 135954432. Throughput: 0: 1665.6, 1: 1675.4. Samples: 33995620. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-14 20:23:58,344][60425] Avg episode reward: [(0, '76.920'), (1, '72.050')] [2023-10-14 20:24:00,692][61585] Updated weights for policy 1, policy_version 66310 (0.0007) [2023-10-14 20:24:01,000][61552] Updated weights for policy 0, policy_version 66472 (0.0008) [2023-10-14 20:24:01,053][61585] Updated weights for policy 1, policy_version 66320 (0.0009) [2023-10-14 20:24:01,376][61552] Updated weights for policy 0, policy_version 66482 (0.0010) [2023-10-14 20:24:01,411][61585] Updated weights for policy 1, policy_version 66330 (0.0008) [2023-10-14 20:24:01,738][61552] Updated weights for policy 0, policy_version 66492 (0.0009) [2023-10-14 20:24:03,344][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 136019968. Throughput: 0: 1676.9, 1: 1671.0. Samples: 34007026. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-14 20:24:03,345][60425] Avg episode reward: [(0, '76.630'), (1, '75.890')] [2023-10-14 20:24:05,502][61585] Updated weights for policy 1, policy_version 66340 (0.0009) [2023-10-14 20:24:05,865][61585] Updated weights for policy 1, policy_version 66350 (0.0007) [2023-10-14 20:24:05,968][61552] Updated weights for policy 0, policy_version 66502 (0.0008) [2023-10-14 20:24:06,235][61585] Updated weights for policy 1, policy_version 66360 (0.0008) [2023-10-14 20:24:06,343][61552] Updated weights for policy 0, policy_version 66512 (0.0008) [2023-10-14 20:24:06,715][61552] Updated weights for policy 0, policy_version 66522 (0.0007) [2023-10-14 20:24:08,343][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 136085504. Throughput: 0: 1655.9, 1: 1657.2. Samples: 34025476. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-14 20:24:08,344][60425] Avg episode reward: [(0, '78.370'), (1, '76.530')] [2023-10-14 20:24:10,483][61585] Updated weights for policy 1, policy_version 66370 (0.0008) [2023-10-14 20:24:10,798][61552] Updated weights for policy 0, policy_version 66532 (0.0007) [2023-10-14 20:24:10,908][61585] Updated weights for policy 1, policy_version 66380 (0.0007) [2023-10-14 20:24:11,184][61552] Updated weights for policy 0, policy_version 66542 (0.0009) [2023-10-14 20:24:11,268][61585] Updated weights for policy 1, policy_version 66390 (0.0008) [2023-10-14 20:24:11,562][61552] Updated weights for policy 0, policy_version 66552 (0.0009) [2023-10-14 20:24:11,636][61585] Updated weights for policy 1, policy_version 66400 (0.0008) [2023-10-14 20:24:13,344][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 136151040. Throughput: 0: 1670.2, 1: 1675.1. Samples: 34045472. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-14 20:24:13,345][60425] Avg episode reward: [(0, '83.110'), (1, '70.300')] [2023-10-14 20:24:15,588][61585] Updated weights for policy 1, policy_version 66410 (0.0007) [2023-10-14 20:24:15,790][61552] Updated weights for policy 0, policy_version 66562 (0.0007) [2023-10-14 20:24:15,957][61585] Updated weights for policy 1, policy_version 66420 (0.0010) [2023-10-14 20:24:16,171][61552] Updated weights for policy 0, policy_version 66572 (0.0008) [2023-10-14 20:24:16,319][61585] Updated weights for policy 1, policy_version 66430 (0.0008) [2023-10-14 20:24:16,541][61552] Updated weights for policy 0, policy_version 66582 (0.0009) [2023-10-14 20:24:16,895][61552] Updated weights for policy 0, policy_version 66592 (0.0007) [2023-10-14 20:24:18,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 136216576. Throughput: 0: 1669.0, 1: 1663.6. Samples: 34056288. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-14 20:24:18,344][60425] Avg episode reward: [(0, '79.780'), (1, '72.180')] [2023-10-14 20:24:20,426][61585] Updated weights for policy 1, policy_version 66440 (0.0009) [2023-10-14 20:24:20,791][61585] Updated weights for policy 1, policy_version 66450 (0.0010) [2023-10-14 20:24:21,117][61552] Updated weights for policy 0, policy_version 66602 (0.0009) [2023-10-14 20:24:21,150][61585] Updated weights for policy 1, policy_version 66460 (0.0008) [2023-10-14 20:24:21,493][61552] Updated weights for policy 0, policy_version 66612 (0.0007) [2023-10-14 20:24:21,852][61552] Updated weights for policy 0, policy_version 66622 (0.0009) [2023-10-14 20:24:23,343][60425] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 136282112. Throughput: 0: 1652.8, 1: 1659.9. Samples: 34074990. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-14 20:24:23,344][60425] Avg episode reward: [(0, '78.000'), (1, '74.170')] [2023-10-14 20:24:25,252][61585] Updated weights for policy 1, policy_version 66470 (0.0008) [2023-10-14 20:24:25,615][61585] Updated weights for policy 1, policy_version 66480 (0.0008) [2023-10-14 20:24:25,858][61552] Updated weights for policy 0, policy_version 66632 (0.0007) [2023-10-14 20:24:25,974][61585] Updated weights for policy 1, policy_version 66490 (0.0008) [2023-10-14 20:24:26,215][61552] Updated weights for policy 0, policy_version 66642 (0.0008) [2023-10-14 20:24:26,588][61552] Updated weights for policy 0, policy_version 66652 (0.0012) [2023-10-14 20:24:28,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 136347648. Throughput: 0: 1667.7, 1: 1673.0. Samples: 34095330. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-14 20:24:28,345][60425] Avg episode reward: [(0, '80.550'), (1, '74.040')] [2023-10-14 20:24:30,180][61585] Updated weights for policy 1, policy_version 66500 (0.0007) [2023-10-14 20:24:30,539][61585] Updated weights for policy 1, policy_version 66510 (0.0007) [2023-10-14 20:24:30,611][61552] Updated weights for policy 0, policy_version 66662 (0.0009) [2023-10-14 20:24:30,898][61585] Updated weights for policy 1, policy_version 66520 (0.0008) [2023-10-14 20:24:30,982][61552] Updated weights for policy 0, policy_version 66672 (0.0009) [2023-10-14 20:24:31,342][61552] Updated weights for policy 0, policy_version 66682 (0.0009) [2023-10-14 20:24:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 136413184. Throughput: 0: 1660.1, 1: 1655.9. Samples: 34106036. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-14 20:24:33,344][60425] Avg episode reward: [(0, '82.160'), (1, '71.890')] [2023-10-14 20:24:35,089][61585] Updated weights for policy 1, policy_version 66530 (0.0007) [2023-10-14 20:24:35,441][61552] Updated weights for policy 0, policy_version 66692 (0.0009) [2023-10-14 20:24:35,459][61585] Updated weights for policy 1, policy_version 66540 (0.0010) [2023-10-14 20:24:35,813][61552] Updated weights for policy 0, policy_version 66702 (0.0007) [2023-10-14 20:24:35,814][61585] Updated weights for policy 1, policy_version 66550 (0.0009) [2023-10-14 20:24:36,180][61585] Updated weights for policy 1, policy_version 66560 (0.0008) [2023-10-14 20:24:36,181][61552] Updated weights for policy 0, policy_version 66712 (0.0010) [2023-10-14 20:24:38,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 136478720. Throughput: 0: 1658.8, 1: 1666.1. Samples: 34125176. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-14 20:24:38,344][60425] Avg episode reward: [(0, '76.830'), (1, '75.430')] [2023-10-14 20:24:40,390][61585] Updated weights for policy 1, policy_version 66570 (0.0008) [2023-10-14 20:24:40,410][61552] Updated weights for policy 0, policy_version 66722 (0.0008) [2023-10-14 20:24:40,747][61585] Updated weights for policy 1, policy_version 66580 (0.0010) [2023-10-14 20:24:40,776][61552] Updated weights for policy 0, policy_version 66732 (0.0008) [2023-10-14 20:24:41,112][61585] Updated weights for policy 1, policy_version 66590 (0.0010) [2023-10-14 20:24:41,146][61552] Updated weights for policy 0, policy_version 66742 (0.0009) [2023-10-14 20:24:41,512][61552] Updated weights for policy 0, policy_version 66752 (0.0009) [2023-10-14 20:24:43,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 136544256. Throughput: 0: 1665.9, 1: 1664.4. Samples: 34145484. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-14 20:24:43,344][60425] Avg episode reward: [(0, '74.900'), (1, '72.460')] [2023-10-14 20:24:45,141][61585] Updated weights for policy 1, policy_version 66600 (0.0009) [2023-10-14 20:24:45,514][61585] Updated weights for policy 1, policy_version 66610 (0.0008) [2023-10-14 20:24:45,748][61552] Updated weights for policy 0, policy_version 66762 (0.0007) [2023-10-14 20:24:45,877][61585] Updated weights for policy 1, policy_version 66620 (0.0008) [2023-10-14 20:24:46,115][61552] Updated weights for policy 0, policy_version 66772 (0.0007) [2023-10-14 20:24:46,480][61552] Updated weights for policy 0, policy_version 66782 (0.0008) [2023-10-14 20:24:48,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 136609792. Throughput: 0: 1659.9, 1: 1647.0. Samples: 34155836. Policy #0 lag: (min: 29.0, avg: 30.6, max: 57.0) [2023-10-14 20:24:48,344][60425] Avg episode reward: [(0, '75.890'), (1, '70.270')] [2023-10-14 20:24:50,183][61585] Updated weights for policy 1, policy_version 66630 (0.0007) [2023-10-14 20:24:50,522][61552] Updated weights for policy 0, policy_version 66792 (0.0009) [2023-10-14 20:24:50,547][61585] Updated weights for policy 1, policy_version 66640 (0.0009) [2023-10-14 20:24:50,896][61552] Updated weights for policy 0, policy_version 66802 (0.0009) [2023-10-14 20:24:50,913][61585] Updated weights for policy 1, policy_version 66650 (0.0008) [2023-10-14 20:24:51,257][61552] Updated weights for policy 0, policy_version 66812 (0.0009) [2023-10-14 20:24:53,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 136675328. Throughput: 0: 1661.9, 1: 1654.8. Samples: 34174726. Policy #0 lag: (min: 29.0, avg: 30.6, max: 57.0) [2023-10-14 20:24:53,344][60425] Avg episode reward: [(0, '73.980'), (1, '74.610')] [2023-10-14 20:24:55,090][61585] Updated weights for policy 1, policy_version 66660 (0.0009) [2023-10-14 20:24:55,490][61585] Updated weights for policy 1, policy_version 66670 (0.0008) [2023-10-14 20:24:55,516][61552] Updated weights for policy 0, policy_version 66822 (0.0008) [2023-10-14 20:24:55,863][61585] Updated weights for policy 1, policy_version 66680 (0.0008) [2023-10-14 20:24:55,885][61552] Updated weights for policy 0, policy_version 66832 (0.0008) [2023-10-14 20:24:56,265][61552] Updated weights for policy 0, policy_version 66842 (0.0009) [2023-10-14 20:24:58,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 136740864. Throughput: 0: 1670.2, 1: 1659.0. Samples: 34195286. Policy #0 lag: (min: 29.0, avg: 30.6, max: 57.0) [2023-10-14 20:24:58,344][60425] Avg episode reward: [(0, '77.610'), (1, '72.470')] [2023-10-14 20:24:59,857][61585] Updated weights for policy 1, policy_version 66690 (0.0008) [2023-10-14 20:25:00,226][61585] Updated weights for policy 1, policy_version 66700 (0.0009) [2023-10-14 20:25:00,390][61552] Updated weights for policy 0, policy_version 66852 (0.0009) [2023-10-14 20:25:00,593][61585] Updated weights for policy 1, policy_version 66710 (0.0009) [2023-10-14 20:25:00,783][61552] Updated weights for policy 0, policy_version 66862 (0.0008) [2023-10-14 20:25:00,962][61585] Updated weights for policy 1, policy_version 66720 (0.0009) [2023-10-14 20:25:01,159][61552] Updated weights for policy 0, policy_version 66872 (0.0008) [2023-10-14 20:25:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 136806400. Throughput: 0: 1659.7, 1: 1649.6. Samples: 34205208. Policy #0 lag: (min: 29.0, avg: 30.6, max: 57.0) [2023-10-14 20:25:03,344][60425] Avg episode reward: [(0, '78.040'), (1, '77.260')] [2023-10-14 20:25:05,154][61585] Updated weights for policy 1, policy_version 66730 (0.0008) [2023-10-14 20:25:05,257][61552] Updated weights for policy 0, policy_version 66882 (0.0008) [2023-10-14 20:25:05,517][61585] Updated weights for policy 1, policy_version 66740 (0.0008) [2023-10-14 20:25:05,619][61552] Updated weights for policy 0, policy_version 66892 (0.0009) [2023-10-14 20:25:05,876][61585] Updated weights for policy 1, policy_version 66750 (0.0007) [2023-10-14 20:25:05,975][61552] Updated weights for policy 0, policy_version 66902 (0.0008) [2023-10-14 20:25:06,343][61552] Updated weights for policy 0, policy_version 66912 (0.0009) [2023-10-14 20:25:08,343][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 136871936. Throughput: 0: 1662.8, 1: 1657.6. Samples: 34224410. Policy #0 lag: (min: 29.0, avg: 30.6, max: 57.0) [2023-10-14 20:25:08,345][60425] Avg episode reward: [(0, '76.270'), (1, '74.520')] [2023-10-14 20:25:09,965][61585] Updated weights for policy 1, policy_version 66760 (0.0007) [2023-10-14 20:25:10,334][61585] Updated weights for policy 1, policy_version 66770 (0.0008) [2023-10-14 20:25:10,336][61552] Updated weights for policy 0, policy_version 66922 (0.0007) [2023-10-14 20:25:10,696][61585] Updated weights for policy 1, policy_version 66780 (0.0009) [2023-10-14 20:25:10,707][61552] Updated weights for policy 0, policy_version 66932 (0.0007) [2023-10-14 20:25:11,068][61552] Updated weights for policy 0, policy_version 66942 (0.0008) [2023-10-14 20:25:13,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 136937472. Throughput: 0: 1673.4, 1: 1655.7. Samples: 34245138. Policy #0 lag: (min: 29.0, avg: 30.6, max: 57.0) [2023-10-14 20:25:13,344][60425] Avg episode reward: [(0, '73.350'), (1, '77.610')] [2023-10-14 20:25:14,978][61552] Updated weights for policy 0, policy_version 66952 (0.0007) [2023-10-14 20:25:15,018][61585] Updated weights for policy 1, policy_version 66790 (0.0009) [2023-10-14 20:25:15,355][61552] Updated weights for policy 0, policy_version 66962 (0.0007) [2023-10-14 20:25:15,384][61585] Updated weights for policy 1, policy_version 66800 (0.0009) [2023-10-14 20:25:15,714][61552] Updated weights for policy 0, policy_version 66972 (0.0007) [2023-10-14 20:25:15,743][61585] Updated weights for policy 1, policy_version 66810 (0.0009) [2023-10-14 20:25:18,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 137003008. Throughput: 0: 1657.4, 1: 1648.3. Samples: 34254792. Policy #0 lag: (min: 29.0, avg: 30.6, max: 57.0) [2023-10-14 20:25:18,344][60425] Avg episode reward: [(0, '76.490'), (1, '76.440')] [2023-10-14 20:25:19,798][61585] Updated weights for policy 1, policy_version 66820 (0.0009) [2023-10-14 20:25:19,834][61552] Updated weights for policy 0, policy_version 66982 (0.0008) [2023-10-14 20:25:20,164][61585] Updated weights for policy 1, policy_version 66830 (0.0008) [2023-10-14 20:25:20,203][61552] Updated weights for policy 0, policy_version 66992 (0.0009) [2023-10-14 20:25:20,525][61585] Updated weights for policy 1, policy_version 66840 (0.0008) [2023-10-14 20:25:20,566][61552] Updated weights for policy 0, policy_version 67002 (0.0008) [2023-10-14 20:25:23,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 137068544. Throughput: 0: 1668.6, 1: 1652.9. Samples: 34274646. Policy #0 lag: (min: 29.0, avg: 30.6, max: 57.0) [2023-10-14 20:25:23,344][60425] Avg episode reward: [(0, '78.070'), (1, '74.690')] [2023-10-14 20:25:24,595][61552] Updated weights for policy 0, policy_version 67012 (0.0009) [2023-10-14 20:25:24,734][61585] Updated weights for policy 1, policy_version 66850 (0.0008) [2023-10-14 20:25:24,966][61552] Updated weights for policy 0, policy_version 67022 (0.0010) [2023-10-14 20:25:25,103][61585] Updated weights for policy 1, policy_version 66860 (0.0007) [2023-10-14 20:25:25,335][61552] Updated weights for policy 0, policy_version 67032 (0.0011) [2023-10-14 20:25:25,463][61585] Updated weights for policy 1, policy_version 66870 (0.0009) [2023-10-14 20:25:25,828][61585] Updated weights for policy 1, policy_version 66880 (0.0010) [2023-10-14 20:25:28,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 137134080. Throughput: 0: 1669.9, 1: 1655.9. Samples: 34295146. Policy #0 lag: (min: 29.0, avg: 30.6, max: 57.0) [2023-10-14 20:25:28,344][60425] Avg episode reward: [(0, '79.260'), (1, '79.350')] [2023-10-14 20:25:29,437][61552] Updated weights for policy 0, policy_version 67042 (0.0007) [2023-10-14 20:25:29,800][61552] Updated weights for policy 0, policy_version 67052 (0.0008) [2023-10-14 20:25:29,996][61585] Updated weights for policy 1, policy_version 66890 (0.0007) [2023-10-14 20:25:30,170][61552] Updated weights for policy 0, policy_version 67062 (0.0007) [2023-10-14 20:25:30,359][61585] Updated weights for policy 1, policy_version 66900 (0.0008) [2023-10-14 20:25:30,530][61552] Updated weights for policy 0, policy_version 67072 (0.0009) [2023-10-14 20:25:30,725][61585] Updated weights for policy 1, policy_version 66910 (0.0007) [2023-10-14 20:25:33,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 137199616. Throughput: 0: 1648.8, 1: 1652.7. Samples: 34304408. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:25:33,344][60425] Avg episode reward: [(0, '75.840'), (1, '76.350')] [2023-10-14 20:25:34,692][61585] Updated weights for policy 1, policy_version 66920 (0.0009) [2023-10-14 20:25:34,738][61552] Updated weights for policy 0, policy_version 67082 (0.0008) [2023-10-14 20:25:35,051][61585] Updated weights for policy 1, policy_version 66930 (0.0007) [2023-10-14 20:25:35,101][61552] Updated weights for policy 0, policy_version 67092 (0.0009) [2023-10-14 20:25:35,424][61585] Updated weights for policy 1, policy_version 66940 (0.0008) [2023-10-14 20:25:35,474][61552] Updated weights for policy 0, policy_version 67102 (0.0007) [2023-10-14 20:25:38,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 137265152. Throughput: 0: 1670.9, 1: 1662.5. Samples: 34324730. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:25:38,344][60425] Avg episode reward: [(0, '75.020'), (1, '77.090')] [2023-10-14 20:25:39,592][61585] Updated weights for policy 1, policy_version 66950 (0.0009) [2023-10-14 20:25:39,597][61552] Updated weights for policy 0, policy_version 67112 (0.0008) [2023-10-14 20:25:39,957][61585] Updated weights for policy 1, policy_version 66960 (0.0007) [2023-10-14 20:25:39,960][61552] Updated weights for policy 0, policy_version 67122 (0.0007) [2023-10-14 20:25:40,315][61585] Updated weights for policy 1, policy_version 66970 (0.0007) [2023-10-14 20:25:40,332][61552] Updated weights for policy 0, policy_version 67132 (0.0008) [2023-10-14 20:25:43,344][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 137330688. Throughput: 0: 1668.7, 1: 1666.4. Samples: 34345370. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:25:43,345][60425] Avg episode reward: [(0, '75.430'), (1, '75.800')] [2023-10-14 20:25:43,353][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000067136_68747264.pth... [2023-10-14 20:25:43,354][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000066976_68583424.pth... [2023-10-14 20:25:43,382][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000065568_67141632.pth [2023-10-14 20:25:43,389][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000065440_67010560.pth [2023-10-14 20:25:44,241][61585] Updated weights for policy 1, policy_version 66980 (0.0008) [2023-10-14 20:25:44,541][61552] Updated weights for policy 0, policy_version 67142 (0.0007) [2023-10-14 20:25:44,629][61585] Updated weights for policy 1, policy_version 66990 (0.0010) [2023-10-14 20:25:44,900][61552] Updated weights for policy 0, policy_version 67152 (0.0007) [2023-10-14 20:25:44,991][61585] Updated weights for policy 1, policy_version 67000 (0.0009) [2023-10-14 20:25:45,270][61552] Updated weights for policy 0, policy_version 67162 (0.0009) [2023-10-14 20:25:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 137396224. Throughput: 0: 1653.7, 1: 1659.6. Samples: 34354306. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:25:48,344][60425] Avg episode reward: [(0, '75.610'), (1, '73.230')] [2023-10-14 20:25:49,045][61585] Updated weights for policy 1, policy_version 67010 (0.0009) [2023-10-14 20:25:49,406][61585] Updated weights for policy 1, policy_version 67020 (0.0008) [2023-10-14 20:25:49,448][61552] Updated weights for policy 0, policy_version 67172 (0.0010) [2023-10-14 20:25:49,776][61585] Updated weights for policy 1, policy_version 67030 (0.0010) [2023-10-14 20:25:49,821][61552] Updated weights for policy 0, policy_version 67182 (0.0009) [2023-10-14 20:25:50,151][61585] Updated weights for policy 1, policy_version 67040 (0.0009) [2023-10-14 20:25:50,184][61552] Updated weights for policy 0, policy_version 67192 (0.0008) [2023-10-14 20:25:53,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 137461760. Throughput: 0: 1672.4, 1: 1671.4. Samples: 34374880. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:25:53,344][60425] Avg episode reward: [(0, '75.920'), (1, '75.630')] [2023-10-14 20:25:54,203][61552] Updated weights for policy 0, policy_version 67202 (0.0007) [2023-10-14 20:25:54,203][61585] Updated weights for policy 1, policy_version 67050 (0.0009) [2023-10-14 20:25:54,574][61585] Updated weights for policy 1, policy_version 67060 (0.0009) [2023-10-14 20:25:54,605][61552] Updated weights for policy 0, policy_version 67212 (0.0007) [2023-10-14 20:25:54,934][61585] Updated weights for policy 1, policy_version 67070 (0.0010) [2023-10-14 20:25:54,967][61552] Updated weights for policy 0, policy_version 67222 (0.0007) [2023-10-14 20:25:55,336][61552] Updated weights for policy 0, policy_version 67232 (0.0008) [2023-10-14 20:25:58,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 137527296. Throughput: 0: 1664.9, 1: 1673.9. Samples: 34395382. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:25:58,345][60425] Avg episode reward: [(0, '75.890'), (1, '77.060')] [2023-10-14 20:25:59,187][61585] Updated weights for policy 1, policy_version 67080 (0.0008) [2023-10-14 20:25:59,362][61552] Updated weights for policy 0, policy_version 67242 (0.0008) [2023-10-14 20:25:59,553][61585] Updated weights for policy 1, policy_version 67090 (0.0008) [2023-10-14 20:25:59,741][61552] Updated weights for policy 0, policy_version 67252 (0.0008) [2023-10-14 20:25:59,928][61585] Updated weights for policy 1, policy_version 67100 (0.0008) [2023-10-14 20:26:00,102][61552] Updated weights for policy 0, policy_version 67262 (0.0009) [2023-10-14 20:26:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 137592832. Throughput: 0: 1658.0, 1: 1666.0. Samples: 34404370. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:26:03,344][60425] Avg episode reward: [(0, '74.760'), (1, '76.120')] [2023-10-14 20:26:03,947][61585] Updated weights for policy 1, policy_version 67110 (0.0009) [2023-10-14 20:26:04,248][61552] Updated weights for policy 0, policy_version 67272 (0.0008) [2023-10-14 20:26:04,308][61585] Updated weights for policy 1, policy_version 67120 (0.0009) [2023-10-14 20:26:04,613][61552] Updated weights for policy 0, policy_version 67282 (0.0007) [2023-10-14 20:26:04,679][61585] Updated weights for policy 1, policy_version 67130 (0.0008) [2023-10-14 20:26:04,981][61552] Updated weights for policy 0, policy_version 67292 (0.0007) [2023-10-14 20:26:08,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 137658368. Throughput: 0: 1669.1, 1: 1678.7. Samples: 34425294. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:26:08,344][60425] Avg episode reward: [(0, '75.210'), (1, '71.690')] [2023-10-14 20:26:08,785][61585] Updated weights for policy 1, policy_version 67140 (0.0009) [2023-10-14 20:26:08,956][61552] Updated weights for policy 0, policy_version 67302 (0.0008) [2023-10-14 20:26:09,152][61585] Updated weights for policy 1, policy_version 67150 (0.0009) [2023-10-14 20:26:09,321][61552] Updated weights for policy 0, policy_version 67312 (0.0008) [2023-10-14 20:26:09,509][61585] Updated weights for policy 1, policy_version 67160 (0.0007) [2023-10-14 20:26:09,689][61552] Updated weights for policy 0, policy_version 67322 (0.0008) [2023-10-14 20:26:13,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 137723904. Throughput: 0: 1667.3, 1: 1681.3. Samples: 34445830. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:26:13,344][60425] Avg episode reward: [(0, '78.600'), (1, '73.000')] [2023-10-14 20:26:13,499][61585] Updated weights for policy 1, policy_version 67170 (0.0007) [2023-10-14 20:26:13,818][61552] Updated weights for policy 0, policy_version 67332 (0.0008) [2023-10-14 20:26:13,864][61585] Updated weights for policy 1, policy_version 67180 (0.0007) [2023-10-14 20:26:14,187][61552] Updated weights for policy 0, policy_version 67342 (0.0008) [2023-10-14 20:26:14,228][61585] Updated weights for policy 1, policy_version 67190 (0.0009) [2023-10-14 20:26:14,548][61552] Updated weights for policy 0, policy_version 67352 (0.0008) [2023-10-14 20:26:14,582][61585] Updated weights for policy 1, policy_version 67200 (0.0007) [2023-10-14 20:26:18,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 137789440. Throughput: 0: 1665.3, 1: 1681.4. Samples: 34455010. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-14 20:26:18,345][60425] Avg episode reward: [(0, '78.700'), (1, '75.140')] [2023-10-14 20:26:18,601][61585] Updated weights for policy 1, policy_version 67210 (0.0009) [2023-10-14 20:26:18,779][61552] Updated weights for policy 0, policy_version 67362 (0.0008) [2023-10-14 20:26:18,968][61585] Updated weights for policy 1, policy_version 67220 (0.0009) [2023-10-14 20:26:19,136][61552] Updated weights for policy 0, policy_version 67372 (0.0008) [2023-10-14 20:26:19,350][61585] Updated weights for policy 1, policy_version 67230 (0.0009) [2023-10-14 20:26:19,503][61552] Updated weights for policy 0, policy_version 67382 (0.0008) [2023-10-14 20:26:19,868][61552] Updated weights for policy 0, policy_version 67392 (0.0007) [2023-10-14 20:26:23,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 137854976. Throughput: 0: 1664.4, 1: 1684.9. Samples: 34475450. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-14 20:26:23,344][60425] Avg episode reward: [(0, '80.110'), (1, '76.920')] [2023-10-14 20:26:23,486][61585] Updated weights for policy 1, policy_version 67240 (0.0008) [2023-10-14 20:26:23,851][61585] Updated weights for policy 1, policy_version 67250 (0.0007) [2023-10-14 20:26:23,973][61552] Updated weights for policy 0, policy_version 67402 (0.0008) [2023-10-14 20:26:24,217][61585] Updated weights for policy 1, policy_version 67260 (0.0009) [2023-10-14 20:26:24,343][61552] Updated weights for policy 0, policy_version 67412 (0.0007) [2023-10-14 20:26:24,716][61552] Updated weights for policy 0, policy_version 67422 (0.0007) [2023-10-14 20:26:28,320][61585] Updated weights for policy 1, policy_version 67270 (0.0008) [2023-10-14 20:26:28,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 137920512. Throughput: 0: 1668.0, 1: 1682.9. Samples: 34496156. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-14 20:26:28,344][60425] Avg episode reward: [(0, '75.490'), (1, '74.900')] [2023-10-14 20:26:28,684][61585] Updated weights for policy 1, policy_version 67280 (0.0010) [2023-10-14 20:26:28,793][61552] Updated weights for policy 0, policy_version 67432 (0.0011) [2023-10-14 20:26:29,058][61585] Updated weights for policy 1, policy_version 67290 (0.0010) [2023-10-14 20:26:29,160][61552] Updated weights for policy 0, policy_version 67442 (0.0008) [2023-10-14 20:26:29,524][61552] Updated weights for policy 0, policy_version 67452 (0.0008) [2023-10-14 20:26:33,159][61585] Updated weights for policy 1, policy_version 67300 (0.0008) [2023-10-14 20:26:33,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 137986048. Throughput: 0: 1668.8, 1: 1685.1. Samples: 34505230. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-14 20:26:33,344][60425] Avg episode reward: [(0, '78.320'), (1, '72.230')] [2023-10-14 20:26:33,550][61585] Updated weights for policy 1, policy_version 67310 (0.0009) [2023-10-14 20:26:33,675][61552] Updated weights for policy 0, policy_version 67462 (0.0008) [2023-10-14 20:26:33,910][61585] Updated weights for policy 1, policy_version 67320 (0.0007) [2023-10-14 20:26:34,050][61552] Updated weights for policy 0, policy_version 67472 (0.0008) [2023-10-14 20:26:34,410][61552] Updated weights for policy 0, policy_version 67482 (0.0009) [2023-10-14 20:26:38,107][61585] Updated weights for policy 1, policy_version 67330 (0.0008) [2023-10-14 20:26:38,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 138051584. Throughput: 0: 1667.6, 1: 1681.7. Samples: 34525602. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-14 20:26:38,344][60425] Avg episode reward: [(0, '77.520'), (1, '69.330')] [2023-10-14 20:26:38,470][61585] Updated weights for policy 1, policy_version 67340 (0.0009) [2023-10-14 20:26:38,584][61552] Updated weights for policy 0, policy_version 67492 (0.0008) [2023-10-14 20:26:38,840][61585] Updated weights for policy 1, policy_version 67350 (0.0008) [2023-10-14 20:26:38,962][61552] Updated weights for policy 0, policy_version 67502 (0.0008) [2023-10-14 20:26:39,199][61585] Updated weights for policy 1, policy_version 67360 (0.0007) [2023-10-14 20:26:39,327][61552] Updated weights for policy 0, policy_version 67512 (0.0010) [2023-10-14 20:26:43,273][61585] Updated weights for policy 1, policy_version 67370 (0.0009) [2023-10-14 20:26:43,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 138117120. Throughput: 0: 1672.4, 1: 1681.3. Samples: 34546296. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-14 20:26:43,344][60425] Avg episode reward: [(0, '80.230'), (1, '75.660')] [2023-10-14 20:26:43,464][61552] Updated weights for policy 0, policy_version 67522 (0.0010) [2023-10-14 20:26:43,634][61585] Updated weights for policy 1, policy_version 67380 (0.0008) [2023-10-14 20:26:43,851][61552] Updated weights for policy 0, policy_version 67532 (0.0007) [2023-10-14 20:26:43,998][61585] Updated weights for policy 1, policy_version 67390 (0.0008) [2023-10-14 20:26:44,221][61552] Updated weights for policy 0, policy_version 67542 (0.0007) [2023-10-14 20:26:44,585][61552] Updated weights for policy 0, policy_version 67552 (0.0008) [2023-10-14 20:26:48,165][61585] Updated weights for policy 1, policy_version 67400 (0.0009) [2023-10-14 20:26:48,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 138182656. Throughput: 0: 1668.6, 1: 1686.0. Samples: 34555330. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-14 20:26:48,344][60425] Avg episode reward: [(0, '79.180'), (1, '77.280')] [2023-10-14 20:26:48,530][61585] Updated weights for policy 1, policy_version 67410 (0.0010) [2023-10-14 20:26:48,759][61552] Updated weights for policy 0, policy_version 67562 (0.0008) [2023-10-14 20:26:48,896][61585] Updated weights for policy 1, policy_version 67420 (0.0008) [2023-10-14 20:26:49,131][61552] Updated weights for policy 0, policy_version 67572 (0.0007) [2023-10-14 20:26:49,507][61552] Updated weights for policy 0, policy_version 67582 (0.0008) [2023-10-14 20:26:53,000][61585] Updated weights for policy 1, policy_version 67430 (0.0008) [2023-10-14 20:26:53,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 138248192. Throughput: 0: 1667.9, 1: 1678.6. Samples: 34575886. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-14 20:26:53,344][60425] Avg episode reward: [(0, '79.440'), (1, '77.410')] [2023-10-14 20:26:53,365][61585] Updated weights for policy 1, policy_version 67440 (0.0009) [2023-10-14 20:26:53,496][61552] Updated weights for policy 0, policy_version 67592 (0.0010) [2023-10-14 20:26:53,732][61585] Updated weights for policy 1, policy_version 67450 (0.0010) [2023-10-14 20:26:53,868][61552] Updated weights for policy 0, policy_version 67602 (0.0008) [2023-10-14 20:26:54,225][61552] Updated weights for policy 0, policy_version 67612 (0.0011) [2023-10-14 20:26:57,915][61585] Updated weights for policy 1, policy_version 67460 (0.0010) [2023-10-14 20:26:58,282][61585] Updated weights for policy 1, policy_version 67470 (0.0007) [2023-10-14 20:26:58,286][61552] Updated weights for policy 0, policy_version 67622 (0.0007) [2023-10-14 20:26:58,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 138313728. Throughput: 0: 1677.4, 1: 1678.5. Samples: 34596844. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-14 20:26:58,344][60425] Avg episode reward: [(0, '78.650'), (1, '77.250')] [2023-10-14 20:26:58,649][61585] Updated weights for policy 1, policy_version 67480 (0.0008) [2023-10-14 20:26:58,650][61552] Updated weights for policy 0, policy_version 67632 (0.0007) [2023-10-14 20:26:59,017][61552] Updated weights for policy 0, policy_version 67642 (0.0009) [2023-10-14 20:27:02,771][61585] Updated weights for policy 1, policy_version 67490 (0.0008) [2023-10-14 20:27:02,999][61552] Updated weights for policy 0, policy_version 67652 (0.0008) [2023-10-14 20:27:03,144][61585] Updated weights for policy 1, policy_version 67500 (0.0008) [2023-10-14 20:27:03,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 138379264. Throughput: 0: 1682.1, 1: 1674.9. Samples: 34606072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:27:03,344][60425] Avg episode reward: [(0, '82.210'), (1, '75.720')] [2023-10-14 20:27:03,366][61552] Updated weights for policy 0, policy_version 67662 (0.0008) [2023-10-14 20:27:03,507][61585] Updated weights for policy 1, policy_version 67510 (0.0008) [2023-10-14 20:27:03,745][61552] Updated weights for policy 0, policy_version 67672 (0.0008) [2023-10-14 20:27:03,860][61585] Updated weights for policy 1, policy_version 67520 (0.0007) [2023-10-14 20:27:07,843][61552] Updated weights for policy 0, policy_version 67682 (0.0008) [2023-10-14 20:27:07,966][61585] Updated weights for policy 1, policy_version 67530 (0.0009) [2023-10-14 20:27:08,197][61552] Updated weights for policy 0, policy_version 67692 (0.0009) [2023-10-14 20:27:08,337][61585] Updated weights for policy 1, policy_version 67540 (0.0008) [2023-10-14 20:27:08,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 138444800. Throughput: 0: 1682.8, 1: 1675.6. Samples: 34626578. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:27:08,344][60425] Avg episode reward: [(0, '81.690'), (1, '72.220')] [2023-10-14 20:27:08,570][61552] Updated weights for policy 0, policy_version 67702 (0.0009) [2023-10-14 20:27:08,711][61585] Updated weights for policy 1, policy_version 67550 (0.0008) [2023-10-14 20:27:08,936][61552] Updated weights for policy 0, policy_version 67712 (0.0008) [2023-10-14 20:27:12,863][61585] Updated weights for policy 1, policy_version 67560 (0.0008) [2023-10-14 20:27:13,038][61552] Updated weights for policy 0, policy_version 67722 (0.0007) [2023-10-14 20:27:13,225][61585] Updated weights for policy 1, policy_version 67570 (0.0007) [2023-10-14 20:27:13,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 138510336. Throughput: 0: 1683.5, 1: 1668.8. Samples: 34647008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:27:13,344][60425] Avg episode reward: [(0, '83.760'), (1, '73.090')] [2023-10-14 20:27:13,410][61552] Updated weights for policy 0, policy_version 67732 (0.0008) [2023-10-14 20:27:13,592][61585] Updated weights for policy 1, policy_version 67580 (0.0008) [2023-10-14 20:27:13,774][61552] Updated weights for policy 0, policy_version 67742 (0.0008) [2023-10-14 20:27:17,693][61585] Updated weights for policy 1, policy_version 67590 (0.0009) [2023-10-14 20:27:17,794][61552] Updated weights for policy 0, policy_version 67752 (0.0009) [2023-10-14 20:27:18,056][61585] Updated weights for policy 1, policy_version 67600 (0.0008) [2023-10-14 20:27:18,155][61552] Updated weights for policy 0, policy_version 67762 (0.0008) [2023-10-14 20:27:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 138575872. Throughput: 0: 1689.2, 1: 1670.9. Samples: 34656438. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:27:18,344][60425] Avg episode reward: [(0, '78.370'), (1, '79.380')] [2023-10-14 20:27:18,420][61585] Updated weights for policy 1, policy_version 67610 (0.0009) [2023-10-14 20:27:18,526][61552] Updated weights for policy 0, policy_version 67772 (0.0008) [2023-10-14 20:27:22,405][61552] Updated weights for policy 0, policy_version 67782 (0.0009) [2023-10-14 20:27:22,649][61585] Updated weights for policy 1, policy_version 67620 (0.0007) [2023-10-14 20:27:22,774][61552] Updated weights for policy 0, policy_version 67792 (0.0009) [2023-10-14 20:27:23,050][61585] Updated weights for policy 1, policy_version 67630 (0.0007) [2023-10-14 20:27:23,137][61552] Updated weights for policy 0, policy_version 67802 (0.0008) [2023-10-14 20:27:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 138641408. Throughput: 0: 1694.3, 1: 1674.2. Samples: 34677186. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:27:23,344][60425] Avg episode reward: [(0, '76.710'), (1, '76.520')] [2023-10-14 20:27:23,421][61585] Updated weights for policy 1, policy_version 67640 (0.0008) [2023-10-14 20:27:27,189][61552] Updated weights for policy 0, policy_version 67812 (0.0008) [2023-10-14 20:27:27,192][61585] Updated weights for policy 1, policy_version 67650 (0.0009) [2023-10-14 20:27:27,561][61585] Updated weights for policy 1, policy_version 67660 (0.0009) [2023-10-14 20:27:27,569][61552] Updated weights for policy 0, policy_version 67822 (0.0009) [2023-10-14 20:27:27,932][61585] Updated weights for policy 1, policy_version 67670 (0.0009) [2023-10-14 20:27:27,935][61552] Updated weights for policy 0, policy_version 67832 (0.0009) [2023-10-14 20:27:28,289][61585] Updated weights for policy 1, policy_version 67680 (0.0009) [2023-10-14 20:27:28,343][60425] Fps is (10 sec: 19660.7, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 138772480. Throughput: 0: 1677.0, 1: 1659.1. Samples: 34696420. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:27:28,344][60425] Avg episode reward: [(0, '81.130'), (1, '73.310')] [2023-10-14 20:27:32,048][61552] Updated weights for policy 0, policy_version 67842 (0.0009) [2023-10-14 20:27:32,422][61552] Updated weights for policy 0, policy_version 67852 (0.0009) [2023-10-14 20:27:32,598][61585] Updated weights for policy 1, policy_version 67690 (0.0007) [2023-10-14 20:27:32,799][61552] Updated weights for policy 0, policy_version 67862 (0.0007) [2023-10-14 20:27:32,965][61585] Updated weights for policy 1, policy_version 67700 (0.0007) [2023-10-14 20:27:33,155][61552] Updated weights for policy 0, policy_version 67872 (0.0008) [2023-10-14 20:27:33,332][61585] Updated weights for policy 1, policy_version 67710 (0.0009) [2023-10-14 20:27:33,343][60425] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 138805248. Throughput: 0: 1696.3, 1: 1667.4. Samples: 34706698. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:27:33,344][60425] Avg episode reward: [(0, '80.350'), (1, '71.600')] [2023-10-14 20:27:37,203][61552] Updated weights for policy 0, policy_version 67882 (0.0010) [2023-10-14 20:27:37,539][61585] Updated weights for policy 1, policy_version 67720 (0.0009) [2023-10-14 20:27:37,568][61552] Updated weights for policy 0, policy_version 67892 (0.0009) [2023-10-14 20:27:37,912][61585] Updated weights for policy 1, policy_version 67730 (0.0009) [2023-10-14 20:27:37,935][61552] Updated weights for policy 0, policy_version 67902 (0.0009) [2023-10-14 20:27:38,271][61585] Updated weights for policy 1, policy_version 67740 (0.0009) [2023-10-14 20:27:38,343][60425] Fps is (10 sec: 9830.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 138870784. Throughput: 0: 1693.5, 1: 1664.2. Samples: 34726982. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:27:38,344][60425] Avg episode reward: [(0, '78.400'), (1, '78.330')] [2023-10-14 20:27:42,086][61552] Updated weights for policy 0, policy_version 67912 (0.0008) [2023-10-14 20:27:42,295][61585] Updated weights for policy 1, policy_version 67750 (0.0008) [2023-10-14 20:27:42,445][61552] Updated weights for policy 0, policy_version 67922 (0.0008) [2023-10-14 20:27:42,665][61585] Updated weights for policy 1, policy_version 67760 (0.0008) [2023-10-14 20:27:42,817][61552] Updated weights for policy 0, policy_version 67932 (0.0009) [2023-10-14 20:27:43,026][61585] Updated weights for policy 1, policy_version 67770 (0.0008) [2023-10-14 20:27:43,343][60425] Fps is (10 sec: 16383.8, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 138969088. Throughput: 0: 1664.4, 1: 1648.0. Samples: 34745902. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:27:43,344][60425] Avg episode reward: [(0, '76.720'), (1, '79.670')] [2023-10-14 20:27:43,351][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000067936_69566464.pth... [2023-10-14 20:27:43,351][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000067776_69402624.pth... [2023-10-14 20:27:43,389][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000066208_67796992.pth [2023-10-14 20:27:43,389][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000066368_67960832.pth [2023-10-14 20:27:46,911][61552] Updated weights for policy 0, policy_version 67942 (0.0009) [2023-10-14 20:27:47,186][61585] Updated weights for policy 1, policy_version 67780 (0.0010) [2023-10-14 20:27:47,276][61552] Updated weights for policy 0, policy_version 67952 (0.0009) [2023-10-14 20:27:47,554][61585] Updated weights for policy 1, policy_version 67790 (0.0009) [2023-10-14 20:27:47,632][61552] Updated weights for policy 0, policy_version 67962 (0.0008) [2023-10-14 20:27:47,922][61585] Updated weights for policy 1, policy_version 67800 (0.0009) [2023-10-14 20:27:48,343][60425] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 139034624. Throughput: 0: 1680.0, 1: 1662.1. Samples: 34756466. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:27:48,344][60425] Avg episode reward: [(0, '73.370'), (1, '74.610')] [2023-10-14 20:27:51,700][61552] Updated weights for policy 0, policy_version 67972 (0.0009) [2023-10-14 20:27:52,008][61585] Updated weights for policy 1, policy_version 67810 (0.0010) [2023-10-14 20:27:52,058][61552] Updated weights for policy 0, policy_version 67982 (0.0009) [2023-10-14 20:27:52,368][61585] Updated weights for policy 1, policy_version 67820 (0.0009) [2023-10-14 20:27:52,434][61552] Updated weights for policy 0, policy_version 67992 (0.0008) [2023-10-14 20:27:52,732][61585] Updated weights for policy 1, policy_version 67830 (0.0007) [2023-10-14 20:27:53,098][61585] Updated weights for policy 1, policy_version 67840 (0.0009) [2023-10-14 20:27:53,343][60425] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13440.5). Total num frames: 139100160. Throughput: 0: 1679.0, 1: 1659.7. Samples: 34776816. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:27:53,344][60425] Avg episode reward: [(0, '75.930'), (1, '77.420')] [2023-10-14 20:27:56,539][61552] Updated weights for policy 0, policy_version 68002 (0.0009) [2023-10-14 20:27:56,899][61552] Updated weights for policy 0, policy_version 68012 (0.0007) [2023-10-14 20:27:57,048][61585] Updated weights for policy 1, policy_version 67850 (0.0008) [2023-10-14 20:27:57,276][61552] Updated weights for policy 0, policy_version 68022 (0.0007) [2023-10-14 20:27:57,410][61585] Updated weights for policy 1, policy_version 67860 (0.0009) [2023-10-14 20:27:57,641][61552] Updated weights for policy 0, policy_version 68032 (0.0007) [2023-10-14 20:27:57,784][61585] Updated weights for policy 1, policy_version 67870 (0.0009) [2023-10-14 20:27:58,344][60425] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 13329.3). Total num frames: 139165696. Throughput: 0: 1651.9, 1: 1642.7. Samples: 34795270. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:27:58,345][60425] Avg episode reward: [(0, '75.160'), (1, '75.360')] [2023-10-14 20:28:01,849][61552] Updated weights for policy 0, policy_version 68042 (0.0007) [2023-10-14 20:28:01,892][61585] Updated weights for policy 1, policy_version 67880 (0.0009) [2023-10-14 20:28:02,215][61552] Updated weights for policy 0, policy_version 68052 (0.0007) [2023-10-14 20:28:02,254][61585] Updated weights for policy 1, policy_version 67890 (0.0008) [2023-10-14 20:28:02,588][61552] Updated weights for policy 0, policy_version 68062 (0.0007) [2023-10-14 20:28:02,622][61585] Updated weights for policy 1, policy_version 67900 (0.0007) [2023-10-14 20:28:03,343][60425] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 13329.3). Total num frames: 139231232. Throughput: 0: 1674.2, 1: 1662.9. Samples: 34806606. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:28:03,345][60425] Avg episode reward: [(0, '74.340'), (1, '75.660')] [2023-10-14 20:28:06,638][61552] Updated weights for policy 0, policy_version 68072 (0.0008) [2023-10-14 20:28:06,854][61585] Updated weights for policy 1, policy_version 67910 (0.0009) [2023-10-14 20:28:07,007][61552] Updated weights for policy 0, policy_version 68082 (0.0008) [2023-10-14 20:28:07,225][61585] Updated weights for policy 1, policy_version 67920 (0.0010) [2023-10-14 20:28:07,370][61552] Updated weights for policy 0, policy_version 68092 (0.0009) [2023-10-14 20:28:07,595][61585] Updated weights for policy 1, policy_version 67930 (0.0009) [2023-10-14 20:28:08,343][60425] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 139296768. Throughput: 0: 1662.0, 1: 1656.8. Samples: 34826534. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:28:08,345][60425] Avg episode reward: [(0, '76.990'), (1, '77.270')] [2023-10-14 20:28:11,601][61552] Updated weights for policy 0, policy_version 68102 (0.0010) [2023-10-14 20:28:11,714][61585] Updated weights for policy 1, policy_version 67940 (0.0008) [2023-10-14 20:28:11,961][61552] Updated weights for policy 0, policy_version 68112 (0.0007) [2023-10-14 20:28:12,107][61585] Updated weights for policy 1, policy_version 67950 (0.0007) [2023-10-14 20:28:12,323][61552] Updated weights for policy 0, policy_version 68122 (0.0007) [2023-10-14 20:28:12,470][61585] Updated weights for policy 1, policy_version 67960 (0.0007) [2023-10-14 20:28:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 13329.4). Total num frames: 139362304. Throughput: 0: 1652.6, 1: 1645.9. Samples: 34844856. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:28:13,345][60425] Avg episode reward: [(0, '76.410'), (1, '77.880')] [2023-10-14 20:28:16,592][61585] Updated weights for policy 1, policy_version 67970 (0.0011) [2023-10-14 20:28:16,614][61552] Updated weights for policy 0, policy_version 68132 (0.0009) [2023-10-14 20:28:16,960][61585] Updated weights for policy 1, policy_version 67980 (0.0010) [2023-10-14 20:28:16,982][61552] Updated weights for policy 0, policy_version 68142 (0.0008) [2023-10-14 20:28:17,318][61585] Updated weights for policy 1, policy_version 67990 (0.0008) [2023-10-14 20:28:17,349][61552] Updated weights for policy 0, policy_version 68152 (0.0008) [2023-10-14 20:28:17,679][61585] Updated weights for policy 1, policy_version 68000 (0.0008) [2023-10-14 20:28:18,343][60425] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 139427840. Throughput: 0: 1665.5, 1: 1659.9. Samples: 34856342. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:28:18,344][60425] Avg episode reward: [(0, '74.600'), (1, '73.090')] [2023-10-14 20:28:21,487][61552] Updated weights for policy 0, policy_version 68162 (0.0009) [2023-10-14 20:28:21,591][61585] Updated weights for policy 1, policy_version 68010 (0.0008) [2023-10-14 20:28:21,902][61552] Updated weights for policy 0, policy_version 68172 (0.0009) [2023-10-14 20:28:21,958][61585] Updated weights for policy 1, policy_version 68020 (0.0009) [2023-10-14 20:28:22,267][61552] Updated weights for policy 0, policy_version 68182 (0.0009) [2023-10-14 20:28:22,324][61585] Updated weights for policy 1, policy_version 68030 (0.0008) [2023-10-14 20:28:22,630][61552] Updated weights for policy 0, policy_version 68192 (0.0008) [2023-10-14 20:28:23,343][60425] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 139493376. Throughput: 0: 1653.3, 1: 1660.0. Samples: 34876084. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:28:23,344][60425] Avg episode reward: [(0, '76.540'), (1, '76.100')] [2023-10-14 20:28:26,424][61585] Updated weights for policy 1, policy_version 68040 (0.0008) [2023-10-14 20:28:26,607][61552] Updated weights for policy 0, policy_version 68202 (0.0010) [2023-10-14 20:28:26,779][61585] Updated weights for policy 1, policy_version 68050 (0.0011) [2023-10-14 20:28:26,977][61552] Updated weights for policy 0, policy_version 68212 (0.0009) [2023-10-14 20:28:27,144][61585] Updated weights for policy 1, policy_version 68060 (0.0008) [2023-10-14 20:28:27,344][61552] Updated weights for policy 0, policy_version 68222 (0.0008) [2023-10-14 20:28:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 139558912. Throughput: 0: 1654.8, 1: 1659.5. Samples: 34895042. Policy #0 lag: (min: 21.0, avg: 26.4, max: 53.0) [2023-10-14 20:28:28,344][60425] Avg episode reward: [(0, '80.710'), (1, '79.020')] [2023-10-14 20:28:31,192][61585] Updated weights for policy 1, policy_version 68070 (0.0008) [2023-10-14 20:28:31,415][61552] Updated weights for policy 0, policy_version 68232 (0.0010) [2023-10-14 20:28:31,551][61585] Updated weights for policy 1, policy_version 68080 (0.0007) [2023-10-14 20:28:31,782][61552] Updated weights for policy 0, policy_version 68242 (0.0009) [2023-10-14 20:28:31,929][61585] Updated weights for policy 1, policy_version 68090 (0.0008) [2023-10-14 20:28:32,153][61552] Updated weights for policy 0, policy_version 68252 (0.0009) [2023-10-14 20:28:33,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 139624448. Throughput: 0: 1662.5, 1: 1676.5. Samples: 34906722. Policy #0 lag: (min: 21.0, avg: 26.4, max: 53.0) [2023-10-14 20:28:33,344][60425] Avg episode reward: [(0, '78.690'), (1, '77.180')] [2023-10-14 20:28:36,025][61585] Updated weights for policy 1, policy_version 68100 (0.0009) [2023-10-14 20:28:36,151][61552] Updated weights for policy 0, policy_version 68262 (0.0007) [2023-10-14 20:28:36,385][61585] Updated weights for policy 1, policy_version 68110 (0.0008) [2023-10-14 20:28:36,518][61552] Updated weights for policy 0, policy_version 68272 (0.0008) [2023-10-14 20:28:36,745][61585] Updated weights for policy 1, policy_version 68120 (0.0008) [2023-10-14 20:28:36,882][61552] Updated weights for policy 0, policy_version 68282 (0.0009) [2023-10-14 20:28:38,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 139689984. Throughput: 0: 1651.5, 1: 1659.2. Samples: 34925796. Policy #0 lag: (min: 21.0, avg: 26.4, max: 53.0) [2023-10-14 20:28:38,345][60425] Avg episode reward: [(0, '80.610'), (1, '75.290')] [2023-10-14 20:28:40,956][61552] Updated weights for policy 0, policy_version 68292 (0.0007) [2023-10-14 20:28:41,024][61585] Updated weights for policy 1, policy_version 68130 (0.0008) [2023-10-14 20:28:41,326][61552] Updated weights for policy 0, policy_version 68302 (0.0008) [2023-10-14 20:28:41,391][61585] Updated weights for policy 1, policy_version 68140 (0.0007) [2023-10-14 20:28:41,699][61552] Updated weights for policy 0, policy_version 68312 (0.0010) [2023-10-14 20:28:41,759][61585] Updated weights for policy 1, policy_version 68150 (0.0007) [2023-10-14 20:28:42,118][61585] Updated weights for policy 1, policy_version 68160 (0.0008) [2023-10-14 20:28:43,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 139755520. Throughput: 0: 1668.7, 1: 1673.0. Samples: 34945646. Policy #0 lag: (min: 21.0, avg: 26.4, max: 53.0) [2023-10-14 20:28:43,345][60425] Avg episode reward: [(0, '78.740'), (1, '76.500')] [2023-10-14 20:28:45,684][61552] Updated weights for policy 0, policy_version 68322 (0.0009) [2023-10-14 20:28:46,048][61552] Updated weights for policy 0, policy_version 68332 (0.0009) [2023-10-14 20:28:46,236][61585] Updated weights for policy 1, policy_version 68170 (0.0008) [2023-10-14 20:28:46,430][61552] Updated weights for policy 0, policy_version 68342 (0.0008) [2023-10-14 20:28:46,607][61585] Updated weights for policy 1, policy_version 68180 (0.0009) [2023-10-14 20:28:46,794][61552] Updated weights for policy 0, policy_version 68352 (0.0008) [2023-10-14 20:28:46,966][61585] Updated weights for policy 1, policy_version 68190 (0.0009) [2023-10-14 20:28:48,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 139821056. Throughput: 0: 1666.1, 1: 1676.3. Samples: 34957012. Policy #0 lag: (min: 21.0, avg: 26.4, max: 53.0) [2023-10-14 20:28:48,344][60425] Avg episode reward: [(0, '79.030'), (1, '82.480')] [2023-10-14 20:28:48,345][61248] Saving new best policy, reward=82.480! [2023-10-14 20:28:51,074][61552] Updated weights for policy 0, policy_version 68362 (0.0007) [2023-10-14 20:28:51,167][61585] Updated weights for policy 1, policy_version 68200 (0.0008) [2023-10-14 20:28:51,431][61552] Updated weights for policy 0, policy_version 68372 (0.0010) [2023-10-14 20:28:51,521][61585] Updated weights for policy 1, policy_version 68210 (0.0007) [2023-10-14 20:28:51,797][61552] Updated weights for policy 0, policy_version 68382 (0.0008) [2023-10-14 20:28:51,891][61585] Updated weights for policy 1, policy_version 68220 (0.0007) [2023-10-14 20:28:53,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 139886592. Throughput: 0: 1652.2, 1: 1660.5. Samples: 34975606. Policy #0 lag: (min: 21.0, avg: 26.4, max: 53.0) [2023-10-14 20:28:53,344][60425] Avg episode reward: [(0, '79.780'), (1, '80.020')] [2023-10-14 20:28:55,828][61552] Updated weights for policy 0, policy_version 68392 (0.0009) [2023-10-14 20:28:55,861][61585] Updated weights for policy 1, policy_version 68230 (0.0007) [2023-10-14 20:28:56,198][61552] Updated weights for policy 0, policy_version 68402 (0.0008) [2023-10-14 20:28:56,227][61585] Updated weights for policy 1, policy_version 68240 (0.0009) [2023-10-14 20:28:56,565][61552] Updated weights for policy 0, policy_version 68412 (0.0008) [2023-10-14 20:28:56,588][61585] Updated weights for policy 1, policy_version 68250 (0.0009) [2023-10-14 20:28:58,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 139952128. Throughput: 0: 1672.1, 1: 1679.7. Samples: 34995688. Policy #0 lag: (min: 21.0, avg: 26.4, max: 53.0) [2023-10-14 20:28:58,344][60425] Avg episode reward: [(0, '76.610'), (1, '77.200')] [2023-10-14 20:29:00,650][61552] Updated weights for policy 0, policy_version 68422 (0.0007) [2023-10-14 20:29:00,710][61585] Updated weights for policy 1, policy_version 68260 (0.0008) [2023-10-14 20:29:01,008][61552] Updated weights for policy 0, policy_version 68432 (0.0007) [2023-10-14 20:29:01,105][61585] Updated weights for policy 1, policy_version 68270 (0.0008) [2023-10-14 20:29:01,379][61552] Updated weights for policy 0, policy_version 68442 (0.0009) [2023-10-14 20:29:01,467][61585] Updated weights for policy 1, policy_version 68280 (0.0010) [2023-10-14 20:29:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 140017664. Throughput: 0: 1666.6, 1: 1677.5. Samples: 35006824. Policy #0 lag: (min: 21.0, avg: 26.4, max: 53.0) [2023-10-14 20:29:03,344][60425] Avg episode reward: [(0, '79.680'), (1, '78.020')] [2023-10-14 20:29:05,494][61585] Updated weights for policy 1, policy_version 68290 (0.0009) [2023-10-14 20:29:05,510][61552] Updated weights for policy 0, policy_version 68452 (0.0008) [2023-10-14 20:29:05,861][61585] Updated weights for policy 1, policy_version 68300 (0.0008) [2023-10-14 20:29:05,881][61552] Updated weights for policy 0, policy_version 68462 (0.0008) [2023-10-14 20:29:06,212][61585] Updated weights for policy 1, policy_version 68310 (0.0007) [2023-10-14 20:29:06,244][61552] Updated weights for policy 0, policy_version 68472 (0.0009) [2023-10-14 20:29:06,575][61585] Updated weights for policy 1, policy_version 68320 (0.0008) [2023-10-14 20:29:08,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 140083200. Throughput: 0: 1656.9, 1: 1661.4. Samples: 35025410. Policy #0 lag: (min: 21.0, avg: 26.4, max: 53.0) [2023-10-14 20:29:08,344][60425] Avg episode reward: [(0, '77.180'), (1, '80.910')] [2023-10-14 20:29:10,352][61552] Updated weights for policy 0, policy_version 68482 (0.0008) [2023-10-14 20:29:10,760][61552] Updated weights for policy 0, policy_version 68492 (0.0007) [2023-10-14 20:29:10,798][61585] Updated weights for policy 1, policy_version 68330 (0.0008) [2023-10-14 20:29:11,128][61552] Updated weights for policy 0, policy_version 68502 (0.0008) [2023-10-14 20:29:11,162][61585] Updated weights for policy 1, policy_version 68340 (0.0008) [2023-10-14 20:29:11,497][61552] Updated weights for policy 0, policy_version 68512 (0.0009) [2023-10-14 20:29:11,524][61585] Updated weights for policy 1, policy_version 68350 (0.0008) [2023-10-14 20:29:13,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 140148736. Throughput: 0: 1677.7, 1: 1675.1. Samples: 35045916. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-10-14 20:29:13,344][60425] Avg episode reward: [(0, '80.140'), (1, '76.790')] [2023-10-14 20:29:15,613][61552] Updated weights for policy 0, policy_version 68522 (0.0010) [2023-10-14 20:29:15,652][61585] Updated weights for policy 1, policy_version 68360 (0.0009) [2023-10-14 20:29:15,983][61552] Updated weights for policy 0, policy_version 68532 (0.0008) [2023-10-14 20:29:16,010][61585] Updated weights for policy 1, policy_version 68370 (0.0010) [2023-10-14 20:29:16,351][61552] Updated weights for policy 0, policy_version 68542 (0.0008) [2023-10-14 20:29:16,373][61585] Updated weights for policy 1, policy_version 68380 (0.0008) [2023-10-14 20:29:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 140214272. Throughput: 0: 1665.7, 1: 1662.4. Samples: 35056486. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-10-14 20:29:18,345][60425] Avg episode reward: [(0, '76.100'), (1, '80.980')] [2023-10-14 20:29:20,453][61585] Updated weights for policy 1, policy_version 68390 (0.0010) [2023-10-14 20:29:20,666][61552] Updated weights for policy 0, policy_version 68552 (0.0008) [2023-10-14 20:29:20,813][61585] Updated weights for policy 1, policy_version 68400 (0.0009) [2023-10-14 20:29:21,040][61552] Updated weights for policy 0, policy_version 68562 (0.0007) [2023-10-14 20:29:21,170][61585] Updated weights for policy 1, policy_version 68410 (0.0009) [2023-10-14 20:29:21,405][61552] Updated weights for policy 0, policy_version 68572 (0.0008) [2023-10-14 20:29:23,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 140279808. Throughput: 0: 1662.0, 1: 1663.5. Samples: 35075442. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-10-14 20:29:23,344][60425] Avg episode reward: [(0, '77.530'), (1, '76.730')] [2023-10-14 20:29:25,362][61585] Updated weights for policy 1, policy_version 68420 (0.0009) [2023-10-14 20:29:25,621][61552] Updated weights for policy 0, policy_version 68582 (0.0008) [2023-10-14 20:29:25,717][61585] Updated weights for policy 1, policy_version 68430 (0.0007) [2023-10-14 20:29:25,988][61552] Updated weights for policy 0, policy_version 68592 (0.0009) [2023-10-14 20:29:26,090][61585] Updated weights for policy 1, policy_version 68440 (0.0007) [2023-10-14 20:29:26,355][61552] Updated weights for policy 0, policy_version 68602 (0.0010) [2023-10-14 20:29:28,344][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 140345344. Throughput: 0: 1668.7, 1: 1670.8. Samples: 35095924. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-10-14 20:29:28,345][60425] Avg episode reward: [(0, '76.070'), (1, '76.970')] [2023-10-14 20:29:30,084][61585] Updated weights for policy 1, policy_version 68450 (0.0007) [2023-10-14 20:29:30,287][61552] Updated weights for policy 0, policy_version 68612 (0.0008) [2023-10-14 20:29:30,454][61585] Updated weights for policy 1, policy_version 68460 (0.0008) [2023-10-14 20:29:30,658][61552] Updated weights for policy 0, policy_version 68622 (0.0008) [2023-10-14 20:29:30,815][61585] Updated weights for policy 1, policy_version 68470 (0.0008) [2023-10-14 20:29:31,026][61552] Updated weights for policy 0, policy_version 68632 (0.0008) [2023-10-14 20:29:31,178][61585] Updated weights for policy 1, policy_version 68480 (0.0008) [2023-10-14 20:29:33,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 140410880. Throughput: 0: 1659.9, 1: 1655.5. Samples: 35106204. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-10-14 20:29:33,344][60425] Avg episode reward: [(0, '79.480'), (1, '75.360')] [2023-10-14 20:29:35,152][61552] Updated weights for policy 0, policy_version 68642 (0.0008) [2023-10-14 20:29:35,249][61585] Updated weights for policy 1, policy_version 68490 (0.0008) [2023-10-14 20:29:35,522][61552] Updated weights for policy 0, policy_version 68652 (0.0007) [2023-10-14 20:29:35,605][61585] Updated weights for policy 1, policy_version 68500 (0.0008) [2023-10-14 20:29:35,886][61552] Updated weights for policy 0, policy_version 68662 (0.0009) [2023-10-14 20:29:35,971][61585] Updated weights for policy 1, policy_version 68510 (0.0010) [2023-10-14 20:29:36,243][61552] Updated weights for policy 0, policy_version 68672 (0.0009) [2023-10-14 20:29:38,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 140476416. Throughput: 0: 1668.2, 1: 1665.9. Samples: 35125640. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-10-14 20:29:38,344][60425] Avg episode reward: [(0, '78.660'), (1, '77.790')] [2023-10-14 20:29:39,993][61585] Updated weights for policy 1, policy_version 68520 (0.0008) [2023-10-14 20:29:40,352][61585] Updated weights for policy 1, policy_version 68530 (0.0007) [2023-10-14 20:29:40,424][61552] Updated weights for policy 0, policy_version 68682 (0.0009) [2023-10-14 20:29:40,722][61585] Updated weights for policy 1, policy_version 68540 (0.0007) [2023-10-14 20:29:40,800][61552] Updated weights for policy 0, policy_version 68692 (0.0008) [2023-10-14 20:29:41,169][61552] Updated weights for policy 0, policy_version 68702 (0.0008) [2023-10-14 20:29:43,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 140541952. Throughput: 0: 1669.0, 1: 1677.2. Samples: 35146270. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-10-14 20:29:43,344][60425] Avg episode reward: [(0, '80.800'), (1, '79.070')] [2023-10-14 20:29:43,353][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000068544_70189056.pth... [2023-10-14 20:29:43,354][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000068704_70352896.pth... [2023-10-14 20:29:43,383][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000066976_68583424.pth [2023-10-14 20:29:43,389][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000067136_68747264.pth [2023-10-14 20:29:44,778][61585] Updated weights for policy 1, policy_version 68550 (0.0007) [2023-10-14 20:29:45,158][61585] Updated weights for policy 1, policy_version 68560 (0.0008) [2023-10-14 20:29:45,331][61552] Updated weights for policy 0, policy_version 68712 (0.0007) [2023-10-14 20:29:45,524][61585] Updated weights for policy 1, policy_version 68570 (0.0008) [2023-10-14 20:29:45,699][61552] Updated weights for policy 0, policy_version 68722 (0.0007) [2023-10-14 20:29:46,066][61552] Updated weights for policy 0, policy_version 68732 (0.0009) [2023-10-14 20:29:48,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 140607488. Throughput: 0: 1654.9, 1: 1658.8. Samples: 35155938. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-10-14 20:29:48,344][60425] Avg episode reward: [(0, '79.640'), (1, '78.160')] [2023-10-14 20:29:49,538][61585] Updated weights for policy 1, policy_version 68580 (0.0009) [2023-10-14 20:29:49,912][61585] Updated weights for policy 1, policy_version 68590 (0.0008) [2023-10-14 20:29:50,230][61552] Updated weights for policy 0, policy_version 68742 (0.0008) [2023-10-14 20:29:50,272][61585] Updated weights for policy 1, policy_version 68600 (0.0008) [2023-10-14 20:29:50,598][61552] Updated weights for policy 0, policy_version 68752 (0.0008) [2023-10-14 20:29:50,971][61552] Updated weights for policy 0, policy_version 68762 (0.0010) [2023-10-14 20:29:53,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 140673024. Throughput: 0: 1661.4, 1: 1675.3. Samples: 35175564. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-10-14 20:29:53,344][60425] Avg episode reward: [(0, '76.380'), (1, '74.960')] [2023-10-14 20:29:54,481][61585] Updated weights for policy 1, policy_version 68610 (0.0010) [2023-10-14 20:29:54,906][61585] Updated weights for policy 1, policy_version 68620 (0.0008) [2023-10-14 20:29:55,101][61552] Updated weights for policy 0, policy_version 68772 (0.0010) [2023-10-14 20:29:55,273][61585] Updated weights for policy 1, policy_version 68630 (0.0009) [2023-10-14 20:29:55,485][61552] Updated weights for policy 0, policy_version 68782 (0.0008) [2023-10-14 20:29:55,642][61585] Updated weights for policy 1, policy_version 68640 (0.0009) [2023-10-14 20:29:55,846][61552] Updated weights for policy 0, policy_version 68792 (0.0007) [2023-10-14 20:29:58,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 140738560. Throughput: 0: 1661.9, 1: 1672.0. Samples: 35195940. Policy #0 lag: (min: 30.0, avg: 54.2, max: 56.0) [2023-10-14 20:29:58,344][60425] Avg episode reward: [(0, '85.580'), (1, '75.920')] [2023-10-14 20:29:58,354][61172] Saving new best policy, reward=85.580! [2023-10-14 20:29:59,747][61552] Updated weights for policy 0, policy_version 68802 (0.0008) [2023-10-14 20:29:59,887][61585] Updated weights for policy 1, policy_version 68650 (0.0008) [2023-10-14 20:30:00,113][61552] Updated weights for policy 0, policy_version 68812 (0.0009) [2023-10-14 20:30:00,256][61585] Updated weights for policy 1, policy_version 68660 (0.0008) [2023-10-14 20:30:00,485][61552] Updated weights for policy 0, policy_version 68822 (0.0010) [2023-10-14 20:30:00,614][61585] Updated weights for policy 1, policy_version 68670 (0.0009) [2023-10-14 20:30:00,846][61552] Updated weights for policy 0, policy_version 68832 (0.0011) [2023-10-14 20:30:03,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 140804096. Throughput: 0: 1650.9, 1: 1653.2. Samples: 35205166. Policy #0 lag: (min: 30.0, avg: 54.2, max: 56.0) [2023-10-14 20:30:03,344][60425] Avg episode reward: [(0, '81.040'), (1, '75.870')] [2023-10-14 20:30:04,935][61585] Updated weights for policy 1, policy_version 68680 (0.0008) [2023-10-14 20:30:05,093][61552] Updated weights for policy 0, policy_version 68842 (0.0010) [2023-10-14 20:30:05,298][61585] Updated weights for policy 1, policy_version 68690 (0.0009) [2023-10-14 20:30:05,458][61552] Updated weights for policy 0, policy_version 68852 (0.0009) [2023-10-14 20:30:05,671][61585] Updated weights for policy 1, policy_version 68700 (0.0008) [2023-10-14 20:30:05,810][61552] Updated weights for policy 0, policy_version 68862 (0.0007) [2023-10-14 20:30:08,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 140869632. Throughput: 0: 1659.3, 1: 1670.7. Samples: 35225292. Policy #0 lag: (min: 30.0, avg: 54.2, max: 56.0) [2023-10-14 20:30:08,344][60425] Avg episode reward: [(0, '78.640'), (1, '75.820')] [2023-10-14 20:30:09,804][61585] Updated weights for policy 1, policy_version 68710 (0.0008) [2023-10-14 20:30:09,971][61552] Updated weights for policy 0, policy_version 68872 (0.0009) [2023-10-14 20:30:10,167][61585] Updated weights for policy 1, policy_version 68720 (0.0007) [2023-10-14 20:30:10,334][61552] Updated weights for policy 0, policy_version 68882 (0.0009) [2023-10-14 20:30:10,529][61585] Updated weights for policy 1, policy_version 68730 (0.0007) [2023-10-14 20:30:10,701][61552] Updated weights for policy 0, policy_version 68892 (0.0008) [2023-10-14 20:30:13,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 140935168. Throughput: 0: 1656.9, 1: 1671.5. Samples: 35245704. Policy #0 lag: (min: 30.0, avg: 54.2, max: 56.0) [2023-10-14 20:30:13,345][60425] Avg episode reward: [(0, '81.530'), (1, '75.560')] [2023-10-14 20:30:14,555][61585] Updated weights for policy 1, policy_version 68740 (0.0009) [2023-10-14 20:30:14,896][61552] Updated weights for policy 0, policy_version 68902 (0.0008) [2023-10-14 20:30:14,926][61585] Updated weights for policy 1, policy_version 68750 (0.0010) [2023-10-14 20:30:15,273][61552] Updated weights for policy 0, policy_version 68912 (0.0009) [2023-10-14 20:30:15,292][61585] Updated weights for policy 1, policy_version 68760 (0.0008) [2023-10-14 20:30:15,645][61552] Updated weights for policy 0, policy_version 68922 (0.0009) [2023-10-14 20:30:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 141000704. Throughput: 0: 1645.7, 1: 1659.1. Samples: 35254918. Policy #0 lag: (min: 30.0, avg: 54.2, max: 56.0) [2023-10-14 20:30:18,344][60425] Avg episode reward: [(0, '82.960'), (1, '82.200')] [2023-10-14 20:30:19,354][61585] Updated weights for policy 1, policy_version 68770 (0.0009) [2023-10-14 20:30:19,659][61552] Updated weights for policy 0, policy_version 68932 (0.0010) [2023-10-14 20:30:19,724][61585] Updated weights for policy 1, policy_version 68780 (0.0008) [2023-10-14 20:30:20,028][61552] Updated weights for policy 0, policy_version 68942 (0.0008) [2023-10-14 20:30:20,092][61585] Updated weights for policy 1, policy_version 68790 (0.0007) [2023-10-14 20:30:20,389][61552] Updated weights for policy 0, policy_version 68952 (0.0008) [2023-10-14 20:30:20,446][61585] Updated weights for policy 1, policy_version 68800 (0.0007) [2023-10-14 20:30:23,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 141066240. Throughput: 0: 1656.6, 1: 1664.6. Samples: 35275094. Policy #0 lag: (min: 30.0, avg: 54.2, max: 56.0) [2023-10-14 20:30:23,344][60425] Avg episode reward: [(0, '77.950'), (1, '75.630')] [2023-10-14 20:30:24,409][61552] Updated weights for policy 0, policy_version 68962 (0.0008) [2023-10-14 20:30:24,697][61585] Updated weights for policy 1, policy_version 68810 (0.0008) [2023-10-14 20:30:24,778][61552] Updated weights for policy 0, policy_version 68972 (0.0008) [2023-10-14 20:30:25,065][61585] Updated weights for policy 1, policy_version 68820 (0.0009) [2023-10-14 20:30:25,152][61552] Updated weights for policy 0, policy_version 68982 (0.0008) [2023-10-14 20:30:25,430][61585] Updated weights for policy 1, policy_version 68830 (0.0007) [2023-10-14 20:30:25,520][61552] Updated weights for policy 0, policy_version 68992 (0.0007) [2023-10-14 20:30:28,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 141131776. Throughput: 0: 1659.8, 1: 1656.0. Samples: 35295482. Policy #0 lag: (min: 30.0, avg: 54.2, max: 56.0) [2023-10-14 20:30:28,344][60425] Avg episode reward: [(0, '81.670'), (1, '78.660')] [2023-10-14 20:30:29,365][61585] Updated weights for policy 1, policy_version 68840 (0.0007) [2023-10-14 20:30:29,665][61552] Updated weights for policy 0, policy_version 69002 (0.0008) [2023-10-14 20:30:29,725][61585] Updated weights for policy 1, policy_version 68850 (0.0009) [2023-10-14 20:30:30,027][61552] Updated weights for policy 0, policy_version 69012 (0.0008) [2023-10-14 20:30:30,093][61585] Updated weights for policy 1, policy_version 68860 (0.0007) [2023-10-14 20:30:30,399][61552] Updated weights for policy 0, policy_version 69022 (0.0007) [2023-10-14 20:30:33,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 141197312. Throughput: 0: 1650.7, 1: 1655.1. Samples: 35304698. Policy #0 lag: (min: 30.0, avg: 54.2, max: 56.0) [2023-10-14 20:30:33,344][60425] Avg episode reward: [(0, '82.400'), (1, '75.100')] [2023-10-14 20:30:34,283][61585] Updated weights for policy 1, policy_version 68870 (0.0009) [2023-10-14 20:30:34,564][61552] Updated weights for policy 0, policy_version 69032 (0.0009) [2023-10-14 20:30:34,640][61585] Updated weights for policy 1, policy_version 68880 (0.0008) [2023-10-14 20:30:34,921][61552] Updated weights for policy 0, policy_version 69042 (0.0007) [2023-10-14 20:30:35,009][61585] Updated weights for policy 1, policy_version 68890 (0.0008) [2023-10-14 20:30:35,289][61552] Updated weights for policy 0, policy_version 69052 (0.0008) [2023-10-14 20:30:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 141262848. Throughput: 0: 1663.0, 1: 1653.8. Samples: 35324818. Policy #0 lag: (min: 30.0, avg: 54.2, max: 56.0) [2023-10-14 20:30:38,344][60425] Avg episode reward: [(0, '77.920'), (1, '76.030')] [2023-10-14 20:30:39,290][61585] Updated weights for policy 1, policy_version 68900 (0.0007) [2023-10-14 20:30:39,475][61552] Updated weights for policy 0, policy_version 69062 (0.0009) [2023-10-14 20:30:39,690][61585] Updated weights for policy 1, policy_version 68910 (0.0008) [2023-10-14 20:30:39,841][61552] Updated weights for policy 0, policy_version 69072 (0.0009) [2023-10-14 20:30:40,056][61585] Updated weights for policy 1, policy_version 68920 (0.0008) [2023-10-14 20:30:40,209][61552] Updated weights for policy 0, policy_version 69082 (0.0008) [2023-10-14 20:30:43,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 141328384. Throughput: 0: 1658.7, 1: 1657.7. Samples: 35345180. Policy #0 lag: (min: 31.0, avg: 31.5, max: 47.0) [2023-10-14 20:30:43,344][60425] Avg episode reward: [(0, '78.230'), (1, '77.590')] [2023-10-14 20:30:43,959][61585] Updated weights for policy 1, policy_version 68930 (0.0007) [2023-10-14 20:30:44,338][61585] Updated weights for policy 1, policy_version 68940 (0.0007) [2023-10-14 20:30:44,482][61552] Updated weights for policy 0, policy_version 69092 (0.0009) [2023-10-14 20:30:44,692][61585] Updated weights for policy 1, policy_version 68950 (0.0008) [2023-10-14 20:30:44,852][61552] Updated weights for policy 0, policy_version 69102 (0.0009) [2023-10-14 20:30:45,058][61585] Updated weights for policy 1, policy_version 68960 (0.0008) [2023-10-14 20:30:45,216][61552] Updated weights for policy 0, policy_version 69112 (0.0009) [2023-10-14 20:30:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 141393920. Throughput: 0: 1653.6, 1: 1657.6. Samples: 35354170. Policy #0 lag: (min: 31.0, avg: 31.5, max: 47.0) [2023-10-14 20:30:48,344][60425] Avg episode reward: [(0, '82.480'), (1, '78.680')] [2023-10-14 20:30:49,183][61585] Updated weights for policy 1, policy_version 68970 (0.0008) [2023-10-14 20:30:49,395][61552] Updated weights for policy 0, policy_version 69122 (0.0009) [2023-10-14 20:30:49,544][61585] Updated weights for policy 1, policy_version 68980 (0.0008) [2023-10-14 20:30:49,761][61552] Updated weights for policy 0, policy_version 69132 (0.0009) [2023-10-14 20:30:49,906][61585] Updated weights for policy 1, policy_version 68990 (0.0008) [2023-10-14 20:30:50,120][61552] Updated weights for policy 0, policy_version 69142 (0.0010) [2023-10-14 20:30:50,489][61552] Updated weights for policy 0, policy_version 69152 (0.0009) [2023-10-14 20:30:53,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 141459456. Throughput: 0: 1659.6, 1: 1662.8. Samples: 35374802. Policy #0 lag: (min: 31.0, avg: 31.5, max: 47.0) [2023-10-14 20:30:53,344][60425] Avg episode reward: [(0, '80.340'), (1, '77.500')] [2023-10-14 20:30:53,877][61585] Updated weights for policy 1, policy_version 69000 (0.0009) [2023-10-14 20:30:54,245][61585] Updated weights for policy 1, policy_version 69010 (0.0008) [2023-10-14 20:30:54,418][61552] Updated weights for policy 0, policy_version 69162 (0.0009) [2023-10-14 20:30:54,604][61585] Updated weights for policy 1, policy_version 69020 (0.0007) [2023-10-14 20:30:54,776][61552] Updated weights for policy 0, policy_version 69172 (0.0009) [2023-10-14 20:30:55,142][61552] Updated weights for policy 0, policy_version 69182 (0.0009) [2023-10-14 20:30:58,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 141524992. Throughput: 0: 1669.4, 1: 1663.4. Samples: 35395680. Policy #0 lag: (min: 31.0, avg: 31.5, max: 47.0) [2023-10-14 20:30:58,344][60425] Avg episode reward: [(0, '82.930'), (1, '80.150')] [2023-10-14 20:30:58,786][61585] Updated weights for policy 1, policy_version 69030 (0.0007) [2023-10-14 20:30:59,144][61585] Updated weights for policy 1, policy_version 69040 (0.0007) [2023-10-14 20:30:59,343][61552] Updated weights for policy 0, policy_version 69192 (0.0009) [2023-10-14 20:30:59,511][61585] Updated weights for policy 1, policy_version 69050 (0.0008) [2023-10-14 20:30:59,703][61552] Updated weights for policy 0, policy_version 69202 (0.0008) [2023-10-14 20:31:00,072][61552] Updated weights for policy 0, policy_version 69212 (0.0011) [2023-10-14 20:31:03,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 141590528. Throughput: 0: 1661.6, 1: 1665.1. Samples: 35404622. Policy #0 lag: (min: 31.0, avg: 31.5, max: 47.0) [2023-10-14 20:31:03,344][60425] Avg episode reward: [(0, '80.920'), (1, '79.530')] [2023-10-14 20:31:03,620][61585] Updated weights for policy 1, policy_version 69060 (0.0008) [2023-10-14 20:31:03,991][61585] Updated weights for policy 1, policy_version 69070 (0.0008) [2023-10-14 20:31:04,199][61552] Updated weights for policy 0, policy_version 69222 (0.0010) [2023-10-14 20:31:04,365][61585] Updated weights for policy 1, policy_version 69080 (0.0009) [2023-10-14 20:31:04,574][61552] Updated weights for policy 0, policy_version 69232 (0.0007) [2023-10-14 20:31:04,942][61552] Updated weights for policy 0, policy_version 69242 (0.0010) [2023-10-14 20:31:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 141656064. Throughput: 0: 1661.8, 1: 1672.8. Samples: 35425152. Policy #0 lag: (min: 31.0, avg: 31.5, max: 47.0) [2023-10-14 20:31:08,344][60425] Avg episode reward: [(0, '77.120'), (1, '73.810')] [2023-10-14 20:31:08,441][61585] Updated weights for policy 1, policy_version 69090 (0.0008) [2023-10-14 20:31:08,818][61585] Updated weights for policy 1, policy_version 69100 (0.0009) [2023-10-14 20:31:09,057][61552] Updated weights for policy 0, policy_version 69252 (0.0007) [2023-10-14 20:31:09,182][61585] Updated weights for policy 1, policy_version 69110 (0.0008) [2023-10-14 20:31:09,430][61552] Updated weights for policy 0, policy_version 69262 (0.0007) [2023-10-14 20:31:09,543][61585] Updated weights for policy 1, policy_version 69120 (0.0008) [2023-10-14 20:31:09,791][61552] Updated weights for policy 0, policy_version 69272 (0.0008) [2023-10-14 20:31:13,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 141721600. Throughput: 0: 1660.0, 1: 1677.8. Samples: 35445686. Policy #0 lag: (min: 31.0, avg: 31.5, max: 47.0) [2023-10-14 20:31:13,344][60425] Avg episode reward: [(0, '76.690'), (1, '83.580')] [2023-10-14 20:31:13,701][61585] Updated weights for policy 1, policy_version 69130 (0.0009) [2023-10-14 20:31:13,816][61552] Updated weights for policy 0, policy_version 69282 (0.0009) [2023-10-14 20:31:14,068][61585] Updated weights for policy 1, policy_version 69140 (0.0008) [2023-10-14 20:31:14,186][61552] Updated weights for policy 0, policy_version 69292 (0.0009) [2023-10-14 20:31:14,435][61585] Updated weights for policy 1, policy_version 69150 (0.0007) [2023-10-14 20:31:14,501][61248] Saving new best policy, reward=83.580! [2023-10-14 20:31:14,560][61552] Updated weights for policy 0, policy_version 69302 (0.0010) [2023-10-14 20:31:14,913][61552] Updated weights for policy 0, policy_version 69312 (0.0007) [2023-10-14 20:31:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 141787136. Throughput: 0: 1658.7, 1: 1676.4. Samples: 35454780. Policy #0 lag: (min: 31.0, avg: 31.5, max: 47.0) [2023-10-14 20:31:18,344][60425] Avg episode reward: [(0, '81.040'), (1, '83.600')] [2023-10-14 20:31:18,534][61585] Updated weights for policy 1, policy_version 69160 (0.0009) [2023-10-14 20:31:18,902][61585] Updated weights for policy 1, policy_version 69170 (0.0008) [2023-10-14 20:31:19,032][61552] Updated weights for policy 0, policy_version 69322 (0.0009) [2023-10-14 20:31:19,261][61585] Updated weights for policy 1, policy_version 69180 (0.0009) [2023-10-14 20:31:19,401][61248] Saving new best policy, reward=83.600! [2023-10-14 20:31:19,410][61552] Updated weights for policy 0, policy_version 69332 (0.0009) [2023-10-14 20:31:19,778][61552] Updated weights for policy 0, policy_version 69342 (0.0011) [2023-10-14 20:31:23,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 141852672. Throughput: 0: 1657.6, 1: 1680.8. Samples: 35475046. Policy #0 lag: (min: 31.0, avg: 31.5, max: 47.0) [2023-10-14 20:31:23,344][60425] Avg episode reward: [(0, '82.600'), (1, '77.590')] [2023-10-14 20:31:23,446][61585] Updated weights for policy 1, policy_version 69190 (0.0008) [2023-10-14 20:31:23,815][61585] Updated weights for policy 1, policy_version 69200 (0.0009) [2023-10-14 20:31:23,967][61552] Updated weights for policy 0, policy_version 69352 (0.0008) [2023-10-14 20:31:24,175][61585] Updated weights for policy 1, policy_version 69210 (0.0007) [2023-10-14 20:31:24,330][61552] Updated weights for policy 0, policy_version 69362 (0.0009) [2023-10-14 20:31:24,709][61552] Updated weights for policy 0, policy_version 69372 (0.0008) [2023-10-14 20:31:28,306][61585] Updated weights for policy 1, policy_version 69220 (0.0011) [2023-10-14 20:31:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 141918208. Throughput: 0: 1659.1, 1: 1681.1. Samples: 35495490. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-14 20:31:28,344][60425] Avg episode reward: [(0, '75.790'), (1, '79.650')] [2023-10-14 20:31:28,712][61585] Updated weights for policy 1, policy_version 69230 (0.0009) [2023-10-14 20:31:29,001][61552] Updated weights for policy 0, policy_version 69382 (0.0008) [2023-10-14 20:31:29,080][61585] Updated weights for policy 1, policy_version 69240 (0.0008) [2023-10-14 20:31:29,398][61552] Updated weights for policy 0, policy_version 69392 (0.0008) [2023-10-14 20:31:29,764][61552] Updated weights for policy 0, policy_version 69402 (0.0008) [2023-10-14 20:31:33,239][61585] Updated weights for policy 1, policy_version 69250 (0.0008) [2023-10-14 20:31:33,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 141983744. Throughput: 0: 1662.4, 1: 1675.6. Samples: 35504376. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-14 20:31:33,344][60425] Avg episode reward: [(0, '77.180'), (1, '75.230')] [2023-10-14 20:31:33,609][61585] Updated weights for policy 1, policy_version 69260 (0.0009) [2023-10-14 20:31:33,848][61552] Updated weights for policy 0, policy_version 69412 (0.0008) [2023-10-14 20:31:33,969][61585] Updated weights for policy 1, policy_version 69270 (0.0009) [2023-10-14 20:31:34,214][61552] Updated weights for policy 0, policy_version 69422 (0.0009) [2023-10-14 20:31:34,329][61585] Updated weights for policy 1, policy_version 69280 (0.0008) [2023-10-14 20:31:34,590][61552] Updated weights for policy 0, policy_version 69432 (0.0009) [2023-10-14 20:31:38,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 142049280. Throughput: 0: 1662.3, 1: 1672.4. Samples: 35524866. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-14 20:31:38,344][60425] Avg episode reward: [(0, '78.180'), (1, '77.110')] [2023-10-14 20:31:38,527][61585] Updated weights for policy 1, policy_version 69290 (0.0010) [2023-10-14 20:31:38,739][61552] Updated weights for policy 0, policy_version 69442 (0.0009) [2023-10-14 20:31:38,895][61585] Updated weights for policy 1, policy_version 69300 (0.0007) [2023-10-14 20:31:39,106][61552] Updated weights for policy 0, policy_version 69452 (0.0007) [2023-10-14 20:31:39,258][61585] Updated weights for policy 1, policy_version 69310 (0.0007) [2023-10-14 20:31:39,472][61552] Updated weights for policy 0, policy_version 69462 (0.0007) [2023-10-14 20:31:39,835][61552] Updated weights for policy 0, policy_version 69472 (0.0008) [2023-10-14 20:31:43,170][61585] Updated weights for policy 1, policy_version 69320 (0.0009) [2023-10-14 20:31:43,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 142114816. Throughput: 0: 1656.6, 1: 1674.2. Samples: 35545564. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-14 20:31:43,345][60425] Avg episode reward: [(0, '80.480'), (1, '78.380')] [2023-10-14 20:31:43,355][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000069472_71139328.pth... [2023-10-14 20:31:43,387][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000067936_69566464.pth [2023-10-14 20:31:43,523][61585] Updated weights for policy 1, policy_version 69330 (0.0009) [2023-10-14 20:31:43,891][61552] Updated weights for policy 0, policy_version 69482 (0.0009) [2023-10-14 20:31:43,892][61585] Updated weights for policy 1, policy_version 69340 (0.0007) [2023-10-14 20:31:44,028][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000069344_71008256.pth... [2023-10-14 20:31:44,056][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000067776_69402624.pth [2023-10-14 20:31:44,257][61552] Updated weights for policy 0, policy_version 69492 (0.0007) [2023-10-14 20:31:44,633][61552] Updated weights for policy 0, policy_version 69502 (0.0010) [2023-10-14 20:31:48,126][61585] Updated weights for policy 1, policy_version 69350 (0.0009) [2023-10-14 20:31:48,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 142180352. Throughput: 0: 1657.9, 1: 1673.2. Samples: 35554520. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-14 20:31:48,344][60425] Avg episode reward: [(0, '77.590'), (1, '79.980')] [2023-10-14 20:31:48,495][61585] Updated weights for policy 1, policy_version 69360 (0.0010) [2023-10-14 20:31:48,706][61552] Updated weights for policy 0, policy_version 69512 (0.0009) [2023-10-14 20:31:48,867][61585] Updated weights for policy 1, policy_version 69370 (0.0008) [2023-10-14 20:31:49,070][61552] Updated weights for policy 0, policy_version 69522 (0.0007) [2023-10-14 20:31:49,439][61552] Updated weights for policy 0, policy_version 69532 (0.0009) [2023-10-14 20:31:52,887][61585] Updated weights for policy 1, policy_version 69380 (0.0008) [2023-10-14 20:31:53,251][61585] Updated weights for policy 1, policy_version 69390 (0.0007) [2023-10-14 20:31:53,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 142245888. Throughput: 0: 1664.1, 1: 1668.7. Samples: 35575130. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-14 20:31:53,344][60425] Avg episode reward: [(0, '82.640'), (1, '71.330')] [2023-10-14 20:31:53,455][61552] Updated weights for policy 0, policy_version 69542 (0.0008) [2023-10-14 20:31:53,616][61585] Updated weights for policy 1, policy_version 69400 (0.0007) [2023-10-14 20:31:53,817][61552] Updated weights for policy 0, policy_version 69552 (0.0009) [2023-10-14 20:31:54,195][61552] Updated weights for policy 0, policy_version 69562 (0.0009) [2023-10-14 20:31:57,515][61585] Updated weights for policy 1, policy_version 69410 (0.0008) [2023-10-14 20:31:57,875][61585] Updated weights for policy 1, policy_version 69420 (0.0008) [2023-10-14 20:31:58,241][61585] Updated weights for policy 1, policy_version 69430 (0.0008) [2023-10-14 20:31:58,270][61552] Updated weights for policy 0, policy_version 69572 (0.0009) [2023-10-14 20:31:58,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 142311424. Throughput: 0: 1669.0, 1: 1665.0. Samples: 35595718. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-14 20:31:58,345][60425] Avg episode reward: [(0, '74.950'), (1, '75.960')] [2023-10-14 20:31:58,602][61585] Updated weights for policy 1, policy_version 69440 (0.0008) [2023-10-14 20:31:58,631][61552] Updated weights for policy 0, policy_version 69582 (0.0009) [2023-10-14 20:31:58,999][61552] Updated weights for policy 0, policy_version 69592 (0.0010) [2023-10-14 20:32:02,832][61585] Updated weights for policy 1, policy_version 69450 (0.0009) [2023-10-14 20:32:03,104][61552] Updated weights for policy 0, policy_version 69602 (0.0010) [2023-10-14 20:32:03,192][61585] Updated weights for policy 1, policy_version 69460 (0.0008) [2023-10-14 20:32:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 142376960. Throughput: 0: 1671.8, 1: 1668.1. Samples: 35605074. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-14 20:32:03,344][60425] Avg episode reward: [(0, '78.150'), (1, '75.910')] [2023-10-14 20:32:03,467][61552] Updated weights for policy 0, policy_version 69612 (0.0007) [2023-10-14 20:32:03,559][61585] Updated weights for policy 1, policy_version 69470 (0.0007) [2023-10-14 20:32:03,843][61552] Updated weights for policy 0, policy_version 69622 (0.0008) [2023-10-14 20:32:04,207][61552] Updated weights for policy 0, policy_version 69632 (0.0008) [2023-10-14 20:32:07,574][61585] Updated weights for policy 1, policy_version 69480 (0.0010) [2023-10-14 20:32:07,945][61585] Updated weights for policy 1, policy_version 69490 (0.0009) [2023-10-14 20:32:08,307][61585] Updated weights for policy 1, policy_version 69500 (0.0009) [2023-10-14 20:32:08,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 142442496. Throughput: 0: 1675.7, 1: 1672.7. Samples: 35625722. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-14 20:32:08,344][60425] Avg episode reward: [(0, '80.860'), (1, '79.070')] [2023-10-14 20:32:08,359][61552] Updated weights for policy 0, policy_version 69642 (0.0007) [2023-10-14 20:32:08,725][61552] Updated weights for policy 0, policy_version 69652 (0.0009) [2023-10-14 20:32:09,091][61552] Updated weights for policy 0, policy_version 69662 (0.0011) [2023-10-14 20:32:12,474][61585] Updated weights for policy 1, policy_version 69510 (0.0010) [2023-10-14 20:32:12,836][61585] Updated weights for policy 1, policy_version 69520 (0.0009) [2023-10-14 20:32:13,192][61552] Updated weights for policy 0, policy_version 69672 (0.0009) [2023-10-14 20:32:13,206][61585] Updated weights for policy 1, policy_version 69530 (0.0009) [2023-10-14 20:32:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 142508032. Throughput: 0: 1678.5, 1: 1659.3. Samples: 35645690. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 20:32:13,344][60425] Avg episode reward: [(0, '77.740'), (1, '80.300')] [2023-10-14 20:32:13,562][61552] Updated weights for policy 0, policy_version 69682 (0.0008) [2023-10-14 20:32:13,930][61552] Updated weights for policy 0, policy_version 69692 (0.0008) [2023-10-14 20:32:17,435][61585] Updated weights for policy 1, policy_version 69540 (0.0009) [2023-10-14 20:32:17,794][61585] Updated weights for policy 1, policy_version 69550 (0.0008) [2023-10-14 20:32:18,082][61552] Updated weights for policy 0, policy_version 69702 (0.0008) [2023-10-14 20:32:18,156][61585] Updated weights for policy 1, policy_version 69560 (0.0007) [2023-10-14 20:32:18,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 142573568. Throughput: 0: 1676.2, 1: 1675.4. Samples: 35655200. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 20:32:18,344][60425] Avg episode reward: [(0, '74.950'), (1, '77.520')] [2023-10-14 20:32:18,471][61552] Updated weights for policy 0, policy_version 69712 (0.0007) [2023-10-14 20:32:18,845][61552] Updated weights for policy 0, policy_version 69722 (0.0008) [2023-10-14 20:32:22,197][61585] Updated weights for policy 1, policy_version 69570 (0.0007) [2023-10-14 20:32:22,567][61585] Updated weights for policy 1, policy_version 69580 (0.0008) [2023-10-14 20:32:22,831][61552] Updated weights for policy 0, policy_version 69732 (0.0009) [2023-10-14 20:32:22,932][61585] Updated weights for policy 1, policy_version 69590 (0.0010) [2023-10-14 20:32:23,209][61552] Updated weights for policy 0, policy_version 69742 (0.0007) [2023-10-14 20:32:23,292][61585] Updated weights for policy 1, policy_version 69600 (0.0009) [2023-10-14 20:32:23,343][60425] Fps is (10 sec: 16384.4, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 142671872. Throughput: 0: 1675.3, 1: 1676.2. Samples: 35675684. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 20:32:23,344][60425] Avg episode reward: [(0, '76.570'), (1, '75.150')] [2023-10-14 20:32:23,585][61552] Updated weights for policy 0, policy_version 69752 (0.0007) [2023-10-14 20:32:27,520][61585] Updated weights for policy 1, policy_version 69610 (0.0011) [2023-10-14 20:32:27,700][61552] Updated weights for policy 0, policy_version 69762 (0.0007) [2023-10-14 20:32:27,890][61585] Updated weights for policy 1, policy_version 69620 (0.0009) [2023-10-14 20:32:28,066][61552] Updated weights for policy 0, policy_version 69772 (0.0007) [2023-10-14 20:32:28,251][61585] Updated weights for policy 1, policy_version 69630 (0.0008) [2023-10-14 20:32:28,344][60425] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 142737408. Throughput: 0: 1673.7, 1: 1657.2. Samples: 35695454. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 20:32:28,345][60425] Avg episode reward: [(0, '74.770'), (1, '76.580')] [2023-10-14 20:32:28,435][61552] Updated weights for policy 0, policy_version 69782 (0.0009) [2023-10-14 20:32:28,800][61552] Updated weights for policy 0, policy_version 69792 (0.0007) [2023-10-14 20:32:32,300][61585] Updated weights for policy 1, policy_version 69640 (0.0007) [2023-10-14 20:32:32,663][61585] Updated weights for policy 1, policy_version 69650 (0.0009) [2023-10-14 20:32:32,819][61552] Updated weights for policy 0, policy_version 69802 (0.0009) [2023-10-14 20:32:33,032][61585] Updated weights for policy 1, policy_version 69660 (0.0008) [2023-10-14 20:32:33,197][61552] Updated weights for policy 0, policy_version 69812 (0.0008) [2023-10-14 20:32:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 142802944. Throughput: 0: 1679.7, 1: 1671.3. Samples: 35705318. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 20:32:33,344][60425] Avg episode reward: [(0, '71.620'), (1, '77.230')] [2023-10-14 20:32:33,559][61552] Updated weights for policy 0, policy_version 69822 (0.0008) [2023-10-14 20:32:37,147][61585] Updated weights for policy 1, policy_version 69670 (0.0007) [2023-10-14 20:32:37,509][61585] Updated weights for policy 1, policy_version 69680 (0.0008) [2023-10-14 20:32:37,645][61552] Updated weights for policy 0, policy_version 69832 (0.0008) [2023-10-14 20:32:37,879][61585] Updated weights for policy 1, policy_version 69690 (0.0009) [2023-10-14 20:32:38,011][61552] Updated weights for policy 0, policy_version 69842 (0.0008) [2023-10-14 20:32:38,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 142868480. Throughput: 0: 1677.8, 1: 1669.6. Samples: 35725762. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 20:32:38,344][60425] Avg episode reward: [(0, '73.720'), (1, '73.990')] [2023-10-14 20:32:38,386][61552] Updated weights for policy 0, policy_version 69852 (0.0007) [2023-10-14 20:32:42,007][61585] Updated weights for policy 1, policy_version 69700 (0.0009) [2023-10-14 20:32:42,344][61552] Updated weights for policy 0, policy_version 69862 (0.0007) [2023-10-14 20:32:42,360][61585] Updated weights for policy 1, policy_version 69710 (0.0009) [2023-10-14 20:32:42,712][61552] Updated weights for policy 0, policy_version 69872 (0.0008) [2023-10-14 20:32:42,723][61585] Updated weights for policy 1, policy_version 69720 (0.0008) [2023-10-14 20:32:43,067][61552] Updated weights for policy 0, policy_version 69882 (0.0007) [2023-10-14 20:32:43,343][60425] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 13329.3). Total num frames: 142966784. Throughput: 0: 1663.2, 1: 1651.4. Samples: 35744876. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 20:32:43,344][60425] Avg episode reward: [(0, '74.010'), (1, '71.260')] [2023-10-14 20:32:46,815][61585] Updated weights for policy 1, policy_version 69730 (0.0008) [2023-10-14 20:32:47,185][61585] Updated weights for policy 1, policy_version 69740 (0.0008) [2023-10-14 20:32:47,336][61552] Updated weights for policy 0, policy_version 69892 (0.0008) [2023-10-14 20:32:47,553][61585] Updated weights for policy 1, policy_version 69750 (0.0009) [2023-10-14 20:32:47,713][61552] Updated weights for policy 0, policy_version 69902 (0.0009) [2023-10-14 20:32:47,915][61585] Updated weights for policy 1, policy_version 69760 (0.0009) [2023-10-14 20:32:48,080][61552] Updated weights for policy 0, policy_version 69912 (0.0009) [2023-10-14 20:32:48,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 142999552. Throughput: 0: 1676.0, 1: 1666.5. Samples: 35755488. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 20:32:48,344][60425] Avg episode reward: [(0, '74.400'), (1, '71.910')] [2023-10-14 20:32:52,023][61585] Updated weights for policy 1, policy_version 69770 (0.0009) [2023-10-14 20:32:52,079][61552] Updated weights for policy 0, policy_version 69922 (0.0009) [2023-10-14 20:32:52,382][61585] Updated weights for policy 1, policy_version 69780 (0.0009) [2023-10-14 20:32:52,457][61552] Updated weights for policy 0, policy_version 69932 (0.0008) [2023-10-14 20:32:52,748][61585] Updated weights for policy 1, policy_version 69790 (0.0008) [2023-10-14 20:32:52,816][61552] Updated weights for policy 0, policy_version 69942 (0.0007) [2023-10-14 20:32:53,183][61552] Updated weights for policy 0, policy_version 69952 (0.0007) [2023-10-14 20:32:53,343][60425] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 143097856. Throughput: 0: 1672.5, 1: 1657.6. Samples: 35775580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:32:53,344][60425] Avg episode reward: [(0, '74.430'), (1, '71.750')] [2023-10-14 20:32:56,935][61585] Updated weights for policy 1, policy_version 69800 (0.0009) [2023-10-14 20:32:57,309][61585] Updated weights for policy 1, policy_version 69810 (0.0008) [2023-10-14 20:32:57,339][61552] Updated weights for policy 0, policy_version 69962 (0.0009) [2023-10-14 20:32:57,679][61585] Updated weights for policy 1, policy_version 69820 (0.0009) [2023-10-14 20:32:57,694][61552] Updated weights for policy 0, policy_version 69972 (0.0008) [2023-10-14 20:32:58,062][61552] Updated weights for policy 0, policy_version 69982 (0.0010) [2023-10-14 20:32:58,343][60425] Fps is (10 sec: 16383.6, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 143163392. Throughput: 0: 1654.3, 1: 1647.1. Samples: 35794252. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:32:58,344][60425] Avg episode reward: [(0, '80.990'), (1, '72.870')] [2023-10-14 20:33:01,787][61585] Updated weights for policy 1, policy_version 69830 (0.0009) [2023-10-14 20:33:02,156][61585] Updated weights for policy 1, policy_version 69840 (0.0008) [2023-10-14 20:33:02,364][61552] Updated weights for policy 0, policy_version 69992 (0.0009) [2023-10-14 20:33:02,515][61585] Updated weights for policy 1, policy_version 69850 (0.0008) [2023-10-14 20:33:02,727][61552] Updated weights for policy 0, policy_version 70002 (0.0008) [2023-10-14 20:33:03,086][61552] Updated weights for policy 0, policy_version 70012 (0.0010) [2023-10-14 20:33:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 143228928. Throughput: 0: 1669.9, 1: 1665.2. Samples: 35805280. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:33:03,344][60425] Avg episode reward: [(0, '78.760'), (1, '74.710')] [2023-10-14 20:33:06,705][61585] Updated weights for policy 1, policy_version 69860 (0.0009) [2023-10-14 20:33:07,092][61585] Updated weights for policy 1, policy_version 69870 (0.0008) [2023-10-14 20:33:07,295][61552] Updated weights for policy 0, policy_version 70022 (0.0009) [2023-10-14 20:33:07,458][61585] Updated weights for policy 1, policy_version 69880 (0.0008) [2023-10-14 20:33:07,671][61552] Updated weights for policy 0, policy_version 70032 (0.0009) [2023-10-14 20:33:08,037][61552] Updated weights for policy 0, policy_version 70042 (0.0009) [2023-10-14 20:33:08,343][60425] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 143294464. Throughput: 0: 1671.8, 1: 1655.9. Samples: 35825432. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:33:08,344][60425] Avg episode reward: [(0, '79.540'), (1, '73.570')] [2023-10-14 20:33:11,583][61585] Updated weights for policy 1, policy_version 69890 (0.0008) [2023-10-14 20:33:11,920][61552] Updated weights for policy 0, policy_version 70052 (0.0007) [2023-10-14 20:33:11,949][61585] Updated weights for policy 1, policy_version 69900 (0.0009) [2023-10-14 20:33:12,283][61552] Updated weights for policy 0, policy_version 70062 (0.0007) [2023-10-14 20:33:12,317][61585] Updated weights for policy 1, policy_version 69910 (0.0007) [2023-10-14 20:33:12,659][61552] Updated weights for policy 0, policy_version 70072 (0.0007) [2023-10-14 20:33:12,676][61585] Updated weights for policy 1, policy_version 69920 (0.0007) [2023-10-14 20:33:13,343][60425] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 143360000. Throughput: 0: 1657.8, 1: 1648.1. Samples: 35844218. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:33:13,344][60425] Avg episode reward: [(0, '80.050'), (1, '69.660')] [2023-10-14 20:33:16,591][61552] Updated weights for policy 0, policy_version 70082 (0.0008) [2023-10-14 20:33:16,716][61585] Updated weights for policy 1, policy_version 69930 (0.0010) [2023-10-14 20:33:16,970][61552] Updated weights for policy 0, policy_version 70092 (0.0009) [2023-10-14 20:33:17,080][61585] Updated weights for policy 1, policy_version 69940 (0.0009) [2023-10-14 20:33:17,343][61552] Updated weights for policy 0, policy_version 70102 (0.0008) [2023-10-14 20:33:17,435][61585] Updated weights for policy 1, policy_version 69950 (0.0008) [2023-10-14 20:33:17,706][61552] Updated weights for policy 0, policy_version 70112 (0.0008) [2023-10-14 20:33:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 143425536. Throughput: 0: 1675.7, 1: 1667.3. Samples: 35855756. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:33:18,344][60425] Avg episode reward: [(0, '80.880'), (1, '74.990')] [2023-10-14 20:33:21,458][61585] Updated weights for policy 1, policy_version 69960 (0.0009) [2023-10-14 20:33:21,801][61552] Updated weights for policy 0, policy_version 70122 (0.0008) [2023-10-14 20:33:21,829][61585] Updated weights for policy 1, policy_version 69970 (0.0008) [2023-10-14 20:33:22,166][61552] Updated weights for policy 0, policy_version 70132 (0.0008) [2023-10-14 20:33:22,197][61585] Updated weights for policy 1, policy_version 69980 (0.0007) [2023-10-14 20:33:22,524][61552] Updated weights for policy 0, policy_version 70142 (0.0007) [2023-10-14 20:33:23,343][60425] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 143491072. Throughput: 0: 1665.0, 1: 1659.2. Samples: 35875350. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:33:23,344][60425] Avg episode reward: [(0, '81.730'), (1, '76.500')] [2023-10-14 20:33:26,369][61585] Updated weights for policy 1, policy_version 69990 (0.0007) [2023-10-14 20:33:26,589][61552] Updated weights for policy 0, policy_version 70152 (0.0008) [2023-10-14 20:33:26,737][61585] Updated weights for policy 1, policy_version 70000 (0.0010) [2023-10-14 20:33:26,966][61552] Updated weights for policy 0, policy_version 70162 (0.0010) [2023-10-14 20:33:27,097][61585] Updated weights for policy 1, policy_version 70010 (0.0008) [2023-10-14 20:33:27,333][61552] Updated weights for policy 0, policy_version 70172 (0.0008) [2023-10-14 20:33:28,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 143556608. Throughput: 0: 1656.6, 1: 1662.2. Samples: 35894224. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:33:28,344][60425] Avg episode reward: [(0, '78.650'), (1, '70.910')] [2023-10-14 20:33:31,198][61585] Updated weights for policy 1, policy_version 70020 (0.0008) [2023-10-14 20:33:31,425][61552] Updated weights for policy 0, policy_version 70182 (0.0008) [2023-10-14 20:33:31,562][61585] Updated weights for policy 1, policy_version 70030 (0.0007) [2023-10-14 20:33:31,798][61552] Updated weights for policy 0, policy_version 70192 (0.0007) [2023-10-14 20:33:31,929][61585] Updated weights for policy 1, policy_version 70040 (0.0009) [2023-10-14 20:33:32,176][61552] Updated weights for policy 0, policy_version 70202 (0.0007) [2023-10-14 20:33:33,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 143622144. Throughput: 0: 1670.3, 1: 1668.8. Samples: 35905748. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:33:33,344][60425] Avg episode reward: [(0, '78.080'), (1, '74.930')] [2023-10-14 20:33:35,988][61585] Updated weights for policy 1, policy_version 70050 (0.0009) [2023-10-14 20:33:36,124][61552] Updated weights for policy 0, policy_version 70212 (0.0007) [2023-10-14 20:33:36,349][61585] Updated weights for policy 1, policy_version 70060 (0.0007) [2023-10-14 20:33:36,483][61552] Updated weights for policy 0, policy_version 70222 (0.0010) [2023-10-14 20:33:36,702][61585] Updated weights for policy 1, policy_version 70070 (0.0009) [2023-10-14 20:33:36,861][61552] Updated weights for policy 0, policy_version 70232 (0.0010) [2023-10-14 20:33:37,070][61585] Updated weights for policy 1, policy_version 70080 (0.0009) [2023-10-14 20:33:38,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 143687680. Throughput: 0: 1663.2, 1: 1659.7. Samples: 35925114. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-14 20:33:38,344][60425] Avg episode reward: [(0, '75.050'), (1, '75.770')] [2023-10-14 20:33:41,072][61552] Updated weights for policy 0, policy_version 70242 (0.0009) [2023-10-14 20:33:41,328][61585] Updated weights for policy 1, policy_version 70090 (0.0009) [2023-10-14 20:33:41,438][61552] Updated weights for policy 0, policy_version 70252 (0.0009) [2023-10-14 20:33:41,691][61585] Updated weights for policy 1, policy_version 70100 (0.0008) [2023-10-14 20:33:41,815][61552] Updated weights for policy 0, policy_version 70262 (0.0009) [2023-10-14 20:33:42,055][61585] Updated weights for policy 1, policy_version 70110 (0.0008) [2023-10-14 20:33:42,184][61552] Updated weights for policy 0, policy_version 70272 (0.0008) [2023-10-14 20:33:43,344][60425] Fps is (10 sec: 13106.7, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 143753216. Throughput: 0: 1666.0, 1: 1673.9. Samples: 35944548. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-14 20:33:43,345][60425] Avg episode reward: [(0, '76.960'), (1, '77.750')] [2023-10-14 20:33:43,357][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000070272_71958528.pth... [2023-10-14 20:33:43,357][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000070112_71794688.pth... [2023-10-14 20:33:43,395][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000068544_70189056.pth [2023-10-14 20:33:43,396][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000068704_70352896.pth [2023-10-14 20:33:43,400][61248] Saving a milestone ./train_atari/atari_roadrunner_APPO/checkpoint_p1/milestones/checkpoint_000070112_71794688.pth [2023-10-14 20:33:43,400][61172] Saving a milestone ./train_atari/atari_roadrunner_APPO/checkpoint_p0/milestones/checkpoint_000070272_71958528.pth [2023-10-14 20:33:46,050][61585] Updated weights for policy 1, policy_version 70120 (0.0010) [2023-10-14 20:33:46,407][61585] Updated weights for policy 1, policy_version 70130 (0.0007) [2023-10-14 20:33:46,458][61552] Updated weights for policy 0, policy_version 70282 (0.0007) [2023-10-14 20:33:46,765][61585] Updated weights for policy 1, policy_version 70140 (0.0010) [2023-10-14 20:33:46,830][61552] Updated weights for policy 0, policy_version 70292 (0.0008) [2023-10-14 20:33:47,195][61552] Updated weights for policy 0, policy_version 70302 (0.0011) [2023-10-14 20:33:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 143818752. Throughput: 0: 1678.2, 1: 1676.6. Samples: 35956244. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-14 20:33:48,344][60425] Avg episode reward: [(0, '75.530'), (1, '75.320')] [2023-10-14 20:33:51,062][61585] Updated weights for policy 1, policy_version 70150 (0.0010) [2023-10-14 20:33:51,438][61585] Updated weights for policy 1, policy_version 70160 (0.0009) [2023-10-14 20:33:51,460][61552] Updated weights for policy 0, policy_version 70312 (0.0008) [2023-10-14 20:33:51,802][61585] Updated weights for policy 1, policy_version 70170 (0.0008) [2023-10-14 20:33:51,829][61552] Updated weights for policy 0, policy_version 70322 (0.0007) [2023-10-14 20:33:52,199][61552] Updated weights for policy 0, policy_version 70332 (0.0008) [2023-10-14 20:33:53,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 143884288. Throughput: 0: 1661.3, 1: 1658.8. Samples: 35974838. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-14 20:33:53,344][60425] Avg episode reward: [(0, '78.780'), (1, '81.370')] [2023-10-14 20:33:56,010][61585] Updated weights for policy 1, policy_version 70180 (0.0008) [2023-10-14 20:33:56,243][61552] Updated weights for policy 0, policy_version 70342 (0.0008) [2023-10-14 20:33:56,377][61585] Updated weights for policy 1, policy_version 70190 (0.0008) [2023-10-14 20:33:56,600][61552] Updated weights for policy 0, policy_version 70352 (0.0009) [2023-10-14 20:33:56,744][61585] Updated weights for policy 1, policy_version 70200 (0.0009) [2023-10-14 20:33:56,968][61552] Updated weights for policy 0, policy_version 70362 (0.0007) [2023-10-14 20:33:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 143949824. Throughput: 0: 1662.2, 1: 1673.4. Samples: 35994320. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-14 20:33:58,344][60425] Avg episode reward: [(0, '74.930'), (1, '76.630')] [2023-10-14 20:34:00,655][61585] Updated weights for policy 1, policy_version 70210 (0.0008) [2023-10-14 20:34:00,903][61552] Updated weights for policy 0, policy_version 70372 (0.0007) [2023-10-14 20:34:01,027][61585] Updated weights for policy 1, policy_version 70220 (0.0009) [2023-10-14 20:34:01,269][61552] Updated weights for policy 0, policy_version 70382 (0.0010) [2023-10-14 20:34:01,386][61585] Updated weights for policy 1, policy_version 70230 (0.0009) [2023-10-14 20:34:01,640][61552] Updated weights for policy 0, policy_version 70392 (0.0009) [2023-10-14 20:34:01,743][61585] Updated weights for policy 1, policy_version 70240 (0.0008) [2023-10-14 20:34:03,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 144015360. Throughput: 0: 1671.1, 1: 1665.3. Samples: 36005896. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-14 20:34:03,344][60425] Avg episode reward: [(0, '75.180'), (1, '76.360')] [2023-10-14 20:34:05,794][61552] Updated weights for policy 0, policy_version 70402 (0.0009) [2023-10-14 20:34:05,839][61585] Updated weights for policy 1, policy_version 70250 (0.0007) [2023-10-14 20:34:06,165][61552] Updated weights for policy 0, policy_version 70412 (0.0009) [2023-10-14 20:34:06,203][61585] Updated weights for policy 1, policy_version 70260 (0.0007) [2023-10-14 20:34:06,525][61552] Updated weights for policy 0, policy_version 70422 (0.0009) [2023-10-14 20:34:06,563][61585] Updated weights for policy 1, policy_version 70270 (0.0010) [2023-10-14 20:34:06,885][61552] Updated weights for policy 0, policy_version 70432 (0.0011) [2023-10-14 20:34:08,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 144080896. Throughput: 0: 1660.5, 1: 1650.1. Samples: 36024326. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-14 20:34:08,344][60425] Avg episode reward: [(0, '78.060'), (1, '77.320')] [2023-10-14 20:34:10,611][61585] Updated weights for policy 1, policy_version 70280 (0.0008) [2023-10-14 20:34:10,901][61552] Updated weights for policy 0, policy_version 70442 (0.0009) [2023-10-14 20:34:10,972][61585] Updated weights for policy 1, policy_version 70290 (0.0010) [2023-10-14 20:34:11,274][61552] Updated weights for policy 0, policy_version 70452 (0.0009) [2023-10-14 20:34:11,328][61585] Updated weights for policy 1, policy_version 70300 (0.0009) [2023-10-14 20:34:11,640][61552] Updated weights for policy 0, policy_version 70462 (0.0007) [2023-10-14 20:34:13,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 144146432. Throughput: 0: 1673.8, 1: 1671.1. Samples: 36044748. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-14 20:34:13,344][60425] Avg episode reward: [(0, '74.980'), (1, '72.760')] [2023-10-14 20:34:15,335][61585] Updated weights for policy 1, policy_version 70310 (0.0008) [2023-10-14 20:34:15,689][61585] Updated weights for policy 1, policy_version 70320 (0.0007) [2023-10-14 20:34:15,826][61552] Updated weights for policy 0, policy_version 70472 (0.0007) [2023-10-14 20:34:16,049][61585] Updated weights for policy 1, policy_version 70330 (0.0010) [2023-10-14 20:34:16,183][61552] Updated weights for policy 0, policy_version 70482 (0.0008) [2023-10-14 20:34:16,560][61552] Updated weights for policy 0, policy_version 70492 (0.0008) [2023-10-14 20:34:18,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 144211968. Throughput: 0: 1668.7, 1: 1660.6. Samples: 36055568. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-14 20:34:18,344][60425] Avg episode reward: [(0, '73.510'), (1, '78.100')] [2023-10-14 20:34:20,277][61585] Updated weights for policy 1, policy_version 70340 (0.0009) [2023-10-14 20:34:20,639][61585] Updated weights for policy 1, policy_version 70350 (0.0011) [2023-10-14 20:34:20,696][61552] Updated weights for policy 0, policy_version 70502 (0.0009) [2023-10-14 20:34:21,012][61585] Updated weights for policy 1, policy_version 70360 (0.0008) [2023-10-14 20:34:21,058][61552] Updated weights for policy 0, policy_version 70512 (0.0008) [2023-10-14 20:34:21,418][61552] Updated weights for policy 0, policy_version 70522 (0.0008) [2023-10-14 20:34:23,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 144277504. Throughput: 0: 1657.5, 1: 1661.7. Samples: 36074480. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-14 20:34:23,344][60425] Avg episode reward: [(0, '77.430'), (1, '73.570')] [2023-10-14 20:34:25,099][61585] Updated weights for policy 1, policy_version 70370 (0.0010) [2023-10-14 20:34:25,469][61585] Updated weights for policy 1, policy_version 70380 (0.0011) [2023-10-14 20:34:25,563][61552] Updated weights for policy 0, policy_version 70532 (0.0008) [2023-10-14 20:34:25,832][61585] Updated weights for policy 1, policy_version 70390 (0.0009) [2023-10-14 20:34:25,917][61552] Updated weights for policy 0, policy_version 70542 (0.0007) [2023-10-14 20:34:26,188][61585] Updated weights for policy 1, policy_version 70400 (0.0008) [2023-10-14 20:34:26,283][61552] Updated weights for policy 0, policy_version 70552 (0.0008) [2023-10-14 20:34:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 144343040. Throughput: 0: 1668.6, 1: 1672.0. Samples: 36094878. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-14 20:34:28,344][60425] Avg episode reward: [(0, '79.250'), (1, '78.600')] [2023-10-14 20:34:30,376][61552] Updated weights for policy 0, policy_version 70562 (0.0008) [2023-10-14 20:34:30,472][61585] Updated weights for policy 1, policy_version 70410 (0.0008) [2023-10-14 20:34:30,742][61552] Updated weights for policy 0, policy_version 70572 (0.0010) [2023-10-14 20:34:30,831][61585] Updated weights for policy 1, policy_version 70420 (0.0008) [2023-10-14 20:34:31,118][61552] Updated weights for policy 0, policy_version 70582 (0.0009) [2023-10-14 20:34:31,191][61585] Updated weights for policy 1, policy_version 70430 (0.0007) [2023-10-14 20:34:31,484][61552] Updated weights for policy 0, policy_version 70592 (0.0009) [2023-10-14 20:34:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 144408576. Throughput: 0: 1655.6, 1: 1654.2. Samples: 36105186. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-14 20:34:33,344][60425] Avg episode reward: [(0, '76.780'), (1, '76.440')] [2023-10-14 20:34:35,208][61585] Updated weights for policy 1, policy_version 70440 (0.0008) [2023-10-14 20:34:35,572][61585] Updated weights for policy 1, policy_version 70450 (0.0007) [2023-10-14 20:34:35,630][61552] Updated weights for policy 0, policy_version 70602 (0.0007) [2023-10-14 20:34:35,936][61585] Updated weights for policy 1, policy_version 70460 (0.0008) [2023-10-14 20:34:35,984][61552] Updated weights for policy 0, policy_version 70612 (0.0007) [2023-10-14 20:34:36,355][61552] Updated weights for policy 0, policy_version 70622 (0.0009) [2023-10-14 20:34:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 144474112. Throughput: 0: 1657.1, 1: 1667.9. Samples: 36124464. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-14 20:34:38,344][60425] Avg episode reward: [(0, '83.350'), (1, '82.230')] [2023-10-14 20:34:40,189][61585] Updated weights for policy 1, policy_version 70470 (0.0008) [2023-10-14 20:34:40,539][61552] Updated weights for policy 0, policy_version 70632 (0.0008) [2023-10-14 20:34:40,577][61585] Updated weights for policy 1, policy_version 70480 (0.0007) [2023-10-14 20:34:40,924][61552] Updated weights for policy 0, policy_version 70642 (0.0008) [2023-10-14 20:34:40,934][61585] Updated weights for policy 1, policy_version 70490 (0.0007) [2023-10-14 20:34:41,287][61552] Updated weights for policy 0, policy_version 70652 (0.0010) [2023-10-14 20:34:43,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 144539648. Throughput: 0: 1668.2, 1: 1676.4. Samples: 36144826. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-14 20:34:43,344][60425] Avg episode reward: [(0, '80.100'), (1, '77.360')] [2023-10-14 20:34:44,808][61585] Updated weights for policy 1, policy_version 70500 (0.0007) [2023-10-14 20:34:45,177][61585] Updated weights for policy 1, policy_version 70510 (0.0008) [2023-10-14 20:34:45,232][61552] Updated weights for policy 0, policy_version 70662 (0.0008) [2023-10-14 20:34:45,532][61585] Updated weights for policy 1, policy_version 70520 (0.0009) [2023-10-14 20:34:45,593][61552] Updated weights for policy 0, policy_version 70672 (0.0009) [2023-10-14 20:34:45,963][61552] Updated weights for policy 0, policy_version 70682 (0.0008) [2023-10-14 20:34:48,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 144605184. Throughput: 0: 1646.3, 1: 1655.3. Samples: 36154466. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-14 20:34:48,344][60425] Avg episode reward: [(0, '81.950'), (1, '80.900')] [2023-10-14 20:34:49,740][61585] Updated weights for policy 1, policy_version 70530 (0.0009) [2023-10-14 20:34:50,094][61585] Updated weights for policy 1, policy_version 70540 (0.0009) [2023-10-14 20:34:50,202][61552] Updated weights for policy 0, policy_version 70692 (0.0010) [2023-10-14 20:34:50,462][61585] Updated weights for policy 1, policy_version 70550 (0.0007) [2023-10-14 20:34:50,566][61552] Updated weights for policy 0, policy_version 70702 (0.0007) [2023-10-14 20:34:50,822][61585] Updated weights for policy 1, policy_version 70560 (0.0007) [2023-10-14 20:34:50,938][61552] Updated weights for policy 0, policy_version 70712 (0.0008) [2023-10-14 20:34:53,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 144670720. Throughput: 0: 1652.3, 1: 1674.3. Samples: 36174022. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-14 20:34:53,344][60425] Avg episode reward: [(0, '80.280'), (1, '81.150')] [2023-10-14 20:34:55,041][61585] Updated weights for policy 1, policy_version 70570 (0.0008) [2023-10-14 20:34:55,053][61552] Updated weights for policy 0, policy_version 70722 (0.0008) [2023-10-14 20:34:55,411][61585] Updated weights for policy 1, policy_version 70580 (0.0007) [2023-10-14 20:34:55,423][61552] Updated weights for policy 0, policy_version 70732 (0.0007) [2023-10-14 20:34:55,764][61585] Updated weights for policy 1, policy_version 70590 (0.0008) [2023-10-14 20:34:55,791][61552] Updated weights for policy 0, policy_version 70742 (0.0008) [2023-10-14 20:34:56,164][61552] Updated weights for policy 0, policy_version 70752 (0.0010) [2023-10-14 20:34:58,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 144736256. Throughput: 0: 1663.5, 1: 1669.5. Samples: 36194732. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-14 20:34:58,344][60425] Avg episode reward: [(0, '80.720'), (1, '75.630')] [2023-10-14 20:34:59,830][61585] Updated weights for policy 1, policy_version 70600 (0.0008) [2023-10-14 20:35:00,096][61552] Updated weights for policy 0, policy_version 70762 (0.0007) [2023-10-14 20:35:00,192][61585] Updated weights for policy 1, policy_version 70610 (0.0008) [2023-10-14 20:35:00,472][61552] Updated weights for policy 0, policy_version 70772 (0.0007) [2023-10-14 20:35:00,548][61585] Updated weights for policy 1, policy_version 70620 (0.0008) [2023-10-14 20:35:00,831][61552] Updated weights for policy 0, policy_version 70782 (0.0009) [2023-10-14 20:35:03,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 144801792. Throughput: 0: 1650.8, 1: 1654.1. Samples: 36204286. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-14 20:35:03,344][60425] Avg episode reward: [(0, '79.590'), (1, '75.580')] [2023-10-14 20:35:04,658][61585] Updated weights for policy 1, policy_version 70630 (0.0008) [2023-10-14 20:35:05,027][61585] Updated weights for policy 1, policy_version 70640 (0.0009) [2023-10-14 20:35:05,070][61552] Updated weights for policy 0, policy_version 70792 (0.0009) [2023-10-14 20:35:05,391][61585] Updated weights for policy 1, policy_version 70650 (0.0007) [2023-10-14 20:35:05,436][61552] Updated weights for policy 0, policy_version 70802 (0.0010) [2023-10-14 20:35:05,802][61552] Updated weights for policy 0, policy_version 70812 (0.0010) [2023-10-14 20:35:08,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 144867328. Throughput: 0: 1664.1, 1: 1666.9. Samples: 36224378. Policy #0 lag: (min: 26.0, avg: 29.1, max: 58.0) [2023-10-14 20:35:08,344][60425] Avg episode reward: [(0, '78.200'), (1, '72.810')] [2023-10-14 20:35:09,585][61585] Updated weights for policy 1, policy_version 70660 (0.0008) [2023-10-14 20:35:09,805][61552] Updated weights for policy 0, policy_version 70822 (0.0009) [2023-10-14 20:35:09,947][61585] Updated weights for policy 1, policy_version 70670 (0.0009) [2023-10-14 20:35:10,163][61552] Updated weights for policy 0, policy_version 70832 (0.0007) [2023-10-14 20:35:10,314][61585] Updated weights for policy 1, policy_version 70680 (0.0010) [2023-10-14 20:35:10,536][61552] Updated weights for policy 0, policy_version 70842 (0.0008) [2023-10-14 20:35:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 144932864. Throughput: 0: 1672.4, 1: 1665.1. Samples: 36245064. Policy #0 lag: (min: 26.0, avg: 29.1, max: 58.0) [2023-10-14 20:35:13,344][60425] Avg episode reward: [(0, '75.510'), (1, '73.610')] [2023-10-14 20:35:14,538][61585] Updated weights for policy 1, policy_version 70690 (0.0008) [2023-10-14 20:35:14,794][61552] Updated weights for policy 0, policy_version 70852 (0.0009) [2023-10-14 20:35:14,902][61585] Updated weights for policy 1, policy_version 70700 (0.0008) [2023-10-14 20:35:15,161][61552] Updated weights for policy 0, policy_version 70862 (0.0007) [2023-10-14 20:35:15,270][61585] Updated weights for policy 1, policy_version 70710 (0.0008) [2023-10-14 20:35:15,532][61552] Updated weights for policy 0, policy_version 70872 (0.0009) [2023-10-14 20:35:15,630][61585] Updated weights for policy 1, policy_version 70720 (0.0008) [2023-10-14 20:35:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 144998400. Throughput: 0: 1659.0, 1: 1651.2. Samples: 36254146. Policy #0 lag: (min: 26.0, avg: 29.1, max: 58.0) [2023-10-14 20:35:18,344][60425] Avg episode reward: [(0, '76.630'), (1, '70.380')] [2023-10-14 20:35:19,667][61552] Updated weights for policy 0, policy_version 70882 (0.0009) [2023-10-14 20:35:19,740][61585] Updated weights for policy 1, policy_version 70730 (0.0008) [2023-10-14 20:35:20,038][61552] Updated weights for policy 0, policy_version 70892 (0.0009) [2023-10-14 20:35:20,101][61585] Updated weights for policy 1, policy_version 70740 (0.0009) [2023-10-14 20:35:20,406][61552] Updated weights for policy 0, policy_version 70902 (0.0007) [2023-10-14 20:35:20,469][61585] Updated weights for policy 1, policy_version 70750 (0.0009) [2023-10-14 20:35:20,783][61552] Updated weights for policy 0, policy_version 70912 (0.0009) [2023-10-14 20:35:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 145063936. Throughput: 0: 1666.4, 1: 1661.6. Samples: 36274224. Policy #0 lag: (min: 26.0, avg: 29.1, max: 58.0) [2023-10-14 20:35:23,344][60425] Avg episode reward: [(0, '77.910'), (1, '75.760')] [2023-10-14 20:35:24,606][61585] Updated weights for policy 1, policy_version 70760 (0.0009) [2023-10-14 20:35:24,947][61552] Updated weights for policy 0, policy_version 70922 (0.0008) [2023-10-14 20:35:24,968][61585] Updated weights for policy 1, policy_version 70770 (0.0008) [2023-10-14 20:35:25,319][61552] Updated weights for policy 0, policy_version 70932 (0.0008) [2023-10-14 20:35:25,346][61585] Updated weights for policy 1, policy_version 70780 (0.0008) [2023-10-14 20:35:25,681][61552] Updated weights for policy 0, policy_version 70942 (0.0008) [2023-10-14 20:35:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 145129472. Throughput: 0: 1665.2, 1: 1661.8. Samples: 36294540. Policy #0 lag: (min: 26.0, avg: 29.1, max: 58.0) [2023-10-14 20:35:28,344][60425] Avg episode reward: [(0, '74.730'), (1, '72.890')] [2023-10-14 20:35:29,416][61585] Updated weights for policy 1, policy_version 70790 (0.0010) [2023-10-14 20:35:29,776][61585] Updated weights for policy 1, policy_version 70800 (0.0010) [2023-10-14 20:35:30,009][61552] Updated weights for policy 0, policy_version 70952 (0.0008) [2023-10-14 20:35:30,138][61585] Updated weights for policy 1, policy_version 70810 (0.0007) [2023-10-14 20:35:30,382][61552] Updated weights for policy 0, policy_version 70962 (0.0009) [2023-10-14 20:35:30,752][61552] Updated weights for policy 0, policy_version 70972 (0.0011) [2023-10-14 20:35:33,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 145195008. Throughput: 0: 1659.6, 1: 1657.1. Samples: 36303720. Policy #0 lag: (min: 26.0, avg: 29.1, max: 58.0) [2023-10-14 20:35:33,345][60425] Avg episode reward: [(0, '78.740'), (1, '74.780')] [2023-10-14 20:35:34,141][61585] Updated weights for policy 1, policy_version 70820 (0.0009) [2023-10-14 20:35:34,504][61585] Updated weights for policy 1, policy_version 70830 (0.0007) [2023-10-14 20:35:34,760][61552] Updated weights for policy 0, policy_version 70982 (0.0010) [2023-10-14 20:35:34,869][61585] Updated weights for policy 1, policy_version 70840 (0.0009) [2023-10-14 20:35:35,131][61552] Updated weights for policy 0, policy_version 70992 (0.0010) [2023-10-14 20:35:35,497][61552] Updated weights for policy 0, policy_version 71002 (0.0011) [2023-10-14 20:35:38,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 145260544. Throughput: 0: 1671.5, 1: 1662.8. Samples: 36324066. Policy #0 lag: (min: 26.0, avg: 29.1, max: 58.0) [2023-10-14 20:35:38,344][60425] Avg episode reward: [(0, '75.260'), (1, '80.320')] [2023-10-14 20:35:38,958][61585] Updated weights for policy 1, policy_version 70850 (0.0007) [2023-10-14 20:35:39,324][61585] Updated weights for policy 1, policy_version 70860 (0.0009) [2023-10-14 20:35:39,486][61552] Updated weights for policy 0, policy_version 71012 (0.0007) [2023-10-14 20:35:39,683][61585] Updated weights for policy 1, policy_version 70870 (0.0009) [2023-10-14 20:35:39,845][61552] Updated weights for policy 0, policy_version 71022 (0.0009) [2023-10-14 20:35:40,050][61585] Updated weights for policy 1, policy_version 70880 (0.0008) [2023-10-14 20:35:40,210][61552] Updated weights for policy 0, policy_version 71032 (0.0009) [2023-10-14 20:35:43,344][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 145326080. Throughput: 0: 1667.6, 1: 1661.2. Samples: 36344528. Policy #0 lag: (min: 26.0, avg: 29.1, max: 58.0) [2023-10-14 20:35:43,345][60425] Avg episode reward: [(0, '77.840'), (1, '79.750')] [2023-10-14 20:35:43,356][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000071040_72744960.pth... [2023-10-14 20:35:43,357][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000070880_72581120.pth... [2023-10-14 20:35:43,387][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000069472_71139328.pth [2023-10-14 20:35:43,392][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000069344_71008256.pth [2023-10-14 20:35:44,336][61552] Updated weights for policy 0, policy_version 71042 (0.0010) [2023-10-14 20:35:44,391][61585] Updated weights for policy 1, policy_version 70890 (0.0007) [2023-10-14 20:35:44,709][61552] Updated weights for policy 0, policy_version 71052 (0.0007) [2023-10-14 20:35:44,759][61585] Updated weights for policy 1, policy_version 70900 (0.0007) [2023-10-14 20:35:45,070][61552] Updated weights for policy 0, policy_version 71062 (0.0008) [2023-10-14 20:35:45,112][61585] Updated weights for policy 1, policy_version 70910 (0.0009) [2023-10-14 20:35:45,433][61552] Updated weights for policy 0, policy_version 71072 (0.0010) [2023-10-14 20:35:48,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 145391616. Throughput: 0: 1656.6, 1: 1659.0. Samples: 36353488. Policy #0 lag: (min: 26.0, avg: 29.1, max: 58.0) [2023-10-14 20:35:48,344][60425] Avg episode reward: [(0, '76.190'), (1, '76.670')] [2023-10-14 20:35:49,384][61585] Updated weights for policy 1, policy_version 70920 (0.0008) [2023-10-14 20:35:49,680][61552] Updated weights for policy 0, policy_version 71082 (0.0010) [2023-10-14 20:35:49,752][61585] Updated weights for policy 1, policy_version 70930 (0.0009) [2023-10-14 20:35:50,055][61552] Updated weights for policy 0, policy_version 71092 (0.0008) [2023-10-14 20:35:50,118][61585] Updated weights for policy 1, policy_version 70940 (0.0008) [2023-10-14 20:35:50,412][61552] Updated weights for policy 0, policy_version 71102 (0.0008) [2023-10-14 20:35:53,344][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 145457152. Throughput: 0: 1663.0, 1: 1658.1. Samples: 36373828. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-14 20:35:53,345][60425] Avg episode reward: [(0, '79.800'), (1, '78.950')] [2023-10-14 20:35:54,255][61585] Updated weights for policy 1, policy_version 70950 (0.0009) [2023-10-14 20:35:54,595][61552] Updated weights for policy 0, policy_version 71112 (0.0009) [2023-10-14 20:35:54,635][61585] Updated weights for policy 1, policy_version 70960 (0.0008) [2023-10-14 20:35:54,965][61552] Updated weights for policy 0, policy_version 71122 (0.0007) [2023-10-14 20:35:54,997][61585] Updated weights for policy 1, policy_version 70970 (0.0008) [2023-10-14 20:35:55,342][61552] Updated weights for policy 0, policy_version 71132 (0.0008) [2023-10-14 20:35:58,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 145522688. Throughput: 0: 1652.8, 1: 1659.0. Samples: 36394096. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-14 20:35:58,344][60425] Avg episode reward: [(0, '73.420'), (1, '75.300')] [2023-10-14 20:35:59,151][61585] Updated weights for policy 1, policy_version 70980 (0.0007) [2023-10-14 20:35:59,426][61552] Updated weights for policy 0, policy_version 71142 (0.0009) [2023-10-14 20:35:59,513][61585] Updated weights for policy 1, policy_version 70990 (0.0007) [2023-10-14 20:35:59,792][61552] Updated weights for policy 0, policy_version 71152 (0.0010) [2023-10-14 20:35:59,876][61585] Updated weights for policy 1, policy_version 71000 (0.0010) [2023-10-14 20:36:00,163][61552] Updated weights for policy 0, policy_version 71162 (0.0007) [2023-10-14 20:36:03,344][60425] Fps is (10 sec: 13106.7, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 145588224. Throughput: 0: 1648.8, 1: 1661.7. Samples: 36403120. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-14 20:36:03,345][60425] Avg episode reward: [(0, '73.590'), (1, '73.050')] [2023-10-14 20:36:04,015][61585] Updated weights for policy 1, policy_version 71010 (0.0007) [2023-10-14 20:36:04,361][61552] Updated weights for policy 0, policy_version 71172 (0.0009) [2023-10-14 20:36:04,377][61585] Updated weights for policy 1, policy_version 71020 (0.0008) [2023-10-14 20:36:04,724][61552] Updated weights for policy 0, policy_version 71182 (0.0008) [2023-10-14 20:36:04,745][61585] Updated weights for policy 1, policy_version 71030 (0.0008) [2023-10-14 20:36:05,082][61552] Updated weights for policy 0, policy_version 71192 (0.0009) [2023-10-14 20:36:05,106][61585] Updated weights for policy 1, policy_version 71040 (0.0008) [2023-10-14 20:36:08,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 145653760. Throughput: 0: 1655.9, 1: 1662.4. Samples: 36423548. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-14 20:36:08,344][60425] Avg episode reward: [(0, '73.220'), (1, '69.800')] [2023-10-14 20:36:09,181][61585] Updated weights for policy 1, policy_version 71050 (0.0009) [2023-10-14 20:36:09,303][61552] Updated weights for policy 0, policy_version 71202 (0.0009) [2023-10-14 20:36:09,549][61585] Updated weights for policy 1, policy_version 71060 (0.0007) [2023-10-14 20:36:09,663][61552] Updated weights for policy 0, policy_version 71212 (0.0007) [2023-10-14 20:36:09,908][61585] Updated weights for policy 1, policy_version 71070 (0.0009) [2023-10-14 20:36:10,029][61552] Updated weights for policy 0, policy_version 71222 (0.0008) [2023-10-14 20:36:10,402][61552] Updated weights for policy 0, policy_version 71232 (0.0008) [2023-10-14 20:36:13,343][60425] Fps is (10 sec: 13108.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 145719296. Throughput: 0: 1662.0, 1: 1663.3. Samples: 36444176. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-14 20:36:13,344][60425] Avg episode reward: [(0, '74.230'), (1, '73.110')] [2023-10-14 20:36:14,125][61585] Updated weights for policy 1, policy_version 71080 (0.0009) [2023-10-14 20:36:14,502][61585] Updated weights for policy 1, policy_version 71090 (0.0007) [2023-10-14 20:36:14,632][61552] Updated weights for policy 0, policy_version 71242 (0.0009) [2023-10-14 20:36:14,877][61585] Updated weights for policy 1, policy_version 71100 (0.0008) [2023-10-14 20:36:15,002][61552] Updated weights for policy 0, policy_version 71252 (0.0008) [2023-10-14 20:36:15,376][61552] Updated weights for policy 0, policy_version 71262 (0.0007) [2023-10-14 20:36:18,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 145784832. Throughput: 0: 1654.4, 1: 1662.3. Samples: 36452970. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-14 20:36:18,344][60425] Avg episode reward: [(0, '72.860'), (1, '76.910')] [2023-10-14 20:36:18,790][61585] Updated weights for policy 1, policy_version 71110 (0.0007) [2023-10-14 20:36:19,161][61585] Updated weights for policy 1, policy_version 71120 (0.0008) [2023-10-14 20:36:19,299][61552] Updated weights for policy 0, policy_version 71272 (0.0007) [2023-10-14 20:36:19,523][61585] Updated weights for policy 1, policy_version 71130 (0.0007) [2023-10-14 20:36:19,668][61552] Updated weights for policy 0, policy_version 71282 (0.0008) [2023-10-14 20:36:20,047][61552] Updated weights for policy 0, policy_version 71292 (0.0009) [2023-10-14 20:36:23,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 145850368. Throughput: 0: 1658.5, 1: 1662.0. Samples: 36473488. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-14 20:36:23,344][60425] Avg episode reward: [(0, '77.590'), (1, '75.300')] [2023-10-14 20:36:23,643][61585] Updated weights for policy 1, policy_version 71140 (0.0007) [2023-10-14 20:36:24,008][61585] Updated weights for policy 1, policy_version 71150 (0.0011) [2023-10-14 20:36:24,306][61552] Updated weights for policy 0, policy_version 71302 (0.0009) [2023-10-14 20:36:24,377][61585] Updated weights for policy 1, policy_version 71160 (0.0009) [2023-10-14 20:36:24,675][61552] Updated weights for policy 0, policy_version 71312 (0.0009) [2023-10-14 20:36:25,049][61552] Updated weights for policy 0, policy_version 71322 (0.0008) [2023-10-14 20:36:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 145915904. Throughput: 0: 1652.9, 1: 1665.4. Samples: 36493848. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-14 20:36:28,344][60425] Avg episode reward: [(0, '76.940'), (1, '75.000')] [2023-10-14 20:36:28,460][61585] Updated weights for policy 1, policy_version 71170 (0.0009) [2023-10-14 20:36:28,826][61585] Updated weights for policy 1, policy_version 71180 (0.0007) [2023-10-14 20:36:29,082][61552] Updated weights for policy 0, policy_version 71332 (0.0008) [2023-10-14 20:36:29,188][61585] Updated weights for policy 1, policy_version 71190 (0.0009) [2023-10-14 20:36:29,442][61552] Updated weights for policy 0, policy_version 71342 (0.0009) [2023-10-14 20:36:29,560][61585] Updated weights for policy 1, policy_version 71200 (0.0008) [2023-10-14 20:36:29,813][61552] Updated weights for policy 0, policy_version 71352 (0.0010) [2023-10-14 20:36:33,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 145981440. Throughput: 0: 1656.9, 1: 1669.2. Samples: 36503160. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-14 20:36:33,344][60425] Avg episode reward: [(0, '73.500'), (1, '74.710')] [2023-10-14 20:36:33,509][61585] Updated weights for policy 1, policy_version 71210 (0.0010) [2023-10-14 20:36:33,728][61552] Updated weights for policy 0, policy_version 71362 (0.0008) [2023-10-14 20:36:33,872][61585] Updated weights for policy 1, policy_version 71220 (0.0009) [2023-10-14 20:36:34,090][61552] Updated weights for policy 0, policy_version 71372 (0.0009) [2023-10-14 20:36:34,238][61585] Updated weights for policy 1, policy_version 71230 (0.0008) [2023-10-14 20:36:34,454][61552] Updated weights for policy 0, policy_version 71382 (0.0009) [2023-10-14 20:36:34,824][61552] Updated weights for policy 0, policy_version 71392 (0.0008) [2023-10-14 20:36:38,271][61585] Updated weights for policy 1, policy_version 71240 (0.0008) [2023-10-14 20:36:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 146046976. Throughput: 0: 1659.5, 1: 1675.7. Samples: 36523914. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-14 20:36:38,344][60425] Avg episode reward: [(0, '78.500'), (1, '75.070')] [2023-10-14 20:36:38,642][61585] Updated weights for policy 1, policy_version 71250 (0.0008) [2023-10-14 20:36:38,935][61552] Updated weights for policy 0, policy_version 71402 (0.0008) [2023-10-14 20:36:39,005][61585] Updated weights for policy 1, policy_version 71260 (0.0008) [2023-10-14 20:36:39,294][61552] Updated weights for policy 0, policy_version 71412 (0.0008) [2023-10-14 20:36:39,659][61552] Updated weights for policy 0, policy_version 71422 (0.0007) [2023-10-14 20:36:43,152][61585] Updated weights for policy 1, policy_version 71270 (0.0008) [2023-10-14 20:36:43,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 146112512. Throughput: 0: 1664.6, 1: 1681.8. Samples: 36544682. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-14 20:36:43,344][60425] Avg episode reward: [(0, '78.520'), (1, '76.550')] [2023-10-14 20:36:43,517][61585] Updated weights for policy 1, policy_version 71280 (0.0008) [2023-10-14 20:36:43,748][61552] Updated weights for policy 0, policy_version 71432 (0.0008) [2023-10-14 20:36:43,886][61585] Updated weights for policy 1, policy_version 71290 (0.0009) [2023-10-14 20:36:44,118][61552] Updated weights for policy 0, policy_version 71442 (0.0009) [2023-10-14 20:36:44,492][61552] Updated weights for policy 0, policy_version 71452 (0.0007) [2023-10-14 20:36:47,996][61585] Updated weights for policy 1, policy_version 71300 (0.0008) [2023-10-14 20:36:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 146178048. Throughput: 0: 1665.7, 1: 1678.1. Samples: 36553588. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-14 20:36:48,344][60425] Avg episode reward: [(0, '79.430'), (1, '81.480')] [2023-10-14 20:36:48,361][61585] Updated weights for policy 1, policy_version 71310 (0.0009) [2023-10-14 20:36:48,674][61552] Updated weights for policy 0, policy_version 71462 (0.0008) [2023-10-14 20:36:48,730][61585] Updated weights for policy 1, policy_version 71320 (0.0009) [2023-10-14 20:36:49,042][61552] Updated weights for policy 0, policy_version 71472 (0.0009) [2023-10-14 20:36:49,407][61552] Updated weights for policy 0, policy_version 71482 (0.0008) [2023-10-14 20:36:52,952][61585] Updated weights for policy 1, policy_version 71330 (0.0007) [2023-10-14 20:36:53,316][61585] Updated weights for policy 1, policy_version 71340 (0.0007) [2023-10-14 20:36:53,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 146243584. Throughput: 0: 1665.1, 1: 1675.6. Samples: 36573876. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-14 20:36:53,344][60425] Avg episode reward: [(0, '79.970'), (1, '77.080')] [2023-10-14 20:36:53,559][61552] Updated weights for policy 0, policy_version 71492 (0.0009) [2023-10-14 20:36:53,685][61585] Updated weights for policy 1, policy_version 71350 (0.0008) [2023-10-14 20:36:53,919][61552] Updated weights for policy 0, policy_version 71502 (0.0009) [2023-10-14 20:36:54,045][61585] Updated weights for policy 1, policy_version 71360 (0.0009) [2023-10-14 20:36:54,287][61552] Updated weights for policy 0, policy_version 71512 (0.0007) [2023-10-14 20:36:57,975][61585] Updated weights for policy 1, policy_version 71370 (0.0009) [2023-10-14 20:36:58,338][61585] Updated weights for policy 1, policy_version 71380 (0.0008) [2023-10-14 20:36:58,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 146309120. Throughput: 0: 1664.7, 1: 1675.1. Samples: 36594468. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-14 20:36:58,344][60425] Avg episode reward: [(0, '77.430'), (1, '72.600')] [2023-10-14 20:36:58,347][61552] Updated weights for policy 0, policy_version 71522 (0.0007) [2023-10-14 20:36:58,703][61585] Updated weights for policy 1, policy_version 71390 (0.0007) [2023-10-14 20:36:58,720][61552] Updated weights for policy 0, policy_version 71532 (0.0009) [2023-10-14 20:36:59,081][61552] Updated weights for policy 0, policy_version 71542 (0.0011) [2023-10-14 20:36:59,452][61552] Updated weights for policy 0, policy_version 71552 (0.0010) [2023-10-14 20:37:02,875][61585] Updated weights for policy 1, policy_version 71400 (0.0007) [2023-10-14 20:37:03,267][61585] Updated weights for policy 1, policy_version 71410 (0.0008) [2023-10-14 20:37:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 146374656. Throughput: 0: 1669.2, 1: 1682.1. Samples: 36603778. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-14 20:37:03,344][60425] Avg episode reward: [(0, '76.440'), (1, '76.490')] [2023-10-14 20:37:03,627][61585] Updated weights for policy 1, policy_version 71420 (0.0009) [2023-10-14 20:37:03,682][61552] Updated weights for policy 0, policy_version 71562 (0.0007) [2023-10-14 20:37:04,052][61552] Updated weights for policy 0, policy_version 71572 (0.0008) [2023-10-14 20:37:04,425][61552] Updated weights for policy 0, policy_version 71582 (0.0009) [2023-10-14 20:37:07,794][61585] Updated weights for policy 1, policy_version 71430 (0.0008) [2023-10-14 20:37:08,170][61585] Updated weights for policy 1, policy_version 71440 (0.0007) [2023-10-14 20:37:08,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 146440192. Throughput: 0: 1665.5, 1: 1675.7. Samples: 36623842. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-14 20:37:08,344][60425] Avg episode reward: [(0, '79.260'), (1, '76.780')] [2023-10-14 20:37:08,506][61552] Updated weights for policy 0, policy_version 71592 (0.0009) [2023-10-14 20:37:08,525][61585] Updated weights for policy 1, policy_version 71450 (0.0008) [2023-10-14 20:37:08,874][61552] Updated weights for policy 0, policy_version 71602 (0.0008) [2023-10-14 20:37:09,236][61552] Updated weights for policy 0, policy_version 71612 (0.0008) [2023-10-14 20:37:12,689][61585] Updated weights for policy 1, policy_version 71460 (0.0009) [2023-10-14 20:37:13,041][61585] Updated weights for policy 1, policy_version 71470 (0.0008) [2023-10-14 20:37:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 146505728. Throughput: 0: 1664.4, 1: 1670.3. Samples: 36643912. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-14 20:37:13,344][60425] Avg episode reward: [(0, '80.140'), (1, '76.020')] [2023-10-14 20:37:13,403][61552] Updated weights for policy 0, policy_version 71622 (0.0008) [2023-10-14 20:37:13,405][61585] Updated weights for policy 1, policy_version 71480 (0.0008) [2023-10-14 20:37:13,762][61552] Updated weights for policy 0, policy_version 71632 (0.0008) [2023-10-14 20:37:14,139][61552] Updated weights for policy 0, policy_version 71642 (0.0010) [2023-10-14 20:37:17,560][61585] Updated weights for policy 1, policy_version 71490 (0.0007) [2023-10-14 20:37:17,924][61585] Updated weights for policy 1, policy_version 71500 (0.0007) [2023-10-14 20:37:18,283][61585] Updated weights for policy 1, policy_version 71510 (0.0007) [2023-10-14 20:37:18,299][61552] Updated weights for policy 0, policy_version 71652 (0.0008) [2023-10-14 20:37:18,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 146571264. Throughput: 0: 1662.3, 1: 1674.5. Samples: 36653314. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-14 20:37:18,344][60425] Avg episode reward: [(0, '77.860'), (1, '74.250')] [2023-10-14 20:37:18,648][61585] Updated weights for policy 1, policy_version 71520 (0.0010) [2023-10-14 20:37:18,651][61552] Updated weights for policy 0, policy_version 71662 (0.0008) [2023-10-14 20:37:19,016][61552] Updated weights for policy 0, policy_version 71672 (0.0007) [2023-10-14 20:37:22,728][61585] Updated weights for policy 1, policy_version 71530 (0.0008) [2023-10-14 20:37:23,096][61585] Updated weights for policy 1, policy_version 71540 (0.0009) [2023-10-14 20:37:23,144][61552] Updated weights for policy 0, policy_version 71682 (0.0009) [2023-10-14 20:37:23,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 146636800. Throughput: 0: 1660.0, 1: 1665.9. Samples: 36673584. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-14 20:37:23,344][60425] Avg episode reward: [(0, '77.150'), (1, '72.250')] [2023-10-14 20:37:23,462][61585] Updated weights for policy 1, policy_version 71550 (0.0008) [2023-10-14 20:37:23,504][61552] Updated weights for policy 0, policy_version 71692 (0.0009) [2023-10-14 20:37:23,881][61552] Updated weights for policy 0, policy_version 71702 (0.0009) [2023-10-14 20:37:24,241][61552] Updated weights for policy 0, policy_version 71712 (0.0008) [2023-10-14 20:37:27,662][61585] Updated weights for policy 1, policy_version 71560 (0.0009) [2023-10-14 20:37:28,029][61585] Updated weights for policy 1, policy_version 71570 (0.0008) [2023-10-14 20:37:28,270][61552] Updated weights for policy 0, policy_version 71722 (0.0010) [2023-10-14 20:37:28,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 146702336. Throughput: 0: 1658.9, 1: 1650.8. Samples: 36693616. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-14 20:37:28,344][60425] Avg episode reward: [(0, '77.250'), (1, '71.150')] [2023-10-14 20:37:28,385][61585] Updated weights for policy 1, policy_version 71580 (0.0007) [2023-10-14 20:37:28,640][61552] Updated weights for policy 0, policy_version 71732 (0.0008) [2023-10-14 20:37:29,009][61552] Updated weights for policy 0, policy_version 71742 (0.0009) [2023-10-14 20:37:32,394][61585] Updated weights for policy 1, policy_version 71590 (0.0008) [2023-10-14 20:37:32,755][61585] Updated weights for policy 1, policy_version 71600 (0.0008) [2023-10-14 20:37:33,119][61552] Updated weights for policy 0, policy_version 71752 (0.0008) [2023-10-14 20:37:33,128][61585] Updated weights for policy 1, policy_version 71610 (0.0009) [2023-10-14 20:37:33,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13218.3). Total num frames: 146767872. Throughput: 0: 1662.8, 1: 1666.7. Samples: 36703418. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-14 20:37:33,344][60425] Avg episode reward: [(0, '80.750'), (1, '73.170')] [2023-10-14 20:37:33,493][61552] Updated weights for policy 0, policy_version 71762 (0.0007) [2023-10-14 20:37:33,865][61552] Updated weights for policy 0, policy_version 71772 (0.0008) [2023-10-14 20:37:37,148][61585] Updated weights for policy 1, policy_version 71620 (0.0008) [2023-10-14 20:37:37,519][61585] Updated weights for policy 1, policy_version 71630 (0.0009) [2023-10-14 20:37:37,882][61585] Updated weights for policy 1, policy_version 71640 (0.0008) [2023-10-14 20:37:38,017][61552] Updated weights for policy 0, policy_version 71782 (0.0008) [2023-10-14 20:37:38,343][60425] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 146866176. Throughput: 0: 1663.0, 1: 1669.9. Samples: 36723858. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-14 20:37:38,344][60425] Avg episode reward: [(0, '77.300'), (1, '76.670')] [2023-10-14 20:37:38,392][61552] Updated weights for policy 0, policy_version 71792 (0.0007) [2023-10-14 20:37:38,755][61552] Updated weights for policy 0, policy_version 71802 (0.0007) [2023-10-14 20:37:41,933][61585] Updated weights for policy 1, policy_version 71650 (0.0008) [2023-10-14 20:37:42,304][61585] Updated weights for policy 1, policy_version 71660 (0.0007) [2023-10-14 20:37:42,663][61585] Updated weights for policy 1, policy_version 71670 (0.0007) [2023-10-14 20:37:42,753][61552] Updated weights for policy 0, policy_version 71812 (0.0007) [2023-10-14 20:37:43,025][61585] Updated weights for policy 1, policy_version 71680 (0.0007) [2023-10-14 20:37:43,118][61552] Updated weights for policy 0, policy_version 71822 (0.0009) [2023-10-14 20:37:43,343][60425] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 146931712. Throughput: 0: 1663.3, 1: 1648.9. Samples: 36743516. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-14 20:37:43,344][60425] Avg episode reward: [(0, '76.270'), (1, '80.970')] [2023-10-14 20:37:43,354][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000071680_73400320.pth... [2023-10-14 20:37:43,383][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000070112_71794688.pth [2023-10-14 20:37:43,488][61552] Updated weights for policy 0, policy_version 71832 (0.0008) [2023-10-14 20:37:43,775][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000071840_73564160.pth... [2023-10-14 20:37:43,816][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000070272_71958528.pth [2023-10-14 20:37:47,133][61585] Updated weights for policy 1, policy_version 71690 (0.0007) [2023-10-14 20:37:47,494][61585] Updated weights for policy 1, policy_version 71700 (0.0007) [2023-10-14 20:37:47,699][61552] Updated weights for policy 0, policy_version 71842 (0.0008) [2023-10-14 20:37:47,854][61585] Updated weights for policy 1, policy_version 71710 (0.0009) [2023-10-14 20:37:48,119][61552] Updated weights for policy 0, policy_version 71852 (0.0009) [2023-10-14 20:37:48,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 146997248. Throughput: 0: 1663.2, 1: 1667.0. Samples: 36753636. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-14 20:37:48,344][60425] Avg episode reward: [(0, '80.330'), (1, '74.610')] [2023-10-14 20:37:48,480][61552] Updated weights for policy 0, policy_version 71862 (0.0009) [2023-10-14 20:37:48,855][61552] Updated weights for policy 0, policy_version 71872 (0.0008) [2023-10-14 20:37:52,198][61585] Updated weights for policy 1, policy_version 71720 (0.0010) [2023-10-14 20:37:52,560][61585] Updated weights for policy 1, policy_version 71730 (0.0009) [2023-10-14 20:37:52,867][61552] Updated weights for policy 0, policy_version 71882 (0.0008) [2023-10-14 20:37:52,921][61585] Updated weights for policy 1, policy_version 71740 (0.0007) [2023-10-14 20:37:53,239][61552] Updated weights for policy 0, policy_version 71892 (0.0009) [2023-10-14 20:37:53,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 147062784. Throughput: 0: 1664.9, 1: 1673.8. Samples: 36774086. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-14 20:37:53,344][60425] Avg episode reward: [(0, '78.150'), (1, '77.180')] [2023-10-14 20:37:53,609][61552] Updated weights for policy 0, policy_version 71902 (0.0008) [2023-10-14 20:37:57,167][61585] Updated weights for policy 1, policy_version 71750 (0.0008) [2023-10-14 20:37:57,541][61585] Updated weights for policy 1, policy_version 71760 (0.0008) [2023-10-14 20:37:57,734][61552] Updated weights for policy 0, policy_version 71912 (0.0007) [2023-10-14 20:37:57,898][61585] Updated weights for policy 1, policy_version 71770 (0.0007) [2023-10-14 20:37:58,104][61552] Updated weights for policy 0, policy_version 71922 (0.0009) [2023-10-14 20:37:58,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 147128320. Throughput: 0: 1666.9, 1: 1655.2. Samples: 36793406. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-14 20:37:58,344][60425] Avg episode reward: [(0, '81.750'), (1, '76.470')] [2023-10-14 20:37:58,480][61552] Updated weights for policy 0, policy_version 71932 (0.0010) [2023-10-14 20:38:01,918][61585] Updated weights for policy 1, policy_version 71780 (0.0008) [2023-10-14 20:38:02,279][61585] Updated weights for policy 1, policy_version 71790 (0.0011) [2023-10-14 20:38:02,654][61585] Updated weights for policy 1, policy_version 71800 (0.0009) [2023-10-14 20:38:02,655][61552] Updated weights for policy 0, policy_version 71942 (0.0008) [2023-10-14 20:38:03,018][61552] Updated weights for policy 0, policy_version 71952 (0.0010) [2023-10-14 20:38:03,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 147193856. Throughput: 0: 1667.0, 1: 1669.5. Samples: 36803456. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-14 20:38:03,344][60425] Avg episode reward: [(0, '75.740'), (1, '80.830')] [2023-10-14 20:38:03,392][61552] Updated weights for policy 0, policy_version 71962 (0.0009) [2023-10-14 20:38:06,689][61585] Updated weights for policy 1, policy_version 71810 (0.0007) [2023-10-14 20:38:07,050][61585] Updated weights for policy 1, policy_version 71820 (0.0008) [2023-10-14 20:38:07,405][61585] Updated weights for policy 1, policy_version 71830 (0.0009) [2023-10-14 20:38:07,476][61552] Updated weights for policy 0, policy_version 71972 (0.0007) [2023-10-14 20:38:07,779][61585] Updated weights for policy 1, policy_version 71840 (0.0008) [2023-10-14 20:38:07,840][61552] Updated weights for policy 0, policy_version 71982 (0.0008) [2023-10-14 20:38:08,207][61552] Updated weights for policy 0, policy_version 71992 (0.0009) [2023-10-14 20:38:08,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 147259392. Throughput: 0: 1670.5, 1: 1669.2. Samples: 36823868. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-14 20:38:08,344][60425] Avg episode reward: [(0, '77.160'), (1, '80.140')] [2023-10-14 20:38:11,875][61585] Updated weights for policy 1, policy_version 71850 (0.0009) [2023-10-14 20:38:12,237][61585] Updated weights for policy 1, policy_version 71860 (0.0009) [2023-10-14 20:38:12,331][61552] Updated weights for policy 0, policy_version 72002 (0.0007) [2023-10-14 20:38:12,609][61585] Updated weights for policy 1, policy_version 71870 (0.0007) [2023-10-14 20:38:12,692][61552] Updated weights for policy 0, policy_version 72012 (0.0007) [2023-10-14 20:38:13,063][61552] Updated weights for policy 0, policy_version 72022 (0.0007) [2023-10-14 20:38:13,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 147324928. Throughput: 0: 1665.9, 1: 1654.2. Samples: 36843020. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-14 20:38:13,344][60425] Avg episode reward: [(0, '75.370'), (1, '76.370')] [2023-10-14 20:38:13,434][61552] Updated weights for policy 0, policy_version 72032 (0.0009) [2023-10-14 20:38:16,666][61585] Updated weights for policy 1, policy_version 71880 (0.0007) [2023-10-14 20:38:17,035][61585] Updated weights for policy 1, policy_version 71890 (0.0010) [2023-10-14 20:38:17,417][61585] Updated weights for policy 1, policy_version 71900 (0.0008) [2023-10-14 20:38:17,521][61552] Updated weights for policy 0, policy_version 72042 (0.0010) [2023-10-14 20:38:17,893][61552] Updated weights for policy 0, policy_version 72052 (0.0008) [2023-10-14 20:38:18,268][61552] Updated weights for policy 0, policy_version 72062 (0.0009) [2023-10-14 20:38:18,343][60425] Fps is (10 sec: 16383.6, 60 sec: 14199.4, 300 sec: 13329.4). Total num frames: 147423232. Throughput: 0: 1671.2, 1: 1670.9. Samples: 36853814. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-14 20:38:18,345][60425] Avg episode reward: [(0, '78.770'), (1, '78.470')] [2023-10-14 20:38:21,579][61585] Updated weights for policy 1, policy_version 71910 (0.0009) [2023-10-14 20:38:21,946][61585] Updated weights for policy 1, policy_version 71920 (0.0009) [2023-10-14 20:38:22,310][61585] Updated weights for policy 1, policy_version 71930 (0.0009) [2023-10-14 20:38:22,350][61552] Updated weights for policy 0, policy_version 72072 (0.0008) [2023-10-14 20:38:22,707][61552] Updated weights for policy 0, policy_version 72082 (0.0008) [2023-10-14 20:38:23,073][61552] Updated weights for policy 0, policy_version 72092 (0.0008) [2023-10-14 20:38:23,343][60425] Fps is (10 sec: 16384.2, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 147488768. Throughput: 0: 1673.1, 1: 1658.5. Samples: 36873780. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-14 20:38:23,344][60425] Avg episode reward: [(0, '80.260'), (1, '74.370')] [2023-10-14 20:38:26,340][61585] Updated weights for policy 1, policy_version 71940 (0.0007) [2023-10-14 20:38:26,705][61585] Updated weights for policy 1, policy_version 71950 (0.0009) [2023-10-14 20:38:27,065][61585] Updated weights for policy 1, policy_version 71960 (0.0009) [2023-10-14 20:38:27,212][61552] Updated weights for policy 0, policy_version 72102 (0.0009) [2023-10-14 20:38:27,581][61552] Updated weights for policy 0, policy_version 72112 (0.0008) [2023-10-14 20:38:27,942][61552] Updated weights for policy 0, policy_version 72122 (0.0009) [2023-10-14 20:38:28,343][60425] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 147554304. Throughput: 0: 1654.0, 1: 1659.7. Samples: 36892632. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-14 20:38:28,345][60425] Avg episode reward: [(0, '75.960'), (1, '80.540')] [2023-10-14 20:38:31,116][61585] Updated weights for policy 1, policy_version 71970 (0.0007) [2023-10-14 20:38:31,481][61585] Updated weights for policy 1, policy_version 71980 (0.0008) [2023-10-14 20:38:31,843][61585] Updated weights for policy 1, policy_version 71990 (0.0008) [2023-10-14 20:38:32,086][61552] Updated weights for policy 0, policy_version 72132 (0.0008) [2023-10-14 20:38:32,206][61585] Updated weights for policy 1, policy_version 72000 (0.0007) [2023-10-14 20:38:32,461][61552] Updated weights for policy 0, policy_version 72142 (0.0008) [2023-10-14 20:38:32,824][61552] Updated weights for policy 0, policy_version 72152 (0.0007) [2023-10-14 20:38:33,343][60425] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 147619840. Throughput: 0: 1670.7, 1: 1669.2. Samples: 36903930. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-14 20:38:33,344][60425] Avg episode reward: [(0, '77.240'), (1, '74.070')] [2023-10-14 20:38:36,394][61585] Updated weights for policy 1, policy_version 72010 (0.0010) [2023-10-14 20:38:36,753][61585] Updated weights for policy 1, policy_version 72020 (0.0008) [2023-10-14 20:38:37,015][61552] Updated weights for policy 0, policy_version 72162 (0.0008) [2023-10-14 20:38:37,126][61585] Updated weights for policy 1, policy_version 72030 (0.0008) [2023-10-14 20:38:37,405][61552] Updated weights for policy 0, policy_version 72172 (0.0009) [2023-10-14 20:38:37,771][61552] Updated weights for policy 0, policy_version 72182 (0.0007) [2023-10-14 20:38:38,136][61552] Updated weights for policy 0, policy_version 72192 (0.0007) [2023-10-14 20:38:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 147685376. Throughput: 0: 1673.9, 1: 1656.2. Samples: 36923940. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-14 20:38:38,344][60425] Avg episode reward: [(0, '78.400'), (1, '78.720')] [2023-10-14 20:38:41,245][61585] Updated weights for policy 1, policy_version 72040 (0.0008) [2023-10-14 20:38:41,606][61585] Updated weights for policy 1, policy_version 72050 (0.0008) [2023-10-14 20:38:41,976][61585] Updated weights for policy 1, policy_version 72060 (0.0007) [2023-10-14 20:38:42,351][61552] Updated weights for policy 0, policy_version 72202 (0.0008) [2023-10-14 20:38:42,724][61552] Updated weights for policy 0, policy_version 72212 (0.0011) [2023-10-14 20:38:43,096][61552] Updated weights for policy 0, policy_version 72222 (0.0008) [2023-10-14 20:38:43,343][60425] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 147750912. Throughput: 0: 1657.5, 1: 1670.9. Samples: 36943186. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-14 20:38:43,344][60425] Avg episode reward: [(0, '76.270'), (1, '77.710')] [2023-10-14 20:38:46,056][61585] Updated weights for policy 1, policy_version 72070 (0.0008) [2023-10-14 20:38:46,419][61585] Updated weights for policy 1, policy_version 72080 (0.0009) [2023-10-14 20:38:46,782][61585] Updated weights for policy 1, policy_version 72090 (0.0008) [2023-10-14 20:38:47,073][61552] Updated weights for policy 0, policy_version 72232 (0.0008) [2023-10-14 20:38:47,432][61552] Updated weights for policy 0, policy_version 72242 (0.0010) [2023-10-14 20:38:47,796][61552] Updated weights for policy 0, policy_version 72252 (0.0011) [2023-10-14 20:38:48,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 147816448. Throughput: 0: 1672.4, 1: 1679.3. Samples: 36954286. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-14 20:38:48,344][60425] Avg episode reward: [(0, '79.140'), (1, '78.310')] [2023-10-14 20:38:50,846][61585] Updated weights for policy 1, policy_version 72100 (0.0010) [2023-10-14 20:38:51,213][61585] Updated weights for policy 1, policy_version 72110 (0.0009) [2023-10-14 20:38:51,574][61585] Updated weights for policy 1, policy_version 72120 (0.0009) [2023-10-14 20:38:51,951][61552] Updated weights for policy 0, policy_version 72262 (0.0009) [2023-10-14 20:38:52,316][61552] Updated weights for policy 0, policy_version 72272 (0.0009) [2023-10-14 20:38:52,685][61552] Updated weights for policy 0, policy_version 72282 (0.0008) [2023-10-14 20:38:53,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 147881984. Throughput: 0: 1672.5, 1: 1657.4. Samples: 36973716. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-14 20:38:53,344][60425] Avg episode reward: [(0, '75.580'), (1, '80.210')] [2023-10-14 20:38:55,789][61585] Updated weights for policy 1, policy_version 72130 (0.0009) [2023-10-14 20:38:56,158][61585] Updated weights for policy 1, policy_version 72140 (0.0009) [2023-10-14 20:38:56,517][61585] Updated weights for policy 1, policy_version 72150 (0.0008) [2023-10-14 20:38:56,821][61552] Updated weights for policy 0, policy_version 72292 (0.0007) [2023-10-14 20:38:56,879][61585] Updated weights for policy 1, policy_version 72160 (0.0009) [2023-10-14 20:38:57,191][61552] Updated weights for policy 0, policy_version 72302 (0.0009) [2023-10-14 20:38:57,557][61552] Updated weights for policy 0, policy_version 72312 (0.0008) [2023-10-14 20:38:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 147947520. Throughput: 0: 1652.9, 1: 1677.6. Samples: 36992890. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-14 20:38:58,344][60425] Avg episode reward: [(0, '77.000'), (1, '76.580')] [2023-10-14 20:39:00,775][61585] Updated weights for policy 1, policy_version 72170 (0.0007) [2023-10-14 20:39:01,132][61585] Updated weights for policy 1, policy_version 72180 (0.0010) [2023-10-14 20:39:01,500][61585] Updated weights for policy 1, policy_version 72190 (0.0009) [2023-10-14 20:39:01,626][61552] Updated weights for policy 0, policy_version 72322 (0.0009) [2023-10-14 20:39:01,990][61552] Updated weights for policy 0, policy_version 72332 (0.0010) [2023-10-14 20:39:02,369][61552] Updated weights for policy 0, policy_version 72342 (0.0010) [2023-10-14 20:39:02,733][61552] Updated weights for policy 0, policy_version 72352 (0.0009) [2023-10-14 20:39:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 148013056. Throughput: 0: 1669.3, 1: 1668.8. Samples: 37004032. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-14 20:39:03,344][60425] Avg episode reward: [(0, '73.510'), (1, '74.040')] [2023-10-14 20:39:05,445][61585] Updated weights for policy 1, policy_version 72200 (0.0007) [2023-10-14 20:39:05,818][61585] Updated weights for policy 1, policy_version 72210 (0.0009) [2023-10-14 20:39:06,179][61585] Updated weights for policy 1, policy_version 72220 (0.0008) [2023-10-14 20:39:06,727][61552] Updated weights for policy 0, policy_version 72362 (0.0009) [2023-10-14 20:39:07,104][61552] Updated weights for policy 0, policy_version 72372 (0.0008) [2023-10-14 20:39:07,473][61552] Updated weights for policy 0, policy_version 72382 (0.0008) [2023-10-14 20:39:08,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 148078592. Throughput: 0: 1662.4, 1: 1661.3. Samples: 37023348. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-14 20:39:08,344][60425] Avg episode reward: [(0, '77.500'), (1, '74.000')] [2023-10-14 20:39:10,351][61585] Updated weights for policy 1, policy_version 72230 (0.0009) [2023-10-14 20:39:10,715][61585] Updated weights for policy 1, policy_version 72240 (0.0010) [2023-10-14 20:39:11,077][61585] Updated weights for policy 1, policy_version 72250 (0.0010) [2023-10-14 20:39:11,541][61552] Updated weights for policy 0, policy_version 72392 (0.0008) [2023-10-14 20:39:11,910][61552] Updated weights for policy 0, policy_version 72402 (0.0007) [2023-10-14 20:39:12,280][61552] Updated weights for policy 0, policy_version 72412 (0.0007) [2023-10-14 20:39:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 148144128. Throughput: 0: 1657.3, 1: 1684.7. Samples: 37043022. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-14 20:39:13,345][60425] Avg episode reward: [(0, '77.120'), (1, '77.700')] [2023-10-14 20:39:15,192][61585] Updated weights for policy 1, policy_version 72260 (0.0009) [2023-10-14 20:39:15,553][61585] Updated weights for policy 1, policy_version 72270 (0.0010) [2023-10-14 20:39:15,913][61585] Updated weights for policy 1, policy_version 72280 (0.0009) [2023-10-14 20:39:16,365][61552] Updated weights for policy 0, policy_version 72422 (0.0009) [2023-10-14 20:39:16,733][61552] Updated weights for policy 0, policy_version 72432 (0.0009) [2023-10-14 20:39:17,103][61552] Updated weights for policy 0, policy_version 72442 (0.0008) [2023-10-14 20:39:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 148209664. Throughput: 0: 1669.6, 1: 1661.4. Samples: 37053822. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-14 20:39:18,344][60425] Avg episode reward: [(0, '75.450'), (1, '75.790')] [2023-10-14 20:39:20,137][61585] Updated weights for policy 1, policy_version 72290 (0.0008) [2023-10-14 20:39:20,500][61585] Updated weights for policy 1, policy_version 72300 (0.0009) [2023-10-14 20:39:20,879][61585] Updated weights for policy 1, policy_version 72310 (0.0009) [2023-10-14 20:39:21,239][61585] Updated weights for policy 1, policy_version 72320 (0.0008) [2023-10-14 20:39:21,271][61552] Updated weights for policy 0, policy_version 72452 (0.0009) [2023-10-14 20:39:21,661][61552] Updated weights for policy 0, policy_version 72462 (0.0008) [2023-10-14 20:39:22,031][61552] Updated weights for policy 0, policy_version 72472 (0.0007) [2023-10-14 20:39:23,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 148275200. Throughput: 0: 1650.0, 1: 1665.3. Samples: 37073126. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-14 20:39:23,344][60425] Avg episode reward: [(0, '75.780'), (1, '77.390')] [2023-10-14 20:39:25,433][61585] Updated weights for policy 1, policy_version 72330 (0.0008) [2023-10-14 20:39:25,806][61585] Updated weights for policy 1, policy_version 72340 (0.0007) [2023-10-14 20:39:26,136][61552] Updated weights for policy 0, policy_version 72482 (0.0007) [2023-10-14 20:39:26,177][61585] Updated weights for policy 1, policy_version 72350 (0.0009) [2023-10-14 20:39:26,501][61552] Updated weights for policy 0, policy_version 72492 (0.0010) [2023-10-14 20:39:26,869][61552] Updated weights for policy 0, policy_version 72502 (0.0009) [2023-10-14 20:39:27,247][61552] Updated weights for policy 0, policy_version 72512 (0.0010) [2023-10-14 20:39:28,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 148340736. Throughput: 0: 1657.2, 1: 1672.0. Samples: 37092998. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-14 20:39:28,344][60425] Avg episode reward: [(0, '74.910'), (1, '76.600')] [2023-10-14 20:39:30,435][61585] Updated weights for policy 1, policy_version 72360 (0.0009) [2023-10-14 20:39:30,809][61585] Updated weights for policy 1, policy_version 72370 (0.0010) [2023-10-14 20:39:31,170][61585] Updated weights for policy 1, policy_version 72380 (0.0009) [2023-10-14 20:39:31,400][61552] Updated weights for policy 0, policy_version 72522 (0.0008) [2023-10-14 20:39:31,766][61552] Updated weights for policy 0, policy_version 72532 (0.0009) [2023-10-14 20:39:32,135][61552] Updated weights for policy 0, policy_version 72542 (0.0008) [2023-10-14 20:39:33,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 148406272. Throughput: 0: 1666.5, 1: 1651.8. Samples: 37103610. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-14 20:39:33,344][60425] Avg episode reward: [(0, '78.730'), (1, '75.750')] [2023-10-14 20:39:35,240][61585] Updated weights for policy 1, policy_version 72390 (0.0007) [2023-10-14 20:39:35,607][61585] Updated weights for policy 1, policy_version 72400 (0.0007) [2023-10-14 20:39:35,970][61585] Updated weights for policy 1, policy_version 72410 (0.0008) [2023-10-14 20:39:36,392][61552] Updated weights for policy 0, policy_version 72552 (0.0008) [2023-10-14 20:39:36,765][61552] Updated weights for policy 0, policy_version 72562 (0.0008) [2023-10-14 20:39:37,135][61552] Updated weights for policy 0, policy_version 72572 (0.0008) [2023-10-14 20:39:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 148471808. Throughput: 0: 1648.7, 1: 1665.2. Samples: 37122840. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-14 20:39:38,344][60425] Avg episode reward: [(0, '77.510'), (1, '78.760')] [2023-10-14 20:39:40,066][61585] Updated weights for policy 1, policy_version 72420 (0.0008) [2023-10-14 20:39:40,424][61585] Updated weights for policy 1, policy_version 72430 (0.0008) [2023-10-14 20:39:40,792][61585] Updated weights for policy 1, policy_version 72440 (0.0009) [2023-10-14 20:39:41,307][61552] Updated weights for policy 0, policy_version 72582 (0.0008) [2023-10-14 20:39:41,692][61552] Updated weights for policy 0, policy_version 72592 (0.0010) [2023-10-14 20:39:42,059][61552] Updated weights for policy 0, policy_version 72602 (0.0010) [2023-10-14 20:39:43,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 148537344. Throughput: 0: 1659.1, 1: 1673.7. Samples: 37142864. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-14 20:39:43,344][60425] Avg episode reward: [(0, '76.880'), (1, '81.820')] [2023-10-14 20:39:43,353][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000072448_74186752.pth... [2023-10-14 20:39:43,354][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000072608_74350592.pth... [2023-10-14 20:39:43,384][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000070880_72581120.pth [2023-10-14 20:39:43,390][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000071040_72744960.pth [2023-10-14 20:39:44,844][61585] Updated weights for policy 1, policy_version 72450 (0.0009) [2023-10-14 20:39:45,214][61585] Updated weights for policy 1, policy_version 72460 (0.0011) [2023-10-14 20:39:45,571][61585] Updated weights for policy 1, policy_version 72470 (0.0011) [2023-10-14 20:39:45,935][61585] Updated weights for policy 1, policy_version 72480 (0.0008) [2023-10-14 20:39:46,160][61552] Updated weights for policy 0, policy_version 72612 (0.0009) [2023-10-14 20:39:46,540][61552] Updated weights for policy 0, policy_version 72622 (0.0007) [2023-10-14 20:39:46,903][61552] Updated weights for policy 0, policy_version 72632 (0.0007) [2023-10-14 20:39:48,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 148602880. Throughput: 0: 1662.0, 1: 1656.7. Samples: 37153372. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:39:48,344][60425] Avg episode reward: [(0, '79.190'), (1, '83.710')] [2023-10-14 20:39:48,345][61248] Saving new best policy, reward=83.710! [2023-10-14 20:39:50,071][61585] Updated weights for policy 1, policy_version 72490 (0.0010) [2023-10-14 20:39:50,429][61585] Updated weights for policy 1, policy_version 72500 (0.0009) [2023-10-14 20:39:50,717][61552] Updated weights for policy 0, policy_version 72642 (0.0007) [2023-10-14 20:39:50,805][61585] Updated weights for policy 1, policy_version 72510 (0.0007) [2023-10-14 20:39:51,083][61552] Updated weights for policy 0, policy_version 72652 (0.0008) [2023-10-14 20:39:51,446][61552] Updated weights for policy 0, policy_version 72662 (0.0008) [2023-10-14 20:39:51,818][61552] Updated weights for policy 0, policy_version 72672 (0.0010) [2023-10-14 20:39:53,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 148668416. Throughput: 0: 1646.8, 1: 1670.2. Samples: 37172612. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:39:53,345][60425] Avg episode reward: [(0, '80.210'), (1, '79.740')] [2023-10-14 20:39:54,925][61585] Updated weights for policy 1, policy_version 72520 (0.0008) [2023-10-14 20:39:55,290][61585] Updated weights for policy 1, policy_version 72530 (0.0009) [2023-10-14 20:39:55,663][61585] Updated weights for policy 1, policy_version 72540 (0.0008) [2023-10-14 20:39:55,854][61552] Updated weights for policy 0, policy_version 72682 (0.0007) [2023-10-14 20:39:56,220][61552] Updated weights for policy 0, policy_version 72692 (0.0007) [2023-10-14 20:39:56,596][61552] Updated weights for policy 0, policy_version 72702 (0.0008) [2023-10-14 20:39:58,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 148733952. Throughput: 0: 1671.2, 1: 1667.6. Samples: 37193270. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:39:58,345][60425] Avg episode reward: [(0, '76.200'), (1, '80.290')] [2023-10-14 20:39:59,690][61585] Updated weights for policy 1, policy_version 72550 (0.0008) [2023-10-14 20:40:00,061][61585] Updated weights for policy 1, policy_version 72560 (0.0009) [2023-10-14 20:40:00,437][61585] Updated weights for policy 1, policy_version 72570 (0.0007) [2023-10-14 20:40:00,830][61552] Updated weights for policy 0, policy_version 72712 (0.0007) [2023-10-14 20:40:01,207][61552] Updated weights for policy 0, policy_version 72722 (0.0009) [2023-10-14 20:40:01,565][61552] Updated weights for policy 0, policy_version 72732 (0.0010) [2023-10-14 20:40:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 148799488. Throughput: 0: 1664.4, 1: 1660.1. Samples: 37203424. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:40:03,344][60425] Avg episode reward: [(0, '80.280'), (1, '76.120')] [2023-10-14 20:40:04,612][61585] Updated weights for policy 1, policy_version 72580 (0.0008) [2023-10-14 20:40:04,976][61585] Updated weights for policy 1, policy_version 72590 (0.0008) [2023-10-14 20:40:05,351][61585] Updated weights for policy 1, policy_version 72600 (0.0007) [2023-10-14 20:40:05,638][61552] Updated weights for policy 0, policy_version 72742 (0.0007) [2023-10-14 20:40:06,003][61552] Updated weights for policy 0, policy_version 72752 (0.0008) [2023-10-14 20:40:06,384][61552] Updated weights for policy 0, policy_version 72762 (0.0009) [2023-10-14 20:40:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 148865024. Throughput: 0: 1659.8, 1: 1667.4. Samples: 37222850. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:40:08,344][60425] Avg episode reward: [(0, '78.680'), (1, '77.950')] [2023-10-14 20:40:09,437][61585] Updated weights for policy 1, policy_version 72610 (0.0009) [2023-10-14 20:40:09,799][61585] Updated weights for policy 1, policy_version 72620 (0.0008) [2023-10-14 20:40:10,171][61585] Updated weights for policy 1, policy_version 72630 (0.0008) [2023-10-14 20:40:10,534][61585] Updated weights for policy 1, policy_version 72640 (0.0008) [2023-10-14 20:40:10,630][61552] Updated weights for policy 0, policy_version 72772 (0.0008) [2023-10-14 20:40:11,028][61552] Updated weights for policy 0, policy_version 72782 (0.0008) [2023-10-14 20:40:11,403][61552] Updated weights for policy 0, policy_version 72792 (0.0007) [2023-10-14 20:40:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 148930560. Throughput: 0: 1669.1, 1: 1671.9. Samples: 37243344. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:40:13,345][60425] Avg episode reward: [(0, '77.420'), (1, '78.330')] [2023-10-14 20:40:14,428][61585] Updated weights for policy 1, policy_version 72650 (0.0011) [2023-10-14 20:40:14,788][61585] Updated weights for policy 1, policy_version 72660 (0.0008) [2023-10-14 20:40:15,152][61585] Updated weights for policy 1, policy_version 72670 (0.0009) [2023-10-14 20:40:15,421][61552] Updated weights for policy 0, policy_version 72802 (0.0008) [2023-10-14 20:40:15,790][61552] Updated weights for policy 0, policy_version 72812 (0.0007) [2023-10-14 20:40:16,165][61552] Updated weights for policy 0, policy_version 72822 (0.0008) [2023-10-14 20:40:16,535][61552] Updated weights for policy 0, policy_version 72832 (0.0007) [2023-10-14 20:40:18,344][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 148996096. Throughput: 0: 1660.0, 1: 1662.6. Samples: 37253128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:40:18,345][60425] Avg episode reward: [(0, '78.750'), (1, '82.010')] [2023-10-14 20:40:19,324][61585] Updated weights for policy 1, policy_version 72680 (0.0009) [2023-10-14 20:40:19,687][61585] Updated weights for policy 1, policy_version 72690 (0.0009) [2023-10-14 20:40:20,057][61585] Updated weights for policy 1, policy_version 72700 (0.0009) [2023-10-14 20:40:20,632][61552] Updated weights for policy 0, policy_version 72842 (0.0008) [2023-10-14 20:40:20,998][61552] Updated weights for policy 0, policy_version 72852 (0.0011) [2023-10-14 20:40:21,376][61552] Updated weights for policy 0, policy_version 72862 (0.0011) [2023-10-14 20:40:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 149061632. Throughput: 0: 1658.1, 1: 1669.3. Samples: 37272574. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:40:23,344][60425] Avg episode reward: [(0, '77.730'), (1, '77.350')] [2023-10-14 20:40:24,209][61585] Updated weights for policy 1, policy_version 72710 (0.0008) [2023-10-14 20:40:24,566][61585] Updated weights for policy 1, policy_version 72720 (0.0008) [2023-10-14 20:40:24,933][61585] Updated weights for policy 1, policy_version 72730 (0.0008) [2023-10-14 20:40:25,468][61552] Updated weights for policy 0, policy_version 72872 (0.0007) [2023-10-14 20:40:25,834][61552] Updated weights for policy 0, policy_version 72882 (0.0007) [2023-10-14 20:40:26,214][61552] Updated weights for policy 0, policy_version 72892 (0.0009) [2023-10-14 20:40:28,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 149127168. Throughput: 0: 1675.8, 1: 1665.3. Samples: 37293214. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:40:28,344][60425] Avg episode reward: [(0, '77.840'), (1, '77.160')] [2023-10-14 20:40:29,110][61585] Updated weights for policy 1, policy_version 72740 (0.0008) [2023-10-14 20:40:29,471][61585] Updated weights for policy 1, policy_version 72750 (0.0011) [2023-10-14 20:40:29,835][61585] Updated weights for policy 1, policy_version 72760 (0.0008) [2023-10-14 20:40:30,294][61552] Updated weights for policy 0, policy_version 72902 (0.0009) [2023-10-14 20:40:30,668][61552] Updated weights for policy 0, policy_version 72912 (0.0007) [2023-10-14 20:40:31,044][61552] Updated weights for policy 0, policy_version 72922 (0.0008) [2023-10-14 20:40:33,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 149192704. Throughput: 0: 1662.5, 1: 1660.6. Samples: 37302912. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:40:33,344][60425] Avg episode reward: [(0, '74.090'), (1, '73.550')] [2023-10-14 20:40:33,993][61585] Updated weights for policy 1, policy_version 72770 (0.0008) [2023-10-14 20:40:34,358][61585] Updated weights for policy 1, policy_version 72780 (0.0008) [2023-10-14 20:40:34,723][61585] Updated weights for policy 1, policy_version 72790 (0.0008) [2023-10-14 20:40:35,081][61585] Updated weights for policy 1, policy_version 72800 (0.0009) [2023-10-14 20:40:35,115][61552] Updated weights for policy 0, policy_version 72932 (0.0007) [2023-10-14 20:40:35,483][61552] Updated weights for policy 0, policy_version 72942 (0.0007) [2023-10-14 20:40:35,846][61552] Updated weights for policy 0, policy_version 72952 (0.0010) [2023-10-14 20:40:38,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 149258240. Throughput: 0: 1672.1, 1: 1665.0. Samples: 37322784. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-14 20:40:38,344][60425] Avg episode reward: [(0, '77.840'), (1, '77.640')] [2023-10-14 20:40:39,292][61585] Updated weights for policy 1, policy_version 72810 (0.0007) [2023-10-14 20:40:39,648][61585] Updated weights for policy 1, policy_version 72820 (0.0007) [2023-10-14 20:40:40,016][61585] Updated weights for policy 1, policy_version 72830 (0.0007) [2023-10-14 20:40:40,062][61552] Updated weights for policy 0, policy_version 72962 (0.0008) [2023-10-14 20:40:40,425][61552] Updated weights for policy 0, policy_version 72972 (0.0007) [2023-10-14 20:40:40,787][61552] Updated weights for policy 0, policy_version 72982 (0.0009) [2023-10-14 20:40:41,165][61552] Updated weights for policy 0, policy_version 72992 (0.0011) [2023-10-14 20:40:43,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 149323776. Throughput: 0: 1671.7, 1: 1668.9. Samples: 37343598. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-14 20:40:43,344][60425] Avg episode reward: [(0, '78.520'), (1, '76.130')] [2023-10-14 20:40:44,157][61585] Updated weights for policy 1, policy_version 72840 (0.0010) [2023-10-14 20:40:44,520][61585] Updated weights for policy 1, policy_version 72850 (0.0010) [2023-10-14 20:40:44,889][61585] Updated weights for policy 1, policy_version 72860 (0.0007) [2023-10-14 20:40:45,223][61552] Updated weights for policy 0, policy_version 73002 (0.0008) [2023-10-14 20:40:45,599][61552] Updated weights for policy 0, policy_version 73012 (0.0007) [2023-10-14 20:40:45,981][61552] Updated weights for policy 0, policy_version 73022 (0.0008) [2023-10-14 20:40:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 149389312. Throughput: 0: 1657.7, 1: 1669.5. Samples: 37353150. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-14 20:40:48,344][60425] Avg episode reward: [(0, '79.780'), (1, '81.660')] [2023-10-14 20:40:48,947][61585] Updated weights for policy 1, policy_version 72870 (0.0009) [2023-10-14 20:40:49,307][61585] Updated weights for policy 1, policy_version 72880 (0.0008) [2023-10-14 20:40:49,679][61585] Updated weights for policy 1, policy_version 72890 (0.0009) [2023-10-14 20:40:50,031][61552] Updated weights for policy 0, policy_version 73032 (0.0007) [2023-10-14 20:40:50,385][61552] Updated weights for policy 0, policy_version 73042 (0.0008) [2023-10-14 20:40:50,755][61552] Updated weights for policy 0, policy_version 73052 (0.0008) [2023-10-14 20:40:53,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 149454848. Throughput: 0: 1673.1, 1: 1674.1. Samples: 37373476. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-14 20:40:53,344][60425] Avg episode reward: [(0, '76.790'), (1, '76.230')] [2023-10-14 20:40:53,666][61585] Updated weights for policy 1, policy_version 72900 (0.0007) [2023-10-14 20:40:54,028][61585] Updated weights for policy 1, policy_version 72910 (0.0010) [2023-10-14 20:40:54,397][61585] Updated weights for policy 1, policy_version 72920 (0.0007) [2023-10-14 20:40:54,617][61552] Updated weights for policy 0, policy_version 73062 (0.0008) [2023-10-14 20:40:54,998][61552] Updated weights for policy 0, policy_version 73072 (0.0008) [2023-10-14 20:40:55,368][61552] Updated weights for policy 0, policy_version 73082 (0.0007) [2023-10-14 20:40:58,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 149520384. Throughput: 0: 1681.3, 1: 1671.9. Samples: 37394238. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-14 20:40:58,344][60425] Avg episode reward: [(0, '73.550'), (1, '74.980')] [2023-10-14 20:40:58,498][61585] Updated weights for policy 1, policy_version 72930 (0.0009) [2023-10-14 20:40:58,861][61585] Updated weights for policy 1, policy_version 72940 (0.0008) [2023-10-14 20:40:59,230][61585] Updated weights for policy 1, policy_version 72950 (0.0007) [2023-10-14 20:40:59,259][61552] Updated weights for policy 0, policy_version 73092 (0.0007) [2023-10-14 20:40:59,595][61585] Updated weights for policy 1, policy_version 72960 (0.0008) [2023-10-14 20:40:59,623][61552] Updated weights for policy 0, policy_version 73102 (0.0009) [2023-10-14 20:40:59,994][61552] Updated weights for policy 0, policy_version 73112 (0.0010) [2023-10-14 20:41:03,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 149585920. Throughput: 0: 1665.6, 1: 1674.4. Samples: 37403428. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-14 20:41:03,344][60425] Avg episode reward: [(0, '74.520'), (1, '76.220')] [2023-10-14 20:41:03,670][61585] Updated weights for policy 1, policy_version 72970 (0.0007) [2023-10-14 20:41:03,980][61552] Updated weights for policy 0, policy_version 73122 (0.0008) [2023-10-14 20:41:04,030][61585] Updated weights for policy 1, policy_version 72980 (0.0007) [2023-10-14 20:41:04,343][61552] Updated weights for policy 0, policy_version 73132 (0.0007) [2023-10-14 20:41:04,400][61585] Updated weights for policy 1, policy_version 72990 (0.0010) [2023-10-14 20:41:04,717][61552] Updated weights for policy 0, policy_version 73142 (0.0008) [2023-10-14 20:41:05,085][61552] Updated weights for policy 0, policy_version 73152 (0.0008) [2023-10-14 20:41:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 149651456. Throughput: 0: 1686.6, 1: 1679.9. Samples: 37424068. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-14 20:41:08,344][60425] Avg episode reward: [(0, '72.910'), (1, '75.870')] [2023-10-14 20:41:08,543][61585] Updated weights for policy 1, policy_version 73000 (0.0007) [2023-10-14 20:41:08,915][61585] Updated weights for policy 1, policy_version 73010 (0.0007) [2023-10-14 20:41:09,111][61552] Updated weights for policy 0, policy_version 73162 (0.0008) [2023-10-14 20:41:09,277][61585] Updated weights for policy 1, policy_version 73020 (0.0008) [2023-10-14 20:41:09,487][61552] Updated weights for policy 0, policy_version 73172 (0.0008) [2023-10-14 20:41:09,854][61552] Updated weights for policy 0, policy_version 73182 (0.0008) [2023-10-14 20:41:13,191][61585] Updated weights for policy 1, policy_version 73030 (0.0008) [2023-10-14 20:41:13,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 149716992. Throughput: 0: 1683.9, 1: 1684.0. Samples: 37444768. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-14 20:41:13,344][60425] Avg episode reward: [(0, '73.510'), (1, '79.850')] [2023-10-14 20:41:13,562][61585] Updated weights for policy 1, policy_version 73040 (0.0008) [2023-10-14 20:41:13,928][61585] Updated weights for policy 1, policy_version 73050 (0.0008) [2023-10-14 20:41:14,001][61552] Updated weights for policy 0, policy_version 73192 (0.0009) [2023-10-14 20:41:14,374][61552] Updated weights for policy 0, policy_version 73202 (0.0009) [2023-10-14 20:41:14,730][61552] Updated weights for policy 0, policy_version 73212 (0.0010) [2023-10-14 20:41:18,090][61585] Updated weights for policy 1, policy_version 73060 (0.0007) [2023-10-14 20:41:18,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 149782528. Throughput: 0: 1667.6, 1: 1685.5. Samples: 37453798. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-14 20:41:18,344][60425] Avg episode reward: [(0, '75.740'), (1, '75.490')] [2023-10-14 20:41:18,453][61585] Updated weights for policy 1, policy_version 73070 (0.0008) [2023-10-14 20:41:18,822][61585] Updated weights for policy 1, policy_version 73080 (0.0009) [2023-10-14 20:41:19,059][61552] Updated weights for policy 0, policy_version 73222 (0.0008) [2023-10-14 20:41:19,431][61552] Updated weights for policy 0, policy_version 73232 (0.0008) [2023-10-14 20:41:19,808][61552] Updated weights for policy 0, policy_version 73242 (0.0010) [2023-10-14 20:41:22,924][61585] Updated weights for policy 1, policy_version 73090 (0.0007) [2023-10-14 20:41:23,290][61585] Updated weights for policy 1, policy_version 73100 (0.0009) [2023-10-14 20:41:23,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 149848064. Throughput: 0: 1680.0, 1: 1689.4. Samples: 37474408. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-14 20:41:23,344][60425] Avg episode reward: [(0, '79.720'), (1, '80.980')] [2023-10-14 20:41:23,657][61585] Updated weights for policy 1, policy_version 73110 (0.0009) [2023-10-14 20:41:23,946][61552] Updated weights for policy 0, policy_version 73252 (0.0008) [2023-10-14 20:41:24,019][61585] Updated weights for policy 1, policy_version 73120 (0.0009) [2023-10-14 20:41:24,314][61552] Updated weights for policy 0, policy_version 73262 (0.0009) [2023-10-14 20:41:24,674][61552] Updated weights for policy 0, policy_version 73272 (0.0007) [2023-10-14 20:41:28,058][61585] Updated weights for policy 1, policy_version 73130 (0.0009) [2023-10-14 20:41:28,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 149913600. Throughput: 0: 1682.5, 1: 1689.3. Samples: 37495330. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-10-14 20:41:28,345][60425] Avg episode reward: [(0, '75.760'), (1, '72.450')] [2023-10-14 20:41:28,427][61585] Updated weights for policy 1, policy_version 73140 (0.0009) [2023-10-14 20:41:28,649][61552] Updated weights for policy 0, policy_version 73282 (0.0008) [2023-10-14 20:41:28,790][61585] Updated weights for policy 1, policy_version 73150 (0.0009) [2023-10-14 20:41:29,014][61552] Updated weights for policy 0, policy_version 73292 (0.0007) [2023-10-14 20:41:29,374][61552] Updated weights for policy 0, policy_version 73302 (0.0009) [2023-10-14 20:41:29,736][61552] Updated weights for policy 0, policy_version 73312 (0.0008) [2023-10-14 20:41:32,867][61585] Updated weights for policy 1, policy_version 73160 (0.0010) [2023-10-14 20:41:33,239][61585] Updated weights for policy 1, policy_version 73170 (0.0008) [2023-10-14 20:41:33,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 149979136. Throughput: 0: 1672.6, 1: 1689.8. Samples: 37504458. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-10-14 20:41:33,344][60425] Avg episode reward: [(0, '77.440'), (1, '78.310')] [2023-10-14 20:41:33,593][61585] Updated weights for policy 1, policy_version 73180 (0.0008) [2023-10-14 20:41:33,896][61552] Updated weights for policy 0, policy_version 73322 (0.0008) [2023-10-14 20:41:34,264][61552] Updated weights for policy 0, policy_version 73332 (0.0009) [2023-10-14 20:41:34,646][61552] Updated weights for policy 0, policy_version 73342 (0.0007) [2023-10-14 20:41:37,743][61585] Updated weights for policy 1, policy_version 73190 (0.0009) [2023-10-14 20:41:38,116][61585] Updated weights for policy 1, policy_version 73200 (0.0007) [2023-10-14 20:41:38,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 150044672. Throughput: 0: 1680.1, 1: 1684.8. Samples: 37524898. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-10-14 20:41:38,344][60425] Avg episode reward: [(0, '77.140'), (1, '76.900')] [2023-10-14 20:41:38,492][61585] Updated weights for policy 1, policy_version 73210 (0.0007) [2023-10-14 20:41:38,811][61552] Updated weights for policy 0, policy_version 73352 (0.0008) [2023-10-14 20:41:39,175][61552] Updated weights for policy 0, policy_version 73362 (0.0009) [2023-10-14 20:41:39,546][61552] Updated weights for policy 0, policy_version 73372 (0.0007) [2023-10-14 20:41:42,283][61585] Updated weights for policy 1, policy_version 73220 (0.0010) [2023-10-14 20:41:42,646][61585] Updated weights for policy 1, policy_version 73230 (0.0008) [2023-10-14 20:41:43,011][61585] Updated weights for policy 1, policy_version 73240 (0.0008) [2023-10-14 20:41:43,344][60425] Fps is (10 sec: 16383.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 150142976. Throughput: 0: 1670.5, 1: 1679.2. Samples: 37544978. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-10-14 20:41:43,345][60425] Avg episode reward: [(0, '75.360'), (1, '74.760')] [2023-10-14 20:41:43,356][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000073248_75005952.pth... [2023-10-14 20:41:43,357][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000073376_75137024.pth... [2023-10-14 20:41:43,391][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000071840_73564160.pth [2023-10-14 20:41:43,397][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000071680_73400320.pth [2023-10-14 20:41:43,830][61552] Updated weights for policy 0, policy_version 73382 (0.0008) [2023-10-14 20:41:44,214][61552] Updated weights for policy 0, policy_version 73392 (0.0009) [2023-10-14 20:41:44,591][61552] Updated weights for policy 0, policy_version 73402 (0.0009) [2023-10-14 20:41:47,030][61585] Updated weights for policy 1, policy_version 73250 (0.0007) [2023-10-14 20:41:47,396][61585] Updated weights for policy 1, policy_version 73260 (0.0007) [2023-10-14 20:41:47,760][61585] Updated weights for policy 1, policy_version 73270 (0.0009) [2023-10-14 20:41:48,126][61585] Updated weights for policy 1, policy_version 73280 (0.0008) [2023-10-14 20:41:48,343][60425] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 150208512. Throughput: 0: 1668.5, 1: 1693.4. Samples: 37554714. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-10-14 20:41:48,344][60425] Avg episode reward: [(0, '78.270'), (1, '76.440')] [2023-10-14 20:41:48,559][61552] Updated weights for policy 0, policy_version 73412 (0.0008) [2023-10-14 20:41:48,936][61552] Updated weights for policy 0, policy_version 73422 (0.0008) [2023-10-14 20:41:49,295][61552] Updated weights for policy 0, policy_version 73432 (0.0010) [2023-10-14 20:41:52,162][61585] Updated weights for policy 1, policy_version 73290 (0.0007) [2023-10-14 20:41:52,527][61585] Updated weights for policy 1, policy_version 73300 (0.0009) [2023-10-14 20:41:52,900][61585] Updated weights for policy 1, policy_version 73310 (0.0010) [2023-10-14 20:41:53,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 150274048. Throughput: 0: 1667.3, 1: 1696.6. Samples: 37575442. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-10-14 20:41:53,344][60425] Avg episode reward: [(0, '71.290'), (1, '80.830')] [2023-10-14 20:41:53,435][61552] Updated weights for policy 0, policy_version 73442 (0.0011) [2023-10-14 20:41:53,801][61552] Updated weights for policy 0, policy_version 73452 (0.0009) [2023-10-14 20:41:54,168][61552] Updated weights for policy 0, policy_version 73462 (0.0008) [2023-10-14 20:41:54,528][61552] Updated weights for policy 0, policy_version 73472 (0.0009) [2023-10-14 20:41:56,909][61585] Updated weights for policy 1, policy_version 73320 (0.0009) [2023-10-14 20:41:57,286][61585] Updated weights for policy 1, policy_version 73330 (0.0008) [2023-10-14 20:41:57,646][61585] Updated weights for policy 1, policy_version 73340 (0.0008) [2023-10-14 20:41:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 150339584. Throughput: 0: 1669.3, 1: 1671.7. Samples: 37595114. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-10-14 20:41:58,344][60425] Avg episode reward: [(0, '73.600'), (1, '79.190')] [2023-10-14 20:41:58,671][61552] Updated weights for policy 0, policy_version 73482 (0.0008) [2023-10-14 20:41:59,041][61552] Updated weights for policy 0, policy_version 73492 (0.0009) [2023-10-14 20:41:59,408][61552] Updated weights for policy 0, policy_version 73502 (0.0009) [2023-10-14 20:42:01,623][61585] Updated weights for policy 1, policy_version 73350 (0.0008) [2023-10-14 20:42:01,983][61585] Updated weights for policy 1, policy_version 73360 (0.0007) [2023-10-14 20:42:02,341][61585] Updated weights for policy 1, policy_version 73370 (0.0009) [2023-10-14 20:42:03,344][60425] Fps is (10 sec: 13106.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 150405120. Throughput: 0: 1672.4, 1: 1698.6. Samples: 37605494. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-10-14 20:42:03,345][60425] Avg episode reward: [(0, '73.230'), (1, '75.080')] [2023-10-14 20:42:03,373][61552] Updated weights for policy 0, policy_version 73512 (0.0009) [2023-10-14 20:42:03,752][61552] Updated weights for policy 0, policy_version 73522 (0.0009) [2023-10-14 20:42:04,116][61552] Updated weights for policy 0, policy_version 73532 (0.0010) [2023-10-14 20:42:06,546][61585] Updated weights for policy 1, policy_version 73380 (0.0008) [2023-10-14 20:42:06,915][61585] Updated weights for policy 1, policy_version 73390 (0.0009) [2023-10-14 20:42:07,275][61585] Updated weights for policy 1, policy_version 73400 (0.0007) [2023-10-14 20:42:08,121][61552] Updated weights for policy 0, policy_version 73542 (0.0008) [2023-10-14 20:42:08,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 150470656. Throughput: 0: 1675.2, 1: 1687.4. Samples: 37625724. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-10-14 20:42:08,344][60425] Avg episode reward: [(0, '76.250'), (1, '79.520')] [2023-10-14 20:42:08,497][61552] Updated weights for policy 0, policy_version 73552 (0.0009) [2023-10-14 20:42:08,871][61552] Updated weights for policy 0, policy_version 73562 (0.0009) [2023-10-14 20:42:11,496][61585] Updated weights for policy 1, policy_version 73410 (0.0008) [2023-10-14 20:42:11,860][61585] Updated weights for policy 1, policy_version 73420 (0.0008) [2023-10-14 20:42:12,234][61585] Updated weights for policy 1, policy_version 73430 (0.0008) [2023-10-14 20:42:12,599][61585] Updated weights for policy 1, policy_version 73440 (0.0009) [2023-10-14 20:42:13,003][61552] Updated weights for policy 0, policy_version 73572 (0.0007) [2023-10-14 20:42:13,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 150536192. Throughput: 0: 1671.5, 1: 1659.5. Samples: 37645224. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-10-14 20:42:13,344][60425] Avg episode reward: [(0, '72.610'), (1, '80.400')] [2023-10-14 20:42:13,370][61552] Updated weights for policy 0, policy_version 73582 (0.0007) [2023-10-14 20:42:13,727][61552] Updated weights for policy 0, policy_version 73592 (0.0009) [2023-10-14 20:42:16,733][61585] Updated weights for policy 1, policy_version 73450 (0.0010) [2023-10-14 20:42:17,104][61585] Updated weights for policy 1, policy_version 73460 (0.0008) [2023-10-14 20:42:17,458][61585] Updated weights for policy 1, policy_version 73470 (0.0007) [2023-10-14 20:42:17,701][61552] Updated weights for policy 0, policy_version 73602 (0.0008) [2023-10-14 20:42:18,065][61552] Updated weights for policy 0, policy_version 73612 (0.0010) [2023-10-14 20:42:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 150601728. Throughput: 0: 1671.5, 1: 1684.8. Samples: 37655494. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) [2023-10-14 20:42:18,344][60425] Avg episode reward: [(0, '75.440'), (1, '81.270')] [2023-10-14 20:42:18,432][61552] Updated weights for policy 0, policy_version 73622 (0.0011) [2023-10-14 20:42:18,810][61552] Updated weights for policy 0, policy_version 73632 (0.0010) [2023-10-14 20:42:21,535][61585] Updated weights for policy 1, policy_version 73480 (0.0008) [2023-10-14 20:42:21,896][61585] Updated weights for policy 1, policy_version 73490 (0.0010) [2023-10-14 20:42:22,272][61585] Updated weights for policy 1, policy_version 73500 (0.0009) [2023-10-14 20:42:23,031][61552] Updated weights for policy 0, policy_version 73642 (0.0010) [2023-10-14 20:42:23,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 150667264. Throughput: 0: 1666.5, 1: 1678.2. Samples: 37675408. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) [2023-10-14 20:42:23,344][60425] Avg episode reward: [(0, '79.650'), (1, '78.150')] [2023-10-14 20:42:23,395][61552] Updated weights for policy 0, policy_version 73652 (0.0007) [2023-10-14 20:42:23,764][61552] Updated weights for policy 0, policy_version 73662 (0.0008) [2023-10-14 20:42:26,364][61585] Updated weights for policy 1, policy_version 73510 (0.0008) [2023-10-14 20:42:26,730][61585] Updated weights for policy 1, policy_version 73520 (0.0011) [2023-10-14 20:42:27,113][61585] Updated weights for policy 1, policy_version 73530 (0.0010) [2023-10-14 20:42:27,925][61552] Updated weights for policy 0, policy_version 73672 (0.0010) [2023-10-14 20:42:28,309][61552] Updated weights for policy 0, policy_version 73682 (0.0009) [2023-10-14 20:42:28,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 150732800. Throughput: 0: 1669.6, 1: 1668.4. Samples: 37695188. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) [2023-10-14 20:42:28,344][60425] Avg episode reward: [(0, '76.440'), (1, '76.010')] [2023-10-14 20:42:28,672][61552] Updated weights for policy 0, policy_version 73692 (0.0007) [2023-10-14 20:42:31,011][61585] Updated weights for policy 1, policy_version 73540 (0.0010) [2023-10-14 20:42:31,379][61585] Updated weights for policy 1, policy_version 73550 (0.0010) [2023-10-14 20:42:31,750][61585] Updated weights for policy 1, policy_version 73560 (0.0009) [2023-10-14 20:42:32,935][61552] Updated weights for policy 0, policy_version 73702 (0.0010) [2023-10-14 20:42:33,295][61552] Updated weights for policy 0, policy_version 73712 (0.0009) [2023-10-14 20:42:33,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 150798336. Throughput: 0: 1672.5, 1: 1683.3. Samples: 37705722. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) [2023-10-14 20:42:33,344][60425] Avg episode reward: [(0, '76.690'), (1, '79.510')] [2023-10-14 20:42:33,660][61552] Updated weights for policy 0, policy_version 73722 (0.0009) [2023-10-14 20:42:35,868][61585] Updated weights for policy 1, policy_version 73570 (0.0008) [2023-10-14 20:42:36,239][61585] Updated weights for policy 1, policy_version 73580 (0.0009) [2023-10-14 20:42:36,611][61585] Updated weights for policy 1, policy_version 73590 (0.0009) [2023-10-14 20:42:36,982][61585] Updated weights for policy 1, policy_version 73600 (0.0007) [2023-10-14 20:42:37,909][61552] Updated weights for policy 0, policy_version 73732 (0.0009) [2023-10-14 20:42:38,281][61552] Updated weights for policy 0, policy_version 73742 (0.0008) [2023-10-14 20:42:38,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 150863872. Throughput: 0: 1670.0, 1: 1659.0. Samples: 37725246. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) [2023-10-14 20:42:38,344][60425] Avg episode reward: [(0, '77.580'), (1, '79.110')] [2023-10-14 20:42:38,645][61552] Updated weights for policy 0, policy_version 73752 (0.0009) [2023-10-14 20:42:41,059][61585] Updated weights for policy 1, policy_version 73610 (0.0010) [2023-10-14 20:42:41,421][61585] Updated weights for policy 1, policy_version 73620 (0.0010) [2023-10-14 20:42:41,777][61585] Updated weights for policy 1, policy_version 73630 (0.0008) [2023-10-14 20:42:42,703][61552] Updated weights for policy 0, policy_version 73762 (0.0009) [2023-10-14 20:42:43,067][61552] Updated weights for policy 0, policy_version 73772 (0.0009) [2023-10-14 20:42:43,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 150929408. Throughput: 0: 1662.7, 1: 1673.1. Samples: 37745224. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) [2023-10-14 20:42:43,344][60425] Avg episode reward: [(0, '74.210'), (1, '82.250')] [2023-10-14 20:42:43,445][61552] Updated weights for policy 0, policy_version 73782 (0.0009) [2023-10-14 20:42:43,805][61552] Updated weights for policy 0, policy_version 73792 (0.0008) [2023-10-14 20:42:45,941][61585] Updated weights for policy 1, policy_version 73640 (0.0009) [2023-10-14 20:42:46,311][61585] Updated weights for policy 1, policy_version 73650 (0.0007) [2023-10-14 20:42:46,678][61585] Updated weights for policy 1, policy_version 73660 (0.0010) [2023-10-14 20:42:48,025][61552] Updated weights for policy 0, policy_version 73802 (0.0008) [2023-10-14 20:42:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 150994944. Throughput: 0: 1660.1, 1: 1669.7. Samples: 37755334. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) [2023-10-14 20:42:48,344][60425] Avg episode reward: [(0, '76.020'), (1, '76.950')] [2023-10-14 20:42:48,387][61552] Updated weights for policy 0, policy_version 73812 (0.0009) [2023-10-14 20:42:48,757][61552] Updated weights for policy 0, policy_version 73822 (0.0010) [2023-10-14 20:42:50,870][61585] Updated weights for policy 1, policy_version 73670 (0.0007) [2023-10-14 20:42:51,236][61585] Updated weights for policy 1, policy_version 73680 (0.0010) [2023-10-14 20:42:51,607][61585] Updated weights for policy 1, policy_version 73690 (0.0011) [2023-10-14 20:42:52,776][61552] Updated weights for policy 0, policy_version 73832 (0.0011) [2023-10-14 20:42:53,148][61552] Updated weights for policy 0, policy_version 73842 (0.0007) [2023-10-14 20:42:53,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 151060480. Throughput: 0: 1661.2, 1: 1654.4. Samples: 37774926. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) [2023-10-14 20:42:53,344][60425] Avg episode reward: [(0, '75.540'), (1, '81.330')] [2023-10-14 20:42:53,510][61552] Updated weights for policy 0, policy_version 73852 (0.0008) [2023-10-14 20:42:55,672][61585] Updated weights for policy 1, policy_version 73700 (0.0009) [2023-10-14 20:42:56,042][61585] Updated weights for policy 1, policy_version 73710 (0.0007) [2023-10-14 20:42:56,409][61585] Updated weights for policy 1, policy_version 73720 (0.0007) [2023-10-14 20:42:57,552][61552] Updated weights for policy 0, policy_version 73862 (0.0009) [2023-10-14 20:42:57,913][61552] Updated weights for policy 0, policy_version 73872 (0.0010) [2023-10-14 20:42:58,283][61552] Updated weights for policy 0, policy_version 73882 (0.0007) [2023-10-14 20:42:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 151126016. Throughput: 0: 1652.9, 1: 1678.2. Samples: 37795124. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) [2023-10-14 20:42:58,344][60425] Avg episode reward: [(0, '75.370'), (1, '75.540')] [2023-10-14 20:43:00,392][61585] Updated weights for policy 1, policy_version 73730 (0.0009) [2023-10-14 20:43:00,752][61585] Updated weights for policy 1, policy_version 73740 (0.0007) [2023-10-14 20:43:01,117][61585] Updated weights for policy 1, policy_version 73750 (0.0009) [2023-10-14 20:43:01,477][61585] Updated weights for policy 1, policy_version 73760 (0.0011) [2023-10-14 20:43:02,331][61552] Updated weights for policy 0, policy_version 73892 (0.0008) [2023-10-14 20:43:02,704][61552] Updated weights for policy 0, policy_version 73902 (0.0009) [2023-10-14 20:43:03,083][61552] Updated weights for policy 0, policy_version 73912 (0.0008) [2023-10-14 20:43:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 151191552. Throughput: 0: 1666.4, 1: 1669.6. Samples: 37805616. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) [2023-10-14 20:43:03,344][60425] Avg episode reward: [(0, '73.050'), (1, '78.280')] [2023-10-14 20:43:05,637][61585] Updated weights for policy 1, policy_version 73770 (0.0009) [2023-10-14 20:43:06,001][61585] Updated weights for policy 1, policy_version 73780 (0.0011) [2023-10-14 20:43:06,365][61585] Updated weights for policy 1, policy_version 73790 (0.0011) [2023-10-14 20:43:07,166][61552] Updated weights for policy 0, policy_version 73922 (0.0008) [2023-10-14 20:43:07,529][61552] Updated weights for policy 0, policy_version 73932 (0.0010) [2023-10-14 20:43:07,899][61552] Updated weights for policy 0, policy_version 73942 (0.0008) [2023-10-14 20:43:08,278][61552] Updated weights for policy 0, policy_version 73952 (0.0007) [2023-10-14 20:43:08,343][60425] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 151289856. Throughput: 0: 1671.5, 1: 1662.2. Samples: 37825424. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-14 20:43:08,344][60425] Avg episode reward: [(0, '74.360'), (1, '79.920')] [2023-10-14 20:43:10,543][61585] Updated weights for policy 1, policy_version 73800 (0.0008) [2023-10-14 20:43:10,906][61585] Updated weights for policy 1, policy_version 73810 (0.0011) [2023-10-14 20:43:11,276][61585] Updated weights for policy 1, policy_version 73820 (0.0007) [2023-10-14 20:43:12,399][61552] Updated weights for policy 0, policy_version 73962 (0.0007) [2023-10-14 20:43:12,771][61552] Updated weights for policy 0, policy_version 73972 (0.0007) [2023-10-14 20:43:13,146][61552] Updated weights for policy 0, policy_version 73982 (0.0008) [2023-10-14 20:43:13,343][60425] Fps is (10 sec: 16384.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 151355392. Throughput: 0: 1659.2, 1: 1673.3. Samples: 37845148. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-14 20:43:13,344][60425] Avg episode reward: [(0, '75.950'), (1, '75.580')] [2023-10-14 20:43:15,500][61585] Updated weights for policy 1, policy_version 73830 (0.0010) [2023-10-14 20:43:15,863][61585] Updated weights for policy 1, policy_version 73840 (0.0010) [2023-10-14 20:43:16,239][61585] Updated weights for policy 1, policy_version 73850 (0.0011) [2023-10-14 20:43:17,111][61552] Updated weights for policy 0, policy_version 73992 (0.0009) [2023-10-14 20:43:17,493][61552] Updated weights for policy 0, policy_version 74002 (0.0009) [2023-10-14 20:43:17,856][61552] Updated weights for policy 0, policy_version 74012 (0.0008) [2023-10-14 20:43:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 151420928. Throughput: 0: 1675.2, 1: 1656.5. Samples: 37855652. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-14 20:43:18,344][60425] Avg episode reward: [(0, '76.560'), (1, '76.790')] [2023-10-14 20:43:20,478][61585] Updated weights for policy 1, policy_version 73860 (0.0009) [2023-10-14 20:43:20,840][61585] Updated weights for policy 1, policy_version 73870 (0.0008) [2023-10-14 20:43:21,209][61585] Updated weights for policy 1, policy_version 73880 (0.0008) [2023-10-14 20:43:21,828][61552] Updated weights for policy 0, policy_version 74022 (0.0009) [2023-10-14 20:43:22,188][61552] Updated weights for policy 0, policy_version 74032 (0.0008) [2023-10-14 20:43:22,556][61552] Updated weights for policy 0, policy_version 74042 (0.0011) [2023-10-14 20:43:23,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 151486464. Throughput: 0: 1679.6, 1: 1656.5. Samples: 37875370. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-14 20:43:23,344][60425] Avg episode reward: [(0, '76.710'), (1, '72.410')] [2023-10-14 20:43:25,292][61585] Updated weights for policy 1, policy_version 73890 (0.0010) [2023-10-14 20:43:25,663][61585] Updated weights for policy 1, policy_version 73900 (0.0008) [2023-10-14 20:43:26,019][61585] Updated weights for policy 1, policy_version 73910 (0.0007) [2023-10-14 20:43:26,390][61585] Updated weights for policy 1, policy_version 73920 (0.0009) [2023-10-14 20:43:26,727][61552] Updated weights for policy 0, policy_version 74052 (0.0010) [2023-10-14 20:43:27,093][61552] Updated weights for policy 0, policy_version 74062 (0.0010) [2023-10-14 20:43:27,470][61552] Updated weights for policy 0, policy_version 74072 (0.0010) [2023-10-14 20:43:28,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 151552000. Throughput: 0: 1657.6, 1: 1665.0. Samples: 37894742. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-14 20:43:28,344][60425] Avg episode reward: [(0, '70.300'), (1, '75.440')] [2023-10-14 20:43:30,520][61585] Updated weights for policy 1, policy_version 73930 (0.0009) [2023-10-14 20:43:30,881][61585] Updated weights for policy 1, policy_version 73940 (0.0009) [2023-10-14 20:43:31,234][61585] Updated weights for policy 1, policy_version 73950 (0.0009) [2023-10-14 20:43:31,571][61552] Updated weights for policy 0, policy_version 74082 (0.0009) [2023-10-14 20:43:31,934][61552] Updated weights for policy 0, policy_version 74092 (0.0011) [2023-10-14 20:43:32,295][61552] Updated weights for policy 0, policy_version 74102 (0.0010) [2023-10-14 20:43:32,665][61552] Updated weights for policy 0, policy_version 74112 (0.0007) [2023-10-14 20:43:33,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 151617536. Throughput: 0: 1683.6, 1: 1654.6. Samples: 37905552. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-14 20:43:33,344][60425] Avg episode reward: [(0, '72.280'), (1, '72.250')] [2023-10-14 20:43:35,403][61585] Updated weights for policy 1, policy_version 73960 (0.0008) [2023-10-14 20:43:35,769][61585] Updated weights for policy 1, policy_version 73970 (0.0008) [2023-10-14 20:43:36,140][61585] Updated weights for policy 1, policy_version 73980 (0.0009) [2023-10-14 20:43:36,798][61552] Updated weights for policy 0, policy_version 74122 (0.0008) [2023-10-14 20:43:37,156][61552] Updated weights for policy 0, policy_version 74132 (0.0008) [2023-10-14 20:43:37,525][61552] Updated weights for policy 0, policy_version 74142 (0.0008) [2023-10-14 20:43:38,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 151683072. Throughput: 0: 1680.3, 1: 1660.1. Samples: 37925246. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-14 20:43:38,344][60425] Avg episode reward: [(0, '76.270'), (1, '75.350')] [2023-10-14 20:43:40,133][61585] Updated weights for policy 1, policy_version 73990 (0.0010) [2023-10-14 20:43:40,497][61585] Updated weights for policy 1, policy_version 74000 (0.0009) [2023-10-14 20:43:40,871][61585] Updated weights for policy 1, policy_version 74010 (0.0011) [2023-10-14 20:43:41,628][61552] Updated weights for policy 0, policy_version 74152 (0.0007) [2023-10-14 20:43:42,007][61552] Updated weights for policy 0, policy_version 74162 (0.0011) [2023-10-14 20:43:42,367][61552] Updated weights for policy 0, policy_version 74172 (0.0008) [2023-10-14 20:43:43,344][60425] Fps is (10 sec: 13106.7, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 151748608. Throughput: 0: 1666.3, 1: 1661.6. Samples: 37944878. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-14 20:43:43,345][60425] Avg episode reward: [(0, '74.610'), (1, '75.460')] [2023-10-14 20:43:43,358][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000074016_75792384.pth... [2023-10-14 20:43:43,358][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000074176_75956224.pth... [2023-10-14 20:43:43,394][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000072608_74350592.pth [2023-10-14 20:43:43,398][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000072448_74186752.pth [2023-10-14 20:43:44,893][61585] Updated weights for policy 1, policy_version 74020 (0.0009) [2023-10-14 20:43:45,261][61585] Updated weights for policy 1, policy_version 74030 (0.0009) [2023-10-14 20:43:45,625][61585] Updated weights for policy 1, policy_version 74040 (0.0008) [2023-10-14 20:43:46,470][61552] Updated weights for policy 0, policy_version 74182 (0.0009) [2023-10-14 20:43:46,844][61552] Updated weights for policy 0, policy_version 74192 (0.0010) [2023-10-14 20:43:47,218][61552] Updated weights for policy 0, policy_version 74202 (0.0010) [2023-10-14 20:43:48,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 151814144. Throughput: 0: 1681.0, 1: 1647.2. Samples: 37955382. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-14 20:43:48,344][60425] Avg episode reward: [(0, '72.790'), (1, '78.020')] [2023-10-14 20:43:49,823][61585] Updated weights for policy 1, policy_version 74050 (0.0009) [2023-10-14 20:43:50,183][61585] Updated weights for policy 1, policy_version 74060 (0.0008) [2023-10-14 20:43:50,548][61585] Updated weights for policy 1, policy_version 74070 (0.0007) [2023-10-14 20:43:50,899][61585] Updated weights for policy 1, policy_version 74080 (0.0007) [2023-10-14 20:43:51,043][61552] Updated weights for policy 0, policy_version 74212 (0.0008) [2023-10-14 20:43:51,400][61552] Updated weights for policy 0, policy_version 74222 (0.0008) [2023-10-14 20:43:51,768][61552] Updated weights for policy 0, policy_version 74232 (0.0007) [2023-10-14 20:43:53,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 151879680. Throughput: 0: 1667.6, 1: 1660.2. Samples: 37975176. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:43:53,344][60425] Avg episode reward: [(0, '74.510'), (1, '76.690')] [2023-10-14 20:43:55,048][61585] Updated weights for policy 1, policy_version 74090 (0.0008) [2023-10-14 20:43:55,405][61585] Updated weights for policy 1, policy_version 74100 (0.0011) [2023-10-14 20:43:55,738][61552] Updated weights for policy 0, policy_version 74242 (0.0008) [2023-10-14 20:43:55,767][61585] Updated weights for policy 1, policy_version 74110 (0.0008) [2023-10-14 20:43:56,108][61552] Updated weights for policy 0, policy_version 74252 (0.0008) [2023-10-14 20:43:56,473][61552] Updated weights for policy 0, policy_version 74262 (0.0009) [2023-10-14 20:43:56,837][61552] Updated weights for policy 0, policy_version 74272 (0.0011) [2023-10-14 20:43:58,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 151945216. Throughput: 0: 1676.7, 1: 1665.2. Samples: 37995534. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:43:58,345][60425] Avg episode reward: [(0, '72.530'), (1, '80.840')] [2023-10-14 20:43:59,628][61585] Updated weights for policy 1, policy_version 74120 (0.0009) [2023-10-14 20:43:59,989][61585] Updated weights for policy 1, policy_version 74130 (0.0009) [2023-10-14 20:44:00,359][61585] Updated weights for policy 1, policy_version 74140 (0.0009) [2023-10-14 20:44:00,769][61552] Updated weights for policy 0, policy_version 74282 (0.0007) [2023-10-14 20:44:01,139][61552] Updated weights for policy 0, policy_version 74292 (0.0008) [2023-10-14 20:44:01,493][61552] Updated weights for policy 0, policy_version 74302 (0.0008) [2023-10-14 20:44:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 152010752. Throughput: 0: 1682.5, 1: 1653.1. Samples: 38005756. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:44:03,344][60425] Avg episode reward: [(0, '71.600'), (1, '77.840')] [2023-10-14 20:44:04,399][61585] Updated weights for policy 1, policy_version 74150 (0.0009) [2023-10-14 20:44:04,768][61585] Updated weights for policy 1, policy_version 74160 (0.0008) [2023-10-14 20:44:05,132][61585] Updated weights for policy 1, policy_version 74170 (0.0011) [2023-10-14 20:44:05,717][61552] Updated weights for policy 0, policy_version 74312 (0.0009) [2023-10-14 20:44:06,087][61552] Updated weights for policy 0, policy_version 74322 (0.0009) [2023-10-14 20:44:06,450][61552] Updated weights for policy 0, policy_version 74332 (0.0008) [2023-10-14 20:44:08,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 152076288. Throughput: 0: 1659.0, 1: 1679.6. Samples: 38025606. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:44:08,344][60425] Avg episode reward: [(0, '72.390'), (1, '76.330')] [2023-10-14 20:44:09,086][61585] Updated weights for policy 1, policy_version 74180 (0.0008) [2023-10-14 20:44:09,447][61585] Updated weights for policy 1, policy_version 74190 (0.0009) [2023-10-14 20:44:09,809][61585] Updated weights for policy 1, policy_version 74200 (0.0011) [2023-10-14 20:44:10,586][61552] Updated weights for policy 0, policy_version 74342 (0.0008) [2023-10-14 20:44:10,944][61552] Updated weights for policy 0, policy_version 74352 (0.0009) [2023-10-14 20:44:11,315][61552] Updated weights for policy 0, policy_version 74362 (0.0010) [2023-10-14 20:44:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 152141824. Throughput: 0: 1691.7, 1: 1685.9. Samples: 38046732. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:44:13,344][60425] Avg episode reward: [(0, '68.710'), (1, '79.080')] [2023-10-14 20:44:13,943][61585] Updated weights for policy 1, policy_version 74210 (0.0009) [2023-10-14 20:44:14,300][61585] Updated weights for policy 1, policy_version 74220 (0.0010) [2023-10-14 20:44:14,668][61585] Updated weights for policy 1, policy_version 74230 (0.0009) [2023-10-14 20:44:15,031][61585] Updated weights for policy 1, policy_version 74240 (0.0009) [2023-10-14 20:44:15,263][61552] Updated weights for policy 0, policy_version 74372 (0.0010) [2023-10-14 20:44:15,633][61552] Updated weights for policy 0, policy_version 74382 (0.0007) [2023-10-14 20:44:16,000][61552] Updated weights for policy 0, policy_version 74392 (0.0008) [2023-10-14 20:44:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 152207360. Throughput: 0: 1683.8, 1: 1673.5. Samples: 38056630. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:44:18,344][60425] Avg episode reward: [(0, '68.970'), (1, '77.740')] [2023-10-14 20:44:19,177][61585] Updated weights for policy 1, policy_version 74250 (0.0010) [2023-10-14 20:44:19,546][61585] Updated weights for policy 1, policy_version 74260 (0.0008) [2023-10-14 20:44:19,917][61585] Updated weights for policy 1, policy_version 74270 (0.0007) [2023-10-14 20:44:20,070][61552] Updated weights for policy 0, policy_version 74402 (0.0010) [2023-10-14 20:44:20,442][61552] Updated weights for policy 0, policy_version 74412 (0.0009) [2023-10-14 20:44:20,810][61552] Updated weights for policy 0, policy_version 74422 (0.0007) [2023-10-14 20:44:21,167][61552] Updated weights for policy 0, policy_version 74432 (0.0008) [2023-10-14 20:44:23,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 152272896. Throughput: 0: 1672.7, 1: 1690.5. Samples: 38076592. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:44:23,344][60425] Avg episode reward: [(0, '68.800'), (1, '78.740')] [2023-10-14 20:44:24,167][61585] Updated weights for policy 1, policy_version 74280 (0.0008) [2023-10-14 20:44:24,551][61585] Updated weights for policy 1, policy_version 74290 (0.0008) [2023-10-14 20:44:24,917][61585] Updated weights for policy 1, policy_version 74300 (0.0007) [2023-10-14 20:44:25,226][61552] Updated weights for policy 0, policy_version 74442 (0.0009) [2023-10-14 20:44:25,594][61552] Updated weights for policy 0, policy_version 74452 (0.0008) [2023-10-14 20:44:25,964][61552] Updated weights for policy 0, policy_version 74462 (0.0008) [2023-10-14 20:44:28,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 152338432. Throughput: 0: 1692.9, 1: 1690.8. Samples: 38097140. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:44:28,344][60425] Avg episode reward: [(0, '68.240'), (1, '72.740')] [2023-10-14 20:44:28,957][61585] Updated weights for policy 1, policy_version 74310 (0.0008) [2023-10-14 20:44:29,327][61585] Updated weights for policy 1, policy_version 74320 (0.0008) [2023-10-14 20:44:29,683][61585] Updated weights for policy 1, policy_version 74330 (0.0008) [2023-10-14 20:44:30,121][61552] Updated weights for policy 0, policy_version 74472 (0.0007) [2023-10-14 20:44:30,479][61552] Updated weights for policy 0, policy_version 74482 (0.0008) [2023-10-14 20:44:30,850][61552] Updated weights for policy 0, policy_version 74492 (0.0009) [2023-10-14 20:44:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 152403968. Throughput: 0: 1672.5, 1: 1682.5. Samples: 38106356. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:44:33,344][60425] Avg episode reward: [(0, '70.670'), (1, '78.920')] [2023-10-14 20:44:33,792][61585] Updated weights for policy 1, policy_version 74340 (0.0008) [2023-10-14 20:44:34,158][61585] Updated weights for policy 1, policy_version 74350 (0.0007) [2023-10-14 20:44:34,521][61585] Updated weights for policy 1, policy_version 74360 (0.0007) [2023-10-14 20:44:34,876][61552] Updated weights for policy 0, policy_version 74502 (0.0008) [2023-10-14 20:44:35,251][61552] Updated weights for policy 0, policy_version 74512 (0.0009) [2023-10-14 20:44:35,619][61552] Updated weights for policy 0, policy_version 74522 (0.0008) [2023-10-14 20:44:38,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 152469504. Throughput: 0: 1677.7, 1: 1688.6. Samples: 38126660. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:44:38,344][60425] Avg episode reward: [(0, '71.110'), (1, '77.300')] [2023-10-14 20:44:38,574][61585] Updated weights for policy 1, policy_version 74370 (0.0008) [2023-10-14 20:44:38,937][61585] Updated weights for policy 1, policy_version 74380 (0.0011) [2023-10-14 20:44:39,307][61585] Updated weights for policy 1, policy_version 74390 (0.0010) [2023-10-14 20:44:39,664][61585] Updated weights for policy 1, policy_version 74400 (0.0009) [2023-10-14 20:44:39,688][61552] Updated weights for policy 0, policy_version 74532 (0.0009) [2023-10-14 20:44:40,047][61552] Updated weights for policy 0, policy_version 74542 (0.0009) [2023-10-14 20:44:40,412][61552] Updated weights for policy 0, policy_version 74552 (0.0008) [2023-10-14 20:44:43,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 152535040. Throughput: 0: 1689.9, 1: 1686.6. Samples: 38147474. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:44:43,344][60425] Avg episode reward: [(0, '72.790'), (1, '75.900')] [2023-10-14 20:44:43,824][61585] Updated weights for policy 1, policy_version 74410 (0.0009) [2023-10-14 20:44:44,186][61585] Updated weights for policy 1, policy_version 74420 (0.0010) [2023-10-14 20:44:44,489][61552] Updated weights for policy 0, policy_version 74562 (0.0008) [2023-10-14 20:44:44,556][61585] Updated weights for policy 1, policy_version 74430 (0.0008) [2023-10-14 20:44:44,859][61552] Updated weights for policy 0, policy_version 74572 (0.0008) [2023-10-14 20:44:45,227][61552] Updated weights for policy 0, policy_version 74582 (0.0007) [2023-10-14 20:44:45,591][61552] Updated weights for policy 0, policy_version 74592 (0.0008) [2023-10-14 20:44:48,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 152600576. Throughput: 0: 1664.8, 1: 1683.0. Samples: 38156406. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:44:48,344][60425] Avg episode reward: [(0, '75.990'), (1, '75.310')] [2023-10-14 20:44:48,882][61585] Updated weights for policy 1, policy_version 74440 (0.0010) [2023-10-14 20:44:49,249][61585] Updated weights for policy 1, policy_version 74450 (0.0009) [2023-10-14 20:44:49,611][61585] Updated weights for policy 1, policy_version 74460 (0.0009) [2023-10-14 20:44:49,677][61552] Updated weights for policy 0, policy_version 74602 (0.0009) [2023-10-14 20:44:50,043][61552] Updated weights for policy 0, policy_version 74612 (0.0007) [2023-10-14 20:44:50,406][61552] Updated weights for policy 0, policy_version 74622 (0.0008) [2023-10-14 20:44:53,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 152666112. Throughput: 0: 1685.2, 1: 1676.0. Samples: 38176860. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:44:53,344][60425] Avg episode reward: [(0, '74.980'), (1, '77.350')] [2023-10-14 20:44:53,413][61585] Updated weights for policy 1, policy_version 74470 (0.0008) [2023-10-14 20:44:53,784][61585] Updated weights for policy 1, policy_version 74480 (0.0008) [2023-10-14 20:44:54,153][61585] Updated weights for policy 1, policy_version 74490 (0.0010) [2023-10-14 20:44:54,751][61552] Updated weights for policy 0, policy_version 74632 (0.0009) [2023-10-14 20:44:55,129][61552] Updated weights for policy 0, policy_version 74642 (0.0007) [2023-10-14 20:44:55,498][61552] Updated weights for policy 0, policy_version 74652 (0.0007) [2023-10-14 20:44:58,101][61585] Updated weights for policy 1, policy_version 74500 (0.0009) [2023-10-14 20:44:58,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 152731648. Throughput: 0: 1678.7, 1: 1675.0. Samples: 38197650. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:44:58,344][60425] Avg episode reward: [(0, '73.770'), (1, '76.250')] [2023-10-14 20:44:58,467][61585] Updated weights for policy 1, policy_version 74510 (0.0008) [2023-10-14 20:44:58,824][61585] Updated weights for policy 1, policy_version 74520 (0.0007) [2023-10-14 20:44:59,409][61552] Updated weights for policy 0, policy_version 74662 (0.0007) [2023-10-14 20:44:59,778][61552] Updated weights for policy 0, policy_version 74672 (0.0007) [2023-10-14 20:45:00,144][61552] Updated weights for policy 0, policy_version 74682 (0.0010) [2023-10-14 20:45:02,888][61585] Updated weights for policy 1, policy_version 74530 (0.0008) [2023-10-14 20:45:03,254][61585] Updated weights for policy 1, policy_version 74540 (0.0007) [2023-10-14 20:45:03,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 152797184. Throughput: 0: 1664.9, 1: 1675.6. Samples: 38206954. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:45:03,344][60425] Avg episode reward: [(0, '74.980'), (1, '77.780')] [2023-10-14 20:45:03,619][61585] Updated weights for policy 1, policy_version 74550 (0.0009) [2023-10-14 20:45:03,984][61585] Updated weights for policy 1, policy_version 74560 (0.0010) [2023-10-14 20:45:04,299][61552] Updated weights for policy 0, policy_version 74692 (0.0009) [2023-10-14 20:45:04,668][61552] Updated weights for policy 0, policy_version 74702 (0.0007) [2023-10-14 20:45:05,026][61552] Updated weights for policy 0, policy_version 74712 (0.0009) [2023-10-14 20:45:08,210][61585] Updated weights for policy 1, policy_version 74570 (0.0010) [2023-10-14 20:45:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 152862720. Throughput: 0: 1677.1, 1: 1673.9. Samples: 38227388. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:45:08,345][60425] Avg episode reward: [(0, '75.500'), (1, '79.900')] [2023-10-14 20:45:08,570][61585] Updated weights for policy 1, policy_version 74580 (0.0007) [2023-10-14 20:45:08,940][61585] Updated weights for policy 1, policy_version 74590 (0.0007) [2023-10-14 20:45:09,187][61552] Updated weights for policy 0, policy_version 74722 (0.0007) [2023-10-14 20:45:09,552][61552] Updated weights for policy 0, policy_version 74732 (0.0007) [2023-10-14 20:45:09,920][61552] Updated weights for policy 0, policy_version 74742 (0.0010) [2023-10-14 20:45:10,293][61552] Updated weights for policy 0, policy_version 74752 (0.0010) [2023-10-14 20:45:13,201][61585] Updated weights for policy 1, policy_version 74600 (0.0007) [2023-10-14 20:45:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 152928256. Throughput: 0: 1679.1, 1: 1669.4. Samples: 38247824. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:45:13,344][60425] Avg episode reward: [(0, '75.710'), (1, '72.040')] [2023-10-14 20:45:13,587][61585] Updated weights for policy 1, policy_version 74610 (0.0008) [2023-10-14 20:45:13,962][61585] Updated weights for policy 1, policy_version 74620 (0.0010) [2023-10-14 20:45:14,361][61552] Updated weights for policy 0, policy_version 74762 (0.0008) [2023-10-14 20:45:14,727][61552] Updated weights for policy 0, policy_version 74772 (0.0009) [2023-10-14 20:45:15,103][61552] Updated weights for policy 0, policy_version 74782 (0.0008) [2023-10-14 20:45:18,096][61585] Updated weights for policy 1, policy_version 74630 (0.0009) [2023-10-14 20:45:18,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 152993792. Throughput: 0: 1673.8, 1: 1667.8. Samples: 38256726. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:45:18,344][60425] Avg episode reward: [(0, '75.200'), (1, '75.050')] [2023-10-14 20:45:18,466][61585] Updated weights for policy 1, policy_version 74640 (0.0009) [2023-10-14 20:45:18,828][61585] Updated weights for policy 1, policy_version 74650 (0.0009) [2023-10-14 20:45:19,029][61552] Updated weights for policy 0, policy_version 74792 (0.0009) [2023-10-14 20:45:19,406][61552] Updated weights for policy 0, policy_version 74802 (0.0009) [2023-10-14 20:45:19,766][61552] Updated weights for policy 0, policy_version 74812 (0.0008) [2023-10-14 20:45:23,078][61585] Updated weights for policy 1, policy_version 74660 (0.0007) [2023-10-14 20:45:23,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 153059328. Throughput: 0: 1682.0, 1: 1663.3. Samples: 38277198. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:45:23,344][60425] Avg episode reward: [(0, '74.730'), (1, '74.740')] [2023-10-14 20:45:23,444][61585] Updated weights for policy 1, policy_version 74670 (0.0008) [2023-10-14 20:45:23,809][61585] Updated weights for policy 1, policy_version 74680 (0.0009) [2023-10-14 20:45:23,948][61552] Updated weights for policy 0, policy_version 74822 (0.0009) [2023-10-14 20:45:24,315][61552] Updated weights for policy 0, policy_version 74832 (0.0009) [2023-10-14 20:45:24,676][61552] Updated weights for policy 0, policy_version 74842 (0.0009) [2023-10-14 20:45:27,852][61585] Updated weights for policy 1, policy_version 74690 (0.0009) [2023-10-14 20:45:28,211][61585] Updated weights for policy 1, policy_version 74700 (0.0011) [2023-10-14 20:45:28,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 153124864. Throughput: 0: 1674.3, 1: 1663.5. Samples: 38297674. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:45:28,344][60425] Avg episode reward: [(0, '72.770'), (1, '78.570')] [2023-10-14 20:45:28,575][61585] Updated weights for policy 1, policy_version 74710 (0.0009) [2023-10-14 20:45:28,695][61552] Updated weights for policy 0, policy_version 74852 (0.0008) [2023-10-14 20:45:28,935][61585] Updated weights for policy 1, policy_version 74720 (0.0009) [2023-10-14 20:45:29,062][61552] Updated weights for policy 0, policy_version 74862 (0.0009) [2023-10-14 20:45:29,427][61552] Updated weights for policy 0, policy_version 74872 (0.0010) [2023-10-14 20:45:33,058][61585] Updated weights for policy 1, policy_version 74730 (0.0007) [2023-10-14 20:45:33,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 153190400. Throughput: 0: 1677.9, 1: 1665.6. Samples: 38306868. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-14 20:45:33,344][60425] Avg episode reward: [(0, '71.180'), (1, '77.340')] [2023-10-14 20:45:33,415][61585] Updated weights for policy 1, policy_version 74740 (0.0008) [2023-10-14 20:45:33,507][61552] Updated weights for policy 0, policy_version 74882 (0.0007) [2023-10-14 20:45:33,789][61585] Updated weights for policy 1, policy_version 74750 (0.0007) [2023-10-14 20:45:33,876][61552] Updated weights for policy 0, policy_version 74892 (0.0009) [2023-10-14 20:45:34,240][61552] Updated weights for policy 0, policy_version 74902 (0.0008) [2023-10-14 20:45:34,609][61552] Updated weights for policy 0, policy_version 74912 (0.0008) [2023-10-14 20:45:37,941][61585] Updated weights for policy 1, policy_version 74760 (0.0007) [2023-10-14 20:45:38,301][61585] Updated weights for policy 1, policy_version 74770 (0.0007) [2023-10-14 20:45:38,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 153255936. Throughput: 0: 1678.1, 1: 1664.7. Samples: 38327288. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-14 20:45:38,344][60425] Avg episode reward: [(0, '75.440'), (1, '76.820')] [2023-10-14 20:45:38,667][61585] Updated weights for policy 1, policy_version 74780 (0.0008) [2023-10-14 20:45:38,832][61552] Updated weights for policy 0, policy_version 74922 (0.0010) [2023-10-14 20:45:39,200][61552] Updated weights for policy 0, policy_version 74932 (0.0009) [2023-10-14 20:45:39,570][61552] Updated weights for policy 0, policy_version 74942 (0.0008) [2023-10-14 20:45:42,832][61585] Updated weights for policy 1, policy_version 74790 (0.0010) [2023-10-14 20:45:43,195][61585] Updated weights for policy 1, policy_version 74800 (0.0010) [2023-10-14 20:45:43,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 153321472. Throughput: 0: 1680.4, 1: 1657.6. Samples: 38347856. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-14 20:45:43,345][60425] Avg episode reward: [(0, '77.730'), (1, '75.480')] [2023-10-14 20:45:43,501][61552] Updated weights for policy 0, policy_version 74952 (0.0007) [2023-10-14 20:45:43,558][61585] Updated weights for policy 1, policy_version 74810 (0.0007) [2023-10-14 20:45:43,775][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000074816_76611584.pth... [2023-10-14 20:45:43,808][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000073248_75005952.pth [2023-10-14 20:45:43,872][61552] Updated weights for policy 0, policy_version 74962 (0.0008) [2023-10-14 20:45:44,242][61552] Updated weights for policy 0, policy_version 74972 (0.0007) [2023-10-14 20:45:44,384][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000074976_76775424.pth... [2023-10-14 20:45:44,422][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000073376_75137024.pth [2023-10-14 20:45:47,678][61585] Updated weights for policy 1, policy_version 74820 (0.0008) [2023-10-14 20:45:48,044][61585] Updated weights for policy 1, policy_version 74830 (0.0009) [2023-10-14 20:45:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 153387008. Throughput: 0: 1680.8, 1: 1657.3. Samples: 38357166. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-14 20:45:48,344][60425] Avg episode reward: [(0, '79.290'), (1, '75.980')] [2023-10-14 20:45:48,371][61552] Updated weights for policy 0, policy_version 74982 (0.0007) [2023-10-14 20:45:48,417][61585] Updated weights for policy 1, policy_version 74840 (0.0009) [2023-10-14 20:45:48,740][61552] Updated weights for policy 0, policy_version 74992 (0.0008) [2023-10-14 20:45:49,109][61552] Updated weights for policy 0, policy_version 75002 (0.0008) [2023-10-14 20:45:52,607][61585] Updated weights for policy 1, policy_version 74850 (0.0008) [2023-10-14 20:45:52,966][61585] Updated weights for policy 1, policy_version 74860 (0.0007) [2023-10-14 20:45:53,044][61552] Updated weights for policy 0, policy_version 75012 (0.0008) [2023-10-14 20:45:53,324][61585] Updated weights for policy 1, policy_version 74870 (0.0008) [2023-10-14 20:45:53,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 153452544. Throughput: 0: 1680.5, 1: 1659.0. Samples: 38377666. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-14 20:45:53,344][60425] Avg episode reward: [(0, '76.080'), (1, '77.460')] [2023-10-14 20:45:53,412][61552] Updated weights for policy 0, policy_version 75022 (0.0007) [2023-10-14 20:45:53,692][61585] Updated weights for policy 1, policy_version 74880 (0.0008) [2023-10-14 20:45:53,778][61552] Updated weights for policy 0, policy_version 75032 (0.0008) [2023-10-14 20:45:57,905][61552] Updated weights for policy 0, policy_version 75042 (0.0010) [2023-10-14 20:45:58,067][61585] Updated weights for policy 1, policy_version 74890 (0.0010) [2023-10-14 20:45:58,276][61552] Updated weights for policy 0, policy_version 75052 (0.0008) [2023-10-14 20:45:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 153518080. Throughput: 0: 1684.0, 1: 1655.8. Samples: 38398114. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-14 20:45:58,344][60425] Avg episode reward: [(0, '75.070'), (1, '76.860')] [2023-10-14 20:45:58,441][61585] Updated weights for policy 1, policy_version 74900 (0.0008) [2023-10-14 20:45:58,649][61552] Updated weights for policy 0, policy_version 75062 (0.0008) [2023-10-14 20:45:58,801][61585] Updated weights for policy 1, policy_version 74910 (0.0007) [2023-10-14 20:45:59,019][61552] Updated weights for policy 0, policy_version 75072 (0.0009) [2023-10-14 20:46:03,014][61585] Updated weights for policy 1, policy_version 74920 (0.0007) [2023-10-14 20:46:03,076][61552] Updated weights for policy 0, policy_version 75082 (0.0007) [2023-10-14 20:46:03,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 153583616. Throughput: 0: 1681.7, 1: 1660.1. Samples: 38407106. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-14 20:46:03,344][60425] Avg episode reward: [(0, '76.170'), (1, '75.180')] [2023-10-14 20:46:03,384][61585] Updated weights for policy 1, policy_version 74930 (0.0007) [2023-10-14 20:46:03,450][61552] Updated weights for policy 0, policy_version 75092 (0.0007) [2023-10-14 20:46:03,750][61585] Updated weights for policy 1, policy_version 74940 (0.0009) [2023-10-14 20:46:03,815][61552] Updated weights for policy 0, policy_version 75102 (0.0010) [2023-10-14 20:46:07,738][61585] Updated weights for policy 1, policy_version 74950 (0.0010) [2023-10-14 20:46:08,054][61552] Updated weights for policy 0, policy_version 75112 (0.0008) [2023-10-14 20:46:08,107][61585] Updated weights for policy 1, policy_version 74960 (0.0008) [2023-10-14 20:46:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 153649152. Throughput: 0: 1682.6, 1: 1656.6. Samples: 38427462. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-14 20:46:08,344][60425] Avg episode reward: [(0, '78.570'), (1, '73.990')] [2023-10-14 20:46:08,416][61552] Updated weights for policy 0, policy_version 75122 (0.0008) [2023-10-14 20:46:08,470][61585] Updated weights for policy 1, policy_version 74970 (0.0009) [2023-10-14 20:46:08,792][61552] Updated weights for policy 0, policy_version 75132 (0.0010) [2023-10-14 20:46:12,531][61585] Updated weights for policy 1, policy_version 74980 (0.0009) [2023-10-14 20:46:12,864][61552] Updated weights for policy 0, policy_version 75142 (0.0009) [2023-10-14 20:46:12,890][61585] Updated weights for policy 1, policy_version 74990 (0.0008) [2023-10-14 20:46:13,240][61552] Updated weights for policy 0, policy_version 75152 (0.0009) [2023-10-14 20:46:13,249][61585] Updated weights for policy 1, policy_version 75000 (0.0007) [2023-10-14 20:46:13,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 153714688. Throughput: 0: 1683.3, 1: 1653.8. Samples: 38447844. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-14 20:46:13,344][60425] Avg episode reward: [(0, '76.550'), (1, '76.300')] [2023-10-14 20:46:13,611][61552] Updated weights for policy 0, policy_version 75162 (0.0008) [2023-10-14 20:46:17,496][61552] Updated weights for policy 0, policy_version 75172 (0.0009) [2023-10-14 20:46:17,611][61585] Updated weights for policy 1, policy_version 75010 (0.0007) [2023-10-14 20:46:17,870][61552] Updated weights for policy 0, policy_version 75182 (0.0007) [2023-10-14 20:46:17,981][61585] Updated weights for policy 1, policy_version 75020 (0.0007) [2023-10-14 20:46:18,246][61552] Updated weights for policy 0, policy_version 75192 (0.0009) [2023-10-14 20:46:18,338][61585] Updated weights for policy 1, policy_version 75030 (0.0008) [2023-10-14 20:46:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 153780224. Throughput: 0: 1683.5, 1: 1661.6. Samples: 38457396. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-14 20:46:18,344][60425] Avg episode reward: [(0, '76.590'), (1, '72.300')] [2023-10-14 20:46:18,705][61585] Updated weights for policy 1, policy_version 75040 (0.0008) [2023-10-14 20:46:22,397][61552] Updated weights for policy 0, policy_version 75202 (0.0010) [2023-10-14 20:46:22,728][61585] Updated weights for policy 1, policy_version 75050 (0.0007) [2023-10-14 20:46:22,765][61552] Updated weights for policy 0, policy_version 75212 (0.0008) [2023-10-14 20:46:23,088][61585] Updated weights for policy 1, policy_version 75060 (0.0008) [2023-10-14 20:46:23,122][61552] Updated weights for policy 0, policy_version 75222 (0.0007) [2023-10-14 20:46:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 153845760. Throughput: 0: 1688.2, 1: 1662.8. Samples: 38478080. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-14 20:46:23,344][60425] Avg episode reward: [(0, '75.500'), (1, '75.440')] [2023-10-14 20:46:23,458][61585] Updated weights for policy 1, policy_version 75070 (0.0008) [2023-10-14 20:46:23,491][61552] Updated weights for policy 0, policy_version 75232 (0.0007) [2023-10-14 20:46:27,426][61585] Updated weights for policy 1, policy_version 75080 (0.0009) [2023-10-14 20:46:27,560][61552] Updated weights for policy 0, policy_version 75242 (0.0009) [2023-10-14 20:46:27,786][61585] Updated weights for policy 1, policy_version 75090 (0.0008) [2023-10-14 20:46:27,930][61552] Updated weights for policy 0, policy_version 75252 (0.0009) [2023-10-14 20:46:28,146][61585] Updated weights for policy 1, policy_version 75100 (0.0009) [2023-10-14 20:46:28,298][61552] Updated weights for policy 0, policy_version 75262 (0.0009) [2023-10-14 20:46:28,343][60425] Fps is (10 sec: 16383.7, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 153944064. Throughput: 0: 1675.6, 1: 1650.9. Samples: 38497552. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-14 20:46:28,344][60425] Avg episode reward: [(0, '72.110'), (1, '76.570')] [2023-10-14 20:46:32,177][61585] Updated weights for policy 1, policy_version 75110 (0.0009) [2023-10-14 20:46:32,384][61552] Updated weights for policy 0, policy_version 75272 (0.0007) [2023-10-14 20:46:32,536][61585] Updated weights for policy 1, policy_version 75120 (0.0007) [2023-10-14 20:46:32,749][61552] Updated weights for policy 0, policy_version 75282 (0.0007) [2023-10-14 20:46:32,902][61585] Updated weights for policy 1, policy_version 75130 (0.0008) [2023-10-14 20:46:33,112][61552] Updated weights for policy 0, policy_version 75292 (0.0007) [2023-10-14 20:46:33,343][60425] Fps is (10 sec: 19660.8, 60 sec: 14199.5, 300 sec: 13551.5). Total num frames: 154042368. Throughput: 0: 1683.8, 1: 1660.4. Samples: 38507654. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-14 20:46:33,344][60425] Avg episode reward: [(0, '75.320'), (1, '77.930')] [2023-10-14 20:46:36,871][61585] Updated weights for policy 1, policy_version 75140 (0.0009) [2023-10-14 20:46:37,178][61552] Updated weights for policy 0, policy_version 75302 (0.0010) [2023-10-14 20:46:37,232][61585] Updated weights for policy 1, policy_version 75150 (0.0008) [2023-10-14 20:46:37,542][61552] Updated weights for policy 0, policy_version 75312 (0.0010) [2023-10-14 20:46:37,591][61585] Updated weights for policy 1, policy_version 75160 (0.0007) [2023-10-14 20:46:37,897][61552] Updated weights for policy 0, policy_version 75322 (0.0008) [2023-10-14 20:46:38,343][60425] Fps is (10 sec: 16384.0, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 154107904. Throughput: 0: 1679.0, 1: 1663.9. Samples: 38528096. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-14 20:46:38,344][60425] Avg episode reward: [(0, '74.860'), (1, '77.660')] [2023-10-14 20:46:41,698][61585] Updated weights for policy 1, policy_version 75170 (0.0008) [2023-10-14 20:46:42,000][61552] Updated weights for policy 0, policy_version 75332 (0.0009) [2023-10-14 20:46:42,068][61585] Updated weights for policy 1, policy_version 75180 (0.0008) [2023-10-14 20:46:42,372][61552] Updated weights for policy 0, policy_version 75342 (0.0009) [2023-10-14 20:46:42,445][61585] Updated weights for policy 1, policy_version 75190 (0.0007) [2023-10-14 20:46:42,729][61552] Updated weights for policy 0, policy_version 75352 (0.0007) [2023-10-14 20:46:42,813][61585] Updated weights for policy 1, policy_version 75200 (0.0007) [2023-10-14 20:46:43,344][60425] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 154173440. Throughput: 0: 1658.4, 1: 1652.0. Samples: 38547084. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-14 20:46:43,345][60425] Avg episode reward: [(0, '76.370'), (1, '78.590')] [2023-10-14 20:46:46,814][61552] Updated weights for policy 0, policy_version 75362 (0.0007) [2023-10-14 20:46:46,863][61585] Updated weights for policy 1, policy_version 75210 (0.0008) [2023-10-14 20:46:47,177][61552] Updated weights for policy 0, policy_version 75372 (0.0007) [2023-10-14 20:46:47,228][61585] Updated weights for policy 1, policy_version 75220 (0.0007) [2023-10-14 20:46:47,559][61552] Updated weights for policy 0, policy_version 75382 (0.0008) [2023-10-14 20:46:47,595][61585] Updated weights for policy 1, policy_version 75230 (0.0009) [2023-10-14 20:46:47,920][61552] Updated weights for policy 0, policy_version 75392 (0.0010) [2023-10-14 20:46:48,343][60425] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 154238976. Throughput: 0: 1679.9, 1: 1676.7. Samples: 38558150. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-14 20:46:48,344][60425] Avg episode reward: [(0, '77.390'), (1, '77.760')] [2023-10-14 20:46:51,875][61585] Updated weights for policy 1, policy_version 75240 (0.0008) [2023-10-14 20:46:52,043][61552] Updated weights for policy 0, policy_version 75402 (0.0008) [2023-10-14 20:46:52,247][61585] Updated weights for policy 1, policy_version 75250 (0.0007) [2023-10-14 20:46:52,410][61552] Updated weights for policy 0, policy_version 75412 (0.0008) [2023-10-14 20:46:52,612][61585] Updated weights for policy 1, policy_version 75260 (0.0008) [2023-10-14 20:46:52,775][61552] Updated weights for policy 0, policy_version 75422 (0.0009) [2023-10-14 20:46:53,343][60425] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 154304512. Throughput: 0: 1674.9, 1: 1672.4. Samples: 38578092. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-14 20:46:53,344][60425] Avg episode reward: [(0, '77.170'), (1, '80.950')] [2023-10-14 20:46:56,691][61585] Updated weights for policy 1, policy_version 75270 (0.0009) [2023-10-14 20:46:56,971][61552] Updated weights for policy 0, policy_version 75432 (0.0008) [2023-10-14 20:46:57,060][61585] Updated weights for policy 1, policy_version 75280 (0.0009) [2023-10-14 20:46:57,344][61552] Updated weights for policy 0, policy_version 75442 (0.0009) [2023-10-14 20:46:57,416][61585] Updated weights for policy 1, policy_version 75290 (0.0008) [2023-10-14 20:46:57,706][61552] Updated weights for policy 0, policy_version 75452 (0.0008) [2023-10-14 20:46:58,343][60425] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 154370048. Throughput: 0: 1650.7, 1: 1656.9. Samples: 38596684. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-14 20:46:58,344][60425] Avg episode reward: [(0, '71.990'), (1, '76.040')] [2023-10-14 20:47:01,613][61585] Updated weights for policy 1, policy_version 75300 (0.0009) [2023-10-14 20:47:01,722][61552] Updated weights for policy 0, policy_version 75462 (0.0008) [2023-10-14 20:47:01,977][61585] Updated weights for policy 1, policy_version 75310 (0.0007) [2023-10-14 20:47:02,098][61552] Updated weights for policy 0, policy_version 75472 (0.0008) [2023-10-14 20:47:02,336][61585] Updated weights for policy 1, policy_version 75320 (0.0007) [2023-10-14 20:47:02,461][61552] Updated weights for policy 0, policy_version 75482 (0.0007) [2023-10-14 20:47:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 154435584. Throughput: 0: 1674.9, 1: 1670.9. Samples: 38607958. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-14 20:47:03,344][60425] Avg episode reward: [(0, '75.010'), (1, '79.670')] [2023-10-14 20:47:06,364][61585] Updated weights for policy 1, policy_version 75330 (0.0009) [2023-10-14 20:47:06,627][61552] Updated weights for policy 0, policy_version 75492 (0.0008) [2023-10-14 20:47:06,727][61585] Updated weights for policy 1, policy_version 75340 (0.0009) [2023-10-14 20:47:06,992][61552] Updated weights for policy 0, policy_version 75502 (0.0008) [2023-10-14 20:47:07,095][61585] Updated weights for policy 1, policy_version 75350 (0.0009) [2023-10-14 20:47:07,371][61552] Updated weights for policy 0, policy_version 75512 (0.0008) [2023-10-14 20:47:07,459][61585] Updated weights for policy 1, policy_version 75360 (0.0009) [2023-10-14 20:47:08,343][60425] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 154501120. Throughput: 0: 1663.0, 1: 1660.8. Samples: 38627652. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:47:08,344][60425] Avg episode reward: [(0, '74.490'), (1, '80.640')] [2023-10-14 20:47:11,448][61585] Updated weights for policy 1, policy_version 75370 (0.0008) [2023-10-14 20:47:11,528][61552] Updated weights for policy 0, policy_version 75522 (0.0008) [2023-10-14 20:47:11,813][61585] Updated weights for policy 1, policy_version 75380 (0.0008) [2023-10-14 20:47:11,898][61552] Updated weights for policy 0, policy_version 75532 (0.0008) [2023-10-14 20:47:12,176][61585] Updated weights for policy 1, policy_version 75390 (0.0008) [2023-10-14 20:47:12,265][61552] Updated weights for policy 0, policy_version 75542 (0.0007) [2023-10-14 20:47:12,626][61552] Updated weights for policy 0, policy_version 75552 (0.0010) [2023-10-14 20:47:13,343][60425] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 154566656. Throughput: 0: 1651.0, 1: 1662.2. Samples: 38646646. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:47:13,344][60425] Avg episode reward: [(0, '73.850'), (1, '75.310')] [2023-10-14 20:47:16,384][61585] Updated weights for policy 1, policy_version 75400 (0.0008) [2023-10-14 20:47:16,743][61585] Updated weights for policy 1, policy_version 75410 (0.0009) [2023-10-14 20:47:16,869][61552] Updated weights for policy 0, policy_version 75562 (0.0008) [2023-10-14 20:47:17,097][61585] Updated weights for policy 1, policy_version 75420 (0.0008) [2023-10-14 20:47:17,230][61552] Updated weights for policy 0, policy_version 75572 (0.0008) [2023-10-14 20:47:17,609][61552] Updated weights for policy 0, policy_version 75582 (0.0008) [2023-10-14 20:47:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 154632192. Throughput: 0: 1665.2, 1: 1677.6. Samples: 38658082. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:47:18,344][60425] Avg episode reward: [(0, '74.080'), (1, '79.440')] [2023-10-14 20:47:21,324][61585] Updated weights for policy 1, policy_version 75430 (0.0008) [2023-10-14 20:47:21,676][61585] Updated weights for policy 1, policy_version 75440 (0.0007) [2023-10-14 20:47:21,727][61552] Updated weights for policy 0, policy_version 75592 (0.0007) [2023-10-14 20:47:22,034][61585] Updated weights for policy 1, policy_version 75450 (0.0008) [2023-10-14 20:47:22,100][61552] Updated weights for policy 0, policy_version 75602 (0.0009) [2023-10-14 20:47:22,464][61552] Updated weights for policy 0, policy_version 75612 (0.0008) [2023-10-14 20:47:23,343][60425] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 154697728. Throughput: 0: 1663.2, 1: 1659.9. Samples: 38677634. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:47:23,344][60425] Avg episode reward: [(0, '72.970'), (1, '78.780')] [2023-10-14 20:47:26,190][61585] Updated weights for policy 1, policy_version 75460 (0.0008) [2023-10-14 20:47:26,548][61585] Updated weights for policy 1, policy_version 75470 (0.0008) [2023-10-14 20:47:26,618][61552] Updated weights for policy 0, policy_version 75622 (0.0008) [2023-10-14 20:47:26,918][61585] Updated weights for policy 1, policy_version 75480 (0.0009) [2023-10-14 20:47:26,985][61552] Updated weights for policy 0, policy_version 75632 (0.0008) [2023-10-14 20:47:27,344][61552] Updated weights for policy 0, policy_version 75642 (0.0010) [2023-10-14 20:47:28,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 154763264. Throughput: 0: 1659.3, 1: 1667.0. Samples: 38696766. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:47:28,344][60425] Avg episode reward: [(0, '74.980'), (1, '78.910')] [2023-10-14 20:47:30,968][61585] Updated weights for policy 1, policy_version 75490 (0.0009) [2023-10-14 20:47:31,334][61585] Updated weights for policy 1, policy_version 75500 (0.0010) [2023-10-14 20:47:31,408][61552] Updated weights for policy 0, policy_version 75652 (0.0009) [2023-10-14 20:47:31,703][61585] Updated weights for policy 1, policy_version 75510 (0.0008) [2023-10-14 20:47:31,771][61552] Updated weights for policy 0, policy_version 75662 (0.0007) [2023-10-14 20:47:32,064][61585] Updated weights for policy 1, policy_version 75520 (0.0010) [2023-10-14 20:47:32,141][61552] Updated weights for policy 0, policy_version 75672 (0.0007) [2023-10-14 20:47:33,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 154828800. Throughput: 0: 1665.8, 1: 1671.0. Samples: 38708304. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:47:33,344][60425] Avg episode reward: [(0, '72.920'), (1, '77.580')] [2023-10-14 20:47:36,069][61585] Updated weights for policy 1, policy_version 75530 (0.0007) [2023-10-14 20:47:36,300][61552] Updated weights for policy 0, policy_version 75682 (0.0009) [2023-10-14 20:47:36,432][61585] Updated weights for policy 1, policy_version 75540 (0.0007) [2023-10-14 20:47:36,676][61552] Updated weights for policy 0, policy_version 75692 (0.0008) [2023-10-14 20:47:36,786][61585] Updated weights for policy 1, policy_version 75550 (0.0010) [2023-10-14 20:47:37,035][61552] Updated weights for policy 0, policy_version 75702 (0.0011) [2023-10-14 20:47:37,400][61552] Updated weights for policy 0, policy_version 75712 (0.0009) [2023-10-14 20:47:38,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 154894336. Throughput: 0: 1655.1, 1: 1659.9. Samples: 38727264. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:47:38,344][60425] Avg episode reward: [(0, '71.490'), (1, '78.670')] [2023-10-14 20:47:40,792][61585] Updated weights for policy 1, policy_version 75560 (0.0008) [2023-10-14 20:47:41,156][61585] Updated weights for policy 1, policy_version 75570 (0.0008) [2023-10-14 20:47:41,483][61552] Updated weights for policy 0, policy_version 75722 (0.0007) [2023-10-14 20:47:41,527][61585] Updated weights for policy 1, policy_version 75580 (0.0008) [2023-10-14 20:47:41,856][61552] Updated weights for policy 0, policy_version 75732 (0.0007) [2023-10-14 20:47:42,215][61552] Updated weights for policy 0, policy_version 75742 (0.0008) [2023-10-14 20:47:43,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 154959872. Throughput: 0: 1664.0, 1: 1681.4. Samples: 38747228. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:47:43,344][60425] Avg episode reward: [(0, '72.480'), (1, '71.970')] [2023-10-14 20:47:43,353][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000075744_77561856.pth... [2023-10-14 20:47:43,353][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000075584_77398016.pth... [2023-10-14 20:47:43,384][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000074176_75956224.pth [2023-10-14 20:47:43,389][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000074016_75792384.pth [2023-10-14 20:47:45,514][61585] Updated weights for policy 1, policy_version 75590 (0.0010) [2023-10-14 20:47:45,875][61585] Updated weights for policy 1, policy_version 75600 (0.0007) [2023-10-14 20:47:46,087][61552] Updated weights for policy 0, policy_version 75752 (0.0010) [2023-10-14 20:47:46,237][61585] Updated weights for policy 1, policy_version 75610 (0.0008) [2023-10-14 20:47:46,454][61552] Updated weights for policy 0, policy_version 75762 (0.0009) [2023-10-14 20:47:46,828][61552] Updated weights for policy 0, policy_version 75772 (0.0009) [2023-10-14 20:47:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 155025408. Throughput: 0: 1669.9, 1: 1674.4. Samples: 38758450. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:47:48,344][60425] Avg episode reward: [(0, '74.230'), (1, '74.810')] [2023-10-14 20:47:50,586][61585] Updated weights for policy 1, policy_version 75620 (0.0008) [2023-10-14 20:47:50,916][61552] Updated weights for policy 0, policy_version 75782 (0.0007) [2023-10-14 20:47:50,957][61585] Updated weights for policy 1, policy_version 75630 (0.0007) [2023-10-14 20:47:51,278][61552] Updated weights for policy 0, policy_version 75792 (0.0009) [2023-10-14 20:47:51,325][61585] Updated weights for policy 1, policy_version 75640 (0.0007) [2023-10-14 20:47:51,656][61552] Updated weights for policy 0, policy_version 75802 (0.0007) [2023-10-14 20:47:53,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 155090944. Throughput: 0: 1653.1, 1: 1665.6. Samples: 38776998. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:47:53,344][60425] Avg episode reward: [(0, '71.230'), (1, '73.900')] [2023-10-14 20:47:55,494][61585] Updated weights for policy 1, policy_version 75650 (0.0008) [2023-10-14 20:47:55,784][61552] Updated weights for policy 0, policy_version 75812 (0.0008) [2023-10-14 20:47:55,867][61585] Updated weights for policy 1, policy_version 75660 (0.0008) [2023-10-14 20:47:56,151][61552] Updated weights for policy 0, policy_version 75822 (0.0009) [2023-10-14 20:47:56,222][61585] Updated weights for policy 1, policy_version 75670 (0.0007) [2023-10-14 20:47:56,508][61552] Updated weights for policy 0, policy_version 75832 (0.0008) [2023-10-14 20:47:56,581][61585] Updated weights for policy 1, policy_version 75680 (0.0007) [2023-10-14 20:47:58,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 155156480. Throughput: 0: 1672.9, 1: 1676.7. Samples: 38797380. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:47:58,344][60425] Avg episode reward: [(0, '72.200'), (1, '73.610')] [2023-10-14 20:48:00,583][61585] Updated weights for policy 1, policy_version 75690 (0.0007) [2023-10-14 20:48:00,667][61552] Updated weights for policy 0, policy_version 75842 (0.0009) [2023-10-14 20:48:00,955][61585] Updated weights for policy 1, policy_version 75700 (0.0008) [2023-10-14 20:48:01,039][61552] Updated weights for policy 0, policy_version 75852 (0.0011) [2023-10-14 20:48:01,311][61585] Updated weights for policy 1, policy_version 75710 (0.0008) [2023-10-14 20:48:01,409][61552] Updated weights for policy 0, policy_version 75862 (0.0010) [2023-10-14 20:48:01,780][61552] Updated weights for policy 0, policy_version 75872 (0.0010) [2023-10-14 20:48:03,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 155222016. Throughput: 0: 1672.0, 1: 1666.1. Samples: 38808294. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:48:03,344][60425] Avg episode reward: [(0, '73.110'), (1, '75.240')] [2023-10-14 20:48:05,327][61585] Updated weights for policy 1, policy_version 75720 (0.0008) [2023-10-14 20:48:05,683][61585] Updated weights for policy 1, policy_version 75730 (0.0008) [2023-10-14 20:48:05,993][61552] Updated weights for policy 0, policy_version 75882 (0.0007) [2023-10-14 20:48:06,046][61585] Updated weights for policy 1, policy_version 75740 (0.0010) [2023-10-14 20:48:06,366][61552] Updated weights for policy 0, policy_version 75892 (0.0009) [2023-10-14 20:48:06,747][61552] Updated weights for policy 0, policy_version 75902 (0.0010) [2023-10-14 20:48:08,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 155287552. Throughput: 0: 1654.2, 1: 1670.0. Samples: 38827220. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:48:08,345][60425] Avg episode reward: [(0, '71.630'), (1, '73.680')] [2023-10-14 20:48:10,087][61585] Updated weights for policy 1, policy_version 75750 (0.0010) [2023-10-14 20:48:10,457][61585] Updated weights for policy 1, policy_version 75760 (0.0008) [2023-10-14 20:48:10,824][61585] Updated weights for policy 1, policy_version 75770 (0.0009) [2023-10-14 20:48:10,894][61552] Updated weights for policy 0, policy_version 75912 (0.0007) [2023-10-14 20:48:11,274][61552] Updated weights for policy 0, policy_version 75922 (0.0008) [2023-10-14 20:48:11,641][61552] Updated weights for policy 0, policy_version 75932 (0.0008) [2023-10-14 20:48:13,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 155353088. Throughput: 0: 1664.4, 1: 1680.1. Samples: 38847268. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:48:13,344][60425] Avg episode reward: [(0, '71.830'), (1, '75.870')] [2023-10-14 20:48:14,937][61585] Updated weights for policy 1, policy_version 75780 (0.0009) [2023-10-14 20:48:15,302][61585] Updated weights for policy 1, policy_version 75790 (0.0010) [2023-10-14 20:48:15,664][61585] Updated weights for policy 1, policy_version 75800 (0.0008) [2023-10-14 20:48:15,674][61552] Updated weights for policy 0, policy_version 75942 (0.0009) [2023-10-14 20:48:16,042][61552] Updated weights for policy 0, policy_version 75952 (0.0009) [2023-10-14 20:48:16,410][61552] Updated weights for policy 0, policy_version 75962 (0.0009) [2023-10-14 20:48:18,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 155418624. Throughput: 0: 1660.3, 1: 1654.0. Samples: 38857448. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:48:18,344][60425] Avg episode reward: [(0, '71.690'), (1, '76.420')] [2023-10-14 20:48:19,881][61585] Updated weights for policy 1, policy_version 75810 (0.0010) [2023-10-14 20:48:20,245][61585] Updated weights for policy 1, policy_version 75820 (0.0009) [2023-10-14 20:48:20,579][61552] Updated weights for policy 0, policy_version 75972 (0.0010) [2023-10-14 20:48:20,611][61585] Updated weights for policy 1, policy_version 75830 (0.0008) [2023-10-14 20:48:20,943][61552] Updated weights for policy 0, policy_version 75982 (0.0008) [2023-10-14 20:48:20,985][61585] Updated weights for policy 1, policy_version 75840 (0.0008) [2023-10-14 20:48:21,299][61552] Updated weights for policy 0, policy_version 75992 (0.0008) [2023-10-14 20:48:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 155484160. Throughput: 0: 1651.3, 1: 1666.6. Samples: 38876570. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:48:23,344][60425] Avg episode reward: [(0, '75.490'), (1, '80.490')] [2023-10-14 20:48:25,040][61585] Updated weights for policy 1, policy_version 75850 (0.0008) [2023-10-14 20:48:25,407][61585] Updated weights for policy 1, policy_version 75860 (0.0008) [2023-10-14 20:48:25,591][61552] Updated weights for policy 0, policy_version 76002 (0.0009) [2023-10-14 20:48:25,768][61585] Updated weights for policy 1, policy_version 75870 (0.0007) [2023-10-14 20:48:25,957][61552] Updated weights for policy 0, policy_version 76012 (0.0008) [2023-10-14 20:48:26,331][61552] Updated weights for policy 0, policy_version 76022 (0.0009) [2023-10-14 20:48:26,694][61552] Updated weights for policy 0, policy_version 76032 (0.0008) [2023-10-14 20:48:28,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 155549696. Throughput: 0: 1665.3, 1: 1669.8. Samples: 38897310. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:48:28,344][60425] Avg episode reward: [(0, '74.590'), (1, '78.360')] [2023-10-14 20:48:30,143][61585] Updated weights for policy 1, policy_version 75880 (0.0011) [2023-10-14 20:48:30,521][61585] Updated weights for policy 1, policy_version 75890 (0.0009) [2023-10-14 20:48:30,803][61552] Updated weights for policy 0, policy_version 76042 (0.0007) [2023-10-14 20:48:30,881][61585] Updated weights for policy 1, policy_version 75900 (0.0009) [2023-10-14 20:48:31,182][61552] Updated weights for policy 0, policy_version 76052 (0.0008) [2023-10-14 20:48:31,540][61552] Updated weights for policy 0, policy_version 76062 (0.0008) [2023-10-14 20:48:33,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 155615232. Throughput: 0: 1653.2, 1: 1658.2. Samples: 38907466. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:48:33,344][60425] Avg episode reward: [(0, '76.600'), (1, '82.230')] [2023-10-14 20:48:34,723][61585] Updated weights for policy 1, policy_version 75910 (0.0008) [2023-10-14 20:48:35,079][61585] Updated weights for policy 1, policy_version 75920 (0.0008) [2023-10-14 20:48:35,443][61585] Updated weights for policy 1, policy_version 75930 (0.0007) [2023-10-14 20:48:35,576][61552] Updated weights for policy 0, policy_version 76072 (0.0008) [2023-10-14 20:48:35,931][61552] Updated weights for policy 0, policy_version 76082 (0.0011) [2023-10-14 20:48:36,300][61552] Updated weights for policy 0, policy_version 76092 (0.0009) [2023-10-14 20:48:38,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 155680768. Throughput: 0: 1655.6, 1: 1672.6. Samples: 38926766. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:48:38,344][60425] Avg episode reward: [(0, '76.120'), (1, '82.140')] [2023-10-14 20:48:39,642][61585] Updated weights for policy 1, policy_version 75940 (0.0008) [2023-10-14 20:48:40,003][61585] Updated weights for policy 1, policy_version 75950 (0.0010) [2023-10-14 20:48:40,289][61552] Updated weights for policy 0, policy_version 76102 (0.0009) [2023-10-14 20:48:40,376][61585] Updated weights for policy 1, policy_version 75960 (0.0008) [2023-10-14 20:48:40,651][61552] Updated weights for policy 0, policy_version 76112 (0.0009) [2023-10-14 20:48:41,017][61552] Updated weights for policy 0, policy_version 76122 (0.0008) [2023-10-14 20:48:43,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 155746304. Throughput: 0: 1664.2, 1: 1672.3. Samples: 38947522. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:48:43,344][60425] Avg episode reward: [(0, '79.790'), (1, '75.330')] [2023-10-14 20:48:44,504][61585] Updated weights for policy 1, policy_version 75970 (0.0008) [2023-10-14 20:48:44,866][61585] Updated weights for policy 1, policy_version 75980 (0.0008) [2023-10-14 20:48:45,147][61552] Updated weights for policy 0, policy_version 76132 (0.0008) [2023-10-14 20:48:45,234][61585] Updated weights for policy 1, policy_version 75990 (0.0008) [2023-10-14 20:48:45,503][61552] Updated weights for policy 0, policy_version 76142 (0.0010) [2023-10-14 20:48:45,593][61585] Updated weights for policy 1, policy_version 76000 (0.0008) [2023-10-14 20:48:45,877][61552] Updated weights for policy 0, policy_version 76152 (0.0010) [2023-10-14 20:48:48,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 155811840. Throughput: 0: 1653.1, 1: 1655.8. Samples: 38957196. Policy #0 lag: (min: 1.0, avg: 3.8, max: 27.0) [2023-10-14 20:48:48,344][60425] Avg episode reward: [(0, '76.590'), (1, '77.540')] [2023-10-14 20:48:49,528][61585] Updated weights for policy 1, policy_version 76010 (0.0010) [2023-10-14 20:48:49,896][61585] Updated weights for policy 1, policy_version 76020 (0.0009) [2023-10-14 20:48:49,966][61552] Updated weights for policy 0, policy_version 76162 (0.0010) [2023-10-14 20:48:50,255][61585] Updated weights for policy 1, policy_version 76030 (0.0007) [2023-10-14 20:48:50,333][61552] Updated weights for policy 0, policy_version 76172 (0.0007) [2023-10-14 20:48:50,702][61552] Updated weights for policy 0, policy_version 76182 (0.0012) [2023-10-14 20:48:51,070][61552] Updated weights for policy 0, policy_version 76192 (0.0009) [2023-10-14 20:48:53,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 155877376. Throughput: 0: 1660.8, 1: 1669.7. Samples: 38977090. Policy #0 lag: (min: 1.0, avg: 3.8, max: 27.0) [2023-10-14 20:48:53,344][60425] Avg episode reward: [(0, '77.370'), (1, '77.650')] [2023-10-14 20:48:54,266][61585] Updated weights for policy 1, policy_version 76040 (0.0007) [2023-10-14 20:48:54,633][61585] Updated weights for policy 1, policy_version 76050 (0.0009) [2023-10-14 20:48:55,012][61585] Updated weights for policy 1, policy_version 76060 (0.0008) [2023-10-14 20:48:55,293][61552] Updated weights for policy 0, policy_version 76202 (0.0008) [2023-10-14 20:48:55,660][61552] Updated weights for policy 0, policy_version 76212 (0.0008) [2023-10-14 20:48:56,025][61552] Updated weights for policy 0, policy_version 76222 (0.0009) [2023-10-14 20:48:58,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 155942912. Throughput: 0: 1668.9, 1: 1672.7. Samples: 38997640. Policy #0 lag: (min: 1.0, avg: 3.8, max: 27.0) [2023-10-14 20:48:58,344][60425] Avg episode reward: [(0, '78.890'), (1, '73.970')] [2023-10-14 20:48:59,107][61585] Updated weights for policy 1, policy_version 76070 (0.0007) [2023-10-14 20:48:59,477][61585] Updated weights for policy 1, policy_version 76080 (0.0009) [2023-10-14 20:48:59,838][61585] Updated weights for policy 1, policy_version 76090 (0.0008) [2023-10-14 20:49:00,284][61552] Updated weights for policy 0, policy_version 76232 (0.0009) [2023-10-14 20:49:00,657][61552] Updated weights for policy 0, policy_version 76242 (0.0011) [2023-10-14 20:49:01,022][61552] Updated weights for policy 0, policy_version 76252 (0.0009) [2023-10-14 20:49:03,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 156008448. Throughput: 0: 1655.4, 1: 1668.5. Samples: 39007024. Policy #0 lag: (min: 1.0, avg: 3.8, max: 27.0) [2023-10-14 20:49:03,344][60425] Avg episode reward: [(0, '77.660'), (1, '73.650')] [2023-10-14 20:49:04,230][61585] Updated weights for policy 1, policy_version 76100 (0.0008) [2023-10-14 20:49:04,595][61585] Updated weights for policy 1, policy_version 76110 (0.0009) [2023-10-14 20:49:04,962][61585] Updated weights for policy 1, policy_version 76120 (0.0008) [2023-10-14 20:49:05,081][61552] Updated weights for policy 0, policy_version 76262 (0.0010) [2023-10-14 20:49:05,463][61552] Updated weights for policy 0, policy_version 76272 (0.0009) [2023-10-14 20:49:05,840][61552] Updated weights for policy 0, policy_version 76282 (0.0010) [2023-10-14 20:49:08,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 156073984. Throughput: 0: 1665.7, 1: 1673.7. Samples: 39026844. Policy #0 lag: (min: 1.0, avg: 3.8, max: 27.0) [2023-10-14 20:49:08,344][60425] Avg episode reward: [(0, '76.090'), (1, '80.040')] [2023-10-14 20:49:08,966][61585] Updated weights for policy 1, policy_version 76130 (0.0008) [2023-10-14 20:49:09,330][61585] Updated weights for policy 1, policy_version 76140 (0.0009) [2023-10-14 20:49:09,691][61585] Updated weights for policy 1, policy_version 76150 (0.0009) [2023-10-14 20:49:09,738][61552] Updated weights for policy 0, policy_version 76292 (0.0008) [2023-10-14 20:49:10,051][61585] Updated weights for policy 1, policy_version 76160 (0.0008) [2023-10-14 20:49:10,114][61552] Updated weights for policy 0, policy_version 76302 (0.0009) [2023-10-14 20:49:10,473][61552] Updated weights for policy 0, policy_version 76312 (0.0009) [2023-10-14 20:49:13,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 156139520. Throughput: 0: 1670.1, 1: 1670.0. Samples: 39047612. Policy #0 lag: (min: 1.0, avg: 3.8, max: 27.0) [2023-10-14 20:49:13,345][60425] Avg episode reward: [(0, '79.240'), (1, '79.340')] [2023-10-14 20:49:14,235][61585] Updated weights for policy 1, policy_version 76170 (0.0007) [2023-10-14 20:49:14,601][61552] Updated weights for policy 0, policy_version 76322 (0.0007) [2023-10-14 20:49:14,604][61585] Updated weights for policy 1, policy_version 76180 (0.0008) [2023-10-14 20:49:14,967][61585] Updated weights for policy 1, policy_version 76190 (0.0008) [2023-10-14 20:49:14,975][61552] Updated weights for policy 0, policy_version 76332 (0.0009) [2023-10-14 20:49:15,334][61552] Updated weights for policy 0, policy_version 76342 (0.0008) [2023-10-14 20:49:15,712][61552] Updated weights for policy 0, policy_version 76352 (0.0007) [2023-10-14 20:49:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 156205056. Throughput: 0: 1650.8, 1: 1665.6. Samples: 39056704. Policy #0 lag: (min: 1.0, avg: 3.8, max: 27.0) [2023-10-14 20:49:18,344][60425] Avg episode reward: [(0, '79.950'), (1, '76.940')] [2023-10-14 20:49:19,292][61585] Updated weights for policy 1, policy_version 76200 (0.0010) [2023-10-14 20:49:19,674][61585] Updated weights for policy 1, policy_version 76210 (0.0011) [2023-10-14 20:49:19,827][61552] Updated weights for policy 0, policy_version 76362 (0.0010) [2023-10-14 20:49:20,030][61585] Updated weights for policy 1, policy_version 76220 (0.0009) [2023-10-14 20:49:20,199][61552] Updated weights for policy 0, policy_version 76372 (0.0007) [2023-10-14 20:49:20,561][61552] Updated weights for policy 0, policy_version 76382 (0.0007) [2023-10-14 20:49:23,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 156270592. Throughput: 0: 1667.3, 1: 1663.2. Samples: 39076638. Policy #0 lag: (min: 1.0, avg: 3.8, max: 27.0) [2023-10-14 20:49:23,344][60425] Avg episode reward: [(0, '77.330'), (1, '77.190')] [2023-10-14 20:49:24,087][61585] Updated weights for policy 1, policy_version 76230 (0.0007) [2023-10-14 20:49:24,461][61585] Updated weights for policy 1, policy_version 76240 (0.0007) [2023-10-14 20:49:24,743][61552] Updated weights for policy 0, policy_version 76392 (0.0008) [2023-10-14 20:49:24,824][61585] Updated weights for policy 1, policy_version 76250 (0.0007) [2023-10-14 20:49:25,117][61552] Updated weights for policy 0, policy_version 76402 (0.0009) [2023-10-14 20:49:25,491][61552] Updated weights for policy 0, policy_version 76412 (0.0009) [2023-10-14 20:49:28,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 156336128. Throughput: 0: 1658.0, 1: 1667.4. Samples: 39097162. Policy #0 lag: (min: 1.0, avg: 3.8, max: 27.0) [2023-10-14 20:49:28,344][60425] Avg episode reward: [(0, '77.910'), (1, '76.890')] [2023-10-14 20:49:28,824][61585] Updated weights for policy 1, policy_version 76260 (0.0007) [2023-10-14 20:49:29,191][61585] Updated weights for policy 1, policy_version 76270 (0.0008) [2023-10-14 20:49:29,548][61585] Updated weights for policy 1, policy_version 76280 (0.0009) [2023-10-14 20:49:29,603][61552] Updated weights for policy 0, policy_version 76422 (0.0010) [2023-10-14 20:49:29,961][61552] Updated weights for policy 0, policy_version 76432 (0.0009) [2023-10-14 20:49:30,326][61552] Updated weights for policy 0, policy_version 76442 (0.0009) [2023-10-14 20:49:33,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 156401664. Throughput: 0: 1645.2, 1: 1669.4. Samples: 39106352. Policy #0 lag: (min: 1.0, avg: 3.8, max: 27.0) [2023-10-14 20:49:33,344][60425] Avg episode reward: [(0, '76.630'), (1, '75.130')] [2023-10-14 20:49:33,633][61585] Updated weights for policy 1, policy_version 76290 (0.0008) [2023-10-14 20:49:33,999][61585] Updated weights for policy 1, policy_version 76300 (0.0010) [2023-10-14 20:49:34,366][61585] Updated weights for policy 1, policy_version 76310 (0.0009) [2023-10-14 20:49:34,406][61552] Updated weights for policy 0, policy_version 76452 (0.0010) [2023-10-14 20:49:34,737][61585] Updated weights for policy 1, policy_version 76320 (0.0009) [2023-10-14 20:49:34,776][61552] Updated weights for policy 0, policy_version 76462 (0.0007) [2023-10-14 20:49:35,146][61552] Updated weights for policy 0, policy_version 76472 (0.0008) [2023-10-14 20:49:38,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 156467200. Throughput: 0: 1664.2, 1: 1663.6. Samples: 39126840. Policy #0 lag: (min: 10.0, avg: 11.3, max: 35.0) [2023-10-14 20:49:38,344][60425] Avg episode reward: [(0, '78.890'), (1, '79.460')] [2023-10-14 20:49:38,982][61585] Updated weights for policy 1, policy_version 76330 (0.0007) [2023-10-14 20:49:39,195][61552] Updated weights for policy 0, policy_version 76482 (0.0008) [2023-10-14 20:49:39,332][61585] Updated weights for policy 1, policy_version 76340 (0.0008) [2023-10-14 20:49:39,567][61552] Updated weights for policy 0, policy_version 76492 (0.0008) [2023-10-14 20:49:39,701][61585] Updated weights for policy 1, policy_version 76350 (0.0008) [2023-10-14 20:49:39,935][61552] Updated weights for policy 0, policy_version 76502 (0.0009) [2023-10-14 20:49:40,296][61552] Updated weights for policy 0, policy_version 76512 (0.0010) [2023-10-14 20:49:43,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 156532736. Throughput: 0: 1665.6, 1: 1662.7. Samples: 39147412. Policy #0 lag: (min: 10.0, avg: 11.3, max: 35.0) [2023-10-14 20:49:43,344][60425] Avg episode reward: [(0, '75.340'), (1, '79.310')] [2023-10-14 20:49:43,350][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000076352_78184448.pth... [2023-10-14 20:49:43,350][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000076512_78348288.pth... [2023-10-14 20:49:43,389][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000074816_76611584.pth [2023-10-14 20:49:43,390][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000074976_76775424.pth [2023-10-14 20:49:43,787][61585] Updated weights for policy 1, policy_version 76360 (0.0008) [2023-10-14 20:49:44,157][61585] Updated weights for policy 1, policy_version 76370 (0.0009) [2023-10-14 20:49:44,515][61585] Updated weights for policy 1, policy_version 76380 (0.0009) [2023-10-14 20:49:44,588][61552] Updated weights for policy 0, policy_version 76522 (0.0008) [2023-10-14 20:49:44,959][61552] Updated weights for policy 0, policy_version 76532 (0.0010) [2023-10-14 20:49:45,330][61552] Updated weights for policy 0, policy_version 76542 (0.0008) [2023-10-14 20:49:48,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 156598272. Throughput: 0: 1658.8, 1: 1663.7. Samples: 39156536. Policy #0 lag: (min: 10.0, avg: 11.3, max: 35.0) [2023-10-14 20:49:48,344][60425] Avg episode reward: [(0, '77.840'), (1, '77.560')] [2023-10-14 20:49:48,455][61585] Updated weights for policy 1, policy_version 76390 (0.0008) [2023-10-14 20:49:48,826][61585] Updated weights for policy 1, policy_version 76400 (0.0007) [2023-10-14 20:49:49,189][61585] Updated weights for policy 1, policy_version 76410 (0.0007) [2023-10-14 20:49:49,435][61552] Updated weights for policy 0, policy_version 76552 (0.0008) [2023-10-14 20:49:49,809][61552] Updated weights for policy 0, policy_version 76562 (0.0009) [2023-10-14 20:49:50,169][61552] Updated weights for policy 0, policy_version 76572 (0.0010) [2023-10-14 20:49:53,319][61585] Updated weights for policy 1, policy_version 76420 (0.0007) [2023-10-14 20:49:53,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 156663808. Throughput: 0: 1669.3, 1: 1669.8. Samples: 39177106. Policy #0 lag: (min: 10.0, avg: 11.3, max: 35.0) [2023-10-14 20:49:53,344][60425] Avg episode reward: [(0, '75.600'), (1, '75.710')] [2023-10-14 20:49:53,686][61585] Updated weights for policy 1, policy_version 76430 (0.0007) [2023-10-14 20:49:54,052][61585] Updated weights for policy 1, policy_version 76440 (0.0009) [2023-10-14 20:49:54,291][61552] Updated weights for policy 0, policy_version 76582 (0.0007) [2023-10-14 20:49:54,674][61552] Updated weights for policy 0, policy_version 76592 (0.0008) [2023-10-14 20:49:55,039][61552] Updated weights for policy 0, policy_version 76602 (0.0007) [2023-10-14 20:49:58,228][61585] Updated weights for policy 1, policy_version 76450 (0.0008) [2023-10-14 20:49:58,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 156729344. Throughput: 0: 1667.5, 1: 1665.8. Samples: 39197608. Policy #0 lag: (min: 10.0, avg: 11.3, max: 35.0) [2023-10-14 20:49:58,344][60425] Avg episode reward: [(0, '74.010'), (1, '75.520')] [2023-10-14 20:49:58,606][61585] Updated weights for policy 1, policy_version 76460 (0.0009) [2023-10-14 20:49:58,972][61585] Updated weights for policy 1, policy_version 76470 (0.0008) [2023-10-14 20:49:58,980][61552] Updated weights for policy 0, policy_version 76612 (0.0008) [2023-10-14 20:49:59,330][61585] Updated weights for policy 1, policy_version 76480 (0.0009) [2023-10-14 20:49:59,335][61552] Updated weights for policy 0, policy_version 76622 (0.0009) [2023-10-14 20:49:59,697][61552] Updated weights for policy 0, policy_version 76632 (0.0008) [2023-10-14 20:50:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 156794880. Throughput: 0: 1665.6, 1: 1667.2. Samples: 39206680. Policy #0 lag: (min: 10.0, avg: 11.3, max: 35.0) [2023-10-14 20:50:03,344][60425] Avg episode reward: [(0, '76.770'), (1, '76.930')] [2023-10-14 20:50:03,434][61585] Updated weights for policy 1, policy_version 76490 (0.0011) [2023-10-14 20:50:03,799][61585] Updated weights for policy 1, policy_version 76500 (0.0011) [2023-10-14 20:50:03,940][61552] Updated weights for policy 0, policy_version 76642 (0.0010) [2023-10-14 20:50:04,161][61585] Updated weights for policy 1, policy_version 76510 (0.0010) [2023-10-14 20:50:04,316][61552] Updated weights for policy 0, policy_version 76652 (0.0007) [2023-10-14 20:50:04,695][61552] Updated weights for policy 0, policy_version 76662 (0.0010) [2023-10-14 20:50:05,060][61552] Updated weights for policy 0, policy_version 76672 (0.0009) [2023-10-14 20:50:08,182][61585] Updated weights for policy 1, policy_version 76520 (0.0008) [2023-10-14 20:50:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 156860416. Throughput: 0: 1667.5, 1: 1677.2. Samples: 39227146. Policy #0 lag: (min: 10.0, avg: 11.3, max: 35.0) [2023-10-14 20:50:08,344][60425] Avg episode reward: [(0, '72.820'), (1, '79.680')] [2023-10-14 20:50:08,549][61585] Updated weights for policy 1, policy_version 76530 (0.0007) [2023-10-14 20:50:08,911][61585] Updated weights for policy 1, policy_version 76540 (0.0009) [2023-10-14 20:50:09,241][61552] Updated weights for policy 0, policy_version 76682 (0.0008) [2023-10-14 20:50:09,608][61552] Updated weights for policy 0, policy_version 76692 (0.0008) [2023-10-14 20:50:09,984][61552] Updated weights for policy 0, policy_version 76702 (0.0007) [2023-10-14 20:50:13,091][61585] Updated weights for policy 1, policy_version 76550 (0.0008) [2023-10-14 20:50:13,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 156925952. Throughput: 0: 1672.1, 1: 1673.1. Samples: 39247700. Policy #0 lag: (min: 10.0, avg: 11.3, max: 35.0) [2023-10-14 20:50:13,345][60425] Avg episode reward: [(0, '73.500'), (1, '80.850')] [2023-10-14 20:50:13,445][61585] Updated weights for policy 1, policy_version 76560 (0.0009) [2023-10-14 20:50:13,818][61585] Updated weights for policy 1, policy_version 76570 (0.0008) [2023-10-14 20:50:14,077][61552] Updated weights for policy 0, policy_version 76712 (0.0007) [2023-10-14 20:50:14,451][61552] Updated weights for policy 0, policy_version 76722 (0.0009) [2023-10-14 20:50:14,813][61552] Updated weights for policy 0, policy_version 76732 (0.0012) [2023-10-14 20:50:17,989][61585] Updated weights for policy 1, policy_version 76580 (0.0008) [2023-10-14 20:50:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 156991488. Throughput: 0: 1670.4, 1: 1671.2. Samples: 39256726. Policy #0 lag: (min: 10.0, avg: 11.3, max: 35.0) [2023-10-14 20:50:18,344][60425] Avg episode reward: [(0, '73.580'), (1, '74.320')] [2023-10-14 20:50:18,346][61585] Updated weights for policy 1, policy_version 76590 (0.0009) [2023-10-14 20:50:18,713][61585] Updated weights for policy 1, policy_version 76600 (0.0008) [2023-10-14 20:50:19,018][61552] Updated weights for policy 0, policy_version 76742 (0.0009) [2023-10-14 20:50:19,385][61552] Updated weights for policy 0, policy_version 76752 (0.0010) [2023-10-14 20:50:19,760][61552] Updated weights for policy 0, policy_version 76762 (0.0009) [2023-10-14 20:50:22,758][61585] Updated weights for policy 1, policy_version 76610 (0.0009) [2023-10-14 20:50:23,127][61585] Updated weights for policy 1, policy_version 76620 (0.0010) [2023-10-14 20:50:23,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 157057024. Throughput: 0: 1669.7, 1: 1672.7. Samples: 39277250. Policy #0 lag: (min: 10.0, avg: 11.3, max: 35.0) [2023-10-14 20:50:23,344][60425] Avg episode reward: [(0, '73.410'), (1, '74.130')] [2023-10-14 20:50:23,483][61585] Updated weights for policy 1, policy_version 76630 (0.0007) [2023-10-14 20:50:23,775][61552] Updated weights for policy 0, policy_version 76772 (0.0010) [2023-10-14 20:50:23,851][61585] Updated weights for policy 1, policy_version 76640 (0.0008) [2023-10-14 20:50:24,147][61552] Updated weights for policy 0, policy_version 76782 (0.0009) [2023-10-14 20:50:24,514][61552] Updated weights for policy 0, policy_version 76792 (0.0010) [2023-10-14 20:50:27,772][61585] Updated weights for policy 1, policy_version 76650 (0.0008) [2023-10-14 20:50:28,132][61585] Updated weights for policy 1, policy_version 76660 (0.0008) [2023-10-14 20:50:28,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 157122560. Throughput: 0: 1671.2, 1: 1666.9. Samples: 39297630. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-14 20:50:28,344][60425] Avg episode reward: [(0, '75.060'), (1, '74.660')] [2023-10-14 20:50:28,489][61585] Updated weights for policy 1, policy_version 76670 (0.0009) [2023-10-14 20:50:28,687][61552] Updated weights for policy 0, policy_version 76802 (0.0008) [2023-10-14 20:50:29,055][61552] Updated weights for policy 0, policy_version 76812 (0.0007) [2023-10-14 20:50:29,434][61552] Updated weights for policy 0, policy_version 76822 (0.0008) [2023-10-14 20:50:29,799][61552] Updated weights for policy 0, policy_version 76832 (0.0012) [2023-10-14 20:50:32,567][61585] Updated weights for policy 1, policy_version 76680 (0.0007) [2023-10-14 20:50:32,928][61585] Updated weights for policy 1, policy_version 76690 (0.0007) [2023-10-14 20:50:33,290][61585] Updated weights for policy 1, policy_version 76700 (0.0008) [2023-10-14 20:50:33,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 157188096. Throughput: 0: 1666.9, 1: 1678.8. Samples: 39307094. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-14 20:50:33,344][60425] Avg episode reward: [(0, '76.960'), (1, '77.150')] [2023-10-14 20:50:33,938][61552] Updated weights for policy 0, policy_version 76842 (0.0008) [2023-10-14 20:50:34,301][61552] Updated weights for policy 0, policy_version 76852 (0.0008) [2023-10-14 20:50:34,670][61552] Updated weights for policy 0, policy_version 76862 (0.0010) [2023-10-14 20:50:37,485][61585] Updated weights for policy 1, policy_version 76710 (0.0010) [2023-10-14 20:50:37,849][61585] Updated weights for policy 1, policy_version 76720 (0.0007) [2023-10-14 20:50:38,209][61585] Updated weights for policy 1, policy_version 76730 (0.0009) [2023-10-14 20:50:38,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 157253632. Throughput: 0: 1668.9, 1: 1677.1. Samples: 39327676. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-14 20:50:38,344][60425] Avg episode reward: [(0, '76.710'), (1, '75.720')] [2023-10-14 20:50:38,696][61552] Updated weights for policy 0, policy_version 76872 (0.0007) [2023-10-14 20:50:39,063][61552] Updated weights for policy 0, policy_version 76882 (0.0007) [2023-10-14 20:50:39,441][61552] Updated weights for policy 0, policy_version 76892 (0.0008) [2023-10-14 20:50:42,338][61585] Updated weights for policy 1, policy_version 76740 (0.0008) [2023-10-14 20:50:42,696][61585] Updated weights for policy 1, policy_version 76750 (0.0007) [2023-10-14 20:50:43,065][61585] Updated weights for policy 1, policy_version 76760 (0.0008) [2023-10-14 20:50:43,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 157319168. Throughput: 0: 1670.0, 1: 1671.5. Samples: 39347972. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-14 20:50:43,344][60425] Avg episode reward: [(0, '77.180'), (1, '79.940')] [2023-10-14 20:50:43,533][61552] Updated weights for policy 0, policy_version 76902 (0.0008) [2023-10-14 20:50:43,915][61552] Updated weights for policy 0, policy_version 76912 (0.0011) [2023-10-14 20:50:44,286][61552] Updated weights for policy 0, policy_version 76922 (0.0011) [2023-10-14 20:50:47,092][61585] Updated weights for policy 1, policy_version 76770 (0.0008) [2023-10-14 20:50:47,459][61585] Updated weights for policy 1, policy_version 76780 (0.0009) [2023-10-14 20:50:47,819][61585] Updated weights for policy 1, policy_version 76790 (0.0008) [2023-10-14 20:50:48,183][61585] Updated weights for policy 1, policy_version 76800 (0.0008) [2023-10-14 20:50:48,343][60425] Fps is (10 sec: 16384.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 157417472. Throughput: 0: 1667.6, 1: 1680.3. Samples: 39357336. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-14 20:50:48,344][60425] Avg episode reward: [(0, '81.500'), (1, '75.440')] [2023-10-14 20:50:48,441][61552] Updated weights for policy 0, policy_version 76932 (0.0025) [2023-10-14 20:50:48,794][61552] Updated weights for policy 0, policy_version 76942 (0.0010) [2023-10-14 20:50:49,172][61552] Updated weights for policy 0, policy_version 76952 (0.0008) [2023-10-14 20:50:52,382][61585] Updated weights for policy 1, policy_version 76810 (0.0008) [2023-10-14 20:50:52,751][61585] Updated weights for policy 1, policy_version 76820 (0.0007) [2023-10-14 20:50:53,122][61585] Updated weights for policy 1, policy_version 76830 (0.0007) [2023-10-14 20:50:53,230][61552] Updated weights for policy 0, policy_version 76962 (0.0010) [2023-10-14 20:50:53,343][60425] Fps is (10 sec: 16384.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 157483008. Throughput: 0: 1671.2, 1: 1675.6. Samples: 39377752. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-14 20:50:53,344][60425] Avg episode reward: [(0, '78.730'), (1, '79.730')] [2023-10-14 20:50:53,610][61552] Updated weights for policy 0, policy_version 76972 (0.0010) [2023-10-14 20:50:53,984][61552] Updated weights for policy 0, policy_version 76982 (0.0010) [2023-10-14 20:50:54,352][61552] Updated weights for policy 0, policy_version 76992 (0.0011) [2023-10-14 20:50:57,314][61585] Updated weights for policy 1, policy_version 76840 (0.0007) [2023-10-14 20:50:57,679][61585] Updated weights for policy 1, policy_version 76850 (0.0010) [2023-10-14 20:50:58,041][61585] Updated weights for policy 1, policy_version 76860 (0.0008) [2023-10-14 20:50:58,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 157548544. Throughput: 0: 1674.9, 1: 1656.2. Samples: 39397602. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-14 20:50:58,345][60425] Avg episode reward: [(0, '78.100'), (1, '77.790')] [2023-10-14 20:50:58,373][61552] Updated weights for policy 0, policy_version 77002 (0.0007) [2023-10-14 20:50:58,735][61552] Updated weights for policy 0, policy_version 77012 (0.0008) [2023-10-14 20:50:59,096][61552] Updated weights for policy 0, policy_version 77022 (0.0007) [2023-10-14 20:51:01,877][61585] Updated weights for policy 1, policy_version 76870 (0.0008) [2023-10-14 20:51:02,239][61585] Updated weights for policy 1, policy_version 76880 (0.0007) [2023-10-14 20:51:02,599][61585] Updated weights for policy 1, policy_version 76890 (0.0007) [2023-10-14 20:51:03,224][61552] Updated weights for policy 0, policy_version 77032 (0.0007) [2023-10-14 20:51:03,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 157614080. Throughput: 0: 1677.6, 1: 1673.0. Samples: 39407506. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-14 20:51:03,344][60425] Avg episode reward: [(0, '83.210'), (1, '80.620')] [2023-10-14 20:51:03,589][61552] Updated weights for policy 0, policy_version 77042 (0.0009) [2023-10-14 20:51:03,962][61552] Updated weights for policy 0, policy_version 77052 (0.0009) [2023-10-14 20:51:06,660][61585] Updated weights for policy 1, policy_version 76900 (0.0008) [2023-10-14 20:51:07,034][61585] Updated weights for policy 1, policy_version 76910 (0.0007) [2023-10-14 20:51:07,404][61585] Updated weights for policy 1, policy_version 76920 (0.0007) [2023-10-14 20:51:08,046][61552] Updated weights for policy 0, policy_version 77062 (0.0008) [2023-10-14 20:51:08,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 157679616. Throughput: 0: 1675.2, 1: 1671.2. Samples: 39427840. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-14 20:51:08,344][60425] Avg episode reward: [(0, '75.370'), (1, '79.300')] [2023-10-14 20:51:08,410][61552] Updated weights for policy 0, policy_version 77072 (0.0007) [2023-10-14 20:51:08,772][61552] Updated weights for policy 0, policy_version 77082 (0.0009) [2023-10-14 20:51:11,482][61585] Updated weights for policy 1, policy_version 76930 (0.0007) [2023-10-14 20:51:11,838][61585] Updated weights for policy 1, policy_version 76940 (0.0010) [2023-10-14 20:51:12,198][61585] Updated weights for policy 1, policy_version 76950 (0.0011) [2023-10-14 20:51:12,566][61585] Updated weights for policy 1, policy_version 76960 (0.0010) [2023-10-14 20:51:12,768][61552] Updated weights for policy 0, policy_version 77092 (0.0007) [2023-10-14 20:51:13,134][61552] Updated weights for policy 0, policy_version 77102 (0.0008) [2023-10-14 20:51:13,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 157745152. Throughput: 0: 1672.1, 1: 1654.4. Samples: 39447324. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-14 20:51:13,344][60425] Avg episode reward: [(0, '77.180'), (1, '76.050')] [2023-10-14 20:51:13,495][61552] Updated weights for policy 0, policy_version 77112 (0.0008) [2023-10-14 20:51:16,712][61585] Updated weights for policy 1, policy_version 76970 (0.0007) [2023-10-14 20:51:17,076][61585] Updated weights for policy 1, policy_version 76980 (0.0007) [2023-10-14 20:51:17,433][61585] Updated weights for policy 1, policy_version 76990 (0.0008) [2023-10-14 20:51:17,494][61552] Updated weights for policy 0, policy_version 77122 (0.0010) [2023-10-14 20:51:17,869][61552] Updated weights for policy 0, policy_version 77132 (0.0010) [2023-10-14 20:51:18,234][61552] Updated weights for policy 0, policy_version 77142 (0.0009) [2023-10-14 20:51:18,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 157810688. Throughput: 0: 1676.8, 1: 1674.5. Samples: 39457906. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-14 20:51:18,344][60425] Avg episode reward: [(0, '76.290'), (1, '80.850')] [2023-10-14 20:51:18,598][61552] Updated weights for policy 0, policy_version 77152 (0.0007) [2023-10-14 20:51:21,543][61585] Updated weights for policy 1, policy_version 77000 (0.0011) [2023-10-14 20:51:21,903][61585] Updated weights for policy 1, policy_version 77010 (0.0010) [2023-10-14 20:51:22,263][61585] Updated weights for policy 1, policy_version 77020 (0.0009) [2023-10-14 20:51:22,652][61552] Updated weights for policy 0, policy_version 77162 (0.0008) [2023-10-14 20:51:23,018][61552] Updated weights for policy 0, policy_version 77172 (0.0007) [2023-10-14 20:51:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 157876224. Throughput: 0: 1676.4, 1: 1661.6. Samples: 39477884. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-14 20:51:23,344][60425] Avg episode reward: [(0, '78.920'), (1, '77.980')] [2023-10-14 20:51:23,391][61552] Updated weights for policy 0, policy_version 77182 (0.0009) [2023-10-14 20:51:26,419][61585] Updated weights for policy 1, policy_version 77030 (0.0009) [2023-10-14 20:51:26,780][61585] Updated weights for policy 1, policy_version 77040 (0.0009) [2023-10-14 20:51:27,147][61585] Updated weights for policy 1, policy_version 77050 (0.0010) [2023-10-14 20:51:27,575][61552] Updated weights for policy 0, policy_version 77192 (0.0010) [2023-10-14 20:51:27,936][61552] Updated weights for policy 0, policy_version 77202 (0.0009) [2023-10-14 20:51:28,307][61552] Updated weights for policy 0, policy_version 77212 (0.0007) [2023-10-14 20:51:28,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13218.3). Total num frames: 157941760. Throughput: 0: 1666.9, 1: 1654.2. Samples: 39497422. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-14 20:51:28,344][60425] Avg episode reward: [(0, '74.690'), (1, '81.480')] [2023-10-14 20:51:31,378][61585] Updated weights for policy 1, policy_version 77060 (0.0009) [2023-10-14 20:51:31,753][61585] Updated weights for policy 1, policy_version 77070 (0.0008) [2023-10-14 20:51:32,121][61585] Updated weights for policy 1, policy_version 77080 (0.0008) [2023-10-14 20:51:32,388][61552] Updated weights for policy 0, policy_version 77222 (0.0007) [2023-10-14 20:51:32,776][61552] Updated weights for policy 0, policy_version 77232 (0.0009) [2023-10-14 20:51:33,152][61552] Updated weights for policy 0, policy_version 77242 (0.0008) [2023-10-14 20:51:33,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13218.3). Total num frames: 158007296. Throughput: 0: 1684.3, 1: 1673.6. Samples: 39508442. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-14 20:51:33,344][60425] Avg episode reward: [(0, '74.610'), (1, '78.380')] [2023-10-14 20:51:36,252][61585] Updated weights for policy 1, policy_version 77090 (0.0007) [2023-10-14 20:51:36,612][61585] Updated weights for policy 1, policy_version 77100 (0.0008) [2023-10-14 20:51:36,982][61585] Updated weights for policy 1, policy_version 77110 (0.0010) [2023-10-14 20:51:37,169][61552] Updated weights for policy 0, policy_version 77252 (0.0008) [2023-10-14 20:51:37,342][61585] Updated weights for policy 1, policy_version 77120 (0.0008) [2023-10-14 20:51:37,547][61552] Updated weights for policy 0, policy_version 77262 (0.0008) [2023-10-14 20:51:37,900][61552] Updated weights for policy 0, policy_version 77272 (0.0010) [2023-10-14 20:51:38,343][60425] Fps is (10 sec: 16383.7, 60 sec: 14199.4, 300 sec: 13329.4). Total num frames: 158105600. Throughput: 0: 1682.6, 1: 1666.8. Samples: 39528476. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-14 20:51:38,344][60425] Avg episode reward: [(0, '75.810'), (1, '80.290')] [2023-10-14 20:51:41,293][61585] Updated weights for policy 1, policy_version 77130 (0.0009) [2023-10-14 20:51:41,666][61585] Updated weights for policy 1, policy_version 77140 (0.0008) [2023-10-14 20:51:42,026][61585] Updated weights for policy 1, policy_version 77150 (0.0009) [2023-10-14 20:51:42,105][61552] Updated weights for policy 0, policy_version 77282 (0.0011) [2023-10-14 20:51:42,476][61552] Updated weights for policy 0, policy_version 77292 (0.0007) [2023-10-14 20:51:42,840][61552] Updated weights for policy 0, policy_version 77302 (0.0008) [2023-10-14 20:51:43,206][61552] Updated weights for policy 0, policy_version 77312 (0.0010) [2023-10-14 20:51:43,343][60425] Fps is (10 sec: 16384.2, 60 sec: 14199.5, 300 sec: 13329.4). Total num frames: 158171136. Throughput: 0: 1660.5, 1: 1678.3. Samples: 39547844. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-14 20:51:43,344][60425] Avg episode reward: [(0, '75.200'), (1, '78.640')] [2023-10-14 20:51:43,355][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000077312_79167488.pth... [2023-10-14 20:51:43,355][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000077152_79003648.pth... [2023-10-14 20:51:43,386][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000075744_77561856.pth [2023-10-14 20:51:43,386][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000075584_77398016.pth [2023-10-14 20:51:46,334][61585] Updated weights for policy 1, policy_version 77160 (0.0009) [2023-10-14 20:51:46,718][61585] Updated weights for policy 1, policy_version 77170 (0.0008) [2023-10-14 20:51:47,082][61585] Updated weights for policy 1, policy_version 77180 (0.0008) [2023-10-14 20:51:47,437][61552] Updated weights for policy 0, policy_version 77322 (0.0008) [2023-10-14 20:51:47,801][61552] Updated weights for policy 0, policy_version 77332 (0.0008) [2023-10-14 20:51:48,174][61552] Updated weights for policy 0, policy_version 77342 (0.0009) [2023-10-14 20:51:48,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 158236672. Throughput: 0: 1671.2, 1: 1688.0. Samples: 39558674. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-14 20:51:48,344][60425] Avg episode reward: [(0, '78.280'), (1, '78.050')] [2023-10-14 20:51:51,049][61585] Updated weights for policy 1, policy_version 77190 (0.0008) [2023-10-14 20:51:51,408][61585] Updated weights for policy 1, policy_version 77200 (0.0012) [2023-10-14 20:51:51,778][61585] Updated weights for policy 1, policy_version 77210 (0.0008) [2023-10-14 20:51:52,421][61552] Updated weights for policy 0, policy_version 77352 (0.0008) [2023-10-14 20:51:52,803][61552] Updated weights for policy 0, policy_version 77362 (0.0007) [2023-10-14 20:51:53,168][61552] Updated weights for policy 0, policy_version 77372 (0.0007) [2023-10-14 20:51:53,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 158302208. Throughput: 0: 1669.4, 1: 1667.2. Samples: 39577988. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-14 20:51:53,344][60425] Avg episode reward: [(0, '77.200'), (1, '81.710')] [2023-10-14 20:51:55,755][61585] Updated weights for policy 1, policy_version 77220 (0.0008) [2023-10-14 20:51:56,121][61585] Updated weights for policy 1, policy_version 77230 (0.0008) [2023-10-14 20:51:56,489][61585] Updated weights for policy 1, policy_version 77240 (0.0008) [2023-10-14 20:51:57,294][61552] Updated weights for policy 0, policy_version 77382 (0.0009) [2023-10-14 20:51:57,660][61552] Updated weights for policy 0, policy_version 77392 (0.0008) [2023-10-14 20:51:58,032][61552] Updated weights for policy 0, policy_version 77402 (0.0010) [2023-10-14 20:51:58,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 158367744. Throughput: 0: 1657.2, 1: 1688.2. Samples: 39597868. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-14 20:51:58,344][60425] Avg episode reward: [(0, '75.170'), (1, '78.120')] [2023-10-14 20:52:00,640][61585] Updated weights for policy 1, policy_version 77250 (0.0007) [2023-10-14 20:52:01,008][61585] Updated weights for policy 1, policy_version 77260 (0.0009) [2023-10-14 20:52:01,379][61585] Updated weights for policy 1, policy_version 77270 (0.0008) [2023-10-14 20:52:01,743][61585] Updated weights for policy 1, policy_version 77280 (0.0009) [2023-10-14 20:52:01,908][61552] Updated weights for policy 0, policy_version 77412 (0.0010) [2023-10-14 20:52:02,287][61552] Updated weights for policy 0, policy_version 77422 (0.0007) [2023-10-14 20:52:02,650][61552] Updated weights for policy 0, policy_version 77432 (0.0008) [2023-10-14 20:52:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 158433280. Throughput: 0: 1671.7, 1: 1678.2. Samples: 39608654. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-14 20:52:03,344][60425] Avg episode reward: [(0, '71.770'), (1, '73.410')] [2023-10-14 20:52:05,681][61585] Updated weights for policy 1, policy_version 77290 (0.0008) [2023-10-14 20:52:06,051][61585] Updated weights for policy 1, policy_version 77300 (0.0010) [2023-10-14 20:52:06,408][61585] Updated weights for policy 1, policy_version 77310 (0.0009) [2023-10-14 20:52:06,603][61552] Updated weights for policy 0, policy_version 77442 (0.0009) [2023-10-14 20:52:06,979][61552] Updated weights for policy 0, policy_version 77452 (0.0010) [2023-10-14 20:52:07,353][61552] Updated weights for policy 0, policy_version 77462 (0.0011) [2023-10-14 20:52:07,720][61552] Updated weights for policy 0, policy_version 77472 (0.0010) [2023-10-14 20:52:08,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 158498816. Throughput: 0: 1672.9, 1: 1667.0. Samples: 39628178. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-14 20:52:08,344][60425] Avg episode reward: [(0, '77.660'), (1, '76.640')] [2023-10-14 20:52:10,365][61585] Updated weights for policy 1, policy_version 77320 (0.0008) [2023-10-14 20:52:10,727][61585] Updated weights for policy 1, policy_version 77330 (0.0008) [2023-10-14 20:52:11,080][61585] Updated weights for policy 1, policy_version 77340 (0.0009) [2023-10-14 20:52:11,802][61552] Updated weights for policy 0, policy_version 77482 (0.0007) [2023-10-14 20:52:12,179][61552] Updated weights for policy 0, policy_version 77492 (0.0008) [2023-10-14 20:52:12,544][61552] Updated weights for policy 0, policy_version 77502 (0.0009) [2023-10-14 20:52:13,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 158564352. Throughput: 0: 1658.9, 1: 1692.6. Samples: 39648240. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-14 20:52:13,344][60425] Avg episode reward: [(0, '77.860'), (1, '80.260')] [2023-10-14 20:52:15,169][61585] Updated weights for policy 1, policy_version 77350 (0.0009) [2023-10-14 20:52:15,535][61585] Updated weights for policy 1, policy_version 77360 (0.0007) [2023-10-14 20:52:15,903][61585] Updated weights for policy 1, policy_version 77370 (0.0009) [2023-10-14 20:52:16,494][61552] Updated weights for policy 0, policy_version 77512 (0.0007) [2023-10-14 20:52:16,865][61552] Updated weights for policy 0, policy_version 77522 (0.0008) [2023-10-14 20:52:17,224][61552] Updated weights for policy 0, policy_version 77532 (0.0008) [2023-10-14 20:52:18,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 158629888. Throughput: 0: 1674.5, 1: 1671.1. Samples: 39658994. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-14 20:52:18,344][60425] Avg episode reward: [(0, '74.900'), (1, '80.510')] [2023-10-14 20:52:19,968][61585] Updated weights for policy 1, policy_version 77380 (0.0010) [2023-10-14 20:52:20,345][61585] Updated weights for policy 1, policy_version 77390 (0.0010) [2023-10-14 20:52:20,706][61585] Updated weights for policy 1, policy_version 77400 (0.0010) [2023-10-14 20:52:21,291][61552] Updated weights for policy 0, policy_version 77542 (0.0009) [2023-10-14 20:52:21,665][61552] Updated weights for policy 0, policy_version 77552 (0.0008) [2023-10-14 20:52:22,035][61552] Updated weights for policy 0, policy_version 77562 (0.0007) [2023-10-14 20:52:23,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 158695424. Throughput: 0: 1658.4, 1: 1669.3. Samples: 39678224. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-14 20:52:23,344][60425] Avg episode reward: [(0, '77.160'), (1, '76.780')] [2023-10-14 20:52:24,911][61585] Updated weights for policy 1, policy_version 77410 (0.0010) [2023-10-14 20:52:25,274][61585] Updated weights for policy 1, policy_version 77420 (0.0010) [2023-10-14 20:52:25,641][61585] Updated weights for policy 1, policy_version 77430 (0.0007) [2023-10-14 20:52:26,014][61585] Updated weights for policy 1, policy_version 77440 (0.0007) [2023-10-14 20:52:26,228][61552] Updated weights for policy 0, policy_version 77572 (0.0007) [2023-10-14 20:52:26,596][61552] Updated weights for policy 0, policy_version 77582 (0.0008) [2023-10-14 20:52:26,965][61552] Updated weights for policy 0, policy_version 77592 (0.0007) [2023-10-14 20:52:28,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 158760960. Throughput: 0: 1663.3, 1: 1675.6. Samples: 39698098. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-14 20:52:28,344][60425] Avg episode reward: [(0, '79.230'), (1, '79.150')] [2023-10-14 20:52:30,067][61585] Updated weights for policy 1, policy_version 77450 (0.0011) [2023-10-14 20:52:30,439][61585] Updated weights for policy 1, policy_version 77460 (0.0010) [2023-10-14 20:52:30,794][61585] Updated weights for policy 1, policy_version 77470 (0.0011) [2023-10-14 20:52:31,104][61552] Updated weights for policy 0, policy_version 77602 (0.0008) [2023-10-14 20:52:31,462][61552] Updated weights for policy 0, policy_version 77612 (0.0008) [2023-10-14 20:52:31,828][61552] Updated weights for policy 0, policy_version 77622 (0.0009) [2023-10-14 20:52:32,191][61552] Updated weights for policy 0, policy_version 77632 (0.0007) [2023-10-14 20:52:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 158826496. Throughput: 0: 1678.7, 1: 1654.0. Samples: 39708644. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-14 20:52:33,344][60425] Avg episode reward: [(0, '78.820'), (1, '74.550')] [2023-10-14 20:52:35,053][61585] Updated weights for policy 1, policy_version 77480 (0.0010) [2023-10-14 20:52:35,433][61585] Updated weights for policy 1, policy_version 77490 (0.0007) [2023-10-14 20:52:35,811][61585] Updated weights for policy 1, policy_version 77500 (0.0010) [2023-10-14 20:52:36,289][61552] Updated weights for policy 0, policy_version 77642 (0.0010) [2023-10-14 20:52:36,660][61552] Updated weights for policy 0, policy_version 77652 (0.0009) [2023-10-14 20:52:37,028][61552] Updated weights for policy 0, policy_version 77662 (0.0010) [2023-10-14 20:52:38,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 158892032. Throughput: 0: 1665.3, 1: 1672.4. Samples: 39728184. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-14 20:52:38,344][60425] Avg episode reward: [(0, '81.470'), (1, '80.990')] [2023-10-14 20:52:40,042][61585] Updated weights for policy 1, policy_version 77510 (0.0011) [2023-10-14 20:52:40,413][61585] Updated weights for policy 1, policy_version 77520 (0.0007) [2023-10-14 20:52:40,774][61585] Updated weights for policy 1, policy_version 77530 (0.0008) [2023-10-14 20:52:40,929][61552] Updated weights for policy 0, policy_version 77672 (0.0008) [2023-10-14 20:52:41,294][61552] Updated weights for policy 0, policy_version 77682 (0.0008) [2023-10-14 20:52:41,656][61552] Updated weights for policy 0, policy_version 77692 (0.0009) [2023-10-14 20:52:43,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 158957568. Throughput: 0: 1673.7, 1: 1675.8. Samples: 39748598. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-14 20:52:43,344][60425] Avg episode reward: [(0, '81.220'), (1, '75.470')] [2023-10-14 20:52:44,896][61585] Updated weights for policy 1, policy_version 77540 (0.0007) [2023-10-14 20:52:45,257][61585] Updated weights for policy 1, policy_version 77550 (0.0007) [2023-10-14 20:52:45,622][61585] Updated weights for policy 1, policy_version 77560 (0.0007) [2023-10-14 20:52:45,963][61552] Updated weights for policy 0, policy_version 77702 (0.0008) [2023-10-14 20:52:46,329][61552] Updated weights for policy 0, policy_version 77712 (0.0010) [2023-10-14 20:52:46,706][61552] Updated weights for policy 0, policy_version 77722 (0.0009) [2023-10-14 20:52:48,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 159023104. Throughput: 0: 1687.2, 1: 1657.5. Samples: 39759164. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-14 20:52:48,344][60425] Avg episode reward: [(0, '76.300'), (1, '75.480')] [2023-10-14 20:52:49,710][61585] Updated weights for policy 1, policy_version 77570 (0.0008) [2023-10-14 20:52:50,080][61585] Updated weights for policy 1, policy_version 77580 (0.0009) [2023-10-14 20:52:50,455][61585] Updated weights for policy 1, policy_version 77590 (0.0007) [2023-10-14 20:52:50,646][61552] Updated weights for policy 0, policy_version 77732 (0.0008) [2023-10-14 20:52:50,823][61585] Updated weights for policy 1, policy_version 77600 (0.0007) [2023-10-14 20:52:51,021][61552] Updated weights for policy 0, policy_version 77742 (0.0008) [2023-10-14 20:52:51,383][61552] Updated weights for policy 0, policy_version 77752 (0.0009) [2023-10-14 20:52:53,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 159088640. Throughput: 0: 1659.1, 1: 1678.7. Samples: 39778384. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-14 20:52:53,344][60425] Avg episode reward: [(0, '76.220'), (1, '79.950')] [2023-10-14 20:52:54,619][61585] Updated weights for policy 1, policy_version 77610 (0.0010) [2023-10-14 20:52:54,968][61585] Updated weights for policy 1, policy_version 77620 (0.0008) [2023-10-14 20:52:55,338][61585] Updated weights for policy 1, policy_version 77630 (0.0011) [2023-10-14 20:52:55,494][61552] Updated weights for policy 0, policy_version 77762 (0.0007) [2023-10-14 20:52:55,866][61552] Updated weights for policy 0, policy_version 77772 (0.0007) [2023-10-14 20:52:56,231][61552] Updated weights for policy 0, policy_version 77782 (0.0010) [2023-10-14 20:52:56,604][61552] Updated weights for policy 0, policy_version 77792 (0.0008) [2023-10-14 20:52:58,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 159154176. Throughput: 0: 1679.6, 1: 1672.1. Samples: 39799068. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-14 20:52:58,344][60425] Avg episode reward: [(0, '76.550'), (1, '77.000')] [2023-10-14 20:52:59,302][61585] Updated weights for policy 1, policy_version 77640 (0.0009) [2023-10-14 20:52:59,664][61585] Updated weights for policy 1, policy_version 77650 (0.0008) [2023-10-14 20:53:00,033][61585] Updated weights for policy 1, policy_version 77660 (0.0009) [2023-10-14 20:53:00,660][61552] Updated weights for policy 0, policy_version 77802 (0.0009) [2023-10-14 20:53:01,020][61552] Updated weights for policy 0, policy_version 77812 (0.0009) [2023-10-14 20:53:01,385][61552] Updated weights for policy 0, policy_version 77822 (0.0011) [2023-10-14 20:53:03,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 159219712. Throughput: 0: 1673.3, 1: 1666.6. Samples: 39809292. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-14 20:53:03,344][60425] Avg episode reward: [(0, '77.120'), (1, '77.370')] [2023-10-14 20:53:04,303][61585] Updated weights for policy 1, policy_version 77670 (0.0008) [2023-10-14 20:53:04,671][61585] Updated weights for policy 1, policy_version 77680 (0.0009) [2023-10-14 20:53:05,043][61585] Updated weights for policy 1, policy_version 77690 (0.0007) [2023-10-14 20:53:05,400][61552] Updated weights for policy 0, policy_version 77832 (0.0009) [2023-10-14 20:53:05,770][61552] Updated weights for policy 0, policy_version 77842 (0.0007) [2023-10-14 20:53:06,135][61552] Updated weights for policy 0, policy_version 77852 (0.0009) [2023-10-14 20:53:08,344][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 159285248. Throughput: 0: 1673.5, 1: 1679.9. Samples: 39829128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-14 20:53:08,345][60425] Avg episode reward: [(0, '78.770'), (1, '79.630')] [2023-10-14 20:53:09,184][61585] Updated weights for policy 1, policy_version 77700 (0.0008) [2023-10-14 20:53:09,552][61585] Updated weights for policy 1, policy_version 77710 (0.0008) [2023-10-14 20:53:09,916][61585] Updated weights for policy 1, policy_version 77720 (0.0009) [2023-10-14 20:53:10,136][61552] Updated weights for policy 0, policy_version 77862 (0.0009) [2023-10-14 20:53:10,520][61552] Updated weights for policy 0, policy_version 77872 (0.0010) [2023-10-14 20:53:10,884][61552] Updated weights for policy 0, policy_version 77882 (0.0009) [2023-10-14 20:53:13,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 159350784. Throughput: 0: 1680.5, 1: 1685.2. Samples: 39849558. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-14 20:53:13,344][60425] Avg episode reward: [(0, '74.750'), (1, '80.560')] [2023-10-14 20:53:13,928][61585] Updated weights for policy 1, policy_version 77730 (0.0009) [2023-10-14 20:53:14,293][61585] Updated weights for policy 1, policy_version 77740 (0.0010) [2023-10-14 20:53:14,659][61585] Updated weights for policy 1, policy_version 77750 (0.0008) [2023-10-14 20:53:15,022][61585] Updated weights for policy 1, policy_version 77760 (0.0009) [2023-10-14 20:53:15,074][61552] Updated weights for policy 0, policy_version 77892 (0.0009) [2023-10-14 20:53:15,449][61552] Updated weights for policy 0, policy_version 77902 (0.0009) [2023-10-14 20:53:15,805][61552] Updated weights for policy 0, policy_version 77912 (0.0009) [2023-10-14 20:53:18,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 159416320. Throughput: 0: 1667.7, 1: 1679.1. Samples: 39859250. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-14 20:53:18,344][60425] Avg episode reward: [(0, '76.850'), (1, '85.460')] [2023-10-14 20:53:18,346][61248] Saving new best policy, reward=85.460! [2023-10-14 20:53:19,151][61585] Updated weights for policy 1, policy_version 77770 (0.0009) [2023-10-14 20:53:19,521][61585] Updated weights for policy 1, policy_version 77780 (0.0009) [2023-10-14 20:53:19,886][61585] Updated weights for policy 1, policy_version 77790 (0.0009) [2023-10-14 20:53:19,891][61552] Updated weights for policy 0, policy_version 77922 (0.0009) [2023-10-14 20:53:20,260][61552] Updated weights for policy 0, policy_version 77932 (0.0009) [2023-10-14 20:53:20,617][61552] Updated weights for policy 0, policy_version 77942 (0.0007) [2023-10-14 20:53:20,987][61552] Updated weights for policy 0, policy_version 77952 (0.0010) [2023-10-14 20:53:23,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 159481856. Throughput: 0: 1670.9, 1: 1683.8. Samples: 39879148. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-14 20:53:23,345][60425] Avg episode reward: [(0, '76.380'), (1, '77.590')] [2023-10-14 20:53:23,917][61585] Updated weights for policy 1, policy_version 77800 (0.0010) [2023-10-14 20:53:24,286][61585] Updated weights for policy 1, policy_version 77810 (0.0010) [2023-10-14 20:53:24,655][61585] Updated weights for policy 1, policy_version 77820 (0.0009) [2023-10-14 20:53:25,224][61552] Updated weights for policy 0, policy_version 77962 (0.0007) [2023-10-14 20:53:25,580][61552] Updated weights for policy 0, policy_version 77972 (0.0009) [2023-10-14 20:53:25,954][61552] Updated weights for policy 0, policy_version 77982 (0.0007) [2023-10-14 20:53:28,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 159547392. Throughput: 0: 1677.5, 1: 1683.5. Samples: 39899840. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-14 20:53:28,344][60425] Avg episode reward: [(0, '77.510'), (1, '83.480')] [2023-10-14 20:53:28,712][61585] Updated weights for policy 1, policy_version 77830 (0.0008) [2023-10-14 20:53:29,081][61585] Updated weights for policy 1, policy_version 77840 (0.0007) [2023-10-14 20:53:29,450][61585] Updated weights for policy 1, policy_version 77850 (0.0010) [2023-10-14 20:53:30,004][61552] Updated weights for policy 0, policy_version 77992 (0.0008) [2023-10-14 20:53:30,380][61552] Updated weights for policy 0, policy_version 78002 (0.0007) [2023-10-14 20:53:30,751][61552] Updated weights for policy 0, policy_version 78012 (0.0007) [2023-10-14 20:53:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 159612928. Throughput: 0: 1653.1, 1: 1682.2. Samples: 39909252. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-14 20:53:33,344][60425] Avg episode reward: [(0, '79.960'), (1, '85.520')] [2023-10-14 20:53:33,557][61585] Updated weights for policy 1, policy_version 77860 (0.0010) [2023-10-14 20:53:33,922][61585] Updated weights for policy 1, policy_version 77870 (0.0007) [2023-10-14 20:53:34,284][61585] Updated weights for policy 1, policy_version 77880 (0.0007) [2023-10-14 20:53:34,565][61248] Saving new best policy, reward=85.520! [2023-10-14 20:53:34,816][61552] Updated weights for policy 0, policy_version 78022 (0.0009) [2023-10-14 20:53:35,181][61552] Updated weights for policy 0, policy_version 78032 (0.0009) [2023-10-14 20:53:35,552][61552] Updated weights for policy 0, policy_version 78042 (0.0008) [2023-10-14 20:53:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 159678464. Throughput: 0: 1676.8, 1: 1684.4. Samples: 39929636. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-14 20:53:38,344][60425] Avg episode reward: [(0, '79.710'), (1, '78.560')] [2023-10-14 20:53:38,410][61585] Updated weights for policy 1, policy_version 77890 (0.0009) [2023-10-14 20:53:38,777][61585] Updated weights for policy 1, policy_version 77900 (0.0007) [2023-10-14 20:53:39,136][61585] Updated weights for policy 1, policy_version 77910 (0.0007) [2023-10-14 20:53:39,489][61552] Updated weights for policy 0, policy_version 78052 (0.0007) [2023-10-14 20:53:39,505][61585] Updated weights for policy 1, policy_version 77920 (0.0008) [2023-10-14 20:53:39,859][61552] Updated weights for policy 0, policy_version 78062 (0.0007) [2023-10-14 20:53:40,226][61552] Updated weights for policy 0, policy_version 78072 (0.0008) [2023-10-14 20:53:43,344][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 159744000. Throughput: 0: 1678.9, 1: 1686.0. Samples: 39950490. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:53:43,345][60425] Avg episode reward: [(0, '81.070'), (1, '79.690')] [2023-10-14 20:53:43,355][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000078080_79953920.pth... [2023-10-14 20:53:43,372][61585] Updated weights for policy 1, policy_version 77930 (0.0009) [2023-10-14 20:53:43,395][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000076512_78348288.pth [2023-10-14 20:53:43,400][61172] Saving a milestone ./train_atari/atari_roadrunner_APPO/checkpoint_p0/milestones/checkpoint_000078080_79953920.pth [2023-10-14 20:53:43,731][61585] Updated weights for policy 1, policy_version 77940 (0.0011) [2023-10-14 20:53:44,099][61585] Updated weights for policy 1, policy_version 77950 (0.0010) [2023-10-14 20:53:44,167][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000077952_79822848.pth... [2023-10-14 20:53:44,206][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000076352_78184448.pth [2023-10-14 20:53:44,211][61248] Saving a milestone ./train_atari/atari_roadrunner_APPO/checkpoint_p1/milestones/checkpoint_000077952_79822848.pth [2023-10-14 20:53:44,397][61552] Updated weights for policy 0, policy_version 78082 (0.0009) [2023-10-14 20:53:44,772][61552] Updated weights for policy 0, policy_version 78092 (0.0008) [2023-10-14 20:53:45,147][61552] Updated weights for policy 0, policy_version 78102 (0.0008) [2023-10-14 20:53:45,518][61552] Updated weights for policy 0, policy_version 78112 (0.0007) [2023-10-14 20:53:48,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 159809536. Throughput: 0: 1657.6, 1: 1681.8. Samples: 39959566. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:53:48,344][60425] Avg episode reward: [(0, '76.350'), (1, '82.330')] [2023-10-14 20:53:48,371][61585] Updated weights for policy 1, policy_version 77960 (0.0008) [2023-10-14 20:53:48,746][61585] Updated weights for policy 1, policy_version 77970 (0.0007) [2023-10-14 20:53:49,112][61585] Updated weights for policy 1, policy_version 77980 (0.0008) [2023-10-14 20:53:49,582][61552] Updated weights for policy 0, policy_version 78122 (0.0008) [2023-10-14 20:53:49,948][61552] Updated weights for policy 0, policy_version 78132 (0.0008) [2023-10-14 20:53:50,302][61552] Updated weights for policy 0, policy_version 78142 (0.0009) [2023-10-14 20:53:53,195][61585] Updated weights for policy 1, policy_version 77990 (0.0010) [2023-10-14 20:53:53,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 159875072. Throughput: 0: 1671.9, 1: 1680.7. Samples: 39979994. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:53:53,344][60425] Avg episode reward: [(0, '77.480'), (1, '82.550')] [2023-10-14 20:53:53,563][61585] Updated weights for policy 1, policy_version 78000 (0.0009) [2023-10-14 20:53:53,934][61585] Updated weights for policy 1, policy_version 78010 (0.0008) [2023-10-14 20:53:54,292][61552] Updated weights for policy 0, policy_version 78152 (0.0009) [2023-10-14 20:53:54,647][61552] Updated weights for policy 0, policy_version 78162 (0.0010) [2023-10-14 20:53:55,017][61552] Updated weights for policy 0, policy_version 78172 (0.0010) [2023-10-14 20:53:58,176][61585] Updated weights for policy 1, policy_version 78020 (0.0010) [2023-10-14 20:53:58,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 159940608. Throughput: 0: 1683.7, 1: 1678.7. Samples: 40000870. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:53:58,344][60425] Avg episode reward: [(0, '75.470'), (1, '77.230')] [2023-10-14 20:53:58,543][61585] Updated weights for policy 1, policy_version 78030 (0.0010) [2023-10-14 20:53:58,916][61585] Updated weights for policy 1, policy_version 78040 (0.0008) [2023-10-14 20:53:59,193][61552] Updated weights for policy 0, policy_version 78182 (0.0008) [2023-10-14 20:53:59,575][61552] Updated weights for policy 0, policy_version 78192 (0.0008) [2023-10-14 20:53:59,943][61552] Updated weights for policy 0, policy_version 78202 (0.0009) [2023-10-14 20:54:02,848][61585] Updated weights for policy 1, policy_version 78050 (0.0009) [2023-10-14 20:54:03,215][61585] Updated weights for policy 1, policy_version 78060 (0.0008) [2023-10-14 20:54:03,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 160006144. Throughput: 0: 1665.9, 1: 1680.6. Samples: 40009842. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:54:03,344][60425] Avg episode reward: [(0, '79.520'), (1, '78.930')] [2023-10-14 20:54:03,584][61585] Updated weights for policy 1, policy_version 78070 (0.0010) [2023-10-14 20:54:03,958][61585] Updated weights for policy 1, policy_version 78080 (0.0008) [2023-10-14 20:54:04,008][61552] Updated weights for policy 0, policy_version 78212 (0.0010) [2023-10-14 20:54:04,381][61552] Updated weights for policy 0, policy_version 78222 (0.0008) [2023-10-14 20:54:04,752][61552] Updated weights for policy 0, policy_version 78232 (0.0010) [2023-10-14 20:54:08,082][61585] Updated weights for policy 1, policy_version 78090 (0.0007) [2023-10-14 20:54:08,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 160071680. Throughput: 0: 1677.8, 1: 1683.1. Samples: 40030388. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:54:08,344][60425] Avg episode reward: [(0, '80.490'), (1, '82.410')] [2023-10-14 20:54:08,455][61585] Updated weights for policy 1, policy_version 78100 (0.0007) [2023-10-14 20:54:08,823][61585] Updated weights for policy 1, policy_version 78110 (0.0009) [2023-10-14 20:54:08,994][61552] Updated weights for policy 0, policy_version 78242 (0.0010) [2023-10-14 20:54:09,357][61552] Updated weights for policy 0, policy_version 78252 (0.0008) [2023-10-14 20:54:09,726][61552] Updated weights for policy 0, policy_version 78262 (0.0009) [2023-10-14 20:54:10,099][61552] Updated weights for policy 0, policy_version 78272 (0.0010) [2023-10-14 20:54:12,880][61585] Updated weights for policy 1, policy_version 78120 (0.0009) [2023-10-14 20:54:13,241][61585] Updated weights for policy 1, policy_version 78130 (0.0010) [2023-10-14 20:54:13,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 160137216. Throughput: 0: 1674.9, 1: 1674.3. Samples: 40050554. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:54:13,344][60425] Avg episode reward: [(0, '79.020'), (1, '77.180')] [2023-10-14 20:54:13,604][61585] Updated weights for policy 1, policy_version 78140 (0.0011) [2023-10-14 20:54:14,187][61552] Updated weights for policy 0, policy_version 78282 (0.0009) [2023-10-14 20:54:14,562][61552] Updated weights for policy 0, policy_version 78292 (0.0009) [2023-10-14 20:54:14,927][61552] Updated weights for policy 0, policy_version 78302 (0.0007) [2023-10-14 20:54:17,688][61585] Updated weights for policy 1, policy_version 78150 (0.0010) [2023-10-14 20:54:18,048][61585] Updated weights for policy 1, policy_version 78160 (0.0011) [2023-10-14 20:54:18,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 160202752. Throughput: 0: 1671.3, 1: 1677.3. Samples: 40059936. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:54:18,344][60425] Avg episode reward: [(0, '75.720'), (1, '79.960')] [2023-10-14 20:54:18,413][61585] Updated weights for policy 1, policy_version 78170 (0.0009) [2023-10-14 20:54:19,085][61552] Updated weights for policy 0, policy_version 78312 (0.0009) [2023-10-14 20:54:19,450][61552] Updated weights for policy 0, policy_version 78322 (0.0009) [2023-10-14 20:54:19,819][61552] Updated weights for policy 0, policy_version 78332 (0.0008) [2023-10-14 20:54:22,442][61585] Updated weights for policy 1, policy_version 78180 (0.0008) [2023-10-14 20:54:22,805][61585] Updated weights for policy 1, policy_version 78190 (0.0008) [2023-10-14 20:54:23,171][61585] Updated weights for policy 1, policy_version 78200 (0.0009) [2023-10-14 20:54:23,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 160268288. Throughput: 0: 1672.1, 1: 1679.1. Samples: 40080440. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:54:23,344][60425] Avg episode reward: [(0, '74.940'), (1, '80.650')] [2023-10-14 20:54:23,882][61552] Updated weights for policy 0, policy_version 78342 (0.0008) [2023-10-14 20:54:24,249][61552] Updated weights for policy 0, policy_version 78352 (0.0008) [2023-10-14 20:54:24,612][61552] Updated weights for policy 0, policy_version 78362 (0.0009) [2023-10-14 20:54:27,329][61585] Updated weights for policy 1, policy_version 78210 (0.0007) [2023-10-14 20:54:27,706][61585] Updated weights for policy 1, policy_version 78220 (0.0010) [2023-10-14 20:54:28,061][61585] Updated weights for policy 1, policy_version 78230 (0.0009) [2023-10-14 20:54:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 160333824. Throughput: 0: 1671.1, 1: 1667.7. Samples: 40100736. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:54:28,344][60425] Avg episode reward: [(0, '77.140'), (1, '81.610')] [2023-10-14 20:54:28,434][61585] Updated weights for policy 1, policy_version 78240 (0.0007) [2023-10-14 20:54:28,651][61552] Updated weights for policy 0, policy_version 78372 (0.0008) [2023-10-14 20:54:29,031][61552] Updated weights for policy 0, policy_version 78382 (0.0010) [2023-10-14 20:54:29,398][61552] Updated weights for policy 0, policy_version 78392 (0.0009) [2023-10-14 20:54:32,470][61585] Updated weights for policy 1, policy_version 78250 (0.0008) [2023-10-14 20:54:32,845][61585] Updated weights for policy 1, policy_version 78260 (0.0007) [2023-10-14 20:54:33,216][61585] Updated weights for policy 1, policy_version 78270 (0.0008) [2023-10-14 20:54:33,343][60425] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 160432128. Throughput: 0: 1671.3, 1: 1680.8. Samples: 40110408. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-14 20:54:33,344][60425] Avg episode reward: [(0, '77.970'), (1, '78.380')] [2023-10-14 20:54:33,412][61552] Updated weights for policy 0, policy_version 78402 (0.0009) [2023-10-14 20:54:33,772][61552] Updated weights for policy 0, policy_version 78412 (0.0010) [2023-10-14 20:54:34,137][61552] Updated weights for policy 0, policy_version 78422 (0.0010) [2023-10-14 20:54:34,506][61552] Updated weights for policy 0, policy_version 78432 (0.0011) [2023-10-14 20:54:37,194][61585] Updated weights for policy 1, policy_version 78280 (0.0008) [2023-10-14 20:54:37,546][61585] Updated weights for policy 1, policy_version 78290 (0.0009) [2023-10-14 20:54:37,906][61585] Updated weights for policy 1, policy_version 78300 (0.0008) [2023-10-14 20:54:38,343][60425] Fps is (10 sec: 16384.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 160497664. Throughput: 0: 1677.5, 1: 1682.9. Samples: 40131212. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-14 20:54:38,344][60425] Avg episode reward: [(0, '77.190'), (1, '80.570')] [2023-10-14 20:54:38,630][61552] Updated weights for policy 0, policy_version 78442 (0.0010) [2023-10-14 20:54:39,006][61552] Updated weights for policy 0, policy_version 78452 (0.0007) [2023-10-14 20:54:39,360][61552] Updated weights for policy 0, policy_version 78462 (0.0007) [2023-10-14 20:54:41,955][61585] Updated weights for policy 1, policy_version 78310 (0.0008) [2023-10-14 20:54:42,334][61585] Updated weights for policy 1, policy_version 78320 (0.0008) [2023-10-14 20:54:42,700][61585] Updated weights for policy 1, policy_version 78330 (0.0009) [2023-10-14 20:54:43,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 160563200. Throughput: 0: 1673.3, 1: 1661.2. Samples: 40150924. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-14 20:54:43,344][60425] Avg episode reward: [(0, '81.120'), (1, '79.040')] [2023-10-14 20:54:43,357][61552] Updated weights for policy 0, policy_version 78472 (0.0007) [2023-10-14 20:54:43,713][61552] Updated weights for policy 0, policy_version 78482 (0.0008) [2023-10-14 20:54:44,075][61552] Updated weights for policy 0, policy_version 78492 (0.0009) [2023-10-14 20:54:46,833][61585] Updated weights for policy 1, policy_version 78340 (0.0009) [2023-10-14 20:54:47,191][61585] Updated weights for policy 1, policy_version 78350 (0.0007) [2023-10-14 20:54:47,553][61585] Updated weights for policy 1, policy_version 78360 (0.0007) [2023-10-14 20:54:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 160628736. Throughput: 0: 1674.5, 1: 1682.9. Samples: 40160928. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-14 20:54:48,344][60425] Avg episode reward: [(0, '77.010'), (1, '80.400')] [2023-10-14 20:54:48,532][61552] Updated weights for policy 0, policy_version 78502 (0.0009) [2023-10-14 20:54:48,906][61552] Updated weights for policy 0, policy_version 78512 (0.0007) [2023-10-14 20:54:49,266][61552] Updated weights for policy 0, policy_version 78522 (0.0007) [2023-10-14 20:54:51,877][61585] Updated weights for policy 1, policy_version 78370 (0.0009) [2023-10-14 20:54:52,235][61585] Updated weights for policy 1, policy_version 78380 (0.0007) [2023-10-14 20:54:52,592][61585] Updated weights for policy 1, policy_version 78390 (0.0007) [2023-10-14 20:54:52,969][61585] Updated weights for policy 1, policy_version 78400 (0.0009) [2023-10-14 20:54:53,333][61552] Updated weights for policy 0, policy_version 78532 (0.0010) [2023-10-14 20:54:53,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 160694272. Throughput: 0: 1672.4, 1: 1679.0. Samples: 40181200. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-14 20:54:53,344][60425] Avg episode reward: [(0, '79.240'), (1, '82.010')] [2023-10-14 20:54:53,701][61552] Updated weights for policy 0, policy_version 78542 (0.0008) [2023-10-14 20:54:54,078][61552] Updated weights for policy 0, policy_version 78552 (0.0009) [2023-10-14 20:54:57,144][61585] Updated weights for policy 1, policy_version 78410 (0.0007) [2023-10-14 20:54:57,519][61585] Updated weights for policy 1, policy_version 78420 (0.0007) [2023-10-14 20:54:57,886][61585] Updated weights for policy 1, policy_version 78430 (0.0008) [2023-10-14 20:54:58,039][61552] Updated weights for policy 0, policy_version 78562 (0.0008) [2023-10-14 20:54:58,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 160759808. Throughput: 0: 1680.3, 1: 1656.6. Samples: 40200714. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-14 20:54:58,344][60425] Avg episode reward: [(0, '82.250'), (1, '80.630')] [2023-10-14 20:54:58,404][61552] Updated weights for policy 0, policy_version 78572 (0.0008) [2023-10-14 20:54:58,768][61552] Updated weights for policy 0, policy_version 78582 (0.0007) [2023-10-14 20:54:59,138][61552] Updated weights for policy 0, policy_version 78592 (0.0009) [2023-10-14 20:55:02,114][61585] Updated weights for policy 1, policy_version 78440 (0.0009) [2023-10-14 20:55:02,481][61585] Updated weights for policy 1, policy_version 78450 (0.0009) [2023-10-14 20:55:02,845][61585] Updated weights for policy 1, policy_version 78460 (0.0008) [2023-10-14 20:55:03,213][61552] Updated weights for policy 0, policy_version 78602 (0.0008) [2023-10-14 20:55:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 160825344. Throughput: 0: 1677.6, 1: 1672.8. Samples: 40210702. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-14 20:55:03,344][60425] Avg episode reward: [(0, '80.930'), (1, '84.200')] [2023-10-14 20:55:03,593][61552] Updated weights for policy 0, policy_version 78612 (0.0007) [2023-10-14 20:55:03,957][61552] Updated weights for policy 0, policy_version 78622 (0.0009) [2023-10-14 20:55:06,888][61585] Updated weights for policy 1, policy_version 78470 (0.0008) [2023-10-14 20:55:07,253][61585] Updated weights for policy 1, policy_version 78480 (0.0011) [2023-10-14 20:55:07,623][61585] Updated weights for policy 1, policy_version 78490 (0.0010) [2023-10-14 20:55:07,910][61552] Updated weights for policy 0, policy_version 78632 (0.0010) [2023-10-14 20:55:08,282][61552] Updated weights for policy 0, policy_version 78642 (0.0008) [2023-10-14 20:55:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 160890880. Throughput: 0: 1678.4, 1: 1667.0. Samples: 40230986. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-14 20:55:08,344][60425] Avg episode reward: [(0, '76.620'), (1, '83.770')] [2023-10-14 20:55:08,642][61552] Updated weights for policy 0, policy_version 78652 (0.0009) [2023-10-14 20:55:11,853][61585] Updated weights for policy 1, policy_version 78500 (0.0008) [2023-10-14 20:55:12,219][61585] Updated weights for policy 1, policy_version 78510 (0.0009) [2023-10-14 20:55:12,580][61585] Updated weights for policy 1, policy_version 78520 (0.0010) [2023-10-14 20:55:12,730][61552] Updated weights for policy 0, policy_version 78662 (0.0009) [2023-10-14 20:55:13,100][61552] Updated weights for policy 0, policy_version 78672 (0.0007) [2023-10-14 20:55:13,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 160956416. Throughput: 0: 1676.3, 1: 1652.8. Samples: 40250544. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-14 20:55:13,344][60425] Avg episode reward: [(0, '74.820'), (1, '82.490')] [2023-10-14 20:55:13,469][61552] Updated weights for policy 0, policy_version 78682 (0.0007) [2023-10-14 20:55:16,563][61585] Updated weights for policy 1, policy_version 78530 (0.0008) [2023-10-14 20:55:16,931][61585] Updated weights for policy 1, policy_version 78540 (0.0007) [2023-10-14 20:55:17,297][61585] Updated weights for policy 1, policy_version 78550 (0.0008) [2023-10-14 20:55:17,546][61552] Updated weights for policy 0, policy_version 78692 (0.0008) [2023-10-14 20:55:17,664][61585] Updated weights for policy 1, policy_version 78560 (0.0008) [2023-10-14 20:55:17,920][61552] Updated weights for policy 0, policy_version 78702 (0.0007) [2023-10-14 20:55:18,295][61552] Updated weights for policy 0, policy_version 78712 (0.0009) [2023-10-14 20:55:18,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 161021952. Throughput: 0: 1677.4, 1: 1666.1. Samples: 40260866. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-14 20:55:18,344][60425] Avg episode reward: [(0, '75.390'), (1, '80.010')] [2023-10-14 20:55:21,647][61585] Updated weights for policy 1, policy_version 78570 (0.0007) [2023-10-14 20:55:22,006][61585] Updated weights for policy 1, policy_version 78580 (0.0007) [2023-10-14 20:55:22,364][61585] Updated weights for policy 1, policy_version 78590 (0.0007) [2023-10-14 20:55:22,468][61552] Updated weights for policy 0, policy_version 78722 (0.0008) [2023-10-14 20:55:22,833][61552] Updated weights for policy 0, policy_version 78732 (0.0012) [2023-10-14 20:55:23,206][61552] Updated weights for policy 0, policy_version 78742 (0.0010) [2023-10-14 20:55:23,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 161087488. Throughput: 0: 1679.4, 1: 1654.4. Samples: 40281236. Policy #0 lag: (min: 19.0, avg: 26.7, max: 51.0) [2023-10-14 20:55:23,344][60425] Avg episode reward: [(0, '73.840'), (1, '82.910')] [2023-10-14 20:55:23,580][61552] Updated weights for policy 0, policy_version 78752 (0.0009) [2023-10-14 20:55:26,483][61585] Updated weights for policy 1, policy_version 78600 (0.0008) [2023-10-14 20:55:26,844][61585] Updated weights for policy 1, policy_version 78610 (0.0008) [2023-10-14 20:55:27,198][61585] Updated weights for policy 1, policy_version 78620 (0.0009) [2023-10-14 20:55:27,714][61552] Updated weights for policy 0, policy_version 78762 (0.0010) [2023-10-14 20:55:28,079][61552] Updated weights for policy 0, policy_version 78772 (0.0007) [2023-10-14 20:55:28,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 161153024. Throughput: 0: 1668.2, 1: 1654.3. Samples: 40300436. Policy #0 lag: (min: 19.0, avg: 26.7, max: 51.0) [2023-10-14 20:55:28,344][60425] Avg episode reward: [(0, '74.560'), (1, '81.880')] [2023-10-14 20:55:28,434][61552] Updated weights for policy 0, policy_version 78782 (0.0007) [2023-10-14 20:55:31,389][61585] Updated weights for policy 1, policy_version 78630 (0.0009) [2023-10-14 20:55:31,740][61585] Updated weights for policy 1, policy_version 78640 (0.0007) [2023-10-14 20:55:32,110][61585] Updated weights for policy 1, policy_version 78650 (0.0007) [2023-10-14 20:55:32,426][61552] Updated weights for policy 0, policy_version 78792 (0.0009) [2023-10-14 20:55:32,799][61552] Updated weights for policy 0, policy_version 78802 (0.0009) [2023-10-14 20:55:33,169][61552] Updated weights for policy 0, policy_version 78812 (0.0011) [2023-10-14 20:55:33,343][60425] Fps is (10 sec: 16383.9, 60 sec: 13653.4, 300 sec: 13551.5). Total num frames: 161251328. Throughput: 0: 1677.6, 1: 1660.4. Samples: 40311136. Policy #0 lag: (min: 19.0, avg: 26.7, max: 51.0) [2023-10-14 20:55:33,344][60425] Avg episode reward: [(0, '75.650'), (1, '76.820')] [2023-10-14 20:55:36,242][61585] Updated weights for policy 1, policy_version 78660 (0.0009) [2023-10-14 20:55:36,612][61585] Updated weights for policy 1, policy_version 78670 (0.0008) [2023-10-14 20:55:36,979][61585] Updated weights for policy 1, policy_version 78680 (0.0007) [2023-10-14 20:55:37,306][61552] Updated weights for policy 0, policy_version 78822 (0.0009) [2023-10-14 20:55:37,699][61552] Updated weights for policy 0, policy_version 78832 (0.0008) [2023-10-14 20:55:38,064][61552] Updated weights for policy 0, policy_version 78842 (0.0008) [2023-10-14 20:55:38,343][60425] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13551.5). Total num frames: 161316864. Throughput: 0: 1680.0, 1: 1650.9. Samples: 40331090. Policy #0 lag: (min: 19.0, avg: 26.7, max: 51.0) [2023-10-14 20:55:38,344][60425] Avg episode reward: [(0, '72.490'), (1, '80.730')] [2023-10-14 20:55:41,085][61585] Updated weights for policy 1, policy_version 78690 (0.0008) [2023-10-14 20:55:41,448][61585] Updated weights for policy 1, policy_version 78700 (0.0009) [2023-10-14 20:55:41,811][61585] Updated weights for policy 1, policy_version 78710 (0.0008) [2023-10-14 20:55:42,179][61585] Updated weights for policy 1, policy_version 78720 (0.0008) [2023-10-14 20:55:42,280][61552] Updated weights for policy 0, policy_version 78852 (0.0009) [2023-10-14 20:55:42,659][61552] Updated weights for policy 0, policy_version 78862 (0.0008) [2023-10-14 20:55:43,036][61552] Updated weights for policy 0, policy_version 78872 (0.0009) [2023-10-14 20:55:43,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 161382400. Throughput: 0: 1653.8, 1: 1670.7. Samples: 40350318. Policy #0 lag: (min: 19.0, avg: 26.7, max: 51.0) [2023-10-14 20:55:43,345][60425] Avg episode reward: [(0, '68.880'), (1, '82.260')] [2023-10-14 20:55:43,354][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000078880_80773120.pth... [2023-10-14 20:55:43,354][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000078720_80609280.pth... [2023-10-14 20:55:43,388][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000077312_79167488.pth [2023-10-14 20:55:43,395][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000077152_79003648.pth [2023-10-14 20:55:46,369][61585] Updated weights for policy 1, policy_version 78730 (0.0009) [2023-10-14 20:55:46,734][61585] Updated weights for policy 1, policy_version 78740 (0.0008) [2023-10-14 20:55:47,097][61585] Updated weights for policy 1, policy_version 78750 (0.0008) [2023-10-14 20:55:47,193][61552] Updated weights for policy 0, policy_version 78882 (0.0010) [2023-10-14 20:55:47,560][61552] Updated weights for policy 0, policy_version 78892 (0.0009) [2023-10-14 20:55:47,940][61552] Updated weights for policy 0, policy_version 78902 (0.0007) [2023-10-14 20:55:48,310][61552] Updated weights for policy 0, policy_version 78912 (0.0008) [2023-10-14 20:55:48,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 161447936. Throughput: 0: 1665.5, 1: 1678.5. Samples: 40361182. Policy #0 lag: (min: 19.0, avg: 26.7, max: 51.0) [2023-10-14 20:55:48,344][60425] Avg episode reward: [(0, '75.960'), (1, '79.230')] [2023-10-14 20:55:51,206][61585] Updated weights for policy 1, policy_version 78760 (0.0009) [2023-10-14 20:55:51,576][61585] Updated weights for policy 1, policy_version 78770 (0.0007) [2023-10-14 20:55:51,949][61585] Updated weights for policy 1, policy_version 78780 (0.0007) [2023-10-14 20:55:52,362][61552] Updated weights for policy 0, policy_version 78922 (0.0007) [2023-10-14 20:55:52,730][61552] Updated weights for policy 0, policy_version 78932 (0.0008) [2023-10-14 20:55:53,100][61552] Updated weights for policy 0, policy_version 78942 (0.0009) [2023-10-14 20:55:53,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 161513472. Throughput: 0: 1669.1, 1: 1658.7. Samples: 40380738. Policy #0 lag: (min: 19.0, avg: 26.7, max: 51.0) [2023-10-14 20:55:53,344][60425] Avg episode reward: [(0, '75.420'), (1, '76.250')] [2023-10-14 20:55:55,962][61585] Updated weights for policy 1, policy_version 78790 (0.0008) [2023-10-14 20:55:56,320][61585] Updated weights for policy 1, policy_version 78800 (0.0007) [2023-10-14 20:55:56,691][61585] Updated weights for policy 1, policy_version 78810 (0.0008) [2023-10-14 20:55:57,242][61552] Updated weights for policy 0, policy_version 78952 (0.0010) [2023-10-14 20:55:57,604][61552] Updated weights for policy 0, policy_version 78962 (0.0009) [2023-10-14 20:55:57,970][61552] Updated weights for policy 0, policy_version 78972 (0.0008) [2023-10-14 20:55:58,343][60425] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 161579008. Throughput: 0: 1651.3, 1: 1672.5. Samples: 40400116. Policy #0 lag: (min: 19.0, avg: 26.7, max: 51.0) [2023-10-14 20:55:58,345][60425] Avg episode reward: [(0, '78.280'), (1, '82.840')] [2023-10-14 20:56:00,624][61585] Updated weights for policy 1, policy_version 78820 (0.0007) [2023-10-14 20:56:00,983][61585] Updated weights for policy 1, policy_version 78830 (0.0009) [2023-10-14 20:56:01,350][61585] Updated weights for policy 1, policy_version 78840 (0.0008) [2023-10-14 20:56:01,981][61552] Updated weights for policy 0, policy_version 78982 (0.0008) [2023-10-14 20:56:02,345][61552] Updated weights for policy 0, policy_version 78992 (0.0009) [2023-10-14 20:56:02,717][61552] Updated weights for policy 0, policy_version 79002 (0.0008) [2023-10-14 20:56:03,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 161644544. Throughput: 0: 1668.5, 1: 1669.7. Samples: 40411084. Policy #0 lag: (min: 19.0, avg: 26.7, max: 51.0) [2023-10-14 20:56:03,344][60425] Avg episode reward: [(0, '70.750'), (1, '79.580')] [2023-10-14 20:56:05,379][61585] Updated weights for policy 1, policy_version 78850 (0.0008) [2023-10-14 20:56:05,745][61585] Updated weights for policy 1, policy_version 78860 (0.0009) [2023-10-14 20:56:06,114][61585] Updated weights for policy 1, policy_version 78870 (0.0008) [2023-10-14 20:56:06,487][61585] Updated weights for policy 1, policy_version 78880 (0.0009) [2023-10-14 20:56:06,826][61552] Updated weights for policy 0, policy_version 79012 (0.0009) [2023-10-14 20:56:07,202][61552] Updated weights for policy 0, policy_version 79022 (0.0010) [2023-10-14 20:56:07,566][61552] Updated weights for policy 0, policy_version 79032 (0.0009) [2023-10-14 20:56:08,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 161710080. Throughput: 0: 1663.9, 1: 1662.0. Samples: 40430902. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:56:08,344][60425] Avg episode reward: [(0, '75.400'), (1, '82.100')] [2023-10-14 20:56:10,561][61585] Updated weights for policy 1, policy_version 78890 (0.0011) [2023-10-14 20:56:10,922][61585] Updated weights for policy 1, policy_version 78900 (0.0009) [2023-10-14 20:56:11,283][61585] Updated weights for policy 1, policy_version 78910 (0.0010) [2023-10-14 20:56:11,638][61552] Updated weights for policy 0, policy_version 79042 (0.0010) [2023-10-14 20:56:12,002][61552] Updated weights for policy 0, policy_version 79052 (0.0011) [2023-10-14 20:56:12,375][61552] Updated weights for policy 0, policy_version 79062 (0.0010) [2023-10-14 20:56:12,730][61552] Updated weights for policy 0, policy_version 79072 (0.0010) [2023-10-14 20:56:13,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 161775616. Throughput: 0: 1650.7, 1: 1686.0. Samples: 40450590. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:56:13,344][60425] Avg episode reward: [(0, '75.780'), (1, '81.690')] [2023-10-14 20:56:15,235][61585] Updated weights for policy 1, policy_version 78920 (0.0010) [2023-10-14 20:56:15,606][61585] Updated weights for policy 1, policy_version 78930 (0.0007) [2023-10-14 20:56:15,978][61585] Updated weights for policy 1, policy_version 78940 (0.0008) [2023-10-14 20:56:16,809][61552] Updated weights for policy 0, policy_version 79082 (0.0010) [2023-10-14 20:56:17,190][61552] Updated weights for policy 0, policy_version 79092 (0.0007) [2023-10-14 20:56:17,553][61552] Updated weights for policy 0, policy_version 79102 (0.0008) [2023-10-14 20:56:18,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 161841152. Throughput: 0: 1668.8, 1: 1669.2. Samples: 40461348. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:56:18,344][60425] Avg episode reward: [(0, '75.050'), (1, '80.090')] [2023-10-14 20:56:20,123][61585] Updated weights for policy 1, policy_version 78950 (0.0010) [2023-10-14 20:56:20,495][61585] Updated weights for policy 1, policy_version 78960 (0.0011) [2023-10-14 20:56:20,850][61585] Updated weights for policy 1, policy_version 78970 (0.0010) [2023-10-14 20:56:21,666][61552] Updated weights for policy 0, policy_version 79112 (0.0007) [2023-10-14 20:56:22,040][61552] Updated weights for policy 0, policy_version 79122 (0.0008) [2023-10-14 20:56:22,407][61552] Updated weights for policy 0, policy_version 79132 (0.0010) [2023-10-14 20:56:23,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 161906688. Throughput: 0: 1664.7, 1: 1669.8. Samples: 40481144. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:56:23,344][60425] Avg episode reward: [(0, '77.860'), (1, '80.600')] [2023-10-14 20:56:24,921][61585] Updated weights for policy 1, policy_version 78980 (0.0010) [2023-10-14 20:56:25,289][61585] Updated weights for policy 1, policy_version 78990 (0.0008) [2023-10-14 20:56:25,657][61585] Updated weights for policy 1, policy_version 79000 (0.0007) [2023-10-14 20:56:26,385][61552] Updated weights for policy 0, policy_version 79142 (0.0009) [2023-10-14 20:56:26,752][61552] Updated weights for policy 0, policy_version 79152 (0.0010) [2023-10-14 20:56:27,120][61552] Updated weights for policy 0, policy_version 79162 (0.0009) [2023-10-14 20:56:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 161972224. Throughput: 0: 1663.7, 1: 1680.1. Samples: 40500788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:56:28,344][60425] Avg episode reward: [(0, '75.800'), (1, '78.210')] [2023-10-14 20:56:29,808][61585] Updated weights for policy 1, policy_version 79010 (0.0009) [2023-10-14 20:56:30,171][61585] Updated weights for policy 1, policy_version 79020 (0.0009) [2023-10-14 20:56:30,548][61585] Updated weights for policy 1, policy_version 79030 (0.0009) [2023-10-14 20:56:30,913][61585] Updated weights for policy 1, policy_version 79040 (0.0008) [2023-10-14 20:56:31,015][61552] Updated weights for policy 0, policy_version 79172 (0.0009) [2023-10-14 20:56:31,387][61552] Updated weights for policy 0, policy_version 79182 (0.0010) [2023-10-14 20:56:31,756][61552] Updated weights for policy 0, policy_version 79192 (0.0009) [2023-10-14 20:56:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 162037760. Throughput: 0: 1684.4, 1: 1655.2. Samples: 40511466. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:56:33,344][60425] Avg episode reward: [(0, '80.850'), (1, '80.080')] [2023-10-14 20:56:35,205][61585] Updated weights for policy 1, policy_version 79050 (0.0008) [2023-10-14 20:56:35,579][61585] Updated weights for policy 1, policy_version 79060 (0.0011) [2023-10-14 20:56:35,862][61552] Updated weights for policy 0, policy_version 79202 (0.0008) [2023-10-14 20:56:35,939][61585] Updated weights for policy 1, policy_version 79070 (0.0011) [2023-10-14 20:56:36,233][61552] Updated weights for policy 0, policy_version 79212 (0.0008) [2023-10-14 20:56:36,611][61552] Updated weights for policy 0, policy_version 79222 (0.0009) [2023-10-14 20:56:36,970][61552] Updated weights for policy 0, policy_version 79232 (0.0009) [2023-10-14 20:56:38,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 162103296. Throughput: 0: 1662.4, 1: 1671.9. Samples: 40530778. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:56:38,344][60425] Avg episode reward: [(0, '76.390'), (1, '75.780')] [2023-10-14 20:56:40,113][61585] Updated weights for policy 1, policy_version 79080 (0.0007) [2023-10-14 20:56:40,475][61585] Updated weights for policy 1, policy_version 79090 (0.0008) [2023-10-14 20:56:40,836][61585] Updated weights for policy 1, policy_version 79100 (0.0007) [2023-10-14 20:56:41,169][61552] Updated weights for policy 0, policy_version 79242 (0.0009) [2023-10-14 20:56:41,543][61552] Updated weights for policy 0, policy_version 79252 (0.0011) [2023-10-14 20:56:41,902][61552] Updated weights for policy 0, policy_version 79262 (0.0008) [2023-10-14 20:56:43,344][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 162168832. Throughput: 0: 1673.7, 1: 1680.8. Samples: 40551072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:56:43,345][60425] Avg episode reward: [(0, '79.640'), (1, '78.970')] [2023-10-14 20:56:44,850][61585] Updated weights for policy 1, policy_version 79110 (0.0009) [2023-10-14 20:56:45,204][61585] Updated weights for policy 1, policy_version 79120 (0.0009) [2023-10-14 20:56:45,574][61585] Updated weights for policy 1, policy_version 79130 (0.0009) [2023-10-14 20:56:46,100][61552] Updated weights for policy 0, policy_version 79272 (0.0010) [2023-10-14 20:56:46,473][61552] Updated weights for policy 0, policy_version 79282 (0.0010) [2023-10-14 20:56:46,851][61552] Updated weights for policy 0, policy_version 79292 (0.0009) [2023-10-14 20:56:48,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 162234368. Throughput: 0: 1682.1, 1: 1662.2. Samples: 40561580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:56:48,344][60425] Avg episode reward: [(0, '75.680'), (1, '77.450')] [2023-10-14 20:56:49,802][61585] Updated weights for policy 1, policy_version 79140 (0.0010) [2023-10-14 20:56:50,165][61585] Updated weights for policy 1, policy_version 79150 (0.0007) [2023-10-14 20:56:50,525][61585] Updated weights for policy 1, policy_version 79160 (0.0008) [2023-10-14 20:56:50,829][61552] Updated weights for policy 0, policy_version 79302 (0.0008) [2023-10-14 20:56:51,188][61552] Updated weights for policy 0, policy_version 79312 (0.0007) [2023-10-14 20:56:51,560][61552] Updated weights for policy 0, policy_version 79322 (0.0009) [2023-10-14 20:56:53,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 162299904. Throughput: 0: 1658.7, 1: 1672.8. Samples: 40580818. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:56:53,344][60425] Avg episode reward: [(0, '79.280'), (1, '80.760')] [2023-10-14 20:56:54,484][61585] Updated weights for policy 1, policy_version 79170 (0.0008) [2023-10-14 20:56:54,851][61585] Updated weights for policy 1, policy_version 79180 (0.0009) [2023-10-14 20:56:55,218][61585] Updated weights for policy 1, policy_version 79190 (0.0007) [2023-10-14 20:56:55,588][61585] Updated weights for policy 1, policy_version 79200 (0.0007) [2023-10-14 20:56:55,590][61552] Updated weights for policy 0, policy_version 79332 (0.0008) [2023-10-14 20:56:55,961][61552] Updated weights for policy 0, policy_version 79342 (0.0008) [2023-10-14 20:56:56,325][61552] Updated weights for policy 0, policy_version 79352 (0.0009) [2023-10-14 20:56:58,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 162365440. Throughput: 0: 1679.7, 1: 1673.8. Samples: 40601496. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:56:58,345][60425] Avg episode reward: [(0, '77.180'), (1, '78.890')] [2023-10-14 20:56:59,589][61585] Updated weights for policy 1, policy_version 79210 (0.0010) [2023-10-14 20:56:59,954][61585] Updated weights for policy 1, policy_version 79220 (0.0009) [2023-10-14 20:57:00,325][61585] Updated weights for policy 1, policy_version 79230 (0.0008) [2023-10-14 20:57:00,541][61552] Updated weights for policy 0, policy_version 79362 (0.0009) [2023-10-14 20:57:00,895][61552] Updated weights for policy 0, policy_version 79372 (0.0008) [2023-10-14 20:57:01,266][61552] Updated weights for policy 0, policy_version 79382 (0.0010) [2023-10-14 20:57:01,635][61552] Updated weights for policy 0, policy_version 79392 (0.0009) [2023-10-14 20:57:03,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 162430976. Throughput: 0: 1676.0, 1: 1661.2. Samples: 40611524. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-14 20:57:03,344][60425] Avg episode reward: [(0, '78.420'), (1, '76.520')] [2023-10-14 20:57:04,431][61585] Updated weights for policy 1, policy_version 79240 (0.0008) [2023-10-14 20:57:04,794][61585] Updated weights for policy 1, policy_version 79250 (0.0009) [2023-10-14 20:57:05,163][61585] Updated weights for policy 1, policy_version 79260 (0.0008) [2023-10-14 20:57:05,798][61552] Updated weights for policy 0, policy_version 79402 (0.0010) [2023-10-14 20:57:06,165][61552] Updated weights for policy 0, policy_version 79412 (0.0009) [2023-10-14 20:57:06,525][61552] Updated weights for policy 0, policy_version 79422 (0.0009) [2023-10-14 20:57:08,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 162496512. Throughput: 0: 1656.8, 1: 1675.0. Samples: 40631076. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-14 20:57:08,344][60425] Avg episode reward: [(0, '78.830'), (1, '78.320')] [2023-10-14 20:57:09,336][61585] Updated weights for policy 1, policy_version 79270 (0.0008) [2023-10-14 20:57:09,700][61585] Updated weights for policy 1, policy_version 79280 (0.0010) [2023-10-14 20:57:10,074][61585] Updated weights for policy 1, policy_version 79290 (0.0009) [2023-10-14 20:57:10,647][61552] Updated weights for policy 0, policy_version 79432 (0.0008) [2023-10-14 20:57:11,025][61552] Updated weights for policy 0, policy_version 79442 (0.0008) [2023-10-14 20:57:11,390][61552] Updated weights for policy 0, policy_version 79452 (0.0008) [2023-10-14 20:57:13,343][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 162562048. Throughput: 0: 1678.7, 1: 1672.6. Samples: 40651600. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-14 20:57:13,344][60425] Avg episode reward: [(0, '77.820'), (1, '82.250')] [2023-10-14 20:57:14,279][61585] Updated weights for policy 1, policy_version 79300 (0.0010) [2023-10-14 20:57:14,649][61585] Updated weights for policy 1, policy_version 79310 (0.0009) [2023-10-14 20:57:15,009][61585] Updated weights for policy 1, policy_version 79320 (0.0009) [2023-10-14 20:57:15,474][61552] Updated weights for policy 0, policy_version 79462 (0.0009) [2023-10-14 20:57:15,844][61552] Updated weights for policy 0, policy_version 79472 (0.0007) [2023-10-14 20:57:16,205][61552] Updated weights for policy 0, policy_version 79482 (0.0007) [2023-10-14 20:57:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 162627584. Throughput: 0: 1664.3, 1: 1664.6. Samples: 40661264. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-14 20:57:18,344][60425] Avg episode reward: [(0, '78.640'), (1, '82.510')] [2023-10-14 20:57:19,300][61585] Updated weights for policy 1, policy_version 79330 (0.0010) [2023-10-14 20:57:19,663][61585] Updated weights for policy 1, policy_version 79340 (0.0008) [2023-10-14 20:57:20,042][61585] Updated weights for policy 1, policy_version 79350 (0.0009) [2023-10-14 20:57:20,321][61552] Updated weights for policy 0, policy_version 79492 (0.0008) [2023-10-14 20:57:20,400][61585] Updated weights for policy 1, policy_version 79360 (0.0008) [2023-10-14 20:57:20,690][61552] Updated weights for policy 0, policy_version 79502 (0.0009) [2023-10-14 20:57:21,059][61552] Updated weights for policy 0, policy_version 79512 (0.0010) [2023-10-14 20:57:23,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 162693120. Throughput: 0: 1668.0, 1: 1671.2. Samples: 40681044. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-14 20:57:23,344][60425] Avg episode reward: [(0, '77.850'), (1, '82.000')] [2023-10-14 20:57:24,506][61585] Updated weights for policy 1, policy_version 79370 (0.0007) [2023-10-14 20:57:24,871][61585] Updated weights for policy 1, policy_version 79380 (0.0008) [2023-10-14 20:57:25,235][61552] Updated weights for policy 0, policy_version 79522 (0.0011) [2023-10-14 20:57:25,237][61585] Updated weights for policy 1, policy_version 79390 (0.0010) [2023-10-14 20:57:25,608][61552] Updated weights for policy 0, policy_version 79532 (0.0010) [2023-10-14 20:57:25,974][61552] Updated weights for policy 0, policy_version 79542 (0.0008) [2023-10-14 20:57:26,347][61552] Updated weights for policy 0, policy_version 79552 (0.0007) [2023-10-14 20:57:28,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 162758656. Throughput: 0: 1672.2, 1: 1669.1. Samples: 40701430. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-14 20:57:28,344][60425] Avg episode reward: [(0, '78.610'), (1, '79.480')] [2023-10-14 20:57:29,354][61585] Updated weights for policy 1, policy_version 79400 (0.0009) [2023-10-14 20:57:29,718][61585] Updated weights for policy 1, policy_version 79410 (0.0008) [2023-10-14 20:57:30,073][61585] Updated weights for policy 1, policy_version 79420 (0.0007) [2023-10-14 20:57:30,323][61552] Updated weights for policy 0, policy_version 79562 (0.0010) [2023-10-14 20:57:30,693][61552] Updated weights for policy 0, policy_version 79572 (0.0010) [2023-10-14 20:57:31,062][61552] Updated weights for policy 0, policy_version 79582 (0.0010) [2023-10-14 20:57:33,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 162824192. Throughput: 0: 1657.3, 1: 1666.5. Samples: 40711152. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-14 20:57:33,344][60425] Avg episode reward: [(0, '79.300'), (1, '80.650')] [2023-10-14 20:57:34,205][61585] Updated weights for policy 1, policy_version 79430 (0.0007) [2023-10-14 20:57:34,587][61585] Updated weights for policy 1, policy_version 79440 (0.0010) [2023-10-14 20:57:34,941][61585] Updated weights for policy 1, policy_version 79450 (0.0008) [2023-10-14 20:57:35,225][61552] Updated weights for policy 0, policy_version 79592 (0.0007) [2023-10-14 20:57:35,590][61552] Updated weights for policy 0, policy_version 79602 (0.0007) [2023-10-14 20:57:35,955][61552] Updated weights for policy 0, policy_version 79612 (0.0008) [2023-10-14 20:57:38,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 162889728. Throughput: 0: 1665.3, 1: 1672.2. Samples: 40731006. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-14 20:57:38,345][60425] Avg episode reward: [(0, '76.490'), (1, '82.510')] [2023-10-14 20:57:39,023][61585] Updated weights for policy 1, policy_version 79460 (0.0008) [2023-10-14 20:57:39,380][61585] Updated weights for policy 1, policy_version 79470 (0.0009) [2023-10-14 20:57:39,754][61585] Updated weights for policy 1, policy_version 79480 (0.0010) [2023-10-14 20:57:40,262][61552] Updated weights for policy 0, policy_version 79622 (0.0009) [2023-10-14 20:57:40,631][61552] Updated weights for policy 0, policy_version 79632 (0.0011) [2023-10-14 20:57:40,998][61552] Updated weights for policy 0, policy_version 79642 (0.0011) [2023-10-14 20:57:43,344][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 162955264. Throughput: 0: 1665.6, 1: 1667.8. Samples: 40751496. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-14 20:57:43,345][60425] Avg episode reward: [(0, '76.190'), (1, '78.680')] [2023-10-14 20:57:43,356][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000079648_81559552.pth... [2023-10-14 20:57:43,356][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000079488_81395712.pth... [2023-10-14 20:57:43,393][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000077952_79822848.pth [2023-10-14 20:57:43,397][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000078080_79953920.pth [2023-10-14 20:57:43,707][61585] Updated weights for policy 1, policy_version 79490 (0.0010) [2023-10-14 20:57:44,071][61585] Updated weights for policy 1, policy_version 79500 (0.0009) [2023-10-14 20:57:44,438][61585] Updated weights for policy 1, policy_version 79510 (0.0008) [2023-10-14 20:57:44,793][61585] Updated weights for policy 1, policy_version 79520 (0.0008) [2023-10-14 20:57:45,210][61552] Updated weights for policy 0, policy_version 79652 (0.0008) [2023-10-14 20:57:45,585][61552] Updated weights for policy 0, policy_version 79662 (0.0008) [2023-10-14 20:57:45,952][61552] Updated weights for policy 0, policy_version 79672 (0.0008) [2023-10-14 20:57:48,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 163020800. Throughput: 0: 1659.5, 1: 1671.1. Samples: 40761400. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-14 20:57:48,344][60425] Avg episode reward: [(0, '74.010'), (1, '80.050')] [2023-10-14 20:57:48,761][61585] Updated weights for policy 1, policy_version 79530 (0.0008) [2023-10-14 20:57:49,128][61585] Updated weights for policy 1, policy_version 79540 (0.0008) [2023-10-14 20:57:49,501][61585] Updated weights for policy 1, policy_version 79550 (0.0009) [2023-10-14 20:57:49,982][61552] Updated weights for policy 0, policy_version 79682 (0.0009) [2023-10-14 20:57:50,352][61552] Updated weights for policy 0, policy_version 79692 (0.0010) [2023-10-14 20:57:50,726][61552] Updated weights for policy 0, policy_version 79702 (0.0010) [2023-10-14 20:57:51,107][61552] Updated weights for policy 0, policy_version 79712 (0.0011) [2023-10-14 20:57:53,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 163086336. Throughput: 0: 1666.8, 1: 1667.9. Samples: 40781136. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:57:53,344][60425] Avg episode reward: [(0, '77.910'), (1, '81.940')] [2023-10-14 20:57:53,602][61585] Updated weights for policy 1, policy_version 79560 (0.0008) [2023-10-14 20:57:53,963][61585] Updated weights for policy 1, policy_version 79570 (0.0007) [2023-10-14 20:57:54,344][61585] Updated weights for policy 1, policy_version 79580 (0.0009) [2023-10-14 20:57:55,123][61552] Updated weights for policy 0, policy_version 79722 (0.0009) [2023-10-14 20:57:55,497][61552] Updated weights for policy 0, policy_version 79732 (0.0007) [2023-10-14 20:57:55,867][61552] Updated weights for policy 0, policy_version 79742 (0.0008) [2023-10-14 20:57:58,309][61585] Updated weights for policy 1, policy_version 79590 (0.0009) [2023-10-14 20:57:58,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 163151872. Throughput: 0: 1665.5, 1: 1668.0. Samples: 40801608. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:57:58,344][60425] Avg episode reward: [(0, '73.930'), (1, '81.680')] [2023-10-14 20:57:58,672][61585] Updated weights for policy 1, policy_version 79600 (0.0009) [2023-10-14 20:57:59,043][61585] Updated weights for policy 1, policy_version 79610 (0.0008) [2023-10-14 20:58:00,069][61552] Updated weights for policy 0, policy_version 79752 (0.0009) [2023-10-14 20:58:00,446][61552] Updated weights for policy 0, policy_version 79762 (0.0008) [2023-10-14 20:58:00,819][61552] Updated weights for policy 0, policy_version 79772 (0.0010) [2023-10-14 20:58:03,118][61585] Updated weights for policy 1, policy_version 79620 (0.0008) [2023-10-14 20:58:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 163217408. Throughput: 0: 1655.9, 1: 1670.5. Samples: 40810952. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:58:03,344][60425] Avg episode reward: [(0, '72.810'), (1, '81.080')] [2023-10-14 20:58:03,481][61585] Updated weights for policy 1, policy_version 79630 (0.0008) [2023-10-14 20:58:03,845][61585] Updated weights for policy 1, policy_version 79640 (0.0007) [2023-10-14 20:58:04,634][61552] Updated weights for policy 0, policy_version 79782 (0.0008) [2023-10-14 20:58:04,996][61552] Updated weights for policy 0, policy_version 79792 (0.0008) [2023-10-14 20:58:05,362][61552] Updated weights for policy 0, policy_version 79802 (0.0007) [2023-10-14 20:58:07,888][61585] Updated weights for policy 1, policy_version 79650 (0.0010) [2023-10-14 20:58:08,245][61585] Updated weights for policy 1, policy_version 79660 (0.0011) [2023-10-14 20:58:08,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 163282944. Throughput: 0: 1674.2, 1: 1674.2. Samples: 40831724. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:58:08,344][60425] Avg episode reward: [(0, '77.100'), (1, '75.100')] [2023-10-14 20:58:08,613][61585] Updated weights for policy 1, policy_version 79670 (0.0007) [2023-10-14 20:58:08,982][61585] Updated weights for policy 1, policy_version 79680 (0.0009) [2023-10-14 20:58:09,353][61552] Updated weights for policy 0, policy_version 79812 (0.0009) [2023-10-14 20:58:09,720][61552] Updated weights for policy 0, policy_version 79822 (0.0007) [2023-10-14 20:58:10,089][61552] Updated weights for policy 0, policy_version 79832 (0.0009) [2023-10-14 20:58:13,164][61585] Updated weights for policy 1, policy_version 79690 (0.0007) [2023-10-14 20:58:13,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 163348480. Throughput: 0: 1684.9, 1: 1674.0. Samples: 40852584. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:58:13,344][60425] Avg episode reward: [(0, '69.380'), (1, '80.160')] [2023-10-14 20:58:13,539][61585] Updated weights for policy 1, policy_version 79700 (0.0011) [2023-10-14 20:58:13,860][61552] Updated weights for policy 0, policy_version 79842 (0.0010) [2023-10-14 20:58:13,909][61585] Updated weights for policy 1, policy_version 79710 (0.0009) [2023-10-14 20:58:14,224][61552] Updated weights for policy 0, policy_version 79852 (0.0010) [2023-10-14 20:58:14,596][61552] Updated weights for policy 0, policy_version 79862 (0.0009) [2023-10-14 20:58:14,963][61552] Updated weights for policy 0, policy_version 79872 (0.0009) [2023-10-14 20:58:18,077][61585] Updated weights for policy 1, policy_version 79720 (0.0010) [2023-10-14 20:58:18,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 163414016. Throughput: 0: 1671.9, 1: 1672.6. Samples: 40861652. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:58:18,344][60425] Avg episode reward: [(0, '73.560'), (1, '78.340')] [2023-10-14 20:58:18,433][61585] Updated weights for policy 1, policy_version 79730 (0.0007) [2023-10-14 20:58:18,795][61585] Updated weights for policy 1, policy_version 79740 (0.0007) [2023-10-14 20:58:19,177][61552] Updated weights for policy 0, policy_version 79882 (0.0007) [2023-10-14 20:58:19,542][61552] Updated weights for policy 0, policy_version 79892 (0.0007) [2023-10-14 20:58:19,903][61552] Updated weights for policy 0, policy_version 79902 (0.0009) [2023-10-14 20:58:23,054][61585] Updated weights for policy 1, policy_version 79750 (0.0009) [2023-10-14 20:58:23,344][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 163479552. Throughput: 0: 1688.6, 1: 1674.3. Samples: 40882336. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:58:23,345][60425] Avg episode reward: [(0, '72.990'), (1, '80.270')] [2023-10-14 20:58:23,430][61585] Updated weights for policy 1, policy_version 79760 (0.0007) [2023-10-14 20:58:23,800][61585] Updated weights for policy 1, policy_version 79770 (0.0007) [2023-10-14 20:58:24,016][61552] Updated weights for policy 0, policy_version 79912 (0.0008) [2023-10-14 20:58:24,378][61552] Updated weights for policy 0, policy_version 79922 (0.0008) [2023-10-14 20:58:24,747][61552] Updated weights for policy 0, policy_version 79932 (0.0009) [2023-10-14 20:58:27,663][61585] Updated weights for policy 1, policy_version 79780 (0.0008) [2023-10-14 20:58:28,023][61585] Updated weights for policy 1, policy_version 79790 (0.0009) [2023-10-14 20:58:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 163545088. Throughput: 0: 1688.4, 1: 1668.9. Samples: 40902572. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:58:28,344][60425] Avg episode reward: [(0, '74.170'), (1, '79.880')] [2023-10-14 20:58:28,383][61585] Updated weights for policy 1, policy_version 79800 (0.0009) [2023-10-14 20:58:28,911][61552] Updated weights for policy 0, policy_version 79942 (0.0009) [2023-10-14 20:58:29,287][61552] Updated weights for policy 0, policy_version 79952 (0.0009) [2023-10-14 20:58:29,649][61552] Updated weights for policy 0, policy_version 79962 (0.0009) [2023-10-14 20:58:32,425][61585] Updated weights for policy 1, policy_version 79810 (0.0010) [2023-10-14 20:58:32,789][61585] Updated weights for policy 1, policy_version 79820 (0.0007) [2023-10-14 20:58:33,152][61585] Updated weights for policy 1, policy_version 79830 (0.0008) [2023-10-14 20:58:33,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 163610624. Throughput: 0: 1672.8, 1: 1675.5. Samples: 40912070. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:58:33,344][60425] Avg episode reward: [(0, '72.130'), (1, '73.480')] [2023-10-14 20:58:33,504][61552] Updated weights for policy 0, policy_version 79972 (0.0009) [2023-10-14 20:58:33,512][61585] Updated weights for policy 1, policy_version 79840 (0.0007) [2023-10-14 20:58:33,872][61552] Updated weights for policy 0, policy_version 79982 (0.0009) [2023-10-14 20:58:34,244][61552] Updated weights for policy 0, policy_version 79992 (0.0009) [2023-10-14 20:58:37,778][61585] Updated weights for policy 1, policy_version 79850 (0.0009) [2023-10-14 20:58:38,151][61585] Updated weights for policy 1, policy_version 79860 (0.0007) [2023-10-14 20:58:38,342][61552] Updated weights for policy 0, policy_version 80002 (0.0007) [2023-10-14 20:58:38,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 163676160. Throughput: 0: 1692.9, 1: 1676.2. Samples: 40932746. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:58:38,344][60425] Avg episode reward: [(0, '74.740'), (1, '79.490')] [2023-10-14 20:58:38,506][61585] Updated weights for policy 1, policy_version 79870 (0.0008) [2023-10-14 20:58:38,711][61552] Updated weights for policy 0, policy_version 80012 (0.0009) [2023-10-14 20:58:39,080][61552] Updated weights for policy 0, policy_version 80022 (0.0007) [2023-10-14 20:58:39,440][61552] Updated weights for policy 0, policy_version 80032 (0.0007) [2023-10-14 20:58:42,637][61585] Updated weights for policy 1, policy_version 79880 (0.0012) [2023-10-14 20:58:42,988][61585] Updated weights for policy 1, policy_version 79890 (0.0009) [2023-10-14 20:58:43,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 163741696. Throughput: 0: 1694.4, 1: 1668.5. Samples: 40952938. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 20:58:43,344][60425] Avg episode reward: [(0, '74.900'), (1, '74.340')] [2023-10-14 20:58:43,350][61585] Updated weights for policy 1, policy_version 79900 (0.0007) [2023-10-14 20:58:43,652][61552] Updated weights for policy 0, policy_version 80042 (0.0009) [2023-10-14 20:58:44,025][61552] Updated weights for policy 0, policy_version 80052 (0.0008) [2023-10-14 20:58:44,388][61552] Updated weights for policy 0, policy_version 80062 (0.0009) [2023-10-14 20:58:47,455][61585] Updated weights for policy 1, policy_version 79910 (0.0008) [2023-10-14 20:58:47,827][61585] Updated weights for policy 1, policy_version 79920 (0.0009) [2023-10-14 20:58:48,189][61585] Updated weights for policy 1, policy_version 79930 (0.0009) [2023-10-14 20:58:48,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 163807232. Throughput: 0: 1682.6, 1: 1684.7. Samples: 40962480. Policy #0 lag: (min: 28.0, avg: 31.4, max: 60.0) [2023-10-14 20:58:48,344][60425] Avg episode reward: [(0, '76.700'), (1, '84.760')] [2023-10-14 20:58:48,585][61552] Updated weights for policy 0, policy_version 80072 (0.0010) [2023-10-14 20:58:48,937][61552] Updated weights for policy 0, policy_version 80082 (0.0010) [2023-10-14 20:58:49,301][61552] Updated weights for policy 0, policy_version 80092 (0.0011) [2023-10-14 20:58:52,318][61585] Updated weights for policy 1, policy_version 79940 (0.0008) [2023-10-14 20:58:52,683][61585] Updated weights for policy 1, policy_version 79950 (0.0007) [2023-10-14 20:58:53,052][61585] Updated weights for policy 1, policy_version 79960 (0.0007) [2023-10-14 20:58:53,343][60425] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 163905536. Throughput: 0: 1678.0, 1: 1682.0. Samples: 40982924. Policy #0 lag: (min: 28.0, avg: 31.4, max: 60.0) [2023-10-14 20:58:53,344][60425] Avg episode reward: [(0, '83.350'), (1, '74.450')] [2023-10-14 20:58:53,538][61552] Updated weights for policy 0, policy_version 80102 (0.0009) [2023-10-14 20:58:53,909][61552] Updated weights for policy 0, policy_version 80112 (0.0007) [2023-10-14 20:58:54,273][61552] Updated weights for policy 0, policy_version 80122 (0.0009) [2023-10-14 20:58:57,252][61585] Updated weights for policy 1, policy_version 79970 (0.0007) [2023-10-14 20:58:57,611][61585] Updated weights for policy 1, policy_version 79980 (0.0009) [2023-10-14 20:58:57,980][61585] Updated weights for policy 1, policy_version 79990 (0.0007) [2023-10-14 20:58:58,344][61585] Updated weights for policy 1, policy_version 80000 (0.0009) [2023-10-14 20:58:58,343][60425] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 163971072. Throughput: 0: 1668.9, 1: 1670.1. Samples: 41002840. Policy #0 lag: (min: 28.0, avg: 31.4, max: 60.0) [2023-10-14 20:58:58,344][60425] Avg episode reward: [(0, '81.050'), (1, '78.350')] [2023-10-14 20:58:58,382][61552] Updated weights for policy 0, policy_version 80132 (0.0009) [2023-10-14 20:58:58,744][61552] Updated weights for policy 0, policy_version 80142 (0.0009) [2023-10-14 20:58:59,124][61552] Updated weights for policy 0, policy_version 80152 (0.0008) [2023-10-14 20:59:02,479][61585] Updated weights for policy 1, policy_version 80010 (0.0009) [2023-10-14 20:59:02,853][61585] Updated weights for policy 1, policy_version 80020 (0.0008) [2023-10-14 20:59:03,164][61552] Updated weights for policy 0, policy_version 80162 (0.0007) [2023-10-14 20:59:03,213][61585] Updated weights for policy 1, policy_version 80030 (0.0008) [2023-10-14 20:59:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 164036608. Throughput: 0: 1667.7, 1: 1686.4. Samples: 41012586. Policy #0 lag: (min: 28.0, avg: 31.4, max: 60.0) [2023-10-14 20:59:03,344][60425] Avg episode reward: [(0, '77.860'), (1, '77.260')] [2023-10-14 20:59:03,526][61552] Updated weights for policy 0, policy_version 80172 (0.0010) [2023-10-14 20:59:03,897][61552] Updated weights for policy 0, policy_version 80182 (0.0010) [2023-10-14 20:59:04,258][61552] Updated weights for policy 0, policy_version 80192 (0.0007) [2023-10-14 20:59:07,302][61585] Updated weights for policy 1, policy_version 80040 (0.0007) [2023-10-14 20:59:07,660][61585] Updated weights for policy 1, policy_version 80050 (0.0009) [2023-10-14 20:59:08,024][61585] Updated weights for policy 1, policy_version 80060 (0.0009) [2023-10-14 20:59:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 164102144. Throughput: 0: 1668.4, 1: 1682.5. Samples: 41033126. Policy #0 lag: (min: 28.0, avg: 31.4, max: 60.0) [2023-10-14 20:59:08,344][60425] Avg episode reward: [(0, '83.040'), (1, '83.930')] [2023-10-14 20:59:08,376][61552] Updated weights for policy 0, policy_version 80202 (0.0008) [2023-10-14 20:59:08,743][61552] Updated weights for policy 0, policy_version 80212 (0.0008) [2023-10-14 20:59:09,102][61552] Updated weights for policy 0, policy_version 80222 (0.0009) [2023-10-14 20:59:12,153][61585] Updated weights for policy 1, policy_version 80070 (0.0008) [2023-10-14 20:59:12,515][61585] Updated weights for policy 1, policy_version 80080 (0.0007) [2023-10-14 20:59:12,881][61585] Updated weights for policy 1, policy_version 80090 (0.0007) [2023-10-14 20:59:13,237][61552] Updated weights for policy 0, policy_version 80232 (0.0007) [2023-10-14 20:59:13,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 164167680. Throughput: 0: 1674.8, 1: 1669.5. Samples: 41053066. Policy #0 lag: (min: 28.0, avg: 31.4, max: 60.0) [2023-10-14 20:59:13,344][60425] Avg episode reward: [(0, '81.340'), (1, '82.520')] [2023-10-14 20:59:13,606][61552] Updated weights for policy 0, policy_version 80242 (0.0008) [2023-10-14 20:59:13,989][61552] Updated weights for policy 0, policy_version 80252 (0.0008) [2023-10-14 20:59:16,797][61585] Updated weights for policy 1, policy_version 80100 (0.0008) [2023-10-14 20:59:17,163][61585] Updated weights for policy 1, policy_version 80110 (0.0011) [2023-10-14 20:59:17,512][61585] Updated weights for policy 1, policy_version 80120 (0.0010) [2023-10-14 20:59:17,947][61552] Updated weights for policy 0, policy_version 80262 (0.0008) [2023-10-14 20:59:18,320][61552] Updated weights for policy 0, policy_version 80272 (0.0007) [2023-10-14 20:59:18,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 164233216. Throughput: 0: 1673.2, 1: 1679.7. Samples: 41062950. Policy #0 lag: (min: 28.0, avg: 31.4, max: 60.0) [2023-10-14 20:59:18,344][60425] Avg episode reward: [(0, '81.070'), (1, '85.080')] [2023-10-14 20:59:18,681][61552] Updated weights for policy 0, policy_version 80282 (0.0009) [2023-10-14 20:59:21,457][61585] Updated weights for policy 1, policy_version 80130 (0.0009) [2023-10-14 20:59:21,827][61585] Updated weights for policy 1, policy_version 80140 (0.0010) [2023-10-14 20:59:22,191][61585] Updated weights for policy 1, policy_version 80150 (0.0008) [2023-10-14 20:59:22,548][61585] Updated weights for policy 1, policy_version 80160 (0.0008) [2023-10-14 20:59:22,732][61552] Updated weights for policy 0, policy_version 80292 (0.0010) [2023-10-14 20:59:23,101][61552] Updated weights for policy 0, policy_version 80302 (0.0009) [2023-10-14 20:59:23,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 164298752. Throughput: 0: 1668.7, 1: 1676.8. Samples: 41083296. Policy #0 lag: (min: 28.0, avg: 31.4, max: 60.0) [2023-10-14 20:59:23,344][60425] Avg episode reward: [(0, '77.170'), (1, '80.270')] [2023-10-14 20:59:23,470][61552] Updated weights for policy 0, policy_version 80312 (0.0008) [2023-10-14 20:59:26,701][61585] Updated weights for policy 1, policy_version 80170 (0.0009) [2023-10-14 20:59:27,071][61585] Updated weights for policy 1, policy_version 80180 (0.0008) [2023-10-14 20:59:27,437][61585] Updated weights for policy 1, policy_version 80190 (0.0007) [2023-10-14 20:59:27,566][61552] Updated weights for policy 0, policy_version 80322 (0.0008) [2023-10-14 20:59:27,946][61552] Updated weights for policy 0, policy_version 80332 (0.0009) [2023-10-14 20:59:28,320][61552] Updated weights for policy 0, policy_version 80342 (0.0010) [2023-10-14 20:59:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 164364288. Throughput: 0: 1668.4, 1: 1665.0. Samples: 41102942. Policy #0 lag: (min: 28.0, avg: 31.4, max: 60.0) [2023-10-14 20:59:28,344][60425] Avg episode reward: [(0, '79.090'), (1, '77.650')] [2023-10-14 20:59:28,694][61552] Updated weights for policy 0, policy_version 80352 (0.0009) [2023-10-14 20:59:31,633][61585] Updated weights for policy 1, policy_version 80200 (0.0007) [2023-10-14 20:59:32,009][61585] Updated weights for policy 1, policy_version 80210 (0.0008) [2023-10-14 20:59:32,378][61585] Updated weights for policy 1, policy_version 80220 (0.0009) [2023-10-14 20:59:32,763][61552] Updated weights for policy 0, policy_version 80362 (0.0007) [2023-10-14 20:59:33,132][61552] Updated weights for policy 0, policy_version 80372 (0.0007) [2023-10-14 20:59:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 164429824. Throughput: 0: 1678.8, 1: 1679.8. Samples: 41113618. Policy #0 lag: (min: 28.0, avg: 31.4, max: 60.0) [2023-10-14 20:59:33,344][60425] Avg episode reward: [(0, '70.640'), (1, '79.790')] [2023-10-14 20:59:33,495][61552] Updated weights for policy 0, policy_version 80382 (0.0007) [2023-10-14 20:59:36,410][61585] Updated weights for policy 1, policy_version 80230 (0.0008) [2023-10-14 20:59:36,785][61585] Updated weights for policy 1, policy_version 80240 (0.0008) [2023-10-14 20:59:37,143][61585] Updated weights for policy 1, policy_version 80250 (0.0007) [2023-10-14 20:59:37,377][61552] Updated weights for policy 0, policy_version 80392 (0.0007) [2023-10-14 20:59:37,742][61552] Updated weights for policy 0, policy_version 80402 (0.0008) [2023-10-14 20:59:38,118][61552] Updated weights for policy 0, policy_version 80412 (0.0009) [2023-10-14 20:59:38,343][60425] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 13440.4). Total num frames: 164528128. Throughput: 0: 1683.0, 1: 1668.4. Samples: 41133738. Policy #0 lag: (min: 3.0, avg: 26.8, max: 32.0) [2023-10-14 20:59:38,344][60425] Avg episode reward: [(0, '77.010'), (1, '74.800')] [2023-10-14 20:59:41,170][61585] Updated weights for policy 1, policy_version 80260 (0.0007) [2023-10-14 20:59:41,537][61585] Updated weights for policy 1, policy_version 80270 (0.0008) [2023-10-14 20:59:41,893][61585] Updated weights for policy 1, policy_version 80280 (0.0008) [2023-10-14 20:59:42,232][61552] Updated weights for policy 0, policy_version 80422 (0.0008) [2023-10-14 20:59:42,607][61552] Updated weights for policy 0, policy_version 80432 (0.0008) [2023-10-14 20:59:42,966][61552] Updated weights for policy 0, policy_version 80442 (0.0007) [2023-10-14 20:59:43,344][60425] Fps is (10 sec: 16383.5, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 164593664. Throughput: 0: 1670.1, 1: 1669.3. Samples: 41153114. Policy #0 lag: (min: 3.0, avg: 26.8, max: 32.0) [2023-10-14 20:59:43,345][60425] Avg episode reward: [(0, '78.790'), (1, '79.200')] [2023-10-14 20:59:43,355][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000080288_82214912.pth... [2023-10-14 20:59:43,355][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000080448_82378752.pth... [2023-10-14 20:59:43,391][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000078720_80609280.pth [2023-10-14 20:59:43,400][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000078880_80773120.pth [2023-10-14 20:59:45,882][61585] Updated weights for policy 1, policy_version 80290 (0.0009) [2023-10-14 20:59:46,245][61585] Updated weights for policy 1, policy_version 80300 (0.0008) [2023-10-14 20:59:46,616][61585] Updated weights for policy 1, policy_version 80310 (0.0008) [2023-10-14 20:59:46,976][61585] Updated weights for policy 1, policy_version 80320 (0.0007) [2023-10-14 20:59:47,182][61552] Updated weights for policy 0, policy_version 80452 (0.0010) [2023-10-14 20:59:47,557][61552] Updated weights for policy 0, policy_version 80462 (0.0010) [2023-10-14 20:59:47,927][61552] Updated weights for policy 0, policy_version 80472 (0.0009) [2023-10-14 20:59:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13440.4). Total num frames: 164659200. Throughput: 0: 1681.0, 1: 1684.0. Samples: 41164010. Policy #0 lag: (min: 3.0, avg: 26.8, max: 32.0) [2023-10-14 20:59:48,344][60425] Avg episode reward: [(0, '76.610'), (1, '76.450')] [2023-10-14 20:59:51,110][61585] Updated weights for policy 1, policy_version 80330 (0.0010) [2023-10-14 20:59:51,479][61585] Updated weights for policy 1, policy_version 80340 (0.0008) [2023-10-14 20:59:51,846][61585] Updated weights for policy 1, policy_version 80350 (0.0009) [2023-10-14 20:59:52,079][61552] Updated weights for policy 0, policy_version 80482 (0.0007) [2023-10-14 20:59:52,448][61552] Updated weights for policy 0, policy_version 80492 (0.0008) [2023-10-14 20:59:52,816][61552] Updated weights for policy 0, policy_version 80502 (0.0008) [2023-10-14 20:59:53,177][61552] Updated weights for policy 0, policy_version 80512 (0.0009) [2023-10-14 20:59:53,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 164724736. Throughput: 0: 1682.3, 1: 1659.4. Samples: 41183500. Policy #0 lag: (min: 3.0, avg: 26.8, max: 32.0) [2023-10-14 20:59:53,345][60425] Avg episode reward: [(0, '75.510'), (1, '77.330')] [2023-10-14 20:59:55,935][61585] Updated weights for policy 1, policy_version 80360 (0.0008) [2023-10-14 20:59:56,297][61585] Updated weights for policy 1, policy_version 80370 (0.0010) [2023-10-14 20:59:56,665][61585] Updated weights for policy 1, policy_version 80380 (0.0011) [2023-10-14 20:59:57,299][61552] Updated weights for policy 0, policy_version 80522 (0.0007) [2023-10-14 20:59:57,658][61552] Updated weights for policy 0, policy_version 80532 (0.0007) [2023-10-14 20:59:58,030][61552] Updated weights for policy 0, policy_version 80542 (0.0008) [2023-10-14 20:59:58,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 164790272. Throughput: 0: 1663.5, 1: 1672.3. Samples: 41203176. Policy #0 lag: (min: 3.0, avg: 26.8, max: 32.0) [2023-10-14 20:59:58,344][60425] Avg episode reward: [(0, '78.740'), (1, '75.360')] [2023-10-14 21:00:00,678][61585] Updated weights for policy 1, policy_version 80390 (0.0011) [2023-10-14 21:00:01,051][61585] Updated weights for policy 1, policy_version 80400 (0.0010) [2023-10-14 21:00:01,415][61585] Updated weights for policy 1, policy_version 80410 (0.0009) [2023-10-14 21:00:02,031][61552] Updated weights for policy 0, policy_version 80552 (0.0009) [2023-10-14 21:00:02,403][61552] Updated weights for policy 0, policy_version 80562 (0.0009) [2023-10-14 21:00:02,765][61552] Updated weights for policy 0, policy_version 80572 (0.0008) [2023-10-14 21:00:03,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 164855808. Throughput: 0: 1680.9, 1: 1674.5. Samples: 41213944. Policy #0 lag: (min: 3.0, avg: 26.8, max: 32.0) [2023-10-14 21:00:03,344][60425] Avg episode reward: [(0, '72.950'), (1, '79.780')] [2023-10-14 21:00:05,571][61585] Updated weights for policy 1, policy_version 80420 (0.0010) [2023-10-14 21:00:05,941][61585] Updated weights for policy 1, policy_version 80430 (0.0009) [2023-10-14 21:00:06,302][61585] Updated weights for policy 1, policy_version 80440 (0.0008) [2023-10-14 21:00:07,029][61552] Updated weights for policy 0, policy_version 80582 (0.0008) [2023-10-14 21:00:07,402][61552] Updated weights for policy 0, policy_version 80592 (0.0008) [2023-10-14 21:00:07,765][61552] Updated weights for policy 0, policy_version 80602 (0.0009) [2023-10-14 21:00:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 164921344. Throughput: 0: 1680.0, 1: 1656.8. Samples: 41233452. Policy #0 lag: (min: 3.0, avg: 26.8, max: 32.0) [2023-10-14 21:00:08,344][60425] Avg episode reward: [(0, '77.270'), (1, '77.710')] [2023-10-14 21:00:10,357][61585] Updated weights for policy 1, policy_version 80450 (0.0009) [2023-10-14 21:00:10,723][61585] Updated weights for policy 1, policy_version 80460 (0.0011) [2023-10-14 21:00:11,100][61585] Updated weights for policy 1, policy_version 80470 (0.0009) [2023-10-14 21:00:11,464][61585] Updated weights for policy 1, policy_version 80480 (0.0011) [2023-10-14 21:00:11,748][61552] Updated weights for policy 0, policy_version 80612 (0.0009) [2023-10-14 21:00:12,116][61552] Updated weights for policy 0, policy_version 80622 (0.0010) [2023-10-14 21:00:12,489][61552] Updated weights for policy 0, policy_version 80632 (0.0007) [2023-10-14 21:00:13,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 164986880. Throughput: 0: 1654.2, 1: 1680.9. Samples: 41253020. Policy #0 lag: (min: 3.0, avg: 26.8, max: 32.0) [2023-10-14 21:00:13,344][60425] Avg episode reward: [(0, '79.270'), (1, '83.470')] [2023-10-14 21:00:15,556][61585] Updated weights for policy 1, policy_version 80490 (0.0010) [2023-10-14 21:00:15,921][61585] Updated weights for policy 1, policy_version 80500 (0.0010) [2023-10-14 21:00:16,285][61585] Updated weights for policy 1, policy_version 80510 (0.0010) [2023-10-14 21:00:16,793][61552] Updated weights for policy 0, policy_version 80642 (0.0008) [2023-10-14 21:00:17,162][61552] Updated weights for policy 0, policy_version 80652 (0.0008) [2023-10-14 21:00:17,525][61552] Updated weights for policy 0, policy_version 80662 (0.0007) [2023-10-14 21:00:17,882][61552] Updated weights for policy 0, policy_version 80672 (0.0009) [2023-10-14 21:00:18,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 165052416. Throughput: 0: 1672.7, 1: 1664.2. Samples: 41263782. Policy #0 lag: (min: 3.0, avg: 26.8, max: 32.0) [2023-10-14 21:00:18,344][60425] Avg episode reward: [(0, '78.150'), (1, '78.570')] [2023-10-14 21:00:20,383][61585] Updated weights for policy 1, policy_version 80520 (0.0008) [2023-10-14 21:00:20,755][61585] Updated weights for policy 1, policy_version 80530 (0.0007) [2023-10-14 21:00:21,119][61585] Updated weights for policy 1, policy_version 80540 (0.0008) [2023-10-14 21:00:22,125][61552] Updated weights for policy 0, policy_version 80682 (0.0009) [2023-10-14 21:00:22,483][61552] Updated weights for policy 0, policy_version 80692 (0.0011) [2023-10-14 21:00:22,864][61552] Updated weights for policy 0, policy_version 80702 (0.0008) [2023-10-14 21:00:23,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 165117952. Throughput: 0: 1667.5, 1: 1662.7. Samples: 41283598. Policy #0 lag: (min: 3.0, avg: 26.8, max: 32.0) [2023-10-14 21:00:23,344][60425] Avg episode reward: [(0, '76.130'), (1, '80.440')] [2023-10-14 21:00:25,304][61585] Updated weights for policy 1, policy_version 80550 (0.0010) [2023-10-14 21:00:25,677][61585] Updated weights for policy 1, policy_version 80560 (0.0008) [2023-10-14 21:00:26,041][61585] Updated weights for policy 1, policy_version 80570 (0.0007) [2023-10-14 21:00:27,018][61552] Updated weights for policy 0, policy_version 80712 (0.0010) [2023-10-14 21:00:27,395][61552] Updated weights for policy 0, policy_version 80722 (0.0012) [2023-10-14 21:00:27,769][61552] Updated weights for policy 0, policy_version 80732 (0.0010) [2023-10-14 21:00:28,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 165183488. Throughput: 0: 1652.5, 1: 1675.7. Samples: 41302880. Policy #0 lag: (min: 1.0, avg: 7.2, max: 33.0) [2023-10-14 21:00:28,344][60425] Avg episode reward: [(0, '77.870'), (1, '79.220')] [2023-10-14 21:00:30,047][61585] Updated weights for policy 1, policy_version 80580 (0.0010) [2023-10-14 21:00:30,414][61585] Updated weights for policy 1, policy_version 80590 (0.0009) [2023-10-14 21:00:30,779][61585] Updated weights for policy 1, policy_version 80600 (0.0009) [2023-10-14 21:00:31,851][61552] Updated weights for policy 0, policy_version 80742 (0.0009) [2023-10-14 21:00:32,222][61552] Updated weights for policy 0, policy_version 80752 (0.0009) [2023-10-14 21:00:32,580][61552] Updated weights for policy 0, policy_version 80762 (0.0009) [2023-10-14 21:00:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 165249024. Throughput: 0: 1661.5, 1: 1654.8. Samples: 41313242. Policy #0 lag: (min: 1.0, avg: 7.2, max: 33.0) [2023-10-14 21:00:33,344][60425] Avg episode reward: [(0, '74.760'), (1, '80.740')] [2023-10-14 21:00:34,966][61585] Updated weights for policy 1, policy_version 80610 (0.0008) [2023-10-14 21:00:35,337][61585] Updated weights for policy 1, policy_version 80620 (0.0009) [2023-10-14 21:00:35,695][61585] Updated weights for policy 1, policy_version 80630 (0.0010) [2023-10-14 21:00:36,058][61585] Updated weights for policy 1, policy_version 80640 (0.0008) [2023-10-14 21:00:36,448][61552] Updated weights for policy 0, policy_version 80772 (0.0010) [2023-10-14 21:00:36,824][61552] Updated weights for policy 0, policy_version 80782 (0.0008) [2023-10-14 21:00:37,192][61552] Updated weights for policy 0, policy_version 80792 (0.0008) [2023-10-14 21:00:38,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 165314560. Throughput: 0: 1654.0, 1: 1675.3. Samples: 41333320. Policy #0 lag: (min: 1.0, avg: 7.2, max: 33.0) [2023-10-14 21:00:38,344][60425] Avg episode reward: [(0, '77.980'), (1, '79.450')] [2023-10-14 21:00:40,289][61585] Updated weights for policy 1, policy_version 80650 (0.0011) [2023-10-14 21:00:40,653][61585] Updated weights for policy 1, policy_version 80660 (0.0009) [2023-10-14 21:00:41,015][61585] Updated weights for policy 1, policy_version 80670 (0.0008) [2023-10-14 21:00:41,210][61552] Updated weights for policy 0, policy_version 80802 (0.0009) [2023-10-14 21:00:41,573][61552] Updated weights for policy 0, policy_version 80812 (0.0007) [2023-10-14 21:00:41,935][61552] Updated weights for policy 0, policy_version 80822 (0.0009) [2023-10-14 21:00:42,303][61552] Updated weights for policy 0, policy_version 80832 (0.0009) [2023-10-14 21:00:43,343][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 165380096. Throughput: 0: 1653.5, 1: 1682.9. Samples: 41353318. Policy #0 lag: (min: 1.0, avg: 7.2, max: 33.0) [2023-10-14 21:00:43,345][60425] Avg episode reward: [(0, '78.820'), (1, '77.430')] [2023-10-14 21:00:44,983][61585] Updated weights for policy 1, policy_version 80680 (0.0008) [2023-10-14 21:00:45,350][61585] Updated weights for policy 1, policy_version 80690 (0.0009) [2023-10-14 21:00:45,710][61585] Updated weights for policy 1, policy_version 80700 (0.0007) [2023-10-14 21:00:46,435][61552] Updated weights for policy 0, policy_version 80842 (0.0011) [2023-10-14 21:00:46,801][61552] Updated weights for policy 0, policy_version 80852 (0.0009) [2023-10-14 21:00:47,159][61552] Updated weights for policy 0, policy_version 80862 (0.0007) [2023-10-14 21:00:48,344][60425] Fps is (10 sec: 13105.9, 60 sec: 13107.0, 300 sec: 13329.3). Total num frames: 165445632. Throughput: 0: 1665.6, 1: 1666.1. Samples: 41363876. Policy #0 lag: (min: 1.0, avg: 7.2, max: 33.0) [2023-10-14 21:00:48,345][60425] Avg episode reward: [(0, '78.190'), (1, '81.870')] [2023-10-14 21:00:49,813][61585] Updated weights for policy 1, policy_version 80710 (0.0008) [2023-10-14 21:00:50,174][61585] Updated weights for policy 1, policy_version 80720 (0.0008) [2023-10-14 21:00:50,536][61585] Updated weights for policy 1, policy_version 80730 (0.0008) [2023-10-14 21:00:51,404][61552] Updated weights for policy 0, policy_version 80872 (0.0008) [2023-10-14 21:00:51,775][61552] Updated weights for policy 0, policy_version 80882 (0.0009) [2023-10-14 21:00:52,140][61552] Updated weights for policy 0, policy_version 80892 (0.0010) [2023-10-14 21:00:53,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 165511168. Throughput: 0: 1652.1, 1: 1683.5. Samples: 41383552. Policy #0 lag: (min: 1.0, avg: 7.2, max: 33.0) [2023-10-14 21:00:53,344][60425] Avg episode reward: [(0, '76.150'), (1, '71.780')] [2023-10-14 21:00:54,551][61585] Updated weights for policy 1, policy_version 80740 (0.0008) [2023-10-14 21:00:54,908][61585] Updated weights for policy 1, policy_version 80750 (0.0011) [2023-10-14 21:00:55,280][61585] Updated weights for policy 1, policy_version 80760 (0.0009) [2023-10-14 21:00:56,159][61552] Updated weights for policy 0, policy_version 80902 (0.0009) [2023-10-14 21:00:56,536][61552] Updated weights for policy 0, policy_version 80912 (0.0008) [2023-10-14 21:00:56,903][61552] Updated weights for policy 0, policy_version 80922 (0.0009) [2023-10-14 21:00:58,343][60425] Fps is (10 sec: 13108.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 165576704. Throughput: 0: 1669.9, 1: 1682.0. Samples: 41403856. Policy #0 lag: (min: 1.0, avg: 7.2, max: 33.0) [2023-10-14 21:00:58,344][60425] Avg episode reward: [(0, '74.610'), (1, '71.770')] [2023-10-14 21:00:59,274][61585] Updated weights for policy 1, policy_version 80770 (0.0010) [2023-10-14 21:00:59,640][61585] Updated weights for policy 1, policy_version 80780 (0.0008) [2023-10-14 21:01:00,007][61585] Updated weights for policy 1, policy_version 80790 (0.0007) [2023-10-14 21:01:00,375][61585] Updated weights for policy 1, policy_version 80800 (0.0009) [2023-10-14 21:01:00,871][61552] Updated weights for policy 0, policy_version 80932 (0.0008) [2023-10-14 21:01:01,230][61552] Updated weights for policy 0, policy_version 80942 (0.0009) [2023-10-14 21:01:01,598][61552] Updated weights for policy 0, policy_version 80952 (0.0009) [2023-10-14 21:01:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 165642240. Throughput: 0: 1674.4, 1: 1670.1. Samples: 41414284. Policy #0 lag: (min: 1.0, avg: 7.2, max: 33.0) [2023-10-14 21:01:03,344][60425] Avg episode reward: [(0, '73.550'), (1, '75.750')] [2023-10-14 21:01:04,415][61585] Updated weights for policy 1, policy_version 80810 (0.0008) [2023-10-14 21:01:04,783][61585] Updated weights for policy 1, policy_version 80820 (0.0009) [2023-10-14 21:01:05,145][61585] Updated weights for policy 1, policy_version 80830 (0.0007) [2023-10-14 21:01:05,583][61552] Updated weights for policy 0, policy_version 80962 (0.0010) [2023-10-14 21:01:05,954][61552] Updated weights for policy 0, policy_version 80972 (0.0007) [2023-10-14 21:01:06,322][61552] Updated weights for policy 0, policy_version 80982 (0.0011) [2023-10-14 21:01:06,692][61552] Updated weights for policy 0, policy_version 80992 (0.0012) [2023-10-14 21:01:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 165707776. Throughput: 0: 1651.1, 1: 1686.7. Samples: 41433800. Policy #0 lag: (min: 1.0, avg: 7.2, max: 33.0) [2023-10-14 21:01:08,344][60425] Avg episode reward: [(0, '75.490'), (1, '78.980')] [2023-10-14 21:01:09,254][61585] Updated weights for policy 1, policy_version 80840 (0.0010) [2023-10-14 21:01:09,626][61585] Updated weights for policy 1, policy_version 80850 (0.0010) [2023-10-14 21:01:09,981][61585] Updated weights for policy 1, policy_version 80860 (0.0009) [2023-10-14 21:01:10,827][61552] Updated weights for policy 0, policy_version 81002 (0.0009) [2023-10-14 21:01:11,192][61552] Updated weights for policy 0, policy_version 81012 (0.0010) [2023-10-14 21:01:11,569][61552] Updated weights for policy 0, policy_version 81022 (0.0010) [2023-10-14 21:01:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 165773312. Throughput: 0: 1675.6, 1: 1687.2. Samples: 41454204. Policy #0 lag: (min: 1.0, avg: 7.2, max: 33.0) [2023-10-14 21:01:13,344][60425] Avg episode reward: [(0, '78.090'), (1, '73.530')] [2023-10-14 21:01:13,937][61585] Updated weights for policy 1, policy_version 80870 (0.0010) [2023-10-14 21:01:14,299][61585] Updated weights for policy 1, policy_version 80880 (0.0012) [2023-10-14 21:01:14,672][61585] Updated weights for policy 1, policy_version 80890 (0.0008) [2023-10-14 21:01:15,860][61552] Updated weights for policy 0, policy_version 81032 (0.0010) [2023-10-14 21:01:16,220][61552] Updated weights for policy 0, policy_version 81042 (0.0008) [2023-10-14 21:01:16,591][61552] Updated weights for policy 0, policy_version 81052 (0.0007) [2023-10-14 21:01:18,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 165838848. Throughput: 0: 1684.3, 1: 1678.9. Samples: 41464588. Policy #0 lag: (min: 1.0, avg: 7.2, max: 33.0) [2023-10-14 21:01:18,345][60425] Avg episode reward: [(0, '76.470'), (1, '79.120')] [2023-10-14 21:01:18,643][61585] Updated weights for policy 1, policy_version 80900 (0.0008) [2023-10-14 21:01:19,004][61585] Updated weights for policy 1, policy_version 80910 (0.0007) [2023-10-14 21:01:19,370][61585] Updated weights for policy 1, policy_version 80920 (0.0008) [2023-10-14 21:01:20,626][61552] Updated weights for policy 0, policy_version 81062 (0.0008) [2023-10-14 21:01:20,998][61552] Updated weights for policy 0, policy_version 81072 (0.0010) [2023-10-14 21:01:21,366][61552] Updated weights for policy 0, policy_version 81082 (0.0010) [2023-10-14 21:01:23,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 165904384. Throughput: 0: 1665.4, 1: 1690.4. Samples: 41484332. Policy #0 lag: (min: 17.0, avg: 20.3, max: 49.0) [2023-10-14 21:01:23,344][60425] Avg episode reward: [(0, '77.220'), (1, '80.020')] [2023-10-14 21:01:23,406][61585] Updated weights for policy 1, policy_version 80930 (0.0007) [2023-10-14 21:01:23,766][61585] Updated weights for policy 1, policy_version 80940 (0.0008) [2023-10-14 21:01:24,130][61585] Updated weights for policy 1, policy_version 80950 (0.0007) [2023-10-14 21:01:24,489][61585] Updated weights for policy 1, policy_version 80960 (0.0008) [2023-10-14 21:01:25,366][61552] Updated weights for policy 0, policy_version 81092 (0.0010) [2023-10-14 21:01:25,733][61552] Updated weights for policy 0, policy_version 81102 (0.0011) [2023-10-14 21:01:26,104][61552] Updated weights for policy 0, policy_version 81112 (0.0009) [2023-10-14 21:01:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 165969920. Throughput: 0: 1680.0, 1: 1693.8. Samples: 41505136. Policy #0 lag: (min: 17.0, avg: 20.3, max: 49.0) [2023-10-14 21:01:28,344][60425] Avg episode reward: [(0, '78.860'), (1, '75.010')] [2023-10-14 21:01:28,539][61585] Updated weights for policy 1, policy_version 80970 (0.0008) [2023-10-14 21:01:28,910][61585] Updated weights for policy 1, policy_version 80980 (0.0009) [2023-10-14 21:01:29,268][61585] Updated weights for policy 1, policy_version 80990 (0.0008) [2023-10-14 21:01:30,247][61552] Updated weights for policy 0, policy_version 81122 (0.0008) [2023-10-14 21:01:30,610][61552] Updated weights for policy 0, policy_version 81132 (0.0008) [2023-10-14 21:01:30,981][61552] Updated weights for policy 0, policy_version 81142 (0.0007) [2023-10-14 21:01:31,349][61552] Updated weights for policy 0, policy_version 81152 (0.0008) [2023-10-14 21:01:33,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 166035456. Throughput: 0: 1669.4, 1: 1687.7. Samples: 41514942. Policy #0 lag: (min: 17.0, avg: 20.3, max: 49.0) [2023-10-14 21:01:33,344][60425] Avg episode reward: [(0, '79.540'), (1, '77.820')] [2023-10-14 21:01:33,459][61585] Updated weights for policy 1, policy_version 81000 (0.0009) [2023-10-14 21:01:33,820][61585] Updated weights for policy 1, policy_version 81010 (0.0011) [2023-10-14 21:01:34,187][61585] Updated weights for policy 1, policy_version 81020 (0.0007) [2023-10-14 21:01:35,516][61552] Updated weights for policy 0, policy_version 81162 (0.0008) [2023-10-14 21:01:35,882][61552] Updated weights for policy 0, policy_version 81172 (0.0009) [2023-10-14 21:01:36,244][61552] Updated weights for policy 0, policy_version 81182 (0.0007) [2023-10-14 21:01:38,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 166100992. Throughput: 0: 1673.5, 1: 1688.1. Samples: 41534828. Policy #0 lag: (min: 17.0, avg: 20.3, max: 49.0) [2023-10-14 21:01:38,344][60425] Avg episode reward: [(0, '80.320'), (1, '81.760')] [2023-10-14 21:01:38,416][61585] Updated weights for policy 1, policy_version 81030 (0.0008) [2023-10-14 21:01:38,775][61585] Updated weights for policy 1, policy_version 81040 (0.0008) [2023-10-14 21:01:39,133][61585] Updated weights for policy 1, policy_version 81050 (0.0011) [2023-10-14 21:01:40,170][61552] Updated weights for policy 0, policy_version 81192 (0.0007) [2023-10-14 21:01:40,537][61552] Updated weights for policy 0, policy_version 81202 (0.0008) [2023-10-14 21:01:40,907][61552] Updated weights for policy 0, policy_version 81212 (0.0007) [2023-10-14 21:01:43,296][61585] Updated weights for policy 1, policy_version 81060 (0.0007) [2023-10-14 21:01:43,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 166166528. Throughput: 0: 1687.2, 1: 1685.9. Samples: 41555646. Policy #0 lag: (min: 17.0, avg: 20.3, max: 49.0) [2023-10-14 21:01:43,344][60425] Avg episode reward: [(0, '83.010'), (1, '84.480')] [2023-10-14 21:01:43,354][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000081216_83165184.pth... [2023-10-14 21:01:43,384][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000079648_81559552.pth [2023-10-14 21:01:43,659][61585] Updated weights for policy 1, policy_version 81070 (0.0010) [2023-10-14 21:01:44,026][61585] Updated weights for policy 1, policy_version 81080 (0.0010) [2023-10-14 21:01:44,312][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000081088_83034112.pth... [2023-10-14 21:01:44,341][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000079488_81395712.pth [2023-10-14 21:01:44,890][61552] Updated weights for policy 0, policy_version 81222 (0.0008) [2023-10-14 21:01:45,248][61552] Updated weights for policy 0, policy_version 81232 (0.0011) [2023-10-14 21:01:45,616][61552] Updated weights for policy 0, policy_version 81242 (0.0009) [2023-10-14 21:01:48,048][61585] Updated weights for policy 1, policy_version 81090 (0.0010) [2023-10-14 21:01:48,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.5, 300 sec: 13329.4). Total num frames: 166232064. Throughput: 0: 1662.0, 1: 1688.4. Samples: 41565050. Policy #0 lag: (min: 17.0, avg: 20.3, max: 49.0) [2023-10-14 21:01:48,344][60425] Avg episode reward: [(0, '80.390'), (1, '75.910')] [2023-10-14 21:01:48,410][61585] Updated weights for policy 1, policy_version 81100 (0.0007) [2023-10-14 21:01:48,774][61585] Updated weights for policy 1, policy_version 81110 (0.0010) [2023-10-14 21:01:49,135][61585] Updated weights for policy 1, policy_version 81120 (0.0010) [2023-10-14 21:01:49,678][61552] Updated weights for policy 0, policy_version 81252 (0.0009) [2023-10-14 21:01:50,042][61552] Updated weights for policy 0, policy_version 81262 (0.0009) [2023-10-14 21:01:50,404][61552] Updated weights for policy 0, policy_version 81272 (0.0009) [2023-10-14 21:01:53,161][61585] Updated weights for policy 1, policy_version 81130 (0.0009) [2023-10-14 21:01:53,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 166297600. Throughput: 0: 1683.7, 1: 1682.3. Samples: 41585272. Policy #0 lag: (min: 17.0, avg: 20.3, max: 49.0) [2023-10-14 21:01:53,344][60425] Avg episode reward: [(0, '80.290'), (1, '75.790')] [2023-10-14 21:01:53,533][61585] Updated weights for policy 1, policy_version 81140 (0.0008) [2023-10-14 21:01:53,900][61585] Updated weights for policy 1, policy_version 81150 (0.0010) [2023-10-14 21:01:54,495][61552] Updated weights for policy 0, policy_version 81282 (0.0008) [2023-10-14 21:01:54,855][61552] Updated weights for policy 0, policy_version 81292 (0.0012) [2023-10-14 21:01:55,233][61552] Updated weights for policy 0, policy_version 81302 (0.0007) [2023-10-14 21:01:55,595][61552] Updated weights for policy 0, policy_version 81312 (0.0008) [2023-10-14 21:01:58,140][61585] Updated weights for policy 1, policy_version 81160 (0.0008) [2023-10-14 21:01:58,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 166363136. Throughput: 0: 1692.4, 1: 1678.4. Samples: 41605894. Policy #0 lag: (min: 17.0, avg: 20.3, max: 49.0) [2023-10-14 21:01:58,344][60425] Avg episode reward: [(0, '82.930'), (1, '80.340')] [2023-10-14 21:01:58,501][61585] Updated weights for policy 1, policy_version 81170 (0.0008) [2023-10-14 21:01:58,866][61585] Updated weights for policy 1, policy_version 81180 (0.0010) [2023-10-14 21:01:59,897][61552] Updated weights for policy 0, policy_version 81322 (0.0011) [2023-10-14 21:02:00,269][61552] Updated weights for policy 0, policy_version 81332 (0.0008) [2023-10-14 21:02:00,651][61552] Updated weights for policy 0, policy_version 81342 (0.0010) [2023-10-14 21:02:02,946][61585] Updated weights for policy 1, policy_version 81190 (0.0008) [2023-10-14 21:02:03,314][61585] Updated weights for policy 1, policy_version 81200 (0.0008) [2023-10-14 21:02:03,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 166428672. Throughput: 0: 1660.8, 1: 1677.7. Samples: 41614820. Policy #0 lag: (min: 17.0, avg: 20.3, max: 49.0) [2023-10-14 21:02:03,345][60425] Avg episode reward: [(0, '80.270'), (1, '84.720')] [2023-10-14 21:02:03,683][61585] Updated weights for policy 1, policy_version 81210 (0.0007) [2023-10-14 21:02:04,772][61552] Updated weights for policy 0, policy_version 81352 (0.0009) [2023-10-14 21:02:05,139][61552] Updated weights for policy 0, policy_version 81362 (0.0009) [2023-10-14 21:02:05,508][61552] Updated weights for policy 0, policy_version 81372 (0.0008) [2023-10-14 21:02:07,816][61585] Updated weights for policy 1, policy_version 81220 (0.0008) [2023-10-14 21:02:08,171][61585] Updated weights for policy 1, policy_version 81230 (0.0007) [2023-10-14 21:02:08,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 166494208. Throughput: 0: 1677.5, 1: 1672.0. Samples: 41635062. Policy #0 lag: (min: 17.0, avg: 20.3, max: 49.0) [2023-10-14 21:02:08,344][60425] Avg episode reward: [(0, '77.900'), (1, '78.070')] [2023-10-14 21:02:08,539][61585] Updated weights for policy 1, policy_version 81240 (0.0008) [2023-10-14 21:02:09,653][61552] Updated weights for policy 0, policy_version 81382 (0.0009) [2023-10-14 21:02:10,013][61552] Updated weights for policy 0, policy_version 81392 (0.0009) [2023-10-14 21:02:10,378][61552] Updated weights for policy 0, policy_version 81402 (0.0011) [2023-10-14 21:02:12,781][61585] Updated weights for policy 1, policy_version 81250 (0.0011) [2023-10-14 21:02:13,154][61585] Updated weights for policy 1, policy_version 81260 (0.0009) [2023-10-14 21:02:13,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 166559744. Throughput: 0: 1678.0, 1: 1661.6. Samples: 41655416. Policy #0 lag: (min: 17.0, avg: 20.3, max: 49.0) [2023-10-14 21:02:13,344][60425] Avg episode reward: [(0, '80.180'), (1, '76.340')] [2023-10-14 21:02:13,519][61585] Updated weights for policy 1, policy_version 81270 (0.0008) [2023-10-14 21:02:13,890][61585] Updated weights for policy 1, policy_version 81280 (0.0008) [2023-10-14 21:02:14,431][61552] Updated weights for policy 0, policy_version 81412 (0.0009) [2023-10-14 21:02:14,804][61552] Updated weights for policy 0, policy_version 81422 (0.0009) [2023-10-14 21:02:15,172][61552] Updated weights for policy 0, policy_version 81432 (0.0008) [2023-10-14 21:02:18,035][61585] Updated weights for policy 1, policy_version 81290 (0.0009) [2023-10-14 21:02:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 166625280. Throughput: 0: 1660.4, 1: 1667.2. Samples: 41664686. Policy #0 lag: (min: 29.0, avg: 39.6, max: 61.0) [2023-10-14 21:02:18,344][60425] Avg episode reward: [(0, '75.240'), (1, '80.140')] [2023-10-14 21:02:18,401][61585] Updated weights for policy 1, policy_version 81300 (0.0012) [2023-10-14 21:02:18,764][61585] Updated weights for policy 1, policy_version 81310 (0.0011) [2023-10-14 21:02:19,311][61552] Updated weights for policy 0, policy_version 81442 (0.0010) [2023-10-14 21:02:19,685][61552] Updated weights for policy 0, policy_version 81452 (0.0008) [2023-10-14 21:02:20,050][61552] Updated weights for policy 0, policy_version 81462 (0.0008) [2023-10-14 21:02:20,423][61552] Updated weights for policy 0, policy_version 81472 (0.0007) [2023-10-14 21:02:22,699][61585] Updated weights for policy 1, policy_version 81320 (0.0008) [2023-10-14 21:02:23,072][61585] Updated weights for policy 1, policy_version 81330 (0.0007) [2023-10-14 21:02:23,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 166690816. Throughput: 0: 1669.6, 1: 1671.5. Samples: 41685176. Policy #0 lag: (min: 29.0, avg: 39.6, max: 61.0) [2023-10-14 21:02:23,345][60425] Avg episode reward: [(0, '76.190'), (1, '76.670')] [2023-10-14 21:02:23,439][61585] Updated weights for policy 1, policy_version 81340 (0.0008) [2023-10-14 21:02:24,476][61552] Updated weights for policy 0, policy_version 81482 (0.0009) [2023-10-14 21:02:24,847][61552] Updated weights for policy 0, policy_version 81492 (0.0010) [2023-10-14 21:02:25,222][61552] Updated weights for policy 0, policy_version 81502 (0.0009) [2023-10-14 21:02:27,689][61585] Updated weights for policy 1, policy_version 81350 (0.0008) [2023-10-14 21:02:28,058][61585] Updated weights for policy 1, policy_version 81360 (0.0007) [2023-10-14 21:02:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 166756352. Throughput: 0: 1662.2, 1: 1662.0. Samples: 41705236. Policy #0 lag: (min: 29.0, avg: 39.6, max: 61.0) [2023-10-14 21:02:28,344][60425] Avg episode reward: [(0, '76.270'), (1, '81.340')] [2023-10-14 21:02:28,418][61585] Updated weights for policy 1, policy_version 81370 (0.0008) [2023-10-14 21:02:29,372][61552] Updated weights for policy 0, policy_version 81512 (0.0008) [2023-10-14 21:02:29,742][61552] Updated weights for policy 0, policy_version 81522 (0.0008) [2023-10-14 21:02:30,115][61552] Updated weights for policy 0, policy_version 81532 (0.0008) [2023-10-14 21:02:32,438][61585] Updated weights for policy 1, policy_version 81380 (0.0009) [2023-10-14 21:02:32,803][61585] Updated weights for policy 1, policy_version 81390 (0.0007) [2023-10-14 21:02:33,175][61585] Updated weights for policy 1, policy_version 81400 (0.0007) [2023-10-14 21:02:33,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 166821888. Throughput: 0: 1657.8, 1: 1663.8. Samples: 41714522. Policy #0 lag: (min: 29.0, avg: 39.6, max: 61.0) [2023-10-14 21:02:33,344][60425] Avg episode reward: [(0, '79.620'), (1, '81.410')] [2023-10-14 21:02:34,192][61552] Updated weights for policy 0, policy_version 81542 (0.0009) [2023-10-14 21:02:34,571][61552] Updated weights for policy 0, policy_version 81552 (0.0009) [2023-10-14 21:02:34,927][61552] Updated weights for policy 0, policy_version 81562 (0.0009) [2023-10-14 21:02:37,278][61585] Updated weights for policy 1, policy_version 81410 (0.0008) [2023-10-14 21:02:37,633][61585] Updated weights for policy 1, policy_version 81420 (0.0010) [2023-10-14 21:02:38,005][61585] Updated weights for policy 1, policy_version 81430 (0.0007) [2023-10-14 21:02:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 166887424. Throughput: 0: 1657.6, 1: 1670.2. Samples: 41735024. Policy #0 lag: (min: 29.0, avg: 39.6, max: 61.0) [2023-10-14 21:02:38,344][60425] Avg episode reward: [(0, '77.420'), (1, '79.800')] [2023-10-14 21:02:38,369][61585] Updated weights for policy 1, policy_version 81440 (0.0007) [2023-10-14 21:02:39,011][61552] Updated weights for policy 0, policy_version 81572 (0.0008) [2023-10-14 21:02:39,388][61552] Updated weights for policy 0, policy_version 81582 (0.0009) [2023-10-14 21:02:39,751][61552] Updated weights for policy 0, policy_version 81592 (0.0010) [2023-10-14 21:02:42,553][61585] Updated weights for policy 1, policy_version 81450 (0.0009) [2023-10-14 21:02:42,913][61585] Updated weights for policy 1, policy_version 81460 (0.0007) [2023-10-14 21:02:43,287][61585] Updated weights for policy 1, policy_version 81470 (0.0009) [2023-10-14 21:02:43,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 166952960. Throughput: 0: 1660.1, 1: 1658.4. Samples: 41755228. Policy #0 lag: (min: 29.0, avg: 39.6, max: 61.0) [2023-10-14 21:02:43,344][60425] Avg episode reward: [(0, '78.960'), (1, '73.130')] [2023-10-14 21:02:43,894][61552] Updated weights for policy 0, policy_version 81602 (0.0009) [2023-10-14 21:02:44,306][61552] Updated weights for policy 0, policy_version 81612 (0.0009) [2023-10-14 21:02:44,675][61552] Updated weights for policy 0, policy_version 81622 (0.0008) [2023-10-14 21:02:45,039][61552] Updated weights for policy 0, policy_version 81632 (0.0008) [2023-10-14 21:02:47,254][61585] Updated weights for policy 1, policy_version 81480 (0.0011) [2023-10-14 21:02:47,609][61585] Updated weights for policy 1, policy_version 81490 (0.0012) [2023-10-14 21:02:47,973][61585] Updated weights for policy 1, policy_version 81500 (0.0008) [2023-10-14 21:02:48,343][60425] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 167051264. Throughput: 0: 1660.0, 1: 1673.6. Samples: 41764828. Policy #0 lag: (min: 29.0, avg: 39.6, max: 61.0) [2023-10-14 21:02:48,344][60425] Avg episode reward: [(0, '78.170'), (1, '79.080')] [2023-10-14 21:02:49,176][61552] Updated weights for policy 0, policy_version 81642 (0.0009) [2023-10-14 21:02:49,542][61552] Updated weights for policy 0, policy_version 81652 (0.0008) [2023-10-14 21:02:49,908][61552] Updated weights for policy 0, policy_version 81662 (0.0010) [2023-10-14 21:02:52,092][61585] Updated weights for policy 1, policy_version 81510 (0.0008) [2023-10-14 21:02:52,466][61585] Updated weights for policy 1, policy_version 81520 (0.0007) [2023-10-14 21:02:52,836][61585] Updated weights for policy 1, policy_version 81530 (0.0008) [2023-10-14 21:02:53,343][60425] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 167116800. Throughput: 0: 1668.3, 1: 1677.3. Samples: 41785614. Policy #0 lag: (min: 29.0, avg: 39.6, max: 61.0) [2023-10-14 21:02:53,344][60425] Avg episode reward: [(0, '78.040'), (1, '80.500')] [2023-10-14 21:02:53,951][61552] Updated weights for policy 0, policy_version 81672 (0.0008) [2023-10-14 21:02:54,316][61552] Updated weights for policy 0, policy_version 81682 (0.0010) [2023-10-14 21:02:54,681][61552] Updated weights for policy 0, policy_version 81692 (0.0009) [2023-10-14 21:02:56,887][61585] Updated weights for policy 1, policy_version 81540 (0.0009) [2023-10-14 21:02:57,255][61585] Updated weights for policy 1, policy_version 81550 (0.0008) [2023-10-14 21:02:57,626][61585] Updated weights for policy 1, policy_version 81560 (0.0008) [2023-10-14 21:02:58,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 167182336. Throughput: 0: 1668.7, 1: 1660.8. Samples: 41805244. Policy #0 lag: (min: 29.0, avg: 39.6, max: 61.0) [2023-10-14 21:02:58,345][60425] Avg episode reward: [(0, '76.270'), (1, '85.150')] [2023-10-14 21:02:58,938][61552] Updated weights for policy 0, policy_version 81702 (0.0009) [2023-10-14 21:02:59,303][61552] Updated weights for policy 0, policy_version 81712 (0.0007) [2023-10-14 21:02:59,671][61552] Updated weights for policy 0, policy_version 81722 (0.0011) [2023-10-14 21:03:01,630][61585] Updated weights for policy 1, policy_version 81570 (0.0009) [2023-10-14 21:03:01,989][61585] Updated weights for policy 1, policy_version 81580 (0.0011) [2023-10-14 21:03:02,357][61585] Updated weights for policy 1, policy_version 81590 (0.0010) [2023-10-14 21:03:02,721][61585] Updated weights for policy 1, policy_version 81600 (0.0011) [2023-10-14 21:03:03,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 167247872. Throughput: 0: 1665.6, 1: 1679.9. Samples: 41815232. Policy #0 lag: (min: 29.0, avg: 39.6, max: 61.0) [2023-10-14 21:03:03,344][60425] Avg episode reward: [(0, '76.310'), (1, '76.150')] [2023-10-14 21:03:03,838][61552] Updated weights for policy 0, policy_version 81732 (0.0010) [2023-10-14 21:03:04,203][61552] Updated weights for policy 0, policy_version 81742 (0.0007) [2023-10-14 21:03:04,566][61552] Updated weights for policy 0, policy_version 81752 (0.0010) [2023-10-14 21:03:07,014][61585] Updated weights for policy 1, policy_version 81610 (0.0009) [2023-10-14 21:03:07,388][61585] Updated weights for policy 1, policy_version 81620 (0.0010) [2023-10-14 21:03:07,753][61585] Updated weights for policy 1, policy_version 81630 (0.0010) [2023-10-14 21:03:08,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 167313408. Throughput: 0: 1668.0, 1: 1674.1. Samples: 41835574. Policy #0 lag: (min: 29.0, avg: 39.6, max: 61.0) [2023-10-14 21:03:08,344][60425] Avg episode reward: [(0, '77.660'), (1, '78.430')] [2023-10-14 21:03:08,675][61552] Updated weights for policy 0, policy_version 81762 (0.0008) [2023-10-14 21:03:09,048][61552] Updated weights for policy 0, policy_version 81772 (0.0009) [2023-10-14 21:03:09,407][61552] Updated weights for policy 0, policy_version 81782 (0.0008) [2023-10-14 21:03:09,771][61552] Updated weights for policy 0, policy_version 81792 (0.0009) [2023-10-14 21:03:11,846][61585] Updated weights for policy 1, policy_version 81640 (0.0010) [2023-10-14 21:03:12,207][61585] Updated weights for policy 1, policy_version 81650 (0.0008) [2023-10-14 21:03:12,571][61585] Updated weights for policy 1, policy_version 81660 (0.0009) [2023-10-14 21:03:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 167378944. Throughput: 0: 1669.8, 1: 1659.3. Samples: 41855046. Policy #0 lag: (min: 9.0, avg: 13.4, max: 37.0) [2023-10-14 21:03:13,344][60425] Avg episode reward: [(0, '81.600'), (1, '79.100')] [2023-10-14 21:03:13,707][61552] Updated weights for policy 0, policy_version 81802 (0.0011) [2023-10-14 21:03:14,072][61552] Updated weights for policy 0, policy_version 81812 (0.0008) [2023-10-14 21:03:14,434][61552] Updated weights for policy 0, policy_version 81822 (0.0007) [2023-10-14 21:03:16,569][61585] Updated weights for policy 1, policy_version 81670 (0.0010) [2023-10-14 21:03:16,938][61585] Updated weights for policy 1, policy_version 81680 (0.0011) [2023-10-14 21:03:17,314][61585] Updated weights for policy 1, policy_version 81690 (0.0007) [2023-10-14 21:03:18,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 167444480. Throughput: 0: 1668.8, 1: 1688.3. Samples: 41865590. Policy #0 lag: (min: 9.0, avg: 13.4, max: 37.0) [2023-10-14 21:03:18,344][60425] Avg episode reward: [(0, '83.020'), (1, '84.130')] [2023-10-14 21:03:18,600][61552] Updated weights for policy 0, policy_version 81832 (0.0007) [2023-10-14 21:03:18,960][61552] Updated weights for policy 0, policy_version 81842 (0.0009) [2023-10-14 21:03:19,326][61552] Updated weights for policy 0, policy_version 81852 (0.0008) [2023-10-14 21:03:21,496][61585] Updated weights for policy 1, policy_version 81700 (0.0008) [2023-10-14 21:03:21,861][61585] Updated weights for policy 1, policy_version 81710 (0.0009) [2023-10-14 21:03:22,226][61585] Updated weights for policy 1, policy_version 81720 (0.0009) [2023-10-14 21:03:23,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 167510016. Throughput: 0: 1675.9, 1: 1671.0. Samples: 41885634. Policy #0 lag: (min: 9.0, avg: 13.4, max: 37.0) [2023-10-14 21:03:23,344][60425] Avg episode reward: [(0, '74.270'), (1, '76.210')] [2023-10-14 21:03:23,394][61552] Updated weights for policy 0, policy_version 81862 (0.0009) [2023-10-14 21:03:23,776][61552] Updated weights for policy 0, policy_version 81872 (0.0007) [2023-10-14 21:03:24,135][61552] Updated weights for policy 0, policy_version 81882 (0.0009) [2023-10-14 21:03:26,286][61585] Updated weights for policy 1, policy_version 81730 (0.0009) [2023-10-14 21:03:26,659][61585] Updated weights for policy 1, policy_version 81740 (0.0009) [2023-10-14 21:03:27,027][61585] Updated weights for policy 1, policy_version 81750 (0.0008) [2023-10-14 21:03:27,389][61585] Updated weights for policy 1, policy_version 81760 (0.0008) [2023-10-14 21:03:28,034][61552] Updated weights for policy 0, policy_version 81892 (0.0009) [2023-10-14 21:03:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 167575552. Throughput: 0: 1674.3, 1: 1664.7. Samples: 41905482. Policy #0 lag: (min: 9.0, avg: 13.4, max: 37.0) [2023-10-14 21:03:28,344][60425] Avg episode reward: [(0, '74.770'), (1, '79.300')] [2023-10-14 21:03:28,404][61552] Updated weights for policy 0, policy_version 81902 (0.0008) [2023-10-14 21:03:28,775][61552] Updated weights for policy 0, policy_version 81912 (0.0007) [2023-10-14 21:03:31,424][61585] Updated weights for policy 1, policy_version 81770 (0.0011) [2023-10-14 21:03:31,791][61585] Updated weights for policy 1, policy_version 81780 (0.0009) [2023-10-14 21:03:32,150][61585] Updated weights for policy 1, policy_version 81790 (0.0009) [2023-10-14 21:03:32,801][61552] Updated weights for policy 0, policy_version 81922 (0.0007) [2023-10-14 21:03:33,172][61552] Updated weights for policy 0, policy_version 81932 (0.0007) [2023-10-14 21:03:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 167641088. Throughput: 0: 1678.4, 1: 1678.4. Samples: 41915884. Policy #0 lag: (min: 9.0, avg: 13.4, max: 37.0) [2023-10-14 21:03:33,344][60425] Avg episode reward: [(0, '78.800'), (1, '81.590')] [2023-10-14 21:03:33,525][61552] Updated weights for policy 0, policy_version 81942 (0.0008) [2023-10-14 21:03:33,891][61552] Updated weights for policy 0, policy_version 81952 (0.0008) [2023-10-14 21:03:36,254][61585] Updated weights for policy 1, policy_version 81800 (0.0009) [2023-10-14 21:03:36,609][61585] Updated weights for policy 1, policy_version 81810 (0.0007) [2023-10-14 21:03:36,978][61585] Updated weights for policy 1, policy_version 81820 (0.0007) [2023-10-14 21:03:37,996][61552] Updated weights for policy 0, policy_version 81962 (0.0010) [2023-10-14 21:03:38,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 167706624. Throughput: 0: 1678.5, 1: 1657.2. Samples: 41935720. Policy #0 lag: (min: 9.0, avg: 13.4, max: 37.0) [2023-10-14 21:03:38,344][60425] Avg episode reward: [(0, '75.260'), (1, '75.050')] [2023-10-14 21:03:38,358][61552] Updated weights for policy 0, policy_version 81972 (0.0009) [2023-10-14 21:03:38,734][61552] Updated weights for policy 0, policy_version 81982 (0.0008) [2023-10-14 21:03:41,211][61585] Updated weights for policy 1, policy_version 81830 (0.0007) [2023-10-14 21:03:41,582][61585] Updated weights for policy 1, policy_version 81840 (0.0009) [2023-10-14 21:03:41,941][61585] Updated weights for policy 1, policy_version 81850 (0.0009) [2023-10-14 21:03:42,773][61552] Updated weights for policy 0, policy_version 81992 (0.0010) [2023-10-14 21:03:43,148][61552] Updated weights for policy 0, policy_version 82002 (0.0011) [2023-10-14 21:03:43,344][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 167772160. Throughput: 0: 1675.2, 1: 1669.7. Samples: 41955768. Policy #0 lag: (min: 9.0, avg: 13.4, max: 37.0) [2023-10-14 21:03:43,345][60425] Avg episode reward: [(0, '76.090'), (1, '72.720')] [2023-10-14 21:03:43,357][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000081856_83820544.pth... [2023-10-14 21:03:43,393][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000080288_82214912.pth [2023-10-14 21:03:43,516][61552] Updated weights for policy 0, policy_version 82012 (0.0010) [2023-10-14 21:03:43,657][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000082016_83984384.pth... [2023-10-14 21:03:43,686][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000080448_82378752.pth [2023-10-14 21:03:46,044][61585] Updated weights for policy 1, policy_version 81860 (0.0009) [2023-10-14 21:03:46,419][61585] Updated weights for policy 1, policy_version 81870 (0.0008) [2023-10-14 21:03:46,777][61585] Updated weights for policy 1, policy_version 81880 (0.0008) [2023-10-14 21:03:47,688][61552] Updated weights for policy 0, policy_version 82022 (0.0008) [2023-10-14 21:03:48,055][61552] Updated weights for policy 0, policy_version 82032 (0.0008) [2023-10-14 21:03:48,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 167837696. Throughput: 0: 1686.7, 1: 1674.2. Samples: 41966472. Policy #0 lag: (min: 9.0, avg: 13.4, max: 37.0) [2023-10-14 21:03:48,344][60425] Avg episode reward: [(0, '76.370'), (1, '77.040')] [2023-10-14 21:03:48,431][61552] Updated weights for policy 0, policy_version 82042 (0.0008) [2023-10-14 21:03:50,940][61585] Updated weights for policy 1, policy_version 81890 (0.0008) [2023-10-14 21:03:51,313][61585] Updated weights for policy 1, policy_version 81900 (0.0010) [2023-10-14 21:03:51,671][61585] Updated weights for policy 1, policy_version 81910 (0.0010) [2023-10-14 21:03:52,031][61585] Updated weights for policy 1, policy_version 81920 (0.0008) [2023-10-14 21:03:52,507][61552] Updated weights for policy 0, policy_version 82052 (0.0008) [2023-10-14 21:03:52,880][61552] Updated weights for policy 0, policy_version 82062 (0.0008) [2023-10-14 21:03:53,253][61552] Updated weights for policy 0, policy_version 82072 (0.0007) [2023-10-14 21:03:53,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 167903232. Throughput: 0: 1685.7, 1: 1656.8. Samples: 41985988. Policy #0 lag: (min: 9.0, avg: 13.4, max: 37.0) [2023-10-14 21:03:53,344][60425] Avg episode reward: [(0, '76.730'), (1, '78.610')] [2023-10-14 21:03:56,161][61585] Updated weights for policy 1, policy_version 81930 (0.0009) [2023-10-14 21:03:56,525][61585] Updated weights for policy 1, policy_version 81940 (0.0008) [2023-10-14 21:03:56,891][61585] Updated weights for policy 1, policy_version 81950 (0.0008) [2023-10-14 21:03:57,245][61552] Updated weights for policy 0, policy_version 82082 (0.0009) [2023-10-14 21:03:57,617][61552] Updated weights for policy 0, policy_version 82092 (0.0008) [2023-10-14 21:03:57,984][61552] Updated weights for policy 0, policy_version 82102 (0.0009) [2023-10-14 21:03:58,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 167968768. Throughput: 0: 1675.2, 1: 1668.3. Samples: 42005506. Policy #0 lag: (min: 9.0, avg: 13.4, max: 37.0) [2023-10-14 21:03:58,344][60425] Avg episode reward: [(0, '75.570'), (1, '73.940')] [2023-10-14 21:03:58,354][61552] Updated weights for policy 0, policy_version 82112 (0.0007) [2023-10-14 21:04:01,053][61585] Updated weights for policy 1, policy_version 81960 (0.0008) [2023-10-14 21:04:01,410][61585] Updated weights for policy 1, policy_version 81970 (0.0008) [2023-10-14 21:04:01,780][61585] Updated weights for policy 1, policy_version 81980 (0.0007) [2023-10-14 21:04:02,360][61552] Updated weights for policy 0, policy_version 82122 (0.0007) [2023-10-14 21:04:02,735][61552] Updated weights for policy 0, policy_version 82132 (0.0008) [2023-10-14 21:04:03,098][61552] Updated weights for policy 0, policy_version 82142 (0.0007) [2023-10-14 21:04:03,343][60425] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 168067072. Throughput: 0: 1686.2, 1: 1660.6. Samples: 42016196. Policy #0 lag: (min: 2.0, avg: 4.6, max: 34.0) [2023-10-14 21:04:03,344][60425] Avg episode reward: [(0, '74.070'), (1, '73.650')] [2023-10-14 21:04:05,737][61585] Updated weights for policy 1, policy_version 81990 (0.0008) [2023-10-14 21:04:06,099][61585] Updated weights for policy 1, policy_version 82000 (0.0007) [2023-10-14 21:04:06,461][61585] Updated weights for policy 1, policy_version 82010 (0.0010) [2023-10-14 21:04:07,068][61552] Updated weights for policy 0, policy_version 82152 (0.0009) [2023-10-14 21:04:07,432][61552] Updated weights for policy 0, policy_version 82162 (0.0008) [2023-10-14 21:04:07,812][61552] Updated weights for policy 0, policy_version 82172 (0.0008) [2023-10-14 21:04:08,343][60425] Fps is (10 sec: 16383.9, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 168132608. Throughput: 0: 1686.5, 1: 1647.8. Samples: 42035678. Policy #0 lag: (min: 2.0, avg: 4.6, max: 34.0) [2023-10-14 21:04:08,344][60425] Avg episode reward: [(0, '77.100'), (1, '77.910')] [2023-10-14 21:04:10,467][61585] Updated weights for policy 1, policy_version 82020 (0.0008) [2023-10-14 21:04:10,838][61585] Updated weights for policy 1, policy_version 82030 (0.0008) [2023-10-14 21:04:11,205][61585] Updated weights for policy 1, policy_version 82040 (0.0007) [2023-10-14 21:04:12,140][61552] Updated weights for policy 0, policy_version 82182 (0.0009) [2023-10-14 21:04:12,513][61552] Updated weights for policy 0, policy_version 82192 (0.0009) [2023-10-14 21:04:12,879][61552] Updated weights for policy 0, policy_version 82202 (0.0008) [2023-10-14 21:04:13,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 168198144. Throughput: 0: 1665.3, 1: 1675.1. Samples: 42055800. Policy #0 lag: (min: 2.0, avg: 4.6, max: 34.0) [2023-10-14 21:04:13,344][60425] Avg episode reward: [(0, '76.740'), (1, '74.260')] [2023-10-14 21:04:15,180][61585] Updated weights for policy 1, policy_version 82050 (0.0008) [2023-10-14 21:04:15,541][61585] Updated weights for policy 1, policy_version 82060 (0.0008) [2023-10-14 21:04:15,904][61585] Updated weights for policy 1, policy_version 82070 (0.0008) [2023-10-14 21:04:16,274][61585] Updated weights for policy 1, policy_version 82080 (0.0008) [2023-10-14 21:04:17,108][61552] Updated weights for policy 0, policy_version 82212 (0.0009) [2023-10-14 21:04:17,483][61552] Updated weights for policy 0, policy_version 82222 (0.0009) [2023-10-14 21:04:17,853][61552] Updated weights for policy 0, policy_version 82232 (0.0009) [2023-10-14 21:04:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 168263680. Throughput: 0: 1678.5, 1: 1655.4. Samples: 42065908. Policy #0 lag: (min: 2.0, avg: 4.6, max: 34.0) [2023-10-14 21:04:18,344][60425] Avg episode reward: [(0, '74.040'), (1, '73.610')] [2023-10-14 21:04:20,334][61585] Updated weights for policy 1, policy_version 82090 (0.0008) [2023-10-14 21:04:20,701][61585] Updated weights for policy 1, policy_version 82100 (0.0007) [2023-10-14 21:04:21,083][61585] Updated weights for policy 1, policy_version 82110 (0.0007) [2023-10-14 21:04:21,917][61552] Updated weights for policy 0, policy_version 82242 (0.0008) [2023-10-14 21:04:22,312][61552] Updated weights for policy 0, policy_version 82252 (0.0009) [2023-10-14 21:04:22,693][61552] Updated weights for policy 0, policy_version 82262 (0.0009) [2023-10-14 21:04:23,063][61552] Updated weights for policy 0, policy_version 82272 (0.0009) [2023-10-14 21:04:23,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 168329216. Throughput: 0: 1673.8, 1: 1665.2. Samples: 42085974. Policy #0 lag: (min: 2.0, avg: 4.6, max: 34.0) [2023-10-14 21:04:23,344][60425] Avg episode reward: [(0, '76.130'), (1, '77.980')] [2023-10-14 21:04:25,058][61585] Updated weights for policy 1, policy_version 82120 (0.0007) [2023-10-14 21:04:25,425][61585] Updated weights for policy 1, policy_version 82130 (0.0008) [2023-10-14 21:04:25,794][61585] Updated weights for policy 1, policy_version 82140 (0.0010) [2023-10-14 21:04:27,209][61552] Updated weights for policy 0, policy_version 82282 (0.0008) [2023-10-14 21:04:27,576][61552] Updated weights for policy 0, policy_version 82292 (0.0008) [2023-10-14 21:04:27,939][61552] Updated weights for policy 0, policy_version 82302 (0.0009) [2023-10-14 21:04:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 168394752. Throughput: 0: 1654.4, 1: 1682.0. Samples: 42105904. Policy #0 lag: (min: 2.0, avg: 4.6, max: 34.0) [2023-10-14 21:04:28,344][60425] Avg episode reward: [(0, '78.410'), (1, '76.060')] [2023-10-14 21:04:29,914][61585] Updated weights for policy 1, policy_version 82150 (0.0008) [2023-10-14 21:04:30,274][61585] Updated weights for policy 1, policy_version 82160 (0.0011) [2023-10-14 21:04:30,646][61585] Updated weights for policy 1, policy_version 82170 (0.0010) [2023-10-14 21:04:32,103][61552] Updated weights for policy 0, policy_version 82312 (0.0008) [2023-10-14 21:04:32,479][61552] Updated weights for policy 0, policy_version 82322 (0.0008) [2023-10-14 21:04:32,855][61552] Updated weights for policy 0, policy_version 82332 (0.0009) [2023-10-14 21:04:33,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 168460288. Throughput: 0: 1665.8, 1: 1659.5. Samples: 42116108. Policy #0 lag: (min: 2.0, avg: 4.6, max: 34.0) [2023-10-14 21:04:33,344][60425] Avg episode reward: [(0, '79.410'), (1, '79.640')] [2023-10-14 21:04:34,767][61585] Updated weights for policy 1, policy_version 82180 (0.0009) [2023-10-14 21:04:35,136][61585] Updated weights for policy 1, policy_version 82190 (0.0008) [2023-10-14 21:04:35,500][61585] Updated weights for policy 1, policy_version 82200 (0.0007) [2023-10-14 21:04:37,032][61552] Updated weights for policy 0, policy_version 82342 (0.0008) [2023-10-14 21:04:37,393][61552] Updated weights for policy 0, policy_version 82352 (0.0008) [2023-10-14 21:04:37,759][61552] Updated weights for policy 0, policy_version 82362 (0.0008) [2023-10-14 21:04:38,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 168525824. Throughput: 0: 1663.0, 1: 1672.4. Samples: 42136082. Policy #0 lag: (min: 2.0, avg: 4.6, max: 34.0) [2023-10-14 21:04:38,344][60425] Avg episode reward: [(0, '76.490'), (1, '76.900')] [2023-10-14 21:04:39,636][61585] Updated weights for policy 1, policy_version 82210 (0.0009) [2023-10-14 21:04:40,045][61585] Updated weights for policy 1, policy_version 82220 (0.0009) [2023-10-14 21:04:40,407][61585] Updated weights for policy 1, policy_version 82230 (0.0010) [2023-10-14 21:04:40,771][61585] Updated weights for policy 1, policy_version 82240 (0.0008) [2023-10-14 21:04:41,773][61552] Updated weights for policy 0, policy_version 82372 (0.0008) [2023-10-14 21:04:42,142][61552] Updated weights for policy 0, policy_version 82382 (0.0007) [2023-10-14 21:04:42,510][61552] Updated weights for policy 0, policy_version 82392 (0.0009) [2023-10-14 21:04:43,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 168591360. Throughput: 0: 1650.8, 1: 1682.8. Samples: 42155520. Policy #0 lag: (min: 2.0, avg: 4.6, max: 34.0) [2023-10-14 21:04:43,345][60425] Avg episode reward: [(0, '75.580'), (1, '80.570')] [2023-10-14 21:04:44,770][61585] Updated weights for policy 1, policy_version 82250 (0.0008) [2023-10-14 21:04:45,132][61585] Updated weights for policy 1, policy_version 82260 (0.0008) [2023-10-14 21:04:45,494][61585] Updated weights for policy 1, policy_version 82270 (0.0009) [2023-10-14 21:04:46,453][61552] Updated weights for policy 0, policy_version 82402 (0.0008) [2023-10-14 21:04:46,825][61552] Updated weights for policy 0, policy_version 82412 (0.0011) [2023-10-14 21:04:47,181][61552] Updated weights for policy 0, policy_version 82422 (0.0012) [2023-10-14 21:04:47,544][61552] Updated weights for policy 0, policy_version 82432 (0.0010) [2023-10-14 21:04:48,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 168656896. Throughput: 0: 1670.0, 1: 1655.8. Samples: 42165856. Policy #0 lag: (min: 2.0, avg: 4.6, max: 34.0) [2023-10-14 21:04:48,344][60425] Avg episode reward: [(0, '76.230'), (1, '80.630')] [2023-10-14 21:04:49,686][61585] Updated weights for policy 1, policy_version 82280 (0.0007) [2023-10-14 21:04:50,043][61585] Updated weights for policy 1, policy_version 82290 (0.0010) [2023-10-14 21:04:50,406][61585] Updated weights for policy 1, policy_version 82300 (0.0009) [2023-10-14 21:04:51,709][61552] Updated weights for policy 0, policy_version 82442 (0.0008) [2023-10-14 21:04:52,067][61552] Updated weights for policy 0, policy_version 82452 (0.0009) [2023-10-14 21:04:52,442][61552] Updated weights for policy 0, policy_version 82462 (0.0007) [2023-10-14 21:04:53,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 168722432. Throughput: 0: 1656.3, 1: 1681.2. Samples: 42185868. Policy #0 lag: (min: 2.0, avg: 4.6, max: 34.0) [2023-10-14 21:04:53,344][60425] Avg episode reward: [(0, '81.600'), (1, '74.600')] [2023-10-14 21:04:54,600][61585] Updated weights for policy 1, policy_version 82310 (0.0007) [2023-10-14 21:04:54,959][61585] Updated weights for policy 1, policy_version 82320 (0.0009) [2023-10-14 21:04:55,323][61585] Updated weights for policy 1, policy_version 82330 (0.0010) [2023-10-14 21:04:56,493][61552] Updated weights for policy 0, policy_version 82472 (0.0007) [2023-10-14 21:04:56,862][61552] Updated weights for policy 0, policy_version 82482 (0.0009) [2023-10-14 21:04:57,228][61552] Updated weights for policy 0, policy_version 82492 (0.0007) [2023-10-14 21:04:58,343][60425] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 168787968. Throughput: 0: 1658.4, 1: 1668.3. Samples: 42205504. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-14 21:04:58,344][60425] Avg episode reward: [(0, '76.860'), (1, '80.120')] [2023-10-14 21:04:59,579][61585] Updated weights for policy 1, policy_version 82340 (0.0011) [2023-10-14 21:04:59,939][61585] Updated weights for policy 1, policy_version 82350 (0.0009) [2023-10-14 21:05:00,307][61585] Updated weights for policy 1, policy_version 82360 (0.0008) [2023-10-14 21:05:01,426][61552] Updated weights for policy 0, policy_version 82502 (0.0009) [2023-10-14 21:05:01,793][61552] Updated weights for policy 0, policy_version 82512 (0.0010) [2023-10-14 21:05:02,163][61552] Updated weights for policy 0, policy_version 82522 (0.0009) [2023-10-14 21:05:03,344][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 168853504. Throughput: 0: 1671.1, 1: 1659.5. Samples: 42215786. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-14 21:05:03,345][60425] Avg episode reward: [(0, '77.250'), (1, '74.710')] [2023-10-14 21:05:04,463][61585] Updated weights for policy 1, policy_version 82370 (0.0008) [2023-10-14 21:05:04,827][61585] Updated weights for policy 1, policy_version 82380 (0.0009) [2023-10-14 21:05:05,196][61585] Updated weights for policy 1, policy_version 82390 (0.0007) [2023-10-14 21:05:05,554][61585] Updated weights for policy 1, policy_version 82400 (0.0007) [2023-10-14 21:05:06,168][61552] Updated weights for policy 0, policy_version 82532 (0.0007) [2023-10-14 21:05:06,522][61552] Updated weights for policy 0, policy_version 82542 (0.0008) [2023-10-14 21:05:06,887][61552] Updated weights for policy 0, policy_version 82552 (0.0009) [2023-10-14 21:05:08,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 168919040. Throughput: 0: 1656.5, 1: 1668.7. Samples: 42235608. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-14 21:05:08,345][60425] Avg episode reward: [(0, '79.550'), (1, '74.100')] [2023-10-14 21:05:09,612][61585] Updated weights for policy 1, policy_version 82410 (0.0010) [2023-10-14 21:05:09,972][61585] Updated weights for policy 1, policy_version 82420 (0.0010) [2023-10-14 21:05:10,344][61585] Updated weights for policy 1, policy_version 82430 (0.0008) [2023-10-14 21:05:11,059][61552] Updated weights for policy 0, policy_version 82562 (0.0008) [2023-10-14 21:05:11,431][61552] Updated weights for policy 0, policy_version 82572 (0.0010) [2023-10-14 21:05:11,798][61552] Updated weights for policy 0, policy_version 82582 (0.0010) [2023-10-14 21:05:12,170][61552] Updated weights for policy 0, policy_version 82592 (0.0007) [2023-10-14 21:05:13,344][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 168984576. Throughput: 0: 1669.8, 1: 1661.3. Samples: 42255804. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-14 21:05:13,345][60425] Avg episode reward: [(0, '75.090'), (1, '74.920')] [2023-10-14 21:05:14,554][61585] Updated weights for policy 1, policy_version 82440 (0.0011) [2023-10-14 21:05:14,915][61585] Updated weights for policy 1, policy_version 82450 (0.0010) [2023-10-14 21:05:15,277][61585] Updated weights for policy 1, policy_version 82460 (0.0010) [2023-10-14 21:05:16,114][61552] Updated weights for policy 0, policy_version 82602 (0.0008) [2023-10-14 21:05:16,480][61552] Updated weights for policy 0, policy_version 82612 (0.0010) [2023-10-14 21:05:16,862][61552] Updated weights for policy 0, policy_version 82622 (0.0010) [2023-10-14 21:05:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 169050112. Throughput: 0: 1678.4, 1: 1651.7. Samples: 42265966. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-14 21:05:18,344][60425] Avg episode reward: [(0, '76.560'), (1, '78.590')] [2023-10-14 21:05:19,532][61585] Updated weights for policy 1, policy_version 82470 (0.0010) [2023-10-14 21:05:19,890][61585] Updated weights for policy 1, policy_version 82480 (0.0009) [2023-10-14 21:05:20,259][61585] Updated weights for policy 1, policy_version 82490 (0.0007) [2023-10-14 21:05:20,916][61552] Updated weights for policy 0, policy_version 82632 (0.0007) [2023-10-14 21:05:21,276][61552] Updated weights for policy 0, policy_version 82642 (0.0010) [2023-10-14 21:05:21,650][61552] Updated weights for policy 0, policy_version 82652 (0.0008) [2023-10-14 21:05:23,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 169115648. Throughput: 0: 1660.6, 1: 1660.6. Samples: 42285536. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-14 21:05:23,344][60425] Avg episode reward: [(0, '79.640'), (1, '75.360')] [2023-10-14 21:05:24,389][61585] Updated weights for policy 1, policy_version 82500 (0.0007) [2023-10-14 21:05:24,786][61585] Updated weights for policy 1, policy_version 82510 (0.0010) [2023-10-14 21:05:25,157][61585] Updated weights for policy 1, policy_version 82520 (0.0010) [2023-10-14 21:05:25,611][61552] Updated weights for policy 0, policy_version 82662 (0.0008) [2023-10-14 21:05:25,977][61552] Updated weights for policy 0, policy_version 82672 (0.0007) [2023-10-14 21:05:26,344][61552] Updated weights for policy 0, policy_version 82682 (0.0009) [2023-10-14 21:05:28,344][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 169181184. Throughput: 0: 1686.5, 1: 1658.8. Samples: 42306058. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-14 21:05:28,345][60425] Avg episode reward: [(0, '78.100'), (1, '77.680')] [2023-10-14 21:05:29,296][61585] Updated weights for policy 1, policy_version 82530 (0.0010) [2023-10-14 21:05:29,659][61585] Updated weights for policy 1, policy_version 82540 (0.0010) [2023-10-14 21:05:30,016][61585] Updated weights for policy 1, policy_version 82550 (0.0008) [2023-10-14 21:05:30,382][61585] Updated weights for policy 1, policy_version 82560 (0.0009) [2023-10-14 21:05:30,400][61552] Updated weights for policy 0, policy_version 82692 (0.0009) [2023-10-14 21:05:30,766][61552] Updated weights for policy 0, policy_version 82702 (0.0007) [2023-10-14 21:05:31,134][61552] Updated weights for policy 0, policy_version 82712 (0.0007) [2023-10-14 21:05:33,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 169246720. Throughput: 0: 1674.5, 1: 1658.3. Samples: 42315830. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-14 21:05:33,344][60425] Avg episode reward: [(0, '71.600'), (1, '74.890')] [2023-10-14 21:05:34,631][61585] Updated weights for policy 1, policy_version 82570 (0.0007) [2023-10-14 21:05:34,998][61585] Updated weights for policy 1, policy_version 82580 (0.0008) [2023-10-14 21:05:35,302][61552] Updated weights for policy 0, policy_version 82722 (0.0007) [2023-10-14 21:05:35,358][61585] Updated weights for policy 1, policy_version 82590 (0.0009) [2023-10-14 21:05:35,660][61552] Updated weights for policy 0, policy_version 82732 (0.0008) [2023-10-14 21:05:36,024][61552] Updated weights for policy 0, policy_version 82742 (0.0008) [2023-10-14 21:05:36,389][61552] Updated weights for policy 0, policy_version 82752 (0.0008) [2023-10-14 21:05:38,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 169312256. Throughput: 0: 1666.4, 1: 1656.9. Samples: 42335416. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-14 21:05:38,344][60425] Avg episode reward: [(0, '77.060'), (1, '79.320')] [2023-10-14 21:05:39,309][61585] Updated weights for policy 1, policy_version 82600 (0.0011) [2023-10-14 21:05:39,670][61585] Updated weights for policy 1, policy_version 82610 (0.0009) [2023-10-14 21:05:40,044][61585] Updated weights for policy 1, policy_version 82620 (0.0009) [2023-10-14 21:05:40,319][61552] Updated weights for policy 0, policy_version 82762 (0.0008) [2023-10-14 21:05:40,684][61552] Updated weights for policy 0, policy_version 82772 (0.0011) [2023-10-14 21:05:41,050][61552] Updated weights for policy 0, policy_version 82782 (0.0009) [2023-10-14 21:05:43,344][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 169377792. Throughput: 0: 1684.4, 1: 1668.6. Samples: 42356386. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-14 21:05:43,345][60425] Avg episode reward: [(0, '79.710'), (1, '81.720')] [2023-10-14 21:05:43,355][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000082624_84606976.pth... [2023-10-14 21:05:43,355][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000082784_84770816.pth... [2023-10-14 21:05:43,386][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000081216_83165184.pth [2023-10-14 21:05:43,396][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000081088_83034112.pth [2023-10-14 21:05:44,043][61585] Updated weights for policy 1, policy_version 82630 (0.0009) [2023-10-14 21:05:44,405][61585] Updated weights for policy 1, policy_version 82640 (0.0008) [2023-10-14 21:05:44,784][61585] Updated weights for policy 1, policy_version 82650 (0.0008) [2023-10-14 21:05:45,277][61552] Updated weights for policy 0, policy_version 82792 (0.0008) [2023-10-14 21:05:45,644][61552] Updated weights for policy 0, policy_version 82802 (0.0009) [2023-10-14 21:05:46,010][61552] Updated weights for policy 0, policy_version 82812 (0.0009) [2023-10-14 21:05:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 169443328. Throughput: 0: 1668.7, 1: 1670.8. Samples: 42366062. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-14 21:05:48,344][60425] Avg episode reward: [(0, '74.520'), (1, '74.340')] [2023-10-14 21:05:48,938][61585] Updated weights for policy 1, policy_version 82660 (0.0010) [2023-10-14 21:05:49,304][61585] Updated weights for policy 1, policy_version 82670 (0.0009) [2023-10-14 21:05:49,667][61585] Updated weights for policy 1, policy_version 82680 (0.0009) [2023-10-14 21:05:50,087][61552] Updated weights for policy 0, policy_version 82822 (0.0009) [2023-10-14 21:05:50,456][61552] Updated weights for policy 0, policy_version 82832 (0.0008) [2023-10-14 21:05:50,823][61552] Updated weights for policy 0, policy_version 82842 (0.0007) [2023-10-14 21:05:53,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 169508864. Throughput: 0: 1673.6, 1: 1670.4. Samples: 42386086. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:05:53,344][60425] Avg episode reward: [(0, '73.930'), (1, '78.490')] [2023-10-14 21:05:53,617][61585] Updated weights for policy 1, policy_version 82690 (0.0009) [2023-10-14 21:05:53,975][61585] Updated weights for policy 1, policy_version 82700 (0.0007) [2023-10-14 21:05:54,341][61585] Updated weights for policy 1, policy_version 82710 (0.0007) [2023-10-14 21:05:54,705][61585] Updated weights for policy 1, policy_version 82720 (0.0008) [2023-10-14 21:05:54,941][61552] Updated weights for policy 0, policy_version 82852 (0.0008) [2023-10-14 21:05:55,315][61552] Updated weights for policy 0, policy_version 82862 (0.0007) [2023-10-14 21:05:55,684][61552] Updated weights for policy 0, policy_version 82872 (0.0009) [2023-10-14 21:05:58,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 169574400. Throughput: 0: 1681.0, 1: 1673.7. Samples: 42406764. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:05:58,344][60425] Avg episode reward: [(0, '78.770'), (1, '80.640')] [2023-10-14 21:05:58,818][61585] Updated weights for policy 1, policy_version 82730 (0.0011) [2023-10-14 21:05:59,185][61585] Updated weights for policy 1, policy_version 82740 (0.0010) [2023-10-14 21:05:59,554][61585] Updated weights for policy 1, policy_version 82750 (0.0009) [2023-10-14 21:05:59,964][61552] Updated weights for policy 0, policy_version 82882 (0.0008) [2023-10-14 21:06:00,367][61552] Updated weights for policy 0, policy_version 82892 (0.0009) [2023-10-14 21:06:00,736][61552] Updated weights for policy 0, policy_version 82902 (0.0008) [2023-10-14 21:06:01,107][61552] Updated weights for policy 0, policy_version 82912 (0.0008) [2023-10-14 21:06:03,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 169639936. Throughput: 0: 1666.3, 1: 1675.1. Samples: 42416328. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:06:03,344][60425] Avg episode reward: [(0, '71.490'), (1, '79.830')] [2023-10-14 21:06:03,658][61585] Updated weights for policy 1, policy_version 82760 (0.0007) [2023-10-14 21:06:04,033][61585] Updated weights for policy 1, policy_version 82770 (0.0007) [2023-10-14 21:06:04,393][61585] Updated weights for policy 1, policy_version 82780 (0.0007) [2023-10-14 21:06:05,095][61552] Updated weights for policy 0, policy_version 82922 (0.0008) [2023-10-14 21:06:05,458][61552] Updated weights for policy 0, policy_version 82932 (0.0007) [2023-10-14 21:06:05,828][61552] Updated weights for policy 0, policy_version 82942 (0.0009) [2023-10-14 21:06:08,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 169705472. Throughput: 0: 1674.4, 1: 1679.5. Samples: 42436464. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:06:08,345][60425] Avg episode reward: [(0, '72.730'), (1, '78.180')] [2023-10-14 21:06:08,417][61585] Updated weights for policy 1, policy_version 82790 (0.0008) [2023-10-14 21:06:08,783][61585] Updated weights for policy 1, policy_version 82800 (0.0009) [2023-10-14 21:06:09,159][61585] Updated weights for policy 1, policy_version 82810 (0.0007) [2023-10-14 21:06:09,801][61552] Updated weights for policy 0, policy_version 82952 (0.0009) [2023-10-14 21:06:10,171][61552] Updated weights for policy 0, policy_version 82962 (0.0008) [2023-10-14 21:06:10,546][61552] Updated weights for policy 0, policy_version 82972 (0.0011) [2023-10-14 21:06:13,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 169771008. Throughput: 0: 1672.5, 1: 1681.5. Samples: 42456986. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:06:13,345][60425] Avg episode reward: [(0, '76.340'), (1, '77.850')] [2023-10-14 21:06:13,423][61585] Updated weights for policy 1, policy_version 82820 (0.0009) [2023-10-14 21:06:13,836][61585] Updated weights for policy 1, policy_version 82830 (0.0007) [2023-10-14 21:06:14,193][61585] Updated weights for policy 1, policy_version 82840 (0.0008) [2023-10-14 21:06:14,649][61552] Updated weights for policy 0, policy_version 82982 (0.0008) [2023-10-14 21:06:15,015][61552] Updated weights for policy 0, policy_version 82992 (0.0008) [2023-10-14 21:06:15,385][61552] Updated weights for policy 0, policy_version 83002 (0.0007) [2023-10-14 21:06:18,187][61585] Updated weights for policy 1, policy_version 82850 (0.0009) [2023-10-14 21:06:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 169836544. Throughput: 0: 1657.5, 1: 1680.9. Samples: 42466058. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:06:18,344][60425] Avg episode reward: [(0, '76.930'), (1, '76.850')] [2023-10-14 21:06:18,550][61585] Updated weights for policy 1, policy_version 82860 (0.0008) [2023-10-14 21:06:18,919][61585] Updated weights for policy 1, policy_version 82870 (0.0008) [2023-10-14 21:06:19,282][61585] Updated weights for policy 1, policy_version 82880 (0.0008) [2023-10-14 21:06:19,518][61552] Updated weights for policy 0, policy_version 83012 (0.0007) [2023-10-14 21:06:19,883][61552] Updated weights for policy 0, policy_version 83022 (0.0007) [2023-10-14 21:06:20,259][61552] Updated weights for policy 0, policy_version 83032 (0.0011) [2023-10-14 21:06:23,344][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 169902080. Throughput: 0: 1675.8, 1: 1684.9. Samples: 42486650. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:06:23,345][60425] Avg episode reward: [(0, '79.010'), (1, '75.400')] [2023-10-14 21:06:23,442][61585] Updated weights for policy 1, policy_version 82890 (0.0007) [2023-10-14 21:06:23,818][61585] Updated weights for policy 1, policy_version 82900 (0.0008) [2023-10-14 21:06:24,187][61585] Updated weights for policy 1, policy_version 82910 (0.0009) [2023-10-14 21:06:24,318][61552] Updated weights for policy 0, policy_version 83042 (0.0011) [2023-10-14 21:06:24,692][61552] Updated weights for policy 0, policy_version 83052 (0.0009) [2023-10-14 21:06:25,067][61552] Updated weights for policy 0, policy_version 83062 (0.0009) [2023-10-14 21:06:25,434][61552] Updated weights for policy 0, policy_version 83072 (0.0010) [2023-10-14 21:06:28,344][61585] Updated weights for policy 1, policy_version 82920 (0.0009) [2023-10-14 21:06:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 169967616. Throughput: 0: 1671.1, 1: 1674.7. Samples: 42506946. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:06:28,345][60425] Avg episode reward: [(0, '80.490'), (1, '75.350')] [2023-10-14 21:06:28,697][61585] Updated weights for policy 1, policy_version 82930 (0.0008) [2023-10-14 21:06:29,063][61585] Updated weights for policy 1, policy_version 82940 (0.0007) [2023-10-14 21:06:29,457][61552] Updated weights for policy 0, policy_version 83082 (0.0010) [2023-10-14 21:06:29,834][61552] Updated weights for policy 0, policy_version 83092 (0.0009) [2023-10-14 21:06:30,186][61552] Updated weights for policy 0, policy_version 83102 (0.0010) [2023-10-14 21:06:33,030][61585] Updated weights for policy 1, policy_version 82950 (0.0008) [2023-10-14 21:06:33,343][60425] Fps is (10 sec: 13107.7, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 170033152. Throughput: 0: 1660.7, 1: 1673.0. Samples: 42516080. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:06:33,344][60425] Avg episode reward: [(0, '80.720'), (1, '76.200')] [2023-10-14 21:06:33,392][61585] Updated weights for policy 1, policy_version 82960 (0.0008) [2023-10-14 21:06:33,757][61585] Updated weights for policy 1, policy_version 82970 (0.0008) [2023-10-14 21:06:34,169][61552] Updated weights for policy 0, policy_version 83112 (0.0008) [2023-10-14 21:06:34,544][61552] Updated weights for policy 0, policy_version 83122 (0.0008) [2023-10-14 21:06:34,912][61552] Updated weights for policy 0, policy_version 83132 (0.0007) [2023-10-14 21:06:37,783][61585] Updated weights for policy 1, policy_version 82980 (0.0008) [2023-10-14 21:06:38,148][61585] Updated weights for policy 1, policy_version 82990 (0.0008) [2023-10-14 21:06:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 170098688. Throughput: 0: 1676.5, 1: 1678.6. Samples: 42537066. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:06:38,344][60425] Avg episode reward: [(0, '78.410'), (1, '76.750')] [2023-10-14 21:06:38,517][61585] Updated weights for policy 1, policy_version 83000 (0.0009) [2023-10-14 21:06:39,007][61552] Updated weights for policy 0, policy_version 83142 (0.0009) [2023-10-14 21:06:39,376][61552] Updated weights for policy 0, policy_version 83152 (0.0011) [2023-10-14 21:06:39,750][61552] Updated weights for policy 0, policy_version 83162 (0.0009) [2023-10-14 21:06:42,562][61585] Updated weights for policy 1, policy_version 83010 (0.0009) [2023-10-14 21:06:42,934][61585] Updated weights for policy 1, policy_version 83020 (0.0009) [2023-10-14 21:06:43,296][61585] Updated weights for policy 1, policy_version 83030 (0.0009) [2023-10-14 21:06:43,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 170164224. Throughput: 0: 1675.8, 1: 1670.3. Samples: 42557336. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:06:43,344][60425] Avg episode reward: [(0, '88.540'), (1, '79.360')] [2023-10-14 21:06:43,351][61172] Saving new best policy, reward=88.540! [2023-10-14 21:06:43,670][61585] Updated weights for policy 1, policy_version 83040 (0.0009) [2023-10-14 21:06:44,005][61552] Updated weights for policy 0, policy_version 83172 (0.0009) [2023-10-14 21:06:44,385][61552] Updated weights for policy 0, policy_version 83182 (0.0008) [2023-10-14 21:06:44,759][61552] Updated weights for policy 0, policy_version 83192 (0.0008) [2023-10-14 21:06:47,735][61585] Updated weights for policy 1, policy_version 83050 (0.0008) [2023-10-14 21:06:48,100][61585] Updated weights for policy 1, policy_version 83060 (0.0008) [2023-10-14 21:06:48,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 170229760. Throughput: 0: 1660.9, 1: 1676.9. Samples: 42566532. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-14 21:06:48,344][60425] Avg episode reward: [(0, '79.360'), (1, '76.490')] [2023-10-14 21:06:48,459][61585] Updated weights for policy 1, policy_version 83070 (0.0009) [2023-10-14 21:06:48,887][61552] Updated weights for policy 0, policy_version 83202 (0.0010) [2023-10-14 21:06:49,255][61552] Updated weights for policy 0, policy_version 83212 (0.0008) [2023-10-14 21:06:49,616][61552] Updated weights for policy 0, policy_version 83222 (0.0007) [2023-10-14 21:06:49,983][61552] Updated weights for policy 0, policy_version 83232 (0.0008) [2023-10-14 21:06:52,592][61585] Updated weights for policy 1, policy_version 83080 (0.0008) [2023-10-14 21:06:52,949][61585] Updated weights for policy 1, policy_version 83090 (0.0009) [2023-10-14 21:06:53,319][61585] Updated weights for policy 1, policy_version 83100 (0.0008) [2023-10-14 21:06:53,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 170295296. Throughput: 0: 1673.3, 1: 1676.9. Samples: 42587222. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-14 21:06:53,344][60425] Avg episode reward: [(0, '81.410'), (1, '77.320')] [2023-10-14 21:06:53,997][61552] Updated weights for policy 0, policy_version 83242 (0.0008) [2023-10-14 21:06:54,378][61552] Updated weights for policy 0, policy_version 83252 (0.0008) [2023-10-14 21:06:54,737][61552] Updated weights for policy 0, policy_version 83262 (0.0008) [2023-10-14 21:06:57,301][61585] Updated weights for policy 1, policy_version 83110 (0.0007) [2023-10-14 21:06:57,664][61585] Updated weights for policy 1, policy_version 83120 (0.0007) [2023-10-14 21:06:58,032][61585] Updated weights for policy 1, policy_version 83130 (0.0010) [2023-10-14 21:06:58,343][60425] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 170393600. Throughput: 0: 1674.0, 1: 1668.3. Samples: 42607390. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-14 21:06:58,344][60425] Avg episode reward: [(0, '77.690'), (1, '77.160')] [2023-10-14 21:06:58,859][61552] Updated weights for policy 0, policy_version 83272 (0.0008) [2023-10-14 21:06:59,232][61552] Updated weights for policy 0, policy_version 83282 (0.0008) [2023-10-14 21:06:59,592][61552] Updated weights for policy 0, policy_version 83292 (0.0008) [2023-10-14 21:07:02,195][61585] Updated weights for policy 1, policy_version 83140 (0.0009) [2023-10-14 21:07:02,592][61585] Updated weights for policy 1, policy_version 83150 (0.0009) [2023-10-14 21:07:02,955][61585] Updated weights for policy 1, policy_version 83160 (0.0009) [2023-10-14 21:07:03,343][60425] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 170459136. Throughput: 0: 1671.1, 1: 1687.2. Samples: 42617180. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-14 21:07:03,344][60425] Avg episode reward: [(0, '79.200'), (1, '75.570')] [2023-10-14 21:07:03,684][61552] Updated weights for policy 0, policy_version 83302 (0.0008) [2023-10-14 21:07:04,045][61552] Updated weights for policy 0, policy_version 83312 (0.0011) [2023-10-14 21:07:04,410][61552] Updated weights for policy 0, policy_version 83322 (0.0010) [2023-10-14 21:07:07,054][61585] Updated weights for policy 1, policy_version 83170 (0.0010) [2023-10-14 21:07:07,422][61585] Updated weights for policy 1, policy_version 83180 (0.0009) [2023-10-14 21:07:07,789][61585] Updated weights for policy 1, policy_version 83190 (0.0010) [2023-10-14 21:07:08,152][61585] Updated weights for policy 1, policy_version 83200 (0.0008) [2023-10-14 21:07:08,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 170524672. Throughput: 0: 1677.4, 1: 1679.5. Samples: 42637710. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-14 21:07:08,344][60425] Avg episode reward: [(0, '74.800'), (1, '77.560')] [2023-10-14 21:07:08,443][61552] Updated weights for policy 0, policy_version 83332 (0.0008) [2023-10-14 21:07:08,805][61552] Updated weights for policy 0, policy_version 83342 (0.0008) [2023-10-14 21:07:09,176][61552] Updated weights for policy 0, policy_version 83352 (0.0007) [2023-10-14 21:07:12,259][61585] Updated weights for policy 1, policy_version 83210 (0.0007) [2023-10-14 21:07:12,638][61585] Updated weights for policy 1, policy_version 83220 (0.0007) [2023-10-14 21:07:13,006][61585] Updated weights for policy 1, policy_version 83230 (0.0007) [2023-10-14 21:07:13,150][61552] Updated weights for policy 0, policy_version 83362 (0.0008) [2023-10-14 21:07:13,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 170590208. Throughput: 0: 1680.5, 1: 1662.8. Samples: 42657394. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-14 21:07:13,344][60425] Avg episode reward: [(0, '77.640'), (1, '73.590')] [2023-10-14 21:07:13,520][61552] Updated weights for policy 0, policy_version 83372 (0.0009) [2023-10-14 21:07:13,882][61552] Updated weights for policy 0, policy_version 83382 (0.0007) [2023-10-14 21:07:14,251][61552] Updated weights for policy 0, policy_version 83392 (0.0007) [2023-10-14 21:07:17,157][61585] Updated weights for policy 1, policy_version 83240 (0.0008) [2023-10-14 21:07:17,511][61585] Updated weights for policy 1, policy_version 83250 (0.0010) [2023-10-14 21:07:17,875][61585] Updated weights for policy 1, policy_version 83260 (0.0011) [2023-10-14 21:07:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 170655744. Throughput: 0: 1678.4, 1: 1681.8. Samples: 42667290. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-14 21:07:18,344][60425] Avg episode reward: [(0, '76.330'), (1, '73.800')] [2023-10-14 21:07:18,525][61552] Updated weights for policy 0, policy_version 83402 (0.0009) [2023-10-14 21:07:18,893][61552] Updated weights for policy 0, policy_version 83412 (0.0007) [2023-10-14 21:07:19,263][61552] Updated weights for policy 0, policy_version 83422 (0.0008) [2023-10-14 21:07:21,913][61585] Updated weights for policy 1, policy_version 83270 (0.0008) [2023-10-14 21:07:22,276][61585] Updated weights for policy 1, policy_version 83280 (0.0010) [2023-10-14 21:07:22,654][61585] Updated weights for policy 1, policy_version 83290 (0.0011) [2023-10-14 21:07:23,172][61552] Updated weights for policy 0, policy_version 83432 (0.0007) [2023-10-14 21:07:23,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 170721280. Throughput: 0: 1679.4, 1: 1675.3. Samples: 42688026. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-14 21:07:23,344][60425] Avg episode reward: [(0, '81.540'), (1, '75.660')] [2023-10-14 21:07:23,540][61552] Updated weights for policy 0, policy_version 83442 (0.0007) [2023-10-14 21:07:23,906][61552] Updated weights for policy 0, policy_version 83452 (0.0009) [2023-10-14 21:07:26,758][61585] Updated weights for policy 1, policy_version 83300 (0.0008) [2023-10-14 21:07:27,124][61585] Updated weights for policy 1, policy_version 83310 (0.0009) [2023-10-14 21:07:27,496][61585] Updated weights for policy 1, policy_version 83320 (0.0010) [2023-10-14 21:07:28,087][61552] Updated weights for policy 0, policy_version 83462 (0.0010) [2023-10-14 21:07:28,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 170786816. Throughput: 0: 1686.3, 1: 1654.7. Samples: 42707682. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-14 21:07:28,344][60425] Avg episode reward: [(0, '76.700'), (1, '78.890')] [2023-10-14 21:07:28,464][61552] Updated weights for policy 0, policy_version 83472 (0.0011) [2023-10-14 21:07:28,838][61552] Updated weights for policy 0, policy_version 83482 (0.0009) [2023-10-14 21:07:31,551][61585] Updated weights for policy 1, policy_version 83330 (0.0008) [2023-10-14 21:07:31,917][61585] Updated weights for policy 1, policy_version 83340 (0.0007) [2023-10-14 21:07:32,276][61585] Updated weights for policy 1, policy_version 83350 (0.0007) [2023-10-14 21:07:32,642][61585] Updated weights for policy 1, policy_version 83360 (0.0010) [2023-10-14 21:07:32,926][61552] Updated weights for policy 0, policy_version 83492 (0.0008) [2023-10-14 21:07:33,296][61552] Updated weights for policy 0, policy_version 83502 (0.0011) [2023-10-14 21:07:33,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 170852352. Throughput: 0: 1683.9, 1: 1679.0. Samples: 42717862. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-14 21:07:33,344][60425] Avg episode reward: [(0, '81.810'), (1, '79.080')] [2023-10-14 21:07:33,662][61552] Updated weights for policy 0, policy_version 83512 (0.0009) [2023-10-14 21:07:36,735][61585] Updated weights for policy 1, policy_version 83370 (0.0010) [2023-10-14 21:07:37,093][61585] Updated weights for policy 1, policy_version 83380 (0.0012) [2023-10-14 21:07:37,463][61585] Updated weights for policy 1, policy_version 83390 (0.0008) [2023-10-14 21:07:37,954][61552] Updated weights for policy 0, policy_version 83522 (0.0009) [2023-10-14 21:07:38,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 170917888. Throughput: 0: 1687.6, 1: 1665.2. Samples: 42738096. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-14 21:07:38,344][60425] Avg episode reward: [(0, '78.960'), (1, '79.670')] [2023-10-14 21:07:38,354][61552] Updated weights for policy 0, policy_version 83532 (0.0009) [2023-10-14 21:07:38,719][61552] Updated weights for policy 0, policy_version 83542 (0.0008) [2023-10-14 21:07:39,091][61552] Updated weights for policy 0, policy_version 83552 (0.0007) [2023-10-14 21:07:41,615][61585] Updated weights for policy 1, policy_version 83400 (0.0008) [2023-10-14 21:07:41,979][61585] Updated weights for policy 1, policy_version 83410 (0.0007) [2023-10-14 21:07:42,345][61585] Updated weights for policy 1, policy_version 83420 (0.0010) [2023-10-14 21:07:43,118][61552] Updated weights for policy 0, policy_version 83562 (0.0009) [2023-10-14 21:07:43,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 170983424. Throughput: 0: 1683.5, 1: 1658.2. Samples: 42757764. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-14 21:07:43,344][60425] Avg episode reward: [(0, '80.520'), (1, '78.090')] [2023-10-14 21:07:43,354][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000083424_85426176.pth... [2023-10-14 21:07:43,391][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000081856_83820544.pth [2023-10-14 21:07:43,494][61552] Updated weights for policy 0, policy_version 83572 (0.0007) [2023-10-14 21:07:43,868][61552] Updated weights for policy 0, policy_version 83582 (0.0007) [2023-10-14 21:07:43,936][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000083584_85590016.pth... [2023-10-14 21:07:43,971][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000082016_83984384.pth [2023-10-14 21:07:46,390][61585] Updated weights for policy 1, policy_version 83430 (0.0009) [2023-10-14 21:07:46,756][61585] Updated weights for policy 1, policy_version 83440 (0.0009) [2023-10-14 21:07:47,121][61585] Updated weights for policy 1, policy_version 83450 (0.0010) [2023-10-14 21:07:47,756][61552] Updated weights for policy 0, policy_version 83592 (0.0009) [2023-10-14 21:07:48,121][61552] Updated weights for policy 0, policy_version 83602 (0.0008) [2023-10-14 21:07:48,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 171048960. Throughput: 0: 1686.4, 1: 1670.8. Samples: 42768256. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-14 21:07:48,344][60425] Avg episode reward: [(0, '81.220'), (1, '78.930')] [2023-10-14 21:07:48,491][61552] Updated weights for policy 0, policy_version 83612 (0.0011) [2023-10-14 21:07:51,237][61585] Updated weights for policy 1, policy_version 83460 (0.0009) [2023-10-14 21:07:51,644][61585] Updated weights for policy 1, policy_version 83470 (0.0007) [2023-10-14 21:07:52,013][61585] Updated weights for policy 1, policy_version 83480 (0.0007) [2023-10-14 21:07:52,738][61552] Updated weights for policy 0, policy_version 83622 (0.0010) [2023-10-14 21:07:53,101][61552] Updated weights for policy 0, policy_version 83632 (0.0011) [2023-10-14 21:07:53,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 171114496. Throughput: 0: 1682.6, 1: 1661.1. Samples: 42788174. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-14 21:07:53,344][60425] Avg episode reward: [(0, '83.320'), (1, '76.320')] [2023-10-14 21:07:53,477][61552] Updated weights for policy 0, policy_version 83642 (0.0008) [2023-10-14 21:07:56,145][61585] Updated weights for policy 1, policy_version 83490 (0.0008) [2023-10-14 21:07:56,516][61585] Updated weights for policy 1, policy_version 83500 (0.0008) [2023-10-14 21:07:56,878][61585] Updated weights for policy 1, policy_version 83510 (0.0008) [2023-10-14 21:07:57,240][61585] Updated weights for policy 1, policy_version 83520 (0.0008) [2023-10-14 21:07:57,626][61552] Updated weights for policy 0, policy_version 83652 (0.0011) [2023-10-14 21:07:57,989][61552] Updated weights for policy 0, policy_version 83662 (0.0008) [2023-10-14 21:07:58,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 171180032. Throughput: 0: 1672.8, 1: 1671.2. Samples: 42807874. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-14 21:07:58,344][60425] Avg episode reward: [(0, '78.970'), (1, '79.060')] [2023-10-14 21:07:58,354][61552] Updated weights for policy 0, policy_version 83672 (0.0007) [2023-10-14 21:08:01,302][61585] Updated weights for policy 1, policy_version 83530 (0.0008) [2023-10-14 21:08:01,665][61585] Updated weights for policy 1, policy_version 83540 (0.0011) [2023-10-14 21:08:02,042][61585] Updated weights for policy 1, policy_version 83550 (0.0009) [2023-10-14 21:08:02,143][61552] Updated weights for policy 0, policy_version 83682 (0.0007) [2023-10-14 21:08:02,511][61552] Updated weights for policy 0, policy_version 83692 (0.0007) [2023-10-14 21:08:02,882][61552] Updated weights for policy 0, policy_version 83702 (0.0008) [2023-10-14 21:08:03,241][61552] Updated weights for policy 0, policy_version 83712 (0.0008) [2023-10-14 21:08:03,345][60425] Fps is (10 sec: 16380.9, 60 sec: 13652.9, 300 sec: 13440.4). Total num frames: 171278336. Throughput: 0: 1680.5, 1: 1679.5. Samples: 42818494. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-14 21:08:03,346][60425] Avg episode reward: [(0, '79.590'), (1, '83.450')] [2023-10-14 21:08:06,021][61585] Updated weights for policy 1, policy_version 83560 (0.0010) [2023-10-14 21:08:06,392][61585] Updated weights for policy 1, policy_version 83570 (0.0010) [2023-10-14 21:08:06,754][61585] Updated weights for policy 1, policy_version 83580 (0.0009) [2023-10-14 21:08:07,407][61552] Updated weights for policy 0, policy_version 83722 (0.0008) [2023-10-14 21:08:07,787][61552] Updated weights for policy 0, policy_version 83732 (0.0009) [2023-10-14 21:08:08,156][61552] Updated weights for policy 0, policy_version 83742 (0.0008) [2023-10-14 21:08:08,343][60425] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 171343872. Throughput: 0: 1678.4, 1: 1659.3. Samples: 42838222. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-14 21:08:08,344][60425] Avg episode reward: [(0, '81.450'), (1, '81.620')] [2023-10-14 21:08:10,629][61585] Updated weights for policy 1, policy_version 83590 (0.0007) [2023-10-14 21:08:10,996][61585] Updated weights for policy 1, policy_version 83600 (0.0008) [2023-10-14 21:08:11,371][61585] Updated weights for policy 1, policy_version 83610 (0.0009) [2023-10-14 21:08:12,381][61552] Updated weights for policy 0, policy_version 83752 (0.0009) [2023-10-14 21:08:12,737][61552] Updated weights for policy 0, policy_version 83762 (0.0011) [2023-10-14 21:08:13,109][61552] Updated weights for policy 0, policy_version 83772 (0.0010) [2023-10-14 21:08:13,343][60425] Fps is (10 sec: 13109.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 171409408. Throughput: 0: 1659.3, 1: 1683.6. Samples: 42858114. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-14 21:08:13,344][60425] Avg episode reward: [(0, '81.600'), (1, '77.910')] [2023-10-14 21:08:15,354][61585] Updated weights for policy 1, policy_version 83620 (0.0008) [2023-10-14 21:08:15,725][61585] Updated weights for policy 1, policy_version 83630 (0.0007) [2023-10-14 21:08:16,098][61585] Updated weights for policy 1, policy_version 83640 (0.0009) [2023-10-14 21:08:17,167][61552] Updated weights for policy 0, policy_version 83782 (0.0008) [2023-10-14 21:08:17,526][61552] Updated weights for policy 0, policy_version 83792 (0.0009) [2023-10-14 21:08:17,889][61552] Updated weights for policy 0, policy_version 83802 (0.0009) [2023-10-14 21:08:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 171474944. Throughput: 0: 1676.9, 1: 1672.0. Samples: 42868564. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-14 21:08:18,344][60425] Avg episode reward: [(0, '80.980'), (1, '83.840')] [2023-10-14 21:08:20,124][61585] Updated weights for policy 1, policy_version 83650 (0.0008) [2023-10-14 21:08:20,489][61585] Updated weights for policy 1, policy_version 83660 (0.0009) [2023-10-14 21:08:20,854][61585] Updated weights for policy 1, policy_version 83670 (0.0007) [2023-10-14 21:08:21,216][61585] Updated weights for policy 1, policy_version 83680 (0.0008) [2023-10-14 21:08:21,969][61552] Updated weights for policy 0, policy_version 83812 (0.0010) [2023-10-14 21:08:22,339][61552] Updated weights for policy 0, policy_version 83822 (0.0011) [2023-10-14 21:08:22,708][61552] Updated weights for policy 0, policy_version 83832 (0.0008) [2023-10-14 21:08:23,344][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 171540480. Throughput: 0: 1671.1, 1: 1669.1. Samples: 42888406. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-14 21:08:23,345][60425] Avg episode reward: [(0, '78.350'), (1, '80.720')] [2023-10-14 21:08:25,547][61585] Updated weights for policy 1, policy_version 83690 (0.0007) [2023-10-14 21:08:25,920][61585] Updated weights for policy 1, policy_version 83700 (0.0007) [2023-10-14 21:08:26,278][61585] Updated weights for policy 1, policy_version 83710 (0.0009) [2023-10-14 21:08:26,968][61552] Updated weights for policy 0, policy_version 83842 (0.0007) [2023-10-14 21:08:27,382][61552] Updated weights for policy 0, policy_version 83852 (0.0010) [2023-10-14 21:08:27,751][61552] Updated weights for policy 0, policy_version 83862 (0.0009) [2023-10-14 21:08:28,122][61552] Updated weights for policy 0, policy_version 83872 (0.0008) [2023-10-14 21:08:28,344][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 171606016. Throughput: 0: 1653.0, 1: 1687.8. Samples: 42908098. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-14 21:08:28,345][60425] Avg episode reward: [(0, '79.910'), (1, '82.700')] [2023-10-14 21:08:30,293][61585] Updated weights for policy 1, policy_version 83720 (0.0007) [2023-10-14 21:08:30,657][61585] Updated weights for policy 1, policy_version 83730 (0.0007) [2023-10-14 21:08:31,025][61585] Updated weights for policy 1, policy_version 83740 (0.0007) [2023-10-14 21:08:32,191][61552] Updated weights for policy 0, policy_version 83882 (0.0007) [2023-10-14 21:08:32,559][61552] Updated weights for policy 0, policy_version 83892 (0.0008) [2023-10-14 21:08:32,932][61552] Updated weights for policy 0, policy_version 83902 (0.0007) [2023-10-14 21:08:33,343][60425] Fps is (10 sec: 13107.7, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 171671552. Throughput: 0: 1669.4, 1: 1671.2. Samples: 42918582. Policy #0 lag: (min: 22.0, avg: 22.4, max: 35.0) [2023-10-14 21:08:33,344][60425] Avg episode reward: [(0, '85.260'), (1, '78.290')] [2023-10-14 21:08:35,172][61585] Updated weights for policy 1, policy_version 83750 (0.0008) [2023-10-14 21:08:35,533][61585] Updated weights for policy 1, policy_version 83760 (0.0008) [2023-10-14 21:08:35,888][61585] Updated weights for policy 1, policy_version 83770 (0.0008) [2023-10-14 21:08:36,952][61552] Updated weights for policy 0, policy_version 83912 (0.0008) [2023-10-14 21:08:37,322][61552] Updated weights for policy 0, policy_version 83922 (0.0010) [2023-10-14 21:08:37,692][61552] Updated weights for policy 0, policy_version 83932 (0.0008) [2023-10-14 21:08:38,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 171737088. Throughput: 0: 1668.9, 1: 1679.5. Samples: 42938852. Policy #0 lag: (min: 22.0, avg: 22.4, max: 35.0) [2023-10-14 21:08:38,344][60425] Avg episode reward: [(0, '85.030'), (1, '78.840')] [2023-10-14 21:08:40,018][61585] Updated weights for policy 1, policy_version 83780 (0.0010) [2023-10-14 21:08:40,385][61585] Updated weights for policy 1, policy_version 83790 (0.0009) [2023-10-14 21:08:40,747][61585] Updated weights for policy 1, policy_version 83800 (0.0007) [2023-10-14 21:08:41,818][61552] Updated weights for policy 0, policy_version 83942 (0.0008) [2023-10-14 21:08:42,184][61552] Updated weights for policy 0, policy_version 83952 (0.0007) [2023-10-14 21:08:42,560][61552] Updated weights for policy 0, policy_version 83962 (0.0009) [2023-10-14 21:08:43,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 171802624. Throughput: 0: 1655.5, 1: 1692.6. Samples: 42958538. Policy #0 lag: (min: 22.0, avg: 22.4, max: 35.0) [2023-10-14 21:08:43,345][60425] Avg episode reward: [(0, '78.510'), (1, '77.820')] [2023-10-14 21:08:44,847][61585] Updated weights for policy 1, policy_version 83810 (0.0009) [2023-10-14 21:08:45,219][61585] Updated weights for policy 1, policy_version 83820 (0.0009) [2023-10-14 21:08:45,587][61585] Updated weights for policy 1, policy_version 83830 (0.0009) [2023-10-14 21:08:45,954][61585] Updated weights for policy 1, policy_version 83840 (0.0009) [2023-10-14 21:08:46,552][61552] Updated weights for policy 0, policy_version 83972 (0.0008) [2023-10-14 21:08:46,931][61552] Updated weights for policy 0, policy_version 83982 (0.0008) [2023-10-14 21:08:47,294][61552] Updated weights for policy 0, policy_version 83992 (0.0008) [2023-10-14 21:08:48,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 171868160. Throughput: 0: 1679.9, 1: 1666.2. Samples: 42969066. Policy #0 lag: (min: 22.0, avg: 22.4, max: 35.0) [2023-10-14 21:08:48,344][60425] Avg episode reward: [(0, '78.350'), (1, '83.060')] [2023-10-14 21:08:50,082][61585] Updated weights for policy 1, policy_version 83850 (0.0009) [2023-10-14 21:08:50,447][61585] Updated weights for policy 1, policy_version 83860 (0.0010) [2023-10-14 21:08:50,811][61585] Updated weights for policy 1, policy_version 83870 (0.0009) [2023-10-14 21:08:51,340][61552] Updated weights for policy 0, policy_version 84002 (0.0008) [2023-10-14 21:08:51,707][61552] Updated weights for policy 0, policy_version 84012 (0.0007) [2023-10-14 21:08:52,078][61552] Updated weights for policy 0, policy_version 84022 (0.0007) [2023-10-14 21:08:52,446][61552] Updated weights for policy 0, policy_version 84032 (0.0007) [2023-10-14 21:08:53,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 171933696. Throughput: 0: 1670.6, 1: 1677.8. Samples: 42988902. Policy #0 lag: (min: 22.0, avg: 22.4, max: 35.0) [2023-10-14 21:08:53,344][60425] Avg episode reward: [(0, '83.300'), (1, '77.980')] [2023-10-14 21:08:54,853][61585] Updated weights for policy 1, policy_version 83880 (0.0010) [2023-10-14 21:08:55,218][61585] Updated weights for policy 1, policy_version 83890 (0.0009) [2023-10-14 21:08:55,587][61585] Updated weights for policy 1, policy_version 83900 (0.0007) [2023-10-14 21:08:56,518][61552] Updated weights for policy 0, policy_version 84042 (0.0008) [2023-10-14 21:08:56,884][61552] Updated weights for policy 0, policy_version 84052 (0.0010) [2023-10-14 21:08:57,245][61552] Updated weights for policy 0, policy_version 84062 (0.0007) [2023-10-14 21:08:58,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 171999232. Throughput: 0: 1667.1, 1: 1683.5. Samples: 43008888. Policy #0 lag: (min: 22.0, avg: 22.4, max: 35.0) [2023-10-14 21:08:58,344][60425] Avg episode reward: [(0, '80.290'), (1, '73.160')] [2023-10-14 21:08:59,611][61585] Updated weights for policy 1, policy_version 83910 (0.0010) [2023-10-14 21:08:59,978][61585] Updated weights for policy 1, policy_version 83920 (0.0009) [2023-10-14 21:09:00,337][61585] Updated weights for policy 1, policy_version 83930 (0.0008) [2023-10-14 21:09:01,407][61552] Updated weights for policy 0, policy_version 84072 (0.0008) [2023-10-14 21:09:01,771][61552] Updated weights for policy 0, policy_version 84082 (0.0007) [2023-10-14 21:09:02,142][61552] Updated weights for policy 0, policy_version 84092 (0.0009) [2023-10-14 21:09:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.6, 300 sec: 13329.4). Total num frames: 172064768. Throughput: 0: 1680.6, 1: 1669.4. Samples: 43019312. Policy #0 lag: (min: 22.0, avg: 22.4, max: 35.0) [2023-10-14 21:09:03,344][60425] Avg episode reward: [(0, '80.250'), (1, '78.800')] [2023-10-14 21:09:04,360][61585] Updated weights for policy 1, policy_version 83940 (0.0009) [2023-10-14 21:09:04,734][61585] Updated weights for policy 1, policy_version 83950 (0.0010) [2023-10-14 21:09:05,105][61585] Updated weights for policy 1, policy_version 83960 (0.0009) [2023-10-14 21:09:06,034][61552] Updated weights for policy 0, policy_version 84102 (0.0009) [2023-10-14 21:09:06,404][61552] Updated weights for policy 0, policy_version 84112 (0.0010) [2023-10-14 21:09:06,768][61552] Updated weights for policy 0, policy_version 84122 (0.0008) [2023-10-14 21:09:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 172130304. Throughput: 0: 1668.9, 1: 1683.5. Samples: 43039266. Policy #0 lag: (min: 22.0, avg: 22.4, max: 35.0) [2023-10-14 21:09:08,344][60425] Avg episode reward: [(0, '80.530'), (1, '79.200')] [2023-10-14 21:09:09,274][61585] Updated weights for policy 1, policy_version 83970 (0.0009) [2023-10-14 21:09:09,640][61585] Updated weights for policy 1, policy_version 83980 (0.0012) [2023-10-14 21:09:10,000][61585] Updated weights for policy 1, policy_version 83990 (0.0010) [2023-10-14 21:09:10,366][61585] Updated weights for policy 1, policy_version 84000 (0.0009) [2023-10-14 21:09:10,675][61552] Updated weights for policy 0, policy_version 84132 (0.0008) [2023-10-14 21:09:11,054][61552] Updated weights for policy 0, policy_version 84142 (0.0009) [2023-10-14 21:09:11,418][61552] Updated weights for policy 0, policy_version 84152 (0.0009) [2023-10-14 21:09:13,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 172195840. Throughput: 0: 1688.8, 1: 1680.5. Samples: 43059716. Policy #0 lag: (min: 22.0, avg: 22.4, max: 35.0) [2023-10-14 21:09:13,344][60425] Avg episode reward: [(0, '77.650'), (1, '81.200')] [2023-10-14 21:09:14,475][61585] Updated weights for policy 1, policy_version 84010 (0.0008) [2023-10-14 21:09:14,846][61585] Updated weights for policy 1, policy_version 84020 (0.0008) [2023-10-14 21:09:15,201][61585] Updated weights for policy 1, policy_version 84030 (0.0009) [2023-10-14 21:09:15,528][61552] Updated weights for policy 0, policy_version 84162 (0.0009) [2023-10-14 21:09:15,940][61552] Updated weights for policy 0, policy_version 84172 (0.0009) [2023-10-14 21:09:16,311][61552] Updated weights for policy 0, policy_version 84182 (0.0009) [2023-10-14 21:09:16,676][61552] Updated weights for policy 0, policy_version 84192 (0.0011) [2023-10-14 21:09:18,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 172261376. Throughput: 0: 1693.6, 1: 1669.5. Samples: 43069922. Policy #0 lag: (min: 22.0, avg: 22.4, max: 35.0) [2023-10-14 21:09:18,344][60425] Avg episode reward: [(0, '76.300'), (1, '79.250')] [2023-10-14 21:09:19,349][61585] Updated weights for policy 1, policy_version 84040 (0.0008) [2023-10-14 21:09:19,717][61585] Updated weights for policy 1, policy_version 84050 (0.0008) [2023-10-14 21:09:20,076][61585] Updated weights for policy 1, policy_version 84060 (0.0007) [2023-10-14 21:09:20,515][61552] Updated weights for policy 0, policy_version 84202 (0.0007) [2023-10-14 21:09:20,872][61552] Updated weights for policy 0, policy_version 84212 (0.0007) [2023-10-14 21:09:21,242][61552] Updated weights for policy 0, policy_version 84222 (0.0008) [2023-10-14 21:09:23,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 172326912. Throughput: 0: 1675.7, 1: 1680.5. Samples: 43089884. Policy #0 lag: (min: 22.0, avg: 22.4, max: 35.0) [2023-10-14 21:09:23,344][60425] Avg episode reward: [(0, '79.030'), (1, '76.410')] [2023-10-14 21:09:24,065][61585] Updated weights for policy 1, policy_version 84070 (0.0010) [2023-10-14 21:09:24,436][61585] Updated weights for policy 1, policy_version 84080 (0.0010) [2023-10-14 21:09:24,796][61585] Updated weights for policy 1, policy_version 84090 (0.0011) [2023-10-14 21:09:25,238][61552] Updated weights for policy 0, policy_version 84232 (0.0009) [2023-10-14 21:09:25,599][61552] Updated weights for policy 0, policy_version 84242 (0.0007) [2023-10-14 21:09:25,969][61552] Updated weights for policy 0, policy_version 84252 (0.0009) [2023-10-14 21:09:28,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 172392448. Throughput: 0: 1702.5, 1: 1678.5. Samples: 43110684. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-14 21:09:28,344][60425] Avg episode reward: [(0, '81.450'), (1, '78.040')] [2023-10-14 21:09:29,003][61585] Updated weights for policy 1, policy_version 84100 (0.0007) [2023-10-14 21:09:29,391][61585] Updated weights for policy 1, policy_version 84110 (0.0007) [2023-10-14 21:09:29,754][61585] Updated weights for policy 1, policy_version 84120 (0.0010) [2023-10-14 21:09:30,109][61552] Updated weights for policy 0, policy_version 84262 (0.0009) [2023-10-14 21:09:30,468][61552] Updated weights for policy 0, policy_version 84272 (0.0007) [2023-10-14 21:09:30,834][61552] Updated weights for policy 0, policy_version 84282 (0.0009) [2023-10-14 21:09:33,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 172457984. Throughput: 0: 1683.6, 1: 1673.3. Samples: 43120126. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-14 21:09:33,345][60425] Avg episode reward: [(0, '77.530'), (1, '76.530')] [2023-10-14 21:09:33,639][61585] Updated weights for policy 1, policy_version 84130 (0.0009) [2023-10-14 21:09:33,995][61585] Updated weights for policy 1, policy_version 84140 (0.0008) [2023-10-14 21:09:34,363][61585] Updated weights for policy 1, policy_version 84150 (0.0011) [2023-10-14 21:09:34,728][61585] Updated weights for policy 1, policy_version 84160 (0.0008) [2023-10-14 21:09:34,794][61552] Updated weights for policy 0, policy_version 84292 (0.0008) [2023-10-14 21:09:35,172][61552] Updated weights for policy 0, policy_version 84302 (0.0007) [2023-10-14 21:09:35,541][61552] Updated weights for policy 0, policy_version 84312 (0.0007) [2023-10-14 21:09:38,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 172523520. Throughput: 0: 1680.3, 1: 1682.3. Samples: 43140216. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-14 21:09:38,344][60425] Avg episode reward: [(0, '77.120'), (1, '81.350')] [2023-10-14 21:09:38,874][61585] Updated weights for policy 1, policy_version 84170 (0.0008) [2023-10-14 21:09:39,242][61585] Updated weights for policy 1, policy_version 84180 (0.0007) [2023-10-14 21:09:39,593][61552] Updated weights for policy 0, policy_version 84322 (0.0007) [2023-10-14 21:09:39,615][61585] Updated weights for policy 1, policy_version 84190 (0.0008) [2023-10-14 21:09:39,967][61552] Updated weights for policy 0, policy_version 84332 (0.0009) [2023-10-14 21:09:40,340][61552] Updated weights for policy 0, policy_version 84342 (0.0009) [2023-10-14 21:09:40,712][61552] Updated weights for policy 0, policy_version 84352 (0.0008) [2023-10-14 21:09:43,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 172589056. Throughput: 0: 1698.5, 1: 1681.1. Samples: 43160970. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-14 21:09:43,344][60425] Avg episode reward: [(0, '78.290'), (1, '80.120')] [2023-10-14 21:09:43,353][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000084352_86376448.pth... [2023-10-14 21:09:43,353][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000084192_86212608.pth... [2023-10-14 21:09:43,395][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000082624_84606976.pth [2023-10-14 21:09:43,395][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000082784_84770816.pth [2023-10-14 21:09:43,829][61585] Updated weights for policy 1, policy_version 84200 (0.0008) [2023-10-14 21:09:44,191][61585] Updated weights for policy 1, policy_version 84210 (0.0010) [2023-10-14 21:09:44,558][61585] Updated weights for policy 1, policy_version 84220 (0.0008) [2023-10-14 21:09:44,889][61552] Updated weights for policy 0, policy_version 84362 (0.0008) [2023-10-14 21:09:45,265][61552] Updated weights for policy 0, policy_version 84372 (0.0011) [2023-10-14 21:09:45,626][61552] Updated weights for policy 0, policy_version 84382 (0.0008) [2023-10-14 21:09:48,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 172654592. Throughput: 0: 1672.7, 1: 1677.4. Samples: 43170068. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-14 21:09:48,344][60425] Avg episode reward: [(0, '80.210'), (1, '76.620')] [2023-10-14 21:09:48,542][61585] Updated weights for policy 1, policy_version 84230 (0.0008) [2023-10-14 21:09:48,905][61585] Updated weights for policy 1, policy_version 84240 (0.0007) [2023-10-14 21:09:49,264][61585] Updated weights for policy 1, policy_version 84250 (0.0008) [2023-10-14 21:09:49,690][61552] Updated weights for policy 0, policy_version 84392 (0.0010) [2023-10-14 21:09:50,055][61552] Updated weights for policy 0, policy_version 84402 (0.0009) [2023-10-14 21:09:50,422][61552] Updated weights for policy 0, policy_version 84412 (0.0009) [2023-10-14 21:09:53,227][61585] Updated weights for policy 1, policy_version 84260 (0.0008) [2023-10-14 21:09:53,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 172720128. Throughput: 0: 1684.8, 1: 1678.7. Samples: 43190624. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-14 21:09:53,344][60425] Avg episode reward: [(0, '82.480'), (1, '77.470')] [2023-10-14 21:09:53,582][61585] Updated weights for policy 1, policy_version 84270 (0.0008) [2023-10-14 21:09:53,945][61585] Updated weights for policy 1, policy_version 84280 (0.0010) [2023-10-14 21:09:54,597][61552] Updated weights for policy 0, policy_version 84422 (0.0009) [2023-10-14 21:09:54,965][61552] Updated weights for policy 0, policy_version 84432 (0.0007) [2023-10-14 21:09:55,340][61552] Updated weights for policy 0, policy_version 84442 (0.0008) [2023-10-14 21:09:58,065][61585] Updated weights for policy 1, policy_version 84290 (0.0009) [2023-10-14 21:09:58,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 172785664. Throughput: 0: 1688.8, 1: 1682.7. Samples: 43211432. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-14 21:09:58,344][60425] Avg episode reward: [(0, '79.590'), (1, '81.920')] [2023-10-14 21:09:58,424][61585] Updated weights for policy 1, policy_version 84300 (0.0009) [2023-10-14 21:09:58,780][61585] Updated weights for policy 1, policy_version 84310 (0.0010) [2023-10-14 21:09:59,141][61585] Updated weights for policy 1, policy_version 84320 (0.0010) [2023-10-14 21:09:59,347][61552] Updated weights for policy 0, policy_version 84452 (0.0008) [2023-10-14 21:09:59,710][61552] Updated weights for policy 0, policy_version 84462 (0.0010) [2023-10-14 21:10:00,084][61552] Updated weights for policy 0, policy_version 84472 (0.0008) [2023-10-14 21:10:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 172851200. Throughput: 0: 1666.6, 1: 1682.0. Samples: 43220610. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-14 21:10:03,344][60425] Avg episode reward: [(0, '78.270'), (1, '78.100')] [2023-10-14 21:10:03,427][61585] Updated weights for policy 1, policy_version 84330 (0.0008) [2023-10-14 21:10:03,799][61585] Updated weights for policy 1, policy_version 84340 (0.0010) [2023-10-14 21:10:04,023][61552] Updated weights for policy 0, policy_version 84482 (0.0009) [2023-10-14 21:10:04,159][61585] Updated weights for policy 1, policy_version 84350 (0.0008) [2023-10-14 21:10:04,433][61552] Updated weights for policy 0, policy_version 84492 (0.0009) [2023-10-14 21:10:04,795][61552] Updated weights for policy 0, policy_version 84502 (0.0009) [2023-10-14 21:10:05,167][61552] Updated weights for policy 0, policy_version 84512 (0.0007) [2023-10-14 21:10:08,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 172916736. Throughput: 0: 1689.4, 1: 1674.9. Samples: 43241278. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-14 21:10:08,344][60425] Avg episode reward: [(0, '80.070'), (1, '80.730')] [2023-10-14 21:10:08,451][61585] Updated weights for policy 1, policy_version 84360 (0.0008) [2023-10-14 21:10:08,822][61585] Updated weights for policy 1, policy_version 84370 (0.0008) [2023-10-14 21:10:09,199][61585] Updated weights for policy 1, policy_version 84380 (0.0010) [2023-10-14 21:10:09,201][61552] Updated weights for policy 0, policy_version 84522 (0.0007) [2023-10-14 21:10:09,568][61552] Updated weights for policy 0, policy_version 84532 (0.0007) [2023-10-14 21:10:09,933][61552] Updated weights for policy 0, policy_version 84542 (0.0010) [2023-10-14 21:10:13,082][61585] Updated weights for policy 1, policy_version 84390 (0.0008) [2023-10-14 21:10:13,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 172982272. Throughput: 0: 1685.4, 1: 1677.4. Samples: 43262010. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-14 21:10:13,344][60425] Avg episode reward: [(0, '81.750'), (1, '79.100')] [2023-10-14 21:10:13,448][61585] Updated weights for policy 1, policy_version 84400 (0.0010) [2023-10-14 21:10:13,813][61585] Updated weights for policy 1, policy_version 84410 (0.0010) [2023-10-14 21:10:14,060][61552] Updated weights for policy 0, policy_version 84552 (0.0008) [2023-10-14 21:10:14,426][61552] Updated weights for policy 0, policy_version 84562 (0.0007) [2023-10-14 21:10:14,784][61552] Updated weights for policy 0, policy_version 84572 (0.0008) [2023-10-14 21:10:18,018][61585] Updated weights for policy 1, policy_version 84420 (0.0007) [2023-10-14 21:10:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 173047808. Throughput: 0: 1671.2, 1: 1681.2. Samples: 43270982. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-14 21:10:18,344][60425] Avg episode reward: [(0, '80.360'), (1, '81.190')] [2023-10-14 21:10:18,419][61585] Updated weights for policy 1, policy_version 84430 (0.0008) [2023-10-14 21:10:18,787][61585] Updated weights for policy 1, policy_version 84440 (0.0009) [2023-10-14 21:10:18,985][61552] Updated weights for policy 0, policy_version 84582 (0.0007) [2023-10-14 21:10:19,354][61552] Updated weights for policy 0, policy_version 84592 (0.0008) [2023-10-14 21:10:19,726][61552] Updated weights for policy 0, policy_version 84602 (0.0008) [2023-10-14 21:10:22,963][61585] Updated weights for policy 1, policy_version 84450 (0.0009) [2023-10-14 21:10:23,335][61585] Updated weights for policy 1, policy_version 84460 (0.0007) [2023-10-14 21:10:23,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 173113344. Throughput: 0: 1677.8, 1: 1678.9. Samples: 43291266. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-14 21:10:23,344][60425] Avg episode reward: [(0, '78.680'), (1, '75.860')] [2023-10-14 21:10:23,716][61585] Updated weights for policy 1, policy_version 84470 (0.0008) [2023-10-14 21:10:23,801][61552] Updated weights for policy 0, policy_version 84612 (0.0010) [2023-10-14 21:10:24,080][61585] Updated weights for policy 1, policy_version 84480 (0.0008) [2023-10-14 21:10:24,161][61552] Updated weights for policy 0, policy_version 84622 (0.0009) [2023-10-14 21:10:24,530][61552] Updated weights for policy 0, policy_version 84632 (0.0008) [2023-10-14 21:10:28,100][61585] Updated weights for policy 1, policy_version 84490 (0.0007) [2023-10-14 21:10:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 173178880. Throughput: 0: 1671.6, 1: 1677.9. Samples: 43311698. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-14 21:10:28,344][60425] Avg episode reward: [(0, '78.710'), (1, '78.060')] [2023-10-14 21:10:28,476][61585] Updated weights for policy 1, policy_version 84500 (0.0009) [2023-10-14 21:10:28,683][61552] Updated weights for policy 0, policy_version 84642 (0.0010) [2023-10-14 21:10:28,835][61585] Updated weights for policy 1, policy_version 84510 (0.0008) [2023-10-14 21:10:29,049][61552] Updated weights for policy 0, policy_version 84652 (0.0008) [2023-10-14 21:10:29,420][61552] Updated weights for policy 0, policy_version 84662 (0.0011) [2023-10-14 21:10:29,781][61552] Updated weights for policy 0, policy_version 84672 (0.0009) [2023-10-14 21:10:32,697][61585] Updated weights for policy 1, policy_version 84520 (0.0008) [2023-10-14 21:10:33,061][61585] Updated weights for policy 1, policy_version 84530 (0.0008) [2023-10-14 21:10:33,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 173244416. Throughput: 0: 1668.7, 1: 1685.4. Samples: 43321002. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-14 21:10:33,344][60425] Avg episode reward: [(0, '79.300'), (1, '79.970')] [2023-10-14 21:10:33,425][61585] Updated weights for policy 1, policy_version 84540 (0.0008) [2023-10-14 21:10:33,905][61552] Updated weights for policy 0, policy_version 84682 (0.0010) [2023-10-14 21:10:34,270][61552] Updated weights for policy 0, policy_version 84692 (0.0008) [2023-10-14 21:10:34,634][61552] Updated weights for policy 0, policy_version 84702 (0.0009) [2023-10-14 21:10:37,334][61585] Updated weights for policy 1, policy_version 84550 (0.0009) [2023-10-14 21:10:37,693][61585] Updated weights for policy 1, policy_version 84560 (0.0012) [2023-10-14 21:10:38,057][61585] Updated weights for policy 1, policy_version 84570 (0.0007) [2023-10-14 21:10:38,343][60425] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 173342720. Throughput: 0: 1670.1, 1: 1686.2. Samples: 43341658. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-14 21:10:38,345][60425] Avg episode reward: [(0, '76.910'), (1, '78.730')] [2023-10-14 21:10:38,672][61552] Updated weights for policy 0, policy_version 84712 (0.0007) [2023-10-14 21:10:39,044][61552] Updated weights for policy 0, policy_version 84722 (0.0010) [2023-10-14 21:10:39,401][61552] Updated weights for policy 0, policy_version 84732 (0.0008) [2023-10-14 21:10:42,130][61585] Updated weights for policy 1, policy_version 84580 (0.0008) [2023-10-14 21:10:42,487][61585] Updated weights for policy 1, policy_version 84590 (0.0011) [2023-10-14 21:10:42,861][61585] Updated weights for policy 1, policy_version 84600 (0.0008) [2023-10-14 21:10:43,343][60425] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 173408256. Throughput: 0: 1669.5, 1: 1665.8. Samples: 43361520. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-14 21:10:43,344][60425] Avg episode reward: [(0, '74.670'), (1, '80.050')] [2023-10-14 21:10:43,500][61552] Updated weights for policy 0, policy_version 84742 (0.0009) [2023-10-14 21:10:43,873][61552] Updated weights for policy 0, policy_version 84752 (0.0010) [2023-10-14 21:10:44,246][61552] Updated weights for policy 0, policy_version 84762 (0.0008) [2023-10-14 21:10:46,912][61585] Updated weights for policy 1, policy_version 84610 (0.0007) [2023-10-14 21:10:47,287][61585] Updated weights for policy 1, policy_version 84620 (0.0011) [2023-10-14 21:10:47,650][61585] Updated weights for policy 1, policy_version 84630 (0.0008) [2023-10-14 21:10:48,008][61585] Updated weights for policy 1, policy_version 84640 (0.0011) [2023-10-14 21:10:48,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 173473792. Throughput: 0: 1666.8, 1: 1682.2. Samples: 43371318. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-14 21:10:48,344][60425] Avg episode reward: [(0, '75.980'), (1, '78.560')] [2023-10-14 21:10:48,651][61552] Updated weights for policy 0, policy_version 84772 (0.0009) [2023-10-14 21:10:49,022][61552] Updated weights for policy 0, policy_version 84782 (0.0010) [2023-10-14 21:10:49,402][61552] Updated weights for policy 0, policy_version 84792 (0.0008) [2023-10-14 21:10:52,053][61585] Updated weights for policy 1, policy_version 84650 (0.0009) [2023-10-14 21:10:52,428][61585] Updated weights for policy 1, policy_version 84660 (0.0009) [2023-10-14 21:10:52,790][61585] Updated weights for policy 1, policy_version 84670 (0.0007) [2023-10-14 21:10:53,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 173539328. Throughput: 0: 1654.8, 1: 1686.6. Samples: 43391642. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-14 21:10:53,344][60425] Avg episode reward: [(0, '81.250'), (1, '85.170')] [2023-10-14 21:10:53,629][61552] Updated weights for policy 0, policy_version 84802 (0.0008) [2023-10-14 21:10:54,012][61552] Updated weights for policy 0, policy_version 84812 (0.0009) [2023-10-14 21:10:54,383][61552] Updated weights for policy 0, policy_version 84822 (0.0008) [2023-10-14 21:10:54,754][61552] Updated weights for policy 0, policy_version 84832 (0.0008) [2023-10-14 21:10:56,796][61585] Updated weights for policy 1, policy_version 84680 (0.0010) [2023-10-14 21:10:57,171][61585] Updated weights for policy 1, policy_version 84690 (0.0009) [2023-10-14 21:10:57,527][61585] Updated weights for policy 1, policy_version 84700 (0.0010) [2023-10-14 21:10:58,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 173604864. Throughput: 0: 1653.6, 1: 1656.1. Samples: 43410948. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-14 21:10:58,344][60425] Avg episode reward: [(0, '80.860'), (1, '77.580')] [2023-10-14 21:10:58,763][61552] Updated weights for policy 0, policy_version 84842 (0.0007) [2023-10-14 21:10:59,127][61552] Updated weights for policy 0, policy_version 84852 (0.0008) [2023-10-14 21:10:59,499][61552] Updated weights for policy 0, policy_version 84862 (0.0009) [2023-10-14 21:11:01,570][61585] Updated weights for policy 1, policy_version 84710 (0.0008) [2023-10-14 21:11:01,933][61585] Updated weights for policy 1, policy_version 84720 (0.0010) [2023-10-14 21:11:02,298][61585] Updated weights for policy 1, policy_version 84730 (0.0009) [2023-10-14 21:11:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 173670400. Throughput: 0: 1657.6, 1: 1684.9. Samples: 43421398. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-14 21:11:03,344][60425] Avg episode reward: [(0, '76.330'), (1, '81.530')] [2023-10-14 21:11:03,553][61552] Updated weights for policy 0, policy_version 84872 (0.0009) [2023-10-14 21:11:03,930][61552] Updated weights for policy 0, policy_version 84882 (0.0010) [2023-10-14 21:11:04,304][61552] Updated weights for policy 0, policy_version 84892 (0.0008) [2023-10-14 21:11:06,520][61585] Updated weights for policy 1, policy_version 84740 (0.0008) [2023-10-14 21:11:06,921][61585] Updated weights for policy 1, policy_version 84750 (0.0009) [2023-10-14 21:11:07,284][61585] Updated weights for policy 1, policy_version 84760 (0.0009) [2023-10-14 21:11:08,333][61552] Updated weights for policy 0, policy_version 84902 (0.0007) [2023-10-14 21:11:08,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 173735936. Throughput: 0: 1668.1, 1: 1676.8. Samples: 43441786. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-14 21:11:08,344][60425] Avg episode reward: [(0, '78.380'), (1, '80.500')] [2023-10-14 21:11:08,709][61552] Updated weights for policy 0, policy_version 84912 (0.0009) [2023-10-14 21:11:09,082][61552] Updated weights for policy 0, policy_version 84922 (0.0008) [2023-10-14 21:11:11,382][61585] Updated weights for policy 1, policy_version 84770 (0.0008) [2023-10-14 21:11:11,746][61585] Updated weights for policy 1, policy_version 84780 (0.0007) [2023-10-14 21:11:12,108][61585] Updated weights for policy 1, policy_version 84790 (0.0009) [2023-10-14 21:11:12,462][61585] Updated weights for policy 1, policy_version 84800 (0.0007) [2023-10-14 21:11:13,220][61552] Updated weights for policy 0, policy_version 84932 (0.0008) [2023-10-14 21:11:13,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 173801472. Throughput: 0: 1673.3, 1: 1654.7. Samples: 43461460. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-14 21:11:13,344][60425] Avg episode reward: [(0, '84.640'), (1, '79.050')] [2023-10-14 21:11:13,588][61552] Updated weights for policy 0, policy_version 84942 (0.0009) [2023-10-14 21:11:13,950][61552] Updated weights for policy 0, policy_version 84952 (0.0008) [2023-10-14 21:11:16,639][61585] Updated weights for policy 1, policy_version 84810 (0.0008) [2023-10-14 21:11:17,009][61585] Updated weights for policy 1, policy_version 84820 (0.0007) [2023-10-14 21:11:17,378][61585] Updated weights for policy 1, policy_version 84830 (0.0007) [2023-10-14 21:11:18,167][61552] Updated weights for policy 0, policy_version 84962 (0.0010) [2023-10-14 21:11:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.5). Total num frames: 173867008. Throughput: 0: 1670.3, 1: 1675.7. Samples: 43471572. Policy #0 lag: (min: 31.0, avg: 40.0, max: 63.0) [2023-10-14 21:11:18,344][60425] Avg episode reward: [(0, '83.200'), (1, '82.270')] [2023-10-14 21:11:18,539][61552] Updated weights for policy 0, policy_version 84972 (0.0007) [2023-10-14 21:11:18,904][61552] Updated weights for policy 0, policy_version 84982 (0.0007) [2023-10-14 21:11:19,283][61552] Updated weights for policy 0, policy_version 84992 (0.0008) [2023-10-14 21:11:21,601][61585] Updated weights for policy 1, policy_version 84840 (0.0010) [2023-10-14 21:11:21,968][61585] Updated weights for policy 1, policy_version 84850 (0.0007) [2023-10-14 21:11:22,334][61585] Updated weights for policy 1, policy_version 84860 (0.0008) [2023-10-14 21:11:23,170][61552] Updated weights for policy 0, policy_version 85002 (0.0007) [2023-10-14 21:11:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 173932544. Throughput: 0: 1672.0, 1: 1660.4. Samples: 43491614. Policy #0 lag: (min: 31.0, avg: 40.0, max: 63.0) [2023-10-14 21:11:23,344][60425] Avg episode reward: [(0, '82.800'), (1, '79.580')] [2023-10-14 21:11:23,542][61552] Updated weights for policy 0, policy_version 85012 (0.0007) [2023-10-14 21:11:23,899][61552] Updated weights for policy 0, policy_version 85022 (0.0009) [2023-10-14 21:11:26,425][61585] Updated weights for policy 1, policy_version 84870 (0.0008) [2023-10-14 21:11:26,795][61585] Updated weights for policy 1, policy_version 84880 (0.0009) [2023-10-14 21:11:27,154][61585] Updated weights for policy 1, policy_version 84890 (0.0007) [2023-10-14 21:11:27,946][61552] Updated weights for policy 0, policy_version 85032 (0.0009) [2023-10-14 21:11:28,319][61552] Updated weights for policy 0, policy_version 85042 (0.0011) [2023-10-14 21:11:28,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 173998080. Throughput: 0: 1672.9, 1: 1661.4. Samples: 43511564. Policy #0 lag: (min: 31.0, avg: 40.0, max: 63.0) [2023-10-14 21:11:28,344][60425] Avg episode reward: [(0, '82.820'), (1, '81.230')] [2023-10-14 21:11:28,680][61552] Updated weights for policy 0, policy_version 85052 (0.0010) [2023-10-14 21:11:31,125][61585] Updated weights for policy 1, policy_version 84900 (0.0009) [2023-10-14 21:11:31,486][61585] Updated weights for policy 1, policy_version 84910 (0.0007) [2023-10-14 21:11:31,853][61585] Updated weights for policy 1, policy_version 84920 (0.0008) [2023-10-14 21:11:32,738][61552] Updated weights for policy 0, policy_version 85062 (0.0009) [2023-10-14 21:11:33,101][61552] Updated weights for policy 0, policy_version 85072 (0.0009) [2023-10-14 21:11:33,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 174063616. Throughput: 0: 1674.1, 1: 1674.8. Samples: 43522018. Policy #0 lag: (min: 31.0, avg: 40.0, max: 63.0) [2023-10-14 21:11:33,344][60425] Avg episode reward: [(0, '82.050'), (1, '74.760')] [2023-10-14 21:11:33,476][61552] Updated weights for policy 0, policy_version 85082 (0.0008) [2023-10-14 21:11:36,023][61585] Updated weights for policy 1, policy_version 84930 (0.0008) [2023-10-14 21:11:36,387][61585] Updated weights for policy 1, policy_version 84940 (0.0010) [2023-10-14 21:11:36,752][61585] Updated weights for policy 1, policy_version 84950 (0.0010) [2023-10-14 21:11:37,114][61585] Updated weights for policy 1, policy_version 84960 (0.0009) [2023-10-14 21:11:37,732][61552] Updated weights for policy 0, policy_version 85092 (0.0008) [2023-10-14 21:11:38,102][61552] Updated weights for policy 0, policy_version 85102 (0.0007) [2023-10-14 21:11:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 174129152. Throughput: 0: 1682.2, 1: 1658.3. Samples: 43541964. Policy #0 lag: (min: 31.0, avg: 40.0, max: 63.0) [2023-10-14 21:11:38,344][60425] Avg episode reward: [(0, '79.470'), (1, '80.040')] [2023-10-14 21:11:38,479][61552] Updated weights for policy 0, policy_version 85112 (0.0007) [2023-10-14 21:11:41,128][61585] Updated weights for policy 1, policy_version 84970 (0.0008) [2023-10-14 21:11:41,490][61585] Updated weights for policy 1, policy_version 84980 (0.0008) [2023-10-14 21:11:41,857][61585] Updated weights for policy 1, policy_version 84990 (0.0009) [2023-10-14 21:11:42,533][61552] Updated weights for policy 0, policy_version 85122 (0.0009) [2023-10-14 21:11:42,941][61552] Updated weights for policy 0, policy_version 85132 (0.0009) [2023-10-14 21:11:43,304][61552] Updated weights for policy 0, policy_version 85142 (0.0007) [2023-10-14 21:11:43,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 174194688. Throughput: 0: 1679.5, 1: 1680.6. Samples: 43562152. Policy #0 lag: (min: 31.0, avg: 40.0, max: 63.0) [2023-10-14 21:11:43,344][60425] Avg episode reward: [(0, '81.820'), (1, '79.480')] [2023-10-14 21:11:43,355][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000084992_87031808.pth... [2023-10-14 21:11:43,394][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000083424_85426176.pth [2023-10-14 21:11:43,674][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000085152_87195648.pth... [2023-10-14 21:11:43,676][61552] Updated weights for policy 0, policy_version 85152 (0.0007) [2023-10-14 21:11:43,703][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000083584_85590016.pth [2023-10-14 21:11:46,027][61585] Updated weights for policy 1, policy_version 85000 (0.0008) [2023-10-14 21:11:46,387][61585] Updated weights for policy 1, policy_version 85010 (0.0009) [2023-10-14 21:11:46,754][61585] Updated weights for policy 1, policy_version 85020 (0.0009) [2023-10-14 21:11:47,705][61552] Updated weights for policy 0, policy_version 85162 (0.0008) [2023-10-14 21:11:48,069][61552] Updated weights for policy 0, policy_version 85172 (0.0010) [2023-10-14 21:11:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 174260224. Throughput: 0: 1681.7, 1: 1677.0. Samples: 43572542. Policy #0 lag: (min: 31.0, avg: 40.0, max: 63.0) [2023-10-14 21:11:48,344][60425] Avg episode reward: [(0, '79.650'), (1, '80.370')] [2023-10-14 21:11:48,436][61552] Updated weights for policy 0, policy_version 85182 (0.0009) [2023-10-14 21:11:50,829][61585] Updated weights for policy 1, policy_version 85030 (0.0010) [2023-10-14 21:11:51,192][61585] Updated weights for policy 1, policy_version 85040 (0.0010) [2023-10-14 21:11:51,564][61585] Updated weights for policy 1, policy_version 85050 (0.0010) [2023-10-14 21:11:52,448][61552] Updated weights for policy 0, policy_version 85192 (0.0007) [2023-10-14 21:11:52,823][61552] Updated weights for policy 0, policy_version 85202 (0.0009) [2023-10-14 21:11:53,191][61552] Updated weights for policy 0, policy_version 85212 (0.0008) [2023-10-14 21:11:53,343][60425] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 174358528. Throughput: 0: 1678.0, 1: 1666.0. Samples: 43592266. Policy #0 lag: (min: 31.0, avg: 40.0, max: 63.0) [2023-10-14 21:11:53,344][60425] Avg episode reward: [(0, '75.580'), (1, '83.800')] [2023-10-14 21:11:55,566][61585] Updated weights for policy 1, policy_version 85060 (0.0010) [2023-10-14 21:11:55,928][61585] Updated weights for policy 1, policy_version 85070 (0.0010) [2023-10-14 21:11:56,290][61585] Updated weights for policy 1, policy_version 85080 (0.0010) [2023-10-14 21:11:57,269][61552] Updated weights for policy 0, policy_version 85222 (0.0007) [2023-10-14 21:11:57,636][61552] Updated weights for policy 0, policy_version 85232 (0.0008) [2023-10-14 21:11:58,008][61552] Updated weights for policy 0, policy_version 85242 (0.0007) [2023-10-14 21:11:58,343][60425] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 174424064. Throughput: 0: 1667.9, 1: 1686.4. Samples: 43612404. Policy #0 lag: (min: 31.0, avg: 40.0, max: 63.0) [2023-10-14 21:11:58,344][60425] Avg episode reward: [(0, '78.970'), (1, '75.360')] [2023-10-14 21:12:00,416][61585] Updated weights for policy 1, policy_version 85090 (0.0009) [2023-10-14 21:12:00,778][61585] Updated weights for policy 1, policy_version 85100 (0.0007) [2023-10-14 21:12:01,143][61585] Updated weights for policy 1, policy_version 85110 (0.0007) [2023-10-14 21:12:01,503][61585] Updated weights for policy 1, policy_version 85120 (0.0009) [2023-10-14 21:12:02,026][61552] Updated weights for policy 0, policy_version 85252 (0.0008) [2023-10-14 21:12:02,393][61552] Updated weights for policy 0, policy_version 85262 (0.0009) [2023-10-14 21:12:02,758][61552] Updated weights for policy 0, policy_version 85272 (0.0010) [2023-10-14 21:12:03,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 174489600. Throughput: 0: 1687.2, 1: 1678.4. Samples: 43623026. Policy #0 lag: (min: 31.0, avg: 40.0, max: 63.0) [2023-10-14 21:12:03,344][60425] Avg episode reward: [(0, '82.610'), (1, '77.730')] [2023-10-14 21:12:05,396][61585] Updated weights for policy 1, policy_version 85130 (0.0007) [2023-10-14 21:12:05,771][61585] Updated weights for policy 1, policy_version 85140 (0.0007) [2023-10-14 21:12:06,132][61585] Updated weights for policy 1, policy_version 85150 (0.0009) [2023-10-14 21:12:06,747][61552] Updated weights for policy 0, policy_version 85282 (0.0010) [2023-10-14 21:12:07,117][61552] Updated weights for policy 0, policy_version 85292 (0.0008) [2023-10-14 21:12:07,486][61552] Updated weights for policy 0, policy_version 85302 (0.0009) [2023-10-14 21:12:07,856][61552] Updated weights for policy 0, policy_version 85312 (0.0009) [2023-10-14 21:12:08,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 174555136. Throughput: 0: 1689.2, 1: 1674.1. Samples: 43642966. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:12:08,344][60425] Avg episode reward: [(0, '84.030'), (1, '78.260')] [2023-10-14 21:12:10,436][61585] Updated weights for policy 1, policy_version 85160 (0.0009) [2023-10-14 21:12:10,804][61585] Updated weights for policy 1, policy_version 85170 (0.0008) [2023-10-14 21:12:11,168][61585] Updated weights for policy 1, policy_version 85180 (0.0008) [2023-10-14 21:12:11,636][61552] Updated weights for policy 0, policy_version 85322 (0.0009) [2023-10-14 21:12:12,003][61552] Updated weights for policy 0, policy_version 85332 (0.0009) [2023-10-14 21:12:12,370][61552] Updated weights for policy 0, policy_version 85342 (0.0008) [2023-10-14 21:12:13,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 174620672. Throughput: 0: 1664.4, 1: 1689.5. Samples: 43662490. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:12:13,344][60425] Avg episode reward: [(0, '77.100'), (1, '81.150')] [2023-10-14 21:12:15,372][61585] Updated weights for policy 1, policy_version 85190 (0.0008) [2023-10-14 21:12:15,726][61585] Updated weights for policy 1, policy_version 85200 (0.0007) [2023-10-14 21:12:16,093][61585] Updated weights for policy 1, policy_version 85210 (0.0009) [2023-10-14 21:12:16,303][61552] Updated weights for policy 0, policy_version 85352 (0.0009) [2023-10-14 21:12:16,672][61552] Updated weights for policy 0, policy_version 85362 (0.0010) [2023-10-14 21:12:17,043][61552] Updated weights for policy 0, policy_version 85372 (0.0008) [2023-10-14 21:12:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 174686208. Throughput: 0: 1694.1, 1: 1670.1. Samples: 43673410. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:12:18,344][60425] Avg episode reward: [(0, '81.250'), (1, '77.160')] [2023-10-14 21:12:20,092][61585] Updated weights for policy 1, policy_version 85220 (0.0009) [2023-10-14 21:12:20,453][61585] Updated weights for policy 1, policy_version 85230 (0.0007) [2023-10-14 21:12:20,818][61585] Updated weights for policy 1, policy_version 85240 (0.0007) [2023-10-14 21:12:21,099][61552] Updated weights for policy 0, policy_version 85382 (0.0009) [2023-10-14 21:12:21,469][61552] Updated weights for policy 0, policy_version 85392 (0.0008) [2023-10-14 21:12:21,840][61552] Updated weights for policy 0, policy_version 85402 (0.0009) [2023-10-14 21:12:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 174751744. Throughput: 0: 1674.4, 1: 1677.2. Samples: 43692784. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:12:23,344][60425] Avg episode reward: [(0, '82.800'), (1, '77.740')] [2023-10-14 21:12:24,784][61585] Updated weights for policy 1, policy_version 85250 (0.0008) [2023-10-14 21:12:25,164][61585] Updated weights for policy 1, policy_version 85260 (0.0008) [2023-10-14 21:12:25,529][61585] Updated weights for policy 1, policy_version 85270 (0.0007) [2023-10-14 21:12:25,843][61552] Updated weights for policy 0, policy_version 85412 (0.0008) [2023-10-14 21:12:25,893][61585] Updated weights for policy 1, policy_version 85280 (0.0008) [2023-10-14 21:12:26,207][61552] Updated weights for policy 0, policy_version 85422 (0.0007) [2023-10-14 21:12:26,581][61552] Updated weights for policy 0, policy_version 85432 (0.0009) [2023-10-14 21:12:28,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 174817280. Throughput: 0: 1674.1, 1: 1683.7. Samples: 43713256. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:12:28,345][60425] Avg episode reward: [(0, '81.380'), (1, '77.590')] [2023-10-14 21:12:29,940][61585] Updated weights for policy 1, policy_version 85290 (0.0011) [2023-10-14 21:12:30,304][61585] Updated weights for policy 1, policy_version 85300 (0.0011) [2023-10-14 21:12:30,635][61552] Updated weights for policy 0, policy_version 85442 (0.0010) [2023-10-14 21:12:30,671][61585] Updated weights for policy 1, policy_version 85310 (0.0007) [2023-10-14 21:12:31,034][61552] Updated weights for policy 0, policy_version 85452 (0.0009) [2023-10-14 21:12:31,409][61552] Updated weights for policy 0, policy_version 85462 (0.0009) [2023-10-14 21:12:31,774][61552] Updated weights for policy 0, policy_version 85472 (0.0009) [2023-10-14 21:12:33,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 174882816. Throughput: 0: 1695.8, 1: 1655.9. Samples: 43723366. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:12:33,344][60425] Avg episode reward: [(0, '79.140'), (1, '75.510')] [2023-10-14 21:12:34,816][61585] Updated weights for policy 1, policy_version 85320 (0.0007) [2023-10-14 21:12:35,180][61585] Updated weights for policy 1, policy_version 85330 (0.0007) [2023-10-14 21:12:35,546][61585] Updated weights for policy 1, policy_version 85340 (0.0008) [2023-10-14 21:12:35,943][61552] Updated weights for policy 0, policy_version 85482 (0.0008) [2023-10-14 21:12:36,307][61552] Updated weights for policy 0, policy_version 85492 (0.0010) [2023-10-14 21:12:36,681][61552] Updated weights for policy 0, policy_version 85502 (0.0010) [2023-10-14 21:12:38,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 174948352. Throughput: 0: 1670.0, 1: 1679.4. Samples: 43742992. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:12:38,344][60425] Avg episode reward: [(0, '80.290'), (1, '76.820')] [2023-10-14 21:12:39,696][61585] Updated weights for policy 1, policy_version 85350 (0.0009) [2023-10-14 21:12:40,059][61585] Updated weights for policy 1, policy_version 85360 (0.0009) [2023-10-14 21:12:40,418][61585] Updated weights for policy 1, policy_version 85370 (0.0010) [2023-10-14 21:12:40,741][61552] Updated weights for policy 0, policy_version 85512 (0.0008) [2023-10-14 21:12:41,112][61552] Updated weights for policy 0, policy_version 85522 (0.0007) [2023-10-14 21:12:41,477][61552] Updated weights for policy 0, policy_version 85532 (0.0008) [2023-10-14 21:12:43,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 175013888. Throughput: 0: 1680.8, 1: 1676.9. Samples: 43763498. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:12:43,344][60425] Avg episode reward: [(0, '77.650'), (1, '79.930')] [2023-10-14 21:12:44,561][61585] Updated weights for policy 1, policy_version 85380 (0.0009) [2023-10-14 21:12:44,937][61585] Updated weights for policy 1, policy_version 85390 (0.0012) [2023-10-14 21:12:45,303][61585] Updated weights for policy 1, policy_version 85400 (0.0008) [2023-10-14 21:12:45,590][61552] Updated weights for policy 0, policy_version 85542 (0.0008) [2023-10-14 21:12:45,953][61552] Updated weights for policy 0, policy_version 85552 (0.0009) [2023-10-14 21:12:46,321][61552] Updated weights for policy 0, policy_version 85562 (0.0009) [2023-10-14 21:12:48,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 175079424. Throughput: 0: 1688.2, 1: 1655.3. Samples: 43773484. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:12:48,344][60425] Avg episode reward: [(0, '75.220'), (1, '81.550')] [2023-10-14 21:12:49,338][61585] Updated weights for policy 1, policy_version 85410 (0.0009) [2023-10-14 21:12:49,695][61585] Updated weights for policy 1, policy_version 85420 (0.0010) [2023-10-14 21:12:50,057][61585] Updated weights for policy 1, policy_version 85430 (0.0010) [2023-10-14 21:12:50,422][61585] Updated weights for policy 1, policy_version 85440 (0.0007) [2023-10-14 21:12:50,531][61552] Updated weights for policy 0, policy_version 85572 (0.0009) [2023-10-14 21:12:50,901][61552] Updated weights for policy 0, policy_version 85582 (0.0007) [2023-10-14 21:12:51,265][61552] Updated weights for policy 0, policy_version 85592 (0.0010) [2023-10-14 21:12:53,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 175144960. Throughput: 0: 1663.3, 1: 1668.7. Samples: 43792904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:12:53,344][60425] Avg episode reward: [(0, '78.310'), (1, '72.050')] [2023-10-14 21:12:54,435][61585] Updated weights for policy 1, policy_version 85450 (0.0007) [2023-10-14 21:12:54,807][61585] Updated weights for policy 1, policy_version 85460 (0.0007) [2023-10-14 21:12:55,169][61585] Updated weights for policy 1, policy_version 85470 (0.0008) [2023-10-14 21:12:55,405][61552] Updated weights for policy 0, policy_version 85602 (0.0010) [2023-10-14 21:12:55,767][61552] Updated weights for policy 0, policy_version 85612 (0.0009) [2023-10-14 21:12:56,134][61552] Updated weights for policy 0, policy_version 85622 (0.0008) [2023-10-14 21:12:56,497][61552] Updated weights for policy 0, policy_version 85632 (0.0009) [2023-10-14 21:12:58,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 175210496. Throughput: 0: 1683.3, 1: 1675.9. Samples: 43813652. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:12:58,344][60425] Avg episode reward: [(0, '76.710'), (1, '79.230')] [2023-10-14 21:12:59,164][61585] Updated weights for policy 1, policy_version 85480 (0.0010) [2023-10-14 21:12:59,527][61585] Updated weights for policy 1, policy_version 85490 (0.0008) [2023-10-14 21:12:59,901][61585] Updated weights for policy 1, policy_version 85500 (0.0007) [2023-10-14 21:13:00,605][61552] Updated weights for policy 0, policy_version 85642 (0.0009) [2023-10-14 21:13:00,971][61552] Updated weights for policy 0, policy_version 85652 (0.0009) [2023-10-14 21:13:01,343][61552] Updated weights for policy 0, policy_version 85662 (0.0009) [2023-10-14 21:13:03,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 175276032. Throughput: 0: 1670.6, 1: 1666.9. Samples: 43823600. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-14 21:13:03,344][60425] Avg episode reward: [(0, '78.140'), (1, '78.080')] [2023-10-14 21:13:04,030][61585] Updated weights for policy 1, policy_version 85510 (0.0009) [2023-10-14 21:13:04,393][61585] Updated weights for policy 1, policy_version 85520 (0.0011) [2023-10-14 21:13:04,756][61585] Updated weights for policy 1, policy_version 85530 (0.0009) [2023-10-14 21:13:05,393][61552] Updated weights for policy 0, policy_version 85672 (0.0010) [2023-10-14 21:13:05,760][61552] Updated weights for policy 0, policy_version 85682 (0.0010) [2023-10-14 21:13:06,122][61552] Updated weights for policy 0, policy_version 85692 (0.0008) [2023-10-14 21:13:08,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 175341568. Throughput: 0: 1671.9, 1: 1678.4. Samples: 43843546. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-14 21:13:08,344][60425] Avg episode reward: [(0, '76.870'), (1, '80.620')] [2023-10-14 21:13:09,050][61585] Updated weights for policy 1, policy_version 85540 (0.0009) [2023-10-14 21:13:09,422][61585] Updated weights for policy 1, policy_version 85550 (0.0008) [2023-10-14 21:13:09,791][61585] Updated weights for policy 1, policy_version 85560 (0.0007) [2023-10-14 21:13:10,304][61552] Updated weights for policy 0, policy_version 85702 (0.0009) [2023-10-14 21:13:10,670][61552] Updated weights for policy 0, policy_version 85712 (0.0007) [2023-10-14 21:13:11,031][61552] Updated weights for policy 0, policy_version 85722 (0.0007) [2023-10-14 21:13:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 175407104. Throughput: 0: 1676.4, 1: 1673.2. Samples: 43863986. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-14 21:13:13,344][60425] Avg episode reward: [(0, '79.450'), (1, '79.450')] [2023-10-14 21:13:13,836][61585] Updated weights for policy 1, policy_version 85570 (0.0008) [2023-10-14 21:13:14,200][61585] Updated weights for policy 1, policy_version 85580 (0.0010) [2023-10-14 21:13:14,570][61585] Updated weights for policy 1, policy_version 85590 (0.0010) [2023-10-14 21:13:14,932][61585] Updated weights for policy 1, policy_version 85600 (0.0008) [2023-10-14 21:13:15,132][61552] Updated weights for policy 0, policy_version 85732 (0.0008) [2023-10-14 21:13:15,508][61552] Updated weights for policy 0, policy_version 85742 (0.0010) [2023-10-14 21:13:15,872][61552] Updated weights for policy 0, policy_version 85752 (0.0009) [2023-10-14 21:13:18,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 175472640. Throughput: 0: 1667.3, 1: 1673.6. Samples: 43873708. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-14 21:13:18,345][60425] Avg episode reward: [(0, '75.870'), (1, '79.520')] [2023-10-14 21:13:19,093][61585] Updated weights for policy 1, policy_version 85610 (0.0010) [2023-10-14 21:13:19,457][61585] Updated weights for policy 1, policy_version 85620 (0.0010) [2023-10-14 21:13:19,826][61585] Updated weights for policy 1, policy_version 85630 (0.0008) [2023-10-14 21:13:20,123][61552] Updated weights for policy 0, policy_version 85762 (0.0009) [2023-10-14 21:13:20,494][61552] Updated weights for policy 0, policy_version 85772 (0.0009) [2023-10-14 21:13:20,870][61552] Updated weights for policy 0, policy_version 85782 (0.0010) [2023-10-14 21:13:21,245][61552] Updated weights for policy 0, policy_version 85792 (0.0009) [2023-10-14 21:13:23,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 175538176. Throughput: 0: 1677.7, 1: 1670.5. Samples: 43893664. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-14 21:13:23,344][60425] Avg episode reward: [(0, '79.010'), (1, '76.040')] [2023-10-14 21:13:23,929][61585] Updated weights for policy 1, policy_version 85640 (0.0008) [2023-10-14 21:13:24,294][61585] Updated weights for policy 1, policy_version 85650 (0.0008) [2023-10-14 21:13:24,662][61585] Updated weights for policy 1, policy_version 85660 (0.0008) [2023-10-14 21:13:25,209][61552] Updated weights for policy 0, policy_version 85802 (0.0009) [2023-10-14 21:13:25,582][61552] Updated weights for policy 0, policy_version 85812 (0.0010) [2023-10-14 21:13:25,950][61552] Updated weights for policy 0, policy_version 85822 (0.0007) [2023-10-14 21:13:28,344][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 175603712. Throughput: 0: 1679.0, 1: 1670.7. Samples: 43914232. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-14 21:13:28,345][60425] Avg episode reward: [(0, '77.600'), (1, '77.250')] [2023-10-14 21:13:28,720][61585] Updated weights for policy 1, policy_version 85670 (0.0009) [2023-10-14 21:13:29,071][61585] Updated weights for policy 1, policy_version 85680 (0.0008) [2023-10-14 21:13:29,448][61585] Updated weights for policy 1, policy_version 85690 (0.0009) [2023-10-14 21:13:29,972][61552] Updated weights for policy 0, policy_version 85832 (0.0008) [2023-10-14 21:13:30,353][61552] Updated weights for policy 0, policy_version 85842 (0.0012) [2023-10-14 21:13:30,725][61552] Updated weights for policy 0, policy_version 85852 (0.0009) [2023-10-14 21:13:33,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 175669248. Throughput: 0: 1663.7, 1: 1669.5. Samples: 43923480. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-14 21:13:33,344][60425] Avg episode reward: [(0, '80.460'), (1, '79.380')] [2023-10-14 21:13:33,559][61585] Updated weights for policy 1, policy_version 85700 (0.0008) [2023-10-14 21:13:33,961][61585] Updated weights for policy 1, policy_version 85710 (0.0007) [2023-10-14 21:13:34,319][61585] Updated weights for policy 1, policy_version 85720 (0.0008) [2023-10-14 21:13:34,826][61552] Updated weights for policy 0, policy_version 85862 (0.0009) [2023-10-14 21:13:35,200][61552] Updated weights for policy 0, policy_version 85872 (0.0009) [2023-10-14 21:13:35,572][61552] Updated weights for policy 0, policy_version 85882 (0.0009) [2023-10-14 21:13:38,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 175734784. Throughput: 0: 1681.1, 1: 1676.6. Samples: 43944002. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-14 21:13:38,344][60425] Avg episode reward: [(0, '79.930'), (1, '79.350')] [2023-10-14 21:13:38,371][61585] Updated weights for policy 1, policy_version 85730 (0.0009) [2023-10-14 21:13:38,740][61585] Updated weights for policy 1, policy_version 85740 (0.0011) [2023-10-14 21:13:39,104][61585] Updated weights for policy 1, policy_version 85750 (0.0010) [2023-10-14 21:13:39,454][61552] Updated weights for policy 0, policy_version 85892 (0.0009) [2023-10-14 21:13:39,470][61585] Updated weights for policy 1, policy_version 85760 (0.0009) [2023-10-14 21:13:39,826][61552] Updated weights for policy 0, policy_version 85902 (0.0009) [2023-10-14 21:13:40,187][61552] Updated weights for policy 0, policy_version 85912 (0.0009) [2023-10-14 21:13:43,344][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 175800320. Throughput: 0: 1684.4, 1: 1668.7. Samples: 43964544. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-14 21:13:43,345][60425] Avg episode reward: [(0, '80.150'), (1, '78.580')] [2023-10-14 21:13:43,354][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000085920_87982080.pth... [2023-10-14 21:13:43,390][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000084352_86376448.pth [2023-10-14 21:13:43,394][61172] Saving a milestone ./train_atari/atari_roadrunner_APPO/checkpoint_p0/milestones/checkpoint_000085920_87982080.pth [2023-10-14 21:13:43,720][61585] Updated weights for policy 1, policy_version 85770 (0.0008) [2023-10-14 21:13:44,079][61585] Updated weights for policy 1, policy_version 85780 (0.0009) [2023-10-14 21:13:44,154][61552] Updated weights for policy 0, policy_version 85922 (0.0009) [2023-10-14 21:13:44,442][61585] Updated weights for policy 1, policy_version 85790 (0.0009) [2023-10-14 21:13:44,516][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000085792_87851008.pth... [2023-10-14 21:13:44,530][61552] Updated weights for policy 0, policy_version 85932 (0.0008) [2023-10-14 21:13:44,551][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000084192_86212608.pth [2023-10-14 21:13:44,555][61248] Saving a milestone ./train_atari/atari_roadrunner_APPO/checkpoint_p1/milestones/checkpoint_000085792_87851008.pth [2023-10-14 21:13:44,896][61552] Updated weights for policy 0, policy_version 85942 (0.0008) [2023-10-14 21:13:45,269][61552] Updated weights for policy 0, policy_version 85952 (0.0007) [2023-10-14 21:13:48,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 175865856. Throughput: 0: 1667.3, 1: 1665.7. Samples: 43973586. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-14 21:13:48,344][60425] Avg episode reward: [(0, '81.770'), (1, '71.850')] [2023-10-14 21:13:48,543][61585] Updated weights for policy 1, policy_version 85800 (0.0009) [2023-10-14 21:13:48,911][61585] Updated weights for policy 1, policy_version 85810 (0.0007) [2023-10-14 21:13:49,266][61585] Updated weights for policy 1, policy_version 85820 (0.0008) [2023-10-14 21:13:49,547][61552] Updated weights for policy 0, policy_version 85962 (0.0008) [2023-10-14 21:13:49,919][61552] Updated weights for policy 0, policy_version 85972 (0.0009) [2023-10-14 21:13:50,294][61552] Updated weights for policy 0, policy_version 85982 (0.0008) [2023-10-14 21:13:53,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 175931392. Throughput: 0: 1678.6, 1: 1664.1. Samples: 43993968. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-14 21:13:53,344][60425] Avg episode reward: [(0, '78.500'), (1, '76.620')] [2023-10-14 21:13:53,470][61585] Updated weights for policy 1, policy_version 85830 (0.0008) [2023-10-14 21:13:53,830][61585] Updated weights for policy 1, policy_version 85840 (0.0008) [2023-10-14 21:13:54,191][61585] Updated weights for policy 1, policy_version 85850 (0.0008) [2023-10-14 21:13:54,372][61552] Updated weights for policy 0, policy_version 85992 (0.0007) [2023-10-14 21:13:54,745][61552] Updated weights for policy 0, policy_version 86002 (0.0009) [2023-10-14 21:13:55,114][61552] Updated weights for policy 0, policy_version 86012 (0.0009) [2023-10-14 21:13:58,289][61585] Updated weights for policy 1, policy_version 85860 (0.0008) [2023-10-14 21:13:58,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 175996928. Throughput: 0: 1674.3, 1: 1669.3. Samples: 44014448. Policy #0 lag: (min: 6.0, avg: 9.5, max: 38.0) [2023-10-14 21:13:58,344][60425] Avg episode reward: [(0, '82.140'), (1, '80.310')] [2023-10-14 21:13:58,655][61585] Updated weights for policy 1, policy_version 85870 (0.0008) [2023-10-14 21:13:59,026][61585] Updated weights for policy 1, policy_version 85880 (0.0008) [2023-10-14 21:13:59,333][61552] Updated weights for policy 0, policy_version 86022 (0.0008) [2023-10-14 21:13:59,701][61552] Updated weights for policy 0, policy_version 86032 (0.0008) [2023-10-14 21:14:00,061][61552] Updated weights for policy 0, policy_version 86042 (0.0009) [2023-10-14 21:14:03,134][61585] Updated weights for policy 1, policy_version 85890 (0.0007) [2023-10-14 21:14:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 176062464. Throughput: 0: 1657.4, 1: 1671.4. Samples: 44023506. Policy #0 lag: (min: 6.0, avg: 9.5, max: 38.0) [2023-10-14 21:14:03,344][60425] Avg episode reward: [(0, '80.850'), (1, '77.100')] [2023-10-14 21:14:03,503][61585] Updated weights for policy 1, policy_version 85900 (0.0008) [2023-10-14 21:14:03,862][61585] Updated weights for policy 1, policy_version 85910 (0.0008) [2023-10-14 21:14:04,206][61552] Updated weights for policy 0, policy_version 86052 (0.0010) [2023-10-14 21:14:04,224][61585] Updated weights for policy 1, policy_version 85920 (0.0010) [2023-10-14 21:14:04,569][61552] Updated weights for policy 0, policy_version 86062 (0.0010) [2023-10-14 21:14:04,940][61552] Updated weights for policy 0, policy_version 86072 (0.0011) [2023-10-14 21:14:08,240][61585] Updated weights for policy 1, policy_version 85930 (0.0011) [2023-10-14 21:14:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 176128000. Throughput: 0: 1666.7, 1: 1678.0. Samples: 44044176. Policy #0 lag: (min: 6.0, avg: 9.5, max: 38.0) [2023-10-14 21:14:08,344][60425] Avg episode reward: [(0, '79.520'), (1, '79.550')] [2023-10-14 21:14:08,610][61585] Updated weights for policy 1, policy_version 85940 (0.0010) [2023-10-14 21:14:08,987][61585] Updated weights for policy 1, policy_version 85950 (0.0009) [2023-10-14 21:14:09,185][61552] Updated weights for policy 0, policy_version 86082 (0.0009) [2023-10-14 21:14:09,552][61552] Updated weights for policy 0, policy_version 86092 (0.0010) [2023-10-14 21:14:09,912][61552] Updated weights for policy 0, policy_version 86102 (0.0011) [2023-10-14 21:14:10,280][61552] Updated weights for policy 0, policy_version 86112 (0.0008) [2023-10-14 21:14:13,079][61585] Updated weights for policy 1, policy_version 85960 (0.0010) [2023-10-14 21:14:13,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 176193536. Throughput: 0: 1659.7, 1: 1678.9. Samples: 44064466. Policy #0 lag: (min: 6.0, avg: 9.5, max: 38.0) [2023-10-14 21:14:13,344][60425] Avg episode reward: [(0, '82.610'), (1, '81.480')] [2023-10-14 21:14:13,451][61585] Updated weights for policy 1, policy_version 85970 (0.0011) [2023-10-14 21:14:13,811][61585] Updated weights for policy 1, policy_version 85980 (0.0011) [2023-10-14 21:14:14,470][61552] Updated weights for policy 0, policy_version 86122 (0.0008) [2023-10-14 21:14:14,828][61552] Updated weights for policy 0, policy_version 86132 (0.0008) [2023-10-14 21:14:15,204][61552] Updated weights for policy 0, policy_version 86142 (0.0007) [2023-10-14 21:14:17,965][61585] Updated weights for policy 1, policy_version 85990 (0.0009) [2023-10-14 21:14:18,340][61585] Updated weights for policy 1, policy_version 86000 (0.0009) [2023-10-14 21:14:18,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 176259072. Throughput: 0: 1652.6, 1: 1682.4. Samples: 44073552. Policy #0 lag: (min: 6.0, avg: 9.5, max: 38.0) [2023-10-14 21:14:18,344][60425] Avg episode reward: [(0, '80.730'), (1, '79.380')] [2023-10-14 21:14:18,703][61585] Updated weights for policy 1, policy_version 86010 (0.0009) [2023-10-14 21:14:19,212][61552] Updated weights for policy 0, policy_version 86152 (0.0009) [2023-10-14 21:14:19,575][61552] Updated weights for policy 0, policy_version 86162 (0.0008) [2023-10-14 21:14:19,943][61552] Updated weights for policy 0, policy_version 86172 (0.0009) [2023-10-14 21:14:22,694][61585] Updated weights for policy 1, policy_version 86020 (0.0009) [2023-10-14 21:14:23,056][61585] Updated weights for policy 1, policy_version 86030 (0.0009) [2023-10-14 21:14:23,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 176324608. Throughput: 0: 1661.6, 1: 1672.0. Samples: 44094014. Policy #0 lag: (min: 6.0, avg: 9.5, max: 38.0) [2023-10-14 21:14:23,344][60425] Avg episode reward: [(0, '81.910'), (1, '78.000')] [2023-10-14 21:14:23,416][61585] Updated weights for policy 1, policy_version 86040 (0.0007) [2023-10-14 21:14:24,063][61552] Updated weights for policy 0, policy_version 86182 (0.0009) [2023-10-14 21:14:24,426][61552] Updated weights for policy 0, policy_version 86192 (0.0011) [2023-10-14 21:14:24,789][61552] Updated weights for policy 0, policy_version 86202 (0.0010) [2023-10-14 21:14:27,504][61585] Updated weights for policy 1, policy_version 86050 (0.0007) [2023-10-14 21:14:27,875][61585] Updated weights for policy 1, policy_version 86060 (0.0008) [2023-10-14 21:14:28,237][61585] Updated weights for policy 1, policy_version 86070 (0.0007) [2023-10-14 21:14:28,343][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 176390144. Throughput: 0: 1656.7, 1: 1669.6. Samples: 44114224. Policy #0 lag: (min: 6.0, avg: 9.5, max: 38.0) [2023-10-14 21:14:28,344][60425] Avg episode reward: [(0, '79.940'), (1, '81.790')] [2023-10-14 21:14:28,597][61585] Updated weights for policy 1, policy_version 86080 (0.0007) [2023-10-14 21:14:28,916][61552] Updated weights for policy 0, policy_version 86212 (0.0008) [2023-10-14 21:14:29,285][61552] Updated weights for policy 0, policy_version 86222 (0.0007) [2023-10-14 21:14:29,653][61552] Updated weights for policy 0, policy_version 86232 (0.0008) [2023-10-14 21:14:32,748][61585] Updated weights for policy 1, policy_version 86090 (0.0010) [2023-10-14 21:14:33,114][61585] Updated weights for policy 1, policy_version 86100 (0.0008) [2023-10-14 21:14:33,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 176455680. Throughput: 0: 1656.5, 1: 1675.6. Samples: 44123532. Policy #0 lag: (min: 6.0, avg: 9.5, max: 38.0) [2023-10-14 21:14:33,344][60425] Avg episode reward: [(0, '81.430'), (1, '81.000')] [2023-10-14 21:14:33,479][61585] Updated weights for policy 1, policy_version 86110 (0.0010) [2023-10-14 21:14:33,733][61552] Updated weights for policy 0, policy_version 86242 (0.0010) [2023-10-14 21:14:34,097][61552] Updated weights for policy 0, policy_version 86252 (0.0011) [2023-10-14 21:14:34,466][61552] Updated weights for policy 0, policy_version 86262 (0.0010) [2023-10-14 21:14:34,829][61552] Updated weights for policy 0, policy_version 86272 (0.0009) [2023-10-14 21:14:37,528][61585] Updated weights for policy 1, policy_version 86120 (0.0010) [2023-10-14 21:14:37,889][61585] Updated weights for policy 1, policy_version 86130 (0.0008) [2023-10-14 21:14:38,259][61585] Updated weights for policy 1, policy_version 86140 (0.0009) [2023-10-14 21:14:38,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 176521216. Throughput: 0: 1662.1, 1: 1675.6. Samples: 44144162. Policy #0 lag: (min: 6.0, avg: 9.5, max: 38.0) [2023-10-14 21:14:38,344][60425] Avg episode reward: [(0, '80.810'), (1, '80.740')] [2023-10-14 21:14:38,937][61552] Updated weights for policy 0, policy_version 86282 (0.0010) [2023-10-14 21:14:39,313][61552] Updated weights for policy 0, policy_version 86292 (0.0007) [2023-10-14 21:14:39,688][61552] Updated weights for policy 0, policy_version 86302 (0.0008) [2023-10-14 21:14:42,535][61585] Updated weights for policy 1, policy_version 86150 (0.0008) [2023-10-14 21:14:42,893][61585] Updated weights for policy 1, policy_version 86160 (0.0008) [2023-10-14 21:14:43,256][61585] Updated weights for policy 1, policy_version 86170 (0.0008) [2023-10-14 21:14:43,343][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 176586752. Throughput: 0: 1672.2, 1: 1660.1. Samples: 44164402. Policy #0 lag: (min: 6.0, avg: 9.5, max: 38.0) [2023-10-14 21:14:43,345][60425] Avg episode reward: [(0, '79.670'), (1, '77.080')] [2023-10-14 21:14:43,716][61552] Updated weights for policy 0, policy_version 86312 (0.0010) [2023-10-14 21:14:44,065][61552] Updated weights for policy 0, policy_version 86322 (0.0010) [2023-10-14 21:14:44,437][61552] Updated weights for policy 0, policy_version 86332 (0.0011) [2023-10-14 21:14:47,280][61585] Updated weights for policy 1, policy_version 86180 (0.0008) [2023-10-14 21:14:47,642][61585] Updated weights for policy 1, policy_version 86190 (0.0008) [2023-10-14 21:14:48,008][61585] Updated weights for policy 1, policy_version 86200 (0.0007) [2023-10-14 21:14:48,343][60425] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 176685056. Throughput: 0: 1671.8, 1: 1672.8. Samples: 44174016. Policy #0 lag: (min: 6.0, avg: 9.5, max: 38.0) [2023-10-14 21:14:48,344][60425] Avg episode reward: [(0, '82.140'), (1, '79.720')] [2023-10-14 21:14:48,574][61552] Updated weights for policy 0, policy_version 86342 (0.0009) [2023-10-14 21:14:48,952][61552] Updated weights for policy 0, policy_version 86352 (0.0010) [2023-10-14 21:14:49,320][61552] Updated weights for policy 0, policy_version 86362 (0.0010) [2023-10-14 21:14:52,134][61585] Updated weights for policy 1, policy_version 86210 (0.0009) [2023-10-14 21:14:52,502][61585] Updated weights for policy 1, policy_version 86220 (0.0007) [2023-10-14 21:14:52,867][61585] Updated weights for policy 1, policy_version 86230 (0.0007) [2023-10-14 21:14:53,224][61585] Updated weights for policy 1, policy_version 86240 (0.0007) [2023-10-14 21:14:53,338][61552] Updated weights for policy 0, policy_version 86372 (0.0009) [2023-10-14 21:14:53,343][60425] Fps is (10 sec: 16384.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 176750592. Throughput: 0: 1674.3, 1: 1667.8. Samples: 44194570. Policy #0 lag: (min: 9.0, avg: 19.4, max: 41.0) [2023-10-14 21:14:53,344][60425] Avg episode reward: [(0, '79.050'), (1, '76.830')] [2023-10-14 21:14:53,708][61552] Updated weights for policy 0, policy_version 86382 (0.0008) [2023-10-14 21:14:54,080][61552] Updated weights for policy 0, policy_version 86392 (0.0008) [2023-10-14 21:14:57,249][61585] Updated weights for policy 1, policy_version 86250 (0.0010) [2023-10-14 21:14:57,610][61585] Updated weights for policy 1, policy_version 86260 (0.0008) [2023-10-14 21:14:57,965][61585] Updated weights for policy 1, policy_version 86270 (0.0008) [2023-10-14 21:14:58,213][61552] Updated weights for policy 0, policy_version 86402 (0.0012) [2023-10-14 21:14:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 176816128. Throughput: 0: 1681.1, 1: 1650.6. Samples: 44214392. Policy #0 lag: (min: 9.0, avg: 19.4, max: 41.0) [2023-10-14 21:14:58,344][60425] Avg episode reward: [(0, '78.300'), (1, '78.960')] [2023-10-14 21:14:58,583][61552] Updated weights for policy 0, policy_version 86412 (0.0008) [2023-10-14 21:14:58,954][61552] Updated weights for policy 0, policy_version 86422 (0.0009) [2023-10-14 21:14:59,325][61552] Updated weights for policy 0, policy_version 86432 (0.0009) [2023-10-14 21:15:02,015][61585] Updated weights for policy 1, policy_version 86280 (0.0007) [2023-10-14 21:15:02,374][61585] Updated weights for policy 1, policy_version 86290 (0.0007) [2023-10-14 21:15:02,740][61585] Updated weights for policy 1, policy_version 86300 (0.0007) [2023-10-14 21:15:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 176881664. Throughput: 0: 1682.2, 1: 1673.1. Samples: 44224540. Policy #0 lag: (min: 9.0, avg: 19.4, max: 41.0) [2023-10-14 21:15:03,344][60425] Avg episode reward: [(0, '80.790'), (1, '80.800')] [2023-10-14 21:15:03,394][61552] Updated weights for policy 0, policy_version 86442 (0.0008) [2023-10-14 21:15:03,767][61552] Updated weights for policy 0, policy_version 86452 (0.0008) [2023-10-14 21:15:04,134][61552] Updated weights for policy 0, policy_version 86462 (0.0010) [2023-10-14 21:15:06,951][61585] Updated weights for policy 1, policy_version 86310 (0.0009) [2023-10-14 21:15:07,324][61585] Updated weights for policy 1, policy_version 86320 (0.0010) [2023-10-14 21:15:07,695][61585] Updated weights for policy 1, policy_version 86330 (0.0010) [2023-10-14 21:15:08,071][61552] Updated weights for policy 0, policy_version 86472 (0.0007) [2023-10-14 21:15:08,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 176947200. Throughput: 0: 1676.0, 1: 1674.9. Samples: 44244806. Policy #0 lag: (min: 9.0, avg: 19.4, max: 41.0) [2023-10-14 21:15:08,344][60425] Avg episode reward: [(0, '79.170'), (1, '78.180')] [2023-10-14 21:15:08,440][61552] Updated weights for policy 0, policy_version 86482 (0.0008) [2023-10-14 21:15:08,817][61552] Updated weights for policy 0, policy_version 86492 (0.0007) [2023-10-14 21:15:11,740][61585] Updated weights for policy 1, policy_version 86340 (0.0008) [2023-10-14 21:15:12,098][61585] Updated weights for policy 1, policy_version 86350 (0.0010) [2023-10-14 21:15:12,459][61585] Updated weights for policy 1, policy_version 86360 (0.0008) [2023-10-14 21:15:12,973][61552] Updated weights for policy 0, policy_version 86502 (0.0009) [2023-10-14 21:15:13,335][61552] Updated weights for policy 0, policy_version 86512 (0.0010) [2023-10-14 21:15:13,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 177012736. Throughput: 0: 1679.2, 1: 1653.0. Samples: 44264174. Policy #0 lag: (min: 9.0, avg: 19.4, max: 41.0) [2023-10-14 21:15:13,344][60425] Avg episode reward: [(0, '77.950'), (1, '75.130')] [2023-10-14 21:15:13,707][61552] Updated weights for policy 0, policy_version 86522 (0.0009) [2023-10-14 21:15:16,520][61585] Updated weights for policy 1, policy_version 86370 (0.0009) [2023-10-14 21:15:16,877][61585] Updated weights for policy 1, policy_version 86380 (0.0010) [2023-10-14 21:15:17,244][61585] Updated weights for policy 1, policy_version 86390 (0.0009) [2023-10-14 21:15:17,613][61585] Updated weights for policy 1, policy_version 86400 (0.0010) [2023-10-14 21:15:17,870][61552] Updated weights for policy 0, policy_version 86532 (0.0009) [2023-10-14 21:15:18,242][61552] Updated weights for policy 0, policy_version 86542 (0.0007) [2023-10-14 21:15:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 177078272. Throughput: 0: 1675.4, 1: 1677.6. Samples: 44274418. Policy #0 lag: (min: 9.0, avg: 19.4, max: 41.0) [2023-10-14 21:15:18,344][60425] Avg episode reward: [(0, '79.000'), (1, '76.280')] [2023-10-14 21:15:18,619][61552] Updated weights for policy 0, policy_version 86552 (0.0007) [2023-10-14 21:15:21,747][61585] Updated weights for policy 1, policy_version 86410 (0.0009) [2023-10-14 21:15:22,109][61585] Updated weights for policy 1, policy_version 86420 (0.0008) [2023-10-14 21:15:22,468][61585] Updated weights for policy 1, policy_version 86430 (0.0010) [2023-10-14 21:15:22,556][61552] Updated weights for policy 0, policy_version 86562 (0.0009) [2023-10-14 21:15:22,927][61552] Updated weights for policy 0, policy_version 86572 (0.0010) [2023-10-14 21:15:23,287][61552] Updated weights for policy 0, policy_version 86582 (0.0010) [2023-10-14 21:15:23,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 177143808. Throughput: 0: 1678.9, 1: 1670.1. Samples: 44294868. Policy #0 lag: (min: 9.0, avg: 19.4, max: 41.0) [2023-10-14 21:15:23,344][60425] Avg episode reward: [(0, '78.450'), (1, '80.280')] [2023-10-14 21:15:23,658][61552] Updated weights for policy 0, policy_version 86592 (0.0007) [2023-10-14 21:15:26,612][61585] Updated weights for policy 1, policy_version 86440 (0.0008) [2023-10-14 21:15:26,981][61585] Updated weights for policy 1, policy_version 86450 (0.0007) [2023-10-14 21:15:27,354][61585] Updated weights for policy 1, policy_version 86460 (0.0009) [2023-10-14 21:15:27,822][61552] Updated weights for policy 0, policy_version 86602 (0.0007) [2023-10-14 21:15:28,193][61552] Updated weights for policy 0, policy_version 86612 (0.0008) [2023-10-14 21:15:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 177209344. Throughput: 0: 1664.6, 1: 1662.6. Samples: 44314128. Policy #0 lag: (min: 9.0, avg: 19.4, max: 41.0) [2023-10-14 21:15:28,344][60425] Avg episode reward: [(0, '74.830'), (1, '76.150')] [2023-10-14 21:15:28,563][61552] Updated weights for policy 0, policy_version 86622 (0.0010) [2023-10-14 21:15:31,282][61585] Updated weights for policy 1, policy_version 86470 (0.0010) [2023-10-14 21:15:31,650][61585] Updated weights for policy 1, policy_version 86480 (0.0008) [2023-10-14 21:15:32,012][61585] Updated weights for policy 1, policy_version 86490 (0.0008) [2023-10-14 21:15:32,634][61552] Updated weights for policy 0, policy_version 86632 (0.0009) [2023-10-14 21:15:33,008][61552] Updated weights for policy 0, policy_version 86642 (0.0011) [2023-10-14 21:15:33,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 177274880. Throughput: 0: 1676.3, 1: 1678.1. Samples: 44324964. Policy #0 lag: (min: 9.0, avg: 19.4, max: 41.0) [2023-10-14 21:15:33,344][60425] Avg episode reward: [(0, '77.940'), (1, '77.100')] [2023-10-14 21:15:33,375][61552] Updated weights for policy 0, policy_version 86652 (0.0011) [2023-10-14 21:15:36,231][61585] Updated weights for policy 1, policy_version 86500 (0.0009) [2023-10-14 21:15:36,593][61585] Updated weights for policy 1, policy_version 86510 (0.0011) [2023-10-14 21:15:36,958][61585] Updated weights for policy 1, policy_version 86520 (0.0010) [2023-10-14 21:15:37,526][61552] Updated weights for policy 0, policy_version 86662 (0.0009) [2023-10-14 21:15:37,898][61552] Updated weights for policy 0, policy_version 86672 (0.0010) [2023-10-14 21:15:38,261][61552] Updated weights for policy 0, policy_version 86682 (0.0007) [2023-10-14 21:15:38,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 177340416. Throughput: 0: 1675.9, 1: 1663.1. Samples: 44344828. Policy #0 lag: (min: 9.0, avg: 19.4, max: 41.0) [2023-10-14 21:15:38,344][60425] Avg episode reward: [(0, '82.020'), (1, '77.270')] [2023-10-14 21:15:41,029][61585] Updated weights for policy 1, policy_version 86530 (0.0009) [2023-10-14 21:15:41,389][61585] Updated weights for policy 1, policy_version 86540 (0.0009) [2023-10-14 21:15:41,755][61585] Updated weights for policy 1, policy_version 86550 (0.0010) [2023-10-14 21:15:42,126][61585] Updated weights for policy 1, policy_version 86560 (0.0009) [2023-10-14 21:15:42,474][61552] Updated weights for policy 0, policy_version 86692 (0.0009) [2023-10-14 21:15:42,858][61552] Updated weights for policy 0, policy_version 86702 (0.0010) [2023-10-14 21:15:43,220][61552] Updated weights for policy 0, policy_version 86712 (0.0011) [2023-10-14 21:15:43,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 177405952. Throughput: 0: 1666.6, 1: 1667.4. Samples: 44364424. Policy #0 lag: (min: 9.0, avg: 19.4, max: 41.0) [2023-10-14 21:15:43,345][60425] Avg episode reward: [(0, '76.130'), (1, '77.830')] [2023-10-14 21:15:43,355][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000086560_88637440.pth... [2023-10-14 21:15:43,395][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000084992_87031808.pth [2023-10-14 21:15:43,515][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000086720_88801280.pth... [2023-10-14 21:15:43,553][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000085152_87195648.pth [2023-10-14 21:15:46,212][61585] Updated weights for policy 1, policy_version 86570 (0.0007) [2023-10-14 21:15:46,582][61585] Updated weights for policy 1, policy_version 86580 (0.0007) [2023-10-14 21:15:46,945][61585] Updated weights for policy 1, policy_version 86590 (0.0008) [2023-10-14 21:15:47,295][61552] Updated weights for policy 0, policy_version 86722 (0.0010) [2023-10-14 21:15:47,674][61552] Updated weights for policy 0, policy_version 86732 (0.0007) [2023-10-14 21:15:48,047][61552] Updated weights for policy 0, policy_version 86742 (0.0009) [2023-10-14 21:15:48,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 177471488. Throughput: 0: 1672.3, 1: 1672.3. Samples: 44375048. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:15:48,344][60425] Avg episode reward: [(0, '76.200'), (1, '75.590')] [2023-10-14 21:15:48,414][61552] Updated weights for policy 0, policy_version 86752 (0.0009) [2023-10-14 21:15:50,959][61585] Updated weights for policy 1, policy_version 86600 (0.0008) [2023-10-14 21:15:51,323][61585] Updated weights for policy 1, policy_version 86610 (0.0011) [2023-10-14 21:15:51,681][61585] Updated weights for policy 1, policy_version 86620 (0.0010) [2023-10-14 21:15:52,648][61552] Updated weights for policy 0, policy_version 86762 (0.0009) [2023-10-14 21:15:53,026][61552] Updated weights for policy 0, policy_version 86772 (0.0009) [2023-10-14 21:15:53,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 177537024. Throughput: 0: 1673.8, 1: 1653.7. Samples: 44394546. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:15:53,344][60425] Avg episode reward: [(0, '78.280'), (1, '76.130')] [2023-10-14 21:15:53,392][61552] Updated weights for policy 0, policy_version 86782 (0.0009) [2023-10-14 21:15:55,931][61585] Updated weights for policy 1, policy_version 86630 (0.0010) [2023-10-14 21:15:56,326][61585] Updated weights for policy 1, policy_version 86640 (0.0009) [2023-10-14 21:15:56,701][61585] Updated weights for policy 1, policy_version 86650 (0.0009) [2023-10-14 21:15:57,447][61552] Updated weights for policy 0, policy_version 86792 (0.0008) [2023-10-14 21:15:57,815][61552] Updated weights for policy 0, policy_version 86802 (0.0007) [2023-10-14 21:15:58,175][61552] Updated weights for policy 0, policy_version 86812 (0.0007) [2023-10-14 21:15:58,343][60425] Fps is (10 sec: 16383.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 177635328. Throughput: 0: 1659.4, 1: 1673.7. Samples: 44414162. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:15:58,344][60425] Avg episode reward: [(0, '79.350'), (1, '75.970')] [2023-10-14 21:16:00,824][61585] Updated weights for policy 1, policy_version 86660 (0.0010) [2023-10-14 21:16:01,192][61585] Updated weights for policy 1, policy_version 86670 (0.0008) [2023-10-14 21:16:01,553][61585] Updated weights for policy 1, policy_version 86680 (0.0009) [2023-10-14 21:16:02,299][61552] Updated weights for policy 0, policy_version 86822 (0.0008) [2023-10-14 21:16:02,667][61552] Updated weights for policy 0, policy_version 86832 (0.0008) [2023-10-14 21:16:03,036][61552] Updated weights for policy 0, policy_version 86842 (0.0007) [2023-10-14 21:16:03,343][60425] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 177700864. Throughput: 0: 1677.1, 1: 1668.1. Samples: 44424952. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:16:03,344][60425] Avg episode reward: [(0, '77.750'), (1, '75.520')] [2023-10-14 21:16:05,622][61585] Updated weights for policy 1, policy_version 86690 (0.0007) [2023-10-14 21:16:05,986][61585] Updated weights for policy 1, policy_version 86700 (0.0010) [2023-10-14 21:16:06,355][61585] Updated weights for policy 1, policy_version 86710 (0.0007) [2023-10-14 21:16:06,724][61585] Updated weights for policy 1, policy_version 86720 (0.0008) [2023-10-14 21:16:07,022][61552] Updated weights for policy 0, policy_version 86852 (0.0008) [2023-10-14 21:16:07,384][61552] Updated weights for policy 0, policy_version 86862 (0.0008) [2023-10-14 21:16:07,749][61552] Updated weights for policy 0, policy_version 86872 (0.0008) [2023-10-14 21:16:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 177766400. Throughput: 0: 1675.5, 1: 1650.8. Samples: 44444550. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:16:08,344][60425] Avg episode reward: [(0, '77.590'), (1, '80.660')] [2023-10-14 21:16:10,817][61585] Updated weights for policy 1, policy_version 86730 (0.0010) [2023-10-14 21:16:11,189][61585] Updated weights for policy 1, policy_version 86740 (0.0009) [2023-10-14 21:16:11,555][61585] Updated weights for policy 1, policy_version 86750 (0.0009) [2023-10-14 21:16:11,817][61552] Updated weights for policy 0, policy_version 86882 (0.0009) [2023-10-14 21:16:12,194][61552] Updated weights for policy 0, policy_version 86892 (0.0008) [2023-10-14 21:16:12,551][61552] Updated weights for policy 0, policy_version 86902 (0.0010) [2023-10-14 21:16:12,928][61552] Updated weights for policy 0, policy_version 86912 (0.0009) [2023-10-14 21:16:13,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 177831936. Throughput: 0: 1663.1, 1: 1671.4. Samples: 44464180. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:16:13,344][60425] Avg episode reward: [(0, '79.400'), (1, '80.420')] [2023-10-14 21:16:15,750][61585] Updated weights for policy 1, policy_version 86760 (0.0009) [2023-10-14 21:16:16,111][61585] Updated weights for policy 1, policy_version 86770 (0.0010) [2023-10-14 21:16:16,465][61585] Updated weights for policy 1, policy_version 86780 (0.0010) [2023-10-14 21:16:16,972][61552] Updated weights for policy 0, policy_version 86922 (0.0007) [2023-10-14 21:16:17,343][61552] Updated weights for policy 0, policy_version 86932 (0.0008) [2023-10-14 21:16:17,704][61552] Updated weights for policy 0, policy_version 86942 (0.0009) [2023-10-14 21:16:18,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 177897472. Throughput: 0: 1672.2, 1: 1664.4. Samples: 44475112. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:16:18,344][60425] Avg episode reward: [(0, '79.510'), (1, '76.030')] [2023-10-14 21:16:20,429][61585] Updated weights for policy 1, policy_version 86790 (0.0009) [2023-10-14 21:16:20,798][61585] Updated weights for policy 1, policy_version 86800 (0.0009) [2023-10-14 21:16:21,158][61585] Updated weights for policy 1, policy_version 86810 (0.0010) [2023-10-14 21:16:21,810][61552] Updated weights for policy 0, policy_version 86952 (0.0011) [2023-10-14 21:16:22,181][61552] Updated weights for policy 0, policy_version 86962 (0.0009) [2023-10-14 21:16:22,551][61552] Updated weights for policy 0, policy_version 86972 (0.0010) [2023-10-14 21:16:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 177963008. Throughput: 0: 1669.6, 1: 1659.3. Samples: 44494630. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:16:23,344][60425] Avg episode reward: [(0, '78.310'), (1, '82.460')] [2023-10-14 21:16:25,429][61585] Updated weights for policy 1, policy_version 86820 (0.0011) [2023-10-14 21:16:25,793][61585] Updated weights for policy 1, policy_version 86830 (0.0009) [2023-10-14 21:16:26,166][61585] Updated weights for policy 1, policy_version 86840 (0.0009) [2023-10-14 21:16:26,646][61552] Updated weights for policy 0, policy_version 86982 (0.0008) [2023-10-14 21:16:27,010][61552] Updated weights for policy 0, policy_version 86992 (0.0008) [2023-10-14 21:16:27,371][61552] Updated weights for policy 0, policy_version 87002 (0.0009) [2023-10-14 21:16:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 178028544. Throughput: 0: 1651.2, 1: 1672.8. Samples: 44514004. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:16:28,344][60425] Avg episode reward: [(0, '80.230'), (1, '83.280')] [2023-10-14 21:16:30,347][61585] Updated weights for policy 1, policy_version 86850 (0.0009) [2023-10-14 21:16:30,714][61585] Updated weights for policy 1, policy_version 86860 (0.0009) [2023-10-14 21:16:31,082][61585] Updated weights for policy 1, policy_version 86870 (0.0008) [2023-10-14 21:16:31,392][61552] Updated weights for policy 0, policy_version 87012 (0.0009) [2023-10-14 21:16:31,444][61585] Updated weights for policy 1, policy_version 86880 (0.0008) [2023-10-14 21:16:31,759][61552] Updated weights for policy 0, policy_version 87022 (0.0011) [2023-10-14 21:16:32,130][61552] Updated weights for policy 0, policy_version 87032 (0.0010) [2023-10-14 21:16:33,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 178094080. Throughput: 0: 1677.6, 1: 1662.0. Samples: 44525330. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:16:33,344][60425] Avg episode reward: [(0, '78.040'), (1, '76.520')] [2023-10-14 21:16:35,544][61585] Updated weights for policy 1, policy_version 86890 (0.0010) [2023-10-14 21:16:35,917][61585] Updated weights for policy 1, policy_version 86900 (0.0008) [2023-10-14 21:16:36,091][61552] Updated weights for policy 0, policy_version 87042 (0.0008) [2023-10-14 21:16:36,275][61585] Updated weights for policy 1, policy_version 86910 (0.0007) [2023-10-14 21:16:36,452][61552] Updated weights for policy 0, policy_version 87052 (0.0009) [2023-10-14 21:16:36,813][61552] Updated weights for policy 0, policy_version 87062 (0.0008) [2023-10-14 21:16:37,184][61552] Updated weights for policy 0, policy_version 87072 (0.0008) [2023-10-14 21:16:38,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 178159616. Throughput: 0: 1664.4, 1: 1667.1. Samples: 44544462. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-14 21:16:38,344][60425] Avg episode reward: [(0, '78.440'), (1, '78.530')] [2023-10-14 21:16:40,383][61585] Updated weights for policy 1, policy_version 86920 (0.0009) [2023-10-14 21:16:40,766][61585] Updated weights for policy 1, policy_version 86930 (0.0009) [2023-10-14 21:16:41,133][61585] Updated weights for policy 1, policy_version 86940 (0.0007) [2023-10-14 21:16:41,197][61552] Updated weights for policy 0, policy_version 87082 (0.0008) [2023-10-14 21:16:41,574][61552] Updated weights for policy 0, policy_version 87092 (0.0007) [2023-10-14 21:16:41,937][61552] Updated weights for policy 0, policy_version 87102 (0.0009) [2023-10-14 21:16:43,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 178225152. Throughput: 0: 1670.1, 1: 1669.5. Samples: 44564442. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-14 21:16:43,344][60425] Avg episode reward: [(0, '75.420'), (1, '81.390')] [2023-10-14 21:16:45,324][61585] Updated weights for policy 1, policy_version 86950 (0.0008) [2023-10-14 21:16:45,691][61585] Updated weights for policy 1, policy_version 86960 (0.0008) [2023-10-14 21:16:45,903][61552] Updated weights for policy 0, policy_version 87112 (0.0008) [2023-10-14 21:16:46,064][61585] Updated weights for policy 1, policy_version 86970 (0.0007) [2023-10-14 21:16:46,271][61552] Updated weights for policy 0, policy_version 87122 (0.0008) [2023-10-14 21:16:46,648][61552] Updated weights for policy 0, policy_version 87132 (0.0009) [2023-10-14 21:16:48,344][60425] Fps is (10 sec: 13106.4, 60 sec: 13653.2, 300 sec: 13329.3). Total num frames: 178290688. Throughput: 0: 1682.2, 1: 1656.2. Samples: 44575186. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-14 21:16:48,344][60425] Avg episode reward: [(0, '81.520'), (1, '81.150')] [2023-10-14 21:16:50,156][61585] Updated weights for policy 1, policy_version 86980 (0.0009) [2023-10-14 21:16:50,521][61585] Updated weights for policy 1, policy_version 86990 (0.0010) [2023-10-14 21:16:50,788][61552] Updated weights for policy 0, policy_version 87142 (0.0007) [2023-10-14 21:16:50,888][61585] Updated weights for policy 1, policy_version 87000 (0.0009) [2023-10-14 21:16:51,157][61552] Updated weights for policy 0, policy_version 87152 (0.0007) [2023-10-14 21:16:51,526][61552] Updated weights for policy 0, policy_version 87162 (0.0011) [2023-10-14 21:16:53,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 178356224. Throughput: 0: 1655.0, 1: 1665.1. Samples: 44593954. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-14 21:16:53,344][60425] Avg episode reward: [(0, '79.790'), (1, '79.070')] [2023-10-14 21:16:55,021][61585] Updated weights for policy 1, policy_version 87010 (0.0009) [2023-10-14 21:16:55,378][61585] Updated weights for policy 1, policy_version 87020 (0.0008) [2023-10-14 21:16:55,740][61585] Updated weights for policy 1, policy_version 87030 (0.0009) [2023-10-14 21:16:55,800][61552] Updated weights for policy 0, policy_version 87172 (0.0007) [2023-10-14 21:16:56,107][61585] Updated weights for policy 1, policy_version 87040 (0.0008) [2023-10-14 21:16:56,169][61552] Updated weights for policy 0, policy_version 87182 (0.0008) [2023-10-14 21:16:56,526][61552] Updated weights for policy 0, policy_version 87192 (0.0010) [2023-10-14 21:16:58,343][60425] Fps is (10 sec: 13108.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 178421760. Throughput: 0: 1670.0, 1: 1664.8. Samples: 44614248. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-14 21:16:58,344][60425] Avg episode reward: [(0, '77.900'), (1, '79.110')] [2023-10-14 21:17:00,178][61585] Updated weights for policy 1, policy_version 87050 (0.0011) [2023-10-14 21:17:00,549][61585] Updated weights for policy 1, policy_version 87060 (0.0008) [2023-10-14 21:17:00,575][61552] Updated weights for policy 0, policy_version 87202 (0.0010) [2023-10-14 21:17:00,915][61585] Updated weights for policy 1, policy_version 87070 (0.0007) [2023-10-14 21:17:00,938][61552] Updated weights for policy 0, policy_version 87212 (0.0009) [2023-10-14 21:17:01,314][61552] Updated weights for policy 0, policy_version 87222 (0.0008) [2023-10-14 21:17:01,669][61552] Updated weights for policy 0, policy_version 87232 (0.0010) [2023-10-14 21:17:03,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 178487296. Throughput: 0: 1675.5, 1: 1649.0. Samples: 44624714. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-14 21:17:03,344][60425] Avg episode reward: [(0, '75.360'), (1, '79.510')] [2023-10-14 21:17:04,935][61585] Updated weights for policy 1, policy_version 87080 (0.0008) [2023-10-14 21:17:05,298][61585] Updated weights for policy 1, policy_version 87090 (0.0009) [2023-10-14 21:17:05,663][61585] Updated weights for policy 1, policy_version 87100 (0.0008) [2023-10-14 21:17:05,742][61552] Updated weights for policy 0, policy_version 87242 (0.0009) [2023-10-14 21:17:06,100][61552] Updated weights for policy 0, policy_version 87252 (0.0008) [2023-10-14 21:17:06,467][61552] Updated weights for policy 0, policy_version 87262 (0.0009) [2023-10-14 21:17:08,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 178552832. Throughput: 0: 1652.4, 1: 1663.5. Samples: 44643846. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-14 21:17:08,344][60425] Avg episode reward: [(0, '78.810'), (1, '78.010')] [2023-10-14 21:17:09,598][61585] Updated weights for policy 1, policy_version 87110 (0.0008) [2023-10-14 21:17:09,952][61585] Updated weights for policy 1, policy_version 87120 (0.0010) [2023-10-14 21:17:10,312][61585] Updated weights for policy 1, policy_version 87130 (0.0008) [2023-10-14 21:17:10,633][61552] Updated weights for policy 0, policy_version 87272 (0.0008) [2023-10-14 21:17:11,008][61552] Updated weights for policy 0, policy_version 87282 (0.0008) [2023-10-14 21:17:11,378][61552] Updated weights for policy 0, policy_version 87292 (0.0011) [2023-10-14 21:17:13,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 178618368. Throughput: 0: 1684.1, 1: 1672.5. Samples: 44665052. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-14 21:17:13,345][60425] Avg episode reward: [(0, '81.240'), (1, '79.790')] [2023-10-14 21:17:14,329][61585] Updated weights for policy 1, policy_version 87140 (0.0007) [2023-10-14 21:17:14,695][61585] Updated weights for policy 1, policy_version 87150 (0.0009) [2023-10-14 21:17:15,061][61585] Updated weights for policy 1, policy_version 87160 (0.0010) [2023-10-14 21:17:15,710][61552] Updated weights for policy 0, policy_version 87302 (0.0009) [2023-10-14 21:17:16,082][61552] Updated weights for policy 0, policy_version 87312 (0.0009) [2023-10-14 21:17:16,451][61552] Updated weights for policy 0, policy_version 87322 (0.0008) [2023-10-14 21:17:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 178683904. Throughput: 0: 1671.8, 1: 1658.1. Samples: 44675174. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-14 21:17:18,344][60425] Avg episode reward: [(0, '83.060'), (1, '80.490')] [2023-10-14 21:17:19,139][61585] Updated weights for policy 1, policy_version 87170 (0.0007) [2023-10-14 21:17:19,504][61585] Updated weights for policy 1, policy_version 87180 (0.0009) [2023-10-14 21:17:19,871][61585] Updated weights for policy 1, policy_version 87190 (0.0009) [2023-10-14 21:17:20,238][61585] Updated weights for policy 1, policy_version 87200 (0.0009) [2023-10-14 21:17:20,695][61552] Updated weights for policy 0, policy_version 87332 (0.0010) [2023-10-14 21:17:21,066][61552] Updated weights for policy 0, policy_version 87342 (0.0007) [2023-10-14 21:17:21,430][61552] Updated weights for policy 0, policy_version 87352 (0.0011) [2023-10-14 21:17:23,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 178749440. Throughput: 0: 1659.6, 1: 1684.8. Samples: 44694960. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-14 21:17:23,344][60425] Avg episode reward: [(0, '78.860'), (1, '76.750')] [2023-10-14 21:17:24,393][61585] Updated weights for policy 1, policy_version 87210 (0.0007) [2023-10-14 21:17:24,770][61585] Updated weights for policy 1, policy_version 87220 (0.0009) [2023-10-14 21:17:25,123][61585] Updated weights for policy 1, policy_version 87230 (0.0010) [2023-10-14 21:17:25,346][61552] Updated weights for policy 0, policy_version 87362 (0.0008) [2023-10-14 21:17:25,712][61552] Updated weights for policy 0, policy_version 87372 (0.0012) [2023-10-14 21:17:26,080][61552] Updated weights for policy 0, policy_version 87382 (0.0009) [2023-10-14 21:17:26,448][61552] Updated weights for policy 0, policy_version 87392 (0.0009) [2023-10-14 21:17:28,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 178814976. Throughput: 0: 1672.1, 1: 1690.7. Samples: 44715768. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-14 21:17:28,344][60425] Avg episode reward: [(0, '80.180'), (1, '71.060')] [2023-10-14 21:17:29,125][61585] Updated weights for policy 1, policy_version 87240 (0.0008) [2023-10-14 21:17:29,512][61585] Updated weights for policy 1, policy_version 87250 (0.0008) [2023-10-14 21:17:29,872][61585] Updated weights for policy 1, policy_version 87260 (0.0011) [2023-10-14 21:17:30,709][61552] Updated weights for policy 0, policy_version 87402 (0.0008) [2023-10-14 21:17:31,075][61552] Updated weights for policy 0, policy_version 87412 (0.0008) [2023-10-14 21:17:31,440][61552] Updated weights for policy 0, policy_version 87422 (0.0010) [2023-10-14 21:17:33,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 178880512. Throughput: 0: 1663.2, 1: 1678.8. Samples: 44725576. Policy #0 lag: (min: 17.0, avg: 31.3, max: 49.0) [2023-10-14 21:17:33,344][60425] Avg episode reward: [(0, '82.060'), (1, '79.960')] [2023-10-14 21:17:33,740][61585] Updated weights for policy 1, policy_version 87270 (0.0009) [2023-10-14 21:17:34,104][61585] Updated weights for policy 1, policy_version 87280 (0.0009) [2023-10-14 21:17:34,471][61585] Updated weights for policy 1, policy_version 87290 (0.0010) [2023-10-14 21:17:35,414][61552] Updated weights for policy 0, policy_version 87432 (0.0009) [2023-10-14 21:17:35,779][61552] Updated weights for policy 0, policy_version 87442 (0.0011) [2023-10-14 21:17:36,155][61552] Updated weights for policy 0, policy_version 87452 (0.0011) [2023-10-14 21:17:38,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 178946048. Throughput: 0: 1670.7, 1: 1696.3. Samples: 44745470. Policy #0 lag: (min: 17.0, avg: 31.3, max: 49.0) [2023-10-14 21:17:38,344][60425] Avg episode reward: [(0, '75.760'), (1, '80.870')] [2023-10-14 21:17:38,577][61585] Updated weights for policy 1, policy_version 87300 (0.0009) [2023-10-14 21:17:38,948][61585] Updated weights for policy 1, policy_version 87310 (0.0010) [2023-10-14 21:17:39,313][61585] Updated weights for policy 1, policy_version 87320 (0.0010) [2023-10-14 21:17:40,403][61552] Updated weights for policy 0, policy_version 87462 (0.0009) [2023-10-14 21:17:40,759][61552] Updated weights for policy 0, policy_version 87472 (0.0008) [2023-10-14 21:17:41,134][61552] Updated weights for policy 0, policy_version 87482 (0.0009) [2023-10-14 21:17:43,344][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 179011584. Throughput: 0: 1670.5, 1: 1704.3. Samples: 44766114. Policy #0 lag: (min: 17.0, avg: 31.3, max: 49.0) [2023-10-14 21:17:43,345][60425] Avg episode reward: [(0, '80.590'), (1, '79.020')] [2023-10-14 21:17:43,357][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000087488_89587712.pth... [2023-10-14 21:17:43,392][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000085920_87982080.pth [2023-10-14 21:17:43,398][61585] Updated weights for policy 1, policy_version 87330 (0.0009) [2023-10-14 21:17:43,755][61585] Updated weights for policy 1, policy_version 87340 (0.0010) [2023-10-14 21:17:44,129][61585] Updated weights for policy 1, policy_version 87350 (0.0009) [2023-10-14 21:17:44,492][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000087360_89456640.pth... [2023-10-14 21:17:44,493][61585] Updated weights for policy 1, policy_version 87360 (0.0007) [2023-10-14 21:17:44,520][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000085792_87851008.pth [2023-10-14 21:17:45,175][61552] Updated weights for policy 0, policy_version 87492 (0.0008) [2023-10-14 21:17:45,555][61552] Updated weights for policy 0, policy_version 87502 (0.0010) [2023-10-14 21:17:45,921][61552] Updated weights for policy 0, policy_version 87512 (0.0010) [2023-10-14 21:17:48,325][61585] Updated weights for policy 1, policy_version 87370 (0.0010) [2023-10-14 21:17:48,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.4, 300 sec: 13329.4). Total num frames: 179077120. Throughput: 0: 1657.1, 1: 1702.8. Samples: 44775908. Policy #0 lag: (min: 17.0, avg: 31.3, max: 49.0) [2023-10-14 21:17:48,344][60425] Avg episode reward: [(0, '76.640'), (1, '75.970')] [2023-10-14 21:17:48,686][61585] Updated weights for policy 1, policy_version 87380 (0.0008) [2023-10-14 21:17:49,052][61585] Updated weights for policy 1, policy_version 87390 (0.0008) [2023-10-14 21:17:49,937][61552] Updated weights for policy 0, policy_version 87522 (0.0010) [2023-10-14 21:17:50,303][61552] Updated weights for policy 0, policy_version 87532 (0.0009) [2023-10-14 21:17:50,680][61552] Updated weights for policy 0, policy_version 87542 (0.0007) [2023-10-14 21:17:51,050][61552] Updated weights for policy 0, policy_version 87552 (0.0008) [2023-10-14 21:17:53,111][61585] Updated weights for policy 1, policy_version 87400 (0.0009) [2023-10-14 21:17:53,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 179142656. Throughput: 0: 1668.3, 1: 1710.8. Samples: 44795906. Policy #0 lag: (min: 17.0, avg: 31.3, max: 49.0) [2023-10-14 21:17:53,344][60425] Avg episode reward: [(0, '78.830'), (1, '78.120')] [2023-10-14 21:17:53,476][61585] Updated weights for policy 1, policy_version 87410 (0.0007) [2023-10-14 21:17:53,840][61585] Updated weights for policy 1, policy_version 87420 (0.0008) [2023-10-14 21:17:55,050][61552] Updated weights for policy 0, policy_version 87562 (0.0007) [2023-10-14 21:17:55,422][61552] Updated weights for policy 0, policy_version 87572 (0.0007) [2023-10-14 21:17:55,799][61552] Updated weights for policy 0, policy_version 87582 (0.0008) [2023-10-14 21:17:58,055][61585] Updated weights for policy 1, policy_version 87430 (0.0009) [2023-10-14 21:17:58,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 179208192. Throughput: 0: 1661.1, 1: 1699.1. Samples: 44816258. Policy #0 lag: (min: 17.0, avg: 31.3, max: 49.0) [2023-10-14 21:17:58,345][60425] Avg episode reward: [(0, '72.870'), (1, '77.820')] [2023-10-14 21:17:58,414][61585] Updated weights for policy 1, policy_version 87440 (0.0009) [2023-10-14 21:17:58,782][61585] Updated weights for policy 1, policy_version 87450 (0.0009) [2023-10-14 21:17:59,982][61552] Updated weights for policy 0, policy_version 87592 (0.0009) [2023-10-14 21:18:00,348][61552] Updated weights for policy 0, policy_version 87602 (0.0007) [2023-10-14 21:18:00,714][61552] Updated weights for policy 0, policy_version 87612 (0.0007) [2023-10-14 21:18:02,912][61585] Updated weights for policy 1, policy_version 87460 (0.0010) [2023-10-14 21:18:03,282][61585] Updated weights for policy 1, policy_version 87470 (0.0008) [2023-10-14 21:18:03,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 179273728. Throughput: 0: 1645.0, 1: 1699.2. Samples: 44825660. Policy #0 lag: (min: 17.0, avg: 31.3, max: 49.0) [2023-10-14 21:18:03,344][60425] Avg episode reward: [(0, '73.280'), (1, '81.630')] [2023-10-14 21:18:03,649][61585] Updated weights for policy 1, policy_version 87480 (0.0010) [2023-10-14 21:18:04,844][61552] Updated weights for policy 0, policy_version 87622 (0.0007) [2023-10-14 21:18:05,203][61552] Updated weights for policy 0, policy_version 87632 (0.0007) [2023-10-14 21:18:05,578][61552] Updated weights for policy 0, policy_version 87642 (0.0007) [2023-10-14 21:18:07,771][61585] Updated weights for policy 1, policy_version 87490 (0.0010) [2023-10-14 21:18:08,147][61585] Updated weights for policy 1, policy_version 87500 (0.0009) [2023-10-14 21:18:08,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 179339264. Throughput: 0: 1665.2, 1: 1688.3. Samples: 44845866. Policy #0 lag: (min: 17.0, avg: 31.3, max: 49.0) [2023-10-14 21:18:08,344][60425] Avg episode reward: [(0, '77.690'), (1, '78.810')] [2023-10-14 21:18:08,505][61585] Updated weights for policy 1, policy_version 87510 (0.0009) [2023-10-14 21:18:08,869][61585] Updated weights for policy 1, policy_version 87520 (0.0007) [2023-10-14 21:18:09,696][61552] Updated weights for policy 0, policy_version 87652 (0.0007) [2023-10-14 21:18:10,074][61552] Updated weights for policy 0, policy_version 87662 (0.0008) [2023-10-14 21:18:10,448][61552] Updated weights for policy 0, policy_version 87672 (0.0008) [2023-10-14 21:18:13,094][61585] Updated weights for policy 1, policy_version 87530 (0.0009) [2023-10-14 21:18:13,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 179404800. Throughput: 0: 1665.3, 1: 1685.2. Samples: 44866540. Policy #0 lag: (min: 17.0, avg: 31.3, max: 49.0) [2023-10-14 21:18:13,344][60425] Avg episode reward: [(0, '75.180'), (1, '76.730')] [2023-10-14 21:18:13,466][61585] Updated weights for policy 1, policy_version 87540 (0.0007) [2023-10-14 21:18:13,826][61585] Updated weights for policy 1, policy_version 87550 (0.0008) [2023-10-14 21:18:14,493][61552] Updated weights for policy 0, policy_version 87682 (0.0008) [2023-10-14 21:18:14,858][61552] Updated weights for policy 0, policy_version 87692 (0.0010) [2023-10-14 21:18:15,243][61552] Updated weights for policy 0, policy_version 87702 (0.0010) [2023-10-14 21:18:15,608][61552] Updated weights for policy 0, policy_version 87712 (0.0009) [2023-10-14 21:18:17,901][61585] Updated weights for policy 1, policy_version 87560 (0.0009) [2023-10-14 21:18:18,271][61585] Updated weights for policy 1, policy_version 87570 (0.0009) [2023-10-14 21:18:18,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 179470336. Throughput: 0: 1646.3, 1: 1687.1. Samples: 44875578. Policy #0 lag: (min: 17.0, avg: 31.3, max: 49.0) [2023-10-14 21:18:18,344][60425] Avg episode reward: [(0, '75.670'), (1, '78.330')] [2023-10-14 21:18:18,636][61585] Updated weights for policy 1, policy_version 87580 (0.0008) [2023-10-14 21:18:19,693][61552] Updated weights for policy 0, policy_version 87722 (0.0008) [2023-10-14 21:18:20,058][61552] Updated weights for policy 0, policy_version 87732 (0.0009) [2023-10-14 21:18:20,426][61552] Updated weights for policy 0, policy_version 87742 (0.0008) [2023-10-14 21:18:22,549][61585] Updated weights for policy 1, policy_version 87590 (0.0009) [2023-10-14 21:18:22,924][61585] Updated weights for policy 1, policy_version 87600 (0.0008) [2023-10-14 21:18:23,287][61585] Updated weights for policy 1, policy_version 87610 (0.0008) [2023-10-14 21:18:23,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 179535872. Throughput: 0: 1666.6, 1: 1686.0. Samples: 44896336. Policy #0 lag: (min: 17.0, avg: 31.3, max: 49.0) [2023-10-14 21:18:23,344][60425] Avg episode reward: [(0, '77.680'), (1, '76.610')] [2023-10-14 21:18:24,570][61552] Updated weights for policy 0, policy_version 87752 (0.0007) [2023-10-14 21:18:24,937][61552] Updated weights for policy 0, policy_version 87762 (0.0011) [2023-10-14 21:18:25,304][61552] Updated weights for policy 0, policy_version 87772 (0.0011) [2023-10-14 21:18:27,244][61585] Updated weights for policy 1, policy_version 87620 (0.0009) [2023-10-14 21:18:27,612][61585] Updated weights for policy 1, policy_version 87630 (0.0008) [2023-10-14 21:18:27,980][61585] Updated weights for policy 1, policy_version 87640 (0.0010) [2023-10-14 21:18:28,343][60425] Fps is (10 sec: 16384.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 179634176. Throughput: 0: 1674.9, 1: 1666.0. Samples: 44916450. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-14 21:18:28,344][60425] Avg episode reward: [(0, '75.540'), (1, '78.580')] [2023-10-14 21:18:29,372][61552] Updated weights for policy 0, policy_version 87782 (0.0008) [2023-10-14 21:18:29,742][61552] Updated weights for policy 0, policy_version 87792 (0.0008) [2023-10-14 21:18:30,123][61552] Updated weights for policy 0, policy_version 87802 (0.0008) [2023-10-14 21:18:32,112][61585] Updated weights for policy 1, policy_version 87650 (0.0008) [2023-10-14 21:18:32,479][61585] Updated weights for policy 1, policy_version 87660 (0.0007) [2023-10-14 21:18:32,842][61585] Updated weights for policy 1, policy_version 87670 (0.0008) [2023-10-14 21:18:33,209][61585] Updated weights for policy 1, policy_version 87680 (0.0007) [2023-10-14 21:18:33,343][60425] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 179699712. Throughput: 0: 1661.6, 1: 1676.1. Samples: 44926108. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-14 21:18:33,344][60425] Avg episode reward: [(0, '75.810'), (1, '79.820')] [2023-10-14 21:18:34,310][61552] Updated weights for policy 0, policy_version 87812 (0.0007) [2023-10-14 21:18:34,684][61552] Updated weights for policy 0, policy_version 87822 (0.0008) [2023-10-14 21:18:35,052][61552] Updated weights for policy 0, policy_version 87832 (0.0008) [2023-10-14 21:18:37,376][61585] Updated weights for policy 1, policy_version 87690 (0.0009) [2023-10-14 21:18:37,743][61585] Updated weights for policy 1, policy_version 87700 (0.0009) [2023-10-14 21:18:38,102][61585] Updated weights for policy 1, policy_version 87710 (0.0007) [2023-10-14 21:18:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 179765248. Throughput: 0: 1674.7, 1: 1673.0. Samples: 44946552. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-14 21:18:38,344][60425] Avg episode reward: [(0, '75.010'), (1, '79.770')] [2023-10-14 21:18:38,997][61552] Updated weights for policy 0, policy_version 87842 (0.0008) [2023-10-14 21:18:39,368][61552] Updated weights for policy 0, policy_version 87852 (0.0011) [2023-10-14 21:18:39,736][61552] Updated weights for policy 0, policy_version 87862 (0.0011) [2023-10-14 21:18:40,102][61552] Updated weights for policy 0, policy_version 87872 (0.0009) [2023-10-14 21:18:42,128][61585] Updated weights for policy 1, policy_version 87720 (0.0009) [2023-10-14 21:18:42,500][61585] Updated weights for policy 1, policy_version 87730 (0.0007) [2023-10-14 21:18:42,855][61585] Updated weights for policy 1, policy_version 87740 (0.0009) [2023-10-14 21:18:43,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 179830784. Throughput: 0: 1676.7, 1: 1662.7. Samples: 44966528. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-14 21:18:43,344][60425] Avg episode reward: [(0, '76.700'), (1, '80.170')] [2023-10-14 21:18:44,229][61552] Updated weights for policy 0, policy_version 87882 (0.0008) [2023-10-14 21:18:44,588][61552] Updated weights for policy 0, policy_version 87892 (0.0007) [2023-10-14 21:18:44,955][61552] Updated weights for policy 0, policy_version 87902 (0.0009) [2023-10-14 21:18:46,837][61585] Updated weights for policy 1, policy_version 87750 (0.0009) [2023-10-14 21:18:47,203][61585] Updated weights for policy 1, policy_version 87760 (0.0011) [2023-10-14 21:18:47,563][61585] Updated weights for policy 1, policy_version 87770 (0.0010) [2023-10-14 21:18:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 179896320. Throughput: 0: 1672.3, 1: 1684.7. Samples: 44976724. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-14 21:18:48,344][60425] Avg episode reward: [(0, '76.860'), (1, '81.010')] [2023-10-14 21:18:49,227][61552] Updated weights for policy 0, policy_version 87912 (0.0008) [2023-10-14 21:18:49,594][61552] Updated weights for policy 0, policy_version 87922 (0.0008) [2023-10-14 21:18:49,960][61552] Updated weights for policy 0, policy_version 87932 (0.0007) [2023-10-14 21:18:51,616][61585] Updated weights for policy 1, policy_version 87780 (0.0009) [2023-10-14 21:18:51,981][61585] Updated weights for policy 1, policy_version 87790 (0.0009) [2023-10-14 21:18:52,351][61585] Updated weights for policy 1, policy_version 87800 (0.0008) [2023-10-14 21:18:53,344][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 179961856. Throughput: 0: 1678.6, 1: 1679.4. Samples: 44996978. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-14 21:18:53,345][60425] Avg episode reward: [(0, '74.880'), (1, '79.600')] [2023-10-14 21:18:53,812][61552] Updated weights for policy 0, policy_version 87942 (0.0009) [2023-10-14 21:18:54,183][61552] Updated weights for policy 0, policy_version 87952 (0.0009) [2023-10-14 21:18:54,556][61552] Updated weights for policy 0, policy_version 87962 (0.0008) [2023-10-14 21:18:56,288][61585] Updated weights for policy 1, policy_version 87810 (0.0009) [2023-10-14 21:18:56,655][61585] Updated weights for policy 1, policy_version 87820 (0.0011) [2023-10-14 21:18:57,027][61585] Updated weights for policy 1, policy_version 87830 (0.0010) [2023-10-14 21:18:57,395][61585] Updated weights for policy 1, policy_version 87840 (0.0011) [2023-10-14 21:18:58,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 180027392. Throughput: 0: 1679.8, 1: 1661.1. Samples: 45016882. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-14 21:18:58,344][60425] Avg episode reward: [(0, '74.000'), (1, '80.510')] [2023-10-14 21:18:58,624][61552] Updated weights for policy 0, policy_version 87972 (0.0009) [2023-10-14 21:18:58,991][61552] Updated weights for policy 0, policy_version 87982 (0.0008) [2023-10-14 21:18:59,353][61552] Updated weights for policy 0, policy_version 87992 (0.0009) [2023-10-14 21:19:01,602][61585] Updated weights for policy 1, policy_version 87850 (0.0010) [2023-10-14 21:19:01,959][61585] Updated weights for policy 1, policy_version 87860 (0.0011) [2023-10-14 21:19:02,321][61585] Updated weights for policy 1, policy_version 87870 (0.0008) [2023-10-14 21:19:03,341][61552] Updated weights for policy 0, policy_version 88002 (0.0009) [2023-10-14 21:19:03,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 180092928. Throughput: 0: 1680.3, 1: 1688.4. Samples: 45027168. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-14 21:19:03,344][60425] Avg episode reward: [(0, '76.450'), (1, '83.490')] [2023-10-14 21:19:03,717][61552] Updated weights for policy 0, policy_version 88012 (0.0009) [2023-10-14 21:19:04,087][61552] Updated weights for policy 0, policy_version 88022 (0.0009) [2023-10-14 21:19:04,460][61552] Updated weights for policy 0, policy_version 88032 (0.0007) [2023-10-14 21:19:06,641][61585] Updated weights for policy 1, policy_version 87880 (0.0011) [2023-10-14 21:19:07,015][61585] Updated weights for policy 1, policy_version 87890 (0.0009) [2023-10-14 21:19:07,380][61585] Updated weights for policy 1, policy_version 87900 (0.0010) [2023-10-14 21:19:08,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 180158464. Throughput: 0: 1677.8, 1: 1671.6. Samples: 45047060. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-14 21:19:08,344][60425] Avg episode reward: [(0, '75.940'), (1, '78.060')] [2023-10-14 21:19:08,594][61552] Updated weights for policy 0, policy_version 88042 (0.0008) [2023-10-14 21:19:08,959][61552] Updated weights for policy 0, policy_version 88052 (0.0008) [2023-10-14 21:19:09,329][61552] Updated weights for policy 0, policy_version 88062 (0.0011) [2023-10-14 21:19:11,313][61585] Updated weights for policy 1, policy_version 87910 (0.0010) [2023-10-14 21:19:11,678][61585] Updated weights for policy 1, policy_version 87920 (0.0008) [2023-10-14 21:19:12,049][61585] Updated weights for policy 1, policy_version 87930 (0.0007) [2023-10-14 21:19:13,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 180224000. Throughput: 0: 1670.1, 1: 1672.1. Samples: 45066850. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-14 21:19:13,344][60425] Avg episode reward: [(0, '77.730'), (1, '81.900')] [2023-10-14 21:19:13,610][61552] Updated weights for policy 0, policy_version 88072 (0.0009) [2023-10-14 21:19:13,984][61552] Updated weights for policy 0, policy_version 88082 (0.0009) [2023-10-14 21:19:14,343][61552] Updated weights for policy 0, policy_version 88092 (0.0010) [2023-10-14 21:19:15,881][61585] Updated weights for policy 1, policy_version 87940 (0.0007) [2023-10-14 21:19:16,244][61585] Updated weights for policy 1, policy_version 87950 (0.0009) [2023-10-14 21:19:16,608][61585] Updated weights for policy 1, policy_version 87960 (0.0008) [2023-10-14 21:19:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 180289536. Throughput: 0: 1667.2, 1: 1688.8. Samples: 45077128. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-14 21:19:18,344][60425] Avg episode reward: [(0, '77.400'), (1, '84.040')] [2023-10-14 21:19:18,476][61552] Updated weights for policy 0, policy_version 88102 (0.0010) [2023-10-14 21:19:18,838][61552] Updated weights for policy 0, policy_version 88112 (0.0011) [2023-10-14 21:19:19,203][61552] Updated weights for policy 0, policy_version 88122 (0.0009) [2023-10-14 21:19:20,770][61585] Updated weights for policy 1, policy_version 87970 (0.0011) [2023-10-14 21:19:21,132][61585] Updated weights for policy 1, policy_version 87980 (0.0010) [2023-10-14 21:19:21,496][61585] Updated weights for policy 1, policy_version 87990 (0.0010) [2023-10-14 21:19:21,859][61585] Updated weights for policy 1, policy_version 88000 (0.0011) [2023-10-14 21:19:23,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 180355072. Throughput: 0: 1668.6, 1: 1666.8. Samples: 45096644. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-14 21:19:23,344][60425] Avg episode reward: [(0, '74.020'), (1, '79.980')] [2023-10-14 21:19:23,371][61552] Updated weights for policy 0, policy_version 88132 (0.0009) [2023-10-14 21:19:23,736][61552] Updated weights for policy 0, policy_version 88142 (0.0007) [2023-10-14 21:19:24,094][61552] Updated weights for policy 0, policy_version 88152 (0.0009) [2023-10-14 21:19:25,980][61585] Updated weights for policy 1, policy_version 88010 (0.0009) [2023-10-14 21:19:26,347][61585] Updated weights for policy 1, policy_version 88020 (0.0009) [2023-10-14 21:19:26,709][61585] Updated weights for policy 1, policy_version 88030 (0.0009) [2023-10-14 21:19:28,199][61552] Updated weights for policy 0, policy_version 88162 (0.0011) [2023-10-14 21:19:28,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 180420608. Throughput: 0: 1669.6, 1: 1677.4. Samples: 45117140. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 21:19:28,344][60425] Avg episode reward: [(0, '73.730'), (1, '80.080')] [2023-10-14 21:19:28,565][61552] Updated weights for policy 0, policy_version 88172 (0.0009) [2023-10-14 21:19:28,931][61552] Updated weights for policy 0, policy_version 88182 (0.0008) [2023-10-14 21:19:29,306][61552] Updated weights for policy 0, policy_version 88192 (0.0008) [2023-10-14 21:19:30,760][61585] Updated weights for policy 1, policy_version 88040 (0.0008) [2023-10-14 21:19:31,120][61585] Updated weights for policy 1, policy_version 88050 (0.0009) [2023-10-14 21:19:31,487][61585] Updated weights for policy 1, policy_version 88060 (0.0009) [2023-10-14 21:19:33,344][60425] Fps is (10 sec: 13106.6, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 180486144. Throughput: 0: 1669.3, 1: 1674.9. Samples: 45127212. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 21:19:33,345][60425] Avg episode reward: [(0, '77.040'), (1, '82.810')] [2023-10-14 21:19:33,422][61552] Updated weights for policy 0, policy_version 88202 (0.0008) [2023-10-14 21:19:33,790][61552] Updated weights for policy 0, policy_version 88212 (0.0007) [2023-10-14 21:19:34,152][61552] Updated weights for policy 0, policy_version 88222 (0.0007) [2023-10-14 21:19:35,628][61585] Updated weights for policy 1, policy_version 88070 (0.0008) [2023-10-14 21:19:36,000][61585] Updated weights for policy 1, policy_version 88080 (0.0010) [2023-10-14 21:19:36,363][61585] Updated weights for policy 1, policy_version 88090 (0.0010) [2023-10-14 21:19:38,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 180551680. Throughput: 0: 1666.0, 1: 1667.2. Samples: 45146970. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 21:19:38,344][60425] Avg episode reward: [(0, '75.740'), (1, '85.420')] [2023-10-14 21:19:38,410][61552] Updated weights for policy 0, policy_version 88232 (0.0009) [2023-10-14 21:19:38,783][61552] Updated weights for policy 0, policy_version 88242 (0.0011) [2023-10-14 21:19:39,158][61552] Updated weights for policy 0, policy_version 88252 (0.0009) [2023-10-14 21:19:40,540][61585] Updated weights for policy 1, policy_version 88100 (0.0009) [2023-10-14 21:19:40,911][61585] Updated weights for policy 1, policy_version 88110 (0.0011) [2023-10-14 21:19:41,276][61585] Updated weights for policy 1, policy_version 88120 (0.0010) [2023-10-14 21:19:43,195][61552] Updated weights for policy 0, policy_version 88262 (0.0010) [2023-10-14 21:19:43,344][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 180617216. Throughput: 0: 1661.4, 1: 1685.5. Samples: 45167492. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 21:19:43,344][60425] Avg episode reward: [(0, '75.680'), (1, '80.020')] [2023-10-14 21:19:43,353][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000088128_90243072.pth... [2023-10-14 21:19:43,392][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000086560_88637440.pth [2023-10-14 21:19:43,568][61552] Updated weights for policy 0, policy_version 88272 (0.0009) [2023-10-14 21:19:43,940][61552] Updated weights for policy 0, policy_version 88282 (0.0009) [2023-10-14 21:19:44,152][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000088288_90406912.pth... [2023-10-14 21:19:44,192][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000086720_88801280.pth [2023-10-14 21:19:45,525][61585] Updated weights for policy 1, policy_version 88130 (0.0008) [2023-10-14 21:19:45,902][61585] Updated weights for policy 1, policy_version 88140 (0.0007) [2023-10-14 21:19:46,264][61585] Updated weights for policy 1, policy_version 88150 (0.0009) [2023-10-14 21:19:46,626][61585] Updated weights for policy 1, policy_version 88160 (0.0011) [2023-10-14 21:19:48,032][61552] Updated weights for policy 0, policy_version 88292 (0.0009) [2023-10-14 21:19:48,344][60425] Fps is (10 sec: 13105.5, 60 sec: 13106.9, 300 sec: 13329.3). Total num frames: 180682752. Throughput: 0: 1660.2, 1: 1677.6. Samples: 45177374. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 21:19:48,345][60425] Avg episode reward: [(0, '77.470'), (1, '83.950')] [2023-10-14 21:19:48,409][61552] Updated weights for policy 0, policy_version 88302 (0.0011) [2023-10-14 21:19:48,776][61552] Updated weights for policy 0, policy_version 88312 (0.0009) [2023-10-14 21:19:50,708][61585] Updated weights for policy 1, policy_version 88170 (0.0010) [2023-10-14 21:19:51,081][61585] Updated weights for policy 1, policy_version 88180 (0.0007) [2023-10-14 21:19:51,437][61585] Updated weights for policy 1, policy_version 88190 (0.0009) [2023-10-14 21:19:52,761][61552] Updated weights for policy 0, policy_version 88322 (0.0009) [2023-10-14 21:19:53,126][61552] Updated weights for policy 0, policy_version 88332 (0.0007) [2023-10-14 21:19:53,343][60425] Fps is (10 sec: 13107.7, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 180748288. Throughput: 0: 1657.5, 1: 1666.2. Samples: 45196624. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 21:19:53,344][60425] Avg episode reward: [(0, '74.190'), (1, '82.700')] [2023-10-14 21:19:53,501][61552] Updated weights for policy 0, policy_version 88342 (0.0008) [2023-10-14 21:19:53,867][61552] Updated weights for policy 0, policy_version 88352 (0.0008) [2023-10-14 21:19:55,599][61585] Updated weights for policy 1, policy_version 88200 (0.0011) [2023-10-14 21:19:55,975][61585] Updated weights for policy 1, policy_version 88210 (0.0008) [2023-10-14 21:19:56,330][61585] Updated weights for policy 1, policy_version 88220 (0.0007) [2023-10-14 21:19:57,978][61552] Updated weights for policy 0, policy_version 88362 (0.0007) [2023-10-14 21:19:58,343][60425] Fps is (10 sec: 13108.8, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 180813824. Throughput: 0: 1663.8, 1: 1672.0. Samples: 45216962. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 21:19:58,344][60425] Avg episode reward: [(0, '77.190'), (1, '81.850')] [2023-10-14 21:19:58,353][61552] Updated weights for policy 0, policy_version 88372 (0.0008) [2023-10-14 21:19:58,737][61552] Updated weights for policy 0, policy_version 88382 (0.0008) [2023-10-14 21:20:00,629][61585] Updated weights for policy 1, policy_version 88230 (0.0007) [2023-10-14 21:20:00,997][61585] Updated weights for policy 1, policy_version 88240 (0.0007) [2023-10-14 21:20:01,354][61585] Updated weights for policy 1, policy_version 88250 (0.0007) [2023-10-14 21:20:03,065][61552] Updated weights for policy 0, policy_version 88392 (0.0008) [2023-10-14 21:20:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 180879360. Throughput: 0: 1667.3, 1: 1657.7. Samples: 45226754. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 21:20:03,344][60425] Avg episode reward: [(0, '73.270'), (1, '78.150')] [2023-10-14 21:20:03,431][61552] Updated weights for policy 0, policy_version 88402 (0.0007) [2023-10-14 21:20:03,795][61552] Updated weights for policy 0, policy_version 88412 (0.0007) [2023-10-14 21:20:05,424][61585] Updated weights for policy 1, policy_version 88260 (0.0007) [2023-10-14 21:20:05,793][61585] Updated weights for policy 1, policy_version 88270 (0.0010) [2023-10-14 21:20:06,163][61585] Updated weights for policy 1, policy_version 88280 (0.0008) [2023-10-14 21:20:07,901][61552] Updated weights for policy 0, policy_version 88422 (0.0007) [2023-10-14 21:20:08,269][61552] Updated weights for policy 0, policy_version 88432 (0.0007) [2023-10-14 21:20:08,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 180944896. Throughput: 0: 1669.4, 1: 1658.9. Samples: 45246420. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 21:20:08,344][60425] Avg episode reward: [(0, '77.650'), (1, '84.800')] [2023-10-14 21:20:08,639][61552] Updated weights for policy 0, policy_version 88442 (0.0010) [2023-10-14 21:20:10,189][61585] Updated weights for policy 1, policy_version 88290 (0.0008) [2023-10-14 21:20:10,551][61585] Updated weights for policy 1, policy_version 88300 (0.0008) [2023-10-14 21:20:10,925][61585] Updated weights for policy 1, policy_version 88310 (0.0007) [2023-10-14 21:20:11,283][61585] Updated weights for policy 1, policy_version 88320 (0.0009) [2023-10-14 21:20:12,522][61552] Updated weights for policy 0, policy_version 88452 (0.0008) [2023-10-14 21:20:12,884][61552] Updated weights for policy 0, policy_version 88462 (0.0008) [2023-10-14 21:20:13,248][61552] Updated weights for policy 0, policy_version 88472 (0.0010) [2023-10-14 21:20:13,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 181010432. Throughput: 0: 1663.1, 1: 1663.5. Samples: 45266836. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 21:20:13,344][60425] Avg episode reward: [(0, '74.240'), (1, '79.600')] [2023-10-14 21:20:15,495][61585] Updated weights for policy 1, policy_version 88330 (0.0011) [2023-10-14 21:20:15,868][61585] Updated weights for policy 1, policy_version 88340 (0.0008) [2023-10-14 21:20:16,240][61585] Updated weights for policy 1, policy_version 88350 (0.0008) [2023-10-14 21:20:17,354][61552] Updated weights for policy 0, policy_version 88482 (0.0009) [2023-10-14 21:20:17,728][61552] Updated weights for policy 0, policy_version 88492 (0.0008) [2023-10-14 21:20:18,092][61552] Updated weights for policy 0, policy_version 88502 (0.0007) [2023-10-14 21:20:18,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 181075968. Throughput: 0: 1672.8, 1: 1653.0. Samples: 45276870. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 21:20:18,344][60425] Avg episode reward: [(0, '78.460'), (1, '77.490')] [2023-10-14 21:20:18,458][61552] Updated weights for policy 0, policy_version 88512 (0.0010) [2023-10-14 21:20:20,376][61585] Updated weights for policy 1, policy_version 88360 (0.0007) [2023-10-14 21:20:20,745][61585] Updated weights for policy 1, policy_version 88370 (0.0007) [2023-10-14 21:20:21,106][61585] Updated weights for policy 1, policy_version 88380 (0.0008) [2023-10-14 21:20:22,479][61552] Updated weights for policy 0, policy_version 88522 (0.0008) [2023-10-14 21:20:22,852][61552] Updated weights for policy 0, policy_version 88532 (0.0007) [2023-10-14 21:20:23,226][61552] Updated weights for policy 0, policy_version 88542 (0.0007) [2023-10-14 21:20:23,343][60425] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 181174272. Throughput: 0: 1679.2, 1: 1652.2. Samples: 45296884. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:20:23,344][60425] Avg episode reward: [(0, '75.610'), (1, '79.250')] [2023-10-14 21:20:25,003][61585] Updated weights for policy 1, policy_version 88390 (0.0007) [2023-10-14 21:20:25,361][61585] Updated weights for policy 1, policy_version 88400 (0.0010) [2023-10-14 21:20:25,727][61585] Updated weights for policy 1, policy_version 88410 (0.0008) [2023-10-14 21:20:27,415][61552] Updated weights for policy 0, policy_version 88552 (0.0009) [2023-10-14 21:20:27,793][61552] Updated weights for policy 0, policy_version 88562 (0.0007) [2023-10-14 21:20:28,158][61552] Updated weights for policy 0, policy_version 88572 (0.0008) [2023-10-14 21:20:28,343][60425] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 181239808. Throughput: 0: 1663.9, 1: 1660.4. Samples: 45317086. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:20:28,344][60425] Avg episode reward: [(0, '77.470'), (1, '79.340')] [2023-10-14 21:20:29,783][61585] Updated weights for policy 1, policy_version 88420 (0.0009) [2023-10-14 21:20:30,146][61585] Updated weights for policy 1, policy_version 88430 (0.0010) [2023-10-14 21:20:30,511][61585] Updated weights for policy 1, policy_version 88440 (0.0009) [2023-10-14 21:20:32,164][61552] Updated weights for policy 0, policy_version 88582 (0.0009) [2023-10-14 21:20:32,537][61552] Updated weights for policy 0, policy_version 88592 (0.0007) [2023-10-14 21:20:32,903][61552] Updated weights for policy 0, policy_version 88602 (0.0007) [2023-10-14 21:20:33,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 181305344. Throughput: 0: 1682.4, 1: 1643.7. Samples: 45327046. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:20:33,344][60425] Avg episode reward: [(0, '75.170'), (1, '78.660')] [2023-10-14 21:20:34,621][61585] Updated weights for policy 1, policy_version 88450 (0.0009) [2023-10-14 21:20:34,985][61585] Updated weights for policy 1, policy_version 88460 (0.0008) [2023-10-14 21:20:35,353][61585] Updated weights for policy 1, policy_version 88470 (0.0010) [2023-10-14 21:20:35,716][61585] Updated weights for policy 1, policy_version 88480 (0.0009) [2023-10-14 21:20:37,064][61552] Updated weights for policy 0, policy_version 88612 (0.0008) [2023-10-14 21:20:37,426][61552] Updated weights for policy 0, policy_version 88622 (0.0009) [2023-10-14 21:20:37,798][61552] Updated weights for policy 0, policy_version 88632 (0.0008) [2023-10-14 21:20:38,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 181370880. Throughput: 0: 1684.0, 1: 1668.2. Samples: 45347472. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:20:38,344][60425] Avg episode reward: [(0, '73.730'), (1, '86.360')] [2023-10-14 21:20:38,345][61248] Saving new best policy, reward=86.360! [2023-10-14 21:20:39,883][61585] Updated weights for policy 1, policy_version 88490 (0.0007) [2023-10-14 21:20:40,270][61585] Updated weights for policy 1, policy_version 88500 (0.0009) [2023-10-14 21:20:40,635][61585] Updated weights for policy 1, policy_version 88510 (0.0007) [2023-10-14 21:20:41,962][61552] Updated weights for policy 0, policy_version 88642 (0.0010) [2023-10-14 21:20:42,339][61552] Updated weights for policy 0, policy_version 88652 (0.0007) [2023-10-14 21:20:42,704][61552] Updated weights for policy 0, policy_version 88662 (0.0007) [2023-10-14 21:20:43,070][61552] Updated weights for policy 0, policy_version 88672 (0.0007) [2023-10-14 21:20:43,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.5, 300 sec: 13440.4). Total num frames: 181436416. Throughput: 0: 1664.0, 1: 1672.2. Samples: 45367092. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:20:43,344][60425] Avg episode reward: [(0, '77.150'), (1, '78.460')] [2023-10-14 21:20:44,859][61585] Updated weights for policy 1, policy_version 88520 (0.0008) [2023-10-14 21:20:45,219][61585] Updated weights for policy 1, policy_version 88530 (0.0009) [2023-10-14 21:20:45,592][61585] Updated weights for policy 1, policy_version 88540 (0.0010) [2023-10-14 21:20:47,063][61552] Updated weights for policy 0, policy_version 88682 (0.0010) [2023-10-14 21:20:47,444][61552] Updated weights for policy 0, policy_version 88692 (0.0008) [2023-10-14 21:20:47,806][61552] Updated weights for policy 0, policy_version 88702 (0.0008) [2023-10-14 21:20:48,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.6, 300 sec: 13440.4). Total num frames: 181501952. Throughput: 0: 1686.5, 1: 1653.8. Samples: 45377068. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:20:48,344][60425] Avg episode reward: [(0, '80.200'), (1, '77.080')] [2023-10-14 21:20:49,786][61585] Updated weights for policy 1, policy_version 88550 (0.0008) [2023-10-14 21:20:50,150][61585] Updated weights for policy 1, policy_version 88560 (0.0007) [2023-10-14 21:20:50,513][61585] Updated weights for policy 1, policy_version 88570 (0.0011) [2023-10-14 21:20:51,968][61552] Updated weights for policy 0, policy_version 88712 (0.0010) [2023-10-14 21:20:52,330][61552] Updated weights for policy 0, policy_version 88722 (0.0011) [2023-10-14 21:20:52,700][61552] Updated weights for policy 0, policy_version 88732 (0.0008) [2023-10-14 21:20:53,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 181567488. Throughput: 0: 1678.1, 1: 1670.3. Samples: 45397096. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:20:53,344][60425] Avg episode reward: [(0, '81.360'), (1, '84.760')] [2023-10-14 21:20:54,547][61585] Updated weights for policy 1, policy_version 88580 (0.0008) [2023-10-14 21:20:54,912][61585] Updated weights for policy 1, policy_version 88590 (0.0008) [2023-10-14 21:20:55,278][61585] Updated weights for policy 1, policy_version 88600 (0.0008) [2023-10-14 21:20:56,751][61552] Updated weights for policy 0, policy_version 88742 (0.0008) [2023-10-14 21:20:57,123][61552] Updated weights for policy 0, policy_version 88752 (0.0008) [2023-10-14 21:20:57,497][61552] Updated weights for policy 0, policy_version 88762 (0.0008) [2023-10-14 21:20:58,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 181633024. Throughput: 0: 1659.3, 1: 1669.5. Samples: 45416634. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:20:58,344][60425] Avg episode reward: [(0, '74.870'), (1, '83.740')] [2023-10-14 21:20:59,443][61585] Updated weights for policy 1, policy_version 88610 (0.0009) [2023-10-14 21:20:59,804][61585] Updated weights for policy 1, policy_version 88620 (0.0008) [2023-10-14 21:21:00,171][61585] Updated weights for policy 1, policy_version 88630 (0.0007) [2023-10-14 21:21:00,532][61585] Updated weights for policy 1, policy_version 88640 (0.0011) [2023-10-14 21:21:01,505][61552] Updated weights for policy 0, policy_version 88772 (0.0007) [2023-10-14 21:21:01,871][61552] Updated weights for policy 0, policy_version 88782 (0.0008) [2023-10-14 21:21:02,232][61552] Updated weights for policy 0, policy_version 88792 (0.0008) [2023-10-14 21:21:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 181698560. Throughput: 0: 1675.6, 1: 1660.2. Samples: 45426984. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:21:03,344][60425] Avg episode reward: [(0, '72.230'), (1, '81.160')] [2023-10-14 21:21:04,575][61585] Updated weights for policy 1, policy_version 88650 (0.0008) [2023-10-14 21:21:04,939][61585] Updated weights for policy 1, policy_version 88660 (0.0007) [2023-10-14 21:21:05,301][61585] Updated weights for policy 1, policy_version 88670 (0.0009) [2023-10-14 21:21:06,351][61552] Updated weights for policy 0, policy_version 88802 (0.0009) [2023-10-14 21:21:06,713][61552] Updated weights for policy 0, policy_version 88812 (0.0007) [2023-10-14 21:21:07,082][61552] Updated weights for policy 0, policy_version 88822 (0.0009) [2023-10-14 21:21:07,448][61552] Updated weights for policy 0, policy_version 88832 (0.0008) [2023-10-14 21:21:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 181764096. Throughput: 0: 1663.0, 1: 1676.9. Samples: 45447182. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:21:08,344][60425] Avg episode reward: [(0, '74.670'), (1, '84.860')] [2023-10-14 21:21:09,253][61585] Updated weights for policy 1, policy_version 88680 (0.0009) [2023-10-14 21:21:09,622][61585] Updated weights for policy 1, policy_version 88690 (0.0009) [2023-10-14 21:21:09,987][61585] Updated weights for policy 1, policy_version 88700 (0.0010) [2023-10-14 21:21:11,351][61552] Updated weights for policy 0, policy_version 88842 (0.0011) [2023-10-14 21:21:11,722][61552] Updated weights for policy 0, policy_version 88852 (0.0010) [2023-10-14 21:21:12,089][61552] Updated weights for policy 0, policy_version 88862 (0.0011) [2023-10-14 21:21:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 181829632. Throughput: 0: 1658.1, 1: 1676.5. Samples: 45467146. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:21:13,344][60425] Avg episode reward: [(0, '78.940'), (1, '82.350')] [2023-10-14 21:21:14,094][61585] Updated weights for policy 1, policy_version 88710 (0.0010) [2023-10-14 21:21:14,455][61585] Updated weights for policy 1, policy_version 88720 (0.0009) [2023-10-14 21:21:14,824][61585] Updated weights for policy 1, policy_version 88730 (0.0007) [2023-10-14 21:21:16,254][61552] Updated weights for policy 0, policy_version 88872 (0.0011) [2023-10-14 21:21:16,615][61552] Updated weights for policy 0, policy_version 88882 (0.0011) [2023-10-14 21:21:16,985][61552] Updated weights for policy 0, policy_version 88892 (0.0009) [2023-10-14 21:21:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 181895168. Throughput: 0: 1669.1, 1: 1675.6. Samples: 45477554. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:21:18,344][60425] Avg episode reward: [(0, '73.490'), (1, '81.340')] [2023-10-14 21:21:19,000][61585] Updated weights for policy 1, policy_version 88740 (0.0007) [2023-10-14 21:21:19,356][61585] Updated weights for policy 1, policy_version 88750 (0.0010) [2023-10-14 21:21:19,715][61585] Updated weights for policy 1, policy_version 88760 (0.0010) [2023-10-14 21:21:21,131][61552] Updated weights for policy 0, policy_version 88902 (0.0008) [2023-10-14 21:21:21,493][61552] Updated weights for policy 0, policy_version 88912 (0.0007) [2023-10-14 21:21:21,863][61552] Updated weights for policy 0, policy_version 88922 (0.0009) [2023-10-14 21:21:23,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 181960704. Throughput: 0: 1649.7, 1: 1676.9. Samples: 45497170. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-14 21:21:23,344][60425] Avg episode reward: [(0, '74.780'), (1, '81.950')] [2023-10-14 21:21:23,680][61585] Updated weights for policy 1, policy_version 88770 (0.0009) [2023-10-14 21:21:24,044][61585] Updated weights for policy 1, policy_version 88780 (0.0010) [2023-10-14 21:21:24,411][61585] Updated weights for policy 1, policy_version 88790 (0.0010) [2023-10-14 21:21:24,784][61585] Updated weights for policy 1, policy_version 88800 (0.0011) [2023-10-14 21:21:25,884][61552] Updated weights for policy 0, policy_version 88932 (0.0008) [2023-10-14 21:21:26,253][61552] Updated weights for policy 0, policy_version 88942 (0.0008) [2023-10-14 21:21:26,628][61552] Updated weights for policy 0, policy_version 88952 (0.0007) [2023-10-14 21:21:28,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 182026240. Throughput: 0: 1656.7, 1: 1685.1. Samples: 45517474. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-14 21:21:28,344][60425] Avg episode reward: [(0, '77.290'), (1, '78.860')] [2023-10-14 21:21:28,767][61585] Updated weights for policy 1, policy_version 88810 (0.0008) [2023-10-14 21:21:29,131][61585] Updated weights for policy 1, policy_version 88820 (0.0009) [2023-10-14 21:21:29,494][61585] Updated weights for policy 1, policy_version 88830 (0.0009) [2023-10-14 21:21:30,712][61552] Updated weights for policy 0, policy_version 88962 (0.0007) [2023-10-14 21:21:31,083][61552] Updated weights for policy 0, policy_version 88972 (0.0008) [2023-10-14 21:21:31,445][61552] Updated weights for policy 0, policy_version 88982 (0.0010) [2023-10-14 21:21:31,815][61552] Updated weights for policy 0, policy_version 88992 (0.0011) [2023-10-14 21:21:33,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 182091776. Throughput: 0: 1659.3, 1: 1689.8. Samples: 45527778. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-14 21:21:33,344][60425] Avg episode reward: [(0, '81.570'), (1, '72.790')] [2023-10-14 21:21:33,555][61585] Updated weights for policy 1, policy_version 88840 (0.0007) [2023-10-14 21:21:33,937][61585] Updated weights for policy 1, policy_version 88850 (0.0011) [2023-10-14 21:21:34,294][61585] Updated weights for policy 1, policy_version 88860 (0.0011) [2023-10-14 21:21:36,118][61552] Updated weights for policy 0, policy_version 89002 (0.0007) [2023-10-14 21:21:36,497][61552] Updated weights for policy 0, policy_version 89012 (0.0007) [2023-10-14 21:21:36,868][61552] Updated weights for policy 0, policy_version 89022 (0.0008) [2023-10-14 21:21:38,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 182157312. Throughput: 0: 1648.7, 1: 1689.4. Samples: 45547308. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-14 21:21:38,344][60425] Avg episode reward: [(0, '76.510'), (1, '80.670')] [2023-10-14 21:21:38,383][61585] Updated weights for policy 1, policy_version 88870 (0.0009) [2023-10-14 21:21:38,753][61585] Updated weights for policy 1, policy_version 88880 (0.0010) [2023-10-14 21:21:39,129][61585] Updated weights for policy 1, policy_version 88890 (0.0008) [2023-10-14 21:21:40,928][61552] Updated weights for policy 0, policy_version 89032 (0.0008) [2023-10-14 21:21:41,302][61552] Updated weights for policy 0, policy_version 89042 (0.0009) [2023-10-14 21:21:41,674][61552] Updated weights for policy 0, policy_version 89052 (0.0010) [2023-10-14 21:21:43,241][61585] Updated weights for policy 1, policy_version 88900 (0.0008) [2023-10-14 21:21:43,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 182222848. Throughput: 0: 1668.1, 1: 1686.4. Samples: 45567584. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-14 21:21:43,344][60425] Avg episode reward: [(0, '80.530'), (1, '77.380')] [2023-10-14 21:21:43,352][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000089056_91193344.pth... [2023-10-14 21:21:43,390][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000087488_89587712.pth [2023-10-14 21:21:43,600][61585] Updated weights for policy 1, policy_version 88910 (0.0009) [2023-10-14 21:21:43,968][61585] Updated weights for policy 1, policy_version 88920 (0.0008) [2023-10-14 21:21:44,254][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000088928_91062272.pth... [2023-10-14 21:21:44,292][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000087360_89456640.pth [2023-10-14 21:21:45,692][61552] Updated weights for policy 0, policy_version 89062 (0.0008) [2023-10-14 21:21:46,063][61552] Updated weights for policy 0, policy_version 89072 (0.0011) [2023-10-14 21:21:46,435][61552] Updated weights for policy 0, policy_version 89082 (0.0010) [2023-10-14 21:21:48,284][61585] Updated weights for policy 1, policy_version 88930 (0.0009) [2023-10-14 21:21:48,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 182288384. Throughput: 0: 1668.9, 1: 1682.0. Samples: 45577772. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-14 21:21:48,344][60425] Avg episode reward: [(0, '78.480'), (1, '80.300')] [2023-10-14 21:21:48,652][61585] Updated weights for policy 1, policy_version 88940 (0.0008) [2023-10-14 21:21:49,013][61585] Updated weights for policy 1, policy_version 88950 (0.0009) [2023-10-14 21:21:49,377][61585] Updated weights for policy 1, policy_version 88960 (0.0008) [2023-10-14 21:21:50,571][61552] Updated weights for policy 0, policy_version 89092 (0.0009) [2023-10-14 21:21:50,928][61552] Updated weights for policy 0, policy_version 89102 (0.0010) [2023-10-14 21:21:51,286][61552] Updated weights for policy 0, policy_version 89112 (0.0011) [2023-10-14 21:21:53,251][61585] Updated weights for policy 1, policy_version 88970 (0.0008) [2023-10-14 21:21:53,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 182353920. Throughput: 0: 1652.4, 1: 1685.4. Samples: 45597382. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-14 21:21:53,344][60425] Avg episode reward: [(0, '81.780'), (1, '77.510')] [2023-10-14 21:21:53,607][61585] Updated weights for policy 1, policy_version 88980 (0.0007) [2023-10-14 21:21:53,979][61585] Updated weights for policy 1, policy_version 88990 (0.0008) [2023-10-14 21:21:55,500][61552] Updated weights for policy 0, policy_version 89122 (0.0009) [2023-10-14 21:21:55,877][61552] Updated weights for policy 0, policy_version 89132 (0.0008) [2023-10-14 21:21:56,241][61552] Updated weights for policy 0, policy_version 89142 (0.0009) [2023-10-14 21:21:56,610][61552] Updated weights for policy 0, policy_version 89152 (0.0010) [2023-10-14 21:21:58,058][61585] Updated weights for policy 1, policy_version 89000 (0.0007) [2023-10-14 21:21:58,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 182419456. Throughput: 0: 1669.3, 1: 1682.1. Samples: 45617958. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-14 21:21:58,344][60425] Avg episode reward: [(0, '78.690'), (1, '77.090')] [2023-10-14 21:21:58,434][61585] Updated weights for policy 1, policy_version 89010 (0.0008) [2023-10-14 21:21:58,790][61585] Updated weights for policy 1, policy_version 89020 (0.0009) [2023-10-14 21:22:00,700][61552] Updated weights for policy 0, policy_version 89162 (0.0009) [2023-10-14 21:22:01,072][61552] Updated weights for policy 0, policy_version 89172 (0.0009) [2023-10-14 21:22:01,433][61552] Updated weights for policy 0, policy_version 89182 (0.0008) [2023-10-14 21:22:03,035][61585] Updated weights for policy 1, policy_version 89030 (0.0009) [2023-10-14 21:22:03,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 182484992. Throughput: 0: 1663.7, 1: 1678.5. Samples: 45627952. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-14 21:22:03,344][60425] Avg episode reward: [(0, '82.930'), (1, '83.860')] [2023-10-14 21:22:03,400][61585] Updated weights for policy 1, policy_version 89040 (0.0009) [2023-10-14 21:22:03,772][61585] Updated weights for policy 1, policy_version 89050 (0.0010) [2023-10-14 21:22:05,680][61552] Updated weights for policy 0, policy_version 89192 (0.0007) [2023-10-14 21:22:06,050][61552] Updated weights for policy 0, policy_version 89202 (0.0008) [2023-10-14 21:22:06,412][61552] Updated weights for policy 0, policy_version 89212 (0.0008) [2023-10-14 21:22:07,845][61585] Updated weights for policy 1, policy_version 89060 (0.0007) [2023-10-14 21:22:08,213][61585] Updated weights for policy 1, policy_version 89070 (0.0007) [2023-10-14 21:22:08,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 182550528. Throughput: 0: 1661.6, 1: 1680.1. Samples: 45647546. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-14 21:22:08,345][60425] Avg episode reward: [(0, '77.820'), (1, '81.000')] [2023-10-14 21:22:08,585][61585] Updated weights for policy 1, policy_version 89080 (0.0009) [2023-10-14 21:22:10,473][61552] Updated weights for policy 0, policy_version 89222 (0.0007) [2023-10-14 21:22:10,847][61552] Updated weights for policy 0, policy_version 89232 (0.0008) [2023-10-14 21:22:11,212][61552] Updated weights for policy 0, policy_version 89242 (0.0008) [2023-10-14 21:22:12,745][61585] Updated weights for policy 1, policy_version 89090 (0.0007) [2023-10-14 21:22:13,121][61585] Updated weights for policy 1, policy_version 89100 (0.0007) [2023-10-14 21:22:13,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 182616064. Throughput: 0: 1674.0, 1: 1672.8. Samples: 45668080. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-14 21:22:13,344][60425] Avg episode reward: [(0, '83.260'), (1, '79.800')] [2023-10-14 21:22:13,482][61585] Updated weights for policy 1, policy_version 89110 (0.0007) [2023-10-14 21:22:13,846][61585] Updated weights for policy 1, policy_version 89120 (0.0009) [2023-10-14 21:22:15,317][61552] Updated weights for policy 0, policy_version 89252 (0.0009) [2023-10-14 21:22:15,687][61552] Updated weights for policy 0, policy_version 89262 (0.0008) [2023-10-14 21:22:16,062][61552] Updated weights for policy 0, policy_version 89272 (0.0009) [2023-10-14 21:22:18,075][61585] Updated weights for policy 1, policy_version 89130 (0.0007) [2023-10-14 21:22:18,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 182681600. Throughput: 0: 1665.3, 1: 1670.3. Samples: 45677880. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-14 21:22:18,344][60425] Avg episode reward: [(0, '79.640'), (1, '82.050')] [2023-10-14 21:22:18,447][61585] Updated weights for policy 1, policy_version 89140 (0.0008) [2023-10-14 21:22:18,819][61585] Updated weights for policy 1, policy_version 89150 (0.0008) [2023-10-14 21:22:20,001][61552] Updated weights for policy 0, policy_version 89282 (0.0009) [2023-10-14 21:22:20,367][61552] Updated weights for policy 0, policy_version 89292 (0.0007) [2023-10-14 21:22:20,727][61552] Updated weights for policy 0, policy_version 89302 (0.0007) [2023-10-14 21:22:21,088][61552] Updated weights for policy 0, policy_version 89312 (0.0007) [2023-10-14 21:22:23,131][61585] Updated weights for policy 1, policy_version 89160 (0.0008) [2023-10-14 21:22:23,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 182747136. Throughput: 0: 1669.6, 1: 1671.1. Samples: 45697638. Policy #0 lag: (min: 0.0, avg: 21.7, max: 32.0) [2023-10-14 21:22:23,344][60425] Avg episode reward: [(0, '78.270'), (1, '78.500')] [2023-10-14 21:22:23,502][61585] Updated weights for policy 1, policy_version 89170 (0.0008) [2023-10-14 21:22:23,859][61585] Updated weights for policy 1, policy_version 89180 (0.0010) [2023-10-14 21:22:25,276][61552] Updated weights for policy 0, policy_version 89322 (0.0007) [2023-10-14 21:22:25,648][61552] Updated weights for policy 0, policy_version 89332 (0.0011) [2023-10-14 21:22:26,017][61552] Updated weights for policy 0, policy_version 89342 (0.0011) [2023-10-14 21:22:27,902][61585] Updated weights for policy 1, policy_version 89190 (0.0008) [2023-10-14 21:22:28,269][61585] Updated weights for policy 1, policy_version 89200 (0.0008) [2023-10-14 21:22:28,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 182812672. Throughput: 0: 1675.8, 1: 1667.4. Samples: 45718026. Policy #0 lag: (min: 0.0, avg: 21.7, max: 32.0) [2023-10-14 21:22:28,344][60425] Avg episode reward: [(0, '77.990'), (1, '80.140')] [2023-10-14 21:22:28,619][61585] Updated weights for policy 1, policy_version 89210 (0.0007) [2023-10-14 21:22:29,921][61552] Updated weights for policy 0, policy_version 89352 (0.0009) [2023-10-14 21:22:30,291][61552] Updated weights for policy 0, policy_version 89362 (0.0008) [2023-10-14 21:22:30,654][61552] Updated weights for policy 0, policy_version 89372 (0.0010) [2023-10-14 21:22:32,540][61585] Updated weights for policy 1, policy_version 89220 (0.0010) [2023-10-14 21:22:32,911][61585] Updated weights for policy 1, policy_version 89230 (0.0011) [2023-10-14 21:22:33,282][61585] Updated weights for policy 1, policy_version 89240 (0.0011) [2023-10-14 21:22:33,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 182878208. Throughput: 0: 1656.9, 1: 1672.8. Samples: 45727608. Policy #0 lag: (min: 0.0, avg: 21.7, max: 32.0) [2023-10-14 21:22:33,345][60425] Avg episode reward: [(0, '76.560'), (1, '76.150')] [2023-10-14 21:22:34,875][61552] Updated weights for policy 0, policy_version 89382 (0.0009) [2023-10-14 21:22:35,239][61552] Updated weights for policy 0, policy_version 89392 (0.0009) [2023-10-14 21:22:35,610][61552] Updated weights for policy 0, policy_version 89402 (0.0010) [2023-10-14 21:22:37,504][61585] Updated weights for policy 1, policy_version 89250 (0.0010) [2023-10-14 21:22:37,862][61585] Updated weights for policy 1, policy_version 89260 (0.0008) [2023-10-14 21:22:38,230][61585] Updated weights for policy 1, policy_version 89270 (0.0008) [2023-10-14 21:22:38,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 182943744. Throughput: 0: 1677.6, 1: 1668.5. Samples: 45747958. Policy #0 lag: (min: 0.0, avg: 21.7, max: 32.0) [2023-10-14 21:22:38,344][60425] Avg episode reward: [(0, '78.230'), (1, '82.120')] [2023-10-14 21:22:38,590][61585] Updated weights for policy 1, policy_version 89280 (0.0009) [2023-10-14 21:22:39,486][61552] Updated weights for policy 0, policy_version 89412 (0.0008) [2023-10-14 21:22:39,859][61552] Updated weights for policy 0, policy_version 89422 (0.0008) [2023-10-14 21:22:40,226][61552] Updated weights for policy 0, policy_version 89432 (0.0010) [2023-10-14 21:22:42,784][61585] Updated weights for policy 1, policy_version 89290 (0.0008) [2023-10-14 21:22:43,146][61585] Updated weights for policy 1, policy_version 89300 (0.0008) [2023-10-14 21:22:43,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 183009280. Throughput: 0: 1685.9, 1: 1658.5. Samples: 45768454. Policy #0 lag: (min: 0.0, avg: 21.7, max: 32.0) [2023-10-14 21:22:43,344][60425] Avg episode reward: [(0, '74.410'), (1, '80.740')] [2023-10-14 21:22:43,501][61585] Updated weights for policy 1, policy_version 89310 (0.0009) [2023-10-14 21:22:44,222][61552] Updated weights for policy 0, policy_version 89442 (0.0008) [2023-10-14 21:22:44,587][61552] Updated weights for policy 0, policy_version 89452 (0.0009) [2023-10-14 21:22:44,957][61552] Updated weights for policy 0, policy_version 89462 (0.0008) [2023-10-14 21:22:45,327][61552] Updated weights for policy 0, policy_version 89472 (0.0008) [2023-10-14 21:22:47,560][61585] Updated weights for policy 1, policy_version 89320 (0.0010) [2023-10-14 21:22:47,927][61585] Updated weights for policy 1, policy_version 89330 (0.0010) [2023-10-14 21:22:48,299][61585] Updated weights for policy 1, policy_version 89340 (0.0009) [2023-10-14 21:22:48,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 183074816. Throughput: 0: 1667.3, 1: 1666.4. Samples: 45777968. Policy #0 lag: (min: 0.0, avg: 21.7, max: 32.0) [2023-10-14 21:22:48,344][60425] Avg episode reward: [(0, '77.220'), (1, '79.260')] [2023-10-14 21:22:49,291][61552] Updated weights for policy 0, policy_version 89482 (0.0007) [2023-10-14 21:22:49,664][61552] Updated weights for policy 0, policy_version 89492 (0.0007) [2023-10-14 21:22:50,039][61552] Updated weights for policy 0, policy_version 89502 (0.0008) [2023-10-14 21:22:52,429][61585] Updated weights for policy 1, policy_version 89350 (0.0011) [2023-10-14 21:22:52,798][61585] Updated weights for policy 1, policy_version 89360 (0.0009) [2023-10-14 21:22:53,159][61585] Updated weights for policy 1, policy_version 89370 (0.0009) [2023-10-14 21:22:53,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 183140352. Throughput: 0: 1693.3, 1: 1665.7. Samples: 45798700. Policy #0 lag: (min: 0.0, avg: 21.7, max: 32.0) [2023-10-14 21:22:53,344][60425] Avg episode reward: [(0, '72.590'), (1, '81.260')] [2023-10-14 21:22:54,305][61552] Updated weights for policy 0, policy_version 89512 (0.0007) [2023-10-14 21:22:54,663][61552] Updated weights for policy 0, policy_version 89522 (0.0009) [2023-10-14 21:22:55,034][61552] Updated weights for policy 0, policy_version 89532 (0.0008) [2023-10-14 21:22:57,138][61585] Updated weights for policy 1, policy_version 89380 (0.0008) [2023-10-14 21:22:57,507][61585] Updated weights for policy 1, policy_version 89390 (0.0009) [2023-10-14 21:22:57,877][61585] Updated weights for policy 1, policy_version 89400 (0.0009) [2023-10-14 21:22:58,343][60425] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 183238656. Throughput: 0: 1694.8, 1: 1648.5. Samples: 45818530. Policy #0 lag: (min: 0.0, avg: 21.7, max: 32.0) [2023-10-14 21:22:58,344][60425] Avg episode reward: [(0, '71.790'), (1, '82.250')] [2023-10-14 21:22:59,120][61552] Updated weights for policy 0, policy_version 89542 (0.0008) [2023-10-14 21:22:59,490][61552] Updated weights for policy 0, policy_version 89552 (0.0009) [2023-10-14 21:22:59,852][61552] Updated weights for policy 0, policy_version 89562 (0.0008) [2023-10-14 21:23:02,032][61585] Updated weights for policy 1, policy_version 89410 (0.0009) [2023-10-14 21:23:02,404][61585] Updated weights for policy 1, policy_version 89420 (0.0010) [2023-10-14 21:23:02,769][61585] Updated weights for policy 1, policy_version 89430 (0.0008) [2023-10-14 21:23:03,132][61585] Updated weights for policy 1, policy_version 89440 (0.0009) [2023-10-14 21:23:03,343][60425] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 183304192. Throughput: 0: 1680.6, 1: 1664.7. Samples: 45828418. Policy #0 lag: (min: 0.0, avg: 21.7, max: 32.0) [2023-10-14 21:23:03,344][60425] Avg episode reward: [(0, '73.730'), (1, '80.440')] [2023-10-14 21:23:04,050][61552] Updated weights for policy 0, policy_version 89572 (0.0009) [2023-10-14 21:23:04,414][61552] Updated weights for policy 0, policy_version 89582 (0.0008) [2023-10-14 21:23:04,782][61552] Updated weights for policy 0, policy_version 89592 (0.0010) [2023-10-14 21:23:07,121][61585] Updated weights for policy 1, policy_version 89450 (0.0007) [2023-10-14 21:23:07,484][61585] Updated weights for policy 1, policy_version 89460 (0.0008) [2023-10-14 21:23:07,850][61585] Updated weights for policy 1, policy_version 89470 (0.0009) [2023-10-14 21:23:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 183369728. Throughput: 0: 1692.2, 1: 1666.4. Samples: 45848774. Policy #0 lag: (min: 0.0, avg: 21.7, max: 32.0) [2023-10-14 21:23:08,344][60425] Avg episode reward: [(0, '68.960'), (1, '82.250')] [2023-10-14 21:23:08,842][61552] Updated weights for policy 0, policy_version 89602 (0.0010) [2023-10-14 21:23:09,208][61552] Updated weights for policy 0, policy_version 89612 (0.0007) [2023-10-14 21:23:09,578][61552] Updated weights for policy 0, policy_version 89622 (0.0010) [2023-10-14 21:23:09,940][61552] Updated weights for policy 0, policy_version 89632 (0.0009) [2023-10-14 21:23:12,007][61585] Updated weights for policy 1, policy_version 89480 (0.0009) [2023-10-14 21:23:12,380][61585] Updated weights for policy 1, policy_version 89490 (0.0007) [2023-10-14 21:23:12,753][61585] Updated weights for policy 1, policy_version 89500 (0.0007) [2023-10-14 21:23:13,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 183435264. Throughput: 0: 1692.5, 1: 1648.2. Samples: 45868356. Policy #0 lag: (min: 0.0, avg: 21.7, max: 32.0) [2023-10-14 21:23:13,344][60425] Avg episode reward: [(0, '73.890'), (1, '78.630')] [2023-10-14 21:23:14,129][61552] Updated weights for policy 0, policy_version 89642 (0.0010) [2023-10-14 21:23:14,502][61552] Updated weights for policy 0, policy_version 89652 (0.0011) [2023-10-14 21:23:14,866][61552] Updated weights for policy 0, policy_version 89662 (0.0010) [2023-10-14 21:23:16,583][61585] Updated weights for policy 1, policy_version 89510 (0.0009) [2023-10-14 21:23:16,943][61585] Updated weights for policy 1, policy_version 89520 (0.0011) [2023-10-14 21:23:17,312][61585] Updated weights for policy 1, policy_version 89530 (0.0008) [2023-10-14 21:23:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 183500800. Throughput: 0: 1680.7, 1: 1673.6. Samples: 45878554. Policy #0 lag: (min: 0.0, avg: 21.7, max: 32.0) [2023-10-14 21:23:18,344][60425] Avg episode reward: [(0, '74.290'), (1, '77.190')] [2023-10-14 21:23:18,771][61552] Updated weights for policy 0, policy_version 89672 (0.0009) [2023-10-14 21:23:19,148][61552] Updated weights for policy 0, policy_version 89682 (0.0010) [2023-10-14 21:23:19,512][61552] Updated weights for policy 0, policy_version 89692 (0.0009) [2023-10-14 21:23:21,504][61585] Updated weights for policy 1, policy_version 89540 (0.0009) [2023-10-14 21:23:21,857][61585] Updated weights for policy 1, policy_version 89550 (0.0011) [2023-10-14 21:23:22,217][61585] Updated weights for policy 1, policy_version 89560 (0.0010) [2023-10-14 21:23:23,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 183566336. Throughput: 0: 1682.9, 1: 1666.1. Samples: 45898664. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 21:23:23,344][60425] Avg episode reward: [(0, '70.920'), (1, '77.700')] [2023-10-14 21:23:23,376][61552] Updated weights for policy 0, policy_version 89702 (0.0009) [2023-10-14 21:23:23,750][61552] Updated weights for policy 0, policy_version 89712 (0.0010) [2023-10-14 21:23:24,118][61552] Updated weights for policy 0, policy_version 89722 (0.0007) [2023-10-14 21:23:26,492][61585] Updated weights for policy 1, policy_version 89570 (0.0011) [2023-10-14 21:23:26,862][61585] Updated weights for policy 1, policy_version 89580 (0.0009) [2023-10-14 21:23:27,224][61585] Updated weights for policy 1, policy_version 89590 (0.0008) [2023-10-14 21:23:27,587][61585] Updated weights for policy 1, policy_version 89600 (0.0008) [2023-10-14 21:23:28,252][61552] Updated weights for policy 0, policy_version 89732 (0.0007) [2023-10-14 21:23:28,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 183631872. Throughput: 0: 1685.2, 1: 1652.7. Samples: 45918658. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 21:23:28,345][60425] Avg episode reward: [(0, '72.800'), (1, '78.720')] [2023-10-14 21:23:28,619][61552] Updated weights for policy 0, policy_version 89742 (0.0008) [2023-10-14 21:23:28,985][61552] Updated weights for policy 0, policy_version 89752 (0.0009) [2023-10-14 21:23:31,822][61585] Updated weights for policy 1, policy_version 89610 (0.0007) [2023-10-14 21:23:32,191][61585] Updated weights for policy 1, policy_version 89620 (0.0010) [2023-10-14 21:23:32,553][61585] Updated weights for policy 1, policy_version 89630 (0.0009) [2023-10-14 21:23:33,129][61552] Updated weights for policy 0, policy_version 89762 (0.0008) [2023-10-14 21:23:33,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 183697408. Throughput: 0: 1680.9, 1: 1672.4. Samples: 45928868. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 21:23:33,344][60425] Avg episode reward: [(0, '73.670'), (1, '77.250')] [2023-10-14 21:23:33,503][61552] Updated weights for policy 0, policy_version 89772 (0.0008) [2023-10-14 21:23:33,871][61552] Updated weights for policy 0, policy_version 89782 (0.0007) [2023-10-14 21:23:34,239][61552] Updated weights for policy 0, policy_version 89792 (0.0007) [2023-10-14 21:23:36,689][61585] Updated weights for policy 1, policy_version 89640 (0.0008) [2023-10-14 21:23:37,046][61585] Updated weights for policy 1, policy_version 89650 (0.0007) [2023-10-14 21:23:37,413][61585] Updated weights for policy 1, policy_version 89660 (0.0008) [2023-10-14 21:23:38,270][61552] Updated weights for policy 0, policy_version 89802 (0.0008) [2023-10-14 21:23:38,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 183762944. Throughput: 0: 1678.4, 1: 1659.8. Samples: 45948920. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 21:23:38,344][60425] Avg episode reward: [(0, '74.480'), (1, '80.570')] [2023-10-14 21:23:38,640][61552] Updated weights for policy 0, policy_version 89812 (0.0008) [2023-10-14 21:23:39,010][61552] Updated weights for policy 0, policy_version 89822 (0.0010) [2023-10-14 21:23:41,617][61585] Updated weights for policy 1, policy_version 89670 (0.0010) [2023-10-14 21:23:41,971][61585] Updated weights for policy 1, policy_version 89680 (0.0008) [2023-10-14 21:23:42,343][61585] Updated weights for policy 1, policy_version 89690 (0.0009) [2023-10-14 21:23:43,172][61552] Updated weights for policy 0, policy_version 89832 (0.0010) [2023-10-14 21:23:43,344][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13329.3). Total num frames: 183828480. Throughput: 0: 1679.1, 1: 1656.8. Samples: 45968642. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 21:23:43,345][60425] Avg episode reward: [(0, '76.520'), (1, '84.160')] [2023-10-14 21:23:43,353][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000089696_91848704.pth... [2023-10-14 21:23:43,389][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000088128_90243072.pth [2023-10-14 21:23:43,545][61552] Updated weights for policy 0, policy_version 89842 (0.0011) [2023-10-14 21:23:43,901][61552] Updated weights for policy 0, policy_version 89852 (0.0008) [2023-10-14 21:23:44,047][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000089856_92012544.pth... [2023-10-14 21:23:44,079][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000088288_90406912.pth [2023-10-14 21:23:46,299][61585] Updated weights for policy 1, policy_version 89700 (0.0010) [2023-10-14 21:23:46,665][61585] Updated weights for policy 1, policy_version 89710 (0.0007) [2023-10-14 21:23:47,034][61585] Updated weights for policy 1, policy_version 89720 (0.0009) [2023-10-14 21:23:48,007][61552] Updated weights for policy 0, policy_version 89862 (0.0010) [2023-10-14 21:23:48,344][60425] Fps is (10 sec: 13106.5, 60 sec: 13653.2, 300 sec: 13329.3). Total num frames: 183894016. Throughput: 0: 1675.6, 1: 1673.3. Samples: 45979120. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 21:23:48,345][60425] Avg episode reward: [(0, '76.740'), (1, '79.090')] [2023-10-14 21:23:48,375][61552] Updated weights for policy 0, policy_version 89872 (0.0010) [2023-10-14 21:23:48,750][61552] Updated weights for policy 0, policy_version 89882 (0.0009) [2023-10-14 21:23:51,203][61585] Updated weights for policy 1, policy_version 89730 (0.0009) [2023-10-14 21:23:51,566][61585] Updated weights for policy 1, policy_version 89740 (0.0009) [2023-10-14 21:23:51,926][61585] Updated weights for policy 1, policy_version 89750 (0.0007) [2023-10-14 21:23:52,301][61585] Updated weights for policy 1, policy_version 89760 (0.0009) [2023-10-14 21:23:52,807][61552] Updated weights for policy 0, policy_version 89892 (0.0008) [2023-10-14 21:23:53,181][61552] Updated weights for policy 0, policy_version 89902 (0.0008) [2023-10-14 21:23:53,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 183959552. Throughput: 0: 1681.7, 1: 1657.8. Samples: 45999052. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 21:23:53,344][60425] Avg episode reward: [(0, '76.720'), (1, '82.600')] [2023-10-14 21:23:53,541][61552] Updated weights for policy 0, policy_version 89912 (0.0009) [2023-10-14 21:23:56,403][61585] Updated weights for policy 1, policy_version 89770 (0.0010) [2023-10-14 21:23:56,770][61585] Updated weights for policy 1, policy_version 89780 (0.0010) [2023-10-14 21:23:57,141][61585] Updated weights for policy 1, policy_version 89790 (0.0007) [2023-10-14 21:23:57,517][61552] Updated weights for policy 0, policy_version 89922 (0.0008) [2023-10-14 21:23:57,882][61552] Updated weights for policy 0, policy_version 89932 (0.0007) [2023-10-14 21:23:58,249][61552] Updated weights for policy 0, policy_version 89942 (0.0007) [2023-10-14 21:23:58,343][60425] Fps is (10 sec: 13108.0, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 184025088. Throughput: 0: 1677.8, 1: 1666.9. Samples: 46018868. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 21:23:58,344][60425] Avg episode reward: [(0, '73.830'), (1, '84.680')] [2023-10-14 21:23:58,612][61552] Updated weights for policy 0, policy_version 89952 (0.0007) [2023-10-14 21:24:01,079][61585] Updated weights for policy 1, policy_version 89800 (0.0009) [2023-10-14 21:24:01,448][61585] Updated weights for policy 1, policy_version 89810 (0.0010) [2023-10-14 21:24:01,820][61585] Updated weights for policy 1, policy_version 89820 (0.0011) [2023-10-14 21:24:02,613][61552] Updated weights for policy 0, policy_version 89962 (0.0009) [2023-10-14 21:24:02,985][61552] Updated weights for policy 0, policy_version 89972 (0.0008) [2023-10-14 21:24:03,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 184090624. Throughput: 0: 1692.4, 1: 1670.6. Samples: 46029888. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 21:24:03,344][60425] Avg episode reward: [(0, '77.480'), (1, '80.870')] [2023-10-14 21:24:03,360][61552] Updated weights for policy 0, policy_version 89982 (0.0007) [2023-10-14 21:24:05,801][61585] Updated weights for policy 1, policy_version 89830 (0.0008) [2023-10-14 21:24:06,163][61585] Updated weights for policy 1, policy_version 89840 (0.0007) [2023-10-14 21:24:06,522][61585] Updated weights for policy 1, policy_version 89850 (0.0007) [2023-10-14 21:24:07,509][61552] Updated weights for policy 0, policy_version 89992 (0.0007) [2023-10-14 21:24:07,875][61552] Updated weights for policy 0, policy_version 90002 (0.0008) [2023-10-14 21:24:08,241][61552] Updated weights for policy 0, policy_version 90012 (0.0008) [2023-10-14 21:24:08,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 184156160. Throughput: 0: 1692.8, 1: 1654.1. Samples: 46049270. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 21:24:08,344][60425] Avg episode reward: [(0, '74.680'), (1, '85.400')] [2023-10-14 21:24:10,675][61585] Updated weights for policy 1, policy_version 89860 (0.0008) [2023-10-14 21:24:11,036][61585] Updated weights for policy 1, policy_version 89870 (0.0009) [2023-10-14 21:24:11,402][61585] Updated weights for policy 1, policy_version 89880 (0.0008) [2023-10-14 21:24:12,399][61552] Updated weights for policy 0, policy_version 90022 (0.0007) [2023-10-14 21:24:12,763][61552] Updated weights for policy 0, policy_version 90032 (0.0009) [2023-10-14 21:24:13,127][61552] Updated weights for policy 0, policy_version 90042 (0.0009) [2023-10-14 21:24:13,343][60425] Fps is (10 sec: 16384.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 184254464. Throughput: 0: 1672.5, 1: 1678.5. Samples: 46069450. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 21:24:13,344][60425] Avg episode reward: [(0, '78.520'), (1, '85.050')] [2023-10-14 21:24:15,363][61585] Updated weights for policy 1, policy_version 89890 (0.0007) [2023-10-14 21:24:15,731][61585] Updated weights for policy 1, policy_version 89900 (0.0007) [2023-10-14 21:24:16,087][61585] Updated weights for policy 1, policy_version 89910 (0.0008) [2023-10-14 21:24:16,460][61585] Updated weights for policy 1, policy_version 89920 (0.0008) [2023-10-14 21:24:17,238][61552] Updated weights for policy 0, policy_version 90052 (0.0008) [2023-10-14 21:24:17,602][61552] Updated weights for policy 0, policy_version 90062 (0.0009) [2023-10-14 21:24:17,967][61552] Updated weights for policy 0, policy_version 90072 (0.0011) [2023-10-14 21:24:18,343][60425] Fps is (10 sec: 16383.9, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 184320000. Throughput: 0: 1685.6, 1: 1669.5. Samples: 46079848. Policy #0 lag: (min: 17.0, avg: 27.7, max: 49.0) [2023-10-14 21:24:18,344][60425] Avg episode reward: [(0, '74.260'), (1, '83.610')] [2023-10-14 21:24:20,607][61585] Updated weights for policy 1, policy_version 89930 (0.0011) [2023-10-14 21:24:20,969][61585] Updated weights for policy 1, policy_version 89940 (0.0010) [2023-10-14 21:24:21,331][61585] Updated weights for policy 1, policy_version 89950 (0.0011) [2023-10-14 21:24:22,126][61552] Updated weights for policy 0, policy_version 90082 (0.0010) [2023-10-14 21:24:22,488][61552] Updated weights for policy 0, policy_version 90092 (0.0008) [2023-10-14 21:24:22,862][61552] Updated weights for policy 0, policy_version 90102 (0.0008) [2023-10-14 21:24:23,232][61552] Updated weights for policy 0, policy_version 90112 (0.0008) [2023-10-14 21:24:23,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 184385536. Throughput: 0: 1685.7, 1: 1666.8. Samples: 46099784. Policy #0 lag: (min: 17.0, avg: 27.7, max: 49.0) [2023-10-14 21:24:23,344][60425] Avg episode reward: [(0, '78.920'), (1, '81.940')] [2023-10-14 21:24:25,403][61585] Updated weights for policy 1, policy_version 89960 (0.0009) [2023-10-14 21:24:25,775][61585] Updated weights for policy 1, policy_version 89970 (0.0008) [2023-10-14 21:24:26,140][61585] Updated weights for policy 1, policy_version 89980 (0.0008) [2023-10-14 21:24:27,267][61552] Updated weights for policy 0, policy_version 90122 (0.0007) [2023-10-14 21:24:27,639][61552] Updated weights for policy 0, policy_version 90132 (0.0009) [2023-10-14 21:24:28,010][61552] Updated weights for policy 0, policy_version 90142 (0.0010) [2023-10-14 21:24:28,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 184451072. Throughput: 0: 1668.0, 1: 1688.5. Samples: 46119684. Policy #0 lag: (min: 17.0, avg: 27.7, max: 49.0) [2023-10-14 21:24:28,345][60425] Avg episode reward: [(0, '79.200'), (1, '81.720')] [2023-10-14 21:24:30,292][61585] Updated weights for policy 1, policy_version 89990 (0.0008) [2023-10-14 21:24:30,649][61585] Updated weights for policy 1, policy_version 90000 (0.0008) [2023-10-14 21:24:31,020][61585] Updated weights for policy 1, policy_version 90010 (0.0009) [2023-10-14 21:24:31,989][61552] Updated weights for policy 0, policy_version 90152 (0.0008) [2023-10-14 21:24:32,359][61552] Updated weights for policy 0, policy_version 90162 (0.0007) [2023-10-14 21:24:32,729][61552] Updated weights for policy 0, policy_version 90172 (0.0007) [2023-10-14 21:24:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 184516608. Throughput: 0: 1685.8, 1: 1664.9. Samples: 46129896. Policy #0 lag: (min: 17.0, avg: 27.7, max: 49.0) [2023-10-14 21:24:33,344][60425] Avg episode reward: [(0, '78.550'), (1, '81.680')] [2023-10-14 21:24:35,319][61585] Updated weights for policy 1, policy_version 90020 (0.0010) [2023-10-14 21:24:35,680][61585] Updated weights for policy 1, policy_version 90030 (0.0010) [2023-10-14 21:24:36,046][61585] Updated weights for policy 1, policy_version 90040 (0.0009) [2023-10-14 21:24:36,942][61552] Updated weights for policy 0, policy_version 90182 (0.0008) [2023-10-14 21:24:37,314][61552] Updated weights for policy 0, policy_version 90192 (0.0007) [2023-10-14 21:24:37,674][61552] Updated weights for policy 0, policy_version 90202 (0.0007) [2023-10-14 21:24:38,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 184582144. Throughput: 0: 1681.2, 1: 1666.6. Samples: 46149706. Policy #0 lag: (min: 17.0, avg: 27.7, max: 49.0) [2023-10-14 21:24:38,345][60425] Avg episode reward: [(0, '79.770'), (1, '79.410')] [2023-10-14 21:24:40,093][61585] Updated weights for policy 1, policy_version 90050 (0.0008) [2023-10-14 21:24:40,463][61585] Updated weights for policy 1, policy_version 90060 (0.0008) [2023-10-14 21:24:40,831][61585] Updated weights for policy 1, policy_version 90070 (0.0008) [2023-10-14 21:24:41,193][61585] Updated weights for policy 1, policy_version 90080 (0.0007) [2023-10-14 21:24:41,656][61552] Updated weights for policy 0, policy_version 90212 (0.0010) [2023-10-14 21:24:42,027][61552] Updated weights for policy 0, policy_version 90222 (0.0010) [2023-10-14 21:24:42,397][61552] Updated weights for policy 0, policy_version 90232 (0.0009) [2023-10-14 21:24:43,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.5). Total num frames: 184647680. Throughput: 0: 1659.1, 1: 1679.9. Samples: 46169128. Policy #0 lag: (min: 17.0, avg: 27.7, max: 49.0) [2023-10-14 21:24:43,345][60425] Avg episode reward: [(0, '80.410'), (1, '82.780')] [2023-10-14 21:24:45,300][61585] Updated weights for policy 1, policy_version 90090 (0.0009) [2023-10-14 21:24:45,662][61585] Updated weights for policy 1, policy_version 90100 (0.0008) [2023-10-14 21:24:46,028][61585] Updated weights for policy 1, policy_version 90110 (0.0009) [2023-10-14 21:24:46,498][61552] Updated weights for policy 0, policy_version 90242 (0.0008) [2023-10-14 21:24:46,866][61552] Updated weights for policy 0, policy_version 90252 (0.0009) [2023-10-14 21:24:47,241][61552] Updated weights for policy 0, policy_version 90262 (0.0009) [2023-10-14 21:24:47,612][61552] Updated weights for policy 0, policy_version 90272 (0.0010) [2023-10-14 21:24:48,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13653.5, 300 sec: 13440.4). Total num frames: 184713216. Throughput: 0: 1675.4, 1: 1653.2. Samples: 46179674. Policy #0 lag: (min: 17.0, avg: 27.7, max: 49.0) [2023-10-14 21:24:48,344][60425] Avg episode reward: [(0, '78.070'), (1, '77.970')] [2023-10-14 21:24:50,133][61585] Updated weights for policy 1, policy_version 90120 (0.0010) [2023-10-14 21:24:50,490][61585] Updated weights for policy 1, policy_version 90130 (0.0008) [2023-10-14 21:24:50,857][61585] Updated weights for policy 1, policy_version 90140 (0.0007) [2023-10-14 21:24:51,702][61552] Updated weights for policy 0, policy_version 90282 (0.0010) [2023-10-14 21:24:52,063][61552] Updated weights for policy 0, policy_version 90292 (0.0009) [2023-10-14 21:24:52,427][61552] Updated weights for policy 0, policy_version 90302 (0.0010) [2023-10-14 21:24:53,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 184778752. Throughput: 0: 1665.1, 1: 1669.1. Samples: 46199306. Policy #0 lag: (min: 17.0, avg: 27.7, max: 49.0) [2023-10-14 21:24:53,344][60425] Avg episode reward: [(0, '77.240'), (1, '80.600')] [2023-10-14 21:24:55,124][61585] Updated weights for policy 1, policy_version 90150 (0.0007) [2023-10-14 21:24:55,482][61585] Updated weights for policy 1, policy_version 90160 (0.0008) [2023-10-14 21:24:55,848][61585] Updated weights for policy 1, policy_version 90170 (0.0008) [2023-10-14 21:24:56,522][61552] Updated weights for policy 0, policy_version 90312 (0.0008) [2023-10-14 21:24:56,892][61552] Updated weights for policy 0, policy_version 90322 (0.0010) [2023-10-14 21:24:57,259][61552] Updated weights for policy 0, policy_version 90332 (0.0007) [2023-10-14 21:24:58,343][60425] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 184844288. Throughput: 0: 1660.7, 1: 1668.0. Samples: 46219242. Policy #0 lag: (min: 17.0, avg: 27.7, max: 49.0) [2023-10-14 21:24:58,345][60425] Avg episode reward: [(0, '80.410'), (1, '77.000')] [2023-10-14 21:24:59,867][61585] Updated weights for policy 1, policy_version 90180 (0.0008) [2023-10-14 21:25:00,236][61585] Updated weights for policy 1, policy_version 90190 (0.0008) [2023-10-14 21:25:00,604][61585] Updated weights for policy 1, policy_version 90200 (0.0009) [2023-10-14 21:25:01,264][61552] Updated weights for policy 0, policy_version 90342 (0.0011) [2023-10-14 21:25:01,640][61552] Updated weights for policy 0, policy_version 90352 (0.0010) [2023-10-14 21:25:02,015][61552] Updated weights for policy 0, policy_version 90362 (0.0009) [2023-10-14 21:25:03,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 184909824. Throughput: 0: 1683.5, 1: 1655.4. Samples: 46230098. Policy #0 lag: (min: 17.0, avg: 27.7, max: 49.0) [2023-10-14 21:25:03,344][60425] Avg episode reward: [(0, '77.080'), (1, '78.870')] [2023-10-14 21:25:04,763][61585] Updated weights for policy 1, policy_version 90210 (0.0008) [2023-10-14 21:25:05,125][61585] Updated weights for policy 1, policy_version 90220 (0.0007) [2023-10-14 21:25:05,488][61585] Updated weights for policy 1, policy_version 90230 (0.0007) [2023-10-14 21:25:05,849][61585] Updated weights for policy 1, policy_version 90240 (0.0008) [2023-10-14 21:25:05,932][61552] Updated weights for policy 0, policy_version 90372 (0.0008) [2023-10-14 21:25:06,295][61552] Updated weights for policy 0, policy_version 90382 (0.0009) [2023-10-14 21:25:06,662][61552] Updated weights for policy 0, policy_version 90392 (0.0009) [2023-10-14 21:25:08,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 184975360. Throughput: 0: 1666.9, 1: 1667.6. Samples: 46249838. Policy #0 lag: (min: 17.0, avg: 27.7, max: 49.0) [2023-10-14 21:25:08,345][60425] Avg episode reward: [(0, '78.290'), (1, '74.910')] [2023-10-14 21:25:09,768][61585] Updated weights for policy 1, policy_version 90250 (0.0007) [2023-10-14 21:25:10,140][61585] Updated weights for policy 1, policy_version 90260 (0.0008) [2023-10-14 21:25:10,500][61585] Updated weights for policy 1, policy_version 90270 (0.0009) [2023-10-14 21:25:10,668][61552] Updated weights for policy 0, policy_version 90402 (0.0008) [2023-10-14 21:25:11,026][61552] Updated weights for policy 0, policy_version 90412 (0.0008) [2023-10-14 21:25:11,397][61552] Updated weights for policy 0, policy_version 90422 (0.0009) [2023-10-14 21:25:11,763][61552] Updated weights for policy 0, policy_version 90432 (0.0009) [2023-10-14 21:25:13,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 185040896. Throughput: 0: 1678.6, 1: 1671.4. Samples: 46270434. Policy #0 lag: (min: 17.0, avg: 27.7, max: 49.0) [2023-10-14 21:25:13,345][60425] Avg episode reward: [(0, '77.060'), (1, '75.810')] [2023-10-14 21:25:14,643][61585] Updated weights for policy 1, policy_version 90280 (0.0009) [2023-10-14 21:25:15,005][61585] Updated weights for policy 1, policy_version 90290 (0.0011) [2023-10-14 21:25:15,371][61585] Updated weights for policy 1, policy_version 90300 (0.0008) [2023-10-14 21:25:15,848][61552] Updated weights for policy 0, policy_version 90442 (0.0010) [2023-10-14 21:25:16,218][61552] Updated weights for policy 0, policy_version 90452 (0.0009) [2023-10-14 21:25:16,582][61552] Updated weights for policy 0, policy_version 90462 (0.0008) [2023-10-14 21:25:18,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 185106432. Throughput: 0: 1687.5, 1: 1659.4. Samples: 46280506. Policy #0 lag: (min: 2.0, avg: 3.7, max: 29.0) [2023-10-14 21:25:18,344][60425] Avg episode reward: [(0, '76.800'), (1, '82.190')] [2023-10-14 21:25:19,227][61585] Updated weights for policy 1, policy_version 90310 (0.0007) [2023-10-14 21:25:19,588][61585] Updated weights for policy 1, policy_version 90320 (0.0009) [2023-10-14 21:25:19,938][61585] Updated weights for policy 1, policy_version 90330 (0.0009) [2023-10-14 21:25:20,717][61552] Updated weights for policy 0, policy_version 90472 (0.0010) [2023-10-14 21:25:21,087][61552] Updated weights for policy 0, policy_version 90482 (0.0009) [2023-10-14 21:25:21,451][61552] Updated weights for policy 0, policy_version 90492 (0.0009) [2023-10-14 21:25:23,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 185171968. Throughput: 0: 1660.6, 1: 1682.9. Samples: 46300162. Policy #0 lag: (min: 2.0, avg: 3.7, max: 29.0) [2023-10-14 21:25:23,344][60425] Avg episode reward: [(0, '73.690'), (1, '79.680')] [2023-10-14 21:25:24,144][61585] Updated weights for policy 1, policy_version 90340 (0.0009) [2023-10-14 21:25:24,520][61585] Updated weights for policy 1, policy_version 90350 (0.0008) [2023-10-14 21:25:24,882][61585] Updated weights for policy 1, policy_version 90360 (0.0008) [2023-10-14 21:25:25,608][61552] Updated weights for policy 0, policy_version 90502 (0.0007) [2023-10-14 21:25:25,971][61552] Updated weights for policy 0, policy_version 90512 (0.0008) [2023-10-14 21:25:26,345][61552] Updated weights for policy 0, policy_version 90522 (0.0010) [2023-10-14 21:25:28,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 185237504. Throughput: 0: 1685.0, 1: 1684.4. Samples: 46320750. Policy #0 lag: (min: 2.0, avg: 3.7, max: 29.0) [2023-10-14 21:25:28,345][60425] Avg episode reward: [(0, '79.140'), (1, '80.580')] [2023-10-14 21:25:28,982][61585] Updated weights for policy 1, policy_version 90370 (0.0007) [2023-10-14 21:25:29,350][61585] Updated weights for policy 1, policy_version 90380 (0.0009) [2023-10-14 21:25:29,707][61585] Updated weights for policy 1, policy_version 90390 (0.0010) [2023-10-14 21:25:30,068][61585] Updated weights for policy 1, policy_version 90400 (0.0010) [2023-10-14 21:25:30,367][61552] Updated weights for policy 0, policy_version 90532 (0.0008) [2023-10-14 21:25:30,727][61552] Updated weights for policy 0, policy_version 90542 (0.0008) [2023-10-14 21:25:31,091][61552] Updated weights for policy 0, policy_version 90552 (0.0009) [2023-10-14 21:25:33,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 185303040. Throughput: 0: 1676.0, 1: 1678.9. Samples: 46330646. Policy #0 lag: (min: 2.0, avg: 3.7, max: 29.0) [2023-10-14 21:25:33,344][60425] Avg episode reward: [(0, '76.250'), (1, '82.220')] [2023-10-14 21:25:34,320][61585] Updated weights for policy 1, policy_version 90410 (0.0008) [2023-10-14 21:25:34,685][61585] Updated weights for policy 1, policy_version 90420 (0.0008) [2023-10-14 21:25:35,047][61585] Updated weights for policy 1, policy_version 90430 (0.0009) [2023-10-14 21:25:35,079][61552] Updated weights for policy 0, policy_version 90562 (0.0008) [2023-10-14 21:25:35,445][61552] Updated weights for policy 0, policy_version 90572 (0.0010) [2023-10-14 21:25:35,812][61552] Updated weights for policy 0, policy_version 90582 (0.0011) [2023-10-14 21:25:36,178][61552] Updated weights for policy 0, policy_version 90592 (0.0009) [2023-10-14 21:25:38,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 185368576. Throughput: 0: 1669.1, 1: 1684.8. Samples: 46350234. Policy #0 lag: (min: 2.0, avg: 3.7, max: 29.0) [2023-10-14 21:25:38,344][60425] Avg episode reward: [(0, '78.440'), (1, '82.940')] [2023-10-14 21:25:38,971][61585] Updated weights for policy 1, policy_version 90440 (0.0008) [2023-10-14 21:25:39,338][61585] Updated weights for policy 1, policy_version 90450 (0.0007) [2023-10-14 21:25:39,697][61585] Updated weights for policy 1, policy_version 90460 (0.0007) [2023-10-14 21:25:40,591][61552] Updated weights for policy 0, policy_version 90602 (0.0008) [2023-10-14 21:25:40,971][61552] Updated weights for policy 0, policy_version 90612 (0.0011) [2023-10-14 21:25:41,331][61552] Updated weights for policy 0, policy_version 90622 (0.0007) [2023-10-14 21:25:43,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 185434112. Throughput: 0: 1681.1, 1: 1685.0. Samples: 46370716. Policy #0 lag: (min: 2.0, avg: 3.7, max: 29.0) [2023-10-14 21:25:43,344][60425] Avg episode reward: [(0, '77.590'), (1, '75.100')] [2023-10-14 21:25:43,356][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000090464_92635136.pth... [2023-10-14 21:25:43,356][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000090624_92798976.pth... [2023-10-14 21:25:43,386][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000088928_91062272.pth [2023-10-14 21:25:43,389][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000089056_91193344.pth [2023-10-14 21:25:43,766][61585] Updated weights for policy 1, policy_version 90470 (0.0010) [2023-10-14 21:25:44,120][61585] Updated weights for policy 1, policy_version 90480 (0.0009) [2023-10-14 21:25:44,494][61585] Updated weights for policy 1, policy_version 90490 (0.0009) [2023-10-14 21:25:45,409][61552] Updated weights for policy 0, policy_version 90632 (0.0008) [2023-10-14 21:25:45,772][61552] Updated weights for policy 0, policy_version 90642 (0.0007) [2023-10-14 21:25:46,128][61552] Updated weights for policy 0, policy_version 90652 (0.0008) [2023-10-14 21:25:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 185499648. Throughput: 0: 1660.2, 1: 1679.3. Samples: 46380376. Policy #0 lag: (min: 2.0, avg: 3.7, max: 29.0) [2023-10-14 21:25:48,344][60425] Avg episode reward: [(0, '76.950'), (1, '81.060')] [2023-10-14 21:25:48,590][61585] Updated weights for policy 1, policy_version 90500 (0.0009) [2023-10-14 21:25:48,964][61585] Updated weights for policy 1, policy_version 90510 (0.0009) [2023-10-14 21:25:49,335][61585] Updated weights for policy 1, policy_version 90520 (0.0007) [2023-10-14 21:25:50,282][61552] Updated weights for policy 0, policy_version 90662 (0.0007) [2023-10-14 21:25:50,642][61552] Updated weights for policy 0, policy_version 90672 (0.0007) [2023-10-14 21:25:51,014][61552] Updated weights for policy 0, policy_version 90682 (0.0008) [2023-10-14 21:25:53,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 185565184. Throughput: 0: 1657.7, 1: 1684.1. Samples: 46400220. Policy #0 lag: (min: 2.0, avg: 3.7, max: 29.0) [2023-10-14 21:25:53,344][60425] Avg episode reward: [(0, '83.640'), (1, '82.550')] [2023-10-14 21:25:53,454][61585] Updated weights for policy 1, policy_version 90530 (0.0008) [2023-10-14 21:25:53,832][61585] Updated weights for policy 1, policy_version 90540 (0.0007) [2023-10-14 21:25:54,196][61585] Updated weights for policy 1, policy_version 90550 (0.0009) [2023-10-14 21:25:54,553][61585] Updated weights for policy 1, policy_version 90560 (0.0009) [2023-10-14 21:25:55,062][61552] Updated weights for policy 0, policy_version 90692 (0.0009) [2023-10-14 21:25:55,437][61552] Updated weights for policy 0, policy_version 90702 (0.0008) [2023-10-14 21:25:55,812][61552] Updated weights for policy 0, policy_version 90712 (0.0010) [2023-10-14 21:25:58,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 185630720. Throughput: 0: 1664.6, 1: 1683.5. Samples: 46421098. Policy #0 lag: (min: 2.0, avg: 3.7, max: 29.0) [2023-10-14 21:25:58,345][60425] Avg episode reward: [(0, '80.440'), (1, '78.170')] [2023-10-14 21:25:58,662][61585] Updated weights for policy 1, policy_version 90570 (0.0011) [2023-10-14 21:25:59,031][61585] Updated weights for policy 1, policy_version 90580 (0.0011) [2023-10-14 21:25:59,402][61585] Updated weights for policy 1, policy_version 90590 (0.0009) [2023-10-14 21:25:59,817][61552] Updated weights for policy 0, policy_version 90722 (0.0009) [2023-10-14 21:26:00,196][61552] Updated weights for policy 0, policy_version 90732 (0.0009) [2023-10-14 21:26:00,559][61552] Updated weights for policy 0, policy_version 90742 (0.0008) [2023-10-14 21:26:00,924][61552] Updated weights for policy 0, policy_version 90752 (0.0009) [2023-10-14 21:26:03,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 185696256. Throughput: 0: 1648.0, 1: 1684.6. Samples: 46430472. Policy #0 lag: (min: 2.0, avg: 3.7, max: 29.0) [2023-10-14 21:26:03,344][60425] Avg episode reward: [(0, '77.100'), (1, '76.370')] [2023-10-14 21:26:03,571][61585] Updated weights for policy 1, policy_version 90600 (0.0008) [2023-10-14 21:26:03,935][61585] Updated weights for policy 1, policy_version 90610 (0.0008) [2023-10-14 21:26:04,295][61585] Updated weights for policy 1, policy_version 90620 (0.0007) [2023-10-14 21:26:04,912][61552] Updated weights for policy 0, policy_version 90762 (0.0009) [2023-10-14 21:26:05,278][61552] Updated weights for policy 0, policy_version 90772 (0.0011) [2023-10-14 21:26:05,650][61552] Updated weights for policy 0, policy_version 90782 (0.0007) [2023-10-14 21:26:08,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 185761792. Throughput: 0: 1671.6, 1: 1678.8. Samples: 46450934. Policy #0 lag: (min: 2.0, avg: 3.7, max: 29.0) [2023-10-14 21:26:08,344][60425] Avg episode reward: [(0, '77.310'), (1, '81.270')] [2023-10-14 21:26:08,379][61585] Updated weights for policy 1, policy_version 90630 (0.0009) [2023-10-14 21:26:08,735][61585] Updated weights for policy 1, policy_version 90640 (0.0007) [2023-10-14 21:26:09,090][61585] Updated weights for policy 1, policy_version 90650 (0.0008) [2023-10-14 21:26:09,753][61552] Updated weights for policy 0, policy_version 90792 (0.0008) [2023-10-14 21:26:10,136][61552] Updated weights for policy 0, policy_version 90802 (0.0009) [2023-10-14 21:26:10,507][61552] Updated weights for policy 0, policy_version 90812 (0.0008) [2023-10-14 21:26:13,019][61585] Updated weights for policy 1, policy_version 90660 (0.0008) [2023-10-14 21:26:13,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 185827328. Throughput: 0: 1672.9, 1: 1679.3. Samples: 46471600. Policy #0 lag: (min: 2.0, avg: 3.7, max: 29.0) [2023-10-14 21:26:13,344][60425] Avg episode reward: [(0, '83.790'), (1, '78.870')] [2023-10-14 21:26:13,391][61585] Updated weights for policy 1, policy_version 90670 (0.0009) [2023-10-14 21:26:13,767][61585] Updated weights for policy 1, policy_version 90680 (0.0009) [2023-10-14 21:26:14,579][61552] Updated weights for policy 0, policy_version 90822 (0.0010) [2023-10-14 21:26:14,938][61552] Updated weights for policy 0, policy_version 90832 (0.0011) [2023-10-14 21:26:15,299][61552] Updated weights for policy 0, policy_version 90842 (0.0009) [2023-10-14 21:26:17,840][61585] Updated weights for policy 1, policy_version 90690 (0.0009) [2023-10-14 21:26:18,200][61585] Updated weights for policy 1, policy_version 90700 (0.0008) [2023-10-14 21:26:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 185892864. Throughput: 0: 1655.8, 1: 1678.9. Samples: 46480708. Policy #0 lag: (min: 16.0, avg: 36.4, max: 48.0) [2023-10-14 21:26:18,344][60425] Avg episode reward: [(0, '83.130'), (1, '79.420')] [2023-10-14 21:26:18,567][61585] Updated weights for policy 1, policy_version 90710 (0.0012) [2023-10-14 21:26:18,930][61585] Updated weights for policy 1, policy_version 90720 (0.0010) [2023-10-14 21:26:19,283][61552] Updated weights for policy 0, policy_version 90852 (0.0007) [2023-10-14 21:26:19,653][61552] Updated weights for policy 0, policy_version 90862 (0.0008) [2023-10-14 21:26:20,027][61552] Updated weights for policy 0, policy_version 90872 (0.0010) [2023-10-14 21:26:22,934][61585] Updated weights for policy 1, policy_version 90730 (0.0009) [2023-10-14 21:26:23,304][61585] Updated weights for policy 1, policy_version 90740 (0.0007) [2023-10-14 21:26:23,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 185958400. Throughput: 0: 1675.1, 1: 1685.2. Samples: 46501448. Policy #0 lag: (min: 16.0, avg: 36.4, max: 48.0) [2023-10-14 21:26:23,344][60425] Avg episode reward: [(0, '82.600'), (1, '78.360')] [2023-10-14 21:26:23,667][61585] Updated weights for policy 1, policy_version 90750 (0.0007) [2023-10-14 21:26:24,058][61552] Updated weights for policy 0, policy_version 90882 (0.0009) [2023-10-14 21:26:24,431][61552] Updated weights for policy 0, policy_version 90892 (0.0008) [2023-10-14 21:26:24,796][61552] Updated weights for policy 0, policy_version 90902 (0.0010) [2023-10-14 21:26:25,162][61552] Updated weights for policy 0, policy_version 90912 (0.0007) [2023-10-14 21:26:27,696][61585] Updated weights for policy 1, policy_version 90760 (0.0008) [2023-10-14 21:26:28,055][61585] Updated weights for policy 1, policy_version 90770 (0.0009) [2023-10-14 21:26:28,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 186023936. Throughput: 0: 1683.3, 1: 1672.7. Samples: 46521738. Policy #0 lag: (min: 16.0, avg: 36.4, max: 48.0) [2023-10-14 21:26:28,344][60425] Avg episode reward: [(0, '79.520'), (1, '79.170')] [2023-10-14 21:26:28,417][61585] Updated weights for policy 1, policy_version 90780 (0.0011) [2023-10-14 21:26:29,332][61552] Updated weights for policy 0, policy_version 90922 (0.0011) [2023-10-14 21:26:29,697][61552] Updated weights for policy 0, policy_version 90932 (0.0008) [2023-10-14 21:26:30,067][61552] Updated weights for policy 0, policy_version 90942 (0.0010) [2023-10-14 21:26:32,530][61585] Updated weights for policy 1, policy_version 90790 (0.0009) [2023-10-14 21:26:32,891][61585] Updated weights for policy 1, policy_version 90800 (0.0008) [2023-10-14 21:26:33,254][61585] Updated weights for policy 1, policy_version 90810 (0.0008) [2023-10-14 21:26:33,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 186089472. Throughput: 0: 1665.5, 1: 1681.3. Samples: 46530982. Policy #0 lag: (min: 16.0, avg: 36.4, max: 48.0) [2023-10-14 21:26:33,344][60425] Avg episode reward: [(0, '78.620'), (1, '83.480')] [2023-10-14 21:26:34,141][61552] Updated weights for policy 0, policy_version 90952 (0.0009) [2023-10-14 21:26:34,503][61552] Updated weights for policy 0, policy_version 90962 (0.0009) [2023-10-14 21:26:34,867][61552] Updated weights for policy 0, policy_version 90972 (0.0009) [2023-10-14 21:26:37,444][61585] Updated weights for policy 1, policy_version 90820 (0.0009) [2023-10-14 21:26:37,799][61585] Updated weights for policy 1, policy_version 90830 (0.0009) [2023-10-14 21:26:38,165][61585] Updated weights for policy 1, policy_version 90840 (0.0007) [2023-10-14 21:26:38,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 186155008. Throughput: 0: 1689.3, 1: 1680.1. Samples: 46551846. Policy #0 lag: (min: 16.0, avg: 36.4, max: 48.0) [2023-10-14 21:26:38,344][60425] Avg episode reward: [(0, '86.000'), (1, '83.490')] [2023-10-14 21:26:38,896][61552] Updated weights for policy 0, policy_version 90982 (0.0008) [2023-10-14 21:26:39,259][61552] Updated weights for policy 0, policy_version 90992 (0.0007) [2023-10-14 21:26:39,630][61552] Updated weights for policy 0, policy_version 91002 (0.0007) [2023-10-14 21:26:42,250][61585] Updated weights for policy 1, policy_version 90850 (0.0007) [2023-10-14 21:26:42,617][61585] Updated weights for policy 1, policy_version 90860 (0.0007) [2023-10-14 21:26:42,983][61585] Updated weights for policy 1, policy_version 90870 (0.0007) [2023-10-14 21:26:43,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 186220544. Throughput: 0: 1691.3, 1: 1662.8. Samples: 46572036. Policy #0 lag: (min: 16.0, avg: 36.4, max: 48.0) [2023-10-14 21:26:43,344][60425] Avg episode reward: [(0, '81.770'), (1, '83.440')] [2023-10-14 21:26:43,345][61585] Updated weights for policy 1, policy_version 90880 (0.0010) [2023-10-14 21:26:43,798][61552] Updated weights for policy 0, policy_version 91012 (0.0008) [2023-10-14 21:26:44,166][61552] Updated weights for policy 0, policy_version 91022 (0.0009) [2023-10-14 21:26:44,534][61552] Updated weights for policy 0, policy_version 91032 (0.0008) [2023-10-14 21:26:47,422][61585] Updated weights for policy 1, policy_version 90890 (0.0010) [2023-10-14 21:26:47,783][61585] Updated weights for policy 1, policy_version 90900 (0.0007) [2023-10-14 21:26:48,150][61585] Updated weights for policy 1, policy_version 90910 (0.0008) [2023-10-14 21:26:48,343][60425] Fps is (10 sec: 16384.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 186318848. Throughput: 0: 1681.2, 1: 1677.6. Samples: 46581620. Policy #0 lag: (min: 16.0, avg: 36.4, max: 48.0) [2023-10-14 21:26:48,344][60425] Avg episode reward: [(0, '77.190'), (1, '83.120')] [2023-10-14 21:26:48,875][61552] Updated weights for policy 0, policy_version 91042 (0.0008) [2023-10-14 21:26:49,244][61552] Updated weights for policy 0, policy_version 91052 (0.0008) [2023-10-14 21:26:49,614][61552] Updated weights for policy 0, policy_version 91062 (0.0010) [2023-10-14 21:26:49,979][61552] Updated weights for policy 0, policy_version 91072 (0.0008) [2023-10-14 21:26:52,298][61585] Updated weights for policy 1, policy_version 90920 (0.0007) [2023-10-14 21:26:52,668][61585] Updated weights for policy 1, policy_version 90930 (0.0011) [2023-10-14 21:26:53,033][61585] Updated weights for policy 1, policy_version 90940 (0.0008) [2023-10-14 21:26:53,344][60425] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 186384384. Throughput: 0: 1680.1, 1: 1676.6. Samples: 46601988. Policy #0 lag: (min: 16.0, avg: 36.4, max: 48.0) [2023-10-14 21:26:53,345][60425] Avg episode reward: [(0, '78.040'), (1, '82.180')] [2023-10-14 21:26:54,132][61552] Updated weights for policy 0, policy_version 91082 (0.0007) [2023-10-14 21:26:54,513][61552] Updated weights for policy 0, policy_version 91092 (0.0007) [2023-10-14 21:26:54,879][61552] Updated weights for policy 0, policy_version 91102 (0.0007) [2023-10-14 21:26:57,175][61585] Updated weights for policy 1, policy_version 90950 (0.0008) [2023-10-14 21:26:57,531][61585] Updated weights for policy 1, policy_version 90960 (0.0007) [2023-10-14 21:26:57,900][61585] Updated weights for policy 1, policy_version 90970 (0.0009) [2023-10-14 21:26:58,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 186449920. Throughput: 0: 1677.5, 1: 1656.8. Samples: 46621646. Policy #0 lag: (min: 16.0, avg: 36.4, max: 48.0) [2023-10-14 21:26:58,344][60425] Avg episode reward: [(0, '77.750'), (1, '78.750')] [2023-10-14 21:26:59,064][61552] Updated weights for policy 0, policy_version 91112 (0.0011) [2023-10-14 21:26:59,432][61552] Updated weights for policy 0, policy_version 91122 (0.0010) [2023-10-14 21:26:59,796][61552] Updated weights for policy 0, policy_version 91132 (0.0009) [2023-10-14 21:27:02,036][61585] Updated weights for policy 1, policy_version 90980 (0.0011) [2023-10-14 21:27:02,396][61585] Updated weights for policy 1, policy_version 90990 (0.0010) [2023-10-14 21:27:02,769][61585] Updated weights for policy 1, policy_version 91000 (0.0008) [2023-10-14 21:27:03,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 186515456. Throughput: 0: 1676.5, 1: 1675.1. Samples: 46631528. Policy #0 lag: (min: 16.0, avg: 36.4, max: 48.0) [2023-10-14 21:27:03,344][60425] Avg episode reward: [(0, '81.370'), (1, '83.490')] [2023-10-14 21:27:03,975][61552] Updated weights for policy 0, policy_version 91142 (0.0010) [2023-10-14 21:27:04,353][61552] Updated weights for policy 0, policy_version 91152 (0.0010) [2023-10-14 21:27:04,718][61552] Updated weights for policy 0, policy_version 91162 (0.0010) [2023-10-14 21:27:06,876][61585] Updated weights for policy 1, policy_version 91010 (0.0009) [2023-10-14 21:27:07,234][61585] Updated weights for policy 1, policy_version 91020 (0.0009) [2023-10-14 21:27:07,605][61585] Updated weights for policy 1, policy_version 91030 (0.0007) [2023-10-14 21:27:07,975][61585] Updated weights for policy 1, policy_version 91040 (0.0008) [2023-10-14 21:27:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 186580992. Throughput: 0: 1675.8, 1: 1670.6. Samples: 46652036. Policy #0 lag: (min: 16.0, avg: 36.4, max: 48.0) [2023-10-14 21:27:08,344][60425] Avg episode reward: [(0, '82.330'), (1, '82.470')] [2023-10-14 21:27:08,723][61552] Updated weights for policy 0, policy_version 91172 (0.0009) [2023-10-14 21:27:09,093][61552] Updated weights for policy 0, policy_version 91182 (0.0007) [2023-10-14 21:27:09,459][61552] Updated weights for policy 0, policy_version 91192 (0.0008) [2023-10-14 21:27:12,159][61585] Updated weights for policy 1, policy_version 91050 (0.0009) [2023-10-14 21:27:12,529][61585] Updated weights for policy 1, policy_version 91060 (0.0008) [2023-10-14 21:27:12,905][61585] Updated weights for policy 1, policy_version 91070 (0.0007) [2023-10-14 21:27:13,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 186646528. Throughput: 0: 1674.3, 1: 1654.7. Samples: 46671546. Policy #0 lag: (min: 16.0, avg: 36.4, max: 48.0) [2023-10-14 21:27:13,344][60425] Avg episode reward: [(0, '75.530'), (1, '75.330')] [2023-10-14 21:27:13,424][61552] Updated weights for policy 0, policy_version 91202 (0.0009) [2023-10-14 21:27:13,795][61552] Updated weights for policy 0, policy_version 91212 (0.0011) [2023-10-14 21:27:14,168][61552] Updated weights for policy 0, policy_version 91222 (0.0008) [2023-10-14 21:27:14,537][61552] Updated weights for policy 0, policy_version 91232 (0.0007) [2023-10-14 21:27:17,228][61585] Updated weights for policy 1, policy_version 91080 (0.0009) [2023-10-14 21:27:17,593][61585] Updated weights for policy 1, policy_version 91090 (0.0009) [2023-10-14 21:27:17,957][61585] Updated weights for policy 1, policy_version 91100 (0.0007) [2023-10-14 21:27:18,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 186712064. Throughput: 0: 1680.0, 1: 1664.9. Samples: 46681502. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-14 21:27:18,344][60425] Avg episode reward: [(0, '73.920'), (1, '77.120')] [2023-10-14 21:27:18,619][61552] Updated weights for policy 0, policy_version 91242 (0.0008) [2023-10-14 21:27:18,987][61552] Updated weights for policy 0, policy_version 91252 (0.0009) [2023-10-14 21:27:19,349][61552] Updated weights for policy 0, policy_version 91262 (0.0008) [2023-10-14 21:27:22,068][61585] Updated weights for policy 1, policy_version 91110 (0.0007) [2023-10-14 21:27:22,426][61585] Updated weights for policy 1, policy_version 91120 (0.0007) [2023-10-14 21:27:22,799][61585] Updated weights for policy 1, policy_version 91130 (0.0007) [2023-10-14 21:27:23,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 186777600. Throughput: 0: 1671.5, 1: 1663.2. Samples: 46701906. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-14 21:27:23,344][60425] Avg episode reward: [(0, '76.950'), (1, '79.310')] [2023-10-14 21:27:23,518][61552] Updated weights for policy 0, policy_version 91272 (0.0008) [2023-10-14 21:27:23,899][61552] Updated weights for policy 0, policy_version 91282 (0.0009) [2023-10-14 21:27:24,258][61552] Updated weights for policy 0, policy_version 91292 (0.0007) [2023-10-14 21:27:26,752][61585] Updated weights for policy 1, policy_version 91140 (0.0008) [2023-10-14 21:27:27,112][61585] Updated weights for policy 1, policy_version 91150 (0.0008) [2023-10-14 21:27:27,481][61585] Updated weights for policy 1, policy_version 91160 (0.0011) [2023-10-14 21:27:28,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 186843136. Throughput: 0: 1669.5, 1: 1652.1. Samples: 46721508. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-14 21:27:28,344][60425] Avg episode reward: [(0, '78.970'), (1, '81.630')] [2023-10-14 21:27:28,351][61552] Updated weights for policy 0, policy_version 91302 (0.0007) [2023-10-14 21:27:28,716][61552] Updated weights for policy 0, policy_version 91312 (0.0009) [2023-10-14 21:27:29,079][61552] Updated weights for policy 0, policy_version 91322 (0.0010) [2023-10-14 21:27:31,670][61585] Updated weights for policy 1, policy_version 91170 (0.0010) [2023-10-14 21:27:32,037][61585] Updated weights for policy 1, policy_version 91180 (0.0007) [2023-10-14 21:27:32,409][61585] Updated weights for policy 1, policy_version 91190 (0.0008) [2023-10-14 21:27:32,767][61585] Updated weights for policy 1, policy_version 91200 (0.0009) [2023-10-14 21:27:32,994][61552] Updated weights for policy 0, policy_version 91332 (0.0010) [2023-10-14 21:27:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 186908672. Throughput: 0: 1669.5, 1: 1665.6. Samples: 46731700. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-14 21:27:33,344][60425] Avg episode reward: [(0, '77.030'), (1, '81.670')] [2023-10-14 21:27:33,356][61552] Updated weights for policy 0, policy_version 91342 (0.0008) [2023-10-14 21:27:33,728][61552] Updated weights for policy 0, policy_version 91352 (0.0007) [2023-10-14 21:27:36,668][61585] Updated weights for policy 1, policy_version 91210 (0.0009) [2023-10-14 21:27:37,028][61585] Updated weights for policy 1, policy_version 91220 (0.0010) [2023-10-14 21:27:37,397][61585] Updated weights for policy 1, policy_version 91230 (0.0009) [2023-10-14 21:27:37,785][61552] Updated weights for policy 0, policy_version 91362 (0.0008) [2023-10-14 21:27:38,146][61552] Updated weights for policy 0, policy_version 91372 (0.0007) [2023-10-14 21:27:38,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 186974208. Throughput: 0: 1677.2, 1: 1657.1. Samples: 46752030. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-14 21:27:38,344][60425] Avg episode reward: [(0, '81.030'), (1, '78.350')] [2023-10-14 21:27:38,519][61552] Updated weights for policy 0, policy_version 91382 (0.0007) [2023-10-14 21:27:38,881][61552] Updated weights for policy 0, policy_version 91392 (0.0010) [2023-10-14 21:27:41,579][61585] Updated weights for policy 1, policy_version 91240 (0.0011) [2023-10-14 21:27:41,943][61585] Updated weights for policy 1, policy_version 91250 (0.0007) [2023-10-14 21:27:42,314][61585] Updated weights for policy 1, policy_version 91260 (0.0007) [2023-10-14 21:27:42,949][61552] Updated weights for policy 0, policy_version 91402 (0.0007) [2023-10-14 21:27:43,323][61552] Updated weights for policy 0, policy_version 91412 (0.0011) [2023-10-14 21:27:43,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 187039744. Throughput: 0: 1676.0, 1: 1656.8. Samples: 46771620. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-14 21:27:43,344][60425] Avg episode reward: [(0, '77.020'), (1, '80.370')] [2023-10-14 21:27:43,354][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000091264_93454336.pth... [2023-10-14 21:27:43,389][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000089696_91848704.pth [2023-10-14 21:27:43,694][61552] Updated weights for policy 0, policy_version 91422 (0.0008) [2023-10-14 21:27:43,767][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000091424_93618176.pth... [2023-10-14 21:27:43,810][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000089856_92012544.pth [2023-10-14 21:27:46,393][61585] Updated weights for policy 1, policy_version 91270 (0.0010) [2023-10-14 21:27:46,764][61585] Updated weights for policy 1, policy_version 91280 (0.0010) [2023-10-14 21:27:47,138][61585] Updated weights for policy 1, policy_version 91290 (0.0010) [2023-10-14 21:27:47,643][61552] Updated weights for policy 0, policy_version 91432 (0.0010) [2023-10-14 21:27:48,019][61552] Updated weights for policy 0, policy_version 91442 (0.0010) [2023-10-14 21:27:48,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 187105280. Throughput: 0: 1679.1, 1: 1669.6. Samples: 46782218. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-14 21:27:48,344][60425] Avg episode reward: [(0, '77.280'), (1, '78.560')] [2023-10-14 21:27:48,377][61552] Updated weights for policy 0, policy_version 91452 (0.0009) [2023-10-14 21:27:51,343][61585] Updated weights for policy 1, policy_version 91300 (0.0007) [2023-10-14 21:27:51,704][61585] Updated weights for policy 1, policy_version 91310 (0.0009) [2023-10-14 21:27:52,063][61585] Updated weights for policy 1, policy_version 91320 (0.0007) [2023-10-14 21:27:52,418][61552] Updated weights for policy 0, policy_version 91462 (0.0009) [2023-10-14 21:27:52,792][61552] Updated weights for policy 0, policy_version 91472 (0.0011) [2023-10-14 21:27:53,161][61552] Updated weights for policy 0, policy_version 91482 (0.0010) [2023-10-14 21:27:53,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 187170816. Throughput: 0: 1683.9, 1: 1654.3. Samples: 46802254. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-14 21:27:53,344][60425] Avg episode reward: [(0, '78.390'), (1, '77.310')] [2023-10-14 21:27:56,297][61585] Updated weights for policy 1, policy_version 91330 (0.0009) [2023-10-14 21:27:56,715][61585] Updated weights for policy 1, policy_version 91340 (0.0010) [2023-10-14 21:27:57,087][61585] Updated weights for policy 1, policy_version 91350 (0.0009) [2023-10-14 21:27:57,280][61552] Updated weights for policy 0, policy_version 91492 (0.0008) [2023-10-14 21:27:57,443][61585] Updated weights for policy 1, policy_version 91360 (0.0008) [2023-10-14 21:27:57,647][61552] Updated weights for policy 0, policy_version 91502 (0.0008) [2023-10-14 21:27:58,006][61552] Updated weights for policy 0, policy_version 91512 (0.0008) [2023-10-14 21:27:58,343][60425] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 187269120. Throughput: 0: 1671.0, 1: 1666.0. Samples: 46821712. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-14 21:27:58,344][60425] Avg episode reward: [(0, '79.430'), (1, '81.780')] [2023-10-14 21:28:01,435][61585] Updated weights for policy 1, policy_version 91370 (0.0010) [2023-10-14 21:28:01,794][61585] Updated weights for policy 1, policy_version 91380 (0.0010) [2023-10-14 21:28:02,047][61552] Updated weights for policy 0, policy_version 91522 (0.0008) [2023-10-14 21:28:02,154][61585] Updated weights for policy 1, policy_version 91390 (0.0008) [2023-10-14 21:28:02,416][61552] Updated weights for policy 0, policy_version 91532 (0.0008) [2023-10-14 21:28:02,776][61552] Updated weights for policy 0, policy_version 91542 (0.0008) [2023-10-14 21:28:03,145][61552] Updated weights for policy 0, policy_version 91552 (0.0007) [2023-10-14 21:28:03,343][60425] Fps is (10 sec: 16383.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 187334656. Throughput: 0: 1680.9, 1: 1679.2. Samples: 46832710. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-14 21:28:03,344][60425] Avg episode reward: [(0, '80.350'), (1, '77.930')] [2023-10-14 21:28:06,019][61585] Updated weights for policy 1, policy_version 91400 (0.0008) [2023-10-14 21:28:06,392][61585] Updated weights for policy 1, policy_version 91410 (0.0009) [2023-10-14 21:28:06,748][61585] Updated weights for policy 1, policy_version 91420 (0.0008) [2023-10-14 21:28:07,416][61552] Updated weights for policy 0, policy_version 91562 (0.0010) [2023-10-14 21:28:07,784][61552] Updated weights for policy 0, policy_version 91572 (0.0010) [2023-10-14 21:28:08,158][61552] Updated weights for policy 0, policy_version 91582 (0.0010) [2023-10-14 21:28:08,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 187400192. Throughput: 0: 1682.5, 1: 1663.4. Samples: 46852472. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-14 21:28:08,344][60425] Avg episode reward: [(0, '76.630'), (1, '80.170')] [2023-10-14 21:28:10,653][61585] Updated weights for policy 1, policy_version 91430 (0.0007) [2023-10-14 21:28:11,015][61585] Updated weights for policy 1, policy_version 91440 (0.0008) [2023-10-14 21:28:11,375][61585] Updated weights for policy 1, policy_version 91450 (0.0010) [2023-10-14 21:28:12,368][61552] Updated weights for policy 0, policy_version 91592 (0.0009) [2023-10-14 21:28:12,744][61552] Updated weights for policy 0, policy_version 91602 (0.0009) [2023-10-14 21:28:13,110][61552] Updated weights for policy 0, policy_version 91612 (0.0007) [2023-10-14 21:28:13,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 187465728. Throughput: 0: 1661.3, 1: 1683.9. Samples: 46872044. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-14 21:28:13,344][60425] Avg episode reward: [(0, '83.290'), (1, '81.230')] [2023-10-14 21:28:15,495][61585] Updated weights for policy 1, policy_version 91460 (0.0010) [2023-10-14 21:28:15,866][61585] Updated weights for policy 1, policy_version 91470 (0.0008) [2023-10-14 21:28:16,226][61585] Updated weights for policy 1, policy_version 91480 (0.0009) [2023-10-14 21:28:17,393][61552] Updated weights for policy 0, policy_version 91622 (0.0009) [2023-10-14 21:28:17,755][61552] Updated weights for policy 0, policy_version 91632 (0.0007) [2023-10-14 21:28:18,129][61552] Updated weights for policy 0, policy_version 91642 (0.0008) [2023-10-14 21:28:18,343][60425] Fps is (10 sec: 9830.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 187498496. Throughput: 0: 1676.5, 1: 1678.3. Samples: 46882666. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-14 21:28:18,344][60425] Avg episode reward: [(0, '79.730'), (1, '81.690')] [2023-10-14 21:28:20,387][61585] Updated weights for policy 1, policy_version 91490 (0.0008) [2023-10-14 21:28:20,751][61585] Updated weights for policy 1, policy_version 91500 (0.0009) [2023-10-14 21:28:21,124][61585] Updated weights for policy 1, policy_version 91510 (0.0007) [2023-10-14 21:28:21,487][61585] Updated weights for policy 1, policy_version 91520 (0.0008) [2023-10-14 21:28:22,167][61552] Updated weights for policy 0, policy_version 91652 (0.0009) [2023-10-14 21:28:22,538][61552] Updated weights for policy 0, policy_version 91662 (0.0008) [2023-10-14 21:28:22,900][61552] Updated weights for policy 0, policy_version 91672 (0.0011) [2023-10-14 21:28:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 187596800. Throughput: 0: 1670.5, 1: 1668.3. Samples: 46902276. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-14 21:28:23,344][60425] Avg episode reward: [(0, '74.450'), (1, '77.190')] [2023-10-14 21:28:25,723][61585] Updated weights for policy 1, policy_version 91530 (0.0008) [2023-10-14 21:28:26,087][61585] Updated weights for policy 1, policy_version 91540 (0.0009) [2023-10-14 21:28:26,457][61585] Updated weights for policy 1, policy_version 91550 (0.0008) [2023-10-14 21:28:27,075][61552] Updated weights for policy 0, policy_version 91682 (0.0007) [2023-10-14 21:28:27,438][61552] Updated weights for policy 0, policy_version 91692 (0.0009) [2023-10-14 21:28:27,809][61552] Updated weights for policy 0, policy_version 91702 (0.0009) [2023-10-14 21:28:28,174][61552] Updated weights for policy 0, policy_version 91712 (0.0007) [2023-10-14 21:28:28,343][60425] Fps is (10 sec: 16384.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 187662336. Throughput: 0: 1660.1, 1: 1682.9. Samples: 46922058. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-14 21:28:28,344][60425] Avg episode reward: [(0, '73.920'), (1, '79.590')] [2023-10-14 21:28:30,500][61585] Updated weights for policy 1, policy_version 91560 (0.0008) [2023-10-14 21:28:30,860][61585] Updated weights for policy 1, policy_version 91570 (0.0009) [2023-10-14 21:28:31,218][61585] Updated weights for policy 1, policy_version 91580 (0.0008) [2023-10-14 21:28:32,195][61552] Updated weights for policy 0, policy_version 91722 (0.0008) [2023-10-14 21:28:32,563][61552] Updated weights for policy 0, policy_version 91732 (0.0007) [2023-10-14 21:28:32,934][61552] Updated weights for policy 0, policy_version 91742 (0.0007) [2023-10-14 21:28:33,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 187727872. Throughput: 0: 1675.4, 1: 1669.4. Samples: 46932734. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-14 21:28:33,344][60425] Avg episode reward: [(0, '76.600'), (1, '81.140')] [2023-10-14 21:28:35,267][61585] Updated weights for policy 1, policy_version 91590 (0.0009) [2023-10-14 21:28:35,636][61585] Updated weights for policy 1, policy_version 91600 (0.0011) [2023-10-14 21:28:36,007][61585] Updated weights for policy 1, policy_version 91610 (0.0008) [2023-10-14 21:28:36,874][61552] Updated weights for policy 0, policy_version 91752 (0.0008) [2023-10-14 21:28:37,236][61552] Updated weights for policy 0, policy_version 91762 (0.0008) [2023-10-14 21:28:37,615][61552] Updated weights for policy 0, policy_version 91772 (0.0008) [2023-10-14 21:28:38,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 187793408. Throughput: 0: 1671.2, 1: 1670.3. Samples: 46952626. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-14 21:28:38,344][60425] Avg episode reward: [(0, '72.760'), (1, '82.730')] [2023-10-14 21:28:40,151][61585] Updated weights for policy 1, policy_version 91620 (0.0008) [2023-10-14 21:28:40,520][61585] Updated weights for policy 1, policy_version 91630 (0.0007) [2023-10-14 21:28:40,884][61585] Updated weights for policy 1, policy_version 91640 (0.0007) [2023-10-14 21:28:41,566][61552] Updated weights for policy 0, policy_version 91782 (0.0010) [2023-10-14 21:28:41,938][61552] Updated weights for policy 0, policy_version 91792 (0.0009) [2023-10-14 21:28:42,310][61552] Updated weights for policy 0, policy_version 91802 (0.0007) [2023-10-14 21:28:43,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.5). Total num frames: 187858944. Throughput: 0: 1658.0, 1: 1685.9. Samples: 46972186. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-14 21:28:43,344][60425] Avg episode reward: [(0, '68.730'), (1, '80.330')] [2023-10-14 21:28:45,035][61585] Updated weights for policy 1, policy_version 91650 (0.0007) [2023-10-14 21:28:45,456][61585] Updated weights for policy 1, policy_version 91660 (0.0007) [2023-10-14 21:28:45,823][61585] Updated weights for policy 1, policy_version 91670 (0.0007) [2023-10-14 21:28:46,187][61585] Updated weights for policy 1, policy_version 91680 (0.0007) [2023-10-14 21:28:46,394][61552] Updated weights for policy 0, policy_version 91812 (0.0009) [2023-10-14 21:28:46,751][61552] Updated weights for policy 0, policy_version 91822 (0.0008) [2023-10-14 21:28:47,124][61552] Updated weights for policy 0, policy_version 91832 (0.0007) [2023-10-14 21:28:48,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 187924480. Throughput: 0: 1677.6, 1: 1661.8. Samples: 46982984. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-14 21:28:48,344][60425] Avg episode reward: [(0, '71.140'), (1, '80.090')] [2023-10-14 21:28:50,299][61585] Updated weights for policy 1, policy_version 91690 (0.0009) [2023-10-14 21:28:50,660][61585] Updated weights for policy 1, policy_version 91700 (0.0008) [2023-10-14 21:28:51,023][61585] Updated weights for policy 1, policy_version 91710 (0.0008) [2023-10-14 21:28:51,054][61552] Updated weights for policy 0, policy_version 91842 (0.0009) [2023-10-14 21:28:51,477][61552] Updated weights for policy 0, policy_version 91852 (0.0009) [2023-10-14 21:28:51,846][61552] Updated weights for policy 0, policy_version 91862 (0.0008) [2023-10-14 21:28:52,223][61552] Updated weights for policy 0, policy_version 91872 (0.0009) [2023-10-14 21:28:53,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 187990016. Throughput: 0: 1665.4, 1: 1666.3. Samples: 47002398. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-14 21:28:53,344][60425] Avg episode reward: [(0, '70.460'), (1, '78.270')] [2023-10-14 21:28:54,974][61585] Updated weights for policy 1, policy_version 91720 (0.0008) [2023-10-14 21:28:55,346][61585] Updated weights for policy 1, policy_version 91730 (0.0008) [2023-10-14 21:28:55,710][61585] Updated weights for policy 1, policy_version 91740 (0.0008) [2023-10-14 21:28:56,092][61552] Updated weights for policy 0, policy_version 91882 (0.0009) [2023-10-14 21:28:56,461][61552] Updated weights for policy 0, policy_version 91892 (0.0008) [2023-10-14 21:28:56,827][61552] Updated weights for policy 0, policy_version 91902 (0.0011) [2023-10-14 21:28:58,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 188055552. Throughput: 0: 1671.5, 1: 1677.0. Samples: 47022726. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-14 21:28:58,344][60425] Avg episode reward: [(0, '70.700'), (1, '76.670')] [2023-10-14 21:28:59,707][61585] Updated weights for policy 1, policy_version 91750 (0.0008) [2023-10-14 21:29:00,075][61585] Updated weights for policy 1, policy_version 91760 (0.0009) [2023-10-14 21:29:00,446][61585] Updated weights for policy 1, policy_version 91770 (0.0011) [2023-10-14 21:29:00,952][61552] Updated weights for policy 0, policy_version 91912 (0.0008) [2023-10-14 21:29:01,318][61552] Updated weights for policy 0, policy_version 91922 (0.0009) [2023-10-14 21:29:01,681][61552] Updated weights for policy 0, policy_version 91932 (0.0011) [2023-10-14 21:29:03,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 188121088. Throughput: 0: 1681.5, 1: 1656.4. Samples: 47032870. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-14 21:29:03,344][60425] Avg episode reward: [(0, '68.610'), (1, '86.320')] [2023-10-14 21:29:04,468][61585] Updated weights for policy 1, policy_version 91780 (0.0009) [2023-10-14 21:29:04,826][61585] Updated weights for policy 1, policy_version 91790 (0.0009) [2023-10-14 21:29:05,191][61585] Updated weights for policy 1, policy_version 91800 (0.0007) [2023-10-14 21:29:05,836][61552] Updated weights for policy 0, policy_version 91942 (0.0008) [2023-10-14 21:29:06,212][61552] Updated weights for policy 0, policy_version 91952 (0.0008) [2023-10-14 21:29:06,575][61552] Updated weights for policy 0, policy_version 91962 (0.0009) [2023-10-14 21:29:08,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 188186624. Throughput: 0: 1661.7, 1: 1676.9. Samples: 47052512. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-14 21:29:08,344][60425] Avg episode reward: [(0, '69.550'), (1, '77.320')] [2023-10-14 21:29:09,290][61585] Updated weights for policy 1, policy_version 91810 (0.0007) [2023-10-14 21:29:09,661][61585] Updated weights for policy 1, policy_version 91820 (0.0007) [2023-10-14 21:29:10,017][61585] Updated weights for policy 1, policy_version 91830 (0.0007) [2023-10-14 21:29:10,384][61585] Updated weights for policy 1, policy_version 91840 (0.0007) [2023-10-14 21:29:10,794][61552] Updated weights for policy 0, policy_version 91972 (0.0008) [2023-10-14 21:29:11,162][61552] Updated weights for policy 0, policy_version 91982 (0.0008) [2023-10-14 21:29:11,526][61552] Updated weights for policy 0, policy_version 91992 (0.0008) [2023-10-14 21:29:13,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 188252160. Throughput: 0: 1668.3, 1: 1680.4. Samples: 47072750. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 21:29:13,345][60425] Avg episode reward: [(0, '70.610'), (1, '83.160')] [2023-10-14 21:29:14,504][61585] Updated weights for policy 1, policy_version 91850 (0.0009) [2023-10-14 21:29:14,872][61585] Updated weights for policy 1, policy_version 91860 (0.0007) [2023-10-14 21:29:15,234][61585] Updated weights for policy 1, policy_version 91870 (0.0008) [2023-10-14 21:29:15,676][61552] Updated weights for policy 0, policy_version 92002 (0.0008) [2023-10-14 21:29:16,041][61552] Updated weights for policy 0, policy_version 92012 (0.0008) [2023-10-14 21:29:16,415][61552] Updated weights for policy 0, policy_version 92022 (0.0008) [2023-10-14 21:29:16,770][61552] Updated weights for policy 0, policy_version 92032 (0.0010) [2023-10-14 21:29:18,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 188317696. Throughput: 0: 1677.3, 1: 1664.8. Samples: 47083128. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 21:29:18,344][60425] Avg episode reward: [(0, '72.230'), (1, '75.670')] [2023-10-14 21:29:19,305][61585] Updated weights for policy 1, policy_version 91880 (0.0007) [2023-10-14 21:29:19,682][61585] Updated weights for policy 1, policy_version 91890 (0.0008) [2023-10-14 21:29:20,052][61585] Updated weights for policy 1, policy_version 91900 (0.0008) [2023-10-14 21:29:20,766][61552] Updated weights for policy 0, policy_version 92042 (0.0009) [2023-10-14 21:29:21,134][61552] Updated weights for policy 0, policy_version 92052 (0.0009) [2023-10-14 21:29:21,500][61552] Updated weights for policy 0, policy_version 92062 (0.0011) [2023-10-14 21:29:23,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 188383232. Throughput: 0: 1651.4, 1: 1680.1. Samples: 47102544. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 21:29:23,344][60425] Avg episode reward: [(0, '75.580'), (1, '80.760')] [2023-10-14 21:29:24,116][61585] Updated weights for policy 1, policy_version 91910 (0.0010) [2023-10-14 21:29:24,484][61585] Updated weights for policy 1, policy_version 91920 (0.0010) [2023-10-14 21:29:24,839][61585] Updated weights for policy 1, policy_version 91930 (0.0009) [2023-10-14 21:29:25,737][61552] Updated weights for policy 0, policy_version 92072 (0.0008) [2023-10-14 21:29:26,100][61552] Updated weights for policy 0, policy_version 92082 (0.0009) [2023-10-14 21:29:26,474][61552] Updated weights for policy 0, policy_version 92092 (0.0012) [2023-10-14 21:29:28,343][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.1, 300 sec: 13329.4). Total num frames: 188448768. Throughput: 0: 1677.6, 1: 1678.5. Samples: 47123210. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 21:29:28,344][60425] Avg episode reward: [(0, '77.190'), (1, '79.500')] [2023-10-14 21:29:29,040][61585] Updated weights for policy 1, policy_version 91940 (0.0008) [2023-10-14 21:29:29,406][61585] Updated weights for policy 1, policy_version 91950 (0.0007) [2023-10-14 21:29:29,773][61585] Updated weights for policy 1, policy_version 91960 (0.0008) [2023-10-14 21:29:30,550][61552] Updated weights for policy 0, policy_version 92102 (0.0008) [2023-10-14 21:29:30,920][61552] Updated weights for policy 0, policy_version 92112 (0.0007) [2023-10-14 21:29:31,291][61552] Updated weights for policy 0, policy_version 92122 (0.0008) [2023-10-14 21:29:33,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 188514304. Throughput: 0: 1664.4, 1: 1672.6. Samples: 47133148. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 21:29:33,344][60425] Avg episode reward: [(0, '78.540'), (1, '80.770')] [2023-10-14 21:29:34,079][61585] Updated weights for policy 1, policy_version 91970 (0.0010) [2023-10-14 21:29:34,488][61585] Updated weights for policy 1, policy_version 91980 (0.0010) [2023-10-14 21:29:34,851][61585] Updated weights for policy 1, policy_version 91990 (0.0009) [2023-10-14 21:29:35,216][61585] Updated weights for policy 1, policy_version 92000 (0.0009) [2023-10-14 21:29:35,414][61552] Updated weights for policy 0, policy_version 92132 (0.0009) [2023-10-14 21:29:35,791][61552] Updated weights for policy 0, policy_version 92142 (0.0009) [2023-10-14 21:29:36,158][61552] Updated weights for policy 0, policy_version 92152 (0.0008) [2023-10-14 21:29:38,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 188579840. Throughput: 0: 1658.8, 1: 1679.7. Samples: 47152634. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 21:29:38,344][60425] Avg episode reward: [(0, '74.870'), (1, '81.050')] [2023-10-14 21:29:39,195][61585] Updated weights for policy 1, policy_version 92010 (0.0008) [2023-10-14 21:29:39,553][61585] Updated weights for policy 1, policy_version 92020 (0.0010) [2023-10-14 21:29:39,913][61585] Updated weights for policy 1, policy_version 92030 (0.0007) [2023-10-14 21:29:40,458][61552] Updated weights for policy 0, policy_version 92162 (0.0009) [2023-10-14 21:29:40,863][61552] Updated weights for policy 0, policy_version 92172 (0.0007) [2023-10-14 21:29:41,235][61552] Updated weights for policy 0, policy_version 92182 (0.0008) [2023-10-14 21:29:41,601][61552] Updated weights for policy 0, policy_version 92192 (0.0007) [2023-10-14 21:29:43,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 188645376. Throughput: 0: 1668.2, 1: 1677.2. Samples: 47173272. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 21:29:43,344][60425] Avg episode reward: [(0, '77.460'), (1, '77.840')] [2023-10-14 21:29:43,352][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000092192_94404608.pth... [2023-10-14 21:29:43,352][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000092032_94240768.pth... [2023-10-14 21:29:43,381][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000090624_92798976.pth [2023-10-14 21:29:43,392][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000090464_92635136.pth [2023-10-14 21:29:43,997][61585] Updated weights for policy 1, policy_version 92040 (0.0008) [2023-10-14 21:29:44,357][61585] Updated weights for policy 1, policy_version 92050 (0.0007) [2023-10-14 21:29:44,733][61585] Updated weights for policy 1, policy_version 92060 (0.0009) [2023-10-14 21:29:45,741][61552] Updated weights for policy 0, policy_version 92202 (0.0009) [2023-10-14 21:29:46,119][61552] Updated weights for policy 0, policy_version 92212 (0.0008) [2023-10-14 21:29:46,484][61552] Updated weights for policy 0, policy_version 92222 (0.0008) [2023-10-14 21:29:48,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 188710912. Throughput: 0: 1663.9, 1: 1675.4. Samples: 47183138. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 21:29:48,344][60425] Avg episode reward: [(0, '77.670'), (1, '78.500')] [2023-10-14 21:29:48,895][61585] Updated weights for policy 1, policy_version 92070 (0.0008) [2023-10-14 21:29:49,264][61585] Updated weights for policy 1, policy_version 92080 (0.0008) [2023-10-14 21:29:49,622][61585] Updated weights for policy 1, policy_version 92090 (0.0007) [2023-10-14 21:29:50,726][61552] Updated weights for policy 0, policy_version 92232 (0.0010) [2023-10-14 21:29:51,088][61552] Updated weights for policy 0, policy_version 92242 (0.0007) [2023-10-14 21:29:51,459][61552] Updated weights for policy 0, policy_version 92252 (0.0008) [2023-10-14 21:29:53,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 188776448. Throughput: 0: 1661.7, 1: 1673.1. Samples: 47202580. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 21:29:53,344][60425] Avg episode reward: [(0, '77.510'), (1, '80.200')] [2023-10-14 21:29:53,563][61585] Updated weights for policy 1, policy_version 92100 (0.0010) [2023-10-14 21:29:53,930][61585] Updated weights for policy 1, policy_version 92110 (0.0009) [2023-10-14 21:29:54,291][61585] Updated weights for policy 1, policy_version 92120 (0.0009) [2023-10-14 21:29:55,481][61552] Updated weights for policy 0, policy_version 92262 (0.0010) [2023-10-14 21:29:55,854][61552] Updated weights for policy 0, policy_version 92272 (0.0008) [2023-10-14 21:29:56,218][61552] Updated weights for policy 0, policy_version 92282 (0.0009) [2023-10-14 21:29:58,332][61585] Updated weights for policy 1, policy_version 92130 (0.0008) [2023-10-14 21:29:58,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 188841984. Throughput: 0: 1674.3, 1: 1673.9. Samples: 47223420. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 21:29:58,345][60425] Avg episode reward: [(0, '77.330'), (1, '77.290')] [2023-10-14 21:29:58,705][61585] Updated weights for policy 1, policy_version 92140 (0.0007) [2023-10-14 21:29:59,078][61585] Updated weights for policy 1, policy_version 92150 (0.0007) [2023-10-14 21:29:59,435][61585] Updated weights for policy 1, policy_version 92160 (0.0009) [2023-10-14 21:30:00,298][61552] Updated weights for policy 0, policy_version 92292 (0.0007) [2023-10-14 21:30:00,657][61552] Updated weights for policy 0, policy_version 92302 (0.0007) [2023-10-14 21:30:01,025][61552] Updated weights for policy 0, policy_version 92312 (0.0009) [2023-10-14 21:30:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 188907520. Throughput: 0: 1660.4, 1: 1673.7. Samples: 47233164. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 21:30:03,344][60425] Avg episode reward: [(0, '78.990'), (1, '80.500')] [2023-10-14 21:30:03,391][61585] Updated weights for policy 1, policy_version 92170 (0.0008) [2023-10-14 21:30:03,753][61585] Updated weights for policy 1, policy_version 92180 (0.0008) [2023-10-14 21:30:04,122][61585] Updated weights for policy 1, policy_version 92190 (0.0010) [2023-10-14 21:30:04,984][61552] Updated weights for policy 0, policy_version 92322 (0.0010) [2023-10-14 21:30:05,348][61552] Updated weights for policy 0, policy_version 92332 (0.0010) [2023-10-14 21:30:05,720][61552] Updated weights for policy 0, policy_version 92342 (0.0008) [2023-10-14 21:30:06,080][61552] Updated weights for policy 0, policy_version 92352 (0.0008) [2023-10-14 21:30:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 188973056. Throughput: 0: 1673.9, 1: 1678.3. Samples: 47253396. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-14 21:30:08,344][60425] Avg episode reward: [(0, '76.270'), (1, '77.560')] [2023-10-14 21:30:08,382][61585] Updated weights for policy 1, policy_version 92200 (0.0009) [2023-10-14 21:30:08,740][61585] Updated weights for policy 1, policy_version 92210 (0.0009) [2023-10-14 21:30:09,107][61585] Updated weights for policy 1, policy_version 92220 (0.0010) [2023-10-14 21:30:10,076][61552] Updated weights for policy 0, policy_version 92362 (0.0008) [2023-10-14 21:30:10,444][61552] Updated weights for policy 0, policy_version 92372 (0.0007) [2023-10-14 21:30:10,818][61552] Updated weights for policy 0, policy_version 92382 (0.0007) [2023-10-14 21:30:13,133][61585] Updated weights for policy 1, policy_version 92230 (0.0010) [2023-10-14 21:30:13,344][60425] Fps is (10 sec: 13106.6, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 189038592. Throughput: 0: 1671.9, 1: 1674.4. Samples: 47273794. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:30:13,345][60425] Avg episode reward: [(0, '79.170'), (1, '75.600')] [2023-10-14 21:30:13,493][61585] Updated weights for policy 1, policy_version 92240 (0.0010) [2023-10-14 21:30:13,868][61585] Updated weights for policy 1, policy_version 92250 (0.0008) [2023-10-14 21:30:14,950][61552] Updated weights for policy 0, policy_version 92392 (0.0009) [2023-10-14 21:30:15,334][61552] Updated weights for policy 0, policy_version 92402 (0.0007) [2023-10-14 21:30:15,706][61552] Updated weights for policy 0, policy_version 92412 (0.0008) [2023-10-14 21:30:18,040][61585] Updated weights for policy 1, policy_version 92260 (0.0009) [2023-10-14 21:30:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 189104128. Throughput: 0: 1658.2, 1: 1672.5. Samples: 47283028. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:30:18,344][60425] Avg episode reward: [(0, '79.720'), (1, '81.270')] [2023-10-14 21:30:18,409][61585] Updated weights for policy 1, policy_version 92270 (0.0008) [2023-10-14 21:30:18,786][61585] Updated weights for policy 1, policy_version 92280 (0.0009) [2023-10-14 21:30:19,677][61552] Updated weights for policy 0, policy_version 92422 (0.0007) [2023-10-14 21:30:20,048][61552] Updated weights for policy 0, policy_version 92432 (0.0008) [2023-10-14 21:30:20,420][61552] Updated weights for policy 0, policy_version 92442 (0.0007) [2023-10-14 21:30:22,970][61585] Updated weights for policy 1, policy_version 92290 (0.0010) [2023-10-14 21:30:23,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 189169664. Throughput: 0: 1675.0, 1: 1674.3. Samples: 47303350. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:30:23,344][60425] Avg episode reward: [(0, '82.720'), (1, '83.080')] [2023-10-14 21:30:23,388][61585] Updated weights for policy 1, policy_version 92300 (0.0008) [2023-10-14 21:30:23,749][61585] Updated weights for policy 1, policy_version 92310 (0.0008) [2023-10-14 21:30:24,109][61585] Updated weights for policy 1, policy_version 92320 (0.0008) [2023-10-14 21:30:24,341][61552] Updated weights for policy 0, policy_version 92452 (0.0007) [2023-10-14 21:30:24,704][61552] Updated weights for policy 0, policy_version 92462 (0.0010) [2023-10-14 21:30:25,067][61552] Updated weights for policy 0, policy_version 92472 (0.0010) [2023-10-14 21:30:28,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 189235200. Throughput: 0: 1674.7, 1: 1668.1. Samples: 47323702. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:30:28,344][60425] Avg episode reward: [(0, '77.290'), (1, '82.650')] [2023-10-14 21:30:28,379][61585] Updated weights for policy 1, policy_version 92330 (0.0009) [2023-10-14 21:30:28,749][61585] Updated weights for policy 1, policy_version 92340 (0.0007) [2023-10-14 21:30:29,111][61585] Updated weights for policy 1, policy_version 92350 (0.0007) [2023-10-14 21:30:29,389][61552] Updated weights for policy 0, policy_version 92482 (0.0009) [2023-10-14 21:30:29,791][61552] Updated weights for policy 0, policy_version 92492 (0.0008) [2023-10-14 21:30:30,151][61552] Updated weights for policy 0, policy_version 92502 (0.0008) [2023-10-14 21:30:30,515][61552] Updated weights for policy 0, policy_version 92512 (0.0009) [2023-10-14 21:30:33,019][61585] Updated weights for policy 1, policy_version 92360 (0.0008) [2023-10-14 21:30:33,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 189300736. Throughput: 0: 1654.0, 1: 1672.6. Samples: 47332834. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:30:33,344][60425] Avg episode reward: [(0, '75.990'), (1, '79.500')] [2023-10-14 21:30:33,391][61585] Updated weights for policy 1, policy_version 92370 (0.0007) [2023-10-14 21:30:33,757][61585] Updated weights for policy 1, policy_version 92380 (0.0008) [2023-10-14 21:30:34,636][61552] Updated weights for policy 0, policy_version 92522 (0.0010) [2023-10-14 21:30:35,007][61552] Updated weights for policy 0, policy_version 92532 (0.0007) [2023-10-14 21:30:35,370][61552] Updated weights for policy 0, policy_version 92542 (0.0007) [2023-10-14 21:30:37,898][61585] Updated weights for policy 1, policy_version 92390 (0.0008) [2023-10-14 21:30:38,266][61585] Updated weights for policy 1, policy_version 92400 (0.0008) [2023-10-14 21:30:38,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 189366272. Throughput: 0: 1675.7, 1: 1676.0. Samples: 47353406. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:30:38,344][60425] Avg episode reward: [(0, '76.660'), (1, '80.070')] [2023-10-14 21:30:38,631][61585] Updated weights for policy 1, policy_version 92410 (0.0007) [2023-10-14 21:30:39,501][61552] Updated weights for policy 0, policy_version 92552 (0.0007) [2023-10-14 21:30:39,870][61552] Updated weights for policy 0, policy_version 92562 (0.0008) [2023-10-14 21:30:40,234][61552] Updated weights for policy 0, policy_version 92572 (0.0009) [2023-10-14 21:30:42,669][61585] Updated weights for policy 1, policy_version 92420 (0.0008) [2023-10-14 21:30:43,033][61585] Updated weights for policy 1, policy_version 92430 (0.0007) [2023-10-14 21:30:43,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 189431808. Throughput: 0: 1668.1, 1: 1669.9. Samples: 47373630. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:30:43,344][60425] Avg episode reward: [(0, '75.070'), (1, '82.340')] [2023-10-14 21:30:43,400][61585] Updated weights for policy 1, policy_version 92440 (0.0009) [2023-10-14 21:30:44,352][61552] Updated weights for policy 0, policy_version 92582 (0.0007) [2023-10-14 21:30:44,712][61552] Updated weights for policy 0, policy_version 92592 (0.0008) [2023-10-14 21:30:45,080][61552] Updated weights for policy 0, policy_version 92602 (0.0008) [2023-10-14 21:30:47,431][61585] Updated weights for policy 1, policy_version 92450 (0.0008) [2023-10-14 21:30:47,796][61585] Updated weights for policy 1, policy_version 92460 (0.0009) [2023-10-14 21:30:48,171][61585] Updated weights for policy 1, policy_version 92470 (0.0010) [2023-10-14 21:30:48,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 189497344. Throughput: 0: 1654.9, 1: 1677.2. Samples: 47383106. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:30:48,344][60425] Avg episode reward: [(0, '77.620'), (1, '83.490')] [2023-10-14 21:30:48,533][61585] Updated weights for policy 1, policy_version 92480 (0.0009) [2023-10-14 21:30:49,183][61552] Updated weights for policy 0, policy_version 92612 (0.0008) [2023-10-14 21:30:49,559][61552] Updated weights for policy 0, policy_version 92622 (0.0009) [2023-10-14 21:30:49,923][61552] Updated weights for policy 0, policy_version 92632 (0.0010) [2023-10-14 21:30:52,625][61585] Updated weights for policy 1, policy_version 92490 (0.0009) [2023-10-14 21:30:52,991][61585] Updated weights for policy 1, policy_version 92500 (0.0010) [2023-10-14 21:30:53,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 189562880. Throughput: 0: 1664.5, 1: 1677.2. Samples: 47403772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:30:53,344][60425] Avg episode reward: [(0, '71.870'), (1, '81.040')] [2023-10-14 21:30:53,356][61585] Updated weights for policy 1, policy_version 92510 (0.0007) [2023-10-14 21:30:54,104][61552] Updated weights for policy 0, policy_version 92642 (0.0009) [2023-10-14 21:30:54,473][61552] Updated weights for policy 0, policy_version 92652 (0.0009) [2023-10-14 21:30:54,848][61552] Updated weights for policy 0, policy_version 92662 (0.0009) [2023-10-14 21:30:55,207][61552] Updated weights for policy 0, policy_version 92672 (0.0009) [2023-10-14 21:30:57,541][61585] Updated weights for policy 1, policy_version 92520 (0.0009) [2023-10-14 21:30:57,899][61585] Updated weights for policy 1, policy_version 92530 (0.0009) [2023-10-14 21:30:58,272][61585] Updated weights for policy 1, policy_version 92540 (0.0008) [2023-10-14 21:30:58,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 189628416. Throughput: 0: 1665.7, 1: 1664.6. Samples: 47423658. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:30:58,345][60425] Avg episode reward: [(0, '75.160'), (1, '80.910')] [2023-10-14 21:30:59,285][61552] Updated weights for policy 0, policy_version 92682 (0.0009) [2023-10-14 21:30:59,657][61552] Updated weights for policy 0, policy_version 92692 (0.0011) [2023-10-14 21:31:00,036][61552] Updated weights for policy 0, policy_version 92702 (0.0007) [2023-10-14 21:31:02,370][61585] Updated weights for policy 1, policy_version 92550 (0.0009) [2023-10-14 21:31:02,742][61585] Updated weights for policy 1, policy_version 92560 (0.0007) [2023-10-14 21:31:03,104][61585] Updated weights for policy 1, policy_version 92570 (0.0008) [2023-10-14 21:31:03,343][60425] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 189726720. Throughput: 0: 1659.7, 1: 1681.7. Samples: 47433390. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:31:03,344][60425] Avg episode reward: [(0, '78.240'), (1, '75.530')] [2023-10-14 21:31:04,124][61552] Updated weights for policy 0, policy_version 92712 (0.0007) [2023-10-14 21:31:04,479][61552] Updated weights for policy 0, policy_version 92722 (0.0009) [2023-10-14 21:31:04,846][61552] Updated weights for policy 0, policy_version 92732 (0.0010) [2023-10-14 21:31:07,284][61585] Updated weights for policy 1, policy_version 92580 (0.0011) [2023-10-14 21:31:07,643][61585] Updated weights for policy 1, policy_version 92590 (0.0008) [2023-10-14 21:31:08,009][61585] Updated weights for policy 1, policy_version 92600 (0.0008) [2023-10-14 21:31:08,343][60425] Fps is (10 sec: 16384.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 189792256. Throughput: 0: 1665.6, 1: 1686.5. Samples: 47454194. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:31:08,344][60425] Avg episode reward: [(0, '80.300'), (1, '81.540')] [2023-10-14 21:31:08,830][61552] Updated weights for policy 0, policy_version 92742 (0.0008) [2023-10-14 21:31:09,199][61552] Updated weights for policy 0, policy_version 92752 (0.0008) [2023-10-14 21:31:09,560][61552] Updated weights for policy 0, policy_version 92762 (0.0010) [2023-10-14 21:31:12,040][61585] Updated weights for policy 1, policy_version 92610 (0.0011) [2023-10-14 21:31:12,447][61585] Updated weights for policy 1, policy_version 92620 (0.0007) [2023-10-14 21:31:12,811][61585] Updated weights for policy 1, policy_version 92630 (0.0010) [2023-10-14 21:31:13,182][61585] Updated weights for policy 1, policy_version 92640 (0.0011) [2023-10-14 21:31:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 189857792. Throughput: 0: 1667.4, 1: 1670.2. Samples: 47473896. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:31:13,344][60425] Avg episode reward: [(0, '80.130'), (1, '83.560')] [2023-10-14 21:31:13,816][61552] Updated weights for policy 0, policy_version 92772 (0.0008) [2023-10-14 21:31:14,173][61552] Updated weights for policy 0, policy_version 92782 (0.0008) [2023-10-14 21:31:14,542][61552] Updated weights for policy 0, policy_version 92792 (0.0008) [2023-10-14 21:31:17,467][61585] Updated weights for policy 1, policy_version 92650 (0.0008) [2023-10-14 21:31:17,838][61585] Updated weights for policy 1, policy_version 92660 (0.0007) [2023-10-14 21:31:18,216][61585] Updated weights for policy 1, policy_version 92670 (0.0008) [2023-10-14 21:31:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 189923328. Throughput: 0: 1668.2, 1: 1682.3. Samples: 47483608. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:31:18,344][60425] Avg episode reward: [(0, '78.850'), (1, '82.440')] [2023-10-14 21:31:18,723][61552] Updated weights for policy 0, policy_version 92802 (0.0008) [2023-10-14 21:31:19,096][61552] Updated weights for policy 0, policy_version 92812 (0.0008) [2023-10-14 21:31:19,467][61552] Updated weights for policy 0, policy_version 92822 (0.0008) [2023-10-14 21:31:19,830][61552] Updated weights for policy 0, policy_version 92832 (0.0010) [2023-10-14 21:31:22,106][61585] Updated weights for policy 1, policy_version 92680 (0.0008) [2023-10-14 21:31:22,470][61585] Updated weights for policy 1, policy_version 92690 (0.0008) [2023-10-14 21:31:22,836][61585] Updated weights for policy 1, policy_version 92700 (0.0011) [2023-10-14 21:31:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 189988864. Throughput: 0: 1672.8, 1: 1680.7. Samples: 47504314. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:31:23,344][60425] Avg episode reward: [(0, '79.770'), (1, '78.610')] [2023-10-14 21:31:23,784][61552] Updated weights for policy 0, policy_version 92842 (0.0007) [2023-10-14 21:31:24,156][61552] Updated weights for policy 0, policy_version 92852 (0.0007) [2023-10-14 21:31:24,522][61552] Updated weights for policy 0, policy_version 92862 (0.0008) [2023-10-14 21:31:26,807][61585] Updated weights for policy 1, policy_version 92710 (0.0008) [2023-10-14 21:31:27,180][61585] Updated weights for policy 1, policy_version 92720 (0.0009) [2023-10-14 21:31:27,546][61585] Updated weights for policy 1, policy_version 92730 (0.0010) [2023-10-14 21:31:28,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 190054400. Throughput: 0: 1683.8, 1: 1661.1. Samples: 47524148. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:31:28,344][60425] Avg episode reward: [(0, '75.160'), (1, '82.360')] [2023-10-14 21:31:28,558][61552] Updated weights for policy 0, policy_version 92872 (0.0011) [2023-10-14 21:31:28,923][61552] Updated weights for policy 0, policy_version 92882 (0.0008) [2023-10-14 21:31:29,285][61552] Updated weights for policy 0, policy_version 92892 (0.0009) [2023-10-14 21:31:31,562][61585] Updated weights for policy 1, policy_version 92740 (0.0009) [2023-10-14 21:31:31,926][61585] Updated weights for policy 1, policy_version 92750 (0.0009) [2023-10-14 21:31:32,291][61585] Updated weights for policy 1, policy_version 92760 (0.0007) [2023-10-14 21:31:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 190119936. Throughput: 0: 1680.6, 1: 1681.1. Samples: 47534382. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:31:33,344][60425] Avg episode reward: [(0, '78.040'), (1, '84.300')] [2023-10-14 21:31:33,394][61552] Updated weights for policy 0, policy_version 92902 (0.0009) [2023-10-14 21:31:33,757][61552] Updated weights for policy 0, policy_version 92912 (0.0011) [2023-10-14 21:31:34,127][61552] Updated weights for policy 0, policy_version 92922 (0.0010) [2023-10-14 21:31:36,351][61585] Updated weights for policy 1, policy_version 92770 (0.0007) [2023-10-14 21:31:36,712][61585] Updated weights for policy 1, policy_version 92780 (0.0008) [2023-10-14 21:31:37,077][61585] Updated weights for policy 1, policy_version 92790 (0.0010) [2023-10-14 21:31:37,431][61585] Updated weights for policy 1, policy_version 92800 (0.0009) [2023-10-14 21:31:38,166][61552] Updated weights for policy 0, policy_version 92932 (0.0010) [2023-10-14 21:31:38,344][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 190185472. Throughput: 0: 1686.7, 1: 1669.5. Samples: 47554802. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:31:38,345][60425] Avg episode reward: [(0, '76.650'), (1, '86.660')] [2023-10-14 21:31:38,346][61248] Saving new best policy, reward=86.660! [2023-10-14 21:31:38,533][61552] Updated weights for policy 0, policy_version 92942 (0.0011) [2023-10-14 21:31:38,898][61552] Updated weights for policy 0, policy_version 92952 (0.0011) [2023-10-14 21:31:41,639][61585] Updated weights for policy 1, policy_version 92810 (0.0008) [2023-10-14 21:31:42,021][61585] Updated weights for policy 1, policy_version 92820 (0.0008) [2023-10-14 21:31:42,374][61585] Updated weights for policy 1, policy_version 92830 (0.0007) [2023-10-14 21:31:42,898][61552] Updated weights for policy 0, policy_version 92962 (0.0009) [2023-10-14 21:31:43,265][61552] Updated weights for policy 0, policy_version 92972 (0.0011) [2023-10-14 21:31:43,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 190251008. Throughput: 0: 1689.6, 1: 1669.9. Samples: 47574832. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:31:43,344][60425] Avg episode reward: [(0, '77.370'), (1, '82.300')] [2023-10-14 21:31:43,353][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000092832_95059968.pth... [2023-10-14 21:31:43,389][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000091264_93454336.pth [2023-10-14 21:31:43,631][61552] Updated weights for policy 0, policy_version 92982 (0.0009) [2023-10-14 21:31:43,998][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000092992_95223808.pth... [2023-10-14 21:31:44,001][61552] Updated weights for policy 0, policy_version 92992 (0.0008) [2023-10-14 21:31:44,038][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000091424_93618176.pth [2023-10-14 21:31:46,352][61585] Updated weights for policy 1, policy_version 92840 (0.0008) [2023-10-14 21:31:46,712][61585] Updated weights for policy 1, policy_version 92850 (0.0009) [2023-10-14 21:31:47,082][61585] Updated weights for policy 1, policy_version 92860 (0.0007) [2023-10-14 21:31:47,962][61552] Updated weights for policy 0, policy_version 93002 (0.0009) [2023-10-14 21:31:48,329][61552] Updated weights for policy 0, policy_version 93012 (0.0009) [2023-10-14 21:31:48,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 190316544. Throughput: 0: 1694.8, 1: 1684.1. Samples: 47585444. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:31:48,344][60425] Avg episode reward: [(0, '76.290'), (1, '79.880')] [2023-10-14 21:31:48,699][61552] Updated weights for policy 0, policy_version 93022 (0.0009) [2023-10-14 21:31:51,205][61585] Updated weights for policy 1, policy_version 92870 (0.0008) [2023-10-14 21:31:51,567][61585] Updated weights for policy 1, policy_version 92880 (0.0011) [2023-10-14 21:31:51,936][61585] Updated weights for policy 1, policy_version 92890 (0.0008) [2023-10-14 21:31:52,941][61552] Updated weights for policy 0, policy_version 93032 (0.0009) [2023-10-14 21:31:53,315][61552] Updated weights for policy 0, policy_version 93042 (0.0008) [2023-10-14 21:31:53,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 190382080. Throughput: 0: 1688.6, 1: 1667.6. Samples: 47605222. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:31:53,344][60425] Avg episode reward: [(0, '75.790'), (1, '76.620')] [2023-10-14 21:31:53,677][61552] Updated weights for policy 0, policy_version 93052 (0.0009) [2023-10-14 21:31:55,945][61585] Updated weights for policy 1, policy_version 92900 (0.0007) [2023-10-14 21:31:56,317][61585] Updated weights for policy 1, policy_version 92910 (0.0008) [2023-10-14 21:31:56,682][61585] Updated weights for policy 1, policy_version 92920 (0.0008) [2023-10-14 21:31:57,703][61552] Updated weights for policy 0, policy_version 93062 (0.0008) [2023-10-14 21:31:58,068][61552] Updated weights for policy 0, policy_version 93072 (0.0007) [2023-10-14 21:31:58,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 190447616. Throughput: 0: 1684.1, 1: 1678.5. Samples: 47625214. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:31:58,344][60425] Avg episode reward: [(0, '79.130'), (1, '79.600')] [2023-10-14 21:31:58,442][61552] Updated weights for policy 0, policy_version 93082 (0.0008) [2023-10-14 21:32:00,654][61585] Updated weights for policy 1, policy_version 92930 (0.0009) [2023-10-14 21:32:01,052][61585] Updated weights for policy 1, policy_version 92940 (0.0009) [2023-10-14 21:32:01,411][61585] Updated weights for policy 1, policy_version 92950 (0.0009) [2023-10-14 21:32:01,767][61585] Updated weights for policy 1, policy_version 92960 (0.0008) [2023-10-14 21:32:02,491][61552] Updated weights for policy 0, policy_version 93092 (0.0007) [2023-10-14 21:32:02,851][61552] Updated weights for policy 0, policy_version 93102 (0.0007) [2023-10-14 21:32:03,226][61552] Updated weights for policy 0, policy_version 93112 (0.0007) [2023-10-14 21:32:03,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 190513152. Throughput: 0: 1691.4, 1: 1688.4. Samples: 47635696. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:32:03,344][60425] Avg episode reward: [(0, '79.290'), (1, '79.470')] [2023-10-14 21:32:05,752][61585] Updated weights for policy 1, policy_version 92970 (0.0007) [2023-10-14 21:32:06,124][61585] Updated weights for policy 1, policy_version 92980 (0.0008) [2023-10-14 21:32:06,477][61585] Updated weights for policy 1, policy_version 92990 (0.0011) [2023-10-14 21:32:07,375][61552] Updated weights for policy 0, policy_version 93122 (0.0007) [2023-10-14 21:32:07,774][61552] Updated weights for policy 0, policy_version 93132 (0.0008) [2023-10-14 21:32:08,140][61552] Updated weights for policy 0, policy_version 93142 (0.0008) [2023-10-14 21:32:08,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 190578688. Throughput: 0: 1692.0, 1: 1664.3. Samples: 47655344. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:32:08,344][60425] Avg episode reward: [(0, '78.160'), (1, '78.960')] [2023-10-14 21:32:08,516][61552] Updated weights for policy 0, policy_version 93152 (0.0009) [2023-10-14 21:32:10,460][61585] Updated weights for policy 1, policy_version 93000 (0.0008) [2023-10-14 21:32:10,823][61585] Updated weights for policy 1, policy_version 93010 (0.0008) [2023-10-14 21:32:11,180][61585] Updated weights for policy 1, policy_version 93020 (0.0009) [2023-10-14 21:32:12,583][61552] Updated weights for policy 0, policy_version 93162 (0.0008) [2023-10-14 21:32:12,936][61552] Updated weights for policy 0, policy_version 93172 (0.0008) [2023-10-14 21:32:13,298][61552] Updated weights for policy 0, policy_version 93182 (0.0009) [2023-10-14 21:32:13,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 190644224. Throughput: 0: 1668.8, 1: 1695.0. Samples: 47675520. Policy #0 lag: (min: 29.0, avg: 34.4, max: 61.0) [2023-10-14 21:32:13,344][60425] Avg episode reward: [(0, '75.860'), (1, '77.200')] [2023-10-14 21:32:15,184][61585] Updated weights for policy 1, policy_version 93030 (0.0008) [2023-10-14 21:32:15,556][61585] Updated weights for policy 1, policy_version 93040 (0.0008) [2023-10-14 21:32:15,926][61585] Updated weights for policy 1, policy_version 93050 (0.0009) [2023-10-14 21:32:17,512][61552] Updated weights for policy 0, policy_version 93192 (0.0009) [2023-10-14 21:32:17,884][61552] Updated weights for policy 0, policy_version 93202 (0.0008) [2023-10-14 21:32:18,245][61552] Updated weights for policy 0, policy_version 93212 (0.0007) [2023-10-14 21:32:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 190709760. Throughput: 0: 1683.2, 1: 1676.2. Samples: 47685552. Policy #0 lag: (min: 29.0, avg: 34.4, max: 61.0) [2023-10-14 21:32:18,344][60425] Avg episode reward: [(0, '79.890'), (1, '79.980')] [2023-10-14 21:32:19,969][61585] Updated weights for policy 1, policy_version 93060 (0.0008) [2023-10-14 21:32:20,327][61585] Updated weights for policy 1, policy_version 93070 (0.0007) [2023-10-14 21:32:20,695][61585] Updated weights for policy 1, policy_version 93080 (0.0008) [2023-10-14 21:32:22,291][61552] Updated weights for policy 0, policy_version 93222 (0.0009) [2023-10-14 21:32:22,661][61552] Updated weights for policy 0, policy_version 93232 (0.0008) [2023-10-14 21:32:23,032][61552] Updated weights for policy 0, policy_version 93242 (0.0009) [2023-10-14 21:32:23,343][60425] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 190808064. Throughput: 0: 1676.7, 1: 1679.1. Samples: 47705812. Policy #0 lag: (min: 29.0, avg: 34.4, max: 61.0) [2023-10-14 21:32:23,344][60425] Avg episode reward: [(0, '81.830'), (1, '82.000')] [2023-10-14 21:32:25,040][61585] Updated weights for policy 1, policy_version 93090 (0.0009) [2023-10-14 21:32:25,405][61585] Updated weights for policy 1, policy_version 93100 (0.0009) [2023-10-14 21:32:25,775][61585] Updated weights for policy 1, policy_version 93110 (0.0007) [2023-10-14 21:32:26,134][61585] Updated weights for policy 1, policy_version 93120 (0.0007) [2023-10-14 21:32:26,844][61552] Updated weights for policy 0, policy_version 93252 (0.0009) [2023-10-14 21:32:27,208][61552] Updated weights for policy 0, policy_version 93262 (0.0010) [2023-10-14 21:32:27,573][61552] Updated weights for policy 0, policy_version 93272 (0.0008) [2023-10-14 21:32:28,343][60425] Fps is (10 sec: 16383.6, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 190873600. Throughput: 0: 1651.1, 1: 1695.9. Samples: 47725446. Policy #0 lag: (min: 29.0, avg: 34.4, max: 61.0) [2023-10-14 21:32:28,344][60425] Avg episode reward: [(0, '76.810'), (1, '80.160')] [2023-10-14 21:32:30,111][61585] Updated weights for policy 1, policy_version 93130 (0.0008) [2023-10-14 21:32:30,470][61585] Updated weights for policy 1, policy_version 93140 (0.0007) [2023-10-14 21:32:30,843][61585] Updated weights for policy 1, policy_version 93150 (0.0009) [2023-10-14 21:32:31,849][61552] Updated weights for policy 0, policy_version 93282 (0.0009) [2023-10-14 21:32:32,217][61552] Updated weights for policy 0, policy_version 93292 (0.0010) [2023-10-14 21:32:32,581][61552] Updated weights for policy 0, policy_version 93302 (0.0007) [2023-10-14 21:32:32,947][61552] Updated weights for policy 0, policy_version 93312 (0.0007) [2023-10-14 21:32:33,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 190939136. Throughput: 0: 1668.1, 1: 1671.0. Samples: 47735706. Policy #0 lag: (min: 29.0, avg: 34.4, max: 61.0) [2023-10-14 21:32:33,344][60425] Avg episode reward: [(0, '78.750'), (1, '80.300')] [2023-10-14 21:32:34,979][61585] Updated weights for policy 1, policy_version 93160 (0.0011) [2023-10-14 21:32:35,340][61585] Updated weights for policy 1, policy_version 93170 (0.0008) [2023-10-14 21:32:35,714][61585] Updated weights for policy 1, policy_version 93180 (0.0008) [2023-10-14 21:32:37,072][61552] Updated weights for policy 0, policy_version 93322 (0.0010) [2023-10-14 21:32:37,433][61552] Updated weights for policy 0, policy_version 93332 (0.0010) [2023-10-14 21:32:37,801][61552] Updated weights for policy 0, policy_version 93342 (0.0011) [2023-10-14 21:32:38,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 191004672. Throughput: 0: 1670.8, 1: 1684.9. Samples: 47756230. Policy #0 lag: (min: 29.0, avg: 34.4, max: 61.0) [2023-10-14 21:32:38,344][60425] Avg episode reward: [(0, '78.950'), (1, '83.440')] [2023-10-14 21:32:39,738][61585] Updated weights for policy 1, policy_version 93190 (0.0011) [2023-10-14 21:32:40,106][61585] Updated weights for policy 1, policy_version 93200 (0.0009) [2023-10-14 21:32:40,468][61585] Updated weights for policy 1, policy_version 93210 (0.0011) [2023-10-14 21:32:41,709][61552] Updated weights for policy 0, policy_version 93352 (0.0011) [2023-10-14 21:32:42,082][61552] Updated weights for policy 0, policy_version 93362 (0.0008) [2023-10-14 21:32:42,453][61552] Updated weights for policy 0, policy_version 93372 (0.0008) [2023-10-14 21:32:43,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 191070208. Throughput: 0: 1649.6, 1: 1699.9. Samples: 47775946. Policy #0 lag: (min: 29.0, avg: 34.4, max: 61.0) [2023-10-14 21:32:43,345][60425] Avg episode reward: [(0, '79.110'), (1, '78.680')] [2023-10-14 21:32:44,404][61585] Updated weights for policy 1, policy_version 93220 (0.0009) [2023-10-14 21:32:44,775][61585] Updated weights for policy 1, policy_version 93230 (0.0007) [2023-10-14 21:32:45,141][61585] Updated weights for policy 1, policy_version 93240 (0.0008) [2023-10-14 21:32:46,431][61552] Updated weights for policy 0, policy_version 93382 (0.0008) [2023-10-14 21:32:46,796][61552] Updated weights for policy 0, policy_version 93392 (0.0009) [2023-10-14 21:32:47,173][61552] Updated weights for policy 0, policy_version 93402 (0.0009) [2023-10-14 21:32:48,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 191135744. Throughput: 0: 1673.8, 1: 1675.0. Samples: 47786392. Policy #0 lag: (min: 29.0, avg: 34.4, max: 61.0) [2023-10-14 21:32:48,344][60425] Avg episode reward: [(0, '70.560'), (1, '77.630')] [2023-10-14 21:32:49,237][61585] Updated weights for policy 1, policy_version 93250 (0.0008) [2023-10-14 21:32:49,602][61585] Updated weights for policy 1, policy_version 93260 (0.0009) [2023-10-14 21:32:49,975][61585] Updated weights for policy 1, policy_version 93270 (0.0009) [2023-10-14 21:32:50,348][61585] Updated weights for policy 1, policy_version 93280 (0.0008) [2023-10-14 21:32:51,394][61552] Updated weights for policy 0, policy_version 93412 (0.0010) [2023-10-14 21:32:51,792][61552] Updated weights for policy 0, policy_version 93422 (0.0009) [2023-10-14 21:32:52,162][61552] Updated weights for policy 0, policy_version 93432 (0.0008) [2023-10-14 21:32:53,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 191201280. Throughput: 0: 1660.7, 1: 1698.1. Samples: 47806492. Policy #0 lag: (min: 29.0, avg: 34.4, max: 61.0) [2023-10-14 21:32:53,344][60425] Avg episode reward: [(0, '77.300'), (1, '80.560')] [2023-10-14 21:32:54,450][61585] Updated weights for policy 1, policy_version 93290 (0.0009) [2023-10-14 21:32:54,814][61585] Updated weights for policy 1, policy_version 93300 (0.0010) [2023-10-14 21:32:55,179][61585] Updated weights for policy 1, policy_version 93310 (0.0011) [2023-10-14 21:32:56,333][61552] Updated weights for policy 0, policy_version 93442 (0.0009) [2023-10-14 21:32:56,699][61552] Updated weights for policy 0, policy_version 93452 (0.0008) [2023-10-14 21:32:57,071][61552] Updated weights for policy 0, policy_version 93462 (0.0010) [2023-10-14 21:32:57,436][61552] Updated weights for policy 0, policy_version 93472 (0.0008) [2023-10-14 21:32:58,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 191266816. Throughput: 0: 1655.5, 1: 1687.2. Samples: 47825942. Policy #0 lag: (min: 29.0, avg: 34.4, max: 61.0) [2023-10-14 21:32:58,344][60425] Avg episode reward: [(0, '75.360'), (1, '81.160')] [2023-10-14 21:32:59,400][61585] Updated weights for policy 1, policy_version 93320 (0.0008) [2023-10-14 21:32:59,771][61585] Updated weights for policy 1, policy_version 93330 (0.0008) [2023-10-14 21:33:00,128][61585] Updated weights for policy 1, policy_version 93340 (0.0007) [2023-10-14 21:33:01,466][61552] Updated weights for policy 0, policy_version 93482 (0.0008) [2023-10-14 21:33:01,827][61552] Updated weights for policy 0, policy_version 93492 (0.0008) [2023-10-14 21:33:02,190][61552] Updated weights for policy 0, policy_version 93502 (0.0009) [2023-10-14 21:33:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 191332352. Throughput: 0: 1671.7, 1: 1675.3. Samples: 47836168. Policy #0 lag: (min: 29.0, avg: 34.4, max: 61.0) [2023-10-14 21:33:03,344][60425] Avg episode reward: [(0, '76.730'), (1, '77.540')] [2023-10-14 21:33:04,127][61585] Updated weights for policy 1, policy_version 93350 (0.0008) [2023-10-14 21:33:04,490][61585] Updated weights for policy 1, policy_version 93360 (0.0009) [2023-10-14 21:33:04,856][61585] Updated weights for policy 1, policy_version 93370 (0.0009) [2023-10-14 21:33:06,332][61552] Updated weights for policy 0, policy_version 93512 (0.0009) [2023-10-14 21:33:06,703][61552] Updated weights for policy 0, policy_version 93522 (0.0009) [2023-10-14 21:33:07,058][61552] Updated weights for policy 0, policy_version 93532 (0.0010) [2023-10-14 21:33:08,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 191397888. Throughput: 0: 1659.8, 1: 1683.6. Samples: 47856266. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 21:33:08,344][60425] Avg episode reward: [(0, '73.190'), (1, '79.890')] [2023-10-14 21:33:08,895][61585] Updated weights for policy 1, policy_version 93380 (0.0008) [2023-10-14 21:33:09,264][61585] Updated weights for policy 1, policy_version 93390 (0.0007) [2023-10-14 21:33:09,624][61585] Updated weights for policy 1, policy_version 93400 (0.0007) [2023-10-14 21:33:11,203][61552] Updated weights for policy 0, policy_version 93542 (0.0010) [2023-10-14 21:33:11,564][61552] Updated weights for policy 0, policy_version 93552 (0.0010) [2023-10-14 21:33:11,929][61552] Updated weights for policy 0, policy_version 93562 (0.0011) [2023-10-14 21:33:13,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 191463424. Throughput: 0: 1671.0, 1: 1690.3. Samples: 47876704. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 21:33:13,344][60425] Avg episode reward: [(0, '73.290'), (1, '76.860')] [2023-10-14 21:33:13,441][61585] Updated weights for policy 1, policy_version 93410 (0.0008) [2023-10-14 21:33:13,800][61585] Updated weights for policy 1, policy_version 93420 (0.0009) [2023-10-14 21:33:14,164][61585] Updated weights for policy 1, policy_version 93430 (0.0009) [2023-10-14 21:33:14,531][61585] Updated weights for policy 1, policy_version 93440 (0.0008) [2023-10-14 21:33:16,019][61552] Updated weights for policy 0, policy_version 93572 (0.0009) [2023-10-14 21:33:16,390][61552] Updated weights for policy 0, policy_version 93582 (0.0007) [2023-10-14 21:33:16,757][61552] Updated weights for policy 0, policy_version 93592 (0.0010) [2023-10-14 21:33:18,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 191528960. Throughput: 0: 1678.4, 1: 1683.1. Samples: 47886972. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 21:33:18,344][60425] Avg episode reward: [(0, '72.120'), (1, '81.010')] [2023-10-14 21:33:18,665][61585] Updated weights for policy 1, policy_version 93450 (0.0010) [2023-10-14 21:33:19,031][61585] Updated weights for policy 1, policy_version 93460 (0.0007) [2023-10-14 21:33:19,388][61585] Updated weights for policy 1, policy_version 93470 (0.0009) [2023-10-14 21:33:20,918][61552] Updated weights for policy 0, policy_version 93602 (0.0009) [2023-10-14 21:33:21,283][61552] Updated weights for policy 0, policy_version 93612 (0.0008) [2023-10-14 21:33:21,645][61552] Updated weights for policy 0, policy_version 93622 (0.0007) [2023-10-14 21:33:22,015][61552] Updated weights for policy 0, policy_version 93632 (0.0010) [2023-10-14 21:33:23,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 191594496. Throughput: 0: 1655.1, 1: 1686.3. Samples: 47906590. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 21:33:23,344][60425] Avg episode reward: [(0, '74.660'), (1, '80.770')] [2023-10-14 21:33:23,436][61585] Updated weights for policy 1, policy_version 93480 (0.0007) [2023-10-14 21:33:23,787][61585] Updated weights for policy 1, policy_version 93490 (0.0007) [2023-10-14 21:33:24,147][61585] Updated weights for policy 1, policy_version 93500 (0.0008) [2023-10-14 21:33:26,107][61552] Updated weights for policy 0, policy_version 93642 (0.0009) [2023-10-14 21:33:26,478][61552] Updated weights for policy 0, policy_version 93652 (0.0008) [2023-10-14 21:33:26,845][61552] Updated weights for policy 0, policy_version 93662 (0.0009) [2023-10-14 21:33:28,308][61585] Updated weights for policy 1, policy_version 93510 (0.0010) [2023-10-14 21:33:28,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 191660032. Throughput: 0: 1675.3, 1: 1682.0. Samples: 47927022. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 21:33:28,344][60425] Avg episode reward: [(0, '71.710'), (1, '77.680')] [2023-10-14 21:33:28,680][61585] Updated weights for policy 1, policy_version 93520 (0.0007) [2023-10-14 21:33:29,047][61585] Updated weights for policy 1, policy_version 93530 (0.0007) [2023-10-14 21:33:30,847][61552] Updated weights for policy 0, policy_version 93672 (0.0010) [2023-10-14 21:33:31,218][61552] Updated weights for policy 0, policy_version 93682 (0.0008) [2023-10-14 21:33:31,585][61552] Updated weights for policy 0, policy_version 93692 (0.0008) [2023-10-14 21:33:33,155][61585] Updated weights for policy 1, policy_version 93540 (0.0010) [2023-10-14 21:33:33,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 191725568. Throughput: 0: 1671.3, 1: 1682.6. Samples: 47937318. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 21:33:33,344][60425] Avg episode reward: [(0, '75.180'), (1, '81.460')] [2023-10-14 21:33:33,519][61585] Updated weights for policy 1, policy_version 93550 (0.0010) [2023-10-14 21:33:33,891][61585] Updated weights for policy 1, policy_version 93560 (0.0008) [2023-10-14 21:33:35,636][61552] Updated weights for policy 0, policy_version 93702 (0.0008) [2023-10-14 21:33:35,997][61552] Updated weights for policy 0, policy_version 93712 (0.0010) [2023-10-14 21:33:36,364][61552] Updated weights for policy 0, policy_version 93722 (0.0009) [2023-10-14 21:33:37,805][61585] Updated weights for policy 1, policy_version 93570 (0.0008) [2023-10-14 21:33:38,172][61585] Updated weights for policy 1, policy_version 93580 (0.0007) [2023-10-14 21:33:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 191791104. Throughput: 0: 1657.1, 1: 1686.5. Samples: 47956952. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 21:33:38,344][60425] Avg episode reward: [(0, '77.580'), (1, '81.140')] [2023-10-14 21:33:38,545][61585] Updated weights for policy 1, policy_version 93590 (0.0008) [2023-10-14 21:33:38,908][61585] Updated weights for policy 1, policy_version 93600 (0.0010) [2023-10-14 21:33:40,661][61552] Updated weights for policy 0, policy_version 93732 (0.0009) [2023-10-14 21:33:41,051][61552] Updated weights for policy 0, policy_version 93742 (0.0008) [2023-10-14 21:33:41,409][61552] Updated weights for policy 0, policy_version 93752 (0.0008) [2023-10-14 21:33:43,208][61585] Updated weights for policy 1, policy_version 93610 (0.0009) [2023-10-14 21:33:43,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 191856640. Throughput: 0: 1671.6, 1: 1689.7. Samples: 47977200. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 21:33:43,344][60425] Avg episode reward: [(0, '73.560'), (1, '82.100')] [2023-10-14 21:33:43,352][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000093760_96010240.pth... [2023-10-14 21:33:43,392][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000092192_94404608.pth [2023-10-14 21:33:43,397][61172] Saving a milestone ./train_atari/atari_roadrunner_APPO/checkpoint_p0/milestones/checkpoint_000093760_96010240.pth [2023-10-14 21:33:43,586][61585] Updated weights for policy 1, policy_version 93620 (0.0007) [2023-10-14 21:33:43,948][61585] Updated weights for policy 1, policy_version 93630 (0.0008) [2023-10-14 21:33:44,015][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000093632_95879168.pth... [2023-10-14 21:33:44,057][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000092032_94240768.pth [2023-10-14 21:33:44,063][61248] Saving a milestone ./train_atari/atari_roadrunner_APPO/checkpoint_p1/milestones/checkpoint_000093632_95879168.pth [2023-10-14 21:33:45,496][61552] Updated weights for policy 0, policy_version 93762 (0.0008) [2023-10-14 21:33:45,865][61552] Updated weights for policy 0, policy_version 93772 (0.0009) [2023-10-14 21:33:46,237][61552] Updated weights for policy 0, policy_version 93782 (0.0009) [2023-10-14 21:33:46,601][61552] Updated weights for policy 0, policy_version 93792 (0.0011) [2023-10-14 21:33:47,903][61585] Updated weights for policy 1, policy_version 93640 (0.0009) [2023-10-14 21:33:48,260][61585] Updated weights for policy 1, policy_version 93650 (0.0009) [2023-10-14 21:33:48,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 191922176. Throughput: 0: 1664.2, 1: 1691.5. Samples: 47987176. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 21:33:48,344][60425] Avg episode reward: [(0, '79.900'), (1, '81.930')] [2023-10-14 21:33:48,622][61585] Updated weights for policy 1, policy_version 93660 (0.0008) [2023-10-14 21:33:50,702][61552] Updated weights for policy 0, policy_version 93802 (0.0009) [2023-10-14 21:33:51,062][61552] Updated weights for policy 0, policy_version 93812 (0.0008) [2023-10-14 21:33:51,425][61552] Updated weights for policy 0, policy_version 93822 (0.0009) [2023-10-14 21:33:52,679][61585] Updated weights for policy 1, policy_version 93670 (0.0007) [2023-10-14 21:33:53,049][61585] Updated weights for policy 1, policy_version 93680 (0.0007) [2023-10-14 21:33:53,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 191987712. Throughput: 0: 1656.7, 1: 1686.5. Samples: 48006708. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 21:33:53,344][60425] Avg episode reward: [(0, '77.150'), (1, '82.300')] [2023-10-14 21:33:53,418][61585] Updated weights for policy 1, policy_version 93690 (0.0008) [2023-10-14 21:33:55,696][61552] Updated weights for policy 0, policy_version 93832 (0.0008) [2023-10-14 21:33:56,062][61552] Updated weights for policy 0, policy_version 93842 (0.0010) [2023-10-14 21:33:56,425][61552] Updated weights for policy 0, policy_version 93852 (0.0010) [2023-10-14 21:33:57,523][61585] Updated weights for policy 1, policy_version 93700 (0.0010) [2023-10-14 21:33:57,885][61585] Updated weights for policy 1, policy_version 93710 (0.0009) [2023-10-14 21:33:58,250][61585] Updated weights for policy 1, policy_version 93720 (0.0009) [2023-10-14 21:33:58,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 192053248. Throughput: 0: 1666.5, 1: 1675.9. Samples: 48027114. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 21:33:58,344][60425] Avg episode reward: [(0, '78.680'), (1, '76.420')] [2023-10-14 21:34:00,480][61552] Updated weights for policy 0, policy_version 93862 (0.0010) [2023-10-14 21:34:00,849][61552] Updated weights for policy 0, policy_version 93872 (0.0009) [2023-10-14 21:34:01,227][61552] Updated weights for policy 0, policy_version 93882 (0.0008) [2023-10-14 21:34:02,291][61585] Updated weights for policy 1, policy_version 93730 (0.0008) [2023-10-14 21:34:02,657][61585] Updated weights for policy 1, policy_version 93740 (0.0007) [2023-10-14 21:34:03,027][61585] Updated weights for policy 1, policy_version 93750 (0.0008) [2023-10-14 21:34:03,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 192118784. Throughput: 0: 1660.5, 1: 1688.4. Samples: 48037670. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-14 21:34:03,344][60425] Avg episode reward: [(0, '74.060'), (1, '80.020')] [2023-10-14 21:34:03,398][61585] Updated weights for policy 1, policy_version 93760 (0.0009) [2023-10-14 21:34:05,113][61552] Updated weights for policy 0, policy_version 93892 (0.0008) [2023-10-14 21:34:05,484][61552] Updated weights for policy 0, policy_version 93902 (0.0007) [2023-10-14 21:34:05,847][61552] Updated weights for policy 0, policy_version 93912 (0.0008) [2023-10-14 21:34:07,400][61585] Updated weights for policy 1, policy_version 93770 (0.0011) [2023-10-14 21:34:07,759][61585] Updated weights for policy 1, policy_version 93780 (0.0010) [2023-10-14 21:34:08,119][61585] Updated weights for policy 1, policy_version 93790 (0.0011) [2023-10-14 21:34:08,343][60425] Fps is (10 sec: 16384.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 192217088. Throughput: 0: 1666.0, 1: 1685.6. Samples: 48057412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:34:08,344][60425] Avg episode reward: [(0, '78.710'), (1, '80.780')] [2023-10-14 21:34:10,001][61552] Updated weights for policy 0, policy_version 93922 (0.0009) [2023-10-14 21:34:10,367][61552] Updated weights for policy 0, policy_version 93932 (0.0010) [2023-10-14 21:34:10,739][61552] Updated weights for policy 0, policy_version 93942 (0.0010) [2023-10-14 21:34:11,098][61552] Updated weights for policy 0, policy_version 93952 (0.0008) [2023-10-14 21:34:12,287][61585] Updated weights for policy 1, policy_version 93800 (0.0011) [2023-10-14 21:34:12,656][61585] Updated weights for policy 1, policy_version 93810 (0.0008) [2023-10-14 21:34:13,026][61585] Updated weights for policy 1, policy_version 93820 (0.0009) [2023-10-14 21:34:13,343][60425] Fps is (10 sec: 16384.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 192282624. Throughput: 0: 1675.1, 1: 1664.6. Samples: 48077310. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:34:13,344][60425] Avg episode reward: [(0, '75.740'), (1, '80.110')] [2023-10-14 21:34:15,172][61552] Updated weights for policy 0, policy_version 93962 (0.0008) [2023-10-14 21:34:15,546][61552] Updated weights for policy 0, policy_version 93972 (0.0007) [2023-10-14 21:34:15,912][61552] Updated weights for policy 0, policy_version 93982 (0.0007) [2023-10-14 21:34:17,075][61585] Updated weights for policy 1, policy_version 93830 (0.0009) [2023-10-14 21:34:17,446][61585] Updated weights for policy 1, policy_version 93840 (0.0008) [2023-10-14 21:34:17,809][61585] Updated weights for policy 1, policy_version 93850 (0.0011) [2023-10-14 21:34:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 192348160. Throughput: 0: 1657.6, 1: 1681.4. Samples: 48087570. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:34:18,344][60425] Avg episode reward: [(0, '74.740'), (1, '77.450')] [2023-10-14 21:34:20,002][61552] Updated weights for policy 0, policy_version 93992 (0.0009) [2023-10-14 21:34:20,367][61552] Updated weights for policy 0, policy_version 94002 (0.0008) [2023-10-14 21:34:20,736][61552] Updated weights for policy 0, policy_version 94012 (0.0007) [2023-10-14 21:34:21,914][61585] Updated weights for policy 1, policy_version 93860 (0.0008) [2023-10-14 21:34:22,288][61585] Updated weights for policy 1, policy_version 93870 (0.0007) [2023-10-14 21:34:22,649][61585] Updated weights for policy 1, policy_version 93880 (0.0007) [2023-10-14 21:34:23,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 192413696. Throughput: 0: 1676.8, 1: 1679.6. Samples: 48107990. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:34:23,344][60425] Avg episode reward: [(0, '76.130'), (1, '81.160')] [2023-10-14 21:34:24,902][61552] Updated weights for policy 0, policy_version 94022 (0.0009) [2023-10-14 21:34:25,271][61552] Updated weights for policy 0, policy_version 94032 (0.0009) [2023-10-14 21:34:25,645][61552] Updated weights for policy 0, policy_version 94042 (0.0007) [2023-10-14 21:34:26,711][61585] Updated weights for policy 1, policy_version 93890 (0.0011) [2023-10-14 21:34:27,074][61585] Updated weights for policy 1, policy_version 93900 (0.0011) [2023-10-14 21:34:27,444][61585] Updated weights for policy 1, policy_version 93910 (0.0008) [2023-10-14 21:34:27,802][61585] Updated weights for policy 1, policy_version 93920 (0.0008) [2023-10-14 21:34:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 192479232. Throughput: 0: 1682.6, 1: 1661.8. Samples: 48127698. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:34:28,344][60425] Avg episode reward: [(0, '75.760'), (1, '76.000')] [2023-10-14 21:34:29,808][61552] Updated weights for policy 0, policy_version 94052 (0.0008) [2023-10-14 21:34:30,203][61552] Updated weights for policy 0, policy_version 94062 (0.0010) [2023-10-14 21:34:30,572][61552] Updated weights for policy 0, policy_version 94072 (0.0008) [2023-10-14 21:34:31,992][61585] Updated weights for policy 1, policy_version 93930 (0.0007) [2023-10-14 21:34:32,357][61585] Updated weights for policy 1, policy_version 93940 (0.0009) [2023-10-14 21:34:32,724][61585] Updated weights for policy 1, policy_version 93950 (0.0007) [2023-10-14 21:34:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 192544768. Throughput: 0: 1665.7, 1: 1686.8. Samples: 48138040. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:34:33,344][60425] Avg episode reward: [(0, '77.760'), (1, '79.870')] [2023-10-14 21:34:34,569][61552] Updated weights for policy 0, policy_version 94082 (0.0007) [2023-10-14 21:34:34,951][61552] Updated weights for policy 0, policy_version 94092 (0.0010) [2023-10-14 21:34:35,318][61552] Updated weights for policy 0, policy_version 94102 (0.0008) [2023-10-14 21:34:35,680][61552] Updated weights for policy 0, policy_version 94112 (0.0008) [2023-10-14 21:34:36,882][61585] Updated weights for policy 1, policy_version 93960 (0.0009) [2023-10-14 21:34:37,246][61585] Updated weights for policy 1, policy_version 93970 (0.0007) [2023-10-14 21:34:37,623][61585] Updated weights for policy 1, policy_version 93980 (0.0007) [2023-10-14 21:34:38,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 192610304. Throughput: 0: 1684.8, 1: 1681.3. Samples: 48158180. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:34:38,344][60425] Avg episode reward: [(0, '79.370'), (1, '83.380')] [2023-10-14 21:34:39,780][61552] Updated weights for policy 0, policy_version 94122 (0.0009) [2023-10-14 21:34:40,152][61552] Updated weights for policy 0, policy_version 94132 (0.0007) [2023-10-14 21:34:40,518][61552] Updated weights for policy 0, policy_version 94142 (0.0008) [2023-10-14 21:34:41,578][61585] Updated weights for policy 1, policy_version 93990 (0.0008) [2023-10-14 21:34:41,938][61585] Updated weights for policy 1, policy_version 94000 (0.0009) [2023-10-14 21:34:42,301][61585] Updated weights for policy 1, policy_version 94010 (0.0008) [2023-10-14 21:34:43,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 192675840. Throughput: 0: 1688.2, 1: 1665.6. Samples: 48178038. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:34:43,345][60425] Avg episode reward: [(0, '82.420'), (1, '83.350')] [2023-10-14 21:34:44,575][61552] Updated weights for policy 0, policy_version 94152 (0.0009) [2023-10-14 21:34:44,940][61552] Updated weights for policy 0, policy_version 94162 (0.0007) [2023-10-14 21:34:45,304][61552] Updated weights for policy 0, policy_version 94172 (0.0008) [2023-10-14 21:34:46,484][61585] Updated weights for policy 1, policy_version 94020 (0.0009) [2023-10-14 21:34:46,859][61585] Updated weights for policy 1, policy_version 94030 (0.0009) [2023-10-14 21:34:47,214][61585] Updated weights for policy 1, policy_version 94040 (0.0008) [2023-10-14 21:34:48,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 192741376. Throughput: 0: 1665.7, 1: 1680.5. Samples: 48188252. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:34:48,344][60425] Avg episode reward: [(0, '79.740'), (1, '82.890')] [2023-10-14 21:34:49,512][61552] Updated weights for policy 0, policy_version 94182 (0.0009) [2023-10-14 21:34:49,885][61552] Updated weights for policy 0, policy_version 94192 (0.0010) [2023-10-14 21:34:50,252][61552] Updated weights for policy 0, policy_version 94202 (0.0009) [2023-10-14 21:34:51,327][61585] Updated weights for policy 1, policy_version 94050 (0.0008) [2023-10-14 21:34:51,687][61585] Updated weights for policy 1, policy_version 94060 (0.0008) [2023-10-14 21:34:52,055][61585] Updated weights for policy 1, policy_version 94070 (0.0009) [2023-10-14 21:34:52,412][61585] Updated weights for policy 1, policy_version 94080 (0.0010) [2023-10-14 21:34:53,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 192806912. Throughput: 0: 1678.2, 1: 1666.0. Samples: 48207904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:34:53,344][60425] Avg episode reward: [(0, '80.120'), (1, '80.780')] [2023-10-14 21:34:54,354][61552] Updated weights for policy 0, policy_version 94212 (0.0009) [2023-10-14 21:34:54,719][61552] Updated weights for policy 0, policy_version 94222 (0.0008) [2023-10-14 21:34:55,090][61552] Updated weights for policy 0, policy_version 94232 (0.0008) [2023-10-14 21:34:56,335][61585] Updated weights for policy 1, policy_version 94090 (0.0009) [2023-10-14 21:34:56,700][61585] Updated weights for policy 1, policy_version 94100 (0.0009) [2023-10-14 21:34:57,066][61585] Updated weights for policy 1, policy_version 94110 (0.0009) [2023-10-14 21:34:58,344][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 192872448. Throughput: 0: 1676.0, 1: 1677.6. Samples: 48228224. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:34:58,345][60425] Avg episode reward: [(0, '78.960'), (1, '83.620')] [2023-10-14 21:34:59,196][61552] Updated weights for policy 0, policy_version 94242 (0.0008) [2023-10-14 21:34:59,559][61552] Updated weights for policy 0, policy_version 94252 (0.0007) [2023-10-14 21:34:59,932][61552] Updated weights for policy 0, policy_version 94262 (0.0007) [2023-10-14 21:35:00,300][61552] Updated weights for policy 0, policy_version 94272 (0.0013) [2023-10-14 21:35:01,068][61585] Updated weights for policy 1, policy_version 94120 (0.0008) [2023-10-14 21:35:01,434][61585] Updated weights for policy 1, policy_version 94130 (0.0010) [2023-10-14 21:35:01,799][61585] Updated weights for policy 1, policy_version 94140 (0.0011) [2023-10-14 21:35:03,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 192937984. Throughput: 0: 1666.0, 1: 1692.5. Samples: 48238702. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:35:03,344][60425] Avg episode reward: [(0, '79.630'), (1, '77.910')] [2023-10-14 21:35:04,270][61552] Updated weights for policy 0, policy_version 94282 (0.0007) [2023-10-14 21:35:04,639][61552] Updated weights for policy 0, policy_version 94292 (0.0008) [2023-10-14 21:35:05,018][61552] Updated weights for policy 0, policy_version 94302 (0.0011) [2023-10-14 21:35:05,989][61585] Updated weights for policy 1, policy_version 94150 (0.0008) [2023-10-14 21:35:06,353][61585] Updated weights for policy 1, policy_version 94160 (0.0007) [2023-10-14 21:35:06,711][61585] Updated weights for policy 1, policy_version 94170 (0.0007) [2023-10-14 21:35:08,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 193003520. Throughput: 0: 1674.1, 1: 1667.0. Samples: 48258340. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:35:08,344][60425] Avg episode reward: [(0, '75.930'), (1, '81.650')] [2023-10-14 21:35:09,173][61552] Updated weights for policy 0, policy_version 94312 (0.0010) [2023-10-14 21:35:09,549][61552] Updated weights for policy 0, policy_version 94322 (0.0010) [2023-10-14 21:35:09,917][61552] Updated weights for policy 0, policy_version 94332 (0.0009) [2023-10-14 21:35:10,688][61585] Updated weights for policy 1, policy_version 94180 (0.0007) [2023-10-14 21:35:11,050][61585] Updated weights for policy 1, policy_version 94190 (0.0008) [2023-10-14 21:35:11,418][61585] Updated weights for policy 1, policy_version 94200 (0.0008) [2023-10-14 21:35:13,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 193069056. Throughput: 0: 1683.5, 1: 1678.5. Samples: 48278988. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:35:13,344][60425] Avg episode reward: [(0, '78.810'), (1, '82.830')] [2023-10-14 21:35:13,821][61552] Updated weights for policy 0, policy_version 94342 (0.0007) [2023-10-14 21:35:14,186][61552] Updated weights for policy 0, policy_version 94352 (0.0008) [2023-10-14 21:35:14,544][61552] Updated weights for policy 0, policy_version 94362 (0.0011) [2023-10-14 21:35:15,714][61585] Updated weights for policy 1, policy_version 94210 (0.0010) [2023-10-14 21:35:16,074][61585] Updated weights for policy 1, policy_version 94220 (0.0008) [2023-10-14 21:35:16,441][61585] Updated weights for policy 1, policy_version 94230 (0.0009) [2023-10-14 21:35:16,800][61585] Updated weights for policy 1, policy_version 94240 (0.0007) [2023-10-14 21:35:18,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 193134592. Throughput: 0: 1678.4, 1: 1677.2. Samples: 48289042. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:35:18,344][60425] Avg episode reward: [(0, '75.540'), (1, '80.880')] [2023-10-14 21:35:18,593][61552] Updated weights for policy 0, policy_version 94372 (0.0009) [2023-10-14 21:35:18,955][61552] Updated weights for policy 0, policy_version 94382 (0.0008) [2023-10-14 21:35:19,327][61552] Updated weights for policy 0, policy_version 94392 (0.0010) [2023-10-14 21:35:20,846][61585] Updated weights for policy 1, policy_version 94250 (0.0008) [2023-10-14 21:35:21,208][61585] Updated weights for policy 1, policy_version 94260 (0.0010) [2023-10-14 21:35:21,566][61585] Updated weights for policy 1, policy_version 94270 (0.0009) [2023-10-14 21:35:23,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 193200128. Throughput: 0: 1683.8, 1: 1659.3. Samples: 48308618. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:35:23,344][60425] Avg episode reward: [(0, '80.510'), (1, '74.260')] [2023-10-14 21:35:23,362][61552] Updated weights for policy 0, policy_version 94402 (0.0009) [2023-10-14 21:35:23,735][61552] Updated weights for policy 0, policy_version 94412 (0.0007) [2023-10-14 21:35:24,098][61552] Updated weights for policy 0, policy_version 94422 (0.0007) [2023-10-14 21:35:24,468][61552] Updated weights for policy 0, policy_version 94432 (0.0008) [2023-10-14 21:35:25,797][61585] Updated weights for policy 1, policy_version 94280 (0.0008) [2023-10-14 21:35:26,176][61585] Updated weights for policy 1, policy_version 94290 (0.0010) [2023-10-14 21:35:26,539][61585] Updated weights for policy 1, policy_version 94300 (0.0009) [2023-10-14 21:35:28,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 193265664. Throughput: 0: 1682.0, 1: 1676.6. Samples: 48329174. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:35:28,344][60425] Avg episode reward: [(0, '77.950'), (1, '84.230')] [2023-10-14 21:35:28,406][61552] Updated weights for policy 0, policy_version 94442 (0.0007) [2023-10-14 21:35:28,782][61552] Updated weights for policy 0, policy_version 94452 (0.0007) [2023-10-14 21:35:29,153][61552] Updated weights for policy 0, policy_version 94462 (0.0009) [2023-10-14 21:35:30,686][61585] Updated weights for policy 1, policy_version 94310 (0.0008) [2023-10-14 21:35:31,038][61585] Updated weights for policy 1, policy_version 94320 (0.0009) [2023-10-14 21:35:31,398][61585] Updated weights for policy 1, policy_version 94330 (0.0009) [2023-10-14 21:35:33,215][61552] Updated weights for policy 0, policy_version 94472 (0.0008) [2023-10-14 21:35:33,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 193331200. Throughput: 0: 1683.8, 1: 1670.3. Samples: 48339186. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:35:33,344][60425] Avg episode reward: [(0, '78.540'), (1, '79.740')] [2023-10-14 21:35:33,576][61552] Updated weights for policy 0, policy_version 94482 (0.0007) [2023-10-14 21:35:33,949][61552] Updated weights for policy 0, policy_version 94492 (0.0007) [2023-10-14 21:35:35,508][61585] Updated weights for policy 1, policy_version 94340 (0.0008) [2023-10-14 21:35:35,876][61585] Updated weights for policy 1, policy_version 94350 (0.0010) [2023-10-14 21:35:36,238][61585] Updated weights for policy 1, policy_version 94360 (0.0009) [2023-10-14 21:35:38,039][61552] Updated weights for policy 0, policy_version 94502 (0.0007) [2023-10-14 21:35:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 193396736. Throughput: 0: 1691.8, 1: 1664.6. Samples: 48358940. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:35:38,344][60425] Avg episode reward: [(0, '81.520'), (1, '80.510')] [2023-10-14 21:35:38,408][61552] Updated weights for policy 0, policy_version 94512 (0.0011) [2023-10-14 21:35:38,779][61552] Updated weights for policy 0, policy_version 94522 (0.0010) [2023-10-14 21:35:40,360][61585] Updated weights for policy 1, policy_version 94370 (0.0009) [2023-10-14 21:35:40,730][61585] Updated weights for policy 1, policy_version 94380 (0.0010) [2023-10-14 21:35:41,103][61585] Updated weights for policy 1, policy_version 94390 (0.0008) [2023-10-14 21:35:41,469][61585] Updated weights for policy 1, policy_version 94400 (0.0008) [2023-10-14 21:35:42,774][61552] Updated weights for policy 0, policy_version 94532 (0.0008) [2023-10-14 21:35:43,150][61552] Updated weights for policy 0, policy_version 94542 (0.0009) [2023-10-14 21:35:43,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 193462272. Throughput: 0: 1686.6, 1: 1670.1. Samples: 48379276. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:35:43,344][60425] Avg episode reward: [(0, '83.710'), (1, '81.750')] [2023-10-14 21:35:43,351][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000094400_96665600.pth... [2023-10-14 21:35:43,386][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000092832_95059968.pth [2023-10-14 21:35:43,520][61552] Updated weights for policy 0, policy_version 94552 (0.0007) [2023-10-14 21:35:43,809][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000094560_96829440.pth... [2023-10-14 21:35:43,849][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000092992_95223808.pth [2023-10-14 21:35:45,670][61585] Updated weights for policy 1, policy_version 94410 (0.0012) [2023-10-14 21:35:46,031][61585] Updated weights for policy 1, policy_version 94420 (0.0011) [2023-10-14 21:35:46,405][61585] Updated weights for policy 1, policy_version 94430 (0.0009) [2023-10-14 21:35:47,530][61552] Updated weights for policy 0, policy_version 94562 (0.0008) [2023-10-14 21:35:47,899][61552] Updated weights for policy 0, policy_version 94572 (0.0008) [2023-10-14 21:35:48,262][61552] Updated weights for policy 0, policy_version 94582 (0.0009) [2023-10-14 21:35:48,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 193527808. Throughput: 0: 1688.1, 1: 1653.8. Samples: 48389090. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:35:48,344][60425] Avg episode reward: [(0, '78.860'), (1, '84.270')] [2023-10-14 21:35:48,630][61552] Updated weights for policy 0, policy_version 94592 (0.0011) [2023-10-14 21:35:50,547][61585] Updated weights for policy 1, policy_version 94440 (0.0009) [2023-10-14 21:35:50,915][61585] Updated weights for policy 1, policy_version 94450 (0.0008) [2023-10-14 21:35:51,286][61585] Updated weights for policy 1, policy_version 94460 (0.0007) [2023-10-14 21:35:52,949][61552] Updated weights for policy 0, policy_version 94602 (0.0007) [2023-10-14 21:35:53,309][61552] Updated weights for policy 0, policy_version 94612 (0.0007) [2023-10-14 21:35:53,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 193593344. Throughput: 0: 1685.6, 1: 1657.1. Samples: 48408758. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:35:53,344][60425] Avg episode reward: [(0, '82.670'), (1, '81.300')] [2023-10-14 21:35:53,680][61552] Updated weights for policy 0, policy_version 94622 (0.0007) [2023-10-14 21:35:55,069][61585] Updated weights for policy 1, policy_version 94470 (0.0010) [2023-10-14 21:35:55,431][61585] Updated weights for policy 1, policy_version 94480 (0.0009) [2023-10-14 21:35:55,798][61585] Updated weights for policy 1, policy_version 94490 (0.0008) [2023-10-14 21:35:57,641][61552] Updated weights for policy 0, policy_version 94632 (0.0008) [2023-10-14 21:35:58,000][61552] Updated weights for policy 0, policy_version 94642 (0.0008) [2023-10-14 21:35:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 193658880. Throughput: 0: 1667.1, 1: 1670.0. Samples: 48429156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:35:58,344][60425] Avg episode reward: [(0, '82.540'), (1, '82.070')] [2023-10-14 21:35:58,370][61552] Updated weights for policy 0, policy_version 94652 (0.0007) [2023-10-14 21:35:59,780][61585] Updated weights for policy 1, policy_version 94500 (0.0008) [2023-10-14 21:36:00,155][61585] Updated weights for policy 1, policy_version 94510 (0.0008) [2023-10-14 21:36:00,523][61585] Updated weights for policy 1, policy_version 94520 (0.0007) [2023-10-14 21:36:02,691][61552] Updated weights for policy 0, policy_version 94662 (0.0008) [2023-10-14 21:36:03,069][61552] Updated weights for policy 0, policy_version 94672 (0.0010) [2023-10-14 21:36:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 193724416. Throughput: 0: 1676.9, 1: 1652.1. Samples: 48438846. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:36:03,344][60425] Avg episode reward: [(0, '79.180'), (1, '81.890')] [2023-10-14 21:36:03,442][61552] Updated weights for policy 0, policy_version 94682 (0.0009) [2023-10-14 21:36:04,673][61585] Updated weights for policy 1, policy_version 94530 (0.0009) [2023-10-14 21:36:05,049][61585] Updated weights for policy 1, policy_version 94540 (0.0010) [2023-10-14 21:36:05,421][61585] Updated weights for policy 1, policy_version 94550 (0.0008) [2023-10-14 21:36:05,782][61585] Updated weights for policy 1, policy_version 94560 (0.0008) [2023-10-14 21:36:07,685][61552] Updated weights for policy 0, policy_version 94692 (0.0009) [2023-10-14 21:36:08,054][61552] Updated weights for policy 0, policy_version 94702 (0.0010) [2023-10-14 21:36:08,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 193789952. Throughput: 0: 1672.4, 1: 1673.9. Samples: 48459204. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-14 21:36:08,344][60425] Avg episode reward: [(0, '82.930'), (1, '81.500')] [2023-10-14 21:36:08,424][61552] Updated weights for policy 0, policy_version 94712 (0.0008) [2023-10-14 21:36:09,740][61585] Updated weights for policy 1, policy_version 94570 (0.0011) [2023-10-14 21:36:10,109][61585] Updated weights for policy 1, policy_version 94580 (0.0008) [2023-10-14 21:36:10,478][61585] Updated weights for policy 1, policy_version 94590 (0.0008) [2023-10-14 21:36:12,407][61552] Updated weights for policy 0, policy_version 94722 (0.0009) [2023-10-14 21:36:12,762][61552] Updated weights for policy 0, policy_version 94732 (0.0007) [2023-10-14 21:36:13,131][61552] Updated weights for policy 0, policy_version 94742 (0.0008) [2023-10-14 21:36:13,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 193855488. Throughput: 0: 1664.9, 1: 1678.1. Samples: 48479610. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-14 21:36:13,344][60425] Avg episode reward: [(0, '76.980'), (1, '79.830')] [2023-10-14 21:36:13,502][61552] Updated weights for policy 0, policy_version 94752 (0.0008) [2023-10-14 21:36:14,638][61585] Updated weights for policy 1, policy_version 94600 (0.0008) [2023-10-14 21:36:15,015][61585] Updated weights for policy 1, policy_version 94610 (0.0007) [2023-10-14 21:36:15,382][61585] Updated weights for policy 1, policy_version 94620 (0.0007) [2023-10-14 21:36:17,699][61552] Updated weights for policy 0, policy_version 94762 (0.0009) [2023-10-14 21:36:18,074][61552] Updated weights for policy 0, policy_version 94772 (0.0009) [2023-10-14 21:36:18,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 193921024. Throughput: 0: 1672.0, 1: 1655.7. Samples: 48488934. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-14 21:36:18,344][60425] Avg episode reward: [(0, '77.650'), (1, '80.660')] [2023-10-14 21:36:18,445][61552] Updated weights for policy 0, policy_version 94782 (0.0009) [2023-10-14 21:36:19,656][61585] Updated weights for policy 1, policy_version 94630 (0.0009) [2023-10-14 21:36:20,021][61585] Updated weights for policy 1, policy_version 94640 (0.0012) [2023-10-14 21:36:20,388][61585] Updated weights for policy 1, policy_version 94650 (0.0007) [2023-10-14 21:36:22,563][61552] Updated weights for policy 0, policy_version 94792 (0.0009) [2023-10-14 21:36:22,942][61552] Updated weights for policy 0, policy_version 94802 (0.0009) [2023-10-14 21:36:23,311][61552] Updated weights for policy 0, policy_version 94812 (0.0009) [2023-10-14 21:36:23,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 193986560. Throughput: 0: 1666.9, 1: 1669.6. Samples: 48509082. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-14 21:36:23,344][60425] Avg episode reward: [(0, '79.780'), (1, '84.500')] [2023-10-14 21:36:24,560][61585] Updated weights for policy 1, policy_version 94660 (0.0008) [2023-10-14 21:36:24,934][61585] Updated weights for policy 1, policy_version 94670 (0.0009) [2023-10-14 21:36:25,289][61585] Updated weights for policy 1, policy_version 94680 (0.0010) [2023-10-14 21:36:27,400][61552] Updated weights for policy 0, policy_version 94822 (0.0008) [2023-10-14 21:36:27,768][61552] Updated weights for policy 0, policy_version 94832 (0.0008) [2023-10-14 21:36:28,135][61552] Updated weights for policy 0, policy_version 94842 (0.0007) [2023-10-14 21:36:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 194052096. Throughput: 0: 1657.0, 1: 1669.1. Samples: 48528948. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-14 21:36:28,344][60425] Avg episode reward: [(0, '80.500'), (1, '78.490')] [2023-10-14 21:36:29,364][61585] Updated weights for policy 1, policy_version 94690 (0.0009) [2023-10-14 21:36:29,735][61585] Updated weights for policy 1, policy_version 94700 (0.0010) [2023-10-14 21:36:30,103][61585] Updated weights for policy 1, policy_version 94710 (0.0010) [2023-10-14 21:36:30,453][61585] Updated weights for policy 1, policy_version 94720 (0.0009) [2023-10-14 21:36:32,143][61552] Updated weights for policy 0, policy_version 94852 (0.0007) [2023-10-14 21:36:32,507][61552] Updated weights for policy 0, policy_version 94862 (0.0007) [2023-10-14 21:36:32,886][61552] Updated weights for policy 0, policy_version 94872 (0.0007) [2023-10-14 21:36:33,343][60425] Fps is (10 sec: 16384.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 194150400. Throughput: 0: 1670.0, 1: 1653.2. Samples: 48538636. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-14 21:36:33,344][60425] Avg episode reward: [(0, '77.150'), (1, '84.280')] [2023-10-14 21:36:34,584][61585] Updated weights for policy 1, policy_version 94730 (0.0009) [2023-10-14 21:36:34,943][61585] Updated weights for policy 1, policy_version 94740 (0.0012) [2023-10-14 21:36:35,311][61585] Updated weights for policy 1, policy_version 94750 (0.0007) [2023-10-14 21:36:36,836][61552] Updated weights for policy 0, policy_version 94882 (0.0008) [2023-10-14 21:36:37,208][61552] Updated weights for policy 0, policy_version 94892 (0.0007) [2023-10-14 21:36:37,572][61552] Updated weights for policy 0, policy_version 94902 (0.0008) [2023-10-14 21:36:37,937][61552] Updated weights for policy 0, policy_version 94912 (0.0008) [2023-10-14 21:36:38,344][60425] Fps is (10 sec: 16383.6, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 194215936. Throughput: 0: 1674.6, 1: 1673.7. Samples: 48559434. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-14 21:36:38,345][60425] Avg episode reward: [(0, '79.720'), (1, '79.880')] [2023-10-14 21:36:39,383][61585] Updated weights for policy 1, policy_version 94760 (0.0009) [2023-10-14 21:36:39,753][61585] Updated weights for policy 1, policy_version 94770 (0.0010) [2023-10-14 21:36:40,115][61585] Updated weights for policy 1, policy_version 94780 (0.0008) [2023-10-14 21:36:42,126][61552] Updated weights for policy 0, policy_version 94922 (0.0009) [2023-10-14 21:36:42,495][61552] Updated weights for policy 0, policy_version 94932 (0.0009) [2023-10-14 21:36:42,854][61552] Updated weights for policy 0, policy_version 94942 (0.0008) [2023-10-14 21:36:43,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 194281472. Throughput: 0: 1658.0, 1: 1673.1. Samples: 48579058. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-14 21:36:43,345][60425] Avg episode reward: [(0, '78.740'), (1, '79.120')] [2023-10-14 21:36:44,087][61585] Updated weights for policy 1, policy_version 94790 (0.0008) [2023-10-14 21:36:44,452][61585] Updated weights for policy 1, policy_version 94800 (0.0009) [2023-10-14 21:36:44,813][61585] Updated weights for policy 1, policy_version 94810 (0.0010) [2023-10-14 21:36:46,889][61552] Updated weights for policy 0, policy_version 94952 (0.0009) [2023-10-14 21:36:47,263][61552] Updated weights for policy 0, policy_version 94962 (0.0010) [2023-10-14 21:36:47,621][61552] Updated weights for policy 0, policy_version 94972 (0.0008) [2023-10-14 21:36:48,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 194347008. Throughput: 0: 1671.6, 1: 1667.0. Samples: 48589080. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-14 21:36:48,344][60425] Avg episode reward: [(0, '80.420'), (1, '81.600')] [2023-10-14 21:36:48,965][61585] Updated weights for policy 1, policy_version 94820 (0.0009) [2023-10-14 21:36:49,332][61585] Updated weights for policy 1, policy_version 94830 (0.0009) [2023-10-14 21:36:49,694][61585] Updated weights for policy 1, policy_version 94840 (0.0009) [2023-10-14 21:36:51,799][61552] Updated weights for policy 0, policy_version 94982 (0.0008) [2023-10-14 21:36:52,184][61552] Updated weights for policy 0, policy_version 94992 (0.0008) [2023-10-14 21:36:52,556][61552] Updated weights for policy 0, policy_version 95002 (0.0007) [2023-10-14 21:36:53,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 194412544. Throughput: 0: 1668.4, 1: 1668.2. Samples: 48609350. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-14 21:36:53,344][60425] Avg episode reward: [(0, '80.240'), (1, '85.400')] [2023-10-14 21:36:53,730][61585] Updated weights for policy 1, policy_version 94850 (0.0008) [2023-10-14 21:36:54,093][61585] Updated weights for policy 1, policy_version 94860 (0.0009) [2023-10-14 21:36:54,456][61585] Updated weights for policy 1, policy_version 94870 (0.0008) [2023-10-14 21:36:54,821][61585] Updated weights for policy 1, policy_version 94880 (0.0010) [2023-10-14 21:36:56,613][61552] Updated weights for policy 0, policy_version 95012 (0.0009) [2023-10-14 21:36:56,991][61552] Updated weights for policy 0, policy_version 95022 (0.0008) [2023-10-14 21:36:57,354][61552] Updated weights for policy 0, policy_version 95032 (0.0008) [2023-10-14 21:36:58,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 194478080. Throughput: 0: 1655.2, 1: 1664.9. Samples: 48629016. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-14 21:36:58,345][60425] Avg episode reward: [(0, '80.300'), (1, '85.140')] [2023-10-14 21:36:59,021][61585] Updated weights for policy 1, policy_version 94890 (0.0009) [2023-10-14 21:36:59,384][61585] Updated weights for policy 1, policy_version 94900 (0.0009) [2023-10-14 21:36:59,749][61585] Updated weights for policy 1, policy_version 94910 (0.0009) [2023-10-14 21:37:01,346][61552] Updated weights for policy 0, policy_version 95042 (0.0008) [2023-10-14 21:37:01,708][61552] Updated weights for policy 0, policy_version 95052 (0.0010) [2023-10-14 21:37:02,078][61552] Updated weights for policy 0, policy_version 95062 (0.0009) [2023-10-14 21:37:02,448][61552] Updated weights for policy 0, policy_version 95072 (0.0010) [2023-10-14 21:37:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 194543616. Throughput: 0: 1677.7, 1: 1666.1. Samples: 48639406. Policy #0 lag: (min: 18.0, avg: 23.6, max: 50.0) [2023-10-14 21:37:03,344][60425] Avg episode reward: [(0, '78.420'), (1, '84.610')] [2023-10-14 21:37:03,910][61585] Updated weights for policy 1, policy_version 94920 (0.0009) [2023-10-14 21:37:04,280][61585] Updated weights for policy 1, policy_version 94930 (0.0009) [2023-10-14 21:37:04,651][61585] Updated weights for policy 1, policy_version 94940 (0.0008) [2023-10-14 21:37:06,642][61552] Updated weights for policy 0, policy_version 95082 (0.0009) [2023-10-14 21:37:07,013][61552] Updated weights for policy 0, policy_version 95092 (0.0011) [2023-10-14 21:37:07,389][61552] Updated weights for policy 0, policy_version 95102 (0.0008) [2023-10-14 21:37:08,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 194609152. Throughput: 0: 1670.3, 1: 1679.9. Samples: 48659840. Policy #0 lag: (min: 18.0, avg: 23.6, max: 50.0) [2023-10-14 21:37:08,344][60425] Avg episode reward: [(0, '80.810'), (1, '81.390')] [2023-10-14 21:37:08,703][61585] Updated weights for policy 1, policy_version 94950 (0.0008) [2023-10-14 21:37:09,074][61585] Updated weights for policy 1, policy_version 94960 (0.0008) [2023-10-14 21:37:09,450][61585] Updated weights for policy 1, policy_version 94970 (0.0010) [2023-10-14 21:37:11,520][61552] Updated weights for policy 0, policy_version 95112 (0.0010) [2023-10-14 21:37:11,887][61552] Updated weights for policy 0, policy_version 95122 (0.0009) [2023-10-14 21:37:12,255][61552] Updated weights for policy 0, policy_version 95132 (0.0010) [2023-10-14 21:37:13,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 194674688. Throughput: 0: 1664.0, 1: 1682.9. Samples: 48679558. Policy #0 lag: (min: 18.0, avg: 23.6, max: 50.0) [2023-10-14 21:37:13,344][60425] Avg episode reward: [(0, '79.690'), (1, '82.690')] [2023-10-14 21:37:13,507][61585] Updated weights for policy 1, policy_version 94980 (0.0008) [2023-10-14 21:37:13,879][61585] Updated weights for policy 1, policy_version 94990 (0.0007) [2023-10-14 21:37:14,239][61585] Updated weights for policy 1, policy_version 95000 (0.0007) [2023-10-14 21:37:16,248][61552] Updated weights for policy 0, policy_version 95142 (0.0008) [2023-10-14 21:37:16,612][61552] Updated weights for policy 0, policy_version 95152 (0.0010) [2023-10-14 21:37:16,981][61552] Updated weights for policy 0, policy_version 95162 (0.0012) [2023-10-14 21:37:18,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 194740224. Throughput: 0: 1684.2, 1: 1678.3. Samples: 48689950. Policy #0 lag: (min: 18.0, avg: 23.6, max: 50.0) [2023-10-14 21:37:18,344][60425] Avg episode reward: [(0, '83.420'), (1, '84.070')] [2023-10-14 21:37:18,497][61585] Updated weights for policy 1, policy_version 95010 (0.0009) [2023-10-14 21:37:18,873][61585] Updated weights for policy 1, policy_version 95020 (0.0007) [2023-10-14 21:37:19,246][61585] Updated weights for policy 1, policy_version 95030 (0.0007) [2023-10-14 21:37:19,601][61585] Updated weights for policy 1, policy_version 95040 (0.0008) [2023-10-14 21:37:21,043][61552] Updated weights for policy 0, policy_version 95172 (0.0010) [2023-10-14 21:37:21,417][61552] Updated weights for policy 0, policy_version 95182 (0.0009) [2023-10-14 21:37:21,784][61552] Updated weights for policy 0, policy_version 95192 (0.0008) [2023-10-14 21:37:23,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13329.4). Total num frames: 194805760. Throughput: 0: 1662.8, 1: 1676.9. Samples: 48709722. Policy #0 lag: (min: 18.0, avg: 23.6, max: 50.0) [2023-10-14 21:37:23,344][60425] Avg episode reward: [(0, '82.190'), (1, '82.880')] [2023-10-14 21:37:23,458][61585] Updated weights for policy 1, policy_version 95050 (0.0008) [2023-10-14 21:37:23,824][61585] Updated weights for policy 1, policy_version 95060 (0.0009) [2023-10-14 21:37:24,200][61585] Updated weights for policy 1, policy_version 95070 (0.0009) [2023-10-14 21:37:25,905][61552] Updated weights for policy 0, policy_version 95202 (0.0009) [2023-10-14 21:37:26,277][61552] Updated weights for policy 0, policy_version 95212 (0.0010) [2023-10-14 21:37:26,642][61552] Updated weights for policy 0, policy_version 95222 (0.0010) [2023-10-14 21:37:27,007][61552] Updated weights for policy 0, policy_version 95232 (0.0009) [2023-10-14 21:37:28,299][61585] Updated weights for policy 1, policy_version 95080 (0.0007) [2023-10-14 21:37:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 194871296. Throughput: 0: 1674.3, 1: 1675.7. Samples: 48729806. Policy #0 lag: (min: 18.0, avg: 23.6, max: 50.0) [2023-10-14 21:37:28,344][60425] Avg episode reward: [(0, '78.540'), (1, '80.190')] [2023-10-14 21:37:28,665][61585] Updated weights for policy 1, policy_version 95090 (0.0007) [2023-10-14 21:37:29,024][61585] Updated weights for policy 1, policy_version 95100 (0.0009) [2023-10-14 21:37:30,994][61552] Updated weights for policy 0, policy_version 95242 (0.0007) [2023-10-14 21:37:31,363][61552] Updated weights for policy 0, policy_version 95252 (0.0008) [2023-10-14 21:37:31,736][61552] Updated weights for policy 0, policy_version 95262 (0.0008) [2023-10-14 21:37:33,078][61585] Updated weights for policy 1, policy_version 95110 (0.0008) [2023-10-14 21:37:33,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 194936832. Throughput: 0: 1689.2, 1: 1677.0. Samples: 48740560. Policy #0 lag: (min: 18.0, avg: 23.6, max: 50.0) [2023-10-14 21:37:33,344][60425] Avg episode reward: [(0, '85.840'), (1, '85.580')] [2023-10-14 21:37:33,444][61585] Updated weights for policy 1, policy_version 95120 (0.0008) [2023-10-14 21:37:33,813][61585] Updated weights for policy 1, policy_version 95130 (0.0010) [2023-10-14 21:37:35,662][61552] Updated weights for policy 0, policy_version 95272 (0.0009) [2023-10-14 21:37:36,030][61552] Updated weights for policy 0, policy_version 95282 (0.0008) [2023-10-14 21:37:36,393][61552] Updated weights for policy 0, policy_version 95292 (0.0010) [2023-10-14 21:37:37,984][61585] Updated weights for policy 1, policy_version 95140 (0.0009) [2023-10-14 21:37:38,343][61585] Updated weights for policy 1, policy_version 95150 (0.0009) [2023-10-14 21:37:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 195002368. Throughput: 0: 1668.7, 1: 1678.4. Samples: 48759970. Policy #0 lag: (min: 18.0, avg: 23.6, max: 50.0) [2023-10-14 21:37:38,344][60425] Avg episode reward: [(0, '81.430'), (1, '80.670')] [2023-10-14 21:37:38,699][61585] Updated weights for policy 1, policy_version 95160 (0.0007) [2023-10-14 21:37:40,596][61552] Updated weights for policy 0, policy_version 95302 (0.0009) [2023-10-14 21:37:40,973][61552] Updated weights for policy 0, policy_version 95312 (0.0007) [2023-10-14 21:37:41,338][61552] Updated weights for policy 0, policy_version 95322 (0.0009) [2023-10-14 21:37:42,732][61585] Updated weights for policy 1, policy_version 95170 (0.0008) [2023-10-14 21:37:43,106][61585] Updated weights for policy 1, policy_version 95180 (0.0008) [2023-10-14 21:37:43,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 195067904. Throughput: 0: 1688.3, 1: 1677.2. Samples: 48780460. Policy #0 lag: (min: 18.0, avg: 23.6, max: 50.0) [2023-10-14 21:37:43,344][60425] Avg episode reward: [(0, '79.250'), (1, '82.740')] [2023-10-14 21:37:43,356][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000095328_97615872.pth... [2023-10-14 21:37:43,392][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000093760_96010240.pth [2023-10-14 21:37:43,466][61585] Updated weights for policy 1, policy_version 95190 (0.0007) [2023-10-14 21:37:43,820][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000095200_97484800.pth... [2023-10-14 21:37:43,822][61585] Updated weights for policy 1, policy_version 95200 (0.0008) [2023-10-14 21:37:43,851][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000093632_95879168.pth [2023-10-14 21:37:45,410][61552] Updated weights for policy 0, policy_version 95332 (0.0007) [2023-10-14 21:37:45,782][61552] Updated weights for policy 0, policy_version 95342 (0.0009) [2023-10-14 21:37:46,155][61552] Updated weights for policy 0, policy_version 95352 (0.0007) [2023-10-14 21:37:47,988][61585] Updated weights for policy 1, policy_version 95210 (0.0007) [2023-10-14 21:37:48,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 195133440. Throughput: 0: 1674.1, 1: 1679.9. Samples: 48790336. Policy #0 lag: (min: 18.0, avg: 23.6, max: 50.0) [2023-10-14 21:37:48,344][60425] Avg episode reward: [(0, '76.750'), (1, '86.120')] [2023-10-14 21:37:48,350][61585] Updated weights for policy 1, policy_version 95220 (0.0007) [2023-10-14 21:37:48,710][61585] Updated weights for policy 1, policy_version 95230 (0.0008) [2023-10-14 21:37:50,289][61552] Updated weights for policy 0, policy_version 95362 (0.0010) [2023-10-14 21:37:50,656][61552] Updated weights for policy 0, policy_version 95372 (0.0008) [2023-10-14 21:37:51,027][61552] Updated weights for policy 0, policy_version 95382 (0.0007) [2023-10-14 21:37:51,398][61552] Updated weights for policy 0, policy_version 95392 (0.0008) [2023-10-14 21:37:52,922][61585] Updated weights for policy 1, policy_version 95240 (0.0007) [2023-10-14 21:37:53,304][61585] Updated weights for policy 1, policy_version 95250 (0.0008) [2023-10-14 21:37:53,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 195198976. Throughput: 0: 1662.8, 1: 1674.0. Samples: 48809994. Policy #0 lag: (min: 18.0, avg: 23.6, max: 50.0) [2023-10-14 21:37:53,344][60425] Avg episode reward: [(0, '76.290'), (1, '80.860')] [2023-10-14 21:37:53,670][61585] Updated weights for policy 1, policy_version 95260 (0.0008) [2023-10-14 21:37:55,249][61552] Updated weights for policy 0, policy_version 95402 (0.0007) [2023-10-14 21:37:55,615][61552] Updated weights for policy 0, policy_version 95412 (0.0007) [2023-10-14 21:37:55,982][61552] Updated weights for policy 0, policy_version 95422 (0.0009) [2023-10-14 21:37:57,556][61585] Updated weights for policy 1, policy_version 95270 (0.0010) [2023-10-14 21:37:57,935][61585] Updated weights for policy 1, policy_version 95280 (0.0009) [2023-10-14 21:37:58,297][61585] Updated weights for policy 1, policy_version 95290 (0.0008) [2023-10-14 21:37:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.3, 300 sec: 13329.4). Total num frames: 195264512. Throughput: 0: 1687.7, 1: 1663.7. Samples: 48830370. Policy #0 lag: (min: 18.0, avg: 23.6, max: 50.0) [2023-10-14 21:37:58,344][60425] Avg episode reward: [(0, '75.900'), (1, '83.920')] [2023-10-14 21:37:59,838][61552] Updated weights for policy 0, policy_version 95432 (0.0007) [2023-10-14 21:38:00,198][61552] Updated weights for policy 0, policy_version 95442 (0.0007) [2023-10-14 21:38:00,567][61552] Updated weights for policy 0, policy_version 95452 (0.0008) [2023-10-14 21:38:02,453][61585] Updated weights for policy 1, policy_version 95300 (0.0008) [2023-10-14 21:38:02,823][61585] Updated weights for policy 1, policy_version 95310 (0.0008) [2023-10-14 21:38:03,188][61585] Updated weights for policy 1, policy_version 95320 (0.0007) [2023-10-14 21:38:03,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 195330048. Throughput: 0: 1661.7, 1: 1679.1. Samples: 48840286. Policy #0 lag: (min: 12.0, avg: 19.6, max: 44.0) [2023-10-14 21:38:03,344][60425] Avg episode reward: [(0, '77.360'), (1, '79.820')] [2023-10-14 21:38:04,626][61552] Updated weights for policy 0, policy_version 95462 (0.0009) [2023-10-14 21:38:04,994][61552] Updated weights for policy 0, policy_version 95472 (0.0011) [2023-10-14 21:38:05,364][61552] Updated weights for policy 0, policy_version 95482 (0.0011) [2023-10-14 21:38:07,223][61585] Updated weights for policy 1, policy_version 95330 (0.0007) [2023-10-14 21:38:07,586][61585] Updated weights for policy 1, policy_version 95340 (0.0007) [2023-10-14 21:38:07,951][61585] Updated weights for policy 1, policy_version 95350 (0.0008) [2023-10-14 21:38:08,307][61585] Updated weights for policy 1, policy_version 95360 (0.0008) [2023-10-14 21:38:08,343][60425] Fps is (10 sec: 16383.9, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 195428352. Throughput: 0: 1677.9, 1: 1685.7. Samples: 48861084. Policy #0 lag: (min: 12.0, avg: 19.6, max: 44.0) [2023-10-14 21:38:08,344][60425] Avg episode reward: [(0, '73.480'), (1, '84.800')] [2023-10-14 21:38:09,487][61552] Updated weights for policy 0, policy_version 95492 (0.0010) [2023-10-14 21:38:09,853][61552] Updated weights for policy 0, policy_version 95502 (0.0008) [2023-10-14 21:38:10,224][61552] Updated weights for policy 0, policy_version 95512 (0.0008) [2023-10-14 21:38:12,303][61585] Updated weights for policy 1, policy_version 95370 (0.0009) [2023-10-14 21:38:12,676][61585] Updated weights for policy 1, policy_version 95380 (0.0009) [2023-10-14 21:38:13,041][61585] Updated weights for policy 1, policy_version 95390 (0.0008) [2023-10-14 21:38:13,343][60425] Fps is (10 sec: 16383.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 195493888. Throughput: 0: 1694.2, 1: 1669.0. Samples: 48881150. Policy #0 lag: (min: 12.0, avg: 19.6, max: 44.0) [2023-10-14 21:38:13,344][60425] Avg episode reward: [(0, '75.940'), (1, '81.160')] [2023-10-14 21:38:14,264][61552] Updated weights for policy 0, policy_version 95522 (0.0008) [2023-10-14 21:38:14,643][61552] Updated weights for policy 0, policy_version 95532 (0.0008) [2023-10-14 21:38:15,015][61552] Updated weights for policy 0, policy_version 95542 (0.0008) [2023-10-14 21:38:15,375][61552] Updated weights for policy 0, policy_version 95552 (0.0008) [2023-10-14 21:38:17,114][61585] Updated weights for policy 1, policy_version 95400 (0.0009) [2023-10-14 21:38:17,483][61585] Updated weights for policy 1, policy_version 95410 (0.0010) [2023-10-14 21:38:17,851][61585] Updated weights for policy 1, policy_version 95420 (0.0009) [2023-10-14 21:38:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 195559424. Throughput: 0: 1658.6, 1: 1684.4. Samples: 48890996. Policy #0 lag: (min: 12.0, avg: 19.6, max: 44.0) [2023-10-14 21:38:18,344][60425] Avg episode reward: [(0, '78.900'), (1, '87.430')] [2023-10-14 21:38:18,345][61248] Saving new best policy, reward=87.430! [2023-10-14 21:38:19,413][61552] Updated weights for policy 0, policy_version 95562 (0.0009) [2023-10-14 21:38:19,776][61552] Updated weights for policy 0, policy_version 95572 (0.0007) [2023-10-14 21:38:20,135][61552] Updated weights for policy 0, policy_version 95582 (0.0007) [2023-10-14 21:38:21,884][61585] Updated weights for policy 1, policy_version 95430 (0.0009) [2023-10-14 21:38:22,248][61585] Updated weights for policy 1, policy_version 95440 (0.0008) [2023-10-14 21:38:22,606][61585] Updated weights for policy 1, policy_version 95450 (0.0008) [2023-10-14 21:38:23,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 195624960. Throughput: 0: 1685.7, 1: 1683.2. Samples: 48911570. Policy #0 lag: (min: 12.0, avg: 19.6, max: 44.0) [2023-10-14 21:38:23,344][60425] Avg episode reward: [(0, '77.620'), (1, '83.560')] [2023-10-14 21:38:24,391][61552] Updated weights for policy 0, policy_version 95592 (0.0007) [2023-10-14 21:38:24,761][61552] Updated weights for policy 0, policy_version 95602 (0.0010) [2023-10-14 21:38:25,128][61552] Updated weights for policy 0, policy_version 95612 (0.0010) [2023-10-14 21:38:26,675][61585] Updated weights for policy 1, policy_version 95460 (0.0009) [2023-10-14 21:38:27,039][61585] Updated weights for policy 1, policy_version 95470 (0.0007) [2023-10-14 21:38:27,408][61585] Updated weights for policy 1, policy_version 95480 (0.0007) [2023-10-14 21:38:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 195690496. Throughput: 0: 1683.2, 1: 1664.4. Samples: 48931102. Policy #0 lag: (min: 12.0, avg: 19.6, max: 44.0) [2023-10-14 21:38:28,344][60425] Avg episode reward: [(0, '77.610'), (1, '84.810')] [2023-10-14 21:38:29,212][61552] Updated weights for policy 0, policy_version 95622 (0.0009) [2023-10-14 21:38:29,589][61552] Updated weights for policy 0, policy_version 95632 (0.0007) [2023-10-14 21:38:29,960][61552] Updated weights for policy 0, policy_version 95642 (0.0007) [2023-10-14 21:38:31,614][61585] Updated weights for policy 1, policy_version 95490 (0.0008) [2023-10-14 21:38:31,983][61585] Updated weights for policy 1, policy_version 95500 (0.0007) [2023-10-14 21:38:32,349][61585] Updated weights for policy 1, policy_version 95510 (0.0008) [2023-10-14 21:38:32,716][61585] Updated weights for policy 1, policy_version 95520 (0.0009) [2023-10-14 21:38:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 195756032. Throughput: 0: 1669.6, 1: 1686.6. Samples: 48941366. Policy #0 lag: (min: 12.0, avg: 19.6, max: 44.0) [2023-10-14 21:38:33,344][60425] Avg episode reward: [(0, '76.500'), (1, '85.040')] [2023-10-14 21:38:34,012][61552] Updated weights for policy 0, policy_version 95652 (0.0008) [2023-10-14 21:38:34,375][61552] Updated weights for policy 0, policy_version 95662 (0.0007) [2023-10-14 21:38:34,747][61552] Updated weights for policy 0, policy_version 95672 (0.0008) [2023-10-14 21:38:36,864][61585] Updated weights for policy 1, policy_version 95530 (0.0010) [2023-10-14 21:38:37,239][61585] Updated weights for policy 1, policy_version 95540 (0.0008) [2023-10-14 21:38:37,602][61585] Updated weights for policy 1, policy_version 95550 (0.0009) [2023-10-14 21:38:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 195821568. Throughput: 0: 1690.9, 1: 1678.5. Samples: 48961618. Policy #0 lag: (min: 12.0, avg: 19.6, max: 44.0) [2023-10-14 21:38:38,344][60425] Avg episode reward: [(0, '82.300'), (1, '78.120')] [2023-10-14 21:38:38,738][61552] Updated weights for policy 0, policy_version 95682 (0.0009) [2023-10-14 21:38:39,109][61552] Updated weights for policy 0, policy_version 95692 (0.0007) [2023-10-14 21:38:39,474][61552] Updated weights for policy 0, policy_version 95702 (0.0008) [2023-10-14 21:38:39,840][61552] Updated weights for policy 0, policy_version 95712 (0.0010) [2023-10-14 21:38:41,872][61585] Updated weights for policy 1, policy_version 95560 (0.0009) [2023-10-14 21:38:42,254][61585] Updated weights for policy 1, policy_version 95570 (0.0009) [2023-10-14 21:38:42,615][61585] Updated weights for policy 1, policy_version 95580 (0.0007) [2023-10-14 21:38:43,343][60425] Fps is (10 sec: 13106.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 195887104. Throughput: 0: 1687.4, 1: 1662.1. Samples: 48981096. Policy #0 lag: (min: 12.0, avg: 19.6, max: 44.0) [2023-10-14 21:38:43,345][60425] Avg episode reward: [(0, '75.390'), (1, '80.100')] [2023-10-14 21:38:43,963][61552] Updated weights for policy 0, policy_version 95722 (0.0010) [2023-10-14 21:38:44,337][61552] Updated weights for policy 0, policy_version 95732 (0.0011) [2023-10-14 21:38:44,709][61552] Updated weights for policy 0, policy_version 95742 (0.0010) [2023-10-14 21:38:46,567][61585] Updated weights for policy 1, policy_version 95590 (0.0010) [2023-10-14 21:38:46,930][61585] Updated weights for policy 1, policy_version 95600 (0.0009) [2023-10-14 21:38:47,301][61585] Updated weights for policy 1, policy_version 95610 (0.0009) [2023-10-14 21:38:48,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 195952640. Throughput: 0: 1678.3, 1: 1676.4. Samples: 48991250. Policy #0 lag: (min: 12.0, avg: 19.6, max: 44.0) [2023-10-14 21:38:48,344][60425] Avg episode reward: [(0, '78.020'), (1, '78.460')] [2023-10-14 21:38:48,910][61552] Updated weights for policy 0, policy_version 95752 (0.0007) [2023-10-14 21:38:49,265][61552] Updated weights for policy 0, policy_version 95762 (0.0009) [2023-10-14 21:38:49,641][61552] Updated weights for policy 0, policy_version 95772 (0.0009) [2023-10-14 21:38:51,564][61585] Updated weights for policy 1, policy_version 95620 (0.0008) [2023-10-14 21:38:51,933][61585] Updated weights for policy 1, policy_version 95630 (0.0007) [2023-10-14 21:38:52,296][61585] Updated weights for policy 1, policy_version 95640 (0.0008) [2023-10-14 21:38:53,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 196018176. Throughput: 0: 1681.8, 1: 1661.9. Samples: 49011548. Policy #0 lag: (min: 12.0, avg: 19.6, max: 44.0) [2023-10-14 21:38:53,344][60425] Avg episode reward: [(0, '76.690'), (1, '79.960')] [2023-10-14 21:38:53,718][61552] Updated weights for policy 0, policy_version 95782 (0.0007) [2023-10-14 21:38:54,089][61552] Updated weights for policy 0, policy_version 95792 (0.0008) [2023-10-14 21:38:54,456][61552] Updated weights for policy 0, policy_version 95802 (0.0007) [2023-10-14 21:38:56,379][61585] Updated weights for policy 1, policy_version 95650 (0.0007) [2023-10-14 21:38:56,751][61585] Updated weights for policy 1, policy_version 95660 (0.0007) [2023-10-14 21:38:57,108][61585] Updated weights for policy 1, policy_version 95670 (0.0007) [2023-10-14 21:38:57,465][61585] Updated weights for policy 1, policy_version 95680 (0.0010) [2023-10-14 21:38:58,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 196083712. Throughput: 0: 1683.6, 1: 1657.9. Samples: 49031516. Policy #0 lag: (min: 12.0, avg: 19.6, max: 44.0) [2023-10-14 21:38:58,344][60425] Avg episode reward: [(0, '79.070'), (1, '75.930')] [2023-10-14 21:38:58,520][61552] Updated weights for policy 0, policy_version 95812 (0.0010) [2023-10-14 21:38:58,900][61552] Updated weights for policy 0, policy_version 95822 (0.0008) [2023-10-14 21:38:59,255][61552] Updated weights for policy 0, policy_version 95832 (0.0007) [2023-10-14 21:39:01,588][61585] Updated weights for policy 1, policy_version 95690 (0.0008) [2023-10-14 21:39:01,957][61585] Updated weights for policy 1, policy_version 95700 (0.0007) [2023-10-14 21:39:02,316][61585] Updated weights for policy 1, policy_version 95710 (0.0007) [2023-10-14 21:39:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13329.4). Total num frames: 196149248. Throughput: 0: 1681.5, 1: 1666.4. Samples: 49041650. Policy #0 lag: (min: 11.0, avg: 19.0, max: 43.0) [2023-10-14 21:39:03,344][60425] Avg episode reward: [(0, '74.740'), (1, '79.050')] [2023-10-14 21:39:03,347][61552] Updated weights for policy 0, policy_version 95842 (0.0008) [2023-10-14 21:39:03,719][61552] Updated weights for policy 0, policy_version 95852 (0.0007) [2023-10-14 21:39:04,075][61552] Updated weights for policy 0, policy_version 95862 (0.0008) [2023-10-14 21:39:04,445][61552] Updated weights for policy 0, policy_version 95872 (0.0007) [2023-10-14 21:39:06,431][61585] Updated weights for policy 1, policy_version 95720 (0.0009) [2023-10-14 21:39:06,792][61585] Updated weights for policy 1, policy_version 95730 (0.0007) [2023-10-14 21:39:07,161][61585] Updated weights for policy 1, policy_version 95740 (0.0009) [2023-10-14 21:39:08,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 196214784. Throughput: 0: 1684.0, 1: 1657.1. Samples: 49061924. Policy #0 lag: (min: 11.0, avg: 19.0, max: 43.0) [2023-10-14 21:39:08,344][60425] Avg episode reward: [(0, '77.810'), (1, '79.130')] [2023-10-14 21:39:08,383][61552] Updated weights for policy 0, policy_version 95882 (0.0010) [2023-10-14 21:39:08,758][61552] Updated weights for policy 0, policy_version 95892 (0.0011) [2023-10-14 21:39:09,126][61552] Updated weights for policy 0, policy_version 95902 (0.0011) [2023-10-14 21:39:11,309][61585] Updated weights for policy 1, policy_version 95750 (0.0009) [2023-10-14 21:39:11,664][61585] Updated weights for policy 1, policy_version 95760 (0.0007) [2023-10-14 21:39:12,035][61585] Updated weights for policy 1, policy_version 95770 (0.0008) [2023-10-14 21:39:13,215][61552] Updated weights for policy 0, policy_version 95912 (0.0009) [2023-10-14 21:39:13,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 196280320. Throughput: 0: 1690.4, 1: 1663.0. Samples: 49082008. Policy #0 lag: (min: 11.0, avg: 19.0, max: 43.0) [2023-10-14 21:39:13,344][60425] Avg episode reward: [(0, '80.880'), (1, '80.150')] [2023-10-14 21:39:13,589][61552] Updated weights for policy 0, policy_version 95922 (0.0010) [2023-10-14 21:39:13,954][61552] Updated weights for policy 0, policy_version 95932 (0.0008) [2023-10-14 21:39:16,158][61585] Updated weights for policy 1, policy_version 95780 (0.0010) [2023-10-14 21:39:16,522][61585] Updated weights for policy 1, policy_version 95790 (0.0009) [2023-10-14 21:39:16,884][61585] Updated weights for policy 1, policy_version 95800 (0.0007) [2023-10-14 21:39:18,121][61552] Updated weights for policy 0, policy_version 95942 (0.0008) [2023-10-14 21:39:18,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 196345856. Throughput: 0: 1687.1, 1: 1665.4. Samples: 49092226. Policy #0 lag: (min: 11.0, avg: 19.0, max: 43.0) [2023-10-14 21:39:18,344][60425] Avg episode reward: [(0, '74.560'), (1, '79.800')] [2023-10-14 21:39:18,488][61552] Updated weights for policy 0, policy_version 95952 (0.0009) [2023-10-14 21:39:18,853][61552] Updated weights for policy 0, policy_version 95962 (0.0009) [2023-10-14 21:39:20,980][61585] Updated weights for policy 1, policy_version 95810 (0.0008) [2023-10-14 21:39:21,337][61585] Updated weights for policy 1, policy_version 95820 (0.0007) [2023-10-14 21:39:21,709][61585] Updated weights for policy 1, policy_version 95830 (0.0008) [2023-10-14 21:39:22,077][61585] Updated weights for policy 1, policy_version 95840 (0.0009) [2023-10-14 21:39:22,962][61552] Updated weights for policy 0, policy_version 95972 (0.0011) [2023-10-14 21:39:23,322][61552] Updated weights for policy 0, policy_version 95982 (0.0010) [2023-10-14 21:39:23,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 196411392. Throughput: 0: 1682.9, 1: 1653.1. Samples: 49111738. Policy #0 lag: (min: 11.0, avg: 19.0, max: 43.0) [2023-10-14 21:39:23,344][60425] Avg episode reward: [(0, '77.090'), (1, '85.060')] [2023-10-14 21:39:23,692][61552] Updated weights for policy 0, policy_version 95992 (0.0008) [2023-10-14 21:39:25,994][61585] Updated weights for policy 1, policy_version 95850 (0.0007) [2023-10-14 21:39:26,361][61585] Updated weights for policy 1, policy_version 95860 (0.0007) [2023-10-14 21:39:26,715][61585] Updated weights for policy 1, policy_version 95870 (0.0009) [2023-10-14 21:39:27,850][61552] Updated weights for policy 0, policy_version 96002 (0.0010) [2023-10-14 21:39:28,214][61552] Updated weights for policy 0, policy_version 96012 (0.0010) [2023-10-14 21:39:28,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 196476928. Throughput: 0: 1680.2, 1: 1680.3. Samples: 49132320. Policy #0 lag: (min: 11.0, avg: 19.0, max: 43.0) [2023-10-14 21:39:28,345][60425] Avg episode reward: [(0, '77.090'), (1, '80.360')] [2023-10-14 21:39:28,583][61552] Updated weights for policy 0, policy_version 96022 (0.0011) [2023-10-14 21:39:28,952][61552] Updated weights for policy 0, policy_version 96032 (0.0010) [2023-10-14 21:39:30,636][61585] Updated weights for policy 1, policy_version 95880 (0.0008) [2023-10-14 21:39:31,027][61585] Updated weights for policy 1, policy_version 95890 (0.0010) [2023-10-14 21:39:31,395][61585] Updated weights for policy 1, policy_version 95900 (0.0008) [2023-10-14 21:39:33,039][61552] Updated weights for policy 0, policy_version 96042 (0.0007) [2023-10-14 21:39:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 196542464. Throughput: 0: 1681.2, 1: 1671.5. Samples: 49142122. Policy #0 lag: (min: 11.0, avg: 19.0, max: 43.0) [2023-10-14 21:39:33,344][60425] Avg episode reward: [(0, '78.990'), (1, '86.170')] [2023-10-14 21:39:33,407][61552] Updated weights for policy 0, policy_version 96052 (0.0010) [2023-10-14 21:39:33,777][61552] Updated weights for policy 0, policy_version 96062 (0.0011) [2023-10-14 21:39:35,489][61585] Updated weights for policy 1, policy_version 95910 (0.0010) [2023-10-14 21:39:35,864][61585] Updated weights for policy 1, policy_version 95920 (0.0008) [2023-10-14 21:39:36,219][61585] Updated weights for policy 1, policy_version 95930 (0.0009) [2023-10-14 21:39:37,991][61552] Updated weights for policy 0, policy_version 96072 (0.0008) [2023-10-14 21:39:38,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 196608000. Throughput: 0: 1680.7, 1: 1664.2. Samples: 49162072. Policy #0 lag: (min: 11.0, avg: 19.0, max: 43.0) [2023-10-14 21:39:38,345][60425] Avg episode reward: [(0, '80.350'), (1, '82.910')] [2023-10-14 21:39:38,362][61552] Updated weights for policy 0, policy_version 96082 (0.0007) [2023-10-14 21:39:38,739][61552] Updated weights for policy 0, policy_version 96092 (0.0008) [2023-10-14 21:39:40,314][61585] Updated weights for policy 1, policy_version 95940 (0.0008) [2023-10-14 21:39:40,686][61585] Updated weights for policy 1, policy_version 95950 (0.0008) [2023-10-14 21:39:41,063][61585] Updated weights for policy 1, policy_version 95960 (0.0009) [2023-10-14 21:39:42,778][61552] Updated weights for policy 0, policy_version 96102 (0.0008) [2023-10-14 21:39:43,146][61552] Updated weights for policy 0, policy_version 96112 (0.0008) [2023-10-14 21:39:43,344][60425] Fps is (10 sec: 13106.5, 60 sec: 13107.1, 300 sec: 13329.3). Total num frames: 196673536. Throughput: 0: 1670.6, 1: 1686.5. Samples: 49182586. Policy #0 lag: (min: 11.0, avg: 19.0, max: 43.0) [2023-10-14 21:39:43,345][60425] Avg episode reward: [(0, '80.230'), (1, '85.240')] [2023-10-14 21:39:43,357][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000095968_98271232.pth... [2023-10-14 21:39:43,390][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000094400_96665600.pth [2023-10-14 21:39:43,519][61552] Updated weights for policy 0, policy_version 96122 (0.0008) [2023-10-14 21:39:43,741][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000096128_98435072.pth... [2023-10-14 21:39:43,788][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000094560_96829440.pth [2023-10-14 21:39:45,286][61585] Updated weights for policy 1, policy_version 95970 (0.0009) [2023-10-14 21:39:45,641][61585] Updated weights for policy 1, policy_version 95980 (0.0011) [2023-10-14 21:39:46,011][61585] Updated weights for policy 1, policy_version 95990 (0.0009) [2023-10-14 21:39:46,370][61585] Updated weights for policy 1, policy_version 96000 (0.0011) [2023-10-14 21:39:47,631][61552] Updated weights for policy 0, policy_version 96132 (0.0009) [2023-10-14 21:39:48,000][61552] Updated weights for policy 0, policy_version 96142 (0.0009) [2023-10-14 21:39:48,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 196739072. Throughput: 0: 1675.3, 1: 1675.0. Samples: 49192416. Policy #0 lag: (min: 11.0, avg: 19.0, max: 43.0) [2023-10-14 21:39:48,344][60425] Avg episode reward: [(0, '80.960'), (1, '80.370')] [2023-10-14 21:39:48,374][61552] Updated weights for policy 0, policy_version 96152 (0.0010) [2023-10-14 21:39:50,534][61585] Updated weights for policy 1, policy_version 96010 (0.0008) [2023-10-14 21:39:50,901][61585] Updated weights for policy 1, policy_version 96020 (0.0009) [2023-10-14 21:39:51,265][61585] Updated weights for policy 1, policy_version 96030 (0.0009) [2023-10-14 21:39:52,451][61552] Updated weights for policy 0, policy_version 96162 (0.0008) [2023-10-14 21:39:52,812][61552] Updated weights for policy 0, policy_version 96172 (0.0008) [2023-10-14 21:39:53,186][61552] Updated weights for policy 0, policy_version 96182 (0.0008) [2023-10-14 21:39:53,343][60425] Fps is (10 sec: 13107.9, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 196804608. Throughput: 0: 1674.3, 1: 1666.1. Samples: 49212242. Policy #0 lag: (min: 11.0, avg: 19.0, max: 43.0) [2023-10-14 21:39:53,344][60425] Avg episode reward: [(0, '79.850'), (1, '77.310')] [2023-10-14 21:39:53,560][61552] Updated weights for policy 0, policy_version 96192 (0.0008) [2023-10-14 21:39:55,302][61585] Updated weights for policy 1, policy_version 96040 (0.0008) [2023-10-14 21:39:55,659][61585] Updated weights for policy 1, policy_version 96050 (0.0008) [2023-10-14 21:39:56,022][61585] Updated weights for policy 1, policy_version 96060 (0.0008) [2023-10-14 21:39:57,572][61552] Updated weights for policy 0, policy_version 96202 (0.0009) [2023-10-14 21:39:57,948][61552] Updated weights for policy 0, policy_version 96212 (0.0010) [2023-10-14 21:39:58,307][61552] Updated weights for policy 0, policy_version 96222 (0.0008) [2023-10-14 21:39:58,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 196870144. Throughput: 0: 1660.0, 1: 1684.3. Samples: 49232504. Policy #0 lag: (min: 11.0, avg: 19.0, max: 43.0) [2023-10-14 21:39:58,344][60425] Avg episode reward: [(0, '78.690'), (1, '81.670')] [2023-10-14 21:40:00,148][61585] Updated weights for policy 1, policy_version 96070 (0.0008) [2023-10-14 21:40:00,518][61585] Updated weights for policy 1, policy_version 96080 (0.0007) [2023-10-14 21:40:00,877][61585] Updated weights for policy 1, policy_version 96090 (0.0008) [2023-10-14 21:40:02,221][61552] Updated weights for policy 0, policy_version 96232 (0.0010) [2023-10-14 21:40:02,586][61552] Updated weights for policy 0, policy_version 96242 (0.0007) [2023-10-14 21:40:02,955][61552] Updated weights for policy 0, policy_version 96252 (0.0010) [2023-10-14 21:40:03,343][60425] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 196968448. Throughput: 0: 1674.5, 1: 1663.8. Samples: 49242452. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:40:03,344][60425] Avg episode reward: [(0, '76.110'), (1, '72.570')] [2023-10-14 21:40:04,968][61585] Updated weights for policy 1, policy_version 96100 (0.0008) [2023-10-14 21:40:05,329][61585] Updated weights for policy 1, policy_version 96110 (0.0008) [2023-10-14 21:40:05,688][61585] Updated weights for policy 1, policy_version 96120 (0.0007) [2023-10-14 21:40:07,148][61552] Updated weights for policy 0, policy_version 96262 (0.0010) [2023-10-14 21:40:07,520][61552] Updated weights for policy 0, policy_version 96272 (0.0009) [2023-10-14 21:40:07,892][61552] Updated weights for policy 0, policy_version 96282 (0.0008) [2023-10-14 21:40:08,343][60425] Fps is (10 sec: 16384.0, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 197033984. Throughput: 0: 1679.9, 1: 1674.8. Samples: 49262698. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:40:08,344][60425] Avg episode reward: [(0, '79.430'), (1, '77.460')] [2023-10-14 21:40:09,723][61585] Updated weights for policy 1, policy_version 96130 (0.0010) [2023-10-14 21:40:10,099][61585] Updated weights for policy 1, policy_version 96140 (0.0010) [2023-10-14 21:40:10,454][61585] Updated weights for policy 1, policy_version 96150 (0.0007) [2023-10-14 21:40:10,820][61585] Updated weights for policy 1, policy_version 96160 (0.0008) [2023-10-14 21:40:11,929][61552] Updated weights for policy 0, policy_version 96292 (0.0007) [2023-10-14 21:40:12,307][61552] Updated weights for policy 0, policy_version 96302 (0.0007) [2023-10-14 21:40:12,675][61552] Updated weights for policy 0, policy_version 96312 (0.0008) [2023-10-14 21:40:13,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 197099520. Throughput: 0: 1657.2, 1: 1677.4. Samples: 49282376. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:40:13,344][60425] Avg episode reward: [(0, '75.780'), (1, '73.830')] [2023-10-14 21:40:14,957][61585] Updated weights for policy 1, policy_version 96170 (0.0010) [2023-10-14 21:40:15,332][61585] Updated weights for policy 1, policy_version 96180 (0.0008) [2023-10-14 21:40:15,694][61585] Updated weights for policy 1, policy_version 96190 (0.0008) [2023-10-14 21:40:16,663][61552] Updated weights for policy 0, policy_version 96322 (0.0009) [2023-10-14 21:40:17,022][61552] Updated weights for policy 0, policy_version 96332 (0.0008) [2023-10-14 21:40:17,394][61552] Updated weights for policy 0, policy_version 96342 (0.0009) [2023-10-14 21:40:17,763][61552] Updated weights for policy 0, policy_version 96352 (0.0011) [2023-10-14 21:40:18,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 197165056. Throughput: 0: 1682.7, 1: 1661.6. Samples: 49292614. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:40:18,344][60425] Avg episode reward: [(0, '77.530'), (1, '82.320')] [2023-10-14 21:40:19,669][61585] Updated weights for policy 1, policy_version 96200 (0.0007) [2023-10-14 21:40:20,030][61585] Updated weights for policy 1, policy_version 96210 (0.0010) [2023-10-14 21:40:20,396][61585] Updated weights for policy 1, policy_version 96220 (0.0008) [2023-10-14 21:40:21,820][61552] Updated weights for policy 0, policy_version 96362 (0.0011) [2023-10-14 21:40:22,180][61552] Updated weights for policy 0, policy_version 96372 (0.0011) [2023-10-14 21:40:22,555][61552] Updated weights for policy 0, policy_version 96382 (0.0008) [2023-10-14 21:40:23,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 197230592. Throughput: 0: 1675.7, 1: 1675.1. Samples: 49312856. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:40:23,344][60425] Avg episode reward: [(0, '76.980'), (1, '77.540')] [2023-10-14 21:40:24,536][61585] Updated weights for policy 1, policy_version 96230 (0.0009) [2023-10-14 21:40:24,909][61585] Updated weights for policy 1, policy_version 96240 (0.0008) [2023-10-14 21:40:25,272][61585] Updated weights for policy 1, policy_version 96250 (0.0008) [2023-10-14 21:40:26,601][61552] Updated weights for policy 0, policy_version 96392 (0.0009) [2023-10-14 21:40:26,972][61552] Updated weights for policy 0, policy_version 96402 (0.0010) [2023-10-14 21:40:27,341][61552] Updated weights for policy 0, policy_version 96412 (0.0010) [2023-10-14 21:40:28,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 197296128. Throughput: 0: 1659.0, 1: 1674.2. Samples: 49332582. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:40:28,344][60425] Avg episode reward: [(0, '77.630'), (1, '80.620')] [2023-10-14 21:40:29,380][61585] Updated weights for policy 1, policy_version 96260 (0.0009) [2023-10-14 21:40:29,736][61585] Updated weights for policy 1, policy_version 96270 (0.0008) [2023-10-14 21:40:30,095][61585] Updated weights for policy 1, policy_version 96280 (0.0010) [2023-10-14 21:40:31,547][61552] Updated weights for policy 0, policy_version 96422 (0.0009) [2023-10-14 21:40:31,925][61552] Updated weights for policy 0, policy_version 96432 (0.0011) [2023-10-14 21:40:32,291][61552] Updated weights for policy 0, policy_version 96442 (0.0010) [2023-10-14 21:40:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 197361664. Throughput: 0: 1683.1, 1: 1659.7. Samples: 49342840. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:40:33,344][60425] Avg episode reward: [(0, '78.700'), (1, '79.610')] [2023-10-14 21:40:34,148][61585] Updated weights for policy 1, policy_version 96290 (0.0009) [2023-10-14 21:40:34,510][61585] Updated weights for policy 1, policy_version 96300 (0.0007) [2023-10-14 21:40:34,870][61585] Updated weights for policy 1, policy_version 96310 (0.0011) [2023-10-14 21:40:35,233][61585] Updated weights for policy 1, policy_version 96320 (0.0009) [2023-10-14 21:40:36,601][61552] Updated weights for policy 0, policy_version 96452 (0.0007) [2023-10-14 21:40:36,968][61552] Updated weights for policy 0, policy_version 96462 (0.0007) [2023-10-14 21:40:37,338][61552] Updated weights for policy 0, policy_version 96472 (0.0007) [2023-10-14 21:40:38,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 197427200. Throughput: 0: 1671.1, 1: 1677.7. Samples: 49362942. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:40:38,345][60425] Avg episode reward: [(0, '76.530'), (1, '87.760')] [2023-10-14 21:40:38,346][61248] Saving new best policy, reward=87.760! [2023-10-14 21:40:39,355][61585] Updated weights for policy 1, policy_version 96330 (0.0009) [2023-10-14 21:40:39,709][61585] Updated weights for policy 1, policy_version 96340 (0.0008) [2023-10-14 21:40:40,077][61585] Updated weights for policy 1, policy_version 96350 (0.0011) [2023-10-14 21:40:41,606][61552] Updated weights for policy 0, policy_version 96482 (0.0011) [2023-10-14 21:40:41,967][61552] Updated weights for policy 0, policy_version 96492 (0.0009) [2023-10-14 21:40:42,330][61552] Updated weights for policy 0, policy_version 96502 (0.0007) [2023-10-14 21:40:42,704][61552] Updated weights for policy 0, policy_version 96512 (0.0008) [2023-10-14 21:40:43,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 197492736. Throughput: 0: 1660.7, 1: 1680.5. Samples: 49382862. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:40:43,345][60425] Avg episode reward: [(0, '78.840'), (1, '83.870')] [2023-10-14 21:40:44,035][61585] Updated weights for policy 1, policy_version 96360 (0.0007) [2023-10-14 21:40:44,392][61585] Updated weights for policy 1, policy_version 96370 (0.0007) [2023-10-14 21:40:44,761][61585] Updated weights for policy 1, policy_version 96380 (0.0007) [2023-10-14 21:40:46,642][61552] Updated weights for policy 0, policy_version 96522 (0.0010) [2023-10-14 21:40:47,008][61552] Updated weights for policy 0, policy_version 96532 (0.0010) [2023-10-14 21:40:47,377][61552] Updated weights for policy 0, policy_version 96542 (0.0007) [2023-10-14 21:40:48,343][60425] Fps is (10 sec: 13107.7, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 197558272. Throughput: 0: 1676.7, 1: 1674.0. Samples: 49393234. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:40:48,344][60425] Avg episode reward: [(0, '81.740'), (1, '81.250')] [2023-10-14 21:40:48,947][61585] Updated weights for policy 1, policy_version 96390 (0.0009) [2023-10-14 21:40:49,312][61585] Updated weights for policy 1, policy_version 96400 (0.0009) [2023-10-14 21:40:49,683][61585] Updated weights for policy 1, policy_version 96410 (0.0009) [2023-10-14 21:40:51,423][61552] Updated weights for policy 0, policy_version 96552 (0.0009) [2023-10-14 21:40:51,798][61552] Updated weights for policy 0, policy_version 96562 (0.0007) [2023-10-14 21:40:52,163][61552] Updated weights for policy 0, policy_version 96572 (0.0008) [2023-10-14 21:40:53,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 197623808. Throughput: 0: 1659.6, 1: 1682.4. Samples: 49413084. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:40:53,344][60425] Avg episode reward: [(0, '76.420'), (1, '81.260')] [2023-10-14 21:40:53,838][61585] Updated weights for policy 1, policy_version 96420 (0.0008) [2023-10-14 21:40:54,200][61585] Updated weights for policy 1, policy_version 96430 (0.0007) [2023-10-14 21:40:54,555][61585] Updated weights for policy 1, policy_version 96440 (0.0007) [2023-10-14 21:40:56,161][61552] Updated weights for policy 0, policy_version 96582 (0.0009) [2023-10-14 21:40:56,523][61552] Updated weights for policy 0, policy_version 96592 (0.0008) [2023-10-14 21:40:56,887][61552] Updated weights for policy 0, policy_version 96602 (0.0008) [2023-10-14 21:40:58,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 197689344. Throughput: 0: 1673.8, 1: 1682.2. Samples: 49433398. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-14 21:40:58,345][60425] Avg episode reward: [(0, '80.240'), (1, '77.190')] [2023-10-14 21:40:58,667][61585] Updated weights for policy 1, policy_version 96450 (0.0009) [2023-10-14 21:40:59,025][61585] Updated weights for policy 1, policy_version 96460 (0.0009) [2023-10-14 21:40:59,384][61585] Updated weights for policy 1, policy_version 96470 (0.0008) [2023-10-14 21:40:59,748][61585] Updated weights for policy 1, policy_version 96480 (0.0011) [2023-10-14 21:41:00,750][61552] Updated weights for policy 0, policy_version 96612 (0.0007) [2023-10-14 21:41:01,126][61552] Updated weights for policy 0, policy_version 96622 (0.0009) [2023-10-14 21:41:01,492][61552] Updated weights for policy 0, policy_version 96632 (0.0009) [2023-10-14 21:41:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 197754880. Throughput: 0: 1678.3, 1: 1677.1. Samples: 49443608. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-14 21:41:03,344][60425] Avg episode reward: [(0, '79.120'), (1, '79.740')] [2023-10-14 21:41:03,834][61585] Updated weights for policy 1, policy_version 96490 (0.0008) [2023-10-14 21:41:04,199][61585] Updated weights for policy 1, policy_version 96500 (0.0009) [2023-10-14 21:41:04,559][61585] Updated weights for policy 1, policy_version 96510 (0.0010) [2023-10-14 21:41:05,539][61552] Updated weights for policy 0, policy_version 96642 (0.0008) [2023-10-14 21:41:05,906][61552] Updated weights for policy 0, policy_version 96652 (0.0008) [2023-10-14 21:41:06,274][61552] Updated weights for policy 0, policy_version 96662 (0.0007) [2023-10-14 21:41:06,636][61552] Updated weights for policy 0, policy_version 96672 (0.0008) [2023-10-14 21:41:08,344][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.1, 300 sec: 13440.4). Total num frames: 197820416. Throughput: 0: 1659.5, 1: 1684.0. Samples: 49463314. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-14 21:41:08,345][60425] Avg episode reward: [(0, '79.990'), (1, '79.180')] [2023-10-14 21:41:08,588][61585] Updated weights for policy 1, policy_version 96520 (0.0009) [2023-10-14 21:41:08,960][61585] Updated weights for policy 1, policy_version 96530 (0.0008) [2023-10-14 21:41:09,323][61585] Updated weights for policy 1, policy_version 96540 (0.0008) [2023-10-14 21:41:10,700][61552] Updated weights for policy 0, policy_version 96682 (0.0009) [2023-10-14 21:41:11,071][61552] Updated weights for policy 0, policy_version 96692 (0.0008) [2023-10-14 21:41:11,442][61552] Updated weights for policy 0, policy_version 96702 (0.0009) [2023-10-14 21:41:13,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 197885952. Throughput: 0: 1684.7, 1: 1686.3. Samples: 49484276. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-14 21:41:13,344][60425] Avg episode reward: [(0, '82.030'), (1, '79.880')] [2023-10-14 21:41:13,447][61585] Updated weights for policy 1, policy_version 96550 (0.0007) [2023-10-14 21:41:13,819][61585] Updated weights for policy 1, policy_version 96560 (0.0008) [2023-10-14 21:41:14,178][61585] Updated weights for policy 1, policy_version 96570 (0.0009) [2023-10-14 21:41:15,404][61552] Updated weights for policy 0, policy_version 96712 (0.0008) [2023-10-14 21:41:15,773][61552] Updated weights for policy 0, policy_version 96722 (0.0007) [2023-10-14 21:41:16,136][61552] Updated weights for policy 0, policy_version 96732 (0.0008) [2023-10-14 21:41:18,281][61585] Updated weights for policy 1, policy_version 96580 (0.0008) [2023-10-14 21:41:18,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 197951488. Throughput: 0: 1672.4, 1: 1686.4. Samples: 49493986. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-14 21:41:18,344][60425] Avg episode reward: [(0, '78.310'), (1, '75.150')] [2023-10-14 21:41:18,639][61585] Updated weights for policy 1, policy_version 96590 (0.0010) [2023-10-14 21:41:19,001][61585] Updated weights for policy 1, policy_version 96600 (0.0008) [2023-10-14 21:41:20,273][61552] Updated weights for policy 0, policy_version 96742 (0.0008) [2023-10-14 21:41:20,647][61552] Updated weights for policy 0, policy_version 96752 (0.0010) [2023-10-14 21:41:21,022][61552] Updated weights for policy 0, policy_version 96762 (0.0010) [2023-10-14 21:41:23,083][61585] Updated weights for policy 1, policy_version 96610 (0.0007) [2023-10-14 21:41:23,343][60425] Fps is (10 sec: 13107.0, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 198017024. Throughput: 0: 1671.3, 1: 1686.8. Samples: 49514054. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-14 21:41:23,344][60425] Avg episode reward: [(0, '81.380'), (1, '77.820')] [2023-10-14 21:41:23,459][61585] Updated weights for policy 1, policy_version 96620 (0.0008) [2023-10-14 21:41:23,816][61585] Updated weights for policy 1, policy_version 96630 (0.0008) [2023-10-14 21:41:24,179][61585] Updated weights for policy 1, policy_version 96640 (0.0008) [2023-10-14 21:41:25,121][61552] Updated weights for policy 0, policy_version 96772 (0.0009) [2023-10-14 21:41:25,495][61552] Updated weights for policy 0, policy_version 96782 (0.0008) [2023-10-14 21:41:25,879][61552] Updated weights for policy 0, policy_version 96792 (0.0008) [2023-10-14 21:41:28,096][61585] Updated weights for policy 1, policy_version 96650 (0.0007) [2023-10-14 21:41:28,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 198082560. Throughput: 0: 1692.6, 1: 1685.7. Samples: 49534886. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-14 21:41:28,344][60425] Avg episode reward: [(0, '79.510'), (1, '79.690')] [2023-10-14 21:41:28,470][61585] Updated weights for policy 1, policy_version 96660 (0.0007) [2023-10-14 21:41:28,833][61585] Updated weights for policy 1, policy_version 96670 (0.0008) [2023-10-14 21:41:30,020][61552] Updated weights for policy 0, policy_version 96802 (0.0009) [2023-10-14 21:41:30,394][61552] Updated weights for policy 0, policy_version 96812 (0.0008) [2023-10-14 21:41:30,758][61552] Updated weights for policy 0, policy_version 96822 (0.0009) [2023-10-14 21:41:31,120][61552] Updated weights for policy 0, policy_version 96832 (0.0009) [2023-10-14 21:41:32,880][61585] Updated weights for policy 1, policy_version 96680 (0.0010) [2023-10-14 21:41:33,245][61585] Updated weights for policy 1, policy_version 96690 (0.0009) [2023-10-14 21:41:33,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 198148096. Throughput: 0: 1673.2, 1: 1689.8. Samples: 49544570. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-14 21:41:33,344][60425] Avg episode reward: [(0, '86.310'), (1, '79.700')] [2023-10-14 21:41:33,611][61585] Updated weights for policy 1, policy_version 96700 (0.0007) [2023-10-14 21:41:35,113][61552] Updated weights for policy 0, policy_version 96842 (0.0009) [2023-10-14 21:41:35,487][61552] Updated weights for policy 0, policy_version 96852 (0.0008) [2023-10-14 21:41:35,850][61552] Updated weights for policy 0, policy_version 96862 (0.0009) [2023-10-14 21:41:37,485][61585] Updated weights for policy 1, policy_version 96710 (0.0010) [2023-10-14 21:41:37,857][61585] Updated weights for policy 1, policy_version 96720 (0.0009) [2023-10-14 21:41:38,226][61585] Updated weights for policy 1, policy_version 96730 (0.0010) [2023-10-14 21:41:38,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 198213632. Throughput: 0: 1680.3, 1: 1695.1. Samples: 49564974. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-14 21:41:38,345][60425] Avg episode reward: [(0, '79.410'), (1, '78.980')] [2023-10-14 21:41:39,966][61552] Updated weights for policy 0, policy_version 96872 (0.0010) [2023-10-14 21:41:40,347][61552] Updated weights for policy 0, policy_version 96882 (0.0011) [2023-10-14 21:41:40,718][61552] Updated weights for policy 0, policy_version 96892 (0.0009) [2023-10-14 21:41:42,270][61585] Updated weights for policy 1, policy_version 96740 (0.0007) [2023-10-14 21:41:42,645][61585] Updated weights for policy 1, policy_version 96750 (0.0008) [2023-10-14 21:41:43,006][61585] Updated weights for policy 1, policy_version 96760 (0.0008) [2023-10-14 21:41:43,343][60425] Fps is (10 sec: 16384.1, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 198311936. Throughput: 0: 1692.9, 1: 1681.8. Samples: 49585258. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-14 21:41:43,344][60425] Avg episode reward: [(0, '80.700'), (1, '81.070')] [2023-10-14 21:41:43,353][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000096768_99090432.pth... [2023-10-14 21:41:43,353][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000096896_99221504.pth... [2023-10-14 21:41:43,393][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000095200_97484800.pth [2023-10-14 21:41:43,399][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000095328_97615872.pth [2023-10-14 21:41:44,541][61552] Updated weights for policy 0, policy_version 96902 (0.0008) [2023-10-14 21:41:44,914][61552] Updated weights for policy 0, policy_version 96912 (0.0008) [2023-10-14 21:41:45,286][61552] Updated weights for policy 0, policy_version 96922 (0.0007) [2023-10-14 21:41:47,050][61585] Updated weights for policy 1, policy_version 96770 (0.0010) [2023-10-14 21:41:47,410][61585] Updated weights for policy 1, policy_version 96780 (0.0009) [2023-10-14 21:41:47,779][61585] Updated weights for policy 1, policy_version 96790 (0.0009) [2023-10-14 21:41:48,136][61585] Updated weights for policy 1, policy_version 96800 (0.0009) [2023-10-14 21:41:48,343][60425] Fps is (10 sec: 16384.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 198377472. Throughput: 0: 1664.2, 1: 1700.4. Samples: 49595016. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-14 21:41:48,344][60425] Avg episode reward: [(0, '79.200'), (1, '79.120')] [2023-10-14 21:41:49,519][61552] Updated weights for policy 0, policy_version 96932 (0.0008) [2023-10-14 21:41:49,887][61552] Updated weights for policy 0, policy_version 96942 (0.0008) [2023-10-14 21:41:50,261][61552] Updated weights for policy 0, policy_version 96952 (0.0008) [2023-10-14 21:41:52,334][61585] Updated weights for policy 1, policy_version 96810 (0.0009) [2023-10-14 21:41:52,703][61585] Updated weights for policy 1, policy_version 96820 (0.0007) [2023-10-14 21:41:53,077][61585] Updated weights for policy 1, policy_version 96830 (0.0009) [2023-10-14 21:41:53,343][60425] Fps is (10 sec: 13107.1, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 198443008. Throughput: 0: 1689.9, 1: 1696.2. Samples: 49615686. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-14 21:41:53,344][60425] Avg episode reward: [(0, '84.310'), (1, '80.280')] [2023-10-14 21:41:54,365][61552] Updated weights for policy 0, policy_version 96962 (0.0008) [2023-10-14 21:41:54,728][61552] Updated weights for policy 0, policy_version 96972 (0.0008) [2023-10-14 21:41:55,104][61552] Updated weights for policy 0, policy_version 96982 (0.0008) [2023-10-14 21:41:55,477][61552] Updated weights for policy 0, policy_version 96992 (0.0009) [2023-10-14 21:41:57,103][61585] Updated weights for policy 1, policy_version 96840 (0.0008) [2023-10-14 21:41:57,466][61585] Updated weights for policy 1, policy_version 96850 (0.0010) [2023-10-14 21:41:57,831][61585] Updated weights for policy 1, policy_version 96860 (0.0010) [2023-10-14 21:41:58,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 198508544. Throughput: 0: 1685.1, 1: 1672.4. Samples: 49635368. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-14 21:41:58,345][60425] Avg episode reward: [(0, '81.520'), (1, '80.820')] [2023-10-14 21:41:59,415][61552] Updated weights for policy 0, policy_version 97002 (0.0008) [2023-10-14 21:41:59,789][61552] Updated weights for policy 0, policy_version 97012 (0.0010) [2023-10-14 21:42:00,149][61552] Updated weights for policy 0, policy_version 97022 (0.0010) [2023-10-14 21:42:02,173][61585] Updated weights for policy 1, policy_version 96870 (0.0011) [2023-10-14 21:42:02,547][61585] Updated weights for policy 1, policy_version 96880 (0.0011) [2023-10-14 21:42:02,914][61585] Updated weights for policy 1, policy_version 96890 (0.0011) [2023-10-14 21:42:03,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 198574080. Throughput: 0: 1666.5, 1: 1695.9. Samples: 49645296. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-14 21:42:03,344][60425] Avg episode reward: [(0, '82.250'), (1, '79.770')] [2023-10-14 21:42:04,449][61552] Updated weights for policy 0, policy_version 97032 (0.0009) [2023-10-14 21:42:04,821][61552] Updated weights for policy 0, policy_version 97042 (0.0009) [2023-10-14 21:42:05,181][61552] Updated weights for policy 0, policy_version 97052 (0.0010) [2023-10-14 21:42:06,987][61585] Updated weights for policy 1, policy_version 96900 (0.0010) [2023-10-14 21:42:07,339][61585] Updated weights for policy 1, policy_version 96910 (0.0007) [2023-10-14 21:42:07,700][61585] Updated weights for policy 1, policy_version 96920 (0.0010) [2023-10-14 21:42:08,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 198639616. Throughput: 0: 1677.6, 1: 1689.0. Samples: 49665554. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-10-14 21:42:08,344][60425] Avg episode reward: [(0, '82.500'), (1, '79.500')] [2023-10-14 21:42:09,209][61552] Updated weights for policy 0, policy_version 97062 (0.0010) [2023-10-14 21:42:09,573][61552] Updated weights for policy 0, policy_version 97072 (0.0009) [2023-10-14 21:42:09,945][61552] Updated weights for policy 0, policy_version 97082 (0.0010) [2023-10-14 21:42:11,742][61585] Updated weights for policy 1, policy_version 96930 (0.0010) [2023-10-14 21:42:12,118][61585] Updated weights for policy 1, policy_version 96940 (0.0009) [2023-10-14 21:42:12,488][61585] Updated weights for policy 1, policy_version 96950 (0.0007) [2023-10-14 21:42:12,848][61585] Updated weights for policy 1, policy_version 96960 (0.0008) [2023-10-14 21:42:13,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 198705152. Throughput: 0: 1677.6, 1: 1660.9. Samples: 49685120. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-10-14 21:42:13,344][60425] Avg episode reward: [(0, '80.220'), (1, '76.830')] [2023-10-14 21:42:13,950][61552] Updated weights for policy 0, policy_version 97092 (0.0010) [2023-10-14 21:42:14,321][61552] Updated weights for policy 0, policy_version 97102 (0.0011) [2023-10-14 21:42:14,681][61552] Updated weights for policy 0, policy_version 97112 (0.0009) [2023-10-14 21:42:16,879][61585] Updated weights for policy 1, policy_version 96970 (0.0007) [2023-10-14 21:42:17,243][61585] Updated weights for policy 1, policy_version 96980 (0.0008) [2023-10-14 21:42:17,610][61585] Updated weights for policy 1, policy_version 96990 (0.0008) [2023-10-14 21:42:18,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 198770688. Throughput: 0: 1664.9, 1: 1683.7. Samples: 49695256. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-10-14 21:42:18,344][60425] Avg episode reward: [(0, '83.110'), (1, '79.840')] [2023-10-14 21:42:18,964][61552] Updated weights for policy 0, policy_version 97122 (0.0010) [2023-10-14 21:42:19,337][61552] Updated weights for policy 0, policy_version 97132 (0.0010) [2023-10-14 21:42:19,700][61552] Updated weights for policy 0, policy_version 97142 (0.0008) [2023-10-14 21:42:20,073][61552] Updated weights for policy 0, policy_version 97152 (0.0009) [2023-10-14 21:42:21,575][61585] Updated weights for policy 1, policy_version 97000 (0.0007) [2023-10-14 21:42:21,940][61585] Updated weights for policy 1, policy_version 97010 (0.0007) [2023-10-14 21:42:22,306][61585] Updated weights for policy 1, policy_version 97020 (0.0008) [2023-10-14 21:42:23,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 198836224. Throughput: 0: 1671.0, 1: 1673.1. Samples: 49715456. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-10-14 21:42:23,344][60425] Avg episode reward: [(0, '79.660'), (1, '77.860')] [2023-10-14 21:42:24,220][61552] Updated weights for policy 0, policy_version 97162 (0.0009) [2023-10-14 21:42:24,589][61552] Updated weights for policy 0, policy_version 97172 (0.0008) [2023-10-14 21:42:24,975][61552] Updated weights for policy 0, policy_version 97182 (0.0010) [2023-10-14 21:42:26,404][61585] Updated weights for policy 1, policy_version 97030 (0.0008) [2023-10-14 21:42:26,771][61585] Updated weights for policy 1, policy_version 97040 (0.0009) [2023-10-14 21:42:27,127][61585] Updated weights for policy 1, policy_version 97050 (0.0009) [2023-10-14 21:42:28,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 198901760. Throughput: 0: 1665.0, 1: 1667.3. Samples: 49735210. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-10-14 21:42:28,345][60425] Avg episode reward: [(0, '78.260'), (1, '82.720')] [2023-10-14 21:42:28,875][61552] Updated weights for policy 0, policy_version 97192 (0.0011) [2023-10-14 21:42:29,238][61552] Updated weights for policy 0, policy_version 97202 (0.0008) [2023-10-14 21:42:29,610][61552] Updated weights for policy 0, policy_version 97212 (0.0008) [2023-10-14 21:42:31,097][61585] Updated weights for policy 1, policy_version 97060 (0.0009) [2023-10-14 21:42:31,465][61585] Updated weights for policy 1, policy_version 97070 (0.0010) [2023-10-14 21:42:31,817][61585] Updated weights for policy 1, policy_version 97080 (0.0007) [2023-10-14 21:42:33,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 198967296. Throughput: 0: 1664.4, 1: 1680.0. Samples: 49745514. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-10-14 21:42:33,344][60425] Avg episode reward: [(0, '78.800'), (1, '77.500')] [2023-10-14 21:42:33,862][61552] Updated weights for policy 0, policy_version 97222 (0.0011) [2023-10-14 21:42:34,236][61552] Updated weights for policy 0, policy_version 97232 (0.0009) [2023-10-14 21:42:34,612][61552] Updated weights for policy 0, policy_version 97242 (0.0008) [2023-10-14 21:42:35,948][61585] Updated weights for policy 1, policy_version 97090 (0.0009) [2023-10-14 21:42:36,306][61585] Updated weights for policy 1, policy_version 97100 (0.0010) [2023-10-14 21:42:36,671][61585] Updated weights for policy 1, policy_version 97110 (0.0012) [2023-10-14 21:42:37,036][61585] Updated weights for policy 1, policy_version 97120 (0.0011) [2023-10-14 21:42:38,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13653.4, 300 sec: 13440.4). Total num frames: 199032832. Throughput: 0: 1664.7, 1: 1658.6. Samples: 49765232. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-10-14 21:42:38,344][60425] Avg episode reward: [(0, '80.930'), (1, '78.800')] [2023-10-14 21:42:38,694][61552] Updated weights for policy 0, policy_version 97252 (0.0010) [2023-10-14 21:42:39,068][61552] Updated weights for policy 0, policy_version 97262 (0.0011) [2023-10-14 21:42:39,436][61552] Updated weights for policy 0, policy_version 97272 (0.0009) [2023-10-14 21:42:41,179][61585] Updated weights for policy 1, policy_version 97130 (0.0008) [2023-10-14 21:42:41,544][61585] Updated weights for policy 1, policy_version 97140 (0.0008) [2023-10-14 21:42:41,919][61585] Updated weights for policy 1, policy_version 97150 (0.0009) [2023-10-14 21:42:43,344][60425] Fps is (10 sec: 13106.8, 60 sec: 13107.2, 300 sec: 13440.4). Total num frames: 199098368. Throughput: 0: 1665.9, 1: 1669.2. Samples: 49785450. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-10-14 21:42:43,345][60425] Avg episode reward: [(0, '78.380'), (1, '79.830')] [2023-10-14 21:42:43,586][61552] Updated weights for policy 0, policy_version 97282 (0.0010) [2023-10-14 21:42:43,956][61552] Updated weights for policy 0, policy_version 97292 (0.0008) [2023-10-14 21:42:44,315][61552] Updated weights for policy 0, policy_version 97302 (0.0009) [2023-10-14 21:42:44,693][61552] Updated weights for policy 0, policy_version 97312 (0.0008) [2023-10-14 21:42:45,990][61585] Updated weights for policy 1, policy_version 97160 (0.0009) [2023-10-14 21:42:46,358][61585] Updated weights for policy 1, policy_version 97170 (0.0008) [2023-10-14 21:42:46,723][61585] Updated weights for policy 1, policy_version 97180 (0.0009) [2023-10-14 21:42:48,344][60425] Fps is (10 sec: 13106.0, 60 sec: 13107.0, 300 sec: 13440.4). Total num frames: 199163904. Throughput: 0: 1669.0, 1: 1673.4. Samples: 49795708. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-10-14 21:42:48,345][60425] Avg episode reward: [(0, '73.010'), (1, '83.260')] [2023-10-14 21:42:48,795][61552] Updated weights for policy 0, policy_version 97322 (0.0009) [2023-10-14 21:42:49,154][61552] Updated weights for policy 0, policy_version 97332 (0.0007) [2023-10-14 21:42:49,523][61552] Updated weights for policy 0, policy_version 97342 (0.0007) [2023-10-14 21:42:50,930][61585] Updated weights for policy 1, policy_version 97190 (0.0007) [2023-10-14 21:42:51,292][61585] Updated weights for policy 1, policy_version 97200 (0.0009) [2023-10-14 21:42:51,654][61585] Updated weights for policy 1, policy_version 97210 (0.0009) [2023-10-14 21:42:53,343][60425] Fps is (10 sec: 13107.7, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 199229440. Throughput: 0: 1668.8, 1: 1660.5. Samples: 49815372. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-10-14 21:42:53,344][60425] Avg episode reward: [(0, '79.150'), (1, '80.630')] [2023-10-14 21:42:53,715][61552] Updated weights for policy 0, policy_version 97352 (0.0007) [2023-10-14 21:42:54,080][61552] Updated weights for policy 0, policy_version 97362 (0.0007) [2023-10-14 21:42:54,445][61552] Updated weights for policy 0, policy_version 97372 (0.0008) [2023-10-14 21:42:55,963][61585] Updated weights for policy 1, policy_version 97220 (0.0007) [2023-10-14 21:42:56,357][61585] Updated weights for policy 1, policy_version 97230 (0.0009) [2023-10-14 21:42:56,714][61585] Updated weights for policy 1, policy_version 97240 (0.0007) [2023-10-14 21:42:58,343][60425] Fps is (10 sec: 13108.5, 60 sec: 13107.3, 300 sec: 13440.4). Total num frames: 199294976. Throughput: 0: 1672.1, 1: 1678.8. Samples: 49835914. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-10-14 21:42:58,344][60425] Avg episode reward: [(0, '82.820'), (1, '83.840')] [2023-10-14 21:42:58,437][61552] Updated weights for policy 0, policy_version 97382 (0.0010) [2023-10-14 21:42:58,815][61552] Updated weights for policy 0, policy_version 97392 (0.0009) [2023-10-14 21:42:59,177][61552] Updated weights for policy 0, policy_version 97402 (0.0007) [2023-10-14 21:43:00,727][61585] Updated weights for policy 1, policy_version 97250 (0.0007) [2023-10-14 21:43:01,092][61585] Updated weights for policy 1, policy_version 97260 (0.0009) [2023-10-14 21:43:01,446][61585] Updated weights for policy 1, policy_version 97270 (0.0009) [2023-10-14 21:43:01,802][61585] Updated weights for policy 1, policy_version 97280 (0.0011) [2023-10-14 21:43:03,132][61552] Updated weights for policy 0, policy_version 97412 (0.0007) [2023-10-14 21:43:03,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 199360512. Throughput: 0: 1673.7, 1: 1679.0. Samples: 49846130. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-10-14 21:43:03,344][60425] Avg episode reward: [(0, '81.650'), (1, '83.010')] [2023-10-14 21:43:03,499][61552] Updated weights for policy 0, policy_version 97422 (0.0008) [2023-10-14 21:43:03,872][61552] Updated weights for policy 0, policy_version 97432 (0.0008) [2023-10-14 21:43:06,078][61585] Updated weights for policy 1, policy_version 97290 (0.0007) [2023-10-14 21:43:06,433][61585] Updated weights for policy 1, policy_version 97300 (0.0010) [2023-10-14 21:43:06,792][61585] Updated weights for policy 1, policy_version 97310 (0.0011) [2023-10-14 21:43:08,155][61552] Updated weights for policy 0, policy_version 97442 (0.0009) [2023-10-14 21:43:08,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 199426048. Throughput: 0: 1679.2, 1: 1662.4. Samples: 49865828. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-10-14 21:43:08,344][60425] Avg episode reward: [(0, '82.960'), (1, '76.500')] [2023-10-14 21:43:08,526][61552] Updated weights for policy 0, policy_version 97452 (0.0009) [2023-10-14 21:43:08,904][61552] Updated weights for policy 0, policy_version 97462 (0.0009) [2023-10-14 21:43:09,276][61552] Updated weights for policy 0, policy_version 97472 (0.0008) [2023-10-14 21:43:10,765][61585] Updated weights for policy 1, policy_version 97320 (0.0010) [2023-10-14 21:43:11,124][61585] Updated weights for policy 1, policy_version 97330 (0.0007) [2023-10-14 21:43:11,486][61585] Updated weights for policy 1, policy_version 97340 (0.0011) [2023-10-14 21:43:13,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 199491584. Throughput: 0: 1680.6, 1: 1671.0. Samples: 49886032. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-14 21:43:13,345][60425] Avg episode reward: [(0, '78.500'), (1, '81.910')] [2023-10-14 21:43:13,398][61552] Updated weights for policy 0, policy_version 97482 (0.0009) [2023-10-14 21:43:13,762][61552] Updated weights for policy 0, policy_version 97492 (0.0010) [2023-10-14 21:43:14,123][61552] Updated weights for policy 0, policy_version 97502 (0.0009) [2023-10-14 21:43:15,577][61585] Updated weights for policy 1, policy_version 97350 (0.0008) [2023-10-14 21:43:15,946][61585] Updated weights for policy 1, policy_version 97360 (0.0008) [2023-10-14 21:43:16,311][61585] Updated weights for policy 1, policy_version 97370 (0.0009) [2023-10-14 21:43:18,015][61552] Updated weights for policy 0, policy_version 97512 (0.0008) [2023-10-14 21:43:18,343][60425] Fps is (10 sec: 13107.2, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 199557120. Throughput: 0: 1679.0, 1: 1664.9. Samples: 49895992. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-14 21:43:18,344][60425] Avg episode reward: [(0, '80.400'), (1, '84.190')] [2023-10-14 21:43:18,376][61552] Updated weights for policy 0, policy_version 97522 (0.0009) [2023-10-14 21:43:18,743][61552] Updated weights for policy 0, policy_version 97532 (0.0009) [2023-10-14 21:43:20,204][61585] Updated weights for policy 1, policy_version 97380 (0.0008) [2023-10-14 21:43:20,568][61585] Updated weights for policy 1, policy_version 97390 (0.0008) [2023-10-14 21:43:20,922][61585] Updated weights for policy 1, policy_version 97400 (0.0011) [2023-10-14 21:43:22,996][61552] Updated weights for policy 0, policy_version 97542 (0.0009) [2023-10-14 21:43:23,343][60425] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 199622656. Throughput: 0: 1681.0, 1: 1669.0. Samples: 49915982. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-14 21:43:23,344][60425] Avg episode reward: [(0, '78.690'), (1, '84.840')] [2023-10-14 21:43:23,361][61552] Updated weights for policy 0, policy_version 97552 (0.0007) [2023-10-14 21:43:23,733][61552] Updated weights for policy 0, policy_version 97562 (0.0007) [2023-10-14 21:43:25,183][61585] Updated weights for policy 1, policy_version 97410 (0.0008) [2023-10-14 21:43:25,549][61585] Updated weights for policy 1, policy_version 97420 (0.0009) [2023-10-14 21:43:25,913][61585] Updated weights for policy 1, policy_version 97430 (0.0009) [2023-10-14 21:43:26,271][61585] Updated weights for policy 1, policy_version 97440 (0.0009) [2023-10-14 21:43:27,747][61552] Updated weights for policy 0, policy_version 97572 (0.0011) [2023-10-14 21:43:28,130][61552] Updated weights for policy 0, policy_version 97582 (0.0012) [2023-10-14 21:43:28,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.2, 300 sec: 13329.3). Total num frames: 199688192. Throughput: 0: 1677.5, 1: 1678.5. Samples: 49936472. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-14 21:43:28,344][60425] Avg episode reward: [(0, '81.700'), (1, '80.670')] [2023-10-14 21:43:28,487][61552] Updated weights for policy 0, policy_version 97592 (0.0011) [2023-10-14 21:43:30,289][61585] Updated weights for policy 1, policy_version 97450 (0.0011) [2023-10-14 21:43:30,659][61585] Updated weights for policy 1, policy_version 97460 (0.0009) [2023-10-14 21:43:31,021][61585] Updated weights for policy 1, policy_version 97470 (0.0010) [2023-10-14 21:43:32,409][61552] Updated weights for policy 0, policy_version 97602 (0.0009) [2023-10-14 21:43:32,778][61552] Updated weights for policy 0, policy_version 97612 (0.0007) [2023-10-14 21:43:33,148][61552] Updated weights for policy 0, policy_version 97622 (0.0008) [2023-10-14 21:43:33,343][60425] Fps is (10 sec: 13107.3, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 199753728. Throughput: 0: 1680.3, 1: 1664.1. Samples: 49946200. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-14 21:43:33,344][60425] Avg episode reward: [(0, '76.260'), (1, '79.110')] [2023-10-14 21:43:33,521][61552] Updated weights for policy 0, policy_version 97632 (0.0008) [2023-10-14 21:43:35,069][61585] Updated weights for policy 1, policy_version 97480 (0.0008) [2023-10-14 21:43:35,433][61585] Updated weights for policy 1, policy_version 97490 (0.0012) [2023-10-14 21:43:35,806][61585] Updated weights for policy 1, policy_version 97500 (0.0010) [2023-10-14 21:43:37,576][61552] Updated weights for policy 0, policy_version 97642 (0.0009) [2023-10-14 21:43:37,940][61552] Updated weights for policy 0, policy_version 97652 (0.0009) [2023-10-14 21:43:38,307][61552] Updated weights for policy 0, policy_version 97662 (0.0009) [2023-10-14 21:43:38,343][60425] Fps is (10 sec: 13107.5, 60 sec: 13107.2, 300 sec: 13329.4). Total num frames: 199819264. Throughput: 0: 1681.5, 1: 1675.5. Samples: 49966438. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-14 21:43:38,344][60425] Avg episode reward: [(0, '84.270'), (1, '78.030')] [2023-10-14 21:43:39,869][61585] Updated weights for policy 1, policy_version 97510 (0.0009) [2023-10-14 21:43:40,236][61585] Updated weights for policy 1, policy_version 97520 (0.0007) [2023-10-14 21:43:40,592][61585] Updated weights for policy 1, policy_version 97530 (0.0007) [2023-10-14 21:43:42,450][61552] Updated weights for policy 0, policy_version 97672 (0.0009) [2023-10-14 21:43:42,829][61552] Updated weights for policy 0, policy_version 97682 (0.0010) [2023-10-14 21:43:43,191][61552] Updated weights for policy 0, policy_version 97692 (0.0010) [2023-10-14 21:43:43,344][60425] Fps is (10 sec: 16383.3, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 199917568. Throughput: 0: 1664.8, 1: 1683.9. Samples: 49986608. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-14 21:43:43,345][60425] Avg episode reward: [(0, '75.190'), (1, '75.310')] [2023-10-14 21:43:43,356][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000097696_100040704.pth... [2023-10-14 21:43:43,356][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000097536_99876864.pth... [2023-10-14 21:43:43,389][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000095968_98271232.pth [2023-10-14 21:43:43,394][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000096128_98435072.pth [2023-10-14 21:43:44,787][61585] Updated weights for policy 1, policy_version 97540 (0.0009) [2023-10-14 21:43:45,164][61585] Updated weights for policy 1, policy_version 97550 (0.0009) [2023-10-14 21:43:45,526][61585] Updated weights for policy 1, policy_version 97560 (0.0010) [2023-10-14 21:43:47,420][61552] Updated weights for policy 0, policy_version 97702 (0.0009) [2023-10-14 21:43:47,786][61552] Updated weights for policy 0, policy_version 97712 (0.0009) [2023-10-14 21:43:48,160][61552] Updated weights for policy 0, policy_version 97722 (0.0009) [2023-10-14 21:43:48,343][60425] Fps is (10 sec: 13106.9, 60 sec: 13107.4, 300 sec: 13329.3). Total num frames: 199950336. Throughput: 0: 1679.5, 1: 1658.3. Samples: 49996332. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-14 21:43:48,344][60425] Avg episode reward: [(0, '75.780'), (1, '74.890')] [2023-10-14 21:43:49,609][61585] Updated weights for policy 1, policy_version 97570 (0.0009) [2023-10-14 21:43:49,988][61585] Updated weights for policy 1, policy_version 97580 (0.0007) [2023-10-14 21:43:50,341][61585] Updated weights for policy 1, policy_version 97590 (0.0009) [2023-10-14 21:43:50,699][61585] Updated weights for policy 1, policy_version 97600 (0.0009) [2023-10-14 21:43:52,278][61552] Updated weights for policy 0, policy_version 97732 (0.0009) [2023-10-14 21:43:52,646][61552] Updated weights for policy 0, policy_version 97742 (0.0007) [2023-10-14 21:43:53,009][61552] Updated weights for policy 0, policy_version 97752 (0.0008) [2023-10-14 21:43:53,343][60425] Fps is (10 sec: 13107.6, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 200048640. Throughput: 0: 1677.6, 1: 1676.2. Samples: 50016748. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-14 21:43:53,344][60425] Avg episode reward: [(0, '78.660'), (1, '76.910')] [2023-10-14 21:43:54,696][61585] Updated weights for policy 1, policy_version 97610 (0.0011) [2023-10-14 21:43:55,065][61585] Updated weights for policy 1, policy_version 97620 (0.0008) [2023-10-14 21:43:55,417][61585] Updated weights for policy 1, policy_version 97630 (0.0007) [2023-10-14 21:43:57,017][61552] Updated weights for policy 0, policy_version 97762 (0.0008) [2023-10-14 21:43:57,379][61552] Updated weights for policy 0, policy_version 97772 (0.0009) [2023-10-14 21:43:57,746][61552] Updated weights for policy 0, policy_version 97782 (0.0009) [2023-10-14 21:43:58,115][61552] Updated weights for policy 0, policy_version 97792 (0.0008) [2023-10-14 21:43:58,343][60425] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13440.4). Total num frames: 200114176. Throughput: 0: 1663.9, 1: 1690.2. Samples: 50036964. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) [2023-10-14 21:43:58,345][60425] Avg episode reward: [(0, '81.680'), (1, '78.530')] [2023-10-14 21:43:59,284][61585] Updated weights for policy 1, policy_version 97640 (0.0008) [2023-10-14 21:43:59,651][61585] Updated weights for policy 1, policy_version 97650 (0.0007) [2023-10-14 21:44:00,007][61585] Updated weights for policy 1, policy_version 97660 (0.0007) [2023-10-14 21:44:02,294][61552] Updated weights for policy 0, policy_version 97802 (0.0009) [2023-10-14 21:44:02,665][61552] Updated weights for policy 0, policy_version 97812 (0.0007) [2023-10-14 21:44:03,036][61552] Updated weights for policy 0, policy_version 97822 (0.0007) [2023-10-14 21:44:03,100][61597] Stopping RolloutWorker_w11... [2023-10-14 21:44:03,100][61589] Stopping RolloutWorker_w3... [2023-10-14 21:44:03,100][61592] Stopping RolloutWorker_w6... [2023-10-14 21:44:03,100][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000097664_100007936.pth... [2023-10-14 21:44:03,101][61589] Loop rollout_proc3_evt_loop terminating... [2023-10-14 21:44:03,101][61597] Loop rollout_proc11_evt_loop terminating... [2023-10-14 21:44:03,100][60425] Component RolloutWorker_w11 stopped! [2023-10-14 21:44:03,101][61592] Loop rollout_proc6_evt_loop terminating... [2023-10-14 21:44:03,101][61587] Stopping RolloutWorker_w1... [2023-10-14 21:44:03,101][61596] Stopping RolloutWorker_w9... [2023-10-14 21:44:03,101][60425] Component RolloutWorker_w3 stopped! [2023-10-14 21:44:03,101][61172] Stopping Batcher_0... [2023-10-14 21:44:03,101][61587] Loop rollout_proc1_evt_loop terminating... [2023-10-14 21:44:03,101][61596] Loop rollout_proc9_evt_loop terminating... [2023-10-14 21:44:03,101][60425] Component RolloutWorker_w6 stopped! [2023-10-14 21:44:03,102][60425] Component Batcher_0 stopped! [2023-10-14 21:44:03,102][61584] Stopping RolloutWorker_w0... [2023-10-14 21:44:03,101][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000097824_100171776.pth... [2023-10-14 21:44:03,102][60425] Component Batcher_1 stopped! [2023-10-14 21:44:03,102][61584] Loop rollout_proc0_evt_loop terminating... [2023-10-14 21:44:03,102][60425] Component RolloutWorker_w1 stopped! [2023-10-14 21:44:03,102][60425] Component RolloutWorker_w9 stopped! [2023-10-14 21:44:03,103][60425] Component RolloutWorker_w0 stopped! [2023-10-14 21:44:03,104][61598] Stopping RolloutWorker_w12... [2023-10-14 21:44:03,104][61591] Stopping RolloutWorker_w4... [2023-10-14 21:44:03,104][61588] Stopping RolloutWorker_w2... [2023-10-14 21:44:03,104][61595] Stopping RolloutWorker_w10... [2023-10-14 21:44:03,104][61591] Loop rollout_proc4_evt_loop terminating... [2023-10-14 21:44:03,104][61598] Loop rollout_proc12_evt_loop terminating... [2023-10-14 21:44:03,104][61595] Loop rollout_proc10_evt_loop terminating... [2023-10-14 21:44:03,104][61588] Loop rollout_proc2_evt_loop terminating... [2023-10-14 21:44:03,104][60425] Component RolloutWorker_w4 stopped! [2023-10-14 21:44:03,105][60425] Component RolloutWorker_w12 stopped! [2023-10-14 21:44:03,106][61599] Stopping RolloutWorker_w13... [2023-10-14 21:44:03,106][61590] Stopping RolloutWorker_w5... [2023-10-14 21:44:03,106][61590] Loop rollout_proc5_evt_loop terminating... [2023-10-14 21:44:03,106][61599] Loop rollout_proc13_evt_loop terminating... [2023-10-14 21:44:03,106][60425] Component RolloutWorker_w10 stopped! [2023-10-14 21:44:03,106][60425] Component RolloutWorker_w2 stopped! [2023-10-14 21:44:03,107][61594] Stopping RolloutWorker_w7... [2023-10-14 21:44:03,107][60425] Component RolloutWorker_w5 stopped! [2023-10-14 21:44:03,107][62179] Stopping RolloutWorker_w15... [2023-10-14 21:44:03,108][61594] Loop rollout_proc7_evt_loop terminating... [2023-10-14 21:44:03,108][62179] Loop rollout_proc15_evt_loop terminating... [2023-10-14 21:44:03,108][60425] Component RolloutWorker_w13 stopped! [2023-10-14 21:44:03,108][62147] Stopping RolloutWorker_w14... [2023-10-14 21:44:03,108][62147] Loop rollout_proc14_evt_loop terminating... [2023-10-14 21:44:03,108][60425] Component RolloutWorker_w7 stopped! [2023-10-14 21:44:03,109][60425] Component RolloutWorker_w15 stopped! [2023-10-14 21:44:03,109][61593] Stopping RolloutWorker_w8... [2023-10-14 21:44:03,110][61593] Loop rollout_proc8_evt_loop terminating... [2023-10-14 21:44:03,110][60425] Component RolloutWorker_w14 stopped! [2023-10-14 21:44:03,110][60425] Component RolloutWorker_w8 stopped! [2023-10-14 21:44:03,101][61248] Stopping Batcher_1... [2023-10-14 21:44:03,102][61172] Loop batcher_evt_loop terminating... [2023-10-14 21:44:03,118][61585] Weights refcount: 2 0 [2023-10-14 21:44:03,120][61585] Stopping InferenceWorker_p1-w0... [2023-10-14 21:44:03,120][61585] Loop inference_proc1-0_evt_loop terminating... [2023-10-14 21:44:03,120][60425] Component InferenceWorker_p1-w0 stopped! [2023-10-14 21:44:03,122][61552] Weights refcount: 2 0 [2023-10-14 21:44:03,123][61552] Stopping InferenceWorker_p0-w0... [2023-10-14 21:44:03,124][61552] Loop inference_proc0-0_evt_loop terminating... [2023-10-14 21:44:03,124][60425] Component InferenceWorker_p0-w0 stopped! [2023-10-14 21:44:03,123][61248] Loop batcher_evt_loop terminating... [2023-10-14 21:44:03,134][61172] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000096896_99221504.pth [2023-10-14 21:44:03,134][61248] Removing ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000096768_99090432.pth [2023-10-14 21:44:03,138][61172] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p0/checkpoint_000097824_100171776.pth... [2023-10-14 21:44:03,139][61248] Saving ./train_atari/atari_roadrunner_APPO/checkpoint_p1/checkpoint_000097664_100007936.pth... [2023-10-14 21:44:03,181][61172] Stopping LearnerWorker_p0... [2023-10-14 21:44:03,181][61172] Loop learner_proc0_evt_loop terminating... [2023-10-14 21:44:03,181][61248] Stopping LearnerWorker_p1... [2023-10-14 21:44:03,181][60425] Component LearnerWorker_p0 stopped! [2023-10-14 21:44:03,182][61248] Loop learner_proc1_evt_loop terminating... [2023-10-14 21:44:03,182][60425] Component LearnerWorker_p1 stopped! [2023-10-14 21:44:03,182][60425] Waiting for process learner_proc0 to stop... [2023-10-14 21:44:04,190][60425] Waiting for process learner_proc1 to stop... [2023-10-14 21:44:04,190][60425] Waiting for process inference_proc0-0 to join... [2023-10-14 21:44:04,191][60425] Waiting for process inference_proc1-0 to join... [2023-10-14 21:44:04,192][60425] Waiting for process rollout_proc0 to join... [2023-10-14 21:44:04,192][60425] Waiting for process rollout_proc1 to join... [2023-10-14 21:44:04,193][60425] Waiting for process rollout_proc2 to join... [2023-10-14 21:44:04,193][60425] Waiting for process rollout_proc3 to join... [2023-10-14 21:44:04,194][60425] Waiting for process rollout_proc4 to join... [2023-10-14 21:44:04,195][60425] Waiting for process rollout_proc5 to join... [2023-10-14 21:44:04,195][60425] Waiting for process rollout_proc6 to join... [2023-10-14 21:44:04,196][60425] Waiting for process rollout_proc7 to join... [2023-10-14 21:44:04,196][60425] Waiting for process rollout_proc8 to join... [2023-10-14 21:44:04,197][60425] Waiting for process rollout_proc9 to join... [2023-10-14 21:44:04,197][60425] Waiting for process rollout_proc10 to join... [2023-10-14 21:44:04,198][60425] Waiting for process rollout_proc11 to join... [2023-10-14 21:44:04,198][60425] Waiting for process rollout_proc12 to join... [2023-10-14 21:44:04,199][60425] Waiting for process rollout_proc13 to join... [2023-10-14 21:44:04,199][60425] Waiting for process rollout_proc14 to join... [2023-10-14 21:44:04,199][60425] Waiting for process rollout_proc15 to join... [2023-10-14 21:44:04,200][60425] Batcher 0 profile tree view: batching: 167.7907, releasing_batches: 0.0906 [2023-10-14 21:44:04,200][60425] Batcher 1 profile tree view: batching: 168.3423, releasing_batches: 0.0899 [2023-10-14 21:44:04,200][60425] InferenceWorker_p0-w0 profile tree view: wait_policy: 0.0000 wait_policy_total: 2710.2179 update_model: 209.0475 weight_update: 0.0007 one_step: 0.0024 handle_policy_step: 11392.6289 deserialize: 65.5934, stack: 192.4240, obs_to_device_normalize: 2550.4512, forward: 5151.3263, prepare_outputs: 2473.5796, send_messages: 460.0514 [2023-10-14 21:44:04,201][60425] InferenceWorker_p1-w0 profile tree view: wait_policy: 0.0000 wait_policy_total: 2640.1650 update_model: 204.8171 weight_update: 0.0009 one_step: 0.0014 handle_policy_step: 11452.6986 deserialize: 65.0443, stack: 193.8784, obs_to_device_normalize: 2567.2602, forward: 5200.4252, prepare_outputs: 2455.2605, send_messages: 469.1981 [2023-10-14 21:44:04,201][60425] Learner 0 profile tree view: misc: 0.0179, prepare_batch: 269.1621 train: 3623.9578 epoch_init: 0.1877, minibatch_init: 13.3242, losses_postprocess: 889.0283, kl_divergence: 32.2508, update: 386.3817, after_optimizer: 2118.4999 calculate_losses: 167.4489 losses_init: 0.3941, forward_head: 56.1042, bptt_initial: 1.4244, bptt: 1.9732, tail: 38.6835, advantages_returns: 11.2305, losses: 44.0140 [2023-10-14 21:44:04,201][60425] Learner 1 profile tree view: misc: 0.0186, prepare_batch: 269.6281 train: 3588.8269 epoch_init: 0.1870, minibatch_init: 12.9557, losses_postprocess: 885.6367, kl_divergence: 31.5201, update: 382.3021, after_optimizer: 2092.8822 calculate_losses: 166.7033 losses_init: 0.3979, forward_head: 56.0081, bptt_initial: 1.4316, bptt: 2.0523, tail: 38.2380, advantages_returns: 11.1520, losses: 43.6645 [2023-10-14 21:44:04,202][60425] RolloutWorker_w0 profile tree view: wait_for_trajectories: 1.2349, enqueue_policy_requests: 405.1679, process_policy_outputs: 188.0444, env_step: 7866.8206, finalize_trajectories: 3.4325, complete_rollouts: 2.9318 post_env_step: 374.0776 process_env_step: 83.5876 [2023-10-14 21:44:04,202][60425] RolloutWorker_w15 profile tree view: wait_for_trajectories: 1.2162, enqueue_policy_requests: 410.1872, process_policy_outputs: 192.3525, env_step: 7703.3751, finalize_trajectories: 3.4361, complete_rollouts: 2.8917 post_env_step: 379.4705 process_env_step: 84.2997 [2023-10-14 21:44:04,202][60425] Loop Runner_EvtLoop terminating... [2023-10-14 21:44:04,203][60425] Runner profile tree view: main_loop: 15016.2286 [2023-10-14 21:44:04,203][60425] Collected {0: 100171776, 1: 100007936}, FPS: 13330.9