[2023-10-11 19:05:59,964][70582] Saving configuration to ./train_atari/atari_gopher_APPO/config.json... [2023-10-11 19:06:00,281][70582] Rollout worker 0 uses device cpu [2023-10-11 19:06:00,282][70582] Rollout worker 1 uses device cpu [2023-10-11 19:06:00,283][70582] Rollout worker 2 uses device cpu [2023-10-11 19:06:00,283][70582] Rollout worker 3 uses device cpu [2023-10-11 19:06:00,284][70582] Rollout worker 4 uses device cpu [2023-10-11 19:06:00,284][70582] Rollout worker 5 uses device cpu [2023-10-11 19:06:00,285][70582] Rollout worker 6 uses device cpu [2023-10-11 19:06:00,285][70582] Rollout worker 7 uses device cpu [2023-10-11 19:06:00,286][70582] Rollout worker 8 uses device cpu [2023-10-11 19:06:00,286][70582] Rollout worker 9 uses device cpu [2023-10-11 19:06:00,286][70582] Rollout worker 10 uses device cpu [2023-10-11 19:06:00,287][70582] Rollout worker 11 uses device cpu [2023-10-11 19:06:00,287][70582] Rollout worker 12 uses device cpu [2023-10-11 19:06:00,288][70582] Rollout worker 13 uses device cpu [2023-10-11 19:06:00,288][70582] Rollout worker 14 uses device cpu [2023-10-11 19:06:00,288][70582] Rollout worker 15 uses device cpu [2023-10-11 19:06:00,605][70582] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-10-11 19:06:00,605][70582] InferenceWorker_p0-w0: min num requests: 2 [2023-10-11 19:06:00,608][70582] Using GPUs [1] for process 1 (actually maps to GPUs [1]) [2023-10-11 19:06:00,608][70582] InferenceWorker_p1-w0: min num requests: 2 [2023-10-11 19:06:00,654][70582] Starting all processes... [2023-10-11 19:06:00,654][70582] Starting process learner_proc0 [2023-10-11 19:06:02,372][70582] Starting process learner_proc1 [2023-10-11 19:06:02,375][71353] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-10-11 19:06:02,375][71353] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 [2023-10-11 19:06:02,394][71353] Num visible devices: 1 [2023-10-11 19:06:02,416][71353] Setting fixed seed 1234 [2023-10-11 19:06:02,417][71353] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-10-11 19:06:02,417][71353] Initializing actor-critic model on device cuda:0 [2023-10-11 19:06:02,417][71353] RunningMeanStd input shape: (4, 84, 84) [2023-10-11 19:06:02,418][71353] RunningMeanStd input shape: (1,) [2023-10-11 19:06:02,429][71353] ConvEncoder: input_channels=4 [2023-10-11 19:06:02,612][71353] Conv encoder output size: 512 [2023-10-11 19:06:02,614][71353] Created Actor Critic model with architecture: [2023-10-11 19:06:02,614][71353] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): MultiInputEncoder( (encoders): ModuleDict( (obs): ConvEncoder( (enc): RecursiveScriptModule( original_name=ConvEncoderImpl (conv_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Conv2d) (1): RecursiveScriptModule(original_name=ReLU) (2): RecursiveScriptModule(original_name=Conv2d) (3): RecursiveScriptModule(original_name=ReLU) (4): RecursiveScriptModule(original_name=Conv2d) (5): RecursiveScriptModule(original_name=ReLU) ) (mlp_layers): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=ReLU) ) ) ) ) ) (core): ModelCoreIdentity() (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=512, out_features=1, bias=True) (action_parameterization): ActionParameterizationDefault( (distribution_linear): Linear(in_features=512, out_features=8, bias=True) ) ) [2023-10-11 19:06:03,161][71353] Using optimizer [2023-10-11 19:06:03,162][71353] No checkpoints found [2023-10-11 19:06:03,162][71353] Did not load from checkpoint, starting from scratch! [2023-10-11 19:06:03,162][71353] Initialized policy 0 weights for model version 0 [2023-10-11 19:06:03,164][71353] LearnerWorker_p0 finished initialization! [2023-10-11 19:06:03,164][71353] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-10-11 19:06:04,164][70582] Starting all processes... [2023-10-11 19:06:04,167][71431] Using GPUs [1] for process 1 (actually maps to GPUs [1]) [2023-10-11 19:06:04,168][71431] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [1]) for learning process 1 [2023-10-11 19:06:04,173][70582] Starting process inference_proc0-0 [2023-10-11 19:06:04,174][70582] Starting process inference_proc1-0 [2023-10-11 19:06:04,174][70582] Starting process rollout_proc0 [2023-10-11 19:06:04,186][71431] Num visible devices: 1 [2023-10-11 19:06:04,174][70582] Starting process rollout_proc1 [2023-10-11 19:06:04,204][71431] Setting fixed seed 1234 [2023-10-11 19:06:04,174][70582] Starting process rollout_proc2 [2023-10-11 19:06:04,205][71431] Using GPUs [0] for process 1 (actually maps to GPUs [1]) [2023-10-11 19:06:04,205][71431] Initializing actor-critic model on device cuda:0 [2023-10-11 19:06:04,206][71431] RunningMeanStd input shape: (4, 84, 84) [2023-10-11 19:06:04,206][71431] RunningMeanStd input shape: (1,) [2023-10-11 19:06:04,175][70582] Starting process rollout_proc3 [2023-10-11 19:06:04,180][70582] Starting process rollout_proc4 [2023-10-11 19:06:04,180][70582] Starting process rollout_proc5 [2023-10-11 19:06:04,181][70582] Starting process rollout_proc6 [2023-10-11 19:06:04,181][70582] Starting process rollout_proc7 [2023-10-11 19:06:04,218][71431] ConvEncoder: input_channels=4 [2023-10-11 19:06:04,182][70582] Starting process rollout_proc8 [2023-10-11 19:06:04,182][70582] Starting process rollout_proc9 [2023-10-11 19:06:04,186][70582] Starting process rollout_proc10 [2023-10-11 19:06:04,188][70582] Starting process rollout_proc11 [2023-10-11 19:06:04,189][70582] Starting process rollout_proc12 [2023-10-11 19:06:04,204][70582] Starting process rollout_proc13 [2023-10-11 19:06:04,683][71431] Conv encoder output size: 512 [2023-10-11 19:06:04,686][71431] Created Actor Critic model with architecture: [2023-10-11 19:06:04,686][71431] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): MultiInputEncoder( (encoders): ModuleDict( (obs): ConvEncoder( (enc): RecursiveScriptModule( original_name=ConvEncoderImpl (conv_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Conv2d) (1): RecursiveScriptModule(original_name=ReLU) (2): RecursiveScriptModule(original_name=Conv2d) (3): RecursiveScriptModule(original_name=ReLU) (4): RecursiveScriptModule(original_name=Conv2d) (5): RecursiveScriptModule(original_name=ReLU) ) (mlp_layers): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=ReLU) ) ) ) ) ) (core): ModelCoreIdentity() (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=512, out_features=1, bias=True) (action_parameterization): ActionParameterizationDefault( (distribution_linear): Linear(in_features=512, out_features=8, bias=True) ) ) [2023-10-11 19:06:05,547][71431] Using optimizer [2023-10-11 19:06:05,548][71431] No checkpoints found [2023-10-11 19:06:05,548][71431] Did not load from checkpoint, starting from scratch! [2023-10-11 19:06:05,548][71431] Initialized policy 1 weights for model version 0 [2023-10-11 19:06:05,550][71431] LearnerWorker_p1 finished initialization! [2023-10-11 19:06:05,551][71431] Using GPUs [0] for process 1 (actually maps to GPUs [1]) [2023-10-11 19:06:06,381][70582] Starting process rollout_proc14 [2023-10-11 19:06:06,386][71637] Worker 2 uses CPU cores [4, 5] [2023-10-11 19:06:06,394][70582] Starting process rollout_proc15 [2023-10-11 19:06:06,399][71635] Using GPUs [1] for process 1 (actually maps to GPUs [1]) [2023-10-11 19:06:06,400][71635] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [1]) for inference process 1 [2023-10-11 19:06:06,417][71642] Worker 6 uses CPU cores [12, 13] [2023-10-11 19:06:06,418][71635] Num visible devices: 1 [2023-10-11 19:06:06,440][71601] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-10-11 19:06:06,441][71601] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 [2023-10-11 19:06:06,479][71640] Worker 4 uses CPU cores [8, 9] [2023-10-11 19:06:06,496][71601] Num visible devices: 1 [2023-10-11 19:06:06,622][71641] Worker 5 uses CPU cores [10, 11] [2023-10-11 19:06:06,663][71639] Worker 3 uses CPU cores [6, 7] [2023-10-11 19:06:06,713][71638] Worker 1 uses CPU cores [2, 3] [2023-10-11 19:06:06,716][71644] Worker 8 uses CPU cores [16, 17] [2023-10-11 19:06:06,743][71643] Worker 7 uses CPU cores [14, 15] [2023-10-11 19:06:06,762][71648] Worker 12 uses CPU cores [24, 25] [2023-10-11 19:06:06,763][71634] Worker 0 uses CPU cores [0, 1] [2023-10-11 19:06:06,976][71645] Worker 9 uses CPU cores [18, 19] [2023-10-11 19:06:07,038][71649] Worker 13 uses CPU cores [26, 27] [2023-10-11 19:06:07,073][71647] Worker 10 uses CPU cores [20, 21] [2023-10-11 19:06:07,146][71635] RunningMeanStd input shape: (4, 84, 84) [2023-10-11 19:06:07,146][71635] RunningMeanStd input shape: (1,) [2023-10-11 19:06:07,158][71635] ConvEncoder: input_channels=4 [2023-10-11 19:06:07,164][71646] Worker 11 uses CPU cores [22, 23] [2023-10-11 19:06:07,252][71601] RunningMeanStd input shape: (4, 84, 84) [2023-10-11 19:06:07,253][71601] RunningMeanStd input shape: (1,) [2023-10-11 19:06:07,264][71601] ConvEncoder: input_channels=4 [2023-10-11 19:06:07,294][71635] Conv encoder output size: 512 [2023-10-11 19:06:07,371][71601] Conv encoder output size: 512 [2023-10-11 19:06:08,355][72289] Worker 14 uses CPU cores [28, 29] [2023-10-11 19:06:08,452][70582] Inference worker 1-0 is ready! [2023-10-11 19:06:08,453][72321] Worker 15 uses CPU cores [30, 31] [2023-10-11 19:06:08,453][70582] Inference worker 0-0 is ready! [2023-10-11 19:06:08,454][70582] All inference workers are ready! Signal rollout workers to start! [2023-10-11 19:06:08,455][71634] EnvRunner 0-0 uses policy 0 [2023-10-11 19:06:08,455][71642] EnvRunner 6-0 uses policy 0 [2023-10-11 19:06:08,455][71637] EnvRunner 2-0 uses policy 0 [2023-10-11 19:06:08,455][71644] EnvRunner 8-0 uses policy 0 [2023-10-11 19:06:08,455][71649] EnvRunner 13-0 uses policy 1 [2023-10-11 19:06:08,455][71640] EnvRunner 4-0 uses policy 0 [2023-10-11 19:06:08,455][71638] EnvRunner 1-0 uses policy 1 [2023-10-11 19:06:08,455][71641] EnvRunner 5-0 uses policy 1 [2023-10-11 19:06:08,455][71647] EnvRunner 10-0 uses policy 0 [2023-10-11 19:06:08,455][71645] EnvRunner 9-0 uses policy 1 [2023-10-11 19:06:08,455][70582] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan, 1: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-10-11 19:06:08,455][71643] EnvRunner 7-0 uses policy 1 [2023-10-11 19:06:08,455][71648] EnvRunner 12-0 uses policy 0 [2023-10-11 19:06:08,455][71639] EnvRunner 3-0 uses policy 1 [2023-10-11 19:06:08,455][71646] EnvRunner 11-0 uses policy 1 [2023-10-11 19:06:08,528][72289] EnvRunner 14-0 uses policy 0 [2023-10-11 19:06:08,560][72321] EnvRunner 15-0 uses policy 1 [2023-10-11 19:06:10,592][70582] Heartbeat connected on Batcher_0 [2023-10-11 19:06:10,595][70582] Heartbeat connected on LearnerWorker_p0 [2023-10-11 19:06:10,598][70582] Heartbeat connected on Batcher_1 [2023-10-11 19:06:10,601][70582] Heartbeat connected on LearnerWorker_p1 [2023-10-11 19:06:10,608][70582] Heartbeat connected on InferenceWorker_p0-w0 [2023-10-11 19:06:10,613][70582] Heartbeat connected on InferenceWorker_p1-w0 [2023-10-11 19:06:10,615][70582] Heartbeat connected on RolloutWorker_w0 [2023-10-11 19:06:10,616][70582] Heartbeat connected on RolloutWorker_w1 [2023-10-11 19:06:10,617][70582] Heartbeat connected on RolloutWorker_w2 [2023-10-11 19:06:10,621][70582] Heartbeat connected on RolloutWorker_w3 [2023-10-11 19:06:10,624][70582] Heartbeat connected on RolloutWorker_w4 [2023-10-11 19:06:10,628][70582] Heartbeat connected on RolloutWorker_w5 [2023-10-11 19:06:10,631][70582] Heartbeat connected on RolloutWorker_w6 [2023-10-11 19:06:10,632][70582] Heartbeat connected on RolloutWorker_w7 [2023-10-11 19:06:10,636][70582] Heartbeat connected on RolloutWorker_w8 [2023-10-11 19:06:10,638][70582] Heartbeat connected on RolloutWorker_w9 [2023-10-11 19:06:10,639][70582] Heartbeat connected on RolloutWorker_w10 [2023-10-11 19:06:10,641][70582] Heartbeat connected on RolloutWorker_w11 [2023-10-11 19:06:10,647][70582] Heartbeat connected on RolloutWorker_w13 [2023-10-11 19:06:10,649][70582] Heartbeat connected on RolloutWorker_w12 [2023-10-11 19:06:10,651][70582] Heartbeat connected on RolloutWorker_w14 [2023-10-11 19:06:10,653][70582] Heartbeat connected on RolloutWorker_w15 [2023-10-11 19:06:11,034][70582] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 401.7, 1: 311.0. Samples: 1838. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-10-11 19:06:16,034][70582] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 989.4, 1: 946.4. Samples: 14670. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-10-11 19:06:16,034][70582] Avg episode reward: [(0, '6.895'), (1, '4.667')] [2023-10-11 19:06:18,040][71601] Updated weights for policy 0, policy_version 10 (0.0008) [2023-10-11 19:06:18,240][71635] Updated weights for policy 1, policy_version 10 (0.0008) [2023-10-11 19:06:18,396][71601] Updated weights for policy 0, policy_version 20 (0.0009) [2023-10-11 19:06:18,587][71635] Updated weights for policy 1, policy_version 20 (0.0007) [2023-10-11 19:06:18,763][71601] Updated weights for policy 0, policy_version 30 (0.0007) [2023-10-11 19:06:18,952][71635] Updated weights for policy 1, policy_version 30 (0.0009) [2023-10-11 19:06:21,034][70582] Fps is (10 sec: 6553.8, 60 sec: 5210.2, 300 sec: 5210.2). Total num frames: 65536. Throughput: 0: 1300.6, 1: 1266.6. Samples: 32292. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-11 19:06:21,034][70582] Avg episode reward: [(0, '5.348'), (1, '3.413')] [2023-10-11 19:06:21,230][71601] Updated weights for policy 0, policy_version 40 (0.0007) [2023-10-11 19:06:21,253][71635] Updated weights for policy 1, policy_version 40 (0.0007) [2023-10-11 19:06:21,599][71601] Updated weights for policy 0, policy_version 50 (0.0008) [2023-10-11 19:06:21,609][71635] Updated weights for policy 1, policy_version 50 (0.0008) [2023-10-11 19:06:21,969][71601] Updated weights for policy 0, policy_version 60 (0.0007) [2023-10-11 19:06:21,972][71635] Updated weights for policy 1, policy_version 60 (0.0009) [2023-10-11 19:06:24,999][71601] Updated weights for policy 0, policy_version 70 (0.0007) [2023-10-11 19:06:25,368][71601] Updated weights for policy 0, policy_version 80 (0.0008) [2023-10-11 19:06:25,408][71635] Updated weights for policy 1, policy_version 70 (0.0008) [2023-10-11 19:06:25,733][71601] Updated weights for policy 0, policy_version 90 (0.0007) [2023-10-11 19:06:25,774][71635] Updated weights for policy 1, policy_version 80 (0.0007) [2023-10-11 19:06:26,034][70582] Fps is (10 sec: 16383.8, 60 sec: 9320.4, 300 sec: 9320.4). Total num frames: 163840. Throughput: 0: 1530.3, 1: 1518.7. Samples: 53596. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:06:26,035][70582] Avg episode reward: [(0, '5.317'), (1, '4.015')] [2023-10-11 19:06:26,131][71635] Updated weights for policy 1, policy_version 90 (0.0008) [2023-10-11 19:06:29,156][71601] Updated weights for policy 0, policy_version 100 (0.0008) [2023-10-11 19:06:29,483][71635] Updated weights for policy 1, policy_version 100 (0.0007) [2023-10-11 19:06:29,523][71601] Updated weights for policy 0, policy_version 110 (0.0008) [2023-10-11 19:06:29,849][71635] Updated weights for policy 1, policy_version 110 (0.0007) [2023-10-11 19:06:29,886][71601] Updated weights for policy 0, policy_version 120 (0.0008) [2023-10-11 19:06:30,220][71635] Updated weights for policy 1, policy_version 120 (0.0007) [2023-10-11 19:06:31,034][70582] Fps is (10 sec: 19660.5, 60 sec: 11610.3, 300 sec: 11610.3). Total num frames: 262144. Throughput: 0: 1438.7, 1: 1415.3. Samples: 64440. Policy #0 lag: (min: 22.0, avg: 30.0, max: 54.0) [2023-10-11 19:06:31,035][70582] Avg episode reward: [(0, '5.744'), (1, '4.065')] [2023-10-11 19:06:31,036][71431] Saving new best policy, reward=4.065! [2023-10-11 19:06:31,036][71353] Saving new best policy, reward=5.744! [2023-10-11 19:06:33,697][71601] Updated weights for policy 0, policy_version 130 (0.0009) [2023-10-11 19:06:34,018][71635] Updated weights for policy 1, policy_version 130 (0.0008) [2023-10-11 19:06:34,070][71601] Updated weights for policy 0, policy_version 140 (0.0010) [2023-10-11 19:06:34,371][71635] Updated weights for policy 1, policy_version 140 (0.0009) [2023-10-11 19:06:34,439][71601] Updated weights for policy 0, policy_version 150 (0.0008) [2023-10-11 19:06:34,745][71635] Updated weights for policy 1, policy_version 150 (0.0008) [2023-10-11 19:06:34,808][71601] Updated weights for policy 0, policy_version 160 (0.0008) [2023-10-11 19:06:35,106][71635] Updated weights for policy 1, policy_version 160 (0.0010) [2023-10-11 19:06:36,034][70582] Fps is (10 sec: 16384.2, 60 sec: 11881.7, 300 sec: 11881.7). Total num frames: 327680. Throughput: 0: 1548.7, 1: 1542.4. Samples: 85248. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) [2023-10-11 19:06:36,034][70582] Avg episode reward: [(0, '6.210'), (1, '4.840')] [2023-10-11 19:06:36,035][71353] Saving new best policy, reward=6.210! [2023-10-11 19:06:36,035][71431] Saving new best policy, reward=4.840! [2023-10-11 19:06:38,540][71601] Updated weights for policy 0, policy_version 170 (0.0007) [2023-10-11 19:06:38,826][71635] Updated weights for policy 1, policy_version 170 (0.0007) [2023-10-11 19:06:38,895][71601] Updated weights for policy 0, policy_version 180 (0.0009) [2023-10-11 19:06:39,181][71635] Updated weights for policy 1, policy_version 180 (0.0007) [2023-10-11 19:06:39,263][71601] Updated weights for policy 0, policy_version 190 (0.0008) [2023-10-11 19:06:39,549][71635] Updated weights for policy 1, policy_version 190 (0.0009) [2023-10-11 19:06:41,034][70582] Fps is (10 sec: 13107.4, 60 sec: 12069.8, 300 sec: 12069.8). Total num frames: 393216. Throughput: 0: 1639.5, 1: 1626.8. Samples: 106414. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-10-11 19:06:41,034][70582] Avg episode reward: [(0, '6.540'), (1, '5.590')] [2023-10-11 19:06:41,039][71353] Saving new best policy, reward=6.540! [2023-10-11 19:06:41,040][71431] Saving new best policy, reward=5.590! [2023-10-11 19:06:43,047][71601] Updated weights for policy 0, policy_version 200 (0.0009) [2023-10-11 19:06:43,296][71635] Updated weights for policy 1, policy_version 200 (0.0007) [2023-10-11 19:06:43,420][71601] Updated weights for policy 0, policy_version 210 (0.0007) [2023-10-11 19:06:43,661][71635] Updated weights for policy 1, policy_version 210 (0.0008) [2023-10-11 19:06:43,780][71601] Updated weights for policy 0, policy_version 220 (0.0008) [2023-10-11 19:06:44,014][71635] Updated weights for policy 1, policy_version 220 (0.0009) [2023-10-11 19:06:46,034][70582] Fps is (10 sec: 13107.3, 60 sec: 12207.8, 300 sec: 12207.8). Total num frames: 458752. Throughput: 0: 1576.0, 1: 1565.8. Samples: 118064. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:06:46,034][70582] Avg episode reward: [(0, '6.700'), (1, '5.640')] [2023-10-11 19:06:46,035][71353] Saving new best policy, reward=6.700! [2023-10-11 19:06:46,035][71431] Saving new best policy, reward=5.640! [2023-10-11 19:06:47,383][71601] Updated weights for policy 0, policy_version 230 (0.0009) [2023-10-11 19:06:47,756][71601] Updated weights for policy 0, policy_version 240 (0.0009) [2023-10-11 19:06:47,794][71635] Updated weights for policy 1, policy_version 230 (0.0009) [2023-10-11 19:06:48,125][71601] Updated weights for policy 0, policy_version 250 (0.0008) [2023-10-11 19:06:48,159][71635] Updated weights for policy 1, policy_version 240 (0.0007) [2023-10-11 19:06:48,515][71635] Updated weights for policy 1, policy_version 250 (0.0008) [2023-10-11 19:06:51,034][70582] Fps is (10 sec: 13107.1, 60 sec: 12313.4, 300 sec: 12313.4). Total num frames: 524288. Throughput: 0: 1645.1, 1: 1622.2. Samples: 139118. Policy #0 lag: (min: 4.0, avg: 10.7, max: 36.0) [2023-10-11 19:06:51,034][70582] Avg episode reward: [(0, '6.880'), (1, '6.460')] [2023-10-11 19:06:51,035][71353] Saving new best policy, reward=6.880! [2023-10-11 19:06:51,035][71431] Saving new best policy, reward=6.460! [2023-10-11 19:06:51,862][71601] Updated weights for policy 0, policy_version 260 (0.0008) [2023-10-11 19:06:52,236][71601] Updated weights for policy 0, policy_version 270 (0.0008) [2023-10-11 19:06:52,244][71635] Updated weights for policy 1, policy_version 260 (0.0008) [2023-10-11 19:06:52,603][71601] Updated weights for policy 0, policy_version 280 (0.0008) [2023-10-11 19:06:52,614][71635] Updated weights for policy 1, policy_version 270 (0.0008) [2023-10-11 19:06:52,976][71635] Updated weights for policy 1, policy_version 280 (0.0008) [2023-10-11 19:06:56,034][70582] Fps is (10 sec: 13107.2, 60 sec: 12396.9, 300 sec: 12396.9). Total num frames: 589824. Throughput: 0: 1786.1, 1: 1767.5. Samples: 161748. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-10-11 19:06:56,034][70582] Avg episode reward: [(0, '7.660'), (1, '6.180')] [2023-10-11 19:06:56,039][71353] Saving new best policy, reward=7.660! [2023-10-11 19:06:56,361][71601] Updated weights for policy 0, policy_version 290 (0.0009) [2023-10-11 19:06:56,574][71635] Updated weights for policy 1, policy_version 290 (0.0009) [2023-10-11 19:06:56,736][71601] Updated weights for policy 0, policy_version 300 (0.0008) [2023-10-11 19:06:56,934][71635] Updated weights for policy 1, policy_version 300 (0.0007) [2023-10-11 19:06:57,102][71601] Updated weights for policy 0, policy_version 310 (0.0008) [2023-10-11 19:06:57,301][71635] Updated weights for policy 1, policy_version 310 (0.0007) [2023-10-11 19:06:57,476][71601] Updated weights for policy 0, policy_version 320 (0.0009) [2023-10-11 19:06:57,669][71635] Updated weights for policy 1, policy_version 320 (0.0009) [2023-10-11 19:07:01,034][70582] Fps is (10 sec: 13106.8, 60 sec: 12464.3, 300 sec: 12464.3). Total num frames: 655360. Throughput: 0: 1750.6, 1: 1738.0. Samples: 171658. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-11 19:07:01,035][70582] Avg episode reward: [(0, '8.360'), (1, '6.630')] [2023-10-11 19:07:01,150][71601] Updated weights for policy 0, policy_version 330 (0.0010) [2023-10-11 19:07:01,324][71635] Updated weights for policy 1, policy_version 330 (0.0007) [2023-10-11 19:07:01,513][71601] Updated weights for policy 0, policy_version 340 (0.0008) [2023-10-11 19:07:01,688][71635] Updated weights for policy 1, policy_version 340 (0.0007) [2023-10-11 19:07:01,880][71601] Updated weights for policy 0, policy_version 350 (0.0009) [2023-10-11 19:07:01,952][71353] Saving new best policy, reward=8.360! [2023-10-11 19:07:02,060][71635] Updated weights for policy 1, policy_version 350 (0.0009) [2023-10-11 19:07:02,131][71431] Saving new best policy, reward=6.630! [2023-10-11 19:07:05,625][71601] Updated weights for policy 0, policy_version 360 (0.0008) [2023-10-11 19:07:05,832][71635] Updated weights for policy 1, policy_version 360 (0.0008) [2023-10-11 19:07:06,004][71601] Updated weights for policy 0, policy_version 370 (0.0007) [2023-10-11 19:07:06,034][70582] Fps is (10 sec: 13107.2, 60 sec: 12520.2, 300 sec: 12520.2). Total num frames: 720896. Throughput: 0: 1804.5, 1: 1793.3. Samples: 194196. Policy #0 lag: (min: 26.0, avg: 26.5, max: 41.0) [2023-10-11 19:07:06,034][70582] Avg episode reward: [(0, '8.780'), (1, '6.480')] [2023-10-11 19:07:06,202][71635] Updated weights for policy 1, policy_version 370 (0.0008) [2023-10-11 19:07:06,373][71601] Updated weights for policy 0, policy_version 380 (0.0009) [2023-10-11 19:07:06,528][71353] Saving new best policy, reward=8.780! [2023-10-11 19:07:06,557][71635] Updated weights for policy 1, policy_version 380 (0.0008) [2023-10-11 19:07:10,010][71601] Updated weights for policy 0, policy_version 390 (0.0007) [2023-10-11 19:07:10,323][71635] Updated weights for policy 1, policy_version 390 (0.0008) [2023-10-11 19:07:10,368][71601] Updated weights for policy 0, policy_version 400 (0.0008) [2023-10-11 19:07:10,688][71635] Updated weights for policy 1, policy_version 400 (0.0008) [2023-10-11 19:07:10,748][71601] Updated weights for policy 0, policy_version 410 (0.0009) [2023-10-11 19:07:11,034][70582] Fps is (10 sec: 16384.2, 60 sec: 13653.3, 300 sec: 13090.7). Total num frames: 819200. Throughput: 0: 1801.5, 1: 1800.1. Samples: 215668. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-11 19:07:11,035][70582] Avg episode reward: [(0, '8.500'), (1, '6.780')] [2023-10-11 19:07:11,050][71635] Updated weights for policy 1, policy_version 410 (0.0009) [2023-10-11 19:07:11,266][71431] Saving new best policy, reward=6.780! [2023-10-11 19:07:14,419][71601] Updated weights for policy 0, policy_version 420 (0.0008) [2023-10-11 19:07:14,782][71635] Updated weights for policy 1, policy_version 420 (0.0007) [2023-10-11 19:07:14,787][71601] Updated weights for policy 0, policy_version 430 (0.0010) [2023-10-11 19:07:15,153][71635] Updated weights for policy 1, policy_version 430 (0.0009) [2023-10-11 19:07:15,155][71601] Updated weights for policy 0, policy_version 440 (0.0007) [2023-10-11 19:07:15,519][71635] Updated weights for policy 1, policy_version 440 (0.0007) [2023-10-11 19:07:16,034][70582] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 13576.9). Total num frames: 917504. Throughput: 0: 1807.0, 1: 1799.9. Samples: 226752. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-11 19:07:16,034][70582] Avg episode reward: [(0, '8.460'), (1, '6.800')] [2023-10-11 19:07:16,035][71431] Saving new best policy, reward=6.800! [2023-10-11 19:07:18,871][71601] Updated weights for policy 0, policy_version 450 (0.0008) [2023-10-11 19:07:19,225][71635] Updated weights for policy 1, policy_version 450 (0.0007) [2023-10-11 19:07:19,248][71601] Updated weights for policy 0, policy_version 460 (0.0009) [2023-10-11 19:07:19,594][71635] Updated weights for policy 1, policy_version 460 (0.0007) [2023-10-11 19:07:19,610][71601] Updated weights for policy 0, policy_version 470 (0.0009) [2023-10-11 19:07:19,954][71635] Updated weights for policy 1, policy_version 470 (0.0008) [2023-10-11 19:07:19,989][71601] Updated weights for policy 0, policy_version 480 (0.0009) [2023-10-11 19:07:20,327][71635] Updated weights for policy 1, policy_version 480 (0.0010) [2023-10-11 19:07:21,034][70582] Fps is (10 sec: 16384.3, 60 sec: 15291.7, 300 sec: 13544.5). Total num frames: 983040. Throughput: 0: 1815.4, 1: 1812.4. Samples: 248500. Policy #0 lag: (min: 1.0, avg: 9.9, max: 33.0) [2023-10-11 19:07:21,034][70582] Avg episode reward: [(0, '8.810'), (1, '6.920')] [2023-10-11 19:07:21,035][71353] Saving new best policy, reward=8.810! [2023-10-11 19:07:21,035][71431] Saving new best policy, reward=6.920! [2023-10-11 19:07:23,647][71601] Updated weights for policy 0, policy_version 490 (0.0009) [2023-10-11 19:07:23,942][71635] Updated weights for policy 1, policy_version 490 (0.0008) [2023-10-11 19:07:24,012][71601] Updated weights for policy 0, policy_version 500 (0.0008) [2023-10-11 19:07:24,313][71635] Updated weights for policy 1, policy_version 500 (0.0009) [2023-10-11 19:07:24,389][71601] Updated weights for policy 0, policy_version 510 (0.0008) [2023-10-11 19:07:24,681][71635] Updated weights for policy 1, policy_version 510 (0.0007) [2023-10-11 19:07:26,034][70582] Fps is (10 sec: 13106.7, 60 sec: 14745.5, 300 sec: 13516.3). Total num frames: 1048576. Throughput: 0: 1814.1, 1: 1803.8. Samples: 269224. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-11 19:07:26,036][70582] Avg episode reward: [(0, '9.120'), (1, '6.860')] [2023-10-11 19:07:26,047][71353] Saving new best policy, reward=9.120! [2023-10-11 19:07:28,139][71601] Updated weights for policy 0, policy_version 520 (0.0009) [2023-10-11 19:07:28,337][71635] Updated weights for policy 1, policy_version 520 (0.0008) [2023-10-11 19:07:28,502][71601] Updated weights for policy 0, policy_version 530 (0.0009) [2023-10-11 19:07:28,707][71635] Updated weights for policy 1, policy_version 530 (0.0007) [2023-10-11 19:07:28,870][71601] Updated weights for policy 0, policy_version 540 (0.0009) [2023-10-11 19:07:29,070][71635] Updated weights for policy 1, policy_version 540 (0.0008) [2023-10-11 19:07:31,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 13491.5). Total num frames: 1114112. Throughput: 0: 1809.7, 1: 1810.7. Samples: 280984. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:07:31,035][70582] Avg episode reward: [(0, '9.580'), (1, '7.780')] [2023-10-11 19:07:31,036][71353] Saving new best policy, reward=9.580! [2023-10-11 19:07:31,037][71431] Saving new best policy, reward=7.780! [2023-10-11 19:07:32,731][71601] Updated weights for policy 0, policy_version 550 (0.0008) [2023-10-11 19:07:32,875][71635] Updated weights for policy 1, policy_version 550 (0.0008) [2023-10-11 19:07:33,114][71601] Updated weights for policy 0, policy_version 560 (0.0008) [2023-10-11 19:07:33,238][71635] Updated weights for policy 1, policy_version 560 (0.0007) [2023-10-11 19:07:33,489][71601] Updated weights for policy 0, policy_version 570 (0.0008) [2023-10-11 19:07:33,606][71635] Updated weights for policy 1, policy_version 570 (0.0007) [2023-10-11 19:07:36,034][70582] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 13469.6). Total num frames: 1179648. Throughput: 0: 1795.5, 1: 1805.1. Samples: 301142. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-10-11 19:07:36,034][70582] Avg episode reward: [(0, '9.670'), (1, '7.730')] [2023-10-11 19:07:36,035][71353] Saving new best policy, reward=9.670! [2023-10-11 19:07:37,178][71601] Updated weights for policy 0, policy_version 580 (0.0008) [2023-10-11 19:07:37,427][71635] Updated weights for policy 1, policy_version 580 (0.0008) [2023-10-11 19:07:37,550][71601] Updated weights for policy 0, policy_version 590 (0.0010) [2023-10-11 19:07:37,782][71635] Updated weights for policy 1, policy_version 590 (0.0008) [2023-10-11 19:07:37,916][71601] Updated weights for policy 0, policy_version 600 (0.0007) [2023-10-11 19:07:38,145][71635] Updated weights for policy 1, policy_version 600 (0.0009) [2023-10-11 19:07:41,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13450.0). Total num frames: 1245184. Throughput: 0: 1792.0, 1: 1800.1. Samples: 323394. Policy #0 lag: (min: 17.0, avg: 20.4, max: 49.0) [2023-10-11 19:07:41,034][70582] Avg episode reward: [(0, '9.120'), (1, '8.300')] [2023-10-11 19:07:41,042][71431] Saving new best policy, reward=8.300! [2023-10-11 19:07:41,915][71601] Updated weights for policy 0, policy_version 610 (0.0010) [2023-10-11 19:07:41,981][71635] Updated weights for policy 1, policy_version 610 (0.0007) [2023-10-11 19:07:42,312][71601] Updated weights for policy 0, policy_version 620 (0.0007) [2023-10-11 19:07:42,357][71635] Updated weights for policy 1, policy_version 620 (0.0009) [2023-10-11 19:07:42,681][71601] Updated weights for policy 0, policy_version 630 (0.0007) [2023-10-11 19:07:42,724][71635] Updated weights for policy 1, policy_version 630 (0.0007) [2023-10-11 19:07:43,063][71601] Updated weights for policy 0, policy_version 640 (0.0007) [2023-10-11 19:07:43,089][71635] Updated weights for policy 1, policy_version 640 (0.0008) [2023-10-11 19:07:46,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 13432.4). Total num frames: 1310720. Throughput: 0: 1786.4, 1: 1796.8. Samples: 332906. Policy #0 lag: (min: 28.0, avg: 32.6, max: 60.0) [2023-10-11 19:07:46,035][70582] Avg episode reward: [(0, '8.910'), (1, '9.210')] [2023-10-11 19:07:46,037][71431] Saving new best policy, reward=9.210! [2023-10-11 19:07:46,683][71601] Updated weights for policy 0, policy_version 650 (0.0008) [2023-10-11 19:07:46,930][71635] Updated weights for policy 1, policy_version 650 (0.0008) [2023-10-11 19:07:47,057][71601] Updated weights for policy 0, policy_version 660 (0.0007) [2023-10-11 19:07:47,290][71635] Updated weights for policy 1, policy_version 660 (0.0008) [2023-10-11 19:07:47,418][71601] Updated weights for policy 0, policy_version 670 (0.0007) [2023-10-11 19:07:47,657][71635] Updated weights for policy 1, policy_version 670 (0.0007) [2023-10-11 19:07:51,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13416.6). Total num frames: 1376256. Throughput: 0: 1785.6, 1: 1793.8. Samples: 355268. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-11 19:07:51,034][70582] Avg episode reward: [(0, '9.750'), (1, '9.850')] [2023-10-11 19:07:51,192][71601] Updated weights for policy 0, policy_version 680 (0.0007) [2023-10-11 19:07:51,377][71635] Updated weights for policy 1, policy_version 680 (0.0009) [2023-10-11 19:07:51,565][71601] Updated weights for policy 0, policy_version 690 (0.0007) [2023-10-11 19:07:51,743][71635] Updated weights for policy 1, policy_version 690 (0.0008) [2023-10-11 19:07:51,934][71601] Updated weights for policy 0, policy_version 700 (0.0007) [2023-10-11 19:07:52,073][71353] Saving new best policy, reward=9.750! [2023-10-11 19:07:52,120][71635] Updated weights for policy 1, policy_version 700 (0.0009) [2023-10-11 19:07:52,264][71431] Saving new best policy, reward=9.850! [2023-10-11 19:07:55,568][71601] Updated weights for policy 0, policy_version 710 (0.0007) [2023-10-11 19:07:55,937][71601] Updated weights for policy 0, policy_version 720 (0.0008) [2023-10-11 19:07:56,030][71635] Updated weights for policy 1, policy_version 710 (0.0008) [2023-10-11 19:07:56,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13402.2). Total num frames: 1441792. Throughput: 0: 1804.0, 1: 1798.4. Samples: 377772. Policy #0 lag: (min: 15.0, avg: 19.1, max: 47.0) [2023-10-11 19:07:56,034][70582] Avg episode reward: [(0, '9.870'), (1, '10.530')] [2023-10-11 19:07:56,314][71601] Updated weights for policy 0, policy_version 730 (0.0009) [2023-10-11 19:07:56,396][71635] Updated weights for policy 1, policy_version 720 (0.0008) [2023-10-11 19:07:56,526][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000000736_753664.pth... [2023-10-11 19:07:56,572][71353] Saving new best policy, reward=9.870! [2023-10-11 19:07:56,759][71635] Updated weights for policy 1, policy_version 730 (0.0008) [2023-10-11 19:07:56,980][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000000736_753664.pth... [2023-10-11 19:07:57,011][71431] Saving new best policy, reward=10.530! [2023-10-11 19:08:00,035][71601] Updated weights for policy 0, policy_version 740 (0.0007) [2023-10-11 19:08:00,410][71601] Updated weights for policy 0, policy_version 750 (0.0007) [2023-10-11 19:08:00,546][71635] Updated weights for policy 1, policy_version 740 (0.0008) [2023-10-11 19:08:00,788][71601] Updated weights for policy 0, policy_version 760 (0.0008) [2023-10-11 19:08:00,912][71635] Updated weights for policy 1, policy_version 750 (0.0007) [2023-10-11 19:08:01,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13389.1). Total num frames: 1507328. Throughput: 0: 1786.7, 1: 1791.2. Samples: 387758. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-11 19:08:01,034][70582] Avg episode reward: [(0, '10.690'), (1, '10.800')] [2023-10-11 19:08:01,086][71353] Saving new best policy, reward=10.690! [2023-10-11 19:08:01,282][71635] Updated weights for policy 1, policy_version 760 (0.0007) [2023-10-11 19:08:01,569][71431] Saving new best policy, reward=10.800! [2023-10-11 19:08:04,630][71601] Updated weights for policy 0, policy_version 770 (0.0008) [2023-10-11 19:08:05,004][71601] Updated weights for policy 0, policy_version 780 (0.0008) [2023-10-11 19:08:05,184][71635] Updated weights for policy 1, policy_version 770 (0.0007) [2023-10-11 19:08:05,379][71601] Updated weights for policy 0, policy_version 790 (0.0008) [2023-10-11 19:08:05,552][71635] Updated weights for policy 1, policy_version 780 (0.0008) [2023-10-11 19:08:05,758][71601] Updated weights for policy 0, policy_version 800 (0.0009) [2023-10-11 19:08:05,926][71635] Updated weights for policy 1, policy_version 790 (0.0007) [2023-10-11 19:08:06,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 13655.8). Total num frames: 1605632. Throughput: 0: 1802.4, 1: 1791.4. Samples: 410220. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-11 19:08:06,035][70582] Avg episode reward: [(0, '11.080'), (1, '11.130')] [2023-10-11 19:08:06,036][71353] Saving new best policy, reward=11.080! [2023-10-11 19:08:06,284][71431] Saving new best policy, reward=11.130! [2023-10-11 19:08:06,284][71635] Updated weights for policy 1, policy_version 800 (0.0007) [2023-10-11 19:08:09,340][71601] Updated weights for policy 0, policy_version 810 (0.0010) [2023-10-11 19:08:09,706][71601] Updated weights for policy 0, policy_version 820 (0.0009) [2023-10-11 19:08:09,975][71635] Updated weights for policy 1, policy_version 810 (0.0007) [2023-10-11 19:08:10,069][71601] Updated weights for policy 0, policy_version 830 (0.0007) [2023-10-11 19:08:10,344][71635] Updated weights for policy 1, policy_version 820 (0.0010) [2023-10-11 19:08:10,721][71635] Updated weights for policy 1, policy_version 830 (0.0008) [2023-10-11 19:08:11,034][70582] Fps is (10 sec: 19660.4, 60 sec: 14745.6, 300 sec: 13900.7). Total num frames: 1703936. Throughput: 0: 1784.1, 1: 1798.4. Samples: 430434. Policy #0 lag: (min: 12.0, avg: 12.1, max: 17.0) [2023-10-11 19:08:11,035][70582] Avg episode reward: [(0, '11.710'), (1, '11.030')] [2023-10-11 19:08:11,044][71353] Saving new best policy, reward=11.710! [2023-10-11 19:08:13,914][71601] Updated weights for policy 0, policy_version 840 (0.0009) [2023-10-11 19:08:14,277][71601] Updated weights for policy 0, policy_version 850 (0.0008) [2023-10-11 19:08:14,489][71635] Updated weights for policy 1, policy_version 840 (0.0007) [2023-10-11 19:08:14,652][71601] Updated weights for policy 0, policy_version 860 (0.0009) [2023-10-11 19:08:14,858][71635] Updated weights for policy 1, policy_version 850 (0.0008) [2023-10-11 19:08:15,223][71635] Updated weights for policy 1, policy_version 860 (0.0008) [2023-10-11 19:08:16,034][70582] Fps is (10 sec: 16384.4, 60 sec: 14199.5, 300 sec: 13869.7). Total num frames: 1769472. Throughput: 0: 1801.4, 1: 1788.3. Samples: 442522. Policy #0 lag: (min: 8.0, avg: 30.6, max: 40.0) [2023-10-11 19:08:16,034][70582] Avg episode reward: [(0, '11.400'), (1, '9.980')] [2023-10-11 19:08:18,416][71601] Updated weights for policy 0, policy_version 870 (0.0009) [2023-10-11 19:08:18,791][71601] Updated weights for policy 0, policy_version 880 (0.0009) [2023-10-11 19:08:19,009][71635] Updated weights for policy 1, policy_version 870 (0.0008) [2023-10-11 19:08:19,164][71601] Updated weights for policy 0, policy_version 890 (0.0008) [2023-10-11 19:08:19,366][71635] Updated weights for policy 1, policy_version 880 (0.0007) [2023-10-11 19:08:19,736][71635] Updated weights for policy 1, policy_version 890 (0.0010) [2023-10-11 19:08:21,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 13840.9). Total num frames: 1835008. Throughput: 0: 1789.4, 1: 1798.0. Samples: 462576. Policy #0 lag: (min: 16.0, avg: 30.6, max: 48.0) [2023-10-11 19:08:21,034][70582] Avg episode reward: [(0, '11.730'), (1, '9.130')] [2023-10-11 19:08:21,035][71353] Saving new best policy, reward=11.730! [2023-10-11 19:08:22,654][71601] Updated weights for policy 0, policy_version 900 (0.0009) [2023-10-11 19:08:23,030][71601] Updated weights for policy 0, policy_version 910 (0.0007) [2023-10-11 19:08:23,394][71601] Updated weights for policy 0, policy_version 920 (0.0008) [2023-10-11 19:08:23,453][71635] Updated weights for policy 1, policy_version 900 (0.0010) [2023-10-11 19:08:23,808][71635] Updated weights for policy 1, policy_version 910 (0.0008) [2023-10-11 19:08:24,175][71635] Updated weights for policy 1, policy_version 920 (0.0008) [2023-10-11 19:08:26,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 13814.2). Total num frames: 1900544. Throughput: 0: 1803.3, 1: 1784.9. Samples: 484864. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:08:26,034][70582] Avg episode reward: [(0, '11.720'), (1, '9.370')] [2023-10-11 19:08:27,098][71601] Updated weights for policy 0, policy_version 930 (0.0009) [2023-10-11 19:08:27,503][71601] Updated weights for policy 0, policy_version 940 (0.0007) [2023-10-11 19:08:27,864][71635] Updated weights for policy 1, policy_version 930 (0.0008) [2023-10-11 19:08:27,874][71601] Updated weights for policy 0, policy_version 950 (0.0007) [2023-10-11 19:08:28,225][71635] Updated weights for policy 1, policy_version 940 (0.0007) [2023-10-11 19:08:28,249][71601] Updated weights for policy 0, policy_version 960 (0.0007) [2023-10-11 19:08:28,597][71635] Updated weights for policy 1, policy_version 950 (0.0009) [2023-10-11 19:08:28,965][71635] Updated weights for policy 1, policy_version 960 (0.0009) [2023-10-11 19:08:31,034][70582] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 13789.4). Total num frames: 1966080. Throughput: 0: 1808.9, 1: 1803.1. Samples: 495446. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-10-11 19:08:31,035][70582] Avg episode reward: [(0, '12.330'), (1, '9.330')] [2023-10-11 19:08:31,036][71353] Saving new best policy, reward=12.330! [2023-10-11 19:08:32,002][71601] Updated weights for policy 0, policy_version 970 (0.0009) [2023-10-11 19:08:32,370][71601] Updated weights for policy 0, policy_version 980 (0.0008) [2023-10-11 19:08:32,645][71635] Updated weights for policy 1, policy_version 970 (0.0008) [2023-10-11 19:08:32,749][71601] Updated weights for policy 0, policy_version 990 (0.0009) [2023-10-11 19:08:33,009][71635] Updated weights for policy 1, policy_version 980 (0.0007) [2023-10-11 19:08:33,383][71635] Updated weights for policy 1, policy_version 990 (0.0008) [2023-10-11 19:08:36,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 13766.3). Total num frames: 2031616. Throughput: 0: 1809.2, 1: 1789.0. Samples: 517188. Policy #0 lag: (min: 21.0, avg: 21.1, max: 28.0) [2023-10-11 19:08:36,035][70582] Avg episode reward: [(0, '12.270'), (1, '9.260')] [2023-10-11 19:08:36,506][71601] Updated weights for policy 0, policy_version 1000 (0.0008) [2023-10-11 19:08:36,881][71601] Updated weights for policy 0, policy_version 1010 (0.0009) [2023-10-11 19:08:37,039][71635] Updated weights for policy 1, policy_version 1000 (0.0009) [2023-10-11 19:08:37,250][71601] Updated weights for policy 0, policy_version 1020 (0.0007) [2023-10-11 19:08:37,404][71635] Updated weights for policy 1, policy_version 1010 (0.0007) [2023-10-11 19:08:37,765][71635] Updated weights for policy 1, policy_version 1020 (0.0008) [2023-10-11 19:08:41,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 13744.7). Total num frames: 2097152. Throughput: 0: 1816.0, 1: 1788.7. Samples: 539982. Policy #0 lag: (min: 13.0, avg: 19.2, max: 45.0) [2023-10-11 19:08:41,034][70582] Avg episode reward: [(0, '12.270'), (1, '10.020')] [2023-10-11 19:08:41,048][71601] Updated weights for policy 0, policy_version 1030 (0.0009) [2023-10-11 19:08:41,413][71601] Updated weights for policy 0, policy_version 1040 (0.0008) [2023-10-11 19:08:41,529][71635] Updated weights for policy 1, policy_version 1030 (0.0008) [2023-10-11 19:08:41,788][71601] Updated weights for policy 0, policy_version 1050 (0.0007) [2023-10-11 19:08:41,898][71635] Updated weights for policy 1, policy_version 1040 (0.0010) [2023-10-11 19:08:42,259][71635] Updated weights for policy 1, policy_version 1050 (0.0009) [2023-10-11 19:08:45,380][71601] Updated weights for policy 0, policy_version 1060 (0.0010) [2023-10-11 19:08:45,748][71601] Updated weights for policy 0, policy_version 1070 (0.0009) [2023-10-11 19:08:46,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13724.5). Total num frames: 2162688. Throughput: 0: 1815.2, 1: 1787.6. Samples: 549886. Policy #0 lag: (min: 3.0, avg: 6.0, max: 35.0) [2023-10-11 19:08:46,034][70582] Avg episode reward: [(0, '11.670'), (1, '9.020')] [2023-10-11 19:08:46,053][71635] Updated weights for policy 1, policy_version 1060 (0.0008) [2023-10-11 19:08:46,124][71601] Updated weights for policy 0, policy_version 1080 (0.0008) [2023-10-11 19:08:46,423][71635] Updated weights for policy 1, policy_version 1070 (0.0008) [2023-10-11 19:08:46,796][71635] Updated weights for policy 1, policy_version 1080 (0.0010) [2023-10-11 19:08:49,904][71601] Updated weights for policy 0, policy_version 1090 (0.0008) [2023-10-11 19:08:50,277][71601] Updated weights for policy 0, policy_version 1100 (0.0010) [2023-10-11 19:08:50,620][71635] Updated weights for policy 1, policy_version 1090 (0.0010) [2023-10-11 19:08:50,657][71601] Updated weights for policy 0, policy_version 1110 (0.0010) [2023-10-11 19:08:50,999][71635] Updated weights for policy 1, policy_version 1100 (0.0009) [2023-10-11 19:08:51,028][71601] Updated weights for policy 0, policy_version 1120 (0.0008) [2023-10-11 19:08:51,034][70582] Fps is (10 sec: 16382.6, 60 sec: 14745.4, 300 sec: 13907.0). Total num frames: 2260992. Throughput: 0: 1815.9, 1: 1788.8. Samples: 572432. Policy #0 lag: (min: 6.0, avg: 8.0, max: 36.0) [2023-10-11 19:08:51,035][70582] Avg episode reward: [(0, '10.150'), (1, '8.690')] [2023-10-11 19:08:51,368][71635] Updated weights for policy 1, policy_version 1110 (0.0007) [2023-10-11 19:08:51,732][71635] Updated weights for policy 1, policy_version 1120 (0.0007) [2023-10-11 19:08:54,584][71601] Updated weights for policy 0, policy_version 1130 (0.0007) [2023-10-11 19:08:54,955][71601] Updated weights for policy 0, policy_version 1140 (0.0007) [2023-10-11 19:08:55,325][71601] Updated weights for policy 0, policy_version 1150 (0.0008) [2023-10-11 19:08:55,467][71635] Updated weights for policy 1, policy_version 1130 (0.0007) [2023-10-11 19:08:55,835][71635] Updated weights for policy 1, policy_version 1140 (0.0008) [2023-10-11 19:08:56,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 13883.2). Total num frames: 2326528. Throughput: 0: 1819.4, 1: 1806.0. Samples: 593576. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:08:56,034][70582] Avg episode reward: [(0, '9.400'), (1, '9.170')] [2023-10-11 19:08:56,211][71635] Updated weights for policy 1, policy_version 1150 (0.0007) [2023-10-11 19:08:58,918][71601] Updated weights for policy 0, policy_version 1160 (0.0008) [2023-10-11 19:08:59,294][71601] Updated weights for policy 0, policy_version 1170 (0.0010) [2023-10-11 19:08:59,663][71601] Updated weights for policy 0, policy_version 1180 (0.0008) [2023-10-11 19:08:59,871][71635] Updated weights for policy 1, policy_version 1160 (0.0007) [2023-10-11 19:09:00,243][71635] Updated weights for policy 1, policy_version 1170 (0.0007) [2023-10-11 19:09:00,611][71635] Updated weights for policy 1, policy_version 1180 (0.0007) [2023-10-11 19:09:01,034][70582] Fps is (10 sec: 16384.8, 60 sec: 15291.6, 300 sec: 14050.6). Total num frames: 2424832. Throughput: 0: 1821.9, 1: 1795.3. Samples: 605298. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:09:01,036][70582] Avg episode reward: [(0, '9.920'), (1, '8.840')] [2023-10-11 19:09:03,439][71601] Updated weights for policy 0, policy_version 1190 (0.0008) [2023-10-11 19:09:03,811][71601] Updated weights for policy 0, policy_version 1200 (0.0007) [2023-10-11 19:09:04,172][71635] Updated weights for policy 1, policy_version 1190 (0.0007) [2023-10-11 19:09:04,180][71601] Updated weights for policy 0, policy_version 1210 (0.0008) [2023-10-11 19:09:04,540][71635] Updated weights for policy 1, policy_version 1200 (0.0009) [2023-10-11 19:09:04,909][71635] Updated weights for policy 1, policy_version 1210 (0.0010) [2023-10-11 19:09:06,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14024.0). Total num frames: 2490368. Throughput: 0: 1829.9, 1: 1808.9. Samples: 626326. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:09:06,035][70582] Avg episode reward: [(0, '10.780'), (1, '8.950')] [2023-10-11 19:09:07,830][71601] Updated weights for policy 0, policy_version 1220 (0.0009) [2023-10-11 19:09:08,193][71601] Updated weights for policy 0, policy_version 1230 (0.0009) [2023-10-11 19:09:08,564][71601] Updated weights for policy 0, policy_version 1240 (0.0009) [2023-10-11 19:09:08,685][71635] Updated weights for policy 1, policy_version 1220 (0.0010) [2023-10-11 19:09:09,057][71635] Updated weights for policy 1, policy_version 1230 (0.0007) [2023-10-11 19:09:09,416][71635] Updated weights for policy 1, policy_version 1240 (0.0009) [2023-10-11 19:09:11,034][70582] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 13998.9). Total num frames: 2555904. Throughput: 0: 1821.3, 1: 1805.4. Samples: 648062. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:09:11,034][70582] Avg episode reward: [(0, '11.210'), (1, '9.130')] [2023-10-11 19:09:12,177][71601] Updated weights for policy 0, policy_version 1250 (0.0008) [2023-10-11 19:09:12,559][71601] Updated weights for policy 0, policy_version 1260 (0.0009) [2023-10-11 19:09:12,929][71601] Updated weights for policy 0, policy_version 1270 (0.0007) [2023-10-11 19:09:13,253][71635] Updated weights for policy 1, policy_version 1250 (0.0010) [2023-10-11 19:09:13,301][71601] Updated weights for policy 0, policy_version 1280 (0.0010) [2023-10-11 19:09:13,623][71635] Updated weights for policy 1, policy_version 1260 (0.0008) [2023-10-11 19:09:13,990][71635] Updated weights for policy 1, policy_version 1270 (0.0011) [2023-10-11 19:09:14,366][71635] Updated weights for policy 1, policy_version 1280 (0.0008) [2023-10-11 19:09:16,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13975.2). Total num frames: 2621440. Throughput: 0: 1823.8, 1: 1817.2. Samples: 659290. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:09:16,035][70582] Avg episode reward: [(0, '11.290'), (1, '9.560')] [2023-10-11 19:09:17,078][71601] Updated weights for policy 0, policy_version 1290 (0.0009) [2023-10-11 19:09:17,454][71601] Updated weights for policy 0, policy_version 1300 (0.0007) [2023-10-11 19:09:17,784][71635] Updated weights for policy 1, policy_version 1290 (0.0008) [2023-10-11 19:09:17,833][71601] Updated weights for policy 0, policy_version 1310 (0.0007) [2023-10-11 19:09:18,153][71635] Updated weights for policy 1, policy_version 1300 (0.0007) [2023-10-11 19:09:18,522][71635] Updated weights for policy 1, policy_version 1310 (0.0008) [2023-10-11 19:09:21,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 13952.6). Total num frames: 2686976. Throughput: 0: 1827.6, 1: 1815.7. Samples: 681138. Policy #0 lag: (min: 26.0, avg: 30.5, max: 58.0) [2023-10-11 19:09:21,035][70582] Avg episode reward: [(0, '11.170'), (1, '9.340')] [2023-10-11 19:09:21,436][71601] Updated weights for policy 0, policy_version 1320 (0.0007) [2023-10-11 19:09:21,803][71601] Updated weights for policy 0, policy_version 1330 (0.0009) [2023-10-11 19:09:22,179][71601] Updated weights for policy 0, policy_version 1340 (0.0008) [2023-10-11 19:09:22,228][71635] Updated weights for policy 1, policy_version 1320 (0.0008) [2023-10-11 19:09:22,600][71635] Updated weights for policy 1, policy_version 1330 (0.0008) [2023-10-11 19:09:22,966][71635] Updated weights for policy 1, policy_version 1340 (0.0008) [2023-10-11 19:09:25,857][71601] Updated weights for policy 0, policy_version 1350 (0.0008) [2023-10-11 19:09:26,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13931.2). Total num frames: 2752512. Throughput: 0: 1826.2, 1: 1818.7. Samples: 704002. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-11 19:09:26,035][70582] Avg episode reward: [(0, '10.550'), (1, '10.380')] [2023-10-11 19:09:26,229][71601] Updated weights for policy 0, policy_version 1360 (0.0007) [2023-10-11 19:09:26,545][71635] Updated weights for policy 1, policy_version 1350 (0.0008) [2023-10-11 19:09:26,607][71601] Updated weights for policy 0, policy_version 1370 (0.0008) [2023-10-11 19:09:26,911][71635] Updated weights for policy 1, policy_version 1360 (0.0007) [2023-10-11 19:09:27,276][71635] Updated weights for policy 1, policy_version 1370 (0.0007) [2023-10-11 19:09:30,324][71601] Updated weights for policy 0, policy_version 1380 (0.0008) [2023-10-11 19:09:30,695][71601] Updated weights for policy 0, policy_version 1390 (0.0007) [2023-10-11 19:09:31,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13910.9). Total num frames: 2818048. Throughput: 0: 1822.7, 1: 1822.3. Samples: 713910. Policy #0 lag: (min: 10.0, avg: 10.9, max: 31.0) [2023-10-11 19:09:31,034][70582] Avg episode reward: [(0, '11.080'), (1, '10.450')] [2023-10-11 19:09:31,069][71635] Updated weights for policy 1, policy_version 1380 (0.0008) [2023-10-11 19:09:31,069][71601] Updated weights for policy 0, policy_version 1400 (0.0008) [2023-10-11 19:09:31,444][71635] Updated weights for policy 1, policy_version 1390 (0.0009) [2023-10-11 19:09:31,802][71635] Updated weights for policy 1, policy_version 1400 (0.0008) [2023-10-11 19:09:34,642][71601] Updated weights for policy 0, policy_version 1410 (0.0008) [2023-10-11 19:09:35,011][71601] Updated weights for policy 0, policy_version 1420 (0.0007) [2023-10-11 19:09:35,400][71601] Updated weights for policy 0, policy_version 1430 (0.0009) [2023-10-11 19:09:35,599][71635] Updated weights for policy 1, policy_version 1410 (0.0007) [2023-10-11 19:09:35,767][71601] Updated weights for policy 0, policy_version 1440 (0.0009) [2023-10-11 19:09:35,958][71635] Updated weights for policy 1, policy_version 1420 (0.0010) [2023-10-11 19:09:36,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14049.4). Total num frames: 2916352. Throughput: 0: 1822.3, 1: 1819.9. Samples: 736324. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-11 19:09:36,034][70582] Avg episode reward: [(0, '10.940'), (1, '11.180')] [2023-10-11 19:09:36,332][71635] Updated weights for policy 1, policy_version 1430 (0.0010) [2023-10-11 19:09:36,693][71431] Saving new best policy, reward=11.180! [2023-10-11 19:09:36,695][71635] Updated weights for policy 1, policy_version 1440 (0.0010) [2023-10-11 19:09:39,635][71601] Updated weights for policy 0, policy_version 1450 (0.0009) [2023-10-11 19:09:40,015][71601] Updated weights for policy 0, policy_version 1460 (0.0008) [2023-10-11 19:09:40,398][71601] Updated weights for policy 0, policy_version 1470 (0.0010) [2023-10-11 19:09:40,512][71635] Updated weights for policy 1, policy_version 1450 (0.0009) [2023-10-11 19:09:40,888][71635] Updated weights for policy 1, policy_version 1460 (0.0008) [2023-10-11 19:09:41,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14027.2). Total num frames: 2981888. Throughput: 0: 1814.4, 1: 1816.1. Samples: 756950. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-10-11 19:09:41,035][70582] Avg episode reward: [(0, '11.650'), (1, '12.730')] [2023-10-11 19:09:41,261][71635] Updated weights for policy 1, policy_version 1470 (0.0009) [2023-10-11 19:09:41,335][71431] Saving new best policy, reward=12.730! [2023-10-11 19:09:43,985][71601] Updated weights for policy 0, policy_version 1480 (0.0009) [2023-10-11 19:09:44,355][71601] Updated weights for policy 0, policy_version 1490 (0.0010) [2023-10-11 19:09:44,731][71601] Updated weights for policy 0, policy_version 1500 (0.0010) [2023-10-11 19:09:44,991][71635] Updated weights for policy 1, policy_version 1480 (0.0007) [2023-10-11 19:09:45,367][71635] Updated weights for policy 1, policy_version 1490 (0.0008) [2023-10-11 19:09:45,733][71635] Updated weights for policy 1, policy_version 1500 (0.0007) [2023-10-11 19:09:46,034][70582] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14156.7). Total num frames: 3080192. Throughput: 0: 1810.2, 1: 1816.7. Samples: 768506. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-10-11 19:09:46,034][70582] Avg episode reward: [(0, '11.840'), (1, '13.000')] [2023-10-11 19:09:46,035][71431] Saving new best policy, reward=13.000! [2023-10-11 19:09:48,446][71601] Updated weights for policy 0, policy_version 1510 (0.0010) [2023-10-11 19:09:48,821][71601] Updated weights for policy 0, policy_version 1520 (0.0009) [2023-10-11 19:09:49,192][71601] Updated weights for policy 0, policy_version 1530 (0.0008) [2023-10-11 19:09:49,546][71635] Updated weights for policy 1, policy_version 1510 (0.0007) [2023-10-11 19:09:49,914][71635] Updated weights for policy 1, policy_version 1520 (0.0008) [2023-10-11 19:09:50,286][71635] Updated weights for policy 1, policy_version 1530 (0.0007) [2023-10-11 19:09:51,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.8, 300 sec: 14133.1). Total num frames: 3145728. Throughput: 0: 1804.9, 1: 1819.6. Samples: 789430. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-11 19:09:51,034][70582] Avg episode reward: [(0, '12.360'), (1, '12.150')] [2023-10-11 19:09:51,035][71353] Saving new best policy, reward=12.360! [2023-10-11 19:09:52,916][71601] Updated weights for policy 0, policy_version 1540 (0.0008) [2023-10-11 19:09:53,294][71601] Updated weights for policy 0, policy_version 1550 (0.0007) [2023-10-11 19:09:53,667][71601] Updated weights for policy 0, policy_version 1560 (0.0008) [2023-10-11 19:09:53,778][71635] Updated weights for policy 1, policy_version 1540 (0.0008) [2023-10-11 19:09:54,148][71635] Updated weights for policy 1, policy_version 1550 (0.0007) [2023-10-11 19:09:54,519][71635] Updated weights for policy 1, policy_version 1560 (0.0008) [2023-10-11 19:09:56,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14110.6). Total num frames: 3211264. Throughput: 0: 1803.8, 1: 1818.1. Samples: 811048. Policy #0 lag: (min: 17.0, avg: 27.6, max: 49.0) [2023-10-11 19:09:56,034][70582] Avg episode reward: [(0, '11.840'), (1, '11.510')] [2023-10-11 19:09:56,043][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000001568_1605632.pth... [2023-10-11 19:09:56,043][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000001568_1605632.pth... [2023-10-11 19:09:57,443][71601] Updated weights for policy 0, policy_version 1570 (0.0009) [2023-10-11 19:09:57,854][71601] Updated weights for policy 0, policy_version 1580 (0.0009) [2023-10-11 19:09:58,160][71635] Updated weights for policy 1, policy_version 1570 (0.0008) [2023-10-11 19:09:58,222][71601] Updated weights for policy 0, policy_version 1590 (0.0008) [2023-10-11 19:09:58,530][71635] Updated weights for policy 1, policy_version 1580 (0.0008) [2023-10-11 19:09:58,588][71601] Updated weights for policy 0, policy_version 1600 (0.0008) [2023-10-11 19:09:58,895][71635] Updated weights for policy 1, policy_version 1590 (0.0009) [2023-10-11 19:09:59,273][71635] Updated weights for policy 1, policy_version 1600 (0.0009) [2023-10-11 19:10:01,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14089.0). Total num frames: 3276800. Throughput: 0: 1803.0, 1: 1818.2. Samples: 822242. Policy #0 lag: (min: 30.0, avg: 30.1, max: 34.0) [2023-10-11 19:10:01,035][70582] Avg episode reward: [(0, '12.330'), (1, '10.380')] [2023-10-11 19:10:02,425][71601] Updated weights for policy 0, policy_version 1610 (0.0009) [2023-10-11 19:10:02,802][71601] Updated weights for policy 0, policy_version 1620 (0.0007) [2023-10-11 19:10:02,894][71635] Updated weights for policy 1, policy_version 1610 (0.0008) [2023-10-11 19:10:03,177][71601] Updated weights for policy 0, policy_version 1630 (0.0007) [2023-10-11 19:10:03,262][71635] Updated weights for policy 1, policy_version 1620 (0.0009) [2023-10-11 19:10:03,631][71635] Updated weights for policy 1, policy_version 1630 (0.0008) [2023-10-11 19:10:06,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14068.3). Total num frames: 3342336. Throughput: 0: 1795.7, 1: 1815.7. Samples: 843646. Policy #0 lag: (min: 13.0, avg: 24.1, max: 45.0) [2023-10-11 19:10:06,034][70582] Avg episode reward: [(0, '11.500'), (1, '10.590')] [2023-10-11 19:10:06,967][71601] Updated weights for policy 0, policy_version 1640 (0.0007) [2023-10-11 19:10:07,341][71601] Updated weights for policy 0, policy_version 1650 (0.0008) [2023-10-11 19:10:07,603][71635] Updated weights for policy 1, policy_version 1640 (0.0008) [2023-10-11 19:10:07,713][71601] Updated weights for policy 0, policy_version 1660 (0.0008) [2023-10-11 19:10:07,982][71635] Updated weights for policy 1, policy_version 1650 (0.0008) [2023-10-11 19:10:08,346][71635] Updated weights for policy 1, policy_version 1660 (0.0007) [2023-10-11 19:10:11,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14048.5). Total num frames: 3407872. Throughput: 0: 1799.1, 1: 1812.1. Samples: 866506. Policy #0 lag: (min: 4.0, avg: 10.2, max: 36.0) [2023-10-11 19:10:11,034][70582] Avg episode reward: [(0, '12.360'), (1, '10.010')] [2023-10-11 19:10:11,318][71601] Updated weights for policy 0, policy_version 1670 (0.0008) [2023-10-11 19:10:11,684][71601] Updated weights for policy 0, policy_version 1680 (0.0008) [2023-10-11 19:10:11,950][71635] Updated weights for policy 1, policy_version 1670 (0.0007) [2023-10-11 19:10:12,059][71601] Updated weights for policy 0, policy_version 1690 (0.0008) [2023-10-11 19:10:12,320][71635] Updated weights for policy 1, policy_version 1680 (0.0008) [2023-10-11 19:10:12,677][71635] Updated weights for policy 1, policy_version 1690 (0.0010) [2023-10-11 19:10:15,832][71601] Updated weights for policy 0, policy_version 1700 (0.0009) [2023-10-11 19:10:16,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14029.5). Total num frames: 3473408. Throughput: 0: 1802.0, 1: 1811.4. Samples: 876514. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:10:16,034][70582] Avg episode reward: [(0, '12.080'), (1, '10.090')] [2023-10-11 19:10:16,213][71601] Updated weights for policy 0, policy_version 1710 (0.0008) [2023-10-11 19:10:16,335][71635] Updated weights for policy 1, policy_version 1700 (0.0008) [2023-10-11 19:10:16,580][71601] Updated weights for policy 0, policy_version 1720 (0.0007) [2023-10-11 19:10:16,689][71635] Updated weights for policy 1, policy_version 1710 (0.0008) [2023-10-11 19:10:17,060][71635] Updated weights for policy 1, policy_version 1720 (0.0008) [2023-10-11 19:10:20,313][71601] Updated weights for policy 0, policy_version 1730 (0.0008) [2023-10-11 19:10:20,687][71601] Updated weights for policy 0, policy_version 1740 (0.0008) [2023-10-11 19:10:20,796][71635] Updated weights for policy 1, policy_version 1730 (0.0008) [2023-10-11 19:10:21,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14011.3). Total num frames: 3538944. Throughput: 0: 1801.9, 1: 1818.0. Samples: 899220. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-11 19:10:21,034][70582] Avg episode reward: [(0, '11.230'), (1, '9.760')] [2023-10-11 19:10:21,058][71601] Updated weights for policy 0, policy_version 1750 (0.0008) [2023-10-11 19:10:21,167][71635] Updated weights for policy 1, policy_version 1740 (0.0007) [2023-10-11 19:10:21,436][71601] Updated weights for policy 0, policy_version 1760 (0.0007) [2023-10-11 19:10:21,539][71635] Updated weights for policy 1, policy_version 1750 (0.0008) [2023-10-11 19:10:21,907][71635] Updated weights for policy 1, policy_version 1760 (0.0007) [2023-10-11 19:10:25,162][71601] Updated weights for policy 0, policy_version 1770 (0.0007) [2023-10-11 19:10:25,495][71635] Updated weights for policy 1, policy_version 1770 (0.0008) [2023-10-11 19:10:25,536][71601] Updated weights for policy 0, policy_version 1780 (0.0007) [2023-10-11 19:10:25,860][71635] Updated weights for policy 1, policy_version 1780 (0.0008) [2023-10-11 19:10:25,911][71601] Updated weights for policy 0, policy_version 1790 (0.0008) [2023-10-11 19:10:26,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14120.9). Total num frames: 3637248. Throughput: 0: 1819.0, 1: 1818.7. Samples: 920644. Policy #0 lag: (min: 8.0, avg: 34.6, max: 40.0) [2023-10-11 19:10:26,035][70582] Avg episode reward: [(0, '11.430'), (1, '9.910')] [2023-10-11 19:10:26,218][71635] Updated weights for policy 1, policy_version 1790 (0.0008) [2023-10-11 19:10:29,509][71601] Updated weights for policy 0, policy_version 1800 (0.0008) [2023-10-11 19:10:29,884][71635] Updated weights for policy 1, policy_version 1800 (0.0009) [2023-10-11 19:10:29,885][71601] Updated weights for policy 0, policy_version 1810 (0.0009) [2023-10-11 19:10:30,249][71635] Updated weights for policy 1, policy_version 1810 (0.0008) [2023-10-11 19:10:30,267][71601] Updated weights for policy 0, policy_version 1820 (0.0008) [2023-10-11 19:10:30,631][71635] Updated weights for policy 1, policy_version 1820 (0.0009) [2023-10-11 19:10:31,034][70582] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14226.4). Total num frames: 3735552. Throughput: 0: 1806.9, 1: 1818.3. Samples: 931640. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:10:31,034][70582] Avg episode reward: [(0, '10.940'), (1, '11.040')] [2023-10-11 19:10:33,909][71601] Updated weights for policy 0, policy_version 1830 (0.0009) [2023-10-11 19:10:34,277][71601] Updated weights for policy 0, policy_version 1840 (0.0008) [2023-10-11 19:10:34,367][71635] Updated weights for policy 1, policy_version 1830 (0.0007) [2023-10-11 19:10:34,648][71601] Updated weights for policy 0, policy_version 1850 (0.0007) [2023-10-11 19:10:34,720][71635] Updated weights for policy 1, policy_version 1840 (0.0007) [2023-10-11 19:10:35,085][71635] Updated weights for policy 1, policy_version 1850 (0.0010) [2023-10-11 19:10:36,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14205.5). Total num frames: 3801088. Throughput: 0: 1828.9, 1: 1816.7. Samples: 953480. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:10:36,034][70582] Avg episode reward: [(0, '11.660'), (1, '11.260')] [2023-10-11 19:10:38,347][71601] Updated weights for policy 0, policy_version 1860 (0.0007) [2023-10-11 19:10:38,719][71601] Updated weights for policy 0, policy_version 1870 (0.0008) [2023-10-11 19:10:38,896][71635] Updated weights for policy 1, policy_version 1860 (0.0009) [2023-10-11 19:10:39,096][71601] Updated weights for policy 0, policy_version 1880 (0.0010) [2023-10-11 19:10:39,253][71635] Updated weights for policy 1, policy_version 1870 (0.0009) [2023-10-11 19:10:39,619][71635] Updated weights for policy 1, policy_version 1880 (0.0012) [2023-10-11 19:10:41,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14185.3). Total num frames: 3866624. Throughput: 0: 1809.7, 1: 1812.4. Samples: 974044. Policy #0 lag: (min: 26.0, avg: 31.3, max: 58.0) [2023-10-11 19:10:41,035][70582] Avg episode reward: [(0, '12.250'), (1, '11.370')] [2023-10-11 19:10:42,853][71601] Updated weights for policy 0, policy_version 1890 (0.0010) [2023-10-11 19:10:43,251][71601] Updated weights for policy 0, policy_version 1900 (0.0008) [2023-10-11 19:10:43,513][71635] Updated weights for policy 1, policy_version 1890 (0.0010) [2023-10-11 19:10:43,629][71601] Updated weights for policy 0, policy_version 1910 (0.0009) [2023-10-11 19:10:43,878][71635] Updated weights for policy 1, policy_version 1900 (0.0007) [2023-10-11 19:10:44,000][71601] Updated weights for policy 0, policy_version 1920 (0.0009) [2023-10-11 19:10:44,245][71635] Updated weights for policy 1, policy_version 1910 (0.0008) [2023-10-11 19:10:44,610][71635] Updated weights for policy 1, policy_version 1920 (0.0011) [2023-10-11 19:10:46,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14165.9). Total num frames: 3932160. Throughput: 0: 1823.2, 1: 1815.8. Samples: 985998. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:10:46,034][70582] Avg episode reward: [(0, '12.160'), (1, '11.880')] [2023-10-11 19:10:47,637][71601] Updated weights for policy 0, policy_version 1930 (0.0007) [2023-10-11 19:10:48,013][71601] Updated weights for policy 0, policy_version 1940 (0.0007) [2023-10-11 19:10:48,314][71635] Updated weights for policy 1, policy_version 1930 (0.0008) [2023-10-11 19:10:48,387][71601] Updated weights for policy 0, policy_version 1950 (0.0008) [2023-10-11 19:10:48,680][71635] Updated weights for policy 1, policy_version 1940 (0.0010) [2023-10-11 19:10:49,056][71635] Updated weights for policy 1, policy_version 1950 (0.0010) [2023-10-11 19:10:51,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14147.2). Total num frames: 3997696. Throughput: 0: 1816.7, 1: 1803.0. Samples: 1006534. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:10:51,034][70582] Avg episode reward: [(0, '12.210'), (1, '11.850')] [2023-10-11 19:10:51,953][71601] Updated weights for policy 0, policy_version 1960 (0.0008) [2023-10-11 19:10:52,325][71601] Updated weights for policy 0, policy_version 1970 (0.0008) [2023-10-11 19:10:52,694][71601] Updated weights for policy 0, policy_version 1980 (0.0008) [2023-10-11 19:10:52,830][71635] Updated weights for policy 1, policy_version 1960 (0.0008) [2023-10-11 19:10:53,203][71635] Updated weights for policy 1, policy_version 1970 (0.0007) [2023-10-11 19:10:53,570][71635] Updated weights for policy 1, policy_version 1980 (0.0008) [2023-10-11 19:10:56,034][70582] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14129.1). Total num frames: 4063232. Throughput: 0: 1813.5, 1: 1803.7. Samples: 1029280. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-11 19:10:56,035][70582] Avg episode reward: [(0, '11.310'), (1, '12.370')] [2023-10-11 19:10:56,328][71601] Updated weights for policy 0, policy_version 1990 (0.0008) [2023-10-11 19:10:56,700][71601] Updated weights for policy 0, policy_version 2000 (0.0007) [2023-10-11 19:10:57,071][71601] Updated weights for policy 0, policy_version 2010 (0.0009) [2023-10-11 19:10:57,146][71635] Updated weights for policy 1, policy_version 1990 (0.0008) [2023-10-11 19:10:57,516][71635] Updated weights for policy 1, policy_version 2000 (0.0009) [2023-10-11 19:10:57,880][71635] Updated weights for policy 1, policy_version 2010 (0.0009) [2023-10-11 19:11:00,810][71601] Updated weights for policy 0, policy_version 2020 (0.0009) [2023-10-11 19:11:01,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14111.7). Total num frames: 4128768. Throughput: 0: 1814.4, 1: 1803.4. Samples: 1039312. Policy #0 lag: (min: 1.0, avg: 9.7, max: 33.0) [2023-10-11 19:11:01,035][70582] Avg episode reward: [(0, '11.480'), (1, '12.110')] [2023-10-11 19:11:01,178][71601] Updated weights for policy 0, policy_version 2030 (0.0011) [2023-10-11 19:11:01,555][71601] Updated weights for policy 0, policy_version 2040 (0.0009) [2023-10-11 19:11:01,702][71635] Updated weights for policy 1, policy_version 2020 (0.0008) [2023-10-11 19:11:02,065][71635] Updated weights for policy 1, policy_version 2030 (0.0008) [2023-10-11 19:11:02,438][71635] Updated weights for policy 1, policy_version 2040 (0.0009) [2023-10-11 19:11:05,246][71601] Updated weights for policy 0, policy_version 2050 (0.0008) [2023-10-11 19:11:05,634][71601] Updated weights for policy 0, policy_version 2060 (0.0009) [2023-10-11 19:11:06,010][71601] Updated weights for policy 0, policy_version 2070 (0.0008) [2023-10-11 19:11:06,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 4194304. Throughput: 0: 1813.0, 1: 1797.7. Samples: 1061702. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:11:06,035][70582] Avg episode reward: [(0, '11.030'), (1, '12.290')] [2023-10-11 19:11:06,257][71635] Updated weights for policy 1, policy_version 2050 (0.0009) [2023-10-11 19:11:06,390][71601] Updated weights for policy 0, policy_version 2080 (0.0008) [2023-10-11 19:11:06,615][71635] Updated weights for policy 1, policy_version 2060 (0.0008) [2023-10-11 19:11:06,986][71635] Updated weights for policy 1, policy_version 2070 (0.0009) [2023-10-11 19:11:07,357][71635] Updated weights for policy 1, policy_version 2080 (0.0007) [2023-10-11 19:11:09,981][71601] Updated weights for policy 0, policy_version 2090 (0.0007) [2023-10-11 19:11:10,360][71601] Updated weights for policy 0, policy_version 2100 (0.0007) [2023-10-11 19:11:10,744][71601] Updated weights for policy 0, policy_version 2110 (0.0007) [2023-10-11 19:11:11,034][70582] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 4292608. Throughput: 0: 1813.4, 1: 1806.8. Samples: 1083554. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-11 19:11:11,035][70582] Avg episode reward: [(0, '10.860'), (1, '11.750')] [2023-10-11 19:11:11,037][71635] Updated weights for policy 1, policy_version 2090 (0.0009) [2023-10-11 19:11:11,396][71635] Updated weights for policy 1, policy_version 2100 (0.0007) [2023-10-11 19:11:11,774][71635] Updated weights for policy 1, policy_version 2110 (0.0008) [2023-10-11 19:11:14,536][71601] Updated weights for policy 0, policy_version 2120 (0.0008) [2023-10-11 19:11:14,910][71601] Updated weights for policy 0, policy_version 2130 (0.0007) [2023-10-11 19:11:15,292][71601] Updated weights for policy 0, policy_version 2140 (0.0008) [2023-10-11 19:11:15,431][71635] Updated weights for policy 1, policy_version 2120 (0.0007) [2023-10-11 19:11:15,794][71635] Updated weights for policy 1, policy_version 2130 (0.0009) [2023-10-11 19:11:16,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 4358144. Throughput: 0: 1816.8, 1: 1803.3. Samples: 1094546. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:11:16,034][70582] Avg episode reward: [(0, '11.340'), (1, '12.850')] [2023-10-11 19:11:16,159][71635] Updated weights for policy 1, policy_version 2140 (0.0010) [2023-10-11 19:11:19,124][71601] Updated weights for policy 0, policy_version 2150 (0.0008) [2023-10-11 19:11:19,496][71601] Updated weights for policy 0, policy_version 2160 (0.0007) [2023-10-11 19:11:19,728][71635] Updated weights for policy 1, policy_version 2150 (0.0008) [2023-10-11 19:11:19,875][71601] Updated weights for policy 0, policy_version 2170 (0.0007) [2023-10-11 19:11:20,099][71635] Updated weights for policy 1, policy_version 2160 (0.0009) [2023-10-11 19:11:20,471][71635] Updated weights for policy 1, policy_version 2170 (0.0008) [2023-10-11 19:11:21,034][70582] Fps is (10 sec: 16384.7, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 4456448. Throughput: 0: 1812.7, 1: 1814.7. Samples: 1116712. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-10-11 19:11:21,034][70582] Avg episode reward: [(0, '10.020'), (1, '12.490')] [2023-10-11 19:11:23,278][71601] Updated weights for policy 0, policy_version 2180 (0.0008) [2023-10-11 19:11:23,652][71601] Updated weights for policy 0, policy_version 2190 (0.0008) [2023-10-11 19:11:24,023][71601] Updated weights for policy 0, policy_version 2200 (0.0009) [2023-10-11 19:11:24,076][71635] Updated weights for policy 1, policy_version 2180 (0.0009) [2023-10-11 19:11:24,452][71635] Updated weights for policy 1, policy_version 2190 (0.0009) [2023-10-11 19:11:24,824][71635] Updated weights for policy 1, policy_version 2200 (0.0009) [2023-10-11 19:11:26,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 4521984. Throughput: 0: 1818.9, 1: 1813.7. Samples: 1137512. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-10-11 19:11:26,034][70582] Avg episode reward: [(0, '10.890'), (1, '12.690')] [2023-10-11 19:11:27,688][71601] Updated weights for policy 0, policy_version 2210 (0.0008) [2023-10-11 19:11:28,072][71601] Updated weights for policy 0, policy_version 2220 (0.0007) [2023-10-11 19:11:28,441][71601] Updated weights for policy 0, policy_version 2230 (0.0007) [2023-10-11 19:11:28,532][71635] Updated weights for policy 1, policy_version 2210 (0.0009) [2023-10-11 19:11:28,808][71601] Updated weights for policy 0, policy_version 2240 (0.0008) [2023-10-11 19:11:28,894][71635] Updated weights for policy 1, policy_version 2220 (0.0008) [2023-10-11 19:11:29,265][71635] Updated weights for policy 1, policy_version 2230 (0.0009) [2023-10-11 19:11:29,631][71635] Updated weights for policy 1, policy_version 2240 (0.0010) [2023-10-11 19:11:31,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 4587520. Throughput: 0: 1816.8, 1: 1815.5. Samples: 1149450. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:11:31,035][70582] Avg episode reward: [(0, '10.550'), (1, '12.870')] [2023-10-11 19:11:32,513][71601] Updated weights for policy 0, policy_version 2250 (0.0010) [2023-10-11 19:11:32,882][71601] Updated weights for policy 0, policy_version 2260 (0.0011) [2023-10-11 19:11:33,251][71601] Updated weights for policy 0, policy_version 2270 (0.0009) [2023-10-11 19:11:33,406][71635] Updated weights for policy 1, policy_version 2250 (0.0008) [2023-10-11 19:11:33,765][71635] Updated weights for policy 1, policy_version 2260 (0.0007) [2023-10-11 19:11:34,136][71635] Updated weights for policy 1, policy_version 2270 (0.0008) [2023-10-11 19:11:36,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 4653056. Throughput: 0: 1820.7, 1: 1815.8. Samples: 1170178. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:11:36,034][70582] Avg episode reward: [(0, '11.840'), (1, '12.390')] [2023-10-11 19:11:36,926][71601] Updated weights for policy 0, policy_version 2280 (0.0009) [2023-10-11 19:11:37,309][71601] Updated weights for policy 0, policy_version 2290 (0.0009) [2023-10-11 19:11:37,673][71601] Updated weights for policy 0, policy_version 2300 (0.0009) [2023-10-11 19:11:37,941][71635] Updated weights for policy 1, policy_version 2280 (0.0007) [2023-10-11 19:11:38,333][71635] Updated weights for policy 1, policy_version 2290 (0.0008) [2023-10-11 19:11:38,777][71635] Updated weights for policy 1, policy_version 2302 (0.0011) [2023-10-11 19:11:41,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 4718592. Throughput: 0: 1815.7, 1: 1807.7. Samples: 1192330. Policy #0 lag: (min: 9.0, avg: 19.9, max: 41.0) [2023-10-11 19:11:41,034][70582] Avg episode reward: [(0, '13.830'), (1, '11.020')] [2023-10-11 19:11:41,041][71353] Saving new best policy, reward=13.830! [2023-10-11 19:11:41,513][71601] Updated weights for policy 0, policy_version 2310 (0.0008) [2023-10-11 19:11:41,894][71601] Updated weights for policy 0, policy_version 2320 (0.0008) [2023-10-11 19:11:42,261][71601] Updated weights for policy 0, policy_version 2330 (0.0008) [2023-10-11 19:11:42,555][71635] Updated weights for policy 1, policy_version 2312 (0.0007) [2023-10-11 19:11:42,921][71635] Updated weights for policy 1, policy_version 2322 (0.0009) [2023-10-11 19:11:43,299][71635] Updated weights for policy 1, policy_version 2332 (0.0008) [2023-10-11 19:11:45,972][71601] Updated weights for policy 0, policy_version 2340 (0.0007) [2023-10-11 19:11:46,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 4784128. Throughput: 0: 1817.9, 1: 1810.2. Samples: 1202576. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-11 19:11:46,035][70582] Avg episode reward: [(0, '13.680'), (1, '10.320')] [2023-10-11 19:11:46,359][71601] Updated weights for policy 0, policy_version 2350 (0.0008) [2023-10-11 19:11:46,729][71601] Updated weights for policy 0, policy_version 2360 (0.0009) [2023-10-11 19:11:46,955][71635] Updated weights for policy 1, policy_version 2342 (0.0008) [2023-10-11 19:11:47,317][71635] Updated weights for policy 1, policy_version 2352 (0.0008) [2023-10-11 19:11:47,684][71635] Updated weights for policy 1, policy_version 2362 (0.0008) [2023-10-11 19:11:50,389][71601] Updated weights for policy 0, policy_version 2370 (0.0007) [2023-10-11 19:11:50,760][71601] Updated weights for policy 0, policy_version 2380 (0.0010) [2023-10-11 19:11:51,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 4849664. Throughput: 0: 1818.8, 1: 1813.4. Samples: 1225148. Policy #0 lag: (min: 14.0, avg: 21.0, max: 46.0) [2023-10-11 19:11:51,034][70582] Avg episode reward: [(0, '13.240'), (1, '9.640')] [2023-10-11 19:11:51,137][71601] Updated weights for policy 0, policy_version 2390 (0.0010) [2023-10-11 19:11:51,456][71635] Updated weights for policy 1, policy_version 2372 (0.0008) [2023-10-11 19:11:51,512][71601] Updated weights for policy 0, policy_version 2400 (0.0007) [2023-10-11 19:11:51,830][71635] Updated weights for policy 1, policy_version 2382 (0.0008) [2023-10-11 19:11:52,202][71635] Updated weights for policy 1, policy_version 2392 (0.0007) [2023-10-11 19:11:55,197][71601] Updated weights for policy 0, policy_version 2410 (0.0009) [2023-10-11 19:11:55,565][71601] Updated weights for policy 0, policy_version 2420 (0.0008) [2023-10-11 19:11:55,820][71635] Updated weights for policy 1, policy_version 2402 (0.0008) [2023-10-11 19:11:55,940][71601] Updated weights for policy 0, policy_version 2430 (0.0009) [2023-10-11 19:11:56,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 4947968. Throughput: 0: 1821.8, 1: 1814.9. Samples: 1247206. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-11 19:11:56,034][70582] Avg episode reward: [(0, '12.760'), (1, '9.900')] [2023-10-11 19:11:56,044][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000002432_2490368.pth... [2023-10-11 19:11:56,083][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000000736_753664.pth [2023-10-11 19:11:56,188][71635] Updated weights for policy 1, policy_version 2412 (0.0009) [2023-10-11 19:11:56,559][71635] Updated weights for policy 1, policy_version 2422 (0.0009) [2023-10-11 19:11:56,922][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000002432_2490368.pth... [2023-10-11 19:11:56,927][71635] Updated weights for policy 1, policy_version 2432 (0.0007) [2023-10-11 19:11:56,951][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000000736_753664.pth [2023-10-11 19:11:59,647][71601] Updated weights for policy 0, policy_version 2440 (0.0008) [2023-10-11 19:12:00,032][71601] Updated weights for policy 0, policy_version 2450 (0.0008) [2023-10-11 19:12:00,404][71601] Updated weights for policy 0, policy_version 2460 (0.0008) [2023-10-11 19:12:00,494][71635] Updated weights for policy 1, policy_version 2442 (0.0007) [2023-10-11 19:12:00,865][71635] Updated weights for policy 1, policy_version 2452 (0.0007) [2023-10-11 19:12:01,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 5013504. Throughput: 0: 1817.0, 1: 1816.9. Samples: 1258072. Policy #0 lag: (min: 31.0, avg: 32.0, max: 53.0) [2023-10-11 19:12:01,034][70582] Avg episode reward: [(0, '10.440'), (1, '10.540')] [2023-10-11 19:12:01,227][71635] Updated weights for policy 1, policy_version 2462 (0.0011) [2023-10-11 19:12:04,121][71601] Updated weights for policy 0, policy_version 2470 (0.0008) [2023-10-11 19:12:04,495][71601] Updated weights for policy 0, policy_version 2480 (0.0010) [2023-10-11 19:12:04,876][71601] Updated weights for policy 0, policy_version 2490 (0.0007) [2023-10-11 19:12:04,988][71635] Updated weights for policy 1, policy_version 2472 (0.0007) [2023-10-11 19:12:05,364][71635] Updated weights for policy 1, policy_version 2482 (0.0009) [2023-10-11 19:12:05,731][71635] Updated weights for policy 1, policy_version 2492 (0.0009) [2023-10-11 19:12:06,034][70582] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 5111808. Throughput: 0: 1819.2, 1: 1813.6. Samples: 1280190. Policy #0 lag: (min: 31.0, avg: 32.0, max: 53.0) [2023-10-11 19:12:06,035][70582] Avg episode reward: [(0, '11.000'), (1, '10.180')] [2023-10-11 19:12:08,484][71601] Updated weights for policy 0, policy_version 2500 (0.0008) [2023-10-11 19:12:08,857][71601] Updated weights for policy 0, policy_version 2510 (0.0008) [2023-10-11 19:12:09,238][71601] Updated weights for policy 0, policy_version 2520 (0.0009) [2023-10-11 19:12:09,383][71635] Updated weights for policy 1, policy_version 2502 (0.0008) [2023-10-11 19:12:09,753][71635] Updated weights for policy 1, policy_version 2512 (0.0010) [2023-10-11 19:12:10,121][71635] Updated weights for policy 1, policy_version 2522 (0.0008) [2023-10-11 19:12:11,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 5177344. Throughput: 0: 1813.9, 1: 1812.2. Samples: 1300686. Policy #0 lag: (min: 18.0, avg: 18.0, max: 19.0) [2023-10-11 19:12:11,034][70582] Avg episode reward: [(0, '12.070'), (1, '11.380')] [2023-10-11 19:12:13,009][71601] Updated weights for policy 0, policy_version 2530 (0.0009) [2023-10-11 19:12:13,412][71601] Updated weights for policy 0, policy_version 2540 (0.0007) [2023-10-11 19:12:13,707][71635] Updated weights for policy 1, policy_version 2532 (0.0009) [2023-10-11 19:12:13,786][71601] Updated weights for policy 0, policy_version 2550 (0.0008) [2023-10-11 19:12:14,072][71635] Updated weights for policy 1, policy_version 2542 (0.0008) [2023-10-11 19:12:14,161][71601] Updated weights for policy 0, policy_version 2560 (0.0009) [2023-10-11 19:12:14,434][71635] Updated weights for policy 1, policy_version 2552 (0.0010) [2023-10-11 19:12:16,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 5242880. Throughput: 0: 1819.8, 1: 1815.6. Samples: 1313042. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:12:16,034][70582] Avg episode reward: [(0, '11.380'), (1, '13.360')] [2023-10-11 19:12:16,035][71431] Saving new best policy, reward=13.360! [2023-10-11 19:12:17,785][71601] Updated weights for policy 0, policy_version 2570 (0.0009) [2023-10-11 19:12:18,101][71635] Updated weights for policy 1, policy_version 2562 (0.0010) [2023-10-11 19:12:18,165][71601] Updated weights for policy 0, policy_version 2580 (0.0009) [2023-10-11 19:12:18,456][71635] Updated weights for policy 1, policy_version 2572 (0.0008) [2023-10-11 19:12:18,533][71601] Updated weights for policy 0, policy_version 2590 (0.0008) [2023-10-11 19:12:18,820][71635] Updated weights for policy 1, policy_version 2582 (0.0010) [2023-10-11 19:12:19,185][71635] Updated weights for policy 1, policy_version 2592 (0.0011) [2023-10-11 19:12:21,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 5308416. Throughput: 0: 1805.1, 1: 1817.4. Samples: 1333192. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:12:21,034][70582] Avg episode reward: [(0, '11.370'), (1, '12.700')] [2023-10-11 19:12:22,261][71601] Updated weights for policy 0, policy_version 2600 (0.0009) [2023-10-11 19:12:22,640][71601] Updated weights for policy 0, policy_version 2610 (0.0008) [2023-10-11 19:12:22,956][71635] Updated weights for policy 1, policy_version 2602 (0.0008) [2023-10-11 19:12:23,007][71601] Updated weights for policy 0, policy_version 2620 (0.0009) [2023-10-11 19:12:23,329][71635] Updated weights for policy 1, policy_version 2612 (0.0008) [2023-10-11 19:12:23,690][71635] Updated weights for policy 1, policy_version 2622 (0.0009) [2023-10-11 19:12:26,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 5373952. Throughput: 0: 1814.1, 1: 1821.6. Samples: 1355940. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:12:26,035][70582] Avg episode reward: [(0, '11.900'), (1, '13.000')] [2023-10-11 19:12:26,564][71601] Updated weights for policy 0, policy_version 2630 (0.0010) [2023-10-11 19:12:26,937][71601] Updated weights for policy 0, policy_version 2640 (0.0010) [2023-10-11 19:12:27,314][71601] Updated weights for policy 0, policy_version 2650 (0.0009) [2023-10-11 19:12:27,488][71635] Updated weights for policy 1, policy_version 2632 (0.0007) [2023-10-11 19:12:27,865][71635] Updated weights for policy 1, policy_version 2642 (0.0009) [2023-10-11 19:12:28,240][71635] Updated weights for policy 1, policy_version 2652 (0.0010) [2023-10-11 19:12:31,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 5439488. Throughput: 0: 1808.5, 1: 1820.6. Samples: 1365886. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:12:31,035][70582] Avg episode reward: [(0, '11.770'), (1, '12.930')] [2023-10-11 19:12:31,130][71601] Updated weights for policy 0, policy_version 2660 (0.0009) [2023-10-11 19:12:31,512][71601] Updated weights for policy 0, policy_version 2670 (0.0011) [2023-10-11 19:12:31,890][71601] Updated weights for policy 0, policy_version 2680 (0.0009) [2023-10-11 19:12:32,094][71635] Updated weights for policy 1, policy_version 2662 (0.0010) [2023-10-11 19:12:32,466][71635] Updated weights for policy 1, policy_version 2672 (0.0008) [2023-10-11 19:12:32,837][71635] Updated weights for policy 1, policy_version 2682 (0.0008) [2023-10-11 19:12:35,501][71601] Updated weights for policy 0, policy_version 2690 (0.0007) [2023-10-11 19:12:35,885][71601] Updated weights for policy 0, policy_version 2700 (0.0007) [2023-10-11 19:12:36,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 5505024. Throughput: 0: 1808.4, 1: 1813.6. Samples: 1388140. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:12:36,034][70582] Avg episode reward: [(0, '12.740'), (1, '11.420')] [2023-10-11 19:12:36,249][71601] Updated weights for policy 0, policy_version 2710 (0.0007) [2023-10-11 19:12:36,491][71635] Updated weights for policy 1, policy_version 2692 (0.0007) [2023-10-11 19:12:36,627][71601] Updated weights for policy 0, policy_version 2720 (0.0008) [2023-10-11 19:12:36,853][71635] Updated weights for policy 1, policy_version 2702 (0.0010) [2023-10-11 19:12:37,217][71635] Updated weights for policy 1, policy_version 2712 (0.0007) [2023-10-11 19:12:40,462][71601] Updated weights for policy 0, policy_version 2730 (0.0007) [2023-10-11 19:12:40,840][71601] Updated weights for policy 0, policy_version 2740 (0.0007) [2023-10-11 19:12:40,961][71635] Updated weights for policy 1, policy_version 2722 (0.0010) [2023-10-11 19:12:41,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 5570560. Throughput: 0: 1814.5, 1: 1813.5. Samples: 1410466. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:12:41,035][70582] Avg episode reward: [(0, '12.990'), (1, '11.400')] [2023-10-11 19:12:41,210][71601] Updated weights for policy 0, policy_version 2750 (0.0008) [2023-10-11 19:12:41,326][71635] Updated weights for policy 1, policy_version 2732 (0.0009) [2023-10-11 19:12:41,698][71635] Updated weights for policy 1, policy_version 2742 (0.0009) [2023-10-11 19:12:42,060][71635] Updated weights for policy 1, policy_version 2752 (0.0008) [2023-10-11 19:12:44,828][71601] Updated weights for policy 0, policy_version 2760 (0.0008) [2023-10-11 19:12:45,197][71601] Updated weights for policy 0, policy_version 2770 (0.0007) [2023-10-11 19:12:45,567][71601] Updated weights for policy 0, policy_version 2780 (0.0007) [2023-10-11 19:12:45,820][71635] Updated weights for policy 1, policy_version 2762 (0.0009) [2023-10-11 19:12:46,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 5668864. Throughput: 0: 1808.5, 1: 1808.4. Samples: 1420832. Policy #0 lag: (min: 25.0, avg: 30.2, max: 57.0) [2023-10-11 19:12:46,034][70582] Avg episode reward: [(0, '12.810'), (1, '10.950')] [2023-10-11 19:12:46,192][71635] Updated weights for policy 1, policy_version 2772 (0.0008) [2023-10-11 19:12:46,559][71635] Updated weights for policy 1, policy_version 2782 (0.0010) [2023-10-11 19:12:49,303][71601] Updated weights for policy 0, policy_version 2790 (0.0010) [2023-10-11 19:12:49,676][71601] Updated weights for policy 0, policy_version 2800 (0.0011) [2023-10-11 19:12:50,055][71601] Updated weights for policy 0, policy_version 2810 (0.0007) [2023-10-11 19:12:50,276][71635] Updated weights for policy 1, policy_version 2792 (0.0008) [2023-10-11 19:12:50,632][71635] Updated weights for policy 1, policy_version 2802 (0.0010) [2023-10-11 19:12:51,007][71635] Updated weights for policy 1, policy_version 2812 (0.0009) [2023-10-11 19:12:51,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 5734400. Throughput: 0: 1810.4, 1: 1802.7. Samples: 1442782. Policy #0 lag: (min: 25.0, avg: 30.2, max: 57.0) [2023-10-11 19:12:51,035][70582] Avg episode reward: [(0, '11.950'), (1, '9.720')] [2023-10-11 19:12:53,892][71601] Updated weights for policy 0, policy_version 2820 (0.0007) [2023-10-11 19:12:54,273][71601] Updated weights for policy 0, policy_version 2830 (0.0008) [2023-10-11 19:12:54,647][71601] Updated weights for policy 0, policy_version 2840 (0.0009) [2023-10-11 19:12:54,664][71635] Updated weights for policy 1, policy_version 2822 (0.0007) [2023-10-11 19:12:55,036][71635] Updated weights for policy 1, policy_version 2832 (0.0008) [2023-10-11 19:12:55,412][71635] Updated weights for policy 1, policy_version 2842 (0.0008) [2023-10-11 19:12:56,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 5832704. Throughput: 0: 1804.5, 1: 1814.7. Samples: 1463550. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) [2023-10-11 19:12:56,035][70582] Avg episode reward: [(0, '11.500'), (1, '10.120')] [2023-10-11 19:12:58,246][71601] Updated weights for policy 0, policy_version 2850 (0.0008) [2023-10-11 19:12:58,642][71601] Updated weights for policy 0, policy_version 2860 (0.0007) [2023-10-11 19:12:59,010][71601] Updated weights for policy 0, policy_version 2870 (0.0009) [2023-10-11 19:12:59,322][71635] Updated weights for policy 1, policy_version 2852 (0.0010) [2023-10-11 19:12:59,385][71601] Updated weights for policy 0, policy_version 2880 (0.0007) [2023-10-11 19:12:59,699][71635] Updated weights for policy 1, policy_version 2862 (0.0008) [2023-10-11 19:13:00,065][71635] Updated weights for policy 1, policy_version 2872 (0.0009) [2023-10-11 19:13:01,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 5898240. Throughput: 0: 1814.6, 1: 1797.4. Samples: 1475582. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) [2023-10-11 19:13:01,035][70582] Avg episode reward: [(0, '10.820'), (1, '10.180')] [2023-10-11 19:13:02,931][71601] Updated weights for policy 0, policy_version 2890 (0.0008) [2023-10-11 19:13:03,315][71601] Updated weights for policy 0, policy_version 2900 (0.0009) [2023-10-11 19:13:03,701][71601] Updated weights for policy 0, policy_version 2910 (0.0010) [2023-10-11 19:13:03,796][71635] Updated weights for policy 1, policy_version 2882 (0.0008) [2023-10-11 19:13:04,168][71635] Updated weights for policy 1, policy_version 2892 (0.0009) [2023-10-11 19:13:04,533][71635] Updated weights for policy 1, policy_version 2902 (0.0007) [2023-10-11 19:13:04,901][71635] Updated weights for policy 1, policy_version 2912 (0.0009) [2023-10-11 19:13:06,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 5963776. Throughput: 0: 1815.2, 1: 1812.8. Samples: 1496452. Policy #0 lag: (min: 15.0, avg: 19.2, max: 47.0) [2023-10-11 19:13:06,035][70582] Avg episode reward: [(0, '10.740'), (1, '9.690')] [2023-10-11 19:13:07,384][71601] Updated weights for policy 0, policy_version 2920 (0.0008) [2023-10-11 19:13:07,746][71601] Updated weights for policy 0, policy_version 2930 (0.0007) [2023-10-11 19:13:08,124][71601] Updated weights for policy 0, policy_version 2940 (0.0008) [2023-10-11 19:13:08,609][71635] Updated weights for policy 1, policy_version 2922 (0.0009) [2023-10-11 19:13:08,987][71635] Updated weights for policy 1, policy_version 2932 (0.0008) [2023-10-11 19:13:09,357][71635] Updated weights for policy 1, policy_version 2942 (0.0008) [2023-10-11 19:13:11,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 6029312. Throughput: 0: 1812.9, 1: 1800.4. Samples: 1518538. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-11 19:13:11,035][70582] Avg episode reward: [(0, '12.130'), (1, '10.830')] [2023-10-11 19:13:11,764][71601] Updated weights for policy 0, policy_version 2950 (0.0009) [2023-10-11 19:13:12,133][71601] Updated weights for policy 0, policy_version 2960 (0.0007) [2023-10-11 19:13:12,514][71601] Updated weights for policy 0, policy_version 2970 (0.0008) [2023-10-11 19:13:12,992][71635] Updated weights for policy 1, policy_version 2952 (0.0007) [2023-10-11 19:13:13,371][71635] Updated weights for policy 1, policy_version 2962 (0.0010) [2023-10-11 19:13:13,740][71635] Updated weights for policy 1, policy_version 2972 (0.0008) [2023-10-11 19:13:16,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 6094848. Throughput: 0: 1817.2, 1: 1813.2. Samples: 1529256. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-11 19:13:16,035][70582] Avg episode reward: [(0, '13.340'), (1, '10.330')] [2023-10-11 19:13:16,078][71601] Updated weights for policy 0, policy_version 2980 (0.0008) [2023-10-11 19:13:16,460][71601] Updated weights for policy 0, policy_version 2990 (0.0010) [2023-10-11 19:13:16,832][71601] Updated weights for policy 0, policy_version 3000 (0.0010) [2023-10-11 19:13:17,267][71635] Updated weights for policy 1, policy_version 2982 (0.0009) [2023-10-11 19:13:17,634][71635] Updated weights for policy 1, policy_version 2992 (0.0008) [2023-10-11 19:13:18,010][71635] Updated weights for policy 1, policy_version 3002 (0.0009) [2023-10-11 19:13:20,692][71601] Updated weights for policy 0, policy_version 3010 (0.0008) [2023-10-11 19:13:21,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 6160384. Throughput: 0: 1816.6, 1: 1811.5. Samples: 1551402. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-11 19:13:21,035][70582] Avg episode reward: [(0, '13.430'), (1, '10.610')] [2023-10-11 19:13:21,067][71601] Updated weights for policy 0, policy_version 3020 (0.0009) [2023-10-11 19:13:21,443][71601] Updated weights for policy 0, policy_version 3030 (0.0010) [2023-10-11 19:13:21,755][71635] Updated weights for policy 1, policy_version 3012 (0.0007) [2023-10-11 19:13:21,810][71601] Updated weights for policy 0, policy_version 3040 (0.0008) [2023-10-11 19:13:22,127][71635] Updated weights for policy 1, policy_version 3022 (0.0008) [2023-10-11 19:13:22,497][71635] Updated weights for policy 1, policy_version 3032 (0.0008) [2023-10-11 19:13:25,463][71601] Updated weights for policy 0, policy_version 3050 (0.0009) [2023-10-11 19:13:25,829][71601] Updated weights for policy 0, policy_version 3060 (0.0011) [2023-10-11 19:13:26,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 6225920. Throughput: 0: 1817.5, 1: 1810.4. Samples: 1573720. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-11 19:13:26,035][70582] Avg episode reward: [(0, '13.570'), (1, '11.430')] [2023-10-11 19:13:26,113][71635] Updated weights for policy 1, policy_version 3042 (0.0007) [2023-10-11 19:13:26,201][71601] Updated weights for policy 0, policy_version 3070 (0.0009) [2023-10-11 19:13:26,481][71635] Updated weights for policy 1, policy_version 3052 (0.0008) [2023-10-11 19:13:26,845][71635] Updated weights for policy 1, policy_version 3062 (0.0007) [2023-10-11 19:13:27,221][71635] Updated weights for policy 1, policy_version 3072 (0.0008) [2023-10-11 19:13:29,931][71601] Updated weights for policy 0, policy_version 3080 (0.0008) [2023-10-11 19:13:30,304][71601] Updated weights for policy 0, policy_version 3090 (0.0009) [2023-10-11 19:13:30,678][71601] Updated weights for policy 0, policy_version 3100 (0.0009) [2023-10-11 19:13:30,892][71635] Updated weights for policy 1, policy_version 3082 (0.0008) [2023-10-11 19:13:31,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 6324224. Throughput: 0: 1815.1, 1: 1816.0. Samples: 1584230. Policy #0 lag: (min: 3.0, avg: 5.3, max: 35.0) [2023-10-11 19:13:31,035][70582] Avg episode reward: [(0, '13.290'), (1, '11.800')] [2023-10-11 19:13:31,259][71635] Updated weights for policy 1, policy_version 3092 (0.0008) [2023-10-11 19:13:31,622][71635] Updated weights for policy 1, policy_version 3102 (0.0007) [2023-10-11 19:13:34,335][71601] Updated weights for policy 0, policy_version 3110 (0.0008) [2023-10-11 19:13:34,710][71601] Updated weights for policy 0, policy_version 3120 (0.0008) [2023-10-11 19:13:35,091][71601] Updated weights for policy 0, policy_version 3130 (0.0009) [2023-10-11 19:13:35,355][71635] Updated weights for policy 1, policy_version 3112 (0.0007) [2023-10-11 19:13:35,712][71635] Updated weights for policy 1, policy_version 3122 (0.0010) [2023-10-11 19:13:36,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 6389760. Throughput: 0: 1821.6, 1: 1818.1. Samples: 1606568. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:13:36,035][70582] Avg episode reward: [(0, '11.800'), (1, '11.690')] [2023-10-11 19:13:36,084][71635] Updated weights for policy 1, policy_version 3132 (0.0008) [2023-10-11 19:13:38,611][71601] Updated weights for policy 0, policy_version 3140 (0.0007) [2023-10-11 19:13:38,991][71601] Updated weights for policy 0, policy_version 3150 (0.0008) [2023-10-11 19:13:39,359][71601] Updated weights for policy 0, policy_version 3160 (0.0009) [2023-10-11 19:13:39,732][71635] Updated weights for policy 1, policy_version 3142 (0.0008) [2023-10-11 19:13:40,103][71635] Updated weights for policy 1, policy_version 3152 (0.0007) [2023-10-11 19:13:40,480][71635] Updated weights for policy 1, policy_version 3162 (0.0008) [2023-10-11 19:13:41,034][70582] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 6488064. Throughput: 0: 1828.9, 1: 1817.7. Samples: 1627650. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:13:41,035][70582] Avg episode reward: [(0, '11.970'), (1, '12.780')] [2023-10-11 19:13:43,184][71601] Updated weights for policy 0, policy_version 3170 (0.0008) [2023-10-11 19:13:43,581][71601] Updated weights for policy 0, policy_version 3180 (0.0009) [2023-10-11 19:13:43,961][71601] Updated weights for policy 0, policy_version 3190 (0.0009) [2023-10-11 19:13:44,112][71635] Updated weights for policy 1, policy_version 3172 (0.0010) [2023-10-11 19:13:44,330][71601] Updated weights for policy 0, policy_version 3200 (0.0008) [2023-10-11 19:13:44,481][71635] Updated weights for policy 1, policy_version 3182 (0.0010) [2023-10-11 19:13:44,849][71635] Updated weights for policy 1, policy_version 3192 (0.0011) [2023-10-11 19:13:46,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 6553600. Throughput: 0: 1821.6, 1: 1823.6. Samples: 1639618. Policy #0 lag: (min: 29.0, avg: 38.6, max: 61.0) [2023-10-11 19:13:46,035][70582] Avg episode reward: [(0, '11.750'), (1, '11.940')] [2023-10-11 19:13:47,962][71601] Updated weights for policy 0, policy_version 3210 (0.0007) [2023-10-11 19:13:48,342][71601] Updated weights for policy 0, policy_version 3220 (0.0008) [2023-10-11 19:13:48,572][71635] Updated weights for policy 1, policy_version 3202 (0.0009) [2023-10-11 19:13:48,719][71601] Updated weights for policy 0, policy_version 3230 (0.0007) [2023-10-11 19:13:48,942][71635] Updated weights for policy 1, policy_version 3212 (0.0010) [2023-10-11 19:13:49,312][71635] Updated weights for policy 1, policy_version 3222 (0.0011) [2023-10-11 19:13:49,668][71635] Updated weights for policy 1, policy_version 3232 (0.0007) [2023-10-11 19:13:51,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 6619136. Throughput: 0: 1815.4, 1: 1819.7. Samples: 1660032. Policy #0 lag: (min: 29.0, avg: 38.6, max: 61.0) [2023-10-11 19:13:51,035][70582] Avg episode reward: [(0, '10.640'), (1, '11.670')] [2023-10-11 19:13:52,320][71601] Updated weights for policy 0, policy_version 3240 (0.0008) [2023-10-11 19:13:52,701][71601] Updated weights for policy 0, policy_version 3250 (0.0010) [2023-10-11 19:13:53,072][71601] Updated weights for policy 0, policy_version 3260 (0.0009) [2023-10-11 19:13:53,509][71635] Updated weights for policy 1, policy_version 3242 (0.0009) [2023-10-11 19:13:53,882][71635] Updated weights for policy 1, policy_version 3252 (0.0010) [2023-10-11 19:13:54,255][71635] Updated weights for policy 1, policy_version 3262 (0.0011) [2023-10-11 19:13:56,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 6684672. Throughput: 0: 1814.3, 1: 1827.6. Samples: 1682422. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-11 19:13:56,035][70582] Avg episode reward: [(0, '11.020'), (1, '12.740')] [2023-10-11 19:13:56,047][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000003264_3342336.pth... [2023-10-11 19:13:56,047][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000003264_3342336.pth... [2023-10-11 19:13:56,077][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000001568_1605632.pth [2023-10-11 19:13:56,087][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000001568_1605632.pth [2023-10-11 19:13:56,883][71601] Updated weights for policy 0, policy_version 3270 (0.0011) [2023-10-11 19:13:57,271][71601] Updated weights for policy 0, policy_version 3280 (0.0010) [2023-10-11 19:13:57,635][71601] Updated weights for policy 0, policy_version 3290 (0.0010) [2023-10-11 19:13:57,907][71635] Updated weights for policy 1, policy_version 3272 (0.0011) [2023-10-11 19:13:58,290][71635] Updated weights for policy 1, policy_version 3282 (0.0007) [2023-10-11 19:13:58,657][71635] Updated weights for policy 1, policy_version 3292 (0.0007) [2023-10-11 19:14:01,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 6750208. Throughput: 0: 1809.3, 1: 1820.7. Samples: 1692604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:14:01,034][70582] Avg episode reward: [(0, '11.590'), (1, '12.300')] [2023-10-11 19:14:01,405][71601] Updated weights for policy 0, policy_version 3300 (0.0008) [2023-10-11 19:14:01,784][71601] Updated weights for policy 0, policy_version 3310 (0.0010) [2023-10-11 19:14:02,154][71601] Updated weights for policy 0, policy_version 3320 (0.0007) [2023-10-11 19:14:02,336][71635] Updated weights for policy 1, policy_version 3302 (0.0007) [2023-10-11 19:14:02,707][71635] Updated weights for policy 1, policy_version 3312 (0.0008) [2023-10-11 19:14:03,069][71635] Updated weights for policy 1, policy_version 3322 (0.0008) [2023-10-11 19:14:05,899][71601] Updated weights for policy 0, policy_version 3330 (0.0009) [2023-10-11 19:14:06,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 6815744. Throughput: 0: 1807.9, 1: 1817.8. Samples: 1714558. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:14:06,034][70582] Avg episode reward: [(0, '11.850'), (1, '12.980')] [2023-10-11 19:14:06,269][71601] Updated weights for policy 0, policy_version 3340 (0.0007) [2023-10-11 19:14:06,644][71601] Updated weights for policy 0, policy_version 3350 (0.0007) [2023-10-11 19:14:06,822][71635] Updated weights for policy 1, policy_version 3332 (0.0009) [2023-10-11 19:14:07,018][71601] Updated weights for policy 0, policy_version 3360 (0.0009) [2023-10-11 19:14:07,186][71635] Updated weights for policy 1, policy_version 3342 (0.0009) [2023-10-11 19:14:07,561][71635] Updated weights for policy 1, policy_version 3352 (0.0009) [2023-10-11 19:14:10,720][71601] Updated weights for policy 0, policy_version 3370 (0.0009) [2023-10-11 19:14:11,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.6, 300 sec: 14440.1). Total num frames: 6881280. Throughput: 0: 1810.5, 1: 1810.5. Samples: 1736662. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:14:11,034][70582] Avg episode reward: [(0, '12.310'), (1, '13.810')] [2023-10-11 19:14:11,042][71431] Saving new best policy, reward=13.810! [2023-10-11 19:14:11,092][71601] Updated weights for policy 0, policy_version 3380 (0.0009) [2023-10-11 19:14:11,285][71635] Updated weights for policy 1, policy_version 3362 (0.0007) [2023-10-11 19:14:11,469][71601] Updated weights for policy 0, policy_version 3390 (0.0008) [2023-10-11 19:14:11,652][71635] Updated weights for policy 1, policy_version 3372 (0.0009) [2023-10-11 19:14:12,017][71635] Updated weights for policy 1, policy_version 3382 (0.0011) [2023-10-11 19:14:12,385][71635] Updated weights for policy 1, policy_version 3392 (0.0010) [2023-10-11 19:14:15,249][71601] Updated weights for policy 0, policy_version 3400 (0.0007) [2023-10-11 19:14:15,615][71601] Updated weights for policy 0, policy_version 3410 (0.0007) [2023-10-11 19:14:15,994][71601] Updated weights for policy 0, policy_version 3420 (0.0008) [2023-10-11 19:14:16,022][71635] Updated weights for policy 1, policy_version 3402 (0.0009) [2023-10-11 19:14:16,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 6946816. Throughput: 0: 1804.5, 1: 1806.1. Samples: 1746710. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:14:16,035][70582] Avg episode reward: [(0, '11.750'), (1, '12.890')] [2023-10-11 19:14:16,382][71635] Updated weights for policy 1, policy_version 3412 (0.0008) [2023-10-11 19:14:16,753][71635] Updated weights for policy 1, policy_version 3422 (0.0009) [2023-10-11 19:14:19,642][71601] Updated weights for policy 0, policy_version 3430 (0.0008) [2023-10-11 19:14:20,009][71601] Updated weights for policy 0, policy_version 3440 (0.0008) [2023-10-11 19:14:20,385][71635] Updated weights for policy 1, policy_version 3432 (0.0008) [2023-10-11 19:14:20,390][71601] Updated weights for policy 0, policy_version 3450 (0.0008) [2023-10-11 19:14:20,749][71635] Updated weights for policy 1, policy_version 3442 (0.0009) [2023-10-11 19:14:21,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 7045120. Throughput: 0: 1810.4, 1: 1810.3. Samples: 1769498. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:14:21,035][70582] Avg episode reward: [(0, '12.070'), (1, '12.760')] [2023-10-11 19:14:21,123][71635] Updated weights for policy 1, policy_version 3452 (0.0010) [2023-10-11 19:14:24,067][71601] Updated weights for policy 0, policy_version 3460 (0.0008) [2023-10-11 19:14:24,445][71601] Updated weights for policy 0, policy_version 3470 (0.0010) [2023-10-11 19:14:24,824][71601] Updated weights for policy 0, policy_version 3480 (0.0007) [2023-10-11 19:14:24,985][71635] Updated weights for policy 1, policy_version 3462 (0.0008) [2023-10-11 19:14:25,361][71635] Updated weights for policy 1, policy_version 3472 (0.0011) [2023-10-11 19:14:25,735][71635] Updated weights for policy 1, policy_version 3482 (0.0009) [2023-10-11 19:14:26,034][70582] Fps is (10 sec: 19661.2, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 7143424. Throughput: 0: 1796.6, 1: 1811.2. Samples: 1790000. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-11 19:14:26,034][70582] Avg episode reward: [(0, '12.910'), (1, '13.260')] [2023-10-11 19:14:28,597][71601] Updated weights for policy 0, policy_version 3490 (0.0007) [2023-10-11 19:14:28,998][71601] Updated weights for policy 0, policy_version 3500 (0.0009) [2023-10-11 19:14:29,368][71601] Updated weights for policy 0, policy_version 3510 (0.0007) [2023-10-11 19:14:29,469][71635] Updated weights for policy 1, policy_version 3492 (0.0010) [2023-10-11 19:14:29,741][71601] Updated weights for policy 0, policy_version 3520 (0.0008) [2023-10-11 19:14:29,838][71635] Updated weights for policy 1, policy_version 3502 (0.0009) [2023-10-11 19:14:30,195][71635] Updated weights for policy 1, policy_version 3512 (0.0009) [2023-10-11 19:14:31,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 7208960. Throughput: 0: 1810.1, 1: 1800.0. Samples: 1802074. Policy #0 lag: (min: 31.0, avg: 31.8, max: 50.0) [2023-10-11 19:14:31,034][70582] Avg episode reward: [(0, '13.890'), (1, '11.500')] [2023-10-11 19:14:31,035][71353] Saving new best policy, reward=13.890! [2023-10-11 19:14:33,375][71601] Updated weights for policy 0, policy_version 3530 (0.0007) [2023-10-11 19:14:33,754][71601] Updated weights for policy 0, policy_version 3540 (0.0008) [2023-10-11 19:14:34,089][71635] Updated weights for policy 1, policy_version 3522 (0.0009) [2023-10-11 19:14:34,129][71601] Updated weights for policy 0, policy_version 3550 (0.0009) [2023-10-11 19:14:34,464][71635] Updated weights for policy 1, policy_version 3532 (0.0009) [2023-10-11 19:14:34,829][71635] Updated weights for policy 1, policy_version 3542 (0.0007) [2023-10-11 19:14:35,193][71635] Updated weights for policy 1, policy_version 3552 (0.0008) [2023-10-11 19:14:36,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 7274496. Throughput: 0: 1805.5, 1: 1809.0. Samples: 1822686. Policy #0 lag: (min: 31.0, avg: 31.8, max: 50.0) [2023-10-11 19:14:36,034][70582] Avg episode reward: [(0, '14.100'), (1, '11.060')] [2023-10-11 19:14:36,035][71353] Saving new best policy, reward=14.100! [2023-10-11 19:14:37,841][71601] Updated weights for policy 0, policy_version 3560 (0.0009) [2023-10-11 19:14:38,215][71601] Updated weights for policy 0, policy_version 3570 (0.0007) [2023-10-11 19:14:38,596][71601] Updated weights for policy 0, policy_version 3580 (0.0007) [2023-10-11 19:14:38,986][71635] Updated weights for policy 1, policy_version 3562 (0.0009) [2023-10-11 19:14:39,360][71635] Updated weights for policy 1, policy_version 3572 (0.0009) [2023-10-11 19:14:39,727][71635] Updated weights for policy 1, policy_version 3582 (0.0009) [2023-10-11 19:14:41,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 7340032. Throughput: 0: 1799.5, 1: 1792.6. Samples: 1844066. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:14:41,035][70582] Avg episode reward: [(0, '14.900'), (1, '11.510')] [2023-10-11 19:14:41,047][71353] Saving new best policy, reward=14.900! [2023-10-11 19:14:42,472][71601] Updated weights for policy 0, policy_version 3590 (0.0007) [2023-10-11 19:14:42,836][71601] Updated weights for policy 0, policy_version 3600 (0.0007) [2023-10-11 19:14:43,212][71601] Updated weights for policy 0, policy_version 3610 (0.0008) [2023-10-11 19:14:43,390][71635] Updated weights for policy 1, policy_version 3592 (0.0008) [2023-10-11 19:14:43,754][71635] Updated weights for policy 1, policy_version 3602 (0.0007) [2023-10-11 19:14:44,125][71635] Updated weights for policy 1, policy_version 3612 (0.0009) [2023-10-11 19:14:46,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 7405568. Throughput: 0: 1800.2, 1: 1809.8. Samples: 1855052. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:14:46,035][70582] Avg episode reward: [(0, '13.420'), (1, '11.140')] [2023-10-11 19:14:46,920][71601] Updated weights for policy 0, policy_version 3620 (0.0007) [2023-10-11 19:14:47,283][71601] Updated weights for policy 0, policy_version 3630 (0.0007) [2023-10-11 19:14:47,655][71601] Updated weights for policy 0, policy_version 3640 (0.0009) [2023-10-11 19:14:47,779][71635] Updated weights for policy 1, policy_version 3622 (0.0008) [2023-10-11 19:14:48,147][71635] Updated weights for policy 1, policy_version 3632 (0.0009) [2023-10-11 19:14:48,519][71635] Updated weights for policy 1, policy_version 3642 (0.0008) [2023-10-11 19:14:51,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 7471104. Throughput: 0: 1806.1, 1: 1798.1. Samples: 1876750. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:14:51,034][70582] Avg episode reward: [(0, '12.020'), (1, '11.850')] [2023-10-11 19:14:51,305][71601] Updated weights for policy 0, policy_version 3650 (0.0010) [2023-10-11 19:14:51,685][71601] Updated weights for policy 0, policy_version 3660 (0.0010) [2023-10-11 19:14:52,061][71601] Updated weights for policy 0, policy_version 3670 (0.0009) [2023-10-11 19:14:52,156][71635] Updated weights for policy 1, policy_version 3652 (0.0009) [2023-10-11 19:14:52,426][71601] Updated weights for policy 0, policy_version 3680 (0.0009) [2023-10-11 19:14:52,523][71635] Updated weights for policy 1, policy_version 3662 (0.0008) [2023-10-11 19:14:52,888][71635] Updated weights for policy 1, policy_version 3672 (0.0008) [2023-10-11 19:14:56,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 7536640. Throughput: 0: 1808.5, 1: 1807.0. Samples: 1899358. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:14:56,035][70582] Avg episode reward: [(0, '12.140'), (1, '11.370')] [2023-10-11 19:14:56,106][71601] Updated weights for policy 0, policy_version 3690 (0.0008) [2023-10-11 19:14:56,475][71635] Updated weights for policy 1, policy_version 3682 (0.0007) [2023-10-11 19:14:56,477][71601] Updated weights for policy 0, policy_version 3700 (0.0007) [2023-10-11 19:14:56,841][71635] Updated weights for policy 1, policy_version 3692 (0.0008) [2023-10-11 19:14:56,865][71601] Updated weights for policy 0, policy_version 3710 (0.0009) [2023-10-11 19:14:57,207][71635] Updated weights for policy 1, policy_version 3702 (0.0009) [2023-10-11 19:14:57,575][71635] Updated weights for policy 1, policy_version 3712 (0.0009) [2023-10-11 19:15:00,505][71601] Updated weights for policy 0, policy_version 3720 (0.0010) [2023-10-11 19:15:00,883][71601] Updated weights for policy 0, policy_version 3730 (0.0009) [2023-10-11 19:15:01,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 7602176. Throughput: 0: 1803.4, 1: 1807.6. Samples: 1909204. Policy #0 lag: (min: 31.0, avg: 32.3, max: 55.0) [2023-10-11 19:15:01,034][70582] Avg episode reward: [(0, '10.970'), (1, '11.140')] [2023-10-11 19:15:01,252][71601] Updated weights for policy 0, policy_version 3740 (0.0008) [2023-10-11 19:15:01,268][71635] Updated weights for policy 1, policy_version 3722 (0.0007) [2023-10-11 19:15:01,639][71635] Updated weights for policy 1, policy_version 3732 (0.0009) [2023-10-11 19:15:02,011][71635] Updated weights for policy 1, policy_version 3742 (0.0008) [2023-10-11 19:15:05,066][71601] Updated weights for policy 0, policy_version 3750 (0.0007) [2023-10-11 19:15:05,431][71601] Updated weights for policy 0, policy_version 3760 (0.0009) [2023-10-11 19:15:05,727][71635] Updated weights for policy 1, policy_version 3752 (0.0007) [2023-10-11 19:15:05,809][71601] Updated weights for policy 0, policy_version 3770 (0.0010) [2023-10-11 19:15:06,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 7667712. Throughput: 0: 1800.2, 1: 1800.8. Samples: 1931542. Policy #0 lag: (min: 31.0, avg: 32.3, max: 55.0) [2023-10-11 19:15:06,034][70582] Avg episode reward: [(0, '11.010'), (1, '10.680')] [2023-10-11 19:15:06,103][71635] Updated weights for policy 1, policy_version 3762 (0.0008) [2023-10-11 19:15:06,472][71635] Updated weights for policy 1, policy_version 3772 (0.0008) [2023-10-11 19:15:09,753][71601] Updated weights for policy 0, policy_version 3780 (0.0008) [2023-10-11 19:15:10,119][71601] Updated weights for policy 0, policy_version 3790 (0.0008) [2023-10-11 19:15:10,309][71635] Updated weights for policy 1, policy_version 3782 (0.0008) [2023-10-11 19:15:10,485][71601] Updated weights for policy 0, policy_version 3800 (0.0008) [2023-10-11 19:15:10,680][71635] Updated weights for policy 1, policy_version 3792 (0.0008) [2023-10-11 19:15:11,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 7766016. Throughput: 0: 1804.8, 1: 1813.1. Samples: 1952808. Policy #0 lag: (min: 16.0, avg: 35.8, max: 48.0) [2023-10-11 19:15:11,035][70582] Avg episode reward: [(0, '10.690'), (1, '11.590')] [2023-10-11 19:15:11,056][71635] Updated weights for policy 1, policy_version 3802 (0.0007) [2023-10-11 19:15:14,124][71601] Updated weights for policy 0, policy_version 3810 (0.0009) [2023-10-11 19:15:14,518][71601] Updated weights for policy 0, policy_version 3820 (0.0009) [2023-10-11 19:15:14,722][71635] Updated weights for policy 1, policy_version 3812 (0.0008) [2023-10-11 19:15:14,895][71601] Updated weights for policy 0, policy_version 3830 (0.0007) [2023-10-11 19:15:15,092][71635] Updated weights for policy 1, policy_version 3822 (0.0008) [2023-10-11 19:15:15,271][71601] Updated weights for policy 0, policy_version 3840 (0.0008) [2023-10-11 19:15:15,456][71635] Updated weights for policy 1, policy_version 3832 (0.0007) [2023-10-11 19:15:16,034][70582] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 7864320. Throughput: 0: 1795.2, 1: 1805.8. Samples: 1964120. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-11 19:15:16,035][70582] Avg episode reward: [(0, '10.920'), (1, '12.380')] [2023-10-11 19:15:18,697][71601] Updated weights for policy 0, policy_version 3850 (0.0008) [2023-10-11 19:15:19,080][71601] Updated weights for policy 0, policy_version 3860 (0.0009) [2023-10-11 19:15:19,283][71635] Updated weights for policy 1, policy_version 3842 (0.0008) [2023-10-11 19:15:19,444][71601] Updated weights for policy 0, policy_version 3870 (0.0008) [2023-10-11 19:15:19,650][71635] Updated weights for policy 1, policy_version 3852 (0.0010) [2023-10-11 19:15:20,025][71635] Updated weights for policy 1, policy_version 3862 (0.0011) [2023-10-11 19:15:20,392][71635] Updated weights for policy 1, policy_version 3872 (0.0012) [2023-10-11 19:15:21,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 7929856. Throughput: 0: 1801.6, 1: 1812.5. Samples: 1985322. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-11 19:15:21,034][70582] Avg episode reward: [(0, '10.690'), (1, '13.170')] [2023-10-11 19:15:23,204][71601] Updated weights for policy 0, policy_version 3880 (0.0009) [2023-10-11 19:15:23,580][71601] Updated weights for policy 0, policy_version 3890 (0.0008) [2023-10-11 19:15:23,959][71601] Updated weights for policy 0, policy_version 3900 (0.0009) [2023-10-11 19:15:24,045][71635] Updated weights for policy 1, policy_version 3882 (0.0010) [2023-10-11 19:15:24,408][71635] Updated weights for policy 1, policy_version 3892 (0.0009) [2023-10-11 19:15:24,779][71635] Updated weights for policy 1, policy_version 3902 (0.0010) [2023-10-11 19:15:26,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 7995392. Throughput: 0: 1805.8, 1: 1813.7. Samples: 2006944. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-11 19:15:26,034][70582] Avg episode reward: [(0, '13.020'), (1, '13.250')] [2023-10-11 19:15:27,646][71601] Updated weights for policy 0, policy_version 3910 (0.0010) [2023-10-11 19:15:28,010][71601] Updated weights for policy 0, policy_version 3920 (0.0007) [2023-10-11 19:15:28,384][71601] Updated weights for policy 0, policy_version 3930 (0.0008) [2023-10-11 19:15:28,459][71635] Updated weights for policy 1, policy_version 3912 (0.0007) [2023-10-11 19:15:28,837][71635] Updated weights for policy 1, policy_version 3922 (0.0009) [2023-10-11 19:15:29,199][71635] Updated weights for policy 1, policy_version 3932 (0.0010) [2023-10-11 19:15:31,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 8060928. Throughput: 0: 1813.5, 1: 1814.6. Samples: 2018314. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-11 19:15:31,035][70582] Avg episode reward: [(0, '14.190'), (1, '13.990')] [2023-10-11 19:15:31,037][71431] Saving new best policy, reward=13.990! [2023-10-11 19:15:32,154][71601] Updated weights for policy 0, policy_version 3940 (0.0008) [2023-10-11 19:15:32,528][71601] Updated weights for policy 0, policy_version 3950 (0.0008) [2023-10-11 19:15:32,852][71635] Updated weights for policy 1, policy_version 3942 (0.0008) [2023-10-11 19:15:32,903][71601] Updated weights for policy 0, policy_version 3960 (0.0008) [2023-10-11 19:15:33,215][71635] Updated weights for policy 1, policy_version 3952 (0.0007) [2023-10-11 19:15:33,595][71635] Updated weights for policy 1, policy_version 3962 (0.0008) [2023-10-11 19:15:36,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 8126464. Throughput: 0: 1804.7, 1: 1809.3. Samples: 2039380. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-10-11 19:15:36,034][70582] Avg episode reward: [(0, '13.550'), (1, '12.930')] [2023-10-11 19:15:36,587][71601] Updated weights for policy 0, policy_version 3970 (0.0007) [2023-10-11 19:15:36,964][71601] Updated weights for policy 0, policy_version 3980 (0.0009) [2023-10-11 19:15:37,334][71601] Updated weights for policy 0, policy_version 3990 (0.0008) [2023-10-11 19:15:37,372][71635] Updated weights for policy 1, policy_version 3972 (0.0008) [2023-10-11 19:15:37,701][71601] Updated weights for policy 0, policy_version 4000 (0.0010) [2023-10-11 19:15:37,740][71635] Updated weights for policy 1, policy_version 3982 (0.0009) [2023-10-11 19:15:38,118][71635] Updated weights for policy 1, policy_version 3992 (0.0009) [2023-10-11 19:15:41,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 8192000. Throughput: 0: 1809.2, 1: 1803.7. Samples: 2061936. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-10-11 19:15:41,034][70582] Avg episode reward: [(0, '14.170'), (1, '12.200')] [2023-10-11 19:15:41,534][71601] Updated weights for policy 0, policy_version 4010 (0.0009) [2023-10-11 19:15:41,813][71635] Updated weights for policy 1, policy_version 4002 (0.0009) [2023-10-11 19:15:41,913][71601] Updated weights for policy 0, policy_version 4020 (0.0008) [2023-10-11 19:15:42,178][71635] Updated weights for policy 1, policy_version 4012 (0.0007) [2023-10-11 19:15:42,281][71601] Updated weights for policy 0, policy_version 4030 (0.0007) [2023-10-11 19:15:42,540][71635] Updated weights for policy 1, policy_version 4022 (0.0009) [2023-10-11 19:15:42,911][71635] Updated weights for policy 1, policy_version 4032 (0.0009) [2023-10-11 19:15:46,024][71601] Updated weights for policy 0, policy_version 4040 (0.0008) [2023-10-11 19:15:46,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 8257536. Throughput: 0: 1809.1, 1: 1806.2. Samples: 2071892. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:15:46,034][70582] Avg episode reward: [(0, '11.820'), (1, '11.360')] [2023-10-11 19:15:46,393][71601] Updated weights for policy 0, policy_version 4050 (0.0007) [2023-10-11 19:15:46,563][71635] Updated weights for policy 1, policy_version 4042 (0.0007) [2023-10-11 19:15:46,773][71601] Updated weights for policy 0, policy_version 4060 (0.0007) [2023-10-11 19:15:46,932][71635] Updated weights for policy 1, policy_version 4052 (0.0008) [2023-10-11 19:15:47,294][71635] Updated weights for policy 1, policy_version 4062 (0.0009) [2023-10-11 19:15:50,650][71601] Updated weights for policy 0, policy_version 4070 (0.0007) [2023-10-11 19:15:51,016][71601] Updated weights for policy 0, policy_version 4080 (0.0008) [2023-10-11 19:15:51,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 8323072. Throughput: 0: 1805.5, 1: 1810.7. Samples: 2094268. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:15:51,034][70582] Avg episode reward: [(0, '11.150'), (1, '10.930')] [2023-10-11 19:15:51,035][71635] Updated weights for policy 1, policy_version 4072 (0.0008) [2023-10-11 19:15:51,391][71601] Updated weights for policy 0, policy_version 4090 (0.0007) [2023-10-11 19:15:51,398][71635] Updated weights for policy 1, policy_version 4082 (0.0007) [2023-10-11 19:15:51,770][71635] Updated weights for policy 1, policy_version 4092 (0.0008) [2023-10-11 19:15:54,982][71601] Updated weights for policy 0, policy_version 4100 (0.0007) [2023-10-11 19:15:55,365][71601] Updated weights for policy 0, policy_version 4110 (0.0007) [2023-10-11 19:15:55,561][71635] Updated weights for policy 1, policy_version 4102 (0.0008) [2023-10-11 19:15:55,730][71601] Updated weights for policy 0, policy_version 4120 (0.0007) [2023-10-11 19:15:55,923][71635] Updated weights for policy 1, policy_version 4112 (0.0007) [2023-10-11 19:15:56,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 8421376. Throughput: 0: 1819.3, 1: 1819.9. Samples: 2116570. Policy #0 lag: (min: 2.0, avg: 8.9, max: 34.0) [2023-10-11 19:15:56,034][70582] Avg episode reward: [(0, '11.470'), (1, '11.400')] [2023-10-11 19:15:56,042][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000004128_4227072.pth... [2023-10-11 19:15:56,071][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000002432_2490368.pth [2023-10-11 19:15:56,295][71635] Updated weights for policy 1, policy_version 4122 (0.0007) [2023-10-11 19:15:56,509][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000004128_4227072.pth... [2023-10-11 19:15:56,549][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000002432_2490368.pth [2023-10-11 19:15:59,448][71601] Updated weights for policy 0, policy_version 4130 (0.0009) [2023-10-11 19:15:59,859][71601] Updated weights for policy 0, policy_version 4140 (0.0010) [2023-10-11 19:16:00,065][71635] Updated weights for policy 1, policy_version 4132 (0.0010) [2023-10-11 19:16:00,238][71601] Updated weights for policy 0, policy_version 4150 (0.0008) [2023-10-11 19:16:00,438][71635] Updated weights for policy 1, policy_version 4142 (0.0008) [2023-10-11 19:16:00,604][71601] Updated weights for policy 0, policy_version 4160 (0.0007) [2023-10-11 19:16:00,804][71635] Updated weights for policy 1, policy_version 4152 (0.0008) [2023-10-11 19:16:01,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 8486912. Throughput: 0: 1813.3, 1: 1808.9. Samples: 2127120. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:16:01,035][70582] Avg episode reward: [(0, '12.440'), (1, '11.430')] [2023-10-11 19:16:04,308][71601] Updated weights for policy 0, policy_version 4170 (0.0008) [2023-10-11 19:16:04,514][71635] Updated weights for policy 1, policy_version 4162 (0.0007) [2023-10-11 19:16:04,678][71601] Updated weights for policy 0, policy_version 4180 (0.0007) [2023-10-11 19:16:04,890][71635] Updated weights for policy 1, policy_version 4172 (0.0007) [2023-10-11 19:16:05,055][71601] Updated weights for policy 0, policy_version 4190 (0.0007) [2023-10-11 19:16:05,258][71635] Updated weights for policy 1, policy_version 4182 (0.0007) [2023-10-11 19:16:05,624][71635] Updated weights for policy 1, policy_version 4192 (0.0007) [2023-10-11 19:16:06,034][70582] Fps is (10 sec: 16384.4, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 8585216. Throughput: 0: 1823.1, 1: 1812.8. Samples: 2148938. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:16:06,034][70582] Avg episode reward: [(0, '13.300'), (1, '12.880')] [2023-10-11 19:16:08,748][71601] Updated weights for policy 0, policy_version 4200 (0.0009) [2023-10-11 19:16:09,124][71601] Updated weights for policy 0, policy_version 4210 (0.0009) [2023-10-11 19:16:09,351][71635] Updated weights for policy 1, policy_version 4202 (0.0008) [2023-10-11 19:16:09,503][71601] Updated weights for policy 0, policy_version 4220 (0.0010) [2023-10-11 19:16:09,706][71635] Updated weights for policy 1, policy_version 4212 (0.0008) [2023-10-11 19:16:10,079][71635] Updated weights for policy 1, policy_version 4222 (0.0007) [2023-10-11 19:16:11,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 8650752. Throughput: 0: 1800.1, 1: 1806.8. Samples: 2169258. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:16:11,034][70582] Avg episode reward: [(0, '13.400'), (1, '13.300')] [2023-10-11 19:16:13,107][71601] Updated weights for policy 0, policy_version 4230 (0.0007) [2023-10-11 19:16:13,473][71601] Updated weights for policy 0, policy_version 4240 (0.0008) [2023-10-11 19:16:13,840][71601] Updated weights for policy 0, policy_version 4250 (0.0007) [2023-10-11 19:16:13,844][71635] Updated weights for policy 1, policy_version 4232 (0.0008) [2023-10-11 19:16:14,222][71635] Updated weights for policy 1, policy_version 4242 (0.0009) [2023-10-11 19:16:14,592][71635] Updated weights for policy 1, policy_version 4252 (0.0010) [2023-10-11 19:16:16,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 8716288. Throughput: 0: 1812.8, 1: 1815.3. Samples: 2181576. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:16:16,034][70582] Avg episode reward: [(0, '13.030'), (1, '13.210')] [2023-10-11 19:16:17,576][71601] Updated weights for policy 0, policy_version 4260 (0.0007) [2023-10-11 19:16:17,935][71601] Updated weights for policy 0, policy_version 4270 (0.0009) [2023-10-11 19:16:18,288][71635] Updated weights for policy 1, policy_version 4262 (0.0009) [2023-10-11 19:16:18,311][71601] Updated weights for policy 0, policy_version 4280 (0.0007) [2023-10-11 19:16:18,656][71635] Updated weights for policy 1, policy_version 4272 (0.0009) [2023-10-11 19:16:19,019][71635] Updated weights for policy 1, policy_version 4282 (0.0008) [2023-10-11 19:16:21,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 8781824. Throughput: 0: 1802.9, 1: 1806.6. Samples: 2201808. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) [2023-10-11 19:16:21,034][70582] Avg episode reward: [(0, '12.400'), (1, '11.980')] [2023-10-11 19:16:22,025][71601] Updated weights for policy 0, policy_version 4290 (0.0008) [2023-10-11 19:16:22,395][71601] Updated weights for policy 0, policy_version 4300 (0.0008) [2023-10-11 19:16:22,775][71601] Updated weights for policy 0, policy_version 4310 (0.0008) [2023-10-11 19:16:22,822][71635] Updated weights for policy 1, policy_version 4292 (0.0008) [2023-10-11 19:16:23,146][71601] Updated weights for policy 0, policy_version 4320 (0.0009) [2023-10-11 19:16:23,177][71635] Updated weights for policy 1, policy_version 4302 (0.0007) [2023-10-11 19:16:23,553][71635] Updated weights for policy 1, policy_version 4312 (0.0009) [2023-10-11 19:16:26,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 8847360. Throughput: 0: 1799.3, 1: 1807.1. Samples: 2224226. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) [2023-10-11 19:16:26,034][70582] Avg episode reward: [(0, '12.860'), (1, '10.760')] [2023-10-11 19:16:26,931][71601] Updated weights for policy 0, policy_version 4330 (0.0011) [2023-10-11 19:16:27,283][71635] Updated weights for policy 1, policy_version 4322 (0.0008) [2023-10-11 19:16:27,307][71601] Updated weights for policy 0, policy_version 4340 (0.0009) [2023-10-11 19:16:27,642][71635] Updated weights for policy 1, policy_version 4332 (0.0009) [2023-10-11 19:16:27,689][71601] Updated weights for policy 0, policy_version 4350 (0.0007) [2023-10-11 19:16:28,017][71635] Updated weights for policy 1, policy_version 4342 (0.0008) [2023-10-11 19:16:28,379][71635] Updated weights for policy 1, policy_version 4352 (0.0008) [2023-10-11 19:16:31,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 8912896. Throughput: 0: 1800.3, 1: 1804.2. Samples: 2234094. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:16:31,034][70582] Avg episode reward: [(0, '12.540'), (1, '11.270')] [2023-10-11 19:16:31,345][71601] Updated weights for policy 0, policy_version 4360 (0.0009) [2023-10-11 19:16:31,719][71601] Updated weights for policy 0, policy_version 4370 (0.0009) [2023-10-11 19:16:32,018][71635] Updated weights for policy 1, policy_version 4362 (0.0009) [2023-10-11 19:16:32,099][71601] Updated weights for policy 0, policy_version 4380 (0.0008) [2023-10-11 19:16:32,384][71635] Updated weights for policy 1, policy_version 4372 (0.0009) [2023-10-11 19:16:32,760][71635] Updated weights for policy 1, policy_version 4382 (0.0012) [2023-10-11 19:16:35,985][71601] Updated weights for policy 0, policy_version 4390 (0.0009) [2023-10-11 19:16:36,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 8978432. Throughput: 0: 1802.3, 1: 1798.7. Samples: 2256318. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:16:36,035][70582] Avg episode reward: [(0, '12.410'), (1, '10.770')] [2023-10-11 19:16:36,360][71601] Updated weights for policy 0, policy_version 4400 (0.0009) [2023-10-11 19:16:36,539][71635] Updated weights for policy 1, policy_version 4392 (0.0009) [2023-10-11 19:16:36,726][71601] Updated weights for policy 0, policy_version 4410 (0.0008) [2023-10-11 19:16:36,908][71635] Updated weights for policy 1, policy_version 4402 (0.0008) [2023-10-11 19:16:37,279][71635] Updated weights for policy 1, policy_version 4412 (0.0009) [2023-10-11 19:16:40,307][71601] Updated weights for policy 0, policy_version 4420 (0.0007) [2023-10-11 19:16:40,681][71601] Updated weights for policy 0, policy_version 4430 (0.0008) [2023-10-11 19:16:40,925][71635] Updated weights for policy 1, policy_version 4422 (0.0008) [2023-10-11 19:16:41,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 9043968. Throughput: 0: 1804.4, 1: 1799.8. Samples: 2278760. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-11 19:16:41,035][70582] Avg episode reward: [(0, '12.330'), (1, '11.660')] [2023-10-11 19:16:41,052][71601] Updated weights for policy 0, policy_version 4440 (0.0008) [2023-10-11 19:16:41,293][71635] Updated weights for policy 1, policy_version 4432 (0.0009) [2023-10-11 19:16:41,664][71635] Updated weights for policy 1, policy_version 4442 (0.0010) [2023-10-11 19:16:44,767][71601] Updated weights for policy 0, policy_version 4450 (0.0008) [2023-10-11 19:16:45,160][71601] Updated weights for policy 0, policy_version 4460 (0.0008) [2023-10-11 19:16:45,395][71635] Updated weights for policy 1, policy_version 4452 (0.0009) [2023-10-11 19:16:45,530][71601] Updated weights for policy 0, policy_version 4470 (0.0007) [2023-10-11 19:16:45,768][71635] Updated weights for policy 1, policy_version 4462 (0.0008) [2023-10-11 19:16:45,904][71601] Updated weights for policy 0, policy_version 4480 (0.0007) [2023-10-11 19:16:46,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 9142272. Throughput: 0: 1794.8, 1: 1802.7. Samples: 2289008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:16:46,034][70582] Avg episode reward: [(0, '12.100'), (1, '11.170')] [2023-10-11 19:16:46,142][71635] Updated weights for policy 1, policy_version 4472 (0.0007) [2023-10-11 19:16:49,677][71601] Updated weights for policy 0, policy_version 4490 (0.0010) [2023-10-11 19:16:49,915][71635] Updated weights for policy 1, policy_version 4482 (0.0008) [2023-10-11 19:16:50,044][71601] Updated weights for policy 0, policy_version 4500 (0.0007) [2023-10-11 19:16:50,284][71635] Updated weights for policy 1, policy_version 4492 (0.0009) [2023-10-11 19:16:50,417][71601] Updated weights for policy 0, policy_version 4510 (0.0008) [2023-10-11 19:16:50,646][71635] Updated weights for policy 1, policy_version 4502 (0.0008) [2023-10-11 19:16:51,012][71635] Updated weights for policy 1, policy_version 4512 (0.0008) [2023-10-11 19:16:51,034][70582] Fps is (10 sec: 19661.2, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 9240576. Throughput: 0: 1800.2, 1: 1803.9. Samples: 2311124. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:16:51,034][70582] Avg episode reward: [(0, '11.940'), (1, '11.520')] [2023-10-11 19:16:54,061][71601] Updated weights for policy 0, policy_version 4520 (0.0010) [2023-10-11 19:16:54,427][71601] Updated weights for policy 0, policy_version 4530 (0.0009) [2023-10-11 19:16:54,691][71635] Updated weights for policy 1, policy_version 4522 (0.0007) [2023-10-11 19:16:54,798][71601] Updated weights for policy 0, policy_version 4540 (0.0009) [2023-10-11 19:16:55,057][71635] Updated weights for policy 1, policy_version 4532 (0.0010) [2023-10-11 19:16:55,421][71635] Updated weights for policy 1, policy_version 4542 (0.0010) [2023-10-11 19:16:56,034][70582] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 9306112. Throughput: 0: 1794.6, 1: 1807.1. Samples: 2331338. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:16:56,035][70582] Avg episode reward: [(0, '13.580'), (1, '11.610')] [2023-10-11 19:16:58,535][71601] Updated weights for policy 0, policy_version 4550 (0.0008) [2023-10-11 19:16:58,913][71601] Updated weights for policy 0, policy_version 4560 (0.0010) [2023-10-11 19:16:59,278][71601] Updated weights for policy 0, policy_version 4570 (0.0009) [2023-10-11 19:16:59,292][71635] Updated weights for policy 1, policy_version 4552 (0.0009) [2023-10-11 19:16:59,669][71635] Updated weights for policy 1, policy_version 4562 (0.0008) [2023-10-11 19:17:00,038][71635] Updated weights for policy 1, policy_version 4572 (0.0009) [2023-10-11 19:17:01,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 9371648. Throughput: 0: 1805.6, 1: 1800.3. Samples: 2343842. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:17:01,034][70582] Avg episode reward: [(0, '13.020'), (1, '11.690')] [2023-10-11 19:17:02,850][71601] Updated weights for policy 0, policy_version 4580 (0.0009) [2023-10-11 19:17:03,221][71601] Updated weights for policy 0, policy_version 4590 (0.0009) [2023-10-11 19:17:03,592][71601] Updated weights for policy 0, policy_version 4600 (0.0007) [2023-10-11 19:17:03,673][71635] Updated weights for policy 1, policy_version 4582 (0.0007) [2023-10-11 19:17:04,026][71635] Updated weights for policy 1, policy_version 4592 (0.0008) [2023-10-11 19:17:04,391][71635] Updated weights for policy 1, policy_version 4602 (0.0010) [2023-10-11 19:17:06,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 9437184. Throughput: 0: 1792.1, 1: 1812.7. Samples: 2364022. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-11 19:17:06,035][70582] Avg episode reward: [(0, '13.250'), (1, '11.310')] [2023-10-11 19:17:07,380][71601] Updated weights for policy 0, policy_version 4610 (0.0008) [2023-10-11 19:17:07,750][71601] Updated weights for policy 0, policy_version 4620 (0.0010) [2023-10-11 19:17:08,120][71601] Updated weights for policy 0, policy_version 4630 (0.0010) [2023-10-11 19:17:08,130][71635] Updated weights for policy 1, policy_version 4612 (0.0007) [2023-10-11 19:17:08,495][71635] Updated weights for policy 1, policy_version 4622 (0.0009) [2023-10-11 19:17:08,498][71601] Updated weights for policy 0, policy_version 4640 (0.0008) [2023-10-11 19:17:08,856][71635] Updated weights for policy 1, policy_version 4632 (0.0010) [2023-10-11 19:17:11,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 9502720. Throughput: 0: 1798.0, 1: 1809.1. Samples: 2386546. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-11 19:17:11,035][70582] Avg episode reward: [(0, '12.970'), (1, '10.990')] [2023-10-11 19:17:12,268][71601] Updated weights for policy 0, policy_version 4650 (0.0007) [2023-10-11 19:17:12,498][71635] Updated weights for policy 1, policy_version 4642 (0.0010) [2023-10-11 19:17:12,633][71601] Updated weights for policy 0, policy_version 4660 (0.0007) [2023-10-11 19:17:12,855][71635] Updated weights for policy 1, policy_version 4652 (0.0008) [2023-10-11 19:17:13,002][71601] Updated weights for policy 0, policy_version 4670 (0.0007) [2023-10-11 19:17:13,223][71635] Updated weights for policy 1, policy_version 4662 (0.0011) [2023-10-11 19:17:13,588][71635] Updated weights for policy 1, policy_version 4672 (0.0009) [2023-10-11 19:17:16,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 9568256. Throughput: 0: 1802.5, 1: 1816.4. Samples: 2396944. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-11 19:17:16,035][70582] Avg episode reward: [(0, '11.730'), (1, '10.950')] [2023-10-11 19:17:16,692][71601] Updated weights for policy 0, policy_version 4680 (0.0009) [2023-10-11 19:17:17,072][71601] Updated weights for policy 0, policy_version 4690 (0.0009) [2023-10-11 19:17:17,395][71635] Updated weights for policy 1, policy_version 4682 (0.0009) [2023-10-11 19:17:17,437][71601] Updated weights for policy 0, policy_version 4700 (0.0007) [2023-10-11 19:17:17,760][71635] Updated weights for policy 1, policy_version 4692 (0.0008) [2023-10-11 19:17:18,129][71635] Updated weights for policy 1, policy_version 4702 (0.0009) [2023-10-11 19:17:21,014][71601] Updated weights for policy 0, policy_version 4710 (0.0007) [2023-10-11 19:17:21,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 9633792. Throughput: 0: 1810.3, 1: 1806.5. Samples: 2419076. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-11 19:17:21,035][70582] Avg episode reward: [(0, '11.550'), (1, '11.110')] [2023-10-11 19:17:21,386][71601] Updated weights for policy 0, policy_version 4720 (0.0007) [2023-10-11 19:17:21,759][71601] Updated weights for policy 0, policy_version 4730 (0.0007) [2023-10-11 19:17:21,883][71635] Updated weights for policy 1, policy_version 4712 (0.0008) [2023-10-11 19:17:22,249][71635] Updated weights for policy 1, policy_version 4722 (0.0008) [2023-10-11 19:17:22,623][71635] Updated weights for policy 1, policy_version 4732 (0.0008) [2023-10-11 19:17:25,487][71601] Updated weights for policy 0, policy_version 4740 (0.0007) [2023-10-11 19:17:25,861][71601] Updated weights for policy 0, policy_version 4750 (0.0009) [2023-10-11 19:17:26,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 9699328. Throughput: 0: 1815.4, 1: 1803.6. Samples: 2441616. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:17:26,034][70582] Avg episode reward: [(0, '11.340'), (1, '11.750')] [2023-10-11 19:17:26,237][71601] Updated weights for policy 0, policy_version 4760 (0.0007) [2023-10-11 19:17:26,297][71635] Updated weights for policy 1, policy_version 4742 (0.0008) [2023-10-11 19:17:26,671][71635] Updated weights for policy 1, policy_version 4752 (0.0008) [2023-10-11 19:17:27,035][71635] Updated weights for policy 1, policy_version 4762 (0.0007) [2023-10-11 19:17:30,037][71601] Updated weights for policy 0, policy_version 4770 (0.0010) [2023-10-11 19:17:30,440][71601] Updated weights for policy 0, policy_version 4780 (0.0007) [2023-10-11 19:17:30,760][71635] Updated weights for policy 1, policy_version 4772 (0.0008) [2023-10-11 19:17:30,813][71601] Updated weights for policy 0, policy_version 4790 (0.0009) [2023-10-11 19:17:31,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 9764864. Throughput: 0: 1810.9, 1: 1805.2. Samples: 2451734. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:17:31,034][70582] Avg episode reward: [(0, '12.030'), (1, '11.020')] [2023-10-11 19:17:31,136][71635] Updated weights for policy 1, policy_version 4782 (0.0009) [2023-10-11 19:17:31,181][71601] Updated weights for policy 0, policy_version 4800 (0.0008) [2023-10-11 19:17:31,503][71635] Updated weights for policy 1, policy_version 4792 (0.0009) [2023-10-11 19:17:34,846][71601] Updated weights for policy 0, policy_version 4810 (0.0009) [2023-10-11 19:17:35,140][71635] Updated weights for policy 1, policy_version 4802 (0.0007) [2023-10-11 19:17:35,206][71601] Updated weights for policy 0, policy_version 4820 (0.0007) [2023-10-11 19:17:35,512][71635] Updated weights for policy 1, policy_version 4812 (0.0007) [2023-10-11 19:17:35,580][71601] Updated weights for policy 0, policy_version 4830 (0.0009) [2023-10-11 19:17:35,882][71635] Updated weights for policy 1, policy_version 4822 (0.0011) [2023-10-11 19:17:36,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 9863168. Throughput: 0: 1818.8, 1: 1806.4. Samples: 2474260. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-11 19:17:36,035][70582] Avg episode reward: [(0, '12.130'), (1, '10.950')] [2023-10-11 19:17:36,240][71635] Updated weights for policy 1, policy_version 4832 (0.0011) [2023-10-11 19:17:39,472][71601] Updated weights for policy 0, policy_version 4840 (0.0009) [2023-10-11 19:17:39,851][71601] Updated weights for policy 0, policy_version 4850 (0.0010) [2023-10-11 19:17:40,042][71635] Updated weights for policy 1, policy_version 4842 (0.0008) [2023-10-11 19:17:40,218][71601] Updated weights for policy 0, policy_version 4860 (0.0008) [2023-10-11 19:17:40,415][71635] Updated weights for policy 1, policy_version 4852 (0.0008) [2023-10-11 19:17:40,774][71635] Updated weights for policy 1, policy_version 4862 (0.0009) [2023-10-11 19:17:41,034][70582] Fps is (10 sec: 19660.2, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 9961472. Throughput: 0: 1810.4, 1: 1817.5. Samples: 2494596. Policy #0 lag: (min: 12.0, avg: 18.6, max: 44.0) [2023-10-11 19:17:41,035][70582] Avg episode reward: [(0, '11.270'), (1, '10.060')] [2023-10-11 19:17:43,935][71601] Updated weights for policy 0, policy_version 4870 (0.0007) [2023-10-11 19:17:44,304][71601] Updated weights for policy 0, policy_version 4880 (0.0008) [2023-10-11 19:17:44,578][71635] Updated weights for policy 1, policy_version 4872 (0.0009) [2023-10-11 19:17:44,682][71601] Updated weights for policy 0, policy_version 4890 (0.0009) [2023-10-11 19:17:44,955][71635] Updated weights for policy 1, policy_version 4882 (0.0008) [2023-10-11 19:17:45,322][71635] Updated weights for policy 1, policy_version 4892 (0.0007) [2023-10-11 19:17:46,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 10027008. Throughput: 0: 1815.5, 1: 1806.8. Samples: 2506850. Policy #0 lag: (min: 12.0, avg: 18.6, max: 44.0) [2023-10-11 19:17:46,035][70582] Avg episode reward: [(0, '10.880'), (1, '10.550')] [2023-10-11 19:17:48,489][71601] Updated weights for policy 0, policy_version 4900 (0.0008) [2023-10-11 19:17:48,854][71601] Updated weights for policy 0, policy_version 4910 (0.0009) [2023-10-11 19:17:49,142][71635] Updated weights for policy 1, policy_version 4902 (0.0008) [2023-10-11 19:17:49,225][71601] Updated weights for policy 0, policy_version 4920 (0.0009) [2023-10-11 19:17:49,510][71635] Updated weights for policy 1, policy_version 4912 (0.0008) [2023-10-11 19:17:49,871][71635] Updated weights for policy 1, policy_version 4922 (0.0011) [2023-10-11 19:17:51,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 10092544. Throughput: 0: 1810.6, 1: 1812.5. Samples: 2527064. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-11 19:17:51,034][70582] Avg episode reward: [(0, '11.670'), (1, '11.080')] [2023-10-11 19:17:53,027][71601] Updated weights for policy 0, policy_version 4930 (0.0008) [2023-10-11 19:17:53,393][71601] Updated weights for policy 0, policy_version 4940 (0.0008) [2023-10-11 19:17:53,400][71635] Updated weights for policy 1, policy_version 4932 (0.0009) [2023-10-11 19:17:53,760][71601] Updated weights for policy 0, policy_version 4950 (0.0008) [2023-10-11 19:17:53,772][71635] Updated weights for policy 1, policy_version 4942 (0.0008) [2023-10-11 19:17:54,133][71601] Updated weights for policy 0, policy_version 4960 (0.0009) [2023-10-11 19:17:54,137][71635] Updated weights for policy 1, policy_version 4952 (0.0007) [2023-10-11 19:17:56,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 10158080. Throughput: 0: 1802.1, 1: 1800.0. Samples: 2548644. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-11 19:17:56,035][70582] Avg episode reward: [(0, '11.150'), (1, '12.480')] [2023-10-11 19:17:56,045][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000004960_5079040.pth... [2023-10-11 19:17:56,045][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000004960_5079040.pth... [2023-10-11 19:17:56,082][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000003264_3342336.pth [2023-10-11 19:17:56,086][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000003264_3342336.pth [2023-10-11 19:17:57,836][71635] Updated weights for policy 1, policy_version 4962 (0.0009) [2023-10-11 19:17:57,873][71601] Updated weights for policy 0, policy_version 4970 (0.0008) [2023-10-11 19:17:58,202][71635] Updated weights for policy 1, policy_version 4972 (0.0008) [2023-10-11 19:17:58,249][71601] Updated weights for policy 0, policy_version 4980 (0.0008) [2023-10-11 19:17:58,571][71635] Updated weights for policy 1, policy_version 4982 (0.0007) [2023-10-11 19:17:58,622][71601] Updated weights for policy 0, policy_version 4990 (0.0007) [2023-10-11 19:17:58,930][71635] Updated weights for policy 1, policy_version 4992 (0.0008) [2023-10-11 19:18:01,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 10223616. Throughput: 0: 1808.4, 1: 1807.9. Samples: 2559676. Policy #0 lag: (min: 28.0, avg: 39.6, max: 40.0) [2023-10-11 19:18:01,034][70582] Avg episode reward: [(0, '12.370'), (1, '12.580')] [2023-10-11 19:18:02,276][71601] Updated weights for policy 0, policy_version 5000 (0.0007) [2023-10-11 19:18:02,650][71601] Updated weights for policy 0, policy_version 5010 (0.0007) [2023-10-11 19:18:02,698][71635] Updated weights for policy 1, policy_version 5002 (0.0007) [2023-10-11 19:18:03,018][71601] Updated weights for policy 0, policy_version 5020 (0.0008) [2023-10-11 19:18:03,062][71635] Updated weights for policy 1, policy_version 5012 (0.0008) [2023-10-11 19:18:03,435][71635] Updated weights for policy 1, policy_version 5022 (0.0007) [2023-10-11 19:18:06,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 10289152. Throughput: 0: 1794.3, 1: 1805.4. Samples: 2581062. Policy #0 lag: (min: 28.0, avg: 39.6, max: 40.0) [2023-10-11 19:18:06,035][70582] Avg episode reward: [(0, '12.670'), (1, '11.180')] [2023-10-11 19:18:06,639][71601] Updated weights for policy 0, policy_version 5030 (0.0010) [2023-10-11 19:18:07,021][71601] Updated weights for policy 0, policy_version 5040 (0.0008) [2023-10-11 19:18:07,271][71635] Updated weights for policy 1, policy_version 5032 (0.0009) [2023-10-11 19:18:07,390][71601] Updated weights for policy 0, policy_version 5050 (0.0008) [2023-10-11 19:18:07,643][71635] Updated weights for policy 1, policy_version 5042 (0.0008) [2023-10-11 19:18:08,006][71635] Updated weights for policy 1, policy_version 5052 (0.0007) [2023-10-11 19:18:11,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 10354688. Throughput: 0: 1799.9, 1: 1799.6. Samples: 2603596. Policy #0 lag: (min: 27.0, avg: 34.9, max: 59.0) [2023-10-11 19:18:11,035][70582] Avg episode reward: [(0, '12.240'), (1, '10.960')] [2023-10-11 19:18:11,078][71601] Updated weights for policy 0, policy_version 5060 (0.0008) [2023-10-11 19:18:11,449][71601] Updated weights for policy 0, policy_version 5070 (0.0008) [2023-10-11 19:18:11,745][71635] Updated weights for policy 1, policy_version 5062 (0.0009) [2023-10-11 19:18:11,820][71601] Updated weights for policy 0, policy_version 5080 (0.0009) [2023-10-11 19:18:12,117][71635] Updated weights for policy 1, policy_version 5072 (0.0007) [2023-10-11 19:18:12,486][71635] Updated weights for policy 1, policy_version 5082 (0.0010) [2023-10-11 19:18:15,568][71601] Updated weights for policy 0, policy_version 5090 (0.0008) [2023-10-11 19:18:15,978][71601] Updated weights for policy 0, policy_version 5100 (0.0008) [2023-10-11 19:18:16,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 10420224. Throughput: 0: 1797.3, 1: 1795.1. Samples: 2613394. Policy #0 lag: (min: 27.0, avg: 34.9, max: 59.0) [2023-10-11 19:18:16,035][70582] Avg episode reward: [(0, '13.300'), (1, '10.910')] [2023-10-11 19:18:16,336][71635] Updated weights for policy 1, policy_version 5092 (0.0008) [2023-10-11 19:18:16,358][71601] Updated weights for policy 0, policy_version 5110 (0.0009) [2023-10-11 19:18:16,696][71635] Updated weights for policy 1, policy_version 5102 (0.0009) [2023-10-11 19:18:16,730][71601] Updated weights for policy 0, policy_version 5120 (0.0007) [2023-10-11 19:18:17,066][71635] Updated weights for policy 1, policy_version 5112 (0.0008) [2023-10-11 19:18:20,308][71601] Updated weights for policy 0, policy_version 5130 (0.0009) [2023-10-11 19:18:20,688][71601] Updated weights for policy 0, policy_version 5140 (0.0008) [2023-10-11 19:18:20,865][71635] Updated weights for policy 1, policy_version 5122 (0.0009) [2023-10-11 19:18:21,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 10485760. Throughput: 0: 1797.5, 1: 1792.9. Samples: 2635828. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-10-11 19:18:21,035][70582] Avg episode reward: [(0, '13.130'), (1, '11.140')] [2023-10-11 19:18:21,061][71601] Updated weights for policy 0, policy_version 5150 (0.0007) [2023-10-11 19:18:21,235][71635] Updated weights for policy 1, policy_version 5132 (0.0009) [2023-10-11 19:18:21,612][71635] Updated weights for policy 1, policy_version 5142 (0.0011) [2023-10-11 19:18:21,975][71635] Updated weights for policy 1, policy_version 5152 (0.0007) [2023-10-11 19:18:24,662][71601] Updated weights for policy 0, policy_version 5160 (0.0009) [2023-10-11 19:18:25,037][71601] Updated weights for policy 0, policy_version 5170 (0.0008) [2023-10-11 19:18:25,423][71601] Updated weights for policy 0, policy_version 5180 (0.0008) [2023-10-11 19:18:25,707][71635] Updated weights for policy 1, policy_version 5162 (0.0007) [2023-10-11 19:18:26,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 10584064. Throughput: 0: 1807.7, 1: 1807.9. Samples: 2657298. Policy #0 lag: (min: 26.0, avg: 27.5, max: 47.0) [2023-10-11 19:18:26,035][70582] Avg episode reward: [(0, '12.560'), (1, '13.130')] [2023-10-11 19:18:26,076][71635] Updated weights for policy 1, policy_version 5172 (0.0007) [2023-10-11 19:18:26,452][71635] Updated weights for policy 1, policy_version 5182 (0.0008) [2023-10-11 19:18:29,185][71601] Updated weights for policy 0, policy_version 5190 (0.0010) [2023-10-11 19:18:29,554][71601] Updated weights for policy 0, policy_version 5200 (0.0010) [2023-10-11 19:18:29,916][71601] Updated weights for policy 0, policy_version 5210 (0.0008) [2023-10-11 19:18:30,218][71635] Updated weights for policy 1, policy_version 5192 (0.0011) [2023-10-11 19:18:30,595][71635] Updated weights for policy 1, policy_version 5202 (0.0008) [2023-10-11 19:18:30,962][71635] Updated weights for policy 1, policy_version 5212 (0.0007) [2023-10-11 19:18:31,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 10649600. Throughput: 0: 1803.7, 1: 1790.7. Samples: 2668598. Policy #0 lag: (min: 26.0, avg: 27.5, max: 47.0) [2023-10-11 19:18:31,034][70582] Avg episode reward: [(0, '11.330'), (1, '12.890')] [2023-10-11 19:18:33,542][71601] Updated weights for policy 0, policy_version 5220 (0.0007) [2023-10-11 19:18:33,925][71601] Updated weights for policy 0, policy_version 5230 (0.0008) [2023-10-11 19:18:34,289][71601] Updated weights for policy 0, policy_version 5240 (0.0007) [2023-10-11 19:18:34,546][71635] Updated weights for policy 1, policy_version 5222 (0.0007) [2023-10-11 19:18:34,922][71635] Updated weights for policy 1, policy_version 5232 (0.0007) [2023-10-11 19:18:35,285][71635] Updated weights for policy 1, policy_version 5242 (0.0009) [2023-10-11 19:18:36,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 10747904. Throughput: 0: 1815.8, 1: 1806.2. Samples: 2690052. Policy #0 lag: (min: 1.0, avg: 10.4, max: 33.0) [2023-10-11 19:18:36,035][70582] Avg episode reward: [(0, '10.970'), (1, '13.600')] [2023-10-11 19:18:38,010][71601] Updated weights for policy 0, policy_version 5250 (0.0007) [2023-10-11 19:18:38,377][71601] Updated weights for policy 0, policy_version 5260 (0.0007) [2023-10-11 19:18:38,749][71601] Updated weights for policy 0, policy_version 5270 (0.0008) [2023-10-11 19:18:38,916][71635] Updated weights for policy 1, policy_version 5252 (0.0008) [2023-10-11 19:18:39,116][71601] Updated weights for policy 0, policy_version 5280 (0.0008) [2023-10-11 19:18:39,285][71635] Updated weights for policy 1, policy_version 5262 (0.0008) [2023-10-11 19:18:39,659][71635] Updated weights for policy 1, policy_version 5272 (0.0009) [2023-10-11 19:18:41,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 10813440. Throughput: 0: 1819.3, 1: 1793.3. Samples: 2711210. Policy #0 lag: (min: 1.0, avg: 10.4, max: 33.0) [2023-10-11 19:18:41,035][70582] Avg episode reward: [(0, '11.570'), (1, '12.040')] [2023-10-11 19:18:42,756][71601] Updated weights for policy 0, policy_version 5290 (0.0008) [2023-10-11 19:18:43,132][71601] Updated weights for policy 0, policy_version 5300 (0.0008) [2023-10-11 19:18:43,359][71635] Updated weights for policy 1, policy_version 5282 (0.0008) [2023-10-11 19:18:43,508][71601] Updated weights for policy 0, policy_version 5310 (0.0007) [2023-10-11 19:18:43,722][71635] Updated weights for policy 1, policy_version 5292 (0.0007) [2023-10-11 19:18:44,078][71635] Updated weights for policy 1, policy_version 5302 (0.0011) [2023-10-11 19:18:44,451][71635] Updated weights for policy 1, policy_version 5312 (0.0010) [2023-10-11 19:18:46,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 10878976. Throughput: 0: 1814.1, 1: 1807.2. Samples: 2722636. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 19:18:46,034][70582] Avg episode reward: [(0, '11.270'), (1, '10.810')] [2023-10-11 19:18:47,320][71601] Updated weights for policy 0, policy_version 5320 (0.0008) [2023-10-11 19:18:47,688][71601] Updated weights for policy 0, policy_version 5330 (0.0008) [2023-10-11 19:18:48,074][71601] Updated weights for policy 0, policy_version 5340 (0.0008) [2023-10-11 19:18:48,333][71635] Updated weights for policy 1, policy_version 5322 (0.0007) [2023-10-11 19:18:48,708][71635] Updated weights for policy 1, policy_version 5332 (0.0008) [2023-10-11 19:18:49,069][71635] Updated weights for policy 1, policy_version 5342 (0.0008) [2023-10-11 19:18:51,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 10944512. Throughput: 0: 1815.3, 1: 1795.7. Samples: 2743554. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 19:18:51,034][70582] Avg episode reward: [(0, '11.740'), (1, '10.370')] [2023-10-11 19:18:51,835][71601] Updated weights for policy 0, policy_version 5350 (0.0007) [2023-10-11 19:18:52,199][71601] Updated weights for policy 0, policy_version 5360 (0.0007) [2023-10-11 19:18:52,567][71601] Updated weights for policy 0, policy_version 5370 (0.0007) [2023-10-11 19:18:52,754][71635] Updated weights for policy 1, policy_version 5352 (0.0009) [2023-10-11 19:18:53,110][71635] Updated weights for policy 1, policy_version 5362 (0.0008) [2023-10-11 19:18:53,484][71635] Updated weights for policy 1, policy_version 5372 (0.0007) [2023-10-11 19:18:56,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 11010048. Throughput: 0: 1809.9, 1: 1795.6. Samples: 2765844. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-11 19:18:56,035][70582] Avg episode reward: [(0, '11.730'), (1, '10.280')] [2023-10-11 19:18:56,307][71601] Updated weights for policy 0, policy_version 5380 (0.0009) [2023-10-11 19:18:56,682][71601] Updated weights for policy 0, policy_version 5390 (0.0007) [2023-10-11 19:18:57,068][71601] Updated weights for policy 0, policy_version 5400 (0.0007) [2023-10-11 19:18:57,176][71635] Updated weights for policy 1, policy_version 5382 (0.0009) [2023-10-11 19:18:57,533][71635] Updated weights for policy 1, policy_version 5392 (0.0008) [2023-10-11 19:18:57,905][71635] Updated weights for policy 1, policy_version 5402 (0.0009) [2023-10-11 19:19:00,607][71601] Updated weights for policy 0, policy_version 5410 (0.0008) [2023-10-11 19:19:01,022][71601] Updated weights for policy 0, policy_version 5420 (0.0008) [2023-10-11 19:19:01,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 11075584. Throughput: 0: 1812.5, 1: 1797.6. Samples: 2775848. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-11 19:19:01,034][70582] Avg episode reward: [(0, '10.960'), (1, '10.710')] [2023-10-11 19:19:01,397][71601] Updated weights for policy 0, policy_version 5430 (0.0011) [2023-10-11 19:19:01,673][71635] Updated weights for policy 1, policy_version 5412 (0.0010) [2023-10-11 19:19:01,768][71601] Updated weights for policy 0, policy_version 5440 (0.0008) [2023-10-11 19:19:02,040][71635] Updated weights for policy 1, policy_version 5422 (0.0010) [2023-10-11 19:19:02,401][71635] Updated weights for policy 1, policy_version 5432 (0.0010) [2023-10-11 19:19:05,534][71601] Updated weights for policy 0, policy_version 5450 (0.0008) [2023-10-11 19:19:05,903][71601] Updated weights for policy 0, policy_version 5460 (0.0008) [2023-10-11 19:19:06,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 11141120. Throughput: 0: 1814.8, 1: 1802.0. Samples: 2798582. Policy #0 lag: (min: 2.0, avg: 3.8, max: 29.0) [2023-10-11 19:19:06,035][70582] Avg episode reward: [(0, '12.140'), (1, '11.690')] [2023-10-11 19:19:06,094][71635] Updated weights for policy 1, policy_version 5442 (0.0010) [2023-10-11 19:19:06,275][71601] Updated weights for policy 0, policy_version 5470 (0.0008) [2023-10-11 19:19:06,459][71635] Updated weights for policy 1, policy_version 5452 (0.0007) [2023-10-11 19:19:06,824][71635] Updated weights for policy 1, policy_version 5462 (0.0007) [2023-10-11 19:19:07,188][71635] Updated weights for policy 1, policy_version 5472 (0.0007) [2023-10-11 19:19:10,093][71601] Updated weights for policy 0, policy_version 5480 (0.0007) [2023-10-11 19:19:10,472][71601] Updated weights for policy 0, policy_version 5490 (0.0007) [2023-10-11 19:19:10,836][71601] Updated weights for policy 0, policy_version 5500 (0.0008) [2023-10-11 19:19:10,943][71635] Updated weights for policy 1, policy_version 5482 (0.0007) [2023-10-11 19:19:11,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 11239424. Throughput: 0: 1822.4, 1: 1802.1. Samples: 2820404. Policy #0 lag: (min: 17.0, avg: 32.7, max: 49.0) [2023-10-11 19:19:11,035][70582] Avg episode reward: [(0, '13.140'), (1, '11.860')] [2023-10-11 19:19:11,311][71635] Updated weights for policy 1, policy_version 5492 (0.0007) [2023-10-11 19:19:11,682][71635] Updated weights for policy 1, policy_version 5502 (0.0008) [2023-10-11 19:19:14,444][71601] Updated weights for policy 0, policy_version 5510 (0.0010) [2023-10-11 19:19:14,810][71601] Updated weights for policy 0, policy_version 5520 (0.0009) [2023-10-11 19:19:15,186][71601] Updated weights for policy 0, policy_version 5530 (0.0008) [2023-10-11 19:19:15,566][71635] Updated weights for policy 1, policy_version 5512 (0.0008) [2023-10-11 19:19:15,946][71635] Updated weights for policy 1, policy_version 5522 (0.0009) [2023-10-11 19:19:16,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 11304960. Throughput: 0: 1811.5, 1: 1799.3. Samples: 2831084. Policy #0 lag: (min: 17.0, avg: 32.7, max: 49.0) [2023-10-11 19:19:16,034][70582] Avg episode reward: [(0, '12.750'), (1, '12.370')] [2023-10-11 19:19:16,317][71635] Updated weights for policy 1, policy_version 5532 (0.0009) [2023-10-11 19:19:18,694][71601] Updated weights for policy 0, policy_version 5540 (0.0008) [2023-10-11 19:19:19,063][71601] Updated weights for policy 0, policy_version 5550 (0.0009) [2023-10-11 19:19:19,433][71601] Updated weights for policy 0, policy_version 5560 (0.0010) [2023-10-11 19:19:20,019][71635] Updated weights for policy 1, policy_version 5542 (0.0009) [2023-10-11 19:19:20,393][71635] Updated weights for policy 1, policy_version 5552 (0.0011) [2023-10-11 19:19:20,764][71635] Updated weights for policy 1, policy_version 5562 (0.0010) [2023-10-11 19:19:21,034][70582] Fps is (10 sec: 16384.4, 60 sec: 15291.8, 300 sec: 14440.1). Total num frames: 11403264. Throughput: 0: 1807.9, 1: 1800.8. Samples: 2852444. Policy #0 lag: (min: 17.0, avg: 21.3, max: 45.0) [2023-10-11 19:19:21,034][70582] Avg episode reward: [(0, '13.010'), (1, '12.160')] [2023-10-11 19:19:23,167][71601] Updated weights for policy 0, policy_version 5570 (0.0010) [2023-10-11 19:19:23,546][71601] Updated weights for policy 0, policy_version 5580 (0.0008) [2023-10-11 19:19:23,915][71601] Updated weights for policy 0, policy_version 5590 (0.0008) [2023-10-11 19:19:24,287][71601] Updated weights for policy 0, policy_version 5600 (0.0009) [2023-10-11 19:19:24,493][71635] Updated weights for policy 1, policy_version 5572 (0.0010) [2023-10-11 19:19:24,858][71635] Updated weights for policy 1, policy_version 5582 (0.0008) [2023-10-11 19:19:25,215][71635] Updated weights for policy 1, policy_version 5592 (0.0009) [2023-10-11 19:19:26,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 11468800. Throughput: 0: 1805.6, 1: 1802.6. Samples: 2873580. Policy #0 lag: (min: 17.0, avg: 21.3, max: 45.0) [2023-10-11 19:19:26,034][70582] Avg episode reward: [(0, '11.850'), (1, '12.260')] [2023-10-11 19:19:28,044][71601] Updated weights for policy 0, policy_version 5610 (0.0008) [2023-10-11 19:19:28,409][71601] Updated weights for policy 0, policy_version 5620 (0.0011) [2023-10-11 19:19:28,778][71601] Updated weights for policy 0, policy_version 5630 (0.0008) [2023-10-11 19:19:28,889][71635] Updated weights for policy 1, policy_version 5602 (0.0009) [2023-10-11 19:19:29,258][71635] Updated weights for policy 1, policy_version 5612 (0.0008) [2023-10-11 19:19:29,635][71635] Updated weights for policy 1, policy_version 5622 (0.0010) [2023-10-11 19:19:29,990][71635] Updated weights for policy 1, policy_version 5632 (0.0010) [2023-10-11 19:19:31,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 11534336. Throughput: 0: 1811.4, 1: 1801.6. Samples: 2885220. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-11 19:19:31,035][70582] Avg episode reward: [(0, '11.710'), (1, '10.710')] [2023-10-11 19:19:32,520][71601] Updated weights for policy 0, policy_version 5640 (0.0010) [2023-10-11 19:19:32,909][71601] Updated weights for policy 0, policy_version 5650 (0.0008) [2023-10-11 19:19:33,283][71601] Updated weights for policy 0, policy_version 5660 (0.0009) [2023-10-11 19:19:33,536][71635] Updated weights for policy 1, policy_version 5642 (0.0007) [2023-10-11 19:19:33,901][71635] Updated weights for policy 1, policy_version 5652 (0.0007) [2023-10-11 19:19:34,270][71635] Updated weights for policy 1, policy_version 5662 (0.0008) [2023-10-11 19:19:36,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 11599872. Throughput: 0: 1808.8, 1: 1803.3. Samples: 2906100. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-11 19:19:36,035][70582] Avg episode reward: [(0, '11.930'), (1, '11.280')] [2023-10-11 19:19:36,966][71601] Updated weights for policy 0, policy_version 5670 (0.0008) [2023-10-11 19:19:37,337][71601] Updated weights for policy 0, policy_version 5680 (0.0008) [2023-10-11 19:19:37,722][71601] Updated weights for policy 0, policy_version 5690 (0.0010) [2023-10-11 19:19:37,865][71635] Updated weights for policy 1, policy_version 5672 (0.0009) [2023-10-11 19:19:38,239][71635] Updated weights for policy 1, policy_version 5682 (0.0010) [2023-10-11 19:19:38,598][71635] Updated weights for policy 1, policy_version 5692 (0.0008) [2023-10-11 19:19:41,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 11665408. Throughput: 0: 1815.9, 1: 1810.0. Samples: 2929006. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-11 19:19:41,034][70582] Avg episode reward: [(0, '11.760'), (1, '11.070')] [2023-10-11 19:19:41,396][71601] Updated weights for policy 0, policy_version 5700 (0.0008) [2023-10-11 19:19:41,769][71601] Updated weights for policy 0, policy_version 5710 (0.0008) [2023-10-11 19:19:42,143][71601] Updated weights for policy 0, policy_version 5720 (0.0008) [2023-10-11 19:19:42,265][71635] Updated weights for policy 1, policy_version 5702 (0.0008) [2023-10-11 19:19:42,623][71635] Updated weights for policy 1, policy_version 5712 (0.0010) [2023-10-11 19:19:42,997][71635] Updated weights for policy 1, policy_version 5722 (0.0008) [2023-10-11 19:19:45,828][71601] Updated weights for policy 0, policy_version 5730 (0.0008) [2023-10-11 19:19:46,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 11730944. Throughput: 0: 1812.2, 1: 1812.7. Samples: 2938968. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-11 19:19:46,035][70582] Avg episode reward: [(0, '11.820'), (1, '11.100')] [2023-10-11 19:19:46,230][71601] Updated weights for policy 0, policy_version 5740 (0.0007) [2023-10-11 19:19:46,607][71601] Updated weights for policy 0, policy_version 5750 (0.0007) [2023-10-11 19:19:46,754][71635] Updated weights for policy 1, policy_version 5732 (0.0009) [2023-10-11 19:19:46,981][71601] Updated weights for policy 0, policy_version 5760 (0.0008) [2023-10-11 19:19:47,125][71635] Updated weights for policy 1, policy_version 5742 (0.0007) [2023-10-11 19:19:47,495][71635] Updated weights for policy 1, policy_version 5752 (0.0010) [2023-10-11 19:19:50,571][71601] Updated weights for policy 0, policy_version 5770 (0.0009) [2023-10-11 19:19:50,956][71601] Updated weights for policy 0, policy_version 5780 (0.0010) [2023-10-11 19:19:51,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 11796480. Throughput: 0: 1813.3, 1: 1810.5. Samples: 2961652. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:19:51,034][70582] Avg episode reward: [(0, '11.100'), (1, '11.940')] [2023-10-11 19:19:51,229][71635] Updated weights for policy 1, policy_version 5762 (0.0010) [2023-10-11 19:19:51,328][71601] Updated weights for policy 0, policy_version 5790 (0.0008) [2023-10-11 19:19:51,597][71635] Updated weights for policy 1, policy_version 5772 (0.0009) [2023-10-11 19:19:51,967][71635] Updated weights for policy 1, policy_version 5782 (0.0009) [2023-10-11 19:19:52,339][71635] Updated weights for policy 1, policy_version 5792 (0.0009) [2023-10-11 19:19:54,976][71601] Updated weights for policy 0, policy_version 5800 (0.0009) [2023-10-11 19:19:55,351][71601] Updated weights for policy 0, policy_version 5810 (0.0007) [2023-10-11 19:19:55,732][71601] Updated weights for policy 0, policy_version 5820 (0.0008) [2023-10-11 19:19:56,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 11894784. Throughput: 0: 1811.1, 1: 1808.8. Samples: 2983302. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:19:56,035][70582] Avg episode reward: [(0, '12.170'), (1, '11.960')] [2023-10-11 19:19:56,045][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000005824_5963776.pth... [2023-10-11 19:19:56,078][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000004128_4227072.pth [2023-10-11 19:19:56,170][71635] Updated weights for policy 1, policy_version 5802 (0.0008) [2023-10-11 19:19:56,542][71635] Updated weights for policy 1, policy_version 5812 (0.0008) [2023-10-11 19:19:56,912][71635] Updated weights for policy 1, policy_version 5822 (0.0008) [2023-10-11 19:19:56,985][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000005824_5963776.pth... [2023-10-11 19:19:57,014][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000004128_4227072.pth [2023-10-11 19:19:59,427][71601] Updated weights for policy 0, policy_version 5830 (0.0007) [2023-10-11 19:19:59,799][71601] Updated weights for policy 0, policy_version 5840 (0.0007) [2023-10-11 19:20:00,179][71601] Updated weights for policy 0, policy_version 5850 (0.0008) [2023-10-11 19:20:00,742][71635] Updated weights for policy 1, policy_version 5832 (0.0007) [2023-10-11 19:20:01,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 11960320. Throughput: 0: 1808.7, 1: 1809.6. Samples: 2993904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:20:01,035][70582] Avg episode reward: [(0, '13.130'), (1, '12.190')] [2023-10-11 19:20:01,114][71635] Updated weights for policy 1, policy_version 5842 (0.0009) [2023-10-11 19:20:01,486][71635] Updated weights for policy 1, policy_version 5852 (0.0007) [2023-10-11 19:20:03,972][71601] Updated weights for policy 0, policy_version 5860 (0.0008) [2023-10-11 19:20:04,350][71601] Updated weights for policy 0, policy_version 5870 (0.0008) [2023-10-11 19:20:04,719][71601] Updated weights for policy 0, policy_version 5880 (0.0008) [2023-10-11 19:20:05,161][71635] Updated weights for policy 1, policy_version 5862 (0.0009) [2023-10-11 19:20:05,525][71635] Updated weights for policy 1, policy_version 5872 (0.0010) [2023-10-11 19:20:05,894][71635] Updated weights for policy 1, policy_version 5882 (0.0010) [2023-10-11 19:20:06,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 12025856. Throughput: 0: 1819.4, 1: 1810.4. Samples: 3015782. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:20:06,034][70582] Avg episode reward: [(0, '14.120'), (1, '12.880')] [2023-10-11 19:20:08,466][71601] Updated weights for policy 0, policy_version 5890 (0.0008) [2023-10-11 19:20:08,853][71601] Updated weights for policy 0, policy_version 5900 (0.0009) [2023-10-11 19:20:09,221][71601] Updated weights for policy 0, policy_version 5910 (0.0009) [2023-10-11 19:20:09,596][71601] Updated weights for policy 0, policy_version 5920 (0.0008) [2023-10-11 19:20:09,628][71635] Updated weights for policy 1, policy_version 5892 (0.0010) [2023-10-11 19:20:10,005][71635] Updated weights for policy 1, policy_version 5902 (0.0009) [2023-10-11 19:20:10,366][71635] Updated weights for policy 1, policy_version 5912 (0.0009) [2023-10-11 19:20:11,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 12124160. Throughput: 0: 1805.7, 1: 1817.0. Samples: 3036604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:20:11,035][70582] Avg episode reward: [(0, '13.630'), (1, '13.780')] [2023-10-11 19:20:13,269][71601] Updated weights for policy 0, policy_version 5930 (0.0007) [2023-10-11 19:20:13,638][71601] Updated weights for policy 0, policy_version 5940 (0.0007) [2023-10-11 19:20:13,957][71635] Updated weights for policy 1, policy_version 5922 (0.0009) [2023-10-11 19:20:14,014][71601] Updated weights for policy 0, policy_version 5950 (0.0009) [2023-10-11 19:20:14,315][71635] Updated weights for policy 1, policy_version 5932 (0.0008) [2023-10-11 19:20:14,683][71635] Updated weights for policy 1, policy_version 5942 (0.0008) [2023-10-11 19:20:15,060][71635] Updated weights for policy 1, policy_version 5952 (0.0010) [2023-10-11 19:20:16,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 12189696. Throughput: 0: 1819.0, 1: 1817.0. Samples: 3048838. Policy #0 lag: (min: 17.0, avg: 32.1, max: 49.0) [2023-10-11 19:20:16,034][70582] Avg episode reward: [(0, '13.520'), (1, '14.140')] [2023-10-11 19:20:16,035][71431] Saving new best policy, reward=14.140! [2023-10-11 19:20:17,607][71601] Updated weights for policy 0, policy_version 5960 (0.0008) [2023-10-11 19:20:17,980][71601] Updated weights for policy 0, policy_version 5970 (0.0010) [2023-10-11 19:20:18,365][71601] Updated weights for policy 0, policy_version 5980 (0.0009) [2023-10-11 19:20:18,803][71635] Updated weights for policy 1, policy_version 5962 (0.0008) [2023-10-11 19:20:19,178][71635] Updated weights for policy 1, policy_version 5972 (0.0008) [2023-10-11 19:20:19,551][71635] Updated weights for policy 1, policy_version 5982 (0.0008) [2023-10-11 19:20:21,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 12255232. Throughput: 0: 1818.4, 1: 1823.6. Samples: 3069986. Policy #0 lag: (min: 17.0, avg: 32.1, max: 49.0) [2023-10-11 19:20:21,035][70582] Avg episode reward: [(0, '12.370'), (1, '14.190')] [2023-10-11 19:20:21,036][71431] Saving new best policy, reward=14.190! [2023-10-11 19:20:22,114][71601] Updated weights for policy 0, policy_version 5990 (0.0008) [2023-10-11 19:20:22,495][71601] Updated weights for policy 0, policy_version 6000 (0.0009) [2023-10-11 19:20:22,867][71601] Updated weights for policy 0, policy_version 6010 (0.0008) [2023-10-11 19:20:23,317][71635] Updated weights for policy 1, policy_version 5992 (0.0008) [2023-10-11 19:20:23,683][71635] Updated weights for policy 1, policy_version 6002 (0.0007) [2023-10-11 19:20:24,051][71635] Updated weights for policy 1, policy_version 6012 (0.0009) [2023-10-11 19:20:26,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 12320768. Throughput: 0: 1813.2, 1: 1814.4. Samples: 3092248. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 19:20:26,034][70582] Avg episode reward: [(0, '11.340'), (1, '12.510')] [2023-10-11 19:20:26,458][71601] Updated weights for policy 0, policy_version 6020 (0.0008) [2023-10-11 19:20:26,832][71601] Updated weights for policy 0, policy_version 6030 (0.0007) [2023-10-11 19:20:27,208][71601] Updated weights for policy 0, policy_version 6040 (0.0007) [2023-10-11 19:20:27,783][71635] Updated weights for policy 1, policy_version 6022 (0.0009) [2023-10-11 19:20:28,157][71635] Updated weights for policy 1, policy_version 6032 (0.0009) [2023-10-11 19:20:28,525][71635] Updated weights for policy 1, policy_version 6042 (0.0007) [2023-10-11 19:20:31,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 12386304. Throughput: 0: 1815.7, 1: 1824.4. Samples: 3102772. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 19:20:31,034][70582] Avg episode reward: [(0, '12.340'), (1, '12.610')] [2023-10-11 19:20:31,130][71601] Updated weights for policy 0, policy_version 6050 (0.0008) [2023-10-11 19:20:31,503][71601] Updated weights for policy 0, policy_version 6060 (0.0007) [2023-10-11 19:20:31,889][71601] Updated weights for policy 0, policy_version 6070 (0.0008) [2023-10-11 19:20:32,255][71601] Updated weights for policy 0, policy_version 6080 (0.0008) [2023-10-11 19:20:32,256][71635] Updated weights for policy 1, policy_version 6052 (0.0008) [2023-10-11 19:20:32,622][71635] Updated weights for policy 1, policy_version 6062 (0.0009) [2023-10-11 19:20:32,994][71635] Updated weights for policy 1, policy_version 6072 (0.0008) [2023-10-11 19:20:35,792][71601] Updated weights for policy 0, policy_version 6090 (0.0008) [2023-10-11 19:20:36,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 12451840. Throughput: 0: 1813.0, 1: 1813.2. Samples: 3124828. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-11 19:20:36,034][70582] Avg episode reward: [(0, '13.840'), (1, '12.840')] [2023-10-11 19:20:36,165][71601] Updated weights for policy 0, policy_version 6100 (0.0008) [2023-10-11 19:20:36,545][71601] Updated weights for policy 0, policy_version 6110 (0.0009) [2023-10-11 19:20:36,593][71635] Updated weights for policy 1, policy_version 6082 (0.0009) [2023-10-11 19:20:36,960][71635] Updated weights for policy 1, policy_version 6092 (0.0008) [2023-10-11 19:20:37,328][71635] Updated weights for policy 1, policy_version 6102 (0.0007) [2023-10-11 19:20:37,685][71635] Updated weights for policy 1, policy_version 6112 (0.0008) [2023-10-11 19:20:40,216][71601] Updated weights for policy 0, policy_version 6120 (0.0010) [2023-10-11 19:20:40,585][71601] Updated weights for policy 0, policy_version 6130 (0.0007) [2023-10-11 19:20:40,963][71601] Updated weights for policy 0, policy_version 6140 (0.0007) [2023-10-11 19:20:41,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 12517376. Throughput: 0: 1815.7, 1: 1815.8. Samples: 3146718. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-11 19:20:41,035][70582] Avg episode reward: [(0, '13.240'), (1, '11.460')] [2023-10-11 19:20:41,437][71635] Updated weights for policy 1, policy_version 6122 (0.0007) [2023-10-11 19:20:41,811][71635] Updated weights for policy 1, policy_version 6132 (0.0008) [2023-10-11 19:20:42,174][71635] Updated weights for policy 1, policy_version 6142 (0.0008) [2023-10-11 19:20:44,653][71601] Updated weights for policy 0, policy_version 6150 (0.0009) [2023-10-11 19:20:45,022][71601] Updated weights for policy 0, policy_version 6160 (0.0010) [2023-10-11 19:20:45,400][71601] Updated weights for policy 0, policy_version 6170 (0.0007) [2023-10-11 19:20:45,856][71635] Updated weights for policy 1, policy_version 6152 (0.0009) [2023-10-11 19:20:46,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 12615680. Throughput: 0: 1814.2, 1: 1817.4. Samples: 3157326. Policy #0 lag: (min: 10.0, avg: 10.2, max: 20.0) [2023-10-11 19:20:46,034][70582] Avg episode reward: [(0, '14.530'), (1, '12.980')] [2023-10-11 19:20:46,222][71635] Updated weights for policy 1, policy_version 6162 (0.0007) [2023-10-11 19:20:46,585][71635] Updated weights for policy 1, policy_version 6172 (0.0009) [2023-10-11 19:20:48,902][71601] Updated weights for policy 0, policy_version 6180 (0.0007) [2023-10-11 19:20:49,272][71601] Updated weights for policy 0, policy_version 6190 (0.0008) [2023-10-11 19:20:49,659][71601] Updated weights for policy 0, policy_version 6200 (0.0010) [2023-10-11 19:20:50,247][71635] Updated weights for policy 1, policy_version 6182 (0.0008) [2023-10-11 19:20:50,618][71635] Updated weights for policy 1, policy_version 6192 (0.0009) [2023-10-11 19:20:50,981][71635] Updated weights for policy 1, policy_version 6202 (0.0009) [2023-10-11 19:20:51,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 12681216. Throughput: 0: 1815.5, 1: 1818.2. Samples: 3179300. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-11 19:20:51,034][70582] Avg episode reward: [(0, '14.280'), (1, '13.010')] [2023-10-11 19:20:53,375][71601] Updated weights for policy 0, policy_version 6210 (0.0010) [2023-10-11 19:20:53,745][71601] Updated weights for policy 0, policy_version 6220 (0.0008) [2023-10-11 19:20:54,123][71601] Updated weights for policy 0, policy_version 6230 (0.0010) [2023-10-11 19:20:54,492][71601] Updated weights for policy 0, policy_version 6240 (0.0008) [2023-10-11 19:20:54,717][71635] Updated weights for policy 1, policy_version 6212 (0.0007) [2023-10-11 19:20:55,081][71635] Updated weights for policy 1, policy_version 6222 (0.0007) [2023-10-11 19:20:55,452][71635] Updated weights for policy 1, policy_version 6232 (0.0008) [2023-10-11 19:20:56,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 12779520. Throughput: 0: 1822.4, 1: 1817.1. Samples: 3200382. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-11 19:20:56,034][70582] Avg episode reward: [(0, '12.370'), (1, '13.510')] [2023-10-11 19:20:58,314][71601] Updated weights for policy 0, policy_version 6250 (0.0008) [2023-10-11 19:20:58,690][71601] Updated weights for policy 0, policy_version 6260 (0.0009) [2023-10-11 19:20:59,059][71601] Updated weights for policy 0, policy_version 6270 (0.0008) [2023-10-11 19:20:59,154][71635] Updated weights for policy 1, policy_version 6242 (0.0007) [2023-10-11 19:20:59,521][71635] Updated weights for policy 1, policy_version 6252 (0.0010) [2023-10-11 19:20:59,895][71635] Updated weights for policy 1, policy_version 6262 (0.0008) [2023-10-11 19:21:00,266][71635] Updated weights for policy 1, policy_version 6272 (0.0010) [2023-10-11 19:21:01,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 12845056. Throughput: 0: 1818.5, 1: 1810.0. Samples: 3212122. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:21:01,035][70582] Avg episode reward: [(0, '12.590'), (1, '14.380')] [2023-10-11 19:21:01,036][71431] Saving new best policy, reward=14.380! [2023-10-11 19:21:02,808][71601] Updated weights for policy 0, policy_version 6280 (0.0008) [2023-10-11 19:21:03,189][71601] Updated weights for policy 0, policy_version 6290 (0.0009) [2023-10-11 19:21:03,552][71601] Updated weights for policy 0, policy_version 6300 (0.0008) [2023-10-11 19:21:03,887][71635] Updated weights for policy 1, policy_version 6282 (0.0007) [2023-10-11 19:21:04,258][71635] Updated weights for policy 1, policy_version 6292 (0.0007) [2023-10-11 19:21:04,626][71635] Updated weights for policy 1, policy_version 6302 (0.0008) [2023-10-11 19:21:06,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 12910592. Throughput: 0: 1816.0, 1: 1811.2. Samples: 3233208. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:21:06,034][70582] Avg episode reward: [(0, '11.170'), (1, '16.260')] [2023-10-11 19:21:06,035][71431] Saving new best policy, reward=16.260! [2023-10-11 19:21:07,090][71601] Updated weights for policy 0, policy_version 6310 (0.0008) [2023-10-11 19:21:07,459][71601] Updated weights for policy 0, policy_version 6320 (0.0007) [2023-10-11 19:21:07,838][71601] Updated weights for policy 0, policy_version 6330 (0.0007) [2023-10-11 19:21:08,432][71635] Updated weights for policy 1, policy_version 6312 (0.0008) [2023-10-11 19:21:08,796][71635] Updated weights for policy 1, policy_version 6322 (0.0008) [2023-10-11 19:21:09,170][71635] Updated weights for policy 1, policy_version 6332 (0.0010) [2023-10-11 19:21:11,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 12976128. Throughput: 0: 1827.0, 1: 1808.8. Samples: 3255856. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:21:11,034][70582] Avg episode reward: [(0, '12.120'), (1, '15.050')] [2023-10-11 19:21:11,431][71601] Updated weights for policy 0, policy_version 6340 (0.0008) [2023-10-11 19:21:11,798][71601] Updated weights for policy 0, policy_version 6350 (0.0007) [2023-10-11 19:21:12,177][71601] Updated weights for policy 0, policy_version 6360 (0.0008) [2023-10-11 19:21:12,901][71635] Updated weights for policy 1, policy_version 6342 (0.0008) [2023-10-11 19:21:13,278][71635] Updated weights for policy 1, policy_version 6352 (0.0008) [2023-10-11 19:21:13,642][71635] Updated weights for policy 1, policy_version 6362 (0.0009) [2023-10-11 19:21:15,901][71601] Updated weights for policy 0, policy_version 6370 (0.0008) [2023-10-11 19:21:16,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 13041664. Throughput: 0: 1823.1, 1: 1811.1. Samples: 3266312. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:21:16,035][70582] Avg episode reward: [(0, '12.750'), (1, '15.040')] [2023-10-11 19:21:16,307][71601] Updated weights for policy 0, policy_version 6380 (0.0009) [2023-10-11 19:21:16,682][71601] Updated weights for policy 0, policy_version 6390 (0.0009) [2023-10-11 19:21:17,064][71601] Updated weights for policy 0, policy_version 6400 (0.0008) [2023-10-11 19:21:17,267][71635] Updated weights for policy 1, policy_version 6372 (0.0008) [2023-10-11 19:21:17,633][71635] Updated weights for policy 1, policy_version 6382 (0.0007) [2023-10-11 19:21:17,999][71635] Updated weights for policy 1, policy_version 6392 (0.0008) [2023-10-11 19:21:20,701][71601] Updated weights for policy 0, policy_version 6410 (0.0009) [2023-10-11 19:21:21,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 13107200. Throughput: 0: 1825.7, 1: 1812.7. Samples: 3288556. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:21:21,035][70582] Avg episode reward: [(0, '13.480'), (1, '13.000')] [2023-10-11 19:21:21,082][71601] Updated weights for policy 0, policy_version 6420 (0.0008) [2023-10-11 19:21:21,447][71601] Updated weights for policy 0, policy_version 6430 (0.0008) [2023-10-11 19:21:21,662][71635] Updated weights for policy 1, policy_version 6402 (0.0007) [2023-10-11 19:21:22,033][71635] Updated weights for policy 1, policy_version 6412 (0.0007) [2023-10-11 19:21:22,400][71635] Updated weights for policy 1, policy_version 6422 (0.0009) [2023-10-11 19:21:22,767][71635] Updated weights for policy 1, policy_version 6432 (0.0007) [2023-10-11 19:21:25,164][71601] Updated weights for policy 0, policy_version 6440 (0.0009) [2023-10-11 19:21:25,529][71601] Updated weights for policy 0, policy_version 6450 (0.0007) [2023-10-11 19:21:25,906][71601] Updated weights for policy 0, policy_version 6460 (0.0007) [2023-10-11 19:21:26,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 13172736. Throughput: 0: 1827.8, 1: 1817.8. Samples: 3310772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:21:26,034][70582] Avg episode reward: [(0, '13.920'), (1, '11.450')] [2023-10-11 19:21:26,385][71635] Updated weights for policy 1, policy_version 6442 (0.0007) [2023-10-11 19:21:26,748][71635] Updated weights for policy 1, policy_version 6452 (0.0008) [2023-10-11 19:21:27,111][71635] Updated weights for policy 1, policy_version 6462 (0.0009) [2023-10-11 19:21:29,568][71601] Updated weights for policy 0, policy_version 6470 (0.0007) [2023-10-11 19:21:29,943][71601] Updated weights for policy 0, policy_version 6480 (0.0009) [2023-10-11 19:21:30,317][71601] Updated weights for policy 0, policy_version 6490 (0.0009) [2023-10-11 19:21:30,895][71635] Updated weights for policy 1, policy_version 6472 (0.0007) [2023-10-11 19:21:31,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 13271040. Throughput: 0: 1831.3, 1: 1819.2. Samples: 3321598. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:21:31,034][70582] Avg episode reward: [(0, '14.450'), (1, '12.610')] [2023-10-11 19:21:31,269][71635] Updated weights for policy 1, policy_version 6482 (0.0007) [2023-10-11 19:21:31,637][71635] Updated weights for policy 1, policy_version 6492 (0.0008) [2023-10-11 19:21:33,807][71601] Updated weights for policy 0, policy_version 6500 (0.0009) [2023-10-11 19:21:34,170][71601] Updated weights for policy 0, policy_version 6510 (0.0008) [2023-10-11 19:21:34,541][71601] Updated weights for policy 0, policy_version 6520 (0.0008) [2023-10-11 19:21:35,278][71635] Updated weights for policy 1, policy_version 6502 (0.0007) [2023-10-11 19:21:35,642][71635] Updated weights for policy 1, policy_version 6512 (0.0007) [2023-10-11 19:21:36,019][71635] Updated weights for policy 1, policy_version 6522 (0.0008) [2023-10-11 19:21:36,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 13336576. Throughput: 0: 1828.8, 1: 1820.0. Samples: 3343498. Policy #0 lag: (min: 10.0, avg: 30.6, max: 32.0) [2023-10-11 19:21:36,034][70582] Avg episode reward: [(0, '14.220'), (1, '12.610')] [2023-10-11 19:21:38,353][71601] Updated weights for policy 0, policy_version 6530 (0.0007) [2023-10-11 19:21:38,722][71601] Updated weights for policy 0, policy_version 6540 (0.0007) [2023-10-11 19:21:39,102][71601] Updated weights for policy 0, policy_version 6550 (0.0008) [2023-10-11 19:21:39,469][71601] Updated weights for policy 0, policy_version 6560 (0.0010) [2023-10-11 19:21:39,793][71635] Updated weights for policy 1, policy_version 6532 (0.0009) [2023-10-11 19:21:40,168][71635] Updated weights for policy 1, policy_version 6542 (0.0009) [2023-10-11 19:21:40,534][71635] Updated weights for policy 1, policy_version 6552 (0.0009) [2023-10-11 19:21:41,034][70582] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 13434880. Throughput: 0: 1832.0, 1: 1824.9. Samples: 3364944. Policy #0 lag: (min: 10.0, avg: 30.6, max: 32.0) [2023-10-11 19:21:41,035][70582] Avg episode reward: [(0, '12.730'), (1, '12.780')] [2023-10-11 19:21:43,072][71601] Updated weights for policy 0, policy_version 6570 (0.0007) [2023-10-11 19:21:43,456][71601] Updated weights for policy 0, policy_version 6580 (0.0007) [2023-10-11 19:21:43,825][71601] Updated weights for policy 0, policy_version 6590 (0.0008) [2023-10-11 19:21:44,306][71635] Updated weights for policy 1, policy_version 6562 (0.0009) [2023-10-11 19:21:44,687][71635] Updated weights for policy 1, policy_version 6572 (0.0008) [2023-10-11 19:21:45,049][71635] Updated weights for policy 1, policy_version 6582 (0.0008) [2023-10-11 19:21:45,423][71635] Updated weights for policy 1, policy_version 6592 (0.0008) [2023-10-11 19:21:46,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 13500416. Throughput: 0: 1825.3, 1: 1818.9. Samples: 3376108. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:21:46,034][70582] Avg episode reward: [(0, '12.280'), (1, '13.100')] [2023-10-11 19:21:47,335][71601] Updated weights for policy 0, policy_version 6600 (0.0007) [2023-10-11 19:21:47,710][71601] Updated weights for policy 0, policy_version 6610 (0.0010) [2023-10-11 19:21:48,083][71601] Updated weights for policy 0, policy_version 6620 (0.0008) [2023-10-11 19:21:49,014][71635] Updated weights for policy 1, policy_version 6602 (0.0009) [2023-10-11 19:21:49,373][71635] Updated weights for policy 1, policy_version 6612 (0.0008) [2023-10-11 19:21:49,739][71635] Updated weights for policy 1, policy_version 6622 (0.0009) [2023-10-11 19:21:51,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 13565952. Throughput: 0: 1833.9, 1: 1819.7. Samples: 3397622. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:21:51,034][70582] Avg episode reward: [(0, '12.000'), (1, '12.900')] [2023-10-11 19:21:51,910][71601] Updated weights for policy 0, policy_version 6630 (0.0009) [2023-10-11 19:21:52,278][71601] Updated weights for policy 0, policy_version 6640 (0.0009) [2023-10-11 19:21:52,655][71601] Updated weights for policy 0, policy_version 6650 (0.0009) [2023-10-11 19:21:53,420][71635] Updated weights for policy 1, policy_version 6632 (0.0008) [2023-10-11 19:21:53,795][71635] Updated weights for policy 1, policy_version 6642 (0.0009) [2023-10-11 19:21:54,161][71635] Updated weights for policy 1, policy_version 6652 (0.0009) [2023-10-11 19:21:56,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 13631488. Throughput: 0: 1816.9, 1: 1815.6. Samples: 3419320. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:21:56,034][70582] Avg episode reward: [(0, '11.940'), (1, '13.910')] [2023-10-11 19:21:56,044][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000006656_6815744.pth... [2023-10-11 19:21:56,044][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000006656_6815744.pth... [2023-10-11 19:21:56,076][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000004960_5079040.pth [2023-10-11 19:21:56,076][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000004960_5079040.pth [2023-10-11 19:21:56,476][71601] Updated weights for policy 0, policy_version 6660 (0.0008) [2023-10-11 19:21:56,854][71601] Updated weights for policy 0, policy_version 6670 (0.0007) [2023-10-11 19:21:57,228][71601] Updated weights for policy 0, policy_version 6680 (0.0007) [2023-10-11 19:21:57,922][71635] Updated weights for policy 1, policy_version 6662 (0.0008) [2023-10-11 19:21:58,294][71635] Updated weights for policy 1, policy_version 6672 (0.0007) [2023-10-11 19:21:58,661][71635] Updated weights for policy 1, policy_version 6682 (0.0007) [2023-10-11 19:22:00,947][71601] Updated weights for policy 0, policy_version 6690 (0.0007) [2023-10-11 19:22:01,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 13697024. Throughput: 0: 1822.0, 1: 1816.5. Samples: 3430044. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:22:01,034][70582] Avg episode reward: [(0, '11.470'), (1, '15.300')] [2023-10-11 19:22:01,340][71601] Updated weights for policy 0, policy_version 6700 (0.0007) [2023-10-11 19:22:01,722][71601] Updated weights for policy 0, policy_version 6710 (0.0007) [2023-10-11 19:22:02,100][71601] Updated weights for policy 0, policy_version 6720 (0.0007) [2023-10-11 19:22:02,487][71635] Updated weights for policy 1, policy_version 6692 (0.0008) [2023-10-11 19:22:02,844][71635] Updated weights for policy 1, policy_version 6702 (0.0008) [2023-10-11 19:22:03,219][71635] Updated weights for policy 1, policy_version 6712 (0.0007) [2023-10-11 19:22:05,750][71601] Updated weights for policy 0, policy_version 6730 (0.0007) [2023-10-11 19:22:06,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 13762560. Throughput: 0: 1823.7, 1: 1808.3. Samples: 3451996. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:22:06,034][70582] Avg episode reward: [(0, '11.660'), (1, '15.600')] [2023-10-11 19:22:06,122][71601] Updated weights for policy 0, policy_version 6740 (0.0008) [2023-10-11 19:22:06,500][71601] Updated weights for policy 0, policy_version 6750 (0.0009) [2023-10-11 19:22:06,840][71635] Updated weights for policy 1, policy_version 6722 (0.0009) [2023-10-11 19:22:07,210][71635] Updated weights for policy 1, policy_version 6732 (0.0007) [2023-10-11 19:22:07,585][71635] Updated weights for policy 1, policy_version 6742 (0.0009) [2023-10-11 19:22:07,951][71635] Updated weights for policy 1, policy_version 6752 (0.0010) [2023-10-11 19:22:10,146][71601] Updated weights for policy 0, policy_version 6760 (0.0008) [2023-10-11 19:22:10,509][71601] Updated weights for policy 0, policy_version 6770 (0.0008) [2023-10-11 19:22:10,881][71601] Updated weights for policy 0, policy_version 6780 (0.0008) [2023-10-11 19:22:11,037][70582] Fps is (10 sec: 16377.8, 60 sec: 14744.7, 300 sec: 14551.0). Total num frames: 13860864. Throughput: 0: 1820.6, 1: 1811.1. Samples: 3474212. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-11 19:22:11,040][70582] Avg episode reward: [(0, '10.600'), (1, '14.710')] [2023-10-11 19:22:11,589][71635] Updated weights for policy 1, policy_version 6762 (0.0008) [2023-10-11 19:22:11,960][71635] Updated weights for policy 1, policy_version 6772 (0.0009) [2023-10-11 19:22:12,325][71635] Updated weights for policy 1, policy_version 6782 (0.0009) [2023-10-11 19:22:14,426][71601] Updated weights for policy 0, policy_version 6790 (0.0009) [2023-10-11 19:22:14,807][71601] Updated weights for policy 0, policy_version 6800 (0.0010) [2023-10-11 19:22:15,191][71601] Updated weights for policy 0, policy_version 6810 (0.0008) [2023-10-11 19:22:16,027][71635] Updated weights for policy 1, policy_version 6792 (0.0007) [2023-10-11 19:22:16,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 13926400. Throughput: 0: 1820.8, 1: 1816.4. Samples: 3485272. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-11 19:22:16,035][70582] Avg episode reward: [(0, '11.570'), (1, '13.280')] [2023-10-11 19:22:16,401][71635] Updated weights for policy 1, policy_version 6802 (0.0007) [2023-10-11 19:22:16,763][71635] Updated weights for policy 1, policy_version 6812 (0.0008) [2023-10-11 19:22:18,962][71601] Updated weights for policy 0, policy_version 6820 (0.0009) [2023-10-11 19:22:19,331][71601] Updated weights for policy 0, policy_version 6830 (0.0007) [2023-10-11 19:22:19,710][71601] Updated weights for policy 0, policy_version 6840 (0.0008) [2023-10-11 19:22:20,487][71635] Updated weights for policy 1, policy_version 6822 (0.0009) [2023-10-11 19:22:20,858][71635] Updated weights for policy 1, policy_version 6832 (0.0007) [2023-10-11 19:22:21,034][70582] Fps is (10 sec: 13112.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 13991936. Throughput: 0: 1818.8, 1: 1818.3. Samples: 3507170. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-11 19:22:21,035][70582] Avg episode reward: [(0, '12.200'), (1, '12.100')] [2023-10-11 19:22:21,229][71635] Updated weights for policy 1, policy_version 6842 (0.0009) [2023-10-11 19:22:23,413][71601] Updated weights for policy 0, policy_version 6850 (0.0008) [2023-10-11 19:22:23,792][71601] Updated weights for policy 0, policy_version 6860 (0.0008) [2023-10-11 19:22:24,154][71601] Updated weights for policy 0, policy_version 6870 (0.0008) [2023-10-11 19:22:24,525][71601] Updated weights for policy 0, policy_version 6880 (0.0008) [2023-10-11 19:22:24,999][71635] Updated weights for policy 1, policy_version 6852 (0.0011) [2023-10-11 19:22:25,379][71635] Updated weights for policy 1, policy_version 6862 (0.0010) [2023-10-11 19:22:25,740][71635] Updated weights for policy 1, policy_version 6872 (0.0010) [2023-10-11 19:22:26,034][70582] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 14090240. Throughput: 0: 1815.0, 1: 1817.8. Samples: 3528420. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-11 19:22:26,034][70582] Avg episode reward: [(0, '12.410'), (1, '12.020')] [2023-10-11 19:22:28,227][71601] Updated weights for policy 0, policy_version 6890 (0.0007) [2023-10-11 19:22:28,608][71601] Updated weights for policy 0, policy_version 6900 (0.0008) [2023-10-11 19:22:28,980][71601] Updated weights for policy 0, policy_version 6910 (0.0009) [2023-10-11 19:22:29,468][71635] Updated weights for policy 1, policy_version 6882 (0.0010) [2023-10-11 19:22:29,839][71635] Updated weights for policy 1, policy_version 6892 (0.0009) [2023-10-11 19:22:30,204][71635] Updated weights for policy 1, policy_version 6902 (0.0009) [2023-10-11 19:22:30,573][71635] Updated weights for policy 1, policy_version 6912 (0.0008) [2023-10-11 19:22:31,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 14155776. Throughput: 0: 1819.7, 1: 1810.7. Samples: 3539476. Policy #0 lag: (min: 24.0, avg: 48.9, max: 56.0) [2023-10-11 19:22:31,034][70582] Avg episode reward: [(0, '11.960'), (1, '10.890')] [2023-10-11 19:22:32,608][71601] Updated weights for policy 0, policy_version 6920 (0.0009) [2023-10-11 19:22:32,975][71601] Updated weights for policy 0, policy_version 6930 (0.0007) [2023-10-11 19:22:33,356][71601] Updated weights for policy 0, policy_version 6940 (0.0009) [2023-10-11 19:22:34,376][71635] Updated weights for policy 1, policy_version 6922 (0.0007) [2023-10-11 19:22:34,743][71635] Updated weights for policy 1, policy_version 6932 (0.0009) [2023-10-11 19:22:35,112][71635] Updated weights for policy 1, policy_version 6942 (0.0009) [2023-10-11 19:22:36,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 14221312. Throughput: 0: 1813.4, 1: 1814.1. Samples: 3560860. Policy #0 lag: (min: 24.0, avg: 48.9, max: 56.0) [2023-10-11 19:22:36,035][70582] Avg episode reward: [(0, '11.500'), (1, '11.610')] [2023-10-11 19:22:37,028][71601] Updated weights for policy 0, policy_version 6950 (0.0009) [2023-10-11 19:22:37,398][71601] Updated weights for policy 0, policy_version 6960 (0.0008) [2023-10-11 19:22:37,765][71601] Updated weights for policy 0, policy_version 6970 (0.0009) [2023-10-11 19:22:38,842][71635] Updated weights for policy 1, policy_version 6952 (0.0008) [2023-10-11 19:22:39,204][71635] Updated weights for policy 1, policy_version 6962 (0.0010) [2023-10-11 19:22:39,561][71635] Updated weights for policy 1, policy_version 6972 (0.0010) [2023-10-11 19:22:41,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 14286848. Throughput: 0: 1828.9, 1: 1802.9. Samples: 3582748. Policy #0 lag: (min: 31.0, avg: 31.2, max: 39.0) [2023-10-11 19:22:41,034][70582] Avg episode reward: [(0, '10.470'), (1, '11.280')] [2023-10-11 19:22:41,481][71601] Updated weights for policy 0, policy_version 6980 (0.0008) [2023-10-11 19:22:41,865][71601] Updated weights for policy 0, policy_version 6990 (0.0009) [2023-10-11 19:22:42,234][71601] Updated weights for policy 0, policy_version 7000 (0.0010) [2023-10-11 19:22:43,266][71635] Updated weights for policy 1, policy_version 6982 (0.0010) [2023-10-11 19:22:43,646][71635] Updated weights for policy 1, policy_version 6992 (0.0008) [2023-10-11 19:22:44,016][71635] Updated weights for policy 1, policy_version 7002 (0.0010) [2023-10-11 19:22:46,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 14352384. Throughput: 0: 1826.2, 1: 1811.2. Samples: 3593726. Policy #0 lag: (min: 31.0, avg: 31.2, max: 39.0) [2023-10-11 19:22:46,034][70582] Avg episode reward: [(0, '10.840'), (1, '11.370')] [2023-10-11 19:22:46,069][71601] Updated weights for policy 0, policy_version 7010 (0.0009) [2023-10-11 19:22:46,442][71601] Updated weights for policy 0, policy_version 7020 (0.0008) [2023-10-11 19:22:46,808][71601] Updated weights for policy 0, policy_version 7030 (0.0008) [2023-10-11 19:22:47,184][71601] Updated weights for policy 0, policy_version 7040 (0.0008) [2023-10-11 19:22:47,706][71635] Updated weights for policy 1, policy_version 7012 (0.0010) [2023-10-11 19:22:48,081][71635] Updated weights for policy 1, policy_version 7022 (0.0009) [2023-10-11 19:22:48,439][71635] Updated weights for policy 1, policy_version 7032 (0.0008) [2023-10-11 19:22:50,720][71601] Updated weights for policy 0, policy_version 7050 (0.0010) [2023-10-11 19:22:51,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 14417920. Throughput: 0: 1823.4, 1: 1807.8. Samples: 3615398. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:22:51,034][70582] Avg episode reward: [(0, '11.040'), (1, '11.890')] [2023-10-11 19:22:51,085][71601] Updated weights for policy 0, policy_version 7060 (0.0008) [2023-10-11 19:22:51,456][71601] Updated weights for policy 0, policy_version 7070 (0.0011) [2023-10-11 19:22:52,189][71635] Updated weights for policy 1, policy_version 7042 (0.0009) [2023-10-11 19:22:52,560][71635] Updated weights for policy 1, policy_version 7052 (0.0009) [2023-10-11 19:22:52,929][71635] Updated weights for policy 1, policy_version 7062 (0.0007) [2023-10-11 19:22:53,289][71635] Updated weights for policy 1, policy_version 7072 (0.0007) [2023-10-11 19:22:55,143][71601] Updated weights for policy 0, policy_version 7080 (0.0009) [2023-10-11 19:22:55,517][71601] Updated weights for policy 0, policy_version 7090 (0.0008) [2023-10-11 19:22:55,890][71601] Updated weights for policy 0, policy_version 7100 (0.0007) [2023-10-11 19:22:56,034][70582] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 14483456. Throughput: 0: 1822.6, 1: 1797.8. Samples: 3637116. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:22:56,035][70582] Avg episode reward: [(0, '11.030'), (1, '11.370')] [2023-10-11 19:22:57,126][71635] Updated weights for policy 1, policy_version 7082 (0.0008) [2023-10-11 19:22:57,497][71635] Updated weights for policy 1, policy_version 7092 (0.0007) [2023-10-11 19:22:57,854][71635] Updated weights for policy 1, policy_version 7102 (0.0007) [2023-10-11 19:22:59,449][71601] Updated weights for policy 0, policy_version 7110 (0.0007) [2023-10-11 19:22:59,827][71601] Updated weights for policy 0, policy_version 7120 (0.0009) [2023-10-11 19:23:00,194][71601] Updated weights for policy 0, policy_version 7130 (0.0008) [2023-10-11 19:23:01,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 14581760. Throughput: 0: 1819.7, 1: 1792.3. Samples: 3647810. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-10-11 19:23:01,034][70582] Avg episode reward: [(0, '10.960'), (1, '10.610')] [2023-10-11 19:23:01,405][71635] Updated weights for policy 1, policy_version 7112 (0.0008) [2023-10-11 19:23:01,765][71635] Updated weights for policy 1, policy_version 7122 (0.0009) [2023-10-11 19:23:02,142][71635] Updated weights for policy 1, policy_version 7132 (0.0008) [2023-10-11 19:23:04,015][71601] Updated weights for policy 0, policy_version 7140 (0.0007) [2023-10-11 19:23:04,383][71601] Updated weights for policy 0, policy_version 7150 (0.0009) [2023-10-11 19:23:04,760][71601] Updated weights for policy 0, policy_version 7160 (0.0009) [2023-10-11 19:23:05,842][71635] Updated weights for policy 1, policy_version 7142 (0.0009) [2023-10-11 19:23:06,034][70582] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 14647296. Throughput: 0: 1819.4, 1: 1802.3. Samples: 3670146. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-11 19:23:06,034][70582] Avg episode reward: [(0, '12.060'), (1, '11.810')] [2023-10-11 19:23:06,222][71635] Updated weights for policy 1, policy_version 7152 (0.0009) [2023-10-11 19:23:06,589][71635] Updated weights for policy 1, policy_version 7162 (0.0009) [2023-10-11 19:23:08,399][71601] Updated weights for policy 0, policy_version 7170 (0.0008) [2023-10-11 19:23:08,779][71601] Updated weights for policy 0, policy_version 7180 (0.0008) [2023-10-11 19:23:09,143][71601] Updated weights for policy 0, policy_version 7190 (0.0010) [2023-10-11 19:23:09,517][71601] Updated weights for policy 0, policy_version 7200 (0.0009) [2023-10-11 19:23:10,138][71635] Updated weights for policy 1, policy_version 7172 (0.0008) [2023-10-11 19:23:10,503][71635] Updated weights for policy 1, policy_version 7182 (0.0007) [2023-10-11 19:23:10,877][71635] Updated weights for policy 1, policy_version 7192 (0.0007) [2023-10-11 19:23:11,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14200.3, 300 sec: 14551.2). Total num frames: 14712832. Throughput: 0: 1823.1, 1: 1811.9. Samples: 3691996. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-11 19:23:11,035][70582] Avg episode reward: [(0, '12.650'), (1, '12.830')] [2023-10-11 19:23:13,288][71601] Updated weights for policy 0, policy_version 7210 (0.0007) [2023-10-11 19:23:13,650][71601] Updated weights for policy 0, policy_version 7220 (0.0011) [2023-10-11 19:23:14,036][71601] Updated weights for policy 0, policy_version 7230 (0.0009) [2023-10-11 19:23:14,515][71635] Updated weights for policy 1, policy_version 7202 (0.0007) [2023-10-11 19:23:14,880][71635] Updated weights for policy 1, policy_version 7212 (0.0007) [2023-10-11 19:23:15,252][71635] Updated weights for policy 1, policy_version 7222 (0.0009) [2023-10-11 19:23:15,616][71635] Updated weights for policy 1, policy_version 7232 (0.0010) [2023-10-11 19:23:16,034][70582] Fps is (10 sec: 16383.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 14811136. Throughput: 0: 1823.0, 1: 1814.5. Samples: 3703164. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-11 19:23:16,035][70582] Avg episode reward: [(0, '12.190'), (1, '12.860')] [2023-10-11 19:23:17,550][71601] Updated weights for policy 0, policy_version 7240 (0.0009) [2023-10-11 19:23:17,925][71601] Updated weights for policy 0, policy_version 7250 (0.0008) [2023-10-11 19:23:18,295][71601] Updated weights for policy 0, policy_version 7260 (0.0008) [2023-10-11 19:23:19,263][71635] Updated weights for policy 1, policy_version 7242 (0.0008) [2023-10-11 19:23:19,633][71635] Updated weights for policy 1, policy_version 7252 (0.0008) [2023-10-11 19:23:19,993][71635] Updated weights for policy 1, policy_version 7262 (0.0009) [2023-10-11 19:23:21,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 14876672. Throughput: 0: 1818.8, 1: 1817.1. Samples: 3724476. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-11 19:23:21,034][70582] Avg episode reward: [(0, '13.940'), (1, '12.870')] [2023-10-11 19:23:21,992][71601] Updated weights for policy 0, policy_version 7270 (0.0008) [2023-10-11 19:23:22,360][71601] Updated weights for policy 0, policy_version 7280 (0.0007) [2023-10-11 19:23:22,731][71601] Updated weights for policy 0, policy_version 7290 (0.0007) [2023-10-11 19:23:23,923][71635] Updated weights for policy 1, policy_version 7272 (0.0010) [2023-10-11 19:23:24,296][71635] Updated weights for policy 1, policy_version 7282 (0.0011) [2023-10-11 19:23:24,660][71635] Updated weights for policy 1, policy_version 7292 (0.0009) [2023-10-11 19:23:26,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 14942208. Throughput: 0: 1813.0, 1: 1823.6. Samples: 3746396. Policy #0 lag: (min: 1.0, avg: 11.3, max: 33.0) [2023-10-11 19:23:26,035][70582] Avg episode reward: [(0, '13.810'), (1, '12.350')] [2023-10-11 19:23:26,425][71601] Updated weights for policy 0, policy_version 7300 (0.0008) [2023-10-11 19:23:26,797][71601] Updated weights for policy 0, policy_version 7310 (0.0011) [2023-10-11 19:23:27,178][71601] Updated weights for policy 0, policy_version 7320 (0.0010) [2023-10-11 19:23:28,302][71635] Updated weights for policy 1, policy_version 7302 (0.0010) [2023-10-11 19:23:28,666][71635] Updated weights for policy 1, policy_version 7312 (0.0010) [2023-10-11 19:23:29,032][71635] Updated weights for policy 1, policy_version 7322 (0.0011) [2023-10-11 19:23:30,988][71601] Updated weights for policy 0, policy_version 7330 (0.0011) [2023-10-11 19:23:31,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 15007744. Throughput: 0: 1809.0, 1: 1824.5. Samples: 3757232. Policy #0 lag: (min: 1.0, avg: 11.3, max: 33.0) [2023-10-11 19:23:31,034][70582] Avg episode reward: [(0, '13.670'), (1, '10.700')] [2023-10-11 19:23:31,369][71601] Updated weights for policy 0, policy_version 7340 (0.0010) [2023-10-11 19:23:31,738][71601] Updated weights for policy 0, policy_version 7350 (0.0008) [2023-10-11 19:23:32,113][71601] Updated weights for policy 0, policy_version 7360 (0.0007) [2023-10-11 19:23:32,711][71635] Updated weights for policy 1, policy_version 7332 (0.0009) [2023-10-11 19:23:33,077][71635] Updated weights for policy 1, policy_version 7342 (0.0007) [2023-10-11 19:23:33,445][71635] Updated weights for policy 1, policy_version 7352 (0.0007) [2023-10-11 19:23:35,808][71601] Updated weights for policy 0, policy_version 7370 (0.0010) [2023-10-11 19:23:36,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 15073280. Throughput: 0: 1811.4, 1: 1821.8. Samples: 3778892. Policy #0 lag: (min: 13.0, avg: 24.3, max: 45.0) [2023-10-11 19:23:36,035][70582] Avg episode reward: [(0, '14.450'), (1, '11.180')] [2023-10-11 19:23:36,177][71601] Updated weights for policy 0, policy_version 7380 (0.0009) [2023-10-11 19:23:36,555][71601] Updated weights for policy 0, policy_version 7390 (0.0011) [2023-10-11 19:23:37,140][71635] Updated weights for policy 1, policy_version 7362 (0.0008) [2023-10-11 19:23:37,512][71635] Updated weights for policy 1, policy_version 7372 (0.0010) [2023-10-11 19:23:37,873][71635] Updated weights for policy 1, policy_version 7382 (0.0007) [2023-10-11 19:23:38,245][71635] Updated weights for policy 1, policy_version 7392 (0.0007) [2023-10-11 19:23:40,051][71601] Updated weights for policy 0, policy_version 7400 (0.0009) [2023-10-11 19:23:40,429][71601] Updated weights for policy 0, policy_version 7410 (0.0009) [2023-10-11 19:23:40,818][71601] Updated weights for policy 0, policy_version 7420 (0.0010) [2023-10-11 19:23:41,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 15171584. Throughput: 0: 1811.1, 1: 1822.9. Samples: 3800646. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:23:41,034][70582] Avg episode reward: [(0, '14.150'), (1, '11.110')] [2023-10-11 19:23:42,149][71635] Updated weights for policy 1, policy_version 7402 (0.0009) [2023-10-11 19:23:42,512][71635] Updated weights for policy 1, policy_version 7412 (0.0007) [2023-10-11 19:23:42,880][71635] Updated weights for policy 1, policy_version 7422 (0.0009) [2023-10-11 19:23:44,580][71601] Updated weights for policy 0, policy_version 7430 (0.0008) [2023-10-11 19:23:44,962][71601] Updated weights for policy 0, policy_version 7440 (0.0008) [2023-10-11 19:23:45,332][71601] Updated weights for policy 0, policy_version 7450 (0.0008) [2023-10-11 19:23:46,034][70582] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 15237120. Throughput: 0: 1811.3, 1: 1821.2. Samples: 3811272. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:23:46,034][70582] Avg episode reward: [(0, '12.650'), (1, '12.270')] [2023-10-11 19:23:46,569][71635] Updated weights for policy 1, policy_version 7432 (0.0010) [2023-10-11 19:23:46,937][71635] Updated weights for policy 1, policy_version 7442 (0.0010) [2023-10-11 19:23:47,302][71635] Updated weights for policy 1, policy_version 7452 (0.0010) [2023-10-11 19:23:48,926][71601] Updated weights for policy 0, policy_version 7460 (0.0008) [2023-10-11 19:23:49,294][71601] Updated weights for policy 0, policy_version 7470 (0.0008) [2023-10-11 19:23:49,668][71601] Updated weights for policy 0, policy_version 7480 (0.0008) [2023-10-11 19:23:50,974][71635] Updated weights for policy 1, policy_version 7462 (0.0010) [2023-10-11 19:23:51,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 15302656. Throughput: 0: 1817.9, 1: 1811.6. Samples: 3833472. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-11 19:23:51,035][70582] Avg episode reward: [(0, '12.420'), (1, '13.120')] [2023-10-11 19:23:51,339][71635] Updated weights for policy 1, policy_version 7472 (0.0008) [2023-10-11 19:23:51,714][71635] Updated weights for policy 1, policy_version 7482 (0.0008) [2023-10-11 19:23:53,253][71601] Updated weights for policy 0, policy_version 7490 (0.0007) [2023-10-11 19:23:53,622][71601] Updated weights for policy 0, policy_version 7500 (0.0007) [2023-10-11 19:23:54,000][71601] Updated weights for policy 0, policy_version 7510 (0.0007) [2023-10-11 19:23:54,380][71601] Updated weights for policy 0, policy_version 7520 (0.0007) [2023-10-11 19:23:55,320][71635] Updated weights for policy 1, policy_version 7492 (0.0010) [2023-10-11 19:23:55,696][71635] Updated weights for policy 1, policy_version 7502 (0.0010) [2023-10-11 19:23:56,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 15368192. Throughput: 0: 1815.7, 1: 1818.9. Samples: 3855554. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-11 19:23:56,034][70582] Avg episode reward: [(0, '11.440'), (1, '13.630')] [2023-10-11 19:23:56,042][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000007520_7700480.pth... [2023-10-11 19:23:56,053][71635] Updated weights for policy 1, policy_version 7512 (0.0008) [2023-10-11 19:23:56,071][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000005824_5963776.pth [2023-10-11 19:23:56,345][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000007520_7700480.pth... [2023-10-11 19:23:56,373][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000005824_5963776.pth [2023-10-11 19:23:58,097][71601] Updated weights for policy 0, policy_version 7530 (0.0008) [2023-10-11 19:23:58,473][71601] Updated weights for policy 0, policy_version 7540 (0.0008) [2023-10-11 19:23:58,848][71601] Updated weights for policy 0, policy_version 7550 (0.0007) [2023-10-11 19:23:59,696][71635] Updated weights for policy 1, policy_version 7522 (0.0008) [2023-10-11 19:24:00,078][71635] Updated weights for policy 1, policy_version 7532 (0.0010) [2023-10-11 19:24:00,448][71635] Updated weights for policy 1, policy_version 7542 (0.0009) [2023-10-11 19:24:00,822][71635] Updated weights for policy 1, policy_version 7552 (0.0008) [2023-10-11 19:24:01,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 15466496. Throughput: 0: 1810.4, 1: 1816.5. Samples: 3866370. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:24:01,034][70582] Avg episode reward: [(0, '11.380'), (1, '13.910')] [2023-10-11 19:24:02,547][71601] Updated weights for policy 0, policy_version 7560 (0.0008) [2023-10-11 19:24:02,923][71601] Updated weights for policy 0, policy_version 7570 (0.0007) [2023-10-11 19:24:03,292][71601] Updated weights for policy 0, policy_version 7580 (0.0009) [2023-10-11 19:24:04,355][71635] Updated weights for policy 1, policy_version 7562 (0.0008) [2023-10-11 19:24:04,727][71635] Updated weights for policy 1, policy_version 7572 (0.0007) [2023-10-11 19:24:05,089][71635] Updated weights for policy 1, policy_version 7582 (0.0008) [2023-10-11 19:24:06,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 15532032. Throughput: 0: 1816.8, 1: 1827.1. Samples: 3888452. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:24:06,034][70582] Avg episode reward: [(0, '12.680'), (1, '13.220')] [2023-10-11 19:24:06,833][71601] Updated weights for policy 0, policy_version 7590 (0.0010) [2023-10-11 19:24:07,207][71601] Updated weights for policy 0, policy_version 7600 (0.0007) [2023-10-11 19:24:07,579][71601] Updated weights for policy 0, policy_version 7610 (0.0008) [2023-10-11 19:24:08,851][71635] Updated weights for policy 1, policy_version 7592 (0.0008) [2023-10-11 19:24:09,216][71635] Updated weights for policy 1, policy_version 7602 (0.0010) [2023-10-11 19:24:09,583][71635] Updated weights for policy 1, policy_version 7612 (0.0008) [2023-10-11 19:24:11,034][70582] Fps is (10 sec: 13106.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 15597568. Throughput: 0: 1826.1, 1: 1820.4. Samples: 3910490. Policy #0 lag: (min: 28.0, avg: 40.3, max: 60.0) [2023-10-11 19:24:11,035][70582] Avg episode reward: [(0, '11.930'), (1, '12.510')] [2023-10-11 19:24:11,167][71601] Updated weights for policy 0, policy_version 7620 (0.0008) [2023-10-11 19:24:11,530][71601] Updated weights for policy 0, policy_version 7630 (0.0008) [2023-10-11 19:24:11,909][71601] Updated weights for policy 0, policy_version 7640 (0.0007) [2023-10-11 19:24:13,151][71635] Updated weights for policy 1, policy_version 7622 (0.0008) [2023-10-11 19:24:13,509][71635] Updated weights for policy 1, policy_version 7632 (0.0008) [2023-10-11 19:24:13,880][71635] Updated weights for policy 1, policy_version 7642 (0.0008) [2023-10-11 19:24:15,596][71601] Updated weights for policy 0, policy_version 7650 (0.0009) [2023-10-11 19:24:15,996][71601] Updated weights for policy 0, policy_version 7660 (0.0008) [2023-10-11 19:24:16,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 15663104. Throughput: 0: 1828.3, 1: 1820.7. Samples: 3921434. Policy #0 lag: (min: 28.0, avg: 40.3, max: 60.0) [2023-10-11 19:24:16,035][70582] Avg episode reward: [(0, '11.840'), (1, '10.220')] [2023-10-11 19:24:16,367][71601] Updated weights for policy 0, policy_version 7670 (0.0009) [2023-10-11 19:24:16,745][71601] Updated weights for policy 0, policy_version 7680 (0.0010) [2023-10-11 19:24:17,654][71635] Updated weights for policy 1, policy_version 7652 (0.0010) [2023-10-11 19:24:18,026][71635] Updated weights for policy 1, policy_version 7662 (0.0009) [2023-10-11 19:24:18,392][71635] Updated weights for policy 1, policy_version 7672 (0.0009) [2023-10-11 19:24:20,651][71601] Updated weights for policy 0, policy_version 7690 (0.0010) [2023-10-11 19:24:21,017][71601] Updated weights for policy 0, policy_version 7700 (0.0010) [2023-10-11 19:24:21,034][70582] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 15728640. Throughput: 0: 1829.5, 1: 1824.8. Samples: 3943334. Policy #0 lag: (min: 25.0, avg: 33.4, max: 57.0) [2023-10-11 19:24:21,034][70582] Avg episode reward: [(0, '12.770'), (1, '10.630')] [2023-10-11 19:24:21,398][71601] Updated weights for policy 0, policy_version 7710 (0.0009) [2023-10-11 19:24:21,924][71635] Updated weights for policy 1, policy_version 7682 (0.0009) [2023-10-11 19:24:22,290][71635] Updated weights for policy 1, policy_version 7692 (0.0007) [2023-10-11 19:24:22,658][71635] Updated weights for policy 1, policy_version 7702 (0.0008) [2023-10-11 19:24:23,028][71635] Updated weights for policy 1, policy_version 7712 (0.0009) [2023-10-11 19:24:25,120][71601] Updated weights for policy 0, policy_version 7720 (0.0007) [2023-10-11 19:24:25,493][71601] Updated weights for policy 0, policy_version 7730 (0.0008) [2023-10-11 19:24:25,864][71601] Updated weights for policy 0, policy_version 7740 (0.0007) [2023-10-11 19:24:26,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 15826944. Throughput: 0: 1833.1, 1: 1836.1. Samples: 3965760. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-11 19:24:26,034][70582] Avg episode reward: [(0, '12.820'), (1, '11.220')] [2023-10-11 19:24:26,546][71635] Updated weights for policy 1, policy_version 7722 (0.0009) [2023-10-11 19:24:26,913][71635] Updated weights for policy 1, policy_version 7732 (0.0009) [2023-10-11 19:24:27,285][71635] Updated weights for policy 1, policy_version 7742 (0.0007) [2023-10-11 19:24:29,510][71601] Updated weights for policy 0, policy_version 7750 (0.0008) [2023-10-11 19:24:29,885][71601] Updated weights for policy 0, policy_version 7760 (0.0009) [2023-10-11 19:24:30,253][71601] Updated weights for policy 0, policy_version 7770 (0.0009) [2023-10-11 19:24:31,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 15892480. Throughput: 0: 1833.3, 1: 1842.6. Samples: 3976686. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-11 19:24:31,034][70582] Avg episode reward: [(0, '12.610'), (1, '12.100')] [2023-10-11 19:24:31,092][71635] Updated weights for policy 1, policy_version 7752 (0.0008) [2023-10-11 19:24:31,450][71635] Updated weights for policy 1, policy_version 7762 (0.0010) [2023-10-11 19:24:31,827][71635] Updated weights for policy 1, policy_version 7772 (0.0011) [2023-10-11 19:24:33,854][71601] Updated weights for policy 0, policy_version 7780 (0.0008) [2023-10-11 19:24:34,219][71601] Updated weights for policy 0, policy_version 7790 (0.0008) [2023-10-11 19:24:34,593][71601] Updated weights for policy 0, policy_version 7800 (0.0008) [2023-10-11 19:24:35,511][71635] Updated weights for policy 1, policy_version 7782 (0.0010) [2023-10-11 19:24:35,897][71635] Updated weights for policy 1, policy_version 7792 (0.0011) [2023-10-11 19:24:36,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 15958016. Throughput: 0: 1827.5, 1: 1839.0. Samples: 3998466. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 19:24:36,035][70582] Avg episode reward: [(0, '12.120'), (1, '13.810')] [2023-10-11 19:24:36,257][71635] Updated weights for policy 1, policy_version 7802 (0.0010) [2023-10-11 19:24:38,259][71601] Updated weights for policy 0, policy_version 7810 (0.0008) [2023-10-11 19:24:38,626][71601] Updated weights for policy 0, policy_version 7820 (0.0007) [2023-10-11 19:24:39,008][71601] Updated weights for policy 0, policy_version 7830 (0.0009) [2023-10-11 19:24:39,384][71601] Updated weights for policy 0, policy_version 7840 (0.0008) [2023-10-11 19:24:39,915][71635] Updated weights for policy 1, policy_version 7812 (0.0009) [2023-10-11 19:24:40,287][71635] Updated weights for policy 1, policy_version 7822 (0.0008) [2023-10-11 19:24:40,655][71635] Updated weights for policy 1, policy_version 7832 (0.0010) [2023-10-11 19:24:41,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 16056320. Throughput: 0: 1825.4, 1: 1824.3. Samples: 4019788. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 19:24:41,035][70582] Avg episode reward: [(0, '11.280'), (1, '14.610')] [2023-10-11 19:24:43,022][71601] Updated weights for policy 0, policy_version 7850 (0.0007) [2023-10-11 19:24:43,396][71601] Updated weights for policy 0, policy_version 7860 (0.0007) [2023-10-11 19:24:43,779][71601] Updated weights for policy 0, policy_version 7870 (0.0008) [2023-10-11 19:24:44,361][71635] Updated weights for policy 1, policy_version 7842 (0.0008) [2023-10-11 19:24:44,726][71635] Updated weights for policy 1, policy_version 7852 (0.0007) [2023-10-11 19:24:45,094][71635] Updated weights for policy 1, policy_version 7862 (0.0008) [2023-10-11 19:24:45,463][71635] Updated weights for policy 1, policy_version 7872 (0.0008) [2023-10-11 19:24:46,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 16121856. Throughput: 0: 1827.4, 1: 1830.3. Samples: 4030964. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:24:46,035][70582] Avg episode reward: [(0, '10.060'), (1, '15.870')] [2023-10-11 19:24:47,271][71601] Updated weights for policy 0, policy_version 7880 (0.0007) [2023-10-11 19:24:47,637][71601] Updated weights for policy 0, policy_version 7890 (0.0010) [2023-10-11 19:24:48,007][71601] Updated weights for policy 0, policy_version 7900 (0.0010) [2023-10-11 19:24:49,147][71635] Updated weights for policy 1, policy_version 7882 (0.0007) [2023-10-11 19:24:49,523][71635] Updated weights for policy 1, policy_version 7892 (0.0007) [2023-10-11 19:24:49,886][71635] Updated weights for policy 1, policy_version 7902 (0.0009) [2023-10-11 19:24:51,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 16187392. Throughput: 0: 1836.6, 1: 1818.7. Samples: 4052942. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:24:51,034][70582] Avg episode reward: [(0, '10.480'), (1, '15.340')] [2023-10-11 19:24:51,632][71601] Updated weights for policy 0, policy_version 7910 (0.0008) [2023-10-11 19:24:51,993][71601] Updated weights for policy 0, policy_version 7920 (0.0007) [2023-10-11 19:24:52,370][71601] Updated weights for policy 0, policy_version 7930 (0.0008) [2023-10-11 19:24:53,524][71635] Updated weights for policy 1, policy_version 7912 (0.0008) [2023-10-11 19:24:53,895][71635] Updated weights for policy 1, policy_version 7922 (0.0009) [2023-10-11 19:24:54,258][71635] Updated weights for policy 1, policy_version 7932 (0.0010) [2023-10-11 19:24:56,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 16252928. Throughput: 0: 1829.5, 1: 1829.8. Samples: 4075156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:24:56,035][70582] Avg episode reward: [(0, '10.930'), (1, '14.440')] [2023-10-11 19:24:56,110][71601] Updated weights for policy 0, policy_version 7940 (0.0009) [2023-10-11 19:24:56,484][71601] Updated weights for policy 0, policy_version 7950 (0.0009) [2023-10-11 19:24:56,848][71601] Updated weights for policy 0, policy_version 7960 (0.0009) [2023-10-11 19:24:57,965][71635] Updated weights for policy 1, policy_version 7942 (0.0011) [2023-10-11 19:24:58,339][71635] Updated weights for policy 1, policy_version 7952 (0.0009) [2023-10-11 19:24:58,705][71635] Updated weights for policy 1, policy_version 7962 (0.0007) [2023-10-11 19:25:00,905][71601] Updated weights for policy 0, policy_version 7970 (0.0009) [2023-10-11 19:25:01,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 16318464. Throughput: 0: 1826.0, 1: 1824.5. Samples: 4085708. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:25:01,035][70582] Avg episode reward: [(0, '11.500'), (1, '13.180')] [2023-10-11 19:25:01,301][71601] Updated weights for policy 0, policy_version 7980 (0.0008) [2023-10-11 19:25:01,682][71601] Updated weights for policy 0, policy_version 7990 (0.0008) [2023-10-11 19:25:02,043][71601] Updated weights for policy 0, policy_version 8000 (0.0008) [2023-10-11 19:25:02,513][71635] Updated weights for policy 1, policy_version 7972 (0.0007) [2023-10-11 19:25:02,870][71635] Updated weights for policy 1, policy_version 7982 (0.0008) [2023-10-11 19:25:03,249][71635] Updated weights for policy 1, policy_version 7992 (0.0008) [2023-10-11 19:25:05,716][71601] Updated weights for policy 0, policy_version 8010 (0.0008) [2023-10-11 19:25:06,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 16384000. Throughput: 0: 1816.9, 1: 1823.1. Samples: 4107132. Policy #0 lag: (min: 1.0, avg: 7.1, max: 33.0) [2023-10-11 19:25:06,035][70582] Avg episode reward: [(0, '12.040'), (1, '12.470')] [2023-10-11 19:25:06,090][71601] Updated weights for policy 0, policy_version 8020 (0.0010) [2023-10-11 19:25:06,456][71601] Updated weights for policy 0, policy_version 8030 (0.0007) [2023-10-11 19:25:06,957][71635] Updated weights for policy 1, policy_version 8002 (0.0007) [2023-10-11 19:25:07,326][71635] Updated weights for policy 1, policy_version 8012 (0.0007) [2023-10-11 19:25:07,702][71635] Updated weights for policy 1, policy_version 8022 (0.0008) [2023-10-11 19:25:08,070][71635] Updated weights for policy 1, policy_version 8032 (0.0009) [2023-10-11 19:25:10,079][71601] Updated weights for policy 0, policy_version 8040 (0.0009) [2023-10-11 19:25:10,454][71601] Updated weights for policy 0, policy_version 8050 (0.0010) [2023-10-11 19:25:10,823][71601] Updated weights for policy 0, policy_version 8060 (0.0008) [2023-10-11 19:25:11,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 16482304. Throughput: 0: 1814.8, 1: 1819.5. Samples: 4129302. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-11 19:25:11,034][70582] Avg episode reward: [(0, '11.700'), (1, '13.430')] [2023-10-11 19:25:11,747][71635] Updated weights for policy 1, policy_version 8042 (0.0008) [2023-10-11 19:25:12,115][71635] Updated weights for policy 1, policy_version 8052 (0.0007) [2023-10-11 19:25:12,487][71635] Updated weights for policy 1, policy_version 8062 (0.0008) [2023-10-11 19:25:14,660][71601] Updated weights for policy 0, policy_version 8070 (0.0008) [2023-10-11 19:25:15,036][71601] Updated weights for policy 0, policy_version 8080 (0.0008) [2023-10-11 19:25:15,419][71601] Updated weights for policy 0, policy_version 8090 (0.0010) [2023-10-11 19:25:16,034][70582] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 16547840. Throughput: 0: 1816.8, 1: 1816.2. Samples: 4140168. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-11 19:25:16,034][70582] Avg episode reward: [(0, '11.680'), (1, '12.580')] [2023-10-11 19:25:16,162][71635] Updated weights for policy 1, policy_version 8072 (0.0010) [2023-10-11 19:25:16,526][71635] Updated weights for policy 1, policy_version 8082 (0.0010) [2023-10-11 19:25:16,893][71635] Updated weights for policy 1, policy_version 8092 (0.0010) [2023-10-11 19:25:19,191][71601] Updated weights for policy 0, policy_version 8100 (0.0009) [2023-10-11 19:25:19,564][71601] Updated weights for policy 0, policy_version 8110 (0.0011) [2023-10-11 19:25:19,935][71601] Updated weights for policy 0, policy_version 8120 (0.0007) [2023-10-11 19:25:20,662][71635] Updated weights for policy 1, policy_version 8102 (0.0011) [2023-10-11 19:25:21,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 16613376. Throughput: 0: 1821.8, 1: 1821.9. Samples: 4162434. Policy #0 lag: (min: 2.0, avg: 2.0, max: 3.0) [2023-10-11 19:25:21,034][70582] Avg episode reward: [(0, '13.560'), (1, '12.640')] [2023-10-11 19:25:21,045][71635] Updated weights for policy 1, policy_version 8112 (0.0009) [2023-10-11 19:25:21,415][71635] Updated weights for policy 1, policy_version 8122 (0.0010) [2023-10-11 19:25:23,531][71601] Updated weights for policy 0, policy_version 8130 (0.0008) [2023-10-11 19:25:23,899][71601] Updated weights for policy 0, policy_version 8140 (0.0007) [2023-10-11 19:25:24,279][71601] Updated weights for policy 0, policy_version 8150 (0.0008) [2023-10-11 19:25:24,644][71601] Updated weights for policy 0, policy_version 8160 (0.0008) [2023-10-11 19:25:24,983][71635] Updated weights for policy 1, policy_version 8132 (0.0008) [2023-10-11 19:25:25,350][71635] Updated weights for policy 1, policy_version 8142 (0.0011) [2023-10-11 19:25:25,714][71635] Updated weights for policy 1, policy_version 8152 (0.0011) [2023-10-11 19:25:26,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 16711680. Throughput: 0: 1816.0, 1: 1826.0. Samples: 4183678. Policy #0 lag: (min: 2.0, avg: 2.0, max: 3.0) [2023-10-11 19:25:26,034][70582] Avg episode reward: [(0, '13.050'), (1, '12.530')] [2023-10-11 19:25:28,345][71601] Updated weights for policy 0, policy_version 8170 (0.0007) [2023-10-11 19:25:28,725][71601] Updated weights for policy 0, policy_version 8180 (0.0007) [2023-10-11 19:25:29,095][71601] Updated weights for policy 0, policy_version 8190 (0.0008) [2023-10-11 19:25:29,326][71635] Updated weights for policy 1, policy_version 8162 (0.0009) [2023-10-11 19:25:29,686][71635] Updated weights for policy 1, policy_version 8172 (0.0008) [2023-10-11 19:25:30,057][71635] Updated weights for policy 1, policy_version 8182 (0.0008) [2023-10-11 19:25:30,430][71635] Updated weights for policy 1, policy_version 8192 (0.0008) [2023-10-11 19:25:31,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 16777216. Throughput: 0: 1824.0, 1: 1828.1. Samples: 4195308. Policy #0 lag: (min: 15.0, avg: 17.7, max: 47.0) [2023-10-11 19:25:31,034][70582] Avg episode reward: [(0, '13.470'), (1, '12.060')] [2023-10-11 19:25:32,664][71601] Updated weights for policy 0, policy_version 8200 (0.0007) [2023-10-11 19:25:33,033][71601] Updated weights for policy 0, policy_version 8210 (0.0007) [2023-10-11 19:25:33,409][71601] Updated weights for policy 0, policy_version 8220 (0.0009) [2023-10-11 19:25:34,106][71635] Updated weights for policy 1, policy_version 8202 (0.0008) [2023-10-11 19:25:34,463][71635] Updated weights for policy 1, policy_version 8212 (0.0008) [2023-10-11 19:25:34,831][71635] Updated weights for policy 1, policy_version 8222 (0.0007) [2023-10-11 19:25:36,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 16842752. Throughput: 0: 1809.4, 1: 1827.8. Samples: 4216616. Policy #0 lag: (min: 15.0, avg: 17.7, max: 47.0) [2023-10-11 19:25:36,035][70582] Avg episode reward: [(0, '14.140'), (1, '13.020')] [2023-10-11 19:25:37,005][71601] Updated weights for policy 0, policy_version 8230 (0.0009) [2023-10-11 19:25:37,382][71601] Updated weights for policy 0, policy_version 8240 (0.0008) [2023-10-11 19:25:37,754][71601] Updated weights for policy 0, policy_version 8250 (0.0007) [2023-10-11 19:25:38,375][71635] Updated weights for policy 1, policy_version 8232 (0.0007) [2023-10-11 19:25:38,747][71635] Updated weights for policy 1, policy_version 8242 (0.0008) [2023-10-11 19:25:39,104][71635] Updated weights for policy 1, policy_version 8252 (0.0008) [2023-10-11 19:25:41,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 16908288. Throughput: 0: 1810.8, 1: 1831.3. Samples: 4239050. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-10-11 19:25:41,034][70582] Avg episode reward: [(0, '12.660'), (1, '12.830')] [2023-10-11 19:25:41,402][71601] Updated weights for policy 0, policy_version 8260 (0.0008) [2023-10-11 19:25:41,781][71601] Updated weights for policy 0, policy_version 8270 (0.0009) [2023-10-11 19:25:42,150][71601] Updated weights for policy 0, policy_version 8280 (0.0010) [2023-10-11 19:25:42,891][71635] Updated weights for policy 1, policy_version 8262 (0.0009) [2023-10-11 19:25:43,267][71635] Updated weights for policy 1, policy_version 8272 (0.0008) [2023-10-11 19:25:43,628][71635] Updated weights for policy 1, policy_version 8282 (0.0008) [2023-10-11 19:25:45,902][71601] Updated weights for policy 0, policy_version 8290 (0.0008) [2023-10-11 19:25:46,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 16973824. Throughput: 0: 1816.4, 1: 1828.8. Samples: 4249746. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-10-11 19:25:46,035][70582] Avg episode reward: [(0, '13.360'), (1, '13.900')] [2023-10-11 19:25:46,294][71601] Updated weights for policy 0, policy_version 8300 (0.0007) [2023-10-11 19:25:46,671][71601] Updated weights for policy 0, policy_version 8310 (0.0007) [2023-10-11 19:25:47,036][71601] Updated weights for policy 0, policy_version 8320 (0.0007) [2023-10-11 19:25:47,383][71635] Updated weights for policy 1, policy_version 8292 (0.0009) [2023-10-11 19:25:47,756][71635] Updated weights for policy 1, policy_version 8302 (0.0008) [2023-10-11 19:25:48,125][71635] Updated weights for policy 1, policy_version 8312 (0.0010) [2023-10-11 19:25:50,617][71601] Updated weights for policy 0, policy_version 8330 (0.0011) [2023-10-11 19:25:50,990][71601] Updated weights for policy 0, policy_version 8340 (0.0007) [2023-10-11 19:25:51,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 17039360. Throughput: 0: 1822.3, 1: 1834.7. Samples: 4271696. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:25:51,035][70582] Avg episode reward: [(0, '13.700'), (1, '14.220')] [2023-10-11 19:25:51,372][71601] Updated weights for policy 0, policy_version 8350 (0.0008) [2023-10-11 19:25:51,731][71635] Updated weights for policy 1, policy_version 8322 (0.0007) [2023-10-11 19:25:52,101][71635] Updated weights for policy 1, policy_version 8332 (0.0007) [2023-10-11 19:25:52,468][71635] Updated weights for policy 1, policy_version 8342 (0.0010) [2023-10-11 19:25:52,841][71635] Updated weights for policy 1, policy_version 8352 (0.0011) [2023-10-11 19:25:55,200][71601] Updated weights for policy 0, policy_version 8360 (0.0007) [2023-10-11 19:25:55,570][71601] Updated weights for policy 0, policy_version 8370 (0.0008) [2023-10-11 19:25:55,941][71601] Updated weights for policy 0, policy_version 8380 (0.0008) [2023-10-11 19:25:56,034][70582] Fps is (10 sec: 13107.7, 60 sec: 14199.6, 300 sec: 14440.1). Total num frames: 17104896. Throughput: 0: 1822.8, 1: 1830.2. Samples: 4293688. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:25:56,034][70582] Avg episode reward: [(0, '13.440'), (1, '15.070')] [2023-10-11 19:25:56,088][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000008384_8585216.pth... [2023-10-11 19:25:56,116][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000006656_6815744.pth [2023-10-11 19:25:56,120][71353] Saving a milestone ./train_atari/atari_gopher_APPO/checkpoint_p0/milestones/checkpoint_000008384_8585216.pth [2023-10-11 19:25:56,433][71635] Updated weights for policy 1, policy_version 8362 (0.0010) [2023-10-11 19:25:56,804][71635] Updated weights for policy 1, policy_version 8372 (0.0009) [2023-10-11 19:25:57,183][71635] Updated weights for policy 1, policy_version 8382 (0.0008) [2023-10-11 19:25:57,253][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000008384_8585216.pth... [2023-10-11 19:25:57,281][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000006656_6815744.pth [2023-10-11 19:25:57,285][71431] Saving a milestone ./train_atari/atari_gopher_APPO/checkpoint_p1/milestones/checkpoint_000008384_8585216.pth [2023-10-11 19:25:59,648][71601] Updated weights for policy 0, policy_version 8390 (0.0009) [2023-10-11 19:26:00,017][71601] Updated weights for policy 0, policy_version 8400 (0.0011) [2023-10-11 19:26:00,393][71601] Updated weights for policy 0, policy_version 8410 (0.0011) [2023-10-11 19:26:01,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 17203200. Throughput: 0: 1818.5, 1: 1827.6. Samples: 4304244. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-11 19:26:01,034][70582] Avg episode reward: [(0, '12.810'), (1, '13.650')] [2023-10-11 19:26:01,054][71635] Updated weights for policy 1, policy_version 8392 (0.0007) [2023-10-11 19:26:01,411][71635] Updated weights for policy 1, policy_version 8402 (0.0009) [2023-10-11 19:26:01,784][71635] Updated weights for policy 1, policy_version 8412 (0.0007) [2023-10-11 19:26:04,178][71601] Updated weights for policy 0, policy_version 8420 (0.0009) [2023-10-11 19:26:04,556][71601] Updated weights for policy 0, policy_version 8430 (0.0011) [2023-10-11 19:26:04,924][71601] Updated weights for policy 0, policy_version 8440 (0.0009) [2023-10-11 19:26:05,607][71635] Updated weights for policy 1, policy_version 8422 (0.0009) [2023-10-11 19:26:06,004][71635] Updated weights for policy 1, policy_version 8432 (0.0008) [2023-10-11 19:26:06,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 17268736. Throughput: 0: 1815.2, 1: 1825.0. Samples: 4326240. Policy #0 lag: (min: 21.0, avg: 21.5, max: 37.0) [2023-10-11 19:26:06,034][70582] Avg episode reward: [(0, '11.630'), (1, '12.940')] [2023-10-11 19:26:06,379][71635] Updated weights for policy 1, policy_version 8442 (0.0009) [2023-10-11 19:26:08,404][71601] Updated weights for policy 0, policy_version 8450 (0.0008) [2023-10-11 19:26:08,773][71601] Updated weights for policy 0, policy_version 8460 (0.0007) [2023-10-11 19:26:09,149][71601] Updated weights for policy 0, policy_version 8470 (0.0008) [2023-10-11 19:26:09,518][71601] Updated weights for policy 0, policy_version 8480 (0.0010) [2023-10-11 19:26:09,938][71635] Updated weights for policy 1, policy_version 8452 (0.0009) [2023-10-11 19:26:10,310][71635] Updated weights for policy 1, policy_version 8462 (0.0007) [2023-10-11 19:26:10,672][71635] Updated weights for policy 1, policy_version 8472 (0.0010) [2023-10-11 19:26:11,034][70582] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 17367040. Throughput: 0: 1818.0, 1: 1821.8. Samples: 4347472. Policy #0 lag: (min: 21.0, avg: 21.5, max: 37.0) [2023-10-11 19:26:11,035][70582] Avg episode reward: [(0, '12.000'), (1, '12.980')] [2023-10-11 19:26:13,278][71601] Updated weights for policy 0, policy_version 8490 (0.0009) [2023-10-11 19:26:13,657][71601] Updated weights for policy 0, policy_version 8500 (0.0008) [2023-10-11 19:26:14,036][71601] Updated weights for policy 0, policy_version 8510 (0.0009) [2023-10-11 19:26:14,322][71635] Updated weights for policy 1, policy_version 8482 (0.0009) [2023-10-11 19:26:14,690][71635] Updated weights for policy 1, policy_version 8492 (0.0008) [2023-10-11 19:26:15,057][71635] Updated weights for policy 1, policy_version 8502 (0.0008) [2023-10-11 19:26:15,424][71635] Updated weights for policy 1, policy_version 8512 (0.0008) [2023-10-11 19:26:16,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 17432576. Throughput: 0: 1816.7, 1: 1821.2. Samples: 4359014. Policy #0 lag: (min: 26.0, avg: 26.4, max: 39.0) [2023-10-11 19:26:16,034][70582] Avg episode reward: [(0, '12.800'), (1, '12.420')] [2023-10-11 19:26:17,695][71601] Updated weights for policy 0, policy_version 8520 (0.0010) [2023-10-11 19:26:18,071][71601] Updated weights for policy 0, policy_version 8530 (0.0007) [2023-10-11 19:26:18,440][71601] Updated weights for policy 0, policy_version 8540 (0.0007) [2023-10-11 19:26:19,154][71635] Updated weights for policy 1, policy_version 8522 (0.0009) [2023-10-11 19:26:19,521][71635] Updated weights for policy 1, policy_version 8532 (0.0009) [2023-10-11 19:26:19,895][71635] Updated weights for policy 1, policy_version 8542 (0.0007) [2023-10-11 19:26:21,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 17498112. Throughput: 0: 1820.2, 1: 1821.3. Samples: 4380482. Policy #0 lag: (min: 26.0, avg: 26.4, max: 39.0) [2023-10-11 19:26:21,034][70582] Avg episode reward: [(0, '12.850'), (1, '12.600')] [2023-10-11 19:26:21,936][71601] Updated weights for policy 0, policy_version 8550 (0.0010) [2023-10-11 19:26:22,298][71601] Updated weights for policy 0, policy_version 8560 (0.0008) [2023-10-11 19:26:22,671][71601] Updated weights for policy 0, policy_version 8570 (0.0008) [2023-10-11 19:26:23,525][71635] Updated weights for policy 1, policy_version 8552 (0.0008) [2023-10-11 19:26:23,898][71635] Updated weights for policy 1, policy_version 8562 (0.0008) [2023-10-11 19:26:24,274][71635] Updated weights for policy 1, policy_version 8572 (0.0007) [2023-10-11 19:26:26,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 17563648. Throughput: 0: 1824.7, 1: 1816.3. Samples: 4402900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:26:26,035][70582] Avg episode reward: [(0, '13.590'), (1, '13.570')] [2023-10-11 19:26:26,316][71601] Updated weights for policy 0, policy_version 8580 (0.0008) [2023-10-11 19:26:26,682][71601] Updated weights for policy 0, policy_version 8590 (0.0008) [2023-10-11 19:26:27,063][71601] Updated weights for policy 0, policy_version 8600 (0.0009) [2023-10-11 19:26:27,939][71635] Updated weights for policy 1, policy_version 8582 (0.0009) [2023-10-11 19:26:28,317][71635] Updated weights for policy 1, policy_version 8592 (0.0009) [2023-10-11 19:26:28,686][71635] Updated weights for policy 1, policy_version 8602 (0.0008) [2023-10-11 19:26:30,902][71601] Updated weights for policy 0, policy_version 8610 (0.0009) [2023-10-11 19:26:31,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 17629184. Throughput: 0: 1822.9, 1: 1817.4. Samples: 4413558. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:26:31,035][70582] Avg episode reward: [(0, '13.640'), (1, '12.840')] [2023-10-11 19:26:31,297][71601] Updated weights for policy 0, policy_version 8620 (0.0009) [2023-10-11 19:26:31,670][71601] Updated weights for policy 0, policy_version 8630 (0.0008) [2023-10-11 19:26:32,043][71601] Updated weights for policy 0, policy_version 8640 (0.0008) [2023-10-11 19:26:32,326][71635] Updated weights for policy 1, policy_version 8612 (0.0009) [2023-10-11 19:26:32,682][71635] Updated weights for policy 1, policy_version 8622 (0.0009) [2023-10-11 19:26:33,047][71635] Updated weights for policy 1, policy_version 8632 (0.0009) [2023-10-11 19:26:35,636][71601] Updated weights for policy 0, policy_version 8650 (0.0009) [2023-10-11 19:26:36,006][71601] Updated weights for policy 0, policy_version 8660 (0.0009) [2023-10-11 19:26:36,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 17694720. Throughput: 0: 1821.1, 1: 1813.2. Samples: 4435238. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:26:36,034][70582] Avg episode reward: [(0, '13.700'), (1, '12.050')] [2023-10-11 19:26:36,382][71601] Updated weights for policy 0, policy_version 8670 (0.0009) [2023-10-11 19:26:36,784][71635] Updated weights for policy 1, policy_version 8642 (0.0008) [2023-10-11 19:26:37,154][71635] Updated weights for policy 1, policy_version 8652 (0.0008) [2023-10-11 19:26:37,519][71635] Updated weights for policy 1, policy_version 8662 (0.0008) [2023-10-11 19:26:37,887][71635] Updated weights for policy 1, policy_version 8672 (0.0008) [2023-10-11 19:26:39,934][71601] Updated weights for policy 0, policy_version 8680 (0.0010) [2023-10-11 19:26:40,302][71601] Updated weights for policy 0, policy_version 8690 (0.0009) [2023-10-11 19:26:40,673][71601] Updated weights for policy 0, policy_version 8700 (0.0007) [2023-10-11 19:26:41,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 17793024. Throughput: 0: 1821.2, 1: 1813.3. Samples: 4457238. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-11 19:26:41,034][70582] Avg episode reward: [(0, '13.440'), (1, '10.950')] [2023-10-11 19:26:41,731][71635] Updated weights for policy 1, policy_version 8682 (0.0008) [2023-10-11 19:26:42,108][71635] Updated weights for policy 1, policy_version 8692 (0.0008) [2023-10-11 19:26:42,467][71635] Updated weights for policy 1, policy_version 8702 (0.0009) [2023-10-11 19:26:44,227][71601] Updated weights for policy 0, policy_version 8710 (0.0008) [2023-10-11 19:26:44,606][71601] Updated weights for policy 0, policy_version 8720 (0.0008) [2023-10-11 19:26:44,971][71601] Updated weights for policy 0, policy_version 8730 (0.0008) [2023-10-11 19:26:46,033][71635] Updated weights for policy 1, policy_version 8712 (0.0010) [2023-10-11 19:26:46,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 17858560. Throughput: 0: 1832.8, 1: 1814.6. Samples: 4468376. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-11 19:26:46,035][70582] Avg episode reward: [(0, '12.580'), (1, '10.720')] [2023-10-11 19:26:46,398][71635] Updated weights for policy 1, policy_version 8722 (0.0010) [2023-10-11 19:26:46,775][71635] Updated weights for policy 1, policy_version 8732 (0.0009) [2023-10-11 19:26:48,660][71601] Updated weights for policy 0, policy_version 8740 (0.0008) [2023-10-11 19:26:49,036][71601] Updated weights for policy 0, policy_version 8750 (0.0008) [2023-10-11 19:26:49,406][71601] Updated weights for policy 0, policy_version 8760 (0.0008) [2023-10-11 19:26:50,372][71635] Updated weights for policy 1, policy_version 8742 (0.0008) [2023-10-11 19:26:50,750][71635] Updated weights for policy 1, policy_version 8752 (0.0011) [2023-10-11 19:26:51,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 17924096. Throughput: 0: 1825.2, 1: 1819.2. Samples: 4490234. Policy #0 lag: (min: 9.0, avg: 14.7, max: 41.0) [2023-10-11 19:26:51,034][70582] Avg episode reward: [(0, '12.210'), (1, '10.620')] [2023-10-11 19:26:51,114][71635] Updated weights for policy 1, policy_version 8762 (0.0011) [2023-10-11 19:26:52,972][71601] Updated weights for policy 0, policy_version 8770 (0.0007) [2023-10-11 19:26:53,337][71601] Updated weights for policy 0, policy_version 8780 (0.0008) [2023-10-11 19:26:53,709][71601] Updated weights for policy 0, policy_version 8790 (0.0007) [2023-10-11 19:26:54,072][71601] Updated weights for policy 0, policy_version 8800 (0.0008) [2023-10-11 19:26:54,984][71635] Updated weights for policy 1, policy_version 8772 (0.0010) [2023-10-11 19:26:55,377][71635] Updated weights for policy 1, policy_version 8782 (0.0007) [2023-10-11 19:26:55,735][71635] Updated weights for policy 1, policy_version 8792 (0.0008) [2023-10-11 19:26:56,034][70582] Fps is (10 sec: 16383.6, 60 sec: 15291.6, 300 sec: 14662.3). Total num frames: 18022400. Throughput: 0: 1840.2, 1: 1823.1. Samples: 4512320. Policy #0 lag: (min: 9.0, avg: 14.7, max: 41.0) [2023-10-11 19:26:56,036][70582] Avg episode reward: [(0, '10.380'), (1, '11.250')] [2023-10-11 19:26:57,776][71601] Updated weights for policy 0, policy_version 8810 (0.0011) [2023-10-11 19:26:58,148][71601] Updated weights for policy 0, policy_version 8820 (0.0007) [2023-10-11 19:26:58,517][71601] Updated weights for policy 0, policy_version 8830 (0.0007) [2023-10-11 19:26:59,371][71635] Updated weights for policy 1, policy_version 8802 (0.0008) [2023-10-11 19:26:59,740][71635] Updated weights for policy 1, policy_version 8812 (0.0007) [2023-10-11 19:27:00,111][71635] Updated weights for policy 1, policy_version 8822 (0.0007) [2023-10-11 19:27:00,481][71635] Updated weights for policy 1, policy_version 8832 (0.0008) [2023-10-11 19:27:01,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 18087936. Throughput: 0: 1824.3, 1: 1820.5. Samples: 4523032. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:27:01,034][70582] Avg episode reward: [(0, '9.950'), (1, '11.430')] [2023-10-11 19:27:02,128][71601] Updated weights for policy 0, policy_version 8840 (0.0007) [2023-10-11 19:27:02,494][71601] Updated weights for policy 0, policy_version 8850 (0.0010) [2023-10-11 19:27:02,879][71601] Updated weights for policy 0, policy_version 8860 (0.0010) [2023-10-11 19:27:04,139][71635] Updated weights for policy 1, policy_version 8842 (0.0011) [2023-10-11 19:27:04,503][71635] Updated weights for policy 1, policy_version 8852 (0.0010) [2023-10-11 19:27:04,874][71635] Updated weights for policy 1, policy_version 8862 (0.0008) [2023-10-11 19:27:06,034][70582] Fps is (10 sec: 13107.7, 60 sec: 14745.6, 300 sec: 14551.4). Total num frames: 18153472. Throughput: 0: 1835.3, 1: 1817.2. Samples: 4544846. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:27:06,034][70582] Avg episode reward: [(0, '10.410'), (1, '13.680')] [2023-10-11 19:27:06,547][71601] Updated weights for policy 0, policy_version 8870 (0.0012) [2023-10-11 19:27:06,925][71601] Updated weights for policy 0, policy_version 8880 (0.0009) [2023-10-11 19:27:07,309][71601] Updated weights for policy 0, policy_version 8890 (0.0007) [2023-10-11 19:27:08,590][71635] Updated weights for policy 1, policy_version 8872 (0.0008) [2023-10-11 19:27:08,958][71635] Updated weights for policy 1, policy_version 8882 (0.0008) [2023-10-11 19:27:09,327][71635] Updated weights for policy 1, policy_version 8892 (0.0009) [2023-10-11 19:27:11,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 18219008. Throughput: 0: 1828.7, 1: 1820.7. Samples: 4567124. Policy #0 lag: (min: 17.0, avg: 21.0, max: 49.0) [2023-10-11 19:27:11,034][70582] Avg episode reward: [(0, '11.510'), (1, '14.290')] [2023-10-11 19:27:11,104][71601] Updated weights for policy 0, policy_version 8900 (0.0007) [2023-10-11 19:27:11,473][71601] Updated weights for policy 0, policy_version 8910 (0.0008) [2023-10-11 19:27:11,849][71601] Updated weights for policy 0, policy_version 8920 (0.0009) [2023-10-11 19:27:12,965][71635] Updated weights for policy 1, policy_version 8902 (0.0010) [2023-10-11 19:27:13,335][71635] Updated weights for policy 1, policy_version 8912 (0.0007) [2023-10-11 19:27:13,697][71635] Updated weights for policy 1, policy_version 8922 (0.0007) [2023-10-11 19:27:15,660][71601] Updated weights for policy 0, policy_version 8930 (0.0008) [2023-10-11 19:27:16,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 18284544. Throughput: 0: 1827.2, 1: 1822.1. Samples: 4577776. Policy #0 lag: (min: 17.0, avg: 21.0, max: 49.0) [2023-10-11 19:27:16,034][70582] Avg episode reward: [(0, '12.170'), (1, '15.560')] [2023-10-11 19:27:16,051][71601] Updated weights for policy 0, policy_version 8940 (0.0008) [2023-10-11 19:27:16,432][71601] Updated weights for policy 0, policy_version 8950 (0.0009) [2023-10-11 19:27:16,797][71601] Updated weights for policy 0, policy_version 8960 (0.0009) [2023-10-11 19:27:17,479][71635] Updated weights for policy 1, policy_version 8932 (0.0009) [2023-10-11 19:27:17,860][71635] Updated weights for policy 1, policy_version 8942 (0.0009) [2023-10-11 19:27:18,219][71635] Updated weights for policy 1, policy_version 8952 (0.0009) [2023-10-11 19:27:20,478][71601] Updated weights for policy 0, policy_version 8970 (0.0009) [2023-10-11 19:27:20,849][71601] Updated weights for policy 0, policy_version 8980 (0.0009) [2023-10-11 19:27:21,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 18350080. Throughput: 0: 1828.2, 1: 1820.2. Samples: 4599414. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:27:21,034][70582] Avg episode reward: [(0, '12.660'), (1, '15.040')] [2023-10-11 19:27:21,215][71601] Updated weights for policy 0, policy_version 8990 (0.0008) [2023-10-11 19:27:21,863][71635] Updated weights for policy 1, policy_version 8962 (0.0008) [2023-10-11 19:27:22,228][71635] Updated weights for policy 1, policy_version 8972 (0.0008) [2023-10-11 19:27:22,608][71635] Updated weights for policy 1, policy_version 8982 (0.0008) [2023-10-11 19:27:22,969][71635] Updated weights for policy 1, policy_version 8992 (0.0008) [2023-10-11 19:27:24,955][71601] Updated weights for policy 0, policy_version 9000 (0.0009) [2023-10-11 19:27:25,321][71601] Updated weights for policy 0, policy_version 9010 (0.0011) [2023-10-11 19:27:25,700][71601] Updated weights for policy 0, policy_version 9020 (0.0007) [2023-10-11 19:27:26,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 18448384. Throughput: 0: 1825.7, 1: 1821.5. Samples: 4621366. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:27:26,035][70582] Avg episode reward: [(0, '13.030'), (1, '15.940')] [2023-10-11 19:27:26,725][71635] Updated weights for policy 1, policy_version 9002 (0.0010) [2023-10-11 19:27:27,085][71635] Updated weights for policy 1, policy_version 9012 (0.0008) [2023-10-11 19:27:27,456][71635] Updated weights for policy 1, policy_version 9022 (0.0008) [2023-10-11 19:27:29,260][71601] Updated weights for policy 0, policy_version 9030 (0.0008) [2023-10-11 19:27:29,638][71601] Updated weights for policy 0, policy_version 9040 (0.0009) [2023-10-11 19:27:30,010][71601] Updated weights for policy 0, policy_version 9050 (0.0008) [2023-10-11 19:27:31,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 18513920. Throughput: 0: 1820.6, 1: 1823.3. Samples: 4632350. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:27:31,034][70582] Avg episode reward: [(0, '12.250'), (1, '14.450')] [2023-10-11 19:27:31,080][71635] Updated weights for policy 1, policy_version 9032 (0.0009) [2023-10-11 19:27:31,440][71635] Updated weights for policy 1, policy_version 9042 (0.0007) [2023-10-11 19:27:31,812][71635] Updated weights for policy 1, policy_version 9052 (0.0008) [2023-10-11 19:27:33,661][71601] Updated weights for policy 0, policy_version 9060 (0.0009) [2023-10-11 19:27:34,046][71601] Updated weights for policy 0, policy_version 9070 (0.0008) [2023-10-11 19:27:34,419][71601] Updated weights for policy 0, policy_version 9080 (0.0007) [2023-10-11 19:27:35,300][71635] Updated weights for policy 1, policy_version 9062 (0.0008) [2023-10-11 19:27:35,661][71635] Updated weights for policy 1, policy_version 9072 (0.0007) [2023-10-11 19:27:36,032][71635] Updated weights for policy 1, policy_version 9082 (0.0008) [2023-10-11 19:27:36,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 18579456. Throughput: 0: 1821.4, 1: 1825.1. Samples: 4654326. Policy #0 lag: (min: 26.0, avg: 29.0, max: 58.0) [2023-10-11 19:27:36,035][70582] Avg episode reward: [(0, '11.900'), (1, '12.210')] [2023-10-11 19:27:38,103][71601] Updated weights for policy 0, policy_version 9090 (0.0009) [2023-10-11 19:27:38,469][71601] Updated weights for policy 0, policy_version 9100 (0.0009) [2023-10-11 19:27:38,847][71601] Updated weights for policy 0, policy_version 9110 (0.0008) [2023-10-11 19:27:39,219][71601] Updated weights for policy 0, policy_version 9120 (0.0008) [2023-10-11 19:27:39,836][71635] Updated weights for policy 1, policy_version 9092 (0.0009) [2023-10-11 19:27:40,227][71635] Updated weights for policy 1, policy_version 9102 (0.0010) [2023-10-11 19:27:40,589][71635] Updated weights for policy 1, policy_version 9112 (0.0009) [2023-10-11 19:27:41,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 18677760. Throughput: 0: 1816.9, 1: 1814.5. Samples: 4675730. Policy #0 lag: (min: 26.0, avg: 29.0, max: 58.0) [2023-10-11 19:27:41,034][70582] Avg episode reward: [(0, '11.540'), (1, '11.910')] [2023-10-11 19:27:42,803][71601] Updated weights for policy 0, policy_version 9130 (0.0010) [2023-10-11 19:27:43,180][71601] Updated weights for policy 0, policy_version 9140 (0.0011) [2023-10-11 19:27:43,549][71601] Updated weights for policy 0, policy_version 9150 (0.0010) [2023-10-11 19:27:44,368][71635] Updated weights for policy 1, policy_version 9122 (0.0011) [2023-10-11 19:27:44,745][71635] Updated weights for policy 1, policy_version 9132 (0.0010) [2023-10-11 19:27:45,108][71635] Updated weights for policy 1, policy_version 9142 (0.0009) [2023-10-11 19:27:45,475][71635] Updated weights for policy 1, policy_version 9152 (0.0009) [2023-10-11 19:27:46,034][70582] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 18743296. Throughput: 0: 1816.1, 1: 1815.4. Samples: 4686450. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:27:46,034][70582] Avg episode reward: [(0, '11.000'), (1, '12.360')] [2023-10-11 19:27:47,268][71601] Updated weights for policy 0, policy_version 9160 (0.0010) [2023-10-11 19:27:47,635][71601] Updated weights for policy 0, policy_version 9170 (0.0010) [2023-10-11 19:27:48,008][71601] Updated weights for policy 0, policy_version 9180 (0.0009) [2023-10-11 19:27:49,292][71635] Updated weights for policy 1, policy_version 9162 (0.0010) [2023-10-11 19:27:49,655][71635] Updated weights for policy 1, policy_version 9172 (0.0010) [2023-10-11 19:27:50,022][71635] Updated weights for policy 1, policy_version 9182 (0.0009) [2023-10-11 19:27:51,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 18808832. Throughput: 0: 1812.8, 1: 1815.3. Samples: 4708110. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:27:51,035][70582] Avg episode reward: [(0, '11.180'), (1, '12.530')] [2023-10-11 19:27:51,772][71601] Updated weights for policy 0, policy_version 9190 (0.0009) [2023-10-11 19:27:52,140][71601] Updated weights for policy 0, policy_version 9200 (0.0008) [2023-10-11 19:27:52,513][71601] Updated weights for policy 0, policy_version 9210 (0.0007) [2023-10-11 19:27:53,784][71635] Updated weights for policy 1, policy_version 9192 (0.0008) [2023-10-11 19:27:54,146][71635] Updated weights for policy 1, policy_version 9202 (0.0008) [2023-10-11 19:27:54,522][71635] Updated weights for policy 1, policy_version 9212 (0.0009) [2023-10-11 19:27:56,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.6, 300 sec: 14551.2). Total num frames: 18874368. Throughput: 0: 1807.2, 1: 1805.4. Samples: 4729694. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:27:56,034][70582] Avg episode reward: [(0, '12.670'), (1, '13.480')] [2023-10-11 19:27:56,044][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000009216_9437184.pth... [2023-10-11 19:27:56,077][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000007520_7700480.pth [2023-10-11 19:27:56,221][71601] Updated weights for policy 0, policy_version 9220 (0.0007) [2023-10-11 19:27:56,591][71601] Updated weights for policy 0, policy_version 9230 (0.0007) [2023-10-11 19:27:56,966][71601] Updated weights for policy 0, policy_version 9240 (0.0008) [2023-10-11 19:27:57,265][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000009248_9469952.pth... [2023-10-11 19:27:57,303][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000007520_7700480.pth [2023-10-11 19:27:58,206][71635] Updated weights for policy 1, policy_version 9222 (0.0008) [2023-10-11 19:27:58,566][71635] Updated weights for policy 1, policy_version 9232 (0.0007) [2023-10-11 19:27:58,935][71635] Updated weights for policy 1, policy_version 9242 (0.0008) [2023-10-11 19:28:00,906][71601] Updated weights for policy 0, policy_version 9250 (0.0010) [2023-10-11 19:28:01,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 18939904. Throughput: 0: 1806.4, 1: 1811.3. Samples: 4740576. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:28:01,035][70582] Avg episode reward: [(0, '12.920'), (1, '13.900')] [2023-10-11 19:28:01,295][71601] Updated weights for policy 0, policy_version 9260 (0.0011) [2023-10-11 19:28:01,670][71601] Updated weights for policy 0, policy_version 9270 (0.0011) [2023-10-11 19:28:02,049][71601] Updated weights for policy 0, policy_version 9280 (0.0010) [2023-10-11 19:28:02,462][71635] Updated weights for policy 1, policy_version 9252 (0.0008) [2023-10-11 19:28:02,840][71635] Updated weights for policy 1, policy_version 9262 (0.0009) [2023-10-11 19:28:03,207][71635] Updated weights for policy 1, policy_version 9272 (0.0007) [2023-10-11 19:28:05,745][71601] Updated weights for policy 0, policy_version 9290 (0.0009) [2023-10-11 19:28:06,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 19005440. Throughput: 0: 1797.4, 1: 1811.5. Samples: 4761814. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:28:06,034][70582] Avg episode reward: [(0, '12.420'), (1, '12.060')] [2023-10-11 19:28:06,109][71601] Updated weights for policy 0, policy_version 9300 (0.0009) [2023-10-11 19:28:06,485][71601] Updated weights for policy 0, policy_version 9310 (0.0009) [2023-10-11 19:28:06,947][71635] Updated weights for policy 1, policy_version 9282 (0.0007) [2023-10-11 19:28:07,304][71635] Updated weights for policy 1, policy_version 9292 (0.0009) [2023-10-11 19:28:07,672][71635] Updated weights for policy 1, policy_version 9302 (0.0011) [2023-10-11 19:28:08,031][71635] Updated weights for policy 1, policy_version 9312 (0.0007) [2023-10-11 19:28:10,178][71601] Updated weights for policy 0, policy_version 9320 (0.0008) [2023-10-11 19:28:10,539][71601] Updated weights for policy 0, policy_version 9330 (0.0008) [2023-10-11 19:28:10,916][71601] Updated weights for policy 0, policy_version 9340 (0.0009) [2023-10-11 19:28:11,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14440.2). Total num frames: 19070976. Throughput: 0: 1801.2, 1: 1811.5. Samples: 4783934. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:28:11,034][70582] Avg episode reward: [(0, '13.000'), (1, '11.330')] [2023-10-11 19:28:11,689][71635] Updated weights for policy 1, policy_version 9322 (0.0008) [2023-10-11 19:28:12,051][71635] Updated weights for policy 1, policy_version 9332 (0.0007) [2023-10-11 19:28:12,417][71635] Updated weights for policy 1, policy_version 9342 (0.0007) [2023-10-11 19:28:14,493][71601] Updated weights for policy 0, policy_version 9350 (0.0007) [2023-10-11 19:28:14,864][71601] Updated weights for policy 0, policy_version 9360 (0.0009) [2023-10-11 19:28:15,230][71601] Updated weights for policy 0, policy_version 9370 (0.0010) [2023-10-11 19:28:16,002][71635] Updated weights for policy 1, policy_version 9352 (0.0007) [2023-10-11 19:28:16,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 19169280. Throughput: 0: 1800.2, 1: 1812.0. Samples: 4794898. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:28:16,034][70582] Avg episode reward: [(0, '12.050'), (1, '11.460')] [2023-10-11 19:28:16,377][71635] Updated weights for policy 1, policy_version 9362 (0.0008) [2023-10-11 19:28:16,736][71635] Updated weights for policy 1, policy_version 9372 (0.0007) [2023-10-11 19:28:19,114][71601] Updated weights for policy 0, policy_version 9380 (0.0009) [2023-10-11 19:28:19,487][71601] Updated weights for policy 0, policy_version 9390 (0.0008) [2023-10-11 19:28:19,862][71601] Updated weights for policy 0, policy_version 9400 (0.0007) [2023-10-11 19:28:20,489][71635] Updated weights for policy 1, policy_version 9382 (0.0007) [2023-10-11 19:28:20,848][71635] Updated weights for policy 1, policy_version 9392 (0.0009) [2023-10-11 19:28:21,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 19234816. Throughput: 0: 1808.5, 1: 1810.9. Samples: 4817196. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:28:21,034][70582] Avg episode reward: [(0, '12.230'), (1, '11.910')] [2023-10-11 19:28:21,222][71635] Updated weights for policy 1, policy_version 9402 (0.0007) [2023-10-11 19:28:23,731][71601] Updated weights for policy 0, policy_version 9410 (0.0008) [2023-10-11 19:28:24,104][71601] Updated weights for policy 0, policy_version 9420 (0.0009) [2023-10-11 19:28:24,473][71601] Updated weights for policy 0, policy_version 9430 (0.0010) [2023-10-11 19:28:24,842][71601] Updated weights for policy 0, policy_version 9440 (0.0009) [2023-10-11 19:28:25,221][71635] Updated weights for policy 1, policy_version 9412 (0.0008) [2023-10-11 19:28:25,621][71635] Updated weights for policy 1, policy_version 9422 (0.0007) [2023-10-11 19:28:25,984][71635] Updated weights for policy 1, policy_version 9432 (0.0008) [2023-10-11 19:28:26,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 19300352. Throughput: 0: 1799.4, 1: 1822.6. Samples: 4838718. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:28:26,034][70582] Avg episode reward: [(0, '11.910'), (1, '11.950')] [2023-10-11 19:28:28,504][71601] Updated weights for policy 0, policy_version 9450 (0.0007) [2023-10-11 19:28:28,870][71601] Updated weights for policy 0, policy_version 9460 (0.0009) [2023-10-11 19:28:29,248][71601] Updated weights for policy 0, policy_version 9470 (0.0008) [2023-10-11 19:28:29,594][71635] Updated weights for policy 1, policy_version 9442 (0.0007) [2023-10-11 19:28:29,962][71635] Updated weights for policy 1, policy_version 9452 (0.0007) [2023-10-11 19:28:30,325][71635] Updated weights for policy 1, policy_version 9462 (0.0007) [2023-10-11 19:28:30,698][71635] Updated weights for policy 1, policy_version 9472 (0.0009) [2023-10-11 19:28:31,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 19398656. Throughput: 0: 1821.9, 1: 1817.8. Samples: 4850236. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:28:31,035][70582] Avg episode reward: [(0, '10.840'), (1, '12.200')] [2023-10-11 19:28:32,965][71601] Updated weights for policy 0, policy_version 9480 (0.0008) [2023-10-11 19:28:33,336][71601] Updated weights for policy 0, policy_version 9490 (0.0007) [2023-10-11 19:28:33,708][71601] Updated weights for policy 0, policy_version 9500 (0.0009) [2023-10-11 19:28:34,266][71635] Updated weights for policy 1, policy_version 9482 (0.0008) [2023-10-11 19:28:34,633][71635] Updated weights for policy 1, policy_version 9492 (0.0008) [2023-10-11 19:28:35,014][71635] Updated weights for policy 1, policy_version 9502 (0.0009) [2023-10-11 19:28:36,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 19464192. Throughput: 0: 1802.8, 1: 1824.5. Samples: 4871340. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:28:36,035][70582] Avg episode reward: [(0, '10.130'), (1, '13.240')] [2023-10-11 19:28:37,495][71601] Updated weights for policy 0, policy_version 9510 (0.0009) [2023-10-11 19:28:37,871][71601] Updated weights for policy 0, policy_version 9520 (0.0008) [2023-10-11 19:28:38,243][71601] Updated weights for policy 0, policy_version 9530 (0.0007) [2023-10-11 19:28:38,585][71635] Updated weights for policy 1, policy_version 9512 (0.0007) [2023-10-11 19:28:38,964][71635] Updated weights for policy 1, policy_version 9522 (0.0009) [2023-10-11 19:28:39,333][71635] Updated weights for policy 1, policy_version 9532 (0.0009) [2023-10-11 19:28:41,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 19529728. Throughput: 0: 1805.3, 1: 1829.7. Samples: 4893272. Policy #0 lag: (min: 9.0, avg: 28.1, max: 41.0) [2023-10-11 19:28:41,035][70582] Avg episode reward: [(0, '10.920'), (1, '11.830')] [2023-10-11 19:28:41,832][71601] Updated weights for policy 0, policy_version 9540 (0.0007) [2023-10-11 19:28:42,208][71601] Updated weights for policy 0, policy_version 9550 (0.0011) [2023-10-11 19:28:42,575][71601] Updated weights for policy 0, policy_version 9560 (0.0010) [2023-10-11 19:28:42,948][71635] Updated weights for policy 1, policy_version 9542 (0.0008) [2023-10-11 19:28:43,321][71635] Updated weights for policy 1, policy_version 9552 (0.0009) [2023-10-11 19:28:43,696][71635] Updated weights for policy 1, policy_version 9562 (0.0009) [2023-10-11 19:28:46,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 19595264. Throughput: 0: 1806.2, 1: 1820.7. Samples: 4903786. Policy #0 lag: (min: 9.0, avg: 28.1, max: 41.0) [2023-10-11 19:28:46,034][70582] Avg episode reward: [(0, '12.300'), (1, '12.480')] [2023-10-11 19:28:46,369][71601] Updated weights for policy 0, policy_version 9570 (0.0008) [2023-10-11 19:28:46,775][71601] Updated weights for policy 0, policy_version 9580 (0.0010) [2023-10-11 19:28:47,146][71601] Updated weights for policy 0, policy_version 9590 (0.0008) [2023-10-11 19:28:47,522][71601] Updated weights for policy 0, policy_version 9600 (0.0007) [2023-10-11 19:28:47,526][71635] Updated weights for policy 1, policy_version 9572 (0.0009) [2023-10-11 19:28:47,897][71635] Updated weights for policy 1, policy_version 9582 (0.0009) [2023-10-11 19:28:48,269][71635] Updated weights for policy 1, policy_version 9592 (0.0007) [2023-10-11 19:28:51,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 19660800. Throughput: 0: 1816.9, 1: 1823.7. Samples: 4925642. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:28:51,034][70582] Avg episode reward: [(0, '12.500'), (1, '12.340')] [2023-10-11 19:28:51,123][71601] Updated weights for policy 0, policy_version 9610 (0.0007) [2023-10-11 19:28:51,509][71601] Updated weights for policy 0, policy_version 9620 (0.0007) [2023-10-11 19:28:51,883][71601] Updated weights for policy 0, policy_version 9630 (0.0008) [2023-10-11 19:28:51,897][71635] Updated weights for policy 1, policy_version 9602 (0.0010) [2023-10-11 19:28:52,265][71635] Updated weights for policy 1, policy_version 9612 (0.0009) [2023-10-11 19:28:52,630][71635] Updated weights for policy 1, policy_version 9622 (0.0007) [2023-10-11 19:28:53,001][71635] Updated weights for policy 1, policy_version 9632 (0.0008) [2023-10-11 19:28:55,514][71601] Updated weights for policy 0, policy_version 9640 (0.0008) [2023-10-11 19:28:55,879][71601] Updated weights for policy 0, policy_version 9650 (0.0007) [2023-10-11 19:28:56,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 19726336. Throughput: 0: 1827.8, 1: 1826.3. Samples: 4948370. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:28:56,034][70582] Avg episode reward: [(0, '12.590'), (1, '13.240')] [2023-10-11 19:28:56,251][71601] Updated weights for policy 0, policy_version 9660 (0.0010) [2023-10-11 19:28:56,759][71635] Updated weights for policy 1, policy_version 9642 (0.0008) [2023-10-11 19:28:57,114][71635] Updated weights for policy 1, policy_version 9652 (0.0009) [2023-10-11 19:28:57,479][71635] Updated weights for policy 1, policy_version 9662 (0.0009) [2023-10-11 19:28:59,968][71601] Updated weights for policy 0, policy_version 9670 (0.0008) [2023-10-11 19:29:00,349][71601] Updated weights for policy 0, policy_version 9680 (0.0009) [2023-10-11 19:29:00,715][71601] Updated weights for policy 0, policy_version 9690 (0.0007) [2023-10-11 19:29:01,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 19824640. Throughput: 0: 1814.0, 1: 1823.6. Samples: 4958594. Policy #0 lag: (min: 25.0, avg: 29.2, max: 57.0) [2023-10-11 19:29:01,034][70582] Avg episode reward: [(0, '13.140'), (1, '13.560')] [2023-10-11 19:29:01,247][71635] Updated weights for policy 1, policy_version 9672 (0.0009) [2023-10-11 19:29:01,602][71635] Updated weights for policy 1, policy_version 9682 (0.0008) [2023-10-11 19:29:01,973][71635] Updated weights for policy 1, policy_version 9692 (0.0008) [2023-10-11 19:29:04,468][71601] Updated weights for policy 0, policy_version 9700 (0.0009) [2023-10-11 19:29:04,837][71601] Updated weights for policy 0, policy_version 9710 (0.0010) [2023-10-11 19:29:05,208][71601] Updated weights for policy 0, policy_version 9720 (0.0009) [2023-10-11 19:29:05,668][71635] Updated weights for policy 1, policy_version 9702 (0.0009) [2023-10-11 19:29:06,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 19890176. Throughput: 0: 1822.1, 1: 1813.5. Samples: 4980800. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:29:06,035][70582] Avg episode reward: [(0, '12.950'), (1, '12.850')] [2023-10-11 19:29:06,037][71635] Updated weights for policy 1, policy_version 9712 (0.0007) [2023-10-11 19:29:06,414][71635] Updated weights for policy 1, policy_version 9722 (0.0008) [2023-10-11 19:29:08,939][71601] Updated weights for policy 0, policy_version 9730 (0.0010) [2023-10-11 19:29:09,306][71601] Updated weights for policy 0, policy_version 9740 (0.0011) [2023-10-11 19:29:09,676][71601] Updated weights for policy 0, policy_version 9750 (0.0010) [2023-10-11 19:29:10,048][71601] Updated weights for policy 0, policy_version 9760 (0.0007) [2023-10-11 19:29:10,083][71635] Updated weights for policy 1, policy_version 9732 (0.0007) [2023-10-11 19:29:10,473][71635] Updated weights for policy 1, policy_version 9742 (0.0007) [2023-10-11 19:29:10,849][71635] Updated weights for policy 1, policy_version 9752 (0.0009) [2023-10-11 19:29:11,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 19955712. Throughput: 0: 1804.8, 1: 1815.9. Samples: 5001650. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:29:11,035][70582] Avg episode reward: [(0, '12.900'), (1, '12.740')] [2023-10-11 19:29:13,768][71601] Updated weights for policy 0, policy_version 9770 (0.0010) [2023-10-11 19:29:14,137][71601] Updated weights for policy 0, policy_version 9780 (0.0009) [2023-10-11 19:29:14,513][71601] Updated weights for policy 0, policy_version 9790 (0.0007) [2023-10-11 19:29:14,543][71635] Updated weights for policy 1, policy_version 9762 (0.0008) [2023-10-11 19:29:14,909][71635] Updated weights for policy 1, policy_version 9772 (0.0009) [2023-10-11 19:29:15,281][71635] Updated weights for policy 1, policy_version 9782 (0.0010) [2023-10-11 19:29:15,643][71635] Updated weights for policy 1, policy_version 9792 (0.0010) [2023-10-11 19:29:16,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 20054016. Throughput: 0: 1810.6, 1: 1815.7. Samples: 5013418. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-10-11 19:29:16,034][70582] Avg episode reward: [(0, '13.630'), (1, '13.010')] [2023-10-11 19:29:18,071][71601] Updated weights for policy 0, policy_version 9800 (0.0007) [2023-10-11 19:29:18,446][71601] Updated weights for policy 0, policy_version 9810 (0.0007) [2023-10-11 19:29:18,825][71601] Updated weights for policy 0, policy_version 9820 (0.0008) [2023-10-11 19:29:19,243][71635] Updated weights for policy 1, policy_version 9802 (0.0007) [2023-10-11 19:29:19,616][71635] Updated weights for policy 1, policy_version 9812 (0.0008) [2023-10-11 19:29:19,973][71635] Updated weights for policy 1, policy_version 9822 (0.0009) [2023-10-11 19:29:21,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 20119552. Throughput: 0: 1805.2, 1: 1818.0. Samples: 5034384. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-10-11 19:29:21,034][70582] Avg episode reward: [(0, '12.440'), (1, '13.030')] [2023-10-11 19:29:22,433][71601] Updated weights for policy 0, policy_version 9830 (0.0007) [2023-10-11 19:29:22,799][71601] Updated weights for policy 0, policy_version 9840 (0.0008) [2023-10-11 19:29:23,169][71601] Updated weights for policy 0, policy_version 9850 (0.0007) [2023-10-11 19:29:23,772][71635] Updated weights for policy 1, policy_version 9832 (0.0008) [2023-10-11 19:29:24,134][71635] Updated weights for policy 1, policy_version 9842 (0.0007) [2023-10-11 19:29:24,497][71635] Updated weights for policy 1, policy_version 9852 (0.0007) [2023-10-11 19:29:26,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 20185088. Throughput: 0: 1806.4, 1: 1816.0. Samples: 5056284. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-10-11 19:29:26,035][70582] Avg episode reward: [(0, '11.680'), (1, '13.980')] [2023-10-11 19:29:26,975][71601] Updated weights for policy 0, policy_version 9860 (0.0009) [2023-10-11 19:29:27,350][71601] Updated weights for policy 0, policy_version 9870 (0.0008) [2023-10-11 19:29:27,725][71601] Updated weights for policy 0, policy_version 9880 (0.0011) [2023-10-11 19:29:28,213][71635] Updated weights for policy 1, policy_version 9862 (0.0009) [2023-10-11 19:29:28,580][71635] Updated weights for policy 1, policy_version 9872 (0.0011) [2023-10-11 19:29:28,946][71635] Updated weights for policy 1, policy_version 9882 (0.0009) [2023-10-11 19:29:31,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 20250624. Throughput: 0: 1807.2, 1: 1826.1. Samples: 5067288. Policy #0 lag: (min: 26.0, avg: 26.3, max: 37.0) [2023-10-11 19:29:31,035][70582] Avg episode reward: [(0, '11.100'), (1, '14.090')] [2023-10-11 19:29:31,473][71601] Updated weights for policy 0, policy_version 9890 (0.0009) [2023-10-11 19:29:31,840][71601] Updated weights for policy 0, policy_version 9900 (0.0009) [2023-10-11 19:29:32,222][71601] Updated weights for policy 0, policy_version 9910 (0.0007) [2023-10-11 19:29:32,591][71601] Updated weights for policy 0, policy_version 9920 (0.0007) [2023-10-11 19:29:32,644][71635] Updated weights for policy 1, policy_version 9892 (0.0008) [2023-10-11 19:29:33,010][71635] Updated weights for policy 1, policy_version 9902 (0.0007) [2023-10-11 19:29:33,365][71635] Updated weights for policy 1, policy_version 9912 (0.0009) [2023-10-11 19:29:36,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 20316160. Throughput: 0: 1811.6, 1: 1824.0. Samples: 5089244. Policy #0 lag: (min: 26.0, avg: 26.3, max: 37.0) [2023-10-11 19:29:36,034][70582] Avg episode reward: [(0, '12.180'), (1, '13.630')] [2023-10-11 19:29:36,344][71601] Updated weights for policy 0, policy_version 9930 (0.0008) [2023-10-11 19:29:36,716][71601] Updated weights for policy 0, policy_version 9940 (0.0008) [2023-10-11 19:29:36,886][71635] Updated weights for policy 1, policy_version 9922 (0.0009) [2023-10-11 19:29:37,097][71601] Updated weights for policy 0, policy_version 9950 (0.0008) [2023-10-11 19:29:37,258][71635] Updated weights for policy 1, policy_version 9932 (0.0008) [2023-10-11 19:29:37,617][71635] Updated weights for policy 1, policy_version 9942 (0.0007) [2023-10-11 19:29:37,984][71635] Updated weights for policy 1, policy_version 9952 (0.0008) [2023-10-11 19:29:40,881][71601] Updated weights for policy 0, policy_version 9960 (0.0008) [2023-10-11 19:29:41,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 20381696. Throughput: 0: 1805.9, 1: 1824.9. Samples: 5111756. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:29:41,035][70582] Avg episode reward: [(0, '11.270'), (1, '14.260')] [2023-10-11 19:29:41,255][71601] Updated weights for policy 0, policy_version 9970 (0.0007) [2023-10-11 19:29:41,637][71601] Updated weights for policy 0, policy_version 9980 (0.0009) [2023-10-11 19:29:41,657][71635] Updated weights for policy 1, policy_version 9962 (0.0008) [2023-10-11 19:29:42,021][71635] Updated weights for policy 1, policy_version 9972 (0.0007) [2023-10-11 19:29:42,388][71635] Updated weights for policy 1, policy_version 9982 (0.0009) [2023-10-11 19:29:45,162][71601] Updated weights for policy 0, policy_version 9990 (0.0008) [2023-10-11 19:29:45,536][71601] Updated weights for policy 0, policy_version 10000 (0.0007) [2023-10-11 19:29:45,911][71601] Updated weights for policy 0, policy_version 10010 (0.0009) [2023-10-11 19:29:46,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 20447232. Throughput: 0: 1798.4, 1: 1826.1. Samples: 5121700. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:29:46,034][70582] Avg episode reward: [(0, '12.260'), (1, '14.720')] [2023-10-11 19:29:46,208][71635] Updated weights for policy 1, policy_version 9992 (0.0007) [2023-10-11 19:29:46,569][71635] Updated weights for policy 1, policy_version 10002 (0.0007) [2023-10-11 19:29:46,939][71635] Updated weights for policy 1, policy_version 10012 (0.0007) [2023-10-11 19:29:49,637][71601] Updated weights for policy 0, policy_version 10020 (0.0007) [2023-10-11 19:29:50,010][71601] Updated weights for policy 0, policy_version 10030 (0.0007) [2023-10-11 19:29:50,382][71601] Updated weights for policy 0, policy_version 10040 (0.0008) [2023-10-11 19:29:50,589][71635] Updated weights for policy 1, policy_version 10022 (0.0007) [2023-10-11 19:29:50,956][71635] Updated weights for policy 1, policy_version 10032 (0.0009) [2023-10-11 19:29:51,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 20545536. Throughput: 0: 1806.8, 1: 1829.8. Samples: 5144450. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:29:51,035][70582] Avg episode reward: [(0, '11.810'), (1, '13.930')] [2023-10-11 19:29:51,315][71635] Updated weights for policy 1, policy_version 10042 (0.0007) [2023-10-11 19:29:54,138][71601] Updated weights for policy 0, policy_version 10050 (0.0007) [2023-10-11 19:29:54,518][71601] Updated weights for policy 0, policy_version 10060 (0.0011) [2023-10-11 19:29:54,876][71601] Updated weights for policy 0, policy_version 10070 (0.0008) [2023-10-11 19:29:55,117][71635] Updated weights for policy 1, policy_version 10052 (0.0007) [2023-10-11 19:29:55,255][71601] Updated weights for policy 0, policy_version 10080 (0.0007) [2023-10-11 19:29:55,512][71635] Updated weights for policy 1, policy_version 10062 (0.0007) [2023-10-11 19:29:55,881][71635] Updated weights for policy 1, policy_version 10072 (0.0008) [2023-10-11 19:29:56,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 20611072. Throughput: 0: 1804.4, 1: 1826.6. Samples: 5165044. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-11 19:29:56,034][70582] Avg episode reward: [(0, '12.940'), (1, '13.780')] [2023-10-11 19:29:56,047][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000010080_10321920.pth... [2023-10-11 19:29:56,083][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000008384_8585216.pth [2023-10-11 19:29:56,177][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000010080_10321920.pth... [2023-10-11 19:29:56,213][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000008384_8585216.pth [2023-10-11 19:29:58,914][71601] Updated weights for policy 0, policy_version 10090 (0.0008) [2023-10-11 19:29:59,290][71601] Updated weights for policy 0, policy_version 10100 (0.0008) [2023-10-11 19:29:59,596][71635] Updated weights for policy 1, policy_version 10082 (0.0009) [2023-10-11 19:29:59,663][71601] Updated weights for policy 0, policy_version 10110 (0.0007) [2023-10-11 19:29:59,970][71635] Updated weights for policy 1, policy_version 10092 (0.0007) [2023-10-11 19:30:00,338][71635] Updated weights for policy 1, policy_version 10102 (0.0007) [2023-10-11 19:30:00,707][71635] Updated weights for policy 1, policy_version 10112 (0.0008) [2023-10-11 19:30:01,034][70582] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 20709376. Throughput: 0: 1807.2, 1: 1823.3. Samples: 5176792. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-11 19:30:01,034][70582] Avg episode reward: [(0, '13.620'), (1, '12.310')] [2023-10-11 19:30:03,321][71601] Updated weights for policy 0, policy_version 10120 (0.0008) [2023-10-11 19:30:03,692][71601] Updated weights for policy 0, policy_version 10130 (0.0008) [2023-10-11 19:30:04,072][71601] Updated weights for policy 0, policy_version 10140 (0.0009) [2023-10-11 19:30:04,433][71635] Updated weights for policy 1, policy_version 10122 (0.0009) [2023-10-11 19:30:04,801][71635] Updated weights for policy 1, policy_version 10132 (0.0010) [2023-10-11 19:30:05,164][71635] Updated weights for policy 1, policy_version 10142 (0.0008) [2023-10-11 19:30:06,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 20774912. Throughput: 0: 1804.5, 1: 1823.3. Samples: 5197634. Policy #0 lag: (min: 9.0, avg: 15.6, max: 41.0) [2023-10-11 19:30:06,034][70582] Avg episode reward: [(0, '14.790'), (1, '11.770')] [2023-10-11 19:30:07,720][71601] Updated weights for policy 0, policy_version 10150 (0.0007) [2023-10-11 19:30:08,093][71601] Updated weights for policy 0, policy_version 10160 (0.0007) [2023-10-11 19:30:08,464][71601] Updated weights for policy 0, policy_version 10170 (0.0008) [2023-10-11 19:30:08,781][71635] Updated weights for policy 1, policy_version 10152 (0.0008) [2023-10-11 19:30:09,146][71635] Updated weights for policy 1, policy_version 10162 (0.0007) [2023-10-11 19:30:09,519][71635] Updated weights for policy 1, policy_version 10172 (0.0010) [2023-10-11 19:30:11,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 20840448. Throughput: 0: 1809.6, 1: 1819.0. Samples: 5219570. Policy #0 lag: (min: 9.0, avg: 15.6, max: 41.0) [2023-10-11 19:30:11,034][70582] Avg episode reward: [(0, '14.650'), (1, '12.210')] [2023-10-11 19:30:12,019][71601] Updated weights for policy 0, policy_version 10180 (0.0010) [2023-10-11 19:30:12,380][71601] Updated weights for policy 0, policy_version 10190 (0.0008) [2023-10-11 19:30:12,757][71601] Updated weights for policy 0, policy_version 10200 (0.0009) [2023-10-11 19:30:13,232][71635] Updated weights for policy 1, policy_version 10182 (0.0007) [2023-10-11 19:30:13,605][71635] Updated weights for policy 1, policy_version 10192 (0.0009) [2023-10-11 19:30:13,977][71635] Updated weights for policy 1, policy_version 10202 (0.0008) [2023-10-11 19:30:16,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 20905984. Throughput: 0: 1814.2, 1: 1820.6. Samples: 5230854. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-10-11 19:30:16,034][70582] Avg episode reward: [(0, '14.500'), (1, '12.250')] [2023-10-11 19:30:16,522][71601] Updated weights for policy 0, policy_version 10210 (0.0008) [2023-10-11 19:30:16,907][71601] Updated weights for policy 0, policy_version 10220 (0.0008) [2023-10-11 19:30:17,286][71601] Updated weights for policy 0, policy_version 10230 (0.0007) [2023-10-11 19:30:17,558][71635] Updated weights for policy 1, policy_version 10212 (0.0007) [2023-10-11 19:30:17,647][71601] Updated weights for policy 0, policy_version 10240 (0.0007) [2023-10-11 19:30:17,920][71635] Updated weights for policy 1, policy_version 10222 (0.0007) [2023-10-11 19:30:18,286][71635] Updated weights for policy 1, policy_version 10232 (0.0011) [2023-10-11 19:30:21,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 20971520. Throughput: 0: 1809.1, 1: 1822.8. Samples: 5252676. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-10-11 19:30:21,034][70582] Avg episode reward: [(0, '13.030'), (1, '12.160')] [2023-10-11 19:30:21,391][71601] Updated weights for policy 0, policy_version 10250 (0.0010) [2023-10-11 19:30:21,750][71635] Updated weights for policy 1, policy_version 10242 (0.0010) [2023-10-11 19:30:21,772][71601] Updated weights for policy 0, policy_version 10260 (0.0009) [2023-10-11 19:30:22,108][71635] Updated weights for policy 1, policy_version 10252 (0.0009) [2023-10-11 19:30:22,141][71601] Updated weights for policy 0, policy_version 10270 (0.0008) [2023-10-11 19:30:22,471][71635] Updated weights for policy 1, policy_version 10262 (0.0010) [2023-10-11 19:30:22,840][71635] Updated weights for policy 1, policy_version 10272 (0.0011) [2023-10-11 19:30:25,716][71601] Updated weights for policy 0, policy_version 10280 (0.0009) [2023-10-11 19:30:26,034][70582] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 21037056. Throughput: 0: 1818.1, 1: 1820.5. Samples: 5275496. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-10-11 19:30:26,035][70582] Avg episode reward: [(0, '11.920'), (1, '12.800')] [2023-10-11 19:30:26,087][71601] Updated weights for policy 0, policy_version 10290 (0.0007) [2023-10-11 19:30:26,465][71601] Updated weights for policy 0, policy_version 10300 (0.0008) [2023-10-11 19:30:26,666][71635] Updated weights for policy 1, policy_version 10282 (0.0007) [2023-10-11 19:30:27,028][71635] Updated weights for policy 1, policy_version 10292 (0.0008) [2023-10-11 19:30:27,389][71635] Updated weights for policy 1, policy_version 10302 (0.0009) [2023-10-11 19:30:30,143][71601] Updated weights for policy 0, policy_version 10310 (0.0007) [2023-10-11 19:30:30,510][71601] Updated weights for policy 0, policy_version 10320 (0.0008) [2023-10-11 19:30:30,893][71601] Updated weights for policy 0, policy_version 10330 (0.0010) [2023-10-11 19:30:31,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 21102592. Throughput: 0: 1822.2, 1: 1816.2. Samples: 5285428. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-11 19:30:31,034][70582] Avg episode reward: [(0, '12.700'), (1, '13.120')] [2023-10-11 19:30:31,349][71635] Updated weights for policy 1, policy_version 10312 (0.0009) [2023-10-11 19:30:31,713][71635] Updated weights for policy 1, policy_version 10322 (0.0010) [2023-10-11 19:30:32,088][71635] Updated weights for policy 1, policy_version 10332 (0.0010) [2023-10-11 19:30:34,623][71601] Updated weights for policy 0, policy_version 10340 (0.0008) [2023-10-11 19:30:34,998][71601] Updated weights for policy 0, policy_version 10350 (0.0007) [2023-10-11 19:30:35,373][71601] Updated weights for policy 0, policy_version 10360 (0.0009) [2023-10-11 19:30:35,697][71635] Updated weights for policy 1, policy_version 10342 (0.0009) [2023-10-11 19:30:36,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 21200896. Throughput: 0: 1823.1, 1: 1814.7. Samples: 5308152. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-11 19:30:36,035][70582] Avg episode reward: [(0, '11.440'), (1, '12.380')] [2023-10-11 19:30:36,073][71635] Updated weights for policy 1, policy_version 10352 (0.0008) [2023-10-11 19:30:36,444][71635] Updated weights for policy 1, policy_version 10362 (0.0008) [2023-10-11 19:30:38,923][71601] Updated weights for policy 0, policy_version 10370 (0.0007) [2023-10-11 19:30:39,297][71601] Updated weights for policy 0, policy_version 10380 (0.0010) [2023-10-11 19:30:39,670][71601] Updated weights for policy 0, policy_version 10390 (0.0008) [2023-10-11 19:30:40,043][71601] Updated weights for policy 0, policy_version 10400 (0.0007) [2023-10-11 19:30:40,156][71635] Updated weights for policy 1, policy_version 10372 (0.0009) [2023-10-11 19:30:40,552][71635] Updated weights for policy 1, policy_version 10382 (0.0009) [2023-10-11 19:30:40,924][71635] Updated weights for policy 1, policy_version 10392 (0.0008) [2023-10-11 19:30:41,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 21266432. Throughput: 0: 1829.2, 1: 1814.7. Samples: 5329020. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-11 19:30:41,034][70582] Avg episode reward: [(0, '11.600'), (1, '12.580')] [2023-10-11 19:30:43,840][71601] Updated weights for policy 0, policy_version 10410 (0.0010) [2023-10-11 19:30:44,213][71601] Updated weights for policy 0, policy_version 10420 (0.0011) [2023-10-11 19:30:44,477][71635] Updated weights for policy 1, policy_version 10402 (0.0007) [2023-10-11 19:30:44,587][71601] Updated weights for policy 0, policy_version 10430 (0.0010) [2023-10-11 19:30:44,845][71635] Updated weights for policy 1, policy_version 10412 (0.0007) [2023-10-11 19:30:45,218][71635] Updated weights for policy 1, policy_version 10422 (0.0008) [2023-10-11 19:30:45,579][71635] Updated weights for policy 1, policy_version 10432 (0.0008) [2023-10-11 19:30:46,034][70582] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 21364736. Throughput: 0: 1828.4, 1: 1820.0. Samples: 5340968. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-11 19:30:46,034][70582] Avg episode reward: [(0, '10.780'), (1, '12.780')] [2023-10-11 19:30:48,311][71601] Updated weights for policy 0, policy_version 10440 (0.0010) [2023-10-11 19:30:48,687][71601] Updated weights for policy 0, policy_version 10450 (0.0009) [2023-10-11 19:30:49,064][71601] Updated weights for policy 0, policy_version 10460 (0.0008) [2023-10-11 19:30:49,306][71635] Updated weights for policy 1, policy_version 10442 (0.0009) [2023-10-11 19:30:49,670][71635] Updated weights for policy 1, policy_version 10452 (0.0010) [2023-10-11 19:30:50,034][71635] Updated weights for policy 1, policy_version 10462 (0.0011) [2023-10-11 19:30:51,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 21430272. Throughput: 0: 1829.8, 1: 1817.4. Samples: 5361756. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-11 19:30:51,034][70582] Avg episode reward: [(0, '10.670'), (1, '12.420')] [2023-10-11 19:30:52,612][71601] Updated weights for policy 0, policy_version 10470 (0.0007) [2023-10-11 19:30:52,983][71601] Updated weights for policy 0, policy_version 10480 (0.0008) [2023-10-11 19:30:53,357][71601] Updated weights for policy 0, policy_version 10490 (0.0007) [2023-10-11 19:30:53,688][71635] Updated weights for policy 1, policy_version 10472 (0.0007) [2023-10-11 19:30:54,057][71635] Updated weights for policy 1, policy_version 10482 (0.0008) [2023-10-11 19:30:54,423][71635] Updated weights for policy 1, policy_version 10492 (0.0008) [2023-10-11 19:30:56,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 21495808. Throughput: 0: 1826.9, 1: 1821.9. Samples: 5383766. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-11 19:30:56,034][70582] Avg episode reward: [(0, '11.370'), (1, '12.340')] [2023-10-11 19:30:57,113][71601] Updated weights for policy 0, policy_version 10500 (0.0008) [2023-10-11 19:30:57,477][71601] Updated weights for policy 0, policy_version 10510 (0.0007) [2023-10-11 19:30:57,857][71601] Updated weights for policy 0, policy_version 10520 (0.0009) [2023-10-11 19:30:58,162][71635] Updated weights for policy 1, policy_version 10502 (0.0009) [2023-10-11 19:30:58,525][71635] Updated weights for policy 1, policy_version 10512 (0.0008) [2023-10-11 19:30:58,894][71635] Updated weights for policy 1, policy_version 10522 (0.0008) [2023-10-11 19:31:01,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 21561344. Throughput: 0: 1819.9, 1: 1814.9. Samples: 5394420. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-11 19:31:01,034][70582] Avg episode reward: [(0, '10.930'), (1, '12.740')] [2023-10-11 19:31:01,594][71601] Updated weights for policy 0, policy_version 10530 (0.0007) [2023-10-11 19:31:01,965][71601] Updated weights for policy 0, policy_version 10540 (0.0007) [2023-10-11 19:31:02,335][71601] Updated weights for policy 0, policy_version 10550 (0.0007) [2023-10-11 19:31:02,576][71635] Updated weights for policy 1, policy_version 10532 (0.0008) [2023-10-11 19:31:02,702][71601] Updated weights for policy 0, policy_version 10560 (0.0007) [2023-10-11 19:31:02,949][71635] Updated weights for policy 1, policy_version 10542 (0.0007) [2023-10-11 19:31:03,309][71635] Updated weights for policy 1, policy_version 10552 (0.0010) [2023-10-11 19:31:06,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 21626880. Throughput: 0: 1824.0, 1: 1810.5. Samples: 5416228. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-11 19:31:06,034][70582] Avg episode reward: [(0, '12.970'), (1, '12.030')] [2023-10-11 19:31:06,606][71601] Updated weights for policy 0, policy_version 10570 (0.0009) [2023-10-11 19:31:06,977][71601] Updated weights for policy 0, policy_version 10580 (0.0008) [2023-10-11 19:31:07,177][71635] Updated weights for policy 1, policy_version 10562 (0.0010) [2023-10-11 19:31:07,349][71601] Updated weights for policy 0, policy_version 10590 (0.0008) [2023-10-11 19:31:07,544][71635] Updated weights for policy 1, policy_version 10572 (0.0009) [2023-10-11 19:31:07,918][71635] Updated weights for policy 1, policy_version 10582 (0.0008) [2023-10-11 19:31:08,292][71635] Updated weights for policy 1, policy_version 10592 (0.0008) [2023-10-11 19:31:10,807][71601] Updated weights for policy 0, policy_version 10600 (0.0008) [2023-10-11 19:31:11,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 21692416. Throughput: 0: 1823.9, 1: 1801.8. Samples: 5438652. Policy #0 lag: (min: 25.0, avg: 37.8, max: 57.0) [2023-10-11 19:31:11,034][70582] Avg episode reward: [(0, '13.150'), (1, '12.510')] [2023-10-11 19:31:11,181][71601] Updated weights for policy 0, policy_version 10610 (0.0008) [2023-10-11 19:31:11,550][71601] Updated weights for policy 0, policy_version 10620 (0.0009) [2023-10-11 19:31:11,984][71635] Updated weights for policy 1, policy_version 10602 (0.0009) [2023-10-11 19:31:12,355][71635] Updated weights for policy 1, policy_version 10612 (0.0009) [2023-10-11 19:31:12,721][71635] Updated weights for policy 1, policy_version 10622 (0.0007) [2023-10-11 19:31:15,356][71601] Updated weights for policy 0, policy_version 10630 (0.0007) [2023-10-11 19:31:15,731][71601] Updated weights for policy 0, policy_version 10640 (0.0007) [2023-10-11 19:31:16,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 21757952. Throughput: 0: 1817.9, 1: 1805.6. Samples: 5448488. Policy #0 lag: (min: 25.0, avg: 37.8, max: 57.0) [2023-10-11 19:31:16,034][70582] Avg episode reward: [(0, '12.840'), (1, '11.910')] [2023-10-11 19:31:16,105][71601] Updated weights for policy 0, policy_version 10650 (0.0007) [2023-10-11 19:31:16,338][71635] Updated weights for policy 1, policy_version 10632 (0.0008) [2023-10-11 19:31:16,713][71635] Updated weights for policy 1, policy_version 10642 (0.0007) [2023-10-11 19:31:17,082][71635] Updated weights for policy 1, policy_version 10652 (0.0008) [2023-10-11 19:31:19,934][71601] Updated weights for policy 0, policy_version 10660 (0.0009) [2023-10-11 19:31:20,306][71601] Updated weights for policy 0, policy_version 10670 (0.0008) [2023-10-11 19:31:20,684][71601] Updated weights for policy 0, policy_version 10680 (0.0007) [2023-10-11 19:31:20,854][71635] Updated weights for policy 1, policy_version 10662 (0.0007) [2023-10-11 19:31:21,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 21856256. Throughput: 0: 1813.3, 1: 1806.5. Samples: 5471046. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-11 19:31:21,034][70582] Avg episode reward: [(0, '13.230'), (1, '12.320')] [2023-10-11 19:31:21,220][71635] Updated weights for policy 1, policy_version 10672 (0.0008) [2023-10-11 19:31:21,580][71635] Updated weights for policy 1, policy_version 10682 (0.0007) [2023-10-11 19:31:24,236][71601] Updated weights for policy 0, policy_version 10690 (0.0007) [2023-10-11 19:31:24,609][71601] Updated weights for policy 0, policy_version 10700 (0.0008) [2023-10-11 19:31:24,985][71601] Updated weights for policy 0, policy_version 10710 (0.0009) [2023-10-11 19:31:25,346][71601] Updated weights for policy 0, policy_version 10720 (0.0008) [2023-10-11 19:31:25,468][71635] Updated weights for policy 1, policy_version 10692 (0.0008) [2023-10-11 19:31:25,873][71635] Updated weights for policy 1, policy_version 10702 (0.0008) [2023-10-11 19:31:26,034][70582] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 21921792. Throughput: 0: 1810.6, 1: 1811.5. Samples: 5492016. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-11 19:31:26,035][70582] Avg episode reward: [(0, '12.910'), (1, '12.910')] [2023-10-11 19:31:26,237][71635] Updated weights for policy 1, policy_version 10712 (0.0008) [2023-10-11 19:31:28,906][71601] Updated weights for policy 0, policy_version 10730 (0.0009) [2023-10-11 19:31:29,276][71601] Updated weights for policy 0, policy_version 10740 (0.0010) [2023-10-11 19:31:29,644][71601] Updated weights for policy 0, policy_version 10750 (0.0009) [2023-10-11 19:31:29,914][71635] Updated weights for policy 1, policy_version 10722 (0.0009) [2023-10-11 19:31:30,283][71635] Updated weights for policy 1, policy_version 10732 (0.0008) [2023-10-11 19:31:30,665][71635] Updated weights for policy 1, policy_version 10742 (0.0008) [2023-10-11 19:31:31,034][70582] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 22020096. Throughput: 0: 1815.2, 1: 1796.2. Samples: 5503482. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-11 19:31:31,035][70582] Avg episode reward: [(0, '11.720'), (1, '12.800')] [2023-10-11 19:31:31,036][71635] Updated weights for policy 1, policy_version 10752 (0.0009) [2023-10-11 19:31:33,305][71601] Updated weights for policy 0, policy_version 10760 (0.0011) [2023-10-11 19:31:33,671][71601] Updated weights for policy 0, policy_version 10770 (0.0010) [2023-10-11 19:31:34,029][71601] Updated weights for policy 0, policy_version 10780 (0.0011) [2023-10-11 19:31:34,754][71635] Updated weights for policy 1, policy_version 10762 (0.0009) [2023-10-11 19:31:35,136][71635] Updated weights for policy 1, policy_version 10772 (0.0011) [2023-10-11 19:31:35,494][71635] Updated weights for policy 1, policy_version 10782 (0.0010) [2023-10-11 19:31:36,034][70582] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 22085632. Throughput: 0: 1811.5, 1: 1808.6. Samples: 5524658. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-11 19:31:36,034][70582] Avg episode reward: [(0, '12.500'), (1, '12.530')] [2023-10-11 19:31:37,816][71601] Updated weights for policy 0, policy_version 10790 (0.0009) [2023-10-11 19:31:38,193][71601] Updated weights for policy 0, policy_version 10800 (0.0007) [2023-10-11 19:31:38,566][71601] Updated weights for policy 0, policy_version 10810 (0.0007) [2023-10-11 19:31:39,206][71635] Updated weights for policy 1, policy_version 10792 (0.0010) [2023-10-11 19:31:39,562][71635] Updated weights for policy 1, policy_version 10802 (0.0010) [2023-10-11 19:31:39,927][71635] Updated weights for policy 1, policy_version 10812 (0.0010) [2023-10-11 19:31:41,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 22151168. Throughput: 0: 1807.7, 1: 1792.2. Samples: 5545760. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-11 19:31:41,034][70582] Avg episode reward: [(0, '12.910'), (1, '12.130')] [2023-10-11 19:31:42,390][71601] Updated weights for policy 0, policy_version 10820 (0.0010) [2023-10-11 19:31:42,763][71601] Updated weights for policy 0, policy_version 10830 (0.0008) [2023-10-11 19:31:43,129][71601] Updated weights for policy 0, policy_version 10840 (0.0009) [2023-10-11 19:31:43,597][71635] Updated weights for policy 1, policy_version 10822 (0.0007) [2023-10-11 19:31:43,965][71635] Updated weights for policy 1, policy_version 10832 (0.0009) [2023-10-11 19:31:44,325][71635] Updated weights for policy 1, policy_version 10842 (0.0009) [2023-10-11 19:31:46,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 22216704. Throughput: 0: 1813.5, 1: 1805.1. Samples: 5557254. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:31:46,035][70582] Avg episode reward: [(0, '12.310'), (1, '12.940')] [2023-10-11 19:31:46,800][71601] Updated weights for policy 0, policy_version 10850 (0.0009) [2023-10-11 19:31:47,165][71601] Updated weights for policy 0, policy_version 10860 (0.0007) [2023-10-11 19:31:47,543][71601] Updated weights for policy 0, policy_version 10870 (0.0007) [2023-10-11 19:31:47,911][71601] Updated weights for policy 0, policy_version 10880 (0.0008) [2023-10-11 19:31:47,976][71635] Updated weights for policy 1, policy_version 10852 (0.0007) [2023-10-11 19:31:48,353][71635] Updated weights for policy 1, policy_version 10862 (0.0007) [2023-10-11 19:31:48,719][71635] Updated weights for policy 1, policy_version 10872 (0.0009) [2023-10-11 19:31:51,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 22282240. Throughput: 0: 1812.6, 1: 1797.3. Samples: 5578672. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:31:51,034][70582] Avg episode reward: [(0, '12.030'), (1, '12.740')] [2023-10-11 19:31:51,467][71601] Updated weights for policy 0, policy_version 10890 (0.0008) [2023-10-11 19:31:51,840][71601] Updated weights for policy 0, policy_version 10900 (0.0009) [2023-10-11 19:31:52,210][71601] Updated weights for policy 0, policy_version 10910 (0.0008) [2023-10-11 19:31:52,623][71635] Updated weights for policy 1, policy_version 10882 (0.0007) [2023-10-11 19:31:52,999][71635] Updated weights for policy 1, policy_version 10892 (0.0009) [2023-10-11 19:31:53,367][71635] Updated weights for policy 1, policy_version 10902 (0.0010) [2023-10-11 19:31:53,739][71635] Updated weights for policy 1, policy_version 10912 (0.0008) [2023-10-11 19:31:55,937][71601] Updated weights for policy 0, policy_version 10920 (0.0008) [2023-10-11 19:31:56,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 22347776. Throughput: 0: 1817.2, 1: 1800.4. Samples: 5601444. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:31:56,034][70582] Avg episode reward: [(0, '11.120'), (1, '13.440')] [2023-10-11 19:31:56,042][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000010912_11173888.pth... [2023-10-11 19:31:56,075][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000009216_9437184.pth [2023-10-11 19:31:56,304][71601] Updated weights for policy 0, policy_version 10930 (0.0007) [2023-10-11 19:31:56,687][71601] Updated weights for policy 0, policy_version 10940 (0.0007) [2023-10-11 19:31:56,831][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000010944_11206656.pth... [2023-10-11 19:31:56,863][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000009248_9469952.pth [2023-10-11 19:31:57,345][71635] Updated weights for policy 1, policy_version 10922 (0.0007) [2023-10-11 19:31:57,703][71635] Updated weights for policy 1, policy_version 10932 (0.0008) [2023-10-11 19:31:58,071][71635] Updated weights for policy 1, policy_version 10942 (0.0008) [2023-10-11 19:32:00,447][71601] Updated weights for policy 0, policy_version 10950 (0.0009) [2023-10-11 19:32:00,822][71601] Updated weights for policy 0, policy_version 10960 (0.0010) [2023-10-11 19:32:01,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 22413312. Throughput: 0: 1821.1, 1: 1803.0. Samples: 5611574. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-11 19:32:01,035][70582] Avg episode reward: [(0, '10.060'), (1, '15.800')] [2023-10-11 19:32:01,192][71601] Updated weights for policy 0, policy_version 10970 (0.0010) [2023-10-11 19:32:01,801][71635] Updated weights for policy 1, policy_version 10952 (0.0009) [2023-10-11 19:32:02,172][71635] Updated weights for policy 1, policy_version 10962 (0.0008) [2023-10-11 19:32:02,537][71635] Updated weights for policy 1, policy_version 10972 (0.0009) [2023-10-11 19:32:04,786][71601] Updated weights for policy 0, policy_version 10980 (0.0009) [2023-10-11 19:32:05,159][71601] Updated weights for policy 0, policy_version 10990 (0.0007) [2023-10-11 19:32:05,530][71601] Updated weights for policy 0, policy_version 11000 (0.0007) [2023-10-11 19:32:06,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 22511616. Throughput: 0: 1820.7, 1: 1803.5. Samples: 5634136. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-11 19:32:06,035][70582] Avg episode reward: [(0, '10.550'), (1, '16.460')] [2023-10-11 19:32:06,036][71431] Saving new best policy, reward=16.460! [2023-10-11 19:32:06,437][71635] Updated weights for policy 1, policy_version 10982 (0.0011) [2023-10-11 19:32:06,799][71635] Updated weights for policy 1, policy_version 10992 (0.0010) [2023-10-11 19:32:07,161][71635] Updated weights for policy 1, policy_version 11002 (0.0007) [2023-10-11 19:32:08,996][71601] Updated weights for policy 0, policy_version 11010 (0.0007) [2023-10-11 19:32:09,369][71601] Updated weights for policy 0, policy_version 11020 (0.0009) [2023-10-11 19:32:09,743][71601] Updated weights for policy 0, policy_version 11030 (0.0007) [2023-10-11 19:32:10,112][71601] Updated weights for policy 0, policy_version 11040 (0.0008) [2023-10-11 19:32:10,739][71635] Updated weights for policy 1, policy_version 11012 (0.0008) [2023-10-11 19:32:11,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 22577152. Throughput: 0: 1824.5, 1: 1814.8. Samples: 5655782. Policy #0 lag: (min: 26.0, avg: 26.0, max: 30.0) [2023-10-11 19:32:11,035][70582] Avg episode reward: [(0, '10.620'), (1, '16.220')] [2023-10-11 19:32:11,109][71635] Updated weights for policy 1, policy_version 11022 (0.0008) [2023-10-11 19:32:11,475][71635] Updated weights for policy 1, policy_version 11032 (0.0008) [2023-10-11 19:32:13,854][71601] Updated weights for policy 0, policy_version 11050 (0.0009) [2023-10-11 19:32:14,228][71601] Updated weights for policy 0, policy_version 11060 (0.0010) [2023-10-11 19:32:14,600][71601] Updated weights for policy 0, policy_version 11070 (0.0009) [2023-10-11 19:32:15,087][71635] Updated weights for policy 1, policy_version 11042 (0.0009) [2023-10-11 19:32:15,460][71635] Updated weights for policy 1, policy_version 11052 (0.0009) [2023-10-11 19:32:15,830][71635] Updated weights for policy 1, policy_version 11062 (0.0011) [2023-10-11 19:32:16,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 22642688. Throughput: 0: 1820.9, 1: 1819.7. Samples: 5667312. Policy #0 lag: (min: 26.0, avg: 26.0, max: 30.0) [2023-10-11 19:32:16,034][70582] Avg episode reward: [(0, '10.630'), (1, '16.030')] [2023-10-11 19:32:16,199][71635] Updated weights for policy 1, policy_version 11072 (0.0011) [2023-10-11 19:32:18,308][71601] Updated weights for policy 0, policy_version 11080 (0.0009) [2023-10-11 19:32:18,673][71601] Updated weights for policy 0, policy_version 11090 (0.0010) [2023-10-11 19:32:19,050][71601] Updated weights for policy 0, policy_version 11100 (0.0010) [2023-10-11 19:32:19,902][71635] Updated weights for policy 1, policy_version 11082 (0.0008) [2023-10-11 19:32:20,264][71635] Updated weights for policy 1, policy_version 11092 (0.0007) [2023-10-11 19:32:20,631][71635] Updated weights for policy 1, policy_version 11102 (0.0008) [2023-10-11 19:32:21,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 22740992. Throughput: 0: 1820.4, 1: 1822.5. Samples: 5688590. Policy #0 lag: (min: 26.0, avg: 26.0, max: 30.0) [2023-10-11 19:32:21,034][70582] Avg episode reward: [(0, '11.870'), (1, '14.090')] [2023-10-11 19:32:22,712][71601] Updated weights for policy 0, policy_version 11110 (0.0008) [2023-10-11 19:32:23,090][71601] Updated weights for policy 0, policy_version 11120 (0.0007) [2023-10-11 19:32:23,469][71601] Updated weights for policy 0, policy_version 11130 (0.0009) [2023-10-11 19:32:24,217][71635] Updated weights for policy 1, policy_version 11112 (0.0009) [2023-10-11 19:32:24,579][71635] Updated weights for policy 1, policy_version 11122 (0.0010) [2023-10-11 19:32:24,957][71635] Updated weights for policy 1, policy_version 11132 (0.0010) [2023-10-11 19:32:26,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 22806528. Throughput: 0: 1822.8, 1: 1828.0. Samples: 5710050. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:32:26,035][70582] Avg episode reward: [(0, '13.350'), (1, '12.060')] [2023-10-11 19:32:27,195][71601] Updated weights for policy 0, policy_version 11140 (0.0009) [2023-10-11 19:32:27,571][71601] Updated weights for policy 0, policy_version 11150 (0.0008) [2023-10-11 19:32:27,937][71601] Updated weights for policy 0, policy_version 11160 (0.0008) [2023-10-11 19:32:28,758][71635] Updated weights for policy 1, policy_version 11142 (0.0008) [2023-10-11 19:32:29,124][71635] Updated weights for policy 1, policy_version 11152 (0.0007) [2023-10-11 19:32:29,499][71635] Updated weights for policy 1, policy_version 11162 (0.0008) [2023-10-11 19:32:31,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 22872064. Throughput: 0: 1816.0, 1: 1824.5. Samples: 5721076. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:32:31,034][70582] Avg episode reward: [(0, '12.190'), (1, '11.210')] [2023-10-11 19:32:31,699][71601] Updated weights for policy 0, policy_version 11170 (0.0010) [2023-10-11 19:32:32,062][71601] Updated weights for policy 0, policy_version 11180 (0.0007) [2023-10-11 19:32:32,440][71601] Updated weights for policy 0, policy_version 11190 (0.0010) [2023-10-11 19:32:32,813][71601] Updated weights for policy 0, policy_version 11200 (0.0011) [2023-10-11 19:32:33,145][71635] Updated weights for policy 1, policy_version 11172 (0.0008) [2023-10-11 19:32:33,518][71635] Updated weights for policy 1, policy_version 11182 (0.0007) [2023-10-11 19:32:33,882][71635] Updated weights for policy 1, policy_version 11192 (0.0008) [2023-10-11 19:32:36,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 22937600. Throughput: 0: 1812.6, 1: 1821.6. Samples: 5742214. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:32:36,034][70582] Avg episode reward: [(0, '11.870'), (1, '11.490')] [2023-10-11 19:32:36,637][71601] Updated weights for policy 0, policy_version 11210 (0.0007) [2023-10-11 19:32:37,000][71601] Updated weights for policy 0, policy_version 11220 (0.0009) [2023-10-11 19:32:37,381][71601] Updated weights for policy 0, policy_version 11230 (0.0009) [2023-10-11 19:32:37,682][71635] Updated weights for policy 1, policy_version 11202 (0.0007) [2023-10-11 19:32:38,046][71635] Updated weights for policy 1, policy_version 11212 (0.0007) [2023-10-11 19:32:38,427][71635] Updated weights for policy 1, policy_version 11222 (0.0010) [2023-10-11 19:32:38,787][71635] Updated weights for policy 1, policy_version 11232 (0.0009) [2023-10-11 19:32:41,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 23003136. Throughput: 0: 1806.4, 1: 1824.1. Samples: 5764816. Policy #0 lag: (min: 17.0, avg: 24.4, max: 49.0) [2023-10-11 19:32:41,035][70582] Avg episode reward: [(0, '11.860'), (1, '13.000')] [2023-10-11 19:32:41,133][71601] Updated weights for policy 0, policy_version 11240 (0.0009) [2023-10-11 19:32:41,498][71601] Updated weights for policy 0, policy_version 11250 (0.0008) [2023-10-11 19:32:41,872][71601] Updated weights for policy 0, policy_version 11260 (0.0008) [2023-10-11 19:32:42,429][71635] Updated weights for policy 1, policy_version 11242 (0.0008) [2023-10-11 19:32:42,794][71635] Updated weights for policy 1, policy_version 11252 (0.0009) [2023-10-11 19:32:43,169][71635] Updated weights for policy 1, policy_version 11262 (0.0007) [2023-10-11 19:32:45,503][71601] Updated weights for policy 0, policy_version 11270 (0.0008) [2023-10-11 19:32:45,876][71601] Updated weights for policy 0, policy_version 11280 (0.0009) [2023-10-11 19:32:46,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 23068672. Throughput: 0: 1807.4, 1: 1822.3. Samples: 5774908. Policy #0 lag: (min: 17.0, avg: 24.4, max: 49.0) [2023-10-11 19:32:46,034][70582] Avg episode reward: [(0, '11.960'), (1, '14.590')] [2023-10-11 19:32:46,250][71601] Updated weights for policy 0, policy_version 11290 (0.0007) [2023-10-11 19:32:46,744][71635] Updated weights for policy 1, policy_version 11272 (0.0008) [2023-10-11 19:32:47,115][71635] Updated weights for policy 1, policy_version 11282 (0.0010) [2023-10-11 19:32:47,482][71635] Updated weights for policy 1, policy_version 11292 (0.0011) [2023-10-11 19:32:49,887][71601] Updated weights for policy 0, policy_version 11300 (0.0008) [2023-10-11 19:32:50,261][71601] Updated weights for policy 0, policy_version 11310 (0.0007) [2023-10-11 19:32:50,634][71601] Updated weights for policy 0, policy_version 11320 (0.0007) [2023-10-11 19:32:51,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 23166976. Throughput: 0: 1808.2, 1: 1824.3. Samples: 5797596. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:32:51,034][70582] Avg episode reward: [(0, '11.140'), (1, '16.580')] [2023-10-11 19:32:51,154][71635] Updated weights for policy 1, policy_version 11302 (0.0008) [2023-10-11 19:32:51,524][71635] Updated weights for policy 1, policy_version 11312 (0.0009) [2023-10-11 19:32:51,897][71635] Updated weights for policy 1, policy_version 11322 (0.0009) [2023-10-11 19:32:52,115][71431] Saving new best policy, reward=16.580! [2023-10-11 19:32:54,310][71601] Updated weights for policy 0, policy_version 11330 (0.0007) [2023-10-11 19:32:54,675][71601] Updated weights for policy 0, policy_version 11340 (0.0008) [2023-10-11 19:32:55,060][71601] Updated weights for policy 0, policy_version 11350 (0.0009) [2023-10-11 19:32:55,431][71601] Updated weights for policy 0, policy_version 11360 (0.0008) [2023-10-11 19:32:55,592][71635] Updated weights for policy 1, policy_version 11332 (0.0007) [2023-10-11 19:32:55,996][71635] Updated weights for policy 1, policy_version 11342 (0.0007) [2023-10-11 19:32:56,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 23232512. Throughput: 0: 1804.9, 1: 1823.9. Samples: 5819076. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:32:56,034][70582] Avg episode reward: [(0, '12.470'), (1, '17.900')] [2023-10-11 19:32:56,358][71635] Updated weights for policy 1, policy_version 11352 (0.0007) [2023-10-11 19:32:56,649][71431] Saving new best policy, reward=17.900! [2023-10-11 19:32:59,040][71601] Updated weights for policy 0, policy_version 11370 (0.0010) [2023-10-11 19:32:59,399][71601] Updated weights for policy 0, policy_version 11380 (0.0010) [2023-10-11 19:32:59,776][71601] Updated weights for policy 0, policy_version 11390 (0.0011) [2023-10-11 19:33:00,120][71635] Updated weights for policy 1, policy_version 11362 (0.0008) [2023-10-11 19:33:00,498][71635] Updated weights for policy 1, policy_version 11372 (0.0007) [2023-10-11 19:33:00,861][71635] Updated weights for policy 1, policy_version 11382 (0.0009) [2023-10-11 19:33:01,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 23298048. Throughput: 0: 1807.4, 1: 1814.1. Samples: 5830280. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:33:01,034][70582] Avg episode reward: [(0, '13.010'), (1, '17.640')] [2023-10-11 19:33:01,220][71635] Updated weights for policy 1, policy_version 11392 (0.0010) [2023-10-11 19:33:03,369][71601] Updated weights for policy 0, policy_version 11400 (0.0007) [2023-10-11 19:33:03,744][71601] Updated weights for policy 0, policy_version 11410 (0.0007) [2023-10-11 19:33:04,122][71601] Updated weights for policy 0, policy_version 11420 (0.0008) [2023-10-11 19:33:04,915][71635] Updated weights for policy 1, policy_version 11402 (0.0008) [2023-10-11 19:33:05,287][71635] Updated weights for policy 1, policy_version 11412 (0.0007) [2023-10-11 19:33:05,648][71635] Updated weights for policy 1, policy_version 11422 (0.0008) [2023-10-11 19:33:06,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 23396352. Throughput: 0: 1812.7, 1: 1811.3. Samples: 5851668. Policy #0 lag: (min: 24.0, avg: 47.6, max: 56.0) [2023-10-11 19:33:06,035][70582] Avg episode reward: [(0, '12.760'), (1, '14.730')] [2023-10-11 19:33:07,802][71601] Updated weights for policy 0, policy_version 11430 (0.0009) [2023-10-11 19:33:08,172][71601] Updated weights for policy 0, policy_version 11440 (0.0010) [2023-10-11 19:33:08,551][71601] Updated weights for policy 0, policy_version 11450 (0.0011) [2023-10-11 19:33:09,189][71635] Updated weights for policy 1, policy_version 11432 (0.0009) [2023-10-11 19:33:09,549][71635] Updated weights for policy 1, policy_version 11442 (0.0010) [2023-10-11 19:33:09,924][71635] Updated weights for policy 1, policy_version 11452 (0.0009) [2023-10-11 19:33:11,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 23461888. Throughput: 0: 1814.9, 1: 1815.2. Samples: 5873402. Policy #0 lag: (min: 24.0, avg: 47.6, max: 56.0) [2023-10-11 19:33:11,035][70582] Avg episode reward: [(0, '13.620'), (1, '15.040')] [2023-10-11 19:33:12,365][71601] Updated weights for policy 0, policy_version 11460 (0.0010) [2023-10-11 19:33:12,746][71601] Updated weights for policy 0, policy_version 11470 (0.0008) [2023-10-11 19:33:13,113][71601] Updated weights for policy 0, policy_version 11480 (0.0008) [2023-10-11 19:33:13,595][71635] Updated weights for policy 1, policy_version 11462 (0.0009) [2023-10-11 19:33:13,954][71635] Updated weights for policy 1, policy_version 11472 (0.0009) [2023-10-11 19:33:14,325][71635] Updated weights for policy 1, policy_version 11482 (0.0008) [2023-10-11 19:33:16,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 23527424. Throughput: 0: 1817.5, 1: 1823.4. Samples: 5884918. Policy #0 lag: (min: 24.0, avg: 47.6, max: 56.0) [2023-10-11 19:33:16,034][70582] Avg episode reward: [(0, '13.940'), (1, '12.630')] [2023-10-11 19:33:16,961][71601] Updated weights for policy 0, policy_version 11490 (0.0008) [2023-10-11 19:33:17,333][71601] Updated weights for policy 0, policy_version 11500 (0.0008) [2023-10-11 19:33:17,701][71601] Updated weights for policy 0, policy_version 11510 (0.0007) [2023-10-11 19:33:17,934][71635] Updated weights for policy 1, policy_version 11492 (0.0008) [2023-10-11 19:33:18,068][71601] Updated weights for policy 0, policy_version 11520 (0.0008) [2023-10-11 19:33:18,303][71635] Updated weights for policy 1, policy_version 11502 (0.0008) [2023-10-11 19:33:18,669][71635] Updated weights for policy 1, policy_version 11512 (0.0007) [2023-10-11 19:33:21,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 23592960. Throughput: 0: 1812.6, 1: 1830.8. Samples: 5906166. Policy #0 lag: (min: 4.0, avg: 13.3, max: 36.0) [2023-10-11 19:33:21,035][70582] Avg episode reward: [(0, '12.990'), (1, '12.950')] [2023-10-11 19:33:21,900][71601] Updated weights for policy 0, policy_version 11530 (0.0007) [2023-10-11 19:33:22,270][71601] Updated weights for policy 0, policy_version 11540 (0.0009) [2023-10-11 19:33:22,392][71635] Updated weights for policy 1, policy_version 11522 (0.0008) [2023-10-11 19:33:22,647][71601] Updated weights for policy 0, policy_version 11550 (0.0008) [2023-10-11 19:33:22,747][71635] Updated weights for policy 1, policy_version 11532 (0.0007) [2023-10-11 19:33:23,120][71635] Updated weights for policy 1, policy_version 11542 (0.0008) [2023-10-11 19:33:23,489][71635] Updated weights for policy 1, policy_version 11552 (0.0008) [2023-10-11 19:33:26,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 23658496. Throughput: 0: 1818.1, 1: 1828.2. Samples: 5928902. Policy #0 lag: (min: 4.0, avg: 13.3, max: 36.0) [2023-10-11 19:33:26,034][70582] Avg episode reward: [(0, '12.720'), (1, '14.300')] [2023-10-11 19:33:26,386][71601] Updated weights for policy 0, policy_version 11560 (0.0007) [2023-10-11 19:33:26,767][71601] Updated weights for policy 0, policy_version 11570 (0.0008) [2023-10-11 19:33:27,136][71601] Updated weights for policy 0, policy_version 11580 (0.0010) [2023-10-11 19:33:27,314][71635] Updated weights for policy 1, policy_version 11562 (0.0008) [2023-10-11 19:33:27,674][71635] Updated weights for policy 1, policy_version 11572 (0.0008) [2023-10-11 19:33:28,048][71635] Updated weights for policy 1, policy_version 11582 (0.0007) [2023-10-11 19:33:30,891][71601] Updated weights for policy 0, policy_version 11590 (0.0007) [2023-10-11 19:33:31,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 23724032. Throughput: 0: 1815.1, 1: 1824.5. Samples: 5938690. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:33:31,034][70582] Avg episode reward: [(0, '13.800'), (1, '12.980')] [2023-10-11 19:33:31,261][71601] Updated weights for policy 0, policy_version 11600 (0.0007) [2023-10-11 19:33:31,633][71601] Updated weights for policy 0, policy_version 11610 (0.0007) [2023-10-11 19:33:31,686][71635] Updated weights for policy 1, policy_version 11592 (0.0007) [2023-10-11 19:33:32,061][71635] Updated weights for policy 1, policy_version 11602 (0.0009) [2023-10-11 19:33:32,426][71635] Updated weights for policy 1, policy_version 11612 (0.0009) [2023-10-11 19:33:35,296][71601] Updated weights for policy 0, policy_version 11620 (0.0010) [2023-10-11 19:33:35,674][71601] Updated weights for policy 0, policy_version 11630 (0.0009) [2023-10-11 19:33:36,034][70582] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 23789568. Throughput: 0: 1816.9, 1: 1820.0. Samples: 5961258. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:33:36,035][70582] Avg episode reward: [(0, '11.510'), (1, '13.870')] [2023-10-11 19:33:36,041][71601] Updated weights for policy 0, policy_version 11640 (0.0007) [2023-10-11 19:33:36,218][71635] Updated weights for policy 1, policy_version 11622 (0.0009) [2023-10-11 19:33:36,581][71635] Updated weights for policy 1, policy_version 11632 (0.0008) [2023-10-11 19:33:36,956][71635] Updated weights for policy 1, policy_version 11642 (0.0008) [2023-10-11 19:33:39,647][71601] Updated weights for policy 0, policy_version 11650 (0.0009) [2023-10-11 19:33:40,015][71601] Updated weights for policy 0, policy_version 11660 (0.0009) [2023-10-11 19:33:40,396][71601] Updated weights for policy 0, policy_version 11670 (0.0009) [2023-10-11 19:33:40,764][71601] Updated weights for policy 0, policy_version 11680 (0.0009) [2023-10-11 19:33:40,783][71635] Updated weights for policy 1, policy_version 11652 (0.0008) [2023-10-11 19:33:41,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 23887872. Throughput: 0: 1831.4, 1: 1813.8. Samples: 5983110. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:33:41,034][70582] Avg episode reward: [(0, '11.590'), (1, '14.070')] [2023-10-11 19:33:41,182][71635] Updated weights for policy 1, policy_version 11662 (0.0009) [2023-10-11 19:33:41,556][71635] Updated weights for policy 1, policy_version 11672 (0.0009) [2023-10-11 19:33:44,486][71601] Updated weights for policy 0, policy_version 11690 (0.0009) [2023-10-11 19:33:44,860][71601] Updated weights for policy 0, policy_version 11700 (0.0008) [2023-10-11 19:33:45,160][71635] Updated weights for policy 1, policy_version 11682 (0.0010) [2023-10-11 19:33:45,227][71601] Updated weights for policy 0, policy_version 11710 (0.0010) [2023-10-11 19:33:45,521][71635] Updated weights for policy 1, policy_version 11692 (0.0008) [2023-10-11 19:33:45,891][71635] Updated weights for policy 1, policy_version 11702 (0.0007) [2023-10-11 19:33:46,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 23953408. Throughput: 0: 1815.9, 1: 1815.0. Samples: 5993672. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:33:46,035][70582] Avg episode reward: [(0, '11.470'), (1, '12.930')] [2023-10-11 19:33:46,254][71635] Updated weights for policy 1, policy_version 11712 (0.0007) [2023-10-11 19:33:48,758][71601] Updated weights for policy 0, policy_version 11720 (0.0008) [2023-10-11 19:33:49,131][71601] Updated weights for policy 0, policy_version 11730 (0.0009) [2023-10-11 19:33:49,511][71601] Updated weights for policy 0, policy_version 11740 (0.0009) [2023-10-11 19:33:50,011][71635] Updated weights for policy 1, policy_version 11722 (0.0009) [2023-10-11 19:33:50,372][71635] Updated weights for policy 1, policy_version 11732 (0.0008) [2023-10-11 19:33:50,737][71635] Updated weights for policy 1, policy_version 11742 (0.0008) [2023-10-11 19:33:51,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 24051712. Throughput: 0: 1821.5, 1: 1813.5. Samples: 6015244. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:33:51,035][70582] Avg episode reward: [(0, '11.190'), (1, '12.690')] [2023-10-11 19:33:53,235][71601] Updated weights for policy 0, policy_version 11750 (0.0008) [2023-10-11 19:33:53,604][71601] Updated weights for policy 0, policy_version 11760 (0.0007) [2023-10-11 19:33:53,976][71601] Updated weights for policy 0, policy_version 11770 (0.0010) [2023-10-11 19:33:54,479][71635] Updated weights for policy 1, policy_version 11752 (0.0011) [2023-10-11 19:33:54,851][71635] Updated weights for policy 1, policy_version 11762 (0.0009) [2023-10-11 19:33:55,218][71635] Updated weights for policy 1, policy_version 11772 (0.0009) [2023-10-11 19:33:56,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 24117248. Throughput: 0: 1806.9, 1: 1808.0. Samples: 6036074. Policy #0 lag: (min: 9.0, avg: 29.4, max: 41.0) [2023-10-11 19:33:56,035][70582] Avg episode reward: [(0, '12.320'), (1, '11.710')] [2023-10-11 19:33:56,046][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000011776_12058624.pth... [2023-10-11 19:33:56,047][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000011776_12058624.pth... [2023-10-11 19:33:56,088][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000010080_10321920.pth [2023-10-11 19:33:56,088][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000010080_10321920.pth [2023-10-11 19:33:57,666][71601] Updated weights for policy 0, policy_version 11780 (0.0007) [2023-10-11 19:33:58,032][71601] Updated weights for policy 0, policy_version 11790 (0.0008) [2023-10-11 19:33:58,398][71601] Updated weights for policy 0, policy_version 11800 (0.0007) [2023-10-11 19:33:58,974][71635] Updated weights for policy 1, policy_version 11782 (0.0010) [2023-10-11 19:33:59,349][71635] Updated weights for policy 1, policy_version 11792 (0.0009) [2023-10-11 19:33:59,716][71635] Updated weights for policy 1, policy_version 11802 (0.0007) [2023-10-11 19:34:01,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 24182784. Throughput: 0: 1817.1, 1: 1804.5. Samples: 6047888. Policy #0 lag: (min: 9.0, avg: 29.4, max: 41.0) [2023-10-11 19:34:01,035][70582] Avg episode reward: [(0, '15.330'), (1, '12.430')] [2023-10-11 19:34:01,036][71353] Saving new best policy, reward=15.330! [2023-10-11 19:34:02,127][71601] Updated weights for policy 0, policy_version 11810 (0.0008) [2023-10-11 19:34:02,498][71601] Updated weights for policy 0, policy_version 11820 (0.0008) [2023-10-11 19:34:02,871][71601] Updated weights for policy 0, policy_version 11830 (0.0007) [2023-10-11 19:34:03,247][71601] Updated weights for policy 0, policy_version 11840 (0.0008) [2023-10-11 19:34:03,360][71635] Updated weights for policy 1, policy_version 11812 (0.0008) [2023-10-11 19:34:03,725][71635] Updated weights for policy 1, policy_version 11822 (0.0008) [2023-10-11 19:34:04,090][71635] Updated weights for policy 1, policy_version 11832 (0.0008) [2023-10-11 19:34:06,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 24248320. Throughput: 0: 1819.2, 1: 1805.6. Samples: 6069280. Policy #0 lag: (min: 9.0, avg: 29.4, max: 41.0) [2023-10-11 19:34:06,035][70582] Avg episode reward: [(0, '16.280'), (1, '12.380')] [2023-10-11 19:34:06,036][71353] Saving new best policy, reward=16.280! [2023-10-11 19:34:06,825][71601] Updated weights for policy 0, policy_version 11850 (0.0009) [2023-10-11 19:34:07,194][71601] Updated weights for policy 0, policy_version 11860 (0.0010) [2023-10-11 19:34:07,573][71601] Updated weights for policy 0, policy_version 11870 (0.0007) [2023-10-11 19:34:07,797][71635] Updated weights for policy 1, policy_version 11842 (0.0010) [2023-10-11 19:34:08,169][71635] Updated weights for policy 1, policy_version 11852 (0.0009) [2023-10-11 19:34:08,542][71635] Updated weights for policy 1, policy_version 11862 (0.0008) [2023-10-11 19:34:08,912][71635] Updated weights for policy 1, policy_version 11872 (0.0008) [2023-10-11 19:34:11,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 24313856. Throughput: 0: 1820.8, 1: 1803.6. Samples: 6092002. Policy #0 lag: (min: 26.0, avg: 28.6, max: 58.0) [2023-10-11 19:34:11,034][70582] Avg episode reward: [(0, '15.770'), (1, '12.660')] [2023-10-11 19:34:11,128][71601] Updated weights for policy 0, policy_version 11880 (0.0008) [2023-10-11 19:34:11,504][71601] Updated weights for policy 0, policy_version 11890 (0.0010) [2023-10-11 19:34:11,877][71601] Updated weights for policy 0, policy_version 11900 (0.0007) [2023-10-11 19:34:12,464][71635] Updated weights for policy 1, policy_version 11882 (0.0010) [2023-10-11 19:34:12,825][71635] Updated weights for policy 1, policy_version 11892 (0.0009) [2023-10-11 19:34:13,187][71635] Updated weights for policy 1, policy_version 11902 (0.0009) [2023-10-11 19:34:15,557][71601] Updated weights for policy 0, policy_version 11910 (0.0008) [2023-10-11 19:34:15,925][71601] Updated weights for policy 0, policy_version 11920 (0.0008) [2023-10-11 19:34:16,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 24379392. Throughput: 0: 1818.2, 1: 1806.7. Samples: 6101808. Policy #0 lag: (min: 26.0, avg: 28.6, max: 58.0) [2023-10-11 19:34:16,034][70582] Avg episode reward: [(0, '16.830'), (1, '13.700')] [2023-10-11 19:34:16,300][71601] Updated weights for policy 0, policy_version 11930 (0.0007) [2023-10-11 19:34:16,517][71353] Saving new best policy, reward=16.830! [2023-10-11 19:34:17,037][71635] Updated weights for policy 1, policy_version 11912 (0.0008) [2023-10-11 19:34:17,399][71635] Updated weights for policy 1, policy_version 11922 (0.0008) [2023-10-11 19:34:17,768][71635] Updated weights for policy 1, policy_version 11932 (0.0008) [2023-10-11 19:34:20,105][71601] Updated weights for policy 0, policy_version 11940 (0.0009) [2023-10-11 19:34:20,481][71601] Updated weights for policy 0, policy_version 11950 (0.0007) [2023-10-11 19:34:20,854][71601] Updated weights for policy 0, policy_version 11960 (0.0007) [2023-10-11 19:34:21,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 24444928. Throughput: 0: 1820.5, 1: 1811.4. Samples: 6124696. Policy #0 lag: (min: 26.0, avg: 28.6, max: 58.0) [2023-10-11 19:34:21,035][70582] Avg episode reward: [(0, '14.990'), (1, '13.580')] [2023-10-11 19:34:21,401][71635] Updated weights for policy 1, policy_version 11942 (0.0008) [2023-10-11 19:34:21,766][71635] Updated weights for policy 1, policy_version 11952 (0.0009) [2023-10-11 19:34:22,135][71635] Updated weights for policy 1, policy_version 11962 (0.0007) [2023-10-11 19:34:24,638][71601] Updated weights for policy 0, policy_version 11970 (0.0008) [2023-10-11 19:34:25,008][71601] Updated weights for policy 0, policy_version 11980 (0.0009) [2023-10-11 19:34:25,385][71601] Updated weights for policy 0, policy_version 11990 (0.0007) [2023-10-11 19:34:25,762][71601] Updated weights for policy 0, policy_version 12000 (0.0008) [2023-10-11 19:34:25,936][71635] Updated weights for policy 1, policy_version 11972 (0.0009) [2023-10-11 19:34:26,034][70582] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 24543232. Throughput: 0: 1812.0, 1: 1811.9. Samples: 6146186. Policy #0 lag: (min: 27.0, avg: 38.4, max: 59.0) [2023-10-11 19:34:26,035][70582] Avg episode reward: [(0, '14.320'), (1, '14.260')] [2023-10-11 19:34:26,328][71635] Updated weights for policy 1, policy_version 11982 (0.0010) [2023-10-11 19:34:26,696][71635] Updated weights for policy 1, policy_version 11992 (0.0010) [2023-10-11 19:34:29,409][71601] Updated weights for policy 0, policy_version 12010 (0.0008) [2023-10-11 19:34:29,780][71601] Updated weights for policy 0, policy_version 12020 (0.0009) [2023-10-11 19:34:30,152][71601] Updated weights for policy 0, policy_version 12030 (0.0007) [2023-10-11 19:34:30,215][71635] Updated weights for policy 1, policy_version 12002 (0.0009) [2023-10-11 19:34:30,583][71635] Updated weights for policy 1, policy_version 12012 (0.0010) [2023-10-11 19:34:30,938][71635] Updated weights for policy 1, policy_version 12022 (0.0009) [2023-10-11 19:34:31,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 24608768. Throughput: 0: 1818.9, 1: 1817.0. Samples: 6157284. Policy #0 lag: (min: 27.0, avg: 38.4, max: 59.0) [2023-10-11 19:34:31,034][70582] Avg episode reward: [(0, '13.000'), (1, '14.380')] [2023-10-11 19:34:31,302][71635] Updated weights for policy 1, policy_version 12032 (0.0008) [2023-10-11 19:34:33,761][71601] Updated weights for policy 0, policy_version 12040 (0.0010) [2023-10-11 19:34:34,136][71601] Updated weights for policy 0, policy_version 12050 (0.0008) [2023-10-11 19:34:34,512][71601] Updated weights for policy 0, policy_version 12060 (0.0008) [2023-10-11 19:34:35,064][71635] Updated weights for policy 1, policy_version 12042 (0.0009) [2023-10-11 19:34:35,432][71635] Updated weights for policy 1, policy_version 12052 (0.0008) [2023-10-11 19:34:35,804][71635] Updated weights for policy 1, policy_version 12062 (0.0009) [2023-10-11 19:34:36,034][70582] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 24707072. Throughput: 0: 1819.2, 1: 1819.4. Samples: 6178980. Policy #0 lag: (min: 13.0, avg: 14.2, max: 37.0) [2023-10-11 19:34:36,035][70582] Avg episode reward: [(0, '12.160'), (1, '14.970')] [2023-10-11 19:34:38,290][71601] Updated weights for policy 0, policy_version 12070 (0.0007) [2023-10-11 19:34:38,662][71601] Updated weights for policy 0, policy_version 12080 (0.0011) [2023-10-11 19:34:39,039][71601] Updated weights for policy 0, policy_version 12090 (0.0009) [2023-10-11 19:34:39,569][71635] Updated weights for policy 1, policy_version 12072 (0.0007) [2023-10-11 19:34:39,938][71635] Updated weights for policy 1, policy_version 12082 (0.0007) [2023-10-11 19:34:40,293][71635] Updated weights for policy 1, policy_version 12092 (0.0009) [2023-10-11 19:34:41,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 24772608. Throughput: 0: 1820.7, 1: 1819.3. Samples: 6199870. Policy #0 lag: (min: 13.0, avg: 14.2, max: 37.0) [2023-10-11 19:34:41,034][70582] Avg episode reward: [(0, '11.960'), (1, '14.020')] [2023-10-11 19:34:42,645][71601] Updated weights for policy 0, policy_version 12100 (0.0010) [2023-10-11 19:34:43,021][71601] Updated weights for policy 0, policy_version 12110 (0.0007) [2023-10-11 19:34:43,383][71601] Updated weights for policy 0, policy_version 12120 (0.0007) [2023-10-11 19:34:44,007][71635] Updated weights for policy 1, policy_version 12102 (0.0009) [2023-10-11 19:34:44,379][71635] Updated weights for policy 1, policy_version 12112 (0.0009) [2023-10-11 19:34:44,752][71635] Updated weights for policy 1, policy_version 12122 (0.0011) [2023-10-11 19:34:46,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 24838144. Throughput: 0: 1823.4, 1: 1815.7. Samples: 6211644. Policy #0 lag: (min: 13.0, avg: 14.2, max: 37.0) [2023-10-11 19:34:46,034][70582] Avg episode reward: [(0, '12.120'), (1, '12.430')] [2023-10-11 19:34:46,986][71601] Updated weights for policy 0, policy_version 12130 (0.0007) [2023-10-11 19:34:47,363][71601] Updated weights for policy 0, policy_version 12140 (0.0008) [2023-10-11 19:34:47,739][71601] Updated weights for policy 0, policy_version 12150 (0.0008) [2023-10-11 19:34:48,106][71601] Updated weights for policy 0, policy_version 12160 (0.0009) [2023-10-11 19:34:48,545][71635] Updated weights for policy 1, policy_version 12132 (0.0009) [2023-10-11 19:34:48,904][71635] Updated weights for policy 1, policy_version 12142 (0.0008) [2023-10-11 19:34:49,278][71635] Updated weights for policy 1, policy_version 12152 (0.0008) [2023-10-11 19:34:51,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 24903680. Throughput: 0: 1820.5, 1: 1812.8. Samples: 6232782. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-11 19:34:51,035][70582] Avg episode reward: [(0, '13.240'), (1, '12.950')] [2023-10-11 19:34:51,920][71601] Updated weights for policy 0, policy_version 12170 (0.0008) [2023-10-11 19:34:52,293][71601] Updated weights for policy 0, policy_version 12180 (0.0008) [2023-10-11 19:34:52,664][71601] Updated weights for policy 0, policy_version 12190 (0.0008) [2023-10-11 19:34:52,938][71635] Updated weights for policy 1, policy_version 12162 (0.0009) [2023-10-11 19:34:53,294][71635] Updated weights for policy 1, policy_version 12172 (0.0008) [2023-10-11 19:34:53,668][71635] Updated weights for policy 1, policy_version 12182 (0.0009) [2023-10-11 19:34:54,040][71635] Updated weights for policy 1, policy_version 12192 (0.0008) [2023-10-11 19:34:56,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 24969216. Throughput: 0: 1821.2, 1: 1808.5. Samples: 6255340. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-11 19:34:56,034][70582] Avg episode reward: [(0, '13.200'), (1, '11.580')] [2023-10-11 19:34:56,192][71601] Updated weights for policy 0, policy_version 12200 (0.0009) [2023-10-11 19:34:56,571][71601] Updated weights for policy 0, policy_version 12210 (0.0007) [2023-10-11 19:34:56,950][71601] Updated weights for policy 0, policy_version 12220 (0.0009) [2023-10-11 19:34:57,778][71635] Updated weights for policy 1, policy_version 12202 (0.0007) [2023-10-11 19:34:58,140][71635] Updated weights for policy 1, policy_version 12212 (0.0009) [2023-10-11 19:34:58,515][71635] Updated weights for policy 1, policy_version 12222 (0.0008) [2023-10-11 19:35:00,619][71601] Updated weights for policy 0, policy_version 12230 (0.0009) [2023-10-11 19:35:00,989][71601] Updated weights for policy 0, policy_version 12240 (0.0010) [2023-10-11 19:35:01,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 25034752. Throughput: 0: 1823.4, 1: 1816.8. Samples: 6265618. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-11 19:35:01,034][70582] Avg episode reward: [(0, '14.840'), (1, '12.350')] [2023-10-11 19:35:01,362][71601] Updated weights for policy 0, policy_version 12250 (0.0010) [2023-10-11 19:35:02,292][71635] Updated weights for policy 1, policy_version 12232 (0.0008) [2023-10-11 19:35:02,655][71635] Updated weights for policy 1, policy_version 12242 (0.0011) [2023-10-11 19:35:03,022][71635] Updated weights for policy 1, policy_version 12252 (0.0008) [2023-10-11 19:35:05,043][71601] Updated weights for policy 0, policy_version 12260 (0.0009) [2023-10-11 19:35:05,415][71601] Updated weights for policy 0, policy_version 12270 (0.0008) [2023-10-11 19:35:05,786][71601] Updated weights for policy 0, policy_version 12280 (0.0007) [2023-10-11 19:35:06,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 25100288. Throughput: 0: 1820.8, 1: 1809.4. Samples: 6288056. Policy #0 lag: (min: 9.0, avg: 36.8, max: 40.0) [2023-10-11 19:35:06,034][70582] Avg episode reward: [(0, '14.270'), (1, '13.330')] [2023-10-11 19:35:06,683][71635] Updated weights for policy 1, policy_version 12262 (0.0007) [2023-10-11 19:35:07,048][71635] Updated weights for policy 1, policy_version 12272 (0.0008) [2023-10-11 19:35:07,408][71635] Updated weights for policy 1, policy_version 12282 (0.0008) [2023-10-11 19:35:09,590][71601] Updated weights for policy 0, policy_version 12290 (0.0010) [2023-10-11 19:35:09,953][71601] Updated weights for policy 0, policy_version 12300 (0.0009) [2023-10-11 19:35:10,333][71601] Updated weights for policy 0, policy_version 12310 (0.0008) [2023-10-11 19:35:10,703][71601] Updated weights for policy 0, policy_version 12320 (0.0007) [2023-10-11 19:35:11,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 25198592. Throughput: 0: 1819.5, 1: 1814.9. Samples: 6309732. Policy #0 lag: (min: 9.0, avg: 36.8, max: 40.0) [2023-10-11 19:35:11,034][70582] Avg episode reward: [(0, '13.090'), (1, '13.530')] [2023-10-11 19:35:11,092][71635] Updated weights for policy 1, policy_version 12292 (0.0009) [2023-10-11 19:35:11,482][71635] Updated weights for policy 1, policy_version 12302 (0.0009) [2023-10-11 19:35:11,851][71635] Updated weights for policy 1, policy_version 12312 (0.0009) [2023-10-11 19:35:14,316][71601] Updated weights for policy 0, policy_version 12330 (0.0011) [2023-10-11 19:35:14,692][71601] Updated weights for policy 0, policy_version 12340 (0.0009) [2023-10-11 19:35:15,066][71601] Updated weights for policy 0, policy_version 12350 (0.0009) [2023-10-11 19:35:15,580][71635] Updated weights for policy 1, policy_version 12322 (0.0008) [2023-10-11 19:35:15,954][71635] Updated weights for policy 1, policy_version 12332 (0.0011) [2023-10-11 19:35:16,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 25264128. Throughput: 0: 1818.3, 1: 1812.5. Samples: 6320674. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-11 19:35:16,035][70582] Avg episode reward: [(0, '13.510'), (1, '13.700')] [2023-10-11 19:35:16,330][71635] Updated weights for policy 1, policy_version 12342 (0.0009) [2023-10-11 19:35:16,697][71635] Updated weights for policy 1, policy_version 12352 (0.0008) [2023-10-11 19:35:18,700][71601] Updated weights for policy 0, policy_version 12360 (0.0008) [2023-10-11 19:35:19,069][71601] Updated weights for policy 0, policy_version 12370 (0.0010) [2023-10-11 19:35:19,442][71601] Updated weights for policy 0, policy_version 12380 (0.0010) [2023-10-11 19:35:20,492][71635] Updated weights for policy 1, policy_version 12362 (0.0010) [2023-10-11 19:35:20,870][71635] Updated weights for policy 1, policy_version 12372 (0.0010) [2023-10-11 19:35:21,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 25329664. Throughput: 0: 1816.7, 1: 1809.3. Samples: 6342152. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-11 19:35:21,035][70582] Avg episode reward: [(0, '11.070'), (1, '14.710')] [2023-10-11 19:35:21,238][71635] Updated weights for policy 1, policy_version 12382 (0.0008) [2023-10-11 19:35:23,144][71601] Updated weights for policy 0, policy_version 12390 (0.0009) [2023-10-11 19:35:23,515][71601] Updated weights for policy 0, policy_version 12400 (0.0009) [2023-10-11 19:35:23,883][71601] Updated weights for policy 0, policy_version 12410 (0.0008) [2023-10-11 19:35:24,884][71635] Updated weights for policy 1, policy_version 12392 (0.0007) [2023-10-11 19:35:25,249][71635] Updated weights for policy 1, policy_version 12402 (0.0008) [2023-10-11 19:35:25,616][71635] Updated weights for policy 1, policy_version 12412 (0.0007) [2023-10-11 19:35:26,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 25427968. Throughput: 0: 1814.9, 1: 1820.6. Samples: 6363468. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-11 19:35:26,034][70582] Avg episode reward: [(0, '12.130'), (1, '14.120')] [2023-10-11 19:35:27,776][71601] Updated weights for policy 0, policy_version 12420 (0.0009) [2023-10-11 19:35:28,152][71601] Updated weights for policy 0, policy_version 12430 (0.0009) [2023-10-11 19:35:28,524][71601] Updated weights for policy 0, policy_version 12440 (0.0007) [2023-10-11 19:35:29,259][71635] Updated weights for policy 1, policy_version 12422 (0.0008) [2023-10-11 19:35:29,635][71635] Updated weights for policy 1, policy_version 12432 (0.0010) [2023-10-11 19:35:30,001][71635] Updated weights for policy 1, policy_version 12442 (0.0010) [2023-10-11 19:35:31,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 25493504. Throughput: 0: 1811.1, 1: 1810.8. Samples: 6374630. Policy #0 lag: (min: 8.0, avg: 29.7, max: 40.0) [2023-10-11 19:35:31,034][70582] Avg episode reward: [(0, '11.750'), (1, '14.480')] [2023-10-11 19:35:32,376][71601] Updated weights for policy 0, policy_version 12450 (0.0009) [2023-10-11 19:35:32,746][71601] Updated weights for policy 0, policy_version 12460 (0.0007) [2023-10-11 19:35:33,129][71601] Updated weights for policy 0, policy_version 12470 (0.0009) [2023-10-11 19:35:33,492][71601] Updated weights for policy 0, policy_version 12480 (0.0007) [2023-10-11 19:35:33,669][71635] Updated weights for policy 1, policy_version 12452 (0.0008) [2023-10-11 19:35:34,027][71635] Updated weights for policy 1, policy_version 12462 (0.0008) [2023-10-11 19:35:34,394][71635] Updated weights for policy 1, policy_version 12472 (0.0010) [2023-10-11 19:35:36,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 25559040. Throughput: 0: 1806.0, 1: 1818.7. Samples: 6395896. Policy #0 lag: (min: 8.0, avg: 29.7, max: 40.0) [2023-10-11 19:35:36,035][70582] Avg episode reward: [(0, '12.630'), (1, '13.790')] [2023-10-11 19:35:37,210][71601] Updated weights for policy 0, policy_version 12490 (0.0008) [2023-10-11 19:35:37,588][71601] Updated weights for policy 0, policy_version 12500 (0.0008) [2023-10-11 19:35:37,968][71601] Updated weights for policy 0, policy_version 12510 (0.0009) [2023-10-11 19:35:38,118][71635] Updated weights for policy 1, policy_version 12482 (0.0010) [2023-10-11 19:35:38,503][71635] Updated weights for policy 1, policy_version 12492 (0.0009) [2023-10-11 19:35:38,856][71635] Updated weights for policy 1, policy_version 12502 (0.0008) [2023-10-11 19:35:39,228][71635] Updated weights for policy 1, policy_version 12512 (0.0008) [2023-10-11 19:35:41,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 25624576. Throughput: 0: 1801.1, 1: 1817.5. Samples: 6418176. Policy #0 lag: (min: 8.0, avg: 29.7, max: 40.0) [2023-10-11 19:35:41,035][70582] Avg episode reward: [(0, '13.420'), (1, '14.090')] [2023-10-11 19:35:41,678][71601] Updated weights for policy 0, policy_version 12520 (0.0008) [2023-10-11 19:35:42,050][71601] Updated weights for policy 0, policy_version 12530 (0.0008) [2023-10-11 19:35:42,431][71601] Updated weights for policy 0, policy_version 12540 (0.0009) [2023-10-11 19:35:42,904][71635] Updated weights for policy 1, policy_version 12522 (0.0008) [2023-10-11 19:35:43,279][71635] Updated weights for policy 1, policy_version 12532 (0.0010) [2023-10-11 19:35:43,642][71635] Updated weights for policy 1, policy_version 12542 (0.0010) [2023-10-11 19:35:46,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 25690112. Throughput: 0: 1801.3, 1: 1820.3. Samples: 6428592. Policy #0 lag: (min: 17.0, avg: 29.7, max: 49.0) [2023-10-11 19:35:46,034][70582] Avg episode reward: [(0, '13.180'), (1, '12.900')] [2023-10-11 19:35:46,181][71601] Updated weights for policy 0, policy_version 12550 (0.0009) [2023-10-11 19:35:46,548][71601] Updated weights for policy 0, policy_version 12560 (0.0008) [2023-10-11 19:35:46,925][71601] Updated weights for policy 0, policy_version 12570 (0.0007) [2023-10-11 19:35:47,452][71635] Updated weights for policy 1, policy_version 12552 (0.0009) [2023-10-11 19:35:47,826][71635] Updated weights for policy 1, policy_version 12562 (0.0008) [2023-10-11 19:35:48,188][71635] Updated weights for policy 1, policy_version 12572 (0.0007) [2023-10-11 19:35:50,468][71601] Updated weights for policy 0, policy_version 12580 (0.0008) [2023-10-11 19:35:50,835][71601] Updated weights for policy 0, policy_version 12590 (0.0007) [2023-10-11 19:35:51,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 25755648. Throughput: 0: 1801.9, 1: 1811.1. Samples: 6450638. Policy #0 lag: (min: 17.0, avg: 29.7, max: 49.0) [2023-10-11 19:35:51,035][70582] Avg episode reward: [(0, '13.480'), (1, '13.480')] [2023-10-11 19:35:51,209][71601] Updated weights for policy 0, policy_version 12600 (0.0007) [2023-10-11 19:35:51,886][71635] Updated weights for policy 1, policy_version 12582 (0.0009) [2023-10-11 19:35:52,256][71635] Updated weights for policy 1, policy_version 12592 (0.0007) [2023-10-11 19:35:52,623][71635] Updated weights for policy 1, policy_version 12602 (0.0009) [2023-10-11 19:35:54,927][71601] Updated weights for policy 0, policy_version 12610 (0.0008) [2023-10-11 19:35:55,293][71601] Updated weights for policy 0, policy_version 12620 (0.0007) [2023-10-11 19:35:55,677][71601] Updated weights for policy 0, policy_version 12630 (0.0008) [2023-10-11 19:35:56,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 25821184. Throughput: 0: 1814.8, 1: 1807.7. Samples: 6472748. Policy #0 lag: (min: 17.0, avg: 29.7, max: 49.0) [2023-10-11 19:35:56,034][70582] Avg episode reward: [(0, '13.820'), (1, '11.760')] [2023-10-11 19:35:56,042][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000012640_12943360.pth... [2023-10-11 19:35:56,047][71601] Updated weights for policy 0, policy_version 12640 (0.0007) [2023-10-11 19:35:56,071][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000010944_11206656.pth [2023-10-11 19:35:56,322][71635] Updated weights for policy 1, policy_version 12612 (0.0008) [2023-10-11 19:35:56,718][71635] Updated weights for policy 1, policy_version 12622 (0.0008) [2023-10-11 19:35:57,084][71635] Updated weights for policy 1, policy_version 12632 (0.0008) [2023-10-11 19:35:57,369][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000012640_12943360.pth... [2023-10-11 19:35:57,409][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000010912_11173888.pth [2023-10-11 19:35:59,754][71601] Updated weights for policy 0, policy_version 12650 (0.0008) [2023-10-11 19:36:00,123][71601] Updated weights for policy 0, policy_version 12660 (0.0008) [2023-10-11 19:36:00,496][71601] Updated weights for policy 0, policy_version 12670 (0.0011) [2023-10-11 19:36:00,907][71635] Updated weights for policy 1, policy_version 12642 (0.0009) [2023-10-11 19:36:01,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 25919488. Throughput: 0: 1807.8, 1: 1809.2. Samples: 6483438. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-11 19:36:01,034][70582] Avg episode reward: [(0, '12.830'), (1, '11.900')] [2023-10-11 19:36:01,280][71635] Updated weights for policy 1, policy_version 12652 (0.0009) [2023-10-11 19:36:01,642][71635] Updated weights for policy 1, policy_version 12662 (0.0010) [2023-10-11 19:36:02,008][71635] Updated weights for policy 1, policy_version 12672 (0.0010) [2023-10-11 19:36:04,059][71601] Updated weights for policy 0, policy_version 12680 (0.0010) [2023-10-11 19:36:04,421][71601] Updated weights for policy 0, policy_version 12690 (0.0008) [2023-10-11 19:36:04,794][71601] Updated weights for policy 0, policy_version 12700 (0.0008) [2023-10-11 19:36:05,665][71635] Updated weights for policy 1, policy_version 12682 (0.0008) [2023-10-11 19:36:06,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 25985024. Throughput: 0: 1819.3, 1: 1808.0. Samples: 6505376. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-11 19:36:06,034][70582] Avg episode reward: [(0, '12.490'), (1, '11.990')] [2023-10-11 19:36:06,042][71635] Updated weights for policy 1, policy_version 12692 (0.0010) [2023-10-11 19:36:06,398][71635] Updated weights for policy 1, policy_version 12702 (0.0008) [2023-10-11 19:36:08,492][71601] Updated weights for policy 0, policy_version 12710 (0.0009) [2023-10-11 19:36:08,857][71601] Updated weights for policy 0, policy_version 12720 (0.0008) [2023-10-11 19:36:09,236][71601] Updated weights for policy 0, policy_version 12730 (0.0009) [2023-10-11 19:36:10,135][71635] Updated weights for policy 1, policy_version 12712 (0.0009) [2023-10-11 19:36:10,506][71635] Updated weights for policy 1, policy_version 12722 (0.0007) [2023-10-11 19:36:10,866][71635] Updated weights for policy 1, policy_version 12732 (0.0008) [2023-10-11 19:36:11,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 26083328. Throughput: 0: 1813.6, 1: 1815.8. Samples: 6526794. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:36:11,035][70582] Avg episode reward: [(0, '11.940'), (1, '11.290')] [2023-10-11 19:36:12,978][71601] Updated weights for policy 0, policy_version 12740 (0.0010) [2023-10-11 19:36:13,348][71601] Updated weights for policy 0, policy_version 12750 (0.0010) [2023-10-11 19:36:13,722][71601] Updated weights for policy 0, policy_version 12760 (0.0010) [2023-10-11 19:36:14,507][71635] Updated weights for policy 1, policy_version 12742 (0.0008) [2023-10-11 19:36:14,883][71635] Updated weights for policy 1, policy_version 12752 (0.0008) [2023-10-11 19:36:15,247][71635] Updated weights for policy 1, policy_version 12762 (0.0010) [2023-10-11 19:36:16,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 26148864. Throughput: 0: 1820.8, 1: 1808.3. Samples: 6537936. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:36:16,034][70582] Avg episode reward: [(0, '11.820'), (1, '11.270')] [2023-10-11 19:36:17,585][71601] Updated weights for policy 0, policy_version 12770 (0.0010) [2023-10-11 19:36:17,953][71601] Updated weights for policy 0, policy_version 12780 (0.0008) [2023-10-11 19:36:18,326][71601] Updated weights for policy 0, policy_version 12790 (0.0007) [2023-10-11 19:36:18,696][71601] Updated weights for policy 0, policy_version 12800 (0.0007) [2023-10-11 19:36:18,936][71635] Updated weights for policy 1, policy_version 12772 (0.0010) [2023-10-11 19:36:19,299][71635] Updated weights for policy 1, policy_version 12782 (0.0008) [2023-10-11 19:36:19,659][71635] Updated weights for policy 1, policy_version 12792 (0.0009) [2023-10-11 19:36:21,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 26214400. Throughput: 0: 1812.5, 1: 1815.0. Samples: 6559134. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:36:21,035][70582] Avg episode reward: [(0, '12.380'), (1, '13.130')] [2023-10-11 19:36:22,531][71601] Updated weights for policy 0, policy_version 12810 (0.0007) [2023-10-11 19:36:22,921][71601] Updated weights for policy 0, policy_version 12820 (0.0008) [2023-10-11 19:36:23,286][71601] Updated weights for policy 0, policy_version 12830 (0.0008) [2023-10-11 19:36:23,366][71635] Updated weights for policy 1, policy_version 12802 (0.0009) [2023-10-11 19:36:23,735][71635] Updated weights for policy 1, policy_version 12812 (0.0009) [2023-10-11 19:36:24,098][71635] Updated weights for policy 1, policy_version 12822 (0.0008) [2023-10-11 19:36:24,477][71635] Updated weights for policy 1, policy_version 12832 (0.0009) [2023-10-11 19:36:26,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 26279936. Throughput: 0: 1810.1, 1: 1806.3. Samples: 6580914. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-10-11 19:36:26,034][70582] Avg episode reward: [(0, '13.440'), (1, '14.490')] [2023-10-11 19:36:26,959][71601] Updated weights for policy 0, policy_version 12840 (0.0007) [2023-10-11 19:36:27,326][71601] Updated weights for policy 0, policy_version 12850 (0.0008) [2023-10-11 19:36:27,697][71601] Updated weights for policy 0, policy_version 12860 (0.0011) [2023-10-11 19:36:28,156][71635] Updated weights for policy 1, policy_version 12842 (0.0009) [2023-10-11 19:36:28,516][71635] Updated weights for policy 1, policy_version 12852 (0.0007) [2023-10-11 19:36:28,881][71635] Updated weights for policy 1, policy_version 12862 (0.0008) [2023-10-11 19:36:31,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 26345472. Throughput: 0: 1810.0, 1: 1808.4. Samples: 6591418. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-10-11 19:36:31,034][70582] Avg episode reward: [(0, '13.140'), (1, '15.350')] [2023-10-11 19:36:31,306][71601] Updated weights for policy 0, policy_version 12870 (0.0009) [2023-10-11 19:36:31,676][71601] Updated weights for policy 0, policy_version 12880 (0.0007) [2023-10-11 19:36:32,050][71601] Updated weights for policy 0, policy_version 12890 (0.0007) [2023-10-11 19:36:32,598][71635] Updated weights for policy 1, policy_version 12872 (0.0008) [2023-10-11 19:36:32,956][71635] Updated weights for policy 1, policy_version 12882 (0.0009) [2023-10-11 19:36:33,327][71635] Updated weights for policy 1, policy_version 12892 (0.0010) [2023-10-11 19:36:35,709][71601] Updated weights for policy 0, policy_version 12900 (0.0009) [2023-10-11 19:36:36,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 26411008. Throughput: 0: 1807.4, 1: 1806.4. Samples: 6613260. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-10-11 19:36:36,035][70582] Avg episode reward: [(0, '12.250'), (1, '17.050')] [2023-10-11 19:36:36,078][71601] Updated weights for policy 0, policy_version 12910 (0.0007) [2023-10-11 19:36:36,458][71601] Updated weights for policy 0, policy_version 12920 (0.0009) [2023-10-11 19:36:36,964][71635] Updated weights for policy 1, policy_version 12902 (0.0008) [2023-10-11 19:36:37,331][71635] Updated weights for policy 1, policy_version 12912 (0.0007) [2023-10-11 19:36:37,701][71635] Updated weights for policy 1, policy_version 12922 (0.0008) [2023-10-11 19:36:40,089][71601] Updated weights for policy 0, policy_version 12930 (0.0008) [2023-10-11 19:36:40,452][71601] Updated weights for policy 0, policy_version 12940 (0.0011) [2023-10-11 19:36:40,818][71601] Updated weights for policy 0, policy_version 12950 (0.0009) [2023-10-11 19:36:41,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 26476544. Throughput: 0: 1813.0, 1: 1812.6. Samples: 6635900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:36:41,034][70582] Avg episode reward: [(0, '12.390'), (1, '16.080')] [2023-10-11 19:36:41,191][71601] Updated weights for policy 0, policy_version 12960 (0.0008) [2023-10-11 19:36:41,446][71635] Updated weights for policy 1, policy_version 12932 (0.0008) [2023-10-11 19:36:41,845][71635] Updated weights for policy 1, policy_version 12942 (0.0009) [2023-10-11 19:36:42,208][71635] Updated weights for policy 1, policy_version 12952 (0.0008) [2023-10-11 19:36:44,957][71601] Updated weights for policy 0, policy_version 12970 (0.0007) [2023-10-11 19:36:45,323][71601] Updated weights for policy 0, policy_version 12980 (0.0007) [2023-10-11 19:36:45,696][71601] Updated weights for policy 0, policy_version 12990 (0.0007) [2023-10-11 19:36:45,908][71635] Updated weights for policy 1, policy_version 12962 (0.0009) [2023-10-11 19:36:46,034][70582] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 26574848. Throughput: 0: 1806.3, 1: 1811.6. Samples: 6646244. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:36:46,034][70582] Avg episode reward: [(0, '11.150'), (1, '15.550')] [2023-10-11 19:36:46,265][71635] Updated weights for policy 1, policy_version 12972 (0.0009) [2023-10-11 19:36:46,634][71635] Updated weights for policy 1, policy_version 12982 (0.0009) [2023-10-11 19:36:46,995][71635] Updated weights for policy 1, policy_version 12992 (0.0009) [2023-10-11 19:36:49,329][71601] Updated weights for policy 0, policy_version 13000 (0.0008) [2023-10-11 19:36:49,696][71601] Updated weights for policy 0, policy_version 13010 (0.0008) [2023-10-11 19:36:50,064][71601] Updated weights for policy 0, policy_version 13020 (0.0007) [2023-10-11 19:36:50,859][71635] Updated weights for policy 1, policy_version 13002 (0.0009) [2023-10-11 19:36:51,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 26640384. Throughput: 0: 1809.2, 1: 1806.0. Samples: 6668060. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:36:51,034][70582] Avg episode reward: [(0, '11.120'), (1, '15.620')] [2023-10-11 19:36:51,219][71635] Updated weights for policy 1, policy_version 13012 (0.0007) [2023-10-11 19:36:51,584][71635] Updated weights for policy 1, policy_version 13022 (0.0007) [2023-10-11 19:36:53,730][71601] Updated weights for policy 0, policy_version 13030 (0.0009) [2023-10-11 19:36:54,098][71601] Updated weights for policy 0, policy_version 13040 (0.0009) [2023-10-11 19:36:54,464][71601] Updated weights for policy 0, policy_version 13050 (0.0009) [2023-10-11 19:36:55,229][71635] Updated weights for policy 1, policy_version 13032 (0.0010) [2023-10-11 19:36:55,597][71635] Updated weights for policy 1, policy_version 13042 (0.0008) [2023-10-11 19:36:55,968][71635] Updated weights for policy 1, policy_version 13052 (0.0009) [2023-10-11 19:36:56,034][70582] Fps is (10 sec: 13106.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 26705920. Throughput: 0: 1810.6, 1: 1811.1. Samples: 6689772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:36:56,035][70582] Avg episode reward: [(0, '11.340'), (1, '13.860')] [2023-10-11 19:36:58,191][71601] Updated weights for policy 0, policy_version 13060 (0.0007) [2023-10-11 19:36:58,566][71601] Updated weights for policy 0, policy_version 13070 (0.0008) [2023-10-11 19:36:58,931][71601] Updated weights for policy 0, policy_version 13080 (0.0008) [2023-10-11 19:36:59,799][71635] Updated weights for policy 1, policy_version 13062 (0.0009) [2023-10-11 19:37:00,158][71635] Updated weights for policy 1, policy_version 13072 (0.0008) [2023-10-11 19:37:00,533][71635] Updated weights for policy 1, policy_version 13082 (0.0009) [2023-10-11 19:37:01,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 26804224. Throughput: 0: 1817.3, 1: 1805.0. Samples: 6700938. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:37:01,034][70582] Avg episode reward: [(0, '10.950'), (1, '12.440')] [2023-10-11 19:37:02,645][71601] Updated weights for policy 0, policy_version 13090 (0.0009) [2023-10-11 19:37:03,007][71601] Updated weights for policy 0, policy_version 13100 (0.0009) [2023-10-11 19:37:03,383][71601] Updated weights for policy 0, policy_version 13110 (0.0009) [2023-10-11 19:37:03,750][71601] Updated weights for policy 0, policy_version 13120 (0.0007) [2023-10-11 19:37:04,191][71635] Updated weights for policy 1, policy_version 13092 (0.0009) [2023-10-11 19:37:04,554][71635] Updated weights for policy 1, policy_version 13102 (0.0009) [2023-10-11 19:37:04,927][71635] Updated weights for policy 1, policy_version 13112 (0.0011) [2023-10-11 19:37:06,034][70582] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 26869760. Throughput: 0: 1811.5, 1: 1810.6. Samples: 6722128. Policy #0 lag: (min: 31.0, avg: 32.0, max: 52.0) [2023-10-11 19:37:06,034][70582] Avg episode reward: [(0, '12.390'), (1, '12.140')] [2023-10-11 19:37:07,426][71601] Updated weights for policy 0, policy_version 13130 (0.0009) [2023-10-11 19:37:07,797][71601] Updated weights for policy 0, policy_version 13140 (0.0007) [2023-10-11 19:37:08,177][71601] Updated weights for policy 0, policy_version 13150 (0.0007) [2023-10-11 19:37:08,614][71635] Updated weights for policy 1, policy_version 13122 (0.0009) [2023-10-11 19:37:08,979][71635] Updated weights for policy 1, policy_version 13132 (0.0008) [2023-10-11 19:37:09,343][71635] Updated weights for policy 1, policy_version 13142 (0.0009) [2023-10-11 19:37:09,709][71635] Updated weights for policy 1, policy_version 13152 (0.0008) [2023-10-11 19:37:11,034][70582] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 26935296. Throughput: 0: 1816.6, 1: 1804.5. Samples: 6743862. Policy #0 lag: (min: 31.0, avg: 32.0, max: 52.0) [2023-10-11 19:37:11,035][70582] Avg episode reward: [(0, '13.380'), (1, '11.360')] [2023-10-11 19:37:11,787][71601] Updated weights for policy 0, policy_version 13160 (0.0008) [2023-10-11 19:37:12,160][71601] Updated weights for policy 0, policy_version 13170 (0.0008) [2023-10-11 19:37:12,530][71601] Updated weights for policy 0, policy_version 13180 (0.0009) [2023-10-11 19:37:13,496][71635] Updated weights for policy 1, policy_version 13162 (0.0009) [2023-10-11 19:37:13,862][71635] Updated weights for policy 1, policy_version 13172 (0.0009) [2023-10-11 19:37:14,237][71635] Updated weights for policy 1, policy_version 13182 (0.0008) [2023-10-11 19:37:16,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 27000832. Throughput: 0: 1821.0, 1: 1819.7. Samples: 6755252. Policy #0 lag: (min: 31.0, avg: 32.0, max: 52.0) [2023-10-11 19:37:16,034][70582] Avg episode reward: [(0, '13.270'), (1, '11.590')] [2023-10-11 19:37:16,131][71601] Updated weights for policy 0, policy_version 13190 (0.0007) [2023-10-11 19:37:16,503][71601] Updated weights for policy 0, policy_version 13200 (0.0008) [2023-10-11 19:37:16,878][71601] Updated weights for policy 0, policy_version 13210 (0.0007) [2023-10-11 19:37:17,778][71635] Updated weights for policy 1, policy_version 13192 (0.0007) [2023-10-11 19:37:18,141][71635] Updated weights for policy 1, policy_version 13202 (0.0007) [2023-10-11 19:37:18,511][71635] Updated weights for policy 1, policy_version 13212 (0.0007) [2023-10-11 19:37:20,609][71601] Updated weights for policy 0, policy_version 13220 (0.0010) [2023-10-11 19:37:20,983][71601] Updated weights for policy 0, policy_version 13230 (0.0008) [2023-10-11 19:37:21,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 27066368. Throughput: 0: 1823.8, 1: 1814.4. Samples: 6776978. Policy #0 lag: (min: 11.0, avg: 11.3, max: 22.0) [2023-10-11 19:37:21,035][70582] Avg episode reward: [(0, '12.920'), (1, '12.790')] [2023-10-11 19:37:21,351][71601] Updated weights for policy 0, policy_version 13240 (0.0009) [2023-10-11 19:37:22,112][71635] Updated weights for policy 1, policy_version 13222 (0.0007) [2023-10-11 19:37:22,482][71635] Updated weights for policy 1, policy_version 13232 (0.0007) [2023-10-11 19:37:22,851][71635] Updated weights for policy 1, policy_version 13242 (0.0010) [2023-10-11 19:37:25,001][71601] Updated weights for policy 0, policy_version 13250 (0.0010) [2023-10-11 19:37:25,372][71601] Updated weights for policy 0, policy_version 13260 (0.0010) [2023-10-11 19:37:25,739][71601] Updated weights for policy 0, policy_version 13270 (0.0009) [2023-10-11 19:37:26,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 27131904. Throughput: 0: 1821.9, 1: 1813.4. Samples: 6799486. Policy #0 lag: (min: 11.0, avg: 11.3, max: 22.0) [2023-10-11 19:37:26,034][70582] Avg episode reward: [(0, '11.780'), (1, '14.590')] [2023-10-11 19:37:26,110][71601] Updated weights for policy 0, policy_version 13280 (0.0007) [2023-10-11 19:37:26,558][71635] Updated weights for policy 1, policy_version 13252 (0.0009) [2023-10-11 19:37:26,954][71635] Updated weights for policy 1, policy_version 13262 (0.0009) [2023-10-11 19:37:27,319][71635] Updated weights for policy 1, policy_version 13272 (0.0010) [2023-10-11 19:37:29,669][71601] Updated weights for policy 0, policy_version 13290 (0.0010) [2023-10-11 19:37:30,044][71601] Updated weights for policy 0, policy_version 13300 (0.0011) [2023-10-11 19:37:30,415][71601] Updated weights for policy 0, policy_version 13310 (0.0009) [2023-10-11 19:37:30,928][71635] Updated weights for policy 1, policy_version 13282 (0.0008) [2023-10-11 19:37:31,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 27230208. Throughput: 0: 1825.3, 1: 1815.9. Samples: 6810096. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-11 19:37:31,035][70582] Avg episode reward: [(0, '12.040'), (1, '14.260')] [2023-10-11 19:37:31,298][71635] Updated weights for policy 1, policy_version 13292 (0.0009) [2023-10-11 19:37:31,668][71635] Updated weights for policy 1, policy_version 13302 (0.0010) [2023-10-11 19:37:32,037][71635] Updated weights for policy 1, policy_version 13312 (0.0009) [2023-10-11 19:37:34,038][71601] Updated weights for policy 0, policy_version 13320 (0.0008) [2023-10-11 19:37:34,410][71601] Updated weights for policy 0, policy_version 13330 (0.0011) [2023-10-11 19:37:34,798][71601] Updated weights for policy 0, policy_version 13340 (0.0008) [2023-10-11 19:37:35,797][71635] Updated weights for policy 1, policy_version 13322 (0.0009) [2023-10-11 19:37:36,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 27295744. Throughput: 0: 1822.0, 1: 1818.1. Samples: 6831862. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-11 19:37:36,035][70582] Avg episode reward: [(0, '11.440'), (1, '15.270')] [2023-10-11 19:37:36,159][71635] Updated weights for policy 1, policy_version 13332 (0.0010) [2023-10-11 19:37:36,524][71635] Updated weights for policy 1, policy_version 13342 (0.0011) [2023-10-11 19:37:38,373][71601] Updated weights for policy 0, policy_version 13350 (0.0007) [2023-10-11 19:37:38,736][71601] Updated weights for policy 0, policy_version 13360 (0.0008) [2023-10-11 19:37:39,115][71601] Updated weights for policy 0, policy_version 13370 (0.0009) [2023-10-11 19:37:40,198][71635] Updated weights for policy 1, policy_version 13352 (0.0007) [2023-10-11 19:37:40,565][71635] Updated weights for policy 1, policy_version 13362 (0.0007) [2023-10-11 19:37:40,942][71635] Updated weights for policy 1, policy_version 13372 (0.0008) [2023-10-11 19:37:41,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 27361280. Throughput: 0: 1828.2, 1: 1821.3. Samples: 6853998. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-11 19:37:41,034][70582] Avg episode reward: [(0, '11.900'), (1, '14.450')] [2023-10-11 19:37:42,747][71601] Updated weights for policy 0, policy_version 13380 (0.0007) [2023-10-11 19:37:43,113][71601] Updated weights for policy 0, policy_version 13390 (0.0010) [2023-10-11 19:37:43,491][71601] Updated weights for policy 0, policy_version 13400 (0.0010) [2023-10-11 19:37:44,666][71635] Updated weights for policy 1, policy_version 13382 (0.0009) [2023-10-11 19:37:45,032][71635] Updated weights for policy 1, policy_version 13392 (0.0008) [2023-10-11 19:37:45,390][71635] Updated weights for policy 1, policy_version 13402 (0.0008) [2023-10-11 19:37:46,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 27459584. Throughput: 0: 1818.3, 1: 1826.4. Samples: 6864952. Policy #0 lag: (min: 20.0, avg: 20.8, max: 39.0) [2023-10-11 19:37:46,035][70582] Avg episode reward: [(0, '13.650'), (1, '14.400')] [2023-10-11 19:37:47,197][71601] Updated weights for policy 0, policy_version 13410 (0.0008) [2023-10-11 19:37:47,570][71601] Updated weights for policy 0, policy_version 13420 (0.0009) [2023-10-11 19:37:47,948][71601] Updated weights for policy 0, policy_version 13430 (0.0008) [2023-10-11 19:37:48,312][71601] Updated weights for policy 0, policy_version 13440 (0.0008) [2023-10-11 19:37:49,198][71635] Updated weights for policy 1, policy_version 13412 (0.0009) [2023-10-11 19:37:49,564][71635] Updated weights for policy 1, policy_version 13422 (0.0009) [2023-10-11 19:37:49,927][71635] Updated weights for policy 1, policy_version 13432 (0.0008) [2023-10-11 19:37:51,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 27525120. Throughput: 0: 1834.7, 1: 1823.2. Samples: 6886736. Policy #0 lag: (min: 20.0, avg: 20.8, max: 39.0) [2023-10-11 19:37:51,035][70582] Avg episode reward: [(0, '14.930'), (1, '14.870')] [2023-10-11 19:37:52,181][71601] Updated weights for policy 0, policy_version 13450 (0.0007) [2023-10-11 19:37:52,556][71601] Updated weights for policy 0, policy_version 13460 (0.0007) [2023-10-11 19:37:52,933][71601] Updated weights for policy 0, policy_version 13470 (0.0009) [2023-10-11 19:37:53,625][71635] Updated weights for policy 1, policy_version 13442 (0.0008) [2023-10-11 19:37:53,996][71635] Updated weights for policy 1, policy_version 13452 (0.0007) [2023-10-11 19:37:54,357][71635] Updated weights for policy 1, policy_version 13462 (0.0009) [2023-10-11 19:37:54,725][71635] Updated weights for policy 1, policy_version 13472 (0.0009) [2023-10-11 19:37:56,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 27590656. Throughput: 0: 1830.0, 1: 1824.0. Samples: 6908290. Policy #0 lag: (min: 20.0, avg: 20.8, max: 39.0) [2023-10-11 19:37:56,035][70582] Avg episode reward: [(0, '14.080'), (1, '15.150')] [2023-10-11 19:37:56,043][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000013472_13795328.pth... [2023-10-11 19:37:56,043][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000013472_13795328.pth... [2023-10-11 19:37:56,073][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000011776_12058624.pth [2023-10-11 19:37:56,081][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000011776_12058624.pth [2023-10-11 19:37:56,655][71601] Updated weights for policy 0, policy_version 13480 (0.0007) [2023-10-11 19:37:57,023][71601] Updated weights for policy 0, policy_version 13490 (0.0011) [2023-10-11 19:37:57,402][71601] Updated weights for policy 0, policy_version 13500 (0.0007) [2023-10-11 19:37:58,477][71635] Updated weights for policy 1, policy_version 13482 (0.0011) [2023-10-11 19:37:58,849][71635] Updated weights for policy 1, policy_version 13492 (0.0008) [2023-10-11 19:37:59,218][71635] Updated weights for policy 1, policy_version 13502 (0.0009) [2023-10-11 19:38:01,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 27656192. Throughput: 0: 1827.6, 1: 1820.8. Samples: 6919434. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:38:01,034][70582] Avg episode reward: [(0, '14.800'), (1, '13.960')] [2023-10-11 19:38:01,108][71601] Updated weights for policy 0, policy_version 13510 (0.0009) [2023-10-11 19:38:01,479][71601] Updated weights for policy 0, policy_version 13520 (0.0008) [2023-10-11 19:38:01,866][71601] Updated weights for policy 0, policy_version 13530 (0.0009) [2023-10-11 19:38:02,962][71635] Updated weights for policy 1, policy_version 13512 (0.0009) [2023-10-11 19:38:03,334][71635] Updated weights for policy 1, policy_version 13522 (0.0008) [2023-10-11 19:38:03,702][71635] Updated weights for policy 1, policy_version 13532 (0.0007) [2023-10-11 19:38:05,528][71601] Updated weights for policy 0, policy_version 13540 (0.0008) [2023-10-11 19:38:05,901][71601] Updated weights for policy 0, policy_version 13550 (0.0007) [2023-10-11 19:38:06,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 27721728. Throughput: 0: 1830.0, 1: 1815.4. Samples: 6941018. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:38:06,034][70582] Avg episode reward: [(0, '14.610'), (1, '14.410')] [2023-10-11 19:38:06,276][71601] Updated weights for policy 0, policy_version 13560 (0.0007) [2023-10-11 19:38:07,356][71635] Updated weights for policy 1, policy_version 13542 (0.0009) [2023-10-11 19:38:07,732][71635] Updated weights for policy 1, policy_version 13552 (0.0008) [2023-10-11 19:38:08,094][71635] Updated weights for policy 1, policy_version 13562 (0.0008) [2023-10-11 19:38:09,707][71601] Updated weights for policy 0, policy_version 13570 (0.0009) [2023-10-11 19:38:10,074][71601] Updated weights for policy 0, policy_version 13580 (0.0011) [2023-10-11 19:38:10,450][71601] Updated weights for policy 0, policy_version 13590 (0.0007) [2023-10-11 19:38:10,822][71601] Updated weights for policy 0, policy_version 13600 (0.0009) [2023-10-11 19:38:11,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 27820032. Throughput: 0: 1828.8, 1: 1809.1. Samples: 6963194. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:38:11,034][70582] Avg episode reward: [(0, '13.360'), (1, '13.220')] [2023-10-11 19:38:11,875][71635] Updated weights for policy 1, policy_version 13572 (0.0009) [2023-10-11 19:38:12,284][71635] Updated weights for policy 1, policy_version 13582 (0.0008) [2023-10-11 19:38:12,641][71635] Updated weights for policy 1, policy_version 13592 (0.0009) [2023-10-11 19:38:14,425][71601] Updated weights for policy 0, policy_version 13610 (0.0008) [2023-10-11 19:38:14,804][71601] Updated weights for policy 0, policy_version 13620 (0.0008) [2023-10-11 19:38:15,174][71601] Updated weights for policy 0, policy_version 13630 (0.0008) [2023-10-11 19:38:16,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 27885568. Throughput: 0: 1839.3, 1: 1805.4. Samples: 6974110. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:38:16,035][70582] Avg episode reward: [(0, '13.330'), (1, '12.180')] [2023-10-11 19:38:16,301][71635] Updated weights for policy 1, policy_version 13602 (0.0007) [2023-10-11 19:38:16,661][71635] Updated weights for policy 1, policy_version 13612 (0.0009) [2023-10-11 19:38:17,027][71635] Updated weights for policy 1, policy_version 13622 (0.0009) [2023-10-11 19:38:17,398][71635] Updated weights for policy 1, policy_version 13632 (0.0011) [2023-10-11 19:38:18,743][71601] Updated weights for policy 0, policy_version 13640 (0.0009) [2023-10-11 19:38:19,119][71601] Updated weights for policy 0, policy_version 13650 (0.0009) [2023-10-11 19:38:19,491][71601] Updated weights for policy 0, policy_version 13660 (0.0009) [2023-10-11 19:38:21,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 27951104. Throughput: 0: 1831.0, 1: 1816.8. Samples: 6996014. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:38:21,035][70582] Avg episode reward: [(0, '13.870'), (1, '12.800')] [2023-10-11 19:38:21,070][71635] Updated weights for policy 1, policy_version 13642 (0.0007) [2023-10-11 19:38:21,447][71635] Updated weights for policy 1, policy_version 13652 (0.0008) [2023-10-11 19:38:21,803][71635] Updated weights for policy 1, policy_version 13662 (0.0007) [2023-10-11 19:38:23,251][71601] Updated weights for policy 0, policy_version 13670 (0.0009) [2023-10-11 19:38:23,624][71601] Updated weights for policy 0, policy_version 13680 (0.0010) [2023-10-11 19:38:23,998][71601] Updated weights for policy 0, policy_version 13690 (0.0009) [2023-10-11 19:38:25,466][71635] Updated weights for policy 1, policy_version 13672 (0.0007) [2023-10-11 19:38:25,849][71635] Updated weights for policy 1, policy_version 13682 (0.0010) [2023-10-11 19:38:26,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 28016640. Throughput: 0: 1832.3, 1: 1814.9. Samples: 7018124. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-11 19:38:26,035][70582] Avg episode reward: [(0, '12.760'), (1, '13.270')] [2023-10-11 19:38:26,219][71635] Updated weights for policy 1, policy_version 13692 (0.0008) [2023-10-11 19:38:27,766][71601] Updated weights for policy 0, policy_version 13700 (0.0009) [2023-10-11 19:38:28,134][71601] Updated weights for policy 0, policy_version 13710 (0.0008) [2023-10-11 19:38:28,512][71601] Updated weights for policy 0, policy_version 13720 (0.0008) [2023-10-11 19:38:29,899][71635] Updated weights for policy 1, policy_version 13702 (0.0009) [2023-10-11 19:38:30,270][71635] Updated weights for policy 1, policy_version 13712 (0.0009) [2023-10-11 19:38:30,641][71635] Updated weights for policy 1, policy_version 13722 (0.0008) [2023-10-11 19:38:31,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 28114944. Throughput: 0: 1832.7, 1: 1808.7. Samples: 7028812. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-11 19:38:31,035][70582] Avg episode reward: [(0, '12.460'), (1, '12.780')] [2023-10-11 19:38:32,265][71601] Updated weights for policy 0, policy_version 13730 (0.0010) [2023-10-11 19:38:32,649][71601] Updated weights for policy 0, policy_version 13740 (0.0010) [2023-10-11 19:38:33,025][71601] Updated weights for policy 0, policy_version 13750 (0.0010) [2023-10-11 19:38:33,382][71601] Updated weights for policy 0, policy_version 13760 (0.0010) [2023-10-11 19:38:34,358][71635] Updated weights for policy 1, policy_version 13732 (0.0007) [2023-10-11 19:38:34,719][71635] Updated weights for policy 1, policy_version 13742 (0.0007) [2023-10-11 19:38:35,095][71635] Updated weights for policy 1, policy_version 13752 (0.0008) [2023-10-11 19:38:36,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 28180480. Throughput: 0: 1826.1, 1: 1817.6. Samples: 7050706. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-11 19:38:36,035][70582] Avg episode reward: [(0, '13.190'), (1, '13.330')] [2023-10-11 19:38:37,157][71601] Updated weights for policy 0, policy_version 13770 (0.0008) [2023-10-11 19:38:37,523][71601] Updated weights for policy 0, policy_version 13780 (0.0008) [2023-10-11 19:38:37,896][71601] Updated weights for policy 0, policy_version 13790 (0.0009) [2023-10-11 19:38:38,733][71635] Updated weights for policy 1, policy_version 13762 (0.0007) [2023-10-11 19:38:39,098][71635] Updated weights for policy 1, policy_version 13772 (0.0010) [2023-10-11 19:38:39,464][71635] Updated weights for policy 1, policy_version 13782 (0.0012) [2023-10-11 19:38:39,839][71635] Updated weights for policy 1, policy_version 13792 (0.0007) [2023-10-11 19:38:41,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 28246016. Throughput: 0: 1833.1, 1: 1812.4. Samples: 7072334. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:38:41,034][70582] Avg episode reward: [(0, '12.130'), (1, '13.300')] [2023-10-11 19:38:41,452][71601] Updated weights for policy 0, policy_version 13800 (0.0007) [2023-10-11 19:38:41,834][71601] Updated weights for policy 0, policy_version 13810 (0.0008) [2023-10-11 19:38:42,211][71601] Updated weights for policy 0, policy_version 13820 (0.0007) [2023-10-11 19:38:43,592][71635] Updated weights for policy 1, policy_version 13802 (0.0007) [2023-10-11 19:38:43,960][71635] Updated weights for policy 1, policy_version 13812 (0.0009) [2023-10-11 19:38:44,326][71635] Updated weights for policy 1, policy_version 13822 (0.0008) [2023-10-11 19:38:45,800][71601] Updated weights for policy 0, policy_version 13830 (0.0008) [2023-10-11 19:38:46,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 28311552. Throughput: 0: 1830.5, 1: 1814.8. Samples: 7083470. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:38:46,034][70582] Avg episode reward: [(0, '12.640'), (1, '13.490')] [2023-10-11 19:38:46,177][71601] Updated weights for policy 0, policy_version 13840 (0.0009) [2023-10-11 19:38:46,556][71601] Updated weights for policy 0, policy_version 13850 (0.0008) [2023-10-11 19:38:48,118][71635] Updated weights for policy 1, policy_version 13832 (0.0010) [2023-10-11 19:38:48,488][71635] Updated weights for policy 1, policy_version 13842 (0.0008) [2023-10-11 19:38:48,868][71635] Updated weights for policy 1, policy_version 13852 (0.0007) [2023-10-11 19:38:50,224][71601] Updated weights for policy 0, policy_version 13860 (0.0011) [2023-10-11 19:38:50,599][71601] Updated weights for policy 0, policy_version 13870 (0.0009) [2023-10-11 19:38:50,964][71601] Updated weights for policy 0, policy_version 13880 (0.0009) [2023-10-11 19:38:51,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 28377088. Throughput: 0: 1833.3, 1: 1812.1. Samples: 7105062. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:38:51,034][70582] Avg episode reward: [(0, '13.270'), (1, '14.850')] [2023-10-11 19:38:52,611][71635] Updated weights for policy 1, policy_version 13862 (0.0007) [2023-10-11 19:38:52,980][71635] Updated weights for policy 1, policy_version 13872 (0.0007) [2023-10-11 19:38:53,350][71635] Updated weights for policy 1, policy_version 13882 (0.0007) [2023-10-11 19:38:54,601][71601] Updated weights for policy 0, policy_version 13890 (0.0008) [2023-10-11 19:38:54,984][71601] Updated weights for policy 0, policy_version 13900 (0.0007) [2023-10-11 19:38:55,342][71601] Updated weights for policy 0, policy_version 13910 (0.0007) [2023-10-11 19:38:55,715][71601] Updated weights for policy 0, policy_version 13920 (0.0008) [2023-10-11 19:38:56,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 28475392. Throughput: 0: 1823.7, 1: 1813.7. Samples: 7126876. Policy #0 lag: (min: 18.0, avg: 18.5, max: 31.0) [2023-10-11 19:38:56,035][70582] Avg episode reward: [(0, '12.990'), (1, '14.760')] [2023-10-11 19:38:56,978][71635] Updated weights for policy 1, policy_version 13892 (0.0009) [2023-10-11 19:38:57,355][71635] Updated weights for policy 1, policy_version 13902 (0.0011) [2023-10-11 19:38:57,722][71635] Updated weights for policy 1, policy_version 13912 (0.0008) [2023-10-11 19:38:59,510][71601] Updated weights for policy 0, policy_version 13930 (0.0010) [2023-10-11 19:38:59,879][71601] Updated weights for policy 0, policy_version 13940 (0.0008) [2023-10-11 19:39:00,250][71601] Updated weights for policy 0, policy_version 13950 (0.0007) [2023-10-11 19:39:01,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 28540928. Throughput: 0: 1821.7, 1: 1819.2. Samples: 7137950. Policy #0 lag: (min: 18.0, avg: 18.5, max: 31.0) [2023-10-11 19:39:01,035][70582] Avg episode reward: [(0, '14.240'), (1, '14.580')] [2023-10-11 19:39:01,493][71635] Updated weights for policy 1, policy_version 13922 (0.0008) [2023-10-11 19:39:01,865][71635] Updated weights for policy 1, policy_version 13932 (0.0011) [2023-10-11 19:39:02,227][71635] Updated weights for policy 1, policy_version 13942 (0.0009) [2023-10-11 19:39:02,594][71635] Updated weights for policy 1, policy_version 13952 (0.0008) [2023-10-11 19:39:03,804][71601] Updated weights for policy 0, policy_version 13960 (0.0007) [2023-10-11 19:39:04,175][71601] Updated weights for policy 0, policy_version 13970 (0.0009) [2023-10-11 19:39:04,548][71601] Updated weights for policy 0, policy_version 13980 (0.0008) [2023-10-11 19:39:06,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 28606464. Throughput: 0: 1822.4, 1: 1809.6. Samples: 7159456. Policy #0 lag: (min: 24.0, avg: 49.5, max: 56.0) [2023-10-11 19:39:06,034][70582] Avg episode reward: [(0, '16.070'), (1, '14.580')] [2023-10-11 19:39:06,327][71635] Updated weights for policy 1, policy_version 13962 (0.0007) [2023-10-11 19:39:06,699][71635] Updated weights for policy 1, policy_version 13972 (0.0008) [2023-10-11 19:39:07,066][71635] Updated weights for policy 1, policy_version 13982 (0.0010) [2023-10-11 19:39:08,300][71601] Updated weights for policy 0, policy_version 13990 (0.0008) [2023-10-11 19:39:08,675][71601] Updated weights for policy 0, policy_version 14000 (0.0009) [2023-10-11 19:39:09,057][71601] Updated weights for policy 0, policy_version 14010 (0.0008) [2023-10-11 19:39:10,656][71635] Updated weights for policy 1, policy_version 13992 (0.0011) [2023-10-11 19:39:11,031][71635] Updated weights for policy 1, policy_version 14002 (0.0008) [2023-10-11 19:39:11,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 28672000. Throughput: 0: 1820.4, 1: 1815.8. Samples: 7181752. Policy #0 lag: (min: 24.0, avg: 49.5, max: 56.0) [2023-10-11 19:39:11,035][70582] Avg episode reward: [(0, '15.470'), (1, '11.910')] [2023-10-11 19:39:11,404][71635] Updated weights for policy 1, policy_version 14012 (0.0007) [2023-10-11 19:39:12,797][71601] Updated weights for policy 0, policy_version 14020 (0.0009) [2023-10-11 19:39:13,174][71601] Updated weights for policy 0, policy_version 14030 (0.0008) [2023-10-11 19:39:13,555][71601] Updated weights for policy 0, policy_version 14040 (0.0007) [2023-10-11 19:39:15,133][71635] Updated weights for policy 1, policy_version 14022 (0.0008) [2023-10-11 19:39:15,502][71635] Updated weights for policy 1, policy_version 14032 (0.0008) [2023-10-11 19:39:15,874][71635] Updated weights for policy 1, policy_version 14042 (0.0007) [2023-10-11 19:39:16,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 28737536. Throughput: 0: 1819.7, 1: 1809.9. Samples: 7192144. Policy #0 lag: (min: 24.0, avg: 49.5, max: 56.0) [2023-10-11 19:39:16,034][70582] Avg episode reward: [(0, '15.800'), (1, '11.700')] [2023-10-11 19:39:17,149][71601] Updated weights for policy 0, policy_version 14050 (0.0007) [2023-10-11 19:39:17,523][71601] Updated weights for policy 0, policy_version 14060 (0.0008) [2023-10-11 19:39:17,891][71601] Updated weights for policy 0, policy_version 14070 (0.0008) [2023-10-11 19:39:18,268][71601] Updated weights for policy 0, policy_version 14080 (0.0009) [2023-10-11 19:39:19,441][71635] Updated weights for policy 1, policy_version 14052 (0.0009) [2023-10-11 19:39:19,808][71635] Updated weights for policy 1, policy_version 14062 (0.0007) [2023-10-11 19:39:20,175][71635] Updated weights for policy 1, policy_version 14072 (0.0008) [2023-10-11 19:39:21,034][70582] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 28835840. Throughput: 0: 1822.9, 1: 1814.0. Samples: 7214366. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-11 19:39:21,034][70582] Avg episode reward: [(0, '14.620'), (1, '10.700')] [2023-10-11 19:39:22,084][71601] Updated weights for policy 0, policy_version 14090 (0.0007) [2023-10-11 19:39:22,449][71601] Updated weights for policy 0, policy_version 14100 (0.0008) [2023-10-11 19:39:22,821][71601] Updated weights for policy 0, policy_version 14110 (0.0008) [2023-10-11 19:39:23,806][71635] Updated weights for policy 1, policy_version 14082 (0.0009) [2023-10-11 19:39:24,185][71635] Updated weights for policy 1, policy_version 14092 (0.0009) [2023-10-11 19:39:24,553][71635] Updated weights for policy 1, policy_version 14102 (0.0008) [2023-10-11 19:39:24,915][71635] Updated weights for policy 1, policy_version 14112 (0.0008) [2023-10-11 19:39:26,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 28901376. Throughput: 0: 1817.9, 1: 1814.6. Samples: 7235798. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-11 19:39:26,035][70582] Avg episode reward: [(0, '13.680'), (1, '10.660')] [2023-10-11 19:39:26,486][71601] Updated weights for policy 0, policy_version 14120 (0.0007) [2023-10-11 19:39:26,859][71601] Updated weights for policy 0, policy_version 14130 (0.0011) [2023-10-11 19:39:27,234][71601] Updated weights for policy 0, policy_version 14140 (0.0008) [2023-10-11 19:39:28,648][71635] Updated weights for policy 1, policy_version 14122 (0.0007) [2023-10-11 19:39:29,015][71635] Updated weights for policy 1, policy_version 14132 (0.0010) [2023-10-11 19:39:29,381][71635] Updated weights for policy 1, policy_version 14142 (0.0010) [2023-10-11 19:39:30,837][71601] Updated weights for policy 0, policy_version 14150 (0.0009) [2023-10-11 19:39:31,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 28966912. Throughput: 0: 1817.1, 1: 1814.3. Samples: 7246880. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-11 19:39:31,035][70582] Avg episode reward: [(0, '14.150'), (1, '11.440')] [2023-10-11 19:39:31,204][71601] Updated weights for policy 0, policy_version 14160 (0.0010) [2023-10-11 19:39:31,578][71601] Updated weights for policy 0, policy_version 14170 (0.0009) [2023-10-11 19:39:33,243][71635] Updated weights for policy 1, policy_version 14152 (0.0009) [2023-10-11 19:39:33,599][71635] Updated weights for policy 1, policy_version 14162 (0.0007) [2023-10-11 19:39:33,966][71635] Updated weights for policy 1, policy_version 14172 (0.0007) [2023-10-11 19:39:35,445][71601] Updated weights for policy 0, policy_version 14180 (0.0009) [2023-10-11 19:39:35,809][71601] Updated weights for policy 0, policy_version 14190 (0.0009) [2023-10-11 19:39:36,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 29032448. Throughput: 0: 1811.7, 1: 1813.2. Samples: 7268184. Policy #0 lag: (min: 30.0, avg: 30.0, max: 33.0) [2023-10-11 19:39:36,034][70582] Avg episode reward: [(0, '14.140'), (1, '12.930')] [2023-10-11 19:39:36,185][71601] Updated weights for policy 0, policy_version 14200 (0.0008) [2023-10-11 19:39:37,622][71635] Updated weights for policy 1, policy_version 14182 (0.0007) [2023-10-11 19:39:37,989][71635] Updated weights for policy 1, policy_version 14192 (0.0007) [2023-10-11 19:39:38,357][71635] Updated weights for policy 1, policy_version 14202 (0.0008) [2023-10-11 19:39:40,075][71601] Updated weights for policy 0, policy_version 14210 (0.0007) [2023-10-11 19:39:40,444][71601] Updated weights for policy 0, policy_version 14220 (0.0008) [2023-10-11 19:39:40,820][71601] Updated weights for policy 0, policy_version 14230 (0.0010) [2023-10-11 19:39:41,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 29097984. Throughput: 0: 1817.1, 1: 1815.1. Samples: 7290324. Policy #0 lag: (min: 30.0, avg: 30.0, max: 33.0) [2023-10-11 19:39:41,035][70582] Avg episode reward: [(0, '14.030'), (1, '15.810')] [2023-10-11 19:39:41,194][71601] Updated weights for policy 0, policy_version 14240 (0.0007) [2023-10-11 19:39:42,109][71635] Updated weights for policy 1, policy_version 14212 (0.0008) [2023-10-11 19:39:42,511][71635] Updated weights for policy 1, policy_version 14222 (0.0008) [2023-10-11 19:39:42,884][71635] Updated weights for policy 1, policy_version 14232 (0.0007) [2023-10-11 19:39:44,792][71601] Updated weights for policy 0, policy_version 14250 (0.0008) [2023-10-11 19:39:45,153][71601] Updated weights for policy 0, policy_version 14260 (0.0009) [2023-10-11 19:39:45,536][71601] Updated weights for policy 0, policy_version 14270 (0.0010) [2023-10-11 19:39:46,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 29196288. Throughput: 0: 1805.8, 1: 1812.3. Samples: 7300762. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:39:46,034][70582] Avg episode reward: [(0, '13.790'), (1, '15.770')] [2023-10-11 19:39:46,312][71635] Updated weights for policy 1, policy_version 14242 (0.0009) [2023-10-11 19:39:46,675][71635] Updated weights for policy 1, policy_version 14252 (0.0008) [2023-10-11 19:39:47,043][71635] Updated weights for policy 1, policy_version 14262 (0.0009) [2023-10-11 19:39:47,415][71635] Updated weights for policy 1, policy_version 14272 (0.0008) [2023-10-11 19:39:49,342][71601] Updated weights for policy 0, policy_version 14280 (0.0008) [2023-10-11 19:39:49,715][71601] Updated weights for policy 0, policy_version 14290 (0.0010) [2023-10-11 19:39:50,090][71601] Updated weights for policy 0, policy_version 14300 (0.0009) [2023-10-11 19:39:51,034][70582] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 29261824. Throughput: 0: 1818.0, 1: 1819.6. Samples: 7323152. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:39:51,034][70582] Avg episode reward: [(0, '13.400'), (1, '14.960')] [2023-10-11 19:39:51,307][71635] Updated weights for policy 1, policy_version 14282 (0.0011) [2023-10-11 19:39:51,680][71635] Updated weights for policy 1, policy_version 14292 (0.0011) [2023-10-11 19:39:52,052][71635] Updated weights for policy 1, policy_version 14302 (0.0010) [2023-10-11 19:39:53,629][71601] Updated weights for policy 0, policy_version 14310 (0.0007) [2023-10-11 19:39:54,013][71601] Updated weights for policy 0, policy_version 14320 (0.0008) [2023-10-11 19:39:54,371][71601] Updated weights for policy 0, policy_version 14330 (0.0009) [2023-10-11 19:39:55,979][71635] Updated weights for policy 1, policy_version 14312 (0.0008) [2023-10-11 19:39:56,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 29327360. Throughput: 0: 1810.4, 1: 1816.9. Samples: 7344984. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:39:56,034][70582] Avg episode reward: [(0, '13.160'), (1, '13.900')] [2023-10-11 19:39:56,045][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000014336_14680064.pth... [2023-10-11 19:39:56,079][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000012640_12943360.pth [2023-10-11 19:39:56,358][71635] Updated weights for policy 1, policy_version 14322 (0.0008) [2023-10-11 19:39:56,727][71635] Updated weights for policy 1, policy_version 14332 (0.0009) [2023-10-11 19:39:56,867][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000014336_14680064.pth... [2023-10-11 19:39:56,905][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000012640_12943360.pth [2023-10-11 19:39:57,913][71601] Updated weights for policy 0, policy_version 14340 (0.0009) [2023-10-11 19:39:58,283][71601] Updated weights for policy 0, policy_version 14350 (0.0010) [2023-10-11 19:39:58,658][71601] Updated weights for policy 0, policy_version 14360 (0.0008) [2023-10-11 19:40:00,406][71635] Updated weights for policy 1, policy_version 14342 (0.0007) [2023-10-11 19:40:00,765][71635] Updated weights for policy 1, policy_version 14352 (0.0007) [2023-10-11 19:40:01,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 29392896. Throughput: 0: 1819.6, 1: 1815.8. Samples: 7355736. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-11 19:40:01,034][70582] Avg episode reward: [(0, '13.740'), (1, '11.570')] [2023-10-11 19:40:01,135][71635] Updated weights for policy 1, policy_version 14362 (0.0008) [2023-10-11 19:40:02,240][71601] Updated weights for policy 0, policy_version 14370 (0.0008) [2023-10-11 19:40:02,614][71601] Updated weights for policy 0, policy_version 14380 (0.0010) [2023-10-11 19:40:02,991][71601] Updated weights for policy 0, policy_version 14390 (0.0009) [2023-10-11 19:40:03,356][71601] Updated weights for policy 0, policy_version 14400 (0.0008) [2023-10-11 19:40:04,813][71635] Updated weights for policy 1, policy_version 14372 (0.0008) [2023-10-11 19:40:05,182][71635] Updated weights for policy 1, policy_version 14382 (0.0008) [2023-10-11 19:40:05,544][71635] Updated weights for policy 1, policy_version 14392 (0.0008) [2023-10-11 19:40:06,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 29491200. Throughput: 0: 1820.0, 1: 1816.0. Samples: 7377990. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-11 19:40:06,034][70582] Avg episode reward: [(0, '14.430'), (1, '12.020')] [2023-10-11 19:40:07,147][71601] Updated weights for policy 0, policy_version 14410 (0.0009) [2023-10-11 19:40:07,517][71601] Updated weights for policy 0, policy_version 14420 (0.0009) [2023-10-11 19:40:07,895][71601] Updated weights for policy 0, policy_version 14430 (0.0008) [2023-10-11 19:40:09,090][71635] Updated weights for policy 1, policy_version 14402 (0.0009) [2023-10-11 19:40:09,459][71635] Updated weights for policy 1, policy_version 14412 (0.0009) [2023-10-11 19:40:09,824][71635] Updated weights for policy 1, policy_version 14422 (0.0009) [2023-10-11 19:40:10,192][71635] Updated weights for policy 1, policy_version 14432 (0.0009) [2023-10-11 19:40:11,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 29556736. Throughput: 0: 1818.9, 1: 1814.0. Samples: 7399276. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-11 19:40:11,034][70582] Avg episode reward: [(0, '15.090'), (1, '12.690')] [2023-10-11 19:40:11,580][71601] Updated weights for policy 0, policy_version 14440 (0.0008) [2023-10-11 19:40:11,949][71601] Updated weights for policy 0, policy_version 14450 (0.0007) [2023-10-11 19:40:12,324][71601] Updated weights for policy 0, policy_version 14460 (0.0007) [2023-10-11 19:40:13,973][71635] Updated weights for policy 1, policy_version 14442 (0.0010) [2023-10-11 19:40:14,333][71635] Updated weights for policy 1, policy_version 14452 (0.0007) [2023-10-11 19:40:14,708][71635] Updated weights for policy 1, policy_version 14462 (0.0011) [2023-10-11 19:40:15,834][71601] Updated weights for policy 0, policy_version 14470 (0.0009) [2023-10-11 19:40:16,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 29622272. Throughput: 0: 1824.2, 1: 1818.9. Samples: 7410822. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:40:16,035][70582] Avg episode reward: [(0, '14.160'), (1, '12.970')] [2023-10-11 19:40:16,204][71601] Updated weights for policy 0, policy_version 14480 (0.0011) [2023-10-11 19:40:16,575][71601] Updated weights for policy 0, policy_version 14490 (0.0011) [2023-10-11 19:40:18,223][71635] Updated weights for policy 1, policy_version 14472 (0.0008) [2023-10-11 19:40:18,583][71635] Updated weights for policy 1, policy_version 14482 (0.0007) [2023-10-11 19:40:18,948][71635] Updated weights for policy 1, policy_version 14492 (0.0008) [2023-10-11 19:40:20,362][71601] Updated weights for policy 0, policy_version 14500 (0.0009) [2023-10-11 19:40:20,738][71601] Updated weights for policy 0, policy_version 14510 (0.0007) [2023-10-11 19:40:21,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 29687808. Throughput: 0: 1825.2, 1: 1820.4. Samples: 7432232. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:40:21,034][70582] Avg episode reward: [(0, '13.730'), (1, '13.660')] [2023-10-11 19:40:21,108][71601] Updated weights for policy 0, policy_version 14520 (0.0010) [2023-10-11 19:40:22,550][71635] Updated weights for policy 1, policy_version 14502 (0.0008) [2023-10-11 19:40:22,915][71635] Updated weights for policy 1, policy_version 14512 (0.0007) [2023-10-11 19:40:23,289][71635] Updated weights for policy 1, policy_version 14522 (0.0009) [2023-10-11 19:40:24,911][71601] Updated weights for policy 0, policy_version 14530 (0.0008) [2023-10-11 19:40:25,281][71601] Updated weights for policy 0, policy_version 14540 (0.0009) [2023-10-11 19:40:25,654][71601] Updated weights for policy 0, policy_version 14550 (0.0010) [2023-10-11 19:40:26,029][71601] Updated weights for policy 0, policy_version 14560 (0.0010) [2023-10-11 19:40:26,034][70582] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 29786112. Throughput: 0: 1819.1, 1: 1819.7. Samples: 7454070. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 19:40:26,034][70582] Avg episode reward: [(0, '15.440'), (1, '13.530')] [2023-10-11 19:40:27,010][71635] Updated weights for policy 1, policy_version 14532 (0.0010) [2023-10-11 19:40:27,409][71635] Updated weights for policy 1, policy_version 14542 (0.0008) [2023-10-11 19:40:27,771][71635] Updated weights for policy 1, policy_version 14552 (0.0009) [2023-10-11 19:40:29,648][71601] Updated weights for policy 0, policy_version 14570 (0.0009) [2023-10-11 19:40:30,019][71601] Updated weights for policy 0, policy_version 14580 (0.0007) [2023-10-11 19:40:30,391][71601] Updated weights for policy 0, policy_version 14590 (0.0010) [2023-10-11 19:40:31,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 29851648. Throughput: 0: 1825.0, 1: 1821.1. Samples: 7464836. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 19:40:31,035][70582] Avg episode reward: [(0, '15.720'), (1, '14.460')] [2023-10-11 19:40:31,438][71635] Updated weights for policy 1, policy_version 14562 (0.0010) [2023-10-11 19:40:31,799][71635] Updated weights for policy 1, policy_version 14572 (0.0010) [2023-10-11 19:40:32,175][71635] Updated weights for policy 1, policy_version 14582 (0.0011) [2023-10-11 19:40:32,537][71635] Updated weights for policy 1, policy_version 14592 (0.0009) [2023-10-11 19:40:33,966][71601] Updated weights for policy 0, policy_version 14600 (0.0011) [2023-10-11 19:40:34,333][71601] Updated weights for policy 0, policy_version 14610 (0.0011) [2023-10-11 19:40:34,711][71601] Updated weights for policy 0, policy_version 14620 (0.0011) [2023-10-11 19:40:36,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 29917184. Throughput: 0: 1817.1, 1: 1820.0. Samples: 7486820. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 19:40:36,034][70582] Avg episode reward: [(0, '15.580'), (1, '14.270')] [2023-10-11 19:40:36,086][71635] Updated weights for policy 1, policy_version 14602 (0.0007) [2023-10-11 19:40:36,460][71635] Updated weights for policy 1, policy_version 14612 (0.0008) [2023-10-11 19:40:36,822][71635] Updated weights for policy 1, policy_version 14622 (0.0008) [2023-10-11 19:40:38,369][71601] Updated weights for policy 0, policy_version 14630 (0.0009) [2023-10-11 19:40:38,738][71601] Updated weights for policy 0, policy_version 14640 (0.0007) [2023-10-11 19:40:39,117][71601] Updated weights for policy 0, policy_version 14650 (0.0008) [2023-10-11 19:40:40,532][71635] Updated weights for policy 1, policy_version 14632 (0.0011) [2023-10-11 19:40:40,904][71635] Updated weights for policy 1, policy_version 14642 (0.0008) [2023-10-11 19:40:41,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 29982720. Throughput: 0: 1823.6, 1: 1819.9. Samples: 7508938. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-11 19:40:41,034][70582] Avg episode reward: [(0, '14.520'), (1, '15.070')] [2023-10-11 19:40:41,267][71635] Updated weights for policy 1, policy_version 14652 (0.0009) [2023-10-11 19:40:42,758][71601] Updated weights for policy 0, policy_version 14660 (0.0009) [2023-10-11 19:40:43,131][71601] Updated weights for policy 0, policy_version 14670 (0.0008) [2023-10-11 19:40:43,511][71601] Updated weights for policy 0, policy_version 14680 (0.0008) [2023-10-11 19:40:44,736][71635] Updated weights for policy 1, policy_version 14662 (0.0009) [2023-10-11 19:40:45,098][71635] Updated weights for policy 1, policy_version 14672 (0.0008) [2023-10-11 19:40:45,464][71635] Updated weights for policy 1, policy_version 14682 (0.0009) [2023-10-11 19:40:46,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 30081024. Throughput: 0: 1814.3, 1: 1832.5. Samples: 7519840. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-11 19:40:46,035][70582] Avg episode reward: [(0, '12.870'), (1, '13.970')] [2023-10-11 19:40:47,061][71601] Updated weights for policy 0, policy_version 14690 (0.0007) [2023-10-11 19:40:47,429][71601] Updated weights for policy 0, policy_version 14700 (0.0010) [2023-10-11 19:40:47,806][71601] Updated weights for policy 0, policy_version 14710 (0.0010) [2023-10-11 19:40:48,175][71601] Updated weights for policy 0, policy_version 14720 (0.0009) [2023-10-11 19:40:49,159][71635] Updated weights for policy 1, policy_version 14692 (0.0009) [2023-10-11 19:40:49,513][71635] Updated weights for policy 1, policy_version 14702 (0.0011) [2023-10-11 19:40:49,880][71635] Updated weights for policy 1, policy_version 14712 (0.0007) [2023-10-11 19:40:51,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 30146560. Throughput: 0: 1816.7, 1: 1823.7. Samples: 7541806. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-11 19:40:51,035][70582] Avg episode reward: [(0, '11.810'), (1, '12.780')] [2023-10-11 19:40:52,002][71601] Updated weights for policy 0, policy_version 14730 (0.0007) [2023-10-11 19:40:52,374][71601] Updated weights for policy 0, policy_version 14740 (0.0008) [2023-10-11 19:40:52,743][71601] Updated weights for policy 0, policy_version 14750 (0.0010) [2023-10-11 19:40:53,779][71635] Updated weights for policy 1, policy_version 14722 (0.0008) [2023-10-11 19:40:54,145][71635] Updated weights for policy 1, policy_version 14732 (0.0007) [2023-10-11 19:40:54,514][71635] Updated weights for policy 1, policy_version 14742 (0.0007) [2023-10-11 19:40:54,885][71635] Updated weights for policy 1, policy_version 14752 (0.0009) [2023-10-11 19:40:56,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 30212096. Throughput: 0: 1819.4, 1: 1830.8. Samples: 7563536. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-11 19:40:56,034][70582] Avg episode reward: [(0, '11.590'), (1, '12.640')] [2023-10-11 19:40:56,365][71601] Updated weights for policy 0, policy_version 14760 (0.0008) [2023-10-11 19:40:56,732][71601] Updated weights for policy 0, policy_version 14770 (0.0007) [2023-10-11 19:40:57,107][71601] Updated weights for policy 0, policy_version 14780 (0.0008) [2023-10-11 19:40:58,552][71635] Updated weights for policy 1, policy_version 14762 (0.0007) [2023-10-11 19:40:58,919][71635] Updated weights for policy 1, policy_version 14772 (0.0009) [2023-10-11 19:40:59,296][71635] Updated weights for policy 1, policy_version 14782 (0.0010) [2023-10-11 19:41:00,818][71601] Updated weights for policy 0, policy_version 14790 (0.0008) [2023-10-11 19:41:01,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 30277632. Throughput: 0: 1815.9, 1: 1827.2. Samples: 7574760. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-11 19:41:01,035][70582] Avg episode reward: [(0, '13.370'), (1, '11.980')] [2023-10-11 19:41:01,188][71601] Updated weights for policy 0, policy_version 14800 (0.0010) [2023-10-11 19:41:01,555][71601] Updated weights for policy 0, policy_version 14810 (0.0007) [2023-10-11 19:41:02,903][71635] Updated weights for policy 1, policy_version 14792 (0.0010) [2023-10-11 19:41:03,275][71635] Updated weights for policy 1, policy_version 14802 (0.0011) [2023-10-11 19:41:03,632][71635] Updated weights for policy 1, policy_version 14812 (0.0007) [2023-10-11 19:41:05,333][71601] Updated weights for policy 0, policy_version 14820 (0.0007) [2023-10-11 19:41:05,712][71601] Updated weights for policy 0, policy_version 14830 (0.0009) [2023-10-11 19:41:06,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 30343168. Throughput: 0: 1816.6, 1: 1831.0. Samples: 7596376. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-11 19:41:06,035][70582] Avg episode reward: [(0, '13.830'), (1, '12.310')] [2023-10-11 19:41:06,093][71601] Updated weights for policy 0, policy_version 14840 (0.0007) [2023-10-11 19:41:07,269][71635] Updated weights for policy 1, policy_version 14822 (0.0008) [2023-10-11 19:41:07,640][71635] Updated weights for policy 1, policy_version 14832 (0.0010) [2023-10-11 19:41:08,000][71635] Updated weights for policy 1, policy_version 14842 (0.0007) [2023-10-11 19:41:09,856][71601] Updated weights for policy 0, policy_version 14850 (0.0008) [2023-10-11 19:41:10,224][71601] Updated weights for policy 0, policy_version 14860 (0.0007) [2023-10-11 19:41:10,613][71601] Updated weights for policy 0, policy_version 14870 (0.0008) [2023-10-11 19:41:10,986][71601] Updated weights for policy 0, policy_version 14880 (0.0008) [2023-10-11 19:41:11,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 30441472. Throughput: 0: 1822.5, 1: 1827.2. Samples: 7618306. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:41:11,035][70582] Avg episode reward: [(0, '13.420'), (1, '13.310')] [2023-10-11 19:41:11,877][71635] Updated weights for policy 1, policy_version 14852 (0.0008) [2023-10-11 19:41:12,272][71635] Updated weights for policy 1, policy_version 14862 (0.0009) [2023-10-11 19:41:12,633][71635] Updated weights for policy 1, policy_version 14872 (0.0010) [2023-10-11 19:41:14,591][71601] Updated weights for policy 0, policy_version 14890 (0.0010) [2023-10-11 19:41:14,958][71601] Updated weights for policy 0, policy_version 14900 (0.0010) [2023-10-11 19:41:15,326][71601] Updated weights for policy 0, policy_version 14910 (0.0012) [2023-10-11 19:41:16,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 30507008. Throughput: 0: 1816.2, 1: 1821.2. Samples: 7628520. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:41:16,035][70582] Avg episode reward: [(0, '13.380'), (1, '13.500')] [2023-10-11 19:41:16,535][71635] Updated weights for policy 1, policy_version 14882 (0.0010) [2023-10-11 19:41:16,900][71635] Updated weights for policy 1, policy_version 14892 (0.0008) [2023-10-11 19:41:17,263][71635] Updated weights for policy 1, policy_version 14902 (0.0009) [2023-10-11 19:41:17,631][71635] Updated weights for policy 1, policy_version 14912 (0.0009) [2023-10-11 19:41:18,985][71601] Updated weights for policy 0, policy_version 14920 (0.0008) [2023-10-11 19:41:19,358][71601] Updated weights for policy 0, policy_version 14930 (0.0008) [2023-10-11 19:41:19,721][71601] Updated weights for policy 0, policy_version 14940 (0.0007) [2023-10-11 19:41:21,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 30572544. Throughput: 0: 1814.4, 1: 1817.5. Samples: 7650256. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-11 19:41:21,035][70582] Avg episode reward: [(0, '13.130'), (1, '13.820')] [2023-10-11 19:41:21,196][71635] Updated weights for policy 1, policy_version 14922 (0.0009) [2023-10-11 19:41:21,558][71635] Updated weights for policy 1, policy_version 14932 (0.0008) [2023-10-11 19:41:21,923][71635] Updated weights for policy 1, policy_version 14942 (0.0009) [2023-10-11 19:41:23,525][71601] Updated weights for policy 0, policy_version 14950 (0.0008) [2023-10-11 19:41:23,894][71601] Updated weights for policy 0, policy_version 14960 (0.0009) [2023-10-11 19:41:24,266][71601] Updated weights for policy 0, policy_version 14970 (0.0011) [2023-10-11 19:41:25,670][71635] Updated weights for policy 1, policy_version 14952 (0.0009) [2023-10-11 19:41:26,033][71635] Updated weights for policy 1, policy_version 14962 (0.0007) [2023-10-11 19:41:26,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 30638080. Throughput: 0: 1810.9, 1: 1821.5. Samples: 7672394. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-11 19:41:26,034][70582] Avg episode reward: [(0, '13.300'), (1, '14.380')] [2023-10-11 19:41:26,391][71635] Updated weights for policy 1, policy_version 14972 (0.0007) [2023-10-11 19:41:27,953][71601] Updated weights for policy 0, policy_version 14980 (0.0008) [2023-10-11 19:41:28,327][71601] Updated weights for policy 0, policy_version 14990 (0.0007) [2023-10-11 19:41:28,693][71601] Updated weights for policy 0, policy_version 15000 (0.0008) [2023-10-11 19:41:30,129][71635] Updated weights for policy 1, policy_version 14982 (0.0008) [2023-10-11 19:41:30,498][71635] Updated weights for policy 1, policy_version 14992 (0.0008) [2023-10-11 19:41:30,867][71635] Updated weights for policy 1, policy_version 15002 (0.0007) [2023-10-11 19:41:31,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 30703616. Throughput: 0: 1817.6, 1: 1812.0. Samples: 7683168. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-11 19:41:31,034][70582] Avg episode reward: [(0, '13.710'), (1, '14.420')] [2023-10-11 19:41:32,482][71601] Updated weights for policy 0, policy_version 15010 (0.0007) [2023-10-11 19:41:32,850][71601] Updated weights for policy 0, policy_version 15020 (0.0008) [2023-10-11 19:41:33,233][71601] Updated weights for policy 0, policy_version 15030 (0.0009) [2023-10-11 19:41:33,598][71601] Updated weights for policy 0, policy_version 15040 (0.0009) [2023-10-11 19:41:34,567][71635] Updated weights for policy 1, policy_version 15012 (0.0007) [2023-10-11 19:41:34,936][71635] Updated weights for policy 1, policy_version 15022 (0.0007) [2023-10-11 19:41:35,296][71635] Updated weights for policy 1, policy_version 15032 (0.0007) [2023-10-11 19:41:36,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 30801920. Throughput: 0: 1810.6, 1: 1822.9. Samples: 7705312. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-11 19:41:36,034][70582] Avg episode reward: [(0, '14.840'), (1, '15.320')] [2023-10-11 19:41:37,311][71601] Updated weights for policy 0, policy_version 15050 (0.0008) [2023-10-11 19:41:37,687][71601] Updated weights for policy 0, policy_version 15060 (0.0008) [2023-10-11 19:41:38,061][71601] Updated weights for policy 0, policy_version 15070 (0.0008) [2023-10-11 19:41:39,102][71635] Updated weights for policy 1, policy_version 15042 (0.0008) [2023-10-11 19:41:39,472][71635] Updated weights for policy 1, policy_version 15052 (0.0011) [2023-10-11 19:41:39,836][71635] Updated weights for policy 1, policy_version 15062 (0.0011) [2023-10-11 19:41:40,198][71635] Updated weights for policy 1, policy_version 15072 (0.0007) [2023-10-11 19:41:41,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 30867456. Throughput: 0: 1810.6, 1: 1811.6. Samples: 7726534. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-11 19:41:41,035][70582] Avg episode reward: [(0, '14.150'), (1, '14.930')] [2023-10-11 19:41:41,866][71601] Updated weights for policy 0, policy_version 15080 (0.0008) [2023-10-11 19:41:42,236][71601] Updated weights for policy 0, policy_version 15090 (0.0008) [2023-10-11 19:41:42,607][71601] Updated weights for policy 0, policy_version 15100 (0.0008) [2023-10-11 19:41:43,909][71635] Updated weights for policy 1, policy_version 15082 (0.0009) [2023-10-11 19:41:44,277][71635] Updated weights for policy 1, policy_version 15092 (0.0011) [2023-10-11 19:41:44,642][71635] Updated weights for policy 1, policy_version 15102 (0.0007) [2023-10-11 19:41:46,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 30932992. Throughput: 0: 1808.9, 1: 1816.7. Samples: 7737912. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-11 19:41:46,035][70582] Avg episode reward: [(0, '13.590'), (1, '14.650')] [2023-10-11 19:41:46,180][71601] Updated weights for policy 0, policy_version 15110 (0.0008) [2023-10-11 19:41:46,550][71601] Updated weights for policy 0, policy_version 15120 (0.0008) [2023-10-11 19:41:46,923][71601] Updated weights for policy 0, policy_version 15130 (0.0009) [2023-10-11 19:41:48,185][71635] Updated weights for policy 1, policy_version 15112 (0.0010) [2023-10-11 19:41:48,558][71635] Updated weights for policy 1, policy_version 15122 (0.0008) [2023-10-11 19:41:48,915][71635] Updated weights for policy 1, policy_version 15132 (0.0009) [2023-10-11 19:41:50,666][71601] Updated weights for policy 0, policy_version 15140 (0.0007) [2023-10-11 19:41:51,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 30998528. Throughput: 0: 1811.3, 1: 1807.3. Samples: 7759214. Policy #0 lag: (min: 24.0, avg: 48.7, max: 56.0) [2023-10-11 19:41:51,035][70582] Avg episode reward: [(0, '13.680'), (1, '15.030')] [2023-10-11 19:41:51,037][71601] Updated weights for policy 0, policy_version 15150 (0.0009) [2023-10-11 19:41:51,406][71601] Updated weights for policy 0, policy_version 15160 (0.0009) [2023-10-11 19:41:52,631][71635] Updated weights for policy 1, policy_version 15142 (0.0010) [2023-10-11 19:41:53,004][71635] Updated weights for policy 1, policy_version 15152 (0.0007) [2023-10-11 19:41:53,374][71635] Updated weights for policy 1, policy_version 15162 (0.0009) [2023-10-11 19:41:54,824][71601] Updated weights for policy 0, policy_version 15170 (0.0008) [2023-10-11 19:41:55,193][71601] Updated weights for policy 0, policy_version 15180 (0.0009) [2023-10-11 19:41:55,559][71601] Updated weights for policy 0, policy_version 15190 (0.0007) [2023-10-11 19:41:55,937][71601] Updated weights for policy 0, policy_version 15200 (0.0009) [2023-10-11 19:41:56,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 31096832. Throughput: 0: 1815.1, 1: 1816.4. Samples: 7781724. Policy #0 lag: (min: 24.0, avg: 48.7, max: 56.0) [2023-10-11 19:41:56,034][70582] Avg episode reward: [(0, '12.420'), (1, '13.470')] [2023-10-11 19:41:56,042][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000015200_15564800.pth... [2023-10-11 19:41:56,042][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000015168_15532032.pth... [2023-10-11 19:41:56,072][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000013472_13795328.pth [2023-10-11 19:41:56,080][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000013472_13795328.pth [2023-10-11 19:41:57,067][71635] Updated weights for policy 1, policy_version 15172 (0.0007) [2023-10-11 19:41:57,471][71635] Updated weights for policy 1, policy_version 15182 (0.0009) [2023-10-11 19:41:57,841][71635] Updated weights for policy 1, policy_version 15192 (0.0008) [2023-10-11 19:41:59,589][71601] Updated weights for policy 0, policy_version 15210 (0.0011) [2023-10-11 19:41:59,962][71601] Updated weights for policy 0, policy_version 15220 (0.0010) [2023-10-11 19:42:00,329][71601] Updated weights for policy 0, policy_version 15230 (0.0010) [2023-10-11 19:42:01,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 31162368. Throughput: 0: 1821.9, 1: 1823.1. Samples: 7792546. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:42:01,035][70582] Avg episode reward: [(0, '12.280'), (1, '13.620')] [2023-10-11 19:42:01,408][71635] Updated weights for policy 1, policy_version 15202 (0.0008) [2023-10-11 19:42:01,773][71635] Updated weights for policy 1, policy_version 15212 (0.0007) [2023-10-11 19:42:02,134][71635] Updated weights for policy 1, policy_version 15222 (0.0007) [2023-10-11 19:42:02,506][71635] Updated weights for policy 1, policy_version 15232 (0.0007) [2023-10-11 19:42:04,000][71601] Updated weights for policy 0, policy_version 15240 (0.0009) [2023-10-11 19:42:04,374][71601] Updated weights for policy 0, policy_version 15250 (0.0008) [2023-10-11 19:42:04,748][71601] Updated weights for policy 0, policy_version 15260 (0.0008) [2023-10-11 19:42:06,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 31227904. Throughput: 0: 1826.8, 1: 1825.2. Samples: 7814592. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:42:06,035][70582] Avg episode reward: [(0, '12.320'), (1, '13.970')] [2023-10-11 19:42:06,251][71635] Updated weights for policy 1, policy_version 15242 (0.0009) [2023-10-11 19:42:06,621][71635] Updated weights for policy 1, policy_version 15252 (0.0008) [2023-10-11 19:42:06,986][71635] Updated weights for policy 1, policy_version 15262 (0.0007) [2023-10-11 19:42:08,415][71601] Updated weights for policy 0, policy_version 15270 (0.0007) [2023-10-11 19:42:08,778][71601] Updated weights for policy 0, policy_version 15280 (0.0008) [2023-10-11 19:42:09,160][71601] Updated weights for policy 0, policy_version 15290 (0.0009) [2023-10-11 19:42:10,619][71635] Updated weights for policy 1, policy_version 15272 (0.0009) [2023-10-11 19:42:10,993][71635] Updated weights for policy 1, policy_version 15282 (0.0008) [2023-10-11 19:42:11,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 31293440. Throughput: 0: 1824.9, 1: 1826.3. Samples: 7836702. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:42:11,035][70582] Avg episode reward: [(0, '12.380'), (1, '13.070')] [2023-10-11 19:42:11,353][71635] Updated weights for policy 1, policy_version 15292 (0.0009) [2023-10-11 19:42:13,071][71601] Updated weights for policy 0, policy_version 15300 (0.0009) [2023-10-11 19:42:13,442][71601] Updated weights for policy 0, policy_version 15310 (0.0007) [2023-10-11 19:42:13,821][71601] Updated weights for policy 0, policy_version 15320 (0.0008) [2023-10-11 19:42:14,888][71635] Updated weights for policy 1, policy_version 15302 (0.0007) [2023-10-11 19:42:15,255][71635] Updated weights for policy 1, policy_version 15312 (0.0009) [2023-10-11 19:42:15,623][71635] Updated weights for policy 1, policy_version 15322 (0.0007) [2023-10-11 19:42:16,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 31391744. Throughput: 0: 1828.2, 1: 1829.4. Samples: 7847760. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-10-11 19:42:16,034][70582] Avg episode reward: [(0, '13.180'), (1, '12.810')] [2023-10-11 19:42:17,378][71601] Updated weights for policy 0, policy_version 15330 (0.0011) [2023-10-11 19:42:17,750][71601] Updated weights for policy 0, policy_version 15340 (0.0011) [2023-10-11 19:42:18,123][71601] Updated weights for policy 0, policy_version 15350 (0.0007) [2023-10-11 19:42:18,493][71601] Updated weights for policy 0, policy_version 15360 (0.0007) [2023-10-11 19:42:19,174][71635] Updated weights for policy 1, policy_version 15332 (0.0007) [2023-10-11 19:42:19,535][71635] Updated weights for policy 1, policy_version 15342 (0.0008) [2023-10-11 19:42:19,900][71635] Updated weights for policy 1, policy_version 15352 (0.0010) [2023-10-11 19:42:21,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 31457280. Throughput: 0: 1827.2, 1: 1821.0. Samples: 7869482. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-10-11 19:42:21,035][70582] Avg episode reward: [(0, '12.880'), (1, '13.200')] [2023-10-11 19:42:22,396][71601] Updated weights for policy 0, policy_version 15370 (0.0010) [2023-10-11 19:42:22,772][71601] Updated weights for policy 0, policy_version 15380 (0.0011) [2023-10-11 19:42:23,150][71601] Updated weights for policy 0, policy_version 15390 (0.0011) [2023-10-11 19:42:23,661][71635] Updated weights for policy 1, policy_version 15362 (0.0010) [2023-10-11 19:42:24,028][71635] Updated weights for policy 1, policy_version 15372 (0.0009) [2023-10-11 19:42:24,394][71635] Updated weights for policy 1, policy_version 15382 (0.0010) [2023-10-11 19:42:24,763][71635] Updated weights for policy 1, policy_version 15392 (0.0011) [2023-10-11 19:42:26,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 31522816. Throughput: 0: 1825.7, 1: 1829.5. Samples: 7891018. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-10-11 19:42:26,035][70582] Avg episode reward: [(0, '13.370'), (1, '13.840')] [2023-10-11 19:42:26,875][71601] Updated weights for policy 0, policy_version 15400 (0.0010) [2023-10-11 19:42:27,255][71601] Updated weights for policy 0, policy_version 15410 (0.0009) [2023-10-11 19:42:27,622][71601] Updated weights for policy 0, policy_version 15420 (0.0010) [2023-10-11 19:42:28,598][71635] Updated weights for policy 1, policy_version 15402 (0.0009) [2023-10-11 19:42:28,966][71635] Updated weights for policy 1, policy_version 15412 (0.0010) [2023-10-11 19:42:29,329][71635] Updated weights for policy 1, policy_version 15422 (0.0010) [2023-10-11 19:42:31,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 31588352. Throughput: 0: 1821.3, 1: 1820.7. Samples: 7901804. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:42:31,035][70582] Avg episode reward: [(0, '14.070'), (1, '13.700')] [2023-10-11 19:42:31,287][71601] Updated weights for policy 0, policy_version 15430 (0.0009) [2023-10-11 19:42:31,660][71601] Updated weights for policy 0, policy_version 15440 (0.0010) [2023-10-11 19:42:32,032][71601] Updated weights for policy 0, policy_version 15450 (0.0010) [2023-10-11 19:42:32,992][71635] Updated weights for policy 1, policy_version 15432 (0.0008) [2023-10-11 19:42:33,367][71635] Updated weights for policy 1, policy_version 15442 (0.0009) [2023-10-11 19:42:33,736][71635] Updated weights for policy 1, policy_version 15452 (0.0008) [2023-10-11 19:42:35,719][71601] Updated weights for policy 0, policy_version 15460 (0.0008) [2023-10-11 19:42:36,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 31653888. Throughput: 0: 1819.2, 1: 1828.4. Samples: 7923352. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:42:36,034][70582] Avg episode reward: [(0, '13.860'), (1, '13.220')] [2023-10-11 19:42:36,089][71601] Updated weights for policy 0, policy_version 15470 (0.0008) [2023-10-11 19:42:36,464][71601] Updated weights for policy 0, policy_version 15480 (0.0009) [2023-10-11 19:42:37,473][71635] Updated weights for policy 1, policy_version 15462 (0.0009) [2023-10-11 19:42:37,845][71635] Updated weights for policy 1, policy_version 15472 (0.0008) [2023-10-11 19:42:38,200][71635] Updated weights for policy 1, policy_version 15482 (0.0007) [2023-10-11 19:42:40,209][71601] Updated weights for policy 0, policy_version 15490 (0.0008) [2023-10-11 19:42:40,577][71601] Updated weights for policy 0, policy_version 15500 (0.0009) [2023-10-11 19:42:40,940][71601] Updated weights for policy 0, policy_version 15510 (0.0007) [2023-10-11 19:42:41,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 31719424. Throughput: 0: 1825.9, 1: 1823.7. Samples: 7945958. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:42:41,035][70582] Avg episode reward: [(0, '13.380'), (1, '12.800')] [2023-10-11 19:42:41,316][71601] Updated weights for policy 0, policy_version 15520 (0.0007) [2023-10-11 19:42:42,013][71635] Updated weights for policy 1, policy_version 15492 (0.0009) [2023-10-11 19:42:42,389][71635] Updated weights for policy 1, policy_version 15502 (0.0009) [2023-10-11 19:42:42,757][71635] Updated weights for policy 1, policy_version 15512 (0.0009) [2023-10-11 19:42:45,009][71601] Updated weights for policy 0, policy_version 15530 (0.0008) [2023-10-11 19:42:45,378][71601] Updated weights for policy 0, policy_version 15540 (0.0009) [2023-10-11 19:42:45,748][71601] Updated weights for policy 0, policy_version 15550 (0.0008) [2023-10-11 19:42:46,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 31817728. Throughput: 0: 1815.8, 1: 1823.4. Samples: 7956308. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:42:46,034][70582] Avg episode reward: [(0, '14.380'), (1, '11.880')] [2023-10-11 19:42:46,481][71635] Updated weights for policy 1, policy_version 15522 (0.0008) [2023-10-11 19:42:46,856][71635] Updated weights for policy 1, policy_version 15532 (0.0007) [2023-10-11 19:42:47,224][71635] Updated weights for policy 1, policy_version 15542 (0.0011) [2023-10-11 19:42:47,582][71635] Updated weights for policy 1, policy_version 15552 (0.0008) [2023-10-11 19:42:49,291][71601] Updated weights for policy 0, policy_version 15560 (0.0008) [2023-10-11 19:42:49,666][71601] Updated weights for policy 0, policy_version 15570 (0.0008) [2023-10-11 19:42:50,035][71601] Updated weights for policy 0, policy_version 15580 (0.0007) [2023-10-11 19:42:51,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 31883264. Throughput: 0: 1823.1, 1: 1822.5. Samples: 7978644. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:42:51,035][70582] Avg episode reward: [(0, '14.670'), (1, '11.090')] [2023-10-11 19:42:51,226][71635] Updated weights for policy 1, policy_version 15562 (0.0008) [2023-10-11 19:42:51,584][71635] Updated weights for policy 1, policy_version 15572 (0.0008) [2023-10-11 19:42:51,957][71635] Updated weights for policy 1, policy_version 15582 (0.0008) [2023-10-11 19:42:53,677][71601] Updated weights for policy 0, policy_version 15590 (0.0008) [2023-10-11 19:42:54,045][71601] Updated weights for policy 0, policy_version 15600 (0.0008) [2023-10-11 19:42:54,417][71601] Updated weights for policy 0, policy_version 15610 (0.0009) [2023-10-11 19:42:55,650][71635] Updated weights for policy 1, policy_version 15592 (0.0009) [2023-10-11 19:42:56,008][71635] Updated weights for policy 1, policy_version 15602 (0.0008) [2023-10-11 19:42:56,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 31948800. Throughput: 0: 1818.9, 1: 1822.3. Samples: 8000558. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-11 19:42:56,034][70582] Avg episode reward: [(0, '13.730'), (1, '12.050')] [2023-10-11 19:42:56,371][71635] Updated weights for policy 1, policy_version 15612 (0.0008) [2023-10-11 19:42:58,135][71601] Updated weights for policy 0, policy_version 15620 (0.0007) [2023-10-11 19:42:58,506][71601] Updated weights for policy 0, policy_version 15630 (0.0010) [2023-10-11 19:42:58,890][71601] Updated weights for policy 0, policy_version 15640 (0.0009) [2023-10-11 19:43:00,177][71635] Updated weights for policy 1, policy_version 15622 (0.0008) [2023-10-11 19:43:00,548][71635] Updated weights for policy 1, policy_version 15632 (0.0008) [2023-10-11 19:43:00,906][71635] Updated weights for policy 1, policy_version 15642 (0.0007) [2023-10-11 19:43:01,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 32014336. Throughput: 0: 1817.5, 1: 1817.9. Samples: 8011350. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-11 19:43:01,035][70582] Avg episode reward: [(0, '13.800'), (1, '12.970')] [2023-10-11 19:43:02,472][71601] Updated weights for policy 0, policy_version 15650 (0.0008) [2023-10-11 19:43:02,844][71601] Updated weights for policy 0, policy_version 15660 (0.0008) [2023-10-11 19:43:03,213][71601] Updated weights for policy 0, policy_version 15670 (0.0007) [2023-10-11 19:43:03,583][71601] Updated weights for policy 0, policy_version 15680 (0.0007) [2023-10-11 19:43:04,597][71635] Updated weights for policy 1, policy_version 15652 (0.0007) [2023-10-11 19:43:04,960][71635] Updated weights for policy 1, policy_version 15662 (0.0007) [2023-10-11 19:43:05,325][71635] Updated weights for policy 1, policy_version 15672 (0.0007) [2023-10-11 19:43:06,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 32112640. Throughput: 0: 1814.4, 1: 1818.7. Samples: 8032970. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-11 19:43:06,034][70582] Avg episode reward: [(0, '13.650'), (1, '12.870')] [2023-10-11 19:43:07,274][71601] Updated weights for policy 0, policy_version 15690 (0.0007) [2023-10-11 19:43:07,640][71601] Updated weights for policy 0, policy_version 15700 (0.0010) [2023-10-11 19:43:08,009][71601] Updated weights for policy 0, policy_version 15710 (0.0009) [2023-10-11 19:43:09,045][71635] Updated weights for policy 1, policy_version 15682 (0.0009) [2023-10-11 19:43:09,403][71635] Updated weights for policy 1, policy_version 15692 (0.0008) [2023-10-11 19:43:09,774][71635] Updated weights for policy 1, policy_version 15702 (0.0007) [2023-10-11 19:43:10,145][71635] Updated weights for policy 1, policy_version 15712 (0.0008) [2023-10-11 19:43:11,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 32178176. Throughput: 0: 1818.7, 1: 1809.7. Samples: 8054298. Policy #0 lag: (min: 1.0, avg: 6.8, max: 33.0) [2023-10-11 19:43:11,035][70582] Avg episode reward: [(0, '13.390'), (1, '13.860')] [2023-10-11 19:43:11,581][71601] Updated weights for policy 0, policy_version 15720 (0.0009) [2023-10-11 19:43:11,957][71601] Updated weights for policy 0, policy_version 15730 (0.0008) [2023-10-11 19:43:12,328][71601] Updated weights for policy 0, policy_version 15740 (0.0009) [2023-10-11 19:43:13,729][71635] Updated weights for policy 1, policy_version 15722 (0.0008) [2023-10-11 19:43:14,099][71635] Updated weights for policy 1, policy_version 15732 (0.0008) [2023-10-11 19:43:14,463][71635] Updated weights for policy 1, policy_version 15742 (0.0009) [2023-10-11 19:43:16,007][71601] Updated weights for policy 0, policy_version 15750 (0.0007) [2023-10-11 19:43:16,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 32243712. Throughput: 0: 1826.2, 1: 1818.1. Samples: 8065798. Policy #0 lag: (min: 1.0, avg: 6.8, max: 33.0) [2023-10-11 19:43:16,034][70582] Avg episode reward: [(0, '14.060'), (1, '14.510')] [2023-10-11 19:43:16,380][71601] Updated weights for policy 0, policy_version 15760 (0.0007) [2023-10-11 19:43:16,759][71601] Updated weights for policy 0, policy_version 15770 (0.0008) [2023-10-11 19:43:18,138][71635] Updated weights for policy 1, policy_version 15752 (0.0010) [2023-10-11 19:43:18,511][71635] Updated weights for policy 1, policy_version 15762 (0.0007) [2023-10-11 19:43:18,871][71635] Updated weights for policy 1, policy_version 15772 (0.0009) [2023-10-11 19:43:20,390][71601] Updated weights for policy 0, policy_version 15780 (0.0009) [2023-10-11 19:43:20,753][71601] Updated weights for policy 0, policy_version 15790 (0.0009) [2023-10-11 19:43:21,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 32309248. Throughput: 0: 1827.6, 1: 1811.9. Samples: 8087130. Policy #0 lag: (min: 1.0, avg: 6.8, max: 33.0) [2023-10-11 19:43:21,034][70582] Avg episode reward: [(0, '13.060'), (1, '13.710')] [2023-10-11 19:43:21,126][71601] Updated weights for policy 0, policy_version 15800 (0.0007) [2023-10-11 19:43:22,604][71635] Updated weights for policy 1, policy_version 15782 (0.0009) [2023-10-11 19:43:22,972][71635] Updated weights for policy 1, policy_version 15792 (0.0011) [2023-10-11 19:43:23,330][71635] Updated weights for policy 1, policy_version 15802 (0.0008) [2023-10-11 19:43:24,730][71601] Updated weights for policy 0, policy_version 15810 (0.0007) [2023-10-11 19:43:25,095][71601] Updated weights for policy 0, policy_version 15820 (0.0007) [2023-10-11 19:43:25,473][71601] Updated weights for policy 0, policy_version 15830 (0.0010) [2023-10-11 19:43:25,842][71601] Updated weights for policy 0, policy_version 15840 (0.0009) [2023-10-11 19:43:26,034][70582] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 32407552. Throughput: 0: 1815.2, 1: 1810.3. Samples: 8109106. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-11 19:43:26,035][70582] Avg episode reward: [(0, '12.910'), (1, '12.970')] [2023-10-11 19:43:27,037][71635] Updated weights for policy 1, policy_version 15812 (0.0009) [2023-10-11 19:43:27,432][71635] Updated weights for policy 1, policy_version 15822 (0.0010) [2023-10-11 19:43:27,803][71635] Updated weights for policy 1, policy_version 15832 (0.0009) [2023-10-11 19:43:29,629][71601] Updated weights for policy 0, policy_version 15850 (0.0009) [2023-10-11 19:43:29,997][71601] Updated weights for policy 0, policy_version 15860 (0.0008) [2023-10-11 19:43:30,374][71601] Updated weights for policy 0, policy_version 15870 (0.0008) [2023-10-11 19:43:31,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 32473088. Throughput: 0: 1828.1, 1: 1811.1. Samples: 8120072. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-11 19:43:31,034][70582] Avg episode reward: [(0, '12.860'), (1, '12.910')] [2023-10-11 19:43:31,595][71635] Updated weights for policy 1, policy_version 15842 (0.0011) [2023-10-11 19:43:31,968][71635] Updated weights for policy 1, policy_version 15852 (0.0008) [2023-10-11 19:43:32,343][71635] Updated weights for policy 1, policy_version 15862 (0.0011) [2023-10-11 19:43:32,717][71635] Updated weights for policy 1, policy_version 15872 (0.0010) [2023-10-11 19:43:34,141][71601] Updated weights for policy 0, policy_version 15880 (0.0008) [2023-10-11 19:43:34,510][71601] Updated weights for policy 0, policy_version 15890 (0.0007) [2023-10-11 19:43:34,885][71601] Updated weights for policy 0, policy_version 15900 (0.0009) [2023-10-11 19:43:36,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 32538624. Throughput: 0: 1819.8, 1: 1811.0. Samples: 8142028. Policy #0 lag: (min: 0.0, avg: 25.5, max: 32.0) [2023-10-11 19:43:36,034][70582] Avg episode reward: [(0, '12.400'), (1, '11.980')] [2023-10-11 19:43:36,377][71635] Updated weights for policy 1, policy_version 15882 (0.0007) [2023-10-11 19:43:36,740][71635] Updated weights for policy 1, policy_version 15892 (0.0007) [2023-10-11 19:43:37,100][71635] Updated weights for policy 1, policy_version 15902 (0.0009) [2023-10-11 19:43:38,709][71601] Updated weights for policy 0, policy_version 15910 (0.0008) [2023-10-11 19:43:39,082][71601] Updated weights for policy 0, policy_version 15920 (0.0008) [2023-10-11 19:43:39,453][71601] Updated weights for policy 0, policy_version 15930 (0.0009) [2023-10-11 19:43:40,750][71635] Updated weights for policy 1, policy_version 15912 (0.0008) [2023-10-11 19:43:41,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 32604160. Throughput: 0: 1818.0, 1: 1810.3. Samples: 8163830. Policy #0 lag: (min: 0.0, avg: 25.5, max: 32.0) [2023-10-11 19:43:41,035][70582] Avg episode reward: [(0, '13.280'), (1, '12.040')] [2023-10-11 19:43:41,112][71635] Updated weights for policy 1, policy_version 15922 (0.0009) [2023-10-11 19:43:41,482][71635] Updated weights for policy 1, policy_version 15932 (0.0009) [2023-10-11 19:43:43,203][71601] Updated weights for policy 0, policy_version 15940 (0.0008) [2023-10-11 19:43:43,565][71601] Updated weights for policy 0, policy_version 15950 (0.0008) [2023-10-11 19:43:43,935][71601] Updated weights for policy 0, policy_version 15960 (0.0008) [2023-10-11 19:43:45,218][71635] Updated weights for policy 1, policy_version 15942 (0.0008) [2023-10-11 19:43:45,581][71635] Updated weights for policy 1, policy_version 15952 (0.0010) [2023-10-11 19:43:45,959][71635] Updated weights for policy 1, policy_version 15962 (0.0009) [2023-10-11 19:43:46,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 32669696. Throughput: 0: 1823.8, 1: 1808.1. Samples: 8174786. Policy #0 lag: (min: 0.0, avg: 25.5, max: 32.0) [2023-10-11 19:43:46,034][70582] Avg episode reward: [(0, '13.530'), (1, '12.150')] [2023-10-11 19:43:47,624][71601] Updated weights for policy 0, policy_version 15970 (0.0008) [2023-10-11 19:43:47,996][71601] Updated weights for policy 0, policy_version 15980 (0.0009) [2023-10-11 19:43:48,371][71601] Updated weights for policy 0, policy_version 15990 (0.0007) [2023-10-11 19:43:48,739][71601] Updated weights for policy 0, policy_version 16000 (0.0008) [2023-10-11 19:43:49,657][71635] Updated weights for policy 1, policy_version 15972 (0.0009) [2023-10-11 19:43:50,037][71635] Updated weights for policy 1, policy_version 15982 (0.0010) [2023-10-11 19:43:50,395][71635] Updated weights for policy 1, policy_version 15992 (0.0012) [2023-10-11 19:43:51,034][70582] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 32768000. Throughput: 0: 1820.5, 1: 1815.0. Samples: 8196568. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-11 19:43:51,034][70582] Avg episode reward: [(0, '13.040'), (1, '13.310')] [2023-10-11 19:43:52,385][71601] Updated weights for policy 0, policy_version 16010 (0.0009) [2023-10-11 19:43:52,750][71601] Updated weights for policy 0, policy_version 16020 (0.0010) [2023-10-11 19:43:53,125][71601] Updated weights for policy 0, policy_version 16030 (0.0007) [2023-10-11 19:43:54,226][71635] Updated weights for policy 1, policy_version 16002 (0.0008) [2023-10-11 19:43:54,595][71635] Updated weights for policy 1, policy_version 16012 (0.0009) [2023-10-11 19:43:54,966][71635] Updated weights for policy 1, policy_version 16022 (0.0007) [2023-10-11 19:43:55,332][71635] Updated weights for policy 1, policy_version 16032 (0.0008) [2023-10-11 19:43:56,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 32833536. Throughput: 0: 1813.1, 1: 1815.2. Samples: 8217568. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-11 19:43:56,034][70582] Avg episode reward: [(0, '13.540'), (1, '12.950')] [2023-10-11 19:43:56,043][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000016032_16416768.pth... [2023-10-11 19:43:56,043][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000016032_16416768.pth... [2023-10-11 19:43:56,074][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000014336_14680064.pth [2023-10-11 19:43:56,075][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000014336_14680064.pth [2023-10-11 19:43:56,905][71601] Updated weights for policy 0, policy_version 16040 (0.0009) [2023-10-11 19:43:57,276][71601] Updated weights for policy 0, policy_version 16050 (0.0010) [2023-10-11 19:43:57,660][71601] Updated weights for policy 0, policy_version 16060 (0.0011) [2023-10-11 19:43:58,953][71635] Updated weights for policy 1, policy_version 16042 (0.0008) [2023-10-11 19:43:59,324][71635] Updated weights for policy 1, policy_version 16052 (0.0008) [2023-10-11 19:43:59,680][71635] Updated weights for policy 1, policy_version 16062 (0.0007) [2023-10-11 19:44:01,034][70582] Fps is (10 sec: 13106.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 32899072. Throughput: 0: 1807.4, 1: 1812.2. Samples: 8228682. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-11 19:44:01,035][70582] Avg episode reward: [(0, '14.920'), (1, '14.400')] [2023-10-11 19:44:01,452][71601] Updated weights for policy 0, policy_version 16070 (0.0009) [2023-10-11 19:44:01,818][71601] Updated weights for policy 0, policy_version 16080 (0.0008) [2023-10-11 19:44:02,200][71601] Updated weights for policy 0, policy_version 16090 (0.0007) [2023-10-11 19:44:03,430][71635] Updated weights for policy 1, policy_version 16072 (0.0008) [2023-10-11 19:44:03,798][71635] Updated weights for policy 1, policy_version 16082 (0.0009) [2023-10-11 19:44:04,163][71635] Updated weights for policy 1, policy_version 16092 (0.0008) [2023-10-11 19:44:05,890][71601] Updated weights for policy 0, policy_version 16100 (0.0009) [2023-10-11 19:44:06,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 32964608. Throughput: 0: 1802.4, 1: 1814.0. Samples: 8249864. Policy #0 lag: (min: 31.0, avg: 45.9, max: 63.0) [2023-10-11 19:44:06,034][70582] Avg episode reward: [(0, '13.640'), (1, '14.470')] [2023-10-11 19:44:06,251][71601] Updated weights for policy 0, policy_version 16110 (0.0010) [2023-10-11 19:44:06,622][71601] Updated weights for policy 0, policy_version 16120 (0.0007) [2023-10-11 19:44:07,965][71635] Updated weights for policy 1, policy_version 16102 (0.0008) [2023-10-11 19:44:08,333][71635] Updated weights for policy 1, policy_version 16112 (0.0008) [2023-10-11 19:44:08,707][71635] Updated weights for policy 1, policy_version 16122 (0.0008) [2023-10-11 19:44:10,293][71601] Updated weights for policy 0, policy_version 16130 (0.0008) [2023-10-11 19:44:10,666][71601] Updated weights for policy 0, policy_version 16140 (0.0008) [2023-10-11 19:44:11,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 33030144. Throughput: 0: 1817.4, 1: 1807.6. Samples: 8272232. Policy #0 lag: (min: 31.0, avg: 45.9, max: 63.0) [2023-10-11 19:44:11,035][70582] Avg episode reward: [(0, '12.840'), (1, '14.950')] [2023-10-11 19:44:11,047][71601] Updated weights for policy 0, policy_version 16150 (0.0010) [2023-10-11 19:44:11,423][71601] Updated weights for policy 0, policy_version 16160 (0.0010) [2023-10-11 19:44:12,541][71635] Updated weights for policy 1, policy_version 16132 (0.0008) [2023-10-11 19:44:12,939][71635] Updated weights for policy 1, policy_version 16142 (0.0010) [2023-10-11 19:44:13,312][71635] Updated weights for policy 1, policy_version 16152 (0.0011) [2023-10-11 19:44:15,056][71601] Updated weights for policy 0, policy_version 16170 (0.0007) [2023-10-11 19:44:15,429][71601] Updated weights for policy 0, policy_version 16180 (0.0007) [2023-10-11 19:44:15,798][71601] Updated weights for policy 0, policy_version 16190 (0.0007) [2023-10-11 19:44:16,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 33128448. Throughput: 0: 1799.4, 1: 1808.3. Samples: 8282416. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:44:16,035][70582] Avg episode reward: [(0, '13.270'), (1, '14.660')] [2023-10-11 19:44:16,920][71635] Updated weights for policy 1, policy_version 16162 (0.0009) [2023-10-11 19:44:17,286][71635] Updated weights for policy 1, policy_version 16172 (0.0007) [2023-10-11 19:44:17,653][71635] Updated weights for policy 1, policy_version 16182 (0.0007) [2023-10-11 19:44:18,029][71635] Updated weights for policy 1, policy_version 16192 (0.0007) [2023-10-11 19:44:19,491][71601] Updated weights for policy 0, policy_version 16200 (0.0008) [2023-10-11 19:44:19,865][71601] Updated weights for policy 0, policy_version 16210 (0.0008) [2023-10-11 19:44:20,236][71601] Updated weights for policy 0, policy_version 16220 (0.0009) [2023-10-11 19:44:21,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 33193984. Throughput: 0: 1808.8, 1: 1800.9. Samples: 8304468. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:44:21,035][70582] Avg episode reward: [(0, '11.160'), (1, '12.960')] [2023-10-11 19:44:21,777][71635] Updated weights for policy 1, policy_version 16202 (0.0009) [2023-10-11 19:44:22,148][71635] Updated weights for policy 1, policy_version 16212 (0.0010) [2023-10-11 19:44:22,510][71635] Updated weights for policy 1, policy_version 16222 (0.0009) [2023-10-11 19:44:24,007][71601] Updated weights for policy 0, policy_version 16230 (0.0008) [2023-10-11 19:44:24,393][71601] Updated weights for policy 0, policy_version 16240 (0.0008) [2023-10-11 19:44:24,761][71601] Updated weights for policy 0, policy_version 16250 (0.0009) [2023-10-11 19:44:26,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 33259520. Throughput: 0: 1804.4, 1: 1801.7. Samples: 8326104. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:44:26,034][70582] Avg episode reward: [(0, '10.640'), (1, '11.970')] [2023-10-11 19:44:26,289][71635] Updated weights for policy 1, policy_version 16232 (0.0008) [2023-10-11 19:44:26,654][71635] Updated weights for policy 1, policy_version 16242 (0.0009) [2023-10-11 19:44:27,012][71635] Updated weights for policy 1, policy_version 16252 (0.0008) [2023-10-11 19:44:28,345][71601] Updated weights for policy 0, policy_version 16260 (0.0008) [2023-10-11 19:44:28,718][71601] Updated weights for policy 0, policy_version 16270 (0.0007) [2023-10-11 19:44:29,084][71601] Updated weights for policy 0, policy_version 16280 (0.0010) [2023-10-11 19:44:30,825][71635] Updated weights for policy 1, policy_version 16262 (0.0008) [2023-10-11 19:44:31,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 33325056. Throughput: 0: 1810.2, 1: 1802.6. Samples: 8337362. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-11 19:44:31,034][70582] Avg episode reward: [(0, '11.670'), (1, '11.160')] [2023-10-11 19:44:31,184][71635] Updated weights for policy 1, policy_version 16272 (0.0007) [2023-10-11 19:44:31,559][71635] Updated weights for policy 1, policy_version 16282 (0.0007) [2023-10-11 19:44:32,872][71601] Updated weights for policy 0, policy_version 16290 (0.0010) [2023-10-11 19:44:33,236][71601] Updated weights for policy 0, policy_version 16300 (0.0010) [2023-10-11 19:44:33,609][71601] Updated weights for policy 0, policy_version 16310 (0.0008) [2023-10-11 19:44:33,978][71601] Updated weights for policy 0, policy_version 16320 (0.0008) [2023-10-11 19:44:35,389][71635] Updated weights for policy 1, policy_version 16292 (0.0009) [2023-10-11 19:44:35,758][71635] Updated weights for policy 1, policy_version 16302 (0.0010) [2023-10-11 19:44:36,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 33390592. Throughput: 0: 1810.9, 1: 1796.3. Samples: 8358892. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-11 19:44:36,034][70582] Avg episode reward: [(0, '10.920'), (1, '11.450')] [2023-10-11 19:44:36,123][71635] Updated weights for policy 1, policy_version 16312 (0.0010) [2023-10-11 19:44:37,654][71601] Updated weights for policy 0, policy_version 16330 (0.0009) [2023-10-11 19:44:38,029][71601] Updated weights for policy 0, policy_version 16340 (0.0009) [2023-10-11 19:44:38,409][71601] Updated weights for policy 0, policy_version 16350 (0.0008) [2023-10-11 19:44:39,694][71635] Updated weights for policy 1, policy_version 16322 (0.0008) [2023-10-11 19:44:40,065][71635] Updated weights for policy 1, policy_version 16332 (0.0010) [2023-10-11 19:44:40,420][71635] Updated weights for policy 1, policy_version 16342 (0.0010) [2023-10-11 19:44:40,789][71635] Updated weights for policy 1, policy_version 16352 (0.0011) [2023-10-11 19:44:41,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 33488896. Throughput: 0: 1818.8, 1: 1810.2. Samples: 8380872. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-11 19:44:41,034][70582] Avg episode reward: [(0, '11.730'), (1, '11.900')] [2023-10-11 19:44:41,872][71601] Updated weights for policy 0, policy_version 16360 (0.0008) [2023-10-11 19:44:42,250][71601] Updated weights for policy 0, policy_version 16370 (0.0007) [2023-10-11 19:44:42,621][71601] Updated weights for policy 0, policy_version 16380 (0.0008) [2023-10-11 19:44:44,482][71635] Updated weights for policy 1, policy_version 16362 (0.0009) [2023-10-11 19:44:44,850][71635] Updated weights for policy 1, policy_version 16372 (0.0009) [2023-10-11 19:44:45,210][71635] Updated weights for policy 1, policy_version 16382 (0.0007) [2023-10-11 19:44:46,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 33554432. Throughput: 0: 1824.8, 1: 1803.3. Samples: 8391946. Policy #0 lag: (min: 8.0, avg: 33.9, max: 40.0) [2023-10-11 19:44:46,034][70582] Avg episode reward: [(0, '11.740'), (1, '12.920')] [2023-10-11 19:44:46,318][71601] Updated weights for policy 0, policy_version 16390 (0.0008) [2023-10-11 19:44:46,683][71601] Updated weights for policy 0, policy_version 16400 (0.0007) [2023-10-11 19:44:47,052][71601] Updated weights for policy 0, policy_version 16410 (0.0008) [2023-10-11 19:44:48,916][71635] Updated weights for policy 1, policy_version 16392 (0.0011) [2023-10-11 19:44:49,284][71635] Updated weights for policy 1, policy_version 16402 (0.0010) [2023-10-11 19:44:49,654][71635] Updated weights for policy 1, policy_version 16412 (0.0010) [2023-10-11 19:44:50,775][71601] Updated weights for policy 0, policy_version 16420 (0.0008) [2023-10-11 19:44:51,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 33619968. Throughput: 0: 1827.0, 1: 1816.4. Samples: 8413814. Policy #0 lag: (min: 8.0, avg: 33.9, max: 40.0) [2023-10-11 19:44:51,034][70582] Avg episode reward: [(0, '12.720'), (1, '13.380')] [2023-10-11 19:44:51,153][71601] Updated weights for policy 0, policy_version 16430 (0.0008) [2023-10-11 19:44:51,523][71601] Updated weights for policy 0, policy_version 16440 (0.0007) [2023-10-11 19:44:53,307][71635] Updated weights for policy 1, policy_version 16422 (0.0010) [2023-10-11 19:44:53,674][71635] Updated weights for policy 1, policy_version 16432 (0.0010) [2023-10-11 19:44:54,043][71635] Updated weights for policy 1, policy_version 16442 (0.0010) [2023-10-11 19:44:55,091][71601] Updated weights for policy 0, policy_version 16450 (0.0009) [2023-10-11 19:44:55,465][71601] Updated weights for policy 0, policy_version 16460 (0.0008) [2023-10-11 19:44:55,834][71601] Updated weights for policy 0, policy_version 16470 (0.0009) [2023-10-11 19:44:56,034][70582] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 33685504. Throughput: 0: 1823.9, 1: 1809.1. Samples: 8435716. Policy #0 lag: (min: 8.0, avg: 33.9, max: 40.0) [2023-10-11 19:44:56,035][70582] Avg episode reward: [(0, '13.910'), (1, '13.960')] [2023-10-11 19:44:56,204][71601] Updated weights for policy 0, policy_version 16480 (0.0009) [2023-10-11 19:44:57,898][71635] Updated weights for policy 1, policy_version 16452 (0.0008) [2023-10-11 19:44:58,300][71635] Updated weights for policy 1, policy_version 16462 (0.0008) [2023-10-11 19:44:58,680][71635] Updated weights for policy 1, policy_version 16472 (0.0007) [2023-10-11 19:44:59,861][71601] Updated weights for policy 0, policy_version 16490 (0.0008) [2023-10-11 19:45:00,229][71601] Updated weights for policy 0, policy_version 16500 (0.0008) [2023-10-11 19:45:00,610][71601] Updated weights for policy 0, policy_version 16510 (0.0011) [2023-10-11 19:45:01,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 33783808. Throughput: 0: 1832.8, 1: 1819.3. Samples: 8446758. Policy #0 lag: (min: 6.0, avg: 11.4, max: 38.0) [2023-10-11 19:45:01,034][70582] Avg episode reward: [(0, '12.830'), (1, '13.460')] [2023-10-11 19:45:02,353][71635] Updated weights for policy 1, policy_version 16482 (0.0009) [2023-10-11 19:45:02,718][71635] Updated weights for policy 1, policy_version 16492 (0.0007) [2023-10-11 19:45:03,089][71635] Updated weights for policy 1, policy_version 16502 (0.0007) [2023-10-11 19:45:03,461][71635] Updated weights for policy 1, policy_version 16512 (0.0009) [2023-10-11 19:45:04,273][71601] Updated weights for policy 0, policy_version 16520 (0.0007) [2023-10-11 19:45:04,640][71601] Updated weights for policy 0, policy_version 16530 (0.0008) [2023-10-11 19:45:05,019][71601] Updated weights for policy 0, policy_version 16540 (0.0007) [2023-10-11 19:45:06,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 33849344. Throughput: 0: 1828.6, 1: 1803.6. Samples: 8467914. Policy #0 lag: (min: 6.0, avg: 11.4, max: 38.0) [2023-10-11 19:45:06,035][70582] Avg episode reward: [(0, '14.150'), (1, '12.840')] [2023-10-11 19:45:07,341][71635] Updated weights for policy 1, policy_version 16522 (0.0010) [2023-10-11 19:45:07,707][71635] Updated weights for policy 1, policy_version 16532 (0.0009) [2023-10-11 19:45:08,080][71635] Updated weights for policy 1, policy_version 16542 (0.0007) [2023-10-11 19:45:08,734][71601] Updated weights for policy 0, policy_version 16550 (0.0008) [2023-10-11 19:45:09,090][71601] Updated weights for policy 0, policy_version 16560 (0.0010) [2023-10-11 19:45:09,480][71601] Updated weights for policy 0, policy_version 16570 (0.0010) [2023-10-11 19:45:11,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 33914880. Throughput: 0: 1833.9, 1: 1797.3. Samples: 8489506. Policy #0 lag: (min: 9.0, avg: 14.2, max: 36.0) [2023-10-11 19:45:11,035][70582] Avg episode reward: [(0, '14.070'), (1, '13.260')] [2023-10-11 19:45:11,857][71635] Updated weights for policy 1, policy_version 16552 (0.0008) [2023-10-11 19:45:12,225][71635] Updated weights for policy 1, policy_version 16562 (0.0008) [2023-10-11 19:45:12,590][71635] Updated weights for policy 1, policy_version 16572 (0.0007) [2023-10-11 19:45:13,252][71601] Updated weights for policy 0, policy_version 16580 (0.0008) [2023-10-11 19:45:13,617][71601] Updated weights for policy 0, policy_version 16590 (0.0007) [2023-10-11 19:45:13,986][71601] Updated weights for policy 0, policy_version 16600 (0.0008) [2023-10-11 19:45:16,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 33980416. Throughput: 0: 1825.7, 1: 1797.6. Samples: 8500412. Policy #0 lag: (min: 9.0, avg: 14.2, max: 36.0) [2023-10-11 19:45:16,034][70582] Avg episode reward: [(0, '13.820'), (1, '13.580')] [2023-10-11 19:45:16,196][71635] Updated weights for policy 1, policy_version 16582 (0.0007) [2023-10-11 19:45:16,564][71635] Updated weights for policy 1, policy_version 16592 (0.0009) [2023-10-11 19:45:16,926][71635] Updated weights for policy 1, policy_version 16602 (0.0008) [2023-10-11 19:45:17,627][71601] Updated weights for policy 0, policy_version 16610 (0.0010) [2023-10-11 19:45:17,999][71601] Updated weights for policy 0, policy_version 16620 (0.0008) [2023-10-11 19:45:18,376][71601] Updated weights for policy 0, policy_version 16630 (0.0009) [2023-10-11 19:45:18,747][71601] Updated weights for policy 0, policy_version 16640 (0.0009) [2023-10-11 19:45:20,519][71635] Updated weights for policy 1, policy_version 16612 (0.0009) [2023-10-11 19:45:20,887][71635] Updated weights for policy 1, policy_version 16622 (0.0008) [2023-10-11 19:45:21,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 34045952. Throughput: 0: 1827.5, 1: 1805.6. Samples: 8522380. Policy #0 lag: (min: 9.0, avg: 14.2, max: 36.0) [2023-10-11 19:45:21,034][70582] Avg episode reward: [(0, '14.750'), (1, '13.480')] [2023-10-11 19:45:21,249][71635] Updated weights for policy 1, policy_version 16632 (0.0008) [2023-10-11 19:45:22,285][71601] Updated weights for policy 0, policy_version 16650 (0.0010) [2023-10-11 19:45:22,658][71601] Updated weights for policy 0, policy_version 16660 (0.0010) [2023-10-11 19:45:23,033][71601] Updated weights for policy 0, policy_version 16670 (0.0009) [2023-10-11 19:45:24,971][71635] Updated weights for policy 1, policy_version 16642 (0.0008) [2023-10-11 19:45:25,334][71635] Updated weights for policy 1, policy_version 16652 (0.0009) [2023-10-11 19:45:25,698][71635] Updated weights for policy 1, policy_version 16662 (0.0008) [2023-10-11 19:45:26,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 34111488. Throughput: 0: 1825.2, 1: 1812.8. Samples: 8544582. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-11 19:45:26,034][70582] Avg episode reward: [(0, '14.240'), (1, '13.150')] [2023-10-11 19:45:26,064][71635] Updated weights for policy 1, policy_version 16672 (0.0009) [2023-10-11 19:45:26,857][71601] Updated weights for policy 0, policy_version 16680 (0.0009) [2023-10-11 19:45:27,216][71601] Updated weights for policy 0, policy_version 16690 (0.0009) [2023-10-11 19:45:27,583][71601] Updated weights for policy 0, policy_version 16700 (0.0010) [2023-10-11 19:45:29,821][71635] Updated weights for policy 1, policy_version 16682 (0.0011) [2023-10-11 19:45:30,196][71635] Updated weights for policy 1, policy_version 16692 (0.0009) [2023-10-11 19:45:30,561][71635] Updated weights for policy 1, policy_version 16702 (0.0008) [2023-10-11 19:45:31,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 34209792. Throughput: 0: 1822.8, 1: 1802.8. Samples: 8555100. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-11 19:45:31,035][70582] Avg episode reward: [(0, '13.290'), (1, '13.200')] [2023-10-11 19:45:31,244][71601] Updated weights for policy 0, policy_version 16710 (0.0007) [2023-10-11 19:45:31,609][71601] Updated weights for policy 0, policy_version 16720 (0.0009) [2023-10-11 19:45:31,991][71601] Updated weights for policy 0, policy_version 16730 (0.0009) [2023-10-11 19:45:34,192][71635] Updated weights for policy 1, policy_version 16712 (0.0009) [2023-10-11 19:45:34,561][71635] Updated weights for policy 1, policy_version 16722 (0.0007) [2023-10-11 19:45:34,924][71635] Updated weights for policy 1, policy_version 16732 (0.0007) [2023-10-11 19:45:35,755][71601] Updated weights for policy 0, policy_version 16740 (0.0009) [2023-10-11 19:45:36,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 34275328. Throughput: 0: 1823.4, 1: 1807.1. Samples: 8577186. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-11 19:45:36,035][70582] Avg episode reward: [(0, '13.230'), (1, '13.570')] [2023-10-11 19:45:36,121][71601] Updated weights for policy 0, policy_version 16750 (0.0008) [2023-10-11 19:45:36,482][71601] Updated weights for policy 0, policy_version 16760 (0.0009) [2023-10-11 19:45:38,620][71635] Updated weights for policy 1, policy_version 16742 (0.0007) [2023-10-11 19:45:38,984][71635] Updated weights for policy 1, policy_version 16752 (0.0011) [2023-10-11 19:45:39,353][71635] Updated weights for policy 1, policy_version 16762 (0.0010) [2023-10-11 19:45:40,122][71601] Updated weights for policy 0, policy_version 16770 (0.0007) [2023-10-11 19:45:40,492][71601] Updated weights for policy 0, policy_version 16780 (0.0010) [2023-10-11 19:45:40,873][71601] Updated weights for policy 0, policy_version 16790 (0.0008) [2023-10-11 19:45:41,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 34340864. Throughput: 0: 1818.3, 1: 1803.9. Samples: 8598714. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:45:41,034][70582] Avg episode reward: [(0, '12.170'), (1, '14.350')] [2023-10-11 19:45:41,244][71601] Updated weights for policy 0, policy_version 16800 (0.0008) [2023-10-11 19:45:43,088][71635] Updated weights for policy 1, policy_version 16772 (0.0007) [2023-10-11 19:45:43,493][71635] Updated weights for policy 1, policy_version 16782 (0.0010) [2023-10-11 19:45:43,861][71635] Updated weights for policy 1, policy_version 16792 (0.0009) [2023-10-11 19:45:44,908][71601] Updated weights for policy 0, policy_version 16810 (0.0008) [2023-10-11 19:45:45,284][71601] Updated weights for policy 0, policy_version 16820 (0.0007) [2023-10-11 19:45:45,662][71601] Updated weights for policy 0, policy_version 16830 (0.0008) [2023-10-11 19:45:46,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 34439168. Throughput: 0: 1818.0, 1: 1813.9. Samples: 8610194. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:45:46,035][70582] Avg episode reward: [(0, '13.830'), (1, '14.320')] [2023-10-11 19:45:47,469][71635] Updated weights for policy 1, policy_version 16802 (0.0008) [2023-10-11 19:45:47,824][71635] Updated weights for policy 1, policy_version 16812 (0.0010) [2023-10-11 19:45:48,191][71635] Updated weights for policy 1, policy_version 16822 (0.0008) [2023-10-11 19:45:48,565][71635] Updated weights for policy 1, policy_version 16832 (0.0008) [2023-10-11 19:45:49,140][71601] Updated weights for policy 0, policy_version 16840 (0.0008) [2023-10-11 19:45:49,513][71601] Updated weights for policy 0, policy_version 16850 (0.0009) [2023-10-11 19:45:49,882][71601] Updated weights for policy 0, policy_version 16860 (0.0008) [2023-10-11 19:45:51,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 34504704. Throughput: 0: 1818.8, 1: 1817.5. Samples: 8631546. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-11 19:45:51,034][70582] Avg episode reward: [(0, '14.000'), (1, '14.790')] [2023-10-11 19:45:52,256][71635] Updated weights for policy 1, policy_version 16842 (0.0008) [2023-10-11 19:45:52,623][71635] Updated weights for policy 1, policy_version 16852 (0.0011) [2023-10-11 19:45:52,991][71635] Updated weights for policy 1, policy_version 16862 (0.0009) [2023-10-11 19:45:53,657][71601] Updated weights for policy 0, policy_version 16870 (0.0009) [2023-10-11 19:45:54,025][71601] Updated weights for policy 0, policy_version 16880 (0.0009) [2023-10-11 19:45:54,402][71601] Updated weights for policy 0, policy_version 16890 (0.0010) [2023-10-11 19:45:56,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 34570240. Throughput: 0: 1824.9, 1: 1820.8. Samples: 8653564. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-11 19:45:56,035][70582] Avg episode reward: [(0, '14.110'), (1, '12.780')] [2023-10-11 19:45:56,048][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000016896_17301504.pth... [2023-10-11 19:45:56,048][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000016864_17268736.pth... [2023-10-11 19:45:56,098][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000015168_15532032.pth [2023-10-11 19:45:56,099][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000015200_15564800.pth [2023-10-11 19:45:56,104][71431] Saving a milestone ./train_atari/atari_gopher_APPO/checkpoint_p1/milestones/checkpoint_000016864_17268736.pth [2023-10-11 19:45:56,105][71353] Saving a milestone ./train_atari/atari_gopher_APPO/checkpoint_p0/milestones/checkpoint_000016896_17301504.pth [2023-10-11 19:45:56,676][71635] Updated weights for policy 1, policy_version 16872 (0.0008) [2023-10-11 19:45:57,037][71635] Updated weights for policy 1, policy_version 16882 (0.0007) [2023-10-11 19:45:57,413][71635] Updated weights for policy 1, policy_version 16892 (0.0008) [2023-10-11 19:45:58,081][71601] Updated weights for policy 0, policy_version 16900 (0.0009) [2023-10-11 19:45:58,453][71601] Updated weights for policy 0, policy_version 16910 (0.0008) [2023-10-11 19:45:58,822][71601] Updated weights for policy 0, policy_version 16920 (0.0007) [2023-10-11 19:46:01,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 34635776. Throughput: 0: 1821.9, 1: 1820.2. Samples: 8664306. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-11 19:46:01,035][70582] Avg episode reward: [(0, '14.540'), (1, '13.610')] [2023-10-11 19:46:01,071][71635] Updated weights for policy 1, policy_version 16902 (0.0008) [2023-10-11 19:46:01,439][71635] Updated weights for policy 1, policy_version 16912 (0.0009) [2023-10-11 19:46:01,814][71635] Updated weights for policy 1, policy_version 16922 (0.0011) [2023-10-11 19:46:02,600][71601] Updated weights for policy 0, policy_version 16930 (0.0007) [2023-10-11 19:46:02,972][71601] Updated weights for policy 0, policy_version 16940 (0.0009) [2023-10-11 19:46:03,356][71601] Updated weights for policy 0, policy_version 16950 (0.0009) [2023-10-11 19:46:03,719][71601] Updated weights for policy 0, policy_version 16960 (0.0008) [2023-10-11 19:46:05,543][71635] Updated weights for policy 1, policy_version 16932 (0.0009) [2023-10-11 19:46:05,902][71635] Updated weights for policy 1, policy_version 16942 (0.0007) [2023-10-11 19:46:06,034][70582] Fps is (10 sec: 13107.8, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 34701312. Throughput: 0: 1822.7, 1: 1815.8. Samples: 8686114. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-11 19:46:06,034][70582] Avg episode reward: [(0, '14.040'), (1, '12.480')] [2023-10-11 19:46:06,267][71635] Updated weights for policy 1, policy_version 16952 (0.0008) [2023-10-11 19:46:07,266][71601] Updated weights for policy 0, policy_version 16970 (0.0009) [2023-10-11 19:46:07,627][71601] Updated weights for policy 0, policy_version 16980 (0.0010) [2023-10-11 19:46:08,001][71601] Updated weights for policy 0, policy_version 16990 (0.0010) [2023-10-11 19:46:10,173][71635] Updated weights for policy 1, policy_version 16962 (0.0008) [2023-10-11 19:46:10,545][71635] Updated weights for policy 1, policy_version 16972 (0.0010) [2023-10-11 19:46:10,908][71635] Updated weights for policy 1, policy_version 16982 (0.0010) [2023-10-11 19:46:11,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 34766848. Throughput: 0: 1818.0, 1: 1818.4. Samples: 8708218. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-11 19:46:11,034][70582] Avg episode reward: [(0, '13.680'), (1, '13.570')] [2023-10-11 19:46:11,284][71635] Updated weights for policy 1, policy_version 16992 (0.0007) [2023-10-11 19:46:11,729][71601] Updated weights for policy 0, policy_version 17000 (0.0008) [2023-10-11 19:46:12,109][71601] Updated weights for policy 0, policy_version 17010 (0.0008) [2023-10-11 19:46:12,474][71601] Updated weights for policy 0, policy_version 17020 (0.0008) [2023-10-11 19:46:14,969][71635] Updated weights for policy 1, policy_version 17002 (0.0007) [2023-10-11 19:46:15,326][71635] Updated weights for policy 1, policy_version 17012 (0.0011) [2023-10-11 19:46:15,695][71635] Updated weights for policy 1, policy_version 17022 (0.0011) [2023-10-11 19:46:16,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 34865152. Throughput: 0: 1815.7, 1: 1812.4. Samples: 8718364. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-11 19:46:16,035][70582] Avg episode reward: [(0, '12.530'), (1, '12.400')] [2023-10-11 19:46:16,184][71601] Updated weights for policy 0, policy_version 17030 (0.0007) [2023-10-11 19:46:16,552][71601] Updated weights for policy 0, policy_version 17040 (0.0008) [2023-10-11 19:46:16,921][71601] Updated weights for policy 0, policy_version 17050 (0.0008) [2023-10-11 19:46:19,484][71635] Updated weights for policy 1, policy_version 17032 (0.0008) [2023-10-11 19:46:19,855][71635] Updated weights for policy 1, policy_version 17042 (0.0008) [2023-10-11 19:46:20,215][71635] Updated weights for policy 1, policy_version 17052 (0.0007) [2023-10-11 19:46:20,773][71601] Updated weights for policy 0, policy_version 17060 (0.0010) [2023-10-11 19:46:21,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 34930688. Throughput: 0: 1816.1, 1: 1822.0. Samples: 8740900. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-11 19:46:21,034][70582] Avg episode reward: [(0, '12.210'), (1, '11.390')] [2023-10-11 19:46:21,156][71601] Updated weights for policy 0, policy_version 17070 (0.0008) [2023-10-11 19:46:21,525][71601] Updated weights for policy 0, policy_version 17080 (0.0009) [2023-10-11 19:46:23,781][71635] Updated weights for policy 1, policy_version 17062 (0.0008) [2023-10-11 19:46:24,138][71635] Updated weights for policy 1, policy_version 17072 (0.0009) [2023-10-11 19:46:24,507][71635] Updated weights for policy 1, policy_version 17082 (0.0008) [2023-10-11 19:46:25,104][71601] Updated weights for policy 0, policy_version 17090 (0.0008) [2023-10-11 19:46:25,485][71601] Updated weights for policy 0, policy_version 17100 (0.0009) [2023-10-11 19:46:25,856][71601] Updated weights for policy 0, policy_version 17110 (0.0007) [2023-10-11 19:46:26,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 34996224. Throughput: 0: 1822.2, 1: 1814.0. Samples: 8762342. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-11 19:46:26,034][70582] Avg episode reward: [(0, '11.810'), (1, '12.170')] [2023-10-11 19:46:26,218][71601] Updated weights for policy 0, policy_version 17120 (0.0009) [2023-10-11 19:46:28,213][71635] Updated weights for policy 1, policy_version 17092 (0.0007) [2023-10-11 19:46:28,609][71635] Updated weights for policy 1, policy_version 17102 (0.0010) [2023-10-11 19:46:28,980][71635] Updated weights for policy 1, policy_version 17112 (0.0009) [2023-10-11 19:46:29,852][71601] Updated weights for policy 0, policy_version 17130 (0.0008) [2023-10-11 19:46:30,225][71601] Updated weights for policy 0, policy_version 17140 (0.0008) [2023-10-11 19:46:30,603][71601] Updated weights for policy 0, policy_version 17150 (0.0007) [2023-10-11 19:46:31,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 35094528. Throughput: 0: 1820.2, 1: 1815.6. Samples: 8773806. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-11 19:46:31,035][70582] Avg episode reward: [(0, '11.910'), (1, '12.720')] [2023-10-11 19:46:32,644][71635] Updated weights for policy 1, policy_version 17122 (0.0009) [2023-10-11 19:46:33,020][71635] Updated weights for policy 1, policy_version 17132 (0.0009) [2023-10-11 19:46:33,387][71635] Updated weights for policy 1, policy_version 17142 (0.0010) [2023-10-11 19:46:33,754][71635] Updated weights for policy 1, policy_version 17152 (0.0007) [2023-10-11 19:46:34,271][71601] Updated weights for policy 0, policy_version 17160 (0.0008) [2023-10-11 19:46:34,645][71601] Updated weights for policy 0, policy_version 17170 (0.0008) [2023-10-11 19:46:35,032][71601] Updated weights for policy 0, policy_version 17180 (0.0010) [2023-10-11 19:46:36,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 35160064. Throughput: 0: 1820.0, 1: 1812.5. Samples: 8795010. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-11 19:46:36,034][70582] Avg episode reward: [(0, '12.860'), (1, '14.030')] [2023-10-11 19:46:37,424][71635] Updated weights for policy 1, policy_version 17162 (0.0007) [2023-10-11 19:46:37,795][71635] Updated weights for policy 1, policy_version 17172 (0.0007) [2023-10-11 19:46:38,157][71635] Updated weights for policy 1, policy_version 17182 (0.0009) [2023-10-11 19:46:38,632][71601] Updated weights for policy 0, policy_version 17190 (0.0007) [2023-10-11 19:46:38,994][71601] Updated weights for policy 0, policy_version 17200 (0.0009) [2023-10-11 19:46:39,368][71601] Updated weights for policy 0, policy_version 17210 (0.0009) [2023-10-11 19:46:41,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 35225600. Throughput: 0: 1817.9, 1: 1820.8. Samples: 8817304. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-11 19:46:41,035][70582] Avg episode reward: [(0, '13.640'), (1, '14.950')] [2023-10-11 19:46:41,846][71635] Updated weights for policy 1, policy_version 17192 (0.0008) [2023-10-11 19:46:42,204][71635] Updated weights for policy 1, policy_version 17202 (0.0008) [2023-10-11 19:46:42,573][71635] Updated weights for policy 1, policy_version 17212 (0.0011) [2023-10-11 19:46:43,150][71601] Updated weights for policy 0, policy_version 17220 (0.0007) [2023-10-11 19:46:43,509][71601] Updated weights for policy 0, policy_version 17230 (0.0008) [2023-10-11 19:46:43,888][71601] Updated weights for policy 0, policy_version 17240 (0.0009) [2023-10-11 19:46:46,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 35291136. Throughput: 0: 1822.4, 1: 1823.2. Samples: 8828360. Policy #0 lag: (min: 26.0, avg: 26.2, max: 35.0) [2023-10-11 19:46:46,035][70582] Avg episode reward: [(0, '14.580'), (1, '14.980')] [2023-10-11 19:46:46,230][71635] Updated weights for policy 1, policy_version 17222 (0.0010) [2023-10-11 19:46:46,594][71635] Updated weights for policy 1, policy_version 17232 (0.0007) [2023-10-11 19:46:46,961][71635] Updated weights for policy 1, policy_version 17242 (0.0008) [2023-10-11 19:46:47,456][71601] Updated weights for policy 0, policy_version 17250 (0.0009) [2023-10-11 19:46:47,822][71601] Updated weights for policy 0, policy_version 17260 (0.0009) [2023-10-11 19:46:48,201][71601] Updated weights for policy 0, policy_version 17270 (0.0008) [2023-10-11 19:46:48,565][71601] Updated weights for policy 0, policy_version 17280 (0.0008) [2023-10-11 19:46:50,563][71635] Updated weights for policy 1, policy_version 17252 (0.0009) [2023-10-11 19:46:50,932][71635] Updated weights for policy 1, policy_version 17262 (0.0009) [2023-10-11 19:46:51,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 35356672. Throughput: 0: 1820.4, 1: 1829.7. Samples: 8850370. Policy #0 lag: (min: 26.0, avg: 26.2, max: 35.0) [2023-10-11 19:46:51,035][70582] Avg episode reward: [(0, '13.980'), (1, '14.530')] [2023-10-11 19:46:51,288][71635] Updated weights for policy 1, policy_version 17272 (0.0008) [2023-10-11 19:46:52,272][71601] Updated weights for policy 0, policy_version 17290 (0.0009) [2023-10-11 19:46:52,638][71601] Updated weights for policy 0, policy_version 17300 (0.0008) [2023-10-11 19:46:53,013][71601] Updated weights for policy 0, policy_version 17310 (0.0007) [2023-10-11 19:46:55,173][71635] Updated weights for policy 1, policy_version 17282 (0.0008) [2023-10-11 19:46:55,532][71635] Updated weights for policy 1, policy_version 17292 (0.0009) [2023-10-11 19:46:55,905][71635] Updated weights for policy 1, policy_version 17302 (0.0008) [2023-10-11 19:46:56,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.6, 300 sec: 14440.1). Total num frames: 35422208. Throughput: 0: 1832.9, 1: 1827.6. Samples: 8872940. Policy #0 lag: (min: 26.0, avg: 26.2, max: 35.0) [2023-10-11 19:46:56,034][70582] Avg episode reward: [(0, '12.960'), (1, '15.060')] [2023-10-11 19:46:56,264][71635] Updated weights for policy 1, policy_version 17312 (0.0008) [2023-10-11 19:46:56,703][71601] Updated weights for policy 0, policy_version 17320 (0.0010) [2023-10-11 19:46:57,084][71601] Updated weights for policy 0, policy_version 17330 (0.0009) [2023-10-11 19:46:57,449][71601] Updated weights for policy 0, policy_version 17340 (0.0008) [2023-10-11 19:47:00,051][71635] Updated weights for policy 1, policy_version 17322 (0.0009) [2023-10-11 19:47:00,425][71635] Updated weights for policy 1, policy_version 17332 (0.0008) [2023-10-11 19:47:00,792][71635] Updated weights for policy 1, policy_version 17342 (0.0007) [2023-10-11 19:47:01,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 35520512. Throughput: 0: 1832.2, 1: 1825.4. Samples: 8882958. Policy #0 lag: (min: 15.0, avg: 22.7, max: 47.0) [2023-10-11 19:47:01,035][70582] Avg episode reward: [(0, '12.650'), (1, '14.570')] [2023-10-11 19:47:01,264][71601] Updated weights for policy 0, policy_version 17350 (0.0009) [2023-10-11 19:47:01,633][71601] Updated weights for policy 0, policy_version 17360 (0.0007) [2023-10-11 19:47:02,009][71601] Updated weights for policy 0, policy_version 17370 (0.0008) [2023-10-11 19:47:04,755][71635] Updated weights for policy 1, policy_version 17352 (0.0009) [2023-10-11 19:47:05,123][71635] Updated weights for policy 1, policy_version 17362 (0.0007) [2023-10-11 19:47:05,489][71635] Updated weights for policy 1, policy_version 17372 (0.0009) [2023-10-11 19:47:05,762][71601] Updated weights for policy 0, policy_version 17380 (0.0009) [2023-10-11 19:47:06,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 35586048. Throughput: 0: 1830.9, 1: 1820.5. Samples: 8905212. Policy #0 lag: (min: 15.0, avg: 22.7, max: 47.0) [2023-10-11 19:47:06,035][70582] Avg episode reward: [(0, '12.490'), (1, '14.070')] [2023-10-11 19:47:06,131][71601] Updated weights for policy 0, policy_version 17390 (0.0007) [2023-10-11 19:47:06,513][71601] Updated weights for policy 0, policy_version 17400 (0.0008) [2023-10-11 19:47:09,095][71635] Updated weights for policy 1, policy_version 17382 (0.0007) [2023-10-11 19:47:09,473][71635] Updated weights for policy 1, policy_version 17392 (0.0008) [2023-10-11 19:47:09,831][71635] Updated weights for policy 1, policy_version 17402 (0.0011) [2023-10-11 19:47:10,117][71601] Updated weights for policy 0, policy_version 17410 (0.0011) [2023-10-11 19:47:10,488][71601] Updated weights for policy 0, policy_version 17420 (0.0008) [2023-10-11 19:47:10,854][71601] Updated weights for policy 0, policy_version 17430 (0.0011) [2023-10-11 19:47:11,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 35651584. Throughput: 0: 1824.5, 1: 1810.7. Samples: 8925928. Policy #0 lag: (min: 15.0, avg: 22.7, max: 47.0) [2023-10-11 19:47:11,034][70582] Avg episode reward: [(0, '13.350'), (1, '12.890')] [2023-10-11 19:47:11,229][71601] Updated weights for policy 0, policy_version 17440 (0.0010) [2023-10-11 19:47:13,709][71635] Updated weights for policy 1, policy_version 17412 (0.0009) [2023-10-11 19:47:14,109][71635] Updated weights for policy 1, policy_version 17422 (0.0008) [2023-10-11 19:47:14,482][71635] Updated weights for policy 1, policy_version 17432 (0.0010) [2023-10-11 19:47:14,950][71601] Updated weights for policy 0, policy_version 17450 (0.0008) [2023-10-11 19:47:15,320][71601] Updated weights for policy 0, policy_version 17460 (0.0007) [2023-10-11 19:47:15,693][71601] Updated weights for policy 0, policy_version 17470 (0.0007) [2023-10-11 19:47:16,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 35749888. Throughput: 0: 1822.8, 1: 1820.4. Samples: 8937746. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 19:47:16,035][70582] Avg episode reward: [(0, '14.440'), (1, '12.510')] [2023-10-11 19:47:18,005][71635] Updated weights for policy 1, policy_version 17442 (0.0009) [2023-10-11 19:47:18,377][71635] Updated weights for policy 1, policy_version 17452 (0.0008) [2023-10-11 19:47:18,753][71635] Updated weights for policy 1, policy_version 17462 (0.0008) [2023-10-11 19:47:19,121][71635] Updated weights for policy 1, policy_version 17472 (0.0008) [2023-10-11 19:47:19,327][71601] Updated weights for policy 0, policy_version 17480 (0.0007) [2023-10-11 19:47:19,704][71601] Updated weights for policy 0, policy_version 17490 (0.0009) [2023-10-11 19:47:20,076][71601] Updated weights for policy 0, policy_version 17500 (0.0009) [2023-10-11 19:47:21,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 35815424. Throughput: 0: 1824.3, 1: 1805.5. Samples: 8958348. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 19:47:21,035][70582] Avg episode reward: [(0, '14.940'), (1, '11.790')] [2023-10-11 19:47:22,628][71635] Updated weights for policy 1, policy_version 17482 (0.0008) [2023-10-11 19:47:22,984][71635] Updated weights for policy 1, policy_version 17492 (0.0007) [2023-10-11 19:47:23,355][71635] Updated weights for policy 1, policy_version 17502 (0.0007) [2023-10-11 19:47:23,629][71601] Updated weights for policy 0, policy_version 17510 (0.0008) [2023-10-11 19:47:24,016][71601] Updated weights for policy 0, policy_version 17520 (0.0009) [2023-10-11 19:47:24,386][71601] Updated weights for policy 0, policy_version 17530 (0.0008) [2023-10-11 19:47:26,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 35880960. Throughput: 0: 1817.5, 1: 1797.8. Samples: 8979990. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 19:47:26,035][70582] Avg episode reward: [(0, '16.330'), (1, '11.930')] [2023-10-11 19:47:27,041][71635] Updated weights for policy 1, policy_version 17512 (0.0007) [2023-10-11 19:47:27,414][71635] Updated weights for policy 1, policy_version 17522 (0.0011) [2023-10-11 19:47:27,791][71635] Updated weights for policy 1, policy_version 17532 (0.0009) [2023-10-11 19:47:28,041][71601] Updated weights for policy 0, policy_version 17540 (0.0008) [2023-10-11 19:47:28,411][71601] Updated weights for policy 0, policy_version 17550 (0.0008) [2023-10-11 19:47:28,771][71601] Updated weights for policy 0, policy_version 17560 (0.0009) [2023-10-11 19:47:31,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 35946496. Throughput: 0: 1813.5, 1: 1796.5. Samples: 8990806. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 19:47:31,034][70582] Avg episode reward: [(0, '16.880'), (1, '12.790')] [2023-10-11 19:47:31,035][71353] Saving new best policy, reward=16.880! [2023-10-11 19:47:31,558][71635] Updated weights for policy 1, policy_version 17542 (0.0008) [2023-10-11 19:47:31,922][71635] Updated weights for policy 1, policy_version 17552 (0.0007) [2023-10-11 19:47:32,282][71635] Updated weights for policy 1, policy_version 17562 (0.0007) [2023-10-11 19:47:32,604][71601] Updated weights for policy 0, policy_version 17570 (0.0009) [2023-10-11 19:47:32,978][71601] Updated weights for policy 0, policy_version 17580 (0.0008) [2023-10-11 19:47:33,352][71601] Updated weights for policy 0, policy_version 17590 (0.0007) [2023-10-11 19:47:33,723][71601] Updated weights for policy 0, policy_version 17600 (0.0007) [2023-10-11 19:47:35,832][71635] Updated weights for policy 1, policy_version 17572 (0.0007) [2023-10-11 19:47:36,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 36012032. Throughput: 0: 1819.1, 1: 1791.6. Samples: 9012852. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 19:47:36,034][70582] Avg episode reward: [(0, '15.400'), (1, '13.450')] [2023-10-11 19:47:36,203][71635] Updated weights for policy 1, policy_version 17582 (0.0007) [2023-10-11 19:47:36,575][71635] Updated weights for policy 1, policy_version 17592 (0.0007) [2023-10-11 19:47:37,397][71601] Updated weights for policy 0, policy_version 17610 (0.0008) [2023-10-11 19:47:37,772][71601] Updated weights for policy 0, policy_version 17620 (0.0008) [2023-10-11 19:47:38,149][71601] Updated weights for policy 0, policy_version 17630 (0.0010) [2023-10-11 19:47:40,464][71635] Updated weights for policy 1, policy_version 17602 (0.0007) [2023-10-11 19:47:40,829][71635] Updated weights for policy 1, policy_version 17612 (0.0008) [2023-10-11 19:47:41,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 36077568. Throughput: 0: 1807.7, 1: 1808.5. Samples: 9035668. Policy #0 lag: (min: 24.0, avg: 48.6, max: 56.0) [2023-10-11 19:47:41,034][70582] Avg episode reward: [(0, '14.480'), (1, '14.860')] [2023-10-11 19:47:41,198][71635] Updated weights for policy 1, policy_version 17622 (0.0008) [2023-10-11 19:47:41,558][71635] Updated weights for policy 1, policy_version 17632 (0.0008) [2023-10-11 19:47:41,912][71601] Updated weights for policy 0, policy_version 17640 (0.0007) [2023-10-11 19:47:42,289][71601] Updated weights for policy 0, policy_version 17650 (0.0007) [2023-10-11 19:47:42,657][71601] Updated weights for policy 0, policy_version 17660 (0.0007) [2023-10-11 19:47:45,158][71635] Updated weights for policy 1, policy_version 17642 (0.0009) [2023-10-11 19:47:45,527][71635] Updated weights for policy 1, policy_version 17652 (0.0007) [2023-10-11 19:47:45,897][71635] Updated weights for policy 1, policy_version 17662 (0.0007) [2023-10-11 19:47:46,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 36175872. Throughput: 0: 1806.6, 1: 1804.8. Samples: 9045470. Policy #0 lag: (min: 24.0, avg: 48.6, max: 56.0) [2023-10-11 19:47:46,034][70582] Avg episode reward: [(0, '14.300'), (1, '15.090')] [2023-10-11 19:47:46,309][71601] Updated weights for policy 0, policy_version 17670 (0.0008) [2023-10-11 19:47:46,670][71601] Updated weights for policy 0, policy_version 17680 (0.0010) [2023-10-11 19:47:47,044][71601] Updated weights for policy 0, policy_version 17690 (0.0007) [2023-10-11 19:47:49,544][71635] Updated weights for policy 1, policy_version 17672 (0.0008) [2023-10-11 19:47:49,910][71635] Updated weights for policy 1, policy_version 17682 (0.0008) [2023-10-11 19:47:50,274][71635] Updated weights for policy 1, policy_version 17692 (0.0007) [2023-10-11 19:47:50,585][71601] Updated weights for policy 0, policy_version 17700 (0.0007) [2023-10-11 19:47:50,968][71601] Updated weights for policy 0, policy_version 17710 (0.0009) [2023-10-11 19:47:51,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 36241408. Throughput: 0: 1810.0, 1: 1810.8. Samples: 9068148. Policy #0 lag: (min: 24.0, avg: 48.6, max: 56.0) [2023-10-11 19:47:51,035][70582] Avg episode reward: [(0, '13.300'), (1, '15.640')] [2023-10-11 19:47:51,340][71601] Updated weights for policy 0, policy_version 17720 (0.0008) [2023-10-11 19:47:53,975][71635] Updated weights for policy 1, policy_version 17702 (0.0009) [2023-10-11 19:47:54,349][71635] Updated weights for policy 1, policy_version 17712 (0.0007) [2023-10-11 19:47:54,707][71635] Updated weights for policy 1, policy_version 17722 (0.0007) [2023-10-11 19:47:55,117][71601] Updated weights for policy 0, policy_version 17730 (0.0009) [2023-10-11 19:47:55,485][71601] Updated weights for policy 0, policy_version 17740 (0.0008) [2023-10-11 19:47:55,857][71601] Updated weights for policy 0, policy_version 17750 (0.0008) [2023-10-11 19:47:56,034][70582] Fps is (10 sec: 13106.5, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 36306944. Throughput: 0: 1812.5, 1: 1816.0. Samples: 9089212. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:47:56,035][70582] Avg episode reward: [(0, '13.720'), (1, '15.450')] [2023-10-11 19:47:56,047][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000017728_18153472.pth... [2023-10-11 19:47:56,082][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000016032_16416768.pth [2023-10-11 19:47:56,228][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000017760_18186240.pth... [2023-10-11 19:47:56,231][71601] Updated weights for policy 0, policy_version 17760 (0.0008) [2023-10-11 19:47:56,257][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000016032_16416768.pth [2023-10-11 19:47:58,548][71635] Updated weights for policy 1, policy_version 17732 (0.0008) [2023-10-11 19:47:58,947][71635] Updated weights for policy 1, policy_version 17742 (0.0007) [2023-10-11 19:47:59,308][71635] Updated weights for policy 1, policy_version 17752 (0.0010) [2023-10-11 19:48:00,069][71601] Updated weights for policy 0, policy_version 17770 (0.0008) [2023-10-11 19:48:00,440][71601] Updated weights for policy 0, policy_version 17780 (0.0010) [2023-10-11 19:48:00,819][71601] Updated weights for policy 0, policy_version 17790 (0.0009) [2023-10-11 19:48:01,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 36405248. Throughput: 0: 1810.9, 1: 1816.6. Samples: 9100984. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:48:01,034][70582] Avg episode reward: [(0, '14.470'), (1, '15.570')] [2023-10-11 19:48:02,881][71635] Updated weights for policy 1, policy_version 17762 (0.0009) [2023-10-11 19:48:03,257][71635] Updated weights for policy 1, policy_version 17772 (0.0009) [2023-10-11 19:48:03,622][71635] Updated weights for policy 1, policy_version 17782 (0.0008) [2023-10-11 19:48:03,989][71635] Updated weights for policy 1, policy_version 17792 (0.0010) [2023-10-11 19:48:04,575][71601] Updated weights for policy 0, policy_version 17800 (0.0008) [2023-10-11 19:48:04,950][71601] Updated weights for policy 0, policy_version 17810 (0.0009) [2023-10-11 19:48:05,333][71601] Updated weights for policy 0, policy_version 17820 (0.0007) [2023-10-11 19:48:06,034][70582] Fps is (10 sec: 16384.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 36470784. Throughput: 0: 1815.9, 1: 1819.8. Samples: 9121952. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-11 19:48:06,034][70582] Avg episode reward: [(0, '14.160'), (1, '13.080')] [2023-10-11 19:48:07,687][71635] Updated weights for policy 1, policy_version 17802 (0.0011) [2023-10-11 19:48:08,050][71635] Updated weights for policy 1, policy_version 17812 (0.0011) [2023-10-11 19:48:08,419][71635] Updated weights for policy 1, policy_version 17822 (0.0007) [2023-10-11 19:48:09,011][71601] Updated weights for policy 0, policy_version 17830 (0.0008) [2023-10-11 19:48:09,387][71601] Updated weights for policy 0, policy_version 17840 (0.0009) [2023-10-11 19:48:09,755][71601] Updated weights for policy 0, policy_version 17850 (0.0007) [2023-10-11 19:48:11,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 36536320. Throughput: 0: 1814.1, 1: 1822.7. Samples: 9143644. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-11 19:48:11,034][70582] Avg episode reward: [(0, '15.410'), (1, '13.070')] [2023-10-11 19:48:12,134][71635] Updated weights for policy 1, policy_version 17832 (0.0009) [2023-10-11 19:48:12,505][71635] Updated weights for policy 1, policy_version 17842 (0.0007) [2023-10-11 19:48:12,863][71635] Updated weights for policy 1, policy_version 17852 (0.0009) [2023-10-11 19:48:13,358][71601] Updated weights for policy 0, policy_version 17860 (0.0007) [2023-10-11 19:48:13,734][71601] Updated weights for policy 0, policy_version 17870 (0.0007) [2023-10-11 19:48:14,114][71601] Updated weights for policy 0, policy_version 17880 (0.0009) [2023-10-11 19:48:16,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 36601856. Throughput: 0: 1825.4, 1: 1820.5. Samples: 9154872. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-11 19:48:16,035][70582] Avg episode reward: [(0, '15.050'), (1, '12.150')] [2023-10-11 19:48:16,488][71635] Updated weights for policy 1, policy_version 17862 (0.0007) [2023-10-11 19:48:16,857][71635] Updated weights for policy 1, policy_version 17872 (0.0007) [2023-10-11 19:48:17,225][71635] Updated weights for policy 1, policy_version 17882 (0.0007) [2023-10-11 19:48:17,679][71601] Updated weights for policy 0, policy_version 17890 (0.0009) [2023-10-11 19:48:18,043][71601] Updated weights for policy 0, policy_version 17900 (0.0007) [2023-10-11 19:48:18,415][71601] Updated weights for policy 0, policy_version 17910 (0.0008) [2023-10-11 19:48:18,786][71601] Updated weights for policy 0, policy_version 17920 (0.0007) [2023-10-11 19:48:20,811][71635] Updated weights for policy 1, policy_version 17892 (0.0009) [2023-10-11 19:48:21,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 36667392. Throughput: 0: 1819.0, 1: 1825.6. Samples: 9176860. Policy #0 lag: (min: 22.0, avg: 26.1, max: 54.0) [2023-10-11 19:48:21,034][70582] Avg episode reward: [(0, '15.960'), (1, '12.630')] [2023-10-11 19:48:21,184][71635] Updated weights for policy 1, policy_version 17902 (0.0011) [2023-10-11 19:48:21,556][71635] Updated weights for policy 1, policy_version 17912 (0.0009) [2023-10-11 19:48:22,460][71601] Updated weights for policy 0, policy_version 17930 (0.0007) [2023-10-11 19:48:22,829][71601] Updated weights for policy 0, policy_version 17940 (0.0009) [2023-10-11 19:48:23,204][71601] Updated weights for policy 0, policy_version 17950 (0.0011) [2023-10-11 19:48:25,310][71635] Updated weights for policy 1, policy_version 17922 (0.0009) [2023-10-11 19:48:25,686][71635] Updated weights for policy 1, policy_version 17932 (0.0007) [2023-10-11 19:48:26,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 36732928. Throughput: 0: 1825.7, 1: 1822.3. Samples: 9199828. Policy #0 lag: (min: 22.0, avg: 26.1, max: 54.0) [2023-10-11 19:48:26,034][70582] Avg episode reward: [(0, '17.420'), (1, '13.040')] [2023-10-11 19:48:26,042][71353] Saving new best policy, reward=17.420! [2023-10-11 19:48:26,051][71635] Updated weights for policy 1, policy_version 17942 (0.0007) [2023-10-11 19:48:26,417][71635] Updated weights for policy 1, policy_version 17952 (0.0008) [2023-10-11 19:48:26,828][71601] Updated weights for policy 0, policy_version 17960 (0.0008) [2023-10-11 19:48:27,204][71601] Updated weights for policy 0, policy_version 17970 (0.0007) [2023-10-11 19:48:27,578][71601] Updated weights for policy 0, policy_version 17980 (0.0010) [2023-10-11 19:48:29,957][71635] Updated weights for policy 1, policy_version 17962 (0.0009) [2023-10-11 19:48:30,321][71635] Updated weights for policy 1, policy_version 17972 (0.0009) [2023-10-11 19:48:30,693][71635] Updated weights for policy 1, policy_version 17982 (0.0008) [2023-10-11 19:48:31,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 36831232. Throughput: 0: 1827.3, 1: 1823.6. Samples: 9209762. Policy #0 lag: (min: 22.0, avg: 26.1, max: 54.0) [2023-10-11 19:48:31,034][70582] Avg episode reward: [(0, '17.100'), (1, '14.900')] [2023-10-11 19:48:31,296][71601] Updated weights for policy 0, policy_version 17990 (0.0009) [2023-10-11 19:48:31,660][71601] Updated weights for policy 0, policy_version 18000 (0.0011) [2023-10-11 19:48:32,034][71601] Updated weights for policy 0, policy_version 18010 (0.0009) [2023-10-11 19:48:34,417][71635] Updated weights for policy 1, policy_version 17992 (0.0008) [2023-10-11 19:48:34,776][71635] Updated weights for policy 1, policy_version 18002 (0.0007) [2023-10-11 19:48:35,144][71635] Updated weights for policy 1, policy_version 18012 (0.0008) [2023-10-11 19:48:35,855][71601] Updated weights for policy 0, policy_version 18020 (0.0008) [2023-10-11 19:48:36,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 36896768. Throughput: 0: 1819.7, 1: 1823.8. Samples: 9232108. Policy #0 lag: (min: 38.0, avg: 53.9, max: 56.0) [2023-10-11 19:48:36,035][70582] Avg episode reward: [(0, '16.570'), (1, '14.750')] [2023-10-11 19:48:36,220][71601] Updated weights for policy 0, policy_version 18030 (0.0008) [2023-10-11 19:48:36,588][71601] Updated weights for policy 0, policy_version 18040 (0.0008) [2023-10-11 19:48:38,913][71635] Updated weights for policy 1, policy_version 18022 (0.0008) [2023-10-11 19:48:39,277][71635] Updated weights for policy 1, policy_version 18032 (0.0010) [2023-10-11 19:48:39,635][71635] Updated weights for policy 1, policy_version 18042 (0.0008) [2023-10-11 19:48:40,201][71601] Updated weights for policy 0, policy_version 18050 (0.0007) [2023-10-11 19:48:40,572][71601] Updated weights for policy 0, policy_version 18060 (0.0008) [2023-10-11 19:48:40,943][71601] Updated weights for policy 0, policy_version 18070 (0.0007) [2023-10-11 19:48:41,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 36962304. Throughput: 0: 1824.3, 1: 1825.2. Samples: 9253436. Policy #0 lag: (min: 38.0, avg: 53.9, max: 56.0) [2023-10-11 19:48:41,035][70582] Avg episode reward: [(0, '16.360'), (1, '14.940')] [2023-10-11 19:48:41,324][71601] Updated weights for policy 0, policy_version 18080 (0.0008) [2023-10-11 19:48:43,282][71635] Updated weights for policy 1, policy_version 18052 (0.0008) [2023-10-11 19:48:43,653][71635] Updated weights for policy 1, policy_version 18062 (0.0008) [2023-10-11 19:48:44,020][71635] Updated weights for policy 1, policy_version 18072 (0.0009) [2023-10-11 19:48:45,062][71601] Updated weights for policy 0, policy_version 18090 (0.0007) [2023-10-11 19:48:45,430][71601] Updated weights for policy 0, policy_version 18100 (0.0009) [2023-10-11 19:48:45,799][71601] Updated weights for policy 0, policy_version 18110 (0.0008) [2023-10-11 19:48:46,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 37060608. Throughput: 0: 1825.7, 1: 1823.7. Samples: 9265206. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 19:48:46,034][70582] Avg episode reward: [(0, '14.730'), (1, '14.700')] [2023-10-11 19:48:47,734][71635] Updated weights for policy 1, policy_version 18082 (0.0010) [2023-10-11 19:48:48,144][71635] Updated weights for policy 1, policy_version 18092 (0.0010) [2023-10-11 19:48:48,511][71635] Updated weights for policy 1, policy_version 18102 (0.0009) [2023-10-11 19:48:48,878][71635] Updated weights for policy 1, policy_version 18112 (0.0011) [2023-10-11 19:48:49,514][71601] Updated weights for policy 0, policy_version 18120 (0.0009) [2023-10-11 19:48:49,891][71601] Updated weights for policy 0, policy_version 18130 (0.0008) [2023-10-11 19:48:50,253][71601] Updated weights for policy 0, policy_version 18140 (0.0008) [2023-10-11 19:48:51,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 37126144. Throughput: 0: 1828.9, 1: 1825.6. Samples: 9286404. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 19:48:51,034][70582] Avg episode reward: [(0, '14.000'), (1, '14.000')] [2023-10-11 19:48:52,473][71635] Updated weights for policy 1, policy_version 18122 (0.0010) [2023-10-11 19:48:52,832][71635] Updated weights for policy 1, policy_version 18132 (0.0007) [2023-10-11 19:48:53,199][71635] Updated weights for policy 1, policy_version 18142 (0.0008) [2023-10-11 19:48:53,828][71601] Updated weights for policy 0, policy_version 18150 (0.0010) [2023-10-11 19:48:54,192][71601] Updated weights for policy 0, policy_version 18160 (0.0008) [2023-10-11 19:48:54,578][71601] Updated weights for policy 0, policy_version 18170 (0.0007) [2023-10-11 19:48:56,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 37191680. Throughput: 0: 1827.9, 1: 1825.0. Samples: 9308024. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 19:48:56,035][70582] Avg episode reward: [(0, '12.970'), (1, '14.450')] [2023-10-11 19:48:57,041][71635] Updated weights for policy 1, policy_version 18152 (0.0009) [2023-10-11 19:48:57,406][71635] Updated weights for policy 1, policy_version 18162 (0.0010) [2023-10-11 19:48:57,775][71635] Updated weights for policy 1, policy_version 18172 (0.0009) [2023-10-11 19:48:58,139][71601] Updated weights for policy 0, policy_version 18180 (0.0009) [2023-10-11 19:48:58,512][71601] Updated weights for policy 0, policy_version 18190 (0.0007) [2023-10-11 19:48:58,879][71601] Updated weights for policy 0, policy_version 18200 (0.0008) [2023-10-11 19:49:01,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 37257216. Throughput: 0: 1822.8, 1: 1824.5. Samples: 9318998. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 19:49:01,035][70582] Avg episode reward: [(0, '12.870'), (1, '15.170')] [2023-10-11 19:49:01,544][71635] Updated weights for policy 1, policy_version 18182 (0.0009) [2023-10-11 19:49:01,909][71635] Updated weights for policy 1, policy_version 18192 (0.0008) [2023-10-11 19:49:02,280][71635] Updated weights for policy 1, policy_version 18202 (0.0008) [2023-10-11 19:49:02,675][71601] Updated weights for policy 0, policy_version 18210 (0.0008) [2023-10-11 19:49:03,045][71601] Updated weights for policy 0, policy_version 18220 (0.0007) [2023-10-11 19:49:03,423][71601] Updated weights for policy 0, policy_version 18230 (0.0007) [2023-10-11 19:49:03,793][71601] Updated weights for policy 0, policy_version 18240 (0.0009) [2023-10-11 19:49:05,998][71635] Updated weights for policy 1, policy_version 18212 (0.0008) [2023-10-11 19:49:06,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 37322752. Throughput: 0: 1819.5, 1: 1816.4. Samples: 9340478. Policy #0 lag: (min: 0.0, avg: 24.7, max: 32.0) [2023-10-11 19:49:06,034][70582] Avg episode reward: [(0, '13.150'), (1, '15.010')] [2023-10-11 19:49:06,372][71635] Updated weights for policy 1, policy_version 18222 (0.0009) [2023-10-11 19:49:06,734][71635] Updated weights for policy 1, policy_version 18232 (0.0010) [2023-10-11 19:49:07,328][71601] Updated weights for policy 0, policy_version 18250 (0.0009) [2023-10-11 19:49:07,704][71601] Updated weights for policy 0, policy_version 18260 (0.0010) [2023-10-11 19:49:08,071][71601] Updated weights for policy 0, policy_version 18270 (0.0011) [2023-10-11 19:49:10,481][71635] Updated weights for policy 1, policy_version 18242 (0.0010) [2023-10-11 19:49:10,857][71635] Updated weights for policy 1, policy_version 18252 (0.0012) [2023-10-11 19:49:11,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 37388288. Throughput: 0: 1813.0, 1: 1813.2. Samples: 9363008. Policy #0 lag: (min: 0.0, avg: 24.7, max: 32.0) [2023-10-11 19:49:11,034][70582] Avg episode reward: [(0, '14.080'), (1, '14.770')] [2023-10-11 19:49:11,217][71635] Updated weights for policy 1, policy_version 18262 (0.0011) [2023-10-11 19:49:11,592][71635] Updated weights for policy 1, policy_version 18272 (0.0009) [2023-10-11 19:49:12,014][71601] Updated weights for policy 0, policy_version 18280 (0.0009) [2023-10-11 19:49:12,390][71601] Updated weights for policy 0, policy_version 18290 (0.0009) [2023-10-11 19:49:12,764][71601] Updated weights for policy 0, policy_version 18300 (0.0011) [2023-10-11 19:49:15,304][71635] Updated weights for policy 1, policy_version 18282 (0.0008) [2023-10-11 19:49:15,675][71635] Updated weights for policy 1, policy_version 18292 (0.0009) [2023-10-11 19:49:16,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 37453824. Throughput: 0: 1813.7, 1: 1808.6. Samples: 9372768. Policy #0 lag: (min: 0.0, avg: 24.7, max: 32.0) [2023-10-11 19:49:16,035][70582] Avg episode reward: [(0, '14.020'), (1, '13.750')] [2023-10-11 19:49:16,043][71635] Updated weights for policy 1, policy_version 18302 (0.0009) [2023-10-11 19:49:16,450][71601] Updated weights for policy 0, policy_version 18310 (0.0009) [2023-10-11 19:49:16,828][71601] Updated weights for policy 0, policy_version 18320 (0.0009) [2023-10-11 19:49:17,210][71601] Updated weights for policy 0, policy_version 18330 (0.0008) [2023-10-11 19:49:19,901][71635] Updated weights for policy 1, policy_version 18312 (0.0009) [2023-10-11 19:49:20,279][71635] Updated weights for policy 1, policy_version 18322 (0.0008) [2023-10-11 19:49:20,655][71635] Updated weights for policy 1, policy_version 18332 (0.0008) [2023-10-11 19:49:20,900][71601] Updated weights for policy 0, policy_version 18340 (0.0009) [2023-10-11 19:49:21,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 37552128. Throughput: 0: 1817.3, 1: 1807.8. Samples: 9395238. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:49:21,035][70582] Avg episode reward: [(0, '15.100'), (1, '13.600')] [2023-10-11 19:49:21,263][71601] Updated weights for policy 0, policy_version 18350 (0.0008) [2023-10-11 19:49:21,635][71601] Updated weights for policy 0, policy_version 18360 (0.0008) [2023-10-11 19:49:24,425][71635] Updated weights for policy 1, policy_version 18342 (0.0009) [2023-10-11 19:49:24,794][71635] Updated weights for policy 1, policy_version 18352 (0.0008) [2023-10-11 19:49:25,163][71635] Updated weights for policy 1, policy_version 18362 (0.0008) [2023-10-11 19:49:25,376][71601] Updated weights for policy 0, policy_version 18370 (0.0008) [2023-10-11 19:49:25,750][71601] Updated weights for policy 0, policy_version 18380 (0.0007) [2023-10-11 19:49:26,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 37617664. Throughput: 0: 1817.2, 1: 1798.5. Samples: 9416144. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:49:26,034][70582] Avg episode reward: [(0, '15.520'), (1, '14.190')] [2023-10-11 19:49:26,128][71601] Updated weights for policy 0, policy_version 18390 (0.0009) [2023-10-11 19:49:26,490][71601] Updated weights for policy 0, policy_version 18400 (0.0009) [2023-10-11 19:49:28,857][71635] Updated weights for policy 1, policy_version 18372 (0.0009) [2023-10-11 19:49:29,220][71635] Updated weights for policy 1, policy_version 18382 (0.0009) [2023-10-11 19:49:29,592][71635] Updated weights for policy 1, policy_version 18392 (0.0007) [2023-10-11 19:49:30,190][71601] Updated weights for policy 0, policy_version 18410 (0.0008) [2023-10-11 19:49:30,558][71601] Updated weights for policy 0, policy_version 18420 (0.0009) [2023-10-11 19:49:30,931][71601] Updated weights for policy 0, policy_version 18430 (0.0009) [2023-10-11 19:49:31,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 37715968. Throughput: 0: 1809.0, 1: 1798.3. Samples: 9427534. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:49:31,034][70582] Avg episode reward: [(0, '16.240'), (1, '14.120')] [2023-10-11 19:49:33,263][71635] Updated weights for policy 1, policy_version 18402 (0.0010) [2023-10-11 19:49:33,626][71635] Updated weights for policy 1, policy_version 18412 (0.0010) [2023-10-11 19:49:33,986][71635] Updated weights for policy 1, policy_version 18422 (0.0009) [2023-10-11 19:49:34,351][71635] Updated weights for policy 1, policy_version 18432 (0.0009) [2023-10-11 19:49:34,666][71601] Updated weights for policy 0, policy_version 18440 (0.0008) [2023-10-11 19:49:35,036][71601] Updated weights for policy 0, policy_version 18450 (0.0009) [2023-10-11 19:49:35,409][71601] Updated weights for policy 0, policy_version 18460 (0.0007) [2023-10-11 19:49:36,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 37781504. Throughput: 0: 1804.8, 1: 1797.4. Samples: 9448504. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:49:36,035][70582] Avg episode reward: [(0, '15.370'), (1, '15.110')] [2023-10-11 19:49:38,158][71635] Updated weights for policy 1, policy_version 18442 (0.0007) [2023-10-11 19:49:38,526][71635] Updated weights for policy 1, policy_version 18452 (0.0008) [2023-10-11 19:49:38,895][71635] Updated weights for policy 1, policy_version 18462 (0.0008) [2023-10-11 19:49:39,004][71601] Updated weights for policy 0, policy_version 18470 (0.0009) [2023-10-11 19:49:39,380][71601] Updated weights for policy 0, policy_version 18480 (0.0009) [2023-10-11 19:49:39,747][71601] Updated weights for policy 0, policy_version 18490 (0.0008) [2023-10-11 19:49:41,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 37847040. Throughput: 0: 1803.5, 1: 1795.7. Samples: 9469992. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:49:41,035][70582] Avg episode reward: [(0, '14.690'), (1, '14.390')] [2023-10-11 19:49:42,604][71635] Updated weights for policy 1, policy_version 18472 (0.0008) [2023-10-11 19:49:42,974][71635] Updated weights for policy 1, policy_version 18482 (0.0008) [2023-10-11 19:49:43,338][71635] Updated weights for policy 1, policy_version 18492 (0.0007) [2023-10-11 19:49:43,488][71601] Updated weights for policy 0, policy_version 18500 (0.0009) [2023-10-11 19:49:43,861][71601] Updated weights for policy 0, policy_version 18510 (0.0009) [2023-10-11 19:49:44,237][71601] Updated weights for policy 0, policy_version 18520 (0.0010) [2023-10-11 19:49:46,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 37912576. Throughput: 0: 1810.4, 1: 1800.0. Samples: 9481462. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:49:46,034][70582] Avg episode reward: [(0, '13.390'), (1, '12.270')] [2023-10-11 19:49:46,956][71635] Updated weights for policy 1, policy_version 18502 (0.0007) [2023-10-11 19:49:47,317][71635] Updated weights for policy 1, policy_version 18512 (0.0009) [2023-10-11 19:49:47,691][71635] Updated weights for policy 1, policy_version 18522 (0.0009) [2023-10-11 19:49:47,939][71601] Updated weights for policy 0, policy_version 18530 (0.0008) [2023-10-11 19:49:48,313][71601] Updated weights for policy 0, policy_version 18540 (0.0008) [2023-10-11 19:49:48,679][71601] Updated weights for policy 0, policy_version 18550 (0.0008) [2023-10-11 19:49:49,046][71601] Updated weights for policy 0, policy_version 18560 (0.0008) [2023-10-11 19:49:51,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 37978112. Throughput: 0: 1804.4, 1: 1800.5. Samples: 9502700. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:49:51,035][70582] Avg episode reward: [(0, '11.940'), (1, '12.270')] [2023-10-11 19:49:51,325][71635] Updated weights for policy 1, policy_version 18532 (0.0010) [2023-10-11 19:49:51,694][71635] Updated weights for policy 1, policy_version 18542 (0.0010) [2023-10-11 19:49:52,054][71635] Updated weights for policy 1, policy_version 18552 (0.0011) [2023-10-11 19:49:52,622][71601] Updated weights for policy 0, policy_version 18570 (0.0010) [2023-10-11 19:49:52,989][71601] Updated weights for policy 0, policy_version 18580 (0.0010) [2023-10-11 19:49:53,361][71601] Updated weights for policy 0, policy_version 18590 (0.0010) [2023-10-11 19:49:55,715][71635] Updated weights for policy 1, policy_version 18562 (0.0008) [2023-10-11 19:49:56,034][70582] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 38043648. Throughput: 0: 1811.5, 1: 1804.2. Samples: 9525714. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 19:49:56,035][70582] Avg episode reward: [(0, '13.230'), (1, '11.940')] [2023-10-11 19:49:56,044][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000018592_19038208.pth... [2023-10-11 19:49:56,076][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000016896_17301504.pth [2023-10-11 19:49:56,080][71635] Updated weights for policy 1, policy_version 18572 (0.0007) [2023-10-11 19:49:56,446][71635] Updated weights for policy 1, policy_version 18582 (0.0008) [2023-10-11 19:49:56,818][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000018592_19038208.pth... [2023-10-11 19:49:56,818][71635] Updated weights for policy 1, policy_version 18592 (0.0009) [2023-10-11 19:49:56,851][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000016864_17268736.pth [2023-10-11 19:49:57,108][71601] Updated weights for policy 0, policy_version 18600 (0.0010) [2023-10-11 19:49:57,481][71601] Updated weights for policy 0, policy_version 18610 (0.0009) [2023-10-11 19:49:57,855][71601] Updated weights for policy 0, policy_version 18620 (0.0009) [2023-10-11 19:50:00,605][71635] Updated weights for policy 1, policy_version 18602 (0.0007) [2023-10-11 19:50:00,970][71635] Updated weights for policy 1, policy_version 18612 (0.0009) [2023-10-11 19:50:01,033][70582] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 38109184. Throughput: 0: 1813.9, 1: 1804.7. Samples: 9535602. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 19:50:01,034][70582] Avg episode reward: [(0, '14.310'), (1, '12.230')] [2023-10-11 19:50:01,334][71635] Updated weights for policy 1, policy_version 18622 (0.0007) [2023-10-11 19:50:01,579][71601] Updated weights for policy 0, policy_version 18630 (0.0007) [2023-10-11 19:50:01,959][71601] Updated weights for policy 0, policy_version 18640 (0.0010) [2023-10-11 19:50:02,324][71601] Updated weights for policy 0, policy_version 18650 (0.0007) [2023-10-11 19:50:04,991][71635] Updated weights for policy 1, policy_version 18632 (0.0008) [2023-10-11 19:50:05,366][71635] Updated weights for policy 1, policy_version 18642 (0.0007) [2023-10-11 19:50:05,729][71635] Updated weights for policy 1, policy_version 18652 (0.0010) [2023-10-11 19:50:05,904][71601] Updated weights for policy 0, policy_version 18660 (0.0007) [2023-10-11 19:50:06,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 38207488. Throughput: 0: 1815.5, 1: 1809.9. Samples: 9558380. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 19:50:06,035][70582] Avg episode reward: [(0, '14.330'), (1, '12.770')] [2023-10-11 19:50:06,268][71601] Updated weights for policy 0, policy_version 18670 (0.0009) [2023-10-11 19:50:06,638][71601] Updated weights for policy 0, policy_version 18680 (0.0008) [2023-10-11 19:50:09,423][71635] Updated weights for policy 1, policy_version 18662 (0.0011) [2023-10-11 19:50:09,788][71635] Updated weights for policy 1, policy_version 18672 (0.0010) [2023-10-11 19:50:10,158][71635] Updated weights for policy 1, policy_version 18682 (0.0009) [2023-10-11 19:50:10,358][71601] Updated weights for policy 0, policy_version 18690 (0.0010) [2023-10-11 19:50:10,726][71601] Updated weights for policy 0, policy_version 18700 (0.0009) [2023-10-11 19:50:11,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 38273024. Throughput: 0: 1816.5, 1: 1813.8. Samples: 9579506. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 19:50:11,034][70582] Avg episode reward: [(0, '14.870'), (1, '13.620')] [2023-10-11 19:50:11,089][71601] Updated weights for policy 0, policy_version 18710 (0.0007) [2023-10-11 19:50:11,470][71601] Updated weights for policy 0, policy_version 18720 (0.0007) [2023-10-11 19:50:13,882][71635] Updated weights for policy 1, policy_version 18692 (0.0008) [2023-10-11 19:50:14,255][71635] Updated weights for policy 1, policy_version 18702 (0.0010) [2023-10-11 19:50:14,612][71635] Updated weights for policy 1, policy_version 18712 (0.0009) [2023-10-11 19:50:15,025][71601] Updated weights for policy 0, policy_version 18730 (0.0008) [2023-10-11 19:50:15,396][71601] Updated weights for policy 0, policy_version 18740 (0.0008) [2023-10-11 19:50:15,761][71601] Updated weights for policy 0, policy_version 18750 (0.0009) [2023-10-11 19:50:16,034][70582] Fps is (10 sec: 16384.2, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 38371328. Throughput: 0: 1820.3, 1: 1813.4. Samples: 9591050. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 19:50:16,034][70582] Avg episode reward: [(0, '14.260'), (1, '13.510')] [2023-10-11 19:50:18,290][71635] Updated weights for policy 1, policy_version 18722 (0.0007) [2023-10-11 19:50:18,709][71635] Updated weights for policy 1, policy_version 18732 (0.0010) [2023-10-11 19:50:19,063][71635] Updated weights for policy 1, policy_version 18742 (0.0008) [2023-10-11 19:50:19,431][71635] Updated weights for policy 1, policy_version 18752 (0.0009) [2023-10-11 19:50:19,626][71601] Updated weights for policy 0, policy_version 18760 (0.0010) [2023-10-11 19:50:19,999][71601] Updated weights for policy 0, policy_version 18770 (0.0008) [2023-10-11 19:50:20,370][71601] Updated weights for policy 0, policy_version 18780 (0.0007) [2023-10-11 19:50:21,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 38436864. Throughput: 0: 1820.1, 1: 1812.9. Samples: 9611988. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 19:50:21,035][70582] Avg episode reward: [(0, '14.800'), (1, '14.370')] [2023-10-11 19:50:23,210][71635] Updated weights for policy 1, policy_version 18762 (0.0007) [2023-10-11 19:50:23,568][71635] Updated weights for policy 1, policy_version 18772 (0.0009) [2023-10-11 19:50:23,938][71635] Updated weights for policy 1, policy_version 18782 (0.0010) [2023-10-11 19:50:24,097][71601] Updated weights for policy 0, policy_version 18790 (0.0007) [2023-10-11 19:50:24,472][71601] Updated weights for policy 0, policy_version 18800 (0.0007) [2023-10-11 19:50:24,850][71601] Updated weights for policy 0, policy_version 18810 (0.0008) [2023-10-11 19:50:26,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 38502400. Throughput: 0: 1814.9, 1: 1811.1. Samples: 9633158. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 19:50:26,034][70582] Avg episode reward: [(0, '14.710'), (1, '13.750')] [2023-10-11 19:50:27,686][71635] Updated weights for policy 1, policy_version 18792 (0.0008) [2023-10-11 19:50:28,050][71635] Updated weights for policy 1, policy_version 18802 (0.0008) [2023-10-11 19:50:28,422][71635] Updated weights for policy 1, policy_version 18812 (0.0007) [2023-10-11 19:50:28,544][71601] Updated weights for policy 0, policy_version 18820 (0.0008) [2023-10-11 19:50:28,927][71601] Updated weights for policy 0, policy_version 18830 (0.0009) [2023-10-11 19:50:29,297][71601] Updated weights for policy 0, policy_version 18840 (0.0011) [2023-10-11 19:50:31,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 38567936. Throughput: 0: 1816.4, 1: 1815.2. Samples: 9644884. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-10-11 19:50:31,034][70582] Avg episode reward: [(0, '14.520'), (1, '15.080')] [2023-10-11 19:50:31,990][71635] Updated weights for policy 1, policy_version 18822 (0.0007) [2023-10-11 19:50:32,361][71635] Updated weights for policy 1, policy_version 18832 (0.0009) [2023-10-11 19:50:32,730][71635] Updated weights for policy 1, policy_version 18842 (0.0010) [2023-10-11 19:50:33,010][71601] Updated weights for policy 0, policy_version 18850 (0.0012) [2023-10-11 19:50:33,379][71601] Updated weights for policy 0, policy_version 18860 (0.0008) [2023-10-11 19:50:33,758][71601] Updated weights for policy 0, policy_version 18870 (0.0007) [2023-10-11 19:50:34,127][71601] Updated weights for policy 0, policy_version 18880 (0.0009) [2023-10-11 19:50:36,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 38633472. Throughput: 0: 1818.5, 1: 1814.2. Samples: 9666170. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-10-11 19:50:36,035][70582] Avg episode reward: [(0, '16.180'), (1, '13.750')] [2023-10-11 19:50:36,416][71635] Updated weights for policy 1, policy_version 18852 (0.0008) [2023-10-11 19:50:36,788][71635] Updated weights for policy 1, policy_version 18862 (0.0009) [2023-10-11 19:50:37,151][71635] Updated weights for policy 1, policy_version 18872 (0.0007) [2023-10-11 19:50:37,704][71601] Updated weights for policy 0, policy_version 18890 (0.0008) [2023-10-11 19:50:38,078][71601] Updated weights for policy 0, policy_version 18900 (0.0007) [2023-10-11 19:50:38,449][71601] Updated weights for policy 0, policy_version 18910 (0.0009) [2023-10-11 19:50:41,008][71635] Updated weights for policy 1, policy_version 18882 (0.0008) [2023-10-11 19:50:41,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 38699008. Throughput: 0: 1814.5, 1: 1808.3. Samples: 9688738. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-10-11 19:50:41,034][70582] Avg episode reward: [(0, '14.930'), (1, '13.850')] [2023-10-11 19:50:41,380][71635] Updated weights for policy 1, policy_version 18892 (0.0007) [2023-10-11 19:50:41,750][71635] Updated weights for policy 1, policy_version 18902 (0.0007) [2023-10-11 19:50:42,112][71635] Updated weights for policy 1, policy_version 18912 (0.0008) [2023-10-11 19:50:42,118][71601] Updated weights for policy 0, policy_version 18920 (0.0007) [2023-10-11 19:50:42,492][71601] Updated weights for policy 0, policy_version 18930 (0.0009) [2023-10-11 19:50:42,869][71601] Updated weights for policy 0, policy_version 18940 (0.0010) [2023-10-11 19:50:45,736][71635] Updated weights for policy 1, policy_version 18922 (0.0011) [2023-10-11 19:50:46,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 38764544. Throughput: 0: 1812.4, 1: 1810.5. Samples: 9698634. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-10-11 19:50:46,034][70582] Avg episode reward: [(0, '15.510'), (1, '14.360')] [2023-10-11 19:50:46,108][71635] Updated weights for policy 1, policy_version 18932 (0.0010) [2023-10-11 19:50:46,467][71635] Updated weights for policy 1, policy_version 18942 (0.0008) [2023-10-11 19:50:46,593][71601] Updated weights for policy 0, policy_version 18950 (0.0008) [2023-10-11 19:50:46,963][71601] Updated weights for policy 0, policy_version 18960 (0.0010) [2023-10-11 19:50:47,327][71601] Updated weights for policy 0, policy_version 18970 (0.0009) [2023-10-11 19:50:50,136][71635] Updated weights for policy 1, policy_version 18952 (0.0008) [2023-10-11 19:50:50,500][71635] Updated weights for policy 1, policy_version 18962 (0.0008) [2023-10-11 19:50:50,867][71635] Updated weights for policy 1, policy_version 18972 (0.0009) [2023-10-11 19:50:50,933][71601] Updated weights for policy 0, policy_version 18980 (0.0008) [2023-10-11 19:50:51,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 38862848. Throughput: 0: 1809.6, 1: 1808.4. Samples: 9721190. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:50:51,034][70582] Avg episode reward: [(0, '15.130'), (1, '14.260')] [2023-10-11 19:50:51,317][71601] Updated weights for policy 0, policy_version 18990 (0.0009) [2023-10-11 19:50:51,692][71601] Updated weights for policy 0, policy_version 19000 (0.0009) [2023-10-11 19:50:54,632][71635] Updated weights for policy 1, policy_version 18982 (0.0008) [2023-10-11 19:50:54,994][71635] Updated weights for policy 1, policy_version 18992 (0.0007) [2023-10-11 19:50:55,354][71635] Updated weights for policy 1, policy_version 19002 (0.0008) [2023-10-11 19:50:55,515][71601] Updated weights for policy 0, policy_version 19010 (0.0007) [2023-10-11 19:50:55,886][71601] Updated weights for policy 0, policy_version 19020 (0.0008) [2023-10-11 19:50:56,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 38928384. Throughput: 0: 1813.4, 1: 1815.4. Samples: 9742804. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:50:56,034][70582] Avg episode reward: [(0, '14.680'), (1, '13.930')] [2023-10-11 19:50:56,257][71601] Updated weights for policy 0, policy_version 19030 (0.0010) [2023-10-11 19:50:56,623][71601] Updated weights for policy 0, policy_version 19040 (0.0010) [2023-10-11 19:50:59,084][71635] Updated weights for policy 1, policy_version 19012 (0.0009) [2023-10-11 19:50:59,452][71635] Updated weights for policy 1, policy_version 19022 (0.0011) [2023-10-11 19:50:59,827][71635] Updated weights for policy 1, policy_version 19032 (0.0009) [2023-10-11 19:51:00,504][71601] Updated weights for policy 0, policy_version 19050 (0.0008) [2023-10-11 19:51:00,866][71601] Updated weights for policy 0, policy_version 19060 (0.0008) [2023-10-11 19:51:01,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 38993920. Throughput: 0: 1804.8, 1: 1814.6. Samples: 9753926. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:51:01,035][70582] Avg episode reward: [(0, '13.750'), (1, '12.140')] [2023-10-11 19:51:01,253][71601] Updated weights for policy 0, policy_version 19070 (0.0008) [2023-10-11 19:51:03,572][71635] Updated weights for policy 1, policy_version 19042 (0.0008) [2023-10-11 19:51:03,934][71635] Updated weights for policy 1, policy_version 19052 (0.0009) [2023-10-11 19:51:04,308][71635] Updated weights for policy 1, policy_version 19062 (0.0010) [2023-10-11 19:51:04,673][71635] Updated weights for policy 1, policy_version 19072 (0.0010) [2023-10-11 19:51:04,930][71601] Updated weights for policy 0, policy_version 19080 (0.0009) [2023-10-11 19:51:05,305][71601] Updated weights for policy 0, policy_version 19090 (0.0009) [2023-10-11 19:51:05,679][71601] Updated weights for policy 0, policy_version 19100 (0.0009) [2023-10-11 19:51:06,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 39092224. Throughput: 0: 1816.3, 1: 1829.7. Samples: 9776060. Policy #0 lag: (min: 26.0, avg: 27.1, max: 44.0) [2023-10-11 19:51:06,034][70582] Avg episode reward: [(0, '14.140'), (1, '13.230')] [2023-10-11 19:51:08,129][71635] Updated weights for policy 1, policy_version 19082 (0.0007) [2023-10-11 19:51:08,491][71635] Updated weights for policy 1, policy_version 19092 (0.0010) [2023-10-11 19:51:08,857][71635] Updated weights for policy 1, policy_version 19102 (0.0009) [2023-10-11 19:51:09,436][71601] Updated weights for policy 0, policy_version 19110 (0.0008) [2023-10-11 19:51:09,800][71601] Updated weights for policy 0, policy_version 19120 (0.0007) [2023-10-11 19:51:10,176][71601] Updated weights for policy 0, policy_version 19130 (0.0009) [2023-10-11 19:51:11,034][70582] Fps is (10 sec: 16384.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 39157760. Throughput: 0: 1812.5, 1: 1821.1. Samples: 9796670. Policy #0 lag: (min: 26.0, avg: 27.1, max: 44.0) [2023-10-11 19:51:11,034][70582] Avg episode reward: [(0, '13.540'), (1, '13.760')] [2023-10-11 19:51:12,709][71635] Updated weights for policy 1, policy_version 19112 (0.0008) [2023-10-11 19:51:13,079][71635] Updated weights for policy 1, policy_version 19122 (0.0010) [2023-10-11 19:51:13,455][71635] Updated weights for policy 1, policy_version 19132 (0.0008) [2023-10-11 19:51:13,744][71601] Updated weights for policy 0, policy_version 19140 (0.0010) [2023-10-11 19:51:14,113][71601] Updated weights for policy 0, policy_version 19150 (0.0010) [2023-10-11 19:51:14,490][71601] Updated weights for policy 0, policy_version 19160 (0.0007) [2023-10-11 19:51:16,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 39223296. Throughput: 0: 1816.3, 1: 1819.4. Samples: 9808492. Policy #0 lag: (min: 26.0, avg: 27.1, max: 44.0) [2023-10-11 19:51:16,035][70582] Avg episode reward: [(0, '13.260'), (1, '12.490')] [2023-10-11 19:51:17,381][71635] Updated weights for policy 1, policy_version 19142 (0.0009) [2023-10-11 19:51:17,750][71635] Updated weights for policy 1, policy_version 19152 (0.0007) [2023-10-11 19:51:18,112][71635] Updated weights for policy 1, policy_version 19162 (0.0008) [2023-10-11 19:51:18,239][71601] Updated weights for policy 0, policy_version 19170 (0.0008) [2023-10-11 19:51:18,621][71601] Updated weights for policy 0, policy_version 19180 (0.0007) [2023-10-11 19:51:18,995][71601] Updated weights for policy 0, policy_version 19190 (0.0010) [2023-10-11 19:51:19,374][71601] Updated weights for policy 0, policy_version 19200 (0.0009) [2023-10-11 19:51:21,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 39288832. Throughput: 0: 1813.7, 1: 1809.5. Samples: 9829214. Policy #0 lag: (min: 26.0, avg: 27.1, max: 44.0) [2023-10-11 19:51:21,035][70582] Avg episode reward: [(0, '12.580'), (1, '13.190')] [2023-10-11 19:51:21,734][71635] Updated weights for policy 1, policy_version 19172 (0.0007) [2023-10-11 19:51:22,098][71635] Updated weights for policy 1, policy_version 19182 (0.0007) [2023-10-11 19:51:22,462][71635] Updated weights for policy 1, policy_version 19192 (0.0008) [2023-10-11 19:51:22,986][71601] Updated weights for policy 0, policy_version 19210 (0.0008) [2023-10-11 19:51:23,356][71601] Updated weights for policy 0, policy_version 19220 (0.0009) [2023-10-11 19:51:23,725][71601] Updated weights for policy 0, policy_version 19230 (0.0009) [2023-10-11 19:51:26,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 39354368. Throughput: 0: 1810.4, 1: 1817.1. Samples: 9851976. Policy #0 lag: (min: 26.0, avg: 35.4, max: 58.0) [2023-10-11 19:51:26,035][70582] Avg episode reward: [(0, '11.610'), (1, '13.020')] [2023-10-11 19:51:26,278][71635] Updated weights for policy 1, policy_version 19202 (0.0007) [2023-10-11 19:51:26,643][71635] Updated weights for policy 1, policy_version 19212 (0.0007) [2023-10-11 19:51:27,024][71635] Updated weights for policy 1, policy_version 19222 (0.0008) [2023-10-11 19:51:27,386][71635] Updated weights for policy 1, policy_version 19232 (0.0007) [2023-10-11 19:51:27,426][71601] Updated weights for policy 0, policy_version 19240 (0.0008) [2023-10-11 19:51:27,803][71601] Updated weights for policy 0, policy_version 19250 (0.0008) [2023-10-11 19:51:28,172][71601] Updated weights for policy 0, policy_version 19260 (0.0008) [2023-10-11 19:51:31,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 39419904. Throughput: 0: 1808.3, 1: 1816.3. Samples: 9861744. Policy #0 lag: (min: 26.0, avg: 35.4, max: 58.0) [2023-10-11 19:51:31,035][70582] Avg episode reward: [(0, '11.320'), (1, '12.940')] [2023-10-11 19:51:31,110][71635] Updated weights for policy 1, policy_version 19242 (0.0012) [2023-10-11 19:51:31,478][71635] Updated weights for policy 1, policy_version 19252 (0.0010) [2023-10-11 19:51:31,849][71635] Updated weights for policy 1, policy_version 19262 (0.0009) [2023-10-11 19:51:31,954][71601] Updated weights for policy 0, policy_version 19270 (0.0008) [2023-10-11 19:51:32,325][71601] Updated weights for policy 0, policy_version 19280 (0.0010) [2023-10-11 19:51:32,702][71601] Updated weights for policy 0, policy_version 19290 (0.0008) [2023-10-11 19:51:35,299][71635] Updated weights for policy 1, policy_version 19272 (0.0009) [2023-10-11 19:51:35,666][71635] Updated weights for policy 1, policy_version 19282 (0.0008) [2023-10-11 19:51:36,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 39485440. Throughput: 0: 1808.6, 1: 1823.5. Samples: 9884634. Policy #0 lag: (min: 26.0, avg: 35.4, max: 58.0) [2023-10-11 19:51:36,035][70582] Avg episode reward: [(0, '12.760'), (1, '13.530')] [2023-10-11 19:51:36,040][71635] Updated weights for policy 1, policy_version 19292 (0.0007) [2023-10-11 19:51:36,457][71601] Updated weights for policy 0, policy_version 19300 (0.0007) [2023-10-11 19:51:36,828][71601] Updated weights for policy 0, policy_version 19310 (0.0008) [2023-10-11 19:51:37,196][71601] Updated weights for policy 0, policy_version 19320 (0.0008) [2023-10-11 19:51:39,754][71635] Updated weights for policy 1, policy_version 19302 (0.0008) [2023-10-11 19:51:40,118][71635] Updated weights for policy 1, policy_version 19312 (0.0007) [2023-10-11 19:51:40,475][71635] Updated weights for policy 1, policy_version 19322 (0.0008) [2023-10-11 19:51:40,940][71601] Updated weights for policy 0, policy_version 19330 (0.0009) [2023-10-11 19:51:41,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 39583744. Throughput: 0: 1809.6, 1: 1824.7. Samples: 9906348. Policy #0 lag: (min: 26.0, avg: 35.4, max: 58.0) [2023-10-11 19:51:41,034][70582] Avg episode reward: [(0, '13.450'), (1, '14.530')] [2023-10-11 19:51:41,318][71601] Updated weights for policy 0, policy_version 19340 (0.0009) [2023-10-11 19:51:41,695][71601] Updated weights for policy 0, policy_version 19350 (0.0009) [2023-10-11 19:51:42,068][71601] Updated weights for policy 0, policy_version 19360 (0.0010) [2023-10-11 19:51:44,105][71635] Updated weights for policy 1, policy_version 19332 (0.0010) [2023-10-11 19:51:44,465][71635] Updated weights for policy 1, policy_version 19342 (0.0009) [2023-10-11 19:51:44,837][71635] Updated weights for policy 1, policy_version 19352 (0.0010) [2023-10-11 19:51:45,742][71601] Updated weights for policy 0, policy_version 19370 (0.0009) [2023-10-11 19:51:46,034][70582] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 39649280. Throughput: 0: 1809.7, 1: 1823.2. Samples: 9917408. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:51:46,034][70582] Avg episode reward: [(0, '13.600'), (1, '13.710')] [2023-10-11 19:51:46,114][71601] Updated weights for policy 0, policy_version 19380 (0.0008) [2023-10-11 19:51:46,501][71601] Updated weights for policy 0, policy_version 19390 (0.0009) [2023-10-11 19:51:48,677][71635] Updated weights for policy 1, policy_version 19362 (0.0010) [2023-10-11 19:51:49,082][71635] Updated weights for policy 1, policy_version 19372 (0.0008) [2023-10-11 19:51:49,451][71635] Updated weights for policy 1, policy_version 19382 (0.0009) [2023-10-11 19:51:49,815][71635] Updated weights for policy 1, policy_version 19392 (0.0008) [2023-10-11 19:51:50,143][71601] Updated weights for policy 0, policy_version 19400 (0.0009) [2023-10-11 19:51:50,517][71601] Updated weights for policy 0, policy_version 19410 (0.0010) [2023-10-11 19:51:50,892][71601] Updated weights for policy 0, policy_version 19420 (0.0008) [2023-10-11 19:51:51,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 39714816. Throughput: 0: 1807.6, 1: 1817.6. Samples: 9939192. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:51:51,035][70582] Avg episode reward: [(0, '14.190'), (1, '13.860')] [2023-10-11 19:51:53,468][71635] Updated weights for policy 1, policy_version 19402 (0.0007) [2023-10-11 19:51:53,830][71635] Updated weights for policy 1, policy_version 19412 (0.0007) [2023-10-11 19:51:54,195][71635] Updated weights for policy 1, policy_version 19422 (0.0009) [2023-10-11 19:51:54,443][71601] Updated weights for policy 0, policy_version 19430 (0.0007) [2023-10-11 19:51:54,819][71601] Updated weights for policy 0, policy_version 19440 (0.0008) [2023-10-11 19:51:55,179][71601] Updated weights for policy 0, policy_version 19450 (0.0010) [2023-10-11 19:51:56,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 39813120. Throughput: 0: 1817.6, 1: 1815.2. Samples: 9960146. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:51:56,034][70582] Avg episode reward: [(0, '13.280'), (1, '12.360')] [2023-10-11 19:51:56,044][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000019456_19922944.pth... [2023-10-11 19:51:56,044][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000019424_19890176.pth... [2023-10-11 19:51:56,083][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000017760_18186240.pth [2023-10-11 19:51:56,084][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000017728_18153472.pth [2023-10-11 19:51:57,824][71635] Updated weights for policy 1, policy_version 19432 (0.0008) [2023-10-11 19:51:58,194][71635] Updated weights for policy 1, policy_version 19442 (0.0009) [2023-10-11 19:51:58,556][71635] Updated weights for policy 1, policy_version 19452 (0.0008) [2023-10-11 19:51:58,946][71601] Updated weights for policy 0, policy_version 19460 (0.0008) [2023-10-11 19:51:59,310][71601] Updated weights for policy 0, policy_version 19470 (0.0008) [2023-10-11 19:51:59,683][71601] Updated weights for policy 0, policy_version 19480 (0.0007) [2023-10-11 19:52:01,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 39878656. Throughput: 0: 1811.8, 1: 1823.5. Samples: 9972082. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-11 19:52:01,034][70582] Avg episode reward: [(0, '13.430'), (1, '13.640')] [2023-10-11 19:52:02,225][71635] Updated weights for policy 1, policy_version 19462 (0.0009) [2023-10-11 19:52:02,603][71635] Updated weights for policy 1, policy_version 19472 (0.0008) [2023-10-11 19:52:02,973][71635] Updated weights for policy 1, policy_version 19482 (0.0007) [2023-10-11 19:52:03,339][71601] Updated weights for policy 0, policy_version 19490 (0.0008) [2023-10-11 19:52:03,721][71601] Updated weights for policy 0, policy_version 19500 (0.0009) [2023-10-11 19:52:04,095][71601] Updated weights for policy 0, policy_version 19510 (0.0009) [2023-10-11 19:52:04,462][71601] Updated weights for policy 0, policy_version 19520 (0.0010) [2023-10-11 19:52:06,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 39944192. Throughput: 0: 1809.8, 1: 1827.0. Samples: 9992870. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-11 19:52:06,035][70582] Avg episode reward: [(0, '13.290'), (1, '13.420')] [2023-10-11 19:52:06,564][71635] Updated weights for policy 1, policy_version 19492 (0.0009) [2023-10-11 19:52:06,923][71635] Updated weights for policy 1, policy_version 19502 (0.0008) [2023-10-11 19:52:07,296][71635] Updated weights for policy 1, policy_version 19512 (0.0007) [2023-10-11 19:52:08,106][71601] Updated weights for policy 0, policy_version 19530 (0.0008) [2023-10-11 19:52:08,477][71601] Updated weights for policy 0, policy_version 19540 (0.0009) [2023-10-11 19:52:08,848][71601] Updated weights for policy 0, policy_version 19550 (0.0008) [2023-10-11 19:52:11,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 40009728. Throughput: 0: 1810.7, 1: 1829.3. Samples: 10015774. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-11 19:52:11,034][70582] Avg episode reward: [(0, '13.630'), (1, '13.440')] [2023-10-11 19:52:11,102][71635] Updated weights for policy 1, policy_version 19522 (0.0008) [2023-10-11 19:52:11,468][71635] Updated weights for policy 1, policy_version 19532 (0.0008) [2023-10-11 19:52:11,835][71635] Updated weights for policy 1, policy_version 19542 (0.0007) [2023-10-11 19:52:12,198][71635] Updated weights for policy 1, policy_version 19552 (0.0008) [2023-10-11 19:52:12,690][71601] Updated weights for policy 0, policy_version 19560 (0.0010) [2023-10-11 19:52:13,065][71601] Updated weights for policy 0, policy_version 19570 (0.0007) [2023-10-11 19:52:13,445][71601] Updated weights for policy 0, policy_version 19580 (0.0008) [2023-10-11 19:52:15,933][71635] Updated weights for policy 1, policy_version 19562 (0.0009) [2023-10-11 19:52:16,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 40075264. Throughput: 0: 1820.5, 1: 1827.4. Samples: 10025902. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-11 19:52:16,034][70582] Avg episode reward: [(0, '13.530'), (1, '13.870')] [2023-10-11 19:52:16,295][71635] Updated weights for policy 1, policy_version 19572 (0.0010) [2023-10-11 19:52:16,665][71635] Updated weights for policy 1, policy_version 19582 (0.0008) [2023-10-11 19:52:16,954][71601] Updated weights for policy 0, policy_version 19590 (0.0007) [2023-10-11 19:52:17,322][71601] Updated weights for policy 0, policy_version 19600 (0.0009) [2023-10-11 19:52:17,702][71601] Updated weights for policy 0, policy_version 19610 (0.0010) [2023-10-11 19:52:20,490][71635] Updated weights for policy 1, policy_version 19592 (0.0010) [2023-10-11 19:52:20,856][71635] Updated weights for policy 1, policy_version 19602 (0.0011) [2023-10-11 19:52:21,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 40140800. Throughput: 0: 1821.0, 1: 1819.9. Samples: 10048476. Policy #0 lag: (min: 34.0, avg: 54.3, max: 56.0) [2023-10-11 19:52:21,034][70582] Avg episode reward: [(0, '13.790'), (1, '13.200')] [2023-10-11 19:52:21,225][71635] Updated weights for policy 1, policy_version 19612 (0.0010) [2023-10-11 19:52:21,493][71601] Updated weights for policy 0, policy_version 19620 (0.0010) [2023-10-11 19:52:21,856][71601] Updated weights for policy 0, policy_version 19630 (0.0007) [2023-10-11 19:52:22,232][71601] Updated weights for policy 0, policy_version 19640 (0.0009) [2023-10-11 19:52:24,857][71635] Updated weights for policy 1, policy_version 19622 (0.0008) [2023-10-11 19:52:25,221][71635] Updated weights for policy 1, policy_version 19632 (0.0009) [2023-10-11 19:52:25,598][71635] Updated weights for policy 1, policy_version 19642 (0.0009) [2023-10-11 19:52:25,912][71601] Updated weights for policy 0, policy_version 19650 (0.0008) [2023-10-11 19:52:26,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 40239104. Throughput: 0: 1818.2, 1: 1825.0. Samples: 10070294. Policy #0 lag: (min: 34.0, avg: 54.3, max: 56.0) [2023-10-11 19:52:26,034][70582] Avg episode reward: [(0, '13.270'), (1, '14.310')] [2023-10-11 19:52:26,292][71601] Updated weights for policy 0, policy_version 19660 (0.0008) [2023-10-11 19:52:26,657][71601] Updated weights for policy 0, policy_version 19670 (0.0009) [2023-10-11 19:52:27,012][71601] Updated weights for policy 0, policy_version 19680 (0.0008) [2023-10-11 19:52:29,224][71635] Updated weights for policy 1, policy_version 19652 (0.0010) [2023-10-11 19:52:29,579][71635] Updated weights for policy 1, policy_version 19662 (0.0009) [2023-10-11 19:52:29,954][71635] Updated weights for policy 1, policy_version 19672 (0.0008) [2023-10-11 19:52:30,706][71601] Updated weights for policy 0, policy_version 19690 (0.0007) [2023-10-11 19:52:31,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 40304640. Throughput: 0: 1817.1, 1: 1819.1. Samples: 10081042. Policy #0 lag: (min: 34.0, avg: 54.3, max: 56.0) [2023-10-11 19:52:31,035][70582] Avg episode reward: [(0, '13.850'), (1, '13.520')] [2023-10-11 19:52:31,085][71601] Updated weights for policy 0, policy_version 19700 (0.0007) [2023-10-11 19:52:31,459][71601] Updated weights for policy 0, policy_version 19710 (0.0008) [2023-10-11 19:52:33,601][71635] Updated weights for policy 1, policy_version 19682 (0.0009) [2023-10-11 19:52:33,968][71635] Updated weights for policy 1, policy_version 19692 (0.0007) [2023-10-11 19:52:34,330][71635] Updated weights for policy 1, policy_version 19702 (0.0008) [2023-10-11 19:52:34,697][71635] Updated weights for policy 1, policy_version 19712 (0.0008) [2023-10-11 19:52:34,983][71601] Updated weights for policy 0, policy_version 19720 (0.0009) [2023-10-11 19:52:35,364][71601] Updated weights for policy 0, policy_version 19730 (0.0008) [2023-10-11 19:52:35,747][71601] Updated weights for policy 0, policy_version 19740 (0.0008) [2023-10-11 19:52:36,034][70582] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 40402944. Throughput: 0: 1819.1, 1: 1819.3. Samples: 10102920. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:52:36,034][70582] Avg episode reward: [(0, '13.570'), (1, '13.710')] [2023-10-11 19:52:38,451][71635] Updated weights for policy 1, policy_version 19722 (0.0010) [2023-10-11 19:52:38,820][71635] Updated weights for policy 1, policy_version 19732 (0.0009) [2023-10-11 19:52:39,191][71635] Updated weights for policy 1, policy_version 19742 (0.0008) [2023-10-11 19:52:39,402][71601] Updated weights for policy 0, policy_version 19750 (0.0008) [2023-10-11 19:52:39,767][71601] Updated weights for policy 0, policy_version 19760 (0.0009) [2023-10-11 19:52:40,145][71601] Updated weights for policy 0, policy_version 19770 (0.0007) [2023-10-11 19:52:41,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 40468480. Throughput: 0: 1815.8, 1: 1816.3. Samples: 10123590. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:52:41,035][70582] Avg episode reward: [(0, '13.410'), (1, '13.800')] [2023-10-11 19:52:42,954][71635] Updated weights for policy 1, policy_version 19752 (0.0007) [2023-10-11 19:52:43,317][71635] Updated weights for policy 1, policy_version 19762 (0.0008) [2023-10-11 19:52:43,686][71635] Updated weights for policy 1, policy_version 19772 (0.0007) [2023-10-11 19:52:43,770][71601] Updated weights for policy 0, policy_version 19780 (0.0008) [2023-10-11 19:52:44,138][71601] Updated weights for policy 0, policy_version 19790 (0.0007) [2023-10-11 19:52:44,506][71601] Updated weights for policy 0, policy_version 19800 (0.0007) [2023-10-11 19:52:46,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 40534016. Throughput: 0: 1817.3, 1: 1816.4. Samples: 10135602. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:52:46,035][70582] Avg episode reward: [(0, '13.510'), (1, '13.140')] [2023-10-11 19:52:47,346][71635] Updated weights for policy 1, policy_version 19782 (0.0008) [2023-10-11 19:52:47,707][71635] Updated weights for policy 1, policy_version 19792 (0.0009) [2023-10-11 19:52:48,088][71635] Updated weights for policy 1, policy_version 19802 (0.0009) [2023-10-11 19:52:48,249][71601] Updated weights for policy 0, policy_version 19810 (0.0009) [2023-10-11 19:52:48,620][71601] Updated weights for policy 0, policy_version 19820 (0.0009) [2023-10-11 19:52:48,987][71601] Updated weights for policy 0, policy_version 19830 (0.0008) [2023-10-11 19:52:49,359][71601] Updated weights for policy 0, policy_version 19840 (0.0009) [2023-10-11 19:52:51,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 40599552. Throughput: 0: 1818.1, 1: 1817.0. Samples: 10156452. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:52:51,034][70582] Avg episode reward: [(0, '13.650'), (1, '13.120')] [2023-10-11 19:52:51,748][71635] Updated weights for policy 1, policy_version 19812 (0.0008) [2023-10-11 19:52:52,117][71635] Updated weights for policy 1, policy_version 19822 (0.0010) [2023-10-11 19:52:52,477][71635] Updated weights for policy 1, policy_version 19832 (0.0009) [2023-10-11 19:52:52,963][71601] Updated weights for policy 0, policy_version 19850 (0.0008) [2023-10-11 19:52:53,340][71601] Updated weights for policy 0, policy_version 19860 (0.0008) [2023-10-11 19:52:53,702][71601] Updated weights for policy 0, policy_version 19870 (0.0007) [2023-10-11 19:52:56,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 40665088. Throughput: 0: 1822.1, 1: 1816.8. Samples: 10179526. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:52:56,034][70582] Avg episode reward: [(0, '14.340'), (1, '12.680')] [2023-10-11 19:52:56,173][71635] Updated weights for policy 1, policy_version 19842 (0.0007) [2023-10-11 19:52:56,541][71635] Updated weights for policy 1, policy_version 19852 (0.0009) [2023-10-11 19:52:56,906][71635] Updated weights for policy 1, policy_version 19862 (0.0007) [2023-10-11 19:52:57,272][71635] Updated weights for policy 1, policy_version 19872 (0.0008) [2023-10-11 19:52:57,503][71601] Updated weights for policy 0, policy_version 19880 (0.0008) [2023-10-11 19:52:57,869][71601] Updated weights for policy 0, policy_version 19890 (0.0007) [2023-10-11 19:52:58,237][71601] Updated weights for policy 0, policy_version 19900 (0.0008) [2023-10-11 19:53:00,851][71635] Updated weights for policy 1, policy_version 19882 (0.0008) [2023-10-11 19:53:01,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 40730624. Throughput: 0: 1818.6, 1: 1819.2. Samples: 10189606. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:53:01,035][70582] Avg episode reward: [(0, '14.850'), (1, '12.660')] [2023-10-11 19:53:01,219][71635] Updated weights for policy 1, policy_version 19892 (0.0007) [2023-10-11 19:53:01,581][71635] Updated weights for policy 1, policy_version 19902 (0.0007) [2023-10-11 19:53:01,904][71601] Updated weights for policy 0, policy_version 19910 (0.0008) [2023-10-11 19:53:02,281][71601] Updated weights for policy 0, policy_version 19920 (0.0009) [2023-10-11 19:53:02,639][71601] Updated weights for policy 0, policy_version 19930 (0.0009) [2023-10-11 19:53:05,184][71635] Updated weights for policy 1, policy_version 19912 (0.0009) [2023-10-11 19:53:05,556][71635] Updated weights for policy 1, policy_version 19922 (0.0008) [2023-10-11 19:53:05,934][71635] Updated weights for policy 1, policy_version 19932 (0.0009) [2023-10-11 19:53:06,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 40796160. Throughput: 0: 1818.9, 1: 1822.2. Samples: 10212328. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:53:06,034][70582] Avg episode reward: [(0, '15.640'), (1, '12.040')] [2023-10-11 19:53:06,293][71601] Updated weights for policy 0, policy_version 19940 (0.0010) [2023-10-11 19:53:06,677][71601] Updated weights for policy 0, policy_version 19950 (0.0010) [2023-10-11 19:53:07,044][71601] Updated weights for policy 0, policy_version 19960 (0.0010) [2023-10-11 19:53:09,832][71635] Updated weights for policy 1, policy_version 19942 (0.0007) [2023-10-11 19:53:10,194][71635] Updated weights for policy 1, policy_version 19952 (0.0008) [2023-10-11 19:53:10,559][71635] Updated weights for policy 1, policy_version 19962 (0.0008) [2023-10-11 19:53:10,739][71601] Updated weights for policy 0, policy_version 19970 (0.0009) [2023-10-11 19:53:11,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 40894464. Throughput: 0: 1819.9, 1: 1816.4. Samples: 10233926. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:53:11,034][70582] Avg episode reward: [(0, '15.050'), (1, '12.110')] [2023-10-11 19:53:11,107][71601] Updated weights for policy 0, policy_version 19980 (0.0007) [2023-10-11 19:53:11,479][71601] Updated weights for policy 0, policy_version 19990 (0.0008) [2023-10-11 19:53:11,852][71601] Updated weights for policy 0, policy_version 20000 (0.0007) [2023-10-11 19:53:14,136][71635] Updated weights for policy 1, policy_version 19972 (0.0008) [2023-10-11 19:53:14,510][71635] Updated weights for policy 1, policy_version 19982 (0.0008) [2023-10-11 19:53:14,878][71635] Updated weights for policy 1, policy_version 19992 (0.0010) [2023-10-11 19:53:15,553][71601] Updated weights for policy 0, policy_version 20010 (0.0008) [2023-10-11 19:53:15,923][71601] Updated weights for policy 0, policy_version 20020 (0.0007) [2023-10-11 19:53:16,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 40960000. Throughput: 0: 1823.3, 1: 1814.3. Samples: 10244734. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:53:16,034][70582] Avg episode reward: [(0, '13.530'), (1, '13.520')] [2023-10-11 19:53:16,294][71601] Updated weights for policy 0, policy_version 20030 (0.0008) [2023-10-11 19:53:18,522][71635] Updated weights for policy 1, policy_version 20002 (0.0009) [2023-10-11 19:53:18,881][71635] Updated weights for policy 1, policy_version 20012 (0.0009) [2023-10-11 19:53:19,249][71635] Updated weights for policy 1, policy_version 20022 (0.0010) [2023-10-11 19:53:19,617][71635] Updated weights for policy 1, policy_version 20032 (0.0010) [2023-10-11 19:53:19,972][71601] Updated weights for policy 0, policy_version 20040 (0.0009) [2023-10-11 19:53:20,347][71601] Updated weights for policy 0, policy_version 20050 (0.0008) [2023-10-11 19:53:20,709][71601] Updated weights for policy 0, policy_version 20060 (0.0009) [2023-10-11 19:53:21,034][70582] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 41058304. Throughput: 0: 1819.8, 1: 1821.7. Samples: 10266788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:53:21,034][70582] Avg episode reward: [(0, '13.720'), (1, '12.840')] [2023-10-11 19:53:23,504][71635] Updated weights for policy 1, policy_version 20042 (0.0009) [2023-10-11 19:53:23,876][71635] Updated weights for policy 1, policy_version 20052 (0.0009) [2023-10-11 19:53:24,251][71635] Updated weights for policy 1, policy_version 20062 (0.0008) [2023-10-11 19:53:24,427][71601] Updated weights for policy 0, policy_version 20070 (0.0008) [2023-10-11 19:53:24,790][71601] Updated weights for policy 0, policy_version 20080 (0.0007) [2023-10-11 19:53:25,164][71601] Updated weights for policy 0, policy_version 20090 (0.0010) [2023-10-11 19:53:26,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 41123840. Throughput: 0: 1819.6, 1: 1822.6. Samples: 10287486. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:53:26,035][70582] Avg episode reward: [(0, '15.290'), (1, '13.240')] [2023-10-11 19:53:27,884][71635] Updated weights for policy 1, policy_version 20072 (0.0007) [2023-10-11 19:53:28,256][71635] Updated weights for policy 1, policy_version 20082 (0.0007) [2023-10-11 19:53:28,622][71635] Updated weights for policy 1, policy_version 20092 (0.0007) [2023-10-11 19:53:28,793][71601] Updated weights for policy 0, policy_version 20100 (0.0010) [2023-10-11 19:53:29,168][71601] Updated weights for policy 0, policy_version 20110 (0.0011) [2023-10-11 19:53:29,543][71601] Updated weights for policy 0, policy_version 20120 (0.0011) [2023-10-11 19:53:31,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 41189376. Throughput: 0: 1818.1, 1: 1824.0. Samples: 10299498. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-11 19:53:31,034][70582] Avg episode reward: [(0, '15.020'), (1, '14.210')] [2023-10-11 19:53:32,499][71635] Updated weights for policy 1, policy_version 20102 (0.0008) [2023-10-11 19:53:32,869][71635] Updated weights for policy 1, policy_version 20112 (0.0009) [2023-10-11 19:53:33,125][71601] Updated weights for policy 0, policy_version 20130 (0.0009) [2023-10-11 19:53:33,234][71635] Updated weights for policy 1, policy_version 20122 (0.0008) [2023-10-11 19:53:33,492][71601] Updated weights for policy 0, policy_version 20140 (0.0007) [2023-10-11 19:53:33,856][71601] Updated weights for policy 0, policy_version 20150 (0.0007) [2023-10-11 19:53:34,225][71601] Updated weights for policy 0, policy_version 20160 (0.0008) [2023-10-11 19:53:36,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 41254912. Throughput: 0: 1822.0, 1: 1813.8. Samples: 10320064. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-11 19:53:36,035][70582] Avg episode reward: [(0, '15.330'), (1, '13.690')] [2023-10-11 19:53:36,975][71635] Updated weights for policy 1, policy_version 20132 (0.0008) [2023-10-11 19:53:37,339][71635] Updated weights for policy 1, policy_version 20142 (0.0010) [2023-10-11 19:53:37,709][71635] Updated weights for policy 1, policy_version 20152 (0.0008) [2023-10-11 19:53:37,906][71601] Updated weights for policy 0, policy_version 20170 (0.0008) [2023-10-11 19:53:38,280][71601] Updated weights for policy 0, policy_version 20180 (0.0008) [2023-10-11 19:53:38,643][71601] Updated weights for policy 0, policy_version 20190 (0.0008) [2023-10-11 19:53:41,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 41320448. Throughput: 0: 1821.1, 1: 1810.3. Samples: 10342940. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-11 19:53:41,034][70582] Avg episode reward: [(0, '15.440'), (1, '13.180')] [2023-10-11 19:53:41,459][71635] Updated weights for policy 1, policy_version 20162 (0.0010) [2023-10-11 19:53:41,830][71635] Updated weights for policy 1, policy_version 20172 (0.0008) [2023-10-11 19:53:42,197][71635] Updated weights for policy 1, policy_version 20182 (0.0007) [2023-10-11 19:53:42,219][71601] Updated weights for policy 0, policy_version 20200 (0.0008) [2023-10-11 19:53:42,558][71635] Updated weights for policy 1, policy_version 20192 (0.0008) [2023-10-11 19:53:42,592][71601] Updated weights for policy 0, policy_version 20210 (0.0007) [2023-10-11 19:53:42,968][71601] Updated weights for policy 0, policy_version 20220 (0.0009) [2023-10-11 19:53:46,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 41385984. Throughput: 0: 1822.8, 1: 1809.3. Samples: 10353050. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-11 19:53:46,034][70582] Avg episode reward: [(0, '14.340'), (1, '13.710')] [2023-10-11 19:53:46,147][71635] Updated weights for policy 1, policy_version 20202 (0.0008) [2023-10-11 19:53:46,519][71635] Updated weights for policy 1, policy_version 20212 (0.0008) [2023-10-11 19:53:46,668][71601] Updated weights for policy 0, policy_version 20230 (0.0009) [2023-10-11 19:53:46,872][71635] Updated weights for policy 1, policy_version 20222 (0.0009) [2023-10-11 19:53:47,033][71601] Updated weights for policy 0, policy_version 20240 (0.0007) [2023-10-11 19:53:47,401][71601] Updated weights for policy 0, policy_version 20250 (0.0008) [2023-10-11 19:53:50,761][71635] Updated weights for policy 1, policy_version 20232 (0.0007) [2023-10-11 19:53:51,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 41451520. Throughput: 0: 1827.0, 1: 1806.4. Samples: 10375832. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:53:51,034][70582] Avg episode reward: [(0, '13.900'), (1, '14.730')] [2023-10-11 19:53:51,095][71601] Updated weights for policy 0, policy_version 20260 (0.0008) [2023-10-11 19:53:51,120][71635] Updated weights for policy 1, policy_version 20242 (0.0007) [2023-10-11 19:53:51,476][71601] Updated weights for policy 0, policy_version 20270 (0.0007) [2023-10-11 19:53:51,484][71635] Updated weights for policy 1, policy_version 20252 (0.0009) [2023-10-11 19:53:51,849][71601] Updated weights for policy 0, policy_version 20280 (0.0010) [2023-10-11 19:53:55,024][71635] Updated weights for policy 1, policy_version 20262 (0.0010) [2023-10-11 19:53:55,392][71635] Updated weights for policy 1, policy_version 20272 (0.0010) [2023-10-11 19:53:55,744][71601] Updated weights for policy 0, policy_version 20290 (0.0010) [2023-10-11 19:53:55,767][71635] Updated weights for policy 1, policy_version 20282 (0.0009) [2023-10-11 19:53:56,034][70582] Fps is (10 sec: 16383.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 41549824. Throughput: 0: 1821.4, 1: 1818.9. Samples: 10397738. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:53:56,036][70582] Avg episode reward: [(0, '14.860'), (1, '13.350')] [2023-10-11 19:53:56,043][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000020288_20774912.pth... [2023-10-11 19:53:56,080][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000018592_19038208.pth [2023-10-11 19:53:56,109][71601] Updated weights for policy 0, policy_version 20300 (0.0009) [2023-10-11 19:53:56,477][71601] Updated weights for policy 0, policy_version 20310 (0.0012) [2023-10-11 19:53:56,847][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000020320_20807680.pth... [2023-10-11 19:53:56,852][71601] Updated weights for policy 0, policy_version 20320 (0.0007) [2023-10-11 19:53:56,887][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000018592_19038208.pth [2023-10-11 19:53:59,569][71635] Updated weights for policy 1, policy_version 20292 (0.0007) [2023-10-11 19:53:59,943][71635] Updated weights for policy 1, policy_version 20302 (0.0011) [2023-10-11 19:54:00,311][71635] Updated weights for policy 1, policy_version 20312 (0.0011) [2023-10-11 19:54:00,619][71601] Updated weights for policy 0, policy_version 20330 (0.0008) [2023-10-11 19:54:00,984][71601] Updated weights for policy 0, policy_version 20340 (0.0007) [2023-10-11 19:54:01,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 41615360. Throughput: 0: 1823.1, 1: 1809.8. Samples: 10408214. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:54:01,034][70582] Avg episode reward: [(0, '13.290'), (1, '14.020')] [2023-10-11 19:54:01,354][71601] Updated weights for policy 0, policy_version 20350 (0.0007) [2023-10-11 19:54:03,967][71635] Updated weights for policy 1, policy_version 20322 (0.0008) [2023-10-11 19:54:04,334][71635] Updated weights for policy 1, policy_version 20332 (0.0008) [2023-10-11 19:54:04,699][71635] Updated weights for policy 1, policy_version 20342 (0.0008) [2023-10-11 19:54:05,074][71635] Updated weights for policy 1, policy_version 20352 (0.0009) [2023-10-11 19:54:05,101][71601] Updated weights for policy 0, policy_version 20360 (0.0007) [2023-10-11 19:54:05,475][71601] Updated weights for policy 0, policy_version 20370 (0.0011) [2023-10-11 19:54:05,848][71601] Updated weights for policy 0, policy_version 20380 (0.0010) [2023-10-11 19:54:06,034][70582] Fps is (10 sec: 16384.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 41713664. Throughput: 0: 1822.6, 1: 1810.8. Samples: 10430294. Policy #0 lag: (min: 18.0, avg: 21.0, max: 50.0) [2023-10-11 19:54:06,035][70582] Avg episode reward: [(0, '14.670'), (1, '14.040')] [2023-10-11 19:54:08,702][71635] Updated weights for policy 1, policy_version 20362 (0.0008) [2023-10-11 19:54:09,070][71635] Updated weights for policy 1, policy_version 20372 (0.0009) [2023-10-11 19:54:09,430][71635] Updated weights for policy 1, policy_version 20382 (0.0010) [2023-10-11 19:54:09,594][71601] Updated weights for policy 0, policy_version 20390 (0.0008) [2023-10-11 19:54:09,961][71601] Updated weights for policy 0, policy_version 20400 (0.0010) [2023-10-11 19:54:10,335][71601] Updated weights for policy 0, policy_version 20410 (0.0010) [2023-10-11 19:54:11,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 41779200. Throughput: 0: 1822.7, 1: 1806.4. Samples: 10450798. Policy #0 lag: (min: 18.0, avg: 21.0, max: 50.0) [2023-10-11 19:54:11,035][70582] Avg episode reward: [(0, '13.180'), (1, '15.780')] [2023-10-11 19:54:13,206][71635] Updated weights for policy 1, policy_version 20392 (0.0009) [2023-10-11 19:54:13,570][71635] Updated weights for policy 1, policy_version 20402 (0.0010) [2023-10-11 19:54:13,932][71635] Updated weights for policy 1, policy_version 20412 (0.0010) [2023-10-11 19:54:14,008][71601] Updated weights for policy 0, policy_version 20420 (0.0008) [2023-10-11 19:54:14,382][71601] Updated weights for policy 0, policy_version 20430 (0.0008) [2023-10-11 19:54:14,748][71601] Updated weights for policy 0, policy_version 20440 (0.0007) [2023-10-11 19:54:16,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 41844736. Throughput: 0: 1820.5, 1: 1811.2. Samples: 10462926. Policy #0 lag: (min: 18.0, avg: 21.0, max: 50.0) [2023-10-11 19:54:16,035][70582] Avg episode reward: [(0, '12.520'), (1, '15.860')] [2023-10-11 19:54:17,747][71635] Updated weights for policy 1, policy_version 20422 (0.0008) [2023-10-11 19:54:18,115][71635] Updated weights for policy 1, policy_version 20432 (0.0009) [2023-10-11 19:54:18,302][71601] Updated weights for policy 0, policy_version 20450 (0.0008) [2023-10-11 19:54:18,485][71635] Updated weights for policy 1, policy_version 20442 (0.0007) [2023-10-11 19:54:18,680][71601] Updated weights for policy 0, policy_version 20460 (0.0008) [2023-10-11 19:54:19,049][71601] Updated weights for policy 0, policy_version 20470 (0.0010) [2023-10-11 19:54:19,422][71601] Updated weights for policy 0, policy_version 20480 (0.0008) [2023-10-11 19:54:21,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 41910272. Throughput: 0: 1822.6, 1: 1805.5. Samples: 10483328. Policy #0 lag: (min: 18.0, avg: 21.0, max: 50.0) [2023-10-11 19:54:21,035][70582] Avg episode reward: [(0, '13.220'), (1, '15.550')] [2023-10-11 19:54:22,154][71635] Updated weights for policy 1, policy_version 20452 (0.0008) [2023-10-11 19:54:22,519][71635] Updated weights for policy 1, policy_version 20462 (0.0010) [2023-10-11 19:54:22,882][71635] Updated weights for policy 1, policy_version 20472 (0.0010) [2023-10-11 19:54:23,069][71601] Updated weights for policy 0, policy_version 20490 (0.0009) [2023-10-11 19:54:23,450][71601] Updated weights for policy 0, policy_version 20500 (0.0010) [2023-10-11 19:54:23,811][71601] Updated weights for policy 0, policy_version 20510 (0.0007) [2023-10-11 19:54:26,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 41975808. Throughput: 0: 1819.5, 1: 1802.0. Samples: 10505910. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-11 19:54:26,034][70582] Avg episode reward: [(0, '12.740'), (1, '15.850')] [2023-10-11 19:54:26,708][71635] Updated weights for policy 1, policy_version 20482 (0.0010) [2023-10-11 19:54:27,079][71635] Updated weights for policy 1, policy_version 20492 (0.0007) [2023-10-11 19:54:27,437][71635] Updated weights for policy 1, policy_version 20502 (0.0009) [2023-10-11 19:54:27,628][71601] Updated weights for policy 0, policy_version 20520 (0.0009) [2023-10-11 19:54:27,807][71635] Updated weights for policy 1, policy_version 20512 (0.0008) [2023-10-11 19:54:27,993][71601] Updated weights for policy 0, policy_version 20530 (0.0007) [2023-10-11 19:54:28,374][71601] Updated weights for policy 0, policy_version 20540 (0.0007) [2023-10-11 19:54:31,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 42041344. Throughput: 0: 1819.0, 1: 1800.9. Samples: 10515948. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-11 19:54:31,034][70582] Avg episode reward: [(0, '11.860'), (1, '15.760')] [2023-10-11 19:54:31,490][71635] Updated weights for policy 1, policy_version 20522 (0.0009) [2023-10-11 19:54:31,856][71635] Updated weights for policy 1, policy_version 20532 (0.0007) [2023-10-11 19:54:32,067][71601] Updated weights for policy 0, policy_version 20550 (0.0008) [2023-10-11 19:54:32,226][71635] Updated weights for policy 1, policy_version 20542 (0.0008) [2023-10-11 19:54:32,444][71601] Updated weights for policy 0, policy_version 20560 (0.0008) [2023-10-11 19:54:32,819][71601] Updated weights for policy 0, policy_version 20570 (0.0008) [2023-10-11 19:54:35,830][71635] Updated weights for policy 1, policy_version 20552 (0.0008) [2023-10-11 19:54:36,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 42106880. Throughput: 0: 1811.4, 1: 1801.7. Samples: 10538420. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-11 19:54:36,034][70582] Avg episode reward: [(0, '13.280'), (1, '15.450')] [2023-10-11 19:54:36,194][71635] Updated weights for policy 1, policy_version 20562 (0.0007) [2023-10-11 19:54:36,470][71601] Updated weights for policy 0, policy_version 20580 (0.0007) [2023-10-11 19:54:36,556][71635] Updated weights for policy 1, policy_version 20572 (0.0008) [2023-10-11 19:54:36,856][71601] Updated weights for policy 0, policy_version 20590 (0.0009) [2023-10-11 19:54:37,233][71601] Updated weights for policy 0, policy_version 20600 (0.0009) [2023-10-11 19:54:40,283][71635] Updated weights for policy 1, policy_version 20582 (0.0010) [2023-10-11 19:54:40,631][71635] Updated weights for policy 1, policy_version 20592 (0.0008) [2023-10-11 19:54:40,949][71601] Updated weights for policy 0, policy_version 20610 (0.0009) [2023-10-11 19:54:40,996][71635] Updated weights for policy 1, policy_version 20602 (0.0010) [2023-10-11 19:54:41,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 42172416. Throughput: 0: 1816.5, 1: 1806.7. Samples: 10560778. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-11 19:54:41,034][70582] Avg episode reward: [(0, '13.450'), (1, '16.360')] [2023-10-11 19:54:41,321][71601] Updated weights for policy 0, policy_version 20620 (0.0009) [2023-10-11 19:54:41,683][71601] Updated weights for policy 0, policy_version 20630 (0.0010) [2023-10-11 19:54:42,055][71601] Updated weights for policy 0, policy_version 20640 (0.0008) [2023-10-11 19:54:44,634][71635] Updated weights for policy 1, policy_version 20612 (0.0008) [2023-10-11 19:54:45,003][71635] Updated weights for policy 1, policy_version 20622 (0.0007) [2023-10-11 19:54:45,381][71635] Updated weights for policy 1, policy_version 20632 (0.0007) [2023-10-11 19:54:45,763][71601] Updated weights for policy 0, policy_version 20650 (0.0007) [2023-10-11 19:54:46,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 42270720. Throughput: 0: 1812.1, 1: 1805.9. Samples: 10571024. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:54:46,035][70582] Avg episode reward: [(0, '13.180'), (1, '15.980')] [2023-10-11 19:54:46,127][71601] Updated weights for policy 0, policy_version 20660 (0.0007) [2023-10-11 19:54:46,500][71601] Updated weights for policy 0, policy_version 20670 (0.0007) [2023-10-11 19:54:49,078][71635] Updated weights for policy 1, policy_version 20642 (0.0007) [2023-10-11 19:54:49,435][71635] Updated weights for policy 1, policy_version 20652 (0.0007) [2023-10-11 19:54:49,805][71635] Updated weights for policy 1, policy_version 20662 (0.0009) [2023-10-11 19:54:50,126][71601] Updated weights for policy 0, policy_version 20680 (0.0008) [2023-10-11 19:54:50,169][71635] Updated weights for policy 1, policy_version 20672 (0.0007) [2023-10-11 19:54:50,506][71601] Updated weights for policy 0, policy_version 20690 (0.0008) [2023-10-11 19:54:50,873][71601] Updated weights for policy 0, policy_version 20700 (0.0008) [2023-10-11 19:54:51,034][70582] Fps is (10 sec: 19660.8, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 42369024. Throughput: 0: 1811.8, 1: 1814.4. Samples: 10593472. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:54:51,034][70582] Avg episode reward: [(0, '13.620'), (1, '17.630')] [2023-10-11 19:54:53,950][71635] Updated weights for policy 1, policy_version 20682 (0.0008) [2023-10-11 19:54:54,312][71635] Updated weights for policy 1, policy_version 20692 (0.0009) [2023-10-11 19:54:54,686][71635] Updated weights for policy 1, policy_version 20702 (0.0009) [2023-10-11 19:54:54,724][71601] Updated weights for policy 0, policy_version 20710 (0.0008) [2023-10-11 19:54:55,099][71601] Updated weights for policy 0, policy_version 20720 (0.0008) [2023-10-11 19:54:55,468][71601] Updated weights for policy 0, policy_version 20730 (0.0007) [2023-10-11 19:54:56,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 42434560. Throughput: 0: 1813.1, 1: 1805.9. Samples: 10613652. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:54:56,035][70582] Avg episode reward: [(0, '14.820'), (1, '16.500')] [2023-10-11 19:54:58,338][71635] Updated weights for policy 1, policy_version 20712 (0.0007) [2023-10-11 19:54:58,700][71635] Updated weights for policy 1, policy_version 20722 (0.0009) [2023-10-11 19:54:59,061][71635] Updated weights for policy 1, policy_version 20732 (0.0009) [2023-10-11 19:54:59,079][71601] Updated weights for policy 0, policy_version 20740 (0.0007) [2023-10-11 19:54:59,443][71601] Updated weights for policy 0, policy_version 20750 (0.0009) [2023-10-11 19:54:59,819][71601] Updated weights for policy 0, policy_version 20760 (0.0008) [2023-10-11 19:55:01,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 42500096. Throughput: 0: 1809.1, 1: 1812.4. Samples: 10625892. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-10-11 19:55:01,034][70582] Avg episode reward: [(0, '13.790'), (1, '16.130')] [2023-10-11 19:55:02,777][71635] Updated weights for policy 1, policy_version 20742 (0.0007) [2023-10-11 19:55:03,137][71635] Updated weights for policy 1, policy_version 20752 (0.0007) [2023-10-11 19:55:03,504][71635] Updated weights for policy 1, policy_version 20762 (0.0007) [2023-10-11 19:55:03,710][71601] Updated weights for policy 0, policy_version 20770 (0.0010) [2023-10-11 19:55:04,072][71601] Updated weights for policy 0, policy_version 20780 (0.0007) [2023-10-11 19:55:04,448][71601] Updated weights for policy 0, policy_version 20790 (0.0007) [2023-10-11 19:55:04,811][71601] Updated weights for policy 0, policy_version 20800 (0.0010) [2023-10-11 19:55:06,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 42565632. Throughput: 0: 1812.5, 1: 1815.9. Samples: 10646606. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-10-11 19:55:06,035][70582] Avg episode reward: [(0, '14.330'), (1, '14.870')] [2023-10-11 19:55:07,276][71635] Updated weights for policy 1, policy_version 20772 (0.0008) [2023-10-11 19:55:07,646][71635] Updated weights for policy 1, policy_version 20782 (0.0008) [2023-10-11 19:55:08,015][71635] Updated weights for policy 1, policy_version 20792 (0.0010) [2023-10-11 19:55:08,579][71601] Updated weights for policy 0, policy_version 20810 (0.0007) [2023-10-11 19:55:08,954][71601] Updated weights for policy 0, policy_version 20820 (0.0008) [2023-10-11 19:55:09,323][71601] Updated weights for policy 0, policy_version 20830 (0.0009) [2023-10-11 19:55:11,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 42631168. Throughput: 0: 1800.6, 1: 1819.1. Samples: 10668798. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-10-11 19:55:11,035][70582] Avg episode reward: [(0, '15.010'), (1, '13.490')] [2023-10-11 19:55:11,651][71635] Updated weights for policy 1, policy_version 20802 (0.0007) [2023-10-11 19:55:12,025][71635] Updated weights for policy 1, policy_version 20812 (0.0008) [2023-10-11 19:55:12,386][71635] Updated weights for policy 1, policy_version 20822 (0.0009) [2023-10-11 19:55:12,748][71635] Updated weights for policy 1, policy_version 20832 (0.0007) [2023-10-11 19:55:12,963][71601] Updated weights for policy 0, policy_version 20840 (0.0007) [2023-10-11 19:55:13,344][71601] Updated weights for policy 0, policy_version 20850 (0.0007) [2023-10-11 19:55:13,705][71601] Updated weights for policy 0, policy_version 20860 (0.0007) [2023-10-11 19:55:16,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 42696704. Throughput: 0: 1813.6, 1: 1817.7. Samples: 10679354. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-10-11 19:55:16,034][70582] Avg episode reward: [(0, '14.870'), (1, '13.830')] [2023-10-11 19:55:16,544][71635] Updated weights for policy 1, policy_version 20842 (0.0008) [2023-10-11 19:55:16,908][71635] Updated weights for policy 1, policy_version 20852 (0.0009) [2023-10-11 19:55:17,285][71635] Updated weights for policy 1, policy_version 20862 (0.0007) [2023-10-11 19:55:17,326][71601] Updated weights for policy 0, policy_version 20870 (0.0009) [2023-10-11 19:55:17,702][71601] Updated weights for policy 0, policy_version 20880 (0.0009) [2023-10-11 19:55:18,071][71601] Updated weights for policy 0, policy_version 20890 (0.0007) [2023-10-11 19:55:21,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 42762240. Throughput: 0: 1808.6, 1: 1817.3. Samples: 10701586. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-11 19:55:21,034][70582] Avg episode reward: [(0, '14.820'), (1, '14.700')] [2023-10-11 19:55:21,087][71635] Updated weights for policy 1, policy_version 20872 (0.0007) [2023-10-11 19:55:21,454][71635] Updated weights for policy 1, policy_version 20882 (0.0007) [2023-10-11 19:55:21,815][71635] Updated weights for policy 1, policy_version 20892 (0.0008) [2023-10-11 19:55:21,915][71601] Updated weights for policy 0, policy_version 20900 (0.0008) [2023-10-11 19:55:22,297][71601] Updated weights for policy 0, policy_version 20910 (0.0009) [2023-10-11 19:55:22,668][71601] Updated weights for policy 0, policy_version 20920 (0.0007) [2023-10-11 19:55:25,546][71635] Updated weights for policy 1, policy_version 20902 (0.0008) [2023-10-11 19:55:25,917][71635] Updated weights for policy 1, policy_version 20912 (0.0007) [2023-10-11 19:55:26,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 42827776. Throughput: 0: 1802.1, 1: 1823.4. Samples: 10723926. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-11 19:55:26,035][70582] Avg episode reward: [(0, '15.030'), (1, '15.330')] [2023-10-11 19:55:26,277][71635] Updated weights for policy 1, policy_version 20922 (0.0007) [2023-10-11 19:55:26,420][71601] Updated weights for policy 0, policy_version 20930 (0.0008) [2023-10-11 19:55:26,793][71601] Updated weights for policy 0, policy_version 20940 (0.0007) [2023-10-11 19:55:27,157][71601] Updated weights for policy 0, policy_version 20950 (0.0008) [2023-10-11 19:55:27,532][71601] Updated weights for policy 0, policy_version 20960 (0.0009) [2023-10-11 19:55:29,943][71635] Updated weights for policy 1, policy_version 20932 (0.0008) [2023-10-11 19:55:30,303][71635] Updated weights for policy 1, policy_version 20942 (0.0007) [2023-10-11 19:55:30,668][71635] Updated weights for policy 1, policy_version 20952 (0.0009) [2023-10-11 19:55:31,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 42926080. Throughput: 0: 1807.6, 1: 1813.7. Samples: 10733986. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-11 19:55:31,035][70582] Avg episode reward: [(0, '15.250'), (1, '14.490')] [2023-10-11 19:55:31,164][71601] Updated weights for policy 0, policy_version 20970 (0.0010) [2023-10-11 19:55:31,544][71601] Updated weights for policy 0, policy_version 20980 (0.0010) [2023-10-11 19:55:31,909][71601] Updated weights for policy 0, policy_version 20990 (0.0008) [2023-10-11 19:55:34,325][71635] Updated weights for policy 1, policy_version 20962 (0.0008) [2023-10-11 19:55:34,685][71635] Updated weights for policy 1, policy_version 20972 (0.0009) [2023-10-11 19:55:35,053][71635] Updated weights for policy 1, policy_version 20982 (0.0007) [2023-10-11 19:55:35,422][71635] Updated weights for policy 1, policy_version 20992 (0.0009) [2023-10-11 19:55:35,664][71601] Updated weights for policy 0, policy_version 21000 (0.0007) [2023-10-11 19:55:36,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 42991616. Throughput: 0: 1806.9, 1: 1818.7. Samples: 10756626. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-11 19:55:36,034][70582] Avg episode reward: [(0, '15.040'), (1, '15.250')] [2023-10-11 19:55:36,036][71601] Updated weights for policy 0, policy_version 21010 (0.0008) [2023-10-11 19:55:36,415][71601] Updated weights for policy 0, policy_version 21020 (0.0007) [2023-10-11 19:55:39,113][71635] Updated weights for policy 1, policy_version 21002 (0.0008) [2023-10-11 19:55:39,472][71635] Updated weights for policy 1, policy_version 21012 (0.0011) [2023-10-11 19:55:39,840][71635] Updated weights for policy 1, policy_version 21022 (0.0007) [2023-10-11 19:55:40,088][71601] Updated weights for policy 0, policy_version 21030 (0.0008) [2023-10-11 19:55:40,451][71601] Updated weights for policy 0, policy_version 21040 (0.0009) [2023-10-11 19:55:40,824][71601] Updated weights for policy 0, policy_version 21050 (0.0009) [2023-10-11 19:55:41,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 43057152. Throughput: 0: 1822.5, 1: 1817.9. Samples: 10777470. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:55:41,034][70582] Avg episode reward: [(0, '14.590'), (1, '13.750')] [2023-10-11 19:55:43,530][71635] Updated weights for policy 1, policy_version 21032 (0.0009) [2023-10-11 19:55:43,890][71635] Updated weights for policy 1, policy_version 21042 (0.0009) [2023-10-11 19:55:44,259][71635] Updated weights for policy 1, policy_version 21052 (0.0008) [2023-10-11 19:55:44,669][71601] Updated weights for policy 0, policy_version 21060 (0.0008) [2023-10-11 19:55:45,043][71601] Updated weights for policy 0, policy_version 21070 (0.0008) [2023-10-11 19:55:45,427][71601] Updated weights for policy 0, policy_version 21080 (0.0009) [2023-10-11 19:55:46,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 43155456. Throughput: 0: 1807.9, 1: 1821.7. Samples: 10789226. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:55:46,035][70582] Avg episode reward: [(0, '14.900'), (1, '14.700')] [2023-10-11 19:55:48,118][71635] Updated weights for policy 1, policy_version 21062 (0.0007) [2023-10-11 19:55:48,484][71635] Updated weights for policy 1, policy_version 21072 (0.0008) [2023-10-11 19:55:48,839][71635] Updated weights for policy 1, policy_version 21082 (0.0008) [2023-10-11 19:55:49,189][71601] Updated weights for policy 0, policy_version 21090 (0.0007) [2023-10-11 19:55:49,562][71601] Updated weights for policy 0, policy_version 21100 (0.0008) [2023-10-11 19:55:49,932][71601] Updated weights for policy 0, policy_version 21110 (0.0009) [2023-10-11 19:55:50,314][71601] Updated weights for policy 0, policy_version 21120 (0.0010) [2023-10-11 19:55:51,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 43220992. Throughput: 0: 1821.4, 1: 1807.8. Samples: 10809922. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:55:51,034][70582] Avg episode reward: [(0, '15.120'), (1, '14.700')] [2023-10-11 19:55:52,488][71635] Updated weights for policy 1, policy_version 21092 (0.0008) [2023-10-11 19:55:52,847][71635] Updated weights for policy 1, policy_version 21102 (0.0007) [2023-10-11 19:55:53,212][71635] Updated weights for policy 1, policy_version 21112 (0.0007) [2023-10-11 19:55:53,946][71601] Updated weights for policy 0, policy_version 21130 (0.0010) [2023-10-11 19:55:54,314][71601] Updated weights for policy 0, policy_version 21140 (0.0010) [2023-10-11 19:55:54,693][71601] Updated weights for policy 0, policy_version 21150 (0.0010) [2023-10-11 19:55:56,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 43286528. Throughput: 0: 1806.8, 1: 1809.7. Samples: 10831538. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:55:56,034][70582] Avg episode reward: [(0, '17.570'), (1, '17.580')] [2023-10-11 19:55:56,043][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000021120_21626880.pth... [2023-10-11 19:55:56,043][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000021152_21659648.pth... [2023-10-11 19:55:56,073][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000019424_19890176.pth [2023-10-11 19:55:56,077][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000019456_19922944.pth [2023-10-11 19:55:56,080][71353] Saving new best policy, reward=17.570! [2023-10-11 19:55:56,895][71635] Updated weights for policy 1, policy_version 21122 (0.0008) [2023-10-11 19:55:57,256][71635] Updated weights for policy 1, policy_version 21132 (0.0009) [2023-10-11 19:55:57,627][71635] Updated weights for policy 1, policy_version 21142 (0.0008) [2023-10-11 19:55:57,988][71635] Updated weights for policy 1, policy_version 21152 (0.0007) [2023-10-11 19:55:58,473][71601] Updated weights for policy 0, policy_version 21160 (0.0008) [2023-10-11 19:55:58,853][71601] Updated weights for policy 0, policy_version 21170 (0.0009) [2023-10-11 19:55:59,216][71601] Updated weights for policy 0, policy_version 21180 (0.0009) [2023-10-11 19:56:01,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 43352064. Throughput: 0: 1814.7, 1: 1808.2. Samples: 10842384. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:56:01,034][70582] Avg episode reward: [(0, '19.240'), (1, '18.170')] [2023-10-11 19:56:01,035][71353] Saving new best policy, reward=19.240! [2023-10-11 19:56:01,035][71431] Saving new best policy, reward=18.170! [2023-10-11 19:56:01,673][71635] Updated weights for policy 1, policy_version 21162 (0.0008) [2023-10-11 19:56:02,038][71635] Updated weights for policy 1, policy_version 21172 (0.0008) [2023-10-11 19:56:02,404][71635] Updated weights for policy 1, policy_version 21182 (0.0009) [2023-10-11 19:56:02,778][71601] Updated weights for policy 0, policy_version 21190 (0.0010) [2023-10-11 19:56:03,157][71601] Updated weights for policy 0, policy_version 21200 (0.0009) [2023-10-11 19:56:03,520][71601] Updated weights for policy 0, policy_version 21210 (0.0007) [2023-10-11 19:56:06,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 43417600. Throughput: 0: 1793.9, 1: 1807.9. Samples: 10863668. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:56:06,034][70582] Avg episode reward: [(0, '19.230'), (1, '18.850')] [2023-10-11 19:56:06,159][71635] Updated weights for policy 1, policy_version 21192 (0.0008) [2023-10-11 19:56:06,526][71635] Updated weights for policy 1, policy_version 21202 (0.0008) [2023-10-11 19:56:06,893][71635] Updated weights for policy 1, policy_version 21212 (0.0009) [2023-10-11 19:56:07,039][71431] Saving new best policy, reward=18.850! [2023-10-11 19:56:07,370][71601] Updated weights for policy 0, policy_version 21220 (0.0009) [2023-10-11 19:56:07,769][71601] Updated weights for policy 0, policy_version 21230 (0.0007) [2023-10-11 19:56:08,144][71601] Updated weights for policy 0, policy_version 21240 (0.0010) [2023-10-11 19:56:10,426][71635] Updated weights for policy 1, policy_version 21222 (0.0008) [2023-10-11 19:56:10,798][71635] Updated weights for policy 1, policy_version 21232 (0.0011) [2023-10-11 19:56:11,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 43483136. Throughput: 0: 1792.5, 1: 1812.0. Samples: 10886128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:56:11,035][70582] Avg episode reward: [(0, '20.280'), (1, '18.970')] [2023-10-11 19:56:11,044][71353] Saving new best policy, reward=20.280! [2023-10-11 19:56:11,171][71635] Updated weights for policy 1, policy_version 21242 (0.0008) [2023-10-11 19:56:11,388][71431] Saving new best policy, reward=18.970! [2023-10-11 19:56:11,974][71601] Updated weights for policy 0, policy_version 21250 (0.0008) [2023-10-11 19:56:12,342][71601] Updated weights for policy 0, policy_version 21260 (0.0008) [2023-10-11 19:56:12,717][71601] Updated weights for policy 0, policy_version 21270 (0.0007) [2023-10-11 19:56:13,091][71601] Updated weights for policy 0, policy_version 21280 (0.0007) [2023-10-11 19:56:14,817][71635] Updated weights for policy 1, policy_version 21252 (0.0008) [2023-10-11 19:56:15,188][71635] Updated weights for policy 1, policy_version 21262 (0.0008) [2023-10-11 19:56:15,545][71635] Updated weights for policy 1, policy_version 21272 (0.0010) [2023-10-11 19:56:16,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 43581440. Throughput: 0: 1791.1, 1: 1816.7. Samples: 10896334. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) [2023-10-11 19:56:16,034][70582] Avg episode reward: [(0, '22.160'), (1, '18.520')] [2023-10-11 19:56:16,035][71353] Saving new best policy, reward=22.160! [2023-10-11 19:56:16,642][71601] Updated weights for policy 0, policy_version 21290 (0.0008) [2023-10-11 19:56:17,011][71601] Updated weights for policy 0, policy_version 21300 (0.0008) [2023-10-11 19:56:17,384][71601] Updated weights for policy 0, policy_version 21310 (0.0007) [2023-10-11 19:56:19,171][71635] Updated weights for policy 1, policy_version 21282 (0.0009) [2023-10-11 19:56:19,533][71635] Updated weights for policy 1, policy_version 21292 (0.0009) [2023-10-11 19:56:19,907][71635] Updated weights for policy 1, policy_version 21302 (0.0008) [2023-10-11 19:56:20,266][71635] Updated weights for policy 1, policy_version 21312 (0.0010) [2023-10-11 19:56:21,033][71601] Updated weights for policy 0, policy_version 21320 (0.0008) [2023-10-11 19:56:21,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 43646976. Throughput: 0: 1796.3, 1: 1814.3. Samples: 10919102. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) [2023-10-11 19:56:21,035][70582] Avg episode reward: [(0, '20.420'), (1, '17.030')] [2023-10-11 19:56:21,416][71601] Updated weights for policy 0, policy_version 21330 (0.0009) [2023-10-11 19:56:21,780][71601] Updated weights for policy 0, policy_version 21340 (0.0007) [2023-10-11 19:56:24,101][71635] Updated weights for policy 1, policy_version 21322 (0.0007) [2023-10-11 19:56:24,473][71635] Updated weights for policy 1, policy_version 21332 (0.0008) [2023-10-11 19:56:24,835][71635] Updated weights for policy 1, policy_version 21342 (0.0007) [2023-10-11 19:56:25,540][71601] Updated weights for policy 0, policy_version 21350 (0.0008) [2023-10-11 19:56:25,905][71601] Updated weights for policy 0, policy_version 21360 (0.0007) [2023-10-11 19:56:26,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 43712512. Throughput: 0: 1803.2, 1: 1817.0. Samples: 10940382. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) [2023-10-11 19:56:26,035][70582] Avg episode reward: [(0, '18.650'), (1, '15.960')] [2023-10-11 19:56:26,273][71601] Updated weights for policy 0, policy_version 21370 (0.0008) [2023-10-11 19:56:28,541][71635] Updated weights for policy 1, policy_version 21352 (0.0009) [2023-10-11 19:56:28,901][71635] Updated weights for policy 1, policy_version 21362 (0.0009) [2023-10-11 19:56:29,262][71635] Updated weights for policy 1, policy_version 21372 (0.0009) [2023-10-11 19:56:30,084][71601] Updated weights for policy 0, policy_version 21380 (0.0008) [2023-10-11 19:56:30,454][71601] Updated weights for policy 0, policy_version 21390 (0.0009) [2023-10-11 19:56:30,816][71601] Updated weights for policy 0, policy_version 21400 (0.0007) [2023-10-11 19:56:31,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 43778048. Throughput: 0: 1791.3, 1: 1817.3. Samples: 10951614. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) [2023-10-11 19:56:31,034][70582] Avg episode reward: [(0, '19.090'), (1, '17.020')] [2023-10-11 19:56:33,049][71635] Updated weights for policy 1, policy_version 21382 (0.0008) [2023-10-11 19:56:33,416][71635] Updated weights for policy 1, policy_version 21392 (0.0007) [2023-10-11 19:56:33,785][71635] Updated weights for policy 1, policy_version 21402 (0.0009) [2023-10-11 19:56:34,307][71601] Updated weights for policy 0, policy_version 21410 (0.0007) [2023-10-11 19:56:34,683][71601] Updated weights for policy 0, policy_version 21420 (0.0008) [2023-10-11 19:56:35,057][71601] Updated weights for policy 0, policy_version 21430 (0.0008) [2023-10-11 19:56:35,427][71601] Updated weights for policy 0, policy_version 21440 (0.0009) [2023-10-11 19:56:36,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 43876352. Throughput: 0: 1799.9, 1: 1817.4. Samples: 10972702. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-11 19:56:36,034][70582] Avg episode reward: [(0, '20.300'), (1, '16.420')] [2023-10-11 19:56:37,489][71635] Updated weights for policy 1, policy_version 21412 (0.0011) [2023-10-11 19:56:37,853][71635] Updated weights for policy 1, policy_version 21422 (0.0010) [2023-10-11 19:56:38,219][71635] Updated weights for policy 1, policy_version 21432 (0.0008) [2023-10-11 19:56:39,088][71601] Updated weights for policy 0, policy_version 21450 (0.0009) [2023-10-11 19:56:39,449][71601] Updated weights for policy 0, policy_version 21460 (0.0008) [2023-10-11 19:56:39,832][71601] Updated weights for policy 0, policy_version 21470 (0.0008) [2023-10-11 19:56:41,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 43941888. Throughput: 0: 1799.0, 1: 1821.9. Samples: 10994478. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-11 19:56:41,035][70582] Avg episode reward: [(0, '20.070'), (1, '19.090')] [2023-10-11 19:56:41,046][71431] Saving new best policy, reward=19.090! [2023-10-11 19:56:41,941][71635] Updated weights for policy 1, policy_version 21442 (0.0009) [2023-10-11 19:56:42,306][71635] Updated weights for policy 1, policy_version 21452 (0.0008) [2023-10-11 19:56:42,675][71635] Updated weights for policy 1, policy_version 21462 (0.0010) [2023-10-11 19:56:43,042][71635] Updated weights for policy 1, policy_version 21472 (0.0007) [2023-10-11 19:56:43,570][71601] Updated weights for policy 0, policy_version 21480 (0.0010) [2023-10-11 19:56:43,938][71601] Updated weights for policy 0, policy_version 21490 (0.0008) [2023-10-11 19:56:44,306][71601] Updated weights for policy 0, policy_version 21500 (0.0008) [2023-10-11 19:56:46,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 44007424. Throughput: 0: 1803.4, 1: 1823.9. Samples: 11005614. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-11 19:56:46,035][70582] Avg episode reward: [(0, '19.080'), (1, '18.440')] [2023-10-11 19:56:46,625][71635] Updated weights for policy 1, policy_version 21482 (0.0011) [2023-10-11 19:56:46,988][71635] Updated weights for policy 1, policy_version 21492 (0.0007) [2023-10-11 19:56:47,354][71635] Updated weights for policy 1, policy_version 21502 (0.0008) [2023-10-11 19:56:48,138][71601] Updated weights for policy 0, policy_version 21510 (0.0007) [2023-10-11 19:56:48,505][71601] Updated weights for policy 0, policy_version 21520 (0.0007) [2023-10-11 19:56:48,877][71601] Updated weights for policy 0, policy_version 21530 (0.0010) [2023-10-11 19:56:51,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 44072960. Throughput: 0: 1804.3, 1: 1828.5. Samples: 11027146. Policy #0 lag: (min: 10.0, avg: 10.1, max: 15.0) [2023-10-11 19:56:51,035][70582] Avg episode reward: [(0, '19.480'), (1, '17.160')] [2023-10-11 19:56:51,074][71635] Updated weights for policy 1, policy_version 21512 (0.0010) [2023-10-11 19:56:51,448][71635] Updated weights for policy 1, policy_version 21522 (0.0008) [2023-10-11 19:56:51,803][71635] Updated weights for policy 1, policy_version 21532 (0.0009) [2023-10-11 19:56:52,694][71601] Updated weights for policy 0, policy_version 21540 (0.0009) [2023-10-11 19:56:53,066][71601] Updated weights for policy 0, policy_version 21550 (0.0008) [2023-10-11 19:56:53,440][71601] Updated weights for policy 0, policy_version 21560 (0.0009) [2023-10-11 19:56:55,434][71635] Updated weights for policy 1, policy_version 21542 (0.0009) [2023-10-11 19:56:55,808][71635] Updated weights for policy 1, policy_version 21552 (0.0009) [2023-10-11 19:56:56,034][70582] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 44138496. Throughput: 0: 1821.1, 1: 1825.8. Samples: 11050238. Policy #0 lag: (min: 10.0, avg: 10.1, max: 15.0) [2023-10-11 19:56:56,035][70582] Avg episode reward: [(0, '19.560'), (1, '15.210')] [2023-10-11 19:56:56,176][71635] Updated weights for policy 1, policy_version 21562 (0.0009) [2023-10-11 19:56:57,029][71601] Updated weights for policy 0, policy_version 21570 (0.0010) [2023-10-11 19:56:57,405][71601] Updated weights for policy 0, policy_version 21580 (0.0009) [2023-10-11 19:56:57,771][71601] Updated weights for policy 0, policy_version 21590 (0.0009) [2023-10-11 19:56:58,142][71601] Updated weights for policy 0, policy_version 21600 (0.0007) [2023-10-11 19:56:59,860][71635] Updated weights for policy 1, policy_version 21572 (0.0007) [2023-10-11 19:57:00,224][71635] Updated weights for policy 1, policy_version 21582 (0.0008) [2023-10-11 19:57:00,590][71635] Updated weights for policy 1, policy_version 21592 (0.0007) [2023-10-11 19:57:01,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 44236800. Throughput: 0: 1818.2, 1: 1824.9. Samples: 11060274. Policy #0 lag: (min: 10.0, avg: 10.1, max: 15.0) [2023-10-11 19:57:01,034][70582] Avg episode reward: [(0, '19.940'), (1, '13.580')] [2023-10-11 19:57:01,732][71601] Updated weights for policy 0, policy_version 21610 (0.0007) [2023-10-11 19:57:02,104][71601] Updated weights for policy 0, policy_version 21620 (0.0007) [2023-10-11 19:57:02,478][71601] Updated weights for policy 0, policy_version 21630 (0.0009) [2023-10-11 19:57:04,343][71635] Updated weights for policy 1, policy_version 21602 (0.0008) [2023-10-11 19:57:04,710][71635] Updated weights for policy 1, policy_version 21612 (0.0009) [2023-10-11 19:57:05,077][71635] Updated weights for policy 1, policy_version 21622 (0.0009) [2023-10-11 19:57:05,443][71635] Updated weights for policy 1, policy_version 21632 (0.0008) [2023-10-11 19:57:06,034][70582] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 44302336. Throughput: 0: 1818.3, 1: 1822.2. Samples: 11082924. Policy #0 lag: (min: 10.0, avg: 10.1, max: 15.0) [2023-10-11 19:57:06,035][70582] Avg episode reward: [(0, '19.300'), (1, '12.720')] [2023-10-11 19:57:06,100][71601] Updated weights for policy 0, policy_version 21640 (0.0009) [2023-10-11 19:57:06,468][71601] Updated weights for policy 0, policy_version 21650 (0.0007) [2023-10-11 19:57:06,851][71601] Updated weights for policy 0, policy_version 21660 (0.0007) [2023-10-11 19:57:09,280][71635] Updated weights for policy 1, policy_version 21642 (0.0007) [2023-10-11 19:57:09,657][71635] Updated weights for policy 1, policy_version 21652 (0.0008) [2023-10-11 19:57:10,029][71635] Updated weights for policy 1, policy_version 21662 (0.0007) [2023-10-11 19:57:10,610][71601] Updated weights for policy 0, policy_version 21670 (0.0009) [2023-10-11 19:57:10,992][71601] Updated weights for policy 0, policy_version 21680 (0.0007) [2023-10-11 19:57:11,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 44367872. Throughput: 0: 1821.7, 1: 1813.5. Samples: 11103964. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-11 19:57:11,034][70582] Avg episode reward: [(0, '18.190'), (1, '12.720')] [2023-10-11 19:57:11,364][71601] Updated weights for policy 0, policy_version 21690 (0.0007) [2023-10-11 19:57:13,695][71635] Updated weights for policy 1, policy_version 21672 (0.0007) [2023-10-11 19:57:14,061][71635] Updated weights for policy 1, policy_version 21682 (0.0010) [2023-10-11 19:57:14,428][71635] Updated weights for policy 1, policy_version 21692 (0.0009) [2023-10-11 19:57:14,984][71601] Updated weights for policy 0, policy_version 21700 (0.0007) [2023-10-11 19:57:15,354][71601] Updated weights for policy 0, policy_version 21710 (0.0008) [2023-10-11 19:57:15,733][71601] Updated weights for policy 0, policy_version 21720 (0.0010) [2023-10-11 19:57:16,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 44466176. Throughput: 0: 1821.9, 1: 1819.9. Samples: 11115496. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-11 19:57:16,035][70582] Avg episode reward: [(0, '18.450'), (1, '12.880')] [2023-10-11 19:57:18,154][71635] Updated weights for policy 1, policy_version 21702 (0.0009) [2023-10-11 19:57:18,519][71635] Updated weights for policy 1, policy_version 21712 (0.0010) [2023-10-11 19:57:18,891][71635] Updated weights for policy 1, policy_version 21722 (0.0009) [2023-10-11 19:57:19,499][71601] Updated weights for policy 0, policy_version 21730 (0.0011) [2023-10-11 19:57:19,877][71601] Updated weights for policy 0, policy_version 21740 (0.0011) [2023-10-11 19:57:20,243][71601] Updated weights for policy 0, policy_version 21750 (0.0010) [2023-10-11 19:57:20,624][71601] Updated weights for policy 0, policy_version 21760 (0.0009) [2023-10-11 19:57:21,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 44531712. Throughput: 0: 1824.8, 1: 1820.7. Samples: 11136752. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-11 19:57:21,035][70582] Avg episode reward: [(0, '18.990'), (1, '14.570')] [2023-10-11 19:57:22,372][71635] Updated weights for policy 1, policy_version 21732 (0.0007) [2023-10-11 19:57:22,739][71635] Updated weights for policy 1, policy_version 21742 (0.0010) [2023-10-11 19:57:23,108][71635] Updated weights for policy 1, policy_version 21752 (0.0011) [2023-10-11 19:57:24,231][71601] Updated weights for policy 0, policy_version 21770 (0.0008) [2023-10-11 19:57:24,601][71601] Updated weights for policy 0, policy_version 21780 (0.0007) [2023-10-11 19:57:24,977][71601] Updated weights for policy 0, policy_version 21790 (0.0007) [2023-10-11 19:57:26,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 44597248. Throughput: 0: 1822.6, 1: 1819.2. Samples: 11158360. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-10-11 19:57:26,034][70582] Avg episode reward: [(0, '19.280'), (1, '15.770')] [2023-10-11 19:57:26,885][71635] Updated weights for policy 1, policy_version 21762 (0.0009) [2023-10-11 19:57:27,258][71635] Updated weights for policy 1, policy_version 21772 (0.0008) [2023-10-11 19:57:27,621][71635] Updated weights for policy 1, policy_version 21782 (0.0010) [2023-10-11 19:57:27,986][71635] Updated weights for policy 1, policy_version 21792 (0.0010) [2023-10-11 19:57:28,548][71601] Updated weights for policy 0, policy_version 21800 (0.0009) [2023-10-11 19:57:28,920][71601] Updated weights for policy 0, policy_version 21810 (0.0009) [2023-10-11 19:57:29,289][71601] Updated weights for policy 0, policy_version 21820 (0.0008) [2023-10-11 19:57:31,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 44662784. Throughput: 0: 1825.1, 1: 1816.5. Samples: 11169482. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-10-11 19:57:31,034][70582] Avg episode reward: [(0, '19.360'), (1, '15.080')] [2023-10-11 19:57:31,836][71635] Updated weights for policy 1, policy_version 21802 (0.0008) [2023-10-11 19:57:32,191][71635] Updated weights for policy 1, policy_version 21812 (0.0007) [2023-10-11 19:57:32,565][71635] Updated weights for policy 1, policy_version 21822 (0.0008) [2023-10-11 19:57:32,998][71601] Updated weights for policy 0, policy_version 21830 (0.0008) [2023-10-11 19:57:33,369][71601] Updated weights for policy 0, policy_version 21840 (0.0007) [2023-10-11 19:57:33,736][71601] Updated weights for policy 0, policy_version 21850 (0.0008) [2023-10-11 19:57:36,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 44728320. Throughput: 0: 1832.0, 1: 1815.2. Samples: 11191270. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-10-11 19:57:36,034][70582] Avg episode reward: [(0, '18.990'), (1, '15.640')] [2023-10-11 19:57:36,193][71635] Updated weights for policy 1, policy_version 21832 (0.0010) [2023-10-11 19:57:36,560][71635] Updated weights for policy 1, policy_version 21842 (0.0011) [2023-10-11 19:57:36,925][71635] Updated weights for policy 1, policy_version 21852 (0.0010) [2023-10-11 19:57:37,398][71601] Updated weights for policy 0, policy_version 21860 (0.0008) [2023-10-11 19:57:37,772][71601] Updated weights for policy 0, policy_version 21870 (0.0008) [2023-10-11 19:57:38,134][71601] Updated weights for policy 0, policy_version 21880 (0.0008) [2023-10-11 19:57:40,723][71635] Updated weights for policy 1, policy_version 21862 (0.0008) [2023-10-11 19:57:41,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 44793856. Throughput: 0: 1823.9, 1: 1818.7. Samples: 11214152. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-10-11 19:57:41,034][70582] Avg episode reward: [(0, '21.130'), (1, '15.970')] [2023-10-11 19:57:41,093][71635] Updated weights for policy 1, policy_version 21872 (0.0007) [2023-10-11 19:57:41,452][71635] Updated weights for policy 1, policy_version 21882 (0.0007) [2023-10-11 19:57:41,871][71601] Updated weights for policy 0, policy_version 21890 (0.0008) [2023-10-11 19:57:42,289][71601] Updated weights for policy 0, policy_version 21900 (0.0010) [2023-10-11 19:57:42,664][71601] Updated weights for policy 0, policy_version 21910 (0.0009) [2023-10-11 19:57:43,033][71601] Updated weights for policy 0, policy_version 21920 (0.0007) [2023-10-11 19:57:44,991][71635] Updated weights for policy 1, policy_version 21892 (0.0008) [2023-10-11 19:57:45,370][71635] Updated weights for policy 1, policy_version 21902 (0.0008) [2023-10-11 19:57:45,726][71635] Updated weights for policy 1, policy_version 21912 (0.0007) [2023-10-11 19:57:46,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 44892160. Throughput: 0: 1824.7, 1: 1816.8. Samples: 11224144. Policy #0 lag: (min: 6.0, avg: 12.7, max: 38.0) [2023-10-11 19:57:46,034][70582] Avg episode reward: [(0, '22.820'), (1, '15.910')] [2023-10-11 19:57:46,035][71353] Saving new best policy, reward=22.820! [2023-10-11 19:57:46,710][71601] Updated weights for policy 0, policy_version 21930 (0.0008) [2023-10-11 19:57:47,079][71601] Updated weights for policy 0, policy_version 21940 (0.0009) [2023-10-11 19:57:47,454][71601] Updated weights for policy 0, policy_version 21950 (0.0008) [2023-10-11 19:57:49,261][71635] Updated weights for policy 1, policy_version 21922 (0.0009) [2023-10-11 19:57:49,631][71635] Updated weights for policy 1, policy_version 21932 (0.0007) [2023-10-11 19:57:50,001][71635] Updated weights for policy 1, policy_version 21942 (0.0008) [2023-10-11 19:57:50,366][71635] Updated weights for policy 1, policy_version 21952 (0.0009) [2023-10-11 19:57:51,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 44957696. Throughput: 0: 1817.2, 1: 1819.8. Samples: 11246590. Policy #0 lag: (min: 6.0, avg: 12.7, max: 38.0) [2023-10-11 19:57:51,035][70582] Avg episode reward: [(0, '22.500'), (1, '14.610')] [2023-10-11 19:57:51,065][71601] Updated weights for policy 0, policy_version 21960 (0.0009) [2023-10-11 19:57:51,443][71601] Updated weights for policy 0, policy_version 21970 (0.0010) [2023-10-11 19:57:51,812][71601] Updated weights for policy 0, policy_version 21980 (0.0012) [2023-10-11 19:57:54,041][71635] Updated weights for policy 1, policy_version 21962 (0.0011) [2023-10-11 19:57:54,414][71635] Updated weights for policy 1, policy_version 21972 (0.0007) [2023-10-11 19:57:54,770][71635] Updated weights for policy 1, policy_version 21982 (0.0008) [2023-10-11 19:57:55,499][71601] Updated weights for policy 0, policy_version 21990 (0.0007) [2023-10-11 19:57:55,877][71601] Updated weights for policy 0, policy_version 22000 (0.0007) [2023-10-11 19:57:56,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 45023232. Throughput: 0: 1819.8, 1: 1833.2. Samples: 11268350. Policy #0 lag: (min: 6.0, avg: 12.7, max: 38.0) [2023-10-11 19:57:56,035][70582] Avg episode reward: [(0, '23.160'), (1, '15.850')] [2023-10-11 19:57:56,044][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000021984_22511616.pth... [2023-10-11 19:57:56,078][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000020288_20774912.pth [2023-10-11 19:57:56,252][71601] Updated weights for policy 0, policy_version 22010 (0.0007) [2023-10-11 19:57:56,475][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000022016_22544384.pth... [2023-10-11 19:57:56,503][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000020320_20807680.pth [2023-10-11 19:57:56,506][71353] Saving new best policy, reward=23.160! [2023-10-11 19:57:58,383][71635] Updated weights for policy 1, policy_version 21992 (0.0007) [2023-10-11 19:57:58,747][71635] Updated weights for policy 1, policy_version 22002 (0.0008) [2023-10-11 19:57:59,119][71635] Updated weights for policy 1, policy_version 22012 (0.0010) [2023-10-11 19:57:59,831][71601] Updated weights for policy 0, policy_version 22020 (0.0007) [2023-10-11 19:58:00,193][71601] Updated weights for policy 0, policy_version 22030 (0.0009) [2023-10-11 19:58:00,571][71601] Updated weights for policy 0, policy_version 22040 (0.0008) [2023-10-11 19:58:01,034][70582] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 45121536. Throughput: 0: 1827.6, 1: 1824.1. Samples: 11279826. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-11 19:58:01,034][70582] Avg episode reward: [(0, '21.780'), (1, '16.770')] [2023-10-11 19:58:02,925][71635] Updated weights for policy 1, policy_version 22022 (0.0010) [2023-10-11 19:58:03,290][71635] Updated weights for policy 1, policy_version 22032 (0.0008) [2023-10-11 19:58:03,655][71635] Updated weights for policy 1, policy_version 22042 (0.0008) [2023-10-11 19:58:04,253][71601] Updated weights for policy 0, policy_version 22050 (0.0008) [2023-10-11 19:58:04,625][71601] Updated weights for policy 0, policy_version 22060 (0.0010) [2023-10-11 19:58:04,989][71601] Updated weights for policy 0, policy_version 22070 (0.0010) [2023-10-11 19:58:05,362][71601] Updated weights for policy 0, policy_version 22080 (0.0007) [2023-10-11 19:58:06,034][70582] Fps is (10 sec: 16384.5, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 45187072. Throughput: 0: 1821.2, 1: 1828.6. Samples: 11300994. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-11 19:58:06,034][70582] Avg episode reward: [(0, '21.510'), (1, '17.210')] [2023-10-11 19:58:07,330][71635] Updated weights for policy 1, policy_version 22052 (0.0007) [2023-10-11 19:58:07,700][71635] Updated weights for policy 1, policy_version 22062 (0.0008) [2023-10-11 19:58:08,066][71635] Updated weights for policy 1, policy_version 22072 (0.0008) [2023-10-11 19:58:09,076][71601] Updated weights for policy 0, policy_version 22090 (0.0011) [2023-10-11 19:58:09,436][71601] Updated weights for policy 0, policy_version 22100 (0.0008) [2023-10-11 19:58:09,817][71601] Updated weights for policy 0, policy_version 22110 (0.0011) [2023-10-11 19:58:11,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 45252608. Throughput: 0: 1827.1, 1: 1821.6. Samples: 11322548. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-11 19:58:11,034][70582] Avg episode reward: [(0, '24.620'), (1, '17.580')] [2023-10-11 19:58:11,042][71353] Saving new best policy, reward=24.620! [2023-10-11 19:58:11,832][71635] Updated weights for policy 1, policy_version 22082 (0.0008) [2023-10-11 19:58:12,204][71635] Updated weights for policy 1, policy_version 22092 (0.0007) [2023-10-11 19:58:12,568][71635] Updated weights for policy 1, policy_version 22102 (0.0008) [2023-10-11 19:58:12,934][71635] Updated weights for policy 1, policy_version 22112 (0.0009) [2023-10-11 19:58:13,532][71601] Updated weights for policy 0, policy_version 22120 (0.0008) [2023-10-11 19:58:13,901][71601] Updated weights for policy 0, policy_version 22130 (0.0010) [2023-10-11 19:58:14,275][71601] Updated weights for policy 0, policy_version 22140 (0.0010) [2023-10-11 19:58:16,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 45318144. Throughput: 0: 1826.9, 1: 1825.7. Samples: 11333852. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-11 19:58:16,035][70582] Avg episode reward: [(0, '26.820'), (1, '20.270')] [2023-10-11 19:58:16,036][71353] Saving new best policy, reward=26.820! [2023-10-11 19:58:16,036][71431] Saving new best policy, reward=20.270! [2023-10-11 19:58:16,659][71635] Updated weights for policy 1, policy_version 22122 (0.0010) [2023-10-11 19:58:17,029][71635] Updated weights for policy 1, policy_version 22132 (0.0008) [2023-10-11 19:58:17,401][71635] Updated weights for policy 1, policy_version 22142 (0.0008) [2023-10-11 19:58:18,019][71601] Updated weights for policy 0, policy_version 22150 (0.0007) [2023-10-11 19:58:18,387][71601] Updated weights for policy 0, policy_version 22160 (0.0008) [2023-10-11 19:58:18,760][71601] Updated weights for policy 0, policy_version 22170 (0.0007) [2023-10-11 19:58:21,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 45383680. Throughput: 0: 1821.4, 1: 1826.9. Samples: 11355444. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:58:21,034][70582] Avg episode reward: [(0, '26.650'), (1, '19.580')] [2023-10-11 19:58:21,113][71635] Updated weights for policy 1, policy_version 22152 (0.0009) [2023-10-11 19:58:21,483][71635] Updated weights for policy 1, policy_version 22162 (0.0010) [2023-10-11 19:58:21,855][71635] Updated weights for policy 1, policy_version 22172 (0.0008) [2023-10-11 19:58:22,246][71601] Updated weights for policy 0, policy_version 22180 (0.0009) [2023-10-11 19:58:22,622][71601] Updated weights for policy 0, policy_version 22190 (0.0012) [2023-10-11 19:58:22,991][71601] Updated weights for policy 0, policy_version 22200 (0.0007) [2023-10-11 19:58:25,585][71635] Updated weights for policy 1, policy_version 22182 (0.0010) [2023-10-11 19:58:25,936][71635] Updated weights for policy 1, policy_version 22192 (0.0007) [2023-10-11 19:58:26,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 45449216. Throughput: 0: 1827.2, 1: 1821.0. Samples: 11378324. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:58:26,034][70582] Avg episode reward: [(0, '25.020'), (1, '18.920')] [2023-10-11 19:58:26,305][71635] Updated weights for policy 1, policy_version 22202 (0.0009) [2023-10-11 19:58:26,634][71601] Updated weights for policy 0, policy_version 22210 (0.0007) [2023-10-11 19:58:27,044][71601] Updated weights for policy 0, policy_version 22220 (0.0008) [2023-10-11 19:58:27,419][71601] Updated weights for policy 0, policy_version 22230 (0.0007) [2023-10-11 19:58:27,784][71601] Updated weights for policy 0, policy_version 22240 (0.0009) [2023-10-11 19:58:29,944][71635] Updated weights for policy 1, policy_version 22212 (0.0009) [2023-10-11 19:58:30,314][71635] Updated weights for policy 1, policy_version 22222 (0.0008) [2023-10-11 19:58:30,675][71635] Updated weights for policy 1, policy_version 22232 (0.0008) [2023-10-11 19:58:31,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 45547520. Throughput: 0: 1829.0, 1: 1822.6. Samples: 11388468. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:58:31,034][70582] Avg episode reward: [(0, '27.760'), (1, '21.360')] [2023-10-11 19:58:31,035][71431] Saving new best policy, reward=21.360! [2023-10-11 19:58:31,403][71601] Updated weights for policy 0, policy_version 22250 (0.0008) [2023-10-11 19:58:31,775][71601] Updated weights for policy 0, policy_version 22260 (0.0007) [2023-10-11 19:58:32,146][71601] Updated weights for policy 0, policy_version 22270 (0.0007) [2023-10-11 19:58:32,215][71353] Saving new best policy, reward=27.760! [2023-10-11 19:58:34,486][71635] Updated weights for policy 1, policy_version 22242 (0.0007) [2023-10-11 19:58:34,846][71635] Updated weights for policy 1, policy_version 22252 (0.0007) [2023-10-11 19:58:35,220][71635] Updated weights for policy 1, policy_version 22262 (0.0009) [2023-10-11 19:58:35,578][71635] Updated weights for policy 1, policy_version 22272 (0.0009) [2023-10-11 19:58:35,791][71601] Updated weights for policy 0, policy_version 22280 (0.0008) [2023-10-11 19:58:36,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 45613056. Throughput: 0: 1833.6, 1: 1823.1. Samples: 11411138. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:58:36,035][70582] Avg episode reward: [(0, '28.740'), (1, '22.450')] [2023-10-11 19:58:36,036][71431] Saving new best policy, reward=22.450! [2023-10-11 19:58:36,174][71601] Updated weights for policy 0, policy_version 22290 (0.0008) [2023-10-11 19:58:36,538][71601] Updated weights for policy 0, policy_version 22300 (0.0007) [2023-10-11 19:58:36,685][71353] Saving new best policy, reward=28.740! [2023-10-11 19:58:39,519][71635] Updated weights for policy 1, policy_version 22282 (0.0008) [2023-10-11 19:58:39,895][71635] Updated weights for policy 1, policy_version 22292 (0.0007) [2023-10-11 19:58:40,224][71601] Updated weights for policy 0, policy_version 22310 (0.0008) [2023-10-11 19:58:40,254][71635] Updated weights for policy 1, policy_version 22302 (0.0008) [2023-10-11 19:58:40,611][71601] Updated weights for policy 0, policy_version 22320 (0.0009) [2023-10-11 19:58:40,970][71601] Updated weights for policy 0, policy_version 22330 (0.0010) [2023-10-11 19:58:41,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 45678592. Throughput: 0: 1823.1, 1: 1804.2. Samples: 11431576. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:58:41,034][70582] Avg episode reward: [(0, '34.360'), (1, '20.490')] [2023-10-11 19:58:41,189][71353] Saving new best policy, reward=34.360! [2023-10-11 19:58:43,927][71635] Updated weights for policy 1, policy_version 22312 (0.0010) [2023-10-11 19:58:44,283][71635] Updated weights for policy 1, policy_version 22322 (0.0010) [2023-10-11 19:58:44,615][71601] Updated weights for policy 0, policy_version 22340 (0.0009) [2023-10-11 19:58:44,643][71635] Updated weights for policy 1, policy_version 22332 (0.0008) [2023-10-11 19:58:44,988][71601] Updated weights for policy 0, policy_version 22350 (0.0008) [2023-10-11 19:58:45,356][71601] Updated weights for policy 0, policy_version 22360 (0.0007) [2023-10-11 19:58:46,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 45776896. Throughput: 0: 1829.8, 1: 1811.3. Samples: 11443676. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:58:46,034][70582] Avg episode reward: [(0, '34.960'), (1, '22.270')] [2023-10-11 19:58:46,035][71353] Saving new best policy, reward=34.960! [2023-10-11 19:58:48,491][71635] Updated weights for policy 1, policy_version 22342 (0.0009) [2023-10-11 19:58:48,848][71635] Updated weights for policy 1, policy_version 22352 (0.0008) [2023-10-11 19:58:49,062][71601] Updated weights for policy 0, policy_version 22370 (0.0007) [2023-10-11 19:58:49,217][71635] Updated weights for policy 1, policy_version 22362 (0.0007) [2023-10-11 19:58:49,433][71601] Updated weights for policy 0, policy_version 22380 (0.0007) [2023-10-11 19:58:49,798][71601] Updated weights for policy 0, policy_version 22390 (0.0007) [2023-10-11 19:58:50,176][71601] Updated weights for policy 0, policy_version 22400 (0.0008) [2023-10-11 19:58:51,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 45842432. Throughput: 0: 1827.8, 1: 1805.3. Samples: 11464486. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:58:51,034][70582] Avg episode reward: [(0, '34.210'), (1, '24.530')] [2023-10-11 19:58:51,035][71431] Saving new best policy, reward=24.530! [2023-10-11 19:58:52,795][71635] Updated weights for policy 1, policy_version 22372 (0.0008) [2023-10-11 19:58:53,159][71635] Updated weights for policy 1, policy_version 22382 (0.0011) [2023-10-11 19:58:53,528][71635] Updated weights for policy 1, policy_version 22392 (0.0008) [2023-10-11 19:58:53,792][71601] Updated weights for policy 0, policy_version 22410 (0.0008) [2023-10-11 19:58:54,174][71601] Updated weights for policy 0, policy_version 22420 (0.0011) [2023-10-11 19:58:54,546][71601] Updated weights for policy 0, policy_version 22430 (0.0010) [2023-10-11 19:58:56,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 45907968. Throughput: 0: 1834.1, 1: 1806.7. Samples: 11486386. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-11 19:58:56,035][70582] Avg episode reward: [(0, '34.150'), (1, '23.730')] [2023-10-11 19:58:57,128][71635] Updated weights for policy 1, policy_version 22402 (0.0008) [2023-10-11 19:58:57,498][71635] Updated weights for policy 1, policy_version 22412 (0.0009) [2023-10-11 19:58:57,861][71635] Updated weights for policy 1, policy_version 22422 (0.0008) [2023-10-11 19:58:58,203][71601] Updated weights for policy 0, policy_version 22440 (0.0008) [2023-10-11 19:58:58,221][71635] Updated weights for policy 1, policy_version 22432 (0.0007) [2023-10-11 19:58:58,577][71601] Updated weights for policy 0, policy_version 22450 (0.0010) [2023-10-11 19:58:58,943][71601] Updated weights for policy 0, policy_version 22460 (0.0008) [2023-10-11 19:59:01,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 45973504. Throughput: 0: 1823.1, 1: 1803.8. Samples: 11497060. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-11 19:59:01,035][70582] Avg episode reward: [(0, '34.980'), (1, '23.170')] [2023-10-11 19:59:01,036][71353] Saving new best policy, reward=34.980! [2023-10-11 19:59:01,961][71635] Updated weights for policy 1, policy_version 22442 (0.0011) [2023-10-11 19:59:02,327][71635] Updated weights for policy 1, policy_version 22452 (0.0008) [2023-10-11 19:59:02,578][71601] Updated weights for policy 0, policy_version 22470 (0.0007) [2023-10-11 19:59:02,695][71635] Updated weights for policy 1, policy_version 22462 (0.0007) [2023-10-11 19:59:02,947][71601] Updated weights for policy 0, policy_version 22480 (0.0007) [2023-10-11 19:59:03,321][71601] Updated weights for policy 0, policy_version 22490 (0.0008) [2023-10-11 19:59:06,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 46039040. Throughput: 0: 1833.0, 1: 1798.5. Samples: 11518864. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-11 19:59:06,035][70582] Avg episode reward: [(0, '35.900'), (1, '22.380')] [2023-10-11 19:59:06,036][71353] Saving new best policy, reward=35.900! [2023-10-11 19:59:06,468][71635] Updated weights for policy 1, policy_version 22472 (0.0008) [2023-10-11 19:59:06,829][71635] Updated weights for policy 1, policy_version 22482 (0.0008) [2023-10-11 19:59:06,889][71601] Updated weights for policy 0, policy_version 22500 (0.0008) [2023-10-11 19:59:07,193][71635] Updated weights for policy 1, policy_version 22492 (0.0007) [2023-10-11 19:59:07,265][71601] Updated weights for policy 0, policy_version 22510 (0.0008) [2023-10-11 19:59:07,626][71601] Updated weights for policy 0, policy_version 22520 (0.0008) [2023-10-11 19:59:10,813][71635] Updated weights for policy 1, policy_version 22502 (0.0009) [2023-10-11 19:59:11,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 46104576. Throughput: 0: 1827.4, 1: 1800.1. Samples: 11541562. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-11 19:59:11,035][70582] Avg episode reward: [(0, '36.320'), (1, '22.920')] [2023-10-11 19:59:11,047][71353] Saving new best policy, reward=36.320! [2023-10-11 19:59:11,173][71635] Updated weights for policy 1, policy_version 22512 (0.0010) [2023-10-11 19:59:11,427][71601] Updated weights for policy 0, policy_version 22530 (0.0007) [2023-10-11 19:59:11,548][71635] Updated weights for policy 1, policy_version 22522 (0.0008) [2023-10-11 19:59:11,795][71601] Updated weights for policy 0, policy_version 22540 (0.0008) [2023-10-11 19:59:12,175][71601] Updated weights for policy 0, policy_version 22550 (0.0008) [2023-10-11 19:59:12,548][71601] Updated weights for policy 0, policy_version 22560 (0.0009) [2023-10-11 19:59:15,216][71635] Updated weights for policy 1, policy_version 22532 (0.0008) [2023-10-11 19:59:15,577][71635] Updated weights for policy 1, policy_version 22542 (0.0010) [2023-10-11 19:59:15,943][71635] Updated weights for policy 1, policy_version 22552 (0.0009) [2023-10-11 19:59:16,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 46170112. Throughput: 0: 1829.7, 1: 1797.8. Samples: 11551708. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-11 19:59:16,034][70582] Avg episode reward: [(0, '30.450'), (1, '21.150')] [2023-10-11 19:59:16,301][71601] Updated weights for policy 0, policy_version 22570 (0.0007) [2023-10-11 19:59:16,670][71601] Updated weights for policy 0, policy_version 22580 (0.0007) [2023-10-11 19:59:17,033][71601] Updated weights for policy 0, policy_version 22590 (0.0007) [2023-10-11 19:59:19,658][71635] Updated weights for policy 1, policy_version 22562 (0.0008) [2023-10-11 19:59:20,037][71635] Updated weights for policy 1, policy_version 22572 (0.0010) [2023-10-11 19:59:20,403][71635] Updated weights for policy 1, policy_version 22582 (0.0009) [2023-10-11 19:59:20,597][71601] Updated weights for policy 0, policy_version 22600 (0.0008) [2023-10-11 19:59:20,762][71635] Updated weights for policy 1, policy_version 22592 (0.0009) [2023-10-11 19:59:20,976][71601] Updated weights for policy 0, policy_version 22610 (0.0007) [2023-10-11 19:59:21,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 46268416. Throughput: 0: 1826.9, 1: 1800.0. Samples: 11574352. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-11 19:59:21,034][70582] Avg episode reward: [(0, '29.640'), (1, '20.500')] [2023-10-11 19:59:21,340][71601] Updated weights for policy 0, policy_version 22620 (0.0008) [2023-10-11 19:59:24,646][71635] Updated weights for policy 1, policy_version 22602 (0.0008) [2023-10-11 19:59:24,936][71601] Updated weights for policy 0, policy_version 22630 (0.0007) [2023-10-11 19:59:25,006][71635] Updated weights for policy 1, policy_version 22612 (0.0009) [2023-10-11 19:59:25,303][71601] Updated weights for policy 0, policy_version 22640 (0.0008) [2023-10-11 19:59:25,367][71635] Updated weights for policy 1, policy_version 22622 (0.0007) [2023-10-11 19:59:25,684][71601] Updated weights for policy 0, policy_version 22650 (0.0007) [2023-10-11 19:59:26,034][70582] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 46366720. Throughput: 0: 1822.6, 1: 1806.1. Samples: 11594866. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-11 19:59:26,034][70582] Avg episode reward: [(0, '27.930'), (1, '22.750')] [2023-10-11 19:59:28,974][71635] Updated weights for policy 1, policy_version 22632 (0.0010) [2023-10-11 19:59:29,336][71635] Updated weights for policy 1, policy_version 22642 (0.0009) [2023-10-11 19:59:29,407][71601] Updated weights for policy 0, policy_version 22660 (0.0010) [2023-10-11 19:59:29,699][71635] Updated weights for policy 1, policy_version 22652 (0.0008) [2023-10-11 19:59:29,780][71601] Updated weights for policy 0, policy_version 22670 (0.0008) [2023-10-11 19:59:30,142][71601] Updated weights for policy 0, policy_version 22680 (0.0011) [2023-10-11 19:59:31,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 46432256. Throughput: 0: 1828.8, 1: 1805.2. Samples: 11607208. Policy #0 lag: (min: 31.0, avg: 43.3, max: 63.0) [2023-10-11 19:59:31,035][70582] Avg episode reward: [(0, '25.140'), (1, '22.440')] [2023-10-11 19:59:33,383][71635] Updated weights for policy 1, policy_version 22662 (0.0009) [2023-10-11 19:59:33,754][71635] Updated weights for policy 1, policy_version 22672 (0.0009) [2023-10-11 19:59:33,924][71601] Updated weights for policy 0, policy_version 22690 (0.0008) [2023-10-11 19:59:34,121][71635] Updated weights for policy 1, policy_version 22682 (0.0008) [2023-10-11 19:59:34,294][71601] Updated weights for policy 0, policy_version 22700 (0.0007) [2023-10-11 19:59:34,662][71601] Updated weights for policy 0, policy_version 22710 (0.0007) [2023-10-11 19:59:35,023][71601] Updated weights for policy 0, policy_version 22720 (0.0008) [2023-10-11 19:59:36,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 46497792. Throughput: 0: 1819.1, 1: 1806.0. Samples: 11627618. Policy #0 lag: (min: 31.0, avg: 43.3, max: 63.0) [2023-10-11 19:59:36,034][70582] Avg episode reward: [(0, '22.970'), (1, '22.350')] [2023-10-11 19:59:37,822][71635] Updated weights for policy 1, policy_version 22692 (0.0008) [2023-10-11 19:59:38,196][71635] Updated weights for policy 1, policy_version 22702 (0.0010) [2023-10-11 19:59:38,557][71601] Updated weights for policy 0, policy_version 22730 (0.0007) [2023-10-11 19:59:38,566][71635] Updated weights for policy 1, policy_version 22712 (0.0010) [2023-10-11 19:59:38,928][71601] Updated weights for policy 0, policy_version 22740 (0.0007) [2023-10-11 19:59:39,294][71601] Updated weights for policy 0, policy_version 22750 (0.0007) [2023-10-11 19:59:41,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 46563328. Throughput: 0: 1826.8, 1: 1810.4. Samples: 11650056. Policy #0 lag: (min: 31.0, avg: 43.3, max: 63.0) [2023-10-11 19:59:41,034][70582] Avg episode reward: [(0, '19.990'), (1, '21.630')] [2023-10-11 19:59:42,255][71635] Updated weights for policy 1, policy_version 22722 (0.0009) [2023-10-11 19:59:42,623][71635] Updated weights for policy 1, policy_version 22732 (0.0008) [2023-10-11 19:59:42,978][71601] Updated weights for policy 0, policy_version 22760 (0.0008) [2023-10-11 19:59:42,979][71635] Updated weights for policy 1, policy_version 22742 (0.0007) [2023-10-11 19:59:43,345][71635] Updated weights for policy 1, policy_version 22752 (0.0008) [2023-10-11 19:59:43,355][71601] Updated weights for policy 0, policy_version 22770 (0.0007) [2023-10-11 19:59:43,728][71601] Updated weights for policy 0, policy_version 22780 (0.0008) [2023-10-11 19:59:46,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 46628864. Throughput: 0: 1821.0, 1: 1815.0. Samples: 11660680. Policy #0 lag: (min: 31.0, avg: 43.3, max: 63.0) [2023-10-11 19:59:46,034][70582] Avg episode reward: [(0, '20.220'), (1, '23.230')] [2023-10-11 19:59:47,133][71635] Updated weights for policy 1, policy_version 22762 (0.0009) [2023-10-11 19:59:47,491][71635] Updated weights for policy 1, policy_version 22772 (0.0008) [2023-10-11 19:59:47,508][71601] Updated weights for policy 0, policy_version 22790 (0.0010) [2023-10-11 19:59:47,858][71635] Updated weights for policy 1, policy_version 22782 (0.0009) [2023-10-11 19:59:47,882][71601] Updated weights for policy 0, policy_version 22800 (0.0007) [2023-10-11 19:59:48,246][71601] Updated weights for policy 0, policy_version 22810 (0.0008) [2023-10-11 19:59:51,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 46694400. Throughput: 0: 1826.0, 1: 1821.6. Samples: 11683004. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:59:51,034][70582] Avg episode reward: [(0, '23.180'), (1, '27.140')] [2023-10-11 19:59:51,035][71431] Saving new best policy, reward=27.140! [2023-10-11 19:59:51,605][71635] Updated weights for policy 1, policy_version 22792 (0.0007) [2023-10-11 19:59:51,966][71635] Updated weights for policy 1, policy_version 22802 (0.0007) [2023-10-11 19:59:52,035][71601] Updated weights for policy 0, policy_version 22820 (0.0008) [2023-10-11 19:59:52,328][71635] Updated weights for policy 1, policy_version 22812 (0.0007) [2023-10-11 19:59:52,405][71601] Updated weights for policy 0, policy_version 22830 (0.0008) [2023-10-11 19:59:52,774][71601] Updated weights for policy 0, policy_version 22840 (0.0008) [2023-10-11 19:59:55,936][71635] Updated weights for policy 1, policy_version 22822 (0.0008) [2023-10-11 19:59:56,034][70582] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 46759936. Throughput: 0: 1824.8, 1: 1826.2. Samples: 11705860. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 19:59:56,035][70582] Avg episode reward: [(0, '23.890'), (1, '26.150')] [2023-10-11 19:59:56,044][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000022848_23396352.pth... [2023-10-11 19:59:56,079][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000021152_21659648.pth [2023-10-11 19:59:56,309][71635] Updated weights for policy 1, policy_version 22832 (0.0007) [2023-10-11 19:59:56,532][71601] Updated weights for policy 0, policy_version 22850 (0.0009) [2023-10-11 19:59:56,664][71635] Updated weights for policy 1, policy_version 22842 (0.0008) [2023-10-11 19:59:56,878][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000022848_23396352.pth... [2023-10-11 19:59:56,905][71601] Updated weights for policy 0, policy_version 22860 (0.0009) [2023-10-11 19:59:56,907][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000021120_21626880.pth [2023-10-11 19:59:57,272][71601] Updated weights for policy 0, policy_version 22870 (0.0008) [2023-10-11 19:59:57,651][71601] Updated weights for policy 0, policy_version 22880 (0.0009) [2023-10-11 20:00:00,470][71635] Updated weights for policy 1, policy_version 22852 (0.0008) [2023-10-11 20:00:00,839][71635] Updated weights for policy 1, policy_version 22862 (0.0008) [2023-10-11 20:00:01,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 46825472. Throughput: 0: 1821.2, 1: 1825.1. Samples: 11715792. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:00:01,034][70582] Avg episode reward: [(0, '27.510'), (1, '24.750')] [2023-10-11 20:00:01,204][71635] Updated weights for policy 1, policy_version 22872 (0.0008) [2023-10-11 20:00:01,415][71601] Updated weights for policy 0, policy_version 22890 (0.0008) [2023-10-11 20:00:01,795][71601] Updated weights for policy 0, policy_version 22900 (0.0008) [2023-10-11 20:00:02,162][71601] Updated weights for policy 0, policy_version 22910 (0.0009) [2023-10-11 20:00:04,951][71635] Updated weights for policy 1, policy_version 22882 (0.0008) [2023-10-11 20:00:05,325][71635] Updated weights for policy 1, policy_version 22892 (0.0008) [2023-10-11 20:00:05,697][71635] Updated weights for policy 1, policy_version 22902 (0.0008) [2023-10-11 20:00:05,814][71601] Updated weights for policy 0, policy_version 22920 (0.0008) [2023-10-11 20:00:06,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 46891008. Throughput: 0: 1817.5, 1: 1820.8. Samples: 11738074. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:00:06,034][70582] Avg episode reward: [(0, '27.490'), (1, '24.130')] [2023-10-11 20:00:06,063][71635] Updated weights for policy 1, policy_version 22912 (0.0008) [2023-10-11 20:00:06,189][71601] Updated weights for policy 0, policy_version 22930 (0.0007) [2023-10-11 20:00:06,555][71601] Updated weights for policy 0, policy_version 22940 (0.0007) [2023-10-11 20:00:09,932][71635] Updated weights for policy 1, policy_version 22922 (0.0009) [2023-10-11 20:00:10,099][71601] Updated weights for policy 0, policy_version 22950 (0.0008) [2023-10-11 20:00:10,290][71635] Updated weights for policy 1, policy_version 22932 (0.0008) [2023-10-11 20:00:10,471][71601] Updated weights for policy 0, policy_version 22960 (0.0007) [2023-10-11 20:00:10,651][71635] Updated weights for policy 1, policy_version 22942 (0.0007) [2023-10-11 20:00:10,848][71601] Updated weights for policy 0, policy_version 22970 (0.0007) [2023-10-11 20:00:11,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 46989312. Throughput: 0: 1822.8, 1: 1821.5. Samples: 11758862. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-11 20:00:11,035][70582] Avg episode reward: [(0, '28.820'), (1, '23.740')] [2023-10-11 20:00:14,250][71635] Updated weights for policy 1, policy_version 22952 (0.0008) [2023-10-11 20:00:14,592][71601] Updated weights for policy 0, policy_version 22980 (0.0008) [2023-10-11 20:00:14,607][71635] Updated weights for policy 1, policy_version 22962 (0.0008) [2023-10-11 20:00:14,957][71601] Updated weights for policy 0, policy_version 22990 (0.0008) [2023-10-11 20:00:14,980][71635] Updated weights for policy 1, policy_version 22972 (0.0007) [2023-10-11 20:00:15,325][71601] Updated weights for policy 0, policy_version 23000 (0.0009) [2023-10-11 20:00:16,034][70582] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 47087616. Throughput: 0: 1814.0, 1: 1814.5. Samples: 11770488. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-11 20:00:16,034][70582] Avg episode reward: [(0, '31.690'), (1, '23.930')] [2023-10-11 20:00:18,674][71635] Updated weights for policy 1, policy_version 22982 (0.0007) [2023-10-11 20:00:19,041][71601] Updated weights for policy 0, policy_version 23010 (0.0010) [2023-10-11 20:00:19,042][71635] Updated weights for policy 1, policy_version 22992 (0.0008) [2023-10-11 20:00:19,402][71601] Updated weights for policy 0, policy_version 23020 (0.0008) [2023-10-11 20:00:19,405][71635] Updated weights for policy 1, policy_version 23002 (0.0008) [2023-10-11 20:00:19,777][71601] Updated weights for policy 0, policy_version 23030 (0.0010) [2023-10-11 20:00:20,148][71601] Updated weights for policy 0, policy_version 23040 (0.0009) [2023-10-11 20:00:21,034][70582] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 47153152. Throughput: 0: 1815.7, 1: 1823.5. Samples: 11791384. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-11 20:00:21,034][70582] Avg episode reward: [(0, '34.370'), (1, '26.720')] [2023-10-11 20:00:23,004][71635] Updated weights for policy 1, policy_version 23012 (0.0009) [2023-10-11 20:00:23,370][71635] Updated weights for policy 1, policy_version 23022 (0.0009) [2023-10-11 20:00:23,746][71635] Updated weights for policy 1, policy_version 23032 (0.0007) [2023-10-11 20:00:23,975][71601] Updated weights for policy 0, policy_version 23050 (0.0008) [2023-10-11 20:00:24,345][71601] Updated weights for policy 0, policy_version 23060 (0.0009) [2023-10-11 20:00:24,713][71601] Updated weights for policy 0, policy_version 23070 (0.0009) [2023-10-11 20:00:26,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 47218688. Throughput: 0: 1805.8, 1: 1810.3. Samples: 11812780. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:00:26,034][70582] Avg episode reward: [(0, '36.540'), (1, '28.040')] [2023-10-11 20:00:26,041][71353] Saving new best policy, reward=36.540! [2023-10-11 20:00:26,041][71431] Saving new best policy, reward=28.040! [2023-10-11 20:00:27,634][71635] Updated weights for policy 1, policy_version 23042 (0.0008) [2023-10-11 20:00:28,011][71635] Updated weights for policy 1, policy_version 23052 (0.0008) [2023-10-11 20:00:28,372][71635] Updated weights for policy 1, policy_version 23062 (0.0007) [2023-10-11 20:00:28,377][71601] Updated weights for policy 0, policy_version 23080 (0.0008) [2023-10-11 20:00:28,743][71635] Updated weights for policy 1, policy_version 23072 (0.0008) [2023-10-11 20:00:28,747][71601] Updated weights for policy 0, policy_version 23090 (0.0007) [2023-10-11 20:00:29,116][71601] Updated weights for policy 0, policy_version 23100 (0.0009) [2023-10-11 20:00:31,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 47284224. Throughput: 0: 1816.6, 1: 1815.7. Samples: 11824136. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:00:31,034][70582] Avg episode reward: [(0, '38.720'), (1, '29.260')] [2023-10-11 20:00:31,035][71431] Saving new best policy, reward=29.260! [2023-10-11 20:00:31,035][71353] Saving new best policy, reward=38.720! [2023-10-11 20:00:32,281][71635] Updated weights for policy 1, policy_version 23082 (0.0008) [2023-10-11 20:00:32,654][71635] Updated weights for policy 1, policy_version 23092 (0.0007) [2023-10-11 20:00:32,836][71601] Updated weights for policy 0, policy_version 23110 (0.0009) [2023-10-11 20:00:33,013][71635] Updated weights for policy 1, policy_version 23102 (0.0007) [2023-10-11 20:00:33,216][71601] Updated weights for policy 0, policy_version 23120 (0.0010) [2023-10-11 20:00:33,595][71601] Updated weights for policy 0, policy_version 23130 (0.0009) [2023-10-11 20:00:36,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 47349760. Throughput: 0: 1803.9, 1: 1806.5. Samples: 11845472. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:00:36,034][70582] Avg episode reward: [(0, '43.720'), (1, '28.300')] [2023-10-11 20:00:36,035][71353] Saving new best policy, reward=43.720! [2023-10-11 20:00:36,646][71635] Updated weights for policy 1, policy_version 23112 (0.0010) [2023-10-11 20:00:37,016][71635] Updated weights for policy 1, policy_version 23122 (0.0008) [2023-10-11 20:00:37,232][71601] Updated weights for policy 0, policy_version 23140 (0.0008) [2023-10-11 20:00:37,384][71635] Updated weights for policy 1, policy_version 23132 (0.0010) [2023-10-11 20:00:37,591][71601] Updated weights for policy 0, policy_version 23150 (0.0007) [2023-10-11 20:00:37,967][71601] Updated weights for policy 0, policy_version 23160 (0.0007) [2023-10-11 20:00:41,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 47415296. Throughput: 0: 1803.9, 1: 1802.6. Samples: 11868150. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:00:41,034][70582] Avg episode reward: [(0, '44.210'), (1, '28.810')] [2023-10-11 20:00:41,040][71353] Saving new best policy, reward=44.210! [2023-10-11 20:00:41,069][71635] Updated weights for policy 1, policy_version 23142 (0.0008) [2023-10-11 20:00:41,438][71635] Updated weights for policy 1, policy_version 23152 (0.0007) [2023-10-11 20:00:41,805][71635] Updated weights for policy 1, policy_version 23162 (0.0007) [2023-10-11 20:00:41,877][71601] Updated weights for policy 0, policy_version 23170 (0.0007) [2023-10-11 20:00:42,252][71601] Updated weights for policy 0, policy_version 23180 (0.0007) [2023-10-11 20:00:42,618][71601] Updated weights for policy 0, policy_version 23190 (0.0007) [2023-10-11 20:00:42,991][71601] Updated weights for policy 0, policy_version 23200 (0.0007) [2023-10-11 20:00:45,559][71635] Updated weights for policy 1, policy_version 23172 (0.0009) [2023-10-11 20:00:45,918][71635] Updated weights for policy 1, policy_version 23182 (0.0009) [2023-10-11 20:00:46,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 47480832. Throughput: 0: 1803.9, 1: 1799.1. Samples: 11877924. Policy #0 lag: (min: 27.0, avg: 33.8, max: 59.0) [2023-10-11 20:00:46,034][70582] Avg episode reward: [(0, '50.340'), (1, '31.820')] [2023-10-11 20:00:46,035][71353] Saving new best policy, reward=50.340! [2023-10-11 20:00:46,284][71635] Updated weights for policy 1, policy_version 23192 (0.0009) [2023-10-11 20:00:46,578][71431] Saving new best policy, reward=31.820! [2023-10-11 20:00:46,747][71601] Updated weights for policy 0, policy_version 23210 (0.0007) [2023-10-11 20:00:47,122][71601] Updated weights for policy 0, policy_version 23220 (0.0009) [2023-10-11 20:00:47,493][71601] Updated weights for policy 0, policy_version 23230 (0.0009) [2023-10-11 20:00:49,931][71635] Updated weights for policy 1, policy_version 23202 (0.0009) [2023-10-11 20:00:50,297][71635] Updated weights for policy 1, policy_version 23212 (0.0008) [2023-10-11 20:00:50,666][71635] Updated weights for policy 1, policy_version 23222 (0.0008) [2023-10-11 20:00:51,026][71635] Updated weights for policy 1, policy_version 23232 (0.0008) [2023-10-11 20:00:51,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 47579136. Throughput: 0: 1806.5, 1: 1809.2. Samples: 11900784. Policy #0 lag: (min: 27.0, avg: 33.8, max: 59.0) [2023-10-11 20:00:51,035][70582] Avg episode reward: [(0, '53.190'), (1, '32.470')] [2023-10-11 20:00:51,036][71431] Saving new best policy, reward=32.470! [2023-10-11 20:00:51,091][71601] Updated weights for policy 0, policy_version 23240 (0.0008) [2023-10-11 20:00:51,461][71601] Updated weights for policy 0, policy_version 23250 (0.0008) [2023-10-11 20:00:51,834][71601] Updated weights for policy 0, policy_version 23260 (0.0007) [2023-10-11 20:00:51,983][71353] Saving new best policy, reward=53.190! [2023-10-11 20:00:54,793][71635] Updated weights for policy 1, policy_version 23242 (0.0010) [2023-10-11 20:00:55,161][71635] Updated weights for policy 1, policy_version 23252 (0.0008) [2023-10-11 20:00:55,515][71635] Updated weights for policy 1, policy_version 23262 (0.0008) [2023-10-11 20:00:55,545][71601] Updated weights for policy 0, policy_version 23270 (0.0010) [2023-10-11 20:00:55,919][71601] Updated weights for policy 0, policy_version 23280 (0.0009) [2023-10-11 20:00:56,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 47644672. Throughput: 0: 1819.8, 1: 1813.6. Samples: 11922364. Policy #0 lag: (min: 27.0, avg: 33.8, max: 59.0) [2023-10-11 20:00:56,034][70582] Avg episode reward: [(0, '52.880'), (1, '28.440')] [2023-10-11 20:00:56,282][71601] Updated weights for policy 0, policy_version 23290 (0.0008) [2023-10-11 20:00:59,184][71635] Updated weights for policy 1, policy_version 23272 (0.0009) [2023-10-11 20:00:59,541][71635] Updated weights for policy 1, policy_version 23282 (0.0008) [2023-10-11 20:00:59,821][71601] Updated weights for policy 0, policy_version 23300 (0.0008) [2023-10-11 20:00:59,905][71635] Updated weights for policy 1, policy_version 23292 (0.0008) [2023-10-11 20:01:00,196][71601] Updated weights for policy 0, policy_version 23310 (0.0008) [2023-10-11 20:01:00,568][71601] Updated weights for policy 0, policy_version 23320 (0.0008) [2023-10-11 20:01:01,034][70582] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 47742976. Throughput: 0: 1814.5, 1: 1819.8. Samples: 11934032. Policy #0 lag: (min: 6.0, avg: 6.2, max: 16.0) [2023-10-11 20:01:01,035][70582] Avg episode reward: [(0, '57.830'), (1, '29.760')] [2023-10-11 20:01:01,036][71353] Saving new best policy, reward=57.830! [2023-10-11 20:01:03,758][71635] Updated weights for policy 1, policy_version 23302 (0.0009) [2023-10-11 20:01:04,127][71635] Updated weights for policy 1, policy_version 23312 (0.0010) [2023-10-11 20:01:04,195][71601] Updated weights for policy 0, policy_version 23330 (0.0008) [2023-10-11 20:01:04,495][71635] Updated weights for policy 1, policy_version 23322 (0.0007) [2023-10-11 20:01:04,567][71601] Updated weights for policy 0, policy_version 23340 (0.0007) [2023-10-11 20:01:04,927][71601] Updated weights for policy 0, policy_version 23350 (0.0007) [2023-10-11 20:01:05,299][71601] Updated weights for policy 0, policy_version 23360 (0.0007) [2023-10-11 20:01:06,034][70582] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 47808512. Throughput: 0: 1828.9, 1: 1813.8. Samples: 11955306. Policy #0 lag: (min: 6.0, avg: 6.2, max: 16.0) [2023-10-11 20:01:06,034][70582] Avg episode reward: [(0, '58.080'), (1, '29.690')] [2023-10-11 20:01:06,035][71353] Saving new best policy, reward=58.080! [2023-10-11 20:01:08,176][71635] Updated weights for policy 1, policy_version 23332 (0.0007) [2023-10-11 20:01:08,531][71635] Updated weights for policy 1, policy_version 23342 (0.0008) [2023-10-11 20:01:08,910][71635] Updated weights for policy 1, policy_version 23352 (0.0009) [2023-10-11 20:01:08,981][71601] Updated weights for policy 0, policy_version 23370 (0.0008) [2023-10-11 20:01:09,355][71601] Updated weights for policy 0, policy_version 23380 (0.0007) [2023-10-11 20:01:09,732][71601] Updated weights for policy 0, policy_version 23390 (0.0010) [2023-10-11 20:01:11,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 47874048. Throughput: 0: 1821.8, 1: 1814.3. Samples: 11976404. Policy #0 lag: (min: 6.0, avg: 6.2, max: 16.0) [2023-10-11 20:01:11,034][70582] Avg episode reward: [(0, '60.180'), (1, '30.740')] [2023-10-11 20:01:11,043][71353] Saving new best policy, reward=60.180! [2023-10-11 20:01:12,682][71635] Updated weights for policy 1, policy_version 23362 (0.0009) [2023-10-11 20:01:13,047][71635] Updated weights for policy 1, policy_version 23372 (0.0007) [2023-10-11 20:01:13,348][71601] Updated weights for policy 0, policy_version 23400 (0.0007) [2023-10-11 20:01:13,409][71635] Updated weights for policy 1, policy_version 23382 (0.0007) [2023-10-11 20:01:13,730][71601] Updated weights for policy 0, policy_version 23410 (0.0009) [2023-10-11 20:01:13,771][71635] Updated weights for policy 1, policy_version 23392 (0.0007) [2023-10-11 20:01:14,098][71601] Updated weights for policy 0, policy_version 23420 (0.0010) [2023-10-11 20:01:16,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 47939584. Throughput: 0: 1825.0, 1: 1817.2. Samples: 11988034. Policy #0 lag: (min: 6.0, avg: 6.2, max: 16.0) [2023-10-11 20:01:16,034][70582] Avg episode reward: [(0, '67.260'), (1, '30.050')] [2023-10-11 20:01:16,035][71353] Saving new best policy, reward=67.260! [2023-10-11 20:01:17,409][71635] Updated weights for policy 1, policy_version 23402 (0.0008) [2023-10-11 20:01:17,766][71635] Updated weights for policy 1, policy_version 23412 (0.0009) [2023-10-11 20:01:17,871][71601] Updated weights for policy 0, policy_version 23430 (0.0008) [2023-10-11 20:01:18,136][71635] Updated weights for policy 1, policy_version 23422 (0.0007) [2023-10-11 20:01:18,238][71601] Updated weights for policy 0, policy_version 23440 (0.0008) [2023-10-11 20:01:18,618][71601] Updated weights for policy 0, policy_version 23450 (0.0010) [2023-10-11 20:01:21,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 48005120. Throughput: 0: 1823.8, 1: 1815.8. Samples: 12009252. Policy #0 lag: (min: 20.0, avg: 26.9, max: 52.0) [2023-10-11 20:01:21,035][70582] Avg episode reward: [(0, '61.950'), (1, '31.350')] [2023-10-11 20:01:21,950][71635] Updated weights for policy 1, policy_version 23432 (0.0007) [2023-10-11 20:01:22,240][71601] Updated weights for policy 0, policy_version 23460 (0.0009) [2023-10-11 20:01:22,317][71635] Updated weights for policy 1, policy_version 23442 (0.0008) [2023-10-11 20:01:22,613][71601] Updated weights for policy 0, policy_version 23470 (0.0008) [2023-10-11 20:01:22,694][71635] Updated weights for policy 1, policy_version 23452 (0.0008) [2023-10-11 20:01:22,977][71601] Updated weights for policy 0, policy_version 23480 (0.0007) [2023-10-11 20:01:26,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 48070656. Throughput: 0: 1826.3, 1: 1818.7. Samples: 12032176. Policy #0 lag: (min: 20.0, avg: 26.9, max: 52.0) [2023-10-11 20:01:26,035][70582] Avg episode reward: [(0, '69.960'), (1, '31.020')] [2023-10-11 20:01:26,044][71353] Saving new best policy, reward=69.960! [2023-10-11 20:01:26,300][71635] Updated weights for policy 1, policy_version 23462 (0.0007) [2023-10-11 20:01:26,668][71635] Updated weights for policy 1, policy_version 23472 (0.0009) [2023-10-11 20:01:26,746][71601] Updated weights for policy 0, policy_version 23490 (0.0008) [2023-10-11 20:01:27,027][71635] Updated weights for policy 1, policy_version 23482 (0.0008) [2023-10-11 20:01:27,112][71601] Updated weights for policy 0, policy_version 23500 (0.0008) [2023-10-11 20:01:27,486][71601] Updated weights for policy 0, policy_version 23510 (0.0008) [2023-10-11 20:01:27,858][71601] Updated weights for policy 0, policy_version 23520 (0.0011) [2023-10-11 20:01:30,832][71635] Updated weights for policy 1, policy_version 23492 (0.0007) [2023-10-11 20:01:31,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 48136192. Throughput: 0: 1827.4, 1: 1822.4. Samples: 12042164. Policy #0 lag: (min: 20.0, avg: 26.9, max: 52.0) [2023-10-11 20:01:31,034][70582] Avg episode reward: [(0, '67.900'), (1, '28.980')] [2023-10-11 20:01:31,203][71635] Updated weights for policy 1, policy_version 23502 (0.0008) [2023-10-11 20:01:31,572][71635] Updated weights for policy 1, policy_version 23512 (0.0007) [2023-10-11 20:01:31,634][71601] Updated weights for policy 0, policy_version 23530 (0.0007) [2023-10-11 20:01:32,005][71601] Updated weights for policy 0, policy_version 23540 (0.0007) [2023-10-11 20:01:32,374][71601] Updated weights for policy 0, policy_version 23550 (0.0009) [2023-10-11 20:01:35,316][71635] Updated weights for policy 1, policy_version 23522 (0.0007) [2023-10-11 20:01:35,690][71635] Updated weights for policy 1, policy_version 23532 (0.0008) [2023-10-11 20:01:35,969][71601] Updated weights for policy 0, policy_version 23560 (0.0008) [2023-10-11 20:01:36,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 48201728. Throughput: 0: 1821.3, 1: 1818.1. Samples: 12064558. Policy #0 lag: (min: 20.0, avg: 26.9, max: 52.0) [2023-10-11 20:01:36,034][70582] Avg episode reward: [(0, '71.050'), (1, '30.170')] [2023-10-11 20:01:36,059][71635] Updated weights for policy 1, policy_version 23542 (0.0008) [2023-10-11 20:01:36,331][71601] Updated weights for policy 0, policy_version 23570 (0.0007) [2023-10-11 20:01:36,426][71635] Updated weights for policy 1, policy_version 23552 (0.0007) [2023-10-11 20:01:36,702][71601] Updated weights for policy 0, policy_version 23580 (0.0008) [2023-10-11 20:01:36,848][71353] Saving new best policy, reward=71.050! [2023-10-11 20:01:40,052][71635] Updated weights for policy 1, policy_version 23562 (0.0008) [2023-10-11 20:01:40,424][71635] Updated weights for policy 1, policy_version 23572 (0.0009) [2023-10-11 20:01:40,472][71601] Updated weights for policy 0, policy_version 23590 (0.0007) [2023-10-11 20:01:40,790][71635] Updated weights for policy 1, policy_version 23582 (0.0009) [2023-10-11 20:01:40,849][71601] Updated weights for policy 0, policy_version 23600 (0.0008) [2023-10-11 20:01:41,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 48300032. Throughput: 0: 1813.2, 1: 1829.4. Samples: 12086280. Policy #0 lag: (min: 26.0, avg: 34.2, max: 58.0) [2023-10-11 20:01:41,034][70582] Avg episode reward: [(0, '68.170'), (1, '30.860')] [2023-10-11 20:01:41,224][71601] Updated weights for policy 0, policy_version 23610 (0.0009) [2023-10-11 20:01:44,517][71635] Updated weights for policy 1, policy_version 23592 (0.0007) [2023-10-11 20:01:44,894][71635] Updated weights for policy 1, policy_version 23602 (0.0008) [2023-10-11 20:01:44,911][71601] Updated weights for policy 0, policy_version 23620 (0.0009) [2023-10-11 20:01:45,259][71635] Updated weights for policy 1, policy_version 23612 (0.0008) [2023-10-11 20:01:45,279][71601] Updated weights for policy 0, policy_version 23630 (0.0007) [2023-10-11 20:01:45,657][71601] Updated weights for policy 0, policy_version 23640 (0.0009) [2023-10-11 20:01:46,034][70582] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 48398336. Throughput: 0: 1808.6, 1: 1818.7. Samples: 12097262. Policy #0 lag: (min: 26.0, avg: 34.2, max: 58.0) [2023-10-11 20:01:46,035][70582] Avg episode reward: [(0, '66.500'), (1, '31.910')] [2023-10-11 20:01:48,907][71635] Updated weights for policy 1, policy_version 23622 (0.0010) [2023-10-11 20:01:49,271][71635] Updated weights for policy 1, policy_version 23632 (0.0009) [2023-10-11 20:01:49,380][71601] Updated weights for policy 0, policy_version 23650 (0.0008) [2023-10-11 20:01:49,639][71635] Updated weights for policy 1, policy_version 23642 (0.0009) [2023-10-11 20:01:49,757][71601] Updated weights for policy 0, policy_version 23660 (0.0007) [2023-10-11 20:01:50,132][71601] Updated weights for policy 0, policy_version 23670 (0.0007) [2023-10-11 20:01:50,498][71601] Updated weights for policy 0, policy_version 23680 (0.0007) [2023-10-11 20:01:51,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 48463872. Throughput: 0: 1809.9, 1: 1825.2. Samples: 12118882. Policy #0 lag: (min: 26.0, avg: 34.2, max: 58.0) [2023-10-11 20:01:51,035][70582] Avg episode reward: [(0, '69.200'), (1, '34.690')] [2023-10-11 20:01:51,035][71431] Saving new best policy, reward=34.690! [2023-10-11 20:01:53,285][71635] Updated weights for policy 1, policy_version 23652 (0.0007) [2023-10-11 20:01:53,653][71635] Updated weights for policy 1, policy_version 23662 (0.0007) [2023-10-11 20:01:54,019][71635] Updated weights for policy 1, policy_version 23672 (0.0008) [2023-10-11 20:01:54,165][71601] Updated weights for policy 0, policy_version 23690 (0.0009) [2023-10-11 20:01:54,534][71601] Updated weights for policy 0, policy_version 23700 (0.0007) [2023-10-11 20:01:54,921][71601] Updated weights for policy 0, policy_version 23710 (0.0009) [2023-10-11 20:01:56,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 48529408. Throughput: 0: 1807.0, 1: 1824.4. Samples: 12139814. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:01:56,034][70582] Avg episode reward: [(0, '66.880'), (1, '36.930')] [2023-10-11 20:01:56,045][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000023712_24281088.pth... [2023-10-11 20:01:56,045][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000023680_24248320.pth... [2023-10-11 20:01:56,075][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000022016_22544384.pth [2023-10-11 20:01:56,084][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000021984_22511616.pth [2023-10-11 20:01:56,088][71431] Saving new best policy, reward=36.930! [2023-10-11 20:01:57,802][71635] Updated weights for policy 1, policy_version 23682 (0.0008) [2023-10-11 20:01:58,160][71635] Updated weights for policy 1, policy_version 23692 (0.0008) [2023-10-11 20:01:58,530][71635] Updated weights for policy 1, policy_version 23702 (0.0008) [2023-10-11 20:01:58,556][71601] Updated weights for policy 0, policy_version 23720 (0.0008) [2023-10-11 20:01:58,891][71635] Updated weights for policy 1, policy_version 23712 (0.0009) [2023-10-11 20:01:58,933][71601] Updated weights for policy 0, policy_version 23730 (0.0009) [2023-10-11 20:01:59,310][71601] Updated weights for policy 0, policy_version 23740 (0.0008) [2023-10-11 20:02:01,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 48594944. Throughput: 0: 1806.8, 1: 1825.1. Samples: 12151472. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:02:01,035][70582] Avg episode reward: [(0, '67.280'), (1, '35.480')] [2023-10-11 20:02:02,608][71635] Updated weights for policy 1, policy_version 23722 (0.0011) [2023-10-11 20:02:02,971][71635] Updated weights for policy 1, policy_version 23732 (0.0008) [2023-10-11 20:02:03,012][71601] Updated weights for policy 0, policy_version 23750 (0.0008) [2023-10-11 20:02:03,340][71635] Updated weights for policy 1, policy_version 23742 (0.0009) [2023-10-11 20:02:03,381][71601] Updated weights for policy 0, policy_version 23760 (0.0007) [2023-10-11 20:02:03,750][71601] Updated weights for policy 0, policy_version 23770 (0.0008) [2023-10-11 20:02:06,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 48660480. Throughput: 0: 1805.4, 1: 1819.0. Samples: 12172352. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:02:06,035][70582] Avg episode reward: [(0, '64.880'), (1, '34.510')] [2023-10-11 20:02:07,055][71635] Updated weights for policy 1, policy_version 23752 (0.0008) [2023-10-11 20:02:07,363][71601] Updated weights for policy 0, policy_version 23780 (0.0008) [2023-10-11 20:02:07,416][71635] Updated weights for policy 1, policy_version 23762 (0.0008) [2023-10-11 20:02:07,729][71601] Updated weights for policy 0, policy_version 23790 (0.0008) [2023-10-11 20:02:07,778][71635] Updated weights for policy 1, policy_version 23772 (0.0008) [2023-10-11 20:02:08,113][71601] Updated weights for policy 0, policy_version 23800 (0.0010) [2023-10-11 20:02:11,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 48726016. Throughput: 0: 1803.6, 1: 1814.8. Samples: 12195002. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:02:11,035][70582] Avg episode reward: [(0, '64.880'), (1, '37.000')] [2023-10-11 20:02:11,047][71431] Saving new best policy, reward=37.000! [2023-10-11 20:02:11,463][71635] Updated weights for policy 1, policy_version 23782 (0.0011) [2023-10-11 20:02:11,830][71635] Updated weights for policy 1, policy_version 23792 (0.0010) [2023-10-11 20:02:12,011][71601] Updated weights for policy 0, policy_version 23810 (0.0010) [2023-10-11 20:02:12,193][71635] Updated weights for policy 1, policy_version 23802 (0.0007) [2023-10-11 20:02:12,384][71601] Updated weights for policy 0, policy_version 23820 (0.0008) [2023-10-11 20:02:12,747][71601] Updated weights for policy 0, policy_version 23830 (0.0009) [2023-10-11 20:02:13,125][71601] Updated weights for policy 0, policy_version 23840 (0.0008) [2023-10-11 20:02:15,886][71635] Updated weights for policy 1, policy_version 23812 (0.0008) [2023-10-11 20:02:16,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 48791552. Throughput: 0: 1801.6, 1: 1811.7. Samples: 12204764. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) [2023-10-11 20:02:16,034][70582] Avg episode reward: [(0, '66.030'), (1, '36.290')] [2023-10-11 20:02:16,255][71635] Updated weights for policy 1, policy_version 23822 (0.0009) [2023-10-11 20:02:16,622][71635] Updated weights for policy 1, policy_version 23832 (0.0010) [2023-10-11 20:02:17,270][71601] Updated weights for policy 0, policy_version 23850 (0.0009) [2023-10-11 20:02:17,642][71601] Updated weights for policy 0, policy_version 23860 (0.0008) [2023-10-11 20:02:18,018][71601] Updated weights for policy 0, policy_version 23870 (0.0009) [2023-10-11 20:02:20,158][71635] Updated weights for policy 1, policy_version 23842 (0.0008) [2023-10-11 20:02:20,525][71635] Updated weights for policy 1, policy_version 23852 (0.0007) [2023-10-11 20:02:20,904][71635] Updated weights for policy 1, policy_version 23862 (0.0008) [2023-10-11 20:02:21,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 48857088. Throughput: 0: 1801.7, 1: 1811.0. Samples: 12227128. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) [2023-10-11 20:02:21,034][70582] Avg episode reward: [(0, '69.340'), (1, '38.960')] [2023-10-11 20:02:21,258][71431] Saving new best policy, reward=38.960! [2023-10-11 20:02:21,264][71635] Updated weights for policy 1, policy_version 23872 (0.0007) [2023-10-11 20:02:21,632][71601] Updated weights for policy 0, policy_version 23880 (0.0008) [2023-10-11 20:02:22,006][71601] Updated weights for policy 0, policy_version 23890 (0.0008) [2023-10-11 20:02:22,386][71601] Updated weights for policy 0, policy_version 23900 (0.0007) [2023-10-11 20:02:24,982][71635] Updated weights for policy 1, policy_version 23882 (0.0010) [2023-10-11 20:02:25,351][71635] Updated weights for policy 1, policy_version 23892 (0.0010) [2023-10-11 20:02:25,721][71635] Updated weights for policy 1, policy_version 23902 (0.0008) [2023-10-11 20:02:25,956][71601] Updated weights for policy 0, policy_version 23910 (0.0007) [2023-10-11 20:02:26,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 48955392. Throughput: 0: 1808.6, 1: 1813.2. Samples: 12249262. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) [2023-10-11 20:02:26,035][70582] Avg episode reward: [(0, '68.590'), (1, '38.620')] [2023-10-11 20:02:26,322][71601] Updated weights for policy 0, policy_version 23920 (0.0009) [2023-10-11 20:02:26,690][71601] Updated weights for policy 0, policy_version 23930 (0.0011) [2023-10-11 20:02:29,406][71635] Updated weights for policy 1, policy_version 23912 (0.0010) [2023-10-11 20:02:29,780][71635] Updated weights for policy 1, policy_version 23922 (0.0008) [2023-10-11 20:02:30,150][71635] Updated weights for policy 1, policy_version 23932 (0.0009) [2023-10-11 20:02:30,339][71601] Updated weights for policy 0, policy_version 23940 (0.0008) [2023-10-11 20:02:30,706][71601] Updated weights for policy 0, policy_version 23950 (0.0009) [2023-10-11 20:02:31,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 49020928. Throughput: 0: 1805.8, 1: 1814.6. Samples: 12260180. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) [2023-10-11 20:02:31,035][70582] Avg episode reward: [(0, '68.960'), (1, '38.380')] [2023-10-11 20:02:31,073][71601] Updated weights for policy 0, policy_version 23960 (0.0008) [2023-10-11 20:02:33,894][71635] Updated weights for policy 1, policy_version 23942 (0.0008) [2023-10-11 20:02:34,258][71635] Updated weights for policy 1, policy_version 23952 (0.0010) [2023-10-11 20:02:34,627][71635] Updated weights for policy 1, policy_version 23962 (0.0008) [2023-10-11 20:02:34,868][71601] Updated weights for policy 0, policy_version 23970 (0.0007) [2023-10-11 20:02:35,234][71601] Updated weights for policy 0, policy_version 23980 (0.0007) [2023-10-11 20:02:35,613][71601] Updated weights for policy 0, policy_version 23990 (0.0007) [2023-10-11 20:02:35,984][71601] Updated weights for policy 0, policy_version 24000 (0.0008) [2023-10-11 20:02:36,034][70582] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 49119232. Throughput: 0: 1804.6, 1: 1816.7. Samples: 12281840. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-11 20:02:36,035][70582] Avg episode reward: [(0, '77.880'), (1, '38.730')] [2023-10-11 20:02:36,036][71353] Saving new best policy, reward=77.880! [2023-10-11 20:02:38,197][71635] Updated weights for policy 1, policy_version 23972 (0.0007) [2023-10-11 20:02:38,569][71635] Updated weights for policy 1, policy_version 23982 (0.0008) [2023-10-11 20:02:38,945][71635] Updated weights for policy 1, policy_version 23992 (0.0008) [2023-10-11 20:02:39,711][71601] Updated weights for policy 0, policy_version 24010 (0.0010) [2023-10-11 20:02:40,089][71601] Updated weights for policy 0, policy_version 24020 (0.0009) [2023-10-11 20:02:40,459][71601] Updated weights for policy 0, policy_version 24030 (0.0009) [2023-10-11 20:02:41,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 49184768. Throughput: 0: 1805.4, 1: 1814.5. Samples: 12302710. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-11 20:02:41,035][70582] Avg episode reward: [(0, '80.640'), (1, '44.240')] [2023-10-11 20:02:41,046][71353] Saving new best policy, reward=80.640! [2023-10-11 20:02:41,046][71431] Saving new best policy, reward=44.240! [2023-10-11 20:02:42,740][71635] Updated weights for policy 1, policy_version 24002 (0.0008) [2023-10-11 20:02:43,107][71635] Updated weights for policy 1, policy_version 24012 (0.0007) [2023-10-11 20:02:43,484][71635] Updated weights for policy 1, policy_version 24022 (0.0007) [2023-10-11 20:02:43,852][71635] Updated weights for policy 1, policy_version 24032 (0.0008) [2023-10-11 20:02:43,950][71601] Updated weights for policy 0, policy_version 24040 (0.0008) [2023-10-11 20:02:44,322][71601] Updated weights for policy 0, policy_version 24050 (0.0007) [2023-10-11 20:02:44,697][71601] Updated weights for policy 0, policy_version 24060 (0.0008) [2023-10-11 20:02:46,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 49250304. Throughput: 0: 1814.3, 1: 1816.3. Samples: 12314850. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-11 20:02:46,035][70582] Avg episode reward: [(0, '74.470'), (1, '43.270')] [2023-10-11 20:02:47,564][71635] Updated weights for policy 1, policy_version 24042 (0.0010) [2023-10-11 20:02:47,947][71635] Updated weights for policy 1, policy_version 24052 (0.0011) [2023-10-11 20:02:48,307][71635] Updated weights for policy 1, policy_version 24062 (0.0008) [2023-10-11 20:02:48,320][71601] Updated weights for policy 0, policy_version 24070 (0.0008) [2023-10-11 20:02:48,679][71601] Updated weights for policy 0, policy_version 24080 (0.0007) [2023-10-11 20:02:49,054][71601] Updated weights for policy 0, policy_version 24090 (0.0007) [2023-10-11 20:02:51,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 49315840. Throughput: 0: 1809.7, 1: 1816.3. Samples: 12335522. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:02:51,035][70582] Avg episode reward: [(0, '74.590'), (1, '48.830')] [2023-10-11 20:02:51,036][71431] Saving new best policy, reward=48.830! [2023-10-11 20:02:52,021][71635] Updated weights for policy 1, policy_version 24072 (0.0008) [2023-10-11 20:02:52,383][71635] Updated weights for policy 1, policy_version 24082 (0.0009) [2023-10-11 20:02:52,754][71635] Updated weights for policy 1, policy_version 24092 (0.0008) [2023-10-11 20:02:52,765][71601] Updated weights for policy 0, policy_version 24100 (0.0009) [2023-10-11 20:02:53,129][71601] Updated weights for policy 0, policy_version 24110 (0.0009) [2023-10-11 20:02:53,499][71601] Updated weights for policy 0, policy_version 24120 (0.0007) [2023-10-11 20:02:56,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 49381376. Throughput: 0: 1809.3, 1: 1817.4. Samples: 12358204. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:02:56,034][70582] Avg episode reward: [(0, '73.920'), (1, '50.650')] [2023-10-11 20:02:56,053][71431] Saving new best policy, reward=50.650! [2023-10-11 20:02:56,549][71635] Updated weights for policy 1, policy_version 24102 (0.0008) [2023-10-11 20:02:56,913][71635] Updated weights for policy 1, policy_version 24112 (0.0009) [2023-10-11 20:02:57,140][71601] Updated weights for policy 0, policy_version 24130 (0.0008) [2023-10-11 20:02:57,276][71635] Updated weights for policy 1, policy_version 24122 (0.0007) [2023-10-11 20:02:57,505][71601] Updated weights for policy 0, policy_version 24140 (0.0007) [2023-10-11 20:02:57,875][71601] Updated weights for policy 0, policy_version 24150 (0.0007) [2023-10-11 20:02:58,246][71601] Updated weights for policy 0, policy_version 24160 (0.0011) [2023-10-11 20:03:00,938][71635] Updated weights for policy 1, policy_version 24132 (0.0009) [2023-10-11 20:03:01,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 49446912. Throughput: 0: 1813.6, 1: 1817.5. Samples: 12368164. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:03:01,035][70582] Avg episode reward: [(0, '76.960'), (1, '48.940')] [2023-10-11 20:03:01,302][71635] Updated weights for policy 1, policy_version 24142 (0.0010) [2023-10-11 20:03:01,667][71635] Updated weights for policy 1, policy_version 24152 (0.0007) [2023-10-11 20:03:01,962][71601] Updated weights for policy 0, policy_version 24170 (0.0009) [2023-10-11 20:03:02,326][71601] Updated weights for policy 0, policy_version 24180 (0.0010) [2023-10-11 20:03:02,708][71601] Updated weights for policy 0, policy_version 24190 (0.0010) [2023-10-11 20:03:05,514][71635] Updated weights for policy 1, policy_version 24162 (0.0008) [2023-10-11 20:03:05,883][71635] Updated weights for policy 1, policy_version 24172 (0.0008) [2023-10-11 20:03:06,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 49512448. Throughput: 0: 1822.2, 1: 1815.9. Samples: 12390846. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:03:06,035][70582] Avg episode reward: [(0, '80.920'), (1, '45.720')] [2023-10-11 20:03:06,036][71353] Saving new best policy, reward=80.920! [2023-10-11 20:03:06,248][71635] Updated weights for policy 1, policy_version 24182 (0.0010) [2023-10-11 20:03:06,568][71601] Updated weights for policy 0, policy_version 24200 (0.0007) [2023-10-11 20:03:06,610][71635] Updated weights for policy 1, policy_version 24192 (0.0010) [2023-10-11 20:03:06,947][71601] Updated weights for policy 0, policy_version 24210 (0.0007) [2023-10-11 20:03:07,315][71601] Updated weights for policy 0, policy_version 24220 (0.0008) [2023-10-11 20:03:10,239][71635] Updated weights for policy 1, policy_version 24202 (0.0011) [2023-10-11 20:03:10,603][71635] Updated weights for policy 1, policy_version 24212 (0.0010) [2023-10-11 20:03:10,938][71601] Updated weights for policy 0, policy_version 24230 (0.0008) [2023-10-11 20:03:10,975][71635] Updated weights for policy 1, policy_version 24222 (0.0007) [2023-10-11 20:03:11,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 49577984. Throughput: 0: 1823.2, 1: 1819.7. Samples: 12413188. Policy #0 lag: (min: 0.0, avg: 19.5, max: 32.0) [2023-10-11 20:03:11,034][70582] Avg episode reward: [(0, '89.330'), (1, '49.590')] [2023-10-11 20:03:11,315][71601] Updated weights for policy 0, policy_version 24240 (0.0007) [2023-10-11 20:03:11,691][71601] Updated weights for policy 0, policy_version 24250 (0.0007) [2023-10-11 20:03:11,912][71353] Saving new best policy, reward=89.330! [2023-10-11 20:03:14,743][71635] Updated weights for policy 1, policy_version 24232 (0.0007) [2023-10-11 20:03:15,119][71635] Updated weights for policy 1, policy_version 24242 (0.0010) [2023-10-11 20:03:15,388][71601] Updated weights for policy 0, policy_version 24260 (0.0008) [2023-10-11 20:03:15,487][71635] Updated weights for policy 1, policy_version 24252 (0.0008) [2023-10-11 20:03:15,757][71601] Updated weights for policy 0, policy_version 24270 (0.0007) [2023-10-11 20:03:16,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 49676288. Throughput: 0: 1820.8, 1: 1815.5. Samples: 12423814. Policy #0 lag: (min: 0.0, avg: 19.5, max: 32.0) [2023-10-11 20:03:16,035][70582] Avg episode reward: [(0, '89.890'), (1, '47.050')] [2023-10-11 20:03:16,131][71601] Updated weights for policy 0, policy_version 24280 (0.0008) [2023-10-11 20:03:16,417][71353] Saving new best policy, reward=89.890! [2023-10-11 20:03:19,152][71635] Updated weights for policy 1, policy_version 24262 (0.0009) [2023-10-11 20:03:19,513][71635] Updated weights for policy 1, policy_version 24272 (0.0008) [2023-10-11 20:03:19,777][71601] Updated weights for policy 0, policy_version 24290 (0.0008) [2023-10-11 20:03:19,882][71635] Updated weights for policy 1, policy_version 24282 (0.0007) [2023-10-11 20:03:20,153][71601] Updated weights for policy 0, policy_version 24300 (0.0008) [2023-10-11 20:03:20,524][71601] Updated weights for policy 0, policy_version 24310 (0.0011) [2023-10-11 20:03:20,907][71601] Updated weights for policy 0, policy_version 24320 (0.0009) [2023-10-11 20:03:21,034][70582] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 49774592. Throughput: 0: 1821.8, 1: 1820.1. Samples: 12445724. Policy #0 lag: (min: 0.0, avg: 19.5, max: 32.0) [2023-10-11 20:03:21,034][70582] Avg episode reward: [(0, '91.000'), (1, '46.490')] [2023-10-11 20:03:21,035][71353] Saving new best policy, reward=91.000! [2023-10-11 20:03:23,645][71635] Updated weights for policy 1, policy_version 24292 (0.0010) [2023-10-11 20:03:24,001][71635] Updated weights for policy 1, policy_version 24302 (0.0009) [2023-10-11 20:03:24,367][71635] Updated weights for policy 1, policy_version 24312 (0.0008) [2023-10-11 20:03:24,384][71601] Updated weights for policy 0, policy_version 24330 (0.0007) [2023-10-11 20:03:24,746][71601] Updated weights for policy 0, policy_version 24340 (0.0007) [2023-10-11 20:03:25,120][71601] Updated weights for policy 0, policy_version 24350 (0.0010) [2023-10-11 20:03:26,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 49840128. Throughput: 0: 1823.1, 1: 1809.1. Samples: 12466158. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-11 20:03:26,034][70582] Avg episode reward: [(0, '94.740'), (1, '44.540')] [2023-10-11 20:03:26,045][71353] Saving new best policy, reward=94.740! [2023-10-11 20:03:28,063][71635] Updated weights for policy 1, policy_version 24322 (0.0007) [2023-10-11 20:03:28,429][71635] Updated weights for policy 1, policy_version 24332 (0.0007) [2023-10-11 20:03:28,768][71601] Updated weights for policy 0, policy_version 24360 (0.0008) [2023-10-11 20:03:28,787][71635] Updated weights for policy 1, policy_version 24342 (0.0007) [2023-10-11 20:03:29,137][71601] Updated weights for policy 0, policy_version 24370 (0.0008) [2023-10-11 20:03:29,165][71635] Updated weights for policy 1, policy_version 24352 (0.0009) [2023-10-11 20:03:29,511][71601] Updated weights for policy 0, policy_version 24380 (0.0010) [2023-10-11 20:03:31,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 49905664. Throughput: 0: 1821.2, 1: 1818.8. Samples: 12478654. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-11 20:03:31,035][70582] Avg episode reward: [(0, '94.770'), (1, '39.450')] [2023-10-11 20:03:31,036][71353] Saving new best policy, reward=94.770! [2023-10-11 20:03:32,739][71635] Updated weights for policy 1, policy_version 24362 (0.0008) [2023-10-11 20:03:33,102][71635] Updated weights for policy 1, policy_version 24372 (0.0008) [2023-10-11 20:03:33,468][71635] Updated weights for policy 1, policy_version 24382 (0.0008) [2023-10-11 20:03:33,514][71601] Updated weights for policy 0, policy_version 24390 (0.0008) [2023-10-11 20:03:33,881][71601] Updated weights for policy 0, policy_version 24400 (0.0008) [2023-10-11 20:03:34,249][71601] Updated weights for policy 0, policy_version 24410 (0.0010) [2023-10-11 20:03:36,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 49971200. Throughput: 0: 1814.9, 1: 1815.2. Samples: 12498874. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-11 20:03:36,034][70582] Avg episode reward: [(0, '106.510'), (1, '41.250')] [2023-10-11 20:03:36,035][71353] Saving new best policy, reward=106.510! [2023-10-11 20:03:37,166][71635] Updated weights for policy 1, policy_version 24392 (0.0008) [2023-10-11 20:03:37,530][71635] Updated weights for policy 1, policy_version 24402 (0.0008) [2023-10-11 20:03:37,885][71601] Updated weights for policy 0, policy_version 24420 (0.0008) [2023-10-11 20:03:37,903][71635] Updated weights for policy 1, policy_version 24412 (0.0008) [2023-10-11 20:03:38,253][71601] Updated weights for policy 0, policy_version 24430 (0.0008) [2023-10-11 20:03:38,631][71601] Updated weights for policy 0, policy_version 24440 (0.0007) [2023-10-11 20:03:41,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 50036736. Throughput: 0: 1818.7, 1: 1812.4. Samples: 12521602. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-11 20:03:41,034][70582] Avg episode reward: [(0, '106.150'), (1, '40.930')] [2023-10-11 20:03:41,649][71635] Updated weights for policy 1, policy_version 24422 (0.0010) [2023-10-11 20:03:42,023][71635] Updated weights for policy 1, policy_version 24432 (0.0010) [2023-10-11 20:03:42,200][71601] Updated weights for policy 0, policy_version 24450 (0.0007) [2023-10-11 20:03:42,392][71635] Updated weights for policy 1, policy_version 24442 (0.0007) [2023-10-11 20:03:42,566][71601] Updated weights for policy 0, policy_version 24460 (0.0009) [2023-10-11 20:03:42,943][71601] Updated weights for policy 0, policy_version 24470 (0.0009) [2023-10-11 20:03:43,317][71601] Updated weights for policy 0, policy_version 24480 (0.0008) [2023-10-11 20:03:46,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 50102272. Throughput: 0: 1815.7, 1: 1812.6. Samples: 12531438. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:03:46,035][70582] Avg episode reward: [(0, '101.300'), (1, '44.460')] [2023-10-11 20:03:46,111][71635] Updated weights for policy 1, policy_version 24452 (0.0009) [2023-10-11 20:03:46,481][71635] Updated weights for policy 1, policy_version 24462 (0.0007) [2023-10-11 20:03:46,852][71635] Updated weights for policy 1, policy_version 24472 (0.0008) [2023-10-11 20:03:46,991][71601] Updated weights for policy 0, policy_version 24490 (0.0007) [2023-10-11 20:03:47,365][71601] Updated weights for policy 0, policy_version 24500 (0.0007) [2023-10-11 20:03:47,734][71601] Updated weights for policy 0, policy_version 24510 (0.0008) [2023-10-11 20:03:50,551][71635] Updated weights for policy 1, policy_version 24482 (0.0008) [2023-10-11 20:03:50,912][71635] Updated weights for policy 1, policy_version 24492 (0.0007) [2023-10-11 20:03:51,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 50167808. Throughput: 0: 1811.0, 1: 1812.7. Samples: 12553910. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:03:51,034][70582] Avg episode reward: [(0, '99.040'), (1, '40.470')] [2023-10-11 20:03:51,280][71635] Updated weights for policy 1, policy_version 24502 (0.0008) [2023-10-11 20:03:51,521][71601] Updated weights for policy 0, policy_version 24520 (0.0008) [2023-10-11 20:03:51,646][71635] Updated weights for policy 1, policy_version 24512 (0.0008) [2023-10-11 20:03:51,901][71601] Updated weights for policy 0, policy_version 24530 (0.0009) [2023-10-11 20:03:52,282][71601] Updated weights for policy 0, policy_version 24540 (0.0010) [2023-10-11 20:03:55,500][71635] Updated weights for policy 1, policy_version 24522 (0.0008) [2023-10-11 20:03:55,866][71635] Updated weights for policy 1, policy_version 24532 (0.0008) [2023-10-11 20:03:55,962][71601] Updated weights for policy 0, policy_version 24550 (0.0008) [2023-10-11 20:03:56,034][70582] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 50233344. Throughput: 0: 1810.4, 1: 1815.8. Samples: 12576368. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:03:56,035][70582] Avg episode reward: [(0, '101.840'), (1, '37.650')] [2023-10-11 20:03:56,234][71635] Updated weights for policy 1, policy_version 24542 (0.0009) [2023-10-11 20:03:56,302][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000024544_25133056.pth... [2023-10-11 20:03:56,332][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000022848_23396352.pth [2023-10-11 20:03:56,343][71601] Updated weights for policy 0, policy_version 24560 (0.0008) [2023-10-11 20:03:56,704][71601] Updated weights for policy 0, policy_version 24570 (0.0010) [2023-10-11 20:03:56,929][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000024576_25165824.pth... [2023-10-11 20:03:56,958][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000022848_23396352.pth [2023-10-11 20:04:00,124][71635] Updated weights for policy 1, policy_version 24552 (0.0010) [2023-10-11 20:04:00,319][71601] Updated weights for policy 0, policy_version 24580 (0.0010) [2023-10-11 20:04:00,504][71635] Updated weights for policy 1, policy_version 24562 (0.0008) [2023-10-11 20:04:00,688][71601] Updated weights for policy 0, policy_version 24590 (0.0009) [2023-10-11 20:04:00,876][71635] Updated weights for policy 1, policy_version 24572 (0.0007) [2023-10-11 20:04:01,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 50331648. Throughput: 0: 1810.6, 1: 1801.8. Samples: 12586372. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:04:01,034][70582] Avg episode reward: [(0, '100.290'), (1, '37.950')] [2023-10-11 20:04:01,066][71601] Updated weights for policy 0, policy_version 24600 (0.0008) [2023-10-11 20:04:04,507][71635] Updated weights for policy 1, policy_version 24582 (0.0008) [2023-10-11 20:04:04,778][71601] Updated weights for policy 0, policy_version 24610 (0.0007) [2023-10-11 20:04:04,879][71635] Updated weights for policy 1, policy_version 24592 (0.0008) [2023-10-11 20:04:05,149][71601] Updated weights for policy 0, policy_version 24620 (0.0007) [2023-10-11 20:04:05,250][71635] Updated weights for policy 1, policy_version 24602 (0.0009) [2023-10-11 20:04:05,518][71601] Updated weights for policy 0, policy_version 24630 (0.0010) [2023-10-11 20:04:05,894][71601] Updated weights for policy 0, policy_version 24640 (0.0007) [2023-10-11 20:04:06,034][70582] Fps is (10 sec: 19661.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 50429952. Throughput: 0: 1818.2, 1: 1812.8. Samples: 12609122. Policy #0 lag: (min: 19.0, avg: 19.3, max: 31.0) [2023-10-11 20:04:06,035][70582] Avg episode reward: [(0, '96.290'), (1, '35.380')] [2023-10-11 20:04:09,013][71635] Updated weights for policy 1, policy_version 24612 (0.0008) [2023-10-11 20:04:09,378][71635] Updated weights for policy 1, policy_version 24622 (0.0008) [2023-10-11 20:04:09,462][71601] Updated weights for policy 0, policy_version 24650 (0.0008) [2023-10-11 20:04:09,742][71635] Updated weights for policy 1, policy_version 24632 (0.0008) [2023-10-11 20:04:09,829][71601] Updated weights for policy 0, policy_version 24660 (0.0008) [2023-10-11 20:04:10,193][71601] Updated weights for policy 0, policy_version 24670 (0.0009) [2023-10-11 20:04:11,034][70582] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 50495488. Throughput: 0: 1812.7, 1: 1800.5. Samples: 12628754. Policy #0 lag: (min: 19.0, avg: 19.3, max: 31.0) [2023-10-11 20:04:11,035][70582] Avg episode reward: [(0, '96.490'), (1, '40.160')] [2023-10-11 20:04:13,494][71635] Updated weights for policy 1, policy_version 24642 (0.0008) [2023-10-11 20:04:13,863][71635] Updated weights for policy 1, policy_version 24652 (0.0007) [2023-10-11 20:04:14,031][71601] Updated weights for policy 0, policy_version 24680 (0.0008) [2023-10-11 20:04:14,225][71635] Updated weights for policy 1, policy_version 24662 (0.0008) [2023-10-11 20:04:14,406][71601] Updated weights for policy 0, policy_version 24690 (0.0008) [2023-10-11 20:04:14,591][71635] Updated weights for policy 1, policy_version 24672 (0.0009) [2023-10-11 20:04:14,773][71601] Updated weights for policy 0, policy_version 24700 (0.0009) [2023-10-11 20:04:16,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 50561024. Throughput: 0: 1805.2, 1: 1808.8. Samples: 12641284. Policy #0 lag: (min: 19.0, avg: 19.3, max: 31.0) [2023-10-11 20:04:16,035][70582] Avg episode reward: [(0, '96.490'), (1, '44.040')] [2023-10-11 20:04:18,294][71635] Updated weights for policy 1, policy_version 24682 (0.0011) [2023-10-11 20:04:18,664][71635] Updated weights for policy 1, policy_version 24692 (0.0008) [2023-10-11 20:04:18,668][71601] Updated weights for policy 0, policy_version 24710 (0.0009) [2023-10-11 20:04:19,023][71635] Updated weights for policy 1, policy_version 24702 (0.0008) [2023-10-11 20:04:19,034][71601] Updated weights for policy 0, policy_version 24720 (0.0010) [2023-10-11 20:04:19,411][71601] Updated weights for policy 0, policy_version 24730 (0.0009) [2023-10-11 20:04:21,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 50626560. Throughput: 0: 1815.1, 1: 1795.0. Samples: 12661326. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-11 20:04:21,034][70582] Avg episode reward: [(0, '96.490'), (1, '41.710')] [2023-10-11 20:04:22,586][71635] Updated weights for policy 1, policy_version 24712 (0.0010) [2023-10-11 20:04:22,945][71635] Updated weights for policy 1, policy_version 24722 (0.0009) [2023-10-11 20:04:23,112][71601] Updated weights for policy 0, policy_version 24740 (0.0009) [2023-10-11 20:04:23,302][71635] Updated weights for policy 1, policy_version 24732 (0.0008) [2023-10-11 20:04:23,480][71601] Updated weights for policy 0, policy_version 24750 (0.0007) [2023-10-11 20:04:23,846][71601] Updated weights for policy 0, policy_version 24760 (0.0009) [2023-10-11 20:04:26,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 50692096. Throughput: 0: 1808.4, 1: 1803.5. Samples: 12684138. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-11 20:04:26,035][70582] Avg episode reward: [(0, '101.400'), (1, '39.740')] [2023-10-11 20:04:27,084][71635] Updated weights for policy 1, policy_version 24742 (0.0009) [2023-10-11 20:04:27,450][71635] Updated weights for policy 1, policy_version 24752 (0.0010) [2023-10-11 20:04:27,483][71601] Updated weights for policy 0, policy_version 24770 (0.0011) [2023-10-11 20:04:27,815][71635] Updated weights for policy 1, policy_version 24762 (0.0009) [2023-10-11 20:04:27,849][71601] Updated weights for policy 0, policy_version 24780 (0.0008) [2023-10-11 20:04:28,214][71601] Updated weights for policy 0, policy_version 24790 (0.0009) [2023-10-11 20:04:28,585][71601] Updated weights for policy 0, policy_version 24800 (0.0010) [2023-10-11 20:04:31,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 50757632. Throughput: 0: 1818.1, 1: 1802.9. Samples: 12694382. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-11 20:04:31,034][70582] Avg episode reward: [(0, '104.020'), (1, '38.460')] [2023-10-11 20:04:31,581][71635] Updated weights for policy 1, policy_version 24772 (0.0007) [2023-10-11 20:04:31,955][71635] Updated weights for policy 1, policy_version 24782 (0.0007) [2023-10-11 20:04:32,291][71601] Updated weights for policy 0, policy_version 24810 (0.0007) [2023-10-11 20:04:32,314][71635] Updated weights for policy 1, policy_version 24792 (0.0008) [2023-10-11 20:04:32,663][71601] Updated weights for policy 0, policy_version 24820 (0.0009) [2023-10-11 20:04:33,023][71601] Updated weights for policy 0, policy_version 24830 (0.0009) [2023-10-11 20:04:36,034][70582] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 50823168. Throughput: 0: 1810.4, 1: 1804.9. Samples: 12716602. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-11 20:04:36,034][70582] Avg episode reward: [(0, '107.070'), (1, '41.680')] [2023-10-11 20:04:36,035][71353] Saving new best policy, reward=107.070! [2023-10-11 20:04:36,099][71635] Updated weights for policy 1, policy_version 24802 (0.0010) [2023-10-11 20:04:36,473][71635] Updated weights for policy 1, policy_version 24812 (0.0010) [2023-10-11 20:04:36,720][71601] Updated weights for policy 0, policy_version 24840 (0.0008) [2023-10-11 20:04:36,840][71635] Updated weights for policy 1, policy_version 24822 (0.0008) [2023-10-11 20:04:37,089][71601] Updated weights for policy 0, policy_version 24850 (0.0008) [2023-10-11 20:04:37,198][71635] Updated weights for policy 1, policy_version 24832 (0.0008) [2023-10-11 20:04:37,464][71601] Updated weights for policy 0, policy_version 24860 (0.0009) [2023-10-11 20:04:40,875][71635] Updated weights for policy 1, policy_version 24842 (0.0007) [2023-10-11 20:04:41,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 50888704. Throughput: 0: 1802.8, 1: 1811.9. Samples: 12739028. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:04:41,034][70582] Avg episode reward: [(0, '110.070'), (1, '43.810')] [2023-10-11 20:04:41,208][71601] Updated weights for policy 0, policy_version 24870 (0.0008) [2023-10-11 20:04:41,237][71635] Updated weights for policy 1, policy_version 24852 (0.0007) [2023-10-11 20:04:41,572][71601] Updated weights for policy 0, policy_version 24880 (0.0007) [2023-10-11 20:04:41,606][71635] Updated weights for policy 1, policy_version 24862 (0.0008) [2023-10-11 20:04:41,952][71601] Updated weights for policy 0, policy_version 24890 (0.0007) [2023-10-11 20:04:42,176][71353] Saving new best policy, reward=110.070! [2023-10-11 20:04:45,500][71635] Updated weights for policy 1, policy_version 24872 (0.0009) [2023-10-11 20:04:45,773][71601] Updated weights for policy 0, policy_version 24900 (0.0008) [2023-10-11 20:04:45,888][71635] Updated weights for policy 1, policy_version 24882 (0.0009) [2023-10-11 20:04:46,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 50954240. Throughput: 0: 1804.8, 1: 1804.9. Samples: 12748810. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:04:46,034][70582] Avg episode reward: [(0, '116.120'), (1, '46.140')] [2023-10-11 20:04:46,146][71601] Updated weights for policy 0, policy_version 24910 (0.0008) [2023-10-11 20:04:46,245][71635] Updated weights for policy 1, policy_version 24892 (0.0008) [2023-10-11 20:04:46,515][71601] Updated weights for policy 0, policy_version 24920 (0.0008) [2023-10-11 20:04:46,810][71353] Saving new best policy, reward=116.120! [2023-10-11 20:04:49,862][71635] Updated weights for policy 1, policy_version 24902 (0.0008) [2023-10-11 20:04:50,206][71601] Updated weights for policy 0, policy_version 24930 (0.0008) [2023-10-11 20:04:50,221][71635] Updated weights for policy 1, policy_version 24912 (0.0010) [2023-10-11 20:04:50,576][71601] Updated weights for policy 0, policy_version 24940 (0.0008) [2023-10-11 20:04:50,585][71635] Updated weights for policy 1, policy_version 24922 (0.0010) [2023-10-11 20:04:50,940][71601] Updated weights for policy 0, policy_version 24950 (0.0008) [2023-10-11 20:04:51,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 51052544. Throughput: 0: 1801.9, 1: 1807.7. Samples: 12771554. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:04:51,034][70582] Avg episode reward: [(0, '115.660'), (1, '46.580')] [2023-10-11 20:04:51,308][71601] Updated weights for policy 0, policy_version 24960 (0.0009) [2023-10-11 20:04:54,276][71635] Updated weights for policy 1, policy_version 24932 (0.0008) [2023-10-11 20:04:54,647][71635] Updated weights for policy 1, policy_version 24942 (0.0007) [2023-10-11 20:04:55,011][71635] Updated weights for policy 1, policy_version 24952 (0.0008) [2023-10-11 20:04:55,036][71601] Updated weights for policy 0, policy_version 24970 (0.0007) [2023-10-11 20:04:55,407][71601] Updated weights for policy 0, policy_version 24980 (0.0008) [2023-10-11 20:04:55,774][71601] Updated weights for policy 0, policy_version 24990 (0.0007) [2023-10-11 20:04:56,034][70582] Fps is (10 sec: 19660.4, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 51150848. Throughput: 0: 1812.4, 1: 1806.0. Samples: 12791578. Policy #0 lag: (min: 14.0, avg: 21.3, max: 46.0) [2023-10-11 20:04:56,035][70582] Avg episode reward: [(0, '101.170'), (1, '44.350')] [2023-10-11 20:04:58,697][71635] Updated weights for policy 1, policy_version 24962 (0.0008) [2023-10-11 20:04:59,052][71635] Updated weights for policy 1, policy_version 24972 (0.0008) [2023-10-11 20:04:59,415][71635] Updated weights for policy 1, policy_version 24982 (0.0009) [2023-10-11 20:04:59,571][71601] Updated weights for policy 0, policy_version 25000 (0.0008) [2023-10-11 20:04:59,781][71635] Updated weights for policy 1, policy_version 24992 (0.0009) [2023-10-11 20:04:59,937][71601] Updated weights for policy 0, policy_version 25010 (0.0008) [2023-10-11 20:05:00,304][71601] Updated weights for policy 0, policy_version 25020 (0.0008) [2023-10-11 20:05:01,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 51216384. Throughput: 0: 1804.7, 1: 1811.9. Samples: 12804030. Policy #0 lag: (min: 14.0, avg: 21.3, max: 46.0) [2023-10-11 20:05:01,035][70582] Avg episode reward: [(0, '102.880'), (1, '47.090')] [2023-10-11 20:05:03,496][71635] Updated weights for policy 1, policy_version 25002 (0.0010) [2023-10-11 20:05:03,868][71635] Updated weights for policy 1, policy_version 25012 (0.0010) [2023-10-11 20:05:04,007][71601] Updated weights for policy 0, policy_version 25030 (0.0009) [2023-10-11 20:05:04,229][71635] Updated weights for policy 1, policy_version 25022 (0.0010) [2023-10-11 20:05:04,377][71601] Updated weights for policy 0, policy_version 25040 (0.0007) [2023-10-11 20:05:04,750][71601] Updated weights for policy 0, policy_version 25050 (0.0007) [2023-10-11 20:05:06,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 51281920. Throughput: 0: 1816.5, 1: 1805.9. Samples: 12824338. Policy #0 lag: (min: 14.0, avg: 21.3, max: 46.0) [2023-10-11 20:05:06,035][70582] Avg episode reward: [(0, '103.020'), (1, '42.210')] [2023-10-11 20:05:07,974][71635] Updated weights for policy 1, policy_version 25032 (0.0007) [2023-10-11 20:05:08,340][71635] Updated weights for policy 1, policy_version 25042 (0.0009) [2023-10-11 20:05:08,450][71601] Updated weights for policy 0, policy_version 25060 (0.0009) [2023-10-11 20:05:08,701][71635] Updated weights for policy 1, policy_version 25052 (0.0007) [2023-10-11 20:05:08,811][71601] Updated weights for policy 0, policy_version 25070 (0.0007) [2023-10-11 20:05:09,188][71601] Updated weights for policy 0, policy_version 25080 (0.0010) [2023-10-11 20:05:11,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 51347456. Throughput: 0: 1806.1, 1: 1801.8. Samples: 12846494. Policy #0 lag: (min: 14.0, avg: 21.3, max: 46.0) [2023-10-11 20:05:11,034][70582] Avg episode reward: [(0, '103.120'), (1, '49.080')] [2023-10-11 20:05:12,436][71635] Updated weights for policy 1, policy_version 25062 (0.0008) [2023-10-11 20:05:12,800][71635] Updated weights for policy 1, policy_version 25072 (0.0009) [2023-10-11 20:05:12,960][71601] Updated weights for policy 0, policy_version 25090 (0.0008) [2023-10-11 20:05:13,168][71635] Updated weights for policy 1, policy_version 25082 (0.0008) [2023-10-11 20:05:13,331][71601] Updated weights for policy 0, policy_version 25100 (0.0007) [2023-10-11 20:05:13,702][71601] Updated weights for policy 0, policy_version 25110 (0.0007) [2023-10-11 20:05:14,074][71601] Updated weights for policy 0, policy_version 25120 (0.0009) [2023-10-11 20:05:16,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 51412992. Throughput: 0: 1814.6, 1: 1805.8. Samples: 12857298. Policy #0 lag: (min: 10.0, avg: 11.6, max: 38.0) [2023-10-11 20:05:16,035][70582] Avg episode reward: [(0, '120.700'), (1, '48.090')] [2023-10-11 20:05:16,036][71353] Saving new best policy, reward=120.700! [2023-10-11 20:05:16,806][71635] Updated weights for policy 1, policy_version 25092 (0.0008) [2023-10-11 20:05:17,181][71635] Updated weights for policy 1, policy_version 25102 (0.0009) [2023-10-11 20:05:17,536][71635] Updated weights for policy 1, policy_version 25112 (0.0008) [2023-10-11 20:05:17,611][71601] Updated weights for policy 0, policy_version 25130 (0.0007) [2023-10-11 20:05:17,980][71601] Updated weights for policy 0, policy_version 25140 (0.0007) [2023-10-11 20:05:18,349][71601] Updated weights for policy 0, policy_version 25150 (0.0009) [2023-10-11 20:05:21,034][70582] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 51478528. Throughput: 0: 1812.5, 1: 1808.2. Samples: 12879536. Policy #0 lag: (min: 10.0, avg: 11.6, max: 38.0) [2023-10-11 20:05:21,035][70582] Avg episode reward: [(0, '121.860'), (1, '51.210')] [2023-10-11 20:05:21,037][71431] Saving new best policy, reward=51.210! [2023-10-11 20:05:21,037][71353] Saving new best policy, reward=121.860! [2023-10-11 20:05:21,241][71635] Updated weights for policy 1, policy_version 25122 (0.0007) [2023-10-11 20:05:21,612][71635] Updated weights for policy 1, policy_version 25132 (0.0009) [2023-10-11 20:05:21,978][71635] Updated weights for policy 1, policy_version 25142 (0.0008) [2023-10-11 20:05:22,070][71601] Updated weights for policy 0, policy_version 25160 (0.0009) [2023-10-11 20:05:22,340][71635] Updated weights for policy 1, policy_version 25152 (0.0008) [2023-10-11 20:05:22,444][71601] Updated weights for policy 0, policy_version 25170 (0.0007) [2023-10-11 20:05:22,824][71601] Updated weights for policy 0, policy_version 25180 (0.0007) [2023-10-11 20:05:25,966][71635] Updated weights for policy 1, policy_version 25162 (0.0007) [2023-10-11 20:05:26,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 51544064. Throughput: 0: 1815.3, 1: 1813.5. Samples: 12902328. Policy #0 lag: (min: 10.0, avg: 11.6, max: 38.0) [2023-10-11 20:05:26,035][70582] Avg episode reward: [(0, '127.110'), (1, '48.180')] [2023-10-11 20:05:26,046][71353] Saving new best policy, reward=127.110! [2023-10-11 20:05:26,344][71635] Updated weights for policy 1, policy_version 25172 (0.0007) [2023-10-11 20:05:26,523][71601] Updated weights for policy 0, policy_version 25190 (0.0008) [2023-10-11 20:05:26,710][71635] Updated weights for policy 1, policy_version 25182 (0.0009) [2023-10-11 20:05:26,888][71601] Updated weights for policy 0, policy_version 25200 (0.0007) [2023-10-11 20:05:27,262][71601] Updated weights for policy 0, policy_version 25210 (0.0009) [2023-10-11 20:05:30,509][71635] Updated weights for policy 1, policy_version 25192 (0.0009) [2023-10-11 20:05:30,893][71635] Updated weights for policy 1, policy_version 25202 (0.0008) [2023-10-11 20:05:30,955][71601] Updated weights for policy 0, policy_version 25220 (0.0009) [2023-10-11 20:05:31,034][70582] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 51609600. Throughput: 0: 1813.8, 1: 1815.6. Samples: 12912134. Policy #0 lag: (min: 10.0, avg: 11.6, max: 38.0) [2023-10-11 20:05:31,034][70582] Avg episode reward: [(0, '138.710'), (1, '46.980')] [2023-10-11 20:05:31,251][71635] Updated weights for policy 1, policy_version 25212 (0.0008) [2023-10-11 20:05:31,327][71601] Updated weights for policy 0, policy_version 25230 (0.0008) [2023-10-11 20:05:31,689][71601] Updated weights for policy 0, policy_version 25240 (0.0008) [2023-10-11 20:05:31,990][71353] Saving new best policy, reward=138.710! [2023-10-11 20:05:34,998][71635] Updated weights for policy 1, policy_version 25222 (0.0010) [2023-10-11 20:05:35,360][71635] Updated weights for policy 1, policy_version 25232 (0.0009) [2023-10-11 20:05:35,365][71601] Updated weights for policy 0, policy_version 25250 (0.0009) [2023-10-11 20:05:35,726][71635] Updated weights for policy 1, policy_version 25242 (0.0008) [2023-10-11 20:05:35,749][71601] Updated weights for policy 0, policy_version 25260 (0.0007) [2023-10-11 20:05:36,034][70582] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 51707904. Throughput: 0: 1813.0, 1: 1812.3. Samples: 12934690. Policy #0 lag: (min: 25.0, avg: 36.8, max: 57.0) [2023-10-11 20:05:36,034][70582] Avg episode reward: [(0, '138.760'), (1, '45.550')] [2023-10-11 20:05:36,112][71601] Updated weights for policy 0, policy_version 25270 (0.0007) [2023-10-11 20:05:36,473][71353] Saving new best policy, reward=138.760! [2023-10-11 20:05:36,475][71601] Updated weights for policy 0, policy_version 25280 (0.0008) [2023-10-11 20:05:39,412][71635] Updated weights for policy 1, policy_version 25252 (0.0008) [2023-10-11 20:05:39,780][71635] Updated weights for policy 1, policy_version 25262 (0.0011) [2023-10-11 20:05:40,146][71635] Updated weights for policy 1, policy_version 25272 (0.0009) [2023-10-11 20:05:40,215][71601] Updated weights for policy 0, policy_version 25290 (0.0009) [2023-10-11 20:05:40,584][71601] Updated weights for policy 0, policy_version 25300 (0.0007) [2023-10-11 20:05:40,955][71601] Updated weights for policy 0, policy_version 25310 (0.0007) [2023-10-11 20:05:41,034][70582] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 51806208. Throughput: 0: 1816.2, 1: 1817.6. Samples: 12955102. Policy #0 lag: (min: 25.0, avg: 36.8, max: 57.0) [2023-10-11 20:05:41,035][70582] Avg episode reward: [(0, '141.130'), (1, '51.110')] [2023-10-11 20:05:41,043][71353] Saving new best policy, reward=141.130! [2023-10-11 20:05:44,063][71635] Updated weights for policy 1, policy_version 25282 (0.0009) [2023-10-11 20:05:44,427][71635] Updated weights for policy 1, policy_version 25292 (0.0009) [2023-10-11 20:05:44,676][71601] Updated weights for policy 0, policy_version 25320 (0.0007) [2023-10-11 20:05:44,803][71635] Updated weights for policy 1, policy_version 25302 (0.0007) [2023-10-11 20:05:45,046][71601] Updated weights for policy 0, policy_version 25330 (0.0008) [2023-10-11 20:05:45,162][71635] Updated weights for policy 1, policy_version 25312 (0.0007) [2023-10-11 20:05:45,419][71601] Updated weights for policy 0, policy_version 25340 (0.0009) [2023-10-11 20:05:46,034][70582] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14551.2). Total num frames: 51871744. Throughput: 0: 1813.6, 1: 1810.9. Samples: 12967130. Policy #0 lag: (min: 25.0, avg: 36.8, max: 57.0) [2023-10-11 20:05:46,034][70582] Avg episode reward: [(0, '144.410'), (1, '48.990')] [2023-10-11 20:05:46,035][71353] Saving new best policy, reward=144.410! [2023-10-11 20:05:48,882][71635] Updated weights for policy 1, policy_version 25322 (0.0008) [2023-10-11 20:05:49,250][71635] Updated weights for policy 1, policy_version 25332 (0.0008) [2023-10-11 20:05:49,258][71601] Updated weights for policy 0, policy_version 25350 (0.0008) [2023-10-11 20:05:49,612][71635] Updated weights for policy 1, policy_version 25342 (0.0008) [2023-10-11 20:05:49,618][71601] Updated weights for policy 0, policy_version 25360 (0.0008) [2023-10-11 20:05:49,995][71601] Updated weights for policy 0, policy_version 25370 (0.0008) [2023-10-11 20:05:51,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 51937280. Throughput: 0: 1814.0, 1: 1820.2. Samples: 12987874. Policy #0 lag: (min: 18.0, avg: 25.9, max: 50.0) [2023-10-11 20:05:51,034][70582] Avg episode reward: [(0, '150.400'), (1, '50.400')] [2023-10-11 20:05:51,035][71353] Saving new best policy, reward=150.400! [2023-10-11 20:05:52,992][71635] Updated weights for policy 1, policy_version 25352 (0.0007) [2023-10-11 20:05:53,354][71635] Updated weights for policy 1, policy_version 25362 (0.0009) [2023-10-11 20:05:53,645][71601] Updated weights for policy 0, policy_version 25380 (0.0008) [2023-10-11 20:05:53,727][71635] Updated weights for policy 1, policy_version 25372 (0.0008) [2023-10-11 20:05:54,016][71601] Updated weights for policy 0, policy_version 25390 (0.0009) [2023-10-11 20:05:54,385][71601] Updated weights for policy 0, policy_version 25400 (0.0007) [2023-10-11 20:05:56,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 52002816. Throughput: 0: 1805.5, 1: 1814.0. Samples: 13009370. Policy #0 lag: (min: 18.0, avg: 25.9, max: 50.0) [2023-10-11 20:05:56,034][70582] Avg episode reward: [(0, '146.820'), (1, '45.910')] [2023-10-11 20:05:56,046][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000025408_26017792.pth... [2023-10-11 20:05:56,046][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000025376_25985024.pth... [2023-10-11 20:05:56,080][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000023680_24248320.pth [2023-10-11 20:05:56,084][71431] Saving a milestone ./train_atari/atari_gopher_APPO/checkpoint_p1/milestones/checkpoint_000025376_25985024.pth [2023-10-11 20:05:56,084][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000023712_24281088.pth [2023-10-11 20:05:56,090][71353] Saving a milestone ./train_atari/atari_gopher_APPO/checkpoint_p0/milestones/checkpoint_000025408_26017792.pth [2023-10-11 20:05:57,579][71635] Updated weights for policy 1, policy_version 25382 (0.0008) [2023-10-11 20:05:57,952][71635] Updated weights for policy 1, policy_version 25392 (0.0010) [2023-10-11 20:05:58,112][71601] Updated weights for policy 0, policy_version 25410 (0.0010) [2023-10-11 20:05:58,313][71635] Updated weights for policy 1, policy_version 25402 (0.0008) [2023-10-11 20:05:58,477][71601] Updated weights for policy 0, policy_version 25420 (0.0007) [2023-10-11 20:05:58,850][71601] Updated weights for policy 0, policy_version 25430 (0.0010) [2023-10-11 20:05:59,230][71601] Updated weights for policy 0, policy_version 25440 (0.0011) [2023-10-11 20:06:01,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 52068352. Throughput: 0: 1809.0, 1: 1818.0. Samples: 13020510. Policy #0 lag: (min: 18.0, avg: 25.9, max: 50.0) [2023-10-11 20:06:01,034][70582] Avg episode reward: [(0, '143.630'), (1, '41.700')] [2023-10-11 20:06:02,050][71635] Updated weights for policy 1, policy_version 25412 (0.0009) [2023-10-11 20:06:02,417][71635] Updated weights for policy 1, policy_version 25422 (0.0007) [2023-10-11 20:06:02,775][71635] Updated weights for policy 1, policy_version 25432 (0.0008) [2023-10-11 20:06:02,988][71601] Updated weights for policy 0, policy_version 25450 (0.0007) [2023-10-11 20:06:03,360][71601] Updated weights for policy 0, policy_version 25460 (0.0007) [2023-10-11 20:06:03,735][71601] Updated weights for policy 0, policy_version 25470 (0.0008) [2023-10-11 20:06:06,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 52133888. Throughput: 0: 1796.6, 1: 1806.4. Samples: 13041672. Policy #0 lag: (min: 18.0, avg: 25.9, max: 50.0) [2023-10-11 20:06:06,034][70582] Avg episode reward: [(0, '144.190'), (1, '44.550')] [2023-10-11 20:06:06,487][71635] Updated weights for policy 1, policy_version 25442 (0.0009) [2023-10-11 20:06:06,855][71635] Updated weights for policy 1, policy_version 25452 (0.0008) [2023-10-11 20:06:07,216][71635] Updated weights for policy 1, policy_version 25462 (0.0007) [2023-10-11 20:06:07,487][71601] Updated weights for policy 0, policy_version 25480 (0.0008) [2023-10-11 20:06:07,583][71635] Updated weights for policy 1, policy_version 25472 (0.0007) [2023-10-11 20:06:07,863][71601] Updated weights for policy 0, policy_version 25490 (0.0009) [2023-10-11 20:06:08,227][71601] Updated weights for policy 0, policy_version 25500 (0.0007) [2023-10-11 20:06:11,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 52199424. Throughput: 0: 1797.3, 1: 1810.4. Samples: 13064676. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-11 20:06:11,034][70582] Avg episode reward: [(0, '143.590'), (1, '44.740')] [2023-10-11 20:06:11,124][71635] Updated weights for policy 1, policy_version 25482 (0.0008) [2023-10-11 20:06:11,494][71635] Updated weights for policy 1, policy_version 25492 (0.0008) [2023-10-11 20:06:11,858][71635] Updated weights for policy 1, policy_version 25502 (0.0008) [2023-10-11 20:06:11,875][71601] Updated weights for policy 0, policy_version 25510 (0.0008) [2023-10-11 20:06:12,253][71601] Updated weights for policy 0, policy_version 25520 (0.0007) [2023-10-11 20:06:12,628][71601] Updated weights for policy 0, policy_version 25530 (0.0008) [2023-10-11 20:06:15,482][71635] Updated weights for policy 1, policy_version 25512 (0.0011) [2023-10-11 20:06:15,858][71635] Updated weights for policy 1, policy_version 25522 (0.0009) [2023-10-11 20:06:16,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 52264960. Throughput: 0: 1798.7, 1: 1808.9. Samples: 13074480. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-11 20:06:16,035][70582] Avg episode reward: [(0, '143.690'), (1, '45.260')] [2023-10-11 20:06:16,233][71635] Updated weights for policy 1, policy_version 25532 (0.0009) [2023-10-11 20:06:16,412][71601] Updated weights for policy 0, policy_version 25540 (0.0008) [2023-10-11 20:06:16,775][71601] Updated weights for policy 0, policy_version 25550 (0.0008) [2023-10-11 20:06:17,146][71601] Updated weights for policy 0, policy_version 25560 (0.0008) [2023-10-11 20:06:20,174][71635] Updated weights for policy 1, policy_version 25542 (0.0009) [2023-10-11 20:06:20,552][71635] Updated weights for policy 1, policy_version 25552 (0.0008) [2023-10-11 20:06:20,810][71601] Updated weights for policy 0, policy_version 25570 (0.0007) [2023-10-11 20:06:20,915][71635] Updated weights for policy 1, policy_version 25562 (0.0008) [2023-10-11 20:06:21,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 52330496. Throughput: 0: 1798.5, 1: 1811.2. Samples: 13097126. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-11 20:06:21,035][70582] Avg episode reward: [(0, '145.730'), (1, '46.680')] [2023-10-11 20:06:21,177][71601] Updated weights for policy 0, policy_version 25580 (0.0007) [2023-10-11 20:06:21,560][71601] Updated weights for policy 0, policy_version 25590 (0.0007) [2023-10-11 20:06:21,924][71601] Updated weights for policy 0, policy_version 25600 (0.0007) [2023-10-11 20:06:24,532][71635] Updated weights for policy 1, policy_version 25572 (0.0008) [2023-10-11 20:06:24,906][71635] Updated weights for policy 1, policy_version 25582 (0.0010) [2023-10-11 20:06:25,278][71635] Updated weights for policy 1, policy_version 25592 (0.0007) [2023-10-11 20:06:25,489][71601] Updated weights for policy 0, policy_version 25610 (0.0008) [2023-10-11 20:06:25,866][71601] Updated weights for policy 0, policy_version 25620 (0.0010) [2023-10-11 20:06:26,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 52428800. Throughput: 0: 1812.8, 1: 1817.2. Samples: 13118450. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-11 20:06:26,035][70582] Avg episode reward: [(0, '133.820'), (1, '45.790')] [2023-10-11 20:06:26,237][71601] Updated weights for policy 0, policy_version 25630 (0.0009) [2023-10-11 20:06:29,053][71635] Updated weights for policy 1, policy_version 25602 (0.0009) [2023-10-11 20:06:29,430][71635] Updated weights for policy 1, policy_version 25612 (0.0008) [2023-10-11 20:06:29,793][71635] Updated weights for policy 1, policy_version 25622 (0.0007) [2023-10-11 20:06:29,965][71601] Updated weights for policy 0, policy_version 25640 (0.0008) [2023-10-11 20:06:30,163][71635] Updated weights for policy 1, policy_version 25632 (0.0009) [2023-10-11 20:06:30,335][71601] Updated weights for policy 0, policy_version 25650 (0.0009) [2023-10-11 20:06:30,708][71601] Updated weights for policy 0, policy_version 25660 (0.0008) [2023-10-11 20:06:31,034][70582] Fps is (10 sec: 19661.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 52527104. Throughput: 0: 1801.0, 1: 1812.8. Samples: 13129752. Policy #0 lag: (min: 9.0, avg: 14.4, max: 41.0) [2023-10-11 20:06:31,034][70582] Avg episode reward: [(0, '133.820'), (1, '50.220')] [2023-10-11 20:06:34,043][71635] Updated weights for policy 1, policy_version 25642 (0.0009) [2023-10-11 20:06:34,401][71601] Updated weights for policy 0, policy_version 25670 (0.0008) [2023-10-11 20:06:34,409][71635] Updated weights for policy 1, policy_version 25652 (0.0008) [2023-10-11 20:06:34,769][71601] Updated weights for policy 0, policy_version 25680 (0.0009) [2023-10-11 20:06:34,783][71635] Updated weights for policy 1, policy_version 25662 (0.0009) [2023-10-11 20:06:35,132][71601] Updated weights for policy 0, policy_version 25690 (0.0008) [2023-10-11 20:06:36,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 52592640. Throughput: 0: 1813.6, 1: 1816.2. Samples: 13151214. Policy #0 lag: (min: 9.0, avg: 14.4, max: 41.0) [2023-10-11 20:06:36,035][70582] Avg episode reward: [(0, '146.230'), (1, '49.690')] [2023-10-11 20:06:38,402][71635] Updated weights for policy 1, policy_version 25672 (0.0011) [2023-10-11 20:06:38,772][71635] Updated weights for policy 1, policy_version 25682 (0.0008) [2023-10-11 20:06:38,890][71601] Updated weights for policy 0, policy_version 25700 (0.0009) [2023-10-11 20:06:39,135][71635] Updated weights for policy 1, policy_version 25692 (0.0008) [2023-10-11 20:06:39,259][71601] Updated weights for policy 0, policy_version 25710 (0.0010) [2023-10-11 20:06:39,631][71601] Updated weights for policy 0, policy_version 25720 (0.0008) [2023-10-11 20:06:41,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 52658176. Throughput: 0: 1810.3, 1: 1809.3. Samples: 13172250. Policy #0 lag: (min: 9.0, avg: 14.4, max: 41.0) [2023-10-11 20:06:41,034][70582] Avg episode reward: [(0, '158.730'), (1, '44.870')] [2023-10-11 20:06:41,046][71353] Saving new best policy, reward=158.730! [2023-10-11 20:06:42,777][71635] Updated weights for policy 1, policy_version 25702 (0.0009) [2023-10-11 20:06:43,139][71635] Updated weights for policy 1, policy_version 25712 (0.0011) [2023-10-11 20:06:43,272][71601] Updated weights for policy 0, policy_version 25730 (0.0008) [2023-10-11 20:06:43,512][71635] Updated weights for policy 1, policy_version 25722 (0.0007) [2023-10-11 20:06:43,640][71601] Updated weights for policy 0, policy_version 25740 (0.0009) [2023-10-11 20:06:44,015][71601] Updated weights for policy 0, policy_version 25750 (0.0008) [2023-10-11 20:06:44,384][71601] Updated weights for policy 0, policy_version 25760 (0.0008) [2023-10-11 20:06:46,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 52723712. Throughput: 0: 1814.9, 1: 1815.4. Samples: 13183874. Policy #0 lag: (min: 9.0, avg: 14.4, max: 41.0) [2023-10-11 20:06:46,034][70582] Avg episode reward: [(0, '158.490'), (1, '48.020')] [2023-10-11 20:06:47,065][71635] Updated weights for policy 1, policy_version 25732 (0.0008) [2023-10-11 20:06:47,431][71635] Updated weights for policy 1, policy_version 25742 (0.0008) [2023-10-11 20:06:47,804][71635] Updated weights for policy 1, policy_version 25752 (0.0007) [2023-10-11 20:06:48,169][71601] Updated weights for policy 0, policy_version 25770 (0.0009) [2023-10-11 20:06:48,551][71601] Updated weights for policy 0, policy_version 25780 (0.0007) [2023-10-11 20:06:48,920][71601] Updated weights for policy 0, policy_version 25790 (0.0008) [2023-10-11 20:06:51,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 52789248. Throughput: 0: 1808.0, 1: 1819.5. Samples: 13204912. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:06:51,035][70582] Avg episode reward: [(0, '165.720'), (1, '55.310')] [2023-10-11 20:06:51,036][71353] Saving new best policy, reward=165.720! [2023-10-11 20:06:51,036][71431] Saving new best policy, reward=55.310! [2023-10-11 20:06:51,551][71635] Updated weights for policy 1, policy_version 25762 (0.0008) [2023-10-11 20:06:51,925][71635] Updated weights for policy 1, policy_version 25772 (0.0009) [2023-10-11 20:06:52,289][71635] Updated weights for policy 1, policy_version 25782 (0.0007) [2023-10-11 20:06:52,651][71601] Updated weights for policy 0, policy_version 25800 (0.0008) [2023-10-11 20:06:52,652][71635] Updated weights for policy 1, policy_version 25792 (0.0007) [2023-10-11 20:06:53,022][71601] Updated weights for policy 0, policy_version 25810 (0.0010) [2023-10-11 20:06:53,398][71601] Updated weights for policy 0, policy_version 25820 (0.0009) [2023-10-11 20:06:56,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 52854784. Throughput: 0: 1808.1, 1: 1811.9. Samples: 13227574. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:06:56,034][70582] Avg episode reward: [(0, '164.190'), (1, '51.060')] [2023-10-11 20:06:56,233][71635] Updated weights for policy 1, policy_version 25802 (0.0008) [2023-10-11 20:06:56,610][71635] Updated weights for policy 1, policy_version 25812 (0.0008) [2023-10-11 20:06:56,968][71635] Updated weights for policy 1, policy_version 25822 (0.0009) [2023-10-11 20:06:56,975][71601] Updated weights for policy 0, policy_version 25830 (0.0008) [2023-10-11 20:06:57,350][71601] Updated weights for policy 0, policy_version 25840 (0.0010) [2023-10-11 20:06:57,715][71601] Updated weights for policy 0, policy_version 25850 (0.0010) [2023-10-11 20:07:00,705][71635] Updated weights for policy 1, policy_version 25832 (0.0009) [2023-10-11 20:07:01,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 52920320. Throughput: 0: 1812.1, 1: 1813.8. Samples: 13237648. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:07:01,034][70582] Avg episode reward: [(0, '170.370'), (1, '47.690')] [2023-10-11 20:07:01,035][71353] Saving new best policy, reward=170.370! [2023-10-11 20:07:01,072][71635] Updated weights for policy 1, policy_version 25842 (0.0010) [2023-10-11 20:07:01,434][71635] Updated weights for policy 1, policy_version 25852 (0.0007) [2023-10-11 20:07:01,440][71601] Updated weights for policy 0, policy_version 25860 (0.0009) [2023-10-11 20:07:01,817][71601] Updated weights for policy 0, policy_version 25870 (0.0010) [2023-10-11 20:07:02,183][71601] Updated weights for policy 0, policy_version 25880 (0.0008) [2023-10-11 20:07:05,367][71635] Updated weights for policy 1, policy_version 25862 (0.0008) [2023-10-11 20:07:05,729][71635] Updated weights for policy 1, policy_version 25872 (0.0009) [2023-10-11 20:07:05,867][71601] Updated weights for policy 0, policy_version 25890 (0.0009) [2023-10-11 20:07:06,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 52985856. Throughput: 0: 1814.0, 1: 1808.5. Samples: 13260136. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:07:06,035][70582] Avg episode reward: [(0, '170.370'), (1, '47.040')] [2023-10-11 20:07:06,103][71635] Updated weights for policy 1, policy_version 25882 (0.0008) [2023-10-11 20:07:06,243][71601] Updated weights for policy 0, policy_version 25900 (0.0008) [2023-10-11 20:07:06,614][71601] Updated weights for policy 0, policy_version 25910 (0.0010) [2023-10-11 20:07:06,987][71601] Updated weights for policy 0, policy_version 25920 (0.0010) [2023-10-11 20:07:09,838][71635] Updated weights for policy 1, policy_version 25892 (0.0010) [2023-10-11 20:07:10,199][71635] Updated weights for policy 1, policy_version 25902 (0.0010) [2023-10-11 20:07:10,569][71635] Updated weights for policy 1, policy_version 25912 (0.0007) [2023-10-11 20:07:10,676][71601] Updated weights for policy 0, policy_version 25930 (0.0009) [2023-10-11 20:07:11,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 53084160. Throughput: 0: 1819.9, 1: 1816.2. Samples: 13282072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:07:11,034][70582] Avg episode reward: [(0, '169.900'), (1, '40.570')] [2023-10-11 20:07:11,054][71601] Updated weights for policy 0, policy_version 25940 (0.0008) [2023-10-11 20:07:11,423][71601] Updated weights for policy 0, policy_version 25950 (0.0008) [2023-10-11 20:07:14,301][71635] Updated weights for policy 1, policy_version 25922 (0.0008) [2023-10-11 20:07:14,678][71635] Updated weights for policy 1, policy_version 25932 (0.0007) [2023-10-11 20:07:15,036][71635] Updated weights for policy 1, policy_version 25942 (0.0007) [2023-10-11 20:07:15,156][71601] Updated weights for policy 0, policy_version 25960 (0.0007) [2023-10-11 20:07:15,411][71635] Updated weights for policy 1, policy_version 25952 (0.0007) [2023-10-11 20:07:15,537][71601] Updated weights for policy 0, policy_version 25970 (0.0008) [2023-10-11 20:07:15,912][71601] Updated weights for policy 0, policy_version 25980 (0.0009) [2023-10-11 20:07:16,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 53149696. Throughput: 0: 1818.6, 1: 1811.6. Samples: 13293112. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-11 20:07:16,035][70582] Avg episode reward: [(0, '191.780'), (1, '35.490')] [2023-10-11 20:07:16,063][71353] Saving new best policy, reward=191.780! [2023-10-11 20:07:18,973][71635] Updated weights for policy 1, policy_version 25962 (0.0010) [2023-10-11 20:07:19,333][71635] Updated weights for policy 1, policy_version 25972 (0.0009) [2023-10-11 20:07:19,484][71601] Updated weights for policy 0, policy_version 25990 (0.0008) [2023-10-11 20:07:19,702][71635] Updated weights for policy 1, policy_version 25982 (0.0008) [2023-10-11 20:07:19,863][71601] Updated weights for policy 0, policy_version 26000 (0.0008) [2023-10-11 20:07:20,236][71601] Updated weights for policy 0, policy_version 26010 (0.0011) [2023-10-11 20:07:21,034][70582] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 53248000. Throughput: 0: 1823.6, 1: 1813.2. Samples: 13314866. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-11 20:07:21,035][70582] Avg episode reward: [(0, '181.280'), (1, '38.150')] [2023-10-11 20:07:23,478][71635] Updated weights for policy 1, policy_version 25992 (0.0008) [2023-10-11 20:07:23,845][71635] Updated weights for policy 1, policy_version 26002 (0.0010) [2023-10-11 20:07:23,973][71601] Updated weights for policy 0, policy_version 26020 (0.0008) [2023-10-11 20:07:24,212][71635] Updated weights for policy 1, policy_version 26012 (0.0010) [2023-10-11 20:07:24,346][71601] Updated weights for policy 0, policy_version 26030 (0.0008) [2023-10-11 20:07:24,716][71601] Updated weights for policy 0, policy_version 26040 (0.0009) [2023-10-11 20:07:26,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 53313536. Throughput: 0: 1818.6, 1: 1812.1. Samples: 13335634. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-11 20:07:26,034][70582] Avg episode reward: [(0, '181.190'), (1, '43.360')] [2023-10-11 20:07:27,677][71635] Updated weights for policy 1, policy_version 26022 (0.0007) [2023-10-11 20:07:28,047][71635] Updated weights for policy 1, policy_version 26032 (0.0007) [2023-10-11 20:07:28,406][71635] Updated weights for policy 1, policy_version 26042 (0.0007) [2023-10-11 20:07:28,511][71601] Updated weights for policy 0, policy_version 26050 (0.0009) [2023-10-11 20:07:28,891][71601] Updated weights for policy 0, policy_version 26060 (0.0007) [2023-10-11 20:07:29,264][71601] Updated weights for policy 0, policy_version 26070 (0.0008) [2023-10-11 20:07:29,632][71601] Updated weights for policy 0, policy_version 26080 (0.0007) [2023-10-11 20:07:31,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 53379072. Throughput: 0: 1822.6, 1: 1814.0. Samples: 13347524. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-11 20:07:31,034][70582] Avg episode reward: [(0, '181.620'), (1, '48.330')] [2023-10-11 20:07:32,159][71635] Updated weights for policy 1, policy_version 26052 (0.0008) [2023-10-11 20:07:32,530][71635] Updated weights for policy 1, policy_version 26062 (0.0011) [2023-10-11 20:07:32,899][71635] Updated weights for policy 1, policy_version 26072 (0.0008) [2023-10-11 20:07:33,294][71601] Updated weights for policy 0, policy_version 26090 (0.0008) [2023-10-11 20:07:33,662][71601] Updated weights for policy 0, policy_version 26100 (0.0007) [2023-10-11 20:07:34,035][71601] Updated weights for policy 0, policy_version 26110 (0.0010) [2023-10-11 20:07:36,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 53444608. Throughput: 0: 1827.9, 1: 1808.4. Samples: 13368542. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:07:36,034][70582] Avg episode reward: [(0, '181.490'), (1, '48.350')] [2023-10-11 20:07:36,774][71635] Updated weights for policy 1, policy_version 26082 (0.0008) [2023-10-11 20:07:37,134][71635] Updated weights for policy 1, policy_version 26092 (0.0007) [2023-10-11 20:07:37,507][71635] Updated weights for policy 1, policy_version 26102 (0.0009) [2023-10-11 20:07:37,640][71601] Updated weights for policy 0, policy_version 26120 (0.0008) [2023-10-11 20:07:37,868][71635] Updated weights for policy 1, policy_version 26112 (0.0008) [2023-10-11 20:07:38,026][71601] Updated weights for policy 0, policy_version 26130 (0.0009) [2023-10-11 20:07:38,408][71601] Updated weights for policy 0, policy_version 26140 (0.0009) [2023-10-11 20:07:41,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 53510144. Throughput: 0: 1830.2, 1: 1812.4. Samples: 13391492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:07:41,034][70582] Avg episode reward: [(0, '180.280'), (1, '46.310')] [2023-10-11 20:07:41,378][71635] Updated weights for policy 1, policy_version 26122 (0.0008) [2023-10-11 20:07:41,745][71635] Updated weights for policy 1, policy_version 26132 (0.0010) [2023-10-11 20:07:42,059][71601] Updated weights for policy 0, policy_version 26150 (0.0007) [2023-10-11 20:07:42,114][71635] Updated weights for policy 1, policy_version 26142 (0.0010) [2023-10-11 20:07:42,422][71601] Updated weights for policy 0, policy_version 26160 (0.0008) [2023-10-11 20:07:42,798][71601] Updated weights for policy 0, policy_version 26170 (0.0007) [2023-10-11 20:07:45,848][71635] Updated weights for policy 1, policy_version 26152 (0.0009) [2023-10-11 20:07:46,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 53575680. Throughput: 0: 1826.0, 1: 1815.4. Samples: 13401514. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:07:46,034][70582] Avg episode reward: [(0, '180.280'), (1, '51.680')] [2023-10-11 20:07:46,229][71635] Updated weights for policy 1, policy_version 26162 (0.0010) [2023-10-11 20:07:46,550][71601] Updated weights for policy 0, policy_version 26180 (0.0008) [2023-10-11 20:07:46,599][71635] Updated weights for policy 1, policy_version 26172 (0.0007) [2023-10-11 20:07:46,924][71601] Updated weights for policy 0, policy_version 26190 (0.0008) [2023-10-11 20:07:47,293][71601] Updated weights for policy 0, policy_version 26200 (0.0007) [2023-10-11 20:07:50,386][71635] Updated weights for policy 1, policy_version 26182 (0.0009) [2023-10-11 20:07:50,758][71635] Updated weights for policy 1, policy_version 26192 (0.0010) [2023-10-11 20:07:51,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 53641216. Throughput: 0: 1820.1, 1: 1822.3. Samples: 13424044. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:07:51,034][70582] Avg episode reward: [(0, '180.280'), (1, '52.800')] [2023-10-11 20:07:51,109][71601] Updated weights for policy 0, policy_version 26210 (0.0008) [2023-10-11 20:07:51,121][71635] Updated weights for policy 1, policy_version 26202 (0.0007) [2023-10-11 20:07:51,479][71601] Updated weights for policy 0, policy_version 26220 (0.0008) [2023-10-11 20:07:51,859][71601] Updated weights for policy 0, policy_version 26230 (0.0010) [2023-10-11 20:07:52,227][71601] Updated weights for policy 0, policy_version 26240 (0.0011) [2023-10-11 20:07:54,681][71635] Updated weights for policy 1, policy_version 26212 (0.0008) [2023-10-11 20:07:55,052][71635] Updated weights for policy 1, policy_version 26222 (0.0008) [2023-10-11 20:07:55,416][71635] Updated weights for policy 1, policy_version 26232 (0.0008) [2023-10-11 20:07:55,956][71601] Updated weights for policy 0, policy_version 26250 (0.0007) [2023-10-11 20:07:56,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 53739520. Throughput: 0: 1820.1, 1: 1818.0. Samples: 13445786. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:07:56,034][70582] Avg episode reward: [(0, '183.110'), (1, '52.190')] [2023-10-11 20:07:56,042][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000026240_26869760.pth... [2023-10-11 20:07:56,078][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000024544_25133056.pth [2023-10-11 20:07:56,336][71601] Updated weights for policy 0, policy_version 26260 (0.0008) [2023-10-11 20:07:56,700][71601] Updated weights for policy 0, policy_version 26270 (0.0007) [2023-10-11 20:07:56,775][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000026272_26902528.pth... [2023-10-11 20:07:56,804][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000024576_25165824.pth [2023-10-11 20:07:59,107][71635] Updated weights for policy 1, policy_version 26242 (0.0009) [2023-10-11 20:07:59,467][71635] Updated weights for policy 1, policy_version 26252 (0.0008) [2023-10-11 20:07:59,835][71635] Updated weights for policy 1, policy_version 26262 (0.0008) [2023-10-11 20:08:00,200][71635] Updated weights for policy 1, policy_version 26272 (0.0010) [2023-10-11 20:08:00,374][71601] Updated weights for policy 0, policy_version 26280 (0.0007) [2023-10-11 20:08:00,745][71601] Updated weights for policy 0, policy_version 26290 (0.0007) [2023-10-11 20:08:01,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 53805056. Throughput: 0: 1816.9, 1: 1816.2. Samples: 13456602. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:08:01,034][70582] Avg episode reward: [(0, '187.940'), (1, '61.050')] [2023-10-11 20:08:01,035][71431] Saving new best policy, reward=61.050! [2023-10-11 20:08:01,122][71601] Updated weights for policy 0, policy_version 26300 (0.0010) [2023-10-11 20:08:03,916][71635] Updated weights for policy 1, policy_version 26282 (0.0009) [2023-10-11 20:08:04,284][71635] Updated weights for policy 1, policy_version 26292 (0.0010) [2023-10-11 20:08:04,659][71635] Updated weights for policy 1, policy_version 26302 (0.0009) [2023-10-11 20:08:04,810][71601] Updated weights for policy 0, policy_version 26310 (0.0007) [2023-10-11 20:08:05,181][71601] Updated weights for policy 0, policy_version 26320 (0.0011) [2023-10-11 20:08:05,558][71601] Updated weights for policy 0, policy_version 26330 (0.0008) [2023-10-11 20:08:06,034][70582] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 53903360. Throughput: 0: 1815.9, 1: 1816.3. Samples: 13478316. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:08:06,034][70582] Avg episode reward: [(0, '197.450'), (1, '67.580')] [2023-10-11 20:08:06,035][71353] Saving new best policy, reward=197.450! [2023-10-11 20:08:06,035][71431] Saving new best policy, reward=67.580! [2023-10-11 20:08:08,533][71635] Updated weights for policy 1, policy_version 26312 (0.0010) [2023-10-11 20:08:08,899][71635] Updated weights for policy 1, policy_version 26322 (0.0007) [2023-10-11 20:08:09,149][71601] Updated weights for policy 0, policy_version 26340 (0.0007) [2023-10-11 20:08:09,269][71635] Updated weights for policy 1, policy_version 26332 (0.0007) [2023-10-11 20:08:09,516][71601] Updated weights for policy 0, policy_version 26350 (0.0009) [2023-10-11 20:08:09,888][71601] Updated weights for policy 0, policy_version 26360 (0.0007) [2023-10-11 20:08:11,034][70582] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 53968896. Throughput: 0: 1815.1, 1: 1814.3. Samples: 13498958. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:08:11,035][70582] Avg episode reward: [(0, '197.400'), (1, '74.220')] [2023-10-11 20:08:11,046][71431] Saving new best policy, reward=74.220! [2023-10-11 20:08:12,950][71635] Updated weights for policy 1, policy_version 26342 (0.0008) [2023-10-11 20:08:13,306][71635] Updated weights for policy 1, policy_version 26352 (0.0008) [2023-10-11 20:08:13,517][71601] Updated weights for policy 0, policy_version 26370 (0.0008) [2023-10-11 20:08:13,672][71635] Updated weights for policy 1, policy_version 26362 (0.0009) [2023-10-11 20:08:13,887][71601] Updated weights for policy 0, policy_version 26380 (0.0007) [2023-10-11 20:08:14,256][71601] Updated weights for policy 0, policy_version 26390 (0.0010) [2023-10-11 20:08:14,629][71601] Updated weights for policy 0, policy_version 26400 (0.0010) [2023-10-11 20:08:16,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 54034432. Throughput: 0: 1818.4, 1: 1814.0. Samples: 13510982. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:08:16,034][70582] Avg episode reward: [(0, '196.880'), (1, '75.310')] [2023-10-11 20:08:16,035][71431] Saving new best policy, reward=75.310! [2023-10-11 20:08:17,422][71635] Updated weights for policy 1, policy_version 26372 (0.0009) [2023-10-11 20:08:17,800][71635] Updated weights for policy 1, policy_version 26382 (0.0008) [2023-10-11 20:08:18,158][71635] Updated weights for policy 1, policy_version 26392 (0.0007) [2023-10-11 20:08:18,202][71601] Updated weights for policy 0, policy_version 26410 (0.0007) [2023-10-11 20:08:18,567][71601] Updated weights for policy 0, policy_version 26420 (0.0008) [2023-10-11 20:08:18,935][71601] Updated weights for policy 0, policy_version 26430 (0.0007) [2023-10-11 20:08:21,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 54099968. Throughput: 0: 1814.1, 1: 1807.4. Samples: 13531510. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-11 20:08:21,035][70582] Avg episode reward: [(0, '190.960'), (1, '77.140')] [2023-10-11 20:08:21,036][71431] Saving new best policy, reward=77.140! [2023-10-11 20:08:21,881][71635] Updated weights for policy 1, policy_version 26402 (0.0007) [2023-10-11 20:08:22,244][71635] Updated weights for policy 1, policy_version 26412 (0.0008) [2023-10-11 20:08:22,606][71635] Updated weights for policy 1, policy_version 26422 (0.0007) [2023-10-11 20:08:22,768][71601] Updated weights for policy 0, policy_version 26440 (0.0007) [2023-10-11 20:08:22,972][71635] Updated weights for policy 1, policy_version 26432 (0.0007) [2023-10-11 20:08:23,129][71601] Updated weights for policy 0, policy_version 26450 (0.0009) [2023-10-11 20:08:23,501][71601] Updated weights for policy 0, policy_version 26460 (0.0009) [2023-10-11 20:08:26,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 54165504. Throughput: 0: 1810.5, 1: 1802.7. Samples: 13554088. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-11 20:08:26,035][70582] Avg episode reward: [(0, '177.120'), (1, '79.650')] [2023-10-11 20:08:26,047][71431] Saving new best policy, reward=79.650! [2023-10-11 20:08:26,664][71635] Updated weights for policy 1, policy_version 26442 (0.0010) [2023-10-11 20:08:27,032][71635] Updated weights for policy 1, policy_version 26452 (0.0009) [2023-10-11 20:08:27,355][71601] Updated weights for policy 0, policy_version 26470 (0.0010) [2023-10-11 20:08:27,396][71635] Updated weights for policy 1, policy_version 26462 (0.0009) [2023-10-11 20:08:27,731][71601] Updated weights for policy 0, policy_version 26480 (0.0009) [2023-10-11 20:08:28,098][71601] Updated weights for policy 0, policy_version 26490 (0.0009) [2023-10-11 20:08:31,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 54231040. Throughput: 0: 1806.8, 1: 1802.6. Samples: 13563936. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-11 20:08:31,035][70582] Avg episode reward: [(0, '184.080'), (1, '80.360')] [2023-10-11 20:08:31,223][71635] Updated weights for policy 1, policy_version 26472 (0.0007) [2023-10-11 20:08:31,577][71635] Updated weights for policy 1, policy_version 26482 (0.0007) [2023-10-11 20:08:31,736][71601] Updated weights for policy 0, policy_version 26500 (0.0008) [2023-10-11 20:08:31,944][71635] Updated weights for policy 1, policy_version 26492 (0.0008) [2023-10-11 20:08:32,088][71431] Saving new best policy, reward=80.360! [2023-10-11 20:08:32,094][71601] Updated weights for policy 0, policy_version 26510 (0.0008) [2023-10-11 20:08:32,462][71601] Updated weights for policy 0, policy_version 26520 (0.0009) [2023-10-11 20:08:35,717][71635] Updated weights for policy 1, policy_version 26502 (0.0010) [2023-10-11 20:08:36,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 54296576. Throughput: 0: 1812.0, 1: 1802.4. Samples: 13586688. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-11 20:08:36,034][70582] Avg episode reward: [(0, '171.900'), (1, '77.400')] [2023-10-11 20:08:36,081][71635] Updated weights for policy 1, policy_version 26512 (0.0009) [2023-10-11 20:08:36,173][71601] Updated weights for policy 0, policy_version 26530 (0.0010) [2023-10-11 20:08:36,444][71635] Updated weights for policy 1, policy_version 26522 (0.0007) [2023-10-11 20:08:36,546][71601] Updated weights for policy 0, policy_version 26540 (0.0008) [2023-10-11 20:08:36,922][71601] Updated weights for policy 0, policy_version 26550 (0.0008) [2023-10-11 20:08:37,285][71601] Updated weights for policy 0, policy_version 26560 (0.0007) [2023-10-11 20:08:40,290][71635] Updated weights for policy 1, policy_version 26532 (0.0008) [2023-10-11 20:08:40,660][71635] Updated weights for policy 1, policy_version 26542 (0.0009) [2023-10-11 20:08:41,033][71635] Updated weights for policy 1, policy_version 26552 (0.0008) [2023-10-11 20:08:41,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 54362112. Throughput: 0: 1810.2, 1: 1812.1. Samples: 13608790. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-11 20:08:41,034][70582] Avg episode reward: [(0, '171.490'), (1, '77.060')] [2023-10-11 20:08:41,150][71601] Updated weights for policy 0, policy_version 26570 (0.0010) [2023-10-11 20:08:41,514][71601] Updated weights for policy 0, policy_version 26580 (0.0007) [2023-10-11 20:08:41,881][71601] Updated weights for policy 0, policy_version 26590 (0.0008) [2023-10-11 20:08:44,611][71635] Updated weights for policy 1, policy_version 26562 (0.0009) [2023-10-11 20:08:44,987][71635] Updated weights for policy 1, policy_version 26572 (0.0008) [2023-10-11 20:08:45,353][71635] Updated weights for policy 1, policy_version 26582 (0.0010) [2023-10-11 20:08:45,501][71601] Updated weights for policy 0, policy_version 26600 (0.0007) [2023-10-11 20:08:45,716][71635] Updated weights for policy 1, policy_version 26592 (0.0007) [2023-10-11 20:08:45,876][71601] Updated weights for policy 0, policy_version 26610 (0.0007) [2023-10-11 20:08:46,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 54460416. Throughput: 0: 1808.3, 1: 1803.5. Samples: 13619138. Policy #0 lag: (min: 7.0, avg: 7.5, max: 21.0) [2023-10-11 20:08:46,035][70582] Avg episode reward: [(0, '161.880'), (1, '74.780')] [2023-10-11 20:08:46,241][71601] Updated weights for policy 0, policy_version 26620 (0.0007) [2023-10-11 20:08:49,503][71635] Updated weights for policy 1, policy_version 26602 (0.0008) [2023-10-11 20:08:49,822][71601] Updated weights for policy 0, policy_version 26630 (0.0008) [2023-10-11 20:08:49,855][71635] Updated weights for policy 1, policy_version 26612 (0.0008) [2023-10-11 20:08:50,196][71601] Updated weights for policy 0, policy_version 26640 (0.0008) [2023-10-11 20:08:50,223][71635] Updated weights for policy 1, policy_version 26622 (0.0009) [2023-10-11 20:08:50,558][71601] Updated weights for policy 0, policy_version 26650 (0.0008) [2023-10-11 20:08:51,034][70582] Fps is (10 sec: 19661.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 54558720. Throughput: 0: 1812.2, 1: 1816.8. Samples: 13641624. Policy #0 lag: (min: 7.0, avg: 7.5, max: 21.0) [2023-10-11 20:08:51,034][70582] Avg episode reward: [(0, '161.880'), (1, '80.280')] [2023-10-11 20:08:53,793][71635] Updated weights for policy 1, policy_version 26632 (0.0009) [2023-10-11 20:08:54,160][71635] Updated weights for policy 1, policy_version 26642 (0.0009) [2023-10-11 20:08:54,249][71601] Updated weights for policy 0, policy_version 26660 (0.0008) [2023-10-11 20:08:54,525][71635] Updated weights for policy 1, policy_version 26652 (0.0007) [2023-10-11 20:08:54,614][71601] Updated weights for policy 0, policy_version 26670 (0.0007) [2023-10-11 20:08:54,988][71601] Updated weights for policy 0, policy_version 26680 (0.0011) [2023-10-11 20:08:56,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 54624256. Throughput: 0: 1803.9, 1: 1807.8. Samples: 13661484. Policy #0 lag: (min: 7.0, avg: 7.5, max: 21.0) [2023-10-11 20:08:56,034][70582] Avg episode reward: [(0, '161.880'), (1, '74.700')] [2023-10-11 20:08:58,175][71635] Updated weights for policy 1, policy_version 26662 (0.0008) [2023-10-11 20:08:58,538][71635] Updated weights for policy 1, policy_version 26672 (0.0007) [2023-10-11 20:08:58,634][71601] Updated weights for policy 0, policy_version 26690 (0.0009) [2023-10-11 20:08:58,906][71635] Updated weights for policy 1, policy_version 26682 (0.0008) [2023-10-11 20:08:59,002][71601] Updated weights for policy 0, policy_version 26700 (0.0009) [2023-10-11 20:08:59,375][71601] Updated weights for policy 0, policy_version 26710 (0.0010) [2023-10-11 20:08:59,751][71601] Updated weights for policy 0, policy_version 26720 (0.0010) [2023-10-11 20:09:01,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 54689792. Throughput: 0: 1809.2, 1: 1818.9. Samples: 13674248. Policy #0 lag: (min: 24.0, avg: 44.5, max: 56.0) [2023-10-11 20:09:01,034][70582] Avg episode reward: [(0, '161.880'), (1, '68.550')] [2023-10-11 20:09:02,688][71635] Updated weights for policy 1, policy_version 26692 (0.0009) [2023-10-11 20:09:03,049][71635] Updated weights for policy 1, policy_version 26702 (0.0008) [2023-10-11 20:09:03,410][71635] Updated weights for policy 1, policy_version 26712 (0.0009) [2023-10-11 20:09:03,454][71601] Updated weights for policy 0, policy_version 26730 (0.0008) [2023-10-11 20:09:03,822][71601] Updated weights for policy 0, policy_version 26740 (0.0008) [2023-10-11 20:09:04,202][71601] Updated weights for policy 0, policy_version 26750 (0.0008) [2023-10-11 20:09:06,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 54755328. Throughput: 0: 1808.6, 1: 1812.6. Samples: 13694466. Policy #0 lag: (min: 24.0, avg: 44.5, max: 56.0) [2023-10-11 20:09:06,035][70582] Avg episode reward: [(0, '161.880'), (1, '69.570')] [2023-10-11 20:09:07,081][71635] Updated weights for policy 1, policy_version 26722 (0.0008) [2023-10-11 20:09:07,453][71635] Updated weights for policy 1, policy_version 26732 (0.0007) [2023-10-11 20:09:07,812][71635] Updated weights for policy 1, policy_version 26742 (0.0008) [2023-10-11 20:09:08,003][71601] Updated weights for policy 0, policy_version 26760 (0.0007) [2023-10-11 20:09:08,185][71635] Updated weights for policy 1, policy_version 26752 (0.0009) [2023-10-11 20:09:08,384][71601] Updated weights for policy 0, policy_version 26770 (0.0007) [2023-10-11 20:09:08,761][71601] Updated weights for policy 0, policy_version 26780 (0.0008) [2023-10-11 20:09:11,034][70582] Fps is (10 sec: 13106.6, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 54820864. Throughput: 0: 1809.6, 1: 1810.7. Samples: 13717004. Policy #0 lag: (min: 24.0, avg: 44.5, max: 56.0) [2023-10-11 20:09:11,036][70582] Avg episode reward: [(0, '170.920'), (1, '62.120')] [2023-10-11 20:09:11,945][71635] Updated weights for policy 1, policy_version 26762 (0.0007) [2023-10-11 20:09:12,300][71635] Updated weights for policy 1, policy_version 26772 (0.0007) [2023-10-11 20:09:12,509][71601] Updated weights for policy 0, policy_version 26790 (0.0008) [2023-10-11 20:09:12,674][71635] Updated weights for policy 1, policy_version 26782 (0.0008) [2023-10-11 20:09:12,890][71601] Updated weights for policy 0, policy_version 26800 (0.0010) [2023-10-11 20:09:13,265][71601] Updated weights for policy 0, policy_version 26810 (0.0008) [2023-10-11 20:09:16,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 54886400. Throughput: 0: 1815.7, 1: 1809.0. Samples: 13727048. Policy #0 lag: (min: 24.0, avg: 44.5, max: 56.0) [2023-10-11 20:09:16,035][70582] Avg episode reward: [(0, '176.850'), (1, '63.630')] [2023-10-11 20:09:16,439][71635] Updated weights for policy 1, policy_version 26792 (0.0007) [2023-10-11 20:09:16,810][71635] Updated weights for policy 1, policy_version 26802 (0.0008) [2023-10-11 20:09:17,033][71601] Updated weights for policy 0, policy_version 26820 (0.0009) [2023-10-11 20:09:17,182][71635] Updated weights for policy 1, policy_version 26812 (0.0007) [2023-10-11 20:09:17,401][71601] Updated weights for policy 0, policy_version 26830 (0.0008) [2023-10-11 20:09:17,786][71601] Updated weights for policy 0, policy_version 26840 (0.0011) [2023-10-11 20:09:20,947][71635] Updated weights for policy 1, policy_version 26822 (0.0010) [2023-10-11 20:09:21,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 54951936. Throughput: 0: 1808.1, 1: 1804.4. Samples: 13749252. Policy #0 lag: (min: 24.0, avg: 44.5, max: 56.0) [2023-10-11 20:09:21,035][70582] Avg episode reward: [(0, '177.030'), (1, '63.590')] [2023-10-11 20:09:21,319][71635] Updated weights for policy 1, policy_version 26832 (0.0009) [2023-10-11 20:09:21,566][71601] Updated weights for policy 0, policy_version 26850 (0.0009) [2023-10-11 20:09:21,687][71635] Updated weights for policy 1, policy_version 26842 (0.0011) [2023-10-11 20:09:21,933][71601] Updated weights for policy 0, policy_version 26860 (0.0008) [2023-10-11 20:09:22,301][71601] Updated weights for policy 0, policy_version 26870 (0.0009) [2023-10-11 20:09:22,671][71601] Updated weights for policy 0, policy_version 26880 (0.0008) [2023-10-11 20:09:25,254][71635] Updated weights for policy 1, policy_version 26852 (0.0007) [2023-10-11 20:09:25,625][71635] Updated weights for policy 1, policy_version 26862 (0.0007) [2023-10-11 20:09:25,996][71635] Updated weights for policy 1, policy_version 26872 (0.0007) [2023-10-11 20:09:26,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 55017472. Throughput: 0: 1808.2, 1: 1813.9. Samples: 13771786. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:09:26,034][70582] Avg episode reward: [(0, '172.600'), (1, '62.550')] [2023-10-11 20:09:26,482][71601] Updated weights for policy 0, policy_version 26890 (0.0008) [2023-10-11 20:09:26,846][71601] Updated weights for policy 0, policy_version 26900 (0.0010) [2023-10-11 20:09:27,223][71601] Updated weights for policy 0, policy_version 26910 (0.0009) [2023-10-11 20:09:29,795][71635] Updated weights for policy 1, policy_version 26882 (0.0008) [2023-10-11 20:09:30,160][71635] Updated weights for policy 1, policy_version 26892 (0.0007) [2023-10-11 20:09:30,525][71635] Updated weights for policy 1, policy_version 26902 (0.0008) [2023-10-11 20:09:30,858][71601] Updated weights for policy 0, policy_version 26920 (0.0008) [2023-10-11 20:09:30,892][71635] Updated weights for policy 1, policy_version 26912 (0.0007) [2023-10-11 20:09:31,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 55115776. Throughput: 0: 1808.0, 1: 1811.7. Samples: 13782020. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:09:31,034][70582] Avg episode reward: [(0, '172.830'), (1, '66.720')] [2023-10-11 20:09:31,226][71601] Updated weights for policy 0, policy_version 26930 (0.0009) [2023-10-11 20:09:31,598][71601] Updated weights for policy 0, policy_version 26940 (0.0010) [2023-10-11 20:09:34,610][71635] Updated weights for policy 1, policy_version 26922 (0.0008) [2023-10-11 20:09:34,971][71635] Updated weights for policy 1, policy_version 26932 (0.0008) [2023-10-11 20:09:35,245][71601] Updated weights for policy 0, policy_version 26950 (0.0008) [2023-10-11 20:09:35,341][71635] Updated weights for policy 1, policy_version 26942 (0.0008) [2023-10-11 20:09:35,617][71601] Updated weights for policy 0, policy_version 26960 (0.0007) [2023-10-11 20:09:35,994][71601] Updated weights for policy 0, policy_version 26970 (0.0008) [2023-10-11 20:09:36,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 55181312. Throughput: 0: 1805.5, 1: 1818.4. Samples: 13804698. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:09:36,035][70582] Avg episode reward: [(0, '172.920'), (1, '63.580')] [2023-10-11 20:09:38,992][71635] Updated weights for policy 1, policy_version 26952 (0.0009) [2023-10-11 20:09:39,348][71635] Updated weights for policy 1, policy_version 26962 (0.0008) [2023-10-11 20:09:39,602][71601] Updated weights for policy 0, policy_version 26980 (0.0007) [2023-10-11 20:09:39,710][71635] Updated weights for policy 1, policy_version 26972 (0.0008) [2023-10-11 20:09:39,972][71601] Updated weights for policy 0, policy_version 26990 (0.0008) [2023-10-11 20:09:40,339][71601] Updated weights for policy 0, policy_version 27000 (0.0009) [2023-10-11 20:09:41,034][70582] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 55279616. Throughput: 0: 1817.0, 1: 1815.3. Samples: 13824938. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:09:41,034][70582] Avg episode reward: [(0, '172.920'), (1, '65.600')] [2023-10-11 20:09:43,317][71635] Updated weights for policy 1, policy_version 26982 (0.0010) [2023-10-11 20:09:43,690][71635] Updated weights for policy 1, policy_version 26992 (0.0012) [2023-10-11 20:09:44,049][71635] Updated weights for policy 1, policy_version 27002 (0.0009) [2023-10-11 20:09:44,108][71601] Updated weights for policy 0, policy_version 27010 (0.0009) [2023-10-11 20:09:44,482][71601] Updated weights for policy 0, policy_version 27020 (0.0007) [2023-10-11 20:09:44,862][71601] Updated weights for policy 0, policy_version 27030 (0.0008) [2023-10-11 20:09:45,236][71601] Updated weights for policy 0, policy_version 27040 (0.0008) [2023-10-11 20:09:46,034][70582] Fps is (10 sec: 16384.5, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 55345152. Throughput: 0: 1805.3, 1: 1818.2. Samples: 13837304. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-10-11 20:09:46,034][70582] Avg episode reward: [(0, '183.160'), (1, '69.580')] [2023-10-11 20:09:47,678][71635] Updated weights for policy 1, policy_version 27012 (0.0009) [2023-10-11 20:09:48,039][71635] Updated weights for policy 1, policy_version 27022 (0.0011) [2023-10-11 20:09:48,414][71635] Updated weights for policy 1, policy_version 27032 (0.0010) [2023-10-11 20:09:48,884][71601] Updated weights for policy 0, policy_version 27050 (0.0008) [2023-10-11 20:09:49,263][71601] Updated weights for policy 0, policy_version 27060 (0.0008) [2023-10-11 20:09:49,632][71601] Updated weights for policy 0, policy_version 27070 (0.0008) [2023-10-11 20:09:51,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 55410688. Throughput: 0: 1812.6, 1: 1818.1. Samples: 13857848. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-10-11 20:09:51,034][70582] Avg episode reward: [(0, '183.030'), (1, '74.530')] [2023-10-11 20:09:52,019][71635] Updated weights for policy 1, policy_version 27042 (0.0008) [2023-10-11 20:09:52,380][71635] Updated weights for policy 1, policy_version 27052 (0.0010) [2023-10-11 20:09:52,745][71635] Updated weights for policy 1, policy_version 27062 (0.0008) [2023-10-11 20:09:53,111][71635] Updated weights for policy 1, policy_version 27072 (0.0007) [2023-10-11 20:09:53,354][71601] Updated weights for policy 0, policy_version 27080 (0.0008) [2023-10-11 20:09:53,730][71601] Updated weights for policy 0, policy_version 27090 (0.0009) [2023-10-11 20:09:54,105][71601] Updated weights for policy 0, policy_version 27100 (0.0009) [2023-10-11 20:09:56,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 55476224. Throughput: 0: 1806.9, 1: 1826.3. Samples: 13880500. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-10-11 20:09:56,034][70582] Avg episode reward: [(0, '183.030'), (1, '67.970')] [2023-10-11 20:09:56,046][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000027072_27721728.pth... [2023-10-11 20:09:56,046][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000027104_27754496.pth... [2023-10-11 20:09:56,075][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000025376_25985024.pth [2023-10-11 20:09:56,081][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000025408_26017792.pth [2023-10-11 20:09:56,802][71635] Updated weights for policy 1, policy_version 27082 (0.0009) [2023-10-11 20:09:57,178][71635] Updated weights for policy 1, policy_version 27092 (0.0010) [2023-10-11 20:09:57,542][71635] Updated weights for policy 1, policy_version 27102 (0.0010) [2023-10-11 20:09:57,795][71601] Updated weights for policy 0, policy_version 27110 (0.0009) [2023-10-11 20:09:58,172][71601] Updated weights for policy 0, policy_version 27120 (0.0009) [2023-10-11 20:09:58,549][71601] Updated weights for policy 0, policy_version 27130 (0.0008) [2023-10-11 20:10:01,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 55541760. Throughput: 0: 1814.2, 1: 1825.6. Samples: 13890842. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-10-11 20:10:01,035][70582] Avg episode reward: [(0, '188.020'), (1, '67.070')] [2023-10-11 20:10:01,250][71635] Updated weights for policy 1, policy_version 27112 (0.0007) [2023-10-11 20:10:01,612][71635] Updated weights for policy 1, policy_version 27122 (0.0007) [2023-10-11 20:10:01,980][71635] Updated weights for policy 1, policy_version 27132 (0.0007) [2023-10-11 20:10:02,203][71601] Updated weights for policy 0, policy_version 27140 (0.0007) [2023-10-11 20:10:02,564][71601] Updated weights for policy 0, policy_version 27150 (0.0011) [2023-10-11 20:10:02,935][71601] Updated weights for policy 0, policy_version 27160 (0.0011) [2023-10-11 20:10:05,653][71635] Updated weights for policy 1, policy_version 27142 (0.0008) [2023-10-11 20:10:06,011][71635] Updated weights for policy 1, policy_version 27152 (0.0007) [2023-10-11 20:10:06,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 55607296. Throughput: 0: 1810.0, 1: 1833.7. Samples: 13913220. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-10-11 20:10:06,035][70582] Avg episode reward: [(0, '222.850'), (1, '71.430')] [2023-10-11 20:10:06,036][71353] Saving new best policy, reward=222.850! [2023-10-11 20:10:06,386][71635] Updated weights for policy 1, policy_version 27162 (0.0009) [2023-10-11 20:10:06,557][71601] Updated weights for policy 0, policy_version 27170 (0.0010) [2023-10-11 20:10:06,936][71601] Updated weights for policy 0, policy_version 27180 (0.0010) [2023-10-11 20:10:07,310][71601] Updated weights for policy 0, policy_version 27190 (0.0008) [2023-10-11 20:10:07,686][71601] Updated weights for policy 0, policy_version 27200 (0.0010) [2023-10-11 20:10:10,055][71635] Updated weights for policy 1, policy_version 27172 (0.0009) [2023-10-11 20:10:10,427][71635] Updated weights for policy 1, policy_version 27182 (0.0008) [2023-10-11 20:10:10,794][71635] Updated weights for policy 1, policy_version 27192 (0.0009) [2023-10-11 20:10:11,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.6, 300 sec: 14440.1). Total num frames: 55672832. Throughput: 0: 1816.9, 1: 1826.9. Samples: 13935758. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:10:11,034][70582] Avg episode reward: [(0, '223.030'), (1, '71.430')] [2023-10-11 20:10:11,256][71601] Updated weights for policy 0, policy_version 27210 (0.0009) [2023-10-11 20:10:11,618][71601] Updated weights for policy 0, policy_version 27220 (0.0009) [2023-10-11 20:10:11,987][71601] Updated weights for policy 0, policy_version 27230 (0.0008) [2023-10-11 20:10:12,057][71353] Saving new best policy, reward=223.030! [2023-10-11 20:10:14,301][71635] Updated weights for policy 1, policy_version 27202 (0.0009) [2023-10-11 20:10:14,672][71635] Updated weights for policy 1, policy_version 27212 (0.0009) [2023-10-11 20:10:15,039][71635] Updated weights for policy 1, policy_version 27222 (0.0010) [2023-10-11 20:10:15,403][71635] Updated weights for policy 1, policy_version 27232 (0.0008) [2023-10-11 20:10:15,634][71601] Updated weights for policy 0, policy_version 27240 (0.0008) [2023-10-11 20:10:15,997][71601] Updated weights for policy 0, policy_version 27250 (0.0011) [2023-10-11 20:10:16,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 55771136. Throughput: 0: 1817.8, 1: 1835.1. Samples: 13946402. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:10:16,034][70582] Avg episode reward: [(0, '234.120'), (1, '71.430')] [2023-10-11 20:10:16,362][71601] Updated weights for policy 0, policy_version 27260 (0.0008) [2023-10-11 20:10:16,511][71353] Saving new best policy, reward=234.120! [2023-10-11 20:10:19,103][71635] Updated weights for policy 1, policy_version 27242 (0.0010) [2023-10-11 20:10:19,473][71635] Updated weights for policy 1, policy_version 27252 (0.0007) [2023-10-11 20:10:19,840][71635] Updated weights for policy 1, policy_version 27262 (0.0007) [2023-10-11 20:10:20,306][71601] Updated weights for policy 0, policy_version 27270 (0.0011) [2023-10-11 20:10:20,670][71601] Updated weights for policy 0, policy_version 27280 (0.0011) [2023-10-11 20:10:21,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 55836672. Throughput: 0: 1815.2, 1: 1825.7. Samples: 13968540. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:10:21,035][70582] Avg episode reward: [(0, '228.630'), (1, '73.600')] [2023-10-11 20:10:21,048][71601] Updated weights for policy 0, policy_version 27290 (0.0009) [2023-10-11 20:10:23,624][71635] Updated weights for policy 1, policy_version 27272 (0.0007) [2023-10-11 20:10:23,982][71635] Updated weights for policy 1, policy_version 27282 (0.0009) [2023-10-11 20:10:24,354][71635] Updated weights for policy 1, policy_version 27292 (0.0009) [2023-10-11 20:10:24,713][71601] Updated weights for policy 0, policy_version 27300 (0.0009) [2023-10-11 20:10:25,086][71601] Updated weights for policy 0, policy_version 27310 (0.0008) [2023-10-11 20:10:25,448][71601] Updated weights for policy 0, policy_version 27320 (0.0008) [2023-10-11 20:10:26,034][70582] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 55934976. Throughput: 0: 1823.9, 1: 1834.5. Samples: 13989564. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:10:26,035][70582] Avg episode reward: [(0, '222.840'), (1, '75.460')] [2023-10-11 20:10:28,090][71635] Updated weights for policy 1, policy_version 27302 (0.0008) [2023-10-11 20:10:28,454][71635] Updated weights for policy 1, policy_version 27312 (0.0008) [2023-10-11 20:10:28,819][71635] Updated weights for policy 1, policy_version 27322 (0.0008) [2023-10-11 20:10:28,962][71601] Updated weights for policy 0, policy_version 27330 (0.0008) [2023-10-11 20:10:29,330][71601] Updated weights for policy 0, policy_version 27340 (0.0007) [2023-10-11 20:10:29,697][71601] Updated weights for policy 0, policy_version 27350 (0.0008) [2023-10-11 20:10:30,070][71601] Updated weights for policy 0, policy_version 27360 (0.0008) [2023-10-11 20:10:31,034][70582] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 56000512. Throughput: 0: 1820.8, 1: 1825.8. Samples: 14001400. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:10:31,034][70582] Avg episode reward: [(0, '215.090'), (1, '80.570')] [2023-10-11 20:10:31,035][71431] Saving new best policy, reward=80.570! [2023-10-11 20:10:32,724][71635] Updated weights for policy 1, policy_version 27332 (0.0009) [2023-10-11 20:10:33,085][71635] Updated weights for policy 1, policy_version 27342 (0.0008) [2023-10-11 20:10:33,449][71635] Updated weights for policy 1, policy_version 27352 (0.0008) [2023-10-11 20:10:33,796][71601] Updated weights for policy 0, policy_version 27370 (0.0007) [2023-10-11 20:10:34,173][71601] Updated weights for policy 0, policy_version 27380 (0.0009) [2023-10-11 20:10:34,539][71601] Updated weights for policy 0, policy_version 27390 (0.0009) [2023-10-11 20:10:36,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 56066048. Throughput: 0: 1823.3, 1: 1828.2. Samples: 14022168. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:10:36,034][70582] Avg episode reward: [(0, '200.480'), (1, '80.900')] [2023-10-11 20:10:36,035][71431] Saving new best policy, reward=80.900! [2023-10-11 20:10:37,152][71635] Updated weights for policy 1, policy_version 27362 (0.0007) [2023-10-11 20:10:37,522][71635] Updated weights for policy 1, policy_version 27372 (0.0008) [2023-10-11 20:10:37,898][71635] Updated weights for policy 1, policy_version 27382 (0.0007) [2023-10-11 20:10:38,271][71635] Updated weights for policy 1, policy_version 27392 (0.0007) [2023-10-11 20:10:38,316][71601] Updated weights for policy 0, policy_version 27400 (0.0008) [2023-10-11 20:10:38,679][71601] Updated weights for policy 0, policy_version 27410 (0.0010) [2023-10-11 20:10:39,054][71601] Updated weights for policy 0, policy_version 27420 (0.0009) [2023-10-11 20:10:41,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 56131584. Throughput: 0: 1819.9, 1: 1819.1. Samples: 14044252. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:10:41,034][70582] Avg episode reward: [(0, '197.080'), (1, '83.540')] [2023-10-11 20:10:41,045][71431] Saving new best policy, reward=83.540! [2023-10-11 20:10:42,106][71635] Updated weights for policy 1, policy_version 27402 (0.0008) [2023-10-11 20:10:42,455][71635] Updated weights for policy 1, policy_version 27412 (0.0008) [2023-10-11 20:10:42,784][71601] Updated weights for policy 0, policy_version 27430 (0.0009) [2023-10-11 20:10:42,817][71635] Updated weights for policy 1, policy_version 27422 (0.0009) [2023-10-11 20:10:43,177][71601] Updated weights for policy 0, policy_version 27440 (0.0009) [2023-10-11 20:10:43,562][71601] Updated weights for policy 0, policy_version 27450 (0.0007) [2023-10-11 20:10:46,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 56197120. Throughput: 0: 1818.7, 1: 1814.7. Samples: 14054344. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:10:46,035][70582] Avg episode reward: [(0, '201.360'), (1, '83.750')] [2023-10-11 20:10:46,037][71431] Saving new best policy, reward=83.750! [2023-10-11 20:10:46,587][71635] Updated weights for policy 1, policy_version 27432 (0.0009) [2023-10-11 20:10:46,974][71635] Updated weights for policy 1, policy_version 27442 (0.0009) [2023-10-11 20:10:47,258][71601] Updated weights for policy 0, policy_version 27460 (0.0008) [2023-10-11 20:10:47,333][71635] Updated weights for policy 1, policy_version 27452 (0.0009) [2023-10-11 20:10:47,625][71601] Updated weights for policy 0, policy_version 27470 (0.0010) [2023-10-11 20:10:47,998][71601] Updated weights for policy 0, policy_version 27480 (0.0008) [2023-10-11 20:10:51,008][71635] Updated weights for policy 1, policy_version 27462 (0.0008) [2023-10-11 20:10:51,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 56262656. Throughput: 0: 1813.6, 1: 1814.4. Samples: 14076478. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:10:51,035][70582] Avg episode reward: [(0, '210.010'), (1, '83.540')] [2023-10-11 20:10:51,374][71635] Updated weights for policy 1, policy_version 27472 (0.0008) [2023-10-11 20:10:51,678][71601] Updated weights for policy 0, policy_version 27490 (0.0008) [2023-10-11 20:10:51,743][71635] Updated weights for policy 1, policy_version 27482 (0.0008) [2023-10-11 20:10:52,048][71601] Updated weights for policy 0, policy_version 27500 (0.0008) [2023-10-11 20:10:52,409][71601] Updated weights for policy 0, policy_version 27510 (0.0010) [2023-10-11 20:10:52,782][71601] Updated weights for policy 0, policy_version 27520 (0.0009) [2023-10-11 20:10:55,483][71635] Updated weights for policy 1, policy_version 27492 (0.0008) [2023-10-11 20:10:55,858][71635] Updated weights for policy 1, policy_version 27502 (0.0009) [2023-10-11 20:10:56,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 56328192. Throughput: 0: 1801.6, 1: 1824.1. Samples: 14098914. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-11 20:10:56,034][70582] Avg episode reward: [(0, '197.610'), (1, '90.510')] [2023-10-11 20:10:56,229][71635] Updated weights for policy 1, policy_version 27512 (0.0008) [2023-10-11 20:10:56,524][71431] Saving new best policy, reward=90.510! [2023-10-11 20:10:56,560][71601] Updated weights for policy 0, policy_version 27530 (0.0009) [2023-10-11 20:10:56,928][71601] Updated weights for policy 0, policy_version 27540 (0.0009) [2023-10-11 20:10:57,306][71601] Updated weights for policy 0, policy_version 27550 (0.0007) [2023-10-11 20:10:59,832][71635] Updated weights for policy 1, policy_version 27522 (0.0009) [2023-10-11 20:11:00,197][71635] Updated weights for policy 1, policy_version 27532 (0.0007) [2023-10-11 20:11:00,567][71635] Updated weights for policy 1, policy_version 27542 (0.0008) [2023-10-11 20:11:00,931][71635] Updated weights for policy 1, policy_version 27552 (0.0008) [2023-10-11 20:11:01,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 56426496. Throughput: 0: 1803.3, 1: 1808.6. Samples: 14108938. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-11 20:11:01,035][70582] Avg episode reward: [(0, '194.840'), (1, '89.700')] [2023-10-11 20:11:01,059][71601] Updated weights for policy 0, policy_version 27560 (0.0008) [2023-10-11 20:11:01,437][71601] Updated weights for policy 0, policy_version 27570 (0.0008) [2023-10-11 20:11:01,812][71601] Updated weights for policy 0, policy_version 27580 (0.0008) [2023-10-11 20:11:04,665][71635] Updated weights for policy 1, policy_version 27562 (0.0007) [2023-10-11 20:11:05,037][71635] Updated weights for policy 1, policy_version 27572 (0.0007) [2023-10-11 20:11:05,401][71635] Updated weights for policy 1, policy_version 27582 (0.0007) [2023-10-11 20:11:05,558][71601] Updated weights for policy 0, policy_version 27590 (0.0008) [2023-10-11 20:11:05,929][71601] Updated weights for policy 0, policy_version 27600 (0.0007) [2023-10-11 20:11:06,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 56492032. Throughput: 0: 1805.8, 1: 1817.1. Samples: 14131572. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-11 20:11:06,034][70582] Avg episode reward: [(0, '168.670'), (1, '102.670')] [2023-10-11 20:11:06,035][71431] Saving new best policy, reward=102.670! [2023-10-11 20:11:06,300][71601] Updated weights for policy 0, policy_version 27610 (0.0007) [2023-10-11 20:11:09,155][71635] Updated weights for policy 1, policy_version 27592 (0.0007) [2023-10-11 20:11:09,516][71635] Updated weights for policy 1, policy_version 27602 (0.0010) [2023-10-11 20:11:09,886][71635] Updated weights for policy 1, policy_version 27612 (0.0008) [2023-10-11 20:11:09,894][71601] Updated weights for policy 0, policy_version 27620 (0.0008) [2023-10-11 20:11:10,264][71601] Updated weights for policy 0, policy_version 27630 (0.0007) [2023-10-11 20:11:10,630][71601] Updated weights for policy 0, policy_version 27640 (0.0007) [2023-10-11 20:11:11,034][70582] Fps is (10 sec: 16384.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 56590336. Throughput: 0: 1809.8, 1: 1805.4. Samples: 14152248. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-11 20:11:11,034][70582] Avg episode reward: [(0, '171.390'), (1, '101.800')] [2023-10-11 20:11:13,651][71635] Updated weights for policy 1, policy_version 27622 (0.0010) [2023-10-11 20:11:14,022][71635] Updated weights for policy 1, policy_version 27632 (0.0007) [2023-10-11 20:11:14,384][71635] Updated weights for policy 1, policy_version 27642 (0.0009) [2023-10-11 20:11:14,401][71601] Updated weights for policy 0, policy_version 27650 (0.0007) [2023-10-11 20:11:14,766][71601] Updated weights for policy 0, policy_version 27660 (0.0009) [2023-10-11 20:11:15,140][71601] Updated weights for policy 0, policy_version 27670 (0.0009) [2023-10-11 20:11:15,507][71601] Updated weights for policy 0, policy_version 27680 (0.0010) [2023-10-11 20:11:16,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 56655872. Throughput: 0: 1804.3, 1: 1817.5. Samples: 14164380. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-11 20:11:16,035][70582] Avg episode reward: [(0, '162.530'), (1, '105.870')] [2023-10-11 20:11:16,036][71431] Saving new best policy, reward=105.870! [2023-10-11 20:11:18,000][71635] Updated weights for policy 1, policy_version 27652 (0.0007) [2023-10-11 20:11:18,363][71635] Updated weights for policy 1, policy_version 27662 (0.0008) [2023-10-11 20:11:18,741][71635] Updated weights for policy 1, policy_version 27672 (0.0008) [2023-10-11 20:11:19,420][71601] Updated weights for policy 0, policy_version 27690 (0.0010) [2023-10-11 20:11:19,787][71601] Updated weights for policy 0, policy_version 27700 (0.0010) [2023-10-11 20:11:20,157][71601] Updated weights for policy 0, policy_version 27710 (0.0008) [2023-10-11 20:11:21,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 56721408. Throughput: 0: 1810.6, 1: 1805.5. Samples: 14184892. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-11 20:11:21,034][70582] Avg episode reward: [(0, '157.940'), (1, '111.410')] [2023-10-11 20:11:21,035][71431] Saving new best policy, reward=111.410! [2023-10-11 20:11:22,540][71635] Updated weights for policy 1, policy_version 27682 (0.0009) [2023-10-11 20:11:22,907][71635] Updated weights for policy 1, policy_version 27692 (0.0007) [2023-10-11 20:11:23,274][71635] Updated weights for policy 1, policy_version 27702 (0.0010) [2023-10-11 20:11:23,645][71635] Updated weights for policy 1, policy_version 27712 (0.0008) [2023-10-11 20:11:23,818][71601] Updated weights for policy 0, policy_version 27720 (0.0010) [2023-10-11 20:11:24,190][71601] Updated weights for policy 0, policy_version 27730 (0.0008) [2023-10-11 20:11:24,565][71601] Updated weights for policy 0, policy_version 27740 (0.0008) [2023-10-11 20:11:26,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 56786944. Throughput: 0: 1802.5, 1: 1805.6. Samples: 14206618. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-11 20:11:26,035][70582] Avg episode reward: [(0, '143.820'), (1, '114.210')] [2023-10-11 20:11:26,045][71431] Saving new best policy, reward=114.210! [2023-10-11 20:11:27,355][71635] Updated weights for policy 1, policy_version 27722 (0.0010) [2023-10-11 20:11:27,707][71635] Updated weights for policy 1, policy_version 27732 (0.0009) [2023-10-11 20:11:28,068][71635] Updated weights for policy 1, policy_version 27742 (0.0007) [2023-10-11 20:11:28,231][71601] Updated weights for policy 0, policy_version 27750 (0.0007) [2023-10-11 20:11:28,608][71601] Updated weights for policy 0, policy_version 27760 (0.0008) [2023-10-11 20:11:28,977][71601] Updated weights for policy 0, policy_version 27770 (0.0008) [2023-10-11 20:11:31,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 56852480. Throughput: 0: 1815.1, 1: 1807.5. Samples: 14217360. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-11 20:11:31,034][70582] Avg episode reward: [(0, '123.320'), (1, '110.890')] [2023-10-11 20:11:31,811][71635] Updated weights for policy 1, policy_version 27752 (0.0007) [2023-10-11 20:11:32,179][71635] Updated weights for policy 1, policy_version 27762 (0.0007) [2023-10-11 20:11:32,545][71635] Updated weights for policy 1, policy_version 27772 (0.0007) [2023-10-11 20:11:32,639][71601] Updated weights for policy 0, policy_version 27780 (0.0009) [2023-10-11 20:11:33,009][71601] Updated weights for policy 0, policy_version 27790 (0.0007) [2023-10-11 20:11:33,385][71601] Updated weights for policy 0, policy_version 27800 (0.0007) [2023-10-11 20:11:36,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 56918016. Throughput: 0: 1806.5, 1: 1808.3. Samples: 14239142. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-11 20:11:36,035][70582] Avg episode reward: [(0, '125.200'), (1, '111.160')] [2023-10-11 20:11:36,288][71635] Updated weights for policy 1, policy_version 27782 (0.0009) [2023-10-11 20:11:36,655][71635] Updated weights for policy 1, policy_version 27792 (0.0009) [2023-10-11 20:11:37,015][71601] Updated weights for policy 0, policy_version 27810 (0.0009) [2023-10-11 20:11:37,023][71635] Updated weights for policy 1, policy_version 27802 (0.0008) [2023-10-11 20:11:37,385][71601] Updated weights for policy 0, policy_version 27820 (0.0010) [2023-10-11 20:11:37,758][71601] Updated weights for policy 0, policy_version 27830 (0.0007) [2023-10-11 20:11:38,132][71601] Updated weights for policy 0, policy_version 27840 (0.0007) [2023-10-11 20:11:40,741][71635] Updated weights for policy 1, policy_version 27812 (0.0007) [2023-10-11 20:11:41,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 56983552. Throughput: 0: 1814.6, 1: 1806.0. Samples: 14261844. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-11 20:11:41,035][70582] Avg episode reward: [(0, '110.930'), (1, '118.070')] [2023-10-11 20:11:41,114][71635] Updated weights for policy 1, policy_version 27822 (0.0010) [2023-10-11 20:11:41,476][71635] Updated weights for policy 1, policy_version 27832 (0.0007) [2023-10-11 20:11:41,766][71431] Saving new best policy, reward=118.070! [2023-10-11 20:11:41,803][71601] Updated weights for policy 0, policy_version 27850 (0.0007) [2023-10-11 20:11:42,173][71601] Updated weights for policy 0, policy_version 27860 (0.0008) [2023-10-11 20:11:42,554][71601] Updated weights for policy 0, policy_version 27870 (0.0009) [2023-10-11 20:11:45,172][71635] Updated weights for policy 1, policy_version 27842 (0.0008) [2023-10-11 20:11:45,552][71635] Updated weights for policy 1, policy_version 27852 (0.0007) [2023-10-11 20:11:45,913][71635] Updated weights for policy 1, policy_version 27862 (0.0007) [2023-10-11 20:11:46,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 57049088. Throughput: 0: 1814.6, 1: 1805.0. Samples: 14271822. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-11 20:11:46,034][70582] Avg episode reward: [(0, '103.450'), (1, '113.910')] [2023-10-11 20:11:46,139][71601] Updated weights for policy 0, policy_version 27880 (0.0007) [2023-10-11 20:11:46,265][71635] Updated weights for policy 1, policy_version 27872 (0.0007) [2023-10-11 20:11:46,506][71601] Updated weights for policy 0, policy_version 27890 (0.0007) [2023-10-11 20:11:46,872][71601] Updated weights for policy 0, policy_version 27900 (0.0009) [2023-10-11 20:11:49,847][71635] Updated weights for policy 1, policy_version 27882 (0.0009) [2023-10-11 20:11:50,213][71635] Updated weights for policy 1, policy_version 27892 (0.0009) [2023-10-11 20:11:50,564][71601] Updated weights for policy 0, policy_version 27910 (0.0008) [2023-10-11 20:11:50,576][71635] Updated weights for policy 1, policy_version 27902 (0.0007) [2023-10-11 20:11:50,940][71601] Updated weights for policy 0, policy_version 27920 (0.0009) [2023-10-11 20:11:51,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 57147392. Throughput: 0: 1814.8, 1: 1806.9. Samples: 14294550. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-11 20:11:51,034][70582] Avg episode reward: [(0, '103.450'), (1, '119.990')] [2023-10-11 20:11:51,035][71431] Saving new best policy, reward=119.990! [2023-10-11 20:11:51,316][71601] Updated weights for policy 0, policy_version 27930 (0.0008) [2023-10-11 20:11:54,079][71635] Updated weights for policy 1, policy_version 27912 (0.0008) [2023-10-11 20:11:54,435][71635] Updated weights for policy 1, policy_version 27922 (0.0008) [2023-10-11 20:11:54,804][71635] Updated weights for policy 1, policy_version 27932 (0.0008) [2023-10-11 20:11:55,040][71601] Updated weights for policy 0, policy_version 27940 (0.0009) [2023-10-11 20:11:55,416][71601] Updated weights for policy 0, policy_version 27950 (0.0008) [2023-10-11 20:11:55,781][71601] Updated weights for policy 0, policy_version 27960 (0.0008) [2023-10-11 20:11:56,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 57212928. Throughput: 0: 1820.8, 1: 1806.5. Samples: 14315478. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-11 20:11:56,035][70582] Avg episode reward: [(0, '103.450'), (1, '136.240')] [2023-10-11 20:11:56,044][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000027936_28606464.pth... [2023-10-11 20:11:56,075][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000027968_28639232.pth... [2023-10-11 20:11:56,084][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000026240_26869760.pth [2023-10-11 20:11:56,089][71431] Saving new best policy, reward=136.240! [2023-10-11 20:11:56,113][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000026272_26902528.pth [2023-10-11 20:11:58,450][71635] Updated weights for policy 1, policy_version 27942 (0.0007) [2023-10-11 20:11:58,821][71635] Updated weights for policy 1, policy_version 27952 (0.0009) [2023-10-11 20:11:59,192][71635] Updated weights for policy 1, policy_version 27962 (0.0010) [2023-10-11 20:11:59,360][71601] Updated weights for policy 0, policy_version 27970 (0.0008) [2023-10-11 20:11:59,727][71601] Updated weights for policy 0, policy_version 27980 (0.0010) [2023-10-11 20:12:00,102][71601] Updated weights for policy 0, policy_version 27990 (0.0010) [2023-10-11 20:12:00,471][71601] Updated weights for policy 0, policy_version 28000 (0.0008) [2023-10-11 20:12:01,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 57311232. Throughput: 0: 1816.7, 1: 1808.0. Samples: 14327488. Policy #0 lag: (min: 2.0, avg: 9.9, max: 34.0) [2023-10-11 20:12:01,035][70582] Avg episode reward: [(0, '103.450'), (1, '130.350')] [2023-10-11 20:12:02,919][71635] Updated weights for policy 1, policy_version 27972 (0.0009) [2023-10-11 20:12:03,281][71635] Updated weights for policy 1, policy_version 27982 (0.0007) [2023-10-11 20:12:03,644][71635] Updated weights for policy 1, policy_version 27992 (0.0007) [2023-10-11 20:12:04,403][71601] Updated weights for policy 0, policy_version 28010 (0.0008) [2023-10-11 20:12:04,768][71601] Updated weights for policy 0, policy_version 28020 (0.0008) [2023-10-11 20:12:05,130][71601] Updated weights for policy 0, policy_version 28030 (0.0007) [2023-10-11 20:12:06,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 57376768. Throughput: 0: 1814.0, 1: 1816.7. Samples: 14348278. Policy #0 lag: (min: 2.0, avg: 9.9, max: 34.0) [2023-10-11 20:12:06,035][70582] Avg episode reward: [(0, '103.450'), (1, '125.610')] [2023-10-11 20:12:07,271][71635] Updated weights for policy 1, policy_version 28002 (0.0009) [2023-10-11 20:12:07,641][71635] Updated weights for policy 1, policy_version 28012 (0.0007) [2023-10-11 20:12:08,012][71635] Updated weights for policy 1, policy_version 28022 (0.0008) [2023-10-11 20:12:08,372][71635] Updated weights for policy 1, policy_version 28032 (0.0007) [2023-10-11 20:12:08,573][71601] Updated weights for policy 0, policy_version 28040 (0.0007) [2023-10-11 20:12:08,950][71601] Updated weights for policy 0, policy_version 28050 (0.0007) [2023-10-11 20:12:09,317][71601] Updated weights for policy 0, policy_version 28060 (0.0010) [2023-10-11 20:12:11,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 57442304. Throughput: 0: 1816.0, 1: 1824.5. Samples: 14370440. Policy #0 lag: (min: 2.0, avg: 9.9, max: 34.0) [2023-10-11 20:12:11,034][70582] Avg episode reward: [(0, '98.720'), (1, '125.010')] [2023-10-11 20:12:12,202][71635] Updated weights for policy 1, policy_version 28042 (0.0008) [2023-10-11 20:12:12,571][71635] Updated weights for policy 1, policy_version 28052 (0.0008) [2023-10-11 20:12:12,936][71635] Updated weights for policy 1, policy_version 28062 (0.0007) [2023-10-11 20:12:13,005][71601] Updated weights for policy 0, policy_version 28070 (0.0010) [2023-10-11 20:12:13,389][71601] Updated weights for policy 0, policy_version 28080 (0.0009) [2023-10-11 20:12:13,768][71601] Updated weights for policy 0, policy_version 28090 (0.0010) [2023-10-11 20:12:16,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 57507840. Throughput: 0: 1812.3, 1: 1824.2. Samples: 14381004. Policy #0 lag: (min: 2.0, avg: 9.9, max: 34.0) [2023-10-11 20:12:16,034][70582] Avg episode reward: [(0, '98.020'), (1, '129.430')] [2023-10-11 20:12:16,609][71635] Updated weights for policy 1, policy_version 28072 (0.0008) [2023-10-11 20:12:16,977][71635] Updated weights for policy 1, policy_version 28082 (0.0009) [2023-10-11 20:12:17,342][71635] Updated weights for policy 1, policy_version 28092 (0.0007) [2023-10-11 20:12:17,638][71601] Updated weights for policy 0, policy_version 28100 (0.0010) [2023-10-11 20:12:18,013][71601] Updated weights for policy 0, policy_version 28110 (0.0008) [2023-10-11 20:12:18,383][71601] Updated weights for policy 0, policy_version 28120 (0.0008) [2023-10-11 20:12:21,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 57573376. Throughput: 0: 1817.5, 1: 1828.5. Samples: 14403212. Policy #0 lag: (min: 2.0, avg: 9.9, max: 34.0) [2023-10-11 20:12:21,034][70582] Avg episode reward: [(0, '103.970'), (1, '126.760')] [2023-10-11 20:12:21,223][71635] Updated weights for policy 1, policy_version 28102 (0.0009) [2023-10-11 20:12:21,594][71635] Updated weights for policy 1, policy_version 28112 (0.0010) [2023-10-11 20:12:21,966][71635] Updated weights for policy 1, policy_version 28122 (0.0009) [2023-10-11 20:12:22,193][71601] Updated weights for policy 0, policy_version 28130 (0.0009) [2023-10-11 20:12:22,564][71601] Updated weights for policy 0, policy_version 28140 (0.0009) [2023-10-11 20:12:22,932][71601] Updated weights for policy 0, policy_version 28150 (0.0009) [2023-10-11 20:12:23,308][71601] Updated weights for policy 0, policy_version 28160 (0.0007) [2023-10-11 20:12:25,717][71635] Updated weights for policy 1, policy_version 28132 (0.0008) [2023-10-11 20:12:26,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 57638912. Throughput: 0: 1815.9, 1: 1825.3. Samples: 14425700. Policy #0 lag: (min: 1.0, avg: 9.5, max: 33.0) [2023-10-11 20:12:26,034][70582] Avg episode reward: [(0, '119.840'), (1, '131.230')] [2023-10-11 20:12:26,088][71635] Updated weights for policy 1, policy_version 28142 (0.0009) [2023-10-11 20:12:26,445][71635] Updated weights for policy 1, policy_version 28152 (0.0008) [2023-10-11 20:12:27,123][71601] Updated weights for policy 0, policy_version 28170 (0.0008) [2023-10-11 20:12:27,490][71601] Updated weights for policy 0, policy_version 28180 (0.0009) [2023-10-11 20:12:27,867][71601] Updated weights for policy 0, policy_version 28190 (0.0007) [2023-10-11 20:12:30,203][71635] Updated weights for policy 1, policy_version 28162 (0.0008) [2023-10-11 20:12:30,569][71635] Updated weights for policy 1, policy_version 28172 (0.0009) [2023-10-11 20:12:30,944][71635] Updated weights for policy 1, policy_version 28182 (0.0007) [2023-10-11 20:12:31,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 57704448. Throughput: 0: 1815.1, 1: 1825.2. Samples: 14435636. Policy #0 lag: (min: 1.0, avg: 9.5, max: 33.0) [2023-10-11 20:12:31,034][70582] Avg episode reward: [(0, '119.780'), (1, '131.010')] [2023-10-11 20:12:31,307][71635] Updated weights for policy 1, policy_version 28192 (0.0007) [2023-10-11 20:12:31,454][71601] Updated weights for policy 0, policy_version 28200 (0.0007) [2023-10-11 20:12:31,830][71601] Updated weights for policy 0, policy_version 28210 (0.0007) [2023-10-11 20:12:32,202][71601] Updated weights for policy 0, policy_version 28220 (0.0008) [2023-10-11 20:12:35,159][71635] Updated weights for policy 1, policy_version 28202 (0.0009) [2023-10-11 20:12:35,523][71635] Updated weights for policy 1, policy_version 28212 (0.0008) [2023-10-11 20:12:35,886][71635] Updated weights for policy 1, policy_version 28222 (0.0007) [2023-10-11 20:12:35,963][71601] Updated weights for policy 0, policy_version 28230 (0.0010) [2023-10-11 20:12:36,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 57802752. Throughput: 0: 1814.4, 1: 1824.5. Samples: 14458302. Policy #0 lag: (min: 1.0, avg: 9.5, max: 33.0) [2023-10-11 20:12:36,034][70582] Avg episode reward: [(0, '122.620'), (1, '131.110')] [2023-10-11 20:12:36,346][71601] Updated weights for policy 0, policy_version 28240 (0.0009) [2023-10-11 20:12:36,714][71601] Updated weights for policy 0, policy_version 28250 (0.0008) [2023-10-11 20:12:39,596][71635] Updated weights for policy 1, policy_version 28232 (0.0007) [2023-10-11 20:12:39,972][71635] Updated weights for policy 1, policy_version 28242 (0.0008) [2023-10-11 20:12:40,309][71601] Updated weights for policy 0, policy_version 28260 (0.0009) [2023-10-11 20:12:40,331][71635] Updated weights for policy 1, policy_version 28252 (0.0010) [2023-10-11 20:12:40,678][71601] Updated weights for policy 0, policy_version 28270 (0.0010) [2023-10-11 20:12:41,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 57868288. Throughput: 0: 1820.6, 1: 1824.7. Samples: 14479516. Policy #0 lag: (min: 1.0, avg: 9.5, max: 33.0) [2023-10-11 20:12:41,035][70582] Avg episode reward: [(0, '121.500'), (1, '134.850')] [2023-10-11 20:12:41,057][71601] Updated weights for policy 0, policy_version 28280 (0.0008) [2023-10-11 20:12:44,000][71635] Updated weights for policy 1, policy_version 28262 (0.0010) [2023-10-11 20:12:44,362][71635] Updated weights for policy 1, policy_version 28272 (0.0009) [2023-10-11 20:12:44,729][71635] Updated weights for policy 1, policy_version 28282 (0.0008) [2023-10-11 20:12:44,824][71601] Updated weights for policy 0, policy_version 28290 (0.0007) [2023-10-11 20:12:45,199][71601] Updated weights for policy 0, policy_version 28300 (0.0009) [2023-10-11 20:12:45,571][71601] Updated weights for policy 0, policy_version 28310 (0.0009) [2023-10-11 20:12:45,940][71601] Updated weights for policy 0, policy_version 28320 (0.0008) [2023-10-11 20:12:46,034][70582] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 57966592. Throughput: 0: 1810.9, 1: 1821.7. Samples: 14490954. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-11 20:12:46,034][70582] Avg episode reward: [(0, '121.600'), (1, '140.620')] [2023-10-11 20:12:46,035][71431] Saving new best policy, reward=140.620! [2023-10-11 20:12:48,378][71635] Updated weights for policy 1, policy_version 28292 (0.0007) [2023-10-11 20:12:48,746][71635] Updated weights for policy 1, policy_version 28302 (0.0008) [2023-10-11 20:12:49,118][71635] Updated weights for policy 1, policy_version 28312 (0.0010) [2023-10-11 20:12:49,572][71601] Updated weights for policy 0, policy_version 28330 (0.0009) [2023-10-11 20:12:49,949][71601] Updated weights for policy 0, policy_version 28340 (0.0008) [2023-10-11 20:12:50,309][71601] Updated weights for policy 0, policy_version 28350 (0.0007) [2023-10-11 20:12:51,034][70582] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 58032128. Throughput: 0: 1821.0, 1: 1815.6. Samples: 14511922. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-11 20:12:51,034][70582] Avg episode reward: [(0, '120.740'), (1, '140.520')] [2023-10-11 20:12:52,856][71635] Updated weights for policy 1, policy_version 28322 (0.0007) [2023-10-11 20:12:53,235][71635] Updated weights for policy 1, policy_version 28332 (0.0010) [2023-10-11 20:12:53,584][71635] Updated weights for policy 1, policy_version 28342 (0.0009) [2023-10-11 20:12:53,946][71635] Updated weights for policy 1, policy_version 28352 (0.0009) [2023-10-11 20:12:54,022][71601] Updated weights for policy 0, policy_version 28360 (0.0008) [2023-10-11 20:12:54,391][71601] Updated weights for policy 0, policy_version 28370 (0.0010) [2023-10-11 20:12:54,758][71601] Updated weights for policy 0, policy_version 28380 (0.0011) [2023-10-11 20:12:56,034][70582] Fps is (10 sec: 13106.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 58097664. Throughput: 0: 1815.2, 1: 1814.6. Samples: 14533782. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-11 20:12:56,035][70582] Avg episode reward: [(0, '122.080'), (1, '137.510')] [2023-10-11 20:12:57,301][71635] Updated weights for policy 1, policy_version 28362 (0.0009) [2023-10-11 20:12:57,658][71635] Updated weights for policy 1, policy_version 28372 (0.0008) [2023-10-11 20:12:58,028][71635] Updated weights for policy 1, policy_version 28382 (0.0009) [2023-10-11 20:12:58,615][71601] Updated weights for policy 0, policy_version 28390 (0.0009) [2023-10-11 20:12:59,001][71601] Updated weights for policy 0, policy_version 28400 (0.0007) [2023-10-11 20:12:59,373][71601] Updated weights for policy 0, policy_version 28410 (0.0009) [2023-10-11 20:13:01,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 58163200. Throughput: 0: 1826.7, 1: 1818.1. Samples: 14545018. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-11 20:13:01,034][70582] Avg episode reward: [(0, '97.980'), (1, '136.510')] [2023-10-11 20:13:01,641][71635] Updated weights for policy 1, policy_version 28392 (0.0007) [2023-10-11 20:13:02,002][71635] Updated weights for policy 1, policy_version 28402 (0.0008) [2023-10-11 20:13:02,372][71635] Updated weights for policy 1, policy_version 28412 (0.0008) [2023-10-11 20:13:03,078][71601] Updated weights for policy 0, policy_version 28420 (0.0009) [2023-10-11 20:13:03,440][71601] Updated weights for policy 0, policy_version 28430 (0.0008) [2023-10-11 20:13:03,812][71601] Updated weights for policy 0, policy_version 28440 (0.0008) [2023-10-11 20:13:06,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 58228736. Throughput: 0: 1807.6, 1: 1816.1. Samples: 14566280. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-11 20:13:06,035][70582] Avg episode reward: [(0, '98.020'), (1, '125.720')] [2023-10-11 20:13:06,061][71635] Updated weights for policy 1, policy_version 28422 (0.0007) [2023-10-11 20:13:06,440][71635] Updated weights for policy 1, policy_version 28432 (0.0007) [2023-10-11 20:13:06,809][71635] Updated weights for policy 1, policy_version 28442 (0.0009) [2023-10-11 20:13:07,500][71601] Updated weights for policy 0, policy_version 28450 (0.0010) [2023-10-11 20:13:07,878][71601] Updated weights for policy 0, policy_version 28460 (0.0008) [2023-10-11 20:13:08,254][71601] Updated weights for policy 0, policy_version 28470 (0.0009) [2023-10-11 20:13:08,627][71601] Updated weights for policy 0, policy_version 28480 (0.0007) [2023-10-11 20:13:10,528][71635] Updated weights for policy 1, policy_version 28452 (0.0008) [2023-10-11 20:13:10,894][71635] Updated weights for policy 1, policy_version 28462 (0.0007) [2023-10-11 20:13:11,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 58294272. Throughput: 0: 1805.8, 1: 1819.6. Samples: 14588846. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) [2023-10-11 20:13:11,034][70582] Avg episode reward: [(0, '106.700'), (1, '120.710')] [2023-10-11 20:13:11,252][71635] Updated weights for policy 1, policy_version 28472 (0.0007) [2023-10-11 20:13:12,108][71601] Updated weights for policy 0, policy_version 28490 (0.0009) [2023-10-11 20:13:12,476][71601] Updated weights for policy 0, policy_version 28500 (0.0010) [2023-10-11 20:13:12,846][71601] Updated weights for policy 0, policy_version 28510 (0.0007) [2023-10-11 20:13:15,003][71635] Updated weights for policy 1, policy_version 28482 (0.0008) [2023-10-11 20:13:15,364][71635] Updated weights for policy 1, policy_version 28492 (0.0008) [2023-10-11 20:13:15,729][71635] Updated weights for policy 1, policy_version 28502 (0.0008) [2023-10-11 20:13:16,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 58359808. Throughput: 0: 1810.0, 1: 1821.4. Samples: 14599050. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) [2023-10-11 20:13:16,034][70582] Avg episode reward: [(0, '106.810'), (1, '119.450')] [2023-10-11 20:13:16,093][71635] Updated weights for policy 1, policy_version 28512 (0.0010) [2023-10-11 20:13:16,405][71601] Updated weights for policy 0, policy_version 28520 (0.0011) [2023-10-11 20:13:16,770][71601] Updated weights for policy 0, policy_version 28530 (0.0011) [2023-10-11 20:13:17,145][71601] Updated weights for policy 0, policy_version 28540 (0.0008) [2023-10-11 20:13:19,777][71635] Updated weights for policy 1, policy_version 28522 (0.0011) [2023-10-11 20:13:20,144][71635] Updated weights for policy 1, policy_version 28532 (0.0008) [2023-10-11 20:13:20,505][71635] Updated weights for policy 1, policy_version 28542 (0.0009) [2023-10-11 20:13:20,871][71601] Updated weights for policy 0, policy_version 28550 (0.0009) [2023-10-11 20:13:21,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 58458112. Throughput: 0: 1813.8, 1: 1828.5. Samples: 14622206. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) [2023-10-11 20:13:21,034][70582] Avg episode reward: [(0, '106.820'), (1, '119.770')] [2023-10-11 20:13:21,230][71601] Updated weights for policy 0, policy_version 28560 (0.0008) [2023-10-11 20:13:21,608][71601] Updated weights for policy 0, policy_version 28570 (0.0008) [2023-10-11 20:13:24,207][71635] Updated weights for policy 1, policy_version 28552 (0.0009) [2023-10-11 20:13:24,579][71635] Updated weights for policy 1, policy_version 28562 (0.0008) [2023-10-11 20:13:24,947][71635] Updated weights for policy 1, policy_version 28572 (0.0007) [2023-10-11 20:13:25,287][71601] Updated weights for policy 0, policy_version 28580 (0.0009) [2023-10-11 20:13:25,654][71601] Updated weights for policy 0, policy_version 28590 (0.0008) [2023-10-11 20:13:26,030][71601] Updated weights for policy 0, policy_version 28600 (0.0008) [2023-10-11 20:13:26,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 58523648. Throughput: 0: 1814.5, 1: 1826.0. Samples: 14643338. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) [2023-10-11 20:13:26,034][70582] Avg episode reward: [(0, '106.820'), (1, '119.140')] [2023-10-11 20:13:28,756][71635] Updated weights for policy 1, policy_version 28582 (0.0008) [2023-10-11 20:13:29,128][71635] Updated weights for policy 1, policy_version 28592 (0.0009) [2023-10-11 20:13:29,498][71635] Updated weights for policy 1, policy_version 28602 (0.0008) [2023-10-11 20:13:29,749][71601] Updated weights for policy 0, policy_version 28610 (0.0008) [2023-10-11 20:13:30,121][71601] Updated weights for policy 0, policy_version 28620 (0.0007) [2023-10-11 20:13:30,490][71601] Updated weights for policy 0, policy_version 28630 (0.0008) [2023-10-11 20:13:30,861][71601] Updated weights for policy 0, policy_version 28640 (0.0007) [2023-10-11 20:13:31,034][70582] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 58621952. Throughput: 0: 1819.0, 1: 1827.7. Samples: 14655054. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-11 20:13:31,034][70582] Avg episode reward: [(0, '109.100'), (1, '122.070')] [2023-10-11 20:13:33,272][71635] Updated weights for policy 1, policy_version 28612 (0.0007) [2023-10-11 20:13:33,646][71635] Updated weights for policy 1, policy_version 28622 (0.0009) [2023-10-11 20:13:34,015][71635] Updated weights for policy 1, policy_version 28632 (0.0008) [2023-10-11 20:13:34,510][71601] Updated weights for policy 0, policy_version 28650 (0.0009) [2023-10-11 20:13:34,879][71601] Updated weights for policy 0, policy_version 28660 (0.0010) [2023-10-11 20:13:35,252][71601] Updated weights for policy 0, policy_version 28670 (0.0007) [2023-10-11 20:13:36,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 58687488. Throughput: 0: 1821.5, 1: 1819.7. Samples: 14675776. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-11 20:13:36,035][70582] Avg episode reward: [(0, '110.390'), (1, '118.610')] [2023-10-11 20:13:37,742][71635] Updated weights for policy 1, policy_version 28642 (0.0007) [2023-10-11 20:13:38,119][71635] Updated weights for policy 1, policy_version 28652 (0.0007) [2023-10-11 20:13:38,486][71635] Updated weights for policy 1, policy_version 28662 (0.0007) [2023-10-11 20:13:38,851][71635] Updated weights for policy 1, policy_version 28672 (0.0008) [2023-10-11 20:13:38,875][71601] Updated weights for policy 0, policy_version 28680 (0.0008) [2023-10-11 20:13:39,237][71601] Updated weights for policy 0, policy_version 28690 (0.0009) [2023-10-11 20:13:39,622][71601] Updated weights for policy 0, policy_version 28700 (0.0008) [2023-10-11 20:13:41,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 58753024. Throughput: 0: 1821.5, 1: 1813.6. Samples: 14697362. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-11 20:13:41,035][70582] Avg episode reward: [(0, '110.820'), (1, '131.870')] [2023-10-11 20:13:42,462][71635] Updated weights for policy 1, policy_version 28682 (0.0008) [2023-10-11 20:13:42,829][71635] Updated weights for policy 1, policy_version 28692 (0.0009) [2023-10-11 20:13:43,198][71635] Updated weights for policy 1, policy_version 28702 (0.0009) [2023-10-11 20:13:43,308][71601] Updated weights for policy 0, policy_version 28710 (0.0009) [2023-10-11 20:13:43,690][71601] Updated weights for policy 0, policy_version 28720 (0.0010) [2023-10-11 20:13:44,061][71601] Updated weights for policy 0, policy_version 28730 (0.0010) [2023-10-11 20:13:46,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 58818560. Throughput: 0: 1815.0, 1: 1813.5. Samples: 14708298. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-11 20:13:46,034][70582] Avg episode reward: [(0, '118.450'), (1, '116.870')] [2023-10-11 20:13:46,953][71635] Updated weights for policy 1, policy_version 28712 (0.0008) [2023-10-11 20:13:47,313][71635] Updated weights for policy 1, policy_version 28722 (0.0007) [2023-10-11 20:13:47,588][71601] Updated weights for policy 0, policy_version 28740 (0.0008) [2023-10-11 20:13:47,672][71635] Updated weights for policy 1, policy_version 28732 (0.0008) [2023-10-11 20:13:47,965][71601] Updated weights for policy 0, policy_version 28750 (0.0008) [2023-10-11 20:13:48,328][71601] Updated weights for policy 0, policy_version 28760 (0.0007) [2023-10-11 20:13:51,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 58884096. Throughput: 0: 1829.8, 1: 1807.6. Samples: 14729962. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-11 20:13:51,034][70582] Avg episode reward: [(0, '118.410'), (1, '111.650')] [2023-10-11 20:13:51,440][71635] Updated weights for policy 1, policy_version 28742 (0.0007) [2023-10-11 20:13:51,832][71635] Updated weights for policy 1, policy_version 28752 (0.0007) [2023-10-11 20:13:52,110][71601] Updated weights for policy 0, policy_version 28770 (0.0008) [2023-10-11 20:13:52,205][71635] Updated weights for policy 1, policy_version 28762 (0.0007) [2023-10-11 20:13:52,480][71601] Updated weights for policy 0, policy_version 28780 (0.0007) [2023-10-11 20:13:52,854][71601] Updated weights for policy 0, policy_version 28790 (0.0007) [2023-10-11 20:13:53,232][71601] Updated weights for policy 0, policy_version 28800 (0.0007) [2023-10-11 20:13:55,835][71635] Updated weights for policy 1, policy_version 28772 (0.0010) [2023-10-11 20:13:56,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 58949632. Throughput: 0: 1839.2, 1: 1808.1. Samples: 14752976. Policy #0 lag: (min: 24.0, avg: 44.6, max: 56.0) [2023-10-11 20:13:56,035][70582] Avg episode reward: [(0, '118.410'), (1, '113.790')] [2023-10-11 20:13:56,044][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000028800_29491200.pth... [2023-10-11 20:13:56,076][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000027104_27754496.pth [2023-10-11 20:13:56,195][71635] Updated weights for policy 1, policy_version 28782 (0.0010) [2023-10-11 20:13:56,559][71635] Updated weights for policy 1, policy_version 28792 (0.0008) [2023-10-11 20:13:56,813][71601] Updated weights for policy 0, policy_version 28810 (0.0009) [2023-10-11 20:13:56,848][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000028800_29491200.pth... [2023-10-11 20:13:56,877][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000027072_27721728.pth [2023-10-11 20:13:57,188][71601] Updated weights for policy 0, policy_version 28820 (0.0011) [2023-10-11 20:13:57,558][71601] Updated weights for policy 0, policy_version 28830 (0.0010) [2023-10-11 20:14:00,283][71635] Updated weights for policy 1, policy_version 28802 (0.0009) [2023-10-11 20:14:00,654][71635] Updated weights for policy 1, policy_version 28812 (0.0007) [2023-10-11 20:14:01,017][71635] Updated weights for policy 1, policy_version 28822 (0.0007) [2023-10-11 20:14:01,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 59015168. Throughput: 0: 1839.5, 1: 1805.8. Samples: 14763086. Policy #0 lag: (min: 24.0, avg: 44.6, max: 56.0) [2023-10-11 20:14:01,035][70582] Avg episode reward: [(0, '118.930'), (1, '108.630')] [2023-10-11 20:14:01,086][71601] Updated weights for policy 0, policy_version 28840 (0.0008) [2023-10-11 20:14:01,376][71635] Updated weights for policy 1, policy_version 28832 (0.0007) [2023-10-11 20:14:01,453][71601] Updated weights for policy 0, policy_version 28850 (0.0008) [2023-10-11 20:14:01,823][71601] Updated weights for policy 0, policy_version 28860 (0.0007) [2023-10-11 20:14:05,027][71635] Updated weights for policy 1, policy_version 28842 (0.0010) [2023-10-11 20:14:05,394][71635] Updated weights for policy 1, policy_version 28852 (0.0009) [2023-10-11 20:14:05,680][71601] Updated weights for policy 0, policy_version 28870 (0.0007) [2023-10-11 20:14:05,754][71635] Updated weights for policy 1, policy_version 28862 (0.0008) [2023-10-11 20:14:06,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 59113472. Throughput: 0: 1835.0, 1: 1805.4. Samples: 14786024. Policy #0 lag: (min: 24.0, avg: 44.6, max: 56.0) [2023-10-11 20:14:06,035][70582] Avg episode reward: [(0, '124.100'), (1, '108.630')] [2023-10-11 20:14:06,045][71601] Updated weights for policy 0, policy_version 28880 (0.0008) [2023-10-11 20:14:06,421][71601] Updated weights for policy 0, policy_version 28890 (0.0010) [2023-10-11 20:14:09,399][71635] Updated weights for policy 1, policy_version 28872 (0.0010) [2023-10-11 20:14:09,758][71635] Updated weights for policy 1, policy_version 28882 (0.0011) [2023-10-11 20:14:10,124][71635] Updated weights for policy 1, policy_version 28892 (0.0009) [2023-10-11 20:14:10,154][71601] Updated weights for policy 0, policy_version 28900 (0.0009) [2023-10-11 20:14:10,529][71601] Updated weights for policy 0, policy_version 28910 (0.0007) [2023-10-11 20:14:10,896][71601] Updated weights for policy 0, policy_version 28920 (0.0008) [2023-10-11 20:14:11,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 59179008. Throughput: 0: 1829.2, 1: 1803.7. Samples: 14806820. Policy #0 lag: (min: 24.0, avg: 44.6, max: 56.0) [2023-10-11 20:14:11,035][70582] Avg episode reward: [(0, '124.130'), (1, '113.560')] [2023-10-11 20:14:13,879][71635] Updated weights for policy 1, policy_version 28902 (0.0010) [2023-10-11 20:14:14,245][71635] Updated weights for policy 1, policy_version 28912 (0.0007) [2023-10-11 20:14:14,400][71601] Updated weights for policy 0, policy_version 28930 (0.0007) [2023-10-11 20:14:14,616][71635] Updated weights for policy 1, policy_version 28922 (0.0009) [2023-10-11 20:14:14,775][71601] Updated weights for policy 0, policy_version 28940 (0.0007) [2023-10-11 20:14:15,145][71601] Updated weights for policy 0, policy_version 28950 (0.0007) [2023-10-11 20:14:15,519][71601] Updated weights for policy 0, policy_version 28960 (0.0007) [2023-10-11 20:14:16,034][70582] Fps is (10 sec: 16384.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 59277312. Throughput: 0: 1838.2, 1: 1806.2. Samples: 14819050. Policy #0 lag: (min: 19.0, avg: 21.7, max: 51.0) [2023-10-11 20:14:16,034][70582] Avg episode reward: [(0, '138.480'), (1, '114.110')] [2023-10-11 20:14:18,299][71635] Updated weights for policy 1, policy_version 28932 (0.0008) [2023-10-11 20:14:18,680][71635] Updated weights for policy 1, policy_version 28942 (0.0010) [2023-10-11 20:14:19,033][71635] Updated weights for policy 1, policy_version 28952 (0.0009) [2023-10-11 20:14:19,245][71601] Updated weights for policy 0, policy_version 28970 (0.0007) [2023-10-11 20:14:19,622][71601] Updated weights for policy 0, policy_version 28980 (0.0007) [2023-10-11 20:14:19,994][71601] Updated weights for policy 0, policy_version 28990 (0.0007) [2023-10-11 20:14:21,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 59342848. Throughput: 0: 1828.8, 1: 1812.0. Samples: 14839610. Policy #0 lag: (min: 19.0, avg: 21.7, max: 51.0) [2023-10-11 20:14:21,034][70582] Avg episode reward: [(0, '137.820'), (1, '107.950')] [2023-10-11 20:14:22,783][71635] Updated weights for policy 1, policy_version 28962 (0.0009) [2023-10-11 20:14:23,159][71635] Updated weights for policy 1, policy_version 28972 (0.0011) [2023-10-11 20:14:23,513][71635] Updated weights for policy 1, policy_version 28982 (0.0007) [2023-10-11 20:14:23,828][71601] Updated weights for policy 0, policy_version 29000 (0.0009) [2023-10-11 20:14:23,882][71635] Updated weights for policy 1, policy_version 28992 (0.0007) [2023-10-11 20:14:24,199][71601] Updated weights for policy 0, policy_version 29010 (0.0011) [2023-10-11 20:14:24,569][71601] Updated weights for policy 0, policy_version 29020 (0.0009) [2023-10-11 20:14:26,034][70582] Fps is (10 sec: 13106.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 59408384. Throughput: 0: 1827.5, 1: 1808.7. Samples: 14860990. Policy #0 lag: (min: 19.0, avg: 21.7, max: 51.0) [2023-10-11 20:14:26,035][70582] Avg episode reward: [(0, '146.140'), (1, '96.210')] [2023-10-11 20:14:27,686][71635] Updated weights for policy 1, policy_version 29002 (0.0007) [2023-10-11 20:14:28,048][71635] Updated weights for policy 1, policy_version 29012 (0.0009) [2023-10-11 20:14:28,301][71601] Updated weights for policy 0, policy_version 29030 (0.0009) [2023-10-11 20:14:28,416][71635] Updated weights for policy 1, policy_version 29022 (0.0007) [2023-10-11 20:14:28,681][71601] Updated weights for policy 0, policy_version 29040 (0.0010) [2023-10-11 20:14:29,056][71601] Updated weights for policy 0, policy_version 29050 (0.0009) [2023-10-11 20:14:31,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 59473920. Throughput: 0: 1828.7, 1: 1811.4. Samples: 14872104. Policy #0 lag: (min: 19.0, avg: 21.7, max: 51.0) [2023-10-11 20:14:31,034][70582] Avg episode reward: [(0, '146.220'), (1, '99.050')] [2023-10-11 20:14:32,140][71635] Updated weights for policy 1, policy_version 29032 (0.0008) [2023-10-11 20:14:32,512][71635] Updated weights for policy 1, policy_version 29042 (0.0008) [2023-10-11 20:14:32,696][71601] Updated weights for policy 0, policy_version 29060 (0.0010) [2023-10-11 20:14:32,883][71635] Updated weights for policy 1, policy_version 29052 (0.0009) [2023-10-11 20:14:33,066][71601] Updated weights for policy 0, policy_version 29070 (0.0008) [2023-10-11 20:14:33,442][71601] Updated weights for policy 0, policy_version 29080 (0.0007) [2023-10-11 20:14:36,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 59539456. Throughput: 0: 1823.7, 1: 1807.1. Samples: 14893348. Policy #0 lag: (min: 19.0, avg: 21.7, max: 51.0) [2023-10-11 20:14:36,035][70582] Avg episode reward: [(0, '160.220'), (1, '104.170')] [2023-10-11 20:14:36,587][71635] Updated weights for policy 1, policy_version 29062 (0.0008) [2023-10-11 20:14:36,950][71635] Updated weights for policy 1, policy_version 29072 (0.0009) [2023-10-11 20:14:37,098][71601] Updated weights for policy 0, policy_version 29090 (0.0007) [2023-10-11 20:14:37,304][71635] Updated weights for policy 1, policy_version 29082 (0.0007) [2023-10-11 20:14:37,465][71601] Updated weights for policy 0, policy_version 29100 (0.0008) [2023-10-11 20:14:37,828][71601] Updated weights for policy 0, policy_version 29110 (0.0010) [2023-10-11 20:14:38,207][71601] Updated weights for policy 0, policy_version 29120 (0.0009) [2023-10-11 20:14:41,011][71635] Updated weights for policy 1, policy_version 29092 (0.0008) [2023-10-11 20:14:41,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 59604992. Throughput: 0: 1817.1, 1: 1812.7. Samples: 14916316. Policy #0 lag: (min: 31.0, avg: 31.9, max: 52.0) [2023-10-11 20:14:41,034][70582] Avg episode reward: [(0, '162.910'), (1, '107.120')] [2023-10-11 20:14:41,380][71635] Updated weights for policy 1, policy_version 29102 (0.0009) [2023-10-11 20:14:41,745][71635] Updated weights for policy 1, policy_version 29112 (0.0008) [2023-10-11 20:14:42,035][71601] Updated weights for policy 0, policy_version 29130 (0.0007) [2023-10-11 20:14:42,407][71601] Updated weights for policy 0, policy_version 29140 (0.0007) [2023-10-11 20:14:42,770][71601] Updated weights for policy 0, policy_version 29150 (0.0009) [2023-10-11 20:14:45,453][71635] Updated weights for policy 1, policy_version 29122 (0.0009) [2023-10-11 20:14:45,830][71635] Updated weights for policy 1, policy_version 29132 (0.0011) [2023-10-11 20:14:46,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 59670528. Throughput: 0: 1809.4, 1: 1813.3. Samples: 14926106. Policy #0 lag: (min: 31.0, avg: 31.9, max: 52.0) [2023-10-11 20:14:46,034][70582] Avg episode reward: [(0, '162.430'), (1, '112.730')] [2023-10-11 20:14:46,209][71635] Updated weights for policy 1, policy_version 29142 (0.0010) [2023-10-11 20:14:46,545][71601] Updated weights for policy 0, policy_version 29160 (0.0008) [2023-10-11 20:14:46,568][71635] Updated weights for policy 1, policy_version 29152 (0.0008) [2023-10-11 20:14:46,915][71601] Updated weights for policy 0, policy_version 29170 (0.0010) [2023-10-11 20:14:47,290][71601] Updated weights for policy 0, policy_version 29180 (0.0009) [2023-10-11 20:14:50,306][71635] Updated weights for policy 1, policy_version 29162 (0.0010) [2023-10-11 20:14:50,671][71635] Updated weights for policy 1, policy_version 29172 (0.0007) [2023-10-11 20:14:50,913][71601] Updated weights for policy 0, policy_version 29190 (0.0008) [2023-10-11 20:14:51,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 59736064. Throughput: 0: 1804.1, 1: 1809.7. Samples: 14948646. Policy #0 lag: (min: 31.0, avg: 31.9, max: 52.0) [2023-10-11 20:14:51,034][70582] Avg episode reward: [(0, '164.130'), (1, '110.370')] [2023-10-11 20:14:51,045][71635] Updated weights for policy 1, policy_version 29182 (0.0008) [2023-10-11 20:14:51,277][71601] Updated weights for policy 0, policy_version 29200 (0.0007) [2023-10-11 20:14:51,654][71601] Updated weights for policy 0, policy_version 29210 (0.0009) [2023-10-11 20:14:54,621][71635] Updated weights for policy 1, policy_version 29192 (0.0009) [2023-10-11 20:14:54,992][71635] Updated weights for policy 1, policy_version 29202 (0.0007) [2023-10-11 20:14:55,350][71635] Updated weights for policy 1, policy_version 29212 (0.0007) [2023-10-11 20:14:55,366][71601] Updated weights for policy 0, policy_version 29220 (0.0008) [2023-10-11 20:14:55,730][71601] Updated weights for policy 0, policy_version 29230 (0.0009) [2023-10-11 20:14:56,034][70582] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 59834368. Throughput: 0: 1807.1, 1: 1817.6. Samples: 14969930. Policy #0 lag: (min: 31.0, avg: 31.9, max: 52.0) [2023-10-11 20:14:56,035][70582] Avg episode reward: [(0, '168.190'), (1, '110.370')] [2023-10-11 20:14:56,108][71601] Updated weights for policy 0, policy_version 29240 (0.0008) [2023-10-11 20:14:59,085][71635] Updated weights for policy 1, policy_version 29222 (0.0008) [2023-10-11 20:14:59,455][71635] Updated weights for policy 1, policy_version 29232 (0.0008) [2023-10-11 20:14:59,780][71601] Updated weights for policy 0, policy_version 29250 (0.0008) [2023-10-11 20:14:59,820][71635] Updated weights for policy 1, policy_version 29242 (0.0009) [2023-10-11 20:15:00,154][71601] Updated weights for policy 0, policy_version 29260 (0.0010) [2023-10-11 20:15:00,518][71601] Updated weights for policy 0, policy_version 29270 (0.0009) [2023-10-11 20:15:00,890][71601] Updated weights for policy 0, policy_version 29280 (0.0009) [2023-10-11 20:15:01,034][70582] Fps is (10 sec: 19660.9, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 59932672. Throughput: 0: 1795.4, 1: 1809.0. Samples: 14981248. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:15:01,034][70582] Avg episode reward: [(0, '176.820'), (1, '111.140')] [2023-10-11 20:15:03,564][71635] Updated weights for policy 1, policy_version 29252 (0.0009) [2023-10-11 20:15:03,921][71635] Updated weights for policy 1, policy_version 29262 (0.0009) [2023-10-11 20:15:04,287][71635] Updated weights for policy 1, policy_version 29272 (0.0008) [2023-10-11 20:15:04,677][71601] Updated weights for policy 0, policy_version 29290 (0.0007) [2023-10-11 20:15:05,057][71601] Updated weights for policy 0, policy_version 29300 (0.0007) [2023-10-11 20:15:05,417][71601] Updated weights for policy 0, policy_version 29310 (0.0009) [2023-10-11 20:15:06,034][70582] Fps is (10 sec: 16384.5, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 59998208. Throughput: 0: 1804.2, 1: 1819.8. Samples: 15002690. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:15:06,034][70582] Avg episode reward: [(0, '175.060'), (1, '111.240')] [2023-10-11 20:15:07,985][71635] Updated weights for policy 1, policy_version 29282 (0.0010) [2023-10-11 20:15:08,353][71635] Updated weights for policy 1, policy_version 29292 (0.0008) [2023-10-11 20:15:08,717][71635] Updated weights for policy 1, policy_version 29302 (0.0008) [2023-10-11 20:15:09,090][71635] Updated weights for policy 1, policy_version 29312 (0.0009) [2023-10-11 20:15:09,212][71601] Updated weights for policy 0, policy_version 29320 (0.0009) [2023-10-11 20:15:09,581][71601] Updated weights for policy 0, policy_version 29330 (0.0011) [2023-10-11 20:15:09,949][71601] Updated weights for policy 0, policy_version 29340 (0.0010) [2023-10-11 20:15:11,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 60063744. Throughput: 0: 1796.2, 1: 1819.6. Samples: 15023702. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:15:11,034][70582] Avg episode reward: [(0, '202.310'), (1, '123.070')] [2023-10-11 20:15:12,823][71635] Updated weights for policy 1, policy_version 29322 (0.0007) [2023-10-11 20:15:13,191][71635] Updated weights for policy 1, policy_version 29332 (0.0010) [2023-10-11 20:15:13,550][71635] Updated weights for policy 1, policy_version 29342 (0.0008) [2023-10-11 20:15:13,608][71601] Updated weights for policy 0, policy_version 29350 (0.0010) [2023-10-11 20:15:13,989][71601] Updated weights for policy 0, policy_version 29360 (0.0011) [2023-10-11 20:15:14,366][71601] Updated weights for policy 0, policy_version 29370 (0.0010) [2023-10-11 20:15:16,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 60129280. Throughput: 0: 1807.7, 1: 1823.2. Samples: 15035496. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:15:16,034][70582] Avg episode reward: [(0, '201.630'), (1, '120.280')] [2023-10-11 20:15:17,216][71635] Updated weights for policy 1, policy_version 29352 (0.0008) [2023-10-11 20:15:17,579][71635] Updated weights for policy 1, policy_version 29362 (0.0007) [2023-10-11 20:15:17,942][71635] Updated weights for policy 1, policy_version 29372 (0.0008) [2023-10-11 20:15:18,134][71601] Updated weights for policy 0, policy_version 29380 (0.0009) [2023-10-11 20:15:18,502][71601] Updated weights for policy 0, policy_version 29390 (0.0009) [2023-10-11 20:15:18,880][71601] Updated weights for policy 0, policy_version 29400 (0.0009) [2023-10-11 20:15:21,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 60194816. Throughput: 0: 1802.7, 1: 1821.3. Samples: 15056428. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:15:21,035][70582] Avg episode reward: [(0, '216.900'), (1, '120.310')] [2023-10-11 20:15:21,656][71635] Updated weights for policy 1, policy_version 29382 (0.0009) [2023-10-11 20:15:22,031][71635] Updated weights for policy 1, policy_version 29392 (0.0009) [2023-10-11 20:15:22,399][71635] Updated weights for policy 1, policy_version 29402 (0.0009) [2023-10-11 20:15:22,494][71601] Updated weights for policy 0, policy_version 29410 (0.0010) [2023-10-11 20:15:22,864][71601] Updated weights for policy 0, policy_version 29420 (0.0007) [2023-10-11 20:15:23,232][71601] Updated weights for policy 0, policy_version 29430 (0.0009) [2023-10-11 20:15:23,606][71601] Updated weights for policy 0, policy_version 29440 (0.0009) [2023-10-11 20:15:25,959][71635] Updated weights for policy 1, policy_version 29412 (0.0008) [2023-10-11 20:15:26,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 60260352. Throughput: 0: 1803.9, 1: 1818.3. Samples: 15079314. Policy #0 lag: (min: 19.0, avg: 22.0, max: 51.0) [2023-10-11 20:15:26,035][70582] Avg episode reward: [(0, '205.870'), (1, '116.690')] [2023-10-11 20:15:26,335][71635] Updated weights for policy 1, policy_version 29422 (0.0009) [2023-10-11 20:15:26,703][71635] Updated weights for policy 1, policy_version 29432 (0.0008) [2023-10-11 20:15:27,363][71601] Updated weights for policy 0, policy_version 29450 (0.0008) [2023-10-11 20:15:27,724][71601] Updated weights for policy 0, policy_version 29460 (0.0008) [2023-10-11 20:15:28,101][71601] Updated weights for policy 0, policy_version 29470 (0.0009) [2023-10-11 20:15:30,306][71635] Updated weights for policy 1, policy_version 29442 (0.0008) [2023-10-11 20:15:30,675][71635] Updated weights for policy 1, policy_version 29452 (0.0007) [2023-10-11 20:15:31,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 60325888. Throughput: 0: 1806.4, 1: 1821.9. Samples: 15089380. Policy #0 lag: (min: 19.0, avg: 22.0, max: 51.0) [2023-10-11 20:15:31,034][70582] Avg episode reward: [(0, '210.380'), (1, '120.470')] [2023-10-11 20:15:31,041][71635] Updated weights for policy 1, policy_version 29462 (0.0008) [2023-10-11 20:15:31,408][71635] Updated weights for policy 1, policy_version 29472 (0.0010) [2023-10-11 20:15:31,895][71601] Updated weights for policy 0, policy_version 29480 (0.0008) [2023-10-11 20:15:32,271][71601] Updated weights for policy 0, policy_version 29490 (0.0009) [2023-10-11 20:15:32,651][71601] Updated weights for policy 0, policy_version 29500 (0.0008) [2023-10-11 20:15:35,185][71635] Updated weights for policy 1, policy_version 29482 (0.0008) [2023-10-11 20:15:35,554][71635] Updated weights for policy 1, policy_version 29492 (0.0009) [2023-10-11 20:15:35,936][71635] Updated weights for policy 1, policy_version 29502 (0.0009) [2023-10-11 20:15:36,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 60424192. Throughput: 0: 1808.2, 1: 1821.3. Samples: 15111972. Policy #0 lag: (min: 19.0, avg: 22.0, max: 51.0) [2023-10-11 20:15:36,035][70582] Avg episode reward: [(0, '205.610'), (1, '121.880')] [2023-10-11 20:15:36,245][71601] Updated weights for policy 0, policy_version 29510 (0.0009) [2023-10-11 20:15:36,612][71601] Updated weights for policy 0, policy_version 29520 (0.0009) [2023-10-11 20:15:36,989][71601] Updated weights for policy 0, policy_version 29530 (0.0007) [2023-10-11 20:15:39,627][71635] Updated weights for policy 1, policy_version 29512 (0.0009) [2023-10-11 20:15:39,992][71635] Updated weights for policy 1, policy_version 29522 (0.0011) [2023-10-11 20:15:40,359][71635] Updated weights for policy 1, policy_version 29532 (0.0007) [2023-10-11 20:15:40,659][71601] Updated weights for policy 0, policy_version 29540 (0.0008) [2023-10-11 20:15:41,031][71601] Updated weights for policy 0, policy_version 29550 (0.0009) [2023-10-11 20:15:41,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 60489728. Throughput: 0: 1816.0, 1: 1823.3. Samples: 15133700. Policy #0 lag: (min: 19.0, avg: 22.0, max: 51.0) [2023-10-11 20:15:41,035][70582] Avg episode reward: [(0, '205.740'), (1, '127.770')] [2023-10-11 20:15:41,390][71601] Updated weights for policy 0, policy_version 29560 (0.0009) [2023-10-11 20:15:44,031][71635] Updated weights for policy 1, policy_version 29542 (0.0010) [2023-10-11 20:15:44,398][71635] Updated weights for policy 1, policy_version 29552 (0.0008) [2023-10-11 20:15:44,767][71635] Updated weights for policy 1, policy_version 29562 (0.0008) [2023-10-11 20:15:45,048][71601] Updated weights for policy 0, policy_version 29570 (0.0009) [2023-10-11 20:15:45,417][71601] Updated weights for policy 0, policy_version 29580 (0.0008) [2023-10-11 20:15:45,796][71601] Updated weights for policy 0, policy_version 29590 (0.0009) [2023-10-11 20:15:46,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 60555264. Throughput: 0: 1807.4, 1: 1826.5. Samples: 15144772. Policy #0 lag: (min: 19.0, avg: 22.0, max: 51.0) [2023-10-11 20:15:46,034][70582] Avg episode reward: [(0, '214.450'), (1, '110.850')] [2023-10-11 20:15:46,163][71601] Updated weights for policy 0, policy_version 29600 (0.0007) [2023-10-11 20:15:48,642][71635] Updated weights for policy 1, policy_version 29572 (0.0007) [2023-10-11 20:15:49,009][71635] Updated weights for policy 1, policy_version 29582 (0.0008) [2023-10-11 20:15:49,372][71635] Updated weights for policy 1, policy_version 29592 (0.0010) [2023-10-11 20:15:49,925][71601] Updated weights for policy 0, policy_version 29610 (0.0007) [2023-10-11 20:15:50,295][71601] Updated weights for policy 0, policy_version 29620 (0.0008) [2023-10-11 20:15:50,666][71601] Updated weights for policy 0, policy_version 29630 (0.0009) [2023-10-11 20:15:51,034][70582] Fps is (10 sec: 16384.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 60653568. Throughput: 0: 1814.4, 1: 1816.5. Samples: 15166080. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) [2023-10-11 20:15:51,034][70582] Avg episode reward: [(0, '206.200'), (1, '116.470')] [2023-10-11 20:15:52,885][71635] Updated weights for policy 1, policy_version 29602 (0.0010) [2023-10-11 20:15:53,252][71635] Updated weights for policy 1, policy_version 29612 (0.0009) [2023-10-11 20:15:53,608][71635] Updated weights for policy 1, policy_version 29622 (0.0009) [2023-10-11 20:15:53,975][71635] Updated weights for policy 1, policy_version 29632 (0.0008) [2023-10-11 20:15:54,444][71601] Updated weights for policy 0, policy_version 29640 (0.0007) [2023-10-11 20:15:54,819][71601] Updated weights for policy 0, policy_version 29650 (0.0007) [2023-10-11 20:15:55,185][71601] Updated weights for policy 0, policy_version 29660 (0.0008) [2023-10-11 20:15:56,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 60719104. Throughput: 0: 1808.9, 1: 1820.9. Samples: 15187044. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) [2023-10-11 20:15:56,034][70582] Avg episode reward: [(0, '200.290'), (1, '113.630')] [2023-10-11 20:15:56,046][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000029664_30375936.pth... [2023-10-11 20:15:56,046][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000029632_30343168.pth... [2023-10-11 20:15:56,085][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000027968_28639232.pth [2023-10-11 20:15:56,090][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000027936_28606464.pth [2023-10-11 20:15:57,588][71635] Updated weights for policy 1, policy_version 29642 (0.0009) [2023-10-11 20:15:57,956][71635] Updated weights for policy 1, policy_version 29652 (0.0008) [2023-10-11 20:15:58,334][71635] Updated weights for policy 1, policy_version 29662 (0.0009) [2023-10-11 20:15:58,814][71601] Updated weights for policy 0, policy_version 29670 (0.0007) [2023-10-11 20:15:59,191][71601] Updated weights for policy 0, policy_version 29680 (0.0008) [2023-10-11 20:15:59,561][71601] Updated weights for policy 0, policy_version 29690 (0.0007) [2023-10-11 20:16:01,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 60784640. Throughput: 0: 1812.9, 1: 1817.2. Samples: 15198854. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) [2023-10-11 20:16:01,035][70582] Avg episode reward: [(0, '193.650'), (1, '119.840')] [2023-10-11 20:16:01,993][71635] Updated weights for policy 1, policy_version 29672 (0.0011) [2023-10-11 20:16:02,355][71635] Updated weights for policy 1, policy_version 29682 (0.0009) [2023-10-11 20:16:02,721][71635] Updated weights for policy 1, policy_version 29692 (0.0009) [2023-10-11 20:16:03,274][71601] Updated weights for policy 0, policy_version 29700 (0.0008) [2023-10-11 20:16:03,648][71601] Updated weights for policy 0, policy_version 29710 (0.0009) [2023-10-11 20:16:04,025][71601] Updated weights for policy 0, policy_version 29720 (0.0010) [2023-10-11 20:16:06,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 60850176. Throughput: 0: 1806.3, 1: 1824.8. Samples: 15219830. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) [2023-10-11 20:16:06,034][70582] Avg episode reward: [(0, '203.250'), (1, '126.840')] [2023-10-11 20:16:06,419][71635] Updated weights for policy 1, policy_version 29702 (0.0008) [2023-10-11 20:16:06,778][71635] Updated weights for policy 1, policy_version 29712 (0.0008) [2023-10-11 20:16:07,158][71635] Updated weights for policy 1, policy_version 29722 (0.0010) [2023-10-11 20:16:07,654][71601] Updated weights for policy 0, policy_version 29730 (0.0007) [2023-10-11 20:16:08,026][71601] Updated weights for policy 0, policy_version 29740 (0.0007) [2023-10-11 20:16:08,409][71601] Updated weights for policy 0, policy_version 29750 (0.0008) [2023-10-11 20:16:08,774][71601] Updated weights for policy 0, policy_version 29760 (0.0010) [2023-10-11 20:16:11,011][71635] Updated weights for policy 1, policy_version 29732 (0.0009) [2023-10-11 20:16:11,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 60915712. Throughput: 0: 1809.5, 1: 1824.0. Samples: 15242824. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:16:11,034][70582] Avg episode reward: [(0, '197.590'), (1, '136.480')] [2023-10-11 20:16:11,400][71635] Updated weights for policy 1, policy_version 29742 (0.0010) [2023-10-11 20:16:11,777][71635] Updated weights for policy 1, policy_version 29752 (0.0009) [2023-10-11 20:16:12,453][71601] Updated weights for policy 0, policy_version 29770 (0.0007) [2023-10-11 20:16:12,826][71601] Updated weights for policy 0, policy_version 29780 (0.0007) [2023-10-11 20:16:13,182][71601] Updated weights for policy 0, policy_version 29790 (0.0009) [2023-10-11 20:16:15,525][71635] Updated weights for policy 1, policy_version 29762 (0.0009) [2023-10-11 20:16:15,884][71635] Updated weights for policy 1, policy_version 29772 (0.0009) [2023-10-11 20:16:16,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 60981248. Throughput: 0: 1813.0, 1: 1818.0. Samples: 15252774. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:16:16,035][70582] Avg episode reward: [(0, '200.190'), (1, '139.140')] [2023-10-11 20:16:16,251][71635] Updated weights for policy 1, policy_version 29782 (0.0008) [2023-10-11 20:16:16,616][71635] Updated weights for policy 1, policy_version 29792 (0.0007) [2023-10-11 20:16:16,830][71601] Updated weights for policy 0, policy_version 29800 (0.0008) [2023-10-11 20:16:17,198][71601] Updated weights for policy 0, policy_version 29810 (0.0007) [2023-10-11 20:16:17,566][71601] Updated weights for policy 0, policy_version 29820 (0.0011) [2023-10-11 20:16:20,279][71635] Updated weights for policy 1, policy_version 29802 (0.0009) [2023-10-11 20:16:20,642][71635] Updated weights for policy 1, policy_version 29812 (0.0007) [2023-10-11 20:16:21,016][71635] Updated weights for policy 1, policy_version 29822 (0.0007) [2023-10-11 20:16:21,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 61046784. Throughput: 0: 1813.6, 1: 1815.5. Samples: 15275280. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:16:21,035][70582] Avg episode reward: [(0, '194.580'), (1, '145.190')] [2023-10-11 20:16:21,082][71431] Saving new best policy, reward=145.190! [2023-10-11 20:16:21,224][71601] Updated weights for policy 0, policy_version 29830 (0.0009) [2023-10-11 20:16:21,592][71601] Updated weights for policy 0, policy_version 29840 (0.0009) [2023-10-11 20:16:21,953][71601] Updated weights for policy 0, policy_version 29850 (0.0009) [2023-10-11 20:16:24,765][71635] Updated weights for policy 1, policy_version 29832 (0.0007) [2023-10-11 20:16:25,129][71635] Updated weights for policy 1, policy_version 29842 (0.0007) [2023-10-11 20:16:25,487][71635] Updated weights for policy 1, policy_version 29852 (0.0008) [2023-10-11 20:16:25,628][71601] Updated weights for policy 0, policy_version 29860 (0.0009) [2023-10-11 20:16:25,994][71601] Updated weights for policy 0, policy_version 29870 (0.0008) [2023-10-11 20:16:26,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 61145088. Throughput: 0: 1818.8, 1: 1814.1. Samples: 15297182. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:16:26,034][70582] Avg episode reward: [(0, '194.580'), (1, '141.960')] [2023-10-11 20:16:26,362][71601] Updated weights for policy 0, policy_version 29880 (0.0007) [2023-10-11 20:16:29,159][71635] Updated weights for policy 1, policy_version 29862 (0.0009) [2023-10-11 20:16:29,526][71635] Updated weights for policy 1, policy_version 29872 (0.0008) [2023-10-11 20:16:29,909][71635] Updated weights for policy 1, policy_version 29882 (0.0007) [2023-10-11 20:16:30,066][71601] Updated weights for policy 0, policy_version 29890 (0.0010) [2023-10-11 20:16:30,432][71601] Updated weights for policy 0, policy_version 29900 (0.0009) [2023-10-11 20:16:30,807][71601] Updated weights for policy 0, policy_version 29910 (0.0009) [2023-10-11 20:16:31,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 61210624. Throughput: 0: 1821.5, 1: 1812.6. Samples: 15308306. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:16:31,034][70582] Avg episode reward: [(0, '195.400'), (1, '142.020')] [2023-10-11 20:16:31,187][71601] Updated weights for policy 0, policy_version 29920 (0.0008) [2023-10-11 20:16:33,724][71635] Updated weights for policy 1, policy_version 29892 (0.0008) [2023-10-11 20:16:34,083][71635] Updated weights for policy 1, policy_version 29902 (0.0008) [2023-10-11 20:16:34,449][71635] Updated weights for policy 1, policy_version 29912 (0.0008) [2023-10-11 20:16:34,924][71601] Updated weights for policy 0, policy_version 29930 (0.0009) [2023-10-11 20:16:35,291][71601] Updated weights for policy 0, policy_version 29940 (0.0008) [2023-10-11 20:16:35,665][71601] Updated weights for policy 0, policy_version 29950 (0.0009) [2023-10-11 20:16:36,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 61308928. Throughput: 0: 1819.9, 1: 1827.8. Samples: 15330226. Policy #0 lag: (min: 12.0, avg: 16.7, max: 44.0) [2023-10-11 20:16:36,035][70582] Avg episode reward: [(0, '195.310'), (1, '151.170')] [2023-10-11 20:16:36,036][71431] Saving new best policy, reward=151.170! [2023-10-11 20:16:38,075][71635] Updated weights for policy 1, policy_version 29922 (0.0009) [2023-10-11 20:16:38,442][71635] Updated weights for policy 1, policy_version 29932 (0.0008) [2023-10-11 20:16:38,805][71635] Updated weights for policy 1, policy_version 29942 (0.0008) [2023-10-11 20:16:39,171][71635] Updated weights for policy 1, policy_version 29952 (0.0007) [2023-10-11 20:16:39,221][71601] Updated weights for policy 0, policy_version 29960 (0.0008) [2023-10-11 20:16:39,579][71601] Updated weights for policy 0, policy_version 29970 (0.0009) [2023-10-11 20:16:39,954][71601] Updated weights for policy 0, policy_version 29980 (0.0007) [2023-10-11 20:16:41,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 61374464. Throughput: 0: 1827.6, 1: 1816.8. Samples: 15351040. Policy #0 lag: (min: 12.0, avg: 16.7, max: 44.0) [2023-10-11 20:16:41,034][70582] Avg episode reward: [(0, '185.590'), (1, '150.580')] [2023-10-11 20:16:42,692][71635] Updated weights for policy 1, policy_version 29962 (0.0009) [2023-10-11 20:16:43,066][71635] Updated weights for policy 1, policy_version 29972 (0.0009) [2023-10-11 20:16:43,428][71635] Updated weights for policy 1, policy_version 29982 (0.0007) [2023-10-11 20:16:43,874][71601] Updated weights for policy 0, policy_version 29990 (0.0009) [2023-10-11 20:16:44,249][71601] Updated weights for policy 0, policy_version 30000 (0.0008) [2023-10-11 20:16:44,621][71601] Updated weights for policy 0, policy_version 30010 (0.0008) [2023-10-11 20:16:46,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 61440000. Throughput: 0: 1822.2, 1: 1819.6. Samples: 15362732. Policy #0 lag: (min: 12.0, avg: 16.7, max: 44.0) [2023-10-11 20:16:46,035][70582] Avg episode reward: [(0, '175.790'), (1, '144.400')] [2023-10-11 20:16:47,073][71635] Updated weights for policy 1, policy_version 29992 (0.0009) [2023-10-11 20:16:47,443][71635] Updated weights for policy 1, policy_version 30002 (0.0008) [2023-10-11 20:16:47,813][71635] Updated weights for policy 1, policy_version 30012 (0.0010) [2023-10-11 20:16:48,245][71601] Updated weights for policy 0, policy_version 30020 (0.0008) [2023-10-11 20:16:48,607][71601] Updated weights for policy 0, policy_version 30030 (0.0008) [2023-10-11 20:16:48,970][71601] Updated weights for policy 0, policy_version 30040 (0.0008) [2023-10-11 20:16:51,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 61505536. Throughput: 0: 1824.9, 1: 1818.6. Samples: 15383788. Policy #0 lag: (min: 12.0, avg: 16.7, max: 44.0) [2023-10-11 20:16:51,035][70582] Avg episode reward: [(0, '175.890'), (1, '144.350')] [2023-10-11 20:16:51,430][71635] Updated weights for policy 1, policy_version 30022 (0.0011) [2023-10-11 20:16:51,800][71635] Updated weights for policy 1, policy_version 30032 (0.0009) [2023-10-11 20:16:52,167][71635] Updated weights for policy 1, policy_version 30042 (0.0009) [2023-10-11 20:16:52,780][71601] Updated weights for policy 0, policy_version 30050 (0.0009) [2023-10-11 20:16:53,144][71601] Updated weights for policy 0, policy_version 30060 (0.0007) [2023-10-11 20:16:53,520][71601] Updated weights for policy 0, policy_version 30070 (0.0008) [2023-10-11 20:16:53,892][71601] Updated weights for policy 0, policy_version 30080 (0.0008) [2023-10-11 20:16:56,032][71635] Updated weights for policy 1, policy_version 30052 (0.0008) [2023-10-11 20:16:56,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 61571072. Throughput: 0: 1821.6, 1: 1820.1. Samples: 15406700. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-11 20:16:56,035][70582] Avg episode reward: [(0, '161.970'), (1, '144.260')] [2023-10-11 20:16:56,421][71635] Updated weights for policy 1, policy_version 30062 (0.0009) [2023-10-11 20:16:56,786][71635] Updated weights for policy 1, policy_version 30072 (0.0011) [2023-10-11 20:16:57,506][71601] Updated weights for policy 0, policy_version 30090 (0.0009) [2023-10-11 20:16:57,882][71601] Updated weights for policy 0, policy_version 30100 (0.0011) [2023-10-11 20:16:58,240][71601] Updated weights for policy 0, policy_version 30110 (0.0009) [2023-10-11 20:17:00,298][71635] Updated weights for policy 1, policy_version 30082 (0.0009) [2023-10-11 20:17:00,664][71635] Updated weights for policy 1, policy_version 30092 (0.0007) [2023-10-11 20:17:01,028][71635] Updated weights for policy 1, policy_version 30102 (0.0008) [2023-10-11 20:17:01,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 61636608. Throughput: 0: 1820.1, 1: 1820.8. Samples: 15416612. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-11 20:17:01,034][70582] Avg episode reward: [(0, '172.250'), (1, '157.890')] [2023-10-11 20:17:01,389][71431] Saving new best policy, reward=157.890! [2023-10-11 20:17:01,390][71635] Updated weights for policy 1, policy_version 30112 (0.0009) [2023-10-11 20:17:01,984][71601] Updated weights for policy 0, policy_version 30120 (0.0009) [2023-10-11 20:17:02,345][71601] Updated weights for policy 0, policy_version 30130 (0.0009) [2023-10-11 20:17:02,716][71601] Updated weights for policy 0, policy_version 30140 (0.0007) [2023-10-11 20:17:05,106][71635] Updated weights for policy 1, policy_version 30122 (0.0012) [2023-10-11 20:17:05,480][71635] Updated weights for policy 1, policy_version 30132 (0.0009) [2023-10-11 20:17:05,838][71635] Updated weights for policy 1, policy_version 30142 (0.0009) [2023-10-11 20:17:06,034][70582] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 61734912. Throughput: 0: 1821.5, 1: 1825.7. Samples: 15439404. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-11 20:17:06,034][70582] Avg episode reward: [(0, '162.360'), (1, '152.880')] [2023-10-11 20:17:06,449][71601] Updated weights for policy 0, policy_version 30150 (0.0009) [2023-10-11 20:17:06,821][71601] Updated weights for policy 0, policy_version 30160 (0.0008) [2023-10-11 20:17:07,185][71601] Updated weights for policy 0, policy_version 30170 (0.0007) [2023-10-11 20:17:09,635][71635] Updated weights for policy 1, policy_version 30152 (0.0007) [2023-10-11 20:17:09,996][71635] Updated weights for policy 1, policy_version 30162 (0.0007) [2023-10-11 20:17:10,367][71635] Updated weights for policy 1, policy_version 30172 (0.0008) [2023-10-11 20:17:10,806][71601] Updated weights for policy 0, policy_version 30180 (0.0007) [2023-10-11 20:17:11,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 61800448. Throughput: 0: 1818.0, 1: 1820.1. Samples: 15460896. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-11 20:17:11,034][70582] Avg episode reward: [(0, '173.000'), (1, '150.340')] [2023-10-11 20:17:11,182][71601] Updated weights for policy 0, policy_version 30190 (0.0007) [2023-10-11 20:17:11,553][71601] Updated weights for policy 0, policy_version 30200 (0.0008) [2023-10-11 20:17:14,042][71635] Updated weights for policy 1, policy_version 30182 (0.0007) [2023-10-11 20:17:14,411][71635] Updated weights for policy 1, policy_version 30192 (0.0007) [2023-10-11 20:17:14,780][71635] Updated weights for policy 1, policy_version 30202 (0.0009) [2023-10-11 20:17:15,138][71601] Updated weights for policy 0, policy_version 30210 (0.0011) [2023-10-11 20:17:15,512][71601] Updated weights for policy 0, policy_version 30220 (0.0008) [2023-10-11 20:17:15,890][71601] Updated weights for policy 0, policy_version 30230 (0.0007) [2023-10-11 20:17:16,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 61865984. Throughput: 0: 1814.4, 1: 1821.4. Samples: 15471914. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-11 20:17:16,034][70582] Avg episode reward: [(0, '145.250'), (1, '154.620')] [2023-10-11 20:17:16,251][71601] Updated weights for policy 0, policy_version 30240 (0.0008) [2023-10-11 20:17:18,349][71635] Updated weights for policy 1, policy_version 30212 (0.0010) [2023-10-11 20:17:18,718][71635] Updated weights for policy 1, policy_version 30222 (0.0010) [2023-10-11 20:17:19,095][71635] Updated weights for policy 1, policy_version 30232 (0.0010) [2023-10-11 20:17:19,987][71601] Updated weights for policy 0, policy_version 30250 (0.0009) [2023-10-11 20:17:20,364][71601] Updated weights for policy 0, policy_version 30260 (0.0009) [2023-10-11 20:17:20,745][71601] Updated weights for policy 0, policy_version 30270 (0.0009) [2023-10-11 20:17:21,034][70582] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 61964288. Throughput: 0: 1821.7, 1: 1810.4. Samples: 15493670. Policy #0 lag: (min: 10.0, avg: 12.0, max: 38.0) [2023-10-11 20:17:21,034][70582] Avg episode reward: [(0, '143.590'), (1, '158.190')] [2023-10-11 20:17:21,035][71431] Saving new best policy, reward=158.190! [2023-10-11 20:17:22,671][71635] Updated weights for policy 1, policy_version 30242 (0.0009) [2023-10-11 20:17:23,035][71635] Updated weights for policy 1, policy_version 30252 (0.0007) [2023-10-11 20:17:23,408][71635] Updated weights for policy 1, policy_version 30262 (0.0007) [2023-10-11 20:17:23,776][71635] Updated weights for policy 1, policy_version 30272 (0.0009) [2023-10-11 20:17:24,398][71601] Updated weights for policy 0, policy_version 30280 (0.0009) [2023-10-11 20:17:24,769][71601] Updated weights for policy 0, policy_version 30290 (0.0007) [2023-10-11 20:17:25,146][71601] Updated weights for policy 0, policy_version 30300 (0.0007) [2023-10-11 20:17:26,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 62029824. Throughput: 0: 1822.2, 1: 1826.0. Samples: 15515210. Policy #0 lag: (min: 10.0, avg: 12.0, max: 38.0) [2023-10-11 20:17:26,035][70582] Avg episode reward: [(0, '143.520'), (1, '167.570')] [2023-10-11 20:17:26,049][71431] Saving new best policy, reward=167.570! [2023-10-11 20:17:27,615][71635] Updated weights for policy 1, policy_version 30282 (0.0008) [2023-10-11 20:17:27,982][71635] Updated weights for policy 1, policy_version 30292 (0.0007) [2023-10-11 20:17:28,344][71635] Updated weights for policy 1, policy_version 30302 (0.0008) [2023-10-11 20:17:28,870][71601] Updated weights for policy 0, policy_version 30310 (0.0009) [2023-10-11 20:17:29,248][71601] Updated weights for policy 0, policy_version 30320 (0.0009) [2023-10-11 20:17:29,625][71601] Updated weights for policy 0, policy_version 30330 (0.0008) [2023-10-11 20:17:31,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 62095360. Throughput: 0: 1825.3, 1: 1820.6. Samples: 15526796. Policy #0 lag: (min: 10.0, avg: 12.0, max: 38.0) [2023-10-11 20:17:31,035][70582] Avg episode reward: [(0, '144.630'), (1, '154.390')] [2023-10-11 20:17:32,055][71635] Updated weights for policy 1, policy_version 30312 (0.0007) [2023-10-11 20:17:32,413][71635] Updated weights for policy 1, policy_version 30322 (0.0008) [2023-10-11 20:17:32,785][71635] Updated weights for policy 1, policy_version 30332 (0.0008) [2023-10-11 20:17:33,148][71601] Updated weights for policy 0, policy_version 30340 (0.0008) [2023-10-11 20:17:33,525][71601] Updated weights for policy 0, policy_version 30350 (0.0008) [2023-10-11 20:17:33,901][71601] Updated weights for policy 0, policy_version 30360 (0.0011) [2023-10-11 20:17:36,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 62160896. Throughput: 0: 1826.8, 1: 1823.0. Samples: 15548032. Policy #0 lag: (min: 10.0, avg: 12.0, max: 38.0) [2023-10-11 20:17:36,035][70582] Avg episode reward: [(0, '140.790'), (1, '157.000')] [2023-10-11 20:17:36,588][71635] Updated weights for policy 1, policy_version 30342 (0.0009) [2023-10-11 20:17:36,952][71635] Updated weights for policy 1, policy_version 30352 (0.0007) [2023-10-11 20:17:37,319][71635] Updated weights for policy 1, policy_version 30362 (0.0007) [2023-10-11 20:17:37,555][71601] Updated weights for policy 0, policy_version 30370 (0.0009) [2023-10-11 20:17:37,929][71601] Updated weights for policy 0, policy_version 30380 (0.0009) [2023-10-11 20:17:38,296][71601] Updated weights for policy 0, policy_version 30390 (0.0007) [2023-10-11 20:17:38,665][71601] Updated weights for policy 0, policy_version 30400 (0.0009) [2023-10-11 20:17:40,972][71635] Updated weights for policy 1, policy_version 30372 (0.0009) [2023-10-11 20:17:41,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 62226432. Throughput: 0: 1824.1, 1: 1824.9. Samples: 15570906. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-11 20:17:41,034][70582] Avg episode reward: [(0, '129.720'), (1, '149.770')] [2023-10-11 20:17:41,381][71635] Updated weights for policy 1, policy_version 30382 (0.0010) [2023-10-11 20:17:41,754][71635] Updated weights for policy 1, policy_version 30392 (0.0008) [2023-10-11 20:17:42,371][71601] Updated weights for policy 0, policy_version 30410 (0.0008) [2023-10-11 20:17:42,741][71601] Updated weights for policy 0, policy_version 30420 (0.0008) [2023-10-11 20:17:43,112][71601] Updated weights for policy 0, policy_version 30430 (0.0007) [2023-10-11 20:17:45,283][71635] Updated weights for policy 1, policy_version 30402 (0.0010) [2023-10-11 20:17:45,654][71635] Updated weights for policy 1, policy_version 30412 (0.0009) [2023-10-11 20:17:46,022][71635] Updated weights for policy 1, policy_version 30422 (0.0010) [2023-10-11 20:17:46,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 62291968. Throughput: 0: 1826.5, 1: 1823.0. Samples: 15580838. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-11 20:17:46,034][70582] Avg episode reward: [(0, '138.770'), (1, '153.890')] [2023-10-11 20:17:46,388][71635] Updated weights for policy 1, policy_version 30432 (0.0010) [2023-10-11 20:17:46,753][71601] Updated weights for policy 0, policy_version 30440 (0.0008) [2023-10-11 20:17:47,125][71601] Updated weights for policy 0, policy_version 30450 (0.0009) [2023-10-11 20:17:47,499][71601] Updated weights for policy 0, policy_version 30460 (0.0008) [2023-10-11 20:17:50,218][71635] Updated weights for policy 1, policy_version 30442 (0.0008) [2023-10-11 20:17:50,592][71635] Updated weights for policy 1, policy_version 30452 (0.0008) [2023-10-11 20:17:50,955][71635] Updated weights for policy 1, policy_version 30462 (0.0008) [2023-10-11 20:17:51,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 62390272. Throughput: 0: 1825.6, 1: 1825.2. Samples: 15603688. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-11 20:17:51,034][70582] Avg episode reward: [(0, '141.700'), (1, '161.810')] [2023-10-11 20:17:51,335][71601] Updated weights for policy 0, policy_version 30470 (0.0009) [2023-10-11 20:17:51,701][71601] Updated weights for policy 0, policy_version 30480 (0.0007) [2023-10-11 20:17:52,066][71601] Updated weights for policy 0, policy_version 30490 (0.0007) [2023-10-11 20:17:54,317][71635] Updated weights for policy 1, policy_version 30472 (0.0007) [2023-10-11 20:17:54,690][71635] Updated weights for policy 1, policy_version 30482 (0.0007) [2023-10-11 20:17:55,053][71635] Updated weights for policy 1, policy_version 30492 (0.0007) [2023-10-11 20:17:55,886][71601] Updated weights for policy 0, policy_version 30500 (0.0009) [2023-10-11 20:17:56,034][70582] Fps is (10 sec: 16383.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 62455808. Throughput: 0: 1817.9, 1: 1826.6. Samples: 15624898. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-11 20:17:56,035][70582] Avg episode reward: [(0, '134.750'), (1, '143.300')] [2023-10-11 20:17:56,046][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000030496_31227904.pth... [2023-10-11 20:17:56,081][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000028800_29491200.pth [2023-10-11 20:17:56,260][71601] Updated weights for policy 0, policy_version 30510 (0.0009) [2023-10-11 20:17:56,642][71601] Updated weights for policy 0, policy_version 30520 (0.0009) [2023-10-11 20:17:56,935][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000030528_31260672.pth... [2023-10-11 20:17:56,964][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000028800_29491200.pth [2023-10-11 20:17:58,758][71635] Updated weights for policy 1, policy_version 30502 (0.0008) [2023-10-11 20:17:59,125][71635] Updated weights for policy 1, policy_version 30512 (0.0007) [2023-10-11 20:17:59,498][71635] Updated weights for policy 1, policy_version 30522 (0.0007) [2023-10-11 20:18:00,365][71601] Updated weights for policy 0, policy_version 30530 (0.0010) [2023-10-11 20:18:00,741][71601] Updated weights for policy 0, policy_version 30540 (0.0011) [2023-10-11 20:18:01,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 62521344. Throughput: 0: 1820.7, 1: 1833.8. Samples: 15636366. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-11 20:18:01,034][70582] Avg episode reward: [(0, '134.660'), (1, '141.070')] [2023-10-11 20:18:01,109][71601] Updated weights for policy 0, policy_version 30550 (0.0009) [2023-10-11 20:18:01,491][71601] Updated weights for policy 0, policy_version 30560 (0.0009) [2023-10-11 20:18:03,039][71635] Updated weights for policy 1, policy_version 30532 (0.0008) [2023-10-11 20:18:03,409][71635] Updated weights for policy 1, policy_version 30542 (0.0008) [2023-10-11 20:18:03,773][71635] Updated weights for policy 1, policy_version 30552 (0.0007) [2023-10-11 20:18:05,079][71601] Updated weights for policy 0, policy_version 30570 (0.0007) [2023-10-11 20:18:05,460][71601] Updated weights for policy 0, policy_version 30580 (0.0008) [2023-10-11 20:18:05,831][71601] Updated weights for policy 0, policy_version 30590 (0.0007) [2023-10-11 20:18:06,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 62619648. Throughput: 0: 1809.6, 1: 1833.7. Samples: 15657620. Policy #0 lag: (min: 28.0, avg: 33.5, max: 60.0) [2023-10-11 20:18:06,035][70582] Avg episode reward: [(0, '136.840'), (1, '141.280')] [2023-10-11 20:18:07,510][71635] Updated weights for policy 1, policy_version 30562 (0.0010) [2023-10-11 20:18:07,875][71635] Updated weights for policy 1, policy_version 30572 (0.0008) [2023-10-11 20:18:08,239][71635] Updated weights for policy 1, policy_version 30582 (0.0007) [2023-10-11 20:18:08,607][71635] Updated weights for policy 1, policy_version 30592 (0.0007) [2023-10-11 20:18:09,520][71601] Updated weights for policy 0, policy_version 30600 (0.0007) [2023-10-11 20:18:09,899][71601] Updated weights for policy 0, policy_version 30610 (0.0008) [2023-10-11 20:18:10,269][71601] Updated weights for policy 0, policy_version 30620 (0.0009) [2023-10-11 20:18:11,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 62685184. Throughput: 0: 1810.8, 1: 1833.3. Samples: 15679192. Policy #0 lag: (min: 28.0, avg: 33.5, max: 60.0) [2023-10-11 20:18:11,035][70582] Avg episode reward: [(0, '134.720'), (1, '136.910')] [2023-10-11 20:18:12,330][71635] Updated weights for policy 1, policy_version 30602 (0.0009) [2023-10-11 20:18:12,701][71635] Updated weights for policy 1, policy_version 30612 (0.0007) [2023-10-11 20:18:13,072][71635] Updated weights for policy 1, policy_version 30622 (0.0007) [2023-10-11 20:18:13,950][71601] Updated weights for policy 0, policy_version 30630 (0.0008) [2023-10-11 20:18:14,324][71601] Updated weights for policy 0, policy_version 30640 (0.0010) [2023-10-11 20:18:14,693][71601] Updated weights for policy 0, policy_version 30650 (0.0007) [2023-10-11 20:18:16,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 62750720. Throughput: 0: 1812.0, 1: 1830.3. Samples: 15690696. Policy #0 lag: (min: 28.0, avg: 33.5, max: 60.0) [2023-10-11 20:18:16,035][70582] Avg episode reward: [(0, '132.200'), (1, '127.560')] [2023-10-11 20:18:16,791][71635] Updated weights for policy 1, policy_version 30632 (0.0009) [2023-10-11 20:18:17,153][71635] Updated weights for policy 1, policy_version 30642 (0.0007) [2023-10-11 20:18:17,526][71635] Updated weights for policy 1, policy_version 30652 (0.0008) [2023-10-11 20:18:18,277][71601] Updated weights for policy 0, policy_version 30660 (0.0008) [2023-10-11 20:18:18,648][71601] Updated weights for policy 0, policy_version 30670 (0.0011) [2023-10-11 20:18:19,027][71601] Updated weights for policy 0, policy_version 30680 (0.0010) [2023-10-11 20:18:21,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 62816256. Throughput: 0: 1811.9, 1: 1840.1. Samples: 15712372. Policy #0 lag: (min: 28.0, avg: 33.5, max: 60.0) [2023-10-11 20:18:21,034][70582] Avg episode reward: [(0, '136.430'), (1, '128.020')] [2023-10-11 20:18:21,213][71635] Updated weights for policy 1, policy_version 30662 (0.0008) [2023-10-11 20:18:21,582][71635] Updated weights for policy 1, policy_version 30672 (0.0009) [2023-10-11 20:18:21,949][71635] Updated weights for policy 1, policy_version 30682 (0.0009) [2023-10-11 20:18:22,838][71601] Updated weights for policy 0, policy_version 30690 (0.0009) [2023-10-11 20:18:23,220][71601] Updated weights for policy 0, policy_version 30700 (0.0008) [2023-10-11 20:18:23,593][71601] Updated weights for policy 0, policy_version 30710 (0.0008) [2023-10-11 20:18:23,968][71601] Updated weights for policy 0, policy_version 30720 (0.0008) [2023-10-11 20:18:25,676][71635] Updated weights for policy 1, policy_version 30692 (0.0007) [2023-10-11 20:18:26,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 62881792. Throughput: 0: 1810.6, 1: 1839.2. Samples: 15735148. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:18:26,034][70582] Avg episode reward: [(0, '138.000'), (1, '131.430')] [2023-10-11 20:18:26,083][71635] Updated weights for policy 1, policy_version 30702 (0.0009) [2023-10-11 20:18:26,448][71635] Updated weights for policy 1, policy_version 30712 (0.0011) [2023-10-11 20:18:27,566][71601] Updated weights for policy 0, policy_version 30730 (0.0009) [2023-10-11 20:18:27,946][71601] Updated weights for policy 0, policy_version 30740 (0.0007) [2023-10-11 20:18:28,304][71601] Updated weights for policy 0, policy_version 30750 (0.0007) [2023-10-11 20:18:30,062][71635] Updated weights for policy 1, policy_version 30722 (0.0010) [2023-10-11 20:18:30,436][71635] Updated weights for policy 1, policy_version 30732 (0.0010) [2023-10-11 20:18:30,800][71635] Updated weights for policy 1, policy_version 30742 (0.0010) [2023-10-11 20:18:31,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 62947328. Throughput: 0: 1808.6, 1: 1840.2. Samples: 15745032. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:18:31,034][70582] Avg episode reward: [(0, '142.640'), (1, '123.090')] [2023-10-11 20:18:31,172][71635] Updated weights for policy 1, policy_version 30752 (0.0009) [2023-10-11 20:18:31,916][71601] Updated weights for policy 0, policy_version 30760 (0.0008) [2023-10-11 20:18:32,283][71601] Updated weights for policy 0, policy_version 30770 (0.0008) [2023-10-11 20:18:32,658][71601] Updated weights for policy 0, policy_version 30780 (0.0008) [2023-10-11 20:18:34,709][71635] Updated weights for policy 1, policy_version 30762 (0.0008) [2023-10-11 20:18:35,089][71635] Updated weights for policy 1, policy_version 30772 (0.0007) [2023-10-11 20:18:35,450][71635] Updated weights for policy 1, policy_version 30782 (0.0008) [2023-10-11 20:18:36,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 63045632. Throughput: 0: 1813.3, 1: 1835.0. Samples: 15767862. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:18:36,035][70582] Avg episode reward: [(0, '139.030'), (1, '122.990')] [2023-10-11 20:18:36,215][71601] Updated weights for policy 0, policy_version 30790 (0.0010) [2023-10-11 20:18:36,597][71601] Updated weights for policy 0, policy_version 30800 (0.0007) [2023-10-11 20:18:36,964][71601] Updated weights for policy 0, policy_version 30810 (0.0008) [2023-10-11 20:18:39,087][71635] Updated weights for policy 1, policy_version 30792 (0.0007) [2023-10-11 20:18:39,450][71635] Updated weights for policy 1, policy_version 30802 (0.0008) [2023-10-11 20:18:39,820][71635] Updated weights for policy 1, policy_version 30812 (0.0007) [2023-10-11 20:18:40,769][71601] Updated weights for policy 0, policy_version 30820 (0.0008) [2023-10-11 20:18:41,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 63111168. Throughput: 0: 1820.2, 1: 1833.3. Samples: 15789306. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:18:41,035][70582] Avg episode reward: [(0, '139.740'), (1, '130.550')] [2023-10-11 20:18:41,151][71601] Updated weights for policy 0, policy_version 30830 (0.0007) [2023-10-11 20:18:41,518][71601] Updated weights for policy 0, policy_version 30840 (0.0007) [2023-10-11 20:18:43,455][71635] Updated weights for policy 1, policy_version 30822 (0.0009) [2023-10-11 20:18:43,823][71635] Updated weights for policy 1, policy_version 30832 (0.0009) [2023-10-11 20:18:44,194][71635] Updated weights for policy 1, policy_version 30842 (0.0009) [2023-10-11 20:18:45,189][71601] Updated weights for policy 0, policy_version 30850 (0.0007) [2023-10-11 20:18:45,562][71601] Updated weights for policy 0, policy_version 30860 (0.0007) [2023-10-11 20:18:45,929][71601] Updated weights for policy 0, policy_version 30870 (0.0009) [2023-10-11 20:18:46,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 63176704. Throughput: 0: 1819.2, 1: 1829.3. Samples: 15800548. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:18:46,034][70582] Avg episode reward: [(0, '134.270'), (1, '130.350')] [2023-10-11 20:18:46,304][71601] Updated weights for policy 0, policy_version 30880 (0.0009) [2023-10-11 20:18:47,866][71635] Updated weights for policy 1, policy_version 30852 (0.0009) [2023-10-11 20:18:48,233][71635] Updated weights for policy 1, policy_version 30862 (0.0010) [2023-10-11 20:18:48,605][71635] Updated weights for policy 1, policy_version 30872 (0.0008) [2023-10-11 20:18:50,083][71601] Updated weights for policy 0, policy_version 30890 (0.0009) [2023-10-11 20:18:50,449][71601] Updated weights for policy 0, policy_version 30900 (0.0008) [2023-10-11 20:18:50,820][71601] Updated weights for policy 0, policy_version 30910 (0.0008) [2023-10-11 20:18:51,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 63275008. Throughput: 0: 1823.3, 1: 1828.6. Samples: 15821954. Policy #0 lag: (min: 29.0, avg: 32.6, max: 61.0) [2023-10-11 20:18:51,034][70582] Avg episode reward: [(0, '137.000'), (1, '140.250')] [2023-10-11 20:18:52,354][71635] Updated weights for policy 1, policy_version 30882 (0.0008) [2023-10-11 20:18:52,725][71635] Updated weights for policy 1, policy_version 30892 (0.0007) [2023-10-11 20:18:53,094][71635] Updated weights for policy 1, policy_version 30902 (0.0008) [2023-10-11 20:18:53,452][71635] Updated weights for policy 1, policy_version 30912 (0.0008) [2023-10-11 20:18:54,528][71601] Updated weights for policy 0, policy_version 30920 (0.0009) [2023-10-11 20:18:54,892][71601] Updated weights for policy 0, policy_version 30930 (0.0010) [2023-10-11 20:18:55,267][71601] Updated weights for policy 0, policy_version 30940 (0.0007) [2023-10-11 20:18:56,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 63340544. Throughput: 0: 1815.2, 1: 1829.3. Samples: 15843196. Policy #0 lag: (min: 29.0, avg: 32.6, max: 61.0) [2023-10-11 20:18:56,034][70582] Avg episode reward: [(0, '137.290'), (1, '149.030')] [2023-10-11 20:18:57,075][71635] Updated weights for policy 1, policy_version 30922 (0.0007) [2023-10-11 20:18:57,445][71635] Updated weights for policy 1, policy_version 30932 (0.0010) [2023-10-11 20:18:57,813][71635] Updated weights for policy 1, policy_version 30942 (0.0009) [2023-10-11 20:18:58,939][71601] Updated weights for policy 0, policy_version 30950 (0.0009) [2023-10-11 20:18:59,313][71601] Updated weights for policy 0, policy_version 30960 (0.0008) [2023-10-11 20:18:59,682][71601] Updated weights for policy 0, policy_version 30970 (0.0008) [2023-10-11 20:19:01,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 63406080. Throughput: 0: 1817.1, 1: 1834.0. Samples: 15854994. Policy #0 lag: (min: 29.0, avg: 32.6, max: 61.0) [2023-10-11 20:19:01,034][70582] Avg episode reward: [(0, '132.840'), (1, '145.590')] [2023-10-11 20:19:01,559][71635] Updated weights for policy 1, policy_version 30952 (0.0007) [2023-10-11 20:19:01,929][71635] Updated weights for policy 1, policy_version 30962 (0.0007) [2023-10-11 20:19:02,300][71635] Updated weights for policy 1, policy_version 30972 (0.0008) [2023-10-11 20:19:03,389][71601] Updated weights for policy 0, policy_version 30980 (0.0009) [2023-10-11 20:19:03,755][71601] Updated weights for policy 0, policy_version 30990 (0.0008) [2023-10-11 20:19:04,117][71601] Updated weights for policy 0, policy_version 31000 (0.0008) [2023-10-11 20:19:06,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 63471616. Throughput: 0: 1815.3, 1: 1827.6. Samples: 15876302. Policy #0 lag: (min: 29.0, avg: 32.6, max: 61.0) [2023-10-11 20:19:06,034][70582] Avg episode reward: [(0, '138.620'), (1, '148.590')] [2023-10-11 20:19:06,135][71635] Updated weights for policy 1, policy_version 30982 (0.0009) [2023-10-11 20:19:06,504][71635] Updated weights for policy 1, policy_version 30992 (0.0007) [2023-10-11 20:19:06,865][71635] Updated weights for policy 1, policy_version 31002 (0.0009) [2023-10-11 20:19:07,655][71601] Updated weights for policy 0, policy_version 31010 (0.0007) [2023-10-11 20:19:08,032][71601] Updated weights for policy 0, policy_version 31020 (0.0009) [2023-10-11 20:19:08,406][71601] Updated weights for policy 0, policy_version 31030 (0.0008) [2023-10-11 20:19:08,777][71601] Updated weights for policy 0, policy_version 31040 (0.0009) [2023-10-11 20:19:10,384][71635] Updated weights for policy 1, policy_version 31012 (0.0008) [2023-10-11 20:19:10,749][71635] Updated weights for policy 1, policy_version 31022 (0.0007) [2023-10-11 20:19:11,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 63537152. Throughput: 0: 1820.6, 1: 1821.2. Samples: 15899030. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-11 20:19:11,034][70582] Avg episode reward: [(0, '143.590'), (1, '149.100')] [2023-10-11 20:19:11,115][71635] Updated weights for policy 1, policy_version 31032 (0.0008) [2023-10-11 20:19:12,483][71601] Updated weights for policy 0, policy_version 31050 (0.0010) [2023-10-11 20:19:12,856][71601] Updated weights for policy 0, policy_version 31060 (0.0009) [2023-10-11 20:19:13,223][71601] Updated weights for policy 0, policy_version 31070 (0.0010) [2023-10-11 20:19:14,756][71635] Updated weights for policy 1, policy_version 31042 (0.0009) [2023-10-11 20:19:15,130][71635] Updated weights for policy 1, policy_version 31052 (0.0007) [2023-10-11 20:19:15,493][71635] Updated weights for policy 1, policy_version 31062 (0.0008) [2023-10-11 20:19:15,860][71635] Updated weights for policy 1, policy_version 31072 (0.0008) [2023-10-11 20:19:16,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 63635456. Throughput: 0: 1820.1, 1: 1832.1. Samples: 15909380. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-11 20:19:16,035][70582] Avg episode reward: [(0, '134.850'), (1, '140.620')] [2023-10-11 20:19:16,965][71601] Updated weights for policy 0, policy_version 31080 (0.0009) [2023-10-11 20:19:17,326][71601] Updated weights for policy 0, policy_version 31090 (0.0007) [2023-10-11 20:19:17,698][71601] Updated weights for policy 0, policy_version 31100 (0.0008) [2023-10-11 20:19:19,434][71635] Updated weights for policy 1, policy_version 31082 (0.0009) [2023-10-11 20:19:19,807][71635] Updated weights for policy 1, policy_version 31092 (0.0011) [2023-10-11 20:19:20,179][71635] Updated weights for policy 1, policy_version 31102 (0.0010) [2023-10-11 20:19:21,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 63700992. Throughput: 0: 1817.2, 1: 1824.6. Samples: 15931744. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-11 20:19:21,034][70582] Avg episode reward: [(0, '134.670'), (1, '136.340')] [2023-10-11 20:19:21,454][71601] Updated weights for policy 0, policy_version 31110 (0.0009) [2023-10-11 20:19:21,827][71601] Updated weights for policy 0, policy_version 31120 (0.0010) [2023-10-11 20:19:22,195][71601] Updated weights for policy 0, policy_version 31130 (0.0009) [2023-10-11 20:19:23,941][71635] Updated weights for policy 1, policy_version 31112 (0.0009) [2023-10-11 20:19:24,303][71635] Updated weights for policy 1, policy_version 31122 (0.0009) [2023-10-11 20:19:24,661][71635] Updated weights for policy 1, policy_version 31132 (0.0009) [2023-10-11 20:19:25,924][71601] Updated weights for policy 0, policy_version 31140 (0.0009) [2023-10-11 20:19:26,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 63766528. Throughput: 0: 1815.9, 1: 1828.0. Samples: 15953278. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-11 20:19:26,035][70582] Avg episode reward: [(0, '141.070'), (1, '130.780')] [2023-10-11 20:19:26,299][71601] Updated weights for policy 0, policy_version 31150 (0.0009) [2023-10-11 20:19:26,681][71601] Updated weights for policy 0, policy_version 31160 (0.0008) [2023-10-11 20:19:28,411][71635] Updated weights for policy 1, policy_version 31142 (0.0008) [2023-10-11 20:19:28,780][71635] Updated weights for policy 1, policy_version 31152 (0.0007) [2023-10-11 20:19:29,147][71635] Updated weights for policy 1, policy_version 31162 (0.0008) [2023-10-11 20:19:30,319][71601] Updated weights for policy 0, policy_version 31170 (0.0010) [2023-10-11 20:19:30,692][71601] Updated weights for policy 0, policy_version 31180 (0.0007) [2023-10-11 20:19:31,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 63832064. Throughput: 0: 1815.9, 1: 1827.6. Samples: 15964506. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-11 20:19:31,034][70582] Avg episode reward: [(0, '141.280'), (1, '138.370')] [2023-10-11 20:19:31,060][71601] Updated weights for policy 0, policy_version 31190 (0.0009) [2023-10-11 20:19:31,435][71601] Updated weights for policy 0, policy_version 31200 (0.0009) [2023-10-11 20:19:32,821][71635] Updated weights for policy 1, policy_version 31172 (0.0011) [2023-10-11 20:19:33,183][71635] Updated weights for policy 1, policy_version 31182 (0.0008) [2023-10-11 20:19:33,546][71635] Updated weights for policy 1, policy_version 31192 (0.0007) [2023-10-11 20:19:35,168][71601] Updated weights for policy 0, policy_version 31210 (0.0008) [2023-10-11 20:19:35,538][71601] Updated weights for policy 0, policy_version 31220 (0.0009) [2023-10-11 20:19:35,911][71601] Updated weights for policy 0, policy_version 31230 (0.0008) [2023-10-11 20:19:36,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 63930368. Throughput: 0: 1815.9, 1: 1830.9. Samples: 15986062. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 20:19:36,035][70582] Avg episode reward: [(0, '141.370'), (1, '130.750')] [2023-10-11 20:19:37,105][71635] Updated weights for policy 1, policy_version 31202 (0.0007) [2023-10-11 20:19:37,470][71635] Updated weights for policy 1, policy_version 31212 (0.0008) [2023-10-11 20:19:37,839][71635] Updated weights for policy 1, policy_version 31222 (0.0007) [2023-10-11 20:19:38,206][71635] Updated weights for policy 1, policy_version 31232 (0.0008) [2023-10-11 20:19:39,568][71601] Updated weights for policy 0, policy_version 31240 (0.0009) [2023-10-11 20:19:39,933][71601] Updated weights for policy 0, policy_version 31250 (0.0010) [2023-10-11 20:19:40,305][71601] Updated weights for policy 0, policy_version 31260 (0.0007) [2023-10-11 20:19:41,034][70582] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 63995904. Throughput: 0: 1821.3, 1: 1830.6. Samples: 16007532. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 20:19:41,035][70582] Avg episode reward: [(0, '129.830'), (1, '133.400')] [2023-10-11 20:19:42,028][71635] Updated weights for policy 1, policy_version 31242 (0.0008) [2023-10-11 20:19:42,398][71635] Updated weights for policy 1, policy_version 31252 (0.0008) [2023-10-11 20:19:42,769][71635] Updated weights for policy 1, policy_version 31262 (0.0008) [2023-10-11 20:19:43,948][71601] Updated weights for policy 0, policy_version 31270 (0.0010) [2023-10-11 20:19:44,346][71601] Updated weights for policy 0, policy_version 31280 (0.0011) [2023-10-11 20:19:44,713][71601] Updated weights for policy 0, policy_version 31290 (0.0010) [2023-10-11 20:19:46,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 64061440. Throughput: 0: 1817.3, 1: 1825.8. Samples: 16018932. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 20:19:46,034][70582] Avg episode reward: [(0, '127.550'), (1, '133.400')] [2023-10-11 20:19:46,415][71635] Updated weights for policy 1, policy_version 31272 (0.0008) [2023-10-11 20:19:46,779][71635] Updated weights for policy 1, policy_version 31282 (0.0008) [2023-10-11 20:19:47,139][71635] Updated weights for policy 1, policy_version 31292 (0.0009) [2023-10-11 20:19:48,513][71601] Updated weights for policy 0, policy_version 31300 (0.0008) [2023-10-11 20:19:48,894][71601] Updated weights for policy 0, policy_version 31310 (0.0008) [2023-10-11 20:19:49,269][71601] Updated weights for policy 0, policy_version 31320 (0.0008) [2023-10-11 20:19:50,800][71635] Updated weights for policy 1, policy_version 31302 (0.0007) [2023-10-11 20:19:51,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 64126976. Throughput: 0: 1817.2, 1: 1826.1. Samples: 16040252. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 20:19:51,035][70582] Avg episode reward: [(0, '127.860'), (1, '135.570')] [2023-10-11 20:19:51,169][71635] Updated weights for policy 1, policy_version 31312 (0.0011) [2023-10-11 20:19:51,541][71635] Updated weights for policy 1, policy_version 31322 (0.0010) [2023-10-11 20:19:52,994][71601] Updated weights for policy 0, policy_version 31330 (0.0009) [2023-10-11 20:19:53,368][71601] Updated weights for policy 0, policy_version 31340 (0.0008) [2023-10-11 20:19:53,741][71601] Updated weights for policy 0, policy_version 31350 (0.0008) [2023-10-11 20:19:54,110][71601] Updated weights for policy 0, policy_version 31360 (0.0010) [2023-10-11 20:19:55,172][71635] Updated weights for policy 1, policy_version 31332 (0.0008) [2023-10-11 20:19:55,538][71635] Updated weights for policy 1, policy_version 31342 (0.0011) [2023-10-11 20:19:55,905][71635] Updated weights for policy 1, policy_version 31352 (0.0007) [2023-10-11 20:19:56,034][70582] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 64192512. Throughput: 0: 1814.0, 1: 1823.1. Samples: 16062702. Policy #0 lag: (min: 5.0, avg: 15.8, max: 37.0) [2023-10-11 20:19:56,035][70582] Avg episode reward: [(0, '127.860'), (1, '126.740')] [2023-10-11 20:19:56,046][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000031360_32112640.pth... [2023-10-11 20:19:56,086][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000029664_30375936.pth [2023-10-11 20:19:56,197][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000031360_32112640.pth... [2023-10-11 20:19:56,232][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000029632_30343168.pth [2023-10-11 20:19:57,716][71601] Updated weights for policy 0, policy_version 31370 (0.0009) [2023-10-11 20:19:58,083][71601] Updated weights for policy 0, policy_version 31380 (0.0007) [2023-10-11 20:19:58,458][71601] Updated weights for policy 0, policy_version 31390 (0.0007) [2023-10-11 20:19:59,690][71635] Updated weights for policy 1, policy_version 31362 (0.0010) [2023-10-11 20:20:00,112][71635] Updated weights for policy 1, policy_version 31372 (0.0008) [2023-10-11 20:20:00,480][71635] Updated weights for policy 1, policy_version 31382 (0.0008) [2023-10-11 20:20:00,847][71635] Updated weights for policy 1, policy_version 31392 (0.0007) [2023-10-11 20:20:01,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 64290816. Throughput: 0: 1821.0, 1: 1823.7. Samples: 16073392. Policy #0 lag: (min: 5.0, avg: 15.8, max: 37.0) [2023-10-11 20:20:01,034][70582] Avg episode reward: [(0, '128.590'), (1, '130.600')] [2023-10-11 20:20:02,136][71601] Updated weights for policy 0, policy_version 31400 (0.0009) [2023-10-11 20:20:02,506][71601] Updated weights for policy 0, policy_version 31410 (0.0008) [2023-10-11 20:20:02,877][71601] Updated weights for policy 0, policy_version 31420 (0.0008) [2023-10-11 20:20:04,581][71635] Updated weights for policy 1, policy_version 31402 (0.0008) [2023-10-11 20:20:04,947][71635] Updated weights for policy 1, policy_version 31412 (0.0010) [2023-10-11 20:20:05,316][71635] Updated weights for policy 1, policy_version 31422 (0.0011) [2023-10-11 20:20:06,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 64356352. Throughput: 0: 1822.1, 1: 1820.4. Samples: 16095660. Policy #0 lag: (min: 5.0, avg: 15.8, max: 37.0) [2023-10-11 20:20:06,035][70582] Avg episode reward: [(0, '128.480'), (1, '128.360')] [2023-10-11 20:20:06,526][71601] Updated weights for policy 0, policy_version 31430 (0.0008) [2023-10-11 20:20:06,894][71601] Updated weights for policy 0, policy_version 31440 (0.0009) [2023-10-11 20:20:07,265][71601] Updated weights for policy 0, policy_version 31450 (0.0010) [2023-10-11 20:20:08,998][71635] Updated weights for policy 1, policy_version 31432 (0.0010) [2023-10-11 20:20:09,360][71635] Updated weights for policy 1, policy_version 31442 (0.0010) [2023-10-11 20:20:09,735][71635] Updated weights for policy 1, policy_version 31452 (0.0010) [2023-10-11 20:20:10,929][71601] Updated weights for policy 0, policy_version 31460 (0.0010) [2023-10-11 20:20:11,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 64421888. Throughput: 0: 1822.0, 1: 1818.7. Samples: 16117112. Policy #0 lag: (min: 5.0, avg: 15.8, max: 37.0) [2023-10-11 20:20:11,034][70582] Avg episode reward: [(0, '135.970'), (1, '131.410')] [2023-10-11 20:20:11,304][71601] Updated weights for policy 0, policy_version 31470 (0.0007) [2023-10-11 20:20:11,675][71601] Updated weights for policy 0, policy_version 31480 (0.0009) [2023-10-11 20:20:13,325][71635] Updated weights for policy 1, policy_version 31462 (0.0008) [2023-10-11 20:20:13,698][71635] Updated weights for policy 1, policy_version 31472 (0.0008) [2023-10-11 20:20:14,058][71635] Updated weights for policy 1, policy_version 31482 (0.0010) [2023-10-11 20:20:15,342][71601] Updated weights for policy 0, policy_version 31490 (0.0008) [2023-10-11 20:20:15,713][71601] Updated weights for policy 0, policy_version 31500 (0.0007) [2023-10-11 20:20:16,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 64487424. Throughput: 0: 1823.5, 1: 1815.3. Samples: 16128254. Policy #0 lag: (min: 5.0, avg: 15.8, max: 37.0) [2023-10-11 20:20:16,035][70582] Avg episode reward: [(0, '135.830'), (1, '128.410')] [2023-10-11 20:20:16,079][71601] Updated weights for policy 0, policy_version 31510 (0.0009) [2023-10-11 20:20:16,447][71601] Updated weights for policy 0, policy_version 31520 (0.0007) [2023-10-11 20:20:17,830][71635] Updated weights for policy 1, policy_version 31492 (0.0007) [2023-10-11 20:20:18,200][71635] Updated weights for policy 1, policy_version 31502 (0.0009) [2023-10-11 20:20:18,562][71635] Updated weights for policy 1, policy_version 31512 (0.0009) [2023-10-11 20:20:20,125][71601] Updated weights for policy 0, policy_version 31530 (0.0010) [2023-10-11 20:20:20,507][71601] Updated weights for policy 0, policy_version 31540 (0.0009) [2023-10-11 20:20:20,870][71601] Updated weights for policy 0, policy_version 31550 (0.0007) [2023-10-11 20:20:21,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 64585728. Throughput: 0: 1826.8, 1: 1814.9. Samples: 16149938. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:20:21,034][70582] Avg episode reward: [(0, '135.740'), (1, '129.150')] [2023-10-11 20:20:22,260][71635] Updated weights for policy 1, policy_version 31522 (0.0009) [2023-10-11 20:20:22,620][71635] Updated weights for policy 1, policy_version 31532 (0.0009) [2023-10-11 20:20:22,988][71635] Updated weights for policy 1, policy_version 31542 (0.0010) [2023-10-11 20:20:23,352][71635] Updated weights for policy 1, policy_version 31552 (0.0009) [2023-10-11 20:20:24,510][71601] Updated weights for policy 0, policy_version 31560 (0.0010) [2023-10-11 20:20:24,885][71601] Updated weights for policy 0, policy_version 31570 (0.0011) [2023-10-11 20:20:25,264][71601] Updated weights for policy 0, policy_version 31580 (0.0010) [2023-10-11 20:20:26,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 64651264. Throughput: 0: 1826.3, 1: 1811.6. Samples: 16171238. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:20:26,035][70582] Avg episode reward: [(0, '140.390'), (1, '130.460')] [2023-10-11 20:20:27,193][71635] Updated weights for policy 1, policy_version 31562 (0.0011) [2023-10-11 20:20:27,560][71635] Updated weights for policy 1, policy_version 31572 (0.0011) [2023-10-11 20:20:27,931][71635] Updated weights for policy 1, policy_version 31582 (0.0007) [2023-10-11 20:20:28,890][71601] Updated weights for policy 0, policy_version 31590 (0.0010) [2023-10-11 20:20:29,261][71601] Updated weights for policy 0, policy_version 31600 (0.0007) [2023-10-11 20:20:29,631][71601] Updated weights for policy 0, policy_version 31610 (0.0008) [2023-10-11 20:20:31,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 64716800. Throughput: 0: 1828.6, 1: 1812.3. Samples: 16182770. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:20:31,034][70582] Avg episode reward: [(0, '145.930'), (1, '130.340')] [2023-10-11 20:20:31,405][71635] Updated weights for policy 1, policy_version 31592 (0.0007) [2023-10-11 20:20:31,779][71635] Updated weights for policy 1, policy_version 31602 (0.0007) [2023-10-11 20:20:32,145][71635] Updated weights for policy 1, policy_version 31612 (0.0007) [2023-10-11 20:20:33,312][71601] Updated weights for policy 0, policy_version 31620 (0.0008) [2023-10-11 20:20:33,685][71601] Updated weights for policy 0, policy_version 31630 (0.0007) [2023-10-11 20:20:34,057][71601] Updated weights for policy 0, policy_version 31640 (0.0009) [2023-10-11 20:20:35,878][71635] Updated weights for policy 1, policy_version 31622 (0.0008) [2023-10-11 20:20:36,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 64782336. Throughput: 0: 1827.2, 1: 1812.8. Samples: 16204056. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:20:36,034][70582] Avg episode reward: [(0, '145.920'), (1, '132.520')] [2023-10-11 20:20:36,240][71635] Updated weights for policy 1, policy_version 31632 (0.0008) [2023-10-11 20:20:36,609][71635] Updated weights for policy 1, policy_version 31642 (0.0010) [2023-10-11 20:20:37,792][71601] Updated weights for policy 0, policy_version 31650 (0.0009) [2023-10-11 20:20:38,166][71601] Updated weights for policy 0, policy_version 31660 (0.0010) [2023-10-11 20:20:38,540][71601] Updated weights for policy 0, policy_version 31670 (0.0008) [2023-10-11 20:20:38,906][71601] Updated weights for policy 0, policy_version 31680 (0.0009) [2023-10-11 20:20:40,237][71635] Updated weights for policy 1, policy_version 31652 (0.0009) [2023-10-11 20:20:40,607][71635] Updated weights for policy 1, policy_version 31662 (0.0010) [2023-10-11 20:20:40,965][71635] Updated weights for policy 1, policy_version 31672 (0.0008) [2023-10-11 20:20:41,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 64847872. Throughput: 0: 1826.9, 1: 1810.5. Samples: 16226384. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:20:41,034][70582] Avg episode reward: [(0, '145.900'), (1, '131.930')] [2023-10-11 20:20:42,420][71601] Updated weights for policy 0, policy_version 31690 (0.0009) [2023-10-11 20:20:42,794][71601] Updated weights for policy 0, policy_version 31700 (0.0010) [2023-10-11 20:20:43,167][71601] Updated weights for policy 0, policy_version 31710 (0.0011) [2023-10-11 20:20:44,782][71635] Updated weights for policy 1, policy_version 31682 (0.0009) [2023-10-11 20:20:45,197][71635] Updated weights for policy 1, policy_version 31692 (0.0008) [2023-10-11 20:20:45,554][71635] Updated weights for policy 1, policy_version 31702 (0.0008) [2023-10-11 20:20:45,920][71635] Updated weights for policy 1, policy_version 31712 (0.0008) [2023-10-11 20:20:46,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 64946176. Throughput: 0: 1826.5, 1: 1807.9. Samples: 16236940. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:20:46,034][70582] Avg episode reward: [(0, '145.900'), (1, '137.880')] [2023-10-11 20:20:46,870][71601] Updated weights for policy 0, policy_version 31720 (0.0010) [2023-10-11 20:20:47,237][71601] Updated weights for policy 0, policy_version 31730 (0.0009) [2023-10-11 20:20:47,605][71601] Updated weights for policy 0, policy_version 31740 (0.0009) [2023-10-11 20:20:49,591][71635] Updated weights for policy 1, policy_version 31722 (0.0008) [2023-10-11 20:20:49,966][71635] Updated weights for policy 1, policy_version 31732 (0.0009) [2023-10-11 20:20:50,327][71635] Updated weights for policy 1, policy_version 31742 (0.0007) [2023-10-11 20:20:51,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 65011712. Throughput: 0: 1825.8, 1: 1811.9. Samples: 16259358. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:20:51,034][70582] Avg episode reward: [(0, '145.900'), (1, '143.890')] [2023-10-11 20:20:51,335][71601] Updated weights for policy 0, policy_version 31750 (0.0008) [2023-10-11 20:20:51,706][71601] Updated weights for policy 0, policy_version 31760 (0.0007) [2023-10-11 20:20:52,083][71601] Updated weights for policy 0, policy_version 31770 (0.0010) [2023-10-11 20:20:54,003][71635] Updated weights for policy 1, policy_version 31752 (0.0010) [2023-10-11 20:20:54,376][71635] Updated weights for policy 1, policy_version 31762 (0.0010) [2023-10-11 20:20:54,733][71635] Updated weights for policy 1, policy_version 31772 (0.0008) [2023-10-11 20:20:55,707][71601] Updated weights for policy 0, policy_version 31780 (0.0009) [2023-10-11 20:20:56,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 65077248. Throughput: 0: 1830.3, 1: 1811.5. Samples: 16280994. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:20:56,034][70582] Avg episode reward: [(0, '153.750'), (1, '143.370')] [2023-10-11 20:20:56,082][71601] Updated weights for policy 0, policy_version 31790 (0.0009) [2023-10-11 20:20:56,445][71601] Updated weights for policy 0, policy_version 31800 (0.0010) [2023-10-11 20:20:58,456][71635] Updated weights for policy 1, policy_version 31782 (0.0009) [2023-10-11 20:20:58,815][71635] Updated weights for policy 1, policy_version 31792 (0.0008) [2023-10-11 20:20:59,183][71635] Updated weights for policy 1, policy_version 31802 (0.0009) [2023-10-11 20:21:00,045][71601] Updated weights for policy 0, policy_version 31810 (0.0008) [2023-10-11 20:21:00,417][71601] Updated weights for policy 0, policy_version 31820 (0.0007) [2023-10-11 20:21:00,793][71601] Updated weights for policy 0, policy_version 31830 (0.0008) [2023-10-11 20:21:01,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 65142784. Throughput: 0: 1833.3, 1: 1818.4. Samples: 16292576. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:21:01,034][70582] Avg episode reward: [(0, '162.750'), (1, '139.830')] [2023-10-11 20:21:01,155][71601] Updated weights for policy 0, policy_version 31840 (0.0009) [2023-10-11 20:21:03,008][71635] Updated weights for policy 1, policy_version 31812 (0.0008) [2023-10-11 20:21:03,368][71635] Updated weights for policy 1, policy_version 31822 (0.0007) [2023-10-11 20:21:03,735][71635] Updated weights for policy 1, policy_version 31832 (0.0008) [2023-10-11 20:21:05,007][71601] Updated weights for policy 0, policy_version 31850 (0.0010) [2023-10-11 20:21:05,383][71601] Updated weights for policy 0, policy_version 31860 (0.0008) [2023-10-11 20:21:05,758][71601] Updated weights for policy 0, policy_version 31870 (0.0010) [2023-10-11 20:21:06,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 65241088. Throughput: 0: 1826.8, 1: 1811.9. Samples: 16313678. Policy #0 lag: (min: 24.0, avg: 51.4, max: 56.0) [2023-10-11 20:21:06,035][70582] Avg episode reward: [(0, '174.740'), (1, '149.190')] [2023-10-11 20:21:07,340][71635] Updated weights for policy 1, policy_version 31842 (0.0009) [2023-10-11 20:21:07,706][71635] Updated weights for policy 1, policy_version 31852 (0.0011) [2023-10-11 20:21:08,069][71635] Updated weights for policy 1, policy_version 31862 (0.0010) [2023-10-11 20:21:08,433][71635] Updated weights for policy 1, policy_version 31872 (0.0009) [2023-10-11 20:21:09,324][71601] Updated weights for policy 0, policy_version 31880 (0.0010) [2023-10-11 20:21:09,704][71601] Updated weights for policy 0, policy_version 31890 (0.0009) [2023-10-11 20:21:10,071][71601] Updated weights for policy 0, policy_version 31900 (0.0010) [2023-10-11 20:21:11,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 65306624. Throughput: 0: 1825.5, 1: 1820.1. Samples: 16335288. Policy #0 lag: (min: 24.0, avg: 51.4, max: 56.0) [2023-10-11 20:21:11,035][70582] Avg episode reward: [(0, '171.110'), (1, '151.960')] [2023-10-11 20:21:11,963][71635] Updated weights for policy 1, policy_version 31882 (0.0007) [2023-10-11 20:21:12,322][71635] Updated weights for policy 1, policy_version 31892 (0.0008) [2023-10-11 20:21:12,686][71635] Updated weights for policy 1, policy_version 31902 (0.0008) [2023-10-11 20:21:13,757][71601] Updated weights for policy 0, policy_version 31910 (0.0009) [2023-10-11 20:21:14,135][71601] Updated weights for policy 0, policy_version 31920 (0.0011) [2023-10-11 20:21:14,507][71601] Updated weights for policy 0, policy_version 31930 (0.0008) [2023-10-11 20:21:16,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 65372160. Throughput: 0: 1826.0, 1: 1821.3. Samples: 16346896. Policy #0 lag: (min: 24.0, avg: 51.4, max: 56.0) [2023-10-11 20:21:16,035][70582] Avg episode reward: [(0, '168.660'), (1, '164.490')] [2023-10-11 20:21:16,383][71635] Updated weights for policy 1, policy_version 31912 (0.0009) [2023-10-11 20:21:16,748][71635] Updated weights for policy 1, policy_version 31922 (0.0008) [2023-10-11 20:21:17,117][71635] Updated weights for policy 1, policy_version 31932 (0.0011) [2023-10-11 20:21:18,211][71601] Updated weights for policy 0, policy_version 31940 (0.0008) [2023-10-11 20:21:18,582][71601] Updated weights for policy 0, policy_version 31950 (0.0009) [2023-10-11 20:21:18,948][71601] Updated weights for policy 0, policy_version 31960 (0.0011) [2023-10-11 20:21:20,893][71635] Updated weights for policy 1, policy_version 31942 (0.0008) [2023-10-11 20:21:21,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 65437696. Throughput: 0: 1820.8, 1: 1819.2. Samples: 16367858. Policy #0 lag: (min: 24.0, avg: 51.4, max: 56.0) [2023-10-11 20:21:21,035][70582] Avg episode reward: [(0, '173.590'), (1, '156.270')] [2023-10-11 20:21:21,257][71635] Updated weights for policy 1, policy_version 31952 (0.0007) [2023-10-11 20:21:21,620][71635] Updated weights for policy 1, policy_version 31962 (0.0007) [2023-10-11 20:21:22,734][71601] Updated weights for policy 0, policy_version 31970 (0.0009) [2023-10-11 20:21:23,100][71601] Updated weights for policy 0, policy_version 31980 (0.0009) [2023-10-11 20:21:23,469][71601] Updated weights for policy 0, policy_version 31990 (0.0009) [2023-10-11 20:21:23,839][71601] Updated weights for policy 0, policy_version 32000 (0.0009) [2023-10-11 20:21:25,299][71635] Updated weights for policy 1, policy_version 31972 (0.0009) [2023-10-11 20:21:25,662][71635] Updated weights for policy 1, policy_version 31982 (0.0011) [2023-10-11 20:21:26,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 65503232. Throughput: 0: 1819.2, 1: 1827.8. Samples: 16390500. Policy #0 lag: (min: 16.0, avg: 38.3, max: 48.0) [2023-10-11 20:21:26,034][70582] Avg episode reward: [(0, '178.000'), (1, '149.650')] [2023-10-11 20:21:26,037][71635] Updated weights for policy 1, policy_version 31992 (0.0011) [2023-10-11 20:21:27,609][71601] Updated weights for policy 0, policy_version 32010 (0.0007) [2023-10-11 20:21:27,975][71601] Updated weights for policy 0, policy_version 32020 (0.0007) [2023-10-11 20:21:28,339][71601] Updated weights for policy 0, policy_version 32030 (0.0007) [2023-10-11 20:21:29,736][71635] Updated weights for policy 1, policy_version 32002 (0.0010) [2023-10-11 20:21:30,139][71635] Updated weights for policy 1, policy_version 32012 (0.0007) [2023-10-11 20:21:30,505][71635] Updated weights for policy 1, policy_version 32022 (0.0009) [2023-10-11 20:21:30,873][71635] Updated weights for policy 1, policy_version 32032 (0.0010) [2023-10-11 20:21:31,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 65601536. Throughput: 0: 1810.7, 1: 1825.9. Samples: 16400590. Policy #0 lag: (min: 16.0, avg: 38.3, max: 48.0) [2023-10-11 20:21:31,035][70582] Avg episode reward: [(0, '173.290'), (1, '137.420')] [2023-10-11 20:21:31,990][71601] Updated weights for policy 0, policy_version 32040 (0.0009) [2023-10-11 20:21:32,364][71601] Updated weights for policy 0, policy_version 32050 (0.0008) [2023-10-11 20:21:32,733][71601] Updated weights for policy 0, policy_version 32060 (0.0010) [2023-10-11 20:21:34,578][71635] Updated weights for policy 1, policy_version 32042 (0.0007) [2023-10-11 20:21:34,945][71635] Updated weights for policy 1, policy_version 32052 (0.0007) [2023-10-11 20:21:35,304][71635] Updated weights for policy 1, policy_version 32062 (0.0007) [2023-10-11 20:21:36,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 65667072. Throughput: 0: 1812.3, 1: 1832.2. Samples: 16423360. Policy #0 lag: (min: 16.0, avg: 38.3, max: 48.0) [2023-10-11 20:21:36,035][70582] Avg episode reward: [(0, '185.020'), (1, '159.550')] [2023-10-11 20:21:36,433][71601] Updated weights for policy 0, policy_version 32070 (0.0009) [2023-10-11 20:21:36,801][71601] Updated weights for policy 0, policy_version 32080 (0.0008) [2023-10-11 20:21:37,171][71601] Updated weights for policy 0, policy_version 32090 (0.0008) [2023-10-11 20:21:39,001][71635] Updated weights for policy 1, policy_version 32072 (0.0008) [2023-10-11 20:21:39,362][71635] Updated weights for policy 1, policy_version 32082 (0.0008) [2023-10-11 20:21:39,737][71635] Updated weights for policy 1, policy_version 32092 (0.0008) [2023-10-11 20:21:40,756][71601] Updated weights for policy 0, policy_version 32100 (0.0010) [2023-10-11 20:21:41,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 65732608. Throughput: 0: 1812.1, 1: 1833.8. Samples: 16445060. Policy #0 lag: (min: 16.0, avg: 38.3, max: 48.0) [2023-10-11 20:21:41,034][70582] Avg episode reward: [(0, '184.900'), (1, '156.830')] [2023-10-11 20:21:41,130][71601] Updated weights for policy 0, policy_version 32110 (0.0008) [2023-10-11 20:21:41,490][71601] Updated weights for policy 0, policy_version 32120 (0.0008) [2023-10-11 20:21:43,217][71635] Updated weights for policy 1, policy_version 32102 (0.0008) [2023-10-11 20:21:43,579][71635] Updated weights for policy 1, policy_version 32112 (0.0008) [2023-10-11 20:21:43,941][71635] Updated weights for policy 1, policy_version 32122 (0.0011) [2023-10-11 20:21:45,306][71601] Updated weights for policy 0, policy_version 32130 (0.0009) [2023-10-11 20:21:45,687][71601] Updated weights for policy 0, policy_version 32140 (0.0008) [2023-10-11 20:21:46,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 65798144. Throughput: 0: 1807.9, 1: 1829.8. Samples: 16456274. Policy #0 lag: (min: 16.0, avg: 38.3, max: 48.0) [2023-10-11 20:21:46,035][70582] Avg episode reward: [(0, '181.860'), (1, '154.010')] [2023-10-11 20:21:46,067][71601] Updated weights for policy 0, policy_version 32150 (0.0009) [2023-10-11 20:21:46,437][71601] Updated weights for policy 0, policy_version 32160 (0.0010) [2023-10-11 20:21:47,581][71635] Updated weights for policy 1, policy_version 32132 (0.0010) [2023-10-11 20:21:47,946][71635] Updated weights for policy 1, policy_version 32142 (0.0007) [2023-10-11 20:21:48,320][71635] Updated weights for policy 1, policy_version 32152 (0.0007) [2023-10-11 20:21:50,147][71601] Updated weights for policy 0, policy_version 32170 (0.0008) [2023-10-11 20:21:50,524][71601] Updated weights for policy 0, policy_version 32180 (0.0010) [2023-10-11 20:21:50,897][71601] Updated weights for policy 0, policy_version 32190 (0.0008) [2023-10-11 20:21:51,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 65896448. Throughput: 0: 1811.2, 1: 1845.1. Samples: 16478212. Policy #0 lag: (min: 30.0, avg: 37.3, max: 62.0) [2023-10-11 20:21:51,034][70582] Avg episode reward: [(0, '197.500'), (1, '152.260')] [2023-10-11 20:21:51,879][71635] Updated weights for policy 1, policy_version 32162 (0.0007) [2023-10-11 20:21:52,241][71635] Updated weights for policy 1, policy_version 32172 (0.0007) [2023-10-11 20:21:52,608][71635] Updated weights for policy 1, policy_version 32182 (0.0007) [2023-10-11 20:21:52,961][71635] Updated weights for policy 1, policy_version 32192 (0.0008) [2023-10-11 20:21:54,697][71601] Updated weights for policy 0, policy_version 32200 (0.0008) [2023-10-11 20:21:55,070][71601] Updated weights for policy 0, policy_version 32210 (0.0008) [2023-10-11 20:21:55,447][71601] Updated weights for policy 0, policy_version 32220 (0.0008) [2023-10-11 20:21:56,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 65961984. Throughput: 0: 1814.1, 1: 1849.5. Samples: 16500152. Policy #0 lag: (min: 30.0, avg: 37.3, max: 62.0) [2023-10-11 20:21:56,035][70582] Avg episode reward: [(0, '186.190'), (1, '158.530')] [2023-10-11 20:21:56,047][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000032192_32964608.pth... [2023-10-11 20:21:56,048][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000032224_32997376.pth... [2023-10-11 20:21:56,081][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000030528_31260672.pth [2023-10-11 20:21:56,087][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000030496_31227904.pth [2023-10-11 20:21:56,657][71635] Updated weights for policy 1, policy_version 32202 (0.0009) [2023-10-11 20:21:57,018][71635] Updated weights for policy 1, policy_version 32212 (0.0008) [2023-10-11 20:21:57,391][71635] Updated weights for policy 1, policy_version 32222 (0.0008) [2023-10-11 20:21:59,151][71601] Updated weights for policy 0, policy_version 32230 (0.0008) [2023-10-11 20:21:59,522][71601] Updated weights for policy 0, policy_version 32240 (0.0009) [2023-10-11 20:21:59,899][71601] Updated weights for policy 0, policy_version 32250 (0.0010) [2023-10-11 20:22:01,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 66027520. Throughput: 0: 1808.4, 1: 1848.0. Samples: 16511432. Policy #0 lag: (min: 30.0, avg: 37.3, max: 62.0) [2023-10-11 20:22:01,034][70582] Avg episode reward: [(0, '192.770'), (1, '157.760')] [2023-10-11 20:22:01,092][71635] Updated weights for policy 1, policy_version 32232 (0.0007) [2023-10-11 20:22:01,445][71635] Updated weights for policy 1, policy_version 32242 (0.0008) [2023-10-11 20:22:01,812][71635] Updated weights for policy 1, policy_version 32252 (0.0010) [2023-10-11 20:22:03,480][71601] Updated weights for policy 0, policy_version 32260 (0.0009) [2023-10-11 20:22:03,858][71601] Updated weights for policy 0, policy_version 32270 (0.0008) [2023-10-11 20:22:04,215][71601] Updated weights for policy 0, policy_version 32280 (0.0009) [2023-10-11 20:22:05,547][71635] Updated weights for policy 1, policy_version 32262 (0.0009) [2023-10-11 20:22:05,907][71635] Updated weights for policy 1, policy_version 32272 (0.0007) [2023-10-11 20:22:06,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 66093056. Throughput: 0: 1823.8, 1: 1844.2. Samples: 16532918. Policy #0 lag: (min: 30.0, avg: 37.3, max: 62.0) [2023-10-11 20:22:06,034][70582] Avg episode reward: [(0, '184.970'), (1, '154.980')] [2023-10-11 20:22:06,269][71635] Updated weights for policy 1, policy_version 32282 (0.0007) [2023-10-11 20:22:07,867][71601] Updated weights for policy 0, policy_version 32290 (0.0008) [2023-10-11 20:22:08,242][71601] Updated weights for policy 0, policy_version 32300 (0.0009) [2023-10-11 20:22:08,613][71601] Updated weights for policy 0, policy_version 32310 (0.0007) [2023-10-11 20:22:08,988][71601] Updated weights for policy 0, policy_version 32320 (0.0010) [2023-10-11 20:22:09,932][71635] Updated weights for policy 1, policy_version 32292 (0.0007) [2023-10-11 20:22:10,296][71635] Updated weights for policy 1, policy_version 32302 (0.0007) [2023-10-11 20:22:10,671][71635] Updated weights for policy 1, policy_version 32312 (0.0007) [2023-10-11 20:22:11,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 66191360. Throughput: 0: 1822.9, 1: 1832.8. Samples: 16555008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:22:11,034][70582] Avg episode reward: [(0, '182.550'), (1, '157.580')] [2023-10-11 20:22:12,599][71601] Updated weights for policy 0, policy_version 32330 (0.0010) [2023-10-11 20:22:12,971][71601] Updated weights for policy 0, policy_version 32340 (0.0010) [2023-10-11 20:22:13,356][71601] Updated weights for policy 0, policy_version 32350 (0.0009) [2023-10-11 20:22:14,426][71635] Updated weights for policy 1, policy_version 32322 (0.0007) [2023-10-11 20:22:14,792][71635] Updated weights for policy 1, policy_version 32332 (0.0008) [2023-10-11 20:22:15,154][71635] Updated weights for policy 1, policy_version 32342 (0.0008) [2023-10-11 20:22:15,519][71635] Updated weights for policy 1, policy_version 32352 (0.0011) [2023-10-11 20:22:16,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 66256896. Throughput: 0: 1824.5, 1: 1848.0. Samples: 16565852. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:22:16,034][70582] Avg episode reward: [(0, '185.750'), (1, '159.260')] [2023-10-11 20:22:16,948][71601] Updated weights for policy 0, policy_version 32360 (0.0008) [2023-10-11 20:22:17,332][71601] Updated weights for policy 0, policy_version 32370 (0.0007) [2023-10-11 20:22:17,695][71601] Updated weights for policy 0, policy_version 32380 (0.0008) [2023-10-11 20:22:19,217][71635] Updated weights for policy 1, policy_version 32362 (0.0007) [2023-10-11 20:22:19,590][71635] Updated weights for policy 1, policy_version 32372 (0.0007) [2023-10-11 20:22:19,961][71635] Updated weights for policy 1, policy_version 32382 (0.0007) [2023-10-11 20:22:21,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 66322432. Throughput: 0: 1827.9, 1: 1832.7. Samples: 16588086. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:22:21,035][70582] Avg episode reward: [(0, '184.050'), (1, '160.090')] [2023-10-11 20:22:21,381][71601] Updated weights for policy 0, policy_version 32390 (0.0008) [2023-10-11 20:22:21,763][71601] Updated weights for policy 0, policy_version 32400 (0.0007) [2023-10-11 20:22:22,132][71601] Updated weights for policy 0, policy_version 32410 (0.0008) [2023-10-11 20:22:23,424][71635] Updated weights for policy 1, policy_version 32392 (0.0008) [2023-10-11 20:22:23,804][71635] Updated weights for policy 1, policy_version 32402 (0.0008) [2023-10-11 20:22:24,168][71635] Updated weights for policy 1, policy_version 32412 (0.0008) [2023-10-11 20:22:25,671][71601] Updated weights for policy 0, policy_version 32420 (0.0008) [2023-10-11 20:22:26,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 66387968. Throughput: 0: 1828.4, 1: 1843.9. Samples: 16610312. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:22:26,034][70582] Avg episode reward: [(0, '195.180'), (1, '161.830')] [2023-10-11 20:22:26,052][71601] Updated weights for policy 0, policy_version 32430 (0.0008) [2023-10-11 20:22:26,426][71601] Updated weights for policy 0, policy_version 32440 (0.0009) [2023-10-11 20:22:27,845][71635] Updated weights for policy 1, policy_version 32422 (0.0008) [2023-10-11 20:22:28,206][71635] Updated weights for policy 1, policy_version 32432 (0.0009) [2023-10-11 20:22:28,578][71635] Updated weights for policy 1, policy_version 32442 (0.0008) [2023-10-11 20:22:30,006][71601] Updated weights for policy 0, policy_version 32450 (0.0010) [2023-10-11 20:22:30,377][71601] Updated weights for policy 0, policy_version 32460 (0.0008) [2023-10-11 20:22:30,757][71601] Updated weights for policy 0, policy_version 32470 (0.0009) [2023-10-11 20:22:31,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 66453504. Throughput: 0: 1828.7, 1: 1828.4. Samples: 16620840. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:22:31,034][70582] Avg episode reward: [(0, '190.270'), (1, '163.080')] [2023-10-11 20:22:31,124][71601] Updated weights for policy 0, policy_version 32480 (0.0008) [2023-10-11 20:22:32,252][71635] Updated weights for policy 1, policy_version 32452 (0.0008) [2023-10-11 20:22:32,620][71635] Updated weights for policy 1, policy_version 32462 (0.0008) [2023-10-11 20:22:32,992][71635] Updated weights for policy 1, policy_version 32472 (0.0008) [2023-10-11 20:22:34,793][71601] Updated weights for policy 0, policy_version 32490 (0.0008) [2023-10-11 20:22:35,163][71601] Updated weights for policy 0, policy_version 32500 (0.0008) [2023-10-11 20:22:35,531][71601] Updated weights for policy 0, policy_version 32510 (0.0008) [2023-10-11 20:22:36,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 66551808. Throughput: 0: 1832.5, 1: 1828.8. Samples: 16642972. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-11 20:22:36,034][70582] Avg episode reward: [(0, '189.960'), (1, '163.170')] [2023-10-11 20:22:36,602][71635] Updated weights for policy 1, policy_version 32482 (0.0009) [2023-10-11 20:22:36,965][71635] Updated weights for policy 1, policy_version 32492 (0.0008) [2023-10-11 20:22:37,346][71635] Updated weights for policy 1, policy_version 32502 (0.0008) [2023-10-11 20:22:37,703][71635] Updated weights for policy 1, policy_version 32512 (0.0008) [2023-10-11 20:22:39,189][71601] Updated weights for policy 0, policy_version 32520 (0.0008) [2023-10-11 20:22:39,559][71601] Updated weights for policy 0, policy_version 32530 (0.0009) [2023-10-11 20:22:39,933][71601] Updated weights for policy 0, policy_version 32540 (0.0007) [2023-10-11 20:22:41,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 66617344. Throughput: 0: 1830.6, 1: 1822.5. Samples: 16664542. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-11 20:22:41,034][70582] Avg episode reward: [(0, '187.150'), (1, '152.720')] [2023-10-11 20:22:41,447][71635] Updated weights for policy 1, policy_version 32522 (0.0009) [2023-10-11 20:22:41,814][71635] Updated weights for policy 1, policy_version 32532 (0.0011) [2023-10-11 20:22:42,182][71635] Updated weights for policy 1, policy_version 32542 (0.0011) [2023-10-11 20:22:43,627][71601] Updated weights for policy 0, policy_version 32550 (0.0009) [2023-10-11 20:22:44,009][71601] Updated weights for policy 0, policy_version 32560 (0.0008) [2023-10-11 20:22:44,366][71601] Updated weights for policy 0, policy_version 32570 (0.0010) [2023-10-11 20:22:45,884][71635] Updated weights for policy 1, policy_version 32552 (0.0011) [2023-10-11 20:22:46,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 66682880. Throughput: 0: 1831.0, 1: 1821.5. Samples: 16675796. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-11 20:22:46,035][70582] Avg episode reward: [(0, '179.670'), (1, '154.310')] [2023-10-11 20:22:46,248][71635] Updated weights for policy 1, policy_version 32562 (0.0009) [2023-10-11 20:22:46,620][71635] Updated weights for policy 1, policy_version 32572 (0.0008) [2023-10-11 20:22:48,025][71601] Updated weights for policy 0, policy_version 32580 (0.0010) [2023-10-11 20:22:48,393][71601] Updated weights for policy 0, policy_version 32590 (0.0007) [2023-10-11 20:22:48,761][71601] Updated weights for policy 0, policy_version 32600 (0.0007) [2023-10-11 20:22:50,441][71635] Updated weights for policy 1, policy_version 32582 (0.0009) [2023-10-11 20:22:50,796][71635] Updated weights for policy 1, policy_version 32592 (0.0008) [2023-10-11 20:22:51,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 66748416. Throughput: 0: 1825.1, 1: 1823.0. Samples: 16697084. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-11 20:22:51,034][70582] Avg episode reward: [(0, '179.670'), (1, '156.560')] [2023-10-11 20:22:51,163][71635] Updated weights for policy 1, policy_version 32602 (0.0007) [2023-10-11 20:22:52,449][71601] Updated weights for policy 0, policy_version 32610 (0.0009) [2023-10-11 20:22:52,818][71601] Updated weights for policy 0, policy_version 32620 (0.0009) [2023-10-11 20:22:53,192][71601] Updated weights for policy 0, policy_version 32630 (0.0007) [2023-10-11 20:22:53,555][71601] Updated weights for policy 0, policy_version 32640 (0.0007) [2023-10-11 20:22:54,838][71635] Updated weights for policy 1, policy_version 32612 (0.0007) [2023-10-11 20:22:55,197][71635] Updated weights for policy 1, policy_version 32622 (0.0009) [2023-10-11 20:22:55,557][71635] Updated weights for policy 1, policy_version 32632 (0.0007) [2023-10-11 20:22:56,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 66846720. Throughput: 0: 1829.1, 1: 1815.4. Samples: 16719010. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:22:56,035][70582] Avg episode reward: [(0, '175.700'), (1, '158.980')] [2023-10-11 20:22:57,493][71601] Updated weights for policy 0, policy_version 32650 (0.0009) [2023-10-11 20:22:57,863][71601] Updated weights for policy 0, policy_version 32660 (0.0009) [2023-10-11 20:22:58,233][71601] Updated weights for policy 0, policy_version 32670 (0.0010) [2023-10-11 20:22:59,451][71635] Updated weights for policy 1, policy_version 32642 (0.0007) [2023-10-11 20:22:59,823][71635] Updated weights for policy 1, policy_version 32652 (0.0007) [2023-10-11 20:23:00,187][71635] Updated weights for policy 1, policy_version 32662 (0.0008) [2023-10-11 20:23:00,543][71635] Updated weights for policy 1, policy_version 32672 (0.0010) [2023-10-11 20:23:01,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 66912256. Throughput: 0: 1825.6, 1: 1809.7. Samples: 16729442. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:23:01,035][70582] Avg episode reward: [(0, '175.730'), (1, '153.220')] [2023-10-11 20:23:02,014][71601] Updated weights for policy 0, policy_version 32680 (0.0007) [2023-10-11 20:23:02,397][71601] Updated weights for policy 0, policy_version 32690 (0.0008) [2023-10-11 20:23:02,763][71601] Updated weights for policy 0, policy_version 32700 (0.0009) [2023-10-11 20:23:04,214][71635] Updated weights for policy 1, policy_version 32682 (0.0010) [2023-10-11 20:23:04,592][71635] Updated weights for policy 1, policy_version 32692 (0.0008) [2023-10-11 20:23:04,950][71635] Updated weights for policy 1, policy_version 32702 (0.0008) [2023-10-11 20:23:06,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 66977792. Throughput: 0: 1822.9, 1: 1811.3. Samples: 16751626. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:23:06,034][70582] Avg episode reward: [(0, '178.240'), (1, '139.490')] [2023-10-11 20:23:06,439][71601] Updated weights for policy 0, policy_version 32710 (0.0007) [2023-10-11 20:23:06,814][71601] Updated weights for policy 0, policy_version 32720 (0.0008) [2023-10-11 20:23:07,180][71601] Updated weights for policy 0, policy_version 32730 (0.0009) [2023-10-11 20:23:08,543][71635] Updated weights for policy 1, policy_version 32712 (0.0007) [2023-10-11 20:23:08,900][71635] Updated weights for policy 1, policy_version 32722 (0.0009) [2023-10-11 20:23:09,263][71635] Updated weights for policy 1, policy_version 32732 (0.0010) [2023-10-11 20:23:10,876][71601] Updated weights for policy 0, policy_version 32740 (0.0009) [2023-10-11 20:23:11,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 67043328. Throughput: 0: 1823.7, 1: 1815.2. Samples: 16774062. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:23:11,034][70582] Avg episode reward: [(0, '170.780'), (1, '143.540')] [2023-10-11 20:23:11,246][71601] Updated weights for policy 0, policy_version 32750 (0.0009) [2023-10-11 20:23:11,616][71601] Updated weights for policy 0, policy_version 32760 (0.0010) [2023-10-11 20:23:13,011][71635] Updated weights for policy 1, policy_version 32742 (0.0008) [2023-10-11 20:23:13,375][71635] Updated weights for policy 1, policy_version 32752 (0.0008) [2023-10-11 20:23:13,744][71635] Updated weights for policy 1, policy_version 32762 (0.0009) [2023-10-11 20:23:15,163][71601] Updated weights for policy 0, policy_version 32770 (0.0007) [2023-10-11 20:23:15,533][71601] Updated weights for policy 0, policy_version 32780 (0.0008) [2023-10-11 20:23:15,916][71601] Updated weights for policy 0, policy_version 32790 (0.0008) [2023-10-11 20:23:16,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 67108864. Throughput: 0: 1825.3, 1: 1818.6. Samples: 16784816. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:23:16,034][70582] Avg episode reward: [(0, '170.970'), (1, '137.710')] [2023-10-11 20:23:16,282][71601] Updated weights for policy 0, policy_version 32800 (0.0011) [2023-10-11 20:23:17,499][71635] Updated weights for policy 1, policy_version 32772 (0.0011) [2023-10-11 20:23:17,862][71635] Updated weights for policy 1, policy_version 32782 (0.0010) [2023-10-11 20:23:18,224][71635] Updated weights for policy 1, policy_version 32792 (0.0010) [2023-10-11 20:23:19,935][71601] Updated weights for policy 0, policy_version 32810 (0.0008) [2023-10-11 20:23:20,302][71601] Updated weights for policy 0, policy_version 32820 (0.0007) [2023-10-11 20:23:20,667][71601] Updated weights for policy 0, policy_version 32830 (0.0009) [2023-10-11 20:23:21,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 67207168. Throughput: 0: 1820.4, 1: 1819.4. Samples: 16806764. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 20:23:21,035][70582] Avg episode reward: [(0, '171.050'), (1, '125.270')] [2023-10-11 20:23:21,963][71635] Updated weights for policy 1, policy_version 32802 (0.0010) [2023-10-11 20:23:22,320][71635] Updated weights for policy 1, policy_version 32812 (0.0008) [2023-10-11 20:23:22,685][71635] Updated weights for policy 1, policy_version 32822 (0.0008) [2023-10-11 20:23:23,055][71635] Updated weights for policy 1, policy_version 32832 (0.0008) [2023-10-11 20:23:24,300][71601] Updated weights for policy 0, policy_version 32840 (0.0008) [2023-10-11 20:23:24,679][71601] Updated weights for policy 0, policy_version 32850 (0.0009) [2023-10-11 20:23:25,060][71601] Updated weights for policy 0, policy_version 32860 (0.0008) [2023-10-11 20:23:26,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 67272704. Throughput: 0: 1820.7, 1: 1814.3. Samples: 16828114. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 20:23:26,035][70582] Avg episode reward: [(0, '171.050'), (1, '125.350')] [2023-10-11 20:23:26,743][71635] Updated weights for policy 1, policy_version 32842 (0.0009) [2023-10-11 20:23:27,110][71635] Updated weights for policy 1, policy_version 32852 (0.0009) [2023-10-11 20:23:27,470][71635] Updated weights for policy 1, policy_version 32862 (0.0007) [2023-10-11 20:23:28,814][71601] Updated weights for policy 0, policy_version 32870 (0.0008) [2023-10-11 20:23:29,195][71601] Updated weights for policy 0, policy_version 32880 (0.0009) [2023-10-11 20:23:29,570][71601] Updated weights for policy 0, policy_version 32890 (0.0009) [2023-10-11 20:23:31,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 67338240. Throughput: 0: 1822.9, 1: 1815.6. Samples: 16839528. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 20:23:31,035][70582] Avg episode reward: [(0, '171.050'), (1, '123.740')] [2023-10-11 20:23:31,295][71635] Updated weights for policy 1, policy_version 32872 (0.0010) [2023-10-11 20:23:31,676][71635] Updated weights for policy 1, policy_version 32882 (0.0009) [2023-10-11 20:23:32,055][71635] Updated weights for policy 1, policy_version 32892 (0.0010) [2023-10-11 20:23:33,111][71601] Updated weights for policy 0, policy_version 32900 (0.0011) [2023-10-11 20:23:33,475][71601] Updated weights for policy 0, policy_version 32910 (0.0007) [2023-10-11 20:23:33,856][71601] Updated weights for policy 0, policy_version 32920 (0.0007) [2023-10-11 20:23:35,735][71635] Updated weights for policy 1, policy_version 32902 (0.0009) [2023-10-11 20:23:36,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 67403776. Throughput: 0: 1820.8, 1: 1809.4. Samples: 16860444. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 20:23:36,034][70582] Avg episode reward: [(0, '171.050'), (1, '123.500')] [2023-10-11 20:23:36,094][71635] Updated weights for policy 1, policy_version 32912 (0.0007) [2023-10-11 20:23:36,467][71635] Updated weights for policy 1, policy_version 32922 (0.0008) [2023-10-11 20:23:37,575][71601] Updated weights for policy 0, policy_version 32930 (0.0008) [2023-10-11 20:23:37,943][71601] Updated weights for policy 0, policy_version 32940 (0.0010) [2023-10-11 20:23:38,312][71601] Updated weights for policy 0, policy_version 32950 (0.0010) [2023-10-11 20:23:38,684][71601] Updated weights for policy 0, policy_version 32960 (0.0010) [2023-10-11 20:23:40,209][71635] Updated weights for policy 1, policy_version 32932 (0.0009) [2023-10-11 20:23:40,569][71635] Updated weights for policy 1, policy_version 32942 (0.0007) [2023-10-11 20:23:40,933][71635] Updated weights for policy 1, policy_version 32952 (0.0009) [2023-10-11 20:23:41,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 67469312. Throughput: 0: 1817.3, 1: 1817.6. Samples: 16882582. Policy #0 lag: (min: 6.0, avg: 13.6, max: 38.0) [2023-10-11 20:23:41,034][70582] Avg episode reward: [(0, '171.050'), (1, '107.490')] [2023-10-11 20:23:42,498][71601] Updated weights for policy 0, policy_version 32970 (0.0008) [2023-10-11 20:23:42,871][71601] Updated weights for policy 0, policy_version 32980 (0.0009) [2023-10-11 20:23:43,237][71601] Updated weights for policy 0, policy_version 32990 (0.0009) [2023-10-11 20:23:44,728][71635] Updated weights for policy 1, policy_version 32962 (0.0007) [2023-10-11 20:23:45,096][71635] Updated weights for policy 1, policy_version 32972 (0.0009) [2023-10-11 20:23:45,468][71635] Updated weights for policy 1, policy_version 32982 (0.0009) [2023-10-11 20:23:45,830][71635] Updated weights for policy 1, policy_version 32992 (0.0009) [2023-10-11 20:23:46,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 67567616. Throughput: 0: 1820.7, 1: 1813.7. Samples: 16892990. Policy #0 lag: (min: 6.0, avg: 13.6, max: 38.0) [2023-10-11 20:23:46,035][70582] Avg episode reward: [(0, '176.300'), (1, '119.900')] [2023-10-11 20:23:46,807][71601] Updated weights for policy 0, policy_version 33000 (0.0008) [2023-10-11 20:23:47,168][71601] Updated weights for policy 0, policy_version 33010 (0.0007) [2023-10-11 20:23:47,548][71601] Updated weights for policy 0, policy_version 33020 (0.0009) [2023-10-11 20:23:49,536][71635] Updated weights for policy 1, policy_version 33002 (0.0008) [2023-10-11 20:23:49,908][71635] Updated weights for policy 1, policy_version 33012 (0.0010) [2023-10-11 20:23:50,284][71635] Updated weights for policy 1, policy_version 33022 (0.0008) [2023-10-11 20:23:51,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 67633152. Throughput: 0: 1821.2, 1: 1820.3. Samples: 16915494. Policy #0 lag: (min: 6.0, avg: 13.6, max: 38.0) [2023-10-11 20:23:51,034][70582] Avg episode reward: [(0, '167.070'), (1, '114.160')] [2023-10-11 20:23:51,292][71601] Updated weights for policy 0, policy_version 33030 (0.0009) [2023-10-11 20:23:51,670][71601] Updated weights for policy 0, policy_version 33040 (0.0008) [2023-10-11 20:23:52,036][71601] Updated weights for policy 0, policy_version 33050 (0.0008) [2023-10-11 20:23:53,822][71635] Updated weights for policy 1, policy_version 33032 (0.0007) [2023-10-11 20:23:54,191][71635] Updated weights for policy 1, policy_version 33042 (0.0010) [2023-10-11 20:23:54,560][71635] Updated weights for policy 1, policy_version 33052 (0.0009) [2023-10-11 20:23:55,736][71601] Updated weights for policy 0, policy_version 33060 (0.0009) [2023-10-11 20:23:56,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 67698688. Throughput: 0: 1814.8, 1: 1814.0. Samples: 16937360. Policy #0 lag: (min: 6.0, avg: 13.6, max: 38.0) [2023-10-11 20:23:56,034][70582] Avg episode reward: [(0, '177.090'), (1, '112.250')] [2023-10-11 20:23:56,044][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000033056_33849344.pth... [2023-10-11 20:23:56,083][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000031360_32112640.pth [2023-10-11 20:23:56,110][71601] Updated weights for policy 0, policy_version 33070 (0.0010) [2023-10-11 20:23:56,493][71601] Updated weights for policy 0, policy_version 33080 (0.0010) [2023-10-11 20:23:56,787][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000033088_33882112.pth... [2023-10-11 20:23:56,816][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000031360_32112640.pth [2023-10-11 20:23:58,230][71635] Updated weights for policy 1, policy_version 33062 (0.0009) [2023-10-11 20:23:58,610][71635] Updated weights for policy 1, policy_version 33072 (0.0008) [2023-10-11 20:23:58,977][71635] Updated weights for policy 1, policy_version 33082 (0.0007) [2023-10-11 20:24:00,252][71601] Updated weights for policy 0, policy_version 33090 (0.0008) [2023-10-11 20:24:00,611][71601] Updated weights for policy 0, policy_version 33100 (0.0007) [2023-10-11 20:24:00,988][71601] Updated weights for policy 0, policy_version 33110 (0.0008) [2023-10-11 20:24:01,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 67764224. Throughput: 0: 1811.6, 1: 1822.2. Samples: 16948336. Policy #0 lag: (min: 6.0, avg: 13.6, max: 38.0) [2023-10-11 20:24:01,034][70582] Avg episode reward: [(0, '153.990'), (1, '105.560')] [2023-10-11 20:24:01,363][71601] Updated weights for policy 0, policy_version 33120 (0.0008) [2023-10-11 20:24:02,714][71635] Updated weights for policy 1, policy_version 33092 (0.0008) [2023-10-11 20:24:03,079][71635] Updated weights for policy 1, policy_version 33102 (0.0008) [2023-10-11 20:24:03,454][71635] Updated weights for policy 1, policy_version 33112 (0.0007) [2023-10-11 20:24:04,996][71601] Updated weights for policy 0, policy_version 33130 (0.0010) [2023-10-11 20:24:05,369][71601] Updated weights for policy 0, policy_version 33140 (0.0010) [2023-10-11 20:24:05,741][71601] Updated weights for policy 0, policy_version 33150 (0.0009) [2023-10-11 20:24:06,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 67862528. Throughput: 0: 1816.2, 1: 1812.4. Samples: 16970048. Policy #0 lag: (min: 1.0, avg: 9.7, max: 33.0) [2023-10-11 20:24:06,034][70582] Avg episode reward: [(0, '153.990'), (1, '102.410')] [2023-10-11 20:24:07,089][71635] Updated weights for policy 1, policy_version 33122 (0.0009) [2023-10-11 20:24:07,450][71635] Updated weights for policy 1, policy_version 33132 (0.0008) [2023-10-11 20:24:07,814][71635] Updated weights for policy 1, policy_version 33142 (0.0008) [2023-10-11 20:24:08,179][71635] Updated weights for policy 1, policy_version 33152 (0.0008) [2023-10-11 20:24:09,379][71601] Updated weights for policy 0, policy_version 33160 (0.0008) [2023-10-11 20:24:09,748][71601] Updated weights for policy 0, policy_version 33170 (0.0008) [2023-10-11 20:24:10,123][71601] Updated weights for policy 0, policy_version 33180 (0.0007) [2023-10-11 20:24:11,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 67928064. Throughput: 0: 1815.3, 1: 1813.6. Samples: 16991414. Policy #0 lag: (min: 1.0, avg: 9.7, max: 33.0) [2023-10-11 20:24:11,034][70582] Avg episode reward: [(0, '155.270'), (1, '100.130')] [2023-10-11 20:24:11,822][71635] Updated weights for policy 1, policy_version 33162 (0.0007) [2023-10-11 20:24:12,190][71635] Updated weights for policy 1, policy_version 33172 (0.0009) [2023-10-11 20:24:12,563][71635] Updated weights for policy 1, policy_version 33182 (0.0008) [2023-10-11 20:24:13,744][71601] Updated weights for policy 0, policy_version 33190 (0.0009) [2023-10-11 20:24:14,119][71601] Updated weights for policy 0, policy_version 33200 (0.0010) [2023-10-11 20:24:14,488][71601] Updated weights for policy 0, policy_version 33210 (0.0007) [2023-10-11 20:24:16,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 67993600. Throughput: 0: 1818.0, 1: 1812.7. Samples: 17002912. Policy #0 lag: (min: 1.0, avg: 9.7, max: 33.0) [2023-10-11 20:24:16,035][70582] Avg episode reward: [(0, '144.660'), (1, '95.770')] [2023-10-11 20:24:16,202][71635] Updated weights for policy 1, policy_version 33192 (0.0007) [2023-10-11 20:24:16,578][71635] Updated weights for policy 1, policy_version 33202 (0.0009) [2023-10-11 20:24:16,944][71635] Updated weights for policy 1, policy_version 33212 (0.0008) [2023-10-11 20:24:18,143][71601] Updated weights for policy 0, policy_version 33220 (0.0008) [2023-10-11 20:24:18,513][71601] Updated weights for policy 0, policy_version 33230 (0.0007) [2023-10-11 20:24:18,882][71601] Updated weights for policy 0, policy_version 33240 (0.0009) [2023-10-11 20:24:20,589][71635] Updated weights for policy 1, policy_version 33222 (0.0008) [2023-10-11 20:24:20,957][71635] Updated weights for policy 1, policy_version 33232 (0.0008) [2023-10-11 20:24:21,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 68059136. Throughput: 0: 1815.4, 1: 1823.1. Samples: 17024176. Policy #0 lag: (min: 1.0, avg: 9.7, max: 33.0) [2023-10-11 20:24:21,034][70582] Avg episode reward: [(0, '144.950'), (1, '110.450')] [2023-10-11 20:24:21,328][71635] Updated weights for policy 1, policy_version 33242 (0.0008) [2023-10-11 20:24:22,404][71601] Updated weights for policy 0, policy_version 33250 (0.0007) [2023-10-11 20:24:22,782][71601] Updated weights for policy 0, policy_version 33260 (0.0010) [2023-10-11 20:24:23,144][71601] Updated weights for policy 0, policy_version 33270 (0.0009) [2023-10-11 20:24:23,516][71601] Updated weights for policy 0, policy_version 33280 (0.0009) [2023-10-11 20:24:25,079][71635] Updated weights for policy 1, policy_version 33252 (0.0009) [2023-10-11 20:24:25,446][71635] Updated weights for policy 1, policy_version 33262 (0.0009) [2023-10-11 20:24:25,809][71635] Updated weights for policy 1, policy_version 33272 (0.0009) [2023-10-11 20:24:26,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 68124672. Throughput: 0: 1828.1, 1: 1822.4. Samples: 17046854. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-11 20:24:26,034][70582] Avg episode reward: [(0, '144.950'), (1, '107.060')] [2023-10-11 20:24:27,278][71601] Updated weights for policy 0, policy_version 33290 (0.0010) [2023-10-11 20:24:27,654][71601] Updated weights for policy 0, policy_version 33300 (0.0009) [2023-10-11 20:24:28,022][71601] Updated weights for policy 0, policy_version 33310 (0.0007) [2023-10-11 20:24:29,408][71635] Updated weights for policy 1, policy_version 33282 (0.0010) [2023-10-11 20:24:29,779][71635] Updated weights for policy 1, policy_version 33292 (0.0009) [2023-10-11 20:24:30,147][71635] Updated weights for policy 1, policy_version 33302 (0.0009) [2023-10-11 20:24:30,512][71635] Updated weights for policy 1, policy_version 33312 (0.0008) [2023-10-11 20:24:31,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 68222976. Throughput: 0: 1825.6, 1: 1825.5. Samples: 17057292. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-11 20:24:31,035][70582] Avg episode reward: [(0, '145.110'), (1, '106.520')] [2023-10-11 20:24:31,623][71601] Updated weights for policy 0, policy_version 33320 (0.0008) [2023-10-11 20:24:31,996][71601] Updated weights for policy 0, policy_version 33330 (0.0007) [2023-10-11 20:24:32,361][71601] Updated weights for policy 0, policy_version 33340 (0.0009) [2023-10-11 20:24:34,226][71635] Updated weights for policy 1, policy_version 33322 (0.0008) [2023-10-11 20:24:34,599][71635] Updated weights for policy 1, policy_version 33332 (0.0011) [2023-10-11 20:24:34,966][71635] Updated weights for policy 1, policy_version 33342 (0.0007) [2023-10-11 20:24:36,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 68288512. Throughput: 0: 1824.5, 1: 1821.6. Samples: 17079566. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-11 20:24:36,034][70582] Avg episode reward: [(0, '145.470'), (1, '108.020')] [2023-10-11 20:24:36,137][71601] Updated weights for policy 0, policy_version 33350 (0.0010) [2023-10-11 20:24:36,497][71601] Updated weights for policy 0, policy_version 33360 (0.0010) [2023-10-11 20:24:36,866][71601] Updated weights for policy 0, policy_version 33370 (0.0010) [2023-10-11 20:24:38,594][71635] Updated weights for policy 1, policy_version 33352 (0.0008) [2023-10-11 20:24:38,955][71635] Updated weights for policy 1, policy_version 33362 (0.0008) [2023-10-11 20:24:39,323][71635] Updated weights for policy 1, policy_version 33372 (0.0008) [2023-10-11 20:24:40,562][71601] Updated weights for policy 0, policy_version 33380 (0.0008) [2023-10-11 20:24:40,929][71601] Updated weights for policy 0, policy_version 33390 (0.0007) [2023-10-11 20:24:41,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 68354048. Throughput: 0: 1825.2, 1: 1822.2. Samples: 17101496. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-11 20:24:41,034][70582] Avg episode reward: [(0, '142.520'), (1, '100.720')] [2023-10-11 20:24:41,312][71601] Updated weights for policy 0, policy_version 33400 (0.0010) [2023-10-11 20:24:42,914][71635] Updated weights for policy 1, policy_version 33382 (0.0009) [2023-10-11 20:24:43,285][71635] Updated weights for policy 1, policy_version 33392 (0.0008) [2023-10-11 20:24:43,653][71635] Updated weights for policy 1, policy_version 33402 (0.0009) [2023-10-11 20:24:44,995][71601] Updated weights for policy 0, policy_version 33410 (0.0008) [2023-10-11 20:24:45,374][71601] Updated weights for policy 0, policy_version 33420 (0.0008) [2023-10-11 20:24:45,741][71601] Updated weights for policy 0, policy_version 33430 (0.0008) [2023-10-11 20:24:46,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 68419584. Throughput: 0: 1830.8, 1: 1813.3. Samples: 17112322. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-11 20:24:46,035][70582] Avg episode reward: [(0, '144.170'), (1, '100.720')] [2023-10-11 20:24:46,116][71601] Updated weights for policy 0, policy_version 33440 (0.0009) [2023-10-11 20:24:47,351][71635] Updated weights for policy 1, policy_version 33412 (0.0008) [2023-10-11 20:24:47,720][71635] Updated weights for policy 1, policy_version 33422 (0.0010) [2023-10-11 20:24:48,092][71635] Updated weights for policy 1, policy_version 33432 (0.0011) [2023-10-11 20:24:49,746][71601] Updated weights for policy 0, policy_version 33450 (0.0009) [2023-10-11 20:24:50,111][71601] Updated weights for policy 0, policy_version 33460 (0.0008) [2023-10-11 20:24:50,487][71601] Updated weights for policy 0, policy_version 33470 (0.0009) [2023-10-11 20:24:51,034][70582] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 68517888. Throughput: 0: 1830.6, 1: 1818.6. Samples: 17134264. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-11 20:24:51,035][70582] Avg episode reward: [(0, '144.240'), (1, '117.300')] [2023-10-11 20:24:51,754][71635] Updated weights for policy 1, policy_version 33442 (0.0010) [2023-10-11 20:24:52,115][71635] Updated weights for policy 1, policy_version 33452 (0.0007) [2023-10-11 20:24:52,492][71635] Updated weights for policy 1, policy_version 33462 (0.0010) [2023-10-11 20:24:52,851][71635] Updated weights for policy 1, policy_version 33472 (0.0008) [2023-10-11 20:24:54,014][71601] Updated weights for policy 0, policy_version 33480 (0.0007) [2023-10-11 20:24:54,390][71601] Updated weights for policy 0, policy_version 33490 (0.0008) [2023-10-11 20:24:54,756][71601] Updated weights for policy 0, policy_version 33500 (0.0009) [2023-10-11 20:24:56,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 68583424. Throughput: 0: 1836.1, 1: 1826.1. Samples: 17156214. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-11 20:24:56,035][70582] Avg episode reward: [(0, '161.010'), (1, '122.370')] [2023-10-11 20:24:56,645][71635] Updated weights for policy 1, policy_version 33482 (0.0009) [2023-10-11 20:24:57,011][71635] Updated weights for policy 1, policy_version 33492 (0.0007) [2023-10-11 20:24:57,387][71635] Updated weights for policy 1, policy_version 33502 (0.0007) [2023-10-11 20:24:58,459][71601] Updated weights for policy 0, policy_version 33510 (0.0007) [2023-10-11 20:24:58,833][71601] Updated weights for policy 0, policy_version 33520 (0.0007) [2023-10-11 20:24:59,192][71601] Updated weights for policy 0, policy_version 33530 (0.0009) [2023-10-11 20:25:01,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 68648960. Throughput: 0: 1827.0, 1: 1827.1. Samples: 17167346. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-11 20:25:01,035][70582] Avg episode reward: [(0, '160.640'), (1, '108.720')] [2023-10-11 20:25:01,286][71635] Updated weights for policy 1, policy_version 33512 (0.0008) [2023-10-11 20:25:01,656][71635] Updated weights for policy 1, policy_version 33522 (0.0007) [2023-10-11 20:25:02,028][71635] Updated weights for policy 1, policy_version 33532 (0.0007) [2023-10-11 20:25:02,903][71601] Updated weights for policy 0, policy_version 33540 (0.0009) [2023-10-11 20:25:03,279][71601] Updated weights for policy 0, policy_version 33550 (0.0008) [2023-10-11 20:25:03,640][71601] Updated weights for policy 0, policy_version 33560 (0.0008) [2023-10-11 20:25:05,691][71635] Updated weights for policy 1, policy_version 33542 (0.0009) [2023-10-11 20:25:06,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 68714496. Throughput: 0: 1834.5, 1: 1824.3. Samples: 17188822. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-11 20:25:06,035][70582] Avg episode reward: [(0, '148.750'), (1, '115.740')] [2023-10-11 20:25:06,066][71635] Updated weights for policy 1, policy_version 33552 (0.0008) [2023-10-11 20:25:06,441][71635] Updated weights for policy 1, policy_version 33562 (0.0011) [2023-10-11 20:25:07,337][71601] Updated weights for policy 0, policy_version 33570 (0.0008) [2023-10-11 20:25:07,701][71601] Updated weights for policy 0, policy_version 33580 (0.0009) [2023-10-11 20:25:08,069][71601] Updated weights for policy 0, policy_version 33590 (0.0009) [2023-10-11 20:25:08,443][71601] Updated weights for policy 0, policy_version 33600 (0.0009) [2023-10-11 20:25:10,098][71635] Updated weights for policy 1, policy_version 33572 (0.0009) [2023-10-11 20:25:10,467][71635] Updated weights for policy 1, policy_version 33582 (0.0009) [2023-10-11 20:25:10,837][71635] Updated weights for policy 1, policy_version 33592 (0.0007) [2023-10-11 20:25:11,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 68780032. Throughput: 0: 1828.2, 1: 1824.4. Samples: 17211224. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-11 20:25:11,034][70582] Avg episode reward: [(0, '160.430'), (1, '135.000')] [2023-10-11 20:25:12,040][71601] Updated weights for policy 0, policy_version 33610 (0.0010) [2023-10-11 20:25:12,404][71601] Updated weights for policy 0, policy_version 33620 (0.0010) [2023-10-11 20:25:12,777][71601] Updated weights for policy 0, policy_version 33630 (0.0010) [2023-10-11 20:25:14,355][71635] Updated weights for policy 1, policy_version 33602 (0.0010) [2023-10-11 20:25:14,722][71635] Updated weights for policy 1, policy_version 33612 (0.0007) [2023-10-11 20:25:15,096][71635] Updated weights for policy 1, policy_version 33622 (0.0008) [2023-10-11 20:25:15,464][71635] Updated weights for policy 1, policy_version 33632 (0.0008) [2023-10-11 20:25:16,034][70582] Fps is (10 sec: 16384.7, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 68878336. Throughput: 0: 1832.0, 1: 1827.1. Samples: 17221950. Policy #0 lag: (min: 17.0, avg: 24.6, max: 49.0) [2023-10-11 20:25:16,034][70582] Avg episode reward: [(0, '160.390'), (1, '133.770')] [2023-10-11 20:25:16,256][71601] Updated weights for policy 0, policy_version 33640 (0.0008) [2023-10-11 20:25:16,628][71601] Updated weights for policy 0, policy_version 33650 (0.0007) [2023-10-11 20:25:17,004][71601] Updated weights for policy 0, policy_version 33660 (0.0009) [2023-10-11 20:25:19,243][71635] Updated weights for policy 1, policy_version 33642 (0.0007) [2023-10-11 20:25:19,610][71635] Updated weights for policy 1, policy_version 33652 (0.0008) [2023-10-11 20:25:19,963][71635] Updated weights for policy 1, policy_version 33662 (0.0009) [2023-10-11 20:25:20,691][71601] Updated weights for policy 0, policy_version 33670 (0.0010) [2023-10-11 20:25:21,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 68943872. Throughput: 0: 1837.9, 1: 1821.3. Samples: 17244228. Policy #0 lag: (min: 17.0, avg: 24.6, max: 49.0) [2023-10-11 20:25:21,035][70582] Avg episode reward: [(0, '159.260'), (1, '134.090')] [2023-10-11 20:25:21,050][71601] Updated weights for policy 0, policy_version 33680 (0.0008) [2023-10-11 20:25:21,418][71601] Updated weights for policy 0, policy_version 33690 (0.0007) [2023-10-11 20:25:23,587][71635] Updated weights for policy 1, policy_version 33672 (0.0009) [2023-10-11 20:25:23,944][71635] Updated weights for policy 1, policy_version 33682 (0.0009) [2023-10-11 20:25:24,305][71635] Updated weights for policy 1, policy_version 33692 (0.0009) [2023-10-11 20:25:24,993][71601] Updated weights for policy 0, policy_version 33700 (0.0008) [2023-10-11 20:25:25,369][71601] Updated weights for policy 0, policy_version 33710 (0.0008) [2023-10-11 20:25:25,738][71601] Updated weights for policy 0, policy_version 33720 (0.0009) [2023-10-11 20:25:26,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 69009408. Throughput: 0: 1830.0, 1: 1825.2. Samples: 17265982. Policy #0 lag: (min: 17.0, avg: 24.6, max: 49.0) [2023-10-11 20:25:26,034][70582] Avg episode reward: [(0, '156.350'), (1, '139.340')] [2023-10-11 20:25:28,033][71635] Updated weights for policy 1, policy_version 33702 (0.0009) [2023-10-11 20:25:28,395][71635] Updated weights for policy 1, policy_version 33712 (0.0011) [2023-10-11 20:25:28,768][71635] Updated weights for policy 1, policy_version 33722 (0.0009) [2023-10-11 20:25:29,398][71601] Updated weights for policy 0, policy_version 33730 (0.0008) [2023-10-11 20:25:29,755][71601] Updated weights for policy 0, policy_version 33740 (0.0007) [2023-10-11 20:25:30,141][71601] Updated weights for policy 0, policy_version 33750 (0.0009) [2023-10-11 20:25:30,513][71601] Updated weights for policy 0, policy_version 33760 (0.0008) [2023-10-11 20:25:31,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 69107712. Throughput: 0: 1845.3, 1: 1826.6. Samples: 17277560. Policy #0 lag: (min: 17.0, avg: 24.6, max: 49.0) [2023-10-11 20:25:31,035][70582] Avg episode reward: [(0, '168.240'), (1, '137.030')] [2023-10-11 20:25:32,461][71635] Updated weights for policy 1, policy_version 33732 (0.0009) [2023-10-11 20:25:32,826][71635] Updated weights for policy 1, policy_version 33742 (0.0010) [2023-10-11 20:25:33,199][71635] Updated weights for policy 1, policy_version 33752 (0.0009) [2023-10-11 20:25:34,301][71601] Updated weights for policy 0, policy_version 33770 (0.0011) [2023-10-11 20:25:34,670][71601] Updated weights for policy 0, policy_version 33780 (0.0010) [2023-10-11 20:25:35,039][71601] Updated weights for policy 0, policy_version 33790 (0.0010) [2023-10-11 20:25:36,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 69173248. Throughput: 0: 1830.9, 1: 1826.2. Samples: 17298832. Policy #0 lag: (min: 21.0, avg: 25.8, max: 53.0) [2023-10-11 20:25:36,035][70582] Avg episode reward: [(0, '168.240'), (1, '129.920')] [2023-10-11 20:25:36,805][71635] Updated weights for policy 1, policy_version 33762 (0.0008) [2023-10-11 20:25:37,166][71635] Updated weights for policy 1, policy_version 33772 (0.0009) [2023-10-11 20:25:37,537][71635] Updated weights for policy 1, policy_version 33782 (0.0007) [2023-10-11 20:25:37,893][71635] Updated weights for policy 1, policy_version 33792 (0.0008) [2023-10-11 20:25:38,494][71601] Updated weights for policy 0, policy_version 33800 (0.0009) [2023-10-11 20:25:38,869][71601] Updated weights for policy 0, policy_version 33810 (0.0008) [2023-10-11 20:25:39,234][71601] Updated weights for policy 0, policy_version 33820 (0.0008) [2023-10-11 20:25:41,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 69238784. Throughput: 0: 1843.3, 1: 1822.4. Samples: 17321174. Policy #0 lag: (min: 21.0, avg: 25.8, max: 53.0) [2023-10-11 20:25:41,035][70582] Avg episode reward: [(0, '168.720'), (1, '138.270')] [2023-10-11 20:25:41,592][71635] Updated weights for policy 1, policy_version 33802 (0.0008) [2023-10-11 20:25:41,966][71635] Updated weights for policy 1, policy_version 33812 (0.0008) [2023-10-11 20:25:42,340][71635] Updated weights for policy 1, policy_version 33822 (0.0008) [2023-10-11 20:25:42,969][71601] Updated weights for policy 0, policy_version 33830 (0.0008) [2023-10-11 20:25:43,348][71601] Updated weights for policy 0, policy_version 33840 (0.0008) [2023-10-11 20:25:43,710][71601] Updated weights for policy 0, policy_version 33850 (0.0008) [2023-10-11 20:25:45,878][71635] Updated weights for policy 1, policy_version 33832 (0.0008) [2023-10-11 20:25:46,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 69304320. Throughput: 0: 1831.2, 1: 1825.6. Samples: 17331904. Policy #0 lag: (min: 21.0, avg: 25.8, max: 53.0) [2023-10-11 20:25:46,035][70582] Avg episode reward: [(0, '194.120'), (1, '137.680')] [2023-10-11 20:25:46,237][71635] Updated weights for policy 1, policy_version 33842 (0.0008) [2023-10-11 20:25:46,609][71635] Updated weights for policy 1, policy_version 33852 (0.0007) [2023-10-11 20:25:47,296][71601] Updated weights for policy 0, policy_version 33860 (0.0008) [2023-10-11 20:25:47,666][71601] Updated weights for policy 0, policy_version 33870 (0.0009) [2023-10-11 20:25:48,045][71601] Updated weights for policy 0, policy_version 33880 (0.0008) [2023-10-11 20:25:50,430][71635] Updated weights for policy 1, policy_version 33862 (0.0008) [2023-10-11 20:25:50,793][71635] Updated weights for policy 1, policy_version 33872 (0.0007) [2023-10-11 20:25:51,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 69369856. Throughput: 0: 1851.2, 1: 1825.5. Samples: 17354272. Policy #0 lag: (min: 21.0, avg: 25.8, max: 53.0) [2023-10-11 20:25:51,035][70582] Avg episode reward: [(0, '194.360'), (1, '138.770')] [2023-10-11 20:25:51,163][71635] Updated weights for policy 1, policy_version 33882 (0.0007) [2023-10-11 20:25:51,676][71601] Updated weights for policy 0, policy_version 33890 (0.0008) [2023-10-11 20:25:52,071][71601] Updated weights for policy 0, policy_version 33900 (0.0009) [2023-10-11 20:25:52,434][71601] Updated weights for policy 0, policy_version 33910 (0.0008) [2023-10-11 20:25:52,807][71601] Updated weights for policy 0, policy_version 33920 (0.0007) [2023-10-11 20:25:54,856][71635] Updated weights for policy 1, policy_version 33892 (0.0009) [2023-10-11 20:25:55,223][71635] Updated weights for policy 1, policy_version 33902 (0.0010) [2023-10-11 20:25:55,588][71635] Updated weights for policy 1, policy_version 33912 (0.0010) [2023-10-11 20:25:56,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 69468160. Throughput: 0: 1853.3, 1: 1819.6. Samples: 17376508. Policy #0 lag: (min: 21.0, avg: 25.8, max: 53.0) [2023-10-11 20:25:56,034][70582] Avg episode reward: [(0, '197.850'), (1, '142.310')] [2023-10-11 20:25:56,042][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000033920_34734080.pth... [2023-10-11 20:25:56,074][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000032192_32964608.pth [2023-10-11 20:25:56,078][71431] Saving a milestone ./train_atari/atari_gopher_APPO/checkpoint_p1/milestones/checkpoint_000033920_34734080.pth [2023-10-11 20:25:56,283][71601] Updated weights for policy 0, policy_version 33930 (0.0007) [2023-10-11 20:25:56,653][71601] Updated weights for policy 0, policy_version 33940 (0.0008) [2023-10-11 20:25:57,031][71601] Updated weights for policy 0, policy_version 33950 (0.0010) [2023-10-11 20:25:57,096][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000033952_34766848.pth... [2023-10-11 20:25:57,134][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000032224_32997376.pth [2023-10-11 20:25:57,140][71353] Saving a milestone ./train_atari/atari_gopher_APPO/checkpoint_p0/milestones/checkpoint_000033952_34766848.pth [2023-10-11 20:25:59,181][71635] Updated weights for policy 1, policy_version 33922 (0.0010) [2023-10-11 20:25:59,554][71635] Updated weights for policy 1, policy_version 33932 (0.0008) [2023-10-11 20:25:59,920][71635] Updated weights for policy 1, policy_version 33942 (0.0008) [2023-10-11 20:26:00,278][71635] Updated weights for policy 1, policy_version 33952 (0.0008) [2023-10-11 20:26:00,806][71601] Updated weights for policy 0, policy_version 33960 (0.0007) [2023-10-11 20:26:01,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 69533696. Throughput: 0: 1853.6, 1: 1824.5. Samples: 17387466. Policy #0 lag: (min: 21.0, avg: 25.8, max: 53.0) [2023-10-11 20:26:01,035][70582] Avg episode reward: [(0, '201.730'), (1, '145.940')] [2023-10-11 20:26:01,174][71601] Updated weights for policy 0, policy_version 33970 (0.0009) [2023-10-11 20:26:01,545][71601] Updated weights for policy 0, policy_version 33980 (0.0010) [2023-10-11 20:26:04,148][71635] Updated weights for policy 1, policy_version 33962 (0.0009) [2023-10-11 20:26:04,513][71635] Updated weights for policy 1, policy_version 33972 (0.0008) [2023-10-11 20:26:04,881][71635] Updated weights for policy 1, policy_version 33982 (0.0007) [2023-10-11 20:26:05,236][71601] Updated weights for policy 0, policy_version 33990 (0.0008) [2023-10-11 20:26:05,609][71601] Updated weights for policy 0, policy_version 34000 (0.0009) [2023-10-11 20:26:05,991][71601] Updated weights for policy 0, policy_version 34010 (0.0008) [2023-10-11 20:26:06,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 69599232. Throughput: 0: 1846.0, 1: 1819.7. Samples: 17409180. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-11 20:26:06,034][70582] Avg episode reward: [(0, '201.730'), (1, '136.880')] [2023-10-11 20:26:08,691][71635] Updated weights for policy 1, policy_version 33992 (0.0008) [2023-10-11 20:26:09,066][71635] Updated weights for policy 1, policy_version 34002 (0.0008) [2023-10-11 20:26:09,433][71635] Updated weights for policy 1, policy_version 34012 (0.0011) [2023-10-11 20:26:09,657][71601] Updated weights for policy 0, policy_version 34020 (0.0008) [2023-10-11 20:26:10,022][71601] Updated weights for policy 0, policy_version 34030 (0.0008) [2023-10-11 20:26:10,387][71601] Updated weights for policy 0, policy_version 34040 (0.0008) [2023-10-11 20:26:11,034][70582] Fps is (10 sec: 16384.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 69697536. Throughput: 0: 1832.8, 1: 1819.3. Samples: 17430326. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-11 20:26:11,034][70582] Avg episode reward: [(0, '201.690'), (1, '132.520')] [2023-10-11 20:26:13,037][71635] Updated weights for policy 1, policy_version 34022 (0.0008) [2023-10-11 20:26:13,400][71635] Updated weights for policy 1, policy_version 34032 (0.0007) [2023-10-11 20:26:13,770][71635] Updated weights for policy 1, policy_version 34042 (0.0007) [2023-10-11 20:26:14,055][71601] Updated weights for policy 0, policy_version 34050 (0.0009) [2023-10-11 20:26:14,436][71601] Updated weights for policy 0, policy_version 34060 (0.0008) [2023-10-11 20:26:14,812][71601] Updated weights for policy 0, policy_version 34070 (0.0009) [2023-10-11 20:26:15,180][71601] Updated weights for policy 0, policy_version 34080 (0.0009) [2023-10-11 20:26:16,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 69763072. Throughput: 0: 1843.3, 1: 1819.7. Samples: 17442396. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-11 20:26:16,034][70582] Avg episode reward: [(0, '215.970'), (1, '119.820')] [2023-10-11 20:26:17,398][71635] Updated weights for policy 1, policy_version 34052 (0.0008) [2023-10-11 20:26:17,763][71635] Updated weights for policy 1, policy_version 34062 (0.0010) [2023-10-11 20:26:18,123][71635] Updated weights for policy 1, policy_version 34072 (0.0010) [2023-10-11 20:26:18,742][71601] Updated weights for policy 0, policy_version 34090 (0.0009) [2023-10-11 20:26:19,114][71601] Updated weights for policy 0, policy_version 34100 (0.0011) [2023-10-11 20:26:19,482][71601] Updated weights for policy 0, policy_version 34110 (0.0009) [2023-10-11 20:26:21,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 69828608. Throughput: 0: 1828.4, 1: 1830.8. Samples: 17463492. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-11 20:26:21,034][70582] Avg episode reward: [(0, '226.900'), (1, '126.750')] [2023-10-11 20:26:21,741][71635] Updated weights for policy 1, policy_version 34082 (0.0010) [2023-10-11 20:26:22,109][71635] Updated weights for policy 1, policy_version 34092 (0.0010) [2023-10-11 20:26:22,482][71635] Updated weights for policy 1, policy_version 34102 (0.0010) [2023-10-11 20:26:22,845][71635] Updated weights for policy 1, policy_version 34112 (0.0009) [2023-10-11 20:26:23,171][71601] Updated weights for policy 0, policy_version 34120 (0.0011) [2023-10-11 20:26:23,545][71601] Updated weights for policy 0, policy_version 34130 (0.0009) [2023-10-11 20:26:23,915][71601] Updated weights for policy 0, policy_version 34140 (0.0009) [2023-10-11 20:26:26,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 69894144. Throughput: 0: 1835.0, 1: 1825.7. Samples: 17485904. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-11 20:26:26,035][70582] Avg episode reward: [(0, '226.830'), (1, '135.010')] [2023-10-11 20:26:26,465][71635] Updated weights for policy 1, policy_version 34122 (0.0010) [2023-10-11 20:26:26,829][71635] Updated weights for policy 1, policy_version 34132 (0.0008) [2023-10-11 20:26:27,198][71635] Updated weights for policy 1, policy_version 34142 (0.0008) [2023-10-11 20:26:27,777][71601] Updated weights for policy 0, policy_version 34150 (0.0009) [2023-10-11 20:26:28,160][71601] Updated weights for policy 0, policy_version 34160 (0.0010) [2023-10-11 20:26:28,529][71601] Updated weights for policy 0, policy_version 34170 (0.0008) [2023-10-11 20:26:31,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 69959680. Throughput: 0: 1827.3, 1: 1819.5. Samples: 17496008. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-11 20:26:31,034][70582] Avg episode reward: [(0, '222.700'), (1, '120.450')] [2023-10-11 20:26:31,088][71635] Updated weights for policy 1, policy_version 34152 (0.0010) [2023-10-11 20:26:31,464][71635] Updated weights for policy 1, policy_version 34162 (0.0007) [2023-10-11 20:26:31,829][71635] Updated weights for policy 1, policy_version 34172 (0.0007) [2023-10-11 20:26:32,310][71601] Updated weights for policy 0, policy_version 34180 (0.0011) [2023-10-11 20:26:32,673][71601] Updated weights for policy 0, policy_version 34190 (0.0010) [2023-10-11 20:26:33,048][71601] Updated weights for policy 0, policy_version 34200 (0.0009) [2023-10-11 20:26:35,418][71635] Updated weights for policy 1, policy_version 34182 (0.0008) [2023-10-11 20:26:35,788][71635] Updated weights for policy 1, policy_version 34192 (0.0007) [2023-10-11 20:26:36,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 70025216. Throughput: 0: 1825.3, 1: 1818.0. Samples: 17518218. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-11 20:26:36,034][70582] Avg episode reward: [(0, '240.150'), (1, '117.170')] [2023-10-11 20:26:36,035][71353] Saving new best policy, reward=240.150! [2023-10-11 20:26:36,157][71635] Updated weights for policy 1, policy_version 34202 (0.0007) [2023-10-11 20:26:36,940][71601] Updated weights for policy 0, policy_version 34210 (0.0007) [2023-10-11 20:26:37,326][71601] Updated weights for policy 0, policy_version 34220 (0.0009) [2023-10-11 20:26:37,708][71601] Updated weights for policy 0, policy_version 34230 (0.0007) [2023-10-11 20:26:38,074][71601] Updated weights for policy 0, policy_version 34240 (0.0008) [2023-10-11 20:26:39,773][71635] Updated weights for policy 1, policy_version 34212 (0.0009) [2023-10-11 20:26:40,134][71635] Updated weights for policy 1, policy_version 34222 (0.0009) [2023-10-11 20:26:40,506][71635] Updated weights for policy 1, policy_version 34232 (0.0008) [2023-10-11 20:26:41,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 70123520. Throughput: 0: 1818.4, 1: 1819.1. Samples: 17540198. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-11 20:26:41,035][70582] Avg episode reward: [(0, '233.090'), (1, '116.930')] [2023-10-11 20:26:41,675][71601] Updated weights for policy 0, policy_version 34250 (0.0008) [2023-10-11 20:26:42,046][71601] Updated weights for policy 0, policy_version 34260 (0.0007) [2023-10-11 20:26:42,412][71601] Updated weights for policy 0, policy_version 34270 (0.0009) [2023-10-11 20:26:44,177][71635] Updated weights for policy 1, policy_version 34242 (0.0008) [2023-10-11 20:26:44,546][71635] Updated weights for policy 1, policy_version 34252 (0.0008) [2023-10-11 20:26:44,903][71635] Updated weights for policy 1, policy_version 34262 (0.0009) [2023-10-11 20:26:45,280][71635] Updated weights for policy 1, policy_version 34272 (0.0009) [2023-10-11 20:26:46,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 70189056. Throughput: 0: 1814.4, 1: 1823.1. Samples: 17551154. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-11 20:26:46,034][70582] Avg episode reward: [(0, '222.530'), (1, '120.000')] [2023-10-11 20:26:46,092][71601] Updated weights for policy 0, policy_version 34280 (0.0007) [2023-10-11 20:26:46,469][71601] Updated weights for policy 0, policy_version 34290 (0.0007) [2023-10-11 20:26:46,831][71601] Updated weights for policy 0, policy_version 34300 (0.0009) [2023-10-11 20:26:49,074][71635] Updated weights for policy 1, policy_version 34282 (0.0007) [2023-10-11 20:26:49,455][71635] Updated weights for policy 1, policy_version 34292 (0.0009) [2023-10-11 20:26:49,818][71635] Updated weights for policy 1, policy_version 34302 (0.0008) [2023-10-11 20:26:50,550][71601] Updated weights for policy 0, policy_version 34310 (0.0008) [2023-10-11 20:26:50,915][71601] Updated weights for policy 0, policy_version 34320 (0.0007) [2023-10-11 20:26:51,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 70254592. Throughput: 0: 1816.3, 1: 1824.2. Samples: 17573002. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-11 20:26:51,035][70582] Avg episode reward: [(0, '222.380'), (1, '96.650')] [2023-10-11 20:26:51,288][71601] Updated weights for policy 0, policy_version 34330 (0.0007) [2023-10-11 20:26:53,358][71635] Updated weights for policy 1, policy_version 34312 (0.0011) [2023-10-11 20:26:53,725][71635] Updated weights for policy 1, policy_version 34322 (0.0011) [2023-10-11 20:26:54,086][71635] Updated weights for policy 1, policy_version 34332 (0.0010) [2023-10-11 20:26:54,971][71601] Updated weights for policy 0, policy_version 34340 (0.0007) [2023-10-11 20:26:55,353][71601] Updated weights for policy 0, policy_version 34350 (0.0009) [2023-10-11 20:26:55,728][71601] Updated weights for policy 0, policy_version 34360 (0.0009) [2023-10-11 20:26:56,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 70352896. Throughput: 0: 1820.6, 1: 1824.8. Samples: 17594368. Policy #0 lag: (min: 31.0, avg: 32.0, max: 53.0) [2023-10-11 20:26:56,034][70582] Avg episode reward: [(0, '222.520'), (1, '87.940')] [2023-10-11 20:26:57,873][71635] Updated weights for policy 1, policy_version 34342 (0.0009) [2023-10-11 20:26:58,242][71635] Updated weights for policy 1, policy_version 34352 (0.0008) [2023-10-11 20:26:58,598][71635] Updated weights for policy 1, policy_version 34362 (0.0010) [2023-10-11 20:26:59,314][71601] Updated weights for policy 0, policy_version 34370 (0.0009) [2023-10-11 20:26:59,688][71601] Updated weights for policy 0, policy_version 34380 (0.0008) [2023-10-11 20:27:00,055][71601] Updated weights for policy 0, policy_version 34390 (0.0007) [2023-10-11 20:27:00,420][71601] Updated weights for policy 0, policy_version 34400 (0.0009) [2023-10-11 20:27:01,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 70418432. Throughput: 0: 1810.8, 1: 1816.7. Samples: 17605632. Policy #0 lag: (min: 31.0, avg: 32.0, max: 53.0) [2023-10-11 20:27:01,034][70582] Avg episode reward: [(0, '237.260'), (1, '88.060')] [2023-10-11 20:27:02,314][71635] Updated weights for policy 1, policy_version 34372 (0.0009) [2023-10-11 20:27:02,685][71635] Updated weights for policy 1, policy_version 34382 (0.0010) [2023-10-11 20:27:03,043][71635] Updated weights for policy 1, policy_version 34392 (0.0010) [2023-10-11 20:27:03,976][71601] Updated weights for policy 0, policy_version 34410 (0.0009) [2023-10-11 20:27:04,346][71601] Updated weights for policy 0, policy_version 34420 (0.0008) [2023-10-11 20:27:04,718][71601] Updated weights for policy 0, policy_version 34430 (0.0008) [2023-10-11 20:27:06,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 70483968. Throughput: 0: 1820.3, 1: 1816.7. Samples: 17627156. Policy #0 lag: (min: 31.0, avg: 32.0, max: 53.0) [2023-10-11 20:27:06,034][70582] Avg episode reward: [(0, '233.280'), (1, '76.870')] [2023-10-11 20:27:06,751][71635] Updated weights for policy 1, policy_version 34402 (0.0011) [2023-10-11 20:27:07,119][71635] Updated weights for policy 1, policy_version 34412 (0.0007) [2023-10-11 20:27:07,488][71635] Updated weights for policy 1, policy_version 34422 (0.0009) [2023-10-11 20:27:07,851][71635] Updated weights for policy 1, policy_version 34432 (0.0009) [2023-10-11 20:27:08,216][71601] Updated weights for policy 0, policy_version 34440 (0.0010) [2023-10-11 20:27:08,595][71601] Updated weights for policy 0, policy_version 34450 (0.0007) [2023-10-11 20:27:08,967][71601] Updated weights for policy 0, policy_version 34460 (0.0007) [2023-10-11 20:27:11,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 70549504. Throughput: 0: 1823.0, 1: 1818.7. Samples: 17649780. Policy #0 lag: (min: 31.0, avg: 32.0, max: 53.0) [2023-10-11 20:27:11,035][70582] Avg episode reward: [(0, '233.280'), (1, '72.900')] [2023-10-11 20:27:11,467][71635] Updated weights for policy 1, policy_version 34442 (0.0007) [2023-10-11 20:27:11,839][71635] Updated weights for policy 1, policy_version 34452 (0.0007) [2023-10-11 20:27:12,205][71635] Updated weights for policy 1, policy_version 34462 (0.0007) [2023-10-11 20:27:12,687][71601] Updated weights for policy 0, policy_version 34470 (0.0008) [2023-10-11 20:27:13,057][71601] Updated weights for policy 0, policy_version 34480 (0.0008) [2023-10-11 20:27:13,431][71601] Updated weights for policy 0, policy_version 34490 (0.0008) [2023-10-11 20:27:15,843][71635] Updated weights for policy 1, policy_version 34472 (0.0012) [2023-10-11 20:27:16,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 70615040. Throughput: 0: 1821.3, 1: 1827.6. Samples: 17660212. Policy #0 lag: (min: 31.0, avg: 32.0, max: 53.0) [2023-10-11 20:27:16,035][70582] Avg episode reward: [(0, '233.280'), (1, '73.950')] [2023-10-11 20:27:16,214][71635] Updated weights for policy 1, policy_version 34482 (0.0009) [2023-10-11 20:27:16,585][71635] Updated weights for policy 1, policy_version 34492 (0.0007) [2023-10-11 20:27:16,926][71601] Updated weights for policy 0, policy_version 34500 (0.0010) [2023-10-11 20:27:17,307][71601] Updated weights for policy 0, policy_version 34510 (0.0008) [2023-10-11 20:27:17,672][71601] Updated weights for policy 0, policy_version 34520 (0.0007) [2023-10-11 20:27:20,433][71635] Updated weights for policy 1, policy_version 34502 (0.0008) [2023-10-11 20:27:20,795][71635] Updated weights for policy 1, policy_version 34512 (0.0007) [2023-10-11 20:27:21,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 70680576. Throughput: 0: 1832.0, 1: 1828.7. Samples: 17682950. Policy #0 lag: (min: 31.0, avg: 32.0, max: 53.0) [2023-10-11 20:27:21,034][70582] Avg episode reward: [(0, '233.280'), (1, '71.380')] [2023-10-11 20:27:21,169][71635] Updated weights for policy 1, policy_version 34522 (0.0008) [2023-10-11 20:27:21,292][71601] Updated weights for policy 0, policy_version 34530 (0.0009) [2023-10-11 20:27:21,660][71601] Updated weights for policy 0, policy_version 34540 (0.0007) [2023-10-11 20:27:22,041][71601] Updated weights for policy 0, policy_version 34550 (0.0007) [2023-10-11 20:27:22,403][71601] Updated weights for policy 0, policy_version 34560 (0.0007) [2023-10-11 20:27:24,986][71635] Updated weights for policy 1, policy_version 34532 (0.0009) [2023-10-11 20:27:25,351][71635] Updated weights for policy 1, policy_version 34542 (0.0009) [2023-10-11 20:27:25,715][71635] Updated weights for policy 1, policy_version 34552 (0.0007) [2023-10-11 20:27:26,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 70778880. Throughput: 0: 1834.8, 1: 1825.5. Samples: 17704908. Policy #0 lag: (min: 17.0, avg: 22.5, max: 49.0) [2023-10-11 20:27:26,034][70582] Avg episode reward: [(0, '246.410'), (1, '71.530')] [2023-10-11 20:27:26,226][71601] Updated weights for policy 0, policy_version 34570 (0.0008) [2023-10-11 20:27:26,602][71601] Updated weights for policy 0, policy_version 34580 (0.0009) [2023-10-11 20:27:26,967][71601] Updated weights for policy 0, policy_version 34590 (0.0009) [2023-10-11 20:27:27,038][71353] Saving new best policy, reward=246.410! [2023-10-11 20:27:29,384][71635] Updated weights for policy 1, policy_version 34562 (0.0009) [2023-10-11 20:27:29,758][71635] Updated weights for policy 1, policy_version 34572 (0.0010) [2023-10-11 20:27:30,118][71635] Updated weights for policy 1, policy_version 34582 (0.0007) [2023-10-11 20:27:30,487][71635] Updated weights for policy 1, policy_version 34592 (0.0009) [2023-10-11 20:27:30,700][71601] Updated weights for policy 0, policy_version 34600 (0.0007) [2023-10-11 20:27:31,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 70844416. Throughput: 0: 1832.0, 1: 1817.5. Samples: 17715384. Policy #0 lag: (min: 17.0, avg: 22.5, max: 49.0) [2023-10-11 20:27:31,035][70582] Avg episode reward: [(0, '246.570'), (1, '74.690')] [2023-10-11 20:27:31,068][71601] Updated weights for policy 0, policy_version 34610 (0.0009) [2023-10-11 20:27:31,445][71601] Updated weights for policy 0, policy_version 34620 (0.0010) [2023-10-11 20:27:31,592][71353] Saving new best policy, reward=246.570! [2023-10-11 20:27:34,172][71635] Updated weights for policy 1, policy_version 34602 (0.0010) [2023-10-11 20:27:34,536][71635] Updated weights for policy 1, policy_version 34612 (0.0008) [2023-10-11 20:27:34,910][71635] Updated weights for policy 1, policy_version 34622 (0.0007) [2023-10-11 20:27:35,279][71601] Updated weights for policy 0, policy_version 34630 (0.0008) [2023-10-11 20:27:35,650][71601] Updated weights for policy 0, policy_version 34640 (0.0007) [2023-10-11 20:27:36,022][71601] Updated weights for policy 0, policy_version 34650 (0.0007) [2023-10-11 20:27:36,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 70909952. Throughput: 0: 1826.4, 1: 1821.6. Samples: 17737160. Policy #0 lag: (min: 17.0, avg: 22.5, max: 49.0) [2023-10-11 20:27:36,034][70582] Avg episode reward: [(0, '246.640'), (1, '69.710')] [2023-10-11 20:27:36,239][71353] Saving new best policy, reward=246.640! [2023-10-11 20:27:38,448][71635] Updated weights for policy 1, policy_version 34632 (0.0009) [2023-10-11 20:27:38,818][71635] Updated weights for policy 1, policy_version 34642 (0.0008) [2023-10-11 20:27:39,177][71635] Updated weights for policy 1, policy_version 34652 (0.0008) [2023-10-11 20:27:39,737][71601] Updated weights for policy 0, policy_version 34660 (0.0008) [2023-10-11 20:27:40,112][71601] Updated weights for policy 0, policy_version 34670 (0.0011) [2023-10-11 20:27:40,488][71601] Updated weights for policy 0, policy_version 34680 (0.0010) [2023-10-11 20:27:41,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 71008256. Throughput: 0: 1819.8, 1: 1821.2. Samples: 17758212. Policy #0 lag: (min: 17.0, avg: 22.5, max: 49.0) [2023-10-11 20:27:41,034][70582] Avg episode reward: [(0, '255.800'), (1, '70.500')] [2023-10-11 20:27:41,044][71353] Saving new best policy, reward=255.800! [2023-10-11 20:27:42,909][71635] Updated weights for policy 1, policy_version 34662 (0.0008) [2023-10-11 20:27:43,279][71635] Updated weights for policy 1, policy_version 34672 (0.0007) [2023-10-11 20:27:43,652][71635] Updated weights for policy 1, policy_version 34682 (0.0010) [2023-10-11 20:27:43,996][71601] Updated weights for policy 0, policy_version 34690 (0.0010) [2023-10-11 20:27:44,355][71601] Updated weights for policy 0, policy_version 34700 (0.0010) [2023-10-11 20:27:44,722][71601] Updated weights for policy 0, policy_version 34710 (0.0009) [2023-10-11 20:27:45,096][71601] Updated weights for policy 0, policy_version 34720 (0.0009) [2023-10-11 20:27:46,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 71073792. Throughput: 0: 1823.6, 1: 1823.2. Samples: 17769738. Policy #0 lag: (min: 17.0, avg: 22.5, max: 49.0) [2023-10-11 20:27:46,035][70582] Avg episode reward: [(0, '255.560'), (1, '63.190')] [2023-10-11 20:27:47,408][71635] Updated weights for policy 1, policy_version 34692 (0.0009) [2023-10-11 20:27:47,772][71635] Updated weights for policy 1, policy_version 34702 (0.0009) [2023-10-11 20:27:48,145][71635] Updated weights for policy 1, policy_version 34712 (0.0008) [2023-10-11 20:27:48,841][71601] Updated weights for policy 0, policy_version 34730 (0.0009) [2023-10-11 20:27:49,207][71601] Updated weights for policy 0, policy_version 34740 (0.0008) [2023-10-11 20:27:49,574][71601] Updated weights for policy 0, policy_version 34750 (0.0007) [2023-10-11 20:27:51,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 71139328. Throughput: 0: 1815.9, 1: 1818.9. Samples: 17790724. Policy #0 lag: (min: 31.0, avg: 31.5, max: 47.0) [2023-10-11 20:27:51,035][70582] Avg episode reward: [(0, '255.560'), (1, '64.120')] [2023-10-11 20:27:51,893][71635] Updated weights for policy 1, policy_version 34722 (0.0010) [2023-10-11 20:27:52,257][71635] Updated weights for policy 1, policy_version 34732 (0.0007) [2023-10-11 20:27:52,629][71635] Updated weights for policy 1, policy_version 34742 (0.0009) [2023-10-11 20:27:52,982][71635] Updated weights for policy 1, policy_version 34752 (0.0009) [2023-10-11 20:27:53,217][71601] Updated weights for policy 0, policy_version 34760 (0.0009) [2023-10-11 20:27:53,604][71601] Updated weights for policy 0, policy_version 34770 (0.0007) [2023-10-11 20:27:53,970][71601] Updated weights for policy 0, policy_version 34780 (0.0008) [2023-10-11 20:27:56,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 71204864. Throughput: 0: 1815.1, 1: 1820.2. Samples: 17813368. Policy #0 lag: (min: 31.0, avg: 31.5, max: 47.0) [2023-10-11 20:27:56,034][70582] Avg episode reward: [(0, '255.560'), (1, '64.120')] [2023-10-11 20:27:56,046][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000034752_35586048.pth... [2023-10-11 20:27:56,046][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000034784_35618816.pth... [2023-10-11 20:27:56,077][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000033088_33882112.pth [2023-10-11 20:27:56,084][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000033056_33849344.pth [2023-10-11 20:27:56,593][71635] Updated weights for policy 1, policy_version 34762 (0.0008) [2023-10-11 20:27:56,956][71635] Updated weights for policy 1, policy_version 34772 (0.0007) [2023-10-11 20:27:57,332][71635] Updated weights for policy 1, policy_version 34782 (0.0007) [2023-10-11 20:27:57,648][71601] Updated weights for policy 0, policy_version 34790 (0.0009) [2023-10-11 20:27:58,017][71601] Updated weights for policy 0, policy_version 34800 (0.0009) [2023-10-11 20:27:58,396][71601] Updated weights for policy 0, policy_version 34810 (0.0007) [2023-10-11 20:28:01,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 71270400. Throughput: 0: 1814.6, 1: 1815.1. Samples: 17823548. Policy #0 lag: (min: 31.0, avg: 31.5, max: 47.0) [2023-10-11 20:28:01,034][70582] Avg episode reward: [(0, '255.560'), (1, '75.890')] [2023-10-11 20:28:01,067][71635] Updated weights for policy 1, policy_version 34792 (0.0008) [2023-10-11 20:28:01,429][71635] Updated weights for policy 1, policy_version 34802 (0.0007) [2023-10-11 20:28:01,795][71635] Updated weights for policy 1, policy_version 34812 (0.0008) [2023-10-11 20:28:02,204][71601] Updated weights for policy 0, policy_version 34820 (0.0007) [2023-10-11 20:28:02,564][71601] Updated weights for policy 0, policy_version 34830 (0.0009) [2023-10-11 20:28:02,934][71601] Updated weights for policy 0, policy_version 34840 (0.0008) [2023-10-11 20:28:05,584][71635] Updated weights for policy 1, policy_version 34822 (0.0009) [2023-10-11 20:28:05,957][71635] Updated weights for policy 1, policy_version 34832 (0.0009) [2023-10-11 20:28:06,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 71335936. Throughput: 0: 1811.1, 1: 1812.4. Samples: 17846012. Policy #0 lag: (min: 31.0, avg: 31.5, max: 47.0) [2023-10-11 20:28:06,035][70582] Avg episode reward: [(0, '255.560'), (1, '73.610')] [2023-10-11 20:28:06,320][71635] Updated weights for policy 1, policy_version 34842 (0.0007) [2023-10-11 20:28:06,716][71601] Updated weights for policy 0, policy_version 34850 (0.0008) [2023-10-11 20:28:07,084][71601] Updated weights for policy 0, policy_version 34860 (0.0007) [2023-10-11 20:28:07,454][71601] Updated weights for policy 0, policy_version 34870 (0.0009) [2023-10-11 20:28:07,820][71601] Updated weights for policy 0, policy_version 34880 (0.0008) [2023-10-11 20:28:09,953][71635] Updated weights for policy 1, policy_version 34852 (0.0008) [2023-10-11 20:28:10,324][71635] Updated weights for policy 1, policy_version 34862 (0.0009) [2023-10-11 20:28:10,686][71635] Updated weights for policy 1, policy_version 34872 (0.0011) [2023-10-11 20:28:11,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 71434240. Throughput: 0: 1806.9, 1: 1814.4. Samples: 17867868. Policy #0 lag: (min: 31.0, avg: 31.5, max: 47.0) [2023-10-11 20:28:11,034][70582] Avg episode reward: [(0, '270.570'), (1, '90.950')] [2023-10-11 20:28:11,043][71353] Saving new best policy, reward=270.570! [2023-10-11 20:28:11,647][71601] Updated weights for policy 0, policy_version 34890 (0.0009) [2023-10-11 20:28:12,016][71601] Updated weights for policy 0, policy_version 34900 (0.0011) [2023-10-11 20:28:12,387][71601] Updated weights for policy 0, policy_version 34910 (0.0009) [2023-10-11 20:28:14,444][71635] Updated weights for policy 1, policy_version 34882 (0.0009) [2023-10-11 20:28:14,809][71635] Updated weights for policy 1, policy_version 34892 (0.0009) [2023-10-11 20:28:15,170][71635] Updated weights for policy 1, policy_version 34902 (0.0007) [2023-10-11 20:28:15,536][71635] Updated weights for policy 1, policy_version 34912 (0.0007) [2023-10-11 20:28:16,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 71499776. Throughput: 0: 1808.8, 1: 1817.2. Samples: 17878552. Policy #0 lag: (min: 31.0, avg: 31.5, max: 47.0) [2023-10-11 20:28:16,034][70582] Avg episode reward: [(0, '270.230'), (1, '89.020')] [2023-10-11 20:28:16,100][71601] Updated weights for policy 0, policy_version 34920 (0.0008) [2023-10-11 20:28:16,467][71601] Updated weights for policy 0, policy_version 34930 (0.0007) [2023-10-11 20:28:16,841][71601] Updated weights for policy 0, policy_version 34940 (0.0007) [2023-10-11 20:28:19,316][71635] Updated weights for policy 1, policy_version 34922 (0.0009) [2023-10-11 20:28:19,687][71635] Updated weights for policy 1, policy_version 34932 (0.0008) [2023-10-11 20:28:20,050][71635] Updated weights for policy 1, policy_version 34942 (0.0008) [2023-10-11 20:28:20,507][71601] Updated weights for policy 0, policy_version 34950 (0.0008) [2023-10-11 20:28:20,883][71601] Updated weights for policy 0, policy_version 34960 (0.0008) [2023-10-11 20:28:21,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 71565312. Throughput: 0: 1812.1, 1: 1819.5. Samples: 17900580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:28:21,034][70582] Avg episode reward: [(0, '279.560'), (1, '89.410')] [2023-10-11 20:28:21,249][71601] Updated weights for policy 0, policy_version 34970 (0.0008) [2023-10-11 20:28:21,469][71353] Saving new best policy, reward=279.560! [2023-10-11 20:28:23,617][71635] Updated weights for policy 1, policy_version 34952 (0.0008) [2023-10-11 20:28:23,987][71635] Updated weights for policy 1, policy_version 34962 (0.0007) [2023-10-11 20:28:24,357][71635] Updated weights for policy 1, policy_version 34972 (0.0009) [2023-10-11 20:28:24,924][71601] Updated weights for policy 0, policy_version 34980 (0.0009) [2023-10-11 20:28:25,289][71601] Updated weights for policy 0, policy_version 34990 (0.0007) [2023-10-11 20:28:25,659][71601] Updated weights for policy 0, policy_version 35000 (0.0008) [2023-10-11 20:28:26,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 71663616. Throughput: 0: 1822.3, 1: 1816.5. Samples: 17921958. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:28:26,034][70582] Avg episode reward: [(0, '269.090'), (1, '88.740')] [2023-10-11 20:28:27,999][71635] Updated weights for policy 1, policy_version 34982 (0.0008) [2023-10-11 20:28:28,374][71635] Updated weights for policy 1, policy_version 34992 (0.0007) [2023-10-11 20:28:28,739][71635] Updated weights for policy 1, policy_version 35002 (0.0008) [2023-10-11 20:28:29,280][71601] Updated weights for policy 0, policy_version 35010 (0.0007) [2023-10-11 20:28:29,656][71601] Updated weights for policy 0, policy_version 35020 (0.0008) [2023-10-11 20:28:30,036][71601] Updated weights for policy 0, policy_version 35030 (0.0008) [2023-10-11 20:28:30,407][71601] Updated weights for policy 0, policy_version 35040 (0.0008) [2023-10-11 20:28:31,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 71729152. Throughput: 0: 1816.3, 1: 1824.4. Samples: 17933568. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:28:31,035][70582] Avg episode reward: [(0, '269.090'), (1, '88.830')] [2023-10-11 20:28:32,286][71635] Updated weights for policy 1, policy_version 35012 (0.0007) [2023-10-11 20:28:32,647][71635] Updated weights for policy 1, policy_version 35022 (0.0007) [2023-10-11 20:28:33,013][71635] Updated weights for policy 1, policy_version 35032 (0.0007) [2023-10-11 20:28:34,020][71601] Updated weights for policy 0, policy_version 35050 (0.0009) [2023-10-11 20:28:34,387][71601] Updated weights for policy 0, policy_version 35060 (0.0011) [2023-10-11 20:28:34,752][71601] Updated weights for policy 0, policy_version 35070 (0.0007) [2023-10-11 20:28:36,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 71794688. Throughput: 0: 1822.1, 1: 1823.4. Samples: 17954770. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:28:36,035][70582] Avg episode reward: [(0, '269.090'), (1, '84.260')] [2023-10-11 20:28:36,754][71635] Updated weights for policy 1, policy_version 35042 (0.0008) [2023-10-11 20:28:37,116][71635] Updated weights for policy 1, policy_version 35052 (0.0009) [2023-10-11 20:28:37,483][71635] Updated weights for policy 1, policy_version 35062 (0.0008) [2023-10-11 20:28:37,851][71635] Updated weights for policy 1, policy_version 35072 (0.0011) [2023-10-11 20:28:38,344][71601] Updated weights for policy 0, policy_version 35080 (0.0007) [2023-10-11 20:28:38,706][71601] Updated weights for policy 0, policy_version 35090 (0.0007) [2023-10-11 20:28:39,077][71601] Updated weights for policy 0, policy_version 35100 (0.0010) [2023-10-11 20:28:41,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 71860224. Throughput: 0: 1817.8, 1: 1823.1. Samples: 17977210. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:28:41,034][70582] Avg episode reward: [(0, '285.140'), (1, '84.260')] [2023-10-11 20:28:41,042][71353] Saving new best policy, reward=285.140! [2023-10-11 20:28:41,537][71635] Updated weights for policy 1, policy_version 35082 (0.0007) [2023-10-11 20:28:41,904][71635] Updated weights for policy 1, policy_version 35092 (0.0007) [2023-10-11 20:28:42,267][71635] Updated weights for policy 1, policy_version 35102 (0.0008) [2023-10-11 20:28:42,768][71601] Updated weights for policy 0, policy_version 35110 (0.0010) [2023-10-11 20:28:43,132][71601] Updated weights for policy 0, policy_version 35120 (0.0007) [2023-10-11 20:28:43,500][71601] Updated weights for policy 0, policy_version 35130 (0.0009) [2023-10-11 20:28:45,935][71635] Updated weights for policy 1, policy_version 35112 (0.0009) [2023-10-11 20:28:46,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 71925760. Throughput: 0: 1819.6, 1: 1823.8. Samples: 17987502. Policy #0 lag: (min: 13.0, avg: 14.3, max: 36.0) [2023-10-11 20:28:46,034][70582] Avg episode reward: [(0, '282.300'), (1, '89.270')] [2023-10-11 20:28:46,308][71635] Updated weights for policy 1, policy_version 35122 (0.0009) [2023-10-11 20:28:46,683][71635] Updated weights for policy 1, policy_version 35132 (0.0010) [2023-10-11 20:28:47,053][71601] Updated weights for policy 0, policy_version 35140 (0.0007) [2023-10-11 20:28:47,423][71601] Updated weights for policy 0, policy_version 35150 (0.0007) [2023-10-11 20:28:47,796][71601] Updated weights for policy 0, policy_version 35160 (0.0009) [2023-10-11 20:28:50,360][71635] Updated weights for policy 1, policy_version 35142 (0.0008) [2023-10-11 20:28:50,720][71635] Updated weights for policy 1, policy_version 35152 (0.0009) [2023-10-11 20:28:51,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 71991296. Throughput: 0: 1823.2, 1: 1826.9. Samples: 18010262. Policy #0 lag: (min: 13.0, avg: 14.3, max: 36.0) [2023-10-11 20:28:51,034][70582] Avg episode reward: [(0, '279.950'), (1, '89.320')] [2023-10-11 20:28:51,092][71635] Updated weights for policy 1, policy_version 35162 (0.0011) [2023-10-11 20:28:51,391][71601] Updated weights for policy 0, policy_version 35170 (0.0008) [2023-10-11 20:28:51,760][71601] Updated weights for policy 0, policy_version 35180 (0.0007) [2023-10-11 20:28:52,129][71601] Updated weights for policy 0, policy_version 35190 (0.0007) [2023-10-11 20:28:52,505][71601] Updated weights for policy 0, policy_version 35200 (0.0007) [2023-10-11 20:28:54,847][71635] Updated weights for policy 1, policy_version 35172 (0.0007) [2023-10-11 20:28:55,212][71635] Updated weights for policy 1, policy_version 35182 (0.0008) [2023-10-11 20:28:55,571][71635] Updated weights for policy 1, policy_version 35192 (0.0008) [2023-10-11 20:28:56,035][70582] Fps is (10 sec: 16381.9, 60 sec: 14745.3, 300 sec: 14662.2). Total num frames: 72089600. Throughput: 0: 1831.7, 1: 1823.9. Samples: 18032372. Policy #0 lag: (min: 13.0, avg: 14.3, max: 36.0) [2023-10-11 20:28:56,036][70582] Avg episode reward: [(0, '294.230'), (1, '87.340')] [2023-10-11 20:28:56,325][71601] Updated weights for policy 0, policy_version 35210 (0.0008) [2023-10-11 20:28:56,708][71601] Updated weights for policy 0, policy_version 35220 (0.0007) [2023-10-11 20:28:57,078][71601] Updated weights for policy 0, policy_version 35230 (0.0008) [2023-10-11 20:28:57,144][71353] Saving new best policy, reward=294.230! [2023-10-11 20:28:59,033][71635] Updated weights for policy 1, policy_version 35202 (0.0008) [2023-10-11 20:28:59,396][71635] Updated weights for policy 1, policy_version 35212 (0.0007) [2023-10-11 20:28:59,759][71635] Updated weights for policy 1, policy_version 35222 (0.0009) [2023-10-11 20:29:00,130][71635] Updated weights for policy 1, policy_version 35232 (0.0009) [2023-10-11 20:29:00,870][71601] Updated weights for policy 0, policy_version 35240 (0.0008) [2023-10-11 20:29:01,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 72155136. Throughput: 0: 1826.9, 1: 1826.4. Samples: 18042952. Policy #0 lag: (min: 13.0, avg: 14.3, max: 36.0) [2023-10-11 20:29:01,035][70582] Avg episode reward: [(0, '288.890'), (1, '90.490')] [2023-10-11 20:29:01,235][71601] Updated weights for policy 0, policy_version 35250 (0.0009) [2023-10-11 20:29:01,612][71601] Updated weights for policy 0, policy_version 35260 (0.0007) [2023-10-11 20:29:03,803][71635] Updated weights for policy 1, policy_version 35242 (0.0009) [2023-10-11 20:29:04,176][71635] Updated weights for policy 1, policy_version 35252 (0.0007) [2023-10-11 20:29:04,548][71635] Updated weights for policy 1, policy_version 35262 (0.0008) [2023-10-11 20:29:05,284][71601] Updated weights for policy 0, policy_version 35270 (0.0007) [2023-10-11 20:29:05,650][71601] Updated weights for policy 0, policy_version 35280 (0.0008) [2023-10-11 20:29:06,034][70582] Fps is (10 sec: 13108.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 72220672. Throughput: 0: 1829.2, 1: 1821.1. Samples: 18064846. Policy #0 lag: (min: 13.0, avg: 14.3, max: 36.0) [2023-10-11 20:29:06,035][70582] Avg episode reward: [(0, '290.590'), (1, '90.620')] [2023-10-11 20:29:06,036][71601] Updated weights for policy 0, policy_version 35290 (0.0010) [2023-10-11 20:29:08,301][71635] Updated weights for policy 1, policy_version 35272 (0.0008) [2023-10-11 20:29:08,672][71635] Updated weights for policy 1, policy_version 35282 (0.0009) [2023-10-11 20:29:09,039][71635] Updated weights for policy 1, policy_version 35292 (0.0010) [2023-10-11 20:29:09,736][71601] Updated weights for policy 0, policy_version 35300 (0.0011) [2023-10-11 20:29:10,104][71601] Updated weights for policy 0, policy_version 35310 (0.0010) [2023-10-11 20:29:10,477][71601] Updated weights for policy 0, policy_version 35320 (0.0009) [2023-10-11 20:29:11,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 72318976. Throughput: 0: 1813.6, 1: 1833.3. Samples: 18086072. Policy #0 lag: (min: 6.0, avg: 7.6, max: 32.0) [2023-10-11 20:29:11,034][70582] Avg episode reward: [(0, '316.120'), (1, '97.050')] [2023-10-11 20:29:11,044][71353] Saving new best policy, reward=316.120! [2023-10-11 20:29:12,666][71635] Updated weights for policy 1, policy_version 35302 (0.0011) [2023-10-11 20:29:13,029][71635] Updated weights for policy 1, policy_version 35312 (0.0008) [2023-10-11 20:29:13,396][71635] Updated weights for policy 1, policy_version 35322 (0.0007) [2023-10-11 20:29:14,100][71601] Updated weights for policy 0, policy_version 35330 (0.0008) [2023-10-11 20:29:14,466][71601] Updated weights for policy 0, policy_version 35340 (0.0010) [2023-10-11 20:29:14,832][71601] Updated weights for policy 0, policy_version 35350 (0.0010) [2023-10-11 20:29:15,204][71601] Updated weights for policy 0, policy_version 35360 (0.0009) [2023-10-11 20:29:16,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 72384512. Throughput: 0: 1822.5, 1: 1821.6. Samples: 18097554. Policy #0 lag: (min: 6.0, avg: 7.6, max: 32.0) [2023-10-11 20:29:16,035][70582] Avg episode reward: [(0, '325.530'), (1, '96.900')] [2023-10-11 20:29:16,035][71353] Saving new best policy, reward=325.530! [2023-10-11 20:29:17,026][71635] Updated weights for policy 1, policy_version 35332 (0.0007) [2023-10-11 20:29:17,391][71635] Updated weights for policy 1, policy_version 35342 (0.0008) [2023-10-11 20:29:17,768][71635] Updated weights for policy 1, policy_version 35352 (0.0010) [2023-10-11 20:29:18,743][71601] Updated weights for policy 0, policy_version 35370 (0.0010) [2023-10-11 20:29:19,114][71601] Updated weights for policy 0, policy_version 35380 (0.0010) [2023-10-11 20:29:19,486][71601] Updated weights for policy 0, policy_version 35390 (0.0009) [2023-10-11 20:29:21,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 72450048. Throughput: 0: 1816.0, 1: 1827.6. Samples: 18118732. Policy #0 lag: (min: 6.0, avg: 7.6, max: 32.0) [2023-10-11 20:29:21,034][70582] Avg episode reward: [(0, '325.260'), (1, '109.700')] [2023-10-11 20:29:21,496][71635] Updated weights for policy 1, policy_version 35362 (0.0011) [2023-10-11 20:29:21,872][71635] Updated weights for policy 1, policy_version 35372 (0.0007) [2023-10-11 20:29:22,230][71635] Updated weights for policy 1, policy_version 35382 (0.0008) [2023-10-11 20:29:22,597][71635] Updated weights for policy 1, policy_version 35392 (0.0009) [2023-10-11 20:29:23,377][71601] Updated weights for policy 0, policy_version 35400 (0.0010) [2023-10-11 20:29:23,749][71601] Updated weights for policy 0, policy_version 35410 (0.0011) [2023-10-11 20:29:24,127][71601] Updated weights for policy 0, policy_version 35420 (0.0008) [2023-10-11 20:29:26,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 72515584. Throughput: 0: 1818.4, 1: 1822.6. Samples: 18141054. Policy #0 lag: (min: 6.0, avg: 7.6, max: 32.0) [2023-10-11 20:29:26,035][70582] Avg episode reward: [(0, '301.020'), (1, '127.450')] [2023-10-11 20:29:26,348][71635] Updated weights for policy 1, policy_version 35402 (0.0010) [2023-10-11 20:29:26,706][71635] Updated weights for policy 1, policy_version 35412 (0.0010) [2023-10-11 20:29:27,079][71635] Updated weights for policy 1, policy_version 35422 (0.0009) [2023-10-11 20:29:27,766][71601] Updated weights for policy 0, policy_version 35430 (0.0009) [2023-10-11 20:29:28,130][71601] Updated weights for policy 0, policy_version 35440 (0.0009) [2023-10-11 20:29:28,512][71601] Updated weights for policy 0, policy_version 35450 (0.0008) [2023-10-11 20:29:30,865][71635] Updated weights for policy 1, policy_version 35432 (0.0008) [2023-10-11 20:29:31,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 72581120. Throughput: 0: 1820.6, 1: 1823.0. Samples: 18151466. Policy #0 lag: (min: 6.0, avg: 7.6, max: 32.0) [2023-10-11 20:29:31,034][70582] Avg episode reward: [(0, '301.130'), (1, '129.730')] [2023-10-11 20:29:31,235][71635] Updated weights for policy 1, policy_version 35442 (0.0007) [2023-10-11 20:29:31,598][71635] Updated weights for policy 1, policy_version 35452 (0.0007) [2023-10-11 20:29:32,318][71601] Updated weights for policy 0, policy_version 35460 (0.0008) [2023-10-11 20:29:32,681][71601] Updated weights for policy 0, policy_version 35470 (0.0008) [2023-10-11 20:29:33,054][71601] Updated weights for policy 0, policy_version 35480 (0.0008) [2023-10-11 20:29:35,311][71635] Updated weights for policy 1, policy_version 35462 (0.0010) [2023-10-11 20:29:35,676][71635] Updated weights for policy 1, policy_version 35472 (0.0008) [2023-10-11 20:29:36,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 72646656. Throughput: 0: 1809.6, 1: 1821.0. Samples: 18173640. Policy #0 lag: (min: 6.0, avg: 7.6, max: 32.0) [2023-10-11 20:29:36,035][70582] Avg episode reward: [(0, '314.260'), (1, '135.960')] [2023-10-11 20:29:36,039][71635] Updated weights for policy 1, policy_version 35482 (0.0008) [2023-10-11 20:29:36,738][71601] Updated weights for policy 0, policy_version 35490 (0.0007) [2023-10-11 20:29:37,113][71601] Updated weights for policy 0, policy_version 35500 (0.0007) [2023-10-11 20:29:37,486][71601] Updated weights for policy 0, policy_version 35510 (0.0009) [2023-10-11 20:29:37,855][71601] Updated weights for policy 0, policy_version 35520 (0.0007) [2023-10-11 20:29:39,880][71635] Updated weights for policy 1, policy_version 35492 (0.0009) [2023-10-11 20:29:40,249][71635] Updated weights for policy 1, policy_version 35502 (0.0011) [2023-10-11 20:29:40,606][71635] Updated weights for policy 1, policy_version 35512 (0.0010) [2023-10-11 20:29:41,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 72744960. Throughput: 0: 1802.1, 1: 1821.1. Samples: 18195410. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-11 20:29:41,034][70582] Avg episode reward: [(0, '317.800'), (1, '145.150')] [2023-10-11 20:29:41,790][71601] Updated weights for policy 0, policy_version 35530 (0.0009) [2023-10-11 20:29:42,168][71601] Updated weights for policy 0, policy_version 35540 (0.0007) [2023-10-11 20:29:42,542][71601] Updated weights for policy 0, policy_version 35550 (0.0009) [2023-10-11 20:29:44,148][71635] Updated weights for policy 1, policy_version 35522 (0.0010) [2023-10-11 20:29:44,510][71635] Updated weights for policy 1, policy_version 35532 (0.0008) [2023-10-11 20:29:44,883][71635] Updated weights for policy 1, policy_version 35542 (0.0007) [2023-10-11 20:29:45,239][71635] Updated weights for policy 1, policy_version 35552 (0.0007) [2023-10-11 20:29:46,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 72810496. Throughput: 0: 1806.0, 1: 1820.3. Samples: 18206136. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-11 20:29:46,035][70582] Avg episode reward: [(0, '317.900'), (1, '146.930')] [2023-10-11 20:29:46,104][71601] Updated weights for policy 0, policy_version 35560 (0.0008) [2023-10-11 20:29:46,473][71601] Updated weights for policy 0, policy_version 35570 (0.0007) [2023-10-11 20:29:46,841][71601] Updated weights for policy 0, policy_version 35580 (0.0009) [2023-10-11 20:29:48,916][71635] Updated weights for policy 1, policy_version 35562 (0.0007) [2023-10-11 20:29:49,289][71635] Updated weights for policy 1, policy_version 35572 (0.0008) [2023-10-11 20:29:49,654][71635] Updated weights for policy 1, policy_version 35582 (0.0007) [2023-10-11 20:29:50,592][71601] Updated weights for policy 0, policy_version 35590 (0.0008) [2023-10-11 20:29:50,971][71601] Updated weights for policy 0, policy_version 35600 (0.0011) [2023-10-11 20:29:51,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 72876032. Throughput: 0: 1806.4, 1: 1821.1. Samples: 18228084. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-11 20:29:51,034][70582] Avg episode reward: [(0, '317.880'), (1, '141.830')] [2023-10-11 20:29:51,343][71601] Updated weights for policy 0, policy_version 35610 (0.0009) [2023-10-11 20:29:53,370][71635] Updated weights for policy 1, policy_version 35592 (0.0009) [2023-10-11 20:29:53,744][71635] Updated weights for policy 1, policy_version 35602 (0.0009) [2023-10-11 20:29:54,112][71635] Updated weights for policy 1, policy_version 35612 (0.0008) [2023-10-11 20:29:54,991][71601] Updated weights for policy 0, policy_version 35620 (0.0008) [2023-10-11 20:29:55,362][71601] Updated weights for policy 0, policy_version 35630 (0.0007) [2023-10-11 20:29:55,734][71601] Updated weights for policy 0, policy_version 35640 (0.0010) [2023-10-11 20:29:56,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.8, 300 sec: 14551.2). Total num frames: 72941568. Throughput: 0: 1820.3, 1: 1812.6. Samples: 18249554. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-11 20:29:56,034][70582] Avg episode reward: [(0, '316.800'), (1, '139.110')] [2023-10-11 20:29:56,044][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000035648_36503552.pth... [2023-10-11 20:29:56,044][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000035616_36470784.pth... [2023-10-11 20:29:56,074][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000033952_34766848.pth [2023-10-11 20:29:56,081][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000033920_34734080.pth [2023-10-11 20:29:57,904][71635] Updated weights for policy 1, policy_version 35622 (0.0009) [2023-10-11 20:29:58,262][71635] Updated weights for policy 1, policy_version 35632 (0.0010) [2023-10-11 20:29:58,630][71635] Updated weights for policy 1, policy_version 35642 (0.0011) [2023-10-11 20:29:59,303][71601] Updated weights for policy 0, policy_version 35650 (0.0008) [2023-10-11 20:29:59,676][71601] Updated weights for policy 0, policy_version 35660 (0.0007) [2023-10-11 20:30:00,049][71601] Updated weights for policy 0, policy_version 35670 (0.0008) [2023-10-11 20:30:00,415][71601] Updated weights for policy 0, policy_version 35680 (0.0009) [2023-10-11 20:30:01,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 73039872. Throughput: 0: 1813.2, 1: 1817.6. Samples: 18260940. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-11 20:30:01,034][70582] Avg episode reward: [(0, '319.860'), (1, '139.110')] [2023-10-11 20:30:02,343][71635] Updated weights for policy 1, policy_version 35652 (0.0009) [2023-10-11 20:30:02,705][71635] Updated weights for policy 1, policy_version 35662 (0.0008) [2023-10-11 20:30:03,075][71635] Updated weights for policy 1, policy_version 35672 (0.0007) [2023-10-11 20:30:04,080][71601] Updated weights for policy 0, policy_version 35690 (0.0007) [2023-10-11 20:30:04,445][71601] Updated weights for policy 0, policy_version 35700 (0.0007) [2023-10-11 20:30:04,816][71601] Updated weights for policy 0, policy_version 35710 (0.0008) [2023-10-11 20:30:06,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 73105408. Throughput: 0: 1818.0, 1: 1810.5. Samples: 18282012. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-11 20:30:06,034][70582] Avg episode reward: [(0, '305.870'), (1, '139.640')] [2023-10-11 20:30:06,826][71635] Updated weights for policy 1, policy_version 35682 (0.0011) [2023-10-11 20:30:07,185][71635] Updated weights for policy 1, policy_version 35692 (0.0008) [2023-10-11 20:30:07,547][71635] Updated weights for policy 1, policy_version 35702 (0.0009) [2023-10-11 20:30:07,918][71635] Updated weights for policy 1, policy_version 35712 (0.0011) [2023-10-11 20:30:08,722][71601] Updated weights for policy 0, policy_version 35720 (0.0009) [2023-10-11 20:30:09,101][71601] Updated weights for policy 0, policy_version 35730 (0.0008) [2023-10-11 20:30:09,469][71601] Updated weights for policy 0, policy_version 35740 (0.0010) [2023-10-11 20:30:11,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 73170944. Throughput: 0: 1804.9, 1: 1814.8. Samples: 18303942. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-11 20:30:11,034][70582] Avg episode reward: [(0, '304.250'), (1, '139.820')] [2023-10-11 20:30:11,568][71635] Updated weights for policy 1, policy_version 35722 (0.0009) [2023-10-11 20:30:11,933][71635] Updated weights for policy 1, policy_version 35732 (0.0011) [2023-10-11 20:30:12,300][71635] Updated weights for policy 1, policy_version 35742 (0.0009) [2023-10-11 20:30:13,151][71601] Updated weights for policy 0, policy_version 35750 (0.0009) [2023-10-11 20:30:13,531][71601] Updated weights for policy 0, policy_version 35760 (0.0007) [2023-10-11 20:30:13,906][71601] Updated weights for policy 0, policy_version 35770 (0.0009) [2023-10-11 20:30:16,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 73236480. Throughput: 0: 1811.8, 1: 1812.9. Samples: 18314576. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-11 20:30:16,034][70582] Avg episode reward: [(0, '292.600'), (1, '144.150')] [2023-10-11 20:30:16,174][71635] Updated weights for policy 1, policy_version 35752 (0.0009) [2023-10-11 20:30:16,541][71635] Updated weights for policy 1, policy_version 35762 (0.0008) [2023-10-11 20:30:16,911][71635] Updated weights for policy 1, policy_version 35772 (0.0009) [2023-10-11 20:30:17,675][71601] Updated weights for policy 0, policy_version 35780 (0.0010) [2023-10-11 20:30:18,050][71601] Updated weights for policy 0, policy_version 35790 (0.0008) [2023-10-11 20:30:18,426][71601] Updated weights for policy 0, policy_version 35800 (0.0010) [2023-10-11 20:30:20,552][71635] Updated weights for policy 1, policy_version 35782 (0.0009) [2023-10-11 20:30:20,917][71635] Updated weights for policy 1, policy_version 35792 (0.0011) [2023-10-11 20:30:21,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 73302016. Throughput: 0: 1796.9, 1: 1811.4. Samples: 18336012. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-11 20:30:21,034][70582] Avg episode reward: [(0, '292.600'), (1, '143.360')] [2023-10-11 20:30:21,283][71635] Updated weights for policy 1, policy_version 35802 (0.0010) [2023-10-11 20:30:22,036][71601] Updated weights for policy 0, policy_version 35810 (0.0010) [2023-10-11 20:30:22,414][71601] Updated weights for policy 0, policy_version 35820 (0.0008) [2023-10-11 20:30:22,778][71601] Updated weights for policy 0, policy_version 35830 (0.0008) [2023-10-11 20:30:23,150][71601] Updated weights for policy 0, policy_version 35840 (0.0009) [2023-10-11 20:30:25,039][71635] Updated weights for policy 1, policy_version 35812 (0.0009) [2023-10-11 20:30:25,414][71635] Updated weights for policy 1, policy_version 35822 (0.0007) [2023-10-11 20:30:25,779][71635] Updated weights for policy 1, policy_version 35832 (0.0007) [2023-10-11 20:30:26,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 73367552. Throughput: 0: 1801.2, 1: 1818.8. Samples: 18358312. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-11 20:30:26,034][70582] Avg episode reward: [(0, '305.740'), (1, '143.500')] [2023-10-11 20:30:27,074][71601] Updated weights for policy 0, policy_version 35850 (0.0007) [2023-10-11 20:30:27,450][71601] Updated weights for policy 0, policy_version 35860 (0.0007) [2023-10-11 20:30:27,830][71601] Updated weights for policy 0, policy_version 35870 (0.0008) [2023-10-11 20:30:29,357][71635] Updated weights for policy 1, policy_version 35842 (0.0008) [2023-10-11 20:30:29,719][71635] Updated weights for policy 1, policy_version 35852 (0.0007) [2023-10-11 20:30:30,085][71635] Updated weights for policy 1, policy_version 35862 (0.0008) [2023-10-11 20:30:30,454][71635] Updated weights for policy 1, policy_version 35872 (0.0009) [2023-10-11 20:30:31,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 73465856. Throughput: 0: 1801.3, 1: 1814.7. Samples: 18368856. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-11 20:30:31,034][70582] Avg episode reward: [(0, '291.700'), (1, '143.650')] [2023-10-11 20:30:31,515][71601] Updated weights for policy 0, policy_version 35880 (0.0007) [2023-10-11 20:30:31,883][71601] Updated weights for policy 0, policy_version 35890 (0.0007) [2023-10-11 20:30:32,263][71601] Updated weights for policy 0, policy_version 35900 (0.0010) [2023-10-11 20:30:34,302][71635] Updated weights for policy 1, policy_version 35882 (0.0007) [2023-10-11 20:30:34,673][71635] Updated weights for policy 1, policy_version 35892 (0.0007) [2023-10-11 20:30:35,032][71635] Updated weights for policy 1, policy_version 35902 (0.0011) [2023-10-11 20:30:35,750][71601] Updated weights for policy 0, policy_version 35910 (0.0008) [2023-10-11 20:30:36,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 73531392. Throughput: 0: 1808.2, 1: 1817.5. Samples: 18391240. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-11 20:30:36,034][70582] Avg episode reward: [(0, '292.910'), (1, '153.770')] [2023-10-11 20:30:36,124][71601] Updated weights for policy 0, policy_version 35920 (0.0010) [2023-10-11 20:30:36,495][71601] Updated weights for policy 0, policy_version 35930 (0.0007) [2023-10-11 20:30:38,880][71635] Updated weights for policy 1, policy_version 35912 (0.0010) [2023-10-11 20:30:39,249][71635] Updated weights for policy 1, policy_version 35922 (0.0010) [2023-10-11 20:30:39,619][71635] Updated weights for policy 1, policy_version 35932 (0.0010) [2023-10-11 20:30:40,080][71601] Updated weights for policy 0, policy_version 35940 (0.0007) [2023-10-11 20:30:40,446][71601] Updated weights for policy 0, policy_version 35950 (0.0008) [2023-10-11 20:30:40,812][71601] Updated weights for policy 0, policy_version 35960 (0.0007) [2023-10-11 20:30:41,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 73596928. Throughput: 0: 1814.8, 1: 1806.1. Samples: 18412498. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-11 20:30:41,035][70582] Avg episode reward: [(0, '278.660'), (1, '152.040')] [2023-10-11 20:30:43,170][71635] Updated weights for policy 1, policy_version 35942 (0.0009) [2023-10-11 20:30:43,534][71635] Updated weights for policy 1, policy_version 35952 (0.0009) [2023-10-11 20:30:43,908][71635] Updated weights for policy 1, policy_version 35962 (0.0008) [2023-10-11 20:30:44,562][71601] Updated weights for policy 0, policy_version 35970 (0.0008) [2023-10-11 20:30:44,928][71601] Updated weights for policy 0, policy_version 35980 (0.0008) [2023-10-11 20:30:45,301][71601] Updated weights for policy 0, policy_version 35990 (0.0008) [2023-10-11 20:30:45,665][71601] Updated weights for policy 0, policy_version 36000 (0.0007) [2023-10-11 20:30:46,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 73695232. Throughput: 0: 1807.8, 1: 1816.8. Samples: 18424048. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-11 20:30:46,034][70582] Avg episode reward: [(0, '266.940'), (1, '152.040')] [2023-10-11 20:30:47,453][71635] Updated weights for policy 1, policy_version 35972 (0.0009) [2023-10-11 20:30:47,821][71635] Updated weights for policy 1, policy_version 35982 (0.0008) [2023-10-11 20:30:48,186][71635] Updated weights for policy 1, policy_version 35992 (0.0007) [2023-10-11 20:30:49,378][71601] Updated weights for policy 0, policy_version 36010 (0.0008) [2023-10-11 20:30:49,751][71601] Updated weights for policy 0, policy_version 36020 (0.0009) [2023-10-11 20:30:50,132][71601] Updated weights for policy 0, policy_version 36030 (0.0007) [2023-10-11 20:30:51,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 73760768. Throughput: 0: 1818.3, 1: 1816.3. Samples: 18445568. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-11 20:30:51,035][70582] Avg episode reward: [(0, '271.720'), (1, '152.040')] [2023-10-11 20:30:51,841][71635] Updated weights for policy 1, policy_version 36002 (0.0009) [2023-10-11 20:30:52,210][71635] Updated weights for policy 1, policy_version 36012 (0.0009) [2023-10-11 20:30:52,579][71635] Updated weights for policy 1, policy_version 36022 (0.0008) [2023-10-11 20:30:52,945][71635] Updated weights for policy 1, policy_version 36032 (0.0007) [2023-10-11 20:30:53,736][71601] Updated weights for policy 0, policy_version 36040 (0.0008) [2023-10-11 20:30:54,104][71601] Updated weights for policy 0, policy_version 36050 (0.0010) [2023-10-11 20:30:54,478][71601] Updated weights for policy 0, policy_version 36060 (0.0008) [2023-10-11 20:30:56,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 73826304. Throughput: 0: 1821.0, 1: 1819.5. Samples: 18467766. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-11 20:30:56,035][70582] Avg episode reward: [(0, '260.830'), (1, '153.240')] [2023-10-11 20:30:56,548][71635] Updated weights for policy 1, policy_version 36042 (0.0007) [2023-10-11 20:30:56,927][71635] Updated weights for policy 1, policy_version 36052 (0.0007) [2023-10-11 20:30:57,294][71635] Updated weights for policy 1, policy_version 36062 (0.0007) [2023-10-11 20:30:58,164][71601] Updated weights for policy 0, policy_version 36070 (0.0009) [2023-10-11 20:30:58,532][71601] Updated weights for policy 0, policy_version 36080 (0.0010) [2023-10-11 20:30:58,906][71601] Updated weights for policy 0, policy_version 36090 (0.0007) [2023-10-11 20:31:00,867][71635] Updated weights for policy 1, policy_version 36072 (0.0007) [2023-10-11 20:31:01,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 73891840. Throughput: 0: 1825.1, 1: 1820.5. Samples: 18478632. Policy #0 lag: (min: 17.0, avg: 27.1, max: 49.0) [2023-10-11 20:31:01,035][70582] Avg episode reward: [(0, '262.860'), (1, '153.350')] [2023-10-11 20:31:01,237][71635] Updated weights for policy 1, policy_version 36082 (0.0007) [2023-10-11 20:31:01,599][71635] Updated weights for policy 1, policy_version 36092 (0.0007) [2023-10-11 20:31:02,515][71601] Updated weights for policy 0, policy_version 36100 (0.0009) [2023-10-11 20:31:02,886][71601] Updated weights for policy 0, policy_version 36110 (0.0010) [2023-10-11 20:31:03,265][71601] Updated weights for policy 0, policy_version 36120 (0.0008) [2023-10-11 20:31:05,233][71635] Updated weights for policy 1, policy_version 36102 (0.0007) [2023-10-11 20:31:05,601][71635] Updated weights for policy 1, policy_version 36112 (0.0008) [2023-10-11 20:31:05,961][71635] Updated weights for policy 1, policy_version 36122 (0.0007) [2023-10-11 20:31:06,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 73957376. Throughput: 0: 1828.3, 1: 1826.1. Samples: 18500458. Policy #0 lag: (min: 17.0, avg: 27.1, max: 49.0) [2023-10-11 20:31:06,034][70582] Avg episode reward: [(0, '262.750'), (1, '152.420')] [2023-10-11 20:31:06,906][71601] Updated weights for policy 0, policy_version 36130 (0.0008) [2023-10-11 20:31:07,275][71601] Updated weights for policy 0, policy_version 36140 (0.0007) [2023-10-11 20:31:07,649][71601] Updated weights for policy 0, policy_version 36150 (0.0009) [2023-10-11 20:31:08,022][71601] Updated weights for policy 0, policy_version 36160 (0.0009) [2023-10-11 20:31:09,662][71635] Updated weights for policy 1, policy_version 36132 (0.0008) [2023-10-11 20:31:10,024][71635] Updated weights for policy 1, policy_version 36142 (0.0009) [2023-10-11 20:31:10,388][71635] Updated weights for policy 1, policy_version 36152 (0.0008) [2023-10-11 20:31:11,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 74055680. Throughput: 0: 1828.1, 1: 1819.9. Samples: 18522474. Policy #0 lag: (min: 17.0, avg: 27.1, max: 49.0) [2023-10-11 20:31:11,035][70582] Avg episode reward: [(0, '255.380'), (1, '137.490')] [2023-10-11 20:31:11,844][71601] Updated weights for policy 0, policy_version 36170 (0.0009) [2023-10-11 20:31:12,224][71601] Updated weights for policy 0, policy_version 36180 (0.0009) [2023-10-11 20:31:12,595][71601] Updated weights for policy 0, policy_version 36190 (0.0009) [2023-10-11 20:31:14,076][71635] Updated weights for policy 1, policy_version 36162 (0.0008) [2023-10-11 20:31:14,442][71635] Updated weights for policy 1, policy_version 36172 (0.0009) [2023-10-11 20:31:14,799][71635] Updated weights for policy 1, policy_version 36182 (0.0007) [2023-10-11 20:31:15,171][71635] Updated weights for policy 1, policy_version 36192 (0.0008) [2023-10-11 20:31:16,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 74121216. Throughput: 0: 1830.1, 1: 1829.3. Samples: 18533530. Policy #0 lag: (min: 17.0, avg: 27.1, max: 49.0) [2023-10-11 20:31:16,034][70582] Avg episode reward: [(0, '241.030'), (1, '147.150')] [2023-10-11 20:31:16,333][71601] Updated weights for policy 0, policy_version 36200 (0.0008) [2023-10-11 20:31:16,707][71601] Updated weights for policy 0, policy_version 36210 (0.0010) [2023-10-11 20:31:17,079][71601] Updated weights for policy 0, policy_version 36220 (0.0007) [2023-10-11 20:31:18,781][71635] Updated weights for policy 1, policy_version 36202 (0.0009) [2023-10-11 20:31:19,155][71635] Updated weights for policy 1, policy_version 36212 (0.0009) [2023-10-11 20:31:19,512][71635] Updated weights for policy 1, policy_version 36222 (0.0008) [2023-10-11 20:31:20,778][71601] Updated weights for policy 0, policy_version 36230 (0.0007) [2023-10-11 20:31:21,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 74186752. Throughput: 0: 1823.8, 1: 1821.1. Samples: 18555260. Policy #0 lag: (min: 17.0, avg: 27.1, max: 49.0) [2023-10-11 20:31:21,035][70582] Avg episode reward: [(0, '239.300'), (1, '144.000')] [2023-10-11 20:31:21,155][71601] Updated weights for policy 0, policy_version 36240 (0.0009) [2023-10-11 20:31:21,526][71601] Updated weights for policy 0, policy_version 36250 (0.0008) [2023-10-11 20:31:23,372][71635] Updated weights for policy 1, policy_version 36232 (0.0007) [2023-10-11 20:31:23,747][71635] Updated weights for policy 1, policy_version 36242 (0.0007) [2023-10-11 20:31:24,108][71635] Updated weights for policy 1, policy_version 36252 (0.0008) [2023-10-11 20:31:25,215][71601] Updated weights for policy 0, policy_version 36260 (0.0009) [2023-10-11 20:31:25,594][71601] Updated weights for policy 0, policy_version 36270 (0.0007) [2023-10-11 20:31:25,961][71601] Updated weights for policy 0, policy_version 36280 (0.0007) [2023-10-11 20:31:26,034][70582] Fps is (10 sec: 13106.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 74252288. Throughput: 0: 1817.6, 1: 1834.9. Samples: 18576860. Policy #0 lag: (min: 17.0, avg: 27.1, max: 49.0) [2023-10-11 20:31:26,035][70582] Avg episode reward: [(0, '218.760'), (1, '149.280')] [2023-10-11 20:31:27,914][71635] Updated weights for policy 1, policy_version 36262 (0.0008) [2023-10-11 20:31:28,291][71635] Updated weights for policy 1, policy_version 36272 (0.0009) [2023-10-11 20:31:28,657][71635] Updated weights for policy 1, policy_version 36282 (0.0010) [2023-10-11 20:31:29,759][71601] Updated weights for policy 0, policy_version 36290 (0.0009) [2023-10-11 20:31:30,129][71601] Updated weights for policy 0, policy_version 36300 (0.0007) [2023-10-11 20:31:30,495][71601] Updated weights for policy 0, policy_version 36310 (0.0009) [2023-10-11 20:31:30,854][71601] Updated weights for policy 0, policy_version 36320 (0.0007) [2023-10-11 20:31:31,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 74350592. Throughput: 0: 1815.9, 1: 1821.4. Samples: 18587726. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:31:31,034][70582] Avg episode reward: [(0, '218.900'), (1, '167.700')] [2023-10-11 20:31:31,035][71431] Saving new best policy, reward=167.700! [2023-10-11 20:31:32,375][71635] Updated weights for policy 1, policy_version 36292 (0.0010) [2023-10-11 20:31:32,744][71635] Updated weights for policy 1, policy_version 36302 (0.0009) [2023-10-11 20:31:33,113][71635] Updated weights for policy 1, policy_version 36312 (0.0007) [2023-10-11 20:31:34,545][71601] Updated weights for policy 0, policy_version 36330 (0.0010) [2023-10-11 20:31:34,911][71601] Updated weights for policy 0, policy_version 36340 (0.0008) [2023-10-11 20:31:35,284][71601] Updated weights for policy 0, policy_version 36350 (0.0008) [2023-10-11 20:31:36,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 74416128. Throughput: 0: 1818.3, 1: 1817.1. Samples: 18609162. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:31:36,035][70582] Avg episode reward: [(0, '220.200'), (1, '167.990')] [2023-10-11 20:31:36,036][71431] Saving new best policy, reward=167.990! [2023-10-11 20:31:36,926][71635] Updated weights for policy 1, policy_version 36322 (0.0008) [2023-10-11 20:31:37,290][71635] Updated weights for policy 1, policy_version 36332 (0.0010) [2023-10-11 20:31:37,650][71635] Updated weights for policy 1, policy_version 36342 (0.0010) [2023-10-11 20:31:38,024][71635] Updated weights for policy 1, policy_version 36352 (0.0010) [2023-10-11 20:31:38,945][71601] Updated weights for policy 0, policy_version 36360 (0.0008) [2023-10-11 20:31:39,319][71601] Updated weights for policy 0, policy_version 36370 (0.0009) [2023-10-11 20:31:39,694][71601] Updated weights for policy 0, policy_version 36380 (0.0008) [2023-10-11 20:31:41,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 74481664. Throughput: 0: 1806.7, 1: 1809.9. Samples: 18630510. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:31:41,034][70582] Avg episode reward: [(0, '221.810'), (1, '170.440')] [2023-10-11 20:31:41,044][71431] Saving new best policy, reward=170.440! [2023-10-11 20:31:41,839][71635] Updated weights for policy 1, policy_version 36362 (0.0008) [2023-10-11 20:31:42,205][71635] Updated weights for policy 1, policy_version 36372 (0.0008) [2023-10-11 20:31:42,571][71635] Updated weights for policy 1, policy_version 36382 (0.0008) [2023-10-11 20:31:43,272][71601] Updated weights for policy 0, policy_version 36390 (0.0007) [2023-10-11 20:31:43,641][71601] Updated weights for policy 0, policy_version 36400 (0.0007) [2023-10-11 20:31:44,014][71601] Updated weights for policy 0, policy_version 36410 (0.0009) [2023-10-11 20:31:46,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 74547200. Throughput: 0: 1810.3, 1: 1808.7. Samples: 18641486. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:31:46,035][70582] Avg episode reward: [(0, '212.320'), (1, '169.260')] [2023-10-11 20:31:46,310][71635] Updated weights for policy 1, policy_version 36392 (0.0008) [2023-10-11 20:31:46,671][71635] Updated weights for policy 1, policy_version 36402 (0.0009) [2023-10-11 20:31:47,043][71635] Updated weights for policy 1, policy_version 36412 (0.0008) [2023-10-11 20:31:47,841][71601] Updated weights for policy 0, policy_version 36420 (0.0007) [2023-10-11 20:31:48,217][71601] Updated weights for policy 0, policy_version 36430 (0.0009) [2023-10-11 20:31:48,597][71601] Updated weights for policy 0, policy_version 36440 (0.0010) [2023-10-11 20:31:50,627][71635] Updated weights for policy 1, policy_version 36422 (0.0007) [2023-10-11 20:31:51,000][71635] Updated weights for policy 1, policy_version 36432 (0.0008) [2023-10-11 20:31:51,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 74612736. Throughput: 0: 1810.4, 1: 1809.3. Samples: 18663344. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:31:51,035][70582] Avg episode reward: [(0, '197.510'), (1, '158.170')] [2023-10-11 20:31:51,365][71635] Updated weights for policy 1, policy_version 36442 (0.0007) [2023-10-11 20:31:52,254][71601] Updated weights for policy 0, policy_version 36450 (0.0008) [2023-10-11 20:31:52,624][71601] Updated weights for policy 0, policy_version 36460 (0.0007) [2023-10-11 20:31:52,993][71601] Updated weights for policy 0, policy_version 36470 (0.0007) [2023-10-11 20:31:53,363][71601] Updated weights for policy 0, policy_version 36480 (0.0007) [2023-10-11 20:31:55,198][71635] Updated weights for policy 1, policy_version 36452 (0.0008) [2023-10-11 20:31:55,565][71635] Updated weights for policy 1, policy_version 36462 (0.0010) [2023-10-11 20:31:55,936][71635] Updated weights for policy 1, policy_version 36472 (0.0009) [2023-10-11 20:31:56,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 74678272. Throughput: 0: 1815.1, 1: 1816.3. Samples: 18685886. Policy #0 lag: (min: 9.0, avg: 35.4, max: 40.0) [2023-10-11 20:31:56,034][70582] Avg episode reward: [(0, '189.170'), (1, '156.040')] [2023-10-11 20:31:56,045][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000036480_37355520.pth... [2023-10-11 20:31:56,086][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000034784_35618816.pth [2023-10-11 20:31:56,230][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000036480_37355520.pth... [2023-10-11 20:31:56,264][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000034752_35586048.pth [2023-10-11 20:31:57,052][71601] Updated weights for policy 0, policy_version 36490 (0.0008) [2023-10-11 20:31:57,426][71601] Updated weights for policy 0, policy_version 36500 (0.0007) [2023-10-11 20:31:57,795][71601] Updated weights for policy 0, policy_version 36510 (0.0011) [2023-10-11 20:31:59,598][71635] Updated weights for policy 1, policy_version 36482 (0.0009) [2023-10-11 20:31:59,957][71635] Updated weights for policy 1, policy_version 36492 (0.0009) [2023-10-11 20:32:00,320][71635] Updated weights for policy 1, policy_version 36502 (0.0011) [2023-10-11 20:32:00,694][71635] Updated weights for policy 1, policy_version 36512 (0.0009) [2023-10-11 20:32:01,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 74776576. Throughput: 0: 1816.0, 1: 1799.6. Samples: 18696234. Policy #0 lag: (min: 9.0, avg: 35.4, max: 40.0) [2023-10-11 20:32:01,034][70582] Avg episode reward: [(0, '189.320'), (1, '163.500')] [2023-10-11 20:32:01,373][71601] Updated weights for policy 0, policy_version 36520 (0.0007) [2023-10-11 20:32:01,745][71601] Updated weights for policy 0, policy_version 36530 (0.0008) [2023-10-11 20:32:02,121][71601] Updated weights for policy 0, policy_version 36540 (0.0007) [2023-10-11 20:32:04,351][71635] Updated weights for policy 1, policy_version 36522 (0.0010) [2023-10-11 20:32:04,714][71635] Updated weights for policy 1, policy_version 36532 (0.0011) [2023-10-11 20:32:05,085][71635] Updated weights for policy 1, policy_version 36542 (0.0007) [2023-10-11 20:32:05,708][71601] Updated weights for policy 0, policy_version 36550 (0.0008) [2023-10-11 20:32:06,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 74842112. Throughput: 0: 1816.8, 1: 1807.4. Samples: 18718350. Policy #0 lag: (min: 9.0, avg: 35.4, max: 40.0) [2023-10-11 20:32:06,034][70582] Avg episode reward: [(0, '191.300'), (1, '163.680')] [2023-10-11 20:32:06,086][71601] Updated weights for policy 0, policy_version 36560 (0.0008) [2023-10-11 20:32:06,456][71601] Updated weights for policy 0, policy_version 36570 (0.0011) [2023-10-11 20:32:08,825][71635] Updated weights for policy 1, policy_version 36552 (0.0009) [2023-10-11 20:32:09,182][71635] Updated weights for policy 1, policy_version 36562 (0.0011) [2023-10-11 20:32:09,545][71635] Updated weights for policy 1, policy_version 36572 (0.0011) [2023-10-11 20:32:10,070][71601] Updated weights for policy 0, policy_version 36580 (0.0007) [2023-10-11 20:32:10,445][71601] Updated weights for policy 0, policy_version 36590 (0.0008) [2023-10-11 20:32:10,806][71601] Updated weights for policy 0, policy_version 36600 (0.0008) [2023-10-11 20:32:11,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 74907648. Throughput: 0: 1818.7, 1: 1794.0. Samples: 18739430. Policy #0 lag: (min: 9.0, avg: 35.4, max: 40.0) [2023-10-11 20:32:11,035][70582] Avg episode reward: [(0, '186.640'), (1, '173.900')] [2023-10-11 20:32:11,046][71431] Saving new best policy, reward=173.900! [2023-10-11 20:32:13,304][71635] Updated weights for policy 1, policy_version 36582 (0.0010) [2023-10-11 20:32:13,683][71635] Updated weights for policy 1, policy_version 36592 (0.0009) [2023-10-11 20:32:14,048][71635] Updated weights for policy 1, policy_version 36602 (0.0009) [2023-10-11 20:32:14,619][71601] Updated weights for policy 0, policy_version 36610 (0.0009) [2023-10-11 20:32:14,982][71601] Updated weights for policy 0, policy_version 36620 (0.0009) [2023-10-11 20:32:15,348][71601] Updated weights for policy 0, policy_version 36630 (0.0009) [2023-10-11 20:32:15,717][71601] Updated weights for policy 0, policy_version 36640 (0.0007) [2023-10-11 20:32:16,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 75005952. Throughput: 0: 1820.8, 1: 1810.7. Samples: 18751146. Policy #0 lag: (min: 9.0, avg: 35.4, max: 40.0) [2023-10-11 20:32:16,035][70582] Avg episode reward: [(0, '186.590'), (1, '153.930')] [2023-10-11 20:32:17,724][71635] Updated weights for policy 1, policy_version 36612 (0.0009) [2023-10-11 20:32:18,090][71635] Updated weights for policy 1, policy_version 36622 (0.0008) [2023-10-11 20:32:18,461][71635] Updated weights for policy 1, policy_version 36632 (0.0007) [2023-10-11 20:32:19,412][71601] Updated weights for policy 0, policy_version 36650 (0.0008) [2023-10-11 20:32:19,782][71601] Updated weights for policy 0, policy_version 36660 (0.0007) [2023-10-11 20:32:20,146][71601] Updated weights for policy 0, policy_version 36670 (0.0009) [2023-10-11 20:32:21,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 75071488. Throughput: 0: 1820.0, 1: 1809.1. Samples: 18772468. Policy #0 lag: (min: 1.0, avg: 8.6, max: 33.0) [2023-10-11 20:32:21,035][70582] Avg episode reward: [(0, '186.590'), (1, '155.200')] [2023-10-11 20:32:21,978][71635] Updated weights for policy 1, policy_version 36642 (0.0009) [2023-10-11 20:32:22,353][71635] Updated weights for policy 1, policy_version 36652 (0.0009) [2023-10-11 20:32:22,714][71635] Updated weights for policy 1, policy_version 36662 (0.0010) [2023-10-11 20:32:23,075][71635] Updated weights for policy 1, policy_version 36672 (0.0008) [2023-10-11 20:32:23,751][71601] Updated weights for policy 0, policy_version 36680 (0.0008) [2023-10-11 20:32:24,122][71601] Updated weights for policy 0, policy_version 36690 (0.0008) [2023-10-11 20:32:24,496][71601] Updated weights for policy 0, policy_version 36700 (0.0009) [2023-10-11 20:32:26,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 75137024. Throughput: 0: 1830.2, 1: 1810.7. Samples: 18794348. Policy #0 lag: (min: 1.0, avg: 8.6, max: 33.0) [2023-10-11 20:32:26,034][70582] Avg episode reward: [(0, '186.590'), (1, '155.530')] [2023-10-11 20:32:26,844][71635] Updated weights for policy 1, policy_version 36682 (0.0011) [2023-10-11 20:32:27,208][71635] Updated weights for policy 1, policy_version 36692 (0.0009) [2023-10-11 20:32:27,576][71635] Updated weights for policy 1, policy_version 36702 (0.0010) [2023-10-11 20:32:28,182][71601] Updated weights for policy 0, policy_version 36710 (0.0008) [2023-10-11 20:32:28,559][71601] Updated weights for policy 0, policy_version 36720 (0.0009) [2023-10-11 20:32:28,938][71601] Updated weights for policy 0, policy_version 36730 (0.0008) [2023-10-11 20:32:31,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 75202560. Throughput: 0: 1828.3, 1: 1814.0. Samples: 18805392. Policy #0 lag: (min: 1.0, avg: 8.6, max: 33.0) [2023-10-11 20:32:31,035][70582] Avg episode reward: [(0, '186.590'), (1, '155.710')] [2023-10-11 20:32:31,366][71635] Updated weights for policy 1, policy_version 36712 (0.0009) [2023-10-11 20:32:31,741][71635] Updated weights for policy 1, policy_version 36722 (0.0009) [2023-10-11 20:32:32,113][71635] Updated weights for policy 1, policy_version 36732 (0.0009) [2023-10-11 20:32:32,770][71601] Updated weights for policy 0, policy_version 36740 (0.0010) [2023-10-11 20:32:33,141][71601] Updated weights for policy 0, policy_version 36750 (0.0009) [2023-10-11 20:32:33,523][71601] Updated weights for policy 0, policy_version 36760 (0.0009) [2023-10-11 20:32:35,945][71635] Updated weights for policy 1, policy_version 36742 (0.0010) [2023-10-11 20:32:36,034][70582] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 75268096. Throughput: 0: 1827.7, 1: 1810.2. Samples: 18827050. Policy #0 lag: (min: 1.0, avg: 8.6, max: 33.0) [2023-10-11 20:32:36,035][70582] Avg episode reward: [(0, '186.590'), (1, '155.550')] [2023-10-11 20:32:36,312][71635] Updated weights for policy 1, policy_version 36752 (0.0010) [2023-10-11 20:32:36,674][71635] Updated weights for policy 1, policy_version 36762 (0.0009) [2023-10-11 20:32:37,053][71601] Updated weights for policy 0, policy_version 36770 (0.0010) [2023-10-11 20:32:37,440][71601] Updated weights for policy 0, policy_version 36780 (0.0010) [2023-10-11 20:32:37,812][71601] Updated weights for policy 0, policy_version 36790 (0.0010) [2023-10-11 20:32:38,183][71601] Updated weights for policy 0, policy_version 36800 (0.0007) [2023-10-11 20:32:40,252][71635] Updated weights for policy 1, policy_version 36772 (0.0010) [2023-10-11 20:32:40,618][71635] Updated weights for policy 1, policy_version 36782 (0.0009) [2023-10-11 20:32:40,986][71635] Updated weights for policy 1, policy_version 36792 (0.0009) [2023-10-11 20:32:41,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 75333632. Throughput: 0: 1826.7, 1: 1810.5. Samples: 18849558. Policy #0 lag: (min: 1.0, avg: 8.6, max: 33.0) [2023-10-11 20:32:41,035][70582] Avg episode reward: [(0, '186.590'), (1, '155.400')] [2023-10-11 20:32:41,869][71601] Updated weights for policy 0, policy_version 36810 (0.0009) [2023-10-11 20:32:42,249][71601] Updated weights for policy 0, policy_version 36820 (0.0009) [2023-10-11 20:32:42,620][71601] Updated weights for policy 0, policy_version 36830 (0.0010) [2023-10-11 20:32:44,679][71635] Updated weights for policy 1, policy_version 36802 (0.0008) [2023-10-11 20:32:45,047][71635] Updated weights for policy 1, policy_version 36812 (0.0009) [2023-10-11 20:32:45,415][71635] Updated weights for policy 1, policy_version 36822 (0.0009) [2023-10-11 20:32:45,768][71635] Updated weights for policy 1, policy_version 36832 (0.0009) [2023-10-11 20:32:46,034][70582] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 75431936. Throughput: 0: 1822.6, 1: 1811.6. Samples: 18859770. Policy #0 lag: (min: 1.0, avg: 8.6, max: 33.0) [2023-10-11 20:32:46,034][70582] Avg episode reward: [(0, '186.590'), (1, '155.310')] [2023-10-11 20:32:46,366][71601] Updated weights for policy 0, policy_version 36840 (0.0008) [2023-10-11 20:32:46,737][71601] Updated weights for policy 0, policy_version 36850 (0.0009) [2023-10-11 20:32:47,104][71601] Updated weights for policy 0, policy_version 36860 (0.0007) [2023-10-11 20:32:49,428][71635] Updated weights for policy 1, policy_version 36842 (0.0008) [2023-10-11 20:32:49,796][71635] Updated weights for policy 1, policy_version 36852 (0.0009) [2023-10-11 20:32:50,165][71635] Updated weights for policy 1, policy_version 36862 (0.0010) [2023-10-11 20:32:50,744][71601] Updated weights for policy 0, policy_version 36870 (0.0007) [2023-10-11 20:32:51,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 75497472. Throughput: 0: 1824.5, 1: 1818.4. Samples: 18882284. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-11 20:32:51,035][70582] Avg episode reward: [(0, '186.590'), (1, '169.370')] [2023-10-11 20:32:51,110][71601] Updated weights for policy 0, policy_version 36880 (0.0008) [2023-10-11 20:32:51,481][71601] Updated weights for policy 0, policy_version 36890 (0.0008) [2023-10-11 20:32:53,797][71635] Updated weights for policy 1, policy_version 36872 (0.0008) [2023-10-11 20:32:54,169][71635] Updated weights for policy 1, policy_version 36882 (0.0008) [2023-10-11 20:32:54,537][71635] Updated weights for policy 1, policy_version 36892 (0.0009) [2023-10-11 20:32:55,161][71601] Updated weights for policy 0, policy_version 36900 (0.0008) [2023-10-11 20:32:55,524][71601] Updated weights for policy 0, policy_version 36910 (0.0008) [2023-10-11 20:32:55,899][71601] Updated weights for policy 0, policy_version 36920 (0.0007) [2023-10-11 20:32:56,034][70582] Fps is (10 sec: 13106.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 75563008. Throughput: 0: 1822.6, 1: 1819.5. Samples: 18903322. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-11 20:32:56,035][70582] Avg episode reward: [(0, '186.590'), (1, '169.420')] [2023-10-11 20:32:58,349][71635] Updated weights for policy 1, policy_version 36902 (0.0009) [2023-10-11 20:32:58,720][71635] Updated weights for policy 1, policy_version 36912 (0.0011) [2023-10-11 20:32:59,078][71635] Updated weights for policy 1, policy_version 36922 (0.0011) [2023-10-11 20:32:59,587][71601] Updated weights for policy 0, policy_version 36930 (0.0008) [2023-10-11 20:32:59,954][71601] Updated weights for policy 0, policy_version 36940 (0.0008) [2023-10-11 20:33:00,315][71601] Updated weights for policy 0, policy_version 36950 (0.0009) [2023-10-11 20:33:00,693][71601] Updated weights for policy 0, policy_version 36960 (0.0009) [2023-10-11 20:33:01,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 75661312. Throughput: 0: 1822.9, 1: 1818.5. Samples: 18915010. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-11 20:33:01,034][70582] Avg episode reward: [(0, '187.680'), (1, '169.900')] [2023-10-11 20:33:02,857][71635] Updated weights for policy 1, policy_version 36932 (0.0010) [2023-10-11 20:33:03,235][71635] Updated weights for policy 1, policy_version 36942 (0.0010) [2023-10-11 20:33:03,598][71635] Updated weights for policy 1, policy_version 36952 (0.0009) [2023-10-11 20:33:04,384][71601] Updated weights for policy 0, policy_version 36970 (0.0010) [2023-10-11 20:33:04,758][71601] Updated weights for policy 0, policy_version 36980 (0.0007) [2023-10-11 20:33:05,130][71601] Updated weights for policy 0, policy_version 36990 (0.0007) [2023-10-11 20:33:06,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 75726848. Throughput: 0: 1821.5, 1: 1813.5. Samples: 18936042. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-11 20:33:06,035][70582] Avg episode reward: [(0, '187.690'), (1, '183.670')] [2023-10-11 20:33:06,038][71431] Saving new best policy, reward=183.670! [2023-10-11 20:33:07,291][71635] Updated weights for policy 1, policy_version 36962 (0.0010) [2023-10-11 20:33:07,649][71635] Updated weights for policy 1, policy_version 36972 (0.0010) [2023-10-11 20:33:08,020][71635] Updated weights for policy 1, policy_version 36982 (0.0008) [2023-10-11 20:33:08,387][71635] Updated weights for policy 1, policy_version 36992 (0.0007) [2023-10-11 20:33:08,832][71601] Updated weights for policy 0, policy_version 37000 (0.0008) [2023-10-11 20:33:09,205][71601] Updated weights for policy 0, policy_version 37010 (0.0009) [2023-10-11 20:33:09,579][71601] Updated weights for policy 0, policy_version 37020 (0.0011) [2023-10-11 20:33:11,034][70582] Fps is (10 sec: 13106.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 75792384. Throughput: 0: 1817.5, 1: 1816.7. Samples: 18957888. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-11 20:33:11,035][70582] Avg episode reward: [(0, '187.690'), (1, '183.710')] [2023-10-11 20:33:11,048][71431] Saving new best policy, reward=183.710! [2023-10-11 20:33:12,037][71635] Updated weights for policy 1, policy_version 37002 (0.0007) [2023-10-11 20:33:12,408][71635] Updated weights for policy 1, policy_version 37012 (0.0009) [2023-10-11 20:33:12,772][71635] Updated weights for policy 1, policy_version 37022 (0.0008) [2023-10-11 20:33:13,300][71601] Updated weights for policy 0, policy_version 37030 (0.0008) [2023-10-11 20:33:13,674][71601] Updated weights for policy 0, policy_version 37040 (0.0007) [2023-10-11 20:33:14,043][71601] Updated weights for policy 0, policy_version 37050 (0.0009) [2023-10-11 20:33:16,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 75857920. Throughput: 0: 1816.9, 1: 1814.0. Samples: 18968784. Policy #0 lag: (min: 24.0, avg: 47.0, max: 56.0) [2023-10-11 20:33:16,035][70582] Avg episode reward: [(0, '197.630'), (1, '183.640')] [2023-10-11 20:33:16,433][71635] Updated weights for policy 1, policy_version 37032 (0.0007) [2023-10-11 20:33:16,806][71635] Updated weights for policy 1, policy_version 37042 (0.0011) [2023-10-11 20:33:17,174][71635] Updated weights for policy 1, policy_version 37052 (0.0010) [2023-10-11 20:33:17,922][71601] Updated weights for policy 0, policy_version 37060 (0.0010) [2023-10-11 20:33:18,296][71601] Updated weights for policy 0, policy_version 37070 (0.0008) [2023-10-11 20:33:18,660][71601] Updated weights for policy 0, policy_version 37080 (0.0009) [2023-10-11 20:33:20,934][71635] Updated weights for policy 1, policy_version 37062 (0.0008) [2023-10-11 20:33:21,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 75923456. Throughput: 0: 1811.3, 1: 1817.2. Samples: 18990332. Policy #0 lag: (min: 24.0, avg: 47.0, max: 56.0) [2023-10-11 20:33:21,035][70582] Avg episode reward: [(0, '197.970'), (1, '183.640')] [2023-10-11 20:33:21,295][71635] Updated weights for policy 1, policy_version 37072 (0.0008) [2023-10-11 20:33:21,661][71635] Updated weights for policy 1, policy_version 37082 (0.0007) [2023-10-11 20:33:22,386][71601] Updated weights for policy 0, policy_version 37090 (0.0008) [2023-10-11 20:33:22,752][71601] Updated weights for policy 0, policy_version 37100 (0.0007) [2023-10-11 20:33:23,130][71601] Updated weights for policy 0, policy_version 37110 (0.0010) [2023-10-11 20:33:23,502][71601] Updated weights for policy 0, policy_version 37120 (0.0010) [2023-10-11 20:33:25,525][71635] Updated weights for policy 1, policy_version 37092 (0.0007) [2023-10-11 20:33:25,895][71635] Updated weights for policy 1, policy_version 37102 (0.0008) [2023-10-11 20:33:26,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 75988992. Throughput: 0: 1808.2, 1: 1824.2. Samples: 19013018. Policy #0 lag: (min: 24.0, avg: 47.0, max: 56.0) [2023-10-11 20:33:26,035][70582] Avg episode reward: [(0, '205.500'), (1, '187.320')] [2023-10-11 20:33:26,265][71635] Updated weights for policy 1, policy_version 37112 (0.0008) [2023-10-11 20:33:26,552][71431] Saving new best policy, reward=187.320! [2023-10-11 20:33:27,184][71601] Updated weights for policy 0, policy_version 37130 (0.0008) [2023-10-11 20:33:27,562][71601] Updated weights for policy 0, policy_version 37140 (0.0009) [2023-10-11 20:33:27,932][71601] Updated weights for policy 0, policy_version 37150 (0.0011) [2023-10-11 20:33:29,952][71635] Updated weights for policy 1, policy_version 37122 (0.0007) [2023-10-11 20:33:30,324][71635] Updated weights for policy 1, policy_version 37132 (0.0009) [2023-10-11 20:33:30,689][71635] Updated weights for policy 1, policy_version 37142 (0.0008) [2023-10-11 20:33:31,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 76054528. Throughput: 0: 1810.2, 1: 1815.5. Samples: 19022924. Policy #0 lag: (min: 24.0, avg: 47.0, max: 56.0) [2023-10-11 20:33:31,034][70582] Avg episode reward: [(0, '189.870'), (1, '191.360')] [2023-10-11 20:33:31,044][71431] Saving new best policy, reward=191.360! [2023-10-11 20:33:31,045][71635] Updated weights for policy 1, policy_version 37152 (0.0009) [2023-10-11 20:33:31,792][71601] Updated weights for policy 0, policy_version 37160 (0.0010) [2023-10-11 20:33:32,165][71601] Updated weights for policy 0, policy_version 37170 (0.0009) [2023-10-11 20:33:32,538][71601] Updated weights for policy 0, policy_version 37180 (0.0009) [2023-10-11 20:33:34,656][71635] Updated weights for policy 1, policy_version 37162 (0.0009) [2023-10-11 20:33:35,021][71635] Updated weights for policy 1, policy_version 37172 (0.0008) [2023-10-11 20:33:35,394][71635] Updated weights for policy 1, policy_version 37182 (0.0007) [2023-10-11 20:33:36,034][70582] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 76152832. Throughput: 0: 1802.8, 1: 1822.4. Samples: 19045416. Policy #0 lag: (min: 24.0, avg: 47.0, max: 56.0) [2023-10-11 20:33:36,034][70582] Avg episode reward: [(0, '175.760'), (1, '178.440')] [2023-10-11 20:33:36,244][71601] Updated weights for policy 0, policy_version 37190 (0.0008) [2023-10-11 20:33:36,615][71601] Updated weights for policy 0, policy_version 37200 (0.0008) [2023-10-11 20:33:36,989][71601] Updated weights for policy 0, policy_version 37210 (0.0007) [2023-10-11 20:33:39,211][71635] Updated weights for policy 1, policy_version 37192 (0.0010) [2023-10-11 20:33:39,573][71635] Updated weights for policy 1, policy_version 37202 (0.0009) [2023-10-11 20:33:39,939][71635] Updated weights for policy 1, policy_version 37212 (0.0009) [2023-10-11 20:33:40,594][71601] Updated weights for policy 0, policy_version 37220 (0.0008) [2023-10-11 20:33:40,962][71601] Updated weights for policy 0, policy_version 37230 (0.0009) [2023-10-11 20:33:41,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 76218368. Throughput: 0: 1813.6, 1: 1814.0. Samples: 19066564. Policy #0 lag: (min: 24.0, avg: 47.0, max: 56.0) [2023-10-11 20:33:41,035][70582] Avg episode reward: [(0, '182.570'), (1, '188.400')] [2023-10-11 20:33:41,342][71601] Updated weights for policy 0, policy_version 37240 (0.0010) [2023-10-11 20:33:43,600][71635] Updated weights for policy 1, policy_version 37222 (0.0009) [2023-10-11 20:33:43,957][71635] Updated weights for policy 1, policy_version 37232 (0.0008) [2023-10-11 20:33:44,333][71635] Updated weights for policy 1, policy_version 37242 (0.0008) [2023-10-11 20:33:45,137][71601] Updated weights for policy 0, policy_version 37250 (0.0010) [2023-10-11 20:33:45,514][71601] Updated weights for policy 0, policy_version 37260 (0.0009) [2023-10-11 20:33:45,893][71601] Updated weights for policy 0, policy_version 37270 (0.0008) [2023-10-11 20:33:46,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 76283904. Throughput: 0: 1796.6, 1: 1825.6. Samples: 19078006. Policy #0 lag: (min: 1.0, avg: 7.7, max: 33.0) [2023-10-11 20:33:46,034][70582] Avg episode reward: [(0, '170.670'), (1, '178.860')] [2023-10-11 20:33:46,264][71601] Updated weights for policy 0, policy_version 37280 (0.0010) [2023-10-11 20:33:47,983][71635] Updated weights for policy 1, policy_version 37252 (0.0009) [2023-10-11 20:33:48,360][71635] Updated weights for policy 1, policy_version 37262 (0.0008) [2023-10-11 20:33:48,723][71635] Updated weights for policy 1, policy_version 37272 (0.0008) [2023-10-11 20:33:49,992][71601] Updated weights for policy 0, policy_version 37290 (0.0007) [2023-10-11 20:33:50,357][71601] Updated weights for policy 0, policy_version 37300 (0.0008) [2023-10-11 20:33:50,726][71601] Updated weights for policy 0, policy_version 37310 (0.0008) [2023-10-11 20:33:51,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.3). Total num frames: 76382208. Throughput: 0: 1804.9, 1: 1818.7. Samples: 19099104. Policy #0 lag: (min: 1.0, avg: 7.7, max: 33.0) [2023-10-11 20:33:51,034][70582] Avg episode reward: [(0, '165.890'), (1, '170.520')] [2023-10-11 20:33:52,448][71635] Updated weights for policy 1, policy_version 37282 (0.0011) [2023-10-11 20:33:52,823][71635] Updated weights for policy 1, policy_version 37292 (0.0011) [2023-10-11 20:33:53,193][71635] Updated weights for policy 1, policy_version 37302 (0.0010) [2023-10-11 20:33:53,558][71635] Updated weights for policy 1, policy_version 37312 (0.0008) [2023-10-11 20:33:54,403][71601] Updated weights for policy 0, policy_version 37320 (0.0008) [2023-10-11 20:33:54,775][71601] Updated weights for policy 0, policy_version 37330 (0.0007) [2023-10-11 20:33:55,151][71601] Updated weights for policy 0, policy_version 37340 (0.0009) [2023-10-11 20:33:56,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 76447744. Throughput: 0: 1796.1, 1: 1816.3. Samples: 19120446. Policy #0 lag: (min: 1.0, avg: 7.7, max: 33.0) [2023-10-11 20:33:56,034][70582] Avg episode reward: [(0, '166.150'), (1, '173.190')] [2023-10-11 20:33:56,041][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000037344_38240256.pth... [2023-10-11 20:33:56,041][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000037312_38207488.pth... [2023-10-11 20:33:56,071][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000035616_36470784.pth [2023-10-11 20:33:56,072][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000035648_36503552.pth [2023-10-11 20:33:57,257][71635] Updated weights for policy 1, policy_version 37322 (0.0010) [2023-10-11 20:33:57,622][71635] Updated weights for policy 1, policy_version 37332 (0.0010) [2023-10-11 20:33:57,988][71635] Updated weights for policy 1, policy_version 37342 (0.0009) [2023-10-11 20:33:58,669][71601] Updated weights for policy 0, policy_version 37350 (0.0008) [2023-10-11 20:33:59,037][71601] Updated weights for policy 0, policy_version 37360 (0.0008) [2023-10-11 20:33:59,410][71601] Updated weights for policy 0, policy_version 37370 (0.0007) [2023-10-11 20:34:01,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 76513280. Throughput: 0: 1813.2, 1: 1814.7. Samples: 19132040. Policy #0 lag: (min: 1.0, avg: 7.7, max: 33.0) [2023-10-11 20:34:01,035][70582] Avg episode reward: [(0, '166.330'), (1, '162.740')] [2023-10-11 20:34:01,710][71635] Updated weights for policy 1, policy_version 37352 (0.0007) [2023-10-11 20:34:02,075][71635] Updated weights for policy 1, policy_version 37362 (0.0009) [2023-10-11 20:34:02,437][71635] Updated weights for policy 1, policy_version 37372 (0.0010) [2023-10-11 20:34:03,057][71601] Updated weights for policy 0, policy_version 37380 (0.0009) [2023-10-11 20:34:03,423][71601] Updated weights for policy 0, policy_version 37390 (0.0007) [2023-10-11 20:34:03,800][71601] Updated weights for policy 0, policy_version 37400 (0.0007) [2023-10-11 20:34:05,988][71635] Updated weights for policy 1, policy_version 37382 (0.0008) [2023-10-11 20:34:06,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 76578816. Throughput: 0: 1810.4, 1: 1817.2. Samples: 19153574. Policy #0 lag: (min: 1.0, avg: 7.7, max: 33.0) [2023-10-11 20:34:06,035][70582] Avg episode reward: [(0, '166.230'), (1, '162.680')] [2023-10-11 20:34:06,357][71635] Updated weights for policy 1, policy_version 37392 (0.0007) [2023-10-11 20:34:06,726][71635] Updated weights for policy 1, policy_version 37402 (0.0011) [2023-10-11 20:34:07,367][71601] Updated weights for policy 0, policy_version 37410 (0.0010) [2023-10-11 20:34:07,746][71601] Updated weights for policy 0, policy_version 37420 (0.0010) [2023-10-11 20:34:08,107][71601] Updated weights for policy 0, policy_version 37430 (0.0009) [2023-10-11 20:34:08,477][71601] Updated weights for policy 0, policy_version 37440 (0.0007) [2023-10-11 20:34:10,221][71635] Updated weights for policy 1, policy_version 37412 (0.0008) [2023-10-11 20:34:10,586][71635] Updated weights for policy 1, policy_version 37422 (0.0009) [2023-10-11 20:34:10,958][71635] Updated weights for policy 1, policy_version 37432 (0.0009) [2023-10-11 20:34:11,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 76644352. Throughput: 0: 1809.5, 1: 1822.3. Samples: 19176446. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:34:11,034][70582] Avg episode reward: [(0, '166.230'), (1, '142.520')] [2023-10-11 20:34:12,159][71601] Updated weights for policy 0, policy_version 37450 (0.0007) [2023-10-11 20:34:12,521][71601] Updated weights for policy 0, policy_version 37460 (0.0009) [2023-10-11 20:34:12,895][71601] Updated weights for policy 0, policy_version 37470 (0.0010) [2023-10-11 20:34:14,611][71635] Updated weights for policy 1, policy_version 37442 (0.0008) [2023-10-11 20:34:14,978][71635] Updated weights for policy 1, policy_version 37452 (0.0010) [2023-10-11 20:34:15,357][71635] Updated weights for policy 1, policy_version 37462 (0.0008) [2023-10-11 20:34:15,721][71635] Updated weights for policy 1, policy_version 37472 (0.0008) [2023-10-11 20:34:16,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 76742656. Throughput: 0: 1815.4, 1: 1830.0. Samples: 19186970. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:34:16,035][70582] Avg episode reward: [(0, '178.680'), (1, '158.000')] [2023-10-11 20:34:16,573][71601] Updated weights for policy 0, policy_version 37480 (0.0009) [2023-10-11 20:34:16,935][71601] Updated weights for policy 0, policy_version 37490 (0.0011) [2023-10-11 20:34:17,312][71601] Updated weights for policy 0, policy_version 37500 (0.0008) [2023-10-11 20:34:19,412][71635] Updated weights for policy 1, policy_version 37482 (0.0010) [2023-10-11 20:34:19,770][71635] Updated weights for policy 1, policy_version 37492 (0.0010) [2023-10-11 20:34:20,137][71635] Updated weights for policy 1, policy_version 37502 (0.0009) [2023-10-11 20:34:20,964][71601] Updated weights for policy 0, policy_version 37510 (0.0008) [2023-10-11 20:34:21,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 76808192. Throughput: 0: 1818.9, 1: 1825.3. Samples: 19209404. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:34:21,035][70582] Avg episode reward: [(0, '177.050'), (1, '157.660')] [2023-10-11 20:34:21,340][71601] Updated weights for policy 0, policy_version 37520 (0.0007) [2023-10-11 20:34:21,719][71601] Updated weights for policy 0, policy_version 37530 (0.0009) [2023-10-11 20:34:23,849][71635] Updated weights for policy 1, policy_version 37512 (0.0009) [2023-10-11 20:34:24,232][71635] Updated weights for policy 1, policy_version 37522 (0.0007) [2023-10-11 20:34:24,606][71635] Updated weights for policy 1, policy_version 37532 (0.0009) [2023-10-11 20:34:25,460][71601] Updated weights for policy 0, policy_version 37540 (0.0010) [2023-10-11 20:34:25,829][71601] Updated weights for policy 0, policy_version 37550 (0.0007) [2023-10-11 20:34:26,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 76873728. Throughput: 0: 1819.6, 1: 1836.5. Samples: 19231086. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:34:26,035][70582] Avg episode reward: [(0, '177.050'), (1, '156.600')] [2023-10-11 20:34:26,201][71601] Updated weights for policy 0, policy_version 37560 (0.0008) [2023-10-11 20:34:28,257][71635] Updated weights for policy 1, policy_version 37542 (0.0009) [2023-10-11 20:34:28,620][71635] Updated weights for policy 1, policy_version 37552 (0.0009) [2023-10-11 20:34:28,990][71635] Updated weights for policy 1, policy_version 37562 (0.0009) [2023-10-11 20:34:29,974][71601] Updated weights for policy 0, policy_version 37570 (0.0009) [2023-10-11 20:34:30,345][71601] Updated weights for policy 0, policy_version 37580 (0.0008) [2023-10-11 20:34:30,705][71601] Updated weights for policy 0, policy_version 37590 (0.0007) [2023-10-11 20:34:31,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 76939264. Throughput: 0: 1826.6, 1: 1823.8. Samples: 19242274. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:34:31,034][70582] Avg episode reward: [(0, '182.060'), (1, '138.840')] [2023-10-11 20:34:31,078][71601] Updated weights for policy 0, policy_version 37600 (0.0007) [2023-10-11 20:34:32,510][71635] Updated weights for policy 1, policy_version 37572 (0.0009) [2023-10-11 20:34:32,879][71635] Updated weights for policy 1, policy_version 37582 (0.0007) [2023-10-11 20:34:33,238][71635] Updated weights for policy 1, policy_version 37592 (0.0009) [2023-10-11 20:34:34,695][71601] Updated weights for policy 0, policy_version 37610 (0.0010) [2023-10-11 20:34:35,066][71601] Updated weights for policy 0, policy_version 37620 (0.0010) [2023-10-11 20:34:35,443][71601] Updated weights for policy 0, policy_version 37630 (0.0008) [2023-10-11 20:34:36,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 77037568. Throughput: 0: 1826.9, 1: 1838.3. Samples: 19264040. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 20:34:36,035][70582] Avg episode reward: [(0, '182.140'), (1, '132.570')] [2023-10-11 20:34:36,974][71635] Updated weights for policy 1, policy_version 37602 (0.0011) [2023-10-11 20:34:37,342][71635] Updated weights for policy 1, policy_version 37612 (0.0007) [2023-10-11 20:34:37,708][71635] Updated weights for policy 1, policy_version 37622 (0.0010) [2023-10-11 20:34:38,079][71635] Updated weights for policy 1, policy_version 37632 (0.0008) [2023-10-11 20:34:39,222][71601] Updated weights for policy 0, policy_version 37640 (0.0008) [2023-10-11 20:34:39,583][71601] Updated weights for policy 0, policy_version 37650 (0.0007) [2023-10-11 20:34:39,955][71601] Updated weights for policy 0, policy_version 37660 (0.0010) [2023-10-11 20:34:41,034][70582] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 77103104. Throughput: 0: 1822.6, 1: 1843.7. Samples: 19285432. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 20:34:41,035][70582] Avg episode reward: [(0, '182.270'), (1, '129.770')] [2023-10-11 20:34:41,854][71635] Updated weights for policy 1, policy_version 37642 (0.0009) [2023-10-11 20:34:42,227][71635] Updated weights for policy 1, policy_version 37652 (0.0009) [2023-10-11 20:34:42,598][71635] Updated weights for policy 1, policy_version 37662 (0.0008) [2023-10-11 20:34:43,650][71601] Updated weights for policy 0, policy_version 37670 (0.0008) [2023-10-11 20:34:44,026][71601] Updated weights for policy 0, policy_version 37680 (0.0010) [2023-10-11 20:34:44,385][71601] Updated weights for policy 0, policy_version 37690 (0.0010) [2023-10-11 20:34:46,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 77168640. Throughput: 0: 1817.8, 1: 1844.5. Samples: 19296846. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 20:34:46,035][70582] Avg episode reward: [(0, '182.270'), (1, '129.860')] [2023-10-11 20:34:46,246][71635] Updated weights for policy 1, policy_version 37672 (0.0007) [2023-10-11 20:34:46,614][71635] Updated weights for policy 1, policy_version 37682 (0.0009) [2023-10-11 20:34:46,976][71635] Updated weights for policy 1, policy_version 37692 (0.0010) [2023-10-11 20:34:48,203][71601] Updated weights for policy 0, policy_version 37700 (0.0010) [2023-10-11 20:34:48,571][71601] Updated weights for policy 0, policy_version 37710 (0.0009) [2023-10-11 20:34:48,941][71601] Updated weights for policy 0, policy_version 37720 (0.0009) [2023-10-11 20:34:50,743][71635] Updated weights for policy 1, policy_version 37702 (0.0009) [2023-10-11 20:34:51,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 77234176. Throughput: 0: 1811.2, 1: 1844.2. Samples: 19318066. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 20:34:51,034][70582] Avg episode reward: [(0, '196.820'), (1, '144.540')] [2023-10-11 20:34:51,114][71635] Updated weights for policy 1, policy_version 37712 (0.0008) [2023-10-11 20:34:51,482][71635] Updated weights for policy 1, policy_version 37722 (0.0008) [2023-10-11 20:34:52,624][71601] Updated weights for policy 0, policy_version 37730 (0.0009) [2023-10-11 20:34:52,990][71601] Updated weights for policy 0, policy_version 37740 (0.0010) [2023-10-11 20:34:53,367][71601] Updated weights for policy 0, policy_version 37750 (0.0009) [2023-10-11 20:34:53,728][71601] Updated weights for policy 0, policy_version 37760 (0.0009) [2023-10-11 20:34:54,940][71635] Updated weights for policy 1, policy_version 37732 (0.0007) [2023-10-11 20:34:55,308][71635] Updated weights for policy 1, policy_version 37742 (0.0007) [2023-10-11 20:34:55,677][71635] Updated weights for policy 1, policy_version 37752 (0.0008) [2023-10-11 20:34:56,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 77332480. Throughput: 0: 1812.4, 1: 1835.4. Samples: 19340596. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 20:34:56,034][70582] Avg episode reward: [(0, '212.070'), (1, '148.080')] [2023-10-11 20:34:57,483][71601] Updated weights for policy 0, policy_version 37770 (0.0009) [2023-10-11 20:34:57,859][71601] Updated weights for policy 0, policy_version 37780 (0.0008) [2023-10-11 20:34:58,240][71601] Updated weights for policy 0, policy_version 37790 (0.0007) [2023-10-11 20:34:59,413][71635] Updated weights for policy 1, policy_version 37762 (0.0010) [2023-10-11 20:34:59,787][71635] Updated weights for policy 1, policy_version 37772 (0.0008) [2023-10-11 20:35:00,161][71635] Updated weights for policy 1, policy_version 37782 (0.0008) [2023-10-11 20:35:00,523][71635] Updated weights for policy 1, policy_version 37792 (0.0007) [2023-10-11 20:35:01,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 77398016. Throughput: 0: 1808.9, 1: 1841.0. Samples: 19351216. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 20:35:01,035][70582] Avg episode reward: [(0, '206.380'), (1, '170.620')] [2023-10-11 20:35:01,845][71601] Updated weights for policy 0, policy_version 37800 (0.0007) [2023-10-11 20:35:02,213][71601] Updated weights for policy 0, policy_version 37810 (0.0007) [2023-10-11 20:35:02,578][71601] Updated weights for policy 0, policy_version 37820 (0.0007) [2023-10-11 20:35:04,038][71635] Updated weights for policy 1, policy_version 37802 (0.0010) [2023-10-11 20:35:04,399][71635] Updated weights for policy 1, policy_version 37812 (0.0010) [2023-10-11 20:35:04,768][71635] Updated weights for policy 1, policy_version 37822 (0.0009) [2023-10-11 20:35:06,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 77463552. Throughput: 0: 1808.8, 1: 1830.4. Samples: 19373166. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-11 20:35:06,034][70582] Avg episode reward: [(0, '206.820'), (1, '160.030')] [2023-10-11 20:35:06,095][71601] Updated weights for policy 0, policy_version 37830 (0.0007) [2023-10-11 20:35:06,464][71601] Updated weights for policy 0, policy_version 37840 (0.0007) [2023-10-11 20:35:06,841][71601] Updated weights for policy 0, policy_version 37850 (0.0008) [2023-10-11 20:35:08,361][71635] Updated weights for policy 1, policy_version 37832 (0.0009) [2023-10-11 20:35:08,736][71635] Updated weights for policy 1, policy_version 37842 (0.0008) [2023-10-11 20:35:09,112][71635] Updated weights for policy 1, policy_version 37852 (0.0008) [2023-10-11 20:35:10,708][71601] Updated weights for policy 0, policy_version 37860 (0.0007) [2023-10-11 20:35:11,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 77529088. Throughput: 0: 1817.5, 1: 1841.1. Samples: 19395722. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-11 20:35:11,034][70582] Avg episode reward: [(0, '228.310'), (1, '146.100')] [2023-10-11 20:35:11,073][71601] Updated weights for policy 0, policy_version 37870 (0.0008) [2023-10-11 20:35:11,454][71601] Updated weights for policy 0, policy_version 37880 (0.0008) [2023-10-11 20:35:12,771][71635] Updated weights for policy 1, policy_version 37862 (0.0010) [2023-10-11 20:35:13,132][71635] Updated weights for policy 1, policy_version 37872 (0.0011) [2023-10-11 20:35:13,506][71635] Updated weights for policy 1, policy_version 37882 (0.0010) [2023-10-11 20:35:14,872][71601] Updated weights for policy 0, policy_version 37890 (0.0008) [2023-10-11 20:35:15,244][71601] Updated weights for policy 0, policy_version 37900 (0.0008) [2023-10-11 20:35:15,617][71601] Updated weights for policy 0, policy_version 37910 (0.0009) [2023-10-11 20:35:15,986][71601] Updated weights for policy 0, policy_version 37920 (0.0009) [2023-10-11 20:35:16,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 77627392. Throughput: 0: 1818.8, 1: 1826.3. Samples: 19406306. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-11 20:35:16,034][70582] Avg episode reward: [(0, '228.270'), (1, '143.540')] [2023-10-11 20:35:17,136][71635] Updated weights for policy 1, policy_version 37892 (0.0010) [2023-10-11 20:35:17,497][71635] Updated weights for policy 1, policy_version 37902 (0.0009) [2023-10-11 20:35:17,867][71635] Updated weights for policy 1, policy_version 37912 (0.0007) [2023-10-11 20:35:19,716][71601] Updated weights for policy 0, policy_version 37930 (0.0007) [2023-10-11 20:35:20,080][71601] Updated weights for policy 0, policy_version 37940 (0.0009) [2023-10-11 20:35:20,446][71601] Updated weights for policy 0, policy_version 37950 (0.0008) [2023-10-11 20:35:21,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 77692928. Throughput: 0: 1820.9, 1: 1837.4. Samples: 19428666. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-11 20:35:21,034][70582] Avg episode reward: [(0, '227.410'), (1, '137.970')] [2023-10-11 20:35:21,735][71635] Updated weights for policy 1, policy_version 37922 (0.0009) [2023-10-11 20:35:22,097][71635] Updated weights for policy 1, policy_version 37932 (0.0007) [2023-10-11 20:35:22,459][71635] Updated weights for policy 1, policy_version 37942 (0.0007) [2023-10-11 20:35:22,825][71635] Updated weights for policy 1, policy_version 37952 (0.0007) [2023-10-11 20:35:24,114][71601] Updated weights for policy 0, policy_version 37960 (0.0009) [2023-10-11 20:35:24,495][71601] Updated weights for policy 0, policy_version 37970 (0.0011) [2023-10-11 20:35:24,872][71601] Updated weights for policy 0, policy_version 37980 (0.0008) [2023-10-11 20:35:26,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 77758464. Throughput: 0: 1830.6, 1: 1832.1. Samples: 19450252. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-11 20:35:26,034][70582] Avg episode reward: [(0, '238.340'), (1, '138.560')] [2023-10-11 20:35:26,395][71635] Updated weights for policy 1, policy_version 37962 (0.0007) [2023-10-11 20:35:26,763][71635] Updated weights for policy 1, policy_version 37972 (0.0007) [2023-10-11 20:35:27,138][71635] Updated weights for policy 1, policy_version 37982 (0.0007) [2023-10-11 20:35:28,592][71601] Updated weights for policy 0, policy_version 37990 (0.0007) [2023-10-11 20:35:28,962][71601] Updated weights for policy 0, policy_version 38000 (0.0009) [2023-10-11 20:35:29,333][71601] Updated weights for policy 0, policy_version 38010 (0.0009) [2023-10-11 20:35:31,031][71635] Updated weights for policy 1, policy_version 37992 (0.0011) [2023-10-11 20:35:31,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 77824000. Throughput: 0: 1829.5, 1: 1833.7. Samples: 19461690. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:35:31,035][70582] Avg episode reward: [(0, '267.950'), (1, '132.350')] [2023-10-11 20:35:31,407][71635] Updated weights for policy 1, policy_version 38002 (0.0008) [2023-10-11 20:35:31,768][71635] Updated weights for policy 1, policy_version 38012 (0.0008) [2023-10-11 20:35:33,028][71601] Updated weights for policy 0, policy_version 38020 (0.0009) [2023-10-11 20:35:33,399][71601] Updated weights for policy 0, policy_version 38030 (0.0009) [2023-10-11 20:35:33,762][71601] Updated weights for policy 0, policy_version 38040 (0.0009) [2023-10-11 20:35:35,528][71635] Updated weights for policy 1, policy_version 38022 (0.0009) [2023-10-11 20:35:35,901][71635] Updated weights for policy 1, policy_version 38032 (0.0008) [2023-10-11 20:35:36,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 77889536. Throughput: 0: 1835.1, 1: 1827.0. Samples: 19482858. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:35:36,034][70582] Avg episode reward: [(0, '252.160'), (1, '134.090')] [2023-10-11 20:35:36,256][71635] Updated weights for policy 1, policy_version 38042 (0.0009) [2023-10-11 20:35:37,500][71601] Updated weights for policy 0, policy_version 38050 (0.0008) [2023-10-11 20:35:37,873][71601] Updated weights for policy 0, policy_version 38060 (0.0007) [2023-10-11 20:35:38,237][71601] Updated weights for policy 0, policy_version 38070 (0.0010) [2023-10-11 20:35:38,609][71601] Updated weights for policy 0, policy_version 38080 (0.0010) [2023-10-11 20:35:39,886][71635] Updated weights for policy 1, policy_version 38052 (0.0009) [2023-10-11 20:35:40,249][71635] Updated weights for policy 1, policy_version 38062 (0.0008) [2023-10-11 20:35:40,623][71635] Updated weights for policy 1, policy_version 38072 (0.0008) [2023-10-11 20:35:41,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 77987840. Throughput: 0: 1826.4, 1: 1819.7. Samples: 19504674. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:35:41,035][70582] Avg episode reward: [(0, '270.890'), (1, '117.410')] [2023-10-11 20:35:42,576][71601] Updated weights for policy 0, policy_version 38090 (0.0009) [2023-10-11 20:35:42,945][71601] Updated weights for policy 0, policy_version 38100 (0.0008) [2023-10-11 20:35:43,315][71601] Updated weights for policy 0, policy_version 38110 (0.0008) [2023-10-11 20:35:44,216][71635] Updated weights for policy 1, policy_version 38082 (0.0010) [2023-10-11 20:35:44,588][71635] Updated weights for policy 1, policy_version 38092 (0.0007) [2023-10-11 20:35:44,951][71635] Updated weights for policy 1, policy_version 38102 (0.0008) [2023-10-11 20:35:45,313][71635] Updated weights for policy 1, policy_version 38112 (0.0007) [2023-10-11 20:35:46,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 78053376. Throughput: 0: 1820.8, 1: 1827.2. Samples: 19515376. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:35:46,035][70582] Avg episode reward: [(0, '251.590'), (1, '113.830')] [2023-10-11 20:35:46,965][71601] Updated weights for policy 0, policy_version 38120 (0.0009) [2023-10-11 20:35:47,345][71601] Updated weights for policy 0, policy_version 38130 (0.0010) [2023-10-11 20:35:47,709][71601] Updated weights for policy 0, policy_version 38140 (0.0009) [2023-10-11 20:35:48,945][71635] Updated weights for policy 1, policy_version 38122 (0.0009) [2023-10-11 20:35:49,307][71635] Updated weights for policy 1, policy_version 38132 (0.0008) [2023-10-11 20:35:49,675][71635] Updated weights for policy 1, policy_version 38142 (0.0010) [2023-10-11 20:35:51,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 78118912. Throughput: 0: 1823.0, 1: 1827.0. Samples: 19537414. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:35:51,034][70582] Avg episode reward: [(0, '256.300'), (1, '114.150')] [2023-10-11 20:35:51,400][71601] Updated weights for policy 0, policy_version 38150 (0.0009) [2023-10-11 20:35:51,771][71601] Updated weights for policy 0, policy_version 38160 (0.0008) [2023-10-11 20:35:52,143][71601] Updated weights for policy 0, policy_version 38170 (0.0007) [2023-10-11 20:35:53,381][71635] Updated weights for policy 1, policy_version 38152 (0.0011) [2023-10-11 20:35:53,753][71635] Updated weights for policy 1, policy_version 38162 (0.0009) [2023-10-11 20:35:54,127][71635] Updated weights for policy 1, policy_version 38172 (0.0008) [2023-10-11 20:35:55,733][71601] Updated weights for policy 0, policy_version 38180 (0.0007) [2023-10-11 20:35:56,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 78184448. Throughput: 0: 1823.8, 1: 1825.6. Samples: 19559942. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:35:56,034][70582] Avg episode reward: [(0, '254.960'), (1, '114.020')] [2023-10-11 20:35:56,042][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000038176_39092224.pth... [2023-10-11 20:35:56,076][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000036480_37355520.pth [2023-10-11 20:35:56,107][71601] Updated weights for policy 0, policy_version 38190 (0.0008) [2023-10-11 20:35:56,475][71601] Updated weights for policy 0, policy_version 38200 (0.0008) [2023-10-11 20:35:56,770][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000038208_39124992.pth... [2023-10-11 20:35:56,799][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000036480_37355520.pth [2023-10-11 20:35:57,793][71635] Updated weights for policy 1, policy_version 38182 (0.0008) [2023-10-11 20:35:58,175][71635] Updated weights for policy 1, policy_version 38192 (0.0008) [2023-10-11 20:35:58,544][71635] Updated weights for policy 1, policy_version 38202 (0.0009) [2023-10-11 20:36:00,091][71601] Updated weights for policy 0, policy_version 38210 (0.0009) [2023-10-11 20:36:00,467][71601] Updated weights for policy 0, policy_version 38220 (0.0008) [2023-10-11 20:36:00,846][71601] Updated weights for policy 0, policy_version 38230 (0.0008) [2023-10-11 20:36:01,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 78249984. Throughput: 0: 1819.3, 1: 1824.6. Samples: 19570284. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-11 20:36:01,035][70582] Avg episode reward: [(0, '264.110'), (1, '115.470')] [2023-10-11 20:36:01,209][71601] Updated weights for policy 0, policy_version 38240 (0.0009) [2023-10-11 20:36:02,307][71635] Updated weights for policy 1, policy_version 38212 (0.0008) [2023-10-11 20:36:02,675][71635] Updated weights for policy 1, policy_version 38222 (0.0008) [2023-10-11 20:36:03,045][71635] Updated weights for policy 1, policy_version 38232 (0.0007) [2023-10-11 20:36:04,818][71601] Updated weights for policy 0, policy_version 38250 (0.0008) [2023-10-11 20:36:05,183][71601] Updated weights for policy 0, policy_version 38260 (0.0007) [2023-10-11 20:36:05,564][71601] Updated weights for policy 0, policy_version 38270 (0.0009) [2023-10-11 20:36:06,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 78348288. Throughput: 0: 1824.4, 1: 1818.9. Samples: 19592614. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-11 20:36:06,034][70582] Avg episode reward: [(0, '264.620'), (1, '115.390')] [2023-10-11 20:36:06,560][71635] Updated weights for policy 1, policy_version 38242 (0.0008) [2023-10-11 20:36:06,929][71635] Updated weights for policy 1, policy_version 38252 (0.0012) [2023-10-11 20:36:07,297][71635] Updated weights for policy 1, policy_version 38262 (0.0010) [2023-10-11 20:36:07,653][71635] Updated weights for policy 1, policy_version 38272 (0.0010) [2023-10-11 20:36:09,274][71601] Updated weights for policy 0, policy_version 38280 (0.0008) [2023-10-11 20:36:09,642][71601] Updated weights for policy 0, policy_version 38290 (0.0008) [2023-10-11 20:36:10,016][71601] Updated weights for policy 0, policy_version 38300 (0.0008) [2023-10-11 20:36:11,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 78413824. Throughput: 0: 1818.4, 1: 1828.3. Samples: 19614352. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-11 20:36:11,034][70582] Avg episode reward: [(0, '252.190'), (1, '110.350')] [2023-10-11 20:36:11,183][71635] Updated weights for policy 1, policy_version 38282 (0.0008) [2023-10-11 20:36:11,536][71635] Updated weights for policy 1, policy_version 38292 (0.0009) [2023-10-11 20:36:11,905][71635] Updated weights for policy 1, policy_version 38302 (0.0008) [2023-10-11 20:36:13,746][71601] Updated weights for policy 0, policy_version 38310 (0.0009) [2023-10-11 20:36:14,125][71601] Updated weights for policy 0, policy_version 38320 (0.0008) [2023-10-11 20:36:14,496][71601] Updated weights for policy 0, policy_version 38330 (0.0008) [2023-10-11 20:36:15,504][71635] Updated weights for policy 1, policy_version 38312 (0.0008) [2023-10-11 20:36:15,877][71635] Updated weights for policy 1, policy_version 38322 (0.0010) [2023-10-11 20:36:16,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 78479360. Throughput: 0: 1821.2, 1: 1826.6. Samples: 19625840. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-11 20:36:16,034][70582] Avg episode reward: [(0, '255.650'), (1, '117.220')] [2023-10-11 20:36:16,238][71635] Updated weights for policy 1, policy_version 38332 (0.0007) [2023-10-11 20:36:18,157][71601] Updated weights for policy 0, policy_version 38340 (0.0008) [2023-10-11 20:36:18,531][71601] Updated weights for policy 0, policy_version 38350 (0.0008) [2023-10-11 20:36:18,900][71601] Updated weights for policy 0, policy_version 38360 (0.0009) [2023-10-11 20:36:20,030][71635] Updated weights for policy 1, policy_version 38342 (0.0009) [2023-10-11 20:36:20,394][71635] Updated weights for policy 1, policy_version 38352 (0.0008) [2023-10-11 20:36:20,760][71635] Updated weights for policy 1, policy_version 38362 (0.0008) [2023-10-11 20:36:21,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 78577664. Throughput: 0: 1818.4, 1: 1832.3. Samples: 19647140. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-11 20:36:21,035][70582] Avg episode reward: [(0, '255.620'), (1, '101.890')] [2023-10-11 20:36:22,605][71601] Updated weights for policy 0, policy_version 38370 (0.0008) [2023-10-11 20:36:22,981][71601] Updated weights for policy 0, policy_version 38380 (0.0009) [2023-10-11 20:36:23,364][71601] Updated weights for policy 0, policy_version 38390 (0.0010) [2023-10-11 20:36:23,733][71601] Updated weights for policy 0, policy_version 38400 (0.0009) [2023-10-11 20:36:24,626][71635] Updated weights for policy 1, policy_version 38372 (0.0008) [2023-10-11 20:36:24,986][71635] Updated weights for policy 1, policy_version 38382 (0.0007) [2023-10-11 20:36:25,357][71635] Updated weights for policy 1, policy_version 38392 (0.0009) [2023-10-11 20:36:26,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 78643200. Throughput: 0: 1825.7, 1: 1824.5. Samples: 19668928. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-11 20:36:26,034][70582] Avg episode reward: [(0, '258.500'), (1, '106.050')] [2023-10-11 20:36:27,439][71601] Updated weights for policy 0, policy_version 38410 (0.0010) [2023-10-11 20:36:27,829][71601] Updated weights for policy 0, policy_version 38420 (0.0010) [2023-10-11 20:36:28,189][71601] Updated weights for policy 0, policy_version 38430 (0.0008) [2023-10-11 20:36:28,859][71635] Updated weights for policy 1, policy_version 38402 (0.0009) [2023-10-11 20:36:29,230][71635] Updated weights for policy 1, policy_version 38412 (0.0008) [2023-10-11 20:36:29,594][71635] Updated weights for policy 1, policy_version 38422 (0.0008) [2023-10-11 20:36:29,958][71635] Updated weights for policy 1, policy_version 38432 (0.0007) [2023-10-11 20:36:31,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 78708736. Throughput: 0: 1828.4, 1: 1825.4. Samples: 19679794. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-11 20:36:31,035][70582] Avg episode reward: [(0, '256.600'), (1, '82.400')] [2023-10-11 20:36:31,778][71601] Updated weights for policy 0, policy_version 38440 (0.0009) [2023-10-11 20:36:32,152][71601] Updated weights for policy 0, policy_version 38450 (0.0011) [2023-10-11 20:36:32,521][71601] Updated weights for policy 0, policy_version 38460 (0.0010) [2023-10-11 20:36:33,550][71635] Updated weights for policy 1, policy_version 38442 (0.0008) [2023-10-11 20:36:33,915][71635] Updated weights for policy 1, policy_version 38452 (0.0008) [2023-10-11 20:36:34,284][71635] Updated weights for policy 1, policy_version 38462 (0.0009) [2023-10-11 20:36:36,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 78774272. Throughput: 0: 1828.9, 1: 1820.7. Samples: 19701646. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-11 20:36:36,035][70582] Avg episode reward: [(0, '257.490'), (1, '79.870')] [2023-10-11 20:36:36,078][71601] Updated weights for policy 0, policy_version 38470 (0.0008) [2023-10-11 20:36:36,456][71601] Updated weights for policy 0, policy_version 38480 (0.0008) [2023-10-11 20:36:36,835][71601] Updated weights for policy 0, policy_version 38490 (0.0008) [2023-10-11 20:36:37,973][71635] Updated weights for policy 1, policy_version 38472 (0.0007) [2023-10-11 20:36:38,346][71635] Updated weights for policy 1, policy_version 38482 (0.0007) [2023-10-11 20:36:38,715][71635] Updated weights for policy 1, policy_version 38492 (0.0007) [2023-10-11 20:36:40,512][71601] Updated weights for policy 0, policy_version 38500 (0.0009) [2023-10-11 20:36:40,885][71601] Updated weights for policy 0, policy_version 38510 (0.0010) [2023-10-11 20:36:41,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 78839808. Throughput: 0: 1821.9, 1: 1834.9. Samples: 19724496. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-11 20:36:41,034][70582] Avg episode reward: [(0, '243.930'), (1, '87.110')] [2023-10-11 20:36:41,261][71601] Updated weights for policy 0, policy_version 38520 (0.0008) [2023-10-11 20:36:42,311][71635] Updated weights for policy 1, policy_version 38502 (0.0010) [2023-10-11 20:36:42,674][71635] Updated weights for policy 1, policy_version 38512 (0.0009) [2023-10-11 20:36:43,041][71635] Updated weights for policy 1, policy_version 38522 (0.0007) [2023-10-11 20:36:44,876][71601] Updated weights for policy 0, policy_version 38530 (0.0008) [2023-10-11 20:36:45,249][71601] Updated weights for policy 0, policy_version 38540 (0.0007) [2023-10-11 20:36:45,619][71601] Updated weights for policy 0, policy_version 38550 (0.0007) [2023-10-11 20:36:45,986][71601] Updated weights for policy 0, policy_version 38560 (0.0007) [2023-10-11 20:36:46,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 78938112. Throughput: 0: 1823.5, 1: 1825.3. Samples: 19734480. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-11 20:36:46,035][70582] Avg episode reward: [(0, '233.000'), (1, '86.250')] [2023-10-11 20:36:46,737][71635] Updated weights for policy 1, policy_version 38532 (0.0011) [2023-10-11 20:36:47,133][71635] Updated weights for policy 1, policy_version 38542 (0.0008) [2023-10-11 20:36:47,501][71635] Updated weights for policy 1, policy_version 38552 (0.0010) [2023-10-11 20:36:49,694][71601] Updated weights for policy 0, policy_version 38570 (0.0010) [2023-10-11 20:36:50,073][71601] Updated weights for policy 0, policy_version 38580 (0.0008) [2023-10-11 20:36:50,449][71601] Updated weights for policy 0, policy_version 38590 (0.0011) [2023-10-11 20:36:51,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 79003648. Throughput: 0: 1819.7, 1: 1839.2. Samples: 19757262. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-11 20:36:51,034][70582] Avg episode reward: [(0, '222.100'), (1, '103.460')] [2023-10-11 20:36:51,278][71635] Updated weights for policy 1, policy_version 38562 (0.0009) [2023-10-11 20:36:51,643][71635] Updated weights for policy 1, policy_version 38572 (0.0010) [2023-10-11 20:36:52,008][71635] Updated weights for policy 1, policy_version 38582 (0.0009) [2023-10-11 20:36:52,379][71635] Updated weights for policy 1, policy_version 38592 (0.0008) [2023-10-11 20:36:54,094][71601] Updated weights for policy 0, policy_version 38600 (0.0010) [2023-10-11 20:36:54,463][71601] Updated weights for policy 0, policy_version 38610 (0.0008) [2023-10-11 20:36:54,831][71601] Updated weights for policy 0, policy_version 38620 (0.0008) [2023-10-11 20:36:56,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 79069184. Throughput: 0: 1820.9, 1: 1836.4. Samples: 19778932. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-11 20:36:56,034][70582] Avg episode reward: [(0, '221.080'), (1, '103.890')] [2023-10-11 20:36:56,052][71635] Updated weights for policy 1, policy_version 38602 (0.0007) [2023-10-11 20:36:56,421][71635] Updated weights for policy 1, policy_version 38612 (0.0009) [2023-10-11 20:36:56,782][71635] Updated weights for policy 1, policy_version 38622 (0.0009) [2023-10-11 20:36:58,698][71601] Updated weights for policy 0, policy_version 38630 (0.0007) [2023-10-11 20:36:59,063][71601] Updated weights for policy 0, policy_version 38640 (0.0007) [2023-10-11 20:36:59,438][71601] Updated weights for policy 0, policy_version 38650 (0.0007) [2023-10-11 20:37:00,445][71635] Updated weights for policy 1, policy_version 38632 (0.0007) [2023-10-11 20:37:00,808][71635] Updated weights for policy 1, policy_version 38642 (0.0007) [2023-10-11 20:37:01,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 79134720. Throughput: 0: 1814.2, 1: 1836.5. Samples: 19790122. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-11 20:37:01,034][70582] Avg episode reward: [(0, '216.180'), (1, '99.670')] [2023-10-11 20:37:01,180][71635] Updated weights for policy 1, policy_version 38652 (0.0007) [2023-10-11 20:37:03,250][71601] Updated weights for policy 0, policy_version 38660 (0.0007) [2023-10-11 20:37:03,631][71601] Updated weights for policy 0, policy_version 38670 (0.0007) [2023-10-11 20:37:03,996][71601] Updated weights for policy 0, policy_version 38680 (0.0011) [2023-10-11 20:37:04,810][71635] Updated weights for policy 1, policy_version 38662 (0.0008) [2023-10-11 20:37:05,183][71635] Updated weights for policy 1, policy_version 38672 (0.0008) [2023-10-11 20:37:05,546][71635] Updated weights for policy 1, policy_version 38682 (0.0009) [2023-10-11 20:37:06,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 79233024. Throughput: 0: 1816.7, 1: 1836.4. Samples: 19811530. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-11 20:37:06,035][70582] Avg episode reward: [(0, '218.180'), (1, '105.950')] [2023-10-11 20:37:07,574][71601] Updated weights for policy 0, policy_version 38690 (0.0010) [2023-10-11 20:37:07,940][71601] Updated weights for policy 0, policy_version 38700 (0.0008) [2023-10-11 20:37:08,305][71601] Updated weights for policy 0, policy_version 38710 (0.0008) [2023-10-11 20:37:08,679][71601] Updated weights for policy 0, policy_version 38720 (0.0008) [2023-10-11 20:37:09,237][71635] Updated weights for policy 1, policy_version 38692 (0.0010) [2023-10-11 20:37:09,612][71635] Updated weights for policy 1, policy_version 38702 (0.0010) [2023-10-11 20:37:09,988][71635] Updated weights for policy 1, policy_version 38712 (0.0008) [2023-10-11 20:37:11,034][70582] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 79298560. Throughput: 0: 1819.4, 1: 1830.4. Samples: 19833170. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-11 20:37:11,035][70582] Avg episode reward: [(0, '211.090'), (1, '105.950')] [2023-10-11 20:37:12,500][71601] Updated weights for policy 0, policy_version 38730 (0.0007) [2023-10-11 20:37:12,874][71601] Updated weights for policy 0, policy_version 38740 (0.0008) [2023-10-11 20:37:13,239][71601] Updated weights for policy 0, policy_version 38750 (0.0007) [2023-10-11 20:37:13,598][71635] Updated weights for policy 1, policy_version 38722 (0.0009) [2023-10-11 20:37:13,959][71635] Updated weights for policy 1, policy_version 38732 (0.0009) [2023-10-11 20:37:14,330][71635] Updated weights for policy 1, policy_version 38742 (0.0008) [2023-10-11 20:37:14,699][71635] Updated weights for policy 1, policy_version 38752 (0.0008) [2023-10-11 20:37:16,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 79364096. Throughput: 0: 1819.0, 1: 1847.4. Samples: 19844782. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-11 20:37:16,034][70582] Avg episode reward: [(0, '211.490'), (1, '108.610')] [2023-10-11 20:37:16,829][71601] Updated weights for policy 0, policy_version 38760 (0.0009) [2023-10-11 20:37:17,190][71601] Updated weights for policy 0, policy_version 38770 (0.0011) [2023-10-11 20:37:17,560][71601] Updated weights for policy 0, policy_version 38780 (0.0011) [2023-10-11 20:37:18,238][71635] Updated weights for policy 1, policy_version 38762 (0.0007) [2023-10-11 20:37:18,608][71635] Updated weights for policy 1, policy_version 38772 (0.0008) [2023-10-11 20:37:18,974][71635] Updated weights for policy 1, policy_version 38782 (0.0009) [2023-10-11 20:37:21,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 79429632. Throughput: 0: 1817.9, 1: 1840.5. Samples: 19866272. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:37:21,034][70582] Avg episode reward: [(0, '219.440'), (1, '110.960')] [2023-10-11 20:37:21,235][71601] Updated weights for policy 0, policy_version 38790 (0.0010) [2023-10-11 20:37:21,611][71601] Updated weights for policy 0, policy_version 38800 (0.0007) [2023-10-11 20:37:21,977][71601] Updated weights for policy 0, policy_version 38810 (0.0009) [2023-10-11 20:37:22,504][71635] Updated weights for policy 1, policy_version 38792 (0.0009) [2023-10-11 20:37:22,865][71635] Updated weights for policy 1, policy_version 38802 (0.0011) [2023-10-11 20:37:23,237][71635] Updated weights for policy 1, policy_version 38812 (0.0010) [2023-10-11 20:37:25,650][71601] Updated weights for policy 0, policy_version 38820 (0.0009) [2023-10-11 20:37:26,025][71601] Updated weights for policy 0, policy_version 38830 (0.0008) [2023-10-11 20:37:26,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 79495168. Throughput: 0: 1820.3, 1: 1839.6. Samples: 19889188. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:37:26,034][70582] Avg episode reward: [(0, '190.890'), (1, '120.080')] [2023-10-11 20:37:26,389][71601] Updated weights for policy 0, policy_version 38840 (0.0008) [2023-10-11 20:37:27,008][71635] Updated weights for policy 1, policy_version 38822 (0.0009) [2023-10-11 20:37:27,366][71635] Updated weights for policy 1, policy_version 38832 (0.0008) [2023-10-11 20:37:27,735][71635] Updated weights for policy 1, policy_version 38842 (0.0007) [2023-10-11 20:37:30,041][71601] Updated weights for policy 0, policy_version 38850 (0.0008) [2023-10-11 20:37:30,415][71601] Updated weights for policy 0, policy_version 38860 (0.0009) [2023-10-11 20:37:30,784][71601] Updated weights for policy 0, policy_version 38870 (0.0011) [2023-10-11 20:37:31,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 79560704. Throughput: 0: 1819.8, 1: 1840.4. Samples: 19899188. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:37:31,034][70582] Avg episode reward: [(0, '192.450'), (1, '120.210')] [2023-10-11 20:37:31,154][71601] Updated weights for policy 0, policy_version 38880 (0.0008) [2023-10-11 20:37:31,382][71635] Updated weights for policy 1, policy_version 38852 (0.0008) [2023-10-11 20:37:31,742][71635] Updated weights for policy 1, policy_version 38862 (0.0007) [2023-10-11 20:37:32,110][71635] Updated weights for policy 1, policy_version 38872 (0.0008) [2023-10-11 20:37:34,774][71601] Updated weights for policy 0, policy_version 38890 (0.0011) [2023-10-11 20:37:35,155][71601] Updated weights for policy 0, policy_version 38900 (0.0010) [2023-10-11 20:37:35,518][71601] Updated weights for policy 0, policy_version 38910 (0.0009) [2023-10-11 20:37:35,634][71635] Updated weights for policy 1, policy_version 38882 (0.0008) [2023-10-11 20:37:36,029][71635] Updated weights for policy 1, policy_version 38892 (0.0009) [2023-10-11 20:37:36,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 79659008. Throughput: 0: 1823.2, 1: 1847.4. Samples: 19922436. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:37:36,034][70582] Avg episode reward: [(0, '167.380'), (1, '125.580')] [2023-10-11 20:37:36,406][71635] Updated weights for policy 1, policy_version 38902 (0.0007) [2023-10-11 20:37:36,770][71635] Updated weights for policy 1, policy_version 38912 (0.0008) [2023-10-11 20:37:39,066][71601] Updated weights for policy 0, policy_version 38920 (0.0010) [2023-10-11 20:37:39,435][71601] Updated weights for policy 0, policy_version 38930 (0.0010) [2023-10-11 20:37:39,809][71601] Updated weights for policy 0, policy_version 38940 (0.0010) [2023-10-11 20:37:40,544][71635] Updated weights for policy 1, policy_version 38922 (0.0009) [2023-10-11 20:37:40,905][71635] Updated weights for policy 1, policy_version 38932 (0.0007) [2023-10-11 20:37:41,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 79724544. Throughput: 0: 1822.8, 1: 1830.3. Samples: 19943322. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:37:41,035][70582] Avg episode reward: [(0, '153.490'), (1, '125.250')] [2023-10-11 20:37:41,277][71635] Updated weights for policy 1, policy_version 38942 (0.0009) [2023-10-11 20:37:43,587][71601] Updated weights for policy 0, policy_version 38950 (0.0011) [2023-10-11 20:37:43,958][71601] Updated weights for policy 0, policy_version 38960 (0.0009) [2023-10-11 20:37:44,330][71601] Updated weights for policy 0, policy_version 38970 (0.0008) [2023-10-11 20:37:45,027][71635] Updated weights for policy 1, policy_version 38952 (0.0009) [2023-10-11 20:37:45,391][71635] Updated weights for policy 1, policy_version 38962 (0.0010) [2023-10-11 20:37:45,766][71635] Updated weights for policy 1, policy_version 38972 (0.0011) [2023-10-11 20:37:46,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 79822848. Throughput: 0: 1828.1, 1: 1837.3. Samples: 19955068. Policy #0 lag: (min: 16.0, avg: 44.1, max: 48.0) [2023-10-11 20:37:46,034][70582] Avg episode reward: [(0, '152.300'), (1, '126.940')] [2023-10-11 20:37:47,861][71601] Updated weights for policy 0, policy_version 38980 (0.0009) [2023-10-11 20:37:48,237][71601] Updated weights for policy 0, policy_version 38990 (0.0010) [2023-10-11 20:37:48,609][71601] Updated weights for policy 0, policy_version 39000 (0.0007) [2023-10-11 20:37:49,662][71635] Updated weights for policy 1, policy_version 38982 (0.0009) [2023-10-11 20:37:50,034][71635] Updated weights for policy 1, policy_version 38992 (0.0009) [2023-10-11 20:37:50,394][71635] Updated weights for policy 1, policy_version 39002 (0.0009) [2023-10-11 20:37:51,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 79888384. Throughput: 0: 1833.7, 1: 1829.1. Samples: 19976356. Policy #0 lag: (min: 16.0, avg: 44.1, max: 48.0) [2023-10-11 20:37:51,034][70582] Avg episode reward: [(0, '132.510'), (1, '128.140')] [2023-10-11 20:37:52,323][71601] Updated weights for policy 0, policy_version 39010 (0.0010) [2023-10-11 20:37:52,686][71601] Updated weights for policy 0, policy_version 39020 (0.0007) [2023-10-11 20:37:53,058][71601] Updated weights for policy 0, policy_version 39030 (0.0007) [2023-10-11 20:37:53,420][71601] Updated weights for policy 0, policy_version 39040 (0.0009) [2023-10-11 20:37:54,015][71635] Updated weights for policy 1, policy_version 39012 (0.0009) [2023-10-11 20:37:54,383][71635] Updated weights for policy 1, policy_version 39022 (0.0010) [2023-10-11 20:37:54,745][71635] Updated weights for policy 1, policy_version 39032 (0.0009) [2023-10-11 20:37:56,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 79953920. Throughput: 0: 1835.1, 1: 1825.2. Samples: 19997880. Policy #0 lag: (min: 16.0, avg: 44.1, max: 48.0) [2023-10-11 20:37:56,034][70582] Avg episode reward: [(0, '136.120'), (1, '128.140')] [2023-10-11 20:37:56,045][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000039040_39976960.pth... [2023-10-11 20:37:56,045][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000039040_39976960.pth... [2023-10-11 20:37:56,082][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000037312_38207488.pth [2023-10-11 20:37:56,084][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000037344_38240256.pth [2023-10-11 20:37:57,285][71601] Updated weights for policy 0, policy_version 39050 (0.0010) [2023-10-11 20:37:57,654][71601] Updated weights for policy 0, policy_version 39060 (0.0009) [2023-10-11 20:37:58,036][71601] Updated weights for policy 0, policy_version 39070 (0.0011) [2023-10-11 20:37:58,307][71635] Updated weights for policy 1, policy_version 39042 (0.0009) [2023-10-11 20:37:58,673][71635] Updated weights for policy 1, policy_version 39052 (0.0009) [2023-10-11 20:37:59,044][71635] Updated weights for policy 1, policy_version 39062 (0.0008) [2023-10-11 20:37:59,411][71635] Updated weights for policy 1, policy_version 39072 (0.0010) [2023-10-11 20:38:01,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 80019456. Throughput: 0: 1829.0, 1: 1818.4. Samples: 20008918. Policy #0 lag: (min: 16.0, avg: 44.1, max: 48.0) [2023-10-11 20:38:01,034][70582] Avg episode reward: [(0, '136.270'), (1, '130.210')] [2023-10-11 20:38:01,696][71601] Updated weights for policy 0, policy_version 39080 (0.0007) [2023-10-11 20:38:02,069][71601] Updated weights for policy 0, policy_version 39090 (0.0007) [2023-10-11 20:38:02,431][71601] Updated weights for policy 0, policy_version 39100 (0.0008) [2023-10-11 20:38:03,157][71635] Updated weights for policy 1, policy_version 39082 (0.0009) [2023-10-11 20:38:03,526][71635] Updated weights for policy 1, policy_version 39092 (0.0011) [2023-10-11 20:38:03,894][71635] Updated weights for policy 1, policy_version 39102 (0.0010) [2023-10-11 20:38:06,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 80084992. Throughput: 0: 1827.1, 1: 1816.9. Samples: 20030252. Policy #0 lag: (min: 16.0, avg: 44.1, max: 48.0) [2023-10-11 20:38:06,035][70582] Avg episode reward: [(0, '132.580'), (1, '119.640')] [2023-10-11 20:38:06,093][71601] Updated weights for policy 0, policy_version 39110 (0.0007) [2023-10-11 20:38:06,458][71601] Updated weights for policy 0, policy_version 39120 (0.0008) [2023-10-11 20:38:06,820][71601] Updated weights for policy 0, policy_version 39130 (0.0009) [2023-10-11 20:38:07,645][71635] Updated weights for policy 1, policy_version 39112 (0.0008) [2023-10-11 20:38:08,016][71635] Updated weights for policy 1, policy_version 39122 (0.0007) [2023-10-11 20:38:08,384][71635] Updated weights for policy 1, policy_version 39132 (0.0010) [2023-10-11 20:38:10,334][71601] Updated weights for policy 0, policy_version 39140 (0.0009) [2023-10-11 20:38:10,703][71601] Updated weights for policy 0, policy_version 39150 (0.0010) [2023-10-11 20:38:11,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 80150528. Throughput: 0: 1825.5, 1: 1815.6. Samples: 20053038. Policy #0 lag: (min: 16.0, avg: 44.1, max: 48.0) [2023-10-11 20:38:11,034][70582] Avg episode reward: [(0, '135.170'), (1, '134.170')] [2023-10-11 20:38:11,087][71601] Updated weights for policy 0, policy_version 39160 (0.0010) [2023-10-11 20:38:12,122][71635] Updated weights for policy 1, policy_version 39142 (0.0009) [2023-10-11 20:38:12,492][71635] Updated weights for policy 1, policy_version 39152 (0.0008) [2023-10-11 20:38:12,871][71635] Updated weights for policy 1, policy_version 39162 (0.0008) [2023-10-11 20:38:14,632][71601] Updated weights for policy 0, policy_version 39170 (0.0007) [2023-10-11 20:38:15,008][71601] Updated weights for policy 0, policy_version 39180 (0.0009) [2023-10-11 20:38:15,380][71601] Updated weights for policy 0, policy_version 39190 (0.0008) [2023-10-11 20:38:15,759][71601] Updated weights for policy 0, policy_version 39200 (0.0011) [2023-10-11 20:38:16,034][70582] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 80248832. Throughput: 0: 1831.5, 1: 1815.9. Samples: 20063320. Policy #0 lag: (min: 39.0, avg: 47.7, max: 48.0) [2023-10-11 20:38:16,034][70582] Avg episode reward: [(0, '136.820'), (1, '132.040')] [2023-10-11 20:38:16,583][71635] Updated weights for policy 1, policy_version 39172 (0.0009) [2023-10-11 20:38:16,953][71635] Updated weights for policy 1, policy_version 39182 (0.0007) [2023-10-11 20:38:17,309][71635] Updated weights for policy 1, policy_version 39192 (0.0007) [2023-10-11 20:38:19,532][71601] Updated weights for policy 0, policy_version 39210 (0.0008) [2023-10-11 20:38:19,904][71601] Updated weights for policy 0, policy_version 39220 (0.0008) [2023-10-11 20:38:20,262][71601] Updated weights for policy 0, policy_version 39230 (0.0009) [2023-10-11 20:38:21,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 80314368. Throughput: 0: 1817.0, 1: 1810.5. Samples: 20085676. Policy #0 lag: (min: 39.0, avg: 47.7, max: 48.0) [2023-10-11 20:38:21,034][70582] Avg episode reward: [(0, '125.870'), (1, '130.330')] [2023-10-11 20:38:21,057][71635] Updated weights for policy 1, policy_version 39202 (0.0008) [2023-10-11 20:38:21,428][71635] Updated weights for policy 1, policy_version 39212 (0.0008) [2023-10-11 20:38:21,797][71635] Updated weights for policy 1, policy_version 39222 (0.0010) [2023-10-11 20:38:22,167][71635] Updated weights for policy 1, policy_version 39232 (0.0009) [2023-10-11 20:38:24,133][71601] Updated weights for policy 0, policy_version 39240 (0.0008) [2023-10-11 20:38:24,509][71601] Updated weights for policy 0, policy_version 39250 (0.0009) [2023-10-11 20:38:24,887][71601] Updated weights for policy 0, policy_version 39260 (0.0012) [2023-10-11 20:38:25,943][71635] Updated weights for policy 1, policy_version 39242 (0.0010) [2023-10-11 20:38:26,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 80379904. Throughput: 0: 1824.4, 1: 1817.2. Samples: 20107196. Policy #0 lag: (min: 39.0, avg: 47.7, max: 48.0) [2023-10-11 20:38:26,034][70582] Avg episode reward: [(0, '144.840'), (1, '136.600')] [2023-10-11 20:38:26,318][71635] Updated weights for policy 1, policy_version 39252 (0.0008) [2023-10-11 20:38:26,682][71635] Updated weights for policy 1, policy_version 39262 (0.0008) [2023-10-11 20:38:28,675][71601] Updated weights for policy 0, policy_version 39270 (0.0009) [2023-10-11 20:38:29,040][71601] Updated weights for policy 0, policy_version 39280 (0.0008) [2023-10-11 20:38:29,417][71601] Updated weights for policy 0, policy_version 39290 (0.0009) [2023-10-11 20:38:30,250][71635] Updated weights for policy 1, policy_version 39272 (0.0009) [2023-10-11 20:38:30,632][71635] Updated weights for policy 1, policy_version 39282 (0.0008) [2023-10-11 20:38:30,996][71635] Updated weights for policy 1, policy_version 39292 (0.0011) [2023-10-11 20:38:31,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 80445440. Throughput: 0: 1820.4, 1: 1814.9. Samples: 20118660. Policy #0 lag: (min: 39.0, avg: 47.7, max: 48.0) [2023-10-11 20:38:31,035][70582] Avg episode reward: [(0, '139.750'), (1, '136.280')] [2023-10-11 20:38:33,084][71601] Updated weights for policy 0, policy_version 39300 (0.0009) [2023-10-11 20:38:33,451][71601] Updated weights for policy 0, policy_version 39310 (0.0009) [2023-10-11 20:38:33,817][71601] Updated weights for policy 0, policy_version 39320 (0.0009) [2023-10-11 20:38:34,740][71635] Updated weights for policy 1, policy_version 39302 (0.0007) [2023-10-11 20:38:35,105][71635] Updated weights for policy 1, policy_version 39312 (0.0007) [2023-10-11 20:38:35,472][71635] Updated weights for policy 1, policy_version 39322 (0.0010) [2023-10-11 20:38:36,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 80543744. Throughput: 0: 1813.9, 1: 1820.3. Samples: 20139898. Policy #0 lag: (min: 39.0, avg: 47.7, max: 48.0) [2023-10-11 20:38:36,035][70582] Avg episode reward: [(0, '138.970'), (1, '136.170')] [2023-10-11 20:38:37,483][71601] Updated weights for policy 0, policy_version 39330 (0.0008) [2023-10-11 20:38:37,850][71601] Updated weights for policy 0, policy_version 39340 (0.0008) [2023-10-11 20:38:38,217][71601] Updated weights for policy 0, policy_version 39350 (0.0008) [2023-10-11 20:38:38,587][71601] Updated weights for policy 0, policy_version 39360 (0.0008) [2023-10-11 20:38:39,047][71635] Updated weights for policy 1, policy_version 39332 (0.0007) [2023-10-11 20:38:39,412][71635] Updated weights for policy 1, policy_version 39342 (0.0010) [2023-10-11 20:38:39,783][71635] Updated weights for policy 1, policy_version 39352 (0.0009) [2023-10-11 20:38:41,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 80609280. Throughput: 0: 1814.1, 1: 1824.3. Samples: 20161606. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:38:41,035][70582] Avg episode reward: [(0, '138.970'), (1, '134.700')] [2023-10-11 20:38:42,182][71601] Updated weights for policy 0, policy_version 39370 (0.0008) [2023-10-11 20:38:42,552][71601] Updated weights for policy 0, policy_version 39380 (0.0009) [2023-10-11 20:38:42,934][71601] Updated weights for policy 0, policy_version 39390 (0.0010) [2023-10-11 20:38:43,355][71635] Updated weights for policy 1, policy_version 39362 (0.0008) [2023-10-11 20:38:43,718][71635] Updated weights for policy 1, policy_version 39372 (0.0009) [2023-10-11 20:38:44,083][71635] Updated weights for policy 1, policy_version 39382 (0.0010) [2023-10-11 20:38:44,455][71635] Updated weights for policy 1, policy_version 39392 (0.0009) [2023-10-11 20:38:46,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 80674816. Throughput: 0: 1821.1, 1: 1816.4. Samples: 20172608. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:38:46,035][70582] Avg episode reward: [(0, '138.970'), (1, '150.130')] [2023-10-11 20:38:46,446][71601] Updated weights for policy 0, policy_version 39400 (0.0010) [2023-10-11 20:38:46,821][71601] Updated weights for policy 0, policy_version 39410 (0.0009) [2023-10-11 20:38:47,186][71601] Updated weights for policy 0, policy_version 39420 (0.0009) [2023-10-11 20:38:48,242][71635] Updated weights for policy 1, policy_version 39402 (0.0011) [2023-10-11 20:38:48,619][71635] Updated weights for policy 1, policy_version 39412 (0.0010) [2023-10-11 20:38:48,988][71635] Updated weights for policy 1, policy_version 39422 (0.0008) [2023-10-11 20:38:50,860][71601] Updated weights for policy 0, policy_version 39430 (0.0008) [2023-10-11 20:38:51,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 80740352. Throughput: 0: 1828.0, 1: 1814.8. Samples: 20194176. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:38:51,035][70582] Avg episode reward: [(0, '138.970'), (1, '151.110')] [2023-10-11 20:38:51,236][71601] Updated weights for policy 0, policy_version 39440 (0.0009) [2023-10-11 20:38:51,616][71601] Updated weights for policy 0, policy_version 39450 (0.0007) [2023-10-11 20:38:52,712][71635] Updated weights for policy 1, policy_version 39432 (0.0008) [2023-10-11 20:38:53,075][71635] Updated weights for policy 1, policy_version 39442 (0.0007) [2023-10-11 20:38:53,434][71635] Updated weights for policy 1, policy_version 39452 (0.0009) [2023-10-11 20:38:55,301][71601] Updated weights for policy 0, policy_version 39460 (0.0008) [2023-10-11 20:38:55,670][71601] Updated weights for policy 0, policy_version 39470 (0.0008) [2023-10-11 20:38:56,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 80805888. Throughput: 0: 1820.4, 1: 1823.0. Samples: 20216990. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:38:56,035][70582] Avg episode reward: [(0, '138.970'), (1, '148.070')] [2023-10-11 20:38:56,044][71601] Updated weights for policy 0, policy_version 39480 (0.0009) [2023-10-11 20:38:57,108][71635] Updated weights for policy 1, policy_version 39462 (0.0008) [2023-10-11 20:38:57,471][71635] Updated weights for policy 1, policy_version 39472 (0.0010) [2023-10-11 20:38:57,846][71635] Updated weights for policy 1, policy_version 39482 (0.0010) [2023-10-11 20:38:59,510][71601] Updated weights for policy 0, policy_version 39490 (0.0010) [2023-10-11 20:38:59,877][71601] Updated weights for policy 0, policy_version 39500 (0.0009) [2023-10-11 20:39:00,243][71601] Updated weights for policy 0, policy_version 39510 (0.0007) [2023-10-11 20:39:00,619][71601] Updated weights for policy 0, policy_version 39520 (0.0007) [2023-10-11 20:39:01,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 80904192. Throughput: 0: 1828.0, 1: 1825.2. Samples: 20227712. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:39:01,034][70582] Avg episode reward: [(0, '138.970'), (1, '149.970')] [2023-10-11 20:39:01,546][71635] Updated weights for policy 1, policy_version 39492 (0.0008) [2023-10-11 20:39:01,901][71635] Updated weights for policy 1, policy_version 39502 (0.0010) [2023-10-11 20:39:02,272][71635] Updated weights for policy 1, policy_version 39512 (0.0009) [2023-10-11 20:39:04,194][71601] Updated weights for policy 0, policy_version 39530 (0.0007) [2023-10-11 20:39:04,567][71601] Updated weights for policy 0, policy_version 39540 (0.0007) [2023-10-11 20:39:04,937][71601] Updated weights for policy 0, policy_version 39550 (0.0008) [2023-10-11 20:39:05,973][71635] Updated weights for policy 1, policy_version 39522 (0.0009) [2023-10-11 20:39:06,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 80969728. Throughput: 0: 1823.8, 1: 1825.6. Samples: 20249900. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-11 20:39:06,035][70582] Avg episode reward: [(0, '142.120'), (1, '150.140')] [2023-10-11 20:39:06,368][71635] Updated weights for policy 1, policy_version 39532 (0.0007) [2023-10-11 20:39:06,737][71635] Updated weights for policy 1, policy_version 39542 (0.0008) [2023-10-11 20:39:07,098][71635] Updated weights for policy 1, policy_version 39552 (0.0009) [2023-10-11 20:39:08,594][71601] Updated weights for policy 0, policy_version 39560 (0.0009) [2023-10-11 20:39:08,955][71601] Updated weights for policy 0, policy_version 39570 (0.0008) [2023-10-11 20:39:09,327][71601] Updated weights for policy 0, policy_version 39580 (0.0008) [2023-10-11 20:39:10,766][71635] Updated weights for policy 1, policy_version 39562 (0.0009) [2023-10-11 20:39:11,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 81035264. Throughput: 0: 1832.9, 1: 1832.1. Samples: 20272120. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-11 20:39:11,034][70582] Avg episode reward: [(0, '162.540'), (1, '142.660')] [2023-10-11 20:39:11,126][71635] Updated weights for policy 1, policy_version 39572 (0.0007) [2023-10-11 20:39:11,493][71635] Updated weights for policy 1, policy_version 39582 (0.0008) [2023-10-11 20:39:13,089][71601] Updated weights for policy 0, policy_version 39590 (0.0008) [2023-10-11 20:39:13,449][71601] Updated weights for policy 0, policy_version 39600 (0.0008) [2023-10-11 20:39:13,818][71601] Updated weights for policy 0, policy_version 39610 (0.0007) [2023-10-11 20:39:15,208][71635] Updated weights for policy 1, policy_version 39592 (0.0009) [2023-10-11 20:39:15,579][71635] Updated weights for policy 1, policy_version 39602 (0.0007) [2023-10-11 20:39:15,949][71635] Updated weights for policy 1, policy_version 39612 (0.0009) [2023-10-11 20:39:16,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 81100800. Throughput: 0: 1822.4, 1: 1828.4. Samples: 20282946. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-11 20:39:16,035][70582] Avg episode reward: [(0, '162.650'), (1, '146.310')] [2023-10-11 20:39:17,417][71601] Updated weights for policy 0, policy_version 39620 (0.0009) [2023-10-11 20:39:17,788][71601] Updated weights for policy 0, policy_version 39630 (0.0010) [2023-10-11 20:39:18,165][71601] Updated weights for policy 0, policy_version 39640 (0.0010) [2023-10-11 20:39:19,690][71635] Updated weights for policy 1, policy_version 39622 (0.0009) [2023-10-11 20:39:20,057][71635] Updated weights for policy 1, policy_version 39632 (0.0008) [2023-10-11 20:39:20,419][71635] Updated weights for policy 1, policy_version 39642 (0.0008) [2023-10-11 20:39:21,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 81199104. Throughput: 0: 1839.4, 1: 1823.2. Samples: 20304716. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-11 20:39:21,034][70582] Avg episode reward: [(0, '174.700'), (1, '148.620')] [2023-10-11 20:39:22,003][71601] Updated weights for policy 0, policy_version 39650 (0.0008) [2023-10-11 20:39:22,375][71601] Updated weights for policy 0, policy_version 39660 (0.0009) [2023-10-11 20:39:22,748][71601] Updated weights for policy 0, policy_version 39670 (0.0008) [2023-10-11 20:39:23,120][71601] Updated weights for policy 0, policy_version 39680 (0.0010) [2023-10-11 20:39:24,113][71635] Updated weights for policy 1, policy_version 39652 (0.0008) [2023-10-11 20:39:24,479][71635] Updated weights for policy 1, policy_version 39662 (0.0008) [2023-10-11 20:39:24,847][71635] Updated weights for policy 1, policy_version 39672 (0.0008) [2023-10-11 20:39:26,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 81264640. Throughput: 0: 1836.1, 1: 1818.3. Samples: 20326052. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-11 20:39:26,034][70582] Avg episode reward: [(0, '177.550'), (1, '172.390')] [2023-10-11 20:39:26,833][71601] Updated weights for policy 0, policy_version 39690 (0.0007) [2023-10-11 20:39:27,218][71601] Updated weights for policy 0, policy_version 39700 (0.0007) [2023-10-11 20:39:27,581][71601] Updated weights for policy 0, policy_version 39710 (0.0009) [2023-10-11 20:39:28,449][71635] Updated weights for policy 1, policy_version 39682 (0.0009) [2023-10-11 20:39:28,819][71635] Updated weights for policy 1, policy_version 39692 (0.0009) [2023-10-11 20:39:29,188][71635] Updated weights for policy 1, policy_version 39702 (0.0007) [2023-10-11 20:39:29,550][71635] Updated weights for policy 1, policy_version 39712 (0.0009) [2023-10-11 20:39:31,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 81330176. Throughput: 0: 1837.5, 1: 1826.8. Samples: 20337500. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-11 20:39:31,035][70582] Avg episode reward: [(0, '194.170'), (1, '161.100')] [2023-10-11 20:39:31,271][71601] Updated weights for policy 0, policy_version 39720 (0.0009) [2023-10-11 20:39:31,646][71601] Updated weights for policy 0, policy_version 39730 (0.0007) [2023-10-11 20:39:32,023][71601] Updated weights for policy 0, policy_version 39740 (0.0010) [2023-10-11 20:39:33,264][71635] Updated weights for policy 1, policy_version 39722 (0.0009) [2023-10-11 20:39:33,638][71635] Updated weights for policy 1, policy_version 39732 (0.0009) [2023-10-11 20:39:34,002][71635] Updated weights for policy 1, policy_version 39742 (0.0010) [2023-10-11 20:39:35,726][71601] Updated weights for policy 0, policy_version 39750 (0.0009) [2023-10-11 20:39:36,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 81395712. Throughput: 0: 1827.2, 1: 1829.6. Samples: 20358734. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-11 20:39:36,035][70582] Avg episode reward: [(0, '193.910'), (1, '147.330')] [2023-10-11 20:39:36,092][71601] Updated weights for policy 0, policy_version 39760 (0.0009) [2023-10-11 20:39:36,468][71601] Updated weights for policy 0, policy_version 39770 (0.0007) [2023-10-11 20:39:37,694][71635] Updated weights for policy 1, policy_version 39752 (0.0008) [2023-10-11 20:39:38,050][71635] Updated weights for policy 1, policy_version 39762 (0.0007) [2023-10-11 20:39:38,411][71635] Updated weights for policy 1, policy_version 39772 (0.0007) [2023-10-11 20:39:40,101][71601] Updated weights for policy 0, policy_version 39780 (0.0010) [2023-10-11 20:39:40,481][71601] Updated weights for policy 0, policy_version 39790 (0.0009) [2023-10-11 20:39:40,842][71601] Updated weights for policy 0, policy_version 39800 (0.0008) [2023-10-11 20:39:41,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 81461248. Throughput: 0: 1820.2, 1: 1822.9. Samples: 20380926. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-11 20:39:41,034][70582] Avg episode reward: [(0, '194.610'), (1, '152.080')] [2023-10-11 20:39:42,049][71635] Updated weights for policy 1, policy_version 39782 (0.0010) [2023-10-11 20:39:42,407][71635] Updated weights for policy 1, policy_version 39792 (0.0008) [2023-10-11 20:39:42,787][71635] Updated weights for policy 1, policy_version 39802 (0.0008) [2023-10-11 20:39:44,618][71601] Updated weights for policy 0, policy_version 39810 (0.0007) [2023-10-11 20:39:44,985][71601] Updated weights for policy 0, policy_version 39820 (0.0007) [2023-10-11 20:39:45,358][71601] Updated weights for policy 0, policy_version 39830 (0.0010) [2023-10-11 20:39:45,736][71601] Updated weights for policy 0, policy_version 39840 (0.0010) [2023-10-11 20:39:46,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 81559552. Throughput: 0: 1820.1, 1: 1820.9. Samples: 20391556. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-11 20:39:46,035][70582] Avg episode reward: [(0, '194.800'), (1, '149.860')] [2023-10-11 20:39:46,501][71635] Updated weights for policy 1, policy_version 39812 (0.0009) [2023-10-11 20:39:46,857][71635] Updated weights for policy 1, policy_version 39822 (0.0011) [2023-10-11 20:39:47,223][71635] Updated weights for policy 1, policy_version 39832 (0.0011) [2023-10-11 20:39:49,545][71601] Updated weights for policy 0, policy_version 39850 (0.0010) [2023-10-11 20:39:49,912][71601] Updated weights for policy 0, policy_version 39860 (0.0009) [2023-10-11 20:39:50,284][71601] Updated weights for policy 0, policy_version 39870 (0.0008) [2023-10-11 20:39:51,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 81625088. Throughput: 0: 1820.6, 1: 1816.9. Samples: 20413588. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-11 20:39:51,035][70582] Avg episode reward: [(0, '194.800'), (1, '146.660')] [2023-10-11 20:39:51,054][71635] Updated weights for policy 1, policy_version 39842 (0.0009) [2023-10-11 20:39:51,452][71635] Updated weights for policy 1, policy_version 39852 (0.0008) [2023-10-11 20:39:51,826][71635] Updated weights for policy 1, policy_version 39862 (0.0007) [2023-10-11 20:39:52,194][71635] Updated weights for policy 1, policy_version 39872 (0.0007) [2023-10-11 20:39:54,099][71601] Updated weights for policy 0, policy_version 39880 (0.0009) [2023-10-11 20:39:54,453][71601] Updated weights for policy 0, policy_version 39890 (0.0007) [2023-10-11 20:39:54,822][71601] Updated weights for policy 0, policy_version 39900 (0.0007) [2023-10-11 20:39:55,827][71635] Updated weights for policy 1, policy_version 39882 (0.0007) [2023-10-11 20:39:56,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 81690624. Throughput: 0: 1812.8, 1: 1815.3. Samples: 20435384. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-11 20:39:56,034][70582] Avg episode reward: [(0, '194.800'), (1, '146.830')] [2023-10-11 20:39:56,044][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000039904_40861696.pth... [2023-10-11 20:39:56,077][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000038208_39124992.pth [2023-10-11 20:39:56,196][71635] Updated weights for policy 1, policy_version 39892 (0.0007) [2023-10-11 20:39:56,555][71635] Updated weights for policy 1, policy_version 39902 (0.0009) [2023-10-11 20:39:56,627][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000039904_40861696.pth... [2023-10-11 20:39:56,656][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000038176_39092224.pth [2023-10-11 20:39:58,560][71601] Updated weights for policy 0, policy_version 39910 (0.0008) [2023-10-11 20:39:58,940][71601] Updated weights for policy 0, policy_version 39920 (0.0009) [2023-10-11 20:39:59,307][71601] Updated weights for policy 0, policy_version 39930 (0.0009) [2023-10-11 20:40:00,191][71635] Updated weights for policy 1, policy_version 39912 (0.0008) [2023-10-11 20:40:00,557][71635] Updated weights for policy 1, policy_version 39922 (0.0007) [2023-10-11 20:40:00,925][71635] Updated weights for policy 1, policy_version 39932 (0.0008) [2023-10-11 20:40:01,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 81756160. Throughput: 0: 1820.1, 1: 1814.5. Samples: 20446504. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-11 20:40:01,034][70582] Avg episode reward: [(0, '194.800'), (1, '151.530')] [2023-10-11 20:40:02,770][71601] Updated weights for policy 0, policy_version 39940 (0.0008) [2023-10-11 20:40:03,136][71601] Updated weights for policy 0, policy_version 39950 (0.0007) [2023-10-11 20:40:03,507][71601] Updated weights for policy 0, policy_version 39960 (0.0008) [2023-10-11 20:40:04,731][71635] Updated weights for policy 1, policy_version 39942 (0.0008) [2023-10-11 20:40:05,099][71635] Updated weights for policy 1, policy_version 39952 (0.0009) [2023-10-11 20:40:05,465][71635] Updated weights for policy 1, policy_version 39962 (0.0007) [2023-10-11 20:40:06,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 81854464. Throughput: 0: 1815.1, 1: 1818.6. Samples: 20468230. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-11 20:40:06,035][70582] Avg episode reward: [(0, '194.800'), (1, '152.670')] [2023-10-11 20:40:07,234][71601] Updated weights for policy 0, policy_version 39970 (0.0007) [2023-10-11 20:40:07,601][71601] Updated weights for policy 0, policy_version 39980 (0.0007) [2023-10-11 20:40:07,968][71601] Updated weights for policy 0, policy_version 39990 (0.0008) [2023-10-11 20:40:08,341][71601] Updated weights for policy 0, policy_version 40000 (0.0009) [2023-10-11 20:40:09,001][71635] Updated weights for policy 1, policy_version 39972 (0.0008) [2023-10-11 20:40:09,367][71635] Updated weights for policy 1, policy_version 39982 (0.0007) [2023-10-11 20:40:09,731][71635] Updated weights for policy 1, policy_version 39992 (0.0010) [2023-10-11 20:40:11,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 81920000. Throughput: 0: 1818.0, 1: 1822.5. Samples: 20489876. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-11 20:40:11,034][70582] Avg episode reward: [(0, '194.800'), (1, '158.950')] [2023-10-11 20:40:12,058][71601] Updated weights for policy 0, policy_version 40010 (0.0007) [2023-10-11 20:40:12,418][71601] Updated weights for policy 0, policy_version 40020 (0.0008) [2023-10-11 20:40:12,793][71601] Updated weights for policy 0, policy_version 40030 (0.0007) [2023-10-11 20:40:13,437][71635] Updated weights for policy 1, policy_version 40002 (0.0008) [2023-10-11 20:40:13,807][71635] Updated weights for policy 1, policy_version 40012 (0.0008) [2023-10-11 20:40:14,167][71635] Updated weights for policy 1, policy_version 40022 (0.0010) [2023-10-11 20:40:14,534][71635] Updated weights for policy 1, policy_version 40032 (0.0007) [2023-10-11 20:40:16,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 81985536. Throughput: 0: 1820.5, 1: 1820.6. Samples: 20501348. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-11 20:40:16,035][70582] Avg episode reward: [(0, '216.570'), (1, '160.250')] [2023-10-11 20:40:16,360][71601] Updated weights for policy 0, policy_version 40040 (0.0008) [2023-10-11 20:40:16,723][71601] Updated weights for policy 0, policy_version 40050 (0.0008) [2023-10-11 20:40:17,101][71601] Updated weights for policy 0, policy_version 40060 (0.0009) [2023-10-11 20:40:18,270][71635] Updated weights for policy 1, policy_version 40042 (0.0009) [2023-10-11 20:40:18,642][71635] Updated weights for policy 1, policy_version 40052 (0.0009) [2023-10-11 20:40:19,001][71635] Updated weights for policy 1, policy_version 40062 (0.0010) [2023-10-11 20:40:20,704][71601] Updated weights for policy 0, policy_version 40070 (0.0008) [2023-10-11 20:40:21,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 82051072. Throughput: 0: 1826.0, 1: 1819.0. Samples: 20522756. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-11 20:40:21,034][70582] Avg episode reward: [(0, '192.390'), (1, '153.250')] [2023-10-11 20:40:21,075][71601] Updated weights for policy 0, policy_version 40080 (0.0007) [2023-10-11 20:40:21,451][71601] Updated weights for policy 0, policy_version 40090 (0.0008) [2023-10-11 20:40:22,683][71635] Updated weights for policy 1, policy_version 40072 (0.0008) [2023-10-11 20:40:23,049][71635] Updated weights for policy 1, policy_version 40082 (0.0008) [2023-10-11 20:40:23,420][71635] Updated weights for policy 1, policy_version 40092 (0.0010) [2023-10-11 20:40:25,088][71601] Updated weights for policy 0, policy_version 40100 (0.0008) [2023-10-11 20:40:25,462][71601] Updated weights for policy 0, policy_version 40110 (0.0007) [2023-10-11 20:40:25,832][71601] Updated weights for policy 0, policy_version 40120 (0.0007) [2023-10-11 20:40:26,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 82116608. Throughput: 0: 1823.4, 1: 1817.5. Samples: 20544768. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-11 20:40:26,034][70582] Avg episode reward: [(0, '198.610'), (1, '138.780')] [2023-10-11 20:40:27,172][71635] Updated weights for policy 1, policy_version 40102 (0.0009) [2023-10-11 20:40:27,546][71635] Updated weights for policy 1, policy_version 40112 (0.0008) [2023-10-11 20:40:27,910][71635] Updated weights for policy 1, policy_version 40122 (0.0007) [2023-10-11 20:40:29,460][71601] Updated weights for policy 0, policy_version 40130 (0.0009) [2023-10-11 20:40:29,832][71601] Updated weights for policy 0, policy_version 40140 (0.0007) [2023-10-11 20:40:30,205][71601] Updated weights for policy 0, policy_version 40150 (0.0007) [2023-10-11 20:40:30,571][71601] Updated weights for policy 0, policy_version 40160 (0.0008) [2023-10-11 20:40:31,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 82214912. Throughput: 0: 1826.5, 1: 1816.4. Samples: 20555488. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-11 20:40:31,035][70582] Avg episode reward: [(0, '197.690'), (1, '151.550')] [2023-10-11 20:40:31,555][71635] Updated weights for policy 1, policy_version 40132 (0.0009) [2023-10-11 20:40:31,935][71635] Updated weights for policy 1, policy_version 40142 (0.0010) [2023-10-11 20:40:32,300][71635] Updated weights for policy 1, policy_version 40152 (0.0011) [2023-10-11 20:40:34,233][71601] Updated weights for policy 0, policy_version 40170 (0.0010) [2023-10-11 20:40:34,607][71601] Updated weights for policy 0, policy_version 40180 (0.0007) [2023-10-11 20:40:34,989][71601] Updated weights for policy 0, policy_version 40190 (0.0009) [2023-10-11 20:40:36,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 82280448. Throughput: 0: 1826.8, 1: 1820.9. Samples: 20577734. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-11 20:40:36,034][71635] Updated weights for policy 1, policy_version 40162 (0.0009) [2023-10-11 20:40:36,034][70582] Avg episode reward: [(0, '197.690'), (1, '152.710')] [2023-10-11 20:40:36,441][71635] Updated weights for policy 1, policy_version 40172 (0.0007) [2023-10-11 20:40:36,802][71635] Updated weights for policy 1, policy_version 40182 (0.0010) [2023-10-11 20:40:37,168][71635] Updated weights for policy 1, policy_version 40192 (0.0008) [2023-10-11 20:40:38,621][71601] Updated weights for policy 0, policy_version 40200 (0.0008) [2023-10-11 20:40:39,002][71601] Updated weights for policy 0, policy_version 40210 (0.0010) [2023-10-11 20:40:39,369][71601] Updated weights for policy 0, policy_version 40220 (0.0008) [2023-10-11 20:40:40,720][71635] Updated weights for policy 1, policy_version 40202 (0.0008) [2023-10-11 20:40:41,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 82345984. Throughput: 0: 1835.7, 1: 1820.2. Samples: 20599898. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-11 20:40:41,034][70582] Avg episode reward: [(0, '197.690'), (1, '152.560')] [2023-10-11 20:40:41,084][71635] Updated weights for policy 1, policy_version 40212 (0.0008) [2023-10-11 20:40:41,448][71635] Updated weights for policy 1, policy_version 40222 (0.0008) [2023-10-11 20:40:43,146][71601] Updated weights for policy 0, policy_version 40230 (0.0007) [2023-10-11 20:40:43,524][71601] Updated weights for policy 0, policy_version 40240 (0.0008) [2023-10-11 20:40:43,896][71601] Updated weights for policy 0, policy_version 40250 (0.0009) [2023-10-11 20:40:45,104][71635] Updated weights for policy 1, policy_version 40232 (0.0009) [2023-10-11 20:40:45,472][71635] Updated weights for policy 1, policy_version 40242 (0.0008) [2023-10-11 20:40:45,833][71635] Updated weights for policy 1, policy_version 40252 (0.0009) [2023-10-11 20:40:46,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 82444288. Throughput: 0: 1825.1, 1: 1822.0. Samples: 20610626. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-11 20:40:46,035][70582] Avg episode reward: [(0, '198.670'), (1, '166.880')] [2023-10-11 20:40:47,465][71601] Updated weights for policy 0, policy_version 40260 (0.0008) [2023-10-11 20:40:47,823][71601] Updated weights for policy 0, policy_version 40270 (0.0008) [2023-10-11 20:40:48,196][71601] Updated weights for policy 0, policy_version 40280 (0.0008) [2023-10-11 20:40:49,482][71635] Updated weights for policy 1, policy_version 40262 (0.0010) [2023-10-11 20:40:49,849][71635] Updated weights for policy 1, policy_version 40272 (0.0011) [2023-10-11 20:40:50,218][71635] Updated weights for policy 1, policy_version 40282 (0.0009) [2023-10-11 20:40:51,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 82509824. Throughput: 0: 1829.0, 1: 1822.1. Samples: 20632528. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-11 20:40:51,034][70582] Avg episode reward: [(0, '184.230'), (1, '164.080')] [2023-10-11 20:40:51,974][71601] Updated weights for policy 0, policy_version 40290 (0.0008) [2023-10-11 20:40:52,345][71601] Updated weights for policy 0, policy_version 40300 (0.0010) [2023-10-11 20:40:52,722][71601] Updated weights for policy 0, policy_version 40310 (0.0008) [2023-10-11 20:40:53,092][71601] Updated weights for policy 0, policy_version 40320 (0.0008) [2023-10-11 20:40:53,810][71635] Updated weights for policy 1, policy_version 40292 (0.0008) [2023-10-11 20:40:54,191][71635] Updated weights for policy 1, policy_version 40302 (0.0009) [2023-10-11 20:40:54,550][71635] Updated weights for policy 1, policy_version 40312 (0.0009) [2023-10-11 20:40:56,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 82575360. Throughput: 0: 1823.6, 1: 1829.3. Samples: 20654258. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-10-11 20:40:56,034][70582] Avg episode reward: [(0, '189.580'), (1, '167.160')] [2023-10-11 20:40:56,890][71601] Updated weights for policy 0, policy_version 40330 (0.0008) [2023-10-11 20:40:57,272][71601] Updated weights for policy 0, policy_version 40340 (0.0008) [2023-10-11 20:40:57,640][71601] Updated weights for policy 0, policy_version 40350 (0.0010) [2023-10-11 20:40:58,306][71635] Updated weights for policy 1, policy_version 40322 (0.0007) [2023-10-11 20:40:58,674][71635] Updated weights for policy 1, policy_version 40332 (0.0010) [2023-10-11 20:40:59,034][71635] Updated weights for policy 1, policy_version 40342 (0.0009) [2023-10-11 20:40:59,400][71635] Updated weights for policy 1, policy_version 40352 (0.0009) [2023-10-11 20:41:01,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 82640896. Throughput: 0: 1820.6, 1: 1827.4. Samples: 20665508. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-10-11 20:41:01,035][70582] Avg episode reward: [(0, '189.460'), (1, '161.720')] [2023-10-11 20:41:01,355][71601] Updated weights for policy 0, policy_version 40360 (0.0008) [2023-10-11 20:41:01,733][71601] Updated weights for policy 0, policy_version 40370 (0.0009) [2023-10-11 20:41:02,103][71601] Updated weights for policy 0, policy_version 40380 (0.0007) [2023-10-11 20:41:03,153][71635] Updated weights for policy 1, policy_version 40362 (0.0010) [2023-10-11 20:41:03,528][71635] Updated weights for policy 1, policy_version 40372 (0.0008) [2023-10-11 20:41:03,890][71635] Updated weights for policy 1, policy_version 40382 (0.0008) [2023-10-11 20:41:05,669][71601] Updated weights for policy 0, policy_version 40390 (0.0010) [2023-10-11 20:41:06,034][71601] Updated weights for policy 0, policy_version 40400 (0.0008) [2023-10-11 20:41:06,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 82706432. Throughput: 0: 1817.1, 1: 1829.3. Samples: 20686844. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-10-11 20:41:06,035][70582] Avg episode reward: [(0, '195.610'), (1, '155.920')] [2023-10-11 20:41:06,405][71601] Updated weights for policy 0, policy_version 40410 (0.0009) [2023-10-11 20:41:07,630][71635] Updated weights for policy 1, policy_version 40392 (0.0007) [2023-10-11 20:41:07,997][71635] Updated weights for policy 1, policy_version 40402 (0.0009) [2023-10-11 20:41:08,362][71635] Updated weights for policy 1, policy_version 40412 (0.0007) [2023-10-11 20:41:10,178][71601] Updated weights for policy 0, policy_version 40420 (0.0009) [2023-10-11 20:41:10,543][71601] Updated weights for policy 0, policy_version 40430 (0.0010) [2023-10-11 20:41:10,907][71601] Updated weights for policy 0, policy_version 40440 (0.0010) [2023-10-11 20:41:11,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 82771968. Throughput: 0: 1824.3, 1: 1829.5. Samples: 20709186. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-10-11 20:41:11,034][70582] Avg episode reward: [(0, '195.770'), (1, '160.700')] [2023-10-11 20:41:12,091][71635] Updated weights for policy 1, policy_version 40422 (0.0007) [2023-10-11 20:41:12,455][71635] Updated weights for policy 1, policy_version 40432 (0.0009) [2023-10-11 20:41:12,820][71635] Updated weights for policy 1, policy_version 40442 (0.0011) [2023-10-11 20:41:14,565][71601] Updated weights for policy 0, policy_version 40450 (0.0009) [2023-10-11 20:41:14,934][71601] Updated weights for policy 0, policy_version 40460 (0.0008) [2023-10-11 20:41:15,309][71601] Updated weights for policy 0, policy_version 40470 (0.0008) [2023-10-11 20:41:15,682][71601] Updated weights for policy 0, policy_version 40480 (0.0008) [2023-10-11 20:41:16,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 82870272. Throughput: 0: 1818.9, 1: 1829.8. Samples: 20719682. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-10-11 20:41:16,035][70582] Avg episode reward: [(0, '194.710'), (1, '147.160')] [2023-10-11 20:41:16,271][71635] Updated weights for policy 1, policy_version 40452 (0.0008) [2023-10-11 20:41:16,640][71635] Updated weights for policy 1, policy_version 40462 (0.0008) [2023-10-11 20:41:17,004][71635] Updated weights for policy 1, policy_version 40472 (0.0010) [2023-10-11 20:41:19,326][71601] Updated weights for policy 0, policy_version 40490 (0.0008) [2023-10-11 20:41:19,692][71601] Updated weights for policy 0, policy_version 40500 (0.0007) [2023-10-11 20:41:20,070][71601] Updated weights for policy 0, policy_version 40510 (0.0008) [2023-10-11 20:41:20,799][71635] Updated weights for policy 1, policy_version 40482 (0.0010) [2023-10-11 20:41:21,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 82935808. Throughput: 0: 1817.1, 1: 1833.6. Samples: 20742016. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:41:21,034][70582] Avg episode reward: [(0, '194.710'), (1, '149.180')] [2023-10-11 20:41:21,215][71635] Updated weights for policy 1, policy_version 40492 (0.0007) [2023-10-11 20:41:21,596][71635] Updated weights for policy 1, policy_version 40502 (0.0007) [2023-10-11 20:41:21,957][71635] Updated weights for policy 1, policy_version 40512 (0.0008) [2023-10-11 20:41:23,667][71601] Updated weights for policy 0, policy_version 40520 (0.0008) [2023-10-11 20:41:24,030][71601] Updated weights for policy 0, policy_version 40530 (0.0008) [2023-10-11 20:41:24,398][71601] Updated weights for policy 0, policy_version 40540 (0.0007) [2023-10-11 20:41:25,661][71635] Updated weights for policy 1, policy_version 40522 (0.0007) [2023-10-11 20:41:26,032][71635] Updated weights for policy 1, policy_version 40532 (0.0008) [2023-10-11 20:41:26,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 83001344. Throughput: 0: 1814.3, 1: 1825.1. Samples: 20763676. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:41:26,035][70582] Avg episode reward: [(0, '194.710'), (1, '149.040')] [2023-10-11 20:41:26,389][71635] Updated weights for policy 1, policy_version 40542 (0.0007) [2023-10-11 20:41:28,179][71601] Updated weights for policy 0, policy_version 40550 (0.0008) [2023-10-11 20:41:28,561][71601] Updated weights for policy 0, policy_version 40560 (0.0009) [2023-10-11 20:41:28,930][71601] Updated weights for policy 0, policy_version 40570 (0.0008) [2023-10-11 20:41:30,131][71635] Updated weights for policy 1, policy_version 40552 (0.0008) [2023-10-11 20:41:30,495][71635] Updated weights for policy 1, policy_version 40562 (0.0008) [2023-10-11 20:41:30,866][71635] Updated weights for policy 1, policy_version 40572 (0.0007) [2023-10-11 20:41:31,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 83099648. Throughput: 0: 1814.7, 1: 1825.7. Samples: 20774446. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:41:31,035][70582] Avg episode reward: [(0, '194.710'), (1, '143.410')] [2023-10-11 20:41:32,568][71601] Updated weights for policy 0, policy_version 40580 (0.0007) [2023-10-11 20:41:32,939][71601] Updated weights for policy 0, policy_version 40590 (0.0007) [2023-10-11 20:41:33,317][71601] Updated weights for policy 0, policy_version 40600 (0.0007) [2023-10-11 20:41:34,527][71635] Updated weights for policy 1, policy_version 40582 (0.0009) [2023-10-11 20:41:34,896][71635] Updated weights for policy 1, policy_version 40592 (0.0011) [2023-10-11 20:41:35,260][71635] Updated weights for policy 1, policy_version 40602 (0.0011) [2023-10-11 20:41:36,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 83165184. Throughput: 0: 1815.2, 1: 1821.7. Samples: 20796186. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:41:36,034][70582] Avg episode reward: [(0, '196.320'), (1, '143.010')] [2023-10-11 20:41:37,069][71601] Updated weights for policy 0, policy_version 40610 (0.0009) [2023-10-11 20:41:37,438][71601] Updated weights for policy 0, policy_version 40620 (0.0008) [2023-10-11 20:41:37,808][71601] Updated weights for policy 0, policy_version 40630 (0.0009) [2023-10-11 20:41:38,179][71601] Updated weights for policy 0, policy_version 40640 (0.0012) [2023-10-11 20:41:38,904][71635] Updated weights for policy 1, policy_version 40612 (0.0010) [2023-10-11 20:41:39,273][71635] Updated weights for policy 1, policy_version 40622 (0.0009) [2023-10-11 20:41:39,649][71635] Updated weights for policy 1, policy_version 40632 (0.0010) [2023-10-11 20:41:41,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 83230720. Throughput: 0: 1810.5, 1: 1812.6. Samples: 20817298. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:41:41,034][70582] Avg episode reward: [(0, '195.550'), (1, '144.690')] [2023-10-11 20:41:42,068][71601] Updated weights for policy 0, policy_version 40650 (0.0010) [2023-10-11 20:41:42,434][71601] Updated weights for policy 0, policy_version 40660 (0.0008) [2023-10-11 20:41:42,799][71601] Updated weights for policy 0, policy_version 40670 (0.0008) [2023-10-11 20:41:43,288][71635] Updated weights for policy 1, policy_version 40642 (0.0009) [2023-10-11 20:41:43,659][71635] Updated weights for policy 1, policy_version 40652 (0.0008) [2023-10-11 20:41:44,029][71635] Updated weights for policy 1, policy_version 40662 (0.0010) [2023-10-11 20:41:44,396][71635] Updated weights for policy 1, policy_version 40672 (0.0007) [2023-10-11 20:41:46,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 83296256. Throughput: 0: 1807.9, 1: 1813.3. Samples: 20828462. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:41:46,034][70582] Avg episode reward: [(0, '195.340'), (1, '138.830')] [2023-10-11 20:41:46,594][71601] Updated weights for policy 0, policy_version 40680 (0.0009) [2023-10-11 20:41:46,967][71601] Updated weights for policy 0, policy_version 40690 (0.0008) [2023-10-11 20:41:47,350][71601] Updated weights for policy 0, policy_version 40700 (0.0007) [2023-10-11 20:41:48,031][71635] Updated weights for policy 1, policy_version 40682 (0.0007) [2023-10-11 20:41:48,394][71635] Updated weights for policy 1, policy_version 40692 (0.0007) [2023-10-11 20:41:48,771][71635] Updated weights for policy 1, policy_version 40702 (0.0008) [2023-10-11 20:41:50,926][71601] Updated weights for policy 0, policy_version 40710 (0.0009) [2023-10-11 20:41:51,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 83361792. Throughput: 0: 1806.3, 1: 1817.9. Samples: 20849934. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:41:51,035][70582] Avg episode reward: [(0, '208.900'), (1, '129.080')] [2023-10-11 20:41:51,297][71601] Updated weights for policy 0, policy_version 40720 (0.0007) [2023-10-11 20:41:51,670][71601] Updated weights for policy 0, policy_version 40730 (0.0007) [2023-10-11 20:41:52,465][71635] Updated weights for policy 1, policy_version 40712 (0.0009) [2023-10-11 20:41:52,836][71635] Updated weights for policy 1, policy_version 40722 (0.0009) [2023-10-11 20:41:53,207][71635] Updated weights for policy 1, policy_version 40732 (0.0008) [2023-10-11 20:41:55,329][71601] Updated weights for policy 0, policy_version 40740 (0.0009) [2023-10-11 20:41:55,698][71601] Updated weights for policy 0, policy_version 40750 (0.0009) [2023-10-11 20:41:56,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 83427328. Throughput: 0: 1808.8, 1: 1819.2. Samples: 20872448. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:41:56,034][70582] Avg episode reward: [(0, '221.520'), (1, '130.480')] [2023-10-11 20:41:56,042][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000040736_41713664.pth... [2023-10-11 20:41:56,065][71601] Updated weights for policy 0, policy_version 40760 (0.0008) [2023-10-11 20:41:56,071][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000039040_39976960.pth [2023-10-11 20:41:56,359][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000040768_41746432.pth... [2023-10-11 20:41:56,390][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000039040_39976960.pth [2023-10-11 20:41:56,963][71635] Updated weights for policy 1, policy_version 40742 (0.0010) [2023-10-11 20:41:57,322][71635] Updated weights for policy 1, policy_version 40752 (0.0010) [2023-10-11 20:41:57,676][71635] Updated weights for policy 1, policy_version 40762 (0.0008) [2023-10-11 20:41:59,831][71601] Updated weights for policy 0, policy_version 40770 (0.0010) [2023-10-11 20:42:00,206][71601] Updated weights for policy 0, policy_version 40780 (0.0008) [2023-10-11 20:42:00,567][71601] Updated weights for policy 0, policy_version 40790 (0.0011) [2023-10-11 20:42:00,941][71601] Updated weights for policy 0, policy_version 40800 (0.0009) [2023-10-11 20:42:01,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 83525632. Throughput: 0: 1805.4, 1: 1820.0. Samples: 20882824. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:42:01,034][70582] Avg episode reward: [(0, '218.240'), (1, '131.310')] [2023-10-11 20:42:01,291][71635] Updated weights for policy 1, policy_version 40772 (0.0008) [2023-10-11 20:42:01,651][71635] Updated weights for policy 1, policy_version 40782 (0.0010) [2023-10-11 20:42:02,018][71635] Updated weights for policy 1, policy_version 40792 (0.0010) [2023-10-11 20:42:04,679][71601] Updated weights for policy 0, policy_version 40810 (0.0009) [2023-10-11 20:42:05,047][71601] Updated weights for policy 0, policy_version 40820 (0.0010) [2023-10-11 20:42:05,426][71601] Updated weights for policy 0, policy_version 40830 (0.0010) [2023-10-11 20:42:05,848][71635] Updated weights for policy 1, policy_version 40802 (0.0010) [2023-10-11 20:42:06,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 83591168. Throughput: 0: 1814.9, 1: 1816.5. Samples: 20905430. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:42:06,035][70582] Avg episode reward: [(0, '229.910'), (1, '131.100')] [2023-10-11 20:42:06,262][71635] Updated weights for policy 1, policy_version 40812 (0.0007) [2023-10-11 20:42:06,638][71635] Updated weights for policy 1, policy_version 40822 (0.0009) [2023-10-11 20:42:06,995][71635] Updated weights for policy 1, policy_version 40832 (0.0008) [2023-10-11 20:42:09,112][71601] Updated weights for policy 0, policy_version 40840 (0.0008) [2023-10-11 20:42:09,483][71601] Updated weights for policy 0, policy_version 40850 (0.0008) [2023-10-11 20:42:09,844][71601] Updated weights for policy 0, policy_version 40860 (0.0009) [2023-10-11 20:42:10,627][71635] Updated weights for policy 1, policy_version 40842 (0.0010) [2023-10-11 20:42:10,995][71635] Updated weights for policy 1, policy_version 40852 (0.0009) [2023-10-11 20:42:11,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 83656704. Throughput: 0: 1798.1, 1: 1822.4. Samples: 20926596. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:42:11,034][70582] Avg episode reward: [(0, '241.530'), (1, '126.130')] [2023-10-11 20:42:11,366][71635] Updated weights for policy 1, policy_version 40862 (0.0010) [2023-10-11 20:42:13,539][71601] Updated weights for policy 0, policy_version 40870 (0.0011) [2023-10-11 20:42:13,907][71601] Updated weights for policy 0, policy_version 40880 (0.0007) [2023-10-11 20:42:14,277][71601] Updated weights for policy 0, policy_version 40890 (0.0009) [2023-10-11 20:42:14,772][71635] Updated weights for policy 1, policy_version 40872 (0.0009) [2023-10-11 20:42:15,138][71635] Updated weights for policy 1, policy_version 40882 (0.0007) [2023-10-11 20:42:15,506][71635] Updated weights for policy 1, policy_version 40892 (0.0008) [2023-10-11 20:42:16,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 83755008. Throughput: 0: 1809.0, 1: 1826.1. Samples: 20938026. Policy #0 lag: (min: 1.0, avg: 14.7, max: 33.0) [2023-10-11 20:42:16,035][70582] Avg episode reward: [(0, '254.520'), (1, '142.240')] [2023-10-11 20:42:17,946][71601] Updated weights for policy 0, policy_version 40900 (0.0008) [2023-10-11 20:42:18,322][71601] Updated weights for policy 0, policy_version 40910 (0.0007) [2023-10-11 20:42:18,697][71601] Updated weights for policy 0, policy_version 40920 (0.0007) [2023-10-11 20:42:19,300][71635] Updated weights for policy 1, policy_version 40902 (0.0009) [2023-10-11 20:42:19,657][71635] Updated weights for policy 1, policy_version 40912 (0.0008) [2023-10-11 20:42:20,026][71635] Updated weights for policy 1, policy_version 40922 (0.0009) [2023-10-11 20:42:21,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 83820544. Throughput: 0: 1800.7, 1: 1825.9. Samples: 20959382. Policy #0 lag: (min: 1.0, avg: 14.7, max: 33.0) [2023-10-11 20:42:21,035][70582] Avg episode reward: [(0, '239.780'), (1, '145.000')] [2023-10-11 20:42:22,280][71601] Updated weights for policy 0, policy_version 40930 (0.0010) [2023-10-11 20:42:22,654][71601] Updated weights for policy 0, policy_version 40940 (0.0009) [2023-10-11 20:42:23,022][71601] Updated weights for policy 0, policy_version 40950 (0.0007) [2023-10-11 20:42:23,399][71601] Updated weights for policy 0, policy_version 40960 (0.0007) [2023-10-11 20:42:23,771][71635] Updated weights for policy 1, policy_version 40932 (0.0011) [2023-10-11 20:42:24,135][71635] Updated weights for policy 1, policy_version 40942 (0.0008) [2023-10-11 20:42:24,501][71635] Updated weights for policy 1, policy_version 40952 (0.0010) [2023-10-11 20:42:26,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 83886080. Throughput: 0: 1809.7, 1: 1830.4. Samples: 20981104. Policy #0 lag: (min: 1.0, avg: 14.7, max: 33.0) [2023-10-11 20:42:26,034][70582] Avg episode reward: [(0, '233.130'), (1, '144.620')] [2023-10-11 20:42:27,108][71601] Updated weights for policy 0, policy_version 40970 (0.0009) [2023-10-11 20:42:27,480][71601] Updated weights for policy 0, policy_version 40980 (0.0009) [2023-10-11 20:42:27,853][71601] Updated weights for policy 0, policy_version 40990 (0.0008) [2023-10-11 20:42:28,294][71635] Updated weights for policy 1, policy_version 40962 (0.0007) [2023-10-11 20:42:28,666][71635] Updated weights for policy 1, policy_version 40972 (0.0011) [2023-10-11 20:42:29,046][71635] Updated weights for policy 1, policy_version 40982 (0.0007) [2023-10-11 20:42:29,410][71635] Updated weights for policy 1, policy_version 40992 (0.0008) [2023-10-11 20:42:31,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 83951616. Throughput: 0: 1813.3, 1: 1827.9. Samples: 20992316. Policy #0 lag: (min: 1.0, avg: 14.7, max: 33.0) [2023-10-11 20:42:31,035][70582] Avg episode reward: [(0, '232.180'), (1, '146.880')] [2023-10-11 20:42:31,651][71601] Updated weights for policy 0, policy_version 41000 (0.0007) [2023-10-11 20:42:32,028][71601] Updated weights for policy 0, policy_version 41010 (0.0007) [2023-10-11 20:42:32,402][71601] Updated weights for policy 0, policy_version 41020 (0.0008) [2023-10-11 20:42:33,084][71635] Updated weights for policy 1, policy_version 41002 (0.0007) [2023-10-11 20:42:33,450][71635] Updated weights for policy 1, policy_version 41012 (0.0007) [2023-10-11 20:42:33,819][71635] Updated weights for policy 1, policy_version 41022 (0.0008) [2023-10-11 20:42:35,951][71601] Updated weights for policy 0, policy_version 41030 (0.0007) [2023-10-11 20:42:36,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 84017152. Throughput: 0: 1822.4, 1: 1825.4. Samples: 21014086. Policy #0 lag: (min: 1.0, avg: 14.7, max: 33.0) [2023-10-11 20:42:36,034][70582] Avg episode reward: [(0, '232.180'), (1, '148.860')] [2023-10-11 20:42:36,316][71601] Updated weights for policy 0, policy_version 41040 (0.0009) [2023-10-11 20:42:36,683][71601] Updated weights for policy 0, policy_version 41050 (0.0008) [2023-10-11 20:42:37,493][71635] Updated weights for policy 1, policy_version 41032 (0.0008) [2023-10-11 20:42:37,854][71635] Updated weights for policy 1, policy_version 41042 (0.0008) [2023-10-11 20:42:38,223][71635] Updated weights for policy 1, policy_version 41052 (0.0010) [2023-10-11 20:42:40,366][71601] Updated weights for policy 0, policy_version 41060 (0.0008) [2023-10-11 20:42:40,736][71601] Updated weights for policy 0, policy_version 41070 (0.0009) [2023-10-11 20:42:41,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 84082688. Throughput: 0: 1824.5, 1: 1824.6. Samples: 21036660. Policy #0 lag: (min: 1.0, avg: 14.7, max: 33.0) [2023-10-11 20:42:41,035][70582] Avg episode reward: [(0, '232.180'), (1, '133.640')] [2023-10-11 20:42:41,113][71601] Updated weights for policy 0, policy_version 41080 (0.0009) [2023-10-11 20:42:42,046][71635] Updated weights for policy 1, policy_version 41062 (0.0008) [2023-10-11 20:42:42,415][71635] Updated weights for policy 1, policy_version 41072 (0.0008) [2023-10-11 20:42:42,779][71635] Updated weights for policy 1, policy_version 41082 (0.0008) [2023-10-11 20:42:44,778][71601] Updated weights for policy 0, policy_version 41090 (0.0008) [2023-10-11 20:42:45,151][71601] Updated weights for policy 0, policy_version 41100 (0.0009) [2023-10-11 20:42:45,527][71601] Updated weights for policy 0, policy_version 41110 (0.0008) [2023-10-11 20:42:45,908][71601] Updated weights for policy 0, policy_version 41120 (0.0007) [2023-10-11 20:42:46,034][70582] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 84180992. Throughput: 0: 1821.9, 1: 1822.8. Samples: 21046836. Policy #0 lag: (min: 28.0, avg: 36.0, max: 60.0) [2023-10-11 20:42:46,035][70582] Avg episode reward: [(0, '232.180'), (1, '133.180')] [2023-10-11 20:42:46,369][71635] Updated weights for policy 1, policy_version 41092 (0.0009) [2023-10-11 20:42:46,730][71635] Updated weights for policy 1, policy_version 41102 (0.0008) [2023-10-11 20:42:47,097][71635] Updated weights for policy 1, policy_version 41112 (0.0008) [2023-10-11 20:42:49,575][71601] Updated weights for policy 0, policy_version 41130 (0.0009) [2023-10-11 20:42:49,947][71601] Updated weights for policy 0, policy_version 41140 (0.0008) [2023-10-11 20:42:50,311][71601] Updated weights for policy 0, policy_version 41150 (0.0010) [2023-10-11 20:42:50,880][71635] Updated weights for policy 1, policy_version 41122 (0.0009) [2023-10-11 20:42:51,034][70582] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 84246528. Throughput: 0: 1827.7, 1: 1818.7. Samples: 21069516. Policy #0 lag: (min: 28.0, avg: 36.0, max: 60.0) [2023-10-11 20:42:51,034][70582] Avg episode reward: [(0, '232.180'), (1, '119.040')] [2023-10-11 20:42:51,303][71635] Updated weights for policy 1, policy_version 41132 (0.0008) [2023-10-11 20:42:51,653][71635] Updated weights for policy 1, policy_version 41142 (0.0009) [2023-10-11 20:42:52,015][71635] Updated weights for policy 1, policy_version 41152 (0.0007) [2023-10-11 20:42:53,879][71601] Updated weights for policy 0, policy_version 41160 (0.0009) [2023-10-11 20:42:54,249][71601] Updated weights for policy 0, policy_version 41170 (0.0010) [2023-10-11 20:42:54,613][71601] Updated weights for policy 0, policy_version 41180 (0.0008) [2023-10-11 20:42:55,542][71635] Updated weights for policy 1, policy_version 41162 (0.0008) [2023-10-11 20:42:55,905][71635] Updated weights for policy 1, policy_version 41172 (0.0011) [2023-10-11 20:42:56,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 84312064. Throughput: 0: 1840.4, 1: 1814.6. Samples: 21091072. Policy #0 lag: (min: 28.0, avg: 36.0, max: 60.0) [2023-10-11 20:42:56,034][70582] Avg episode reward: [(0, '232.180'), (1, '119.040')] [2023-10-11 20:42:56,274][71635] Updated weights for policy 1, policy_version 41182 (0.0012) [2023-10-11 20:42:58,137][71601] Updated weights for policy 0, policy_version 41190 (0.0007) [2023-10-11 20:42:58,510][71601] Updated weights for policy 0, policy_version 41200 (0.0007) [2023-10-11 20:42:58,880][71601] Updated weights for policy 0, policy_version 41210 (0.0009) [2023-10-11 20:42:59,893][71635] Updated weights for policy 1, policy_version 41192 (0.0010) [2023-10-11 20:43:00,249][71635] Updated weights for policy 1, policy_version 41202 (0.0009) [2023-10-11 20:43:00,613][71635] Updated weights for policy 1, policy_version 41212 (0.0007) [2023-10-11 20:43:01,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 84410368. Throughput: 0: 1832.8, 1: 1812.6. Samples: 21102072. Policy #0 lag: (min: 28.0, avg: 36.0, max: 60.0) [2023-10-11 20:43:01,035][70582] Avg episode reward: [(0, '232.180'), (1, '127.120')] [2023-10-11 20:43:02,615][71601] Updated weights for policy 0, policy_version 41220 (0.0008) [2023-10-11 20:43:02,983][71601] Updated weights for policy 0, policy_version 41230 (0.0007) [2023-10-11 20:43:03,358][71601] Updated weights for policy 0, policy_version 41240 (0.0009) [2023-10-11 20:43:04,239][71635] Updated weights for policy 1, policy_version 41222 (0.0009) [2023-10-11 20:43:04,603][71635] Updated weights for policy 1, policy_version 41232 (0.0008) [2023-10-11 20:43:04,968][71635] Updated weights for policy 1, policy_version 41242 (0.0009) [2023-10-11 20:43:06,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 84475904. Throughput: 0: 1843.2, 1: 1813.3. Samples: 21123922. Policy #0 lag: (min: 28.0, avg: 36.0, max: 60.0) [2023-10-11 20:43:06,034][70582] Avg episode reward: [(0, '232.180'), (1, '123.550')] [2023-10-11 20:43:07,099][71601] Updated weights for policy 0, policy_version 41250 (0.0007) [2023-10-11 20:43:07,469][71601] Updated weights for policy 0, policy_version 41260 (0.0008) [2023-10-11 20:43:07,849][71601] Updated weights for policy 0, policy_version 41270 (0.0009) [2023-10-11 20:43:08,228][71601] Updated weights for policy 0, policy_version 41280 (0.0007) [2023-10-11 20:43:08,587][71635] Updated weights for policy 1, policy_version 41252 (0.0008) [2023-10-11 20:43:08,959][71635] Updated weights for policy 1, policy_version 41262 (0.0009) [2023-10-11 20:43:09,335][71635] Updated weights for policy 1, policy_version 41272 (0.0008) [2023-10-11 20:43:11,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 84541440. Throughput: 0: 1837.5, 1: 1819.4. Samples: 21145668. Policy #0 lag: (min: 28.0, avg: 36.0, max: 60.0) [2023-10-11 20:43:11,035][70582] Avg episode reward: [(0, '232.180'), (1, '123.570')] [2023-10-11 20:43:11,811][71601] Updated weights for policy 0, policy_version 41290 (0.0007) [2023-10-11 20:43:12,192][71601] Updated weights for policy 0, policy_version 41300 (0.0007) [2023-10-11 20:43:12,560][71601] Updated weights for policy 0, policy_version 41310 (0.0008) [2023-10-11 20:43:13,096][71635] Updated weights for policy 1, policy_version 41282 (0.0007) [2023-10-11 20:43:13,463][71635] Updated weights for policy 1, policy_version 41292 (0.0008) [2023-10-11 20:43:13,827][71635] Updated weights for policy 1, policy_version 41302 (0.0010) [2023-10-11 20:43:14,193][71635] Updated weights for policy 1, policy_version 41312 (0.0008) [2023-10-11 20:43:16,034][70582] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 84606976. Throughput: 0: 1838.5, 1: 1813.9. Samples: 21156676. Policy #0 lag: (min: 5.0, avg: 5.0, max: 6.0) [2023-10-11 20:43:16,035][70582] Avg episode reward: [(0, '232.180'), (1, '135.400')] [2023-10-11 20:43:16,476][71601] Updated weights for policy 0, policy_version 41320 (0.0007) [2023-10-11 20:43:16,854][71601] Updated weights for policy 0, policy_version 41330 (0.0008) [2023-10-11 20:43:17,217][71601] Updated weights for policy 0, policy_version 41340 (0.0007) [2023-10-11 20:43:17,987][71635] Updated weights for policy 1, policy_version 41322 (0.0009) [2023-10-11 20:43:18,356][71635] Updated weights for policy 1, policy_version 41332 (0.0008) [2023-10-11 20:43:18,715][71635] Updated weights for policy 1, policy_version 41342 (0.0007) [2023-10-11 20:43:20,845][71601] Updated weights for policy 0, policy_version 41350 (0.0008) [2023-10-11 20:43:21,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 84672512. Throughput: 0: 1829.4, 1: 1818.3. Samples: 21178234. Policy #0 lag: (min: 5.0, avg: 5.0, max: 6.0) [2023-10-11 20:43:21,034][70582] Avg episode reward: [(0, '232.180'), (1, '130.780')] [2023-10-11 20:43:21,219][71601] Updated weights for policy 0, policy_version 41360 (0.0010) [2023-10-11 20:43:21,590][71601] Updated weights for policy 0, policy_version 41370 (0.0009) [2023-10-11 20:43:22,390][71635] Updated weights for policy 1, policy_version 41352 (0.0010) [2023-10-11 20:43:22,755][71635] Updated weights for policy 1, policy_version 41362 (0.0008) [2023-10-11 20:43:23,116][71635] Updated weights for policy 1, policy_version 41372 (0.0007) [2023-10-11 20:43:25,273][71601] Updated weights for policy 0, policy_version 41380 (0.0009) [2023-10-11 20:43:25,642][71601] Updated weights for policy 0, policy_version 41390 (0.0009) [2023-10-11 20:43:26,015][71601] Updated weights for policy 0, policy_version 41400 (0.0010) [2023-10-11 20:43:26,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 84738048. Throughput: 0: 1825.4, 1: 1822.4. Samples: 21200814. Policy #0 lag: (min: 5.0, avg: 5.0, max: 6.0) [2023-10-11 20:43:26,035][70582] Avg episode reward: [(0, '253.940'), (1, '146.590')] [2023-10-11 20:43:26,751][71635] Updated weights for policy 1, policy_version 41382 (0.0008) [2023-10-11 20:43:27,129][71635] Updated weights for policy 1, policy_version 41392 (0.0009) [2023-10-11 20:43:27,488][71635] Updated weights for policy 1, policy_version 41402 (0.0010) [2023-10-11 20:43:29,751][71601] Updated weights for policy 0, policy_version 41410 (0.0008) [2023-10-11 20:43:30,128][71601] Updated weights for policy 0, policy_version 41420 (0.0010) [2023-10-11 20:43:30,498][71601] Updated weights for policy 0, policy_version 41430 (0.0007) [2023-10-11 20:43:30,862][71601] Updated weights for policy 0, policy_version 41440 (0.0007) [2023-10-11 20:43:31,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 84836352. Throughput: 0: 1830.0, 1: 1821.9. Samples: 21211170. Policy #0 lag: (min: 5.0, avg: 5.0, max: 6.0) [2023-10-11 20:43:31,035][70582] Avg episode reward: [(0, '292.240'), (1, '135.870')] [2023-10-11 20:43:31,348][71635] Updated weights for policy 1, policy_version 41412 (0.0009) [2023-10-11 20:43:31,711][71635] Updated weights for policy 1, policy_version 41422 (0.0008) [2023-10-11 20:43:32,068][71635] Updated weights for policy 1, policy_version 41432 (0.0008) [2023-10-11 20:43:34,518][71601] Updated weights for policy 0, policy_version 41450 (0.0008) [2023-10-11 20:43:34,892][71601] Updated weights for policy 0, policy_version 41460 (0.0008) [2023-10-11 20:43:35,251][71601] Updated weights for policy 0, policy_version 41470 (0.0009) [2023-10-11 20:43:35,804][71635] Updated weights for policy 1, policy_version 41442 (0.0009) [2023-10-11 20:43:36,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 84901888. Throughput: 0: 1822.5, 1: 1817.3. Samples: 21233308. Policy #0 lag: (min: 5.0, avg: 5.0, max: 6.0) [2023-10-11 20:43:36,034][70582] Avg episode reward: [(0, '283.900'), (1, '136.110')] [2023-10-11 20:43:36,217][71635] Updated weights for policy 1, policy_version 41452 (0.0009) [2023-10-11 20:43:36,589][71635] Updated weights for policy 1, policy_version 41462 (0.0007) [2023-10-11 20:43:36,944][71635] Updated weights for policy 1, policy_version 41472 (0.0007) [2023-10-11 20:43:38,850][71601] Updated weights for policy 0, policy_version 41480 (0.0007) [2023-10-11 20:43:39,224][71601] Updated weights for policy 0, policy_version 41490 (0.0008) [2023-10-11 20:43:39,589][71601] Updated weights for policy 0, policy_version 41500 (0.0010) [2023-10-11 20:43:40,576][71635] Updated weights for policy 1, policy_version 41482 (0.0009) [2023-10-11 20:43:40,942][71635] Updated weights for policy 1, policy_version 41492 (0.0010) [2023-10-11 20:43:41,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 84967424. Throughput: 0: 1823.2, 1: 1819.1. Samples: 21254974. Policy #0 lag: (min: 31.0, avg: 31.3, max: 44.0) [2023-10-11 20:43:41,034][70582] Avg episode reward: [(0, '263.530'), (1, '145.770')] [2023-10-11 20:43:41,314][71635] Updated weights for policy 1, policy_version 41502 (0.0008) [2023-10-11 20:43:43,238][71601] Updated weights for policy 0, policy_version 41510 (0.0009) [2023-10-11 20:43:43,615][71601] Updated weights for policy 0, policy_version 41520 (0.0008) [2023-10-11 20:43:43,983][71601] Updated weights for policy 0, policy_version 41530 (0.0008) [2023-10-11 20:43:44,941][71635] Updated weights for policy 1, policy_version 41512 (0.0009) [2023-10-11 20:43:45,311][71635] Updated weights for policy 1, policy_version 41522 (0.0007) [2023-10-11 20:43:45,685][71635] Updated weights for policy 1, policy_version 41532 (0.0007) [2023-10-11 20:43:46,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 85065728. Throughput: 0: 1825.4, 1: 1822.4. Samples: 21266222. Policy #0 lag: (min: 31.0, avg: 31.3, max: 44.0) [2023-10-11 20:43:46,034][70582] Avg episode reward: [(0, '263.510'), (1, '145.400')] [2023-10-11 20:43:47,615][71601] Updated weights for policy 0, policy_version 41540 (0.0007) [2023-10-11 20:43:47,980][71601] Updated weights for policy 0, policy_version 41550 (0.0008) [2023-10-11 20:43:48,356][71601] Updated weights for policy 0, policy_version 41560 (0.0009) [2023-10-11 20:43:49,414][71635] Updated weights for policy 1, policy_version 41542 (0.0008) [2023-10-11 20:43:49,770][71635] Updated weights for policy 1, policy_version 41552 (0.0011) [2023-10-11 20:43:50,135][71635] Updated weights for policy 1, policy_version 41562 (0.0010) [2023-10-11 20:43:51,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 85131264. Throughput: 0: 1821.6, 1: 1825.6. Samples: 21288046. Policy #0 lag: (min: 31.0, avg: 31.3, max: 44.0) [2023-10-11 20:43:51,035][70582] Avg episode reward: [(0, '263.510'), (1, '145.400')] [2023-10-11 20:43:51,962][71601] Updated weights for policy 0, policy_version 41570 (0.0007) [2023-10-11 20:43:52,328][71601] Updated weights for policy 0, policy_version 41580 (0.0009) [2023-10-11 20:43:52,698][71601] Updated weights for policy 0, policy_version 41590 (0.0007) [2023-10-11 20:43:53,067][71601] Updated weights for policy 0, policy_version 41600 (0.0008) [2023-10-11 20:43:53,786][71635] Updated weights for policy 1, policy_version 41572 (0.0008) [2023-10-11 20:43:54,153][71635] Updated weights for policy 1, policy_version 41582 (0.0007) [2023-10-11 20:43:54,517][71635] Updated weights for policy 1, policy_version 41592 (0.0008) [2023-10-11 20:43:56,034][70582] Fps is (10 sec: 13106.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 85196800. Throughput: 0: 1827.2, 1: 1822.7. Samples: 21309912. Policy #0 lag: (min: 31.0, avg: 31.3, max: 44.0) [2023-10-11 20:43:56,035][70582] Avg episode reward: [(0, '264.710'), (1, '147.810')] [2023-10-11 20:43:56,044][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000041600_42598400.pth... [2023-10-11 20:43:56,045][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000041600_42598400.pth... [2023-10-11 20:43:56,073][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000039904_40861696.pth [2023-10-11 20:43:56,078][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000039904_40861696.pth [2023-10-11 20:43:56,754][71601] Updated weights for policy 0, policy_version 41610 (0.0010) [2023-10-11 20:43:57,123][71601] Updated weights for policy 0, policy_version 41620 (0.0009) [2023-10-11 20:43:57,494][71601] Updated weights for policy 0, policy_version 41630 (0.0008) [2023-10-11 20:43:58,345][71635] Updated weights for policy 1, policy_version 41602 (0.0008) [2023-10-11 20:43:58,715][71635] Updated weights for policy 1, policy_version 41612 (0.0007) [2023-10-11 20:43:59,068][71635] Updated weights for policy 1, policy_version 41622 (0.0008) [2023-10-11 20:43:59,439][71635] Updated weights for policy 1, policy_version 41632 (0.0009) [2023-10-11 20:44:01,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 85262336. Throughput: 0: 1830.7, 1: 1826.2. Samples: 21321236. Policy #0 lag: (min: 31.0, avg: 31.3, max: 44.0) [2023-10-11 20:44:01,034][70582] Avg episode reward: [(0, '264.710'), (1, '150.910')] [2023-10-11 20:44:01,200][71601] Updated weights for policy 0, policy_version 41640 (0.0007) [2023-10-11 20:44:01,568][71601] Updated weights for policy 0, policy_version 41650 (0.0009) [2023-10-11 20:44:01,945][71601] Updated weights for policy 0, policy_version 41660 (0.0009) [2023-10-11 20:44:03,162][71635] Updated weights for policy 1, policy_version 41642 (0.0008) [2023-10-11 20:44:03,529][71635] Updated weights for policy 1, policy_version 41652 (0.0008) [2023-10-11 20:44:03,888][71635] Updated weights for policy 1, policy_version 41662 (0.0007) [2023-10-11 20:44:05,399][71601] Updated weights for policy 0, policy_version 41670 (0.0007) [2023-10-11 20:44:05,776][71601] Updated weights for policy 0, policy_version 41680 (0.0008) [2023-10-11 20:44:06,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 85327872. Throughput: 0: 1832.7, 1: 1821.8. Samples: 21342684. Policy #0 lag: (min: 31.0, avg: 31.3, max: 44.0) [2023-10-11 20:44:06,034][70582] Avg episode reward: [(0, '272.070'), (1, '149.200')] [2023-10-11 20:44:06,147][71601] Updated weights for policy 0, policy_version 41690 (0.0008) [2023-10-11 20:44:07,585][71635] Updated weights for policy 1, policy_version 41672 (0.0010) [2023-10-11 20:44:07,938][71635] Updated weights for policy 1, policy_version 41682 (0.0009) [2023-10-11 20:44:08,311][71635] Updated weights for policy 1, policy_version 41692 (0.0011) [2023-10-11 20:44:09,847][71601] Updated weights for policy 0, policy_version 41700 (0.0009) [2023-10-11 20:44:10,208][71601] Updated weights for policy 0, policy_version 41710 (0.0007) [2023-10-11 20:44:10,588][71601] Updated weights for policy 0, policy_version 41720 (0.0010) [2023-10-11 20:44:11,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 85426176. Throughput: 0: 1826.7, 1: 1819.0. Samples: 21364870. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:44:11,035][70582] Avg episode reward: [(0, '271.980'), (1, '149.240')] [2023-10-11 20:44:11,987][71635] Updated weights for policy 1, policy_version 41702 (0.0007) [2023-10-11 20:44:12,359][71635] Updated weights for policy 1, policy_version 41712 (0.0008) [2023-10-11 20:44:12,738][71635] Updated weights for policy 1, policy_version 41722 (0.0007) [2023-10-11 20:44:14,305][71601] Updated weights for policy 0, policy_version 41730 (0.0008) [2023-10-11 20:44:14,675][71601] Updated weights for policy 0, policy_version 41740 (0.0007) [2023-10-11 20:44:15,048][71601] Updated weights for policy 0, policy_version 41750 (0.0007) [2023-10-11 20:44:15,412][71601] Updated weights for policy 0, policy_version 41760 (0.0007) [2023-10-11 20:44:16,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 85491712. Throughput: 0: 1841.4, 1: 1819.6. Samples: 21375916. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:44:16,034][70582] Avg episode reward: [(0, '272.100'), (1, '149.350')] [2023-10-11 20:44:16,447][71635] Updated weights for policy 1, policy_version 41732 (0.0008) [2023-10-11 20:44:16,807][71635] Updated weights for policy 1, policy_version 41742 (0.0011) [2023-10-11 20:44:17,175][71635] Updated weights for policy 1, policy_version 41752 (0.0010) [2023-10-11 20:44:18,924][71601] Updated weights for policy 0, policy_version 41770 (0.0009) [2023-10-11 20:44:19,296][71601] Updated weights for policy 0, policy_version 41780 (0.0009) [2023-10-11 20:44:19,681][71601] Updated weights for policy 0, policy_version 41790 (0.0009) [2023-10-11 20:44:20,814][71635] Updated weights for policy 1, policy_version 41762 (0.0009) [2023-10-11 20:44:21,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 85557248. Throughput: 0: 1829.9, 1: 1832.4. Samples: 21398116. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:44:21,035][70582] Avg episode reward: [(0, '286.020'), (1, '149.350')] [2023-10-11 20:44:21,248][71635] Updated weights for policy 1, policy_version 41772 (0.0007) [2023-10-11 20:44:21,610][71635] Updated weights for policy 1, policy_version 41782 (0.0011) [2023-10-11 20:44:21,979][71635] Updated weights for policy 1, policy_version 41792 (0.0007) [2023-10-11 20:44:23,328][71601] Updated weights for policy 0, policy_version 41800 (0.0007) [2023-10-11 20:44:23,691][71601] Updated weights for policy 0, policy_version 41810 (0.0008) [2023-10-11 20:44:24,065][71601] Updated weights for policy 0, policy_version 41820 (0.0010) [2023-10-11 20:44:25,557][71635] Updated weights for policy 1, policy_version 41802 (0.0011) [2023-10-11 20:44:25,920][71635] Updated weights for policy 1, policy_version 41812 (0.0007) [2023-10-11 20:44:26,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 85622784. Throughput: 0: 1843.1, 1: 1832.2. Samples: 21420364. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:44:26,034][70582] Avg episode reward: [(0, '271.400'), (1, '158.890')] [2023-10-11 20:44:26,294][71635] Updated weights for policy 1, policy_version 41822 (0.0007) [2023-10-11 20:44:27,598][71601] Updated weights for policy 0, policy_version 41830 (0.0009) [2023-10-11 20:44:27,966][71601] Updated weights for policy 0, policy_version 41840 (0.0009) [2023-10-11 20:44:28,337][71601] Updated weights for policy 0, policy_version 41850 (0.0008) [2023-10-11 20:44:30,069][71635] Updated weights for policy 1, policy_version 41832 (0.0008) [2023-10-11 20:44:30,430][71635] Updated weights for policy 1, policy_version 41842 (0.0009) [2023-10-11 20:44:30,802][71635] Updated weights for policy 1, policy_version 41852 (0.0008) [2023-10-11 20:44:31,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 85721088. Throughput: 0: 1829.1, 1: 1830.2. Samples: 21430892. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:44:31,035][70582] Avg episode reward: [(0, '271.400'), (1, '164.450')] [2023-10-11 20:44:32,094][71601] Updated weights for policy 0, policy_version 41860 (0.0007) [2023-10-11 20:44:32,459][71601] Updated weights for policy 0, policy_version 41870 (0.0007) [2023-10-11 20:44:32,838][71601] Updated weights for policy 0, policy_version 41880 (0.0007) [2023-10-11 20:44:34,453][71635] Updated weights for policy 1, policy_version 41862 (0.0009) [2023-10-11 20:44:34,820][71635] Updated weights for policy 1, policy_version 41872 (0.0009) [2023-10-11 20:44:35,188][71635] Updated weights for policy 1, policy_version 41882 (0.0007) [2023-10-11 20:44:36,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 85786624. Throughput: 0: 1844.1, 1: 1825.7. Samples: 21453188. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:44:36,035][70582] Avg episode reward: [(0, '271.400'), (1, '166.750')] [2023-10-11 20:44:36,551][71601] Updated weights for policy 0, policy_version 41890 (0.0008) [2023-10-11 20:44:36,918][71601] Updated weights for policy 0, policy_version 41900 (0.0009) [2023-10-11 20:44:37,287][71601] Updated weights for policy 0, policy_version 41910 (0.0009) [2023-10-11 20:44:37,658][71601] Updated weights for policy 0, policy_version 41920 (0.0007) [2023-10-11 20:44:38,787][71635] Updated weights for policy 1, policy_version 41892 (0.0007) [2023-10-11 20:44:39,158][71635] Updated weights for policy 1, policy_version 41902 (0.0010) [2023-10-11 20:44:39,533][71635] Updated weights for policy 1, policy_version 41912 (0.0008) [2023-10-11 20:44:41,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 85852160. Throughput: 0: 1840.5, 1: 1821.8. Samples: 21474716. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:44:41,034][70582] Avg episode reward: [(0, '278.890'), (1, '171.130')] [2023-10-11 20:44:41,407][71601] Updated weights for policy 0, policy_version 41930 (0.0009) [2023-10-11 20:44:41,780][71601] Updated weights for policy 0, policy_version 41940 (0.0009) [2023-10-11 20:44:42,159][71601] Updated weights for policy 0, policy_version 41950 (0.0008) [2023-10-11 20:44:43,083][71635] Updated weights for policy 1, policy_version 41922 (0.0007) [2023-10-11 20:44:43,458][71635] Updated weights for policy 1, policy_version 41932 (0.0008) [2023-10-11 20:44:43,821][71635] Updated weights for policy 1, policy_version 41942 (0.0009) [2023-10-11 20:44:44,188][71635] Updated weights for policy 1, policy_version 41952 (0.0008) [2023-10-11 20:44:45,814][71601] Updated weights for policy 0, policy_version 41960 (0.0011) [2023-10-11 20:44:46,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 85917696. Throughput: 0: 1834.4, 1: 1819.8. Samples: 21485674. Policy #0 lag: (min: 9.0, avg: 18.8, max: 41.0) [2023-10-11 20:44:46,034][70582] Avg episode reward: [(0, '277.510'), (1, '180.580')] [2023-10-11 20:44:46,187][71601] Updated weights for policy 0, policy_version 41970 (0.0008) [2023-10-11 20:44:46,554][71601] Updated weights for policy 0, policy_version 41980 (0.0009) [2023-10-11 20:44:47,782][71635] Updated weights for policy 1, policy_version 41962 (0.0010) [2023-10-11 20:44:48,152][71635] Updated weights for policy 1, policy_version 41972 (0.0008) [2023-10-11 20:44:48,506][71635] Updated weights for policy 1, policy_version 41982 (0.0009) [2023-10-11 20:44:50,391][71601] Updated weights for policy 0, policy_version 41990 (0.0009) [2023-10-11 20:44:50,775][71601] Updated weights for policy 0, policy_version 42000 (0.0009) [2023-10-11 20:44:51,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 85983232. Throughput: 0: 1831.2, 1: 1828.7. Samples: 21507380. Policy #0 lag: (min: 9.0, avg: 18.8, max: 41.0) [2023-10-11 20:44:51,034][70582] Avg episode reward: [(0, '292.490'), (1, '168.780')] [2023-10-11 20:44:51,154][71601] Updated weights for policy 0, policy_version 42010 (0.0010) [2023-10-11 20:44:52,199][71635] Updated weights for policy 1, policy_version 41992 (0.0009) [2023-10-11 20:44:52,551][71635] Updated weights for policy 1, policy_version 42002 (0.0011) [2023-10-11 20:44:52,926][71635] Updated weights for policy 1, policy_version 42012 (0.0011) [2023-10-11 20:44:54,679][71601] Updated weights for policy 0, policy_version 42020 (0.0008) [2023-10-11 20:44:55,049][71601] Updated weights for policy 0, policy_version 42030 (0.0008) [2023-10-11 20:44:55,419][71601] Updated weights for policy 0, policy_version 42040 (0.0007) [2023-10-11 20:44:56,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 86081536. Throughput: 0: 1821.2, 1: 1830.0. Samples: 21529176. Policy #0 lag: (min: 9.0, avg: 18.8, max: 41.0) [2023-10-11 20:44:56,034][70582] Avg episode reward: [(0, '294.880'), (1, '158.610')] [2023-10-11 20:44:56,663][71635] Updated weights for policy 1, policy_version 42022 (0.0009) [2023-10-11 20:44:57,033][71635] Updated weights for policy 1, policy_version 42032 (0.0008) [2023-10-11 20:44:57,410][71635] Updated weights for policy 1, policy_version 42042 (0.0008) [2023-10-11 20:44:58,975][71601] Updated weights for policy 0, policy_version 42050 (0.0010) [2023-10-11 20:44:59,358][71601] Updated weights for policy 0, policy_version 42060 (0.0010) [2023-10-11 20:44:59,727][71601] Updated weights for policy 0, policy_version 42070 (0.0010) [2023-10-11 20:45:00,098][71601] Updated weights for policy 0, policy_version 42080 (0.0009) [2023-10-11 20:45:01,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 86147072. Throughput: 0: 1828.4, 1: 1829.0. Samples: 21540498. Policy #0 lag: (min: 9.0, avg: 18.8, max: 41.0) [2023-10-11 20:45:01,034][70582] Avg episode reward: [(0, '294.880'), (1, '153.380')] [2023-10-11 20:45:01,207][71635] Updated weights for policy 1, policy_version 42052 (0.0008) [2023-10-11 20:45:01,577][71635] Updated weights for policy 1, policy_version 42062 (0.0008) [2023-10-11 20:45:01,938][71635] Updated weights for policy 1, policy_version 42072 (0.0008) [2023-10-11 20:45:03,737][71601] Updated weights for policy 0, policy_version 42090 (0.0007) [2023-10-11 20:45:04,103][71601] Updated weights for policy 0, policy_version 42100 (0.0007) [2023-10-11 20:45:04,475][71601] Updated weights for policy 0, policy_version 42110 (0.0007) [2023-10-11 20:45:05,616][71635] Updated weights for policy 1, policy_version 42082 (0.0008) [2023-10-11 20:45:05,988][71635] Updated weights for policy 1, policy_version 42092 (0.0010) [2023-10-11 20:45:06,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 86212608. Throughput: 0: 1819.1, 1: 1824.0. Samples: 21562056. Policy #0 lag: (min: 9.0, avg: 18.8, max: 41.0) [2023-10-11 20:45:06,034][70582] Avg episode reward: [(0, '291.840'), (1, '152.280')] [2023-10-11 20:45:06,349][71635] Updated weights for policy 1, policy_version 42102 (0.0010) [2023-10-11 20:45:06,715][71635] Updated weights for policy 1, policy_version 42112 (0.0011) [2023-10-11 20:45:08,315][71601] Updated weights for policy 0, policy_version 42120 (0.0007) [2023-10-11 20:45:08,684][71601] Updated weights for policy 0, policy_version 42130 (0.0009) [2023-10-11 20:45:09,058][71601] Updated weights for policy 0, policy_version 42140 (0.0009) [2023-10-11 20:45:10,444][71635] Updated weights for policy 1, policy_version 42122 (0.0007) [2023-10-11 20:45:10,806][71635] Updated weights for policy 1, policy_version 42132 (0.0007) [2023-10-11 20:45:11,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 86278144. Throughput: 0: 1818.5, 1: 1820.0. Samples: 21584094. Policy #0 lag: (min: 16.0, avg: 40.8, max: 48.0) [2023-10-11 20:45:11,035][70582] Avg episode reward: [(0, '290.270'), (1, '152.280')] [2023-10-11 20:45:11,178][71635] Updated weights for policy 1, policy_version 42142 (0.0008) [2023-10-11 20:45:12,795][71601] Updated weights for policy 0, policy_version 42150 (0.0008) [2023-10-11 20:45:13,155][71601] Updated weights for policy 0, policy_version 42160 (0.0007) [2023-10-11 20:45:13,530][71601] Updated weights for policy 0, policy_version 42170 (0.0007) [2023-10-11 20:45:14,760][71635] Updated weights for policy 1, policy_version 42152 (0.0007) [2023-10-11 20:45:15,138][71635] Updated weights for policy 1, policy_version 42162 (0.0009) [2023-10-11 20:45:15,491][71635] Updated weights for policy 1, policy_version 42172 (0.0010) [2023-10-11 20:45:16,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 86376448. Throughput: 0: 1822.3, 1: 1825.5. Samples: 21595042. Policy #0 lag: (min: 16.0, avg: 40.8, max: 48.0) [2023-10-11 20:45:16,034][70582] Avg episode reward: [(0, '299.020'), (1, '167.310')] [2023-10-11 20:45:17,086][71601] Updated weights for policy 0, policy_version 42180 (0.0008) [2023-10-11 20:45:17,461][71601] Updated weights for policy 0, policy_version 42190 (0.0007) [2023-10-11 20:45:17,837][71601] Updated weights for policy 0, policy_version 42200 (0.0009) [2023-10-11 20:45:19,229][71635] Updated weights for policy 1, policy_version 42182 (0.0009) [2023-10-11 20:45:19,586][71635] Updated weights for policy 1, policy_version 42192 (0.0008) [2023-10-11 20:45:19,948][71635] Updated weights for policy 1, policy_version 42202 (0.0008) [2023-10-11 20:45:21,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 86441984. Throughput: 0: 1820.7, 1: 1821.4. Samples: 21617082. Policy #0 lag: (min: 16.0, avg: 40.8, max: 48.0) [2023-10-11 20:45:21,035][70582] Avg episode reward: [(0, '298.960'), (1, '181.010')] [2023-10-11 20:45:21,440][71601] Updated weights for policy 0, policy_version 42210 (0.0007) [2023-10-11 20:45:21,811][71601] Updated weights for policy 0, policy_version 42220 (0.0008) [2023-10-11 20:45:22,169][71601] Updated weights for policy 0, policy_version 42230 (0.0009) [2023-10-11 20:45:22,546][71601] Updated weights for policy 0, policy_version 42240 (0.0010) [2023-10-11 20:45:23,614][71635] Updated weights for policy 1, policy_version 42212 (0.0008) [2023-10-11 20:45:23,989][71635] Updated weights for policy 1, policy_version 42222 (0.0010) [2023-10-11 20:45:24,357][71635] Updated weights for policy 1, policy_version 42232 (0.0010) [2023-10-11 20:45:26,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 86507520. Throughput: 0: 1823.4, 1: 1824.6. Samples: 21638874. Policy #0 lag: (min: 16.0, avg: 40.8, max: 48.0) [2023-10-11 20:45:26,034][70582] Avg episode reward: [(0, '300.330'), (1, '181.570')] [2023-10-11 20:45:26,299][71601] Updated weights for policy 0, policy_version 42250 (0.0007) [2023-10-11 20:45:26,675][71601] Updated weights for policy 0, policy_version 42260 (0.0008) [2023-10-11 20:45:27,045][71601] Updated weights for policy 0, policy_version 42270 (0.0007) [2023-10-11 20:45:27,996][71635] Updated weights for policy 1, policy_version 42242 (0.0010) [2023-10-11 20:45:28,374][71635] Updated weights for policy 1, policy_version 42252 (0.0008) [2023-10-11 20:45:28,731][71635] Updated weights for policy 1, policy_version 42262 (0.0009) [2023-10-11 20:45:29,102][71635] Updated weights for policy 1, policy_version 42272 (0.0010) [2023-10-11 20:45:30,493][71601] Updated weights for policy 0, policy_version 42280 (0.0008) [2023-10-11 20:45:30,855][71601] Updated weights for policy 0, policy_version 42290 (0.0009) [2023-10-11 20:45:31,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 86573056. Throughput: 0: 1824.5, 1: 1820.8. Samples: 21649712. Policy #0 lag: (min: 16.0, avg: 40.8, max: 48.0) [2023-10-11 20:45:31,035][70582] Avg episode reward: [(0, '300.360'), (1, '186.630')] [2023-10-11 20:45:31,238][71601] Updated weights for policy 0, policy_version 42300 (0.0010) [2023-10-11 20:45:32,759][71635] Updated weights for policy 1, policy_version 42282 (0.0007) [2023-10-11 20:45:33,116][71635] Updated weights for policy 1, policy_version 42292 (0.0007) [2023-10-11 20:45:33,488][71635] Updated weights for policy 1, policy_version 42302 (0.0007) [2023-10-11 20:45:35,041][71601] Updated weights for policy 0, policy_version 42310 (0.0010) [2023-10-11 20:45:35,428][71601] Updated weights for policy 0, policy_version 42320 (0.0008) [2023-10-11 20:45:35,812][71601] Updated weights for policy 0, policy_version 42330 (0.0009) [2023-10-11 20:45:36,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 86671360. Throughput: 0: 1830.2, 1: 1820.7. Samples: 21671670. Policy #0 lag: (min: 16.0, avg: 40.8, max: 48.0) [2023-10-11 20:45:36,034][70582] Avg episode reward: [(0, '307.500'), (1, '186.930')] [2023-10-11 20:45:37,117][71635] Updated weights for policy 1, policy_version 42312 (0.0008) [2023-10-11 20:45:37,476][71635] Updated weights for policy 1, policy_version 42322 (0.0007) [2023-10-11 20:45:37,841][71635] Updated weights for policy 1, policy_version 42332 (0.0009) [2023-10-11 20:45:39,391][71601] Updated weights for policy 0, policy_version 42340 (0.0010) [2023-10-11 20:45:39,762][71601] Updated weights for policy 0, policy_version 42350 (0.0009) [2023-10-11 20:45:40,133][71601] Updated weights for policy 0, policy_version 42360 (0.0008) [2023-10-11 20:45:41,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 86736896. Throughput: 0: 1822.0, 1: 1820.4. Samples: 21693080. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:45:41,035][70582] Avg episode reward: [(0, '283.880'), (1, '185.320')] [2023-10-11 20:45:41,582][71635] Updated weights for policy 1, policy_version 42342 (0.0008) [2023-10-11 20:45:41,951][71635] Updated weights for policy 1, policy_version 42352 (0.0007) [2023-10-11 20:45:42,321][71635] Updated weights for policy 1, policy_version 42362 (0.0007) [2023-10-11 20:45:43,850][71601] Updated weights for policy 0, policy_version 42370 (0.0011) [2023-10-11 20:45:44,222][71601] Updated weights for policy 0, policy_version 42380 (0.0010) [2023-10-11 20:45:44,588][71601] Updated weights for policy 0, policy_version 42390 (0.0011) [2023-10-11 20:45:44,965][71601] Updated weights for policy 0, policy_version 42400 (0.0009) [2023-10-11 20:45:46,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 86802432. Throughput: 0: 1824.8, 1: 1819.9. Samples: 21704508. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:45:46,034][70582] Avg episode reward: [(0, '283.880'), (1, '190.220')] [2023-10-11 20:45:46,147][71635] Updated weights for policy 1, policy_version 42372 (0.0008) [2023-10-11 20:45:46,506][71635] Updated weights for policy 1, policy_version 42382 (0.0009) [2023-10-11 20:45:46,869][71635] Updated weights for policy 1, policy_version 42392 (0.0008) [2023-10-11 20:45:48,789][71601] Updated weights for policy 0, policy_version 42410 (0.0008) [2023-10-11 20:45:49,159][71601] Updated weights for policy 0, policy_version 42420 (0.0009) [2023-10-11 20:45:49,520][71601] Updated weights for policy 0, policy_version 42430 (0.0010) [2023-10-11 20:45:50,575][71635] Updated weights for policy 1, policy_version 42402 (0.0007) [2023-10-11 20:45:50,948][71635] Updated weights for policy 1, policy_version 42412 (0.0007) [2023-10-11 20:45:51,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 86867968. Throughput: 0: 1823.8, 1: 1821.7. Samples: 21726104. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:45:51,034][70582] Avg episode reward: [(0, '310.580'), (1, '189.960')] [2023-10-11 20:45:51,311][71635] Updated weights for policy 1, policy_version 42422 (0.0009) [2023-10-11 20:45:51,675][71635] Updated weights for policy 1, policy_version 42432 (0.0010) [2023-10-11 20:45:53,237][71601] Updated weights for policy 0, policy_version 42440 (0.0009) [2023-10-11 20:45:53,606][71601] Updated weights for policy 0, policy_version 42450 (0.0007) [2023-10-11 20:45:53,965][71601] Updated weights for policy 0, policy_version 42460 (0.0007) [2023-10-11 20:45:55,386][71635] Updated weights for policy 1, policy_version 42442 (0.0009) [2023-10-11 20:45:55,746][71635] Updated weights for policy 1, policy_version 42452 (0.0010) [2023-10-11 20:45:56,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 86933504. Throughput: 0: 1828.6, 1: 1817.6. Samples: 21748172. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:45:56,034][70582] Avg episode reward: [(0, '309.560'), (1, '190.080')] [2023-10-11 20:45:56,043][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000042464_43483136.pth... [2023-10-11 20:45:56,076][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000040768_41746432.pth [2023-10-11 20:45:56,080][71353] Saving a milestone ./train_atari/atari_gopher_APPO/checkpoint_p0/milestones/checkpoint_000042464_43483136.pth [2023-10-11 20:45:56,120][71635] Updated weights for policy 1, policy_version 42462 (0.0010) [2023-10-11 20:45:56,186][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000042464_43483136.pth... [2023-10-11 20:45:56,225][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000040736_41713664.pth [2023-10-11 20:45:56,230][71431] Saving a milestone ./train_atari/atari_gopher_APPO/checkpoint_p1/milestones/checkpoint_000042464_43483136.pth [2023-10-11 20:45:57,656][71601] Updated weights for policy 0, policy_version 42470 (0.0007) [2023-10-11 20:45:58,020][71601] Updated weights for policy 0, policy_version 42480 (0.0009) [2023-10-11 20:45:58,401][71601] Updated weights for policy 0, policy_version 42490 (0.0008) [2023-10-11 20:45:59,764][71635] Updated weights for policy 1, policy_version 42472 (0.0007) [2023-10-11 20:46:00,126][71635] Updated weights for policy 1, policy_version 42482 (0.0008) [2023-10-11 20:46:00,503][71635] Updated weights for policy 1, policy_version 42492 (0.0009) [2023-10-11 20:46:01,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 87031808. Throughput: 0: 1821.9, 1: 1819.8. Samples: 21758918. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:46:01,035][70582] Avg episode reward: [(0, '309.590'), (1, '184.370')] [2023-10-11 20:46:01,976][71601] Updated weights for policy 0, policy_version 42500 (0.0008) [2023-10-11 20:46:02,341][71601] Updated weights for policy 0, policy_version 42510 (0.0008) [2023-10-11 20:46:02,703][71601] Updated weights for policy 0, policy_version 42520 (0.0009) [2023-10-11 20:46:04,150][71635] Updated weights for policy 1, policy_version 42502 (0.0009) [2023-10-11 20:46:04,519][71635] Updated weights for policy 1, policy_version 42512 (0.0009) [2023-10-11 20:46:04,874][71635] Updated weights for policy 1, policy_version 42522 (0.0009) [2023-10-11 20:46:06,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 87097344. Throughput: 0: 1829.0, 1: 1816.8. Samples: 21781140. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:46:06,034][70582] Avg episode reward: [(0, '309.590'), (1, '187.310')] [2023-10-11 20:46:06,228][71601] Updated weights for policy 0, policy_version 42530 (0.0007) [2023-10-11 20:46:06,590][71601] Updated weights for policy 0, policy_version 42540 (0.0008) [2023-10-11 20:46:06,966][71601] Updated weights for policy 0, policy_version 42550 (0.0009) [2023-10-11 20:46:07,333][71601] Updated weights for policy 0, policy_version 42560 (0.0008) [2023-10-11 20:46:08,544][71635] Updated weights for policy 1, policy_version 42532 (0.0008) [2023-10-11 20:46:08,900][71635] Updated weights for policy 1, policy_version 42542 (0.0009) [2023-10-11 20:46:09,266][71635] Updated weights for policy 1, policy_version 42552 (0.0010) [2023-10-11 20:46:10,919][71601] Updated weights for policy 0, policy_version 42570 (0.0007) [2023-10-11 20:46:11,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 87162880. Throughput: 0: 1834.2, 1: 1819.6. Samples: 21803296. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:46:11,034][70582] Avg episode reward: [(0, '309.590'), (1, '184.000')] [2023-10-11 20:46:11,297][71601] Updated weights for policy 0, policy_version 42580 (0.0007) [2023-10-11 20:46:11,664][71601] Updated weights for policy 0, policy_version 42590 (0.0007) [2023-10-11 20:46:13,070][71635] Updated weights for policy 1, policy_version 42562 (0.0010) [2023-10-11 20:46:13,432][71635] Updated weights for policy 1, policy_version 42572 (0.0009) [2023-10-11 20:46:13,794][71635] Updated weights for policy 1, policy_version 42582 (0.0008) [2023-10-11 20:46:14,159][71635] Updated weights for policy 1, policy_version 42592 (0.0009) [2023-10-11 20:46:15,303][71601] Updated weights for policy 0, policy_version 42600 (0.0008) [2023-10-11 20:46:15,676][71601] Updated weights for policy 0, policy_version 42610 (0.0008) [2023-10-11 20:46:16,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 87228416. Throughput: 0: 1835.1, 1: 1817.3. Samples: 21814070. Policy #0 lag: (min: 12.0, avg: 20.0, max: 44.0) [2023-10-11 20:46:16,034][70582] Avg episode reward: [(0, '336.670'), (1, '175.900')] [2023-10-11 20:46:16,051][71601] Updated weights for policy 0, policy_version 42620 (0.0007) [2023-10-11 20:46:16,198][71353] Saving new best policy, reward=336.670! [2023-10-11 20:46:17,979][71635] Updated weights for policy 1, policy_version 42602 (0.0008) [2023-10-11 20:46:18,342][71635] Updated weights for policy 1, policy_version 42612 (0.0008) [2023-10-11 20:46:18,708][71635] Updated weights for policy 1, policy_version 42622 (0.0008) [2023-10-11 20:46:19,668][71601] Updated weights for policy 0, policy_version 42630 (0.0008) [2023-10-11 20:46:20,040][71601] Updated weights for policy 0, policy_version 42640 (0.0008) [2023-10-11 20:46:20,406][71601] Updated weights for policy 0, policy_version 42650 (0.0008) [2023-10-11 20:46:21,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 87326720. Throughput: 0: 1833.5, 1: 1812.0. Samples: 21835720. Policy #0 lag: (min: 12.0, avg: 20.0, max: 44.0) [2023-10-11 20:46:21,035][70582] Avg episode reward: [(0, '328.180'), (1, '171.920')] [2023-10-11 20:46:22,542][71635] Updated weights for policy 1, policy_version 42632 (0.0009) [2023-10-11 20:46:22,906][71635] Updated weights for policy 1, policy_version 42642 (0.0009) [2023-10-11 20:46:23,275][71635] Updated weights for policy 1, policy_version 42652 (0.0007) [2023-10-11 20:46:24,116][71601] Updated weights for policy 0, policy_version 42660 (0.0008) [2023-10-11 20:46:24,505][71601] Updated weights for policy 0, policy_version 42670 (0.0008) [2023-10-11 20:46:24,877][71601] Updated weights for policy 0, policy_version 42680 (0.0009) [2023-10-11 20:46:26,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 87392256. Throughput: 0: 1830.0, 1: 1809.3. Samples: 21856848. Policy #0 lag: (min: 12.0, avg: 20.0, max: 44.0) [2023-10-11 20:46:26,034][70582] Avg episode reward: [(0, '327.800'), (1, '178.720')] [2023-10-11 20:46:26,984][71635] Updated weights for policy 1, policy_version 42662 (0.0008) [2023-10-11 20:46:27,343][71635] Updated weights for policy 1, policy_version 42672 (0.0008) [2023-10-11 20:46:27,717][71635] Updated weights for policy 1, policy_version 42682 (0.0008) [2023-10-11 20:46:28,580][71601] Updated weights for policy 0, policy_version 42690 (0.0008) [2023-10-11 20:46:28,946][71601] Updated weights for policy 0, policy_version 42700 (0.0010) [2023-10-11 20:46:29,322][71601] Updated weights for policy 0, policy_version 42710 (0.0009) [2023-10-11 20:46:29,688][71601] Updated weights for policy 0, policy_version 42720 (0.0010) [2023-10-11 20:46:31,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 87457792. Throughput: 0: 1828.2, 1: 1807.1. Samples: 21868100. Policy #0 lag: (min: 12.0, avg: 20.0, max: 44.0) [2023-10-11 20:46:31,035][70582] Avg episode reward: [(0, '327.800'), (1, '168.300')] [2023-10-11 20:46:31,475][71635] Updated weights for policy 1, policy_version 42692 (0.0007) [2023-10-11 20:46:31,839][71635] Updated weights for policy 1, policy_version 42702 (0.0008) [2023-10-11 20:46:32,199][71635] Updated weights for policy 1, policy_version 42712 (0.0009) [2023-10-11 20:46:33,383][71601] Updated weights for policy 0, policy_version 42730 (0.0007) [2023-10-11 20:46:33,756][71601] Updated weights for policy 0, policy_version 42740 (0.0007) [2023-10-11 20:46:34,128][71601] Updated weights for policy 0, policy_version 42750 (0.0008) [2023-10-11 20:46:35,801][71635] Updated weights for policy 1, policy_version 42722 (0.0008) [2023-10-11 20:46:36,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 87523328. Throughput: 0: 1826.0, 1: 1807.1. Samples: 21889596. Policy #0 lag: (min: 12.0, avg: 20.0, max: 44.0) [2023-10-11 20:46:36,035][70582] Avg episode reward: [(0, '328.770'), (1, '167.880')] [2023-10-11 20:46:36,157][71635] Updated weights for policy 1, policy_version 42732 (0.0009) [2023-10-11 20:46:36,533][71635] Updated weights for policy 1, policy_version 42742 (0.0009) [2023-10-11 20:46:36,889][71635] Updated weights for policy 1, policy_version 42752 (0.0008) [2023-10-11 20:46:37,817][71601] Updated weights for policy 0, policy_version 42760 (0.0010) [2023-10-11 20:46:38,188][71601] Updated weights for policy 0, policy_version 42770 (0.0008) [2023-10-11 20:46:38,560][71601] Updated weights for policy 0, policy_version 42780 (0.0007) [2023-10-11 20:46:40,524][71635] Updated weights for policy 1, policy_version 42762 (0.0010) [2023-10-11 20:46:40,888][71635] Updated weights for policy 1, policy_version 42772 (0.0012) [2023-10-11 20:46:41,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 87588864. Throughput: 0: 1830.6, 1: 1815.8. Samples: 21912262. Policy #0 lag: (min: 12.0, avg: 20.0, max: 44.0) [2023-10-11 20:46:41,035][70582] Avg episode reward: [(0, '326.980'), (1, '149.700')] [2023-10-11 20:46:41,253][71635] Updated weights for policy 1, policy_version 42782 (0.0010) [2023-10-11 20:46:42,130][71601] Updated weights for policy 0, policy_version 42790 (0.0007) [2023-10-11 20:46:42,500][71601] Updated weights for policy 0, policy_version 42800 (0.0007) [2023-10-11 20:46:42,860][71601] Updated weights for policy 0, policy_version 42810 (0.0008) [2023-10-11 20:46:44,985][71635] Updated weights for policy 1, policy_version 42792 (0.0007) [2023-10-11 20:46:45,351][71635] Updated weights for policy 1, policy_version 42802 (0.0008) [2023-10-11 20:46:45,719][71635] Updated weights for policy 1, policy_version 42812 (0.0008) [2023-10-11 20:46:46,034][70582] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 87687168. Throughput: 0: 1828.5, 1: 1810.6. Samples: 21922676. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-10-11 20:46:46,034][70582] Avg episode reward: [(0, '326.030'), (1, '143.320')] [2023-10-11 20:46:46,509][71601] Updated weights for policy 0, policy_version 42820 (0.0008) [2023-10-11 20:46:46,876][71601] Updated weights for policy 0, policy_version 42830 (0.0009) [2023-10-11 20:46:47,238][71601] Updated weights for policy 0, policy_version 42840 (0.0008) [2023-10-11 20:46:49,403][71635] Updated weights for policy 1, policy_version 42822 (0.0008) [2023-10-11 20:46:49,772][71635] Updated weights for policy 1, policy_version 42832 (0.0010) [2023-10-11 20:46:50,140][71635] Updated weights for policy 1, policy_version 42842 (0.0011) [2023-10-11 20:46:50,905][71601] Updated weights for policy 0, policy_version 42850 (0.0009) [2023-10-11 20:46:51,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 87752704. Throughput: 0: 1832.7, 1: 1818.8. Samples: 21945460. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-10-11 20:46:51,035][70582] Avg episode reward: [(0, '337.550'), (1, '147.430')] [2023-10-11 20:46:51,280][71601] Updated weights for policy 0, policy_version 42860 (0.0009) [2023-10-11 20:46:51,657][71601] Updated weights for policy 0, policy_version 42870 (0.0008) [2023-10-11 20:46:52,024][71353] Saving new best policy, reward=337.550! [2023-10-11 20:46:52,025][71601] Updated weights for policy 0, policy_version 42880 (0.0009) [2023-10-11 20:46:53,837][71635] Updated weights for policy 1, policy_version 42852 (0.0009) [2023-10-11 20:46:54,202][71635] Updated weights for policy 1, policy_version 42862 (0.0008) [2023-10-11 20:46:54,568][71635] Updated weights for policy 1, policy_version 42872 (0.0008) [2023-10-11 20:46:55,829][71601] Updated weights for policy 0, policy_version 42890 (0.0007) [2023-10-11 20:46:56,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 87818240. Throughput: 0: 1827.6, 1: 1812.9. Samples: 21967118. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-10-11 20:46:56,034][70582] Avg episode reward: [(0, '320.660'), (1, '118.550')] [2023-10-11 20:46:56,209][71601] Updated weights for policy 0, policy_version 42900 (0.0007) [2023-10-11 20:46:56,589][71601] Updated weights for policy 0, policy_version 42910 (0.0008) [2023-10-11 20:46:58,157][71635] Updated weights for policy 1, policy_version 42882 (0.0011) [2023-10-11 20:46:58,522][71635] Updated weights for policy 1, policy_version 42892 (0.0007) [2023-10-11 20:46:58,889][71635] Updated weights for policy 1, policy_version 42902 (0.0009) [2023-10-11 20:46:59,244][71635] Updated weights for policy 1, policy_version 42912 (0.0007) [2023-10-11 20:47:00,408][71601] Updated weights for policy 0, policy_version 42920 (0.0007) [2023-10-11 20:47:00,782][71601] Updated weights for policy 0, policy_version 42930 (0.0009) [2023-10-11 20:47:01,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 87883776. Throughput: 0: 1826.6, 1: 1822.7. Samples: 21978288. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-10-11 20:47:01,035][70582] Avg episode reward: [(0, '308.810'), (1, '118.760')] [2023-10-11 20:47:01,169][71601] Updated weights for policy 0, policy_version 42940 (0.0009) [2023-10-11 20:47:02,990][71635] Updated weights for policy 1, policy_version 42922 (0.0010) [2023-10-11 20:47:03,353][71635] Updated weights for policy 1, policy_version 42932 (0.0011) [2023-10-11 20:47:03,720][71635] Updated weights for policy 1, policy_version 42942 (0.0009) [2023-10-11 20:47:04,959][71601] Updated weights for policy 0, policy_version 42950 (0.0007) [2023-10-11 20:47:05,332][71601] Updated weights for policy 0, policy_version 42960 (0.0009) [2023-10-11 20:47:05,710][71601] Updated weights for policy 0, policy_version 42970 (0.0008) [2023-10-11 20:47:06,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 87982080. Throughput: 0: 1824.5, 1: 1821.1. Samples: 21999774. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-10-11 20:47:06,034][70582] Avg episode reward: [(0, '308.570'), (1, '112.360')] [2023-10-11 20:47:07,415][71635] Updated weights for policy 1, policy_version 42952 (0.0009) [2023-10-11 20:47:07,784][71635] Updated weights for policy 1, policy_version 42962 (0.0007) [2023-10-11 20:47:08,151][71635] Updated weights for policy 1, policy_version 42972 (0.0007) [2023-10-11 20:47:09,332][71601] Updated weights for policy 0, policy_version 42980 (0.0008) [2023-10-11 20:47:09,709][71601] Updated weights for policy 0, policy_version 42990 (0.0008) [2023-10-11 20:47:10,086][71601] Updated weights for policy 0, policy_version 43000 (0.0009) [2023-10-11 20:47:11,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 88047616. Throughput: 0: 1828.5, 1: 1825.2. Samples: 22021262. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-10-11 20:47:11,035][70582] Avg episode reward: [(0, '308.590'), (1, '109.370')] [2023-10-11 20:47:11,791][71635] Updated weights for policy 1, policy_version 42982 (0.0008) [2023-10-11 20:47:12,156][71635] Updated weights for policy 1, policy_version 42992 (0.0007) [2023-10-11 20:47:12,523][71635] Updated weights for policy 1, policy_version 43002 (0.0008) [2023-10-11 20:47:13,611][71601] Updated weights for policy 0, policy_version 43010 (0.0009) [2023-10-11 20:47:13,987][71601] Updated weights for policy 0, policy_version 43020 (0.0007) [2023-10-11 20:47:14,353][71601] Updated weights for policy 0, policy_version 43030 (0.0010) [2023-10-11 20:47:14,718][71601] Updated weights for policy 0, policy_version 43040 (0.0009) [2023-10-11 20:47:16,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 88113152. Throughput: 0: 1830.7, 1: 1828.8. Samples: 22032778. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-11 20:47:16,034][70582] Avg episode reward: [(0, '294.550'), (1, '108.940')] [2023-10-11 20:47:16,136][71635] Updated weights for policy 1, policy_version 43012 (0.0008) [2023-10-11 20:47:16,504][71635] Updated weights for policy 1, policy_version 43022 (0.0008) [2023-10-11 20:47:16,864][71635] Updated weights for policy 1, policy_version 43032 (0.0008) [2023-10-11 20:47:18,271][71601] Updated weights for policy 0, policy_version 43050 (0.0010) [2023-10-11 20:47:18,645][71601] Updated weights for policy 0, policy_version 43060 (0.0007) [2023-10-11 20:47:19,014][71601] Updated weights for policy 0, policy_version 43070 (0.0009) [2023-10-11 20:47:20,649][71635] Updated weights for policy 1, policy_version 43042 (0.0009) [2023-10-11 20:47:21,020][71635] Updated weights for policy 1, policy_version 43052 (0.0010) [2023-10-11 20:47:21,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 88178688. Throughput: 0: 1830.2, 1: 1832.8. Samples: 22054428. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-11 20:47:21,034][70582] Avg episode reward: [(0, '295.240'), (1, '112.380')] [2023-10-11 20:47:21,374][71635] Updated weights for policy 1, policy_version 43062 (0.0008) [2023-10-11 20:47:21,737][71635] Updated weights for policy 1, policy_version 43072 (0.0007) [2023-10-11 20:47:22,675][71601] Updated weights for policy 0, policy_version 43080 (0.0009) [2023-10-11 20:47:23,050][71601] Updated weights for policy 0, policy_version 43090 (0.0008) [2023-10-11 20:47:23,416][71601] Updated weights for policy 0, policy_version 43100 (0.0009) [2023-10-11 20:47:25,350][71635] Updated weights for policy 1, policy_version 43082 (0.0008) [2023-10-11 20:47:25,720][71635] Updated weights for policy 1, policy_version 43092 (0.0007) [2023-10-11 20:47:26,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 88244224. Throughput: 0: 1833.1, 1: 1825.6. Samples: 22076902. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-11 20:47:26,035][70582] Avg episode reward: [(0, '302.820'), (1, '108.080')] [2023-10-11 20:47:26,084][71635] Updated weights for policy 1, policy_version 43102 (0.0011) [2023-10-11 20:47:27,106][71601] Updated weights for policy 0, policy_version 43110 (0.0008) [2023-10-11 20:47:27,481][71601] Updated weights for policy 0, policy_version 43120 (0.0010) [2023-10-11 20:47:27,852][71601] Updated weights for policy 0, policy_version 43130 (0.0009) [2023-10-11 20:47:29,825][71635] Updated weights for policy 1, policy_version 43112 (0.0009) [2023-10-11 20:47:30,202][71635] Updated weights for policy 1, policy_version 43122 (0.0007) [2023-10-11 20:47:30,575][71635] Updated weights for policy 1, policy_version 43132 (0.0009) [2023-10-11 20:47:31,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 88342528. Throughput: 0: 1826.4, 1: 1828.8. Samples: 22087162. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-11 20:47:31,034][70582] Avg episode reward: [(0, '280.900'), (1, '109.450')] [2023-10-11 20:47:31,621][71601] Updated weights for policy 0, policy_version 43140 (0.0008) [2023-10-11 20:47:31,997][71601] Updated weights for policy 0, policy_version 43150 (0.0007) [2023-10-11 20:47:32,366][71601] Updated weights for policy 0, policy_version 43160 (0.0008) [2023-10-11 20:47:34,273][71635] Updated weights for policy 1, policy_version 43142 (0.0008) [2023-10-11 20:47:34,643][71635] Updated weights for policy 1, policy_version 43152 (0.0007) [2023-10-11 20:47:34,999][71635] Updated weights for policy 1, policy_version 43162 (0.0008) [2023-10-11 20:47:35,938][71601] Updated weights for policy 0, policy_version 43170 (0.0007) [2023-10-11 20:47:36,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 88408064. Throughput: 0: 1825.7, 1: 1826.2. Samples: 22109796. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-11 20:47:36,034][70582] Avg episode reward: [(0, '264.990'), (1, '109.690')] [2023-10-11 20:47:36,315][71601] Updated weights for policy 0, policy_version 43180 (0.0007) [2023-10-11 20:47:36,682][71601] Updated weights for policy 0, policy_version 43190 (0.0008) [2023-10-11 20:47:37,054][71601] Updated weights for policy 0, policy_version 43200 (0.0007) [2023-10-11 20:47:38,658][71635] Updated weights for policy 1, policy_version 43172 (0.0008) [2023-10-11 20:47:39,028][71635] Updated weights for policy 1, policy_version 43182 (0.0008) [2023-10-11 20:47:39,385][71635] Updated weights for policy 1, policy_version 43192 (0.0008) [2023-10-11 20:47:40,711][71601] Updated weights for policy 0, policy_version 43210 (0.0009) [2023-10-11 20:47:41,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 88473600. Throughput: 0: 1821.9, 1: 1829.1. Samples: 22131416. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-11 20:47:41,035][70582] Avg episode reward: [(0, '249.860'), (1, '102.500')] [2023-10-11 20:47:41,072][71601] Updated weights for policy 0, policy_version 43220 (0.0007) [2023-10-11 20:47:41,458][71601] Updated weights for policy 0, policy_version 43230 (0.0009) [2023-10-11 20:47:43,087][71635] Updated weights for policy 1, policy_version 43202 (0.0007) [2023-10-11 20:47:43,459][71635] Updated weights for policy 1, policy_version 43212 (0.0011) [2023-10-11 20:47:43,817][71635] Updated weights for policy 1, policy_version 43222 (0.0010) [2023-10-11 20:47:44,180][71635] Updated weights for policy 1, policy_version 43232 (0.0008) [2023-10-11 20:47:45,138][71601] Updated weights for policy 0, policy_version 43240 (0.0008) [2023-10-11 20:47:45,505][71601] Updated weights for policy 0, policy_version 43250 (0.0011) [2023-10-11 20:47:45,881][71601] Updated weights for policy 0, policy_version 43260 (0.0007) [2023-10-11 20:47:46,036][70582] Fps is (10 sec: 16379.9, 60 sec: 14745.0, 300 sec: 14662.2). Total num frames: 88571904. Throughput: 0: 1824.3, 1: 1823.3. Samples: 22142436. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:47:46,037][70582] Avg episode reward: [(0, '248.760'), (1, '93.390')] [2023-10-11 20:47:47,921][71635] Updated weights for policy 1, policy_version 43242 (0.0009) [2023-10-11 20:47:48,292][71635] Updated weights for policy 1, policy_version 43252 (0.0010) [2023-10-11 20:47:48,664][71635] Updated weights for policy 1, policy_version 43262 (0.0009) [2023-10-11 20:47:49,462][71601] Updated weights for policy 0, policy_version 43270 (0.0009) [2023-10-11 20:47:49,828][71601] Updated weights for policy 0, policy_version 43280 (0.0011) [2023-10-11 20:47:50,198][71601] Updated weights for policy 0, policy_version 43290 (0.0009) [2023-10-11 20:47:51,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 88637440. Throughput: 0: 1819.5, 1: 1822.6. Samples: 22163668. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:47:51,034][70582] Avg episode reward: [(0, '255.540'), (1, '94.040')] [2023-10-11 20:47:52,407][71635] Updated weights for policy 1, policy_version 43272 (0.0008) [2023-10-11 20:47:52,776][71635] Updated weights for policy 1, policy_version 43282 (0.0008) [2023-10-11 20:47:53,144][71635] Updated weights for policy 1, policy_version 43292 (0.0009) [2023-10-11 20:47:54,030][71601] Updated weights for policy 0, policy_version 43300 (0.0007) [2023-10-11 20:47:54,421][71601] Updated weights for policy 0, policy_version 43310 (0.0009) [2023-10-11 20:47:54,787][71601] Updated weights for policy 0, policy_version 43320 (0.0010) [2023-10-11 20:47:56,034][70582] Fps is (10 sec: 13109.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 88702976. Throughput: 0: 1823.0, 1: 1823.4. Samples: 22185348. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:47:56,035][70582] Avg episode reward: [(0, '241.460'), (1, '94.040')] [2023-10-11 20:47:56,045][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000043296_44335104.pth... [2023-10-11 20:47:56,045][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000043328_44367872.pth... [2023-10-11 20:47:56,074][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000041600_42598400.pth [2023-10-11 20:47:56,084][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000041600_42598400.pth [2023-10-11 20:47:56,807][71635] Updated weights for policy 1, policy_version 43302 (0.0010) [2023-10-11 20:47:57,169][71635] Updated weights for policy 1, policy_version 43312 (0.0007) [2023-10-11 20:47:57,530][71635] Updated weights for policy 1, policy_version 43322 (0.0009) [2023-10-11 20:47:58,432][71601] Updated weights for policy 0, policy_version 43330 (0.0009) [2023-10-11 20:47:58,803][71601] Updated weights for policy 0, policy_version 43340 (0.0007) [2023-10-11 20:47:59,167][71601] Updated weights for policy 0, policy_version 43350 (0.0007) [2023-10-11 20:47:59,537][71601] Updated weights for policy 0, policy_version 43360 (0.0009) [2023-10-11 20:48:01,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 88768512. Throughput: 0: 1820.5, 1: 1823.7. Samples: 22196768. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:48:01,034][70582] Avg episode reward: [(0, '244.200'), (1, '97.720')] [2023-10-11 20:48:01,201][71635] Updated weights for policy 1, policy_version 43332 (0.0007) [2023-10-11 20:48:01,566][71635] Updated weights for policy 1, policy_version 43342 (0.0007) [2023-10-11 20:48:01,931][71635] Updated weights for policy 1, policy_version 43352 (0.0008) [2023-10-11 20:48:03,124][71601] Updated weights for policy 0, policy_version 43370 (0.0008) [2023-10-11 20:48:03,485][71601] Updated weights for policy 0, policy_version 43380 (0.0008) [2023-10-11 20:48:03,863][71601] Updated weights for policy 0, policy_version 43390 (0.0007) [2023-10-11 20:48:05,399][71635] Updated weights for policy 1, policy_version 43362 (0.0008) [2023-10-11 20:48:05,775][71635] Updated weights for policy 1, policy_version 43372 (0.0010) [2023-10-11 20:48:06,034][70582] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 88834048. Throughput: 0: 1823.9, 1: 1823.3. Samples: 22218554. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:48:06,035][70582] Avg episode reward: [(0, '218.580'), (1, '89.270')] [2023-10-11 20:48:06,136][71635] Updated weights for policy 1, policy_version 43382 (0.0009) [2023-10-11 20:48:06,494][71635] Updated weights for policy 1, policy_version 43392 (0.0008) [2023-10-11 20:48:07,639][71601] Updated weights for policy 0, policy_version 43400 (0.0007) [2023-10-11 20:48:08,011][71601] Updated weights for policy 0, policy_version 43410 (0.0007) [2023-10-11 20:48:08,389][71601] Updated weights for policy 0, policy_version 43420 (0.0008) [2023-10-11 20:48:10,222][71635] Updated weights for policy 1, policy_version 43402 (0.0009) [2023-10-11 20:48:10,600][71635] Updated weights for policy 1, policy_version 43412 (0.0009) [2023-10-11 20:48:10,961][71635] Updated weights for policy 1, policy_version 43422 (0.0008) [2023-10-11 20:48:11,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 88932352. Throughput: 0: 1820.4, 1: 1819.4. Samples: 22240692. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:48:11,035][70582] Avg episode reward: [(0, '211.030'), (1, '107.520')] [2023-10-11 20:48:12,031][71601] Updated weights for policy 0, policy_version 43430 (0.0009) [2023-10-11 20:48:12,401][71601] Updated weights for policy 0, policy_version 43440 (0.0008) [2023-10-11 20:48:12,768][71601] Updated weights for policy 0, policy_version 43450 (0.0010) [2023-10-11 20:48:14,552][71635] Updated weights for policy 1, policy_version 43432 (0.0009) [2023-10-11 20:48:14,918][71635] Updated weights for policy 1, policy_version 43442 (0.0008) [2023-10-11 20:48:15,292][71635] Updated weights for policy 1, policy_version 43452 (0.0008) [2023-10-11 20:48:16,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 88997888. Throughput: 0: 1822.6, 1: 1828.3. Samples: 22251450. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:48:16,035][70582] Avg episode reward: [(0, '194.210'), (1, '106.200')] [2023-10-11 20:48:16,423][71601] Updated weights for policy 0, policy_version 43460 (0.0007) [2023-10-11 20:48:16,801][71601] Updated weights for policy 0, policy_version 43470 (0.0008) [2023-10-11 20:48:17,176][71601] Updated weights for policy 0, policy_version 43480 (0.0008) [2023-10-11 20:48:18,916][71635] Updated weights for policy 1, policy_version 43462 (0.0008) [2023-10-11 20:48:19,285][71635] Updated weights for policy 1, policy_version 43472 (0.0008) [2023-10-11 20:48:19,649][71635] Updated weights for policy 1, policy_version 43482 (0.0008) [2023-10-11 20:48:20,894][71601] Updated weights for policy 0, policy_version 43490 (0.0008) [2023-10-11 20:48:21,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 89063424. Throughput: 0: 1825.3, 1: 1818.9. Samples: 22273788. Policy #0 lag: (min: 4.0, avg: 4.1, max: 12.0) [2023-10-11 20:48:21,034][70582] Avg episode reward: [(0, '191.260'), (1, '103.440')] [2023-10-11 20:48:21,263][71601] Updated weights for policy 0, policy_version 43500 (0.0009) [2023-10-11 20:48:21,633][71601] Updated weights for policy 0, policy_version 43510 (0.0009) [2023-10-11 20:48:22,000][71601] Updated weights for policy 0, policy_version 43520 (0.0008) [2023-10-11 20:48:23,418][71635] Updated weights for policy 1, policy_version 43492 (0.0009) [2023-10-11 20:48:23,791][71635] Updated weights for policy 1, policy_version 43502 (0.0009) [2023-10-11 20:48:24,160][71635] Updated weights for policy 1, policy_version 43512 (0.0009) [2023-10-11 20:48:25,733][71601] Updated weights for policy 0, policy_version 43530 (0.0009) [2023-10-11 20:48:26,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 89128960. Throughput: 0: 1820.5, 1: 1825.1. Samples: 22295464. Policy #0 lag: (min: 4.0, avg: 4.1, max: 12.0) [2023-10-11 20:48:26,034][70582] Avg episode reward: [(0, '192.360'), (1, '118.380')] [2023-10-11 20:48:26,120][71601] Updated weights for policy 0, policy_version 43540 (0.0010) [2023-10-11 20:48:26,488][71601] Updated weights for policy 0, policy_version 43550 (0.0008) [2023-10-11 20:48:27,915][71635] Updated weights for policy 1, policy_version 43522 (0.0010) [2023-10-11 20:48:28,280][71635] Updated weights for policy 1, policy_version 43532 (0.0010) [2023-10-11 20:48:28,647][71635] Updated weights for policy 1, policy_version 43542 (0.0009) [2023-10-11 20:48:29,009][71635] Updated weights for policy 1, policy_version 43552 (0.0011) [2023-10-11 20:48:30,055][71601] Updated weights for policy 0, policy_version 43560 (0.0010) [2023-10-11 20:48:30,432][71601] Updated weights for policy 0, policy_version 43570 (0.0009) [2023-10-11 20:48:30,803][71601] Updated weights for policy 0, policy_version 43580 (0.0008) [2023-10-11 20:48:31,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 89227264. Throughput: 0: 1825.3, 1: 1817.0. Samples: 22306332. Policy #0 lag: (min: 4.0, avg: 4.1, max: 12.0) [2023-10-11 20:48:31,035][70582] Avg episode reward: [(0, '193.690'), (1, '115.740')] [2023-10-11 20:48:32,715][71635] Updated weights for policy 1, policy_version 43562 (0.0008) [2023-10-11 20:48:33,084][71635] Updated weights for policy 1, policy_version 43572 (0.0008) [2023-10-11 20:48:33,449][71635] Updated weights for policy 1, policy_version 43582 (0.0009) [2023-10-11 20:48:34,669][71601] Updated weights for policy 0, policy_version 43590 (0.0009) [2023-10-11 20:48:35,047][71601] Updated weights for policy 0, policy_version 43600 (0.0009) [2023-10-11 20:48:35,417][71601] Updated weights for policy 0, policy_version 43610 (0.0009) [2023-10-11 20:48:36,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 89292800. Throughput: 0: 1827.9, 1: 1829.0. Samples: 22328228. Policy #0 lag: (min: 4.0, avg: 4.1, max: 12.0) [2023-10-11 20:48:36,034][70582] Avg episode reward: [(0, '180.210'), (1, '116.410')] [2023-10-11 20:48:37,253][71635] Updated weights for policy 1, policy_version 43592 (0.0009) [2023-10-11 20:48:37,607][71635] Updated weights for policy 1, policy_version 43602 (0.0010) [2023-10-11 20:48:37,985][71635] Updated weights for policy 1, policy_version 43612 (0.0011) [2023-10-11 20:48:39,119][71601] Updated weights for policy 0, policy_version 43620 (0.0011) [2023-10-11 20:48:39,516][71601] Updated weights for policy 0, policy_version 43630 (0.0007) [2023-10-11 20:48:39,881][71601] Updated weights for policy 0, policy_version 43640 (0.0009) [2023-10-11 20:48:41,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 89358336. Throughput: 0: 1821.9, 1: 1822.1. Samples: 22349328. Policy #0 lag: (min: 4.0, avg: 4.1, max: 12.0) [2023-10-11 20:48:41,034][70582] Avg episode reward: [(0, '180.210'), (1, '121.380')] [2023-10-11 20:48:41,627][71635] Updated weights for policy 1, policy_version 43622 (0.0008) [2023-10-11 20:48:41,988][71635] Updated weights for policy 1, policy_version 43632 (0.0007) [2023-10-11 20:48:42,353][71635] Updated weights for policy 1, policy_version 43642 (0.0008) [2023-10-11 20:48:43,444][71601] Updated weights for policy 0, policy_version 43650 (0.0008) [2023-10-11 20:48:43,817][71601] Updated weights for policy 0, policy_version 43660 (0.0009) [2023-10-11 20:48:44,200][71601] Updated weights for policy 0, policy_version 43670 (0.0008) [2023-10-11 20:48:44,568][71601] Updated weights for policy 0, policy_version 43680 (0.0008) [2023-10-11 20:48:46,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14200.0, 300 sec: 14551.2). Total num frames: 89423872. Throughput: 0: 1823.3, 1: 1820.7. Samples: 22360748. Policy #0 lag: (min: 4.0, avg: 4.1, max: 12.0) [2023-10-11 20:48:46,035][70582] Avg episode reward: [(0, '180.210'), (1, '121.550')] [2023-10-11 20:48:46,051][71635] Updated weights for policy 1, policy_version 43652 (0.0008) [2023-10-11 20:48:46,421][71635] Updated weights for policy 1, policy_version 43662 (0.0007) [2023-10-11 20:48:46,793][71635] Updated weights for policy 1, policy_version 43672 (0.0008) [2023-10-11 20:48:48,137][71601] Updated weights for policy 0, policy_version 43690 (0.0008) [2023-10-11 20:48:48,518][71601] Updated weights for policy 0, policy_version 43700 (0.0007) [2023-10-11 20:48:48,885][71601] Updated weights for policy 0, policy_version 43710 (0.0007) [2023-10-11 20:48:50,616][71635] Updated weights for policy 1, policy_version 43682 (0.0008) [2023-10-11 20:48:50,983][71635] Updated weights for policy 1, policy_version 43692 (0.0008) [2023-10-11 20:48:51,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 89489408. Throughput: 0: 1824.9, 1: 1815.3. Samples: 22382364. Policy #0 lag: (min: 18.0, avg: 18.6, max: 34.0) [2023-10-11 20:48:51,035][70582] Avg episode reward: [(0, '180.210'), (1, '131.410')] [2023-10-11 20:48:51,347][71635] Updated weights for policy 1, policy_version 43702 (0.0009) [2023-10-11 20:48:51,715][71635] Updated weights for policy 1, policy_version 43712 (0.0011) [2023-10-11 20:48:52,546][71601] Updated weights for policy 0, policy_version 43720 (0.0009) [2023-10-11 20:48:52,927][71601] Updated weights for policy 0, policy_version 43730 (0.0010) [2023-10-11 20:48:53,287][71601] Updated weights for policy 0, policy_version 43740 (0.0011) [2023-10-11 20:48:55,412][71635] Updated weights for policy 1, policy_version 43722 (0.0010) [2023-10-11 20:48:55,784][71635] Updated weights for policy 1, policy_version 43732 (0.0009) [2023-10-11 20:48:56,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 89554944. Throughput: 0: 1824.6, 1: 1817.5. Samples: 22404586. Policy #0 lag: (min: 18.0, avg: 18.6, max: 34.0) [2023-10-11 20:48:56,035][70582] Avg episode reward: [(0, '180.210'), (1, '132.590')] [2023-10-11 20:48:56,148][71635] Updated weights for policy 1, policy_version 43742 (0.0007) [2023-10-11 20:48:56,984][71601] Updated weights for policy 0, policy_version 43750 (0.0009) [2023-10-11 20:48:57,359][71601] Updated weights for policy 0, policy_version 43760 (0.0007) [2023-10-11 20:48:57,724][71601] Updated weights for policy 0, policy_version 43770 (0.0010) [2023-10-11 20:48:59,833][71635] Updated weights for policy 1, policy_version 43752 (0.0008) [2023-10-11 20:49:00,190][71635] Updated weights for policy 1, policy_version 43762 (0.0009) [2023-10-11 20:49:00,569][71635] Updated weights for policy 1, policy_version 43772 (0.0008) [2023-10-11 20:49:01,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 89653248. Throughput: 0: 1823.3, 1: 1805.7. Samples: 22414756. Policy #0 lag: (min: 18.0, avg: 18.6, max: 34.0) [2023-10-11 20:49:01,035][70582] Avg episode reward: [(0, '180.210'), (1, '136.960')] [2023-10-11 20:49:01,449][71601] Updated weights for policy 0, policy_version 43780 (0.0009) [2023-10-11 20:49:01,820][71601] Updated weights for policy 0, policy_version 43790 (0.0008) [2023-10-11 20:49:02,198][71601] Updated weights for policy 0, policy_version 43800 (0.0007) [2023-10-11 20:49:04,246][71635] Updated weights for policy 1, policy_version 43782 (0.0009) [2023-10-11 20:49:04,610][71635] Updated weights for policy 1, policy_version 43792 (0.0007) [2023-10-11 20:49:04,972][71635] Updated weights for policy 1, policy_version 43802 (0.0008) [2023-10-11 20:49:05,946][71601] Updated weights for policy 0, policy_version 43810 (0.0010) [2023-10-11 20:49:06,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 89718784. Throughput: 0: 1815.4, 1: 1811.5. Samples: 22436998. Policy #0 lag: (min: 18.0, avg: 18.6, max: 34.0) [2023-10-11 20:49:06,035][70582] Avg episode reward: [(0, '180.210'), (1, '143.570')] [2023-10-11 20:49:06,319][71601] Updated weights for policy 0, policy_version 43820 (0.0008) [2023-10-11 20:49:06,682][71601] Updated weights for policy 0, policy_version 43830 (0.0009) [2023-10-11 20:49:07,055][71601] Updated weights for policy 0, policy_version 43840 (0.0009) [2023-10-11 20:49:08,610][71635] Updated weights for policy 1, policy_version 43812 (0.0008) [2023-10-11 20:49:08,973][71635] Updated weights for policy 1, policy_version 43822 (0.0010) [2023-10-11 20:49:09,341][71635] Updated weights for policy 1, policy_version 43832 (0.0007) [2023-10-11 20:49:10,695][71601] Updated weights for policy 0, policy_version 43850 (0.0007) [2023-10-11 20:49:11,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 89784320. Throughput: 0: 1817.0, 1: 1806.5. Samples: 22458522. Policy #0 lag: (min: 18.0, avg: 18.6, max: 34.0) [2023-10-11 20:49:11,034][70582] Avg episode reward: [(0, '180.210'), (1, '141.850')] [2023-10-11 20:49:11,055][71601] Updated weights for policy 0, policy_version 43860 (0.0008) [2023-10-11 20:49:11,428][71601] Updated weights for policy 0, policy_version 43870 (0.0009) [2023-10-11 20:49:13,130][71635] Updated weights for policy 1, policy_version 43842 (0.0009) [2023-10-11 20:49:13,491][71635] Updated weights for policy 1, policy_version 43852 (0.0009) [2023-10-11 20:49:13,854][71635] Updated weights for policy 1, policy_version 43862 (0.0010) [2023-10-11 20:49:14,222][71635] Updated weights for policy 1, policy_version 43872 (0.0008) [2023-10-11 20:49:15,087][71601] Updated weights for policy 0, policy_version 43880 (0.0007) [2023-10-11 20:49:15,459][71601] Updated weights for policy 0, policy_version 43890 (0.0007) [2023-10-11 20:49:15,827][71601] Updated weights for policy 0, policy_version 43900 (0.0007) [2023-10-11 20:49:16,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 89882624. Throughput: 0: 1817.6, 1: 1818.3. Samples: 22469944. Policy #0 lag: (min: 18.0, avg: 18.6, max: 34.0) [2023-10-11 20:49:16,034][70582] Avg episode reward: [(0, '180.210'), (1, '149.270')] [2023-10-11 20:49:17,872][71635] Updated weights for policy 1, policy_version 43882 (0.0007) [2023-10-11 20:49:18,237][71635] Updated weights for policy 1, policy_version 43892 (0.0009) [2023-10-11 20:49:18,614][71635] Updated weights for policy 1, policy_version 43902 (0.0009) [2023-10-11 20:49:19,604][71601] Updated weights for policy 0, policy_version 43910 (0.0007) [2023-10-11 20:49:19,975][71601] Updated weights for policy 0, policy_version 43920 (0.0007) [2023-10-11 20:49:20,346][71601] Updated weights for policy 0, policy_version 43930 (0.0008) [2023-10-11 20:49:21,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 89948160. Throughput: 0: 1819.6, 1: 1810.1. Samples: 22491562. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-10-11 20:49:21,035][70582] Avg episode reward: [(0, '180.210'), (1, '134.240')] [2023-10-11 20:49:22,358][71635] Updated weights for policy 1, policy_version 43912 (0.0008) [2023-10-11 20:49:22,723][71635] Updated weights for policy 1, policy_version 43922 (0.0009) [2023-10-11 20:49:23,087][71635] Updated weights for policy 1, policy_version 43932 (0.0011) [2023-10-11 20:49:24,132][71601] Updated weights for policy 0, policy_version 43940 (0.0010) [2023-10-11 20:49:24,502][71601] Updated weights for policy 0, policy_version 43950 (0.0009) [2023-10-11 20:49:24,870][71601] Updated weights for policy 0, policy_version 43960 (0.0009) [2023-10-11 20:49:26,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 90013696. Throughput: 0: 1815.7, 1: 1818.7. Samples: 22512876. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-10-11 20:49:26,034][70582] Avg episode reward: [(0, '180.210'), (1, '138.690')] [2023-10-11 20:49:26,680][71635] Updated weights for policy 1, policy_version 43942 (0.0009) [2023-10-11 20:49:27,045][71635] Updated weights for policy 1, policy_version 43952 (0.0010) [2023-10-11 20:49:27,413][71635] Updated weights for policy 1, policy_version 43962 (0.0008) [2023-10-11 20:49:28,352][71601] Updated weights for policy 0, policy_version 43970 (0.0007) [2023-10-11 20:49:28,731][71601] Updated weights for policy 0, policy_version 43980 (0.0008) [2023-10-11 20:49:29,100][71601] Updated weights for policy 0, policy_version 43990 (0.0010) [2023-10-11 20:49:29,467][71601] Updated weights for policy 0, policy_version 44000 (0.0009) [2023-10-11 20:49:31,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 90079232. Throughput: 0: 1815.6, 1: 1820.7. Samples: 22524378. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-10-11 20:49:31,034][70582] Avg episode reward: [(0, '201.650'), (1, '142.690')] [2023-10-11 20:49:31,071][71635] Updated weights for policy 1, policy_version 43972 (0.0007) [2023-10-11 20:49:31,433][71635] Updated weights for policy 1, policy_version 43982 (0.0008) [2023-10-11 20:49:31,801][71635] Updated weights for policy 1, policy_version 43992 (0.0007) [2023-10-11 20:49:33,310][71601] Updated weights for policy 0, policy_version 44010 (0.0011) [2023-10-11 20:49:33,669][71601] Updated weights for policy 0, policy_version 44020 (0.0009) [2023-10-11 20:49:34,036][71601] Updated weights for policy 0, policy_version 44030 (0.0008) [2023-10-11 20:49:35,557][71635] Updated weights for policy 1, policy_version 44002 (0.0007) [2023-10-11 20:49:35,917][71635] Updated weights for policy 1, policy_version 44012 (0.0007) [2023-10-11 20:49:36,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 90144768. Throughput: 0: 1813.6, 1: 1823.2. Samples: 22546020. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-10-11 20:49:36,034][70582] Avg episode reward: [(0, '184.290'), (1, '134.800')] [2023-10-11 20:49:36,286][71635] Updated weights for policy 1, policy_version 44022 (0.0008) [2023-10-11 20:49:36,645][71635] Updated weights for policy 1, policy_version 44032 (0.0009) [2023-10-11 20:49:37,839][71601] Updated weights for policy 0, policy_version 44040 (0.0008) [2023-10-11 20:49:38,212][71601] Updated weights for policy 0, policy_version 44050 (0.0008) [2023-10-11 20:49:38,583][71601] Updated weights for policy 0, policy_version 44060 (0.0007) [2023-10-11 20:49:40,319][71635] Updated weights for policy 1, policy_version 44042 (0.0008) [2023-10-11 20:49:40,685][71635] Updated weights for policy 1, policy_version 44052 (0.0008) [2023-10-11 20:49:41,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 90210304. Throughput: 0: 1813.7, 1: 1821.3. Samples: 22568158. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-10-11 20:49:41,034][70582] Avg episode reward: [(0, '198.790'), (1, '129.510')] [2023-10-11 20:49:41,057][71635] Updated weights for policy 1, policy_version 44062 (0.0010) [2023-10-11 20:49:42,263][71601] Updated weights for policy 0, policy_version 44070 (0.0010) [2023-10-11 20:49:42,640][71601] Updated weights for policy 0, policy_version 44080 (0.0008) [2023-10-11 20:49:43,001][71601] Updated weights for policy 0, policy_version 44090 (0.0009) [2023-10-11 20:49:44,727][71635] Updated weights for policy 1, policy_version 44072 (0.0010) [2023-10-11 20:49:45,096][71635] Updated weights for policy 1, policy_version 44082 (0.0008) [2023-10-11 20:49:45,462][71635] Updated weights for policy 1, policy_version 44092 (0.0008) [2023-10-11 20:49:46,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 90308608. Throughput: 0: 1815.7, 1: 1828.1. Samples: 22578728. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-10-11 20:49:46,035][70582] Avg episode reward: [(0, '197.830'), (1, '119.010')] [2023-10-11 20:49:46,620][71601] Updated weights for policy 0, policy_version 44100 (0.0009) [2023-10-11 20:49:47,001][71601] Updated weights for policy 0, policy_version 44110 (0.0008) [2023-10-11 20:49:47,363][71601] Updated weights for policy 0, policy_version 44120 (0.0007) [2023-10-11 20:49:49,196][71635] Updated weights for policy 1, policy_version 44102 (0.0009) [2023-10-11 20:49:49,565][71635] Updated weights for policy 1, policy_version 44112 (0.0009) [2023-10-11 20:49:49,931][71635] Updated weights for policy 1, policy_version 44122 (0.0008) [2023-10-11 20:49:51,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 90374144. Throughput: 0: 1820.0, 1: 1825.1. Samples: 22601026. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-10-11 20:49:51,034][70582] Avg episode reward: [(0, '185.350'), (1, '118.730')] [2023-10-11 20:49:51,089][71601] Updated weights for policy 0, policy_version 44130 (0.0009) [2023-10-11 20:49:51,461][71601] Updated weights for policy 0, policy_version 44140 (0.0008) [2023-10-11 20:49:51,824][71601] Updated weights for policy 0, policy_version 44150 (0.0008) [2023-10-11 20:49:52,199][71601] Updated weights for policy 0, policy_version 44160 (0.0007) [2023-10-11 20:49:53,616][71635] Updated weights for policy 1, policy_version 44132 (0.0007) [2023-10-11 20:49:53,984][71635] Updated weights for policy 1, policy_version 44142 (0.0008) [2023-10-11 20:49:54,349][71635] Updated weights for policy 1, policy_version 44152 (0.0010) [2023-10-11 20:49:55,757][71601] Updated weights for policy 0, policy_version 44170 (0.0008) [2023-10-11 20:49:56,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 90439680. Throughput: 0: 1827.1, 1: 1825.8. Samples: 22622904. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-10-11 20:49:56,035][70582] Avg episode reward: [(0, '180.510'), (1, '117.820')] [2023-10-11 20:49:56,045][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000044160_45219840.pth... [2023-10-11 20:49:56,079][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000042464_43483136.pth [2023-10-11 20:49:56,138][71601] Updated weights for policy 0, policy_version 44180 (0.0011) [2023-10-11 20:49:56,511][71601] Updated weights for policy 0, policy_version 44190 (0.0010) [2023-10-11 20:49:56,577][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000044192_45252608.pth... [2023-10-11 20:49:56,606][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000042464_43483136.pth [2023-10-11 20:49:58,053][71635] Updated weights for policy 1, policy_version 44162 (0.0008) [2023-10-11 20:49:58,409][71635] Updated weights for policy 1, policy_version 44172 (0.0008) [2023-10-11 20:49:58,783][71635] Updated weights for policy 1, policy_version 44182 (0.0008) [2023-10-11 20:49:59,144][71635] Updated weights for policy 1, policy_version 44192 (0.0008) [2023-10-11 20:50:00,169][71601] Updated weights for policy 0, policy_version 44200 (0.0008) [2023-10-11 20:50:00,533][71601] Updated weights for policy 0, policy_version 44210 (0.0010) [2023-10-11 20:50:00,914][71601] Updated weights for policy 0, policy_version 44220 (0.0009) [2023-10-11 20:50:01,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 90505216. Throughput: 0: 1824.9, 1: 1822.7. Samples: 22634084. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-10-11 20:50:01,034][70582] Avg episode reward: [(0, '178.080'), (1, '118.250')] [2023-10-11 20:50:02,825][71635] Updated weights for policy 1, policy_version 44202 (0.0009) [2023-10-11 20:50:03,190][71635] Updated weights for policy 1, policy_version 44212 (0.0010) [2023-10-11 20:50:03,553][71635] Updated weights for policy 1, policy_version 44222 (0.0011) [2023-10-11 20:50:04,598][71601] Updated weights for policy 0, policy_version 44230 (0.0009) [2023-10-11 20:50:04,976][71601] Updated weights for policy 0, policy_version 44240 (0.0009) [2023-10-11 20:50:05,341][71601] Updated weights for policy 0, policy_version 44250 (0.0008) [2023-10-11 20:50:06,034][70582] Fps is (10 sec: 16384.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 90603520. Throughput: 0: 1820.5, 1: 1825.7. Samples: 22655640. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-10-11 20:50:06,034][70582] Avg episode reward: [(0, '182.350'), (1, '131.550')] [2023-10-11 20:50:07,288][71635] Updated weights for policy 1, policy_version 44232 (0.0010) [2023-10-11 20:50:07,657][71635] Updated weights for policy 1, policy_version 44242 (0.0009) [2023-10-11 20:50:08,018][71635] Updated weights for policy 1, policy_version 44252 (0.0009) [2023-10-11 20:50:09,085][71601] Updated weights for policy 0, policy_version 44260 (0.0009) [2023-10-11 20:50:09,480][71601] Updated weights for policy 0, policy_version 44270 (0.0009) [2023-10-11 20:50:09,850][71601] Updated weights for policy 0, policy_version 44280 (0.0008) [2023-10-11 20:50:11,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 90669056. Throughput: 0: 1822.7, 1: 1821.6. Samples: 22676872. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-10-11 20:50:11,035][70582] Avg episode reward: [(0, '182.360'), (1, '124.240')] [2023-10-11 20:50:11,678][71635] Updated weights for policy 1, policy_version 44262 (0.0008) [2023-10-11 20:50:12,038][71635] Updated weights for policy 1, policy_version 44272 (0.0010) [2023-10-11 20:50:12,409][71635] Updated weights for policy 1, policy_version 44282 (0.0008) [2023-10-11 20:50:13,489][71601] Updated weights for policy 0, policy_version 44290 (0.0009) [2023-10-11 20:50:13,870][71601] Updated weights for policy 0, policy_version 44300 (0.0008) [2023-10-11 20:50:14,229][71601] Updated weights for policy 0, policy_version 44310 (0.0008) [2023-10-11 20:50:14,601][71601] Updated weights for policy 0, policy_version 44320 (0.0008) [2023-10-11 20:50:16,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 90734592. Throughput: 0: 1823.7, 1: 1819.4. Samples: 22688318. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-10-11 20:50:16,034][70582] Avg episode reward: [(0, '197.500'), (1, '130.850')] [2023-10-11 20:50:16,090][71635] Updated weights for policy 1, policy_version 44292 (0.0009) [2023-10-11 20:50:16,457][71635] Updated weights for policy 1, policy_version 44302 (0.0009) [2023-10-11 20:50:16,824][71635] Updated weights for policy 1, policy_version 44312 (0.0009) [2023-10-11 20:50:18,269][71601] Updated weights for policy 0, policy_version 44330 (0.0010) [2023-10-11 20:50:18,647][71601] Updated weights for policy 0, policy_version 44340 (0.0007) [2023-10-11 20:50:19,007][71601] Updated weights for policy 0, policy_version 44350 (0.0007) [2023-10-11 20:50:20,359][71635] Updated weights for policy 1, policy_version 44322 (0.0008) [2023-10-11 20:50:20,731][71635] Updated weights for policy 1, policy_version 44332 (0.0008) [2023-10-11 20:50:21,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 90800128. Throughput: 0: 1821.0, 1: 1822.0. Samples: 22709958. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-10-11 20:50:21,035][70582] Avg episode reward: [(0, '199.740'), (1, '122.590')] [2023-10-11 20:50:21,095][71635] Updated weights for policy 1, policy_version 44342 (0.0008) [2023-10-11 20:50:21,469][71635] Updated weights for policy 1, policy_version 44352 (0.0008) [2023-10-11 20:50:22,587][71601] Updated weights for policy 0, policy_version 44360 (0.0010) [2023-10-11 20:50:22,952][71601] Updated weights for policy 0, policy_version 44370 (0.0011) [2023-10-11 20:50:23,326][71601] Updated weights for policy 0, policy_version 44380 (0.0010) [2023-10-11 20:50:25,213][71635] Updated weights for policy 1, policy_version 44362 (0.0007) [2023-10-11 20:50:25,584][71635] Updated weights for policy 1, policy_version 44372 (0.0009) [2023-10-11 20:50:25,956][71635] Updated weights for policy 1, policy_version 44382 (0.0007) [2023-10-11 20:50:26,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 90898432. Throughput: 0: 1821.8, 1: 1820.1. Samples: 22732046. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-11 20:50:26,034][70582] Avg episode reward: [(0, '184.450'), (1, '117.320')] [2023-10-11 20:50:27,153][71601] Updated weights for policy 0, policy_version 44390 (0.0010) [2023-10-11 20:50:27,526][71601] Updated weights for policy 0, policy_version 44400 (0.0008) [2023-10-11 20:50:27,888][71601] Updated weights for policy 0, policy_version 44410 (0.0007) [2023-10-11 20:50:29,651][71635] Updated weights for policy 1, policy_version 44392 (0.0007) [2023-10-11 20:50:30,031][71635] Updated weights for policy 1, policy_version 44402 (0.0008) [2023-10-11 20:50:30,395][71635] Updated weights for policy 1, policy_version 44412 (0.0009) [2023-10-11 20:50:31,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 90963968. Throughput: 0: 1821.1, 1: 1819.8. Samples: 22742570. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-11 20:50:31,034][70582] Avg episode reward: [(0, '184.450'), (1, '114.190')] [2023-10-11 20:50:31,614][71601] Updated weights for policy 0, policy_version 44420 (0.0007) [2023-10-11 20:50:31,976][71601] Updated weights for policy 0, policy_version 44430 (0.0010) [2023-10-11 20:50:32,346][71601] Updated weights for policy 0, policy_version 44440 (0.0008) [2023-10-11 20:50:34,158][71635] Updated weights for policy 1, policy_version 44422 (0.0008) [2023-10-11 20:50:34,523][71635] Updated weights for policy 1, policy_version 44432 (0.0010) [2023-10-11 20:50:34,892][71635] Updated weights for policy 1, policy_version 44442 (0.0008) [2023-10-11 20:50:36,000][71601] Updated weights for policy 0, policy_version 44450 (0.0011) [2023-10-11 20:50:36,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 91029504. Throughput: 0: 1823.7, 1: 1820.2. Samples: 22765002. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-11 20:50:36,034][70582] Avg episode reward: [(0, '192.220'), (1, '115.400')] [2023-10-11 20:50:36,382][71601] Updated weights for policy 0, policy_version 44460 (0.0009) [2023-10-11 20:50:36,756][71601] Updated weights for policy 0, policy_version 44470 (0.0008) [2023-10-11 20:50:37,121][71601] Updated weights for policy 0, policy_version 44480 (0.0007) [2023-10-11 20:50:38,489][71635] Updated weights for policy 1, policy_version 44452 (0.0008) [2023-10-11 20:50:38,855][71635] Updated weights for policy 1, policy_version 44462 (0.0009) [2023-10-11 20:50:39,215][71635] Updated weights for policy 1, policy_version 44472 (0.0008) [2023-10-11 20:50:40,622][71601] Updated weights for policy 0, policy_version 44490 (0.0011) [2023-10-11 20:50:40,995][71601] Updated weights for policy 0, policy_version 44500 (0.0010) [2023-10-11 20:50:41,034][70582] Fps is (10 sec: 13106.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 91095040. Throughput: 0: 1816.0, 1: 1820.8. Samples: 22786560. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-11 20:50:41,035][70582] Avg episode reward: [(0, '177.090'), (1, '115.170')] [2023-10-11 20:50:41,371][71601] Updated weights for policy 0, policy_version 44510 (0.0007) [2023-10-11 20:50:42,994][71635] Updated weights for policy 1, policy_version 44482 (0.0010) [2023-10-11 20:50:43,367][71635] Updated weights for policy 1, policy_version 44492 (0.0008) [2023-10-11 20:50:43,720][71635] Updated weights for policy 1, policy_version 44502 (0.0008) [2023-10-11 20:50:44,092][71635] Updated weights for policy 1, policy_version 44512 (0.0011) [2023-10-11 20:50:45,068][71601] Updated weights for policy 0, policy_version 44520 (0.0007) [2023-10-11 20:50:45,440][71601] Updated weights for policy 0, policy_version 44530 (0.0007) [2023-10-11 20:50:45,810][71601] Updated weights for policy 0, policy_version 44540 (0.0008) [2023-10-11 20:50:46,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 91193344. Throughput: 0: 1818.3, 1: 1818.2. Samples: 22797728. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-11 20:50:46,035][70582] Avg episode reward: [(0, '176.850'), (1, '117.560')] [2023-10-11 20:50:47,872][71635] Updated weights for policy 1, policy_version 44522 (0.0009) [2023-10-11 20:50:48,232][71635] Updated weights for policy 1, policy_version 44532 (0.0010) [2023-10-11 20:50:48,598][71635] Updated weights for policy 1, policy_version 44542 (0.0009) [2023-10-11 20:50:49,620][71601] Updated weights for policy 0, policy_version 44550 (0.0010) [2023-10-11 20:50:49,999][71601] Updated weights for policy 0, policy_version 44560 (0.0010) [2023-10-11 20:50:50,370][71601] Updated weights for policy 0, policy_version 44570 (0.0008) [2023-10-11 20:50:51,034][70582] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 91258880. Throughput: 0: 1820.4, 1: 1814.0. Samples: 22819192. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-11 20:50:51,034][70582] Avg episode reward: [(0, '191.440'), (1, '116.350')] [2023-10-11 20:50:52,386][71635] Updated weights for policy 1, policy_version 44552 (0.0010) [2023-10-11 20:50:52,749][71635] Updated weights for policy 1, policy_version 44562 (0.0009) [2023-10-11 20:50:53,113][71635] Updated weights for policy 1, policy_version 44572 (0.0007) [2023-10-11 20:50:54,112][71601] Updated weights for policy 0, policy_version 44580 (0.0007) [2023-10-11 20:50:54,509][71601] Updated weights for policy 0, policy_version 44590 (0.0008) [2023-10-11 20:50:54,872][71601] Updated weights for policy 0, policy_version 44600 (0.0007) [2023-10-11 20:50:56,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 91324416. Throughput: 0: 1818.3, 1: 1810.8. Samples: 22840182. Policy #0 lag: (min: 6.0, avg: 16.1, max: 38.0) [2023-10-11 20:50:56,035][70582] Avg episode reward: [(0, '196.850'), (1, '129.010')] [2023-10-11 20:50:56,966][71635] Updated weights for policy 1, policy_version 44582 (0.0010) [2023-10-11 20:50:57,338][71635] Updated weights for policy 1, policy_version 44592 (0.0008) [2023-10-11 20:50:57,698][71635] Updated weights for policy 1, policy_version 44602 (0.0008) [2023-10-11 20:50:58,649][71601] Updated weights for policy 0, policy_version 44610 (0.0007) [2023-10-11 20:50:59,016][71601] Updated weights for policy 0, policy_version 44620 (0.0009) [2023-10-11 20:50:59,380][71601] Updated weights for policy 0, policy_version 44630 (0.0009) [2023-10-11 20:50:59,750][71601] Updated weights for policy 0, policy_version 44640 (0.0008) [2023-10-11 20:51:01,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 91389952. Throughput: 0: 1819.1, 1: 1814.2. Samples: 22851816. Policy #0 lag: (min: 6.0, avg: 16.1, max: 38.0) [2023-10-11 20:51:01,034][70582] Avg episode reward: [(0, '189.970'), (1, '129.260')] [2023-10-11 20:51:01,325][71635] Updated weights for policy 1, policy_version 44612 (0.0007) [2023-10-11 20:51:01,690][71635] Updated weights for policy 1, policy_version 44622 (0.0010) [2023-10-11 20:51:02,057][71635] Updated weights for policy 1, policy_version 44632 (0.0009) [2023-10-11 20:51:03,430][71601] Updated weights for policy 0, policy_version 44650 (0.0010) [2023-10-11 20:51:03,799][71601] Updated weights for policy 0, policy_version 44660 (0.0007) [2023-10-11 20:51:04,173][71601] Updated weights for policy 0, policy_version 44670 (0.0007) [2023-10-11 20:51:05,905][71635] Updated weights for policy 1, policy_version 44642 (0.0007) [2023-10-11 20:51:06,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 91455488. Throughput: 0: 1817.9, 1: 1808.3. Samples: 22873136. Policy #0 lag: (min: 6.0, avg: 16.1, max: 38.0) [2023-10-11 20:51:06,034][70582] Avg episode reward: [(0, '189.850'), (1, '137.490')] [2023-10-11 20:51:06,282][71635] Updated weights for policy 1, policy_version 44652 (0.0008) [2023-10-11 20:51:06,646][71635] Updated weights for policy 1, policy_version 44662 (0.0010) [2023-10-11 20:51:07,014][71635] Updated weights for policy 1, policy_version 44672 (0.0009) [2023-10-11 20:51:07,785][71601] Updated weights for policy 0, policy_version 44680 (0.0008) [2023-10-11 20:51:08,153][71601] Updated weights for policy 0, policy_version 44690 (0.0007) [2023-10-11 20:51:08,530][71601] Updated weights for policy 0, policy_version 44700 (0.0007) [2023-10-11 20:51:10,838][71635] Updated weights for policy 1, policy_version 44682 (0.0009) [2023-10-11 20:51:11,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 91521024. Throughput: 0: 1816.8, 1: 1819.7. Samples: 22895688. Policy #0 lag: (min: 6.0, avg: 16.1, max: 38.0) [2023-10-11 20:51:11,035][70582] Avg episode reward: [(0, '196.620'), (1, '148.310')] [2023-10-11 20:51:11,211][71635] Updated weights for policy 1, policy_version 44692 (0.0008) [2023-10-11 20:51:11,583][71635] Updated weights for policy 1, policy_version 44702 (0.0007) [2023-10-11 20:51:12,253][71601] Updated weights for policy 0, policy_version 44710 (0.0008) [2023-10-11 20:51:12,614][71601] Updated weights for policy 0, policy_version 44720 (0.0010) [2023-10-11 20:51:12,997][71601] Updated weights for policy 0, policy_version 44730 (0.0007) [2023-10-11 20:51:15,160][71635] Updated weights for policy 1, policy_version 44712 (0.0007) [2023-10-11 20:51:15,533][71635] Updated weights for policy 1, policy_version 44722 (0.0010) [2023-10-11 20:51:15,894][71635] Updated weights for policy 1, policy_version 44732 (0.0008) [2023-10-11 20:51:16,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 91586560. Throughput: 0: 1817.1, 1: 1805.1. Samples: 22905566. Policy #0 lag: (min: 6.0, avg: 16.1, max: 38.0) [2023-10-11 20:51:16,034][70582] Avg episode reward: [(0, '198.820'), (1, '148.310')] [2023-10-11 20:51:16,552][71601] Updated weights for policy 0, policy_version 44740 (0.0008) [2023-10-11 20:51:16,922][71601] Updated weights for policy 0, policy_version 44750 (0.0009) [2023-10-11 20:51:17,308][71601] Updated weights for policy 0, policy_version 44760 (0.0011) [2023-10-11 20:51:19,484][71635] Updated weights for policy 1, policy_version 44742 (0.0008) [2023-10-11 20:51:19,848][71635] Updated weights for policy 1, policy_version 44752 (0.0008) [2023-10-11 20:51:20,217][71635] Updated weights for policy 1, policy_version 44762 (0.0008) [2023-10-11 20:51:20,918][71601] Updated weights for policy 0, policy_version 44770 (0.0008) [2023-10-11 20:51:21,034][70582] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 91684864. Throughput: 0: 1813.0, 1: 1816.9. Samples: 22928348. Policy #0 lag: (min: 6.0, avg: 16.1, max: 38.0) [2023-10-11 20:51:21,034][70582] Avg episode reward: [(0, '200.920'), (1, '144.620')] [2023-10-11 20:51:21,291][71601] Updated weights for policy 0, policy_version 44780 (0.0011) [2023-10-11 20:51:21,669][71601] Updated weights for policy 0, policy_version 44790 (0.0008) [2023-10-11 20:51:22,041][71601] Updated weights for policy 0, policy_version 44800 (0.0009) [2023-10-11 20:51:23,838][71635] Updated weights for policy 1, policy_version 44772 (0.0007) [2023-10-11 20:51:24,209][71635] Updated weights for policy 1, policy_version 44782 (0.0008) [2023-10-11 20:51:24,574][71635] Updated weights for policy 1, policy_version 44792 (0.0010) [2023-10-11 20:51:25,821][71601] Updated weights for policy 0, policy_version 44810 (0.0007) [2023-10-11 20:51:26,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 91750400. Throughput: 0: 1821.2, 1: 1811.3. Samples: 22950020. Policy #0 lag: (min: 6.0, avg: 16.1, max: 38.0) [2023-10-11 20:51:26,034][70582] Avg episode reward: [(0, '200.200'), (1, '148.460')] [2023-10-11 20:51:26,192][71601] Updated weights for policy 0, policy_version 44820 (0.0009) [2023-10-11 20:51:26,569][71601] Updated weights for policy 0, policy_version 44830 (0.0008) [2023-10-11 20:51:28,173][71635] Updated weights for policy 1, policy_version 44802 (0.0007) [2023-10-11 20:51:28,532][71635] Updated weights for policy 1, policy_version 44812 (0.0007) [2023-10-11 20:51:28,896][71635] Updated weights for policy 1, policy_version 44822 (0.0007) [2023-10-11 20:51:29,265][71635] Updated weights for policy 1, policy_version 44832 (0.0007) [2023-10-11 20:51:30,176][71601] Updated weights for policy 0, policy_version 44840 (0.0009) [2023-10-11 20:51:30,547][71601] Updated weights for policy 0, policy_version 44850 (0.0010) [2023-10-11 20:51:30,915][71601] Updated weights for policy 0, policy_version 44860 (0.0008) [2023-10-11 20:51:31,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 91815936. Throughput: 0: 1814.0, 1: 1818.4. Samples: 22961186. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 20:51:31,035][70582] Avg episode reward: [(0, '190.370'), (1, '149.840')] [2023-10-11 20:51:33,001][71635] Updated weights for policy 1, policy_version 44842 (0.0007) [2023-10-11 20:51:33,366][71635] Updated weights for policy 1, policy_version 44852 (0.0009) [2023-10-11 20:51:33,741][71635] Updated weights for policy 1, policy_version 44862 (0.0010) [2023-10-11 20:51:34,712][71601] Updated weights for policy 0, policy_version 44870 (0.0008) [2023-10-11 20:51:35,080][71601] Updated weights for policy 0, policy_version 44880 (0.0008) [2023-10-11 20:51:35,451][71601] Updated weights for policy 0, policy_version 44890 (0.0009) [2023-10-11 20:51:36,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 91914240. Throughput: 0: 1814.9, 1: 1816.9. Samples: 22982624. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 20:51:36,035][70582] Avg episode reward: [(0, '190.180'), (1, '152.190')] [2023-10-11 20:51:37,394][71635] Updated weights for policy 1, policy_version 44872 (0.0007) [2023-10-11 20:51:37,752][71635] Updated weights for policy 1, policy_version 44882 (0.0007) [2023-10-11 20:51:38,126][71635] Updated weights for policy 1, policy_version 44892 (0.0007) [2023-10-11 20:51:39,217][71601] Updated weights for policy 0, policy_version 44900 (0.0008) [2023-10-11 20:51:39,588][71601] Updated weights for policy 0, policy_version 44910 (0.0007) [2023-10-11 20:51:39,972][71601] Updated weights for policy 0, policy_version 44920 (0.0007) [2023-10-11 20:51:41,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 91979776. Throughput: 0: 1819.6, 1: 1829.4. Samples: 23004390. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 20:51:41,035][70582] Avg episode reward: [(0, '218.420'), (1, '152.230')] [2023-10-11 20:51:41,735][71635] Updated weights for policy 1, policy_version 44902 (0.0010) [2023-10-11 20:51:42,103][71635] Updated weights for policy 1, policy_version 44912 (0.0011) [2023-10-11 20:51:42,470][71635] Updated weights for policy 1, policy_version 44922 (0.0010) [2023-10-11 20:51:43,716][71601] Updated weights for policy 0, policy_version 44930 (0.0008) [2023-10-11 20:51:44,087][71601] Updated weights for policy 0, policy_version 44940 (0.0008) [2023-10-11 20:51:44,468][71601] Updated weights for policy 0, policy_version 44950 (0.0009) [2023-10-11 20:51:44,831][71601] Updated weights for policy 0, policy_version 44960 (0.0009) [2023-10-11 20:51:46,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 92045312. Throughput: 0: 1818.6, 1: 1823.6. Samples: 23015714. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 20:51:46,035][70582] Avg episode reward: [(0, '223.730'), (1, '172.240')] [2023-10-11 20:51:46,218][71635] Updated weights for policy 1, policy_version 44932 (0.0009) [2023-10-11 20:51:46,580][71635] Updated weights for policy 1, policy_version 44942 (0.0010) [2023-10-11 20:51:46,949][71635] Updated weights for policy 1, policy_version 44952 (0.0010) [2023-10-11 20:51:48,371][71601] Updated weights for policy 0, policy_version 44970 (0.0011) [2023-10-11 20:51:48,749][71601] Updated weights for policy 0, policy_version 44980 (0.0008) [2023-10-11 20:51:49,117][71601] Updated weights for policy 0, policy_version 44990 (0.0009) [2023-10-11 20:51:50,704][71635] Updated weights for policy 1, policy_version 44962 (0.0007) [2023-10-11 20:51:51,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 92110848. Throughput: 0: 1820.5, 1: 1823.6. Samples: 23037122. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 20:51:51,035][70582] Avg episode reward: [(0, '218.640'), (1, '162.780')] [2023-10-11 20:51:51,078][71635] Updated weights for policy 1, policy_version 44972 (0.0008) [2023-10-11 20:51:51,435][71635] Updated weights for policy 1, policy_version 44982 (0.0007) [2023-10-11 20:51:51,794][71635] Updated weights for policy 1, policy_version 44992 (0.0007) [2023-10-11 20:51:52,823][71601] Updated weights for policy 0, policy_version 45000 (0.0007) [2023-10-11 20:51:53,195][71601] Updated weights for policy 0, policy_version 45010 (0.0009) [2023-10-11 20:51:53,567][71601] Updated weights for policy 0, policy_version 45020 (0.0009) [2023-10-11 20:51:55,512][71635] Updated weights for policy 1, policy_version 45002 (0.0007) [2023-10-11 20:51:55,883][71635] Updated weights for policy 1, policy_version 45012 (0.0011) [2023-10-11 20:51:56,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 92176384. Throughput: 0: 1822.9, 1: 1818.8. Samples: 23059564. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 20:51:56,035][70582] Avg episode reward: [(0, '216.510'), (1, '161.830')] [2023-10-11 20:51:56,048][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000045024_46104576.pth... [2023-10-11 20:51:56,080][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000043328_44367872.pth [2023-10-11 20:51:56,247][71635] Updated weights for policy 1, policy_version 45022 (0.0011) [2023-10-11 20:51:56,311][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000045024_46104576.pth... [2023-10-11 20:51:56,340][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000043296_44335104.pth [2023-10-11 20:51:57,112][71601] Updated weights for policy 0, policy_version 45030 (0.0009) [2023-10-11 20:51:57,484][71601] Updated weights for policy 0, policy_version 45040 (0.0008) [2023-10-11 20:51:57,860][71601] Updated weights for policy 0, policy_version 45050 (0.0007) [2023-10-11 20:51:59,931][71635] Updated weights for policy 1, policy_version 45032 (0.0011) [2023-10-11 20:52:00,302][71635] Updated weights for policy 1, policy_version 45042 (0.0008) [2023-10-11 20:52:00,663][71635] Updated weights for policy 1, policy_version 45052 (0.0008) [2023-10-11 20:52:01,034][70582] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 92274688. Throughput: 0: 1824.7, 1: 1827.5. Samples: 23069916. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:52:01,034][70582] Avg episode reward: [(0, '220.740'), (1, '168.270')] [2023-10-11 20:52:01,521][71601] Updated weights for policy 0, policy_version 45060 (0.0009) [2023-10-11 20:52:01,886][71601] Updated weights for policy 0, policy_version 45070 (0.0008) [2023-10-11 20:52:02,266][71601] Updated weights for policy 0, policy_version 45080 (0.0007) [2023-10-11 20:52:04,345][71635] Updated weights for policy 1, policy_version 45062 (0.0010) [2023-10-11 20:52:04,716][71635] Updated weights for policy 1, policy_version 45072 (0.0010) [2023-10-11 20:52:05,079][71635] Updated weights for policy 1, policy_version 45082 (0.0007) [2023-10-11 20:52:05,995][71601] Updated weights for policy 0, policy_version 45090 (0.0009) [2023-10-11 20:52:06,034][70582] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 92340224. Throughput: 0: 1822.2, 1: 1823.7. Samples: 23092412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:52:06,034][70582] Avg episode reward: [(0, '218.510'), (1, '167.040')] [2023-10-11 20:52:06,366][71601] Updated weights for policy 0, policy_version 45100 (0.0008) [2023-10-11 20:52:06,733][71601] Updated weights for policy 0, policy_version 45110 (0.0009) [2023-10-11 20:52:07,105][71601] Updated weights for policy 0, policy_version 45120 (0.0009) [2023-10-11 20:52:08,596][71635] Updated weights for policy 1, policy_version 45092 (0.0008) [2023-10-11 20:52:08,967][71635] Updated weights for policy 1, policy_version 45102 (0.0010) [2023-10-11 20:52:09,335][71635] Updated weights for policy 1, policy_version 45112 (0.0008) [2023-10-11 20:52:10,796][71601] Updated weights for policy 0, policy_version 45130 (0.0009) [2023-10-11 20:52:11,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 92405760. Throughput: 0: 1821.5, 1: 1827.9. Samples: 23114240. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:52:11,034][70582] Avg episode reward: [(0, '246.430'), (1, '159.890')] [2023-10-11 20:52:11,171][71601] Updated weights for policy 0, policy_version 45140 (0.0008) [2023-10-11 20:52:11,541][71601] Updated weights for policy 0, policy_version 45150 (0.0010) [2023-10-11 20:52:13,130][71635] Updated weights for policy 1, policy_version 45122 (0.0007) [2023-10-11 20:52:13,499][71635] Updated weights for policy 1, policy_version 45132 (0.0009) [2023-10-11 20:52:13,859][71635] Updated weights for policy 1, policy_version 45142 (0.0011) [2023-10-11 20:52:14,223][71635] Updated weights for policy 1, policy_version 45152 (0.0010) [2023-10-11 20:52:15,341][71601] Updated weights for policy 0, policy_version 45160 (0.0008) [2023-10-11 20:52:15,715][71601] Updated weights for policy 0, policy_version 45170 (0.0009) [2023-10-11 20:52:16,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 92471296. Throughput: 0: 1817.8, 1: 1823.2. Samples: 23125032. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:52:16,034][70582] Avg episode reward: [(0, '245.670'), (1, '160.940')] [2023-10-11 20:52:16,081][71601] Updated weights for policy 0, policy_version 45180 (0.0009) [2023-10-11 20:52:17,992][71635] Updated weights for policy 1, policy_version 45162 (0.0009) [2023-10-11 20:52:18,354][71635] Updated weights for policy 1, policy_version 45172 (0.0009) [2023-10-11 20:52:18,726][71635] Updated weights for policy 1, policy_version 45182 (0.0008) [2023-10-11 20:52:19,787][71601] Updated weights for policy 0, policy_version 45190 (0.0008) [2023-10-11 20:52:20,155][71601] Updated weights for policy 0, policy_version 45200 (0.0009) [2023-10-11 20:52:20,524][71601] Updated weights for policy 0, policy_version 45210 (0.0010) [2023-10-11 20:52:21,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 92569600. Throughput: 0: 1820.4, 1: 1828.9. Samples: 23146844. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:52:21,034][70582] Avg episode reward: [(0, '244.280'), (1, '161.090')] [2023-10-11 20:52:22,428][71635] Updated weights for policy 1, policy_version 45192 (0.0008) [2023-10-11 20:52:22,805][71635] Updated weights for policy 1, policy_version 45202 (0.0007) [2023-10-11 20:52:23,169][71635] Updated weights for policy 1, policy_version 45212 (0.0008) [2023-10-11 20:52:24,201][71601] Updated weights for policy 0, policy_version 45220 (0.0009) [2023-10-11 20:52:24,595][71601] Updated weights for policy 0, policy_version 45230 (0.0007) [2023-10-11 20:52:24,966][71601] Updated weights for policy 0, policy_version 45240 (0.0008) [2023-10-11 20:52:26,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 92635136. Throughput: 0: 1821.5, 1: 1827.5. Samples: 23168594. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:52:26,034][70582] Avg episode reward: [(0, '262.750'), (1, '144.740')] [2023-10-11 20:52:26,616][71635] Updated weights for policy 1, policy_version 45222 (0.0009) [2023-10-11 20:52:26,983][71635] Updated weights for policy 1, policy_version 45232 (0.0009) [2023-10-11 20:52:27,354][71635] Updated weights for policy 1, policy_version 45242 (0.0007) [2023-10-11 20:52:28,571][71601] Updated weights for policy 0, policy_version 45250 (0.0008) [2023-10-11 20:52:28,939][71601] Updated weights for policy 0, policy_version 45260 (0.0007) [2023-10-11 20:52:29,310][71601] Updated weights for policy 0, policy_version 45270 (0.0008) [2023-10-11 20:52:29,678][71601] Updated weights for policy 0, policy_version 45280 (0.0008) [2023-10-11 20:52:30,939][71635] Updated weights for policy 1, policy_version 45252 (0.0009) [2023-10-11 20:52:31,034][70582] Fps is (10 sec: 13106.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 92700672. Throughput: 0: 1821.6, 1: 1833.2. Samples: 23180182. Policy #0 lag: (min: 8.0, avg: 21.3, max: 40.0) [2023-10-11 20:52:31,035][70582] Avg episode reward: [(0, '248.410'), (1, '133.460')] [2023-10-11 20:52:31,305][71635] Updated weights for policy 1, policy_version 45262 (0.0008) [2023-10-11 20:52:31,668][71635] Updated weights for policy 1, policy_version 45272 (0.0009) [2023-10-11 20:52:33,281][71601] Updated weights for policy 0, policy_version 45290 (0.0007) [2023-10-11 20:52:33,655][71601] Updated weights for policy 0, policy_version 45300 (0.0010) [2023-10-11 20:52:34,023][71601] Updated weights for policy 0, policy_version 45310 (0.0010) [2023-10-11 20:52:35,424][71635] Updated weights for policy 1, policy_version 45282 (0.0008) [2023-10-11 20:52:35,797][71635] Updated weights for policy 1, policy_version 45292 (0.0007) [2023-10-11 20:52:36,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 92766208. Throughput: 0: 1822.9, 1: 1829.4. Samples: 23201476. Policy #0 lag: (min: 8.0, avg: 21.3, max: 40.0) [2023-10-11 20:52:36,035][70582] Avg episode reward: [(0, '214.870'), (1, '132.820')] [2023-10-11 20:52:36,164][71635] Updated weights for policy 1, policy_version 45302 (0.0008) [2023-10-11 20:52:36,532][71635] Updated weights for policy 1, policy_version 45312 (0.0008) [2023-10-11 20:52:37,769][71601] Updated weights for policy 0, policy_version 45320 (0.0007) [2023-10-11 20:52:38,145][71601] Updated weights for policy 0, policy_version 45330 (0.0007) [2023-10-11 20:52:38,514][71601] Updated weights for policy 0, policy_version 45340 (0.0007) [2023-10-11 20:52:40,246][71635] Updated weights for policy 1, policy_version 45322 (0.0009) [2023-10-11 20:52:40,605][71635] Updated weights for policy 1, policy_version 45332 (0.0008) [2023-10-11 20:52:40,965][71635] Updated weights for policy 1, policy_version 45342 (0.0008) [2023-10-11 20:52:41,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 92831744. Throughput: 0: 1822.9, 1: 1822.8. Samples: 23223622. Policy #0 lag: (min: 8.0, avg: 21.3, max: 40.0) [2023-10-11 20:52:41,035][70582] Avg episode reward: [(0, '215.250'), (1, '124.910')] [2023-10-11 20:52:42,166][71601] Updated weights for policy 0, policy_version 45350 (0.0009) [2023-10-11 20:52:42,535][71601] Updated weights for policy 0, policy_version 45360 (0.0008) [2023-10-11 20:52:42,908][71601] Updated weights for policy 0, policy_version 45370 (0.0008) [2023-10-11 20:52:44,564][71635] Updated weights for policy 1, policy_version 45352 (0.0007) [2023-10-11 20:52:44,927][71635] Updated weights for policy 1, policy_version 45362 (0.0008) [2023-10-11 20:52:45,296][71635] Updated weights for policy 1, policy_version 45372 (0.0007) [2023-10-11 20:52:46,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 92930048. Throughput: 0: 1819.8, 1: 1833.2. Samples: 23234304. Policy #0 lag: (min: 8.0, avg: 21.3, max: 40.0) [2023-10-11 20:52:46,034][70582] Avg episode reward: [(0, '207.830'), (1, '125.900')] [2023-10-11 20:52:46,534][71601] Updated weights for policy 0, policy_version 45380 (0.0009) [2023-10-11 20:52:46,900][71601] Updated weights for policy 0, policy_version 45390 (0.0011) [2023-10-11 20:52:47,276][71601] Updated weights for policy 0, policy_version 45400 (0.0010) [2023-10-11 20:52:49,124][71635] Updated weights for policy 1, policy_version 45382 (0.0008) [2023-10-11 20:52:49,487][71635] Updated weights for policy 1, policy_version 45392 (0.0010) [2023-10-11 20:52:49,857][71635] Updated weights for policy 1, policy_version 45402 (0.0009) [2023-10-11 20:52:50,797][71601] Updated weights for policy 0, policy_version 45410 (0.0009) [2023-10-11 20:52:51,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 92995584. Throughput: 0: 1826.4, 1: 1820.2. Samples: 23256506. Policy #0 lag: (min: 8.0, avg: 21.3, max: 40.0) [2023-10-11 20:52:51,034][70582] Avg episode reward: [(0, '207.790'), (1, '111.390')] [2023-10-11 20:52:51,169][71601] Updated weights for policy 0, policy_version 45420 (0.0009) [2023-10-11 20:52:51,534][71601] Updated weights for policy 0, policy_version 45430 (0.0008) [2023-10-11 20:52:51,905][71601] Updated weights for policy 0, policy_version 45440 (0.0007) [2023-10-11 20:52:53,724][71635] Updated weights for policy 1, policy_version 45412 (0.0008) [2023-10-11 20:52:54,086][71635] Updated weights for policy 1, policy_version 45422 (0.0007) [2023-10-11 20:52:54,457][71635] Updated weights for policy 1, policy_version 45432 (0.0010) [2023-10-11 20:52:55,580][71601] Updated weights for policy 0, policy_version 45450 (0.0009) [2023-10-11 20:52:55,952][71601] Updated weights for policy 0, policy_version 45460 (0.0007) [2023-10-11 20:52:56,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 93061120. Throughput: 0: 1816.6, 1: 1824.3. Samples: 23278080. Policy #0 lag: (min: 8.0, avg: 21.3, max: 40.0) [2023-10-11 20:52:56,034][70582] Avg episode reward: [(0, '207.980'), (1, '116.970')] [2023-10-11 20:52:56,319][71601] Updated weights for policy 0, policy_version 45470 (0.0007) [2023-10-11 20:52:58,153][71635] Updated weights for policy 1, policy_version 45442 (0.0009) [2023-10-11 20:52:58,518][71635] Updated weights for policy 1, policy_version 45452 (0.0007) [2023-10-11 20:52:58,880][71635] Updated weights for policy 1, policy_version 45462 (0.0007) [2023-10-11 20:52:59,240][71635] Updated weights for policy 1, policy_version 45472 (0.0008) [2023-10-11 20:52:59,815][71601] Updated weights for policy 0, policy_version 45480 (0.0008) [2023-10-11 20:53:00,184][71601] Updated weights for policy 0, policy_version 45490 (0.0009) [2023-10-11 20:53:00,557][71601] Updated weights for policy 0, policy_version 45500 (0.0010) [2023-10-11 20:53:01,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 93159424. Throughput: 0: 1829.2, 1: 1827.0. Samples: 23289562. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-11 20:53:01,035][70582] Avg episode reward: [(0, '210.070'), (1, '116.520')] [2023-10-11 20:53:02,839][71635] Updated weights for policy 1, policy_version 45482 (0.0007) [2023-10-11 20:53:03,202][71635] Updated weights for policy 1, policy_version 45492 (0.0007) [2023-10-11 20:53:03,572][71635] Updated weights for policy 1, policy_version 45502 (0.0008) [2023-10-11 20:53:04,273][71601] Updated weights for policy 0, policy_version 45510 (0.0008) [2023-10-11 20:53:04,649][71601] Updated weights for policy 0, policy_version 45520 (0.0007) [2023-10-11 20:53:05,022][71601] Updated weights for policy 0, policy_version 45530 (0.0007) [2023-10-11 20:53:06,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 93224960. Throughput: 0: 1820.0, 1: 1820.3. Samples: 23310658. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-11 20:53:06,034][70582] Avg episode reward: [(0, '202.650'), (1, '115.700')] [2023-10-11 20:53:07,103][71635] Updated weights for policy 1, policy_version 45512 (0.0007) [2023-10-11 20:53:07,464][71635] Updated weights for policy 1, policy_version 45522 (0.0009) [2023-10-11 20:53:07,836][71635] Updated weights for policy 1, policy_version 45532 (0.0008) [2023-10-11 20:53:08,742][71601] Updated weights for policy 0, policy_version 45540 (0.0011) [2023-10-11 20:53:09,109][71601] Updated weights for policy 0, policy_version 45550 (0.0010) [2023-10-11 20:53:09,474][71601] Updated weights for policy 0, policy_version 45560 (0.0009) [2023-10-11 20:53:11,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 93290496. Throughput: 0: 1829.7, 1: 1817.4. Samples: 23332714. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-11 20:53:11,034][70582] Avg episode reward: [(0, '203.500'), (1, '129.200')] [2023-10-11 20:53:11,560][71635] Updated weights for policy 1, policy_version 45542 (0.0009) [2023-10-11 20:53:11,933][71635] Updated weights for policy 1, policy_version 45552 (0.0010) [2023-10-11 20:53:12,298][71635] Updated weights for policy 1, policy_version 45562 (0.0008) [2023-10-11 20:53:13,236][71601] Updated weights for policy 0, policy_version 45570 (0.0007) [2023-10-11 20:53:13,626][71601] Updated weights for policy 0, policy_version 45580 (0.0007) [2023-10-11 20:53:13,996][71601] Updated weights for policy 0, policy_version 45590 (0.0008) [2023-10-11 20:53:14,373][71601] Updated weights for policy 0, policy_version 45600 (0.0007) [2023-10-11 20:53:15,975][71635] Updated weights for policy 1, policy_version 45572 (0.0007) [2023-10-11 20:53:16,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 93356032. Throughput: 0: 1822.2, 1: 1816.5. Samples: 23343922. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-11 20:53:16,034][70582] Avg episode reward: [(0, '189.930'), (1, '118.560')] [2023-10-11 20:53:16,341][71635] Updated weights for policy 1, policy_version 45582 (0.0010) [2023-10-11 20:53:16,710][71635] Updated weights for policy 1, policy_version 45592 (0.0008) [2023-10-11 20:53:18,003][71601] Updated weights for policy 0, policy_version 45610 (0.0007) [2023-10-11 20:53:18,378][71601] Updated weights for policy 0, policy_version 45620 (0.0007) [2023-10-11 20:53:18,741][71601] Updated weights for policy 0, policy_version 45630 (0.0008) [2023-10-11 20:53:20,496][71635] Updated weights for policy 1, policy_version 45602 (0.0009) [2023-10-11 20:53:20,851][71635] Updated weights for policy 1, policy_version 45612 (0.0009) [2023-10-11 20:53:21,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 93421568. Throughput: 0: 1829.8, 1: 1817.8. Samples: 23365620. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-11 20:53:21,034][70582] Avg episode reward: [(0, '181.100'), (1, '124.530')] [2023-10-11 20:53:21,217][71635] Updated weights for policy 1, policy_version 45622 (0.0008) [2023-10-11 20:53:21,580][71635] Updated weights for policy 1, policy_version 45632 (0.0009) [2023-10-11 20:53:22,549][71601] Updated weights for policy 0, policy_version 45640 (0.0011) [2023-10-11 20:53:22,915][71601] Updated weights for policy 0, policy_version 45650 (0.0010) [2023-10-11 20:53:23,293][71601] Updated weights for policy 0, policy_version 45660 (0.0011) [2023-10-11 20:53:25,516][71635] Updated weights for policy 1, policy_version 45642 (0.0010) [2023-10-11 20:53:25,886][71635] Updated weights for policy 1, policy_version 45652 (0.0009) [2023-10-11 20:53:26,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 93487104. Throughput: 0: 1826.1, 1: 1826.5. Samples: 23387986. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-11 20:53:26,034][70582] Avg episode reward: [(0, '182.450'), (1, '122.700')] [2023-10-11 20:53:26,256][71635] Updated weights for policy 1, policy_version 45662 (0.0008) [2023-10-11 20:53:26,935][71601] Updated weights for policy 0, policy_version 45670 (0.0011) [2023-10-11 20:53:27,314][71601] Updated weights for policy 0, policy_version 45680 (0.0010) [2023-10-11 20:53:27,684][71601] Updated weights for policy 0, policy_version 45690 (0.0007) [2023-10-11 20:53:29,806][71635] Updated weights for policy 1, policy_version 45672 (0.0007) [2023-10-11 20:53:30,170][71635] Updated weights for policy 1, policy_version 45682 (0.0008) [2023-10-11 20:53:30,540][71635] Updated weights for policy 1, policy_version 45692 (0.0009) [2023-10-11 20:53:31,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 93585408. Throughput: 0: 1827.6, 1: 1812.8. Samples: 23398122. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-11 20:53:31,034][70582] Avg episode reward: [(0, '178.110'), (1, '124.980')] [2023-10-11 20:53:31,347][71601] Updated weights for policy 0, policy_version 45700 (0.0008) [2023-10-11 20:53:31,719][71601] Updated weights for policy 0, policy_version 45710 (0.0007) [2023-10-11 20:53:32,097][71601] Updated weights for policy 0, policy_version 45720 (0.0008) [2023-10-11 20:53:34,141][71635] Updated weights for policy 1, policy_version 45702 (0.0008) [2023-10-11 20:53:34,516][71635] Updated weights for policy 1, policy_version 45712 (0.0008) [2023-10-11 20:53:34,895][71635] Updated weights for policy 1, policy_version 45722 (0.0010) [2023-10-11 20:53:35,544][71601] Updated weights for policy 0, policy_version 45730 (0.0008) [2023-10-11 20:53:35,910][71601] Updated weights for policy 0, policy_version 45740 (0.0010) [2023-10-11 20:53:36,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 93650944. Throughput: 0: 1831.2, 1: 1818.6. Samples: 23420744. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-11 20:53:36,034][70582] Avg episode reward: [(0, '178.270'), (1, '119.470')] [2023-10-11 20:53:36,288][71601] Updated weights for policy 0, policy_version 45750 (0.0009) [2023-10-11 20:53:36,650][71601] Updated weights for policy 0, policy_version 45760 (0.0008) [2023-10-11 20:53:38,536][71635] Updated weights for policy 1, policy_version 45732 (0.0010) [2023-10-11 20:53:38,912][71635] Updated weights for policy 1, policy_version 45742 (0.0010) [2023-10-11 20:53:39,282][71635] Updated weights for policy 1, policy_version 45752 (0.0010) [2023-10-11 20:53:40,362][71601] Updated weights for policy 0, policy_version 45770 (0.0010) [2023-10-11 20:53:40,731][71601] Updated weights for policy 0, policy_version 45780 (0.0009) [2023-10-11 20:53:41,034][70582] Fps is (10 sec: 13106.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 93716480. Throughput: 0: 1831.1, 1: 1817.2. Samples: 23442254. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-11 20:53:41,035][70582] Avg episode reward: [(0, '178.270'), (1, '118.440')] [2023-10-11 20:53:41,107][71601] Updated weights for policy 0, policy_version 45790 (0.0008) [2023-10-11 20:53:43,140][71635] Updated weights for policy 1, policy_version 45762 (0.0008) [2023-10-11 20:53:43,494][71635] Updated weights for policy 1, policy_version 45772 (0.0008) [2023-10-11 20:53:43,867][71635] Updated weights for policy 1, policy_version 45782 (0.0009) [2023-10-11 20:53:44,227][71635] Updated weights for policy 1, policy_version 45792 (0.0009) [2023-10-11 20:53:44,710][71601] Updated weights for policy 0, policy_version 45800 (0.0008) [2023-10-11 20:53:45,079][71601] Updated weights for policy 0, policy_version 45810 (0.0008) [2023-10-11 20:53:45,459][71601] Updated weights for policy 0, policy_version 45820 (0.0007) [2023-10-11 20:53:46,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 93814784. Throughput: 0: 1838.1, 1: 1812.6. Samples: 23453846. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-11 20:53:46,035][70582] Avg episode reward: [(0, '177.750'), (1, '118.830')] [2023-10-11 20:53:48,006][71635] Updated weights for policy 1, policy_version 45802 (0.0009) [2023-10-11 20:53:48,381][71635] Updated weights for policy 1, policy_version 45812 (0.0008) [2023-10-11 20:53:48,744][71635] Updated weights for policy 1, policy_version 45822 (0.0008) [2023-10-11 20:53:49,221][71601] Updated weights for policy 0, policy_version 45830 (0.0010) [2023-10-11 20:53:49,586][71601] Updated weights for policy 0, policy_version 45840 (0.0007) [2023-10-11 20:53:49,958][71601] Updated weights for policy 0, policy_version 45850 (0.0007) [2023-10-11 20:53:51,034][70582] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 93880320. Throughput: 0: 1838.3, 1: 1813.4. Samples: 23474982. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-11 20:53:51,034][70582] Avg episode reward: [(0, '177.750'), (1, '124.450')] [2023-10-11 20:53:52,393][71635] Updated weights for policy 1, policy_version 45832 (0.0009) [2023-10-11 20:53:52,764][71635] Updated weights for policy 1, policy_version 45842 (0.0009) [2023-10-11 20:53:53,135][71635] Updated weights for policy 1, policy_version 45852 (0.0009) [2023-10-11 20:53:53,429][71601] Updated weights for policy 0, policy_version 45860 (0.0007) [2023-10-11 20:53:53,797][71601] Updated weights for policy 0, policy_version 45870 (0.0008) [2023-10-11 20:53:54,170][71601] Updated weights for policy 0, policy_version 45880 (0.0009) [2023-10-11 20:53:56,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 93945856. Throughput: 0: 1846.0, 1: 1813.7. Samples: 23497404. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-11 20:53:56,034][70582] Avg episode reward: [(0, '180.250'), (1, '123.550')] [2023-10-11 20:53:56,045][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000045856_46956544.pth... [2023-10-11 20:53:56,045][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000045888_46989312.pth... [2023-10-11 20:53:56,080][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000044192_45252608.pth [2023-10-11 20:53:56,087][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000044160_45219840.pth [2023-10-11 20:53:56,924][71635] Updated weights for policy 1, policy_version 45862 (0.0008) [2023-10-11 20:53:57,296][71635] Updated weights for policy 1, policy_version 45872 (0.0008) [2023-10-11 20:53:57,666][71635] Updated weights for policy 1, policy_version 45882 (0.0010) [2023-10-11 20:53:57,983][71601] Updated weights for policy 0, policy_version 45890 (0.0010) [2023-10-11 20:53:58,382][71601] Updated weights for policy 0, policy_version 45900 (0.0008) [2023-10-11 20:53:58,755][71601] Updated weights for policy 0, policy_version 45910 (0.0008) [2023-10-11 20:53:59,128][71601] Updated weights for policy 0, policy_version 45920 (0.0008) [2023-10-11 20:54:01,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 94011392. Throughput: 0: 1839.0, 1: 1813.9. Samples: 23508302. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-11 20:54:01,035][70582] Avg episode reward: [(0, '195.670'), (1, '125.540')] [2023-10-11 20:54:01,304][71635] Updated weights for policy 1, policy_version 45892 (0.0008) [2023-10-11 20:54:01,668][71635] Updated weights for policy 1, policy_version 45902 (0.0009) [2023-10-11 20:54:02,027][71635] Updated weights for policy 1, policy_version 45912 (0.0009) [2023-10-11 20:54:02,753][71601] Updated weights for policy 0, policy_version 45930 (0.0008) [2023-10-11 20:54:03,121][71601] Updated weights for policy 0, policy_version 45940 (0.0007) [2023-10-11 20:54:03,498][71601] Updated weights for policy 0, policy_version 45950 (0.0007) [2023-10-11 20:54:05,775][71635] Updated weights for policy 1, policy_version 45922 (0.0009) [2023-10-11 20:54:06,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 94076928. Throughput: 0: 1843.8, 1: 1813.0. Samples: 23530178. Policy #0 lag: (min: 16.0, avg: 31.2, max: 32.0) [2023-10-11 20:54:06,034][70582] Avg episode reward: [(0, '193.420'), (1, '131.240')] [2023-10-11 20:54:06,135][71635] Updated weights for policy 1, policy_version 45932 (0.0010) [2023-10-11 20:54:06,500][71635] Updated weights for policy 1, policy_version 45942 (0.0009) [2023-10-11 20:54:06,873][71635] Updated weights for policy 1, policy_version 45952 (0.0008) [2023-10-11 20:54:07,028][71601] Updated weights for policy 0, policy_version 45960 (0.0008) [2023-10-11 20:54:07,402][71601] Updated weights for policy 0, policy_version 45970 (0.0010) [2023-10-11 20:54:07,773][71601] Updated weights for policy 0, policy_version 45980 (0.0010) [2023-10-11 20:54:10,718][71635] Updated weights for policy 1, policy_version 45962 (0.0007) [2023-10-11 20:54:11,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 94142464. Throughput: 0: 1847.8, 1: 1809.5. Samples: 23552562. Policy #0 lag: (min: 16.0, avg: 31.2, max: 32.0) [2023-10-11 20:54:11,034][70582] Avg episode reward: [(0, '201.910'), (1, '129.020')] [2023-10-11 20:54:11,087][71635] Updated weights for policy 1, policy_version 45972 (0.0007) [2023-10-11 20:54:11,314][71601] Updated weights for policy 0, policy_version 45990 (0.0007) [2023-10-11 20:54:11,448][71635] Updated weights for policy 1, policy_version 45982 (0.0008) [2023-10-11 20:54:11,688][71601] Updated weights for policy 0, policy_version 46000 (0.0007) [2023-10-11 20:54:12,053][71601] Updated weights for policy 0, policy_version 46010 (0.0008) [2023-10-11 20:54:15,193][71635] Updated weights for policy 1, policy_version 45992 (0.0009) [2023-10-11 20:54:15,563][71635] Updated weights for policy 1, policy_version 46002 (0.0009) [2023-10-11 20:54:15,806][71601] Updated weights for policy 0, policy_version 46020 (0.0007) [2023-10-11 20:54:15,927][71635] Updated weights for policy 1, policy_version 46012 (0.0007) [2023-10-11 20:54:16,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 94208000. Throughput: 0: 1848.1, 1: 1807.3. Samples: 23562616. Policy #0 lag: (min: 16.0, avg: 31.2, max: 32.0) [2023-10-11 20:54:16,034][70582] Avg episode reward: [(0, '173.310'), (1, '128.840')] [2023-10-11 20:54:16,183][71601] Updated weights for policy 0, policy_version 46030 (0.0008) [2023-10-11 20:54:16,545][71601] Updated weights for policy 0, policy_version 46040 (0.0008) [2023-10-11 20:54:19,607][71635] Updated weights for policy 1, policy_version 46022 (0.0009) [2023-10-11 20:54:19,985][71635] Updated weights for policy 1, policy_version 46032 (0.0010) [2023-10-11 20:54:20,162][71601] Updated weights for policy 0, policy_version 46050 (0.0007) [2023-10-11 20:54:20,349][71635] Updated weights for policy 1, policy_version 46042 (0.0007) [2023-10-11 20:54:20,526][71601] Updated weights for policy 0, policy_version 46060 (0.0007) [2023-10-11 20:54:20,892][71601] Updated weights for policy 0, policy_version 46070 (0.0010) [2023-10-11 20:54:21,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 94306304. Throughput: 0: 1837.8, 1: 1813.7. Samples: 23585062. Policy #0 lag: (min: 16.0, avg: 31.2, max: 32.0) [2023-10-11 20:54:21,034][70582] Avg episode reward: [(0, '173.270'), (1, '129.680')] [2023-10-11 20:54:21,261][71601] Updated weights for policy 0, policy_version 46080 (0.0008) [2023-10-11 20:54:24,091][71635] Updated weights for policy 1, policy_version 46052 (0.0008) [2023-10-11 20:54:24,453][71635] Updated weights for policy 1, policy_version 46062 (0.0008) [2023-10-11 20:54:24,829][71635] Updated weights for policy 1, policy_version 46072 (0.0007) [2023-10-11 20:54:25,069][71601] Updated weights for policy 0, policy_version 46090 (0.0008) [2023-10-11 20:54:25,447][71601] Updated weights for policy 0, policy_version 46100 (0.0008) [2023-10-11 20:54:25,814][71601] Updated weights for policy 0, policy_version 46110 (0.0007) [2023-10-11 20:54:26,034][70582] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 94404608. Throughput: 0: 1826.8, 1: 1799.7. Samples: 23605446. Policy #0 lag: (min: 16.0, avg: 31.2, max: 32.0) [2023-10-11 20:54:26,035][70582] Avg episode reward: [(0, '168.020'), (1, '136.160')] [2023-10-11 20:54:28,644][71635] Updated weights for policy 1, policy_version 46082 (0.0008) [2023-10-11 20:54:29,002][71635] Updated weights for policy 1, policy_version 46092 (0.0009) [2023-10-11 20:54:29,380][71635] Updated weights for policy 1, policy_version 46102 (0.0008) [2023-10-11 20:54:29,551][71601] Updated weights for policy 0, policy_version 46120 (0.0009) [2023-10-11 20:54:29,741][71635] Updated weights for policy 1, policy_version 46112 (0.0008) [2023-10-11 20:54:29,924][71601] Updated weights for policy 0, policy_version 46130 (0.0010) [2023-10-11 20:54:30,299][71601] Updated weights for policy 0, policy_version 46140 (0.0010) [2023-10-11 20:54:31,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 94470144. Throughput: 0: 1828.9, 1: 1808.1. Samples: 23617512. Policy #0 lag: (min: 16.0, avg: 31.2, max: 32.0) [2023-10-11 20:54:31,034][70582] Avg episode reward: [(0, '167.600'), (1, '136.710')] [2023-10-11 20:54:33,408][71635] Updated weights for policy 1, policy_version 46122 (0.0010) [2023-10-11 20:54:33,770][71635] Updated weights for policy 1, policy_version 46132 (0.0010) [2023-10-11 20:54:34,054][71601] Updated weights for policy 0, policy_version 46150 (0.0009) [2023-10-11 20:54:34,128][71635] Updated weights for policy 1, policy_version 46142 (0.0007) [2023-10-11 20:54:34,424][71601] Updated weights for policy 0, policy_version 46160 (0.0010) [2023-10-11 20:54:34,796][71601] Updated weights for policy 0, policy_version 46170 (0.0010) [2023-10-11 20:54:36,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 94535680. Throughput: 0: 1817.5, 1: 1804.6. Samples: 23637978. Policy #0 lag: (min: 0.0, avg: 22.6, max: 32.0) [2023-10-11 20:54:36,035][70582] Avg episode reward: [(0, '171.320'), (1, '141.440')] [2023-10-11 20:54:37,787][71635] Updated weights for policy 1, policy_version 46152 (0.0008) [2023-10-11 20:54:38,152][71635] Updated weights for policy 1, policy_version 46162 (0.0009) [2023-10-11 20:54:38,515][71635] Updated weights for policy 1, policy_version 46172 (0.0010) [2023-10-11 20:54:38,543][71601] Updated weights for policy 0, policy_version 46180 (0.0008) [2023-10-11 20:54:38,918][71601] Updated weights for policy 0, policy_version 46190 (0.0010) [2023-10-11 20:54:39,285][71601] Updated weights for policy 0, policy_version 46200 (0.0009) [2023-10-11 20:54:41,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 94601216. Throughput: 0: 1815.1, 1: 1799.1. Samples: 23660040. Policy #0 lag: (min: 0.0, avg: 22.6, max: 32.0) [2023-10-11 20:54:41,034][70582] Avg episode reward: [(0, '167.690'), (1, '141.440')] [2023-10-11 20:54:42,302][71635] Updated weights for policy 1, policy_version 46182 (0.0008) [2023-10-11 20:54:42,670][71635] Updated weights for policy 1, policy_version 46192 (0.0009) [2023-10-11 20:54:42,996][71601] Updated weights for policy 0, policy_version 46210 (0.0009) [2023-10-11 20:54:43,050][71635] Updated weights for policy 1, policy_version 46202 (0.0008) [2023-10-11 20:54:43,401][71601] Updated weights for policy 0, policy_version 46220 (0.0008) [2023-10-11 20:54:43,769][71601] Updated weights for policy 0, policy_version 46230 (0.0009) [2023-10-11 20:54:44,138][71601] Updated weights for policy 0, policy_version 46240 (0.0008) [2023-10-11 20:54:46,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 94666752. Throughput: 0: 1813.8, 1: 1795.6. Samples: 23670728. Policy #0 lag: (min: 0.0, avg: 22.6, max: 32.0) [2023-10-11 20:54:46,034][70582] Avg episode reward: [(0, '163.880'), (1, '149.110')] [2023-10-11 20:54:46,687][71635] Updated weights for policy 1, policy_version 46212 (0.0008) [2023-10-11 20:54:47,050][71635] Updated weights for policy 1, policy_version 46222 (0.0008) [2023-10-11 20:54:47,418][71635] Updated weights for policy 1, policy_version 46232 (0.0009) [2023-10-11 20:54:47,898][71601] Updated weights for policy 0, policy_version 46250 (0.0007) [2023-10-11 20:54:48,269][71601] Updated weights for policy 0, policy_version 46260 (0.0008) [2023-10-11 20:54:48,641][71601] Updated weights for policy 0, policy_version 46270 (0.0009) [2023-10-11 20:54:51,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 94732288. Throughput: 0: 1809.8, 1: 1800.4. Samples: 23692636. Policy #0 lag: (min: 0.0, avg: 22.6, max: 32.0) [2023-10-11 20:54:51,034][70582] Avg episode reward: [(0, '177.100'), (1, '153.060')] [2023-10-11 20:54:51,087][71635] Updated weights for policy 1, policy_version 46242 (0.0009) [2023-10-11 20:54:51,453][71635] Updated weights for policy 1, policy_version 46252 (0.0009) [2023-10-11 20:54:51,827][71635] Updated weights for policy 1, policy_version 46262 (0.0011) [2023-10-11 20:54:52,196][71635] Updated weights for policy 1, policy_version 46272 (0.0007) [2023-10-11 20:54:52,212][71601] Updated weights for policy 0, policy_version 46280 (0.0007) [2023-10-11 20:54:52,592][71601] Updated weights for policy 0, policy_version 46290 (0.0007) [2023-10-11 20:54:52,970][71601] Updated weights for policy 0, policy_version 46300 (0.0009) [2023-10-11 20:54:55,875][71635] Updated weights for policy 1, policy_version 46282 (0.0007) [2023-10-11 20:54:56,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 94797824. Throughput: 0: 1808.8, 1: 1817.5. Samples: 23715748. Policy #0 lag: (min: 0.0, avg: 22.6, max: 32.0) [2023-10-11 20:54:56,035][70582] Avg episode reward: [(0, '177.000'), (1, '141.700')] [2023-10-11 20:54:56,245][71635] Updated weights for policy 1, policy_version 46292 (0.0008) [2023-10-11 20:54:56,612][71635] Updated weights for policy 1, policy_version 46302 (0.0008) [2023-10-11 20:54:56,742][71601] Updated weights for policy 0, policy_version 46310 (0.0009) [2023-10-11 20:54:57,122][71601] Updated weights for policy 0, policy_version 46320 (0.0008) [2023-10-11 20:54:57,493][71601] Updated weights for policy 0, policy_version 46330 (0.0009) [2023-10-11 20:55:00,244][71635] Updated weights for policy 1, policy_version 46312 (0.0008) [2023-10-11 20:55:00,611][71635] Updated weights for policy 1, policy_version 46322 (0.0010) [2023-10-11 20:55:00,985][71635] Updated weights for policy 1, policy_version 46332 (0.0010) [2023-10-11 20:55:01,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 94863360. Throughput: 0: 1811.9, 1: 1808.9. Samples: 23725552. Policy #0 lag: (min: 0.0, avg: 22.6, max: 32.0) [2023-10-11 20:55:01,034][70582] Avg episode reward: [(0, '175.090'), (1, '137.240')] [2023-10-11 20:55:01,107][71601] Updated weights for policy 0, policy_version 46340 (0.0007) [2023-10-11 20:55:01,479][71601] Updated weights for policy 0, policy_version 46350 (0.0007) [2023-10-11 20:55:01,854][71601] Updated weights for policy 0, policy_version 46360 (0.0007) [2023-10-11 20:55:04,601][71635] Updated weights for policy 1, policy_version 46342 (0.0008) [2023-10-11 20:55:04,970][71635] Updated weights for policy 1, policy_version 46352 (0.0008) [2023-10-11 20:55:05,335][71635] Updated weights for policy 1, policy_version 46362 (0.0007) [2023-10-11 20:55:05,525][71601] Updated weights for policy 0, policy_version 46370 (0.0007) [2023-10-11 20:55:05,893][71601] Updated weights for policy 0, policy_version 46380 (0.0010) [2023-10-11 20:55:06,034][70582] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 94961664. Throughput: 0: 1813.3, 1: 1819.1. Samples: 23748520. Policy #0 lag: (min: 0.0, avg: 22.6, max: 32.0) [2023-10-11 20:55:06,034][70582] Avg episode reward: [(0, '161.090'), (1, '136.430')] [2023-10-11 20:55:06,260][71601] Updated weights for policy 0, policy_version 46390 (0.0008) [2023-10-11 20:55:06,630][71601] Updated weights for policy 0, policy_version 46400 (0.0008) [2023-10-11 20:55:09,057][71635] Updated weights for policy 1, policy_version 46372 (0.0007) [2023-10-11 20:55:09,421][71635] Updated weights for policy 1, policy_version 46382 (0.0008) [2023-10-11 20:55:09,787][71635] Updated weights for policy 1, policy_version 46392 (0.0012) [2023-10-11 20:55:10,399][71601] Updated weights for policy 0, policy_version 46410 (0.0007) [2023-10-11 20:55:10,772][71601] Updated weights for policy 0, policy_version 46420 (0.0007) [2023-10-11 20:55:11,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 95027200. Throughput: 0: 1823.1, 1: 1820.5. Samples: 23769410. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:55:11,034][70582] Avg episode reward: [(0, '167.110'), (1, '123.730')] [2023-10-11 20:55:11,142][71601] Updated weights for policy 0, policy_version 46430 (0.0007) [2023-10-11 20:55:13,537][71635] Updated weights for policy 1, policy_version 46402 (0.0009) [2023-10-11 20:55:13,898][71635] Updated weights for policy 1, policy_version 46412 (0.0007) [2023-10-11 20:55:14,251][71635] Updated weights for policy 1, policy_version 46422 (0.0008) [2023-10-11 20:55:14,618][71635] Updated weights for policy 1, policy_version 46432 (0.0009) [2023-10-11 20:55:14,628][71601] Updated weights for policy 0, policy_version 46440 (0.0009) [2023-10-11 20:55:14,998][71601] Updated weights for policy 0, policy_version 46450 (0.0010) [2023-10-11 20:55:15,370][71601] Updated weights for policy 0, policy_version 46460 (0.0010) [2023-10-11 20:55:16,034][70582] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 95125504. Throughput: 0: 1819.7, 1: 1824.4. Samples: 23781496. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:55:16,034][70582] Avg episode reward: [(0, '157.530'), (1, '119.200')] [2023-10-11 20:55:18,552][71635] Updated weights for policy 1, policy_version 46442 (0.0007) [2023-10-11 20:55:18,922][71635] Updated weights for policy 1, policy_version 46452 (0.0008) [2023-10-11 20:55:18,977][71601] Updated weights for policy 0, policy_version 46470 (0.0009) [2023-10-11 20:55:19,287][71635] Updated weights for policy 1, policy_version 46462 (0.0007) [2023-10-11 20:55:19,337][71601] Updated weights for policy 0, policy_version 46480 (0.0008) [2023-10-11 20:55:19,708][71601] Updated weights for policy 0, policy_version 46490 (0.0010) [2023-10-11 20:55:21,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 95191040. Throughput: 0: 1825.1, 1: 1816.5. Samples: 23801850. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:55:21,034][70582] Avg episode reward: [(0, '154.710'), (1, '119.330')] [2023-10-11 20:55:22,934][71635] Updated weights for policy 1, policy_version 46472 (0.0007) [2023-10-11 20:55:23,304][71635] Updated weights for policy 1, policy_version 46482 (0.0008) [2023-10-11 20:55:23,446][71601] Updated weights for policy 0, policy_version 46500 (0.0009) [2023-10-11 20:55:23,661][71635] Updated weights for policy 1, policy_version 46492 (0.0007) [2023-10-11 20:55:23,808][71601] Updated weights for policy 0, policy_version 46510 (0.0007) [2023-10-11 20:55:24,180][71601] Updated weights for policy 0, policy_version 46520 (0.0008) [2023-10-11 20:55:26,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 95256576. Throughput: 0: 1829.6, 1: 1814.5. Samples: 23824026. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:55:26,035][70582] Avg episode reward: [(0, '154.580'), (1, '122.080')] [2023-10-11 20:55:27,337][71635] Updated weights for policy 1, policy_version 46502 (0.0008) [2023-10-11 20:55:27,701][71635] Updated weights for policy 1, policy_version 46512 (0.0007) [2023-10-11 20:55:27,900][71601] Updated weights for policy 0, policy_version 46530 (0.0007) [2023-10-11 20:55:28,064][71635] Updated weights for policy 1, policy_version 46522 (0.0008) [2023-10-11 20:55:28,292][71601] Updated weights for policy 0, policy_version 46540 (0.0008) [2023-10-11 20:55:28,659][71601] Updated weights for policy 0, policy_version 46550 (0.0007) [2023-10-11 20:55:29,027][71601] Updated weights for policy 0, policy_version 46560 (0.0008) [2023-10-11 20:55:31,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 95322112. Throughput: 0: 1822.1, 1: 1818.4. Samples: 23834548. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:55:31,034][70582] Avg episode reward: [(0, '167.240'), (1, '124.430')] [2023-10-11 20:55:31,676][71635] Updated weights for policy 1, policy_version 46532 (0.0010) [2023-10-11 20:55:32,049][71635] Updated weights for policy 1, policy_version 46542 (0.0008) [2023-10-11 20:55:32,406][71635] Updated weights for policy 1, policy_version 46552 (0.0009) [2023-10-11 20:55:32,648][71601] Updated weights for policy 0, policy_version 46570 (0.0007) [2023-10-11 20:55:33,010][71601] Updated weights for policy 0, policy_version 46580 (0.0008) [2023-10-11 20:55:33,377][71601] Updated weights for policy 0, policy_version 46590 (0.0007) [2023-10-11 20:55:36,024][71635] Updated weights for policy 1, policy_version 46562 (0.0007) [2023-10-11 20:55:36,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 95387648. Throughput: 0: 1830.7, 1: 1818.0. Samples: 23856832. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:55:36,035][70582] Avg episode reward: [(0, '167.170'), (1, '125.150')] [2023-10-11 20:55:36,390][71635] Updated weights for policy 1, policy_version 46572 (0.0012) [2023-10-11 20:55:36,760][71635] Updated weights for policy 1, policy_version 46582 (0.0009) [2023-10-11 20:55:36,979][71601] Updated weights for policy 0, policy_version 46600 (0.0008) [2023-10-11 20:55:37,129][71635] Updated weights for policy 1, policy_version 46592 (0.0009) [2023-10-11 20:55:37,345][71601] Updated weights for policy 0, policy_version 46610 (0.0008) [2023-10-11 20:55:37,723][71601] Updated weights for policy 0, policy_version 46620 (0.0008) [2023-10-11 20:55:40,844][71635] Updated weights for policy 1, policy_version 46602 (0.0010) [2023-10-11 20:55:41,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 95453184. Throughput: 0: 1839.7, 1: 1811.9. Samples: 23880068. Policy #0 lag: (min: 26.0, avg: 34.0, max: 58.0) [2023-10-11 20:55:41,034][70582] Avg episode reward: [(0, '173.630'), (1, '126.530')] [2023-10-11 20:55:41,221][71635] Updated weights for policy 1, policy_version 46612 (0.0009) [2023-10-11 20:55:41,440][71601] Updated weights for policy 0, policy_version 46630 (0.0008) [2023-10-11 20:55:41,584][71635] Updated weights for policy 1, policy_version 46622 (0.0008) [2023-10-11 20:55:41,805][71601] Updated weights for policy 0, policy_version 46640 (0.0007) [2023-10-11 20:55:42,184][71601] Updated weights for policy 0, policy_version 46650 (0.0008) [2023-10-11 20:55:45,247][71635] Updated weights for policy 1, policy_version 46632 (0.0010) [2023-10-11 20:55:45,617][71635] Updated weights for policy 1, policy_version 46642 (0.0008) [2023-10-11 20:55:45,774][71601] Updated weights for policy 0, policy_version 46660 (0.0008) [2023-10-11 20:55:45,976][71635] Updated weights for policy 1, policy_version 46652 (0.0007) [2023-10-11 20:55:46,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 95518720. Throughput: 0: 1836.6, 1: 1815.5. Samples: 23889900. Policy #0 lag: (min: 26.0, avg: 34.0, max: 58.0) [2023-10-11 20:55:46,035][70582] Avg episode reward: [(0, '173.330'), (1, '118.840')] [2023-10-11 20:55:46,152][71601] Updated weights for policy 0, policy_version 46670 (0.0008) [2023-10-11 20:55:46,517][71601] Updated weights for policy 0, policy_version 46680 (0.0010) [2023-10-11 20:55:49,790][71635] Updated weights for policy 1, policy_version 46662 (0.0009) [2023-10-11 20:55:50,103][71601] Updated weights for policy 0, policy_version 46690 (0.0009) [2023-10-11 20:55:50,150][71635] Updated weights for policy 1, policy_version 46672 (0.0009) [2023-10-11 20:55:50,474][71601] Updated weights for policy 0, policy_version 46700 (0.0009) [2023-10-11 20:55:50,508][71635] Updated weights for policy 1, policy_version 46682 (0.0008) [2023-10-11 20:55:50,836][71601] Updated weights for policy 0, policy_version 46710 (0.0008) [2023-10-11 20:55:51,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 95617024. Throughput: 0: 1839.1, 1: 1810.2. Samples: 23912736. Policy #0 lag: (min: 26.0, avg: 34.0, max: 58.0) [2023-10-11 20:55:51,034][70582] Avg episode reward: [(0, '178.260'), (1, '132.780')] [2023-10-11 20:55:51,206][71601] Updated weights for policy 0, policy_version 46720 (0.0007) [2023-10-11 20:55:54,216][71635] Updated weights for policy 1, policy_version 46692 (0.0010) [2023-10-11 20:55:54,584][71635] Updated weights for policy 1, policy_version 46702 (0.0010) [2023-10-11 20:55:54,819][71601] Updated weights for policy 0, policy_version 46730 (0.0008) [2023-10-11 20:55:54,950][71635] Updated weights for policy 1, policy_version 46712 (0.0009) [2023-10-11 20:55:55,182][71601] Updated weights for policy 0, policy_version 46740 (0.0008) [2023-10-11 20:55:55,558][71601] Updated weights for policy 0, policy_version 46750 (0.0009) [2023-10-11 20:55:56,034][70582] Fps is (10 sec: 19660.9, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 95715328. Throughput: 0: 1823.2, 1: 1811.2. Samples: 23932954. Policy #0 lag: (min: 26.0, avg: 34.0, max: 58.0) [2023-10-11 20:55:56,034][70582] Avg episode reward: [(0, '178.260'), (1, '123.560')] [2023-10-11 20:55:56,044][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000046720_47841280.pth... [2023-10-11 20:55:56,044][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000046752_47874048.pth... [2023-10-11 20:55:56,077][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000045024_46104576.pth [2023-10-11 20:55:56,081][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000045024_46104576.pth [2023-10-11 20:55:58,791][71635] Updated weights for policy 1, policy_version 46722 (0.0008) [2023-10-11 20:55:59,155][71635] Updated weights for policy 1, policy_version 46732 (0.0007) [2023-10-11 20:55:59,265][71601] Updated weights for policy 0, policy_version 46760 (0.0009) [2023-10-11 20:55:59,520][71635] Updated weights for policy 1, policy_version 46742 (0.0008) [2023-10-11 20:55:59,623][71601] Updated weights for policy 0, policy_version 46770 (0.0008) [2023-10-11 20:55:59,887][71635] Updated weights for policy 1, policy_version 46752 (0.0007) [2023-10-11 20:56:00,003][71601] Updated weights for policy 0, policy_version 46780 (0.0007) [2023-10-11 20:56:01,034][70582] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 95780864. Throughput: 0: 1834.2, 1: 1809.4. Samples: 23945456. Policy #0 lag: (min: 26.0, avg: 34.0, max: 58.0) [2023-10-11 20:56:01,034][70582] Avg episode reward: [(0, '183.090'), (1, '136.890')] [2023-10-11 20:56:03,613][71601] Updated weights for policy 0, policy_version 46790 (0.0007) [2023-10-11 20:56:03,643][71635] Updated weights for policy 1, policy_version 46762 (0.0008) [2023-10-11 20:56:03,988][71601] Updated weights for policy 0, policy_version 46800 (0.0008) [2023-10-11 20:56:04,012][71635] Updated weights for policy 1, policy_version 46772 (0.0009) [2023-10-11 20:56:04,356][71601] Updated weights for policy 0, policy_version 46810 (0.0007) [2023-10-11 20:56:04,369][71635] Updated weights for policy 1, policy_version 46782 (0.0008) [2023-10-11 20:56:06,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 95846400. Throughput: 0: 1822.9, 1: 1816.9. Samples: 23965644. Policy #0 lag: (min: 26.0, avg: 34.0, max: 58.0) [2023-10-11 20:56:06,035][70582] Avg episode reward: [(0, '188.400'), (1, '135.370')] [2023-10-11 20:56:08,038][71635] Updated weights for policy 1, policy_version 46792 (0.0008) [2023-10-11 20:56:08,197][71601] Updated weights for policy 0, policy_version 46820 (0.0008) [2023-10-11 20:56:08,409][71635] Updated weights for policy 1, policy_version 46802 (0.0008) [2023-10-11 20:56:08,571][71601] Updated weights for policy 0, policy_version 46830 (0.0008) [2023-10-11 20:56:08,790][71635] Updated weights for policy 1, policy_version 46812 (0.0007) [2023-10-11 20:56:08,934][71601] Updated weights for policy 0, policy_version 46840 (0.0008) [2023-10-11 20:56:11,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 95911936. Throughput: 0: 1826.3, 1: 1811.2. Samples: 23987712. Policy #0 lag: (min: 24.0, avg: 45.5, max: 56.0) [2023-10-11 20:56:11,035][70582] Avg episode reward: [(0, '188.410'), (1, '140.960')] [2023-10-11 20:56:12,424][71635] Updated weights for policy 1, policy_version 46822 (0.0008) [2023-10-11 20:56:12,625][71601] Updated weights for policy 0, policy_version 46850 (0.0009) [2023-10-11 20:56:12,796][71635] Updated weights for policy 1, policy_version 46832 (0.0009) [2023-10-11 20:56:12,991][71601] Updated weights for policy 0, policy_version 46860 (0.0008) [2023-10-11 20:56:13,169][71635] Updated weights for policy 1, policy_version 46842 (0.0008) [2023-10-11 20:56:13,357][71601] Updated weights for policy 0, policy_version 46870 (0.0007) [2023-10-11 20:56:13,732][71601] Updated weights for policy 0, policy_version 46880 (0.0008) [2023-10-11 20:56:16,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 95977472. Throughput: 0: 1823.2, 1: 1809.8. Samples: 23998030. Policy #0 lag: (min: 24.0, avg: 45.5, max: 56.0) [2023-10-11 20:56:16,034][70582] Avg episode reward: [(0, '188.180'), (1, '122.640')] [2023-10-11 20:56:16,869][71635] Updated weights for policy 1, policy_version 46852 (0.0009) [2023-10-11 20:56:17,237][71635] Updated weights for policy 1, policy_version 46862 (0.0010) [2023-10-11 20:56:17,546][71601] Updated weights for policy 0, policy_version 46890 (0.0007) [2023-10-11 20:56:17,596][71635] Updated weights for policy 1, policy_version 46872 (0.0008) [2023-10-11 20:56:17,925][71601] Updated weights for policy 0, policy_version 46900 (0.0009) [2023-10-11 20:56:18,294][71601] Updated weights for policy 0, policy_version 46910 (0.0010) [2023-10-11 20:56:21,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 96043008. Throughput: 0: 1818.5, 1: 1804.1. Samples: 24019846. Policy #0 lag: (min: 24.0, avg: 45.5, max: 56.0) [2023-10-11 20:56:21,034][70582] Avg episode reward: [(0, '188.180'), (1, '110.360')] [2023-10-11 20:56:21,393][71635] Updated weights for policy 1, policy_version 46882 (0.0007) [2023-10-11 20:56:21,760][71635] Updated weights for policy 1, policy_version 46892 (0.0008) [2023-10-11 20:56:22,126][71635] Updated weights for policy 1, policy_version 46902 (0.0008) [2023-10-11 20:56:22,173][71601] Updated weights for policy 0, policy_version 46920 (0.0007) [2023-10-11 20:56:22,484][71635] Updated weights for policy 1, policy_version 46912 (0.0007) [2023-10-11 20:56:22,541][71601] Updated weights for policy 0, policy_version 46930 (0.0008) [2023-10-11 20:56:22,902][71601] Updated weights for policy 0, policy_version 46940 (0.0007) [2023-10-11 20:56:26,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 96108544. Throughput: 0: 1802.3, 1: 1807.4. Samples: 24042506. Policy #0 lag: (min: 24.0, avg: 45.5, max: 56.0) [2023-10-11 20:56:26,035][70582] Avg episode reward: [(0, '195.540'), (1, '117.720')] [2023-10-11 20:56:26,365][71635] Updated weights for policy 1, policy_version 46922 (0.0010) [2023-10-11 20:56:26,598][71601] Updated weights for policy 0, policy_version 46950 (0.0008) [2023-10-11 20:56:26,735][71635] Updated weights for policy 1, policy_version 46932 (0.0008) [2023-10-11 20:56:26,967][71601] Updated weights for policy 0, policy_version 46960 (0.0007) [2023-10-11 20:56:27,097][71635] Updated weights for policy 1, policy_version 46942 (0.0008) [2023-10-11 20:56:27,333][71601] Updated weights for policy 0, policy_version 46970 (0.0008) [2023-10-11 20:56:30,592][71635] Updated weights for policy 1, policy_version 46952 (0.0009) [2023-10-11 20:56:30,966][71635] Updated weights for policy 1, policy_version 46962 (0.0009) [2023-10-11 20:56:31,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 96174080. Throughput: 0: 1800.9, 1: 1810.0. Samples: 24052388. Policy #0 lag: (min: 24.0, avg: 45.5, max: 56.0) [2023-10-11 20:56:31,034][71601] Updated weights for policy 0, policy_version 46980 (0.0009) [2023-10-11 20:56:31,035][70582] Avg episode reward: [(0, '187.870'), (1, '106.850')] [2023-10-11 20:56:31,334][71635] Updated weights for policy 1, policy_version 46972 (0.0008) [2023-10-11 20:56:31,408][71601] Updated weights for policy 0, policy_version 46990 (0.0007) [2023-10-11 20:56:31,773][71601] Updated weights for policy 0, policy_version 47000 (0.0007) [2023-10-11 20:56:34,962][71635] Updated weights for policy 1, policy_version 46982 (0.0009) [2023-10-11 20:56:35,337][71635] Updated weights for policy 1, policy_version 46992 (0.0009) [2023-10-11 20:56:35,416][71601] Updated weights for policy 0, policy_version 47010 (0.0008) [2023-10-11 20:56:35,701][71635] Updated weights for policy 1, policy_version 47002 (0.0007) [2023-10-11 20:56:35,777][71601] Updated weights for policy 0, policy_version 47020 (0.0007) [2023-10-11 20:56:36,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 96272384. Throughput: 0: 1798.7, 1: 1811.2. Samples: 24075184. Policy #0 lag: (min: 24.0, avg: 45.5, max: 56.0) [2023-10-11 20:56:36,035][70582] Avg episode reward: [(0, '187.800'), (1, '112.530')] [2023-10-11 20:56:36,157][71601] Updated weights for policy 0, policy_version 47030 (0.0009) [2023-10-11 20:56:36,521][71601] Updated weights for policy 0, policy_version 47040 (0.0008) [2023-10-11 20:56:39,468][71635] Updated weights for policy 1, policy_version 47012 (0.0008) [2023-10-11 20:56:39,833][71635] Updated weights for policy 1, policy_version 47022 (0.0010) [2023-10-11 20:56:40,197][71635] Updated weights for policy 1, policy_version 47032 (0.0009) [2023-10-11 20:56:40,301][71601] Updated weights for policy 0, policy_version 47050 (0.0008) [2023-10-11 20:56:40,665][71601] Updated weights for policy 0, policy_version 47060 (0.0008) [2023-10-11 20:56:41,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 96337920. Throughput: 0: 1811.3, 1: 1815.2. Samples: 24096144. Policy #0 lag: (min: 24.0, avg: 45.5, max: 56.0) [2023-10-11 20:56:41,034][70582] Avg episode reward: [(0, '187.800'), (1, '123.870')] [2023-10-11 20:56:41,046][71601] Updated weights for policy 0, policy_version 47070 (0.0008) [2023-10-11 20:56:44,034][71635] Updated weights for policy 1, policy_version 47042 (0.0008) [2023-10-11 20:56:44,390][71635] Updated weights for policy 1, policy_version 47052 (0.0009) [2023-10-11 20:56:44,751][71635] Updated weights for policy 1, policy_version 47062 (0.0007) [2023-10-11 20:56:44,765][71601] Updated weights for policy 0, policy_version 47080 (0.0008) [2023-10-11 20:56:45,122][71635] Updated weights for policy 1, policy_version 47072 (0.0007) [2023-10-11 20:56:45,126][71601] Updated weights for policy 0, policy_version 47090 (0.0008) [2023-10-11 20:56:45,495][71601] Updated weights for policy 0, policy_version 47100 (0.0008) [2023-10-11 20:56:46,034][70582] Fps is (10 sec: 16384.5, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 96436224. Throughput: 0: 1803.4, 1: 1810.0. Samples: 24108060. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:56:46,034][70582] Avg episode reward: [(0, '187.800'), (1, '113.580')] [2023-10-11 20:56:48,717][71635] Updated weights for policy 1, policy_version 47082 (0.0007) [2023-10-11 20:56:49,040][71601] Updated weights for policy 0, policy_version 47110 (0.0009) [2023-10-11 20:56:49,091][71635] Updated weights for policy 1, policy_version 47092 (0.0007) [2023-10-11 20:56:49,413][71601] Updated weights for policy 0, policy_version 47120 (0.0008) [2023-10-11 20:56:49,454][71635] Updated weights for policy 1, policy_version 47102 (0.0007) [2023-10-11 20:56:49,791][71601] Updated weights for policy 0, policy_version 47130 (0.0007) [2023-10-11 20:56:51,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 96501760. Throughput: 0: 1813.7, 1: 1814.7. Samples: 24128920. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:56:51,034][70582] Avg episode reward: [(0, '187.800'), (1, '114.590')] [2023-10-11 20:56:53,223][71635] Updated weights for policy 1, policy_version 47112 (0.0007) [2023-10-11 20:56:53,449][71601] Updated weights for policy 0, policy_version 47140 (0.0008) [2023-10-11 20:56:53,585][71635] Updated weights for policy 1, policy_version 47122 (0.0007) [2023-10-11 20:56:53,812][71601] Updated weights for policy 0, policy_version 47150 (0.0008) [2023-10-11 20:56:53,940][71635] Updated weights for policy 1, policy_version 47132 (0.0007) [2023-10-11 20:56:54,190][71601] Updated weights for policy 0, policy_version 47160 (0.0009) [2023-10-11 20:56:56,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 96567296. Throughput: 0: 1809.9, 1: 1812.1. Samples: 24150702. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:56:56,035][70582] Avg episode reward: [(0, '187.800'), (1, '111.850')] [2023-10-11 20:56:57,647][71635] Updated weights for policy 1, policy_version 47142 (0.0008) [2023-10-11 20:56:57,792][71601] Updated weights for policy 0, policy_version 47170 (0.0007) [2023-10-11 20:56:58,015][71635] Updated weights for policy 1, policy_version 47152 (0.0008) [2023-10-11 20:56:58,162][71601] Updated weights for policy 0, policy_version 47180 (0.0009) [2023-10-11 20:56:58,379][71635] Updated weights for policy 1, policy_version 47162 (0.0009) [2023-10-11 20:56:58,537][71601] Updated weights for policy 0, policy_version 47190 (0.0007) [2023-10-11 20:56:58,919][71601] Updated weights for policy 0, policy_version 47200 (0.0007) [2023-10-11 20:57:01,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 96632832. Throughput: 0: 1817.5, 1: 1818.9. Samples: 24161666. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:57:01,034][70582] Avg episode reward: [(0, '191.390'), (1, '96.550')] [2023-10-11 20:57:02,266][71635] Updated weights for policy 1, policy_version 47172 (0.0008) [2023-10-11 20:57:02,633][71635] Updated weights for policy 1, policy_version 47182 (0.0009) [2023-10-11 20:57:02,645][71601] Updated weights for policy 0, policy_version 47210 (0.0009) [2023-10-11 20:57:02,997][71635] Updated weights for policy 1, policy_version 47192 (0.0007) [2023-10-11 20:57:03,007][71601] Updated weights for policy 0, policy_version 47220 (0.0008) [2023-10-11 20:57:03,386][71601] Updated weights for policy 0, policy_version 47230 (0.0008) [2023-10-11 20:57:06,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 96698368. Throughput: 0: 1817.1, 1: 1816.2. Samples: 24183344. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:57:06,034][70582] Avg episode reward: [(0, '191.480'), (1, '100.950')] [2023-10-11 20:57:06,611][71635] Updated weights for policy 1, policy_version 47202 (0.0007) [2023-10-11 20:57:06,973][71635] Updated weights for policy 1, policy_version 47212 (0.0008) [2023-10-11 20:57:07,161][71601] Updated weights for policy 0, policy_version 47240 (0.0008) [2023-10-11 20:57:07,343][71635] Updated weights for policy 1, policy_version 47222 (0.0008) [2023-10-11 20:57:07,531][71601] Updated weights for policy 0, policy_version 47250 (0.0008) [2023-10-11 20:57:07,710][71635] Updated weights for policy 1, policy_version 47232 (0.0009) [2023-10-11 20:57:07,899][71601] Updated weights for policy 0, policy_version 47260 (0.0008) [2023-10-11 20:57:11,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 96763904. Throughput: 0: 1818.5, 1: 1811.4. Samples: 24205854. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:57:11,034][70582] Avg episode reward: [(0, '204.660'), (1, '88.630')] [2023-10-11 20:57:11,493][71601] Updated weights for policy 0, policy_version 47270 (0.0007) [2023-10-11 20:57:11,569][71635] Updated weights for policy 1, policy_version 47242 (0.0008) [2023-10-11 20:57:11,866][71601] Updated weights for policy 0, policy_version 47280 (0.0009) [2023-10-11 20:57:11,942][71635] Updated weights for policy 1, policy_version 47252 (0.0010) [2023-10-11 20:57:12,240][71601] Updated weights for policy 0, policy_version 47290 (0.0007) [2023-10-11 20:57:12,316][71635] Updated weights for policy 1, policy_version 47262 (0.0008) [2023-10-11 20:57:15,997][71601] Updated weights for policy 0, policy_version 47300 (0.0009) [2023-10-11 20:57:16,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 96829440. Throughput: 0: 1820.1, 1: 1808.4. Samples: 24215666. Policy #0 lag: (min: 27.0, avg: 39.4, max: 59.0) [2023-10-11 20:57:16,034][70582] Avg episode reward: [(0, '204.730'), (1, '78.920')] [2023-10-11 20:57:16,068][71635] Updated weights for policy 1, policy_version 47272 (0.0007) [2023-10-11 20:57:16,364][71601] Updated weights for policy 0, policy_version 47310 (0.0008) [2023-10-11 20:57:16,429][71635] Updated weights for policy 1, policy_version 47282 (0.0007) [2023-10-11 20:57:16,736][71601] Updated weights for policy 0, policy_version 47320 (0.0008) [2023-10-11 20:57:16,794][71635] Updated weights for policy 1, policy_version 47292 (0.0008) [2023-10-11 20:57:20,464][71635] Updated weights for policy 1, policy_version 47302 (0.0008) [2023-10-11 20:57:20,497][71601] Updated weights for policy 0, policy_version 47330 (0.0007) [2023-10-11 20:57:20,820][71635] Updated weights for policy 1, policy_version 47312 (0.0009) [2023-10-11 20:57:20,865][71601] Updated weights for policy 0, policy_version 47340 (0.0007) [2023-10-11 20:57:21,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 96894976. Throughput: 0: 1816.5, 1: 1805.9. Samples: 24238192. Policy #0 lag: (min: 27.0, avg: 39.4, max: 59.0) [2023-10-11 20:57:21,034][70582] Avg episode reward: [(0, '215.000'), (1, '79.230')] [2023-10-11 20:57:21,185][71635] Updated weights for policy 1, policy_version 47322 (0.0008) [2023-10-11 20:57:21,230][71601] Updated weights for policy 0, policy_version 47350 (0.0008) [2023-10-11 20:57:21,593][71601] Updated weights for policy 0, policy_version 47360 (0.0007) [2023-10-11 20:57:25,025][71635] Updated weights for policy 1, policy_version 47332 (0.0008) [2023-10-11 20:57:25,387][71635] Updated weights for policy 1, policy_version 47342 (0.0008) [2023-10-11 20:57:25,397][71601] Updated weights for policy 0, policy_version 47370 (0.0009) [2023-10-11 20:57:25,754][71635] Updated weights for policy 1, policy_version 47352 (0.0007) [2023-10-11 20:57:25,771][71601] Updated weights for policy 0, policy_version 47380 (0.0009) [2023-10-11 20:57:26,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 96960512. Throughput: 0: 1811.6, 1: 1818.2. Samples: 24259486. Policy #0 lag: (min: 27.0, avg: 39.4, max: 59.0) [2023-10-11 20:57:26,034][70582] Avg episode reward: [(0, '212.430'), (1, '83.780')] [2023-10-11 20:57:26,147][71601] Updated weights for policy 0, policy_version 47390 (0.0007) [2023-10-11 20:57:29,569][71635] Updated weights for policy 1, policy_version 47362 (0.0008) [2023-10-11 20:57:29,879][71601] Updated weights for policy 0, policy_version 47400 (0.0008) [2023-10-11 20:57:29,935][71635] Updated weights for policy 1, policy_version 47372 (0.0007) [2023-10-11 20:57:30,244][71601] Updated weights for policy 0, policy_version 47410 (0.0008) [2023-10-11 20:57:30,295][71635] Updated weights for policy 1, policy_version 47382 (0.0008) [2023-10-11 20:57:30,620][71601] Updated weights for policy 0, policy_version 47420 (0.0007) [2023-10-11 20:57:30,654][71635] Updated weights for policy 1, policy_version 47392 (0.0007) [2023-10-11 20:57:31,034][70582] Fps is (10 sec: 19660.6, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 97091584. Throughput: 0: 1808.8, 1: 1801.2. Samples: 24270510. Policy #0 lag: (min: 27.0, avg: 39.4, max: 59.0) [2023-10-11 20:57:31,034][70582] Avg episode reward: [(0, '212.430'), (1, '90.190')] [2023-10-11 20:57:34,456][71601] Updated weights for policy 0, policy_version 47430 (0.0008) [2023-10-11 20:57:34,575][71635] Updated weights for policy 1, policy_version 47402 (0.0007) [2023-10-11 20:57:34,825][71601] Updated weights for policy 0, policy_version 47440 (0.0007) [2023-10-11 20:57:34,952][71635] Updated weights for policy 1, policy_version 47412 (0.0007) [2023-10-11 20:57:35,198][71601] Updated weights for policy 0, policy_version 47450 (0.0007) [2023-10-11 20:57:35,311][71635] Updated weights for policy 1, policy_version 47422 (0.0009) [2023-10-11 20:57:36,034][70582] Fps is (10 sec: 19660.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 97157120. Throughput: 0: 1817.1, 1: 1814.0. Samples: 24292324. Policy #0 lag: (min: 27.0, avg: 39.4, max: 59.0) [2023-10-11 20:57:36,035][70582] Avg episode reward: [(0, '212.650'), (1, '86.790')] [2023-10-11 20:57:38,930][71601] Updated weights for policy 0, policy_version 47460 (0.0007) [2023-10-11 20:57:39,079][71635] Updated weights for policy 1, policy_version 47432 (0.0008) [2023-10-11 20:57:39,296][71601] Updated weights for policy 0, policy_version 47470 (0.0008) [2023-10-11 20:57:39,438][71635] Updated weights for policy 1, policy_version 47442 (0.0007) [2023-10-11 20:57:39,661][71601] Updated weights for policy 0, policy_version 47480 (0.0008) [2023-10-11 20:57:39,799][71635] Updated weights for policy 1, policy_version 47452 (0.0007) [2023-10-11 20:57:41,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 97222656. Throughput: 0: 1804.4, 1: 1788.7. Samples: 24312392. Policy #0 lag: (min: 27.0, avg: 39.4, max: 59.0) [2023-10-11 20:57:41,035][70582] Avg episode reward: [(0, '214.320'), (1, '87.360')] [2023-10-11 20:57:43,266][71601] Updated weights for policy 0, policy_version 47490 (0.0008) [2023-10-11 20:57:43,521][71635] Updated weights for policy 1, policy_version 47462 (0.0007) [2023-10-11 20:57:43,641][71601] Updated weights for policy 0, policy_version 47500 (0.0008) [2023-10-11 20:57:43,883][71635] Updated weights for policy 1, policy_version 47472 (0.0009) [2023-10-11 20:57:44,013][71601] Updated weights for policy 0, policy_version 47510 (0.0008) [2023-10-11 20:57:44,250][71635] Updated weights for policy 1, policy_version 47482 (0.0008) [2023-10-11 20:57:44,371][71601] Updated weights for policy 0, policy_version 47520 (0.0007) [2023-10-11 20:57:46,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 97288192. Throughput: 0: 1813.8, 1: 1810.7. Samples: 24324766. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-11 20:57:46,035][70582] Avg episode reward: [(0, '224.760'), (1, '88.220')] [2023-10-11 20:57:47,941][71635] Updated weights for policy 1, policy_version 47492 (0.0008) [2023-10-11 20:57:48,018][71601] Updated weights for policy 0, policy_version 47530 (0.0009) [2023-10-11 20:57:48,294][71635] Updated weights for policy 1, policy_version 47502 (0.0007) [2023-10-11 20:57:48,379][71601] Updated weights for policy 0, policy_version 47540 (0.0008) [2023-10-11 20:57:48,657][71635] Updated weights for policy 1, policy_version 47512 (0.0007) [2023-10-11 20:57:48,752][71601] Updated weights for policy 0, policy_version 47550 (0.0009) [2023-10-11 20:57:51,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 97353728. Throughput: 0: 1806.2, 1: 1784.8. Samples: 24344940. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-11 20:57:51,034][70582] Avg episode reward: [(0, '228.840'), (1, '90.840')] [2023-10-11 20:57:52,253][71635] Updated weights for policy 1, policy_version 47522 (0.0009) [2023-10-11 20:57:52,561][71601] Updated weights for policy 0, policy_version 47560 (0.0007) [2023-10-11 20:57:52,626][71635] Updated weights for policy 1, policy_version 47532 (0.0010) [2023-10-11 20:57:52,936][71601] Updated weights for policy 0, policy_version 47570 (0.0007) [2023-10-11 20:57:52,992][71635] Updated weights for policy 1, policy_version 47542 (0.0008) [2023-10-11 20:57:53,306][71601] Updated weights for policy 0, policy_version 47580 (0.0009) [2023-10-11 20:57:53,346][71635] Updated weights for policy 1, policy_version 47552 (0.0008) [2023-10-11 20:57:56,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 97419264. Throughput: 0: 1806.4, 1: 1786.7. Samples: 24367544. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-11 20:57:56,035][70582] Avg episode reward: [(0, '204.820'), (1, '103.470')] [2023-10-11 20:57:56,042][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000047552_48693248.pth... [2023-10-11 20:57:56,042][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000047584_48726016.pth... [2023-10-11 20:57:56,071][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000045856_46956544.pth [2023-10-11 20:57:56,080][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000045888_46989312.pth [2023-10-11 20:57:56,916][71601] Updated weights for policy 0, policy_version 47590 (0.0007) [2023-10-11 20:57:57,107][71635] Updated weights for policy 1, policy_version 47562 (0.0008) [2023-10-11 20:57:57,273][71601] Updated weights for policy 0, policy_version 47600 (0.0007) [2023-10-11 20:57:57,476][71635] Updated weights for policy 1, policy_version 47572 (0.0009) [2023-10-11 20:57:57,653][71601] Updated weights for policy 0, policy_version 47610 (0.0007) [2023-10-11 20:57:57,833][71635] Updated weights for policy 1, policy_version 47582 (0.0007) [2023-10-11 20:58:01,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 97484800. Throughput: 0: 1805.0, 1: 1785.7. Samples: 24377246. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-11 20:58:01,035][70582] Avg episode reward: [(0, '216.110'), (1, '99.630')] [2023-10-11 20:58:01,516][71601] Updated weights for policy 0, policy_version 47620 (0.0008) [2023-10-11 20:58:01,583][71635] Updated weights for policy 1, policy_version 47592 (0.0007) [2023-10-11 20:58:01,882][71601] Updated weights for policy 0, policy_version 47630 (0.0008) [2023-10-11 20:58:01,944][71635] Updated weights for policy 1, policy_version 47602 (0.0007) [2023-10-11 20:58:02,254][71601] Updated weights for policy 0, policy_version 47640 (0.0009) [2023-10-11 20:58:02,314][71635] Updated weights for policy 1, policy_version 47612 (0.0008) [2023-10-11 20:58:05,919][71635] Updated weights for policy 1, policy_version 47622 (0.0008) [2023-10-11 20:58:06,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 97550336. Throughput: 0: 1797.8, 1: 1791.1. Samples: 24399692. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-11 20:58:06,035][70582] Avg episode reward: [(0, '215.900'), (1, '99.620')] [2023-10-11 20:58:06,071][71601] Updated weights for policy 0, policy_version 47650 (0.0007) [2023-10-11 20:58:06,283][71635] Updated weights for policy 1, policy_version 47632 (0.0008) [2023-10-11 20:58:06,446][71601] Updated weights for policy 0, policy_version 47660 (0.0009) [2023-10-11 20:58:06,651][71635] Updated weights for policy 1, policy_version 47642 (0.0009) [2023-10-11 20:58:06,803][71601] Updated weights for policy 0, policy_version 47670 (0.0009) [2023-10-11 20:58:07,175][71601] Updated weights for policy 0, policy_version 47680 (0.0007) [2023-10-11 20:58:10,482][71635] Updated weights for policy 1, policy_version 47652 (0.0008) [2023-10-11 20:58:10,804][71601] Updated weights for policy 0, policy_version 47690 (0.0010) [2023-10-11 20:58:10,855][71635] Updated weights for policy 1, policy_version 47662 (0.0007) [2023-10-11 20:58:11,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 97615872. Throughput: 0: 1818.8, 1: 1805.2. Samples: 24422568. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-11 20:58:11,034][70582] Avg episode reward: [(0, '226.990'), (1, '99.610')] [2023-10-11 20:58:11,181][71601] Updated weights for policy 0, policy_version 47700 (0.0009) [2023-10-11 20:58:11,222][71635] Updated weights for policy 1, policy_version 47672 (0.0007) [2023-10-11 20:58:11,550][71601] Updated weights for policy 0, policy_version 47710 (0.0007) [2023-10-11 20:58:14,989][71635] Updated weights for policy 1, policy_version 47682 (0.0008) [2023-10-11 20:58:15,323][71601] Updated weights for policy 0, policy_version 47720 (0.0008) [2023-10-11 20:58:15,356][71635] Updated weights for policy 1, policy_version 47692 (0.0008) [2023-10-11 20:58:15,690][71601] Updated weights for policy 0, policy_version 47730 (0.0007) [2023-10-11 20:58:15,719][71635] Updated weights for policy 1, policy_version 47702 (0.0008) [2023-10-11 20:58:16,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 97681408. Throughput: 0: 1803.9, 1: 1795.6. Samples: 24432486. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-11 20:58:16,034][70582] Avg episode reward: [(0, '226.940'), (1, '106.080')] [2023-10-11 20:58:16,069][71601] Updated weights for policy 0, policy_version 47740 (0.0009) [2023-10-11 20:58:16,085][71635] Updated weights for policy 1, policy_version 47712 (0.0007) [2023-10-11 20:58:19,701][71601] Updated weights for policy 0, policy_version 47750 (0.0007) [2023-10-11 20:58:19,772][71635] Updated weights for policy 1, policy_version 47722 (0.0008) [2023-10-11 20:58:20,077][71601] Updated weights for policy 0, policy_version 47760 (0.0009) [2023-10-11 20:58:20,129][71635] Updated weights for policy 1, policy_version 47732 (0.0008) [2023-10-11 20:58:20,442][71601] Updated weights for policy 0, policy_version 47770 (0.0009) [2023-10-11 20:58:20,490][71635] Updated weights for policy 1, policy_version 47742 (0.0008) [2023-10-11 20:58:21,034][70582] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 97812480. Throughput: 0: 1815.3, 1: 1804.0. Samples: 24455192. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-11 20:58:21,034][70582] Avg episode reward: [(0, '227.090'), (1, '102.270')] [2023-10-11 20:58:23,984][71601] Updated weights for policy 0, policy_version 47780 (0.0009) [2023-10-11 20:58:24,266][71635] Updated weights for policy 1, policy_version 47752 (0.0010) [2023-10-11 20:58:24,359][71601] Updated weights for policy 0, policy_version 47790 (0.0008) [2023-10-11 20:58:24,635][71635] Updated weights for policy 1, policy_version 47762 (0.0008) [2023-10-11 20:58:24,738][71601] Updated weights for policy 0, policy_version 47800 (0.0007) [2023-10-11 20:58:24,997][71635] Updated weights for policy 1, policy_version 47772 (0.0009) [2023-10-11 20:58:26,034][70582] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 97878016. Throughput: 0: 1807.2, 1: 1800.8. Samples: 24474752. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-11 20:58:26,035][70582] Avg episode reward: [(0, '237.650'), (1, '100.790')] [2023-10-11 20:58:28,404][71601] Updated weights for policy 0, policy_version 47810 (0.0009) [2023-10-11 20:58:28,613][71635] Updated weights for policy 1, policy_version 47782 (0.0009) [2023-10-11 20:58:28,786][71601] Updated weights for policy 0, policy_version 47820 (0.0009) [2023-10-11 20:58:28,981][71635] Updated weights for policy 1, policy_version 47792 (0.0011) [2023-10-11 20:58:29,156][71601] Updated weights for policy 0, policy_version 47830 (0.0008) [2023-10-11 20:58:29,337][71635] Updated weights for policy 1, policy_version 47802 (0.0008) [2023-10-11 20:58:29,528][71601] Updated weights for policy 0, policy_version 47840 (0.0009) [2023-10-11 20:58:31,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 97943552. Throughput: 0: 1812.2, 1: 1808.1. Samples: 24487680. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-11 20:58:31,035][70582] Avg episode reward: [(0, '237.960'), (1, '100.430')] [2023-10-11 20:58:33,138][71635] Updated weights for policy 1, policy_version 47812 (0.0009) [2023-10-11 20:58:33,278][71601] Updated weights for policy 0, policy_version 47850 (0.0009) [2023-10-11 20:58:33,502][71635] Updated weights for policy 1, policy_version 47822 (0.0009) [2023-10-11 20:58:33,647][71601] Updated weights for policy 0, policy_version 47860 (0.0010) [2023-10-11 20:58:33,869][71635] Updated weights for policy 1, policy_version 47832 (0.0008) [2023-10-11 20:58:34,021][71601] Updated weights for policy 0, policy_version 47870 (0.0008) [2023-10-11 20:58:36,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 98009088. Throughput: 0: 1800.0, 1: 1806.3. Samples: 24507224. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-11 20:58:36,034][70582] Avg episode reward: [(0, '233.950'), (1, '105.610')] [2023-10-11 20:58:37,581][71635] Updated weights for policy 1, policy_version 47842 (0.0009) [2023-10-11 20:58:37,677][71601] Updated weights for policy 0, policy_version 47880 (0.0008) [2023-10-11 20:58:37,945][71635] Updated weights for policy 1, policy_version 47852 (0.0007) [2023-10-11 20:58:38,045][71601] Updated weights for policy 0, policy_version 47890 (0.0007) [2023-10-11 20:58:38,312][71635] Updated weights for policy 1, policy_version 47862 (0.0010) [2023-10-11 20:58:38,401][71601] Updated weights for policy 0, policy_version 47900 (0.0007) [2023-10-11 20:58:38,679][71635] Updated weights for policy 1, policy_version 47872 (0.0008) [2023-10-11 20:58:41,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 98074624. Throughput: 0: 1805.6, 1: 1806.5. Samples: 24530086. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-11 20:58:41,034][70582] Avg episode reward: [(0, '234.060'), (1, '108.490')] [2023-10-11 20:58:42,124][71601] Updated weights for policy 0, policy_version 47910 (0.0008) [2023-10-11 20:58:42,496][71601] Updated weights for policy 0, policy_version 47920 (0.0007) [2023-10-11 20:58:42,524][71635] Updated weights for policy 1, policy_version 47882 (0.0008) [2023-10-11 20:58:42,856][71601] Updated weights for policy 0, policy_version 47930 (0.0008) [2023-10-11 20:58:42,889][71635] Updated weights for policy 1, policy_version 47892 (0.0009) [2023-10-11 20:58:43,252][71635] Updated weights for policy 1, policy_version 47902 (0.0007) [2023-10-11 20:58:46,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 98140160. Throughput: 0: 1807.8, 1: 1810.0. Samples: 24540046. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-11 20:58:46,034][70582] Avg episode reward: [(0, '240.130'), (1, '118.400')] [2023-10-11 20:58:46,617][71601] Updated weights for policy 0, policy_version 47940 (0.0008) [2023-10-11 20:58:46,802][71635] Updated weights for policy 1, policy_version 47912 (0.0008) [2023-10-11 20:58:46,982][71601] Updated weights for policy 0, policy_version 47950 (0.0009) [2023-10-11 20:58:47,175][71635] Updated weights for policy 1, policy_version 47922 (0.0007) [2023-10-11 20:58:47,360][71601] Updated weights for policy 0, policy_version 47960 (0.0008) [2023-10-11 20:58:47,533][71635] Updated weights for policy 1, policy_version 47932 (0.0009) [2023-10-11 20:58:51,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 98205696. Throughput: 0: 1813.2, 1: 1805.6. Samples: 24562538. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-11 20:58:51,034][70582] Avg episode reward: [(0, '239.880'), (1, '124.980')] [2023-10-11 20:58:51,113][71601] Updated weights for policy 0, policy_version 47970 (0.0009) [2023-10-11 20:58:51,257][71635] Updated weights for policy 1, policy_version 47942 (0.0010) [2023-10-11 20:58:51,477][71601] Updated weights for policy 0, policy_version 47980 (0.0008) [2023-10-11 20:58:51,619][71635] Updated weights for policy 1, policy_version 47952 (0.0008) [2023-10-11 20:58:51,845][71601] Updated weights for policy 0, policy_version 47990 (0.0007) [2023-10-11 20:58:51,981][71635] Updated weights for policy 1, policy_version 47962 (0.0007) [2023-10-11 20:58:52,209][71601] Updated weights for policy 0, policy_version 48000 (0.0008) [2023-10-11 20:58:55,842][71635] Updated weights for policy 1, policy_version 47972 (0.0008) [2023-10-11 20:58:55,952][71601] Updated weights for policy 0, policy_version 48010 (0.0008) [2023-10-11 20:58:56,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 98271232. Throughput: 0: 1812.4, 1: 1810.3. Samples: 24585594. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-11 20:58:56,035][70582] Avg episode reward: [(0, '213.420'), (1, '123.720')] [2023-10-11 20:58:56,208][71635] Updated weights for policy 1, policy_version 47982 (0.0007) [2023-10-11 20:58:56,327][71601] Updated weights for policy 0, policy_version 48020 (0.0009) [2023-10-11 20:58:56,565][71635] Updated weights for policy 1, policy_version 47992 (0.0007) [2023-10-11 20:58:56,686][71601] Updated weights for policy 0, policy_version 48030 (0.0007) [2023-10-11 20:59:00,278][71601] Updated weights for policy 0, policy_version 48040 (0.0008) [2023-10-11 20:59:00,373][71635] Updated weights for policy 1, policy_version 48002 (0.0009) [2023-10-11 20:59:00,654][71601] Updated weights for policy 0, policy_version 48050 (0.0007) [2023-10-11 20:59:00,733][71635] Updated weights for policy 1, policy_version 48012 (0.0007) [2023-10-11 20:59:01,022][71601] Updated weights for policy 0, policy_version 48060 (0.0009) [2023-10-11 20:59:01,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 98336768. Throughput: 0: 1813.0, 1: 1806.7. Samples: 24595372. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-11 20:59:01,034][70582] Avg episode reward: [(0, '222.530'), (1, '123.790')] [2023-10-11 20:59:01,094][71635] Updated weights for policy 1, policy_version 48022 (0.0007) [2023-10-11 20:59:01,459][71635] Updated weights for policy 1, policy_version 48032 (0.0009) [2023-10-11 20:59:04,800][71601] Updated weights for policy 0, policy_version 48070 (0.0008) [2023-10-11 20:59:05,054][71635] Updated weights for policy 1, policy_version 48042 (0.0008) [2023-10-11 20:59:05,165][71601] Updated weights for policy 0, policy_version 48080 (0.0009) [2023-10-11 20:59:05,418][71635] Updated weights for policy 1, policy_version 48052 (0.0008) [2023-10-11 20:59:05,525][71601] Updated weights for policy 0, policy_version 48090 (0.0008) [2023-10-11 20:59:05,781][71635] Updated weights for policy 1, policy_version 48062 (0.0007) [2023-10-11 20:59:06,034][70582] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 98467840. Throughput: 0: 1810.2, 1: 1811.0. Samples: 24618144. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-11 20:59:06,035][70582] Avg episode reward: [(0, '224.190'), (1, '107.600')] [2023-10-11 20:59:09,262][71601] Updated weights for policy 0, policy_version 48100 (0.0007) [2023-10-11 20:59:09,437][71635] Updated weights for policy 1, policy_version 48072 (0.0007) [2023-10-11 20:59:09,635][71601] Updated weights for policy 0, policy_version 48110 (0.0008) [2023-10-11 20:59:09,815][71635] Updated weights for policy 1, policy_version 48082 (0.0008) [2023-10-11 20:59:09,999][71601] Updated weights for policy 0, policy_version 48120 (0.0008) [2023-10-11 20:59:10,181][71635] Updated weights for policy 1, policy_version 48092 (0.0009) [2023-10-11 20:59:11,034][70582] Fps is (10 sec: 19660.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 98533376. Throughput: 0: 1812.4, 1: 1816.2. Samples: 24638036. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-11 20:59:11,035][70582] Avg episode reward: [(0, '224.040'), (1, '102.920')] [2023-10-11 20:59:13,801][71601] Updated weights for policy 0, policy_version 48130 (0.0008) [2023-10-11 20:59:13,883][71635] Updated weights for policy 1, policy_version 48102 (0.0008) [2023-10-11 20:59:14,171][71601] Updated weights for policy 0, policy_version 48140 (0.0009) [2023-10-11 20:59:14,243][71635] Updated weights for policy 1, policy_version 48112 (0.0008) [2023-10-11 20:59:14,536][71601] Updated weights for policy 0, policy_version 48150 (0.0008) [2023-10-11 20:59:14,616][71635] Updated weights for policy 1, policy_version 48122 (0.0008) [2023-10-11 20:59:14,904][71601] Updated weights for policy 0, policy_version 48160 (0.0007) [2023-10-11 20:59:16,034][70582] Fps is (10 sec: 13107.4, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 98598912. Throughput: 0: 1814.6, 1: 1814.3. Samples: 24650980. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-11 20:59:16,034][70582] Avg episode reward: [(0, '224.050'), (1, '95.250')] [2023-10-11 20:59:18,332][71635] Updated weights for policy 1, policy_version 48132 (0.0008) [2023-10-11 20:59:18,370][71601] Updated weights for policy 0, policy_version 48170 (0.0009) [2023-10-11 20:59:18,700][71635] Updated weights for policy 1, policy_version 48142 (0.0008) [2023-10-11 20:59:18,738][71601] Updated weights for policy 0, policy_version 48180 (0.0007) [2023-10-11 20:59:19,063][71635] Updated weights for policy 1, policy_version 48152 (0.0011) [2023-10-11 20:59:19,109][71601] Updated weights for policy 0, policy_version 48190 (0.0008) [2023-10-11 20:59:21,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 98664448. Throughput: 0: 1820.4, 1: 1812.8. Samples: 24670718. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:59:21,034][70582] Avg episode reward: [(0, '224.270'), (1, '94.730')] [2023-10-11 20:59:22,720][71601] Updated weights for policy 0, policy_version 48200 (0.0008) [2023-10-11 20:59:22,752][71635] Updated weights for policy 1, policy_version 48162 (0.0009) [2023-10-11 20:59:23,100][71601] Updated weights for policy 0, policy_version 48210 (0.0008) [2023-10-11 20:59:23,115][71635] Updated weights for policy 1, policy_version 48172 (0.0007) [2023-10-11 20:59:23,467][71601] Updated weights for policy 0, policy_version 48220 (0.0008) [2023-10-11 20:59:23,493][71635] Updated weights for policy 1, policy_version 48182 (0.0007) [2023-10-11 20:59:23,861][71635] Updated weights for policy 1, policy_version 48192 (0.0009) [2023-10-11 20:59:26,034][70582] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 98729984. Throughput: 0: 1817.9, 1: 1804.7. Samples: 24693102. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:59:26,035][70582] Avg episode reward: [(0, '213.050'), (1, '90.680')] [2023-10-11 20:59:27,263][71601] Updated weights for policy 0, policy_version 48230 (0.0008) [2023-10-11 20:59:27,629][71601] Updated weights for policy 0, policy_version 48240 (0.0008) [2023-10-11 20:59:27,721][71635] Updated weights for policy 1, policy_version 48202 (0.0007) [2023-10-11 20:59:27,997][71601] Updated weights for policy 0, policy_version 48250 (0.0009) [2023-10-11 20:59:28,090][71635] Updated weights for policy 1, policy_version 48212 (0.0007) [2023-10-11 20:59:28,459][71635] Updated weights for policy 1, policy_version 48222 (0.0007) [2023-10-11 20:59:31,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 98795520. Throughput: 0: 1817.1, 1: 1808.4. Samples: 24703196. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:59:31,034][70582] Avg episode reward: [(0, '219.510'), (1, '87.980')] [2023-10-11 20:59:31,762][71601] Updated weights for policy 0, policy_version 48260 (0.0007) [2023-10-11 20:59:32,124][71601] Updated weights for policy 0, policy_version 48270 (0.0007) [2023-10-11 20:59:32,246][71635] Updated weights for policy 1, policy_version 48232 (0.0008) [2023-10-11 20:59:32,496][71601] Updated weights for policy 0, policy_version 48280 (0.0008) [2023-10-11 20:59:32,615][71635] Updated weights for policy 1, policy_version 48242 (0.0008) [2023-10-11 20:59:32,982][71635] Updated weights for policy 1, policy_version 48252 (0.0008) [2023-10-11 20:59:36,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 98861056. Throughput: 0: 1826.1, 1: 1803.4. Samples: 24725864. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:59:36,034][70582] Avg episode reward: [(0, '214.220'), (1, '92.670')] [2023-10-11 20:59:36,067][71601] Updated weights for policy 0, policy_version 48290 (0.0008) [2023-10-11 20:59:36,440][71601] Updated weights for policy 0, policy_version 48300 (0.0008) [2023-10-11 20:59:36,688][71635] Updated weights for policy 1, policy_version 48262 (0.0008) [2023-10-11 20:59:36,800][71601] Updated weights for policy 0, policy_version 48310 (0.0007) [2023-10-11 20:59:37,053][71635] Updated weights for policy 1, policy_version 48272 (0.0008) [2023-10-11 20:59:37,170][71601] Updated weights for policy 0, policy_version 48320 (0.0008) [2023-10-11 20:59:37,415][71635] Updated weights for policy 1, policy_version 48282 (0.0009) [2023-10-11 20:59:40,769][71601] Updated weights for policy 0, policy_version 48330 (0.0007) [2023-10-11 20:59:41,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 98926592. Throughput: 0: 1824.6, 1: 1797.5. Samples: 24748586. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:59:41,034][70582] Avg episode reward: [(0, '225.560'), (1, '98.980')] [2023-10-11 20:59:41,127][71601] Updated weights for policy 0, policy_version 48340 (0.0008) [2023-10-11 20:59:41,147][71635] Updated weights for policy 1, policy_version 48292 (0.0010) [2023-10-11 20:59:41,499][71601] Updated weights for policy 0, policy_version 48350 (0.0008) [2023-10-11 20:59:41,518][71635] Updated weights for policy 1, policy_version 48302 (0.0008) [2023-10-11 20:59:41,882][71635] Updated weights for policy 1, policy_version 48312 (0.0008) [2023-10-11 20:59:45,140][71601] Updated weights for policy 0, policy_version 48360 (0.0009) [2023-10-11 20:59:45,450][71635] Updated weights for policy 1, policy_version 48322 (0.0007) [2023-10-11 20:59:45,514][71601] Updated weights for policy 0, policy_version 48370 (0.0007) [2023-10-11 20:59:45,824][71635] Updated weights for policy 1, policy_version 48332 (0.0007) [2023-10-11 20:59:45,889][71601] Updated weights for policy 0, policy_version 48380 (0.0008) [2023-10-11 20:59:46,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 98992128. Throughput: 0: 1830.2, 1: 1803.2. Samples: 24758876. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 20:59:46,035][70582] Avg episode reward: [(0, '211.970'), (1, '83.090')] [2023-10-11 20:59:46,184][71635] Updated weights for policy 1, policy_version 48342 (0.0009) [2023-10-11 20:59:46,549][71635] Updated weights for policy 1, policy_version 48352 (0.0011) [2023-10-11 20:59:49,461][71601] Updated weights for policy 0, policy_version 48390 (0.0009) [2023-10-11 20:59:49,827][71601] Updated weights for policy 0, policy_version 48400 (0.0010) [2023-10-11 20:59:50,200][71601] Updated weights for policy 0, policy_version 48410 (0.0009) [2023-10-11 20:59:50,287][71635] Updated weights for policy 1, policy_version 48362 (0.0008) [2023-10-11 20:59:50,656][71635] Updated weights for policy 1, policy_version 48372 (0.0008) [2023-10-11 20:59:51,014][71635] Updated weights for policy 1, policy_version 48382 (0.0007) [2023-10-11 20:59:51,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 99090432. Throughput: 0: 1825.7, 1: 1804.6. Samples: 24781504. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-11 20:59:51,034][70582] Avg episode reward: [(0, '213.240'), (1, '77.880')] [2023-10-11 20:59:53,950][71601] Updated weights for policy 0, policy_version 48420 (0.0008) [2023-10-11 20:59:54,322][71601] Updated weights for policy 0, policy_version 48430 (0.0009) [2023-10-11 20:59:54,686][71601] Updated weights for policy 0, policy_version 48440 (0.0008) [2023-10-11 20:59:54,891][71635] Updated weights for policy 1, policy_version 48392 (0.0008) [2023-10-11 20:59:55,247][71635] Updated weights for policy 1, policy_version 48402 (0.0009) [2023-10-11 20:59:55,625][71635] Updated weights for policy 1, policy_version 48412 (0.0008) [2023-10-11 20:59:56,034][70582] Fps is (10 sec: 19660.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 99188736. Throughput: 0: 1830.2, 1: 1819.5. Samples: 24802274. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-11 20:59:56,035][70582] Avg episode reward: [(0, '198.640'), (1, '72.950')] [2023-10-11 20:59:56,044][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000048448_49610752.pth... [2023-10-11 20:59:56,045][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000048416_49577984.pth... [2023-10-11 20:59:56,075][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000046752_47874048.pth [2023-10-11 20:59:56,092][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000046720_47841280.pth [2023-10-11 20:59:58,374][71601] Updated weights for policy 0, policy_version 48450 (0.0007) [2023-10-11 20:59:58,737][71601] Updated weights for policy 0, policy_version 48460 (0.0007) [2023-10-11 20:59:59,112][71601] Updated weights for policy 0, policy_version 48470 (0.0007) [2023-10-11 20:59:59,274][71635] Updated weights for policy 1, policy_version 48422 (0.0009) [2023-10-11 20:59:59,483][71601] Updated weights for policy 0, policy_version 48480 (0.0010) [2023-10-11 20:59:59,641][71635] Updated weights for policy 1, policy_version 48432 (0.0011) [2023-10-11 20:59:59,996][71635] Updated weights for policy 1, policy_version 48442 (0.0008) [2023-10-11 21:00:01,034][70582] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 99254272. Throughput: 0: 1828.9, 1: 1806.4. Samples: 24814570. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-11 21:00:01,035][70582] Avg episode reward: [(0, '204.900'), (1, '74.460')] [2023-10-11 21:00:03,178][71601] Updated weights for policy 0, policy_version 48490 (0.0010) [2023-10-11 21:00:03,549][71601] Updated weights for policy 0, policy_version 48500 (0.0008) [2023-10-11 21:00:03,866][71635] Updated weights for policy 1, policy_version 48452 (0.0008) [2023-10-11 21:00:03,925][71601] Updated weights for policy 0, policy_version 48510 (0.0010) [2023-10-11 21:00:04,228][71635] Updated weights for policy 1, policy_version 48462 (0.0008) [2023-10-11 21:00:04,597][71635] Updated weights for policy 1, policy_version 48472 (0.0008) [2023-10-11 21:00:06,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 99319808. Throughput: 0: 1825.9, 1: 1822.3. Samples: 24834888. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-11 21:00:06,035][70582] Avg episode reward: [(0, '185.620'), (1, '69.710')] [2023-10-11 21:00:07,797][71601] Updated weights for policy 0, policy_version 48520 (0.0009) [2023-10-11 21:00:08,172][71601] Updated weights for policy 0, policy_version 48530 (0.0008) [2023-10-11 21:00:08,351][71635] Updated weights for policy 1, policy_version 48482 (0.0009) [2023-10-11 21:00:08,540][71601] Updated weights for policy 0, policy_version 48540 (0.0008) [2023-10-11 21:00:08,719][71635] Updated weights for policy 1, policy_version 48492 (0.0008) [2023-10-11 21:00:09,088][71635] Updated weights for policy 1, policy_version 48502 (0.0010) [2023-10-11 21:00:09,452][71635] Updated weights for policy 1, policy_version 48512 (0.0009) [2023-10-11 21:00:11,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 99385344. Throughput: 0: 1826.5, 1: 1807.9. Samples: 24856650. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-11 21:00:11,034][70582] Avg episode reward: [(0, '179.410'), (1, '73.450')] [2023-10-11 21:00:12,203][71601] Updated weights for policy 0, policy_version 48550 (0.0008) [2023-10-11 21:00:12,575][71601] Updated weights for policy 0, policy_version 48560 (0.0008) [2023-10-11 21:00:12,944][71601] Updated weights for policy 0, policy_version 48570 (0.0008) [2023-10-11 21:00:13,319][71635] Updated weights for policy 1, policy_version 48522 (0.0009) [2023-10-11 21:00:13,681][71635] Updated weights for policy 1, policy_version 48532 (0.0008) [2023-10-11 21:00:14,043][71635] Updated weights for policy 1, policy_version 48542 (0.0007) [2023-10-11 21:00:16,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 99450880. Throughput: 0: 1825.8, 1: 1822.8. Samples: 24867382. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-11 21:00:16,034][70582] Avg episode reward: [(0, '179.610'), (1, '70.850')] [2023-10-11 21:00:16,673][71601] Updated weights for policy 0, policy_version 48580 (0.0008) [2023-10-11 21:00:17,041][71601] Updated weights for policy 0, policy_version 48590 (0.0007) [2023-10-11 21:00:17,405][71601] Updated weights for policy 0, policy_version 48600 (0.0010) [2023-10-11 21:00:17,679][71635] Updated weights for policy 1, policy_version 48552 (0.0010) [2023-10-11 21:00:18,049][71635] Updated weights for policy 1, policy_version 48562 (0.0010) [2023-10-11 21:00:18,423][71635] Updated weights for policy 1, policy_version 48572 (0.0009) [2023-10-11 21:00:21,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 99516416. Throughput: 0: 1822.2, 1: 1808.3. Samples: 24889234. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-11 21:00:21,034][70582] Avg episode reward: [(0, '170.180'), (1, '76.730')] [2023-10-11 21:00:21,068][71601] Updated weights for policy 0, policy_version 48610 (0.0007) [2023-10-11 21:00:21,427][71601] Updated weights for policy 0, policy_version 48620 (0.0007) [2023-10-11 21:00:21,801][71601] Updated weights for policy 0, policy_version 48630 (0.0007) [2023-10-11 21:00:21,948][71635] Updated weights for policy 1, policy_version 48582 (0.0009) [2023-10-11 21:00:22,176][71601] Updated weights for policy 0, policy_version 48640 (0.0009) [2023-10-11 21:00:22,315][71635] Updated weights for policy 1, policy_version 48592 (0.0008) [2023-10-11 21:00:22,680][71635] Updated weights for policy 1, policy_version 48602 (0.0007) [2023-10-11 21:00:26,034][70582] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 99581952. Throughput: 0: 1819.7, 1: 1815.7. Samples: 24912180. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:00:26,035][70582] Avg episode reward: [(0, '170.280'), (1, '73.070')] [2023-10-11 21:00:26,056][71601] Updated weights for policy 0, policy_version 48650 (0.0009) [2023-10-11 21:00:26,402][71635] Updated weights for policy 1, policy_version 48612 (0.0008) [2023-10-11 21:00:26,424][71601] Updated weights for policy 0, policy_version 48660 (0.0008) [2023-10-11 21:00:26,768][71635] Updated weights for policy 1, policy_version 48622 (0.0008) [2023-10-11 21:00:26,800][71601] Updated weights for policy 0, policy_version 48670 (0.0009) [2023-10-11 21:00:27,140][71635] Updated weights for policy 1, policy_version 48632 (0.0007) [2023-10-11 21:00:30,454][71601] Updated weights for policy 0, policy_version 48680 (0.0008) [2023-10-11 21:00:30,832][71601] Updated weights for policy 0, policy_version 48690 (0.0009) [2023-10-11 21:00:30,876][71635] Updated weights for policy 1, policy_version 48642 (0.0008) [2023-10-11 21:00:31,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 99647488. Throughput: 0: 1812.8, 1: 1811.4. Samples: 24921968. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:00:31,035][70582] Avg episode reward: [(0, '170.280'), (1, '78.640')] [2023-10-11 21:00:31,203][71601] Updated weights for policy 0, policy_version 48700 (0.0008) [2023-10-11 21:00:31,240][71635] Updated weights for policy 1, policy_version 48652 (0.0007) [2023-10-11 21:00:31,614][71635] Updated weights for policy 1, policy_version 48662 (0.0008) [2023-10-11 21:00:31,982][71635] Updated weights for policy 1, policy_version 48672 (0.0007) [2023-10-11 21:00:34,874][71601] Updated weights for policy 0, policy_version 48710 (0.0008) [2023-10-11 21:00:35,244][71601] Updated weights for policy 0, policy_version 48720 (0.0008) [2023-10-11 21:00:35,616][71601] Updated weights for policy 0, policy_version 48730 (0.0009) [2023-10-11 21:00:35,738][71635] Updated weights for policy 1, policy_version 48682 (0.0009) [2023-10-11 21:00:36,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 99745792. Throughput: 0: 1819.6, 1: 1806.3. Samples: 24944672. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:00:36,035][70582] Avg episode reward: [(0, '169.080'), (1, '82.350')] [2023-10-11 21:00:36,108][71635] Updated weights for policy 1, policy_version 48692 (0.0009) [2023-10-11 21:00:36,480][71635] Updated weights for policy 1, policy_version 48702 (0.0010) [2023-10-11 21:00:39,340][71601] Updated weights for policy 0, policy_version 48740 (0.0010) [2023-10-11 21:00:39,709][71601] Updated weights for policy 0, policy_version 48750 (0.0008) [2023-10-11 21:00:40,083][71601] Updated weights for policy 0, policy_version 48760 (0.0007) [2023-10-11 21:00:40,198][71635] Updated weights for policy 1, policy_version 48712 (0.0008) [2023-10-11 21:00:40,567][71635] Updated weights for policy 1, policy_version 48722 (0.0009) [2023-10-11 21:00:40,932][71635] Updated weights for policy 1, policy_version 48732 (0.0008) [2023-10-11 21:00:41,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 99811328. Throughput: 0: 1810.7, 1: 1810.3. Samples: 24965218. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:00:41,035][70582] Avg episode reward: [(0, '160.940'), (1, '86.970')] [2023-10-11 21:00:43,724][71601] Updated weights for policy 0, policy_version 48770 (0.0007) [2023-10-11 21:00:44,096][71601] Updated weights for policy 0, policy_version 48780 (0.0007) [2023-10-11 21:00:44,471][71601] Updated weights for policy 0, policy_version 48790 (0.0008) [2023-10-11 21:00:44,659][71635] Updated weights for policy 1, policy_version 48742 (0.0008) [2023-10-11 21:00:44,848][71601] Updated weights for policy 0, policy_version 48800 (0.0008) [2023-10-11 21:00:45,022][71635] Updated weights for policy 1, policy_version 48752 (0.0008) [2023-10-11 21:00:45,388][71635] Updated weights for policy 1, policy_version 48762 (0.0008) [2023-10-11 21:00:46,034][70582] Fps is (10 sec: 16384.3, 60 sec: 15291.8, 300 sec: 14551.2). Total num frames: 99909632. Throughput: 0: 1815.8, 1: 1800.0. Samples: 24977278. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:00:46,034][70582] Avg episode reward: [(0, '160.940'), (1, '87.080')] [2023-10-11 21:00:48,459][71601] Updated weights for policy 0, policy_version 48810 (0.0007) [2023-10-11 21:00:48,817][71601] Updated weights for policy 0, policy_version 48820 (0.0008) [2023-10-11 21:00:49,003][71635] Updated weights for policy 1, policy_version 48772 (0.0008) [2023-10-11 21:00:49,197][71601] Updated weights for policy 0, policy_version 48830 (0.0009) [2023-10-11 21:00:49,372][71635] Updated weights for policy 1, policy_version 48782 (0.0007) [2023-10-11 21:00:49,747][71635] Updated weights for policy 1, policy_version 48792 (0.0009) [2023-10-11 21:00:51,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 99975168. Throughput: 0: 1816.0, 1: 1807.0. Samples: 24997926. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:00:51,035][70582] Avg episode reward: [(0, '179.450'), (1, '95.410')] [2023-10-11 21:00:53,055][71601] Updated weights for policy 0, policy_version 48840 (0.0008) [2023-10-11 21:00:53,329][71635] Updated weights for policy 1, policy_version 48802 (0.0008) [2023-10-11 21:00:53,426][71601] Updated weights for policy 0, policy_version 48850 (0.0008) [2023-10-11 21:00:53,697][71635] Updated weights for policy 1, policy_version 48812 (0.0008) [2023-10-11 21:00:53,793][71601] Updated weights for policy 0, policy_version 48860 (0.0007) [2023-10-11 21:00:54,056][71635] Updated weights for policy 1, policy_version 48822 (0.0009) [2023-10-11 21:00:54,412][71635] Updated weights for policy 1, policy_version 48832 (0.0010) [2023-10-11 21:00:56,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 100040704. Throughput: 0: 1813.3, 1: 1817.3. Samples: 25020030. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:00:56,035][70582] Avg episode reward: [(0, '182.310'), (1, '90.430')] [2023-10-11 21:00:57,421][71601] Updated weights for policy 0, policy_version 48870 (0.0008) [2023-10-11 21:00:57,797][71601] Updated weights for policy 0, policy_version 48880 (0.0009) [2023-10-11 21:00:58,120][71635] Updated weights for policy 1, policy_version 48842 (0.0008) [2023-10-11 21:00:58,172][71601] Updated weights for policy 0, policy_version 48890 (0.0008) [2023-10-11 21:00:58,486][71635] Updated weights for policy 1, policy_version 48852 (0.0008) [2023-10-11 21:00:58,853][71635] Updated weights for policy 1, policy_version 48862 (0.0011) [2023-10-11 21:01:01,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 100106240. Throughput: 0: 1811.4, 1: 1813.7. Samples: 25030514. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:01:01,035][70582] Avg episode reward: [(0, '166.290'), (1, '94.500')] [2023-10-11 21:01:01,771][71601] Updated weights for policy 0, policy_version 48900 (0.0007) [2023-10-11 21:01:02,135][71601] Updated weights for policy 0, policy_version 48910 (0.0007) [2023-10-11 21:01:02,504][71601] Updated weights for policy 0, policy_version 48920 (0.0008) [2023-10-11 21:01:02,531][71635] Updated weights for policy 1, policy_version 48872 (0.0009) [2023-10-11 21:01:02,897][71635] Updated weights for policy 1, policy_version 48882 (0.0008) [2023-10-11 21:01:03,268][71635] Updated weights for policy 1, policy_version 48892 (0.0008) [2023-10-11 21:01:06,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 100171776. Throughput: 0: 1815.8, 1: 1815.8. Samples: 25052656. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:01:06,034][70582] Avg episode reward: [(0, '170.690'), (1, '87.440')] [2023-10-11 21:01:06,120][71601] Updated weights for policy 0, policy_version 48930 (0.0009) [2023-10-11 21:01:06,487][71601] Updated weights for policy 0, policy_version 48940 (0.0007) [2023-10-11 21:01:06,863][71601] Updated weights for policy 0, policy_version 48950 (0.0008) [2023-10-11 21:01:07,025][71635] Updated weights for policy 1, policy_version 48902 (0.0007) [2023-10-11 21:01:07,241][71601] Updated weights for policy 0, policy_version 48960 (0.0007) [2023-10-11 21:01:07,389][71635] Updated weights for policy 1, policy_version 48912 (0.0008) [2023-10-11 21:01:07,760][71635] Updated weights for policy 1, policy_version 48922 (0.0007) [2023-10-11 21:01:10,937][71601] Updated weights for policy 0, policy_version 48970 (0.0010) [2023-10-11 21:01:11,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 100237312. Throughput: 0: 1820.5, 1: 1813.0. Samples: 25075690. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:01:11,034][70582] Avg episode reward: [(0, '173.340'), (1, '94.620')] [2023-10-11 21:01:11,315][71601] Updated weights for policy 0, policy_version 48980 (0.0008) [2023-10-11 21:01:11,382][71635] Updated weights for policy 1, policy_version 48932 (0.0007) [2023-10-11 21:01:11,689][71601] Updated weights for policy 0, policy_version 48990 (0.0008) [2023-10-11 21:01:11,749][71635] Updated weights for policy 1, policy_version 48942 (0.0008) [2023-10-11 21:01:12,114][71635] Updated weights for policy 1, policy_version 48952 (0.0009) [2023-10-11 21:01:15,321][71601] Updated weights for policy 0, policy_version 49000 (0.0008) [2023-10-11 21:01:15,686][71601] Updated weights for policy 0, policy_version 49010 (0.0007) [2023-10-11 21:01:15,876][71635] Updated weights for policy 1, policy_version 48962 (0.0007) [2023-10-11 21:01:16,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 100302848. Throughput: 0: 1820.2, 1: 1814.7. Samples: 25085536. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:01:16,035][70582] Avg episode reward: [(0, '173.450'), (1, '101.600')] [2023-10-11 21:01:16,050][71601] Updated weights for policy 0, policy_version 49020 (0.0007) [2023-10-11 21:01:16,237][71635] Updated weights for policy 1, policy_version 48972 (0.0007) [2023-10-11 21:01:16,600][71635] Updated weights for policy 1, policy_version 48982 (0.0009) [2023-10-11 21:01:16,963][71635] Updated weights for policy 1, policy_version 48992 (0.0011) [2023-10-11 21:01:19,926][71601] Updated weights for policy 0, policy_version 49030 (0.0009) [2023-10-11 21:01:20,301][71601] Updated weights for policy 0, policy_version 49040 (0.0009) [2023-10-11 21:01:20,620][71635] Updated weights for policy 1, policy_version 49002 (0.0008) [2023-10-11 21:01:20,665][71601] Updated weights for policy 0, policy_version 49050 (0.0009) [2023-10-11 21:01:20,996][71635] Updated weights for policy 1, policy_version 49012 (0.0008) [2023-10-11 21:01:21,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 100401152. Throughput: 0: 1815.9, 1: 1816.5. Samples: 25108130. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:01:21,035][70582] Avg episode reward: [(0, '186.710'), (1, '98.700')] [2023-10-11 21:01:21,359][71635] Updated weights for policy 1, policy_version 49022 (0.0008) [2023-10-11 21:01:24,387][71601] Updated weights for policy 0, policy_version 49060 (0.0009) [2023-10-11 21:01:24,759][71601] Updated weights for policy 0, policy_version 49070 (0.0008) [2023-10-11 21:01:25,142][71601] Updated weights for policy 0, policy_version 49080 (0.0008) [2023-10-11 21:01:25,271][71635] Updated weights for policy 1, policy_version 49032 (0.0009) [2023-10-11 21:01:25,639][71635] Updated weights for policy 1, policy_version 49042 (0.0007) [2023-10-11 21:01:26,004][71635] Updated weights for policy 1, policy_version 49052 (0.0009) [2023-10-11 21:01:26,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 100466688. Throughput: 0: 1819.1, 1: 1815.4. Samples: 25128770. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:01:26,034][70582] Avg episode reward: [(0, '177.590'), (1, '98.120')] [2023-10-11 21:01:28,606][71601] Updated weights for policy 0, policy_version 49090 (0.0008) [2023-10-11 21:01:28,985][71601] Updated weights for policy 0, policy_version 49100 (0.0007) [2023-10-11 21:01:29,346][71601] Updated weights for policy 0, policy_version 49110 (0.0008) [2023-10-11 21:01:29,600][71635] Updated weights for policy 1, policy_version 49062 (0.0009) [2023-10-11 21:01:29,714][71601] Updated weights for policy 0, policy_version 49120 (0.0007) [2023-10-11 21:01:29,966][71635] Updated weights for policy 1, policy_version 49072 (0.0010) [2023-10-11 21:01:30,330][71635] Updated weights for policy 1, policy_version 49082 (0.0010) [2023-10-11 21:01:31,034][70582] Fps is (10 sec: 16384.2, 60 sec: 15291.8, 300 sec: 14551.2). Total num frames: 100564992. Throughput: 0: 1819.3, 1: 1818.2. Samples: 25140964. Policy #0 lag: (min: 18.0, avg: 31.2, max: 32.0) [2023-10-11 21:01:31,034][70582] Avg episode reward: [(0, '178.730'), (1, '94.810')] [2023-10-11 21:01:33,551][71601] Updated weights for policy 0, policy_version 49130 (0.0009) [2023-10-11 21:01:33,915][71601] Updated weights for policy 0, policy_version 49140 (0.0008) [2023-10-11 21:01:34,096][71635] Updated weights for policy 1, policy_version 49092 (0.0010) [2023-10-11 21:01:34,284][71601] Updated weights for policy 0, policy_version 49150 (0.0008) [2023-10-11 21:01:34,459][71635] Updated weights for policy 1, policy_version 49102 (0.0007) [2023-10-11 21:01:34,836][71635] Updated weights for policy 1, policy_version 49112 (0.0007) [2023-10-11 21:01:36,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 100630528. Throughput: 0: 1815.6, 1: 1821.5. Samples: 25161592. Policy #0 lag: (min: 18.0, avg: 31.2, max: 32.0) [2023-10-11 21:01:36,034][70582] Avg episode reward: [(0, '186.820'), (1, '94.940')] [2023-10-11 21:01:38,106][71601] Updated weights for policy 0, policy_version 49160 (0.0008) [2023-10-11 21:01:38,473][71601] Updated weights for policy 0, policy_version 49170 (0.0010) [2023-10-11 21:01:38,516][71635] Updated weights for policy 1, policy_version 49122 (0.0007) [2023-10-11 21:01:38,847][71601] Updated weights for policy 0, policy_version 49180 (0.0009) [2023-10-11 21:01:38,890][71635] Updated weights for policy 1, policy_version 49132 (0.0008) [2023-10-11 21:01:39,260][71635] Updated weights for policy 1, policy_version 49142 (0.0010) [2023-10-11 21:01:39,623][71635] Updated weights for policy 1, policy_version 49152 (0.0010) [2023-10-11 21:01:41,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 100696064. Throughput: 0: 1817.9, 1: 1813.4. Samples: 25183440. Policy #0 lag: (min: 18.0, avg: 31.2, max: 32.0) [2023-10-11 21:01:41,035][70582] Avg episode reward: [(0, '173.260'), (1, '104.870')] [2023-10-11 21:01:42,590][71601] Updated weights for policy 0, policy_version 49190 (0.0008) [2023-10-11 21:01:42,964][71601] Updated weights for policy 0, policy_version 49200 (0.0009) [2023-10-11 21:01:43,307][71635] Updated weights for policy 1, policy_version 49162 (0.0009) [2023-10-11 21:01:43,336][71601] Updated weights for policy 0, policy_version 49210 (0.0007) [2023-10-11 21:01:43,681][71635] Updated weights for policy 1, policy_version 49172 (0.0009) [2023-10-11 21:01:44,040][71635] Updated weights for policy 1, policy_version 49182 (0.0008) [2023-10-11 21:01:46,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 100761600. Throughput: 0: 1824.9, 1: 1820.8. Samples: 25194570. Policy #0 lag: (min: 18.0, avg: 31.2, max: 32.0) [2023-10-11 21:01:46,034][70582] Avg episode reward: [(0, '173.260'), (1, '106.190')] [2023-10-11 21:01:46,981][71601] Updated weights for policy 0, policy_version 49220 (0.0007) [2023-10-11 21:01:47,356][71601] Updated weights for policy 0, policy_version 49230 (0.0007) [2023-10-11 21:01:47,716][71601] Updated weights for policy 0, policy_version 49240 (0.0008) [2023-10-11 21:01:47,808][71635] Updated weights for policy 1, policy_version 49192 (0.0009) [2023-10-11 21:01:48,177][71635] Updated weights for policy 1, policy_version 49202 (0.0008) [2023-10-11 21:01:48,553][71635] Updated weights for policy 1, policy_version 49212 (0.0009) [2023-10-11 21:01:51,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 100827136. Throughput: 0: 1817.7, 1: 1810.9. Samples: 25215942. Policy #0 lag: (min: 18.0, avg: 31.2, max: 32.0) [2023-10-11 21:01:51,034][70582] Avg episode reward: [(0, '166.860'), (1, '106.100')] [2023-10-11 21:01:51,389][71601] Updated weights for policy 0, policy_version 49250 (0.0008) [2023-10-11 21:01:51,766][71601] Updated weights for policy 0, policy_version 49260 (0.0008) [2023-10-11 21:01:52,138][71601] Updated weights for policy 0, policy_version 49270 (0.0007) [2023-10-11 21:01:52,234][71635] Updated weights for policy 1, policy_version 49222 (0.0009) [2023-10-11 21:01:52,506][71601] Updated weights for policy 0, policy_version 49280 (0.0008) [2023-10-11 21:01:52,591][71635] Updated weights for policy 1, policy_version 49232 (0.0010) [2023-10-11 21:01:52,970][71635] Updated weights for policy 1, policy_version 49242 (0.0010) [2023-10-11 21:01:56,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 100892672. Throughput: 0: 1814.0, 1: 1815.8. Samples: 25239030. Policy #0 lag: (min: 18.0, avg: 31.2, max: 32.0) [2023-10-11 21:01:56,034][70582] Avg episode reward: [(0, '166.810'), (1, '107.040')] [2023-10-11 21:01:56,042][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000049248_50429952.pth... [2023-10-11 21:01:56,077][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000047552_48693248.pth [2023-10-11 21:01:56,140][71601] Updated weights for policy 0, policy_version 49290 (0.0009) [2023-10-11 21:01:56,502][71601] Updated weights for policy 0, policy_version 49300 (0.0008) [2023-10-11 21:01:56,734][71635] Updated weights for policy 1, policy_version 49252 (0.0008) [2023-10-11 21:01:56,865][71601] Updated weights for policy 0, policy_version 49310 (0.0008) [2023-10-11 21:01:56,941][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000049312_50495488.pth... [2023-10-11 21:01:56,976][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000047584_48726016.pth [2023-10-11 21:01:57,106][71635] Updated weights for policy 1, policy_version 49262 (0.0008) [2023-10-11 21:01:57,471][71635] Updated weights for policy 1, policy_version 49272 (0.0010) [2023-10-11 21:02:00,656][71601] Updated weights for policy 0, policy_version 49320 (0.0009) [2023-10-11 21:02:01,032][71601] Updated weights for policy 0, policy_version 49330 (0.0008) [2023-10-11 21:02:01,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 100958208. Throughput: 0: 1813.0, 1: 1816.7. Samples: 25248872. Policy #0 lag: (min: 18.0, avg: 31.2, max: 32.0) [2023-10-11 21:02:01,034][70582] Avg episode reward: [(0, '174.600'), (1, '106.360')] [2023-10-11 21:02:01,110][71635] Updated weights for policy 1, policy_version 49282 (0.0009) [2023-10-11 21:02:01,396][71601] Updated weights for policy 0, policy_version 49340 (0.0008) [2023-10-11 21:02:01,480][71635] Updated weights for policy 1, policy_version 49292 (0.0008) [2023-10-11 21:02:01,839][71635] Updated weights for policy 1, policy_version 49302 (0.0009) [2023-10-11 21:02:02,211][71635] Updated weights for policy 1, policy_version 49312 (0.0009) [2023-10-11 21:02:05,124][71601] Updated weights for policy 0, policy_version 49350 (0.0007) [2023-10-11 21:02:05,495][71601] Updated weights for policy 0, policy_version 49360 (0.0008) [2023-10-11 21:02:05,844][71635] Updated weights for policy 1, policy_version 49322 (0.0008) [2023-10-11 21:02:05,867][71601] Updated weights for policy 0, policy_version 49370 (0.0007) [2023-10-11 21:02:06,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 101023744. Throughput: 0: 1809.5, 1: 1822.4. Samples: 25271566. Policy #0 lag: (min: 19.0, avg: 22.2, max: 51.0) [2023-10-11 21:02:06,034][70582] Avg episode reward: [(0, '177.120'), (1, '98.380')] [2023-10-11 21:02:06,209][71635] Updated weights for policy 1, policy_version 49332 (0.0008) [2023-10-11 21:02:06,582][71635] Updated weights for policy 1, policy_version 49342 (0.0008) [2023-10-11 21:02:09,578][71601] Updated weights for policy 0, policy_version 49380 (0.0007) [2023-10-11 21:02:09,957][71601] Updated weights for policy 0, policy_version 49390 (0.0008) [2023-10-11 21:02:10,329][71635] Updated weights for policy 1, policy_version 49352 (0.0007) [2023-10-11 21:02:10,331][71601] Updated weights for policy 0, policy_version 49400 (0.0008) [2023-10-11 21:02:10,687][71635] Updated weights for policy 1, policy_version 49362 (0.0007) [2023-10-11 21:02:11,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 101122048. Throughput: 0: 1816.4, 1: 1825.9. Samples: 25292672. Policy #0 lag: (min: 19.0, avg: 22.2, max: 51.0) [2023-10-11 21:02:11,034][70582] Avg episode reward: [(0, '177.220'), (1, '98.670')] [2023-10-11 21:02:11,053][71635] Updated weights for policy 1, policy_version 49372 (0.0007) [2023-10-11 21:02:13,971][71601] Updated weights for policy 0, policy_version 49410 (0.0007) [2023-10-11 21:02:14,344][71601] Updated weights for policy 0, policy_version 49420 (0.0011) [2023-10-11 21:02:14,634][71635] Updated weights for policy 1, policy_version 49382 (0.0008) [2023-10-11 21:02:14,720][71601] Updated weights for policy 0, policy_version 49430 (0.0008) [2023-10-11 21:02:14,999][71635] Updated weights for policy 1, policy_version 49392 (0.0007) [2023-10-11 21:02:15,092][71601] Updated weights for policy 0, policy_version 49440 (0.0008) [2023-10-11 21:02:15,365][71635] Updated weights for policy 1, policy_version 49402 (0.0008) [2023-10-11 21:02:16,034][70582] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 101220352. Throughput: 0: 1808.1, 1: 1823.2. Samples: 25304374. Policy #0 lag: (min: 19.0, avg: 22.2, max: 51.0) [2023-10-11 21:02:16,035][70582] Avg episode reward: [(0, '177.220'), (1, '100.770')] [2023-10-11 21:02:18,845][71601] Updated weights for policy 0, policy_version 49450 (0.0008) [2023-10-11 21:02:18,991][71635] Updated weights for policy 1, policy_version 49412 (0.0008) [2023-10-11 21:02:19,208][71601] Updated weights for policy 0, policy_version 49460 (0.0010) [2023-10-11 21:02:19,363][71635] Updated weights for policy 1, policy_version 49422 (0.0008) [2023-10-11 21:02:19,573][71601] Updated weights for policy 0, policy_version 49470 (0.0009) [2023-10-11 21:02:19,723][71635] Updated weights for policy 1, policy_version 49432 (0.0008) [2023-10-11 21:02:21,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 101285888. Throughput: 0: 1813.1, 1: 1823.1. Samples: 25325224. Policy #0 lag: (min: 19.0, avg: 22.2, max: 51.0) [2023-10-11 21:02:21,035][70582] Avg episode reward: [(0, '177.220'), (1, '94.710')] [2023-10-11 21:02:23,395][71601] Updated weights for policy 0, policy_version 49480 (0.0007) [2023-10-11 21:02:23,484][71635] Updated weights for policy 1, policy_version 49442 (0.0009) [2023-10-11 21:02:23,768][71601] Updated weights for policy 0, policy_version 49490 (0.0008) [2023-10-11 21:02:23,835][71635] Updated weights for policy 1, policy_version 49452 (0.0009) [2023-10-11 21:02:24,130][71601] Updated weights for policy 0, policy_version 49500 (0.0009) [2023-10-11 21:02:24,204][71635] Updated weights for policy 1, policy_version 49462 (0.0009) [2023-10-11 21:02:24,569][71635] Updated weights for policy 1, policy_version 49472 (0.0007) [2023-10-11 21:02:26,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 101351424. Throughput: 0: 1805.1, 1: 1823.6. Samples: 25346730. Policy #0 lag: (min: 19.0, avg: 22.2, max: 51.0) [2023-10-11 21:02:26,034][70582] Avg episode reward: [(0, '162.920'), (1, '95.280')] [2023-10-11 21:02:27,731][71601] Updated weights for policy 0, policy_version 49510 (0.0011) [2023-10-11 21:02:28,117][71601] Updated weights for policy 0, policy_version 49520 (0.0008) [2023-10-11 21:02:28,264][71635] Updated weights for policy 1, policy_version 49482 (0.0008) [2023-10-11 21:02:28,489][71601] Updated weights for policy 0, policy_version 49530 (0.0008) [2023-10-11 21:02:28,629][71635] Updated weights for policy 1, policy_version 49492 (0.0009) [2023-10-11 21:02:29,003][71635] Updated weights for policy 1, policy_version 49502 (0.0009) [2023-10-11 21:02:31,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 101416960. Throughput: 0: 1808.1, 1: 1825.4. Samples: 25358078. Policy #0 lag: (min: 19.0, avg: 22.2, max: 51.0) [2023-10-11 21:02:31,035][70582] Avg episode reward: [(0, '161.420'), (1, '87.530')] [2023-10-11 21:02:32,264][71601] Updated weights for policy 0, policy_version 49540 (0.0009) [2023-10-11 21:02:32,637][71601] Updated weights for policy 0, policy_version 49550 (0.0008) [2023-10-11 21:02:32,745][71635] Updated weights for policy 1, policy_version 49512 (0.0008) [2023-10-11 21:02:33,004][71601] Updated weights for policy 0, policy_version 49560 (0.0008) [2023-10-11 21:02:33,105][71635] Updated weights for policy 1, policy_version 49522 (0.0008) [2023-10-11 21:02:33,473][71635] Updated weights for policy 1, policy_version 49532 (0.0008) [2023-10-11 21:02:36,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 101482496. Throughput: 0: 1797.0, 1: 1829.3. Samples: 25379128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:02:36,034][70582] Avg episode reward: [(0, '161.420'), (1, '87.370')] [2023-10-11 21:02:36,611][71601] Updated weights for policy 0, policy_version 49570 (0.0009) [2023-10-11 21:02:36,975][71601] Updated weights for policy 0, policy_version 49580 (0.0007) [2023-10-11 21:02:37,077][71635] Updated weights for policy 1, policy_version 49542 (0.0007) [2023-10-11 21:02:37,350][71601] Updated weights for policy 0, policy_version 49590 (0.0007) [2023-10-11 21:02:37,444][71635] Updated weights for policy 1, policy_version 49552 (0.0008) [2023-10-11 21:02:37,722][71601] Updated weights for policy 0, policy_version 49600 (0.0008) [2023-10-11 21:02:37,803][71635] Updated weights for policy 1, policy_version 49562 (0.0010) [2023-10-11 21:02:41,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 101548032. Throughput: 0: 1801.4, 1: 1826.6. Samples: 25402290. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:02:41,035][70582] Avg episode reward: [(0, '161.420'), (1, '96.330')] [2023-10-11 21:02:41,343][71601] Updated weights for policy 0, policy_version 49610 (0.0007) [2023-10-11 21:02:41,422][71635] Updated weights for policy 1, policy_version 49572 (0.0008) [2023-10-11 21:02:41,719][71601] Updated weights for policy 0, policy_version 49620 (0.0010) [2023-10-11 21:02:41,784][71635] Updated weights for policy 1, policy_version 49582 (0.0007) [2023-10-11 21:02:42,099][71601] Updated weights for policy 0, policy_version 49630 (0.0008) [2023-10-11 21:02:42,144][71635] Updated weights for policy 1, policy_version 49592 (0.0008) [2023-10-11 21:02:45,757][71635] Updated weights for policy 1, policy_version 49602 (0.0011) [2023-10-11 21:02:45,822][71601] Updated weights for policy 0, policy_version 49640 (0.0008) [2023-10-11 21:02:46,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 101613568. Throughput: 0: 1799.8, 1: 1823.1. Samples: 25411904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:02:46,034][70582] Avg episode reward: [(0, '161.420'), (1, '95.050')] [2023-10-11 21:02:46,114][71635] Updated weights for policy 1, policy_version 49612 (0.0009) [2023-10-11 21:02:46,194][71601] Updated weights for policy 0, policy_version 49650 (0.0008) [2023-10-11 21:02:46,481][71635] Updated weights for policy 1, policy_version 49622 (0.0007) [2023-10-11 21:02:46,565][71601] Updated weights for policy 0, policy_version 49660 (0.0008) [2023-10-11 21:02:46,850][71635] Updated weights for policy 1, policy_version 49632 (0.0008) [2023-10-11 21:02:50,314][71601] Updated weights for policy 0, policy_version 49670 (0.0009) [2023-10-11 21:02:50,587][71635] Updated weights for policy 1, policy_version 49642 (0.0008) [2023-10-11 21:02:50,681][71601] Updated weights for policy 0, policy_version 49680 (0.0008) [2023-10-11 21:02:50,959][71635] Updated weights for policy 1, policy_version 49652 (0.0007) [2023-10-11 21:02:51,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 101679104. Throughput: 0: 1799.6, 1: 1820.0. Samples: 25434448. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:02:51,034][70582] Avg episode reward: [(0, '161.420'), (1, '94.630')] [2023-10-11 21:02:51,053][71601] Updated weights for policy 0, policy_version 49690 (0.0008) [2023-10-11 21:02:51,322][71635] Updated weights for policy 1, policy_version 49662 (0.0009) [2023-10-11 21:02:54,739][71601] Updated weights for policy 0, policy_version 49700 (0.0007) [2023-10-11 21:02:55,104][71635] Updated weights for policy 1, policy_version 49672 (0.0008) [2023-10-11 21:02:55,109][71601] Updated weights for policy 0, policy_version 49710 (0.0009) [2023-10-11 21:02:55,462][71635] Updated weights for policy 1, policy_version 49682 (0.0007) [2023-10-11 21:02:55,483][71601] Updated weights for policy 0, policy_version 49720 (0.0007) [2023-10-11 21:02:55,824][71635] Updated weights for policy 1, policy_version 49692 (0.0008) [2023-10-11 21:02:56,034][70582] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 101810176. Throughput: 0: 1803.0, 1: 1816.7. Samples: 25455556. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:02:56,034][70582] Avg episode reward: [(0, '161.420'), (1, '91.120')] [2023-10-11 21:02:59,170][71601] Updated weights for policy 0, policy_version 49730 (0.0009) [2023-10-11 21:02:59,532][71601] Updated weights for policy 0, policy_version 49740 (0.0009) [2023-10-11 21:02:59,626][71635] Updated weights for policy 1, policy_version 49702 (0.0007) [2023-10-11 21:02:59,905][71601] Updated weights for policy 0, policy_version 49750 (0.0008) [2023-10-11 21:02:59,993][71635] Updated weights for policy 1, policy_version 49712 (0.0009) [2023-10-11 21:03:00,274][71601] Updated weights for policy 0, policy_version 49760 (0.0008) [2023-10-11 21:03:00,351][71635] Updated weights for policy 1, policy_version 49722 (0.0009) [2023-10-11 21:03:01,034][70582] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 101875712. Throughput: 0: 1797.6, 1: 1817.8. Samples: 25467068. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:03:01,035][70582] Avg episode reward: [(0, '174.410'), (1, '89.990')] [2023-10-11 21:03:03,929][71601] Updated weights for policy 0, policy_version 49770 (0.0009) [2023-10-11 21:03:04,155][71635] Updated weights for policy 1, policy_version 49732 (0.0009) [2023-10-11 21:03:04,303][71601] Updated weights for policy 0, policy_version 49780 (0.0008) [2023-10-11 21:03:04,531][71635] Updated weights for policy 1, policy_version 49742 (0.0008) [2023-10-11 21:03:04,679][71601] Updated weights for policy 0, policy_version 49790 (0.0009) [2023-10-11 21:03:04,905][71635] Updated weights for policy 1, policy_version 49752 (0.0009) [2023-10-11 21:03:06,034][70582] Fps is (10 sec: 13107.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 101941248. Throughput: 0: 1807.2, 1: 1819.9. Samples: 25488440. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:03:06,034][70582] Avg episode reward: [(0, '180.010'), (1, '88.090')] [2023-10-11 21:03:08,443][71635] Updated weights for policy 1, policy_version 49762 (0.0010) [2023-10-11 21:03:08,491][71601] Updated weights for policy 0, policy_version 49800 (0.0007) [2023-10-11 21:03:08,810][71635] Updated weights for policy 1, policy_version 49772 (0.0008) [2023-10-11 21:03:08,867][71601] Updated weights for policy 0, policy_version 49810 (0.0007) [2023-10-11 21:03:09,172][71635] Updated weights for policy 1, policy_version 49782 (0.0008) [2023-10-11 21:03:09,239][71601] Updated weights for policy 0, policy_version 49820 (0.0007) [2023-10-11 21:03:09,542][71635] Updated weights for policy 1, policy_version 49792 (0.0009) [2023-10-11 21:03:11,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 102006784. Throughput: 0: 1804.4, 1: 1816.0. Samples: 25509650. Policy #0 lag: (min: 16.0, avg: 36.1, max: 48.0) [2023-10-11 21:03:11,034][70582] Avg episode reward: [(0, '183.920'), (1, '96.510')] [2023-10-11 21:03:13,034][71601] Updated weights for policy 0, policy_version 49830 (0.0009) [2023-10-11 21:03:13,403][71601] Updated weights for policy 0, policy_version 49840 (0.0007) [2023-10-11 21:03:13,495][71635] Updated weights for policy 1, policy_version 49802 (0.0007) [2023-10-11 21:03:13,788][71601] Updated weights for policy 0, policy_version 49850 (0.0009) [2023-10-11 21:03:13,876][71635] Updated weights for policy 1, policy_version 49812 (0.0008) [2023-10-11 21:03:14,238][71635] Updated weights for policy 1, policy_version 49822 (0.0008) [2023-10-11 21:03:16,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 102072320. Throughput: 0: 1808.9, 1: 1813.6. Samples: 25521088. Policy #0 lag: (min: 16.0, avg: 36.1, max: 48.0) [2023-10-11 21:03:16,034][70582] Avg episode reward: [(0, '180.480'), (1, '85.690')] [2023-10-11 21:03:17,447][71601] Updated weights for policy 0, policy_version 49860 (0.0008) [2023-10-11 21:03:17,806][71601] Updated weights for policy 0, policy_version 49870 (0.0008) [2023-10-11 21:03:17,910][71635] Updated weights for policy 1, policy_version 49832 (0.0009) [2023-10-11 21:03:18,190][71601] Updated weights for policy 0, policy_version 49880 (0.0008) [2023-10-11 21:03:18,277][71635] Updated weights for policy 1, policy_version 49842 (0.0009) [2023-10-11 21:03:18,641][71635] Updated weights for policy 1, policy_version 49852 (0.0009) [2023-10-11 21:03:21,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 102137856. Throughput: 0: 1803.4, 1: 1809.2. Samples: 25541694. Policy #0 lag: (min: 16.0, avg: 36.1, max: 48.0) [2023-10-11 21:03:21,035][70582] Avg episode reward: [(0, '188.720'), (1, '81.110')] [2023-10-11 21:03:22,049][71601] Updated weights for policy 0, policy_version 49890 (0.0007) [2023-10-11 21:03:22,396][71635] Updated weights for policy 1, policy_version 49862 (0.0007) [2023-10-11 21:03:22,422][71601] Updated weights for policy 0, policy_version 49900 (0.0009) [2023-10-11 21:03:22,761][71635] Updated weights for policy 1, policy_version 49872 (0.0008) [2023-10-11 21:03:22,791][71601] Updated weights for policy 0, policy_version 49910 (0.0008) [2023-10-11 21:03:23,126][71635] Updated weights for policy 1, policy_version 49882 (0.0008) [2023-10-11 21:03:23,161][71601] Updated weights for policy 0, policy_version 49920 (0.0008) [2023-10-11 21:03:26,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 102203392. Throughput: 0: 1799.7, 1: 1803.8. Samples: 25564450. Policy #0 lag: (min: 16.0, avg: 36.1, max: 48.0) [2023-10-11 21:03:26,035][70582] Avg episode reward: [(0, '188.990'), (1, '81.480')] [2023-10-11 21:03:26,838][71601] Updated weights for policy 0, policy_version 49930 (0.0009) [2023-10-11 21:03:26,952][71635] Updated weights for policy 1, policy_version 49892 (0.0009) [2023-10-11 21:03:27,204][71601] Updated weights for policy 0, policy_version 49940 (0.0007) [2023-10-11 21:03:27,312][71635] Updated weights for policy 1, policy_version 49902 (0.0007) [2023-10-11 21:03:27,581][71601] Updated weights for policy 0, policy_version 49950 (0.0007) [2023-10-11 21:03:27,677][71635] Updated weights for policy 1, policy_version 49912 (0.0007) [2023-10-11 21:03:31,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 102268928. Throughput: 0: 1805.4, 1: 1802.7. Samples: 25574270. Policy #0 lag: (min: 16.0, avg: 36.1, max: 48.0) [2023-10-11 21:03:31,034][70582] Avg episode reward: [(0, '205.730'), (1, '93.140')] [2023-10-11 21:03:31,353][71601] Updated weights for policy 0, policy_version 49960 (0.0008) [2023-10-11 21:03:31,507][71635] Updated weights for policy 1, policy_version 49922 (0.0010) [2023-10-11 21:03:31,729][71601] Updated weights for policy 0, policy_version 49970 (0.0007) [2023-10-11 21:03:31,866][71635] Updated weights for policy 1, policy_version 49932 (0.0007) [2023-10-11 21:03:32,100][71601] Updated weights for policy 0, policy_version 49980 (0.0007) [2023-10-11 21:03:32,228][71635] Updated weights for policy 1, policy_version 49942 (0.0008) [2023-10-11 21:03:32,590][71635] Updated weights for policy 1, policy_version 49952 (0.0007) [2023-10-11 21:03:35,658][71601] Updated weights for policy 0, policy_version 49990 (0.0008) [2023-10-11 21:03:36,023][71601] Updated weights for policy 0, policy_version 50000 (0.0009) [2023-10-11 21:03:36,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 102334464. Throughput: 0: 1808.5, 1: 1794.2. Samples: 25596570. Policy #0 lag: (min: 16.0, avg: 36.1, max: 48.0) [2023-10-11 21:03:36,035][70582] Avg episode reward: [(0, '201.030'), (1, '91.980')] [2023-10-11 21:03:36,338][71635] Updated weights for policy 1, policy_version 49962 (0.0007) [2023-10-11 21:03:36,389][71601] Updated weights for policy 0, policy_version 50010 (0.0009) [2023-10-11 21:03:36,696][71635] Updated weights for policy 1, policy_version 49972 (0.0008) [2023-10-11 21:03:37,064][71635] Updated weights for policy 1, policy_version 49982 (0.0007) [2023-10-11 21:03:40,118][71601] Updated weights for policy 0, policy_version 50020 (0.0008) [2023-10-11 21:03:40,483][71601] Updated weights for policy 0, policy_version 50030 (0.0010) [2023-10-11 21:03:40,709][71635] Updated weights for policy 1, policy_version 49992 (0.0007) [2023-10-11 21:03:40,856][71601] Updated weights for policy 0, policy_version 50040 (0.0008) [2023-10-11 21:03:41,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 102400000. Throughput: 0: 1819.2, 1: 1808.8. Samples: 25618812. Policy #0 lag: (min: 16.0, avg: 36.1, max: 48.0) [2023-10-11 21:03:41,034][70582] Avg episode reward: [(0, '200.820'), (1, '92.360')] [2023-10-11 21:03:41,082][71635] Updated weights for policy 1, policy_version 50002 (0.0008) [2023-10-11 21:03:41,446][71635] Updated weights for policy 1, policy_version 50012 (0.0008) [2023-10-11 21:03:44,623][71601] Updated weights for policy 0, policy_version 50050 (0.0009) [2023-10-11 21:03:44,996][71601] Updated weights for policy 0, policy_version 50060 (0.0008) [2023-10-11 21:03:45,051][71635] Updated weights for policy 1, policy_version 50022 (0.0008) [2023-10-11 21:03:45,370][71601] Updated weights for policy 0, policy_version 50070 (0.0007) [2023-10-11 21:03:45,425][71635] Updated weights for policy 1, policy_version 50032 (0.0008) [2023-10-11 21:03:45,733][71601] Updated weights for policy 0, policy_version 50080 (0.0008) [2023-10-11 21:03:45,794][71635] Updated weights for policy 1, policy_version 50042 (0.0007) [2023-10-11 21:03:46,034][70582] Fps is (10 sec: 19661.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 102531072. Throughput: 0: 1810.9, 1: 1796.4. Samples: 25629394. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:03:46,034][70582] Avg episode reward: [(0, '200.930'), (1, '95.850')] [2023-10-11 21:03:49,427][71601] Updated weights for policy 0, policy_version 50090 (0.0009) [2023-10-11 21:03:49,534][71635] Updated weights for policy 1, policy_version 50052 (0.0008) [2023-10-11 21:03:49,790][71601] Updated weights for policy 0, policy_version 50100 (0.0009) [2023-10-11 21:03:49,899][71635] Updated weights for policy 1, policy_version 50062 (0.0009) [2023-10-11 21:03:50,165][71601] Updated weights for policy 0, policy_version 50110 (0.0008) [2023-10-11 21:03:50,267][71635] Updated weights for policy 1, policy_version 50072 (0.0008) [2023-10-11 21:03:51,034][70582] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 102596608. Throughput: 0: 1820.4, 1: 1799.7. Samples: 25651344. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:03:51,035][70582] Avg episode reward: [(0, '201.710'), (1, '99.700')] [2023-10-11 21:03:53,949][71601] Updated weights for policy 0, policy_version 50120 (0.0010) [2023-10-11 21:03:53,991][71635] Updated weights for policy 1, policy_version 50082 (0.0007) [2023-10-11 21:03:54,315][71601] Updated weights for policy 0, policy_version 50130 (0.0007) [2023-10-11 21:03:54,360][71635] Updated weights for policy 1, policy_version 50092 (0.0009) [2023-10-11 21:03:54,688][71601] Updated weights for policy 0, policy_version 50140 (0.0008) [2023-10-11 21:03:54,727][71635] Updated weights for policy 1, policy_version 50102 (0.0007) [2023-10-11 21:03:55,093][71635] Updated weights for policy 1, policy_version 50112 (0.0007) [2023-10-11 21:03:56,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 102662144. Throughput: 0: 1805.9, 1: 1790.3. Samples: 25671480. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:03:56,035][70582] Avg episode reward: [(0, '210.540'), (1, '105.620')] [2023-10-11 21:03:56,049][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000050144_51347456.pth... [2023-10-11 21:03:56,049][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000050112_51314688.pth... [2023-10-11 21:03:56,083][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000048448_49610752.pth [2023-10-11 21:03:56,084][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000048416_49577984.pth [2023-10-11 21:03:58,331][71601] Updated weights for policy 0, policy_version 50150 (0.0009) [2023-10-11 21:03:58,703][71601] Updated weights for policy 0, policy_version 50160 (0.0007) [2023-10-11 21:03:58,937][71635] Updated weights for policy 1, policy_version 50122 (0.0007) [2023-10-11 21:03:59,070][71601] Updated weights for policy 0, policy_version 50170 (0.0009) [2023-10-11 21:03:59,308][71635] Updated weights for policy 1, policy_version 50132 (0.0007) [2023-10-11 21:03:59,670][71635] Updated weights for policy 1, policy_version 50142 (0.0010) [2023-10-11 21:04:01,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 102727680. Throughput: 0: 1820.2, 1: 1802.1. Samples: 25684092. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:04:01,034][70582] Avg episode reward: [(0, '208.930'), (1, '106.000')] [2023-10-11 21:04:02,818][71601] Updated weights for policy 0, policy_version 50180 (0.0009) [2023-10-11 21:04:03,194][71601] Updated weights for policy 0, policy_version 50190 (0.0007) [2023-10-11 21:04:03,472][71635] Updated weights for policy 1, policy_version 50152 (0.0010) [2023-10-11 21:04:03,562][71601] Updated weights for policy 0, policy_version 50200 (0.0007) [2023-10-11 21:04:03,842][71635] Updated weights for policy 1, policy_version 50162 (0.0008) [2023-10-11 21:04:04,200][71635] Updated weights for policy 1, policy_version 50172 (0.0009) [2023-10-11 21:04:06,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 102793216. Throughput: 0: 1815.1, 1: 1790.9. Samples: 25703964. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:04:06,035][70582] Avg episode reward: [(0, '225.800'), (1, '113.450')] [2023-10-11 21:04:07,166][71601] Updated weights for policy 0, policy_version 50210 (0.0008) [2023-10-11 21:04:07,527][71601] Updated weights for policy 0, policy_version 50220 (0.0008) [2023-10-11 21:04:07,880][71635] Updated weights for policy 1, policy_version 50182 (0.0008) [2023-10-11 21:04:07,903][71601] Updated weights for policy 0, policy_version 50230 (0.0007) [2023-10-11 21:04:08,241][71635] Updated weights for policy 1, policy_version 50192 (0.0008) [2023-10-11 21:04:08,267][71601] Updated weights for policy 0, policy_version 50240 (0.0007) [2023-10-11 21:04:08,617][71635] Updated weights for policy 1, policy_version 50202 (0.0010) [2023-10-11 21:04:11,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 102858752. Throughput: 0: 1816.9, 1: 1791.2. Samples: 25726812. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:04:11,035][70582] Avg episode reward: [(0, '219.000'), (1, '113.930')] [2023-10-11 21:04:12,005][71601] Updated weights for policy 0, policy_version 50250 (0.0008) [2023-10-11 21:04:12,291][71635] Updated weights for policy 1, policy_version 50212 (0.0010) [2023-10-11 21:04:12,374][71601] Updated weights for policy 0, policy_version 50260 (0.0007) [2023-10-11 21:04:12,649][71635] Updated weights for policy 1, policy_version 50222 (0.0008) [2023-10-11 21:04:12,745][71601] Updated weights for policy 0, policy_version 50270 (0.0007) [2023-10-11 21:04:13,022][71635] Updated weights for policy 1, policy_version 50232 (0.0008) [2023-10-11 21:04:16,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 102924288. Throughput: 0: 1813.3, 1: 1796.1. Samples: 25736694. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:04:16,034][70582] Avg episode reward: [(0, '213.850'), (1, '114.250')] [2023-10-11 21:04:16,367][71601] Updated weights for policy 0, policy_version 50280 (0.0007) [2023-10-11 21:04:16,727][71601] Updated weights for policy 0, policy_version 50290 (0.0007) [2023-10-11 21:04:16,885][71635] Updated weights for policy 1, policy_version 50242 (0.0008) [2023-10-11 21:04:17,102][71601] Updated weights for policy 0, policy_version 50300 (0.0008) [2023-10-11 21:04:17,254][71635] Updated weights for policy 1, policy_version 50252 (0.0008) [2023-10-11 21:04:17,628][71635] Updated weights for policy 1, policy_version 50262 (0.0007) [2023-10-11 21:04:17,994][71635] Updated weights for policy 1, policy_version 50272 (0.0010) [2023-10-11 21:04:20,814][71601] Updated weights for policy 0, policy_version 50310 (0.0010) [2023-10-11 21:04:21,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 102989824. Throughput: 0: 1814.4, 1: 1800.5. Samples: 25759240. Policy #0 lag: (min: 22.0, avg: 25.1, max: 54.0) [2023-10-11 21:04:21,034][70582] Avg episode reward: [(0, '208.880'), (1, '121.720')] [2023-10-11 21:04:21,183][71601] Updated weights for policy 0, policy_version 50320 (0.0009) [2023-10-11 21:04:21,557][71601] Updated weights for policy 0, policy_version 50330 (0.0010) [2023-10-11 21:04:21,867][71635] Updated weights for policy 1, policy_version 50282 (0.0007) [2023-10-11 21:04:22,230][71635] Updated weights for policy 1, policy_version 50292 (0.0009) [2023-10-11 21:04:22,612][71635] Updated weights for policy 1, policy_version 50302 (0.0010) [2023-10-11 21:04:25,200][71601] Updated weights for policy 0, policy_version 50340 (0.0010) [2023-10-11 21:04:25,563][71601] Updated weights for policy 0, policy_version 50350 (0.0009) [2023-10-11 21:04:25,932][71601] Updated weights for policy 0, policy_version 50360 (0.0009) [2023-10-11 21:04:26,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 103055360. Throughput: 0: 1823.4, 1: 1798.0. Samples: 25781776. Policy #0 lag: (min: 22.0, avg: 25.1, max: 54.0) [2023-10-11 21:04:26,035][70582] Avg episode reward: [(0, '199.400'), (1, '130.840')] [2023-10-11 21:04:26,261][71635] Updated weights for policy 1, policy_version 50312 (0.0008) [2023-10-11 21:04:26,626][71635] Updated weights for policy 1, policy_version 50322 (0.0009) [2023-10-11 21:04:26,992][71635] Updated weights for policy 1, policy_version 50332 (0.0009) [2023-10-11 21:04:29,503][71601] Updated weights for policy 0, policy_version 50370 (0.0008) [2023-10-11 21:04:29,888][71601] Updated weights for policy 0, policy_version 50380 (0.0009) [2023-10-11 21:04:30,255][71601] Updated weights for policy 0, policy_version 50390 (0.0009) [2023-10-11 21:04:30,606][71635] Updated weights for policy 1, policy_version 50342 (0.0008) [2023-10-11 21:04:30,617][71601] Updated weights for policy 0, policy_version 50400 (0.0008) [2023-10-11 21:04:30,985][71635] Updated weights for policy 1, policy_version 50352 (0.0010) [2023-10-11 21:04:31,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 103153664. Throughput: 0: 1819.0, 1: 1797.6. Samples: 25792142. Policy #0 lag: (min: 22.0, avg: 25.1, max: 54.0) [2023-10-11 21:04:31,034][70582] Avg episode reward: [(0, '200.690'), (1, '123.020')] [2023-10-11 21:04:31,349][71635] Updated weights for policy 1, policy_version 50362 (0.0010) [2023-10-11 21:04:34,300][71601] Updated weights for policy 0, policy_version 50410 (0.0009) [2023-10-11 21:04:34,671][71601] Updated weights for policy 0, policy_version 50420 (0.0007) [2023-10-11 21:04:35,023][71635] Updated weights for policy 1, policy_version 50372 (0.0008) [2023-10-11 21:04:35,047][71601] Updated weights for policy 0, policy_version 50430 (0.0008) [2023-10-11 21:04:35,387][71635] Updated weights for policy 1, policy_version 50382 (0.0010) [2023-10-11 21:04:35,763][71635] Updated weights for policy 1, policy_version 50392 (0.0008) [2023-10-11 21:04:36,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 103219200. Throughput: 0: 1820.3, 1: 1800.4. Samples: 25814272. Policy #0 lag: (min: 22.0, avg: 25.1, max: 54.0) [2023-10-11 21:04:36,034][70582] Avg episode reward: [(0, '195.710'), (1, '118.330')] [2023-10-11 21:04:38,658][71601] Updated weights for policy 0, policy_version 50440 (0.0008) [2023-10-11 21:04:39,029][71601] Updated weights for policy 0, policy_version 50450 (0.0010) [2023-10-11 21:04:39,408][71601] Updated weights for policy 0, policy_version 50460 (0.0010) [2023-10-11 21:04:39,584][71635] Updated weights for policy 1, policy_version 50402 (0.0008) [2023-10-11 21:04:39,952][71635] Updated weights for policy 1, policy_version 50412 (0.0007) [2023-10-11 21:04:40,319][71635] Updated weights for policy 1, policy_version 50422 (0.0008) [2023-10-11 21:04:40,684][71635] Updated weights for policy 1, policy_version 50432 (0.0009) [2023-10-11 21:04:41,034][70582] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 103317504. Throughput: 0: 1833.4, 1: 1807.2. Samples: 25835306. Policy #0 lag: (min: 22.0, avg: 25.1, max: 54.0) [2023-10-11 21:04:41,034][70582] Avg episode reward: [(0, '189.420'), (1, '113.790')] [2023-10-11 21:04:43,082][71601] Updated weights for policy 0, policy_version 50470 (0.0009) [2023-10-11 21:04:43,449][71601] Updated weights for policy 0, policy_version 50480 (0.0008) [2023-10-11 21:04:43,821][71601] Updated weights for policy 0, policy_version 50490 (0.0009) [2023-10-11 21:04:44,511][71635] Updated weights for policy 1, policy_version 50442 (0.0011) [2023-10-11 21:04:44,877][71635] Updated weights for policy 1, policy_version 50452 (0.0007) [2023-10-11 21:04:45,242][71635] Updated weights for policy 1, policy_version 50462 (0.0007) [2023-10-11 21:04:46,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 103383040. Throughput: 0: 1824.5, 1: 1798.1. Samples: 25847108. Policy #0 lag: (min: 22.0, avg: 25.1, max: 54.0) [2023-10-11 21:04:46,035][70582] Avg episode reward: [(0, '189.540'), (1, '125.260')] [2023-10-11 21:04:47,457][71601] Updated weights for policy 0, policy_version 50500 (0.0010) [2023-10-11 21:04:47,828][71601] Updated weights for policy 0, policy_version 50510 (0.0009) [2023-10-11 21:04:48,200][71601] Updated weights for policy 0, policy_version 50520 (0.0010) [2023-10-11 21:04:49,149][71635] Updated weights for policy 1, policy_version 50472 (0.0008) [2023-10-11 21:04:49,515][71635] Updated weights for policy 1, policy_version 50482 (0.0009) [2023-10-11 21:04:49,882][71635] Updated weights for policy 1, policy_version 50492 (0.0009) [2023-10-11 21:04:51,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 103448576. Throughput: 0: 1837.9, 1: 1817.1. Samples: 25868438. Policy #0 lag: (min: 22.0, avg: 25.1, max: 54.0) [2023-10-11 21:04:51,034][70582] Avg episode reward: [(0, '193.860'), (1, '128.160')] [2023-10-11 21:04:51,858][71601] Updated weights for policy 0, policy_version 50530 (0.0008) [2023-10-11 21:04:52,229][71601] Updated weights for policy 0, policy_version 50540 (0.0007) [2023-10-11 21:04:52,598][71601] Updated weights for policy 0, policy_version 50550 (0.0008) [2023-10-11 21:04:52,971][71601] Updated weights for policy 0, policy_version 50560 (0.0010) [2023-10-11 21:04:53,606][71635] Updated weights for policy 1, policy_version 50502 (0.0009) [2023-10-11 21:04:53,960][71635] Updated weights for policy 1, policy_version 50512 (0.0009) [2023-10-11 21:04:54,329][71635] Updated weights for policy 1, policy_version 50522 (0.0008) [2023-10-11 21:04:56,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 103514112. Throughput: 0: 1838.4, 1: 1800.7. Samples: 25890572. Policy #0 lag: (min: 10.0, avg: 21.0, max: 42.0) [2023-10-11 21:04:56,034][70582] Avg episode reward: [(0, '174.480'), (1, '134.360')] [2023-10-11 21:04:56,683][71601] Updated weights for policy 0, policy_version 50570 (0.0008) [2023-10-11 21:04:57,061][71601] Updated weights for policy 0, policy_version 50580 (0.0008) [2023-10-11 21:04:57,430][71601] Updated weights for policy 0, policy_version 50590 (0.0008) [2023-10-11 21:04:57,743][71635] Updated weights for policy 1, policy_version 50532 (0.0007) [2023-10-11 21:04:58,114][71635] Updated weights for policy 1, policy_version 50542 (0.0007) [2023-10-11 21:04:58,480][71635] Updated weights for policy 1, policy_version 50552 (0.0007) [2023-10-11 21:05:01,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 103579648. Throughput: 0: 1840.7, 1: 1819.4. Samples: 25901396. Policy #0 lag: (min: 10.0, avg: 21.0, max: 42.0) [2023-10-11 21:05:01,035][70582] Avg episode reward: [(0, '160.540'), (1, '135.590')] [2023-10-11 21:05:01,093][71601] Updated weights for policy 0, policy_version 50600 (0.0007) [2023-10-11 21:05:01,468][71601] Updated weights for policy 0, policy_version 50610 (0.0009) [2023-10-11 21:05:01,849][71601] Updated weights for policy 0, policy_version 50620 (0.0008) [2023-10-11 21:05:02,130][71635] Updated weights for policy 1, policy_version 50562 (0.0007) [2023-10-11 21:05:02,505][71635] Updated weights for policy 1, policy_version 50572 (0.0008) [2023-10-11 21:05:02,868][71635] Updated weights for policy 1, policy_version 50582 (0.0007) [2023-10-11 21:05:03,229][71635] Updated weights for policy 1, policy_version 50592 (0.0010) [2023-10-11 21:05:05,603][71601] Updated weights for policy 0, policy_version 50630 (0.0008) [2023-10-11 21:05:05,972][71601] Updated weights for policy 0, policy_version 50640 (0.0007) [2023-10-11 21:05:06,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 103645184. Throughput: 0: 1841.2, 1: 1810.0. Samples: 25923548. Policy #0 lag: (min: 10.0, avg: 21.0, max: 42.0) [2023-10-11 21:05:06,035][70582] Avg episode reward: [(0, '161.130'), (1, '130.520')] [2023-10-11 21:05:06,343][71601] Updated weights for policy 0, policy_version 50650 (0.0007) [2023-10-11 21:05:06,821][71635] Updated weights for policy 1, policy_version 50602 (0.0009) [2023-10-11 21:05:07,195][71635] Updated weights for policy 1, policy_version 50612 (0.0007) [2023-10-11 21:05:07,563][71635] Updated weights for policy 1, policy_version 50622 (0.0008) [2023-10-11 21:05:09,975][71601] Updated weights for policy 0, policy_version 50660 (0.0007) [2023-10-11 21:05:10,347][71601] Updated weights for policy 0, policy_version 50670 (0.0007) [2023-10-11 21:05:10,717][71601] Updated weights for policy 0, policy_version 50680 (0.0007) [2023-10-11 21:05:11,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 103743488. Throughput: 0: 1827.3, 1: 1814.6. Samples: 25945664. Policy #0 lag: (min: 10.0, avg: 21.0, max: 42.0) [2023-10-11 21:05:11,034][70582] Avg episode reward: [(0, '149.530'), (1, '117.180')] [2023-10-11 21:05:11,373][71635] Updated weights for policy 1, policy_version 50632 (0.0007) [2023-10-11 21:05:11,737][71635] Updated weights for policy 1, policy_version 50642 (0.0008) [2023-10-11 21:05:12,109][71635] Updated weights for policy 1, policy_version 50652 (0.0008) [2023-10-11 21:05:14,303][71601] Updated weights for policy 0, policy_version 50690 (0.0008) [2023-10-11 21:05:14,677][71601] Updated weights for policy 0, policy_version 50700 (0.0007) [2023-10-11 21:05:15,052][71601] Updated weights for policy 0, policy_version 50710 (0.0007) [2023-10-11 21:05:15,423][71601] Updated weights for policy 0, policy_version 50720 (0.0009) [2023-10-11 21:05:15,679][71635] Updated weights for policy 1, policy_version 50662 (0.0009) [2023-10-11 21:05:16,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 103809024. Throughput: 0: 1838.8, 1: 1817.1. Samples: 25956658. Policy #0 lag: (min: 10.0, avg: 21.0, max: 42.0) [2023-10-11 21:05:16,035][70582] Avg episode reward: [(0, '151.360'), (1, '106.520')] [2023-10-11 21:05:16,042][71635] Updated weights for policy 1, policy_version 50672 (0.0008) [2023-10-11 21:05:16,410][71635] Updated weights for policy 1, policy_version 50682 (0.0009) [2023-10-11 21:05:19,083][71601] Updated weights for policy 0, policy_version 50730 (0.0008) [2023-10-11 21:05:19,458][71601] Updated weights for policy 0, policy_version 50740 (0.0008) [2023-10-11 21:05:19,821][71601] Updated weights for policy 0, policy_version 50750 (0.0007) [2023-10-11 21:05:20,002][71635] Updated weights for policy 1, policy_version 50692 (0.0007) [2023-10-11 21:05:20,374][71635] Updated weights for policy 1, policy_version 50702 (0.0007) [2023-10-11 21:05:20,736][71635] Updated weights for policy 1, policy_version 50712 (0.0008) [2023-10-11 21:05:21,034][70582] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 103907328. Throughput: 0: 1827.8, 1: 1821.1. Samples: 25978474. Policy #0 lag: (min: 10.0, avg: 21.0, max: 42.0) [2023-10-11 21:05:21,034][70582] Avg episode reward: [(0, '152.410'), (1, '93.430')] [2023-10-11 21:05:23,607][71601] Updated weights for policy 0, policy_version 50760 (0.0009) [2023-10-11 21:05:23,991][71601] Updated weights for policy 0, policy_version 50770 (0.0009) [2023-10-11 21:05:24,356][71601] Updated weights for policy 0, policy_version 50780 (0.0008) [2023-10-11 21:05:24,362][71635] Updated weights for policy 1, policy_version 50722 (0.0011) [2023-10-11 21:05:24,723][71635] Updated weights for policy 1, policy_version 50732 (0.0008) [2023-10-11 21:05:25,088][71635] Updated weights for policy 1, policy_version 50742 (0.0009) [2023-10-11 21:05:25,455][71635] Updated weights for policy 1, policy_version 50752 (0.0007) [2023-10-11 21:05:26,034][70582] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 103972864. Throughput: 0: 1826.3, 1: 1821.0. Samples: 25999436. Policy #0 lag: (min: 10.0, avg: 21.0, max: 42.0) [2023-10-11 21:05:26,035][70582] Avg episode reward: [(0, '138.630'), (1, '90.700')] [2023-10-11 21:05:28,000][71601] Updated weights for policy 0, policy_version 50790 (0.0007) [2023-10-11 21:05:28,369][71601] Updated weights for policy 0, policy_version 50800 (0.0011) [2023-10-11 21:05:28,736][71601] Updated weights for policy 0, policy_version 50810 (0.0009) [2023-10-11 21:05:29,311][71635] Updated weights for policy 1, policy_version 50762 (0.0009) [2023-10-11 21:05:29,689][71635] Updated weights for policy 1, policy_version 50772 (0.0009) [2023-10-11 21:05:30,058][71635] Updated weights for policy 1, policy_version 50782 (0.0009) [2023-10-11 21:05:31,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 104038400. Throughput: 0: 1825.2, 1: 1824.9. Samples: 26011366. Policy #0 lag: (min: 31.0, avg: 32.2, max: 55.0) [2023-10-11 21:05:31,034][70582] Avg episode reward: [(0, '140.200'), (1, '89.200')] [2023-10-11 21:05:32,482][71601] Updated weights for policy 0, policy_version 50820 (0.0008) [2023-10-11 21:05:32,856][71601] Updated weights for policy 0, policy_version 50830 (0.0007) [2023-10-11 21:05:33,231][71601] Updated weights for policy 0, policy_version 50840 (0.0009) [2023-10-11 21:05:33,736][71635] Updated weights for policy 1, policy_version 50792 (0.0008) [2023-10-11 21:05:34,105][71635] Updated weights for policy 1, policy_version 50802 (0.0010) [2023-10-11 21:05:34,472][71635] Updated weights for policy 1, policy_version 50812 (0.0007) [2023-10-11 21:05:36,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 104103936. Throughput: 0: 1819.2, 1: 1819.0. Samples: 26032154. Policy #0 lag: (min: 31.0, avg: 32.2, max: 55.0) [2023-10-11 21:05:36,034][70582] Avg episode reward: [(0, '150.040'), (1, '88.320')] [2023-10-11 21:05:36,856][71601] Updated weights for policy 0, policy_version 50850 (0.0008) [2023-10-11 21:05:37,226][71601] Updated weights for policy 0, policy_version 50860 (0.0007) [2023-10-11 21:05:37,603][71601] Updated weights for policy 0, policy_version 50870 (0.0009) [2023-10-11 21:05:37,971][71601] Updated weights for policy 0, policy_version 50880 (0.0008) [2023-10-11 21:05:38,156][71635] Updated weights for policy 1, policy_version 50822 (0.0007) [2023-10-11 21:05:38,529][71635] Updated weights for policy 1, policy_version 50832 (0.0008) [2023-10-11 21:05:38,893][71635] Updated weights for policy 1, policy_version 50842 (0.0009) [2023-10-11 21:05:41,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 104169472. Throughput: 0: 1817.1, 1: 1826.7. Samples: 26054540. Policy #0 lag: (min: 31.0, avg: 32.2, max: 55.0) [2023-10-11 21:05:41,035][70582] Avg episode reward: [(0, '150.830'), (1, '75.250')] [2023-10-11 21:05:41,659][71601] Updated weights for policy 0, policy_version 50890 (0.0009) [2023-10-11 21:05:42,028][71601] Updated weights for policy 0, policy_version 50900 (0.0010) [2023-10-11 21:05:42,413][71601] Updated weights for policy 0, policy_version 50910 (0.0012) [2023-10-11 21:05:42,604][71635] Updated weights for policy 1, policy_version 50852 (0.0010) [2023-10-11 21:05:42,975][71635] Updated weights for policy 1, policy_version 50862 (0.0009) [2023-10-11 21:05:43,341][71635] Updated weights for policy 1, policy_version 50872 (0.0009) [2023-10-11 21:05:46,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 104235008. Throughput: 0: 1812.6, 1: 1817.4. Samples: 26064746. Policy #0 lag: (min: 31.0, avg: 32.2, max: 55.0) [2023-10-11 21:05:46,034][70582] Avg episode reward: [(0, '135.010'), (1, '71.340')] [2023-10-11 21:05:46,231][71601] Updated weights for policy 0, policy_version 50920 (0.0008) [2023-10-11 21:05:46,598][71601] Updated weights for policy 0, policy_version 50930 (0.0010) [2023-10-11 21:05:46,975][71601] Updated weights for policy 0, policy_version 50940 (0.0010) [2023-10-11 21:05:47,116][71635] Updated weights for policy 1, policy_version 50882 (0.0008) [2023-10-11 21:05:47,476][71635] Updated weights for policy 1, policy_version 50892 (0.0010) [2023-10-11 21:05:47,852][71635] Updated weights for policy 1, policy_version 50902 (0.0009) [2023-10-11 21:05:48,214][71635] Updated weights for policy 1, policy_version 50912 (0.0009) [2023-10-11 21:05:50,682][71601] Updated weights for policy 0, policy_version 50950 (0.0008) [2023-10-11 21:05:51,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 104300544. Throughput: 0: 1812.6, 1: 1818.0. Samples: 26086922. Policy #0 lag: (min: 31.0, avg: 32.2, max: 55.0) [2023-10-11 21:05:51,035][70582] Avg episode reward: [(0, '135.130'), (1, '69.620')] [2023-10-11 21:05:51,051][71601] Updated weights for policy 0, policy_version 50960 (0.0008) [2023-10-11 21:05:51,430][71601] Updated weights for policy 0, policy_version 50970 (0.0007) [2023-10-11 21:05:51,918][71635] Updated weights for policy 1, policy_version 50922 (0.0009) [2023-10-11 21:05:52,277][71635] Updated weights for policy 1, policy_version 50932 (0.0010) [2023-10-11 21:05:52,643][71635] Updated weights for policy 1, policy_version 50942 (0.0010) [2023-10-11 21:05:55,102][71601] Updated weights for policy 0, policy_version 50980 (0.0008) [2023-10-11 21:05:55,475][71601] Updated weights for policy 0, policy_version 50990 (0.0007) [2023-10-11 21:05:55,850][71601] Updated weights for policy 0, policy_version 51000 (0.0009) [2023-10-11 21:05:56,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 104366080. Throughput: 0: 1813.4, 1: 1814.7. Samples: 26108930. Policy #0 lag: (min: 31.0, avg: 32.2, max: 55.0) [2023-10-11 21:05:56,034][70582] Avg episode reward: [(0, '118.240'), (1, '61.250')] [2023-10-11 21:05:56,140][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000051008_52232192.pth... [2023-10-11 21:05:56,179][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000049312_50495488.pth [2023-10-11 21:05:56,185][71353] Saving a milestone ./train_atari/atari_gopher_APPO/checkpoint_p0/milestones/checkpoint_000051008_52232192.pth [2023-10-11 21:05:56,393][71635] Updated weights for policy 1, policy_version 50952 (0.0011) [2023-10-11 21:05:56,756][71635] Updated weights for policy 1, policy_version 50962 (0.0009) [2023-10-11 21:05:57,121][71635] Updated weights for policy 1, policy_version 50972 (0.0007) [2023-10-11 21:05:57,269][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000050976_52199424.pth... [2023-10-11 21:05:57,298][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000049248_50429952.pth [2023-10-11 21:05:57,303][71431] Saving a milestone ./train_atari/atari_gopher_APPO/checkpoint_p1/milestones/checkpoint_000050976_52199424.pth [2023-10-11 21:05:59,577][71601] Updated weights for policy 0, policy_version 51010 (0.0010) [2023-10-11 21:05:59,956][71601] Updated weights for policy 0, policy_version 51020 (0.0010) [2023-10-11 21:06:00,317][71601] Updated weights for policy 0, policy_version 51030 (0.0008) [2023-10-11 21:06:00,687][71601] Updated weights for policy 0, policy_version 51040 (0.0009) [2023-10-11 21:06:00,899][71635] Updated weights for policy 1, policy_version 50982 (0.0008) [2023-10-11 21:06:01,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 104464384. Throughput: 0: 1803.8, 1: 1812.2. Samples: 26119378. Policy #0 lag: (min: 31.0, avg: 32.2, max: 55.0) [2023-10-11 21:06:01,035][70582] Avg episode reward: [(0, '118.230'), (1, '61.960')] [2023-10-11 21:06:01,262][71635] Updated weights for policy 1, policy_version 50992 (0.0007) [2023-10-11 21:06:01,633][71635] Updated weights for policy 1, policy_version 51002 (0.0010) [2023-10-11 21:06:04,286][71601] Updated weights for policy 0, policy_version 51050 (0.0008) [2023-10-11 21:06:04,666][71601] Updated weights for policy 0, policy_version 51060 (0.0007) [2023-10-11 21:06:05,039][71601] Updated weights for policy 0, policy_version 51070 (0.0008) [2023-10-11 21:06:05,309][71635] Updated weights for policy 1, policy_version 51012 (0.0009) [2023-10-11 21:06:05,674][71635] Updated weights for policy 1, policy_version 51022 (0.0007) [2023-10-11 21:06:06,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 104529920. Throughput: 0: 1814.4, 1: 1809.2. Samples: 26141536. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-11 21:06:06,034][70582] Avg episode reward: [(0, '114.750'), (1, '68.010')] [2023-10-11 21:06:06,047][71635] Updated weights for policy 1, policy_version 51032 (0.0007) [2023-10-11 21:06:08,812][71601] Updated weights for policy 0, policy_version 51080 (0.0008) [2023-10-11 21:06:09,193][71601] Updated weights for policy 0, policy_version 51090 (0.0009) [2023-10-11 21:06:09,566][71601] Updated weights for policy 0, policy_version 51100 (0.0011) [2023-10-11 21:06:09,888][71635] Updated weights for policy 1, policy_version 51042 (0.0008) [2023-10-11 21:06:10,255][71635] Updated weights for policy 1, policy_version 51052 (0.0008) [2023-10-11 21:06:10,620][71635] Updated weights for policy 1, policy_version 51062 (0.0007) [2023-10-11 21:06:10,984][71635] Updated weights for policy 1, policy_version 51072 (0.0007) [2023-10-11 21:06:11,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 104628224. Throughput: 0: 1809.8, 1: 1820.2. Samples: 26162788. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-11 21:06:11,035][70582] Avg episode reward: [(0, '117.170'), (1, '67.730')] [2023-10-11 21:06:13,169][71601] Updated weights for policy 0, policy_version 51110 (0.0009) [2023-10-11 21:06:13,544][71601] Updated weights for policy 0, policy_version 51120 (0.0007) [2023-10-11 21:06:13,922][71601] Updated weights for policy 0, policy_version 51130 (0.0010) [2023-10-11 21:06:14,750][71635] Updated weights for policy 1, policy_version 51082 (0.0009) [2023-10-11 21:06:15,118][71635] Updated weights for policy 1, policy_version 51092 (0.0008) [2023-10-11 21:06:15,482][71635] Updated weights for policy 1, policy_version 51102 (0.0009) [2023-10-11 21:06:16,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 104693760. Throughput: 0: 1812.0, 1: 1804.9. Samples: 26174128. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-11 21:06:16,034][70582] Avg episode reward: [(0, '117.020'), (1, '83.660')] [2023-10-11 21:06:17,623][71601] Updated weights for policy 0, policy_version 51140 (0.0009) [2023-10-11 21:06:17,997][71601] Updated weights for policy 0, policy_version 51150 (0.0009) [2023-10-11 21:06:18,370][71601] Updated weights for policy 0, policy_version 51160 (0.0007) [2023-10-11 21:06:19,213][71635] Updated weights for policy 1, policy_version 51112 (0.0012) [2023-10-11 21:06:19,575][71635] Updated weights for policy 1, policy_version 51122 (0.0011) [2023-10-11 21:06:19,953][71635] Updated weights for policy 1, policy_version 51132 (0.0009) [2023-10-11 21:06:21,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 104759296. Throughput: 0: 1808.8, 1: 1814.9. Samples: 26195218. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-11 21:06:21,034][70582] Avg episode reward: [(0, '104.970'), (1, '86.400')] [2023-10-11 21:06:22,058][71601] Updated weights for policy 0, policy_version 51170 (0.0008) [2023-10-11 21:06:22,425][71601] Updated weights for policy 0, policy_version 51180 (0.0008) [2023-10-11 21:06:22,804][71601] Updated weights for policy 0, policy_version 51190 (0.0007) [2023-10-11 21:06:23,176][71601] Updated weights for policy 0, policy_version 51200 (0.0008) [2023-10-11 21:06:23,591][71635] Updated weights for policy 1, policy_version 51142 (0.0009) [2023-10-11 21:06:23,956][71635] Updated weights for policy 1, policy_version 51152 (0.0007) [2023-10-11 21:06:24,325][71635] Updated weights for policy 1, policy_version 51162 (0.0008) [2023-10-11 21:06:26,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 104824832. Throughput: 0: 1807.3, 1: 1811.8. Samples: 26217398. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-11 21:06:26,034][70582] Avg episode reward: [(0, '104.470'), (1, '84.000')] [2023-10-11 21:06:26,925][71601] Updated weights for policy 0, policy_version 51210 (0.0008) [2023-10-11 21:06:27,296][71601] Updated weights for policy 0, policy_version 51220 (0.0008) [2023-10-11 21:06:27,672][71601] Updated weights for policy 0, policy_version 51230 (0.0009) [2023-10-11 21:06:27,866][71635] Updated weights for policy 1, policy_version 51172 (0.0009) [2023-10-11 21:06:28,228][71635] Updated weights for policy 1, policy_version 51182 (0.0008) [2023-10-11 21:06:28,590][71635] Updated weights for policy 1, policy_version 51192 (0.0010) [2023-10-11 21:06:31,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 104890368. Throughput: 0: 1813.3, 1: 1821.0. Samples: 26228288. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-11 21:06:31,035][70582] Avg episode reward: [(0, '104.430'), (1, '79.640')] [2023-10-11 21:06:31,307][71601] Updated weights for policy 0, policy_version 51240 (0.0009) [2023-10-11 21:06:31,674][71601] Updated weights for policy 0, policy_version 51250 (0.0008) [2023-10-11 21:06:32,046][71601] Updated weights for policy 0, policy_version 51260 (0.0007) [2023-10-11 21:06:32,340][71635] Updated weights for policy 1, policy_version 51202 (0.0010) [2023-10-11 21:06:32,710][71635] Updated weights for policy 1, policy_version 51212 (0.0008) [2023-10-11 21:06:33,071][71635] Updated weights for policy 1, policy_version 51222 (0.0007) [2023-10-11 21:06:33,441][71635] Updated weights for policy 1, policy_version 51232 (0.0007) [2023-10-11 21:06:35,671][71601] Updated weights for policy 0, policy_version 51270 (0.0008) [2023-10-11 21:06:36,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 104955904. Throughput: 0: 1813.9, 1: 1814.8. Samples: 26250210. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-11 21:06:36,034][70582] Avg episode reward: [(0, '106.540'), (1, '84.590')] [2023-10-11 21:06:36,042][71601] Updated weights for policy 0, policy_version 51280 (0.0007) [2023-10-11 21:06:36,421][71601] Updated weights for policy 0, policy_version 51290 (0.0007) [2023-10-11 21:06:37,164][71635] Updated weights for policy 1, policy_version 51242 (0.0007) [2023-10-11 21:06:37,526][71635] Updated weights for policy 1, policy_version 51252 (0.0007) [2023-10-11 21:06:37,905][71635] Updated weights for policy 1, policy_version 51262 (0.0008) [2023-10-11 21:06:40,082][71601] Updated weights for policy 0, policy_version 51300 (0.0007) [2023-10-11 21:06:40,461][71601] Updated weights for policy 0, policy_version 51310 (0.0009) [2023-10-11 21:06:40,823][71601] Updated weights for policy 0, policy_version 51320 (0.0009) [2023-10-11 21:06:41,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 105021440. Throughput: 0: 1820.5, 1: 1813.9. Samples: 26272482. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-11 21:06:41,035][70582] Avg episode reward: [(0, '106.530'), (1, '82.960')] [2023-10-11 21:06:41,591][71635] Updated weights for policy 1, policy_version 51272 (0.0008) [2023-10-11 21:06:41,960][71635] Updated weights for policy 1, policy_version 51282 (0.0007) [2023-10-11 21:06:42,331][71635] Updated weights for policy 1, policy_version 51292 (0.0010) [2023-10-11 21:06:44,472][71601] Updated weights for policy 0, policy_version 51330 (0.0009) [2023-10-11 21:06:44,848][71601] Updated weights for policy 0, policy_version 51340 (0.0008) [2023-10-11 21:06:45,221][71601] Updated weights for policy 0, policy_version 51350 (0.0009) [2023-10-11 21:06:45,587][71601] Updated weights for policy 0, policy_version 51360 (0.0007) [2023-10-11 21:06:45,964][71635] Updated weights for policy 1, policy_version 51302 (0.0009) [2023-10-11 21:06:46,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 105119744. Throughput: 0: 1821.1, 1: 1812.9. Samples: 26282906. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:06:46,034][70582] Avg episode reward: [(0, '106.990'), (1, '87.580')] [2023-10-11 21:06:46,320][71635] Updated weights for policy 1, policy_version 51312 (0.0007) [2023-10-11 21:06:46,682][71635] Updated weights for policy 1, policy_version 51322 (0.0008) [2023-10-11 21:06:49,339][71601] Updated weights for policy 0, policy_version 51370 (0.0010) [2023-10-11 21:06:49,709][71601] Updated weights for policy 0, policy_version 51380 (0.0009) [2023-10-11 21:06:50,077][71601] Updated weights for policy 0, policy_version 51390 (0.0009) [2023-10-11 21:06:50,397][71635] Updated weights for policy 1, policy_version 51332 (0.0008) [2023-10-11 21:06:50,770][71635] Updated weights for policy 1, policy_version 51342 (0.0007) [2023-10-11 21:06:51,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 105185280. Throughput: 0: 1821.2, 1: 1816.7. Samples: 26305246. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:06:51,035][70582] Avg episode reward: [(0, '106.990'), (1, '90.170')] [2023-10-11 21:06:51,127][71635] Updated weights for policy 1, policy_version 51352 (0.0007) [2023-10-11 21:06:53,809][71601] Updated weights for policy 0, policy_version 51400 (0.0008) [2023-10-11 21:06:54,191][71601] Updated weights for policy 0, policy_version 51410 (0.0009) [2023-10-11 21:06:54,553][71601] Updated weights for policy 0, policy_version 51420 (0.0007) [2023-10-11 21:06:54,902][71635] Updated weights for policy 1, policy_version 51362 (0.0008) [2023-10-11 21:06:55,270][71635] Updated weights for policy 1, policy_version 51372 (0.0009) [2023-10-11 21:06:55,638][71635] Updated weights for policy 1, policy_version 51382 (0.0007) [2023-10-11 21:06:55,987][71635] Updated weights for policy 1, policy_version 51392 (0.0008) [2023-10-11 21:06:56,034][70582] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 105283584. Throughput: 0: 1820.7, 1: 1815.7. Samples: 26326428. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:06:56,034][70582] Avg episode reward: [(0, '106.990'), (1, '84.350')] [2023-10-11 21:06:58,166][71601] Updated weights for policy 0, policy_version 51430 (0.0007) [2023-10-11 21:06:58,537][71601] Updated weights for policy 0, policy_version 51440 (0.0008) [2023-10-11 21:06:58,907][71601] Updated weights for policy 0, policy_version 51450 (0.0008) [2023-10-11 21:06:59,670][71635] Updated weights for policy 1, policy_version 51402 (0.0007) [2023-10-11 21:07:00,039][71635] Updated weights for policy 1, policy_version 51412 (0.0008) [2023-10-11 21:07:00,399][71635] Updated weights for policy 1, policy_version 51422 (0.0010) [2023-10-11 21:07:01,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 105349120. Throughput: 0: 1823.9, 1: 1818.7. Samples: 26338044. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:07:01,035][70582] Avg episode reward: [(0, '106.990'), (1, '85.040')] [2023-10-11 21:07:02,760][71601] Updated weights for policy 0, policy_version 51460 (0.0008) [2023-10-11 21:07:03,128][71601] Updated weights for policy 0, policy_version 51470 (0.0007) [2023-10-11 21:07:03,499][71601] Updated weights for policy 0, policy_version 51480 (0.0007) [2023-10-11 21:07:04,054][71635] Updated weights for policy 1, policy_version 51432 (0.0010) [2023-10-11 21:07:04,419][71635] Updated weights for policy 1, policy_version 51442 (0.0008) [2023-10-11 21:07:04,793][71635] Updated weights for policy 1, policy_version 51452 (0.0008) [2023-10-11 21:07:06,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 105414656. Throughput: 0: 1822.4, 1: 1821.8. Samples: 26359208. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:07:06,035][70582] Avg episode reward: [(0, '106.990'), (1, '78.820')] [2023-10-11 21:07:07,213][71601] Updated weights for policy 0, policy_version 51490 (0.0008) [2023-10-11 21:07:07,581][71601] Updated weights for policy 0, policy_version 51500 (0.0009) [2023-10-11 21:07:07,963][71601] Updated weights for policy 0, policy_version 51510 (0.0007) [2023-10-11 21:07:08,332][71601] Updated weights for policy 0, policy_version 51520 (0.0007) [2023-10-11 21:07:08,474][71635] Updated weights for policy 1, policy_version 51462 (0.0007) [2023-10-11 21:07:08,837][71635] Updated weights for policy 1, policy_version 51472 (0.0007) [2023-10-11 21:07:09,211][71635] Updated weights for policy 1, policy_version 51482 (0.0008) [2023-10-11 21:07:11,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 105480192. Throughput: 0: 1822.2, 1: 1818.0. Samples: 26381206. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:07:11,035][70582] Avg episode reward: [(0, '116.090'), (1, '87.880')] [2023-10-11 21:07:11,987][71601] Updated weights for policy 0, policy_version 51530 (0.0007) [2023-10-11 21:07:12,353][71601] Updated weights for policy 0, policy_version 51540 (0.0008) [2023-10-11 21:07:12,733][71601] Updated weights for policy 0, policy_version 51550 (0.0010) [2023-10-11 21:07:12,870][71635] Updated weights for policy 1, policy_version 51492 (0.0008) [2023-10-11 21:07:13,242][71635] Updated weights for policy 1, policy_version 51502 (0.0009) [2023-10-11 21:07:13,608][71635] Updated weights for policy 1, policy_version 51512 (0.0008) [2023-10-11 21:07:16,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 105545728. Throughput: 0: 1821.4, 1: 1814.5. Samples: 26391902. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:07:16,034][70582] Avg episode reward: [(0, '108.620'), (1, '71.950')] [2023-10-11 21:07:16,252][71601] Updated weights for policy 0, policy_version 51560 (0.0008) [2023-10-11 21:07:16,626][71601] Updated weights for policy 0, policy_version 51570 (0.0008) [2023-10-11 21:07:16,990][71601] Updated weights for policy 0, policy_version 51580 (0.0007) [2023-10-11 21:07:17,286][71635] Updated weights for policy 1, policy_version 51522 (0.0009) [2023-10-11 21:07:17,649][71635] Updated weights for policy 1, policy_version 51532 (0.0011) [2023-10-11 21:07:18,018][71635] Updated weights for policy 1, policy_version 51542 (0.0011) [2023-10-11 21:07:18,383][71635] Updated weights for policy 1, policy_version 51552 (0.0008) [2023-10-11 21:07:20,655][71601] Updated weights for policy 0, policy_version 51590 (0.0008) [2023-10-11 21:07:21,026][71601] Updated weights for policy 0, policy_version 51600 (0.0010) [2023-10-11 21:07:21,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 105611264. Throughput: 0: 1828.3, 1: 1820.3. Samples: 26414396. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:07:21,034][70582] Avg episode reward: [(0, '115.640'), (1, '74.830')] [2023-10-11 21:07:21,393][71601] Updated weights for policy 0, policy_version 51610 (0.0009) [2023-10-11 21:07:21,954][71635] Updated weights for policy 1, policy_version 51562 (0.0008) [2023-10-11 21:07:22,315][71635] Updated weights for policy 1, policy_version 51572 (0.0008) [2023-10-11 21:07:22,680][71635] Updated weights for policy 1, policy_version 51582 (0.0008) [2023-10-11 21:07:25,028][71601] Updated weights for policy 0, policy_version 51620 (0.0008) [2023-10-11 21:07:25,403][71601] Updated weights for policy 0, policy_version 51630 (0.0010) [2023-10-11 21:07:25,774][71601] Updated weights for policy 0, policy_version 51640 (0.0010) [2023-10-11 21:07:26,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 105676800. Throughput: 0: 1828.2, 1: 1824.5. Samples: 26436854. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:07:26,035][70582] Avg episode reward: [(0, '115.450'), (1, '76.760')] [2023-10-11 21:07:26,272][71635] Updated weights for policy 1, policy_version 51592 (0.0009) [2023-10-11 21:07:26,642][71635] Updated weights for policy 1, policy_version 51602 (0.0009) [2023-10-11 21:07:27,007][71635] Updated weights for policy 1, policy_version 51612 (0.0010) [2023-10-11 21:07:29,391][71601] Updated weights for policy 0, policy_version 51650 (0.0008) [2023-10-11 21:07:29,769][71601] Updated weights for policy 0, policy_version 51660 (0.0007) [2023-10-11 21:07:30,141][71601] Updated weights for policy 0, policy_version 51670 (0.0008) [2023-10-11 21:07:30,509][71601] Updated weights for policy 0, policy_version 51680 (0.0008) [2023-10-11 21:07:30,650][71635] Updated weights for policy 1, policy_version 51622 (0.0009) [2023-10-11 21:07:31,013][71635] Updated weights for policy 1, policy_version 51632 (0.0007) [2023-10-11 21:07:31,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 105775104. Throughput: 0: 1832.2, 1: 1826.0. Samples: 26447528. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:07:31,034][70582] Avg episode reward: [(0, '119.340'), (1, '80.640')] [2023-10-11 21:07:31,384][71635] Updated weights for policy 1, policy_version 51642 (0.0011) [2023-10-11 21:07:34,165][71601] Updated weights for policy 0, policy_version 51690 (0.0008) [2023-10-11 21:07:34,533][71601] Updated weights for policy 0, policy_version 51700 (0.0011) [2023-10-11 21:07:34,907][71601] Updated weights for policy 0, policy_version 51710 (0.0008) [2023-10-11 21:07:35,159][71635] Updated weights for policy 1, policy_version 51652 (0.0008) [2023-10-11 21:07:35,523][71635] Updated weights for policy 1, policy_version 51662 (0.0009) [2023-10-11 21:07:35,887][71635] Updated weights for policy 1, policy_version 51672 (0.0010) [2023-10-11 21:07:36,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 105840640. Throughput: 0: 1824.6, 1: 1829.7. Samples: 26469692. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:07:36,035][70582] Avg episode reward: [(0, '114.700'), (1, '79.920')] [2023-10-11 21:07:38,587][71601] Updated weights for policy 0, policy_version 51720 (0.0008) [2023-10-11 21:07:38,962][71601] Updated weights for policy 0, policy_version 51730 (0.0009) [2023-10-11 21:07:39,326][71601] Updated weights for policy 0, policy_version 51740 (0.0008) [2023-10-11 21:07:39,561][71635] Updated weights for policy 1, policy_version 51682 (0.0011) [2023-10-11 21:07:39,934][71635] Updated weights for policy 1, policy_version 51692 (0.0007) [2023-10-11 21:07:40,295][71635] Updated weights for policy 1, policy_version 51702 (0.0007) [2023-10-11 21:07:40,670][71635] Updated weights for policy 1, policy_version 51712 (0.0008) [2023-10-11 21:07:41,034][70582] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 105938944. Throughput: 0: 1831.4, 1: 1822.0. Samples: 26490828. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:07:41,034][70582] Avg episode reward: [(0, '114.700'), (1, '75.730')] [2023-10-11 21:07:43,030][71601] Updated weights for policy 0, policy_version 51750 (0.0009) [2023-10-11 21:07:43,400][71601] Updated weights for policy 0, policy_version 51760 (0.0009) [2023-10-11 21:07:43,771][71601] Updated weights for policy 0, policy_version 51770 (0.0008) [2023-10-11 21:07:44,453][71635] Updated weights for policy 1, policy_version 51722 (0.0008) [2023-10-11 21:07:44,818][71635] Updated weights for policy 1, policy_version 51732 (0.0008) [2023-10-11 21:07:45,183][71635] Updated weights for policy 1, policy_version 51742 (0.0009) [2023-10-11 21:07:46,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 106004480. Throughput: 0: 1824.7, 1: 1834.1. Samples: 26502690. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:07:46,035][70582] Avg episode reward: [(0, '112.720'), (1, '83.030')] [2023-10-11 21:07:47,429][71601] Updated weights for policy 0, policy_version 51780 (0.0009) [2023-10-11 21:07:47,797][71601] Updated weights for policy 0, policy_version 51790 (0.0008) [2023-10-11 21:07:48,174][71601] Updated weights for policy 0, policy_version 51800 (0.0008) [2023-10-11 21:07:48,862][71635] Updated weights for policy 1, policy_version 51752 (0.0010) [2023-10-11 21:07:49,236][71635] Updated weights for policy 1, policy_version 51762 (0.0010) [2023-10-11 21:07:49,602][71635] Updated weights for policy 1, policy_version 51772 (0.0010) [2023-10-11 21:07:51,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 106070016. Throughput: 0: 1834.2, 1: 1825.6. Samples: 26523896. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:07:51,034][70582] Avg episode reward: [(0, '128.890'), (1, '79.910')] [2023-10-11 21:07:51,829][71601] Updated weights for policy 0, policy_version 51810 (0.0008) [2023-10-11 21:07:52,198][71601] Updated weights for policy 0, policy_version 51820 (0.0010) [2023-10-11 21:07:52,567][71601] Updated weights for policy 0, policy_version 51830 (0.0008) [2023-10-11 21:07:52,942][71601] Updated weights for policy 0, policy_version 51840 (0.0007) [2023-10-11 21:07:53,286][71635] Updated weights for policy 1, policy_version 51782 (0.0008) [2023-10-11 21:07:53,643][71635] Updated weights for policy 1, policy_version 51792 (0.0008) [2023-10-11 21:07:54,006][71635] Updated weights for policy 1, policy_version 51802 (0.0010) [2023-10-11 21:07:56,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 106135552. Throughput: 0: 1836.0, 1: 1834.3. Samples: 26546368. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-10-11 21:07:56,034][70582] Avg episode reward: [(0, '122.180'), (1, '76.640')] [2023-10-11 21:07:56,043][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000051808_53051392.pth... [2023-10-11 21:07:56,076][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000050112_51314688.pth [2023-10-11 21:07:56,451][71601] Updated weights for policy 0, policy_version 51850 (0.0008) [2023-10-11 21:07:56,818][71601] Updated weights for policy 0, policy_version 51860 (0.0007) [2023-10-11 21:07:57,202][71601] Updated weights for policy 0, policy_version 51870 (0.0007) [2023-10-11 21:07:57,270][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000051872_53116928.pth... [2023-10-11 21:07:57,310][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000050144_51347456.pth [2023-10-11 21:07:57,715][71635] Updated weights for policy 1, policy_version 51812 (0.0010) [2023-10-11 21:07:58,084][71635] Updated weights for policy 1, policy_version 51822 (0.0009) [2023-10-11 21:07:58,454][71635] Updated weights for policy 1, policy_version 51832 (0.0009) [2023-10-11 21:08:00,805][71601] Updated weights for policy 0, policy_version 51880 (0.0009) [2023-10-11 21:08:01,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 106201088. Throughput: 0: 1839.4, 1: 1827.4. Samples: 26556908. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-10-11 21:08:01,034][70582] Avg episode reward: [(0, '122.220'), (1, '76.490')] [2023-10-11 21:08:01,184][71601] Updated weights for policy 0, policy_version 51890 (0.0010) [2023-10-11 21:08:01,557][71601] Updated weights for policy 0, policy_version 51900 (0.0007) [2023-10-11 21:08:02,127][71635] Updated weights for policy 1, policy_version 51842 (0.0010) [2023-10-11 21:08:02,500][71635] Updated weights for policy 1, policy_version 51852 (0.0009) [2023-10-11 21:08:02,869][71635] Updated weights for policy 1, policy_version 51862 (0.0008) [2023-10-11 21:08:03,233][71635] Updated weights for policy 1, policy_version 51872 (0.0007) [2023-10-11 21:08:05,231][71601] Updated weights for policy 0, policy_version 51910 (0.0008) [2023-10-11 21:08:05,610][71601] Updated weights for policy 0, policy_version 51920 (0.0009) [2023-10-11 21:08:05,976][71601] Updated weights for policy 0, policy_version 51930 (0.0008) [2023-10-11 21:08:06,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 106266624. Throughput: 0: 1833.8, 1: 1829.3. Samples: 26579236. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-10-11 21:08:06,034][70582] Avg episode reward: [(0, '122.220'), (1, '66.770')] [2023-10-11 21:08:06,855][71635] Updated weights for policy 1, policy_version 51882 (0.0009) [2023-10-11 21:08:07,232][71635] Updated weights for policy 1, policy_version 51892 (0.0008) [2023-10-11 21:08:07,598][71635] Updated weights for policy 1, policy_version 51902 (0.0008) [2023-10-11 21:08:09,608][71601] Updated weights for policy 0, policy_version 51940 (0.0008) [2023-10-11 21:08:09,975][71601] Updated weights for policy 0, policy_version 51950 (0.0007) [2023-10-11 21:08:10,354][71601] Updated weights for policy 0, policy_version 51960 (0.0008) [2023-10-11 21:08:11,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 106364928. Throughput: 0: 1816.4, 1: 1831.3. Samples: 26600998. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-10-11 21:08:11,035][70582] Avg episode reward: [(0, '122.220'), (1, '51.960')] [2023-10-11 21:08:11,222][71635] Updated weights for policy 1, policy_version 51912 (0.0009) [2023-10-11 21:08:11,592][71635] Updated weights for policy 1, policy_version 51922 (0.0009) [2023-10-11 21:08:11,961][71635] Updated weights for policy 1, policy_version 51932 (0.0009) [2023-10-11 21:08:13,882][71601] Updated weights for policy 0, policy_version 51970 (0.0008) [2023-10-11 21:08:14,243][71601] Updated weights for policy 0, policy_version 51980 (0.0010) [2023-10-11 21:08:14,623][71601] Updated weights for policy 0, policy_version 51990 (0.0011) [2023-10-11 21:08:14,977][71601] Updated weights for policy 0, policy_version 52000 (0.0009) [2023-10-11 21:08:15,714][71635] Updated weights for policy 1, policy_version 51942 (0.0008) [2023-10-11 21:08:16,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 106430464. Throughput: 0: 1832.4, 1: 1833.6. Samples: 26612498. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-10-11 21:08:16,034][70582] Avg episode reward: [(0, '122.220'), (1, '51.940')] [2023-10-11 21:08:16,087][71635] Updated weights for policy 1, policy_version 51952 (0.0010) [2023-10-11 21:08:16,455][71635] Updated weights for policy 1, policy_version 51962 (0.0008) [2023-10-11 21:08:18,710][71601] Updated weights for policy 0, policy_version 52010 (0.0009) [2023-10-11 21:08:19,085][71601] Updated weights for policy 0, policy_version 52020 (0.0009) [2023-10-11 21:08:19,452][71601] Updated weights for policy 0, policy_version 52030 (0.0009) [2023-10-11 21:08:20,153][71635] Updated weights for policy 1, policy_version 51972 (0.0009) [2023-10-11 21:08:20,512][71635] Updated weights for policy 1, policy_version 51982 (0.0008) [2023-10-11 21:08:20,878][71635] Updated weights for policy 1, policy_version 51992 (0.0008) [2023-10-11 21:08:21,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 106496000. Throughput: 0: 1826.1, 1: 1827.0. Samples: 26634082. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-10-11 21:08:21,034][70582] Avg episode reward: [(0, '136.790'), (1, '51.930')] [2023-10-11 21:08:23,249][71601] Updated weights for policy 0, policy_version 52040 (0.0007) [2023-10-11 21:08:23,620][71601] Updated weights for policy 0, policy_version 52050 (0.0008) [2023-10-11 21:08:23,998][71601] Updated weights for policy 0, policy_version 52060 (0.0008) [2023-10-11 21:08:24,329][71635] Updated weights for policy 1, policy_version 52002 (0.0007) [2023-10-11 21:08:24,689][71635] Updated weights for policy 1, policy_version 52012 (0.0007) [2023-10-11 21:08:25,060][71635] Updated weights for policy 1, policy_version 52022 (0.0008) [2023-10-11 21:08:25,426][71635] Updated weights for policy 1, policy_version 52032 (0.0007) [2023-10-11 21:08:26,034][70582] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 106594304. Throughput: 0: 1837.5, 1: 1825.5. Samples: 26655666. Policy #0 lag: (min: 8.0, avg: 35.6, max: 40.0) [2023-10-11 21:08:26,035][70582] Avg episode reward: [(0, '146.890'), (1, '53.510')] [2023-10-11 21:08:27,702][71601] Updated weights for policy 0, policy_version 52070 (0.0011) [2023-10-11 21:08:28,074][71601] Updated weights for policy 0, policy_version 52080 (0.0010) [2023-10-11 21:08:28,442][71601] Updated weights for policy 0, policy_version 52090 (0.0009) [2023-10-11 21:08:29,162][71635] Updated weights for policy 1, policy_version 52042 (0.0009) [2023-10-11 21:08:29,534][71635] Updated weights for policy 1, policy_version 52052 (0.0009) [2023-10-11 21:08:29,909][71635] Updated weights for policy 1, policy_version 52062 (0.0009) [2023-10-11 21:08:31,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 106659840. Throughput: 0: 1827.1, 1: 1829.9. Samples: 26667256. Policy #0 lag: (min: 20.0, avg: 45.5, max: 48.0) [2023-10-11 21:08:31,035][70582] Avg episode reward: [(0, '143.240'), (1, '54.760')] [2023-10-11 21:08:32,126][71601] Updated weights for policy 0, policy_version 52100 (0.0010) [2023-10-11 21:08:32,493][71601] Updated weights for policy 0, policy_version 52110 (0.0010) [2023-10-11 21:08:32,867][71601] Updated weights for policy 0, policy_version 52120 (0.0009) [2023-10-11 21:08:33,580][71635] Updated weights for policy 1, policy_version 52072 (0.0008) [2023-10-11 21:08:33,950][71635] Updated weights for policy 1, policy_version 52082 (0.0009) [2023-10-11 21:08:34,322][71635] Updated weights for policy 1, policy_version 52092 (0.0010) [2023-10-11 21:08:36,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 106725376. Throughput: 0: 1839.0, 1: 1816.8. Samples: 26688410. Policy #0 lag: (min: 20.0, avg: 45.5, max: 48.0) [2023-10-11 21:08:36,034][70582] Avg episode reward: [(0, '143.280'), (1, '54.850')] [2023-10-11 21:08:36,428][71601] Updated weights for policy 0, policy_version 52130 (0.0007) [2023-10-11 21:08:36,799][71601] Updated weights for policy 0, policy_version 52140 (0.0009) [2023-10-11 21:08:37,167][71601] Updated weights for policy 0, policy_version 52150 (0.0009) [2023-10-11 21:08:37,541][71601] Updated weights for policy 0, policy_version 52160 (0.0007) [2023-10-11 21:08:38,148][71635] Updated weights for policy 1, policy_version 52102 (0.0008) [2023-10-11 21:08:38,512][71635] Updated weights for policy 1, policy_version 52112 (0.0008) [2023-10-11 21:08:38,880][71635] Updated weights for policy 1, policy_version 52122 (0.0009) [2023-10-11 21:08:41,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 106790912. Throughput: 0: 1842.0, 1: 1819.1. Samples: 26711116. Policy #0 lag: (min: 20.0, avg: 45.5, max: 48.0) [2023-10-11 21:08:41,034][70582] Avg episode reward: [(0, '165.460'), (1, '54.240')] [2023-10-11 21:08:41,147][71601] Updated weights for policy 0, policy_version 52170 (0.0008) [2023-10-11 21:08:41,516][71601] Updated weights for policy 0, policy_version 52180 (0.0009) [2023-10-11 21:08:41,896][71601] Updated weights for policy 0, policy_version 52190 (0.0007) [2023-10-11 21:08:42,588][71635] Updated weights for policy 1, policy_version 52132 (0.0008) [2023-10-11 21:08:42,955][71635] Updated weights for policy 1, policy_version 52142 (0.0007) [2023-10-11 21:08:43,319][71635] Updated weights for policy 1, policy_version 52152 (0.0008) [2023-10-11 21:08:45,480][71601] Updated weights for policy 0, policy_version 52200 (0.0007) [2023-10-11 21:08:45,858][71601] Updated weights for policy 0, policy_version 52210 (0.0007) [2023-10-11 21:08:46,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 106856448. Throughput: 0: 1840.5, 1: 1819.2. Samples: 26721598. Policy #0 lag: (min: 20.0, avg: 45.5, max: 48.0) [2023-10-11 21:08:46,034][70582] Avg episode reward: [(0, '173.320'), (1, '61.400')] [2023-10-11 21:08:46,224][71601] Updated weights for policy 0, policy_version 52220 (0.0008) [2023-10-11 21:08:47,054][71635] Updated weights for policy 1, policy_version 52162 (0.0008) [2023-10-11 21:08:47,422][71635] Updated weights for policy 1, policy_version 52172 (0.0008) [2023-10-11 21:08:47,784][71635] Updated weights for policy 1, policy_version 52182 (0.0007) [2023-10-11 21:08:48,153][71635] Updated weights for policy 1, policy_version 52192 (0.0009) [2023-10-11 21:08:49,848][71601] Updated weights for policy 0, policy_version 52230 (0.0009) [2023-10-11 21:08:50,215][71601] Updated weights for policy 0, policy_version 52240 (0.0007) [2023-10-11 21:08:50,590][71601] Updated weights for policy 0, policy_version 52250 (0.0007) [2023-10-11 21:08:51,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 106954752. Throughput: 0: 1842.7, 1: 1820.7. Samples: 26744090. Policy #0 lag: (min: 20.0, avg: 45.5, max: 48.0) [2023-10-11 21:08:51,035][70582] Avg episode reward: [(0, '172.550'), (1, '75.720')] [2023-10-11 21:08:51,801][71635] Updated weights for policy 1, policy_version 52202 (0.0009) [2023-10-11 21:08:52,171][71635] Updated weights for policy 1, policy_version 52212 (0.0008) [2023-10-11 21:08:52,541][71635] Updated weights for policy 1, policy_version 52222 (0.0009) [2023-10-11 21:08:54,042][71601] Updated weights for policy 0, policy_version 52260 (0.0008) [2023-10-11 21:08:54,411][71601] Updated weights for policy 0, policy_version 52270 (0.0010) [2023-10-11 21:08:54,778][71601] Updated weights for policy 0, policy_version 52280 (0.0010) [2023-10-11 21:08:56,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 107020288. Throughput: 0: 1842.6, 1: 1824.9. Samples: 26766038. Policy #0 lag: (min: 20.0, avg: 45.5, max: 48.0) [2023-10-11 21:08:56,034][70582] Avg episode reward: [(0, '184.050'), (1, '73.110')] [2023-10-11 21:08:56,108][71635] Updated weights for policy 1, policy_version 52232 (0.0007) [2023-10-11 21:08:56,483][71635] Updated weights for policy 1, policy_version 52242 (0.0008) [2023-10-11 21:08:56,843][71635] Updated weights for policy 1, policy_version 52252 (0.0008) [2023-10-11 21:08:58,552][71601] Updated weights for policy 0, policy_version 52290 (0.0009) [2023-10-11 21:08:58,926][71601] Updated weights for policy 0, policy_version 52300 (0.0009) [2023-10-11 21:08:59,295][71601] Updated weights for policy 0, policy_version 52310 (0.0007) [2023-10-11 21:08:59,665][71601] Updated weights for policy 0, policy_version 52320 (0.0007) [2023-10-11 21:09:00,542][71635] Updated weights for policy 1, policy_version 52262 (0.0007) [2023-10-11 21:09:00,913][71635] Updated weights for policy 1, policy_version 52272 (0.0010) [2023-10-11 21:09:01,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 107085824. Throughput: 0: 1843.9, 1: 1818.9. Samples: 26777324. Policy #0 lag: (min: 20.0, avg: 45.5, max: 48.0) [2023-10-11 21:09:01,035][70582] Avg episode reward: [(0, '190.460'), (1, '72.160')] [2023-10-11 21:09:01,287][71635] Updated weights for policy 1, policy_version 52282 (0.0007) [2023-10-11 21:09:03,402][71601] Updated weights for policy 0, policy_version 52330 (0.0007) [2023-10-11 21:09:03,775][71601] Updated weights for policy 0, policy_version 52340 (0.0007) [2023-10-11 21:09:04,148][71601] Updated weights for policy 0, policy_version 52350 (0.0008) [2023-10-11 21:09:04,988][71635] Updated weights for policy 1, policy_version 52292 (0.0009) [2023-10-11 21:09:05,353][71635] Updated weights for policy 1, policy_version 52302 (0.0008) [2023-10-11 21:09:05,716][71635] Updated weights for policy 1, policy_version 52312 (0.0007) [2023-10-11 21:09:06,034][70582] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 107184128. Throughput: 0: 1839.5, 1: 1821.7. Samples: 26798838. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:09:06,034][70582] Avg episode reward: [(0, '203.490'), (1, '77.750')] [2023-10-11 21:09:07,634][71601] Updated weights for policy 0, policy_version 52360 (0.0010) [2023-10-11 21:09:08,002][71601] Updated weights for policy 0, policy_version 52370 (0.0009) [2023-10-11 21:09:08,377][71601] Updated weights for policy 0, policy_version 52380 (0.0007) [2023-10-11 21:09:09,367][71635] Updated weights for policy 1, policy_version 52322 (0.0007) [2023-10-11 21:09:09,727][71635] Updated weights for policy 1, policy_version 52332 (0.0008) [2023-10-11 21:09:10,102][71635] Updated weights for policy 1, policy_version 52342 (0.0007) [2023-10-11 21:09:10,466][71635] Updated weights for policy 1, policy_version 52352 (0.0011) [2023-10-11 21:09:11,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 107249664. Throughput: 0: 1849.6, 1: 1816.4. Samples: 26820634. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:09:11,035][70582] Avg episode reward: [(0, '215.330'), (1, '75.720')] [2023-10-11 21:09:12,061][71601] Updated weights for policy 0, policy_version 52390 (0.0010) [2023-10-11 21:09:12,448][71601] Updated weights for policy 0, policy_version 52400 (0.0008) [2023-10-11 21:09:12,820][71601] Updated weights for policy 0, policy_version 52410 (0.0008) [2023-10-11 21:09:14,222][71635] Updated weights for policy 1, policy_version 52362 (0.0009) [2023-10-11 21:09:14,594][71635] Updated weights for policy 1, policy_version 52372 (0.0009) [2023-10-11 21:09:14,961][71635] Updated weights for policy 1, policy_version 52382 (0.0010) [2023-10-11 21:09:16,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 107315200. Throughput: 0: 1841.1, 1: 1816.0. Samples: 26831826. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:09:16,035][70582] Avg episode reward: [(0, '219.570'), (1, '66.230')] [2023-10-11 21:09:16,427][71601] Updated weights for policy 0, policy_version 52420 (0.0010) [2023-10-11 21:09:16,800][71601] Updated weights for policy 0, policy_version 52430 (0.0011) [2023-10-11 21:09:17,169][71601] Updated weights for policy 0, policy_version 52440 (0.0010) [2023-10-11 21:09:18,606][71635] Updated weights for policy 1, policy_version 52392 (0.0008) [2023-10-11 21:09:18,967][71635] Updated weights for policy 1, policy_version 52402 (0.0007) [2023-10-11 21:09:19,327][71635] Updated weights for policy 1, policy_version 52412 (0.0010) [2023-10-11 21:09:20,916][71601] Updated weights for policy 0, policy_version 52450 (0.0009) [2023-10-11 21:09:21,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 107380736. Throughput: 0: 1844.2, 1: 1822.9. Samples: 26853430. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:09:21,034][70582] Avg episode reward: [(0, '219.570'), (1, '44.260')] [2023-10-11 21:09:21,284][71601] Updated weights for policy 0, policy_version 52460 (0.0009) [2023-10-11 21:09:21,657][71601] Updated weights for policy 0, policy_version 52470 (0.0008) [2023-10-11 21:09:22,031][71601] Updated weights for policy 0, policy_version 52480 (0.0008) [2023-10-11 21:09:23,087][71635] Updated weights for policy 1, policy_version 52422 (0.0008) [2023-10-11 21:09:23,459][71635] Updated weights for policy 1, policy_version 52432 (0.0008) [2023-10-11 21:09:23,828][71635] Updated weights for policy 1, policy_version 52442 (0.0007) [2023-10-11 21:09:25,726][71601] Updated weights for policy 0, policy_version 52490 (0.0009) [2023-10-11 21:09:26,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 107446272. Throughput: 0: 1832.8, 1: 1827.2. Samples: 26875814. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:09:26,034][70582] Avg episode reward: [(0, '219.560'), (1, '39.200')] [2023-10-11 21:09:26,097][71601] Updated weights for policy 0, policy_version 52500 (0.0009) [2023-10-11 21:09:26,462][71601] Updated weights for policy 0, policy_version 52510 (0.0007) [2023-10-11 21:09:27,565][71635] Updated weights for policy 1, policy_version 52452 (0.0010) [2023-10-11 21:09:27,939][71635] Updated weights for policy 1, policy_version 52462 (0.0008) [2023-10-11 21:09:28,306][71635] Updated weights for policy 1, policy_version 52472 (0.0009) [2023-10-11 21:09:30,080][71601] Updated weights for policy 0, policy_version 52520 (0.0008) [2023-10-11 21:09:30,455][71601] Updated weights for policy 0, policy_version 52530 (0.0008) [2023-10-11 21:09:30,835][71601] Updated weights for policy 0, policy_version 52540 (0.0007) [2023-10-11 21:09:31,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 107544576. Throughput: 0: 1835.4, 1: 1825.9. Samples: 26886358. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:09:31,035][70582] Avg episode reward: [(0, '219.760'), (1, '39.600')] [2023-10-11 21:09:32,172][71635] Updated weights for policy 1, policy_version 52482 (0.0008) [2023-10-11 21:09:32,534][71635] Updated weights for policy 1, policy_version 52492 (0.0009) [2023-10-11 21:09:32,903][71635] Updated weights for policy 1, policy_version 52502 (0.0007) [2023-10-11 21:09:33,267][71635] Updated weights for policy 1, policy_version 52512 (0.0007) [2023-10-11 21:09:34,501][71601] Updated weights for policy 0, policy_version 52550 (0.0008) [2023-10-11 21:09:34,878][71601] Updated weights for policy 0, policy_version 52560 (0.0010) [2023-10-11 21:09:35,248][71601] Updated weights for policy 0, policy_version 52570 (0.0010) [2023-10-11 21:09:36,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 107610112. Throughput: 0: 1831.2, 1: 1821.2. Samples: 26908444. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:09:36,034][70582] Avg episode reward: [(0, '219.760'), (1, '42.780')] [2023-10-11 21:09:36,875][71635] Updated weights for policy 1, policy_version 52522 (0.0011) [2023-10-11 21:09:37,248][71635] Updated weights for policy 1, policy_version 52532 (0.0010) [2023-10-11 21:09:37,614][71635] Updated weights for policy 1, policy_version 52542 (0.0007) [2023-10-11 21:09:39,029][71601] Updated weights for policy 0, policy_version 52580 (0.0009) [2023-10-11 21:09:39,394][71601] Updated weights for policy 0, policy_version 52590 (0.0011) [2023-10-11 21:09:39,773][71601] Updated weights for policy 0, policy_version 52600 (0.0007) [2023-10-11 21:09:41,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 107675648. Throughput: 0: 1827.1, 1: 1820.6. Samples: 26930184. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-11 21:09:41,035][70582] Avg episode reward: [(0, '219.770'), (1, '43.770')] [2023-10-11 21:09:41,347][71635] Updated weights for policy 1, policy_version 52552 (0.0007) [2023-10-11 21:09:41,708][71635] Updated weights for policy 1, policy_version 52562 (0.0007) [2023-10-11 21:09:42,081][71635] Updated weights for policy 1, policy_version 52572 (0.0007) [2023-10-11 21:09:43,437][71601] Updated weights for policy 0, policy_version 52610 (0.0010) [2023-10-11 21:09:43,809][71601] Updated weights for policy 0, policy_version 52620 (0.0008) [2023-10-11 21:09:44,173][71601] Updated weights for policy 0, policy_version 52630 (0.0009) [2023-10-11 21:09:44,547][71601] Updated weights for policy 0, policy_version 52640 (0.0010) [2023-10-11 21:09:45,729][71635] Updated weights for policy 1, policy_version 52582 (0.0010) [2023-10-11 21:09:46,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 107741184. Throughput: 0: 1825.5, 1: 1823.3. Samples: 26941516. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-11 21:09:46,034][70582] Avg episode reward: [(0, '219.770'), (1, '44.460')] [2023-10-11 21:09:46,094][71635] Updated weights for policy 1, policy_version 52592 (0.0010) [2023-10-11 21:09:46,456][71635] Updated weights for policy 1, policy_version 52602 (0.0010) [2023-10-11 21:09:48,042][71601] Updated weights for policy 0, policy_version 52650 (0.0011) [2023-10-11 21:09:48,416][71601] Updated weights for policy 0, policy_version 52660 (0.0010) [2023-10-11 21:09:48,772][71601] Updated weights for policy 0, policy_version 52670 (0.0010) [2023-10-11 21:09:49,929][71635] Updated weights for policy 1, policy_version 52612 (0.0009) [2023-10-11 21:09:50,292][71635] Updated weights for policy 1, policy_version 52622 (0.0009) [2023-10-11 21:09:50,658][71635] Updated weights for policy 1, policy_version 52632 (0.0010) [2023-10-11 21:09:51,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 107839488. Throughput: 0: 1824.4, 1: 1822.7. Samples: 26962956. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-11 21:09:51,034][70582] Avg episode reward: [(0, '224.450'), (1, '43.470')] [2023-10-11 21:09:52,516][71601] Updated weights for policy 0, policy_version 52680 (0.0010) [2023-10-11 21:09:52,881][71601] Updated weights for policy 0, policy_version 52690 (0.0007) [2023-10-11 21:09:53,262][71601] Updated weights for policy 0, policy_version 52700 (0.0007) [2023-10-11 21:09:54,394][71635] Updated weights for policy 1, policy_version 52642 (0.0009) [2023-10-11 21:09:54,764][71635] Updated weights for policy 1, policy_version 52652 (0.0009) [2023-10-11 21:09:55,128][71635] Updated weights for policy 1, policy_version 52662 (0.0010) [2023-10-11 21:09:55,486][71635] Updated weights for policy 1, policy_version 52672 (0.0010) [2023-10-11 21:09:56,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 107905024. Throughput: 0: 1815.3, 1: 1823.7. Samples: 26984388. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-11 21:09:56,034][70582] Avg episode reward: [(0, '224.340'), (1, '48.640')] [2023-10-11 21:09:56,044][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000052704_53968896.pth... [2023-10-11 21:09:56,044][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000052672_53936128.pth... [2023-10-11 21:09:56,074][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000051008_52232192.pth [2023-10-11 21:09:56,082][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000050976_52199424.pth [2023-10-11 21:09:57,010][71601] Updated weights for policy 0, policy_version 52710 (0.0007) [2023-10-11 21:09:57,391][71601] Updated weights for policy 0, policy_version 52720 (0.0007) [2023-10-11 21:09:57,754][71601] Updated weights for policy 0, policy_version 52730 (0.0007) [2023-10-11 21:09:59,448][71635] Updated weights for policy 1, policy_version 52682 (0.0011) [2023-10-11 21:09:59,825][71635] Updated weights for policy 1, policy_version 52692 (0.0009) [2023-10-11 21:10:00,180][71635] Updated weights for policy 1, policy_version 52702 (0.0009) [2023-10-11 21:10:01,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 107970560. Throughput: 0: 1818.4, 1: 1819.5. Samples: 26995530. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-11 21:10:01,034][70582] Avg episode reward: [(0, '221.750'), (1, '48.840')] [2023-10-11 21:10:01,511][71601] Updated weights for policy 0, policy_version 52740 (0.0008) [2023-10-11 21:10:01,888][71601] Updated weights for policy 0, policy_version 52750 (0.0009) [2023-10-11 21:10:02,267][71601] Updated weights for policy 0, policy_version 52760 (0.0008) [2023-10-11 21:10:03,700][71635] Updated weights for policy 1, policy_version 52712 (0.0009) [2023-10-11 21:10:04,070][71635] Updated weights for policy 1, policy_version 52722 (0.0008) [2023-10-11 21:10:04,438][71635] Updated weights for policy 1, policy_version 52732 (0.0008) [2023-10-11 21:10:06,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 108036096. Throughput: 0: 1815.8, 1: 1823.8. Samples: 27017212. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-11 21:10:06,035][70582] Avg episode reward: [(0, '223.100'), (1, '54.590')] [2023-10-11 21:10:06,145][71601] Updated weights for policy 0, policy_version 52770 (0.0009) [2023-10-11 21:10:06,511][71601] Updated weights for policy 0, policy_version 52780 (0.0008) [2023-10-11 21:10:06,878][71601] Updated weights for policy 0, policy_version 52790 (0.0009) [2023-10-11 21:10:07,245][71601] Updated weights for policy 0, policy_version 52800 (0.0008) [2023-10-11 21:10:08,022][71635] Updated weights for policy 1, policy_version 52742 (0.0008) [2023-10-11 21:10:08,383][71635] Updated weights for policy 1, policy_version 52752 (0.0008) [2023-10-11 21:10:08,749][71635] Updated weights for policy 1, policy_version 52762 (0.0011) [2023-10-11 21:10:10,972][71601] Updated weights for policy 0, policy_version 52810 (0.0008) [2023-10-11 21:10:11,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 108101632. Throughput: 0: 1821.3, 1: 1825.6. Samples: 27039926. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-11 21:10:11,034][70582] Avg episode reward: [(0, '223.100'), (1, '58.160')] [2023-10-11 21:10:11,347][71601] Updated weights for policy 0, policy_version 52820 (0.0009) [2023-10-11 21:10:11,709][71601] Updated weights for policy 0, policy_version 52830 (0.0007) [2023-10-11 21:10:12,427][71635] Updated weights for policy 1, policy_version 52772 (0.0009) [2023-10-11 21:10:12,792][71635] Updated weights for policy 1, policy_version 52782 (0.0009) [2023-10-11 21:10:13,162][71635] Updated weights for policy 1, policy_version 52792 (0.0008) [2023-10-11 21:10:15,325][71601] Updated weights for policy 0, policy_version 52840 (0.0008) [2023-10-11 21:10:15,705][71601] Updated weights for policy 0, policy_version 52850 (0.0008) [2023-10-11 21:10:16,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 108167168. Throughput: 0: 1813.9, 1: 1823.4. Samples: 27050036. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-11 21:10:16,034][70582] Avg episode reward: [(0, '224.000'), (1, '57.640')] [2023-10-11 21:10:16,068][71601] Updated weights for policy 0, policy_version 52860 (0.0008) [2023-10-11 21:10:16,681][71635] Updated weights for policy 1, policy_version 52802 (0.0008) [2023-10-11 21:10:17,049][71635] Updated weights for policy 1, policy_version 52812 (0.0010) [2023-10-11 21:10:17,415][71635] Updated weights for policy 1, policy_version 52822 (0.0008) [2023-10-11 21:10:17,779][71635] Updated weights for policy 1, policy_version 52832 (0.0008) [2023-10-11 21:10:19,741][71601] Updated weights for policy 0, policy_version 52870 (0.0009) [2023-10-11 21:10:20,110][71601] Updated weights for policy 0, policy_version 52880 (0.0009) [2023-10-11 21:10:20,482][71601] Updated weights for policy 0, policy_version 52890 (0.0010) [2023-10-11 21:10:21,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 108265472. Throughput: 0: 1812.4, 1: 1836.5. Samples: 27072648. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-11 21:10:21,035][70582] Avg episode reward: [(0, '229.230'), (1, '61.560')] [2023-10-11 21:10:21,523][71635] Updated weights for policy 1, policy_version 52842 (0.0008) [2023-10-11 21:10:21,886][71635] Updated weights for policy 1, policy_version 52852 (0.0007) [2023-10-11 21:10:22,246][71635] Updated weights for policy 1, policy_version 52862 (0.0008) [2023-10-11 21:10:23,954][71601] Updated weights for policy 0, policy_version 52900 (0.0008) [2023-10-11 21:10:24,330][71601] Updated weights for policy 0, policy_version 52910 (0.0007) [2023-10-11 21:10:24,696][71601] Updated weights for policy 0, policy_version 52920 (0.0007) [2023-10-11 21:10:26,019][71635] Updated weights for policy 1, policy_version 52872 (0.0009) [2023-10-11 21:10:26,034][70582] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 108331008. Throughput: 0: 1811.6, 1: 1828.1. Samples: 27093972. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-11 21:10:26,035][70582] Avg episode reward: [(0, '229.950'), (1, '62.490')] [2023-10-11 21:10:26,382][71635] Updated weights for policy 1, policy_version 52882 (0.0008) [2023-10-11 21:10:26,755][71635] Updated weights for policy 1, policy_version 52892 (0.0010) [2023-10-11 21:10:28,423][71601] Updated weights for policy 0, policy_version 52930 (0.0008) [2023-10-11 21:10:28,794][71601] Updated weights for policy 0, policy_version 52940 (0.0008) [2023-10-11 21:10:29,155][71601] Updated weights for policy 0, policy_version 52950 (0.0009) [2023-10-11 21:10:29,529][71601] Updated weights for policy 0, policy_version 52960 (0.0010) [2023-10-11 21:10:30,416][71635] Updated weights for policy 1, policy_version 52902 (0.0009) [2023-10-11 21:10:30,789][71635] Updated weights for policy 1, policy_version 52912 (0.0009) [2023-10-11 21:10:31,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 108396544. Throughput: 0: 1814.2, 1: 1828.0. Samples: 27105416. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-11 21:10:31,035][70582] Avg episode reward: [(0, '233.780'), (1, '60.630')] [2023-10-11 21:10:31,149][71635] Updated weights for policy 1, policy_version 52922 (0.0007) [2023-10-11 21:10:33,322][71601] Updated weights for policy 0, policy_version 52970 (0.0008) [2023-10-11 21:10:33,690][71601] Updated weights for policy 0, policy_version 52980 (0.0008) [2023-10-11 21:10:34,051][71601] Updated weights for policy 0, policy_version 52990 (0.0008) [2023-10-11 21:10:34,813][71635] Updated weights for policy 1, policy_version 52932 (0.0008) [2023-10-11 21:10:35,181][71635] Updated weights for policy 1, policy_version 52942 (0.0011) [2023-10-11 21:10:35,547][71635] Updated weights for policy 1, policy_version 52952 (0.0008) [2023-10-11 21:10:36,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 108494848. Throughput: 0: 1813.9, 1: 1828.2. Samples: 27126850. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-11 21:10:36,035][70582] Avg episode reward: [(0, '236.210'), (1, '56.360')] [2023-10-11 21:10:37,649][71601] Updated weights for policy 0, policy_version 53000 (0.0009) [2023-10-11 21:10:38,021][71601] Updated weights for policy 0, policy_version 53010 (0.0007) [2023-10-11 21:10:38,400][71601] Updated weights for policy 0, policy_version 53020 (0.0007) [2023-10-11 21:10:39,077][71635] Updated weights for policy 1, policy_version 52962 (0.0008) [2023-10-11 21:10:39,444][71635] Updated weights for policy 1, policy_version 52972 (0.0010) [2023-10-11 21:10:39,810][71635] Updated weights for policy 1, policy_version 52982 (0.0008) [2023-10-11 21:10:40,182][71635] Updated weights for policy 1, policy_version 52992 (0.0008) [2023-10-11 21:10:41,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 108560384. Throughput: 0: 1813.2, 1: 1825.7. Samples: 27148140. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-11 21:10:41,034][70582] Avg episode reward: [(0, '237.350'), (1, '56.930')] [2023-10-11 21:10:42,227][71601] Updated weights for policy 0, policy_version 53030 (0.0010) [2023-10-11 21:10:42,599][71601] Updated weights for policy 0, policy_version 53040 (0.0010) [2023-10-11 21:10:42,968][71601] Updated weights for policy 0, policy_version 53050 (0.0010) [2023-10-11 21:10:43,890][71635] Updated weights for policy 1, policy_version 53002 (0.0010) [2023-10-11 21:10:44,258][71635] Updated weights for policy 1, policy_version 53012 (0.0008) [2023-10-11 21:10:44,623][71635] Updated weights for policy 1, policy_version 53022 (0.0007) [2023-10-11 21:10:46,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 108625920. Throughput: 0: 1810.6, 1: 1833.5. Samples: 27159518. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-11 21:10:46,035][70582] Avg episode reward: [(0, '228.430'), (1, '53.260')] [2023-10-11 21:10:46,660][71601] Updated weights for policy 0, policy_version 53060 (0.0009) [2023-10-11 21:10:47,039][71601] Updated weights for policy 0, policy_version 53070 (0.0009) [2023-10-11 21:10:47,415][71601] Updated weights for policy 0, policy_version 53080 (0.0011) [2023-10-11 21:10:48,367][71635] Updated weights for policy 1, policy_version 53032 (0.0008) [2023-10-11 21:10:48,736][71635] Updated weights for policy 1, policy_version 53042 (0.0008) [2023-10-11 21:10:49,097][71635] Updated weights for policy 1, policy_version 53052 (0.0008) [2023-10-11 21:10:51,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 108691456. Throughput: 0: 1811.3, 1: 1820.0. Samples: 27180624. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-11 21:10:51,034][70582] Avg episode reward: [(0, '223.600'), (1, '53.780')] [2023-10-11 21:10:51,127][71601] Updated weights for policy 0, policy_version 53090 (0.0010) [2023-10-11 21:10:51,500][71601] Updated weights for policy 0, policy_version 53100 (0.0007) [2023-10-11 21:10:51,877][71601] Updated weights for policy 0, policy_version 53110 (0.0007) [2023-10-11 21:10:52,245][71601] Updated weights for policy 0, policy_version 53120 (0.0007) [2023-10-11 21:10:52,846][71635] Updated weights for policy 1, policy_version 53062 (0.0008) [2023-10-11 21:10:53,216][71635] Updated weights for policy 1, policy_version 53072 (0.0007) [2023-10-11 21:10:53,586][71635] Updated weights for policy 1, policy_version 53082 (0.0007) [2023-10-11 21:10:55,895][71601] Updated weights for policy 0, policy_version 53130 (0.0008) [2023-10-11 21:10:56,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 108756992. Throughput: 0: 1812.7, 1: 1822.0. Samples: 27203484. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:10:56,034][70582] Avg episode reward: [(0, '236.410'), (1, '54.530')] [2023-10-11 21:10:56,264][71601] Updated weights for policy 0, policy_version 53140 (0.0008) [2023-10-11 21:10:56,633][71601] Updated weights for policy 0, policy_version 53150 (0.0007) [2023-10-11 21:10:57,237][71635] Updated weights for policy 1, policy_version 53092 (0.0008) [2023-10-11 21:10:57,598][71635] Updated weights for policy 1, policy_version 53102 (0.0010) [2023-10-11 21:10:57,962][71635] Updated weights for policy 1, policy_version 53112 (0.0011) [2023-10-11 21:11:00,292][71601] Updated weights for policy 0, policy_version 53160 (0.0008) [2023-10-11 21:11:00,660][71601] Updated weights for policy 0, policy_version 53170 (0.0009) [2023-10-11 21:11:01,030][71601] Updated weights for policy 0, policy_version 53180 (0.0010) [2023-10-11 21:11:01,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 108822528. Throughput: 0: 1818.3, 1: 1816.4. Samples: 27213596. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:11:01,034][70582] Avg episode reward: [(0, '238.810'), (1, '49.430')] [2023-10-11 21:11:01,794][71635] Updated weights for policy 1, policy_version 53122 (0.0010) [2023-10-11 21:11:02,161][71635] Updated weights for policy 1, policy_version 53132 (0.0009) [2023-10-11 21:11:02,521][71635] Updated weights for policy 1, policy_version 53142 (0.0009) [2023-10-11 21:11:02,880][71635] Updated weights for policy 1, policy_version 53152 (0.0009) [2023-10-11 21:11:04,699][71601] Updated weights for policy 0, policy_version 53190 (0.0007) [2023-10-11 21:11:05,064][71601] Updated weights for policy 0, policy_version 53200 (0.0010) [2023-10-11 21:11:05,435][71601] Updated weights for policy 0, policy_version 53210 (0.0009) [2023-10-11 21:11:06,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 108920832. Throughput: 0: 1819.7, 1: 1812.9. Samples: 27236118. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:11:06,035][70582] Avg episode reward: [(0, '238.880'), (1, '49.350')] [2023-10-11 21:11:06,681][71635] Updated weights for policy 1, policy_version 53162 (0.0009) [2023-10-11 21:11:07,050][71635] Updated weights for policy 1, policy_version 53172 (0.0007) [2023-10-11 21:11:07,416][71635] Updated weights for policy 1, policy_version 53182 (0.0007) [2023-10-11 21:11:09,073][71601] Updated weights for policy 0, policy_version 53220 (0.0009) [2023-10-11 21:11:09,451][71601] Updated weights for policy 0, policy_version 53230 (0.0009) [2023-10-11 21:11:09,824][71601] Updated weights for policy 0, policy_version 53240 (0.0008) [2023-10-11 21:11:11,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 108986368. Throughput: 0: 1817.1, 1: 1811.9. Samples: 27257278. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:11:11,034][70582] Avg episode reward: [(0, '233.190'), (1, '50.270')] [2023-10-11 21:11:11,161][71635] Updated weights for policy 1, policy_version 53192 (0.0007) [2023-10-11 21:11:11,521][71635] Updated weights for policy 1, policy_version 53202 (0.0008) [2023-10-11 21:11:11,894][71635] Updated weights for policy 1, policy_version 53212 (0.0008) [2023-10-11 21:11:13,530][71601] Updated weights for policy 0, policy_version 53250 (0.0009) [2023-10-11 21:11:13,887][71601] Updated weights for policy 0, policy_version 53260 (0.0008) [2023-10-11 21:11:14,267][71601] Updated weights for policy 0, policy_version 53270 (0.0008) [2023-10-11 21:11:14,643][71601] Updated weights for policy 0, policy_version 53280 (0.0008) [2023-10-11 21:11:15,533][71635] Updated weights for policy 1, policy_version 53222 (0.0009) [2023-10-11 21:11:15,904][71635] Updated weights for policy 1, policy_version 53232 (0.0007) [2023-10-11 21:11:16,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 109051904. Throughput: 0: 1818.4, 1: 1810.5. Samples: 27268718. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:11:16,034][70582] Avg episode reward: [(0, '232.590'), (1, '49.910')] [2023-10-11 21:11:16,276][71635] Updated weights for policy 1, policy_version 53242 (0.0008) [2023-10-11 21:11:18,290][71601] Updated weights for policy 0, policy_version 53290 (0.0008) [2023-10-11 21:11:18,660][71601] Updated weights for policy 0, policy_version 53300 (0.0008) [2023-10-11 21:11:19,035][71601] Updated weights for policy 0, policy_version 53310 (0.0009) [2023-10-11 21:11:19,997][71635] Updated weights for policy 1, policy_version 53252 (0.0008) [2023-10-11 21:11:20,358][71635] Updated weights for policy 1, policy_version 53262 (0.0008) [2023-10-11 21:11:20,731][71635] Updated weights for policy 1, policy_version 53272 (0.0008) [2023-10-11 21:11:21,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 109150208. Throughput: 0: 1823.2, 1: 1809.5. Samples: 27290318. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:11:21,034][70582] Avg episode reward: [(0, '219.510'), (1, '50.810')] [2023-10-11 21:11:22,656][71601] Updated weights for policy 0, policy_version 53320 (0.0008) [2023-10-11 21:11:23,029][71601] Updated weights for policy 0, policy_version 53330 (0.0007) [2023-10-11 21:11:23,392][71601] Updated weights for policy 0, policy_version 53340 (0.0008) [2023-10-11 21:11:24,363][71635] Updated weights for policy 1, policy_version 53282 (0.0008) [2023-10-11 21:11:24,734][71635] Updated weights for policy 1, policy_version 53292 (0.0010) [2023-10-11 21:11:25,109][71635] Updated weights for policy 1, policy_version 53302 (0.0007) [2023-10-11 21:11:25,477][71635] Updated weights for policy 1, policy_version 53312 (0.0009) [2023-10-11 21:11:26,034][70582] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 109215744. Throughput: 0: 1825.1, 1: 1814.2. Samples: 27311912. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:11:26,035][70582] Avg episode reward: [(0, '243.010'), (1, '54.080')] [2023-10-11 21:11:27,209][71601] Updated weights for policy 0, policy_version 53350 (0.0008) [2023-10-11 21:11:27,592][71601] Updated weights for policy 0, policy_version 53360 (0.0008) [2023-10-11 21:11:27,964][71601] Updated weights for policy 0, policy_version 53370 (0.0007) [2023-10-11 21:11:29,223][71635] Updated weights for policy 1, policy_version 53322 (0.0008) [2023-10-11 21:11:29,588][71635] Updated weights for policy 1, policy_version 53332 (0.0008) [2023-10-11 21:11:29,944][71635] Updated weights for policy 1, policy_version 53342 (0.0007) [2023-10-11 21:11:31,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 109281280. Throughput: 0: 1821.4, 1: 1807.9. Samples: 27322838. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:11:31,034][70582] Avg episode reward: [(0, '228.370'), (1, '44.950')] [2023-10-11 21:11:31,597][71601] Updated weights for policy 0, policy_version 53380 (0.0008) [2023-10-11 21:11:31,962][71601] Updated weights for policy 0, policy_version 53390 (0.0008) [2023-10-11 21:11:32,333][71601] Updated weights for policy 0, policy_version 53400 (0.0007) [2023-10-11 21:11:33,689][71635] Updated weights for policy 1, policy_version 53352 (0.0008) [2023-10-11 21:11:34,056][71635] Updated weights for policy 1, policy_version 53362 (0.0009) [2023-10-11 21:11:34,428][71635] Updated weights for policy 1, policy_version 53372 (0.0010) [2023-10-11 21:11:36,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 109346816. Throughput: 0: 1820.1, 1: 1818.9. Samples: 27344378. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:11:36,035][70582] Avg episode reward: [(0, '237.240'), (1, '45.990')] [2023-10-11 21:11:36,047][71601] Updated weights for policy 0, policy_version 53410 (0.0007) [2023-10-11 21:11:36,422][71601] Updated weights for policy 0, policy_version 53420 (0.0008) [2023-10-11 21:11:36,794][71601] Updated weights for policy 0, policy_version 53430 (0.0008) [2023-10-11 21:11:37,163][71601] Updated weights for policy 0, policy_version 53440 (0.0008) [2023-10-11 21:11:38,211][71635] Updated weights for policy 1, policy_version 53382 (0.0009) [2023-10-11 21:11:38,585][71635] Updated weights for policy 1, policy_version 53392 (0.0009) [2023-10-11 21:11:38,954][71635] Updated weights for policy 1, policy_version 53402 (0.0008) [2023-10-11 21:11:40,928][71601] Updated weights for policy 0, policy_version 53450 (0.0009) [2023-10-11 21:11:41,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 109412352. Throughput: 0: 1820.0, 1: 1807.9. Samples: 27366740. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:11:41,034][70582] Avg episode reward: [(0, '227.040'), (1, '43.510')] [2023-10-11 21:11:41,296][71601] Updated weights for policy 0, policy_version 53460 (0.0007) [2023-10-11 21:11:41,674][71601] Updated weights for policy 0, policy_version 53470 (0.0008) [2023-10-11 21:11:42,569][71635] Updated weights for policy 1, policy_version 53412 (0.0007) [2023-10-11 21:11:42,931][71635] Updated weights for policy 1, policy_version 53422 (0.0007) [2023-10-11 21:11:43,297][71635] Updated weights for policy 1, policy_version 53432 (0.0007) [2023-10-11 21:11:45,363][71601] Updated weights for policy 0, policy_version 53480 (0.0008) [2023-10-11 21:11:45,732][71601] Updated weights for policy 0, policy_version 53490 (0.0008) [2023-10-11 21:11:46,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 109477888. Throughput: 0: 1813.4, 1: 1816.8. Samples: 27376954. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:11:46,034][70582] Avg episode reward: [(0, '218.230'), (1, '51.320')] [2023-10-11 21:11:46,109][71601] Updated weights for policy 0, policy_version 53500 (0.0007) [2023-10-11 21:11:46,889][71635] Updated weights for policy 1, policy_version 53442 (0.0008) [2023-10-11 21:11:47,260][71635] Updated weights for policy 1, policy_version 53452 (0.0009) [2023-10-11 21:11:47,633][71635] Updated weights for policy 1, policy_version 53462 (0.0010) [2023-10-11 21:11:48,002][71635] Updated weights for policy 1, policy_version 53472 (0.0008) [2023-10-11 21:11:49,780][71601] Updated weights for policy 0, policy_version 53510 (0.0009) [2023-10-11 21:11:50,150][71601] Updated weights for policy 0, policy_version 53520 (0.0010) [2023-10-11 21:11:50,523][71601] Updated weights for policy 0, policy_version 53530 (0.0009) [2023-10-11 21:11:51,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 109576192. Throughput: 0: 1815.8, 1: 1812.4. Samples: 27399388. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:11:51,035][70582] Avg episode reward: [(0, '222.860'), (1, '51.650')] [2023-10-11 21:11:51,777][71635] Updated weights for policy 1, policy_version 53482 (0.0007) [2023-10-11 21:11:52,147][71635] Updated weights for policy 1, policy_version 53492 (0.0007) [2023-10-11 21:11:52,514][71635] Updated weights for policy 1, policy_version 53502 (0.0009) [2023-10-11 21:11:54,362][71601] Updated weights for policy 0, policy_version 53540 (0.0007) [2023-10-11 21:11:54,745][71601] Updated weights for policy 0, policy_version 53550 (0.0009) [2023-10-11 21:11:55,121][71601] Updated weights for policy 0, policy_version 53560 (0.0007) [2023-10-11 21:11:56,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 109641728. Throughput: 0: 1818.2, 1: 1818.9. Samples: 27420948. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:11:56,034][70582] Avg episode reward: [(0, '214.710'), (1, '56.550')] [2023-10-11 21:11:56,043][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000053568_54853632.pth... [2023-10-11 21:11:56,077][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000051872_53116928.pth [2023-10-11 21:11:56,153][71635] Updated weights for policy 1, policy_version 53512 (0.0009) [2023-10-11 21:11:56,518][71635] Updated weights for policy 1, policy_version 53522 (0.0007) [2023-10-11 21:11:56,893][71635] Updated weights for policy 1, policy_version 53532 (0.0010) [2023-10-11 21:11:57,030][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000053536_54820864.pth... [2023-10-11 21:11:57,067][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000051808_53051392.pth [2023-10-11 21:11:58,905][71601] Updated weights for policy 0, policy_version 53570 (0.0008) [2023-10-11 21:11:59,279][71601] Updated weights for policy 0, policy_version 53580 (0.0008) [2023-10-11 21:11:59,653][71601] Updated weights for policy 0, policy_version 53590 (0.0009) [2023-10-11 21:12:00,013][71601] Updated weights for policy 0, policy_version 53600 (0.0007) [2023-10-11 21:12:00,673][71635] Updated weights for policy 1, policy_version 53542 (0.0008) [2023-10-11 21:12:01,025][71635] Updated weights for policy 1, policy_version 53552 (0.0008) [2023-10-11 21:12:01,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 109707264. Throughput: 0: 1813.0, 1: 1819.2. Samples: 27432168. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:12:01,034][70582] Avg episode reward: [(0, '208.510'), (1, '60.930')] [2023-10-11 21:12:01,395][71635] Updated weights for policy 1, policy_version 53562 (0.0009) [2023-10-11 21:12:03,568][71601] Updated weights for policy 0, policy_version 53610 (0.0009) [2023-10-11 21:12:03,938][71601] Updated weights for policy 0, policy_version 53620 (0.0008) [2023-10-11 21:12:04,308][71601] Updated weights for policy 0, policy_version 53630 (0.0008) [2023-10-11 21:12:05,105][71635] Updated weights for policy 1, policy_version 53572 (0.0010) [2023-10-11 21:12:05,477][71635] Updated weights for policy 1, policy_version 53582 (0.0012) [2023-10-11 21:12:05,842][71635] Updated weights for policy 1, policy_version 53592 (0.0010) [2023-10-11 21:12:06,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 109772800. Throughput: 0: 1809.6, 1: 1822.0. Samples: 27453740. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:12:06,034][70582] Avg episode reward: [(0, '194.190'), (1, '55.300')] [2023-10-11 21:12:07,911][71601] Updated weights for policy 0, policy_version 53640 (0.0009) [2023-10-11 21:12:08,283][71601] Updated weights for policy 0, policy_version 53650 (0.0007) [2023-10-11 21:12:08,648][71601] Updated weights for policy 0, policy_version 53660 (0.0007) [2023-10-11 21:12:09,482][71635] Updated weights for policy 1, policy_version 53602 (0.0008) [2023-10-11 21:12:09,863][71635] Updated weights for policy 1, policy_version 53612 (0.0009) [2023-10-11 21:12:10,222][71635] Updated weights for policy 1, policy_version 53622 (0.0008) [2023-10-11 21:12:10,589][71635] Updated weights for policy 1, policy_version 53632 (0.0008) [2023-10-11 21:12:11,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 109871104. Throughput: 0: 1811.2, 1: 1827.7. Samples: 27475664. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:12:11,035][70582] Avg episode reward: [(0, '193.660'), (1, '59.950')] [2023-10-11 21:12:12,295][71601] Updated weights for policy 0, policy_version 53670 (0.0009) [2023-10-11 21:12:12,671][71601] Updated weights for policy 0, policy_version 53680 (0.0008) [2023-10-11 21:12:13,054][71601] Updated weights for policy 0, policy_version 53690 (0.0011) [2023-10-11 21:12:14,055][71635] Updated weights for policy 1, policy_version 53642 (0.0008) [2023-10-11 21:12:14,414][71635] Updated weights for policy 1, policy_version 53652 (0.0007) [2023-10-11 21:12:14,777][71635] Updated weights for policy 1, policy_version 53662 (0.0009) [2023-10-11 21:12:16,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 109936640. Throughput: 0: 1814.6, 1: 1831.9. Samples: 27486930. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:12:16,034][70582] Avg episode reward: [(0, '177.820'), (1, '62.210')] [2023-10-11 21:12:16,654][71601] Updated weights for policy 0, policy_version 53700 (0.0009) [2023-10-11 21:12:17,032][71601] Updated weights for policy 0, policy_version 53710 (0.0011) [2023-10-11 21:12:17,405][71601] Updated weights for policy 0, policy_version 53720 (0.0008) [2023-10-11 21:12:18,385][71635] Updated weights for policy 1, policy_version 53672 (0.0009) [2023-10-11 21:12:18,759][71635] Updated weights for policy 1, policy_version 53682 (0.0008) [2023-10-11 21:12:19,125][71635] Updated weights for policy 1, policy_version 53692 (0.0010) [2023-10-11 21:12:21,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 110002176. Throughput: 0: 1817.3, 1: 1830.8. Samples: 27508542. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:12:21,034][70582] Avg episode reward: [(0, '177.870'), (1, '63.540')] [2023-10-11 21:12:21,182][71601] Updated weights for policy 0, policy_version 53730 (0.0009) [2023-10-11 21:12:21,557][71601] Updated weights for policy 0, policy_version 53740 (0.0007) [2023-10-11 21:12:21,937][71601] Updated weights for policy 0, policy_version 53750 (0.0008) [2023-10-11 21:12:22,304][71601] Updated weights for policy 0, policy_version 53760 (0.0008) [2023-10-11 21:12:22,581][71635] Updated weights for policy 1, policy_version 53702 (0.0008) [2023-10-11 21:12:22,953][71635] Updated weights for policy 1, policy_version 53712 (0.0007) [2023-10-11 21:12:23,329][71635] Updated weights for policy 1, policy_version 53722 (0.0009) [2023-10-11 21:12:26,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 110067712. Throughput: 0: 1811.2, 1: 1842.8. Samples: 27531172. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:12:26,035][70582] Avg episode reward: [(0, '177.550'), (1, '62.130')] [2023-10-11 21:12:26,074][71601] Updated weights for policy 0, policy_version 53770 (0.0007) [2023-10-11 21:12:26,451][71601] Updated weights for policy 0, policy_version 53780 (0.0007) [2023-10-11 21:12:26,812][71601] Updated weights for policy 0, policy_version 53790 (0.0008) [2023-10-11 21:12:27,133][71635] Updated weights for policy 1, policy_version 53732 (0.0008) [2023-10-11 21:12:27,538][71635] Updated weights for policy 1, policy_version 53742 (0.0007) [2023-10-11 21:12:27,898][71635] Updated weights for policy 1, policy_version 53752 (0.0010) [2023-10-11 21:12:30,454][71601] Updated weights for policy 0, policy_version 53800 (0.0007) [2023-10-11 21:12:30,833][71601] Updated weights for policy 0, policy_version 53810 (0.0007) [2023-10-11 21:12:31,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 110133248. Throughput: 0: 1816.9, 1: 1831.6. Samples: 27541138. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:12:31,034][70582] Avg episode reward: [(0, '187.830'), (1, '59.760')] [2023-10-11 21:12:31,201][71601] Updated weights for policy 0, policy_version 53820 (0.0008) [2023-10-11 21:12:31,596][71635] Updated weights for policy 1, policy_version 53762 (0.0009) [2023-10-11 21:12:31,956][71635] Updated weights for policy 1, policy_version 53772 (0.0008) [2023-10-11 21:12:32,320][71635] Updated weights for policy 1, policy_version 53782 (0.0010) [2023-10-11 21:12:32,684][71635] Updated weights for policy 1, policy_version 53792 (0.0008) [2023-10-11 21:12:34,901][71601] Updated weights for policy 0, policy_version 53830 (0.0007) [2023-10-11 21:12:35,275][71601] Updated weights for policy 0, policy_version 53840 (0.0010) [2023-10-11 21:12:35,648][71601] Updated weights for policy 0, policy_version 53850 (0.0010) [2023-10-11 21:12:36,034][70582] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 110231552. Throughput: 0: 1815.5, 1: 1840.1. Samples: 27563888. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:12:36,034][70582] Avg episode reward: [(0, '178.700'), (1, '62.190')] [2023-10-11 21:12:36,498][71635] Updated weights for policy 1, policy_version 53802 (0.0008) [2023-10-11 21:12:36,880][71635] Updated weights for policy 1, policy_version 53812 (0.0009) [2023-10-11 21:12:37,250][71635] Updated weights for policy 1, policy_version 53822 (0.0008) [2023-10-11 21:12:39,404][71601] Updated weights for policy 0, policy_version 53860 (0.0010) [2023-10-11 21:12:39,760][71601] Updated weights for policy 0, policy_version 53870 (0.0010) [2023-10-11 21:12:40,136][71601] Updated weights for policy 0, policy_version 53880 (0.0008) [2023-10-11 21:12:40,873][71635] Updated weights for policy 1, policy_version 53832 (0.0009) [2023-10-11 21:12:41,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 110297088. Throughput: 0: 1811.9, 1: 1835.6. Samples: 27585086. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:12:41,035][70582] Avg episode reward: [(0, '178.700'), (1, '52.950')] [2023-10-11 21:12:41,246][71635] Updated weights for policy 1, policy_version 53842 (0.0008) [2023-10-11 21:12:41,615][71635] Updated weights for policy 1, policy_version 53852 (0.0007) [2023-10-11 21:12:43,943][71601] Updated weights for policy 0, policy_version 53890 (0.0009) [2023-10-11 21:12:44,316][71601] Updated weights for policy 0, policy_version 53900 (0.0009) [2023-10-11 21:12:44,680][71601] Updated weights for policy 0, policy_version 53910 (0.0007) [2023-10-11 21:12:45,049][71601] Updated weights for policy 0, policy_version 53920 (0.0008) [2023-10-11 21:12:45,189][71635] Updated weights for policy 1, policy_version 53862 (0.0008) [2023-10-11 21:12:45,547][71635] Updated weights for policy 1, policy_version 53872 (0.0008) [2023-10-11 21:12:45,922][71635] Updated weights for policy 1, policy_version 53882 (0.0007) [2023-10-11 21:12:46,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 110362624. Throughput: 0: 1812.0, 1: 1836.5. Samples: 27596352. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:12:46,034][70582] Avg episode reward: [(0, '178.520'), (1, '53.250')] [2023-10-11 21:12:48,916][71601] Updated weights for policy 0, policy_version 53930 (0.0010) [2023-10-11 21:12:49,285][71601] Updated weights for policy 0, policy_version 53940 (0.0008) [2023-10-11 21:12:49,654][71601] Updated weights for policy 0, policy_version 53950 (0.0007) [2023-10-11 21:12:49,659][71635] Updated weights for policy 1, policy_version 53892 (0.0008) [2023-10-11 21:12:50,026][71635] Updated weights for policy 1, policy_version 53902 (0.0009) [2023-10-11 21:12:50,388][71635] Updated weights for policy 1, policy_version 53912 (0.0007) [2023-10-11 21:12:51,034][70582] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 110460928. Throughput: 0: 1815.4, 1: 1832.2. Samples: 27617880. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:12:51,034][70582] Avg episode reward: [(0, '178.460'), (1, '51.340')] [2023-10-11 21:12:53,123][71601] Updated weights for policy 0, policy_version 53960 (0.0008) [2023-10-11 21:12:53,486][71601] Updated weights for policy 0, policy_version 53970 (0.0008) [2023-10-11 21:12:53,853][71601] Updated weights for policy 0, policy_version 53980 (0.0008) [2023-10-11 21:12:53,971][71635] Updated weights for policy 1, policy_version 53922 (0.0007) [2023-10-11 21:12:54,331][71635] Updated weights for policy 1, policy_version 53932 (0.0010) [2023-10-11 21:12:54,701][71635] Updated weights for policy 1, policy_version 53942 (0.0007) [2023-10-11 21:12:55,072][71635] Updated weights for policy 1, policy_version 53952 (0.0007) [2023-10-11 21:12:56,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 110526464. Throughput: 0: 1811.1, 1: 1823.1. Samples: 27639202. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:12:56,035][70582] Avg episode reward: [(0, '182.200'), (1, '45.780')] [2023-10-11 21:12:57,505][71601] Updated weights for policy 0, policy_version 53990 (0.0011) [2023-10-11 21:12:57,881][71601] Updated weights for policy 0, policy_version 54000 (0.0009) [2023-10-11 21:12:58,249][71601] Updated weights for policy 0, policy_version 54010 (0.0010) [2023-10-11 21:12:58,832][71635] Updated weights for policy 1, policy_version 53962 (0.0009) [2023-10-11 21:12:59,186][71635] Updated weights for policy 1, policy_version 53972 (0.0011) [2023-10-11 21:12:59,547][71635] Updated weights for policy 1, policy_version 53982 (0.0009) [2023-10-11 21:13:01,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 110592000. Throughput: 0: 1813.6, 1: 1824.4. Samples: 27650638. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:13:01,035][70582] Avg episode reward: [(0, '177.550'), (1, '48.330')] [2023-10-11 21:13:01,950][71601] Updated weights for policy 0, policy_version 54020 (0.0008) [2023-10-11 21:13:02,314][71601] Updated weights for policy 0, policy_version 54030 (0.0009) [2023-10-11 21:13:02,687][71601] Updated weights for policy 0, policy_version 54040 (0.0008) [2023-10-11 21:13:03,289][71635] Updated weights for policy 1, policy_version 53992 (0.0010) [2023-10-11 21:13:03,650][71635] Updated weights for policy 1, policy_version 54002 (0.0007) [2023-10-11 21:13:04,017][71635] Updated weights for policy 1, policy_version 54012 (0.0007) [2023-10-11 21:13:06,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 110657536. Throughput: 0: 1809.5, 1: 1819.1. Samples: 27671830. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:13:06,035][70582] Avg episode reward: [(0, '177.600'), (1, '47.580')] [2023-10-11 21:13:06,420][71601] Updated weights for policy 0, policy_version 54050 (0.0008) [2023-10-11 21:13:06,796][71601] Updated weights for policy 0, policy_version 54060 (0.0010) [2023-10-11 21:13:07,162][71601] Updated weights for policy 0, policy_version 54070 (0.0008) [2023-10-11 21:13:07,534][71601] Updated weights for policy 0, policy_version 54080 (0.0007) [2023-10-11 21:13:07,661][71635] Updated weights for policy 1, policy_version 54022 (0.0010) [2023-10-11 21:13:08,038][71635] Updated weights for policy 1, policy_version 54032 (0.0011) [2023-10-11 21:13:08,395][71635] Updated weights for policy 1, policy_version 54042 (0.0008) [2023-10-11 21:13:11,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 110723072. Throughput: 0: 1822.0, 1: 1822.9. Samples: 27695190. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:13:11,034][70582] Avg episode reward: [(0, '183.550'), (1, '43.890')] [2023-10-11 21:13:11,138][71601] Updated weights for policy 0, policy_version 54090 (0.0008) [2023-10-11 21:13:11,500][71601] Updated weights for policy 0, policy_version 54100 (0.0007) [2023-10-11 21:13:11,881][71601] Updated weights for policy 0, policy_version 54110 (0.0007) [2023-10-11 21:13:12,083][71635] Updated weights for policy 1, policy_version 54052 (0.0008) [2023-10-11 21:13:12,453][71635] Updated weights for policy 1, policy_version 54062 (0.0009) [2023-10-11 21:13:12,821][71635] Updated weights for policy 1, policy_version 54072 (0.0008) [2023-10-11 21:13:15,691][71601] Updated weights for policy 0, policy_version 54120 (0.0007) [2023-10-11 21:13:16,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 110788608. Throughput: 0: 1818.8, 1: 1824.1. Samples: 27705070. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:13:16,034][70582] Avg episode reward: [(0, '183.590'), (1, '40.580')] [2023-10-11 21:13:16,061][71601] Updated weights for policy 0, policy_version 54130 (0.0007) [2023-10-11 21:13:16,432][71601] Updated weights for policy 0, policy_version 54140 (0.0007) [2023-10-11 21:13:16,505][71635] Updated weights for policy 1, policy_version 54082 (0.0010) [2023-10-11 21:13:16,875][71635] Updated weights for policy 1, policy_version 54092 (0.0007) [2023-10-11 21:13:17,245][71635] Updated weights for policy 1, policy_version 54102 (0.0007) [2023-10-11 21:13:17,608][71635] Updated weights for policy 1, policy_version 54112 (0.0009) [2023-10-11 21:13:20,172][71601] Updated weights for policy 0, policy_version 54150 (0.0007) [2023-10-11 21:13:20,543][71601] Updated weights for policy 0, policy_version 54160 (0.0007) [2023-10-11 21:13:20,922][71601] Updated weights for policy 0, policy_version 54170 (0.0007) [2023-10-11 21:13:21,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 110854144. Throughput: 0: 1821.4, 1: 1828.6. Samples: 27728138. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:13:21,034][70582] Avg episode reward: [(0, '182.470'), (1, '35.590')] [2023-10-11 21:13:21,113][71635] Updated weights for policy 1, policy_version 54122 (0.0007) [2023-10-11 21:13:21,478][71635] Updated weights for policy 1, policy_version 54132 (0.0007) [2023-10-11 21:13:21,849][71635] Updated weights for policy 1, policy_version 54142 (0.0010) [2023-10-11 21:13:24,534][71601] Updated weights for policy 0, policy_version 54180 (0.0008) [2023-10-11 21:13:24,897][71601] Updated weights for policy 0, policy_version 54190 (0.0008) [2023-10-11 21:13:25,266][71601] Updated weights for policy 0, policy_version 54200 (0.0008) [2023-10-11 21:13:25,515][71635] Updated weights for policy 1, policy_version 54152 (0.0008) [2023-10-11 21:13:25,888][71635] Updated weights for policy 1, policy_version 54162 (0.0008) [2023-10-11 21:13:26,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 110952448. Throughput: 0: 1831.6, 1: 1824.4. Samples: 27749608. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:13:26,034][70582] Avg episode reward: [(0, '176.150'), (1, '35.710')] [2023-10-11 21:13:26,253][71635] Updated weights for policy 1, policy_version 54172 (0.0010) [2023-10-11 21:13:28,754][71601] Updated weights for policy 0, policy_version 54210 (0.0007) [2023-10-11 21:13:29,122][71601] Updated weights for policy 0, policy_version 54220 (0.0011) [2023-10-11 21:13:29,500][71601] Updated weights for policy 0, policy_version 54230 (0.0009) [2023-10-11 21:13:29,865][71601] Updated weights for policy 0, policy_version 54240 (0.0008) [2023-10-11 21:13:29,984][71635] Updated weights for policy 1, policy_version 54182 (0.0009) [2023-10-11 21:13:30,346][71635] Updated weights for policy 1, policy_version 54192 (0.0008) [2023-10-11 21:13:30,709][71635] Updated weights for policy 1, policy_version 54202 (0.0008) [2023-10-11 21:13:31,034][70582] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 111050752. Throughput: 0: 1834.9, 1: 1828.7. Samples: 27761216. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:13:31,035][70582] Avg episode reward: [(0, '182.310'), (1, '40.530')] [2023-10-11 21:13:33,676][71601] Updated weights for policy 0, policy_version 54250 (0.0008) [2023-10-11 21:13:34,052][71601] Updated weights for policy 0, policy_version 54260 (0.0008) [2023-10-11 21:13:34,412][71635] Updated weights for policy 1, policy_version 54212 (0.0009) [2023-10-11 21:13:34,421][71601] Updated weights for policy 0, policy_version 54270 (0.0008) [2023-10-11 21:13:34,775][71635] Updated weights for policy 1, policy_version 54222 (0.0007) [2023-10-11 21:13:35,144][71635] Updated weights for policy 1, policy_version 54232 (0.0007) [2023-10-11 21:13:36,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 111116288. Throughput: 0: 1826.9, 1: 1829.6. Samples: 27782422. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:13:36,035][70582] Avg episode reward: [(0, '181.040'), (1, '41.020')] [2023-10-11 21:13:38,121][71601] Updated weights for policy 0, policy_version 54280 (0.0008) [2023-10-11 21:13:38,491][71601] Updated weights for policy 0, policy_version 54290 (0.0008) [2023-10-11 21:13:38,769][71635] Updated weights for policy 1, policy_version 54242 (0.0008) [2023-10-11 21:13:38,857][71601] Updated weights for policy 0, policy_version 54300 (0.0010) [2023-10-11 21:13:39,132][71635] Updated weights for policy 1, policy_version 54252 (0.0008) [2023-10-11 21:13:39,497][71635] Updated weights for policy 1, policy_version 54262 (0.0008) [2023-10-11 21:13:39,860][71635] Updated weights for policy 1, policy_version 54272 (0.0007) [2023-10-11 21:13:41,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 111181824. Throughput: 0: 1826.7, 1: 1829.7. Samples: 27803742. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:13:41,035][70582] Avg episode reward: [(0, '186.380'), (1, '40.410')] [2023-10-11 21:13:42,644][71601] Updated weights for policy 0, policy_version 54310 (0.0010) [2023-10-11 21:13:43,024][71601] Updated weights for policy 0, policy_version 54320 (0.0007) [2023-10-11 21:13:43,401][71601] Updated weights for policy 0, policy_version 54330 (0.0009) [2023-10-11 21:13:43,534][71635] Updated weights for policy 1, policy_version 54282 (0.0010) [2023-10-11 21:13:43,893][71635] Updated weights for policy 1, policy_version 54292 (0.0009) [2023-10-11 21:13:44,261][71635] Updated weights for policy 1, policy_version 54302 (0.0009) [2023-10-11 21:13:46,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 111247360. Throughput: 0: 1829.6, 1: 1821.9. Samples: 27814954. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:13:46,034][70582] Avg episode reward: [(0, '186.200'), (1, '45.610')] [2023-10-11 21:13:46,977][71601] Updated weights for policy 0, policy_version 54340 (0.0008) [2023-10-11 21:13:47,354][71601] Updated weights for policy 0, policy_version 54350 (0.0007) [2023-10-11 21:13:47,718][71601] Updated weights for policy 0, policy_version 54360 (0.0008) [2023-10-11 21:13:47,979][71635] Updated weights for policy 1, policy_version 54312 (0.0008) [2023-10-11 21:13:48,340][71635] Updated weights for policy 1, policy_version 54322 (0.0008) [2023-10-11 21:13:48,713][71635] Updated weights for policy 1, policy_version 54332 (0.0008) [2023-10-11 21:13:51,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 111312896. Throughput: 0: 1824.8, 1: 1828.1. Samples: 27836214. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:13:51,035][70582] Avg episode reward: [(0, '186.200'), (1, '48.860')] [2023-10-11 21:13:51,578][71601] Updated weights for policy 0, policy_version 54370 (0.0009) [2023-10-11 21:13:51,939][71601] Updated weights for policy 0, policy_version 54380 (0.0007) [2023-10-11 21:13:52,311][71601] Updated weights for policy 0, policy_version 54390 (0.0009) [2023-10-11 21:13:52,383][71635] Updated weights for policy 1, policy_version 54342 (0.0008) [2023-10-11 21:13:52,681][71601] Updated weights for policy 0, policy_version 54400 (0.0008) [2023-10-11 21:13:52,760][71635] Updated weights for policy 1, policy_version 54352 (0.0010) [2023-10-11 21:13:53,127][71635] Updated weights for policy 1, policy_version 54362 (0.0008) [2023-10-11 21:13:56,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 111378432. Throughput: 0: 1817.1, 1: 1822.0. Samples: 27858946. Policy #0 lag: (min: 4.0, avg: 7.2, max: 36.0) [2023-10-11 21:13:56,034][70582] Avg episode reward: [(0, '186.200'), (1, '48.090')] [2023-10-11 21:13:56,044][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000054368_55672832.pth... [2023-10-11 21:13:56,082][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000052672_53936128.pth [2023-10-11 21:13:56,268][71601] Updated weights for policy 0, policy_version 54410 (0.0008) [2023-10-11 21:13:56,637][71601] Updated weights for policy 0, policy_version 54420 (0.0009) [2023-10-11 21:13:56,789][71635] Updated weights for policy 1, policy_version 54372 (0.0009) [2023-10-11 21:13:57,017][71601] Updated weights for policy 0, policy_version 54430 (0.0008) [2023-10-11 21:13:57,082][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000054432_55738368.pth... [2023-10-11 21:13:57,110][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000052704_53968896.pth [2023-10-11 21:13:57,159][71635] Updated weights for policy 1, policy_version 54382 (0.0009) [2023-10-11 21:13:57,524][71635] Updated weights for policy 1, policy_version 54392 (0.0010) [2023-10-11 21:14:00,707][71601] Updated weights for policy 0, policy_version 54440 (0.0009) [2023-10-11 21:14:01,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 111443968. Throughput: 0: 1817.2, 1: 1822.8. Samples: 27868874. Policy #0 lag: (min: 4.0, avg: 7.2, max: 36.0) [2023-10-11 21:14:01,034][70582] Avg episode reward: [(0, '205.630'), (1, '54.880')] [2023-10-11 21:14:01,087][71601] Updated weights for policy 0, policy_version 54450 (0.0010) [2023-10-11 21:14:01,424][71635] Updated weights for policy 1, policy_version 54402 (0.0010) [2023-10-11 21:14:01,455][71601] Updated weights for policy 0, policy_version 54460 (0.0009) [2023-10-11 21:14:01,832][71635] Updated weights for policy 1, policy_version 54412 (0.0009) [2023-10-11 21:14:02,203][71635] Updated weights for policy 1, policy_version 54422 (0.0010) [2023-10-11 21:14:02,564][71635] Updated weights for policy 1, policy_version 54432 (0.0008) [2023-10-11 21:14:05,216][71601] Updated weights for policy 0, policy_version 54470 (0.0008) [2023-10-11 21:14:05,580][71601] Updated weights for policy 0, policy_version 54480 (0.0007) [2023-10-11 21:14:05,934][71601] Updated weights for policy 0, policy_version 54490 (0.0008) [2023-10-11 21:14:06,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 111509504. Throughput: 0: 1811.1, 1: 1814.4. Samples: 27891284. Policy #0 lag: (min: 4.0, avg: 7.2, max: 36.0) [2023-10-11 21:14:06,034][70582] Avg episode reward: [(0, '201.760'), (1, '53.510')] [2023-10-11 21:14:06,288][71635] Updated weights for policy 1, policy_version 54442 (0.0008) [2023-10-11 21:14:06,650][71635] Updated weights for policy 1, policy_version 54452 (0.0009) [2023-10-11 21:14:07,016][71635] Updated weights for policy 1, policy_version 54462 (0.0007) [2023-10-11 21:14:09,580][71601] Updated weights for policy 0, policy_version 54500 (0.0009) [2023-10-11 21:14:09,964][71601] Updated weights for policy 0, policy_version 54510 (0.0011) [2023-10-11 21:14:10,324][71601] Updated weights for policy 0, policy_version 54520 (0.0010) [2023-10-11 21:14:10,786][71635] Updated weights for policy 1, policy_version 54472 (0.0011) [2023-10-11 21:14:11,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 111607808. Throughput: 0: 1811.8, 1: 1818.4. Samples: 27912966. Policy #0 lag: (min: 4.0, avg: 7.2, max: 36.0) [2023-10-11 21:14:11,034][70582] Avg episode reward: [(0, '201.460'), (1, '51.700')] [2023-10-11 21:14:11,156][71635] Updated weights for policy 1, policy_version 54482 (0.0007) [2023-10-11 21:14:11,525][71635] Updated weights for policy 1, policy_version 54492 (0.0008) [2023-10-11 21:14:13,917][71601] Updated weights for policy 0, policy_version 54530 (0.0010) [2023-10-11 21:14:14,281][71601] Updated weights for policy 0, policy_version 54540 (0.0011) [2023-10-11 21:14:14,646][71601] Updated weights for policy 0, policy_version 54550 (0.0010) [2023-10-11 21:14:15,012][71601] Updated weights for policy 0, policy_version 54560 (0.0010) [2023-10-11 21:14:15,394][71635] Updated weights for policy 1, policy_version 54502 (0.0009) [2023-10-11 21:14:15,755][71635] Updated weights for policy 1, policy_version 54512 (0.0008) [2023-10-11 21:14:16,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 111673344. Throughput: 0: 1807.9, 1: 1813.2. Samples: 27924166. Policy #0 lag: (min: 4.0, avg: 7.2, max: 36.0) [2023-10-11 21:14:16,035][70582] Avg episode reward: [(0, '179.030'), (1, '49.700')] [2023-10-11 21:14:16,117][71635] Updated weights for policy 1, policy_version 54522 (0.0009) [2023-10-11 21:14:18,785][71601] Updated weights for policy 0, policy_version 54570 (0.0008) [2023-10-11 21:14:19,151][71601] Updated weights for policy 0, policy_version 54580 (0.0009) [2023-10-11 21:14:19,518][71601] Updated weights for policy 0, policy_version 54590 (0.0010) [2023-10-11 21:14:19,732][71635] Updated weights for policy 1, policy_version 54532 (0.0008) [2023-10-11 21:14:20,085][71635] Updated weights for policy 1, policy_version 54542 (0.0009) [2023-10-11 21:14:20,463][71635] Updated weights for policy 1, policy_version 54552 (0.0008) [2023-10-11 21:14:21,034][70582] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 111771648. Throughput: 0: 1815.2, 1: 1812.8. Samples: 27945680. Policy #0 lag: (min: 4.0, avg: 7.2, max: 36.0) [2023-10-11 21:14:21,035][70582] Avg episode reward: [(0, '195.130'), (1, '47.960')] [2023-10-11 21:14:23,346][71601] Updated weights for policy 0, policy_version 54600 (0.0009) [2023-10-11 21:14:23,720][71601] Updated weights for policy 0, policy_version 54610 (0.0009) [2023-10-11 21:14:24,095][71601] Updated weights for policy 0, policy_version 54620 (0.0009) [2023-10-11 21:14:24,119][71635] Updated weights for policy 1, policy_version 54562 (0.0011) [2023-10-11 21:14:24,493][71635] Updated weights for policy 1, policy_version 54572 (0.0009) [2023-10-11 21:14:24,854][71635] Updated weights for policy 1, policy_version 54582 (0.0007) [2023-10-11 21:14:25,219][71635] Updated weights for policy 1, policy_version 54592 (0.0007) [2023-10-11 21:14:26,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 111837184. Throughput: 0: 1811.9, 1: 1809.8. Samples: 27966720. Policy #0 lag: (min: 4.0, avg: 7.2, max: 36.0) [2023-10-11 21:14:26,035][70582] Avg episode reward: [(0, '195.310'), (1, '44.370')] [2023-10-11 21:14:27,821][71601] Updated weights for policy 0, policy_version 54630 (0.0008) [2023-10-11 21:14:28,210][71601] Updated weights for policy 0, policy_version 54640 (0.0007) [2023-10-11 21:14:28,578][71601] Updated weights for policy 0, policy_version 54650 (0.0007) [2023-10-11 21:14:28,787][71635] Updated weights for policy 1, policy_version 54602 (0.0007) [2023-10-11 21:14:29,158][71635] Updated weights for policy 1, policy_version 54612 (0.0009) [2023-10-11 21:14:29,516][71635] Updated weights for policy 1, policy_version 54622 (0.0010) [2023-10-11 21:14:31,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 111902720. Throughput: 0: 1816.8, 1: 1819.1. Samples: 27978572. Policy #0 lag: (min: 9.0, avg: 21.0, max: 41.0) [2023-10-11 21:14:31,034][70582] Avg episode reward: [(0, '185.970'), (1, '45.830')] [2023-10-11 21:14:31,981][71601] Updated weights for policy 0, policy_version 54660 (0.0008) [2023-10-11 21:14:32,362][71601] Updated weights for policy 0, policy_version 54670 (0.0009) [2023-10-11 21:14:32,722][71601] Updated weights for policy 0, policy_version 54680 (0.0010) [2023-10-11 21:14:33,243][71635] Updated weights for policy 1, policy_version 54632 (0.0009) [2023-10-11 21:14:33,614][71635] Updated weights for policy 1, policy_version 54642 (0.0009) [2023-10-11 21:14:33,989][71635] Updated weights for policy 1, policy_version 54652 (0.0009) [2023-10-11 21:14:36,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 111968256. Throughput: 0: 1818.2, 1: 1813.9. Samples: 27999658. Policy #0 lag: (min: 9.0, avg: 21.0, max: 41.0) [2023-10-11 21:14:36,034][70582] Avg episode reward: [(0, '175.360'), (1, '40.560')] [2023-10-11 21:14:36,300][71601] Updated weights for policy 0, policy_version 54690 (0.0008) [2023-10-11 21:14:36,669][71601] Updated weights for policy 0, policy_version 54700 (0.0009) [2023-10-11 21:14:37,047][71601] Updated weights for policy 0, policy_version 54710 (0.0009) [2023-10-11 21:14:37,409][71601] Updated weights for policy 0, policy_version 54720 (0.0009) [2023-10-11 21:14:37,826][71635] Updated weights for policy 1, policy_version 54662 (0.0009) [2023-10-11 21:14:38,199][71635] Updated weights for policy 1, policy_version 54672 (0.0009) [2023-10-11 21:14:38,575][71635] Updated weights for policy 1, policy_version 54682 (0.0009) [2023-10-11 21:14:41,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 112033792. Throughput: 0: 1821.9, 1: 1805.4. Samples: 28022174. Policy #0 lag: (min: 9.0, avg: 21.0, max: 41.0) [2023-10-11 21:14:41,035][70582] Avg episode reward: [(0, '175.440'), (1, '44.230')] [2023-10-11 21:14:41,065][71601] Updated weights for policy 0, policy_version 54730 (0.0011) [2023-10-11 21:14:41,433][71601] Updated weights for policy 0, policy_version 54740 (0.0011) [2023-10-11 21:14:41,807][71601] Updated weights for policy 0, policy_version 54750 (0.0010) [2023-10-11 21:14:42,382][71635] Updated weights for policy 1, policy_version 54692 (0.0007) [2023-10-11 21:14:42,758][71635] Updated weights for policy 1, policy_version 54702 (0.0009) [2023-10-11 21:14:43,117][71635] Updated weights for policy 1, policy_version 54712 (0.0007) [2023-10-11 21:14:45,641][71601] Updated weights for policy 0, policy_version 54760 (0.0008) [2023-10-11 21:14:46,015][71601] Updated weights for policy 0, policy_version 54770 (0.0007) [2023-10-11 21:14:46,034][70582] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 112099328. Throughput: 0: 1823.2, 1: 1807.5. Samples: 28032260. Policy #0 lag: (min: 9.0, avg: 21.0, max: 41.0) [2023-10-11 21:14:46,035][70582] Avg episode reward: [(0, '155.660'), (1, '48.390')] [2023-10-11 21:14:46,389][71601] Updated weights for policy 0, policy_version 54780 (0.0008) [2023-10-11 21:14:46,879][71635] Updated weights for policy 1, policy_version 54722 (0.0008) [2023-10-11 21:14:47,289][71635] Updated weights for policy 1, policy_version 54732 (0.0007) [2023-10-11 21:14:47,660][71635] Updated weights for policy 1, policy_version 54742 (0.0010) [2023-10-11 21:14:48,022][71635] Updated weights for policy 1, policy_version 54752 (0.0009) [2023-10-11 21:14:49,989][71601] Updated weights for policy 0, policy_version 54790 (0.0008) [2023-10-11 21:14:50,361][71601] Updated weights for policy 0, policy_version 54800 (0.0007) [2023-10-11 21:14:50,726][71601] Updated weights for policy 0, policy_version 54810 (0.0007) [2023-10-11 21:14:51,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 112197632. Throughput: 0: 1825.4, 1: 1806.0. Samples: 28054698. Policy #0 lag: (min: 9.0, avg: 21.0, max: 41.0) [2023-10-11 21:14:51,035][70582] Avg episode reward: [(0, '142.930'), (1, '50.060')] [2023-10-11 21:14:51,710][71635] Updated weights for policy 1, policy_version 54762 (0.0009) [2023-10-11 21:14:52,075][71635] Updated weights for policy 1, policy_version 54772 (0.0007) [2023-10-11 21:14:52,447][71635] Updated weights for policy 1, policy_version 54782 (0.0007) [2023-10-11 21:14:54,470][71601] Updated weights for policy 0, policy_version 54820 (0.0008) [2023-10-11 21:14:54,840][71601] Updated weights for policy 0, policy_version 54830 (0.0007) [2023-10-11 21:14:55,216][71601] Updated weights for policy 0, policy_version 54840 (0.0008) [2023-10-11 21:14:56,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 112263168. Throughput: 0: 1817.6, 1: 1809.3. Samples: 28076178. Policy #0 lag: (min: 9.0, avg: 21.0, max: 41.0) [2023-10-11 21:14:56,034][70582] Avg episode reward: [(0, '150.310'), (1, '48.590')] [2023-10-11 21:14:56,192][71635] Updated weights for policy 1, policy_version 54792 (0.0008) [2023-10-11 21:14:56,556][71635] Updated weights for policy 1, policy_version 54802 (0.0008) [2023-10-11 21:14:56,925][71635] Updated weights for policy 1, policy_version 54812 (0.0007) [2023-10-11 21:14:58,919][71601] Updated weights for policy 0, policy_version 54850 (0.0008) [2023-10-11 21:14:59,284][71601] Updated weights for policy 0, policy_version 54860 (0.0008) [2023-10-11 21:14:59,650][71601] Updated weights for policy 0, policy_version 54870 (0.0008) [2023-10-11 21:15:00,027][71601] Updated weights for policy 0, policy_version 54880 (0.0011) [2023-10-11 21:15:00,562][71635] Updated weights for policy 1, policy_version 54822 (0.0007) [2023-10-11 21:15:00,929][71635] Updated weights for policy 1, policy_version 54832 (0.0010) [2023-10-11 21:15:01,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 112328704. Throughput: 0: 1814.9, 1: 1810.6. Samples: 28087314. Policy #0 lag: (min: 9.0, avg: 21.0, max: 41.0) [2023-10-11 21:15:01,035][70582] Avg episode reward: [(0, '150.310'), (1, '48.800')] [2023-10-11 21:15:01,285][71635] Updated weights for policy 1, policy_version 54842 (0.0008) [2023-10-11 21:15:03,735][71601] Updated weights for policy 0, policy_version 54890 (0.0007) [2023-10-11 21:15:04,112][71601] Updated weights for policy 0, policy_version 54900 (0.0010) [2023-10-11 21:15:04,472][71601] Updated weights for policy 0, policy_version 54910 (0.0010) [2023-10-11 21:15:05,068][71635] Updated weights for policy 1, policy_version 54852 (0.0010) [2023-10-11 21:15:05,429][71635] Updated weights for policy 1, policy_version 54862 (0.0009) [2023-10-11 21:15:05,807][71635] Updated weights for policy 1, policy_version 54872 (0.0011) [2023-10-11 21:15:06,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 112394240. Throughput: 0: 1812.3, 1: 1813.5. Samples: 28108842. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-11 21:15:06,034][70582] Avg episode reward: [(0, '164.770'), (1, '51.120')] [2023-10-11 21:15:08,118][71601] Updated weights for policy 0, policy_version 54920 (0.0008) [2023-10-11 21:15:08,492][71601] Updated weights for policy 0, policy_version 54930 (0.0008) [2023-10-11 21:15:08,850][71601] Updated weights for policy 0, policy_version 54940 (0.0010) [2023-10-11 21:15:09,568][71635] Updated weights for policy 1, policy_version 54882 (0.0011) [2023-10-11 21:15:09,933][71635] Updated weights for policy 1, policy_version 54892 (0.0008) [2023-10-11 21:15:10,308][71635] Updated weights for policy 1, policy_version 54902 (0.0008) [2023-10-11 21:15:10,669][71635] Updated weights for policy 1, policy_version 54912 (0.0010) [2023-10-11 21:15:11,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 112492544. Throughput: 0: 1817.6, 1: 1816.9. Samples: 28130270. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-11 21:15:11,034][70582] Avg episode reward: [(0, '183.090'), (1, '50.750')] [2023-10-11 21:15:12,683][71601] Updated weights for policy 0, policy_version 54950 (0.0008) [2023-10-11 21:15:13,060][71601] Updated weights for policy 0, policy_version 54960 (0.0009) [2023-10-11 21:15:13,442][71601] Updated weights for policy 0, policy_version 54970 (0.0011) [2023-10-11 21:15:14,269][71635] Updated weights for policy 1, policy_version 54922 (0.0008) [2023-10-11 21:15:14,632][71635] Updated weights for policy 1, policy_version 54932 (0.0008) [2023-10-11 21:15:15,005][71635] Updated weights for policy 1, policy_version 54942 (0.0009) [2023-10-11 21:15:16,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 112558080. Throughput: 0: 1814.6, 1: 1809.1. Samples: 28141638. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-11 21:15:16,034][70582] Avg episode reward: [(0, '182.860'), (1, '51.500')] [2023-10-11 21:15:17,109][71601] Updated weights for policy 0, policy_version 54980 (0.0009) [2023-10-11 21:15:17,472][71601] Updated weights for policy 0, policy_version 54990 (0.0007) [2023-10-11 21:15:17,846][71601] Updated weights for policy 0, policy_version 55000 (0.0007) [2023-10-11 21:15:18,636][71635] Updated weights for policy 1, policy_version 54952 (0.0008) [2023-10-11 21:15:18,993][71635] Updated weights for policy 1, policy_version 54962 (0.0009) [2023-10-11 21:15:19,352][71635] Updated weights for policy 1, policy_version 54972 (0.0011) [2023-10-11 21:15:21,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 112623616. Throughput: 0: 1814.9, 1: 1814.5. Samples: 28162982. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-11 21:15:21,034][70582] Avg episode reward: [(0, '182.840'), (1, '51.820')] [2023-10-11 21:15:21,457][71601] Updated weights for policy 0, policy_version 55010 (0.0008) [2023-10-11 21:15:21,833][71601] Updated weights for policy 0, policy_version 55020 (0.0008) [2023-10-11 21:15:22,198][71601] Updated weights for policy 0, policy_version 55030 (0.0007) [2023-10-11 21:15:22,574][71601] Updated weights for policy 0, policy_version 55040 (0.0009) [2023-10-11 21:15:23,025][71635] Updated weights for policy 1, policy_version 54982 (0.0010) [2023-10-11 21:15:23,386][71635] Updated weights for policy 1, policy_version 54992 (0.0007) [2023-10-11 21:15:23,756][71635] Updated weights for policy 1, policy_version 55002 (0.0009) [2023-10-11 21:15:26,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 112689152. Throughput: 0: 1811.5, 1: 1818.5. Samples: 28185524. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-11 21:15:26,034][70582] Avg episode reward: [(0, '182.840'), (1, '52.070')] [2023-10-11 21:15:26,352][71601] Updated weights for policy 0, policy_version 55050 (0.0009) [2023-10-11 21:15:26,721][71601] Updated weights for policy 0, policy_version 55060 (0.0010) [2023-10-11 21:15:27,101][71601] Updated weights for policy 0, policy_version 55070 (0.0010) [2023-10-11 21:15:27,547][71635] Updated weights for policy 1, policy_version 55012 (0.0008) [2023-10-11 21:15:27,914][71635] Updated weights for policy 1, policy_version 55022 (0.0010) [2023-10-11 21:15:28,275][71635] Updated weights for policy 1, policy_version 55032 (0.0008) [2023-10-11 21:15:30,763][71601] Updated weights for policy 0, policy_version 55080 (0.0008) [2023-10-11 21:15:31,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 112754688. Throughput: 0: 1813.0, 1: 1822.5. Samples: 28195858. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-11 21:15:31,035][70582] Avg episode reward: [(0, '176.390'), (1, '52.290')] [2023-10-11 21:15:31,146][71601] Updated weights for policy 0, policy_version 55090 (0.0008) [2023-10-11 21:15:31,516][71601] Updated weights for policy 0, policy_version 55100 (0.0008) [2023-10-11 21:15:31,912][71635] Updated weights for policy 1, policy_version 55042 (0.0008) [2023-10-11 21:15:32,277][71635] Updated weights for policy 1, policy_version 55052 (0.0011) [2023-10-11 21:15:32,643][71635] Updated weights for policy 1, policy_version 55062 (0.0009) [2023-10-11 21:15:33,005][71635] Updated weights for policy 1, policy_version 55072 (0.0008) [2023-10-11 21:15:35,206][71601] Updated weights for policy 0, policy_version 55110 (0.0008) [2023-10-11 21:15:35,585][71601] Updated weights for policy 0, policy_version 55120 (0.0010) [2023-10-11 21:15:35,957][71601] Updated weights for policy 0, policy_version 55130 (0.0007) [2023-10-11 21:15:36,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 112820224. Throughput: 0: 1817.8, 1: 1821.6. Samples: 28218472. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-11 21:15:36,034][70582] Avg episode reward: [(0, '178.060'), (1, '58.710')] [2023-10-11 21:15:36,696][71635] Updated weights for policy 1, policy_version 55082 (0.0009) [2023-10-11 21:15:37,054][71635] Updated weights for policy 1, policy_version 55092 (0.0008) [2023-10-11 21:15:37,432][71635] Updated weights for policy 1, policy_version 55102 (0.0010) [2023-10-11 21:15:39,787][71601] Updated weights for policy 0, policy_version 55140 (0.0008) [2023-10-11 21:15:40,150][71601] Updated weights for policy 0, policy_version 55150 (0.0008) [2023-10-11 21:15:40,518][71601] Updated weights for policy 0, policy_version 55160 (0.0009) [2023-10-11 21:15:41,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 112918528. Throughput: 0: 1826.5, 1: 1815.5. Samples: 28240066. Policy #0 lag: (min: 11.0, avg: 11.5, max: 25.0) [2023-10-11 21:15:41,035][70582] Avg episode reward: [(0, '176.670'), (1, '62.950')] [2023-10-11 21:15:41,234][71635] Updated weights for policy 1, policy_version 55112 (0.0009) [2023-10-11 21:15:41,599][71635] Updated weights for policy 1, policy_version 55122 (0.0008) [2023-10-11 21:15:41,972][71635] Updated weights for policy 1, policy_version 55132 (0.0008) [2023-10-11 21:15:44,196][71601] Updated weights for policy 0, policy_version 55170 (0.0008) [2023-10-11 21:15:44,572][71601] Updated weights for policy 0, policy_version 55180 (0.0011) [2023-10-11 21:15:44,946][71601] Updated weights for policy 0, policy_version 55190 (0.0009) [2023-10-11 21:15:45,319][71601] Updated weights for policy 0, policy_version 55200 (0.0011) [2023-10-11 21:15:45,637][71635] Updated weights for policy 1, policy_version 55142 (0.0008) [2023-10-11 21:15:45,999][71635] Updated weights for policy 1, policy_version 55152 (0.0011) [2023-10-11 21:15:46,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 112984064. Throughput: 0: 1817.9, 1: 1815.4. Samples: 28250814. Policy #0 lag: (min: 11.0, avg: 11.5, max: 25.0) [2023-10-11 21:15:46,035][70582] Avg episode reward: [(0, '170.960'), (1, '61.380')] [2023-10-11 21:15:46,368][71635] Updated weights for policy 1, policy_version 55162 (0.0010) [2023-10-11 21:15:49,009][71601] Updated weights for policy 0, policy_version 55210 (0.0009) [2023-10-11 21:15:49,375][71601] Updated weights for policy 0, policy_version 55220 (0.0010) [2023-10-11 21:15:49,762][71601] Updated weights for policy 0, policy_version 55230 (0.0008) [2023-10-11 21:15:50,132][71635] Updated weights for policy 1, policy_version 55172 (0.0011) [2023-10-11 21:15:50,501][71635] Updated weights for policy 1, policy_version 55182 (0.0010) [2023-10-11 21:15:50,873][71635] Updated weights for policy 1, policy_version 55192 (0.0011) [2023-10-11 21:15:51,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 113049600. Throughput: 0: 1820.0, 1: 1808.7. Samples: 28272134. Policy #0 lag: (min: 11.0, avg: 11.5, max: 25.0) [2023-10-11 21:15:51,035][70582] Avg episode reward: [(0, '181.360'), (1, '59.490')] [2023-10-11 21:15:53,333][71601] Updated weights for policy 0, policy_version 55240 (0.0008) [2023-10-11 21:15:53,706][71601] Updated weights for policy 0, policy_version 55250 (0.0009) [2023-10-11 21:15:54,085][71601] Updated weights for policy 0, policy_version 55260 (0.0009) [2023-10-11 21:15:54,579][71635] Updated weights for policy 1, policy_version 55202 (0.0009) [2023-10-11 21:15:54,952][71635] Updated weights for policy 1, policy_version 55212 (0.0007) [2023-10-11 21:15:55,316][71635] Updated weights for policy 1, policy_version 55222 (0.0007) [2023-10-11 21:15:55,680][71635] Updated weights for policy 1, policy_version 55232 (0.0007) [2023-10-11 21:15:56,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 113147904. Throughput: 0: 1817.4, 1: 1813.6. Samples: 28293666. Policy #0 lag: (min: 11.0, avg: 11.5, max: 25.0) [2023-10-11 21:15:56,035][70582] Avg episode reward: [(0, '186.490'), (1, '58.610')] [2023-10-11 21:15:56,046][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000055232_56557568.pth... [2023-10-11 21:15:56,046][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000055264_56590336.pth... [2023-10-11 21:15:56,082][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000053568_54853632.pth [2023-10-11 21:15:56,084][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000053536_54820864.pth [2023-10-11 21:15:57,801][71601] Updated weights for policy 0, policy_version 55270 (0.0009) [2023-10-11 21:15:58,162][71601] Updated weights for policy 0, policy_version 55280 (0.0007) [2023-10-11 21:15:58,537][71601] Updated weights for policy 0, policy_version 55290 (0.0009) [2023-10-11 21:15:59,431][71635] Updated weights for policy 1, policy_version 55242 (0.0008) [2023-10-11 21:15:59,799][71635] Updated weights for policy 1, policy_version 55252 (0.0008) [2023-10-11 21:16:00,157][71635] Updated weights for policy 1, policy_version 55262 (0.0008) [2023-10-11 21:16:01,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 113213440. Throughput: 0: 1819.8, 1: 1806.5. Samples: 28304822. Policy #0 lag: (min: 11.0, avg: 11.5, max: 25.0) [2023-10-11 21:16:01,035][70582] Avg episode reward: [(0, '193.580'), (1, '54.270')] [2023-10-11 21:16:02,232][71601] Updated weights for policy 0, policy_version 55300 (0.0008) [2023-10-11 21:16:02,605][71601] Updated weights for policy 0, policy_version 55310 (0.0008) [2023-10-11 21:16:02,975][71601] Updated weights for policy 0, policy_version 55320 (0.0007) [2023-10-11 21:16:03,643][71635] Updated weights for policy 1, policy_version 55272 (0.0008) [2023-10-11 21:16:04,010][71635] Updated weights for policy 1, policy_version 55282 (0.0010) [2023-10-11 21:16:04,368][71635] Updated weights for policy 1, policy_version 55292 (0.0010) [2023-10-11 21:16:06,034][70582] Fps is (10 sec: 13107.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 113278976. Throughput: 0: 1815.5, 1: 1811.7. Samples: 28326206. Policy #0 lag: (min: 11.0, avg: 11.5, max: 25.0) [2023-10-11 21:16:06,034][70582] Avg episode reward: [(0, '188.460'), (1, '54.630')] [2023-10-11 21:16:06,614][71601] Updated weights for policy 0, policy_version 55330 (0.0008) [2023-10-11 21:16:06,984][71601] Updated weights for policy 0, policy_version 55340 (0.0008) [2023-10-11 21:16:07,353][71601] Updated weights for policy 0, policy_version 55350 (0.0007) [2023-10-11 21:16:07,722][71601] Updated weights for policy 0, policy_version 55360 (0.0009) [2023-10-11 21:16:08,081][71635] Updated weights for policy 1, policy_version 55302 (0.0007) [2023-10-11 21:16:08,452][71635] Updated weights for policy 1, policy_version 55312 (0.0007) [2023-10-11 21:16:08,819][71635] Updated weights for policy 1, policy_version 55322 (0.0008) [2023-10-11 21:16:11,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 113344512. Throughput: 0: 1817.9, 1: 1812.8. Samples: 28348910. Policy #0 lag: (min: 11.0, avg: 11.5, max: 25.0) [2023-10-11 21:16:11,035][70582] Avg episode reward: [(0, '175.390'), (1, '57.860')] [2023-10-11 21:16:11,508][71601] Updated weights for policy 0, policy_version 55370 (0.0007) [2023-10-11 21:16:11,870][71601] Updated weights for policy 0, policy_version 55380 (0.0008) [2023-10-11 21:16:12,243][71601] Updated weights for policy 0, policy_version 55390 (0.0009) [2023-10-11 21:16:12,517][71635] Updated weights for policy 1, policy_version 55332 (0.0010) [2023-10-11 21:16:12,891][71635] Updated weights for policy 1, policy_version 55342 (0.0010) [2023-10-11 21:16:13,254][71635] Updated weights for policy 1, policy_version 55352 (0.0009) [2023-10-11 21:16:15,794][71601] Updated weights for policy 0, policy_version 55400 (0.0009) [2023-10-11 21:16:16,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 113410048. Throughput: 0: 1818.5, 1: 1815.6. Samples: 28359392. Policy #0 lag: (min: 11.0, avg: 11.5, max: 25.0) [2023-10-11 21:16:16,035][70582] Avg episode reward: [(0, '162.320'), (1, '53.010')] [2023-10-11 21:16:16,160][71601] Updated weights for policy 0, policy_version 55410 (0.0008) [2023-10-11 21:16:16,529][71601] Updated weights for policy 0, policy_version 55420 (0.0007) [2023-10-11 21:16:16,848][71635] Updated weights for policy 1, policy_version 55362 (0.0011) [2023-10-11 21:16:17,251][71635] Updated weights for policy 1, policy_version 55372 (0.0007) [2023-10-11 21:16:17,609][71635] Updated weights for policy 1, policy_version 55382 (0.0008) [2023-10-11 21:16:17,972][71635] Updated weights for policy 1, policy_version 55392 (0.0009) [2023-10-11 21:16:20,234][71601] Updated weights for policy 0, policy_version 55430 (0.0008) [2023-10-11 21:16:20,603][71601] Updated weights for policy 0, policy_version 55440 (0.0008) [2023-10-11 21:16:20,981][71601] Updated weights for policy 0, policy_version 55450 (0.0007) [2023-10-11 21:16:21,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 113475584. Throughput: 0: 1811.9, 1: 1816.3. Samples: 28381740. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-11 21:16:21,034][70582] Avg episode reward: [(0, '171.670'), (1, '52.570')] [2023-10-11 21:16:21,629][71635] Updated weights for policy 1, policy_version 55402 (0.0007) [2023-10-11 21:16:21,993][71635] Updated weights for policy 1, policy_version 55412 (0.0007) [2023-10-11 21:16:22,351][71635] Updated weights for policy 1, policy_version 55422 (0.0009) [2023-10-11 21:16:24,642][71601] Updated weights for policy 0, policy_version 55460 (0.0008) [2023-10-11 21:16:25,011][71601] Updated weights for policy 0, policy_version 55470 (0.0007) [2023-10-11 21:16:25,384][71601] Updated weights for policy 0, policy_version 55480 (0.0007) [2023-10-11 21:16:25,979][71635] Updated weights for policy 1, policy_version 55432 (0.0008) [2023-10-11 21:16:26,034][70582] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 113573888. Throughput: 0: 1813.2, 1: 1821.5. Samples: 28403628. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-11 21:16:26,034][70582] Avg episode reward: [(0, '171.680'), (1, '51.100')] [2023-10-11 21:16:26,343][71635] Updated weights for policy 1, policy_version 55442 (0.0008) [2023-10-11 21:16:26,706][71635] Updated weights for policy 1, policy_version 55452 (0.0008) [2023-10-11 21:16:28,963][71601] Updated weights for policy 0, policy_version 55490 (0.0007) [2023-10-11 21:16:29,331][71601] Updated weights for policy 0, policy_version 55500 (0.0007) [2023-10-11 21:16:29,707][71601] Updated weights for policy 0, policy_version 55510 (0.0007) [2023-10-11 21:16:30,082][71601] Updated weights for policy 0, policy_version 55520 (0.0010) [2023-10-11 21:16:30,418][71635] Updated weights for policy 1, policy_version 55462 (0.0009) [2023-10-11 21:16:30,792][71635] Updated weights for policy 1, policy_version 55472 (0.0009) [2023-10-11 21:16:31,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 113639424. Throughput: 0: 1819.8, 1: 1824.4. Samples: 28414804. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-11 21:16:31,035][70582] Avg episode reward: [(0, '172.740'), (1, '59.640')] [2023-10-11 21:16:31,161][71635] Updated weights for policy 1, policy_version 55482 (0.0008) [2023-10-11 21:16:33,793][71601] Updated weights for policy 0, policy_version 55530 (0.0008) [2023-10-11 21:16:34,157][71601] Updated weights for policy 0, policy_version 55540 (0.0009) [2023-10-11 21:16:34,534][71601] Updated weights for policy 0, policy_version 55550 (0.0009) [2023-10-11 21:16:34,883][71635] Updated weights for policy 1, policy_version 55492 (0.0008) [2023-10-11 21:16:35,249][71635] Updated weights for policy 1, policy_version 55502 (0.0010) [2023-10-11 21:16:35,616][71635] Updated weights for policy 1, policy_version 55512 (0.0008) [2023-10-11 21:16:36,034][70582] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 113737728. Throughput: 0: 1819.3, 1: 1826.8. Samples: 28436208. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-11 21:16:36,035][70582] Avg episode reward: [(0, '155.410'), (1, '60.860')] [2023-10-11 21:16:38,035][71601] Updated weights for policy 0, policy_version 55560 (0.0009) [2023-10-11 21:16:38,411][71601] Updated weights for policy 0, policy_version 55570 (0.0008) [2023-10-11 21:16:38,782][71601] Updated weights for policy 0, policy_version 55580 (0.0008) [2023-10-11 21:16:39,382][71635] Updated weights for policy 1, policy_version 55522 (0.0008) [2023-10-11 21:16:39,758][71635] Updated weights for policy 1, policy_version 55532 (0.0007) [2023-10-11 21:16:40,125][71635] Updated weights for policy 1, policy_version 55542 (0.0008) [2023-10-11 21:16:40,496][71635] Updated weights for policy 1, policy_version 55552 (0.0007) [2023-10-11 21:16:41,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 113803264. Throughput: 0: 1827.9, 1: 1821.2. Samples: 28457874. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-11 21:16:41,035][70582] Avg episode reward: [(0, '155.920'), (1, '62.200')] [2023-10-11 21:16:42,470][71601] Updated weights for policy 0, policy_version 55590 (0.0008) [2023-10-11 21:16:42,860][71601] Updated weights for policy 0, policy_version 55600 (0.0007) [2023-10-11 21:16:43,232][71601] Updated weights for policy 0, policy_version 55610 (0.0009) [2023-10-11 21:16:44,172][71635] Updated weights for policy 1, policy_version 55562 (0.0008) [2023-10-11 21:16:44,535][71635] Updated weights for policy 1, policy_version 55572 (0.0008) [2023-10-11 21:16:44,890][71635] Updated weights for policy 1, policy_version 55582 (0.0009) [2023-10-11 21:16:46,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 113868800. Throughput: 0: 1819.3, 1: 1833.9. Samples: 28469216. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-11 21:16:46,035][70582] Avg episode reward: [(0, '160.660'), (1, '64.190')] [2023-10-11 21:16:47,096][71601] Updated weights for policy 0, policy_version 55620 (0.0009) [2023-10-11 21:16:47,473][71601] Updated weights for policy 0, policy_version 55630 (0.0009) [2023-10-11 21:16:47,856][71601] Updated weights for policy 0, policy_version 55640 (0.0011) [2023-10-11 21:16:48,550][71635] Updated weights for policy 1, policy_version 55592 (0.0009) [2023-10-11 21:16:48,915][71635] Updated weights for policy 1, policy_version 55602 (0.0009) [2023-10-11 21:16:49,296][71635] Updated weights for policy 1, policy_version 55612 (0.0011) [2023-10-11 21:16:51,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 113934336. Throughput: 0: 1825.4, 1: 1826.5. Samples: 28490540. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-11 21:16:51,035][70582] Avg episode reward: [(0, '160.590'), (1, '62.840')] [2023-10-11 21:16:51,581][71601] Updated weights for policy 0, policy_version 55650 (0.0009) [2023-10-11 21:16:51,961][71601] Updated weights for policy 0, policy_version 55660 (0.0011) [2023-10-11 21:16:52,337][71601] Updated weights for policy 0, policy_version 55670 (0.0011) [2023-10-11 21:16:52,702][71601] Updated weights for policy 0, policy_version 55680 (0.0009) [2023-10-11 21:16:52,962][71635] Updated weights for policy 1, policy_version 55622 (0.0009) [2023-10-11 21:16:53,324][71635] Updated weights for policy 1, policy_version 55632 (0.0008) [2023-10-11 21:16:53,690][71635] Updated weights for policy 1, policy_version 55642 (0.0007) [2023-10-11 21:16:56,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 113999872. Throughput: 0: 1821.4, 1: 1826.6. Samples: 28513072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:16:56,034][70582] Avg episode reward: [(0, '156.990'), (1, '61.080')] [2023-10-11 21:16:56,347][71601] Updated weights for policy 0, policy_version 55690 (0.0010) [2023-10-11 21:16:56,711][71601] Updated weights for policy 0, policy_version 55700 (0.0009) [2023-10-11 21:16:57,080][71601] Updated weights for policy 0, policy_version 55710 (0.0009) [2023-10-11 21:16:57,437][71635] Updated weights for policy 1, policy_version 55652 (0.0008) [2023-10-11 21:16:57,800][71635] Updated weights for policy 1, policy_version 55662 (0.0008) [2023-10-11 21:16:58,167][71635] Updated weights for policy 1, policy_version 55672 (0.0009) [2023-10-11 21:17:00,638][71601] Updated weights for policy 0, policy_version 55720 (0.0008) [2023-10-11 21:17:01,020][71601] Updated weights for policy 0, policy_version 55730 (0.0009) [2023-10-11 21:17:01,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 114065408. Throughput: 0: 1824.9, 1: 1820.0. Samples: 28523410. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:17:01,035][70582] Avg episode reward: [(0, '139.920'), (1, '61.350')] [2023-10-11 21:17:01,399][71601] Updated weights for policy 0, policy_version 55740 (0.0008) [2023-10-11 21:17:02,019][71635] Updated weights for policy 1, policy_version 55682 (0.0009) [2023-10-11 21:17:02,380][71635] Updated weights for policy 1, policy_version 55692 (0.0010) [2023-10-11 21:17:02,759][71635] Updated weights for policy 1, policy_version 55702 (0.0012) [2023-10-11 21:17:03,119][71635] Updated weights for policy 1, policy_version 55712 (0.0010) [2023-10-11 21:17:04,998][71601] Updated weights for policy 0, policy_version 55750 (0.0009) [2023-10-11 21:17:05,364][71601] Updated weights for policy 0, policy_version 55760 (0.0009) [2023-10-11 21:17:05,728][71601] Updated weights for policy 0, policy_version 55770 (0.0008) [2023-10-11 21:17:06,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 114163712. Throughput: 0: 1827.1, 1: 1813.3. Samples: 28545556. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:17:06,034][70582] Avg episode reward: [(0, '144.040'), (1, '62.670')] [2023-10-11 21:17:06,837][71635] Updated weights for policy 1, policy_version 55722 (0.0008) [2023-10-11 21:17:07,213][71635] Updated weights for policy 1, policy_version 55732 (0.0011) [2023-10-11 21:17:07,583][71635] Updated weights for policy 1, policy_version 55742 (0.0009) [2023-10-11 21:17:09,680][71601] Updated weights for policy 0, policy_version 55780 (0.0010) [2023-10-11 21:17:10,055][71601] Updated weights for policy 0, policy_version 55790 (0.0008) [2023-10-11 21:17:10,423][71601] Updated weights for policy 0, policy_version 55800 (0.0008) [2023-10-11 21:17:11,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 114229248. Throughput: 0: 1823.2, 1: 1813.4. Samples: 28567274. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:17:11,034][70582] Avg episode reward: [(0, '144.270'), (1, '58.300')] [2023-10-11 21:17:11,240][71635] Updated weights for policy 1, policy_version 55752 (0.0010) [2023-10-11 21:17:11,609][71635] Updated weights for policy 1, policy_version 55762 (0.0008) [2023-10-11 21:17:11,975][71635] Updated weights for policy 1, policy_version 55772 (0.0007) [2023-10-11 21:17:14,226][71601] Updated weights for policy 0, policy_version 55810 (0.0009) [2023-10-11 21:17:14,606][71601] Updated weights for policy 0, policy_version 55820 (0.0009) [2023-10-11 21:17:14,983][71601] Updated weights for policy 0, policy_version 55830 (0.0007) [2023-10-11 21:17:15,356][71601] Updated weights for policy 0, policy_version 55840 (0.0007) [2023-10-11 21:17:15,643][71635] Updated weights for policy 1, policy_version 55782 (0.0008) [2023-10-11 21:17:16,004][71635] Updated weights for policy 1, policy_version 55792 (0.0010) [2023-10-11 21:17:16,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 114294784. Throughput: 0: 1817.3, 1: 1811.4. Samples: 28578096. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:17:16,034][70582] Avg episode reward: [(0, '154.940'), (1, '54.880')] [2023-10-11 21:17:16,372][71635] Updated weights for policy 1, policy_version 55802 (0.0009) [2023-10-11 21:17:19,126][71601] Updated weights for policy 0, policy_version 55850 (0.0009) [2023-10-11 21:17:19,500][71601] Updated weights for policy 0, policy_version 55860 (0.0009) [2023-10-11 21:17:19,878][71601] Updated weights for policy 0, policy_version 55870 (0.0008) [2023-10-11 21:17:20,027][71635] Updated weights for policy 1, policy_version 55812 (0.0008) [2023-10-11 21:17:20,395][71635] Updated weights for policy 1, policy_version 55822 (0.0008) [2023-10-11 21:17:20,771][71635] Updated weights for policy 1, policy_version 55832 (0.0008) [2023-10-11 21:17:21,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 114360320. Throughput: 0: 1826.2, 1: 1812.4. Samples: 28599944. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:17:21,034][70582] Avg episode reward: [(0, '155.110'), (1, '55.590')] [2023-10-11 21:17:23,555][71601] Updated weights for policy 0, policy_version 55880 (0.0009) [2023-10-11 21:17:23,919][71601] Updated weights for policy 0, policy_version 55890 (0.0010) [2023-10-11 21:17:24,294][71601] Updated weights for policy 0, policy_version 55900 (0.0008) [2023-10-11 21:17:24,350][71635] Updated weights for policy 1, policy_version 55842 (0.0009) [2023-10-11 21:17:24,727][71635] Updated weights for policy 1, policy_version 55852 (0.0008) [2023-10-11 21:17:25,091][71635] Updated weights for policy 1, policy_version 55862 (0.0007) [2023-10-11 21:17:25,460][71635] Updated weights for policy 1, policy_version 55872 (0.0009) [2023-10-11 21:17:26,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 114458624. Throughput: 0: 1812.8, 1: 1813.0. Samples: 28621032. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:17:26,034][70582] Avg episode reward: [(0, '155.110'), (1, '53.720')] [2023-10-11 21:17:27,944][71601] Updated weights for policy 0, policy_version 55910 (0.0007) [2023-10-11 21:17:28,321][71601] Updated weights for policy 0, policy_version 55920 (0.0008) [2023-10-11 21:17:28,689][71601] Updated weights for policy 0, policy_version 55930 (0.0008) [2023-10-11 21:17:29,108][71635] Updated weights for policy 1, policy_version 55882 (0.0011) [2023-10-11 21:17:29,484][71635] Updated weights for policy 1, policy_version 55892 (0.0011) [2023-10-11 21:17:29,853][71635] Updated weights for policy 1, policy_version 55902 (0.0010) [2023-10-11 21:17:31,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 114524160. Throughput: 0: 1830.7, 1: 1806.5. Samples: 28632888. Policy #0 lag: (min: 28.0, avg: 28.0, max: 30.0) [2023-10-11 21:17:31,034][70582] Avg episode reward: [(0, '155.110'), (1, '51.150')] [2023-10-11 21:17:32,258][71601] Updated weights for policy 0, policy_version 55940 (0.0007) [2023-10-11 21:17:32,629][71601] Updated weights for policy 0, policy_version 55950 (0.0008) [2023-10-11 21:17:32,998][71601] Updated weights for policy 0, policy_version 55960 (0.0007) [2023-10-11 21:17:33,548][71635] Updated weights for policy 1, policy_version 55912 (0.0008) [2023-10-11 21:17:33,925][71635] Updated weights for policy 1, policy_version 55922 (0.0009) [2023-10-11 21:17:34,291][71635] Updated weights for policy 1, policy_version 55932 (0.0008) [2023-10-11 21:17:36,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 114589696. Throughput: 0: 1818.6, 1: 1807.4. Samples: 28653710. Policy #0 lag: (min: 28.0, avg: 28.0, max: 30.0) [2023-10-11 21:17:36,034][70582] Avg episode reward: [(0, '161.860'), (1, '51.550')] [2023-10-11 21:17:36,727][71601] Updated weights for policy 0, policy_version 55970 (0.0008) [2023-10-11 21:17:37,095][71601] Updated weights for policy 0, policy_version 55980 (0.0010) [2023-10-11 21:17:37,463][71601] Updated weights for policy 0, policy_version 55990 (0.0008) [2023-10-11 21:17:37,823][71601] Updated weights for policy 0, policy_version 56000 (0.0010) [2023-10-11 21:17:37,991][71635] Updated weights for policy 1, policy_version 55942 (0.0007) [2023-10-11 21:17:38,352][71635] Updated weights for policy 1, policy_version 55952 (0.0007) [2023-10-11 21:17:38,722][71635] Updated weights for policy 1, policy_version 55962 (0.0009) [2023-10-11 21:17:41,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 114655232. Throughput: 0: 1822.2, 1: 1810.4. Samples: 28676542. Policy #0 lag: (min: 28.0, avg: 28.0, max: 30.0) [2023-10-11 21:17:41,034][70582] Avg episode reward: [(0, '173.400'), (1, '48.720')] [2023-10-11 21:17:41,438][71601] Updated weights for policy 0, policy_version 56010 (0.0008) [2023-10-11 21:17:41,812][71601] Updated weights for policy 0, policy_version 56020 (0.0009) [2023-10-11 21:17:42,193][71601] Updated weights for policy 0, policy_version 56030 (0.0009) [2023-10-11 21:17:42,480][71635] Updated weights for policy 1, policy_version 55972 (0.0009) [2023-10-11 21:17:42,852][71635] Updated weights for policy 1, policy_version 55982 (0.0010) [2023-10-11 21:17:43,206][71635] Updated weights for policy 1, policy_version 55992 (0.0010) [2023-10-11 21:17:45,773][71601] Updated weights for policy 0, policy_version 56040 (0.0009) [2023-10-11 21:17:46,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 114720768. Throughput: 0: 1812.0, 1: 1813.1. Samples: 28686536. Policy #0 lag: (min: 28.0, avg: 28.0, max: 30.0) [2023-10-11 21:17:46,035][70582] Avg episode reward: [(0, '157.400'), (1, '48.140')] [2023-10-11 21:17:46,132][71601] Updated weights for policy 0, policy_version 56050 (0.0009) [2023-10-11 21:17:46,497][71601] Updated weights for policy 0, policy_version 56060 (0.0010) [2023-10-11 21:17:47,131][71635] Updated weights for policy 1, policy_version 56002 (0.0010) [2023-10-11 21:17:47,506][71635] Updated weights for policy 1, policy_version 56012 (0.0009) [2023-10-11 21:17:47,872][71635] Updated weights for policy 1, policy_version 56022 (0.0007) [2023-10-11 21:17:48,242][71635] Updated weights for policy 1, policy_version 56032 (0.0008) [2023-10-11 21:17:50,274][71601] Updated weights for policy 0, policy_version 56070 (0.0010) [2023-10-11 21:17:50,621][71601] Updated weights for policy 0, policy_version 56080 (0.0008) [2023-10-11 21:17:50,985][71601] Updated weights for policy 0, policy_version 56090 (0.0007) [2023-10-11 21:17:51,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 114786304. Throughput: 0: 1812.6, 1: 1824.5. Samples: 28709226. Policy #0 lag: (min: 28.0, avg: 28.0, max: 30.0) [2023-10-11 21:17:51,034][70582] Avg episode reward: [(0, '155.680'), (1, '48.470')] [2023-10-11 21:17:51,962][71635] Updated weights for policy 1, policy_version 56042 (0.0008) [2023-10-11 21:17:52,324][71635] Updated weights for policy 1, policy_version 56052 (0.0010) [2023-10-11 21:17:52,686][71635] Updated weights for policy 1, policy_version 56062 (0.0007) [2023-10-11 21:17:54,682][71601] Updated weights for policy 0, policy_version 56100 (0.0010) [2023-10-11 21:17:55,037][71601] Updated weights for policy 0, policy_version 56110 (0.0009) [2023-10-11 21:17:55,397][71601] Updated weights for policy 0, policy_version 56120 (0.0009) [2023-10-11 21:17:56,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 114884608. Throughput: 0: 1819.2, 1: 1826.0. Samples: 28731310. Policy #0 lag: (min: 28.0, avg: 28.0, max: 30.0) [2023-10-11 21:17:56,035][70582] Avg episode reward: [(0, '155.820'), (1, '48.320')] [2023-10-11 21:17:56,045][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000056128_57475072.pth... [2023-10-11 21:17:56,074][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000054432_55738368.pth [2023-10-11 21:17:56,076][71635] Updated weights for policy 1, policy_version 56072 (0.0007) [2023-10-11 21:17:56,442][71635] Updated weights for policy 1, policy_version 56082 (0.0008) [2023-10-11 21:17:56,800][71635] Updated weights for policy 1, policy_version 56092 (0.0007) [2023-10-11 21:17:56,943][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000056096_57442304.pth... [2023-10-11 21:17:56,971][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000054368_55672832.pth [2023-10-11 21:17:58,967][71601] Updated weights for policy 0, policy_version 56130 (0.0008) [2023-10-11 21:17:59,325][71601] Updated weights for policy 0, policy_version 56140 (0.0010) [2023-10-11 21:17:59,699][71601] Updated weights for policy 0, policy_version 56150 (0.0009) [2023-10-11 21:18:00,060][71601] Updated weights for policy 0, policy_version 56160 (0.0007) [2023-10-11 21:18:00,272][71635] Updated weights for policy 1, policy_version 56102 (0.0007) [2023-10-11 21:18:00,627][71635] Updated weights for policy 1, policy_version 56112 (0.0008) [2023-10-11 21:18:00,997][71635] Updated weights for policy 1, policy_version 56122 (0.0011) [2023-10-11 21:18:01,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 114950144. Throughput: 0: 1830.0, 1: 1833.2. Samples: 28742938. Policy #0 lag: (min: 28.0, avg: 28.0, max: 30.0) [2023-10-11 21:18:01,034][70582] Avg episode reward: [(0, '151.910'), (1, '49.670')] [2023-10-11 21:18:03,728][71601] Updated weights for policy 0, policy_version 56170 (0.0008) [2023-10-11 21:18:04,088][71601] Updated weights for policy 0, policy_version 56180 (0.0008) [2023-10-11 21:18:04,462][71601] Updated weights for policy 0, policy_version 56190 (0.0009) [2023-10-11 21:18:04,687][71635] Updated weights for policy 1, policy_version 56132 (0.0007) [2023-10-11 21:18:05,051][71635] Updated weights for policy 1, policy_version 56142 (0.0008) [2023-10-11 21:18:05,412][71635] Updated weights for policy 1, policy_version 56152 (0.0007) [2023-10-11 21:18:06,034][70582] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 115048448. Throughput: 0: 1824.2, 1: 1836.1. Samples: 28764656. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-11 21:18:06,034][70582] Avg episode reward: [(0, '135.240'), (1, '52.530')] [2023-10-11 21:18:07,842][71601] Updated weights for policy 0, policy_version 56200 (0.0008) [2023-10-11 21:18:08,209][71601] Updated weights for policy 0, policy_version 56210 (0.0010) [2023-10-11 21:18:08,576][71601] Updated weights for policy 0, policy_version 56220 (0.0009) [2023-10-11 21:18:09,026][71635] Updated weights for policy 1, policy_version 56162 (0.0008) [2023-10-11 21:18:09,388][71635] Updated weights for policy 1, policy_version 56172 (0.0009) [2023-10-11 21:18:09,752][71635] Updated weights for policy 1, policy_version 56182 (0.0009) [2023-10-11 21:18:10,116][71635] Updated weights for policy 1, policy_version 56192 (0.0010) [2023-10-11 21:18:11,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 115113984. Throughput: 0: 1838.3, 1: 1835.1. Samples: 28786336. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-11 21:18:11,035][70582] Avg episode reward: [(0, '134.450'), (1, '51.540')] [2023-10-11 21:18:12,316][71601] Updated weights for policy 0, policy_version 56230 (0.0010) [2023-10-11 21:18:12,701][71601] Updated weights for policy 0, policy_version 56240 (0.0008) [2023-10-11 21:18:13,075][71601] Updated weights for policy 0, policy_version 56250 (0.0009) [2023-10-11 21:18:13,846][71635] Updated weights for policy 1, policy_version 56202 (0.0008) [2023-10-11 21:18:14,215][71635] Updated weights for policy 1, policy_version 56212 (0.0010) [2023-10-11 21:18:14,581][71635] Updated weights for policy 1, policy_version 56222 (0.0008) [2023-10-11 21:18:16,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 115179520. Throughput: 0: 1822.0, 1: 1842.7. Samples: 28797800. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-11 21:18:16,034][70582] Avg episode reward: [(0, '144.050'), (1, '57.520')] [2023-10-11 21:18:16,782][71601] Updated weights for policy 0, policy_version 56260 (0.0009) [2023-10-11 21:18:17,161][71601] Updated weights for policy 0, policy_version 56270 (0.0008) [2023-10-11 21:18:17,525][71601] Updated weights for policy 0, policy_version 56280 (0.0008) [2023-10-11 21:18:18,167][71635] Updated weights for policy 1, policy_version 56232 (0.0008) [2023-10-11 21:18:18,541][71635] Updated weights for policy 1, policy_version 56242 (0.0008) [2023-10-11 21:18:18,909][71635] Updated weights for policy 1, policy_version 56252 (0.0008) [2023-10-11 21:18:21,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 115245056. Throughput: 0: 1835.3, 1: 1846.2. Samples: 28819378. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-11 21:18:21,034][70582] Avg episode reward: [(0, '146.760'), (1, '55.890')] [2023-10-11 21:18:21,237][71601] Updated weights for policy 0, policy_version 56290 (0.0008) [2023-10-11 21:18:21,609][71601] Updated weights for policy 0, policy_version 56300 (0.0009) [2023-10-11 21:18:21,984][71601] Updated weights for policy 0, policy_version 56310 (0.0009) [2023-10-11 21:18:22,363][71601] Updated weights for policy 0, policy_version 56320 (0.0007) [2023-10-11 21:18:22,602][71635] Updated weights for policy 1, policy_version 56262 (0.0009) [2023-10-11 21:18:22,967][71635] Updated weights for policy 1, policy_version 56272 (0.0008) [2023-10-11 21:18:23,337][71635] Updated weights for policy 1, policy_version 56282 (0.0009) [2023-10-11 21:18:26,016][71601] Updated weights for policy 0, policy_version 56330 (0.0007) [2023-10-11 21:18:26,034][70582] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 115310592. Throughput: 0: 1830.6, 1: 1844.1. Samples: 28841906. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-11 21:18:26,035][70582] Avg episode reward: [(0, '145.870'), (1, '52.860')] [2023-10-11 21:18:26,394][71601] Updated weights for policy 0, policy_version 56340 (0.0007) [2023-10-11 21:18:26,768][71601] Updated weights for policy 0, policy_version 56350 (0.0009) [2023-10-11 21:18:27,117][71635] Updated weights for policy 1, policy_version 56292 (0.0008) [2023-10-11 21:18:27,487][71635] Updated weights for policy 1, policy_version 56302 (0.0009) [2023-10-11 21:18:27,861][71635] Updated weights for policy 1, policy_version 56312 (0.0007) [2023-10-11 21:18:30,429][71601] Updated weights for policy 0, policy_version 56360 (0.0007) [2023-10-11 21:18:30,799][71601] Updated weights for policy 0, policy_version 56370 (0.0008) [2023-10-11 21:18:31,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 115376128. Throughput: 0: 1836.9, 1: 1840.0. Samples: 28851998. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-11 21:18:31,034][70582] Avg episode reward: [(0, '143.170'), (1, '54.870')] [2023-10-11 21:18:31,170][71601] Updated weights for policy 0, policy_version 56380 (0.0009) [2023-10-11 21:18:31,420][71635] Updated weights for policy 1, policy_version 56322 (0.0008) [2023-10-11 21:18:31,794][71635] Updated weights for policy 1, policy_version 56332 (0.0009) [2023-10-11 21:18:32,153][71635] Updated weights for policy 1, policy_version 56342 (0.0008) [2023-10-11 21:18:32,525][71635] Updated weights for policy 1, policy_version 56352 (0.0008) [2023-10-11 21:18:34,752][71601] Updated weights for policy 0, policy_version 56390 (0.0008) [2023-10-11 21:18:35,122][71601] Updated weights for policy 0, policy_version 56400 (0.0007) [2023-10-11 21:18:35,491][71601] Updated weights for policy 0, policy_version 56410 (0.0007) [2023-10-11 21:18:36,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 115474432. Throughput: 0: 1843.4, 1: 1842.9. Samples: 28875110. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-11 21:18:36,034][70582] Avg episode reward: [(0, '143.890'), (1, '57.810')] [2023-10-11 21:18:36,276][71635] Updated weights for policy 1, policy_version 56362 (0.0009) [2023-10-11 21:18:36,642][71635] Updated weights for policy 1, policy_version 56372 (0.0010) [2023-10-11 21:18:37,016][71635] Updated weights for policy 1, policy_version 56382 (0.0010) [2023-10-11 21:18:39,113][71601] Updated weights for policy 0, policy_version 56420 (0.0008) [2023-10-11 21:18:39,488][71601] Updated weights for policy 0, policy_version 56430 (0.0008) [2023-10-11 21:18:39,857][71601] Updated weights for policy 0, policy_version 56440 (0.0008) [2023-10-11 21:18:40,848][71635] Updated weights for policy 1, policy_version 56392 (0.0008) [2023-10-11 21:18:41,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 115539968. Throughput: 0: 1835.7, 1: 1837.5. Samples: 28896600. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:18:41,034][70582] Avg episode reward: [(0, '149.960'), (1, '57.700')] [2023-10-11 21:18:41,211][71635] Updated weights for policy 1, policy_version 56402 (0.0009) [2023-10-11 21:18:41,572][71635] Updated weights for policy 1, policy_version 56412 (0.0008) [2023-10-11 21:18:43,578][71601] Updated weights for policy 0, policy_version 56450 (0.0008) [2023-10-11 21:18:43,953][71601] Updated weights for policy 0, policy_version 56460 (0.0008) [2023-10-11 21:18:44,323][71601] Updated weights for policy 0, policy_version 56470 (0.0008) [2023-10-11 21:18:44,686][71601] Updated weights for policy 0, policy_version 56480 (0.0007) [2023-10-11 21:18:45,155][71635] Updated weights for policy 1, policy_version 56422 (0.0008) [2023-10-11 21:18:45,521][71635] Updated weights for policy 1, policy_version 56432 (0.0008) [2023-10-11 21:18:45,888][71635] Updated weights for policy 1, policy_version 56442 (0.0008) [2023-10-11 21:18:46,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 115605504. Throughput: 0: 1840.9, 1: 1832.3. Samples: 28908232. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:18:46,034][70582] Avg episode reward: [(0, '154.980'), (1, '57.950')] [2023-10-11 21:18:48,322][71601] Updated weights for policy 0, policy_version 56490 (0.0007) [2023-10-11 21:18:48,702][71601] Updated weights for policy 0, policy_version 56500 (0.0009) [2023-10-11 21:18:49,072][71601] Updated weights for policy 0, policy_version 56510 (0.0008) [2023-10-11 21:18:49,655][71635] Updated weights for policy 1, policy_version 56452 (0.0010) [2023-10-11 21:18:50,029][71635] Updated weights for policy 1, policy_version 56462 (0.0009) [2023-10-11 21:18:50,401][71635] Updated weights for policy 1, policy_version 56472 (0.0008) [2023-10-11 21:18:51,034][70582] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 115703808. Throughput: 0: 1832.2, 1: 1832.3. Samples: 28929558. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:18:51,035][70582] Avg episode reward: [(0, '140.800'), (1, '53.830')] [2023-10-11 21:18:52,747][71601] Updated weights for policy 0, policy_version 56520 (0.0010) [2023-10-11 21:18:53,112][71601] Updated weights for policy 0, policy_version 56530 (0.0009) [2023-10-11 21:18:53,476][71601] Updated weights for policy 0, policy_version 56540 (0.0007) [2023-10-11 21:18:54,029][71635] Updated weights for policy 1, policy_version 56482 (0.0008) [2023-10-11 21:18:54,403][71635] Updated weights for policy 1, policy_version 56492 (0.0008) [2023-10-11 21:18:54,768][71635] Updated weights for policy 1, policy_version 56502 (0.0009) [2023-10-11 21:18:55,132][71635] Updated weights for policy 1, policy_version 56512 (0.0008) [2023-10-11 21:18:56,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 115769344. Throughput: 0: 1830.2, 1: 1831.2. Samples: 28951102. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:18:56,035][70582] Avg episode reward: [(0, '124.960'), (1, '52.700')] [2023-10-11 21:18:57,285][71601] Updated weights for policy 0, policy_version 56550 (0.0009) [2023-10-11 21:18:57,660][71601] Updated weights for policy 0, policy_version 56560 (0.0010) [2023-10-11 21:18:58,020][71601] Updated weights for policy 0, policy_version 56570 (0.0007) [2023-10-11 21:18:58,857][71635] Updated weights for policy 1, policy_version 56522 (0.0008) [2023-10-11 21:18:59,222][71635] Updated weights for policy 1, policy_version 56532 (0.0008) [2023-10-11 21:18:59,588][71635] Updated weights for policy 1, policy_version 56542 (0.0008) [2023-10-11 21:19:01,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 115834880. Throughput: 0: 1823.8, 1: 1834.0. Samples: 28962402. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:19:01,034][70582] Avg episode reward: [(0, '110.780'), (1, '55.370')] [2023-10-11 21:19:01,621][71601] Updated weights for policy 0, policy_version 56580 (0.0009) [2023-10-11 21:19:01,996][71601] Updated weights for policy 0, policy_version 56590 (0.0007) [2023-10-11 21:19:02,360][71601] Updated weights for policy 0, policy_version 56600 (0.0007) [2023-10-11 21:19:03,085][71635] Updated weights for policy 1, policy_version 56552 (0.0008) [2023-10-11 21:19:03,450][71635] Updated weights for policy 1, policy_version 56562 (0.0007) [2023-10-11 21:19:03,818][71635] Updated weights for policy 1, policy_version 56572 (0.0007) [2023-10-11 21:19:05,953][71601] Updated weights for policy 0, policy_version 56610 (0.0009) [2023-10-11 21:19:06,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 115900416. Throughput: 0: 1827.6, 1: 1828.6. Samples: 28983906. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:19:06,034][70582] Avg episode reward: [(0, '105.720'), (1, '52.400')] [2023-10-11 21:19:06,333][71601] Updated weights for policy 0, policy_version 56620 (0.0009) [2023-10-11 21:19:06,697][71601] Updated weights for policy 0, policy_version 56630 (0.0007) [2023-10-11 21:19:07,068][71601] Updated weights for policy 0, policy_version 56640 (0.0008) [2023-10-11 21:19:07,398][71635] Updated weights for policy 1, policy_version 56582 (0.0008) [2023-10-11 21:19:07,763][71635] Updated weights for policy 1, policy_version 56592 (0.0007) [2023-10-11 21:19:08,129][71635] Updated weights for policy 1, policy_version 56602 (0.0008) [2023-10-11 21:19:10,759][71601] Updated weights for policy 0, policy_version 56650 (0.0008) [2023-10-11 21:19:11,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 115965952. Throughput: 0: 1830.6, 1: 1833.2. Samples: 29006780. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:19:11,035][70582] Avg episode reward: [(0, '93.540'), (1, '49.910')] [2023-10-11 21:19:11,137][71601] Updated weights for policy 0, policy_version 56660 (0.0007) [2023-10-11 21:19:11,504][71601] Updated weights for policy 0, policy_version 56670 (0.0007) [2023-10-11 21:19:11,856][71635] Updated weights for policy 1, policy_version 56612 (0.0010) [2023-10-11 21:19:12,218][71635] Updated weights for policy 1, policy_version 56622 (0.0008) [2023-10-11 21:19:12,593][71635] Updated weights for policy 1, policy_version 56632 (0.0010) [2023-10-11 21:19:15,247][71601] Updated weights for policy 0, policy_version 56680 (0.0007) [2023-10-11 21:19:15,621][71601] Updated weights for policy 0, policy_version 56690 (0.0007) [2023-10-11 21:19:15,999][71601] Updated weights for policy 0, policy_version 56700 (0.0010) [2023-10-11 21:19:16,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 116031488. Throughput: 0: 1830.6, 1: 1832.7. Samples: 29016846. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:19:16,035][70582] Avg episode reward: [(0, '85.920'), (1, '50.620')] [2023-10-11 21:19:16,379][71635] Updated weights for policy 1, policy_version 56642 (0.0009) [2023-10-11 21:19:16,748][71635] Updated weights for policy 1, policy_version 56652 (0.0007) [2023-10-11 21:19:17,107][71635] Updated weights for policy 1, policy_version 56662 (0.0009) [2023-10-11 21:19:17,477][71635] Updated weights for policy 1, policy_version 56672 (0.0007) [2023-10-11 21:19:19,693][71601] Updated weights for policy 0, policy_version 56710 (0.0010) [2023-10-11 21:19:20,065][71601] Updated weights for policy 0, policy_version 56720 (0.0010) [2023-10-11 21:19:20,425][71601] Updated weights for policy 0, policy_version 56730 (0.0009) [2023-10-11 21:19:21,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 116129792. Throughput: 0: 1825.4, 1: 1827.9. Samples: 29039508. Policy #0 lag: (min: 15.0, avg: 18.0, max: 47.0) [2023-10-11 21:19:21,035][70582] Avg episode reward: [(0, '78.180'), (1, '51.410')] [2023-10-11 21:19:21,264][71635] Updated weights for policy 1, policy_version 56682 (0.0007) [2023-10-11 21:19:21,633][71635] Updated weights for policy 1, policy_version 56692 (0.0008) [2023-10-11 21:19:22,003][71635] Updated weights for policy 1, policy_version 56702 (0.0008) [2023-10-11 21:19:24,151][71601] Updated weights for policy 0, policy_version 56740 (0.0008) [2023-10-11 21:19:24,520][71601] Updated weights for policy 0, policy_version 56750 (0.0007) [2023-10-11 21:19:24,883][71601] Updated weights for policy 0, policy_version 56760 (0.0007) [2023-10-11 21:19:25,768][71635] Updated weights for policy 1, policy_version 56712 (0.0009) [2023-10-11 21:19:26,034][70582] Fps is (10 sec: 16384.5, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 116195328. Throughput: 0: 1819.9, 1: 1824.5. Samples: 29060598. Policy #0 lag: (min: 15.0, avg: 18.0, max: 47.0) [2023-10-11 21:19:26,034][70582] Avg episode reward: [(0, '66.420'), (1, '54.030')] [2023-10-11 21:19:26,134][71635] Updated weights for policy 1, policy_version 56722 (0.0010) [2023-10-11 21:19:26,492][71635] Updated weights for policy 1, policy_version 56732 (0.0010) [2023-10-11 21:19:28,500][71601] Updated weights for policy 0, policy_version 56770 (0.0008) [2023-10-11 21:19:28,874][71601] Updated weights for policy 0, policy_version 56780 (0.0010) [2023-10-11 21:19:29,240][71601] Updated weights for policy 0, policy_version 56790 (0.0008) [2023-10-11 21:19:29,604][71601] Updated weights for policy 0, policy_version 56800 (0.0010) [2023-10-11 21:19:30,141][71635] Updated weights for policy 1, policy_version 56742 (0.0009) [2023-10-11 21:19:30,514][71635] Updated weights for policy 1, policy_version 56752 (0.0008) [2023-10-11 21:19:30,874][71635] Updated weights for policy 1, policy_version 56762 (0.0009) [2023-10-11 21:19:31,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 116260864. Throughput: 0: 1822.6, 1: 1821.9. Samples: 29072234. Policy #0 lag: (min: 15.0, avg: 18.0, max: 47.0) [2023-10-11 21:19:31,034][70582] Avg episode reward: [(0, '58.470'), (1, '56.340')] [2023-10-11 21:19:33,279][71601] Updated weights for policy 0, policy_version 56810 (0.0010) [2023-10-11 21:19:33,645][71601] Updated weights for policy 0, policy_version 56820 (0.0007) [2023-10-11 21:19:34,027][71601] Updated weights for policy 0, policy_version 56830 (0.0008) [2023-10-11 21:19:34,573][71635] Updated weights for policy 1, policy_version 56772 (0.0008) [2023-10-11 21:19:34,942][71635] Updated weights for policy 1, policy_version 56782 (0.0008) [2023-10-11 21:19:35,311][71635] Updated weights for policy 1, policy_version 56792 (0.0008) [2023-10-11 21:19:36,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 116359168. Throughput: 0: 1824.2, 1: 1818.4. Samples: 29093478. Policy #0 lag: (min: 15.0, avg: 18.0, max: 47.0) [2023-10-11 21:19:36,035][70582] Avg episode reward: [(0, '50.180'), (1, '54.820')] [2023-10-11 21:19:37,882][71601] Updated weights for policy 0, policy_version 56840 (0.0007) [2023-10-11 21:19:38,257][71601] Updated weights for policy 0, policy_version 56850 (0.0007) [2023-10-11 21:19:38,627][71601] Updated weights for policy 0, policy_version 56860 (0.0007) [2023-10-11 21:19:39,007][71635] Updated weights for policy 1, policy_version 56802 (0.0008) [2023-10-11 21:19:39,379][71635] Updated weights for policy 1, policy_version 56812 (0.0009) [2023-10-11 21:19:39,740][71635] Updated weights for policy 1, policy_version 56822 (0.0008) [2023-10-11 21:19:40,106][71635] Updated weights for policy 1, policy_version 56832 (0.0008) [2023-10-11 21:19:41,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 116424704. Throughput: 0: 1819.4, 1: 1821.0. Samples: 29114920. Policy #0 lag: (min: 15.0, avg: 18.0, max: 47.0) [2023-10-11 21:19:41,034][70582] Avg episode reward: [(0, '54.170'), (1, '54.600')] [2023-10-11 21:19:42,298][71601] Updated weights for policy 0, policy_version 56870 (0.0007) [2023-10-11 21:19:42,678][71601] Updated weights for policy 0, policy_version 56880 (0.0007) [2023-10-11 21:19:43,048][71601] Updated weights for policy 0, policy_version 56890 (0.0009) [2023-10-11 21:19:43,739][71635] Updated weights for policy 1, policy_version 56842 (0.0011) [2023-10-11 21:19:44,100][71635] Updated weights for policy 1, policy_version 56852 (0.0011) [2023-10-11 21:19:44,477][71635] Updated weights for policy 1, policy_version 56862 (0.0008) [2023-10-11 21:19:46,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 116490240. Throughput: 0: 1820.8, 1: 1820.1. Samples: 29126242. Policy #0 lag: (min: 15.0, avg: 18.0, max: 47.0) [2023-10-11 21:19:46,034][70582] Avg episode reward: [(0, '59.820'), (1, '59.440')] [2023-10-11 21:19:46,733][71601] Updated weights for policy 0, policy_version 56900 (0.0009) [2023-10-11 21:19:47,111][71601] Updated weights for policy 0, policy_version 56910 (0.0009) [2023-10-11 21:19:47,475][71601] Updated weights for policy 0, policy_version 56920 (0.0010) [2023-10-11 21:19:48,258][71635] Updated weights for policy 1, policy_version 56872 (0.0007) [2023-10-11 21:19:48,619][71635] Updated weights for policy 1, policy_version 56882 (0.0010) [2023-10-11 21:19:48,985][71635] Updated weights for policy 1, policy_version 56892 (0.0008) [2023-10-11 21:19:51,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 116555776. Throughput: 0: 1824.2, 1: 1814.7. Samples: 29147660. Policy #0 lag: (min: 15.0, avg: 18.0, max: 47.0) [2023-10-11 21:19:51,035][70582] Avg episode reward: [(0, '50.860'), (1, '61.590')] [2023-10-11 21:19:51,061][71601] Updated weights for policy 0, policy_version 56930 (0.0007) [2023-10-11 21:19:51,428][71601] Updated weights for policy 0, policy_version 56940 (0.0007) [2023-10-11 21:19:51,803][71601] Updated weights for policy 0, policy_version 56950 (0.0008) [2023-10-11 21:19:52,163][71601] Updated weights for policy 0, policy_version 56960 (0.0008) [2023-10-11 21:19:52,643][71635] Updated weights for policy 1, policy_version 56902 (0.0008) [2023-10-11 21:19:53,005][71635] Updated weights for policy 1, policy_version 56912 (0.0008) [2023-10-11 21:19:53,373][71635] Updated weights for policy 1, policy_version 56922 (0.0007) [2023-10-11 21:19:55,921][71601] Updated weights for policy 0, policy_version 56970 (0.0008) [2023-10-11 21:19:56,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 116621312. Throughput: 0: 1827.6, 1: 1819.3. Samples: 29170894. Policy #0 lag: (min: 15.0, avg: 18.0, max: 47.0) [2023-10-11 21:19:56,035][70582] Avg episode reward: [(0, '51.080'), (1, '64.000')] [2023-10-11 21:19:56,047][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000056928_58294272.pth... [2023-10-11 21:19:56,080][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000055232_56557568.pth [2023-10-11 21:19:56,304][71601] Updated weights for policy 0, policy_version 56980 (0.0007) [2023-10-11 21:19:56,684][71601] Updated weights for policy 0, policy_version 56990 (0.0008) [2023-10-11 21:19:56,758][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000056992_58359808.pth... [2023-10-11 21:19:56,791][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000055264_56590336.pth [2023-10-11 21:19:56,867][71635] Updated weights for policy 1, policy_version 56932 (0.0007) [2023-10-11 21:19:57,232][71635] Updated weights for policy 1, policy_version 56942 (0.0008) [2023-10-11 21:19:57,593][71635] Updated weights for policy 1, policy_version 56952 (0.0009) [2023-10-11 21:20:00,355][71601] Updated weights for policy 0, policy_version 57000 (0.0009) [2023-10-11 21:20:00,724][71601] Updated weights for policy 0, policy_version 57010 (0.0007) [2023-10-11 21:20:01,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 116686848. Throughput: 0: 1827.2, 1: 1820.2. Samples: 29180982. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-11 21:20:01,035][70582] Avg episode reward: [(0, '59.050'), (1, '61.430')] [2023-10-11 21:20:01,105][71601] Updated weights for policy 0, policy_version 57020 (0.0007) [2023-10-11 21:20:01,267][71635] Updated weights for policy 1, policy_version 56962 (0.0008) [2023-10-11 21:20:01,643][71635] Updated weights for policy 1, policy_version 56972 (0.0008) [2023-10-11 21:20:02,018][71635] Updated weights for policy 1, policy_version 56982 (0.0008) [2023-10-11 21:20:02,380][71635] Updated weights for policy 1, policy_version 56992 (0.0007) [2023-10-11 21:20:04,892][71601] Updated weights for policy 0, policy_version 57030 (0.0008) [2023-10-11 21:20:05,260][71601] Updated weights for policy 0, policy_version 57040 (0.0009) [2023-10-11 21:20:05,631][71601] Updated weights for policy 0, policy_version 57050 (0.0007) [2023-10-11 21:20:06,034][70582] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 116785152. Throughput: 0: 1818.6, 1: 1824.8. Samples: 29203460. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-11 21:20:06,034][70582] Avg episode reward: [(0, '56.990'), (1, '56.880')] [2023-10-11 21:20:06,079][71635] Updated weights for policy 1, policy_version 57002 (0.0009) [2023-10-11 21:20:06,457][71635] Updated weights for policy 1, policy_version 57012 (0.0008) [2023-10-11 21:20:06,816][71635] Updated weights for policy 1, policy_version 57022 (0.0008) [2023-10-11 21:20:09,058][71601] Updated weights for policy 0, policy_version 57060 (0.0008) [2023-10-11 21:20:09,431][71601] Updated weights for policy 0, policy_version 57070 (0.0008) [2023-10-11 21:20:09,792][71601] Updated weights for policy 0, policy_version 57080 (0.0008) [2023-10-11 21:20:10,459][71635] Updated weights for policy 1, policy_version 57032 (0.0009) [2023-10-11 21:20:10,828][71635] Updated weights for policy 1, policy_version 57042 (0.0010) [2023-10-11 21:20:11,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 116850688. Throughput: 0: 1825.2, 1: 1824.7. Samples: 29224844. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-11 21:20:11,035][70582] Avg episode reward: [(0, '54.040'), (1, '58.500')] [2023-10-11 21:20:11,192][71635] Updated weights for policy 1, policy_version 57052 (0.0009) [2023-10-11 21:20:13,596][71601] Updated weights for policy 0, policy_version 57090 (0.0008) [2023-10-11 21:20:13,977][71601] Updated weights for policy 0, policy_version 57100 (0.0010) [2023-10-11 21:20:14,348][71601] Updated weights for policy 0, policy_version 57110 (0.0007) [2023-10-11 21:20:14,724][71601] Updated weights for policy 0, policy_version 57120 (0.0007) [2023-10-11 21:20:15,067][71635] Updated weights for policy 1, policy_version 57062 (0.0009) [2023-10-11 21:20:15,438][71635] Updated weights for policy 1, policy_version 57072 (0.0008) [2023-10-11 21:20:15,808][71635] Updated weights for policy 1, policy_version 57082 (0.0008) [2023-10-11 21:20:16,034][70582] Fps is (10 sec: 16383.8, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 116948992. Throughput: 0: 1819.0, 1: 1827.8. Samples: 29236342. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-11 21:20:16,035][70582] Avg episode reward: [(0, '58.930'), (1, '57.550')] [2023-10-11 21:20:18,340][71601] Updated weights for policy 0, policy_version 57130 (0.0008) [2023-10-11 21:20:18,710][71601] Updated weights for policy 0, policy_version 57140 (0.0007) [2023-10-11 21:20:19,076][71601] Updated weights for policy 0, policy_version 57150 (0.0008) [2023-10-11 21:20:19,600][71635] Updated weights for policy 1, policy_version 57092 (0.0010) [2023-10-11 21:20:19,963][71635] Updated weights for policy 1, policy_version 57102 (0.0010) [2023-10-11 21:20:20,330][71635] Updated weights for policy 1, policy_version 57112 (0.0010) [2023-10-11 21:20:21,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 117014528. Throughput: 0: 1818.9, 1: 1826.9. Samples: 29257540. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-11 21:20:21,035][70582] Avg episode reward: [(0, '66.300'), (1, '53.210')] [2023-10-11 21:20:22,772][71601] Updated weights for policy 0, policy_version 57160 (0.0007) [2023-10-11 21:20:23,142][71601] Updated weights for policy 0, policy_version 57170 (0.0008) [2023-10-11 21:20:23,517][71601] Updated weights for policy 0, policy_version 57180 (0.0008) [2023-10-11 21:20:24,233][71635] Updated weights for policy 1, policy_version 57122 (0.0010) [2023-10-11 21:20:24,593][71635] Updated weights for policy 1, policy_version 57132 (0.0008) [2023-10-11 21:20:24,963][71635] Updated weights for policy 1, policy_version 57142 (0.0009) [2023-10-11 21:20:25,326][71635] Updated weights for policy 1, policy_version 57152 (0.0009) [2023-10-11 21:20:26,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 117080064. Throughput: 0: 1823.9, 1: 1820.0. Samples: 29278896. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-11 21:20:26,035][70582] Avg episode reward: [(0, '65.940'), (1, '52.760')] [2023-10-11 21:20:27,201][71601] Updated weights for policy 0, policy_version 57190 (0.0009) [2023-10-11 21:20:27,579][71601] Updated weights for policy 0, policy_version 57200 (0.0008) [2023-10-11 21:20:27,947][71601] Updated weights for policy 0, policy_version 57210 (0.0007) [2023-10-11 21:20:28,775][71635] Updated weights for policy 1, policy_version 57162 (0.0008) [2023-10-11 21:20:29,149][71635] Updated weights for policy 1, policy_version 57172 (0.0008) [2023-10-11 21:20:29,505][71635] Updated weights for policy 1, policy_version 57182 (0.0008) [2023-10-11 21:20:31,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 117145600. Throughput: 0: 1826.3, 1: 1818.1. Samples: 29290240. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-11 21:20:31,035][70582] Avg episode reward: [(0, '66.870'), (1, '50.130')] [2023-10-11 21:20:31,622][71601] Updated weights for policy 0, policy_version 57220 (0.0007) [2023-10-11 21:20:31,986][71601] Updated weights for policy 0, policy_version 57230 (0.0007) [2023-10-11 21:20:32,349][71601] Updated weights for policy 0, policy_version 57240 (0.0007) [2023-10-11 21:20:33,149][71635] Updated weights for policy 1, policy_version 57192 (0.0009) [2023-10-11 21:20:33,505][71635] Updated weights for policy 1, policy_version 57202 (0.0008) [2023-10-11 21:20:33,868][71635] Updated weights for policy 1, policy_version 57212 (0.0008) [2023-10-11 21:20:35,982][71601] Updated weights for policy 0, policy_version 57250 (0.0011) [2023-10-11 21:20:36,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 117211136. Throughput: 0: 1822.5, 1: 1819.5. Samples: 29311550. Policy #0 lag: (min: 13.0, avg: 20.0, max: 45.0) [2023-10-11 21:20:36,034][70582] Avg episode reward: [(0, '69.910'), (1, '52.920')] [2023-10-11 21:20:36,348][71601] Updated weights for policy 0, policy_version 57260 (0.0009) [2023-10-11 21:20:36,721][71601] Updated weights for policy 0, policy_version 57270 (0.0011) [2023-10-11 21:20:37,091][71601] Updated weights for policy 0, policy_version 57280 (0.0010) [2023-10-11 21:20:37,283][71635] Updated weights for policy 1, policy_version 57222 (0.0008) [2023-10-11 21:20:37,652][71635] Updated weights for policy 1, policy_version 57232 (0.0011) [2023-10-11 21:20:38,021][71635] Updated weights for policy 1, policy_version 57242 (0.0009) [2023-10-11 21:20:40,942][71601] Updated weights for policy 0, policy_version 57290 (0.0011) [2023-10-11 21:20:41,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 117276672. Throughput: 0: 1811.9, 1: 1817.3. Samples: 29334206. Policy #0 lag: (min: 13.0, avg: 20.0, max: 45.0) [2023-10-11 21:20:41,034][70582] Avg episode reward: [(0, '64.860'), (1, '54.920')] [2023-10-11 21:20:41,307][71601] Updated weights for policy 0, policy_version 57300 (0.0010) [2023-10-11 21:20:41,677][71601] Updated weights for policy 0, policy_version 57310 (0.0010) [2023-10-11 21:20:41,818][71635] Updated weights for policy 1, policy_version 57252 (0.0007) [2023-10-11 21:20:42,192][71635] Updated weights for policy 1, policy_version 57262 (0.0008) [2023-10-11 21:20:42,558][71635] Updated weights for policy 1, policy_version 57272 (0.0010) [2023-10-11 21:20:45,415][71601] Updated weights for policy 0, policy_version 57320 (0.0008) [2023-10-11 21:20:45,778][71601] Updated weights for policy 0, policy_version 57330 (0.0007) [2023-10-11 21:20:46,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 117342208. Throughput: 0: 1807.6, 1: 1816.9. Samples: 29344084. Policy #0 lag: (min: 13.0, avg: 20.0, max: 45.0) [2023-10-11 21:20:46,034][70582] Avg episode reward: [(0, '62.250'), (1, '53.800')] [2023-10-11 21:20:46,156][71601] Updated weights for policy 0, policy_version 57340 (0.0007) [2023-10-11 21:20:46,236][71635] Updated weights for policy 1, policy_version 57282 (0.0009) [2023-10-11 21:20:46,606][71635] Updated weights for policy 1, policy_version 57292 (0.0008) [2023-10-11 21:20:46,972][71635] Updated weights for policy 1, policy_version 57302 (0.0008) [2023-10-11 21:20:47,344][71635] Updated weights for policy 1, policy_version 57312 (0.0010) [2023-10-11 21:20:49,938][71601] Updated weights for policy 0, policy_version 57350 (0.0008) [2023-10-11 21:20:50,311][71601] Updated weights for policy 0, policy_version 57360 (0.0007) [2023-10-11 21:20:50,691][71601] Updated weights for policy 0, policy_version 57370 (0.0010) [2023-10-11 21:20:51,034][70582] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 117440512. Throughput: 0: 1812.0, 1: 1814.3. Samples: 29366642. Policy #0 lag: (min: 13.0, avg: 20.0, max: 45.0) [2023-10-11 21:20:51,035][70582] Avg episode reward: [(0, '62.350'), (1, '55.310')] [2023-10-11 21:20:51,097][71635] Updated weights for policy 1, policy_version 57322 (0.0007) [2023-10-11 21:20:51,467][71635] Updated weights for policy 1, policy_version 57332 (0.0007) [2023-10-11 21:20:51,829][71635] Updated weights for policy 1, policy_version 57342 (0.0010) [2023-10-11 21:20:54,329][71601] Updated weights for policy 0, policy_version 57380 (0.0009) [2023-10-11 21:20:54,702][71601] Updated weights for policy 0, policy_version 57390 (0.0007) [2023-10-11 21:20:55,073][71601] Updated weights for policy 0, policy_version 57400 (0.0008) [2023-10-11 21:20:55,433][71635] Updated weights for policy 1, policy_version 57352 (0.0007) [2023-10-11 21:20:55,798][71635] Updated weights for policy 1, policy_version 57362 (0.0008) [2023-10-11 21:20:56,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 117506048. Throughput: 0: 1807.7, 1: 1810.0. Samples: 29387642. Policy #0 lag: (min: 13.0, avg: 20.0, max: 45.0) [2023-10-11 21:20:56,034][70582] Avg episode reward: [(0, '64.590'), (1, '55.910')] [2023-10-11 21:20:56,165][71635] Updated weights for policy 1, policy_version 57372 (0.0007) [2023-10-11 21:20:58,868][71601] Updated weights for policy 0, policy_version 57410 (0.0009) [2023-10-11 21:20:59,234][71601] Updated weights for policy 0, policy_version 57420 (0.0010) [2023-10-11 21:20:59,613][71601] Updated weights for policy 0, policy_version 57430 (0.0010) [2023-10-11 21:20:59,916][71635] Updated weights for policy 1, policy_version 57382 (0.0008) [2023-10-11 21:20:59,981][71601] Updated weights for policy 0, policy_version 57440 (0.0009) [2023-10-11 21:21:00,281][71635] Updated weights for policy 1, policy_version 57392 (0.0009) [2023-10-11 21:21:00,651][71635] Updated weights for policy 1, policy_version 57402 (0.0008) [2023-10-11 21:21:01,034][70582] Fps is (10 sec: 16384.5, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 117604352. Throughput: 0: 1810.3, 1: 1818.4. Samples: 29399634. Policy #0 lag: (min: 13.0, avg: 20.0, max: 45.0) [2023-10-11 21:21:01,034][70582] Avg episode reward: [(0, '57.000'), (1, '57.840')] [2023-10-11 21:21:03,560][71601] Updated weights for policy 0, policy_version 57450 (0.0009) [2023-10-11 21:21:03,931][71601] Updated weights for policy 0, policy_version 57460 (0.0008) [2023-10-11 21:21:04,299][71601] Updated weights for policy 0, policy_version 57470 (0.0010) [2023-10-11 21:21:04,456][71635] Updated weights for policy 1, policy_version 57412 (0.0008) [2023-10-11 21:21:04,829][71635] Updated weights for policy 1, policy_version 57422 (0.0007) [2023-10-11 21:21:05,195][71635] Updated weights for policy 1, policy_version 57432 (0.0007) [2023-10-11 21:21:06,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 117669888. Throughput: 0: 1811.2, 1: 1818.2. Samples: 29420866. Policy #0 lag: (min: 13.0, avg: 20.0, max: 45.0) [2023-10-11 21:21:06,035][70582] Avg episode reward: [(0, '53.960'), (1, '56.240')] [2023-10-11 21:21:07,845][71601] Updated weights for policy 0, policy_version 57480 (0.0010) [2023-10-11 21:21:08,222][71601] Updated weights for policy 0, policy_version 57490 (0.0009) [2023-10-11 21:21:08,588][71601] Updated weights for policy 0, policy_version 57500 (0.0010) [2023-10-11 21:21:08,804][71635] Updated weights for policy 1, policy_version 57442 (0.0009) [2023-10-11 21:21:09,168][71635] Updated weights for policy 1, policy_version 57452 (0.0009) [2023-10-11 21:21:09,537][71635] Updated weights for policy 1, policy_version 57462 (0.0012) [2023-10-11 21:21:09,910][71635] Updated weights for policy 1, policy_version 57472 (0.0009) [2023-10-11 21:21:11,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 117735424. Throughput: 0: 1811.8, 1: 1818.0. Samples: 29442238. Policy #0 lag: (min: 31.0, avg: 31.7, max: 48.0) [2023-10-11 21:21:11,035][70582] Avg episode reward: [(0, '56.660'), (1, '59.260')] [2023-10-11 21:21:12,295][71601] Updated weights for policy 0, policy_version 57510 (0.0009) [2023-10-11 21:21:12,667][71601] Updated weights for policy 0, policy_version 57520 (0.0008) [2023-10-11 21:21:13,041][71601] Updated weights for policy 0, policy_version 57530 (0.0010) [2023-10-11 21:21:13,536][71635] Updated weights for policy 1, policy_version 57482 (0.0010) [2023-10-11 21:21:13,906][71635] Updated weights for policy 1, policy_version 57492 (0.0009) [2023-10-11 21:21:14,277][71635] Updated weights for policy 1, policy_version 57502 (0.0009) [2023-10-11 21:21:16,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 117800960. Throughput: 0: 1813.4, 1: 1814.4. Samples: 29453490. Policy #0 lag: (min: 31.0, avg: 31.7, max: 48.0) [2023-10-11 21:21:16,035][70582] Avg episode reward: [(0, '52.530'), (1, '61.500')] [2023-10-11 21:21:16,583][71601] Updated weights for policy 0, policy_version 57540 (0.0009) [2023-10-11 21:21:16,956][71601] Updated weights for policy 0, policy_version 57550 (0.0009) [2023-10-11 21:21:17,323][71601] Updated weights for policy 0, policy_version 57560 (0.0007) [2023-10-11 21:21:18,182][71635] Updated weights for policy 1, policy_version 57512 (0.0007) [2023-10-11 21:21:18,559][71635] Updated weights for policy 1, policy_version 57522 (0.0008) [2023-10-11 21:21:18,934][71635] Updated weights for policy 1, policy_version 57532 (0.0007) [2023-10-11 21:21:20,955][71601] Updated weights for policy 0, policy_version 57570 (0.0007) [2023-10-11 21:21:21,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 117866496. Throughput: 0: 1816.3, 1: 1822.1. Samples: 29475278. Policy #0 lag: (min: 31.0, avg: 31.7, max: 48.0) [2023-10-11 21:21:21,035][70582] Avg episode reward: [(0, '52.650'), (1, '64.120')] [2023-10-11 21:21:21,337][71601] Updated weights for policy 0, policy_version 57580 (0.0009) [2023-10-11 21:21:21,704][71601] Updated weights for policy 0, policy_version 57590 (0.0009) [2023-10-11 21:21:22,078][71601] Updated weights for policy 0, policy_version 57600 (0.0008) [2023-10-11 21:21:22,728][71635] Updated weights for policy 1, policy_version 57542 (0.0007) [2023-10-11 21:21:23,089][71635] Updated weights for policy 1, policy_version 57552 (0.0008) [2023-10-11 21:21:23,455][71635] Updated weights for policy 1, policy_version 57562 (0.0008) [2023-10-11 21:21:25,645][71601] Updated weights for policy 0, policy_version 57610 (0.0008) [2023-10-11 21:21:26,010][71601] Updated weights for policy 0, policy_version 57620 (0.0008) [2023-10-11 21:21:26,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 117932032. Throughput: 0: 1822.1, 1: 1816.2. Samples: 29497930. Policy #0 lag: (min: 31.0, avg: 31.7, max: 48.0) [2023-10-11 21:21:26,035][70582] Avg episode reward: [(0, '52.080'), (1, '62.270')] [2023-10-11 21:21:26,378][71601] Updated weights for policy 0, policy_version 57630 (0.0008) [2023-10-11 21:21:27,180][71635] Updated weights for policy 1, policy_version 57572 (0.0009) [2023-10-11 21:21:27,545][71635] Updated weights for policy 1, policy_version 57582 (0.0010) [2023-10-11 21:21:27,910][71635] Updated weights for policy 1, policy_version 57592 (0.0009) [2023-10-11 21:21:30,053][71601] Updated weights for policy 0, policy_version 57640 (0.0009) [2023-10-11 21:21:30,429][71601] Updated weights for policy 0, policy_version 57650 (0.0009) [2023-10-11 21:21:30,805][71601] Updated weights for policy 0, policy_version 57660 (0.0009) [2023-10-11 21:21:31,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 118030336. Throughput: 0: 1826.8, 1: 1817.3. Samples: 29508068. Policy #0 lag: (min: 31.0, avg: 31.7, max: 48.0) [2023-10-11 21:21:31,035][70582] Avg episode reward: [(0, '51.680'), (1, '62.340')] [2023-10-11 21:21:31,600][71635] Updated weights for policy 1, policy_version 57602 (0.0008) [2023-10-11 21:21:31,968][71635] Updated weights for policy 1, policy_version 57612 (0.0007) [2023-10-11 21:21:32,334][71635] Updated weights for policy 1, policy_version 57622 (0.0008) [2023-10-11 21:21:32,699][71635] Updated weights for policy 1, policy_version 57632 (0.0010) [2023-10-11 21:21:34,533][71601] Updated weights for policy 0, policy_version 57670 (0.0009) [2023-10-11 21:21:34,897][71601] Updated weights for policy 0, policy_version 57680 (0.0009) [2023-10-11 21:21:35,269][71601] Updated weights for policy 0, policy_version 57690 (0.0007) [2023-10-11 21:21:36,034][70582] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 118095872. Throughput: 0: 1829.4, 1: 1817.1. Samples: 29530734. Policy #0 lag: (min: 31.0, avg: 31.7, max: 48.0) [2023-10-11 21:21:36,034][70582] Avg episode reward: [(0, '55.160'), (1, '63.560')] [2023-10-11 21:21:36,384][71635] Updated weights for policy 1, policy_version 57642 (0.0008) [2023-10-11 21:21:36,740][71635] Updated weights for policy 1, policy_version 57652 (0.0010) [2023-10-11 21:21:37,105][71635] Updated weights for policy 1, policy_version 57662 (0.0011) [2023-10-11 21:21:38,908][71601] Updated weights for policy 0, policy_version 57700 (0.0007) [2023-10-11 21:21:39,275][71601] Updated weights for policy 0, policy_version 57710 (0.0009) [2023-10-11 21:21:39,655][71601] Updated weights for policy 0, policy_version 57720 (0.0008) [2023-10-11 21:21:40,723][71635] Updated weights for policy 1, policy_version 57672 (0.0009) [2023-10-11 21:21:41,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 118161408. Throughput: 0: 1830.6, 1: 1828.0. Samples: 29552280. Policy #0 lag: (min: 31.0, avg: 31.7, max: 48.0) [2023-10-11 21:21:41,034][70582] Avg episode reward: [(0, '54.340'), (1, '67.330')] [2023-10-11 21:21:41,084][71635] Updated weights for policy 1, policy_version 57682 (0.0008) [2023-10-11 21:21:41,448][71635] Updated weights for policy 1, policy_version 57692 (0.0009) [2023-10-11 21:21:43,283][71601] Updated weights for policy 0, policy_version 57730 (0.0009) [2023-10-11 21:21:43,652][71601] Updated weights for policy 0, policy_version 57740 (0.0008) [2023-10-11 21:21:44,019][71601] Updated weights for policy 0, policy_version 57750 (0.0008) [2023-10-11 21:21:44,384][71601] Updated weights for policy 0, policy_version 57760 (0.0010) [2023-10-11 21:21:45,249][71635] Updated weights for policy 1, policy_version 57702 (0.0009) [2023-10-11 21:21:45,622][71635] Updated weights for policy 1, policy_version 57712 (0.0009) [2023-10-11 21:21:45,998][71635] Updated weights for policy 1, policy_version 57722 (0.0008) [2023-10-11 21:21:46,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 118226944. Throughput: 0: 1827.9, 1: 1815.5. Samples: 29563588. Policy #0 lag: (min: 31.0, avg: 31.7, max: 48.0) [2023-10-11 21:21:46,035][70582] Avg episode reward: [(0, '53.110'), (1, '69.540')] [2023-10-11 21:21:48,036][71601] Updated weights for policy 0, policy_version 57770 (0.0009) [2023-10-11 21:21:48,413][71601] Updated weights for policy 0, policy_version 57780 (0.0009) [2023-10-11 21:21:48,774][71601] Updated weights for policy 0, policy_version 57790 (0.0009) [2023-10-11 21:21:49,659][71635] Updated weights for policy 1, policy_version 57732 (0.0010) [2023-10-11 21:21:50,024][71635] Updated weights for policy 1, policy_version 57742 (0.0009) [2023-10-11 21:21:50,398][71635] Updated weights for policy 1, policy_version 57752 (0.0009) [2023-10-11 21:21:51,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 118325248. Throughput: 0: 1834.2, 1: 1817.1. Samples: 29585174. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:21:51,035][70582] Avg episode reward: [(0, '58.790'), (1, '65.100')] [2023-10-11 21:21:52,470][71601] Updated weights for policy 0, policy_version 57800 (0.0009) [2023-10-11 21:21:52,839][71601] Updated weights for policy 0, policy_version 57810 (0.0009) [2023-10-11 21:21:53,218][71601] Updated weights for policy 0, policy_version 57820 (0.0008) [2023-10-11 21:21:54,087][71635] Updated weights for policy 1, policy_version 57762 (0.0007) [2023-10-11 21:21:54,451][71635] Updated weights for policy 1, policy_version 57772 (0.0007) [2023-10-11 21:21:54,825][71635] Updated weights for policy 1, policy_version 57782 (0.0008) [2023-10-11 21:21:55,187][71635] Updated weights for policy 1, policy_version 57792 (0.0008) [2023-10-11 21:21:56,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 118390784. Throughput: 0: 1831.8, 1: 1821.4. Samples: 29606632. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:21:56,035][70582] Avg episode reward: [(0, '57.070'), (1, '64.930')] [2023-10-11 21:21:56,044][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000057824_59211776.pth... [2023-10-11 21:21:56,044][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000057792_59179008.pth... [2023-10-11 21:21:56,077][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000056096_57442304.pth [2023-10-11 21:21:56,087][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000056128_57475072.pth [2023-10-11 21:21:57,043][71601] Updated weights for policy 0, policy_version 57830 (0.0008) [2023-10-11 21:21:57,417][71601] Updated weights for policy 0, policy_version 57840 (0.0009) [2023-10-11 21:21:57,784][71601] Updated weights for policy 0, policy_version 57850 (0.0010) [2023-10-11 21:21:58,845][71635] Updated weights for policy 1, policy_version 57802 (0.0009) [2023-10-11 21:21:59,217][71635] Updated weights for policy 1, policy_version 57812 (0.0008) [2023-10-11 21:21:59,585][71635] Updated weights for policy 1, policy_version 57822 (0.0008) [2023-10-11 21:22:01,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 118456320. Throughput: 0: 1826.2, 1: 1824.9. Samples: 29617788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:22:01,034][70582] Avg episode reward: [(0, '59.970'), (1, '64.780')] [2023-10-11 21:22:01,435][71601] Updated weights for policy 0, policy_version 57860 (0.0007) [2023-10-11 21:22:01,798][71601] Updated weights for policy 0, policy_version 57870 (0.0007) [2023-10-11 21:22:02,168][71601] Updated weights for policy 0, policy_version 57880 (0.0007) [2023-10-11 21:22:03,288][71635] Updated weights for policy 1, policy_version 57832 (0.0007) [2023-10-11 21:22:03,648][71635] Updated weights for policy 1, policy_version 57842 (0.0007) [2023-10-11 21:22:04,012][71635] Updated weights for policy 1, policy_version 57852 (0.0007) [2023-10-11 21:22:05,825][71601] Updated weights for policy 0, policy_version 57890 (0.0008) [2023-10-11 21:22:06,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 118521856. Throughput: 0: 1826.7, 1: 1816.7. Samples: 29639230. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:22:06,034][70582] Avg episode reward: [(0, '62.550'), (1, '69.920')] [2023-10-11 21:22:06,199][71601] Updated weights for policy 0, policy_version 57900 (0.0007) [2023-10-11 21:22:06,574][71601] Updated weights for policy 0, policy_version 57910 (0.0008) [2023-10-11 21:22:06,931][71601] Updated weights for policy 0, policy_version 57920 (0.0007) [2023-10-11 21:22:07,666][71635] Updated weights for policy 1, policy_version 57862 (0.0008) [2023-10-11 21:22:08,022][71635] Updated weights for policy 1, policy_version 57872 (0.0009) [2023-10-11 21:22:08,387][71635] Updated weights for policy 1, policy_version 57882 (0.0008) [2023-10-11 21:22:10,587][71601] Updated weights for policy 0, policy_version 57930 (0.0008) [2023-10-11 21:22:10,958][71601] Updated weights for policy 0, policy_version 57940 (0.0008) [2023-10-11 21:22:11,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 118587392. Throughput: 0: 1825.1, 1: 1817.3. Samples: 29661836. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:22:11,034][70582] Avg episode reward: [(0, '67.850'), (1, '70.880')] [2023-10-11 21:22:11,333][71601] Updated weights for policy 0, policy_version 57950 (0.0008) [2023-10-11 21:22:12,214][71635] Updated weights for policy 1, policy_version 57892 (0.0009) [2023-10-11 21:22:12,578][71635] Updated weights for policy 1, policy_version 57902 (0.0008) [2023-10-11 21:22:12,939][71635] Updated weights for policy 1, policy_version 57912 (0.0007) [2023-10-11 21:22:15,116][71601] Updated weights for policy 0, policy_version 57960 (0.0008) [2023-10-11 21:22:15,487][71601] Updated weights for policy 0, policy_version 57970 (0.0007) [2023-10-11 21:22:15,862][71601] Updated weights for policy 0, policy_version 57980 (0.0007) [2023-10-11 21:22:16,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 118685696. Throughput: 0: 1830.7, 1: 1811.4. Samples: 29671964. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:22:16,034][70582] Avg episode reward: [(0, '62.730'), (1, '67.630')] [2023-10-11 21:22:16,546][71635] Updated weights for policy 1, policy_version 57922 (0.0008) [2023-10-11 21:22:16,907][71635] Updated weights for policy 1, policy_version 57932 (0.0008) [2023-10-11 21:22:17,272][71635] Updated weights for policy 1, policy_version 57942 (0.0007) [2023-10-11 21:22:17,652][71635] Updated weights for policy 1, policy_version 57952 (0.0008) [2023-10-11 21:22:19,540][71601] Updated weights for policy 0, policy_version 57990 (0.0010) [2023-10-11 21:22:19,914][71601] Updated weights for policy 0, policy_version 58000 (0.0010) [2023-10-11 21:22:20,291][71601] Updated weights for policy 0, policy_version 58010 (0.0007) [2023-10-11 21:22:21,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 118751232. Throughput: 0: 1828.7, 1: 1812.5. Samples: 29694586. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:22:21,034][70582] Avg episode reward: [(0, '59.370'), (1, '63.520')] [2023-10-11 21:22:21,372][71635] Updated weights for policy 1, policy_version 57962 (0.0008) [2023-10-11 21:22:21,745][71635] Updated weights for policy 1, policy_version 57972 (0.0008) [2023-10-11 21:22:22,106][71635] Updated weights for policy 1, policy_version 57982 (0.0008) [2023-10-11 21:22:23,920][71601] Updated weights for policy 0, policy_version 58020 (0.0009) [2023-10-11 21:22:24,288][71601] Updated weights for policy 0, policy_version 58030 (0.0007) [2023-10-11 21:22:24,660][71601] Updated weights for policy 0, policy_version 58040 (0.0007) [2023-10-11 21:22:25,860][71635] Updated weights for policy 1, policy_version 57992 (0.0009) [2023-10-11 21:22:26,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 118816768. Throughput: 0: 1830.4, 1: 1812.8. Samples: 29716224. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:22:26,034][70582] Avg episode reward: [(0, '58.620'), (1, '64.620')] [2023-10-11 21:22:26,232][71635] Updated weights for policy 1, policy_version 58002 (0.0009) [2023-10-11 21:22:26,605][71635] Updated weights for policy 1, policy_version 58012 (0.0007) [2023-10-11 21:22:28,109][71601] Updated weights for policy 0, policy_version 58050 (0.0008) [2023-10-11 21:22:28,481][71601] Updated weights for policy 0, policy_version 58060 (0.0007) [2023-10-11 21:22:28,852][71601] Updated weights for policy 0, policy_version 58070 (0.0008) [2023-10-11 21:22:29,222][71601] Updated weights for policy 0, policy_version 58080 (0.0007) [2023-10-11 21:22:30,182][71635] Updated weights for policy 1, policy_version 58022 (0.0007) [2023-10-11 21:22:30,547][71635] Updated weights for policy 1, policy_version 58032 (0.0007) [2023-10-11 21:22:30,920][71635] Updated weights for policy 1, policy_version 58042 (0.0007) [2023-10-11 21:22:31,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 118882304. Throughput: 0: 1824.1, 1: 1817.7. Samples: 29727470. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 21:22:31,034][70582] Avg episode reward: [(0, '62.680'), (1, '62.150')] [2023-10-11 21:22:33,033][71601] Updated weights for policy 0, policy_version 58090 (0.0008) [2023-10-11 21:22:33,394][71601] Updated weights for policy 0, policy_version 58100 (0.0010) [2023-10-11 21:22:33,774][71601] Updated weights for policy 0, policy_version 58110 (0.0007) [2023-10-11 21:22:34,449][71635] Updated weights for policy 1, policy_version 58052 (0.0008) [2023-10-11 21:22:34,828][71635] Updated weights for policy 1, policy_version 58062 (0.0011) [2023-10-11 21:22:35,189][71635] Updated weights for policy 1, policy_version 58072 (0.0010) [2023-10-11 21:22:36,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 118980608. Throughput: 0: 1824.5, 1: 1820.4. Samples: 29749194. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 21:22:36,035][70582] Avg episode reward: [(0, '62.590'), (1, '58.570')] [2023-10-11 21:22:37,394][71601] Updated weights for policy 0, policy_version 58120 (0.0008) [2023-10-11 21:22:37,772][71601] Updated weights for policy 0, policy_version 58130 (0.0007) [2023-10-11 21:22:38,133][71601] Updated weights for policy 0, policy_version 58140 (0.0008) [2023-10-11 21:22:38,852][71635] Updated weights for policy 1, policy_version 58082 (0.0011) [2023-10-11 21:22:39,210][71635] Updated weights for policy 1, policy_version 58092 (0.0010) [2023-10-11 21:22:39,584][71635] Updated weights for policy 1, policy_version 58102 (0.0009) [2023-10-11 21:22:39,950][71635] Updated weights for policy 1, policy_version 58112 (0.0008) [2023-10-11 21:22:41,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 119046144. Throughput: 0: 1829.7, 1: 1820.4. Samples: 29770890. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 21:22:41,034][70582] Avg episode reward: [(0, '64.200'), (1, '57.760')] [2023-10-11 21:22:41,978][71601] Updated weights for policy 0, policy_version 58150 (0.0008) [2023-10-11 21:22:42,375][71601] Updated weights for policy 0, policy_version 58160 (0.0008) [2023-10-11 21:22:42,735][71601] Updated weights for policy 0, policy_version 58170 (0.0007) [2023-10-11 21:22:43,687][71635] Updated weights for policy 1, policy_version 58122 (0.0010) [2023-10-11 21:22:44,054][71635] Updated weights for policy 1, policy_version 58132 (0.0010) [2023-10-11 21:22:44,420][71635] Updated weights for policy 1, policy_version 58142 (0.0011) [2023-10-11 21:22:46,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 119111680. Throughput: 0: 1829.8, 1: 1818.2. Samples: 29781946. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 21:22:46,034][70582] Avg episode reward: [(0, '64.420'), (1, '55.720')] [2023-10-11 21:22:46,328][71601] Updated weights for policy 0, policy_version 58180 (0.0008) [2023-10-11 21:22:46,707][71601] Updated weights for policy 0, policy_version 58190 (0.0007) [2023-10-11 21:22:47,079][71601] Updated weights for policy 0, policy_version 58200 (0.0008) [2023-10-11 21:22:48,289][71635] Updated weights for policy 1, policy_version 58152 (0.0009) [2023-10-11 21:22:48,658][71635] Updated weights for policy 1, policy_version 58162 (0.0009) [2023-10-11 21:22:49,028][71635] Updated weights for policy 1, policy_version 58172 (0.0009) [2023-10-11 21:22:50,641][71601] Updated weights for policy 0, policy_version 58210 (0.0010) [2023-10-11 21:22:51,016][71601] Updated weights for policy 0, policy_version 58220 (0.0007) [2023-10-11 21:22:51,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 119177216. Throughput: 0: 1829.1, 1: 1816.9. Samples: 29803300. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 21:22:51,034][70582] Avg episode reward: [(0, '58.920'), (1, '56.740')] [2023-10-11 21:22:51,385][71601] Updated weights for policy 0, policy_version 58230 (0.0009) [2023-10-11 21:22:51,751][71601] Updated weights for policy 0, policy_version 58240 (0.0007) [2023-10-11 21:22:52,693][71635] Updated weights for policy 1, policy_version 58182 (0.0009) [2023-10-11 21:22:53,069][71635] Updated weights for policy 1, policy_version 58192 (0.0011) [2023-10-11 21:22:53,440][71635] Updated weights for policy 1, policy_version 58202 (0.0011) [2023-10-11 21:22:55,354][71601] Updated weights for policy 0, policy_version 58250 (0.0009) [2023-10-11 21:22:55,720][71601] Updated weights for policy 0, policy_version 58260 (0.0010) [2023-10-11 21:22:56,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 119242752. Throughput: 0: 1825.6, 1: 1817.1. Samples: 29825758. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 21:22:56,034][70582] Avg episode reward: [(0, '59.600'), (1, '55.420')] [2023-10-11 21:22:56,091][71601] Updated weights for policy 0, policy_version 58270 (0.0010) [2023-10-11 21:22:57,341][71635] Updated weights for policy 1, policy_version 58212 (0.0011) [2023-10-11 21:22:57,715][71635] Updated weights for policy 1, policy_version 58222 (0.0011) [2023-10-11 21:22:58,074][71635] Updated weights for policy 1, policy_version 58232 (0.0010) [2023-10-11 21:22:59,567][71601] Updated weights for policy 0, policy_version 58280 (0.0011) [2023-10-11 21:22:59,939][71601] Updated weights for policy 0, policy_version 58290 (0.0010) [2023-10-11 21:23:00,311][71601] Updated weights for policy 0, policy_version 58300 (0.0011) [2023-10-11 21:23:01,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 119341056. Throughput: 0: 1834.2, 1: 1819.2. Samples: 29836366. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 21:23:01,035][70582] Avg episode reward: [(0, '53.270'), (1, '56.760')] [2023-10-11 21:23:01,770][71635] Updated weights for policy 1, policy_version 58242 (0.0008) [2023-10-11 21:23:02,137][71635] Updated weights for policy 1, policy_version 58252 (0.0008) [2023-10-11 21:23:02,501][71635] Updated weights for policy 1, policy_version 58262 (0.0011) [2023-10-11 21:23:02,873][71635] Updated weights for policy 1, policy_version 58272 (0.0008) [2023-10-11 21:23:04,346][71601] Updated weights for policy 0, policy_version 58310 (0.0008) [2023-10-11 21:23:04,715][71601] Updated weights for policy 0, policy_version 58320 (0.0009) [2023-10-11 21:23:05,093][71601] Updated weights for policy 0, policy_version 58330 (0.0010) [2023-10-11 21:23:06,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 119406592. Throughput: 0: 1821.4, 1: 1819.5. Samples: 29858426. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 21:23:06,035][70582] Avg episode reward: [(0, '53.040'), (1, '59.270')] [2023-10-11 21:23:06,684][71635] Updated weights for policy 1, policy_version 58282 (0.0009) [2023-10-11 21:23:07,064][71635] Updated weights for policy 1, policy_version 58292 (0.0008) [2023-10-11 21:23:07,426][71635] Updated weights for policy 1, policy_version 58302 (0.0008) [2023-10-11 21:23:08,888][71601] Updated weights for policy 0, policy_version 58340 (0.0010) [2023-10-11 21:23:09,249][71601] Updated weights for policy 0, policy_version 58350 (0.0008) [2023-10-11 21:23:09,622][71601] Updated weights for policy 0, policy_version 58360 (0.0007) [2023-10-11 21:23:11,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 119472128. Throughput: 0: 1824.6, 1: 1807.7. Samples: 29879680. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:23:11,034][70582] Avg episode reward: [(0, '57.910'), (1, '60.280')] [2023-10-11 21:23:11,115][71635] Updated weights for policy 1, policy_version 58312 (0.0010) [2023-10-11 21:23:11,486][71635] Updated weights for policy 1, policy_version 58322 (0.0008) [2023-10-11 21:23:11,851][71635] Updated weights for policy 1, policy_version 58332 (0.0007) [2023-10-11 21:23:13,262][71601] Updated weights for policy 0, policy_version 58370 (0.0007) [2023-10-11 21:23:13,634][71601] Updated weights for policy 0, policy_version 58380 (0.0008) [2023-10-11 21:23:14,003][71601] Updated weights for policy 0, policy_version 58390 (0.0009) [2023-10-11 21:23:14,371][71601] Updated weights for policy 0, policy_version 58400 (0.0007) [2023-10-11 21:23:15,478][71635] Updated weights for policy 1, policy_version 58342 (0.0008) [2023-10-11 21:23:15,852][71635] Updated weights for policy 1, policy_version 58352 (0.0009) [2023-10-11 21:23:16,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 119537664. Throughput: 0: 1827.9, 1: 1803.6. Samples: 29890886. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:23:16,034][70582] Avg episode reward: [(0, '51.420'), (1, '65.740')] [2023-10-11 21:23:16,205][71635] Updated weights for policy 1, policy_version 58362 (0.0007) [2023-10-11 21:23:18,101][71601] Updated weights for policy 0, policy_version 58410 (0.0009) [2023-10-11 21:23:18,464][71601] Updated weights for policy 0, policy_version 58420 (0.0009) [2023-10-11 21:23:18,840][71601] Updated weights for policy 0, policy_version 58430 (0.0009) [2023-10-11 21:23:19,927][71635] Updated weights for policy 1, policy_version 58372 (0.0009) [2023-10-11 21:23:20,298][71635] Updated weights for policy 1, policy_version 58382 (0.0009) [2023-10-11 21:23:20,663][71635] Updated weights for policy 1, policy_version 58392 (0.0009) [2023-10-11 21:23:21,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 119635968. Throughput: 0: 1827.5, 1: 1800.4. Samples: 29912448. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:23:21,035][70582] Avg episode reward: [(0, '50.200'), (1, '60.110')] [2023-10-11 21:23:22,532][71601] Updated weights for policy 0, policy_version 58440 (0.0011) [2023-10-11 21:23:22,913][71601] Updated weights for policy 0, policy_version 58450 (0.0008) [2023-10-11 21:23:23,295][71601] Updated weights for policy 0, policy_version 58460 (0.0008) [2023-10-11 21:23:24,496][71635] Updated weights for policy 1, policy_version 58402 (0.0007) [2023-10-11 21:23:24,873][71635] Updated weights for policy 1, policy_version 58412 (0.0008) [2023-10-11 21:23:25,241][71635] Updated weights for policy 1, policy_version 58422 (0.0010) [2023-10-11 21:23:25,603][71635] Updated weights for policy 1, policy_version 58432 (0.0010) [2023-10-11 21:23:26,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 119701504. Throughput: 0: 1826.8, 1: 1802.1. Samples: 29934190. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:23:26,034][70582] Avg episode reward: [(0, '44.480'), (1, '59.480')] [2023-10-11 21:23:26,912][71601] Updated weights for policy 0, policy_version 58470 (0.0008) [2023-10-11 21:23:27,293][71601] Updated weights for policy 0, policy_version 58480 (0.0010) [2023-10-11 21:23:27,666][71601] Updated weights for policy 0, policy_version 58490 (0.0009) [2023-10-11 21:23:29,274][71635] Updated weights for policy 1, policy_version 58442 (0.0007) [2023-10-11 21:23:29,639][71635] Updated weights for policy 1, policy_version 58452 (0.0008) [2023-10-11 21:23:30,008][71635] Updated weights for policy 1, policy_version 58462 (0.0008) [2023-10-11 21:23:31,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 119767040. Throughput: 0: 1829.7, 1: 1794.4. Samples: 29945030. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:23:31,034][70582] Avg episode reward: [(0, '45.200'), (1, '60.860')] [2023-10-11 21:23:31,351][71601] Updated weights for policy 0, policy_version 58500 (0.0009) [2023-10-11 21:23:31,715][71601] Updated weights for policy 0, policy_version 58510 (0.0008) [2023-10-11 21:23:32,087][71601] Updated weights for policy 0, policy_version 58520 (0.0007) [2023-10-11 21:23:33,775][71635] Updated weights for policy 1, policy_version 58472 (0.0009) [2023-10-11 21:23:34,139][71635] Updated weights for policy 1, policy_version 58482 (0.0009) [2023-10-11 21:23:34,502][71635] Updated weights for policy 1, policy_version 58492 (0.0010) [2023-10-11 21:23:35,696][71601] Updated weights for policy 0, policy_version 58530 (0.0010) [2023-10-11 21:23:36,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 119832576. Throughput: 0: 1825.8, 1: 1805.9. Samples: 29966726. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:23:36,034][70582] Avg episode reward: [(0, '45.860'), (1, '63.040')] [2023-10-11 21:23:36,064][71601] Updated weights for policy 0, policy_version 58540 (0.0008) [2023-10-11 21:23:36,440][71601] Updated weights for policy 0, policy_version 58550 (0.0008) [2023-10-11 21:23:36,804][71601] Updated weights for policy 0, policy_version 58560 (0.0009) [2023-10-11 21:23:38,204][71635] Updated weights for policy 1, policy_version 58502 (0.0008) [2023-10-11 21:23:38,562][71635] Updated weights for policy 1, policy_version 58512 (0.0008) [2023-10-11 21:23:38,940][71635] Updated weights for policy 1, policy_version 58522 (0.0011) [2023-10-11 21:23:40,429][71601] Updated weights for policy 0, policy_version 58570 (0.0009) [2023-10-11 21:23:40,810][71601] Updated weights for policy 0, policy_version 58580 (0.0009) [2023-10-11 21:23:41,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 119898112. Throughput: 0: 1826.2, 1: 1799.2. Samples: 29988902. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:23:41,034][70582] Avg episode reward: [(0, '43.900'), (1, '61.110')] [2023-10-11 21:23:41,176][71601] Updated weights for policy 0, policy_version 58590 (0.0010) [2023-10-11 21:23:42,589][71635] Updated weights for policy 1, policy_version 58532 (0.0009) [2023-10-11 21:23:42,953][71635] Updated weights for policy 1, policy_version 58542 (0.0007) [2023-10-11 21:23:43,314][71635] Updated weights for policy 1, policy_version 58552 (0.0010) [2023-10-11 21:23:44,895][71601] Updated weights for policy 0, policy_version 58600 (0.0008) [2023-10-11 21:23:45,261][71601] Updated weights for policy 0, policy_version 58610 (0.0008) [2023-10-11 21:23:45,625][71601] Updated weights for policy 0, policy_version 58620 (0.0008) [2023-10-11 21:23:46,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 119996416. Throughput: 0: 1818.4, 1: 1810.0. Samples: 29999642. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:23:46,035][70582] Avg episode reward: [(0, '48.410'), (1, '65.320')] [2023-10-11 21:23:47,076][71635] Updated weights for policy 1, policy_version 58562 (0.0009) [2023-10-11 21:23:47,439][71635] Updated weights for policy 1, policy_version 58572 (0.0008) [2023-10-11 21:23:47,797][71635] Updated weights for policy 1, policy_version 58582 (0.0010) [2023-10-11 21:23:48,161][71635] Updated weights for policy 1, policy_version 58592 (0.0008) [2023-10-11 21:23:49,206][71601] Updated weights for policy 0, policy_version 58630 (0.0007) [2023-10-11 21:23:49,568][71601] Updated weights for policy 0, policy_version 58640 (0.0007) [2023-10-11 21:23:49,944][71601] Updated weights for policy 0, policy_version 58650 (0.0007) [2023-10-11 21:23:51,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 120061952. Throughput: 0: 1825.6, 1: 1798.1. Samples: 30021490. Policy #0 lag: (min: 26.0, avg: 35.7, max: 58.0) [2023-10-11 21:23:51,035][70582] Avg episode reward: [(0, '45.690'), (1, '63.340')] [2023-10-11 21:23:52,049][71635] Updated weights for policy 1, policy_version 58602 (0.0009) [2023-10-11 21:23:52,418][71635] Updated weights for policy 1, policy_version 58612 (0.0010) [2023-10-11 21:23:52,780][71635] Updated weights for policy 1, policy_version 58622 (0.0011) [2023-10-11 21:23:53,691][71601] Updated weights for policy 0, policy_version 58660 (0.0008) [2023-10-11 21:23:54,067][71601] Updated weights for policy 0, policy_version 58670 (0.0009) [2023-10-11 21:23:54,440][71601] Updated weights for policy 0, policy_version 58680 (0.0010) [2023-10-11 21:23:56,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 120127488. Throughput: 0: 1826.7, 1: 1803.8. Samples: 30043054. Policy #0 lag: (min: 26.0, avg: 35.7, max: 58.0) [2023-10-11 21:23:56,034][70582] Avg episode reward: [(0, '38.080'), (1, '60.280')] [2023-10-11 21:23:56,043][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000058688_60096512.pth... [2023-10-11 21:23:56,044][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000058624_60030976.pth... [2023-10-11 21:23:56,073][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000056992_58359808.pth [2023-10-11 21:23:56,080][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000056928_58294272.pth [2023-10-11 21:23:56,501][71635] Updated weights for policy 1, policy_version 58632 (0.0010) [2023-10-11 21:23:56,869][71635] Updated weights for policy 1, policy_version 58642 (0.0010) [2023-10-11 21:23:57,238][71635] Updated weights for policy 1, policy_version 58652 (0.0007) [2023-10-11 21:23:58,165][71601] Updated weights for policy 0, policy_version 58690 (0.0009) [2023-10-11 21:23:58,543][71601] Updated weights for policy 0, policy_version 58700 (0.0009) [2023-10-11 21:23:58,906][71601] Updated weights for policy 0, policy_version 58710 (0.0007) [2023-10-11 21:23:59,271][71601] Updated weights for policy 0, policy_version 58720 (0.0009) [2023-10-11 21:24:00,915][71635] Updated weights for policy 1, policy_version 58662 (0.0008) [2023-10-11 21:24:01,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 120193024. Throughput: 0: 1820.5, 1: 1807.3. Samples: 30054138. Policy #0 lag: (min: 26.0, avg: 35.7, max: 58.0) [2023-10-11 21:24:01,035][70582] Avg episode reward: [(0, '40.510'), (1, '66.480')] [2023-10-11 21:24:01,274][71635] Updated weights for policy 1, policy_version 58672 (0.0008) [2023-10-11 21:24:01,655][71635] Updated weights for policy 1, policy_version 58682 (0.0008) [2023-10-11 21:24:02,883][71601] Updated weights for policy 0, policy_version 58730 (0.0007) [2023-10-11 21:24:03,245][71601] Updated weights for policy 0, policy_version 58740 (0.0007) [2023-10-11 21:24:03,617][71601] Updated weights for policy 0, policy_version 58750 (0.0007) [2023-10-11 21:24:05,389][71635] Updated weights for policy 1, policy_version 58692 (0.0008) [2023-10-11 21:24:05,753][71635] Updated weights for policy 1, policy_version 58702 (0.0007) [2023-10-11 21:24:06,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 120258560. Throughput: 0: 1823.5, 1: 1805.6. Samples: 30075754. Policy #0 lag: (min: 26.0, avg: 35.7, max: 58.0) [2023-10-11 21:24:06,034][70582] Avg episode reward: [(0, '36.640'), (1, '64.940')] [2023-10-11 21:24:06,120][71635] Updated weights for policy 1, policy_version 58712 (0.0007) [2023-10-11 21:24:07,281][71601] Updated weights for policy 0, policy_version 58760 (0.0008) [2023-10-11 21:24:07,653][71601] Updated weights for policy 0, policy_version 58770 (0.0008) [2023-10-11 21:24:08,030][71601] Updated weights for policy 0, policy_version 58780 (0.0008) [2023-10-11 21:24:09,753][71635] Updated weights for policy 1, policy_version 58722 (0.0008) [2023-10-11 21:24:10,120][71635] Updated weights for policy 1, policy_version 58732 (0.0008) [2023-10-11 21:24:10,489][71635] Updated weights for policy 1, policy_version 58742 (0.0009) [2023-10-11 21:24:10,857][71635] Updated weights for policy 1, policy_version 58752 (0.0007) [2023-10-11 21:24:11,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 120356864. Throughput: 0: 1815.8, 1: 1818.3. Samples: 30097722. Policy #0 lag: (min: 26.0, avg: 35.7, max: 58.0) [2023-10-11 21:24:11,034][70582] Avg episode reward: [(0, '37.210'), (1, '65.470')] [2023-10-11 21:24:11,695][71601] Updated weights for policy 0, policy_version 58790 (0.0009) [2023-10-11 21:24:12,067][71601] Updated weights for policy 0, policy_version 58800 (0.0007) [2023-10-11 21:24:12,433][71601] Updated weights for policy 0, policy_version 58810 (0.0008) [2023-10-11 21:24:14,358][71635] Updated weights for policy 1, policy_version 58762 (0.0010) [2023-10-11 21:24:14,721][71635] Updated weights for policy 1, policy_version 58772 (0.0009) [2023-10-11 21:24:15,084][71635] Updated weights for policy 1, policy_version 58782 (0.0007) [2023-10-11 21:24:16,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 120422400. Throughput: 0: 1818.5, 1: 1816.9. Samples: 30108624. Policy #0 lag: (min: 26.0, avg: 35.7, max: 58.0) [2023-10-11 21:24:16,034][70582] Avg episode reward: [(0, '40.020'), (1, '65.780')] [2023-10-11 21:24:16,082][71601] Updated weights for policy 0, policy_version 58820 (0.0008) [2023-10-11 21:24:16,479][71601] Updated weights for policy 0, policy_version 58830 (0.0009) [2023-10-11 21:24:16,853][71601] Updated weights for policy 0, policy_version 58840 (0.0010) [2023-10-11 21:24:18,659][71635] Updated weights for policy 1, policy_version 58792 (0.0008) [2023-10-11 21:24:19,031][71635] Updated weights for policy 1, policy_version 58802 (0.0012) [2023-10-11 21:24:19,400][71635] Updated weights for policy 1, policy_version 58812 (0.0010) [2023-10-11 21:24:20,594][71601] Updated weights for policy 0, policy_version 58850 (0.0009) [2023-10-11 21:24:20,963][71601] Updated weights for policy 0, policy_version 58860 (0.0008) [2023-10-11 21:24:21,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 120487936. Throughput: 0: 1821.1, 1: 1821.0. Samples: 30130622. Policy #0 lag: (min: 26.0, avg: 35.7, max: 58.0) [2023-10-11 21:24:21,034][70582] Avg episode reward: [(0, '32.930'), (1, '67.390')] [2023-10-11 21:24:21,327][71601] Updated weights for policy 0, policy_version 58870 (0.0007) [2023-10-11 21:24:21,706][71601] Updated weights for policy 0, policy_version 58880 (0.0008) [2023-10-11 21:24:23,024][71635] Updated weights for policy 1, policy_version 58822 (0.0010) [2023-10-11 21:24:23,396][71635] Updated weights for policy 1, policy_version 58832 (0.0008) [2023-10-11 21:24:23,759][71635] Updated weights for policy 1, policy_version 58842 (0.0008) [2023-10-11 21:24:25,419][71601] Updated weights for policy 0, policy_version 58890 (0.0008) [2023-10-11 21:24:25,789][71601] Updated weights for policy 0, policy_version 58900 (0.0007) [2023-10-11 21:24:26,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 120553472. Throughput: 0: 1815.6, 1: 1827.2. Samples: 30152830. Policy #0 lag: (min: 26.0, avg: 35.7, max: 58.0) [2023-10-11 21:24:26,035][70582] Avg episode reward: [(0, '33.250'), (1, '63.960')] [2023-10-11 21:24:26,156][71601] Updated weights for policy 0, policy_version 58910 (0.0007) [2023-10-11 21:24:27,445][71635] Updated weights for policy 1, policy_version 58852 (0.0007) [2023-10-11 21:24:27,813][71635] Updated weights for policy 1, policy_version 58862 (0.0007) [2023-10-11 21:24:28,175][71635] Updated weights for policy 1, policy_version 58872 (0.0008) [2023-10-11 21:24:29,736][71601] Updated weights for policy 0, policy_version 58920 (0.0009) [2023-10-11 21:24:30,110][71601] Updated weights for policy 0, policy_version 58930 (0.0009) [2023-10-11 21:24:30,488][71601] Updated weights for policy 0, policy_version 58940 (0.0011) [2023-10-11 21:24:31,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 120651776. Throughput: 0: 1817.0, 1: 1823.3. Samples: 30163454. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 21:24:31,034][70582] Avg episode reward: [(0, '35.450'), (1, '66.080')] [2023-10-11 21:24:31,927][71635] Updated weights for policy 1, policy_version 58882 (0.0009) [2023-10-11 21:24:32,285][71635] Updated weights for policy 1, policy_version 58892 (0.0008) [2023-10-11 21:24:32,651][71635] Updated weights for policy 1, policy_version 58902 (0.0008) [2023-10-11 21:24:33,015][71635] Updated weights for policy 1, policy_version 58912 (0.0008) [2023-10-11 21:24:34,033][71601] Updated weights for policy 0, policy_version 58950 (0.0010) [2023-10-11 21:24:34,412][71601] Updated weights for policy 0, policy_version 58960 (0.0008) [2023-10-11 21:24:34,779][71601] Updated weights for policy 0, policy_version 58970 (0.0008) [2023-10-11 21:24:36,034][70582] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 120717312. Throughput: 0: 1811.5, 1: 1830.5. Samples: 30185378. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 21:24:36,034][70582] Avg episode reward: [(0, '34.130'), (1, '69.110')] [2023-10-11 21:24:36,811][71635] Updated weights for policy 1, policy_version 58922 (0.0011) [2023-10-11 21:24:37,177][71635] Updated weights for policy 1, policy_version 58932 (0.0009) [2023-10-11 21:24:37,551][71635] Updated weights for policy 1, policy_version 58942 (0.0007) [2023-10-11 21:24:38,361][71601] Updated weights for policy 0, policy_version 58980 (0.0008) [2023-10-11 21:24:38,733][71601] Updated weights for policy 0, policy_version 58990 (0.0009) [2023-10-11 21:24:39,108][71601] Updated weights for policy 0, policy_version 59000 (0.0009) [2023-10-11 21:24:41,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 120782848. Throughput: 0: 1824.4, 1: 1838.4. Samples: 30207882. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 21:24:41,034][70582] Avg episode reward: [(0, '34.510'), (1, '68.640')] [2023-10-11 21:24:41,110][71635] Updated weights for policy 1, policy_version 58952 (0.0008) [2023-10-11 21:24:41,471][71635] Updated weights for policy 1, policy_version 58962 (0.0009) [2023-10-11 21:24:41,840][71635] Updated weights for policy 1, policy_version 58972 (0.0010) [2023-10-11 21:24:42,794][71601] Updated weights for policy 0, policy_version 59010 (0.0010) [2023-10-11 21:24:43,155][71601] Updated weights for policy 0, policy_version 59020 (0.0009) [2023-10-11 21:24:43,530][71601] Updated weights for policy 0, policy_version 59030 (0.0010) [2023-10-11 21:24:43,898][71601] Updated weights for policy 0, policy_version 59040 (0.0009) [2023-10-11 21:24:45,510][71635] Updated weights for policy 1, policy_version 58982 (0.0009) [2023-10-11 21:24:45,878][71635] Updated weights for policy 1, policy_version 58992 (0.0008) [2023-10-11 21:24:46,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 120848384. Throughput: 0: 1816.8, 1: 1838.1. Samples: 30218606. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 21:24:46,035][70582] Avg episode reward: [(0, '32.340'), (1, '67.240')] [2023-10-11 21:24:46,249][71635] Updated weights for policy 1, policy_version 59002 (0.0009) [2023-10-11 21:24:47,614][71601] Updated weights for policy 0, policy_version 59050 (0.0008) [2023-10-11 21:24:47,974][71601] Updated weights for policy 0, policy_version 59060 (0.0009) [2023-10-11 21:24:48,352][71601] Updated weights for policy 0, policy_version 59070 (0.0009) [2023-10-11 21:24:50,004][71635] Updated weights for policy 1, policy_version 59012 (0.0007) [2023-10-11 21:24:50,366][71635] Updated weights for policy 1, policy_version 59022 (0.0008) [2023-10-11 21:24:50,750][71635] Updated weights for policy 1, policy_version 59032 (0.0010) [2023-10-11 21:24:51,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 120946688. Throughput: 0: 1828.8, 1: 1840.8. Samples: 30240890. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 21:24:51,034][70582] Avg episode reward: [(0, '33.450'), (1, '68.380')] [2023-10-11 21:24:52,084][71601] Updated weights for policy 0, policy_version 59080 (0.0009) [2023-10-11 21:24:52,448][71601] Updated weights for policy 0, policy_version 59090 (0.0009) [2023-10-11 21:24:52,818][71601] Updated weights for policy 0, policy_version 59100 (0.0008) [2023-10-11 21:24:54,110][71635] Updated weights for policy 1, policy_version 59042 (0.0008) [2023-10-11 21:24:54,480][71635] Updated weights for policy 1, policy_version 59052 (0.0008) [2023-10-11 21:24:54,854][71635] Updated weights for policy 1, policy_version 59062 (0.0008) [2023-10-11 21:24:55,209][71635] Updated weights for policy 1, policy_version 59072 (0.0009) [2023-10-11 21:24:56,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 121012224. Throughput: 0: 1830.1, 1: 1828.4. Samples: 30262352. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 21:24:56,034][70582] Avg episode reward: [(0, '37.380'), (1, '72.420')] [2023-10-11 21:24:56,567][71601] Updated weights for policy 0, policy_version 59110 (0.0010) [2023-10-11 21:24:56,943][71601] Updated weights for policy 0, policy_version 59120 (0.0008) [2023-10-11 21:24:57,312][71601] Updated weights for policy 0, policy_version 59130 (0.0008) [2023-10-11 21:24:58,914][71635] Updated weights for policy 1, policy_version 59082 (0.0010) [2023-10-11 21:24:59,277][71635] Updated weights for policy 1, policy_version 59092 (0.0007) [2023-10-11 21:24:59,644][71635] Updated weights for policy 1, policy_version 59102 (0.0009) [2023-10-11 21:25:01,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 121077760. Throughput: 0: 1828.1, 1: 1842.5. Samples: 30273800. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 21:25:01,035][70582] Avg episode reward: [(0, '37.770'), (1, '70.320')] [2023-10-11 21:25:01,277][71601] Updated weights for policy 0, policy_version 59140 (0.0010) [2023-10-11 21:25:01,646][71601] Updated weights for policy 0, policy_version 59150 (0.0008) [2023-10-11 21:25:02,019][71601] Updated weights for policy 0, policy_version 59160 (0.0009) [2023-10-11 21:25:03,375][71635] Updated weights for policy 1, policy_version 59112 (0.0007) [2023-10-11 21:25:03,741][71635] Updated weights for policy 1, policy_version 59122 (0.0007) [2023-10-11 21:25:04,109][71635] Updated weights for policy 1, policy_version 59132 (0.0009) [2023-10-11 21:25:05,748][71601] Updated weights for policy 0, policy_version 59170 (0.0008) [2023-10-11 21:25:06,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 121143296. Throughput: 0: 1822.7, 1: 1829.9. Samples: 30294988. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 21:25:06,035][70582] Avg episode reward: [(0, '39.090'), (1, '69.570')] [2023-10-11 21:25:06,111][71601] Updated weights for policy 0, policy_version 59180 (0.0007) [2023-10-11 21:25:06,483][71601] Updated weights for policy 0, policy_version 59190 (0.0011) [2023-10-11 21:25:06,860][71601] Updated weights for policy 0, policy_version 59200 (0.0010) [2023-10-11 21:25:07,924][71635] Updated weights for policy 1, policy_version 59142 (0.0007) [2023-10-11 21:25:08,298][71635] Updated weights for policy 1, policy_version 59152 (0.0008) [2023-10-11 21:25:08,656][71635] Updated weights for policy 1, policy_version 59162 (0.0008) [2023-10-11 21:25:10,468][71601] Updated weights for policy 0, policy_version 59210 (0.0009) [2023-10-11 21:25:10,842][71601] Updated weights for policy 0, policy_version 59220 (0.0008) [2023-10-11 21:25:11,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 121208832. Throughput: 0: 1819.0, 1: 1834.1. Samples: 30317220. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 21:25:11,035][70582] Avg episode reward: [(0, '38.000'), (1, '64.690')] [2023-10-11 21:25:11,214][71601] Updated weights for policy 0, policy_version 59230 (0.0009) [2023-10-11 21:25:12,250][71635] Updated weights for policy 1, policy_version 59172 (0.0008) [2023-10-11 21:25:12,620][71635] Updated weights for policy 1, policy_version 59182 (0.0009) [2023-10-11 21:25:12,992][71635] Updated weights for policy 1, policy_version 59192 (0.0009) [2023-10-11 21:25:14,619][71601] Updated weights for policy 0, policy_version 59240 (0.0007) [2023-10-11 21:25:14,998][71601] Updated weights for policy 0, policy_version 59250 (0.0007) [2023-10-11 21:25:15,361][71601] Updated weights for policy 0, policy_version 59260 (0.0007) [2023-10-11 21:25:16,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 121307136. Throughput: 0: 1822.4, 1: 1829.3. Samples: 30327782. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-11 21:25:16,035][70582] Avg episode reward: [(0, '41.080'), (1, '64.730')] [2023-10-11 21:25:16,646][71635] Updated weights for policy 1, policy_version 59202 (0.0008) [2023-10-11 21:25:17,006][71635] Updated weights for policy 1, policy_version 59212 (0.0007) [2023-10-11 21:25:17,365][71635] Updated weights for policy 1, policy_version 59222 (0.0008) [2023-10-11 21:25:17,733][71635] Updated weights for policy 1, policy_version 59232 (0.0008) [2023-10-11 21:25:19,183][71601] Updated weights for policy 0, policy_version 59270 (0.0009) [2023-10-11 21:25:19,547][71601] Updated weights for policy 0, policy_version 59280 (0.0008) [2023-10-11 21:25:19,918][71601] Updated weights for policy 0, policy_version 59290 (0.0008) [2023-10-11 21:25:21,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 121372672. Throughput: 0: 1821.1, 1: 1836.5. Samples: 30349970. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-11 21:25:21,035][70582] Avg episode reward: [(0, '43.580'), (1, '65.930')] [2023-10-11 21:25:21,501][71635] Updated weights for policy 1, policy_version 59242 (0.0007) [2023-10-11 21:25:21,872][71635] Updated weights for policy 1, policy_version 59252 (0.0007) [2023-10-11 21:25:22,246][71635] Updated weights for policy 1, policy_version 59262 (0.0007) [2023-10-11 21:25:23,589][71601] Updated weights for policy 0, policy_version 59300 (0.0009) [2023-10-11 21:25:23,964][71601] Updated weights for policy 0, policy_version 59310 (0.0011) [2023-10-11 21:25:24,334][71601] Updated weights for policy 0, policy_version 59320 (0.0010) [2023-10-11 21:25:25,889][71635] Updated weights for policy 1, policy_version 59272 (0.0008) [2023-10-11 21:25:26,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 121438208. Throughput: 0: 1812.0, 1: 1830.4. Samples: 30371790. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-11 21:25:26,035][70582] Avg episode reward: [(0, '45.160'), (1, '65.100')] [2023-10-11 21:25:26,259][71635] Updated weights for policy 1, policy_version 59282 (0.0009) [2023-10-11 21:25:26,630][71635] Updated weights for policy 1, policy_version 59292 (0.0008) [2023-10-11 21:25:28,255][71601] Updated weights for policy 0, policy_version 59330 (0.0008) [2023-10-11 21:25:28,625][71601] Updated weights for policy 0, policy_version 59340 (0.0007) [2023-10-11 21:25:28,986][71601] Updated weights for policy 0, policy_version 59350 (0.0009) [2023-10-11 21:25:29,356][71601] Updated weights for policy 0, policy_version 59360 (0.0010) [2023-10-11 21:25:30,321][71635] Updated weights for policy 1, policy_version 59302 (0.0010) [2023-10-11 21:25:30,690][71635] Updated weights for policy 1, policy_version 59312 (0.0009) [2023-10-11 21:25:31,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 121503744. Throughput: 0: 1824.8, 1: 1827.8. Samples: 30382976. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-11 21:25:31,034][70582] Avg episode reward: [(0, '47.080'), (1, '65.950')] [2023-10-11 21:25:31,063][71635] Updated weights for policy 1, policy_version 59322 (0.0007) [2023-10-11 21:25:33,113][71601] Updated weights for policy 0, policy_version 59370 (0.0012) [2023-10-11 21:25:33,488][71601] Updated weights for policy 0, policy_version 59380 (0.0008) [2023-10-11 21:25:33,860][71601] Updated weights for policy 0, policy_version 59390 (0.0008) [2023-10-11 21:25:34,812][71635] Updated weights for policy 1, policy_version 59332 (0.0008) [2023-10-11 21:25:35,181][71635] Updated weights for policy 1, policy_version 59342 (0.0008) [2023-10-11 21:25:35,533][71635] Updated weights for policy 1, policy_version 59352 (0.0008) [2023-10-11 21:25:36,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 121602048. Throughput: 0: 1806.8, 1: 1824.2. Samples: 30404284. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-11 21:25:36,035][70582] Avg episode reward: [(0, '46.900'), (1, '61.100')] [2023-10-11 21:25:37,606][71601] Updated weights for policy 0, policy_version 59400 (0.0007) [2023-10-11 21:25:37,981][71601] Updated weights for policy 0, policy_version 59410 (0.0008) [2023-10-11 21:25:38,347][71601] Updated weights for policy 0, policy_version 59420 (0.0008) [2023-10-11 21:25:39,123][71635] Updated weights for policy 1, policy_version 59362 (0.0010) [2023-10-11 21:25:39,477][71635] Updated weights for policy 1, policy_version 59372 (0.0007) [2023-10-11 21:25:39,844][71635] Updated weights for policy 1, policy_version 59382 (0.0007) [2023-10-11 21:25:40,215][71635] Updated weights for policy 1, policy_version 59392 (0.0008) [2023-10-11 21:25:41,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 121667584. Throughput: 0: 1806.9, 1: 1822.9. Samples: 30425694. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-11 21:25:41,035][70582] Avg episode reward: [(0, '47.260'), (1, '62.280')] [2023-10-11 21:25:42,021][71601] Updated weights for policy 0, policy_version 59430 (0.0009) [2023-10-11 21:25:42,393][71601] Updated weights for policy 0, policy_version 59440 (0.0009) [2023-10-11 21:25:42,761][71601] Updated weights for policy 0, policy_version 59450 (0.0010) [2023-10-11 21:25:43,794][71635] Updated weights for policy 1, policy_version 59402 (0.0008) [2023-10-11 21:25:44,170][71635] Updated weights for policy 1, policy_version 59412 (0.0008) [2023-10-11 21:25:44,533][71635] Updated weights for policy 1, policy_version 59422 (0.0008) [2023-10-11 21:25:46,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 121733120. Throughput: 0: 1803.3, 1: 1823.7. Samples: 30437016. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-11 21:25:46,034][70582] Avg episode reward: [(0, '46.240'), (1, '66.240')] [2023-10-11 21:25:46,315][71601] Updated weights for policy 0, policy_version 59460 (0.0008) [2023-10-11 21:25:46,705][71601] Updated weights for policy 0, policy_version 59470 (0.0009) [2023-10-11 21:25:47,074][71601] Updated weights for policy 0, policy_version 59480 (0.0007) [2023-10-11 21:25:48,290][71635] Updated weights for policy 1, policy_version 59432 (0.0008) [2023-10-11 21:25:48,652][71635] Updated weights for policy 1, policy_version 59442 (0.0009) [2023-10-11 21:25:49,030][71635] Updated weights for policy 1, policy_version 59452 (0.0008) [2023-10-11 21:25:50,736][71601] Updated weights for policy 0, policy_version 59490 (0.0008) [2023-10-11 21:25:51,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 121798656. Throughput: 0: 1811.4, 1: 1822.4. Samples: 30458508. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-11 21:25:51,034][70582] Avg episode reward: [(0, '52.560'), (1, '68.770')] [2023-10-11 21:25:51,115][71601] Updated weights for policy 0, policy_version 59500 (0.0008) [2023-10-11 21:25:51,473][71601] Updated weights for policy 0, policy_version 59510 (0.0008) [2023-10-11 21:25:51,846][71601] Updated weights for policy 0, policy_version 59520 (0.0007) [2023-10-11 21:25:52,830][71635] Updated weights for policy 1, policy_version 59462 (0.0010) [2023-10-11 21:25:53,197][71635] Updated weights for policy 1, policy_version 59472 (0.0007) [2023-10-11 21:25:53,560][71635] Updated weights for policy 1, policy_version 59482 (0.0011) [2023-10-11 21:25:55,523][71601] Updated weights for policy 0, policy_version 59530 (0.0007) [2023-10-11 21:25:55,890][71601] Updated weights for policy 0, policy_version 59540 (0.0008) [2023-10-11 21:25:56,034][70582] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 121864192. Throughput: 0: 1821.1, 1: 1817.6. Samples: 30480964. Policy #0 lag: (min: 12.0, avg: 35.6, max: 40.0) [2023-10-11 21:25:56,035][70582] Avg episode reward: [(0, '46.290'), (1, '70.520')] [2023-10-11 21:25:56,046][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000059488_60915712.pth... [2023-10-11 21:25:56,076][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000057792_59179008.pth [2023-10-11 21:25:56,080][71431] Saving a milestone ./train_atari/atari_gopher_APPO/checkpoint_p1/milestones/checkpoint_000059488_60915712.pth [2023-10-11 21:25:56,271][71601] Updated weights for policy 0, policy_version 59550 (0.0007) [2023-10-11 21:25:56,338][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000059552_60981248.pth... [2023-10-11 21:25:56,367][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000057824_59211776.pth [2023-10-11 21:25:56,371][71353] Saving a milestone ./train_atari/atari_gopher_APPO/checkpoint_p0/milestones/checkpoint_000059552_60981248.pth [2023-10-11 21:25:57,245][71635] Updated weights for policy 1, policy_version 59492 (0.0009) [2023-10-11 21:25:57,609][71635] Updated weights for policy 1, policy_version 59502 (0.0009) [2023-10-11 21:25:57,977][71635] Updated weights for policy 1, policy_version 59512 (0.0009) [2023-10-11 21:26:00,036][71601] Updated weights for policy 0, policy_version 59560 (0.0007) [2023-10-11 21:26:00,401][71601] Updated weights for policy 0, policy_version 59570 (0.0010) [2023-10-11 21:26:00,781][71601] Updated weights for policy 0, policy_version 59580 (0.0009) [2023-10-11 21:26:01,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 121962496. Throughput: 0: 1815.4, 1: 1812.2. Samples: 30491024. Policy #0 lag: (min: 12.0, avg: 35.6, max: 40.0) [2023-10-11 21:26:01,034][70582] Avg episode reward: [(0, '39.700'), (1, '66.150')] [2023-10-11 21:26:01,631][71635] Updated weights for policy 1, policy_version 59522 (0.0011) [2023-10-11 21:26:02,003][71635] Updated weights for policy 1, policy_version 59532 (0.0009) [2023-10-11 21:26:02,369][71635] Updated weights for policy 1, policy_version 59542 (0.0007) [2023-10-11 21:26:02,733][71635] Updated weights for policy 1, policy_version 59552 (0.0007) [2023-10-11 21:26:04,482][71601] Updated weights for policy 0, policy_version 59590 (0.0008) [2023-10-11 21:26:04,849][71601] Updated weights for policy 0, policy_version 59600 (0.0008) [2023-10-11 21:26:05,219][71601] Updated weights for policy 0, policy_version 59610 (0.0008) [2023-10-11 21:26:06,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 122028032. Throughput: 0: 1823.6, 1: 1809.6. Samples: 30513460. Policy #0 lag: (min: 12.0, avg: 35.6, max: 40.0) [2023-10-11 21:26:06,035][70582] Avg episode reward: [(0, '39.260'), (1, '71.810')] [2023-10-11 21:26:06,570][71635] Updated weights for policy 1, policy_version 59562 (0.0010) [2023-10-11 21:26:06,939][71635] Updated weights for policy 1, policy_version 59572 (0.0008) [2023-10-11 21:26:07,307][71635] Updated weights for policy 1, policy_version 59582 (0.0009) [2023-10-11 21:26:08,802][71601] Updated weights for policy 0, policy_version 59620 (0.0009) [2023-10-11 21:26:09,180][71601] Updated weights for policy 0, policy_version 59630 (0.0009) [2023-10-11 21:26:09,560][71601] Updated weights for policy 0, policy_version 59640 (0.0010) [2023-10-11 21:26:10,988][71635] Updated weights for policy 1, policy_version 59592 (0.0008) [2023-10-11 21:26:11,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 122093568. Throughput: 0: 1823.7, 1: 1809.7. Samples: 30535294. Policy #0 lag: (min: 12.0, avg: 35.6, max: 40.0) [2023-10-11 21:26:11,034][70582] Avg episode reward: [(0, '40.020'), (1, '74.590')] [2023-10-11 21:26:11,349][71635] Updated weights for policy 1, policy_version 59602 (0.0008) [2023-10-11 21:26:11,709][71635] Updated weights for policy 1, policy_version 59612 (0.0008) [2023-10-11 21:26:13,161][71601] Updated weights for policy 0, policy_version 59650 (0.0010) [2023-10-11 21:26:13,535][71601] Updated weights for policy 0, policy_version 59660 (0.0011) [2023-10-11 21:26:13,899][71601] Updated weights for policy 0, policy_version 59670 (0.0009) [2023-10-11 21:26:14,262][71601] Updated weights for policy 0, policy_version 59680 (0.0007) [2023-10-11 21:26:15,579][71635] Updated weights for policy 1, policy_version 59622 (0.0011) [2023-10-11 21:26:15,950][71635] Updated weights for policy 1, policy_version 59632 (0.0009) [2023-10-11 21:26:16,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 122159104. Throughput: 0: 1820.2, 1: 1802.7. Samples: 30546004. Policy #0 lag: (min: 12.0, avg: 35.6, max: 40.0) [2023-10-11 21:26:16,035][70582] Avg episode reward: [(0, '38.360'), (1, '75.520')] [2023-10-11 21:26:16,328][71635] Updated weights for policy 1, policy_version 59642 (0.0011) [2023-10-11 21:26:18,023][71601] Updated weights for policy 0, policy_version 59690 (0.0007) [2023-10-11 21:26:18,398][71601] Updated weights for policy 0, policy_version 59700 (0.0007) [2023-10-11 21:26:18,774][71601] Updated weights for policy 0, policy_version 59710 (0.0010) [2023-10-11 21:26:20,150][71635] Updated weights for policy 1, policy_version 59652 (0.0009) [2023-10-11 21:26:20,511][71635] Updated weights for policy 1, policy_version 59662 (0.0009) [2023-10-11 21:26:20,873][71635] Updated weights for policy 1, policy_version 59672 (0.0007) [2023-10-11 21:26:21,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 122224640. Throughput: 0: 1819.8, 1: 1802.1. Samples: 30567270. Policy #0 lag: (min: 12.0, avg: 35.6, max: 40.0) [2023-10-11 21:26:21,035][70582] Avg episode reward: [(0, '34.900'), (1, '76.690')] [2023-10-11 21:26:22,428][71601] Updated weights for policy 0, policy_version 59720 (0.0010) [2023-10-11 21:26:22,811][71601] Updated weights for policy 0, policy_version 59730 (0.0008) [2023-10-11 21:26:23,184][71601] Updated weights for policy 0, policy_version 59740 (0.0009) [2023-10-11 21:26:24,630][71635] Updated weights for policy 1, policy_version 59682 (0.0010) [2023-10-11 21:26:24,998][71635] Updated weights for policy 1, policy_version 59692 (0.0008) [2023-10-11 21:26:25,370][71635] Updated weights for policy 1, policy_version 59702 (0.0007) [2023-10-11 21:26:25,745][71635] Updated weights for policy 1, policy_version 59712 (0.0007) [2023-10-11 21:26:26,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 122322944. Throughput: 0: 1824.2, 1: 1811.0. Samples: 30589280. Policy #0 lag: (min: 12.0, avg: 35.6, max: 40.0) [2023-10-11 21:26:26,035][70582] Avg episode reward: [(0, '38.540'), (1, '78.590')] [2023-10-11 21:26:26,736][71601] Updated weights for policy 0, policy_version 59750 (0.0010) [2023-10-11 21:26:27,101][71601] Updated weights for policy 0, policy_version 59760 (0.0007) [2023-10-11 21:26:27,480][71601] Updated weights for policy 0, policy_version 59770 (0.0008) [2023-10-11 21:26:29,484][71635] Updated weights for policy 1, policy_version 59722 (0.0010) [2023-10-11 21:26:29,846][71635] Updated weights for policy 1, policy_version 59732 (0.0008) [2023-10-11 21:26:30,224][71635] Updated weights for policy 1, policy_version 59742 (0.0007) [2023-10-11 21:26:31,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 122388480. Throughput: 0: 1830.6, 1: 1794.0. Samples: 30600122. Policy #0 lag: (min: 12.0, avg: 35.6, max: 40.0) [2023-10-11 21:26:31,034][70582] Avg episode reward: [(0, '40.840'), (1, '76.720')] [2023-10-11 21:26:31,185][71601] Updated weights for policy 0, policy_version 59780 (0.0009) [2023-10-11 21:26:31,546][71601] Updated weights for policy 0, policy_version 59790 (0.0009) [2023-10-11 21:26:31,920][71601] Updated weights for policy 0, policy_version 59800 (0.0008) [2023-10-11 21:26:34,056][71635] Updated weights for policy 1, policy_version 59752 (0.0008) [2023-10-11 21:26:34,424][71635] Updated weights for policy 1, policy_version 59762 (0.0009) [2023-10-11 21:26:34,790][71635] Updated weights for policy 1, policy_version 59772 (0.0008) [2023-10-11 21:26:35,697][71601] Updated weights for policy 0, policy_version 59810 (0.0007) [2023-10-11 21:26:36,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 122454016. Throughput: 0: 1816.1, 1: 1809.4. Samples: 30621656. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:26:36,034][70582] Avg episode reward: [(0, '39.770'), (1, '75.890')] [2023-10-11 21:26:36,077][71601] Updated weights for policy 0, policy_version 59820 (0.0007) [2023-10-11 21:26:36,446][71601] Updated weights for policy 0, policy_version 59830 (0.0009) [2023-10-11 21:26:36,817][71601] Updated weights for policy 0, policy_version 59840 (0.0009) [2023-10-11 21:26:38,517][71635] Updated weights for policy 1, policy_version 59782 (0.0008) [2023-10-11 21:26:38,889][71635] Updated weights for policy 1, policy_version 59792 (0.0009) [2023-10-11 21:26:39,260][71635] Updated weights for policy 1, policy_version 59802 (0.0009) [2023-10-11 21:26:40,565][71601] Updated weights for policy 0, policy_version 59850 (0.0009) [2023-10-11 21:26:40,935][71601] Updated weights for policy 0, policy_version 59860 (0.0007) [2023-10-11 21:26:41,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 122519552. Throughput: 0: 1821.7, 1: 1791.9. Samples: 30643576. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:26:41,034][70582] Avg episode reward: [(0, '43.320'), (1, '77.130')] [2023-10-11 21:26:41,315][71601] Updated weights for policy 0, policy_version 59870 (0.0008) [2023-10-11 21:26:42,747][71635] Updated weights for policy 1, policy_version 59812 (0.0007) [2023-10-11 21:26:43,116][71635] Updated weights for policy 1, policy_version 59822 (0.0007) [2023-10-11 21:26:43,477][71635] Updated weights for policy 1, policy_version 59832 (0.0008) [2023-10-11 21:26:45,065][71601] Updated weights for policy 0, policy_version 59880 (0.0008) [2023-10-11 21:26:45,436][71601] Updated weights for policy 0, policy_version 59890 (0.0008) [2023-10-11 21:26:45,799][71601] Updated weights for policy 0, policy_version 59900 (0.0007) [2023-10-11 21:26:46,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 122617856. Throughput: 0: 1820.0, 1: 1813.9. Samples: 30654548. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:26:46,034][70582] Avg episode reward: [(0, '43.440'), (1, '77.330')] [2023-10-11 21:26:47,098][71635] Updated weights for policy 1, policy_version 59842 (0.0009) [2023-10-11 21:26:47,464][71635] Updated weights for policy 1, policy_version 59852 (0.0010) [2023-10-11 21:26:47,833][71635] Updated weights for policy 1, policy_version 59862 (0.0010) [2023-10-11 21:26:48,200][71635] Updated weights for policy 1, policy_version 59872 (0.0009) [2023-10-11 21:26:49,398][71601] Updated weights for policy 0, policy_version 59910 (0.0008) [2023-10-11 21:26:49,758][71601] Updated weights for policy 0, policy_version 59920 (0.0008) [2023-10-11 21:26:50,128][71601] Updated weights for policy 0, policy_version 59930 (0.0009) [2023-10-11 21:26:51,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 122683392. Throughput: 0: 1820.9, 1: 1800.3. Samples: 30676412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:26:51,034][70582] Avg episode reward: [(0, '48.620'), (1, '75.340')] [2023-10-11 21:26:51,893][71635] Updated weights for policy 1, policy_version 59882 (0.0009) [2023-10-11 21:26:52,254][71635] Updated weights for policy 1, policy_version 59892 (0.0009) [2023-10-11 21:26:52,624][71635] Updated weights for policy 1, policy_version 59902 (0.0007) [2023-10-11 21:26:53,640][71601] Updated weights for policy 0, policy_version 59940 (0.0010) [2023-10-11 21:26:54,010][71601] Updated weights for policy 0, policy_version 59950 (0.0008) [2023-10-11 21:26:54,380][71601] Updated weights for policy 0, policy_version 59960 (0.0009) [2023-10-11 21:26:56,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 122748928. Throughput: 0: 1825.8, 1: 1799.8. Samples: 30698446. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:26:56,034][70582] Avg episode reward: [(0, '49.660'), (1, '70.860')] [2023-10-11 21:26:56,306][71635] Updated weights for policy 1, policy_version 59912 (0.0007) [2023-10-11 21:26:56,673][71635] Updated weights for policy 1, policy_version 59922 (0.0011) [2023-10-11 21:26:57,040][71635] Updated weights for policy 1, policy_version 59932 (0.0011) [2023-10-11 21:26:57,948][71601] Updated weights for policy 0, policy_version 59970 (0.0009) [2023-10-11 21:26:58,324][71601] Updated weights for policy 0, policy_version 59980 (0.0008) [2023-10-11 21:26:58,709][71601] Updated weights for policy 0, policy_version 59990 (0.0011) [2023-10-11 21:26:59,071][71601] Updated weights for policy 0, policy_version 60000 (0.0008) [2023-10-11 21:27:00,842][71635] Updated weights for policy 1, policy_version 59942 (0.0009) [2023-10-11 21:27:01,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 122814464. Throughput: 0: 1820.9, 1: 1806.3. Samples: 30709230. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:27:01,034][70582] Avg episode reward: [(0, '49.600'), (1, '69.630')] [2023-10-11 21:27:01,210][71635] Updated weights for policy 1, policy_version 59952 (0.0009) [2023-10-11 21:27:01,581][71635] Updated weights for policy 1, policy_version 59962 (0.0008) [2023-10-11 21:27:02,916][71601] Updated weights for policy 0, policy_version 60010 (0.0007) [2023-10-11 21:27:03,289][71601] Updated weights for policy 0, policy_version 60020 (0.0007) [2023-10-11 21:27:03,663][71601] Updated weights for policy 0, policy_version 60030 (0.0009) [2023-10-11 21:27:05,304][71635] Updated weights for policy 1, policy_version 59972 (0.0008) [2023-10-11 21:27:05,677][71635] Updated weights for policy 1, policy_version 59982 (0.0008) [2023-10-11 21:27:06,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 122880000. Throughput: 0: 1825.1, 1: 1810.4. Samples: 30730866. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:27:06,035][70582] Avg episode reward: [(0, '49.890'), (1, '66.920')] [2023-10-11 21:27:06,043][71635] Updated weights for policy 1, policy_version 59992 (0.0008) [2023-10-11 21:27:07,334][71601] Updated weights for policy 0, policy_version 60040 (0.0007) [2023-10-11 21:27:07,699][71601] Updated weights for policy 0, policy_version 60050 (0.0010) [2023-10-11 21:27:08,066][71601] Updated weights for policy 0, policy_version 60060 (0.0010) [2023-10-11 21:27:09,522][71635] Updated weights for policy 1, policy_version 60002 (0.0008) [2023-10-11 21:27:09,888][71635] Updated weights for policy 1, policy_version 60012 (0.0007) [2023-10-11 21:27:10,259][71635] Updated weights for policy 1, policy_version 60022 (0.0009) [2023-10-11 21:27:10,621][71635] Updated weights for policy 1, policy_version 60032 (0.0008) [2023-10-11 21:27:11,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 122978304. Throughput: 0: 1822.0, 1: 1813.2. Samples: 30752862. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:27:11,035][70582] Avg episode reward: [(0, '46.510'), (1, '67.030')] [2023-10-11 21:27:11,718][71601] Updated weights for policy 0, policy_version 60070 (0.0009) [2023-10-11 21:27:12,103][71601] Updated weights for policy 0, policy_version 60080 (0.0010) [2023-10-11 21:27:12,476][71601] Updated weights for policy 0, policy_version 60090 (0.0010) [2023-10-11 21:27:14,276][71635] Updated weights for policy 1, policy_version 60042 (0.0007) [2023-10-11 21:27:14,649][71635] Updated weights for policy 1, policy_version 60052 (0.0009) [2023-10-11 21:27:15,007][71635] Updated weights for policy 1, policy_version 60062 (0.0008) [2023-10-11 21:27:16,034][70582] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 123043840. Throughput: 0: 1821.4, 1: 1820.8. Samples: 30764024. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-11 21:27:16,034][70582] Avg episode reward: [(0, '49.860'), (1, '66.360')] [2023-10-11 21:27:16,270][71601] Updated weights for policy 0, policy_version 60100 (0.0010) [2023-10-11 21:27:16,652][71601] Updated weights for policy 0, policy_version 60110 (0.0009) [2023-10-11 21:27:17,023][71601] Updated weights for policy 0, policy_version 60120 (0.0008) [2023-10-11 21:27:18,625][71635] Updated weights for policy 1, policy_version 60072 (0.0008) [2023-10-11 21:27:19,004][71635] Updated weights for policy 1, policy_version 60082 (0.0008) [2023-10-11 21:27:19,363][71635] Updated weights for policy 1, policy_version 60092 (0.0009) [2023-10-11 21:27:20,822][71601] Updated weights for policy 0, policy_version 60130 (0.0008) [2023-10-11 21:27:21,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 123109376. Throughput: 0: 1829.2, 1: 1816.1. Samples: 30785692. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-11 21:27:21,034][70582] Avg episode reward: [(0, '52.060'), (1, '67.380')] [2023-10-11 21:27:21,194][71601] Updated weights for policy 0, policy_version 60140 (0.0008) [2023-10-11 21:27:21,572][71601] Updated weights for policy 0, policy_version 60150 (0.0008) [2023-10-11 21:27:21,942][71601] Updated weights for policy 0, policy_version 60160 (0.0008) [2023-10-11 21:27:23,118][71635] Updated weights for policy 1, policy_version 60102 (0.0007) [2023-10-11 21:27:23,481][71635] Updated weights for policy 1, policy_version 60112 (0.0007) [2023-10-11 21:27:23,851][71635] Updated weights for policy 1, policy_version 60122 (0.0010) [2023-10-11 21:27:25,658][71601] Updated weights for policy 0, policy_version 60170 (0.0008) [2023-10-11 21:27:26,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 123174912. Throughput: 0: 1824.3, 1: 1827.8. Samples: 30807920. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-11 21:27:26,034][70582] Avg episode reward: [(0, '51.910'), (1, '66.900')] [2023-10-11 21:27:26,036][71601] Updated weights for policy 0, policy_version 60180 (0.0009) [2023-10-11 21:27:26,396][71601] Updated weights for policy 0, policy_version 60190 (0.0009) [2023-10-11 21:27:27,601][71635] Updated weights for policy 1, policy_version 60132 (0.0010) [2023-10-11 21:27:27,962][71635] Updated weights for policy 1, policy_version 60142 (0.0009) [2023-10-11 21:27:28,325][71635] Updated weights for policy 1, policy_version 60152 (0.0011) [2023-10-11 21:27:30,056][71601] Updated weights for policy 0, policy_version 60200 (0.0008) [2023-10-11 21:27:30,425][71601] Updated weights for policy 0, policy_version 60210 (0.0009) [2023-10-11 21:27:30,800][71601] Updated weights for policy 0, policy_version 60220 (0.0009) [2023-10-11 21:27:31,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 123273216. Throughput: 0: 1821.7, 1: 1818.6. Samples: 30818362. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-11 21:27:31,034][70582] Avg episode reward: [(0, '46.130'), (1, '66.850')] [2023-10-11 21:27:32,109][71635] Updated weights for policy 1, policy_version 60162 (0.0010) [2023-10-11 21:27:32,480][71635] Updated weights for policy 1, policy_version 60172 (0.0008) [2023-10-11 21:27:32,841][71635] Updated weights for policy 1, policy_version 60182 (0.0008) [2023-10-11 21:27:33,209][71635] Updated weights for policy 1, policy_version 60192 (0.0010) [2023-10-11 21:27:34,540][71601] Updated weights for policy 0, policy_version 60230 (0.0007) [2023-10-11 21:27:34,909][71601] Updated weights for policy 0, policy_version 60240 (0.0008) [2023-10-11 21:27:35,284][71601] Updated weights for policy 0, policy_version 60250 (0.0009) [2023-10-11 21:27:36,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 123338752. Throughput: 0: 1820.7, 1: 1818.4. Samples: 30840172. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-11 21:27:36,035][70582] Avg episode reward: [(0, '46.710'), (1, '68.390')] [2023-10-11 21:27:37,169][71635] Updated weights for policy 1, policy_version 60202 (0.0009) [2023-10-11 21:27:37,539][71635] Updated weights for policy 1, policy_version 60212 (0.0009) [2023-10-11 21:27:37,916][71635] Updated weights for policy 1, policy_version 60222 (0.0009) [2023-10-11 21:27:38,835][71601] Updated weights for policy 0, policy_version 60260 (0.0009) [2023-10-11 21:27:39,187][71601] Updated weights for policy 0, policy_version 60270 (0.0009) [2023-10-11 21:27:39,559][71601] Updated weights for policy 0, policy_version 60280 (0.0010) [2023-10-11 21:27:41,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 123404288. Throughput: 0: 1804.6, 1: 1812.0. Samples: 30861194. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-11 21:27:41,035][70582] Avg episode reward: [(0, '44.120'), (1, '69.800')] [2023-10-11 21:27:41,641][71635] Updated weights for policy 1, policy_version 60232 (0.0009) [2023-10-11 21:27:42,013][71635] Updated weights for policy 1, policy_version 60242 (0.0009) [2023-10-11 21:27:42,379][71635] Updated weights for policy 1, policy_version 60252 (0.0009) [2023-10-11 21:27:43,295][71601] Updated weights for policy 0, policy_version 60290 (0.0008) [2023-10-11 21:27:43,666][71601] Updated weights for policy 0, policy_version 60300 (0.0007) [2023-10-11 21:27:44,035][71601] Updated weights for policy 0, policy_version 60310 (0.0008) [2023-10-11 21:27:44,416][71601] Updated weights for policy 0, policy_version 60320 (0.0009) [2023-10-11 21:27:46,005][71635] Updated weights for policy 1, policy_version 60262 (0.0007) [2023-10-11 21:27:46,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 123469824. Throughput: 0: 1813.8, 1: 1815.3. Samples: 30872540. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-11 21:27:46,034][70582] Avg episode reward: [(0, '47.030'), (1, '67.120')] [2023-10-11 21:27:46,371][71635] Updated weights for policy 1, policy_version 60272 (0.0007) [2023-10-11 21:27:46,743][71635] Updated weights for policy 1, policy_version 60282 (0.0010) [2023-10-11 21:27:47,912][71601] Updated weights for policy 0, policy_version 60330 (0.0010) [2023-10-11 21:27:48,281][71601] Updated weights for policy 0, policy_version 60340 (0.0011) [2023-10-11 21:27:48,655][71601] Updated weights for policy 0, policy_version 60350 (0.0010) [2023-10-11 21:27:50,526][71635] Updated weights for policy 1, policy_version 60292 (0.0011) [2023-10-11 21:27:50,892][71635] Updated weights for policy 1, policy_version 60302 (0.0009) [2023-10-11 21:27:51,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 123535360. Throughput: 0: 1814.0, 1: 1815.3. Samples: 30894184. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-11 21:27:51,034][70582] Avg episode reward: [(0, '43.970'), (1, '70.900')] [2023-10-11 21:27:51,263][71635] Updated weights for policy 1, policy_version 60312 (0.0009) [2023-10-11 21:27:52,365][71601] Updated weights for policy 0, policy_version 60360 (0.0010) [2023-10-11 21:27:52,735][71601] Updated weights for policy 0, policy_version 60370 (0.0010) [2023-10-11 21:27:53,113][71601] Updated weights for policy 0, policy_version 60380 (0.0010) [2023-10-11 21:27:54,896][71635] Updated weights for policy 1, policy_version 60322 (0.0008) [2023-10-11 21:27:55,266][71635] Updated weights for policy 1, policy_version 60332 (0.0009) [2023-10-11 21:27:55,625][71635] Updated weights for policy 1, policy_version 60342 (0.0007) [2023-10-11 21:27:55,993][71635] Updated weights for policy 1, policy_version 60352 (0.0007) [2023-10-11 21:27:56,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 123633664. Throughput: 0: 1813.1, 1: 1822.0. Samples: 30916444. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:27:56,035][70582] Avg episode reward: [(0, '38.870'), (1, '70.400')] [2023-10-11 21:27:56,045][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000060352_61800448.pth... [2023-10-11 21:27:56,046][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000060384_61833216.pth... [2023-10-11 21:27:56,077][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000058624_60030976.pth [2023-10-11 21:27:56,080][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000058688_60096512.pth [2023-10-11 21:27:56,837][71601] Updated weights for policy 0, policy_version 60390 (0.0009) [2023-10-11 21:27:57,196][71601] Updated weights for policy 0, policy_version 60400 (0.0008) [2023-10-11 21:27:57,578][71601] Updated weights for policy 0, policy_version 60410 (0.0008) [2023-10-11 21:27:59,748][71635] Updated weights for policy 1, policy_version 60362 (0.0008) [2023-10-11 21:28:00,112][71635] Updated weights for policy 1, policy_version 60372 (0.0009) [2023-10-11 21:28:00,476][71635] Updated weights for policy 1, policy_version 60382 (0.0007) [2023-10-11 21:28:01,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 123699200. Throughput: 0: 1811.9, 1: 1808.7. Samples: 30926950. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:28:01,035][70582] Avg episode reward: [(0, '36.710'), (1, '72.730')] [2023-10-11 21:28:01,367][71601] Updated weights for policy 0, policy_version 60420 (0.0009) [2023-10-11 21:28:01,759][71601] Updated weights for policy 0, policy_version 60430 (0.0007) [2023-10-11 21:28:02,145][71601] Updated weights for policy 0, policy_version 60440 (0.0007) [2023-10-11 21:28:04,066][71635] Updated weights for policy 1, policy_version 60392 (0.0009) [2023-10-11 21:28:04,431][71635] Updated weights for policy 1, policy_version 60402 (0.0007) [2023-10-11 21:28:04,799][71635] Updated weights for policy 1, policy_version 60412 (0.0010) [2023-10-11 21:28:05,839][71601] Updated weights for policy 0, policy_version 60450 (0.0008) [2023-10-11 21:28:06,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 123764736. Throughput: 0: 1810.4, 1: 1818.2. Samples: 30948982. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:28:06,035][70582] Avg episode reward: [(0, '45.040'), (1, '68.790')] [2023-10-11 21:28:06,211][71601] Updated weights for policy 0, policy_version 60460 (0.0009) [2023-10-11 21:28:06,585][71601] Updated weights for policy 0, policy_version 60470 (0.0007) [2023-10-11 21:28:06,950][71601] Updated weights for policy 0, policy_version 60480 (0.0007) [2023-10-11 21:28:08,550][71635] Updated weights for policy 1, policy_version 60422 (0.0010) [2023-10-11 21:28:08,915][71635] Updated weights for policy 1, policy_version 60432 (0.0011) [2023-10-11 21:28:09,287][71635] Updated weights for policy 1, policy_version 60442 (0.0011) [2023-10-11 21:28:10,431][71601] Updated weights for policy 0, policy_version 60490 (0.0010) [2023-10-11 21:28:10,815][71601] Updated weights for policy 0, policy_version 60500 (0.0008) [2023-10-11 21:28:11,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 123830272. Throughput: 0: 1811.2, 1: 1802.7. Samples: 30970546. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:28:11,034][70582] Avg episode reward: [(0, '45.540'), (1, '71.190')] [2023-10-11 21:28:11,180][71601] Updated weights for policy 0, policy_version 60510 (0.0008) [2023-10-11 21:28:13,107][71635] Updated weights for policy 1, policy_version 60452 (0.0009) [2023-10-11 21:28:13,476][71635] Updated weights for policy 1, policy_version 60462 (0.0009) [2023-10-11 21:28:13,840][71635] Updated weights for policy 1, policy_version 60472 (0.0009) [2023-10-11 21:28:14,820][71601] Updated weights for policy 0, policy_version 60520 (0.0009) [2023-10-11 21:28:15,191][71601] Updated weights for policy 0, policy_version 60530 (0.0008) [2023-10-11 21:28:15,563][71601] Updated weights for policy 0, policy_version 60540 (0.0007) [2023-10-11 21:28:16,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 123928576. Throughput: 0: 1816.8, 1: 1816.9. Samples: 30981876. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:28:16,034][70582] Avg episode reward: [(0, '47.030'), (1, '70.610')] [2023-10-11 21:28:17,427][71635] Updated weights for policy 1, policy_version 60482 (0.0008) [2023-10-11 21:28:17,792][71635] Updated weights for policy 1, policy_version 60492 (0.0010) [2023-10-11 21:28:18,144][71635] Updated weights for policy 1, policy_version 60502 (0.0009) [2023-10-11 21:28:18,514][71635] Updated weights for policy 1, policy_version 60512 (0.0008) [2023-10-11 21:28:19,335][71601] Updated weights for policy 0, policy_version 60550 (0.0010) [2023-10-11 21:28:19,713][71601] Updated weights for policy 0, policy_version 60560 (0.0010) [2023-10-11 21:28:20,072][71601] Updated weights for policy 0, policy_version 60570 (0.0007) [2023-10-11 21:28:21,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 123994112. Throughput: 0: 1817.7, 1: 1813.8. Samples: 31003590. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:28:21,034][70582] Avg episode reward: [(0, '43.510'), (1, '66.920')] [2023-10-11 21:28:22,161][71635] Updated weights for policy 1, policy_version 60522 (0.0007) [2023-10-11 21:28:22,546][71635] Updated weights for policy 1, policy_version 60532 (0.0008) [2023-10-11 21:28:22,917][71635] Updated weights for policy 1, policy_version 60542 (0.0009) [2023-10-11 21:28:23,989][71601] Updated weights for policy 0, policy_version 60580 (0.0010) [2023-10-11 21:28:24,356][71601] Updated weights for policy 0, policy_version 60590 (0.0010) [2023-10-11 21:28:24,733][71601] Updated weights for policy 0, policy_version 60600 (0.0009) [2023-10-11 21:28:26,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 124059648. Throughput: 0: 1819.6, 1: 1829.3. Samples: 31025396. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:28:26,034][70582] Avg episode reward: [(0, '45.100'), (1, '68.080')] [2023-10-11 21:28:26,484][71635] Updated weights for policy 1, policy_version 60552 (0.0008) [2023-10-11 21:28:26,848][71635] Updated weights for policy 1, policy_version 60562 (0.0008) [2023-10-11 21:28:27,222][71635] Updated weights for policy 1, policy_version 60572 (0.0009) [2023-10-11 21:28:28,539][71601] Updated weights for policy 0, policy_version 60610 (0.0009) [2023-10-11 21:28:28,920][71601] Updated weights for policy 0, policy_version 60620 (0.0008) [2023-10-11 21:28:29,283][71601] Updated weights for policy 0, policy_version 60630 (0.0008) [2023-10-11 21:28:29,651][71601] Updated weights for policy 0, policy_version 60640 (0.0011) [2023-10-11 21:28:30,897][71635] Updated weights for policy 1, policy_version 60582 (0.0008) [2023-10-11 21:28:31,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 124125184. Throughput: 0: 1823.5, 1: 1829.7. Samples: 31036934. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:28:31,035][70582] Avg episode reward: [(0, '45.800'), (1, '67.230')] [2023-10-11 21:28:31,262][71635] Updated weights for policy 1, policy_version 60592 (0.0007) [2023-10-11 21:28:31,622][71635] Updated weights for policy 1, policy_version 60602 (0.0008) [2023-10-11 21:28:33,321][71601] Updated weights for policy 0, policy_version 60650 (0.0008) [2023-10-11 21:28:33,698][71601] Updated weights for policy 0, policy_version 60660 (0.0007) [2023-10-11 21:28:34,082][71601] Updated weights for policy 0, policy_version 60670 (0.0008) [2023-10-11 21:28:35,402][71635] Updated weights for policy 1, policy_version 60612 (0.0008) [2023-10-11 21:28:35,767][71635] Updated weights for policy 1, policy_version 60622 (0.0007) [2023-10-11 21:28:36,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 124190720. Throughput: 0: 1814.4, 1: 1826.8. Samples: 31058042. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-11 21:28:36,034][70582] Avg episode reward: [(0, '44.760'), (1, '65.500')] [2023-10-11 21:28:36,131][71635] Updated weights for policy 1, policy_version 60632 (0.0007) [2023-10-11 21:28:37,783][71601] Updated weights for policy 0, policy_version 60680 (0.0007) [2023-10-11 21:28:38,151][71601] Updated weights for policy 0, policy_version 60690 (0.0010) [2023-10-11 21:28:38,528][71601] Updated weights for policy 0, policy_version 60700 (0.0009) [2023-10-11 21:28:39,884][71635] Updated weights for policy 1, policy_version 60642 (0.0008) [2023-10-11 21:28:40,255][71635] Updated weights for policy 1, policy_version 60652 (0.0010) [2023-10-11 21:28:40,622][71635] Updated weights for policy 1, policy_version 60662 (0.0010) [2023-10-11 21:28:40,993][71635] Updated weights for policy 1, policy_version 60672 (0.0010) [2023-10-11 21:28:41,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 124289024. Throughput: 0: 1814.9, 1: 1818.0. Samples: 31079924. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-11 21:28:41,035][70582] Avg episode reward: [(0, '39.430'), (1, '64.740')] [2023-10-11 21:28:42,133][71601] Updated weights for policy 0, policy_version 60710 (0.0007) [2023-10-11 21:28:42,499][71601] Updated weights for policy 0, policy_version 60720 (0.0007) [2023-10-11 21:28:42,870][71601] Updated weights for policy 0, policy_version 60730 (0.0010) [2023-10-11 21:28:44,824][71635] Updated weights for policy 1, policy_version 60682 (0.0011) [2023-10-11 21:28:45,181][71635] Updated weights for policy 1, policy_version 60692 (0.0009) [2023-10-11 21:28:45,542][71635] Updated weights for policy 1, policy_version 60702 (0.0007) [2023-10-11 21:28:46,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 124354560. Throughput: 0: 1818.1, 1: 1818.1. Samples: 31090580. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-11 21:28:46,034][70582] Avg episode reward: [(0, '40.720'), (1, '70.780')] [2023-10-11 21:28:46,500][71601] Updated weights for policy 0, policy_version 60740 (0.0010) [2023-10-11 21:28:46,896][71601] Updated weights for policy 0, policy_version 60750 (0.0007) [2023-10-11 21:28:47,269][71601] Updated weights for policy 0, policy_version 60760 (0.0008) [2023-10-11 21:28:49,352][71635] Updated weights for policy 1, policy_version 60712 (0.0009) [2023-10-11 21:28:49,715][71635] Updated weights for policy 1, policy_version 60722 (0.0010) [2023-10-11 21:28:50,082][71635] Updated weights for policy 1, policy_version 60732 (0.0010) [2023-10-11 21:28:50,987][71601] Updated weights for policy 0, policy_version 60770 (0.0008) [2023-10-11 21:28:51,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 124420096. Throughput: 0: 1817.7, 1: 1817.7. Samples: 31112576. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-11 21:28:51,034][70582] Avg episode reward: [(0, '38.750'), (1, '73.810')] [2023-10-11 21:28:51,360][71601] Updated weights for policy 0, policy_version 60780 (0.0007) [2023-10-11 21:28:51,731][71601] Updated weights for policy 0, policy_version 60790 (0.0007) [2023-10-11 21:28:52,110][71601] Updated weights for policy 0, policy_version 60800 (0.0007) [2023-10-11 21:28:53,787][71635] Updated weights for policy 1, policy_version 60742 (0.0010) [2023-10-11 21:28:54,158][71635] Updated weights for policy 1, policy_version 60752 (0.0009) [2023-10-11 21:28:54,518][71635] Updated weights for policy 1, policy_version 60762 (0.0007) [2023-10-11 21:28:55,775][71601] Updated weights for policy 0, policy_version 60810 (0.0008) [2023-10-11 21:28:56,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 124485632. Throughput: 0: 1822.3, 1: 1820.6. Samples: 31134476. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-11 21:28:56,034][70582] Avg episode reward: [(0, '39.440'), (1, '70.630')] [2023-10-11 21:28:56,145][71601] Updated weights for policy 0, policy_version 60820 (0.0009) [2023-10-11 21:28:56,520][71601] Updated weights for policy 0, policy_version 60830 (0.0008) [2023-10-11 21:28:58,164][71635] Updated weights for policy 1, policy_version 60772 (0.0008) [2023-10-11 21:28:58,526][71635] Updated weights for policy 1, policy_version 60782 (0.0010) [2023-10-11 21:28:58,893][71635] Updated weights for policy 1, policy_version 60792 (0.0011) [2023-10-11 21:29:00,095][71601] Updated weights for policy 0, policy_version 60840 (0.0007) [2023-10-11 21:29:00,454][71601] Updated weights for policy 0, policy_version 60850 (0.0008) [2023-10-11 21:29:00,831][71601] Updated weights for policy 0, policy_version 60860 (0.0010) [2023-10-11 21:29:01,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 124583936. Throughput: 0: 1815.1, 1: 1825.7. Samples: 31145714. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-11 21:29:01,034][70582] Avg episode reward: [(0, '41.880'), (1, '66.430')] [2023-10-11 21:29:02,581][71635] Updated weights for policy 1, policy_version 60802 (0.0009) [2023-10-11 21:29:02,951][71635] Updated weights for policy 1, policy_version 60812 (0.0011) [2023-10-11 21:29:03,310][71635] Updated weights for policy 1, policy_version 60822 (0.0009) [2023-10-11 21:29:03,670][71635] Updated weights for policy 1, policy_version 60832 (0.0007) [2023-10-11 21:29:04,573][71601] Updated weights for policy 0, policy_version 60870 (0.0009) [2023-10-11 21:29:04,939][71601] Updated weights for policy 0, policy_version 60880 (0.0010) [2023-10-11 21:29:05,319][71601] Updated weights for policy 0, policy_version 60890 (0.0008) [2023-10-11 21:29:06,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 124649472. Throughput: 0: 1820.6, 1: 1819.2. Samples: 31167378. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-11 21:29:06,035][70582] Avg episode reward: [(0, '42.660'), (1, '65.560')] [2023-10-11 21:29:07,426][71635] Updated weights for policy 1, policy_version 60842 (0.0010) [2023-10-11 21:29:07,800][71635] Updated weights for policy 1, policy_version 60852 (0.0009) [2023-10-11 21:29:08,164][71635] Updated weights for policy 1, policy_version 60862 (0.0007) [2023-10-11 21:29:08,965][71601] Updated weights for policy 0, policy_version 60900 (0.0008) [2023-10-11 21:29:09,345][71601] Updated weights for policy 0, policy_version 60910 (0.0009) [2023-10-11 21:29:09,709][71601] Updated weights for policy 0, policy_version 60920 (0.0008) [2023-10-11 21:29:11,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 124715008. Throughput: 0: 1823.6, 1: 1816.2. Samples: 31189188. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-11 21:29:11,035][70582] Avg episode reward: [(0, '46.270'), (1, '69.920')] [2023-10-11 21:29:11,669][71635] Updated weights for policy 1, policy_version 60872 (0.0008) [2023-10-11 21:29:12,034][71635] Updated weights for policy 1, policy_version 60882 (0.0008) [2023-10-11 21:29:12,398][71635] Updated weights for policy 1, policy_version 60892 (0.0011) [2023-10-11 21:29:13,266][71601] Updated weights for policy 0, policy_version 60930 (0.0008) [2023-10-11 21:29:13,636][71601] Updated weights for policy 0, policy_version 60940 (0.0010) [2023-10-11 21:29:13,999][71601] Updated weights for policy 0, policy_version 60950 (0.0011) [2023-10-11 21:29:14,373][71601] Updated weights for policy 0, policy_version 60960 (0.0007) [2023-10-11 21:29:15,904][71635] Updated weights for policy 1, policy_version 60902 (0.0008) [2023-10-11 21:29:16,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 124780544. Throughput: 0: 1819.7, 1: 1813.1. Samples: 31200412. Policy #0 lag: (min: 31.0, avg: 41.8, max: 63.0) [2023-10-11 21:29:16,034][70582] Avg episode reward: [(0, '46.030'), (1, '73.480')] [2023-10-11 21:29:16,274][71635] Updated weights for policy 1, policy_version 60912 (0.0008) [2023-10-11 21:29:16,640][71635] Updated weights for policy 1, policy_version 60922 (0.0010) [2023-10-11 21:29:18,053][71601] Updated weights for policy 0, policy_version 60970 (0.0007) [2023-10-11 21:29:18,431][71601] Updated weights for policy 0, policy_version 60980 (0.0007) [2023-10-11 21:29:18,802][71601] Updated weights for policy 0, policy_version 60990 (0.0010) [2023-10-11 21:29:20,330][71635] Updated weights for policy 1, policy_version 60932 (0.0010) [2023-10-11 21:29:20,690][71635] Updated weights for policy 1, policy_version 60942 (0.0009) [2023-10-11 21:29:21,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 124846080. Throughput: 0: 1827.1, 1: 1819.1. Samples: 31222124. Policy #0 lag: (min: 31.0, avg: 41.8, max: 63.0) [2023-10-11 21:29:21,036][70582] Avg episode reward: [(0, '46.240'), (1, '71.200')] [2023-10-11 21:29:21,055][71635] Updated weights for policy 1, policy_version 60952 (0.0011) [2023-10-11 21:29:22,620][71601] Updated weights for policy 0, policy_version 61000 (0.0009) [2023-10-11 21:29:22,991][71601] Updated weights for policy 0, policy_version 61010 (0.0008) [2023-10-11 21:29:23,362][71601] Updated weights for policy 0, policy_version 61020 (0.0008) [2023-10-11 21:29:24,761][71635] Updated weights for policy 1, policy_version 60962 (0.0009) [2023-10-11 21:29:25,122][71635] Updated weights for policy 1, policy_version 60972 (0.0007) [2023-10-11 21:29:25,501][71635] Updated weights for policy 1, policy_version 60982 (0.0010) [2023-10-11 21:29:25,869][71635] Updated weights for policy 1, policy_version 60992 (0.0009) [2023-10-11 21:29:26,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 124944384. Throughput: 0: 1823.2, 1: 1821.8. Samples: 31243950. Policy #0 lag: (min: 31.0, avg: 41.8, max: 63.0) [2023-10-11 21:29:26,034][70582] Avg episode reward: [(0, '47.270'), (1, '70.740')] [2023-10-11 21:29:27,040][71601] Updated weights for policy 0, policy_version 61030 (0.0007) [2023-10-11 21:29:27,405][71601] Updated weights for policy 0, policy_version 61040 (0.0008) [2023-10-11 21:29:27,773][71601] Updated weights for policy 0, policy_version 61050 (0.0008) [2023-10-11 21:29:29,679][71635] Updated weights for policy 1, policy_version 61002 (0.0011) [2023-10-11 21:29:30,047][71635] Updated weights for policy 1, policy_version 61012 (0.0010) [2023-10-11 21:29:30,419][71635] Updated weights for policy 1, policy_version 61022 (0.0008) [2023-10-11 21:29:31,034][70582] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 125009920. Throughput: 0: 1820.2, 1: 1825.9. Samples: 31254652. Policy #0 lag: (min: 31.0, avg: 41.8, max: 63.0) [2023-10-11 21:29:31,034][70582] Avg episode reward: [(0, '47.090'), (1, '67.640')] [2023-10-11 21:29:31,412][71601] Updated weights for policy 0, policy_version 61060 (0.0009) [2023-10-11 21:29:31,780][71601] Updated weights for policy 0, policy_version 61070 (0.0008) [2023-10-11 21:29:32,161][71601] Updated weights for policy 0, policy_version 61080 (0.0008) [2023-10-11 21:29:34,081][71635] Updated weights for policy 1, policy_version 61032 (0.0008) [2023-10-11 21:29:34,450][71635] Updated weights for policy 1, policy_version 61042 (0.0007) [2023-10-11 21:29:34,813][71635] Updated weights for policy 1, policy_version 61052 (0.0007) [2023-10-11 21:29:35,953][71601] Updated weights for policy 0, policy_version 61090 (0.0008) [2023-10-11 21:29:36,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 125075456. Throughput: 0: 1825.1, 1: 1823.9. Samples: 31276782. Policy #0 lag: (min: 31.0, avg: 41.8, max: 63.0) [2023-10-11 21:29:36,035][70582] Avg episode reward: [(0, '44.710'), (1, '67.360')] [2023-10-11 21:29:36,360][71601] Updated weights for policy 0, policy_version 61100 (0.0009) [2023-10-11 21:29:36,733][71601] Updated weights for policy 0, policy_version 61110 (0.0008) [2023-10-11 21:29:37,097][71601] Updated weights for policy 0, policy_version 61120 (0.0009) [2023-10-11 21:29:38,445][71635] Updated weights for policy 1, policy_version 61062 (0.0008) [2023-10-11 21:29:38,817][71635] Updated weights for policy 1, policy_version 61072 (0.0008) [2023-10-11 21:29:39,177][71635] Updated weights for policy 1, policy_version 61082 (0.0007) [2023-10-11 21:29:40,845][71601] Updated weights for policy 0, policy_version 61130 (0.0011) [2023-10-11 21:29:41,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 125140992. Throughput: 0: 1822.2, 1: 1830.8. Samples: 31298860. Policy #0 lag: (min: 31.0, avg: 41.8, max: 63.0) [2023-10-11 21:29:41,034][70582] Avg episode reward: [(0, '46.770'), (1, '71.110')] [2023-10-11 21:29:41,228][71601] Updated weights for policy 0, policy_version 61140 (0.0010) [2023-10-11 21:29:41,587][71601] Updated weights for policy 0, policy_version 61150 (0.0009) [2023-10-11 21:29:42,913][71635] Updated weights for policy 1, policy_version 61092 (0.0008) [2023-10-11 21:29:43,274][71635] Updated weights for policy 1, policy_version 61102 (0.0007) [2023-10-11 21:29:43,649][71635] Updated weights for policy 1, policy_version 61112 (0.0008) [2023-10-11 21:29:45,230][71601] Updated weights for policy 0, policy_version 61160 (0.0011) [2023-10-11 21:29:45,596][71601] Updated weights for policy 0, policy_version 61170 (0.0010) [2023-10-11 21:29:45,963][71601] Updated weights for policy 0, policy_version 61180 (0.0010) [2023-10-11 21:29:46,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 125206528. Throughput: 0: 1820.1, 1: 1822.3. Samples: 31309622. Policy #0 lag: (min: 31.0, avg: 41.8, max: 63.0) [2023-10-11 21:29:46,034][70582] Avg episode reward: [(0, '47.180'), (1, '76.930')] [2023-10-11 21:29:47,169][71635] Updated weights for policy 1, policy_version 61122 (0.0008) [2023-10-11 21:29:47,539][71635] Updated weights for policy 1, policy_version 61132 (0.0008) [2023-10-11 21:29:47,907][71635] Updated weights for policy 1, policy_version 61142 (0.0009) [2023-10-11 21:29:48,268][71635] Updated weights for policy 1, policy_version 61152 (0.0009) [2023-10-11 21:29:49,720][71601] Updated weights for policy 0, policy_version 61190 (0.0008) [2023-10-11 21:29:50,084][71601] Updated weights for policy 0, policy_version 61200 (0.0007) [2023-10-11 21:29:50,452][71601] Updated weights for policy 0, policy_version 61210 (0.0008) [2023-10-11 21:29:51,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 125304832. Throughput: 0: 1817.3, 1: 1835.0. Samples: 31331732. Policy #0 lag: (min: 31.0, avg: 41.8, max: 63.0) [2023-10-11 21:29:51,035][70582] Avg episode reward: [(0, '46.020'), (1, '80.680')] [2023-10-11 21:29:52,055][71635] Updated weights for policy 1, policy_version 61162 (0.0008) [2023-10-11 21:29:52,423][71635] Updated weights for policy 1, policy_version 61172 (0.0008) [2023-10-11 21:29:52,782][71635] Updated weights for policy 1, policy_version 61182 (0.0009) [2023-10-11 21:29:54,026][71601] Updated weights for policy 0, policy_version 61220 (0.0007) [2023-10-11 21:29:54,389][71601] Updated weights for policy 0, policy_version 61230 (0.0010) [2023-10-11 21:29:54,769][71601] Updated weights for policy 0, policy_version 61240 (0.0009) [2023-10-11 21:29:56,034][70582] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 125370368. Throughput: 0: 1819.0, 1: 1826.4. Samples: 31353234. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-11 21:29:56,035][70582] Avg episode reward: [(0, '45.560'), (1, '72.290')] [2023-10-11 21:29:56,047][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000061184_62652416.pth... [2023-10-11 21:29:56,047][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000061248_62717952.pth... [2023-10-11 21:29:56,078][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000059552_60981248.pth [2023-10-11 21:29:56,086][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000059488_60915712.pth [2023-10-11 21:29:56,492][71635] Updated weights for policy 1, policy_version 61192 (0.0007) [2023-10-11 21:29:56,850][71635] Updated weights for policy 1, policy_version 61202 (0.0008) [2023-10-11 21:29:57,217][71635] Updated weights for policy 1, policy_version 61212 (0.0008) [2023-10-11 21:29:58,375][71601] Updated weights for policy 0, policy_version 61250 (0.0011) [2023-10-11 21:29:58,749][71601] Updated weights for policy 0, policy_version 61260 (0.0008) [2023-10-11 21:29:59,127][71601] Updated weights for policy 0, policy_version 61270 (0.0010) [2023-10-11 21:29:59,500][71601] Updated weights for policy 0, policy_version 61280 (0.0009) [2023-10-11 21:30:00,802][71635] Updated weights for policy 1, policy_version 61222 (0.0009) [2023-10-11 21:30:01,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 125435904. Throughput: 0: 1819.0, 1: 1829.1. Samples: 31364576. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-11 21:30:01,034][70582] Avg episode reward: [(0, '41.560'), (1, '74.370')] [2023-10-11 21:30:01,163][71635] Updated weights for policy 1, policy_version 61232 (0.0011) [2023-10-11 21:30:01,539][71635] Updated weights for policy 1, policy_version 61242 (0.0009) [2023-10-11 21:30:03,244][71601] Updated weights for policy 0, policy_version 61290 (0.0007) [2023-10-11 21:30:03,619][71601] Updated weights for policy 0, policy_version 61300 (0.0007) [2023-10-11 21:30:03,991][71601] Updated weights for policy 0, policy_version 61310 (0.0009) [2023-10-11 21:30:05,293][71635] Updated weights for policy 1, policy_version 61252 (0.0008) [2023-10-11 21:30:05,659][71635] Updated weights for policy 1, policy_version 61262 (0.0009) [2023-10-11 21:30:06,023][71635] Updated weights for policy 1, policy_version 61272 (0.0007) [2023-10-11 21:30:06,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 125501440. Throughput: 0: 1816.9, 1: 1828.6. Samples: 31386170. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-11 21:30:06,035][70582] Avg episode reward: [(0, '43.660'), (1, '69.800')] [2023-10-11 21:30:07,612][71601] Updated weights for policy 0, policy_version 61320 (0.0007) [2023-10-11 21:30:07,983][71601] Updated weights for policy 0, policy_version 61330 (0.0007) [2023-10-11 21:30:08,359][71601] Updated weights for policy 0, policy_version 61340 (0.0008) [2023-10-11 21:30:09,823][71635] Updated weights for policy 1, policy_version 61282 (0.0009) [2023-10-11 21:30:10,189][71635] Updated weights for policy 1, policy_version 61292 (0.0009) [2023-10-11 21:30:10,547][71635] Updated weights for policy 1, policy_version 61302 (0.0010) [2023-10-11 21:30:10,914][71635] Updated weights for policy 1, policy_version 61312 (0.0011) [2023-10-11 21:30:11,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 125599744. Throughput: 0: 1819.0, 1: 1823.5. Samples: 31407862. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-11 21:30:11,035][70582] Avg episode reward: [(0, '39.730'), (1, '70.860')] [2023-10-11 21:30:12,119][71601] Updated weights for policy 0, policy_version 61350 (0.0009) [2023-10-11 21:30:12,494][71601] Updated weights for policy 0, policy_version 61360 (0.0008) [2023-10-11 21:30:12,874][71601] Updated weights for policy 0, policy_version 61370 (0.0008) [2023-10-11 21:30:14,669][71635] Updated weights for policy 1, policy_version 61322 (0.0007) [2023-10-11 21:30:15,042][71635] Updated weights for policy 1, policy_version 61332 (0.0011) [2023-10-11 21:30:15,417][71635] Updated weights for policy 1, policy_version 61342 (0.0012) [2023-10-11 21:30:16,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 125665280. Throughput: 0: 1817.6, 1: 1819.1. Samples: 31418304. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-11 21:30:16,034][70582] Avg episode reward: [(0, '39.900'), (1, '69.290')] [2023-10-11 21:30:16,558][71601] Updated weights for policy 0, policy_version 61380 (0.0009) [2023-10-11 21:30:16,931][71601] Updated weights for policy 0, policy_version 61390 (0.0010) [2023-10-11 21:30:17,316][71601] Updated weights for policy 0, policy_version 61400 (0.0009) [2023-10-11 21:30:18,995][71635] Updated weights for policy 1, policy_version 61352 (0.0010) [2023-10-11 21:30:19,363][71635] Updated weights for policy 1, policy_version 61362 (0.0011) [2023-10-11 21:30:19,735][71635] Updated weights for policy 1, policy_version 61372 (0.0009) [2023-10-11 21:30:21,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 125730816. Throughput: 0: 1813.2, 1: 1821.7. Samples: 31440354. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-11 21:30:21,035][70582] Avg episode reward: [(0, '38.530'), (1, '66.810')] [2023-10-11 21:30:21,127][71601] Updated weights for policy 0, policy_version 61410 (0.0008) [2023-10-11 21:30:21,538][71601] Updated weights for policy 0, policy_version 61420 (0.0010) [2023-10-11 21:30:21,904][71601] Updated weights for policy 0, policy_version 61430 (0.0008) [2023-10-11 21:30:22,276][71601] Updated weights for policy 0, policy_version 61440 (0.0009) [2023-10-11 21:30:23,439][71635] Updated weights for policy 1, policy_version 61382 (0.0011) [2023-10-11 21:30:23,803][71635] Updated weights for policy 1, policy_version 61392 (0.0009) [2023-10-11 21:30:24,167][71635] Updated weights for policy 1, policy_version 61402 (0.0008) [2023-10-11 21:30:25,830][71601] Updated weights for policy 0, policy_version 61450 (0.0007) [2023-10-11 21:30:26,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 125796352. Throughput: 0: 1815.6, 1: 1817.0. Samples: 31462328. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-11 21:30:26,035][70582] Avg episode reward: [(0, '41.730'), (1, '58.570')] [2023-10-11 21:30:26,207][71601] Updated weights for policy 0, policy_version 61460 (0.0009) [2023-10-11 21:30:26,571][71601] Updated weights for policy 0, policy_version 61470 (0.0008) [2023-10-11 21:30:27,881][71635] Updated weights for policy 1, policy_version 61412 (0.0009) [2023-10-11 21:30:28,254][71635] Updated weights for policy 1, policy_version 61422 (0.0009) [2023-10-11 21:30:28,618][71635] Updated weights for policy 1, policy_version 61432 (0.0009) [2023-10-11 21:30:30,178][71601] Updated weights for policy 0, policy_version 61480 (0.0010) [2023-10-11 21:30:30,541][71601] Updated weights for policy 0, policy_version 61490 (0.0008) [2023-10-11 21:30:30,922][71601] Updated weights for policy 0, policy_version 61500 (0.0009) [2023-10-11 21:30:31,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 125861888. Throughput: 0: 1817.2, 1: 1813.8. Samples: 31473018. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) [2023-10-11 21:30:31,035][70582] Avg episode reward: [(0, '40.760'), (1, '59.250')] [2023-10-11 21:30:32,372][71635] Updated weights for policy 1, policy_version 61442 (0.0008) [2023-10-11 21:30:32,743][71635] Updated weights for policy 1, policy_version 61452 (0.0008) [2023-10-11 21:30:33,122][71635] Updated weights for policy 1, policy_version 61462 (0.0009) [2023-10-11 21:30:33,490][71635] Updated weights for policy 1, policy_version 61472 (0.0008) [2023-10-11 21:30:34,549][71601] Updated weights for policy 0, policy_version 61510 (0.0008) [2023-10-11 21:30:34,921][71601] Updated weights for policy 0, policy_version 61520 (0.0009) [2023-10-11 21:30:35,296][71601] Updated weights for policy 0, policy_version 61530 (0.0008) [2023-10-11 21:30:36,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 125960192. Throughput: 0: 1823.3, 1: 1805.6. Samples: 31495034. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:30:36,034][70582] Avg episode reward: [(0, '41.980'), (1, '60.240')] [2023-10-11 21:30:37,187][71635] Updated weights for policy 1, policy_version 61482 (0.0008) [2023-10-11 21:30:37,569][71635] Updated weights for policy 1, policy_version 61492 (0.0008) [2023-10-11 21:30:37,935][71635] Updated weights for policy 1, policy_version 61502 (0.0010) [2023-10-11 21:30:38,886][71601] Updated weights for policy 0, policy_version 61540 (0.0007) [2023-10-11 21:30:39,249][71601] Updated weights for policy 0, policy_version 61550 (0.0008) [2023-10-11 21:30:39,624][71601] Updated weights for policy 0, policy_version 61560 (0.0008) [2023-10-11 21:30:41,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 126025728. Throughput: 0: 1819.4, 1: 1807.7. Samples: 31516456. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:30:41,034][70582] Avg episode reward: [(0, '43.420'), (1, '59.920')] [2023-10-11 21:30:41,684][71635] Updated weights for policy 1, policy_version 61512 (0.0010) [2023-10-11 21:30:42,047][71635] Updated weights for policy 1, policy_version 61522 (0.0008) [2023-10-11 21:30:42,417][71635] Updated weights for policy 1, policy_version 61532 (0.0009) [2023-10-11 21:30:43,225][71601] Updated weights for policy 0, policy_version 61570 (0.0008) [2023-10-11 21:30:43,603][71601] Updated weights for policy 0, policy_version 61580 (0.0007) [2023-10-11 21:30:43,961][71601] Updated weights for policy 0, policy_version 61590 (0.0007) [2023-10-11 21:30:44,339][71601] Updated weights for policy 0, policy_version 61600 (0.0008) [2023-10-11 21:30:46,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 126091264. Throughput: 0: 1820.7, 1: 1803.3. Samples: 31527656. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:30:46,034][70582] Avg episode reward: [(0, '45.970'), (1, '62.040')] [2023-10-11 21:30:46,188][71635] Updated weights for policy 1, policy_version 61542 (0.0008) [2023-10-11 21:30:46,558][71635] Updated weights for policy 1, policy_version 61552 (0.0007) [2023-10-11 21:30:46,924][71635] Updated weights for policy 1, policy_version 61562 (0.0007) [2023-10-11 21:30:48,054][71601] Updated weights for policy 0, policy_version 61610 (0.0008) [2023-10-11 21:30:48,432][71601] Updated weights for policy 0, policy_version 61620 (0.0007) [2023-10-11 21:30:48,816][71601] Updated weights for policy 0, policy_version 61630 (0.0008) [2023-10-11 21:30:50,712][71635] Updated weights for policy 1, policy_version 61572 (0.0008) [2023-10-11 21:30:51,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 126156800. Throughput: 0: 1822.1, 1: 1800.7. Samples: 31549198. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:30:51,035][70582] Avg episode reward: [(0, '47.290'), (1, '66.600')] [2023-10-11 21:30:51,079][71635] Updated weights for policy 1, policy_version 61582 (0.0009) [2023-10-11 21:30:51,454][71635] Updated weights for policy 1, policy_version 61592 (0.0007) [2023-10-11 21:30:52,448][71601] Updated weights for policy 0, policy_version 61640 (0.0008) [2023-10-11 21:30:52,827][71601] Updated weights for policy 0, policy_version 61650 (0.0007) [2023-10-11 21:30:53,196][71601] Updated weights for policy 0, policy_version 61660 (0.0007) [2023-10-11 21:30:54,958][71635] Updated weights for policy 1, policy_version 61602 (0.0007) [2023-10-11 21:30:55,333][71635] Updated weights for policy 1, policy_version 61612 (0.0008) [2023-10-11 21:30:55,701][71635] Updated weights for policy 1, policy_version 61622 (0.0008) [2023-10-11 21:30:56,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 126222336. Throughput: 0: 1825.5, 1: 1815.4. Samples: 31571700. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:30:56,034][70582] Avg episode reward: [(0, '45.680'), (1, '64.760')] [2023-10-11 21:30:56,067][71635] Updated weights for policy 1, policy_version 61632 (0.0010) [2023-10-11 21:30:56,836][71601] Updated weights for policy 0, policy_version 61670 (0.0009) [2023-10-11 21:30:57,215][71601] Updated weights for policy 0, policy_version 61680 (0.0008) [2023-10-11 21:30:57,589][71601] Updated weights for policy 0, policy_version 61690 (0.0008) [2023-10-11 21:30:59,722][71635] Updated weights for policy 1, policy_version 61642 (0.0011) [2023-10-11 21:31:00,096][71635] Updated weights for policy 1, policy_version 61652 (0.0011) [2023-10-11 21:31:00,456][71635] Updated weights for policy 1, policy_version 61662 (0.0008) [2023-10-11 21:31:01,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 126320640. Throughput: 0: 1830.3, 1: 1818.2. Samples: 31582484. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:31:01,035][70582] Avg episode reward: [(0, '46.490'), (1, '62.600')] [2023-10-11 21:31:01,409][71601] Updated weights for policy 0, policy_version 61700 (0.0008) [2023-10-11 21:31:01,776][71601] Updated weights for policy 0, policy_version 61710 (0.0008) [2023-10-11 21:31:02,153][71601] Updated weights for policy 0, policy_version 61720 (0.0007) [2023-10-11 21:31:04,049][71635] Updated weights for policy 1, policy_version 61672 (0.0010) [2023-10-11 21:31:04,424][71635] Updated weights for policy 1, policy_version 61682 (0.0008) [2023-10-11 21:31:04,793][71635] Updated weights for policy 1, policy_version 61692 (0.0007) [2023-10-11 21:31:05,874][71601] Updated weights for policy 0, policy_version 61730 (0.0008) [2023-10-11 21:31:06,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 126386176. Throughput: 0: 1827.2, 1: 1820.8. Samples: 31604514. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:31:06,035][70582] Avg episode reward: [(0, '48.460'), (1, '66.450')] [2023-10-11 21:31:06,248][71601] Updated weights for policy 0, policy_version 61740 (0.0007) [2023-10-11 21:31:06,620][71601] Updated weights for policy 0, policy_version 61750 (0.0008) [2023-10-11 21:31:06,991][71601] Updated weights for policy 0, policy_version 61760 (0.0007) [2023-10-11 21:31:08,430][71635] Updated weights for policy 1, policy_version 61702 (0.0008) [2023-10-11 21:31:08,790][71635] Updated weights for policy 1, policy_version 61712 (0.0009) [2023-10-11 21:31:09,160][71635] Updated weights for policy 1, policy_version 61722 (0.0009) [2023-10-11 21:31:10,597][71601] Updated weights for policy 0, policy_version 61770 (0.0009) [2023-10-11 21:31:10,966][71601] Updated weights for policy 0, policy_version 61780 (0.0009) [2023-10-11 21:31:11,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 126451712. Throughput: 0: 1820.3, 1: 1826.0. Samples: 31626412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:31:11,034][70582] Avg episode reward: [(0, '46.470'), (1, '70.010')] [2023-10-11 21:31:11,334][71601] Updated weights for policy 0, policy_version 61790 (0.0009) [2023-10-11 21:31:12,787][71635] Updated weights for policy 1, policy_version 61732 (0.0009) [2023-10-11 21:31:13,153][71635] Updated weights for policy 1, policy_version 61742 (0.0007) [2023-10-11 21:31:13,520][71635] Updated weights for policy 1, policy_version 61752 (0.0008) [2023-10-11 21:31:14,988][71601] Updated weights for policy 0, policy_version 61800 (0.0010) [2023-10-11 21:31:15,363][71601] Updated weights for policy 0, policy_version 61810 (0.0008) [2023-10-11 21:31:15,745][71601] Updated weights for policy 0, policy_version 61820 (0.0010) [2023-10-11 21:31:16,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 126550016. Throughput: 0: 1826.7, 1: 1825.3. Samples: 31637358. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-11 21:31:16,035][70582] Avg episode reward: [(0, '44.020'), (1, '73.600')] [2023-10-11 21:31:17,163][71635] Updated weights for policy 1, policy_version 61762 (0.0008) [2023-10-11 21:31:17,522][71635] Updated weights for policy 1, policy_version 61772 (0.0010) [2023-10-11 21:31:17,896][71635] Updated weights for policy 1, policy_version 61782 (0.0009) [2023-10-11 21:31:18,266][71635] Updated weights for policy 1, policy_version 61792 (0.0010) [2023-10-11 21:31:19,317][71601] Updated weights for policy 0, policy_version 61830 (0.0008) [2023-10-11 21:31:19,685][71601] Updated weights for policy 0, policy_version 61840 (0.0009) [2023-10-11 21:31:20,049][71601] Updated weights for policy 0, policy_version 61850 (0.0008) [2023-10-11 21:31:21,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 126615552. Throughput: 0: 1818.5, 1: 1838.7. Samples: 31659608. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-11 21:31:21,035][70582] Avg episode reward: [(0, '39.470'), (1, '71.320')] [2023-10-11 21:31:22,006][71635] Updated weights for policy 1, policy_version 61802 (0.0008) [2023-10-11 21:31:22,387][71635] Updated weights for policy 1, policy_version 61812 (0.0007) [2023-10-11 21:31:22,761][71635] Updated weights for policy 1, policy_version 61822 (0.0008) [2023-10-11 21:31:23,715][71601] Updated weights for policy 0, policy_version 61860 (0.0008) [2023-10-11 21:31:24,085][71601] Updated weights for policy 0, policy_version 61870 (0.0009) [2023-10-11 21:31:24,461][71601] Updated weights for policy 0, policy_version 61880 (0.0009) [2023-10-11 21:31:26,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 126681088. Throughput: 0: 1827.9, 1: 1837.4. Samples: 31681396. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-11 21:31:26,035][70582] Avg episode reward: [(0, '41.080'), (1, '70.290')] [2023-10-11 21:31:26,509][71635] Updated weights for policy 1, policy_version 61832 (0.0008) [2023-10-11 21:31:26,881][71635] Updated weights for policy 1, policy_version 61842 (0.0008) [2023-10-11 21:31:27,248][71635] Updated weights for policy 1, policy_version 61852 (0.0008) [2023-10-11 21:31:28,041][71601] Updated weights for policy 0, policy_version 61890 (0.0009) [2023-10-11 21:31:28,408][71601] Updated weights for policy 0, policy_version 61900 (0.0009) [2023-10-11 21:31:28,784][71601] Updated weights for policy 0, policy_version 61910 (0.0010) [2023-10-11 21:31:29,150][71601] Updated weights for policy 0, policy_version 61920 (0.0009) [2023-10-11 21:31:31,016][71635] Updated weights for policy 1, policy_version 61862 (0.0010) [2023-10-11 21:31:31,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 126746624. Throughput: 0: 1819.0, 1: 1834.6. Samples: 31692068. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-11 21:31:31,034][70582] Avg episode reward: [(0, '40.550'), (1, '65.960')] [2023-10-11 21:31:31,385][71635] Updated weights for policy 1, policy_version 61872 (0.0007) [2023-10-11 21:31:31,762][71635] Updated weights for policy 1, policy_version 61882 (0.0007) [2023-10-11 21:31:32,976][71601] Updated weights for policy 0, policy_version 61930 (0.0009) [2023-10-11 21:31:33,347][71601] Updated weights for policy 0, policy_version 61940 (0.0008) [2023-10-11 21:31:33,729][71601] Updated weights for policy 0, policy_version 61950 (0.0007) [2023-10-11 21:31:35,498][71635] Updated weights for policy 1, policy_version 61892 (0.0009) [2023-10-11 21:31:35,866][71635] Updated weights for policy 1, policy_version 61902 (0.0008) [2023-10-11 21:31:36,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 126812160. Throughput: 0: 1827.3, 1: 1833.8. Samples: 31713948. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-11 21:31:36,034][70582] Avg episode reward: [(0, '37.280'), (1, '63.340')] [2023-10-11 21:31:36,238][71635] Updated weights for policy 1, policy_version 61912 (0.0008) [2023-10-11 21:31:37,402][71601] Updated weights for policy 0, policy_version 61960 (0.0010) [2023-10-11 21:31:37,763][71601] Updated weights for policy 0, policy_version 61970 (0.0011) [2023-10-11 21:31:38,138][71601] Updated weights for policy 0, policy_version 61980 (0.0011) [2023-10-11 21:31:39,741][71635] Updated weights for policy 1, policy_version 61922 (0.0007) [2023-10-11 21:31:40,107][71635] Updated weights for policy 1, policy_version 61932 (0.0007) [2023-10-11 21:31:40,477][71635] Updated weights for policy 1, policy_version 61942 (0.0010) [2023-10-11 21:31:40,850][71635] Updated weights for policy 1, policy_version 61952 (0.0009) [2023-10-11 21:31:41,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 126910464. Throughput: 0: 1828.0, 1: 1821.6. Samples: 31735934. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-11 21:31:41,034][70582] Avg episode reward: [(0, '36.980'), (1, '67.670')] [2023-10-11 21:31:41,807][71601] Updated weights for policy 0, policy_version 61990 (0.0009) [2023-10-11 21:31:42,179][71601] Updated weights for policy 0, policy_version 62000 (0.0010) [2023-10-11 21:31:42,555][71601] Updated weights for policy 0, policy_version 62010 (0.0009) [2023-10-11 21:31:44,559][71635] Updated weights for policy 1, policy_version 61962 (0.0007) [2023-10-11 21:31:44,926][71635] Updated weights for policy 1, policy_version 61972 (0.0009) [2023-10-11 21:31:45,294][71635] Updated weights for policy 1, policy_version 61982 (0.0010) [2023-10-11 21:31:46,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 126976000. Throughput: 0: 1826.0, 1: 1822.5. Samples: 31746664. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-11 21:31:46,035][70582] Avg episode reward: [(0, '37.640'), (1, '67.510')] [2023-10-11 21:31:46,069][71601] Updated weights for policy 0, policy_version 62020 (0.0007) [2023-10-11 21:31:46,446][71601] Updated weights for policy 0, policy_version 62030 (0.0007) [2023-10-11 21:31:46,811][71601] Updated weights for policy 0, policy_version 62040 (0.0009) [2023-10-11 21:31:49,050][71635] Updated weights for policy 1, policy_version 61992 (0.0010) [2023-10-11 21:31:49,418][71635] Updated weights for policy 1, policy_version 62002 (0.0009) [2023-10-11 21:31:49,790][71635] Updated weights for policy 1, policy_version 62012 (0.0009) [2023-10-11 21:31:50,375][71601] Updated weights for policy 0, policy_version 62050 (0.0008) [2023-10-11 21:31:50,747][71601] Updated weights for policy 0, policy_version 62060 (0.0008) [2023-10-11 21:31:51,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 127041536. Throughput: 0: 1834.4, 1: 1816.9. Samples: 31768818. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-11 21:31:51,034][70582] Avg episode reward: [(0, '39.030'), (1, '65.540')] [2023-10-11 21:31:51,126][71601] Updated weights for policy 0, policy_version 62070 (0.0009) [2023-10-11 21:31:51,498][71601] Updated weights for policy 0, policy_version 62080 (0.0008) [2023-10-11 21:31:53,426][71635] Updated weights for policy 1, policy_version 62022 (0.0008) [2023-10-11 21:31:53,790][71635] Updated weights for policy 1, policy_version 62032 (0.0007) [2023-10-11 21:31:54,155][71635] Updated weights for policy 1, policy_version 62042 (0.0009) [2023-10-11 21:31:55,305][71601] Updated weights for policy 0, policy_version 62090 (0.0008) [2023-10-11 21:31:55,678][71601] Updated weights for policy 0, policy_version 62100 (0.0009) [2023-10-11 21:31:56,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 127107072. Throughput: 0: 1828.7, 1: 1812.8. Samples: 31790282. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-11 21:31:56,034][70582] Avg episode reward: [(0, '37.540'), (1, '66.960')] [2023-10-11 21:31:56,041][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000062048_63537152.pth... [2023-10-11 21:31:56,064][71601] Updated weights for policy 0, policy_version 62110 (0.0007) [2023-10-11 21:31:56,076][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000060352_61800448.pth [2023-10-11 21:31:56,135][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000062112_63602688.pth... [2023-10-11 21:31:56,171][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000060384_61833216.pth [2023-10-11 21:31:57,775][71635] Updated weights for policy 1, policy_version 62052 (0.0012) [2023-10-11 21:31:58,140][71635] Updated weights for policy 1, policy_version 62062 (0.0008) [2023-10-11 21:31:58,503][71635] Updated weights for policy 1, policy_version 62072 (0.0007) [2023-10-11 21:31:59,636][71601] Updated weights for policy 0, policy_version 62120 (0.0009) [2023-10-11 21:32:00,008][71601] Updated weights for policy 0, policy_version 62130 (0.0008) [2023-10-11 21:32:00,381][71601] Updated weights for policy 0, policy_version 62140 (0.0007) [2023-10-11 21:32:01,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 127205376. Throughput: 0: 1835.0, 1: 1812.0. Samples: 31801474. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-11 21:32:01,034][70582] Avg episode reward: [(0, '37.580'), (1, '73.260')] [2023-10-11 21:32:02,223][71635] Updated weights for policy 1, policy_version 62082 (0.0008) [2023-10-11 21:32:02,592][71635] Updated weights for policy 1, policy_version 62092 (0.0011) [2023-10-11 21:32:02,963][71635] Updated weights for policy 1, policy_version 62102 (0.0009) [2023-10-11 21:32:03,336][71635] Updated weights for policy 1, policy_version 62112 (0.0011) [2023-10-11 21:32:04,171][71601] Updated weights for policy 0, policy_version 62150 (0.0008) [2023-10-11 21:32:04,550][71601] Updated weights for policy 0, policy_version 62160 (0.0011) [2023-10-11 21:32:04,923][71601] Updated weights for policy 0, policy_version 62170 (0.0009) [2023-10-11 21:32:06,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 127270912. Throughput: 0: 1824.8, 1: 1804.2. Samples: 31822914. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-11 21:32:06,035][70582] Avg episode reward: [(0, '39.470'), (1, '74.470')] [2023-10-11 21:32:07,161][71635] Updated weights for policy 1, policy_version 62122 (0.0009) [2023-10-11 21:32:07,527][71635] Updated weights for policy 1, policy_version 62132 (0.0008) [2023-10-11 21:32:07,895][71635] Updated weights for policy 1, policy_version 62142 (0.0007) [2023-10-11 21:32:08,751][71601] Updated weights for policy 0, policy_version 62180 (0.0009) [2023-10-11 21:32:09,124][71601] Updated weights for policy 0, policy_version 62190 (0.0009) [2023-10-11 21:32:09,494][71601] Updated weights for policy 0, policy_version 62200 (0.0009) [2023-10-11 21:32:11,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 127336448. Throughput: 0: 1823.6, 1: 1804.1. Samples: 31844644. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-11 21:32:11,035][70582] Avg episode reward: [(0, '41.000'), (1, '79.640')] [2023-10-11 21:32:11,632][71635] Updated weights for policy 1, policy_version 62152 (0.0008) [2023-10-11 21:32:12,003][71635] Updated weights for policy 1, policy_version 62162 (0.0008) [2023-10-11 21:32:12,355][71635] Updated weights for policy 1, policy_version 62172 (0.0008) [2023-10-11 21:32:13,136][71601] Updated weights for policy 0, policy_version 62210 (0.0009) [2023-10-11 21:32:13,509][71601] Updated weights for policy 0, policy_version 62220 (0.0008) [2023-10-11 21:32:13,885][71601] Updated weights for policy 0, policy_version 62230 (0.0008) [2023-10-11 21:32:14,248][71601] Updated weights for policy 0, policy_version 62240 (0.0009) [2023-10-11 21:32:16,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 127401984. Throughput: 0: 1826.4, 1: 1806.9. Samples: 31855566. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-11 21:32:16,034][70582] Avg episode reward: [(0, '43.560'), (1, '80.550')] [2023-10-11 21:32:16,069][71635] Updated weights for policy 1, policy_version 62182 (0.0010) [2023-10-11 21:32:16,431][71635] Updated weights for policy 1, policy_version 62192 (0.0009) [2023-10-11 21:32:16,803][71635] Updated weights for policy 1, policy_version 62202 (0.0009) [2023-10-11 21:32:18,024][71601] Updated weights for policy 0, policy_version 62250 (0.0010) [2023-10-11 21:32:18,400][71601] Updated weights for policy 0, policy_version 62260 (0.0007) [2023-10-11 21:32:18,761][71601] Updated weights for policy 0, policy_version 62270 (0.0009) [2023-10-11 21:32:20,393][71635] Updated weights for policy 1, policy_version 62212 (0.0009) [2023-10-11 21:32:20,763][71635] Updated weights for policy 1, policy_version 62222 (0.0007) [2023-10-11 21:32:21,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 127467520. Throughput: 0: 1821.5, 1: 1810.5. Samples: 31877390. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-11 21:32:21,035][70582] Avg episode reward: [(0, '45.030'), (1, '77.680')] [2023-10-11 21:32:21,136][71635] Updated weights for policy 1, policy_version 62232 (0.0009) [2023-10-11 21:32:22,328][71601] Updated weights for policy 0, policy_version 62280 (0.0008) [2023-10-11 21:32:22,706][71601] Updated weights for policy 0, policy_version 62290 (0.0009) [2023-10-11 21:32:23,074][71601] Updated weights for policy 0, policy_version 62300 (0.0007) [2023-10-11 21:32:24,831][71635] Updated weights for policy 1, policy_version 62242 (0.0010) [2023-10-11 21:32:25,200][71635] Updated weights for policy 1, policy_version 62252 (0.0008) [2023-10-11 21:32:25,567][71635] Updated weights for policy 1, policy_version 62262 (0.0009) [2023-10-11 21:32:25,928][71635] Updated weights for policy 1, policy_version 62272 (0.0010) [2023-10-11 21:32:26,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 127565824. Throughput: 0: 1822.8, 1: 1818.3. Samples: 31899784. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-11 21:32:26,034][70582] Avg episode reward: [(0, '48.310'), (1, '77.840')] [2023-10-11 21:32:26,817][71601] Updated weights for policy 0, policy_version 62310 (0.0009) [2023-10-11 21:32:27,190][71601] Updated weights for policy 0, policy_version 62320 (0.0011) [2023-10-11 21:32:27,565][71601] Updated weights for policy 0, policy_version 62330 (0.0010) [2023-10-11 21:32:29,700][71635] Updated weights for policy 1, policy_version 62282 (0.0008) [2023-10-11 21:32:30,074][71635] Updated weights for policy 1, policy_version 62292 (0.0008) [2023-10-11 21:32:30,443][71635] Updated weights for policy 1, policy_version 62302 (0.0008) [2023-10-11 21:32:31,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 127631360. Throughput: 0: 1821.5, 1: 1816.4. Samples: 31910368. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-11 21:32:31,034][70582] Avg episode reward: [(0, '50.750'), (1, '75.600')] [2023-10-11 21:32:31,080][71601] Updated weights for policy 0, policy_version 62340 (0.0009) [2023-10-11 21:32:31,453][71601] Updated weights for policy 0, policy_version 62350 (0.0009) [2023-10-11 21:32:31,827][71601] Updated weights for policy 0, policy_version 62360 (0.0008) [2023-10-11 21:32:34,281][71635] Updated weights for policy 1, policy_version 62312 (0.0007) [2023-10-11 21:32:34,645][71635] Updated weights for policy 1, policy_version 62322 (0.0007) [2023-10-11 21:32:35,005][71635] Updated weights for policy 1, policy_version 62332 (0.0007) [2023-10-11 21:32:35,607][71601] Updated weights for policy 0, policy_version 62370 (0.0009) [2023-10-11 21:32:35,972][71601] Updated weights for policy 0, policy_version 62380 (0.0007) [2023-10-11 21:32:36,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 127696896. Throughput: 0: 1822.7, 1: 1823.4. Samples: 31932896. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-11 21:32:36,035][70582] Avg episode reward: [(0, '53.160'), (1, '75.690')] [2023-10-11 21:32:36,359][71601] Updated weights for policy 0, policy_version 62390 (0.0007) [2023-10-11 21:32:36,728][71601] Updated weights for policy 0, policy_version 62400 (0.0008) [2023-10-11 21:32:38,695][71635] Updated weights for policy 1, policy_version 62342 (0.0008) [2023-10-11 21:32:39,055][71635] Updated weights for policy 1, policy_version 62352 (0.0008) [2023-10-11 21:32:39,419][71635] Updated weights for policy 1, policy_version 62362 (0.0008) [2023-10-11 21:32:40,523][71601] Updated weights for policy 0, policy_version 62410 (0.0009) [2023-10-11 21:32:40,889][71601] Updated weights for policy 0, policy_version 62420 (0.0007) [2023-10-11 21:32:41,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 127762432. Throughput: 0: 1825.3, 1: 1819.1. Samples: 31954278. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:32:41,035][70582] Avg episode reward: [(0, '53.220'), (1, '76.610')] [2023-10-11 21:32:41,262][71601] Updated weights for policy 0, policy_version 62430 (0.0007) [2023-10-11 21:32:42,994][71635] Updated weights for policy 1, policy_version 62372 (0.0009) [2023-10-11 21:32:43,366][71635] Updated weights for policy 1, policy_version 62382 (0.0009) [2023-10-11 21:32:43,747][71635] Updated weights for policy 1, policy_version 62392 (0.0010) [2023-10-11 21:32:45,000][71601] Updated weights for policy 0, policy_version 62440 (0.0009) [2023-10-11 21:32:45,372][71601] Updated weights for policy 0, policy_version 62450 (0.0010) [2023-10-11 21:32:45,736][71601] Updated weights for policy 0, policy_version 62460 (0.0008) [2023-10-11 21:32:46,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 127860736. Throughput: 0: 1814.0, 1: 1829.3. Samples: 31965426. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:32:46,034][70582] Avg episode reward: [(0, '52.380'), (1, '73.270')] [2023-10-11 21:32:47,430][71635] Updated weights for policy 1, policy_version 62402 (0.0008) [2023-10-11 21:32:47,802][71635] Updated weights for policy 1, policy_version 62412 (0.0011) [2023-10-11 21:32:48,175][71635] Updated weights for policy 1, policy_version 62422 (0.0008) [2023-10-11 21:32:48,534][71635] Updated weights for policy 1, policy_version 62432 (0.0008) [2023-10-11 21:32:49,351][71601] Updated weights for policy 0, policy_version 62470 (0.0008) [2023-10-11 21:32:49,723][71601] Updated weights for policy 0, policy_version 62480 (0.0009) [2023-10-11 21:32:50,085][71601] Updated weights for policy 0, policy_version 62490 (0.0009) [2023-10-11 21:32:51,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 127926272. Throughput: 0: 1819.2, 1: 1823.7. Samples: 31986842. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:32:51,034][70582] Avg episode reward: [(0, '52.800'), (1, '71.360')] [2023-10-11 21:32:52,307][71635] Updated weights for policy 1, policy_version 62442 (0.0009) [2023-10-11 21:32:52,678][71635] Updated weights for policy 1, policy_version 62452 (0.0010) [2023-10-11 21:32:53,060][71635] Updated weights for policy 1, policy_version 62462 (0.0007) [2023-10-11 21:32:53,680][71601] Updated weights for policy 0, policy_version 62500 (0.0008) [2023-10-11 21:32:54,053][71601] Updated weights for policy 0, policy_version 62510 (0.0007) [2023-10-11 21:32:54,424][71601] Updated weights for policy 0, policy_version 62520 (0.0008) [2023-10-11 21:32:56,034][70582] Fps is (10 sec: 13106.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 127991808. Throughput: 0: 1815.6, 1: 1820.6. Samples: 32008270. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:32:56,035][70582] Avg episode reward: [(0, '47.480'), (1, '69.750')] [2023-10-11 21:32:56,892][71635] Updated weights for policy 1, policy_version 62472 (0.0008) [2023-10-11 21:32:57,253][71635] Updated weights for policy 1, policy_version 62482 (0.0008) [2023-10-11 21:32:57,621][71635] Updated weights for policy 1, policy_version 62492 (0.0009) [2023-10-11 21:32:58,060][71601] Updated weights for policy 0, policy_version 62530 (0.0008) [2023-10-11 21:32:58,422][71601] Updated weights for policy 0, policy_version 62540 (0.0009) [2023-10-11 21:32:58,800][71601] Updated weights for policy 0, policy_version 62550 (0.0010) [2023-10-11 21:32:59,169][71601] Updated weights for policy 0, policy_version 62560 (0.0010) [2023-10-11 21:33:01,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 128057344. Throughput: 0: 1817.1, 1: 1823.5. Samples: 32019390. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:33:01,034][70582] Avg episode reward: [(0, '44.030'), (1, '68.010')] [2023-10-11 21:33:01,269][71635] Updated weights for policy 1, policy_version 62502 (0.0008) [2023-10-11 21:33:01,649][71635] Updated weights for policy 1, policy_version 62512 (0.0008) [2023-10-11 21:33:02,016][71635] Updated weights for policy 1, policy_version 62522 (0.0008) [2023-10-11 21:33:02,912][71601] Updated weights for policy 0, policy_version 62570 (0.0007) [2023-10-11 21:33:03,288][71601] Updated weights for policy 0, policy_version 62580 (0.0007) [2023-10-11 21:33:03,666][71601] Updated weights for policy 0, policy_version 62590 (0.0009) [2023-10-11 21:33:05,619][71635] Updated weights for policy 1, policy_version 62532 (0.0009) [2023-10-11 21:33:05,988][71635] Updated weights for policy 1, policy_version 62542 (0.0009) [2023-10-11 21:33:06,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 128122880. Throughput: 0: 1819.3, 1: 1822.6. Samples: 32041278. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:33:06,035][70582] Avg episode reward: [(0, '42.500'), (1, '70.750')] [2023-10-11 21:33:06,360][71635] Updated weights for policy 1, policy_version 62552 (0.0009) [2023-10-11 21:33:07,247][71601] Updated weights for policy 0, policy_version 62600 (0.0008) [2023-10-11 21:33:07,613][71601] Updated weights for policy 0, policy_version 62610 (0.0010) [2023-10-11 21:33:07,987][71601] Updated weights for policy 0, policy_version 62620 (0.0009) [2023-10-11 21:33:09,991][71635] Updated weights for policy 1, policy_version 62562 (0.0010) [2023-10-11 21:33:10,354][71635] Updated weights for policy 1, policy_version 62572 (0.0010) [2023-10-11 21:33:10,715][71635] Updated weights for policy 1, policy_version 62582 (0.0010) [2023-10-11 21:33:11,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 128188416. Throughput: 0: 1815.0, 1: 1823.2. Samples: 32063502. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:33:11,034][70582] Avg episode reward: [(0, '41.600'), (1, '75.300')] [2023-10-11 21:33:11,077][71635] Updated weights for policy 1, policy_version 62592 (0.0009) [2023-10-11 21:33:11,712][71601] Updated weights for policy 0, policy_version 62630 (0.0010) [2023-10-11 21:33:12,093][71601] Updated weights for policy 0, policy_version 62640 (0.0007) [2023-10-11 21:33:12,458][71601] Updated weights for policy 0, policy_version 62650 (0.0009) [2023-10-11 21:33:14,823][71635] Updated weights for policy 1, policy_version 62602 (0.0009) [2023-10-11 21:33:15,195][71635] Updated weights for policy 1, policy_version 62612 (0.0008) [2023-10-11 21:33:15,557][71635] Updated weights for policy 1, policy_version 62622 (0.0007) [2023-10-11 21:33:16,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 128286720. Throughput: 0: 1818.6, 1: 1817.0. Samples: 32073968. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:33:16,035][70582] Avg episode reward: [(0, '39.440'), (1, '72.650')] [2023-10-11 21:33:16,187][71601] Updated weights for policy 0, policy_version 62660 (0.0008) [2023-10-11 21:33:16,552][71601] Updated weights for policy 0, policy_version 62670 (0.0008) [2023-10-11 21:33:16,934][71601] Updated weights for policy 0, policy_version 62680 (0.0008) [2023-10-11 21:33:19,301][71635] Updated weights for policy 1, policy_version 62632 (0.0009) [2023-10-11 21:33:19,666][71635] Updated weights for policy 1, policy_version 62642 (0.0008) [2023-10-11 21:33:20,031][71635] Updated weights for policy 1, policy_version 62652 (0.0008) [2023-10-11 21:33:20,603][71601] Updated weights for policy 0, policy_version 62690 (0.0007) [2023-10-11 21:33:20,970][71601] Updated weights for policy 0, policy_version 62700 (0.0008) [2023-10-11 21:33:21,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 128352256. Throughput: 0: 1814.9, 1: 1819.8. Samples: 32096456. Policy #0 lag: (min: 8.0, avg: 33.1, max: 40.0) [2023-10-11 21:33:21,034][70582] Avg episode reward: [(0, '39.910'), (1, '69.980')] [2023-10-11 21:33:21,333][71601] Updated weights for policy 0, policy_version 62710 (0.0007) [2023-10-11 21:33:21,698][71601] Updated weights for policy 0, policy_version 62720 (0.0008) [2023-10-11 21:33:23,695][71635] Updated weights for policy 1, policy_version 62662 (0.0009) [2023-10-11 21:33:24,059][71635] Updated weights for policy 1, policy_version 62672 (0.0010) [2023-10-11 21:33:24,428][71635] Updated weights for policy 1, policy_version 62682 (0.0009) [2023-10-11 21:33:25,457][71601] Updated weights for policy 0, policy_version 62730 (0.0007) [2023-10-11 21:33:25,835][71601] Updated weights for policy 0, policy_version 62740 (0.0009) [2023-10-11 21:33:26,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 128417792. Throughput: 0: 1816.1, 1: 1819.3. Samples: 32117868. Policy #0 lag: (min: 8.0, avg: 33.1, max: 40.0) [2023-10-11 21:33:26,034][70582] Avg episode reward: [(0, '39.560'), (1, '71.000')] [2023-10-11 21:33:26,215][71601] Updated weights for policy 0, policy_version 62750 (0.0008) [2023-10-11 21:33:28,125][71635] Updated weights for policy 1, policy_version 62692 (0.0009) [2023-10-11 21:33:28,499][71635] Updated weights for policy 1, policy_version 62702 (0.0007) [2023-10-11 21:33:28,867][71635] Updated weights for policy 1, policy_version 62712 (0.0009) [2023-10-11 21:33:29,938][71601] Updated weights for policy 0, policy_version 62760 (0.0007) [2023-10-11 21:33:30,299][71601] Updated weights for policy 0, policy_version 62770 (0.0007) [2023-10-11 21:33:30,671][71601] Updated weights for policy 0, policy_version 62780 (0.0007) [2023-10-11 21:33:31,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 128516096. Throughput: 0: 1820.1, 1: 1816.1. Samples: 32129054. Policy #0 lag: (min: 8.0, avg: 33.1, max: 40.0) [2023-10-11 21:33:31,034][70582] Avg episode reward: [(0, '42.520'), (1, '74.170')] [2023-10-11 21:33:32,611][71635] Updated weights for policy 1, policy_version 62722 (0.0008) [2023-10-11 21:33:32,981][71635] Updated weights for policy 1, policy_version 62732 (0.0007) [2023-10-11 21:33:33,338][71635] Updated weights for policy 1, policy_version 62742 (0.0010) [2023-10-11 21:33:33,706][71635] Updated weights for policy 1, policy_version 62752 (0.0009) [2023-10-11 21:33:34,312][71601] Updated weights for policy 0, policy_version 62790 (0.0007) [2023-10-11 21:33:34,677][71601] Updated weights for policy 0, policy_version 62800 (0.0007) [2023-10-11 21:33:35,041][71601] Updated weights for policy 0, policy_version 62810 (0.0008) [2023-10-11 21:33:36,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 128581632. Throughput: 0: 1825.7, 1: 1814.6. Samples: 32150656. Policy #0 lag: (min: 8.0, avg: 33.1, max: 40.0) [2023-10-11 21:33:36,034][70582] Avg episode reward: [(0, '42.260'), (1, '71.570')] [2023-10-11 21:33:37,445][71635] Updated weights for policy 1, policy_version 62762 (0.0008) [2023-10-11 21:33:37,821][71635] Updated weights for policy 1, policy_version 62772 (0.0007) [2023-10-11 21:33:38,192][71635] Updated weights for policy 1, policy_version 62782 (0.0008) [2023-10-11 21:33:38,516][71601] Updated weights for policy 0, policy_version 62820 (0.0008) [2023-10-11 21:33:38,887][71601] Updated weights for policy 0, policy_version 62830 (0.0009) [2023-10-11 21:33:39,265][71601] Updated weights for policy 0, policy_version 62840 (0.0009) [2023-10-11 21:33:41,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 128647168. Throughput: 0: 1831.2, 1: 1821.9. Samples: 32172658. Policy #0 lag: (min: 8.0, avg: 33.1, max: 40.0) [2023-10-11 21:33:41,034][70582] Avg episode reward: [(0, '41.700'), (1, '72.670')] [2023-10-11 21:33:41,648][71635] Updated weights for policy 1, policy_version 62792 (0.0008) [2023-10-11 21:33:42,020][71635] Updated weights for policy 1, policy_version 62802 (0.0008) [2023-10-11 21:33:42,384][71635] Updated weights for policy 1, policy_version 62812 (0.0008) [2023-10-11 21:33:42,889][71601] Updated weights for policy 0, policy_version 62850 (0.0008) [2023-10-11 21:33:43,256][71601] Updated weights for policy 0, policy_version 62860 (0.0007) [2023-10-11 21:33:43,633][71601] Updated weights for policy 0, policy_version 62870 (0.0008) [2023-10-11 21:33:44,001][71601] Updated weights for policy 0, policy_version 62880 (0.0010) [2023-10-11 21:33:46,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 128712704. Throughput: 0: 1823.9, 1: 1821.1. Samples: 32183414. Policy #0 lag: (min: 8.0, avg: 33.1, max: 40.0) [2023-10-11 21:33:46,034][71635] Updated weights for policy 1, policy_version 62822 (0.0009) [2023-10-11 21:33:46,034][70582] Avg episode reward: [(0, '41.840'), (1, '73.410')] [2023-10-11 21:33:46,401][71635] Updated weights for policy 1, policy_version 62832 (0.0007) [2023-10-11 21:33:46,766][71635] Updated weights for policy 1, policy_version 62842 (0.0008) [2023-10-11 21:33:47,814][71601] Updated weights for policy 0, policy_version 62890 (0.0009) [2023-10-11 21:33:48,191][71601] Updated weights for policy 0, policy_version 62900 (0.0010) [2023-10-11 21:33:48,569][71601] Updated weights for policy 0, policy_version 62910 (0.0009) [2023-10-11 21:33:50,544][71635] Updated weights for policy 1, policy_version 62852 (0.0008) [2023-10-11 21:33:50,914][71635] Updated weights for policy 1, policy_version 62862 (0.0008) [2023-10-11 21:33:51,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 128778240. Throughput: 0: 1827.4, 1: 1821.7. Samples: 32205488. Policy #0 lag: (min: 8.0, avg: 33.1, max: 40.0) [2023-10-11 21:33:51,034][70582] Avg episode reward: [(0, '41.340'), (1, '69.360')] [2023-10-11 21:33:51,278][71635] Updated weights for policy 1, policy_version 62872 (0.0007) [2023-10-11 21:33:52,156][71601] Updated weights for policy 0, policy_version 62920 (0.0008) [2023-10-11 21:33:52,541][71601] Updated weights for policy 0, policy_version 62930 (0.0008) [2023-10-11 21:33:52,903][71601] Updated weights for policy 0, policy_version 62940 (0.0008) [2023-10-11 21:33:54,826][71635] Updated weights for policy 1, policy_version 62882 (0.0008) [2023-10-11 21:33:55,196][71635] Updated weights for policy 1, policy_version 62892 (0.0007) [2023-10-11 21:33:55,563][71635] Updated weights for policy 1, policy_version 62902 (0.0008) [2023-10-11 21:33:55,924][71635] Updated weights for policy 1, policy_version 62912 (0.0007) [2023-10-11 21:33:56,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 128876544. Throughput: 0: 1830.2, 1: 1822.8. Samples: 32227884. Policy #0 lag: (min: 8.0, avg: 33.1, max: 40.0) [2023-10-11 21:33:56,034][70582] Avg episode reward: [(0, '45.880'), (1, '69.540')] [2023-10-11 21:33:56,043][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000062944_64454656.pth... [2023-10-11 21:33:56,043][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000062912_64421888.pth... [2023-10-11 21:33:56,073][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000061248_62717952.pth [2023-10-11 21:33:56,082][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000061184_62652416.pth [2023-10-11 21:33:56,445][71601] Updated weights for policy 0, policy_version 62950 (0.0007) [2023-10-11 21:33:56,819][71601] Updated weights for policy 0, policy_version 62960 (0.0009) [2023-10-11 21:33:57,197][71601] Updated weights for policy 0, policy_version 62970 (0.0008) [2023-10-11 21:33:59,571][71635] Updated weights for policy 1, policy_version 62922 (0.0008) [2023-10-11 21:33:59,936][71635] Updated weights for policy 1, policy_version 62932 (0.0009) [2023-10-11 21:34:00,303][71635] Updated weights for policy 1, policy_version 62942 (0.0008) [2023-10-11 21:34:00,903][71601] Updated weights for policy 0, policy_version 62980 (0.0008) [2023-10-11 21:34:01,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 128942080. Throughput: 0: 1828.0, 1: 1833.1. Samples: 32238714. Policy #0 lag: (min: 15.0, avg: 23.2, max: 47.0) [2023-10-11 21:34:01,035][70582] Avg episode reward: [(0, '47.120'), (1, '72.340')] [2023-10-11 21:34:01,273][71601] Updated weights for policy 0, policy_version 62990 (0.0008) [2023-10-11 21:34:01,652][71601] Updated weights for policy 0, policy_version 63000 (0.0008) [2023-10-11 21:34:03,845][71635] Updated weights for policy 1, policy_version 62952 (0.0009) [2023-10-11 21:34:04,208][71635] Updated weights for policy 1, policy_version 62962 (0.0010) [2023-10-11 21:34:04,574][71635] Updated weights for policy 1, policy_version 62972 (0.0009) [2023-10-11 21:34:05,407][71601] Updated weights for policy 0, policy_version 63010 (0.0008) [2023-10-11 21:34:05,786][71601] Updated weights for policy 0, policy_version 63020 (0.0008) [2023-10-11 21:34:06,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 129007616. Throughput: 0: 1827.9, 1: 1825.3. Samples: 32260848. Policy #0 lag: (min: 15.0, avg: 23.2, max: 47.0) [2023-10-11 21:34:06,034][70582] Avg episode reward: [(0, '46.680'), (1, '71.990')] [2023-10-11 21:34:06,148][71601] Updated weights for policy 0, policy_version 63030 (0.0008) [2023-10-11 21:34:06,527][71601] Updated weights for policy 0, policy_version 63040 (0.0007) [2023-10-11 21:34:08,135][71635] Updated weights for policy 1, policy_version 62982 (0.0008) [2023-10-11 21:34:08,498][71635] Updated weights for policy 1, policy_version 62992 (0.0009) [2023-10-11 21:34:08,860][71635] Updated weights for policy 1, policy_version 63002 (0.0011) [2023-10-11 21:34:10,349][71601] Updated weights for policy 0, policy_version 63050 (0.0008) [2023-10-11 21:34:10,734][71601] Updated weights for policy 0, policy_version 63060 (0.0010) [2023-10-11 21:34:11,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 129073152. Throughput: 0: 1820.2, 1: 1843.3. Samples: 32282724. Policy #0 lag: (min: 15.0, avg: 23.2, max: 47.0) [2023-10-11 21:34:11,034][70582] Avg episode reward: [(0, '45.840'), (1, '69.210')] [2023-10-11 21:34:11,089][71601] Updated weights for policy 0, policy_version 63070 (0.0008) [2023-10-11 21:34:12,498][71635] Updated weights for policy 1, policy_version 63012 (0.0010) [2023-10-11 21:34:12,876][71635] Updated weights for policy 1, policy_version 63022 (0.0007) [2023-10-11 21:34:13,238][71635] Updated weights for policy 1, policy_version 63032 (0.0008) [2023-10-11 21:34:14,904][71601] Updated weights for policy 0, policy_version 63080 (0.0007) [2023-10-11 21:34:15,274][71601] Updated weights for policy 0, policy_version 63090 (0.0007) [2023-10-11 21:34:15,637][71601] Updated weights for policy 0, policy_version 63100 (0.0007) [2023-10-11 21:34:16,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 129171456. Throughput: 0: 1821.9, 1: 1830.1. Samples: 32293396. Policy #0 lag: (min: 15.0, avg: 23.2, max: 47.0) [2023-10-11 21:34:16,035][70582] Avg episode reward: [(0, '47.460'), (1, '71.920')] [2023-10-11 21:34:16,894][71635] Updated weights for policy 1, policy_version 63042 (0.0009) [2023-10-11 21:34:17,252][71635] Updated weights for policy 1, policy_version 63052 (0.0011) [2023-10-11 21:34:17,618][71635] Updated weights for policy 1, policy_version 63062 (0.0008) [2023-10-11 21:34:17,985][71635] Updated weights for policy 1, policy_version 63072 (0.0007) [2023-10-11 21:34:19,320][71601] Updated weights for policy 0, policy_version 63110 (0.0009) [2023-10-11 21:34:19,687][71601] Updated weights for policy 0, policy_version 63120 (0.0010) [2023-10-11 21:34:20,055][71601] Updated weights for policy 0, policy_version 63130 (0.0007) [2023-10-11 21:34:21,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 129236992. Throughput: 0: 1819.2, 1: 1845.0. Samples: 32315546. Policy #0 lag: (min: 15.0, avg: 23.2, max: 47.0) [2023-10-11 21:34:21,035][70582] Avg episode reward: [(0, '46.950'), (1, '75.530')] [2023-10-11 21:34:21,737][71635] Updated weights for policy 1, policy_version 63082 (0.0009) [2023-10-11 21:34:22,113][71635] Updated weights for policy 1, policy_version 63092 (0.0012) [2023-10-11 21:34:22,469][71635] Updated weights for policy 1, policy_version 63102 (0.0011) [2023-10-11 21:34:23,760][71601] Updated weights for policy 0, policy_version 63140 (0.0009) [2023-10-11 21:34:24,131][71601] Updated weights for policy 0, policy_version 63150 (0.0010) [2023-10-11 21:34:24,511][71601] Updated weights for policy 0, policy_version 63160 (0.0009) [2023-10-11 21:34:26,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 129302528. Throughput: 0: 1813.2, 1: 1850.6. Samples: 32337532. Policy #0 lag: (min: 15.0, avg: 23.2, max: 47.0) [2023-10-11 21:34:26,035][70582] Avg episode reward: [(0, '47.540'), (1, '74.240')] [2023-10-11 21:34:26,179][71635] Updated weights for policy 1, policy_version 63112 (0.0009) [2023-10-11 21:34:26,556][71635] Updated weights for policy 1, policy_version 63122 (0.0008) [2023-10-11 21:34:26,922][71635] Updated weights for policy 1, policy_version 63132 (0.0009) [2023-10-11 21:34:28,305][71601] Updated weights for policy 0, policy_version 63170 (0.0007) [2023-10-11 21:34:28,687][71601] Updated weights for policy 0, policy_version 63180 (0.0007) [2023-10-11 21:34:29,056][71601] Updated weights for policy 0, policy_version 63190 (0.0009) [2023-10-11 21:34:29,430][71601] Updated weights for policy 0, policy_version 63200 (0.0008) [2023-10-11 21:34:30,583][71635] Updated weights for policy 1, policy_version 63142 (0.0008) [2023-10-11 21:34:30,945][71635] Updated weights for policy 1, policy_version 63152 (0.0011) [2023-10-11 21:34:31,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 129368064. Throughput: 0: 1820.8, 1: 1846.2. Samples: 32348430. Policy #0 lag: (min: 15.0, avg: 23.2, max: 47.0) [2023-10-11 21:34:31,034][70582] Avg episode reward: [(0, '44.950'), (1, '75.080')] [2023-10-11 21:34:31,312][71635] Updated weights for policy 1, policy_version 63162 (0.0009) [2023-10-11 21:34:33,092][71601] Updated weights for policy 0, policy_version 63210 (0.0007) [2023-10-11 21:34:33,463][71601] Updated weights for policy 0, policy_version 63220 (0.0007) [2023-10-11 21:34:33,835][71601] Updated weights for policy 0, policy_version 63230 (0.0008) [2023-10-11 21:34:34,983][71635] Updated weights for policy 1, policy_version 63172 (0.0008) [2023-10-11 21:34:35,349][71635] Updated weights for policy 1, policy_version 63182 (0.0008) [2023-10-11 21:34:35,713][71635] Updated weights for policy 1, policy_version 63192 (0.0008) [2023-10-11 21:34:36,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 129466368. Throughput: 0: 1816.3, 1: 1843.7. Samples: 32370190. Policy #0 lag: (min: 15.0, avg: 23.2, max: 47.0) [2023-10-11 21:34:36,035][70582] Avg episode reward: [(0, '42.650'), (1, '78.480')] [2023-10-11 21:34:37,470][71601] Updated weights for policy 0, policy_version 63240 (0.0008) [2023-10-11 21:34:37,842][71601] Updated weights for policy 0, policy_version 63250 (0.0007) [2023-10-11 21:34:38,227][71601] Updated weights for policy 0, policy_version 63260 (0.0009) [2023-10-11 21:34:39,531][71635] Updated weights for policy 1, policy_version 63202 (0.0007) [2023-10-11 21:34:39,895][71635] Updated weights for policy 1, policy_version 63212 (0.0008) [2023-10-11 21:34:40,255][71635] Updated weights for policy 1, policy_version 63222 (0.0008) [2023-10-11 21:34:40,620][71635] Updated weights for policy 1, policy_version 63232 (0.0008) [2023-10-11 21:34:41,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 129531904. Throughput: 0: 1818.4, 1: 1820.8. Samples: 32391650. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-11 21:34:41,034][70582] Avg episode reward: [(0, '44.060'), (1, '76.860')] [2023-10-11 21:34:41,815][71601] Updated weights for policy 0, policy_version 63270 (0.0008) [2023-10-11 21:34:42,190][71601] Updated weights for policy 0, policy_version 63280 (0.0007) [2023-10-11 21:34:42,563][71601] Updated weights for policy 0, policy_version 63290 (0.0007) [2023-10-11 21:34:44,495][71635] Updated weights for policy 1, policy_version 63242 (0.0008) [2023-10-11 21:34:44,864][71635] Updated weights for policy 1, policy_version 63252 (0.0008) [2023-10-11 21:34:45,230][71635] Updated weights for policy 1, policy_version 63262 (0.0008) [2023-10-11 21:34:46,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 129597440. Throughput: 0: 1821.9, 1: 1821.1. Samples: 32402646. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-11 21:34:46,035][70582] Avg episode reward: [(0, '45.730'), (1, '77.510')] [2023-10-11 21:34:46,076][71601] Updated weights for policy 0, policy_version 63300 (0.0008) [2023-10-11 21:34:46,459][71601] Updated weights for policy 0, policy_version 63310 (0.0010) [2023-10-11 21:34:46,824][71601] Updated weights for policy 0, policy_version 63320 (0.0010) [2023-10-11 21:34:49,045][71635] Updated weights for policy 1, policy_version 63272 (0.0007) [2023-10-11 21:34:49,422][71635] Updated weights for policy 1, policy_version 63282 (0.0008) [2023-10-11 21:34:49,791][71635] Updated weights for policy 1, policy_version 63292 (0.0009) [2023-10-11 21:34:50,452][71601] Updated weights for policy 0, policy_version 63330 (0.0007) [2023-10-11 21:34:50,825][71601] Updated weights for policy 0, policy_version 63340 (0.0009) [2023-10-11 21:34:51,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 129662976. Throughput: 0: 1824.2, 1: 1815.0. Samples: 32424612. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-11 21:34:51,034][70582] Avg episode reward: [(0, '45.190'), (1, '78.720')] [2023-10-11 21:34:51,207][71601] Updated weights for policy 0, policy_version 63350 (0.0008) [2023-10-11 21:34:51,582][71601] Updated weights for policy 0, policy_version 63360 (0.0008) [2023-10-11 21:34:53,563][71635] Updated weights for policy 1, policy_version 63302 (0.0007) [2023-10-11 21:34:53,927][71635] Updated weights for policy 1, policy_version 63312 (0.0007) [2023-10-11 21:34:54,288][71635] Updated weights for policy 1, policy_version 63322 (0.0009) [2023-10-11 21:34:55,313][71601] Updated weights for policy 0, policy_version 63370 (0.0010) [2023-10-11 21:34:55,693][71601] Updated weights for policy 0, policy_version 63380 (0.0010) [2023-10-11 21:34:56,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 129728512. Throughput: 0: 1825.2, 1: 1803.9. Samples: 32446032. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-11 21:34:56,034][70582] Avg episode reward: [(0, '44.920'), (1, '76.340')] [2023-10-11 21:34:56,060][71601] Updated weights for policy 0, policy_version 63390 (0.0009) [2023-10-11 21:34:57,929][71635] Updated weights for policy 1, policy_version 63332 (0.0009) [2023-10-11 21:34:58,289][71635] Updated weights for policy 1, policy_version 63342 (0.0008) [2023-10-11 21:34:58,657][71635] Updated weights for policy 1, policy_version 63352 (0.0008) [2023-10-11 21:34:59,713][71601] Updated weights for policy 0, policy_version 63400 (0.0009) [2023-10-11 21:35:00,078][71601] Updated weights for policy 0, policy_version 63410 (0.0008) [2023-10-11 21:35:00,459][71601] Updated weights for policy 0, policy_version 63420 (0.0010) [2023-10-11 21:35:01,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 129826816. Throughput: 0: 1831.2, 1: 1813.9. Samples: 32457424. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-11 21:35:01,034][70582] Avg episode reward: [(0, '46.560'), (1, '76.330')] [2023-10-11 21:35:02,233][71635] Updated weights for policy 1, policy_version 63362 (0.0009) [2023-10-11 21:35:02,591][71635] Updated weights for policy 1, policy_version 63372 (0.0009) [2023-10-11 21:35:02,958][71635] Updated weights for policy 1, policy_version 63382 (0.0009) [2023-10-11 21:35:03,324][71635] Updated weights for policy 1, policy_version 63392 (0.0007) [2023-10-11 21:35:04,231][71601] Updated weights for policy 0, policy_version 63430 (0.0009) [2023-10-11 21:35:04,608][71601] Updated weights for policy 0, policy_version 63440 (0.0008) [2023-10-11 21:35:04,981][71601] Updated weights for policy 0, policy_version 63450 (0.0007) [2023-10-11 21:35:06,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 129892352. Throughput: 0: 1822.0, 1: 1807.1. Samples: 32478854. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-11 21:35:06,035][70582] Avg episode reward: [(0, '46.840'), (1, '73.910')] [2023-10-11 21:35:06,980][71635] Updated weights for policy 1, policy_version 63402 (0.0009) [2023-10-11 21:35:07,347][71635] Updated weights for policy 1, policy_version 63412 (0.0008) [2023-10-11 21:35:07,726][71635] Updated weights for policy 1, policy_version 63422 (0.0008) [2023-10-11 21:35:08,725][71601] Updated weights for policy 0, policy_version 63460 (0.0008) [2023-10-11 21:35:09,099][71601] Updated weights for policy 0, policy_version 63470 (0.0009) [2023-10-11 21:35:09,480][71601] Updated weights for policy 0, policy_version 63480 (0.0009) [2023-10-11 21:35:11,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 129957888. Throughput: 0: 1825.9, 1: 1799.9. Samples: 32500692. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-11 21:35:11,035][70582] Avg episode reward: [(0, '47.730'), (1, '73.240')] [2023-10-11 21:35:11,704][71635] Updated weights for policy 1, policy_version 63432 (0.0008) [2023-10-11 21:35:12,085][71635] Updated weights for policy 1, policy_version 63442 (0.0010) [2023-10-11 21:35:12,451][71635] Updated weights for policy 1, policy_version 63452 (0.0008) [2023-10-11 21:35:13,113][71601] Updated weights for policy 0, policy_version 63490 (0.0008) [2023-10-11 21:35:13,485][71601] Updated weights for policy 0, policy_version 63500 (0.0007) [2023-10-11 21:35:13,859][71601] Updated weights for policy 0, policy_version 63510 (0.0007) [2023-10-11 21:35:14,223][71601] Updated weights for policy 0, policy_version 63520 (0.0007) [2023-10-11 21:35:15,997][71635] Updated weights for policy 1, policy_version 63462 (0.0008) [2023-10-11 21:35:16,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 130023424. Throughput: 0: 1830.0, 1: 1799.4. Samples: 32511752. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-10-11 21:35:16,034][70582] Avg episode reward: [(0, '46.100'), (1, '71.480')] [2023-10-11 21:35:16,366][71635] Updated weights for policy 1, policy_version 63472 (0.0007) [2023-10-11 21:35:16,730][71635] Updated weights for policy 1, policy_version 63482 (0.0008) [2023-10-11 21:35:17,808][71601] Updated weights for policy 0, policy_version 63530 (0.0009) [2023-10-11 21:35:18,187][71601] Updated weights for policy 0, policy_version 63540 (0.0009) [2023-10-11 21:35:18,562][71601] Updated weights for policy 0, policy_version 63550 (0.0009) [2023-10-11 21:35:20,537][71635] Updated weights for policy 1, policy_version 63492 (0.0008) [2023-10-11 21:35:20,910][71635] Updated weights for policy 1, policy_version 63502 (0.0008) [2023-10-11 21:35:21,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 130088960. Throughput: 0: 1830.4, 1: 1804.5. Samples: 32533760. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 21:35:21,035][70582] Avg episode reward: [(0, '47.440'), (1, '72.260')] [2023-10-11 21:35:21,284][71635] Updated weights for policy 1, policy_version 63512 (0.0009) [2023-10-11 21:35:22,256][71601] Updated weights for policy 0, policy_version 63560 (0.0008) [2023-10-11 21:35:22,617][71601] Updated weights for policy 0, policy_version 63570 (0.0009) [2023-10-11 21:35:23,001][71601] Updated weights for policy 0, policy_version 63580 (0.0007) [2023-10-11 21:35:24,904][71635] Updated weights for policy 1, policy_version 63522 (0.0008) [2023-10-11 21:35:25,272][71635] Updated weights for policy 1, policy_version 63532 (0.0007) [2023-10-11 21:35:25,633][71635] Updated weights for policy 1, policy_version 63542 (0.0010) [2023-10-11 21:35:26,007][71635] Updated weights for policy 1, policy_version 63552 (0.0009) [2023-10-11 21:35:26,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 130187264. Throughput: 0: 1827.6, 1: 1825.5. Samples: 32556036. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 21:35:26,034][70582] Avg episode reward: [(0, '50.710'), (1, '74.250')] [2023-10-11 21:35:26,594][71601] Updated weights for policy 0, policy_version 63590 (0.0009) [2023-10-11 21:35:26,955][71601] Updated weights for policy 0, policy_version 63600 (0.0011) [2023-10-11 21:35:27,325][71601] Updated weights for policy 0, policy_version 63610 (0.0010) [2023-10-11 21:35:29,496][71635] Updated weights for policy 1, policy_version 63562 (0.0010) [2023-10-11 21:35:29,867][71635] Updated weights for policy 1, policy_version 63572 (0.0010) [2023-10-11 21:35:30,231][71635] Updated weights for policy 1, policy_version 63582 (0.0008) [2023-10-11 21:35:31,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 130252800. Throughput: 0: 1824.3, 1: 1822.3. Samples: 32566744. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 21:35:31,034][70582] Avg episode reward: [(0, '52.650'), (1, '78.470')] [2023-10-11 21:35:31,146][71601] Updated weights for policy 0, policy_version 63620 (0.0008) [2023-10-11 21:35:31,525][71601] Updated weights for policy 0, policy_version 63630 (0.0007) [2023-10-11 21:35:31,896][71601] Updated weights for policy 0, policy_version 63640 (0.0010) [2023-10-11 21:35:33,819][71635] Updated weights for policy 1, policy_version 63592 (0.0007) [2023-10-11 21:35:34,193][71635] Updated weights for policy 1, policy_version 63602 (0.0009) [2023-10-11 21:35:34,558][71635] Updated weights for policy 1, policy_version 63612 (0.0007) [2023-10-11 21:35:35,535][71601] Updated weights for policy 0, policy_version 63650 (0.0010) [2023-10-11 21:35:35,896][71601] Updated weights for policy 0, policy_version 63660 (0.0007) [2023-10-11 21:35:36,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 130318336. Throughput: 0: 1820.2, 1: 1823.1. Samples: 32588558. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 21:35:36,035][70582] Avg episode reward: [(0, '57.260'), (1, '77.540')] [2023-10-11 21:35:36,271][71601] Updated weights for policy 0, policy_version 63670 (0.0007) [2023-10-11 21:35:36,646][71601] Updated weights for policy 0, policy_version 63680 (0.0008) [2023-10-11 21:35:38,264][71635] Updated weights for policy 1, policy_version 63622 (0.0008) [2023-10-11 21:35:38,632][71635] Updated weights for policy 1, policy_version 63632 (0.0008) [2023-10-11 21:35:38,999][71635] Updated weights for policy 1, policy_version 63642 (0.0010) [2023-10-11 21:35:40,518][71601] Updated weights for policy 0, policy_version 63690 (0.0011) [2023-10-11 21:35:40,885][71601] Updated weights for policy 0, policy_version 63700 (0.0010) [2023-10-11 21:35:41,034][70582] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 130383872. Throughput: 0: 1819.6, 1: 1824.0. Samples: 32609994. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 21:35:41,035][70582] Avg episode reward: [(0, '56.360'), (1, '77.620')] [2023-10-11 21:35:41,253][71601] Updated weights for policy 0, policy_version 63710 (0.0009) [2023-10-11 21:35:42,819][71635] Updated weights for policy 1, policy_version 63652 (0.0008) [2023-10-11 21:35:43,186][71635] Updated weights for policy 1, policy_version 63662 (0.0007) [2023-10-11 21:35:43,565][71635] Updated weights for policy 1, policy_version 63672 (0.0008) [2023-10-11 21:35:44,916][71601] Updated weights for policy 0, policy_version 63720 (0.0007) [2023-10-11 21:35:45,286][71601] Updated weights for policy 0, policy_version 63730 (0.0007) [2023-10-11 21:35:45,657][71601] Updated weights for policy 0, policy_version 63740 (0.0008) [2023-10-11 21:35:46,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 130482176. Throughput: 0: 1811.1, 1: 1820.2. Samples: 32620832. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 21:35:46,034][70582] Avg episode reward: [(0, '58.750'), (1, '80.260')] [2023-10-11 21:35:47,286][71635] Updated weights for policy 1, policy_version 63682 (0.0008) [2023-10-11 21:35:47,659][71635] Updated weights for policy 1, policy_version 63692 (0.0008) [2023-10-11 21:35:48,021][71635] Updated weights for policy 1, policy_version 63702 (0.0007) [2023-10-11 21:35:48,382][71635] Updated weights for policy 1, policy_version 63712 (0.0007) [2023-10-11 21:35:49,338][71601] Updated weights for policy 0, policy_version 63750 (0.0008) [2023-10-11 21:35:49,701][71601] Updated weights for policy 0, policy_version 63760 (0.0008) [2023-10-11 21:35:50,075][71601] Updated weights for policy 0, policy_version 63770 (0.0007) [2023-10-11 21:35:51,034][70582] Fps is (10 sec: 16384.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 130547712. Throughput: 0: 1818.0, 1: 1817.2. Samples: 32642436. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 21:35:51,034][70582] Avg episode reward: [(0, '63.040'), (1, '79.440')] [2023-10-11 21:35:52,077][71635] Updated weights for policy 1, policy_version 63722 (0.0008) [2023-10-11 21:35:52,447][71635] Updated weights for policy 1, policy_version 63732 (0.0009) [2023-10-11 21:35:52,824][71635] Updated weights for policy 1, policy_version 63742 (0.0008) [2023-10-11 21:35:53,684][71601] Updated weights for policy 0, policy_version 63780 (0.0009) [2023-10-11 21:35:54,059][71601] Updated weights for policy 0, policy_version 63790 (0.0008) [2023-10-11 21:35:54,432][71601] Updated weights for policy 0, policy_version 63800 (0.0010) [2023-10-11 21:35:56,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 130613248. Throughput: 0: 1815.6, 1: 1819.4. Samples: 32664270. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 21:35:56,035][70582] Avg episode reward: [(0, '64.250'), (1, '76.700')] [2023-10-11 21:35:56,046][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000063808_65339392.pth... [2023-10-11 21:35:56,046][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000063744_65273856.pth... [2023-10-11 21:35:56,075][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000062112_63602688.pth [2023-10-11 21:35:56,084][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000062048_63537152.pth [2023-10-11 21:35:56,725][71635] Updated weights for policy 1, policy_version 63752 (0.0009) [2023-10-11 21:35:57,100][71635] Updated weights for policy 1, policy_version 63762 (0.0010) [2023-10-11 21:35:57,465][71635] Updated weights for policy 1, policy_version 63772 (0.0009) [2023-10-11 21:35:58,148][71601] Updated weights for policy 0, policy_version 63810 (0.0009) [2023-10-11 21:35:58,521][71601] Updated weights for policy 0, policy_version 63820 (0.0008) [2023-10-11 21:35:58,894][71601] Updated weights for policy 0, policy_version 63830 (0.0009) [2023-10-11 21:35:59,280][71601] Updated weights for policy 0, policy_version 63840 (0.0008) [2023-10-11 21:36:01,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 130678784. Throughput: 0: 1810.4, 1: 1819.6. Samples: 32675098. Policy #0 lag: (min: 17.0, avg: 25.0, max: 49.0) [2023-10-11 21:36:01,034][70582] Avg episode reward: [(0, '66.030'), (1, '75.100')] [2023-10-11 21:36:01,156][71635] Updated weights for policy 1, policy_version 63782 (0.0008) [2023-10-11 21:36:01,530][71635] Updated weights for policy 1, policy_version 63792 (0.0011) [2023-10-11 21:36:01,898][71635] Updated weights for policy 1, policy_version 63802 (0.0009) [2023-10-11 21:36:02,840][71601] Updated weights for policy 0, policy_version 63850 (0.0008) [2023-10-11 21:36:03,207][71601] Updated weights for policy 0, policy_version 63860 (0.0009) [2023-10-11 21:36:03,582][71601] Updated weights for policy 0, policy_version 63870 (0.0010) [2023-10-11 21:36:05,650][71635] Updated weights for policy 1, policy_version 63812 (0.0010) [2023-10-11 21:36:06,015][71635] Updated weights for policy 1, policy_version 63822 (0.0008) [2023-10-11 21:36:06,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 130744320. Throughput: 0: 1809.1, 1: 1814.1. Samples: 32696802. Policy #0 lag: (min: 17.0, avg: 25.0, max: 49.0) [2023-10-11 21:36:06,034][70582] Avg episode reward: [(0, '62.950'), (1, '70.960')] [2023-10-11 21:36:06,386][71635] Updated weights for policy 1, policy_version 63832 (0.0007) [2023-10-11 21:36:07,185][71601] Updated weights for policy 0, policy_version 63880 (0.0007) [2023-10-11 21:36:07,547][71601] Updated weights for policy 0, policy_version 63890 (0.0008) [2023-10-11 21:36:07,912][71601] Updated weights for policy 0, policy_version 63900 (0.0008) [2023-10-11 21:36:09,910][71635] Updated weights for policy 1, policy_version 63842 (0.0009) [2023-10-11 21:36:10,281][71635] Updated weights for policy 1, policy_version 63852 (0.0007) [2023-10-11 21:36:10,665][71635] Updated weights for policy 1, policy_version 63862 (0.0008) [2023-10-11 21:36:11,034][71635] Updated weights for policy 1, policy_version 63872 (0.0008) [2023-10-11 21:36:11,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 130842624. Throughput: 0: 1807.9, 1: 1810.9. Samples: 32718882. Policy #0 lag: (min: 17.0, avg: 25.0, max: 49.0) [2023-10-11 21:36:11,035][70582] Avg episode reward: [(0, '62.290'), (1, '64.380')] [2023-10-11 21:36:11,599][71601] Updated weights for policy 0, policy_version 63910 (0.0008) [2023-10-11 21:36:11,969][71601] Updated weights for policy 0, policy_version 63920 (0.0008) [2023-10-11 21:36:12,344][71601] Updated weights for policy 0, policy_version 63930 (0.0008) [2023-10-11 21:36:14,721][71635] Updated weights for policy 1, policy_version 63882 (0.0010) [2023-10-11 21:36:15,089][71635] Updated weights for policy 1, policy_version 63892 (0.0009) [2023-10-11 21:36:15,453][71635] Updated weights for policy 1, policy_version 63902 (0.0008) [2023-10-11 21:36:16,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 130908160. Throughput: 0: 1808.7, 1: 1805.7. Samples: 32729394. Policy #0 lag: (min: 17.0, avg: 25.0, max: 49.0) [2023-10-11 21:36:16,034][70582] Avg episode reward: [(0, '59.820'), (1, '65.140')] [2023-10-11 21:36:16,122][71601] Updated weights for policy 0, policy_version 63940 (0.0008) [2023-10-11 21:36:16,506][71601] Updated weights for policy 0, policy_version 63950 (0.0008) [2023-10-11 21:36:16,876][71601] Updated weights for policy 0, policy_version 63960 (0.0008) [2023-10-11 21:36:19,022][71635] Updated weights for policy 1, policy_version 63912 (0.0010) [2023-10-11 21:36:19,396][71635] Updated weights for policy 1, policy_version 63922 (0.0009) [2023-10-11 21:36:19,757][71635] Updated weights for policy 1, policy_version 63932 (0.0010) [2023-10-11 21:36:20,643][71601] Updated weights for policy 0, policy_version 63970 (0.0009) [2023-10-11 21:36:21,010][71601] Updated weights for policy 0, policy_version 63980 (0.0007) [2023-10-11 21:36:21,033][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 130973696. Throughput: 0: 1808.5, 1: 1812.6. Samples: 32751508. Policy #0 lag: (min: 17.0, avg: 25.0, max: 49.0) [2023-10-11 21:36:21,034][70582] Avg episode reward: [(0, '60.710'), (1, '63.900')] [2023-10-11 21:36:21,392][71601] Updated weights for policy 0, policy_version 63990 (0.0007) [2023-10-11 21:36:21,756][71601] Updated weights for policy 0, policy_version 64000 (0.0008) [2023-10-11 21:36:23,633][71635] Updated weights for policy 1, policy_version 63942 (0.0010) [2023-10-11 21:36:23,994][71635] Updated weights for policy 1, policy_version 63952 (0.0011) [2023-10-11 21:36:24,359][71635] Updated weights for policy 1, policy_version 63962 (0.0008) [2023-10-11 21:36:25,489][71601] Updated weights for policy 0, policy_version 64010 (0.0009) [2023-10-11 21:36:25,861][71601] Updated weights for policy 0, policy_version 64020 (0.0007) [2023-10-11 21:36:26,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 131039232. Throughput: 0: 1815.5, 1: 1811.2. Samples: 32773194. Policy #0 lag: (min: 17.0, avg: 25.0, max: 49.0) [2023-10-11 21:36:26,035][70582] Avg episode reward: [(0, '60.090'), (1, '66.110')] [2023-10-11 21:36:26,240][71601] Updated weights for policy 0, policy_version 64030 (0.0007) [2023-10-11 21:36:28,004][71635] Updated weights for policy 1, policy_version 63972 (0.0008) [2023-10-11 21:36:28,380][71635] Updated weights for policy 1, policy_version 63982 (0.0008) [2023-10-11 21:36:28,754][71635] Updated weights for policy 1, policy_version 63992 (0.0009) [2023-10-11 21:36:29,893][71601] Updated weights for policy 0, policy_version 64040 (0.0010) [2023-10-11 21:36:30,253][71601] Updated weights for policy 0, policy_version 64050 (0.0008) [2023-10-11 21:36:30,627][71601] Updated weights for policy 0, policy_version 64060 (0.0007) [2023-10-11 21:36:31,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 131137536. Throughput: 0: 1817.6, 1: 1817.4. Samples: 32784408. Policy #0 lag: (min: 17.0, avg: 25.0, max: 49.0) [2023-10-11 21:36:31,034][70582] Avg episode reward: [(0, '59.620'), (1, '65.860')] [2023-10-11 21:36:32,503][71635] Updated weights for policy 1, policy_version 64002 (0.0010) [2023-10-11 21:36:32,864][71635] Updated weights for policy 1, policy_version 64012 (0.0010) [2023-10-11 21:36:33,241][71635] Updated weights for policy 1, policy_version 64022 (0.0011) [2023-10-11 21:36:33,603][71635] Updated weights for policy 1, policy_version 64032 (0.0008) [2023-10-11 21:36:34,358][71601] Updated weights for policy 0, policy_version 64070 (0.0007) [2023-10-11 21:36:34,727][71601] Updated weights for policy 0, policy_version 64080 (0.0009) [2023-10-11 21:36:35,100][71601] Updated weights for policy 0, policy_version 64090 (0.0008) [2023-10-11 21:36:36,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 131203072. Throughput: 0: 1820.0, 1: 1815.5. Samples: 32806034. Policy #0 lag: (min: 17.0, avg: 25.0, max: 49.0) [2023-10-11 21:36:36,035][70582] Avg episode reward: [(0, '64.080'), (1, '65.630')] [2023-10-11 21:36:37,451][71635] Updated weights for policy 1, policy_version 64042 (0.0011) [2023-10-11 21:36:37,825][71635] Updated weights for policy 1, policy_version 64052 (0.0009) [2023-10-11 21:36:38,184][71635] Updated weights for policy 1, policy_version 64062 (0.0007) [2023-10-11 21:36:38,740][71601] Updated weights for policy 0, policy_version 64100 (0.0009) [2023-10-11 21:36:39,117][71601] Updated weights for policy 0, policy_version 64110 (0.0010) [2023-10-11 21:36:39,489][71601] Updated weights for policy 0, policy_version 64120 (0.0008) [2023-10-11 21:36:41,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 131268608. Throughput: 0: 1819.5, 1: 1810.0. Samples: 32827598. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-10-11 21:36:41,034][70582] Avg episode reward: [(0, '62.670'), (1, '67.270')] [2023-10-11 21:36:42,074][71635] Updated weights for policy 1, policy_version 64072 (0.0008) [2023-10-11 21:36:42,453][71635] Updated weights for policy 1, policy_version 64082 (0.0009) [2023-10-11 21:36:42,822][71635] Updated weights for policy 1, policy_version 64092 (0.0008) [2023-10-11 21:36:43,159][71601] Updated weights for policy 0, policy_version 64130 (0.0007) [2023-10-11 21:36:43,531][71601] Updated weights for policy 0, policy_version 64140 (0.0008) [2023-10-11 21:36:43,906][71601] Updated weights for policy 0, policy_version 64150 (0.0009) [2023-10-11 21:36:44,268][71601] Updated weights for policy 0, policy_version 64160 (0.0009) [2023-10-11 21:36:46,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 131334144. Throughput: 0: 1816.5, 1: 1808.5. Samples: 32838222. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-10-11 21:36:46,035][70582] Avg episode reward: [(0, '63.600'), (1, '70.850')] [2023-10-11 21:36:46,483][71635] Updated weights for policy 1, policy_version 64102 (0.0009) [2023-10-11 21:36:46,846][71635] Updated weights for policy 1, policy_version 64112 (0.0007) [2023-10-11 21:36:47,213][71635] Updated weights for policy 1, policy_version 64122 (0.0008) [2023-10-11 21:36:47,962][71601] Updated weights for policy 0, policy_version 64170 (0.0009) [2023-10-11 21:36:48,331][71601] Updated weights for policy 0, policy_version 64180 (0.0008) [2023-10-11 21:36:48,697][71601] Updated weights for policy 0, policy_version 64190 (0.0007) [2023-10-11 21:36:50,914][71635] Updated weights for policy 1, policy_version 64132 (0.0009) [2023-10-11 21:36:51,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 131399680. Throughput: 0: 1819.8, 1: 1809.3. Samples: 32860112. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-10-11 21:36:51,034][70582] Avg episode reward: [(0, '62.580'), (1, '70.060')] [2023-10-11 21:36:51,268][71635] Updated weights for policy 1, policy_version 64142 (0.0009) [2023-10-11 21:36:51,636][71635] Updated weights for policy 1, policy_version 64152 (0.0008) [2023-10-11 21:36:52,432][71601] Updated weights for policy 0, policy_version 64200 (0.0007) [2023-10-11 21:36:52,798][71601] Updated weights for policy 0, policy_version 64210 (0.0007) [2023-10-11 21:36:53,166][71601] Updated weights for policy 0, policy_version 64220 (0.0008) [2023-10-11 21:36:55,419][71635] Updated weights for policy 1, policy_version 64162 (0.0007) [2023-10-11 21:36:55,792][71635] Updated weights for policy 1, policy_version 64172 (0.0009) [2023-10-11 21:36:56,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 131465216. Throughput: 0: 1820.0, 1: 1823.8. Samples: 32882852. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-10-11 21:36:56,034][70582] Avg episode reward: [(0, '60.890'), (1, '68.330')] [2023-10-11 21:36:56,166][71635] Updated weights for policy 1, policy_version 64182 (0.0007) [2023-10-11 21:36:56,530][71635] Updated weights for policy 1, policy_version 64192 (0.0008) [2023-10-11 21:36:56,808][71601] Updated weights for policy 0, policy_version 64230 (0.0007) [2023-10-11 21:36:57,180][71601] Updated weights for policy 0, policy_version 64240 (0.0007) [2023-10-11 21:36:57,551][71601] Updated weights for policy 0, policy_version 64250 (0.0009) [2023-10-11 21:37:00,130][71635] Updated weights for policy 1, policy_version 64202 (0.0008) [2023-10-11 21:37:00,503][71635] Updated weights for policy 1, policy_version 64212 (0.0007) [2023-10-11 21:37:00,870][71635] Updated weights for policy 1, policy_version 64222 (0.0007) [2023-10-11 21:37:01,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 131563520. Throughput: 0: 1821.6, 1: 1816.3. Samples: 32893100. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-10-11 21:37:01,034][70582] Avg episode reward: [(0, '63.980'), (1, '67.600')] [2023-10-11 21:37:01,178][71601] Updated weights for policy 0, policy_version 64260 (0.0009) [2023-10-11 21:37:01,563][71601] Updated weights for policy 0, policy_version 64270 (0.0008) [2023-10-11 21:37:01,931][71601] Updated weights for policy 0, policy_version 64280 (0.0008) [2023-10-11 21:37:04,574][71635] Updated weights for policy 1, policy_version 64232 (0.0008) [2023-10-11 21:37:04,942][71635] Updated weights for policy 1, policy_version 64242 (0.0009) [2023-10-11 21:37:05,313][71635] Updated weights for policy 1, policy_version 64252 (0.0010) [2023-10-11 21:37:05,645][71601] Updated weights for policy 0, policy_version 64290 (0.0011) [2023-10-11 21:37:06,008][71601] Updated weights for policy 0, policy_version 64300 (0.0008) [2023-10-11 21:37:06,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 131629056. Throughput: 0: 1819.4, 1: 1823.4. Samples: 32915436. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-10-11 21:37:06,035][70582] Avg episode reward: [(0, '62.340'), (1, '67.520')] [2023-10-11 21:37:06,382][71601] Updated weights for policy 0, policy_version 64310 (0.0007) [2023-10-11 21:37:06,759][71601] Updated weights for policy 0, policy_version 64320 (0.0007) [2023-10-11 21:37:09,002][71635] Updated weights for policy 1, policy_version 64262 (0.0009) [2023-10-11 21:37:09,371][71635] Updated weights for policy 1, policy_version 64272 (0.0007) [2023-10-11 21:37:09,733][71635] Updated weights for policy 1, policy_version 64282 (0.0007) [2023-10-11 21:37:10,391][71601] Updated weights for policy 0, policy_version 64330 (0.0007) [2023-10-11 21:37:10,768][71601] Updated weights for policy 0, policy_version 64340 (0.0008) [2023-10-11 21:37:11,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 131694592. Throughput: 0: 1822.9, 1: 1808.5. Samples: 32936610. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-10-11 21:37:11,034][70582] Avg episode reward: [(0, '60.070'), (1, '69.590')] [2023-10-11 21:37:11,147][71601] Updated weights for policy 0, policy_version 64350 (0.0007) [2023-10-11 21:37:13,229][71635] Updated weights for policy 1, policy_version 64292 (0.0009) [2023-10-11 21:37:13,603][71635] Updated weights for policy 1, policy_version 64302 (0.0009) [2023-10-11 21:37:13,966][71635] Updated weights for policy 1, policy_version 64312 (0.0009) [2023-10-11 21:37:14,756][71601] Updated weights for policy 0, policy_version 64360 (0.0008) [2023-10-11 21:37:15,129][71601] Updated weights for policy 0, policy_version 64370 (0.0009) [2023-10-11 21:37:15,503][71601] Updated weights for policy 0, policy_version 64380 (0.0007) [2023-10-11 21:37:16,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 131792896. Throughput: 0: 1825.6, 1: 1814.8. Samples: 32948226. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-10-11 21:37:16,034][70582] Avg episode reward: [(0, '55.840'), (1, '70.140')] [2023-10-11 21:37:17,710][71635] Updated weights for policy 1, policy_version 64322 (0.0010) [2023-10-11 21:37:18,070][71635] Updated weights for policy 1, policy_version 64332 (0.0008) [2023-10-11 21:37:18,438][71635] Updated weights for policy 1, policy_version 64342 (0.0007) [2023-10-11 21:37:18,797][71635] Updated weights for policy 1, policy_version 64352 (0.0011) [2023-10-11 21:37:19,093][71601] Updated weights for policy 0, policy_version 64390 (0.0010) [2023-10-11 21:37:19,463][71601] Updated weights for policy 0, policy_version 64400 (0.0010) [2023-10-11 21:37:19,827][71601] Updated weights for policy 0, policy_version 64410 (0.0010) [2023-10-11 21:37:21,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 131858432. Throughput: 0: 1821.0, 1: 1807.8. Samples: 32969332. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-10-11 21:37:21,035][70582] Avg episode reward: [(0, '54.900'), (1, '74.110')] [2023-10-11 21:37:22,632][71635] Updated weights for policy 1, policy_version 64362 (0.0009) [2023-10-11 21:37:22,992][71635] Updated weights for policy 1, policy_version 64372 (0.0007) [2023-10-11 21:37:23,354][71635] Updated weights for policy 1, policy_version 64382 (0.0007) [2023-10-11 21:37:23,518][71601] Updated weights for policy 0, policy_version 64420 (0.0010) [2023-10-11 21:37:23,893][71601] Updated weights for policy 0, policy_version 64430 (0.0010) [2023-10-11 21:37:24,268][71601] Updated weights for policy 0, policy_version 64440 (0.0009) [2023-10-11 21:37:26,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 131923968. Throughput: 0: 1827.4, 1: 1812.2. Samples: 32991378. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:37:26,035][70582] Avg episode reward: [(0, '56.400'), (1, '69.810')] [2023-10-11 21:37:27,090][71635] Updated weights for policy 1, policy_version 64392 (0.0009) [2023-10-11 21:37:27,466][71635] Updated weights for policy 1, policy_version 64402 (0.0009) [2023-10-11 21:37:27,847][71635] Updated weights for policy 1, policy_version 64412 (0.0010) [2023-10-11 21:37:28,024][71601] Updated weights for policy 0, policy_version 64450 (0.0010) [2023-10-11 21:37:28,411][71601] Updated weights for policy 0, policy_version 64460 (0.0011) [2023-10-11 21:37:28,778][71601] Updated weights for policy 0, policy_version 64470 (0.0010) [2023-10-11 21:37:29,148][71601] Updated weights for policy 0, policy_version 64480 (0.0007) [2023-10-11 21:37:31,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 131989504. Throughput: 0: 1823.6, 1: 1816.5. Samples: 33002026. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:37:31,034][70582] Avg episode reward: [(0, '57.840'), (1, '71.390')] [2023-10-11 21:37:31,490][71635] Updated weights for policy 1, policy_version 64422 (0.0008) [2023-10-11 21:37:31,859][71635] Updated weights for policy 1, policy_version 64432 (0.0007) [2023-10-11 21:37:32,229][71635] Updated weights for policy 1, policy_version 64442 (0.0010) [2023-10-11 21:37:32,962][71601] Updated weights for policy 0, policy_version 64490 (0.0007) [2023-10-11 21:37:33,327][71601] Updated weights for policy 0, policy_version 64500 (0.0008) [2023-10-11 21:37:33,709][71601] Updated weights for policy 0, policy_version 64510 (0.0008) [2023-10-11 21:37:36,021][71635] Updated weights for policy 1, policy_version 64452 (0.0010) [2023-10-11 21:37:36,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 132055040. Throughput: 0: 1818.7, 1: 1813.1. Samples: 33023544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:37:36,035][70582] Avg episode reward: [(0, '58.690'), (1, '73.640')] [2023-10-11 21:37:36,391][71635] Updated weights for policy 1, policy_version 64462 (0.0008) [2023-10-11 21:37:36,764][71635] Updated weights for policy 1, policy_version 64472 (0.0008) [2023-10-11 21:37:37,311][71601] Updated weights for policy 0, policy_version 64520 (0.0009) [2023-10-11 21:37:37,683][71601] Updated weights for policy 0, policy_version 64530 (0.0010) [2023-10-11 21:37:38,055][71601] Updated weights for policy 0, policy_version 64540 (0.0007) [2023-10-11 21:37:40,521][71635] Updated weights for policy 1, policy_version 64482 (0.0008) [2023-10-11 21:37:40,892][71635] Updated weights for policy 1, policy_version 64492 (0.0008) [2023-10-11 21:37:41,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 132120576. Throughput: 0: 1818.7, 1: 1812.1. Samples: 33046238. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:37:41,034][70582] Avg episode reward: [(0, '55.780'), (1, '73.220')] [2023-10-11 21:37:41,259][71635] Updated weights for policy 1, policy_version 64502 (0.0008) [2023-10-11 21:37:41,624][71635] Updated weights for policy 1, policy_version 64512 (0.0009) [2023-10-11 21:37:41,827][71601] Updated weights for policy 0, policy_version 64550 (0.0007) [2023-10-11 21:37:42,192][71601] Updated weights for policy 0, policy_version 64560 (0.0007) [2023-10-11 21:37:42,562][71601] Updated weights for policy 0, policy_version 64570 (0.0007) [2023-10-11 21:37:45,364][71635] Updated weights for policy 1, policy_version 64522 (0.0010) [2023-10-11 21:37:45,735][71635] Updated weights for policy 1, policy_version 64532 (0.0008) [2023-10-11 21:37:46,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 132186112. Throughput: 0: 1817.5, 1: 1809.6. Samples: 33056320. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:37:46,035][70582] Avg episode reward: [(0, '50.610'), (1, '70.730')] [2023-10-11 21:37:46,103][71635] Updated weights for policy 1, policy_version 64542 (0.0008) [2023-10-11 21:37:46,361][71601] Updated weights for policy 0, policy_version 64580 (0.0009) [2023-10-11 21:37:46,744][71601] Updated weights for policy 0, policy_version 64590 (0.0008) [2023-10-11 21:37:47,102][71601] Updated weights for policy 0, policy_version 64600 (0.0010) [2023-10-11 21:37:49,781][71635] Updated weights for policy 1, policy_version 64552 (0.0010) [2023-10-11 21:37:50,152][71635] Updated weights for policy 1, policy_version 64562 (0.0010) [2023-10-11 21:37:50,515][71635] Updated weights for policy 1, policy_version 64572 (0.0010) [2023-10-11 21:37:50,854][71601] Updated weights for policy 0, policy_version 64610 (0.0008) [2023-10-11 21:37:51,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 132284416. Throughput: 0: 1818.0, 1: 1810.8. Samples: 33078732. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:37:51,035][70582] Avg episode reward: [(0, '50.510'), (1, '72.590')] [2023-10-11 21:37:51,221][71601] Updated weights for policy 0, policy_version 64620 (0.0010) [2023-10-11 21:37:51,583][71601] Updated weights for policy 0, policy_version 64630 (0.0008) [2023-10-11 21:37:51,955][71601] Updated weights for policy 0, policy_version 64640 (0.0008) [2023-10-11 21:37:54,226][71635] Updated weights for policy 1, policy_version 64582 (0.0008) [2023-10-11 21:37:54,598][71635] Updated weights for policy 1, policy_version 64592 (0.0008) [2023-10-11 21:37:54,962][71635] Updated weights for policy 1, policy_version 64602 (0.0009) [2023-10-11 21:37:55,609][71601] Updated weights for policy 0, policy_version 64650 (0.0007) [2023-10-11 21:37:55,982][71601] Updated weights for policy 0, policy_version 64660 (0.0008) [2023-10-11 21:37:56,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 132349952. Throughput: 0: 1815.4, 1: 1807.1. Samples: 33099622. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:37:56,035][70582] Avg episode reward: [(0, '49.640'), (1, '70.400')] [2023-10-11 21:37:56,044][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000064608_66158592.pth... [2023-10-11 21:37:56,080][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000062912_64421888.pth [2023-10-11 21:37:56,359][71601] Updated weights for policy 0, policy_version 64670 (0.0009) [2023-10-11 21:37:56,426][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000064672_66224128.pth... [2023-10-11 21:37:56,464][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000062944_64454656.pth [2023-10-11 21:37:58,596][71635] Updated weights for policy 1, policy_version 64612 (0.0008) [2023-10-11 21:37:58,969][71635] Updated weights for policy 1, policy_version 64622 (0.0008) [2023-10-11 21:37:59,333][71635] Updated weights for policy 1, policy_version 64632 (0.0011) [2023-10-11 21:38:00,017][71601] Updated weights for policy 0, policy_version 64680 (0.0009) [2023-10-11 21:38:00,400][71601] Updated weights for policy 0, policy_version 64690 (0.0010) [2023-10-11 21:38:00,771][71601] Updated weights for policy 0, policy_version 64700 (0.0009) [2023-10-11 21:38:01,034][70582] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 132448256. Throughput: 0: 1806.5, 1: 1816.8. Samples: 33111274. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:38:01,034][70582] Avg episode reward: [(0, '50.520'), (1, '66.630')] [2023-10-11 21:38:03,008][71635] Updated weights for policy 1, policy_version 64642 (0.0009) [2023-10-11 21:38:03,383][71635] Updated weights for policy 1, policy_version 64652 (0.0007) [2023-10-11 21:38:03,749][71635] Updated weights for policy 1, policy_version 64662 (0.0007) [2023-10-11 21:38:04,114][71635] Updated weights for policy 1, policy_version 64672 (0.0008) [2023-10-11 21:38:04,475][71601] Updated weights for policy 0, policy_version 64710 (0.0008) [2023-10-11 21:38:04,852][71601] Updated weights for policy 0, policy_version 64720 (0.0007) [2023-10-11 21:38:05,221][71601] Updated weights for policy 0, policy_version 64730 (0.0010) [2023-10-11 21:38:06,034][70582] Fps is (10 sec: 16384.6, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 132513792. Throughput: 0: 1815.4, 1: 1807.8. Samples: 33132374. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 21:38:06,034][70582] Avg episode reward: [(0, '49.190'), (1, '69.860')] [2023-10-11 21:38:07,799][71635] Updated weights for policy 1, policy_version 64682 (0.0009) [2023-10-11 21:38:08,164][71635] Updated weights for policy 1, policy_version 64692 (0.0010) [2023-10-11 21:38:08,534][71635] Updated weights for policy 1, policy_version 64702 (0.0008) [2023-10-11 21:38:08,910][71601] Updated weights for policy 0, policy_version 64740 (0.0009) [2023-10-11 21:38:09,278][71601] Updated weights for policy 0, policy_version 64750 (0.0008) [2023-10-11 21:38:09,648][71601] Updated weights for policy 0, policy_version 64760 (0.0011) [2023-10-11 21:38:11,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 132579328. Throughput: 0: 1800.3, 1: 1812.8. Samples: 33153966. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 21:38:11,035][70582] Avg episode reward: [(0, '49.690'), (1, '71.360')] [2023-10-11 21:38:12,309][71635] Updated weights for policy 1, policy_version 64712 (0.0007) [2023-10-11 21:38:12,688][71635] Updated weights for policy 1, policy_version 64722 (0.0007) [2023-10-11 21:38:13,057][71635] Updated weights for policy 1, policy_version 64732 (0.0009) [2023-10-11 21:38:13,355][71601] Updated weights for policy 0, policy_version 64770 (0.0010) [2023-10-11 21:38:13,728][71601] Updated weights for policy 0, policy_version 64780 (0.0009) [2023-10-11 21:38:14,107][71601] Updated weights for policy 0, policy_version 64790 (0.0009) [2023-10-11 21:38:14,476][71601] Updated weights for policy 0, policy_version 64800 (0.0007) [2023-10-11 21:38:16,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 132644864. Throughput: 0: 1812.3, 1: 1809.6. Samples: 33165012. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 21:38:16,034][70582] Avg episode reward: [(0, '50.150'), (1, '76.040')] [2023-10-11 21:38:16,716][71635] Updated weights for policy 1, policy_version 64742 (0.0009) [2023-10-11 21:38:17,080][71635] Updated weights for policy 1, policy_version 64752 (0.0009) [2023-10-11 21:38:17,450][71635] Updated weights for policy 1, policy_version 64762 (0.0009) [2023-10-11 21:38:18,181][71601] Updated weights for policy 0, policy_version 64810 (0.0007) [2023-10-11 21:38:18,558][71601] Updated weights for policy 0, policy_version 64820 (0.0008) [2023-10-11 21:38:18,925][71601] Updated weights for policy 0, policy_version 64830 (0.0008) [2023-10-11 21:38:21,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 132710400. Throughput: 0: 1813.2, 1: 1811.3. Samples: 33186642. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 21:38:21,034][70582] Avg episode reward: [(0, '54.600'), (1, '77.890')] [2023-10-11 21:38:21,113][71635] Updated weights for policy 1, policy_version 64772 (0.0010) [2023-10-11 21:38:21,479][71635] Updated weights for policy 1, policy_version 64782 (0.0007) [2023-10-11 21:38:21,851][71635] Updated weights for policy 1, policy_version 64792 (0.0009) [2023-10-11 21:38:22,565][71601] Updated weights for policy 0, policy_version 64840 (0.0009) [2023-10-11 21:38:22,939][71601] Updated weights for policy 0, policy_version 64850 (0.0007) [2023-10-11 21:38:23,319][71601] Updated weights for policy 0, policy_version 64860 (0.0007) [2023-10-11 21:38:25,644][71635] Updated weights for policy 1, policy_version 64802 (0.0009) [2023-10-11 21:38:26,012][71635] Updated weights for policy 1, policy_version 64812 (0.0008) [2023-10-11 21:38:26,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 132775936. Throughput: 0: 1815.6, 1: 1809.7. Samples: 33209376. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 21:38:26,034][70582] Avg episode reward: [(0, '56.970'), (1, '82.240')] [2023-10-11 21:38:26,369][71635] Updated weights for policy 1, policy_version 64822 (0.0008) [2023-10-11 21:38:26,741][71635] Updated weights for policy 1, policy_version 64832 (0.0009) [2023-10-11 21:38:26,918][71601] Updated weights for policy 0, policy_version 64870 (0.0007) [2023-10-11 21:38:27,287][71601] Updated weights for policy 0, policy_version 64880 (0.0007) [2023-10-11 21:38:27,655][71601] Updated weights for policy 0, policy_version 64890 (0.0007) [2023-10-11 21:38:30,407][71635] Updated weights for policy 1, policy_version 64842 (0.0011) [2023-10-11 21:38:30,774][71635] Updated weights for policy 1, policy_version 64852 (0.0010) [2023-10-11 21:38:31,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 132841472. Throughput: 0: 1816.5, 1: 1806.7. Samples: 33219364. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 21:38:31,034][70582] Avg episode reward: [(0, '59.090'), (1, '86.930')] [2023-10-11 21:38:31,140][71635] Updated weights for policy 1, policy_version 64862 (0.0007) [2023-10-11 21:38:31,328][71601] Updated weights for policy 0, policy_version 64900 (0.0008) [2023-10-11 21:38:31,696][71601] Updated weights for policy 0, policy_version 64910 (0.0007) [2023-10-11 21:38:32,075][71601] Updated weights for policy 0, policy_version 64920 (0.0007) [2023-10-11 21:38:34,837][71635] Updated weights for policy 1, policy_version 64872 (0.0009) [2023-10-11 21:38:35,194][71635] Updated weights for policy 1, policy_version 64882 (0.0008) [2023-10-11 21:38:35,565][71635] Updated weights for policy 1, policy_version 64892 (0.0009) [2023-10-11 21:38:35,845][71601] Updated weights for policy 0, policy_version 64930 (0.0008) [2023-10-11 21:38:36,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 132939776. Throughput: 0: 1817.0, 1: 1811.9. Samples: 33242032. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 21:38:36,034][70582] Avg episode reward: [(0, '54.500'), (1, '86.110')] [2023-10-11 21:38:36,210][71601] Updated weights for policy 0, policy_version 64940 (0.0009) [2023-10-11 21:38:36,578][71601] Updated weights for policy 0, policy_version 64950 (0.0011) [2023-10-11 21:38:36,947][71601] Updated weights for policy 0, policy_version 64960 (0.0009) [2023-10-11 21:38:39,291][71635] Updated weights for policy 1, policy_version 64902 (0.0008) [2023-10-11 21:38:39,664][71635] Updated weights for policy 1, policy_version 64912 (0.0009) [2023-10-11 21:38:40,027][71635] Updated weights for policy 1, policy_version 64922 (0.0007) [2023-10-11 21:38:40,669][71601] Updated weights for policy 0, policy_version 64970 (0.0007) [2023-10-11 21:38:41,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 133005312. Throughput: 0: 1822.3, 1: 1810.3. Samples: 33263088. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 21:38:41,034][70582] Avg episode reward: [(0, '53.980'), (1, '85.840')] [2023-10-11 21:38:41,045][71601] Updated weights for policy 0, policy_version 64980 (0.0007) [2023-10-11 21:38:41,415][71601] Updated weights for policy 0, policy_version 64990 (0.0007) [2023-10-11 21:38:43,679][71635] Updated weights for policy 1, policy_version 64932 (0.0007) [2023-10-11 21:38:44,045][71635] Updated weights for policy 1, policy_version 64942 (0.0008) [2023-10-11 21:38:44,403][71635] Updated weights for policy 1, policy_version 64952 (0.0009) [2023-10-11 21:38:45,086][71601] Updated weights for policy 0, policy_version 65000 (0.0008) [2023-10-11 21:38:45,472][71601] Updated weights for policy 0, policy_version 65010 (0.0009) [2023-10-11 21:38:45,839][71601] Updated weights for policy 0, policy_version 65020 (0.0008) [2023-10-11 21:38:46,034][70582] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 133103616. Throughput: 0: 1825.1, 1: 1811.2. Samples: 33274912. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 21:38:46,035][70582] Avg episode reward: [(0, '53.370'), (1, '89.600')] [2023-10-11 21:38:48,182][71635] Updated weights for policy 1, policy_version 64962 (0.0008) [2023-10-11 21:38:48,553][71635] Updated weights for policy 1, policy_version 64972 (0.0009) [2023-10-11 21:38:48,917][71635] Updated weights for policy 1, policy_version 64982 (0.0010) [2023-10-11 21:38:49,276][71635] Updated weights for policy 1, policy_version 64992 (0.0009) [2023-10-11 21:38:49,520][71601] Updated weights for policy 0, policy_version 65030 (0.0009) [2023-10-11 21:38:49,882][71601] Updated weights for policy 0, policy_version 65040 (0.0008) [2023-10-11 21:38:50,249][71601] Updated weights for policy 0, policy_version 65050 (0.0008) [2023-10-11 21:38:51,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 133169152. Throughput: 0: 1824.5, 1: 1805.9. Samples: 33295742. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-11 21:38:51,035][70582] Avg episode reward: [(0, '55.040'), (1, '90.780')] [2023-10-11 21:38:52,984][71635] Updated weights for policy 1, policy_version 65002 (0.0008) [2023-10-11 21:38:53,345][71635] Updated weights for policy 1, policy_version 65012 (0.0007) [2023-10-11 21:38:53,712][71635] Updated weights for policy 1, policy_version 65022 (0.0008) [2023-10-11 21:38:53,928][71601] Updated weights for policy 0, policy_version 65060 (0.0010) [2023-10-11 21:38:54,301][71601] Updated weights for policy 0, policy_version 65070 (0.0010) [2023-10-11 21:38:54,681][71601] Updated weights for policy 0, policy_version 65080 (0.0008) [2023-10-11 21:38:56,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 133234688. Throughput: 0: 1827.2, 1: 1799.0. Samples: 33317146. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-11 21:38:56,035][70582] Avg episode reward: [(0, '54.090'), (1, '93.110')] [2023-10-11 21:38:57,575][71635] Updated weights for policy 1, policy_version 65032 (0.0009) [2023-10-11 21:38:57,941][71635] Updated weights for policy 1, policy_version 65042 (0.0008) [2023-10-11 21:38:58,305][71635] Updated weights for policy 1, policy_version 65052 (0.0009) [2023-10-11 21:38:58,377][71601] Updated weights for policy 0, policy_version 65090 (0.0008) [2023-10-11 21:38:58,761][71601] Updated weights for policy 0, policy_version 65100 (0.0010) [2023-10-11 21:38:59,133][71601] Updated weights for policy 0, policy_version 65110 (0.0008) [2023-10-11 21:38:59,515][71601] Updated weights for policy 0, policy_version 65120 (0.0007) [2023-10-11 21:39:01,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 133300224. Throughput: 0: 1826.9, 1: 1800.7. Samples: 33328256. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-11 21:39:01,034][70582] Avg episode reward: [(0, '55.680'), (1, '95.530')] [2023-10-11 21:39:02,179][71635] Updated weights for policy 1, policy_version 65062 (0.0010) [2023-10-11 21:39:02,536][71635] Updated weights for policy 1, policy_version 65072 (0.0009) [2023-10-11 21:39:02,905][71635] Updated weights for policy 1, policy_version 65082 (0.0008) [2023-10-11 21:39:03,257][71601] Updated weights for policy 0, policy_version 65130 (0.0008) [2023-10-11 21:39:03,631][71601] Updated weights for policy 0, policy_version 65140 (0.0008) [2023-10-11 21:39:04,005][71601] Updated weights for policy 0, policy_version 65150 (0.0007) [2023-10-11 21:39:06,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 133365760. Throughput: 0: 1819.4, 1: 1790.6. Samples: 33349092. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-11 21:39:06,034][70582] Avg episode reward: [(0, '54.120'), (1, '94.250')] [2023-10-11 21:39:06,682][71635] Updated weights for policy 1, policy_version 65092 (0.0008) [2023-10-11 21:39:07,051][71635] Updated weights for policy 1, policy_version 65102 (0.0007) [2023-10-11 21:39:07,411][71635] Updated weights for policy 1, policy_version 65112 (0.0008) [2023-10-11 21:39:07,585][71601] Updated weights for policy 0, policy_version 65160 (0.0008) [2023-10-11 21:39:07,958][71601] Updated weights for policy 0, policy_version 65170 (0.0008) [2023-10-11 21:39:08,339][71601] Updated weights for policy 0, policy_version 65180 (0.0009) [2023-10-11 21:39:11,021][71635] Updated weights for policy 1, policy_version 65122 (0.0009) [2023-10-11 21:39:11,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 133431296. Throughput: 0: 1823.3, 1: 1796.6. Samples: 33372270. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-11 21:39:11,034][70582] Avg episode reward: [(0, '58.610'), (1, '86.630')] [2023-10-11 21:39:11,381][71635] Updated weights for policy 1, policy_version 65132 (0.0007) [2023-10-11 21:39:11,743][71635] Updated weights for policy 1, policy_version 65142 (0.0008) [2023-10-11 21:39:11,976][71601] Updated weights for policy 0, policy_version 65190 (0.0008) [2023-10-11 21:39:12,109][71635] Updated weights for policy 1, policy_version 65152 (0.0009) [2023-10-11 21:39:12,349][71601] Updated weights for policy 0, policy_version 65200 (0.0008) [2023-10-11 21:39:12,739][71601] Updated weights for policy 0, policy_version 65210 (0.0009) [2023-10-11 21:39:15,910][71635] Updated weights for policy 1, policy_version 65162 (0.0009) [2023-10-11 21:39:16,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 133496832. Throughput: 0: 1823.3, 1: 1796.2. Samples: 33382242. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-11 21:39:16,034][70582] Avg episode reward: [(0, '60.040'), (1, '84.340')] [2023-10-11 21:39:16,290][71635] Updated weights for policy 1, policy_version 65172 (0.0007) [2023-10-11 21:39:16,310][71601] Updated weights for policy 0, policy_version 65220 (0.0010) [2023-10-11 21:39:16,648][71635] Updated weights for policy 1, policy_version 65182 (0.0009) [2023-10-11 21:39:16,677][71601] Updated weights for policy 0, policy_version 65230 (0.0008) [2023-10-11 21:39:17,056][71601] Updated weights for policy 0, policy_version 65240 (0.0010) [2023-10-11 21:39:20,392][71635] Updated weights for policy 1, policy_version 65192 (0.0008) [2023-10-11 21:39:20,707][71601] Updated weights for policy 0, policy_version 65250 (0.0010) [2023-10-11 21:39:20,750][71635] Updated weights for policy 1, policy_version 65202 (0.0008) [2023-10-11 21:39:21,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 133562368. Throughput: 0: 1828.2, 1: 1790.0. Samples: 33404854. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-11 21:39:21,035][70582] Avg episode reward: [(0, '63.370'), (1, '84.020')] [2023-10-11 21:39:21,075][71601] Updated weights for policy 0, policy_version 65260 (0.0007) [2023-10-11 21:39:21,116][71635] Updated weights for policy 1, policy_version 65212 (0.0008) [2023-10-11 21:39:21,440][71601] Updated weights for policy 0, policy_version 65270 (0.0007) [2023-10-11 21:39:21,817][71601] Updated weights for policy 0, policy_version 65280 (0.0007) [2023-10-11 21:39:24,920][71635] Updated weights for policy 1, policy_version 65222 (0.0010) [2023-10-11 21:39:25,289][71635] Updated weights for policy 1, policy_version 65232 (0.0008) [2023-10-11 21:39:25,480][71601] Updated weights for policy 0, policy_version 65290 (0.0007) [2023-10-11 21:39:25,660][71635] Updated weights for policy 1, policy_version 65242 (0.0007) [2023-10-11 21:39:25,848][71601] Updated weights for policy 0, policy_version 65300 (0.0007) [2023-10-11 21:39:26,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 133660672. Throughput: 0: 1822.7, 1: 1803.9. Samples: 33426284. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-11 21:39:26,034][70582] Avg episode reward: [(0, '64.850'), (1, '91.370')] [2023-10-11 21:39:26,224][71601] Updated weights for policy 0, policy_version 65310 (0.0007) [2023-10-11 21:39:29,380][71635] Updated weights for policy 1, policy_version 65252 (0.0007) [2023-10-11 21:39:29,753][71635] Updated weights for policy 1, policy_version 65262 (0.0008) [2023-10-11 21:39:29,929][71601] Updated weights for policy 0, policy_version 65320 (0.0007) [2023-10-11 21:39:30,127][71635] Updated weights for policy 1, policy_version 65272 (0.0009) [2023-10-11 21:39:30,307][71601] Updated weights for policy 0, policy_version 65330 (0.0009) [2023-10-11 21:39:30,666][71601] Updated weights for policy 0, policy_version 65340 (0.0009) [2023-10-11 21:39:31,034][70582] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 133758976. Throughput: 0: 1827.3, 1: 1786.4. Samples: 33437526. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-11 21:39:31,035][70582] Avg episode reward: [(0, '63.870'), (1, '85.570')] [2023-10-11 21:39:33,789][71635] Updated weights for policy 1, policy_version 65282 (0.0008) [2023-10-11 21:39:34,159][71635] Updated weights for policy 1, policy_version 65292 (0.0008) [2023-10-11 21:39:34,397][71601] Updated weights for policy 0, policy_version 65350 (0.0008) [2023-10-11 21:39:34,511][71635] Updated weights for policy 1, policy_version 65302 (0.0007) [2023-10-11 21:39:34,771][71601] Updated weights for policy 0, policy_version 65360 (0.0007) [2023-10-11 21:39:34,877][71635] Updated weights for policy 1, policy_version 65312 (0.0009) [2023-10-11 21:39:35,150][71601] Updated weights for policy 0, policy_version 65370 (0.0010) [2023-10-11 21:39:36,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 133824512. Throughput: 0: 1825.4, 1: 1810.6. Samples: 33459364. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:39:36,034][70582] Avg episode reward: [(0, '65.200'), (1, '87.530')] [2023-10-11 21:39:38,537][71635] Updated weights for policy 1, policy_version 65322 (0.0007) [2023-10-11 21:39:38,674][71601] Updated weights for policy 0, policy_version 65380 (0.0009) [2023-10-11 21:39:38,907][71635] Updated weights for policy 1, policy_version 65332 (0.0009) [2023-10-11 21:39:39,042][71601] Updated weights for policy 0, policy_version 65390 (0.0009) [2023-10-11 21:39:39,272][71635] Updated weights for policy 1, policy_version 65342 (0.0009) [2023-10-11 21:39:39,412][71601] Updated weights for policy 0, policy_version 65400 (0.0009) [2023-10-11 21:39:41,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 133890048. Throughput: 0: 1830.4, 1: 1798.7. Samples: 33480456. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:39:41,035][70582] Avg episode reward: [(0, '64.560'), (1, '82.540')] [2023-10-11 21:39:43,038][71635] Updated weights for policy 1, policy_version 65352 (0.0008) [2023-10-11 21:39:43,072][71601] Updated weights for policy 0, policy_version 65410 (0.0007) [2023-10-11 21:39:43,420][71635] Updated weights for policy 1, policy_version 65362 (0.0007) [2023-10-11 21:39:43,452][71601] Updated weights for policy 0, policy_version 65420 (0.0007) [2023-10-11 21:39:43,782][71635] Updated weights for policy 1, policy_version 65372 (0.0008) [2023-10-11 21:39:43,820][71601] Updated weights for policy 0, policy_version 65430 (0.0008) [2023-10-11 21:39:44,181][71601] Updated weights for policy 0, policy_version 65440 (0.0009) [2023-10-11 21:39:46,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 133955584. Throughput: 0: 1824.5, 1: 1814.5. Samples: 33492012. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:39:46,035][70582] Avg episode reward: [(0, '67.690'), (1, '80.390')] [2023-10-11 21:39:47,249][71635] Updated weights for policy 1, policy_version 65382 (0.0009) [2023-10-11 21:39:47,622][71635] Updated weights for policy 1, policy_version 65392 (0.0009) [2023-10-11 21:39:47,987][71635] Updated weights for policy 1, policy_version 65402 (0.0007) [2023-10-11 21:39:48,011][71601] Updated weights for policy 0, policy_version 65450 (0.0008) [2023-10-11 21:39:48,381][71601] Updated weights for policy 0, policy_version 65460 (0.0007) [2023-10-11 21:39:48,754][71601] Updated weights for policy 0, policy_version 65470 (0.0007) [2023-10-11 21:39:51,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 134021120. Throughput: 0: 1830.5, 1: 1813.8. Samples: 33513084. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:39:51,034][70582] Avg episode reward: [(0, '64.030'), (1, '77.050')] [2023-10-11 21:39:51,831][71635] Updated weights for policy 1, policy_version 65412 (0.0008) [2023-10-11 21:39:52,205][71635] Updated weights for policy 1, policy_version 65422 (0.0009) [2023-10-11 21:39:52,417][71601] Updated weights for policy 0, policy_version 65480 (0.0007) [2023-10-11 21:39:52,568][71635] Updated weights for policy 1, policy_version 65432 (0.0009) [2023-10-11 21:39:52,792][71601] Updated weights for policy 0, policy_version 65490 (0.0008) [2023-10-11 21:39:53,158][71601] Updated weights for policy 0, policy_version 65500 (0.0010) [2023-10-11 21:39:56,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 134086656. Throughput: 0: 1821.6, 1: 1809.7. Samples: 33535682. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:39:56,034][70582] Avg episode reward: [(0, '61.850'), (1, '76.540')] [2023-10-11 21:39:56,044][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000065504_67076096.pth... [2023-10-11 21:39:56,045][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000065440_67010560.pth... [2023-10-11 21:39:56,074][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000063808_65339392.pth [2023-10-11 21:39:56,082][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000063744_65273856.pth [2023-10-11 21:39:56,305][71635] Updated weights for policy 1, policy_version 65442 (0.0007) [2023-10-11 21:39:56,671][71635] Updated weights for policy 1, policy_version 65452 (0.0007) [2023-10-11 21:39:56,855][71601] Updated weights for policy 0, policy_version 65510 (0.0007) [2023-10-11 21:39:57,038][71635] Updated weights for policy 1, policy_version 65462 (0.0009) [2023-10-11 21:39:57,229][71601] Updated weights for policy 0, policy_version 65520 (0.0009) [2023-10-11 21:39:57,411][71635] Updated weights for policy 1, policy_version 65472 (0.0008) [2023-10-11 21:39:57,604][71601] Updated weights for policy 0, policy_version 65530 (0.0007) [2023-10-11 21:40:01,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 134152192. Throughput: 0: 1819.5, 1: 1814.5. Samples: 33545772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:40:01,034][70582] Avg episode reward: [(0, '58.190'), (1, '76.610')] [2023-10-11 21:40:01,040][71635] Updated weights for policy 1, policy_version 65482 (0.0008) [2023-10-11 21:40:01,297][71601] Updated weights for policy 0, policy_version 65540 (0.0007) [2023-10-11 21:40:01,396][71635] Updated weights for policy 1, policy_version 65492 (0.0009) [2023-10-11 21:40:01,677][71601] Updated weights for policy 0, policy_version 65550 (0.0007) [2023-10-11 21:40:01,765][71635] Updated weights for policy 1, policy_version 65502 (0.0009) [2023-10-11 21:40:02,046][71601] Updated weights for policy 0, policy_version 65560 (0.0010) [2023-10-11 21:40:05,560][71635] Updated weights for policy 1, policy_version 65512 (0.0009) [2023-10-11 21:40:05,766][71601] Updated weights for policy 0, policy_version 65570 (0.0009) [2023-10-11 21:40:05,934][71635] Updated weights for policy 1, policy_version 65522 (0.0008) [2023-10-11 21:40:06,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 134217728. Throughput: 0: 1815.4, 1: 1817.6. Samples: 33568340. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:40:06,034][70582] Avg episode reward: [(0, '57.670'), (1, '75.140')] [2023-10-11 21:40:06,133][71601] Updated weights for policy 0, policy_version 65580 (0.0008) [2023-10-11 21:40:06,301][71635] Updated weights for policy 1, policy_version 65532 (0.0007) [2023-10-11 21:40:06,505][71601] Updated weights for policy 0, policy_version 65590 (0.0007) [2023-10-11 21:40:06,881][71601] Updated weights for policy 0, policy_version 65600 (0.0008) [2023-10-11 21:40:10,065][71635] Updated weights for policy 1, policy_version 65542 (0.0008) [2023-10-11 21:40:10,434][71635] Updated weights for policy 1, policy_version 65552 (0.0007) [2023-10-11 21:40:10,611][71601] Updated weights for policy 0, policy_version 65610 (0.0007) [2023-10-11 21:40:10,790][71635] Updated weights for policy 1, policy_version 65562 (0.0009) [2023-10-11 21:40:10,976][71601] Updated weights for policy 0, policy_version 65620 (0.0007) [2023-10-11 21:40:11,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 134316032. Throughput: 0: 1814.4, 1: 1823.9. Samples: 33590006. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:40:11,034][70582] Avg episode reward: [(0, '58.680'), (1, '76.860')] [2023-10-11 21:40:11,348][71601] Updated weights for policy 0, policy_version 65630 (0.0007) [2023-10-11 21:40:14,542][71635] Updated weights for policy 1, policy_version 65572 (0.0009) [2023-10-11 21:40:14,908][71635] Updated weights for policy 1, policy_version 65582 (0.0009) [2023-10-11 21:40:15,183][71601] Updated weights for policy 0, policy_version 65640 (0.0009) [2023-10-11 21:40:15,269][71635] Updated weights for policy 1, policy_version 65592 (0.0008) [2023-10-11 21:40:15,561][71601] Updated weights for policy 0, policy_version 65650 (0.0010) [2023-10-11 21:40:15,936][71601] Updated weights for policy 0, policy_version 65660 (0.0007) [2023-10-11 21:40:16,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 134381568. Throughput: 0: 1808.4, 1: 1816.9. Samples: 33600664. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:40:16,035][70582] Avg episode reward: [(0, '57.070'), (1, '74.700')] [2023-10-11 21:40:19,009][71635] Updated weights for policy 1, policy_version 65602 (0.0009) [2023-10-11 21:40:19,374][71635] Updated weights for policy 1, policy_version 65612 (0.0008) [2023-10-11 21:40:19,622][71601] Updated weights for policy 0, policy_version 65670 (0.0007) [2023-10-11 21:40:19,733][71635] Updated weights for policy 1, policy_version 65622 (0.0007) [2023-10-11 21:40:19,991][71601] Updated weights for policy 0, policy_version 65680 (0.0008) [2023-10-11 21:40:20,089][71635] Updated weights for policy 1, policy_version 65632 (0.0008) [2023-10-11 21:40:20,366][71601] Updated weights for policy 0, policy_version 65690 (0.0009) [2023-10-11 21:40:21,034][70582] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 134479872. Throughput: 0: 1808.4, 1: 1818.4. Samples: 33622568. Policy #0 lag: (min: 18.0, avg: 23.9, max: 50.0) [2023-10-11 21:40:21,035][70582] Avg episode reward: [(0, '61.340'), (1, '77.970')] [2023-10-11 21:40:23,758][71635] Updated weights for policy 1, policy_version 65642 (0.0007) [2023-10-11 21:40:23,960][71601] Updated weights for policy 0, policy_version 65700 (0.0007) [2023-10-11 21:40:24,123][71635] Updated weights for policy 1, policy_version 65652 (0.0009) [2023-10-11 21:40:24,329][71601] Updated weights for policy 0, policy_version 65710 (0.0007) [2023-10-11 21:40:24,497][71635] Updated weights for policy 1, policy_version 65662 (0.0009) [2023-10-11 21:40:24,693][71601] Updated weights for policy 0, policy_version 65720 (0.0008) [2023-10-11 21:40:26,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 134545408. Throughput: 0: 1800.0, 1: 1811.9. Samples: 33642988. Policy #0 lag: (min: 18.0, avg: 23.9, max: 50.0) [2023-10-11 21:40:26,035][70582] Avg episode reward: [(0, '58.660'), (1, '76.700')] [2023-10-11 21:40:28,256][71635] Updated weights for policy 1, policy_version 65672 (0.0007) [2023-10-11 21:40:28,475][71601] Updated weights for policy 0, policy_version 65730 (0.0009) [2023-10-11 21:40:28,633][71635] Updated weights for policy 1, policy_version 65682 (0.0009) [2023-10-11 21:40:28,842][71601] Updated weights for policy 0, policy_version 65740 (0.0010) [2023-10-11 21:40:28,996][71635] Updated weights for policy 1, policy_version 65692 (0.0011) [2023-10-11 21:40:29,209][71601] Updated weights for policy 0, policy_version 65750 (0.0008) [2023-10-11 21:40:29,578][71601] Updated weights for policy 0, policy_version 65760 (0.0008) [2023-10-11 21:40:31,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 134610944. Throughput: 0: 1811.5, 1: 1817.8. Samples: 33655330. Policy #0 lag: (min: 18.0, avg: 23.9, max: 50.0) [2023-10-11 21:40:31,035][70582] Avg episode reward: [(0, '59.400'), (1, '82.710')] [2023-10-11 21:40:32,581][71635] Updated weights for policy 1, policy_version 65702 (0.0007) [2023-10-11 21:40:32,943][71635] Updated weights for policy 1, policy_version 65712 (0.0008) [2023-10-11 21:40:33,266][71601] Updated weights for policy 0, policy_version 65770 (0.0008) [2023-10-11 21:40:33,315][71635] Updated weights for policy 1, policy_version 65722 (0.0008) [2023-10-11 21:40:33,644][71601] Updated weights for policy 0, policy_version 65780 (0.0007) [2023-10-11 21:40:34,008][71601] Updated weights for policy 0, policy_version 65790 (0.0008) [2023-10-11 21:40:36,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 134676480. Throughput: 0: 1807.3, 1: 1804.2. Samples: 33675604. Policy #0 lag: (min: 18.0, avg: 23.9, max: 50.0) [2023-10-11 21:40:36,034][70582] Avg episode reward: [(0, '58.500'), (1, '86.860')] [2023-10-11 21:40:37,110][71635] Updated weights for policy 1, policy_version 65732 (0.0007) [2023-10-11 21:40:37,473][71635] Updated weights for policy 1, policy_version 65742 (0.0008) [2023-10-11 21:40:37,566][71601] Updated weights for policy 0, policy_version 65800 (0.0008) [2023-10-11 21:40:37,830][71635] Updated weights for policy 1, policy_version 65752 (0.0008) [2023-10-11 21:40:37,945][71601] Updated weights for policy 0, policy_version 65810 (0.0009) [2023-10-11 21:40:38,320][71601] Updated weights for policy 0, policy_version 65820 (0.0008) [2023-10-11 21:40:41,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 134742016. Throughput: 0: 1807.4, 1: 1807.3. Samples: 33698346. Policy #0 lag: (min: 18.0, avg: 23.9, max: 50.0) [2023-10-11 21:40:41,035][70582] Avg episode reward: [(0, '54.590'), (1, '89.770')] [2023-10-11 21:40:41,488][71635] Updated weights for policy 1, policy_version 65762 (0.0009) [2023-10-11 21:40:41,844][71635] Updated weights for policy 1, policy_version 65772 (0.0008) [2023-10-11 21:40:42,026][71601] Updated weights for policy 0, policy_version 65830 (0.0009) [2023-10-11 21:40:42,218][71635] Updated weights for policy 1, policy_version 65782 (0.0008) [2023-10-11 21:40:42,404][71601] Updated weights for policy 0, policy_version 65840 (0.0007) [2023-10-11 21:40:42,577][71635] Updated weights for policy 1, policy_version 65792 (0.0009) [2023-10-11 21:40:42,774][71601] Updated weights for policy 0, policy_version 65850 (0.0009) [2023-10-11 21:40:46,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 134807552. Throughput: 0: 1805.5, 1: 1807.0. Samples: 33708334. Policy #0 lag: (min: 18.0, avg: 23.9, max: 50.0) [2023-10-11 21:40:46,035][70582] Avg episode reward: [(0, '58.320'), (1, '88.350')] [2023-10-11 21:40:46,325][71635] Updated weights for policy 1, policy_version 65802 (0.0008) [2023-10-11 21:40:46,571][71601] Updated weights for policy 0, policy_version 65860 (0.0009) [2023-10-11 21:40:46,692][71635] Updated weights for policy 1, policy_version 65812 (0.0008) [2023-10-11 21:40:46,941][71601] Updated weights for policy 0, policy_version 65870 (0.0008) [2023-10-11 21:40:47,056][71635] Updated weights for policy 1, policy_version 65822 (0.0009) [2023-10-11 21:40:47,307][71601] Updated weights for policy 0, policy_version 65880 (0.0009) [2023-10-11 21:40:50,823][71635] Updated weights for policy 1, policy_version 65832 (0.0007) [2023-10-11 21:40:51,008][71601] Updated weights for policy 0, policy_version 65890 (0.0007) [2023-10-11 21:40:51,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 134873088. Throughput: 0: 1810.6, 1: 1807.2. Samples: 33731140. Policy #0 lag: (min: 18.0, avg: 23.9, max: 50.0) [2023-10-11 21:40:51,035][70582] Avg episode reward: [(0, '58.590'), (1, '86.750')] [2023-10-11 21:40:51,197][71635] Updated weights for policy 1, policy_version 65842 (0.0009) [2023-10-11 21:40:51,369][71601] Updated weights for policy 0, policy_version 65900 (0.0008) [2023-10-11 21:40:51,552][71635] Updated weights for policy 1, policy_version 65852 (0.0008) [2023-10-11 21:40:51,745][71601] Updated weights for policy 0, policy_version 65910 (0.0008) [2023-10-11 21:40:52,114][71601] Updated weights for policy 0, policy_version 65920 (0.0007) [2023-10-11 21:40:55,344][71635] Updated weights for policy 1, policy_version 65862 (0.0009) [2023-10-11 21:40:55,716][71635] Updated weights for policy 1, policy_version 65872 (0.0008) [2023-10-11 21:40:56,003][71601] Updated weights for policy 0, policy_version 65930 (0.0008) [2023-10-11 21:40:56,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 134938624. Throughput: 0: 1813.9, 1: 1813.0. Samples: 33753218. Policy #0 lag: (min: 18.0, avg: 23.9, max: 50.0) [2023-10-11 21:40:56,035][70582] Avg episode reward: [(0, '56.400'), (1, '84.460')] [2023-10-11 21:40:56,087][71635] Updated weights for policy 1, policy_version 65882 (0.0007) [2023-10-11 21:40:56,379][71601] Updated weights for policy 0, policy_version 65940 (0.0008) [2023-10-11 21:40:56,756][71601] Updated weights for policy 0, policy_version 65950 (0.0010) [2023-10-11 21:40:59,818][71635] Updated weights for policy 1, policy_version 65892 (0.0008) [2023-10-11 21:41:00,187][71635] Updated weights for policy 1, policy_version 65902 (0.0007) [2023-10-11 21:41:00,209][71601] Updated weights for policy 0, policy_version 65960 (0.0008) [2023-10-11 21:41:00,541][71635] Updated weights for policy 1, policy_version 65912 (0.0007) [2023-10-11 21:41:00,579][71601] Updated weights for policy 0, policy_version 65970 (0.0008) [2023-10-11 21:41:00,947][71601] Updated weights for policy 0, policy_version 65980 (0.0008) [2023-10-11 21:41:01,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 135036928. Throughput: 0: 1807.6, 1: 1803.3. Samples: 33763152. Policy #0 lag: (min: 18.0, avg: 23.9, max: 50.0) [2023-10-11 21:41:01,034][70582] Avg episode reward: [(0, '56.500'), (1, '85.660')] [2023-10-11 21:41:04,278][71635] Updated weights for policy 1, policy_version 65922 (0.0007) [2023-10-11 21:41:04,636][71635] Updated weights for policy 1, policy_version 65932 (0.0009) [2023-10-11 21:41:04,662][71601] Updated weights for policy 0, policy_version 65990 (0.0008) [2023-10-11 21:41:05,001][71635] Updated weights for policy 1, policy_version 65942 (0.0008) [2023-10-11 21:41:05,041][71601] Updated weights for policy 0, policy_version 66000 (0.0008) [2023-10-11 21:41:05,370][71635] Updated weights for policy 1, policy_version 65952 (0.0009) [2023-10-11 21:41:05,402][71601] Updated weights for policy 0, policy_version 66010 (0.0009) [2023-10-11 21:41:06,034][70582] Fps is (10 sec: 19661.5, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 135135232. Throughput: 0: 1814.1, 1: 1812.4. Samples: 33785760. Policy #0 lag: (min: 0.0, avg: 23.9, max: 32.0) [2023-10-11 21:41:06,034][70582] Avg episode reward: [(0, '57.200'), (1, '80.180')] [2023-10-11 21:41:09,120][71635] Updated weights for policy 1, policy_version 65962 (0.0008) [2023-10-11 21:41:09,204][71601] Updated weights for policy 0, policy_version 66020 (0.0009) [2023-10-11 21:41:09,483][71635] Updated weights for policy 1, policy_version 65972 (0.0009) [2023-10-11 21:41:09,567][71601] Updated weights for policy 0, policy_version 66030 (0.0007) [2023-10-11 21:41:09,855][71635] Updated weights for policy 1, policy_version 65982 (0.0008) [2023-10-11 21:41:09,939][71601] Updated weights for policy 0, policy_version 66040 (0.0008) [2023-10-11 21:41:11,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 135200768. Throughput: 0: 1815.3, 1: 1799.9. Samples: 33805672. Policy #0 lag: (min: 0.0, avg: 23.9, max: 32.0) [2023-10-11 21:41:11,035][70582] Avg episode reward: [(0, '56.710'), (1, '84.270')] [2023-10-11 21:41:13,733][71635] Updated weights for policy 1, policy_version 65992 (0.0008) [2023-10-11 21:41:13,757][71601] Updated weights for policy 0, policy_version 66050 (0.0010) [2023-10-11 21:41:14,097][71635] Updated weights for policy 1, policy_version 66002 (0.0007) [2023-10-11 21:41:14,137][71601] Updated weights for policy 0, policy_version 66060 (0.0009) [2023-10-11 21:41:14,466][71635] Updated weights for policy 1, policy_version 66012 (0.0007) [2023-10-11 21:41:14,497][71601] Updated weights for policy 0, policy_version 66070 (0.0009) [2023-10-11 21:41:14,865][71601] Updated weights for policy 0, policy_version 66080 (0.0009) [2023-10-11 21:41:16,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 135266304. Throughput: 0: 1816.8, 1: 1808.9. Samples: 33818486. Policy #0 lag: (min: 0.0, avg: 23.9, max: 32.0) [2023-10-11 21:41:16,035][70582] Avg episode reward: [(0, '60.630'), (1, '84.770')] [2023-10-11 21:41:17,965][71635] Updated weights for policy 1, policy_version 66022 (0.0008) [2023-10-11 21:41:18,327][71635] Updated weights for policy 1, policy_version 66032 (0.0008) [2023-10-11 21:41:18,537][71601] Updated weights for policy 0, policy_version 66090 (0.0007) [2023-10-11 21:41:18,689][71635] Updated weights for policy 1, policy_version 66042 (0.0009) [2023-10-11 21:41:18,912][71601] Updated weights for policy 0, policy_version 66100 (0.0008) [2023-10-11 21:41:19,279][71601] Updated weights for policy 0, policy_version 66110 (0.0010) [2023-10-11 21:41:21,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 135331840. Throughput: 0: 1808.6, 1: 1802.6. Samples: 33838108. Policy #0 lag: (min: 0.0, avg: 23.9, max: 32.0) [2023-10-11 21:41:21,034][70582] Avg episode reward: [(0, '61.180'), (1, '83.970')] [2023-10-11 21:41:22,469][71635] Updated weights for policy 1, policy_version 66052 (0.0009) [2023-10-11 21:41:22,832][71635] Updated weights for policy 1, policy_version 66062 (0.0009) [2023-10-11 21:41:22,994][71601] Updated weights for policy 0, policy_version 66120 (0.0009) [2023-10-11 21:41:23,206][71635] Updated weights for policy 1, policy_version 66072 (0.0009) [2023-10-11 21:41:23,368][71601] Updated weights for policy 0, policy_version 66130 (0.0010) [2023-10-11 21:41:23,745][71601] Updated weights for policy 0, policy_version 66140 (0.0008) [2023-10-11 21:41:26,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 135397376. Throughput: 0: 1807.4, 1: 1797.1. Samples: 33860548. Policy #0 lag: (min: 0.0, avg: 23.9, max: 32.0) [2023-10-11 21:41:26,034][70582] Avg episode reward: [(0, '61.610'), (1, '83.990')] [2023-10-11 21:41:27,055][71635] Updated weights for policy 1, policy_version 66082 (0.0010) [2023-10-11 21:41:27,426][71635] Updated weights for policy 1, policy_version 66092 (0.0008) [2023-10-11 21:41:27,510][71601] Updated weights for policy 0, policy_version 66150 (0.0009) [2023-10-11 21:41:27,795][71635] Updated weights for policy 1, policy_version 66102 (0.0008) [2023-10-11 21:41:27,877][71601] Updated weights for policy 0, policy_version 66160 (0.0008) [2023-10-11 21:41:28,154][71635] Updated weights for policy 1, policy_version 66112 (0.0007) [2023-10-11 21:41:28,248][71601] Updated weights for policy 0, policy_version 66170 (0.0009) [2023-10-11 21:41:31,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 135462912. Throughput: 0: 1810.4, 1: 1789.0. Samples: 33870310. Policy #0 lag: (min: 0.0, avg: 23.9, max: 32.0) [2023-10-11 21:41:31,035][70582] Avg episode reward: [(0, '61.380'), (1, '84.260')] [2023-10-11 21:41:31,942][71635] Updated weights for policy 1, policy_version 66122 (0.0010) [2023-10-11 21:41:32,171][71601] Updated weights for policy 0, policy_version 66180 (0.0007) [2023-10-11 21:41:32,317][71635] Updated weights for policy 1, policy_version 66132 (0.0008) [2023-10-11 21:41:32,546][71601] Updated weights for policy 0, policy_version 66190 (0.0008) [2023-10-11 21:41:32,677][71635] Updated weights for policy 1, policy_version 66142 (0.0007) [2023-10-11 21:41:32,920][71601] Updated weights for policy 0, policy_version 66200 (0.0007) [2023-10-11 21:41:36,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 135528448. Throughput: 0: 1798.3, 1: 1785.9. Samples: 33892430. Policy #0 lag: (min: 0.0, avg: 23.9, max: 32.0) [2023-10-11 21:41:36,035][70582] Avg episode reward: [(0, '65.140'), (1, '82.900')] [2023-10-11 21:41:36,494][71601] Updated weights for policy 0, policy_version 66210 (0.0008) [2023-10-11 21:41:36,560][71635] Updated weights for policy 1, policy_version 66152 (0.0007) [2023-10-11 21:41:36,854][71601] Updated weights for policy 0, policy_version 66220 (0.0008) [2023-10-11 21:41:36,924][71635] Updated weights for policy 1, policy_version 66162 (0.0008) [2023-10-11 21:41:37,221][71601] Updated weights for policy 0, policy_version 66230 (0.0008) [2023-10-11 21:41:37,285][71635] Updated weights for policy 1, policy_version 66172 (0.0008) [2023-10-11 21:41:37,593][71601] Updated weights for policy 0, policy_version 66240 (0.0008) [2023-10-11 21:41:41,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 135593984. Throughput: 0: 1799.2, 1: 1788.0. Samples: 33914640. Policy #0 lag: (min: 0.0, avg: 23.9, max: 32.0) [2023-10-11 21:41:41,034][70582] Avg episode reward: [(0, '66.470'), (1, '85.850')] [2023-10-11 21:41:41,182][71635] Updated weights for policy 1, policy_version 66182 (0.0008) [2023-10-11 21:41:41,530][71601] Updated weights for policy 0, policy_version 66250 (0.0007) [2023-10-11 21:41:41,554][71635] Updated weights for policy 1, policy_version 66192 (0.0008) [2023-10-11 21:41:41,905][71601] Updated weights for policy 0, policy_version 66260 (0.0007) [2023-10-11 21:41:41,929][71635] Updated weights for policy 1, policy_version 66202 (0.0008) [2023-10-11 21:41:42,270][71601] Updated weights for policy 0, policy_version 66270 (0.0007) [2023-10-11 21:41:45,640][71635] Updated weights for policy 1, policy_version 66212 (0.0009) [2023-10-11 21:41:46,007][71635] Updated weights for policy 1, policy_version 66222 (0.0008) [2023-10-11 21:41:46,010][71601] Updated weights for policy 0, policy_version 66280 (0.0009) [2023-10-11 21:41:46,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 135659520. Throughput: 0: 1797.2, 1: 1784.6. Samples: 33924330. Policy #0 lag: (min: 0.0, avg: 23.9, max: 32.0) [2023-10-11 21:41:46,034][70582] Avg episode reward: [(0, '64.970'), (1, '87.810')] [2023-10-11 21:41:46,371][71635] Updated weights for policy 1, policy_version 66232 (0.0008) [2023-10-11 21:41:46,386][71601] Updated weights for policy 0, policy_version 66290 (0.0009) [2023-10-11 21:41:46,754][71601] Updated weights for policy 0, policy_version 66300 (0.0008) [2023-10-11 21:41:50,066][71635] Updated weights for policy 1, policy_version 66242 (0.0008) [2023-10-11 21:41:50,372][71601] Updated weights for policy 0, policy_version 66310 (0.0007) [2023-10-11 21:41:50,431][71635] Updated weights for policy 1, policy_version 66252 (0.0007) [2023-10-11 21:41:50,740][71601] Updated weights for policy 0, policy_version 66320 (0.0008) [2023-10-11 21:41:50,787][71635] Updated weights for policy 1, policy_version 66262 (0.0008) [2023-10-11 21:41:51,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 135725056. Throughput: 0: 1793.8, 1: 1791.8. Samples: 33947112. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-11 21:41:51,034][70582] Avg episode reward: [(0, '65.870'), (1, '89.520')] [2023-10-11 21:41:51,124][71601] Updated weights for policy 0, policy_version 66330 (0.0009) [2023-10-11 21:41:51,155][71635] Updated weights for policy 1, policy_version 66272 (0.0009) [2023-10-11 21:41:54,750][71601] Updated weights for policy 0, policy_version 66340 (0.0008) [2023-10-11 21:41:54,861][71635] Updated weights for policy 1, policy_version 66282 (0.0008) [2023-10-11 21:41:55,114][71601] Updated weights for policy 0, policy_version 66350 (0.0007) [2023-10-11 21:41:55,215][71635] Updated weights for policy 1, policy_version 66292 (0.0007) [2023-10-11 21:41:55,489][71601] Updated weights for policy 0, policy_version 66360 (0.0007) [2023-10-11 21:41:55,577][71635] Updated weights for policy 1, policy_version 66302 (0.0007) [2023-10-11 21:41:56,034][70582] Fps is (10 sec: 19660.5, 60 sec: 15291.8, 300 sec: 14551.2). Total num frames: 135856128. Throughput: 0: 1803.2, 1: 1801.2. Samples: 33967872. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-11 21:41:56,034][70582] Avg episode reward: [(0, '65.130'), (1, '95.510')] [2023-10-11 21:41:56,045][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000066304_67895296.pth... [2023-10-11 21:41:56,045][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000066368_67960832.pth... [2023-10-11 21:41:56,075][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000064608_66158592.pth [2023-10-11 21:41:56,084][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000064672_66224128.pth [2023-10-11 21:41:59,227][71601] Updated weights for policy 0, policy_version 66370 (0.0007) [2023-10-11 21:41:59,420][71635] Updated weights for policy 1, policy_version 66312 (0.0009) [2023-10-11 21:41:59,612][71601] Updated weights for policy 0, policy_version 66380 (0.0007) [2023-10-11 21:41:59,798][71635] Updated weights for policy 1, policy_version 66322 (0.0009) [2023-10-11 21:41:59,979][71601] Updated weights for policy 0, policy_version 66390 (0.0008) [2023-10-11 21:42:00,168][71635] Updated weights for policy 1, policy_version 66332 (0.0009) [2023-10-11 21:42:00,354][71601] Updated weights for policy 0, policy_version 66400 (0.0007) [2023-10-11 21:42:01,034][70582] Fps is (10 sec: 19660.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 135921664. Throughput: 0: 1785.9, 1: 1797.0. Samples: 33979714. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-11 21:42:01,034][70582] Avg episode reward: [(0, '68.480'), (1, '93.580')] [2023-10-11 21:42:03,752][71635] Updated weights for policy 1, policy_version 66342 (0.0008) [2023-10-11 21:42:04,123][71635] Updated weights for policy 1, policy_version 66352 (0.0010) [2023-10-11 21:42:04,198][71601] Updated weights for policy 0, policy_version 66410 (0.0010) [2023-10-11 21:42:04,491][71635] Updated weights for policy 1, policy_version 66362 (0.0008) [2023-10-11 21:42:04,569][71601] Updated weights for policy 0, policy_version 66420 (0.0009) [2023-10-11 21:42:04,951][71601] Updated weights for policy 0, policy_version 66430 (0.0009) [2023-10-11 21:42:06,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 135987200. Throughput: 0: 1804.8, 1: 1803.8. Samples: 34000496. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-11 21:42:06,035][70582] Avg episode reward: [(0, '67.080'), (1, '90.270')] [2023-10-11 21:42:08,309][71635] Updated weights for policy 1, policy_version 66372 (0.0009) [2023-10-11 21:42:08,560][71601] Updated weights for policy 0, policy_version 66440 (0.0009) [2023-10-11 21:42:08,679][71635] Updated weights for policy 1, policy_version 66382 (0.0008) [2023-10-11 21:42:08,932][71601] Updated weights for policy 0, policy_version 66450 (0.0010) [2023-10-11 21:42:09,037][71635] Updated weights for policy 1, policy_version 66392 (0.0010) [2023-10-11 21:42:09,305][71601] Updated weights for policy 0, policy_version 66460 (0.0009) [2023-10-11 21:42:11,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 136052736. Throughput: 0: 1784.3, 1: 1795.2. Samples: 34021624. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-11 21:42:11,035][70582] Avg episode reward: [(0, '66.210'), (1, '85.220')] [2023-10-11 21:42:12,624][71635] Updated weights for policy 1, policy_version 66402 (0.0009) [2023-10-11 21:42:12,994][71635] Updated weights for policy 1, policy_version 66412 (0.0007) [2023-10-11 21:42:13,215][71601] Updated weights for policy 0, policy_version 66470 (0.0008) [2023-10-11 21:42:13,355][71635] Updated weights for policy 1, policy_version 66422 (0.0007) [2023-10-11 21:42:13,591][71601] Updated weights for policy 0, policy_version 66480 (0.0007) [2023-10-11 21:42:13,722][71635] Updated weights for policy 1, policy_version 66432 (0.0007) [2023-10-11 21:42:13,951][71601] Updated weights for policy 0, policy_version 66490 (0.0008) [2023-10-11 21:42:16,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 136118272. Throughput: 0: 1801.5, 1: 1809.0. Samples: 34032782. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-11 21:42:16,035][70582] Avg episode reward: [(0, '65.110'), (1, '82.400')] [2023-10-11 21:42:17,462][71635] Updated weights for policy 1, policy_version 66442 (0.0008) [2023-10-11 21:42:17,635][71601] Updated weights for policy 0, policy_version 66500 (0.0008) [2023-10-11 21:42:17,829][71635] Updated weights for policy 1, policy_version 66452 (0.0008) [2023-10-11 21:42:18,005][71601] Updated weights for policy 0, policy_version 66510 (0.0008) [2023-10-11 21:42:18,188][71635] Updated weights for policy 1, policy_version 66462 (0.0007) [2023-10-11 21:42:18,382][71601] Updated weights for policy 0, policy_version 66520 (0.0007) [2023-10-11 21:42:21,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 136183808. Throughput: 0: 1790.0, 1: 1806.1. Samples: 34054256. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-11 21:42:21,034][70582] Avg episode reward: [(0, '64.040'), (1, '88.220')] [2023-10-11 21:42:21,893][71635] Updated weights for policy 1, policy_version 66472 (0.0007) [2023-10-11 21:42:21,911][71601] Updated weights for policy 0, policy_version 66530 (0.0007) [2023-10-11 21:42:22,261][71635] Updated weights for policy 1, policy_version 66482 (0.0007) [2023-10-11 21:42:22,278][71601] Updated weights for policy 0, policy_version 66540 (0.0008) [2023-10-11 21:42:22,620][71635] Updated weights for policy 1, policy_version 66492 (0.0008) [2023-10-11 21:42:22,661][71601] Updated weights for policy 0, policy_version 66550 (0.0008) [2023-10-11 21:42:23,029][71601] Updated weights for policy 0, policy_version 66560 (0.0009) [2023-10-11 21:42:26,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 136249344. Throughput: 0: 1796.9, 1: 1812.8. Samples: 34077074. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-11 21:42:26,034][70582] Avg episode reward: [(0, '62.780'), (1, '85.400')] [2023-10-11 21:42:26,320][71635] Updated weights for policy 1, policy_version 66502 (0.0007) [2023-10-11 21:42:26,685][71635] Updated weights for policy 1, policy_version 66512 (0.0008) [2023-10-11 21:42:26,863][71601] Updated weights for policy 0, policy_version 66570 (0.0009) [2023-10-11 21:42:27,064][71635] Updated weights for policy 1, policy_version 66522 (0.0008) [2023-10-11 21:42:27,230][71601] Updated weights for policy 0, policy_version 66580 (0.0008) [2023-10-11 21:42:27,600][71601] Updated weights for policy 0, policy_version 66590 (0.0009) [2023-10-11 21:42:30,927][71635] Updated weights for policy 1, policy_version 66532 (0.0009) [2023-10-11 21:42:31,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 136314880. Throughput: 0: 1796.4, 1: 1811.0. Samples: 34086666. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-11 21:42:31,034][70582] Avg episode reward: [(0, '62.780'), (1, '84.490')] [2023-10-11 21:42:31,299][71635] Updated weights for policy 1, policy_version 66542 (0.0008) [2023-10-11 21:42:31,397][71601] Updated weights for policy 0, policy_version 66600 (0.0009) [2023-10-11 21:42:31,653][71635] Updated weights for policy 1, policy_version 66552 (0.0008) [2023-10-11 21:42:31,769][71601] Updated weights for policy 0, policy_version 66610 (0.0007) [2023-10-11 21:42:32,144][71601] Updated weights for policy 0, policy_version 66620 (0.0008) [2023-10-11 21:42:35,297][71635] Updated weights for policy 1, policy_version 66562 (0.0009) [2023-10-11 21:42:35,664][71635] Updated weights for policy 1, policy_version 66572 (0.0008) [2023-10-11 21:42:35,755][71601] Updated weights for policy 0, policy_version 66630 (0.0009) [2023-10-11 21:42:36,031][71635] Updated weights for policy 1, policy_version 66582 (0.0008) [2023-10-11 21:42:36,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 136380416. Throughput: 0: 1798.6, 1: 1805.4. Samples: 34109294. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-11 21:42:36,035][70582] Avg episode reward: [(0, '61.790'), (1, '87.040')] [2023-10-11 21:42:36,124][71601] Updated weights for policy 0, policy_version 66640 (0.0007) [2023-10-11 21:42:36,393][71635] Updated weights for policy 1, policy_version 66592 (0.0010) [2023-10-11 21:42:36,507][71601] Updated weights for policy 0, policy_version 66650 (0.0008) [2023-10-11 21:42:39,985][71635] Updated weights for policy 1, policy_version 66602 (0.0010) [2023-10-11 21:42:40,215][71601] Updated weights for policy 0, policy_version 66660 (0.0008) [2023-10-11 21:42:40,350][71635] Updated weights for policy 1, policy_version 66612 (0.0008) [2023-10-11 21:42:40,583][71601] Updated weights for policy 0, policy_version 66670 (0.0009) [2023-10-11 21:42:40,712][71635] Updated weights for policy 1, policy_version 66622 (0.0007) [2023-10-11 21:42:40,956][71601] Updated weights for policy 0, policy_version 66680 (0.0010) [2023-10-11 21:42:41,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 136478720. Throughput: 0: 1814.2, 1: 1806.2. Samples: 34130792. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-11 21:42:41,034][70582] Avg episode reward: [(0, '66.080'), (1, '80.870')] [2023-10-11 21:42:44,618][71601] Updated weights for policy 0, policy_version 66690 (0.0008) [2023-10-11 21:42:44,712][71635] Updated weights for policy 1, policy_version 66632 (0.0009) [2023-10-11 21:42:44,982][71601] Updated weights for policy 0, policy_version 66700 (0.0008) [2023-10-11 21:42:45,101][71635] Updated weights for policy 1, policy_version 66642 (0.0009) [2023-10-11 21:42:45,344][71601] Updated weights for policy 0, policy_version 66710 (0.0008) [2023-10-11 21:42:45,460][71635] Updated weights for policy 1, policy_version 66652 (0.0009) [2023-10-11 21:42:45,716][71601] Updated weights for policy 0, policy_version 66720 (0.0008) [2023-10-11 21:42:46,034][70582] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 136577024. Throughput: 0: 1808.9, 1: 1800.9. Samples: 34142156. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-11 21:42:46,035][70582] Avg episode reward: [(0, '60.980'), (1, '81.010')] [2023-10-11 21:42:49,092][71635] Updated weights for policy 1, policy_version 66662 (0.0008) [2023-10-11 21:42:49,337][71601] Updated weights for policy 0, policy_version 66730 (0.0008) [2023-10-11 21:42:49,456][71635] Updated weights for policy 1, policy_version 66672 (0.0009) [2023-10-11 21:42:49,713][71601] Updated weights for policy 0, policy_version 66740 (0.0007) [2023-10-11 21:42:49,819][71635] Updated weights for policy 1, policy_version 66682 (0.0007) [2023-10-11 21:42:50,088][71601] Updated weights for policy 0, policy_version 66750 (0.0008) [2023-10-11 21:42:51,034][70582] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 136642560. Throughput: 0: 1812.8, 1: 1808.5. Samples: 34163456. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-11 21:42:51,035][70582] Avg episode reward: [(0, '64.580'), (1, '84.260')] [2023-10-11 21:42:53,704][71635] Updated weights for policy 1, policy_version 66692 (0.0007) [2023-10-11 21:42:53,807][71601] Updated weights for policy 0, policy_version 66760 (0.0007) [2023-10-11 21:42:54,063][71635] Updated weights for policy 1, policy_version 66702 (0.0008) [2023-10-11 21:42:54,177][71601] Updated weights for policy 0, policy_version 66770 (0.0009) [2023-10-11 21:42:54,435][71635] Updated weights for policy 1, policy_version 66712 (0.0009) [2023-10-11 21:42:54,545][71601] Updated weights for policy 0, policy_version 66780 (0.0009) [2023-10-11 21:42:56,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 136708096. Throughput: 0: 1818.8, 1: 1800.5. Samples: 34184494. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-11 21:42:56,034][70582] Avg episode reward: [(0, '61.160'), (1, '81.270')] [2023-10-11 21:42:58,118][71635] Updated weights for policy 1, policy_version 66722 (0.0009) [2023-10-11 21:42:58,176][71601] Updated weights for policy 0, policy_version 66790 (0.0007) [2023-10-11 21:42:58,481][71635] Updated weights for policy 1, policy_version 66732 (0.0009) [2023-10-11 21:42:58,550][71601] Updated weights for policy 0, policy_version 66800 (0.0008) [2023-10-11 21:42:58,840][71635] Updated weights for policy 1, policy_version 66742 (0.0007) [2023-10-11 21:42:58,929][71601] Updated weights for policy 0, policy_version 66810 (0.0008) [2023-10-11 21:42:59,207][71635] Updated weights for policy 1, policy_version 66752 (0.0008) [2023-10-11 21:43:01,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 136773632. Throughput: 0: 1820.3, 1: 1813.3. Samples: 34196292. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-11 21:43:01,035][70582] Avg episode reward: [(0, '61.260'), (1, '81.990')] [2023-10-11 21:43:02,663][71601] Updated weights for policy 0, policy_version 66820 (0.0008) [2023-10-11 21:43:03,039][71601] Updated weights for policy 0, policy_version 66830 (0.0008) [2023-10-11 21:43:03,051][71635] Updated weights for policy 1, policy_version 66762 (0.0008) [2023-10-11 21:43:03,410][71601] Updated weights for policy 0, policy_version 66840 (0.0008) [2023-10-11 21:43:03,413][71635] Updated weights for policy 1, policy_version 66772 (0.0007) [2023-10-11 21:43:03,796][71635] Updated weights for policy 1, policy_version 66782 (0.0009) [2023-10-11 21:43:06,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 136839168. Throughput: 0: 1815.4, 1: 1794.5. Samples: 34216704. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-11 21:43:06,034][70582] Avg episode reward: [(0, '60.390'), (1, '78.540')] [2023-10-11 21:43:07,192][71601] Updated weights for policy 0, policy_version 66850 (0.0008) [2023-10-11 21:43:07,446][71635] Updated weights for policy 1, policy_version 66792 (0.0007) [2023-10-11 21:43:07,573][71601] Updated weights for policy 0, policy_version 66860 (0.0008) [2023-10-11 21:43:07,816][71635] Updated weights for policy 1, policy_version 66802 (0.0007) [2023-10-11 21:43:07,944][71601] Updated weights for policy 0, policy_version 66870 (0.0007) [2023-10-11 21:43:08,198][71635] Updated weights for policy 1, policy_version 66812 (0.0008) [2023-10-11 21:43:08,312][71601] Updated weights for policy 0, policy_version 66880 (0.0008) [2023-10-11 21:43:11,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 136904704. Throughput: 0: 1815.6, 1: 1795.2. Samples: 34239560. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-11 21:43:11,035][70582] Avg episode reward: [(0, '60.830'), (1, '80.580')] [2023-10-11 21:43:11,870][71635] Updated weights for policy 1, policy_version 66822 (0.0009) [2023-10-11 21:43:11,982][71601] Updated weights for policy 0, policy_version 66890 (0.0009) [2023-10-11 21:43:12,234][71635] Updated weights for policy 1, policy_version 66832 (0.0008) [2023-10-11 21:43:12,352][71601] Updated weights for policy 0, policy_version 66900 (0.0009) [2023-10-11 21:43:12,597][71635] Updated weights for policy 1, policy_version 66842 (0.0008) [2023-10-11 21:43:12,727][71601] Updated weights for policy 0, policy_version 66910 (0.0009) [2023-10-11 21:43:16,034][70582] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 136970240. Throughput: 0: 1817.7, 1: 1795.8. Samples: 34249276. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-11 21:43:16,035][70582] Avg episode reward: [(0, '57.670'), (1, '75.060')] [2023-10-11 21:43:16,352][71635] Updated weights for policy 1, policy_version 66852 (0.0008) [2023-10-11 21:43:16,422][71601] Updated weights for policy 0, policy_version 66920 (0.0009) [2023-10-11 21:43:16,710][71635] Updated weights for policy 1, policy_version 66862 (0.0010) [2023-10-11 21:43:16,791][71601] Updated weights for policy 0, policy_version 66930 (0.0009) [2023-10-11 21:43:17,077][71635] Updated weights for policy 1, policy_version 66872 (0.0008) [2023-10-11 21:43:17,166][71601] Updated weights for policy 0, policy_version 66940 (0.0011) [2023-10-11 21:43:20,806][71635] Updated weights for policy 1, policy_version 66882 (0.0007) [2023-10-11 21:43:20,815][71601] Updated weights for policy 0, policy_version 66950 (0.0008) [2023-10-11 21:43:21,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 137035776. Throughput: 0: 1818.3, 1: 1798.2. Samples: 34272036. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:43:21,034][70582] Avg episode reward: [(0, '58.620'), (1, '75.540')] [2023-10-11 21:43:21,171][71635] Updated weights for policy 1, policy_version 66892 (0.0008) [2023-10-11 21:43:21,189][71601] Updated weights for policy 0, policy_version 66960 (0.0008) [2023-10-11 21:43:21,531][71635] Updated weights for policy 1, policy_version 66902 (0.0008) [2023-10-11 21:43:21,564][71601] Updated weights for policy 0, policy_version 66970 (0.0008) [2023-10-11 21:43:21,904][71635] Updated weights for policy 1, policy_version 66912 (0.0007) [2023-10-11 21:43:25,136][71601] Updated weights for policy 0, policy_version 66980 (0.0009) [2023-10-11 21:43:25,509][71601] Updated weights for policy 0, policy_version 66990 (0.0008) [2023-10-11 21:43:25,710][71635] Updated weights for policy 1, policy_version 66922 (0.0008) [2023-10-11 21:43:25,874][71601] Updated weights for policy 0, policy_version 67000 (0.0007) [2023-10-11 21:43:26,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 137101312. Throughput: 0: 1815.4, 1: 1815.4. Samples: 34294178. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:43:26,035][70582] Avg episode reward: [(0, '62.010'), (1, '79.030')] [2023-10-11 21:43:26,084][71635] Updated weights for policy 1, policy_version 66932 (0.0009) [2023-10-11 21:43:26,445][71635] Updated weights for policy 1, policy_version 66942 (0.0009) [2023-10-11 21:43:29,637][71601] Updated weights for policy 0, policy_version 67010 (0.0008) [2023-10-11 21:43:30,002][71601] Updated weights for policy 0, policy_version 67020 (0.0009) [2023-10-11 21:43:30,182][71635] Updated weights for policy 1, policy_version 66952 (0.0010) [2023-10-11 21:43:30,371][71601] Updated weights for policy 0, policy_version 67030 (0.0008) [2023-10-11 21:43:30,545][71635] Updated weights for policy 1, policy_version 66962 (0.0009) [2023-10-11 21:43:30,740][71601] Updated weights for policy 0, policy_version 67040 (0.0007) [2023-10-11 21:43:30,918][71635] Updated weights for policy 1, policy_version 66972 (0.0010) [2023-10-11 21:43:31,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 137199616. Throughput: 0: 1810.0, 1: 1800.4. Samples: 34304622. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:43:31,035][70582] Avg episode reward: [(0, '58.580'), (1, '81.340')] [2023-10-11 21:43:34,539][71601] Updated weights for policy 0, policy_version 67050 (0.0007) [2023-10-11 21:43:34,696][71635] Updated weights for policy 1, policy_version 66982 (0.0009) [2023-10-11 21:43:34,919][71601] Updated weights for policy 0, policy_version 67060 (0.0008) [2023-10-11 21:43:35,063][71635] Updated weights for policy 1, policy_version 66992 (0.0008) [2023-10-11 21:43:35,294][71601] Updated weights for policy 0, policy_version 67070 (0.0008) [2023-10-11 21:43:35,420][71635] Updated weights for policy 1, policy_version 67002 (0.0008) [2023-10-11 21:43:36,034][70582] Fps is (10 sec: 19661.2, 60 sec: 15291.8, 300 sec: 14551.2). Total num frames: 137297920. Throughput: 0: 1816.9, 1: 1807.0. Samples: 34326530. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:43:36,034][70582] Avg episode reward: [(0, '60.370'), (1, '82.570')] [2023-10-11 21:43:38,859][71601] Updated weights for policy 0, policy_version 67080 (0.0008) [2023-10-11 21:43:39,137][71635] Updated weights for policy 1, policy_version 67012 (0.0008) [2023-10-11 21:43:39,228][71601] Updated weights for policy 0, policy_version 67090 (0.0007) [2023-10-11 21:43:39,507][71635] Updated weights for policy 1, policy_version 67022 (0.0009) [2023-10-11 21:43:39,604][71601] Updated weights for policy 0, policy_version 67100 (0.0007) [2023-10-11 21:43:39,873][71635] Updated weights for policy 1, policy_version 67032 (0.0008) [2023-10-11 21:43:41,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 137363456. Throughput: 0: 1818.3, 1: 1796.7. Samples: 34347174. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:43:41,035][70582] Avg episode reward: [(0, '62.460'), (1, '85.960')] [2023-10-11 21:43:43,308][71601] Updated weights for policy 0, policy_version 67110 (0.0007) [2023-10-11 21:43:43,472][71635] Updated weights for policy 1, policy_version 67042 (0.0007) [2023-10-11 21:43:43,692][71601] Updated weights for policy 0, policy_version 67120 (0.0009) [2023-10-11 21:43:43,835][71635] Updated weights for policy 1, policy_version 67052 (0.0008) [2023-10-11 21:43:44,055][71601] Updated weights for policy 0, policy_version 67130 (0.0008) [2023-10-11 21:43:44,198][71635] Updated weights for policy 1, policy_version 67062 (0.0007) [2023-10-11 21:43:44,573][71635] Updated weights for policy 1, policy_version 67072 (0.0008) [2023-10-11 21:43:46,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 137428992. Throughput: 0: 1817.2, 1: 1810.6. Samples: 34359544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:43:46,035][70582] Avg episode reward: [(0, '64.640'), (1, '90.030')] [2023-10-11 21:43:47,636][71601] Updated weights for policy 0, policy_version 67140 (0.0009) [2023-10-11 21:43:48,003][71601] Updated weights for policy 0, policy_version 67150 (0.0009) [2023-10-11 21:43:48,353][71635] Updated weights for policy 1, policy_version 67082 (0.0008) [2023-10-11 21:43:48,373][71601] Updated weights for policy 0, policy_version 67160 (0.0008) [2023-10-11 21:43:48,714][71635] Updated weights for policy 1, policy_version 67092 (0.0009) [2023-10-11 21:43:49,082][71635] Updated weights for policy 1, policy_version 67102 (0.0011) [2023-10-11 21:43:51,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 137494528. Throughput: 0: 1826.3, 1: 1802.4. Samples: 34379996. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:43:51,034][70582] Avg episode reward: [(0, '68.480'), (1, '95.330')] [2023-10-11 21:43:52,164][71601] Updated weights for policy 0, policy_version 67170 (0.0007) [2023-10-11 21:43:52,530][71601] Updated weights for policy 0, policy_version 67180 (0.0007) [2023-10-11 21:43:52,772][71635] Updated weights for policy 1, policy_version 67112 (0.0007) [2023-10-11 21:43:52,913][71601] Updated weights for policy 0, policy_version 67190 (0.0007) [2023-10-11 21:43:53,131][71635] Updated weights for policy 1, policy_version 67122 (0.0008) [2023-10-11 21:43:53,273][71601] Updated weights for policy 0, policy_version 67200 (0.0009) [2023-10-11 21:43:53,495][71635] Updated weights for policy 1, policy_version 67132 (0.0008) [2023-10-11 21:43:56,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 137560064. Throughput: 0: 1822.4, 1: 1806.0. Samples: 34402836. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:43:56,035][70582] Avg episode reward: [(0, '70.500'), (1, '95.880')] [2023-10-11 21:43:56,043][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000067200_68812800.pth... [2023-10-11 21:43:56,043][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000067136_68747264.pth... [2023-10-11 21:43:56,082][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000065504_67076096.pth [2023-10-11 21:43:56,083][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000065440_67010560.pth [2023-10-11 21:43:56,952][71601] Updated weights for policy 0, policy_version 67210 (0.0007) [2023-10-11 21:43:57,177][71635] Updated weights for policy 1, policy_version 67142 (0.0009) [2023-10-11 21:43:57,330][71601] Updated weights for policy 0, policy_version 67220 (0.0007) [2023-10-11 21:43:57,545][71635] Updated weights for policy 1, policy_version 67152 (0.0008) [2023-10-11 21:43:57,706][71601] Updated weights for policy 0, policy_version 67230 (0.0007) [2023-10-11 21:43:57,900][71635] Updated weights for policy 1, policy_version 67162 (0.0008) [2023-10-11 21:44:01,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 137625600. Throughput: 0: 1824.3, 1: 1804.9. Samples: 34412590. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:44:01,034][70582] Avg episode reward: [(0, '69.160'), (1, '95.360')] [2023-10-11 21:44:01,454][71601] Updated weights for policy 0, policy_version 67240 (0.0008) [2023-10-11 21:44:01,635][71635] Updated weights for policy 1, policy_version 67172 (0.0008) [2023-10-11 21:44:01,821][71601] Updated weights for policy 0, policy_version 67250 (0.0009) [2023-10-11 21:44:02,000][71635] Updated weights for policy 1, policy_version 67182 (0.0009) [2023-10-11 21:44:02,197][71601] Updated weights for policy 0, policy_version 67260 (0.0009) [2023-10-11 21:44:02,368][71635] Updated weights for policy 1, policy_version 67192 (0.0008) [2023-10-11 21:44:05,796][71601] Updated weights for policy 0, policy_version 67270 (0.0009) [2023-10-11 21:44:06,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 137691136. Throughput: 0: 1816.0, 1: 1802.5. Samples: 34434870. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-10-11 21:44:06,034][70582] Avg episode reward: [(0, '68.890'), (1, '98.420')] [2023-10-11 21:44:06,138][71635] Updated weights for policy 1, policy_version 67202 (0.0009) [2023-10-11 21:44:06,162][71601] Updated weights for policy 0, policy_version 67280 (0.0007) [2023-10-11 21:44:06,507][71635] Updated weights for policy 1, policy_version 67212 (0.0007) [2023-10-11 21:44:06,535][71601] Updated weights for policy 0, policy_version 67290 (0.0008) [2023-10-11 21:44:06,868][71635] Updated weights for policy 1, policy_version 67222 (0.0007) [2023-10-11 21:44:07,234][71635] Updated weights for policy 1, policy_version 67232 (0.0008) [2023-10-11 21:44:10,254][71601] Updated weights for policy 0, policy_version 67300 (0.0009) [2023-10-11 21:44:10,628][71601] Updated weights for policy 0, policy_version 67310 (0.0008) [2023-10-11 21:44:10,907][71635] Updated weights for policy 1, policy_version 67242 (0.0008) [2023-10-11 21:44:11,014][71601] Updated weights for policy 0, policy_version 67320 (0.0008) [2023-10-11 21:44:11,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 137756672. Throughput: 0: 1814.4, 1: 1804.1. Samples: 34457014. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-10-11 21:44:11,035][70582] Avg episode reward: [(0, '68.240'), (1, '100.240')] [2023-10-11 21:44:11,276][71635] Updated weights for policy 1, policy_version 67252 (0.0007) [2023-10-11 21:44:11,645][71635] Updated weights for policy 1, policy_version 67262 (0.0009) [2023-10-11 21:44:14,968][71601] Updated weights for policy 0, policy_version 67330 (0.0008) [2023-10-11 21:44:15,344][71601] Updated weights for policy 0, policy_version 67340 (0.0007) [2023-10-11 21:44:15,376][71635] Updated weights for policy 1, policy_version 67272 (0.0007) [2023-10-11 21:44:15,707][71601] Updated weights for policy 0, policy_version 67350 (0.0007) [2023-10-11 21:44:15,746][71635] Updated weights for policy 1, policy_version 67282 (0.0008) [2023-10-11 21:44:16,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 137822208. Throughput: 0: 1811.5, 1: 1802.2. Samples: 34467238. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-10-11 21:44:16,034][70582] Avg episode reward: [(0, '68.190'), (1, '98.810')] [2023-10-11 21:44:16,072][71601] Updated weights for policy 0, policy_version 67360 (0.0007) [2023-10-11 21:44:16,119][71635] Updated weights for policy 1, policy_version 67292 (0.0008) [2023-10-11 21:44:19,686][71601] Updated weights for policy 0, policy_version 67370 (0.0008) [2023-10-11 21:44:19,832][71635] Updated weights for policy 1, policy_version 67302 (0.0009) [2023-10-11 21:44:20,064][71601] Updated weights for policy 0, policy_version 67380 (0.0008) [2023-10-11 21:44:20,202][71635] Updated weights for policy 1, policy_version 67312 (0.0009) [2023-10-11 21:44:20,429][71601] Updated weights for policy 0, policy_version 67390 (0.0008) [2023-10-11 21:44:20,560][71635] Updated weights for policy 1, policy_version 67322 (0.0009) [2023-10-11 21:44:21,034][70582] Fps is (10 sec: 19661.4, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 137953280. Throughput: 0: 1817.6, 1: 1808.2. Samples: 34489688. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-10-11 21:44:21,034][70582] Avg episode reward: [(0, '66.070'), (1, '90.300')] [2023-10-11 21:44:23,986][71601] Updated weights for policy 0, policy_version 67400 (0.0007) [2023-10-11 21:44:24,313][71635] Updated weights for policy 1, policy_version 67332 (0.0008) [2023-10-11 21:44:24,353][71601] Updated weights for policy 0, policy_version 67410 (0.0008) [2023-10-11 21:44:24,681][71635] Updated weights for policy 1, policy_version 67342 (0.0009) [2023-10-11 21:44:24,727][71601] Updated weights for policy 0, policy_version 67420 (0.0009) [2023-10-11 21:44:25,042][71635] Updated weights for policy 1, policy_version 67352 (0.0007) [2023-10-11 21:44:26,034][70582] Fps is (10 sec: 19660.8, 60 sec: 15291.8, 300 sec: 14440.1). Total num frames: 138018816. Throughput: 0: 1808.1, 1: 1808.4. Samples: 34509912. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-10-11 21:44:26,034][70582] Avg episode reward: [(0, '61.780'), (1, '88.450')] [2023-10-11 21:44:28,517][71601] Updated weights for policy 0, policy_version 67430 (0.0009) [2023-10-11 21:44:28,794][71635] Updated weights for policy 1, policy_version 67362 (0.0009) [2023-10-11 21:44:28,891][71601] Updated weights for policy 0, policy_version 67440 (0.0008) [2023-10-11 21:44:29,162][71635] Updated weights for policy 1, policy_version 67372 (0.0009) [2023-10-11 21:44:29,263][71601] Updated weights for policy 0, policy_version 67450 (0.0007) [2023-10-11 21:44:29,528][71635] Updated weights for policy 1, policy_version 67382 (0.0007) [2023-10-11 21:44:29,892][71635] Updated weights for policy 1, policy_version 67392 (0.0007) [2023-10-11 21:44:31,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 138084352. Throughput: 0: 1817.2, 1: 1803.7. Samples: 34522480. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-10-11 21:44:31,034][70582] Avg episode reward: [(0, '61.230'), (1, '88.080')] [2023-10-11 21:44:33,099][71601] Updated weights for policy 0, policy_version 67460 (0.0008) [2023-10-11 21:44:33,469][71601] Updated weights for policy 0, policy_version 67470 (0.0007) [2023-10-11 21:44:33,527][71635] Updated weights for policy 1, policy_version 67402 (0.0007) [2023-10-11 21:44:33,836][71601] Updated weights for policy 0, policy_version 67480 (0.0008) [2023-10-11 21:44:33,888][71635] Updated weights for policy 1, policy_version 67412 (0.0008) [2023-10-11 21:44:34,262][71635] Updated weights for policy 1, policy_version 67422 (0.0007) [2023-10-11 21:44:36,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 138149888. Throughput: 0: 1798.5, 1: 1810.8. Samples: 34542412. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-10-11 21:44:36,034][70582] Avg episode reward: [(0, '60.310'), (1, '85.560')] [2023-10-11 21:44:37,504][71601] Updated weights for policy 0, policy_version 67490 (0.0008) [2023-10-11 21:44:37,877][71601] Updated weights for policy 0, policy_version 67500 (0.0008) [2023-10-11 21:44:37,903][71635] Updated weights for policy 1, policy_version 67432 (0.0009) [2023-10-11 21:44:38,250][71601] Updated weights for policy 0, policy_version 67510 (0.0007) [2023-10-11 21:44:38,276][71635] Updated weights for policy 1, policy_version 67442 (0.0007) [2023-10-11 21:44:38,628][71601] Updated weights for policy 0, policy_version 67520 (0.0007) [2023-10-11 21:44:38,640][71635] Updated weights for policy 1, policy_version 67452 (0.0007) [2023-10-11 21:44:41,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 138215424. Throughput: 0: 1800.0, 1: 1797.8. Samples: 34564736. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-10-11 21:44:41,035][70582] Avg episode reward: [(0, '62.350'), (1, '84.300')] [2023-10-11 21:44:42,335][71601] Updated weights for policy 0, policy_version 67530 (0.0008) [2023-10-11 21:44:42,427][71635] Updated weights for policy 1, policy_version 67462 (0.0008) [2023-10-11 21:44:42,714][71601] Updated weights for policy 0, policy_version 67540 (0.0007) [2023-10-11 21:44:42,796][71635] Updated weights for policy 1, policy_version 67472 (0.0007) [2023-10-11 21:44:43,087][71601] Updated weights for policy 0, policy_version 67550 (0.0008) [2023-10-11 21:44:43,157][71635] Updated weights for policy 1, policy_version 67482 (0.0007) [2023-10-11 21:44:46,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 138280960. Throughput: 0: 1799.2, 1: 1803.7. Samples: 34574720. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-10-11 21:44:46,034][70582] Avg episode reward: [(0, '62.840'), (1, '81.010')] [2023-10-11 21:44:46,629][71601] Updated weights for policy 0, policy_version 67560 (0.0008) [2023-10-11 21:44:46,852][71635] Updated weights for policy 1, policy_version 67492 (0.0009) [2023-10-11 21:44:47,005][71601] Updated weights for policy 0, policy_version 67570 (0.0009) [2023-10-11 21:44:47,230][71635] Updated weights for policy 1, policy_version 67502 (0.0007) [2023-10-11 21:44:47,366][71601] Updated weights for policy 0, policy_version 67580 (0.0008) [2023-10-11 21:44:47,592][71635] Updated weights for policy 1, policy_version 67512 (0.0009) [2023-10-11 21:44:50,926][71601] Updated weights for policy 0, policy_version 67590 (0.0008) [2023-10-11 21:44:51,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 138346496. Throughput: 0: 1815.7, 1: 1804.1. Samples: 34597762. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:44:51,034][70582] Avg episode reward: [(0, '68.820'), (1, '79.110')] [2023-10-11 21:44:51,303][71601] Updated weights for policy 0, policy_version 67600 (0.0010) [2023-10-11 21:44:51,455][71635] Updated weights for policy 1, policy_version 67522 (0.0007) [2023-10-11 21:44:51,673][71601] Updated weights for policy 0, policy_version 67610 (0.0008) [2023-10-11 21:44:51,825][71635] Updated weights for policy 1, policy_version 67532 (0.0008) [2023-10-11 21:44:52,188][71635] Updated weights for policy 1, policy_version 67542 (0.0009) [2023-10-11 21:44:52,549][71635] Updated weights for policy 1, policy_version 67552 (0.0011) [2023-10-11 21:44:55,499][71601] Updated weights for policy 0, policy_version 67620 (0.0009) [2023-10-11 21:44:55,869][71601] Updated weights for policy 0, policy_version 67630 (0.0008) [2023-10-11 21:44:56,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 138412032. Throughput: 0: 1824.2, 1: 1805.3. Samples: 34620344. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:44:56,034][70582] Avg episode reward: [(0, '69.380'), (1, '78.260')] [2023-10-11 21:44:56,241][71601] Updated weights for policy 0, policy_version 67640 (0.0007) [2023-10-11 21:44:56,346][71635] Updated weights for policy 1, policy_version 67562 (0.0009) [2023-10-11 21:44:56,715][71635] Updated weights for policy 1, policy_version 67572 (0.0008) [2023-10-11 21:44:57,093][71635] Updated weights for policy 1, policy_version 67582 (0.0007) [2023-10-11 21:44:59,946][71601] Updated weights for policy 0, policy_version 67650 (0.0007) [2023-10-11 21:45:00,312][71601] Updated weights for policy 0, policy_version 67660 (0.0007) [2023-10-11 21:45:00,678][71601] Updated weights for policy 0, policy_version 67670 (0.0007) [2023-10-11 21:45:00,784][71635] Updated weights for policy 1, policy_version 67592 (0.0010) [2023-10-11 21:45:01,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 138477568. Throughput: 0: 1819.1, 1: 1804.3. Samples: 34630294. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:45:01,035][70582] Avg episode reward: [(0, '70.600'), (1, '79.740')] [2023-10-11 21:45:01,051][71601] Updated weights for policy 0, policy_version 67680 (0.0008) [2023-10-11 21:45:01,162][71635] Updated weights for policy 1, policy_version 67602 (0.0009) [2023-10-11 21:45:01,524][71635] Updated weights for policy 1, policy_version 67612 (0.0011) [2023-10-11 21:45:04,772][71601] Updated weights for policy 0, policy_version 67690 (0.0010) [2023-10-11 21:45:05,144][71635] Updated weights for policy 1, policy_version 67622 (0.0008) [2023-10-11 21:45:05,145][71601] Updated weights for policy 0, policy_version 67700 (0.0008) [2023-10-11 21:45:05,509][71635] Updated weights for policy 1, policy_version 67632 (0.0008) [2023-10-11 21:45:05,511][71601] Updated weights for policy 0, policy_version 67710 (0.0008) [2023-10-11 21:45:05,867][71635] Updated weights for policy 1, policy_version 67642 (0.0008) [2023-10-11 21:45:06,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 138575872. Throughput: 0: 1820.5, 1: 1803.5. Samples: 34652770. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:45:06,034][70582] Avg episode reward: [(0, '75.980'), (1, '81.380')] [2023-10-11 21:45:09,366][71601] Updated weights for policy 0, policy_version 67720 (0.0009) [2023-10-11 21:45:09,630][71635] Updated weights for policy 1, policy_version 67652 (0.0009) [2023-10-11 21:45:09,732][71601] Updated weights for policy 0, policy_version 67730 (0.0007) [2023-10-11 21:45:10,000][71635] Updated weights for policy 1, policy_version 67662 (0.0008) [2023-10-11 21:45:10,113][71601] Updated weights for policy 0, policy_version 67740 (0.0009) [2023-10-11 21:45:10,366][71635] Updated weights for policy 1, policy_version 67672 (0.0009) [2023-10-11 21:45:11,034][70582] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 138674176. Throughput: 0: 1809.7, 1: 1812.7. Samples: 34672922. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:45:11,035][70582] Avg episode reward: [(0, '79.580'), (1, '85.090')] [2023-10-11 21:45:13,659][71601] Updated weights for policy 0, policy_version 67750 (0.0007) [2023-10-11 21:45:14,021][71601] Updated weights for policy 0, policy_version 67760 (0.0010) [2023-10-11 21:45:14,315][71635] Updated weights for policy 1, policy_version 67682 (0.0009) [2023-10-11 21:45:14,394][71601] Updated weights for policy 0, policy_version 67770 (0.0008) [2023-10-11 21:45:14,676][71635] Updated weights for policy 1, policy_version 67692 (0.0008) [2023-10-11 21:45:15,048][71635] Updated weights for policy 1, policy_version 67702 (0.0010) [2023-10-11 21:45:15,409][71635] Updated weights for policy 1, policy_version 67712 (0.0007) [2023-10-11 21:45:16,034][70582] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 138739712. Throughput: 0: 1819.5, 1: 1799.9. Samples: 34685350. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:45:16,034][70582] Avg episode reward: [(0, '84.570'), (1, '84.760')] [2023-10-11 21:45:18,002][71601] Updated weights for policy 0, policy_version 67780 (0.0009) [2023-10-11 21:45:18,360][71601] Updated weights for policy 0, policy_version 67790 (0.0010) [2023-10-11 21:45:18,736][71601] Updated weights for policy 0, policy_version 67800 (0.0009) [2023-10-11 21:45:19,167][71635] Updated weights for policy 1, policy_version 67722 (0.0008) [2023-10-11 21:45:19,526][71635] Updated weights for policy 1, policy_version 67732 (0.0008) [2023-10-11 21:45:19,891][71635] Updated weights for policy 1, policy_version 67742 (0.0009) [2023-10-11 21:45:21,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 138805248. Throughput: 0: 1822.5, 1: 1814.4. Samples: 34706072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:45:21,034][70582] Avg episode reward: [(0, '81.400'), (1, '84.710')] [2023-10-11 21:45:22,424][71601] Updated weights for policy 0, policy_version 67810 (0.0008) [2023-10-11 21:45:22,803][71601] Updated weights for policy 0, policy_version 67820 (0.0007) [2023-10-11 21:45:23,168][71601] Updated weights for policy 0, policy_version 67830 (0.0008) [2023-10-11 21:45:23,513][71635] Updated weights for policy 1, policy_version 67752 (0.0008) [2023-10-11 21:45:23,537][71601] Updated weights for policy 0, policy_version 67840 (0.0008) [2023-10-11 21:45:23,875][71635] Updated weights for policy 1, policy_version 67762 (0.0010) [2023-10-11 21:45:24,252][71635] Updated weights for policy 1, policy_version 67772 (0.0010) [2023-10-11 21:45:26,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 138870784. Throughput: 0: 1821.7, 1: 1808.9. Samples: 34728114. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:45:26,035][70582] Avg episode reward: [(0, '85.520'), (1, '83.940')] [2023-10-11 21:45:27,407][71601] Updated weights for policy 0, policy_version 67850 (0.0008) [2023-10-11 21:45:27,745][71635] Updated weights for policy 1, policy_version 67782 (0.0007) [2023-10-11 21:45:27,775][71601] Updated weights for policy 0, policy_version 67860 (0.0008) [2023-10-11 21:45:28,108][71635] Updated weights for policy 1, policy_version 67792 (0.0007) [2023-10-11 21:45:28,156][71601] Updated weights for policy 0, policy_version 67870 (0.0007) [2023-10-11 21:45:28,464][71635] Updated weights for policy 1, policy_version 67802 (0.0009) [2023-10-11 21:45:31,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 138936320. Throughput: 0: 1819.7, 1: 1818.8. Samples: 34738450. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:45:31,034][70582] Avg episode reward: [(0, '81.490'), (1, '91.220')] [2023-10-11 21:45:31,812][71601] Updated weights for policy 0, policy_version 67880 (0.0007) [2023-10-11 21:45:32,178][71601] Updated weights for policy 0, policy_version 67890 (0.0007) [2023-10-11 21:45:32,252][71635] Updated weights for policy 1, policy_version 67812 (0.0010) [2023-10-11 21:45:32,547][71601] Updated weights for policy 0, policy_version 67900 (0.0008) [2023-10-11 21:45:32,619][71635] Updated weights for policy 1, policy_version 67822 (0.0007) [2023-10-11 21:45:32,978][71635] Updated weights for policy 1, policy_version 67832 (0.0010) [2023-10-11 21:45:36,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 139001856. Throughput: 0: 1810.0, 1: 1809.7. Samples: 34760650. Policy #0 lag: (min: 10.0, avg: 10.3, max: 20.0) [2023-10-11 21:45:36,035][70582] Avg episode reward: [(0, '79.400'), (1, '92.880')] [2023-10-11 21:45:36,275][71601] Updated weights for policy 0, policy_version 67910 (0.0008) [2023-10-11 21:45:36,640][71601] Updated weights for policy 0, policy_version 67920 (0.0010) [2023-10-11 21:45:36,679][71635] Updated weights for policy 1, policy_version 67842 (0.0009) [2023-10-11 21:45:37,008][71601] Updated weights for policy 0, policy_version 67930 (0.0008) [2023-10-11 21:45:37,046][71635] Updated weights for policy 1, policy_version 67852 (0.0008) [2023-10-11 21:45:37,416][71635] Updated weights for policy 1, policy_version 67862 (0.0008) [2023-10-11 21:45:37,791][71635] Updated weights for policy 1, policy_version 67872 (0.0009) [2023-10-11 21:45:40,761][71601] Updated weights for policy 0, policy_version 67940 (0.0008) [2023-10-11 21:45:41,034][70582] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 139067392. Throughput: 0: 1811.5, 1: 1807.0. Samples: 34783180. Policy #0 lag: (min: 10.0, avg: 10.3, max: 20.0) [2023-10-11 21:45:41,035][70582] Avg episode reward: [(0, '81.530'), (1, '93.030')] [2023-10-11 21:45:41,128][71601] Updated weights for policy 0, policy_version 67950 (0.0010) [2023-10-11 21:45:41,506][71601] Updated weights for policy 0, policy_version 67960 (0.0008) [2023-10-11 21:45:41,644][71635] Updated weights for policy 1, policy_version 67882 (0.0007) [2023-10-11 21:45:42,012][71635] Updated weights for policy 1, policy_version 67892 (0.0009) [2023-10-11 21:45:42,375][71635] Updated weights for policy 1, policy_version 67902 (0.0010) [2023-10-11 21:45:45,305][71601] Updated weights for policy 0, policy_version 67970 (0.0008) [2023-10-11 21:45:45,675][71601] Updated weights for policy 0, policy_version 67980 (0.0007) [2023-10-11 21:45:46,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 139132928. Throughput: 0: 1807.1, 1: 1804.8. Samples: 34792830. Policy #0 lag: (min: 10.0, avg: 10.3, max: 20.0) [2023-10-11 21:45:46,034][70582] Avg episode reward: [(0, '82.640'), (1, '95.210')] [2023-10-11 21:45:46,052][71601] Updated weights for policy 0, policy_version 67990 (0.0008) [2023-10-11 21:45:46,143][71635] Updated weights for policy 1, policy_version 67912 (0.0007) [2023-10-11 21:45:46,413][71601] Updated weights for policy 0, policy_version 68000 (0.0009) [2023-10-11 21:45:46,504][71635] Updated weights for policy 1, policy_version 67922 (0.0008) [2023-10-11 21:45:46,870][71635] Updated weights for policy 1, policy_version 67932 (0.0007) [2023-10-11 21:45:50,066][71601] Updated weights for policy 0, policy_version 68010 (0.0007) [2023-10-11 21:45:50,440][71601] Updated weights for policy 0, policy_version 68020 (0.0009) [2023-10-11 21:45:50,613][71635] Updated weights for policy 1, policy_version 67942 (0.0007) [2023-10-11 21:45:50,814][71601] Updated weights for policy 0, policy_version 68030 (0.0009) [2023-10-11 21:45:50,989][71635] Updated weights for policy 1, policy_version 67952 (0.0007) [2023-10-11 21:45:51,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 139231232. Throughput: 0: 1809.0, 1: 1809.3. Samples: 34815596. Policy #0 lag: (min: 10.0, avg: 10.3, max: 20.0) [2023-10-11 21:45:51,035][70582] Avg episode reward: [(0, '78.360'), (1, '98.930')] [2023-10-11 21:45:51,345][71635] Updated weights for policy 1, policy_version 67962 (0.0010) [2023-10-11 21:45:54,601][71601] Updated weights for policy 0, policy_version 68040 (0.0008) [2023-10-11 21:45:54,971][71601] Updated weights for policy 0, policy_version 68050 (0.0008) [2023-10-11 21:45:55,188][71635] Updated weights for policy 1, policy_version 67972 (0.0009) [2023-10-11 21:45:55,336][71601] Updated weights for policy 0, policy_version 68060 (0.0008) [2023-10-11 21:45:55,555][71635] Updated weights for policy 1, policy_version 67982 (0.0008) [2023-10-11 21:45:55,911][71635] Updated weights for policy 1, policy_version 67992 (0.0010) [2023-10-11 21:45:56,034][70582] Fps is (10 sec: 16383.3, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 139296768. Throughput: 0: 1806.6, 1: 1815.6. Samples: 34835922. Policy #0 lag: (min: 10.0, avg: 10.3, max: 20.0) [2023-10-11 21:45:56,035][70582] Avg episode reward: [(0, '77.350'), (1, '101.360')] [2023-10-11 21:45:56,045][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000068064_69697536.pth... [2023-10-11 21:45:56,084][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000066368_67960832.pth [2023-10-11 21:45:56,089][71353] Saving a milestone ./train_atari/atari_gopher_APPO/checkpoint_p0/milestones/checkpoint_000068064_69697536.pth [2023-10-11 21:45:56,204][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000068000_69632000.pth... [2023-10-11 21:45:56,241][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000066304_67895296.pth [2023-10-11 21:45:56,244][71431] Saving a milestone ./train_atari/atari_gopher_APPO/checkpoint_p1/milestones/checkpoint_000068000_69632000.pth [2023-10-11 21:45:58,989][71601] Updated weights for policy 0, policy_version 68070 (0.0010) [2023-10-11 21:45:59,356][71601] Updated weights for policy 0, policy_version 68080 (0.0010) [2023-10-11 21:45:59,561][71635] Updated weights for policy 1, policy_version 68002 (0.0008) [2023-10-11 21:45:59,730][71601] Updated weights for policy 0, policy_version 68090 (0.0008) [2023-10-11 21:45:59,926][71635] Updated weights for policy 1, policy_version 68012 (0.0008) [2023-10-11 21:46:00,289][71635] Updated weights for policy 1, policy_version 68022 (0.0009) [2023-10-11 21:46:00,665][71635] Updated weights for policy 1, policy_version 68032 (0.0008) [2023-10-11 21:46:01,034][70582] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 139395072. Throughput: 0: 1803.4, 1: 1803.7. Samples: 34847670. Policy #0 lag: (min: 10.0, avg: 10.3, max: 20.0) [2023-10-11 21:46:01,035][70582] Avg episode reward: [(0, '74.770'), (1, '101.670')] [2023-10-11 21:46:03,628][71601] Updated weights for policy 0, policy_version 68100 (0.0007) [2023-10-11 21:46:03,996][71601] Updated weights for policy 0, policy_version 68110 (0.0008) [2023-10-11 21:46:04,367][71601] Updated weights for policy 0, policy_version 68120 (0.0008) [2023-10-11 21:46:04,452][71635] Updated weights for policy 1, policy_version 68042 (0.0008) [2023-10-11 21:46:04,817][71635] Updated weights for policy 1, policy_version 68052 (0.0008) [2023-10-11 21:46:05,187][71635] Updated weights for policy 1, policy_version 68062 (0.0009) [2023-10-11 21:46:06,034][70582] Fps is (10 sec: 16384.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 139460608. Throughput: 0: 1800.4, 1: 1804.7. Samples: 34868302. Policy #0 lag: (min: 10.0, avg: 10.3, max: 20.0) [2023-10-11 21:46:06,034][70582] Avg episode reward: [(0, '71.600'), (1, '98.060')] [2023-10-11 21:46:08,157][71601] Updated weights for policy 0, policy_version 68130 (0.0008) [2023-10-11 21:46:08,526][71601] Updated weights for policy 0, policy_version 68140 (0.0010) [2023-10-11 21:46:08,896][71601] Updated weights for policy 0, policy_version 68150 (0.0009) [2023-10-11 21:46:09,048][71635] Updated weights for policy 1, policy_version 68072 (0.0008) [2023-10-11 21:46:09,263][71601] Updated weights for policy 0, policy_version 68160 (0.0007) [2023-10-11 21:46:09,407][71635] Updated weights for policy 1, policy_version 68082 (0.0009) [2023-10-11 21:46:09,772][71635] Updated weights for policy 1, policy_version 68092 (0.0009) [2023-10-11 21:46:11,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 139526144. Throughput: 0: 1790.9, 1: 1791.6. Samples: 34889330. Policy #0 lag: (min: 10.0, avg: 10.3, max: 20.0) [2023-10-11 21:46:11,035][70582] Avg episode reward: [(0, '73.060'), (1, '104.070')] [2023-10-11 21:46:13,079][71601] Updated weights for policy 0, policy_version 68170 (0.0009) [2023-10-11 21:46:13,448][71601] Updated weights for policy 0, policy_version 68180 (0.0007) [2023-10-11 21:46:13,492][71635] Updated weights for policy 1, policy_version 68102 (0.0008) [2023-10-11 21:46:13,827][71601] Updated weights for policy 0, policy_version 68190 (0.0009) [2023-10-11 21:46:13,862][71635] Updated weights for policy 1, policy_version 68112 (0.0009) [2023-10-11 21:46:14,224][71635] Updated weights for policy 1, policy_version 68122 (0.0008) [2023-10-11 21:46:16,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 139591680. Throughput: 0: 1806.0, 1: 1810.0. Samples: 34901172. Policy #0 lag: (min: 10.0, avg: 10.3, max: 20.0) [2023-10-11 21:46:16,034][70582] Avg episode reward: [(0, '71.490'), (1, '104.270')] [2023-10-11 21:46:17,655][71601] Updated weights for policy 0, policy_version 68200 (0.0009) [2023-10-11 21:46:17,792][71635] Updated weights for policy 1, policy_version 68132 (0.0007) [2023-10-11 21:46:18,037][71601] Updated weights for policy 0, policy_version 68210 (0.0008) [2023-10-11 21:46:18,153][71635] Updated weights for policy 1, policy_version 68142 (0.0007) [2023-10-11 21:46:18,399][71601] Updated weights for policy 0, policy_version 68220 (0.0009) [2023-10-11 21:46:18,513][71635] Updated weights for policy 1, policy_version 68152 (0.0007) [2023-10-11 21:46:21,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 139657216. Throughput: 0: 1789.6, 1: 1794.3. Samples: 34921926. Policy #0 lag: (min: 21.0, avg: 28.9, max: 53.0) [2023-10-11 21:46:21,035][70582] Avg episode reward: [(0, '68.500'), (1, '104.140')] [2023-10-11 21:46:22,054][71601] Updated weights for policy 0, policy_version 68230 (0.0008) [2023-10-11 21:46:22,145][71635] Updated weights for policy 1, policy_version 68162 (0.0008) [2023-10-11 21:46:22,421][71601] Updated weights for policy 0, policy_version 68240 (0.0008) [2023-10-11 21:46:22,508][71635] Updated weights for policy 1, policy_version 68172 (0.0007) [2023-10-11 21:46:22,791][71601] Updated weights for policy 0, policy_version 68250 (0.0008) [2023-10-11 21:46:22,880][71635] Updated weights for policy 1, policy_version 68182 (0.0007) [2023-10-11 21:46:23,246][71635] Updated weights for policy 1, policy_version 68192 (0.0007) [2023-10-11 21:46:26,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 139722752. Throughput: 0: 1792.7, 1: 1804.6. Samples: 34945056. Policy #0 lag: (min: 21.0, avg: 28.9, max: 53.0) [2023-10-11 21:46:26,035][70582] Avg episode reward: [(0, '69.540'), (1, '108.090')] [2023-10-11 21:46:26,478][71601] Updated weights for policy 0, policy_version 68260 (0.0009) [2023-10-11 21:46:26,843][71601] Updated weights for policy 0, policy_version 68270 (0.0009) [2023-10-11 21:46:26,905][71635] Updated weights for policy 1, policy_version 68202 (0.0009) [2023-10-11 21:46:27,208][71601] Updated weights for policy 0, policy_version 68280 (0.0009) [2023-10-11 21:46:27,277][71635] Updated weights for policy 1, policy_version 68212 (0.0007) [2023-10-11 21:46:27,641][71635] Updated weights for policy 1, policy_version 68222 (0.0007) [2023-10-11 21:46:30,871][71601] Updated weights for policy 0, policy_version 68290 (0.0009) [2023-10-11 21:46:31,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 139788288. Throughput: 0: 1795.5, 1: 1807.5. Samples: 34954962. Policy #0 lag: (min: 21.0, avg: 28.9, max: 53.0) [2023-10-11 21:46:31,035][70582] Avg episode reward: [(0, '69.270'), (1, '105.710')] [2023-10-11 21:46:31,234][71601] Updated weights for policy 0, policy_version 68300 (0.0008) [2023-10-11 21:46:31,463][71635] Updated weights for policy 1, policy_version 68232 (0.0008) [2023-10-11 21:46:31,599][71601] Updated weights for policy 0, policy_version 68310 (0.0007) [2023-10-11 21:46:31,829][71635] Updated weights for policy 1, policy_version 68242 (0.0008) [2023-10-11 21:46:31,968][71601] Updated weights for policy 0, policy_version 68320 (0.0008) [2023-10-11 21:46:32,193][71635] Updated weights for policy 1, policy_version 68252 (0.0010) [2023-10-11 21:46:35,563][71601] Updated weights for policy 0, policy_version 68330 (0.0010) [2023-10-11 21:46:35,934][71601] Updated weights for policy 0, policy_version 68340 (0.0008) [2023-10-11 21:46:36,024][71635] Updated weights for policy 1, policy_version 68262 (0.0008) [2023-10-11 21:46:36,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 139853824. Throughput: 0: 1795.3, 1: 1803.5. Samples: 34977540. Policy #0 lag: (min: 21.0, avg: 28.9, max: 53.0) [2023-10-11 21:46:36,034][70582] Avg episode reward: [(0, '69.190'), (1, '102.390')] [2023-10-11 21:46:36,306][71601] Updated weights for policy 0, policy_version 68350 (0.0009) [2023-10-11 21:46:36,392][71635] Updated weights for policy 1, policy_version 68272 (0.0007) [2023-10-11 21:46:36,763][71635] Updated weights for policy 1, policy_version 68282 (0.0008) [2023-10-11 21:46:40,073][71601] Updated weights for policy 0, policy_version 68360 (0.0009) [2023-10-11 21:46:40,446][71601] Updated weights for policy 0, policy_version 68370 (0.0007) [2023-10-11 21:46:40,545][71635] Updated weights for policy 1, policy_version 68292 (0.0010) [2023-10-11 21:46:40,813][71601] Updated weights for policy 0, policy_version 68380 (0.0008) [2023-10-11 21:46:40,906][71635] Updated weights for policy 1, policy_version 68302 (0.0008) [2023-10-11 21:46:41,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 139952128. Throughput: 0: 1815.3, 1: 1815.5. Samples: 34999304. Policy #0 lag: (min: 21.0, avg: 28.9, max: 53.0) [2023-10-11 21:46:41,034][70582] Avg episode reward: [(0, '70.300'), (1, '102.250')] [2023-10-11 21:46:41,277][71635] Updated weights for policy 1, policy_version 68312 (0.0010) [2023-10-11 21:46:44,419][71601] Updated weights for policy 0, policy_version 68390 (0.0008) [2023-10-11 21:46:44,796][71601] Updated weights for policy 0, policy_version 68400 (0.0009) [2023-10-11 21:46:44,949][71635] Updated weights for policy 1, policy_version 68322 (0.0009) [2023-10-11 21:46:45,171][71601] Updated weights for policy 0, policy_version 68410 (0.0009) [2023-10-11 21:46:45,318][71635] Updated weights for policy 1, policy_version 68332 (0.0007) [2023-10-11 21:46:45,689][71635] Updated weights for policy 1, policy_version 68342 (0.0009) [2023-10-11 21:46:46,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 140017664. Throughput: 0: 1806.3, 1: 1809.0. Samples: 35010358. Policy #0 lag: (min: 21.0, avg: 28.9, max: 53.0) [2023-10-11 21:46:46,035][70582] Avg episode reward: [(0, '75.120'), (1, '101.910')] [2023-10-11 21:46:46,045][71635] Updated weights for policy 1, policy_version 68352 (0.0008) [2023-10-11 21:46:48,795][71601] Updated weights for policy 0, policy_version 68420 (0.0009) [2023-10-11 21:46:49,178][71601] Updated weights for policy 0, policy_version 68430 (0.0009) [2023-10-11 21:46:49,548][71601] Updated weights for policy 0, policy_version 68440 (0.0008) [2023-10-11 21:46:49,819][71635] Updated weights for policy 1, policy_version 68362 (0.0008) [2023-10-11 21:46:50,187][71635] Updated weights for policy 1, policy_version 68372 (0.0007) [2023-10-11 21:46:50,555][71635] Updated weights for policy 1, policy_version 68382 (0.0009) [2023-10-11 21:46:51,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 140115968. Throughput: 0: 1817.3, 1: 1815.7. Samples: 35031790. Policy #0 lag: (min: 21.0, avg: 28.9, max: 53.0) [2023-10-11 21:46:51,035][70582] Avg episode reward: [(0, '73.110'), (1, '99.340')] [2023-10-11 21:46:53,129][71601] Updated weights for policy 0, policy_version 68450 (0.0008) [2023-10-11 21:46:53,508][71601] Updated weights for policy 0, policy_version 68460 (0.0010) [2023-10-11 21:46:53,883][71601] Updated weights for policy 0, policy_version 68470 (0.0009) [2023-10-11 21:46:54,248][71601] Updated weights for policy 0, policy_version 68480 (0.0008) [2023-10-11 21:46:54,282][71635] Updated weights for policy 1, policy_version 68392 (0.0007) [2023-10-11 21:46:54,650][71635] Updated weights for policy 1, policy_version 68402 (0.0009) [2023-10-11 21:46:55,025][71635] Updated weights for policy 1, policy_version 68412 (0.0007) [2023-10-11 21:46:56,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 140181504. Throughput: 0: 1820.2, 1: 1807.0. Samples: 35052554. Policy #0 lag: (min: 21.0, avg: 28.9, max: 53.0) [2023-10-11 21:46:56,035][70582] Avg episode reward: [(0, '71.470'), (1, '103.280')] [2023-10-11 21:46:58,028][71601] Updated weights for policy 0, policy_version 68490 (0.0011) [2023-10-11 21:46:58,411][71601] Updated weights for policy 0, policy_version 68500 (0.0007) [2023-10-11 21:46:58,587][71635] Updated weights for policy 1, policy_version 68422 (0.0007) [2023-10-11 21:46:58,790][71601] Updated weights for policy 0, policy_version 68510 (0.0008) [2023-10-11 21:46:58,964][71635] Updated weights for policy 1, policy_version 68432 (0.0009) [2023-10-11 21:46:59,322][71635] Updated weights for policy 1, policy_version 68442 (0.0010) [2023-10-11 21:47:01,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 140247040. Throughput: 0: 1819.0, 1: 1810.8. Samples: 35064514. Policy #0 lag: (min: 21.0, avg: 28.9, max: 53.0) [2023-10-11 21:47:01,034][70582] Avg episode reward: [(0, '73.960'), (1, '105.830')] [2023-10-11 21:47:02,596][71601] Updated weights for policy 0, policy_version 68520 (0.0009) [2023-10-11 21:47:02,978][71601] Updated weights for policy 0, policy_version 68530 (0.0008) [2023-10-11 21:47:03,139][71635] Updated weights for policy 1, policy_version 68452 (0.0010) [2023-10-11 21:47:03,344][71601] Updated weights for policy 0, policy_version 68540 (0.0007) [2023-10-11 21:47:03,500][71635] Updated weights for policy 1, policy_version 68462 (0.0008) [2023-10-11 21:47:03,878][71635] Updated weights for policy 1, policy_version 68472 (0.0009) [2023-10-11 21:47:06,034][70582] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 140312576. Throughput: 0: 1817.3, 1: 1802.2. Samples: 35084802. Policy #0 lag: (min: 15.0, avg: 22.9, max: 47.0) [2023-10-11 21:47:06,034][70582] Avg episode reward: [(0, '71.950'), (1, '101.580')] [2023-10-11 21:47:07,098][71601] Updated weights for policy 0, policy_version 68550 (0.0009) [2023-10-11 21:47:07,465][71601] Updated weights for policy 0, policy_version 68560 (0.0009) [2023-10-11 21:47:07,550][71635] Updated weights for policy 1, policy_version 68482 (0.0009) [2023-10-11 21:47:07,835][71601] Updated weights for policy 0, policy_version 68570 (0.0008) [2023-10-11 21:47:07,917][71635] Updated weights for policy 1, policy_version 68492 (0.0008) [2023-10-11 21:47:08,278][71635] Updated weights for policy 1, policy_version 68502 (0.0008) [2023-10-11 21:47:08,646][71635] Updated weights for policy 1, policy_version 68512 (0.0011) [2023-10-11 21:47:11,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 140378112. Throughput: 0: 1809.7, 1: 1794.6. Samples: 35107248. Policy #0 lag: (min: 15.0, avg: 22.9, max: 47.0) [2023-10-11 21:47:11,035][70582] Avg episode reward: [(0, '73.560'), (1, '99.540')] [2023-10-11 21:47:11,687][71601] Updated weights for policy 0, policy_version 68580 (0.0008) [2023-10-11 21:47:12,064][71601] Updated weights for policy 0, policy_version 68590 (0.0008) [2023-10-11 21:47:12,419][71635] Updated weights for policy 1, policy_version 68522 (0.0007) [2023-10-11 21:47:12,431][71601] Updated weights for policy 0, policy_version 68600 (0.0008) [2023-10-11 21:47:12,784][71635] Updated weights for policy 1, policy_version 68532 (0.0008) [2023-10-11 21:47:13,155][71635] Updated weights for policy 1, policy_version 68542 (0.0010) [2023-10-11 21:47:15,978][71601] Updated weights for policy 0, policy_version 68610 (0.0008) [2023-10-11 21:47:16,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 140443648. Throughput: 0: 1810.6, 1: 1794.0. Samples: 35117168. Policy #0 lag: (min: 15.0, avg: 22.9, max: 47.0) [2023-10-11 21:47:16,034][70582] Avg episode reward: [(0, '75.030'), (1, '106.930')] [2023-10-11 21:47:16,356][71601] Updated weights for policy 0, policy_version 68620 (0.0008) [2023-10-11 21:47:16,719][71601] Updated weights for policy 0, policy_version 68630 (0.0008) [2023-10-11 21:47:16,981][71635] Updated weights for policy 1, policy_version 68552 (0.0010) [2023-10-11 21:47:17,085][71601] Updated weights for policy 0, policy_version 68640 (0.0008) [2023-10-11 21:47:17,371][71635] Updated weights for policy 1, policy_version 68562 (0.0009) [2023-10-11 21:47:17,736][71635] Updated weights for policy 1, policy_version 68572 (0.0008) [2023-10-11 21:47:20,710][71601] Updated weights for policy 0, policy_version 68650 (0.0008) [2023-10-11 21:47:21,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 140509184. Throughput: 0: 1814.2, 1: 1793.3. Samples: 35139880. Policy #0 lag: (min: 15.0, avg: 22.9, max: 47.0) [2023-10-11 21:47:21,035][70582] Avg episode reward: [(0, '74.820'), (1, '99.580')] [2023-10-11 21:47:21,077][71601] Updated weights for policy 0, policy_version 68660 (0.0010) [2023-10-11 21:47:21,452][71601] Updated weights for policy 0, policy_version 68670 (0.0008) [2023-10-11 21:47:21,460][71635] Updated weights for policy 1, policy_version 68582 (0.0008) [2023-10-11 21:47:21,818][71635] Updated weights for policy 1, policy_version 68592 (0.0008) [2023-10-11 21:47:22,197][71635] Updated weights for policy 1, policy_version 68602 (0.0009) [2023-10-11 21:47:25,155][71601] Updated weights for policy 0, policy_version 68680 (0.0009) [2023-10-11 21:47:25,516][71601] Updated weights for policy 0, policy_version 68690 (0.0007) [2023-10-11 21:47:25,889][71601] Updated weights for policy 0, policy_version 68700 (0.0009) [2023-10-11 21:47:25,992][71635] Updated weights for policy 1, policy_version 68612 (0.0010) [2023-10-11 21:47:26,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 140574720. Throughput: 0: 1815.1, 1: 1797.9. Samples: 35161888. Policy #0 lag: (min: 15.0, avg: 22.9, max: 47.0) [2023-10-11 21:47:26,034][70582] Avg episode reward: [(0, '70.820'), (1, '104.300')] [2023-10-11 21:47:26,363][71635] Updated weights for policy 1, policy_version 68622 (0.0010) [2023-10-11 21:47:26,734][71635] Updated weights for policy 1, policy_version 68632 (0.0011) [2023-10-11 21:47:29,650][71601] Updated weights for policy 0, policy_version 68710 (0.0010) [2023-10-11 21:47:30,023][71601] Updated weights for policy 0, policy_version 68720 (0.0009) [2023-10-11 21:47:30,392][71601] Updated weights for policy 0, policy_version 68730 (0.0008) [2023-10-11 21:47:30,400][71635] Updated weights for policy 1, policy_version 68642 (0.0010) [2023-10-11 21:47:30,773][71635] Updated weights for policy 1, policy_version 68652 (0.0009) [2023-10-11 21:47:31,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 140673024. Throughput: 0: 1805.3, 1: 1796.1. Samples: 35172420. Policy #0 lag: (min: 15.0, avg: 22.9, max: 47.0) [2023-10-11 21:47:31,035][70582] Avg episode reward: [(0, '72.830'), (1, '99.830')] [2023-10-11 21:47:31,142][71635] Updated weights for policy 1, policy_version 68662 (0.0008) [2023-10-11 21:47:31,511][71635] Updated weights for policy 1, policy_version 68672 (0.0008) [2023-10-11 21:47:34,065][71601] Updated weights for policy 0, policy_version 68740 (0.0008) [2023-10-11 21:47:34,436][71601] Updated weights for policy 0, policy_version 68750 (0.0010) [2023-10-11 21:47:34,800][71601] Updated weights for policy 0, policy_version 68760 (0.0010) [2023-10-11 21:47:35,229][71635] Updated weights for policy 1, policy_version 68682 (0.0007) [2023-10-11 21:47:35,588][71635] Updated weights for policy 1, policy_version 68692 (0.0007) [2023-10-11 21:47:35,960][71635] Updated weights for policy 1, policy_version 68702 (0.0007) [2023-10-11 21:47:36,034][70582] Fps is (10 sec: 19661.0, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 140771328. Throughput: 0: 1814.1, 1: 1798.6. Samples: 35194360. Policy #0 lag: (min: 15.0, avg: 22.9, max: 47.0) [2023-10-11 21:47:36,034][70582] Avg episode reward: [(0, '75.550'), (1, '100.420')] [2023-10-11 21:47:38,498][71601] Updated weights for policy 0, policy_version 68770 (0.0009) [2023-10-11 21:47:38,876][71601] Updated weights for policy 0, policy_version 68780 (0.0010) [2023-10-11 21:47:39,252][71601] Updated weights for policy 0, policy_version 68790 (0.0008) [2023-10-11 21:47:39,594][71635] Updated weights for policy 1, policy_version 68712 (0.0007) [2023-10-11 21:47:39,625][71601] Updated weights for policy 0, policy_version 68800 (0.0007) [2023-10-11 21:47:39,962][71635] Updated weights for policy 1, policy_version 68722 (0.0007) [2023-10-11 21:47:40,336][71635] Updated weights for policy 1, policy_version 68732 (0.0007) [2023-10-11 21:47:41,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 140836864. Throughput: 0: 1802.0, 1: 1811.1. Samples: 35215144. Policy #0 lag: (min: 15.0, avg: 22.9, max: 47.0) [2023-10-11 21:47:41,035][70582] Avg episode reward: [(0, '78.670'), (1, '95.810')] [2023-10-11 21:47:43,490][71601] Updated weights for policy 0, policy_version 68810 (0.0008) [2023-10-11 21:47:43,873][71601] Updated weights for policy 0, policy_version 68820 (0.0008) [2023-10-11 21:47:44,090][71635] Updated weights for policy 1, policy_version 68742 (0.0008) [2023-10-11 21:47:44,237][71601] Updated weights for policy 0, policy_version 68830 (0.0008) [2023-10-11 21:47:44,452][71635] Updated weights for policy 1, policy_version 68752 (0.0008) [2023-10-11 21:47:44,828][71635] Updated weights for policy 1, policy_version 68762 (0.0007) [2023-10-11 21:47:46,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 140902400. Throughput: 0: 1812.3, 1: 1806.7. Samples: 35227368. Policy #0 lag: (min: 15.0, avg: 22.9, max: 47.0) [2023-10-11 21:47:46,035][70582] Avg episode reward: [(0, '75.320'), (1, '94.190')] [2023-10-11 21:47:47,915][71601] Updated weights for policy 0, policy_version 68840 (0.0008) [2023-10-11 21:47:48,274][71601] Updated weights for policy 0, policy_version 68850 (0.0010) [2023-10-11 21:47:48,612][71635] Updated weights for policy 1, policy_version 68772 (0.0008) [2023-10-11 21:47:48,646][71601] Updated weights for policy 0, policy_version 68860 (0.0008) [2023-10-11 21:47:48,976][71635] Updated weights for policy 1, policy_version 68782 (0.0008) [2023-10-11 21:47:49,345][71635] Updated weights for policy 1, policy_version 68792 (0.0008) [2023-10-11 21:47:51,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 140967936. Throughput: 0: 1807.6, 1: 1817.2. Samples: 35247918. Policy #0 lag: (min: 28.0, avg: 28.9, max: 49.0) [2023-10-11 21:47:51,035][70582] Avg episode reward: [(0, '73.710'), (1, '94.210')] [2023-10-11 21:47:52,352][71601] Updated weights for policy 0, policy_version 68870 (0.0008) [2023-10-11 21:47:52,727][71601] Updated weights for policy 0, policy_version 68880 (0.0010) [2023-10-11 21:47:53,026][71635] Updated weights for policy 1, policy_version 68802 (0.0008) [2023-10-11 21:47:53,094][71601] Updated weights for policy 0, policy_version 68890 (0.0008) [2023-10-11 21:47:53,395][71635] Updated weights for policy 1, policy_version 68812 (0.0008) [2023-10-11 21:47:53,754][71635] Updated weights for policy 1, policy_version 68822 (0.0011) [2023-10-11 21:47:54,122][71635] Updated weights for policy 1, policy_version 68832 (0.0010) [2023-10-11 21:47:56,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 141033472. Throughput: 0: 1810.7, 1: 1810.3. Samples: 35270196. Policy #0 lag: (min: 28.0, avg: 28.9, max: 49.0) [2023-10-11 21:47:56,035][70582] Avg episode reward: [(0, '75.630'), (1, '88.100')] [2023-10-11 21:47:56,047][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000068832_70483968.pth... [2023-10-11 21:47:56,048][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000068896_70549504.pth... [2023-10-11 21:47:56,079][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000067136_68747264.pth [2023-10-11 21:47:56,083][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000067200_68812800.pth [2023-10-11 21:47:56,652][71601] Updated weights for policy 0, policy_version 68900 (0.0008) [2023-10-11 21:47:57,037][71601] Updated weights for policy 0, policy_version 68910 (0.0009) [2023-10-11 21:47:57,410][71601] Updated weights for policy 0, policy_version 68920 (0.0009) [2023-10-11 21:47:57,838][71635] Updated weights for policy 1, policy_version 68842 (0.0008) [2023-10-11 21:47:58,205][71635] Updated weights for policy 1, policy_version 68852 (0.0008) [2023-10-11 21:47:58,566][71635] Updated weights for policy 1, policy_version 68862 (0.0007) [2023-10-11 21:48:01,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 141099008. Throughput: 0: 1814.7, 1: 1818.6. Samples: 35280668. Policy #0 lag: (min: 28.0, avg: 28.9, max: 49.0) [2023-10-11 21:48:01,035][70582] Avg episode reward: [(0, '67.120'), (1, '82.260')] [2023-10-11 21:48:01,198][71601] Updated weights for policy 0, policy_version 68930 (0.0007) [2023-10-11 21:48:01,575][71601] Updated weights for policy 0, policy_version 68940 (0.0008) [2023-10-11 21:48:01,949][71601] Updated weights for policy 0, policy_version 68950 (0.0008) [2023-10-11 21:48:02,256][71635] Updated weights for policy 1, policy_version 68872 (0.0008) [2023-10-11 21:48:02,319][71601] Updated weights for policy 0, policy_version 68960 (0.0009) [2023-10-11 21:48:02,612][71635] Updated weights for policy 1, policy_version 68882 (0.0008) [2023-10-11 21:48:02,980][71635] Updated weights for policy 1, policy_version 68892 (0.0010) [2023-10-11 21:48:06,034][70582] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 141164544. Throughput: 0: 1802.0, 1: 1813.3. Samples: 35302568. Policy #0 lag: (min: 28.0, avg: 28.9, max: 49.0) [2023-10-11 21:48:06,034][70582] Avg episode reward: [(0, '68.790'), (1, '83.890')] [2023-10-11 21:48:06,075][71601] Updated weights for policy 0, policy_version 68970 (0.0010) [2023-10-11 21:48:06,450][71601] Updated weights for policy 0, policy_version 68980 (0.0010) [2023-10-11 21:48:06,825][71601] Updated weights for policy 0, policy_version 68990 (0.0009) [2023-10-11 21:48:06,825][71635] Updated weights for policy 1, policy_version 68902 (0.0008) [2023-10-11 21:48:07,222][71635] Updated weights for policy 1, policy_version 68912 (0.0011) [2023-10-11 21:48:07,589][71635] Updated weights for policy 1, policy_version 68922 (0.0010) [2023-10-11 21:48:10,486][71601] Updated weights for policy 0, policy_version 69000 (0.0008) [2023-10-11 21:48:10,856][71601] Updated weights for policy 0, policy_version 69010 (0.0007) [2023-10-11 21:48:11,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 141230080. Throughput: 0: 1812.3, 1: 1810.9. Samples: 35324932. Policy #0 lag: (min: 28.0, avg: 28.9, max: 49.0) [2023-10-11 21:48:11,034][70582] Avg episode reward: [(0, '65.470'), (1, '83.610')] [2023-10-11 21:48:11,138][71635] Updated weights for policy 1, policy_version 68932 (0.0007) [2023-10-11 21:48:11,225][71601] Updated weights for policy 0, policy_version 69020 (0.0008) [2023-10-11 21:48:11,502][71635] Updated weights for policy 1, policy_version 68942 (0.0007) [2023-10-11 21:48:11,864][71635] Updated weights for policy 1, policy_version 68952 (0.0007) [2023-10-11 21:48:15,037][71601] Updated weights for policy 0, policy_version 69030 (0.0007) [2023-10-11 21:48:15,398][71601] Updated weights for policy 0, policy_version 69040 (0.0008) [2023-10-11 21:48:15,583][71635] Updated weights for policy 1, policy_version 68962 (0.0007) [2023-10-11 21:48:15,767][71601] Updated weights for policy 0, policy_version 69050 (0.0008) [2023-10-11 21:48:15,952][71635] Updated weights for policy 1, policy_version 68972 (0.0008) [2023-10-11 21:48:16,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 141328384. Throughput: 0: 1802.4, 1: 1812.6. Samples: 35335096. Policy #0 lag: (min: 28.0, avg: 28.9, max: 49.0) [2023-10-11 21:48:16,034][70582] Avg episode reward: [(0, '66.340'), (1, '87.460')] [2023-10-11 21:48:16,313][71635] Updated weights for policy 1, policy_version 68982 (0.0008) [2023-10-11 21:48:16,680][71635] Updated weights for policy 1, policy_version 68992 (0.0008) [2023-10-11 21:48:19,508][71601] Updated weights for policy 0, policy_version 69060 (0.0007) [2023-10-11 21:48:19,871][71601] Updated weights for policy 0, policy_version 69070 (0.0010) [2023-10-11 21:48:20,242][71601] Updated weights for policy 0, policy_version 69080 (0.0008) [2023-10-11 21:48:20,352][71635] Updated weights for policy 1, policy_version 69002 (0.0007) [2023-10-11 21:48:20,717][71635] Updated weights for policy 1, policy_version 69012 (0.0007) [2023-10-11 21:48:21,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 141393920. Throughput: 0: 1816.8, 1: 1814.7. Samples: 35357780. Policy #0 lag: (min: 28.0, avg: 28.9, max: 49.0) [2023-10-11 21:48:21,034][70582] Avg episode reward: [(0, '65.190'), (1, '87.350')] [2023-10-11 21:48:21,084][71635] Updated weights for policy 1, policy_version 69022 (0.0010) [2023-10-11 21:48:23,911][71601] Updated weights for policy 0, policy_version 69090 (0.0010) [2023-10-11 21:48:24,281][71601] Updated weights for policy 0, policy_version 69100 (0.0010) [2023-10-11 21:48:24,658][71601] Updated weights for policy 0, policy_version 69110 (0.0007) [2023-10-11 21:48:24,879][71635] Updated weights for policy 1, policy_version 69032 (0.0008) [2023-10-11 21:48:25,023][71601] Updated weights for policy 0, policy_version 69120 (0.0009) [2023-10-11 21:48:25,243][71635] Updated weights for policy 1, policy_version 69042 (0.0008) [2023-10-11 21:48:25,606][71635] Updated weights for policy 1, policy_version 69052 (0.0007) [2023-10-11 21:48:26,034][70582] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 141492224. Throughput: 0: 1805.8, 1: 1821.0. Samples: 35378350. Policy #0 lag: (min: 28.0, avg: 28.9, max: 49.0) [2023-10-11 21:48:26,035][70582] Avg episode reward: [(0, '66.280'), (1, '86.830')] [2023-10-11 21:48:28,906][71601] Updated weights for policy 0, policy_version 69130 (0.0010) [2023-10-11 21:48:29,139][71635] Updated weights for policy 1, policy_version 69062 (0.0009) [2023-10-11 21:48:29,284][71601] Updated weights for policy 0, policy_version 69140 (0.0008) [2023-10-11 21:48:29,497][71635] Updated weights for policy 1, policy_version 69072 (0.0009) [2023-10-11 21:48:29,657][71601] Updated weights for policy 0, policy_version 69150 (0.0007) [2023-10-11 21:48:29,861][71635] Updated weights for policy 1, policy_version 69082 (0.0007) [2023-10-11 21:48:31,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 141557760. Throughput: 0: 1817.0, 1: 1815.2. Samples: 35390816. Policy #0 lag: (min: 28.0, avg: 28.9, max: 49.0) [2023-10-11 21:48:31,035][70582] Avg episode reward: [(0, '68.600'), (1, '87.020')] [2023-10-11 21:48:33,288][71601] Updated weights for policy 0, policy_version 69160 (0.0009) [2023-10-11 21:48:33,654][71601] Updated weights for policy 0, policy_version 69170 (0.0011) [2023-10-11 21:48:33,740][71635] Updated weights for policy 1, policy_version 69092 (0.0009) [2023-10-11 21:48:34,024][71601] Updated weights for policy 0, policy_version 69180 (0.0008) [2023-10-11 21:48:34,103][71635] Updated weights for policy 1, policy_version 69102 (0.0009) [2023-10-11 21:48:34,480][71635] Updated weights for policy 1, policy_version 69112 (0.0008) [2023-10-11 21:48:36,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 141623296. Throughput: 0: 1801.7, 1: 1815.4. Samples: 35410686. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-11 21:48:36,035][70582] Avg episode reward: [(0, '69.900'), (1, '85.040')] [2023-10-11 21:48:37,813][71601] Updated weights for policy 0, policy_version 69190 (0.0007) [2023-10-11 21:48:38,178][71601] Updated weights for policy 0, policy_version 69200 (0.0008) [2023-10-11 21:48:38,208][71635] Updated weights for policy 1, policy_version 69122 (0.0008) [2023-10-11 21:48:38,554][71601] Updated weights for policy 0, policy_version 69210 (0.0008) [2023-10-11 21:48:38,570][71635] Updated weights for policy 1, policy_version 69132 (0.0008) [2023-10-11 21:48:38,933][71635] Updated weights for policy 1, policy_version 69142 (0.0008) [2023-10-11 21:48:39,302][71635] Updated weights for policy 1, policy_version 69152 (0.0011) [2023-10-11 21:48:41,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 141688832. Throughput: 0: 1805.3, 1: 1810.2. Samples: 35432896. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-11 21:48:41,034][70582] Avg episode reward: [(0, '72.230'), (1, '87.160')] [2023-10-11 21:48:42,252][71601] Updated weights for policy 0, policy_version 69220 (0.0008) [2023-10-11 21:48:42,620][71601] Updated weights for policy 0, policy_version 69230 (0.0008) [2023-10-11 21:48:42,980][71635] Updated weights for policy 1, policy_version 69162 (0.0007) [2023-10-11 21:48:42,995][71601] Updated weights for policy 0, policy_version 69240 (0.0008) [2023-10-11 21:48:43,350][71635] Updated weights for policy 1, policy_version 69172 (0.0009) [2023-10-11 21:48:43,721][71635] Updated weights for policy 1, policy_version 69182 (0.0011) [2023-10-11 21:48:46,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 141754368. Throughput: 0: 1796.4, 1: 1816.7. Samples: 35443260. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-11 21:48:46,035][70582] Avg episode reward: [(0, '73.580'), (1, '86.520')] [2023-10-11 21:48:46,717][71601] Updated weights for policy 0, policy_version 69250 (0.0009) [2023-10-11 21:48:47,096][71601] Updated weights for policy 0, policy_version 69260 (0.0009) [2023-10-11 21:48:47,450][71635] Updated weights for policy 1, policy_version 69192 (0.0008) [2023-10-11 21:48:47,464][71601] Updated weights for policy 0, policy_version 69270 (0.0009) [2023-10-11 21:48:47,818][71635] Updated weights for policy 1, policy_version 69202 (0.0007) [2023-10-11 21:48:47,836][71601] Updated weights for policy 0, policy_version 69280 (0.0008) [2023-10-11 21:48:48,185][71635] Updated weights for policy 1, policy_version 69212 (0.0010) [2023-10-11 21:48:51,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 141819904. Throughput: 0: 1803.2, 1: 1817.4. Samples: 35465496. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-11 21:48:51,034][70582] Avg episode reward: [(0, '74.590'), (1, '87.580')] [2023-10-11 21:48:51,467][71601] Updated weights for policy 0, policy_version 69290 (0.0009) [2023-10-11 21:48:51,766][71635] Updated weights for policy 1, policy_version 69222 (0.0007) [2023-10-11 21:48:51,844][71601] Updated weights for policy 0, policy_version 69300 (0.0009) [2023-10-11 21:48:52,138][71635] Updated weights for policy 1, policy_version 69232 (0.0007) [2023-10-11 21:48:52,205][71601] Updated weights for policy 0, policy_version 69310 (0.0008) [2023-10-11 21:48:52,495][71635] Updated weights for policy 1, policy_version 69242 (0.0009) [2023-10-11 21:48:56,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.6, 300 sec: 14440.1). Total num frames: 141885440. Throughput: 0: 1803.8, 1: 1823.2. Samples: 35488148. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-11 21:48:56,034][70582] Avg episode reward: [(0, '74.290'), (1, '89.840')] [2023-10-11 21:48:56,102][71601] Updated weights for policy 0, policy_version 69320 (0.0008) [2023-10-11 21:48:56,155][71635] Updated weights for policy 1, policy_version 69252 (0.0009) [2023-10-11 21:48:56,485][71601] Updated weights for policy 0, policy_version 69330 (0.0007) [2023-10-11 21:48:56,526][71635] Updated weights for policy 1, policy_version 69262 (0.0009) [2023-10-11 21:48:56,847][71601] Updated weights for policy 0, policy_version 69340 (0.0008) [2023-10-11 21:48:56,888][71635] Updated weights for policy 1, policy_version 69272 (0.0009) [2023-10-11 21:49:00,541][71635] Updated weights for policy 1, policy_version 69282 (0.0009) [2023-10-11 21:49:00,645][71601] Updated weights for policy 0, policy_version 69350 (0.0008) [2023-10-11 21:49:00,913][71635] Updated weights for policy 1, policy_version 69292 (0.0008) [2023-10-11 21:49:01,017][71601] Updated weights for policy 0, policy_version 69360 (0.0007) [2023-10-11 21:49:01,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 141950976. Throughput: 0: 1793.7, 1: 1823.3. Samples: 35497864. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-11 21:49:01,034][70582] Avg episode reward: [(0, '72.460'), (1, '87.030')] [2023-10-11 21:49:01,272][71635] Updated weights for policy 1, policy_version 69302 (0.0008) [2023-10-11 21:49:01,378][71601] Updated weights for policy 0, policy_version 69370 (0.0008) [2023-10-11 21:49:01,638][71635] Updated weights for policy 1, policy_version 69312 (0.0008) [2023-10-11 21:49:05,053][71601] Updated weights for policy 0, policy_version 69380 (0.0008) [2023-10-11 21:49:05,210][71635] Updated weights for policy 1, policy_version 69322 (0.0008) [2023-10-11 21:49:05,419][71601] Updated weights for policy 0, policy_version 69390 (0.0007) [2023-10-11 21:49:05,576][71635] Updated weights for policy 1, policy_version 69332 (0.0008) [2023-10-11 21:49:05,785][71601] Updated weights for policy 0, policy_version 69400 (0.0007) [2023-10-11 21:49:05,940][71635] Updated weights for policy 1, policy_version 69342 (0.0009) [2023-10-11 21:49:06,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 142049280. Throughput: 0: 1794.1, 1: 1826.8. Samples: 35520724. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-11 21:49:06,035][70582] Avg episode reward: [(0, '67.750'), (1, '85.870')] [2023-10-11 21:49:09,535][71635] Updated weights for policy 1, policy_version 69352 (0.0008) [2023-10-11 21:49:09,637][71601] Updated weights for policy 0, policy_version 69410 (0.0008) [2023-10-11 21:49:09,890][71635] Updated weights for policy 1, policy_version 69362 (0.0010) [2023-10-11 21:49:10,005][71601] Updated weights for policy 0, policy_version 69420 (0.0010) [2023-10-11 21:49:10,252][71635] Updated weights for policy 1, policy_version 69372 (0.0007) [2023-10-11 21:49:10,363][71601] Updated weights for policy 0, policy_version 69430 (0.0008) [2023-10-11 21:49:10,739][71601] Updated weights for policy 0, policy_version 69440 (0.0007) [2023-10-11 21:49:11,034][70582] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 142147584. Throughput: 0: 1798.3, 1: 1818.2. Samples: 35541090. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-11 21:49:11,034][70582] Avg episode reward: [(0, '70.110'), (1, '86.170')] [2023-10-11 21:49:13,991][71635] Updated weights for policy 1, policy_version 69382 (0.0010) [2023-10-11 21:49:14,356][71635] Updated weights for policy 1, policy_version 69392 (0.0009) [2023-10-11 21:49:14,591][71601] Updated weights for policy 0, policy_version 69450 (0.0007) [2023-10-11 21:49:14,714][71635] Updated weights for policy 1, policy_version 69402 (0.0008) [2023-10-11 21:49:14,962][71601] Updated weights for policy 0, policy_version 69460 (0.0007) [2023-10-11 21:49:15,343][71601] Updated weights for policy 0, policy_version 69470 (0.0007) [2023-10-11 21:49:16,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 142213120. Throughput: 0: 1788.3, 1: 1819.5. Samples: 35553166. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-11 21:49:16,034][70582] Avg episode reward: [(0, '69.110'), (1, '91.510')] [2023-10-11 21:49:18,428][71635] Updated weights for policy 1, policy_version 69412 (0.0009) [2023-10-11 21:49:18,794][71635] Updated weights for policy 1, policy_version 69422 (0.0011) [2023-10-11 21:49:19,040][71601] Updated weights for policy 0, policy_version 69480 (0.0008) [2023-10-11 21:49:19,154][71635] Updated weights for policy 1, policy_version 69432 (0.0007) [2023-10-11 21:49:19,418][71601] Updated weights for policy 0, policy_version 69490 (0.0008) [2023-10-11 21:49:19,790][71601] Updated weights for policy 0, policy_version 69500 (0.0010) [2023-10-11 21:49:21,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 142278656. Throughput: 0: 1801.5, 1: 1813.8. Samples: 35573376. Policy #0 lag: (min: 31.0, avg: 32.1, max: 53.0) [2023-10-11 21:49:21,034][70582] Avg episode reward: [(0, '66.060'), (1, '89.110')] [2023-10-11 21:49:22,865][71635] Updated weights for policy 1, policy_version 69442 (0.0008) [2023-10-11 21:49:23,241][71635] Updated weights for policy 1, policy_version 69452 (0.0008) [2023-10-11 21:49:23,499][71601] Updated weights for policy 0, policy_version 69510 (0.0008) [2023-10-11 21:49:23,603][71635] Updated weights for policy 1, policy_version 69462 (0.0008) [2023-10-11 21:49:23,858][71601] Updated weights for policy 0, policy_version 69520 (0.0007) [2023-10-11 21:49:23,977][71635] Updated weights for policy 1, policy_version 69472 (0.0008) [2023-10-11 21:49:24,226][71601] Updated weights for policy 0, policy_version 69530 (0.0008) [2023-10-11 21:49:26,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 142344192. Throughput: 0: 1782.6, 1: 1828.6. Samples: 35595398. Policy #0 lag: (min: 31.0, avg: 32.1, max: 53.0) [2023-10-11 21:49:26,034][70582] Avg episode reward: [(0, '67.720'), (1, '86.650')] [2023-10-11 21:49:27,566][71635] Updated weights for policy 1, policy_version 69482 (0.0011) [2023-10-11 21:49:27,926][71635] Updated weights for policy 1, policy_version 69492 (0.0010) [2023-10-11 21:49:27,984][71601] Updated weights for policy 0, policy_version 69540 (0.0007) [2023-10-11 21:49:28,288][71635] Updated weights for policy 1, policy_version 69502 (0.0009) [2023-10-11 21:49:28,355][71601] Updated weights for policy 0, policy_version 69550 (0.0007) [2023-10-11 21:49:28,740][71601] Updated weights for policy 0, policy_version 69560 (0.0007) [2023-10-11 21:49:31,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 142409728. Throughput: 0: 1801.4, 1: 1814.6. Samples: 35605982. Policy #0 lag: (min: 31.0, avg: 32.1, max: 53.0) [2023-10-11 21:49:31,034][70582] Avg episode reward: [(0, '66.520'), (1, '88.380')] [2023-10-11 21:49:32,161][71635] Updated weights for policy 1, policy_version 69512 (0.0009) [2023-10-11 21:49:32,537][71635] Updated weights for policy 1, policy_version 69522 (0.0009) [2023-10-11 21:49:32,580][71601] Updated weights for policy 0, policy_version 69570 (0.0009) [2023-10-11 21:49:32,902][71635] Updated weights for policy 1, policy_version 69532 (0.0008) [2023-10-11 21:49:32,947][71601] Updated weights for policy 0, policy_version 69580 (0.0008) [2023-10-11 21:49:33,322][71601] Updated weights for policy 0, policy_version 69590 (0.0008) [2023-10-11 21:49:33,691][71601] Updated weights for policy 0, policy_version 69600 (0.0008) [2023-10-11 21:49:36,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 142475264. Throughput: 0: 1779.9, 1: 1817.8. Samples: 35627392. Policy #0 lag: (min: 31.0, avg: 32.1, max: 53.0) [2023-10-11 21:49:36,035][70582] Avg episode reward: [(0, '63.950'), (1, '88.560')] [2023-10-11 21:49:36,758][71635] Updated weights for policy 1, policy_version 69542 (0.0008) [2023-10-11 21:49:37,128][71635] Updated weights for policy 1, policy_version 69552 (0.0008) [2023-10-11 21:49:37,374][71601] Updated weights for policy 0, policy_version 69610 (0.0009) [2023-10-11 21:49:37,492][71635] Updated weights for policy 1, policy_version 69562 (0.0008) [2023-10-11 21:49:37,742][71601] Updated weights for policy 0, policy_version 69620 (0.0008) [2023-10-11 21:49:38,124][71601] Updated weights for policy 0, policy_version 69630 (0.0007) [2023-10-11 21:49:41,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 142540800. Throughput: 0: 1785.1, 1: 1814.1. Samples: 35650114. Policy #0 lag: (min: 31.0, avg: 32.1, max: 53.0) [2023-10-11 21:49:41,035][70582] Avg episode reward: [(0, '61.540'), (1, '88.210')] [2023-10-11 21:49:41,040][71635] Updated weights for policy 1, policy_version 69572 (0.0009) [2023-10-11 21:49:41,405][71635] Updated weights for policy 1, policy_version 69582 (0.0010) [2023-10-11 21:49:41,769][71635] Updated weights for policy 1, policy_version 69592 (0.0008) [2023-10-11 21:49:41,821][71601] Updated weights for policy 0, policy_version 69640 (0.0007) [2023-10-11 21:49:42,193][71601] Updated weights for policy 0, policy_version 69650 (0.0008) [2023-10-11 21:49:42,564][71601] Updated weights for policy 0, policy_version 69660 (0.0008) [2023-10-11 21:49:45,625][71635] Updated weights for policy 1, policy_version 69602 (0.0007) [2023-10-11 21:49:45,999][71635] Updated weights for policy 1, policy_version 69612 (0.0008) [2023-10-11 21:49:46,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 142606336. Throughput: 0: 1790.4, 1: 1810.9. Samples: 35659924. Policy #0 lag: (min: 31.0, avg: 32.1, max: 53.0) [2023-10-11 21:49:46,035][70582] Avg episode reward: [(0, '65.320'), (1, '87.770')] [2023-10-11 21:49:46,275][71601] Updated weights for policy 0, policy_version 69670 (0.0008) [2023-10-11 21:49:46,361][71635] Updated weights for policy 1, policy_version 69622 (0.0009) [2023-10-11 21:49:46,645][71601] Updated weights for policy 0, policy_version 69680 (0.0008) [2023-10-11 21:49:46,729][71635] Updated weights for policy 1, policy_version 69632 (0.0008) [2023-10-11 21:49:47,023][71601] Updated weights for policy 0, policy_version 69690 (0.0008) [2023-10-11 21:49:50,593][71635] Updated weights for policy 1, policy_version 69642 (0.0007) [2023-10-11 21:49:50,810][71601] Updated weights for policy 0, policy_version 69700 (0.0010) [2023-10-11 21:49:50,959][71635] Updated weights for policy 1, policy_version 69652 (0.0007) [2023-10-11 21:49:51,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 142671872. Throughput: 0: 1785.3, 1: 1799.4. Samples: 35682032. Policy #0 lag: (min: 31.0, avg: 32.1, max: 53.0) [2023-10-11 21:49:51,034][70582] Avg episode reward: [(0, '66.270'), (1, '83.950')] [2023-10-11 21:49:51,179][71601] Updated weights for policy 0, policy_version 69710 (0.0007) [2023-10-11 21:49:51,321][71635] Updated weights for policy 1, policy_version 69662 (0.0007) [2023-10-11 21:49:51,553][71601] Updated weights for policy 0, policy_version 69720 (0.0007) [2023-10-11 21:49:55,117][71601] Updated weights for policy 0, policy_version 69730 (0.0007) [2023-10-11 21:49:55,175][71635] Updated weights for policy 1, policy_version 69672 (0.0008) [2023-10-11 21:49:55,495][71601] Updated weights for policy 0, policy_version 69740 (0.0007) [2023-10-11 21:49:55,530][71635] Updated weights for policy 1, policy_version 69682 (0.0008) [2023-10-11 21:49:55,859][71601] Updated weights for policy 0, policy_version 69750 (0.0007) [2023-10-11 21:49:55,896][71635] Updated weights for policy 1, policy_version 69692 (0.0007) [2023-10-11 21:49:56,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 142737408. Throughput: 0: 1803.5, 1: 1807.8. Samples: 35703600. Policy #0 lag: (min: 31.0, avg: 32.1, max: 53.0) [2023-10-11 21:49:56,034][70582] Avg episode reward: [(0, '67.430'), (1, '89.890')] [2023-10-11 21:49:56,043][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000069696_71368704.pth... [2023-10-11 21:49:56,082][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000068000_69632000.pth [2023-10-11 21:49:56,230][71601] Updated weights for policy 0, policy_version 69760 (0.0009) [2023-10-11 21:49:56,231][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000069760_71434240.pth... [2023-10-11 21:49:56,268][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000068064_69697536.pth [2023-10-11 21:49:59,613][71635] Updated weights for policy 1, policy_version 69702 (0.0007) [2023-10-11 21:49:59,980][71635] Updated weights for policy 1, policy_version 69712 (0.0009) [2023-10-11 21:50:00,028][71601] Updated weights for policy 0, policy_version 69770 (0.0008) [2023-10-11 21:50:00,353][71635] Updated weights for policy 1, policy_version 69722 (0.0008) [2023-10-11 21:50:00,399][71601] Updated weights for policy 0, policy_version 69780 (0.0010) [2023-10-11 21:50:00,784][71601] Updated weights for policy 0, policy_version 69790 (0.0009) [2023-10-11 21:50:01,034][70582] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 142868480. Throughput: 0: 1789.6, 1: 1795.4. Samples: 35714492. Policy #0 lag: (min: 31.0, avg: 32.1, max: 53.0) [2023-10-11 21:50:01,034][70582] Avg episode reward: [(0, '65.020'), (1, '95.570')] [2023-10-11 21:50:03,920][71635] Updated weights for policy 1, policy_version 69732 (0.0009) [2023-10-11 21:50:04,290][71635] Updated weights for policy 1, policy_version 69742 (0.0009) [2023-10-11 21:50:04,488][71601] Updated weights for policy 0, policy_version 69800 (0.0010) [2023-10-11 21:50:04,654][71635] Updated weights for policy 1, policy_version 69752 (0.0008) [2023-10-11 21:50:04,861][71601] Updated weights for policy 0, policy_version 69810 (0.0007) [2023-10-11 21:50:05,228][71601] Updated weights for policy 0, policy_version 69820 (0.0007) [2023-10-11 21:50:06,034][70582] Fps is (10 sec: 19661.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 142934016. Throughput: 0: 1805.8, 1: 1812.4. Samples: 35736194. Policy #0 lag: (min: 24.0, avg: 44.3, max: 56.0) [2023-10-11 21:50:06,034][70582] Avg episode reward: [(0, '67.860'), (1, '91.930')] [2023-10-11 21:50:08,338][71635] Updated weights for policy 1, policy_version 69762 (0.0008) [2023-10-11 21:50:08,700][71635] Updated weights for policy 1, policy_version 69772 (0.0008) [2023-10-11 21:50:08,884][71601] Updated weights for policy 0, policy_version 69830 (0.0009) [2023-10-11 21:50:09,069][71635] Updated weights for policy 1, policy_version 69782 (0.0008) [2023-10-11 21:50:09,257][71601] Updated weights for policy 0, policy_version 69840 (0.0007) [2023-10-11 21:50:09,431][71635] Updated weights for policy 1, policy_version 69792 (0.0008) [2023-10-11 21:50:09,623][71601] Updated weights for policy 0, policy_version 69850 (0.0010) [2023-10-11 21:50:11,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 142999552. Throughput: 0: 1797.9, 1: 1798.2. Samples: 35757220. Policy #0 lag: (min: 24.0, avg: 44.3, max: 56.0) [2023-10-11 21:50:11,034][70582] Avg episode reward: [(0, '66.950'), (1, '94.060')] [2023-10-11 21:50:13,100][71635] Updated weights for policy 1, policy_version 69802 (0.0008) [2023-10-11 21:50:13,290][71601] Updated weights for policy 0, policy_version 69860 (0.0008) [2023-10-11 21:50:13,458][71635] Updated weights for policy 1, policy_version 69812 (0.0007) [2023-10-11 21:50:13,657][71601] Updated weights for policy 0, policy_version 69870 (0.0008) [2023-10-11 21:50:13,822][71635] Updated weights for policy 1, policy_version 69822 (0.0007) [2023-10-11 21:50:14,037][71601] Updated weights for policy 0, policy_version 69880 (0.0008) [2023-10-11 21:50:16,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 143065088. Throughput: 0: 1807.6, 1: 1815.8. Samples: 35769034. Policy #0 lag: (min: 24.0, avg: 44.3, max: 56.0) [2023-10-11 21:50:16,034][70582] Avg episode reward: [(0, '66.240'), (1, '103.840')] [2023-10-11 21:50:17,487][71635] Updated weights for policy 1, policy_version 69832 (0.0009) [2023-10-11 21:50:17,809][71601] Updated weights for policy 0, policy_version 69890 (0.0009) [2023-10-11 21:50:17,861][71635] Updated weights for policy 1, policy_version 69842 (0.0010) [2023-10-11 21:50:18,183][71601] Updated weights for policy 0, policy_version 69900 (0.0009) [2023-10-11 21:50:18,218][71635] Updated weights for policy 1, policy_version 69852 (0.0007) [2023-10-11 21:50:18,557][71601] Updated weights for policy 0, policy_version 69910 (0.0009) [2023-10-11 21:50:18,925][71601] Updated weights for policy 0, policy_version 69920 (0.0011) [2023-10-11 21:50:21,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 143130624. Throughput: 0: 1805.8, 1: 1808.9. Samples: 35790052. Policy #0 lag: (min: 24.0, avg: 44.3, max: 56.0) [2023-10-11 21:50:21,034][70582] Avg episode reward: [(0, '69.120'), (1, '103.710')] [2023-10-11 21:50:21,977][71635] Updated weights for policy 1, policy_version 69862 (0.0007) [2023-10-11 21:50:22,372][71635] Updated weights for policy 1, policy_version 69872 (0.0008) [2023-10-11 21:50:22,534][71601] Updated weights for policy 0, policy_version 69930 (0.0008) [2023-10-11 21:50:22,735][71635] Updated weights for policy 1, policy_version 69882 (0.0008) [2023-10-11 21:50:22,905][71601] Updated weights for policy 0, policy_version 69940 (0.0007) [2023-10-11 21:50:23,270][71601] Updated weights for policy 0, policy_version 69950 (0.0007) [2023-10-11 21:50:26,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 143196160. Throughput: 0: 1805.8, 1: 1804.1. Samples: 35812560. Policy #0 lag: (min: 24.0, avg: 44.3, max: 56.0) [2023-10-11 21:50:26,035][70582] Avg episode reward: [(0, '69.280'), (1, '102.340')] [2023-10-11 21:50:26,469][71635] Updated weights for policy 1, policy_version 69892 (0.0008) [2023-10-11 21:50:26,840][71635] Updated weights for policy 1, policy_version 69902 (0.0009) [2023-10-11 21:50:27,097][71601] Updated weights for policy 0, policy_version 69960 (0.0009) [2023-10-11 21:50:27,207][71635] Updated weights for policy 1, policy_version 69912 (0.0007) [2023-10-11 21:50:27,468][71601] Updated weights for policy 0, policy_version 69970 (0.0008) [2023-10-11 21:50:27,830][71601] Updated weights for policy 0, policy_version 69980 (0.0008) [2023-10-11 21:50:31,034][70582] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 143261696. Throughput: 0: 1802.7, 1: 1806.1. Samples: 35822322. Policy #0 lag: (min: 24.0, avg: 44.3, max: 56.0) [2023-10-11 21:50:31,035][70582] Avg episode reward: [(0, '69.130'), (1, '103.020')] [2023-10-11 21:50:31,043][71635] Updated weights for policy 1, policy_version 69922 (0.0007) [2023-10-11 21:50:31,416][71635] Updated weights for policy 1, policy_version 69932 (0.0009) [2023-10-11 21:50:31,598][71601] Updated weights for policy 0, policy_version 69990 (0.0009) [2023-10-11 21:50:31,786][71635] Updated weights for policy 1, policy_version 69942 (0.0007) [2023-10-11 21:50:31,977][71601] Updated weights for policy 0, policy_version 70000 (0.0008) [2023-10-11 21:50:32,154][71635] Updated weights for policy 1, policy_version 69952 (0.0008) [2023-10-11 21:50:32,355][71601] Updated weights for policy 0, policy_version 70010 (0.0009) [2023-10-11 21:50:35,856][71635] Updated weights for policy 1, policy_version 69962 (0.0007) [2023-10-11 21:50:36,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 143327232. Throughput: 0: 1803.7, 1: 1809.8. Samples: 35844638. Policy #0 lag: (min: 24.0, avg: 44.3, max: 56.0) [2023-10-11 21:50:36,034][70582] Avg episode reward: [(0, '70.660'), (1, '97.680')] [2023-10-11 21:50:36,073][71601] Updated weights for policy 0, policy_version 70020 (0.0008) [2023-10-11 21:50:36,219][71635] Updated weights for policy 1, policy_version 69972 (0.0008) [2023-10-11 21:50:36,441][71601] Updated weights for policy 0, policy_version 70030 (0.0008) [2023-10-11 21:50:36,593][71635] Updated weights for policy 1, policy_version 69982 (0.0010) [2023-10-11 21:50:36,818][71601] Updated weights for policy 0, policy_version 70040 (0.0007) [2023-10-11 21:50:40,284][71635] Updated weights for policy 1, policy_version 69992 (0.0008) [2023-10-11 21:50:40,550][71601] Updated weights for policy 0, policy_version 70050 (0.0007) [2023-10-11 21:50:40,644][71635] Updated weights for policy 1, policy_version 70002 (0.0007) [2023-10-11 21:50:40,922][71601] Updated weights for policy 0, policy_version 70060 (0.0007) [2023-10-11 21:50:41,010][71635] Updated weights for policy 1, policy_version 70012 (0.0010) [2023-10-11 21:50:41,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 143392768. Throughput: 0: 1811.3, 1: 1820.6. Samples: 35867038. Policy #0 lag: (min: 24.0, avg: 44.3, max: 56.0) [2023-10-11 21:50:41,034][70582] Avg episode reward: [(0, '73.450'), (1, '97.670')] [2023-10-11 21:50:41,296][71601] Updated weights for policy 0, policy_version 70070 (0.0008) [2023-10-11 21:50:41,676][71601] Updated weights for policy 0, policy_version 70080 (0.0007) [2023-10-11 21:50:44,707][71635] Updated weights for policy 1, policy_version 70022 (0.0011) [2023-10-11 21:50:45,074][71635] Updated weights for policy 1, policy_version 70032 (0.0007) [2023-10-11 21:50:45,362][71601] Updated weights for policy 0, policy_version 70090 (0.0008) [2023-10-11 21:50:45,442][71635] Updated weights for policy 1, policy_version 70042 (0.0008) [2023-10-11 21:50:45,747][71601] Updated weights for policy 0, policy_version 70100 (0.0008) [2023-10-11 21:50:46,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 143491072. Throughput: 0: 1805.2, 1: 1815.6. Samples: 35877432. Policy #0 lag: (min: 24.0, avg: 44.3, max: 56.0) [2023-10-11 21:50:46,034][70582] Avg episode reward: [(0, '75.010'), (1, '91.100')] [2023-10-11 21:50:46,117][71601] Updated weights for policy 0, policy_version 70110 (0.0007) [2023-10-11 21:50:49,174][71635] Updated weights for policy 1, policy_version 70052 (0.0008) [2023-10-11 21:50:49,537][71635] Updated weights for policy 1, policy_version 70062 (0.0008) [2023-10-11 21:50:49,814][71601] Updated weights for policy 0, policy_version 70120 (0.0008) [2023-10-11 21:50:49,904][71635] Updated weights for policy 1, policy_version 70072 (0.0007) [2023-10-11 21:50:50,185][71601] Updated weights for policy 0, policy_version 70130 (0.0008) [2023-10-11 21:50:50,565][71601] Updated weights for policy 0, policy_version 70140 (0.0010) [2023-10-11 21:50:51,034][70582] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 143589376. Throughput: 0: 1809.1, 1: 1820.3. Samples: 35899516. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-11 21:50:51,035][70582] Avg episode reward: [(0, '71.110'), (1, '94.190')] [2023-10-11 21:50:53,549][71635] Updated weights for policy 1, policy_version 70082 (0.0008) [2023-10-11 21:50:53,914][71635] Updated weights for policy 1, policy_version 70092 (0.0007) [2023-10-11 21:50:54,228][71601] Updated weights for policy 0, policy_version 70150 (0.0007) [2023-10-11 21:50:54,273][71635] Updated weights for policy 1, policy_version 70102 (0.0008) [2023-10-11 21:50:54,601][71601] Updated weights for policy 0, policy_version 70160 (0.0008) [2023-10-11 21:50:54,639][71635] Updated weights for policy 1, policy_version 70112 (0.0007) [2023-10-11 21:50:54,963][71601] Updated weights for policy 0, policy_version 70170 (0.0008) [2023-10-11 21:50:56,034][70582] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 143654912. Throughput: 0: 1800.3, 1: 1813.3. Samples: 35919830. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-11 21:50:56,035][70582] Avg episode reward: [(0, '73.400'), (1, '98.270')] [2023-10-11 21:50:58,540][71635] Updated weights for policy 1, policy_version 70122 (0.0008) [2023-10-11 21:50:58,586][71601] Updated weights for policy 0, policy_version 70180 (0.0007) [2023-10-11 21:50:58,914][71635] Updated weights for policy 1, policy_version 70132 (0.0007) [2023-10-11 21:50:58,955][71601] Updated weights for policy 0, policy_version 70190 (0.0008) [2023-10-11 21:50:59,273][71635] Updated weights for policy 1, policy_version 70142 (0.0008) [2023-10-11 21:50:59,335][71601] Updated weights for policy 0, policy_version 70200 (0.0007) [2023-10-11 21:51:01,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 143720448. Throughput: 0: 1812.8, 1: 1816.6. Samples: 35932356. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-11 21:51:01,035][70582] Avg episode reward: [(0, '72.590'), (1, '100.540')] [2023-10-11 21:51:03,073][71601] Updated weights for policy 0, policy_version 70210 (0.0008) [2023-10-11 21:51:03,075][71635] Updated weights for policy 1, policy_version 70152 (0.0007) [2023-10-11 21:51:03,433][71601] Updated weights for policy 0, policy_version 70220 (0.0007) [2023-10-11 21:51:03,442][71635] Updated weights for policy 1, policy_version 70162 (0.0007) [2023-10-11 21:51:03,808][71635] Updated weights for policy 1, policy_version 70172 (0.0007) [2023-10-11 21:51:03,809][71601] Updated weights for policy 0, policy_version 70230 (0.0008) [2023-10-11 21:51:04,179][71601] Updated weights for policy 0, policy_version 70240 (0.0007) [2023-10-11 21:51:06,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 143785984. Throughput: 0: 1801.4, 1: 1798.7. Samples: 35952056. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-11 21:51:06,034][70582] Avg episode reward: [(0, '70.060'), (1, '99.490')] [2023-10-11 21:51:07,471][71635] Updated weights for policy 1, policy_version 70182 (0.0008) [2023-10-11 21:51:07,809][71601] Updated weights for policy 0, policy_version 70250 (0.0008) [2023-10-11 21:51:07,844][71635] Updated weights for policy 1, policy_version 70192 (0.0007) [2023-10-11 21:51:08,186][71601] Updated weights for policy 0, policy_version 70260 (0.0009) [2023-10-11 21:51:08,216][71635] Updated weights for policy 1, policy_version 70202 (0.0008) [2023-10-11 21:51:08,552][71601] Updated weights for policy 0, policy_version 70270 (0.0009) [2023-10-11 21:51:11,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 143851520. Throughput: 0: 1809.5, 1: 1803.3. Samples: 35975134. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-11 21:51:11,035][70582] Avg episode reward: [(0, '68.320'), (1, '98.090')] [2023-10-11 21:51:11,875][71635] Updated weights for policy 1, policy_version 70212 (0.0008) [2023-10-11 21:51:12,217][71601] Updated weights for policy 0, policy_version 70280 (0.0007) [2023-10-11 21:51:12,237][71635] Updated weights for policy 1, policy_version 70222 (0.0007) [2023-10-11 21:51:12,583][71601] Updated weights for policy 0, policy_version 70290 (0.0009) [2023-10-11 21:51:12,602][71635] Updated weights for policy 1, policy_version 70232 (0.0008) [2023-10-11 21:51:12,960][71601] Updated weights for policy 0, policy_version 70300 (0.0009) [2023-10-11 21:51:16,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 143917056. Throughput: 0: 1808.9, 1: 1802.9. Samples: 35984854. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-11 21:51:16,034][70582] Avg episode reward: [(0, '64.400'), (1, '103.010')] [2023-10-11 21:51:16,278][71635] Updated weights for policy 1, policy_version 70242 (0.0008) [2023-10-11 21:51:16,650][71635] Updated weights for policy 1, policy_version 70252 (0.0007) [2023-10-11 21:51:16,807][71601] Updated weights for policy 0, policy_version 70310 (0.0008) [2023-10-11 21:51:17,015][71635] Updated weights for policy 1, policy_version 70262 (0.0008) [2023-10-11 21:51:17,174][71601] Updated weights for policy 0, policy_version 70320 (0.0009) [2023-10-11 21:51:17,386][71635] Updated weights for policy 1, policy_version 70272 (0.0007) [2023-10-11 21:51:17,554][71601] Updated weights for policy 0, policy_version 70330 (0.0010) [2023-10-11 21:51:21,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 143982592. Throughput: 0: 1808.1, 1: 1807.5. Samples: 36007342. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-11 21:51:21,035][70582] Avg episode reward: [(0, '62.320'), (1, '105.160')] [2023-10-11 21:51:21,240][71635] Updated weights for policy 1, policy_version 70282 (0.0008) [2023-10-11 21:51:21,301][71601] Updated weights for policy 0, policy_version 70340 (0.0008) [2023-10-11 21:51:21,607][71635] Updated weights for policy 1, policy_version 70292 (0.0007) [2023-10-11 21:51:21,672][71601] Updated weights for policy 0, policy_version 70350 (0.0008) [2023-10-11 21:51:21,980][71635] Updated weights for policy 1, policy_version 70302 (0.0008) [2023-10-11 21:51:22,041][71601] Updated weights for policy 0, policy_version 70360 (0.0008) [2023-10-11 21:51:25,571][71635] Updated weights for policy 1, policy_version 70312 (0.0008) [2023-10-11 21:51:25,819][71601] Updated weights for policy 0, policy_version 70370 (0.0009) [2023-10-11 21:51:25,940][71635] Updated weights for policy 1, policy_version 70322 (0.0008) [2023-10-11 21:51:26,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 144048128. Throughput: 0: 1810.1, 1: 1811.0. Samples: 36029990. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-11 21:51:26,035][70582] Avg episode reward: [(0, '63.620'), (1, '109.480')] [2023-10-11 21:51:26,188][71601] Updated weights for policy 0, policy_version 70380 (0.0009) [2023-10-11 21:51:26,304][71635] Updated weights for policy 1, policy_version 70332 (0.0007) [2023-10-11 21:51:26,554][71601] Updated weights for policy 0, policy_version 70390 (0.0009) [2023-10-11 21:51:26,930][71601] Updated weights for policy 0, policy_version 70400 (0.0008) [2023-10-11 21:51:29,994][71635] Updated weights for policy 1, policy_version 70342 (0.0007) [2023-10-11 21:51:30,356][71635] Updated weights for policy 1, policy_version 70352 (0.0008) [2023-10-11 21:51:30,723][71635] Updated weights for policy 1, policy_version 70362 (0.0008) [2023-10-11 21:51:30,780][71601] Updated weights for policy 0, policy_version 70410 (0.0007) [2023-10-11 21:51:31,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 144146432. Throughput: 0: 1804.4, 1: 1803.3. Samples: 36039780. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-11 21:51:31,034][70582] Avg episode reward: [(0, '64.100'), (1, '112.430')] [2023-10-11 21:51:31,156][71601] Updated weights for policy 0, policy_version 70420 (0.0008) [2023-10-11 21:51:31,526][71601] Updated weights for policy 0, policy_version 70430 (0.0008) [2023-10-11 21:51:34,350][71635] Updated weights for policy 1, policy_version 70372 (0.0007) [2023-10-11 21:51:34,719][71635] Updated weights for policy 1, policy_version 70382 (0.0009) [2023-10-11 21:51:35,084][71635] Updated weights for policy 1, policy_version 70392 (0.0009) [2023-10-11 21:51:35,209][71601] Updated weights for policy 0, policy_version 70440 (0.0007) [2023-10-11 21:51:35,581][71601] Updated weights for policy 0, policy_version 70450 (0.0008) [2023-10-11 21:51:35,952][71601] Updated weights for policy 0, policy_version 70460 (0.0009) [2023-10-11 21:51:36,034][70582] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 144211968. Throughput: 0: 1807.2, 1: 1809.1. Samples: 36062248. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-11 21:51:36,034][70582] Avg episode reward: [(0, '66.740'), (1, '113.760')] [2023-10-11 21:51:38,650][71635] Updated weights for policy 1, policy_version 70402 (0.0009) [2023-10-11 21:51:39,023][71635] Updated weights for policy 1, policy_version 70412 (0.0007) [2023-10-11 21:51:39,380][71635] Updated weights for policy 1, policy_version 70422 (0.0009) [2023-10-11 21:51:39,748][71635] Updated weights for policy 1, policy_version 70432 (0.0009) [2023-10-11 21:51:39,784][71601] Updated weights for policy 0, policy_version 70470 (0.0009) [2023-10-11 21:51:40,161][71601] Updated weights for policy 0, policy_version 70480 (0.0008) [2023-10-11 21:51:40,525][71601] Updated weights for policy 0, policy_version 70490 (0.0010) [2023-10-11 21:51:41,034][70582] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 144310272. Throughput: 0: 1811.0, 1: 1811.6. Samples: 36082848. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:51:41,034][70582] Avg episode reward: [(0, '66.470'), (1, '115.670')] [2023-10-11 21:51:43,515][71635] Updated weights for policy 1, policy_version 70442 (0.0007) [2023-10-11 21:51:43,876][71635] Updated weights for policy 1, policy_version 70452 (0.0008) [2023-10-11 21:51:44,249][71635] Updated weights for policy 1, policy_version 70462 (0.0010) [2023-10-11 21:51:44,358][71601] Updated weights for policy 0, policy_version 70500 (0.0009) [2023-10-11 21:51:44,739][71601] Updated weights for policy 0, policy_version 70510 (0.0009) [2023-10-11 21:51:45,111][71601] Updated weights for policy 0, policy_version 70520 (0.0008) [2023-10-11 21:51:46,034][70582] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 144375808. Throughput: 0: 1794.6, 1: 1815.4. Samples: 36094804. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:51:46,035][70582] Avg episode reward: [(0, '65.330'), (1, '114.570')] [2023-10-11 21:51:48,092][71635] Updated weights for policy 1, policy_version 70472 (0.0009) [2023-10-11 21:51:48,466][71635] Updated weights for policy 1, policy_version 70482 (0.0009) [2023-10-11 21:51:48,731][71601] Updated weights for policy 0, policy_version 70530 (0.0009) [2023-10-11 21:51:48,826][71635] Updated weights for policy 1, policy_version 70492 (0.0009) [2023-10-11 21:51:49,110][71601] Updated weights for policy 0, policy_version 70540 (0.0009) [2023-10-11 21:51:49,486][71601] Updated weights for policy 0, policy_version 70550 (0.0010) [2023-10-11 21:51:49,857][71601] Updated weights for policy 0, policy_version 70560 (0.0007) [2023-10-11 21:51:51,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 144441344. Throughput: 0: 1808.2, 1: 1814.6. Samples: 36115080. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:51:51,034][70582] Avg episode reward: [(0, '66.560'), (1, '106.850')] [2023-10-11 21:51:52,617][71635] Updated weights for policy 1, policy_version 70502 (0.0009) [2023-10-11 21:51:53,010][71635] Updated weights for policy 1, policy_version 70512 (0.0009) [2023-10-11 21:51:53,369][71635] Updated weights for policy 1, policy_version 70522 (0.0009) [2023-10-11 21:51:53,500][71601] Updated weights for policy 0, policy_version 70570 (0.0009) [2023-10-11 21:51:53,876][71601] Updated weights for policy 0, policy_version 70580 (0.0008) [2023-10-11 21:51:54,248][71601] Updated weights for policy 0, policy_version 70590 (0.0008) [2023-10-11 21:51:56,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 144506880. Throughput: 0: 1788.5, 1: 1810.8. Samples: 36137104. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:51:56,034][70582] Avg episode reward: [(0, '68.840'), (1, '103.040')] [2023-10-11 21:51:56,042][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000070592_72286208.pth... [2023-10-11 21:51:56,042][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000070528_72220672.pth... [2023-10-11 21:51:56,073][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000068896_70549504.pth [2023-10-11 21:51:56,080][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000068832_70483968.pth [2023-10-11 21:51:57,075][71635] Updated weights for policy 1, policy_version 70532 (0.0008) [2023-10-11 21:51:57,440][71635] Updated weights for policy 1, policy_version 70542 (0.0008) [2023-10-11 21:51:57,807][71635] Updated weights for policy 1, policy_version 70552 (0.0007) [2023-10-11 21:51:57,961][71601] Updated weights for policy 0, policy_version 70600 (0.0007) [2023-10-11 21:51:58,332][71601] Updated weights for policy 0, policy_version 70610 (0.0010) [2023-10-11 21:51:58,706][71601] Updated weights for policy 0, policy_version 70620 (0.0008) [2023-10-11 21:52:01,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 144572416. Throughput: 0: 1799.7, 1: 1808.9. Samples: 36147244. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:52:01,035][70582] Avg episode reward: [(0, '68.290'), (1, '100.920')] [2023-10-11 21:52:01,566][71635] Updated weights for policy 1, policy_version 70562 (0.0007) [2023-10-11 21:52:01,933][71635] Updated weights for policy 1, policy_version 70572 (0.0009) [2023-10-11 21:52:02,298][71635] Updated weights for policy 1, policy_version 70582 (0.0010) [2023-10-11 21:52:02,488][71601] Updated weights for policy 0, policy_version 70630 (0.0009) [2023-10-11 21:52:02,659][71635] Updated weights for policy 1, policy_version 70592 (0.0008) [2023-10-11 21:52:02,862][71601] Updated weights for policy 0, policy_version 70640 (0.0008) [2023-10-11 21:52:03,234][71601] Updated weights for policy 0, policy_version 70650 (0.0010) [2023-10-11 21:52:06,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 144637952. Throughput: 0: 1789.4, 1: 1811.8. Samples: 36169396. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:52:06,034][70582] Avg episode reward: [(0, '65.970'), (1, '94.040')] [2023-10-11 21:52:06,390][71635] Updated weights for policy 1, policy_version 70602 (0.0010) [2023-10-11 21:52:06,764][71635] Updated weights for policy 1, policy_version 70612 (0.0011) [2023-10-11 21:52:06,890][71601] Updated weights for policy 0, policy_version 70660 (0.0009) [2023-10-11 21:52:07,122][71635] Updated weights for policy 1, policy_version 70622 (0.0008) [2023-10-11 21:52:07,265][71601] Updated weights for policy 0, policy_version 70670 (0.0009) [2023-10-11 21:52:07,650][71601] Updated weights for policy 0, policy_version 70680 (0.0007) [2023-10-11 21:52:10,858][71635] Updated weights for policy 1, policy_version 70632 (0.0009) [2023-10-11 21:52:11,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 144703488. Throughput: 0: 1784.4, 1: 1815.6. Samples: 36191988. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:52:11,034][70582] Avg episode reward: [(0, '67.220'), (1, '95.620')] [2023-10-11 21:52:11,227][71635] Updated weights for policy 1, policy_version 70642 (0.0009) [2023-10-11 21:52:11,354][71601] Updated weights for policy 0, policy_version 70690 (0.0008) [2023-10-11 21:52:11,588][71635] Updated weights for policy 1, policy_version 70652 (0.0008) [2023-10-11 21:52:11,733][71601] Updated weights for policy 0, policy_version 70700 (0.0008) [2023-10-11 21:52:12,105][71601] Updated weights for policy 0, policy_version 70710 (0.0008) [2023-10-11 21:52:12,474][71601] Updated weights for policy 0, policy_version 70720 (0.0009) [2023-10-11 21:52:15,349][71635] Updated weights for policy 1, policy_version 70662 (0.0008) [2023-10-11 21:52:15,718][71635] Updated weights for policy 1, policy_version 70672 (0.0007) [2023-10-11 21:52:16,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 144769024. Throughput: 0: 1787.0, 1: 1813.1. Samples: 36201782. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:52:16,034][70582] Avg episode reward: [(0, '66.740'), (1, '91.770')] [2023-10-11 21:52:16,087][71635] Updated weights for policy 1, policy_version 70682 (0.0007) [2023-10-11 21:52:16,291][71601] Updated weights for policy 0, policy_version 70730 (0.0007) [2023-10-11 21:52:16,663][71601] Updated weights for policy 0, policy_version 70740 (0.0008) [2023-10-11 21:52:17,041][71601] Updated weights for policy 0, policy_version 70750 (0.0007) [2023-10-11 21:52:19,936][71635] Updated weights for policy 1, policy_version 70692 (0.0008) [2023-10-11 21:52:20,295][71635] Updated weights for policy 1, policy_version 70702 (0.0011) [2023-10-11 21:52:20,666][71635] Updated weights for policy 1, policy_version 70712 (0.0007) [2023-10-11 21:52:20,714][71601] Updated weights for policy 0, policy_version 70760 (0.0007) [2023-10-11 21:52:21,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 144867328. Throughput: 0: 1787.9, 1: 1810.0. Samples: 36224152. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:52:21,034][70582] Avg episode reward: [(0, '68.790'), (1, '98.450')] [2023-10-11 21:52:21,080][71601] Updated weights for policy 0, policy_version 70770 (0.0007) [2023-10-11 21:52:21,449][71601] Updated weights for policy 0, policy_version 70780 (0.0008) [2023-10-11 21:52:24,353][71635] Updated weights for policy 1, policy_version 70722 (0.0008) [2023-10-11 21:52:24,729][71635] Updated weights for policy 1, policy_version 70732 (0.0010) [2023-10-11 21:52:25,102][71635] Updated weights for policy 1, policy_version 70742 (0.0008) [2023-10-11 21:52:25,105][71601] Updated weights for policy 0, policy_version 70790 (0.0008) [2023-10-11 21:52:25,459][71635] Updated weights for policy 1, policy_version 70752 (0.0008) [2023-10-11 21:52:25,490][71601] Updated weights for policy 0, policy_version 70800 (0.0009) [2023-10-11 21:52:25,858][71601] Updated weights for policy 0, policy_version 70810 (0.0011) [2023-10-11 21:52:26,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 144932864. Throughput: 0: 1805.6, 1: 1793.3. Samples: 36244800. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-11 21:52:26,034][70582] Avg episode reward: [(0, '69.000'), (1, '96.790')] [2023-10-11 21:52:29,094][71635] Updated weights for policy 1, policy_version 70762 (0.0009) [2023-10-11 21:52:29,455][71635] Updated weights for policy 1, policy_version 70772 (0.0008) [2023-10-11 21:52:29,569][71601] Updated weights for policy 0, policy_version 70820 (0.0010) [2023-10-11 21:52:29,820][71635] Updated weights for policy 1, policy_version 70782 (0.0007) [2023-10-11 21:52:29,937][71601] Updated weights for policy 0, policy_version 70830 (0.0009) [2023-10-11 21:52:30,313][71601] Updated weights for policy 0, policy_version 70840 (0.0008) [2023-10-11 21:52:31,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 145031168. Throughput: 0: 1798.9, 1: 1798.1. Samples: 36256670. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-11 21:52:31,035][70582] Avg episode reward: [(0, '72.660'), (1, '89.860')] [2023-10-11 21:52:33,576][71635] Updated weights for policy 1, policy_version 70792 (0.0007) [2023-10-11 21:52:33,936][71601] Updated weights for policy 0, policy_version 70850 (0.0008) [2023-10-11 21:52:33,944][71635] Updated weights for policy 1, policy_version 70802 (0.0008) [2023-10-11 21:52:34,312][71601] Updated weights for policy 0, policy_version 70860 (0.0007) [2023-10-11 21:52:34,319][71635] Updated weights for policy 1, policy_version 70812 (0.0008) [2023-10-11 21:52:34,683][71601] Updated weights for policy 0, policy_version 70870 (0.0007) [2023-10-11 21:52:35,059][71601] Updated weights for policy 0, policy_version 70880 (0.0010) [2023-10-11 21:52:36,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 145096704. Throughput: 0: 1810.3, 1: 1799.0. Samples: 36277498. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-11 21:52:36,035][70582] Avg episode reward: [(0, '71.370'), (1, '88.260')] [2023-10-11 21:52:38,192][71635] Updated weights for policy 1, policy_version 70822 (0.0008) [2023-10-11 21:52:38,588][71635] Updated weights for policy 1, policy_version 70832 (0.0007) [2023-10-11 21:52:38,897][71601] Updated weights for policy 0, policy_version 70890 (0.0007) [2023-10-11 21:52:38,955][71635] Updated weights for policy 1, policy_version 70842 (0.0009) [2023-10-11 21:52:39,265][71601] Updated weights for policy 0, policy_version 70900 (0.0009) [2023-10-11 21:52:39,648][71601] Updated weights for policy 0, policy_version 70910 (0.0007) [2023-10-11 21:52:41,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 145162240. Throughput: 0: 1800.7, 1: 1794.1. Samples: 36298874. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-11 21:52:41,035][70582] Avg episode reward: [(0, '72.010'), (1, '87.700')] [2023-10-11 21:52:42,718][71635] Updated weights for policy 1, policy_version 70852 (0.0009) [2023-10-11 21:52:43,075][71635] Updated weights for policy 1, policy_version 70862 (0.0008) [2023-10-11 21:52:43,367][71601] Updated weights for policy 0, policy_version 70920 (0.0008) [2023-10-11 21:52:43,433][71635] Updated weights for policy 1, policy_version 70872 (0.0009) [2023-10-11 21:52:43,737][71601] Updated weights for policy 0, policy_version 70930 (0.0009) [2023-10-11 21:52:44,111][71601] Updated weights for policy 0, policy_version 70940 (0.0010) [2023-10-11 21:52:46,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 145227776. Throughput: 0: 1815.9, 1: 1805.1. Samples: 36310188. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-11 21:52:46,034][70582] Avg episode reward: [(0, '76.650'), (1, '91.110')] [2023-10-11 21:52:47,123][71635] Updated weights for policy 1, policy_version 70882 (0.0008) [2023-10-11 21:52:47,501][71635] Updated weights for policy 1, policy_version 70892 (0.0009) [2023-10-11 21:52:47,849][71601] Updated weights for policy 0, policy_version 70950 (0.0008) [2023-10-11 21:52:47,862][71635] Updated weights for policy 1, policy_version 70902 (0.0008) [2023-10-11 21:52:48,222][71635] Updated weights for policy 1, policy_version 70912 (0.0008) [2023-10-11 21:52:48,223][71601] Updated weights for policy 0, policy_version 70960 (0.0009) [2023-10-11 21:52:48,586][71601] Updated weights for policy 0, policy_version 70970 (0.0009) [2023-10-11 21:52:51,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 145293312. Throughput: 0: 1803.8, 1: 1796.3. Samples: 36331402. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-11 21:52:51,035][70582] Avg episode reward: [(0, '73.480'), (1, '88.020')] [2023-10-11 21:52:52,017][71635] Updated weights for policy 1, policy_version 70922 (0.0007) [2023-10-11 21:52:52,261][71601] Updated weights for policy 0, policy_version 70980 (0.0007) [2023-10-11 21:52:52,382][71635] Updated weights for policy 1, policy_version 70932 (0.0008) [2023-10-11 21:52:52,632][71601] Updated weights for policy 0, policy_version 70990 (0.0007) [2023-10-11 21:52:52,739][71635] Updated weights for policy 1, policy_version 70942 (0.0008) [2023-10-11 21:52:53,002][71601] Updated weights for policy 0, policy_version 71000 (0.0007) [2023-10-11 21:52:56,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 145358848. Throughput: 0: 1810.9, 1: 1794.0. Samples: 36354208. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-11 21:52:56,034][70582] Avg episode reward: [(0, '70.670'), (1, '93.780')] [2023-10-11 21:52:56,408][71635] Updated weights for policy 1, policy_version 70952 (0.0007) [2023-10-11 21:52:56,580][71601] Updated weights for policy 0, policy_version 71010 (0.0009) [2023-10-11 21:52:56,770][71635] Updated weights for policy 1, policy_version 70962 (0.0007) [2023-10-11 21:52:56,947][71601] Updated weights for policy 0, policy_version 71020 (0.0008) [2023-10-11 21:52:57,144][71635] Updated weights for policy 1, policy_version 70972 (0.0007) [2023-10-11 21:52:57,325][71601] Updated weights for policy 0, policy_version 71030 (0.0007) [2023-10-11 21:52:57,692][71601] Updated weights for policy 0, policy_version 71040 (0.0007) [2023-10-11 21:53:01,016][71635] Updated weights for policy 1, policy_version 70982 (0.0008) [2023-10-11 21:53:01,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 145424384. Throughput: 0: 1812.9, 1: 1795.0. Samples: 36364138. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-11 21:53:01,035][70582] Avg episode reward: [(0, '65.930'), (1, '92.820')] [2023-10-11 21:53:01,383][71635] Updated weights for policy 1, policy_version 70992 (0.0008) [2023-10-11 21:53:01,432][71601] Updated weights for policy 0, policy_version 71050 (0.0008) [2023-10-11 21:53:01,753][71635] Updated weights for policy 1, policy_version 71002 (0.0007) [2023-10-11 21:53:01,806][71601] Updated weights for policy 0, policy_version 71060 (0.0009) [2023-10-11 21:53:02,169][71601] Updated weights for policy 0, policy_version 71070 (0.0007) [2023-10-11 21:53:05,278][71635] Updated weights for policy 1, policy_version 71012 (0.0008) [2023-10-11 21:53:05,640][71635] Updated weights for policy 1, policy_version 71022 (0.0009) [2023-10-11 21:53:05,920][71601] Updated weights for policy 0, policy_version 71080 (0.0008) [2023-10-11 21:53:06,013][71635] Updated weights for policy 1, policy_version 71032 (0.0007) [2023-10-11 21:53:06,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 145489920. Throughput: 0: 1807.9, 1: 1800.9. Samples: 36386550. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-11 21:53:06,035][70582] Avg episode reward: [(0, '67.920'), (1, '94.150')] [2023-10-11 21:53:06,299][71601] Updated weights for policy 0, policy_version 71090 (0.0008) [2023-10-11 21:53:06,662][71601] Updated weights for policy 0, policy_version 71100 (0.0009) [2023-10-11 21:53:09,692][71635] Updated weights for policy 1, policy_version 71042 (0.0008) [2023-10-11 21:53:10,057][71635] Updated weights for policy 1, policy_version 71052 (0.0009) [2023-10-11 21:53:10,396][71601] Updated weights for policy 0, policy_version 71110 (0.0008) [2023-10-11 21:53:10,426][71635] Updated weights for policy 1, policy_version 71062 (0.0007) [2023-10-11 21:53:10,760][71601] Updated weights for policy 0, policy_version 71120 (0.0007) [2023-10-11 21:53:10,789][71635] Updated weights for policy 1, policy_version 71072 (0.0008) [2023-10-11 21:53:11,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 145588224. Throughput: 0: 1817.6, 1: 1816.8. Samples: 36408352. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-10-11 21:53:11,034][70582] Avg episode reward: [(0, '69.970'), (1, '92.240')] [2023-10-11 21:53:11,130][71601] Updated weights for policy 0, policy_version 71130 (0.0008) [2023-10-11 21:53:14,562][71635] Updated weights for policy 1, policy_version 71082 (0.0007) [2023-10-11 21:53:14,828][71601] Updated weights for policy 0, policy_version 71140 (0.0008) [2023-10-11 21:53:14,928][71635] Updated weights for policy 1, policy_version 71092 (0.0008) [2023-10-11 21:53:15,204][71601] Updated weights for policy 0, policy_version 71150 (0.0009) [2023-10-11 21:53:15,293][71635] Updated weights for policy 1, policy_version 71102 (0.0008) [2023-10-11 21:53:15,567][71601] Updated weights for policy 0, policy_version 71160 (0.0008) [2023-10-11 21:53:16,034][70582] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 145686528. Throughput: 0: 1815.1, 1: 1808.4. Samples: 36419724. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-10-11 21:53:16,035][70582] Avg episode reward: [(0, '66.400'), (1, '91.220')] [2023-10-11 21:53:18,947][71635] Updated weights for policy 1, policy_version 71112 (0.0008) [2023-10-11 21:53:19,311][71635] Updated weights for policy 1, policy_version 71122 (0.0008) [2023-10-11 21:53:19,421][71601] Updated weights for policy 0, policy_version 71170 (0.0009) [2023-10-11 21:53:19,682][71635] Updated weights for policy 1, policy_version 71132 (0.0007) [2023-10-11 21:53:19,785][71601] Updated weights for policy 0, policy_version 71180 (0.0008) [2023-10-11 21:53:20,167][71601] Updated weights for policy 0, policy_version 71190 (0.0007) [2023-10-11 21:53:20,539][71601] Updated weights for policy 0, policy_version 71200 (0.0008) [2023-10-11 21:53:21,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 145752064. Throughput: 0: 1815.3, 1: 1823.0. Samples: 36441222. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-10-11 21:53:21,035][70582] Avg episode reward: [(0, '71.750'), (1, '92.440')] [2023-10-11 21:53:23,407][71635] Updated weights for policy 1, policy_version 71142 (0.0009) [2023-10-11 21:53:23,797][71635] Updated weights for policy 1, policy_version 71152 (0.0007) [2023-10-11 21:53:24,150][71635] Updated weights for policy 1, policy_version 71162 (0.0008) [2023-10-11 21:53:24,184][71601] Updated weights for policy 0, policy_version 71210 (0.0009) [2023-10-11 21:53:24,553][71601] Updated weights for policy 0, policy_version 71220 (0.0009) [2023-10-11 21:53:24,925][71601] Updated weights for policy 0, policy_version 71230 (0.0008) [2023-10-11 21:53:26,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 145817600. Throughput: 0: 1803.2, 1: 1822.4. Samples: 36462026. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-10-11 21:53:26,035][70582] Avg episode reward: [(0, '67.790'), (1, '89.700')] [2023-10-11 21:53:27,814][71635] Updated weights for policy 1, policy_version 71172 (0.0009) [2023-10-11 21:53:28,190][71635] Updated weights for policy 1, policy_version 71182 (0.0010) [2023-10-11 21:53:28,536][71601] Updated weights for policy 0, policy_version 71240 (0.0007) [2023-10-11 21:53:28,562][71635] Updated weights for policy 1, policy_version 71192 (0.0008) [2023-10-11 21:53:28,909][71601] Updated weights for policy 0, policy_version 71250 (0.0008) [2023-10-11 21:53:29,270][71601] Updated weights for policy 0, policy_version 71260 (0.0010) [2023-10-11 21:53:31,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 145883136. Throughput: 0: 1808.6, 1: 1827.0. Samples: 36473790. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-10-11 21:53:31,034][70582] Avg episode reward: [(0, '71.270'), (1, '96.240')] [2023-10-11 21:53:32,203][71635] Updated weights for policy 1, policy_version 71202 (0.0007) [2023-10-11 21:53:32,567][71635] Updated weights for policy 1, policy_version 71212 (0.0008) [2023-10-11 21:53:32,937][71635] Updated weights for policy 1, policy_version 71222 (0.0008) [2023-10-11 21:53:33,073][71601] Updated weights for policy 0, policy_version 71270 (0.0009) [2023-10-11 21:53:33,296][71635] Updated weights for policy 1, policy_version 71232 (0.0009) [2023-10-11 21:53:33,447][71601] Updated weights for policy 0, policy_version 71280 (0.0007) [2023-10-11 21:53:33,812][71601] Updated weights for policy 0, policy_version 71290 (0.0008) [2023-10-11 21:53:36,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 145948672. Throughput: 0: 1805.0, 1: 1818.0. Samples: 36494436. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-10-11 21:53:36,034][70582] Avg episode reward: [(0, '70.310'), (1, '93.720')] [2023-10-11 21:53:37,084][71635] Updated weights for policy 1, policy_version 71242 (0.0008) [2023-10-11 21:53:37,459][71635] Updated weights for policy 1, policy_version 71252 (0.0009) [2023-10-11 21:53:37,677][71601] Updated weights for policy 0, policy_version 71300 (0.0008) [2023-10-11 21:53:37,818][71635] Updated weights for policy 1, policy_version 71262 (0.0009) [2023-10-11 21:53:38,044][71601] Updated weights for policy 0, policy_version 71310 (0.0009) [2023-10-11 21:53:38,421][71601] Updated weights for policy 0, policy_version 71320 (0.0010) [2023-10-11 21:53:41,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 146014208. Throughput: 0: 1797.1, 1: 1821.5. Samples: 36517044. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-10-11 21:53:41,035][70582] Avg episode reward: [(0, '73.840'), (1, '95.500')] [2023-10-11 21:53:41,421][71635] Updated weights for policy 1, policy_version 71272 (0.0007) [2023-10-11 21:53:41,789][71635] Updated weights for policy 1, policy_version 71282 (0.0008) [2023-10-11 21:53:42,162][71635] Updated weights for policy 1, policy_version 71292 (0.0009) [2023-10-11 21:53:42,318][71601] Updated weights for policy 0, policy_version 71330 (0.0009) [2023-10-11 21:53:42,693][71601] Updated weights for policy 0, policy_version 71340 (0.0010) [2023-10-11 21:53:43,068][71601] Updated weights for policy 0, policy_version 71350 (0.0009) [2023-10-11 21:53:43,446][71601] Updated weights for policy 0, policy_version 71360 (0.0009) [2023-10-11 21:53:45,767][71635] Updated weights for policy 1, policy_version 71302 (0.0008) [2023-10-11 21:53:46,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 146079744. Throughput: 0: 1797.2, 1: 1821.1. Samples: 36526962. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-10-11 21:53:46,035][70582] Avg episode reward: [(0, '77.870'), (1, '96.100')] [2023-10-11 21:53:46,135][71635] Updated weights for policy 1, policy_version 71312 (0.0007) [2023-10-11 21:53:46,496][71635] Updated weights for policy 1, policy_version 71322 (0.0008) [2023-10-11 21:53:47,129][71601] Updated weights for policy 0, policy_version 71370 (0.0007) [2023-10-11 21:53:47,498][71601] Updated weights for policy 0, policy_version 71380 (0.0009) [2023-10-11 21:53:47,883][71601] Updated weights for policy 0, policy_version 71390 (0.0009) [2023-10-11 21:53:50,352][71635] Updated weights for policy 1, policy_version 71332 (0.0009) [2023-10-11 21:53:50,714][71635] Updated weights for policy 1, policy_version 71342 (0.0010) [2023-10-11 21:53:51,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 146145280. Throughput: 0: 1803.1, 1: 1817.6. Samples: 36549480. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-10-11 21:53:51,034][70582] Avg episode reward: [(0, '76.480'), (1, '102.140')] [2023-10-11 21:53:51,081][71635] Updated weights for policy 1, policy_version 71352 (0.0009) [2023-10-11 21:53:51,404][71601] Updated weights for policy 0, policy_version 71400 (0.0009) [2023-10-11 21:53:51,781][71601] Updated weights for policy 0, policy_version 71410 (0.0007) [2023-10-11 21:53:52,160][71601] Updated weights for policy 0, policy_version 71420 (0.0009) [2023-10-11 21:53:54,767][71635] Updated weights for policy 1, policy_version 71362 (0.0007) [2023-10-11 21:53:55,131][71635] Updated weights for policy 1, policy_version 71372 (0.0007) [2023-10-11 21:53:55,495][71635] Updated weights for policy 1, policy_version 71382 (0.0007) [2023-10-11 21:53:55,752][71601] Updated weights for policy 0, policy_version 71430 (0.0007) [2023-10-11 21:53:55,866][71635] Updated weights for policy 1, policy_version 71392 (0.0008) [2023-10-11 21:53:56,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 146243584. Throughput: 0: 1816.7, 1: 1815.6. Samples: 36571804. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-11 21:53:56,034][70582] Avg episode reward: [(0, '75.960'), (1, '99.620')] [2023-10-11 21:53:56,045][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000071392_73105408.pth... [2023-10-11 21:53:56,086][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000069696_71368704.pth [2023-10-11 21:53:56,118][71601] Updated weights for policy 0, policy_version 71440 (0.0007) [2023-10-11 21:53:56,498][71601] Updated weights for policy 0, policy_version 71450 (0.0007) [2023-10-11 21:53:56,715][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000071456_73170944.pth... [2023-10-11 21:53:56,754][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000069760_71434240.pth [2023-10-11 21:53:59,480][71635] Updated weights for policy 1, policy_version 71402 (0.0009) [2023-10-11 21:53:59,857][71635] Updated weights for policy 1, policy_version 71412 (0.0009) [2023-10-11 21:54:00,129][71601] Updated weights for policy 0, policy_version 71460 (0.0009) [2023-10-11 21:54:00,218][71635] Updated weights for policy 1, policy_version 71422 (0.0007) [2023-10-11 21:54:00,503][71601] Updated weights for policy 0, policy_version 71470 (0.0009) [2023-10-11 21:54:00,878][71601] Updated weights for policy 0, policy_version 71480 (0.0008) [2023-10-11 21:54:01,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 146309120. Throughput: 0: 1807.1, 1: 1811.8. Samples: 36582576. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-11 21:54:01,034][70582] Avg episode reward: [(0, '75.640'), (1, '97.590')] [2023-10-11 21:54:03,955][71635] Updated weights for policy 1, policy_version 71432 (0.0009) [2023-10-11 21:54:04,327][71635] Updated weights for policy 1, policy_version 71442 (0.0009) [2023-10-11 21:54:04,540][71601] Updated weights for policy 0, policy_version 71490 (0.0007) [2023-10-11 21:54:04,696][71635] Updated weights for policy 1, policy_version 71452 (0.0009) [2023-10-11 21:54:04,911][71601] Updated weights for policy 0, policy_version 71500 (0.0007) [2023-10-11 21:54:05,295][71601] Updated weights for policy 0, policy_version 71510 (0.0008) [2023-10-11 21:54:05,669][71601] Updated weights for policy 0, policy_version 71520 (0.0009) [2023-10-11 21:54:06,034][70582] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 146407424. Throughput: 0: 1819.0, 1: 1811.9. Samples: 36604610. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-11 21:54:06,035][70582] Avg episode reward: [(0, '74.460'), (1, '99.290')] [2023-10-11 21:54:08,573][71635] Updated weights for policy 1, policy_version 71462 (0.0008) [2023-10-11 21:54:08,956][71635] Updated weights for policy 1, policy_version 71472 (0.0009) [2023-10-11 21:54:09,145][71601] Updated weights for policy 0, policy_version 71530 (0.0008) [2023-10-11 21:54:09,325][71635] Updated weights for policy 1, policy_version 71482 (0.0008) [2023-10-11 21:54:09,512][71601] Updated weights for policy 0, policy_version 71540 (0.0008) [2023-10-11 21:54:09,892][71601] Updated weights for policy 0, policy_version 71550 (0.0010) [2023-10-11 21:54:11,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 146472960. Throughput: 0: 1823.6, 1: 1797.5. Samples: 36624976. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-11 21:54:11,035][70582] Avg episode reward: [(0, '74.480'), (1, '99.430')] [2023-10-11 21:54:13,007][71635] Updated weights for policy 1, policy_version 71492 (0.0008) [2023-10-11 21:54:13,379][71635] Updated weights for policy 1, policy_version 71502 (0.0009) [2023-10-11 21:54:13,705][71601] Updated weights for policy 0, policy_version 71560 (0.0008) [2023-10-11 21:54:13,751][71635] Updated weights for policy 1, policy_version 71512 (0.0008) [2023-10-11 21:54:14,077][71601] Updated weights for policy 0, policy_version 71570 (0.0008) [2023-10-11 21:54:14,446][71601] Updated weights for policy 0, policy_version 71580 (0.0007) [2023-10-11 21:54:16,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 146538496. Throughput: 0: 1824.8, 1: 1806.7. Samples: 36637206. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-11 21:54:16,035][70582] Avg episode reward: [(0, '75.690'), (1, '98.350')] [2023-10-11 21:54:17,606][71635] Updated weights for policy 1, policy_version 71522 (0.0009) [2023-10-11 21:54:17,981][71635] Updated weights for policy 1, policy_version 71532 (0.0009) [2023-10-11 21:54:18,328][71601] Updated weights for policy 0, policy_version 71590 (0.0008) [2023-10-11 21:54:18,347][71635] Updated weights for policy 1, policy_version 71542 (0.0007) [2023-10-11 21:54:18,694][71601] Updated weights for policy 0, policy_version 71600 (0.0009) [2023-10-11 21:54:18,721][71635] Updated weights for policy 1, policy_version 71552 (0.0008) [2023-10-11 21:54:19,071][71601] Updated weights for policy 0, policy_version 71610 (0.0007) [2023-10-11 21:54:21,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 146604032. Throughput: 0: 1819.1, 1: 1798.7. Samples: 36657240. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-11 21:54:21,035][70582] Avg episode reward: [(0, '76.610'), (1, '97.420')] [2023-10-11 21:54:22,339][71635] Updated weights for policy 1, policy_version 71562 (0.0007) [2023-10-11 21:54:22,679][71601] Updated weights for policy 0, policy_version 71620 (0.0008) [2023-10-11 21:54:22,698][71635] Updated weights for policy 1, policy_version 71572 (0.0007) [2023-10-11 21:54:23,049][71601] Updated weights for policy 0, policy_version 71630 (0.0008) [2023-10-11 21:54:23,069][71635] Updated weights for policy 1, policy_version 71582 (0.0008) [2023-10-11 21:54:23,421][71601] Updated weights for policy 0, policy_version 71640 (0.0008) [2023-10-11 21:54:26,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 146669568. Throughput: 0: 1825.3, 1: 1803.1. Samples: 36680322. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-11 21:54:26,035][70582] Avg episode reward: [(0, '80.100'), (1, '102.840')] [2023-10-11 21:54:26,800][71635] Updated weights for policy 1, policy_version 71592 (0.0008) [2023-10-11 21:54:27,081][71601] Updated weights for policy 0, policy_version 71650 (0.0008) [2023-10-11 21:54:27,175][71635] Updated weights for policy 1, policy_version 71602 (0.0008) [2023-10-11 21:54:27,446][71601] Updated weights for policy 0, policy_version 71660 (0.0008) [2023-10-11 21:54:27,542][71635] Updated weights for policy 1, policy_version 71612 (0.0008) [2023-10-11 21:54:27,818][71601] Updated weights for policy 0, policy_version 71670 (0.0007) [2023-10-11 21:54:28,189][71601] Updated weights for policy 0, policy_version 71680 (0.0008) [2023-10-11 21:54:31,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 146735104. Throughput: 0: 1827.1, 1: 1801.1. Samples: 36690232. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-11 21:54:31,034][70582] Avg episode reward: [(0, '76.440'), (1, '98.780')] [2023-10-11 21:54:31,334][71635] Updated weights for policy 1, policy_version 71622 (0.0009) [2023-10-11 21:54:31,696][71635] Updated weights for policy 1, policy_version 71632 (0.0008) [2023-10-11 21:54:31,728][71601] Updated weights for policy 0, policy_version 71690 (0.0008) [2023-10-11 21:54:32,071][71635] Updated weights for policy 1, policy_version 71642 (0.0008) [2023-10-11 21:54:32,107][71601] Updated weights for policy 0, policy_version 71700 (0.0007) [2023-10-11 21:54:32,473][71601] Updated weights for policy 0, policy_version 71710 (0.0008) [2023-10-11 21:54:35,818][71635] Updated weights for policy 1, policy_version 71652 (0.0007) [2023-10-11 21:54:36,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 146800640. Throughput: 0: 1828.6, 1: 1800.4. Samples: 36712786. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-11 21:54:36,034][70582] Avg episode reward: [(0, '76.270'), (1, '98.060')] [2023-10-11 21:54:36,141][71601] Updated weights for policy 0, policy_version 71720 (0.0009) [2023-10-11 21:54:36,187][71635] Updated weights for policy 1, policy_version 71662 (0.0008) [2023-10-11 21:54:36,519][71601] Updated weights for policy 0, policy_version 71730 (0.0008) [2023-10-11 21:54:36,553][71635] Updated weights for policy 1, policy_version 71672 (0.0007) [2023-10-11 21:54:36,892][71601] Updated weights for policy 0, policy_version 71740 (0.0007) [2023-10-11 21:54:40,053][71635] Updated weights for policy 1, policy_version 71682 (0.0008) [2023-10-11 21:54:40,411][71635] Updated weights for policy 1, policy_version 71692 (0.0008) [2023-10-11 21:54:40,643][71601] Updated weights for policy 0, policy_version 71750 (0.0008) [2023-10-11 21:54:40,779][71635] Updated weights for policy 1, policy_version 71702 (0.0007) [2023-10-11 21:54:41,006][71601] Updated weights for policy 0, policy_version 71760 (0.0007) [2023-10-11 21:54:41,034][70582] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 146866176. Throughput: 0: 1818.2, 1: 1810.7. Samples: 36735108. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-11 21:54:41,035][70582] Avg episode reward: [(0, '77.780'), (1, '98.400')] [2023-10-11 21:54:41,143][71635] Updated weights for policy 1, policy_version 71712 (0.0009) [2023-10-11 21:54:41,386][71601] Updated weights for policy 0, policy_version 71770 (0.0007) [2023-10-11 21:54:44,687][71635] Updated weights for policy 1, policy_version 71722 (0.0011) [2023-10-11 21:54:45,053][71635] Updated weights for policy 1, policy_version 71732 (0.0009) [2023-10-11 21:54:45,059][71601] Updated weights for policy 0, policy_version 71780 (0.0008) [2023-10-11 21:54:45,420][71601] Updated weights for policy 0, policy_version 71790 (0.0008) [2023-10-11 21:54:45,422][71635] Updated weights for policy 1, policy_version 71742 (0.0007) [2023-10-11 21:54:45,793][71601] Updated weights for policy 0, policy_version 71800 (0.0008) [2023-10-11 21:54:46,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 146964480. Throughput: 0: 1819.8, 1: 1805.8. Samples: 36745728. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-11 21:54:46,035][70582] Avg episode reward: [(0, '78.980'), (1, '97.490')] [2023-10-11 21:54:49,238][71635] Updated weights for policy 1, policy_version 71752 (0.0008) [2023-10-11 21:54:49,494][71601] Updated weights for policy 0, policy_version 71810 (0.0009) [2023-10-11 21:54:49,597][71635] Updated weights for policy 1, policy_version 71762 (0.0008) [2023-10-11 21:54:49,868][71601] Updated weights for policy 0, policy_version 71820 (0.0008) [2023-10-11 21:54:49,977][71635] Updated weights for policy 1, policy_version 71772 (0.0007) [2023-10-11 21:54:50,246][71601] Updated weights for policy 0, policy_version 71830 (0.0008) [2023-10-11 21:54:50,610][71601] Updated weights for policy 0, policy_version 71840 (0.0009) [2023-10-11 21:54:51,034][70582] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 147062784. Throughput: 0: 1819.0, 1: 1809.6. Samples: 36767896. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-11 21:54:51,035][70582] Avg episode reward: [(0, '79.180'), (1, '99.220')] [2023-10-11 21:54:53,686][71635] Updated weights for policy 1, policy_version 71782 (0.0008) [2023-10-11 21:54:54,050][71635] Updated weights for policy 1, policy_version 71792 (0.0010) [2023-10-11 21:54:54,188][71601] Updated weights for policy 0, policy_version 71850 (0.0008) [2023-10-11 21:54:54,421][71635] Updated weights for policy 1, policy_version 71802 (0.0008) [2023-10-11 21:54:54,549][71601] Updated weights for policy 0, policy_version 71860 (0.0008) [2023-10-11 21:54:54,927][71601] Updated weights for policy 0, policy_version 71870 (0.0010) [2023-10-11 21:54:56,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 147128320. Throughput: 0: 1819.3, 1: 1814.5. Samples: 36788498. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-11 21:54:56,035][70582] Avg episode reward: [(0, '76.390'), (1, '100.690')] [2023-10-11 21:54:58,027][71635] Updated weights for policy 1, policy_version 71812 (0.0009) [2023-10-11 21:54:58,400][71635] Updated weights for policy 1, policy_version 71822 (0.0009) [2023-10-11 21:54:58,509][71601] Updated weights for policy 0, policy_version 71880 (0.0007) [2023-10-11 21:54:58,761][71635] Updated weights for policy 1, policy_version 71832 (0.0007) [2023-10-11 21:54:58,884][71601] Updated weights for policy 0, policy_version 71890 (0.0008) [2023-10-11 21:54:59,255][71601] Updated weights for policy 0, policy_version 71900 (0.0010) [2023-10-11 21:55:01,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 147193856. Throughput: 0: 1816.1, 1: 1818.6. Samples: 36800766. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-11 21:55:01,034][70582] Avg episode reward: [(0, '77.690'), (1, '102.320')] [2023-10-11 21:55:02,465][71635] Updated weights for policy 1, policy_version 71842 (0.0007) [2023-10-11 21:55:02,838][71635] Updated weights for policy 1, policy_version 71852 (0.0007) [2023-10-11 21:55:02,938][71601] Updated weights for policy 0, policy_version 71910 (0.0008) [2023-10-11 21:55:03,212][71635] Updated weights for policy 1, policy_version 71862 (0.0008) [2023-10-11 21:55:03,311][71601] Updated weights for policy 0, policy_version 71920 (0.0007) [2023-10-11 21:55:03,579][71635] Updated weights for policy 1, policy_version 71872 (0.0008) [2023-10-11 21:55:03,688][71601] Updated weights for policy 0, policy_version 71930 (0.0007) [2023-10-11 21:55:06,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 147259392. Throughput: 0: 1823.8, 1: 1820.4. Samples: 36821228. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-11 21:55:06,035][70582] Avg episode reward: [(0, '72.650'), (1, '104.330')] [2023-10-11 21:55:07,162][71635] Updated weights for policy 1, policy_version 71882 (0.0008) [2023-10-11 21:55:07,377][71601] Updated weights for policy 0, policy_version 71940 (0.0009) [2023-10-11 21:55:07,523][71635] Updated weights for policy 1, policy_version 71892 (0.0009) [2023-10-11 21:55:07,757][71601] Updated weights for policy 0, policy_version 71950 (0.0009) [2023-10-11 21:55:07,888][71635] Updated weights for policy 1, policy_version 71902 (0.0008) [2023-10-11 21:55:08,124][71601] Updated weights for policy 0, policy_version 71960 (0.0008) [2023-10-11 21:55:11,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 147324928. Throughput: 0: 1825.3, 1: 1816.3. Samples: 36844192. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-11 21:55:11,034][70582] Avg episode reward: [(0, '71.630'), (1, '102.260')] [2023-10-11 21:55:11,668][71635] Updated weights for policy 1, policy_version 71912 (0.0008) [2023-10-11 21:55:11,869][71601] Updated weights for policy 0, policy_version 71970 (0.0008) [2023-10-11 21:55:12,034][71635] Updated weights for policy 1, policy_version 71922 (0.0008) [2023-10-11 21:55:12,242][71601] Updated weights for policy 0, policy_version 71980 (0.0007) [2023-10-11 21:55:12,398][71635] Updated weights for policy 1, policy_version 71932 (0.0008) [2023-10-11 21:55:12,616][71601] Updated weights for policy 0, policy_version 71990 (0.0008) [2023-10-11 21:55:12,981][71601] Updated weights for policy 0, policy_version 72000 (0.0009) [2023-10-11 21:55:16,030][71635] Updated weights for policy 1, policy_version 71942 (0.0008) [2023-10-11 21:55:16,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 147390464. Throughput: 0: 1820.2, 1: 1813.9. Samples: 36853766. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-11 21:55:16,035][70582] Avg episode reward: [(0, '73.650'), (1, '104.160')] [2023-10-11 21:55:16,399][71635] Updated weights for policy 1, policy_version 71952 (0.0009) [2023-10-11 21:55:16,777][71635] Updated weights for policy 1, policy_version 71962 (0.0008) [2023-10-11 21:55:16,854][71601] Updated weights for policy 0, policy_version 72010 (0.0008) [2023-10-11 21:55:17,226][71601] Updated weights for policy 0, policy_version 72020 (0.0009) [2023-10-11 21:55:17,599][71601] Updated weights for policy 0, policy_version 72030 (0.0008) [2023-10-11 21:55:20,457][71635] Updated weights for policy 1, policy_version 71972 (0.0008) [2023-10-11 21:55:20,825][71635] Updated weights for policy 1, policy_version 71982 (0.0007) [2023-10-11 21:55:21,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 147456000. Throughput: 0: 1819.5, 1: 1821.2. Samples: 36876618. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-11 21:55:21,034][70582] Avg episode reward: [(0, '73.030'), (1, '101.810')] [2023-10-11 21:55:21,188][71635] Updated weights for policy 1, policy_version 71992 (0.0008) [2023-10-11 21:55:21,478][71601] Updated weights for policy 0, policy_version 72040 (0.0007) [2023-10-11 21:55:21,865][71601] Updated weights for policy 0, policy_version 72050 (0.0010) [2023-10-11 21:55:22,242][71601] Updated weights for policy 0, policy_version 72060 (0.0010) [2023-10-11 21:55:25,035][71635] Updated weights for policy 1, policy_version 72002 (0.0008) [2023-10-11 21:55:25,398][71635] Updated weights for policy 1, policy_version 72012 (0.0011) [2023-10-11 21:55:25,769][71635] Updated weights for policy 1, policy_version 72022 (0.0008) [2023-10-11 21:55:25,961][71601] Updated weights for policy 0, policy_version 72070 (0.0007) [2023-10-11 21:55:26,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 147521536. Throughput: 0: 1817.0, 1: 1817.9. Samples: 36898678. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-11 21:55:26,035][70582] Avg episode reward: [(0, '69.430'), (1, '101.060')] [2023-10-11 21:55:26,124][71635] Updated weights for policy 1, policy_version 72032 (0.0007) [2023-10-11 21:55:26,330][71601] Updated weights for policy 0, policy_version 72080 (0.0008) [2023-10-11 21:55:26,699][71601] Updated weights for policy 0, policy_version 72090 (0.0009) [2023-10-11 21:55:29,854][71635] Updated weights for policy 1, policy_version 72042 (0.0009) [2023-10-11 21:55:30,222][71635] Updated weights for policy 1, policy_version 72052 (0.0008) [2023-10-11 21:55:30,404][71601] Updated weights for policy 0, policy_version 72100 (0.0008) [2023-10-11 21:55:30,592][71635] Updated weights for policy 1, policy_version 72062 (0.0009) [2023-10-11 21:55:30,774][71601] Updated weights for policy 0, policy_version 72110 (0.0009) [2023-10-11 21:55:31,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 147619840. Throughput: 0: 1811.5, 1: 1818.8. Samples: 36909090. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-11 21:55:31,035][70582] Avg episode reward: [(0, '70.520'), (1, '101.020')] [2023-10-11 21:55:31,151][71601] Updated weights for policy 0, policy_version 72120 (0.0008) [2023-10-11 21:55:34,285][71635] Updated weights for policy 1, policy_version 72072 (0.0007) [2023-10-11 21:55:34,659][71635] Updated weights for policy 1, policy_version 72082 (0.0009) [2023-10-11 21:55:34,820][71601] Updated weights for policy 0, policy_version 72130 (0.0009) [2023-10-11 21:55:35,018][71635] Updated weights for policy 1, policy_version 72092 (0.0007) [2023-10-11 21:55:35,195][71601] Updated weights for policy 0, policy_version 72140 (0.0008) [2023-10-11 21:55:35,573][71601] Updated weights for policy 0, policy_version 72150 (0.0009) [2023-10-11 21:55:35,952][71601] Updated weights for policy 0, policy_version 72160 (0.0011) [2023-10-11 21:55:36,034][70582] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 147718144. Throughput: 0: 1812.2, 1: 1821.9. Samples: 36931432. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-11 21:55:36,035][70582] Avg episode reward: [(0, '69.960'), (1, '98.230')] [2023-10-11 21:55:38,878][71635] Updated weights for policy 1, policy_version 72102 (0.0009) [2023-10-11 21:55:39,257][71635] Updated weights for policy 1, policy_version 72112 (0.0010) [2023-10-11 21:55:39,614][71635] Updated weights for policy 1, policy_version 72122 (0.0007) [2023-10-11 21:55:39,730][71601] Updated weights for policy 0, policy_version 72170 (0.0008) [2023-10-11 21:55:40,102][71601] Updated weights for policy 0, policy_version 72180 (0.0010) [2023-10-11 21:55:40,463][71601] Updated weights for policy 0, policy_version 72190 (0.0011) [2023-10-11 21:55:41,034][70582] Fps is (10 sec: 16384.4, 60 sec: 15291.8, 300 sec: 14551.2). Total num frames: 147783680. Throughput: 0: 1811.2, 1: 1813.9. Samples: 36951626. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-11 21:55:41,034][70582] Avg episode reward: [(0, '71.340'), (1, '98.260')] [2023-10-11 21:55:43,382][71635] Updated weights for policy 1, policy_version 72132 (0.0008) [2023-10-11 21:55:43,747][71635] Updated weights for policy 1, policy_version 72142 (0.0010) [2023-10-11 21:55:44,037][71601] Updated weights for policy 0, policy_version 72200 (0.0009) [2023-10-11 21:55:44,109][71635] Updated weights for policy 1, policy_version 72152 (0.0008) [2023-10-11 21:55:44,411][71601] Updated weights for policy 0, policy_version 72210 (0.0009) [2023-10-11 21:55:44,779][71601] Updated weights for policy 0, policy_version 72220 (0.0009) [2023-10-11 21:55:46,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 147849216. Throughput: 0: 1814.8, 1: 1819.4. Samples: 36964306. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-11 21:55:46,035][70582] Avg episode reward: [(0, '72.040'), (1, '96.850')] [2023-10-11 21:55:47,901][71635] Updated weights for policy 1, policy_version 72162 (0.0008) [2023-10-11 21:55:48,268][71635] Updated weights for policy 1, policy_version 72172 (0.0007) [2023-10-11 21:55:48,400][71601] Updated weights for policy 0, policy_version 72230 (0.0007) [2023-10-11 21:55:48,620][71635] Updated weights for policy 1, policy_version 72182 (0.0009) [2023-10-11 21:55:48,765][71601] Updated weights for policy 0, policy_version 72240 (0.0008) [2023-10-11 21:55:48,980][71635] Updated weights for policy 1, policy_version 72192 (0.0010) [2023-10-11 21:55:49,138][71601] Updated weights for policy 0, policy_version 72250 (0.0007) [2023-10-11 21:55:51,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 147914752. Throughput: 0: 1810.0, 1: 1806.1. Samples: 36983952. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-11 21:55:51,034][70582] Avg episode reward: [(0, '72.450'), (1, '99.340')] [2023-10-11 21:55:52,634][71635] Updated weights for policy 1, policy_version 72202 (0.0009) [2023-10-11 21:55:52,804][71601] Updated weights for policy 0, policy_version 72260 (0.0007) [2023-10-11 21:55:52,996][71635] Updated weights for policy 1, policy_version 72212 (0.0007) [2023-10-11 21:55:53,172][71601] Updated weights for policy 0, policy_version 72270 (0.0008) [2023-10-11 21:55:53,366][71635] Updated weights for policy 1, policy_version 72222 (0.0007) [2023-10-11 21:55:53,550][71601] Updated weights for policy 0, policy_version 72280 (0.0009) [2023-10-11 21:55:56,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 147980288. Throughput: 0: 1803.1, 1: 1810.2. Samples: 37006790. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-11 21:55:56,034][70582] Avg episode reward: [(0, '75.010'), (1, '98.050')] [2023-10-11 21:55:56,044][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000072288_74022912.pth... [2023-10-11 21:55:56,044][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000072224_73957376.pth... [2023-10-11 21:55:56,085][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000070528_72220672.pth [2023-10-11 21:55:56,086][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000070592_72286208.pth [2023-10-11 21:55:57,091][71635] Updated weights for policy 1, policy_version 72232 (0.0011) [2023-10-11 21:55:57,243][71601] Updated weights for policy 0, policy_version 72290 (0.0009) [2023-10-11 21:55:57,453][71635] Updated weights for policy 1, policy_version 72242 (0.0007) [2023-10-11 21:55:57,618][71601] Updated weights for policy 0, policy_version 72300 (0.0007) [2023-10-11 21:55:57,820][71635] Updated weights for policy 1, policy_version 72252 (0.0008) [2023-10-11 21:55:57,989][71601] Updated weights for policy 0, policy_version 72310 (0.0007) [2023-10-11 21:55:58,354][71601] Updated weights for policy 0, policy_version 72320 (0.0008) [2023-10-11 21:56:01,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 148045824. Throughput: 0: 1809.0, 1: 1811.0. Samples: 37016664. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-11 21:56:01,034][70582] Avg episode reward: [(0, '78.450'), (1, '96.520')] [2023-10-11 21:56:01,704][71635] Updated weights for policy 1, policy_version 72262 (0.0008) [2023-10-11 21:56:01,919][71601] Updated weights for policy 0, policy_version 72330 (0.0007) [2023-10-11 21:56:02,068][71635] Updated weights for policy 1, policy_version 72272 (0.0007) [2023-10-11 21:56:02,291][71601] Updated weights for policy 0, policy_version 72340 (0.0008) [2023-10-11 21:56:02,431][71635] Updated weights for policy 1, policy_version 72282 (0.0008) [2023-10-11 21:56:02,659][71601] Updated weights for policy 0, policy_version 72350 (0.0008) [2023-10-11 21:56:06,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 148111360. Throughput: 0: 1810.9, 1: 1805.4. Samples: 37039354. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-11 21:56:06,034][70582] Avg episode reward: [(0, '77.760'), (1, '98.920')] [2023-10-11 21:56:06,155][71635] Updated weights for policy 1, policy_version 72292 (0.0008) [2023-10-11 21:56:06,372][71601] Updated weights for policy 0, policy_version 72360 (0.0007) [2023-10-11 21:56:06,529][71635] Updated weights for policy 1, policy_version 72302 (0.0009) [2023-10-11 21:56:06,747][71601] Updated weights for policy 0, policy_version 72370 (0.0008) [2023-10-11 21:56:06,906][71635] Updated weights for policy 1, policy_version 72312 (0.0008) [2023-10-11 21:56:07,120][71601] Updated weights for policy 0, policy_version 72380 (0.0009) [2023-10-11 21:56:10,618][71635] Updated weights for policy 1, policy_version 72322 (0.0007) [2023-10-11 21:56:10,843][71601] Updated weights for policy 0, policy_version 72390 (0.0008) [2023-10-11 21:56:10,989][71635] Updated weights for policy 1, policy_version 72332 (0.0008) [2023-10-11 21:56:11,034][70582] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 148176896. Throughput: 0: 1810.8, 1: 1821.9. Samples: 37062152. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-11 21:56:11,035][70582] Avg episode reward: [(0, '72.550'), (1, '97.760')] [2023-10-11 21:56:11,218][71601] Updated weights for policy 0, policy_version 72400 (0.0007) [2023-10-11 21:56:11,353][71635] Updated weights for policy 1, policy_version 72342 (0.0007) [2023-10-11 21:56:11,589][71601] Updated weights for policy 0, policy_version 72410 (0.0007) [2023-10-11 21:56:11,723][71635] Updated weights for policy 1, policy_version 72352 (0.0007) [2023-10-11 21:56:15,195][71635] Updated weights for policy 1, policy_version 72362 (0.0009) [2023-10-11 21:56:15,421][71601] Updated weights for policy 0, policy_version 72420 (0.0009) [2023-10-11 21:56:15,561][71635] Updated weights for policy 1, policy_version 72372 (0.0008) [2023-10-11 21:56:15,793][71601] Updated weights for policy 0, policy_version 72430 (0.0008) [2023-10-11 21:56:15,932][71635] Updated weights for policy 1, policy_version 72382 (0.0008) [2023-10-11 21:56:16,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 148275200. Throughput: 0: 1811.0, 1: 1810.2. Samples: 37072044. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-11 21:56:16,035][70582] Avg episode reward: [(0, '74.710'), (1, '95.950')] [2023-10-11 21:56:16,167][71601] Updated weights for policy 0, policy_version 72440 (0.0008) [2023-10-11 21:56:19,507][71635] Updated weights for policy 1, policy_version 72392 (0.0009) [2023-10-11 21:56:19,884][71635] Updated weights for policy 1, policy_version 72402 (0.0010) [2023-10-11 21:56:19,991][71601] Updated weights for policy 0, policy_version 72450 (0.0008) [2023-10-11 21:56:20,251][71635] Updated weights for policy 1, policy_version 72412 (0.0007) [2023-10-11 21:56:20,370][71601] Updated weights for policy 0, policy_version 72460 (0.0009) [2023-10-11 21:56:20,734][71601] Updated weights for policy 0, policy_version 72470 (0.0008) [2023-10-11 21:56:21,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 148340736. Throughput: 0: 1807.1, 1: 1819.6. Samples: 37094634. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-11 21:56:21,035][70582] Avg episode reward: [(0, '75.760'), (1, '96.070')] [2023-10-11 21:56:21,110][71601] Updated weights for policy 0, policy_version 72480 (0.0009) [2023-10-11 21:56:24,061][71635] Updated weights for policy 1, policy_version 72422 (0.0008) [2023-10-11 21:56:24,447][71635] Updated weights for policy 1, policy_version 72432 (0.0007) [2023-10-11 21:56:24,800][71601] Updated weights for policy 0, policy_version 72490 (0.0008) [2023-10-11 21:56:24,818][71635] Updated weights for policy 1, policy_version 72442 (0.0007) [2023-10-11 21:56:25,166][71601] Updated weights for policy 0, policy_version 72500 (0.0007) [2023-10-11 21:56:25,536][71601] Updated weights for policy 0, policy_version 72510 (0.0008) [2023-10-11 21:56:26,034][70582] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14551.2). Total num frames: 148439040. Throughput: 0: 1806.9, 1: 1815.8. Samples: 37114646. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-11 21:56:26,034][70582] Avg episode reward: [(0, '74.520'), (1, '99.530')] [2023-10-11 21:56:28,571][71635] Updated weights for policy 1, policy_version 72452 (0.0008) [2023-10-11 21:56:28,934][71635] Updated weights for policy 1, policy_version 72462 (0.0007) [2023-10-11 21:56:29,301][71635] Updated weights for policy 1, policy_version 72472 (0.0007) [2023-10-11 21:56:29,330][71601] Updated weights for policy 0, policy_version 72520 (0.0008) [2023-10-11 21:56:29,706][71601] Updated weights for policy 0, policy_version 72530 (0.0009) [2023-10-11 21:56:30,083][71601] Updated weights for policy 0, policy_version 72540 (0.0008) [2023-10-11 21:56:31,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 148504576. Throughput: 0: 1799.9, 1: 1816.4. Samples: 37127042. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-11 21:56:31,035][70582] Avg episode reward: [(0, '74.450'), (1, '99.400')] [2023-10-11 21:56:32,895][71635] Updated weights for policy 1, policy_version 72482 (0.0008) [2023-10-11 21:56:33,258][71635] Updated weights for policy 1, policy_version 72492 (0.0009) [2023-10-11 21:56:33,630][71635] Updated weights for policy 1, policy_version 72502 (0.0010) [2023-10-11 21:56:33,911][71601] Updated weights for policy 0, policy_version 72550 (0.0008) [2023-10-11 21:56:33,988][71635] Updated weights for policy 1, policy_version 72512 (0.0007) [2023-10-11 21:56:34,282][71601] Updated weights for policy 0, policy_version 72560 (0.0010) [2023-10-11 21:56:34,666][71601] Updated weights for policy 0, policy_version 72570 (0.0007) [2023-10-11 21:56:36,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 148570112. Throughput: 0: 1807.0, 1: 1817.1. Samples: 37147036. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-11 21:56:36,034][70582] Avg episode reward: [(0, '74.650'), (1, '99.010')] [2023-10-11 21:56:37,777][71635] Updated weights for policy 1, policy_version 72522 (0.0011) [2023-10-11 21:56:38,149][71635] Updated weights for policy 1, policy_version 72532 (0.0008) [2023-10-11 21:56:38,395][71601] Updated weights for policy 0, policy_version 72580 (0.0007) [2023-10-11 21:56:38,510][71635] Updated weights for policy 1, policy_version 72542 (0.0008) [2023-10-11 21:56:38,763][71601] Updated weights for policy 0, policy_version 72590 (0.0007) [2023-10-11 21:56:39,139][71601] Updated weights for policy 0, policy_version 72600 (0.0009) [2023-10-11 21:56:41,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 148635648. Throughput: 0: 1791.5, 1: 1804.7. Samples: 37168618. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-11 21:56:41,034][70582] Avg episode reward: [(0, '72.120'), (1, '101.360')] [2023-10-11 21:56:42,218][71635] Updated weights for policy 1, policy_version 72552 (0.0009) [2023-10-11 21:56:42,581][71635] Updated weights for policy 1, policy_version 72562 (0.0008) [2023-10-11 21:56:42,945][71635] Updated weights for policy 1, policy_version 72572 (0.0008) [2023-10-11 21:56:42,951][71601] Updated weights for policy 0, policy_version 72610 (0.0008) [2023-10-11 21:56:43,310][71601] Updated weights for policy 0, policy_version 72620 (0.0008) [2023-10-11 21:56:43,676][71601] Updated weights for policy 0, policy_version 72630 (0.0010) [2023-10-11 21:56:44,044][71601] Updated weights for policy 0, policy_version 72640 (0.0010) [2023-10-11 21:56:46,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 148701184. Throughput: 0: 1804.9, 1: 1811.2. Samples: 37179386. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-11 21:56:46,034][70582] Avg episode reward: [(0, '69.650'), (1, '102.450')] [2023-10-11 21:56:46,726][71635] Updated weights for policy 1, policy_version 72582 (0.0008) [2023-10-11 21:56:47,100][71635] Updated weights for policy 1, policy_version 72592 (0.0008) [2023-10-11 21:56:47,456][71635] Updated weights for policy 1, policy_version 72602 (0.0008) [2023-10-11 21:56:47,681][71601] Updated weights for policy 0, policy_version 72650 (0.0008) [2023-10-11 21:56:48,061][71601] Updated weights for policy 0, policy_version 72660 (0.0009) [2023-10-11 21:56:48,428][71601] Updated weights for policy 0, policy_version 72670 (0.0008) [2023-10-11 21:56:51,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 148766720. Throughput: 0: 1789.1, 1: 1815.2. Samples: 37201548. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-11 21:56:51,034][70582] Avg episode reward: [(0, '69.150'), (1, '102.120')] [2023-10-11 21:56:51,305][71635] Updated weights for policy 1, policy_version 72612 (0.0007) [2023-10-11 21:56:51,671][71635] Updated weights for policy 1, policy_version 72622 (0.0008) [2023-10-11 21:56:52,036][71635] Updated weights for policy 1, policy_version 72632 (0.0009) [2023-10-11 21:56:52,188][71601] Updated weights for policy 0, policy_version 72680 (0.0007) [2023-10-11 21:56:52,570][71601] Updated weights for policy 0, policy_version 72690 (0.0007) [2023-10-11 21:56:52,938][71601] Updated weights for policy 0, policy_version 72700 (0.0009) [2023-10-11 21:56:55,722][71635] Updated weights for policy 1, policy_version 72642 (0.0009) [2023-10-11 21:56:56,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 148832256. Throughput: 0: 1789.8, 1: 1809.3. Samples: 37224108. Policy #0 lag: (min: 27.0, avg: 33.9, max: 59.0) [2023-10-11 21:56:56,034][70582] Avg episode reward: [(0, '69.670'), (1, '99.990')] [2023-10-11 21:56:56,094][71635] Updated weights for policy 1, policy_version 72652 (0.0010) [2023-10-11 21:56:56,460][71635] Updated weights for policy 1, policy_version 72662 (0.0009) [2023-10-11 21:56:56,512][71601] Updated weights for policy 0, policy_version 72710 (0.0007) [2023-10-11 21:56:56,820][71635] Updated weights for policy 1, policy_version 72672 (0.0008) [2023-10-11 21:56:56,882][71601] Updated weights for policy 0, policy_version 72720 (0.0007) [2023-10-11 21:56:57,246][71601] Updated weights for policy 0, policy_version 72730 (0.0008) [2023-10-11 21:57:00,375][71635] Updated weights for policy 1, policy_version 72682 (0.0011) [2023-10-11 21:57:00,746][71635] Updated weights for policy 1, policy_version 72692 (0.0008) [2023-10-11 21:57:01,028][71601] Updated weights for policy 0, policy_version 72740 (0.0007) [2023-10-11 21:57:01,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 148897792. Throughput: 0: 1797.4, 1: 1807.5. Samples: 37234264. Policy #0 lag: (min: 27.0, avg: 33.9, max: 59.0) [2023-10-11 21:57:01,034][70582] Avg episode reward: [(0, '71.110'), (1, '102.090')] [2023-10-11 21:57:01,114][71635] Updated weights for policy 1, policy_version 72702 (0.0008) [2023-10-11 21:57:01,402][71601] Updated weights for policy 0, policy_version 72750 (0.0007) [2023-10-11 21:57:01,768][71601] Updated weights for policy 0, policy_version 72760 (0.0008) [2023-10-11 21:57:04,826][71635] Updated weights for policy 1, policy_version 72712 (0.0009) [2023-10-11 21:57:05,189][71635] Updated weights for policy 1, policy_version 72722 (0.0008) [2023-10-11 21:57:05,523][71601] Updated weights for policy 0, policy_version 72770 (0.0008) [2023-10-11 21:57:05,564][71635] Updated weights for policy 1, policy_version 72732 (0.0009) [2023-10-11 21:57:05,905][71601] Updated weights for policy 0, policy_version 72780 (0.0010) [2023-10-11 21:57:06,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 148996096. Throughput: 0: 1793.5, 1: 1810.5. Samples: 37256816. Policy #0 lag: (min: 27.0, avg: 33.9, max: 59.0) [2023-10-11 21:57:06,034][70582] Avg episode reward: [(0, '76.380'), (1, '110.070')] [2023-10-11 21:57:06,272][71601] Updated weights for policy 0, policy_version 72790 (0.0009) [2023-10-11 21:57:06,644][71601] Updated weights for policy 0, policy_version 72800 (0.0008) [2023-10-11 21:57:09,134][71635] Updated weights for policy 1, policy_version 72742 (0.0007) [2023-10-11 21:57:09,507][71635] Updated weights for policy 1, policy_version 72752 (0.0011) [2023-10-11 21:57:09,878][71635] Updated weights for policy 1, policy_version 72762 (0.0010) [2023-10-11 21:57:10,293][71601] Updated weights for policy 0, policy_version 72810 (0.0009) [2023-10-11 21:57:10,666][71601] Updated weights for policy 0, policy_version 72820 (0.0011) [2023-10-11 21:57:11,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 149061632. Throughput: 0: 1811.1, 1: 1808.7. Samples: 37277538. Policy #0 lag: (min: 27.0, avg: 33.9, max: 59.0) [2023-10-11 21:57:11,035][70582] Avg episode reward: [(0, '75.380'), (1, '107.930')] [2023-10-11 21:57:11,036][71601] Updated weights for policy 0, policy_version 72830 (0.0010) [2023-10-11 21:57:13,503][71635] Updated weights for policy 1, policy_version 72772 (0.0008) [2023-10-11 21:57:13,864][71635] Updated weights for policy 1, policy_version 72782 (0.0010) [2023-10-11 21:57:14,228][71635] Updated weights for policy 1, policy_version 72792 (0.0011) [2023-10-11 21:57:14,810][71601] Updated weights for policy 0, policy_version 72840 (0.0008) [2023-10-11 21:57:15,184][71601] Updated weights for policy 0, policy_version 72850 (0.0008) [2023-10-11 21:57:15,551][71601] Updated weights for policy 0, policy_version 72860 (0.0008) [2023-10-11 21:57:16,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 149159936. Throughput: 0: 1798.3, 1: 1814.0. Samples: 37289598. Policy #0 lag: (min: 27.0, avg: 33.9, max: 59.0) [2023-10-11 21:57:16,035][70582] Avg episode reward: [(0, '74.150'), (1, '108.310')] [2023-10-11 21:57:17,958][71635] Updated weights for policy 1, policy_version 72802 (0.0009) [2023-10-11 21:57:18,330][71635] Updated weights for policy 1, policy_version 72812 (0.0010) [2023-10-11 21:57:18,700][71635] Updated weights for policy 1, policy_version 72822 (0.0008) [2023-10-11 21:57:19,061][71635] Updated weights for policy 1, policy_version 72832 (0.0007) [2023-10-11 21:57:19,219][71601] Updated weights for policy 0, policy_version 72870 (0.0009) [2023-10-11 21:57:19,585][71601] Updated weights for policy 0, policy_version 72880 (0.0010) [2023-10-11 21:57:19,957][71601] Updated weights for policy 0, policy_version 72890 (0.0010) [2023-10-11 21:57:21,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 149225472. Throughput: 0: 1811.5, 1: 1817.2. Samples: 37310328. Policy #0 lag: (min: 27.0, avg: 33.9, max: 59.0) [2023-10-11 21:57:21,035][70582] Avg episode reward: [(0, '77.130'), (1, '107.610')] [2023-10-11 21:57:22,702][71635] Updated weights for policy 1, policy_version 72842 (0.0008) [2023-10-11 21:57:23,061][71635] Updated weights for policy 1, policy_version 72852 (0.0007) [2023-10-11 21:57:23,428][71635] Updated weights for policy 1, policy_version 72862 (0.0007) [2023-10-11 21:57:23,664][71601] Updated weights for policy 0, policy_version 72900 (0.0010) [2023-10-11 21:57:24,046][71601] Updated weights for policy 0, policy_version 72910 (0.0010) [2023-10-11 21:57:24,412][71601] Updated weights for policy 0, policy_version 72920 (0.0008) [2023-10-11 21:57:26,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 149291008. Throughput: 0: 1805.5, 1: 1824.2. Samples: 37331952. Policy #0 lag: (min: 27.0, avg: 33.9, max: 59.0) [2023-10-11 21:57:26,035][70582] Avg episode reward: [(0, '79.960'), (1, '110.810')] [2023-10-11 21:57:27,241][71635] Updated weights for policy 1, policy_version 72872 (0.0011) [2023-10-11 21:57:27,598][71635] Updated weights for policy 1, policy_version 72882 (0.0011) [2023-10-11 21:57:27,969][71635] Updated weights for policy 1, policy_version 72892 (0.0010) [2023-10-11 21:57:28,125][71601] Updated weights for policy 0, policy_version 72930 (0.0007) [2023-10-11 21:57:28,496][71601] Updated weights for policy 0, policy_version 72940 (0.0008) [2023-10-11 21:57:28,865][71601] Updated weights for policy 0, policy_version 72950 (0.0009) [2023-10-11 21:57:29,237][71601] Updated weights for policy 0, policy_version 72960 (0.0010) [2023-10-11 21:57:31,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 149356544. Throughput: 0: 1812.4, 1: 1819.2. Samples: 37342810. Policy #0 lag: (min: 27.0, avg: 33.9, max: 59.0) [2023-10-11 21:57:31,034][70582] Avg episode reward: [(0, '83.280'), (1, '110.170')] [2023-10-11 21:57:31,674][71635] Updated weights for policy 1, policy_version 72902 (0.0008) [2023-10-11 21:57:32,039][71635] Updated weights for policy 1, policy_version 72912 (0.0008) [2023-10-11 21:57:32,412][71635] Updated weights for policy 1, policy_version 72922 (0.0010) [2023-10-11 21:57:33,137][71601] Updated weights for policy 0, policy_version 72970 (0.0007) [2023-10-11 21:57:33,507][71601] Updated weights for policy 0, policy_version 72980 (0.0007) [2023-10-11 21:57:33,881][71601] Updated weights for policy 0, policy_version 72990 (0.0007) [2023-10-11 21:57:36,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 149422080. Throughput: 0: 1798.2, 1: 1820.9. Samples: 37364410. Policy #0 lag: (min: 27.0, avg: 33.9, max: 59.0) [2023-10-11 21:57:36,034][70582] Avg episode reward: [(0, '83.930'), (1, '107.550')] [2023-10-11 21:57:36,173][71635] Updated weights for policy 1, policy_version 72932 (0.0008) [2023-10-11 21:57:36,532][71635] Updated weights for policy 1, policy_version 72942 (0.0008) [2023-10-11 21:57:36,905][71635] Updated weights for policy 1, policy_version 72952 (0.0009) [2023-10-11 21:57:37,630][71601] Updated weights for policy 0, policy_version 73000 (0.0008) [2023-10-11 21:57:38,011][71601] Updated weights for policy 0, policy_version 73010 (0.0008) [2023-10-11 21:57:38,379][71601] Updated weights for policy 0, policy_version 73020 (0.0007) [2023-10-11 21:57:40,646][71635] Updated weights for policy 1, policy_version 72962 (0.0007) [2023-10-11 21:57:41,015][71635] Updated weights for policy 1, policy_version 72972 (0.0008) [2023-10-11 21:57:41,034][70582] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 149487616. Throughput: 0: 1799.7, 1: 1824.5. Samples: 37387198. Policy #0 lag: (min: 25.0, avg: 30.0, max: 57.0) [2023-10-11 21:57:41,035][70582] Avg episode reward: [(0, '90.990'), (1, '110.090')] [2023-10-11 21:57:41,377][71635] Updated weights for policy 1, policy_version 72982 (0.0008) [2023-10-11 21:57:41,747][71635] Updated weights for policy 1, policy_version 72992 (0.0007) [2023-10-11 21:57:42,090][71601] Updated weights for policy 0, policy_version 73030 (0.0010) [2023-10-11 21:57:42,455][71601] Updated weights for policy 0, policy_version 73040 (0.0009) [2023-10-11 21:57:42,827][71601] Updated weights for policy 0, policy_version 73050 (0.0008) [2023-10-11 21:57:45,460][71635] Updated weights for policy 1, policy_version 73002 (0.0008) [2023-10-11 21:57:45,823][71635] Updated weights for policy 1, policy_version 73012 (0.0008) [2023-10-11 21:57:46,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 149553152. Throughput: 0: 1795.9, 1: 1825.4. Samples: 37397222. Policy #0 lag: (min: 25.0, avg: 30.0, max: 57.0) [2023-10-11 21:57:46,034][70582] Avg episode reward: [(0, '88.250'), (1, '108.610')] [2023-10-11 21:57:46,188][71635] Updated weights for policy 1, policy_version 73022 (0.0009) [2023-10-11 21:57:46,350][71601] Updated weights for policy 0, policy_version 73060 (0.0008) [2023-10-11 21:57:46,726][71601] Updated weights for policy 0, policy_version 73070 (0.0009) [2023-10-11 21:57:47,100][71601] Updated weights for policy 0, policy_version 73080 (0.0008) [2023-10-11 21:57:49,949][71635] Updated weights for policy 1, policy_version 73032 (0.0009) [2023-10-11 21:57:50,306][71635] Updated weights for policy 1, policy_version 73042 (0.0009) [2023-10-11 21:57:50,645][71601] Updated weights for policy 0, policy_version 73090 (0.0008) [2023-10-11 21:57:50,674][71635] Updated weights for policy 1, policy_version 73052 (0.0008) [2023-10-11 21:57:51,017][71601] Updated weights for policy 0, policy_version 73100 (0.0009) [2023-10-11 21:57:51,034][70582] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 149651456. Throughput: 0: 1807.2, 1: 1822.0. Samples: 37420132. Policy #0 lag: (min: 25.0, avg: 30.0, max: 57.0) [2023-10-11 21:57:51,034][70582] Avg episode reward: [(0, '86.320'), (1, '108.720')] [2023-10-11 21:57:51,387][71601] Updated weights for policy 0, policy_version 73110 (0.0009) [2023-10-11 21:57:51,763][71601] Updated weights for policy 0, policy_version 73120 (0.0008) [2023-10-11 21:57:54,500][71635] Updated weights for policy 1, policy_version 73062 (0.0007) [2023-10-11 21:57:54,893][71635] Updated weights for policy 1, policy_version 73072 (0.0007) [2023-10-11 21:57:55,247][71635] Updated weights for policy 1, policy_version 73082 (0.0007) [2023-10-11 21:57:55,432][71601] Updated weights for policy 0, policy_version 73130 (0.0009) [2023-10-11 21:57:55,814][71601] Updated weights for policy 0, policy_version 73140 (0.0009) [2023-10-11 21:57:56,034][70582] Fps is (10 sec: 16383.4, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 149716992. Throughput: 0: 1811.9, 1: 1818.7. Samples: 37440918. Policy #0 lag: (min: 25.0, avg: 30.0, max: 57.0) [2023-10-11 21:57:56,035][70582] Avg episode reward: [(0, '84.400'), (1, '105.970')] [2023-10-11 21:57:56,047][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000073088_74842112.pth... [2023-10-11 21:57:56,081][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000071392_73105408.pth [2023-10-11 21:57:56,174][71601] Updated weights for policy 0, policy_version 73150 (0.0010) [2023-10-11 21:57:56,245][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000073152_74907648.pth... [2023-10-11 21:57:56,282][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000071456_73170944.pth [2023-10-11 21:57:58,978][71635] Updated weights for policy 1, policy_version 73092 (0.0008) [2023-10-11 21:57:59,349][71635] Updated weights for policy 1, policy_version 73102 (0.0010) [2023-10-11 21:57:59,721][71635] Updated weights for policy 1, policy_version 73112 (0.0010) [2023-10-11 21:57:59,986][71601] Updated weights for policy 0, policy_version 73160 (0.0008) [2023-10-11 21:58:00,356][71601] Updated weights for policy 0, policy_version 73170 (0.0008) [2023-10-11 21:58:00,726][71601] Updated weights for policy 0, policy_version 73180 (0.0008) [2023-10-11 21:58:01,034][70582] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 149815296. Throughput: 0: 1809.6, 1: 1808.9. Samples: 37452432. Policy #0 lag: (min: 25.0, avg: 30.0, max: 57.0) [2023-10-11 21:58:01,034][70582] Avg episode reward: [(0, '82.940'), (1, '109.050')] [2023-10-11 21:58:03,375][71635] Updated weights for policy 1, policy_version 73122 (0.0009) [2023-10-11 21:58:03,735][71635] Updated weights for policy 1, policy_version 73132 (0.0007) [2023-10-11 21:58:04,101][71635] Updated weights for policy 1, policy_version 73142 (0.0008) [2023-10-11 21:58:04,418][71601] Updated weights for policy 0, policy_version 73190 (0.0007) [2023-10-11 21:58:04,467][71635] Updated weights for policy 1, policy_version 73152 (0.0009) [2023-10-11 21:58:04,780][71601] Updated weights for policy 0, policy_version 73200 (0.0008) [2023-10-11 21:58:05,161][71601] Updated weights for policy 0, policy_version 73210 (0.0009) [2023-10-11 21:58:06,034][70582] Fps is (10 sec: 16384.4, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 149880832. Throughput: 0: 1817.9, 1: 1810.7. Samples: 37473614. Policy #0 lag: (min: 25.0, avg: 30.0, max: 57.0) [2023-10-11 21:58:06,035][70582] Avg episode reward: [(0, '84.880'), (1, '110.240')] [2023-10-11 21:58:08,081][71635] Updated weights for policy 1, policy_version 73162 (0.0007) [2023-10-11 21:58:08,455][71635] Updated weights for policy 1, policy_version 73172 (0.0009) [2023-10-11 21:58:08,771][71601] Updated weights for policy 0, policy_version 73220 (0.0009) [2023-10-11 21:58:08,811][71635] Updated weights for policy 1, policy_version 73182 (0.0008) [2023-10-11 21:58:09,151][71601] Updated weights for policy 0, policy_version 73230 (0.0009) [2023-10-11 21:58:09,520][71601] Updated weights for policy 0, policy_version 73240 (0.0008) [2023-10-11 21:58:11,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 149946368. Throughput: 0: 1818.8, 1: 1814.1. Samples: 37495436. Policy #0 lag: (min: 25.0, avg: 30.0, max: 57.0) [2023-10-11 21:58:11,035][70582] Avg episode reward: [(0, '79.580'), (1, '114.290')] [2023-10-11 21:58:12,437][71635] Updated weights for policy 1, policy_version 73192 (0.0009) [2023-10-11 21:58:12,791][71635] Updated weights for policy 1, policy_version 73202 (0.0009) [2023-10-11 21:58:13,111][71601] Updated weights for policy 0, policy_version 73250 (0.0008) [2023-10-11 21:58:13,160][71635] Updated weights for policy 1, policy_version 73212 (0.0008) [2023-10-11 21:58:13,473][71601] Updated weights for policy 0, policy_version 73260 (0.0007) [2023-10-11 21:58:13,857][71601] Updated weights for policy 0, policy_version 73270 (0.0008) [2023-10-11 21:58:14,215][71601] Updated weights for policy 0, policy_version 73280 (0.0009) [2023-10-11 21:58:16,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 150011904. Throughput: 0: 1823.4, 1: 1814.4. Samples: 37506514. Policy #0 lag: (min: 25.0, avg: 30.0, max: 57.0) [2023-10-11 21:58:16,035][70582] Avg episode reward: [(0, '76.940'), (1, '113.410')] [2023-10-11 21:58:16,875][71635] Updated weights for policy 1, policy_version 73222 (0.0009) [2023-10-11 21:58:17,244][71635] Updated weights for policy 1, policy_version 73232 (0.0011) [2023-10-11 21:58:17,606][71635] Updated weights for policy 1, policy_version 73242 (0.0010) [2023-10-11 21:58:17,895][71601] Updated weights for policy 0, policy_version 73290 (0.0008) [2023-10-11 21:58:18,267][71601] Updated weights for policy 0, policy_version 73300 (0.0007) [2023-10-11 21:58:18,644][71601] Updated weights for policy 0, policy_version 73310 (0.0009) [2023-10-11 21:58:21,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 150077440. Throughput: 0: 1831.6, 1: 1810.4. Samples: 37528298. Policy #0 lag: (min: 25.0, avg: 30.0, max: 57.0) [2023-10-11 21:58:21,034][70582] Avg episode reward: [(0, '70.580'), (1, '112.490')] [2023-10-11 21:58:21,286][71635] Updated weights for policy 1, policy_version 73252 (0.0008) [2023-10-11 21:58:21,658][71635] Updated weights for policy 1, policy_version 73262 (0.0007) [2023-10-11 21:58:22,037][71635] Updated weights for policy 1, policy_version 73272 (0.0010) [2023-10-11 21:58:22,317][71601] Updated weights for policy 0, policy_version 73320 (0.0008) [2023-10-11 21:58:22,697][71601] Updated weights for policy 0, policy_version 73330 (0.0007) [2023-10-11 21:58:23,070][71601] Updated weights for policy 0, policy_version 73340 (0.0009) [2023-10-11 21:58:25,736][71635] Updated weights for policy 1, policy_version 73282 (0.0008) [2023-10-11 21:58:26,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 150142976. Throughput: 0: 1828.8, 1: 1812.7. Samples: 37551068. Policy #0 lag: (min: 25.0, avg: 30.0, max: 57.0) [2023-10-11 21:58:26,034][70582] Avg episode reward: [(0, '66.860'), (1, '110.770')] [2023-10-11 21:58:26,106][71635] Updated weights for policy 1, policy_version 73292 (0.0009) [2023-10-11 21:58:26,469][71635] Updated weights for policy 1, policy_version 73302 (0.0007) [2023-10-11 21:58:26,828][71601] Updated weights for policy 0, policy_version 73350 (0.0008) [2023-10-11 21:58:26,835][71635] Updated weights for policy 1, policy_version 73312 (0.0009) [2023-10-11 21:58:27,196][71601] Updated weights for policy 0, policy_version 73360 (0.0008) [2023-10-11 21:58:27,560][71601] Updated weights for policy 0, policy_version 73370 (0.0009) [2023-10-11 21:58:30,473][71635] Updated weights for policy 1, policy_version 73322 (0.0011) [2023-10-11 21:58:30,846][71635] Updated weights for policy 1, policy_version 73332 (0.0009) [2023-10-11 21:58:31,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 150208512. Throughput: 0: 1826.9, 1: 1813.9. Samples: 37561058. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:58:31,035][70582] Avg episode reward: [(0, '70.190'), (1, '108.490')] [2023-10-11 21:58:31,209][71635] Updated weights for policy 1, policy_version 73342 (0.0008) [2023-10-11 21:58:31,311][71601] Updated weights for policy 0, policy_version 73380 (0.0008) [2023-10-11 21:58:31,671][71601] Updated weights for policy 0, policy_version 73390 (0.0010) [2023-10-11 21:58:32,047][71601] Updated weights for policy 0, policy_version 73400 (0.0011) [2023-10-11 21:58:34,895][71635] Updated weights for policy 1, policy_version 73352 (0.0011) [2023-10-11 21:58:35,255][71635] Updated weights for policy 1, policy_version 73362 (0.0008) [2023-10-11 21:58:35,620][71635] Updated weights for policy 1, policy_version 73372 (0.0007) [2023-10-11 21:58:35,638][71601] Updated weights for policy 0, policy_version 73410 (0.0009) [2023-10-11 21:58:36,009][71601] Updated weights for policy 0, policy_version 73420 (0.0008) [2023-10-11 21:58:36,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 150306816. Throughput: 0: 1821.0, 1: 1809.6. Samples: 37583510. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:58:36,035][70582] Avg episode reward: [(0, '73.990'), (1, '108.210')] [2023-10-11 21:58:36,385][71601] Updated weights for policy 0, policy_version 73430 (0.0009) [2023-10-11 21:58:36,755][71601] Updated weights for policy 0, policy_version 73440 (0.0009) [2023-10-11 21:58:39,400][71635] Updated weights for policy 1, policy_version 73382 (0.0008) [2023-10-11 21:58:39,775][71635] Updated weights for policy 1, policy_version 73392 (0.0009) [2023-10-11 21:58:40,149][71635] Updated weights for policy 1, policy_version 73402 (0.0008) [2023-10-11 21:58:40,407][71601] Updated weights for policy 0, policy_version 73450 (0.0009) [2023-10-11 21:58:40,786][71601] Updated weights for policy 0, policy_version 73460 (0.0007) [2023-10-11 21:58:41,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 150372352. Throughput: 0: 1821.7, 1: 1807.4. Samples: 37604230. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:58:41,035][70582] Avg episode reward: [(0, '73.440'), (1, '114.770')] [2023-10-11 21:58:41,149][71601] Updated weights for policy 0, policy_version 73470 (0.0008) [2023-10-11 21:58:43,819][71635] Updated weights for policy 1, policy_version 73412 (0.0009) [2023-10-11 21:58:44,182][71635] Updated weights for policy 1, policy_version 73422 (0.0008) [2023-10-11 21:58:44,549][71635] Updated weights for policy 1, policy_version 73432 (0.0010) [2023-10-11 21:58:44,725][71601] Updated weights for policy 0, policy_version 73480 (0.0008) [2023-10-11 21:58:45,087][71601] Updated weights for policy 0, policy_version 73490 (0.0008) [2023-10-11 21:58:45,468][71601] Updated weights for policy 0, policy_version 73500 (0.0008) [2023-10-11 21:58:46,034][70582] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 150470656. Throughput: 0: 1830.8, 1: 1815.0. Samples: 37616494. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:58:46,035][70582] Avg episode reward: [(0, '77.290'), (1, '121.090')] [2023-10-11 21:58:48,337][71635] Updated weights for policy 1, policy_version 73442 (0.0008) [2023-10-11 21:58:48,711][71635] Updated weights for policy 1, policy_version 73452 (0.0007) [2023-10-11 21:58:49,073][71635] Updated weights for policy 1, policy_version 73462 (0.0008) [2023-10-11 21:58:49,124][71601] Updated weights for policy 0, policy_version 73510 (0.0008) [2023-10-11 21:58:49,442][71635] Updated weights for policy 1, policy_version 73472 (0.0010) [2023-10-11 21:58:49,493][71601] Updated weights for policy 0, policy_version 73520 (0.0009) [2023-10-11 21:58:49,868][71601] Updated weights for policy 0, policy_version 73530 (0.0009) [2023-10-11 21:58:51,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 150536192. Throughput: 0: 1825.3, 1: 1811.0. Samples: 37637248. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:58:51,035][70582] Avg episode reward: [(0, '77.590'), (1, '120.770')] [2023-10-11 21:58:53,216][71635] Updated weights for policy 1, policy_version 73482 (0.0010) [2023-10-11 21:58:53,590][71635] Updated weights for policy 1, policy_version 73492 (0.0011) [2023-10-11 21:58:53,671][71601] Updated weights for policy 0, policy_version 73540 (0.0008) [2023-10-11 21:58:53,957][71635] Updated weights for policy 1, policy_version 73502 (0.0008) [2023-10-11 21:58:54,049][71601] Updated weights for policy 0, policy_version 73550 (0.0008) [2023-10-11 21:58:54,416][71601] Updated weights for policy 0, policy_version 73560 (0.0007) [2023-10-11 21:58:56,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 150601728. Throughput: 0: 1822.0, 1: 1807.2. Samples: 37658752. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:58:56,035][70582] Avg episode reward: [(0, '77.560'), (1, '114.930')] [2023-10-11 21:58:57,576][71635] Updated weights for policy 1, policy_version 73512 (0.0008) [2023-10-11 21:58:57,939][71635] Updated weights for policy 1, policy_version 73522 (0.0009) [2023-10-11 21:58:58,244][71601] Updated weights for policy 0, policy_version 73570 (0.0008) [2023-10-11 21:58:58,304][71635] Updated weights for policy 1, policy_version 73532 (0.0007) [2023-10-11 21:58:58,608][71601] Updated weights for policy 0, policy_version 73580 (0.0008) [2023-10-11 21:58:58,977][71601] Updated weights for policy 0, policy_version 73590 (0.0008) [2023-10-11 21:58:59,349][71601] Updated weights for policy 0, policy_version 73600 (0.0009) [2023-10-11 21:59:01,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 150667264. Throughput: 0: 1819.2, 1: 1816.5. Samples: 37670118. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:59:01,034][70582] Avg episode reward: [(0, '73.490'), (1, '115.260')] [2023-10-11 21:59:02,056][71635] Updated weights for policy 1, policy_version 73542 (0.0007) [2023-10-11 21:59:02,431][71635] Updated weights for policy 1, policy_version 73552 (0.0008) [2023-10-11 21:59:02,796][71635] Updated weights for policy 1, policy_version 73562 (0.0009) [2023-10-11 21:59:03,155][71601] Updated weights for policy 0, policy_version 73610 (0.0007) [2023-10-11 21:59:03,529][71601] Updated weights for policy 0, policy_version 73620 (0.0007) [2023-10-11 21:59:03,904][71601] Updated weights for policy 0, policy_version 73630 (0.0009) [2023-10-11 21:59:06,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 150732800. Throughput: 0: 1811.0, 1: 1814.5. Samples: 37691446. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:59:06,035][70582] Avg episode reward: [(0, '74.170'), (1, '111.790')] [2023-10-11 21:59:06,460][71635] Updated weights for policy 1, policy_version 73572 (0.0008) [2023-10-11 21:59:06,821][71635] Updated weights for policy 1, policy_version 73582 (0.0008) [2023-10-11 21:59:07,194][71635] Updated weights for policy 1, policy_version 73592 (0.0007) [2023-10-11 21:59:07,659][71601] Updated weights for policy 0, policy_version 73640 (0.0009) [2023-10-11 21:59:08,036][71601] Updated weights for policy 0, policy_version 73650 (0.0010) [2023-10-11 21:59:08,397][71601] Updated weights for policy 0, policy_version 73660 (0.0008) [2023-10-11 21:59:10,820][71635] Updated weights for policy 1, policy_version 73602 (0.0007) [2023-10-11 21:59:11,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 150798336. Throughput: 0: 1820.7, 1: 1812.3. Samples: 37714554. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:59:11,034][70582] Avg episode reward: [(0, '77.140'), (1, '110.100')] [2023-10-11 21:59:11,182][71635] Updated weights for policy 1, policy_version 73612 (0.0008) [2023-10-11 21:59:11,558][71635] Updated weights for policy 1, policy_version 73622 (0.0008) [2023-10-11 21:59:11,924][71635] Updated weights for policy 1, policy_version 73632 (0.0009) [2023-10-11 21:59:11,950][71601] Updated weights for policy 0, policy_version 73670 (0.0007) [2023-10-11 21:59:12,316][71601] Updated weights for policy 0, policy_version 73680 (0.0007) [2023-10-11 21:59:12,684][71601] Updated weights for policy 0, policy_version 73690 (0.0008) [2023-10-11 21:59:15,585][71635] Updated weights for policy 1, policy_version 73642 (0.0008) [2023-10-11 21:59:15,954][71635] Updated weights for policy 1, policy_version 73652 (0.0011) [2023-10-11 21:59:16,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 150863872. Throughput: 0: 1825.1, 1: 1809.8. Samples: 37724626. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 21:59:16,034][70582] Avg episode reward: [(0, '77.250'), (1, '109.200')] [2023-10-11 21:59:16,324][71635] Updated weights for policy 1, policy_version 73662 (0.0008) [2023-10-11 21:59:16,332][71601] Updated weights for policy 0, policy_version 73700 (0.0010) [2023-10-11 21:59:16,707][71601] Updated weights for policy 0, policy_version 73710 (0.0009) [2023-10-11 21:59:17,085][71601] Updated weights for policy 0, policy_version 73720 (0.0009) [2023-10-11 21:59:19,984][71635] Updated weights for policy 1, policy_version 73672 (0.0009) [2023-10-11 21:59:20,349][71635] Updated weights for policy 1, policy_version 73682 (0.0011) [2023-10-11 21:59:20,717][71635] Updated weights for policy 1, policy_version 73692 (0.0010) [2023-10-11 21:59:20,782][71601] Updated weights for policy 0, policy_version 73730 (0.0007) [2023-10-11 21:59:21,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 150962176. Throughput: 0: 1826.6, 1: 1819.5. Samples: 37747586. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-11 21:59:21,034][70582] Avg episode reward: [(0, '74.680'), (1, '114.830')] [2023-10-11 21:59:21,141][71601] Updated weights for policy 0, policy_version 73740 (0.0010) [2023-10-11 21:59:21,518][71601] Updated weights for policy 0, policy_version 73750 (0.0008) [2023-10-11 21:59:21,890][71601] Updated weights for policy 0, policy_version 73760 (0.0008) [2023-10-11 21:59:24,608][71635] Updated weights for policy 1, policy_version 73702 (0.0009) [2023-10-11 21:59:24,988][71635] Updated weights for policy 1, policy_version 73712 (0.0007) [2023-10-11 21:59:25,355][71635] Updated weights for policy 1, policy_version 73722 (0.0008) [2023-10-11 21:59:25,588][71601] Updated weights for policy 0, policy_version 73770 (0.0009) [2023-10-11 21:59:25,963][71601] Updated weights for policy 0, policy_version 73780 (0.0009) [2023-10-11 21:59:26,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 151027712. Throughput: 0: 1828.5, 1: 1828.9. Samples: 37768816. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-11 21:59:26,034][70582] Avg episode reward: [(0, '69.920'), (1, '121.440')] [2023-10-11 21:59:26,336][71601] Updated weights for policy 0, policy_version 73790 (0.0010) [2023-10-11 21:59:28,961][71635] Updated weights for policy 1, policy_version 73732 (0.0007) [2023-10-11 21:59:29,326][71635] Updated weights for policy 1, policy_version 73742 (0.0009) [2023-10-11 21:59:29,695][71635] Updated weights for policy 1, policy_version 73752 (0.0008) [2023-10-11 21:59:29,950][71601] Updated weights for policy 0, policy_version 73800 (0.0008) [2023-10-11 21:59:30,320][71601] Updated weights for policy 0, policy_version 73810 (0.0010) [2023-10-11 21:59:30,689][71601] Updated weights for policy 0, policy_version 73820 (0.0008) [2023-10-11 21:59:31,034][70582] Fps is (10 sec: 16383.8, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 151126016. Throughput: 0: 1819.2, 1: 1822.8. Samples: 37780384. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-11 21:59:31,034][70582] Avg episode reward: [(0, '70.930'), (1, '118.550')] [2023-10-11 21:59:33,458][71635] Updated weights for policy 1, policy_version 73762 (0.0008) [2023-10-11 21:59:33,822][71635] Updated weights for policy 1, policy_version 73772 (0.0010) [2023-10-11 21:59:34,194][71635] Updated weights for policy 1, policy_version 73782 (0.0009) [2023-10-11 21:59:34,354][71601] Updated weights for policy 0, policy_version 73830 (0.0007) [2023-10-11 21:59:34,559][71635] Updated weights for policy 1, policy_version 73792 (0.0007) [2023-10-11 21:59:34,717][71601] Updated weights for policy 0, policy_version 73840 (0.0007) [2023-10-11 21:59:35,102][71601] Updated weights for policy 0, policy_version 73850 (0.0007) [2023-10-11 21:59:36,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 151191552. Throughput: 0: 1826.2, 1: 1826.5. Samples: 37801618. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-11 21:59:36,034][70582] Avg episode reward: [(0, '68.550'), (1, '109.870')] [2023-10-11 21:59:38,219][71635] Updated weights for policy 1, policy_version 73802 (0.0007) [2023-10-11 21:59:38,589][71635] Updated weights for policy 1, policy_version 73812 (0.0007) [2023-10-11 21:59:38,709][71601] Updated weights for policy 0, policy_version 73860 (0.0007) [2023-10-11 21:59:38,950][71635] Updated weights for policy 1, policy_version 73822 (0.0009) [2023-10-11 21:59:39,072][71601] Updated weights for policy 0, policy_version 73870 (0.0007) [2023-10-11 21:59:39,441][71601] Updated weights for policy 0, policy_version 73880 (0.0008) [2023-10-11 21:59:41,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 151257088. Throughput: 0: 1832.1, 1: 1823.8. Samples: 37823268. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-11 21:59:41,034][70582] Avg episode reward: [(0, '73.350'), (1, '108.340')] [2023-10-11 21:59:42,548][71635] Updated weights for policy 1, policy_version 73832 (0.0010) [2023-10-11 21:59:42,919][71635] Updated weights for policy 1, policy_version 73842 (0.0008) [2023-10-11 21:59:42,958][71601] Updated weights for policy 0, policy_version 73890 (0.0008) [2023-10-11 21:59:43,284][71635] Updated weights for policy 1, policy_version 73852 (0.0007) [2023-10-11 21:59:43,325][71601] Updated weights for policy 0, policy_version 73900 (0.0008) [2023-10-11 21:59:43,699][71601] Updated weights for policy 0, policy_version 73910 (0.0008) [2023-10-11 21:59:44,078][71601] Updated weights for policy 0, policy_version 73920 (0.0008) [2023-10-11 21:59:46,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 151322624. Throughput: 0: 1829.3, 1: 1820.1. Samples: 37834344. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-11 21:59:46,035][70582] Avg episode reward: [(0, '73.270'), (1, '107.830')] [2023-10-11 21:59:47,002][71635] Updated weights for policy 1, policy_version 73862 (0.0008) [2023-10-11 21:59:47,369][71635] Updated weights for policy 1, policy_version 73872 (0.0008) [2023-10-11 21:59:47,728][71635] Updated weights for policy 1, policy_version 73882 (0.0008) [2023-10-11 21:59:47,782][71601] Updated weights for policy 0, policy_version 73930 (0.0008) [2023-10-11 21:59:48,159][71601] Updated weights for policy 0, policy_version 73940 (0.0010) [2023-10-11 21:59:48,529][71601] Updated weights for policy 0, policy_version 73950 (0.0011) [2023-10-11 21:59:51,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 151388160. Throughput: 0: 1836.3, 1: 1817.7. Samples: 37855874. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-11 21:59:51,034][70582] Avg episode reward: [(0, '72.910'), (1, '108.210')] [2023-10-11 21:59:51,487][71635] Updated weights for policy 1, policy_version 73892 (0.0008) [2023-10-11 21:59:51,856][71635] Updated weights for policy 1, policy_version 73902 (0.0007) [2023-10-11 21:59:52,224][71635] Updated weights for policy 1, policy_version 73912 (0.0007) [2023-10-11 21:59:52,345][71601] Updated weights for policy 0, policy_version 73960 (0.0009) [2023-10-11 21:59:52,720][71601] Updated weights for policy 0, policy_version 73970 (0.0009) [2023-10-11 21:59:53,095][71601] Updated weights for policy 0, policy_version 73980 (0.0009) [2023-10-11 21:59:56,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 151453696. Throughput: 0: 1830.3, 1: 1814.5. Samples: 37878572. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-11 21:59:56,035][70582] Avg episode reward: [(0, '72.060'), (1, '107.080')] [2023-10-11 21:59:56,039][71635] Updated weights for policy 1, policy_version 73922 (0.0008) [2023-10-11 21:59:56,045][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000073984_75759616.pth... [2023-10-11 21:59:56,080][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000072288_74022912.pth [2023-10-11 21:59:56,395][71635] Updated weights for policy 1, policy_version 73932 (0.0008) [2023-10-11 21:59:56,774][71635] Updated weights for policy 1, policy_version 73942 (0.0010) [2023-10-11 21:59:56,785][71601] Updated weights for policy 0, policy_version 73990 (0.0008) [2023-10-11 21:59:57,126][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000073952_75726848.pth... [2023-10-11 21:59:57,126][71635] Updated weights for policy 1, policy_version 73952 (0.0008) [2023-10-11 21:59:57,152][71601] Updated weights for policy 0, policy_version 74000 (0.0008) [2023-10-11 21:59:57,154][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000072224_73957376.pth [2023-10-11 21:59:57,514][71601] Updated weights for policy 0, policy_version 74010 (0.0007) [2023-10-11 22:00:00,766][71635] Updated weights for policy 1, policy_version 73962 (0.0009) [2023-10-11 22:00:01,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 151519232. Throughput: 0: 1830.3, 1: 1812.3. Samples: 37888540. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-11 22:00:01,034][70582] Avg episode reward: [(0, '74.110'), (1, '108.790')] [2023-10-11 22:00:01,058][71601] Updated weights for policy 0, policy_version 74020 (0.0007) [2023-10-11 22:00:01,126][71635] Updated weights for policy 1, policy_version 73972 (0.0008) [2023-10-11 22:00:01,424][71601] Updated weights for policy 0, policy_version 74030 (0.0007) [2023-10-11 22:00:01,484][71635] Updated weights for policy 1, policy_version 73982 (0.0007) [2023-10-11 22:00:01,800][71601] Updated weights for policy 0, policy_version 74040 (0.0008) [2023-10-11 22:00:05,287][71635] Updated weights for policy 1, policy_version 73992 (0.0008) [2023-10-11 22:00:05,569][71601] Updated weights for policy 0, policy_version 74050 (0.0007) [2023-10-11 22:00:05,650][71635] Updated weights for policy 1, policy_version 74002 (0.0007) [2023-10-11 22:00:05,937][71601] Updated weights for policy 0, policy_version 74060 (0.0007) [2023-10-11 22:00:06,026][71635] Updated weights for policy 1, policy_version 74012 (0.0008) [2023-10-11 22:00:06,034][70582] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 151584768. Throughput: 0: 1830.9, 1: 1802.9. Samples: 37911106. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-11 22:00:06,034][70582] Avg episode reward: [(0, '74.820'), (1, '108.860')] [2023-10-11 22:00:06,316][71601] Updated weights for policy 0, policy_version 74070 (0.0007) [2023-10-11 22:00:06,701][71601] Updated weights for policy 0, policy_version 74080 (0.0007) [2023-10-11 22:00:10,031][71635] Updated weights for policy 1, policy_version 74022 (0.0008) [2023-10-11 22:00:10,322][71601] Updated weights for policy 0, policy_version 74090 (0.0008) [2023-10-11 22:00:10,397][71635] Updated weights for policy 1, policy_version 74032 (0.0008) [2023-10-11 22:00:10,693][71601] Updated weights for policy 0, policy_version 74100 (0.0009) [2023-10-11 22:00:10,764][71635] Updated weights for policy 1, policy_version 74042 (0.0008) [2023-10-11 22:00:11,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 151683072. Throughput: 0: 1823.2, 1: 1812.2. Samples: 37932406. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-11 22:00:11,034][70582] Avg episode reward: [(0, '77.230'), (1, '106.880')] [2023-10-11 22:00:11,067][71601] Updated weights for policy 0, policy_version 74110 (0.0008) [2023-10-11 22:00:14,771][71635] Updated weights for policy 1, policy_version 74052 (0.0008) [2023-10-11 22:00:14,917][71601] Updated weights for policy 0, policy_version 74120 (0.0008) [2023-10-11 22:00:15,134][71635] Updated weights for policy 1, policy_version 74062 (0.0008) [2023-10-11 22:00:15,299][71601] Updated weights for policy 0, policy_version 74130 (0.0008) [2023-10-11 22:00:15,492][71635] Updated weights for policy 1, policy_version 74072 (0.0008) [2023-10-11 22:00:15,658][71601] Updated weights for policy 0, policy_version 74140 (0.0009) [2023-10-11 22:00:16,034][70582] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 151781376. Throughput: 0: 1826.5, 1: 1795.4. Samples: 37943370. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-11 22:00:16,035][70582] Avg episode reward: [(0, '78.650'), (1, '101.920')] [2023-10-11 22:00:19,203][71635] Updated weights for policy 1, policy_version 74082 (0.0009) [2023-10-11 22:00:19,303][71601] Updated weights for policy 0, policy_version 74150 (0.0010) [2023-10-11 22:00:19,579][71635] Updated weights for policy 1, policy_version 74092 (0.0009) [2023-10-11 22:00:19,667][71601] Updated weights for policy 0, policy_version 74160 (0.0008) [2023-10-11 22:00:19,938][71635] Updated weights for policy 1, policy_version 74102 (0.0007) [2023-10-11 22:00:20,038][71601] Updated weights for policy 0, policy_version 74170 (0.0007) [2023-10-11 22:00:20,311][71635] Updated weights for policy 1, policy_version 74112 (0.0009) [2023-10-11 22:00:21,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 151846912. Throughput: 0: 1816.8, 1: 1815.0. Samples: 37965052. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-11 22:00:21,034][70582] Avg episode reward: [(0, '82.820'), (1, '95.760')] [2023-10-11 22:00:23,701][71601] Updated weights for policy 0, policy_version 74180 (0.0007) [2023-10-11 22:00:24,051][71635] Updated weights for policy 1, policy_version 74122 (0.0008) [2023-10-11 22:00:24,066][71601] Updated weights for policy 0, policy_version 74190 (0.0007) [2023-10-11 22:00:24,415][71635] Updated weights for policy 1, policy_version 74132 (0.0008) [2023-10-11 22:00:24,442][71601] Updated weights for policy 0, policy_version 74200 (0.0008) [2023-10-11 22:00:24,778][71635] Updated weights for policy 1, policy_version 74142 (0.0009) [2023-10-11 22:00:26,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 151912448. Throughput: 0: 1816.5, 1: 1786.0. Samples: 37985382. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-11 22:00:26,035][70582] Avg episode reward: [(0, '79.990'), (1, '102.790')] [2023-10-11 22:00:28,096][71601] Updated weights for policy 0, policy_version 74210 (0.0009) [2023-10-11 22:00:28,398][71635] Updated weights for policy 1, policy_version 74152 (0.0008) [2023-10-11 22:00:28,466][71601] Updated weights for policy 0, policy_version 74220 (0.0007) [2023-10-11 22:00:28,763][71635] Updated weights for policy 1, policy_version 74162 (0.0009) [2023-10-11 22:00:28,835][71601] Updated weights for policy 0, policy_version 74230 (0.0009) [2023-10-11 22:00:29,121][71635] Updated weights for policy 1, policy_version 74172 (0.0007) [2023-10-11 22:00:29,203][71601] Updated weights for policy 0, policy_version 74240 (0.0008) [2023-10-11 22:00:31,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 151977984. Throughput: 0: 1815.8, 1: 1810.8. Samples: 37997538. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-11 22:00:31,034][70582] Avg episode reward: [(0, '82.770'), (1, '100.410')] [2023-10-11 22:00:32,719][71635] Updated weights for policy 1, policy_version 74182 (0.0008) [2023-10-11 22:00:32,879][71601] Updated weights for policy 0, policy_version 74250 (0.0007) [2023-10-11 22:00:33,086][71635] Updated weights for policy 1, policy_version 74192 (0.0008) [2023-10-11 22:00:33,244][71601] Updated weights for policy 0, policy_version 74260 (0.0008) [2023-10-11 22:00:33,454][71635] Updated weights for policy 1, policy_version 74202 (0.0007) [2023-10-11 22:00:33,615][71601] Updated weights for policy 0, policy_version 74270 (0.0007) [2023-10-11 22:00:36,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 152043520. Throughput: 0: 1812.7, 1: 1792.5. Samples: 38018110. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-11 22:00:36,035][70582] Avg episode reward: [(0, '86.110'), (1, '98.910')] [2023-10-11 22:00:37,097][71635] Updated weights for policy 1, policy_version 74212 (0.0007) [2023-10-11 22:00:37,397][71601] Updated weights for policy 0, policy_version 74280 (0.0008) [2023-10-11 22:00:37,452][71635] Updated weights for policy 1, policy_version 74222 (0.0008) [2023-10-11 22:00:37,772][71601] Updated weights for policy 0, policy_version 74290 (0.0008) [2023-10-11 22:00:37,820][71635] Updated weights for policy 1, policy_version 74232 (0.0008) [2023-10-11 22:00:38,143][71601] Updated weights for policy 0, policy_version 74300 (0.0009) [2023-10-11 22:00:41,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 152109056. Throughput: 0: 1810.0, 1: 1798.5. Samples: 38040950. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-11 22:00:41,034][70582] Avg episode reward: [(0, '87.540'), (1, '98.720')] [2023-10-11 22:00:41,525][71635] Updated weights for policy 1, policy_version 74242 (0.0007) [2023-10-11 22:00:41,819][71601] Updated weights for policy 0, policy_version 74310 (0.0007) [2023-10-11 22:00:41,889][71635] Updated weights for policy 1, policy_version 74252 (0.0007) [2023-10-11 22:00:42,183][71601] Updated weights for policy 0, policy_version 74320 (0.0007) [2023-10-11 22:00:42,255][71635] Updated weights for policy 1, policy_version 74262 (0.0009) [2023-10-11 22:00:42,556][71601] Updated weights for policy 0, policy_version 74330 (0.0009) [2023-10-11 22:00:42,608][71635] Updated weights for policy 1, policy_version 74272 (0.0009) [2023-10-11 22:00:46,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 152174592. Throughput: 0: 1808.4, 1: 1801.2. Samples: 38050970. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-11 22:00:46,034][70582] Avg episode reward: [(0, '85.870'), (1, '100.970')] [2023-10-11 22:00:46,166][71601] Updated weights for policy 0, policy_version 74340 (0.0010) [2023-10-11 22:00:46,272][71635] Updated weights for policy 1, policy_version 74282 (0.0009) [2023-10-11 22:00:46,537][71601] Updated weights for policy 0, policy_version 74350 (0.0008) [2023-10-11 22:00:46,642][71635] Updated weights for policy 1, policy_version 74292 (0.0007) [2023-10-11 22:00:46,915][71601] Updated weights for policy 0, policy_version 74360 (0.0008) [2023-10-11 22:00:47,001][71635] Updated weights for policy 1, policy_version 74302 (0.0007) [2023-10-11 22:00:50,570][71601] Updated weights for policy 0, policy_version 74370 (0.0008) [2023-10-11 22:00:50,598][71635] Updated weights for policy 1, policy_version 74312 (0.0007) [2023-10-11 22:00:50,936][71601] Updated weights for policy 0, policy_version 74380 (0.0008) [2023-10-11 22:00:50,959][71635] Updated weights for policy 1, policy_version 74322 (0.0007) [2023-10-11 22:00:51,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 152240128. Throughput: 0: 1806.7, 1: 1805.3. Samples: 38073644. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-11 22:00:51,034][70582] Avg episode reward: [(0, '84.870'), (1, '91.630')] [2023-10-11 22:00:51,310][71601] Updated weights for policy 0, policy_version 74390 (0.0009) [2023-10-11 22:00:51,327][71635] Updated weights for policy 1, policy_version 74332 (0.0007) [2023-10-11 22:00:51,678][71601] Updated weights for policy 0, policy_version 74400 (0.0007) [2023-10-11 22:00:55,188][71635] Updated weights for policy 1, policy_version 74342 (0.0007) [2023-10-11 22:00:55,494][71601] Updated weights for policy 0, policy_version 74410 (0.0008) [2023-10-11 22:00:55,566][71635] Updated weights for policy 1, policy_version 74352 (0.0009) [2023-10-11 22:00:55,856][71601] Updated weights for policy 0, policy_version 74420 (0.0008) [2023-10-11 22:00:55,925][71635] Updated weights for policy 1, policy_version 74362 (0.0009) [2023-10-11 22:00:56,034][70582] Fps is (10 sec: 13106.7, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 152305664. Throughput: 0: 1807.5, 1: 1812.0. Samples: 38095286. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-11 22:00:56,035][70582] Avg episode reward: [(0, '84.090'), (1, '94.260')] [2023-10-11 22:00:56,224][71601] Updated weights for policy 0, policy_version 74430 (0.0008) [2023-10-11 22:00:59,747][71635] Updated weights for policy 1, policy_version 74372 (0.0007) [2023-10-11 22:00:59,946][71601] Updated weights for policy 0, policy_version 74440 (0.0008) [2023-10-11 22:01:00,103][71635] Updated weights for policy 1, policy_version 74382 (0.0007) [2023-10-11 22:01:00,313][71601] Updated weights for policy 0, policy_version 74450 (0.0007) [2023-10-11 22:01:00,473][71635] Updated weights for policy 1, policy_version 74392 (0.0008) [2023-10-11 22:01:00,691][71601] Updated weights for policy 0, policy_version 74460 (0.0008) [2023-10-11 22:01:01,034][70582] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 152436736. Throughput: 0: 1805.9, 1: 1808.7. Samples: 38106026. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:01:01,035][70582] Avg episode reward: [(0, '83.010'), (1, '92.610')] [2023-10-11 22:01:04,291][71635] Updated weights for policy 1, policy_version 74402 (0.0008) [2023-10-11 22:01:04,417][71601] Updated weights for policy 0, policy_version 74470 (0.0007) [2023-10-11 22:01:04,659][71635] Updated weights for policy 1, policy_version 74412 (0.0008) [2023-10-11 22:01:04,794][71601] Updated weights for policy 0, policy_version 74480 (0.0007) [2023-10-11 22:01:05,020][71635] Updated weights for policy 1, policy_version 74422 (0.0007) [2023-10-11 22:01:05,164][71601] Updated weights for policy 0, policy_version 74490 (0.0007) [2023-10-11 22:01:05,390][71635] Updated weights for policy 1, policy_version 74432 (0.0008) [2023-10-11 22:01:06,034][70582] Fps is (10 sec: 19661.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 152502272. Throughput: 0: 1812.9, 1: 1814.0. Samples: 38128262. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:01:06,035][70582] Avg episode reward: [(0, '79.420'), (1, '95.950')] [2023-10-11 22:01:08,699][71601] Updated weights for policy 0, policy_version 74500 (0.0007) [2023-10-11 22:01:09,067][71601] Updated weights for policy 0, policy_version 74510 (0.0008) [2023-10-11 22:01:09,268][71635] Updated weights for policy 1, policy_version 74442 (0.0007) [2023-10-11 22:01:09,437][71601] Updated weights for policy 0, policy_version 74520 (0.0008) [2023-10-11 22:01:09,632][71635] Updated weights for policy 1, policy_version 74452 (0.0008) [2023-10-11 22:01:10,006][71635] Updated weights for policy 1, policy_version 74462 (0.0008) [2023-10-11 22:01:11,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 152567808. Throughput: 0: 1812.3, 1: 1810.0. Samples: 38148386. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:01:11,034][70582] Avg episode reward: [(0, '82.420'), (1, '93.700')] [2023-10-11 22:01:13,139][71601] Updated weights for policy 0, policy_version 74530 (0.0007) [2023-10-11 22:01:13,505][71601] Updated weights for policy 0, policy_version 74540 (0.0007) [2023-10-11 22:01:13,878][71601] Updated weights for policy 0, policy_version 74550 (0.0007) [2023-10-11 22:01:13,881][71635] Updated weights for policy 1, policy_version 74472 (0.0007) [2023-10-11 22:01:14,251][71635] Updated weights for policy 1, policy_version 74482 (0.0007) [2023-10-11 22:01:14,254][71601] Updated weights for policy 0, policy_version 74560 (0.0007) [2023-10-11 22:01:14,608][71635] Updated weights for policy 1, policy_version 74492 (0.0008) [2023-10-11 22:01:16,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 152633344. Throughput: 0: 1818.0, 1: 1810.1. Samples: 38160804. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:01:16,035][70582] Avg episode reward: [(0, '76.920'), (1, '89.530')] [2023-10-11 22:01:17,911][71601] Updated weights for policy 0, policy_version 74570 (0.0007) [2023-10-11 22:01:18,267][71601] Updated weights for policy 0, policy_version 74580 (0.0007) [2023-10-11 22:01:18,349][71635] Updated weights for policy 1, policy_version 74502 (0.0008) [2023-10-11 22:01:18,643][71601] Updated weights for policy 0, policy_version 74590 (0.0007) [2023-10-11 22:01:18,708][71635] Updated weights for policy 1, policy_version 74512 (0.0008) [2023-10-11 22:01:19,068][71635] Updated weights for policy 1, policy_version 74522 (0.0010) [2023-10-11 22:01:21,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 152698880. Throughput: 0: 1823.8, 1: 1799.7. Samples: 38181164. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:01:21,034][70582] Avg episode reward: [(0, '78.640'), (1, '91.020')] [2023-10-11 22:01:22,283][71601] Updated weights for policy 0, policy_version 74600 (0.0007) [2023-10-11 22:01:22,632][71635] Updated weights for policy 1, policy_version 74532 (0.0008) [2023-10-11 22:01:22,651][71601] Updated weights for policy 0, policy_version 74610 (0.0007) [2023-10-11 22:01:23,001][71635] Updated weights for policy 1, policy_version 74542 (0.0008) [2023-10-11 22:01:23,017][71601] Updated weights for policy 0, policy_version 74620 (0.0008) [2023-10-11 22:01:23,377][71635] Updated weights for policy 1, policy_version 74552 (0.0008) [2023-10-11 22:01:26,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 152764416. Throughput: 0: 1823.0, 1: 1800.8. Samples: 38204018. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:01:26,034][70582] Avg episode reward: [(0, '77.690'), (1, '92.700')] [2023-10-11 22:01:26,818][71601] Updated weights for policy 0, policy_version 74630 (0.0007) [2023-10-11 22:01:27,083][71635] Updated weights for policy 1, policy_version 74562 (0.0010) [2023-10-11 22:01:27,191][71601] Updated weights for policy 0, policy_version 74640 (0.0009) [2023-10-11 22:01:27,448][71635] Updated weights for policy 1, policy_version 74572 (0.0009) [2023-10-11 22:01:27,556][71601] Updated weights for policy 0, policy_version 74650 (0.0007) [2023-10-11 22:01:27,808][71635] Updated weights for policy 1, policy_version 74582 (0.0008) [2023-10-11 22:01:28,168][71635] Updated weights for policy 1, policy_version 74592 (0.0009) [2023-10-11 22:01:31,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 152829952. Throughput: 0: 1817.3, 1: 1800.8. Samples: 38213786. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:01:31,034][70582] Avg episode reward: [(0, '76.090'), (1, '92.400')] [2023-10-11 22:01:31,396][71601] Updated weights for policy 0, policy_version 74660 (0.0009) [2023-10-11 22:01:31,771][71601] Updated weights for policy 0, policy_version 74670 (0.0009) [2023-10-11 22:01:31,905][71635] Updated weights for policy 1, policy_version 74602 (0.0007) [2023-10-11 22:01:32,145][71601] Updated weights for policy 0, policy_version 74680 (0.0007) [2023-10-11 22:01:32,278][71635] Updated weights for policy 1, policy_version 74612 (0.0009) [2023-10-11 22:01:32,652][71635] Updated weights for policy 1, policy_version 74622 (0.0009) [2023-10-11 22:01:35,922][71601] Updated weights for policy 0, policy_version 74690 (0.0009) [2023-10-11 22:01:36,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 152895488. Throughput: 0: 1815.3, 1: 1796.3. Samples: 38236164. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:01:36,034][70582] Avg episode reward: [(0, '72.330'), (1, '92.570')] [2023-10-11 22:01:36,279][71635] Updated weights for policy 1, policy_version 74632 (0.0008) [2023-10-11 22:01:36,290][71601] Updated weights for policy 0, policy_version 74700 (0.0008) [2023-10-11 22:01:36,646][71635] Updated weights for policy 1, policy_version 74642 (0.0007) [2023-10-11 22:01:36,655][71601] Updated weights for policy 0, policy_version 74710 (0.0009) [2023-10-11 22:01:37,001][71635] Updated weights for policy 1, policy_version 74652 (0.0009) [2023-10-11 22:01:37,024][71601] Updated weights for policy 0, policy_version 74720 (0.0008) [2023-10-11 22:01:40,691][71601] Updated weights for policy 0, policy_version 74730 (0.0007) [2023-10-11 22:01:40,841][71635] Updated weights for policy 1, policy_version 74662 (0.0008) [2023-10-11 22:01:41,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 152961024. Throughput: 0: 1828.1, 1: 1807.3. Samples: 38258880. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:01:41,035][70582] Avg episode reward: [(0, '72.510'), (1, '96.820')] [2023-10-11 22:01:41,062][71601] Updated weights for policy 0, policy_version 74740 (0.0007) [2023-10-11 22:01:41,219][71635] Updated weights for policy 1, policy_version 74672 (0.0007) [2023-10-11 22:01:41,431][71601] Updated weights for policy 0, policy_version 74750 (0.0007) [2023-10-11 22:01:41,583][71635] Updated weights for policy 1, policy_version 74682 (0.0007) [2023-10-11 22:01:45,108][71601] Updated weights for policy 0, policy_version 74760 (0.0008) [2023-10-11 22:01:45,288][71635] Updated weights for policy 1, policy_version 74692 (0.0008) [2023-10-11 22:01:45,485][71601] Updated weights for policy 0, policy_version 74770 (0.0009) [2023-10-11 22:01:45,648][71635] Updated weights for policy 1, policy_version 74702 (0.0008) [2023-10-11 22:01:45,860][71601] Updated weights for policy 0, policy_version 74780 (0.0007) [2023-10-11 22:01:46,012][71635] Updated weights for policy 1, policy_version 74712 (0.0008) [2023-10-11 22:01:46,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 153059328. Throughput: 0: 1821.7, 1: 1790.2. Samples: 38268564. Policy #0 lag: (min: 21.0, avg: 29.0, max: 53.0) [2023-10-11 22:01:46,034][70582] Avg episode reward: [(0, '74.430'), (1, '99.500')] [2023-10-11 22:01:49,589][71601] Updated weights for policy 0, policy_version 74790 (0.0008) [2023-10-11 22:01:49,680][71635] Updated weights for policy 1, policy_version 74722 (0.0009) [2023-10-11 22:01:49,960][71601] Updated weights for policy 0, policy_version 74800 (0.0007) [2023-10-11 22:01:50,045][71635] Updated weights for policy 1, policy_version 74732 (0.0009) [2023-10-11 22:01:50,325][71601] Updated weights for policy 0, policy_version 74810 (0.0007) [2023-10-11 22:01:50,411][71635] Updated weights for policy 1, policy_version 74742 (0.0007) [2023-10-11 22:01:50,763][71635] Updated weights for policy 1, policy_version 74752 (0.0008) [2023-10-11 22:01:51,034][70582] Fps is (10 sec: 19661.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 153157632. Throughput: 0: 1822.7, 1: 1794.7. Samples: 38291044. Policy #0 lag: (min: 21.0, avg: 29.0, max: 53.0) [2023-10-11 22:01:51,034][70582] Avg episode reward: [(0, '77.270'), (1, '100.440')] [2023-10-11 22:01:53,942][71601] Updated weights for policy 0, policy_version 74820 (0.0009) [2023-10-11 22:01:54,312][71601] Updated weights for policy 0, policy_version 74830 (0.0011) [2023-10-11 22:01:54,679][71635] Updated weights for policy 1, policy_version 74762 (0.0007) [2023-10-11 22:01:54,680][71601] Updated weights for policy 0, policy_version 74840 (0.0008) [2023-10-11 22:01:55,039][71635] Updated weights for policy 1, policy_version 74772 (0.0009) [2023-10-11 22:01:55,416][71635] Updated weights for policy 1, policy_version 74782 (0.0009) [2023-10-11 22:01:56,034][70582] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 153223168. Throughput: 0: 1815.6, 1: 1799.2. Samples: 38311054. Policy #0 lag: (min: 21.0, avg: 29.0, max: 53.0) [2023-10-11 22:01:56,034][70582] Avg episode reward: [(0, '74.200'), (1, '93.680')] [2023-10-11 22:01:56,042][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000074848_76644352.pth... [2023-10-11 22:01:56,042][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000074784_76578816.pth... [2023-10-11 22:01:56,087][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000073088_74842112.pth [2023-10-11 22:01:56,088][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000073152_74907648.pth [2023-10-11 22:01:58,329][71601] Updated weights for policy 0, policy_version 74850 (0.0007) [2023-10-11 22:01:58,707][71601] Updated weights for policy 0, policy_version 74860 (0.0008) [2023-10-11 22:01:59,066][71601] Updated weights for policy 0, policy_version 74870 (0.0010) [2023-10-11 22:01:59,086][71635] Updated weights for policy 1, policy_version 74792 (0.0009) [2023-10-11 22:01:59,433][71601] Updated weights for policy 0, policy_version 74880 (0.0011) [2023-10-11 22:01:59,450][71635] Updated weights for policy 1, policy_version 74802 (0.0008) [2023-10-11 22:01:59,825][71635] Updated weights for policy 1, policy_version 74812 (0.0010) [2023-10-11 22:02:01,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 153288704. Throughput: 0: 1815.1, 1: 1796.6. Samples: 38323328. Policy #0 lag: (min: 21.0, avg: 29.0, max: 53.0) [2023-10-11 22:02:01,035][70582] Avg episode reward: [(0, '76.790'), (1, '94.660')] [2023-10-11 22:02:03,170][71601] Updated weights for policy 0, policy_version 74890 (0.0007) [2023-10-11 22:02:03,536][71601] Updated weights for policy 0, policy_version 74900 (0.0009) [2023-10-11 22:02:03,658][71635] Updated weights for policy 1, policy_version 74822 (0.0007) [2023-10-11 22:02:03,903][71601] Updated weights for policy 0, policy_version 74910 (0.0008) [2023-10-11 22:02:04,023][71635] Updated weights for policy 1, policy_version 74832 (0.0008) [2023-10-11 22:02:04,395][71635] Updated weights for policy 1, policy_version 74842 (0.0010) [2023-10-11 22:02:06,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 153354240. Throughput: 0: 1805.8, 1: 1802.4. Samples: 38343534. Policy #0 lag: (min: 21.0, avg: 29.0, max: 53.0) [2023-10-11 22:02:06,034][70582] Avg episode reward: [(0, '74.710'), (1, '92.470')] [2023-10-11 22:02:07,660][71601] Updated weights for policy 0, policy_version 74920 (0.0008) [2023-10-11 22:02:08,034][71601] Updated weights for policy 0, policy_version 74930 (0.0007) [2023-10-11 22:02:08,121][71635] Updated weights for policy 1, policy_version 74852 (0.0008) [2023-10-11 22:02:08,401][71601] Updated weights for policy 0, policy_version 74940 (0.0007) [2023-10-11 22:02:08,494][71635] Updated weights for policy 1, policy_version 74862 (0.0008) [2023-10-11 22:02:08,867][71635] Updated weights for policy 1, policy_version 74872 (0.0010) [2023-10-11 22:02:11,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 153419776. Throughput: 0: 1808.0, 1: 1789.3. Samples: 38365900. Policy #0 lag: (min: 21.0, avg: 29.0, max: 53.0) [2023-10-11 22:02:11,034][70582] Avg episode reward: [(0, '76.750'), (1, '98.150')] [2023-10-11 22:02:12,061][71601] Updated weights for policy 0, policy_version 74950 (0.0008) [2023-10-11 22:02:12,434][71601] Updated weights for policy 0, policy_version 74960 (0.0008) [2023-10-11 22:02:12,665][71635] Updated weights for policy 1, policy_version 74882 (0.0009) [2023-10-11 22:02:12,809][71601] Updated weights for policy 0, policy_version 74970 (0.0008) [2023-10-11 22:02:13,031][71635] Updated weights for policy 1, policy_version 74892 (0.0007) [2023-10-11 22:02:13,402][71635] Updated weights for policy 1, policy_version 74902 (0.0010) [2023-10-11 22:02:13,771][71635] Updated weights for policy 1, policy_version 74912 (0.0010) [2023-10-11 22:02:16,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 153485312. Throughput: 0: 1813.7, 1: 1798.6. Samples: 38376342. Policy #0 lag: (min: 21.0, avg: 29.0, max: 53.0) [2023-10-11 22:02:16,034][70582] Avg episode reward: [(0, '76.630'), (1, '93.770')] [2023-10-11 22:02:16,607][71601] Updated weights for policy 0, policy_version 74980 (0.0008) [2023-10-11 22:02:16,965][71601] Updated weights for policy 0, policy_version 74990 (0.0008) [2023-10-11 22:02:17,282][71635] Updated weights for policy 1, policy_version 74922 (0.0008) [2023-10-11 22:02:17,338][71601] Updated weights for policy 0, policy_version 75000 (0.0007) [2023-10-11 22:02:17,645][71635] Updated weights for policy 1, policy_version 74932 (0.0009) [2023-10-11 22:02:18,012][71635] Updated weights for policy 1, policy_version 74942 (0.0011) [2023-10-11 22:02:20,972][71601] Updated weights for policy 0, policy_version 75010 (0.0008) [2023-10-11 22:02:21,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 153550848. Throughput: 0: 1819.0, 1: 1796.7. Samples: 38398868. Policy #0 lag: (min: 21.0, avg: 29.0, max: 53.0) [2023-10-11 22:02:21,034][70582] Avg episode reward: [(0, '76.990'), (1, '96.050')] [2023-10-11 22:02:21,344][71601] Updated weights for policy 0, policy_version 75020 (0.0007) [2023-10-11 22:02:21,711][71635] Updated weights for policy 1, policy_version 74952 (0.0008) [2023-10-11 22:02:21,720][71601] Updated weights for policy 0, policy_version 75030 (0.0009) [2023-10-11 22:02:22,077][71635] Updated weights for policy 1, policy_version 74962 (0.0008) [2023-10-11 22:02:22,091][71601] Updated weights for policy 0, policy_version 75040 (0.0009) [2023-10-11 22:02:22,441][71635] Updated weights for policy 1, policy_version 74972 (0.0008) [2023-10-11 22:02:25,600][71601] Updated weights for policy 0, policy_version 75050 (0.0009) [2023-10-11 22:02:25,980][71601] Updated weights for policy 0, policy_version 75060 (0.0009) [2023-10-11 22:02:26,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 153616384. Throughput: 0: 1814.2, 1: 1796.5. Samples: 38421362. Policy #0 lag: (min: 21.0, avg: 29.0, max: 53.0) [2023-10-11 22:02:26,034][70582] Avg episode reward: [(0, '77.430'), (1, '95.180')] [2023-10-11 22:02:26,183][71635] Updated weights for policy 1, policy_version 74982 (0.0009) [2023-10-11 22:02:26,351][71601] Updated weights for policy 0, policy_version 75070 (0.0008) [2023-10-11 22:02:26,576][71635] Updated weights for policy 1, policy_version 74992 (0.0007) [2023-10-11 22:02:26,947][71635] Updated weights for policy 1, policy_version 75002 (0.0010) [2023-10-11 22:02:30,002][71601] Updated weights for policy 0, policy_version 75080 (0.0008) [2023-10-11 22:02:30,373][71601] Updated weights for policy 0, policy_version 75090 (0.0010) [2023-10-11 22:02:30,712][71635] Updated weights for policy 1, policy_version 75012 (0.0009) [2023-10-11 22:02:30,735][71601] Updated weights for policy 0, policy_version 75100 (0.0009) [2023-10-11 22:02:31,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 153714688. Throughput: 0: 1819.3, 1: 1800.2. Samples: 38431444. Policy #0 lag: (min: 21.0, avg: 29.0, max: 53.0) [2023-10-11 22:02:31,034][70582] Avg episode reward: [(0, '75.360'), (1, '95.270')] [2023-10-11 22:02:31,070][71635] Updated weights for policy 1, policy_version 75022 (0.0008) [2023-10-11 22:02:31,439][71635] Updated weights for policy 1, policy_version 75032 (0.0008) [2023-10-11 22:02:34,489][71601] Updated weights for policy 0, policy_version 75110 (0.0008) [2023-10-11 22:02:34,866][71601] Updated weights for policy 0, policy_version 75120 (0.0008) [2023-10-11 22:02:35,183][71635] Updated weights for policy 1, policy_version 75042 (0.0009) [2023-10-11 22:02:35,232][71601] Updated weights for policy 0, policy_version 75130 (0.0007) [2023-10-11 22:02:35,547][71635] Updated weights for policy 1, policy_version 75052 (0.0008) [2023-10-11 22:02:35,911][71635] Updated weights for policy 1, policy_version 75062 (0.0008) [2023-10-11 22:02:36,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 153780224. Throughput: 0: 1817.5, 1: 1800.1. Samples: 38453838. Policy #0 lag: (min: 2.0, avg: 4.3, max: 34.0) [2023-10-11 22:02:36,035][70582] Avg episode reward: [(0, '74.570'), (1, '94.830')] [2023-10-11 22:02:36,285][71635] Updated weights for policy 1, policy_version 75072 (0.0008) [2023-10-11 22:02:39,018][71601] Updated weights for policy 0, policy_version 75140 (0.0007) [2023-10-11 22:02:39,390][71601] Updated weights for policy 0, policy_version 75150 (0.0007) [2023-10-11 22:02:39,757][71601] Updated weights for policy 0, policy_version 75160 (0.0007) [2023-10-11 22:02:40,075][71635] Updated weights for policy 1, policy_version 75082 (0.0008) [2023-10-11 22:02:40,441][71635] Updated weights for policy 1, policy_version 75092 (0.0008) [2023-10-11 22:02:40,807][71635] Updated weights for policy 1, policy_version 75102 (0.0008) [2023-10-11 22:02:41,034][70582] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 153878528. Throughput: 0: 1815.9, 1: 1811.8. Samples: 38474302. Policy #0 lag: (min: 2.0, avg: 4.3, max: 34.0) [2023-10-11 22:02:41,035][70582] Avg episode reward: [(0, '68.430'), (1, '98.200')] [2023-10-11 22:02:43,416][71601] Updated weights for policy 0, policy_version 75170 (0.0009) [2023-10-11 22:02:43,792][71601] Updated weights for policy 0, policy_version 75180 (0.0009) [2023-10-11 22:02:44,166][71601] Updated weights for policy 0, policy_version 75190 (0.0010) [2023-10-11 22:02:44,458][71635] Updated weights for policy 1, policy_version 75112 (0.0009) [2023-10-11 22:02:44,531][71601] Updated weights for policy 0, policy_version 75200 (0.0009) [2023-10-11 22:02:44,832][71635] Updated weights for policy 1, policy_version 75122 (0.0010) [2023-10-11 22:02:45,191][71635] Updated weights for policy 1, policy_version 75132 (0.0009) [2023-10-11 22:02:46,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 153944064. Throughput: 0: 1819.9, 1: 1809.1. Samples: 38486632. Policy #0 lag: (min: 2.0, avg: 4.3, max: 34.0) [2023-10-11 22:02:46,035][70582] Avg episode reward: [(0, '68.780'), (1, '101.490')] [2023-10-11 22:02:48,370][71601] Updated weights for policy 0, policy_version 75210 (0.0008) [2023-10-11 22:02:48,745][71601] Updated weights for policy 0, policy_version 75220 (0.0009) [2023-10-11 22:02:49,040][71635] Updated weights for policy 1, policy_version 75142 (0.0008) [2023-10-11 22:02:49,105][71601] Updated weights for policy 0, policy_version 75230 (0.0008) [2023-10-11 22:02:49,403][71635] Updated weights for policy 1, policy_version 75152 (0.0009) [2023-10-11 22:02:49,768][71635] Updated weights for policy 1, policy_version 75162 (0.0007) [2023-10-11 22:02:51,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 154009600. Throughput: 0: 1814.0, 1: 1815.4. Samples: 38506856. Policy #0 lag: (min: 2.0, avg: 4.3, max: 34.0) [2023-10-11 22:02:51,034][70582] Avg episode reward: [(0, '70.070'), (1, '99.190')] [2023-10-11 22:02:52,809][71601] Updated weights for policy 0, policy_version 75240 (0.0008) [2023-10-11 22:02:53,193][71601] Updated weights for policy 0, policy_version 75250 (0.0007) [2023-10-11 22:02:53,394][71635] Updated weights for policy 1, policy_version 75172 (0.0007) [2023-10-11 22:02:53,559][71601] Updated weights for policy 0, policy_version 75260 (0.0008) [2023-10-11 22:02:53,748][71635] Updated weights for policy 1, policy_version 75182 (0.0007) [2023-10-11 22:02:54,118][71635] Updated weights for policy 1, policy_version 75192 (0.0010) [2023-10-11 22:02:56,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 154075136. Throughput: 0: 1818.5, 1: 1808.2. Samples: 38529104. Policy #0 lag: (min: 2.0, avg: 4.3, max: 34.0) [2023-10-11 22:02:56,034][70582] Avg episode reward: [(0, '74.620'), (1, '99.530')] [2023-10-11 22:02:57,154][71601] Updated weights for policy 0, policy_version 75270 (0.0007) [2023-10-11 22:02:57,515][71601] Updated weights for policy 0, policy_version 75280 (0.0008) [2023-10-11 22:02:57,756][71635] Updated weights for policy 1, policy_version 75202 (0.0007) [2023-10-11 22:02:57,893][71601] Updated weights for policy 0, policy_version 75290 (0.0008) [2023-10-11 22:02:58,127][71635] Updated weights for policy 1, policy_version 75212 (0.0007) [2023-10-11 22:02:58,497][71635] Updated weights for policy 1, policy_version 75222 (0.0009) [2023-10-11 22:02:58,867][71635] Updated weights for policy 1, policy_version 75232 (0.0008) [2023-10-11 22:03:01,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 154140672. Throughput: 0: 1814.0, 1: 1816.1. Samples: 38539694. Policy #0 lag: (min: 2.0, avg: 4.3, max: 34.0) [2023-10-11 22:03:01,034][70582] Avg episode reward: [(0, '76.340'), (1, '96.700')] [2023-10-11 22:03:01,564][71601] Updated weights for policy 0, policy_version 75300 (0.0007) [2023-10-11 22:03:01,931][71601] Updated weights for policy 0, policy_version 75310 (0.0008) [2023-10-11 22:03:02,302][71601] Updated weights for policy 0, policy_version 75320 (0.0010) [2023-10-11 22:03:02,767][71635] Updated weights for policy 1, policy_version 75242 (0.0009) [2023-10-11 22:03:03,132][71635] Updated weights for policy 1, policy_version 75252 (0.0010) [2023-10-11 22:03:03,498][71635] Updated weights for policy 1, policy_version 75262 (0.0008) [2023-10-11 22:03:06,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 154206208. Throughput: 0: 1814.2, 1: 1798.0. Samples: 38561414. Policy #0 lag: (min: 2.0, avg: 4.3, max: 34.0) [2023-10-11 22:03:06,034][70582] Avg episode reward: [(0, '74.780'), (1, '101.030')] [2023-10-11 22:03:06,070][71601] Updated weights for policy 0, policy_version 75330 (0.0007) [2023-10-11 22:03:06,442][71601] Updated weights for policy 0, policy_version 75340 (0.0007) [2023-10-11 22:03:06,810][71601] Updated weights for policy 0, policy_version 75350 (0.0007) [2023-10-11 22:03:07,161][71635] Updated weights for policy 1, policy_version 75272 (0.0008) [2023-10-11 22:03:07,183][71601] Updated weights for policy 0, policy_version 75360 (0.0007) [2023-10-11 22:03:07,522][71635] Updated weights for policy 1, policy_version 75282 (0.0010) [2023-10-11 22:03:07,888][71635] Updated weights for policy 1, policy_version 75292 (0.0008) [2023-10-11 22:03:10,857][71601] Updated weights for policy 0, policy_version 75370 (0.0008) [2023-10-11 22:03:11,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 154271744. Throughput: 0: 1817.0, 1: 1795.8. Samples: 38583940. Policy #0 lag: (min: 2.0, avg: 4.3, max: 34.0) [2023-10-11 22:03:11,034][70582] Avg episode reward: [(0, '78.120'), (1, '104.720')] [2023-10-11 22:03:11,232][71601] Updated weights for policy 0, policy_version 75380 (0.0008) [2023-10-11 22:03:11,611][71601] Updated weights for policy 0, policy_version 75390 (0.0008) [2023-10-11 22:03:11,769][71635] Updated weights for policy 1, policy_version 75302 (0.0007) [2023-10-11 22:03:12,152][71635] Updated weights for policy 1, policy_version 75312 (0.0008) [2023-10-11 22:03:12,521][71635] Updated weights for policy 1, policy_version 75322 (0.0009) [2023-10-11 22:03:15,213][71601] Updated weights for policy 0, policy_version 75400 (0.0007) [2023-10-11 22:03:15,585][71601] Updated weights for policy 0, policy_version 75410 (0.0008) [2023-10-11 22:03:15,956][71601] Updated weights for policy 0, policy_version 75420 (0.0009) [2023-10-11 22:03:16,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 154337280. Throughput: 0: 1807.0, 1: 1797.1. Samples: 38593626. Policy #0 lag: (min: 2.0, avg: 4.3, max: 34.0) [2023-10-11 22:03:16,034][70582] Avg episode reward: [(0, '77.440'), (1, '104.440')] [2023-10-11 22:03:16,188][71635] Updated weights for policy 1, policy_version 75332 (0.0010) [2023-10-11 22:03:16,559][71635] Updated weights for policy 1, policy_version 75342 (0.0011) [2023-10-11 22:03:16,923][71635] Updated weights for policy 1, policy_version 75352 (0.0010) [2023-10-11 22:03:19,687][71601] Updated weights for policy 0, policy_version 75430 (0.0009) [2023-10-11 22:03:20,052][71601] Updated weights for policy 0, policy_version 75440 (0.0010) [2023-10-11 22:03:20,433][71601] Updated weights for policy 0, policy_version 75450 (0.0007) [2023-10-11 22:03:20,711][71635] Updated weights for policy 1, policy_version 75362 (0.0008) [2023-10-11 22:03:21,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 154435584. Throughput: 0: 1815.7, 1: 1794.3. Samples: 38616288. Policy #0 lag: (min: 2.0, avg: 4.3, max: 34.0) [2023-10-11 22:03:21,034][70582] Avg episode reward: [(0, '79.780'), (1, '103.880')] [2023-10-11 22:03:21,077][71635] Updated weights for policy 1, policy_version 75372 (0.0007) [2023-10-11 22:03:21,440][71635] Updated weights for policy 1, policy_version 75382 (0.0008) [2023-10-11 22:03:21,807][71635] Updated weights for policy 1, policy_version 75392 (0.0008) [2023-10-11 22:03:24,002][71601] Updated weights for policy 0, policy_version 75460 (0.0008) [2023-10-11 22:03:24,362][71601] Updated weights for policy 0, policy_version 75470 (0.0007) [2023-10-11 22:03:24,730][71601] Updated weights for policy 0, policy_version 75480 (0.0007) [2023-10-11 22:03:25,500][71635] Updated weights for policy 1, policy_version 75402 (0.0009) [2023-10-11 22:03:25,865][71635] Updated weights for policy 1, policy_version 75412 (0.0010) [2023-10-11 22:03:26,034][70582] Fps is (10 sec: 16383.4, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 154501120. Throughput: 0: 1821.7, 1: 1810.0. Samples: 38637730. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-11 22:03:26,035][70582] Avg episode reward: [(0, '82.800'), (1, '107.940')] [2023-10-11 22:03:26,227][71635] Updated weights for policy 1, policy_version 75422 (0.0012) [2023-10-11 22:03:28,256][71601] Updated weights for policy 0, policy_version 75490 (0.0007) [2023-10-11 22:03:28,627][71601] Updated weights for policy 0, policy_version 75500 (0.0007) [2023-10-11 22:03:28,999][71601] Updated weights for policy 0, policy_version 75510 (0.0007) [2023-10-11 22:03:29,375][71601] Updated weights for policy 0, policy_version 75520 (0.0007) [2023-10-11 22:03:29,939][71635] Updated weights for policy 1, policy_version 75432 (0.0007) [2023-10-11 22:03:30,306][71635] Updated weights for policy 1, policy_version 75442 (0.0008) [2023-10-11 22:03:30,675][71635] Updated weights for policy 1, policy_version 75452 (0.0009) [2023-10-11 22:03:31,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 154599424. Throughput: 0: 1818.8, 1: 1789.9. Samples: 38649020. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-11 22:03:31,035][70582] Avg episode reward: [(0, '87.280'), (1, '107.310')] [2023-10-11 22:03:33,030][71601] Updated weights for policy 0, policy_version 75530 (0.0008) [2023-10-11 22:03:33,402][71601] Updated weights for policy 0, policy_version 75540 (0.0008) [2023-10-11 22:03:33,772][71601] Updated weights for policy 0, policy_version 75550 (0.0008) [2023-10-11 22:03:34,533][71635] Updated weights for policy 1, policy_version 75462 (0.0008) [2023-10-11 22:03:34,897][71635] Updated weights for policy 1, policy_version 75472 (0.0009) [2023-10-11 22:03:35,263][71635] Updated weights for policy 1, policy_version 75482 (0.0007) [2023-10-11 22:03:36,034][70582] Fps is (10 sec: 16384.7, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 154664960. Throughput: 0: 1828.9, 1: 1809.2. Samples: 38670568. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-11 22:03:36,034][70582] Avg episode reward: [(0, '89.090'), (1, '107.830')] [2023-10-11 22:03:37,602][71601] Updated weights for policy 0, policy_version 75560 (0.0009) [2023-10-11 22:03:37,980][71601] Updated weights for policy 0, policy_version 75570 (0.0010) [2023-10-11 22:03:38,364][71601] Updated weights for policy 0, policy_version 75580 (0.0010) [2023-10-11 22:03:38,893][71635] Updated weights for policy 1, policy_version 75492 (0.0009) [2023-10-11 22:03:39,251][71635] Updated weights for policy 1, policy_version 75502 (0.0008) [2023-10-11 22:03:39,616][71635] Updated weights for policy 1, policy_version 75512 (0.0010) [2023-10-11 22:03:41,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 154730496. Throughput: 0: 1823.9, 1: 1794.8. Samples: 38691944. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-11 22:03:41,035][70582] Avg episode reward: [(0, '87.330'), (1, '114.230')] [2023-10-11 22:03:42,011][71601] Updated weights for policy 0, policy_version 75590 (0.0009) [2023-10-11 22:03:42,383][71601] Updated weights for policy 0, policy_version 75600 (0.0010) [2023-10-11 22:03:42,756][71601] Updated weights for policy 0, policy_version 75610 (0.0009) [2023-10-11 22:03:43,267][71635] Updated weights for policy 1, policy_version 75522 (0.0007) [2023-10-11 22:03:43,645][71635] Updated weights for policy 1, policy_version 75532 (0.0009) [2023-10-11 22:03:44,000][71635] Updated weights for policy 1, policy_version 75542 (0.0009) [2023-10-11 22:03:44,373][71635] Updated weights for policy 1, policy_version 75552 (0.0008) [2023-10-11 22:03:46,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 154796032. Throughput: 0: 1820.8, 1: 1805.3. Samples: 38702868. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-11 22:03:46,035][70582] Avg episode reward: [(0, '88.260'), (1, '113.810')] [2023-10-11 22:03:46,479][71601] Updated weights for policy 0, policy_version 75620 (0.0009) [2023-10-11 22:03:46,864][71601] Updated weights for policy 0, policy_version 75630 (0.0007) [2023-10-11 22:03:47,239][71601] Updated weights for policy 0, policy_version 75640 (0.0008) [2023-10-11 22:03:48,182][71635] Updated weights for policy 1, policy_version 75562 (0.0008) [2023-10-11 22:03:48,558][71635] Updated weights for policy 1, policy_version 75572 (0.0007) [2023-10-11 22:03:48,918][71635] Updated weights for policy 1, policy_version 75582 (0.0008) [2023-10-11 22:03:51,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.2). Total num frames: 154861568. Throughput: 0: 1821.7, 1: 1799.2. Samples: 38724356. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-11 22:03:51,034][70582] Avg episode reward: [(0, '88.720'), (1, '116.880')] [2023-10-11 22:03:51,044][71601] Updated weights for policy 0, policy_version 75650 (0.0008) [2023-10-11 22:03:51,424][71601] Updated weights for policy 0, policy_version 75660 (0.0008) [2023-10-11 22:03:51,794][71601] Updated weights for policy 0, policy_version 75670 (0.0008) [2023-10-11 22:03:52,157][71601] Updated weights for policy 0, policy_version 75680 (0.0007) [2023-10-11 22:03:52,513][71635] Updated weights for policy 1, policy_version 75592 (0.0009) [2023-10-11 22:03:52,878][71635] Updated weights for policy 1, policy_version 75602 (0.0009) [2023-10-11 22:03:53,248][71635] Updated weights for policy 1, policy_version 75612 (0.0009) [2023-10-11 22:03:55,977][71601] Updated weights for policy 0, policy_version 75690 (0.0010) [2023-10-11 22:03:56,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 154927104. Throughput: 0: 1821.9, 1: 1807.6. Samples: 38747266. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-11 22:03:56,034][70582] Avg episode reward: [(0, '89.510'), (1, '115.320')] [2023-10-11 22:03:56,045][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000075616_77430784.pth... [2023-10-11 22:03:56,081][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000073952_75726848.pth [2023-10-11 22:03:56,339][71601] Updated weights for policy 0, policy_version 75700 (0.0010) [2023-10-11 22:03:56,712][71601] Updated weights for policy 0, policy_version 75710 (0.0009) [2023-10-11 22:03:56,785][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000075712_77529088.pth... [2023-10-11 22:03:56,814][71635] Updated weights for policy 1, policy_version 75622 (0.0008) [2023-10-11 22:03:56,818][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000073984_75759616.pth [2023-10-11 22:03:57,183][71635] Updated weights for policy 1, policy_version 75632 (0.0007) [2023-10-11 22:03:57,542][71635] Updated weights for policy 1, policy_version 75642 (0.0009) [2023-10-11 22:04:00,288][71601] Updated weights for policy 0, policy_version 75720 (0.0009) [2023-10-11 22:04:00,662][71601] Updated weights for policy 0, policy_version 75730 (0.0010) [2023-10-11 22:04:01,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 154992640. Throughput: 0: 1824.4, 1: 1812.4. Samples: 38757282. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-11 22:04:01,034][70582] Avg episode reward: [(0, '87.290'), (1, '104.900')] [2023-10-11 22:04:01,036][71601] Updated weights for policy 0, policy_version 75740 (0.0010) [2023-10-11 22:04:01,310][71635] Updated weights for policy 1, policy_version 75652 (0.0008) [2023-10-11 22:04:01,669][71635] Updated weights for policy 1, policy_version 75662 (0.0010) [2023-10-11 22:04:02,034][71635] Updated weights for policy 1, policy_version 75672 (0.0011) [2023-10-11 22:04:04,751][71601] Updated weights for policy 0, policy_version 75750 (0.0008) [2023-10-11 22:04:05,120][71601] Updated weights for policy 0, policy_version 75760 (0.0007) [2023-10-11 22:04:05,493][71601] Updated weights for policy 0, policy_version 75770 (0.0007) [2023-10-11 22:04:05,805][71635] Updated weights for policy 1, policy_version 75682 (0.0009) [2023-10-11 22:04:06,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 155090944. Throughput: 0: 1818.7, 1: 1815.2. Samples: 38779814. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-11 22:04:06,034][70582] Avg episode reward: [(0, '90.210'), (1, '104.200')] [2023-10-11 22:04:06,167][71635] Updated weights for policy 1, policy_version 75692 (0.0007) [2023-10-11 22:04:06,536][71635] Updated weights for policy 1, policy_version 75702 (0.0008) [2023-10-11 22:04:06,903][71635] Updated weights for policy 1, policy_version 75712 (0.0007) [2023-10-11 22:04:09,270][71601] Updated weights for policy 0, policy_version 75780 (0.0009) [2023-10-11 22:04:09,644][71601] Updated weights for policy 0, policy_version 75790 (0.0008) [2023-10-11 22:04:10,021][71601] Updated weights for policy 0, policy_version 75800 (0.0010) [2023-10-11 22:04:10,447][71635] Updated weights for policy 1, policy_version 75722 (0.0007) [2023-10-11 22:04:10,816][71635] Updated weights for policy 1, policy_version 75732 (0.0011) [2023-10-11 22:04:11,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 155156480. Throughput: 0: 1809.1, 1: 1821.7. Samples: 38801118. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-11 22:04:11,035][70582] Avg episode reward: [(0, '88.520'), (1, '96.370')] [2023-10-11 22:04:11,182][71635] Updated weights for policy 1, policy_version 75742 (0.0010) [2023-10-11 22:04:13,559][71601] Updated weights for policy 0, policy_version 75810 (0.0008) [2023-10-11 22:04:13,929][71601] Updated weights for policy 0, policy_version 75820 (0.0009) [2023-10-11 22:04:14,301][71601] Updated weights for policy 0, policy_version 75830 (0.0007) [2023-10-11 22:04:14,677][71601] Updated weights for policy 0, policy_version 75840 (0.0007) [2023-10-11 22:04:15,008][71635] Updated weights for policy 1, policy_version 75752 (0.0010) [2023-10-11 22:04:15,383][71635] Updated weights for policy 1, policy_version 75762 (0.0010) [2023-10-11 22:04:15,763][71635] Updated weights for policy 1, policy_version 75772 (0.0010) [2023-10-11 22:04:16,034][70582] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 155254784. Throughput: 0: 1816.5, 1: 1826.1. Samples: 38812938. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:04:16,035][70582] Avg episode reward: [(0, '86.110'), (1, '95.470')] [2023-10-11 22:04:18,289][71601] Updated weights for policy 0, policy_version 75850 (0.0007) [2023-10-11 22:04:18,666][71601] Updated weights for policy 0, policy_version 75860 (0.0011) [2023-10-11 22:04:19,034][71601] Updated weights for policy 0, policy_version 75870 (0.0010) [2023-10-11 22:04:19,269][71635] Updated weights for policy 1, policy_version 75782 (0.0007) [2023-10-11 22:04:19,640][71635] Updated weights for policy 1, policy_version 75792 (0.0008) [2023-10-11 22:04:20,013][71635] Updated weights for policy 1, policy_version 75802 (0.0011) [2023-10-11 22:04:21,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 155320320. Throughput: 0: 1808.3, 1: 1824.1. Samples: 38834028. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:04:21,035][70582] Avg episode reward: [(0, '88.400'), (1, '93.810')] [2023-10-11 22:04:22,801][71601] Updated weights for policy 0, policy_version 75880 (0.0008) [2023-10-11 22:04:23,179][71601] Updated weights for policy 0, policy_version 75890 (0.0007) [2023-10-11 22:04:23,547][71601] Updated weights for policy 0, policy_version 75900 (0.0008) [2023-10-11 22:04:23,870][71635] Updated weights for policy 1, policy_version 75812 (0.0010) [2023-10-11 22:04:24,237][71635] Updated weights for policy 1, policy_version 75822 (0.0009) [2023-10-11 22:04:24,596][71635] Updated weights for policy 1, policy_version 75832 (0.0008) [2023-10-11 22:04:26,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 155385856. Throughput: 0: 1814.6, 1: 1825.0. Samples: 38855728. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:04:26,034][70582] Avg episode reward: [(0, '81.650'), (1, '89.010')] [2023-10-11 22:04:27,281][71601] Updated weights for policy 0, policy_version 75910 (0.0008) [2023-10-11 22:04:27,656][71601] Updated weights for policy 0, policy_version 75920 (0.0008) [2023-10-11 22:04:28,023][71601] Updated weights for policy 0, policy_version 75930 (0.0009) [2023-10-11 22:04:28,276][71635] Updated weights for policy 1, policy_version 75842 (0.0010) [2023-10-11 22:04:28,642][71635] Updated weights for policy 1, policy_version 75852 (0.0009) [2023-10-11 22:04:29,003][71635] Updated weights for policy 1, policy_version 75862 (0.0010) [2023-10-11 22:04:29,364][71635] Updated weights for policy 1, policy_version 75872 (0.0009) [2023-10-11 22:04:31,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 155451392. Throughput: 0: 1819.0, 1: 1827.2. Samples: 38866946. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:04:31,034][70582] Avg episode reward: [(0, '77.380'), (1, '87.500')] [2023-10-11 22:04:31,698][71601] Updated weights for policy 0, policy_version 75940 (0.0008) [2023-10-11 22:04:32,064][71601] Updated weights for policy 0, policy_version 75950 (0.0010) [2023-10-11 22:04:32,443][71601] Updated weights for policy 0, policy_version 75960 (0.0009) [2023-10-11 22:04:33,078][71635] Updated weights for policy 1, policy_version 75882 (0.0010) [2023-10-11 22:04:33,447][71635] Updated weights for policy 1, policy_version 75892 (0.0009) [2023-10-11 22:04:33,814][71635] Updated weights for policy 1, policy_version 75902 (0.0008) [2023-10-11 22:04:36,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 155516928. Throughput: 0: 1813.7, 1: 1829.1. Samples: 38888282. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:04:36,034][70582] Avg episode reward: [(0, '79.590'), (1, '90.330')] [2023-10-11 22:04:36,113][71601] Updated weights for policy 0, policy_version 75970 (0.0009) [2023-10-11 22:04:36,493][71601] Updated weights for policy 0, policy_version 75980 (0.0009) [2023-10-11 22:04:36,858][71601] Updated weights for policy 0, policy_version 75990 (0.0008) [2023-10-11 22:04:37,231][71601] Updated weights for policy 0, policy_version 76000 (0.0007) [2023-10-11 22:04:37,462][71635] Updated weights for policy 1, policy_version 75912 (0.0008) [2023-10-11 22:04:37,830][71635] Updated weights for policy 1, policy_version 75922 (0.0008) [2023-10-11 22:04:38,205][71635] Updated weights for policy 1, policy_version 75932 (0.0008) [2023-10-11 22:04:40,902][71601] Updated weights for policy 0, policy_version 76010 (0.0007) [2023-10-11 22:04:41,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 155582464. Throughput: 0: 1812.9, 1: 1823.6. Samples: 38910908. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:04:41,034][70582] Avg episode reward: [(0, '76.200'), (1, '91.960')] [2023-10-11 22:04:41,263][71601] Updated weights for policy 0, policy_version 76020 (0.0010) [2023-10-11 22:04:41,635][71601] Updated weights for policy 0, policy_version 76030 (0.0010) [2023-10-11 22:04:42,065][71635] Updated weights for policy 1, policy_version 75942 (0.0011) [2023-10-11 22:04:42,444][71635] Updated weights for policy 1, policy_version 75952 (0.0009) [2023-10-11 22:04:42,813][71635] Updated weights for policy 1, policy_version 75962 (0.0010) [2023-10-11 22:04:45,296][71601] Updated weights for policy 0, policy_version 76040 (0.0008) [2023-10-11 22:04:45,660][71601] Updated weights for policy 0, policy_version 76050 (0.0007) [2023-10-11 22:04:46,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 155648000. Throughput: 0: 1812.4, 1: 1820.0. Samples: 38920742. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:04:46,034][70582] Avg episode reward: [(0, '81.030'), (1, '91.920')] [2023-10-11 22:04:46,042][71601] Updated weights for policy 0, policy_version 76060 (0.0009) [2023-10-11 22:04:46,473][71635] Updated weights for policy 1, policy_version 75972 (0.0008) [2023-10-11 22:04:46,833][71635] Updated weights for policy 1, policy_version 75982 (0.0009) [2023-10-11 22:04:47,205][71635] Updated weights for policy 1, policy_version 75992 (0.0008) [2023-10-11 22:04:49,732][71601] Updated weights for policy 0, policy_version 76070 (0.0007) [2023-10-11 22:04:50,112][71601] Updated weights for policy 0, policy_version 76080 (0.0008) [2023-10-11 22:04:50,484][71601] Updated weights for policy 0, policy_version 76090 (0.0010) [2023-10-11 22:04:51,004][71635] Updated weights for policy 1, policy_version 76002 (0.0009) [2023-10-11 22:04:51,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 155746304. Throughput: 0: 1813.5, 1: 1823.0. Samples: 38943456. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:04:51,036][70582] Avg episode reward: [(0, '80.290'), (1, '95.760')] [2023-10-11 22:04:51,362][71635] Updated weights for policy 1, policy_version 76012 (0.0008) [2023-10-11 22:04:51,721][71635] Updated weights for policy 1, policy_version 76022 (0.0010) [2023-10-11 22:04:52,088][71635] Updated weights for policy 1, policy_version 76032 (0.0009) [2023-10-11 22:04:54,126][71601] Updated weights for policy 0, policy_version 76100 (0.0009) [2023-10-11 22:04:54,498][71601] Updated weights for policy 0, policy_version 76110 (0.0009) [2023-10-11 22:04:54,875][71601] Updated weights for policy 0, policy_version 76120 (0.0010) [2023-10-11 22:04:55,857][71635] Updated weights for policy 1, policy_version 76042 (0.0008) [2023-10-11 22:04:56,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 155811840. Throughput: 0: 1817.9, 1: 1815.2. Samples: 38964606. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:04:56,034][70582] Avg episode reward: [(0, '78.760'), (1, '91.250')] [2023-10-11 22:04:56,226][71635] Updated weights for policy 1, policy_version 76052 (0.0008) [2023-10-11 22:04:56,599][71635] Updated weights for policy 1, policy_version 76062 (0.0007) [2023-10-11 22:04:58,638][71601] Updated weights for policy 0, policy_version 76130 (0.0008) [2023-10-11 22:04:59,008][71601] Updated weights for policy 0, policy_version 76140 (0.0009) [2023-10-11 22:04:59,371][71601] Updated weights for policy 0, policy_version 76150 (0.0009) [2023-10-11 22:04:59,743][71601] Updated weights for policy 0, policy_version 76160 (0.0008) [2023-10-11 22:05:00,096][71635] Updated weights for policy 1, policy_version 76072 (0.0007) [2023-10-11 22:05:00,461][71635] Updated weights for policy 1, policy_version 76082 (0.0007) [2023-10-11 22:05:00,825][71635] Updated weights for policy 1, policy_version 76092 (0.0007) [2023-10-11 22:05:01,034][70582] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 155910144. Throughput: 0: 1815.7, 1: 1811.0. Samples: 38976140. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:05:01,034][70582] Avg episode reward: [(0, '78.660'), (1, '94.890')] [2023-10-11 22:05:03,428][71601] Updated weights for policy 0, policy_version 76170 (0.0010) [2023-10-11 22:05:03,793][71601] Updated weights for policy 0, policy_version 76180 (0.0008) [2023-10-11 22:05:04,162][71601] Updated weights for policy 0, policy_version 76190 (0.0008) [2023-10-11 22:05:04,501][71635] Updated weights for policy 1, policy_version 76102 (0.0008) [2023-10-11 22:05:04,867][71635] Updated weights for policy 1, policy_version 76112 (0.0009) [2023-10-11 22:05:05,236][71635] Updated weights for policy 1, policy_version 76122 (0.0008) [2023-10-11 22:05:06,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 155975680. Throughput: 0: 1811.1, 1: 1812.7. Samples: 38997098. Policy #0 lag: (min: 10.0, avg: 10.4, max: 24.0) [2023-10-11 22:05:06,035][70582] Avg episode reward: [(0, '72.590'), (1, '97.180')] [2023-10-11 22:05:08,125][71601] Updated weights for policy 0, policy_version 76200 (0.0010) [2023-10-11 22:05:08,503][71601] Updated weights for policy 0, policy_version 76210 (0.0008) [2023-10-11 22:05:08,783][71635] Updated weights for policy 1, policy_version 76132 (0.0008) [2023-10-11 22:05:08,869][71601] Updated weights for policy 0, policy_version 76220 (0.0008) [2023-10-11 22:05:09,149][71635] Updated weights for policy 1, policy_version 76142 (0.0007) [2023-10-11 22:05:09,530][71635] Updated weights for policy 1, policy_version 76152 (0.0007) [2023-10-11 22:05:11,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 156041216. Throughput: 0: 1802.1, 1: 1812.9. Samples: 39018404. Policy #0 lag: (min: 10.0, avg: 10.4, max: 24.0) [2023-10-11 22:05:11,034][70582] Avg episode reward: [(0, '75.080'), (1, '105.390')] [2023-10-11 22:05:12,636][71601] Updated weights for policy 0, policy_version 76230 (0.0010) [2023-10-11 22:05:13,004][71601] Updated weights for policy 0, policy_version 76240 (0.0010) [2023-10-11 22:05:13,293][71635] Updated weights for policy 1, policy_version 76162 (0.0009) [2023-10-11 22:05:13,378][71601] Updated weights for policy 0, policy_version 76250 (0.0008) [2023-10-11 22:05:13,669][71635] Updated weights for policy 1, policy_version 76172 (0.0009) [2023-10-11 22:05:14,040][71635] Updated weights for policy 1, policy_version 76182 (0.0008) [2023-10-11 22:05:14,402][71635] Updated weights for policy 1, policy_version 76192 (0.0010) [2023-10-11 22:05:16,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 156106752. Throughput: 0: 1807.6, 1: 1812.8. Samples: 39029866. Policy #0 lag: (min: 10.0, avg: 10.4, max: 24.0) [2023-10-11 22:05:16,034][70582] Avg episode reward: [(0, '78.320'), (1, '99.630')] [2023-10-11 22:05:16,993][71601] Updated weights for policy 0, policy_version 76260 (0.0008) [2023-10-11 22:05:17,363][71601] Updated weights for policy 0, policy_version 76270 (0.0011) [2023-10-11 22:05:17,730][71601] Updated weights for policy 0, policy_version 76280 (0.0007) [2023-10-11 22:05:18,212][71635] Updated weights for policy 1, policy_version 76202 (0.0009) [2023-10-11 22:05:18,590][71635] Updated weights for policy 1, policy_version 76212 (0.0009) [2023-10-11 22:05:18,946][71635] Updated weights for policy 1, policy_version 76222 (0.0008) [2023-10-11 22:05:21,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 156172288. Throughput: 0: 1807.0, 1: 1807.5. Samples: 39050934. Policy #0 lag: (min: 10.0, avg: 10.4, max: 24.0) [2023-10-11 22:05:21,035][70582] Avg episode reward: [(0, '77.180'), (1, '102.050')] [2023-10-11 22:05:21,490][71601] Updated weights for policy 0, policy_version 76290 (0.0007) [2023-10-11 22:05:21,859][71601] Updated weights for policy 0, policy_version 76300 (0.0007) [2023-10-11 22:05:22,220][71601] Updated weights for policy 0, policy_version 76310 (0.0008) [2023-10-11 22:05:22,593][71601] Updated weights for policy 0, policy_version 76320 (0.0008) [2023-10-11 22:05:22,693][71635] Updated weights for policy 1, policy_version 76232 (0.0007) [2023-10-11 22:05:23,060][71635] Updated weights for policy 1, policy_version 76242 (0.0007) [2023-10-11 22:05:23,429][71635] Updated weights for policy 1, policy_version 76252 (0.0008) [2023-10-11 22:05:26,016][71601] Updated weights for policy 0, policy_version 76330 (0.0008) [2023-10-11 22:05:26,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 156237824. Throughput: 0: 1816.2, 1: 1803.5. Samples: 39073794. Policy #0 lag: (min: 10.0, avg: 10.4, max: 24.0) [2023-10-11 22:05:26,035][70582] Avg episode reward: [(0, '74.410'), (1, '106.500')] [2023-10-11 22:05:26,385][71601] Updated weights for policy 0, policy_version 76340 (0.0008) [2023-10-11 22:05:26,756][71601] Updated weights for policy 0, policy_version 76350 (0.0009) [2023-10-11 22:05:27,301][71635] Updated weights for policy 1, policy_version 76262 (0.0009) [2023-10-11 22:05:27,681][71635] Updated weights for policy 1, policy_version 76272 (0.0008) [2023-10-11 22:05:28,051][71635] Updated weights for policy 1, policy_version 76282 (0.0008) [2023-10-11 22:05:30,462][71601] Updated weights for policy 0, policy_version 76360 (0.0010) [2023-10-11 22:05:30,839][71601] Updated weights for policy 0, policy_version 76370 (0.0008) [2023-10-11 22:05:31,034][70582] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 156303360. Throughput: 0: 1816.0, 1: 1801.1. Samples: 39083512. Policy #0 lag: (min: 10.0, avg: 10.4, max: 24.0) [2023-10-11 22:05:31,034][70582] Avg episode reward: [(0, '78.460'), (1, '104.870')] [2023-10-11 22:05:31,220][71601] Updated weights for policy 0, policy_version 76380 (0.0009) [2023-10-11 22:05:31,751][71635] Updated weights for policy 1, policy_version 76292 (0.0008) [2023-10-11 22:05:32,114][71635] Updated weights for policy 1, policy_version 76302 (0.0008) [2023-10-11 22:05:32,482][71635] Updated weights for policy 1, policy_version 76312 (0.0011) [2023-10-11 22:05:34,906][71601] Updated weights for policy 0, policy_version 76390 (0.0009) [2023-10-11 22:05:35,267][71601] Updated weights for policy 0, policy_version 76400 (0.0009) [2023-10-11 22:05:35,644][71601] Updated weights for policy 0, policy_version 76410 (0.0008) [2023-10-11 22:05:36,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 156401664. Throughput: 0: 1817.4, 1: 1795.2. Samples: 39106022. Policy #0 lag: (min: 10.0, avg: 10.4, max: 24.0) [2023-10-11 22:05:36,034][70582] Avg episode reward: [(0, '78.250'), (1, '107.450')] [2023-10-11 22:05:36,154][71635] Updated weights for policy 1, policy_version 76322 (0.0011) [2023-10-11 22:05:36,519][71635] Updated weights for policy 1, policy_version 76332 (0.0008) [2023-10-11 22:05:36,890][71635] Updated weights for policy 1, policy_version 76342 (0.0008) [2023-10-11 22:05:37,252][71635] Updated weights for policy 1, policy_version 76352 (0.0007) [2023-10-11 22:05:39,370][71601] Updated weights for policy 0, policy_version 76420 (0.0009) [2023-10-11 22:05:39,740][71601] Updated weights for policy 0, policy_version 76430 (0.0010) [2023-10-11 22:05:40,107][71601] Updated weights for policy 0, policy_version 76440 (0.0010) [2023-10-11 22:05:41,031][71635] Updated weights for policy 1, policy_version 76362 (0.0009) [2023-10-11 22:05:41,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 156467200. Throughput: 0: 1814.3, 1: 1805.9. Samples: 39127516. Policy #0 lag: (min: 10.0, avg: 10.4, max: 24.0) [2023-10-11 22:05:41,034][70582] Avg episode reward: [(0, '78.540'), (1, '107.470')] [2023-10-11 22:05:41,394][71635] Updated weights for policy 1, policy_version 76372 (0.0008) [2023-10-11 22:05:41,765][71635] Updated weights for policy 1, policy_version 76382 (0.0008) [2023-10-11 22:05:43,775][71601] Updated weights for policy 0, policy_version 76450 (0.0009) [2023-10-11 22:05:44,147][71601] Updated weights for policy 0, policy_version 76460 (0.0009) [2023-10-11 22:05:44,529][71601] Updated weights for policy 0, policy_version 76470 (0.0007) [2023-10-11 22:05:44,903][71601] Updated weights for policy 0, policy_version 76480 (0.0009) [2023-10-11 22:05:45,351][71635] Updated weights for policy 1, policy_version 76392 (0.0010) [2023-10-11 22:05:45,724][71635] Updated weights for policy 1, policy_version 76402 (0.0010) [2023-10-11 22:05:46,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 156532736. Throughput: 0: 1815.7, 1: 1803.3. Samples: 39138996. Policy #0 lag: (min: 10.0, avg: 10.4, max: 24.0) [2023-10-11 22:05:46,034][70582] Avg episode reward: [(0, '81.180'), (1, '107.110')] [2023-10-11 22:05:46,086][71635] Updated weights for policy 1, policy_version 76412 (0.0011) [2023-10-11 22:05:48,601][71601] Updated weights for policy 0, policy_version 76490 (0.0009) [2023-10-11 22:05:48,971][71601] Updated weights for policy 0, policy_version 76500 (0.0011) [2023-10-11 22:05:49,348][71601] Updated weights for policy 0, policy_version 76510 (0.0007) [2023-10-11 22:05:49,896][71635] Updated weights for policy 1, policy_version 76422 (0.0010) [2023-10-11 22:05:50,262][71635] Updated weights for policy 1, policy_version 76432 (0.0009) [2023-10-11 22:05:50,630][71635] Updated weights for policy 1, policy_version 76442 (0.0007) [2023-10-11 22:05:51,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 156631040. Throughput: 0: 1817.5, 1: 1804.6. Samples: 39160092. Policy #0 lag: (min: 10.0, avg: 10.4, max: 24.0) [2023-10-11 22:05:51,034][70582] Avg episode reward: [(0, '81.240'), (1, '107.730')] [2023-10-11 22:05:53,092][71601] Updated weights for policy 0, policy_version 76520 (0.0008) [2023-10-11 22:05:53,468][71601] Updated weights for policy 0, policy_version 76530 (0.0008) [2023-10-11 22:05:53,844][71601] Updated weights for policy 0, policy_version 76540 (0.0008) [2023-10-11 22:05:54,312][71635] Updated weights for policy 1, policy_version 76452 (0.0009) [2023-10-11 22:05:54,684][71635] Updated weights for policy 1, policy_version 76462 (0.0007) [2023-10-11 22:05:55,048][71635] Updated weights for policy 1, policy_version 76472 (0.0008) [2023-10-11 22:05:56,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 156696576. Throughput: 0: 1815.3, 1: 1801.0. Samples: 39181140. Policy #0 lag: (min: 17.0, avg: 21.2, max: 49.0) [2023-10-11 22:05:56,034][70582] Avg episode reward: [(0, '80.270'), (1, '110.920')] [2023-10-11 22:05:56,042][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000076544_78381056.pth... [2023-10-11 22:05:56,042][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000076480_78315520.pth... [2023-10-11 22:05:56,073][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000074848_76644352.pth [2023-10-11 22:05:56,077][71353] Saving a milestone ./train_atari/atari_gopher_APPO/checkpoint_p0/milestones/checkpoint_000076544_78381056.pth [2023-10-11 22:05:56,078][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000074784_76578816.pth [2023-10-11 22:05:56,082][71431] Saving a milestone ./train_atari/atari_gopher_APPO/checkpoint_p1/milestones/checkpoint_000076480_78315520.pth [2023-10-11 22:05:57,540][71601] Updated weights for policy 0, policy_version 76550 (0.0008) [2023-10-11 22:05:57,909][71601] Updated weights for policy 0, policy_version 76560 (0.0008) [2023-10-11 22:05:58,278][71601] Updated weights for policy 0, policy_version 76570 (0.0008) [2023-10-11 22:05:58,669][71635] Updated weights for policy 1, policy_version 76482 (0.0008) [2023-10-11 22:05:59,026][71635] Updated weights for policy 1, policy_version 76492 (0.0009) [2023-10-11 22:05:59,400][71635] Updated weights for policy 1, policy_version 76502 (0.0008) [2023-10-11 22:05:59,760][71635] Updated weights for policy 1, policy_version 76512 (0.0009) [2023-10-11 22:06:01,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 156762112. Throughput: 0: 1814.1, 1: 1807.0. Samples: 39192818. Policy #0 lag: (min: 17.0, avg: 21.2, max: 49.0) [2023-10-11 22:06:01,034][70582] Avg episode reward: [(0, '77.490'), (1, '106.210')] [2023-10-11 22:06:01,826][71601] Updated weights for policy 0, policy_version 76580 (0.0008) [2023-10-11 22:06:02,197][71601] Updated weights for policy 0, policy_version 76590 (0.0008) [2023-10-11 22:06:02,568][71601] Updated weights for policy 0, policy_version 76600 (0.0008) [2023-10-11 22:06:03,464][71635] Updated weights for policy 1, policy_version 76522 (0.0008) [2023-10-11 22:06:03,824][71635] Updated weights for policy 1, policy_version 76532 (0.0008) [2023-10-11 22:06:04,201][71635] Updated weights for policy 1, policy_version 76542 (0.0010) [2023-10-11 22:06:06,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 156827648. Throughput: 0: 1816.2, 1: 1808.0. Samples: 39214024. Policy #0 lag: (min: 17.0, avg: 21.2, max: 49.0) [2023-10-11 22:06:06,035][70582] Avg episode reward: [(0, '82.350'), (1, '106.080')] [2023-10-11 22:06:06,330][71601] Updated weights for policy 0, policy_version 76610 (0.0008) [2023-10-11 22:06:06,703][71601] Updated weights for policy 0, policy_version 76620 (0.0008) [2023-10-11 22:06:07,079][71601] Updated weights for policy 0, policy_version 76630 (0.0007) [2023-10-11 22:06:07,457][71601] Updated weights for policy 0, policy_version 76640 (0.0009) [2023-10-11 22:06:07,828][71635] Updated weights for policy 1, policy_version 76552 (0.0010) [2023-10-11 22:06:08,201][71635] Updated weights for policy 1, policy_version 76562 (0.0009) [2023-10-11 22:06:08,569][71635] Updated weights for policy 1, policy_version 76572 (0.0008) [2023-10-11 22:06:11,034][70582] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 156893184. Throughput: 0: 1810.2, 1: 1814.4. Samples: 39236902. Policy #0 lag: (min: 17.0, avg: 21.2, max: 49.0) [2023-10-11 22:06:11,035][70582] Avg episode reward: [(0, '82.060'), (1, '105.830')] [2023-10-11 22:06:11,298][71601] Updated weights for policy 0, policy_version 76650 (0.0008) [2023-10-11 22:06:11,669][71601] Updated weights for policy 0, policy_version 76660 (0.0010) [2023-10-11 22:06:12,043][71601] Updated weights for policy 0, policy_version 76670 (0.0008) [2023-10-11 22:06:12,403][71635] Updated weights for policy 1, policy_version 76582 (0.0009) [2023-10-11 22:06:12,778][71635] Updated weights for policy 1, policy_version 76592 (0.0009) [2023-10-11 22:06:13,147][71635] Updated weights for policy 1, policy_version 76602 (0.0009) [2023-10-11 22:06:15,816][71601] Updated weights for policy 0, policy_version 76680 (0.0009) [2023-10-11 22:06:16,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 156958720. Throughput: 0: 1809.9, 1: 1817.4. Samples: 39246738. Policy #0 lag: (min: 17.0, avg: 21.2, max: 49.0) [2023-10-11 22:06:16,034][70582] Avg episode reward: [(0, '76.660'), (1, '100.630')] [2023-10-11 22:06:16,190][71601] Updated weights for policy 0, policy_version 76690 (0.0010) [2023-10-11 22:06:16,571][71601] Updated weights for policy 0, policy_version 76700 (0.0010) [2023-10-11 22:06:16,870][71635] Updated weights for policy 1, policy_version 76612 (0.0011) [2023-10-11 22:06:17,245][71635] Updated weights for policy 1, policy_version 76622 (0.0010) [2023-10-11 22:06:17,615][71635] Updated weights for policy 1, policy_version 76632 (0.0009) [2023-10-11 22:06:20,288][71601] Updated weights for policy 0, policy_version 76710 (0.0009) [2023-10-11 22:06:20,656][71601] Updated weights for policy 0, policy_version 76720 (0.0009) [2023-10-11 22:06:21,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 157024256. Throughput: 0: 1807.9, 1: 1815.4. Samples: 39269068. Policy #0 lag: (min: 17.0, avg: 21.2, max: 49.0) [2023-10-11 22:06:21,035][70582] Avg episode reward: [(0, '82.020'), (1, '100.790')] [2023-10-11 22:06:21,041][71601] Updated weights for policy 0, policy_version 76730 (0.0011) [2023-10-11 22:06:21,332][71635] Updated weights for policy 1, policy_version 76642 (0.0008) [2023-10-11 22:06:21,707][71635] Updated weights for policy 1, policy_version 76652 (0.0009) [2023-10-11 22:06:22,066][71635] Updated weights for policy 1, policy_version 76662 (0.0008) [2023-10-11 22:06:22,437][71635] Updated weights for policy 1, policy_version 76672 (0.0008) [2023-10-11 22:06:24,591][71601] Updated weights for policy 0, policy_version 76740 (0.0008) [2023-10-11 22:06:24,964][71601] Updated weights for policy 0, policy_version 76750 (0.0008) [2023-10-11 22:06:25,338][71601] Updated weights for policy 0, policy_version 76760 (0.0009) [2023-10-11 22:06:26,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 157122560. Throughput: 0: 1814.5, 1: 1804.7. Samples: 39290378. Policy #0 lag: (min: 17.0, avg: 21.2, max: 49.0) [2023-10-11 22:06:26,034][70582] Avg episode reward: [(0, '84.120'), (1, '100.670')] [2023-10-11 22:06:26,257][71635] Updated weights for policy 1, policy_version 76682 (0.0007) [2023-10-11 22:06:26,627][71635] Updated weights for policy 1, policy_version 76692 (0.0009) [2023-10-11 22:06:27,008][71635] Updated weights for policy 1, policy_version 76702 (0.0010) [2023-10-11 22:06:29,119][71601] Updated weights for policy 0, policy_version 76770 (0.0009) [2023-10-11 22:06:29,490][71601] Updated weights for policy 0, policy_version 76780 (0.0010) [2023-10-11 22:06:29,867][71601] Updated weights for policy 0, policy_version 76790 (0.0008) [2023-10-11 22:06:30,244][71601] Updated weights for policy 0, policy_version 76800 (0.0008) [2023-10-11 22:06:30,625][71635] Updated weights for policy 1, policy_version 76712 (0.0008) [2023-10-11 22:06:30,990][71635] Updated weights for policy 1, policy_version 76722 (0.0009) [2023-10-11 22:06:31,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 157188096. Throughput: 0: 1805.5, 1: 1803.6. Samples: 39301406. Policy #0 lag: (min: 17.0, avg: 21.2, max: 49.0) [2023-10-11 22:06:31,034][70582] Avg episode reward: [(0, '81.080'), (1, '99.390')] [2023-10-11 22:06:31,366][71635] Updated weights for policy 1, policy_version 76732 (0.0008) [2023-10-11 22:06:33,913][71601] Updated weights for policy 0, policy_version 76810 (0.0010) [2023-10-11 22:06:34,291][71601] Updated weights for policy 0, policy_version 76820 (0.0007) [2023-10-11 22:06:34,661][71601] Updated weights for policy 0, policy_version 76830 (0.0009) [2023-10-11 22:06:35,147][71635] Updated weights for policy 1, policy_version 76742 (0.0007) [2023-10-11 22:06:35,513][71635] Updated weights for policy 1, policy_version 76752 (0.0008) [2023-10-11 22:06:35,876][71635] Updated weights for policy 1, policy_version 76762 (0.0007) [2023-10-11 22:06:36,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 157253632. Throughput: 0: 1816.7, 1: 1804.4. Samples: 39323040. Policy #0 lag: (min: 17.0, avg: 21.2, max: 49.0) [2023-10-11 22:06:36,035][70582] Avg episode reward: [(0, '83.990'), (1, '100.930')] [2023-10-11 22:06:38,461][71601] Updated weights for policy 0, policy_version 76840 (0.0008) [2023-10-11 22:06:38,839][71601] Updated weights for policy 0, policy_version 76850 (0.0009) [2023-10-11 22:06:39,207][71601] Updated weights for policy 0, policy_version 76860 (0.0009) [2023-10-11 22:06:39,619][71635] Updated weights for policy 1, policy_version 76772 (0.0009) [2023-10-11 22:06:39,984][71635] Updated weights for policy 1, policy_version 76782 (0.0008) [2023-10-11 22:06:40,351][71635] Updated weights for policy 1, policy_version 76792 (0.0009) [2023-10-11 22:06:41,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 157351936. Throughput: 0: 1812.6, 1: 1811.6. Samples: 39344230. Policy #0 lag: (min: 17.0, avg: 21.2, max: 49.0) [2023-10-11 22:06:41,034][70582] Avg episode reward: [(0, '82.000'), (1, '102.240')] [2023-10-11 22:06:42,848][71601] Updated weights for policy 0, policy_version 76870 (0.0010) [2023-10-11 22:06:43,235][71601] Updated weights for policy 0, policy_version 76880 (0.0010) [2023-10-11 22:06:43,607][71601] Updated weights for policy 0, policy_version 76890 (0.0008) [2023-10-11 22:06:44,149][71635] Updated weights for policy 1, policy_version 76802 (0.0010) [2023-10-11 22:06:44,515][71635] Updated weights for policy 1, policy_version 76812 (0.0011) [2023-10-11 22:06:44,876][71635] Updated weights for policy 1, policy_version 76822 (0.0010) [2023-10-11 22:06:45,244][71635] Updated weights for policy 1, policy_version 76832 (0.0008) [2023-10-11 22:06:46,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 157417472. Throughput: 0: 1820.2, 1: 1798.8. Samples: 39355676. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-11 22:06:46,035][70582] Avg episode reward: [(0, '82.970'), (1, '95.170')] [2023-10-11 22:06:47,287][71601] Updated weights for policy 0, policy_version 76900 (0.0009) [2023-10-11 22:06:47,652][71601] Updated weights for policy 0, policy_version 76910 (0.0010) [2023-10-11 22:06:48,023][71601] Updated weights for policy 0, policy_version 76920 (0.0010) [2023-10-11 22:06:49,010][71635] Updated weights for policy 1, policy_version 76842 (0.0009) [2023-10-11 22:06:49,368][71635] Updated weights for policy 1, policy_version 76852 (0.0008) [2023-10-11 22:06:49,731][71635] Updated weights for policy 1, policy_version 76862 (0.0009) [2023-10-11 22:06:51,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 157483008. Throughput: 0: 1814.1, 1: 1810.1. Samples: 39377114. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-11 22:06:51,034][70582] Avg episode reward: [(0, '83.300'), (1, '93.360')] [2023-10-11 22:06:51,813][71601] Updated weights for policy 0, policy_version 76930 (0.0008) [2023-10-11 22:06:52,180][71601] Updated weights for policy 0, policy_version 76940 (0.0007) [2023-10-11 22:06:52,545][71601] Updated weights for policy 0, policy_version 76950 (0.0011) [2023-10-11 22:06:52,916][71601] Updated weights for policy 0, policy_version 76960 (0.0008) [2023-10-11 22:06:53,446][71635] Updated weights for policy 1, policy_version 76872 (0.0008) [2023-10-11 22:06:53,816][71635] Updated weights for policy 1, policy_version 76882 (0.0009) [2023-10-11 22:06:54,180][71635] Updated weights for policy 1, policy_version 76892 (0.0008) [2023-10-11 22:06:56,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 157548544. Throughput: 0: 1811.2, 1: 1792.1. Samples: 39399048. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-11 22:06:56,035][70582] Avg episode reward: [(0, '80.860'), (1, '92.590')] [2023-10-11 22:06:56,578][71601] Updated weights for policy 0, policy_version 76970 (0.0008) [2023-10-11 22:06:56,953][71601] Updated weights for policy 0, policy_version 76980 (0.0007) [2023-10-11 22:06:57,328][71601] Updated weights for policy 0, policy_version 76990 (0.0007) [2023-10-11 22:06:58,117][71635] Updated weights for policy 1, policy_version 76902 (0.0007) [2023-10-11 22:06:58,503][71635] Updated weights for policy 1, policy_version 76912 (0.0007) [2023-10-11 22:06:58,865][71635] Updated weights for policy 1, policy_version 76922 (0.0008) [2023-10-11 22:07:00,789][71601] Updated weights for policy 0, policy_version 77000 (0.0009) [2023-10-11 22:07:01,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 157614080. Throughput: 0: 1814.5, 1: 1812.3. Samples: 39409946. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-11 22:07:01,035][70582] Avg episode reward: [(0, '85.650'), (1, '97.020')] [2023-10-11 22:07:01,154][71601] Updated weights for policy 0, policy_version 77010 (0.0008) [2023-10-11 22:07:01,534][71601] Updated weights for policy 0, policy_version 77020 (0.0009) [2023-10-11 22:07:02,293][71635] Updated weights for policy 1, policy_version 76932 (0.0008) [2023-10-11 22:07:02,667][71635] Updated weights for policy 1, policy_version 76942 (0.0008) [2023-10-11 22:07:03,030][71635] Updated weights for policy 1, policy_version 76952 (0.0008) [2023-10-11 22:07:05,225][71601] Updated weights for policy 0, policy_version 77030 (0.0008) [2023-10-11 22:07:05,595][71601] Updated weights for policy 0, policy_version 77040 (0.0007) [2023-10-11 22:07:05,967][71601] Updated weights for policy 0, policy_version 77050 (0.0011) [2023-10-11 22:07:06,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 157679616. Throughput: 0: 1825.1, 1: 1800.0. Samples: 39432194. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-11 22:07:06,034][70582] Avg episode reward: [(0, '84.990'), (1, '90.730')] [2023-10-11 22:07:06,682][71635] Updated weights for policy 1, policy_version 76962 (0.0012) [2023-10-11 22:07:07,057][71635] Updated weights for policy 1, policy_version 76972 (0.0009) [2023-10-11 22:07:07,419][71635] Updated weights for policy 1, policy_version 76982 (0.0010) [2023-10-11 22:07:07,781][71635] Updated weights for policy 1, policy_version 76992 (0.0010) [2023-10-11 22:07:09,630][71601] Updated weights for policy 0, policy_version 77060 (0.0009) [2023-10-11 22:07:09,993][71601] Updated weights for policy 0, policy_version 77070 (0.0010) [2023-10-11 22:07:10,376][71601] Updated weights for policy 0, policy_version 77080 (0.0010) [2023-10-11 22:07:11,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 157777920. Throughput: 0: 1826.8, 1: 1808.8. Samples: 39453980. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-11 22:07:11,034][70582] Avg episode reward: [(0, '86.050'), (1, '91.750')] [2023-10-11 22:07:11,483][71635] Updated weights for policy 1, policy_version 77002 (0.0008) [2023-10-11 22:07:11,850][71635] Updated weights for policy 1, policy_version 77012 (0.0009) [2023-10-11 22:07:12,222][71635] Updated weights for policy 1, policy_version 77022 (0.0010) [2023-10-11 22:07:14,039][71601] Updated weights for policy 0, policy_version 77090 (0.0009) [2023-10-11 22:07:14,414][71601] Updated weights for policy 0, policy_version 77100 (0.0009) [2023-10-11 22:07:14,789][71601] Updated weights for policy 0, policy_version 77110 (0.0007) [2023-10-11 22:07:15,159][71601] Updated weights for policy 0, policy_version 77120 (0.0007) [2023-10-11 22:07:15,856][71635] Updated weights for policy 1, policy_version 77032 (0.0010) [2023-10-11 22:07:16,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 157843456. Throughput: 0: 1831.9, 1: 1809.8. Samples: 39465280. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-11 22:07:16,034][70582] Avg episode reward: [(0, '88.790'), (1, '85.560')] [2023-10-11 22:07:16,213][71635] Updated weights for policy 1, policy_version 77042 (0.0009) [2023-10-11 22:07:16,578][71635] Updated weights for policy 1, policy_version 77052 (0.0008) [2023-10-11 22:07:18,769][71601] Updated weights for policy 0, policy_version 77130 (0.0008) [2023-10-11 22:07:19,139][71601] Updated weights for policy 0, policy_version 77140 (0.0008) [2023-10-11 22:07:19,525][71601] Updated weights for policy 0, policy_version 77150 (0.0010) [2023-10-11 22:07:20,431][71635] Updated weights for policy 1, policy_version 77062 (0.0008) [2023-10-11 22:07:20,801][71635] Updated weights for policy 1, policy_version 77072 (0.0011) [2023-10-11 22:07:21,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 157908992. Throughput: 0: 1827.2, 1: 1812.8. Samples: 39486840. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-11 22:07:21,034][70582] Avg episode reward: [(0, '85.120'), (1, '90.010')] [2023-10-11 22:07:21,167][71635] Updated weights for policy 1, policy_version 77082 (0.0012) [2023-10-11 22:07:23,139][71601] Updated weights for policy 0, policy_version 77160 (0.0008) [2023-10-11 22:07:23,499][71601] Updated weights for policy 0, policy_version 77170 (0.0007) [2023-10-11 22:07:23,866][71601] Updated weights for policy 0, policy_version 77180 (0.0008) [2023-10-11 22:07:24,923][71635] Updated weights for policy 1, policy_version 77092 (0.0009) [2023-10-11 22:07:25,287][71635] Updated weights for policy 1, policy_version 77102 (0.0007) [2023-10-11 22:07:25,647][71635] Updated weights for policy 1, policy_version 77112 (0.0008) [2023-10-11 22:07:26,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 158007296. Throughput: 0: 1835.3, 1: 1818.7. Samples: 39508660. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-11 22:07:26,035][70582] Avg episode reward: [(0, '86.020'), (1, '92.170')] [2023-10-11 22:07:27,546][71601] Updated weights for policy 0, policy_version 77190 (0.0010) [2023-10-11 22:07:27,936][71601] Updated weights for policy 0, policy_version 77200 (0.0007) [2023-10-11 22:07:28,310][71601] Updated weights for policy 0, policy_version 77210 (0.0009) [2023-10-11 22:07:29,376][71635] Updated weights for policy 1, policy_version 77122 (0.0008) [2023-10-11 22:07:29,732][71635] Updated weights for policy 1, policy_version 77132 (0.0008) [2023-10-11 22:07:30,096][71635] Updated weights for policy 1, policy_version 77142 (0.0009) [2023-10-11 22:07:30,463][71635] Updated weights for policy 1, policy_version 77152 (0.0009) [2023-10-11 22:07:31,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 158072832. Throughput: 0: 1828.7, 1: 1811.6. Samples: 39519486. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-11 22:07:31,034][70582] Avg episode reward: [(0, '85.730'), (1, '93.200')] [2023-10-11 22:07:31,961][71601] Updated weights for policy 0, policy_version 77220 (0.0010) [2023-10-11 22:07:32,341][71601] Updated weights for policy 0, policy_version 77230 (0.0009) [2023-10-11 22:07:32,712][71601] Updated weights for policy 0, policy_version 77240 (0.0007) [2023-10-11 22:07:34,324][71635] Updated weights for policy 1, policy_version 77162 (0.0008) [2023-10-11 22:07:34,690][71635] Updated weights for policy 1, policy_version 77172 (0.0012) [2023-10-11 22:07:35,059][71635] Updated weights for policy 1, policy_version 77182 (0.0008) [2023-10-11 22:07:36,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 158138368. Throughput: 0: 1835.2, 1: 1815.6. Samples: 39541402. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:07:36,034][70582] Avg episode reward: [(0, '88.670'), (1, '94.650')] [2023-10-11 22:07:36,400][71601] Updated weights for policy 0, policy_version 77250 (0.0007) [2023-10-11 22:07:36,783][71601] Updated weights for policy 0, policy_version 77260 (0.0009) [2023-10-11 22:07:37,154][71601] Updated weights for policy 0, policy_version 77270 (0.0008) [2023-10-11 22:07:37,526][71601] Updated weights for policy 0, policy_version 77280 (0.0010) [2023-10-11 22:07:38,807][71635] Updated weights for policy 1, policy_version 77192 (0.0010) [2023-10-11 22:07:39,167][71635] Updated weights for policy 1, policy_version 77202 (0.0010) [2023-10-11 22:07:39,537][71635] Updated weights for policy 1, policy_version 77212 (0.0009) [2023-10-11 22:07:41,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 158203904. Throughput: 0: 1842.9, 1: 1804.6. Samples: 39563186. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:07:41,035][70582] Avg episode reward: [(0, '85.670'), (1, '89.150')] [2023-10-11 22:07:41,139][71601] Updated weights for policy 0, policy_version 77290 (0.0010) [2023-10-11 22:07:41,502][71601] Updated weights for policy 0, policy_version 77300 (0.0009) [2023-10-11 22:07:41,872][71601] Updated weights for policy 0, policy_version 77310 (0.0010) [2023-10-11 22:07:43,279][71635] Updated weights for policy 1, policy_version 77222 (0.0008) [2023-10-11 22:07:43,660][71635] Updated weights for policy 1, policy_version 77232 (0.0007) [2023-10-11 22:07:44,026][71635] Updated weights for policy 1, policy_version 77242 (0.0010) [2023-10-11 22:07:45,566][71601] Updated weights for policy 0, policy_version 77320 (0.0008) [2023-10-11 22:07:45,938][71601] Updated weights for policy 0, policy_version 77330 (0.0008) [2023-10-11 22:07:46,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 158269440. Throughput: 0: 1837.1, 1: 1809.2. Samples: 39574030. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:07:46,035][70582] Avg episode reward: [(0, '85.600'), (1, '90.570')] [2023-10-11 22:07:46,306][71601] Updated weights for policy 0, policy_version 77340 (0.0008) [2023-10-11 22:07:47,821][71635] Updated weights for policy 1, policy_version 77252 (0.0008) [2023-10-11 22:07:48,180][71635] Updated weights for policy 1, policy_version 77262 (0.0009) [2023-10-11 22:07:48,547][71635] Updated weights for policy 1, policy_version 77272 (0.0007) [2023-10-11 22:07:50,040][71601] Updated weights for policy 0, policy_version 77350 (0.0008) [2023-10-11 22:07:50,407][71601] Updated weights for policy 0, policy_version 77360 (0.0009) [2023-10-11 22:07:50,777][71601] Updated weights for policy 0, policy_version 77370 (0.0009) [2023-10-11 22:07:51,034][70582] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 158367744. Throughput: 0: 1830.4, 1: 1802.8. Samples: 39595686. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:07:51,034][70582] Avg episode reward: [(0, '85.310'), (1, '89.670')] [2023-10-11 22:07:52,392][71635] Updated weights for policy 1, policy_version 77282 (0.0007) [2023-10-11 22:07:52,763][71635] Updated weights for policy 1, policy_version 77292 (0.0008) [2023-10-11 22:07:53,129][71635] Updated weights for policy 1, policy_version 77302 (0.0009) [2023-10-11 22:07:53,505][71635] Updated weights for policy 1, policy_version 77312 (0.0010) [2023-10-11 22:07:54,497][71601] Updated weights for policy 0, policy_version 77380 (0.0009) [2023-10-11 22:07:54,865][71601] Updated weights for policy 0, policy_version 77390 (0.0007) [2023-10-11 22:07:55,240][71601] Updated weights for policy 0, policy_version 77400 (0.0007) [2023-10-11 22:07:56,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 158433280. Throughput: 0: 1824.1, 1: 1796.6. Samples: 39616912. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:07:56,034][70582] Avg episode reward: [(0, '78.700'), (1, '95.920')] [2023-10-11 22:07:56,043][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000077408_79265792.pth... [2023-10-11 22:07:56,044][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000077312_79167488.pth... [2023-10-11 22:07:56,073][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000075712_77529088.pth [2023-10-11 22:07:56,079][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000075616_77430784.pth [2023-10-11 22:07:57,281][71635] Updated weights for policy 1, policy_version 77322 (0.0008) [2023-10-11 22:07:57,657][71635] Updated weights for policy 1, policy_version 77332 (0.0009) [2023-10-11 22:07:58,028][71635] Updated weights for policy 1, policy_version 77342 (0.0009) [2023-10-11 22:07:59,001][71601] Updated weights for policy 0, policy_version 77410 (0.0008) [2023-10-11 22:07:59,368][71601] Updated weights for policy 0, policy_version 77420 (0.0009) [2023-10-11 22:07:59,745][71601] Updated weights for policy 0, policy_version 77430 (0.0010) [2023-10-11 22:08:00,115][71601] Updated weights for policy 0, policy_version 77440 (0.0011) [2023-10-11 22:08:01,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 158498816. Throughput: 0: 1819.4, 1: 1794.4. Samples: 39627900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:08:01,034][70582] Avg episode reward: [(0, '78.620'), (1, '96.760')] [2023-10-11 22:08:01,663][71635] Updated weights for policy 1, policy_version 77352 (0.0010) [2023-10-11 22:08:02,036][71635] Updated weights for policy 1, policy_version 77362 (0.0009) [2023-10-11 22:08:02,399][71635] Updated weights for policy 1, policy_version 77372 (0.0010) [2023-10-11 22:08:03,662][71601] Updated weights for policy 0, policy_version 77450 (0.0010) [2023-10-11 22:08:04,036][71601] Updated weights for policy 0, policy_version 77460 (0.0010) [2023-10-11 22:08:04,414][71601] Updated weights for policy 0, policy_version 77470 (0.0009) [2023-10-11 22:08:06,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 158564352. Throughput: 0: 1815.6, 1: 1795.9. Samples: 39649358. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:08:06,035][70582] Avg episode reward: [(0, '84.860'), (1, '97.890')] [2023-10-11 22:08:06,188][71635] Updated weights for policy 1, policy_version 77382 (0.0010) [2023-10-11 22:08:06,552][71635] Updated weights for policy 1, policy_version 77392 (0.0009) [2023-10-11 22:08:06,931][71635] Updated weights for policy 1, policy_version 77402 (0.0008) [2023-10-11 22:08:08,354][71601] Updated weights for policy 0, policy_version 77480 (0.0012) [2023-10-11 22:08:08,731][71601] Updated weights for policy 0, policy_version 77490 (0.0010) [2023-10-11 22:08:09,092][71601] Updated weights for policy 0, policy_version 77500 (0.0010) [2023-10-11 22:08:10,369][71635] Updated weights for policy 1, policy_version 77412 (0.0008) [2023-10-11 22:08:10,745][71635] Updated weights for policy 1, policy_version 77422 (0.0009) [2023-10-11 22:08:11,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 158629888. Throughput: 0: 1805.5, 1: 1810.0. Samples: 39671354. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:08:11,034][70582] Avg episode reward: [(0, '87.480'), (1, '100.710')] [2023-10-11 22:08:11,106][71635] Updated weights for policy 1, policy_version 77432 (0.0007) [2023-10-11 22:08:12,919][71601] Updated weights for policy 0, policy_version 77510 (0.0009) [2023-10-11 22:08:13,297][71601] Updated weights for policy 0, policy_version 77520 (0.0008) [2023-10-11 22:08:13,671][71601] Updated weights for policy 0, policy_version 77530 (0.0007) [2023-10-11 22:08:14,743][71635] Updated weights for policy 1, policy_version 77442 (0.0009) [2023-10-11 22:08:15,102][71635] Updated weights for policy 1, policy_version 77452 (0.0008) [2023-10-11 22:08:15,465][71635] Updated weights for policy 1, policy_version 77462 (0.0007) [2023-10-11 22:08:15,841][71635] Updated weights for policy 1, policy_version 77472 (0.0009) [2023-10-11 22:08:16,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 158728192. Throughput: 0: 1811.6, 1: 1794.8. Samples: 39681776. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:08:16,034][70582] Avg episode reward: [(0, '82.880'), (1, '95.060')] [2023-10-11 22:08:17,058][71601] Updated weights for policy 0, policy_version 77540 (0.0008) [2023-10-11 22:08:17,426][71601] Updated weights for policy 0, policy_version 77550 (0.0007) [2023-10-11 22:08:17,794][71601] Updated weights for policy 0, policy_version 77560 (0.0008) [2023-10-11 22:08:19,560][71635] Updated weights for policy 1, policy_version 77482 (0.0011) [2023-10-11 22:08:19,932][71635] Updated weights for policy 1, policy_version 77492 (0.0010) [2023-10-11 22:08:20,308][71635] Updated weights for policy 1, policy_version 77502 (0.0010) [2023-10-11 22:08:21,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 158793728. Throughput: 0: 1810.7, 1: 1803.9. Samples: 39704056. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:08:21,035][70582] Avg episode reward: [(0, '87.890'), (1, '95.400')] [2023-10-11 22:08:21,572][71601] Updated weights for policy 0, policy_version 77570 (0.0009) [2023-10-11 22:08:21,953][71601] Updated weights for policy 0, policy_version 77580 (0.0008) [2023-10-11 22:08:22,317][71601] Updated weights for policy 0, policy_version 77590 (0.0008) [2023-10-11 22:08:22,688][71601] Updated weights for policy 0, policy_version 77600 (0.0011) [2023-10-11 22:08:23,871][71635] Updated weights for policy 1, policy_version 77512 (0.0008) [2023-10-11 22:08:24,245][71635] Updated weights for policy 1, policy_version 77522 (0.0008) [2023-10-11 22:08:24,622][71635] Updated weights for policy 1, policy_version 77532 (0.0010) [2023-10-11 22:08:26,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 158859264. Throughput: 0: 1801.3, 1: 1806.5. Samples: 39725536. Policy #0 lag: (min: 18.0, avg: 27.0, max: 50.0) [2023-10-11 22:08:26,035][70582] Avg episode reward: [(0, '87.500'), (1, '95.650')] [2023-10-11 22:08:26,383][71601] Updated weights for policy 0, policy_version 77610 (0.0008) [2023-10-11 22:08:26,760][71601] Updated weights for policy 0, policy_version 77620 (0.0007) [2023-10-11 22:08:27,132][71601] Updated weights for policy 0, policy_version 77630 (0.0007) [2023-10-11 22:08:28,546][71635] Updated weights for policy 1, policy_version 77542 (0.0008) [2023-10-11 22:08:28,936][71635] Updated weights for policy 1, policy_version 77552 (0.0008) [2023-10-11 22:08:29,305][71635] Updated weights for policy 1, policy_version 77562 (0.0009) [2023-10-11 22:08:30,841][71601] Updated weights for policy 0, policy_version 77640 (0.0008) [2023-10-11 22:08:31,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 158924800. Throughput: 0: 1804.4, 1: 1813.4. Samples: 39736828. Policy #0 lag: (min: 18.0, avg: 27.0, max: 50.0) [2023-10-11 22:08:31,034][70582] Avg episode reward: [(0, '88.490'), (1, '102.040')] [2023-10-11 22:08:31,221][71601] Updated weights for policy 0, policy_version 77650 (0.0011) [2023-10-11 22:08:31,595][71601] Updated weights for policy 0, policy_version 77660 (0.0010) [2023-10-11 22:08:33,101][71635] Updated weights for policy 1, policy_version 77572 (0.0007) [2023-10-11 22:08:33,471][71635] Updated weights for policy 1, policy_version 77582 (0.0008) [2023-10-11 22:08:33,848][71635] Updated weights for policy 1, policy_version 77592 (0.0009) [2023-10-11 22:08:35,365][71601] Updated weights for policy 0, policy_version 77670 (0.0008) [2023-10-11 22:08:35,743][71601] Updated weights for policy 0, policy_version 77680 (0.0007) [2023-10-11 22:08:36,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 158990336. Throughput: 0: 1803.2, 1: 1801.1. Samples: 39757882. Policy #0 lag: (min: 18.0, avg: 27.0, max: 50.0) [2023-10-11 22:08:36,034][70582] Avg episode reward: [(0, '93.520'), (1, '98.580')] [2023-10-11 22:08:36,120][71601] Updated weights for policy 0, policy_version 77690 (0.0009) [2023-10-11 22:08:37,462][71635] Updated weights for policy 1, policy_version 77602 (0.0010) [2023-10-11 22:08:37,825][71635] Updated weights for policy 1, policy_version 77612 (0.0007) [2023-10-11 22:08:38,183][71635] Updated weights for policy 1, policy_version 77622 (0.0009) [2023-10-11 22:08:38,551][71635] Updated weights for policy 1, policy_version 77632 (0.0010) [2023-10-11 22:08:39,686][71601] Updated weights for policy 0, policy_version 77700 (0.0010) [2023-10-11 22:08:40,055][71601] Updated weights for policy 0, policy_version 77710 (0.0010) [2023-10-11 22:08:40,436][71601] Updated weights for policy 0, policy_version 77720 (0.0009) [2023-10-11 22:08:41,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 159088640. Throughput: 0: 1812.5, 1: 1807.2. Samples: 39779794. Policy #0 lag: (min: 18.0, avg: 27.0, max: 50.0) [2023-10-11 22:08:41,034][70582] Avg episode reward: [(0, '89.320'), (1, '98.500')] [2023-10-11 22:08:42,411][71635] Updated weights for policy 1, policy_version 77642 (0.0008) [2023-10-11 22:08:42,764][71635] Updated weights for policy 1, policy_version 77652 (0.0008) [2023-10-11 22:08:43,136][71635] Updated weights for policy 1, policy_version 77662 (0.0007) [2023-10-11 22:08:44,052][71601] Updated weights for policy 0, policy_version 77730 (0.0008) [2023-10-11 22:08:44,431][71601] Updated weights for policy 0, policy_version 77740 (0.0008) [2023-10-11 22:08:44,801][71601] Updated weights for policy 0, policy_version 77750 (0.0010) [2023-10-11 22:08:45,175][71601] Updated weights for policy 0, policy_version 77760 (0.0009) [2023-10-11 22:08:46,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 159154176. Throughput: 0: 1813.1, 1: 1807.8. Samples: 39790840. Policy #0 lag: (min: 18.0, avg: 27.0, max: 50.0) [2023-10-11 22:08:46,035][70582] Avg episode reward: [(0, '85.640'), (1, '98.340')] [2023-10-11 22:08:47,072][71635] Updated weights for policy 1, policy_version 77672 (0.0007) [2023-10-11 22:08:47,431][71635] Updated weights for policy 1, policy_version 77682 (0.0008) [2023-10-11 22:08:47,796][71635] Updated weights for policy 1, policy_version 77692 (0.0007) [2023-10-11 22:08:48,869][71601] Updated weights for policy 0, policy_version 77770 (0.0009) [2023-10-11 22:08:49,238][71601] Updated weights for policy 0, policy_version 77780 (0.0009) [2023-10-11 22:08:49,613][71601] Updated weights for policy 0, policy_version 77790 (0.0010) [2023-10-11 22:08:51,034][70582] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 159219712. Throughput: 0: 1817.8, 1: 1800.4. Samples: 39812176. Policy #0 lag: (min: 18.0, avg: 27.0, max: 50.0) [2023-10-11 22:08:51,035][70582] Avg episode reward: [(0, '84.350'), (1, '98.950')] [2023-10-11 22:08:51,420][71635] Updated weights for policy 1, policy_version 77702 (0.0009) [2023-10-11 22:08:51,790][71635] Updated weights for policy 1, policy_version 77712 (0.0010) [2023-10-11 22:08:52,157][71635] Updated weights for policy 1, policy_version 77722 (0.0008) [2023-10-11 22:08:53,543][71601] Updated weights for policy 0, policy_version 77800 (0.0008) [2023-10-11 22:08:53,921][71601] Updated weights for policy 0, policy_version 77810 (0.0008) [2023-10-11 22:08:54,291][71601] Updated weights for policy 0, policy_version 77820 (0.0008) [2023-10-11 22:08:55,772][71635] Updated weights for policy 1, policy_version 77732 (0.0007) [2023-10-11 22:08:56,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 159285248. Throughput: 0: 1813.1, 1: 1811.0. Samples: 39834436. Policy #0 lag: (min: 18.0, avg: 27.0, max: 50.0) [2023-10-11 22:08:56,034][70582] Avg episode reward: [(0, '82.700'), (1, '98.790')] [2023-10-11 22:08:56,141][71635] Updated weights for policy 1, policy_version 77742 (0.0008) [2023-10-11 22:08:56,511][71635] Updated weights for policy 1, policy_version 77752 (0.0010) [2023-10-11 22:08:58,101][71601] Updated weights for policy 0, policy_version 77830 (0.0008) [2023-10-11 22:08:58,481][71601] Updated weights for policy 0, policy_version 77840 (0.0007) [2023-10-11 22:08:58,862][71601] Updated weights for policy 0, policy_version 77850 (0.0009) [2023-10-11 22:09:00,137][71635] Updated weights for policy 1, policy_version 77762 (0.0009) [2023-10-11 22:09:00,499][71635] Updated weights for policy 1, policy_version 77772 (0.0008) [2023-10-11 22:09:00,861][71635] Updated weights for policy 1, policy_version 77782 (0.0007) [2023-10-11 22:09:01,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 159350784. Throughput: 0: 1822.8, 1: 1808.0. Samples: 39845164. Policy #0 lag: (min: 18.0, avg: 27.0, max: 50.0) [2023-10-11 22:09:01,035][70582] Avg episode reward: [(0, '82.240'), (1, '104.250')] [2023-10-11 22:09:01,231][71635] Updated weights for policy 1, policy_version 77792 (0.0007) [2023-10-11 22:09:02,460][71601] Updated weights for policy 0, policy_version 77860 (0.0008) [2023-10-11 22:09:02,833][71601] Updated weights for policy 0, policy_version 77870 (0.0007) [2023-10-11 22:09:03,212][71601] Updated weights for policy 0, policy_version 77880 (0.0007) [2023-10-11 22:09:04,840][71635] Updated weights for policy 1, policy_version 77802 (0.0008) [2023-10-11 22:09:05,197][71635] Updated weights for policy 1, policy_version 77812 (0.0010) [2023-10-11 22:09:05,564][71635] Updated weights for policy 1, policy_version 77822 (0.0010) [2023-10-11 22:09:06,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 159449088. Throughput: 0: 1807.2, 1: 1820.5. Samples: 39867302. Policy #0 lag: (min: 18.0, avg: 27.0, max: 50.0) [2023-10-11 22:09:06,034][70582] Avg episode reward: [(0, '85.870'), (1, '98.780')] [2023-10-11 22:09:06,717][71601] Updated weights for policy 0, policy_version 77890 (0.0007) [2023-10-11 22:09:07,085][71601] Updated weights for policy 0, policy_version 77900 (0.0008) [2023-10-11 22:09:07,451][71601] Updated weights for policy 0, policy_version 77910 (0.0008) [2023-10-11 22:09:07,826][71601] Updated weights for policy 0, policy_version 77920 (0.0008) [2023-10-11 22:09:09,408][71635] Updated weights for policy 1, policy_version 77832 (0.0010) [2023-10-11 22:09:09,779][71635] Updated weights for policy 1, policy_version 77842 (0.0010) [2023-10-11 22:09:10,144][71635] Updated weights for policy 1, policy_version 77852 (0.0009) [2023-10-11 22:09:11,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 159514624. Throughput: 0: 1813.7, 1: 1805.7. Samples: 39888410. Policy #0 lag: (min: 18.0, avg: 27.0, max: 50.0) [2023-10-11 22:09:11,034][70582] Avg episode reward: [(0, '80.420'), (1, '97.160')] [2023-10-11 22:09:11,540][71601] Updated weights for policy 0, policy_version 77930 (0.0009) [2023-10-11 22:09:11,911][71601] Updated weights for policy 0, policy_version 77940 (0.0007) [2023-10-11 22:09:12,289][71601] Updated weights for policy 0, policy_version 77950 (0.0010) [2023-10-11 22:09:13,833][71635] Updated weights for policy 1, policy_version 77862 (0.0008) [2023-10-11 22:09:14,205][71635] Updated weights for policy 1, policy_version 77872 (0.0008) [2023-10-11 22:09:14,568][71635] Updated weights for policy 1, policy_version 77882 (0.0008) [2023-10-11 22:09:15,862][71601] Updated weights for policy 0, policy_version 77960 (0.0009) [2023-10-11 22:09:16,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 159580160. Throughput: 0: 1813.7, 1: 1814.1. Samples: 39900082. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:09:16,034][70582] Avg episode reward: [(0, '86.830'), (1, '98.550')] [2023-10-11 22:09:16,234][71601] Updated weights for policy 0, policy_version 77970 (0.0010) [2023-10-11 22:09:16,605][71601] Updated weights for policy 0, policy_version 77980 (0.0007) [2023-10-11 22:09:18,314][71635] Updated weights for policy 1, policy_version 77892 (0.0010) [2023-10-11 22:09:18,673][71635] Updated weights for policy 1, policy_version 77902 (0.0008) [2023-10-11 22:09:19,035][71635] Updated weights for policy 1, policy_version 77912 (0.0008) [2023-10-11 22:09:20,203][71601] Updated weights for policy 0, policy_version 77990 (0.0010) [2023-10-11 22:09:20,570][71601] Updated weights for policy 0, policy_version 78000 (0.0010) [2023-10-11 22:09:20,947][71601] Updated weights for policy 0, policy_version 78010 (0.0009) [2023-10-11 22:09:21,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 159645696. Throughput: 0: 1818.4, 1: 1816.0. Samples: 39921428. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:09:21,034][70582] Avg episode reward: [(0, '81.320'), (1, '95.250')] [2023-10-11 22:09:22,750][71635] Updated weights for policy 1, policy_version 77922 (0.0011) [2023-10-11 22:09:23,114][71635] Updated weights for policy 1, policy_version 77932 (0.0008) [2023-10-11 22:09:23,477][71635] Updated weights for policy 1, policy_version 77942 (0.0008) [2023-10-11 22:09:23,840][71635] Updated weights for policy 1, policy_version 77952 (0.0009) [2023-10-11 22:09:24,549][71601] Updated weights for policy 0, policy_version 78020 (0.0009) [2023-10-11 22:09:24,924][71601] Updated weights for policy 0, policy_version 78030 (0.0007) [2023-10-11 22:09:25,304][71601] Updated weights for policy 0, policy_version 78040 (0.0010) [2023-10-11 22:09:26,034][70582] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 159744000. Throughput: 0: 1815.8, 1: 1818.4. Samples: 39943332. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:09:26,035][70582] Avg episode reward: [(0, '83.850'), (1, '95.920')] [2023-10-11 22:09:27,406][71635] Updated weights for policy 1, policy_version 77962 (0.0009) [2023-10-11 22:09:27,764][71635] Updated weights for policy 1, policy_version 77972 (0.0008) [2023-10-11 22:09:28,133][71635] Updated weights for policy 1, policy_version 77982 (0.0007) [2023-10-11 22:09:28,828][71601] Updated weights for policy 0, policy_version 78050 (0.0008) [2023-10-11 22:09:29,203][71601] Updated weights for policy 0, policy_version 78060 (0.0010) [2023-10-11 22:09:29,571][71601] Updated weights for policy 0, policy_version 78070 (0.0009) [2023-10-11 22:09:29,935][71601] Updated weights for policy 0, policy_version 78080 (0.0008) [2023-10-11 22:09:31,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 159809536. Throughput: 0: 1822.8, 1: 1818.8. Samples: 39954708. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:09:31,034][70582] Avg episode reward: [(0, '87.060'), (1, '99.210')] [2023-10-11 22:09:31,667][71635] Updated weights for policy 1, policy_version 77992 (0.0008) [2023-10-11 22:09:32,041][71635] Updated weights for policy 1, policy_version 78002 (0.0007) [2023-10-11 22:09:32,406][71635] Updated weights for policy 1, policy_version 78012 (0.0008) [2023-10-11 22:09:33,561][71601] Updated weights for policy 0, policy_version 78090 (0.0007) [2023-10-11 22:09:33,931][71601] Updated weights for policy 0, policy_version 78100 (0.0009) [2023-10-11 22:09:34,299][71601] Updated weights for policy 0, policy_version 78110 (0.0008) [2023-10-11 22:09:36,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 159875072. Throughput: 0: 1816.9, 1: 1824.0. Samples: 39976018. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:09:36,035][70582] Avg episode reward: [(0, '88.560'), (1, '97.680')] [2023-10-11 22:09:36,135][71635] Updated weights for policy 1, policy_version 78022 (0.0010) [2023-10-11 22:09:36,497][71635] Updated weights for policy 1, policy_version 78032 (0.0011) [2023-10-11 22:09:36,867][71635] Updated weights for policy 1, policy_version 78042 (0.0011) [2023-10-11 22:09:38,117][71601] Updated weights for policy 0, policy_version 78120 (0.0007) [2023-10-11 22:09:38,504][71601] Updated weights for policy 0, policy_version 78130 (0.0007) [2023-10-11 22:09:38,864][71601] Updated weights for policy 0, policy_version 78140 (0.0008) [2023-10-11 22:09:40,557][71635] Updated weights for policy 1, policy_version 78052 (0.0009) [2023-10-11 22:09:40,926][71635] Updated weights for policy 1, policy_version 78062 (0.0008) [2023-10-11 22:09:41,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 159940608. Throughput: 0: 1830.5, 1: 1816.2. Samples: 39998540. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:09:41,035][70582] Avg episode reward: [(0, '88.780'), (1, '93.600')] [2023-10-11 22:09:41,284][71635] Updated weights for policy 1, policy_version 78072 (0.0008) [2023-10-11 22:09:42,710][71601] Updated weights for policy 0, policy_version 78150 (0.0009) [2023-10-11 22:09:43,087][71601] Updated weights for policy 0, policy_version 78160 (0.0007) [2023-10-11 22:09:43,460][71601] Updated weights for policy 0, policy_version 78170 (0.0007) [2023-10-11 22:09:45,252][71635] Updated weights for policy 1, policy_version 78082 (0.0008) [2023-10-11 22:09:45,617][71635] Updated weights for policy 1, policy_version 78092 (0.0009) [2023-10-11 22:09:45,983][71635] Updated weights for policy 1, policy_version 78102 (0.0008) [2023-10-11 22:09:46,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 160006144. Throughput: 0: 1815.9, 1: 1814.9. Samples: 40008552. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:09:46,034][70582] Avg episode reward: [(0, '89.570'), (1, '100.550')] [2023-10-11 22:09:46,343][71635] Updated weights for policy 1, policy_version 78112 (0.0007) [2023-10-11 22:09:47,172][71601] Updated weights for policy 0, policy_version 78180 (0.0009) [2023-10-11 22:09:47,535][71601] Updated weights for policy 0, policy_version 78190 (0.0008) [2023-10-11 22:09:47,910][71601] Updated weights for policy 0, policy_version 78200 (0.0008) [2023-10-11 22:09:50,110][71635] Updated weights for policy 1, policy_version 78122 (0.0008) [2023-10-11 22:09:50,475][71635] Updated weights for policy 1, policy_version 78132 (0.0009) [2023-10-11 22:09:50,838][71635] Updated weights for policy 1, policy_version 78142 (0.0008) [2023-10-11 22:09:51,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 160104448. Throughput: 0: 1821.3, 1: 1804.6. Samples: 40030470. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:09:51,035][70582] Avg episode reward: [(0, '85.870'), (1, '98.480')] [2023-10-11 22:09:51,596][71601] Updated weights for policy 0, policy_version 78210 (0.0008) [2023-10-11 22:09:51,977][71601] Updated weights for policy 0, policy_version 78220 (0.0011) [2023-10-11 22:09:52,354][71601] Updated weights for policy 0, policy_version 78230 (0.0010) [2023-10-11 22:09:52,721][71601] Updated weights for policy 0, policy_version 78240 (0.0009) [2023-10-11 22:09:54,427][71635] Updated weights for policy 1, policy_version 78152 (0.0008) [2023-10-11 22:09:54,796][71635] Updated weights for policy 1, policy_version 78162 (0.0007) [2023-10-11 22:09:55,155][71635] Updated weights for policy 1, policy_version 78172 (0.0009) [2023-10-11 22:09:56,034][70582] Fps is (10 sec: 16383.4, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 160169984. Throughput: 0: 1819.2, 1: 1814.8. Samples: 40051944. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:09:56,035][70582] Avg episode reward: [(0, '86.500'), (1, '101.180')] [2023-10-11 22:09:56,049][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000078240_80117760.pth... [2023-10-11 22:09:56,049][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000078176_80052224.pth... [2023-10-11 22:09:56,084][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000076480_78315520.pth [2023-10-11 22:09:56,088][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000076544_78381056.pth [2023-10-11 22:09:56,595][71601] Updated weights for policy 0, policy_version 78250 (0.0008) [2023-10-11 22:09:56,969][71601] Updated weights for policy 0, policy_version 78260 (0.0008) [2023-10-11 22:09:57,335][71601] Updated weights for policy 0, policy_version 78270 (0.0008) [2023-10-11 22:09:58,908][71635] Updated weights for policy 1, policy_version 78182 (0.0008) [2023-10-11 22:09:59,292][71635] Updated weights for policy 1, policy_version 78192 (0.0008) [2023-10-11 22:09:59,657][71635] Updated weights for policy 1, policy_version 78202 (0.0007) [2023-10-11 22:10:01,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 160235520. Throughput: 0: 1815.4, 1: 1813.9. Samples: 40063400. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:10:01,035][70582] Avg episode reward: [(0, '80.490'), (1, '106.660')] [2023-10-11 22:10:01,053][71601] Updated weights for policy 0, policy_version 78280 (0.0008) [2023-10-11 22:10:01,412][71601] Updated weights for policy 0, policy_version 78290 (0.0010) [2023-10-11 22:10:01,784][71601] Updated weights for policy 0, policy_version 78300 (0.0008) [2023-10-11 22:10:03,459][71635] Updated weights for policy 1, policy_version 78212 (0.0008) [2023-10-11 22:10:03,833][71635] Updated weights for policy 1, policy_version 78222 (0.0010) [2023-10-11 22:10:04,196][71635] Updated weights for policy 1, policy_version 78232 (0.0010) [2023-10-11 22:10:05,424][71601] Updated weights for policy 0, policy_version 78310 (0.0007) [2023-10-11 22:10:05,805][71601] Updated weights for policy 0, policy_version 78320 (0.0008) [2023-10-11 22:10:06,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 160301056. Throughput: 0: 1819.4, 1: 1816.4. Samples: 40085040. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-10-11 22:10:06,035][70582] Avg episode reward: [(0, '82.440'), (1, '107.740')] [2023-10-11 22:10:06,179][71601] Updated weights for policy 0, policy_version 78330 (0.0008) [2023-10-11 22:10:08,021][71635] Updated weights for policy 1, policy_version 78242 (0.0011) [2023-10-11 22:10:08,389][71635] Updated weights for policy 1, policy_version 78252 (0.0009) [2023-10-11 22:10:08,762][71635] Updated weights for policy 1, policy_version 78262 (0.0008) [2023-10-11 22:10:09,130][71635] Updated weights for policy 1, policy_version 78272 (0.0009) [2023-10-11 22:10:09,739][71601] Updated weights for policy 0, policy_version 78340 (0.0009) [2023-10-11 22:10:10,122][71601] Updated weights for policy 0, policy_version 78350 (0.0009) [2023-10-11 22:10:10,490][71601] Updated weights for policy 0, policy_version 78360 (0.0010) [2023-10-11 22:10:11,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 160399360. Throughput: 0: 1824.3, 1: 1801.8. Samples: 40106506. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-10-11 22:10:11,035][70582] Avg episode reward: [(0, '86.710'), (1, '109.550')] [2023-10-11 22:10:12,750][71635] Updated weights for policy 1, policy_version 78282 (0.0009) [2023-10-11 22:10:13,118][71635] Updated weights for policy 1, policy_version 78292 (0.0008) [2023-10-11 22:10:13,492][71635] Updated weights for policy 1, policy_version 78302 (0.0007) [2023-10-11 22:10:14,239][71601] Updated weights for policy 0, policy_version 78370 (0.0011) [2023-10-11 22:10:14,606][71601] Updated weights for policy 0, policy_version 78380 (0.0008) [2023-10-11 22:10:14,989][71601] Updated weights for policy 0, policy_version 78390 (0.0008) [2023-10-11 22:10:15,364][71601] Updated weights for policy 0, policy_version 78400 (0.0009) [2023-10-11 22:10:16,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 160464896. Throughput: 0: 1812.5, 1: 1811.4. Samples: 40117782. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-10-11 22:10:16,035][70582] Avg episode reward: [(0, '80.090'), (1, '110.610')] [2023-10-11 22:10:17,256][71635] Updated weights for policy 1, policy_version 78312 (0.0008) [2023-10-11 22:10:17,631][71635] Updated weights for policy 1, policy_version 78322 (0.0011) [2023-10-11 22:10:17,998][71635] Updated weights for policy 1, policy_version 78332 (0.0007) [2023-10-11 22:10:18,929][71601] Updated weights for policy 0, policy_version 78410 (0.0008) [2023-10-11 22:10:19,306][71601] Updated weights for policy 0, policy_version 78420 (0.0008) [2023-10-11 22:10:19,674][71601] Updated weights for policy 0, policy_version 78430 (0.0009) [2023-10-11 22:10:21,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 160530432. Throughput: 0: 1822.8, 1: 1805.3. Samples: 40139282. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-10-11 22:10:21,034][70582] Avg episode reward: [(0, '83.460'), (1, '111.750')] [2023-10-11 22:10:21,657][71635] Updated weights for policy 1, policy_version 78342 (0.0009) [2023-10-11 22:10:22,013][71635] Updated weights for policy 1, policy_version 78352 (0.0008) [2023-10-11 22:10:22,382][71635] Updated weights for policy 1, policy_version 78362 (0.0009) [2023-10-11 22:10:23,289][71601] Updated weights for policy 0, policy_version 78440 (0.0009) [2023-10-11 22:10:23,667][71601] Updated weights for policy 0, policy_version 78450 (0.0008) [2023-10-11 22:10:24,033][71601] Updated weights for policy 0, policy_version 78460 (0.0010) [2023-10-11 22:10:26,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14199.6, 300 sec: 14551.2). Total num frames: 160595968. Throughput: 0: 1817.3, 1: 1810.9. Samples: 40161808. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-10-11 22:10:26,034][70582] Avg episode reward: [(0, '82.530'), (1, '111.170')] [2023-10-11 22:10:26,046][71635] Updated weights for policy 1, policy_version 78372 (0.0008) [2023-10-11 22:10:26,411][71635] Updated weights for policy 1, policy_version 78382 (0.0007) [2023-10-11 22:10:26,779][71635] Updated weights for policy 1, policy_version 78392 (0.0009) [2023-10-11 22:10:27,711][71601] Updated weights for policy 0, policy_version 78470 (0.0008) [2023-10-11 22:10:28,091][71601] Updated weights for policy 0, policy_version 78480 (0.0007) [2023-10-11 22:10:28,459][71601] Updated weights for policy 0, policy_version 78490 (0.0008) [2023-10-11 22:10:30,498][71635] Updated weights for policy 1, policy_version 78402 (0.0009) [2023-10-11 22:10:30,862][71635] Updated weights for policy 1, policy_version 78412 (0.0007) [2023-10-11 22:10:31,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 160661504. Throughput: 0: 1820.0, 1: 1814.7. Samples: 40172112. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-10-11 22:10:31,034][70582] Avg episode reward: [(0, '81.390'), (1, '112.710')] [2023-10-11 22:10:31,228][71635] Updated weights for policy 1, policy_version 78422 (0.0007) [2023-10-11 22:10:31,594][71635] Updated weights for policy 1, policy_version 78432 (0.0007) [2023-10-11 22:10:32,138][71601] Updated weights for policy 0, policy_version 78500 (0.0008) [2023-10-11 22:10:32,505][71601] Updated weights for policy 0, policy_version 78510 (0.0008) [2023-10-11 22:10:32,872][71601] Updated weights for policy 0, policy_version 78520 (0.0007) [2023-10-11 22:10:35,252][71635] Updated weights for policy 1, policy_version 78442 (0.0008) [2023-10-11 22:10:35,617][71635] Updated weights for policy 1, policy_version 78452 (0.0008) [2023-10-11 22:10:35,983][71635] Updated weights for policy 1, policy_version 78462 (0.0011) [2023-10-11 22:10:36,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 160727040. Throughput: 0: 1822.4, 1: 1820.8. Samples: 40194412. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-10-11 22:10:36,035][70582] Avg episode reward: [(0, '84.910'), (1, '114.440')] [2023-10-11 22:10:36,608][71601] Updated weights for policy 0, policy_version 78530 (0.0008) [2023-10-11 22:10:36,979][71601] Updated weights for policy 0, policy_version 78540 (0.0008) [2023-10-11 22:10:37,351][71601] Updated weights for policy 0, policy_version 78550 (0.0009) [2023-10-11 22:10:37,731][71601] Updated weights for policy 0, policy_version 78560 (0.0008) [2023-10-11 22:10:39,709][71635] Updated weights for policy 1, policy_version 78472 (0.0008) [2023-10-11 22:10:40,068][71635] Updated weights for policy 1, policy_version 78482 (0.0008) [2023-10-11 22:10:40,439][71635] Updated weights for policy 1, policy_version 78492 (0.0011) [2023-10-11 22:10:41,034][70582] Fps is (10 sec: 16383.2, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 160825344. Throughput: 0: 1823.2, 1: 1821.2. Samples: 40215942. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-10-11 22:10:41,035][70582] Avg episode reward: [(0, '83.060'), (1, '108.720')] [2023-10-11 22:10:41,533][71601] Updated weights for policy 0, policy_version 78570 (0.0008) [2023-10-11 22:10:41,905][71601] Updated weights for policy 0, policy_version 78580 (0.0008) [2023-10-11 22:10:42,272][71601] Updated weights for policy 0, policy_version 78590 (0.0008) [2023-10-11 22:10:44,345][71635] Updated weights for policy 1, policy_version 78502 (0.0009) [2023-10-11 22:10:44,746][71635] Updated weights for policy 1, policy_version 78512 (0.0008) [2023-10-11 22:10:45,105][71635] Updated weights for policy 1, policy_version 78522 (0.0008) [2023-10-11 22:10:45,997][71601] Updated weights for policy 0, policy_version 78600 (0.0007) [2023-10-11 22:10:46,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 160890880. Throughput: 0: 1825.5, 1: 1807.5. Samples: 40226882. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-10-11 22:10:46,035][70582] Avg episode reward: [(0, '85.030'), (1, '101.360')] [2023-10-11 22:10:46,374][71601] Updated weights for policy 0, policy_version 78610 (0.0008) [2023-10-11 22:10:46,745][71601] Updated weights for policy 0, policy_version 78620 (0.0008) [2023-10-11 22:10:48,588][71635] Updated weights for policy 1, policy_version 78532 (0.0007) [2023-10-11 22:10:48,964][71635] Updated weights for policy 1, policy_version 78542 (0.0008) [2023-10-11 22:10:49,326][71635] Updated weights for policy 1, policy_version 78552 (0.0009) [2023-10-11 22:10:50,304][71601] Updated weights for policy 0, policy_version 78630 (0.0008) [2023-10-11 22:10:50,671][71601] Updated weights for policy 0, policy_version 78640 (0.0010) [2023-10-11 22:10:51,034][70582] Fps is (10 sec: 13107.8, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 160956416. Throughput: 0: 1818.4, 1: 1816.7. Samples: 40248620. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-10-11 22:10:51,034][70582] Avg episode reward: [(0, '79.170'), (1, '100.260')] [2023-10-11 22:10:51,041][71601] Updated weights for policy 0, policy_version 78650 (0.0010) [2023-10-11 22:10:53,038][71635] Updated weights for policy 1, policy_version 78562 (0.0009) [2023-10-11 22:10:53,400][71635] Updated weights for policy 1, policy_version 78572 (0.0009) [2023-10-11 22:10:53,770][71635] Updated weights for policy 1, policy_version 78582 (0.0007) [2023-10-11 22:10:54,130][71635] Updated weights for policy 1, policy_version 78592 (0.0010) [2023-10-11 22:10:54,839][71601] Updated weights for policy 0, policy_version 78660 (0.0009) [2023-10-11 22:10:55,196][71601] Updated weights for policy 0, policy_version 78670 (0.0007) [2023-10-11 22:10:55,571][71601] Updated weights for policy 0, policy_version 78680 (0.0007) [2023-10-11 22:10:56,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 161054720. Throughput: 0: 1815.2, 1: 1818.8. Samples: 40270036. Policy #0 lag: (min: 13.0, avg: 20.6, max: 45.0) [2023-10-11 22:10:56,035][70582] Avg episode reward: [(0, '79.030'), (1, '100.280')] [2023-10-11 22:10:57,827][71635] Updated weights for policy 1, policy_version 78602 (0.0009) [2023-10-11 22:10:58,201][71635] Updated weights for policy 1, policy_version 78612 (0.0009) [2023-10-11 22:10:58,556][71635] Updated weights for policy 1, policy_version 78622 (0.0009) [2023-10-11 22:10:59,233][71601] Updated weights for policy 0, policy_version 78690 (0.0008) [2023-10-11 22:10:59,601][71601] Updated weights for policy 0, policy_version 78700 (0.0011) [2023-10-11 22:10:59,968][71601] Updated weights for policy 0, policy_version 78710 (0.0011) [2023-10-11 22:11:00,346][71601] Updated weights for policy 0, policy_version 78720 (0.0011) [2023-10-11 22:11:01,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 161120256. Throughput: 0: 1815.5, 1: 1818.4. Samples: 40281304. Policy #0 lag: (min: 13.0, avg: 20.6, max: 45.0) [2023-10-11 22:11:01,035][70582] Avg episode reward: [(0, '82.320'), (1, '100.000')] [2023-10-11 22:11:02,244][71635] Updated weights for policy 1, policy_version 78632 (0.0009) [2023-10-11 22:11:02,610][71635] Updated weights for policy 1, policy_version 78642 (0.0008) [2023-10-11 22:11:02,984][71635] Updated weights for policy 1, policy_version 78652 (0.0008) [2023-10-11 22:11:04,142][71601] Updated weights for policy 0, policy_version 78730 (0.0009) [2023-10-11 22:11:04,515][71601] Updated weights for policy 0, policy_version 78740 (0.0010) [2023-10-11 22:11:04,880][71601] Updated weights for policy 0, policy_version 78750 (0.0008) [2023-10-11 22:11:06,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 161185792. Throughput: 0: 1818.6, 1: 1813.7. Samples: 40302738. Policy #0 lag: (min: 13.0, avg: 20.6, max: 45.0) [2023-10-11 22:11:06,035][70582] Avg episode reward: [(0, '78.960'), (1, '105.990')] [2023-10-11 22:11:06,642][71635] Updated weights for policy 1, policy_version 78662 (0.0008) [2023-10-11 22:11:07,010][71635] Updated weights for policy 1, policy_version 78672 (0.0008) [2023-10-11 22:11:07,388][71635] Updated weights for policy 1, policy_version 78682 (0.0009) [2023-10-11 22:11:08,568][71601] Updated weights for policy 0, policy_version 78760 (0.0008) [2023-10-11 22:11:08,945][71601] Updated weights for policy 0, policy_version 78770 (0.0010) [2023-10-11 22:11:09,323][71601] Updated weights for policy 0, policy_version 78780 (0.0008) [2023-10-11 22:11:11,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 161251328. Throughput: 0: 1810.3, 1: 1815.9. Samples: 40324984. Policy #0 lag: (min: 13.0, avg: 20.6, max: 45.0) [2023-10-11 22:11:11,035][70582] Avg episode reward: [(0, '78.150'), (1, '102.610')] [2023-10-11 22:11:11,039][71635] Updated weights for policy 1, policy_version 78692 (0.0009) [2023-10-11 22:11:11,395][71635] Updated weights for policy 1, policy_version 78702 (0.0008) [2023-10-11 22:11:11,756][71635] Updated weights for policy 1, policy_version 78712 (0.0009) [2023-10-11 22:11:13,098][71601] Updated weights for policy 0, policy_version 78790 (0.0007) [2023-10-11 22:11:13,480][71601] Updated weights for policy 0, policy_version 78800 (0.0007) [2023-10-11 22:11:13,855][71601] Updated weights for policy 0, policy_version 78810 (0.0007) [2023-10-11 22:11:15,579][71635] Updated weights for policy 1, policy_version 78722 (0.0008) [2023-10-11 22:11:15,951][71635] Updated weights for policy 1, policy_version 78732 (0.0007) [2023-10-11 22:11:16,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 161316864. Throughput: 0: 1816.1, 1: 1814.4. Samples: 40335484. Policy #0 lag: (min: 13.0, avg: 20.6, max: 45.0) [2023-10-11 22:11:16,034][70582] Avg episode reward: [(0, '73.850'), (1, '107.620')] [2023-10-11 22:11:16,329][71635] Updated weights for policy 1, policy_version 78742 (0.0009) [2023-10-11 22:11:16,691][71635] Updated weights for policy 1, policy_version 78752 (0.0008) [2023-10-11 22:11:17,530][71601] Updated weights for policy 0, policy_version 78820 (0.0010) [2023-10-11 22:11:17,904][71601] Updated weights for policy 0, policy_version 78830 (0.0009) [2023-10-11 22:11:18,285][71601] Updated weights for policy 0, policy_version 78840 (0.0007) [2023-10-11 22:11:20,415][71635] Updated weights for policy 1, policy_version 78762 (0.0009) [2023-10-11 22:11:20,780][71635] Updated weights for policy 1, policy_version 78772 (0.0009) [2023-10-11 22:11:21,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 161382400. Throughput: 0: 1804.3, 1: 1813.6. Samples: 40357216. Policy #0 lag: (min: 13.0, avg: 20.6, max: 45.0) [2023-10-11 22:11:21,035][70582] Avg episode reward: [(0, '72.400'), (1, '105.640')] [2023-10-11 22:11:21,141][71635] Updated weights for policy 1, policy_version 78782 (0.0007) [2023-10-11 22:11:21,904][71601] Updated weights for policy 0, policy_version 78850 (0.0009) [2023-10-11 22:11:22,283][71601] Updated weights for policy 0, policy_version 78860 (0.0008) [2023-10-11 22:11:22,650][71601] Updated weights for policy 0, policy_version 78870 (0.0008) [2023-10-11 22:11:23,024][71601] Updated weights for policy 0, policy_version 78880 (0.0007) [2023-10-11 22:11:24,876][71635] Updated weights for policy 1, policy_version 78792 (0.0007) [2023-10-11 22:11:25,239][71635] Updated weights for policy 1, policy_version 78802 (0.0009) [2023-10-11 22:11:25,607][71635] Updated weights for policy 1, policy_version 78812 (0.0008) [2023-10-11 22:11:26,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 161480704. Throughput: 0: 1810.6, 1: 1823.5. Samples: 40379476. Policy #0 lag: (min: 13.0, avg: 20.6, max: 45.0) [2023-10-11 22:11:26,034][70582] Avg episode reward: [(0, '72.190'), (1, '97.600')] [2023-10-11 22:11:26,682][71601] Updated weights for policy 0, policy_version 78890 (0.0010) [2023-10-11 22:11:27,059][71601] Updated weights for policy 0, policy_version 78900 (0.0009) [2023-10-11 22:11:27,433][71601] Updated weights for policy 0, policy_version 78910 (0.0008) [2023-10-11 22:11:29,254][71635] Updated weights for policy 1, policy_version 78822 (0.0008) [2023-10-11 22:11:29,647][71635] Updated weights for policy 1, policy_version 78832 (0.0007) [2023-10-11 22:11:30,010][71635] Updated weights for policy 1, policy_version 78842 (0.0009) [2023-10-11 22:11:31,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 161546240. Throughput: 0: 1808.2, 1: 1824.8. Samples: 40390366. Policy #0 lag: (min: 13.0, avg: 20.6, max: 45.0) [2023-10-11 22:11:31,034][70582] Avg episode reward: [(0, '73.080'), (1, '93.720')] [2023-10-11 22:11:31,217][71601] Updated weights for policy 0, policy_version 78920 (0.0007) [2023-10-11 22:11:31,579][71601] Updated weights for policy 0, policy_version 78930 (0.0008) [2023-10-11 22:11:31,947][71601] Updated weights for policy 0, policy_version 78940 (0.0008) [2023-10-11 22:11:33,539][71635] Updated weights for policy 1, policy_version 78852 (0.0010) [2023-10-11 22:11:33,904][71635] Updated weights for policy 1, policy_version 78862 (0.0008) [2023-10-11 22:11:34,262][71635] Updated weights for policy 1, policy_version 78872 (0.0007) [2023-10-11 22:11:35,593][71601] Updated weights for policy 0, policy_version 78950 (0.0008) [2023-10-11 22:11:35,965][71601] Updated weights for policy 0, policy_version 78960 (0.0008) [2023-10-11 22:11:36,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 161611776. Throughput: 0: 1805.4, 1: 1819.4. Samples: 40411736. Policy #0 lag: (min: 13.0, avg: 20.6, max: 45.0) [2023-10-11 22:11:36,034][70582] Avg episode reward: [(0, '76.030'), (1, '106.000')] [2023-10-11 22:11:36,337][71601] Updated weights for policy 0, policy_version 78970 (0.0009) [2023-10-11 22:11:37,880][71635] Updated weights for policy 1, policy_version 78882 (0.0007) [2023-10-11 22:11:38,250][71635] Updated weights for policy 1, policy_version 78892 (0.0007) [2023-10-11 22:11:38,617][71635] Updated weights for policy 1, policy_version 78902 (0.0010) [2023-10-11 22:11:38,975][71635] Updated weights for policy 1, policy_version 78912 (0.0009) [2023-10-11 22:11:39,845][71601] Updated weights for policy 0, policy_version 78980 (0.0009) [2023-10-11 22:11:40,212][71601] Updated weights for policy 0, policy_version 78990 (0.0011) [2023-10-11 22:11:40,586][71601] Updated weights for policy 0, policy_version 79000 (0.0008) [2023-10-11 22:11:41,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 161710080. Throughput: 0: 1816.4, 1: 1824.9. Samples: 40433896. Policy #0 lag: (min: 29.0, avg: 32.2, max: 61.0) [2023-10-11 22:11:41,035][70582] Avg episode reward: [(0, '77.180'), (1, '107.030')] [2023-10-11 22:11:42,755][71635] Updated weights for policy 1, policy_version 78922 (0.0011) [2023-10-11 22:11:43,126][71635] Updated weights for policy 1, policy_version 78932 (0.0010) [2023-10-11 22:11:43,489][71635] Updated weights for policy 1, policy_version 78942 (0.0007) [2023-10-11 22:11:44,383][71601] Updated weights for policy 0, policy_version 79010 (0.0007) [2023-10-11 22:11:44,761][71601] Updated weights for policy 0, policy_version 79020 (0.0008) [2023-10-11 22:11:45,130][71601] Updated weights for policy 0, policy_version 79030 (0.0008) [2023-10-11 22:11:45,511][71601] Updated weights for policy 0, policy_version 79040 (0.0009) [2023-10-11 22:11:46,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 161775616. Throughput: 0: 1811.6, 1: 1824.1. Samples: 40444910. Policy #0 lag: (min: 29.0, avg: 32.2, max: 61.0) [2023-10-11 22:11:46,035][70582] Avg episode reward: [(0, '77.640'), (1, '103.080')] [2023-10-11 22:11:47,322][71635] Updated weights for policy 1, policy_version 78952 (0.0008) [2023-10-11 22:11:47,694][71635] Updated weights for policy 1, policy_version 78962 (0.0007) [2023-10-11 22:11:48,057][71635] Updated weights for policy 1, policy_version 78972 (0.0008) [2023-10-11 22:11:49,115][71601] Updated weights for policy 0, policy_version 79050 (0.0007) [2023-10-11 22:11:49,482][71601] Updated weights for policy 0, policy_version 79060 (0.0008) [2023-10-11 22:11:49,855][71601] Updated weights for policy 0, policy_version 79070 (0.0007) [2023-10-11 22:11:51,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 161841152. Throughput: 0: 1815.6, 1: 1823.8. Samples: 40466508. Policy #0 lag: (min: 29.0, avg: 32.2, max: 61.0) [2023-10-11 22:11:51,035][70582] Avg episode reward: [(0, '80.530'), (1, '104.510')] [2023-10-11 22:11:51,629][71635] Updated weights for policy 1, policy_version 78982 (0.0008) [2023-10-11 22:11:51,993][71635] Updated weights for policy 1, policy_version 78992 (0.0008) [2023-10-11 22:11:52,368][71635] Updated weights for policy 1, policy_version 79002 (0.0008) [2023-10-11 22:11:53,508][71601] Updated weights for policy 0, policy_version 79080 (0.0008) [2023-10-11 22:11:53,883][71601] Updated weights for policy 0, policy_version 79090 (0.0009) [2023-10-11 22:11:54,250][71601] Updated weights for policy 0, policy_version 79100 (0.0007) [2023-10-11 22:11:56,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 161906688. Throughput: 0: 1819.6, 1: 1818.2. Samples: 40488684. Policy #0 lag: (min: 29.0, avg: 32.2, max: 61.0) [2023-10-11 22:11:56,035][70582] Avg episode reward: [(0, '84.300'), (1, '108.380')] [2023-10-11 22:11:56,044][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000079104_81002496.pth... [2023-10-11 22:11:56,077][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000077408_79265792.pth [2023-10-11 22:11:56,119][71635] Updated weights for policy 1, policy_version 79012 (0.0009) [2023-10-11 22:11:56,486][71635] Updated weights for policy 1, policy_version 79022 (0.0010) [2023-10-11 22:11:56,852][71635] Updated weights for policy 1, policy_version 79032 (0.0010) [2023-10-11 22:11:57,144][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000079040_80936960.pth... [2023-10-11 22:11:57,180][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000077312_79167488.pth [2023-10-11 22:11:57,946][71601] Updated weights for policy 0, policy_version 79110 (0.0008) [2023-10-11 22:11:58,325][71601] Updated weights for policy 0, policy_version 79120 (0.0007) [2023-10-11 22:11:58,687][71601] Updated weights for policy 0, policy_version 79130 (0.0009) [2023-10-11 22:12:00,594][71635] Updated weights for policy 1, policy_version 79042 (0.0010) [2023-10-11 22:12:00,962][71635] Updated weights for policy 1, policy_version 79052 (0.0007) [2023-10-11 22:12:01,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 161972224. Throughput: 0: 1820.3, 1: 1821.4. Samples: 40499358. Policy #0 lag: (min: 29.0, avg: 32.2, max: 61.0) [2023-10-11 22:12:01,035][70582] Avg episode reward: [(0, '88.840'), (1, '110.050')] [2023-10-11 22:12:01,337][71635] Updated weights for policy 1, policy_version 79062 (0.0007) [2023-10-11 22:12:01,706][71635] Updated weights for policy 1, policy_version 79072 (0.0007) [2023-10-11 22:12:02,466][71601] Updated weights for policy 0, policy_version 79140 (0.0007) [2023-10-11 22:12:02,843][71601] Updated weights for policy 0, policy_version 79150 (0.0010) [2023-10-11 22:12:03,219][71601] Updated weights for policy 0, policy_version 79160 (0.0009) [2023-10-11 22:12:05,251][71635] Updated weights for policy 1, policy_version 79082 (0.0007) [2023-10-11 22:12:05,623][71635] Updated weights for policy 1, policy_version 79092 (0.0007) [2023-10-11 22:12:05,987][71635] Updated weights for policy 1, policy_version 79102 (0.0008) [2023-10-11 22:12:06,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 162037760. Throughput: 0: 1823.7, 1: 1827.2. Samples: 40521506. Policy #0 lag: (min: 29.0, avg: 32.2, max: 61.0) [2023-10-11 22:12:06,034][70582] Avg episode reward: [(0, '90.860'), (1, '109.160')] [2023-10-11 22:12:06,901][71601] Updated weights for policy 0, policy_version 79170 (0.0009) [2023-10-11 22:12:07,270][71601] Updated weights for policy 0, policy_version 79180 (0.0008) [2023-10-11 22:12:07,645][71601] Updated weights for policy 0, policy_version 79190 (0.0009) [2023-10-11 22:12:08,017][71601] Updated weights for policy 0, policy_version 79200 (0.0008) [2023-10-11 22:12:09,646][71635] Updated weights for policy 1, policy_version 79112 (0.0008) [2023-10-11 22:12:10,026][71635] Updated weights for policy 1, policy_version 79122 (0.0010) [2023-10-11 22:12:10,384][71635] Updated weights for policy 1, policy_version 79132 (0.0011) [2023-10-11 22:12:11,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 162136064. Throughput: 0: 1811.6, 1: 1819.8. Samples: 40542888. Policy #0 lag: (min: 29.0, avg: 32.2, max: 61.0) [2023-10-11 22:12:11,034][70582] Avg episode reward: [(0, '88.560'), (1, '107.080')] [2023-10-11 22:12:11,778][71601] Updated weights for policy 0, policy_version 79210 (0.0010) [2023-10-11 22:12:12,160][71601] Updated weights for policy 0, policy_version 79220 (0.0009) [2023-10-11 22:12:12,542][71601] Updated weights for policy 0, policy_version 79230 (0.0009) [2023-10-11 22:12:14,148][71635] Updated weights for policy 1, policy_version 79142 (0.0010) [2023-10-11 22:12:14,519][71635] Updated weights for policy 1, policy_version 79152 (0.0008) [2023-10-11 22:12:14,880][71635] Updated weights for policy 1, policy_version 79162 (0.0009) [2023-10-11 22:12:16,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 162201600. Throughput: 0: 1812.7, 1: 1823.9. Samples: 40554012. Policy #0 lag: (min: 29.0, avg: 32.2, max: 61.0) [2023-10-11 22:12:16,035][70582] Avg episode reward: [(0, '87.660'), (1, '112.290')] [2023-10-11 22:12:16,240][71601] Updated weights for policy 0, policy_version 79240 (0.0009) [2023-10-11 22:12:16,616][71601] Updated weights for policy 0, policy_version 79250 (0.0008) [2023-10-11 22:12:16,982][71601] Updated weights for policy 0, policy_version 79260 (0.0011) [2023-10-11 22:12:18,564][71635] Updated weights for policy 1, policy_version 79172 (0.0008) [2023-10-11 22:12:18,959][71635] Updated weights for policy 1, policy_version 79182 (0.0008) [2023-10-11 22:12:19,315][71635] Updated weights for policy 1, policy_version 79192 (0.0007) [2023-10-11 22:12:20,554][71601] Updated weights for policy 0, policy_version 79270 (0.0011) [2023-10-11 22:12:20,935][71601] Updated weights for policy 0, policy_version 79280 (0.0007) [2023-10-11 22:12:21,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 162267136. Throughput: 0: 1817.9, 1: 1826.1. Samples: 40575716. Policy #0 lag: (min: 29.0, avg: 32.2, max: 61.0) [2023-10-11 22:12:21,034][70582] Avg episode reward: [(0, '86.320'), (1, '117.990')] [2023-10-11 22:12:21,301][71601] Updated weights for policy 0, policy_version 79290 (0.0008) [2023-10-11 22:12:23,097][71635] Updated weights for policy 1, policy_version 79202 (0.0010) [2023-10-11 22:12:23,454][71635] Updated weights for policy 1, policy_version 79212 (0.0008) [2023-10-11 22:12:23,823][71635] Updated weights for policy 1, policy_version 79222 (0.0008) [2023-10-11 22:12:24,180][71635] Updated weights for policy 1, policy_version 79232 (0.0010) [2023-10-11 22:12:25,041][71601] Updated weights for policy 0, policy_version 79300 (0.0009) [2023-10-11 22:12:25,405][71601] Updated weights for policy 0, policy_version 79310 (0.0008) [2023-10-11 22:12:25,775][71601] Updated weights for policy 0, policy_version 79320 (0.0011) [2023-10-11 22:12:26,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 162332672. Throughput: 0: 1812.6, 1: 1820.2. Samples: 40597372. Policy #0 lag: (min: 29.0, avg: 32.2, max: 61.0) [2023-10-11 22:12:26,034][70582] Avg episode reward: [(0, '87.100'), (1, '123.960')] [2023-10-11 22:12:27,781][71635] Updated weights for policy 1, policy_version 79242 (0.0010) [2023-10-11 22:12:28,147][71635] Updated weights for policy 1, policy_version 79252 (0.0008) [2023-10-11 22:12:28,519][71635] Updated weights for policy 1, policy_version 79262 (0.0008) [2023-10-11 22:12:29,537][71601] Updated weights for policy 0, policy_version 79330 (0.0010) [2023-10-11 22:12:29,911][71601] Updated weights for policy 0, policy_version 79340 (0.0008) [2023-10-11 22:12:30,286][71601] Updated weights for policy 0, policy_version 79350 (0.0008) [2023-10-11 22:12:30,655][71601] Updated weights for policy 0, policy_version 79360 (0.0007) [2023-10-11 22:12:31,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 162430976. Throughput: 0: 1807.5, 1: 1814.2. Samples: 40607888. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:12:31,034][70582] Avg episode reward: [(0, '81.750'), (1, '118.610')] [2023-10-11 22:12:32,323][71635] Updated weights for policy 1, policy_version 79272 (0.0009) [2023-10-11 22:12:32,686][71635] Updated weights for policy 1, policy_version 79282 (0.0011) [2023-10-11 22:12:33,043][71635] Updated weights for policy 1, policy_version 79292 (0.0010) [2023-10-11 22:12:34,450][71601] Updated weights for policy 0, policy_version 79370 (0.0008) [2023-10-11 22:12:34,830][71601] Updated weights for policy 0, policy_version 79380 (0.0008) [2023-10-11 22:12:35,198][71601] Updated weights for policy 0, policy_version 79390 (0.0007) [2023-10-11 22:12:36,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 162496512. Throughput: 0: 1818.5, 1: 1813.9. Samples: 40629964. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:12:36,034][70582] Avg episode reward: [(0, '79.960'), (1, '115.390')] [2023-10-11 22:12:36,892][71635] Updated weights for policy 1, policy_version 79302 (0.0010) [2023-10-11 22:12:37,257][71635] Updated weights for policy 1, policy_version 79312 (0.0011) [2023-10-11 22:12:37,630][71635] Updated weights for policy 1, policy_version 79322 (0.0009) [2023-10-11 22:12:38,955][71601] Updated weights for policy 0, policy_version 79400 (0.0008) [2023-10-11 22:12:39,338][71601] Updated weights for policy 0, policy_version 79410 (0.0008) [2023-10-11 22:12:39,716][71601] Updated weights for policy 0, policy_version 79420 (0.0007) [2023-10-11 22:12:41,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 162562048. Throughput: 0: 1809.1, 1: 1812.9. Samples: 40651674. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:12:41,034][70582] Avg episode reward: [(0, '84.550'), (1, '115.470')] [2023-10-11 22:12:41,393][71635] Updated weights for policy 1, policy_version 79332 (0.0009) [2023-10-11 22:12:41,764][71635] Updated weights for policy 1, policy_version 79342 (0.0007) [2023-10-11 22:12:42,128][71635] Updated weights for policy 1, policy_version 79352 (0.0010) [2023-10-11 22:12:43,391][71601] Updated weights for policy 0, policy_version 79430 (0.0008) [2023-10-11 22:12:43,759][71601] Updated weights for policy 0, policy_version 79440 (0.0007) [2023-10-11 22:12:44,134][71601] Updated weights for policy 0, policy_version 79450 (0.0008) [2023-10-11 22:12:45,902][71635] Updated weights for policy 1, policy_version 79362 (0.0009) [2023-10-11 22:12:46,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 162627584. Throughput: 0: 1821.2, 1: 1809.6. Samples: 40662746. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:12:46,035][70582] Avg episode reward: [(0, '81.170'), (1, '116.080')] [2023-10-11 22:12:46,269][71635] Updated weights for policy 1, policy_version 79372 (0.0007) [2023-10-11 22:12:46,641][71635] Updated weights for policy 1, policy_version 79382 (0.0009) [2023-10-11 22:12:47,004][71635] Updated weights for policy 1, policy_version 79392 (0.0010) [2023-10-11 22:12:47,864][71601] Updated weights for policy 0, policy_version 79460 (0.0008) [2023-10-11 22:12:48,241][71601] Updated weights for policy 0, policy_version 79470 (0.0007) [2023-10-11 22:12:48,611][71601] Updated weights for policy 0, policy_version 79480 (0.0007) [2023-10-11 22:12:50,680][71635] Updated weights for policy 1, policy_version 79402 (0.0008) [2023-10-11 22:12:51,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 162693120. Throughput: 0: 1812.0, 1: 1804.1. Samples: 40684234. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:12:51,035][70582] Avg episode reward: [(0, '79.550'), (1, '108.110')] [2023-10-11 22:12:51,037][71635] Updated weights for policy 1, policy_version 79412 (0.0010) [2023-10-11 22:12:51,398][71635] Updated weights for policy 1, policy_version 79422 (0.0011) [2023-10-11 22:12:52,269][71601] Updated weights for policy 0, policy_version 79490 (0.0008) [2023-10-11 22:12:52,651][71601] Updated weights for policy 0, policy_version 79500 (0.0012) [2023-10-11 22:12:53,018][71601] Updated weights for policy 0, policy_version 79510 (0.0008) [2023-10-11 22:12:53,397][71601] Updated weights for policy 0, policy_version 79520 (0.0007) [2023-10-11 22:12:55,100][71635] Updated weights for policy 1, policy_version 79432 (0.0008) [2023-10-11 22:12:55,460][71635] Updated weights for policy 1, policy_version 79442 (0.0007) [2023-10-11 22:12:55,832][71635] Updated weights for policy 1, policy_version 79452 (0.0009) [2023-10-11 22:12:56,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 162791424. Throughput: 0: 1817.0, 1: 1814.9. Samples: 40706324. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:12:56,035][70582] Avg episode reward: [(0, '82.780'), (1, '105.060')] [2023-10-11 22:12:57,057][71601] Updated weights for policy 0, policy_version 79530 (0.0008) [2023-10-11 22:12:57,443][71601] Updated weights for policy 0, policy_version 79540 (0.0010) [2023-10-11 22:12:57,806][71601] Updated weights for policy 0, policy_version 79550 (0.0011) [2023-10-11 22:12:59,372][71635] Updated weights for policy 1, policy_version 79462 (0.0007) [2023-10-11 22:12:59,740][71635] Updated weights for policy 1, policy_version 79472 (0.0008) [2023-10-11 22:13:00,108][71635] Updated weights for policy 1, policy_version 79482 (0.0008) [2023-10-11 22:13:01,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 162856960. Throughput: 0: 1818.1, 1: 1807.3. Samples: 40717154. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:13:01,034][70582] Avg episode reward: [(0, '87.070'), (1, '105.160')] [2023-10-11 22:13:01,476][71601] Updated weights for policy 0, policy_version 79560 (0.0008) [2023-10-11 22:13:01,864][71601] Updated weights for policy 0, policy_version 79570 (0.0009) [2023-10-11 22:13:02,237][71601] Updated weights for policy 0, policy_version 79580 (0.0010) [2023-10-11 22:13:03,910][71635] Updated weights for policy 1, policy_version 79492 (0.0008) [2023-10-11 22:13:04,316][71635] Updated weights for policy 1, policy_version 79502 (0.0008) [2023-10-11 22:13:04,689][71635] Updated weights for policy 1, policy_version 79512 (0.0008) [2023-10-11 22:13:05,898][71601] Updated weights for policy 0, policy_version 79590 (0.0009) [2023-10-11 22:13:06,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 162922496. Throughput: 0: 1813.7, 1: 1818.3. Samples: 40739156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:13:06,034][70582] Avg episode reward: [(0, '87.000'), (1, '103.940')] [2023-10-11 22:13:06,273][71601] Updated weights for policy 0, policy_version 79600 (0.0008) [2023-10-11 22:13:06,635][71601] Updated weights for policy 0, policy_version 79610 (0.0010) [2023-10-11 22:13:08,254][71635] Updated weights for policy 1, policy_version 79522 (0.0007) [2023-10-11 22:13:08,629][71635] Updated weights for policy 1, policy_version 79532 (0.0009) [2023-10-11 22:13:08,985][71635] Updated weights for policy 1, policy_version 79542 (0.0008) [2023-10-11 22:13:09,354][71635] Updated weights for policy 1, policy_version 79552 (0.0009) [2023-10-11 22:13:10,253][71601] Updated weights for policy 0, policy_version 79620 (0.0010) [2023-10-11 22:13:10,631][71601] Updated weights for policy 0, policy_version 79630 (0.0008) [2023-10-11 22:13:10,997][71601] Updated weights for policy 0, policy_version 79640 (0.0007) [2023-10-11 22:13:11,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 162988032. Throughput: 0: 1823.1, 1: 1810.1. Samples: 40760868. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:13:11,035][70582] Avg episode reward: [(0, '88.650'), (1, '96.200')] [2023-10-11 22:13:12,961][71635] Updated weights for policy 1, policy_version 79562 (0.0009) [2023-10-11 22:13:13,338][71635] Updated weights for policy 1, policy_version 79572 (0.0008) [2023-10-11 22:13:13,704][71635] Updated weights for policy 1, policy_version 79582 (0.0008) [2023-10-11 22:13:14,676][71601] Updated weights for policy 0, policy_version 79650 (0.0010) [2023-10-11 22:13:15,051][71601] Updated weights for policy 0, policy_version 79660 (0.0009) [2023-10-11 22:13:15,432][71601] Updated weights for policy 0, policy_version 79670 (0.0008) [2023-10-11 22:13:15,798][71601] Updated weights for policy 0, policy_version 79680 (0.0009) [2023-10-11 22:13:16,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 163086336. Throughput: 0: 1819.0, 1: 1823.3. Samples: 40771792. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:13:16,034][70582] Avg episode reward: [(0, '87.320'), (1, '102.340')] [2023-10-11 22:13:17,298][71635] Updated weights for policy 1, policy_version 79592 (0.0010) [2023-10-11 22:13:17,671][71635] Updated weights for policy 1, policy_version 79602 (0.0007) [2023-10-11 22:13:18,029][71635] Updated weights for policy 1, policy_version 79612 (0.0007) [2023-10-11 22:13:19,505][71601] Updated weights for policy 0, policy_version 79690 (0.0010) [2023-10-11 22:13:19,889][71601] Updated weights for policy 0, policy_version 79700 (0.0009) [2023-10-11 22:13:20,267][71601] Updated weights for policy 0, policy_version 79710 (0.0007) [2023-10-11 22:13:21,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 163151872. Throughput: 0: 1819.8, 1: 1823.6. Samples: 40793920. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:13:21,034][70582] Avg episode reward: [(0, '87.630'), (1, '103.740')] [2023-10-11 22:13:21,788][71635] Updated weights for policy 1, policy_version 79622 (0.0007) [2023-10-11 22:13:22,155][71635] Updated weights for policy 1, policy_version 79632 (0.0007) [2023-10-11 22:13:22,523][71635] Updated weights for policy 1, policy_version 79642 (0.0008) [2023-10-11 22:13:23,916][71601] Updated weights for policy 0, policy_version 79720 (0.0009) [2023-10-11 22:13:24,286][71601] Updated weights for policy 0, policy_version 79730 (0.0010) [2023-10-11 22:13:24,662][71601] Updated weights for policy 0, policy_version 79740 (0.0008) [2023-10-11 22:13:26,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 163217408. Throughput: 0: 1817.4, 1: 1822.1. Samples: 40815454. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:13:26,034][70582] Avg episode reward: [(0, '88.760'), (1, '103.110')] [2023-10-11 22:13:26,350][71635] Updated weights for policy 1, policy_version 79652 (0.0009) [2023-10-11 22:13:26,713][71635] Updated weights for policy 1, policy_version 79662 (0.0007) [2023-10-11 22:13:27,082][71635] Updated weights for policy 1, policy_version 79672 (0.0008) [2023-10-11 22:13:28,290][71601] Updated weights for policy 0, policy_version 79750 (0.0009) [2023-10-11 22:13:28,656][71601] Updated weights for policy 0, policy_version 79760 (0.0011) [2023-10-11 22:13:29,034][71601] Updated weights for policy 0, policy_version 79770 (0.0011) [2023-10-11 22:13:30,714][71635] Updated weights for policy 1, policy_version 79682 (0.0010) [2023-10-11 22:13:31,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 163282944. Throughput: 0: 1817.3, 1: 1822.2. Samples: 40826526. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:13:31,034][70582] Avg episode reward: [(0, '91.850'), (1, '106.520')] [2023-10-11 22:13:31,087][71635] Updated weights for policy 1, policy_version 79692 (0.0008) [2023-10-11 22:13:31,454][71635] Updated weights for policy 1, policy_version 79702 (0.0008) [2023-10-11 22:13:31,823][71635] Updated weights for policy 1, policy_version 79712 (0.0007) [2023-10-11 22:13:32,600][71601] Updated weights for policy 0, policy_version 79780 (0.0009) [2023-10-11 22:13:32,977][71601] Updated weights for policy 0, policy_version 79790 (0.0010) [2023-10-11 22:13:33,346][71601] Updated weights for policy 0, policy_version 79800 (0.0010) [2023-10-11 22:13:35,407][71635] Updated weights for policy 1, policy_version 79722 (0.0008) [2023-10-11 22:13:35,776][71635] Updated weights for policy 1, policy_version 79732 (0.0007) [2023-10-11 22:13:36,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 163348480. Throughput: 0: 1821.9, 1: 1823.9. Samples: 40848294. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:13:36,035][70582] Avg episode reward: [(0, '86.370'), (1, '110.700')] [2023-10-11 22:13:36,141][71635] Updated weights for policy 1, policy_version 79742 (0.0007) [2023-10-11 22:13:37,234][71601] Updated weights for policy 0, policy_version 79810 (0.0008) [2023-10-11 22:13:37,657][71601] Updated weights for policy 0, policy_version 79820 (0.0010) [2023-10-11 22:13:38,028][71601] Updated weights for policy 0, policy_version 79830 (0.0008) [2023-10-11 22:13:38,405][71601] Updated weights for policy 0, policy_version 79840 (0.0010) [2023-10-11 22:13:39,879][71635] Updated weights for policy 1, policy_version 79752 (0.0010) [2023-10-11 22:13:40,236][71635] Updated weights for policy 1, policy_version 79762 (0.0009) [2023-10-11 22:13:40,595][71635] Updated weights for policy 1, policy_version 79772 (0.0010) [2023-10-11 22:13:41,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 163446784. Throughput: 0: 1817.9, 1: 1816.7. Samples: 40869880. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:13:41,035][70582] Avg episode reward: [(0, '91.570'), (1, '107.700')] [2023-10-11 22:13:42,000][71601] Updated weights for policy 0, policy_version 79850 (0.0008) [2023-10-11 22:13:42,379][71601] Updated weights for policy 0, policy_version 79860 (0.0008) [2023-10-11 22:13:42,745][71601] Updated weights for policy 0, policy_version 79870 (0.0007) [2023-10-11 22:13:44,087][71635] Updated weights for policy 1, policy_version 79782 (0.0007) [2023-10-11 22:13:44,455][71635] Updated weights for policy 1, policy_version 79792 (0.0009) [2023-10-11 22:13:44,815][71635] Updated weights for policy 1, policy_version 79802 (0.0007) [2023-10-11 22:13:46,034][70582] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 163512320. Throughput: 0: 1818.2, 1: 1824.1. Samples: 40881058. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:13:46,034][70582] Avg episode reward: [(0, '88.760'), (1, '109.540')] [2023-10-11 22:13:46,536][71601] Updated weights for policy 0, policy_version 79880 (0.0010) [2023-10-11 22:13:46,894][71601] Updated weights for policy 0, policy_version 79890 (0.0010) [2023-10-11 22:13:47,270][71601] Updated weights for policy 0, policy_version 79900 (0.0008) [2023-10-11 22:13:48,672][71635] Updated weights for policy 1, policy_version 79812 (0.0008) [2023-10-11 22:13:49,073][71635] Updated weights for policy 1, policy_version 79822 (0.0010) [2023-10-11 22:13:49,434][71635] Updated weights for policy 1, policy_version 79832 (0.0007) [2023-10-11 22:13:51,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 163577856. Throughput: 0: 1823.5, 1: 1813.7. Samples: 40902828. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:13:51,034][70582] Avg episode reward: [(0, '89.070'), (1, '115.880')] [2023-10-11 22:13:51,058][71601] Updated weights for policy 0, policy_version 79910 (0.0008) [2023-10-11 22:13:51,443][71601] Updated weights for policy 0, policy_version 79920 (0.0008) [2023-10-11 22:13:51,808][71601] Updated weights for policy 0, policy_version 79930 (0.0008) [2023-10-11 22:13:53,179][71635] Updated weights for policy 1, policy_version 79842 (0.0008) [2023-10-11 22:13:53,553][71635] Updated weights for policy 1, policy_version 79852 (0.0010) [2023-10-11 22:13:53,930][71635] Updated weights for policy 1, policy_version 79862 (0.0009) [2023-10-11 22:13:54,291][71635] Updated weights for policy 1, policy_version 79872 (0.0008) [2023-10-11 22:13:55,386][71601] Updated weights for policy 0, policy_version 79940 (0.0009) [2023-10-11 22:13:55,755][71601] Updated weights for policy 0, policy_version 79950 (0.0007) [2023-10-11 22:13:56,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.6, 300 sec: 14551.2). Total num frames: 163643392. Throughput: 0: 1824.5, 1: 1820.7. Samples: 40924904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:13:56,034][70582] Avg episode reward: [(0, '86.220'), (1, '122.830')] [2023-10-11 22:13:56,042][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000079872_81788928.pth... [2023-10-11 22:13:56,072][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000078176_80052224.pth [2023-10-11 22:13:56,128][71601] Updated weights for policy 0, policy_version 79960 (0.0007) [2023-10-11 22:13:56,424][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000079968_81887232.pth... [2023-10-11 22:13:56,461][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000078240_80117760.pth [2023-10-11 22:13:57,984][71635] Updated weights for policy 1, policy_version 79882 (0.0007) [2023-10-11 22:13:58,353][71635] Updated weights for policy 1, policy_version 79892 (0.0008) [2023-10-11 22:13:58,726][71635] Updated weights for policy 1, policy_version 79902 (0.0007) [2023-10-11 22:13:59,797][71601] Updated weights for policy 0, policy_version 79970 (0.0009) [2023-10-11 22:14:00,170][71601] Updated weights for policy 0, policy_version 79980 (0.0010) [2023-10-11 22:14:00,545][71601] Updated weights for policy 0, policy_version 79990 (0.0011) [2023-10-11 22:14:00,918][71601] Updated weights for policy 0, policy_version 80000 (0.0009) [2023-10-11 22:14:01,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 163741696. Throughput: 0: 1823.0, 1: 1820.3. Samples: 40935742. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:14:01,034][70582] Avg episode reward: [(0, '84.930'), (1, '121.380')] [2023-10-11 22:14:02,327][71635] Updated weights for policy 1, policy_version 79912 (0.0009) [2023-10-11 22:14:02,704][71635] Updated weights for policy 1, policy_version 79922 (0.0011) [2023-10-11 22:14:03,078][71635] Updated weights for policy 1, policy_version 79932 (0.0011) [2023-10-11 22:14:04,615][71601] Updated weights for policy 0, policy_version 80010 (0.0011) [2023-10-11 22:14:04,982][71601] Updated weights for policy 0, policy_version 80020 (0.0010) [2023-10-11 22:14:05,350][71601] Updated weights for policy 0, policy_version 80030 (0.0009) [2023-10-11 22:14:06,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 163807232. Throughput: 0: 1821.2, 1: 1816.3. Samples: 40957610. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:14:06,034][70582] Avg episode reward: [(0, '85.460'), (1, '121.060')] [2023-10-11 22:14:06,743][71635] Updated weights for policy 1, policy_version 79942 (0.0011) [2023-10-11 22:14:07,105][71635] Updated weights for policy 1, policy_version 79952 (0.0011) [2023-10-11 22:14:07,471][71635] Updated weights for policy 1, policy_version 79962 (0.0011) [2023-10-11 22:14:08,850][71601] Updated weights for policy 0, policy_version 80040 (0.0009) [2023-10-11 22:14:09,230][71601] Updated weights for policy 0, policy_version 80050 (0.0008) [2023-10-11 22:14:09,604][71601] Updated weights for policy 0, policy_version 80060 (0.0009) [2023-10-11 22:14:11,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 163872768. Throughput: 0: 1824.1, 1: 1821.6. Samples: 40979512. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:14:11,034][70582] Avg episode reward: [(0, '87.400'), (1, '118.580')] [2023-10-11 22:14:11,102][71635] Updated weights for policy 1, policy_version 79972 (0.0009) [2023-10-11 22:14:11,464][71635] Updated weights for policy 1, policy_version 79982 (0.0007) [2023-10-11 22:14:11,820][71635] Updated weights for policy 1, policy_version 79992 (0.0008) [2023-10-11 22:14:13,382][71601] Updated weights for policy 0, policy_version 80070 (0.0008) [2023-10-11 22:14:13,759][71601] Updated weights for policy 0, policy_version 80080 (0.0008) [2023-10-11 22:14:14,134][71601] Updated weights for policy 0, policy_version 80090 (0.0010) [2023-10-11 22:14:15,607][71635] Updated weights for policy 1, policy_version 80002 (0.0008) [2023-10-11 22:14:15,962][71635] Updated weights for policy 1, policy_version 80012 (0.0009) [2023-10-11 22:14:16,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 163938304. Throughput: 0: 1821.3, 1: 1824.3. Samples: 40990578. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-11 22:14:16,034][70582] Avg episode reward: [(0, '79.420'), (1, '113.070')] [2023-10-11 22:14:16,327][71635] Updated weights for policy 1, policy_version 80022 (0.0009) [2023-10-11 22:14:16,693][71635] Updated weights for policy 1, policy_version 80032 (0.0009) [2023-10-11 22:14:17,767][71601] Updated weights for policy 0, policy_version 80100 (0.0011) [2023-10-11 22:14:18,140][71601] Updated weights for policy 0, policy_version 80110 (0.0010) [2023-10-11 22:14:18,521][71601] Updated weights for policy 0, policy_version 80120 (0.0010) [2023-10-11 22:14:20,430][71635] Updated weights for policy 1, policy_version 80042 (0.0010) [2023-10-11 22:14:20,802][71635] Updated weights for policy 1, policy_version 80052 (0.0007) [2023-10-11 22:14:21,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 164003840. Throughput: 0: 1815.3, 1: 1824.9. Samples: 41012104. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-11 22:14:21,034][70582] Avg episode reward: [(0, '84.870'), (1, '115.760')] [2023-10-11 22:14:21,166][71635] Updated weights for policy 1, policy_version 80062 (0.0009) [2023-10-11 22:14:22,195][71601] Updated weights for policy 0, policy_version 80130 (0.0009) [2023-10-11 22:14:22,600][71601] Updated weights for policy 0, policy_version 80140 (0.0009) [2023-10-11 22:14:22,977][71601] Updated weights for policy 0, policy_version 80150 (0.0008) [2023-10-11 22:14:23,349][71601] Updated weights for policy 0, policy_version 80160 (0.0007) [2023-10-11 22:14:24,866][71635] Updated weights for policy 1, policy_version 80072 (0.0009) [2023-10-11 22:14:25,240][71635] Updated weights for policy 1, policy_version 80082 (0.0007) [2023-10-11 22:14:25,611][71635] Updated weights for policy 1, policy_version 80092 (0.0007) [2023-10-11 22:14:26,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 164102144. Throughput: 0: 1818.8, 1: 1827.3. Samples: 41033952. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-11 22:14:26,035][70582] Avg episode reward: [(0, '90.410'), (1, '119.840')] [2023-10-11 22:14:27,061][71601] Updated weights for policy 0, policy_version 80170 (0.0007) [2023-10-11 22:14:27,434][71601] Updated weights for policy 0, policy_version 80180 (0.0007) [2023-10-11 22:14:27,800][71601] Updated weights for policy 0, policy_version 80190 (0.0008) [2023-10-11 22:14:29,273][71635] Updated weights for policy 1, policy_version 80102 (0.0007) [2023-10-11 22:14:29,645][71635] Updated weights for policy 1, policy_version 80112 (0.0007) [2023-10-11 22:14:30,015][71635] Updated weights for policy 1, policy_version 80122 (0.0009) [2023-10-11 22:14:31,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 164167680. Throughput: 0: 1818.4, 1: 1820.8. Samples: 41044824. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-11 22:14:31,034][70582] Avg episode reward: [(0, '89.330'), (1, '121.820')] [2023-10-11 22:14:31,660][71601] Updated weights for policy 0, policy_version 80200 (0.0008) [2023-10-11 22:14:32,030][71601] Updated weights for policy 0, policy_version 80210 (0.0007) [2023-10-11 22:14:32,403][71601] Updated weights for policy 0, policy_version 80220 (0.0008) [2023-10-11 22:14:33,689][71635] Updated weights for policy 1, policy_version 80132 (0.0009) [2023-10-11 22:14:34,088][71635] Updated weights for policy 1, policy_version 80142 (0.0010) [2023-10-11 22:14:34,463][71635] Updated weights for policy 1, policy_version 80152 (0.0007) [2023-10-11 22:14:36,011][71601] Updated weights for policy 0, policy_version 80230 (0.0009) [2023-10-11 22:14:36,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 164233216. Throughput: 0: 1820.3, 1: 1822.8. Samples: 41066770. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-11 22:14:36,034][70582] Avg episode reward: [(0, '94.580'), (1, '117.560')] [2023-10-11 22:14:36,389][71601] Updated weights for policy 0, policy_version 80240 (0.0010) [2023-10-11 22:14:36,761][71601] Updated weights for policy 0, policy_version 80250 (0.0011) [2023-10-11 22:14:38,025][71635] Updated weights for policy 1, policy_version 80162 (0.0008) [2023-10-11 22:14:38,393][71635] Updated weights for policy 1, policy_version 80172 (0.0007) [2023-10-11 22:14:38,770][71635] Updated weights for policy 1, policy_version 80182 (0.0008) [2023-10-11 22:14:39,124][71635] Updated weights for policy 1, policy_version 80192 (0.0009) [2023-10-11 22:14:40,525][71601] Updated weights for policy 0, policy_version 80260 (0.0007) [2023-10-11 22:14:40,903][71601] Updated weights for policy 0, policy_version 80270 (0.0009) [2023-10-11 22:14:41,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 164298752. Throughput: 0: 1820.8, 1: 1822.3. Samples: 41088846. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-11 22:14:41,034][70582] Avg episode reward: [(0, '100.290'), (1, '122.380')] [2023-10-11 22:14:41,280][71601] Updated weights for policy 0, policy_version 80280 (0.0007) [2023-10-11 22:14:42,820][71635] Updated weights for policy 1, policy_version 80202 (0.0010) [2023-10-11 22:14:43,183][71635] Updated weights for policy 1, policy_version 80212 (0.0008) [2023-10-11 22:14:43,553][71635] Updated weights for policy 1, policy_version 80222 (0.0007) [2023-10-11 22:14:44,774][71601] Updated weights for policy 0, policy_version 80290 (0.0009) [2023-10-11 22:14:45,155][71601] Updated weights for policy 0, policy_version 80300 (0.0009) [2023-10-11 22:14:45,522][71601] Updated weights for policy 0, policy_version 80310 (0.0008) [2023-10-11 22:14:45,898][71601] Updated weights for policy 0, policy_version 80320 (0.0007) [2023-10-11 22:14:46,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 164397056. Throughput: 0: 1820.5, 1: 1817.4. Samples: 41099450. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-11 22:14:46,034][70582] Avg episode reward: [(0, '102.810'), (1, '119.830')] [2023-10-11 22:14:47,342][71635] Updated weights for policy 1, policy_version 80232 (0.0009) [2023-10-11 22:14:47,710][71635] Updated weights for policy 1, policy_version 80242 (0.0009) [2023-10-11 22:14:48,082][71635] Updated weights for policy 1, policy_version 80252 (0.0008) [2023-10-11 22:14:49,480][71601] Updated weights for policy 0, policy_version 80330 (0.0011) [2023-10-11 22:14:49,845][71601] Updated weights for policy 0, policy_version 80340 (0.0010) [2023-10-11 22:14:50,222][71601] Updated weights for policy 0, policy_version 80350 (0.0009) [2023-10-11 22:14:51,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 164462592. Throughput: 0: 1817.5, 1: 1827.5. Samples: 41121634. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-11 22:14:51,034][70582] Avg episode reward: [(0, '103.580'), (1, '110.370')] [2023-10-11 22:14:51,743][71635] Updated weights for policy 1, policy_version 80262 (0.0007) [2023-10-11 22:14:52,110][71635] Updated weights for policy 1, policy_version 80272 (0.0010) [2023-10-11 22:14:52,473][71635] Updated weights for policy 1, policy_version 80282 (0.0010) [2023-10-11 22:14:53,799][71601] Updated weights for policy 0, policy_version 80360 (0.0009) [2023-10-11 22:14:54,175][71601] Updated weights for policy 0, policy_version 80370 (0.0010) [2023-10-11 22:14:54,547][71601] Updated weights for policy 0, policy_version 80380 (0.0009) [2023-10-11 22:14:56,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 164528128. Throughput: 0: 1820.5, 1: 1826.7. Samples: 41143634. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-11 22:14:56,034][70582] Avg episode reward: [(0, '102.330'), (1, '108.290')] [2023-10-11 22:14:56,180][71635] Updated weights for policy 1, policy_version 80292 (0.0007) [2023-10-11 22:14:56,545][71635] Updated weights for policy 1, policy_version 80302 (0.0007) [2023-10-11 22:14:56,906][71635] Updated weights for policy 1, policy_version 80312 (0.0009) [2023-10-11 22:14:58,155][71601] Updated weights for policy 0, policy_version 80390 (0.0008) [2023-10-11 22:14:58,532][71601] Updated weights for policy 0, policy_version 80400 (0.0007) [2023-10-11 22:14:58,898][71601] Updated weights for policy 0, policy_version 80410 (0.0007) [2023-10-11 22:15:00,385][71635] Updated weights for policy 1, policy_version 80322 (0.0008) [2023-10-11 22:15:00,752][71635] Updated weights for policy 1, policy_version 80332 (0.0007) [2023-10-11 22:15:01,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 164593664. Throughput: 0: 1817.1, 1: 1826.5. Samples: 41154542. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-11 22:15:01,035][70582] Avg episode reward: [(0, '101.080'), (1, '109.800')] [2023-10-11 22:15:01,118][71635] Updated weights for policy 1, policy_version 80342 (0.0007) [2023-10-11 22:15:01,491][71635] Updated weights for policy 1, policy_version 80352 (0.0008) [2023-10-11 22:15:02,583][71601] Updated weights for policy 0, policy_version 80420 (0.0009) [2023-10-11 22:15:02,960][71601] Updated weights for policy 0, policy_version 80430 (0.0007) [2023-10-11 22:15:03,329][71601] Updated weights for policy 0, policy_version 80440 (0.0007) [2023-10-11 22:15:05,291][71635] Updated weights for policy 1, policy_version 80362 (0.0008) [2023-10-11 22:15:05,663][71635] Updated weights for policy 1, policy_version 80372 (0.0008) [2023-10-11 22:15:06,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 164659200. Throughput: 0: 1826.1, 1: 1825.7. Samples: 41176434. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-11 22:15:06,034][70582] Avg episode reward: [(0, '104.540'), (1, '111.760')] [2023-10-11 22:15:06,045][71635] Updated weights for policy 1, policy_version 80382 (0.0008) [2023-10-11 22:15:07,109][71601] Updated weights for policy 0, policy_version 80450 (0.0009) [2023-10-11 22:15:07,483][71601] Updated weights for policy 0, policy_version 80460 (0.0009) [2023-10-11 22:15:07,880][71601] Updated weights for policy 0, policy_version 80470 (0.0009) [2023-10-11 22:15:08,251][71601] Updated weights for policy 0, policy_version 80480 (0.0009) [2023-10-11 22:15:09,750][71635] Updated weights for policy 1, policy_version 80392 (0.0010) [2023-10-11 22:15:10,115][71635] Updated weights for policy 1, policy_version 80402 (0.0007) [2023-10-11 22:15:10,478][71635] Updated weights for policy 1, policy_version 80412 (0.0010) [2023-10-11 22:15:11,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 164757504. Throughput: 0: 1824.6, 1: 1821.4. Samples: 41198022. Policy #0 lag: (min: 20.0, avg: 24.6, max: 52.0) [2023-10-11 22:15:11,035][70582] Avg episode reward: [(0, '104.470'), (1, '103.520')] [2023-10-11 22:15:11,998][71601] Updated weights for policy 0, policy_version 80490 (0.0007) [2023-10-11 22:15:12,372][71601] Updated weights for policy 0, policy_version 80500 (0.0007) [2023-10-11 22:15:12,745][71601] Updated weights for policy 0, policy_version 80510 (0.0008) [2023-10-11 22:15:14,151][71635] Updated weights for policy 1, policy_version 80422 (0.0010) [2023-10-11 22:15:14,516][71635] Updated weights for policy 1, policy_version 80432 (0.0011) [2023-10-11 22:15:14,886][71635] Updated weights for policy 1, policy_version 80442 (0.0011) [2023-10-11 22:15:16,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 164823040. Throughput: 0: 1822.2, 1: 1826.3. Samples: 41209008. Policy #0 lag: (min: 20.0, avg: 24.6, max: 52.0) [2023-10-11 22:15:16,035][70582] Avg episode reward: [(0, '100.800'), (1, '97.250')] [2023-10-11 22:15:16,344][71601] Updated weights for policy 0, policy_version 80520 (0.0007) [2023-10-11 22:15:16,716][71601] Updated weights for policy 0, policy_version 80530 (0.0008) [2023-10-11 22:15:17,081][71601] Updated weights for policy 0, policy_version 80540 (0.0011) [2023-10-11 22:15:18,604][71635] Updated weights for policy 1, policy_version 80452 (0.0009) [2023-10-11 22:15:18,998][71635] Updated weights for policy 1, policy_version 80462 (0.0009) [2023-10-11 22:15:19,361][71635] Updated weights for policy 1, policy_version 80472 (0.0008) [2023-10-11 22:15:20,810][71601] Updated weights for policy 0, policy_version 80550 (0.0010) [2023-10-11 22:15:21,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 164888576. Throughput: 0: 1823.0, 1: 1821.1. Samples: 41230758. Policy #0 lag: (min: 20.0, avg: 24.6, max: 52.0) [2023-10-11 22:15:21,035][70582] Avg episode reward: [(0, '96.880'), (1, '96.260')] [2023-10-11 22:15:21,181][71601] Updated weights for policy 0, policy_version 80560 (0.0012) [2023-10-11 22:15:21,552][71601] Updated weights for policy 0, policy_version 80570 (0.0011) [2023-10-11 22:15:23,025][71635] Updated weights for policy 1, policy_version 80482 (0.0008) [2023-10-11 22:15:23,385][71635] Updated weights for policy 1, policy_version 80492 (0.0011) [2023-10-11 22:15:23,754][71635] Updated weights for policy 1, policy_version 80502 (0.0007) [2023-10-11 22:15:24,123][71635] Updated weights for policy 1, policy_version 80512 (0.0010) [2023-10-11 22:15:25,212][71601] Updated weights for policy 0, policy_version 80580 (0.0009) [2023-10-11 22:15:25,592][71601] Updated weights for policy 0, policy_version 80590 (0.0007) [2023-10-11 22:15:25,963][71601] Updated weights for policy 0, policy_version 80600 (0.0007) [2023-10-11 22:15:26,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 164954112. Throughput: 0: 1817.1, 1: 1822.9. Samples: 41252650. Policy #0 lag: (min: 20.0, avg: 24.6, max: 52.0) [2023-10-11 22:15:26,035][70582] Avg episode reward: [(0, '90.690'), (1, '101.660')] [2023-10-11 22:15:27,896][71635] Updated weights for policy 1, policy_version 80522 (0.0008) [2023-10-11 22:15:28,270][71635] Updated weights for policy 1, policy_version 80532 (0.0007) [2023-10-11 22:15:28,649][71635] Updated weights for policy 1, policy_version 80542 (0.0007) [2023-10-11 22:15:29,696][71601] Updated weights for policy 0, policy_version 80610 (0.0007) [2023-10-11 22:15:30,075][71601] Updated weights for policy 0, policy_version 80620 (0.0008) [2023-10-11 22:15:30,435][71601] Updated weights for policy 0, policy_version 80630 (0.0011) [2023-10-11 22:15:30,804][71601] Updated weights for policy 0, policy_version 80640 (0.0008) [2023-10-11 22:15:31,034][70582] Fps is (10 sec: 16384.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 165052416. Throughput: 0: 1824.0, 1: 1825.2. Samples: 41263660. Policy #0 lag: (min: 20.0, avg: 24.6, max: 52.0) [2023-10-11 22:15:31,034][70582] Avg episode reward: [(0, '87.930'), (1, '96.950')] [2023-10-11 22:15:32,345][71635] Updated weights for policy 1, policy_version 80552 (0.0007) [2023-10-11 22:15:32,708][71635] Updated weights for policy 1, policy_version 80562 (0.0008) [2023-10-11 22:15:33,081][71635] Updated weights for policy 1, policy_version 80572 (0.0007) [2023-10-11 22:15:34,484][71601] Updated weights for policy 0, policy_version 80650 (0.0007) [2023-10-11 22:15:34,864][71601] Updated weights for policy 0, policy_version 80660 (0.0007) [2023-10-11 22:15:35,235][71601] Updated weights for policy 0, policy_version 80670 (0.0007) [2023-10-11 22:15:36,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 165117952. Throughput: 0: 1819.9, 1: 1820.4. Samples: 41285448. Policy #0 lag: (min: 20.0, avg: 24.6, max: 52.0) [2023-10-11 22:15:36,034][70582] Avg episode reward: [(0, '84.910'), (1, '95.920')] [2023-10-11 22:15:36,705][71635] Updated weights for policy 1, policy_version 80582 (0.0008) [2023-10-11 22:15:37,083][71635] Updated weights for policy 1, policy_version 80592 (0.0008) [2023-10-11 22:15:37,446][71635] Updated weights for policy 1, policy_version 80602 (0.0010) [2023-10-11 22:15:38,977][71601] Updated weights for policy 0, policy_version 80680 (0.0007) [2023-10-11 22:15:39,348][71601] Updated weights for policy 0, policy_version 80690 (0.0008) [2023-10-11 22:15:39,725][71601] Updated weights for policy 0, policy_version 80700 (0.0008) [2023-10-11 22:15:41,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 165183488. Throughput: 0: 1816.2, 1: 1821.3. Samples: 41307320. Policy #0 lag: (min: 20.0, avg: 24.6, max: 52.0) [2023-10-11 22:15:41,034][70582] Avg episode reward: [(0, '87.050'), (1, '102.190')] [2023-10-11 22:15:41,050][71635] Updated weights for policy 1, policy_version 80612 (0.0008) [2023-10-11 22:15:41,408][71635] Updated weights for policy 1, policy_version 80622 (0.0007) [2023-10-11 22:15:41,776][71635] Updated weights for policy 1, policy_version 80632 (0.0007) [2023-10-11 22:15:43,501][71601] Updated weights for policy 0, policy_version 80710 (0.0008) [2023-10-11 22:15:43,871][71601] Updated weights for policy 0, policy_version 80720 (0.0008) [2023-10-11 22:15:44,238][71601] Updated weights for policy 0, policy_version 80730 (0.0010) [2023-10-11 22:15:45,462][71635] Updated weights for policy 1, policy_version 80642 (0.0009) [2023-10-11 22:15:45,835][71635] Updated weights for policy 1, policy_version 80652 (0.0010) [2023-10-11 22:15:46,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 165249024. Throughput: 0: 1824.8, 1: 1819.9. Samples: 41318550. Policy #0 lag: (min: 20.0, avg: 24.6, max: 52.0) [2023-10-11 22:15:46,034][70582] Avg episode reward: [(0, '87.960'), (1, '102.410')] [2023-10-11 22:15:46,190][71635] Updated weights for policy 1, policy_version 80662 (0.0010) [2023-10-11 22:15:46,556][71635] Updated weights for policy 1, policy_version 80672 (0.0009) [2023-10-11 22:15:47,680][71601] Updated weights for policy 0, policy_version 80740 (0.0009) [2023-10-11 22:15:48,058][71601] Updated weights for policy 0, policy_version 80750 (0.0008) [2023-10-11 22:15:48,427][71601] Updated weights for policy 0, policy_version 80760 (0.0008) [2023-10-11 22:15:50,188][71635] Updated weights for policy 1, policy_version 80682 (0.0008) [2023-10-11 22:15:50,548][71635] Updated weights for policy 1, policy_version 80692 (0.0007) [2023-10-11 22:15:50,921][71635] Updated weights for policy 1, policy_version 80702 (0.0007) [2023-10-11 22:15:51,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 165347328. Throughput: 0: 1815.7, 1: 1821.7. Samples: 41340120. Policy #0 lag: (min: 20.0, avg: 24.6, max: 52.0) [2023-10-11 22:15:51,035][70582] Avg episode reward: [(0, '88.940'), (1, '102.740')] [2023-10-11 22:15:52,268][71601] Updated weights for policy 0, policy_version 80770 (0.0010) [2023-10-11 22:15:52,646][71601] Updated weights for policy 0, policy_version 80780 (0.0007) [2023-10-11 22:15:53,023][71601] Updated weights for policy 0, policy_version 80790 (0.0007) [2023-10-11 22:15:53,388][71601] Updated weights for policy 0, policy_version 80800 (0.0010) [2023-10-11 22:15:54,733][71635] Updated weights for policy 1, policy_version 80712 (0.0008) [2023-10-11 22:15:55,094][71635] Updated weights for policy 1, policy_version 80722 (0.0008) [2023-10-11 22:15:55,465][71635] Updated weights for policy 1, policy_version 80732 (0.0007) [2023-10-11 22:15:56,034][70582] Fps is (10 sec: 16383.4, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 165412864. Throughput: 0: 1817.7, 1: 1820.7. Samples: 41361750. Policy #0 lag: (min: 20.0, avg: 24.6, max: 52.0) [2023-10-11 22:15:56,035][70582] Avg episode reward: [(0, '85.770'), (1, '99.460')] [2023-10-11 22:15:56,045][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000080800_82739200.pth... [2023-10-11 22:15:56,046][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000080736_82673664.pth... [2023-10-11 22:15:56,086][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000079040_80936960.pth [2023-10-11 22:15:56,087][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000079104_81002496.pth [2023-10-11 22:15:57,256][71601] Updated weights for policy 0, policy_version 80810 (0.0007) [2023-10-11 22:15:57,627][71601] Updated weights for policy 0, policy_version 80820 (0.0008) [2023-10-11 22:15:58,005][71601] Updated weights for policy 0, policy_version 80830 (0.0008) [2023-10-11 22:15:59,189][71635] Updated weights for policy 1, policy_version 80742 (0.0007) [2023-10-11 22:15:59,556][71635] Updated weights for policy 1, policy_version 80752 (0.0008) [2023-10-11 22:15:59,918][71635] Updated weights for policy 1, policy_version 80762 (0.0008) [2023-10-11 22:16:01,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 165478400. Throughput: 0: 1818.6, 1: 1818.3. Samples: 41372670. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-11 22:16:01,035][70582] Avg episode reward: [(0, '74.770'), (1, '97.600')] [2023-10-11 22:16:01,646][71601] Updated weights for policy 0, policy_version 80840 (0.0009) [2023-10-11 22:16:02,017][71601] Updated weights for policy 0, policy_version 80850 (0.0007) [2023-10-11 22:16:02,388][71601] Updated weights for policy 0, policy_version 80860 (0.0008) [2023-10-11 22:16:03,627][71635] Updated weights for policy 1, policy_version 80772 (0.0008) [2023-10-11 22:16:03,984][71635] Updated weights for policy 1, policy_version 80782 (0.0008) [2023-10-11 22:16:04,355][71635] Updated weights for policy 1, policy_version 80792 (0.0008) [2023-10-11 22:16:06,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 165543936. Throughput: 0: 1817.7, 1: 1820.9. Samples: 41394494. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-11 22:16:06,035][70582] Avg episode reward: [(0, '77.560'), (1, '97.540')] [2023-10-11 22:16:06,040][71601] Updated weights for policy 0, policy_version 80870 (0.0008) [2023-10-11 22:16:06,417][71601] Updated weights for policy 0, policy_version 80880 (0.0009) [2023-10-11 22:16:06,784][71601] Updated weights for policy 0, policy_version 80890 (0.0009) [2023-10-11 22:16:08,050][71635] Updated weights for policy 1, policy_version 80802 (0.0008) [2023-10-11 22:16:08,423][71635] Updated weights for policy 1, policy_version 80812 (0.0011) [2023-10-11 22:16:08,784][71635] Updated weights for policy 1, policy_version 80822 (0.0011) [2023-10-11 22:16:09,149][71635] Updated weights for policy 1, policy_version 80832 (0.0011) [2023-10-11 22:16:10,521][71601] Updated weights for policy 0, policy_version 80900 (0.0009) [2023-10-11 22:16:10,893][71601] Updated weights for policy 0, policy_version 80910 (0.0008) [2023-10-11 22:16:11,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 165609472. Throughput: 0: 1823.2, 1: 1818.4. Samples: 41416518. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-11 22:16:11,034][70582] Avg episode reward: [(0, '76.190'), (1, '110.370')] [2023-10-11 22:16:11,258][71601] Updated weights for policy 0, policy_version 80920 (0.0009) [2023-10-11 22:16:12,942][71635] Updated weights for policy 1, policy_version 80842 (0.0011) [2023-10-11 22:16:13,314][71635] Updated weights for policy 1, policy_version 80852 (0.0010) [2023-10-11 22:16:13,689][71635] Updated weights for policy 1, policy_version 80862 (0.0010) [2023-10-11 22:16:14,982][71601] Updated weights for policy 0, policy_version 80930 (0.0010) [2023-10-11 22:16:15,359][71601] Updated weights for policy 0, policy_version 80940 (0.0009) [2023-10-11 22:16:15,728][71601] Updated weights for policy 0, policy_version 80950 (0.0009) [2023-10-11 22:16:16,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 165675008. Throughput: 0: 1811.9, 1: 1819.3. Samples: 41427066. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-11 22:16:16,034][70582] Avg episode reward: [(0, '78.340'), (1, '110.900')] [2023-10-11 22:16:16,094][71601] Updated weights for policy 0, policy_version 80960 (0.0008) [2023-10-11 22:16:17,464][71635] Updated weights for policy 1, policy_version 80872 (0.0008) [2023-10-11 22:16:17,833][71635] Updated weights for policy 1, policy_version 80882 (0.0007) [2023-10-11 22:16:18,208][71635] Updated weights for policy 1, policy_version 80892 (0.0007) [2023-10-11 22:16:19,681][71601] Updated weights for policy 0, policy_version 80970 (0.0011) [2023-10-11 22:16:20,048][71601] Updated weights for policy 0, policy_version 80980 (0.0011) [2023-10-11 22:16:20,414][71601] Updated weights for policy 0, policy_version 80990 (0.0011) [2023-10-11 22:16:21,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 165773312. Throughput: 0: 1818.1, 1: 1812.1. Samples: 41448810. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-11 22:16:21,034][70582] Avg episode reward: [(0, '81.150'), (1, '111.510')] [2023-10-11 22:16:21,836][71635] Updated weights for policy 1, policy_version 80902 (0.0009) [2023-10-11 22:16:22,202][71635] Updated weights for policy 1, policy_version 80912 (0.0010) [2023-10-11 22:16:22,564][71635] Updated weights for policy 1, policy_version 80922 (0.0009) [2023-10-11 22:16:24,194][71601] Updated weights for policy 0, policy_version 81000 (0.0009) [2023-10-11 22:16:24,554][71601] Updated weights for policy 0, policy_version 81010 (0.0007) [2023-10-11 22:16:24,928][71601] Updated weights for policy 0, policy_version 81020 (0.0007) [2023-10-11 22:16:26,034][70582] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 165838848. Throughput: 0: 1811.3, 1: 1809.3. Samples: 41470248. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-11 22:16:26,035][70582] Avg episode reward: [(0, '81.820'), (1, '108.960')] [2023-10-11 22:16:26,345][71635] Updated weights for policy 1, policy_version 80932 (0.0008) [2023-10-11 22:16:26,717][71635] Updated weights for policy 1, policy_version 80942 (0.0011) [2023-10-11 22:16:27,075][71635] Updated weights for policy 1, policy_version 80952 (0.0009) [2023-10-11 22:16:28,596][71601] Updated weights for policy 0, policy_version 81030 (0.0010) [2023-10-11 22:16:28,970][71601] Updated weights for policy 0, policy_version 81040 (0.0008) [2023-10-11 22:16:29,334][71601] Updated weights for policy 0, policy_version 81050 (0.0009) [2023-10-11 22:16:30,879][71635] Updated weights for policy 1, policy_version 80962 (0.0007) [2023-10-11 22:16:31,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 165904384. Throughput: 0: 1813.8, 1: 1806.4. Samples: 41481458. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-11 22:16:31,034][70582] Avg episode reward: [(0, '79.980'), (1, '114.160')] [2023-10-11 22:16:31,238][71635] Updated weights for policy 1, policy_version 80972 (0.0007) [2023-10-11 22:16:31,613][71635] Updated weights for policy 1, policy_version 80982 (0.0007) [2023-10-11 22:16:31,969][71635] Updated weights for policy 1, policy_version 80992 (0.0008) [2023-10-11 22:16:33,052][71601] Updated weights for policy 0, policy_version 81060 (0.0010) [2023-10-11 22:16:33,420][71601] Updated weights for policy 0, policy_version 81070 (0.0008) [2023-10-11 22:16:33,796][71601] Updated weights for policy 0, policy_version 81080 (0.0007) [2023-10-11 22:16:35,662][71635] Updated weights for policy 1, policy_version 81002 (0.0007) [2023-10-11 22:16:36,032][71635] Updated weights for policy 1, policy_version 81012 (0.0008) [2023-10-11 22:16:36,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 165969920. Throughput: 0: 1808.7, 1: 1805.5. Samples: 41502758. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-11 22:16:36,034][70582] Avg episode reward: [(0, '79.130'), (1, '115.650')] [2023-10-11 22:16:36,410][71635] Updated weights for policy 1, policy_version 81022 (0.0008) [2023-10-11 22:16:37,553][71601] Updated weights for policy 0, policy_version 81090 (0.0008) [2023-10-11 22:16:37,924][71601] Updated weights for policy 0, policy_version 81100 (0.0007) [2023-10-11 22:16:38,292][71601] Updated weights for policy 0, policy_version 81110 (0.0010) [2023-10-11 22:16:38,660][71601] Updated weights for policy 0, policy_version 81120 (0.0008) [2023-10-11 22:16:40,148][71635] Updated weights for policy 1, policy_version 81032 (0.0008) [2023-10-11 22:16:40,508][71635] Updated weights for policy 1, policy_version 81042 (0.0008) [2023-10-11 22:16:40,872][71635] Updated weights for policy 1, policy_version 81052 (0.0009) [2023-10-11 22:16:41,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 166068224. Throughput: 0: 1805.9, 1: 1814.3. Samples: 41524658. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-11 22:16:41,034][70582] Avg episode reward: [(0, '78.510'), (1, '113.510')] [2023-10-11 22:16:42,565][71601] Updated weights for policy 0, policy_version 81130 (0.0007) [2023-10-11 22:16:42,930][71601] Updated weights for policy 0, policy_version 81140 (0.0009) [2023-10-11 22:16:43,301][71601] Updated weights for policy 0, policy_version 81150 (0.0008) [2023-10-11 22:16:44,498][71635] Updated weights for policy 1, policy_version 81062 (0.0009) [2023-10-11 22:16:44,867][71635] Updated weights for policy 1, policy_version 81072 (0.0009) [2023-10-11 22:16:45,242][71635] Updated weights for policy 1, policy_version 81082 (0.0008) [2023-10-11 22:16:46,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 166133760. Throughput: 0: 1804.2, 1: 1804.1. Samples: 41535042. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-11 22:16:46,034][70582] Avg episode reward: [(0, '84.150'), (1, '110.960')] [2023-10-11 22:16:47,016][71601] Updated weights for policy 0, policy_version 81160 (0.0009) [2023-10-11 22:16:47,386][71601] Updated weights for policy 0, policy_version 81170 (0.0010) [2023-10-11 22:16:47,761][71601] Updated weights for policy 0, policy_version 81180 (0.0009) [2023-10-11 22:16:48,785][71635] Updated weights for policy 1, policy_version 81092 (0.0007) [2023-10-11 22:16:49,153][71635] Updated weights for policy 1, policy_version 81102 (0.0007) [2023-10-11 22:16:49,519][71635] Updated weights for policy 1, policy_version 81112 (0.0009) [2023-10-11 22:16:51,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 166199296. Throughput: 0: 1796.5, 1: 1815.9. Samples: 41557048. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-11 22:16:51,034][70582] Avg episode reward: [(0, '82.190'), (1, '111.540')] [2023-10-11 22:16:51,591][71601] Updated weights for policy 0, policy_version 81190 (0.0009) [2023-10-11 22:16:51,956][71601] Updated weights for policy 0, policy_version 81200 (0.0007) [2023-10-11 22:16:52,338][71601] Updated weights for policy 0, policy_version 81210 (0.0009) [2023-10-11 22:16:53,418][71635] Updated weights for policy 1, policy_version 81122 (0.0009) [2023-10-11 22:16:53,830][71635] Updated weights for policy 1, policy_version 81132 (0.0007) [2023-10-11 22:16:54,199][71635] Updated weights for policy 1, policy_version 81142 (0.0007) [2023-10-11 22:16:54,563][71635] Updated weights for policy 1, policy_version 81152 (0.0009) [2023-10-11 22:16:55,880][71601] Updated weights for policy 0, policy_version 81220 (0.0008) [2023-10-11 22:16:56,034][70582] Fps is (10 sec: 13106.6, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 166264832. Throughput: 0: 1802.1, 1: 1809.2. Samples: 41579026. Policy #0 lag: (min: 5.0, avg: 18.0, max: 37.0) [2023-10-11 22:16:56,036][70582] Avg episode reward: [(0, '85.470'), (1, '113.920')] [2023-10-11 22:16:56,257][71601] Updated weights for policy 0, policy_version 81230 (0.0008) [2023-10-11 22:16:56,619][71601] Updated weights for policy 0, policy_version 81240 (0.0007) [2023-10-11 22:16:58,169][71635] Updated weights for policy 1, policy_version 81162 (0.0010) [2023-10-11 22:16:58,537][71635] Updated weights for policy 1, policy_version 81172 (0.0011) [2023-10-11 22:16:58,892][71635] Updated weights for policy 1, policy_version 81182 (0.0007) [2023-10-11 22:17:00,245][71601] Updated weights for policy 0, policy_version 81250 (0.0009) [2023-10-11 22:17:00,623][71601] Updated weights for policy 0, policy_version 81260 (0.0009) [2023-10-11 22:17:00,993][71601] Updated weights for policy 0, policy_version 81270 (0.0009) [2023-10-11 22:17:01,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 166330368. Throughput: 0: 1801.7, 1: 1814.4. Samples: 41589792. Policy #0 lag: (min: 5.0, avg: 18.0, max: 37.0) [2023-10-11 22:17:01,034][70582] Avg episode reward: [(0, '82.120'), (1, '116.830')] [2023-10-11 22:17:01,364][71601] Updated weights for policy 0, policy_version 81280 (0.0008) [2023-10-11 22:17:02,689][71635] Updated weights for policy 1, policy_version 81192 (0.0009) [2023-10-11 22:17:03,065][71635] Updated weights for policy 1, policy_version 81202 (0.0009) [2023-10-11 22:17:03,423][71635] Updated weights for policy 1, policy_version 81212 (0.0007) [2023-10-11 22:17:05,104][71601] Updated weights for policy 0, policy_version 81290 (0.0009) [2023-10-11 22:17:05,476][71601] Updated weights for policy 0, policy_version 81300 (0.0011) [2023-10-11 22:17:05,844][71601] Updated weights for policy 0, policy_version 81310 (0.0007) [2023-10-11 22:17:06,034][70582] Fps is (10 sec: 16384.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 166428672. Throughput: 0: 1810.2, 1: 1811.9. Samples: 41611804. Policy #0 lag: (min: 5.0, avg: 18.0, max: 37.0) [2023-10-11 22:17:06,035][70582] Avg episode reward: [(0, '79.460'), (1, '109.720')] [2023-10-11 22:17:06,922][71635] Updated weights for policy 1, policy_version 81222 (0.0007) [2023-10-11 22:17:07,287][71635] Updated weights for policy 1, policy_version 81232 (0.0008) [2023-10-11 22:17:07,654][71635] Updated weights for policy 1, policy_version 81242 (0.0008) [2023-10-11 22:17:09,443][71601] Updated weights for policy 0, policy_version 81320 (0.0008) [2023-10-11 22:17:09,817][71601] Updated weights for policy 0, policy_version 81330 (0.0008) [2023-10-11 22:17:10,196][71601] Updated weights for policy 0, policy_version 81340 (0.0009) [2023-10-11 22:17:11,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 166494208. Throughput: 0: 1807.1, 1: 1814.1. Samples: 41633202. Policy #0 lag: (min: 5.0, avg: 18.0, max: 37.0) [2023-10-11 22:17:11,034][70582] Avg episode reward: [(0, '77.930'), (1, '106.580')] [2023-10-11 22:17:11,467][71635] Updated weights for policy 1, policy_version 81252 (0.0008) [2023-10-11 22:17:11,836][71635] Updated weights for policy 1, policy_version 81262 (0.0007) [2023-10-11 22:17:12,209][71635] Updated weights for policy 1, policy_version 81272 (0.0009) [2023-10-11 22:17:13,819][71601] Updated weights for policy 0, policy_version 81350 (0.0009) [2023-10-11 22:17:14,190][71601] Updated weights for policy 0, policy_version 81360 (0.0010) [2023-10-11 22:17:14,573][71601] Updated weights for policy 0, policy_version 81370 (0.0007) [2023-10-11 22:17:15,674][71635] Updated weights for policy 1, policy_version 81282 (0.0008) [2023-10-11 22:17:16,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 166559744. Throughput: 0: 1809.9, 1: 1813.7. Samples: 41644520. Policy #0 lag: (min: 5.0, avg: 18.0, max: 37.0) [2023-10-11 22:17:16,034][70582] Avg episode reward: [(0, '74.320'), (1, '106.650')] [2023-10-11 22:17:16,043][71635] Updated weights for policy 1, policy_version 81292 (0.0009) [2023-10-11 22:17:16,413][71635] Updated weights for policy 1, policy_version 81302 (0.0009) [2023-10-11 22:17:16,778][71635] Updated weights for policy 1, policy_version 81312 (0.0008) [2023-10-11 22:17:18,297][71601] Updated weights for policy 0, policy_version 81380 (0.0007) [2023-10-11 22:17:18,662][71601] Updated weights for policy 0, policy_version 81390 (0.0009) [2023-10-11 22:17:19,044][71601] Updated weights for policy 0, policy_version 81400 (0.0007) [2023-10-11 22:17:20,552][71635] Updated weights for policy 1, policy_version 81322 (0.0007) [2023-10-11 22:17:20,918][71635] Updated weights for policy 1, policy_version 81332 (0.0009) [2023-10-11 22:17:21,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 166625280. Throughput: 0: 1803.8, 1: 1821.5. Samples: 41665896. Policy #0 lag: (min: 5.0, avg: 18.0, max: 37.0) [2023-10-11 22:17:21,034][70582] Avg episode reward: [(0, '73.790'), (1, '102.490')] [2023-10-11 22:17:21,284][71635] Updated weights for policy 1, policy_version 81342 (0.0008) [2023-10-11 22:17:22,814][71601] Updated weights for policy 0, policy_version 81410 (0.0008) [2023-10-11 22:17:23,189][71601] Updated weights for policy 0, policy_version 81420 (0.0007) [2023-10-11 22:17:23,562][71601] Updated weights for policy 0, policy_version 81430 (0.0007) [2023-10-11 22:17:23,930][71601] Updated weights for policy 0, policy_version 81440 (0.0010) [2023-10-11 22:17:25,067][71635] Updated weights for policy 1, policy_version 81352 (0.0008) [2023-10-11 22:17:25,433][71635] Updated weights for policy 1, policy_version 81362 (0.0007) [2023-10-11 22:17:25,804][71635] Updated weights for policy 1, policy_version 81372 (0.0007) [2023-10-11 22:17:26,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 166723584. Throughput: 0: 1809.6, 1: 1816.3. Samples: 41687824. Policy #0 lag: (min: 5.0, avg: 18.0, max: 37.0) [2023-10-11 22:17:26,034][70582] Avg episode reward: [(0, '75.790'), (1, '97.510')] [2023-10-11 22:17:27,665][71601] Updated weights for policy 0, policy_version 81450 (0.0010) [2023-10-11 22:17:28,041][71601] Updated weights for policy 0, policy_version 81460 (0.0008) [2023-10-11 22:17:28,422][71601] Updated weights for policy 0, policy_version 81470 (0.0008) [2023-10-11 22:17:29,435][71635] Updated weights for policy 1, policy_version 81382 (0.0011) [2023-10-11 22:17:29,797][71635] Updated weights for policy 1, policy_version 81392 (0.0010) [2023-10-11 22:17:30,163][71635] Updated weights for policy 1, policy_version 81402 (0.0009) [2023-10-11 22:17:31,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 166789120. Throughput: 0: 1815.1, 1: 1822.9. Samples: 41698752. Policy #0 lag: (min: 5.0, avg: 18.0, max: 37.0) [2023-10-11 22:17:31,035][70582] Avg episode reward: [(0, '77.150'), (1, '95.170')] [2023-10-11 22:17:31,939][71601] Updated weights for policy 0, policy_version 81480 (0.0010) [2023-10-11 22:17:32,319][71601] Updated weights for policy 0, policy_version 81490 (0.0010) [2023-10-11 22:17:32,690][71601] Updated weights for policy 0, policy_version 81500 (0.0010) [2023-10-11 22:17:33,705][71635] Updated weights for policy 1, policy_version 81412 (0.0010) [2023-10-11 22:17:34,080][71635] Updated weights for policy 1, policy_version 81422 (0.0008) [2023-10-11 22:17:34,448][71635] Updated weights for policy 1, policy_version 81432 (0.0007) [2023-10-11 22:17:36,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 166854656. Throughput: 0: 1813.5, 1: 1821.0. Samples: 41720600. Policy #0 lag: (min: 5.0, avg: 18.0, max: 37.0) [2023-10-11 22:17:36,035][70582] Avg episode reward: [(0, '77.160'), (1, '101.900')] [2023-10-11 22:17:36,394][71601] Updated weights for policy 0, policy_version 81510 (0.0007) [2023-10-11 22:17:36,755][71601] Updated weights for policy 0, policy_version 81520 (0.0009) [2023-10-11 22:17:37,130][71601] Updated weights for policy 0, policy_version 81530 (0.0009) [2023-10-11 22:17:38,288][71635] Updated weights for policy 1, policy_version 81442 (0.0007) [2023-10-11 22:17:38,704][71635] Updated weights for policy 1, policy_version 81452 (0.0009) [2023-10-11 22:17:39,076][71635] Updated weights for policy 1, policy_version 81462 (0.0010) [2023-10-11 22:17:39,430][71635] Updated weights for policy 1, policy_version 81472 (0.0010) [2023-10-11 22:17:41,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 166920192. Throughput: 0: 1816.1, 1: 1819.7. Samples: 41742632. Policy #0 lag: (min: 5.0, avg: 18.0, max: 37.0) [2023-10-11 22:17:41,035][70582] Avg episode reward: [(0, '75.780'), (1, '104.190')] [2023-10-11 22:17:41,060][71601] Updated weights for policy 0, policy_version 81540 (0.0010) [2023-10-11 22:17:41,441][71601] Updated weights for policy 0, policy_version 81550 (0.0007) [2023-10-11 22:17:41,815][71601] Updated weights for policy 0, policy_version 81560 (0.0009) [2023-10-11 22:17:43,187][71635] Updated weights for policy 1, policy_version 81482 (0.0007) [2023-10-11 22:17:43,557][71635] Updated weights for policy 1, policy_version 81492 (0.0007) [2023-10-11 22:17:43,928][71635] Updated weights for policy 1, policy_version 81502 (0.0008) [2023-10-11 22:17:45,600][71601] Updated weights for policy 0, policy_version 81570 (0.0009) [2023-10-11 22:17:45,960][71601] Updated weights for policy 0, policy_version 81580 (0.0008) [2023-10-11 22:17:46,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 166985728. Throughput: 0: 1808.9, 1: 1819.4. Samples: 41753068. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:17:46,034][70582] Avg episode reward: [(0, '76.480'), (1, '102.550')] [2023-10-11 22:17:46,334][71601] Updated weights for policy 0, policy_version 81590 (0.0008) [2023-10-11 22:17:46,706][71601] Updated weights for policy 0, policy_version 81600 (0.0008) [2023-10-11 22:17:47,629][71635] Updated weights for policy 1, policy_version 81512 (0.0010) [2023-10-11 22:17:47,990][71635] Updated weights for policy 1, policy_version 81522 (0.0010) [2023-10-11 22:17:48,364][71635] Updated weights for policy 1, policy_version 81532 (0.0008) [2023-10-11 22:17:50,399][71601] Updated weights for policy 0, policy_version 81610 (0.0010) [2023-10-11 22:17:50,779][71601] Updated weights for policy 0, policy_version 81620 (0.0008) [2023-10-11 22:17:51,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 167051264. Throughput: 0: 1804.5, 1: 1817.4. Samples: 41774790. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:17:51,034][70582] Avg episode reward: [(0, '78.030'), (1, '103.560')] [2023-10-11 22:17:51,147][71601] Updated weights for policy 0, policy_version 81630 (0.0011) [2023-10-11 22:17:52,040][71635] Updated weights for policy 1, policy_version 81542 (0.0007) [2023-10-11 22:17:52,397][71635] Updated weights for policy 1, policy_version 81552 (0.0009) [2023-10-11 22:17:52,765][71635] Updated weights for policy 1, policy_version 81562 (0.0010) [2023-10-11 22:17:54,781][71601] Updated weights for policy 0, policy_version 81640 (0.0009) [2023-10-11 22:17:55,156][71601] Updated weights for policy 0, policy_version 81650 (0.0010) [2023-10-11 22:17:55,530][71601] Updated weights for policy 0, policy_version 81660 (0.0008) [2023-10-11 22:17:56,034][70582] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 167149568. Throughput: 0: 1812.4, 1: 1821.4. Samples: 41796726. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:17:56,035][70582] Avg episode reward: [(0, '81.590'), (1, '103.220')] [2023-10-11 22:17:56,046][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000081664_83623936.pth... [2023-10-11 22:17:56,046][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000081568_83525632.pth... [2023-10-11 22:17:56,082][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000079968_81887232.pth [2023-10-11 22:17:56,083][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000079872_81788928.pth [2023-10-11 22:17:56,449][71635] Updated weights for policy 1, policy_version 81572 (0.0007) [2023-10-11 22:17:56,808][71635] Updated weights for policy 1, policy_version 81582 (0.0009) [2023-10-11 22:17:57,172][71635] Updated weights for policy 1, policy_version 81592 (0.0007) [2023-10-11 22:17:59,191][71601] Updated weights for policy 0, policy_version 81670 (0.0010) [2023-10-11 22:17:59,565][71601] Updated weights for policy 0, policy_version 81680 (0.0009) [2023-10-11 22:17:59,938][71601] Updated weights for policy 0, policy_version 81690 (0.0010) [2023-10-11 22:18:01,015][71635] Updated weights for policy 1, policy_version 81602 (0.0008) [2023-10-11 22:18:01,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 167215104. Throughput: 0: 1808.8, 1: 1824.0. Samples: 41807998. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:18:01,035][70582] Avg episode reward: [(0, '87.170'), (1, '104.330')] [2023-10-11 22:18:01,381][71635] Updated weights for policy 1, policy_version 81612 (0.0009) [2023-10-11 22:18:01,748][71635] Updated weights for policy 1, policy_version 81622 (0.0009) [2023-10-11 22:18:02,117][71635] Updated weights for policy 1, policy_version 81632 (0.0008) [2023-10-11 22:18:03,500][71601] Updated weights for policy 0, policy_version 81700 (0.0010) [2023-10-11 22:18:03,876][71601] Updated weights for policy 0, policy_version 81710 (0.0007) [2023-10-11 22:18:04,244][71601] Updated weights for policy 0, policy_version 81720 (0.0008) [2023-10-11 22:18:05,741][71635] Updated weights for policy 1, policy_version 81642 (0.0008) [2023-10-11 22:18:06,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 167280640. Throughput: 0: 1816.2, 1: 1814.9. Samples: 41829296. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:18:06,034][70582] Avg episode reward: [(0, '89.860'), (1, '106.710')] [2023-10-11 22:18:06,118][71635] Updated weights for policy 1, policy_version 81652 (0.0010) [2023-10-11 22:18:06,485][71635] Updated weights for policy 1, policy_version 81662 (0.0009) [2023-10-11 22:18:07,786][71601] Updated weights for policy 0, policy_version 81730 (0.0008) [2023-10-11 22:18:08,155][71601] Updated weights for policy 0, policy_version 81740 (0.0007) [2023-10-11 22:18:08,523][71601] Updated weights for policy 0, policy_version 81750 (0.0007) [2023-10-11 22:18:08,899][71601] Updated weights for policy 0, policy_version 81760 (0.0007) [2023-10-11 22:18:10,186][71635] Updated weights for policy 1, policy_version 81672 (0.0008) [2023-10-11 22:18:10,547][71635] Updated weights for policy 1, policy_version 81682 (0.0010) [2023-10-11 22:18:10,912][71635] Updated weights for policy 1, policy_version 81692 (0.0010) [2023-10-11 22:18:11,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 167346176. Throughput: 0: 1814.8, 1: 1825.9. Samples: 41851654. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:18:11,034][70582] Avg episode reward: [(0, '87.300'), (1, '106.890')] [2023-10-11 22:18:12,561][71601] Updated weights for policy 0, policy_version 81770 (0.0008) [2023-10-11 22:18:12,942][71601] Updated weights for policy 0, policy_version 81780 (0.0008) [2023-10-11 22:18:13,311][71601] Updated weights for policy 0, policy_version 81790 (0.0008) [2023-10-11 22:18:14,650][71635] Updated weights for policy 1, policy_version 81702 (0.0009) [2023-10-11 22:18:15,012][71635] Updated weights for policy 1, policy_version 81712 (0.0008) [2023-10-11 22:18:15,379][71635] Updated weights for policy 1, policy_version 81722 (0.0007) [2023-10-11 22:18:16,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 167444480. Throughput: 0: 1813.2, 1: 1818.2. Samples: 41862164. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:18:16,034][70582] Avg episode reward: [(0, '89.320'), (1, '105.210')] [2023-10-11 22:18:17,043][71601] Updated weights for policy 0, policy_version 81800 (0.0008) [2023-10-11 22:18:17,409][71601] Updated weights for policy 0, policy_version 81810 (0.0008) [2023-10-11 22:18:17,776][71601] Updated weights for policy 0, policy_version 81820 (0.0008) [2023-10-11 22:18:19,003][71635] Updated weights for policy 1, policy_version 81732 (0.0009) [2023-10-11 22:18:19,374][71635] Updated weights for policy 1, policy_version 81742 (0.0008) [2023-10-11 22:18:19,747][71635] Updated weights for policy 1, policy_version 81752 (0.0011) [2023-10-11 22:18:21,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 167510016. Throughput: 0: 1822.3, 1: 1818.8. Samples: 41884452. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:18:21,034][70582] Avg episode reward: [(0, '89.710'), (1, '105.230')] [2023-10-11 22:18:21,344][71601] Updated weights for policy 0, policy_version 81830 (0.0008) [2023-10-11 22:18:21,711][71601] Updated weights for policy 0, policy_version 81840 (0.0009) [2023-10-11 22:18:22,088][71601] Updated weights for policy 0, policy_version 81850 (0.0007) [2023-10-11 22:18:23,572][71635] Updated weights for policy 1, policy_version 81762 (0.0009) [2023-10-11 22:18:23,968][71635] Updated weights for policy 1, policy_version 81772 (0.0009) [2023-10-11 22:18:24,333][71635] Updated weights for policy 1, policy_version 81782 (0.0008) [2023-10-11 22:18:24,703][71635] Updated weights for policy 1, policy_version 81792 (0.0007) [2023-10-11 22:18:25,817][71601] Updated weights for policy 0, policy_version 81860 (0.0008) [2023-10-11 22:18:26,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 167575552. Throughput: 0: 1824.8, 1: 1814.9. Samples: 41906418. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:18:26,034][70582] Avg episode reward: [(0, '92.830'), (1, '105.050')] [2023-10-11 22:18:26,193][71601] Updated weights for policy 0, policy_version 81870 (0.0009) [2023-10-11 22:18:26,553][71601] Updated weights for policy 0, policy_version 81880 (0.0009) [2023-10-11 22:18:28,273][71635] Updated weights for policy 1, policy_version 81802 (0.0010) [2023-10-11 22:18:28,644][71635] Updated weights for policy 1, policy_version 81812 (0.0010) [2023-10-11 22:18:29,001][71635] Updated weights for policy 1, policy_version 81822 (0.0011) [2023-10-11 22:18:30,167][71601] Updated weights for policy 0, policy_version 81890 (0.0009) [2023-10-11 22:18:30,539][71601] Updated weights for policy 0, policy_version 81900 (0.0008) [2023-10-11 22:18:30,913][71601] Updated weights for policy 0, policy_version 81910 (0.0009) [2023-10-11 22:18:31,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 167641088. Throughput: 0: 1832.0, 1: 1821.9. Samples: 41917494. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:18:31,035][70582] Avg episode reward: [(0, '90.210'), (1, '108.000')] [2023-10-11 22:18:31,288][71601] Updated weights for policy 0, policy_version 81920 (0.0007) [2023-10-11 22:18:32,755][71635] Updated weights for policy 1, policy_version 81832 (0.0009) [2023-10-11 22:18:33,122][71635] Updated weights for policy 1, policy_version 81842 (0.0008) [2023-10-11 22:18:33,488][71635] Updated weights for policy 1, policy_version 81852 (0.0007) [2023-10-11 22:18:34,949][71601] Updated weights for policy 0, policy_version 81930 (0.0009) [2023-10-11 22:18:35,315][71601] Updated weights for policy 0, policy_version 81940 (0.0007) [2023-10-11 22:18:35,691][71601] Updated weights for policy 0, policy_version 81950 (0.0009) [2023-10-11 22:18:36,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 167739392. Throughput: 0: 1836.7, 1: 1816.0. Samples: 41939160. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-11 22:18:36,034][70582] Avg episode reward: [(0, '92.800'), (1, '107.900')] [2023-10-11 22:18:37,201][71635] Updated weights for policy 1, policy_version 81862 (0.0009) [2023-10-11 22:18:37,570][71635] Updated weights for policy 1, policy_version 81872 (0.0008) [2023-10-11 22:18:37,936][71635] Updated weights for policy 1, policy_version 81882 (0.0009) [2023-10-11 22:18:39,271][71601] Updated weights for policy 0, policy_version 81960 (0.0008) [2023-10-11 22:18:39,642][71601] Updated weights for policy 0, policy_version 81970 (0.0007) [2023-10-11 22:18:40,000][71601] Updated weights for policy 0, policy_version 81980 (0.0007) [2023-10-11 22:18:41,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 167804928. Throughput: 0: 1829.8, 1: 1818.0. Samples: 41960876. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-11 22:18:41,035][70582] Avg episode reward: [(0, '93.060'), (1, '106.050')] [2023-10-11 22:18:41,393][71635] Updated weights for policy 1, policy_version 81892 (0.0007) [2023-10-11 22:18:41,761][71635] Updated weights for policy 1, policy_version 81902 (0.0008) [2023-10-11 22:18:42,128][71635] Updated weights for policy 1, policy_version 81912 (0.0008) [2023-10-11 22:18:43,740][71601] Updated weights for policy 0, policy_version 81990 (0.0009) [2023-10-11 22:18:44,106][71601] Updated weights for policy 0, policy_version 82000 (0.0009) [2023-10-11 22:18:44,484][71601] Updated weights for policy 0, policy_version 82010 (0.0011) [2023-10-11 22:18:45,792][71635] Updated weights for policy 1, policy_version 81922 (0.0009) [2023-10-11 22:18:46,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 167870464. Throughput: 0: 1834.7, 1: 1819.3. Samples: 41972430. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-11 22:18:46,034][70582] Avg episode reward: [(0, '90.740'), (1, '105.080')] [2023-10-11 22:18:46,165][71635] Updated weights for policy 1, policy_version 81932 (0.0007) [2023-10-11 22:18:46,527][71635] Updated weights for policy 1, policy_version 81942 (0.0009) [2023-10-11 22:18:46,893][71635] Updated weights for policy 1, policy_version 81952 (0.0009) [2023-10-11 22:18:48,313][71601] Updated weights for policy 0, policy_version 82020 (0.0010) [2023-10-11 22:18:48,686][71601] Updated weights for policy 0, policy_version 82030 (0.0009) [2023-10-11 22:18:49,060][71601] Updated weights for policy 0, policy_version 82040 (0.0008) [2023-10-11 22:18:50,494][71635] Updated weights for policy 1, policy_version 81962 (0.0010) [2023-10-11 22:18:50,867][71635] Updated weights for policy 1, policy_version 81972 (0.0009) [2023-10-11 22:18:51,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 167936000. Throughput: 0: 1830.6, 1: 1826.3. Samples: 41993854. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-11 22:18:51,034][70582] Avg episode reward: [(0, '91.270'), (1, '107.940')] [2023-10-11 22:18:51,226][71635] Updated weights for policy 1, policy_version 81982 (0.0008) [2023-10-11 22:18:52,625][71601] Updated weights for policy 0, policy_version 82050 (0.0010) [2023-10-11 22:18:52,988][71601] Updated weights for policy 0, policy_version 82060 (0.0011) [2023-10-11 22:18:53,352][71601] Updated weights for policy 0, policy_version 82070 (0.0008) [2023-10-11 22:18:53,732][71601] Updated weights for policy 0, policy_version 82080 (0.0007) [2023-10-11 22:18:55,027][71635] Updated weights for policy 1, policy_version 81992 (0.0007) [2023-10-11 22:18:55,386][71635] Updated weights for policy 1, policy_version 82002 (0.0007) [2023-10-11 22:18:55,755][71635] Updated weights for policy 1, policy_version 82012 (0.0010) [2023-10-11 22:18:56,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 168034304. Throughput: 0: 1829.0, 1: 1819.3. Samples: 42015830. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-11 22:18:56,034][70582] Avg episode reward: [(0, '91.680'), (1, '107.540')] [2023-10-11 22:18:57,501][71601] Updated weights for policy 0, policy_version 82090 (0.0010) [2023-10-11 22:18:57,873][71601] Updated weights for policy 0, policy_version 82100 (0.0008) [2023-10-11 22:18:58,243][71601] Updated weights for policy 0, policy_version 82110 (0.0007) [2023-10-11 22:18:59,471][71635] Updated weights for policy 1, policy_version 82022 (0.0008) [2023-10-11 22:18:59,831][71635] Updated weights for policy 1, policy_version 82032 (0.0008) [2023-10-11 22:19:00,199][71635] Updated weights for policy 1, policy_version 82042 (0.0008) [2023-10-11 22:19:01,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 168099840. Throughput: 0: 1829.5, 1: 1824.7. Samples: 42026604. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-11 22:19:01,035][70582] Avg episode reward: [(0, '89.330'), (1, '102.800')] [2023-10-11 22:19:02,014][71601] Updated weights for policy 0, policy_version 82120 (0.0009) [2023-10-11 22:19:02,388][71601] Updated weights for policy 0, policy_version 82130 (0.0008) [2023-10-11 22:19:02,752][71601] Updated weights for policy 0, policy_version 82140 (0.0009) [2023-10-11 22:19:03,716][71635] Updated weights for policy 1, policy_version 82052 (0.0009) [2023-10-11 22:19:04,086][71635] Updated weights for policy 1, policy_version 82062 (0.0010) [2023-10-11 22:19:04,453][71635] Updated weights for policy 1, policy_version 82072 (0.0011) [2023-10-11 22:19:06,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 168165376. Throughput: 0: 1823.7, 1: 1822.3. Samples: 42048524. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-11 22:19:06,035][70582] Avg episode reward: [(0, '83.700'), (1, '109.150')] [2023-10-11 22:19:06,446][71601] Updated weights for policy 0, policy_version 82150 (0.0007) [2023-10-11 22:19:06,823][71601] Updated weights for policy 0, policy_version 82160 (0.0008) [2023-10-11 22:19:07,195][71601] Updated weights for policy 0, policy_version 82170 (0.0007) [2023-10-11 22:19:08,297][71635] Updated weights for policy 1, policy_version 82082 (0.0008) [2023-10-11 22:19:08,672][71635] Updated weights for policy 1, policy_version 82092 (0.0010) [2023-10-11 22:19:09,042][71635] Updated weights for policy 1, policy_version 82102 (0.0009) [2023-10-11 22:19:09,402][71635] Updated weights for policy 1, policy_version 82112 (0.0009) [2023-10-11 22:19:10,922][71601] Updated weights for policy 0, policy_version 82180 (0.0008) [2023-10-11 22:19:11,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 168230912. Throughput: 0: 1817.5, 1: 1828.4. Samples: 42070482. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-11 22:19:11,035][70582] Avg episode reward: [(0, '87.150'), (1, '110.590')] [2023-10-11 22:19:11,298][71601] Updated weights for policy 0, policy_version 82190 (0.0008) [2023-10-11 22:19:11,676][71601] Updated weights for policy 0, policy_version 82200 (0.0007) [2023-10-11 22:19:13,209][71635] Updated weights for policy 1, policy_version 82122 (0.0007) [2023-10-11 22:19:13,580][71635] Updated weights for policy 1, policy_version 82132 (0.0008) [2023-10-11 22:19:13,954][71635] Updated weights for policy 1, policy_version 82142 (0.0009) [2023-10-11 22:19:15,414][71601] Updated weights for policy 0, policy_version 82210 (0.0009) [2023-10-11 22:19:15,787][71601] Updated weights for policy 0, policy_version 82220 (0.0008) [2023-10-11 22:19:16,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 168296448. Throughput: 0: 1816.3, 1: 1821.1. Samples: 42081174. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-11 22:19:16,035][70582] Avg episode reward: [(0, '88.130'), (1, '106.530')] [2023-10-11 22:19:16,166][71601] Updated weights for policy 0, policy_version 82230 (0.0007) [2023-10-11 22:19:16,531][71601] Updated weights for policy 0, policy_version 82240 (0.0008) [2023-10-11 22:19:17,560][71635] Updated weights for policy 1, policy_version 82152 (0.0008) [2023-10-11 22:19:17,925][71635] Updated weights for policy 1, policy_version 82162 (0.0008) [2023-10-11 22:19:18,287][71635] Updated weights for policy 1, policy_version 82172 (0.0007) [2023-10-11 22:19:20,124][71601] Updated weights for policy 0, policy_version 82250 (0.0010) [2023-10-11 22:19:20,507][71601] Updated weights for policy 0, policy_version 82260 (0.0010) [2023-10-11 22:19:20,871][71601] Updated weights for policy 0, policy_version 82270 (0.0008) [2023-10-11 22:19:21,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 168394752. Throughput: 0: 1813.1, 1: 1831.0. Samples: 42103144. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-11 22:19:21,034][70582] Avg episode reward: [(0, '93.540'), (1, '108.450')] [2023-10-11 22:19:21,876][71635] Updated weights for policy 1, policy_version 82182 (0.0007) [2023-10-11 22:19:22,241][71635] Updated weights for policy 1, policy_version 82192 (0.0008) [2023-10-11 22:19:22,611][71635] Updated weights for policy 1, policy_version 82202 (0.0007) [2023-10-11 22:19:24,541][71601] Updated weights for policy 0, policy_version 82280 (0.0009) [2023-10-11 22:19:24,922][71601] Updated weights for policy 0, policy_version 82290 (0.0008) [2023-10-11 22:19:25,298][71601] Updated weights for policy 0, policy_version 82300 (0.0008) [2023-10-11 22:19:26,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 168460288. Throughput: 0: 1811.3, 1: 1829.6. Samples: 42124718. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-11 22:19:26,034][70582] Avg episode reward: [(0, '91.680'), (1, '111.940')] [2023-10-11 22:19:26,375][71635] Updated weights for policy 1, policy_version 82212 (0.0009) [2023-10-11 22:19:26,738][71635] Updated weights for policy 1, policy_version 82222 (0.0010) [2023-10-11 22:19:27,099][71635] Updated weights for policy 1, policy_version 82232 (0.0008) [2023-10-11 22:19:28,956][71601] Updated weights for policy 0, policy_version 82310 (0.0010) [2023-10-11 22:19:29,334][71601] Updated weights for policy 0, policy_version 82320 (0.0010) [2023-10-11 22:19:29,704][71601] Updated weights for policy 0, policy_version 82330 (0.0007) [2023-10-11 22:19:30,888][71635] Updated weights for policy 1, policy_version 82242 (0.0009) [2023-10-11 22:19:31,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 168525824. Throughput: 0: 1809.5, 1: 1826.1. Samples: 42136034. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-11 22:19:31,034][70582] Avg episode reward: [(0, '89.860'), (1, '108.920')] [2023-10-11 22:19:31,261][71635] Updated weights for policy 1, policy_version 82252 (0.0008) [2023-10-11 22:19:31,635][71635] Updated weights for policy 1, policy_version 82262 (0.0008) [2023-10-11 22:19:32,003][71635] Updated weights for policy 1, policy_version 82272 (0.0009) [2023-10-11 22:19:33,431][71601] Updated weights for policy 0, policy_version 82340 (0.0008) [2023-10-11 22:19:33,802][71601] Updated weights for policy 0, policy_version 82350 (0.0007) [2023-10-11 22:19:34,172][71601] Updated weights for policy 0, policy_version 82360 (0.0007) [2023-10-11 22:19:35,574][71635] Updated weights for policy 1, policy_version 82282 (0.0010) [2023-10-11 22:19:35,948][71635] Updated weights for policy 1, policy_version 82292 (0.0008) [2023-10-11 22:19:36,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 168591360. Throughput: 0: 1811.5, 1: 1818.7. Samples: 42157218. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-11 22:19:36,035][70582] Avg episode reward: [(0, '90.690'), (1, '113.180')] [2023-10-11 22:19:36,315][71635] Updated weights for policy 1, policy_version 82302 (0.0009) [2023-10-11 22:19:38,046][71601] Updated weights for policy 0, policy_version 82370 (0.0007) [2023-10-11 22:19:38,415][71601] Updated weights for policy 0, policy_version 82380 (0.0009) [2023-10-11 22:19:38,788][71601] Updated weights for policy 0, policy_version 82390 (0.0010) [2023-10-11 22:19:39,157][71601] Updated weights for policy 0, policy_version 82400 (0.0008) [2023-10-11 22:19:40,084][71635] Updated weights for policy 1, policy_version 82312 (0.0008) [2023-10-11 22:19:40,450][71635] Updated weights for policy 1, policy_version 82322 (0.0012) [2023-10-11 22:19:40,818][71635] Updated weights for policy 1, policy_version 82332 (0.0010) [2023-10-11 22:19:41,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 168689664. Throughput: 0: 1806.4, 1: 1821.0. Samples: 42179062. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-11 22:19:41,034][70582] Avg episode reward: [(0, '92.030'), (1, '114.380')] [2023-10-11 22:19:42,754][71601] Updated weights for policy 0, policy_version 82410 (0.0010) [2023-10-11 22:19:43,124][71601] Updated weights for policy 0, policy_version 82420 (0.0010) [2023-10-11 22:19:43,497][71601] Updated weights for policy 0, policy_version 82430 (0.0010) [2023-10-11 22:19:44,453][71635] Updated weights for policy 1, policy_version 82342 (0.0007) [2023-10-11 22:19:44,824][71635] Updated weights for policy 1, policy_version 82352 (0.0008) [2023-10-11 22:19:45,198][71635] Updated weights for policy 1, policy_version 82362 (0.0009) [2023-10-11 22:19:46,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 168755200. Throughput: 0: 1810.9, 1: 1819.1. Samples: 42189950. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-11 22:19:46,034][70582] Avg episode reward: [(0, '95.990'), (1, '112.410')] [2023-10-11 22:19:47,160][71601] Updated weights for policy 0, policy_version 82440 (0.0008) [2023-10-11 22:19:47,532][71601] Updated weights for policy 0, policy_version 82450 (0.0007) [2023-10-11 22:19:47,896][71601] Updated weights for policy 0, policy_version 82460 (0.0009) [2023-10-11 22:19:48,785][71635] Updated weights for policy 1, policy_version 82372 (0.0007) [2023-10-11 22:19:49,151][71635] Updated weights for policy 1, policy_version 82382 (0.0008) [2023-10-11 22:19:49,515][71635] Updated weights for policy 1, policy_version 82392 (0.0010) [2023-10-11 22:19:51,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 168820736. Throughput: 0: 1808.5, 1: 1821.1. Samples: 42211856. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-11 22:19:51,035][70582] Avg episode reward: [(0, '96.640'), (1, '109.420')] [2023-10-11 22:19:51,639][71601] Updated weights for policy 0, policy_version 82470 (0.0007) [2023-10-11 22:19:52,023][71601] Updated weights for policy 0, policy_version 82480 (0.0007) [2023-10-11 22:19:52,406][71601] Updated weights for policy 0, policy_version 82490 (0.0007) [2023-10-11 22:19:53,100][71635] Updated weights for policy 1, policy_version 82402 (0.0009) [2023-10-11 22:19:53,471][71635] Updated weights for policy 1, policy_version 82412 (0.0007) [2023-10-11 22:19:53,847][71635] Updated weights for policy 1, policy_version 82422 (0.0008) [2023-10-11 22:19:54,221][71635] Updated weights for policy 1, policy_version 82432 (0.0009) [2023-10-11 22:19:56,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 168886272. Throughput: 0: 1814.0, 1: 1824.4. Samples: 42234206. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-11 22:19:56,034][70582] Avg episode reward: [(0, '94.200'), (1, '115.110')] [2023-10-11 22:19:56,044][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000082432_84410368.pth... [2023-10-11 22:19:56,078][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000080736_82673664.pth [2023-10-11 22:19:56,080][71601] Updated weights for policy 0, policy_version 82500 (0.0008) [2023-10-11 22:19:56,444][71601] Updated weights for policy 0, policy_version 82510 (0.0008) [2023-10-11 22:19:56,817][71601] Updated weights for policy 0, policy_version 82520 (0.0008) [2023-10-11 22:19:57,107][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000082528_84508672.pth... [2023-10-11 22:19:57,146][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000080800_82739200.pth [2023-10-11 22:19:57,967][71635] Updated weights for policy 1, policy_version 82442 (0.0007) [2023-10-11 22:19:58,325][71635] Updated weights for policy 1, policy_version 82452 (0.0007) [2023-10-11 22:19:58,690][71635] Updated weights for policy 1, policy_version 82462 (0.0010) [2023-10-11 22:20:00,565][71601] Updated weights for policy 0, policy_version 82530 (0.0008) [2023-10-11 22:20:00,939][71601] Updated weights for policy 0, policy_version 82540 (0.0009) [2023-10-11 22:20:01,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 168951808. Throughput: 0: 1810.2, 1: 1820.1. Samples: 42244536. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-11 22:20:01,034][70582] Avg episode reward: [(0, '99.820'), (1, '112.520')] [2023-10-11 22:20:01,303][71601] Updated weights for policy 0, policy_version 82550 (0.0007) [2023-10-11 22:20:01,677][71601] Updated weights for policy 0, policy_version 82560 (0.0008) [2023-10-11 22:20:02,485][71635] Updated weights for policy 1, policy_version 82472 (0.0009) [2023-10-11 22:20:02,843][71635] Updated weights for policy 1, policy_version 82482 (0.0008) [2023-10-11 22:20:03,202][71635] Updated weights for policy 1, policy_version 82492 (0.0008) [2023-10-11 22:20:05,218][71601] Updated weights for policy 0, policy_version 82570 (0.0008) [2023-10-11 22:20:05,591][71601] Updated weights for policy 0, policy_version 82580 (0.0007) [2023-10-11 22:20:05,968][71601] Updated weights for policy 0, policy_version 82590 (0.0008) [2023-10-11 22:20:06,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 169017344. Throughput: 0: 1814.3, 1: 1821.8. Samples: 42266768. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-11 22:20:06,034][70582] Avg episode reward: [(0, '95.180'), (1, '110.500')] [2023-10-11 22:20:06,898][71635] Updated weights for policy 1, policy_version 82502 (0.0009) [2023-10-11 22:20:07,270][71635] Updated weights for policy 1, policy_version 82512 (0.0009) [2023-10-11 22:20:07,646][71635] Updated weights for policy 1, policy_version 82522 (0.0008) [2023-10-11 22:20:09,768][71601] Updated weights for policy 0, policy_version 82600 (0.0010) [2023-10-11 22:20:10,142][71601] Updated weights for policy 0, policy_version 82610 (0.0008) [2023-10-11 22:20:10,504][71601] Updated weights for policy 0, policy_version 82620 (0.0009) [2023-10-11 22:20:11,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 169115648. Throughput: 0: 1817.6, 1: 1817.2. Samples: 42288286. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-11 22:20:11,034][70582] Avg episode reward: [(0, '88.060'), (1, '108.810')] [2023-10-11 22:20:11,411][71635] Updated weights for policy 1, policy_version 82532 (0.0008) [2023-10-11 22:20:11,788][71635] Updated weights for policy 1, policy_version 82542 (0.0009) [2023-10-11 22:20:12,145][71635] Updated weights for policy 1, policy_version 82552 (0.0008) [2023-10-11 22:20:14,322][71601] Updated weights for policy 0, policy_version 82630 (0.0010) [2023-10-11 22:20:14,697][71601] Updated weights for policy 0, policy_version 82640 (0.0008) [2023-10-11 22:20:15,067][71601] Updated weights for policy 0, policy_version 82650 (0.0007) [2023-10-11 22:20:15,718][71635] Updated weights for policy 1, policy_version 82562 (0.0007) [2023-10-11 22:20:16,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 169181184. Throughput: 0: 1810.1, 1: 1822.2. Samples: 42299488. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-11 22:20:16,034][70582] Avg episode reward: [(0, '87.560'), (1, '110.610')] [2023-10-11 22:20:16,087][71635] Updated weights for policy 1, policy_version 82572 (0.0008) [2023-10-11 22:20:16,457][71635] Updated weights for policy 1, policy_version 82582 (0.0008) [2023-10-11 22:20:16,819][71635] Updated weights for policy 1, policy_version 82592 (0.0009) [2023-10-11 22:20:18,694][71601] Updated weights for policy 0, policy_version 82660 (0.0008) [2023-10-11 22:20:19,058][71601] Updated weights for policy 0, policy_version 82670 (0.0009) [2023-10-11 22:20:19,430][71601] Updated weights for policy 0, policy_version 82680 (0.0009) [2023-10-11 22:20:20,488][71635] Updated weights for policy 1, policy_version 82602 (0.0009) [2023-10-11 22:20:20,859][71635] Updated weights for policy 1, policy_version 82612 (0.0010) [2023-10-11 22:20:21,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 169246720. Throughput: 0: 1819.6, 1: 1823.9. Samples: 42321172. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-11 22:20:21,035][70582] Avg episode reward: [(0, '93.410'), (1, '109.630')] [2023-10-11 22:20:21,227][71635] Updated weights for policy 1, policy_version 82622 (0.0009) [2023-10-11 22:20:22,998][71601] Updated weights for policy 0, policy_version 82690 (0.0007) [2023-10-11 22:20:23,369][71601] Updated weights for policy 0, policy_version 82700 (0.0007) [2023-10-11 22:20:23,741][71601] Updated weights for policy 0, policy_version 82710 (0.0008) [2023-10-11 22:20:24,108][71601] Updated weights for policy 0, policy_version 82720 (0.0009) [2023-10-11 22:20:24,917][71635] Updated weights for policy 1, policy_version 82632 (0.0010) [2023-10-11 22:20:25,280][71635] Updated weights for policy 1, policy_version 82642 (0.0008) [2023-10-11 22:20:25,647][71635] Updated weights for policy 1, policy_version 82652 (0.0009) [2023-10-11 22:20:26,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 169345024. Throughput: 0: 1822.1, 1: 1820.2. Samples: 42342966. Policy #0 lag: (min: 15.0, avg: 30.5, max: 32.0) [2023-10-11 22:20:26,034][70582] Avg episode reward: [(0, '92.470'), (1, '108.340')] [2023-10-11 22:20:27,865][71601] Updated weights for policy 0, policy_version 82730 (0.0007) [2023-10-11 22:20:28,232][71601] Updated weights for policy 0, policy_version 82740 (0.0010) [2023-10-11 22:20:28,617][71601] Updated weights for policy 0, policy_version 82750 (0.0010) [2023-10-11 22:20:29,391][71635] Updated weights for policy 1, policy_version 82662 (0.0008) [2023-10-11 22:20:29,764][71635] Updated weights for policy 1, policy_version 82672 (0.0008) [2023-10-11 22:20:30,128][71635] Updated weights for policy 1, policy_version 82682 (0.0008) [2023-10-11 22:20:31,034][70582] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 169410560. Throughput: 0: 1822.7, 1: 1826.4. Samples: 42354158. Policy #0 lag: (min: 15.0, avg: 30.5, max: 32.0) [2023-10-11 22:20:31,034][70582] Avg episode reward: [(0, '91.940'), (1, '105.580')] [2023-10-11 22:20:32,331][71601] Updated weights for policy 0, policy_version 82760 (0.0010) [2023-10-11 22:20:32,707][71601] Updated weights for policy 0, policy_version 82770 (0.0010) [2023-10-11 22:20:33,084][71601] Updated weights for policy 0, policy_version 82780 (0.0010) [2023-10-11 22:20:33,885][71635] Updated weights for policy 1, policy_version 82692 (0.0009) [2023-10-11 22:20:34,245][71635] Updated weights for policy 1, policy_version 82702 (0.0010) [2023-10-11 22:20:34,609][71635] Updated weights for policy 1, policy_version 82712 (0.0008) [2023-10-11 22:20:36,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 169476096. Throughput: 0: 1820.2, 1: 1824.8. Samples: 42375882. Policy #0 lag: (min: 15.0, avg: 30.5, max: 32.0) [2023-10-11 22:20:36,035][70582] Avg episode reward: [(0, '93.420'), (1, '108.770')] [2023-10-11 22:20:36,815][71601] Updated weights for policy 0, policy_version 82790 (0.0009) [2023-10-11 22:20:37,198][71601] Updated weights for policy 0, policy_version 82800 (0.0008) [2023-10-11 22:20:37,567][71601] Updated weights for policy 0, policy_version 82810 (0.0008) [2023-10-11 22:20:38,118][71635] Updated weights for policy 1, policy_version 82722 (0.0008) [2023-10-11 22:20:38,477][71635] Updated weights for policy 1, policy_version 82732 (0.0010) [2023-10-11 22:20:38,838][71635] Updated weights for policy 1, policy_version 82742 (0.0010) [2023-10-11 22:20:39,202][71635] Updated weights for policy 1, policy_version 82752 (0.0010) [2023-10-11 22:20:41,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 169541632. Throughput: 0: 1813.6, 1: 1825.3. Samples: 42397954. Policy #0 lag: (min: 15.0, avg: 30.5, max: 32.0) [2023-10-11 22:20:41,034][70582] Avg episode reward: [(0, '96.550'), (1, '102.840')] [2023-10-11 22:20:41,193][71601] Updated weights for policy 0, policy_version 82820 (0.0008) [2023-10-11 22:20:41,575][71601] Updated weights for policy 0, policy_version 82830 (0.0011) [2023-10-11 22:20:41,943][71601] Updated weights for policy 0, policy_version 82840 (0.0010) [2023-10-11 22:20:43,113][71635] Updated weights for policy 1, policy_version 82762 (0.0009) [2023-10-11 22:20:43,502][71635] Updated weights for policy 1, policy_version 82772 (0.0009) [2023-10-11 22:20:43,866][71635] Updated weights for policy 1, policy_version 82782 (0.0010) [2023-10-11 22:20:45,835][71601] Updated weights for policy 0, policy_version 82850 (0.0009) [2023-10-11 22:20:46,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 169607168. Throughput: 0: 1816.2, 1: 1823.0. Samples: 42408302. Policy #0 lag: (min: 15.0, avg: 30.5, max: 32.0) [2023-10-11 22:20:46,034][70582] Avg episode reward: [(0, '96.360'), (1, '100.580')] [2023-10-11 22:20:46,204][71601] Updated weights for policy 0, policy_version 82860 (0.0007) [2023-10-11 22:20:46,585][71601] Updated weights for policy 0, policy_version 82870 (0.0007) [2023-10-11 22:20:46,953][71601] Updated weights for policy 0, policy_version 82880 (0.0008) [2023-10-11 22:20:47,504][71635] Updated weights for policy 1, policy_version 82792 (0.0008) [2023-10-11 22:20:47,870][71635] Updated weights for policy 1, policy_version 82802 (0.0008) [2023-10-11 22:20:48,237][71635] Updated weights for policy 1, policy_version 82812 (0.0011) [2023-10-11 22:20:50,636][71601] Updated weights for policy 0, policy_version 82890 (0.0009) [2023-10-11 22:20:51,006][71601] Updated weights for policy 0, policy_version 82900 (0.0010) [2023-10-11 22:20:51,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 169672704. Throughput: 0: 1805.0, 1: 1822.8. Samples: 42430016. Policy #0 lag: (min: 15.0, avg: 30.5, max: 32.0) [2023-10-11 22:20:51,034][70582] Avg episode reward: [(0, '89.330'), (1, '99.720')] [2023-10-11 22:20:51,376][71601] Updated weights for policy 0, policy_version 82910 (0.0008) [2023-10-11 22:20:51,853][71635] Updated weights for policy 1, policy_version 82822 (0.0008) [2023-10-11 22:20:52,217][71635] Updated weights for policy 1, policy_version 82832 (0.0007) [2023-10-11 22:20:52,589][71635] Updated weights for policy 1, policy_version 82842 (0.0009) [2023-10-11 22:20:55,035][71601] Updated weights for policy 0, policy_version 82920 (0.0007) [2023-10-11 22:20:55,410][71601] Updated weights for policy 0, policy_version 82930 (0.0008) [2023-10-11 22:20:55,784][71601] Updated weights for policy 0, policy_version 82940 (0.0007) [2023-10-11 22:20:56,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 169771008. Throughput: 0: 1817.4, 1: 1825.2. Samples: 42452202. Policy #0 lag: (min: 15.0, avg: 30.5, max: 32.0) [2023-10-11 22:20:56,034][70582] Avg episode reward: [(0, '91.260'), (1, '98.200')] [2023-10-11 22:20:56,241][71635] Updated weights for policy 1, policy_version 82852 (0.0009) [2023-10-11 22:20:56,608][71635] Updated weights for policy 1, policy_version 82862 (0.0008) [2023-10-11 22:20:56,982][71635] Updated weights for policy 1, policy_version 82872 (0.0008) [2023-10-11 22:20:59,495][71601] Updated weights for policy 0, policy_version 82950 (0.0009) [2023-10-11 22:20:59,872][71601] Updated weights for policy 0, policy_version 82960 (0.0007) [2023-10-11 22:21:00,242][71601] Updated weights for policy 0, policy_version 82970 (0.0009) [2023-10-11 22:21:00,654][71635] Updated weights for policy 1, policy_version 82882 (0.0009) [2023-10-11 22:21:01,025][71635] Updated weights for policy 1, policy_version 82892 (0.0007) [2023-10-11 22:21:01,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 169836544. Throughput: 0: 1809.0, 1: 1823.1. Samples: 42462932. Policy #0 lag: (min: 15.0, avg: 30.5, max: 32.0) [2023-10-11 22:21:01,034][70582] Avg episode reward: [(0, '92.890'), (1, '100.660')] [2023-10-11 22:21:01,387][71635] Updated weights for policy 1, policy_version 82902 (0.0008) [2023-10-11 22:21:01,751][71635] Updated weights for policy 1, policy_version 82912 (0.0007) [2023-10-11 22:21:03,932][71601] Updated weights for policy 0, policy_version 82980 (0.0010) [2023-10-11 22:21:04,305][71601] Updated weights for policy 0, policy_version 82990 (0.0010) [2023-10-11 22:21:04,675][71601] Updated weights for policy 0, policy_version 83000 (0.0007) [2023-10-11 22:21:05,330][71635] Updated weights for policy 1, policy_version 82922 (0.0009) [2023-10-11 22:21:05,700][71635] Updated weights for policy 1, policy_version 82932 (0.0008) [2023-10-11 22:21:06,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 169902080. Throughput: 0: 1817.0, 1: 1826.5. Samples: 42485128. Policy #0 lag: (min: 15.0, avg: 30.5, max: 32.0) [2023-10-11 22:21:06,034][70582] Avg episode reward: [(0, '101.680'), (1, '98.290')] [2023-10-11 22:21:06,062][71635] Updated weights for policy 1, policy_version 82942 (0.0008) [2023-10-11 22:21:08,245][71601] Updated weights for policy 0, policy_version 83010 (0.0007) [2023-10-11 22:21:08,615][71601] Updated weights for policy 0, policy_version 83020 (0.0007) [2023-10-11 22:21:08,982][71601] Updated weights for policy 0, policy_version 83030 (0.0010) [2023-10-11 22:21:09,359][71601] Updated weights for policy 0, policy_version 83040 (0.0007) [2023-10-11 22:21:09,843][71635] Updated weights for policy 1, policy_version 82952 (0.0007) [2023-10-11 22:21:10,210][71635] Updated weights for policy 1, policy_version 82962 (0.0008) [2023-10-11 22:21:10,574][71635] Updated weights for policy 1, policy_version 82972 (0.0007) [2023-10-11 22:21:11,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 170000384. Throughput: 0: 1809.2, 1: 1823.1. Samples: 42506420. Policy #0 lag: (min: 15.0, avg: 30.5, max: 32.0) [2023-10-11 22:21:11,034][70582] Avg episode reward: [(0, '98.740'), (1, '98.410')] [2023-10-11 22:21:12,995][71601] Updated weights for policy 0, policy_version 83050 (0.0008) [2023-10-11 22:21:13,359][71601] Updated weights for policy 0, policy_version 83060 (0.0008) [2023-10-11 22:21:13,738][71601] Updated weights for policy 0, policy_version 83070 (0.0009) [2023-10-11 22:21:14,123][71635] Updated weights for policy 1, policy_version 82982 (0.0008) [2023-10-11 22:21:14,499][71635] Updated weights for policy 1, policy_version 82992 (0.0008) [2023-10-11 22:21:14,857][71635] Updated weights for policy 1, policy_version 83002 (0.0008) [2023-10-11 22:21:16,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 170065920. Throughput: 0: 1813.3, 1: 1826.1. Samples: 42517932. Policy #0 lag: (min: 15.0, avg: 30.5, max: 32.0) [2023-10-11 22:21:16,034][70582] Avg episode reward: [(0, '96.170'), (1, '98.130')] [2023-10-11 22:21:17,426][71601] Updated weights for policy 0, policy_version 83080 (0.0009) [2023-10-11 22:21:17,798][71601] Updated weights for policy 0, policy_version 83090 (0.0010) [2023-10-11 22:21:18,176][71601] Updated weights for policy 0, policy_version 83100 (0.0009) [2023-10-11 22:21:18,498][71635] Updated weights for policy 1, policy_version 83012 (0.0007) [2023-10-11 22:21:18,869][71635] Updated weights for policy 1, policy_version 83022 (0.0008) [2023-10-11 22:21:19,237][71635] Updated weights for policy 1, policy_version 83032 (0.0008) [2023-10-11 22:21:21,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 170131456. Throughput: 0: 1812.7, 1: 1818.7. Samples: 42539292. Policy #0 lag: (min: 31.0, avg: 31.2, max: 39.0) [2023-10-11 22:21:21,035][70582] Avg episode reward: [(0, '92.710'), (1, '91.340')] [2023-10-11 22:21:22,001][71601] Updated weights for policy 0, policy_version 83110 (0.0009) [2023-10-11 22:21:22,380][71601] Updated weights for policy 0, policy_version 83120 (0.0009) [2023-10-11 22:21:22,741][71601] Updated weights for policy 0, policy_version 83130 (0.0010) [2023-10-11 22:21:22,987][71635] Updated weights for policy 1, policy_version 83042 (0.0008) [2023-10-11 22:21:23,361][71635] Updated weights for policy 1, policy_version 83052 (0.0007) [2023-10-11 22:21:23,734][71635] Updated weights for policy 1, policy_version 83062 (0.0008) [2023-10-11 22:21:24,093][71635] Updated weights for policy 1, policy_version 83072 (0.0009) [2023-10-11 22:21:26,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 170196992. Throughput: 0: 1817.6, 1: 1827.9. Samples: 42562000. Policy #0 lag: (min: 31.0, avg: 31.2, max: 39.0) [2023-10-11 22:21:26,034][70582] Avg episode reward: [(0, '87.550'), (1, '92.510')] [2023-10-11 22:21:26,393][71601] Updated weights for policy 0, policy_version 83140 (0.0010) [2023-10-11 22:21:26,763][71601] Updated weights for policy 0, policy_version 83150 (0.0009) [2023-10-11 22:21:27,147][71601] Updated weights for policy 0, policy_version 83160 (0.0009) [2023-10-11 22:21:27,811][71635] Updated weights for policy 1, policy_version 83082 (0.0009) [2023-10-11 22:21:28,173][71635] Updated weights for policy 1, policy_version 83092 (0.0009) [2023-10-11 22:21:28,548][71635] Updated weights for policy 1, policy_version 83102 (0.0009) [2023-10-11 22:21:30,844][71601] Updated weights for policy 0, policy_version 83170 (0.0009) [2023-10-11 22:21:31,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 170262528. Throughput: 0: 1820.8, 1: 1825.0. Samples: 42572360. Policy #0 lag: (min: 31.0, avg: 31.2, max: 39.0) [2023-10-11 22:21:31,035][70582] Avg episode reward: [(0, '85.410'), (1, '98.030')] [2023-10-11 22:21:31,217][71601] Updated weights for policy 0, policy_version 83180 (0.0009) [2023-10-11 22:21:31,596][71601] Updated weights for policy 0, policy_version 83190 (0.0009) [2023-10-11 22:21:31,962][71601] Updated weights for policy 0, policy_version 83200 (0.0008) [2023-10-11 22:21:32,258][71635] Updated weights for policy 1, policy_version 83112 (0.0009) [2023-10-11 22:21:32,634][71635] Updated weights for policy 1, policy_version 83122 (0.0007) [2023-10-11 22:21:32,991][71635] Updated weights for policy 1, policy_version 83132 (0.0008) [2023-10-11 22:21:35,374][71601] Updated weights for policy 0, policy_version 83210 (0.0008) [2023-10-11 22:21:35,746][71601] Updated weights for policy 0, policy_version 83220 (0.0008) [2023-10-11 22:21:36,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 170328064. Throughput: 0: 1833.2, 1: 1831.1. Samples: 42594908. Policy #0 lag: (min: 31.0, avg: 31.2, max: 39.0) [2023-10-11 22:21:36,034][70582] Avg episode reward: [(0, '90.620'), (1, '99.180')] [2023-10-11 22:21:36,123][71601] Updated weights for policy 0, policy_version 83230 (0.0008) [2023-10-11 22:21:36,601][71635] Updated weights for policy 1, policy_version 83142 (0.0009) [2023-10-11 22:21:36,976][71635] Updated weights for policy 1, policy_version 83152 (0.0007) [2023-10-11 22:21:37,345][71635] Updated weights for policy 1, policy_version 83162 (0.0008) [2023-10-11 22:21:39,753][71601] Updated weights for policy 0, policy_version 83240 (0.0008) [2023-10-11 22:21:40,132][71601] Updated weights for policy 0, policy_version 83250 (0.0007) [2023-10-11 22:21:40,497][71601] Updated weights for policy 0, policy_version 83260 (0.0007) [2023-10-11 22:21:40,970][71635] Updated weights for policy 1, policy_version 83172 (0.0009) [2023-10-11 22:21:41,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 170426368. Throughput: 0: 1826.8, 1: 1832.0. Samples: 42616846. Policy #0 lag: (min: 31.0, avg: 31.2, max: 39.0) [2023-10-11 22:21:41,034][70582] Avg episode reward: [(0, '91.000'), (1, '104.260')] [2023-10-11 22:21:41,327][71635] Updated weights for policy 1, policy_version 83182 (0.0010) [2023-10-11 22:21:41,701][71635] Updated weights for policy 1, policy_version 83192 (0.0009) [2023-10-11 22:21:44,339][71601] Updated weights for policy 0, policy_version 83270 (0.0010) [2023-10-11 22:21:44,729][71601] Updated weights for policy 0, policy_version 83280 (0.0009) [2023-10-11 22:21:45,106][71601] Updated weights for policy 0, policy_version 83290 (0.0009) [2023-10-11 22:21:45,425][71635] Updated weights for policy 1, policy_version 83202 (0.0009) [2023-10-11 22:21:45,787][71635] Updated weights for policy 1, policy_version 83212 (0.0009) [2023-10-11 22:21:46,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 170491904. Throughput: 0: 1831.6, 1: 1829.7. Samples: 42627690. Policy #0 lag: (min: 31.0, avg: 31.2, max: 39.0) [2023-10-11 22:21:46,035][70582] Avg episode reward: [(0, '89.160'), (1, '103.370')] [2023-10-11 22:21:46,147][71635] Updated weights for policy 1, policy_version 83222 (0.0008) [2023-10-11 22:21:46,515][71635] Updated weights for policy 1, policy_version 83232 (0.0009) [2023-10-11 22:21:48,660][71601] Updated weights for policy 0, policy_version 83300 (0.0008) [2023-10-11 22:21:49,034][71601] Updated weights for policy 0, policy_version 83310 (0.0008) [2023-10-11 22:21:49,407][71601] Updated weights for policy 0, policy_version 83320 (0.0011) [2023-10-11 22:21:50,384][71635] Updated weights for policy 1, policy_version 83242 (0.0008) [2023-10-11 22:21:50,751][71635] Updated weights for policy 1, policy_version 83252 (0.0008) [2023-10-11 22:21:51,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 170557440. Throughput: 0: 1822.4, 1: 1821.1. Samples: 42649086. Policy #0 lag: (min: 31.0, avg: 31.2, max: 39.0) [2023-10-11 22:21:51,034][70582] Avg episode reward: [(0, '82.430'), (1, '103.690')] [2023-10-11 22:21:51,121][71635] Updated weights for policy 1, policy_version 83262 (0.0009) [2023-10-11 22:21:53,119][71601] Updated weights for policy 0, policy_version 83330 (0.0009) [2023-10-11 22:21:53,497][71601] Updated weights for policy 0, policy_version 83340 (0.0011) [2023-10-11 22:21:53,865][71601] Updated weights for policy 0, policy_version 83350 (0.0008) [2023-10-11 22:21:54,226][71601] Updated weights for policy 0, policy_version 83360 (0.0009) [2023-10-11 22:21:54,891][71635] Updated weights for policy 1, policy_version 83272 (0.0009) [2023-10-11 22:21:55,255][71635] Updated weights for policy 1, policy_version 83282 (0.0007) [2023-10-11 22:21:55,624][71635] Updated weights for policy 1, policy_version 83292 (0.0008) [2023-10-11 22:21:56,034][70582] Fps is (10 sec: 16383.4, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 170655744. Throughput: 0: 1826.7, 1: 1818.8. Samples: 42670470. Policy #0 lag: (min: 31.0, avg: 31.2, max: 39.0) [2023-10-11 22:21:56,035][70582] Avg episode reward: [(0, '80.150'), (1, '101.960')] [2023-10-11 22:21:56,045][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000083296_85295104.pth... [2023-10-11 22:21:56,045][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000083360_85360640.pth... [2023-10-11 22:21:56,077][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000081568_83525632.pth [2023-10-11 22:21:56,083][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000081664_83623936.pth [2023-10-11 22:21:57,999][71601] Updated weights for policy 0, policy_version 83370 (0.0008) [2023-10-11 22:21:58,372][71601] Updated weights for policy 0, policy_version 83380 (0.0009) [2023-10-11 22:21:58,757][71601] Updated weights for policy 0, policy_version 83390 (0.0009) [2023-10-11 22:21:59,399][71635] Updated weights for policy 1, policy_version 83302 (0.0010) [2023-10-11 22:21:59,763][71635] Updated weights for policy 1, policy_version 83312 (0.0008) [2023-10-11 22:22:00,139][71635] Updated weights for policy 1, policy_version 83322 (0.0007) [2023-10-11 22:22:01,034][70582] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 170721280. Throughput: 0: 1824.3, 1: 1813.0. Samples: 42681612. Policy #0 lag: (min: 31.0, avg: 31.2, max: 39.0) [2023-10-11 22:22:01,035][70582] Avg episode reward: [(0, '83.420'), (1, '102.370')] [2023-10-11 22:22:02,361][71601] Updated weights for policy 0, policy_version 83400 (0.0009) [2023-10-11 22:22:02,746][71601] Updated weights for policy 0, policy_version 83410 (0.0010) [2023-10-11 22:22:03,116][71601] Updated weights for policy 0, policy_version 83420 (0.0007) [2023-10-11 22:22:03,774][71635] Updated weights for policy 1, policy_version 83332 (0.0009) [2023-10-11 22:22:04,146][71635] Updated weights for policy 1, policy_version 83342 (0.0011) [2023-10-11 22:22:04,514][71635] Updated weights for policy 1, policy_version 83352 (0.0009) [2023-10-11 22:22:06,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 170786816. Throughput: 0: 1818.8, 1: 1818.6. Samples: 42702978. Policy #0 lag: (min: 31.0, avg: 31.2, max: 39.0) [2023-10-11 22:22:06,035][70582] Avg episode reward: [(0, '87.490'), (1, '99.100')] [2023-10-11 22:22:06,827][71601] Updated weights for policy 0, policy_version 83430 (0.0007) [2023-10-11 22:22:07,200][71601] Updated weights for policy 0, policy_version 83440 (0.0007) [2023-10-11 22:22:07,569][71601] Updated weights for policy 0, policy_version 83450 (0.0007) [2023-10-11 22:22:08,149][71635] Updated weights for policy 1, policy_version 83362 (0.0009) [2023-10-11 22:22:08,511][71635] Updated weights for policy 1, policy_version 83372 (0.0008) [2023-10-11 22:22:08,882][71635] Updated weights for policy 1, policy_version 83382 (0.0010) [2023-10-11 22:22:09,248][71635] Updated weights for policy 1, policy_version 83392 (0.0010) [2023-10-11 22:22:11,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 170852352. Throughput: 0: 1816.6, 1: 1810.8. Samples: 42725232. Policy #0 lag: (min: 31.0, avg: 31.2, max: 39.0) [2023-10-11 22:22:11,034][70582] Avg episode reward: [(0, '92.180'), (1, '100.910')] [2023-10-11 22:22:11,263][71601] Updated weights for policy 0, policy_version 83460 (0.0009) [2023-10-11 22:22:11,637][71601] Updated weights for policy 0, policy_version 83470 (0.0007) [2023-10-11 22:22:12,006][71601] Updated weights for policy 0, policy_version 83480 (0.0009) [2023-10-11 22:22:13,069][71635] Updated weights for policy 1, policy_version 83402 (0.0011) [2023-10-11 22:22:13,439][71635] Updated weights for policy 1, policy_version 83412 (0.0011) [2023-10-11 22:22:13,817][71635] Updated weights for policy 1, policy_version 83422 (0.0008) [2023-10-11 22:22:15,654][71601] Updated weights for policy 0, policy_version 83490 (0.0008) [2023-10-11 22:22:16,021][71601] Updated weights for policy 0, policy_version 83500 (0.0010) [2023-10-11 22:22:16,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 170917888. Throughput: 0: 1817.0, 1: 1814.5. Samples: 42735780. Policy #0 lag: (min: 10.0, avg: 12.0, max: 33.0) [2023-10-11 22:22:16,034][70582] Avg episode reward: [(0, '94.910'), (1, '103.670')] [2023-10-11 22:22:16,399][71601] Updated weights for policy 0, policy_version 83510 (0.0007) [2023-10-11 22:22:16,777][71601] Updated weights for policy 0, policy_version 83520 (0.0009) [2023-10-11 22:22:17,461][71635] Updated weights for policy 1, policy_version 83432 (0.0008) [2023-10-11 22:22:17,824][71635] Updated weights for policy 1, policy_version 83442 (0.0008) [2023-10-11 22:22:18,190][71635] Updated weights for policy 1, policy_version 83452 (0.0011) [2023-10-11 22:22:20,409][71601] Updated weights for policy 0, policy_version 83530 (0.0010) [2023-10-11 22:22:20,781][71601] Updated weights for policy 0, policy_version 83540 (0.0011) [2023-10-11 22:22:21,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 170983424. Throughput: 0: 1817.3, 1: 1809.3. Samples: 42758106. Policy #0 lag: (min: 10.0, avg: 12.0, max: 33.0) [2023-10-11 22:22:21,035][70582] Avg episode reward: [(0, '99.870'), (1, '104.180')] [2023-10-11 22:22:21,164][71601] Updated weights for policy 0, policy_version 83550 (0.0010) [2023-10-11 22:22:21,841][71635] Updated weights for policy 1, policy_version 83462 (0.0009) [2023-10-11 22:22:22,203][71635] Updated weights for policy 1, policy_version 83472 (0.0008) [2023-10-11 22:22:22,567][71635] Updated weights for policy 1, policy_version 83482 (0.0007) [2023-10-11 22:22:24,661][71601] Updated weights for policy 0, policy_version 83560 (0.0007) [2023-10-11 22:22:25,027][71601] Updated weights for policy 0, policy_version 83570 (0.0007) [2023-10-11 22:22:25,401][71601] Updated weights for policy 0, policy_version 83580 (0.0008) [2023-10-11 22:22:26,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 171081728. Throughput: 0: 1818.5, 1: 1807.7. Samples: 42780024. Policy #0 lag: (min: 10.0, avg: 12.0, max: 33.0) [2023-10-11 22:22:26,034][70582] Avg episode reward: [(0, '99.130'), (1, '101.780')] [2023-10-11 22:22:26,228][71635] Updated weights for policy 1, policy_version 83492 (0.0007) [2023-10-11 22:22:26,587][71635] Updated weights for policy 1, policy_version 83502 (0.0010) [2023-10-11 22:22:26,955][71635] Updated weights for policy 1, policy_version 83512 (0.0010) [2023-10-11 22:22:29,243][71601] Updated weights for policy 0, policy_version 83590 (0.0008) [2023-10-11 22:22:29,608][71601] Updated weights for policy 0, policy_version 83600 (0.0010) [2023-10-11 22:22:29,985][71601] Updated weights for policy 0, policy_version 83610 (0.0008) [2023-10-11 22:22:30,563][71635] Updated weights for policy 1, policy_version 83522 (0.0008) [2023-10-11 22:22:30,929][71635] Updated weights for policy 1, policy_version 83532 (0.0007) [2023-10-11 22:22:31,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 171147264. Throughput: 0: 1824.4, 1: 1811.2. Samples: 42791288. Policy #0 lag: (min: 10.0, avg: 12.0, max: 33.0) [2023-10-11 22:22:31,034][70582] Avg episode reward: [(0, '102.290'), (1, '100.970')] [2023-10-11 22:22:31,289][71635] Updated weights for policy 1, policy_version 83542 (0.0008) [2023-10-11 22:22:31,659][71635] Updated weights for policy 1, policy_version 83552 (0.0007) [2023-10-11 22:22:33,687][71601] Updated weights for policy 0, policy_version 83620 (0.0009) [2023-10-11 22:22:34,070][71601] Updated weights for policy 0, policy_version 83630 (0.0011) [2023-10-11 22:22:34,432][71601] Updated weights for policy 0, policy_version 83640 (0.0010) [2023-10-11 22:22:35,360][71635] Updated weights for policy 1, policy_version 83562 (0.0010) [2023-10-11 22:22:35,742][71635] Updated weights for policy 1, policy_version 83572 (0.0010) [2023-10-11 22:22:36,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 171212800. Throughput: 0: 1820.0, 1: 1816.0. Samples: 42812704. Policy #0 lag: (min: 10.0, avg: 12.0, max: 33.0) [2023-10-11 22:22:36,034][70582] Avg episode reward: [(0, '103.840'), (1, '94.480')] [2023-10-11 22:22:36,102][71635] Updated weights for policy 1, policy_version 83582 (0.0008) [2023-10-11 22:22:38,114][71601] Updated weights for policy 0, policy_version 83650 (0.0010) [2023-10-11 22:22:38,496][71601] Updated weights for policy 0, policy_version 83660 (0.0007) [2023-10-11 22:22:38,866][71601] Updated weights for policy 0, policy_version 83670 (0.0010) [2023-10-11 22:22:39,242][71601] Updated weights for policy 0, policy_version 83680 (0.0007) [2023-10-11 22:22:39,777][71635] Updated weights for policy 1, policy_version 83592 (0.0010) [2023-10-11 22:22:40,140][71635] Updated weights for policy 1, policy_version 83602 (0.0011) [2023-10-11 22:22:40,516][71635] Updated weights for policy 1, policy_version 83612 (0.0010) [2023-10-11 22:22:41,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 171311104. Throughput: 0: 1817.4, 1: 1818.7. Samples: 42834096. Policy #0 lag: (min: 10.0, avg: 12.0, max: 33.0) [2023-10-11 22:22:41,035][70582] Avg episode reward: [(0, '104.750'), (1, '94.620')] [2023-10-11 22:22:42,938][71601] Updated weights for policy 0, policy_version 83690 (0.0010) [2023-10-11 22:22:43,316][71601] Updated weights for policy 0, policy_version 83700 (0.0011) [2023-10-11 22:22:43,687][71601] Updated weights for policy 0, policy_version 83710 (0.0009) [2023-10-11 22:22:44,229][71635] Updated weights for policy 1, policy_version 83622 (0.0008) [2023-10-11 22:22:44,600][71635] Updated weights for policy 1, policy_version 83632 (0.0008) [2023-10-11 22:22:44,972][71635] Updated weights for policy 1, policy_version 83642 (0.0007) [2023-10-11 22:22:46,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 171376640. Throughput: 0: 1818.2, 1: 1822.5. Samples: 42845440. Policy #0 lag: (min: 10.0, avg: 12.0, max: 33.0) [2023-10-11 22:22:46,034][70582] Avg episode reward: [(0, '105.780'), (1, '97.190')] [2023-10-11 22:22:47,379][71601] Updated weights for policy 0, policy_version 83720 (0.0007) [2023-10-11 22:22:47,759][71601] Updated weights for policy 0, policy_version 83730 (0.0007) [2023-10-11 22:22:48,129][71601] Updated weights for policy 0, policy_version 83740 (0.0008) [2023-10-11 22:22:48,712][71635] Updated weights for policy 1, policy_version 83652 (0.0008) [2023-10-11 22:22:49,075][71635] Updated weights for policy 1, policy_version 83662 (0.0010) [2023-10-11 22:22:49,448][71635] Updated weights for policy 1, policy_version 83672 (0.0009) [2023-10-11 22:22:51,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 171442176. Throughput: 0: 1823.7, 1: 1819.1. Samples: 42866904. Policy #0 lag: (min: 10.0, avg: 12.0, max: 33.0) [2023-10-11 22:22:51,034][70582] Avg episode reward: [(0, '101.660'), (1, '97.890')] [2023-10-11 22:22:51,950][71601] Updated weights for policy 0, policy_version 83750 (0.0008) [2023-10-11 22:22:52,339][71601] Updated weights for policy 0, policy_version 83760 (0.0009) [2023-10-11 22:22:52,704][71601] Updated weights for policy 0, policy_version 83770 (0.0008) [2023-10-11 22:22:53,251][71635] Updated weights for policy 1, policy_version 83682 (0.0007) [2023-10-11 22:22:53,625][71635] Updated weights for policy 1, policy_version 83692 (0.0009) [2023-10-11 22:22:53,989][71635] Updated weights for policy 1, policy_version 83702 (0.0008) [2023-10-11 22:22:54,354][71635] Updated weights for policy 1, policy_version 83712 (0.0008) [2023-10-11 22:22:56,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.6, 300 sec: 14551.2). Total num frames: 171507712. Throughput: 0: 1816.3, 1: 1820.6. Samples: 42888892. Policy #0 lag: (min: 10.0, avg: 12.0, max: 33.0) [2023-10-11 22:22:56,034][70582] Avg episode reward: [(0, '105.810'), (1, '99.900')] [2023-10-11 22:22:56,536][71601] Updated weights for policy 0, policy_version 83780 (0.0008) [2023-10-11 22:22:56,903][71601] Updated weights for policy 0, policy_version 83790 (0.0009) [2023-10-11 22:22:57,277][71601] Updated weights for policy 0, policy_version 83800 (0.0009) [2023-10-11 22:22:58,094][71635] Updated weights for policy 1, policy_version 83722 (0.0010) [2023-10-11 22:22:58,464][71635] Updated weights for policy 1, policy_version 83732 (0.0007) [2023-10-11 22:22:58,836][71635] Updated weights for policy 1, policy_version 83742 (0.0008) [2023-10-11 22:23:01,006][71601] Updated weights for policy 0, policy_version 83810 (0.0008) [2023-10-11 22:23:01,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.6, 300 sec: 14551.2). Total num frames: 171573248. Throughput: 0: 1814.0, 1: 1825.2. Samples: 42899546. Policy #0 lag: (min: 10.0, avg: 12.0, max: 33.0) [2023-10-11 22:23:01,034][70582] Avg episode reward: [(0, '104.870'), (1, '103.860')] [2023-10-11 22:23:01,377][71601] Updated weights for policy 0, policy_version 83820 (0.0008) [2023-10-11 22:23:01,759][71601] Updated weights for policy 0, policy_version 83830 (0.0007) [2023-10-11 22:23:02,133][71601] Updated weights for policy 0, policy_version 83840 (0.0009) [2023-10-11 22:23:02,393][71635] Updated weights for policy 1, policy_version 83752 (0.0010) [2023-10-11 22:23:02,760][71635] Updated weights for policy 1, policy_version 83762 (0.0011) [2023-10-11 22:23:03,136][71635] Updated weights for policy 1, policy_version 83772 (0.0009) [2023-10-11 22:23:05,819][71601] Updated weights for policy 0, policy_version 83850 (0.0007) [2023-10-11 22:23:06,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 171638784. Throughput: 0: 1804.1, 1: 1824.1. Samples: 42921374. Policy #0 lag: (min: 10.0, avg: 12.0, max: 33.0) [2023-10-11 22:23:06,034][70582] Avg episode reward: [(0, '100.650'), (1, '103.510')] [2023-10-11 22:23:06,198][71601] Updated weights for policy 0, policy_version 83860 (0.0008) [2023-10-11 22:23:06,579][71601] Updated weights for policy 0, policy_version 83870 (0.0008) [2023-10-11 22:23:06,674][71635] Updated weights for policy 1, policy_version 83782 (0.0008) [2023-10-11 22:23:07,041][71635] Updated weights for policy 1, policy_version 83792 (0.0007) [2023-10-11 22:23:07,416][71635] Updated weights for policy 1, policy_version 83802 (0.0008) [2023-10-11 22:23:10,265][71601] Updated weights for policy 0, policy_version 83880 (0.0008) [2023-10-11 22:23:10,634][71601] Updated weights for policy 0, policy_version 83890 (0.0008) [2023-10-11 22:23:11,006][71635] Updated weights for policy 1, policy_version 83812 (0.0007) [2023-10-11 22:23:11,018][71601] Updated weights for policy 0, policy_version 83900 (0.0007) [2023-10-11 22:23:11,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 171704320. Throughput: 0: 1807.8, 1: 1832.9. Samples: 42943856. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-11 22:23:11,034][70582] Avg episode reward: [(0, '95.470'), (1, '104.190')] [2023-10-11 22:23:11,369][71635] Updated weights for policy 1, policy_version 83822 (0.0009) [2023-10-11 22:23:11,735][71635] Updated weights for policy 1, policy_version 83832 (0.0010) [2023-10-11 22:23:14,582][71601] Updated weights for policy 0, policy_version 83910 (0.0008) [2023-10-11 22:23:14,953][71601] Updated weights for policy 0, policy_version 83920 (0.0009) [2023-10-11 22:23:15,332][71601] Updated weights for policy 0, policy_version 83930 (0.0008) [2023-10-11 22:23:15,388][71635] Updated weights for policy 1, policy_version 83842 (0.0008) [2023-10-11 22:23:15,751][71635] Updated weights for policy 1, policy_version 83852 (0.0007) [2023-10-11 22:23:16,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 171802624. Throughput: 0: 1795.0, 1: 1832.5. Samples: 42954528. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-11 22:23:16,034][70582] Avg episode reward: [(0, '96.870'), (1, '98.590')] [2023-10-11 22:23:16,120][71635] Updated weights for policy 1, policy_version 83862 (0.0008) [2023-10-11 22:23:16,492][71635] Updated weights for policy 1, policy_version 83872 (0.0009) [2023-10-11 22:23:18,983][71601] Updated weights for policy 0, policy_version 83940 (0.0008) [2023-10-11 22:23:19,354][71601] Updated weights for policy 0, policy_version 83950 (0.0008) [2023-10-11 22:23:19,719][71601] Updated weights for policy 0, policy_version 83960 (0.0007) [2023-10-11 22:23:20,306][71635] Updated weights for policy 1, policy_version 83882 (0.0010) [2023-10-11 22:23:20,677][71635] Updated weights for policy 1, policy_version 83892 (0.0011) [2023-10-11 22:23:21,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 171868160. Throughput: 0: 1808.4, 1: 1829.4. Samples: 42976404. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-11 22:23:21,034][70582] Avg episode reward: [(0, '93.300'), (1, '102.710')] [2023-10-11 22:23:21,042][71635] Updated weights for policy 1, policy_version 83902 (0.0009) [2023-10-11 22:23:23,471][71601] Updated weights for policy 0, policy_version 83970 (0.0008) [2023-10-11 22:23:23,851][71601] Updated weights for policy 0, policy_version 83980 (0.0010) [2023-10-11 22:23:24,208][71601] Updated weights for policy 0, policy_version 83990 (0.0010) [2023-10-11 22:23:24,586][71601] Updated weights for policy 0, policy_version 84000 (0.0007) [2023-10-11 22:23:24,824][71635] Updated weights for policy 1, policy_version 83912 (0.0009) [2023-10-11 22:23:25,186][71635] Updated weights for policy 1, policy_version 83922 (0.0008) [2023-10-11 22:23:25,561][71635] Updated weights for policy 1, policy_version 83932 (0.0008) [2023-10-11 22:23:26,034][70582] Fps is (10 sec: 16383.4, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 171966464. Throughput: 0: 1801.8, 1: 1825.8. Samples: 42997340. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-11 22:23:26,035][70582] Avg episode reward: [(0, '96.770'), (1, '103.690')] [2023-10-11 22:23:28,355][71601] Updated weights for policy 0, policy_version 84010 (0.0008) [2023-10-11 22:23:28,718][71601] Updated weights for policy 0, policy_version 84020 (0.0009) [2023-10-11 22:23:29,100][71601] Updated weights for policy 0, policy_version 84030 (0.0009) [2023-10-11 22:23:29,299][71635] Updated weights for policy 1, policy_version 83942 (0.0008) [2023-10-11 22:23:29,660][71635] Updated weights for policy 1, policy_version 83952 (0.0008) [2023-10-11 22:23:30,017][71635] Updated weights for policy 1, policy_version 83962 (0.0007) [2023-10-11 22:23:31,034][70582] Fps is (10 sec: 16383.4, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 172032000. Throughput: 0: 1812.7, 1: 1823.9. Samples: 43009090. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-11 22:23:31,035][70582] Avg episode reward: [(0, '102.300'), (1, '104.430')] [2023-10-11 22:23:32,877][71601] Updated weights for policy 0, policy_version 84040 (0.0010) [2023-10-11 22:23:33,259][71601] Updated weights for policy 0, policy_version 84050 (0.0010) [2023-10-11 22:23:33,631][71601] Updated weights for policy 0, policy_version 84060 (0.0009) [2023-10-11 22:23:33,751][71635] Updated weights for policy 1, policy_version 83972 (0.0008) [2023-10-11 22:23:34,125][71635] Updated weights for policy 1, policy_version 83982 (0.0008) [2023-10-11 22:23:34,496][71635] Updated weights for policy 1, policy_version 83992 (0.0007) [2023-10-11 22:23:36,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 172097536. Throughput: 0: 1792.8, 1: 1825.2. Samples: 43029716. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-11 22:23:36,035][70582] Avg episode reward: [(0, '103.560'), (1, '107.020')] [2023-10-11 22:23:37,431][71601] Updated weights for policy 0, policy_version 84070 (0.0009) [2023-10-11 22:23:37,805][71601] Updated weights for policy 0, policy_version 84080 (0.0009) [2023-10-11 22:23:38,149][71635] Updated weights for policy 1, policy_version 84002 (0.0008) [2023-10-11 22:23:38,177][71601] Updated weights for policy 0, policy_version 84090 (0.0010) [2023-10-11 22:23:38,525][71635] Updated weights for policy 1, policy_version 84012 (0.0009) [2023-10-11 22:23:38,894][71635] Updated weights for policy 1, policy_version 84022 (0.0009) [2023-10-11 22:23:39,249][71635] Updated weights for policy 1, policy_version 84032 (0.0007) [2023-10-11 22:23:41,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 172163072. Throughput: 0: 1795.8, 1: 1823.5. Samples: 43051764. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-11 22:23:41,035][70582] Avg episode reward: [(0, '101.630'), (1, '106.220')] [2023-10-11 22:23:41,789][71601] Updated weights for policy 0, policy_version 84100 (0.0008) [2023-10-11 22:23:42,159][71601] Updated weights for policy 0, policy_version 84110 (0.0007) [2023-10-11 22:23:42,531][71601] Updated weights for policy 0, policy_version 84120 (0.0008) [2023-10-11 22:23:42,999][71635] Updated weights for policy 1, policy_version 84042 (0.0007) [2023-10-11 22:23:43,369][71635] Updated weights for policy 1, policy_version 84052 (0.0007) [2023-10-11 22:23:43,729][71635] Updated weights for policy 1, policy_version 84062 (0.0007) [2023-10-11 22:23:46,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 172228608. Throughput: 0: 1796.1, 1: 1818.7. Samples: 43062212. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-11 22:23:46,035][70582] Avg episode reward: [(0, '106.050'), (1, '102.030')] [2023-10-11 22:23:46,060][71601] Updated weights for policy 0, policy_version 84130 (0.0008) [2023-10-11 22:23:46,435][71601] Updated weights for policy 0, policy_version 84140 (0.0008) [2023-10-11 22:23:46,820][71601] Updated weights for policy 0, policy_version 84150 (0.0008) [2023-10-11 22:23:47,187][71601] Updated weights for policy 0, policy_version 84160 (0.0007) [2023-10-11 22:23:47,361][71635] Updated weights for policy 1, policy_version 84072 (0.0008) [2023-10-11 22:23:47,724][71635] Updated weights for policy 1, policy_version 84082 (0.0010) [2023-10-11 22:23:48,100][71635] Updated weights for policy 1, policy_version 84092 (0.0009) [2023-10-11 22:23:50,796][71601] Updated weights for policy 0, policy_version 84170 (0.0009) [2023-10-11 22:23:51,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 172294144. Throughput: 0: 1807.7, 1: 1822.1. Samples: 43084714. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-11 22:23:51,035][70582] Avg episode reward: [(0, '101.040'), (1, '100.800')] [2023-10-11 22:23:51,172][71601] Updated weights for policy 0, policy_version 84180 (0.0010) [2023-10-11 22:23:51,550][71601] Updated weights for policy 0, policy_version 84190 (0.0009) [2023-10-11 22:23:51,782][71635] Updated weights for policy 1, policy_version 84102 (0.0008) [2023-10-11 22:23:52,146][71635] Updated weights for policy 1, policy_version 84112 (0.0011) [2023-10-11 22:23:52,504][71635] Updated weights for policy 1, policy_version 84122 (0.0010) [2023-10-11 22:23:55,168][71601] Updated weights for policy 0, policy_version 84200 (0.0010) [2023-10-11 22:23:55,533][71601] Updated weights for policy 0, policy_version 84210 (0.0011) [2023-10-11 22:23:55,911][71601] Updated weights for policy 0, policy_version 84220 (0.0007) [2023-10-11 22:23:56,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 172359680. Throughput: 0: 1816.4, 1: 1808.8. Samples: 43106986. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-11 22:23:56,034][70582] Avg episode reward: [(0, '100.300'), (1, '104.530')] [2023-10-11 22:23:56,054][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000084224_86245376.pth... [2023-10-11 22:23:56,083][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000082528_84508672.pth [2023-10-11 22:23:56,125][71635] Updated weights for policy 1, policy_version 84132 (0.0010) [2023-10-11 22:23:56,481][71635] Updated weights for policy 1, policy_version 84142 (0.0008) [2023-10-11 22:23:56,847][71635] Updated weights for policy 1, policy_version 84152 (0.0007) [2023-10-11 22:23:57,133][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000084160_86179840.pth... [2023-10-11 22:23:57,165][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000082432_84410368.pth [2023-10-11 22:23:59,601][71601] Updated weights for policy 0, policy_version 84230 (0.0011) [2023-10-11 22:23:59,971][71601] Updated weights for policy 0, policy_version 84240 (0.0008) [2023-10-11 22:24:00,343][71601] Updated weights for policy 0, policy_version 84250 (0.0011) [2023-10-11 22:24:00,622][71635] Updated weights for policy 1, policy_version 84162 (0.0008) [2023-10-11 22:24:00,985][71635] Updated weights for policy 1, policy_version 84172 (0.0009) [2023-10-11 22:24:01,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 172457984. Throughput: 0: 1823.5, 1: 1807.2. Samples: 43117908. Policy #0 lag: (min: 28.0, avg: 33.5, max: 60.0) [2023-10-11 22:24:01,035][70582] Avg episode reward: [(0, '108.010'), (1, '100.930')] [2023-10-11 22:24:01,350][71635] Updated weights for policy 1, policy_version 84182 (0.0008) [2023-10-11 22:24:01,716][71635] Updated weights for policy 1, policy_version 84192 (0.0010) [2023-10-11 22:24:04,099][71601] Updated weights for policy 0, policy_version 84260 (0.0009) [2023-10-11 22:24:04,478][71601] Updated weights for policy 0, policy_version 84270 (0.0009) [2023-10-11 22:24:04,857][71601] Updated weights for policy 0, policy_version 84280 (0.0010) [2023-10-11 22:24:05,396][71635] Updated weights for policy 1, policy_version 84202 (0.0007) [2023-10-11 22:24:05,757][71635] Updated weights for policy 1, policy_version 84212 (0.0008) [2023-10-11 22:24:06,034][70582] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 172523520. Throughput: 0: 1826.6, 1: 1807.7. Samples: 43139946. Policy #0 lag: (min: 28.0, avg: 33.5, max: 60.0) [2023-10-11 22:24:06,035][70582] Avg episode reward: [(0, '109.470'), (1, '106.760')] [2023-10-11 22:24:06,120][71635] Updated weights for policy 1, policy_version 84222 (0.0008) [2023-10-11 22:24:08,376][71601] Updated weights for policy 0, policy_version 84290 (0.0009) [2023-10-11 22:24:08,741][71601] Updated weights for policy 0, policy_version 84300 (0.0010) [2023-10-11 22:24:09,116][71601] Updated weights for policy 0, policy_version 84310 (0.0007) [2023-10-11 22:24:09,485][71601] Updated weights for policy 0, policy_version 84320 (0.0007) [2023-10-11 22:24:09,852][71635] Updated weights for policy 1, policy_version 84232 (0.0010) [2023-10-11 22:24:10,224][71635] Updated weights for policy 1, policy_version 84242 (0.0009) [2023-10-11 22:24:10,587][71635] Updated weights for policy 1, policy_version 84252 (0.0007) [2023-10-11 22:24:11,034][70582] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 172621824. Throughput: 0: 1825.2, 1: 1812.7. Samples: 43161046. Policy #0 lag: (min: 28.0, avg: 33.5, max: 60.0) [2023-10-11 22:24:11,035][70582] Avg episode reward: [(0, '110.940'), (1, '107.170')] [2023-10-11 22:24:13,202][71601] Updated weights for policy 0, policy_version 84330 (0.0009) [2023-10-11 22:24:13,570][71601] Updated weights for policy 0, policy_version 84340 (0.0007) [2023-10-11 22:24:13,951][71601] Updated weights for policy 0, policy_version 84350 (0.0009) [2023-10-11 22:24:14,397][71635] Updated weights for policy 1, policy_version 84262 (0.0008) [2023-10-11 22:24:14,767][71635] Updated weights for policy 1, policy_version 84272 (0.0009) [2023-10-11 22:24:15,143][71635] Updated weights for policy 1, policy_version 84282 (0.0010) [2023-10-11 22:24:16,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 172687360. Throughput: 0: 1822.1, 1: 1812.2. Samples: 43172632. Policy #0 lag: (min: 28.0, avg: 33.5, max: 60.0) [2023-10-11 22:24:16,035][70582] Avg episode reward: [(0, '114.620'), (1, '109.760')] [2023-10-11 22:24:17,512][71601] Updated weights for policy 0, policy_version 84360 (0.0011) [2023-10-11 22:24:17,887][71601] Updated weights for policy 0, policy_version 84370 (0.0007) [2023-10-11 22:24:18,255][71601] Updated weights for policy 0, policy_version 84380 (0.0007) [2023-10-11 22:24:18,910][71635] Updated weights for policy 1, policy_version 84292 (0.0008) [2023-10-11 22:24:19,276][71635] Updated weights for policy 1, policy_version 84302 (0.0010) [2023-10-11 22:24:19,635][71635] Updated weights for policy 1, policy_version 84312 (0.0007) [2023-10-11 22:24:21,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 172752896. Throughput: 0: 1834.4, 1: 1821.9. Samples: 43194250. Policy #0 lag: (min: 28.0, avg: 33.5, max: 60.0) [2023-10-11 22:24:21,035][70582] Avg episode reward: [(0, '110.360'), (1, '111.290')] [2023-10-11 22:24:21,804][71601] Updated weights for policy 0, policy_version 84390 (0.0008) [2023-10-11 22:24:22,183][71601] Updated weights for policy 0, policy_version 84400 (0.0007) [2023-10-11 22:24:22,553][71601] Updated weights for policy 0, policy_version 84410 (0.0010) [2023-10-11 22:24:23,251][71635] Updated weights for policy 1, policy_version 84322 (0.0007) [2023-10-11 22:24:23,628][71635] Updated weights for policy 1, policy_version 84332 (0.0008) [2023-10-11 22:24:23,997][71635] Updated weights for policy 1, policy_version 84342 (0.0011) [2023-10-11 22:24:24,355][71635] Updated weights for policy 1, policy_version 84352 (0.0010) [2023-10-11 22:24:26,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 172818432. Throughput: 0: 1841.0, 1: 1820.4. Samples: 43216526. Policy #0 lag: (min: 28.0, avg: 33.5, max: 60.0) [2023-10-11 22:24:26,034][70582] Avg episode reward: [(0, '99.190'), (1, '112.400')] [2023-10-11 22:24:26,403][71601] Updated weights for policy 0, policy_version 84420 (0.0009) [2023-10-11 22:24:26,779][71601] Updated weights for policy 0, policy_version 84430 (0.0008) [2023-10-11 22:24:27,162][71601] Updated weights for policy 0, policy_version 84440 (0.0008) [2023-10-11 22:24:28,022][71635] Updated weights for policy 1, policy_version 84362 (0.0010) [2023-10-11 22:24:28,392][71635] Updated weights for policy 1, policy_version 84372 (0.0010) [2023-10-11 22:24:28,767][71635] Updated weights for policy 1, policy_version 84382 (0.0009) [2023-10-11 22:24:30,937][71601] Updated weights for policy 0, policy_version 84450 (0.0010) [2023-10-11 22:24:31,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.6, 300 sec: 14551.2). Total num frames: 172883968. Throughput: 0: 1837.1, 1: 1825.1. Samples: 43227010. Policy #0 lag: (min: 28.0, avg: 33.5, max: 60.0) [2023-10-11 22:24:31,034][70582] Avg episode reward: [(0, '98.950'), (1, '114.090')] [2023-10-11 22:24:31,309][71601] Updated weights for policy 0, policy_version 84460 (0.0010) [2023-10-11 22:24:31,676][71601] Updated weights for policy 0, policy_version 84470 (0.0010) [2023-10-11 22:24:32,049][71601] Updated weights for policy 0, policy_version 84480 (0.0010) [2023-10-11 22:24:32,419][71635] Updated weights for policy 1, policy_version 84392 (0.0009) [2023-10-11 22:24:32,795][71635] Updated weights for policy 1, policy_version 84402 (0.0007) [2023-10-11 22:24:33,152][71635] Updated weights for policy 1, policy_version 84412 (0.0007) [2023-10-11 22:24:35,691][71601] Updated weights for policy 0, policy_version 84490 (0.0008) [2023-10-11 22:24:36,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 172949504. Throughput: 0: 1835.2, 1: 1823.3. Samples: 43249348. Policy #0 lag: (min: 28.0, avg: 33.5, max: 60.0) [2023-10-11 22:24:36,035][70582] Avg episode reward: [(0, '101.840'), (1, '108.750')] [2023-10-11 22:24:36,068][71601] Updated weights for policy 0, policy_version 84500 (0.0009) [2023-10-11 22:24:36,435][71601] Updated weights for policy 0, policy_version 84510 (0.0007) [2023-10-11 22:24:36,761][71635] Updated weights for policy 1, policy_version 84422 (0.0008) [2023-10-11 22:24:37,130][71635] Updated weights for policy 1, policy_version 84432 (0.0008) [2023-10-11 22:24:37,492][71635] Updated weights for policy 1, policy_version 84442 (0.0009) [2023-10-11 22:24:40,129][71601] Updated weights for policy 0, policy_version 84520 (0.0009) [2023-10-11 22:24:40,511][71601] Updated weights for policy 0, policy_version 84530 (0.0007) [2023-10-11 22:24:40,879][71601] Updated weights for policy 0, policy_version 84540 (0.0007) [2023-10-11 22:24:41,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 173047808. Throughput: 0: 1829.4, 1: 1822.5. Samples: 43271324. Policy #0 lag: (min: 28.0, avg: 33.5, max: 60.0) [2023-10-11 22:24:41,034][70582] Avg episode reward: [(0, '109.110'), (1, '115.220')] [2023-10-11 22:24:41,213][71635] Updated weights for policy 1, policy_version 84452 (0.0010) [2023-10-11 22:24:41,582][71635] Updated weights for policy 1, policy_version 84462 (0.0009) [2023-10-11 22:24:41,941][71635] Updated weights for policy 1, policy_version 84472 (0.0007) [2023-10-11 22:24:44,583][71601] Updated weights for policy 0, policy_version 84550 (0.0009) [2023-10-11 22:24:44,954][71601] Updated weights for policy 0, policy_version 84560 (0.0008) [2023-10-11 22:24:45,329][71601] Updated weights for policy 0, policy_version 84570 (0.0007) [2023-10-11 22:24:45,661][71635] Updated weights for policy 1, policy_version 84482 (0.0008) [2023-10-11 22:24:46,022][71635] Updated weights for policy 1, policy_version 84492 (0.0008) [2023-10-11 22:24:46,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 173113344. Throughput: 0: 1827.6, 1: 1819.7. Samples: 43282036. Policy #0 lag: (min: 28.0, avg: 33.5, max: 60.0) [2023-10-11 22:24:46,035][70582] Avg episode reward: [(0, '114.330'), (1, '114.560')] [2023-10-11 22:24:46,385][71635] Updated weights for policy 1, policy_version 84502 (0.0009) [2023-10-11 22:24:46,750][71635] Updated weights for policy 1, policy_version 84512 (0.0010) [2023-10-11 22:24:49,095][71601] Updated weights for policy 0, policy_version 84580 (0.0008) [2023-10-11 22:24:49,468][71601] Updated weights for policy 0, policy_version 84590 (0.0009) [2023-10-11 22:24:49,839][71601] Updated weights for policy 0, policy_version 84600 (0.0007) [2023-10-11 22:24:50,544][71635] Updated weights for policy 1, policy_version 84522 (0.0008) [2023-10-11 22:24:50,908][71635] Updated weights for policy 1, policy_version 84532 (0.0009) [2023-10-11 22:24:51,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 173178880. Throughput: 0: 1823.2, 1: 1820.8. Samples: 43303928. Policy #0 lag: (min: 28.0, avg: 33.5, max: 60.0) [2023-10-11 22:24:51,035][70582] Avg episode reward: [(0, '118.110'), (1, '113.380')] [2023-10-11 22:24:51,276][71635] Updated weights for policy 1, policy_version 84542 (0.0007) [2023-10-11 22:24:53,338][71601] Updated weights for policy 0, policy_version 84610 (0.0007) [2023-10-11 22:24:53,713][71601] Updated weights for policy 0, policy_version 84620 (0.0008) [2023-10-11 22:24:54,095][71601] Updated weights for policy 0, policy_version 84630 (0.0011) [2023-10-11 22:24:54,464][71601] Updated weights for policy 0, policy_version 84640 (0.0009) [2023-10-11 22:24:54,966][71635] Updated weights for policy 1, policy_version 84552 (0.0008) [2023-10-11 22:24:55,332][71635] Updated weights for policy 1, policy_version 84562 (0.0007) [2023-10-11 22:24:55,692][71635] Updated weights for policy 1, policy_version 84572 (0.0007) [2023-10-11 22:24:56,034][70582] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 173277184. Throughput: 0: 1825.4, 1: 1820.4. Samples: 43325106. Policy #0 lag: (min: 10.0, avg: 13.1, max: 37.0) [2023-10-11 22:24:56,035][70582] Avg episode reward: [(0, '114.630'), (1, '116.100')] [2023-10-11 22:24:58,103][71601] Updated weights for policy 0, policy_version 84650 (0.0007) [2023-10-11 22:24:58,477][71601] Updated weights for policy 0, policy_version 84660 (0.0008) [2023-10-11 22:24:58,846][71601] Updated weights for policy 0, policy_version 84670 (0.0008) [2023-10-11 22:24:59,424][71635] Updated weights for policy 1, policy_version 84582 (0.0009) [2023-10-11 22:24:59,788][71635] Updated weights for policy 1, policy_version 84592 (0.0010) [2023-10-11 22:25:00,162][71635] Updated weights for policy 1, policy_version 84602 (0.0010) [2023-10-11 22:25:01,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 173342720. Throughput: 0: 1826.5, 1: 1817.5. Samples: 43336610. Policy #0 lag: (min: 10.0, avg: 13.1, max: 37.0) [2023-10-11 22:25:01,034][70582] Avg episode reward: [(0, '115.140'), (1, '113.060')] [2023-10-11 22:25:02,485][71601] Updated weights for policy 0, policy_version 84680 (0.0008) [2023-10-11 22:25:02,850][71601] Updated weights for policy 0, policy_version 84690 (0.0008) [2023-10-11 22:25:03,224][71601] Updated weights for policy 0, policy_version 84700 (0.0009) [2023-10-11 22:25:03,987][71635] Updated weights for policy 1, policy_version 84612 (0.0009) [2023-10-11 22:25:04,357][71635] Updated weights for policy 1, policy_version 84622 (0.0007) [2023-10-11 22:25:04,724][71635] Updated weights for policy 1, policy_version 84632 (0.0007) [2023-10-11 22:25:06,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 173408256. Throughput: 0: 1822.0, 1: 1813.4. Samples: 43357842. Policy #0 lag: (min: 10.0, avg: 13.1, max: 37.0) [2023-10-11 22:25:06,035][70582] Avg episode reward: [(0, '120.490'), (1, '110.680')] [2023-10-11 22:25:06,995][71601] Updated weights for policy 0, policy_version 84710 (0.0008) [2023-10-11 22:25:07,367][71601] Updated weights for policy 0, policy_version 84720 (0.0008) [2023-10-11 22:25:07,736][71601] Updated weights for policy 0, policy_version 84730 (0.0009) [2023-10-11 22:25:08,483][71635] Updated weights for policy 1, policy_version 84642 (0.0010) [2023-10-11 22:25:08,852][71635] Updated weights for policy 1, policy_version 84652 (0.0008) [2023-10-11 22:25:09,222][71635] Updated weights for policy 1, policy_version 84662 (0.0011) [2023-10-11 22:25:09,593][71635] Updated weights for policy 1, policy_version 84672 (0.0010) [2023-10-11 22:25:11,034][70582] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 173473792. Throughput: 0: 1820.4, 1: 1806.8. Samples: 43379752. Policy #0 lag: (min: 10.0, avg: 13.1, max: 37.0) [2023-10-11 22:25:11,035][70582] Avg episode reward: [(0, '126.000'), (1, '104.320')] [2023-10-11 22:25:11,459][71601] Updated weights for policy 0, policy_version 84740 (0.0008) [2023-10-11 22:25:11,853][71601] Updated weights for policy 0, policy_version 84750 (0.0009) [2023-10-11 22:25:12,225][71601] Updated weights for policy 0, policy_version 84760 (0.0008) [2023-10-11 22:25:13,271][71635] Updated weights for policy 1, policy_version 84682 (0.0010) [2023-10-11 22:25:13,650][71635] Updated weights for policy 1, policy_version 84692 (0.0010) [2023-10-11 22:25:14,017][71635] Updated weights for policy 1, policy_version 84702 (0.0008) [2023-10-11 22:25:15,865][71601] Updated weights for policy 0, policy_version 84770 (0.0009) [2023-10-11 22:25:16,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 173539328. Throughput: 0: 1821.4, 1: 1812.0. Samples: 43390514. Policy #0 lag: (min: 10.0, avg: 13.1, max: 37.0) [2023-10-11 22:25:16,034][70582] Avg episode reward: [(0, '129.980'), (1, '99.200')] [2023-10-11 22:25:16,234][71601] Updated weights for policy 0, policy_version 84780 (0.0011) [2023-10-11 22:25:16,599][71601] Updated weights for policy 0, policy_version 84790 (0.0011) [2023-10-11 22:25:16,972][71601] Updated weights for policy 0, policy_version 84800 (0.0007) [2023-10-11 22:25:17,843][71635] Updated weights for policy 1, policy_version 84712 (0.0009) [2023-10-11 22:25:18,206][71635] Updated weights for policy 1, policy_version 84722 (0.0010) [2023-10-11 22:25:18,566][71635] Updated weights for policy 1, policy_version 84732 (0.0008) [2023-10-11 22:25:20,561][71601] Updated weights for policy 0, policy_version 84810 (0.0009) [2023-10-11 22:25:20,925][71601] Updated weights for policy 0, policy_version 84820 (0.0008) [2023-10-11 22:25:21,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 173604864. Throughput: 0: 1821.2, 1: 1798.0. Samples: 43412210. Policy #0 lag: (min: 10.0, avg: 13.1, max: 37.0) [2023-10-11 22:25:21,035][70582] Avg episode reward: [(0, '128.630'), (1, '102.850')] [2023-10-11 22:25:21,288][71601] Updated weights for policy 0, policy_version 84830 (0.0009) [2023-10-11 22:25:22,232][71635] Updated weights for policy 1, policy_version 84742 (0.0009) [2023-10-11 22:25:22,598][71635] Updated weights for policy 1, policy_version 84752 (0.0008) [2023-10-11 22:25:22,953][71635] Updated weights for policy 1, policy_version 84762 (0.0010) [2023-10-11 22:25:24,959][71601] Updated weights for policy 0, policy_version 84840 (0.0008) [2023-10-11 22:25:25,327][71601] Updated weights for policy 0, policy_version 84850 (0.0007) [2023-10-11 22:25:25,702][71601] Updated weights for policy 0, policy_version 84860 (0.0007) [2023-10-11 22:25:26,034][70582] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 173703168. Throughput: 0: 1818.3, 1: 1801.3. Samples: 43434208. Policy #0 lag: (min: 10.0, avg: 13.1, max: 37.0) [2023-10-11 22:25:26,035][70582] Avg episode reward: [(0, '133.320'), (1, '104.450')] [2023-10-11 22:25:26,666][71635] Updated weights for policy 1, policy_version 84772 (0.0008) [2023-10-11 22:25:27,035][71635] Updated weights for policy 1, policy_version 84782 (0.0008) [2023-10-11 22:25:27,398][71635] Updated weights for policy 1, policy_version 84792 (0.0008) [2023-10-11 22:25:29,460][71601] Updated weights for policy 0, policy_version 84870 (0.0010) [2023-10-11 22:25:29,837][71601] Updated weights for policy 0, policy_version 84880 (0.0009) [2023-10-11 22:25:30,206][71601] Updated weights for policy 0, policy_version 84890 (0.0009) [2023-10-11 22:25:31,034][70582] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 173768704. Throughput: 0: 1815.3, 1: 1804.9. Samples: 43444944. Policy #0 lag: (min: 10.0, avg: 13.1, max: 37.0) [2023-10-11 22:25:31,034][70582] Avg episode reward: [(0, '138.000'), (1, '106.550')] [2023-10-11 22:25:31,054][71635] Updated weights for policy 1, policy_version 84802 (0.0010) [2023-10-11 22:25:31,416][71635] Updated weights for policy 1, policy_version 84812 (0.0008) [2023-10-11 22:25:31,787][71635] Updated weights for policy 1, policy_version 84822 (0.0009) [2023-10-11 22:25:32,158][71635] Updated weights for policy 1, policy_version 84832 (0.0008) [2023-10-11 22:25:34,018][71601] Updated weights for policy 0, policy_version 84900 (0.0009) [2023-10-11 22:25:34,399][71601] Updated weights for policy 0, policy_version 84910 (0.0009) [2023-10-11 22:25:34,776][71601] Updated weights for policy 0, policy_version 84920 (0.0008) [2023-10-11 22:25:35,722][71635] Updated weights for policy 1, policy_version 84842 (0.0007) [2023-10-11 22:25:36,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 173834240. Throughput: 0: 1818.0, 1: 1808.4. Samples: 43467116. Policy #0 lag: (min: 10.0, avg: 13.1, max: 37.0) [2023-10-11 22:25:36,035][70582] Avg episode reward: [(0, '139.990'), (1, '106.060')] [2023-10-11 22:25:36,098][71635] Updated weights for policy 1, policy_version 84852 (0.0007) [2023-10-11 22:25:36,465][71635] Updated weights for policy 1, policy_version 84862 (0.0007) [2023-10-11 22:25:38,468][71601] Updated weights for policy 0, policy_version 84930 (0.0009) [2023-10-11 22:25:38,827][71601] Updated weights for policy 0, policy_version 84940 (0.0009) [2023-10-11 22:25:39,202][71601] Updated weights for policy 0, policy_version 84950 (0.0008) [2023-10-11 22:25:39,573][71601] Updated weights for policy 0, policy_version 84960 (0.0009) [2023-10-11 22:25:40,200][71635] Updated weights for policy 1, policy_version 84872 (0.0009) [2023-10-11 22:25:40,566][71635] Updated weights for policy 1, policy_version 84882 (0.0008) [2023-10-11 22:25:40,931][71635] Updated weights for policy 1, policy_version 84892 (0.0007) [2023-10-11 22:25:41,034][70582] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 173899776. Throughput: 0: 1810.9, 1: 1822.4. Samples: 43488604. Policy #0 lag: (min: 10.0, avg: 13.1, max: 37.0) [2023-10-11 22:25:41,035][70582] Avg episode reward: [(0, '140.090'), (1, '107.500')] [2023-10-11 22:25:43,280][71601] Updated weights for policy 0, policy_version 84970 (0.0007) [2023-10-11 22:25:43,652][71601] Updated weights for policy 0, policy_version 84980 (0.0008) [2023-10-11 22:25:44,039][71601] Updated weights for policy 0, policy_version 84990 (0.0010) [2023-10-11 22:25:44,727][71635] Updated weights for policy 1, policy_version 84902 (0.0008) [2023-10-11 22:25:45,107][71635] Updated weights for policy 1, policy_version 84912 (0.0008) [2023-10-11 22:25:45,461][71635] Updated weights for policy 1, policy_version 84922 (0.0009) [2023-10-11 22:25:46,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 173998080. Throughput: 0: 1812.8, 1: 1813.9. Samples: 43499810. Policy #0 lag: (min: 10.0, avg: 13.1, max: 37.0) [2023-10-11 22:25:46,034][70582] Avg episode reward: [(0, '135.210'), (1, '104.580')] [2023-10-11 22:25:47,692][71601] Updated weights for policy 0, policy_version 85000 (0.0010) [2023-10-11 22:25:48,061][71601] Updated weights for policy 0, policy_version 85010 (0.0010) [2023-10-11 22:25:48,431][71601] Updated weights for policy 0, policy_version 85020 (0.0010) [2023-10-11 22:25:49,109][71635] Updated weights for policy 1, policy_version 84932 (0.0009) [2023-10-11 22:25:49,480][71635] Updated weights for policy 1, policy_version 84942 (0.0010) [2023-10-11 22:25:49,854][71635] Updated weights for policy 1, policy_version 84952 (0.0011) [2023-10-11 22:25:51,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 174063616. Throughput: 0: 1811.4, 1: 1819.6. Samples: 43521238. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 22:25:51,035][70582] Avg episode reward: [(0, '134.610'), (1, '114.000')] [2023-10-11 22:25:52,131][71601] Updated weights for policy 0, policy_version 85030 (0.0010) [2023-10-11 22:25:52,504][71601] Updated weights for policy 0, policy_version 85040 (0.0009) [2023-10-11 22:25:52,887][71601] Updated weights for policy 0, policy_version 85050 (0.0008) [2023-10-11 22:25:53,418][71635] Updated weights for policy 1, policy_version 84962 (0.0010) [2023-10-11 22:25:53,790][71635] Updated weights for policy 1, policy_version 84972 (0.0008) [2023-10-11 22:25:54,157][71635] Updated weights for policy 1, policy_version 84982 (0.0008) [2023-10-11 22:25:54,523][71635] Updated weights for policy 1, policy_version 84992 (0.0007) [2023-10-11 22:25:56,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 174129152. Throughput: 0: 1818.3, 1: 1820.9. Samples: 43543518. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 22:25:56,034][70582] Avg episode reward: [(0, '130.710'), (1, '111.970')] [2023-10-11 22:25:56,043][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000085056_87097344.pth... [2023-10-11 22:25:56,043][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000084992_87031808.pth... [2023-10-11 22:25:56,074][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000083360_85360640.pth [2023-10-11 22:25:56,078][71353] Saving a milestone ./train_atari/atari_gopher_APPO/checkpoint_p0/milestones/checkpoint_000085056_87097344.pth [2023-10-11 22:25:56,083][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000083296_85295104.pth [2023-10-11 22:25:56,087][71431] Saving a milestone ./train_atari/atari_gopher_APPO/checkpoint_p1/milestones/checkpoint_000084992_87031808.pth [2023-10-11 22:25:56,615][71601] Updated weights for policy 0, policy_version 85060 (0.0009) [2023-10-11 22:25:56,980][71601] Updated weights for policy 0, policy_version 85070 (0.0010) [2023-10-11 22:25:57,363][71601] Updated weights for policy 0, policy_version 85080 (0.0008) [2023-10-11 22:25:58,250][71635] Updated weights for policy 1, policy_version 85002 (0.0008) [2023-10-11 22:25:58,621][71635] Updated weights for policy 1, policy_version 85012 (0.0008) [2023-10-11 22:25:58,994][71635] Updated weights for policy 1, policy_version 85022 (0.0009) [2023-10-11 22:26:01,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 174194688. Throughput: 0: 1817.1, 1: 1819.4. Samples: 43554154. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 22:26:01,035][70582] Avg episode reward: [(0, '124.350'), (1, '116.470')] [2023-10-11 22:26:01,065][71601] Updated weights for policy 0, policy_version 85090 (0.0007) [2023-10-11 22:26:01,449][71601] Updated weights for policy 0, policy_version 85100 (0.0007) [2023-10-11 22:26:01,825][71601] Updated weights for policy 0, policy_version 85110 (0.0008) [2023-10-11 22:26:02,200][71601] Updated weights for policy 0, policy_version 85120 (0.0010) [2023-10-11 22:26:02,600][71635] Updated weights for policy 1, policy_version 85032 (0.0007) [2023-10-11 22:26:02,961][71635] Updated weights for policy 1, policy_version 85042 (0.0007) [2023-10-11 22:26:03,323][71635] Updated weights for policy 1, policy_version 85052 (0.0007) [2023-10-11 22:26:05,956][71601] Updated weights for policy 0, policy_version 85130 (0.0007) [2023-10-11 22:26:06,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 174260224. Throughput: 0: 1806.9, 1: 1826.5. Samples: 43575716. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 22:26:06,034][70582] Avg episode reward: [(0, '122.540'), (1, '124.780')] [2023-10-11 22:26:06,327][71601] Updated weights for policy 0, policy_version 85140 (0.0007) [2023-10-11 22:26:06,697][71601] Updated weights for policy 0, policy_version 85150 (0.0009) [2023-10-11 22:26:06,888][71635] Updated weights for policy 1, policy_version 85062 (0.0008) [2023-10-11 22:26:07,243][71635] Updated weights for policy 1, policy_version 85072 (0.0007) [2023-10-11 22:26:07,604][71635] Updated weights for policy 1, policy_version 85082 (0.0008) [2023-10-11 22:26:10,351][71601] Updated weights for policy 0, policy_version 85160 (0.0010) [2023-10-11 22:26:10,727][71601] Updated weights for policy 0, policy_version 85170 (0.0009) [2023-10-11 22:26:11,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 174325760. Throughput: 0: 1815.5, 1: 1829.2. Samples: 43598222. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 22:26:11,035][70582] Avg episode reward: [(0, '116.460'), (1, '126.030')] [2023-10-11 22:26:11,094][71601] Updated weights for policy 0, policy_version 85180 (0.0009) [2023-10-11 22:26:11,341][71635] Updated weights for policy 1, policy_version 85092 (0.0009) [2023-10-11 22:26:11,720][71635] Updated weights for policy 1, policy_version 85102 (0.0009) [2023-10-11 22:26:12,095][71635] Updated weights for policy 1, policy_version 85112 (0.0009) [2023-10-11 22:26:14,909][71601] Updated weights for policy 0, policy_version 85190 (0.0009) [2023-10-11 22:26:15,289][71601] Updated weights for policy 0, policy_version 85200 (0.0008) [2023-10-11 22:26:15,672][71601] Updated weights for policy 0, policy_version 85210 (0.0009) [2023-10-11 22:26:15,772][71635] Updated weights for policy 1, policy_version 85122 (0.0009) [2023-10-11 22:26:16,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 174424064. Throughput: 0: 1805.4, 1: 1827.5. Samples: 43608424. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 22:26:16,034][70582] Avg episode reward: [(0, '110.450'), (1, '126.580')] [2023-10-11 22:26:16,138][71635] Updated weights for policy 1, policy_version 85132 (0.0007) [2023-10-11 22:26:16,507][71635] Updated weights for policy 1, policy_version 85142 (0.0007) [2023-10-11 22:26:16,868][71635] Updated weights for policy 1, policy_version 85152 (0.0007) [2023-10-11 22:26:19,273][71601] Updated weights for policy 0, policy_version 85220 (0.0008) [2023-10-11 22:26:19,647][71601] Updated weights for policy 0, policy_version 85230 (0.0007) [2023-10-11 22:26:20,018][71601] Updated weights for policy 0, policy_version 85240 (0.0007) [2023-10-11 22:26:20,715][71635] Updated weights for policy 1, policy_version 85162 (0.0011) [2023-10-11 22:26:21,034][70582] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 174489600. Throughput: 0: 1809.8, 1: 1824.3. Samples: 43630650. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 22:26:21,034][70582] Avg episode reward: [(0, '110.890'), (1, '124.440')] [2023-10-11 22:26:21,088][71635] Updated weights for policy 1, policy_version 85172 (0.0009) [2023-10-11 22:26:21,456][71635] Updated weights for policy 1, policy_version 85182 (0.0007) [2023-10-11 22:26:23,629][71601] Updated weights for policy 0, policy_version 85250 (0.0007) [2023-10-11 22:26:23,992][71601] Updated weights for policy 0, policy_version 85260 (0.0008) [2023-10-11 22:26:24,370][71601] Updated weights for policy 0, policy_version 85270 (0.0009) [2023-10-11 22:26:24,746][71601] Updated weights for policy 0, policy_version 85280 (0.0007) [2023-10-11 22:26:25,170][71635] Updated weights for policy 1, policy_version 85192 (0.0007) [2023-10-11 22:26:25,545][71635] Updated weights for policy 1, policy_version 85202 (0.0010) [2023-10-11 22:26:25,908][71635] Updated weights for policy 1, policy_version 85212 (0.0009) [2023-10-11 22:26:26,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 174555136. Throughput: 0: 1807.6, 1: 1817.5. Samples: 43651734. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 22:26:26,034][70582] Avg episode reward: [(0, '108.910'), (1, '128.120')] [2023-10-11 22:26:28,551][71601] Updated weights for policy 0, policy_version 85290 (0.0008) [2023-10-11 22:26:28,927][71601] Updated weights for policy 0, policy_version 85300 (0.0010) [2023-10-11 22:26:29,300][71601] Updated weights for policy 0, policy_version 85310 (0.0007) [2023-10-11 22:26:29,585][71635] Updated weights for policy 1, policy_version 85222 (0.0008) [2023-10-11 22:26:29,952][71635] Updated weights for policy 1, policy_version 85232 (0.0009) [2023-10-11 22:26:30,320][71635] Updated weights for policy 1, policy_version 85242 (0.0009) [2023-10-11 22:26:31,034][70582] Fps is (10 sec: 16383.4, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 174653440. Throughput: 0: 1814.2, 1: 1822.6. Samples: 43663466. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 22:26:31,035][70582] Avg episode reward: [(0, '104.350'), (1, '132.340')] [2023-10-11 22:26:32,896][71601] Updated weights for policy 0, policy_version 85320 (0.0009) [2023-10-11 22:26:33,266][71601] Updated weights for policy 0, policy_version 85330 (0.0009) [2023-10-11 22:26:33,646][71601] Updated weights for policy 0, policy_version 85340 (0.0009) [2023-10-11 22:26:34,073][71635] Updated weights for policy 1, policy_version 85252 (0.0008) [2023-10-11 22:26:34,438][71635] Updated weights for policy 1, policy_version 85262 (0.0008) [2023-10-11 22:26:34,814][71635] Updated weights for policy 1, policy_version 85272 (0.0009) [2023-10-11 22:26:36,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 174718976. Throughput: 0: 1812.7, 1: 1817.9. Samples: 43684614. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 22:26:36,035][70582] Avg episode reward: [(0, '111.560'), (1, '126.450')] [2023-10-11 22:26:37,214][71601] Updated weights for policy 0, policy_version 85350 (0.0008) [2023-10-11 22:26:37,587][71601] Updated weights for policy 0, policy_version 85360 (0.0009) [2023-10-11 22:26:37,962][71601] Updated weights for policy 0, policy_version 85370 (0.0009) [2023-10-11 22:26:38,547][71635] Updated weights for policy 1, policy_version 85282 (0.0008) [2023-10-11 22:26:38,915][71635] Updated weights for policy 1, policy_version 85292 (0.0011) [2023-10-11 22:26:39,289][71635] Updated weights for policy 1, policy_version 85302 (0.0010) [2023-10-11 22:26:39,659][71635] Updated weights for policy 1, policy_version 85312 (0.0011) [2023-10-11 22:26:41,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 174784512. Throughput: 0: 1809.0, 1: 1815.6. Samples: 43706624. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 22:26:41,035][70582] Avg episode reward: [(0, '108.430'), (1, '125.310')] [2023-10-11 22:26:41,863][71601] Updated weights for policy 0, policy_version 85380 (0.0008) [2023-10-11 22:26:42,252][71601] Updated weights for policy 0, policy_version 85390 (0.0007) [2023-10-11 22:26:42,623][71601] Updated weights for policy 0, policy_version 85400 (0.0008) [2023-10-11 22:26:43,550][71635] Updated weights for policy 1, policy_version 85322 (0.0009) [2023-10-11 22:26:43,920][71635] Updated weights for policy 1, policy_version 85332 (0.0008) [2023-10-11 22:26:44,289][71635] Updated weights for policy 1, policy_version 85342 (0.0008) [2023-10-11 22:26:46,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 174850048. Throughput: 0: 1809.3, 1: 1821.0. Samples: 43717516. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:26:46,034][70582] Avg episode reward: [(0, '109.370'), (1, '128.820')] [2023-10-11 22:26:46,185][71601] Updated weights for policy 0, policy_version 85410 (0.0007) [2023-10-11 22:26:46,562][71601] Updated weights for policy 0, policy_version 85420 (0.0008) [2023-10-11 22:26:46,940][71601] Updated weights for policy 0, policy_version 85430 (0.0007) [2023-10-11 22:26:47,320][71601] Updated weights for policy 0, policy_version 85440 (0.0009) [2023-10-11 22:26:47,962][71635] Updated weights for policy 1, policy_version 85352 (0.0010) [2023-10-11 22:26:48,337][71635] Updated weights for policy 1, policy_version 85362 (0.0007) [2023-10-11 22:26:48,713][71635] Updated weights for policy 1, policy_version 85372 (0.0007) [2023-10-11 22:26:51,018][71601] Updated weights for policy 0, policy_version 85450 (0.0007) [2023-10-11 22:26:51,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 174915584. Throughput: 0: 1817.7, 1: 1811.5. Samples: 43739030. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:26:51,034][70582] Avg episode reward: [(0, '108.330'), (1, '121.790')] [2023-10-11 22:26:51,392][71601] Updated weights for policy 0, policy_version 85460 (0.0008) [2023-10-11 22:26:51,765][71601] Updated weights for policy 0, policy_version 85470 (0.0010) [2023-10-11 22:26:52,579][71635] Updated weights for policy 1, policy_version 85382 (0.0007) [2023-10-11 22:26:52,945][71635] Updated weights for policy 1, policy_version 85392 (0.0007) [2023-10-11 22:26:53,310][71635] Updated weights for policy 1, policy_version 85402 (0.0008) [2023-10-11 22:26:55,349][71601] Updated weights for policy 0, policy_version 85480 (0.0008) [2023-10-11 22:26:55,728][71601] Updated weights for policy 0, policy_version 85490 (0.0008) [2023-10-11 22:26:56,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 174981120. Throughput: 0: 1817.2, 1: 1805.2. Samples: 43761228. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:26:56,035][70582] Avg episode reward: [(0, '110.320'), (1, '121.400')] [2023-10-11 22:26:56,089][71601] Updated weights for policy 0, policy_version 85500 (0.0007) [2023-10-11 22:26:56,867][71635] Updated weights for policy 1, policy_version 85412 (0.0008) [2023-10-11 22:26:57,244][71635] Updated weights for policy 1, policy_version 85422 (0.0007) [2023-10-11 22:26:57,606][71635] Updated weights for policy 1, policy_version 85432 (0.0010) [2023-10-11 22:26:59,766][71601] Updated weights for policy 0, policy_version 85510 (0.0008) [2023-10-11 22:27:00,142][71601] Updated weights for policy 0, policy_version 85520 (0.0008) [2023-10-11 22:27:00,520][71601] Updated weights for policy 0, policy_version 85530 (0.0009) [2023-10-11 22:27:01,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 175079424. Throughput: 0: 1823.1, 1: 1808.3. Samples: 43771838. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:27:01,035][70582] Avg episode reward: [(0, '112.360'), (1, '121.360')] [2023-10-11 22:27:01,265][71635] Updated weights for policy 1, policy_version 85442 (0.0008) [2023-10-11 22:27:01,631][71635] Updated weights for policy 1, policy_version 85452 (0.0007) [2023-10-11 22:27:01,989][71635] Updated weights for policy 1, policy_version 85462 (0.0009) [2023-10-11 22:27:02,354][71635] Updated weights for policy 1, policy_version 85472 (0.0009) [2023-10-11 22:27:04,306][71601] Updated weights for policy 0, policy_version 85540 (0.0008) [2023-10-11 22:27:04,684][71601] Updated weights for policy 0, policy_version 85550 (0.0007) [2023-10-11 22:27:05,055][71601] Updated weights for policy 0, policy_version 85560 (0.0008) [2023-10-11 22:27:05,857][71635] Updated weights for policy 1, policy_version 85482 (0.0007) [2023-10-11 22:27:06,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 175144960. Throughput: 0: 1826.3, 1: 1811.4. Samples: 43794346. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:27:06,034][70582] Avg episode reward: [(0, '115.170'), (1, '117.770')] [2023-10-11 22:27:06,213][71635] Updated weights for policy 1, policy_version 85492 (0.0007) [2023-10-11 22:27:06,580][71635] Updated weights for policy 1, policy_version 85502 (0.0009) [2023-10-11 22:27:08,862][71601] Updated weights for policy 0, policy_version 85570 (0.0007) [2023-10-11 22:27:09,227][71601] Updated weights for policy 0, policy_version 85580 (0.0009) [2023-10-11 22:27:09,606][71601] Updated weights for policy 0, policy_version 85590 (0.0009) [2023-10-11 22:27:09,971][71601] Updated weights for policy 0, policy_version 85600 (0.0010) [2023-10-11 22:27:10,337][71635] Updated weights for policy 1, policy_version 85512 (0.0009) [2023-10-11 22:27:10,714][71635] Updated weights for policy 1, policy_version 85522 (0.0010) [2023-10-11 22:27:11,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 175210496. Throughput: 0: 1821.8, 1: 1821.7. Samples: 43815692. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:27:11,035][70582] Avg episode reward: [(0, '114.210'), (1, '116.630')] [2023-10-11 22:27:11,081][71635] Updated weights for policy 1, policy_version 85532 (0.0007) [2023-10-11 22:27:13,555][71601] Updated weights for policy 0, policy_version 85610 (0.0010) [2023-10-11 22:27:13,920][71601] Updated weights for policy 0, policy_version 85620 (0.0010) [2023-10-11 22:27:14,294][71601] Updated weights for policy 0, policy_version 85630 (0.0009) [2023-10-11 22:27:14,640][71635] Updated weights for policy 1, policy_version 85542 (0.0008) [2023-10-11 22:27:15,002][71635] Updated weights for policy 1, policy_version 85552 (0.0009) [2023-10-11 22:27:15,362][71635] Updated weights for policy 1, policy_version 85562 (0.0008) [2023-10-11 22:27:16,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 175308800. Throughput: 0: 1823.3, 1: 1818.6. Samples: 43827350. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:27:16,035][70582] Avg episode reward: [(0, '117.160'), (1, '121.490')] [2023-10-11 22:27:17,859][71601] Updated weights for policy 0, policy_version 85640 (0.0008) [2023-10-11 22:27:18,242][71601] Updated weights for policy 0, policy_version 85650 (0.0008) [2023-10-11 22:27:18,610][71601] Updated weights for policy 0, policy_version 85660 (0.0008) [2023-10-11 22:27:19,228][71635] Updated weights for policy 1, policy_version 85572 (0.0009) [2023-10-11 22:27:19,592][71635] Updated weights for policy 1, policy_version 85582 (0.0008) [2023-10-11 22:27:19,951][71635] Updated weights for policy 1, policy_version 85592 (0.0008) [2023-10-11 22:27:21,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 175374336. Throughput: 0: 1823.6, 1: 1820.8. Samples: 43848610. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:27:21,035][70582] Avg episode reward: [(0, '121.230'), (1, '119.330')] [2023-10-11 22:27:22,215][71601] Updated weights for policy 0, policy_version 85670 (0.0009) [2023-10-11 22:27:22,588][71601] Updated weights for policy 0, policy_version 85680 (0.0009) [2023-10-11 22:27:22,959][71601] Updated weights for policy 0, policy_version 85690 (0.0010) [2023-10-11 22:27:23,479][71635] Updated weights for policy 1, policy_version 85602 (0.0008) [2023-10-11 22:27:23,852][71635] Updated weights for policy 1, policy_version 85612 (0.0009) [2023-10-11 22:27:24,220][71635] Updated weights for policy 1, policy_version 85622 (0.0009) [2023-10-11 22:27:24,587][71635] Updated weights for policy 1, policy_version 85632 (0.0007) [2023-10-11 22:27:26,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 175439872. Throughput: 0: 1830.8, 1: 1818.5. Samples: 43870844. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:27:26,034][70582] Avg episode reward: [(0, '120.860'), (1, '112.780')] [2023-10-11 22:27:26,624][71601] Updated weights for policy 0, policy_version 85700 (0.0008) [2023-10-11 22:27:27,002][71601] Updated weights for policy 0, policy_version 85710 (0.0008) [2023-10-11 22:27:27,372][71601] Updated weights for policy 0, policy_version 85720 (0.0008) [2023-10-11 22:27:28,228][71635] Updated weights for policy 1, policy_version 85642 (0.0009) [2023-10-11 22:27:28,609][71635] Updated weights for policy 1, policy_version 85652 (0.0007) [2023-10-11 22:27:28,973][71635] Updated weights for policy 1, policy_version 85662 (0.0008) [2023-10-11 22:27:31,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 175505408. Throughput: 0: 1831.6, 1: 1810.7. Samples: 43881418. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:27:31,034][70582] Avg episode reward: [(0, '116.340'), (1, '116.870')] [2023-10-11 22:27:31,044][71601] Updated weights for policy 0, policy_version 85730 (0.0009) [2023-10-11 22:27:31,412][71601] Updated weights for policy 0, policy_version 85740 (0.0007) [2023-10-11 22:27:31,798][71601] Updated weights for policy 0, policy_version 85750 (0.0007) [2023-10-11 22:27:32,167][71601] Updated weights for policy 0, policy_version 85760 (0.0008) [2023-10-11 22:27:32,729][71635] Updated weights for policy 1, policy_version 85672 (0.0008) [2023-10-11 22:27:33,100][71635] Updated weights for policy 1, policy_version 85682 (0.0009) [2023-10-11 22:27:33,478][71635] Updated weights for policy 1, policy_version 85692 (0.0009) [2023-10-11 22:27:35,817][71601] Updated weights for policy 0, policy_version 85770 (0.0009) [2023-10-11 22:27:36,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 175570944. Throughput: 0: 1830.3, 1: 1819.0. Samples: 43903246. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:27:36,034][70582] Avg episode reward: [(0, '111.150'), (1, '119.460')] [2023-10-11 22:27:36,188][71601] Updated weights for policy 0, policy_version 85780 (0.0008) [2023-10-11 22:27:36,556][71601] Updated weights for policy 0, policy_version 85790 (0.0007) [2023-10-11 22:27:37,237][71635] Updated weights for policy 1, policy_version 85702 (0.0011) [2023-10-11 22:27:37,607][71635] Updated weights for policy 1, policy_version 85712 (0.0010) [2023-10-11 22:27:37,979][71635] Updated weights for policy 1, policy_version 85722 (0.0009) [2023-10-11 22:27:40,337][71601] Updated weights for policy 0, policy_version 85800 (0.0008) [2023-10-11 22:27:40,718][71601] Updated weights for policy 0, policy_version 85810 (0.0008) [2023-10-11 22:27:41,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 175636480. Throughput: 0: 1827.6, 1: 1819.7. Samples: 43925356. Policy #0 lag: (min: 24.0, avg: 48.2, max: 56.0) [2023-10-11 22:27:41,034][70582] Avg episode reward: [(0, '104.960'), (1, '119.240')] [2023-10-11 22:27:41,086][71601] Updated weights for policy 0, policy_version 85820 (0.0007) [2023-10-11 22:27:41,662][71635] Updated weights for policy 1, policy_version 85732 (0.0008) [2023-10-11 22:27:42,034][71635] Updated weights for policy 1, policy_version 85742 (0.0010) [2023-10-11 22:27:42,391][71635] Updated weights for policy 1, policy_version 85752 (0.0007) [2023-10-11 22:27:44,679][71601] Updated weights for policy 0, policy_version 85830 (0.0009) [2023-10-11 22:27:45,060][71601] Updated weights for policy 0, policy_version 85840 (0.0010) [2023-10-11 22:27:45,437][71601] Updated weights for policy 0, policy_version 85850 (0.0010) [2023-10-11 22:27:45,946][71635] Updated weights for policy 1, policy_version 85762 (0.0008) [2023-10-11 22:27:46,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 175734784. Throughput: 0: 1827.8, 1: 1818.6. Samples: 43935926. Policy #0 lag: (min: 24.0, avg: 48.2, max: 56.0) [2023-10-11 22:27:46,034][70582] Avg episode reward: [(0, '104.740'), (1, '119.960')] [2023-10-11 22:27:46,315][71635] Updated weights for policy 1, policy_version 85772 (0.0007) [2023-10-11 22:27:46,690][71635] Updated weights for policy 1, policy_version 85782 (0.0008) [2023-10-11 22:27:47,054][71635] Updated weights for policy 1, policy_version 85792 (0.0009) [2023-10-11 22:27:49,052][71601] Updated weights for policy 0, policy_version 85860 (0.0009) [2023-10-11 22:27:49,426][71601] Updated weights for policy 0, policy_version 85870 (0.0008) [2023-10-11 22:27:49,801][71601] Updated weights for policy 0, policy_version 85880 (0.0008) [2023-10-11 22:27:50,918][71635] Updated weights for policy 1, policy_version 85802 (0.0008) [2023-10-11 22:27:51,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 175800320. Throughput: 0: 1821.1, 1: 1815.2. Samples: 43957976. Policy #0 lag: (min: 24.0, avg: 48.2, max: 56.0) [2023-10-11 22:27:51,034][70582] Avg episode reward: [(0, '102.690'), (1, '122.040')] [2023-10-11 22:27:51,286][71635] Updated weights for policy 1, policy_version 85812 (0.0007) [2023-10-11 22:27:51,652][71635] Updated weights for policy 1, policy_version 85822 (0.0010) [2023-10-11 22:27:53,162][71601] Updated weights for policy 0, policy_version 85890 (0.0008) [2023-10-11 22:27:53,536][71601] Updated weights for policy 0, policy_version 85900 (0.0009) [2023-10-11 22:27:53,905][71601] Updated weights for policy 0, policy_version 85910 (0.0009) [2023-10-11 22:27:54,275][71601] Updated weights for policy 0, policy_version 85920 (0.0010) [2023-10-11 22:27:55,330][71635] Updated weights for policy 1, policy_version 85832 (0.0010) [2023-10-11 22:27:55,694][71635] Updated weights for policy 1, policy_version 85842 (0.0009) [2023-10-11 22:27:56,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 175865856. Throughput: 0: 1837.1, 1: 1806.5. Samples: 43979654. Policy #0 lag: (min: 24.0, avg: 48.2, max: 56.0) [2023-10-11 22:27:56,034][70582] Avg episode reward: [(0, '102.980'), (1, '121.620')] [2023-10-11 22:27:56,043][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000085920_87982080.pth... [2023-10-11 22:27:56,070][71635] Updated weights for policy 1, policy_version 85852 (0.0009) [2023-10-11 22:27:56,081][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000084224_86245376.pth [2023-10-11 22:27:56,212][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000085856_87916544.pth... [2023-10-11 22:27:56,240][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000084160_86179840.pth [2023-10-11 22:27:58,042][71601] Updated weights for policy 0, policy_version 85930 (0.0007) [2023-10-11 22:27:58,408][71601] Updated weights for policy 0, policy_version 85940 (0.0009) [2023-10-11 22:27:58,790][71601] Updated weights for policy 0, policy_version 85950 (0.0009) [2023-10-11 22:27:59,879][71635] Updated weights for policy 1, policy_version 85862 (0.0007) [2023-10-11 22:28:00,238][71635] Updated weights for policy 1, policy_version 85872 (0.0008) [2023-10-11 22:28:00,601][71635] Updated weights for policy 1, policy_version 85882 (0.0010) [2023-10-11 22:28:01,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 175964160. Throughput: 0: 1820.6, 1: 1802.9. Samples: 43990404. Policy #0 lag: (min: 24.0, avg: 48.2, max: 56.0) [2023-10-11 22:28:01,034][70582] Avg episode reward: [(0, '102.930'), (1, '119.970')] [2023-10-11 22:28:02,449][71601] Updated weights for policy 0, policy_version 85960 (0.0009) [2023-10-11 22:28:02,832][71601] Updated weights for policy 0, policy_version 85970 (0.0008) [2023-10-11 22:28:03,200][71601] Updated weights for policy 0, policy_version 85980 (0.0011) [2023-10-11 22:28:04,325][71635] Updated weights for policy 1, policy_version 85892 (0.0009) [2023-10-11 22:28:04,688][71635] Updated weights for policy 1, policy_version 85902 (0.0008) [2023-10-11 22:28:05,060][71635] Updated weights for policy 1, policy_version 85912 (0.0009) [2023-10-11 22:28:06,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 176029696. Throughput: 0: 1830.6, 1: 1808.9. Samples: 44012388. Policy #0 lag: (min: 24.0, avg: 48.2, max: 56.0) [2023-10-11 22:28:06,034][70582] Avg episode reward: [(0, '110.600'), (1, '120.920')] [2023-10-11 22:28:07,069][71601] Updated weights for policy 0, policy_version 85990 (0.0010) [2023-10-11 22:28:07,452][71601] Updated weights for policy 0, policy_version 86000 (0.0008) [2023-10-11 22:28:07,827][71601] Updated weights for policy 0, policy_version 86010 (0.0008) [2023-10-11 22:28:08,719][71635] Updated weights for policy 1, policy_version 85922 (0.0007) [2023-10-11 22:28:09,082][71635] Updated weights for policy 1, policy_version 85932 (0.0007) [2023-10-11 22:28:09,444][71635] Updated weights for policy 1, policy_version 85942 (0.0008) [2023-10-11 22:28:09,808][71635] Updated weights for policy 1, policy_version 85952 (0.0007) [2023-10-11 22:28:11,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 176095232. Throughput: 0: 1814.4, 1: 1807.8. Samples: 44033846. Policy #0 lag: (min: 24.0, avg: 48.2, max: 56.0) [2023-10-11 22:28:11,035][70582] Avg episode reward: [(0, '110.870'), (1, '115.540')] [2023-10-11 22:28:11,604][71601] Updated weights for policy 0, policy_version 86020 (0.0007) [2023-10-11 22:28:11,972][71601] Updated weights for policy 0, policy_version 86030 (0.0007) [2023-10-11 22:28:12,347][71601] Updated weights for policy 0, policy_version 86040 (0.0008) [2023-10-11 22:28:13,566][71635] Updated weights for policy 1, policy_version 85962 (0.0008) [2023-10-11 22:28:13,934][71635] Updated weights for policy 1, policy_version 85972 (0.0007) [2023-10-11 22:28:14,303][71635] Updated weights for policy 1, policy_version 85982 (0.0008) [2023-10-11 22:28:15,926][71601] Updated weights for policy 0, policy_version 86050 (0.0008) [2023-10-11 22:28:16,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 176160768. Throughput: 0: 1816.0, 1: 1819.5. Samples: 44045016. Policy #0 lag: (min: 24.0, avg: 48.2, max: 56.0) [2023-10-11 22:28:16,034][70582] Avg episode reward: [(0, '108.830'), (1, '113.990')] [2023-10-11 22:28:16,304][71601] Updated weights for policy 0, policy_version 86060 (0.0007) [2023-10-11 22:28:16,666][71601] Updated weights for policy 0, policy_version 86070 (0.0008) [2023-10-11 22:28:17,037][71601] Updated weights for policy 0, policy_version 86080 (0.0009) [2023-10-11 22:28:18,018][71635] Updated weights for policy 1, policy_version 85992 (0.0010) [2023-10-11 22:28:18,379][71635] Updated weights for policy 1, policy_version 86002 (0.0009) [2023-10-11 22:28:18,746][71635] Updated weights for policy 1, policy_version 86012 (0.0008) [2023-10-11 22:28:20,696][71601] Updated weights for policy 0, policy_version 86090 (0.0010) [2023-10-11 22:28:21,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 176226304. Throughput: 0: 1820.8, 1: 1812.4. Samples: 44066742. Policy #0 lag: (min: 24.0, avg: 48.2, max: 56.0) [2023-10-11 22:28:21,034][70582] Avg episode reward: [(0, '112.550'), (1, '112.000')] [2023-10-11 22:28:21,070][71601] Updated weights for policy 0, policy_version 86100 (0.0011) [2023-10-11 22:28:21,445][71601] Updated weights for policy 0, policy_version 86110 (0.0010) [2023-10-11 22:28:22,542][71635] Updated weights for policy 1, policy_version 86022 (0.0010) [2023-10-11 22:28:22,915][71635] Updated weights for policy 1, policy_version 86032 (0.0009) [2023-10-11 22:28:23,285][71635] Updated weights for policy 1, policy_version 86042 (0.0008) [2023-10-11 22:28:25,002][71601] Updated weights for policy 0, policy_version 86120 (0.0009) [2023-10-11 22:28:25,367][71601] Updated weights for policy 0, policy_version 86130 (0.0008) [2023-10-11 22:28:25,742][71601] Updated weights for policy 0, policy_version 86140 (0.0008) [2023-10-11 22:28:26,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 176324608. Throughput: 0: 1817.9, 1: 1818.9. Samples: 44089012. Policy #0 lag: (min: 24.0, avg: 48.2, max: 56.0) [2023-10-11 22:28:26,035][70582] Avg episode reward: [(0, '114.990'), (1, '110.630')] [2023-10-11 22:28:26,853][71635] Updated weights for policy 1, policy_version 86052 (0.0010) [2023-10-11 22:28:27,231][71635] Updated weights for policy 1, policy_version 86062 (0.0008) [2023-10-11 22:28:27,586][71635] Updated weights for policy 1, policy_version 86072 (0.0008) [2023-10-11 22:28:29,473][71601] Updated weights for policy 0, policy_version 86150 (0.0008) [2023-10-11 22:28:29,847][71601] Updated weights for policy 0, policy_version 86160 (0.0010) [2023-10-11 22:28:30,217][71601] Updated weights for policy 0, policy_version 86170 (0.0008) [2023-10-11 22:28:31,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 176390144. Throughput: 0: 1827.2, 1: 1818.5. Samples: 44099982. Policy #0 lag: (min: 26.0, avg: 26.4, max: 37.0) [2023-10-11 22:28:31,035][70582] Avg episode reward: [(0, '110.360'), (1, '106.310')] [2023-10-11 22:28:31,360][71635] Updated weights for policy 1, policy_version 86082 (0.0009) [2023-10-11 22:28:31,727][71635] Updated weights for policy 1, policy_version 86092 (0.0009) [2023-10-11 22:28:32,085][71635] Updated weights for policy 1, policy_version 86102 (0.0009) [2023-10-11 22:28:32,454][71635] Updated weights for policy 1, policy_version 86112 (0.0008) [2023-10-11 22:28:33,839][71601] Updated weights for policy 0, policy_version 86180 (0.0010) [2023-10-11 22:28:34,201][71601] Updated weights for policy 0, policy_version 86190 (0.0010) [2023-10-11 22:28:34,576][71601] Updated weights for policy 0, policy_version 86200 (0.0008) [2023-10-11 22:28:36,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 176455680. Throughput: 0: 1822.7, 1: 1814.6. Samples: 44121654. Policy #0 lag: (min: 26.0, avg: 26.4, max: 37.0) [2023-10-11 22:28:36,034][70582] Avg episode reward: [(0, '117.730'), (1, '104.740')] [2023-10-11 22:28:36,100][71635] Updated weights for policy 1, policy_version 86122 (0.0008) [2023-10-11 22:28:36,472][71635] Updated weights for policy 1, policy_version 86132 (0.0007) [2023-10-11 22:28:36,839][71635] Updated weights for policy 1, policy_version 86142 (0.0009) [2023-10-11 22:28:38,326][71601] Updated weights for policy 0, policy_version 86210 (0.0010) [2023-10-11 22:28:38,698][71601] Updated weights for policy 0, policy_version 86220 (0.0008) [2023-10-11 22:28:39,073][71601] Updated weights for policy 0, policy_version 86230 (0.0008) [2023-10-11 22:28:39,438][71601] Updated weights for policy 0, policy_version 86240 (0.0008) [2023-10-11 22:28:40,514][71635] Updated weights for policy 1, policy_version 86152 (0.0008) [2023-10-11 22:28:40,877][71635] Updated weights for policy 1, policy_version 86162 (0.0008) [2023-10-11 22:28:41,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 176521216. Throughput: 0: 1820.0, 1: 1826.1. Samples: 44143732. Policy #0 lag: (min: 26.0, avg: 26.4, max: 37.0) [2023-10-11 22:28:41,035][70582] Avg episode reward: [(0, '117.700'), (1, '100.440')] [2023-10-11 22:28:41,246][71635] Updated weights for policy 1, policy_version 86172 (0.0009) [2023-10-11 22:28:43,199][71601] Updated weights for policy 0, policy_version 86250 (0.0007) [2023-10-11 22:28:43,578][71601] Updated weights for policy 0, policy_version 86260 (0.0007) [2023-10-11 22:28:43,956][71601] Updated weights for policy 0, policy_version 86270 (0.0007) [2023-10-11 22:28:45,030][71635] Updated weights for policy 1, policy_version 86182 (0.0009) [2023-10-11 22:28:45,399][71635] Updated weights for policy 1, policy_version 86192 (0.0007) [2023-10-11 22:28:45,752][71635] Updated weights for policy 1, policy_version 86202 (0.0008) [2023-10-11 22:28:46,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 176619520. Throughput: 0: 1828.3, 1: 1819.1. Samples: 44154536. Policy #0 lag: (min: 26.0, avg: 26.4, max: 37.0) [2023-10-11 22:28:46,034][70582] Avg episode reward: [(0, '118.390'), (1, '96.700')] [2023-10-11 22:28:47,643][71601] Updated weights for policy 0, policy_version 86280 (0.0010) [2023-10-11 22:28:48,013][71601] Updated weights for policy 0, policy_version 86290 (0.0011) [2023-10-11 22:28:48,385][71601] Updated weights for policy 0, policy_version 86300 (0.0009) [2023-10-11 22:28:49,580][71635] Updated weights for policy 1, policy_version 86212 (0.0007) [2023-10-11 22:28:49,946][71635] Updated weights for policy 1, policy_version 86222 (0.0009) [2023-10-11 22:28:50,311][71635] Updated weights for policy 1, policy_version 86232 (0.0007) [2023-10-11 22:28:51,034][70582] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 176685056. Throughput: 0: 1819.5, 1: 1819.5. Samples: 44176142. Policy #0 lag: (min: 26.0, avg: 26.4, max: 37.0) [2023-10-11 22:28:51,034][70582] Avg episode reward: [(0, '119.100'), (1, '97.750')] [2023-10-11 22:28:52,297][71601] Updated weights for policy 0, policy_version 86310 (0.0008) [2023-10-11 22:28:52,661][71601] Updated weights for policy 0, policy_version 86320 (0.0008) [2023-10-11 22:28:53,032][71601] Updated weights for policy 0, policy_version 86330 (0.0008) [2023-10-11 22:28:54,080][71635] Updated weights for policy 1, policy_version 86242 (0.0008) [2023-10-11 22:28:54,442][71635] Updated weights for policy 1, policy_version 86252 (0.0008) [2023-10-11 22:28:54,804][71635] Updated weights for policy 1, policy_version 86262 (0.0009) [2023-10-11 22:28:55,171][71635] Updated weights for policy 1, policy_version 86272 (0.0009) [2023-10-11 22:28:56,034][70582] Fps is (10 sec: 13106.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 176750592. Throughput: 0: 1822.6, 1: 1810.5. Samples: 44197336. Policy #0 lag: (min: 26.0, avg: 26.4, max: 37.0) [2023-10-11 22:28:56,035][70582] Avg episode reward: [(0, '115.930'), (1, '103.460')] [2023-10-11 22:28:56,771][71601] Updated weights for policy 0, policy_version 86340 (0.0007) [2023-10-11 22:28:57,158][71601] Updated weights for policy 0, policy_version 86350 (0.0008) [2023-10-11 22:28:57,536][71601] Updated weights for policy 0, policy_version 86360 (0.0009) [2023-10-11 22:28:58,960][71635] Updated weights for policy 1, policy_version 86282 (0.0010) [2023-10-11 22:28:59,335][71635] Updated weights for policy 1, policy_version 86292 (0.0010) [2023-10-11 22:28:59,707][71635] Updated weights for policy 1, policy_version 86302 (0.0009) [2023-10-11 22:29:01,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 176816128. Throughput: 0: 1819.2, 1: 1816.4. Samples: 44208618. Policy #0 lag: (min: 26.0, avg: 26.4, max: 37.0) [2023-10-11 22:29:01,035][70582] Avg episode reward: [(0, '112.300'), (1, '101.900')] [2023-10-11 22:29:01,116][71601] Updated weights for policy 0, policy_version 86370 (0.0010) [2023-10-11 22:29:01,505][71601] Updated weights for policy 0, policy_version 86380 (0.0009) [2023-10-11 22:29:01,876][71601] Updated weights for policy 0, policy_version 86390 (0.0009) [2023-10-11 22:29:02,239][71601] Updated weights for policy 0, policy_version 86400 (0.0009) [2023-10-11 22:29:03,334][71635] Updated weights for policy 1, policy_version 86312 (0.0010) [2023-10-11 22:29:03,694][71635] Updated weights for policy 1, policy_version 86322 (0.0008) [2023-10-11 22:29:04,063][71635] Updated weights for policy 1, policy_version 86332 (0.0009) [2023-10-11 22:29:05,810][71601] Updated weights for policy 0, policy_version 86410 (0.0007) [2023-10-11 22:29:06,034][70582] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 176881664. Throughput: 0: 1815.8, 1: 1809.0. Samples: 44229858. Policy #0 lag: (min: 26.0, avg: 26.4, max: 37.0) [2023-10-11 22:29:06,034][70582] Avg episode reward: [(0, '111.990'), (1, '104.230')] [2023-10-11 22:29:06,193][71601] Updated weights for policy 0, policy_version 86420 (0.0008) [2023-10-11 22:29:06,560][71601] Updated weights for policy 0, policy_version 86430 (0.0007) [2023-10-11 22:29:07,734][71635] Updated weights for policy 1, policy_version 86342 (0.0008) [2023-10-11 22:29:08,101][71635] Updated weights for policy 1, policy_version 86352 (0.0008) [2023-10-11 22:29:08,473][71635] Updated weights for policy 1, policy_version 86362 (0.0007) [2023-10-11 22:29:10,346][71601] Updated weights for policy 0, policy_version 86440 (0.0008) [2023-10-11 22:29:10,715][71601] Updated weights for policy 0, policy_version 86450 (0.0009) [2023-10-11 22:29:11,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 176947200. Throughput: 0: 1819.7, 1: 1801.7. Samples: 44251974. Policy #0 lag: (min: 26.0, avg: 26.4, max: 37.0) [2023-10-11 22:29:11,034][70582] Avg episode reward: [(0, '111.070'), (1, '105.860')] [2023-10-11 22:29:11,079][71601] Updated weights for policy 0, policy_version 86460 (0.0009) [2023-10-11 22:29:12,134][71635] Updated weights for policy 1, policy_version 86372 (0.0009) [2023-10-11 22:29:12,503][71635] Updated weights for policy 1, policy_version 86382 (0.0010) [2023-10-11 22:29:12,875][71635] Updated weights for policy 1, policy_version 86392 (0.0010) [2023-10-11 22:29:14,829][71601] Updated weights for policy 0, policy_version 86470 (0.0007) [2023-10-11 22:29:15,190][71601] Updated weights for policy 0, policy_version 86480 (0.0008) [2023-10-11 22:29:15,569][71601] Updated weights for policy 0, policy_version 86490 (0.0008) [2023-10-11 22:29:16,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 177045504. Throughput: 0: 1807.5, 1: 1803.4. Samples: 44262472. Policy #0 lag: (min: 26.0, avg: 26.4, max: 37.0) [2023-10-11 22:29:16,035][70582] Avg episode reward: [(0, '107.660'), (1, '109.370')] [2023-10-11 22:29:16,609][71635] Updated weights for policy 1, policy_version 86402 (0.0008) [2023-10-11 22:29:16,978][71635] Updated weights for policy 1, policy_version 86412 (0.0008) [2023-10-11 22:29:17,351][71635] Updated weights for policy 1, policy_version 86422 (0.0007) [2023-10-11 22:29:17,709][71635] Updated weights for policy 1, policy_version 86432 (0.0009) [2023-10-11 22:29:19,252][71601] Updated weights for policy 0, policy_version 86500 (0.0009) [2023-10-11 22:29:19,614][71601] Updated weights for policy 0, policy_version 86510 (0.0007) [2023-10-11 22:29:19,989][71601] Updated weights for policy 0, policy_version 86520 (0.0009) [2023-10-11 22:29:21,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 177111040. Throughput: 0: 1813.9, 1: 1809.5. Samples: 44284704. Policy #0 lag: (min: 26.0, avg: 26.4, max: 37.0) [2023-10-11 22:29:21,034][70582] Avg episode reward: [(0, '103.660'), (1, '106.640')] [2023-10-11 22:29:21,323][71635] Updated weights for policy 1, policy_version 86442 (0.0009) [2023-10-11 22:29:21,693][71635] Updated weights for policy 1, policy_version 86452 (0.0008) [2023-10-11 22:29:22,058][71635] Updated weights for policy 1, policy_version 86462 (0.0009) [2023-10-11 22:29:23,651][71601] Updated weights for policy 0, policy_version 86530 (0.0007) [2023-10-11 22:29:24,015][71601] Updated weights for policy 0, policy_version 86540 (0.0009) [2023-10-11 22:29:24,394][71601] Updated weights for policy 0, policy_version 86550 (0.0008) [2023-10-11 22:29:24,759][71601] Updated weights for policy 0, policy_version 86560 (0.0007) [2023-10-11 22:29:25,671][71635] Updated weights for policy 1, policy_version 86472 (0.0008) [2023-10-11 22:29:26,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 177176576. Throughput: 0: 1809.2, 1: 1815.2. Samples: 44306828. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-11 22:29:26,035][70582] Avg episode reward: [(0, '104.650'), (1, '108.400')] [2023-10-11 22:29:26,038][71635] Updated weights for policy 1, policy_version 86482 (0.0008) [2023-10-11 22:29:26,406][71635] Updated weights for policy 1, policy_version 86492 (0.0008) [2023-10-11 22:29:28,236][71601] Updated weights for policy 0, policy_version 86570 (0.0007) [2023-10-11 22:29:28,604][71601] Updated weights for policy 0, policy_version 86580 (0.0007) [2023-10-11 22:29:28,966][71601] Updated weights for policy 0, policy_version 86590 (0.0009) [2023-10-11 22:29:30,099][71635] Updated weights for policy 1, policy_version 86502 (0.0008) [2023-10-11 22:29:30,457][71635] Updated weights for policy 1, policy_version 86512 (0.0009) [2023-10-11 22:29:30,829][71635] Updated weights for policy 1, policy_version 86522 (0.0007) [2023-10-11 22:29:31,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 177242112. Throughput: 0: 1814.6, 1: 1813.5. Samples: 44317800. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-11 22:29:31,034][70582] Avg episode reward: [(0, '101.860'), (1, '113.290')] [2023-10-11 22:29:32,676][71601] Updated weights for policy 0, policy_version 86600 (0.0008) [2023-10-11 22:29:33,052][71601] Updated weights for policy 0, policy_version 86610 (0.0007) [2023-10-11 22:29:33,434][71601] Updated weights for policy 0, policy_version 86620 (0.0007) [2023-10-11 22:29:34,416][71635] Updated weights for policy 1, policy_version 86532 (0.0012) [2023-10-11 22:29:34,779][71635] Updated weights for policy 1, policy_version 86542 (0.0009) [2023-10-11 22:29:35,149][71635] Updated weights for policy 1, policy_version 86552 (0.0008) [2023-10-11 22:29:36,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 177340416. Throughput: 0: 1815.0, 1: 1823.1. Samples: 44339858. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-11 22:29:36,035][70582] Avg episode reward: [(0, '104.800'), (1, '115.920')] [2023-10-11 22:29:37,090][71601] Updated weights for policy 0, policy_version 86630 (0.0009) [2023-10-11 22:29:37,462][71601] Updated weights for policy 0, policy_version 86640 (0.0008) [2023-10-11 22:29:37,822][71601] Updated weights for policy 0, policy_version 86650 (0.0009) [2023-10-11 22:29:38,903][71635] Updated weights for policy 1, policy_version 86562 (0.0010) [2023-10-11 22:29:39,280][71635] Updated weights for policy 1, policy_version 86572 (0.0009) [2023-10-11 22:29:39,634][71635] Updated weights for policy 1, policy_version 86582 (0.0007) [2023-10-11 22:29:40,000][71635] Updated weights for policy 1, policy_version 86592 (0.0008) [2023-10-11 22:29:41,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 177405952. Throughput: 0: 1827.3, 1: 1827.6. Samples: 44361806. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-11 22:29:41,035][70582] Avg episode reward: [(0, '107.970'), (1, '119.250')] [2023-10-11 22:29:41,534][71601] Updated weights for policy 0, policy_version 86660 (0.0009) [2023-10-11 22:29:41,915][71601] Updated weights for policy 0, policy_version 86670 (0.0010) [2023-10-11 22:29:42,288][71601] Updated weights for policy 0, policy_version 86680 (0.0009) [2023-10-11 22:29:43,707][71635] Updated weights for policy 1, policy_version 86602 (0.0009) [2023-10-11 22:29:44,077][71635] Updated weights for policy 1, policy_version 86612 (0.0009) [2023-10-11 22:29:44,456][71635] Updated weights for policy 1, policy_version 86622 (0.0009) [2023-10-11 22:29:46,001][71601] Updated weights for policy 0, policy_version 86690 (0.0007) [2023-10-11 22:29:46,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 177471488. Throughput: 0: 1828.3, 1: 1824.8. Samples: 44373008. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-11 22:29:46,035][70582] Avg episode reward: [(0, '112.110'), (1, '122.840')] [2023-10-11 22:29:46,379][71601] Updated weights for policy 0, policy_version 86700 (0.0009) [2023-10-11 22:29:46,738][71601] Updated weights for policy 0, policy_version 86710 (0.0008) [2023-10-11 22:29:47,113][71601] Updated weights for policy 0, policy_version 86720 (0.0008) [2023-10-11 22:29:48,106][71635] Updated weights for policy 1, policy_version 86632 (0.0007) [2023-10-11 22:29:48,471][71635] Updated weights for policy 1, policy_version 86642 (0.0007) [2023-10-11 22:29:48,839][71635] Updated weights for policy 1, policy_version 86652 (0.0007) [2023-10-11 22:29:50,740][71601] Updated weights for policy 0, policy_version 86730 (0.0009) [2023-10-11 22:29:51,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 177537024. Throughput: 0: 1821.9, 1: 1827.7. Samples: 44394088. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-11 22:29:51,034][70582] Avg episode reward: [(0, '106.470'), (1, '116.390')] [2023-10-11 22:29:51,117][71601] Updated weights for policy 0, policy_version 86740 (0.0008) [2023-10-11 22:29:51,492][71601] Updated weights for policy 0, policy_version 86750 (0.0008) [2023-10-11 22:29:52,591][71635] Updated weights for policy 1, policy_version 86662 (0.0008) [2023-10-11 22:29:52,961][71635] Updated weights for policy 1, policy_version 86672 (0.0008) [2023-10-11 22:29:53,336][71635] Updated weights for policy 1, policy_version 86682 (0.0007) [2023-10-11 22:29:55,219][71601] Updated weights for policy 0, policy_version 86760 (0.0008) [2023-10-11 22:29:55,588][71601] Updated weights for policy 0, policy_version 86770 (0.0009) [2023-10-11 22:29:55,962][71601] Updated weights for policy 0, policy_version 86780 (0.0008) [2023-10-11 22:29:56,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 177602560. Throughput: 0: 1820.0, 1: 1832.4. Samples: 44416332. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-11 22:29:56,035][70582] Avg episode reward: [(0, '112.590'), (1, '120.060')] [2023-10-11 22:29:56,045][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000086688_88768512.pth... [2023-10-11 22:29:56,079][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000084992_87031808.pth [2023-10-11 22:29:56,108][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000086784_88866816.pth... [2023-10-11 22:29:56,137][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000085056_87097344.pth [2023-10-11 22:29:57,020][71635] Updated weights for policy 1, policy_version 86692 (0.0009) [2023-10-11 22:29:57,395][71635] Updated weights for policy 1, policy_version 86702 (0.0007) [2023-10-11 22:29:57,753][71635] Updated weights for policy 1, policy_version 86712 (0.0007) [2023-10-11 22:29:59,706][71601] Updated weights for policy 0, policy_version 86790 (0.0008) [2023-10-11 22:30:00,071][71601] Updated weights for policy 0, policy_version 86800 (0.0008) [2023-10-11 22:30:00,442][71601] Updated weights for policy 0, policy_version 86810 (0.0010) [2023-10-11 22:30:01,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 177700864. Throughput: 0: 1822.9, 1: 1828.7. Samples: 44426792. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-11 22:30:01,034][70582] Avg episode reward: [(0, '114.300'), (1, '116.890')] [2023-10-11 22:30:01,382][71635] Updated weights for policy 1, policy_version 86722 (0.0009) [2023-10-11 22:30:01,752][71635] Updated weights for policy 1, policy_version 86732 (0.0008) [2023-10-11 22:30:02,118][71635] Updated weights for policy 1, policy_version 86742 (0.0009) [2023-10-11 22:30:02,484][71635] Updated weights for policy 1, policy_version 86752 (0.0008) [2023-10-11 22:30:04,043][71601] Updated weights for policy 0, policy_version 86820 (0.0008) [2023-10-11 22:30:04,414][71601] Updated weights for policy 0, policy_version 86830 (0.0008) [2023-10-11 22:30:04,783][71601] Updated weights for policy 0, policy_version 86840 (0.0008) [2023-10-11 22:30:06,034][70582] Fps is (10 sec: 16384.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 177766400. Throughput: 0: 1823.0, 1: 1829.9. Samples: 44449082. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-11 22:30:06,034][70582] Avg episode reward: [(0, '114.960'), (1, '108.710')] [2023-10-11 22:30:06,185][71635] Updated weights for policy 1, policy_version 86762 (0.0007) [2023-10-11 22:30:06,546][71635] Updated weights for policy 1, policy_version 86772 (0.0009) [2023-10-11 22:30:06,908][71635] Updated weights for policy 1, policy_version 86782 (0.0010) [2023-10-11 22:30:08,369][71601] Updated weights for policy 0, policy_version 86850 (0.0010) [2023-10-11 22:30:08,744][71601] Updated weights for policy 0, policy_version 86860 (0.0009) [2023-10-11 22:30:09,127][71601] Updated weights for policy 0, policy_version 86870 (0.0009) [2023-10-11 22:30:09,502][71601] Updated weights for policy 0, policy_version 86880 (0.0008) [2023-10-11 22:30:10,694][71635] Updated weights for policy 1, policy_version 86792 (0.0009) [2023-10-11 22:30:11,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 177831936. Throughput: 0: 1826.1, 1: 1825.0. Samples: 44471128. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-11 22:30:11,035][70582] Avg episode reward: [(0, '111.190'), (1, '113.200')] [2023-10-11 22:30:11,058][71635] Updated weights for policy 1, policy_version 86802 (0.0008) [2023-10-11 22:30:11,434][71635] Updated weights for policy 1, policy_version 86812 (0.0008) [2023-10-11 22:30:13,194][71601] Updated weights for policy 0, policy_version 86890 (0.0007) [2023-10-11 22:30:13,573][71601] Updated weights for policy 0, policy_version 86900 (0.0007) [2023-10-11 22:30:13,953][71601] Updated weights for policy 0, policy_version 86910 (0.0007) [2023-10-11 22:30:15,114][71635] Updated weights for policy 1, policy_version 86822 (0.0009) [2023-10-11 22:30:15,488][71635] Updated weights for policy 1, policy_version 86832 (0.0010) [2023-10-11 22:30:15,850][71635] Updated weights for policy 1, policy_version 86842 (0.0009) [2023-10-11 22:30:16,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 177897472. Throughput: 0: 1824.5, 1: 1825.2. Samples: 44482034. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-11 22:30:16,034][70582] Avg episode reward: [(0, '112.200'), (1, '113.540')] [2023-10-11 22:30:17,575][71601] Updated weights for policy 0, policy_version 86920 (0.0008) [2023-10-11 22:30:17,939][71601] Updated weights for policy 0, policy_version 86930 (0.0009) [2023-10-11 22:30:18,307][71601] Updated weights for policy 0, policy_version 86940 (0.0010) [2023-10-11 22:30:19,533][71635] Updated weights for policy 1, policy_version 86852 (0.0009) [2023-10-11 22:30:19,901][71635] Updated weights for policy 1, policy_version 86862 (0.0010) [2023-10-11 22:30:20,269][71635] Updated weights for policy 1, policy_version 86872 (0.0010) [2023-10-11 22:30:21,034][70582] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 177995776. Throughput: 0: 1830.9, 1: 1822.2. Samples: 44504248. Policy #0 lag: (min: 6.0, avg: 13.7, max: 38.0) [2023-10-11 22:30:21,034][70582] Avg episode reward: [(0, '121.130'), (1, '111.620')] [2023-10-11 22:30:21,882][71601] Updated weights for policy 0, policy_version 86950 (0.0007) [2023-10-11 22:30:22,260][71601] Updated weights for policy 0, policy_version 86960 (0.0007) [2023-10-11 22:30:22,622][71601] Updated weights for policy 0, policy_version 86970 (0.0009) [2023-10-11 22:30:23,771][71635] Updated weights for policy 1, policy_version 86882 (0.0009) [2023-10-11 22:30:24,127][71635] Updated weights for policy 1, policy_version 86892 (0.0011) [2023-10-11 22:30:24,502][71635] Updated weights for policy 1, policy_version 86902 (0.0010) [2023-10-11 22:30:24,871][71635] Updated weights for policy 1, policy_version 86912 (0.0010) [2023-10-11 22:30:26,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 178061312. Throughput: 0: 1825.7, 1: 1819.8. Samples: 44525856. Policy #0 lag: (min: 6.0, avg: 13.7, max: 38.0) [2023-10-11 22:30:26,034][70582] Avg episode reward: [(0, '123.460'), (1, '105.820')] [2023-10-11 22:30:26,502][71601] Updated weights for policy 0, policy_version 86980 (0.0009) [2023-10-11 22:30:26,889][71601] Updated weights for policy 0, policy_version 86990 (0.0011) [2023-10-11 22:30:27,250][71601] Updated weights for policy 0, policy_version 87000 (0.0010) [2023-10-11 22:30:28,578][71635] Updated weights for policy 1, policy_version 86922 (0.0007) [2023-10-11 22:30:28,940][71635] Updated weights for policy 1, policy_version 86932 (0.0007) [2023-10-11 22:30:29,300][71635] Updated weights for policy 1, policy_version 86942 (0.0008) [2023-10-11 22:30:30,863][71601] Updated weights for policy 0, policy_version 87010 (0.0009) [2023-10-11 22:30:31,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 178126848. Throughput: 0: 1825.4, 1: 1818.6. Samples: 44536988. Policy #0 lag: (min: 6.0, avg: 13.7, max: 38.0) [2023-10-11 22:30:31,034][70582] Avg episode reward: [(0, '127.490'), (1, '103.510')] [2023-10-11 22:30:31,221][71601] Updated weights for policy 0, policy_version 87020 (0.0007) [2023-10-11 22:30:31,595][71601] Updated weights for policy 0, policy_version 87030 (0.0007) [2023-10-11 22:30:31,959][71601] Updated weights for policy 0, policy_version 87040 (0.0008) [2023-10-11 22:30:33,010][71635] Updated weights for policy 1, policy_version 86952 (0.0007) [2023-10-11 22:30:33,378][71635] Updated weights for policy 1, policy_version 86962 (0.0010) [2023-10-11 22:30:33,738][71635] Updated weights for policy 1, policy_version 86972 (0.0009) [2023-10-11 22:30:35,509][71601] Updated weights for policy 0, policy_version 87050 (0.0010) [2023-10-11 22:30:35,867][71601] Updated weights for policy 0, policy_version 87060 (0.0007) [2023-10-11 22:30:36,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 178192384. Throughput: 0: 1831.9, 1: 1821.1. Samples: 44558472. Policy #0 lag: (min: 6.0, avg: 13.7, max: 38.0) [2023-10-11 22:30:36,035][70582] Avg episode reward: [(0, '127.670'), (1, '106.160')] [2023-10-11 22:30:36,233][71601] Updated weights for policy 0, policy_version 87070 (0.0009) [2023-10-11 22:30:37,459][71635] Updated weights for policy 1, policy_version 86982 (0.0008) [2023-10-11 22:30:37,839][71635] Updated weights for policy 1, policy_version 86992 (0.0007) [2023-10-11 22:30:38,198][71635] Updated weights for policy 1, policy_version 87002 (0.0011) [2023-10-11 22:30:39,863][71601] Updated weights for policy 0, policy_version 87080 (0.0008) [2023-10-11 22:30:40,233][71601] Updated weights for policy 0, policy_version 87090 (0.0007) [2023-10-11 22:30:40,608][71601] Updated weights for policy 0, policy_version 87100 (0.0009) [2023-10-11 22:30:41,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 178290688. Throughput: 0: 1826.2, 1: 1819.7. Samples: 44580398. Policy #0 lag: (min: 6.0, avg: 13.7, max: 38.0) [2023-10-11 22:30:41,034][70582] Avg episode reward: [(0, '124.810'), (1, '106.270')] [2023-10-11 22:30:41,775][71635] Updated weights for policy 1, policy_version 87012 (0.0007) [2023-10-11 22:30:42,153][71635] Updated weights for policy 1, policy_version 87022 (0.0009) [2023-10-11 22:30:42,512][71635] Updated weights for policy 1, policy_version 87032 (0.0008) [2023-10-11 22:30:44,276][71601] Updated weights for policy 0, policy_version 87110 (0.0009) [2023-10-11 22:30:44,650][71601] Updated weights for policy 0, policy_version 87120 (0.0009) [2023-10-11 22:30:45,023][71601] Updated weights for policy 0, policy_version 87130 (0.0007) [2023-10-11 22:30:46,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 178356224. Throughput: 0: 1837.1, 1: 1821.2. Samples: 44591418. Policy #0 lag: (min: 6.0, avg: 13.7, max: 38.0) [2023-10-11 22:30:46,034][70582] Avg episode reward: [(0, '130.060'), (1, '104.380')] [2023-10-11 22:30:46,299][71635] Updated weights for policy 1, policy_version 87042 (0.0008) [2023-10-11 22:30:46,674][71635] Updated weights for policy 1, policy_version 87052 (0.0011) [2023-10-11 22:30:47,043][71635] Updated weights for policy 1, policy_version 87062 (0.0007) [2023-10-11 22:30:47,412][71635] Updated weights for policy 1, policy_version 87072 (0.0008) [2023-10-11 22:30:48,733][71601] Updated weights for policy 0, policy_version 87140 (0.0007) [2023-10-11 22:30:49,103][71601] Updated weights for policy 0, policy_version 87150 (0.0008) [2023-10-11 22:30:49,477][71601] Updated weights for policy 0, policy_version 87160 (0.0007) [2023-10-11 22:30:50,988][71635] Updated weights for policy 1, policy_version 87082 (0.0009) [2023-10-11 22:30:51,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 178421760. Throughput: 0: 1821.2, 1: 1822.5. Samples: 44613048. Policy #0 lag: (min: 6.0, avg: 13.7, max: 38.0) [2023-10-11 22:30:51,034][70582] Avg episode reward: [(0, '132.950'), (1, '106.830')] [2023-10-11 22:30:51,357][71635] Updated weights for policy 1, policy_version 87092 (0.0008) [2023-10-11 22:30:51,730][71635] Updated weights for policy 1, policy_version 87102 (0.0007) [2023-10-11 22:30:53,149][71601] Updated weights for policy 0, policy_version 87170 (0.0008) [2023-10-11 22:30:53,522][71601] Updated weights for policy 0, policy_version 87180 (0.0009) [2023-10-11 22:30:53,884][71601] Updated weights for policy 0, policy_version 87190 (0.0008) [2023-10-11 22:30:54,251][71601] Updated weights for policy 0, policy_version 87200 (0.0009) [2023-10-11 22:30:55,393][71635] Updated weights for policy 1, policy_version 87112 (0.0008) [2023-10-11 22:30:55,766][71635] Updated weights for policy 1, policy_version 87122 (0.0007) [2023-10-11 22:30:56,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 178487296. Throughput: 0: 1828.5, 1: 1821.0. Samples: 44635358. Policy #0 lag: (min: 6.0, avg: 13.7, max: 38.0) [2023-10-11 22:30:56,034][70582] Avg episode reward: [(0, '127.590'), (1, '115.590')] [2023-10-11 22:30:56,133][71635] Updated weights for policy 1, policy_version 87132 (0.0009) [2023-10-11 22:30:58,040][71601] Updated weights for policy 0, policy_version 87210 (0.0009) [2023-10-11 22:30:58,419][71601] Updated weights for policy 0, policy_version 87220 (0.0008) [2023-10-11 22:30:58,791][71601] Updated weights for policy 0, policy_version 87230 (0.0010) [2023-10-11 22:30:59,891][71635] Updated weights for policy 1, policy_version 87142 (0.0008) [2023-10-11 22:31:00,257][71635] Updated weights for policy 1, policy_version 87152 (0.0009) [2023-10-11 22:31:00,626][71635] Updated weights for policy 1, policy_version 87162 (0.0008) [2023-10-11 22:31:01,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 178585600. Throughput: 0: 1818.4, 1: 1829.6. Samples: 44646190. Policy #0 lag: (min: 6.0, avg: 13.7, max: 38.0) [2023-10-11 22:31:01,034][70582] Avg episode reward: [(0, '126.170'), (1, '117.020')] [2023-10-11 22:31:02,683][71601] Updated weights for policy 0, policy_version 87240 (0.0010) [2023-10-11 22:31:03,056][71601] Updated weights for policy 0, policy_version 87250 (0.0008) [2023-10-11 22:31:03,417][71601] Updated weights for policy 0, policy_version 87260 (0.0007) [2023-10-11 22:31:04,315][71635] Updated weights for policy 1, policy_version 87172 (0.0009) [2023-10-11 22:31:04,683][71635] Updated weights for policy 1, policy_version 87182 (0.0009) [2023-10-11 22:31:05,053][71635] Updated weights for policy 1, policy_version 87192 (0.0008) [2023-10-11 22:31:06,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 178651136. Throughput: 0: 1814.2, 1: 1824.9. Samples: 44668010. Policy #0 lag: (min: 6.0, avg: 13.7, max: 38.0) [2023-10-11 22:31:06,034][70582] Avg episode reward: [(0, '120.090'), (1, '116.200')] [2023-10-11 22:31:07,201][71601] Updated weights for policy 0, policy_version 87270 (0.0007) [2023-10-11 22:31:07,581][71601] Updated weights for policy 0, policy_version 87280 (0.0009) [2023-10-11 22:31:07,965][71601] Updated weights for policy 0, policy_version 87290 (0.0009) [2023-10-11 22:31:08,725][71635] Updated weights for policy 1, policy_version 87202 (0.0008) [2023-10-11 22:31:09,097][71635] Updated weights for policy 1, policy_version 87212 (0.0009) [2023-10-11 22:31:09,456][71635] Updated weights for policy 1, policy_version 87222 (0.0009) [2023-10-11 22:31:09,819][71635] Updated weights for policy 1, policy_version 87232 (0.0010) [2023-10-11 22:31:11,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 178716672. Throughput: 0: 1811.1, 1: 1827.6. Samples: 44689600. Policy #0 lag: (min: 6.0, avg: 13.7, max: 38.0) [2023-10-11 22:31:11,034][70582] Avg episode reward: [(0, '128.470'), (1, '115.730')] [2023-10-11 22:31:11,743][71601] Updated weights for policy 0, policy_version 87300 (0.0009) [2023-10-11 22:31:12,131][71601] Updated weights for policy 0, policy_version 87310 (0.0007) [2023-10-11 22:31:12,499][71601] Updated weights for policy 0, policy_version 87320 (0.0008) [2023-10-11 22:31:13,501][71635] Updated weights for policy 1, policy_version 87242 (0.0008) [2023-10-11 22:31:13,865][71635] Updated weights for policy 1, policy_version 87252 (0.0008) [2023-10-11 22:31:14,230][71635] Updated weights for policy 1, policy_version 87262 (0.0008) [2023-10-11 22:31:16,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 178782208. Throughput: 0: 1811.0, 1: 1824.7. Samples: 44700592. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-11 22:31:16,034][70582] Avg episode reward: [(0, '125.000'), (1, '117.330')] [2023-10-11 22:31:16,066][71601] Updated weights for policy 0, policy_version 87330 (0.0008) [2023-10-11 22:31:16,439][71601] Updated weights for policy 0, policy_version 87340 (0.0009) [2023-10-11 22:31:16,810][71601] Updated weights for policy 0, policy_version 87350 (0.0009) [2023-10-11 22:31:17,186][71601] Updated weights for policy 0, policy_version 87360 (0.0010) [2023-10-11 22:31:18,006][71635] Updated weights for policy 1, policy_version 87272 (0.0008) [2023-10-11 22:31:18,369][71635] Updated weights for policy 1, policy_version 87282 (0.0007) [2023-10-11 22:31:18,740][71635] Updated weights for policy 1, policy_version 87292 (0.0007) [2023-10-11 22:31:20,894][71601] Updated weights for policy 0, policy_version 87370 (0.0010) [2023-10-11 22:31:21,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 178847744. Throughput: 0: 1811.4, 1: 1829.7. Samples: 44722322. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-11 22:31:21,034][70582] Avg episode reward: [(0, '119.300'), (1, '118.810')] [2023-10-11 22:31:21,264][71601] Updated weights for policy 0, policy_version 87380 (0.0007) [2023-10-11 22:31:21,637][71601] Updated weights for policy 0, policy_version 87390 (0.0007) [2023-10-11 22:31:22,552][71635] Updated weights for policy 1, policy_version 87302 (0.0010) [2023-10-11 22:31:22,944][71635] Updated weights for policy 1, policy_version 87312 (0.0011) [2023-10-11 22:31:23,322][71635] Updated weights for policy 1, policy_version 87322 (0.0010) [2023-10-11 22:31:25,255][71601] Updated weights for policy 0, policy_version 87400 (0.0009) [2023-10-11 22:31:25,613][71601] Updated weights for policy 0, policy_version 87410 (0.0009) [2023-10-11 22:31:25,986][71601] Updated weights for policy 0, policy_version 87420 (0.0007) [2023-10-11 22:31:26,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 178913280. Throughput: 0: 1823.2, 1: 1827.0. Samples: 44744656. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-11 22:31:26,034][70582] Avg episode reward: [(0, '116.840'), (1, '121.430')] [2023-10-11 22:31:26,909][71635] Updated weights for policy 1, policy_version 87332 (0.0009) [2023-10-11 22:31:27,283][71635] Updated weights for policy 1, policy_version 87342 (0.0009) [2023-10-11 22:31:27,649][71635] Updated weights for policy 1, policy_version 87352 (0.0008) [2023-10-11 22:31:29,692][71601] Updated weights for policy 0, policy_version 87430 (0.0008) [2023-10-11 22:31:30,063][71601] Updated weights for policy 0, policy_version 87440 (0.0009) [2023-10-11 22:31:30,429][71601] Updated weights for policy 0, policy_version 87450 (0.0007) [2023-10-11 22:31:31,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 179011584. Throughput: 0: 1809.1, 1: 1833.1. Samples: 44755314. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-11 22:31:31,035][70582] Avg episode reward: [(0, '119.020'), (1, '117.480')] [2023-10-11 22:31:31,302][71635] Updated weights for policy 1, policy_version 87362 (0.0007) [2023-10-11 22:31:31,675][71635] Updated weights for policy 1, policy_version 87372 (0.0010) [2023-10-11 22:31:32,039][71635] Updated weights for policy 1, policy_version 87382 (0.0009) [2023-10-11 22:31:32,407][71635] Updated weights for policy 1, policy_version 87392 (0.0009) [2023-10-11 22:31:34,223][71601] Updated weights for policy 0, policy_version 87460 (0.0008) [2023-10-11 22:31:34,591][71601] Updated weights for policy 0, policy_version 87470 (0.0009) [2023-10-11 22:31:34,961][71601] Updated weights for policy 0, policy_version 87480 (0.0008) [2023-10-11 22:31:36,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 179077120. Throughput: 0: 1826.2, 1: 1830.5. Samples: 44777600. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-11 22:31:36,034][70582] Avg episode reward: [(0, '120.490'), (1, '115.840')] [2023-10-11 22:31:36,065][71635] Updated weights for policy 1, policy_version 87402 (0.0010) [2023-10-11 22:31:36,441][71635] Updated weights for policy 1, policy_version 87412 (0.0009) [2023-10-11 22:31:36,802][71635] Updated weights for policy 1, policy_version 87422 (0.0011) [2023-10-11 22:31:38,676][71601] Updated weights for policy 0, policy_version 87490 (0.0008) [2023-10-11 22:31:39,050][71601] Updated weights for policy 0, policy_version 87500 (0.0009) [2023-10-11 22:31:39,421][71601] Updated weights for policy 0, policy_version 87510 (0.0008) [2023-10-11 22:31:39,796][71601] Updated weights for policy 0, policy_version 87520 (0.0011) [2023-10-11 22:31:40,604][71635] Updated weights for policy 1, policy_version 87432 (0.0008) [2023-10-11 22:31:40,977][71635] Updated weights for policy 1, policy_version 87442 (0.0010) [2023-10-11 22:31:41,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 179142656. Throughput: 0: 1811.0, 1: 1826.5. Samples: 44799048. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-11 22:31:41,035][70582] Avg episode reward: [(0, '113.900'), (1, '118.340')] [2023-10-11 22:31:41,339][71635] Updated weights for policy 1, policy_version 87452 (0.0009) [2023-10-11 22:31:43,432][71601] Updated weights for policy 0, policy_version 87530 (0.0008) [2023-10-11 22:31:43,800][71601] Updated weights for policy 0, policy_version 87540 (0.0009) [2023-10-11 22:31:44,174][71601] Updated weights for policy 0, policy_version 87550 (0.0008) [2023-10-11 22:31:45,047][71635] Updated weights for policy 1, policy_version 87462 (0.0008) [2023-10-11 22:31:45,422][71635] Updated weights for policy 1, policy_version 87472 (0.0009) [2023-10-11 22:31:45,786][71635] Updated weights for policy 1, policy_version 87482 (0.0008) [2023-10-11 22:31:46,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 179240960. Throughput: 0: 1823.4, 1: 1819.8. Samples: 44810136. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-11 22:31:46,034][70582] Avg episode reward: [(0, '113.550'), (1, '118.470')] [2023-10-11 22:31:47,817][71601] Updated weights for policy 0, policy_version 87560 (0.0009) [2023-10-11 22:31:48,198][71601] Updated weights for policy 0, policy_version 87570 (0.0008) [2023-10-11 22:31:48,565][71601] Updated weights for policy 0, policy_version 87580 (0.0009) [2023-10-11 22:31:49,379][71635] Updated weights for policy 1, policy_version 87492 (0.0009) [2023-10-11 22:31:49,757][71635] Updated weights for policy 1, policy_version 87502 (0.0008) [2023-10-11 22:31:50,116][71635] Updated weights for policy 1, policy_version 87512 (0.0008) [2023-10-11 22:31:51,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 179306496. Throughput: 0: 1819.8, 1: 1820.8. Samples: 44831838. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-11 22:31:51,034][70582] Avg episode reward: [(0, '120.400'), (1, '110.600')] [2023-10-11 22:31:52,182][71601] Updated weights for policy 0, policy_version 87590 (0.0007) [2023-10-11 22:31:52,561][71601] Updated weights for policy 0, policy_version 87600 (0.0009) [2023-10-11 22:31:52,931][71601] Updated weights for policy 0, policy_version 87610 (0.0007) [2023-10-11 22:31:53,844][71635] Updated weights for policy 1, policy_version 87522 (0.0007) [2023-10-11 22:31:54,208][71635] Updated weights for policy 1, policy_version 87532 (0.0008) [2023-10-11 22:31:54,578][71635] Updated weights for policy 1, policy_version 87542 (0.0008) [2023-10-11 22:31:54,947][71635] Updated weights for policy 1, policy_version 87552 (0.0009) [2023-10-11 22:31:56,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 179372032. Throughput: 0: 1821.2, 1: 1820.7. Samples: 44853482. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-11 22:31:56,034][70582] Avg episode reward: [(0, '112.720'), (1, '112.870')] [2023-10-11 22:31:56,041][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000087616_89718784.pth... [2023-10-11 22:31:56,041][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000087552_89653248.pth... [2023-10-11 22:31:56,076][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000085856_87916544.pth [2023-10-11 22:31:56,083][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000085920_87982080.pth [2023-10-11 22:31:56,618][71601] Updated weights for policy 0, policy_version 87620 (0.0007) [2023-10-11 22:31:56,999][71601] Updated weights for policy 0, policy_version 87630 (0.0007) [2023-10-11 22:31:57,376][71601] Updated weights for policy 0, policy_version 87640 (0.0007) [2023-10-11 22:31:58,632][71635] Updated weights for policy 1, policy_version 87562 (0.0007) [2023-10-11 22:31:59,005][71635] Updated weights for policy 1, policy_version 87572 (0.0010) [2023-10-11 22:31:59,371][71635] Updated weights for policy 1, policy_version 87582 (0.0010) [2023-10-11 22:32:00,867][71601] Updated weights for policy 0, policy_version 87650 (0.0007) [2023-10-11 22:32:01,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 179437568. Throughput: 0: 1825.6, 1: 1819.9. Samples: 44864636. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-11 22:32:01,034][70582] Avg episode reward: [(0, '114.680'), (1, '109.090')] [2023-10-11 22:32:01,243][71601] Updated weights for policy 0, policy_version 87660 (0.0007) [2023-10-11 22:32:01,603][71601] Updated weights for policy 0, policy_version 87670 (0.0009) [2023-10-11 22:32:01,968][71601] Updated weights for policy 0, policy_version 87680 (0.0010) [2023-10-11 22:32:03,021][71635] Updated weights for policy 1, policy_version 87592 (0.0009) [2023-10-11 22:32:03,388][71635] Updated weights for policy 1, policy_version 87602 (0.0007) [2023-10-11 22:32:03,760][71635] Updated weights for policy 1, policy_version 87612 (0.0009) [2023-10-11 22:32:05,681][71601] Updated weights for policy 0, policy_version 87690 (0.0008) [2023-10-11 22:32:06,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 179503104. Throughput: 0: 1824.0, 1: 1819.6. Samples: 44886286. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-11 22:32:06,034][70582] Avg episode reward: [(0, '110.820'), (1, '108.640')] [2023-10-11 22:32:06,049][71601] Updated weights for policy 0, policy_version 87700 (0.0008) [2023-10-11 22:32:06,428][71601] Updated weights for policy 0, policy_version 87710 (0.0008) [2023-10-11 22:32:07,441][71635] Updated weights for policy 1, policy_version 87622 (0.0007) [2023-10-11 22:32:07,834][71635] Updated weights for policy 1, policy_version 87632 (0.0009) [2023-10-11 22:32:08,193][71635] Updated weights for policy 1, policy_version 87642 (0.0009) [2023-10-11 22:32:10,195][71601] Updated weights for policy 0, policy_version 87720 (0.0009) [2023-10-11 22:32:10,557][71601] Updated weights for policy 0, policy_version 87730 (0.0010) [2023-10-11 22:32:10,934][71601] Updated weights for policy 0, policy_version 87740 (0.0010) [2023-10-11 22:32:11,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 179568640. Throughput: 0: 1813.7, 1: 1826.9. Samples: 44908484. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-11 22:32:11,034][70582] Avg episode reward: [(0, '112.850'), (1, '102.510')] [2023-10-11 22:32:11,701][71635] Updated weights for policy 1, policy_version 87652 (0.0007) [2023-10-11 22:32:12,074][71635] Updated weights for policy 1, policy_version 87662 (0.0009) [2023-10-11 22:32:12,440][71635] Updated weights for policy 1, policy_version 87672 (0.0009) [2023-10-11 22:32:14,622][71601] Updated weights for policy 0, policy_version 87750 (0.0010) [2023-10-11 22:32:14,984][71601] Updated weights for policy 0, policy_version 87760 (0.0007) [2023-10-11 22:32:15,356][71601] Updated weights for policy 0, policy_version 87770 (0.0010) [2023-10-11 22:32:15,978][71635] Updated weights for policy 1, policy_version 87682 (0.0009) [2023-10-11 22:32:16,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 179666944. Throughput: 0: 1816.6, 1: 1821.3. Samples: 44919016. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:32:16,034][70582] Avg episode reward: [(0, '114.060'), (1, '102.090')] [2023-10-11 22:32:16,350][71635] Updated weights for policy 1, policy_version 87692 (0.0007) [2023-10-11 22:32:16,724][71635] Updated weights for policy 1, policy_version 87702 (0.0008) [2023-10-11 22:32:17,078][71635] Updated weights for policy 1, policy_version 87712 (0.0008) [2023-10-11 22:32:19,074][71601] Updated weights for policy 0, policy_version 87780 (0.0009) [2023-10-11 22:32:19,448][71601] Updated weights for policy 0, policy_version 87790 (0.0009) [2023-10-11 22:32:19,826][71601] Updated weights for policy 0, policy_version 87800 (0.0009) [2023-10-11 22:32:20,664][71635] Updated weights for policy 1, policy_version 87722 (0.0007) [2023-10-11 22:32:21,028][71635] Updated weights for policy 1, policy_version 87732 (0.0008) [2023-10-11 22:32:21,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 179732480. Throughput: 0: 1815.0, 1: 1830.1. Samples: 44941630. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:32:21,034][70582] Avg episode reward: [(0, '113.800'), (1, '105.480')] [2023-10-11 22:32:21,394][71635] Updated weights for policy 1, policy_version 87742 (0.0007) [2023-10-11 22:32:23,391][71601] Updated weights for policy 0, policy_version 87810 (0.0007) [2023-10-11 22:32:23,750][71601] Updated weights for policy 0, policy_version 87820 (0.0008) [2023-10-11 22:32:24,129][71601] Updated weights for policy 0, policy_version 87830 (0.0010) [2023-10-11 22:32:24,493][71601] Updated weights for policy 0, policy_version 87840 (0.0007) [2023-10-11 22:32:25,099][71635] Updated weights for policy 1, policy_version 87752 (0.0007) [2023-10-11 22:32:25,462][71635] Updated weights for policy 1, policy_version 87762 (0.0007) [2023-10-11 22:32:25,828][71635] Updated weights for policy 1, policy_version 87772 (0.0008) [2023-10-11 22:32:26,034][70582] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 179830784. Throughput: 0: 1825.4, 1: 1823.3. Samples: 44963242. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:32:26,034][70582] Avg episode reward: [(0, '110.420'), (1, '107.530')] [2023-10-11 22:32:28,208][71601] Updated weights for policy 0, policy_version 87850 (0.0008) [2023-10-11 22:32:28,584][71601] Updated weights for policy 0, policy_version 87860 (0.0007) [2023-10-11 22:32:28,957][71601] Updated weights for policy 0, policy_version 87870 (0.0008) [2023-10-11 22:32:29,614][71635] Updated weights for policy 1, policy_version 87782 (0.0009) [2023-10-11 22:32:29,977][71635] Updated weights for policy 1, policy_version 87792 (0.0008) [2023-10-11 22:32:30,346][71635] Updated weights for policy 1, policy_version 87802 (0.0009) [2023-10-11 22:32:31,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 179896320. Throughput: 0: 1818.7, 1: 1837.1. Samples: 44974646. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:32:31,035][70582] Avg episode reward: [(0, '112.250'), (1, '109.660')] [2023-10-11 22:32:32,673][71601] Updated weights for policy 0, policy_version 87880 (0.0008) [2023-10-11 22:32:33,056][71601] Updated weights for policy 0, policy_version 87890 (0.0011) [2023-10-11 22:32:33,431][71601] Updated weights for policy 0, policy_version 87900 (0.0009) [2023-10-11 22:32:34,075][71635] Updated weights for policy 1, policy_version 87812 (0.0009) [2023-10-11 22:32:34,451][71635] Updated weights for policy 1, policy_version 87822 (0.0011) [2023-10-11 22:32:34,821][71635] Updated weights for policy 1, policy_version 87832 (0.0009) [2023-10-11 22:32:36,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 179961856. Throughput: 0: 1821.6, 1: 1823.6. Samples: 44995876. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:32:36,034][70582] Avg episode reward: [(0, '118.730'), (1, '109.570')] [2023-10-11 22:32:37,058][71601] Updated weights for policy 0, policy_version 87910 (0.0008) [2023-10-11 22:32:37,434][71601] Updated weights for policy 0, policy_version 87920 (0.0010) [2023-10-11 22:32:37,811][71601] Updated weights for policy 0, policy_version 87930 (0.0009) [2023-10-11 22:32:38,621][71635] Updated weights for policy 1, policy_version 87842 (0.0009) [2023-10-11 22:32:38,991][71635] Updated weights for policy 1, policy_version 87852 (0.0008) [2023-10-11 22:32:39,346][71635] Updated weights for policy 1, policy_version 87862 (0.0007) [2023-10-11 22:32:39,712][71635] Updated weights for policy 1, policy_version 87872 (0.0007) [2023-10-11 22:32:41,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 180027392. Throughput: 0: 1825.4, 1: 1830.8. Samples: 45018012. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:32:41,035][70582] Avg episode reward: [(0, '123.180'), (1, '102.890')] [2023-10-11 22:32:41,505][71601] Updated weights for policy 0, policy_version 87940 (0.0007) [2023-10-11 22:32:41,893][71601] Updated weights for policy 0, policy_version 87950 (0.0009) [2023-10-11 22:32:42,274][71601] Updated weights for policy 0, policy_version 87960 (0.0010) [2023-10-11 22:32:43,375][71635] Updated weights for policy 1, policy_version 87882 (0.0010) [2023-10-11 22:32:43,740][71635] Updated weights for policy 1, policy_version 87892 (0.0010) [2023-10-11 22:32:44,095][71635] Updated weights for policy 1, policy_version 87902 (0.0010) [2023-10-11 22:32:45,781][71601] Updated weights for policy 0, policy_version 87970 (0.0008) [2023-10-11 22:32:46,034][70582] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 180092928. Throughput: 0: 1821.7, 1: 1829.6. Samples: 45028946. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:32:46,035][70582] Avg episode reward: [(0, '122.110'), (1, '111.920')] [2023-10-11 22:32:46,154][71601] Updated weights for policy 0, policy_version 87980 (0.0009) [2023-10-11 22:32:46,528][71601] Updated weights for policy 0, policy_version 87990 (0.0009) [2023-10-11 22:32:46,882][71601] Updated weights for policy 0, policy_version 88000 (0.0009) [2023-10-11 22:32:47,828][71635] Updated weights for policy 1, policy_version 87912 (0.0007) [2023-10-11 22:32:48,196][71635] Updated weights for policy 1, policy_version 87922 (0.0008) [2023-10-11 22:32:48,565][71635] Updated weights for policy 1, policy_version 87932 (0.0008) [2023-10-11 22:32:50,608][71601] Updated weights for policy 0, policy_version 88010 (0.0008) [2023-10-11 22:32:50,980][71601] Updated weights for policy 0, policy_version 88020 (0.0010) [2023-10-11 22:32:51,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 180158464. Throughput: 0: 1825.4, 1: 1830.7. Samples: 45050812. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:32:51,035][70582] Avg episode reward: [(0, '120.880'), (1, '106.810')] [2023-10-11 22:32:51,349][71601] Updated weights for policy 0, policy_version 88030 (0.0007) [2023-10-11 22:32:52,241][71635] Updated weights for policy 1, policy_version 87942 (0.0007) [2023-10-11 22:32:52,601][71635] Updated weights for policy 1, policy_version 87952 (0.0008) [2023-10-11 22:32:52,962][71635] Updated weights for policy 1, policy_version 87962 (0.0009) [2023-10-11 22:32:55,027][71601] Updated weights for policy 0, policy_version 88040 (0.0007) [2023-10-11 22:32:55,402][71601] Updated weights for policy 0, policy_version 88050 (0.0008) [2023-10-11 22:32:55,784][71601] Updated weights for policy 0, policy_version 88060 (0.0008) [2023-10-11 22:32:56,034][70582] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 180256768. Throughput: 0: 1829.6, 1: 1824.7. Samples: 45072924. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:32:56,034][70582] Avg episode reward: [(0, '124.660'), (1, '112.150')] [2023-10-11 22:32:56,642][71635] Updated weights for policy 1, policy_version 87972 (0.0009) [2023-10-11 22:32:57,031][71635] Updated weights for policy 1, policy_version 87982 (0.0009) [2023-10-11 22:32:57,385][71635] Updated weights for policy 1, policy_version 87992 (0.0008) [2023-10-11 22:32:59,402][71601] Updated weights for policy 0, policy_version 88070 (0.0009) [2023-10-11 22:32:59,784][71601] Updated weights for policy 0, policy_version 88080 (0.0010) [2023-10-11 22:33:00,153][71601] Updated weights for policy 0, policy_version 88090 (0.0010) [2023-10-11 22:33:01,009][71635] Updated weights for policy 1, policy_version 88002 (0.0009) [2023-10-11 22:33:01,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 180322304. Throughput: 0: 1834.6, 1: 1821.1. Samples: 45083522. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:33:01,034][70582] Avg episode reward: [(0, '124.310'), (1, '118.180')] [2023-10-11 22:33:01,371][71635] Updated weights for policy 1, policy_version 88012 (0.0008) [2023-10-11 22:33:01,739][71635] Updated weights for policy 1, policy_version 88022 (0.0011) [2023-10-11 22:33:02,103][71635] Updated weights for policy 1, policy_version 88032 (0.0008) [2023-10-11 22:33:03,904][71601] Updated weights for policy 0, policy_version 88100 (0.0010) [2023-10-11 22:33:04,267][71601] Updated weights for policy 0, policy_version 88110 (0.0010) [2023-10-11 22:33:04,645][71601] Updated weights for policy 0, policy_version 88120 (0.0010) [2023-10-11 22:33:05,905][71635] Updated weights for policy 1, policy_version 88042 (0.0008) [2023-10-11 22:33:06,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 180387840. Throughput: 0: 1820.9, 1: 1813.2. Samples: 45105164. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:33:06,034][70582] Avg episode reward: [(0, '126.360'), (1, '119.120')] [2023-10-11 22:33:06,284][71635] Updated weights for policy 1, policy_version 88052 (0.0010) [2023-10-11 22:33:06,650][71635] Updated weights for policy 1, policy_version 88062 (0.0008) [2023-10-11 22:33:08,445][71601] Updated weights for policy 0, policy_version 88130 (0.0009) [2023-10-11 22:33:08,821][71601] Updated weights for policy 0, policy_version 88140 (0.0008) [2023-10-11 22:33:09,186][71601] Updated weights for policy 0, policy_version 88150 (0.0008) [2023-10-11 22:33:09,563][71601] Updated weights for policy 0, policy_version 88160 (0.0007) [2023-10-11 22:33:10,289][71635] Updated weights for policy 1, policy_version 88072 (0.0008) [2023-10-11 22:33:10,655][71635] Updated weights for policy 1, policy_version 88082 (0.0008) [2023-10-11 22:33:11,033][71635] Updated weights for policy 1, policy_version 88092 (0.0009) [2023-10-11 22:33:11,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 180453376. Throughput: 0: 1814.7, 1: 1817.0. Samples: 45126668. Policy #0 lag: (min: 29.0, avg: 29.0, max: 32.0) [2023-10-11 22:33:11,034][70582] Avg episode reward: [(0, '129.450'), (1, '121.380')] [2023-10-11 22:33:13,157][71601] Updated weights for policy 0, policy_version 88170 (0.0009) [2023-10-11 22:33:13,525][71601] Updated weights for policy 0, policy_version 88180 (0.0009) [2023-10-11 22:33:13,891][71601] Updated weights for policy 0, policy_version 88190 (0.0008) [2023-10-11 22:33:14,703][71635] Updated weights for policy 1, policy_version 88102 (0.0009) [2023-10-11 22:33:15,068][71635] Updated weights for policy 1, policy_version 88112 (0.0010) [2023-10-11 22:33:15,425][71635] Updated weights for policy 1, policy_version 88122 (0.0008) [2023-10-11 22:33:16,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 180551680. Throughput: 0: 1815.4, 1: 1812.9. Samples: 45137922. Policy #0 lag: (min: 29.0, avg: 29.0, max: 32.0) [2023-10-11 22:33:16,035][70582] Avg episode reward: [(0, '125.870'), (1, '112.980')] [2023-10-11 22:33:17,646][71601] Updated weights for policy 0, policy_version 88200 (0.0008) [2023-10-11 22:33:18,018][71601] Updated weights for policy 0, policy_version 88210 (0.0007) [2023-10-11 22:33:18,393][71601] Updated weights for policy 0, policy_version 88220 (0.0007) [2023-10-11 22:33:19,097][71635] Updated weights for policy 1, policy_version 88132 (0.0010) [2023-10-11 22:33:19,462][71635] Updated weights for policy 1, policy_version 88142 (0.0010) [2023-10-11 22:33:19,831][71635] Updated weights for policy 1, policy_version 88152 (0.0009) [2023-10-11 22:33:21,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 180617216. Throughput: 0: 1815.2, 1: 1824.2. Samples: 45159650. Policy #0 lag: (min: 29.0, avg: 29.0, max: 32.0) [2023-10-11 22:33:21,035][70582] Avg episode reward: [(0, '125.160'), (1, '102.640')] [2023-10-11 22:33:21,946][71601] Updated weights for policy 0, policy_version 88230 (0.0009) [2023-10-11 22:33:22,323][71601] Updated weights for policy 0, policy_version 88240 (0.0009) [2023-10-11 22:33:22,694][71601] Updated weights for policy 0, policy_version 88250 (0.0009) [2023-10-11 22:33:23,662][71635] Updated weights for policy 1, policy_version 88162 (0.0010) [2023-10-11 22:33:24,029][71635] Updated weights for policy 1, policy_version 88172 (0.0009) [2023-10-11 22:33:24,388][71635] Updated weights for policy 1, policy_version 88182 (0.0009) [2023-10-11 22:33:24,756][71635] Updated weights for policy 1, policy_version 88192 (0.0007) [2023-10-11 22:33:26,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 180682752. Throughput: 0: 1813.5, 1: 1813.7. Samples: 45181234. Policy #0 lag: (min: 29.0, avg: 29.0, max: 32.0) [2023-10-11 22:33:26,035][70582] Avg episode reward: [(0, '121.370'), (1, '101.100')] [2023-10-11 22:33:26,416][71601] Updated weights for policy 0, policy_version 88260 (0.0010) [2023-10-11 22:33:26,789][71601] Updated weights for policy 0, policy_version 88270 (0.0009) [2023-10-11 22:33:27,157][71601] Updated weights for policy 0, policy_version 88280 (0.0007) [2023-10-11 22:33:28,432][71635] Updated weights for policy 1, policy_version 88202 (0.0007) [2023-10-11 22:33:28,800][71635] Updated weights for policy 1, policy_version 88212 (0.0007) [2023-10-11 22:33:29,169][71635] Updated weights for policy 1, policy_version 88222 (0.0007) [2023-10-11 22:33:31,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 180748288. Throughput: 0: 1816.1, 1: 1815.1. Samples: 45192346. Policy #0 lag: (min: 29.0, avg: 29.0, max: 32.0) [2023-10-11 22:33:31,034][70582] Avg episode reward: [(0, '116.080'), (1, '104.000')] [2023-10-11 22:33:31,118][71601] Updated weights for policy 0, policy_version 88290 (0.0008) [2023-10-11 22:33:31,489][71601] Updated weights for policy 0, policy_version 88300 (0.0009) [2023-10-11 22:33:31,864][71601] Updated weights for policy 0, policy_version 88310 (0.0007) [2023-10-11 22:33:32,228][71601] Updated weights for policy 0, policy_version 88320 (0.0007) [2023-10-11 22:33:32,699][71635] Updated weights for policy 1, policy_version 88232 (0.0010) [2023-10-11 22:33:33,076][71635] Updated weights for policy 1, policy_version 88242 (0.0007) [2023-10-11 22:33:33,438][71635] Updated weights for policy 1, policy_version 88252 (0.0007) [2023-10-11 22:33:36,021][71601] Updated weights for policy 0, policy_version 88330 (0.0007) [2023-10-11 22:33:36,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 180813824. Throughput: 0: 1810.4, 1: 1819.6. Samples: 45214164. Policy #0 lag: (min: 29.0, avg: 29.0, max: 32.0) [2023-10-11 22:33:36,035][70582] Avg episode reward: [(0, '111.810'), (1, '94.440')] [2023-10-11 22:33:36,403][71601] Updated weights for policy 0, policy_version 88340 (0.0009) [2023-10-11 22:33:36,771][71601] Updated weights for policy 0, policy_version 88350 (0.0007) [2023-10-11 22:33:37,158][71635] Updated weights for policy 1, policy_version 88262 (0.0008) [2023-10-11 22:33:37,530][71635] Updated weights for policy 1, policy_version 88272 (0.0007) [2023-10-11 22:33:37,897][71635] Updated weights for policy 1, policy_version 88282 (0.0011) [2023-10-11 22:33:40,383][71601] Updated weights for policy 0, policy_version 88360 (0.0008) [2023-10-11 22:33:40,753][71601] Updated weights for policy 0, policy_version 88370 (0.0007) [2023-10-11 22:33:41,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 180879360. Throughput: 0: 1816.0, 1: 1820.2. Samples: 45236554. Policy #0 lag: (min: 29.0, avg: 29.0, max: 32.0) [2023-10-11 22:33:41,035][70582] Avg episode reward: [(0, '114.830'), (1, '98.700')] [2023-10-11 22:33:41,128][71601] Updated weights for policy 0, policy_version 88380 (0.0008) [2023-10-11 22:33:41,748][71635] Updated weights for policy 1, policy_version 88292 (0.0010) [2023-10-11 22:33:42,115][71635] Updated weights for policy 1, policy_version 88302 (0.0008) [2023-10-11 22:33:42,478][71635] Updated weights for policy 1, policy_version 88312 (0.0008) [2023-10-11 22:33:44,790][71601] Updated weights for policy 0, policy_version 88390 (0.0008) [2023-10-11 22:33:45,164][71601] Updated weights for policy 0, policy_version 88400 (0.0008) [2023-10-11 22:33:45,539][71601] Updated weights for policy 0, policy_version 88410 (0.0007) [2023-10-11 22:33:46,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 180977664. Throughput: 0: 1808.8, 1: 1823.5. Samples: 45246976. Policy #0 lag: (min: 29.0, avg: 29.0, max: 32.0) [2023-10-11 22:33:46,034][70582] Avg episode reward: [(0, '110.140'), (1, '103.470')] [2023-10-11 22:33:46,093][71635] Updated weights for policy 1, policy_version 88322 (0.0008) [2023-10-11 22:33:46,454][71635] Updated weights for policy 1, policy_version 88332 (0.0007) [2023-10-11 22:33:46,822][71635] Updated weights for policy 1, policy_version 88342 (0.0007) [2023-10-11 22:33:47,190][71635] Updated weights for policy 1, policy_version 88352 (0.0007) [2023-10-11 22:33:49,161][71601] Updated weights for policy 0, policy_version 88420 (0.0008) [2023-10-11 22:33:49,531][71601] Updated weights for policy 0, policy_version 88430 (0.0008) [2023-10-11 22:33:49,899][71601] Updated weights for policy 0, policy_version 88440 (0.0008) [2023-10-11 22:33:50,862][71635] Updated weights for policy 1, policy_version 88362 (0.0009) [2023-10-11 22:33:51,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 181043200. Throughput: 0: 1821.1, 1: 1823.7. Samples: 45269178. Policy #0 lag: (min: 29.0, avg: 29.0, max: 32.0) [2023-10-11 22:33:51,034][70582] Avg episode reward: [(0, '107.620'), (1, '96.340')] [2023-10-11 22:33:51,226][71635] Updated weights for policy 1, policy_version 88372 (0.0008) [2023-10-11 22:33:51,587][71635] Updated weights for policy 1, policy_version 88382 (0.0008) [2023-10-11 22:33:53,362][71601] Updated weights for policy 0, policy_version 88450 (0.0008) [2023-10-11 22:33:53,727][71601] Updated weights for policy 0, policy_version 88460 (0.0008) [2023-10-11 22:33:54,094][71601] Updated weights for policy 0, policy_version 88470 (0.0007) [2023-10-11 22:33:54,461][71601] Updated weights for policy 0, policy_version 88480 (0.0009) [2023-10-11 22:33:55,202][71635] Updated weights for policy 1, policy_version 88392 (0.0008) [2023-10-11 22:33:55,574][71635] Updated weights for policy 1, policy_version 88402 (0.0008) [2023-10-11 22:33:55,945][71635] Updated weights for policy 1, policy_version 88412 (0.0009) [2023-10-11 22:33:56,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 181108736. Throughput: 0: 1825.1, 1: 1822.9. Samples: 45290826. Policy #0 lag: (min: 29.0, avg: 29.0, max: 32.0) [2023-10-11 22:33:56,034][70582] Avg episode reward: [(0, '109.830'), (1, '96.580')] [2023-10-11 22:33:56,042][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000088480_90603520.pth... [2023-10-11 22:33:56,077][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000086784_88866816.pth [2023-10-11 22:33:56,084][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000088416_90537984.pth... [2023-10-11 22:33:56,112][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000086688_88768512.pth [2023-10-11 22:33:58,106][71601] Updated weights for policy 0, policy_version 88490 (0.0009) [2023-10-11 22:33:58,473][71601] Updated weights for policy 0, policy_version 88500 (0.0008) [2023-10-11 22:33:58,844][71601] Updated weights for policy 0, policy_version 88510 (0.0007) [2023-10-11 22:33:59,708][71635] Updated weights for policy 1, policy_version 88422 (0.0008) [2023-10-11 22:34:00,074][71635] Updated weights for policy 1, policy_version 88432 (0.0007) [2023-10-11 22:34:00,438][71635] Updated weights for policy 1, policy_version 88442 (0.0007) [2023-10-11 22:34:01,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 181207040. Throughput: 0: 1820.3, 1: 1822.0. Samples: 45301826. Policy #0 lag: (min: 29.0, avg: 29.0, max: 32.0) [2023-10-11 22:34:01,035][70582] Avg episode reward: [(0, '106.750'), (1, '95.370')] [2023-10-11 22:34:02,591][71601] Updated weights for policy 0, policy_version 88520 (0.0010) [2023-10-11 22:34:02,963][71601] Updated weights for policy 0, policy_version 88530 (0.0008) [2023-10-11 22:34:03,339][71601] Updated weights for policy 0, policy_version 88540 (0.0008) [2023-10-11 22:34:04,038][71635] Updated weights for policy 1, policy_version 88452 (0.0008) [2023-10-11 22:34:04,400][71635] Updated weights for policy 1, policy_version 88462 (0.0008) [2023-10-11 22:34:04,764][71635] Updated weights for policy 1, policy_version 88472 (0.0009) [2023-10-11 22:34:06,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 181272576. Throughput: 0: 1824.9, 1: 1816.7. Samples: 45323522. Policy #0 lag: (min: 29.0, avg: 29.0, max: 32.0) [2023-10-11 22:34:06,034][70582] Avg episode reward: [(0, '103.460'), (1, '98.860')] [2023-10-11 22:34:07,128][71601] Updated weights for policy 0, policy_version 88550 (0.0007) [2023-10-11 22:34:07,508][71601] Updated weights for policy 0, policy_version 88560 (0.0007) [2023-10-11 22:34:07,878][71601] Updated weights for policy 0, policy_version 88570 (0.0008) [2023-10-11 22:34:08,474][71635] Updated weights for policy 1, policy_version 88482 (0.0010) [2023-10-11 22:34:08,843][71635] Updated weights for policy 1, policy_version 88492 (0.0008) [2023-10-11 22:34:09,206][71635] Updated weights for policy 1, policy_version 88502 (0.0009) [2023-10-11 22:34:09,573][71635] Updated weights for policy 1, policy_version 88512 (0.0008) [2023-10-11 22:34:11,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 181338112. Throughput: 0: 1829.0, 1: 1824.1. Samples: 45345624. Policy #0 lag: (min: 0.0, avg: 15.7, max: 32.0) [2023-10-11 22:34:11,034][70582] Avg episode reward: [(0, '106.520'), (1, '100.660')] [2023-10-11 22:34:11,512][71601] Updated weights for policy 0, policy_version 88580 (0.0009) [2023-10-11 22:34:11,883][71601] Updated weights for policy 0, policy_version 88590 (0.0008) [2023-10-11 22:34:12,257][71601] Updated weights for policy 0, policy_version 88600 (0.0008) [2023-10-11 22:34:13,456][71635] Updated weights for policy 1, policy_version 88522 (0.0010) [2023-10-11 22:34:13,814][71635] Updated weights for policy 1, policy_version 88532 (0.0008) [2023-10-11 22:34:14,176][71635] Updated weights for policy 1, policy_version 88542 (0.0009) [2023-10-11 22:34:15,826][71601] Updated weights for policy 0, policy_version 88610 (0.0008) [2023-10-11 22:34:16,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 181403648. Throughput: 0: 1830.9, 1: 1819.7. Samples: 45356626. Policy #0 lag: (min: 0.0, avg: 15.7, max: 32.0) [2023-10-11 22:34:16,034][70582] Avg episode reward: [(0, '117.590'), (1, '103.420')] [2023-10-11 22:34:16,210][71601] Updated weights for policy 0, policy_version 88620 (0.0008) [2023-10-11 22:34:16,584][71601] Updated weights for policy 0, policy_version 88630 (0.0009) [2023-10-11 22:34:16,951][71601] Updated weights for policy 0, policy_version 88640 (0.0010) [2023-10-11 22:34:17,961][71635] Updated weights for policy 1, policy_version 88552 (0.0007) [2023-10-11 22:34:18,327][71635] Updated weights for policy 1, policy_version 88562 (0.0009) [2023-10-11 22:34:18,695][71635] Updated weights for policy 1, policy_version 88572 (0.0008) [2023-10-11 22:34:20,769][71601] Updated weights for policy 0, policy_version 88650 (0.0008) [2023-10-11 22:34:21,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 181469184. Throughput: 0: 1833.5, 1: 1813.4. Samples: 45378276. Policy #0 lag: (min: 0.0, avg: 15.7, max: 32.0) [2023-10-11 22:34:21,034][70582] Avg episode reward: [(0, '117.740'), (1, '105.590')] [2023-10-11 22:34:21,151][71601] Updated weights for policy 0, policy_version 88660 (0.0008) [2023-10-11 22:34:21,519][71601] Updated weights for policy 0, policy_version 88670 (0.0008) [2023-10-11 22:34:22,346][71635] Updated weights for policy 1, policy_version 88582 (0.0008) [2023-10-11 22:34:22,713][71635] Updated weights for policy 1, policy_version 88592 (0.0009) [2023-10-11 22:34:23,074][71635] Updated weights for policy 1, policy_version 88602 (0.0009) [2023-10-11 22:34:25,045][71601] Updated weights for policy 0, policy_version 88680 (0.0008) [2023-10-11 22:34:25,430][71601] Updated weights for policy 0, policy_version 88690 (0.0008) [2023-10-11 22:34:25,796][71601] Updated weights for policy 0, policy_version 88700 (0.0007) [2023-10-11 22:34:26,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 181567488. Throughput: 0: 1829.4, 1: 1811.0. Samples: 45400372. Policy #0 lag: (min: 0.0, avg: 15.7, max: 32.0) [2023-10-11 22:34:26,034][70582] Avg episode reward: [(0, '121.610'), (1, '110.550')] [2023-10-11 22:34:26,889][71635] Updated weights for policy 1, policy_version 88612 (0.0007) [2023-10-11 22:34:27,263][71635] Updated weights for policy 1, policy_version 88622 (0.0009) [2023-10-11 22:34:27,632][71635] Updated weights for policy 1, policy_version 88632 (0.0008) [2023-10-11 22:34:29,506][71601] Updated weights for policy 0, policy_version 88710 (0.0007) [2023-10-11 22:34:29,877][71601] Updated weights for policy 0, policy_version 88720 (0.0009) [2023-10-11 22:34:30,248][71601] Updated weights for policy 0, policy_version 88730 (0.0009) [2023-10-11 22:34:31,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 181633024. Throughput: 0: 1837.1, 1: 1810.0. Samples: 45411096. Policy #0 lag: (min: 0.0, avg: 15.7, max: 32.0) [2023-10-11 22:34:31,034][70582] Avg episode reward: [(0, '121.250'), (1, '120.410')] [2023-10-11 22:34:31,210][71635] Updated weights for policy 1, policy_version 88642 (0.0008) [2023-10-11 22:34:31,585][71635] Updated weights for policy 1, policy_version 88652 (0.0009) [2023-10-11 22:34:31,945][71635] Updated weights for policy 1, policy_version 88662 (0.0008) [2023-10-11 22:34:32,307][71635] Updated weights for policy 1, policy_version 88672 (0.0007) [2023-10-11 22:34:33,754][71601] Updated weights for policy 0, policy_version 88740 (0.0008) [2023-10-11 22:34:34,125][71601] Updated weights for policy 0, policy_version 88750 (0.0009) [2023-10-11 22:34:34,495][71601] Updated weights for policy 0, policy_version 88760 (0.0011) [2023-10-11 22:34:35,872][71635] Updated weights for policy 1, policy_version 88682 (0.0009) [2023-10-11 22:34:36,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 181698560. Throughput: 0: 1830.1, 1: 1810.2. Samples: 45432992. Policy #0 lag: (min: 0.0, avg: 15.7, max: 32.0) [2023-10-11 22:34:36,034][70582] Avg episode reward: [(0, '124.190'), (1, '123.420')] [2023-10-11 22:34:36,241][71635] Updated weights for policy 1, policy_version 88692 (0.0009) [2023-10-11 22:34:36,595][71635] Updated weights for policy 1, policy_version 88702 (0.0009) [2023-10-11 22:34:38,041][71601] Updated weights for policy 0, policy_version 88770 (0.0010) [2023-10-11 22:34:38,416][71601] Updated weights for policy 0, policy_version 88780 (0.0009) [2023-10-11 22:34:38,783][71601] Updated weights for policy 0, policy_version 88790 (0.0007) [2023-10-11 22:34:39,160][71601] Updated weights for policy 0, policy_version 88800 (0.0009) [2023-10-11 22:34:40,310][71635] Updated weights for policy 1, policy_version 88712 (0.0009) [2023-10-11 22:34:40,671][71635] Updated weights for policy 1, policy_version 88722 (0.0008) [2023-10-11 22:34:41,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 181764096. Throughput: 0: 1838.9, 1: 1813.1. Samples: 45455166. Policy #0 lag: (min: 0.0, avg: 15.7, max: 32.0) [2023-10-11 22:34:41,035][70582] Avg episode reward: [(0, '123.580'), (1, '118.700')] [2023-10-11 22:34:41,041][71635] Updated weights for policy 1, policy_version 88732 (0.0007) [2023-10-11 22:34:42,851][71601] Updated weights for policy 0, policy_version 88810 (0.0007) [2023-10-11 22:34:43,222][71601] Updated weights for policy 0, policy_version 88820 (0.0008) [2023-10-11 22:34:43,594][71601] Updated weights for policy 0, policy_version 88830 (0.0008) [2023-10-11 22:34:44,776][71635] Updated weights for policy 1, policy_version 88742 (0.0008) [2023-10-11 22:34:45,139][71635] Updated weights for policy 1, policy_version 88752 (0.0007) [2023-10-11 22:34:45,509][71635] Updated weights for policy 1, policy_version 88762 (0.0007) [2023-10-11 22:34:46,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 181862400. Throughput: 0: 1834.4, 1: 1810.2. Samples: 45465830. Policy #0 lag: (min: 0.0, avg: 15.7, max: 32.0) [2023-10-11 22:34:46,034][70582] Avg episode reward: [(0, '129.990'), (1, '120.060')] [2023-10-11 22:34:47,234][71601] Updated weights for policy 0, policy_version 88840 (0.0007) [2023-10-11 22:34:47,601][71601] Updated weights for policy 0, policy_version 88850 (0.0007) [2023-10-11 22:34:47,975][71601] Updated weights for policy 0, policy_version 88860 (0.0007) [2023-10-11 22:34:49,327][71635] Updated weights for policy 1, policy_version 88772 (0.0009) [2023-10-11 22:34:49,701][71635] Updated weights for policy 1, policy_version 88782 (0.0007) [2023-10-11 22:34:50,068][71635] Updated weights for policy 1, policy_version 88792 (0.0009) [2023-10-11 22:34:51,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 181927936. Throughput: 0: 1841.2, 1: 1813.7. Samples: 45487992. Policy #0 lag: (min: 0.0, avg: 15.7, max: 32.0) [2023-10-11 22:34:51,034][70582] Avg episode reward: [(0, '134.120'), (1, '118.490')] [2023-10-11 22:34:51,606][71601] Updated weights for policy 0, policy_version 88870 (0.0009) [2023-10-11 22:34:51,974][71601] Updated weights for policy 0, policy_version 88880 (0.0011) [2023-10-11 22:34:52,352][71601] Updated weights for policy 0, policy_version 88890 (0.0009) [2023-10-11 22:34:53,601][71635] Updated weights for policy 1, policy_version 88802 (0.0010) [2023-10-11 22:34:53,971][71635] Updated weights for policy 1, policy_version 88812 (0.0008) [2023-10-11 22:34:54,329][71635] Updated weights for policy 1, policy_version 88822 (0.0007) [2023-10-11 22:34:54,695][71635] Updated weights for policy 1, policy_version 88832 (0.0008) [2023-10-11 22:34:55,946][71601] Updated weights for policy 0, policy_version 88900 (0.0007) [2023-10-11 22:34:56,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 181993472. Throughput: 0: 1838.3, 1: 1811.7. Samples: 45509876. Policy #0 lag: (min: 0.0, avg: 15.7, max: 32.0) [2023-10-11 22:34:56,034][70582] Avg episode reward: [(0, '141.130'), (1, '121.670')] [2023-10-11 22:34:56,312][71601] Updated weights for policy 0, policy_version 88910 (0.0007) [2023-10-11 22:34:56,690][71601] Updated weights for policy 0, policy_version 88920 (0.0010) [2023-10-11 22:34:58,470][71635] Updated weights for policy 1, policy_version 88842 (0.0008) [2023-10-11 22:34:58,832][71635] Updated weights for policy 1, policy_version 88852 (0.0008) [2023-10-11 22:34:59,207][71635] Updated weights for policy 1, policy_version 88862 (0.0010) [2023-10-11 22:35:00,508][71601] Updated weights for policy 0, policy_version 88930 (0.0011) [2023-10-11 22:35:00,908][71601] Updated weights for policy 0, policy_version 88940 (0.0007) [2023-10-11 22:35:01,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 182059008. Throughput: 0: 1836.0, 1: 1814.8. Samples: 45520912. Policy #0 lag: (min: 0.0, avg: 15.7, max: 32.0) [2023-10-11 22:35:01,035][70582] Avg episode reward: [(0, '139.470'), (1, '122.340')] [2023-10-11 22:35:01,272][71601] Updated weights for policy 0, policy_version 88950 (0.0008) [2023-10-11 22:35:01,643][71601] Updated weights for policy 0, policy_version 88960 (0.0008) [2023-10-11 22:35:02,992][71635] Updated weights for policy 1, policy_version 88872 (0.0008) [2023-10-11 22:35:03,356][71635] Updated weights for policy 1, policy_version 88882 (0.0011) [2023-10-11 22:35:03,722][71635] Updated weights for policy 1, policy_version 88892 (0.0008) [2023-10-11 22:35:05,219][71601] Updated weights for policy 0, policy_version 88970 (0.0011) [2023-10-11 22:35:05,588][71601] Updated weights for policy 0, policy_version 88980 (0.0010) [2023-10-11 22:35:05,957][71601] Updated weights for policy 0, policy_version 88990 (0.0007) [2023-10-11 22:35:06,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 182157312. Throughput: 0: 1835.3, 1: 1815.4. Samples: 45542558. Policy #0 lag: (min: 6.0, avg: 8.4, max: 38.0) [2023-10-11 22:35:06,034][70582] Avg episode reward: [(0, '142.670'), (1, '121.140')] [2023-10-11 22:35:07,398][71635] Updated weights for policy 1, policy_version 88902 (0.0010) [2023-10-11 22:35:07,759][71635] Updated weights for policy 1, policy_version 88912 (0.0009) [2023-10-11 22:35:08,137][71635] Updated weights for policy 1, policy_version 88922 (0.0010) [2023-10-11 22:35:09,701][71601] Updated weights for policy 0, policy_version 89000 (0.0009) [2023-10-11 22:35:10,069][71601] Updated weights for policy 0, policy_version 89010 (0.0010) [2023-10-11 22:35:10,429][71601] Updated weights for policy 0, policy_version 89020 (0.0009) [2023-10-11 22:35:11,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 182222848. Throughput: 0: 1819.2, 1: 1819.6. Samples: 45564116. Policy #0 lag: (min: 6.0, avg: 8.4, max: 38.0) [2023-10-11 22:35:11,035][70582] Avg episode reward: [(0, '146.360'), (1, '124.390')] [2023-10-11 22:35:12,065][71635] Updated weights for policy 1, policy_version 88932 (0.0010) [2023-10-11 22:35:12,450][71635] Updated weights for policy 1, policy_version 88942 (0.0008) [2023-10-11 22:35:12,821][71635] Updated weights for policy 1, policy_version 88952 (0.0007) [2023-10-11 22:35:14,202][71601] Updated weights for policy 0, policy_version 89030 (0.0009) [2023-10-11 22:35:14,574][71601] Updated weights for policy 0, policy_version 89040 (0.0008) [2023-10-11 22:35:14,946][71601] Updated weights for policy 0, policy_version 89050 (0.0007) [2023-10-11 22:35:16,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 182288384. Throughput: 0: 1831.8, 1: 1815.2. Samples: 45575214. Policy #0 lag: (min: 6.0, avg: 8.4, max: 38.0) [2023-10-11 22:35:16,034][70582] Avg episode reward: [(0, '143.000'), (1, '129.990')] [2023-10-11 22:35:16,423][71635] Updated weights for policy 1, policy_version 88962 (0.0008) [2023-10-11 22:35:16,791][71635] Updated weights for policy 1, policy_version 88972 (0.0009) [2023-10-11 22:35:17,163][71635] Updated weights for policy 1, policy_version 88982 (0.0010) [2023-10-11 22:35:17,518][71635] Updated weights for policy 1, policy_version 88992 (0.0009) [2023-10-11 22:35:18,357][71601] Updated weights for policy 0, policy_version 89060 (0.0008) [2023-10-11 22:35:18,735][71601] Updated weights for policy 0, policy_version 89070 (0.0010) [2023-10-11 22:35:19,106][71601] Updated weights for policy 0, policy_version 89080 (0.0010) [2023-10-11 22:35:21,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 182353920. Throughput: 0: 1819.9, 1: 1809.1. Samples: 45596298. Policy #0 lag: (min: 6.0, avg: 8.4, max: 38.0) [2023-10-11 22:35:21,034][70582] Avg episode reward: [(0, '147.600'), (1, '129.480')] [2023-10-11 22:35:21,342][71635] Updated weights for policy 1, policy_version 89002 (0.0009) [2023-10-11 22:35:21,704][71635] Updated weights for policy 1, policy_version 89012 (0.0009) [2023-10-11 22:35:22,076][71635] Updated weights for policy 1, policy_version 89022 (0.0009) [2023-10-11 22:35:22,944][71601] Updated weights for policy 0, policy_version 89090 (0.0008) [2023-10-11 22:35:23,315][71601] Updated weights for policy 0, policy_version 89100 (0.0007) [2023-10-11 22:35:23,690][71601] Updated weights for policy 0, policy_version 89110 (0.0008) [2023-10-11 22:35:24,057][71601] Updated weights for policy 0, policy_version 89120 (0.0008) [2023-10-11 22:35:25,800][71635] Updated weights for policy 1, policy_version 89032 (0.0008) [2023-10-11 22:35:26,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 182419456. Throughput: 0: 1819.8, 1: 1816.8. Samples: 45618814. Policy #0 lag: (min: 6.0, avg: 8.4, max: 38.0) [2023-10-11 22:35:26,034][70582] Avg episode reward: [(0, '141.520'), (1, '131.280')] [2023-10-11 22:35:26,169][71635] Updated weights for policy 1, policy_version 89042 (0.0008) [2023-10-11 22:35:26,536][71635] Updated weights for policy 1, policy_version 89052 (0.0008) [2023-10-11 22:35:27,891][71601] Updated weights for policy 0, policy_version 89130 (0.0010) [2023-10-11 22:35:28,264][71601] Updated weights for policy 0, policy_version 89140 (0.0010) [2023-10-11 22:35:28,626][71601] Updated weights for policy 0, policy_version 89150 (0.0009) [2023-10-11 22:35:30,140][71635] Updated weights for policy 1, policy_version 89062 (0.0007) [2023-10-11 22:35:30,507][71635] Updated weights for policy 1, policy_version 89072 (0.0007) [2023-10-11 22:35:30,875][71635] Updated weights for policy 1, policy_version 89082 (0.0008) [2023-10-11 22:35:31,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 182484992. Throughput: 0: 1820.0, 1: 1807.7. Samples: 45629076. Policy #0 lag: (min: 6.0, avg: 8.4, max: 38.0) [2023-10-11 22:35:31,035][70582] Avg episode reward: [(0, '140.420'), (1, '123.270')] [2023-10-11 22:35:32,427][71601] Updated weights for policy 0, policy_version 89160 (0.0010) [2023-10-11 22:35:32,796][71601] Updated weights for policy 0, policy_version 89170 (0.0010) [2023-10-11 22:35:33,165][71601] Updated weights for policy 0, policy_version 89180 (0.0010) [2023-10-11 22:35:34,514][71635] Updated weights for policy 1, policy_version 89092 (0.0010) [2023-10-11 22:35:34,872][71635] Updated weights for policy 1, policy_version 89102 (0.0008) [2023-10-11 22:35:35,235][71635] Updated weights for policy 1, policy_version 89112 (0.0007) [2023-10-11 22:35:36,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 182583296. Throughput: 0: 1815.6, 1: 1820.5. Samples: 45651618. Policy #0 lag: (min: 6.0, avg: 8.4, max: 38.0) [2023-10-11 22:35:36,034][70582] Avg episode reward: [(0, '141.190'), (1, '124.380')] [2023-10-11 22:35:36,925][71601] Updated weights for policy 0, policy_version 89190 (0.0010) [2023-10-11 22:35:37,292][71601] Updated weights for policy 0, policy_version 89200 (0.0009) [2023-10-11 22:35:37,656][71601] Updated weights for policy 0, policy_version 89210 (0.0010) [2023-10-11 22:35:38,852][71635] Updated weights for policy 1, policy_version 89122 (0.0009) [2023-10-11 22:35:39,227][71635] Updated weights for policy 1, policy_version 89132 (0.0009) [2023-10-11 22:35:39,589][71635] Updated weights for policy 1, policy_version 89142 (0.0011) [2023-10-11 22:35:39,955][71635] Updated weights for policy 1, policy_version 89152 (0.0010) [2023-10-11 22:35:41,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 182648832. Throughput: 0: 1807.3, 1: 1813.2. Samples: 45672798. Policy #0 lag: (min: 6.0, avg: 8.4, max: 38.0) [2023-10-11 22:35:41,034][70582] Avg episode reward: [(0, '135.790'), (1, '130.780')] [2023-10-11 22:35:41,301][71601] Updated weights for policy 0, policy_version 89220 (0.0010) [2023-10-11 22:35:41,666][71601] Updated weights for policy 0, policy_version 89230 (0.0010) [2023-10-11 22:35:42,035][71601] Updated weights for policy 0, policy_version 89240 (0.0008) [2023-10-11 22:35:43,695][71635] Updated weights for policy 1, policy_version 89162 (0.0009) [2023-10-11 22:35:44,061][71635] Updated weights for policy 1, policy_version 89172 (0.0007) [2023-10-11 22:35:44,428][71635] Updated weights for policy 1, policy_version 89182 (0.0009) [2023-10-11 22:35:45,782][71601] Updated weights for policy 0, policy_version 89250 (0.0008) [2023-10-11 22:35:46,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 182714368. Throughput: 0: 1803.4, 1: 1818.9. Samples: 45683914. Policy #0 lag: (min: 6.0, avg: 8.4, max: 38.0) [2023-10-11 22:35:46,034][70582] Avg episode reward: [(0, '127.740'), (1, '132.220')] [2023-10-11 22:35:46,187][71601] Updated weights for policy 0, policy_version 89260 (0.0007) [2023-10-11 22:35:46,562][71601] Updated weights for policy 0, policy_version 89270 (0.0007) [2023-10-11 22:35:46,923][71601] Updated weights for policy 0, policy_version 89280 (0.0007) [2023-10-11 22:35:48,173][71635] Updated weights for policy 1, policy_version 89192 (0.0009) [2023-10-11 22:35:48,539][71635] Updated weights for policy 1, policy_version 89202 (0.0009) [2023-10-11 22:35:48,911][71635] Updated weights for policy 1, policy_version 89212 (0.0010) [2023-10-11 22:35:50,655][71601] Updated weights for policy 0, policy_version 89290 (0.0009) [2023-10-11 22:35:51,021][71601] Updated weights for policy 0, policy_version 89300 (0.0008) [2023-10-11 22:35:51,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 182779904. Throughput: 0: 1803.5, 1: 1806.7. Samples: 45705014. Policy #0 lag: (min: 6.0, avg: 8.4, max: 38.0) [2023-10-11 22:35:51,035][70582] Avg episode reward: [(0, '124.220'), (1, '134.060')] [2023-10-11 22:35:51,390][71601] Updated weights for policy 0, policy_version 89310 (0.0009) [2023-10-11 22:35:52,672][71635] Updated weights for policy 1, policy_version 89222 (0.0010) [2023-10-11 22:35:53,042][71635] Updated weights for policy 1, policy_version 89232 (0.0009) [2023-10-11 22:35:53,410][71635] Updated weights for policy 1, policy_version 89242 (0.0009) [2023-10-11 22:35:55,154][71601] Updated weights for policy 0, policy_version 89320 (0.0008) [2023-10-11 22:35:55,525][71601] Updated weights for policy 0, policy_version 89330 (0.0007) [2023-10-11 22:35:55,896][71601] Updated weights for policy 0, policy_version 89340 (0.0008) [2023-10-11 22:35:56,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 182845440. Throughput: 0: 1817.1, 1: 1804.6. Samples: 45727090. Policy #0 lag: (min: 6.0, avg: 8.4, max: 38.0) [2023-10-11 22:35:56,034][70582] Avg episode reward: [(0, '119.550'), (1, '138.180')] [2023-10-11 22:35:56,044][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000089344_91488256.pth... [2023-10-11 22:35:56,044][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000089248_91389952.pth... [2023-10-11 22:35:56,076][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000087616_89718784.pth [2023-10-11 22:35:56,082][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000087552_89653248.pth [2023-10-11 22:35:57,284][71635] Updated weights for policy 1, policy_version 89252 (0.0010) [2023-10-11 22:35:57,680][71635] Updated weights for policy 1, policy_version 89262 (0.0007) [2023-10-11 22:35:58,043][71635] Updated weights for policy 1, policy_version 89272 (0.0007) [2023-10-11 22:35:59,664][71601] Updated weights for policy 0, policy_version 89350 (0.0009) [2023-10-11 22:36:00,029][71601] Updated weights for policy 0, policy_version 89360 (0.0011) [2023-10-11 22:36:00,391][71601] Updated weights for policy 0, policy_version 89370 (0.0009) [2023-10-11 22:36:01,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 182943744. Throughput: 0: 1798.3, 1: 1807.3. Samples: 45737470. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-11 22:36:01,035][70582] Avg episode reward: [(0, '114.690'), (1, '138.120')] [2023-10-11 22:36:01,737][71635] Updated weights for policy 1, policy_version 89282 (0.0008) [2023-10-11 22:36:02,090][71635] Updated weights for policy 1, policy_version 89292 (0.0008) [2023-10-11 22:36:02,462][71635] Updated weights for policy 1, policy_version 89302 (0.0011) [2023-10-11 22:36:02,829][71635] Updated weights for policy 1, policy_version 89312 (0.0010) [2023-10-11 22:36:03,964][71601] Updated weights for policy 0, policy_version 89380 (0.0008) [2023-10-11 22:36:04,345][71601] Updated weights for policy 0, policy_version 89390 (0.0009) [2023-10-11 22:36:04,716][71601] Updated weights for policy 0, policy_version 89400 (0.0008) [2023-10-11 22:36:06,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 183009280. Throughput: 0: 1816.4, 1: 1808.3. Samples: 45759412. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-11 22:36:06,035][70582] Avg episode reward: [(0, '111.690'), (1, '135.830')] [2023-10-11 22:36:06,499][71635] Updated weights for policy 1, policy_version 89322 (0.0008) [2023-10-11 22:36:06,864][71635] Updated weights for policy 1, policy_version 89332 (0.0010) [2023-10-11 22:36:07,236][71635] Updated weights for policy 1, policy_version 89342 (0.0009) [2023-10-11 22:36:08,435][71601] Updated weights for policy 0, policy_version 89410 (0.0009) [2023-10-11 22:36:08,809][71601] Updated weights for policy 0, policy_version 89420 (0.0007) [2023-10-11 22:36:09,191][71601] Updated weights for policy 0, policy_version 89430 (0.0008) [2023-10-11 22:36:09,551][71601] Updated weights for policy 0, policy_version 89440 (0.0007) [2023-10-11 22:36:11,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 183074816. Throughput: 0: 1804.8, 1: 1809.8. Samples: 45781474. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-11 22:36:11,035][70582] Avg episode reward: [(0, '118.680'), (1, '130.600')] [2023-10-11 22:36:11,091][71635] Updated weights for policy 1, policy_version 89352 (0.0008) [2023-10-11 22:36:11,453][71635] Updated weights for policy 1, policy_version 89362 (0.0007) [2023-10-11 22:36:11,831][71635] Updated weights for policy 1, policy_version 89372 (0.0008) [2023-10-11 22:36:13,319][71601] Updated weights for policy 0, policy_version 89450 (0.0008) [2023-10-11 22:36:13,697][71601] Updated weights for policy 0, policy_version 89460 (0.0009) [2023-10-11 22:36:14,081][71601] Updated weights for policy 0, policy_version 89470 (0.0008) [2023-10-11 22:36:15,606][71635] Updated weights for policy 1, policy_version 89382 (0.0008) [2023-10-11 22:36:15,960][71635] Updated weights for policy 1, policy_version 89392 (0.0009) [2023-10-11 22:36:16,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 183140352. Throughput: 0: 1815.7, 1: 1808.1. Samples: 45792148. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-11 22:36:16,035][70582] Avg episode reward: [(0, '110.320'), (1, '127.960')] [2023-10-11 22:36:16,327][71635] Updated weights for policy 1, policy_version 89402 (0.0008) [2023-10-11 22:36:17,770][71601] Updated weights for policy 0, policy_version 89480 (0.0008) [2023-10-11 22:36:18,138][71601] Updated weights for policy 0, policy_version 89490 (0.0008) [2023-10-11 22:36:18,516][71601] Updated weights for policy 0, policy_version 89500 (0.0008) [2023-10-11 22:36:20,058][71635] Updated weights for policy 1, policy_version 89412 (0.0008) [2023-10-11 22:36:20,429][71635] Updated weights for policy 1, policy_version 89422 (0.0009) [2023-10-11 22:36:20,786][71635] Updated weights for policy 1, policy_version 89432 (0.0008) [2023-10-11 22:36:21,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 183205888. Throughput: 0: 1800.7, 1: 1799.5. Samples: 45813624. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-11 22:36:21,035][70582] Avg episode reward: [(0, '112.470'), (1, '128.110')] [2023-10-11 22:36:22,257][71601] Updated weights for policy 0, policy_version 89510 (0.0008) [2023-10-11 22:36:22,635][71601] Updated weights for policy 0, policy_version 89520 (0.0007) [2023-10-11 22:36:23,006][71601] Updated weights for policy 0, policy_version 89530 (0.0007) [2023-10-11 22:36:24,436][71635] Updated weights for policy 1, policy_version 89442 (0.0008) [2023-10-11 22:36:24,798][71635] Updated weights for policy 1, policy_version 89452 (0.0008) [2023-10-11 22:36:25,164][71635] Updated weights for policy 1, policy_version 89462 (0.0009) [2023-10-11 22:36:25,526][71635] Updated weights for policy 1, policy_version 89472 (0.0009) [2023-10-11 22:36:26,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 183304192. Throughput: 0: 1805.2, 1: 1804.5. Samples: 45835234. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-11 22:36:26,035][70582] Avg episode reward: [(0, '112.060'), (1, '127.380')] [2023-10-11 22:36:26,522][71601] Updated weights for policy 0, policy_version 89540 (0.0010) [2023-10-11 22:36:26,897][71601] Updated weights for policy 0, policy_version 89550 (0.0007) [2023-10-11 22:36:27,274][71601] Updated weights for policy 0, policy_version 89560 (0.0008) [2023-10-11 22:36:29,391][71635] Updated weights for policy 1, policy_version 89482 (0.0010) [2023-10-11 22:36:29,751][71635] Updated weights for policy 1, policy_version 89492 (0.0008) [2023-10-11 22:36:30,124][71635] Updated weights for policy 1, policy_version 89502 (0.0008) [2023-10-11 22:36:31,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 183369728. Throughput: 0: 1810.6, 1: 1796.4. Samples: 45846230. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-11 22:36:31,035][70582] Avg episode reward: [(0, '113.820'), (1, '129.380')] [2023-10-11 22:36:31,061][71601] Updated weights for policy 0, policy_version 89570 (0.0008) [2023-10-11 22:36:31,440][71601] Updated weights for policy 0, policy_version 89580 (0.0008) [2023-10-11 22:36:31,806][71601] Updated weights for policy 0, policy_version 89590 (0.0008) [2023-10-11 22:36:32,184][71601] Updated weights for policy 0, policy_version 89600 (0.0008) [2023-10-11 22:36:33,863][71635] Updated weights for policy 1, policy_version 89512 (0.0009) [2023-10-11 22:36:34,237][71635] Updated weights for policy 1, policy_version 89522 (0.0008) [2023-10-11 22:36:34,604][71635] Updated weights for policy 1, policy_version 89532 (0.0009) [2023-10-11 22:36:35,858][71601] Updated weights for policy 0, policy_version 89610 (0.0008) [2023-10-11 22:36:36,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 183435264. Throughput: 0: 1800.6, 1: 1812.9. Samples: 45867622. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-11 22:36:36,034][70582] Avg episode reward: [(0, '117.770'), (1, '127.760')] [2023-10-11 22:36:36,237][71601] Updated weights for policy 0, policy_version 89620 (0.0011) [2023-10-11 22:36:36,607][71601] Updated weights for policy 0, policy_version 89630 (0.0008) [2023-10-11 22:36:38,144][71635] Updated weights for policy 1, policy_version 89542 (0.0008) [2023-10-11 22:36:38,514][71635] Updated weights for policy 1, policy_version 89552 (0.0007) [2023-10-11 22:36:38,885][71635] Updated weights for policy 1, policy_version 89562 (0.0007) [2023-10-11 22:36:40,352][71601] Updated weights for policy 0, policy_version 89640 (0.0009) [2023-10-11 22:36:40,716][71601] Updated weights for policy 0, policy_version 89650 (0.0007) [2023-10-11 22:36:41,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 183500800. Throughput: 0: 1807.6, 1: 1804.7. Samples: 45889644. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-11 22:36:41,035][70582] Avg episode reward: [(0, '119.510'), (1, '124.080')] [2023-10-11 22:36:41,090][71601] Updated weights for policy 0, policy_version 89660 (0.0007) [2023-10-11 22:36:42,559][71635] Updated weights for policy 1, policy_version 89572 (0.0008) [2023-10-11 22:36:42,925][71635] Updated weights for policy 1, policy_version 89582 (0.0007) [2023-10-11 22:36:43,285][71635] Updated weights for policy 1, policy_version 89592 (0.0008) [2023-10-11 22:36:44,989][71601] Updated weights for policy 0, policy_version 89670 (0.0008) [2023-10-11 22:36:45,362][71601] Updated weights for policy 0, policy_version 89680 (0.0009) [2023-10-11 22:36:45,724][71601] Updated weights for policy 0, policy_version 89690 (0.0009) [2023-10-11 22:36:46,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 183599104. Throughput: 0: 1805.9, 1: 1815.2. Samples: 45900420. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-11 22:36:46,035][70582] Avg episode reward: [(0, '118.800'), (1, '117.740')] [2023-10-11 22:36:47,070][71635] Updated weights for policy 1, policy_version 89602 (0.0007) [2023-10-11 22:36:47,432][71635] Updated weights for policy 1, policy_version 89612 (0.0008) [2023-10-11 22:36:47,797][71635] Updated weights for policy 1, policy_version 89622 (0.0008) [2023-10-11 22:36:48,161][71635] Updated weights for policy 1, policy_version 89632 (0.0009) [2023-10-11 22:36:49,466][71601] Updated weights for policy 0, policy_version 89700 (0.0007) [2023-10-11 22:36:49,833][71601] Updated weights for policy 0, policy_version 89710 (0.0007) [2023-10-11 22:36:50,201][71601] Updated weights for policy 0, policy_version 89720 (0.0010) [2023-10-11 22:36:51,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 183664640. Throughput: 0: 1815.8, 1: 1809.1. Samples: 45922532. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-11 22:36:51,035][70582] Avg episode reward: [(0, '118.330'), (1, '109.600')] [2023-10-11 22:36:51,809][71635] Updated weights for policy 1, policy_version 89642 (0.0010) [2023-10-11 22:36:52,173][71635] Updated weights for policy 1, policy_version 89652 (0.0008) [2023-10-11 22:36:52,534][71635] Updated weights for policy 1, policy_version 89662 (0.0010) [2023-10-11 22:36:53,903][71601] Updated weights for policy 0, policy_version 89730 (0.0009) [2023-10-11 22:36:54,264][71601] Updated weights for policy 0, policy_version 89740 (0.0010) [2023-10-11 22:36:54,635][71601] Updated weights for policy 0, policy_version 89750 (0.0007) [2023-10-11 22:36:54,992][71601] Updated weights for policy 0, policy_version 89760 (0.0008) [2023-10-11 22:36:56,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 183730176. Throughput: 0: 1806.8, 1: 1811.0. Samples: 45944274. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-11 22:36:56,034][70582] Avg episode reward: [(0, '123.040'), (1, '106.120')] [2023-10-11 22:36:56,240][71635] Updated weights for policy 1, policy_version 89672 (0.0008) [2023-10-11 22:36:56,601][71635] Updated weights for policy 1, policy_version 89682 (0.0007) [2023-10-11 22:36:56,963][71635] Updated weights for policy 1, policy_version 89692 (0.0009) [2023-10-11 22:36:58,664][71601] Updated weights for policy 0, policy_version 89770 (0.0007) [2023-10-11 22:36:59,036][71601] Updated weights for policy 0, policy_version 89780 (0.0008) [2023-10-11 22:36:59,404][71601] Updated weights for policy 0, policy_version 89790 (0.0007) [2023-10-11 22:37:00,610][71635] Updated weights for policy 1, policy_version 89702 (0.0007) [2023-10-11 22:37:00,981][71635] Updated weights for policy 1, policy_version 89712 (0.0010) [2023-10-11 22:37:01,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 183795712. Throughput: 0: 1818.4, 1: 1815.6. Samples: 45955676. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:37:01,035][70582] Avg episode reward: [(0, '132.290'), (1, '105.840')] [2023-10-11 22:37:01,342][71635] Updated weights for policy 1, policy_version 89722 (0.0008) [2023-10-11 22:37:03,056][71601] Updated weights for policy 0, policy_version 89800 (0.0007) [2023-10-11 22:37:03,432][71601] Updated weights for policy 0, policy_version 89810 (0.0007) [2023-10-11 22:37:03,796][71601] Updated weights for policy 0, policy_version 89820 (0.0008) [2023-10-11 22:37:04,973][71635] Updated weights for policy 1, policy_version 89732 (0.0008) [2023-10-11 22:37:05,338][71635] Updated weights for policy 1, policy_version 89742 (0.0010) [2023-10-11 22:37:05,704][71635] Updated weights for policy 1, policy_version 89752 (0.0007) [2023-10-11 22:37:06,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 183894016. Throughput: 0: 1811.6, 1: 1818.6. Samples: 45976984. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:37:06,035][70582] Avg episode reward: [(0, '131.870'), (1, '114.130')] [2023-10-11 22:37:07,369][71601] Updated weights for policy 0, policy_version 89830 (0.0009) [2023-10-11 22:37:07,745][71601] Updated weights for policy 0, policy_version 89840 (0.0007) [2023-10-11 22:37:08,125][71601] Updated weights for policy 0, policy_version 89850 (0.0008) [2023-10-11 22:37:09,412][71635] Updated weights for policy 1, policy_version 89762 (0.0008) [2023-10-11 22:37:09,776][71635] Updated weights for policy 1, policy_version 89772 (0.0007) [2023-10-11 22:37:10,142][71635] Updated weights for policy 1, policy_version 89782 (0.0010) [2023-10-11 22:37:10,512][71635] Updated weights for policy 1, policy_version 89792 (0.0010) [2023-10-11 22:37:11,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 183959552. Throughput: 0: 1808.8, 1: 1816.0. Samples: 45998350. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:37:11,035][70582] Avg episode reward: [(0, '128.710'), (1, '111.790')] [2023-10-11 22:37:11,917][71601] Updated weights for policy 0, policy_version 89860 (0.0010) [2023-10-11 22:37:12,283][71601] Updated weights for policy 0, policy_version 89870 (0.0009) [2023-10-11 22:37:12,655][71601] Updated weights for policy 0, policy_version 89880 (0.0007) [2023-10-11 22:37:14,136][71635] Updated weights for policy 1, policy_version 89802 (0.0008) [2023-10-11 22:37:14,493][71635] Updated weights for policy 1, policy_version 89812 (0.0008) [2023-10-11 22:37:14,856][71635] Updated weights for policy 1, policy_version 89822 (0.0009) [2023-10-11 22:37:16,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 184025088. Throughput: 0: 1806.9, 1: 1822.9. Samples: 46009572. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:37:16,034][70582] Avg episode reward: [(0, '128.370'), (1, '111.670')] [2023-10-11 22:37:16,353][71601] Updated weights for policy 0, policy_version 89890 (0.0009) [2023-10-11 22:37:16,733][71601] Updated weights for policy 0, policy_version 89900 (0.0010) [2023-10-11 22:37:17,116][71601] Updated weights for policy 0, policy_version 89910 (0.0008) [2023-10-11 22:37:17,490][71601] Updated weights for policy 0, policy_version 89920 (0.0009) [2023-10-11 22:37:18,489][71635] Updated weights for policy 1, policy_version 89832 (0.0008) [2023-10-11 22:37:18,848][71635] Updated weights for policy 1, policy_version 89842 (0.0009) [2023-10-11 22:37:19,216][71635] Updated weights for policy 1, policy_version 89852 (0.0010) [2023-10-11 22:37:21,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 184090624. Throughput: 0: 1815.6, 1: 1816.9. Samples: 46031086. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:37:21,034][70582] Avg episode reward: [(0, '133.200'), (1, '113.850')] [2023-10-11 22:37:21,171][71601] Updated weights for policy 0, policy_version 89930 (0.0009) [2023-10-11 22:37:21,543][71601] Updated weights for policy 0, policy_version 89940 (0.0009) [2023-10-11 22:37:21,907][71601] Updated weights for policy 0, policy_version 89950 (0.0010) [2023-10-11 22:37:23,062][71635] Updated weights for policy 1, policy_version 89862 (0.0009) [2023-10-11 22:37:23,438][71635] Updated weights for policy 1, policy_version 89872 (0.0008) [2023-10-11 22:37:23,788][71635] Updated weights for policy 1, policy_version 89882 (0.0009) [2023-10-11 22:37:25,725][71601] Updated weights for policy 0, policy_version 89960 (0.0009) [2023-10-11 22:37:26,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 184156160. Throughput: 0: 1820.0, 1: 1825.9. Samples: 46053706. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:37:26,035][70582] Avg episode reward: [(0, '134.230'), (1, '114.320')] [2023-10-11 22:37:26,095][71601] Updated weights for policy 0, policy_version 89970 (0.0007) [2023-10-11 22:37:26,469][71601] Updated weights for policy 0, policy_version 89980 (0.0007) [2023-10-11 22:37:27,476][71635] Updated weights for policy 1, policy_version 89892 (0.0009) [2023-10-11 22:37:27,875][71635] Updated weights for policy 1, policy_version 89902 (0.0011) [2023-10-11 22:37:28,231][71635] Updated weights for policy 1, policy_version 89912 (0.0010) [2023-10-11 22:37:30,119][71601] Updated weights for policy 0, policy_version 89990 (0.0008) [2023-10-11 22:37:30,490][71601] Updated weights for policy 0, policy_version 90000 (0.0011) [2023-10-11 22:37:30,862][71601] Updated weights for policy 0, policy_version 90010 (0.0011) [2023-10-11 22:37:31,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 184221696. Throughput: 0: 1811.7, 1: 1818.9. Samples: 46063794. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:37:31,034][70582] Avg episode reward: [(0, '133.440'), (1, '120.620')] [2023-10-11 22:37:31,712][71635] Updated weights for policy 1, policy_version 89922 (0.0008) [2023-10-11 22:37:32,071][71635] Updated weights for policy 1, policy_version 89932 (0.0010) [2023-10-11 22:37:32,451][71635] Updated weights for policy 1, policy_version 89942 (0.0011) [2023-10-11 22:37:32,807][71635] Updated weights for policy 1, policy_version 89952 (0.0008) [2023-10-11 22:37:34,607][71601] Updated weights for policy 0, policy_version 90020 (0.0008) [2023-10-11 22:37:34,971][71601] Updated weights for policy 0, policy_version 90030 (0.0008) [2023-10-11 22:37:35,336][71601] Updated weights for policy 0, policy_version 90040 (0.0009) [2023-10-11 22:37:36,034][70582] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 184320000. Throughput: 0: 1815.1, 1: 1830.2. Samples: 46086568. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:37:36,034][70582] Avg episode reward: [(0, '128.450'), (1, '116.880')] [2023-10-11 22:37:36,401][71635] Updated weights for policy 1, policy_version 89962 (0.0007) [2023-10-11 22:37:36,766][71635] Updated weights for policy 1, policy_version 89972 (0.0007) [2023-10-11 22:37:37,137][71635] Updated weights for policy 1, policy_version 89982 (0.0007) [2023-10-11 22:37:38,995][71601] Updated weights for policy 0, policy_version 90050 (0.0007) [2023-10-11 22:37:39,363][71601] Updated weights for policy 0, policy_version 90060 (0.0008) [2023-10-11 22:37:39,749][71601] Updated weights for policy 0, policy_version 90070 (0.0011) [2023-10-11 22:37:40,106][71601] Updated weights for policy 0, policy_version 90080 (0.0011) [2023-10-11 22:37:40,805][71635] Updated weights for policy 1, policy_version 89992 (0.0008) [2023-10-11 22:37:41,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 184385536. Throughput: 0: 1807.8, 1: 1827.4. Samples: 46107858. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:37:41,035][70582] Avg episode reward: [(0, '124.010'), (1, '113.210')] [2023-10-11 22:37:41,172][71635] Updated weights for policy 1, policy_version 90002 (0.0009) [2023-10-11 22:37:41,530][71635] Updated weights for policy 1, policy_version 90012 (0.0009) [2023-10-11 22:37:43,831][71601] Updated weights for policy 0, policy_version 90090 (0.0007) [2023-10-11 22:37:44,191][71601] Updated weights for policy 0, policy_version 90100 (0.0008) [2023-10-11 22:37:44,559][71601] Updated weights for policy 0, policy_version 90110 (0.0008) [2023-10-11 22:37:45,202][71635] Updated weights for policy 1, policy_version 90022 (0.0009) [2023-10-11 22:37:45,560][71635] Updated weights for policy 1, policy_version 90032 (0.0008) [2023-10-11 22:37:45,932][71635] Updated weights for policy 1, policy_version 90042 (0.0009) [2023-10-11 22:37:46,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 184451072. Throughput: 0: 1809.7, 1: 1822.2. Samples: 46119110. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:37:46,034][70582] Avg episode reward: [(0, '129.790'), (1, '112.070')] [2023-10-11 22:37:48,190][71601] Updated weights for policy 0, policy_version 90120 (0.0008) [2023-10-11 22:37:48,559][71601] Updated weights for policy 0, policy_version 90130 (0.0009) [2023-10-11 22:37:48,928][71601] Updated weights for policy 0, policy_version 90140 (0.0008) [2023-10-11 22:37:49,781][71635] Updated weights for policy 1, policy_version 90052 (0.0009) [2023-10-11 22:37:50,153][71635] Updated weights for policy 1, policy_version 90062 (0.0008) [2023-10-11 22:37:50,515][71635] Updated weights for policy 1, policy_version 90072 (0.0009) [2023-10-11 22:37:51,034][70582] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 184549376. Throughput: 0: 1809.7, 1: 1825.7. Samples: 46140576. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:37:51,034][70582] Avg episode reward: [(0, '131.180'), (1, '114.460')] [2023-10-11 22:37:52,466][71601] Updated weights for policy 0, policy_version 90150 (0.0009) [2023-10-11 22:37:52,834][71601] Updated weights for policy 0, policy_version 90160 (0.0008) [2023-10-11 22:37:53,212][71601] Updated weights for policy 0, policy_version 90170 (0.0009) [2023-10-11 22:37:54,159][71635] Updated weights for policy 1, policy_version 90082 (0.0008) [2023-10-11 22:37:54,530][71635] Updated weights for policy 1, policy_version 90092 (0.0008) [2023-10-11 22:37:54,900][71635] Updated weights for policy 1, policy_version 90102 (0.0008) [2023-10-11 22:37:55,265][71635] Updated weights for policy 1, policy_version 90112 (0.0008) [2023-10-11 22:37:56,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 184614912. Throughput: 0: 1812.1, 1: 1826.9. Samples: 46162104. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:37:56,034][70582] Avg episode reward: [(0, '122.520'), (1, '116.280')] [2023-10-11 22:37:56,043][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000090176_92340224.pth... [2023-10-11 22:37:56,043][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000090112_92274688.pth... [2023-10-11 22:37:56,073][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000088480_90603520.pth [2023-10-11 22:37:56,078][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000088416_90537984.pth [2023-10-11 22:37:56,962][71601] Updated weights for policy 0, policy_version 90180 (0.0007) [2023-10-11 22:37:57,336][71601] Updated weights for policy 0, policy_version 90190 (0.0007) [2023-10-11 22:37:57,691][71601] Updated weights for policy 0, policy_version 90200 (0.0009) [2023-10-11 22:37:58,808][71635] Updated weights for policy 1, policy_version 90122 (0.0009) [2023-10-11 22:37:59,164][71635] Updated weights for policy 1, policy_version 90132 (0.0010) [2023-10-11 22:37:59,537][71635] Updated weights for policy 1, policy_version 90142 (0.0008) [2023-10-11 22:38:01,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 184680448. Throughput: 0: 1811.7, 1: 1839.0. Samples: 46173856. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-11 22:38:01,034][70582] Avg episode reward: [(0, '122.970'), (1, '109.480')] [2023-10-11 22:38:01,441][71601] Updated weights for policy 0, policy_version 90210 (0.0008) [2023-10-11 22:38:01,823][71601] Updated weights for policy 0, policy_version 90220 (0.0008) [2023-10-11 22:38:02,189][71601] Updated weights for policy 0, policy_version 90230 (0.0008) [2023-10-11 22:38:02,568][71601] Updated weights for policy 0, policy_version 90240 (0.0008) [2023-10-11 22:38:03,342][71635] Updated weights for policy 1, policy_version 90152 (0.0007) [2023-10-11 22:38:03,707][71635] Updated weights for policy 1, policy_version 90162 (0.0008) [2023-10-11 22:38:04,085][71635] Updated weights for policy 1, policy_version 90172 (0.0010) [2023-10-11 22:38:06,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 184745984. Throughput: 0: 1809.1, 1: 1830.3. Samples: 46194858. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-11 22:38:06,035][70582] Avg episode reward: [(0, '116.830'), (1, '111.750')] [2023-10-11 22:38:06,312][71601] Updated weights for policy 0, policy_version 90250 (0.0009) [2023-10-11 22:38:06,681][71601] Updated weights for policy 0, policy_version 90260 (0.0008) [2023-10-11 22:38:07,056][71601] Updated weights for policy 0, policy_version 90270 (0.0007) [2023-10-11 22:38:07,579][71635] Updated weights for policy 1, policy_version 90182 (0.0010) [2023-10-11 22:38:07,942][71635] Updated weights for policy 1, policy_version 90192 (0.0007) [2023-10-11 22:38:08,319][71635] Updated weights for policy 1, policy_version 90202 (0.0007) [2023-10-11 22:38:10,789][71601] Updated weights for policy 0, policy_version 90280 (0.0007) [2023-10-11 22:38:11,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 184811520. Throughput: 0: 1810.1, 1: 1835.1. Samples: 46217740. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-11 22:38:11,035][70582] Avg episode reward: [(0, '126.410'), (1, '110.670')] [2023-10-11 22:38:11,161][71601] Updated weights for policy 0, policy_version 90290 (0.0008) [2023-10-11 22:38:11,535][71601] Updated weights for policy 0, policy_version 90300 (0.0008) [2023-10-11 22:38:12,123][71635] Updated weights for policy 1, policy_version 90212 (0.0007) [2023-10-11 22:38:12,486][71635] Updated weights for policy 1, policy_version 90222 (0.0007) [2023-10-11 22:38:12,849][71635] Updated weights for policy 1, policy_version 90232 (0.0010) [2023-10-11 22:38:15,262][71601] Updated weights for policy 0, policy_version 90310 (0.0008) [2023-10-11 22:38:15,635][71601] Updated weights for policy 0, policy_version 90320 (0.0008) [2023-10-11 22:38:16,003][71601] Updated weights for policy 0, policy_version 90330 (0.0007) [2023-10-11 22:38:16,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 184877056. Throughput: 0: 1805.5, 1: 1836.0. Samples: 46227660. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-11 22:38:16,035][70582] Avg episode reward: [(0, '120.720'), (1, '105.360')] [2023-10-11 22:38:16,631][71635] Updated weights for policy 1, policy_version 90242 (0.0010) [2023-10-11 22:38:17,032][71635] Updated weights for policy 1, policy_version 90252 (0.0009) [2023-10-11 22:38:17,410][71635] Updated weights for policy 1, policy_version 90262 (0.0010) [2023-10-11 22:38:17,770][71635] Updated weights for policy 1, policy_version 90272 (0.0011) [2023-10-11 22:38:19,763][71601] Updated weights for policy 0, policy_version 90340 (0.0009) [2023-10-11 22:38:20,150][71601] Updated weights for policy 0, policy_version 90350 (0.0011) [2023-10-11 22:38:20,521][71601] Updated weights for policy 0, policy_version 90360 (0.0009) [2023-10-11 22:38:21,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 184975360. Throughput: 0: 1807.3, 1: 1830.3. Samples: 46250260. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-11 22:38:21,034][70582] Avg episode reward: [(0, '116.640'), (1, '111.620')] [2023-10-11 22:38:21,528][71635] Updated weights for policy 1, policy_version 90282 (0.0009) [2023-10-11 22:38:21,895][71635] Updated weights for policy 1, policy_version 90292 (0.0010) [2023-10-11 22:38:22,255][71635] Updated weights for policy 1, policy_version 90302 (0.0008) [2023-10-11 22:38:24,246][71601] Updated weights for policy 0, policy_version 90370 (0.0009) [2023-10-11 22:38:24,613][71601] Updated weights for policy 0, policy_version 90380 (0.0007) [2023-10-11 22:38:24,983][71601] Updated weights for policy 0, policy_version 90390 (0.0007) [2023-10-11 22:38:25,358][71601] Updated weights for policy 0, policy_version 90400 (0.0008) [2023-10-11 22:38:25,845][71635] Updated weights for policy 1, policy_version 90312 (0.0009) [2023-10-11 22:38:26,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 185040896. Throughput: 0: 1806.9, 1: 1830.8. Samples: 46271554. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-11 22:38:26,034][70582] Avg episode reward: [(0, '120.160'), (1, '103.450')] [2023-10-11 22:38:26,209][71635] Updated weights for policy 1, policy_version 90322 (0.0008) [2023-10-11 22:38:26,576][71635] Updated weights for policy 1, policy_version 90332 (0.0010) [2023-10-11 22:38:29,088][71601] Updated weights for policy 0, policy_version 90410 (0.0008) [2023-10-11 22:38:29,458][71601] Updated weights for policy 0, policy_version 90420 (0.0008) [2023-10-11 22:38:29,830][71601] Updated weights for policy 0, policy_version 90430 (0.0007) [2023-10-11 22:38:30,296][71635] Updated weights for policy 1, policy_version 90342 (0.0010) [2023-10-11 22:38:30,669][71635] Updated weights for policy 1, policy_version 90352 (0.0007) [2023-10-11 22:38:31,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 185106432. Throughput: 0: 1807.5, 1: 1833.0. Samples: 46282932. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-11 22:38:31,035][70582] Avg episode reward: [(0, '117.920'), (1, '112.470')] [2023-10-11 22:38:31,045][71635] Updated weights for policy 1, policy_version 90362 (0.0007) [2023-10-11 22:38:33,400][71601] Updated weights for policy 0, policy_version 90440 (0.0009) [2023-10-11 22:38:33,779][71601] Updated weights for policy 0, policy_version 90450 (0.0010) [2023-10-11 22:38:34,146][71601] Updated weights for policy 0, policy_version 90460 (0.0010) [2023-10-11 22:38:34,865][71635] Updated weights for policy 1, policy_version 90372 (0.0008) [2023-10-11 22:38:35,237][71635] Updated weights for policy 1, policy_version 90382 (0.0009) [2023-10-11 22:38:35,607][71635] Updated weights for policy 1, policy_version 90392 (0.0010) [2023-10-11 22:38:36,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 185204736. Throughput: 0: 1806.4, 1: 1827.8. Samples: 46304114. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-11 22:38:36,035][70582] Avg episode reward: [(0, '118.990'), (1, '112.220')] [2023-10-11 22:38:37,813][71601] Updated weights for policy 0, policy_version 90470 (0.0011) [2023-10-11 22:38:38,190][71601] Updated weights for policy 0, policy_version 90480 (0.0010) [2023-10-11 22:38:38,565][71601] Updated weights for policy 0, policy_version 90490 (0.0009) [2023-10-11 22:38:39,178][71635] Updated weights for policy 1, policy_version 90402 (0.0008) [2023-10-11 22:38:39,535][71635] Updated weights for policy 1, policy_version 90412 (0.0008) [2023-10-11 22:38:39,906][71635] Updated weights for policy 1, policy_version 90422 (0.0008) [2023-10-11 22:38:40,272][71635] Updated weights for policy 1, policy_version 90432 (0.0008) [2023-10-11 22:38:41,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 185270272. Throughput: 0: 1807.7, 1: 1820.5. Samples: 46325370. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-11 22:38:41,034][70582] Avg episode reward: [(0, '125.400'), (1, '112.700')] [2023-10-11 22:38:42,259][71601] Updated weights for policy 0, policy_version 90500 (0.0010) [2023-10-11 22:38:42,642][71601] Updated weights for policy 0, policy_version 90510 (0.0008) [2023-10-11 22:38:43,022][71601] Updated weights for policy 0, policy_version 90520 (0.0008) [2023-10-11 22:38:43,917][71635] Updated weights for policy 1, policy_version 90442 (0.0010) [2023-10-11 22:38:44,284][71635] Updated weights for policy 1, policy_version 90452 (0.0010) [2023-10-11 22:38:44,648][71635] Updated weights for policy 1, policy_version 90462 (0.0011) [2023-10-11 22:38:46,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 185335808. Throughput: 0: 1805.4, 1: 1809.2. Samples: 46336516. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-11 22:38:46,034][70582] Avg episode reward: [(0, '129.710'), (1, '114.700')] [2023-10-11 22:38:46,769][71601] Updated weights for policy 0, policy_version 90530 (0.0010) [2023-10-11 22:38:47,137][71601] Updated weights for policy 0, policy_version 90540 (0.0009) [2023-10-11 22:38:47,520][71601] Updated weights for policy 0, policy_version 90550 (0.0008) [2023-10-11 22:38:47,888][71601] Updated weights for policy 0, policy_version 90560 (0.0008) [2023-10-11 22:38:48,447][71635] Updated weights for policy 1, policy_version 90472 (0.0007) [2023-10-11 22:38:48,823][71635] Updated weights for policy 1, policy_version 90482 (0.0009) [2023-10-11 22:38:49,180][71635] Updated weights for policy 1, policy_version 90492 (0.0008) [2023-10-11 22:38:51,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 185401344. Throughput: 0: 1811.1, 1: 1811.2. Samples: 46357864. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-11 22:38:51,035][70582] Avg episode reward: [(0, '128.640'), (1, '110.430')] [2023-10-11 22:38:51,732][71601] Updated weights for policy 0, policy_version 90570 (0.0009) [2023-10-11 22:38:52,098][71601] Updated weights for policy 0, policy_version 90580 (0.0009) [2023-10-11 22:38:52,475][71601] Updated weights for policy 0, policy_version 90590 (0.0008) [2023-10-11 22:38:52,802][71635] Updated weights for policy 1, policy_version 90502 (0.0010) [2023-10-11 22:38:53,171][71635] Updated weights for policy 1, policy_version 90512 (0.0009) [2023-10-11 22:38:53,538][71635] Updated weights for policy 1, policy_version 90522 (0.0009) [2023-10-11 22:38:56,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 185466880. Throughput: 0: 1810.5, 1: 1804.7. Samples: 46380424. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-11 22:38:56,034][70582] Avg episode reward: [(0, '129.810'), (1, '115.140')] [2023-10-11 22:38:56,106][71601] Updated weights for policy 0, policy_version 90600 (0.0008) [2023-10-11 22:38:56,475][71601] Updated weights for policy 0, policy_version 90610 (0.0007) [2023-10-11 22:38:56,846][71601] Updated weights for policy 0, policy_version 90620 (0.0010) [2023-10-11 22:38:57,177][71635] Updated weights for policy 1, policy_version 90532 (0.0009) [2023-10-11 22:38:57,541][71635] Updated weights for policy 1, policy_version 90542 (0.0008) [2023-10-11 22:38:57,893][71635] Updated weights for policy 1, policy_version 90552 (0.0007) [2023-10-11 22:39:00,567][71601] Updated weights for policy 0, policy_version 90630 (0.0010) [2023-10-11 22:39:00,936][71601] Updated weights for policy 0, policy_version 90640 (0.0010) [2023-10-11 22:39:01,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 185532416. Throughput: 0: 1811.8, 1: 1804.3. Samples: 46390382. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-11 22:39:01,035][70582] Avg episode reward: [(0, '139.400'), (1, '115.020')] [2023-10-11 22:39:01,304][71601] Updated weights for policy 0, policy_version 90650 (0.0010) [2023-10-11 22:39:01,726][71635] Updated weights for policy 1, policy_version 90562 (0.0009) [2023-10-11 22:39:02,093][71635] Updated weights for policy 1, policy_version 90572 (0.0010) [2023-10-11 22:39:02,459][71635] Updated weights for policy 1, policy_version 90582 (0.0009) [2023-10-11 22:39:02,826][71635] Updated weights for policy 1, policy_version 90592 (0.0010) [2023-10-11 22:39:05,047][71601] Updated weights for policy 0, policy_version 90660 (0.0008) [2023-10-11 22:39:05,414][71601] Updated weights for policy 0, policy_version 90670 (0.0008) [2023-10-11 22:39:05,781][71601] Updated weights for policy 0, policy_version 90680 (0.0007) [2023-10-11 22:39:06,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 185597952. Throughput: 0: 1809.8, 1: 1805.5. Samples: 46412948. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-11 22:39:06,034][70582] Avg episode reward: [(0, '135.460'), (1, '113.210')] [2023-10-11 22:39:06,677][71635] Updated weights for policy 1, policy_version 90602 (0.0010) [2023-10-11 22:39:07,047][71635] Updated weights for policy 1, policy_version 90612 (0.0011) [2023-10-11 22:39:07,420][71635] Updated weights for policy 1, policy_version 90622 (0.0007) [2023-10-11 22:39:09,508][71601] Updated weights for policy 0, policy_version 90690 (0.0007) [2023-10-11 22:39:09,882][71601] Updated weights for policy 0, policy_version 90700 (0.0007) [2023-10-11 22:39:10,260][71601] Updated weights for policy 0, policy_version 90710 (0.0007) [2023-10-11 22:39:10,628][71601] Updated weights for policy 0, policy_version 90720 (0.0007) [2023-10-11 22:39:11,004][71635] Updated weights for policy 1, policy_version 90632 (0.0007) [2023-10-11 22:39:11,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 185696256. Throughput: 0: 1814.4, 1: 1806.3. Samples: 46434482. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-11 22:39:11,034][70582] Avg episode reward: [(0, '137.810'), (1, '115.300')] [2023-10-11 22:39:11,369][71635] Updated weights for policy 1, policy_version 90642 (0.0007) [2023-10-11 22:39:11,734][71635] Updated weights for policy 1, policy_version 90652 (0.0010) [2023-10-11 22:39:14,122][71601] Updated weights for policy 0, policy_version 90730 (0.0008) [2023-10-11 22:39:14,504][71601] Updated weights for policy 0, policy_version 90740 (0.0009) [2023-10-11 22:39:14,866][71601] Updated weights for policy 0, policy_version 90750 (0.0008) [2023-10-11 22:39:15,370][71635] Updated weights for policy 1, policy_version 90662 (0.0009) [2023-10-11 22:39:15,747][71635] Updated weights for policy 1, policy_version 90672 (0.0010) [2023-10-11 22:39:16,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 185761792. Throughput: 0: 1811.7, 1: 1804.4. Samples: 46445654. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-11 22:39:16,035][70582] Avg episode reward: [(0, '145.750'), (1, '112.910')] [2023-10-11 22:39:16,123][71635] Updated weights for policy 1, policy_version 90682 (0.0009) [2023-10-11 22:39:18,430][71601] Updated weights for policy 0, policy_version 90760 (0.0009) [2023-10-11 22:39:18,791][71601] Updated weights for policy 0, policy_version 90770 (0.0008) [2023-10-11 22:39:19,164][71601] Updated weights for policy 0, policy_version 90780 (0.0009) [2023-10-11 22:39:19,847][71635] Updated weights for policy 1, policy_version 90692 (0.0008) [2023-10-11 22:39:20,208][71635] Updated weights for policy 1, policy_version 90702 (0.0008) [2023-10-11 22:39:20,578][71635] Updated weights for policy 1, policy_version 90712 (0.0007) [2023-10-11 22:39:21,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 185860096. Throughput: 0: 1816.3, 1: 1809.0. Samples: 46467252. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-11 22:39:21,034][70582] Avg episode reward: [(0, '142.800'), (1, '116.650')] [2023-10-11 22:39:23,030][71601] Updated weights for policy 0, policy_version 90790 (0.0011) [2023-10-11 22:39:23,407][71601] Updated weights for policy 0, policy_version 90800 (0.0009) [2023-10-11 22:39:23,772][71601] Updated weights for policy 0, policy_version 90810 (0.0010) [2023-10-11 22:39:24,272][71635] Updated weights for policy 1, policy_version 90722 (0.0008) [2023-10-11 22:39:24,635][71635] Updated weights for policy 1, policy_version 90732 (0.0008) [2023-10-11 22:39:25,012][71635] Updated weights for policy 1, policy_version 90742 (0.0009) [2023-10-11 22:39:25,373][71635] Updated weights for policy 1, policy_version 90752 (0.0009) [2023-10-11 22:39:26,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 185925632. Throughput: 0: 1812.3, 1: 1816.2. Samples: 46488654. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-11 22:39:26,035][70582] Avg episode reward: [(0, '145.520'), (1, '115.550')] [2023-10-11 22:39:27,554][71601] Updated weights for policy 0, policy_version 90820 (0.0009) [2023-10-11 22:39:27,917][71601] Updated weights for policy 0, policy_version 90830 (0.0008) [2023-10-11 22:39:28,298][71601] Updated weights for policy 0, policy_version 90840 (0.0007) [2023-10-11 22:39:29,156][71635] Updated weights for policy 1, policy_version 90762 (0.0008) [2023-10-11 22:39:29,524][71635] Updated weights for policy 1, policy_version 90772 (0.0009) [2023-10-11 22:39:29,894][71635] Updated weights for policy 1, policy_version 90782 (0.0007) [2023-10-11 22:39:31,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 185991168. Throughput: 0: 1821.2, 1: 1814.7. Samples: 46500132. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-11 22:39:31,035][70582] Avg episode reward: [(0, '146.570'), (1, '113.180')] [2023-10-11 22:39:31,769][71601] Updated weights for policy 0, policy_version 90850 (0.0010) [2023-10-11 22:39:32,132][71601] Updated weights for policy 0, policy_version 90860 (0.0007) [2023-10-11 22:39:32,508][71601] Updated weights for policy 0, policy_version 90870 (0.0008) [2023-10-11 22:39:32,870][71601] Updated weights for policy 0, policy_version 90880 (0.0007) [2023-10-11 22:39:33,462][71635] Updated weights for policy 1, policy_version 90792 (0.0007) [2023-10-11 22:39:33,825][71635] Updated weights for policy 1, policy_version 90802 (0.0009) [2023-10-11 22:39:34,201][71635] Updated weights for policy 1, policy_version 90812 (0.0009) [2023-10-11 22:39:36,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 186056704. Throughput: 0: 1819.9, 1: 1819.4. Samples: 46521632. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-11 22:39:36,034][70582] Avg episode reward: [(0, '145.930'), (1, '111.740')] [2023-10-11 22:39:36,578][71601] Updated weights for policy 0, policy_version 90890 (0.0008) [2023-10-11 22:39:36,948][71601] Updated weights for policy 0, policy_version 90900 (0.0007) [2023-10-11 22:39:37,321][71601] Updated weights for policy 0, policy_version 90910 (0.0008) [2023-10-11 22:39:37,923][71635] Updated weights for policy 1, policy_version 90822 (0.0008) [2023-10-11 22:39:38,291][71635] Updated weights for policy 1, policy_version 90832 (0.0008) [2023-10-11 22:39:38,656][71635] Updated weights for policy 1, policy_version 90842 (0.0008) [2023-10-11 22:39:40,903][71601] Updated weights for policy 0, policy_version 90920 (0.0008) [2023-10-11 22:39:41,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 186122240. Throughput: 0: 1822.9, 1: 1824.9. Samples: 46544574. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-11 22:39:41,035][70582] Avg episode reward: [(0, '148.280'), (1, '113.500')] [2023-10-11 22:39:41,276][71601] Updated weights for policy 0, policy_version 90930 (0.0009) [2023-10-11 22:39:41,647][71601] Updated weights for policy 0, policy_version 90940 (0.0007) [2023-10-11 22:39:42,430][71635] Updated weights for policy 1, policy_version 90852 (0.0009) [2023-10-11 22:39:42,793][71635] Updated weights for policy 1, policy_version 90862 (0.0007) [2023-10-11 22:39:43,166][71635] Updated weights for policy 1, policy_version 90872 (0.0010) [2023-10-11 22:39:45,419][71601] Updated weights for policy 0, policy_version 90950 (0.0009) [2023-10-11 22:39:45,797][71601] Updated weights for policy 0, policy_version 90960 (0.0009) [2023-10-11 22:39:46,034][70582] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 186187776. Throughput: 0: 1822.4, 1: 1828.1. Samples: 46554656. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-11 22:39:46,035][70582] Avg episode reward: [(0, '146.100'), (1, '119.730')] [2023-10-11 22:39:46,165][71601] Updated weights for policy 0, policy_version 90970 (0.0010) [2023-10-11 22:39:46,751][71635] Updated weights for policy 1, policy_version 90882 (0.0009) [2023-10-11 22:39:47,128][71635] Updated weights for policy 1, policy_version 90892 (0.0007) [2023-10-11 22:39:47,494][71635] Updated weights for policy 1, policy_version 90902 (0.0008) [2023-10-11 22:39:47,858][71635] Updated weights for policy 1, policy_version 90912 (0.0009) [2023-10-11 22:39:50,030][71601] Updated weights for policy 0, policy_version 90980 (0.0007) [2023-10-11 22:39:50,414][71601] Updated weights for policy 0, policy_version 90990 (0.0010) [2023-10-11 22:39:50,775][71601] Updated weights for policy 0, policy_version 91000 (0.0010) [2023-10-11 22:39:51,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 186253312. Throughput: 0: 1821.0, 1: 1834.0. Samples: 46577424. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-11 22:39:51,035][70582] Avg episode reward: [(0, '160.060'), (1, '125.880')] [2023-10-11 22:39:51,529][71635] Updated weights for policy 1, policy_version 90922 (0.0007) [2023-10-11 22:39:51,895][71635] Updated weights for policy 1, policy_version 90932 (0.0008) [2023-10-11 22:39:52,260][71635] Updated weights for policy 1, policy_version 90942 (0.0009) [2023-10-11 22:39:54,353][71601] Updated weights for policy 0, policy_version 91010 (0.0008) [2023-10-11 22:39:54,718][71601] Updated weights for policy 0, policy_version 91020 (0.0008) [2023-10-11 22:39:55,089][71601] Updated weights for policy 0, policy_version 91030 (0.0008) [2023-10-11 22:39:55,460][71601] Updated weights for policy 0, policy_version 91040 (0.0010) [2023-10-11 22:39:55,970][71635] Updated weights for policy 1, policy_version 90952 (0.0010) [2023-10-11 22:39:56,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 186351616. Throughput: 0: 1825.0, 1: 1827.1. Samples: 46598826. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-11 22:39:56,034][70582] Avg episode reward: [(0, '162.960'), (1, '122.010')] [2023-10-11 22:39:56,041][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000091040_93224960.pth... [2023-10-11 22:39:56,075][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000089344_91488256.pth [2023-10-11 22:39:56,348][71635] Updated weights for policy 1, policy_version 90962 (0.0010) [2023-10-11 22:39:56,711][71635] Updated weights for policy 1, policy_version 90972 (0.0011) [2023-10-11 22:39:56,859][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000090976_93159424.pth... [2023-10-11 22:39:56,896][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000089248_91389952.pth [2023-10-11 22:39:59,226][71601] Updated weights for policy 0, policy_version 91050 (0.0008) [2023-10-11 22:39:59,608][71601] Updated weights for policy 0, policy_version 91060 (0.0008) [2023-10-11 22:39:59,985][71601] Updated weights for policy 0, policy_version 91070 (0.0009) [2023-10-11 22:40:00,406][71635] Updated weights for policy 1, policy_version 90982 (0.0010) [2023-10-11 22:40:00,774][71635] Updated weights for policy 1, policy_version 90992 (0.0010) [2023-10-11 22:40:01,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 186417152. Throughput: 0: 1829.6, 1: 1831.9. Samples: 46610420. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-11 22:40:01,034][70582] Avg episode reward: [(0, '157.490'), (1, '117.990')] [2023-10-11 22:40:01,144][71635] Updated weights for policy 1, policy_version 91002 (0.0010) [2023-10-11 22:40:03,693][71601] Updated weights for policy 0, policy_version 91080 (0.0007) [2023-10-11 22:40:04,058][71601] Updated weights for policy 0, policy_version 91090 (0.0010) [2023-10-11 22:40:04,429][71601] Updated weights for policy 0, policy_version 91100 (0.0007) [2023-10-11 22:40:04,813][71635] Updated weights for policy 1, policy_version 91012 (0.0009) [2023-10-11 22:40:05,174][71635] Updated weights for policy 1, policy_version 91022 (0.0007) [2023-10-11 22:40:05,537][71635] Updated weights for policy 1, policy_version 91032 (0.0008) [2023-10-11 22:40:06,034][70582] Fps is (10 sec: 16384.3, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 186515456. Throughput: 0: 1828.8, 1: 1825.5. Samples: 46631696. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-11 22:40:06,034][70582] Avg episode reward: [(0, '163.200'), (1, '121.340')] [2023-10-11 22:40:08,062][71601] Updated weights for policy 0, policy_version 91110 (0.0008) [2023-10-11 22:40:08,434][71601] Updated weights for policy 0, policy_version 91120 (0.0008) [2023-10-11 22:40:08,811][71601] Updated weights for policy 0, policy_version 91130 (0.0008) [2023-10-11 22:40:09,189][71635] Updated weights for policy 1, policy_version 91042 (0.0008) [2023-10-11 22:40:09,549][71635] Updated weights for policy 1, policy_version 91052 (0.0009) [2023-10-11 22:40:09,921][71635] Updated weights for policy 1, policy_version 91062 (0.0010) [2023-10-11 22:40:10,295][71635] Updated weights for policy 1, policy_version 91072 (0.0011) [2023-10-11 22:40:11,034][70582] Fps is (10 sec: 16383.3, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 186580992. Throughput: 0: 1827.1, 1: 1822.7. Samples: 46652894. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-11 22:40:11,036][70582] Avg episode reward: [(0, '154.020'), (1, '124.070')] [2023-10-11 22:40:12,445][71601] Updated weights for policy 0, policy_version 91140 (0.0009) [2023-10-11 22:40:12,812][71601] Updated weights for policy 0, policy_version 91150 (0.0008) [2023-10-11 22:40:13,178][71601] Updated weights for policy 0, policy_version 91160 (0.0008) [2023-10-11 22:40:13,990][71635] Updated weights for policy 1, policy_version 91082 (0.0010) [2023-10-11 22:40:14,359][71635] Updated weights for policy 1, policy_version 91092 (0.0007) [2023-10-11 22:40:14,732][71635] Updated weights for policy 1, policy_version 91102 (0.0009) [2023-10-11 22:40:16,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 186646528. Throughput: 0: 1826.7, 1: 1826.5. Samples: 46664526. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-11 22:40:16,034][70582] Avg episode reward: [(0, '154.950'), (1, '117.120')] [2023-10-11 22:40:16,824][71601] Updated weights for policy 0, policy_version 91170 (0.0008) [2023-10-11 22:40:17,196][71601] Updated weights for policy 0, policy_version 91180 (0.0007) [2023-10-11 22:40:17,564][71601] Updated weights for policy 0, policy_version 91190 (0.0010) [2023-10-11 22:40:17,942][71601] Updated weights for policy 0, policy_version 91200 (0.0008) [2023-10-11 22:40:18,351][71635] Updated weights for policy 1, policy_version 91112 (0.0009) [2023-10-11 22:40:18,717][71635] Updated weights for policy 1, policy_version 91122 (0.0009) [2023-10-11 22:40:19,088][71635] Updated weights for policy 1, policy_version 91132 (0.0009) [2023-10-11 22:40:21,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 186712064. Throughput: 0: 1821.5, 1: 1827.0. Samples: 46685818. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-11 22:40:21,035][70582] Avg episode reward: [(0, '155.800'), (1, '124.080')] [2023-10-11 22:40:21,639][71601] Updated weights for policy 0, policy_version 91210 (0.0007) [2023-10-11 22:40:22,008][71601] Updated weights for policy 0, policy_version 91220 (0.0007) [2023-10-11 22:40:22,388][71601] Updated weights for policy 0, policy_version 91230 (0.0011) [2023-10-11 22:40:22,853][71635] Updated weights for policy 1, policy_version 91142 (0.0009) [2023-10-11 22:40:23,221][71635] Updated weights for policy 1, policy_version 91152 (0.0009) [2023-10-11 22:40:23,580][71635] Updated weights for policy 1, policy_version 91162 (0.0008) [2023-10-11 22:40:26,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 186777600. Throughput: 0: 1821.3, 1: 1819.6. Samples: 46708416. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-11 22:40:26,034][70582] Avg episode reward: [(0, '149.800'), (1, '117.950')] [2023-10-11 22:40:26,162][71601] Updated weights for policy 0, policy_version 91240 (0.0009) [2023-10-11 22:40:26,537][71601] Updated weights for policy 0, policy_version 91250 (0.0007) [2023-10-11 22:40:26,916][71601] Updated weights for policy 0, policy_version 91260 (0.0010) [2023-10-11 22:40:27,274][71635] Updated weights for policy 1, policy_version 91172 (0.0009) [2023-10-11 22:40:27,634][71635] Updated weights for policy 1, policy_version 91182 (0.0008) [2023-10-11 22:40:28,003][71635] Updated weights for policy 1, policy_version 91192 (0.0008) [2023-10-11 22:40:30,513][71601] Updated weights for policy 0, policy_version 91270 (0.0009) [2023-10-11 22:40:30,884][71601] Updated weights for policy 0, policy_version 91280 (0.0008) [2023-10-11 22:40:31,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 186843136. Throughput: 0: 1826.0, 1: 1819.2. Samples: 46718688. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-11 22:40:31,035][70582] Avg episode reward: [(0, '154.230'), (1, '113.850')] [2023-10-11 22:40:31,263][71601] Updated weights for policy 0, policy_version 91290 (0.0007) [2023-10-11 22:40:31,625][71635] Updated weights for policy 1, policy_version 91202 (0.0008) [2023-10-11 22:40:32,000][71635] Updated weights for policy 1, policy_version 91212 (0.0011) [2023-10-11 22:40:32,357][71635] Updated weights for policy 1, policy_version 91222 (0.0010) [2023-10-11 22:40:32,730][71635] Updated weights for policy 1, policy_version 91232 (0.0010) [2023-10-11 22:40:34,815][71601] Updated weights for policy 0, policy_version 91300 (0.0009) [2023-10-11 22:40:35,195][71601] Updated weights for policy 0, policy_version 91310 (0.0008) [2023-10-11 22:40:35,572][71601] Updated weights for policy 0, policy_version 91320 (0.0008) [2023-10-11 22:40:36,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 186941440. Throughput: 0: 1830.7, 1: 1811.9. Samples: 46741342. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-11 22:40:36,035][70582] Avg episode reward: [(0, '152.790'), (1, '111.490')] [2023-10-11 22:40:36,593][71635] Updated weights for policy 1, policy_version 91242 (0.0009) [2023-10-11 22:40:36,965][71635] Updated weights for policy 1, policy_version 91252 (0.0009) [2023-10-11 22:40:37,337][71635] Updated weights for policy 1, policy_version 91262 (0.0010) [2023-10-11 22:40:39,139][71601] Updated weights for policy 0, policy_version 91330 (0.0008) [2023-10-11 22:40:39,516][71601] Updated weights for policy 0, policy_version 91340 (0.0010) [2023-10-11 22:40:39,883][71601] Updated weights for policy 0, policy_version 91350 (0.0009) [2023-10-11 22:40:40,251][71601] Updated weights for policy 0, policy_version 91360 (0.0008) [2023-10-11 22:40:41,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 187006976. Throughput: 0: 1829.3, 1: 1813.4. Samples: 46762746. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-11 22:40:41,034][70582] Avg episode reward: [(0, '153.720'), (1, '108.370')] [2023-10-11 22:40:41,229][71635] Updated weights for policy 1, policy_version 91272 (0.0009) [2023-10-11 22:40:41,597][71635] Updated weights for policy 1, policy_version 91282 (0.0007) [2023-10-11 22:40:41,967][71635] Updated weights for policy 1, policy_version 91292 (0.0007) [2023-10-11 22:40:44,096][71601] Updated weights for policy 0, policy_version 91370 (0.0008) [2023-10-11 22:40:44,466][71601] Updated weights for policy 0, policy_version 91380 (0.0007) [2023-10-11 22:40:44,834][71601] Updated weights for policy 0, policy_version 91390 (0.0008) [2023-10-11 22:40:45,540][71635] Updated weights for policy 1, policy_version 91302 (0.0008) [2023-10-11 22:40:45,907][71635] Updated weights for policy 1, policy_version 91312 (0.0007) [2023-10-11 22:40:46,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 187072512. Throughput: 0: 1825.5, 1: 1810.0. Samples: 46774020. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-11 22:40:46,034][70582] Avg episode reward: [(0, '147.260'), (1, '112.670')] [2023-10-11 22:40:46,272][71635] Updated weights for policy 1, policy_version 91322 (0.0011) [2023-10-11 22:40:48,438][71601] Updated weights for policy 0, policy_version 91400 (0.0008) [2023-10-11 22:40:48,820][71601] Updated weights for policy 0, policy_version 91410 (0.0008) [2023-10-11 22:40:49,185][71601] Updated weights for policy 0, policy_version 91420 (0.0008) [2023-10-11 22:40:49,944][71635] Updated weights for policy 1, policy_version 91332 (0.0009) [2023-10-11 22:40:50,310][71635] Updated weights for policy 1, policy_version 91342 (0.0008) [2023-10-11 22:40:50,679][71635] Updated weights for policy 1, policy_version 91352 (0.0007) [2023-10-11 22:40:51,034][70582] Fps is (10 sec: 16384.2, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 187170816. Throughput: 0: 1820.1, 1: 1818.2. Samples: 46795422. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-11 22:40:51,034][70582] Avg episode reward: [(0, '137.140'), (1, '110.850')] [2023-10-11 22:40:52,610][71601] Updated weights for policy 0, policy_version 91430 (0.0010) [2023-10-11 22:40:52,986][71601] Updated weights for policy 0, policy_version 91440 (0.0007) [2023-10-11 22:40:53,353][71601] Updated weights for policy 0, policy_version 91450 (0.0008) [2023-10-11 22:40:54,340][71635] Updated weights for policy 1, policy_version 91362 (0.0008) [2023-10-11 22:40:54,715][71635] Updated weights for policy 1, policy_version 91372 (0.0008) [2023-10-11 22:40:55,081][71635] Updated weights for policy 1, policy_version 91382 (0.0008) [2023-10-11 22:40:55,451][71635] Updated weights for policy 1, policy_version 91392 (0.0007) [2023-10-11 22:40:56,034][70582] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 187236352. Throughput: 0: 1827.3, 1: 1819.8. Samples: 46817016. Policy #0 lag: (min: 10.0, avg: 17.8, max: 42.0) [2023-10-11 22:40:56,035][70582] Avg episode reward: [(0, '134.600'), (1, '117.880')] [2023-10-11 22:40:57,243][71601] Updated weights for policy 0, policy_version 91460 (0.0009) [2023-10-11 22:40:57,607][71601] Updated weights for policy 0, policy_version 91470 (0.0008) [2023-10-11 22:40:57,985][71601] Updated weights for policy 0, policy_version 91480 (0.0007) [2023-10-11 22:40:59,095][71635] Updated weights for policy 1, policy_version 91402 (0.0010) [2023-10-11 22:40:59,460][71635] Updated weights for policy 1, policy_version 91412 (0.0010) [2023-10-11 22:40:59,835][71635] Updated weights for policy 1, policy_version 91422 (0.0009) [2023-10-11 22:41:01,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 187301888. Throughput: 0: 1817.5, 1: 1816.1. Samples: 46828036. Policy #0 lag: (min: 10.0, avg: 17.8, max: 42.0) [2023-10-11 22:41:01,034][70582] Avg episode reward: [(0, '137.410'), (1, '114.230')] [2023-10-11 22:41:01,760][71601] Updated weights for policy 0, policy_version 91490 (0.0007) [2023-10-11 22:41:02,131][71601] Updated weights for policy 0, policy_version 91500 (0.0007) [2023-10-11 22:41:02,510][71601] Updated weights for policy 0, policy_version 91510 (0.0008) [2023-10-11 22:41:02,875][71601] Updated weights for policy 0, policy_version 91520 (0.0009) [2023-10-11 22:41:03,533][71635] Updated weights for policy 1, policy_version 91432 (0.0009) [2023-10-11 22:41:03,911][71635] Updated weights for policy 1, policy_version 91442 (0.0008) [2023-10-11 22:41:04,273][71635] Updated weights for policy 1, policy_version 91452 (0.0008) [2023-10-11 22:41:06,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 187367424. Throughput: 0: 1822.2, 1: 1814.4. Samples: 46849466. Policy #0 lag: (min: 10.0, avg: 17.8, max: 42.0) [2023-10-11 22:41:06,034][70582] Avg episode reward: [(0, '137.420'), (1, '113.180')] [2023-10-11 22:41:06,552][71601] Updated weights for policy 0, policy_version 91530 (0.0009) [2023-10-11 22:41:06,911][71601] Updated weights for policy 0, policy_version 91540 (0.0011) [2023-10-11 22:41:07,279][71601] Updated weights for policy 0, policy_version 91550 (0.0010) [2023-10-11 22:41:08,041][71635] Updated weights for policy 1, policy_version 91462 (0.0008) [2023-10-11 22:41:08,406][71635] Updated weights for policy 1, policy_version 91472 (0.0010) [2023-10-11 22:41:08,781][71635] Updated weights for policy 1, policy_version 91482 (0.0010) [2023-10-11 22:41:11,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.6, 300 sec: 14551.2). Total num frames: 187432960. Throughput: 0: 1821.3, 1: 1812.4. Samples: 46871932. Policy #0 lag: (min: 10.0, avg: 17.8, max: 42.0) [2023-10-11 22:41:11,034][70582] Avg episode reward: [(0, '127.790'), (1, '115.500')] [2023-10-11 22:41:11,122][71601] Updated weights for policy 0, policy_version 91560 (0.0009) [2023-10-11 22:41:11,495][71601] Updated weights for policy 0, policy_version 91570 (0.0007) [2023-10-11 22:41:11,878][71601] Updated weights for policy 0, policy_version 91580 (0.0008) [2023-10-11 22:41:12,423][71635] Updated weights for policy 1, policy_version 91492 (0.0008) [2023-10-11 22:41:12,803][71635] Updated weights for policy 1, policy_version 91502 (0.0009) [2023-10-11 22:41:13,176][71635] Updated weights for policy 1, policy_version 91512 (0.0008) [2023-10-11 22:41:15,487][71601] Updated weights for policy 0, policy_version 91590 (0.0010) [2023-10-11 22:41:15,852][71601] Updated weights for policy 0, policy_version 91600 (0.0010) [2023-10-11 22:41:16,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 187498496. Throughput: 0: 1813.7, 1: 1813.8. Samples: 46881922. Policy #0 lag: (min: 10.0, avg: 17.8, max: 42.0) [2023-10-11 22:41:16,034][70582] Avg episode reward: [(0, '136.020'), (1, '108.680')] [2023-10-11 22:41:16,227][71601] Updated weights for policy 0, policy_version 91610 (0.0010) [2023-10-11 22:41:16,818][71635] Updated weights for policy 1, policy_version 91522 (0.0008) [2023-10-11 22:41:17,184][71635] Updated weights for policy 1, policy_version 91532 (0.0008) [2023-10-11 22:41:17,553][71635] Updated weights for policy 1, policy_version 91542 (0.0008) [2023-10-11 22:41:17,913][71635] Updated weights for policy 1, policy_version 91552 (0.0007) [2023-10-11 22:41:19,766][71601] Updated weights for policy 0, policy_version 91620 (0.0007) [2023-10-11 22:41:20,129][71601] Updated weights for policy 0, policy_version 91630 (0.0007) [2023-10-11 22:41:20,511][71601] Updated weights for policy 0, policy_version 91640 (0.0008) [2023-10-11 22:41:21,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 187596800. Throughput: 0: 1812.9, 1: 1816.7. Samples: 46904672. Policy #0 lag: (min: 10.0, avg: 17.8, max: 42.0) [2023-10-11 22:41:21,034][70582] Avg episode reward: [(0, '136.110'), (1, '110.770')] [2023-10-11 22:41:21,610][71635] Updated weights for policy 1, policy_version 91562 (0.0007) [2023-10-11 22:41:21,986][71635] Updated weights for policy 1, policy_version 91572 (0.0008) [2023-10-11 22:41:22,358][71635] Updated weights for policy 1, policy_version 91582 (0.0010) [2023-10-11 22:41:24,114][71601] Updated weights for policy 0, policy_version 91650 (0.0008) [2023-10-11 22:41:24,484][71601] Updated weights for policy 0, policy_version 91660 (0.0009) [2023-10-11 22:41:24,857][71601] Updated weights for policy 0, policy_version 91670 (0.0007) [2023-10-11 22:41:25,218][71601] Updated weights for policy 0, policy_version 91680 (0.0007) [2023-10-11 22:41:25,952][71635] Updated weights for policy 1, policy_version 91592 (0.0009) [2023-10-11 22:41:26,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 187662336. Throughput: 0: 1811.2, 1: 1821.2. Samples: 46926206. Policy #0 lag: (min: 10.0, avg: 17.8, max: 42.0) [2023-10-11 22:41:26,034][70582] Avg episode reward: [(0, '134.680'), (1, '115.400')] [2023-10-11 22:41:26,326][71635] Updated weights for policy 1, policy_version 91602 (0.0008) [2023-10-11 22:41:26,698][71635] Updated weights for policy 1, policy_version 91612 (0.0009) [2023-10-11 22:41:28,947][71601] Updated weights for policy 0, policy_version 91690 (0.0008) [2023-10-11 22:41:29,316][71601] Updated weights for policy 0, policy_version 91700 (0.0007) [2023-10-11 22:41:29,691][71601] Updated weights for policy 0, policy_version 91710 (0.0008) [2023-10-11 22:41:30,334][71635] Updated weights for policy 1, policy_version 91622 (0.0008) [2023-10-11 22:41:30,703][71635] Updated weights for policy 1, policy_version 91632 (0.0007) [2023-10-11 22:41:31,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 187727872. Throughput: 0: 1815.5, 1: 1821.9. Samples: 46937700. Policy #0 lag: (min: 10.0, avg: 17.8, max: 42.0) [2023-10-11 22:41:31,034][70582] Avg episode reward: [(0, '127.530'), (1, '118.460')] [2023-10-11 22:41:31,073][71635] Updated weights for policy 1, policy_version 91642 (0.0007) [2023-10-11 22:41:33,407][71601] Updated weights for policy 0, policy_version 91720 (0.0010) [2023-10-11 22:41:33,787][71601] Updated weights for policy 0, policy_version 91730 (0.0008) [2023-10-11 22:41:34,150][71601] Updated weights for policy 0, policy_version 91740 (0.0009) [2023-10-11 22:41:34,950][71635] Updated weights for policy 1, policy_version 91652 (0.0009) [2023-10-11 22:41:35,327][71635] Updated weights for policy 1, policy_version 91662 (0.0009) [2023-10-11 22:41:35,702][71635] Updated weights for policy 1, policy_version 91672 (0.0008) [2023-10-11 22:41:36,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 187826176. Throughput: 0: 1815.2, 1: 1811.9. Samples: 46958646. Policy #0 lag: (min: 10.0, avg: 17.8, max: 42.0) [2023-10-11 22:41:36,035][70582] Avg episode reward: [(0, '122.620'), (1, '113.810')] [2023-10-11 22:41:37,859][71601] Updated weights for policy 0, policy_version 91750 (0.0009) [2023-10-11 22:41:38,223][71601] Updated weights for policy 0, policy_version 91760 (0.0008) [2023-10-11 22:41:38,598][71601] Updated weights for policy 0, policy_version 91770 (0.0011) [2023-10-11 22:41:39,468][71635] Updated weights for policy 1, policy_version 91682 (0.0009) [2023-10-11 22:41:39,839][71635] Updated weights for policy 1, policy_version 91692 (0.0008) [2023-10-11 22:41:40,208][71635] Updated weights for policy 1, policy_version 91702 (0.0008) [2023-10-11 22:41:40,571][71635] Updated weights for policy 1, policy_version 91712 (0.0007) [2023-10-11 22:41:41,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 187891712. Throughput: 0: 1814.1, 1: 1816.7. Samples: 46980404. Policy #0 lag: (min: 10.0, avg: 17.8, max: 42.0) [2023-10-11 22:41:41,034][70582] Avg episode reward: [(0, '123.760'), (1, '114.610')] [2023-10-11 22:41:42,282][71601] Updated weights for policy 0, policy_version 91780 (0.0010) [2023-10-11 22:41:42,664][71601] Updated weights for policy 0, policy_version 91790 (0.0008) [2023-10-11 22:41:43,032][71601] Updated weights for policy 0, policy_version 91800 (0.0009) [2023-10-11 22:41:44,343][71635] Updated weights for policy 1, policy_version 91722 (0.0008) [2023-10-11 22:41:44,707][71635] Updated weights for policy 1, policy_version 91732 (0.0010) [2023-10-11 22:41:45,073][71635] Updated weights for policy 1, policy_version 91742 (0.0010) [2023-10-11 22:41:46,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 187957248. Throughput: 0: 1818.3, 1: 1816.4. Samples: 46991598. Policy #0 lag: (min: 10.0, avg: 17.8, max: 42.0) [2023-10-11 22:41:46,035][70582] Avg episode reward: [(0, '131.740'), (1, '116.040')] [2023-10-11 22:41:46,757][71601] Updated weights for policy 0, policy_version 91810 (0.0007) [2023-10-11 22:41:47,130][71601] Updated weights for policy 0, policy_version 91820 (0.0007) [2023-10-11 22:41:47,499][71601] Updated weights for policy 0, policy_version 91830 (0.0009) [2023-10-11 22:41:47,872][71601] Updated weights for policy 0, policy_version 91840 (0.0012) [2023-10-11 22:41:48,784][71635] Updated weights for policy 1, policy_version 91752 (0.0009) [2023-10-11 22:41:49,145][71635] Updated weights for policy 1, policy_version 91762 (0.0009) [2023-10-11 22:41:49,519][71635] Updated weights for policy 1, policy_version 91772 (0.0008) [2023-10-11 22:41:51,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 188022784. Throughput: 0: 1825.5, 1: 1819.7. Samples: 47013500. Policy #0 lag: (min: 10.0, avg: 17.8, max: 42.0) [2023-10-11 22:41:51,034][70582] Avg episode reward: [(0, '133.750'), (1, '112.740')] [2023-10-11 22:41:51,561][71601] Updated weights for policy 0, policy_version 91850 (0.0008) [2023-10-11 22:41:51,942][71601] Updated weights for policy 0, policy_version 91860 (0.0008) [2023-10-11 22:41:52,313][71601] Updated weights for policy 0, policy_version 91870 (0.0010) [2023-10-11 22:41:53,135][71635] Updated weights for policy 1, policy_version 91782 (0.0007) [2023-10-11 22:41:53,501][71635] Updated weights for policy 1, policy_version 91792 (0.0011) [2023-10-11 22:41:53,873][71635] Updated weights for policy 1, policy_version 91802 (0.0007) [2023-10-11 22:41:56,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 188088320. Throughput: 0: 1830.1, 1: 1817.5. Samples: 47036074. Policy #0 lag: (min: 8.0, avg: 33.9, max: 40.0) [2023-10-11 22:41:56,034][70582] Avg episode reward: [(0, '134.160'), (1, '111.400')] [2023-10-11 22:41:56,040][71601] Updated weights for policy 0, policy_version 91880 (0.0008) [2023-10-11 22:41:56,040][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000091808_94011392.pth... [2023-10-11 22:41:56,070][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000090112_92274688.pth [2023-10-11 22:41:56,408][71601] Updated weights for policy 0, policy_version 91890 (0.0007) [2023-10-11 22:41:56,780][71601] Updated weights for policy 0, policy_version 91900 (0.0009) [2023-10-11 22:41:56,922][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000091904_94109696.pth... [2023-10-11 22:41:56,951][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000090176_92340224.pth [2023-10-11 22:41:57,499][71635] Updated weights for policy 1, policy_version 91812 (0.0008) [2023-10-11 22:41:57,865][71635] Updated weights for policy 1, policy_version 91822 (0.0009) [2023-10-11 22:41:58,228][71635] Updated weights for policy 1, policy_version 91832 (0.0008) [2023-10-11 22:42:00,437][71601] Updated weights for policy 0, policy_version 91910 (0.0010) [2023-10-11 22:42:00,811][71601] Updated weights for policy 0, policy_version 91920 (0.0010) [2023-10-11 22:42:01,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 188153856. Throughput: 0: 1830.3, 1: 1818.6. Samples: 47046122. Policy #0 lag: (min: 8.0, avg: 33.9, max: 40.0) [2023-10-11 22:42:01,034][70582] Avg episode reward: [(0, '133.400'), (1, '108.370')] [2023-10-11 22:42:01,170][71601] Updated weights for policy 0, policy_version 91930 (0.0010) [2023-10-11 22:42:01,946][71635] Updated weights for policy 1, policy_version 91842 (0.0008) [2023-10-11 22:42:02,302][71635] Updated weights for policy 1, policy_version 91852 (0.0009) [2023-10-11 22:42:02,672][71635] Updated weights for policy 1, policy_version 91862 (0.0008) [2023-10-11 22:42:03,039][71635] Updated weights for policy 1, policy_version 91872 (0.0008) [2023-10-11 22:42:04,813][71601] Updated weights for policy 0, policy_version 91940 (0.0009) [2023-10-11 22:42:05,194][71601] Updated weights for policy 0, policy_version 91950 (0.0007) [2023-10-11 22:42:05,565][71601] Updated weights for policy 0, policy_version 91960 (0.0008) [2023-10-11 22:42:06,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 188252160. Throughput: 0: 1830.4, 1: 1810.7. Samples: 47068520. Policy #0 lag: (min: 8.0, avg: 33.9, max: 40.0) [2023-10-11 22:42:06,034][70582] Avg episode reward: [(0, '135.910'), (1, '107.580')] [2023-10-11 22:42:06,814][71635] Updated weights for policy 1, policy_version 91882 (0.0010) [2023-10-11 22:42:07,174][71635] Updated weights for policy 1, policy_version 91892 (0.0011) [2023-10-11 22:42:07,541][71635] Updated weights for policy 1, policy_version 91902 (0.0010) [2023-10-11 22:42:09,233][71601] Updated weights for policy 0, policy_version 91970 (0.0008) [2023-10-11 22:42:09,593][71601] Updated weights for policy 0, policy_version 91980 (0.0007) [2023-10-11 22:42:09,968][71601] Updated weights for policy 0, policy_version 91990 (0.0009) [2023-10-11 22:42:10,341][71601] Updated weights for policy 0, policy_version 92000 (0.0009) [2023-10-11 22:42:11,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 188317696. Throughput: 0: 1827.2, 1: 1809.6. Samples: 47089860. Policy #0 lag: (min: 8.0, avg: 33.9, max: 40.0) [2023-10-11 22:42:11,034][70582] Avg episode reward: [(0, '137.700'), (1, '109.620')] [2023-10-11 22:42:11,185][71635] Updated weights for policy 1, policy_version 91912 (0.0008) [2023-10-11 22:42:11,554][71635] Updated weights for policy 1, policy_version 91922 (0.0009) [2023-10-11 22:42:11,919][71635] Updated weights for policy 1, policy_version 91932 (0.0009) [2023-10-11 22:42:13,920][71601] Updated weights for policy 0, policy_version 92010 (0.0009) [2023-10-11 22:42:14,292][71601] Updated weights for policy 0, policy_version 92020 (0.0009) [2023-10-11 22:42:14,654][71601] Updated weights for policy 0, policy_version 92030 (0.0008) [2023-10-11 22:42:15,598][71635] Updated weights for policy 1, policy_version 91942 (0.0008) [2023-10-11 22:42:15,967][71635] Updated weights for policy 1, policy_version 91952 (0.0009) [2023-10-11 22:42:16,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 188383232. Throughput: 0: 1826.4, 1: 1809.7. Samples: 47101324. Policy #0 lag: (min: 8.0, avg: 33.9, max: 40.0) [2023-10-11 22:42:16,034][70582] Avg episode reward: [(0, '137.990'), (1, '109.510')] [2023-10-11 22:42:16,333][71635] Updated weights for policy 1, policy_version 91962 (0.0010) [2023-10-11 22:42:18,338][71601] Updated weights for policy 0, policy_version 92040 (0.0009) [2023-10-11 22:42:18,719][71601] Updated weights for policy 0, policy_version 92050 (0.0008) [2023-10-11 22:42:19,093][71601] Updated weights for policy 0, policy_version 92060 (0.0009) [2023-10-11 22:42:20,135][71635] Updated weights for policy 1, policy_version 91972 (0.0011) [2023-10-11 22:42:20,499][71635] Updated weights for policy 1, policy_version 91982 (0.0009) [2023-10-11 22:42:20,864][71635] Updated weights for policy 1, policy_version 91992 (0.0008) [2023-10-11 22:42:21,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 188448768. Throughput: 0: 1826.7, 1: 1814.6. Samples: 47122504. Policy #0 lag: (min: 8.0, avg: 33.9, max: 40.0) [2023-10-11 22:42:21,034][70582] Avg episode reward: [(0, '131.010'), (1, '104.580')] [2023-10-11 22:42:22,700][71601] Updated weights for policy 0, policy_version 92070 (0.0007) [2023-10-11 22:42:23,068][71601] Updated weights for policy 0, policy_version 92080 (0.0008) [2023-10-11 22:42:23,440][71601] Updated weights for policy 0, policy_version 92090 (0.0007) [2023-10-11 22:42:24,272][71635] Updated weights for policy 1, policy_version 92002 (0.0009) [2023-10-11 22:42:24,638][71635] Updated weights for policy 1, policy_version 92012 (0.0008) [2023-10-11 22:42:25,008][71635] Updated weights for policy 1, policy_version 92022 (0.0009) [2023-10-11 22:42:25,367][71635] Updated weights for policy 1, policy_version 92032 (0.0009) [2023-10-11 22:42:26,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 188547072. Throughput: 0: 1824.0, 1: 1815.5. Samples: 47144180. Policy #0 lag: (min: 8.0, avg: 33.9, max: 40.0) [2023-10-11 22:42:26,035][70582] Avg episode reward: [(0, '136.480'), (1, '103.210')] [2023-10-11 22:42:27,242][71601] Updated weights for policy 0, policy_version 92100 (0.0008) [2023-10-11 22:42:27,613][71601] Updated weights for policy 0, policy_version 92110 (0.0007) [2023-10-11 22:42:27,987][71601] Updated weights for policy 0, policy_version 92120 (0.0008) [2023-10-11 22:42:29,164][71635] Updated weights for policy 1, policy_version 92042 (0.0010) [2023-10-11 22:42:29,522][71635] Updated weights for policy 1, policy_version 92052 (0.0010) [2023-10-11 22:42:29,894][71635] Updated weights for policy 1, policy_version 92062 (0.0009) [2023-10-11 22:42:31,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 188612608. Throughput: 0: 1821.0, 1: 1817.9. Samples: 47155346. Policy #0 lag: (min: 8.0, avg: 33.9, max: 40.0) [2023-10-11 22:42:31,034][70582] Avg episode reward: [(0, '135.250'), (1, '104.110')] [2023-10-11 22:42:31,703][71601] Updated weights for policy 0, policy_version 92130 (0.0008) [2023-10-11 22:42:32,062][71601] Updated weights for policy 0, policy_version 92140 (0.0011) [2023-10-11 22:42:32,436][71601] Updated weights for policy 0, policy_version 92150 (0.0010) [2023-10-11 22:42:32,801][71601] Updated weights for policy 0, policy_version 92160 (0.0008) [2023-10-11 22:42:33,714][71635] Updated weights for policy 1, policy_version 92072 (0.0008) [2023-10-11 22:42:34,086][71635] Updated weights for policy 1, policy_version 92082 (0.0009) [2023-10-11 22:42:34,450][71635] Updated weights for policy 1, policy_version 92092 (0.0007) [2023-10-11 22:42:36,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 188678144. Throughput: 0: 1811.6, 1: 1818.9. Samples: 47176872. Policy #0 lag: (min: 8.0, avg: 33.9, max: 40.0) [2023-10-11 22:42:36,034][70582] Avg episode reward: [(0, '141.490'), (1, '102.810')] [2023-10-11 22:42:36,442][71601] Updated weights for policy 0, policy_version 92170 (0.0008) [2023-10-11 22:42:36,811][71601] Updated weights for policy 0, policy_version 92180 (0.0010) [2023-10-11 22:42:37,185][71601] Updated weights for policy 0, policy_version 92190 (0.0008) [2023-10-11 22:42:38,033][71635] Updated weights for policy 1, policy_version 92102 (0.0007) [2023-10-11 22:42:38,408][71635] Updated weights for policy 1, policy_version 92112 (0.0008) [2023-10-11 22:42:38,768][71635] Updated weights for policy 1, policy_version 92122 (0.0008) [2023-10-11 22:42:40,962][71601] Updated weights for policy 0, policy_version 92200 (0.0011) [2023-10-11 22:42:41,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 188743680. Throughput: 0: 1815.6, 1: 1823.6. Samples: 47199836. Policy #0 lag: (min: 8.0, avg: 33.9, max: 40.0) [2023-10-11 22:42:41,034][70582] Avg episode reward: [(0, '141.730'), (1, '102.690')] [2023-10-11 22:42:41,329][71601] Updated weights for policy 0, policy_version 92210 (0.0011) [2023-10-11 22:42:41,711][71601] Updated weights for policy 0, policy_version 92220 (0.0010) [2023-10-11 22:42:42,500][71635] Updated weights for policy 1, policy_version 92132 (0.0009) [2023-10-11 22:42:42,867][71635] Updated weights for policy 1, policy_version 92142 (0.0010) [2023-10-11 22:42:43,234][71635] Updated weights for policy 1, policy_version 92152 (0.0007) [2023-10-11 22:42:45,144][71601] Updated weights for policy 0, policy_version 92230 (0.0007) [2023-10-11 22:42:45,527][71601] Updated weights for policy 0, policy_version 92240 (0.0008) [2023-10-11 22:42:45,900][71601] Updated weights for policy 0, policy_version 92250 (0.0009) [2023-10-11 22:42:46,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 188809216. Throughput: 0: 1820.9, 1: 1823.0. Samples: 47210096. Policy #0 lag: (min: 8.0, avg: 33.9, max: 40.0) [2023-10-11 22:42:46,035][70582] Avg episode reward: [(0, '150.070'), (1, '109.120')] [2023-10-11 22:42:46,866][71635] Updated weights for policy 1, policy_version 92162 (0.0008) [2023-10-11 22:42:47,229][71635] Updated weights for policy 1, policy_version 92172 (0.0008) [2023-10-11 22:42:47,592][71635] Updated weights for policy 1, policy_version 92182 (0.0009) [2023-10-11 22:42:47,968][71635] Updated weights for policy 1, policy_version 92192 (0.0009) [2023-10-11 22:42:49,648][71601] Updated weights for policy 0, policy_version 92260 (0.0010) [2023-10-11 22:42:50,024][71601] Updated weights for policy 0, policy_version 92270 (0.0008) [2023-10-11 22:42:50,398][71601] Updated weights for policy 0, policy_version 92280 (0.0010) [2023-10-11 22:42:51,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 188907520. Throughput: 0: 1820.2, 1: 1825.3. Samples: 47232566. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:42:51,034][70582] Avg episode reward: [(0, '147.170'), (1, '108.900')] [2023-10-11 22:42:51,832][71635] Updated weights for policy 1, policy_version 92202 (0.0008) [2023-10-11 22:42:52,206][71635] Updated weights for policy 1, policy_version 92212 (0.0009) [2023-10-11 22:42:52,568][71635] Updated weights for policy 1, policy_version 92222 (0.0007) [2023-10-11 22:42:54,192][71601] Updated weights for policy 0, policy_version 92290 (0.0009) [2023-10-11 22:42:54,560][71601] Updated weights for policy 0, policy_version 92300 (0.0010) [2023-10-11 22:42:54,939][71601] Updated weights for policy 0, policy_version 92310 (0.0008) [2023-10-11 22:42:55,301][71601] Updated weights for policy 0, policy_version 92320 (0.0009) [2023-10-11 22:42:56,034][70582] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 188973056. Throughput: 0: 1815.2, 1: 1825.2. Samples: 47253678. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:42:56,034][70582] Avg episode reward: [(0, '143.310'), (1, '110.340')] [2023-10-11 22:42:56,165][71635] Updated weights for policy 1, policy_version 92232 (0.0011) [2023-10-11 22:42:56,538][71635] Updated weights for policy 1, policy_version 92242 (0.0010) [2023-10-11 22:42:56,907][71635] Updated weights for policy 1, policy_version 92252 (0.0007) [2023-10-11 22:42:58,951][71601] Updated weights for policy 0, policy_version 92330 (0.0007) [2023-10-11 22:42:59,316][71601] Updated weights for policy 0, policy_version 92340 (0.0011) [2023-10-11 22:42:59,689][71601] Updated weights for policy 0, policy_version 92350 (0.0009) [2023-10-11 22:43:00,478][71635] Updated weights for policy 1, policy_version 92262 (0.0007) [2023-10-11 22:43:00,849][71635] Updated weights for policy 1, policy_version 92272 (0.0008) [2023-10-11 22:43:01,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 189038592. Throughput: 0: 1813.7, 1: 1828.2. Samples: 47265210. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:43:01,034][70582] Avg episode reward: [(0, '148.500'), (1, '111.000')] [2023-10-11 22:43:01,215][71635] Updated weights for policy 1, policy_version 92282 (0.0008) [2023-10-11 22:43:03,556][71601] Updated weights for policy 0, policy_version 92360 (0.0008) [2023-10-11 22:43:03,927][71601] Updated weights for policy 0, policy_version 92370 (0.0008) [2023-10-11 22:43:04,306][71601] Updated weights for policy 0, policy_version 92380 (0.0008) [2023-10-11 22:43:04,991][71635] Updated weights for policy 1, policy_version 92292 (0.0009) [2023-10-11 22:43:05,359][71635] Updated weights for policy 1, policy_version 92302 (0.0007) [2023-10-11 22:43:05,715][71635] Updated weights for policy 1, policy_version 92312 (0.0011) [2023-10-11 22:43:06,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 189136896. Throughput: 0: 1813.2, 1: 1829.2. Samples: 47286412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:43:06,034][70582] Avg episode reward: [(0, '144.320'), (1, '109.290')] [2023-10-11 22:43:07,865][71601] Updated weights for policy 0, policy_version 92390 (0.0009) [2023-10-11 22:43:08,230][71601] Updated weights for policy 0, policy_version 92400 (0.0007) [2023-10-11 22:43:08,613][71601] Updated weights for policy 0, policy_version 92410 (0.0008) [2023-10-11 22:43:09,547][71635] Updated weights for policy 1, policy_version 92322 (0.0009) [2023-10-11 22:43:09,906][71635] Updated weights for policy 1, policy_version 92332 (0.0009) [2023-10-11 22:43:10,275][71635] Updated weights for policy 1, policy_version 92342 (0.0008) [2023-10-11 22:43:10,646][71635] Updated weights for policy 1, policy_version 92352 (0.0007) [2023-10-11 22:43:11,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 189202432. Throughput: 0: 1818.1, 1: 1823.8. Samples: 47308064. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:43:11,034][70582] Avg episode reward: [(0, '140.490'), (1, '104.740')] [2023-10-11 22:43:12,356][71601] Updated weights for policy 0, policy_version 92420 (0.0008) [2023-10-11 22:43:12,718][71601] Updated weights for policy 0, policy_version 92430 (0.0008) [2023-10-11 22:43:13,090][71601] Updated weights for policy 0, policy_version 92440 (0.0008) [2023-10-11 22:43:14,117][71635] Updated weights for policy 1, policy_version 92362 (0.0010) [2023-10-11 22:43:14,484][71635] Updated weights for policy 1, policy_version 92372 (0.0007) [2023-10-11 22:43:14,845][71635] Updated weights for policy 1, policy_version 92382 (0.0007) [2023-10-11 22:43:16,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 189267968. Throughput: 0: 1817.2, 1: 1823.0. Samples: 47319154. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:43:16,035][70582] Avg episode reward: [(0, '145.150'), (1, '109.960')] [2023-10-11 22:43:16,663][71601] Updated weights for policy 0, policy_version 92450 (0.0009) [2023-10-11 22:43:17,037][71601] Updated weights for policy 0, policy_version 92460 (0.0010) [2023-10-11 22:43:17,412][71601] Updated weights for policy 0, policy_version 92470 (0.0008) [2023-10-11 22:43:17,779][71601] Updated weights for policy 0, policy_version 92480 (0.0008) [2023-10-11 22:43:18,560][71635] Updated weights for policy 1, policy_version 92392 (0.0008) [2023-10-11 22:43:18,922][71635] Updated weights for policy 1, policy_version 92402 (0.0008) [2023-10-11 22:43:19,292][71635] Updated weights for policy 1, policy_version 92412 (0.0009) [2023-10-11 22:43:21,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 189333504. Throughput: 0: 1821.5, 1: 1821.9. Samples: 47340826. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:43:21,034][70582] Avg episode reward: [(0, '138.430'), (1, '117.060')] [2023-10-11 22:43:21,627][71601] Updated weights for policy 0, policy_version 92490 (0.0009) [2023-10-11 22:43:21,995][71601] Updated weights for policy 0, policy_version 92500 (0.0007) [2023-10-11 22:43:22,376][71601] Updated weights for policy 0, policy_version 92510 (0.0009) [2023-10-11 22:43:23,002][71635] Updated weights for policy 1, policy_version 92422 (0.0010) [2023-10-11 22:43:23,372][71635] Updated weights for policy 1, policy_version 92432 (0.0010) [2023-10-11 22:43:23,730][71635] Updated weights for policy 1, policy_version 92442 (0.0011) [2023-10-11 22:43:26,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 189399040. Throughput: 0: 1817.4, 1: 1820.8. Samples: 47363554. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:43:26,034][70582] Avg episode reward: [(0, '140.030'), (1, '120.020')] [2023-10-11 22:43:26,083][71601] Updated weights for policy 0, policy_version 92520 (0.0008) [2023-10-11 22:43:26,451][71601] Updated weights for policy 0, policy_version 92530 (0.0010) [2023-10-11 22:43:26,833][71601] Updated weights for policy 0, policy_version 92540 (0.0009) [2023-10-11 22:43:27,206][71635] Updated weights for policy 1, policy_version 92452 (0.0009) [2023-10-11 22:43:27,581][71635] Updated weights for policy 1, policy_version 92462 (0.0010) [2023-10-11 22:43:27,949][71635] Updated weights for policy 1, policy_version 92472 (0.0009) [2023-10-11 22:43:30,527][71601] Updated weights for policy 0, policy_version 92550 (0.0010) [2023-10-11 22:43:30,900][71601] Updated weights for policy 0, policy_version 92560 (0.0007) [2023-10-11 22:43:31,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 189464576. Throughput: 0: 1813.2, 1: 1819.1. Samples: 47373548. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:43:31,034][70582] Avg episode reward: [(0, '133.110'), (1, '120.350')] [2023-10-11 22:43:31,279][71601] Updated weights for policy 0, policy_version 92570 (0.0009) [2023-10-11 22:43:31,712][71635] Updated weights for policy 1, policy_version 92482 (0.0007) [2023-10-11 22:43:32,079][71635] Updated weights for policy 1, policy_version 92492 (0.0007) [2023-10-11 22:43:32,449][71635] Updated weights for policy 1, policy_version 92502 (0.0011) [2023-10-11 22:43:32,815][71635] Updated weights for policy 1, policy_version 92512 (0.0008) [2023-10-11 22:43:35,130][71601] Updated weights for policy 0, policy_version 92580 (0.0007) [2023-10-11 22:43:35,507][71601] Updated weights for policy 0, policy_version 92590 (0.0009) [2023-10-11 22:43:35,875][71601] Updated weights for policy 0, policy_version 92600 (0.0007) [2023-10-11 22:43:36,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 189530112. Throughput: 0: 1805.3, 1: 1822.4. Samples: 47395814. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:43:36,035][70582] Avg episode reward: [(0, '135.080'), (1, '125.960')] [2023-10-11 22:43:36,574][71635] Updated weights for policy 1, policy_version 92522 (0.0011) [2023-10-11 22:43:36,939][71635] Updated weights for policy 1, policy_version 92532 (0.0011) [2023-10-11 22:43:37,301][71635] Updated weights for policy 1, policy_version 92542 (0.0010) [2023-10-11 22:43:39,596][71601] Updated weights for policy 0, policy_version 92610 (0.0009) [2023-10-11 22:43:39,972][71601] Updated weights for policy 0, policy_version 92620 (0.0008) [2023-10-11 22:43:40,331][71601] Updated weights for policy 0, policy_version 92630 (0.0008) [2023-10-11 22:43:40,706][71601] Updated weights for policy 0, policy_version 92640 (0.0009) [2023-10-11 22:43:41,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 189628416. Throughput: 0: 1814.8, 1: 1818.6. Samples: 47417184. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:43:41,035][70582] Avg episode reward: [(0, '137.070'), (1, '130.350')] [2023-10-11 22:43:41,142][71635] Updated weights for policy 1, policy_version 92552 (0.0009) [2023-10-11 22:43:41,508][71635] Updated weights for policy 1, policy_version 92562 (0.0010) [2023-10-11 22:43:41,872][71635] Updated weights for policy 1, policy_version 92572 (0.0009) [2023-10-11 22:43:44,411][71601] Updated weights for policy 0, policy_version 92650 (0.0010) [2023-10-11 22:43:44,785][71601] Updated weights for policy 0, policy_version 92660 (0.0008) [2023-10-11 22:43:45,164][71601] Updated weights for policy 0, policy_version 92670 (0.0008) [2023-10-11 22:43:45,676][71635] Updated weights for policy 1, policy_version 92582 (0.0007) [2023-10-11 22:43:46,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 189693952. Throughput: 0: 1809.7, 1: 1810.1. Samples: 47428100. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:43:46,035][70582] Avg episode reward: [(0, '137.360'), (1, '121.080')] [2023-10-11 22:43:46,036][71635] Updated weights for policy 1, policy_version 92592 (0.0008) [2023-10-11 22:43:46,402][71635] Updated weights for policy 1, policy_version 92602 (0.0007) [2023-10-11 22:43:48,816][71601] Updated weights for policy 0, policy_version 92680 (0.0008) [2023-10-11 22:43:49,194][71601] Updated weights for policy 0, policy_version 92690 (0.0008) [2023-10-11 22:43:49,576][71601] Updated weights for policy 0, policy_version 92700 (0.0009) [2023-10-11 22:43:50,193][71635] Updated weights for policy 1, policy_version 92612 (0.0008) [2023-10-11 22:43:50,546][71635] Updated weights for policy 1, policy_version 92622 (0.0008) [2023-10-11 22:43:50,915][71635] Updated weights for policy 1, policy_version 92632 (0.0008) [2023-10-11 22:43:51,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 189759488. Throughput: 0: 1819.2, 1: 1804.2. Samples: 47449468. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:43:51,034][70582] Avg episode reward: [(0, '127.610'), (1, '121.810')] [2023-10-11 22:43:53,073][71601] Updated weights for policy 0, policy_version 92710 (0.0007) [2023-10-11 22:43:53,450][71601] Updated weights for policy 0, policy_version 92720 (0.0008) [2023-10-11 22:43:53,828][71601] Updated weights for policy 0, policy_version 92730 (0.0008) [2023-10-11 22:43:54,698][71635] Updated weights for policy 1, policy_version 92642 (0.0009) [2023-10-11 22:43:55,060][71635] Updated weights for policy 1, policy_version 92652 (0.0008) [2023-10-11 22:43:55,433][71635] Updated weights for policy 1, policy_version 92662 (0.0008) [2023-10-11 22:43:55,798][71635] Updated weights for policy 1, policy_version 92672 (0.0007) [2023-10-11 22:43:56,034][70582] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 189857792. Throughput: 0: 1814.2, 1: 1812.4. Samples: 47471264. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:43:56,035][70582] Avg episode reward: [(0, '121.720'), (1, '122.180')] [2023-10-11 22:43:56,050][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000092736_94961664.pth... [2023-10-11 22:43:56,050][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000092672_94896128.pth... [2023-10-11 22:43:56,081][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000091040_93224960.pth [2023-10-11 22:43:56,086][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000090976_93159424.pth [2023-10-11 22:43:57,447][71601] Updated weights for policy 0, policy_version 92740 (0.0009) [2023-10-11 22:43:57,824][71601] Updated weights for policy 0, policy_version 92750 (0.0011) [2023-10-11 22:43:58,200][71601] Updated weights for policy 0, policy_version 92760 (0.0008) [2023-10-11 22:43:59,406][71635] Updated weights for policy 1, policy_version 92682 (0.0011) [2023-10-11 22:43:59,770][71635] Updated weights for policy 1, policy_version 92692 (0.0008) [2023-10-11 22:44:00,136][71635] Updated weights for policy 1, policy_version 92702 (0.0008) [2023-10-11 22:44:01,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 189923328. Throughput: 0: 1823.4, 1: 1804.7. Samples: 47482418. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:44:01,034][70582] Avg episode reward: [(0, '122.900'), (1, '121.950')] [2023-10-11 22:44:01,999][71601] Updated weights for policy 0, policy_version 92770 (0.0008) [2023-10-11 22:44:02,372][71601] Updated weights for policy 0, policy_version 92780 (0.0009) [2023-10-11 22:44:02,737][71601] Updated weights for policy 0, policy_version 92790 (0.0009) [2023-10-11 22:44:03,106][71601] Updated weights for policy 0, policy_version 92800 (0.0009) [2023-10-11 22:44:03,830][71635] Updated weights for policy 1, policy_version 92712 (0.0007) [2023-10-11 22:44:04,191][71635] Updated weights for policy 1, policy_version 92722 (0.0009) [2023-10-11 22:44:04,558][71635] Updated weights for policy 1, policy_version 92732 (0.0008) [2023-10-11 22:44:06,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 189988864. Throughput: 0: 1817.5, 1: 1813.1. Samples: 47504202. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:44:06,035][70582] Avg episode reward: [(0, '125.540'), (1, '121.620')] [2023-10-11 22:44:06,645][71601] Updated weights for policy 0, policy_version 92810 (0.0010) [2023-10-11 22:44:07,014][71601] Updated weights for policy 0, policy_version 92820 (0.0009) [2023-10-11 22:44:07,399][71601] Updated weights for policy 0, policy_version 92830 (0.0008) [2023-10-11 22:44:08,474][71635] Updated weights for policy 1, policy_version 92742 (0.0008) [2023-10-11 22:44:08,840][71635] Updated weights for policy 1, policy_version 92752 (0.0008) [2023-10-11 22:44:09,207][71635] Updated weights for policy 1, policy_version 92762 (0.0008) [2023-10-11 22:44:11,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 190054400. Throughput: 0: 1817.9, 1: 1802.5. Samples: 47526472. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:44:11,035][70582] Avg episode reward: [(0, '127.640'), (1, '132.220')] [2023-10-11 22:44:11,098][71601] Updated weights for policy 0, policy_version 92840 (0.0010) [2023-10-11 22:44:11,468][71601] Updated weights for policy 0, policy_version 92850 (0.0010) [2023-10-11 22:44:11,845][71601] Updated weights for policy 0, policy_version 92860 (0.0009) [2023-10-11 22:44:12,624][71635] Updated weights for policy 1, policy_version 92772 (0.0009) [2023-10-11 22:44:12,994][71635] Updated weights for policy 1, policy_version 92782 (0.0007) [2023-10-11 22:44:13,365][71635] Updated weights for policy 1, policy_version 92792 (0.0007) [2023-10-11 22:44:15,435][71601] Updated weights for policy 0, policy_version 92870 (0.0010) [2023-10-11 22:44:15,810][71601] Updated weights for policy 0, policy_version 92880 (0.0010) [2023-10-11 22:44:16,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 190119936. Throughput: 0: 1818.0, 1: 1812.1. Samples: 47536906. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:44:16,035][70582] Avg episode reward: [(0, '128.730'), (1, '127.120')] [2023-10-11 22:44:16,181][71601] Updated weights for policy 0, policy_version 92890 (0.0011) [2023-10-11 22:44:16,962][71635] Updated weights for policy 1, policy_version 92802 (0.0007) [2023-10-11 22:44:17,327][71635] Updated weights for policy 1, policy_version 92812 (0.0009) [2023-10-11 22:44:17,691][71635] Updated weights for policy 1, policy_version 92822 (0.0008) [2023-10-11 22:44:18,065][71635] Updated weights for policy 1, policy_version 92832 (0.0008) [2023-10-11 22:44:19,955][71601] Updated weights for policy 0, policy_version 92900 (0.0007) [2023-10-11 22:44:20,322][71601] Updated weights for policy 0, policy_version 92910 (0.0009) [2023-10-11 22:44:20,699][71601] Updated weights for policy 0, policy_version 92920 (0.0007) [2023-10-11 22:44:21,034][70582] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 190218240. Throughput: 0: 1826.6, 1: 1807.9. Samples: 47559364. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:44:21,034][70582] Avg episode reward: [(0, '127.050'), (1, '125.570')] [2023-10-11 22:44:21,809][71635] Updated weights for policy 1, policy_version 92842 (0.0008) [2023-10-11 22:44:22,177][71635] Updated weights for policy 1, policy_version 92852 (0.0008) [2023-10-11 22:44:22,548][71635] Updated weights for policy 1, policy_version 92862 (0.0008) [2023-10-11 22:44:24,374][71601] Updated weights for policy 0, policy_version 92930 (0.0007) [2023-10-11 22:44:24,751][71601] Updated weights for policy 0, policy_version 92940 (0.0008) [2023-10-11 22:44:25,114][71601] Updated weights for policy 0, policy_version 92950 (0.0008) [2023-10-11 22:44:25,491][71601] Updated weights for policy 0, policy_version 92960 (0.0009) [2023-10-11 22:44:26,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 190283776. Throughput: 0: 1822.1, 1: 1809.3. Samples: 47580596. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:44:26,034][70582] Avg episode reward: [(0, '134.490'), (1, '127.700')] [2023-10-11 22:44:26,485][71635] Updated weights for policy 1, policy_version 92872 (0.0009) [2023-10-11 22:44:26,869][71635] Updated weights for policy 1, policy_version 92882 (0.0008) [2023-10-11 22:44:27,242][71635] Updated weights for policy 1, policy_version 92892 (0.0010) [2023-10-11 22:44:29,189][71601] Updated weights for policy 0, policy_version 92970 (0.0008) [2023-10-11 22:44:29,557][71601] Updated weights for policy 0, policy_version 92980 (0.0008) [2023-10-11 22:44:29,936][71601] Updated weights for policy 0, policy_version 92990 (0.0009) [2023-10-11 22:44:30,999][71635] Updated weights for policy 1, policy_version 92902 (0.0009) [2023-10-11 22:44:31,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 190349312. Throughput: 0: 1824.0, 1: 1805.8. Samples: 47591438. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:44:31,034][70582] Avg episode reward: [(0, '135.490'), (1, '122.910')] [2023-10-11 22:44:31,363][71635] Updated weights for policy 1, policy_version 92912 (0.0008) [2023-10-11 22:44:31,724][71635] Updated weights for policy 1, policy_version 92922 (0.0009) [2023-10-11 22:44:33,563][71601] Updated weights for policy 0, policy_version 93000 (0.0008) [2023-10-11 22:44:33,940][71601] Updated weights for policy 0, policy_version 93010 (0.0008) [2023-10-11 22:44:34,309][71601] Updated weights for policy 0, policy_version 93020 (0.0010) [2023-10-11 22:44:35,313][71635] Updated weights for policy 1, policy_version 92932 (0.0009) [2023-10-11 22:44:35,672][71635] Updated weights for policy 1, policy_version 92942 (0.0007) [2023-10-11 22:44:36,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 190414848. Throughput: 0: 1821.6, 1: 1816.9. Samples: 47613202. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:44:36,035][70582] Avg episode reward: [(0, '135.240'), (1, '120.720')] [2023-10-11 22:44:36,036][71635] Updated weights for policy 1, policy_version 92952 (0.0009) [2023-10-11 22:44:37,994][71601] Updated weights for policy 0, policy_version 93030 (0.0008) [2023-10-11 22:44:38,362][71601] Updated weights for policy 0, policy_version 93040 (0.0009) [2023-10-11 22:44:38,741][71601] Updated weights for policy 0, policy_version 93050 (0.0007) [2023-10-11 22:44:39,663][71635] Updated weights for policy 1, policy_version 92962 (0.0007) [2023-10-11 22:44:40,022][71635] Updated weights for policy 1, policy_version 92972 (0.0007) [2023-10-11 22:44:40,390][71635] Updated weights for policy 1, policy_version 92982 (0.0009) [2023-10-11 22:44:40,763][71635] Updated weights for policy 1, policy_version 92992 (0.0007) [2023-10-11 22:44:41,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 190513152. Throughput: 0: 1822.3, 1: 1822.1. Samples: 47635262. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:44:41,035][70582] Avg episode reward: [(0, '135.090'), (1, '118.710')] [2023-10-11 22:44:42,378][71601] Updated weights for policy 0, policy_version 93060 (0.0008) [2023-10-11 22:44:42,754][71601] Updated weights for policy 0, policy_version 93070 (0.0009) [2023-10-11 22:44:43,128][71601] Updated weights for policy 0, policy_version 93080 (0.0009) [2023-10-11 22:44:44,495][71635] Updated weights for policy 1, policy_version 93002 (0.0007) [2023-10-11 22:44:44,861][71635] Updated weights for policy 1, policy_version 93012 (0.0007) [2023-10-11 22:44:45,232][71635] Updated weights for policy 1, policy_version 93022 (0.0008) [2023-10-11 22:44:46,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 190578688. Throughput: 0: 1816.8, 1: 1820.3. Samples: 47646090. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:44:46,034][70582] Avg episode reward: [(0, '137.240'), (1, '122.860')] [2023-10-11 22:44:46,741][71601] Updated weights for policy 0, policy_version 93090 (0.0007) [2023-10-11 22:44:47,118][71601] Updated weights for policy 0, policy_version 93100 (0.0007) [2023-10-11 22:44:47,492][71601] Updated weights for policy 0, policy_version 93110 (0.0007) [2023-10-11 22:44:47,866][71601] Updated weights for policy 0, policy_version 93120 (0.0008) [2023-10-11 22:44:48,907][71635] Updated weights for policy 1, policy_version 93032 (0.0008) [2023-10-11 22:44:49,269][71635] Updated weights for policy 1, policy_version 93042 (0.0009) [2023-10-11 22:44:49,641][71635] Updated weights for policy 1, policy_version 93052 (0.0007) [2023-10-11 22:44:51,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 190644224. Throughput: 0: 1814.5, 1: 1820.2. Samples: 47667764. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 22:44:51,035][70582] Avg episode reward: [(0, '138.180'), (1, '122.640')] [2023-10-11 22:44:51,681][71601] Updated weights for policy 0, policy_version 93130 (0.0009) [2023-10-11 22:44:52,063][71601] Updated weights for policy 0, policy_version 93140 (0.0010) [2023-10-11 22:44:52,423][71601] Updated weights for policy 0, policy_version 93150 (0.0008) [2023-10-11 22:44:53,284][71635] Updated weights for policy 1, policy_version 93062 (0.0009) [2023-10-11 22:44:53,650][71635] Updated weights for policy 1, policy_version 93072 (0.0007) [2023-10-11 22:44:54,014][71635] Updated weights for policy 1, policy_version 93082 (0.0010) [2023-10-11 22:44:56,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.6, 300 sec: 14551.2). Total num frames: 190709760. Throughput: 0: 1809.7, 1: 1822.2. Samples: 47689906. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 22:44:56,034][70582] Avg episode reward: [(0, '139.770'), (1, '122.330')] [2023-10-11 22:44:56,391][71601] Updated weights for policy 0, policy_version 93160 (0.0009) [2023-10-11 22:44:56,764][71601] Updated weights for policy 0, policy_version 93170 (0.0007) [2023-10-11 22:44:57,141][71601] Updated weights for policy 0, policy_version 93180 (0.0008) [2023-10-11 22:44:57,713][71635] Updated weights for policy 1, policy_version 93092 (0.0009) [2023-10-11 22:44:58,083][71635] Updated weights for policy 1, policy_version 93102 (0.0009) [2023-10-11 22:44:58,451][71635] Updated weights for policy 1, policy_version 93112 (0.0011) [2023-10-11 22:45:00,802][71601] Updated weights for policy 0, policy_version 93190 (0.0007) [2023-10-11 22:45:01,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 190775296. Throughput: 0: 1808.6, 1: 1826.2. Samples: 47700470. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 22:45:01,035][70582] Avg episode reward: [(0, '143.680'), (1, '124.890')] [2023-10-11 22:45:01,160][71601] Updated weights for policy 0, policy_version 93200 (0.0008) [2023-10-11 22:45:01,538][71601] Updated weights for policy 0, policy_version 93210 (0.0010) [2023-10-11 22:45:02,290][71635] Updated weights for policy 1, policy_version 93122 (0.0010) [2023-10-11 22:45:02,672][71635] Updated weights for policy 1, policy_version 93132 (0.0009) [2023-10-11 22:45:03,039][71635] Updated weights for policy 1, policy_version 93142 (0.0008) [2023-10-11 22:45:03,403][71635] Updated weights for policy 1, policy_version 93152 (0.0008) [2023-10-11 22:45:05,144][71601] Updated weights for policy 0, policy_version 93220 (0.0008) [2023-10-11 22:45:05,523][71601] Updated weights for policy 0, policy_version 93230 (0.0010) [2023-10-11 22:45:05,894][71601] Updated weights for policy 0, policy_version 93240 (0.0008) [2023-10-11 22:45:06,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 190840832. Throughput: 0: 1809.6, 1: 1818.0. Samples: 47722602. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 22:45:06,034][70582] Avg episode reward: [(0, '144.880'), (1, '130.630')] [2023-10-11 22:45:07,168][71635] Updated weights for policy 1, policy_version 93162 (0.0009) [2023-10-11 22:45:07,528][71635] Updated weights for policy 1, policy_version 93172 (0.0007) [2023-10-11 22:45:07,892][71635] Updated weights for policy 1, policy_version 93182 (0.0007) [2023-10-11 22:45:09,591][71601] Updated weights for policy 0, policy_version 93250 (0.0007) [2023-10-11 22:45:09,952][71601] Updated weights for policy 0, policy_version 93260 (0.0007) [2023-10-11 22:45:10,325][71601] Updated weights for policy 0, policy_version 93270 (0.0007) [2023-10-11 22:45:10,695][71601] Updated weights for policy 0, policy_version 93280 (0.0008) [2023-10-11 22:45:11,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 190939136. Throughput: 0: 1818.4, 1: 1818.2. Samples: 47744242. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 22:45:11,034][70582] Avg episode reward: [(0, '137.180'), (1, '134.210')] [2023-10-11 22:45:11,727][71635] Updated weights for policy 1, policy_version 93192 (0.0010) [2023-10-11 22:45:12,104][71635] Updated weights for policy 1, policy_version 93202 (0.0008) [2023-10-11 22:45:12,471][71635] Updated weights for policy 1, policy_version 93212 (0.0009) [2023-10-11 22:45:14,402][71601] Updated weights for policy 0, policy_version 93290 (0.0009) [2023-10-11 22:45:14,774][71601] Updated weights for policy 0, policy_version 93300 (0.0010) [2023-10-11 22:45:15,142][71601] Updated weights for policy 0, policy_version 93310 (0.0009) [2023-10-11 22:45:16,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 191004672. Throughput: 0: 1815.6, 1: 1822.6. Samples: 47755158. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 22:45:16,035][70582] Avg episode reward: [(0, '144.480'), (1, '133.680')] [2023-10-11 22:45:16,167][71635] Updated weights for policy 1, policy_version 93222 (0.0010) [2023-10-11 22:45:16,534][71635] Updated weights for policy 1, policy_version 93232 (0.0010) [2023-10-11 22:45:16,913][71635] Updated weights for policy 1, policy_version 93242 (0.0011) [2023-10-11 22:45:18,787][71601] Updated weights for policy 0, policy_version 93320 (0.0007) [2023-10-11 22:45:19,157][71601] Updated weights for policy 0, policy_version 93330 (0.0008) [2023-10-11 22:45:19,528][71601] Updated weights for policy 0, policy_version 93340 (0.0009) [2023-10-11 22:45:20,632][71635] Updated weights for policy 1, policy_version 93252 (0.0008) [2023-10-11 22:45:20,998][71635] Updated weights for policy 1, policy_version 93262 (0.0008) [2023-10-11 22:45:21,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 191070208. Throughput: 0: 1816.5, 1: 1814.4. Samples: 47776594. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 22:45:21,034][70582] Avg episode reward: [(0, '141.350'), (1, '139.430')] [2023-10-11 22:45:21,369][71635] Updated weights for policy 1, policy_version 93272 (0.0012) [2023-10-11 22:45:23,209][71601] Updated weights for policy 0, policy_version 93350 (0.0007) [2023-10-11 22:45:23,572][71601] Updated weights for policy 0, policy_version 93360 (0.0009) [2023-10-11 22:45:23,948][71601] Updated weights for policy 0, policy_version 93370 (0.0007) [2023-10-11 22:45:25,104][71635] Updated weights for policy 1, policy_version 93282 (0.0008) [2023-10-11 22:45:25,466][71635] Updated weights for policy 1, policy_version 93292 (0.0010) [2023-10-11 22:45:25,845][71635] Updated weights for policy 1, policy_version 93302 (0.0009) [2023-10-11 22:45:26,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 191135744. Throughput: 0: 1807.7, 1: 1819.7. Samples: 47798494. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 22:45:26,034][70582] Avg episode reward: [(0, '142.650'), (1, '129.180')] [2023-10-11 22:45:26,207][71635] Updated weights for policy 1, policy_version 93312 (0.0008) [2023-10-11 22:45:27,656][71601] Updated weights for policy 0, policy_version 93380 (0.0008) [2023-10-11 22:45:28,029][71601] Updated weights for policy 0, policy_version 93390 (0.0009) [2023-10-11 22:45:28,397][71601] Updated weights for policy 0, policy_version 93400 (0.0011) [2023-10-11 22:45:29,747][71635] Updated weights for policy 1, policy_version 93322 (0.0008) [2023-10-11 22:45:30,110][71635] Updated weights for policy 1, policy_version 93332 (0.0011) [2023-10-11 22:45:30,481][71635] Updated weights for policy 1, policy_version 93342 (0.0010) [2023-10-11 22:45:31,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 191234048. Throughput: 0: 1820.4, 1: 1811.7. Samples: 47809534. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 22:45:31,034][70582] Avg episode reward: [(0, '134.880'), (1, '129.230')] [2023-10-11 22:45:32,009][71601] Updated weights for policy 0, policy_version 93410 (0.0010) [2023-10-11 22:45:32,389][71601] Updated weights for policy 0, policy_version 93420 (0.0011) [2023-10-11 22:45:32,751][71601] Updated weights for policy 0, policy_version 93430 (0.0008) [2023-10-11 22:45:33,124][71601] Updated weights for policy 0, policy_version 93440 (0.0007) [2023-10-11 22:45:34,245][71635] Updated weights for policy 1, policy_version 93352 (0.0009) [2023-10-11 22:45:34,609][71635] Updated weights for policy 1, policy_version 93362 (0.0009) [2023-10-11 22:45:34,968][71635] Updated weights for policy 1, policy_version 93372 (0.0009) [2023-10-11 22:45:36,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 191299584. Throughput: 0: 1819.5, 1: 1821.1. Samples: 47831592. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 22:45:36,035][70582] Avg episode reward: [(0, '133.570'), (1, '131.460')] [2023-10-11 22:45:36,772][71601] Updated weights for policy 0, policy_version 93450 (0.0007) [2023-10-11 22:45:37,151][71601] Updated weights for policy 0, policy_version 93460 (0.0009) [2023-10-11 22:45:37,523][71601] Updated weights for policy 0, policy_version 93470 (0.0007) [2023-10-11 22:45:38,576][71635] Updated weights for policy 1, policy_version 93382 (0.0009) [2023-10-11 22:45:38,927][71635] Updated weights for policy 1, policy_version 93392 (0.0008) [2023-10-11 22:45:39,292][71635] Updated weights for policy 1, policy_version 93402 (0.0009) [2023-10-11 22:45:41,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 191365120. Throughput: 0: 1825.1, 1: 1813.8. Samples: 47853654. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 22:45:41,034][70582] Avg episode reward: [(0, '132.970'), (1, '130.940')] [2023-10-11 22:45:41,221][71601] Updated weights for policy 0, policy_version 93480 (0.0007) [2023-10-11 22:45:41,598][71601] Updated weights for policy 0, policy_version 93490 (0.0007) [2023-10-11 22:45:41,981][71601] Updated weights for policy 0, policy_version 93500 (0.0008) [2023-10-11 22:45:42,962][71635] Updated weights for policy 1, policy_version 93412 (0.0009) [2023-10-11 22:45:43,337][71635] Updated weights for policy 1, policy_version 93422 (0.0007) [2023-10-11 22:45:43,707][71635] Updated weights for policy 1, policy_version 93432 (0.0007) [2023-10-11 22:45:45,456][71601] Updated weights for policy 0, policy_version 93510 (0.0007) [2023-10-11 22:45:45,825][71601] Updated weights for policy 0, policy_version 93520 (0.0008) [2023-10-11 22:45:46,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 191430656. Throughput: 0: 1824.7, 1: 1816.0. Samples: 47864302. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-11 22:45:46,035][70582] Avg episode reward: [(0, '130.360'), (1, '134.070')] [2023-10-11 22:45:46,194][71601] Updated weights for policy 0, policy_version 93530 (0.0008) [2023-10-11 22:45:47,404][71635] Updated weights for policy 1, policy_version 93442 (0.0008) [2023-10-11 22:45:47,773][71635] Updated weights for policy 1, policy_version 93452 (0.0008) [2023-10-11 22:45:48,137][71635] Updated weights for policy 1, policy_version 93462 (0.0008) [2023-10-11 22:45:48,500][71635] Updated weights for policy 1, policy_version 93472 (0.0007) [2023-10-11 22:45:49,893][71601] Updated weights for policy 0, policy_version 93540 (0.0010) [2023-10-11 22:45:50,261][71601] Updated weights for policy 0, policy_version 93550 (0.0010) [2023-10-11 22:45:50,633][71601] Updated weights for policy 0, policy_version 93560 (0.0007) [2023-10-11 22:45:51,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 191528960. Throughput: 0: 1823.9, 1: 1813.5. Samples: 47886286. Policy #0 lag: (min: 29.0, avg: 38.7, max: 61.0) [2023-10-11 22:45:51,034][70582] Avg episode reward: [(0, '130.750'), (1, '131.600')] [2023-10-11 22:45:52,131][71635] Updated weights for policy 1, policy_version 93482 (0.0008) [2023-10-11 22:45:52,506][71635] Updated weights for policy 1, policy_version 93492 (0.0010) [2023-10-11 22:45:52,873][71635] Updated weights for policy 1, policy_version 93502 (0.0009) [2023-10-11 22:45:54,320][71601] Updated weights for policy 0, policy_version 93570 (0.0008) [2023-10-11 22:45:54,692][71601] Updated weights for policy 0, policy_version 93580 (0.0008) [2023-10-11 22:45:55,063][71601] Updated weights for policy 0, policy_version 93590 (0.0009) [2023-10-11 22:45:55,422][71601] Updated weights for policy 0, policy_version 93600 (0.0009) [2023-10-11 22:45:56,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 191594496. Throughput: 0: 1818.4, 1: 1814.6. Samples: 47907730. Policy #0 lag: (min: 29.0, avg: 38.7, max: 61.0) [2023-10-11 22:45:56,035][70582] Avg episode reward: [(0, '130.180'), (1, '131.630')] [2023-10-11 22:45:56,046][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000093600_95846400.pth... [2023-10-11 22:45:56,046][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000093504_95748096.pth... [2023-10-11 22:45:56,076][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000091904_94109696.pth [2023-10-11 22:45:56,079][71353] Saving a milestone ./train_atari/atari_gopher_APPO/checkpoint_p0/milestones/checkpoint_000093600_95846400.pth [2023-10-11 22:45:56,086][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000091808_94011392.pth [2023-10-11 22:45:56,090][71431] Saving a milestone ./train_atari/atari_gopher_APPO/checkpoint_p1/milestones/checkpoint_000093504_95748096.pth [2023-10-11 22:45:56,813][71635] Updated weights for policy 1, policy_version 93512 (0.0007) [2023-10-11 22:45:57,186][71635] Updated weights for policy 1, policy_version 93522 (0.0007) [2023-10-11 22:45:57,557][71635] Updated weights for policy 1, policy_version 93532 (0.0011) [2023-10-11 22:45:59,226][71601] Updated weights for policy 0, policy_version 93610 (0.0008) [2023-10-11 22:45:59,601][71601] Updated weights for policy 0, policy_version 93620 (0.0011) [2023-10-11 22:45:59,981][71601] Updated weights for policy 0, policy_version 93630 (0.0010) [2023-10-11 22:46:01,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 191660032. Throughput: 0: 1824.7, 1: 1817.1. Samples: 47919038. Policy #0 lag: (min: 29.0, avg: 38.7, max: 61.0) [2023-10-11 22:46:01,034][70582] Avg episode reward: [(0, '127.560'), (1, '119.300')] [2023-10-11 22:46:01,152][71635] Updated weights for policy 1, policy_version 93542 (0.0008) [2023-10-11 22:46:01,512][71635] Updated weights for policy 1, policy_version 93552 (0.0007) [2023-10-11 22:46:01,880][71635] Updated weights for policy 1, policy_version 93562 (0.0008) [2023-10-11 22:46:03,547][71601] Updated weights for policy 0, policy_version 93640 (0.0008) [2023-10-11 22:46:03,915][71601] Updated weights for policy 0, policy_version 93650 (0.0008) [2023-10-11 22:46:04,290][71601] Updated weights for policy 0, policy_version 93660 (0.0008) [2023-10-11 22:46:05,628][71635] Updated weights for policy 1, policy_version 93572 (0.0009) [2023-10-11 22:46:05,996][71635] Updated weights for policy 1, policy_version 93582 (0.0008) [2023-10-11 22:46:06,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 191725568. Throughput: 0: 1825.0, 1: 1819.7. Samples: 47940604. Policy #0 lag: (min: 29.0, avg: 38.7, max: 61.0) [2023-10-11 22:46:06,034][70582] Avg episode reward: [(0, '128.520'), (1, '113.270')] [2023-10-11 22:46:06,375][71635] Updated weights for policy 1, policy_version 93592 (0.0009) [2023-10-11 22:46:07,936][71601] Updated weights for policy 0, policy_version 93670 (0.0008) [2023-10-11 22:46:08,305][71601] Updated weights for policy 0, policy_version 93680 (0.0008) [2023-10-11 22:46:08,688][71601] Updated weights for policy 0, policy_version 93690 (0.0008) [2023-10-11 22:46:09,921][71635] Updated weights for policy 1, policy_version 93602 (0.0009) [2023-10-11 22:46:10,293][71635] Updated weights for policy 1, policy_version 93612 (0.0008) [2023-10-11 22:46:10,658][71635] Updated weights for policy 1, policy_version 93622 (0.0008) [2023-10-11 22:46:11,020][71635] Updated weights for policy 1, policy_version 93632 (0.0007) [2023-10-11 22:46:11,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 191823872. Throughput: 0: 1832.3, 1: 1817.8. Samples: 47962752. Policy #0 lag: (min: 29.0, avg: 38.7, max: 61.0) [2023-10-11 22:46:11,035][70582] Avg episode reward: [(0, '118.940'), (1, '111.340')] [2023-10-11 22:46:12,471][71601] Updated weights for policy 0, policy_version 93700 (0.0007) [2023-10-11 22:46:12,836][71601] Updated weights for policy 0, policy_version 93710 (0.0010) [2023-10-11 22:46:13,210][71601] Updated weights for policy 0, policy_version 93720 (0.0010) [2023-10-11 22:46:14,808][71635] Updated weights for policy 1, policy_version 93642 (0.0008) [2023-10-11 22:46:15,175][71635] Updated weights for policy 1, policy_version 93652 (0.0009) [2023-10-11 22:46:15,536][71635] Updated weights for policy 1, policy_version 93662 (0.0008) [2023-10-11 22:46:16,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 191889408. Throughput: 0: 1823.1, 1: 1819.2. Samples: 47973438. Policy #0 lag: (min: 29.0, avg: 38.7, max: 61.0) [2023-10-11 22:46:16,035][70582] Avg episode reward: [(0, '123.750'), (1, '105.760')] [2023-10-11 22:46:16,708][71601] Updated weights for policy 0, policy_version 93730 (0.0011) [2023-10-11 22:46:17,080][71601] Updated weights for policy 0, policy_version 93740 (0.0007) [2023-10-11 22:46:17,460][71601] Updated weights for policy 0, policy_version 93750 (0.0008) [2023-10-11 22:46:17,835][71601] Updated weights for policy 0, policy_version 93760 (0.0009) [2023-10-11 22:46:19,188][71635] Updated weights for policy 1, policy_version 93672 (0.0008) [2023-10-11 22:46:19,553][71635] Updated weights for policy 1, policy_version 93682 (0.0007) [2023-10-11 22:46:19,928][71635] Updated weights for policy 1, policy_version 93692 (0.0007) [2023-10-11 22:46:21,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 191954944. Throughput: 0: 1828.7, 1: 1817.2. Samples: 47995658. Policy #0 lag: (min: 29.0, avg: 38.7, max: 61.0) [2023-10-11 22:46:21,034][70582] Avg episode reward: [(0, '125.090'), (1, '113.860')] [2023-10-11 22:46:21,538][71601] Updated weights for policy 0, policy_version 93770 (0.0010) [2023-10-11 22:46:21,903][71601] Updated weights for policy 0, policy_version 93780 (0.0007) [2023-10-11 22:46:22,269][71601] Updated weights for policy 0, policy_version 93790 (0.0008) [2023-10-11 22:46:23,620][71635] Updated weights for policy 1, policy_version 93702 (0.0009) [2023-10-11 22:46:23,987][71635] Updated weights for policy 1, policy_version 93712 (0.0009) [2023-10-11 22:46:24,353][71635] Updated weights for policy 1, policy_version 93722 (0.0011) [2023-10-11 22:46:26,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 192020480. Throughput: 0: 1825.3, 1: 1818.8. Samples: 48017642. Policy #0 lag: (min: 29.0, avg: 38.7, max: 61.0) [2023-10-11 22:46:26,035][70582] Avg episode reward: [(0, '124.760'), (1, '114.950')] [2023-10-11 22:46:26,070][71601] Updated weights for policy 0, policy_version 93800 (0.0008) [2023-10-11 22:46:26,458][71601] Updated weights for policy 0, policy_version 93810 (0.0008) [2023-10-11 22:46:26,829][71601] Updated weights for policy 0, policy_version 93820 (0.0008) [2023-10-11 22:46:27,864][71635] Updated weights for policy 1, policy_version 93732 (0.0008) [2023-10-11 22:46:28,232][71635] Updated weights for policy 1, policy_version 93742 (0.0008) [2023-10-11 22:46:28,599][71635] Updated weights for policy 1, policy_version 93752 (0.0008) [2023-10-11 22:46:30,685][71601] Updated weights for policy 0, policy_version 93830 (0.0010) [2023-10-11 22:46:31,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 192086016. Throughput: 0: 1824.5, 1: 1818.6. Samples: 48028242. Policy #0 lag: (min: 29.0, avg: 38.7, max: 61.0) [2023-10-11 22:46:31,035][70582] Avg episode reward: [(0, '124.240'), (1, '116.180')] [2023-10-11 22:46:31,057][71601] Updated weights for policy 0, policy_version 93840 (0.0009) [2023-10-11 22:46:31,430][71601] Updated weights for policy 0, policy_version 93850 (0.0011) [2023-10-11 22:46:32,216][71635] Updated weights for policy 1, policy_version 93762 (0.0008) [2023-10-11 22:46:32,583][71635] Updated weights for policy 1, policy_version 93772 (0.0007) [2023-10-11 22:46:32,946][71635] Updated weights for policy 1, policy_version 93782 (0.0008) [2023-10-11 22:46:33,310][71635] Updated weights for policy 1, policy_version 93792 (0.0007) [2023-10-11 22:46:35,123][71601] Updated weights for policy 0, policy_version 93860 (0.0008) [2023-10-11 22:46:35,492][71601] Updated weights for policy 0, policy_version 93870 (0.0008) [2023-10-11 22:46:35,870][71601] Updated weights for policy 0, policy_version 93880 (0.0008) [2023-10-11 22:46:36,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 192151552. Throughput: 0: 1824.9, 1: 1823.9. Samples: 48050480. Policy #0 lag: (min: 29.0, avg: 38.7, max: 61.0) [2023-10-11 22:46:36,034][70582] Avg episode reward: [(0, '124.140'), (1, '116.910')] [2023-10-11 22:46:37,024][71635] Updated weights for policy 1, policy_version 93802 (0.0009) [2023-10-11 22:46:37,391][71635] Updated weights for policy 1, policy_version 93812 (0.0011) [2023-10-11 22:46:37,762][71635] Updated weights for policy 1, policy_version 93822 (0.0009) [2023-10-11 22:46:39,651][71601] Updated weights for policy 0, policy_version 93890 (0.0008) [2023-10-11 22:46:40,029][71601] Updated weights for policy 0, policy_version 93900 (0.0008) [2023-10-11 22:46:40,392][71601] Updated weights for policy 0, policy_version 93910 (0.0007) [2023-10-11 22:46:40,760][71601] Updated weights for policy 0, policy_version 93920 (0.0008) [2023-10-11 22:46:41,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 192249856. Throughput: 0: 1825.6, 1: 1830.1. Samples: 48072234. Policy #0 lag: (min: 29.0, avg: 38.7, max: 61.0) [2023-10-11 22:46:41,034][70582] Avg episode reward: [(0, '127.630'), (1, '115.920')] [2023-10-11 22:46:41,296][71635] Updated weights for policy 1, policy_version 93832 (0.0008) [2023-10-11 22:46:41,656][71635] Updated weights for policy 1, policy_version 93842 (0.0008) [2023-10-11 22:46:42,030][71635] Updated weights for policy 1, policy_version 93852 (0.0008) [2023-10-11 22:46:44,435][71601] Updated weights for policy 0, policy_version 93930 (0.0009) [2023-10-11 22:46:44,806][71601] Updated weights for policy 0, policy_version 93940 (0.0008) [2023-10-11 22:46:45,177][71601] Updated weights for policy 0, policy_version 93950 (0.0009) [2023-10-11 22:46:45,725][71635] Updated weights for policy 1, policy_version 93862 (0.0009) [2023-10-11 22:46:46,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 192315392. Throughput: 0: 1814.8, 1: 1835.4. Samples: 48083294. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:46:46,034][70582] Avg episode reward: [(0, '128.530'), (1, '118.410')] [2023-10-11 22:46:46,113][71635] Updated weights for policy 1, policy_version 93872 (0.0007) [2023-10-11 22:46:46,478][71635] Updated weights for policy 1, policy_version 93882 (0.0008) [2023-10-11 22:46:48,813][71601] Updated weights for policy 0, policy_version 93960 (0.0008) [2023-10-11 22:46:49,183][71601] Updated weights for policy 0, policy_version 93970 (0.0007) [2023-10-11 22:46:49,565][71601] Updated weights for policy 0, policy_version 93980 (0.0009) [2023-10-11 22:46:50,218][71635] Updated weights for policy 1, policy_version 93892 (0.0007) [2023-10-11 22:46:50,592][71635] Updated weights for policy 1, policy_version 93902 (0.0008) [2023-10-11 22:46:50,962][71635] Updated weights for policy 1, policy_version 93912 (0.0009) [2023-10-11 22:46:51,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 192380928. Throughput: 0: 1818.4, 1: 1833.9. Samples: 48104958. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:46:51,034][70582] Avg episode reward: [(0, '127.340'), (1, '117.520')] [2023-10-11 22:46:53,241][71601] Updated weights for policy 0, policy_version 93990 (0.0008) [2023-10-11 22:46:53,610][71601] Updated weights for policy 0, policy_version 94000 (0.0008) [2023-10-11 22:46:53,979][71601] Updated weights for policy 0, policy_version 94010 (0.0008) [2023-10-11 22:46:54,685][71635] Updated weights for policy 1, policy_version 93922 (0.0009) [2023-10-11 22:46:55,046][71635] Updated weights for policy 1, policy_version 93932 (0.0009) [2023-10-11 22:46:55,431][71635] Updated weights for policy 1, policy_version 93942 (0.0010) [2023-10-11 22:46:55,803][71635] Updated weights for policy 1, policy_version 93952 (0.0010) [2023-10-11 22:46:56,034][70582] Fps is (10 sec: 16383.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 192479232. Throughput: 0: 1810.0, 1: 1827.5. Samples: 48126438. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:46:56,035][70582] Avg episode reward: [(0, '124.150'), (1, '120.220')] [2023-10-11 22:46:57,613][71601] Updated weights for policy 0, policy_version 94020 (0.0009) [2023-10-11 22:46:57,986][71601] Updated weights for policy 0, policy_version 94030 (0.0007) [2023-10-11 22:46:58,355][71601] Updated weights for policy 0, policy_version 94040 (0.0007) [2023-10-11 22:46:59,614][71635] Updated weights for policy 1, policy_version 93962 (0.0010) [2023-10-11 22:46:59,975][71635] Updated weights for policy 1, policy_version 93972 (0.0008) [2023-10-11 22:47:00,345][71635] Updated weights for policy 1, policy_version 93982 (0.0008) [2023-10-11 22:47:01,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 192544768. Throughput: 0: 1815.5, 1: 1832.7. Samples: 48137604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:47:01,035][70582] Avg episode reward: [(0, '127.440'), (1, '125.400')] [2023-10-11 22:47:02,056][71601] Updated weights for policy 0, policy_version 94050 (0.0008) [2023-10-11 22:47:02,426][71601] Updated weights for policy 0, policy_version 94060 (0.0009) [2023-10-11 22:47:02,791][71601] Updated weights for policy 0, policy_version 94070 (0.0008) [2023-10-11 22:47:03,160][71601] Updated weights for policy 0, policy_version 94080 (0.0007) [2023-10-11 22:47:04,058][71635] Updated weights for policy 1, policy_version 93992 (0.0009) [2023-10-11 22:47:04,420][71635] Updated weights for policy 1, policy_version 94002 (0.0008) [2023-10-11 22:47:04,783][71635] Updated weights for policy 1, policy_version 94012 (0.0009) [2023-10-11 22:47:06,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 192610304. Throughput: 0: 1806.2, 1: 1824.3. Samples: 48159030. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:47:06,034][70582] Avg episode reward: [(0, '133.630'), (1, '125.340')] [2023-10-11 22:47:07,035][71601] Updated weights for policy 0, policy_version 94090 (0.0008) [2023-10-11 22:47:07,407][71601] Updated weights for policy 0, policy_version 94100 (0.0007) [2023-10-11 22:47:07,775][71601] Updated weights for policy 0, policy_version 94110 (0.0007) [2023-10-11 22:47:08,385][71635] Updated weights for policy 1, policy_version 94022 (0.0009) [2023-10-11 22:47:08,743][71635] Updated weights for policy 1, policy_version 94032 (0.0011) [2023-10-11 22:47:09,121][71635] Updated weights for policy 1, policy_version 94042 (0.0010) [2023-10-11 22:47:11,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 192675840. Throughput: 0: 1801.1, 1: 1829.1. Samples: 48181002. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:47:11,035][70582] Avg episode reward: [(0, '135.220'), (1, '121.330')] [2023-10-11 22:47:11,505][71601] Updated weights for policy 0, policy_version 94120 (0.0007) [2023-10-11 22:47:11,878][71601] Updated weights for policy 0, policy_version 94130 (0.0007) [2023-10-11 22:47:12,263][71601] Updated weights for policy 0, policy_version 94140 (0.0008) [2023-10-11 22:47:12,821][71635] Updated weights for policy 1, policy_version 94052 (0.0009) [2023-10-11 22:47:13,198][71635] Updated weights for policy 1, policy_version 94062 (0.0007) [2023-10-11 22:47:13,569][71635] Updated weights for policy 1, policy_version 94072 (0.0007) [2023-10-11 22:47:15,883][71601] Updated weights for policy 0, policy_version 94150 (0.0011) [2023-10-11 22:47:16,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 192741376. Throughput: 0: 1802.5, 1: 1822.2. Samples: 48191354. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:47:16,034][70582] Avg episode reward: [(0, '134.970'), (1, '120.700')] [2023-10-11 22:47:16,252][71601] Updated weights for policy 0, policy_version 94160 (0.0009) [2023-10-11 22:47:16,621][71601] Updated weights for policy 0, policy_version 94170 (0.0011) [2023-10-11 22:47:17,232][71635] Updated weights for policy 1, policy_version 94082 (0.0007) [2023-10-11 22:47:17,597][71635] Updated weights for policy 1, policy_version 94092 (0.0008) [2023-10-11 22:47:17,959][71635] Updated weights for policy 1, policy_version 94102 (0.0007) [2023-10-11 22:47:18,318][71635] Updated weights for policy 1, policy_version 94112 (0.0007) [2023-10-11 22:47:20,534][71601] Updated weights for policy 0, policy_version 94180 (0.0009) [2023-10-11 22:47:20,907][71601] Updated weights for policy 0, policy_version 94190 (0.0008) [2023-10-11 22:47:21,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 192806912. Throughput: 0: 1800.3, 1: 1821.7. Samples: 48213470. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:47:21,034][70582] Avg episode reward: [(0, '134.160'), (1, '112.760')] [2023-10-11 22:47:21,281][71601] Updated weights for policy 0, policy_version 94200 (0.0007) [2023-10-11 22:47:21,961][71635] Updated weights for policy 1, policy_version 94122 (0.0008) [2023-10-11 22:47:22,328][71635] Updated weights for policy 1, policy_version 94132 (0.0008) [2023-10-11 22:47:22,695][71635] Updated weights for policy 1, policy_version 94142 (0.0007) [2023-10-11 22:47:24,830][71601] Updated weights for policy 0, policy_version 94210 (0.0009) [2023-10-11 22:47:25,198][71601] Updated weights for policy 0, policy_version 94220 (0.0009) [2023-10-11 22:47:25,570][71601] Updated weights for policy 0, policy_version 94230 (0.0010) [2023-10-11 22:47:25,946][71601] Updated weights for policy 0, policy_version 94240 (0.0009) [2023-10-11 22:47:26,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 192905216. Throughput: 0: 1809.0, 1: 1819.7. Samples: 48235526. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:47:26,034][70582] Avg episode reward: [(0, '137.330'), (1, '112.920')] [2023-10-11 22:47:26,429][71635] Updated weights for policy 1, policy_version 94152 (0.0009) [2023-10-11 22:47:26,785][71635] Updated weights for policy 1, policy_version 94162 (0.0010) [2023-10-11 22:47:27,152][71635] Updated weights for policy 1, policy_version 94172 (0.0009) [2023-10-11 22:47:29,741][71601] Updated weights for policy 0, policy_version 94250 (0.0007) [2023-10-11 22:47:30,118][71601] Updated weights for policy 0, policy_version 94260 (0.0007) [2023-10-11 22:47:30,494][71601] Updated weights for policy 0, policy_version 94270 (0.0007) [2023-10-11 22:47:31,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 192970752. Throughput: 0: 1804.4, 1: 1815.2. Samples: 48246174. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:47:31,034][70582] Avg episode reward: [(0, '138.670'), (1, '118.630')] [2023-10-11 22:47:31,090][71635] Updated weights for policy 1, policy_version 94182 (0.0008) [2023-10-11 22:47:31,468][71635] Updated weights for policy 1, policy_version 94192 (0.0008) [2023-10-11 22:47:31,836][71635] Updated weights for policy 1, policy_version 94202 (0.0007) [2023-10-11 22:47:33,958][71601] Updated weights for policy 0, policy_version 94280 (0.0009) [2023-10-11 22:47:34,327][71601] Updated weights for policy 0, policy_version 94290 (0.0010) [2023-10-11 22:47:34,702][71601] Updated weights for policy 0, policy_version 94300 (0.0007) [2023-10-11 22:47:35,356][71635] Updated weights for policy 1, policy_version 94212 (0.0008) [2023-10-11 22:47:35,736][71635] Updated weights for policy 1, policy_version 94222 (0.0011) [2023-10-11 22:47:36,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 193036288. Throughput: 0: 1813.5, 1: 1812.5. Samples: 48268128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:47:36,034][70582] Avg episode reward: [(0, '137.960'), (1, '115.730')] [2023-10-11 22:47:36,106][71635] Updated weights for policy 1, policy_version 94232 (0.0008) [2023-10-11 22:47:38,307][71601] Updated weights for policy 0, policy_version 94310 (0.0008) [2023-10-11 22:47:38,672][71601] Updated weights for policy 0, policy_version 94320 (0.0007) [2023-10-11 22:47:39,046][71601] Updated weights for policy 0, policy_version 94330 (0.0009) [2023-10-11 22:47:39,699][71635] Updated weights for policy 1, policy_version 94242 (0.0007) [2023-10-11 22:47:40,058][71635] Updated weights for policy 1, policy_version 94252 (0.0009) [2023-10-11 22:47:40,434][71635] Updated weights for policy 1, policy_version 94262 (0.0010) [2023-10-11 22:47:40,796][71635] Updated weights for policy 1, policy_version 94272 (0.0010) [2023-10-11 22:47:41,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 193134592. Throughput: 0: 1810.6, 1: 1816.9. Samples: 48289676. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:47:41,034][70582] Avg episode reward: [(0, '138.530'), (1, '115.000')] [2023-10-11 22:47:42,701][71601] Updated weights for policy 0, policy_version 94340 (0.0009) [2023-10-11 22:47:43,081][71601] Updated weights for policy 0, policy_version 94350 (0.0008) [2023-10-11 22:47:43,454][71601] Updated weights for policy 0, policy_version 94360 (0.0008) [2023-10-11 22:47:44,703][71635] Updated weights for policy 1, policy_version 94282 (0.0007) [2023-10-11 22:47:45,074][71635] Updated weights for policy 1, policy_version 94292 (0.0008) [2023-10-11 22:47:45,442][71635] Updated weights for policy 1, policy_version 94302 (0.0008) [2023-10-11 22:47:46,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 193200128. Throughput: 0: 1814.5, 1: 1813.6. Samples: 48300868. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-11 22:47:46,034][70582] Avg episode reward: [(0, '139.080'), (1, '109.790')] [2023-10-11 22:47:47,215][71601] Updated weights for policy 0, policy_version 94370 (0.0008) [2023-10-11 22:47:47,591][71601] Updated weights for policy 0, policy_version 94380 (0.0010) [2023-10-11 22:47:47,961][71601] Updated weights for policy 0, policy_version 94390 (0.0008) [2023-10-11 22:47:48,330][71601] Updated weights for policy 0, policy_version 94400 (0.0007) [2023-10-11 22:47:49,069][71635] Updated weights for policy 1, policy_version 94312 (0.0008) [2023-10-11 22:47:49,430][71635] Updated weights for policy 1, policy_version 94322 (0.0007) [2023-10-11 22:47:49,793][71635] Updated weights for policy 1, policy_version 94332 (0.0008) [2023-10-11 22:47:51,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 193265664. Throughput: 0: 1814.6, 1: 1819.4. Samples: 48322560. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-11 22:47:51,034][70582] Avg episode reward: [(0, '140.520'), (1, '109.110')] [2023-10-11 22:47:52,169][71601] Updated weights for policy 0, policy_version 94410 (0.0007) [2023-10-11 22:47:52,544][71601] Updated weights for policy 0, policy_version 94420 (0.0008) [2023-10-11 22:47:52,911][71601] Updated weights for policy 0, policy_version 94430 (0.0009) [2023-10-11 22:47:53,535][71635] Updated weights for policy 1, policy_version 94342 (0.0008) [2023-10-11 22:47:53,901][71635] Updated weights for policy 1, policy_version 94352 (0.0009) [2023-10-11 22:47:54,269][71635] Updated weights for policy 1, policy_version 94362 (0.0008) [2023-10-11 22:47:56,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 193331200. Throughput: 0: 1817.3, 1: 1816.0. Samples: 48344502. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-11 22:47:56,034][70582] Avg episode reward: [(0, '141.320'), (1, '104.340')] [2023-10-11 22:47:56,045][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000094432_96698368.pth... [2023-10-11 22:47:56,045][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000094368_96632832.pth... [2023-10-11 22:47:56,074][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000092736_94961664.pth [2023-10-11 22:47:56,079][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000092672_94896128.pth [2023-10-11 22:47:56,709][71601] Updated weights for policy 0, policy_version 94440 (0.0008) [2023-10-11 22:47:57,083][71601] Updated weights for policy 0, policy_version 94450 (0.0007) [2023-10-11 22:47:57,456][71601] Updated weights for policy 0, policy_version 94460 (0.0007) [2023-10-11 22:47:57,863][71635] Updated weights for policy 1, policy_version 94372 (0.0008) [2023-10-11 22:47:58,236][71635] Updated weights for policy 1, policy_version 94382 (0.0008) [2023-10-11 22:47:58,599][71635] Updated weights for policy 1, policy_version 94392 (0.0010) [2023-10-11 22:48:01,031][71601] Updated weights for policy 0, policy_version 94470 (0.0007) [2023-10-11 22:48:01,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 193396736. Throughput: 0: 1823.3, 1: 1823.7. Samples: 48355470. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-11 22:48:01,035][70582] Avg episode reward: [(0, '144.770'), (1, '101.590')] [2023-10-11 22:48:01,402][71601] Updated weights for policy 0, policy_version 94480 (0.0008) [2023-10-11 22:48:01,762][71601] Updated weights for policy 0, policy_version 94490 (0.0009) [2023-10-11 22:48:02,086][71635] Updated weights for policy 1, policy_version 94402 (0.0008) [2023-10-11 22:48:02,446][71635] Updated weights for policy 1, policy_version 94412 (0.0009) [2023-10-11 22:48:02,815][71635] Updated weights for policy 1, policy_version 94422 (0.0008) [2023-10-11 22:48:03,185][71635] Updated weights for policy 1, policy_version 94432 (0.0008) [2023-10-11 22:48:05,385][71601] Updated weights for policy 0, policy_version 94500 (0.0010) [2023-10-11 22:48:05,767][71601] Updated weights for policy 0, policy_version 94510 (0.0008) [2023-10-11 22:48:06,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 193462272. Throughput: 0: 1825.2, 1: 1825.6. Samples: 48377756. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-11 22:48:06,034][70582] Avg episode reward: [(0, '143.530'), (1, '97.910')] [2023-10-11 22:48:06,131][71601] Updated weights for policy 0, policy_version 94520 (0.0009) [2023-10-11 22:48:06,925][71635] Updated weights for policy 1, policy_version 94442 (0.0008) [2023-10-11 22:48:07,296][71635] Updated weights for policy 1, policy_version 94452 (0.0007) [2023-10-11 22:48:07,647][71635] Updated weights for policy 1, policy_version 94462 (0.0007) [2023-10-11 22:48:09,742][71601] Updated weights for policy 0, policy_version 94530 (0.0009) [2023-10-11 22:48:10,123][71601] Updated weights for policy 0, policy_version 94540 (0.0009) [2023-10-11 22:48:10,485][71601] Updated weights for policy 0, policy_version 94550 (0.0008) [2023-10-11 22:48:10,866][71601] Updated weights for policy 0, policy_version 94560 (0.0007) [2023-10-11 22:48:11,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 193560576. Throughput: 0: 1828.3, 1: 1819.3. Samples: 48399668. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-11 22:48:11,034][70582] Avg episode reward: [(0, '144.700'), (1, '100.190')] [2023-10-11 22:48:11,514][71635] Updated weights for policy 1, policy_version 94472 (0.0008) [2023-10-11 22:48:11,886][71635] Updated weights for policy 1, policy_version 94482 (0.0007) [2023-10-11 22:48:12,245][71635] Updated weights for policy 1, policy_version 94492 (0.0008) [2023-10-11 22:48:14,505][71601] Updated weights for policy 0, policy_version 94570 (0.0008) [2023-10-11 22:48:14,877][71601] Updated weights for policy 0, policy_version 94580 (0.0008) [2023-10-11 22:48:15,241][71601] Updated weights for policy 0, policy_version 94590 (0.0007) [2023-10-11 22:48:16,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 193626112. Throughput: 0: 1828.2, 1: 1819.7. Samples: 48410330. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-11 22:48:16,034][70582] Avg episode reward: [(0, '140.940'), (1, '107.830')] [2023-10-11 22:48:16,044][71635] Updated weights for policy 1, policy_version 94502 (0.0008) [2023-10-11 22:48:16,422][71635] Updated weights for policy 1, policy_version 94512 (0.0008) [2023-10-11 22:48:16,786][71635] Updated weights for policy 1, policy_version 94522 (0.0007) [2023-10-11 22:48:18,787][71601] Updated weights for policy 0, policy_version 94600 (0.0010) [2023-10-11 22:48:19,165][71601] Updated weights for policy 0, policy_version 94610 (0.0008) [2023-10-11 22:48:19,536][71601] Updated weights for policy 0, policy_version 94620 (0.0008) [2023-10-11 22:48:20,396][71635] Updated weights for policy 1, policy_version 94532 (0.0008) [2023-10-11 22:48:20,764][71635] Updated weights for policy 1, policy_version 94542 (0.0009) [2023-10-11 22:48:21,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 193691648. Throughput: 0: 1819.4, 1: 1823.9. Samples: 48432076. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-11 22:48:21,035][70582] Avg episode reward: [(0, '140.030'), (1, '100.410')] [2023-10-11 22:48:21,140][71635] Updated weights for policy 1, policy_version 94552 (0.0008) [2023-10-11 22:48:23,297][71601] Updated weights for policy 0, policy_version 94630 (0.0009) [2023-10-11 22:48:23,670][71601] Updated weights for policy 0, policy_version 94640 (0.0009) [2023-10-11 22:48:24,038][71601] Updated weights for policy 0, policy_version 94650 (0.0010) [2023-10-11 22:48:24,847][71635] Updated weights for policy 1, policy_version 94562 (0.0008) [2023-10-11 22:48:25,210][71635] Updated weights for policy 1, policy_version 94572 (0.0011) [2023-10-11 22:48:25,582][71635] Updated weights for policy 1, policy_version 94582 (0.0010) [2023-10-11 22:48:25,941][71635] Updated weights for policy 1, policy_version 94592 (0.0008) [2023-10-11 22:48:26,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 193789952. Throughput: 0: 1824.5, 1: 1822.0. Samples: 48453770. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-11 22:48:26,035][70582] Avg episode reward: [(0, '140.470'), (1, '99.250')] [2023-10-11 22:48:27,728][71601] Updated weights for policy 0, policy_version 94660 (0.0007) [2023-10-11 22:48:28,095][71601] Updated weights for policy 0, policy_version 94670 (0.0008) [2023-10-11 22:48:28,474][71601] Updated weights for policy 0, policy_version 94680 (0.0009) [2023-10-11 22:48:29,642][71635] Updated weights for policy 1, policy_version 94602 (0.0008) [2023-10-11 22:48:30,009][71635] Updated weights for policy 1, policy_version 94612 (0.0009) [2023-10-11 22:48:30,374][71635] Updated weights for policy 1, policy_version 94622 (0.0008) [2023-10-11 22:48:31,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 193855488. Throughput: 0: 1825.6, 1: 1821.9. Samples: 48465006. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-11 22:48:31,035][70582] Avg episode reward: [(0, '142.880'), (1, '100.430')] [2023-10-11 22:48:32,107][71601] Updated weights for policy 0, policy_version 94690 (0.0008) [2023-10-11 22:48:32,465][71601] Updated weights for policy 0, policy_version 94700 (0.0008) [2023-10-11 22:48:32,840][71601] Updated weights for policy 0, policy_version 94710 (0.0008) [2023-10-11 22:48:33,218][71601] Updated weights for policy 0, policy_version 94720 (0.0009) [2023-10-11 22:48:34,110][71635] Updated weights for policy 1, policy_version 94632 (0.0007) [2023-10-11 22:48:34,481][71635] Updated weights for policy 1, policy_version 94642 (0.0009) [2023-10-11 22:48:34,851][71635] Updated weights for policy 1, policy_version 94652 (0.0008) [2023-10-11 22:48:36,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 193921024. Throughput: 0: 1828.4, 1: 1821.3. Samples: 48486800. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-11 22:48:36,034][70582] Avg episode reward: [(0, '144.120'), (1, '104.620')] [2023-10-11 22:48:36,933][71601] Updated weights for policy 0, policy_version 94730 (0.0007) [2023-10-11 22:48:37,314][71601] Updated weights for policy 0, policy_version 94740 (0.0010) [2023-10-11 22:48:37,683][71601] Updated weights for policy 0, policy_version 94750 (0.0010) [2023-10-11 22:48:38,514][71635] Updated weights for policy 1, policy_version 94662 (0.0007) [2023-10-11 22:48:38,880][71635] Updated weights for policy 1, policy_version 94672 (0.0008) [2023-10-11 22:48:39,246][71635] Updated weights for policy 1, policy_version 94682 (0.0007) [2023-10-11 22:48:41,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 193986560. Throughput: 0: 1834.7, 1: 1819.4. Samples: 48508938. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-11 22:48:41,034][70582] Avg episode reward: [(0, '144.180'), (1, '103.910')] [2023-10-11 22:48:41,415][71601] Updated weights for policy 0, policy_version 94760 (0.0008) [2023-10-11 22:48:41,782][71601] Updated weights for policy 0, policy_version 94770 (0.0007) [2023-10-11 22:48:42,149][71601] Updated weights for policy 0, policy_version 94780 (0.0008) [2023-10-11 22:48:42,914][71635] Updated weights for policy 1, policy_version 94692 (0.0009) [2023-10-11 22:48:43,282][71635] Updated weights for policy 1, policy_version 94702 (0.0009) [2023-10-11 22:48:43,643][71635] Updated weights for policy 1, policy_version 94712 (0.0009) [2023-10-11 22:48:45,606][71601] Updated weights for policy 0, policy_version 94790 (0.0008) [2023-10-11 22:48:45,966][71601] Updated weights for policy 0, policy_version 94800 (0.0007) [2023-10-11 22:48:46,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 194052096. Throughput: 0: 1833.5, 1: 1816.4. Samples: 48519712. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-11 22:48:46,034][70582] Avg episode reward: [(0, '140.810'), (1, '107.420')] [2023-10-11 22:48:46,345][71601] Updated weights for policy 0, policy_version 94810 (0.0007) [2023-10-11 22:48:47,338][71635] Updated weights for policy 1, policy_version 94722 (0.0010) [2023-10-11 22:48:47,699][71635] Updated weights for policy 1, policy_version 94732 (0.0008) [2023-10-11 22:48:48,063][71635] Updated weights for policy 1, policy_version 94742 (0.0007) [2023-10-11 22:48:48,431][71635] Updated weights for policy 1, policy_version 94752 (0.0009) [2023-10-11 22:48:50,047][71601] Updated weights for policy 0, policy_version 94820 (0.0009) [2023-10-11 22:48:50,412][71601] Updated weights for policy 0, policy_version 94830 (0.0007) [2023-10-11 22:48:50,788][71601] Updated weights for policy 0, policy_version 94840 (0.0008) [2023-10-11 22:48:51,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 194117632. Throughput: 0: 1837.3, 1: 1812.2. Samples: 48541982. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-11 22:48:51,034][70582] Avg episode reward: [(0, '145.580'), (1, '110.380')] [2023-10-11 22:48:52,088][71635] Updated weights for policy 1, policy_version 94762 (0.0009) [2023-10-11 22:48:52,458][71635] Updated weights for policy 1, policy_version 94772 (0.0007) [2023-10-11 22:48:52,822][71635] Updated weights for policy 1, policy_version 94782 (0.0008) [2023-10-11 22:48:54,490][71601] Updated weights for policy 0, policy_version 94850 (0.0010) [2023-10-11 22:48:54,857][71601] Updated weights for policy 0, policy_version 94860 (0.0007) [2023-10-11 22:48:55,233][71601] Updated weights for policy 0, policy_version 94870 (0.0007) [2023-10-11 22:48:55,603][71601] Updated weights for policy 0, policy_version 94880 (0.0007) [2023-10-11 22:48:56,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 194215936. Throughput: 0: 1823.2, 1: 1819.8. Samples: 48563604. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-11 22:48:56,035][70582] Avg episode reward: [(0, '142.140'), (1, '113.570')] [2023-10-11 22:48:56,542][71635] Updated weights for policy 1, policy_version 94792 (0.0009) [2023-10-11 22:48:56,905][71635] Updated weights for policy 1, policy_version 94802 (0.0009) [2023-10-11 22:48:57,264][71635] Updated weights for policy 1, policy_version 94812 (0.0009) [2023-10-11 22:48:59,150][71601] Updated weights for policy 0, policy_version 94890 (0.0008) [2023-10-11 22:48:59,519][71601] Updated weights for policy 0, policy_version 94900 (0.0010) [2023-10-11 22:48:59,902][71601] Updated weights for policy 0, policy_version 94910 (0.0008) [2023-10-11 22:49:01,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 194281472. Throughput: 0: 1833.7, 1: 1818.8. Samples: 48574692. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-11 22:49:01,034][70582] Avg episode reward: [(0, '142.970'), (1, '118.030')] [2023-10-11 22:49:01,048][71635] Updated weights for policy 1, policy_version 94822 (0.0009) [2023-10-11 22:49:01,445][71635] Updated weights for policy 1, policy_version 94832 (0.0009) [2023-10-11 22:49:01,820][71635] Updated weights for policy 1, policy_version 94842 (0.0008) [2023-10-11 22:49:03,574][71601] Updated weights for policy 0, policy_version 94920 (0.0007) [2023-10-11 22:49:03,948][71601] Updated weights for policy 0, policy_version 94930 (0.0009) [2023-10-11 22:49:04,315][71601] Updated weights for policy 0, policy_version 94940 (0.0011) [2023-10-11 22:49:05,317][71635] Updated weights for policy 1, policy_version 94852 (0.0009) [2023-10-11 22:49:05,683][71635] Updated weights for policy 1, policy_version 94862 (0.0009) [2023-10-11 22:49:06,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 194347008. Throughput: 0: 1826.4, 1: 1815.1. Samples: 48595942. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-11 22:49:06,034][70582] Avg episode reward: [(0, '140.790'), (1, '117.930')] [2023-10-11 22:49:06,047][71635] Updated weights for policy 1, policy_version 94872 (0.0010) [2023-10-11 22:49:07,994][71601] Updated weights for policy 0, policy_version 94950 (0.0009) [2023-10-11 22:49:08,371][71601] Updated weights for policy 0, policy_version 94960 (0.0009) [2023-10-11 22:49:08,748][71601] Updated weights for policy 0, policy_version 94970 (0.0007) [2023-10-11 22:49:09,818][71635] Updated weights for policy 1, policy_version 94882 (0.0007) [2023-10-11 22:49:10,184][71635] Updated weights for policy 1, policy_version 94892 (0.0007) [2023-10-11 22:49:10,546][71635] Updated weights for policy 1, policy_version 94902 (0.0007) [2023-10-11 22:49:10,916][71635] Updated weights for policy 1, policy_version 94912 (0.0007) [2023-10-11 22:49:11,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 194445312. Throughput: 0: 1834.6, 1: 1820.1. Samples: 48618232. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-11 22:49:11,035][70582] Avg episode reward: [(0, '137.860'), (1, '118.670')] [2023-10-11 22:49:12,436][71601] Updated weights for policy 0, policy_version 94980 (0.0009) [2023-10-11 22:49:12,808][71601] Updated weights for policy 0, policy_version 94990 (0.0008) [2023-10-11 22:49:13,190][71601] Updated weights for policy 0, policy_version 95000 (0.0008) [2023-10-11 22:49:14,652][71635] Updated weights for policy 1, policy_version 94922 (0.0009) [2023-10-11 22:49:15,028][71635] Updated weights for policy 1, policy_version 94932 (0.0008) [2023-10-11 22:49:15,390][71635] Updated weights for policy 1, policy_version 94942 (0.0007) [2023-10-11 22:49:16,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 194510848. Throughput: 0: 1826.4, 1: 1821.5. Samples: 48629162. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-11 22:49:16,035][70582] Avg episode reward: [(0, '138.830'), (1, '118.370')] [2023-10-11 22:49:17,005][71601] Updated weights for policy 0, policy_version 95010 (0.0008) [2023-10-11 22:49:17,369][71601] Updated weights for policy 0, policy_version 95020 (0.0007) [2023-10-11 22:49:17,746][71601] Updated weights for policy 0, policy_version 95030 (0.0007) [2023-10-11 22:49:18,115][71601] Updated weights for policy 0, policy_version 95040 (0.0009) [2023-10-11 22:49:19,125][71635] Updated weights for policy 1, policy_version 94952 (0.0008) [2023-10-11 22:49:19,497][71635] Updated weights for policy 1, policy_version 94962 (0.0009) [2023-10-11 22:49:19,872][71635] Updated weights for policy 1, policy_version 94972 (0.0009) [2023-10-11 22:49:21,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 194576384. Throughput: 0: 1824.2, 1: 1824.6. Samples: 48650994. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-11 22:49:21,034][70582] Avg episode reward: [(0, '138.280'), (1, '119.870')] [2023-10-11 22:49:21,735][71601] Updated weights for policy 0, policy_version 95050 (0.0007) [2023-10-11 22:49:22,109][71601] Updated weights for policy 0, policy_version 95060 (0.0008) [2023-10-11 22:49:22,474][71601] Updated weights for policy 0, policy_version 95070 (0.0007) [2023-10-11 22:49:23,500][71635] Updated weights for policy 1, policy_version 94982 (0.0008) [2023-10-11 22:49:23,864][71635] Updated weights for policy 1, policy_version 94992 (0.0008) [2023-10-11 22:49:24,223][71635] Updated weights for policy 1, policy_version 95002 (0.0008) [2023-10-11 22:49:26,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 194641920. Throughput: 0: 1820.9, 1: 1827.3. Samples: 48673106. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-11 22:49:26,034][70582] Avg episode reward: [(0, '136.450'), (1, '123.080')] [2023-10-11 22:49:26,351][71601] Updated weights for policy 0, policy_version 95080 (0.0007) [2023-10-11 22:49:26,733][71601] Updated weights for policy 0, policy_version 95090 (0.0008) [2023-10-11 22:49:27,102][71601] Updated weights for policy 0, policy_version 95100 (0.0009) [2023-10-11 22:49:27,998][71635] Updated weights for policy 1, policy_version 95012 (0.0009) [2023-10-11 22:49:28,368][71635] Updated weights for policy 1, policy_version 95022 (0.0009) [2023-10-11 22:49:28,734][71635] Updated weights for policy 1, policy_version 95032 (0.0011) [2023-10-11 22:49:30,728][71601] Updated weights for policy 0, policy_version 95110 (0.0008) [2023-10-11 22:49:31,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 194707456. Throughput: 0: 1816.4, 1: 1828.9. Samples: 48683748. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-11 22:49:31,035][70582] Avg episode reward: [(0, '129.490'), (1, '121.480')] [2023-10-11 22:49:31,104][71601] Updated weights for policy 0, policy_version 95120 (0.0009) [2023-10-11 22:49:31,475][71601] Updated weights for policy 0, policy_version 95130 (0.0008) [2023-10-11 22:49:32,297][71635] Updated weights for policy 1, policy_version 95042 (0.0008) [2023-10-11 22:49:32,671][71635] Updated weights for policy 1, policy_version 95052 (0.0010) [2023-10-11 22:49:33,037][71635] Updated weights for policy 1, policy_version 95062 (0.0007) [2023-10-11 22:49:33,410][71635] Updated weights for policy 1, policy_version 95072 (0.0008) [2023-10-11 22:49:35,197][71601] Updated weights for policy 0, policy_version 95140 (0.0008) [2023-10-11 22:49:35,565][71601] Updated weights for policy 0, policy_version 95150 (0.0007) [2023-10-11 22:49:35,943][71601] Updated weights for policy 0, policy_version 95160 (0.0008) [2023-10-11 22:49:36,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 194772992. Throughput: 0: 1807.3, 1: 1822.2. Samples: 48705310. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-11 22:49:36,035][70582] Avg episode reward: [(0, '127.050'), (1, '119.710')] [2023-10-11 22:49:37,139][71635] Updated weights for policy 1, policy_version 95082 (0.0008) [2023-10-11 22:49:37,505][71635] Updated weights for policy 1, policy_version 95092 (0.0008) [2023-10-11 22:49:37,869][71635] Updated weights for policy 1, policy_version 95102 (0.0008) [2023-10-11 22:49:39,743][71601] Updated weights for policy 0, policy_version 95170 (0.0007) [2023-10-11 22:49:40,110][71601] Updated weights for policy 0, policy_version 95180 (0.0008) [2023-10-11 22:49:40,485][71601] Updated weights for policy 0, policy_version 95190 (0.0007) [2023-10-11 22:49:40,843][71601] Updated weights for policy 0, policy_version 95200 (0.0010) [2023-10-11 22:49:41,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 194871296. Throughput: 0: 1814.4, 1: 1823.9. Samples: 48727328. Policy #0 lag: (min: 8.0, avg: 30.8, max: 40.0) [2023-10-11 22:49:41,035][70582] Avg episode reward: [(0, '124.320'), (1, '114.330')] [2023-10-11 22:49:41,405][71635] Updated weights for policy 1, policy_version 95112 (0.0008) [2023-10-11 22:49:41,774][71635] Updated weights for policy 1, policy_version 95122 (0.0008) [2023-10-11 22:49:42,146][71635] Updated weights for policy 1, policy_version 95132 (0.0008) [2023-10-11 22:49:44,583][71601] Updated weights for policy 0, policy_version 95210 (0.0009) [2023-10-11 22:49:44,949][71601] Updated weights for policy 0, policy_version 95220 (0.0009) [2023-10-11 22:49:45,331][71601] Updated weights for policy 0, policy_version 95230 (0.0008) [2023-10-11 22:49:45,962][71635] Updated weights for policy 1, policy_version 95142 (0.0008) [2023-10-11 22:49:46,034][70582] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 194936832. Throughput: 0: 1805.1, 1: 1820.7. Samples: 48737852. Policy #0 lag: (min: 8.0, avg: 30.8, max: 40.0) [2023-10-11 22:49:46,034][70582] Avg episode reward: [(0, '122.360'), (1, '114.560')] [2023-10-11 22:49:46,347][71635] Updated weights for policy 1, policy_version 95152 (0.0009) [2023-10-11 22:49:46,705][71635] Updated weights for policy 1, policy_version 95162 (0.0008) [2023-10-11 22:49:49,061][71601] Updated weights for policy 0, policy_version 95240 (0.0011) [2023-10-11 22:49:49,431][71601] Updated weights for policy 0, policy_version 95250 (0.0011) [2023-10-11 22:49:49,806][71601] Updated weights for policy 0, policy_version 95260 (0.0008) [2023-10-11 22:49:50,230][71635] Updated weights for policy 1, policy_version 95172 (0.0010) [2023-10-11 22:49:50,605][71635] Updated weights for policy 1, policy_version 95182 (0.0010) [2023-10-11 22:49:50,973][71635] Updated weights for policy 1, policy_version 95192 (0.0007) [2023-10-11 22:49:51,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 195002368. Throughput: 0: 1818.0, 1: 1821.4. Samples: 48759712. Policy #0 lag: (min: 8.0, avg: 30.8, max: 40.0) [2023-10-11 22:49:51,034][70582] Avg episode reward: [(0, '124.480'), (1, '114.490')] [2023-10-11 22:49:53,360][71601] Updated weights for policy 0, policy_version 95270 (0.0009) [2023-10-11 22:49:53,732][71601] Updated weights for policy 0, policy_version 95280 (0.0008) [2023-10-11 22:49:54,105][71601] Updated weights for policy 0, policy_version 95290 (0.0008) [2023-10-11 22:49:54,725][71635] Updated weights for policy 1, policy_version 95202 (0.0007) [2023-10-11 22:49:55,087][71635] Updated weights for policy 1, policy_version 95212 (0.0007) [2023-10-11 22:49:55,454][71635] Updated weights for policy 1, policy_version 95222 (0.0008) [2023-10-11 22:49:55,826][71635] Updated weights for policy 1, policy_version 95232 (0.0008) [2023-10-11 22:49:56,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 195100672. Throughput: 0: 1804.7, 1: 1814.9. Samples: 48781114. Policy #0 lag: (min: 8.0, avg: 30.8, max: 40.0) [2023-10-11 22:49:56,035][70582] Avg episode reward: [(0, '130.680'), (1, '110.620')] [2023-10-11 22:49:56,045][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000095296_97583104.pth... [2023-10-11 22:49:56,045][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000095232_97517568.pth... [2023-10-11 22:49:56,076][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000093504_95748096.pth [2023-10-11 22:49:56,076][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000093600_95846400.pth [2023-10-11 22:49:57,758][71601] Updated weights for policy 0, policy_version 95300 (0.0009) [2023-10-11 22:49:58,124][71601] Updated weights for policy 0, policy_version 95310 (0.0008) [2023-10-11 22:49:58,487][71601] Updated weights for policy 0, policy_version 95320 (0.0007) [2023-10-11 22:49:59,569][71635] Updated weights for policy 1, policy_version 95242 (0.0008) [2023-10-11 22:49:59,945][71635] Updated weights for policy 1, policy_version 95252 (0.0008) [2023-10-11 22:50:00,309][71635] Updated weights for policy 1, policy_version 95262 (0.0008) [2023-10-11 22:50:01,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 195166208. Throughput: 0: 1811.1, 1: 1816.7. Samples: 48792412. Policy #0 lag: (min: 8.0, avg: 30.8, max: 40.0) [2023-10-11 22:50:01,035][70582] Avg episode reward: [(0, '137.410'), (1, '103.510')] [2023-10-11 22:50:02,109][71601] Updated weights for policy 0, policy_version 95330 (0.0010) [2023-10-11 22:50:02,477][71601] Updated weights for policy 0, policy_version 95340 (0.0010) [2023-10-11 22:50:02,856][71601] Updated weights for policy 0, policy_version 95350 (0.0010) [2023-10-11 22:50:03,228][71601] Updated weights for policy 0, policy_version 95360 (0.0010) [2023-10-11 22:50:04,083][71635] Updated weights for policy 1, policy_version 95272 (0.0010) [2023-10-11 22:50:04,456][71635] Updated weights for policy 1, policy_version 95282 (0.0011) [2023-10-11 22:50:04,826][71635] Updated weights for policy 1, policy_version 95292 (0.0010) [2023-10-11 22:50:06,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 195231744. Throughput: 0: 1810.0, 1: 1816.1. Samples: 48814172. Policy #0 lag: (min: 8.0, avg: 30.8, max: 40.0) [2023-10-11 22:50:06,034][70582] Avg episode reward: [(0, '139.580'), (1, '96.340')] [2023-10-11 22:50:06,843][71601] Updated weights for policy 0, policy_version 95370 (0.0007) [2023-10-11 22:50:07,225][71601] Updated weights for policy 0, policy_version 95380 (0.0007) [2023-10-11 22:50:07,589][71601] Updated weights for policy 0, policy_version 95390 (0.0007) [2023-10-11 22:50:08,635][71635] Updated weights for policy 1, policy_version 95302 (0.0009) [2023-10-11 22:50:09,013][71635] Updated weights for policy 1, policy_version 95312 (0.0008) [2023-10-11 22:50:09,376][71635] Updated weights for policy 1, policy_version 95322 (0.0010) [2023-10-11 22:50:11,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 195297280. Throughput: 0: 1812.7, 1: 1807.1. Samples: 48836000. Policy #0 lag: (min: 8.0, avg: 30.8, max: 40.0) [2023-10-11 22:50:11,035][70582] Avg episode reward: [(0, '139.390'), (1, '91.550')] [2023-10-11 22:50:11,393][71601] Updated weights for policy 0, policy_version 95400 (0.0008) [2023-10-11 22:50:11,771][71601] Updated weights for policy 0, policy_version 95410 (0.0009) [2023-10-11 22:50:12,141][71601] Updated weights for policy 0, policy_version 95420 (0.0009) [2023-10-11 22:50:13,159][71635] Updated weights for policy 1, policy_version 95332 (0.0009) [2023-10-11 22:50:13,529][71635] Updated weights for policy 1, policy_version 95342 (0.0008) [2023-10-11 22:50:13,896][71635] Updated weights for policy 1, policy_version 95352 (0.0008) [2023-10-11 22:50:15,735][71601] Updated weights for policy 0, policy_version 95430 (0.0010) [2023-10-11 22:50:16,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 195362816. Throughput: 0: 1812.6, 1: 1814.3. Samples: 48846956. Policy #0 lag: (min: 8.0, avg: 30.8, max: 40.0) [2023-10-11 22:50:16,035][70582] Avg episode reward: [(0, '139.430'), (1, '91.900')] [2023-10-11 22:50:16,107][71601] Updated weights for policy 0, policy_version 95440 (0.0008) [2023-10-11 22:50:16,483][71601] Updated weights for policy 0, policy_version 95450 (0.0007) [2023-10-11 22:50:17,623][71635] Updated weights for policy 1, policy_version 95362 (0.0009) [2023-10-11 22:50:17,993][71635] Updated weights for policy 1, policy_version 95372 (0.0009) [2023-10-11 22:50:18,364][71635] Updated weights for policy 1, policy_version 95382 (0.0008) [2023-10-11 22:50:18,734][71635] Updated weights for policy 1, policy_version 95392 (0.0009) [2023-10-11 22:50:20,191][71601] Updated weights for policy 0, policy_version 95460 (0.0009) [2023-10-11 22:50:20,561][71601] Updated weights for policy 0, policy_version 95470 (0.0011) [2023-10-11 22:50:20,925][71601] Updated weights for policy 0, policy_version 95480 (0.0008) [2023-10-11 22:50:21,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 195428352. Throughput: 0: 1820.9, 1: 1807.0. Samples: 48868564. Policy #0 lag: (min: 8.0, avg: 30.8, max: 40.0) [2023-10-11 22:50:21,035][70582] Avg episode reward: [(0, '134.830'), (1, '89.890')] [2023-10-11 22:50:22,452][71635] Updated weights for policy 1, policy_version 95402 (0.0008) [2023-10-11 22:50:22,814][71635] Updated weights for policy 1, policy_version 95412 (0.0011) [2023-10-11 22:50:23,182][71635] Updated weights for policy 1, policy_version 95422 (0.0009) [2023-10-11 22:50:24,666][71601] Updated weights for policy 0, policy_version 95490 (0.0009) [2023-10-11 22:50:25,032][71601] Updated weights for policy 0, policy_version 95500 (0.0009) [2023-10-11 22:50:25,402][71601] Updated weights for policy 0, policy_version 95510 (0.0007) [2023-10-11 22:50:25,776][71601] Updated weights for policy 0, policy_version 95520 (0.0009) [2023-10-11 22:50:26,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 195526656. Throughput: 0: 1819.0, 1: 1800.6. Samples: 48890210. Policy #0 lag: (min: 8.0, avg: 30.8, max: 40.0) [2023-10-11 22:50:26,035][70582] Avg episode reward: [(0, '136.710'), (1, '88.570')] [2023-10-11 22:50:26,732][71635] Updated weights for policy 1, policy_version 95432 (0.0011) [2023-10-11 22:50:27,091][71635] Updated weights for policy 1, policy_version 95442 (0.0009) [2023-10-11 22:50:27,456][71635] Updated weights for policy 1, policy_version 95452 (0.0010) [2023-10-11 22:50:29,421][71601] Updated weights for policy 0, policy_version 95530 (0.0009) [2023-10-11 22:50:29,785][71601] Updated weights for policy 0, policy_version 95540 (0.0010) [2023-10-11 22:50:30,169][71601] Updated weights for policy 0, policy_version 95550 (0.0009) [2023-10-11 22:50:31,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 195592192. Throughput: 0: 1822.8, 1: 1807.1. Samples: 48901196. Policy #0 lag: (min: 8.0, avg: 30.8, max: 40.0) [2023-10-11 22:50:31,034][70582] Avg episode reward: [(0, '135.720'), (1, '86.100')] [2023-10-11 22:50:31,123][71635] Updated weights for policy 1, policy_version 95462 (0.0010) [2023-10-11 22:50:31,511][71635] Updated weights for policy 1, policy_version 95472 (0.0009) [2023-10-11 22:50:31,881][71635] Updated weights for policy 1, policy_version 95482 (0.0009) [2023-10-11 22:50:33,881][71601] Updated weights for policy 0, policy_version 95560 (0.0009) [2023-10-11 22:50:34,251][71601] Updated weights for policy 0, policy_version 95570 (0.0010) [2023-10-11 22:50:34,627][71601] Updated weights for policy 0, policy_version 95580 (0.0008) [2023-10-11 22:50:35,454][71635] Updated weights for policy 1, policy_version 95492 (0.0008) [2023-10-11 22:50:35,824][71635] Updated weights for policy 1, policy_version 95502 (0.0009) [2023-10-11 22:50:36,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 195657728. Throughput: 0: 1820.3, 1: 1809.1. Samples: 48923032. Policy #0 lag: (min: 8.0, avg: 30.8, max: 40.0) [2023-10-11 22:50:36,035][70582] Avg episode reward: [(0, '141.680'), (1, '89.190')] [2023-10-11 22:50:36,188][71635] Updated weights for policy 1, policy_version 95512 (0.0008) [2023-10-11 22:50:38,504][71601] Updated weights for policy 0, policy_version 95590 (0.0007) [2023-10-11 22:50:38,875][71601] Updated weights for policy 0, policy_version 95600 (0.0008) [2023-10-11 22:50:39,244][71601] Updated weights for policy 0, policy_version 95610 (0.0008) [2023-10-11 22:50:40,039][71635] Updated weights for policy 1, policy_version 95522 (0.0009) [2023-10-11 22:50:40,402][71635] Updated weights for policy 1, policy_version 95532 (0.0010) [2023-10-11 22:50:40,777][71635] Updated weights for policy 1, policy_version 95542 (0.0010) [2023-10-11 22:50:41,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 195723264. Throughput: 0: 1821.8, 1: 1816.5. Samples: 48944834. Policy #0 lag: (min: 20.0, avg: 25.2, max: 52.0) [2023-10-11 22:50:41,034][70582] Avg episode reward: [(0, '143.150'), (1, '93.410')] [2023-10-11 22:50:41,139][71635] Updated weights for policy 1, policy_version 95552 (0.0008) [2023-10-11 22:50:42,909][71601] Updated weights for policy 0, policy_version 95620 (0.0010) [2023-10-11 22:50:43,290][71601] Updated weights for policy 0, policy_version 95630 (0.0008) [2023-10-11 22:50:43,661][71601] Updated weights for policy 0, policy_version 95640 (0.0007) [2023-10-11 22:50:44,826][71635] Updated weights for policy 1, policy_version 95562 (0.0008) [2023-10-11 22:50:45,202][71635] Updated weights for policy 1, policy_version 95572 (0.0007) [2023-10-11 22:50:45,568][71635] Updated weights for policy 1, policy_version 95582 (0.0008) [2023-10-11 22:50:46,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 195821568. Throughput: 0: 1824.4, 1: 1812.0. Samples: 48956052. Policy #0 lag: (min: 20.0, avg: 25.2, max: 52.0) [2023-10-11 22:50:46,035][70582] Avg episode reward: [(0, '143.440'), (1, '96.050')] [2023-10-11 22:50:47,238][71601] Updated weights for policy 0, policy_version 95650 (0.0012) [2023-10-11 22:50:47,617][71601] Updated weights for policy 0, policy_version 95660 (0.0009) [2023-10-11 22:50:47,984][71601] Updated weights for policy 0, policy_version 95670 (0.0010) [2023-10-11 22:50:48,356][71601] Updated weights for policy 0, policy_version 95680 (0.0007) [2023-10-11 22:50:49,228][71635] Updated weights for policy 1, policy_version 95592 (0.0008) [2023-10-11 22:50:49,592][71635] Updated weights for policy 1, policy_version 95602 (0.0007) [2023-10-11 22:50:49,962][71635] Updated weights for policy 1, policy_version 95612 (0.0009) [2023-10-11 22:50:51,034][70582] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 195887104. Throughput: 0: 1821.2, 1: 1816.8. Samples: 48977886. Policy #0 lag: (min: 20.0, avg: 25.2, max: 52.0) [2023-10-11 22:50:51,035][70582] Avg episode reward: [(0, '148.090'), (1, '100.150')] [2023-10-11 22:50:51,985][71601] Updated weights for policy 0, policy_version 95690 (0.0007) [2023-10-11 22:50:52,358][71601] Updated weights for policy 0, policy_version 95700 (0.0009) [2023-10-11 22:50:52,744][71601] Updated weights for policy 0, policy_version 95710 (0.0010) [2023-10-11 22:50:53,620][71635] Updated weights for policy 1, policy_version 95622 (0.0011) [2023-10-11 22:50:53,984][71635] Updated weights for policy 1, policy_version 95632 (0.0009) [2023-10-11 22:50:54,347][71635] Updated weights for policy 1, policy_version 95642 (0.0007) [2023-10-11 22:50:56,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 195952640. Throughput: 0: 1818.0, 1: 1823.8. Samples: 48999880. Policy #0 lag: (min: 20.0, avg: 25.2, max: 52.0) [2023-10-11 22:50:56,035][70582] Avg episode reward: [(0, '132.720'), (1, '100.540')] [2023-10-11 22:50:56,497][71601] Updated weights for policy 0, policy_version 95720 (0.0010) [2023-10-11 22:50:56,860][71601] Updated weights for policy 0, policy_version 95730 (0.0009) [2023-10-11 22:50:57,231][71601] Updated weights for policy 0, policy_version 95740 (0.0008) [2023-10-11 22:50:57,984][71635] Updated weights for policy 1, policy_version 95652 (0.0008) [2023-10-11 22:50:58,344][71635] Updated weights for policy 1, policy_version 95662 (0.0009) [2023-10-11 22:50:58,723][71635] Updated weights for policy 1, policy_version 95672 (0.0011) [2023-10-11 22:51:00,876][71601] Updated weights for policy 0, policy_version 95750 (0.0009) [2023-10-11 22:51:01,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 196018176. Throughput: 0: 1817.9, 1: 1818.7. Samples: 49010602. Policy #0 lag: (min: 20.0, avg: 25.2, max: 52.0) [2023-10-11 22:51:01,034][70582] Avg episode reward: [(0, '130.230'), (1, '102.200')] [2023-10-11 22:51:01,255][71601] Updated weights for policy 0, policy_version 95760 (0.0007) [2023-10-11 22:51:01,625][71601] Updated weights for policy 0, policy_version 95770 (0.0008) [2023-10-11 22:51:02,245][71635] Updated weights for policy 1, policy_version 95682 (0.0008) [2023-10-11 22:51:02,609][71635] Updated weights for policy 1, policy_version 95692 (0.0007) [2023-10-11 22:51:02,972][71635] Updated weights for policy 1, policy_version 95702 (0.0007) [2023-10-11 22:51:03,344][71635] Updated weights for policy 1, policy_version 95712 (0.0008) [2023-10-11 22:51:05,295][71601] Updated weights for policy 0, policy_version 95780 (0.0007) [2023-10-11 22:51:05,662][71601] Updated weights for policy 0, policy_version 95790 (0.0010) [2023-10-11 22:51:06,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 196083712. Throughput: 0: 1812.0, 1: 1834.2. Samples: 49032644. Policy #0 lag: (min: 20.0, avg: 25.2, max: 52.0) [2023-10-11 22:51:06,034][70582] Avg episode reward: [(0, '126.690'), (1, '100.660')] [2023-10-11 22:51:06,035][71601] Updated weights for policy 0, policy_version 95800 (0.0011) [2023-10-11 22:51:06,979][71635] Updated weights for policy 1, policy_version 95722 (0.0011) [2023-10-11 22:51:07,344][71635] Updated weights for policy 1, policy_version 95732 (0.0010) [2023-10-11 22:51:07,716][71635] Updated weights for policy 1, policy_version 95742 (0.0011) [2023-10-11 22:51:09,781][71601] Updated weights for policy 0, policy_version 95810 (0.0007) [2023-10-11 22:51:10,148][71601] Updated weights for policy 0, policy_version 95820 (0.0008) [2023-10-11 22:51:10,522][71601] Updated weights for policy 0, policy_version 95830 (0.0008) [2023-10-11 22:51:10,888][71601] Updated weights for policy 0, policy_version 95840 (0.0007) [2023-10-11 22:51:11,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 196182016. Throughput: 0: 1817.8, 1: 1827.2. Samples: 49054234. Policy #0 lag: (min: 20.0, avg: 25.2, max: 52.0) [2023-10-11 22:51:11,034][70582] Avg episode reward: [(0, '122.760'), (1, '102.660')] [2023-10-11 22:51:11,512][71635] Updated weights for policy 1, policy_version 95752 (0.0010) [2023-10-11 22:51:11,872][71635] Updated weights for policy 1, policy_version 95762 (0.0008) [2023-10-11 22:51:12,246][71635] Updated weights for policy 1, policy_version 95772 (0.0007) [2023-10-11 22:51:14,660][71601] Updated weights for policy 0, policy_version 95850 (0.0007) [2023-10-11 22:51:15,022][71601] Updated weights for policy 0, policy_version 95860 (0.0008) [2023-10-11 22:51:15,407][71601] Updated weights for policy 0, policy_version 95870 (0.0010) [2023-10-11 22:51:15,995][71635] Updated weights for policy 1, policy_version 95782 (0.0008) [2023-10-11 22:51:16,034][70582] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 196247552. Throughput: 0: 1812.1, 1: 1828.0. Samples: 49065002. Policy #0 lag: (min: 20.0, avg: 25.2, max: 52.0) [2023-10-11 22:51:16,034][70582] Avg episode reward: [(0, '120.790'), (1, '105.430')] [2023-10-11 22:51:16,362][71635] Updated weights for policy 1, policy_version 95792 (0.0008) [2023-10-11 22:51:16,734][71635] Updated weights for policy 1, policy_version 95802 (0.0009) [2023-10-11 22:51:19,189][71601] Updated weights for policy 0, policy_version 95880 (0.0008) [2023-10-11 22:51:19,561][71601] Updated weights for policy 0, policy_version 95890 (0.0009) [2023-10-11 22:51:19,939][71601] Updated weights for policy 0, policy_version 95900 (0.0008) [2023-10-11 22:51:20,469][71635] Updated weights for policy 1, policy_version 95812 (0.0008) [2023-10-11 22:51:20,841][71635] Updated weights for policy 1, policy_version 95822 (0.0008) [2023-10-11 22:51:21,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 196313088. Throughput: 0: 1814.1, 1: 1826.7. Samples: 49086868. Policy #0 lag: (min: 20.0, avg: 25.2, max: 52.0) [2023-10-11 22:51:21,034][70582] Avg episode reward: [(0, '123.450'), (1, '106.270')] [2023-10-11 22:51:21,213][71635] Updated weights for policy 1, policy_version 95832 (0.0008) [2023-10-11 22:51:23,699][71601] Updated weights for policy 0, policy_version 95910 (0.0010) [2023-10-11 22:51:24,065][71601] Updated weights for policy 0, policy_version 95920 (0.0010) [2023-10-11 22:51:24,441][71601] Updated weights for policy 0, policy_version 95930 (0.0007) [2023-10-11 22:51:24,769][71635] Updated weights for policy 1, policy_version 95842 (0.0008) [2023-10-11 22:51:25,134][71635] Updated weights for policy 1, policy_version 95852 (0.0008) [2023-10-11 22:51:25,500][71635] Updated weights for policy 1, policy_version 95862 (0.0007) [2023-10-11 22:51:25,870][71635] Updated weights for policy 1, policy_version 95872 (0.0009) [2023-10-11 22:51:26,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 196411392. Throughput: 0: 1806.3, 1: 1822.6. Samples: 49108134. Policy #0 lag: (min: 20.0, avg: 25.2, max: 52.0) [2023-10-11 22:51:26,034][70582] Avg episode reward: [(0, '127.910'), (1, '103.350')] [2023-10-11 22:51:28,099][71601] Updated weights for policy 0, policy_version 95940 (0.0008) [2023-10-11 22:51:28,468][71601] Updated weights for policy 0, policy_version 95950 (0.0009) [2023-10-11 22:51:28,831][71601] Updated weights for policy 0, policy_version 95960 (0.0011) [2023-10-11 22:51:29,689][71635] Updated weights for policy 1, policy_version 95882 (0.0008) [2023-10-11 22:51:30,058][71635] Updated weights for policy 1, policy_version 95892 (0.0009) [2023-10-11 22:51:30,434][71635] Updated weights for policy 1, policy_version 95902 (0.0010) [2023-10-11 22:51:31,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 196476928. Throughput: 0: 1808.1, 1: 1824.9. Samples: 49119536. Policy #0 lag: (min: 20.0, avg: 25.2, max: 52.0) [2023-10-11 22:51:31,034][70582] Avg episode reward: [(0, '135.290'), (1, '95.860')] [2023-10-11 22:51:32,621][71601] Updated weights for policy 0, policy_version 95970 (0.0010) [2023-10-11 22:51:32,990][71601] Updated weights for policy 0, policy_version 95980 (0.0009) [2023-10-11 22:51:33,356][71601] Updated weights for policy 0, policy_version 95990 (0.0007) [2023-10-11 22:51:33,723][71601] Updated weights for policy 0, policy_version 96000 (0.0007) [2023-10-11 22:51:34,252][71635] Updated weights for policy 1, policy_version 95912 (0.0008) [2023-10-11 22:51:34,618][71635] Updated weights for policy 1, policy_version 95922 (0.0010) [2023-10-11 22:51:34,979][71635] Updated weights for policy 1, policy_version 95932 (0.0010) [2023-10-11 22:51:36,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 196542464. Throughput: 0: 1798.8, 1: 1820.6. Samples: 49140758. Policy #0 lag: (min: 20.0, avg: 25.2, max: 52.0) [2023-10-11 22:51:36,035][70582] Avg episode reward: [(0, '132.160'), (1, '95.490')] [2023-10-11 22:51:37,403][71601] Updated weights for policy 0, policy_version 96010 (0.0007) [2023-10-11 22:51:37,776][71601] Updated weights for policy 0, policy_version 96020 (0.0010) [2023-10-11 22:51:38,154][71601] Updated weights for policy 0, policy_version 96030 (0.0008) [2023-10-11 22:51:38,544][71635] Updated weights for policy 1, policy_version 95942 (0.0011) [2023-10-11 22:51:38,923][71635] Updated weights for policy 1, policy_version 95952 (0.0012) [2023-10-11 22:51:39,287][71635] Updated weights for policy 1, policy_version 95962 (0.0010) [2023-10-11 22:51:41,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 196608000. Throughput: 0: 1803.6, 1: 1815.8. Samples: 49162752. Policy #0 lag: (min: 15.0, avg: 23.1, max: 47.0) [2023-10-11 22:51:41,034][70582] Avg episode reward: [(0, '135.790'), (1, '95.350')] [2023-10-11 22:51:41,885][71601] Updated weights for policy 0, policy_version 96040 (0.0008) [2023-10-11 22:51:42,254][71601] Updated weights for policy 0, policy_version 96050 (0.0008) [2023-10-11 22:51:42,627][71601] Updated weights for policy 0, policy_version 96060 (0.0011) [2023-10-11 22:51:43,051][71635] Updated weights for policy 1, policy_version 95972 (0.0007) [2023-10-11 22:51:43,419][71635] Updated weights for policy 1, policy_version 95982 (0.0007) [2023-10-11 22:51:43,794][71635] Updated weights for policy 1, policy_version 95992 (0.0008) [2023-10-11 22:51:46,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 196673536. Throughput: 0: 1804.4, 1: 1812.8. Samples: 49173374. Policy #0 lag: (min: 15.0, avg: 23.1, max: 47.0) [2023-10-11 22:51:46,034][70582] Avg episode reward: [(0, '132.950'), (1, '98.760')] [2023-10-11 22:51:46,308][71601] Updated weights for policy 0, policy_version 96070 (0.0008) [2023-10-11 22:51:46,681][71601] Updated weights for policy 0, policy_version 96080 (0.0008) [2023-10-11 22:51:47,042][71601] Updated weights for policy 0, policy_version 96090 (0.0008) [2023-10-11 22:51:47,412][71635] Updated weights for policy 1, policy_version 96002 (0.0010) [2023-10-11 22:51:47,782][71635] Updated weights for policy 1, policy_version 96012 (0.0008) [2023-10-11 22:51:48,137][71635] Updated weights for policy 1, policy_version 96022 (0.0008) [2023-10-11 22:51:48,502][71635] Updated weights for policy 1, policy_version 96032 (0.0007) [2023-10-11 22:51:50,703][71601] Updated weights for policy 0, policy_version 96100 (0.0008) [2023-10-11 22:51:51,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 196739072. Throughput: 0: 1808.9, 1: 1808.8. Samples: 49195444. Policy #0 lag: (min: 15.0, avg: 23.1, max: 47.0) [2023-10-11 22:51:51,034][70582] Avg episode reward: [(0, '133.330'), (1, '96.260')] [2023-10-11 22:51:51,085][71601] Updated weights for policy 0, policy_version 96110 (0.0008) [2023-10-11 22:51:51,460][71601] Updated weights for policy 0, policy_version 96120 (0.0007) [2023-10-11 22:51:52,253][71635] Updated weights for policy 1, policy_version 96042 (0.0011) [2023-10-11 22:51:52,616][71635] Updated weights for policy 1, policy_version 96052 (0.0012) [2023-10-11 22:51:52,985][71635] Updated weights for policy 1, policy_version 96062 (0.0011) [2023-10-11 22:51:55,056][71601] Updated weights for policy 0, policy_version 96130 (0.0009) [2023-10-11 22:51:55,437][71601] Updated weights for policy 0, policy_version 96140 (0.0007) [2023-10-11 22:51:55,807][71601] Updated weights for policy 0, policy_version 96150 (0.0007) [2023-10-11 22:51:56,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 196804608. Throughput: 0: 1822.5, 1: 1815.2. Samples: 49217934. Policy #0 lag: (min: 15.0, avg: 23.1, max: 47.0) [2023-10-11 22:51:56,034][70582] Avg episode reward: [(0, '134.790'), (1, '92.270')] [2023-10-11 22:51:56,044][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000096064_98369536.pth... [2023-10-11 22:51:56,081][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000094368_96632832.pth [2023-10-11 22:51:56,169][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000096160_98467840.pth... [2023-10-11 22:51:56,171][71601] Updated weights for policy 0, policy_version 96160 (0.0010) [2023-10-11 22:51:56,198][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000094432_96698368.pth [2023-10-11 22:51:56,739][71635] Updated weights for policy 1, policy_version 96072 (0.0009) [2023-10-11 22:51:57,106][71635] Updated weights for policy 1, policy_version 96082 (0.0011) [2023-10-11 22:51:57,466][71635] Updated weights for policy 1, policy_version 96092 (0.0011) [2023-10-11 22:51:59,655][71601] Updated weights for policy 0, policy_version 96170 (0.0010) [2023-10-11 22:52:00,033][71601] Updated weights for policy 0, policy_version 96180 (0.0008) [2023-10-11 22:52:00,418][71601] Updated weights for policy 0, policy_version 96190 (0.0009) [2023-10-11 22:52:01,034][70582] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 196902912. Throughput: 0: 1816.3, 1: 1815.1. Samples: 49228414. Policy #0 lag: (min: 15.0, avg: 23.1, max: 47.0) [2023-10-11 22:52:01,035][70582] Avg episode reward: [(0, '138.860'), (1, '99.900')] [2023-10-11 22:52:01,290][71635] Updated weights for policy 1, policy_version 96102 (0.0008) [2023-10-11 22:52:01,664][71635] Updated weights for policy 1, policy_version 96112 (0.0009) [2023-10-11 22:52:02,034][71635] Updated weights for policy 1, policy_version 96122 (0.0008) [2023-10-11 22:52:04,069][71601] Updated weights for policy 0, policy_version 96200 (0.0009) [2023-10-11 22:52:04,453][71601] Updated weights for policy 0, policy_version 96210 (0.0008) [2023-10-11 22:52:04,824][71601] Updated weights for policy 0, policy_version 96220 (0.0007) [2023-10-11 22:52:05,659][71635] Updated weights for policy 1, policy_version 96132 (0.0007) [2023-10-11 22:52:06,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 196968448. Throughput: 0: 1821.6, 1: 1819.8. Samples: 49250732. Policy #0 lag: (min: 15.0, avg: 23.1, max: 47.0) [2023-10-11 22:52:06,035][70582] Avg episode reward: [(0, '146.450'), (1, '101.620')] [2023-10-11 22:52:06,037][71635] Updated weights for policy 1, policy_version 96142 (0.0009) [2023-10-11 22:52:06,393][71635] Updated weights for policy 1, policy_version 96152 (0.0010) [2023-10-11 22:52:08,581][71601] Updated weights for policy 0, policy_version 96230 (0.0007) [2023-10-11 22:52:08,958][71601] Updated weights for policy 0, policy_version 96240 (0.0009) [2023-10-11 22:52:09,327][71601] Updated weights for policy 0, policy_version 96250 (0.0008) [2023-10-11 22:52:09,973][71635] Updated weights for policy 1, policy_version 96162 (0.0010) [2023-10-11 22:52:10,331][71635] Updated weights for policy 1, policy_version 96172 (0.0009) [2023-10-11 22:52:10,702][71635] Updated weights for policy 1, policy_version 96182 (0.0007) [2023-10-11 22:52:11,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 197033984. Throughput: 0: 1830.1, 1: 1824.5. Samples: 49272592. Policy #0 lag: (min: 15.0, avg: 23.1, max: 47.0) [2023-10-11 22:52:11,035][70582] Avg episode reward: [(0, '153.780'), (1, '100.410')] [2023-10-11 22:52:11,067][71635] Updated weights for policy 1, policy_version 96192 (0.0008) [2023-10-11 22:52:12,978][71601] Updated weights for policy 0, policy_version 96260 (0.0007) [2023-10-11 22:52:13,350][71601] Updated weights for policy 0, policy_version 96270 (0.0008) [2023-10-11 22:52:13,735][71601] Updated weights for policy 0, policy_version 96280 (0.0008) [2023-10-11 22:52:14,820][71635] Updated weights for policy 1, policy_version 96202 (0.0010) [2023-10-11 22:52:15,185][71635] Updated weights for policy 1, policy_version 96212 (0.0010) [2023-10-11 22:52:15,551][71635] Updated weights for policy 1, policy_version 96222 (0.0009) [2023-10-11 22:52:16,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 197132288. Throughput: 0: 1832.9, 1: 1816.2. Samples: 49283746. Policy #0 lag: (min: 15.0, avg: 23.1, max: 47.0) [2023-10-11 22:52:16,034][70582] Avg episode reward: [(0, '156.770'), (1, '103.140')] [2023-10-11 22:52:17,143][71601] Updated weights for policy 0, policy_version 96290 (0.0009) [2023-10-11 22:52:17,502][71601] Updated weights for policy 0, policy_version 96300 (0.0008) [2023-10-11 22:52:17,865][71601] Updated weights for policy 0, policy_version 96310 (0.0007) [2023-10-11 22:52:18,236][71601] Updated weights for policy 0, policy_version 96320 (0.0008) [2023-10-11 22:52:19,360][71635] Updated weights for policy 1, policy_version 96232 (0.0011) [2023-10-11 22:52:19,723][71635] Updated weights for policy 1, policy_version 96242 (0.0012) [2023-10-11 22:52:20,094][71635] Updated weights for policy 1, policy_version 96252 (0.0010) [2023-10-11 22:52:21,034][70582] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 197197824. Throughput: 0: 1841.5, 1: 1818.1. Samples: 49305438. Policy #0 lag: (min: 15.0, avg: 23.1, max: 47.0) [2023-10-11 22:52:21,034][70582] Avg episode reward: [(0, '162.870'), (1, '105.920')] [2023-10-11 22:52:22,028][71601] Updated weights for policy 0, policy_version 96330 (0.0007) [2023-10-11 22:52:22,400][71601] Updated weights for policy 0, policy_version 96340 (0.0011) [2023-10-11 22:52:22,780][71601] Updated weights for policy 0, policy_version 96350 (0.0009) [2023-10-11 22:52:23,742][71635] Updated weights for policy 1, policy_version 96262 (0.0008) [2023-10-11 22:52:24,113][71635] Updated weights for policy 1, policy_version 96272 (0.0010) [2023-10-11 22:52:24,477][71635] Updated weights for policy 1, policy_version 96282 (0.0007) [2023-10-11 22:52:26,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 197263360. Throughput: 0: 1835.9, 1: 1817.1. Samples: 49327136. Policy #0 lag: (min: 15.0, avg: 23.1, max: 47.0) [2023-10-11 22:52:26,034][70582] Avg episode reward: [(0, '160.020'), (1, '102.850')] [2023-10-11 22:52:26,529][71601] Updated weights for policy 0, policy_version 96360 (0.0007) [2023-10-11 22:52:26,902][71601] Updated weights for policy 0, policy_version 96370 (0.0009) [2023-10-11 22:52:27,278][71601] Updated weights for policy 0, policy_version 96380 (0.0011) [2023-10-11 22:52:28,244][71635] Updated weights for policy 1, policy_version 96292 (0.0007) [2023-10-11 22:52:28,612][71635] Updated weights for policy 1, policy_version 96302 (0.0007) [2023-10-11 22:52:28,984][71635] Updated weights for policy 1, policy_version 96312 (0.0008) [2023-10-11 22:52:30,985][71601] Updated weights for policy 0, policy_version 96390 (0.0009) [2023-10-11 22:52:31,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 197328896. Throughput: 0: 1835.7, 1: 1822.9. Samples: 49338012. Policy #0 lag: (min: 15.0, avg: 23.1, max: 47.0) [2023-10-11 22:52:31,034][70582] Avg episode reward: [(0, '161.940'), (1, '104.630')] [2023-10-11 22:52:31,375][71601] Updated weights for policy 0, policy_version 96400 (0.0008) [2023-10-11 22:52:31,753][71601] Updated weights for policy 0, policy_version 96410 (0.0008) [2023-10-11 22:52:32,541][71635] Updated weights for policy 1, policy_version 96322 (0.0009) [2023-10-11 22:52:32,911][71635] Updated weights for policy 1, policy_version 96332 (0.0010) [2023-10-11 22:52:33,275][71635] Updated weights for policy 1, policy_version 96342 (0.0008) [2023-10-11 22:52:33,649][71635] Updated weights for policy 1, policy_version 96352 (0.0008) [2023-10-11 22:52:35,346][71601] Updated weights for policy 0, policy_version 96420 (0.0008) [2023-10-11 22:52:35,717][71601] Updated weights for policy 0, policy_version 96430 (0.0008) [2023-10-11 22:52:36,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 197394432. Throughput: 0: 1832.6, 1: 1820.8. Samples: 49359850. Policy #0 lag: (min: 15.0, avg: 23.1, max: 47.0) [2023-10-11 22:52:36,034][70582] Avg episode reward: [(0, '166.760'), (1, '111.650')] [2023-10-11 22:52:36,093][71601] Updated weights for policy 0, policy_version 96440 (0.0008) [2023-10-11 22:52:37,366][71635] Updated weights for policy 1, policy_version 96362 (0.0008) [2023-10-11 22:52:37,733][71635] Updated weights for policy 1, policy_version 96372 (0.0010) [2023-10-11 22:52:38,112][71635] Updated weights for policy 1, policy_version 96382 (0.0009) [2023-10-11 22:52:39,781][71601] Updated weights for policy 0, policy_version 96450 (0.0008) [2023-10-11 22:52:40,149][71601] Updated weights for policy 0, policy_version 96460 (0.0008) [2023-10-11 22:52:40,525][71601] Updated weights for policy 0, policy_version 96470 (0.0010) [2023-10-11 22:52:40,883][71601] Updated weights for policy 0, policy_version 96480 (0.0007) [2023-10-11 22:52:41,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 197492736. Throughput: 0: 1818.0, 1: 1816.6. Samples: 49381490. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-11 22:52:41,034][70582] Avg episode reward: [(0, '165.740'), (1, '109.830')] [2023-10-11 22:52:41,943][71635] Updated weights for policy 1, policy_version 96392 (0.0009) [2023-10-11 22:52:42,305][71635] Updated weights for policy 1, policy_version 96402 (0.0009) [2023-10-11 22:52:42,678][71635] Updated weights for policy 1, policy_version 96412 (0.0009) [2023-10-11 22:52:44,633][71601] Updated weights for policy 0, policy_version 96490 (0.0007) [2023-10-11 22:52:45,008][71601] Updated weights for policy 0, policy_version 96500 (0.0007) [2023-10-11 22:52:45,377][71601] Updated weights for policy 0, policy_version 96510 (0.0007) [2023-10-11 22:52:46,034][70582] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 197558272. Throughput: 0: 1828.7, 1: 1813.9. Samples: 49392332. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-11 22:52:46,034][70582] Avg episode reward: [(0, '158.920'), (1, '115.010')] [2023-10-11 22:52:46,374][71635] Updated weights for policy 1, policy_version 96422 (0.0009) [2023-10-11 22:52:46,758][71635] Updated weights for policy 1, policy_version 96432 (0.0009) [2023-10-11 22:52:47,128][71635] Updated weights for policy 1, policy_version 96442 (0.0008) [2023-10-11 22:52:49,087][71601] Updated weights for policy 0, policy_version 96520 (0.0008) [2023-10-11 22:52:49,463][71601] Updated weights for policy 0, policy_version 96530 (0.0008) [2023-10-11 22:52:49,839][71601] Updated weights for policy 0, policy_version 96540 (0.0010) [2023-10-11 22:52:50,546][71635] Updated weights for policy 1, policy_version 96452 (0.0009) [2023-10-11 22:52:50,919][71635] Updated weights for policy 1, policy_version 96462 (0.0007) [2023-10-11 22:52:51,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 197623808. Throughput: 0: 1825.8, 1: 1812.9. Samples: 49414474. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-11 22:52:51,034][70582] Avg episode reward: [(0, '159.850'), (1, '119.950')] [2023-10-11 22:52:51,282][71635] Updated weights for policy 1, policy_version 96472 (0.0008) [2023-10-11 22:52:53,300][71601] Updated weights for policy 0, policy_version 96550 (0.0008) [2023-10-11 22:52:53,665][71601] Updated weights for policy 0, policy_version 96560 (0.0009) [2023-10-11 22:52:54,038][71601] Updated weights for policy 0, policy_version 96570 (0.0008) [2023-10-11 22:52:55,033][71635] Updated weights for policy 1, policy_version 96482 (0.0008) [2023-10-11 22:52:55,398][71635] Updated weights for policy 1, policy_version 96492 (0.0008) [2023-10-11 22:52:55,755][71635] Updated weights for policy 1, policy_version 96502 (0.0007) [2023-10-11 22:52:56,034][70582] Fps is (10 sec: 13106.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 197689344. Throughput: 0: 1819.5, 1: 1816.9. Samples: 49436230. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-11 22:52:56,035][70582] Avg episode reward: [(0, '157.100'), (1, '116.760')] [2023-10-11 22:52:56,127][71635] Updated weights for policy 1, policy_version 96512 (0.0007) [2023-10-11 22:52:57,862][71601] Updated weights for policy 0, policy_version 96580 (0.0010) [2023-10-11 22:52:58,233][71601] Updated weights for policy 0, policy_version 96590 (0.0007) [2023-10-11 22:52:58,610][71601] Updated weights for policy 0, policy_version 96600 (0.0007) [2023-10-11 22:52:59,694][71635] Updated weights for policy 1, policy_version 96522 (0.0011) [2023-10-11 22:53:00,053][71635] Updated weights for policy 1, policy_version 96532 (0.0008) [2023-10-11 22:53:00,424][71635] Updated weights for policy 1, policy_version 96542 (0.0010) [2023-10-11 22:53:01,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 197787648. Throughput: 0: 1812.4, 1: 1820.4. Samples: 49447226. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-11 22:53:01,034][70582] Avg episode reward: [(0, '159.580'), (1, '119.760')] [2023-10-11 22:53:02,357][71601] Updated weights for policy 0, policy_version 96610 (0.0008) [2023-10-11 22:53:02,740][71601] Updated weights for policy 0, policy_version 96620 (0.0009) [2023-10-11 22:53:03,117][71601] Updated weights for policy 0, policy_version 96630 (0.0007) [2023-10-11 22:53:03,486][71601] Updated weights for policy 0, policy_version 96640 (0.0008) [2023-10-11 22:53:04,220][71635] Updated weights for policy 1, policy_version 96552 (0.0010) [2023-10-11 22:53:04,587][71635] Updated weights for policy 1, policy_version 96562 (0.0011) [2023-10-11 22:53:04,953][71635] Updated weights for policy 1, policy_version 96572 (0.0007) [2023-10-11 22:53:06,034][70582] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 197853184. Throughput: 0: 1810.8, 1: 1818.0. Samples: 49468732. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-11 22:53:06,035][70582] Avg episode reward: [(0, '153.860'), (1, '108.040')] [2023-10-11 22:53:07,131][71601] Updated weights for policy 0, policy_version 96650 (0.0008) [2023-10-11 22:53:07,508][71601] Updated weights for policy 0, policy_version 96660 (0.0008) [2023-10-11 22:53:07,879][71601] Updated weights for policy 0, policy_version 96670 (0.0007) [2023-10-11 22:53:08,624][71635] Updated weights for policy 1, policy_version 96582 (0.0007) [2023-10-11 22:53:08,987][71635] Updated weights for policy 1, policy_version 96592 (0.0009) [2023-10-11 22:53:09,356][71635] Updated weights for policy 1, policy_version 96602 (0.0007) [2023-10-11 22:53:11,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 197918720. Throughput: 0: 1815.2, 1: 1824.2. Samples: 49490910. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-11 22:53:11,034][70582] Avg episode reward: [(0, '154.060'), (1, '109.310')] [2023-10-11 22:53:11,466][71601] Updated weights for policy 0, policy_version 96680 (0.0008) [2023-10-11 22:53:11,838][71601] Updated weights for policy 0, policy_version 96690 (0.0007) [2023-10-11 22:53:12,215][71601] Updated weights for policy 0, policy_version 96700 (0.0007) [2023-10-11 22:53:12,989][71635] Updated weights for policy 1, policy_version 96612 (0.0009) [2023-10-11 22:53:13,351][71635] Updated weights for policy 1, policy_version 96622 (0.0010) [2023-10-11 22:53:13,725][71635] Updated weights for policy 1, policy_version 96632 (0.0009) [2023-10-11 22:53:16,009][71601] Updated weights for policy 0, policy_version 96710 (0.0008) [2023-10-11 22:53:16,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 197984256. Throughput: 0: 1820.4, 1: 1821.3. Samples: 49501888. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-11 22:53:16,034][70582] Avg episode reward: [(0, '145.080'), (1, '111.580')] [2023-10-11 22:53:16,393][71601] Updated weights for policy 0, policy_version 96720 (0.0010) [2023-10-11 22:53:16,756][71601] Updated weights for policy 0, policy_version 96730 (0.0008) [2023-10-11 22:53:17,462][71635] Updated weights for policy 1, policy_version 96642 (0.0010) [2023-10-11 22:53:17,821][71635] Updated weights for policy 1, policy_version 96652 (0.0008) [2023-10-11 22:53:18,179][71635] Updated weights for policy 1, policy_version 96662 (0.0007) [2023-10-11 22:53:18,542][71635] Updated weights for policy 1, policy_version 96672 (0.0007) [2023-10-11 22:53:20,385][71601] Updated weights for policy 0, policy_version 96740 (0.0009) [2023-10-11 22:53:20,765][71601] Updated weights for policy 0, policy_version 96750 (0.0007) [2023-10-11 22:53:21,033][70582] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 198049792. Throughput: 0: 1823.2, 1: 1817.2. Samples: 49523664. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-11 22:53:21,034][70582] Avg episode reward: [(0, '138.760'), (1, '115.010')] [2023-10-11 22:53:21,141][71601] Updated weights for policy 0, policy_version 96760 (0.0007) [2023-10-11 22:53:22,230][71635] Updated weights for policy 1, policy_version 96682 (0.0009) [2023-10-11 22:53:22,604][71635] Updated weights for policy 1, policy_version 96692 (0.0007) [2023-10-11 22:53:22,981][71635] Updated weights for policy 1, policy_version 96702 (0.0008) [2023-10-11 22:53:24,927][71601] Updated weights for policy 0, policy_version 96770 (0.0007) [2023-10-11 22:53:25,299][71601] Updated weights for policy 0, policy_version 96780 (0.0008) [2023-10-11 22:53:25,665][71601] Updated weights for policy 0, policy_version 96790 (0.0008) [2023-10-11 22:53:26,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 198115328. Throughput: 0: 1825.6, 1: 1826.6. Samples: 49545838. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-11 22:53:26,034][70582] Avg episode reward: [(0, '141.370'), (1, '115.630')] [2023-10-11 22:53:26,044][71601] Updated weights for policy 0, policy_version 96800 (0.0007) [2023-10-11 22:53:26,638][71635] Updated weights for policy 1, policy_version 96712 (0.0011) [2023-10-11 22:53:27,001][71635] Updated weights for policy 1, policy_version 96722 (0.0011) [2023-10-11 22:53:27,376][71635] Updated weights for policy 1, policy_version 96732 (0.0011) [2023-10-11 22:53:29,536][71601] Updated weights for policy 0, policy_version 96810 (0.0007) [2023-10-11 22:53:29,911][71601] Updated weights for policy 0, policy_version 96820 (0.0007) [2023-10-11 22:53:30,283][71601] Updated weights for policy 0, policy_version 96830 (0.0010) [2023-10-11 22:53:31,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 198213632. Throughput: 0: 1825.4, 1: 1827.7. Samples: 49556722. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-11 22:53:31,034][70582] Avg episode reward: [(0, '137.650'), (1, '119.830')] [2023-10-11 22:53:31,072][71635] Updated weights for policy 1, policy_version 96742 (0.0010) [2023-10-11 22:53:31,429][71635] Updated weights for policy 1, policy_version 96752 (0.0008) [2023-10-11 22:53:31,793][71635] Updated weights for policy 1, policy_version 96762 (0.0008) [2023-10-11 22:53:33,854][71601] Updated weights for policy 0, policy_version 96840 (0.0008) [2023-10-11 22:53:34,223][71601] Updated weights for policy 0, policy_version 96850 (0.0010) [2023-10-11 22:53:34,592][71601] Updated weights for policy 0, policy_version 96860 (0.0010) [2023-10-11 22:53:35,503][71635] Updated weights for policy 1, policy_version 96772 (0.0008) [2023-10-11 22:53:35,866][71635] Updated weights for policy 1, policy_version 96782 (0.0009) [2023-10-11 22:53:36,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 198279168. Throughput: 0: 1823.6, 1: 1828.4. Samples: 49578812. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-11 22:53:36,035][70582] Avg episode reward: [(0, '127.150'), (1, '118.430')] [2023-10-11 22:53:36,231][71635] Updated weights for policy 1, policy_version 96792 (0.0009) [2023-10-11 22:53:38,280][71601] Updated weights for policy 0, policy_version 96870 (0.0011) [2023-10-11 22:53:38,653][71601] Updated weights for policy 0, policy_version 96880 (0.0008) [2023-10-11 22:53:39,028][71601] Updated weights for policy 0, policy_version 96890 (0.0008) [2023-10-11 22:53:39,956][71635] Updated weights for policy 1, policy_version 96802 (0.0011) [2023-10-11 22:53:40,335][71635] Updated weights for policy 1, policy_version 96812 (0.0010) [2023-10-11 22:53:40,699][71635] Updated weights for policy 1, policy_version 96822 (0.0010) [2023-10-11 22:53:41,034][70582] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 198344704. Throughput: 0: 1827.3, 1: 1826.0. Samples: 49600632. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-11 22:53:41,035][70582] Avg episode reward: [(0, '123.130'), (1, '122.780')] [2023-10-11 22:53:41,057][71635] Updated weights for policy 1, policy_version 96832 (0.0010) [2023-10-11 22:53:42,763][71601] Updated weights for policy 0, policy_version 96900 (0.0009) [2023-10-11 22:53:43,132][71601] Updated weights for policy 0, policy_version 96910 (0.0007) [2023-10-11 22:53:43,499][71601] Updated weights for policy 0, policy_version 96920 (0.0007) [2023-10-11 22:53:44,613][71635] Updated weights for policy 1, policy_version 96842 (0.0008) [2023-10-11 22:53:44,980][71635] Updated weights for policy 1, policy_version 96852 (0.0010) [2023-10-11 22:53:45,351][71635] Updated weights for policy 1, policy_version 96862 (0.0009) [2023-10-11 22:53:46,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 198443008. Throughput: 0: 1828.3, 1: 1826.2. Samples: 49611676. Policy #0 lag: (min: 14.0, avg: 25.3, max: 46.0) [2023-10-11 22:53:46,035][70582] Avg episode reward: [(0, '113.430'), (1, '121.950')] [2023-10-11 22:53:47,259][71601] Updated weights for policy 0, policy_version 96930 (0.0009) [2023-10-11 22:53:47,626][71601] Updated weights for policy 0, policy_version 96940 (0.0008) [2023-10-11 22:53:48,006][71601] Updated weights for policy 0, policy_version 96950 (0.0008) [2023-10-11 22:53:48,381][71601] Updated weights for policy 0, policy_version 96960 (0.0008) [2023-10-11 22:53:48,943][71635] Updated weights for policy 1, policy_version 96872 (0.0010) [2023-10-11 22:53:49,308][71635] Updated weights for policy 1, policy_version 96882 (0.0009) [2023-10-11 22:53:49,681][71635] Updated weights for policy 1, policy_version 96892 (0.0010) [2023-10-11 22:53:51,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 198508544. Throughput: 0: 1832.6, 1: 1822.6. Samples: 49633214. Policy #0 lag: (min: 14.0, avg: 25.3, max: 46.0) [2023-10-11 22:53:51,035][70582] Avg episode reward: [(0, '118.980'), (1, '121.070')] [2023-10-11 22:53:52,054][71601] Updated weights for policy 0, policy_version 96970 (0.0007) [2023-10-11 22:53:52,419][71601] Updated weights for policy 0, policy_version 96980 (0.0007) [2023-10-11 22:53:52,795][71601] Updated weights for policy 0, policy_version 96990 (0.0009) [2023-10-11 22:53:53,259][71635] Updated weights for policy 1, policy_version 96902 (0.0009) [2023-10-11 22:53:53,624][71635] Updated weights for policy 1, policy_version 96912 (0.0009) [2023-10-11 22:53:54,000][71635] Updated weights for policy 1, policy_version 96922 (0.0009) [2023-10-11 22:53:56,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 198574080. Throughput: 0: 1831.0, 1: 1830.5. Samples: 49655676. Policy #0 lag: (min: 14.0, avg: 25.3, max: 46.0) [2023-10-11 22:53:56,034][70582] Avg episode reward: [(0, '119.740'), (1, '121.140')] [2023-10-11 22:53:56,044][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000096928_99254272.pth... [2023-10-11 22:53:56,073][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000095232_97517568.pth [2023-10-11 22:53:56,363][71601] Updated weights for policy 0, policy_version 97000 (0.0010) [2023-10-11 22:53:56,736][71601] Updated weights for policy 0, policy_version 97010 (0.0008) [2023-10-11 22:53:57,110][71601] Updated weights for policy 0, policy_version 97020 (0.0007) [2023-10-11 22:53:57,258][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000097024_99352576.pth... [2023-10-11 22:53:57,286][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000095296_97583104.pth [2023-10-11 22:53:57,744][71635] Updated weights for policy 1, policy_version 96932 (0.0008) [2023-10-11 22:53:58,111][71635] Updated weights for policy 1, policy_version 96942 (0.0007) [2023-10-11 22:53:58,470][71635] Updated weights for policy 1, policy_version 96952 (0.0008) [2023-10-11 22:54:00,953][71601] Updated weights for policy 0, policy_version 97030 (0.0008) [2023-10-11 22:54:01,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 198639616. Throughput: 0: 1828.2, 1: 1821.9. Samples: 49666144. Policy #0 lag: (min: 14.0, avg: 25.3, max: 46.0) [2023-10-11 22:54:01,035][70582] Avg episode reward: [(0, '120.690'), (1, '120.080')] [2023-10-11 22:54:01,343][71601] Updated weights for policy 0, policy_version 97040 (0.0010) [2023-10-11 22:54:01,717][71601] Updated weights for policy 0, policy_version 97050 (0.0008) [2023-10-11 22:54:02,188][71635] Updated weights for policy 1, policy_version 96962 (0.0008) [2023-10-11 22:54:02,551][71635] Updated weights for policy 1, policy_version 96972 (0.0007) [2023-10-11 22:54:02,923][71635] Updated weights for policy 1, policy_version 96982 (0.0007) [2023-10-11 22:54:03,279][71635] Updated weights for policy 1, policy_version 96992 (0.0008) [2023-10-11 22:54:05,458][71601] Updated weights for policy 0, policy_version 97060 (0.0009) [2023-10-11 22:54:05,825][71601] Updated weights for policy 0, policy_version 97070 (0.0007) [2023-10-11 22:54:06,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 198705152. Throughput: 0: 1819.5, 1: 1833.6. Samples: 49688054. Policy #0 lag: (min: 14.0, avg: 25.3, max: 46.0) [2023-10-11 22:54:06,034][70582] Avg episode reward: [(0, '113.480'), (1, '128.770')] [2023-10-11 22:54:06,200][71601] Updated weights for policy 0, policy_version 97080 (0.0007) [2023-10-11 22:54:07,031][71635] Updated weights for policy 1, policy_version 97002 (0.0007) [2023-10-11 22:54:07,401][71635] Updated weights for policy 1, policy_version 97012 (0.0007) [2023-10-11 22:54:07,777][71635] Updated weights for policy 1, policy_version 97022 (0.0007) [2023-10-11 22:54:09,920][71601] Updated weights for policy 0, policy_version 97090 (0.0009) [2023-10-11 22:54:10,299][71601] Updated weights for policy 0, policy_version 97100 (0.0009) [2023-10-11 22:54:10,668][71601] Updated weights for policy 0, policy_version 97110 (0.0009) [2023-10-11 22:54:11,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 198770688. Throughput: 0: 1820.2, 1: 1833.1. Samples: 49710236. Policy #0 lag: (min: 14.0, avg: 25.3, max: 46.0) [2023-10-11 22:54:11,035][70582] Avg episode reward: [(0, '112.620'), (1, '132.730')] [2023-10-11 22:54:11,043][71601] Updated weights for policy 0, policy_version 97120 (0.0008) [2023-10-11 22:54:11,564][71635] Updated weights for policy 1, policy_version 97032 (0.0008) [2023-10-11 22:54:11,918][71635] Updated weights for policy 1, policy_version 97042 (0.0009) [2023-10-11 22:54:12,280][71635] Updated weights for policy 1, policy_version 97052 (0.0007) [2023-10-11 22:54:14,639][71601] Updated weights for policy 0, policy_version 97130 (0.0007) [2023-10-11 22:54:15,019][71601] Updated weights for policy 0, policy_version 97140 (0.0009) [2023-10-11 22:54:15,384][71601] Updated weights for policy 0, policy_version 97150 (0.0009) [2023-10-11 22:54:16,014][71635] Updated weights for policy 1, policy_version 97062 (0.0010) [2023-10-11 22:54:16,034][70582] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 198868992. Throughput: 0: 1812.1, 1: 1833.4. Samples: 49720772. Policy #0 lag: (min: 14.0, avg: 25.3, max: 46.0) [2023-10-11 22:54:16,035][70582] Avg episode reward: [(0, '112.920'), (1, '136.300')] [2023-10-11 22:54:16,389][71635] Updated weights for policy 1, policy_version 97072 (0.0007) [2023-10-11 22:54:16,755][71635] Updated weights for policy 1, policy_version 97082 (0.0008) [2023-10-11 22:54:19,094][71601] Updated weights for policy 0, policy_version 97160 (0.0009) [2023-10-11 22:54:19,463][71601] Updated weights for policy 0, policy_version 97170 (0.0008) [2023-10-11 22:54:19,834][71601] Updated weights for policy 0, policy_version 97180 (0.0010) [2023-10-11 22:54:20,240][71635] Updated weights for policy 1, policy_version 97092 (0.0007) [2023-10-11 22:54:20,605][71635] Updated weights for policy 1, policy_version 97102 (0.0007) [2023-10-11 22:54:20,974][71635] Updated weights for policy 1, policy_version 97112 (0.0007) [2023-10-11 22:54:21,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 198934528. Throughput: 0: 1811.6, 1: 1832.5. Samples: 49742800. Policy #0 lag: (min: 14.0, avg: 25.3, max: 46.0) [2023-10-11 22:54:21,034][70582] Avg episode reward: [(0, '107.630'), (1, '138.950')] [2023-10-11 22:54:23,435][71601] Updated weights for policy 0, policy_version 97190 (0.0008) [2023-10-11 22:54:23,798][71601] Updated weights for policy 0, policy_version 97200 (0.0008) [2023-10-11 22:54:24,161][71601] Updated weights for policy 0, policy_version 97210 (0.0011) [2023-10-11 22:54:24,591][71635] Updated weights for policy 1, policy_version 97122 (0.0009) [2023-10-11 22:54:24,964][71635] Updated weights for policy 1, policy_version 97132 (0.0007) [2023-10-11 22:54:25,323][71635] Updated weights for policy 1, policy_version 97142 (0.0007) [2023-10-11 22:54:25,692][71635] Updated weights for policy 1, policy_version 97152 (0.0008) [2023-10-11 22:54:26,034][70582] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 199032832. Throughput: 0: 1812.7, 1: 1822.3. Samples: 49764204. Policy #0 lag: (min: 14.0, avg: 25.3, max: 46.0) [2023-10-11 22:54:26,035][70582] Avg episode reward: [(0, '112.800'), (1, '136.330')] [2023-10-11 22:54:27,678][71601] Updated weights for policy 0, policy_version 97220 (0.0010) [2023-10-11 22:54:28,036][71601] Updated weights for policy 0, policy_version 97230 (0.0009) [2023-10-11 22:54:28,416][71601] Updated weights for policy 0, policy_version 97240 (0.0008) [2023-10-11 22:54:29,511][71635] Updated weights for policy 1, policy_version 97162 (0.0008) [2023-10-11 22:54:29,882][71635] Updated weights for policy 1, policy_version 97172 (0.0007) [2023-10-11 22:54:30,256][71635] Updated weights for policy 1, policy_version 97182 (0.0008) [2023-10-11 22:54:31,034][70582] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 199098368. Throughput: 0: 1809.2, 1: 1831.6. Samples: 49775510. Policy #0 lag: (min: 14.0, avg: 25.3, max: 46.0) [2023-10-11 22:54:31,034][70582] Avg episode reward: [(0, '117.380'), (1, '128.330')] [2023-10-11 22:54:32,139][71601] Updated weights for policy 0, policy_version 97250 (0.0008) [2023-10-11 22:54:32,505][71601] Updated weights for policy 0, policy_version 97260 (0.0010) [2023-10-11 22:54:32,881][71601] Updated weights for policy 0, policy_version 97270 (0.0008) [2023-10-11 22:54:33,258][71601] Updated weights for policy 0, policy_version 97280 (0.0008) [2023-10-11 22:54:33,889][71635] Updated weights for policy 1, policy_version 97192 (0.0008) [2023-10-11 22:54:34,260][71635] Updated weights for policy 1, policy_version 97202 (0.0007) [2023-10-11 22:54:34,626][71635] Updated weights for policy 1, policy_version 97212 (0.0007) [2023-10-11 22:54:36,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 199163904. Throughput: 0: 1807.6, 1: 1827.2. Samples: 49796780. Policy #0 lag: (min: 14.0, avg: 25.3, max: 46.0) [2023-10-11 22:54:36,034][70582] Avg episode reward: [(0, '117.990'), (1, '120.440')] [2023-10-11 22:54:37,117][71601] Updated weights for policy 0, policy_version 97290 (0.0008) [2023-10-11 22:54:37,491][71601] Updated weights for policy 0, policy_version 97300 (0.0009) [2023-10-11 22:54:37,873][71601] Updated weights for policy 0, policy_version 97310 (0.0011) [2023-10-11 22:54:38,248][71635] Updated weights for policy 1, policy_version 97222 (0.0008) [2023-10-11 22:54:38,611][71635] Updated weights for policy 1, policy_version 97232 (0.0010) [2023-10-11 22:54:38,983][71635] Updated weights for policy 1, policy_version 97242 (0.0009) [2023-10-11 22:54:41,034][70582] Fps is (10 sec: 13106.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 199229440. Throughput: 0: 1804.2, 1: 1825.7. Samples: 49819024. Policy #0 lag: (min: 14.0, avg: 25.3, max: 46.0) [2023-10-11 22:54:41,035][70582] Avg episode reward: [(0, '124.650'), (1, '119.340')] [2023-10-11 22:54:41,693][71601] Updated weights for policy 0, policy_version 97320 (0.0009) [2023-10-11 22:54:42,077][71601] Updated weights for policy 0, policy_version 97330 (0.0010) [2023-10-11 22:54:42,449][71601] Updated weights for policy 0, policy_version 97340 (0.0008) [2023-10-11 22:54:42,676][71635] Updated weights for policy 1, policy_version 97252 (0.0009) [2023-10-11 22:54:43,049][71635] Updated weights for policy 1, policy_version 97262 (0.0009) [2023-10-11 22:54:43,408][71635] Updated weights for policy 1, policy_version 97272 (0.0008) [2023-10-11 22:54:46,034][70582] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 199294976. Throughput: 0: 1803.6, 1: 1823.1. Samples: 49829342. Policy #0 lag: (min: 14.0, avg: 25.3, max: 46.0) [2023-10-11 22:54:46,035][70582] Avg episode reward: [(0, '123.090'), (1, '118.030')] [2023-10-11 22:54:46,363][71601] Updated weights for policy 0, policy_version 97350 (0.0010) [2023-10-11 22:54:46,739][71601] Updated weights for policy 0, policy_version 97360 (0.0008) [2023-10-11 22:54:47,107][71601] Updated weights for policy 0, policy_version 97370 (0.0007) [2023-10-11 22:54:47,133][71635] Updated weights for policy 1, policy_version 97282 (0.0007) [2023-10-11 22:54:47,493][71635] Updated weights for policy 1, policy_version 97292 (0.0008) [2023-10-11 22:54:47,864][71635] Updated weights for policy 1, policy_version 97302 (0.0010) [2023-10-11 22:54:48,223][71635] Updated weights for policy 1, policy_version 97312 (0.0011) [2023-10-11 22:54:50,658][71601] Updated weights for policy 0, policy_version 97380 (0.0007) [2023-10-11 22:54:51,034][71601] Updated weights for policy 0, policy_version 97390 (0.0010) [2023-10-11 22:54:51,034][70582] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 199360512. Throughput: 0: 1809.0, 1: 1826.0. Samples: 49851630. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:54:51,035][70582] Avg episode reward: [(0, '119.000'), (1, '117.970')] [2023-10-11 22:54:51,399][71601] Updated weights for policy 0, policy_version 97400 (0.0008) [2023-10-11 22:54:51,944][71635] Updated weights for policy 1, policy_version 97322 (0.0011) [2023-10-11 22:54:52,308][71635] Updated weights for policy 1, policy_version 97332 (0.0009) [2023-10-11 22:54:52,674][71635] Updated weights for policy 1, policy_version 97342 (0.0007) [2023-10-11 22:54:55,057][71601] Updated weights for policy 0, policy_version 97410 (0.0008) [2023-10-11 22:54:55,432][71601] Updated weights for policy 0, policy_version 97420 (0.0011) [2023-10-11 22:54:55,797][71601] Updated weights for policy 0, policy_version 97430 (0.0009) [2023-10-11 22:54:56,034][70582] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 199426048. Throughput: 0: 1815.5, 1: 1824.7. Samples: 49874046. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:54:56,034][70582] Avg episode reward: [(0, '120.610'), (1, '118.130')] [2023-10-11 22:54:56,167][71601] Updated weights for policy 0, policy_version 97440 (0.0008) [2023-10-11 22:54:56,464][71635] Updated weights for policy 1, policy_version 97352 (0.0009) [2023-10-11 22:54:56,837][71635] Updated weights for policy 1, policy_version 97362 (0.0008) [2023-10-11 22:54:57,194][71635] Updated weights for policy 1, policy_version 97372 (0.0008) [2023-10-11 22:54:59,871][71601] Updated weights for policy 0, policy_version 97450 (0.0008) [2023-10-11 22:55:00,249][71601] Updated weights for policy 0, policy_version 97460 (0.0009) [2023-10-11 22:55:00,611][71601] Updated weights for policy 0, policy_version 97470 (0.0007) [2023-10-11 22:55:00,860][71635] Updated weights for policy 1, policy_version 97382 (0.0011) [2023-10-11 22:55:01,034][70582] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 199524352. Throughput: 0: 1812.7, 1: 1825.3. Samples: 49884484. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:55:01,034][70582] Avg episode reward: [(0, '124.190'), (1, '111.900')] [2023-10-11 22:55:01,245][71635] Updated weights for policy 1, policy_version 97392 (0.0008) [2023-10-11 22:55:01,605][71635] Updated weights for policy 1, policy_version 97402 (0.0007) [2023-10-11 22:55:04,311][71601] Updated weights for policy 0, policy_version 97480 (0.0007) [2023-10-11 22:55:04,675][71601] Updated weights for policy 0, policy_version 97490 (0.0008) [2023-10-11 22:55:05,050][71601] Updated weights for policy 0, policy_version 97500 (0.0007) [2023-10-11 22:55:05,314][71635] Updated weights for policy 1, policy_version 97412 (0.0009) [2023-10-11 22:55:05,684][71635] Updated weights for policy 1, policy_version 97422 (0.0008) [2023-10-11 22:55:06,034][70582] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 199589888. Throughput: 0: 1819.6, 1: 1824.9. Samples: 49906804. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:55:06,034][70582] Avg episode reward: [(0, '130.410'), (1, '105.630')] [2023-10-11 22:55:06,051][71635] Updated weights for policy 1, policy_version 97432 (0.0008) [2023-10-11 22:55:08,648][71601] Updated weights for policy 0, policy_version 97510 (0.0010) [2023-10-11 22:55:09,018][71601] Updated weights for policy 0, policy_version 97520 (0.0007) [2023-10-11 22:55:09,393][71601] Updated weights for policy 0, policy_version 97530 (0.0008) [2023-10-11 22:55:09,855][71635] Updated weights for policy 1, policy_version 97442 (0.0008) [2023-10-11 22:55:10,222][71635] Updated weights for policy 1, policy_version 97452 (0.0007) [2023-10-11 22:55:10,595][71635] Updated weights for policy 1, policy_version 97462 (0.0008) [2023-10-11 22:55:10,959][71635] Updated weights for policy 1, policy_version 97472 (0.0008) [2023-10-11 22:55:11,034][70582] Fps is (10 sec: 16383.8, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 199688192. Throughput: 0: 1811.3, 1: 1826.8. Samples: 49927916. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:55:11,035][70582] Avg episode reward: [(0, '128.650'), (1, '103.500')] [2023-10-11 22:55:13,037][71601] Updated weights for policy 0, policy_version 97540 (0.0008) [2023-10-11 22:55:13,416][71601] Updated weights for policy 0, policy_version 97550 (0.0008) [2023-10-11 22:55:13,778][71601] Updated weights for policy 0, policy_version 97560 (0.0008) [2023-10-11 22:55:14,710][71635] Updated weights for policy 1, policy_version 97482 (0.0009) [2023-10-11 22:55:15,080][71635] Updated weights for policy 1, policy_version 97492 (0.0007) [2023-10-11 22:55:15,442][71635] Updated weights for policy 1, policy_version 97502 (0.0009) [2023-10-11 22:55:16,034][70582] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 199753728. Throughput: 0: 1820.7, 1: 1819.6. Samples: 49939326. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:55:16,035][70582] Avg episode reward: [(0, '134.690'), (1, '103.550')] [2023-10-11 22:55:17,493][71601] Updated weights for policy 0, policy_version 97570 (0.0011) [2023-10-11 22:55:17,863][71601] Updated weights for policy 0, policy_version 97580 (0.0011) [2023-10-11 22:55:18,237][71601] Updated weights for policy 0, policy_version 97590 (0.0008) [2023-10-11 22:55:18,601][71601] Updated weights for policy 0, policy_version 97600 (0.0009) [2023-10-11 22:55:19,004][71635] Updated weights for policy 1, policy_version 97512 (0.0009) [2023-10-11 22:55:19,369][71635] Updated weights for policy 1, policy_version 97522 (0.0008) [2023-10-11 22:55:19,739][71635] Updated weights for policy 1, policy_version 97532 (0.0009) [2023-10-11 22:55:21,034][70582] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 199819264. Throughput: 0: 1813.7, 1: 1824.9. Samples: 49960520. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:55:21,034][70582] Avg episode reward: [(0, '135.150'), (1, '103.830')] [2023-10-11 22:55:22,384][71601] Updated weights for policy 0, policy_version 97610 (0.0008) [2023-10-11 22:55:22,752][71601] Updated weights for policy 0, policy_version 97620 (0.0009) [2023-10-11 22:55:23,123][71601] Updated weights for policy 0, policy_version 97630 (0.0007) [2023-10-11 22:55:23,476][71635] Updated weights for policy 1, policy_version 97542 (0.0009) [2023-10-11 22:55:23,839][71635] Updated weights for policy 1, policy_version 97552 (0.0010) [2023-10-11 22:55:24,201][71635] Updated weights for policy 1, policy_version 97562 (0.0007) [2023-10-11 22:55:26,034][70582] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 199884800. Throughput: 0: 1817.4, 1: 1820.5. Samples: 49982732. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:55:26,034][70582] Avg episode reward: [(0, '134.730'), (1, '109.650')] [2023-10-11 22:55:26,799][71601] Updated weights for policy 0, policy_version 97640 (0.0009) [2023-10-11 22:55:27,172][71601] Updated weights for policy 0, policy_version 97650 (0.0007) [2023-10-11 22:55:27,545][71601] Updated weights for policy 0, policy_version 97660 (0.0010) [2023-10-11 22:55:27,837][71635] Updated weights for policy 1, policy_version 97572 (0.0008) [2023-10-11 22:55:28,199][71635] Updated weights for policy 1, policy_version 97582 (0.0007) [2023-10-11 22:55:28,561][71635] Updated weights for policy 1, policy_version 97592 (0.0008) [2023-10-11 22:55:31,034][70582] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 199950336. Throughput: 0: 1820.0, 1: 1828.5. Samples: 49993520. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:55:31,034][70582] Avg episode reward: [(0, '127.130'), (1, '107.830')] [2023-10-11 22:55:31,292][71601] Updated weights for policy 0, policy_version 97670 (0.0007) [2023-10-11 22:55:31,668][71601] Updated weights for policy 0, policy_version 97680 (0.0007) [2023-10-11 22:55:32,045][71601] Updated weights for policy 0, policy_version 97690 (0.0008) [2023-10-11 22:55:32,321][71635] Updated weights for policy 1, policy_version 97602 (0.0011) [2023-10-11 22:55:32,688][71635] Updated weights for policy 1, policy_version 97612 (0.0008) [2023-10-11 22:55:33,061][71635] Updated weights for policy 1, policy_version 97622 (0.0007) [2023-10-11 22:55:33,436][71635] Updated weights for policy 1, policy_version 97632 (0.0009) [2023-10-11 22:55:35,745][71601] Updated weights for policy 0, policy_version 97700 (0.0009) [2023-10-11 22:55:36,034][70582] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 200015872. Throughput: 0: 1814.8, 1: 1818.8. Samples: 50015138. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:55:36,035][70582] Avg episode reward: [(0, '131.530'), (1, '107.930')] [2023-10-11 22:55:36,112][71601] Updated weights for policy 0, policy_version 97710 (0.0010) [2023-10-11 22:55:36,487][71601] Updated weights for policy 0, policy_version 97720 (0.0011) [2023-10-11 22:55:37,153][71635] Updated weights for policy 1, policy_version 97642 (0.0010) [2023-10-11 22:55:37,518][71635] Updated weights for policy 1, policy_version 97652 (0.0007) [2023-10-11 22:55:37,885][71635] Updated weights for policy 1, policy_version 97662 (0.0007) [2023-10-11 22:55:40,259][71601] Updated weights for policy 0, policy_version 97730 (0.0009) [2023-10-11 22:55:40,635][71601] Updated weights for policy 0, policy_version 97740 (0.0010) [2023-10-11 22:55:41,010][71601] Updated weights for policy 0, policy_version 97750 (0.0011) [2023-10-11 22:55:41,034][70582] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 200081408. Throughput: 0: 1815.8, 1: 1813.7. Samples: 50037372. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-11 22:55:41,034][70582] Avg episode reward: [(0, '128.890'), (1, '109.300')] [2023-10-11 22:55:41,386][71601] Updated weights for policy 0, policy_version 97760 (0.0007) [2023-10-11 22:55:41,386][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000097760_100106240.pth... [2023-10-11 22:55:41,386][71641] Stopping RolloutWorker_w5... [2023-10-11 22:55:41,387][71639] Stopping RolloutWorker_w3... [2023-10-11 22:55:41,387][71640] Stopping RolloutWorker_w4... [2023-10-11 22:55:41,387][71647] Stopping RolloutWorker_w10... [2023-10-11 22:55:41,387][71646] Stopping RolloutWorker_w11... [2023-10-11 22:55:41,387][71638] Stopping RolloutWorker_w1... [2023-10-11 22:55:41,387][71637] Stopping RolloutWorker_w2... [2023-10-11 22:55:41,387][71641] Loop rollout_proc5_evt_loop terminating... [2023-10-11 22:55:41,387][71642] Stopping RolloutWorker_w6... [2023-10-11 22:55:41,387][71639] Loop rollout_proc3_evt_loop terminating... [2023-10-11 22:55:41,387][71640] Loop rollout_proc4_evt_loop terminating... [2023-10-11 22:55:41,387][71645] Stopping RolloutWorker_w9... [2023-10-11 22:55:41,387][71647] Loop rollout_proc10_evt_loop terminating... [2023-10-11 22:55:41,387][71638] Loop rollout_proc1_evt_loop terminating... [2023-10-11 22:55:41,387][71637] Loop rollout_proc2_evt_loop terminating... [2023-10-11 22:55:41,387][71642] Loop rollout_proc6_evt_loop terminating... [2023-10-11 22:55:41,387][71649] Stopping RolloutWorker_w13... [2023-10-11 22:55:41,387][71646] Loop rollout_proc11_evt_loop terminating... [2023-10-11 22:55:41,387][71645] Loop rollout_proc9_evt_loop terminating... [2023-10-11 22:55:41,387][71649] Loop rollout_proc13_evt_loop terminating... [2023-10-11 22:55:41,387][71431] Stopping Batcher_1... [2023-10-11 22:55:41,387][70582] Component RolloutWorker_w5 stopped! [2023-10-11 22:55:41,388][71644] Stopping RolloutWorker_w8... [2023-10-11 22:55:41,388][71634] Stopping RolloutWorker_w0... [2023-10-11 22:55:41,388][71431] Loop batcher_evt_loop terminating... [2023-10-11 22:55:41,388][71644] Loop rollout_proc8_evt_loop terminating... [2023-10-11 22:55:41,388][70582] Component RolloutWorker_w11 stopped! [2023-10-11 22:55:41,388][71634] Loop rollout_proc0_evt_loop terminating... [2023-10-11 22:55:41,389][70582] Component RolloutWorker_w4 stopped! [2023-10-11 22:55:41,389][71648] Stopping RolloutWorker_w12... [2023-10-11 22:55:41,389][70582] Component RolloutWorker_w10 stopped! [2023-10-11 22:55:41,389][72289] Stopping RolloutWorker_w14... [2023-10-11 22:55:41,389][71648] Loop rollout_proc12_evt_loop terminating... [2023-10-11 22:55:41,390][72289] Loop rollout_proc14_evt_loop terminating... [2023-10-11 22:55:41,390][70582] Component RolloutWorker_w3 stopped! [2023-10-11 22:55:41,390][71643] Stopping RolloutWorker_w7... [2023-10-11 22:55:41,390][70582] Component RolloutWorker_w2 stopped! [2023-10-11 22:55:41,390][72321] Stopping RolloutWorker_w15... [2023-10-11 22:55:41,391][71643] Loop rollout_proc7_evt_loop terminating... [2023-10-11 22:55:41,391][70582] Component RolloutWorker_w1 stopped! [2023-10-11 22:55:41,391][72321] Loop rollout_proc15_evt_loop terminating... [2023-10-11 22:55:41,391][70582] Component RolloutWorker_w6 stopped! [2023-10-11 22:55:41,392][70582] Component RolloutWorker_w9 stopped! [2023-10-11 22:55:41,393][70582] Component RolloutWorker_w13 stopped! [2023-10-11 22:55:41,393][70582] Component Batcher_1 stopped! [2023-10-11 22:55:41,393][70582] Component RolloutWorker_w8 stopped! [2023-10-11 22:55:41,393][70582] Component RolloutWorker_w0 stopped! [2023-10-11 22:55:41,394][70582] Component RolloutWorker_w12 stopped! [2023-10-11 22:55:41,394][70582] Component RolloutWorker_w14 stopped! [2023-10-11 22:55:41,394][70582] Component RolloutWorker_w7 stopped! [2023-10-11 22:55:41,394][70582] Component RolloutWorker_w15 stopped! [2023-10-11 22:55:41,394][70582] Component Batcher_0 stopped! [2023-10-11 22:55:41,392][71353] Stopping Batcher_0... [2023-10-11 22:55:41,406][71601] Weights refcount: 2 0 [2023-10-11 22:55:41,407][71601] Stopping InferenceWorker_p0-w0... [2023-10-11 22:55:41,408][71601] Loop inference_proc0-0_evt_loop terminating... [2023-10-11 22:55:41,408][70582] Component InferenceWorker_p0-w0 stopped! [2023-10-11 22:55:41,410][71635] Weights refcount: 2 0 [2023-10-11 22:55:41,411][71635] Stopping InferenceWorker_p1-w0... [2023-10-11 22:55:41,412][71635] Loop inference_proc1-0_evt_loop terminating... [2023-10-11 22:55:41,412][70582] Component InferenceWorker_p1-w0 stopped! [2023-10-11 22:55:41,420][71353] Loop batcher_evt_loop terminating... [2023-10-11 22:55:41,433][71353] Removing ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000096160_98467840.pth [2023-10-11 22:55:41,439][71353] Saving ./train_atari/atari_gopher_APPO/checkpoint_p0/checkpoint_000097760_100106240.pth... [2023-10-11 22:55:41,494][71353] Stopping LearnerWorker_p0... [2023-10-11 22:55:41,495][71353] Loop learner_proc0_evt_loop terminating... [2023-10-11 22:55:41,494][70582] Component LearnerWorker_p0 stopped! [2023-10-11 22:55:42,466][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000097696_100040704.pth... [2023-10-11 22:55:42,491][71431] Removing ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000096064_98369536.pth [2023-10-11 22:55:42,494][71431] Saving ./train_atari/atari_gopher_APPO/checkpoint_p1/checkpoint_000097696_100040704.pth... [2023-10-11 22:55:42,525][71431] Stopping LearnerWorker_p1... [2023-10-11 22:55:42,525][71431] Loop learner_proc1_evt_loop terminating... [2023-10-11 22:55:42,526][70582] Component LearnerWorker_p1 stopped! [2023-10-11 22:55:42,527][70582] Waiting for process learner_proc0 to stop... [2023-10-11 22:55:42,527][70582] Waiting for process learner_proc1 to stop... [2023-10-11 22:55:43,078][70582] Waiting for process inference_proc0-0 to join... [2023-10-11 22:55:43,079][70582] Waiting for process inference_proc1-0 to join... [2023-10-11 22:55:43,080][70582] Waiting for process rollout_proc0 to join... [2023-10-11 22:55:43,081][70582] Waiting for process rollout_proc1 to join... [2023-10-11 22:55:43,082][70582] Waiting for process rollout_proc2 to join... [2023-10-11 22:55:43,083][70582] Waiting for process rollout_proc3 to join... [2023-10-11 22:55:43,083][70582] Waiting for process rollout_proc4 to join... [2023-10-11 22:55:43,084][70582] Waiting for process rollout_proc5 to join... [2023-10-11 22:55:43,084][70582] Waiting for process rollout_proc6 to join... [2023-10-11 22:55:43,085][70582] Waiting for process rollout_proc7 to join... [2023-10-11 22:55:43,086][70582] Waiting for process rollout_proc8 to join... [2023-10-11 22:55:43,086][70582] Waiting for process rollout_proc9 to join... [2023-10-11 22:55:43,087][70582] Waiting for process rollout_proc10 to join... [2023-10-11 22:55:43,088][70582] Waiting for process rollout_proc11 to join... [2023-10-11 22:55:43,089][70582] Waiting for process rollout_proc12 to join... [2023-10-11 22:55:43,089][70582] Waiting for process rollout_proc13 to join... [2023-10-11 22:55:43,090][70582] Waiting for process rollout_proc14 to join... [2023-10-11 22:55:43,091][70582] Waiting for process rollout_proc15 to join... [2023-10-11 22:55:43,091][70582] Batcher 0 profile tree view: batching: 170.7833, releasing_batches: 0.0896 [2023-10-11 22:55:43,092][70582] Batcher 1 profile tree view: batching: 170.4569, releasing_batches: 0.0909 [2023-10-11 22:55:43,092][70582] InferenceWorker_p0-w0 profile tree view: wait_policy: 0.0000 wait_policy_total: 1819.2777 update_model: 197.0184 weight_update: 0.0007 one_step: 0.0019 handle_policy_step: 11096.7636 deserialize: 62.1916, stack: 190.9073, obs_to_device_normalize: 2481.5919, forward: 4987.6584, prepare_outputs: 2440.1324, send_messages: 453.7255 [2023-10-11 22:55:43,093][70582] InferenceWorker_p1-w0 profile tree view: wait_policy: 0.0000 wait_policy_total: 1789.3089 update_model: 197.2678 weight_update: 0.0007 one_step: 0.0023 handle_policy_step: 11119.8403 deserialize: 63.0442, stack: 189.6042, obs_to_device_normalize: 2472.0577, forward: 5024.8040, prepare_outputs: 2422.8659, send_messages: 462.7961 [2023-10-11 22:55:43,093][70582] Learner 0 profile tree view: misc: 0.0186, prepare_batch: 269.4414 train: 3648.2085 epoch_init: 0.1905, minibatch_init: 13.0049, losses_postprocess: 899.6102, kl_divergence: 31.5251, update: 393.8979, after_optimizer: 2123.6478 calculate_losses: 169.5765 losses_init: 0.3863, forward_head: 59.4199, bptt_initial: 1.4275, bptt: 1.8338, tail: 37.7372, advantages_returns: 11.1008, losses: 44.1626 [2023-10-11 22:55:43,093][70582] Learner 1 profile tree view: misc: 0.0181, prepare_batch: 269.0580 train: 3601.2766 epoch_init: 0.1821, minibatch_init: 13.0661, losses_postprocess: 886.0417, kl_divergence: 31.1369, update: 389.3193, after_optimizer: 2098.7961 calculate_losses: 165.9329 losses_init: 0.4472, forward_head: 55.5390, bptt_initial: 1.4066, bptt: 2.0655, tail: 37.9480, advantages_returns: 11.1336, losses: 43.8258 [2023-10-11 22:55:43,093][70582] RolloutWorker_w0 profile tree view: wait_for_trajectories: 1.2439, enqueue_policy_requests: 400.9926, process_policy_outputs: 186.8155, env_step: 6584.5343, finalize_trajectories: 3.9758, complete_rollouts: 3.0478 post_env_step: 368.9664 process_env_step: 82.3990 [2023-10-11 22:55:43,094][70582] RolloutWorker_w15 profile tree view: wait_for_trajectories: 1.2328, enqueue_policy_requests: 404.2139, process_policy_outputs: 191.9455, env_step: 6392.9990, finalize_trajectories: 3.5330, complete_rollouts: 2.9346 post_env_step: 375.3041 process_env_step: 84.6783 [2023-10-11 22:55:43,094][70582] Loop Runner_EvtLoop terminating... [2023-10-11 22:55:43,095][70582] Runner profile tree view: main_loop: 13782.4412 [2023-10-11 22:55:43,095][70582] Collected {0: 100106240, 1: 100040704}, FPS: 14521.9