diff --git "a/sf_log.txt" "b/sf_log.txt" --- "a/sf_log.txt" +++ "b/sf_log.txt" @@ -1,50 +1,50 @@ -[2024-07-24 17:47:35,504][03928] Saving configuration to /content/train_dir/default_experiment/config.json... -[2024-07-24 17:47:35,509][03928] Rollout worker 0 uses device cpu -[2024-07-24 17:47:35,510][03928] Rollout worker 1 uses device cpu -[2024-07-24 17:47:35,512][03928] Rollout worker 2 uses device cpu -[2024-07-24 17:47:35,514][03928] Rollout worker 3 uses device cpu -[2024-07-24 17:47:35,516][03928] Rollout worker 4 uses device cpu -[2024-07-24 17:47:35,517][03928] Rollout worker 5 uses device cpu -[2024-07-24 17:47:35,518][03928] Rollout worker 6 uses device cpu -[2024-07-24 17:47:35,519][03928] Rollout worker 7 uses device cpu -[2024-07-24 17:47:35,676][03928] Using GPUs [0] for process 0 (actually maps to GPUs [0]) -[2024-07-24 17:47:35,678][03928] InferenceWorker_p0-w0: min num requests: 2 -[2024-07-24 17:47:35,720][03928] Starting all processes... -[2024-07-24 17:47:35,721][03928] Starting process learner_proc0 -[2024-07-24 17:47:36,990][03928] Starting all processes... -[2024-07-24 17:47:37,002][03928] Starting process inference_proc0-0 -[2024-07-24 17:47:37,003][03928] Starting process rollout_proc0 -[2024-07-24 17:47:37,005][03928] Starting process rollout_proc1 -[2024-07-24 17:47:37,007][03928] Starting process rollout_proc2 -[2024-07-24 17:47:37,007][03928] Starting process rollout_proc3 -[2024-07-24 17:47:37,008][03928] Starting process rollout_proc4 -[2024-07-24 17:47:37,009][03928] Starting process rollout_proc5 -[2024-07-24 17:47:37,009][03928] Starting process rollout_proc6 -[2024-07-24 17:47:37,009][03928] Starting process rollout_proc7 -[2024-07-24 17:47:52,502][05149] Worker 0 uses CPU cores [0] -[2024-07-24 17:47:52,703][05135] Using GPUs [0] for process 0 (actually maps to GPUs [0]) -[2024-07-24 17:47:52,706][05135] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 -[2024-07-24 17:47:52,719][05153] Worker 4 uses CPU cores [0] -[2024-07-24 17:47:52,737][05156] Worker 5 uses CPU cores [1] -[2024-07-24 17:47:52,777][05135] Num visible devices: 1 -[2024-07-24 17:47:52,780][05152] Worker 3 uses CPU cores [1] -[2024-07-24 17:47:52,798][05151] Worker 2 uses CPU cores [0] -[2024-07-24 17:47:52,808][05135] Starting seed is not provided -[2024-07-24 17:47:52,808][05135] Using GPUs [0] for process 0 (actually maps to GPUs [0]) -[2024-07-24 17:47:52,808][05135] Initializing actor-critic model on device cuda:0 -[2024-07-24 17:47:52,809][05135] RunningMeanStd input shape: (3, 72, 128) -[2024-07-24 17:47:52,812][05135] RunningMeanStd input shape: (1,) -[2024-07-24 17:47:52,819][05150] Worker 1 uses CPU cores [1] -[2024-07-24 17:47:52,823][05155] Worker 7 uses CPU cores [1] -[2024-07-24 17:47:52,829][05148] Using GPUs [0] for process 0 (actually maps to GPUs [0]) -[2024-07-24 17:47:52,829][05148] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 -[2024-07-24 17:47:52,852][05148] Num visible devices: 1 -[2024-07-24 17:47:52,858][05135] ConvEncoder: input_channels=3 -[2024-07-24 17:47:52,879][05154] Worker 6 uses CPU cores [0] -[2024-07-24 17:47:53,079][05135] Conv encoder output size: 512 -[2024-07-24 17:47:53,079][05135] Policy head output size: 512 -[2024-07-24 17:47:53,131][05135] Created Actor Critic model with architecture: -[2024-07-24 17:47:53,132][05135] ActorCriticSharedWeights( +[2024-07-24 19:00:32,744][02885] Saving configuration to /content/train_dir/default_experiment/config.json... +[2024-07-24 19:00:32,749][02885] Rollout worker 0 uses device cpu +[2024-07-24 19:00:32,750][02885] Rollout worker 1 uses device cpu +[2024-07-24 19:00:32,752][02885] Rollout worker 2 uses device cpu +[2024-07-24 19:00:32,753][02885] Rollout worker 3 uses device cpu +[2024-07-24 19:00:32,755][02885] Rollout worker 4 uses device cpu +[2024-07-24 19:00:32,757][02885] Rollout worker 5 uses device cpu +[2024-07-24 19:00:32,759][02885] Rollout worker 6 uses device cpu +[2024-07-24 19:00:32,761][02885] Rollout worker 7 uses device cpu +[2024-07-24 19:00:32,930][02885] Using GPUs [0] for process 0 (actually maps to GPUs [0]) +[2024-07-24 19:00:32,932][02885] InferenceWorker_p0-w0: min num requests: 2 +[2024-07-24 19:00:32,971][02885] Starting all processes... +[2024-07-24 19:00:32,973][02885] Starting process learner_proc0 +[2024-07-24 19:00:34,308][02885] Starting all processes... +[2024-07-24 19:00:34,317][02885] Starting process inference_proc0-0 +[2024-07-24 19:00:34,317][02885] Starting process rollout_proc0 +[2024-07-24 19:00:34,319][02885] Starting process rollout_proc1 +[2024-07-24 19:00:34,319][02885] Starting process rollout_proc2 +[2024-07-24 19:00:34,319][02885] Starting process rollout_proc3 +[2024-07-24 19:00:34,319][02885] Starting process rollout_proc4 +[2024-07-24 19:00:34,319][02885] Starting process rollout_proc5 +[2024-07-24 19:00:34,319][02885] Starting process rollout_proc6 +[2024-07-24 19:00:34,319][02885] Starting process rollout_proc7 +[2024-07-24 19:00:49,781][05263] Using GPUs [0] for process 0 (actually maps to GPUs [0]) +[2024-07-24 19:00:49,781][05263] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 +[2024-07-24 19:00:49,903][05263] Num visible devices: 1 +[2024-07-24 19:00:49,947][05263] Starting seed is not provided +[2024-07-24 19:00:49,948][05263] Using GPUs [0] for process 0 (actually maps to GPUs [0]) +[2024-07-24 19:00:49,949][05263] Initializing actor-critic model on device cuda:0 +[2024-07-24 19:00:49,950][05263] RunningMeanStd input shape: (3, 72, 128) +[2024-07-24 19:00:49,953][05263] RunningMeanStd input shape: (1,) +[2024-07-24 19:00:50,018][05279] Worker 1 uses CPU cores [1] +[2024-07-24 19:00:50,039][05281] Worker 3 uses CPU cores [1] +[2024-07-24 19:00:50,036][05263] ConvEncoder: input_channels=3 +[2024-07-24 19:00:50,180][05276] Using GPUs [0] for process 0 (actually maps to GPUs [0]) +[2024-07-24 19:00:50,181][05276] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 +[2024-07-24 19:00:50,222][05284] Worker 6 uses CPU cores [0] +[2024-07-24 19:00:50,244][05276] Num visible devices: 1 +[2024-07-24 19:00:50,260][05277] Worker 0 uses CPU cores [0] +[2024-07-24 19:00:50,290][05278] Worker 2 uses CPU cores [0] +[2024-07-24 19:00:50,293][05283] Worker 7 uses CPU cores [1] +[2024-07-24 19:00:50,334][05280] Worker 4 uses CPU cores [0] +[2024-07-24 19:00:50,337][05282] Worker 5 uses CPU cores [1] +[2024-07-24 19:00:50,414][05263] Conv encoder output size: 512 +[2024-07-24 19:00:50,414][05263] Policy head output size: 512 +[2024-07-24 19:00:50,468][05263] Created Actor Critic model with architecture: +[2024-07-24 19:00:50,468][05263] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( @@ -85,905 +85,1039 @@ (distribution_linear): Linear(in_features=512, out_features=5, bias=True) ) ) -[2024-07-24 17:47:53,393][05135] Using optimizer -[2024-07-24 17:47:54,125][05135] No checkpoints found -[2024-07-24 17:47:54,125][05135] Did not load from checkpoint, starting from scratch! -[2024-07-24 17:47:54,125][05135] Initialized policy 0 weights for model version 0 -[2024-07-24 17:47:54,128][05135] LearnerWorker_p0 finished initialization! -[2024-07-24 17:47:54,129][05135] Using GPUs [0] for process 0 (actually maps to GPUs [0]) -[2024-07-24 17:47:54,268][05148] RunningMeanStd input shape: (3, 72, 128) -[2024-07-24 17:47:54,269][05148] RunningMeanStd input shape: (1,) -[2024-07-24 17:47:54,282][05148] ConvEncoder: input_channels=3 -[2024-07-24 17:47:54,385][05148] Conv encoder output size: 512 -[2024-07-24 17:47:54,385][05148] Policy head output size: 512 -[2024-07-24 17:47:54,439][03928] Inference worker 0-0 is ready! -[2024-07-24 17:47:54,441][03928] All inference workers are ready! Signal rollout workers to start! -[2024-07-24 17:47:54,722][05155] Doom resolution: 160x120, resize resolution: (128, 72) -[2024-07-24 17:47:54,728][05152] Doom resolution: 160x120, resize resolution: (128, 72) -[2024-07-24 17:47:54,762][05149] Doom resolution: 160x120, resize resolution: (128, 72) -[2024-07-24 17:47:54,764][05153] Doom resolution: 160x120, resize resolution: (128, 72) -[2024-07-24 17:47:54,772][05151] Doom resolution: 160x120, resize resolution: (128, 72) -[2024-07-24 17:47:54,774][05156] Doom resolution: 160x120, resize resolution: (128, 72) -[2024-07-24 17:47:54,767][05154] Doom resolution: 160x120, resize resolution: (128, 72) -[2024-07-24 17:47:54,861][05150] Doom resolution: 160x120, resize resolution: (128, 72) -[2024-07-24 17:47:55,668][03928] Heartbeat connected on Batcher_0 -[2024-07-24 17:47:55,671][03928] Heartbeat connected on LearnerWorker_p0 -[2024-07-24 17:47:55,714][03928] Heartbeat connected on InferenceWorker_p0-w0 -[2024-07-24 17:47:56,388][03928] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) -[2024-07-24 17:47:56,608][05155] Decorrelating experience for 0 frames... -[2024-07-24 17:47:56,608][05152] Decorrelating experience for 0 frames... -[2024-07-24 17:47:56,895][05154] Decorrelating experience for 0 frames... -[2024-07-24 17:47:56,907][05153] Decorrelating experience for 0 frames... -[2024-07-24 17:47:56,911][05149] Decorrelating experience for 0 frames... -[2024-07-24 17:47:56,917][05151] Decorrelating experience for 0 frames... -[2024-07-24 17:47:59,290][05155] Decorrelating experience for 32 frames... -[2024-07-24 17:47:59,814][05156] Decorrelating experience for 0 frames... -[2024-07-24 17:47:59,825][05154] Decorrelating experience for 32 frames... -[2024-07-24 17:47:59,819][05150] Decorrelating experience for 0 frames... -[2024-07-24 17:47:59,856][05152] Decorrelating experience for 32 frames... -[2024-07-24 17:47:59,867][05149] Decorrelating experience for 32 frames... -[2024-07-24 17:47:59,905][05151] Decorrelating experience for 32 frames... -[2024-07-24 17:48:00,209][05153] Decorrelating experience for 32 frames... -[2024-07-24 17:48:01,388][03928] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) -[2024-07-24 17:48:02,102][05151] Decorrelating experience for 64 frames... -[2024-07-24 17:48:02,329][05156] Decorrelating experience for 32 frames... -[2024-07-24 17:48:02,354][05150] Decorrelating experience for 32 frames... -[2024-07-24 17:48:02,506][05155] Decorrelating experience for 64 frames... -[2024-07-24 17:48:03,928][05155] Decorrelating experience for 96 frames... -[2024-07-24 17:48:04,097][05150] Decorrelating experience for 64 frames... -[2024-07-24 17:48:04,103][03928] Heartbeat connected on RolloutWorker_w7 -[2024-07-24 17:48:04,570][05154] Decorrelating experience for 64 frames... -[2024-07-24 17:48:04,595][05151] Decorrelating experience for 96 frames... -[2024-07-24 17:48:05,011][05149] Decorrelating experience for 64 frames... -[2024-07-24 17:48:05,180][03928] Heartbeat connected on RolloutWorker_w2 -[2024-07-24 17:48:06,098][05150] Decorrelating experience for 96 frames... -[2024-07-24 17:48:06,102][05156] Decorrelating experience for 64 frames... -[2024-07-24 17:48:06,388][03928] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) -[2024-07-24 17:48:06,392][03928] Avg episode reward: [(0, '1.280')] -[2024-07-24 17:48:06,648][03928] Heartbeat connected on RolloutWorker_w1 -[2024-07-24 17:48:06,766][05152] Decorrelating experience for 64 frames... -[2024-07-24 17:48:08,330][05156] Decorrelating experience for 96 frames... -[2024-07-24 17:48:08,791][03928] Heartbeat connected on RolloutWorker_w5 -[2024-07-24 17:48:08,879][05154] Decorrelating experience for 96 frames... -[2024-07-24 17:48:08,885][05153] Decorrelating experience for 64 frames... -[2024-07-24 17:48:09,400][03928] Heartbeat connected on RolloutWorker_w6 -[2024-07-24 17:48:11,388][03928] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 101.9. Samples: 1528. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) -[2024-07-24 17:48:11,392][03928] Avg episode reward: [(0, '3.311')] -[2024-07-24 17:48:12,049][05149] Decorrelating experience for 96 frames... -[2024-07-24 17:48:12,260][05135] Signal inference workers to stop experience collection... -[2024-07-24 17:48:12,282][05148] InferenceWorker_p0-w0: stopping experience collection -[2024-07-24 17:48:12,285][05153] Decorrelating experience for 96 frames... -[2024-07-24 17:48:12,381][05152] Decorrelating experience for 96 frames... -[2024-07-24 17:48:12,399][03928] Heartbeat connected on RolloutWorker_w0 -[2024-07-24 17:48:12,471][03928] Heartbeat connected on RolloutWorker_w4 -[2024-07-24 17:48:12,546][03928] Heartbeat connected on RolloutWorker_w3 -[2024-07-24 17:48:13,855][05135] Signal inference workers to resume experience collection... -[2024-07-24 17:48:13,862][05148] InferenceWorker_p0-w0: resuming experience collection -[2024-07-24 17:48:16,388][03928] Fps is (10 sec: 1228.8, 60 sec: 614.4, 300 sec: 614.4). Total num frames: 12288. Throughput: 0: 172.4. Samples: 3448. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) -[2024-07-24 17:48:16,392][03928] Avg episode reward: [(0, '3.234')] -[2024-07-24 17:48:21,388][03928] Fps is (10 sec: 2867.2, 60 sec: 1146.9, 300 sec: 1146.9). Total num frames: 28672. Throughput: 0: 210.6. Samples: 5264. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2024-07-24 17:48:21,390][03928] Avg episode reward: [(0, '3.563')] -[2024-07-24 17:48:24,277][05148] Updated weights for policy 0, policy_version 10 (0.0309) -[2024-07-24 17:48:26,388][03928] Fps is (10 sec: 3686.4, 60 sec: 1638.4, 300 sec: 1638.4). Total num frames: 49152. Throughput: 0: 365.7. Samples: 10972. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 17:48:26,394][03928] Avg episode reward: [(0, '4.026')] -[2024-07-24 17:48:31,388][03928] Fps is (10 sec: 3686.4, 60 sec: 1872.5, 300 sec: 1872.5). Total num frames: 65536. Throughput: 0: 491.4. Samples: 17200. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2024-07-24 17:48:31,390][03928] Avg episode reward: [(0, '4.367')] -[2024-07-24 17:48:36,394][03928] Fps is (10 sec: 2865.5, 60 sec: 1945.3, 300 sec: 1945.3). Total num frames: 77824. Throughput: 0: 476.4. Samples: 19060. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 17:48:36,396][03928] Avg episode reward: [(0, '4.399')] -[2024-07-24 17:48:36,500][05148] Updated weights for policy 0, policy_version 20 (0.0046) -[2024-07-24 17:48:41,388][03928] Fps is (10 sec: 3276.8, 60 sec: 2184.5, 300 sec: 2184.5). Total num frames: 98304. Throughput: 0: 522.8. Samples: 23528. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 17:48:41,390][03928] Avg episode reward: [(0, '4.341')] -[2024-07-24 17:48:46,388][03928] Fps is (10 sec: 4098.5, 60 sec: 2375.7, 300 sec: 2375.7). Total num frames: 118784. Throughput: 0: 668.5. Samples: 30084. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2024-07-24 17:48:46,390][03928] Avg episode reward: [(0, '4.325')] -[2024-07-24 17:48:46,411][05135] Saving new best policy, reward=4.325! -[2024-07-24 17:48:46,413][05148] Updated weights for policy 0, policy_version 30 (0.0034) -[2024-07-24 17:48:51,388][03928] Fps is (10 sec: 3686.4, 60 sec: 2457.6, 300 sec: 2457.6). Total num frames: 135168. Throughput: 0: 729.8. Samples: 32840. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) -[2024-07-24 17:48:51,391][03928] Avg episode reward: [(0, '4.491')] -[2024-07-24 17:48:51,404][05135] Saving new best policy, reward=4.491! -[2024-07-24 17:48:56,388][03928] Fps is (10 sec: 3276.8, 60 sec: 2525.9, 300 sec: 2525.9). Total num frames: 151552. Throughput: 0: 784.5. Samples: 36830. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-07-24 17:48:56,393][03928] Avg episode reward: [(0, '4.468')] -[2024-07-24 17:48:58,944][05148] Updated weights for policy 0, policy_version 40 (0.0043) -[2024-07-24 17:49:01,388][03928] Fps is (10 sec: 3686.4, 60 sec: 2867.2, 300 sec: 2646.6). Total num frames: 172032. Throughput: 0: 882.2. Samples: 43146. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2024-07-24 17:49:01,394][03928] Avg episode reward: [(0, '4.428')] -[2024-07-24 17:49:06,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3208.5, 300 sec: 2750.2). Total num frames: 192512. Throughput: 0: 912.7. Samples: 46336. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2024-07-24 17:49:06,393][03928] Avg episode reward: [(0, '4.409')] -[2024-07-24 17:49:10,932][05148] Updated weights for policy 0, policy_version 50 (0.0017) -[2024-07-24 17:49:11,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 2730.7). Total num frames: 204800. Throughput: 0: 884.5. Samples: 50774. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-07-24 17:49:11,395][03928] Avg episode reward: [(0, '4.429')] -[2024-07-24 17:49:16,389][03928] Fps is (10 sec: 3276.2, 60 sec: 3549.8, 300 sec: 2815.9). Total num frames: 225280. Throughput: 0: 862.8. Samples: 56028. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 17:49:16,392][03928] Avg episode reward: [(0, '4.423')] -[2024-07-24 17:49:20,928][05148] Updated weights for policy 0, policy_version 60 (0.0029) -[2024-07-24 17:49:21,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3618.1, 300 sec: 2891.3). Total num frames: 245760. Throughput: 0: 894.8. Samples: 59320. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 17:49:21,390][03928] Avg episode reward: [(0, '4.223')] -[2024-07-24 17:49:26,389][03928] Fps is (10 sec: 3277.0, 60 sec: 3481.5, 300 sec: 2867.2). Total num frames: 258048. Throughput: 0: 918.1. Samples: 64842. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 17:49:26,394][03928] Avg episode reward: [(0, '4.287')] -[2024-07-24 17:49:31,388][03928] Fps is (10 sec: 2867.2, 60 sec: 3481.6, 300 sec: 2888.8). Total num frames: 274432. Throughput: 0: 868.9. Samples: 69184. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 17:49:31,393][03928] Avg episode reward: [(0, '4.392')] -[2024-07-24 17:49:31,401][05135] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000067_274432.pth... -[2024-07-24 17:49:33,441][05148] Updated weights for policy 0, policy_version 70 (0.0034) -[2024-07-24 17:49:36,388][03928] Fps is (10 sec: 4096.4, 60 sec: 3686.8, 300 sec: 2990.1). Total num frames: 299008. Throughput: 0: 876.0. Samples: 72260. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 17:49:36,393][03928] Avg episode reward: [(0, '4.597')] -[2024-07-24 17:49:36,398][05135] Saving new best policy, reward=4.597! -[2024-07-24 17:49:41,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3618.1, 300 sec: 3003.7). Total num frames: 315392. Throughput: 0: 928.3. Samples: 78604. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) -[2024-07-24 17:49:41,390][03928] Avg episode reward: [(0, '4.577')] -[2024-07-24 17:49:45,697][05148] Updated weights for policy 0, policy_version 80 (0.0059) -[2024-07-24 17:49:46,388][03928] Fps is (10 sec: 2867.2, 60 sec: 3481.6, 300 sec: 2978.9). Total num frames: 327680. Throughput: 0: 875.9. Samples: 82562. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) -[2024-07-24 17:49:46,393][03928] Avg episode reward: [(0, '4.488')] -[2024-07-24 17:49:51,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3027.5). Total num frames: 348160. Throughput: 0: 865.2. Samples: 85272. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 17:49:51,389][03928] Avg episode reward: [(0, '4.380')] -[2024-07-24 17:49:55,556][05148] Updated weights for policy 0, policy_version 90 (0.0020) -[2024-07-24 17:49:56,390][03928] Fps is (10 sec: 4095.1, 60 sec: 3618.0, 300 sec: 3071.9). Total num frames: 368640. Throughput: 0: 909.7. Samples: 91712. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 17:49:56,395][03928] Avg episode reward: [(0, '4.437')] -[2024-07-24 17:50:01,388][03928] Fps is (10 sec: 3686.2, 60 sec: 3549.8, 300 sec: 3080.2). Total num frames: 385024. Throughput: 0: 901.0. Samples: 96574. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2024-07-24 17:50:01,394][03928] Avg episode reward: [(0, '4.362')] -[2024-07-24 17:50:06,388][03928] Fps is (10 sec: 3277.5, 60 sec: 3481.6, 300 sec: 3087.8). Total num frames: 401408. Throughput: 0: 872.7. Samples: 98592. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2024-07-24 17:50:06,390][03928] Avg episode reward: [(0, '4.471')] -[2024-07-24 17:50:07,868][05148] Updated weights for policy 0, policy_version 100 (0.0030) -[2024-07-24 17:50:11,388][03928] Fps is (10 sec: 3686.6, 60 sec: 3618.1, 300 sec: 3125.1). Total num frames: 421888. Throughput: 0: 887.2. Samples: 104764. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 17:50:11,390][03928] Avg episode reward: [(0, '4.783')] -[2024-07-24 17:50:11,402][05135] Saving new best policy, reward=4.783! -[2024-07-24 17:50:16,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3550.0, 300 sec: 3130.5). Total num frames: 438272. Throughput: 0: 918.9. Samples: 110536. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 17:50:16,390][03928] Avg episode reward: [(0, '4.747')] -[2024-07-24 17:50:19,489][05148] Updated weights for policy 0, policy_version 110 (0.0022) -[2024-07-24 17:50:21,389][03928] Fps is (10 sec: 3276.5, 60 sec: 3481.5, 300 sec: 3135.5). Total num frames: 454656. Throughput: 0: 893.6. Samples: 112474. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 17:50:21,391][03928] Avg episode reward: [(0, '4.800')] -[2024-07-24 17:50:21,402][05135] Saving new best policy, reward=4.800! -[2024-07-24 17:50:26,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3618.2, 300 sec: 3167.6). Total num frames: 475136. Throughput: 0: 866.0. Samples: 117576. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-07-24 17:50:26,394][03928] Avg episode reward: [(0, '4.561')] -[2024-07-24 17:50:29,833][05148] Updated weights for policy 0, policy_version 120 (0.0026) -[2024-07-24 17:50:31,388][03928] Fps is (10 sec: 4096.4, 60 sec: 3686.4, 300 sec: 3197.5). Total num frames: 495616. Throughput: 0: 920.4. Samples: 123978. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2024-07-24 17:50:31,393][03928] Avg episode reward: [(0, '4.471')] -[2024-07-24 17:50:36,392][03928] Fps is (10 sec: 3275.5, 60 sec: 3481.4, 300 sec: 3174.3). Total num frames: 507904. Throughput: 0: 914.1. Samples: 126410. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 17:50:36,400][03928] Avg episode reward: [(0, '4.468')] -[2024-07-24 17:50:41,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3202.3). Total num frames: 528384. Throughput: 0: 866.4. Samples: 130698. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2024-07-24 17:50:41,394][03928] Avg episode reward: [(0, '4.585')] -[2024-07-24 17:50:42,178][05148] Updated weights for policy 0, policy_version 130 (0.0022) -[2024-07-24 17:50:46,388][03928] Fps is (10 sec: 4097.7, 60 sec: 3686.4, 300 sec: 3228.6). Total num frames: 548864. Throughput: 0: 902.2. Samples: 137172. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 17:50:46,390][03928] Avg episode reward: [(0, '4.674')] -[2024-07-24 17:50:51,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3230.0). Total num frames: 565248. Throughput: 0: 928.3. Samples: 140364. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 17:50:51,390][03928] Avg episode reward: [(0, '4.728')] -[2024-07-24 17:50:53,745][05148] Updated weights for policy 0, policy_version 140 (0.0019) -[2024-07-24 17:50:56,388][03928] Fps is (10 sec: 2867.2, 60 sec: 3481.7, 300 sec: 3208.5). Total num frames: 577536. Throughput: 0: 883.2. Samples: 144510. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 17:50:56,391][03928] Avg episode reward: [(0, '4.697')] -[2024-07-24 17:51:01,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3618.2, 300 sec: 3254.7). Total num frames: 602112. Throughput: 0: 885.9. Samples: 150400. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2024-07-24 17:51:01,390][03928] Avg episode reward: [(0, '4.561')] -[2024-07-24 17:51:03,973][05148] Updated weights for policy 0, policy_version 150 (0.0021) -[2024-07-24 17:51:06,388][03928] Fps is (10 sec: 4505.6, 60 sec: 3686.4, 300 sec: 3276.8). Total num frames: 622592. Throughput: 0: 915.7. Samples: 153680. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 17:51:06,393][03928] Avg episode reward: [(0, '4.519')] -[2024-07-24 17:51:11,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3255.8). Total num frames: 634880. Throughput: 0: 916.7. Samples: 158826. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 17:51:11,390][03928] Avg episode reward: [(0, '4.657')] -[2024-07-24 17:51:16,289][05148] Updated weights for policy 0, policy_version 160 (0.0023) -[2024-07-24 17:51:16,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3276.8). Total num frames: 655360. Throughput: 0: 877.6. Samples: 163470. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 17:51:16,389][03928] Avg episode reward: [(0, '4.728')] -[2024-07-24 17:51:21,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3686.5, 300 sec: 3296.8). Total num frames: 675840. Throughput: 0: 894.7. Samples: 166666. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2024-07-24 17:51:21,390][03928] Avg episode reward: [(0, '4.468')] -[2024-07-24 17:51:26,391][03928] Fps is (10 sec: 3685.3, 60 sec: 3617.9, 300 sec: 3296.3). Total num frames: 692224. Throughput: 0: 935.9. Samples: 172818. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 17:51:26,394][03928] Avg episode reward: [(0, '4.711')] -[2024-07-24 17:51:27,256][05148] Updated weights for policy 0, policy_version 170 (0.0030) -[2024-07-24 17:51:31,388][03928] Fps is (10 sec: 2867.2, 60 sec: 3481.6, 300 sec: 3276.8). Total num frames: 704512. Throughput: 0: 881.2. Samples: 176826. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 17:51:31,390][03928] Avg episode reward: [(0, '4.892')] -[2024-07-24 17:51:31,477][05135] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000173_708608.pth... -[2024-07-24 17:51:31,599][05135] Saving new best policy, reward=4.892! -[2024-07-24 17:51:36,388][03928] Fps is (10 sec: 3687.5, 60 sec: 3686.7, 300 sec: 3314.0). Total num frames: 729088. Throughput: 0: 877.3. Samples: 179844. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 17:51:36,390][03928] Avg episode reward: [(0, '4.768')] -[2024-07-24 17:51:38,085][05148] Updated weights for policy 0, policy_version 180 (0.0028) -[2024-07-24 17:51:41,388][03928] Fps is (10 sec: 4505.6, 60 sec: 3686.4, 300 sec: 3331.4). Total num frames: 749568. Throughput: 0: 929.3. Samples: 186330. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 17:51:41,390][03928] Avg episode reward: [(0, '4.802')] -[2024-07-24 17:51:46,391][03928] Fps is (10 sec: 3275.6, 60 sec: 3549.7, 300 sec: 3312.4). Total num frames: 761856. Throughput: 0: 899.0. Samples: 190858. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2024-07-24 17:51:46,394][03928] Avg episode reward: [(0, '4.912')] -[2024-07-24 17:51:46,399][05135] Saving new best policy, reward=4.912! -[2024-07-24 17:51:50,581][05148] Updated weights for policy 0, policy_version 190 (0.0026) -[2024-07-24 17:51:51,388][03928] Fps is (10 sec: 2867.1, 60 sec: 3549.9, 300 sec: 3311.7). Total num frames: 778240. Throughput: 0: 871.0. Samples: 192876. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2024-07-24 17:51:51,394][03928] Avg episode reward: [(0, '5.131')] -[2024-07-24 17:51:51,407][05135] Saving new best policy, reward=5.131! -[2024-07-24 17:51:56,388][03928] Fps is (10 sec: 4097.5, 60 sec: 3754.7, 300 sec: 3345.1). Total num frames: 802816. Throughput: 0: 897.2. Samples: 199200. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2024-07-24 17:51:56,395][03928] Avg episode reward: [(0, '5.337')] -[2024-07-24 17:51:56,399][05135] Saving new best policy, reward=5.337! -[2024-07-24 17:52:01,224][05148] Updated weights for policy 0, policy_version 200 (0.0020) -[2024-07-24 17:52:01,388][03928] Fps is (10 sec: 4096.1, 60 sec: 3618.1, 300 sec: 3343.7). Total num frames: 819200. Throughput: 0: 921.6. Samples: 204940. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-07-24 17:52:01,391][03928] Avg episode reward: [(0, '5.275')] -[2024-07-24 17:52:06,388][03928] Fps is (10 sec: 2867.2, 60 sec: 3481.6, 300 sec: 3326.0). Total num frames: 831488. Throughput: 0: 892.3. Samples: 206818. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 17:52:06,394][03928] Avg episode reward: [(0, '5.096')] -[2024-07-24 17:52:11,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3341.1). Total num frames: 851968. Throughput: 0: 882.4. Samples: 212524. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 17:52:11,390][03928] Avg episode reward: [(0, '5.151')] -[2024-07-24 17:52:12,252][05148] Updated weights for policy 0, policy_version 210 (0.0053) -[2024-07-24 17:52:16,391][03928] Fps is (10 sec: 4504.0, 60 sec: 3686.2, 300 sec: 3371.3). Total num frames: 876544. Throughput: 0: 934.8. Samples: 218894. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 17:52:16,393][03928] Avg episode reward: [(0, '5.156')] -[2024-07-24 17:52:21,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3354.1). Total num frames: 888832. Throughput: 0: 913.6. Samples: 220954. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2024-07-24 17:52:21,389][03928] Avg episode reward: [(0, '5.318')] -[2024-07-24 17:52:24,641][05148] Updated weights for policy 0, policy_version 220 (0.0027) -[2024-07-24 17:52:26,388][03928] Fps is (10 sec: 2868.2, 60 sec: 3550.0, 300 sec: 3352.7). Total num frames: 905216. Throughput: 0: 875.1. Samples: 225710. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 17:52:26,393][03928] Avg episode reward: [(0, '5.438')] -[2024-07-24 17:52:26,398][05135] Saving new best policy, reward=5.438! -[2024-07-24 17:52:31,388][03928] Fps is (10 sec: 2867.2, 60 sec: 3549.9, 300 sec: 3336.4). Total num frames: 917504. Throughput: 0: 867.0. Samples: 229870. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 17:52:31,393][03928] Avg episode reward: [(0, '5.211')] -[2024-07-24 17:52:36,388][03928] Fps is (10 sec: 2867.2, 60 sec: 3413.3, 300 sec: 3335.3). Total num frames: 933888. Throughput: 0: 872.3. Samples: 232130. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 17:52:36,392][03928] Avg episode reward: [(0, '5.211')] -[2024-07-24 17:52:39,228][05148] Updated weights for policy 0, policy_version 230 (0.0039) -[2024-07-24 17:52:41,388][03928] Fps is (10 sec: 2867.2, 60 sec: 3276.8, 300 sec: 3319.9). Total num frames: 946176. Throughput: 0: 821.2. Samples: 236156. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 17:52:41,396][03928] Avg episode reward: [(0, '5.159')] -[2024-07-24 17:52:46,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3413.5, 300 sec: 3333.3). Total num frames: 966656. Throughput: 0: 827.3. Samples: 242168. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2024-07-24 17:52:46,394][03928] Avg episode reward: [(0, '5.756')] -[2024-07-24 17:52:46,416][05135] Saving new best policy, reward=5.756! -[2024-07-24 17:52:49,369][05148] Updated weights for policy 0, policy_version 240 (0.0026) -[2024-07-24 17:52:51,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3481.6, 300 sec: 3346.2). Total num frames: 987136. Throughput: 0: 854.5. Samples: 245272. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) -[2024-07-24 17:52:51,396][03928] Avg episode reward: [(0, '5.792')] -[2024-07-24 17:52:51,415][05135] Saving new best policy, reward=5.792! -[2024-07-24 17:52:56,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3276.8, 300 sec: 3387.9). Total num frames: 999424. Throughput: 0: 829.7. Samples: 249862. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) -[2024-07-24 17:52:56,391][03928] Avg episode reward: [(0, '5.909')] -[2024-07-24 17:52:56,396][05135] Saving new best policy, reward=5.909! -[2024-07-24 17:53:01,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3345.1, 300 sec: 3457.3). Total num frames: 1019904. Throughput: 0: 807.6. Samples: 255232. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-07-24 17:53:01,392][03928] Avg episode reward: [(0, '6.161')] -[2024-07-24 17:53:01,402][05135] Saving new best policy, reward=6.161! -[2024-07-24 17:53:01,642][05148] Updated weights for policy 0, policy_version 250 (0.0034) -[2024-07-24 17:53:06,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3481.6, 300 sec: 3526.7). Total num frames: 1040384. Throughput: 0: 832.3. Samples: 258408. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-07-24 17:53:06,390][03928] Avg episode reward: [(0, '6.466')] -[2024-07-24 17:53:06,425][05135] Saving new best policy, reward=6.466! -[2024-07-24 17:53:11,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3413.3, 300 sec: 3540.6). Total num frames: 1056768. Throughput: 0: 849.3. Samples: 263928. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-07-24 17:53:11,394][03928] Avg episode reward: [(0, '6.500')] -[2024-07-24 17:53:11,403][05135] Saving new best policy, reward=6.500! -[2024-07-24 17:53:13,856][05148] Updated weights for policy 0, policy_version 260 (0.0036) -[2024-07-24 17:53:16,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3277.0, 300 sec: 3540.6). Total num frames: 1073152. Throughput: 0: 851.6. Samples: 268190. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-07-24 17:53:16,390][03928] Avg episode reward: [(0, '6.813')] -[2024-07-24 17:53:16,397][05135] Saving new best policy, reward=6.813! -[2024-07-24 17:53:21,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3413.3, 300 sec: 3540.6). Total num frames: 1093632. Throughput: 0: 869.2. Samples: 271246. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2024-07-24 17:53:21,394][03928] Avg episode reward: [(0, '7.181')] -[2024-07-24 17:53:21,407][05135] Saving new best policy, reward=7.181! -[2024-07-24 17:53:23,601][05148] Updated weights for policy 0, policy_version 270 (0.0017) -[2024-07-24 17:53:26,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3481.6, 300 sec: 3554.5). Total num frames: 1114112. Throughput: 0: 923.5. Samples: 277714. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) -[2024-07-24 17:53:26,393][03928] Avg episode reward: [(0, '7.707')] -[2024-07-24 17:53:26,396][05135] Saving new best policy, reward=7.707! -[2024-07-24 17:53:31,389][03928] Fps is (10 sec: 3276.4, 60 sec: 3481.5, 300 sec: 3554.6). Total num frames: 1126400. Throughput: 0: 877.7. Samples: 281666. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) -[2024-07-24 17:53:31,393][03928] Avg episode reward: [(0, '8.232')] -[2024-07-24 17:53:31,406][05135] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000275_1126400.pth... -[2024-07-24 17:53:31,613][05135] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000067_274432.pth -[2024-07-24 17:53:31,628][05135] Saving new best policy, reward=8.232! -[2024-07-24 17:53:36,048][05148] Updated weights for policy 0, policy_version 280 (0.0025) -[2024-07-24 17:53:36,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3554.5). Total num frames: 1146880. Throughput: 0: 867.8. Samples: 284322. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) -[2024-07-24 17:53:36,390][03928] Avg episode reward: [(0, '8.515')] -[2024-07-24 17:53:36,395][05135] Saving new best policy, reward=8.515! -[2024-07-24 17:53:41,388][03928] Fps is (10 sec: 4096.5, 60 sec: 3686.4, 300 sec: 3554.5). Total num frames: 1167360. Throughput: 0: 908.0. Samples: 290720. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 17:53:41,393][03928] Avg episode reward: [(0, '7.801')] -[2024-07-24 17:53:46,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3540.6). Total num frames: 1179648. Throughput: 0: 896.3. Samples: 295564. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 17:53:46,390][03928] Avg episode reward: [(0, '7.531')] -[2024-07-24 17:53:48,309][05148] Updated weights for policy 0, policy_version 290 (0.0025) -[2024-07-24 17:53:51,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3554.5). Total num frames: 1200128. Throughput: 0: 870.8. Samples: 297594. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 17:53:51,390][03928] Avg episode reward: [(0, '7.313')] -[2024-07-24 17:53:56,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3554.5). Total num frames: 1220608. Throughput: 0: 885.0. Samples: 303754. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-07-24 17:53:56,392][03928] Avg episode reward: [(0, '7.883')] -[2024-07-24 17:53:58,087][05148] Updated weights for policy 0, policy_version 300 (0.0020) -[2024-07-24 17:54:01,391][03928] Fps is (10 sec: 3685.3, 60 sec: 3618.0, 300 sec: 3540.6). Total num frames: 1236992. Throughput: 0: 926.2. Samples: 309870. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) -[2024-07-24 17:54:01,394][03928] Avg episode reward: [(0, '8.880')] -[2024-07-24 17:54:01,404][05135] Saving new best policy, reward=8.880! -[2024-07-24 17:54:06,388][03928] Fps is (10 sec: 2867.0, 60 sec: 3481.6, 300 sec: 3540.6). Total num frames: 1249280. Throughput: 0: 900.1. Samples: 311750. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2024-07-24 17:54:06,394][03928] Avg episode reward: [(0, '9.189')] -[2024-07-24 17:54:06,396][05135] Saving new best policy, reward=9.189! -[2024-07-24 17:54:10,527][05148] Updated weights for policy 0, policy_version 310 (0.0031) -[2024-07-24 17:54:11,388][03928] Fps is (10 sec: 3687.5, 60 sec: 3618.1, 300 sec: 3554.5). Total num frames: 1273856. Throughput: 0: 872.4. Samples: 316970. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-07-24 17:54:11,389][03928] Avg episode reward: [(0, '9.557')] -[2024-07-24 17:54:11,400][05135] Saving new best policy, reward=9.557! -[2024-07-24 17:54:16,388][03928] Fps is (10 sec: 4505.6, 60 sec: 3686.4, 300 sec: 3554.5). Total num frames: 1294336. Throughput: 0: 924.4. Samples: 323264. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-07-24 17:54:16,391][03928] Avg episode reward: [(0, '8.574')] -[2024-07-24 17:54:21,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3554.5). Total num frames: 1306624. Throughput: 0: 917.8. Samples: 325624. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2024-07-24 17:54:21,393][03928] Avg episode reward: [(0, '8.608')] -[2024-07-24 17:54:22,140][05148] Updated weights for policy 0, policy_version 320 (0.0014) -[2024-07-24 17:54:26,388][03928] Fps is (10 sec: 2867.4, 60 sec: 3481.6, 300 sec: 3554.5). Total num frames: 1323008. Throughput: 0: 874.8. Samples: 330084. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2024-07-24 17:54:26,393][03928] Avg episode reward: [(0, '8.180')] -[2024-07-24 17:54:31,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3686.5, 300 sec: 3554.5). Total num frames: 1347584. Throughput: 0: 910.0. Samples: 336512. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-07-24 17:54:31,390][03928] Avg episode reward: [(0, '8.560')] -[2024-07-24 17:54:32,143][05148] Updated weights for policy 0, policy_version 330 (0.0029) -[2024-07-24 17:54:36,391][03928] Fps is (10 sec: 4094.7, 60 sec: 3617.9, 300 sec: 3554.5). Total num frames: 1363968. Throughput: 0: 939.2. Samples: 339862. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2024-07-24 17:54:36,394][03928] Avg episode reward: [(0, '8.343')] -[2024-07-24 17:54:41,389][03928] Fps is (10 sec: 2866.9, 60 sec: 3481.5, 300 sec: 3554.5). Total num frames: 1376256. Throughput: 0: 892.6. Samples: 343920. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2024-07-24 17:54:41,393][03928] Avg episode reward: [(0, '8.515')] -[2024-07-24 17:54:44,407][05148] Updated weights for policy 0, policy_version 340 (0.0042) -[2024-07-24 17:54:46,388][03928] Fps is (10 sec: 3687.6, 60 sec: 3686.4, 300 sec: 3568.4). Total num frames: 1400832. Throughput: 0: 885.0. Samples: 349690. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2024-07-24 17:54:46,390][03928] Avg episode reward: [(0, '9.127')] -[2024-07-24 17:54:51,388][03928] Fps is (10 sec: 4096.4, 60 sec: 3618.1, 300 sec: 3554.5). Total num frames: 1417216. Throughput: 0: 915.6. Samples: 352952. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2024-07-24 17:54:51,390][03928] Avg episode reward: [(0, '9.707')] -[2024-07-24 17:54:51,398][05135] Saving new best policy, reward=9.707! -[2024-07-24 17:54:56,394][03928] Fps is (10 sec: 2865.2, 60 sec: 3481.2, 300 sec: 3540.5). Total num frames: 1429504. Throughput: 0: 907.2. Samples: 357802. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2024-07-24 17:54:56,401][03928] Avg episode reward: [(0, '9.535')] -[2024-07-24 17:54:56,447][05148] Updated weights for policy 0, policy_version 350 (0.0027) -[2024-07-24 17:55:01,390][03928] Fps is (10 sec: 3276.1, 60 sec: 3549.9, 300 sec: 3554.5). Total num frames: 1449984. Throughput: 0: 880.8. Samples: 362900. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 17:55:01,396][03928] Avg episode reward: [(0, '9.454')] -[2024-07-24 17:55:06,301][05148] Updated weights for policy 0, policy_version 360 (0.0021) -[2024-07-24 17:55:06,388][03928] Fps is (10 sec: 4508.7, 60 sec: 3754.7, 300 sec: 3568.4). Total num frames: 1474560. Throughput: 0: 899.9. Samples: 366120. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 17:55:06,394][03928] Avg episode reward: [(0, '9.840')] -[2024-07-24 17:55:06,398][05135] Saving new best policy, reward=9.840! -[2024-07-24 17:55:11,389][03928] Fps is (10 sec: 4096.3, 60 sec: 3618.0, 300 sec: 3568.4). Total num frames: 1490944. Throughput: 0: 929.8. Samples: 371928. Policy #0 lag: (min: 0.0, avg: 0.3, max: 2.0) -[2024-07-24 17:55:11,391][03928] Avg episode reward: [(0, '9.908')] -[2024-07-24 17:55:11,405][05135] Saving new best policy, reward=9.908! -[2024-07-24 17:55:16,388][03928] Fps is (10 sec: 2867.2, 60 sec: 3481.6, 300 sec: 3554.5). Total num frames: 1503232. Throughput: 0: 879.0. Samples: 376068. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2024-07-24 17:55:16,390][03928] Avg episode reward: [(0, '10.292')] -[2024-07-24 17:55:16,395][05135] Saving new best policy, reward=10.292! -[2024-07-24 17:55:18,830][05148] Updated weights for policy 0, policy_version 370 (0.0026) -[2024-07-24 17:55:21,388][03928] Fps is (10 sec: 3277.2, 60 sec: 3618.1, 300 sec: 3554.5). Total num frames: 1523712. Throughput: 0: 870.9. Samples: 379052. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2024-07-24 17:55:21,394][03928] Avg episode reward: [(0, '10.595')] -[2024-07-24 17:55:21,406][05135] Saving new best policy, reward=10.595! -[2024-07-24 17:55:26,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3554.5). Total num frames: 1544192. Throughput: 0: 920.2. Samples: 385330. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2024-07-24 17:55:26,393][03928] Avg episode reward: [(0, '10.916')] -[2024-07-24 17:55:26,403][05135] Saving new best policy, reward=10.916! -[2024-07-24 17:55:30,606][05148] Updated weights for policy 0, policy_version 380 (0.0021) -[2024-07-24 17:55:31,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3554.5). Total num frames: 1556480. Throughput: 0: 883.7. Samples: 389458. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2024-07-24 17:55:31,391][03928] Avg episode reward: [(0, '11.562')] -[2024-07-24 17:55:31,403][05135] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000380_1556480.pth... -[2024-07-24 17:55:31,618][05135] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000173_708608.pth -[2024-07-24 17:55:31,638][05135] Saving new best policy, reward=11.562! -[2024-07-24 17:55:36,388][03928] Fps is (10 sec: 2867.2, 60 sec: 3481.8, 300 sec: 3540.6). Total num frames: 1572864. Throughput: 0: 859.0. Samples: 391606. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2024-07-24 17:55:36,393][03928] Avg episode reward: [(0, '12.422')] -[2024-07-24 17:55:36,397][05135] Saving new best policy, reward=12.422! -[2024-07-24 17:55:41,254][05148] Updated weights for policy 0, policy_version 390 (0.0027) -[2024-07-24 17:55:41,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3686.5, 300 sec: 3554.5). Total num frames: 1597440. Throughput: 0: 894.4. Samples: 398042. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2024-07-24 17:55:41,394][03928] Avg episode reward: [(0, '11.908')] -[2024-07-24 17:55:46,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3549.9, 300 sec: 3554.5). Total num frames: 1613824. Throughput: 0: 898.8. Samples: 403342. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 17:55:46,393][03928] Avg episode reward: [(0, '11.706')] -[2024-07-24 17:55:51,388][03928] Fps is (10 sec: 2867.2, 60 sec: 3481.6, 300 sec: 3554.5). Total num frames: 1626112. Throughput: 0: 871.0. Samples: 405316. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2024-07-24 17:55:51,389][03928] Avg episode reward: [(0, '11.442')] -[2024-07-24 17:55:53,512][05148] Updated weights for policy 0, policy_version 400 (0.0028) -[2024-07-24 17:55:56,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3686.8, 300 sec: 3554.5). Total num frames: 1650688. Throughput: 0: 874.7. Samples: 411288. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 17:55:56,389][03928] Avg episode reward: [(0, '11.429')] -[2024-07-24 17:56:01,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3618.3, 300 sec: 3540.6). Total num frames: 1667072. Throughput: 0: 924.8. Samples: 417684. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-07-24 17:56:01,392][03928] Avg episode reward: [(0, '12.423')] -[2024-07-24 17:56:01,398][05135] Saving new best policy, reward=12.423! -[2024-07-24 17:56:04,712][05148] Updated weights for policy 0, policy_version 410 (0.0038) -[2024-07-24 17:56:06,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3554.5). Total num frames: 1683456. Throughput: 0: 901.4. Samples: 419614. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 17:56:06,397][03928] Avg episode reward: [(0, '13.423')] -[2024-07-24 17:56:06,402][05135] Saving new best policy, reward=13.423! -[2024-07-24 17:56:11,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3554.5). Total num frames: 1703936. Throughput: 0: 873.0. Samples: 424614. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-07-24 17:56:11,393][03928] Avg episode reward: [(0, '13.634')] -[2024-07-24 17:56:11,404][05135] Saving new best policy, reward=13.634! -[2024-07-24 17:56:15,038][05148] Updated weights for policy 0, policy_version 420 (0.0035) -[2024-07-24 17:56:16,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3554.5). Total num frames: 1724416. Throughput: 0: 926.9. Samples: 431170. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 17:56:16,392][03928] Avg episode reward: [(0, '14.456')] -[2024-07-24 17:56:16,398][05135] Saving new best policy, reward=14.456! -[2024-07-24 17:56:21,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3540.6). Total num frames: 1736704. Throughput: 0: 934.4. Samples: 433656. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 17:56:21,393][03928] Avg episode reward: [(0, '14.105')] -[2024-07-24 17:56:26,388][03928] Fps is (10 sec: 2867.2, 60 sec: 3481.6, 300 sec: 3554.5). Total num frames: 1753088. Throughput: 0: 881.3. Samples: 437702. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 17:56:26,389][03928] Avg episode reward: [(0, '14.255')] -[2024-07-24 17:56:27,441][05148] Updated weights for policy 0, policy_version 430 (0.0014) -[2024-07-24 17:56:31,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3554.5). Total num frames: 1777664. Throughput: 0: 908.4. Samples: 444218. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 17:56:31,392][03928] Avg episode reward: [(0, '14.285')] -[2024-07-24 17:56:36,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3540.6). Total num frames: 1794048. Throughput: 0: 938.3. Samples: 447540. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2024-07-24 17:56:36,394][03928] Avg episode reward: [(0, '12.840')] -[2024-07-24 17:56:38,177][05148] Updated weights for policy 0, policy_version 440 (0.0020) -[2024-07-24 17:56:41,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3554.5). Total num frames: 1810432. Throughput: 0: 904.4. Samples: 451986. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 17:56:41,395][03928] Avg episode reward: [(0, '12.887')] -[2024-07-24 17:56:46,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3568.4). Total num frames: 1830912. Throughput: 0: 887.7. Samples: 457632. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2024-07-24 17:56:46,390][03928] Avg episode reward: [(0, '12.672')] -[2024-07-24 17:56:48,924][05148] Updated weights for policy 0, policy_version 450 (0.0028) -[2024-07-24 17:56:51,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3754.7, 300 sec: 3554.5). Total num frames: 1851392. Throughput: 0: 916.4. Samples: 460850. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 17:56:51,389][03928] Avg episode reward: [(0, '12.969')] -[2024-07-24 17:56:56,390][03928] Fps is (10 sec: 3276.1, 60 sec: 3549.7, 300 sec: 3540.6). Total num frames: 1863680. Throughput: 0: 920.4. Samples: 466036. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 17:56:56,392][03928] Avg episode reward: [(0, '14.355')] -[2024-07-24 17:57:01,391][03928] Fps is (10 sec: 2456.9, 60 sec: 3481.4, 300 sec: 3540.6). Total num frames: 1875968. Throughput: 0: 846.8. Samples: 469280. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 17:57:01,398][03928] Avg episode reward: [(0, '14.263')] -[2024-07-24 17:57:03,873][05148] Updated weights for policy 0, policy_version 460 (0.0027) -[2024-07-24 17:57:06,388][03928] Fps is (10 sec: 2867.8, 60 sec: 3481.6, 300 sec: 3526.7). Total num frames: 1892352. Throughput: 0: 834.0. Samples: 471184. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 17:57:06,390][03928] Avg episode reward: [(0, '13.679')] -[2024-07-24 17:57:11,388][03928] Fps is (10 sec: 3687.5, 60 sec: 3481.6, 300 sec: 3512.9). Total num frames: 1912832. Throughput: 0: 890.7. Samples: 477782. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) -[2024-07-24 17:57:11,391][03928] Avg episode reward: [(0, '13.150')] -[2024-07-24 17:57:14,244][05148] Updated weights for policy 0, policy_version 470 (0.0028) -[2024-07-24 17:57:16,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3413.3, 300 sec: 3526.7). Total num frames: 1929216. Throughput: 0: 853.8. Samples: 482640. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2024-07-24 17:57:16,393][03928] Avg episode reward: [(0, '12.833')] -[2024-07-24 17:57:21,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3526.7). Total num frames: 1945600. Throughput: 0: 820.7. Samples: 484472. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2024-07-24 17:57:21,390][03928] Avg episode reward: [(0, '13.420')] -[2024-07-24 17:57:25,932][05148] Updated weights for policy 0, policy_version 480 (0.0035) -[2024-07-24 17:57:26,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3554.5). Total num frames: 1966080. Throughput: 0: 858.6. Samples: 490622. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 17:57:26,395][03928] Avg episode reward: [(0, '14.727')] -[2024-07-24 17:57:26,406][05135] Saving new best policy, reward=14.727! -[2024-07-24 17:57:31,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3413.3, 300 sec: 3554.5). Total num frames: 1982464. Throughput: 0: 860.8. Samples: 496368. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2024-07-24 17:57:31,391][03928] Avg episode reward: [(0, '15.248')] -[2024-07-24 17:57:31,406][05135] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000484_1982464.pth... -[2024-07-24 17:57:31,562][05135] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000275_1126400.pth -[2024-07-24 17:57:31,579][05135] Saving new best policy, reward=15.248! -[2024-07-24 17:57:36,388][03928] Fps is (10 sec: 2867.2, 60 sec: 3345.1, 300 sec: 3554.5). Total num frames: 1994752. Throughput: 0: 831.4. Samples: 498264. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 17:57:36,394][03928] Avg episode reward: [(0, '16.611')] -[2024-07-24 17:57:36,404][05135] Saving new best policy, reward=16.611! -[2024-07-24 17:57:38,558][05148] Updated weights for policy 0, policy_version 490 (0.0040) -[2024-07-24 17:57:41,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 3554.5). Total num frames: 2015232. Throughput: 0: 827.1. Samples: 503254. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 17:57:41,394][03928] Avg episode reward: [(0, '17.272')] -[2024-07-24 17:57:41,460][05135] Saving new best policy, reward=17.272! -[2024-07-24 17:57:46,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3413.3, 300 sec: 3554.5). Total num frames: 2035712. Throughput: 0: 892.8. Samples: 509452. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2024-07-24 17:57:46,392][03928] Avg episode reward: [(0, '16.641')] -[2024-07-24 17:57:49,443][05148] Updated weights for policy 0, policy_version 500 (0.0047) -[2024-07-24 17:57:51,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3345.1, 300 sec: 3568.4). Total num frames: 2052096. Throughput: 0: 906.4. Samples: 511970. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2024-07-24 17:57:51,390][03928] Avg episode reward: [(0, '16.230')] -[2024-07-24 17:57:56,390][03928] Fps is (10 sec: 3276.1, 60 sec: 3413.3, 300 sec: 3554.5). Total num frames: 2068480. Throughput: 0: 847.0. Samples: 515898. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 17:57:56,395][03928] Avg episode reward: [(0, '15.687')] -[2024-07-24 17:58:01,143][05148] Updated weights for policy 0, policy_version 510 (0.0030) -[2024-07-24 17:58:01,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3550.0, 300 sec: 3554.5). Total num frames: 2088960. Throughput: 0: 877.1. Samples: 522110. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-07-24 17:58:01,393][03928] Avg episode reward: [(0, '15.674')] -[2024-07-24 17:58:06,388][03928] Fps is (10 sec: 3687.2, 60 sec: 3549.9, 300 sec: 3554.5). Total num frames: 2105344. Throughput: 0: 904.7. Samples: 525182. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-07-24 17:58:06,395][03928] Avg episode reward: [(0, '15.286')] -[2024-07-24 17:58:11,388][03928] Fps is (10 sec: 2867.1, 60 sec: 3413.3, 300 sec: 3540.6). Total num frames: 2117632. Throughput: 0: 858.7. Samples: 529266. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-07-24 17:58:11,391][03928] Avg episode reward: [(0, '16.167')] -[2024-07-24 17:58:13,887][05148] Updated weights for policy 0, policy_version 520 (0.0038) -[2024-07-24 17:58:16,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3540.6). Total num frames: 2138112. Throughput: 0: 851.4. Samples: 534680. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 17:58:16,396][03928] Avg episode reward: [(0, '16.406')] -[2024-07-24 17:58:21,388][03928] Fps is (10 sec: 4096.2, 60 sec: 3549.9, 300 sec: 3540.6). Total num frames: 2158592. Throughput: 0: 877.3. Samples: 537744. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 17:58:21,390][03928] Avg episode reward: [(0, '16.621')] -[2024-07-24 17:58:25,217][05148] Updated weights for policy 0, policy_version 530 (0.0030) -[2024-07-24 17:58:26,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 3540.6). Total num frames: 2170880. Throughput: 0: 879.4. Samples: 542826. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 17:58:26,392][03928] Avg episode reward: [(0, '17.562')] -[2024-07-24 17:58:26,397][05135] Saving new best policy, reward=17.562! -[2024-07-24 17:58:31,388][03928] Fps is (10 sec: 2867.2, 60 sec: 3413.3, 300 sec: 3526.7). Total num frames: 2187264. Throughput: 0: 841.2. Samples: 547308. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 17:58:31,390][03928] Avg episode reward: [(0, '17.960')] -[2024-07-24 17:58:31,398][05135] Saving new best policy, reward=17.960! -[2024-07-24 17:58:36,281][05148] Updated weights for policy 0, policy_version 540 (0.0021) -[2024-07-24 17:58:36,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3618.1, 300 sec: 3540.6). Total num frames: 2211840. Throughput: 0: 854.3. Samples: 550414. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 17:58:36,390][03928] Avg episode reward: [(0, '17.697')] -[2024-07-24 17:58:41,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3549.9, 300 sec: 3554.5). Total num frames: 2228224. Throughput: 0: 909.0. Samples: 556802. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2024-07-24 17:58:41,390][03928] Avg episode reward: [(0, '18.157')] -[2024-07-24 17:58:41,397][05135] Saving new best policy, reward=18.157! -[2024-07-24 17:58:46,388][03928] Fps is (10 sec: 2867.2, 60 sec: 3413.3, 300 sec: 3526.7). Total num frames: 2240512. Throughput: 0: 855.6. Samples: 560614. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2024-07-24 17:58:46,391][03928] Avg episode reward: [(0, '17.741')] -[2024-07-24 17:58:48,629][05148] Updated weights for policy 0, policy_version 550 (0.0046) -[2024-07-24 17:58:51,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3526.7). Total num frames: 2260992. Throughput: 0: 851.8. Samples: 563514. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 17:58:51,389][03928] Avg episode reward: [(0, '18.106')] -[2024-07-24 17:58:56,388][03928] Fps is (10 sec: 4095.9, 60 sec: 3550.0, 300 sec: 3540.6). Total num frames: 2281472. Throughput: 0: 902.8. Samples: 569892. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-07-24 17:58:56,390][03928] Avg episode reward: [(0, '19.165')] -[2024-07-24 17:58:56,394][05135] Saving new best policy, reward=19.165! -[2024-07-24 17:58:59,781][05148] Updated weights for policy 0, policy_version 560 (0.0018) -[2024-07-24 17:59:01,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3481.6, 300 sec: 3554.5). Total num frames: 2297856. Throughput: 0: 886.2. Samples: 574558. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2024-07-24 17:59:01,394][03928] Avg episode reward: [(0, '18.524')] -[2024-07-24 17:59:06,388][03928] Fps is (10 sec: 3276.9, 60 sec: 3481.6, 300 sec: 3526.7). Total num frames: 2314240. Throughput: 0: 863.5. Samples: 576602. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2024-07-24 17:59:06,391][03928] Avg episode reward: [(0, '18.026')] -[2024-07-24 17:59:10,396][05148] Updated weights for policy 0, policy_version 570 (0.0017) -[2024-07-24 17:59:11,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3540.6). Total num frames: 2338816. Throughput: 0: 896.2. Samples: 583154. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 17:59:11,390][03928] Avg episode reward: [(0, '17.310')] -[2024-07-24 17:59:16,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3618.1, 300 sec: 3554.5). Total num frames: 2355200. Throughput: 0: 925.7. Samples: 588966. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-07-24 17:59:16,392][03928] Avg episode reward: [(0, '16.380')] -[2024-07-24 17:59:21,388][03928] Fps is (10 sec: 2867.2, 60 sec: 3481.6, 300 sec: 3540.6). Total num frames: 2367488. Throughput: 0: 901.2. Samples: 590968. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2024-07-24 17:59:21,392][03928] Avg episode reward: [(0, '16.258')] -[2024-07-24 17:59:22,734][05148] Updated weights for policy 0, policy_version 580 (0.0021) -[2024-07-24 17:59:26,388][03928] Fps is (10 sec: 3276.6, 60 sec: 3618.1, 300 sec: 3526.7). Total num frames: 2387968. Throughput: 0: 883.1. Samples: 596542. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 17:59:26,394][03928] Avg episode reward: [(0, '16.607')] -[2024-07-24 17:59:31,389][03928] Fps is (10 sec: 4504.9, 60 sec: 3754.6, 300 sec: 3554.5). Total num frames: 2412544. Throughput: 0: 941.3. Samples: 602974. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 17:59:31,392][03928] Avg episode reward: [(0, '17.820')] -[2024-07-24 17:59:31,404][05135] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000589_2412544.pth... -[2024-07-24 17:59:31,552][05135] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000380_1556480.pth -[2024-07-24 17:59:32,709][05148] Updated weights for policy 0, policy_version 590 (0.0018) -[2024-07-24 17:59:36,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3549.8, 300 sec: 3554.5). Total num frames: 2424832. Throughput: 0: 924.2. Samples: 605102. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2024-07-24 17:59:36,396][03928] Avg episode reward: [(0, '18.054')] -[2024-07-24 17:59:41,388][03928] Fps is (10 sec: 2867.6, 60 sec: 3549.9, 300 sec: 3526.7). Total num frames: 2441216. Throughput: 0: 884.9. Samples: 609712. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2024-07-24 17:59:41,393][03928] Avg episode reward: [(0, '18.196')] -[2024-07-24 17:59:44,452][05148] Updated weights for policy 0, policy_version 600 (0.0020) -[2024-07-24 17:59:46,391][03928] Fps is (10 sec: 4094.9, 60 sec: 3754.5, 300 sec: 3554.5). Total num frames: 2465792. Throughput: 0: 922.6. Samples: 616076. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 17:59:46,393][03928] Avg episode reward: [(0, '18.771')] -[2024-07-24 17:59:51,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3568.5). Total num frames: 2482176. Throughput: 0: 946.2. Samples: 619182. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-07-24 17:59:51,390][03928] Avg episode reward: [(0, '18.468')] -[2024-07-24 17:59:56,388][03928] Fps is (10 sec: 2868.1, 60 sec: 3549.9, 300 sec: 3540.6). Total num frames: 2494464. Throughput: 0: 890.5. Samples: 623228. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 17:59:56,389][03928] Avg episode reward: [(0, '18.800')] -[2024-07-24 17:59:56,931][05148] Updated weights for policy 0, policy_version 610 (0.0034) -[2024-07-24 18:00:01,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3540.6). Total num frames: 2519040. Throughput: 0: 895.3. Samples: 629256. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 18:00:01,396][03928] Avg episode reward: [(0, '19.406')] -[2024-07-24 18:00:01,405][05135] Saving new best policy, reward=19.406! -[2024-07-24 18:00:06,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3540.6). Total num frames: 2535424. Throughput: 0: 923.1. Samples: 632506. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 18:00:06,392][03928] Avg episode reward: [(0, '19.363')] -[2024-07-24 18:00:06,694][05148] Updated weights for policy 0, policy_version 620 (0.0042) -[2024-07-24 18:00:11,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3554.5). Total num frames: 2551808. Throughput: 0: 909.1. Samples: 637452. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) -[2024-07-24 18:00:11,397][03928] Avg episode reward: [(0, '19.925')] -[2024-07-24 18:00:11,422][05135] Saving new best policy, reward=19.925! -[2024-07-24 18:00:16,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3540.6). Total num frames: 2568192. Throughput: 0: 878.8. Samples: 642518. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2024-07-24 18:00:16,394][03928] Avg episode reward: [(0, '20.714')] -[2024-07-24 18:00:16,397][05135] Saving new best policy, reward=20.714! -[2024-07-24 18:00:18,457][05148] Updated weights for policy 0, policy_version 630 (0.0045) -[2024-07-24 18:00:21,390][03928] Fps is (10 sec: 3685.7, 60 sec: 3686.3, 300 sec: 3540.6). Total num frames: 2588672. Throughput: 0: 902.1. Samples: 645700. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 18:00:21,395][03928] Avg episode reward: [(0, '20.538')] -[2024-07-24 18:00:26,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3618.2, 300 sec: 3554.5). Total num frames: 2605056. Throughput: 0: 927.8. Samples: 651462. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2024-07-24 18:00:26,390][03928] Avg episode reward: [(0, '19.510')] -[2024-07-24 18:00:30,605][05148] Updated weights for policy 0, policy_version 640 (0.0027) -[2024-07-24 18:00:31,388][03928] Fps is (10 sec: 3277.4, 60 sec: 3481.7, 300 sec: 3554.5). Total num frames: 2621440. Throughput: 0: 880.0. Samples: 655674. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2024-07-24 18:00:31,392][03928] Avg episode reward: [(0, '18.659')] -[2024-07-24 18:00:36,390][03928] Fps is (10 sec: 4095.0, 60 sec: 3686.3, 300 sec: 3554.5). Total num frames: 2646016. Throughput: 0: 880.8. Samples: 658820. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 18:00:36,394][03928] Avg episode reward: [(0, '18.538')] -[2024-07-24 18:00:40,230][05148] Updated weights for policy 0, policy_version 650 (0.0023) -[2024-07-24 18:00:41,394][03928] Fps is (10 sec: 4093.5, 60 sec: 3686.0, 300 sec: 3554.4). Total num frames: 2662400. Throughput: 0: 937.3. Samples: 665412. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2024-07-24 18:00:41,396][03928] Avg episode reward: [(0, '18.118')] -[2024-07-24 18:00:46,388][03928] Fps is (10 sec: 2867.9, 60 sec: 3481.8, 300 sec: 3554.5). Total num frames: 2674688. Throughput: 0: 896.3. Samples: 669590. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2024-07-24 18:00:46,393][03928] Avg episode reward: [(0, '19.090')] -[2024-07-24 18:00:51,388][03928] Fps is (10 sec: 3278.8, 60 sec: 3549.9, 300 sec: 3540.6). Total num frames: 2695168. Throughput: 0: 879.9. Samples: 672100. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 18:00:51,390][03928] Avg episode reward: [(0, '19.696')] -[2024-07-24 18:00:52,577][05148] Updated weights for policy 0, policy_version 660 (0.0032) -[2024-07-24 18:00:56,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3554.5). Total num frames: 2715648. Throughput: 0: 911.4. Samples: 678464. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 18:00:56,392][03928] Avg episode reward: [(0, '19.752')] -[2024-07-24 18:01:01,390][03928] Fps is (10 sec: 3685.7, 60 sec: 3549.7, 300 sec: 3554.5). Total num frames: 2732032. Throughput: 0: 908.1. Samples: 683384. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 18:01:01,396][03928] Avg episode reward: [(0, '20.074')] -[2024-07-24 18:01:05,368][05148] Updated weights for policy 0, policy_version 670 (0.0046) -[2024-07-24 18:01:06,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3540.6). Total num frames: 2748416. Throughput: 0: 880.1. Samples: 685302. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 18:01:06,391][03928] Avg episode reward: [(0, '19.223')] -[2024-07-24 18:01:11,388][03928] Fps is (10 sec: 3687.1, 60 sec: 3618.1, 300 sec: 3540.6). Total num frames: 2768896. Throughput: 0: 880.8. Samples: 691096. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 18:01:11,394][03928] Avg episode reward: [(0, '19.820')] -[2024-07-24 18:01:14,889][05148] Updated weights for policy 0, policy_version 680 (0.0022) -[2024-07-24 18:01:16,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3554.5). Total num frames: 2785280. Throughput: 0: 922.0. Samples: 697166. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 18:01:16,390][03928] Avg episode reward: [(0, '19.437')] -[2024-07-24 18:01:21,389][03928] Fps is (10 sec: 3276.5, 60 sec: 3549.9, 300 sec: 3554.5). Total num frames: 2801664. Throughput: 0: 895.1. Samples: 699096. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2024-07-24 18:01:21,391][03928] Avg episode reward: [(0, '19.917')] -[2024-07-24 18:01:26,388][03928] Fps is (10 sec: 2457.6, 60 sec: 3413.3, 300 sec: 3499.0). Total num frames: 2809856. Throughput: 0: 828.2. Samples: 702678. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2024-07-24 18:01:26,389][03928] Avg episode reward: [(0, '20.748')] -[2024-07-24 18:01:26,407][05135] Saving new best policy, reward=20.748! -[2024-07-24 18:01:30,526][05148] Updated weights for policy 0, policy_version 690 (0.0020) -[2024-07-24 18:01:31,388][03928] Fps is (10 sec: 2457.9, 60 sec: 3413.3, 300 sec: 3499.0). Total num frames: 2826240. Throughput: 0: 832.5. Samples: 707054. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 18:01:31,390][03928] Avg episode reward: [(0, '20.377')] -[2024-07-24 18:01:31,403][05135] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000690_2826240.pth... -[2024-07-24 18:01:31,542][05135] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000484_1982464.pth -[2024-07-24 18:01:36,390][03928] Fps is (10 sec: 3276.1, 60 sec: 3276.8, 300 sec: 3498.9). Total num frames: 2842624. Throughput: 0: 835.9. Samples: 709716. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 18:01:36,393][03928] Avg episode reward: [(0, '19.809')] -[2024-07-24 18:01:41,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3277.1, 300 sec: 3485.1). Total num frames: 2859008. Throughput: 0: 782.1. Samples: 713660. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2024-07-24 18:01:41,390][03928] Avg episode reward: [(0, '20.442')] -[2024-07-24 18:01:43,036][05148] Updated weights for policy 0, policy_version 700 (0.0034) -[2024-07-24 18:01:46,388][03928] Fps is (10 sec: 3687.1, 60 sec: 3413.3, 300 sec: 3485.1). Total num frames: 2879488. Throughput: 0: 809.3. Samples: 719802. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-07-24 18:01:46,390][03928] Avg episode reward: [(0, '20.249')] -[2024-07-24 18:01:51,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3345.1, 300 sec: 3499.0). Total num frames: 2895872. Throughput: 0: 835.3. Samples: 722892. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2024-07-24 18:01:51,392][03928] Avg episode reward: [(0, '20.259')] -[2024-07-24 18:01:54,732][05148] Updated weights for policy 0, policy_version 710 (0.0032) -[2024-07-24 18:01:56,388][03928] Fps is (10 sec: 3276.6, 60 sec: 3276.8, 300 sec: 3512.9). Total num frames: 2912256. Throughput: 0: 808.9. Samples: 727498. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 18:01:56,393][03928] Avg episode reward: [(0, '19.961')] -[2024-07-24 18:02:01,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3276.9, 300 sec: 3512.8). Total num frames: 2928640. Throughput: 0: 785.3. Samples: 732506. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 18:02:01,394][03928] Avg episode reward: [(0, '20.125')] -[2024-07-24 18:02:05,411][05148] Updated weights for policy 0, policy_version 720 (0.0025) -[2024-07-24 18:02:06,390][03928] Fps is (10 sec: 4095.4, 60 sec: 3413.2, 300 sec: 3526.7). Total num frames: 2953216. Throughput: 0: 814.5. Samples: 735750. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 18:02:06,394][03928] Avg episode reward: [(0, '19.647')] -[2024-07-24 18:02:11,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3276.8, 300 sec: 3512.8). Total num frames: 2965504. Throughput: 0: 861.1. Samples: 741426. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2024-07-24 18:02:11,394][03928] Avg episode reward: [(0, '19.592')] -[2024-07-24 18:02:16,388][03928] Fps is (10 sec: 2867.8, 60 sec: 3276.8, 300 sec: 3512.8). Total num frames: 2981888. Throughput: 0: 857.5. Samples: 745640. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2024-07-24 18:02:16,390][03928] Avg episode reward: [(0, '19.007')] -[2024-07-24 18:02:17,820][05148] Updated weights for policy 0, policy_version 730 (0.0023) -[2024-07-24 18:02:21,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3345.1, 300 sec: 3512.8). Total num frames: 3002368. Throughput: 0: 869.4. Samples: 748836. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 18:02:21,390][03928] Avg episode reward: [(0, '18.736')] -[2024-07-24 18:02:26,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3549.9, 300 sec: 3526.7). Total num frames: 3022848. Throughput: 0: 920.6. Samples: 755088. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 18:02:26,390][03928] Avg episode reward: [(0, '18.715')] -[2024-07-24 18:02:29,368][05148] Updated weights for policy 0, policy_version 740 (0.0024) -[2024-07-24 18:02:31,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3526.7). Total num frames: 3035136. Throughput: 0: 874.5. Samples: 759156. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2024-07-24 18:02:31,394][03928] Avg episode reward: [(0, '19.211')] -[2024-07-24 18:02:36,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3550.0, 300 sec: 3526.7). Total num frames: 3055616. Throughput: 0: 861.5. Samples: 761658. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 18:02:36,391][03928] Avg episode reward: [(0, '18.984')] -[2024-07-24 18:02:40,026][05148] Updated weights for policy 0, policy_version 750 (0.0042) -[2024-07-24 18:02:41,389][03928] Fps is (10 sec: 4095.5, 60 sec: 3618.1, 300 sec: 3526.7). Total num frames: 3076096. Throughput: 0: 898.7. Samples: 767940. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 18:02:41,392][03928] Avg episode reward: [(0, '20.207')] -[2024-07-24 18:02:46,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3512.8). Total num frames: 3088384. Throughput: 0: 896.0. Samples: 772824. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-07-24 18:02:46,390][03928] Avg episode reward: [(0, '21.139')] -[2024-07-24 18:02:46,392][05135] Saving new best policy, reward=21.139! -[2024-07-24 18:02:51,388][03928] Fps is (10 sec: 2867.6, 60 sec: 3481.6, 300 sec: 3512.9). Total num frames: 3104768. Throughput: 0: 866.0. Samples: 774716. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2024-07-24 18:02:51,390][03928] Avg episode reward: [(0, '21.132')] -[2024-07-24 18:02:52,666][05148] Updated weights for policy 0, policy_version 760 (0.0029) -[2024-07-24 18:02:56,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3512.8). Total num frames: 3125248. Throughput: 0: 868.5. Samples: 780508. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 18:02:56,395][03928] Avg episode reward: [(0, '19.634')] -[2024-07-24 18:03:01,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3512.8). Total num frames: 3141632. Throughput: 0: 900.7. Samples: 786170. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 18:03:01,390][03928] Avg episode reward: [(0, '19.616')] -[2024-07-24 18:03:04,842][05148] Updated weights for policy 0, policy_version 770 (0.0015) -[2024-07-24 18:03:06,391][03928] Fps is (10 sec: 3275.6, 60 sec: 3413.3, 300 sec: 3526.7). Total num frames: 3158016. Throughput: 0: 871.3. Samples: 788048. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 18:03:06,394][03928] Avg episode reward: [(0, '19.844')] -[2024-07-24 18:03:11,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3512.8). Total num frames: 3174400. Throughput: 0: 845.4. Samples: 793130. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 18:03:11,390][03928] Avg episode reward: [(0, '18.447')] -[2024-07-24 18:03:15,652][05148] Updated weights for policy 0, policy_version 780 (0.0026) -[2024-07-24 18:03:16,388][03928] Fps is (10 sec: 3687.7, 60 sec: 3549.9, 300 sec: 3512.8). Total num frames: 3194880. Throughput: 0: 886.7. Samples: 799058. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 18:03:16,390][03928] Avg episode reward: [(0, '18.362')] -[2024-07-24 18:03:21,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 3512.8). Total num frames: 3207168. Throughput: 0: 884.3. Samples: 801450. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2024-07-24 18:03:21,393][03928] Avg episode reward: [(0, '19.209')] -[2024-07-24 18:03:26,388][03928] Fps is (10 sec: 2867.0, 60 sec: 3345.0, 300 sec: 3512.8). Total num frames: 3223552. Throughput: 0: 833.3. Samples: 805440. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 18:03:26,392][03928] Avg episode reward: [(0, '19.174')] -[2024-07-24 18:03:28,459][05148] Updated weights for policy 0, policy_version 790 (0.0018) -[2024-07-24 18:03:31,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3549.9, 300 sec: 3512.8). Total num frames: 3248128. Throughput: 0: 863.7. Samples: 811692. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 18:03:31,390][03928] Avg episode reward: [(0, '19.706')] -[2024-07-24 18:03:31,400][05135] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000793_3248128.pth... -[2024-07-24 18:03:31,534][05135] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000589_2412544.pth -[2024-07-24 18:03:36,388][03928] Fps is (10 sec: 4096.2, 60 sec: 3481.6, 300 sec: 3512.8). Total num frames: 3264512. Throughput: 0: 890.6. Samples: 814794. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2024-07-24 18:03:36,392][03928] Avg episode reward: [(0, '19.738')] -[2024-07-24 18:03:40,798][05148] Updated weights for policy 0, policy_version 800 (0.0048) -[2024-07-24 18:03:41,388][03928] Fps is (10 sec: 2867.2, 60 sec: 3345.1, 300 sec: 3512.8). Total num frames: 3276800. Throughput: 0: 854.3. Samples: 818950. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2024-07-24 18:03:41,395][03928] Avg episode reward: [(0, '19.912')] -[2024-07-24 18:03:46,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3512.8). Total num frames: 3297280. Throughput: 0: 851.6. Samples: 824490. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 18:03:46,391][03928] Avg episode reward: [(0, '20.931')] -[2024-07-24 18:03:50,775][05148] Updated weights for policy 0, policy_version 810 (0.0021) -[2024-07-24 18:03:51,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3549.9, 300 sec: 3512.8). Total num frames: 3317760. Throughput: 0: 878.8. Samples: 827590. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) -[2024-07-24 18:03:51,390][03928] Avg episode reward: [(0, '20.755')] -[2024-07-24 18:03:56,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 3499.0). Total num frames: 3330048. Throughput: 0: 881.3. Samples: 832788. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 18:03:56,390][03928] Avg episode reward: [(0, '20.034')] -[2024-07-24 18:04:01,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3512.8). Total num frames: 3350528. Throughput: 0: 847.3. Samples: 837188. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2024-07-24 18:04:01,394][03928] Avg episode reward: [(0, '20.330')] -[2024-07-24 18:04:03,276][05148] Updated weights for policy 0, policy_version 820 (0.0017) -[2024-07-24 18:04:06,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3550.1, 300 sec: 3499.0). Total num frames: 3371008. Throughput: 0: 866.7. Samples: 840450. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 18:04:06,395][03928] Avg episode reward: [(0, '20.146')] -[2024-07-24 18:04:11,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3499.0). Total num frames: 3387392. Throughput: 0: 917.3. Samples: 846716. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 18:04:11,390][03928] Avg episode reward: [(0, '19.309')] -[2024-07-24 18:04:15,446][05148] Updated weights for policy 0, policy_version 830 (0.0024) -[2024-07-24 18:04:16,388][03928] Fps is (10 sec: 2867.2, 60 sec: 3413.3, 300 sec: 3499.0). Total num frames: 3399680. Throughput: 0: 863.9. Samples: 850566. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2024-07-24 18:04:16,390][03928] Avg episode reward: [(0, '18.963')] -[2024-07-24 18:04:21,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3499.0). Total num frames: 3420160. Throughput: 0: 856.6. Samples: 853342. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 18:04:21,395][03928] Avg episode reward: [(0, '19.258')] -[2024-07-24 18:04:25,600][05148] Updated weights for policy 0, policy_version 840 (0.0032) -[2024-07-24 18:04:26,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3618.2, 300 sec: 3485.1). Total num frames: 3440640. Throughput: 0: 902.3. Samples: 859552. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 18:04:26,392][03928] Avg episode reward: [(0, '19.035')] -[2024-07-24 18:04:31,393][03928] Fps is (10 sec: 3275.2, 60 sec: 3413.1, 300 sec: 3485.0). Total num frames: 3452928. Throughput: 0: 874.9. Samples: 863866. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 18:04:31,401][03928] Avg episode reward: [(0, '20.335')] -[2024-07-24 18:04:36,388][03928] Fps is (10 sec: 2867.2, 60 sec: 3413.3, 300 sec: 3485.1). Total num frames: 3469312. Throughput: 0: 849.3. Samples: 865810. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 18:04:36,395][03928] Avg episode reward: [(0, '21.311')] -[2024-07-24 18:04:36,398][05135] Saving new best policy, reward=21.311! -[2024-07-24 18:04:38,586][05148] Updated weights for policy 0, policy_version 850 (0.0018) -[2024-07-24 18:04:41,388][03928] Fps is (10 sec: 3688.3, 60 sec: 3549.9, 300 sec: 3471.2). Total num frames: 3489792. Throughput: 0: 867.2. Samples: 871812. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2024-07-24 18:04:41,390][03928] Avg episode reward: [(0, '22.288')] -[2024-07-24 18:04:41,400][05135] Saving new best policy, reward=22.288! -[2024-07-24 18:04:46,390][03928] Fps is (10 sec: 3685.7, 60 sec: 3481.5, 300 sec: 3471.2). Total num frames: 3506176. Throughput: 0: 891.2. Samples: 877294. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 18:04:46,394][03928] Avg episode reward: [(0, '22.407')] -[2024-07-24 18:04:46,401][05135] Saving new best policy, reward=22.407! -[2024-07-24 18:04:51,388][03928] Fps is (10 sec: 2867.2, 60 sec: 3345.1, 300 sec: 3471.2). Total num frames: 3518464. Throughput: 0: 858.4. Samples: 879080. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 18:04:51,390][03928] Avg episode reward: [(0, '22.836')] -[2024-07-24 18:04:51,406][05135] Saving new best policy, reward=22.836! -[2024-07-24 18:04:51,680][05148] Updated weights for policy 0, policy_version 860 (0.0018) -[2024-07-24 18:04:56,388][03928] Fps is (10 sec: 3277.5, 60 sec: 3481.6, 300 sec: 3457.3). Total num frames: 3538944. Throughput: 0: 835.6. Samples: 884318. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 18:04:56,393][03928] Avg episode reward: [(0, '23.839')] -[2024-07-24 18:04:56,399][05135] Saving new best policy, reward=23.839! -[2024-07-24 18:05:01,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3481.6, 300 sec: 3471.2). Total num frames: 3559424. Throughput: 0: 885.5. Samples: 890412. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 18:05:01,395][03928] Avg episode reward: [(0, '21.871')] -[2024-07-24 18:05:01,522][05148] Updated weights for policy 0, policy_version 870 (0.0035) -[2024-07-24 18:05:06,388][03928] Fps is (10 sec: 3686.2, 60 sec: 3413.3, 300 sec: 3471.2). Total num frames: 3575808. Throughput: 0: 874.6. Samples: 892698. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-07-24 18:05:06,398][03928] Avg episode reward: [(0, '21.223')] -[2024-07-24 18:05:11,391][03928] Fps is (10 sec: 3275.7, 60 sec: 3413.1, 300 sec: 3471.1). Total num frames: 3592192. Throughput: 0: 835.0. Samples: 897130. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-07-24 18:05:11,399][03928] Avg episode reward: [(0, '21.649')] -[2024-07-24 18:05:13,694][05148] Updated weights for policy 0, policy_version 880 (0.0056) -[2024-07-24 18:05:16,388][03928] Fps is (10 sec: 3686.5, 60 sec: 3549.9, 300 sec: 3471.2). Total num frames: 3612672. Throughput: 0: 884.1. Samples: 903646. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 18:05:16,396][03928] Avg episode reward: [(0, '19.774')] -[2024-07-24 18:05:21,388][03928] Fps is (10 sec: 3687.6, 60 sec: 3481.6, 300 sec: 3471.2). Total num frames: 3629056. Throughput: 0: 912.9. Samples: 906890. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2024-07-24 18:05:21,393][03928] Avg episode reward: [(0, '20.340')] -[2024-07-24 18:05:26,307][05148] Updated weights for policy 0, policy_version 890 (0.0022) -[2024-07-24 18:05:26,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 3471.2). Total num frames: 3645440. Throughput: 0: 862.6. Samples: 910628. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 18:05:26,391][03928] Avg episode reward: [(0, '20.521')] -[2024-07-24 18:05:31,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3481.9, 300 sec: 3443.4). Total num frames: 3661824. Throughput: 0: 859.8. Samples: 915984. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 18:05:31,390][03928] Avg episode reward: [(0, '20.443')] -[2024-07-24 18:05:31,406][05135] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000895_3665920.pth... -[2024-07-24 18:05:31,529][05135] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000690_2826240.pth -[2024-07-24 18:05:36,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3457.4). Total num frames: 3682304. Throughput: 0: 885.6. Samples: 918930. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 18:05:36,390][03928] Avg episode reward: [(0, '21.409')] -[2024-07-24 18:05:36,714][05148] Updated weights for policy 0, policy_version 900 (0.0019) -[2024-07-24 18:05:41,391][03928] Fps is (10 sec: 3275.8, 60 sec: 3413.2, 300 sec: 3457.3). Total num frames: 3694592. Throughput: 0: 879.7. Samples: 923906. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 18:05:41,393][03928] Avg episode reward: [(0, '21.583')] -[2024-07-24 18:05:46,388][03928] Fps is (10 sec: 3276.7, 60 sec: 3481.7, 300 sec: 3457.3). Total num frames: 3715072. Throughput: 0: 852.8. Samples: 928790. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2024-07-24 18:05:46,391][03928] Avg episode reward: [(0, '22.659')] -[2024-07-24 18:05:50,046][05148] Updated weights for policy 0, policy_version 910 (0.0027) -[2024-07-24 18:05:51,388][03928] Fps is (10 sec: 3277.7, 60 sec: 3481.6, 300 sec: 3429.5). Total num frames: 3727360. Throughput: 0: 858.8. Samples: 931342. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 18:05:51,393][03928] Avg episode reward: [(0, '22.964')] -[2024-07-24 18:05:56,391][03928] Fps is (10 sec: 2456.7, 60 sec: 3344.9, 300 sec: 3415.6). Total num frames: 3739648. Throughput: 0: 846.9. Samples: 935240. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 18:05:56,394][03928] Avg episode reward: [(0, '21.320')] -[2024-07-24 18:06:01,388][03928] Fps is (10 sec: 2867.3, 60 sec: 3276.8, 300 sec: 3415.6). Total num frames: 3756032. Throughput: 0: 786.0. Samples: 939016. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-07-24 18:06:01,394][03928] Avg episode reward: [(0, '21.737')] -[2024-07-24 18:06:04,144][05148] Updated weights for policy 0, policy_version 920 (0.0029) -[2024-07-24 18:06:06,388][03928] Fps is (10 sec: 3687.8, 60 sec: 3345.1, 300 sec: 3415.6). Total num frames: 3776512. Throughput: 0: 778.0. Samples: 941898. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 18:06:06,390][03928] Avg episode reward: [(0, '21.229')] -[2024-07-24 18:06:11,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3413.5, 300 sec: 3429.5). Total num frames: 3796992. Throughput: 0: 837.8. Samples: 948330. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 18:06:11,393][03928] Avg episode reward: [(0, '20.337')] -[2024-07-24 18:06:15,098][05148] Updated weights for policy 0, policy_version 930 (0.0019) -[2024-07-24 18:06:16,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3276.8, 300 sec: 3415.7). Total num frames: 3809280. Throughput: 0: 824.0. Samples: 953066. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-07-24 18:06:16,393][03928] Avg episode reward: [(0, '19.589')] -[2024-07-24 18:06:21,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3345.1, 300 sec: 3457.3). Total num frames: 3829760. Throughput: 0: 804.8. Samples: 955148. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 18:06:21,392][03928] Avg episode reward: [(0, '18.518')] -[2024-07-24 18:06:26,161][05148] Updated weights for policy 0, policy_version 940 (0.0024) -[2024-07-24 18:06:26,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3413.3, 300 sec: 3471.2). Total num frames: 3850240. Throughput: 0: 833.7. Samples: 961420. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 18:06:26,395][03928] Avg episode reward: [(0, '19.677')] -[2024-07-24 18:06:31,388][03928] Fps is (10 sec: 3686.3, 60 sec: 3413.3, 300 sec: 3471.2). Total num frames: 3866624. Throughput: 0: 850.7. Samples: 967072. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2024-07-24 18:06:31,394][03928] Avg episode reward: [(0, '20.345')] -[2024-07-24 18:06:36,388][03928] Fps is (10 sec: 2867.2, 60 sec: 3276.8, 300 sec: 3457.3). Total num frames: 3878912. Throughput: 0: 838.0. Samples: 969050. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 18:06:36,394][03928] Avg episode reward: [(0, '19.104')] -[2024-07-24 18:06:38,424][05148] Updated weights for policy 0, policy_version 950 (0.0020) -[2024-07-24 18:06:41,389][03928] Fps is (10 sec: 3686.1, 60 sec: 3481.7, 300 sec: 3471.2). Total num frames: 3903488. Throughput: 0: 874.1. Samples: 974570. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 18:06:41,396][03928] Avg episode reward: [(0, '20.782')] -[2024-07-24 18:06:46,388][03928] Fps is (10 sec: 4505.6, 60 sec: 3481.6, 300 sec: 3485.1). Total num frames: 3923968. Throughput: 0: 937.2. Samples: 981192. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 18:06:46,391][03928] Avg episode reward: [(0, '21.816')] -[2024-07-24 18:06:48,710][05148] Updated weights for policy 0, policy_version 960 (0.0034) -[2024-07-24 18:06:51,388][03928] Fps is (10 sec: 3277.2, 60 sec: 3481.6, 300 sec: 3471.2). Total num frames: 3936256. Throughput: 0: 921.8. Samples: 983378. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 18:06:51,392][03928] Avg episode reward: [(0, '21.747')] -[2024-07-24 18:06:56,388][03928] Fps is (10 sec: 3276.8, 60 sec: 3618.4, 300 sec: 3485.1). Total num frames: 3956736. Throughput: 0: 878.6. Samples: 987866. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2024-07-24 18:06:56,390][03928] Avg episode reward: [(0, '21.512')] -[2024-07-24 18:07:00,142][05148] Updated weights for policy 0, policy_version 970 (0.0033) -[2024-07-24 18:07:01,388][03928] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3471.2). Total num frames: 3977216. Throughput: 0: 915.2. Samples: 994248. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2024-07-24 18:07:01,390][03928] Avg episode reward: [(0, '22.009')] -[2024-07-24 18:07:06,388][03928] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3485.1). Total num frames: 3993600. Throughput: 0: 937.6. Samples: 997340. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) -[2024-07-24 18:07:06,393][03928] Avg episode reward: [(0, '21.690')] -[2024-07-24 18:07:10,532][05135] Stopping Batcher_0... -[2024-07-24 18:07:10,533][05135] Loop batcher_evt_loop terminating... -[2024-07-24 18:07:10,535][05135] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... -[2024-07-24 18:07:10,547][03928] Component Batcher_0 stopped! -[2024-07-24 18:07:10,629][03928] Component RolloutWorker_w6 stopped! -[2024-07-24 18:07:10,629][05154] Stopping RolloutWorker_w6... -[2024-07-24 18:07:10,666][03928] Component RolloutWorker_w0 stopped! -[2024-07-24 18:07:10,652][05154] Loop rollout_proc6_evt_loop terminating... -[2024-07-24 18:07:10,670][05148] Weights refcount: 2 0 -[2024-07-24 18:07:10,665][05149] Stopping RolloutWorker_w0... -[2024-07-24 18:07:10,676][05149] Loop rollout_proc0_evt_loop terminating... -[2024-07-24 18:07:10,681][03928] Component InferenceWorker_p0-w0 stopped! -[2024-07-24 18:07:10,684][03928] Component RolloutWorker_w3 stopped! -[2024-07-24 18:07:10,681][05152] Stopping RolloutWorker_w3... -[2024-07-24 18:07:10,687][05152] Loop rollout_proc3_evt_loop terminating... -[2024-07-24 18:07:10,688][05148] Stopping InferenceWorker_p0-w0... -[2024-07-24 18:07:10,689][05148] Loop inference_proc0-0_evt_loop terminating... -[2024-07-24 18:07:10,696][03928] Component RolloutWorker_w7 stopped! -[2024-07-24 18:07:10,699][05155] Stopping RolloutWorker_w7... -[2024-07-24 18:07:10,714][03928] Component RolloutWorker_w4 stopped! -[2024-07-24 18:07:10,703][05155] Loop rollout_proc7_evt_loop terminating... -[2024-07-24 18:07:10,714][05153] Stopping RolloutWorker_w4... -[2024-07-24 18:07:10,726][05150] Stopping RolloutWorker_w1... -[2024-07-24 18:07:10,722][03928] Component RolloutWorker_w1 stopped! -[2024-07-24 18:07:10,730][05150] Loop rollout_proc1_evt_loop terminating... -[2024-07-24 18:07:10,721][05153] Loop rollout_proc4_evt_loop terminating... -[2024-07-24 18:07:10,733][03928] Component RolloutWorker_w5 stopped! -[2024-07-24 18:07:10,733][05156] Stopping RolloutWorker_w5... -[2024-07-24 18:07:10,740][05156] Loop rollout_proc5_evt_loop terminating... -[2024-07-24 18:07:10,746][05151] Stopping RolloutWorker_w2... -[2024-07-24 18:07:10,746][03928] Component RolloutWorker_w2 stopped! -[2024-07-24 18:07:10,749][05151] Loop rollout_proc2_evt_loop terminating... -[2024-07-24 18:07:10,760][05135] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000793_3248128.pth -[2024-07-24 18:07:10,779][05135] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... -[2024-07-24 18:07:10,957][05135] Stopping LearnerWorker_p0... -[2024-07-24 18:07:10,957][05135] Loop learner_proc0_evt_loop terminating... -[2024-07-24 18:07:10,956][03928] Component LearnerWorker_p0 stopped! -[2024-07-24 18:07:10,960][03928] Waiting for process learner_proc0 to stop... -[2024-07-24 18:07:12,362][03928] Waiting for process inference_proc0-0 to join... -[2024-07-24 18:07:12,365][03928] Waiting for process rollout_proc0 to join... -[2024-07-24 18:07:14,367][03928] Waiting for process rollout_proc1 to join... -[2024-07-24 18:07:14,370][03928] Waiting for process rollout_proc2 to join... -[2024-07-24 18:07:14,376][03928] Waiting for process rollout_proc3 to join... -[2024-07-24 18:07:14,379][03928] Waiting for process rollout_proc4 to join... -[2024-07-24 18:07:14,383][03928] Waiting for process rollout_proc5 to join... -[2024-07-24 18:07:14,388][03928] Waiting for process rollout_proc6 to join... -[2024-07-24 18:07:14,391][03928] Waiting for process rollout_proc7 to join... -[2024-07-24 18:07:14,394][03928] Batcher 0 profile tree view: -batching: 28.4688, releasing_batches: 0.0284 -[2024-07-24 18:07:14,398][03928] InferenceWorker_p0-w0 profile tree view: +[2024-07-24 19:00:50,719][05263] Using optimizer +[2024-07-24 19:00:51,485][05263] No checkpoints found +[2024-07-24 19:00:51,485][05263] Did not load from checkpoint, starting from scratch! +[2024-07-24 19:00:51,486][05263] Initialized policy 0 weights for model version 0 +[2024-07-24 19:00:51,489][05263] Using GPUs [0] for process 0 (actually maps to GPUs [0]) +[2024-07-24 19:00:51,503][05263] LearnerWorker_p0 finished initialization! +[2024-07-24 19:00:51,612][05276] RunningMeanStd input shape: (3, 72, 128) +[2024-07-24 19:00:51,613][05276] RunningMeanStd input shape: (1,) +[2024-07-24 19:00:51,627][05276] ConvEncoder: input_channels=3 +[2024-07-24 19:00:51,731][05276] Conv encoder output size: 512 +[2024-07-24 19:00:51,732][05276] Policy head output size: 512 +[2024-07-24 19:00:51,787][02885] Inference worker 0-0 is ready! +[2024-07-24 19:00:51,791][02885] All inference workers are ready! Signal rollout workers to start! +[2024-07-24 19:00:52,149][05278] Doom resolution: 160x120, resize resolution: (128, 72) +[2024-07-24 19:00:52,159][05280] Doom resolution: 160x120, resize resolution: (128, 72) +[2024-07-24 19:00:52,182][05282] Doom resolution: 160x120, resize resolution: (128, 72) +[2024-07-24 19:00:52,246][05283] Doom resolution: 160x120, resize resolution: (128, 72) +[2024-07-24 19:00:52,248][05279] Doom resolution: 160x120, resize resolution: (128, 72) +[2024-07-24 19:00:52,260][05284] Doom resolution: 160x120, resize resolution: (128, 72) +[2024-07-24 19:00:52,271][05281] Doom resolution: 160x120, resize resolution: (128, 72) +[2024-07-24 19:00:52,277][05277] Doom resolution: 160x120, resize resolution: (128, 72) +[2024-07-24 19:00:52,921][02885] Heartbeat connected on Batcher_0 +[2024-07-24 19:00:52,929][02885] Heartbeat connected on LearnerWorker_p0 +[2024-07-24 19:00:52,980][02885] Heartbeat connected on InferenceWorker_p0-w0 +[2024-07-24 19:00:53,635][02885] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) +[2024-07-24 19:00:54,705][05280] Decorrelating experience for 0 frames... +[2024-07-24 19:00:54,707][05284] Decorrelating experience for 0 frames... +[2024-07-24 19:00:54,708][05282] Decorrelating experience for 0 frames... +[2024-07-24 19:00:54,713][05278] Decorrelating experience for 0 frames... +[2024-07-24 19:00:54,711][05283] Decorrelating experience for 0 frames... +[2024-07-24 19:00:54,710][05277] Decorrelating experience for 0 frames... +[2024-07-24 19:00:54,712][05279] Decorrelating experience for 0 frames... +[2024-07-24 19:00:55,973][05282] Decorrelating experience for 32 frames... +[2024-07-24 19:00:55,975][05279] Decorrelating experience for 32 frames... +[2024-07-24 19:00:56,254][05280] Decorrelating experience for 32 frames... +[2024-07-24 19:00:56,257][05278] Decorrelating experience for 32 frames... +[2024-07-24 19:00:56,280][05277] Decorrelating experience for 32 frames... +[2024-07-24 19:00:57,699][05284] Decorrelating experience for 32 frames... +[2024-07-24 19:00:57,855][05281] Decorrelating experience for 0 frames... +[2024-07-24 19:00:57,958][05280] Decorrelating experience for 64 frames... +[2024-07-24 19:00:57,998][05283] Decorrelating experience for 32 frames... +[2024-07-24 19:00:58,240][05282] Decorrelating experience for 64 frames... +[2024-07-24 19:00:58,635][02885] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) +[2024-07-24 19:00:58,951][05279] Decorrelating experience for 64 frames... +[2024-07-24 19:00:59,410][05283] Decorrelating experience for 64 frames... +[2024-07-24 19:00:59,427][05282] Decorrelating experience for 96 frames... +[2024-07-24 19:00:59,477][05277] Decorrelating experience for 64 frames... +[2024-07-24 19:00:59,622][05280] Decorrelating experience for 96 frames... +[2024-07-24 19:00:59,621][05278] Decorrelating experience for 64 frames... +[2024-07-24 19:00:59,650][02885] Heartbeat connected on RolloutWorker_w5 +[2024-07-24 19:00:59,997][02885] Heartbeat connected on RolloutWorker_w4 +[2024-07-24 19:01:00,022][05284] Decorrelating experience for 64 frames... +[2024-07-24 19:01:00,580][05277] Decorrelating experience for 96 frames... +[2024-07-24 19:01:00,738][02885] Heartbeat connected on RolloutWorker_w0 +[2024-07-24 19:01:01,108][05281] Decorrelating experience for 32 frames... +[2024-07-24 19:01:01,171][05283] Decorrelating experience for 96 frames... +[2024-07-24 19:01:01,388][05279] Decorrelating experience for 96 frames... +[2024-07-24 19:01:01,789][05278] Decorrelating experience for 96 frames... +[2024-07-24 19:01:01,796][02885] Heartbeat connected on RolloutWorker_w7 +[2024-07-24 19:01:01,952][02885] Heartbeat connected on RolloutWorker_w1 +[2024-07-24 19:01:02,150][02885] Heartbeat connected on RolloutWorker_w2 +[2024-07-24 19:01:03,235][05284] Decorrelating experience for 96 frames... +[2024-07-24 19:01:03,636][02885] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 1.6. Samples: 16. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) +[2024-07-24 19:01:03,640][02885] Avg episode reward: [(0, '1.636')] +[2024-07-24 19:01:03,989][02885] Heartbeat connected on RolloutWorker_w6 +[2024-07-24 19:01:04,476][05281] Decorrelating experience for 64 frames... +[2024-07-24 19:01:05,336][05263] Signal inference workers to stop experience collection... +[2024-07-24 19:01:05,356][05276] InferenceWorker_p0-w0: stopping experience collection +[2024-07-24 19:01:05,653][05281] Decorrelating experience for 96 frames... +[2024-07-24 19:01:05,762][02885] Heartbeat connected on RolloutWorker_w3 +[2024-07-24 19:01:07,675][05263] Signal inference workers to resume experience collection... +[2024-07-24 19:01:07,681][05276] InferenceWorker_p0-w0: resuming experience collection +[2024-07-24 19:01:08,635][02885] Fps is (10 sec: 409.6, 60 sec: 273.1, 300 sec: 273.1). Total num frames: 4096. Throughput: 0: 165.1. Samples: 2476. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) +[2024-07-24 19:01:08,645][02885] Avg episode reward: [(0, '3.006')] +[2024-07-24 19:01:13,639][02885] Fps is (10 sec: 1637.9, 60 sec: 819.0, 300 sec: 819.0). Total num frames: 16384. Throughput: 0: 252.1. Samples: 5044. Policy #0 lag: (min: 0.0, avg: 1.2, max: 2.0) +[2024-07-24 19:01:13,641][02885] Avg episode reward: [(0, '3.488')] +[2024-07-24 19:01:18,635][02885] Fps is (10 sec: 3276.8, 60 sec: 1474.6, 300 sec: 1474.6). Total num frames: 36864. Throughput: 0: 283.0. Samples: 7074. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-07-24 19:01:18,637][02885] Avg episode reward: [(0, '3.901')] +[2024-07-24 19:01:19,394][05276] Updated weights for policy 0, policy_version 10 (0.0029) +[2024-07-24 19:01:23,635][02885] Fps is (10 sec: 3687.9, 60 sec: 1774.9, 300 sec: 1774.9). Total num frames: 53248. Throughput: 0: 439.5. Samples: 13186. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-07-24 19:01:23,638][02885] Avg episode reward: [(0, '4.391')] +[2024-07-24 19:01:28,636][02885] Fps is (10 sec: 3276.6, 60 sec: 1989.5, 300 sec: 1989.5). Total num frames: 69632. Throughput: 0: 511.0. Samples: 17886. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-07-24 19:01:28,640][02885] Avg episode reward: [(0, '4.462')] +[2024-07-24 19:01:32,918][05276] Updated weights for policy 0, policy_version 20 (0.0029) +[2024-07-24 19:01:33,635][02885] Fps is (10 sec: 2867.2, 60 sec: 2048.0, 300 sec: 2048.0). Total num frames: 81920. Throughput: 0: 494.0. Samples: 19762. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-07-24 19:01:33,642][02885] Avg episode reward: [(0, '4.469')] +[2024-07-24 19:01:38,635][02885] Fps is (10 sec: 3686.6, 60 sec: 2366.6, 300 sec: 2366.6). Total num frames: 106496. Throughput: 0: 562.3. Samples: 25304. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-07-24 19:01:38,643][02885] Avg episode reward: [(0, '4.501')] +[2024-07-24 19:01:38,656][05263] Saving new best policy, reward=4.501! +[2024-07-24 19:01:42,389][05276] Updated weights for policy 0, policy_version 30 (0.0037) +[2024-07-24 19:01:43,635][02885] Fps is (10 sec: 4505.6, 60 sec: 2539.5, 300 sec: 2539.5). Total num frames: 126976. Throughput: 0: 706.5. Samples: 31794. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-07-24 19:01:43,644][02885] Avg episode reward: [(0, '4.591')] +[2024-07-24 19:01:43,647][05263] Saving new best policy, reward=4.591! +[2024-07-24 19:01:48,638][02885] Fps is (10 sec: 3275.9, 60 sec: 2531.9, 300 sec: 2531.9). Total num frames: 139264. Throughput: 0: 750.8. Samples: 33804. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:01:48,645][02885] Avg episode reward: [(0, '4.535')] +[2024-07-24 19:01:53,635][02885] Fps is (10 sec: 3276.8, 60 sec: 2662.4, 300 sec: 2662.4). Total num frames: 159744. Throughput: 0: 803.1. Samples: 38614. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-07-24 19:01:53,637][02885] Avg episode reward: [(0, '4.483')] +[2024-07-24 19:01:54,522][05276] Updated weights for policy 0, policy_version 40 (0.0028) +[2024-07-24 19:01:58,635][02885] Fps is (10 sec: 4097.2, 60 sec: 3003.7, 300 sec: 2772.7). Total num frames: 180224. Throughput: 0: 894.7. Samples: 45300. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-07-24 19:01:58,637][02885] Avg episode reward: [(0, '4.361')] +[2024-07-24 19:02:03,637][02885] Fps is (10 sec: 3685.6, 60 sec: 3276.7, 300 sec: 2808.6). Total num frames: 196608. Throughput: 0: 914.4. Samples: 48222. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-07-24 19:02:03,639][02885] Avg episode reward: [(0, '4.306')] +[2024-07-24 19:02:06,142][05276] Updated weights for policy 0, policy_version 50 (0.0025) +[2024-07-24 19:02:08,635][02885] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 2839.9). Total num frames: 212992. Throughput: 0: 870.0. Samples: 52336. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-07-24 19:02:08,637][02885] Avg episode reward: [(0, '4.285')] +[2024-07-24 19:02:13,635][02885] Fps is (10 sec: 3687.1, 60 sec: 3618.4, 300 sec: 2918.4). Total num frames: 233472. Throughput: 0: 908.1. Samples: 58750. Policy #0 lag: (min: 0.0, avg: 0.3, max: 2.0) +[2024-07-24 19:02:13,638][02885] Avg episode reward: [(0, '4.294')] +[2024-07-24 19:02:15,774][05276] Updated weights for policy 0, policy_version 60 (0.0032) +[2024-07-24 19:02:18,635][02885] Fps is (10 sec: 4096.0, 60 sec: 3618.1, 300 sec: 2987.7). Total num frames: 253952. Throughput: 0: 941.2. Samples: 62118. Policy #0 lag: (min: 0.0, avg: 0.3, max: 2.0) +[2024-07-24 19:02:18,638][02885] Avg episode reward: [(0, '4.365')] +[2024-07-24 19:02:23,637][02885] Fps is (10 sec: 3276.2, 60 sec: 3549.8, 300 sec: 2958.2). Total num frames: 266240. Throughput: 0: 920.9. Samples: 66746. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-07-24 19:02:23,639][02885] Avg episode reward: [(0, '4.450')] +[2024-07-24 19:02:27,693][05276] Updated weights for policy 0, policy_version 70 (0.0023) +[2024-07-24 19:02:28,635][02885] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3061.2). Total num frames: 290816. Throughput: 0: 902.0. Samples: 72386. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-07-24 19:02:28,637][02885] Avg episode reward: [(0, '4.459')] +[2024-07-24 19:02:28,648][05263] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000071_290816.pth... +[2024-07-24 19:02:33,635][02885] Fps is (10 sec: 4506.3, 60 sec: 3822.9, 300 sec: 3112.9). Total num frames: 311296. Throughput: 0: 930.2. Samples: 75662. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-07-24 19:02:33,641][02885] Avg episode reward: [(0, '4.285')] +[2024-07-24 19:02:38,635][02885] Fps is (10 sec: 3276.7, 60 sec: 3618.1, 300 sec: 3081.7). Total num frames: 323584. Throughput: 0: 944.6. Samples: 81120. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-07-24 19:02:38,638][02885] Avg episode reward: [(0, '4.347')] +[2024-07-24 19:02:38,770][05276] Updated weights for policy 0, policy_version 80 (0.0019) +[2024-07-24 19:02:43,635][02885] Fps is (10 sec: 2867.2, 60 sec: 3549.9, 300 sec: 3090.6). Total num frames: 339968. Throughput: 0: 898.9. Samples: 85752. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-07-24 19:02:43,641][02885] Avg episode reward: [(0, '4.348')] +[2024-07-24 19:02:48,635][02885] Fps is (10 sec: 4096.0, 60 sec: 3754.8, 300 sec: 3169.9). Total num frames: 364544. Throughput: 0: 907.3. Samples: 89050. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:02:48,644][02885] Avg episode reward: [(0, '4.545')] +[2024-07-24 19:02:49,213][05276] Updated weights for policy 0, policy_version 90 (0.0037) +[2024-07-24 19:02:53,635][02885] Fps is (10 sec: 4096.1, 60 sec: 3686.4, 300 sec: 3174.4). Total num frames: 380928. Throughput: 0: 959.4. Samples: 95508. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-07-24 19:02:53,644][02885] Avg episode reward: [(0, '4.607')] +[2024-07-24 19:02:53,647][05263] Saving new best policy, reward=4.607! +[2024-07-24 19:02:58,635][02885] Fps is (10 sec: 3276.9, 60 sec: 3618.1, 300 sec: 3178.5). Total num frames: 397312. Throughput: 0: 905.5. Samples: 99496. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-07-24 19:02:58,637][02885] Avg episode reward: [(0, '4.548')] +[2024-07-24 19:03:01,162][05276] Updated weights for policy 0, policy_version 100 (0.0037) +[2024-07-24 19:03:03,635][02885] Fps is (10 sec: 3686.3, 60 sec: 3686.5, 300 sec: 3213.8). Total num frames: 417792. Throughput: 0: 898.3. Samples: 102542. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:03:03,644][02885] Avg episode reward: [(0, '4.482')] +[2024-07-24 19:03:08,637][02885] Fps is (10 sec: 4095.2, 60 sec: 3754.5, 300 sec: 3246.4). Total num frames: 438272. Throughput: 0: 941.9. Samples: 109130. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-07-24 19:03:08,640][02885] Avg episode reward: [(0, '4.401')] +[2024-07-24 19:03:11,596][05276] Updated weights for policy 0, policy_version 110 (0.0034) +[2024-07-24 19:03:13,635][02885] Fps is (10 sec: 3686.5, 60 sec: 3686.4, 300 sec: 3247.5). Total num frames: 454656. Throughput: 0: 920.8. Samples: 113822. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-07-24 19:03:13,639][02885] Avg episode reward: [(0, '4.449')] +[2024-07-24 19:03:18,635][02885] Fps is (10 sec: 3277.4, 60 sec: 3618.1, 300 sec: 3248.6). Total num frames: 471040. Throughput: 0: 895.3. Samples: 115950. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-07-24 19:03:18,638][02885] Avg episode reward: [(0, '4.500')] +[2024-07-24 19:03:22,440][05276] Updated weights for policy 0, policy_version 120 (0.0027) +[2024-07-24 19:03:23,635][02885] Fps is (10 sec: 4095.9, 60 sec: 3823.0, 300 sec: 3304.1). Total num frames: 495616. Throughput: 0: 920.9. Samples: 122562. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-07-24 19:03:23,637][02885] Avg episode reward: [(0, '4.457')] +[2024-07-24 19:03:28,635][02885] Fps is (10 sec: 4095.9, 60 sec: 3686.4, 300 sec: 3303.2). Total num frames: 512000. Throughput: 0: 944.0. Samples: 128234. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-07-24 19:03:28,641][02885] Avg episode reward: [(0, '4.466')] +[2024-07-24 19:03:33,635][02885] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3302.4). Total num frames: 528384. Throughput: 0: 915.4. Samples: 130242. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-07-24 19:03:33,642][02885] Avg episode reward: [(0, '4.670')] +[2024-07-24 19:03:33,645][05263] Saving new best policy, reward=4.670! +[2024-07-24 19:03:34,522][05276] Updated weights for policy 0, policy_version 130 (0.0020) +[2024-07-24 19:03:38,635][02885] Fps is (10 sec: 3686.5, 60 sec: 3754.7, 300 sec: 3326.4). Total num frames: 548864. Throughput: 0: 900.8. Samples: 136046. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:03:38,637][02885] Avg episode reward: [(0, '4.678')] +[2024-07-24 19:03:38,654][05263] Saving new best policy, reward=4.678! +[2024-07-24 19:03:43,640][02885] Fps is (10 sec: 4094.1, 60 sec: 3822.6, 300 sec: 3349.0). Total num frames: 569344. Throughput: 0: 959.8. Samples: 142692. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:03:43,642][02885] Avg episode reward: [(0, '4.675')] +[2024-07-24 19:03:44,654][05276] Updated weights for policy 0, policy_version 140 (0.0021) +[2024-07-24 19:03:48,641][02885] Fps is (10 sec: 3274.8, 60 sec: 3617.8, 300 sec: 3323.5). Total num frames: 581632. Throughput: 0: 937.2. Samples: 144720. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-07-24 19:03:48,647][02885] Avg episode reward: [(0, '4.669')] +[2024-07-24 19:03:53,635][02885] Fps is (10 sec: 3278.4, 60 sec: 3686.4, 300 sec: 3345.1). Total num frames: 602112. Throughput: 0: 896.0. Samples: 149450. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:03:53,638][02885] Avg episode reward: [(0, '4.539')] +[2024-07-24 19:03:55,950][05276] Updated weights for policy 0, policy_version 150 (0.0014) +[2024-07-24 19:03:58,635][02885] Fps is (10 sec: 4098.5, 60 sec: 3754.7, 300 sec: 3365.4). Total num frames: 622592. Throughput: 0: 940.0. Samples: 156122. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:03:58,638][02885] Avg episode reward: [(0, '4.640')] +[2024-07-24 19:04:03,635][02885] Fps is (10 sec: 3686.3, 60 sec: 3686.4, 300 sec: 3363.0). Total num frames: 638976. Throughput: 0: 960.0. Samples: 159152. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-07-24 19:04:03,638][02885] Avg episode reward: [(0, '4.783')] +[2024-07-24 19:04:03,643][05263] Saving new best policy, reward=4.783! +[2024-07-24 19:04:08,067][05276] Updated weights for policy 0, policy_version 160 (0.0031) +[2024-07-24 19:04:08,635][02885] Fps is (10 sec: 3276.8, 60 sec: 3618.2, 300 sec: 3360.8). Total num frames: 655360. Throughput: 0: 903.6. Samples: 163222. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-07-24 19:04:08,638][02885] Avg episode reward: [(0, '4.751')] +[2024-07-24 19:04:13,635][02885] Fps is (10 sec: 3686.5, 60 sec: 3686.4, 300 sec: 3379.2). Total num frames: 675840. Throughput: 0: 920.2. Samples: 169644. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-07-24 19:04:13,638][02885] Avg episode reward: [(0, '4.863')] +[2024-07-24 19:04:13,703][05263] Saving new best policy, reward=4.863! +[2024-07-24 19:04:17,474][05276] Updated weights for policy 0, policy_version 170 (0.0030) +[2024-07-24 19:04:18,635][02885] Fps is (10 sec: 4096.1, 60 sec: 3754.7, 300 sec: 3396.7). Total num frames: 696320. Throughput: 0: 943.7. Samples: 172708. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-07-24 19:04:18,638][02885] Avg episode reward: [(0, '5.076')] +[2024-07-24 19:04:18,645][05263] Saving new best policy, reward=5.076! +[2024-07-24 19:04:23,635][02885] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3393.8). Total num frames: 712704. Throughput: 0: 920.9. Samples: 177488. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-07-24 19:04:23,640][02885] Avg episode reward: [(0, '4.977')] +[2024-07-24 19:04:28,635][02885] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3391.1). Total num frames: 729088. Throughput: 0: 894.2. Samples: 182928. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-07-24 19:04:28,641][02885] Avg episode reward: [(0, '5.188')] +[2024-07-24 19:04:28,672][05263] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000179_733184.pth... +[2024-07-24 19:04:28,801][05263] Saving new best policy, reward=5.188! +[2024-07-24 19:04:29,787][05276] Updated weights for policy 0, policy_version 180 (0.0030) +[2024-07-24 19:04:33,635][02885] Fps is (10 sec: 4096.0, 60 sec: 3754.7, 300 sec: 3425.7). Total num frames: 753664. Throughput: 0: 919.6. Samples: 186098. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:04:33,642][02885] Avg episode reward: [(0, '5.170')] +[2024-07-24 19:04:38,635][02885] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3422.4). Total num frames: 770048. Throughput: 0: 941.6. Samples: 191820. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-07-24 19:04:38,643][02885] Avg episode reward: [(0, '5.347')] +[2024-07-24 19:04:38,655][05263] Saving new best policy, reward=5.347! +[2024-07-24 19:04:41,588][05276] Updated weights for policy 0, policy_version 190 (0.0025) +[2024-07-24 19:04:43,639][02885] Fps is (10 sec: 3275.5, 60 sec: 3618.2, 300 sec: 3419.2). Total num frames: 786432. Throughput: 0: 893.5. Samples: 196334. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-07-24 19:04:43,641][02885] Avg episode reward: [(0, '5.545')] +[2024-07-24 19:04:43,645][05263] Saving new best policy, reward=5.545! +[2024-07-24 19:04:48,635][02885] Fps is (10 sec: 3686.4, 60 sec: 3755.0, 300 sec: 3433.7). Total num frames: 806912. Throughput: 0: 895.7. Samples: 199458. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:04:48,643][02885] Avg episode reward: [(0, '5.754')] +[2024-07-24 19:04:48,652][05263] Saving new best policy, reward=5.754! +[2024-07-24 19:04:51,251][05276] Updated weights for policy 0, policy_version 200 (0.0038) +[2024-07-24 19:04:53,635][02885] Fps is (10 sec: 3687.8, 60 sec: 3686.4, 300 sec: 3430.4). Total num frames: 823296. Throughput: 0: 951.6. Samples: 206042. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:04:53,637][02885] Avg episode reward: [(0, '6.051')] +[2024-07-24 19:04:53,700][05263] Saving new best policy, reward=6.051! +[2024-07-24 19:04:58,635][02885] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3427.3). Total num frames: 839680. Throughput: 0: 899.7. Samples: 210132. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-07-24 19:04:58,638][02885] Avg episode reward: [(0, '5.924')] +[2024-07-24 19:05:03,305][05276] Updated weights for policy 0, policy_version 210 (0.0030) +[2024-07-24 19:05:03,635][02885] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3440.6). Total num frames: 860160. Throughput: 0: 890.4. Samples: 212774. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-07-24 19:05:03,639][02885] Avg episode reward: [(0, '5.694')] +[2024-07-24 19:05:08,635][02885] Fps is (10 sec: 4096.0, 60 sec: 3754.7, 300 sec: 3453.5). Total num frames: 880640. Throughput: 0: 934.2. Samples: 219526. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:05:08,638][02885] Avg episode reward: [(0, '6.067')] +[2024-07-24 19:05:08,645][05263] Saving new best policy, reward=6.067! +[2024-07-24 19:05:13,635][02885] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3450.1). Total num frames: 897024. Throughput: 0: 924.2. Samples: 224518. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:05:13,637][02885] Avg episode reward: [(0, '6.120')] +[2024-07-24 19:05:13,645][05263] Saving new best policy, reward=6.120! +[2024-07-24 19:05:14,901][05276] Updated weights for policy 0, policy_version 220 (0.0040) +[2024-07-24 19:05:18,635][02885] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3446.8). Total num frames: 913408. Throughput: 0: 897.5. Samples: 226484. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:05:18,644][02885] Avg episode reward: [(0, '6.464')] +[2024-07-24 19:05:18,656][05263] Saving new best policy, reward=6.464! +[2024-07-24 19:05:23,635][02885] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3458.8). Total num frames: 933888. Throughput: 0: 909.2. Samples: 232736. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:05:23,642][02885] Avg episode reward: [(0, '6.678')] +[2024-07-24 19:05:23,645][05263] Saving new best policy, reward=6.678! +[2024-07-24 19:05:24,894][05276] Updated weights for policy 0, policy_version 230 (0.0029) +[2024-07-24 19:05:28,635][02885] Fps is (10 sec: 4095.9, 60 sec: 3754.7, 300 sec: 3470.4). Total num frames: 954368. Throughput: 0: 943.6. Samples: 238792. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:05:28,647][02885] Avg episode reward: [(0, '6.206')] +[2024-07-24 19:05:33,635][02885] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3452.3). Total num frames: 966656. Throughput: 0: 918.3. Samples: 240782. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-07-24 19:05:33,644][02885] Avg episode reward: [(0, '6.393')] +[2024-07-24 19:05:36,933][05276] Updated weights for policy 0, policy_version 240 (0.0027) +[2024-07-24 19:05:38,635][02885] Fps is (10 sec: 3276.9, 60 sec: 3618.1, 300 sec: 3463.6). Total num frames: 987136. Throughput: 0: 895.6. Samples: 246342. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:05:38,642][02885] Avg episode reward: [(0, '6.489')] +[2024-07-24 19:05:43,635][02885] Fps is (10 sec: 4505.5, 60 sec: 3754.9, 300 sec: 3488.7). Total num frames: 1011712. Throughput: 0: 951.5. Samples: 252950. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-07-24 19:05:43,642][02885] Avg episode reward: [(0, '6.765')] +[2024-07-24 19:05:43,649][05263] Saving new best policy, reward=6.765! +[2024-07-24 19:05:47,982][05276] Updated weights for policy 0, policy_version 250 (0.0026) +[2024-07-24 19:05:48,635][02885] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3471.2). Total num frames: 1024000. Throughput: 0: 943.7. Samples: 255242. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:05:48,641][02885] Avg episode reward: [(0, '6.583')] +[2024-07-24 19:05:53,635][02885] Fps is (10 sec: 2867.2, 60 sec: 3618.1, 300 sec: 3526.7). Total num frames: 1040384. Throughput: 0: 891.1. Samples: 259626. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-07-24 19:05:53,638][02885] Avg episode reward: [(0, '6.849')] +[2024-07-24 19:05:53,642][05263] Saving new best policy, reward=6.849! +[2024-07-24 19:05:58,339][05276] Updated weights for policy 0, policy_version 260 (0.0030) +[2024-07-24 19:05:58,635][02885] Fps is (10 sec: 4095.9, 60 sec: 3754.7, 300 sec: 3610.0). Total num frames: 1064960. Throughput: 0: 924.9. Samples: 266140. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-07-24 19:05:58,641][02885] Avg episode reward: [(0, '7.340')] +[2024-07-24 19:05:58,657][05263] Saving new best policy, reward=7.340! +[2024-07-24 19:06:03,639][02885] Fps is (10 sec: 4094.4, 60 sec: 3686.2, 300 sec: 3651.6). Total num frames: 1081344. Throughput: 0: 953.7. Samples: 269404. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-07-24 19:06:03,647][02885] Avg episode reward: [(0, '7.908')] +[2024-07-24 19:06:03,650][05263] Saving new best policy, reward=7.908! +[2024-07-24 19:06:08,635][02885] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3665.6). Total num frames: 1097728. Throughput: 0: 907.4. Samples: 273570. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-07-24 19:06:08,638][02885] Avg episode reward: [(0, '8.361')] +[2024-07-24 19:06:08,649][05263] Saving new best policy, reward=8.361! +[2024-07-24 19:06:10,463][05276] Updated weights for policy 0, policy_version 270 (0.0028) +[2024-07-24 19:06:13,635][02885] Fps is (10 sec: 3687.8, 60 sec: 3686.4, 300 sec: 3665.6). Total num frames: 1118208. Throughput: 0: 905.2. Samples: 279524. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:06:13,637][02885] Avg episode reward: [(0, '8.412')] +[2024-07-24 19:06:13,640][05263] Saving new best policy, reward=8.412! +[2024-07-24 19:06:18,635][02885] Fps is (10 sec: 4096.1, 60 sec: 3754.7, 300 sec: 3679.5). Total num frames: 1138688. Throughput: 0: 927.8. Samples: 282532. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-07-24 19:06:18,644][02885] Avg episode reward: [(0, '7.957')] +[2024-07-24 19:06:21,583][05276] Updated weights for policy 0, policy_version 280 (0.0018) +[2024-07-24 19:06:23,635][02885] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3665.6). Total num frames: 1150976. Throughput: 0: 912.9. Samples: 287424. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-07-24 19:06:23,638][02885] Avg episode reward: [(0, '8.225')] +[2024-07-24 19:06:28,635][02885] Fps is (10 sec: 2867.2, 60 sec: 3549.9, 300 sec: 3679.5). Total num frames: 1167360. Throughput: 0: 876.3. Samples: 292384. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-07-24 19:06:28,640][02885] Avg episode reward: [(0, '9.086')] +[2024-07-24 19:06:28,742][05263] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000286_1171456.pth... +[2024-07-24 19:06:28,877][05263] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000071_290816.pth +[2024-07-24 19:06:28,896][05263] Saving new best policy, reward=9.086! +[2024-07-24 19:06:32,734][05276] Updated weights for policy 0, policy_version 290 (0.0019) +[2024-07-24 19:06:33,635][02885] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3665.6). Total num frames: 1187840. Throughput: 0: 892.2. Samples: 295390. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-07-24 19:06:33,637][02885] Avg episode reward: [(0, '8.912')] +[2024-07-24 19:06:38,636][02885] Fps is (10 sec: 3686.1, 60 sec: 3618.1, 300 sec: 3651.7). Total num frames: 1204224. Throughput: 0: 925.0. Samples: 301252. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:06:38,641][02885] Avg episode reward: [(0, '8.902')] +[2024-07-24 19:06:43,635][02885] Fps is (10 sec: 3276.7, 60 sec: 3481.6, 300 sec: 3665.6). Total num frames: 1220608. Throughput: 0: 869.7. Samples: 305278. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:06:43,638][02885] Avg episode reward: [(0, '9.065')] +[2024-07-24 19:06:45,365][05276] Updated weights for policy 0, policy_version 300 (0.0016) +[2024-07-24 19:06:48,635][02885] Fps is (10 sec: 3686.8, 60 sec: 3618.1, 300 sec: 3665.6). Total num frames: 1241088. Throughput: 0: 863.1. Samples: 308242. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-07-24 19:06:48,637][02885] Avg episode reward: [(0, '9.563')] +[2024-07-24 19:06:48,647][05263] Saving new best policy, reward=9.563! +[2024-07-24 19:06:53,635][02885] Fps is (10 sec: 3686.5, 60 sec: 3618.1, 300 sec: 3651.7). Total num frames: 1257472. Throughput: 0: 902.9. Samples: 314202. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:06:53,643][02885] Avg episode reward: [(0, '9.847')] +[2024-07-24 19:06:53,649][05263] Saving new best policy, reward=9.847! +[2024-07-24 19:06:57,476][05276] Updated weights for policy 0, policy_version 310 (0.0031) +[2024-07-24 19:06:58,635][02885] Fps is (10 sec: 2867.2, 60 sec: 3413.3, 300 sec: 3637.8). Total num frames: 1269760. Throughput: 0: 856.1. Samples: 318048. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-07-24 19:06:58,637][02885] Avg episode reward: [(0, '9.700')] +[2024-07-24 19:07:03,635][02885] Fps is (10 sec: 3276.8, 60 sec: 3481.8, 300 sec: 3651.7). Total num frames: 1290240. Throughput: 0: 841.1. Samples: 320382. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-07-24 19:07:03,642][02885] Avg episode reward: [(0, '9.034')] +[2024-07-24 19:07:08,635][02885] Fps is (10 sec: 3686.4, 60 sec: 3481.6, 300 sec: 3637.8). Total num frames: 1306624. Throughput: 0: 865.7. Samples: 326382. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:07:08,647][02885] Avg episode reward: [(0, '9.043')] +[2024-07-24 19:07:08,685][05276] Updated weights for policy 0, policy_version 320 (0.0030) +[2024-07-24 19:07:13,635][02885] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 3623.9). Total num frames: 1323008. Throughput: 0: 865.3. Samples: 331322. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-07-24 19:07:13,640][02885] Avg episode reward: [(0, '9.209')] +[2024-07-24 19:07:18,635][02885] Fps is (10 sec: 2867.2, 60 sec: 3276.8, 300 sec: 3623.9). Total num frames: 1335296. Throughput: 0: 840.0. Samples: 333188. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-07-24 19:07:18,638][02885] Avg episode reward: [(0, '10.245')] +[2024-07-24 19:07:18,648][05263] Saving new best policy, reward=10.245! +[2024-07-24 19:07:21,602][05276] Updated weights for policy 0, policy_version 330 (0.0027) +[2024-07-24 19:07:23,635][02885] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 3610.0). Total num frames: 1355776. Throughput: 0: 831.3. Samples: 338658. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-07-24 19:07:23,637][02885] Avg episode reward: [(0, '10.631')] +[2024-07-24 19:07:23,714][05263] Saving new best policy, reward=10.631! +[2024-07-24 19:07:28,637][02885] Fps is (10 sec: 4095.1, 60 sec: 3481.5, 300 sec: 3610.0). Total num frames: 1376256. Throughput: 0: 878.7. Samples: 344822. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:07:28,641][02885] Avg episode reward: [(0, '11.530')] +[2024-07-24 19:07:28,651][05263] Saving new best policy, reward=11.530! +[2024-07-24 19:07:33,635][02885] Fps is (10 sec: 3276.8, 60 sec: 3345.1, 300 sec: 3610.0). Total num frames: 1388544. Throughput: 0: 854.0. Samples: 346674. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:07:33,638][02885] Avg episode reward: [(0, '12.418')] +[2024-07-24 19:07:33,642][05263] Saving new best policy, reward=12.418! +[2024-07-24 19:07:33,976][05276] Updated weights for policy 0, policy_version 340 (0.0018) +[2024-07-24 19:07:38,635][02885] Fps is (10 sec: 3277.5, 60 sec: 3413.4, 300 sec: 3623.9). Total num frames: 1409024. Throughput: 0: 831.7. Samples: 351628. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:07:38,640][02885] Avg episode reward: [(0, '12.641')] +[2024-07-24 19:07:38,652][05263] Saving new best policy, reward=12.641! +[2024-07-24 19:07:43,625][05276] Updated weights for policy 0, policy_version 350 (0.0019) +[2024-07-24 19:07:43,635][02885] Fps is (10 sec: 4505.6, 60 sec: 3549.9, 300 sec: 3623.9). Total num frames: 1433600. Throughput: 0: 889.4. Samples: 358070. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:07:43,639][02885] Avg episode reward: [(0, '13.516')] +[2024-07-24 19:07:43,641][05263] Saving new best policy, reward=13.516! +[2024-07-24 19:07:48,635][02885] Fps is (10 sec: 3686.4, 60 sec: 3413.3, 300 sec: 3610.0). Total num frames: 1445888. Throughput: 0: 901.3. Samples: 360940. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:07:48,643][02885] Avg episode reward: [(0, '12.950')] +[2024-07-24 19:07:53,635][02885] Fps is (10 sec: 2867.2, 60 sec: 3413.3, 300 sec: 3610.0). Total num frames: 1462272. Throughput: 0: 857.0. Samples: 364948. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:07:53,640][02885] Avg episode reward: [(0, '12.262')] +[2024-07-24 19:07:55,806][05276] Updated weights for policy 0, policy_version 360 (0.0027) +[2024-07-24 19:07:58,635][02885] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3610.0). Total num frames: 1482752. Throughput: 0: 889.1. Samples: 371330. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-07-24 19:07:58,638][02885] Avg episode reward: [(0, '12.085')] +[2024-07-24 19:08:03,635][02885] Fps is (10 sec: 4096.0, 60 sec: 3549.9, 300 sec: 3610.1). Total num frames: 1503232. Throughput: 0: 921.2. Samples: 374644. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-07-24 19:08:03,642][02885] Avg episode reward: [(0, '12.483')] +[2024-07-24 19:08:07,197][05276] Updated weights for policy 0, policy_version 370 (0.0019) +[2024-07-24 19:08:08,635][02885] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3610.0). Total num frames: 1519616. Throughput: 0: 899.2. Samples: 379120. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-07-24 19:08:08,641][02885] Avg episode reward: [(0, '13.015')] +[2024-07-24 19:08:13,635][02885] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3623.9). Total num frames: 1540096. Throughput: 0: 885.3. Samples: 384660. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:08:13,638][02885] Avg episode reward: [(0, '13.185')] +[2024-07-24 19:08:17,390][05276] Updated weights for policy 0, policy_version 380 (0.0026) +[2024-07-24 19:08:18,635][02885] Fps is (10 sec: 4096.0, 60 sec: 3754.7, 300 sec: 3610.0). Total num frames: 1560576. Throughput: 0: 918.7. Samples: 388014. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-07-24 19:08:18,640][02885] Avg episode reward: [(0, '12.609')] +[2024-07-24 19:08:23,635][02885] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3596.2). Total num frames: 1572864. Throughput: 0: 932.6. Samples: 393596. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:08:23,642][02885] Avg episode reward: [(0, '12.087')] +[2024-07-24 19:08:28,635][02885] Fps is (10 sec: 3276.8, 60 sec: 3618.3, 300 sec: 3610.0). Total num frames: 1593344. Throughput: 0: 892.0. Samples: 398210. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:08:28,639][02885] Avg episode reward: [(0, '11.816')] +[2024-07-24 19:08:28,663][05263] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000389_1593344.pth... +[2024-07-24 19:08:28,786][05263] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000179_733184.pth +[2024-07-24 19:08:29,356][05276] Updated weights for policy 0, policy_version 390 (0.0029) +[2024-07-24 19:08:33,635][02885] Fps is (10 sec: 4096.0, 60 sec: 3754.7, 300 sec: 3610.0). Total num frames: 1613824. Throughput: 0: 900.5. Samples: 401462. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-07-24 19:08:33,641][02885] Avg episode reward: [(0, '12.589')] +[2024-07-24 19:08:38,635][02885] Fps is (10 sec: 4096.0, 60 sec: 3754.7, 300 sec: 3610.1). Total num frames: 1634304. Throughput: 0: 956.1. Samples: 407974. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-07-24 19:08:38,637][02885] Avg episode reward: [(0, '12.170')] +[2024-07-24 19:08:39,966][05276] Updated weights for policy 0, policy_version 400 (0.0046) +[2024-07-24 19:08:43,635][02885] Fps is (10 sec: 3276.7, 60 sec: 3549.8, 300 sec: 3610.1). Total num frames: 1646592. Throughput: 0: 905.8. Samples: 412092. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-07-24 19:08:43,642][02885] Avg episode reward: [(0, '12.007')] +[2024-07-24 19:08:48,635][02885] Fps is (10 sec: 3276.8, 60 sec: 3686.4, 300 sec: 3610.0). Total num frames: 1667072. Throughput: 0: 899.7. Samples: 415132. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:08:48,642][02885] Avg episode reward: [(0, '11.975')] +[2024-07-24 19:08:50,627][05276] Updated weights for policy 0, policy_version 410 (0.0028) +[2024-07-24 19:08:53,635][02885] Fps is (10 sec: 4505.8, 60 sec: 3822.9, 300 sec: 3623.9). Total num frames: 1691648. Throughput: 0: 944.7. Samples: 421630. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:08:53,640][02885] Avg episode reward: [(0, '12.036')] +[2024-07-24 19:08:58,644][02885] Fps is (10 sec: 3683.0, 60 sec: 3685.8, 300 sec: 3609.9). Total num frames: 1703936. Throughput: 0: 929.2. Samples: 426484. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-07-24 19:08:58,652][02885] Avg episode reward: [(0, '13.080')] +[2024-07-24 19:09:02,713][05276] Updated weights for policy 0, policy_version 420 (0.0040) +[2024-07-24 19:09:03,636][02885] Fps is (10 sec: 2867.0, 60 sec: 3618.1, 300 sec: 3610.0). Total num frames: 1720320. Throughput: 0: 899.2. Samples: 428478. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-07-24 19:09:03,641][02885] Avg episode reward: [(0, '14.214')] +[2024-07-24 19:09:03,643][05263] Saving new best policy, reward=14.214! +[2024-07-24 19:09:08,635][02885] Fps is (10 sec: 4099.8, 60 sec: 3754.7, 300 sec: 3623.9). Total num frames: 1744896. Throughput: 0: 913.8. Samples: 434716. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:09:08,640][02885] Avg episode reward: [(0, '15.723')] +[2024-07-24 19:09:08,648][05263] Saving new best policy, reward=15.723! +[2024-07-24 19:09:13,635][02885] Fps is (10 sec: 3686.6, 60 sec: 3618.1, 300 sec: 3596.1). Total num frames: 1757184. Throughput: 0: 935.1. Samples: 440288. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:09:13,638][02885] Avg episode reward: [(0, '16.336')] +[2024-07-24 19:09:13,640][05263] Saving new best policy, reward=16.336! +[2024-07-24 19:09:13,952][05276] Updated weights for policy 0, policy_version 430 (0.0028) +[2024-07-24 19:09:18,635][02885] Fps is (10 sec: 2867.2, 60 sec: 3549.9, 300 sec: 3596.1). Total num frames: 1773568. Throughput: 0: 904.5. Samples: 442164. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:09:18,638][02885] Avg episode reward: [(0, '15.848')] +[2024-07-24 19:09:23,635][02885] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3610.0). Total num frames: 1794048. Throughput: 0: 878.4. Samples: 447504. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:09:23,638][02885] Avg episode reward: [(0, '16.002')] +[2024-07-24 19:09:25,221][05276] Updated weights for policy 0, policy_version 440 (0.0042) +[2024-07-24 19:09:28,635][02885] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3596.1). Total num frames: 1814528. Throughput: 0: 933.9. Samples: 454118. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:09:28,641][02885] Avg episode reward: [(0, '15.808')] +[2024-07-24 19:09:33,636][02885] Fps is (10 sec: 3276.5, 60 sec: 3549.8, 300 sec: 3582.3). Total num frames: 1826816. Throughput: 0: 914.2. Samples: 456272. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:09:33,638][02885] Avg episode reward: [(0, '14.954')] +[2024-07-24 19:09:36,936][05276] Updated weights for policy 0, policy_version 450 (0.0028) +[2024-07-24 19:09:38,635][02885] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3596.2). Total num frames: 1847296. Throughput: 0: 878.4. Samples: 461156. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-07-24 19:09:38,637][02885] Avg episode reward: [(0, '14.885')] +[2024-07-24 19:09:43,635][02885] Fps is (10 sec: 4506.0, 60 sec: 3754.7, 300 sec: 3610.0). Total num frames: 1871872. Throughput: 0: 919.5. Samples: 467854. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:09:43,637][02885] Avg episode reward: [(0, '14.594')] +[2024-07-24 19:09:46,803][05276] Updated weights for policy 0, policy_version 460 (0.0013) +[2024-07-24 19:09:48,636][02885] Fps is (10 sec: 4095.5, 60 sec: 3686.3, 300 sec: 3610.0). Total num frames: 1888256. Throughput: 0: 942.5. Samples: 470890. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:09:48,643][02885] Avg episode reward: [(0, '13.417')] +[2024-07-24 19:09:53,635][02885] Fps is (10 sec: 2867.2, 60 sec: 3481.6, 300 sec: 3596.1). Total num frames: 1900544. Throughput: 0: 889.7. Samples: 474752. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-07-24 19:09:53,643][02885] Avg episode reward: [(0, '13.898')] +[2024-07-24 19:09:58,499][05276] Updated weights for policy 0, policy_version 470 (0.0017) +[2024-07-24 19:09:58,635][02885] Fps is (10 sec: 3686.8, 60 sec: 3687.0, 300 sec: 3610.0). Total num frames: 1925120. Throughput: 0: 906.5. Samples: 481080. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-07-24 19:09:58,644][02885] Avg episode reward: [(0, '14.036')] +[2024-07-24 19:10:03,635][02885] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3596.1). Total num frames: 1941504. Throughput: 0: 936.4. Samples: 484300. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:10:03,638][02885] Avg episode reward: [(0, '15.425')] +[2024-07-24 19:10:08,635][02885] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3596.1). Total num frames: 1957888. Throughput: 0: 920.0. Samples: 488902. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:10:08,643][02885] Avg episode reward: [(0, '15.407')] +[2024-07-24 19:10:10,821][05276] Updated weights for policy 0, policy_version 480 (0.0029) +[2024-07-24 19:10:13,635][02885] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3610.0). Total num frames: 1978368. Throughput: 0: 892.0. Samples: 494258. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:10:13,637][02885] Avg episode reward: [(0, '16.655')] +[2024-07-24 19:10:13,641][05263] Saving new best policy, reward=16.655! +[2024-07-24 19:10:18,635][02885] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3596.1). Total num frames: 1994752. Throughput: 0: 913.7. Samples: 497386. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-07-24 19:10:18,637][02885] Avg episode reward: [(0, '17.423')] +[2024-07-24 19:10:18,678][05263] Saving new best policy, reward=17.423! +[2024-07-24 19:10:21,254][05276] Updated weights for policy 0, policy_version 490 (0.0017) +[2024-07-24 19:10:23,637][02885] Fps is (10 sec: 3276.3, 60 sec: 3618.0, 300 sec: 3582.2). Total num frames: 2011136. Throughput: 0: 923.7. Samples: 502724. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:10:23,639][02885] Avg episode reward: [(0, '17.974')] +[2024-07-24 19:10:23,642][05263] Saving new best policy, reward=17.974! +[2024-07-24 19:10:28,635][02885] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3596.1). Total num frames: 2027520. Throughput: 0: 871.5. Samples: 507072. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:10:28,644][02885] Avg episode reward: [(0, '17.706')] +[2024-07-24 19:10:28,657][05263] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000495_2027520.pth... +[2024-07-24 19:10:28,801][05263] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000286_1171456.pth +[2024-07-24 19:10:32,995][05276] Updated weights for policy 0, policy_version 500 (0.0041) +[2024-07-24 19:10:33,635][02885] Fps is (10 sec: 3687.0, 60 sec: 3686.5, 300 sec: 3596.1). Total num frames: 2048000. Throughput: 0: 873.6. Samples: 510202. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:10:33,637][02885] Avg episode reward: [(0, '16.992')] +[2024-07-24 19:10:38,635][02885] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3582.3). Total num frames: 2068480. Throughput: 0: 931.7. Samples: 516678. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:10:38,638][02885] Avg episode reward: [(0, '16.003')] +[2024-07-24 19:10:43,635][02885] Fps is (10 sec: 3276.7, 60 sec: 3481.6, 300 sec: 3582.3). Total num frames: 2080768. Throughput: 0: 881.1. Samples: 520732. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:10:43,640][02885] Avg episode reward: [(0, '15.598')] +[2024-07-24 19:10:45,110][05276] Updated weights for policy 0, policy_version 510 (0.0023) +[2024-07-24 19:10:48,635][02885] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3596.1). Total num frames: 2101248. Throughput: 0: 873.6. Samples: 523612. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:10:48,638][02885] Avg episode reward: [(0, '15.392')] +[2024-07-24 19:10:53,635][02885] Fps is (10 sec: 4096.2, 60 sec: 3686.4, 300 sec: 3582.3). Total num frames: 2121728. Throughput: 0: 912.0. Samples: 529940. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:10:53,643][02885] Avg episode reward: [(0, '16.098')] +[2024-07-24 19:10:55,182][05276] Updated weights for policy 0, policy_version 520 (0.0024) +[2024-07-24 19:10:58,635][02885] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3582.3). Total num frames: 2138112. Throughput: 0: 900.7. Samples: 534790. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:10:58,639][02885] Avg episode reward: [(0, '15.426')] +[2024-07-24 19:11:03,635][02885] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3582.3). Total num frames: 2154496. Throughput: 0: 876.1. Samples: 536810. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:11:03,638][02885] Avg episode reward: [(0, '15.869')] +[2024-07-24 19:11:06,932][05276] Updated weights for policy 0, policy_version 530 (0.0033) +[2024-07-24 19:11:08,635][02885] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3582.3). Total num frames: 2174976. Throughput: 0: 899.8. Samples: 543212. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:11:08,638][02885] Avg episode reward: [(0, '15.887')] +[2024-07-24 19:11:13,635][02885] Fps is (10 sec: 4096.0, 60 sec: 3618.1, 300 sec: 3582.3). Total num frames: 2195456. Throughput: 0: 935.3. Samples: 549160. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:11:13,639][02885] Avg episode reward: [(0, '15.765')] +[2024-07-24 19:11:18,635][02885] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3582.3). Total num frames: 2207744. Throughput: 0: 908.1. Samples: 551066. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:11:18,637][02885] Avg episode reward: [(0, '15.782')] +[2024-07-24 19:11:19,216][05276] Updated weights for policy 0, policy_version 540 (0.0029) +[2024-07-24 19:11:23,635][02885] Fps is (10 sec: 3276.8, 60 sec: 3618.2, 300 sec: 3596.2). Total num frames: 2228224. Throughput: 0: 885.5. Samples: 556526. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-07-24 19:11:23,637][02885] Avg episode reward: [(0, '16.210')] +[2024-07-24 19:11:28,636][02885] Fps is (10 sec: 4095.6, 60 sec: 3686.3, 300 sec: 3596.1). Total num frames: 2248704. Throughput: 0: 938.9. Samples: 562982. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-07-24 19:11:28,638][02885] Avg episode reward: [(0, '15.603')] +[2024-07-24 19:11:28,839][05276] Updated weights for policy 0, policy_version 550 (0.0034) +[2024-07-24 19:11:33,635][02885] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3596.2). Total num frames: 2265088. Throughput: 0: 923.5. Samples: 565170. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-07-24 19:11:33,642][02885] Avg episode reward: [(0, '15.352')] +[2024-07-24 19:11:38,635][02885] Fps is (10 sec: 3277.1, 60 sec: 3549.9, 300 sec: 3596.2). Total num frames: 2281472. Throughput: 0: 885.3. Samples: 569780. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:11:38,638][02885] Avg episode reward: [(0, '15.446')] +[2024-07-24 19:11:40,780][05276] Updated weights for policy 0, policy_version 560 (0.0030) +[2024-07-24 19:11:43,635][02885] Fps is (10 sec: 4096.0, 60 sec: 3754.7, 300 sec: 3610.0). Total num frames: 2306048. Throughput: 0: 921.1. Samples: 576240. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:11:43,637][02885] Avg episode reward: [(0, '15.976')] +[2024-07-24 19:11:48,635][02885] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3610.0). Total num frames: 2322432. Throughput: 0: 946.7. Samples: 579410. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:11:48,646][02885] Avg episode reward: [(0, '15.645')] +[2024-07-24 19:11:52,974][05276] Updated weights for policy 0, policy_version 570 (0.0027) +[2024-07-24 19:11:53,635][02885] Fps is (10 sec: 2867.2, 60 sec: 3549.9, 300 sec: 3610.0). Total num frames: 2334720. Throughput: 0: 891.7. Samples: 583338. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:11:53,640][02885] Avg episode reward: [(0, '17.462')] +[2024-07-24 19:11:58,635][02885] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3610.0). Total num frames: 2355200. Throughput: 0: 893.5. Samples: 589366. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:11:58,642][02885] Avg episode reward: [(0, '17.984')] +[2024-07-24 19:11:58,725][05263] Saving new best policy, reward=17.984! +[2024-07-24 19:12:02,589][05276] Updated weights for policy 0, policy_version 580 (0.0023) +[2024-07-24 19:12:03,635][02885] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3623.9). Total num frames: 2375680. Throughput: 0: 920.3. Samples: 592478. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:12:03,638][02885] Avg episode reward: [(0, '17.287')] +[2024-07-24 19:12:08,637][02885] Fps is (10 sec: 3685.7, 60 sec: 3618.0, 300 sec: 3623.9). Total num frames: 2392064. Throughput: 0: 909.6. Samples: 597460. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-07-24 19:12:08,639][02885] Avg episode reward: [(0, '17.543')] +[2024-07-24 19:12:13,635][02885] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3637.8). Total num frames: 2408448. Throughput: 0: 880.0. Samples: 602580. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-07-24 19:12:13,642][02885] Avg episode reward: [(0, '16.879')] +[2024-07-24 19:12:14,788][05276] Updated weights for policy 0, policy_version 590 (0.0032) +[2024-07-24 19:12:18,635][02885] Fps is (10 sec: 4096.8, 60 sec: 3754.7, 300 sec: 3651.7). Total num frames: 2433024. Throughput: 0: 903.7. Samples: 605838. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:12:18,637][02885] Avg episode reward: [(0, '16.983')] +[2024-07-24 19:12:23,635][02885] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3623.9). Total num frames: 2445312. Throughput: 0: 930.6. Samples: 611656. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:12:23,639][02885] Avg episode reward: [(0, '17.198')] +[2024-07-24 19:12:26,777][05276] Updated weights for policy 0, policy_version 600 (0.0027) +[2024-07-24 19:12:28,635][02885] Fps is (10 sec: 2867.2, 60 sec: 3549.9, 300 sec: 3637.8). Total num frames: 2461696. Throughput: 0: 878.4. Samples: 615766. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-07-24 19:12:28,637][02885] Avg episode reward: [(0, '17.929')] +[2024-07-24 19:12:28,653][05263] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000601_2461696.pth... +[2024-07-24 19:12:28,778][05263] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000389_1593344.pth +[2024-07-24 19:12:33,635][02885] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3637.8). Total num frames: 2482176. Throughput: 0: 877.7. Samples: 618906. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-07-24 19:12:33,639][02885] Avg episode reward: [(0, '18.573')] +[2024-07-24 19:12:33,715][05263] Saving new best policy, reward=18.573! +[2024-07-24 19:12:36,531][05276] Updated weights for policy 0, policy_version 610 (0.0028) +[2024-07-24 19:12:38,637][02885] Fps is (10 sec: 4095.3, 60 sec: 3686.3, 300 sec: 3623.9). Total num frames: 2502656. Throughput: 0: 935.3. Samples: 625428. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-07-24 19:12:38,641][02885] Avg episode reward: [(0, '17.718')] +[2024-07-24 19:12:43,635][02885] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3623.9). Total num frames: 2514944. Throughput: 0: 897.8. Samples: 629766. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-07-24 19:12:43,637][02885] Avg episode reward: [(0, '17.657')] +[2024-07-24 19:12:48,635][02885] Fps is (10 sec: 3277.4, 60 sec: 3549.9, 300 sec: 3637.8). Total num frames: 2535424. Throughput: 0: 880.4. Samples: 632098. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:12:48,641][02885] Avg episode reward: [(0, '18.603')] +[2024-07-24 19:12:48,650][05263] Saving new best policy, reward=18.603! +[2024-07-24 19:12:48,911][05276] Updated weights for policy 0, policy_version 620 (0.0034) +[2024-07-24 19:12:53,635][02885] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3637.8). Total num frames: 2555904. Throughput: 0: 909.5. Samples: 638386. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-07-24 19:12:53,640][02885] Avg episode reward: [(0, '17.031')] +[2024-07-24 19:12:58,635][02885] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3623.9). Total num frames: 2572288. Throughput: 0: 914.2. Samples: 643720. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:12:58,642][02885] Avg episode reward: [(0, '17.422')] +[2024-07-24 19:13:00,597][05276] Updated weights for policy 0, policy_version 630 (0.0032) +[2024-07-24 19:13:03,635][02885] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3623.9). Total num frames: 2588672. Throughput: 0: 884.6. Samples: 645644. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-07-24 19:13:03,638][02885] Avg episode reward: [(0, '17.887')] +[2024-07-24 19:13:08,635][02885] Fps is (10 sec: 3686.4, 60 sec: 3618.2, 300 sec: 3623.9). Total num frames: 2609152. Throughput: 0: 889.8. Samples: 651698. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:13:08,637][02885] Avg episode reward: [(0, '17.724')] +[2024-07-24 19:13:10,686][05276] Updated weights for policy 0, policy_version 640 (0.0025) +[2024-07-24 19:13:13,635][02885] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3623.9). Total num frames: 2629632. Throughput: 0: 939.8. Samples: 658056. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:13:13,642][02885] Avg episode reward: [(0, '17.214')] +[2024-07-24 19:13:18,635][02885] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3623.9). Total num frames: 2641920. Throughput: 0: 913.0. Samples: 659990. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:13:18,637][02885] Avg episode reward: [(0, '17.762')] +[2024-07-24 19:13:23,175][05276] Updated weights for policy 0, policy_version 650 (0.0014) +[2024-07-24 19:13:23,635][02885] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3623.9). Total num frames: 2662400. Throughput: 0: 873.5. Samples: 664734. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-07-24 19:13:23,638][02885] Avg episode reward: [(0, '17.650')] +[2024-07-24 19:13:28,635][02885] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3623.9). Total num frames: 2682880. Throughput: 0: 917.6. Samples: 671056. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:13:28,638][02885] Avg episode reward: [(0, '19.130')] +[2024-07-24 19:13:28,645][05263] Saving new best policy, reward=19.130! +[2024-07-24 19:13:33,635][02885] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3610.0). Total num frames: 2699264. Throughput: 0: 923.2. Samples: 673640. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-07-24 19:13:33,639][02885] Avg episode reward: [(0, '18.757')] +[2024-07-24 19:13:35,150][05276] Updated weights for policy 0, policy_version 660 (0.0038) +[2024-07-24 19:13:38,635][02885] Fps is (10 sec: 3276.8, 60 sec: 3550.0, 300 sec: 3623.9). Total num frames: 2715648. Throughput: 0: 871.0. Samples: 677580. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:13:38,637][02885] Avg episode reward: [(0, '19.933')] +[2024-07-24 19:13:38,653][05263] Saving new best policy, reward=19.933! +[2024-07-24 19:13:43,635][02885] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3623.9). Total num frames: 2736128. Throughput: 0: 896.1. Samples: 684044. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:13:43,640][02885] Avg episode reward: [(0, '18.922')] +[2024-07-24 19:13:45,272][05276] Updated weights for policy 0, policy_version 670 (0.0028) +[2024-07-24 19:13:48,635][02885] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3596.1). Total num frames: 2752512. Throughput: 0: 924.5. Samples: 687248. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:13:48,641][02885] Avg episode reward: [(0, '17.557')] +[2024-07-24 19:13:53,635][02885] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3610.1). Total num frames: 2768896. Throughput: 0: 888.4. Samples: 691676. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-07-24 19:13:53,637][02885] Avg episode reward: [(0, '16.301')] +[2024-07-24 19:13:57,667][05276] Updated weights for policy 0, policy_version 680 (0.0030) +[2024-07-24 19:13:58,635][02885] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3623.9). Total num frames: 2789376. Throughput: 0: 864.5. Samples: 696960. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-07-24 19:13:58,638][02885] Avg episode reward: [(0, '17.352')] +[2024-07-24 19:14:03,635][02885] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3610.0). Total num frames: 2809856. Throughput: 0: 894.7. Samples: 700250. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-07-24 19:14:03,639][02885] Avg episode reward: [(0, '18.486')] +[2024-07-24 19:14:08,635][02885] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3610.0). Total num frames: 2822144. Throughput: 0: 911.5. Samples: 705750. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:14:08,641][02885] Avg episode reward: [(0, '19.721')] +[2024-07-24 19:14:08,957][05276] Updated weights for policy 0, policy_version 690 (0.0013) +[2024-07-24 19:14:13,635][02885] Fps is (10 sec: 2867.2, 60 sec: 3481.6, 300 sec: 3610.0). Total num frames: 2838528. Throughput: 0: 869.3. Samples: 710176. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:14:13,637][02885] Avg episode reward: [(0, '18.238')] +[2024-07-24 19:14:18,635][02885] Fps is (10 sec: 4095.9, 60 sec: 3686.4, 300 sec: 3623.9). Total num frames: 2863104. Throughput: 0: 884.9. Samples: 713462. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-07-24 19:14:18,638][02885] Avg episode reward: [(0, '19.821')] +[2024-07-24 19:14:19,388][05276] Updated weights for policy 0, policy_version 700 (0.0014) +[2024-07-24 19:14:23,635][02885] Fps is (10 sec: 4096.0, 60 sec: 3618.1, 300 sec: 3610.0). Total num frames: 2879488. Throughput: 0: 936.8. Samples: 719736. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:14:23,642][02885] Avg episode reward: [(0, '20.043')] +[2024-07-24 19:14:23,646][05263] Saving new best policy, reward=20.043! +[2024-07-24 19:14:28,635][02885] Fps is (10 sec: 2867.2, 60 sec: 3481.6, 300 sec: 3610.0). Total num frames: 2891776. Throughput: 0: 879.2. Samples: 723608. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-07-24 19:14:28,644][02885] Avg episode reward: [(0, '20.434')] +[2024-07-24 19:14:28,665][05263] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000706_2891776.pth... +[2024-07-24 19:14:28,833][05263] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000495_2027520.pth +[2024-07-24 19:14:28,853][05263] Saving new best policy, reward=20.434! +[2024-07-24 19:14:32,006][05276] Updated weights for policy 0, policy_version 710 (0.0022) +[2024-07-24 19:14:33,635][02885] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3610.0). Total num frames: 2912256. Throughput: 0: 867.6. Samples: 726288. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:14:33,640][02885] Avg episode reward: [(0, '20.224')] +[2024-07-24 19:14:38,635][02885] Fps is (10 sec: 4505.6, 60 sec: 3686.4, 300 sec: 3610.0). Total num frames: 2936832. Throughput: 0: 913.3. Samples: 732776. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:14:38,640][02885] Avg episode reward: [(0, '20.487')] +[2024-07-24 19:14:38,650][05263] Saving new best policy, reward=20.487! +[2024-07-24 19:14:43,060][05276] Updated weights for policy 0, policy_version 720 (0.0030) +[2024-07-24 19:14:43,635][02885] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3596.2). Total num frames: 2949120. Throughput: 0: 905.3. Samples: 737698. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:14:43,637][02885] Avg episode reward: [(0, '21.995')] +[2024-07-24 19:14:43,645][05263] Saving new best policy, reward=21.995! +[2024-07-24 19:14:48,635][02885] Fps is (10 sec: 2867.2, 60 sec: 3549.9, 300 sec: 3610.0). Total num frames: 2965504. Throughput: 0: 875.3. Samples: 739640. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:14:48,642][02885] Avg episode reward: [(0, '22.002')] +[2024-07-24 19:14:48,652][05263] Saving new best policy, reward=22.002! +[2024-07-24 19:14:53,635][02885] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3596.2). Total num frames: 2985984. Throughput: 0: 889.6. Samples: 745784. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-07-24 19:14:53,638][02885] Avg episode reward: [(0, '20.175')] +[2024-07-24 19:14:53,932][05276] Updated weights for policy 0, policy_version 730 (0.0022) +[2024-07-24 19:14:58,638][02885] Fps is (10 sec: 3685.3, 60 sec: 3549.7, 300 sec: 3596.1). Total num frames: 3002368. Throughput: 0: 918.1. Samples: 751494. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-07-24 19:14:58,643][02885] Avg episode reward: [(0, '19.457')] +[2024-07-24 19:15:03,636][02885] Fps is (10 sec: 3276.4, 60 sec: 3481.5, 300 sec: 3596.1). Total num frames: 3018752. Throughput: 0: 887.0. Samples: 753378. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:15:03,639][02885] Avg episode reward: [(0, '20.056')] +[2024-07-24 19:15:06,883][05276] Updated weights for policy 0, policy_version 740 (0.0036) +[2024-07-24 19:15:08,635][02885] Fps is (10 sec: 3277.8, 60 sec: 3549.9, 300 sec: 3582.3). Total num frames: 3035136. Throughput: 0: 857.2. Samples: 758312. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:15:08,643][02885] Avg episode reward: [(0, '20.591')] +[2024-07-24 19:15:13,635][02885] Fps is (10 sec: 3686.9, 60 sec: 3618.1, 300 sec: 3596.1). Total num frames: 3055616. Throughput: 0: 899.5. Samples: 764084. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-07-24 19:15:13,645][02885] Avg episode reward: [(0, '18.422')] +[2024-07-24 19:15:18,635][02885] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 3582.3). Total num frames: 3067904. Throughput: 0: 886.0. Samples: 766160. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:15:18,638][02885] Avg episode reward: [(0, '19.206')] +[2024-07-24 19:15:20,040][05276] Updated weights for policy 0, policy_version 750 (0.0028) +[2024-07-24 19:15:23,635][02885] Fps is (10 sec: 2457.6, 60 sec: 3345.1, 300 sec: 3568.4). Total num frames: 3080192. Throughput: 0: 818.4. Samples: 769606. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-07-24 19:15:23,643][02885] Avg episode reward: [(0, '19.109')] +[2024-07-24 19:15:28,635][02885] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3568.4). Total num frames: 3100672. Throughput: 0: 836.0. Samples: 775320. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-07-24 19:15:28,637][02885] Avg episode reward: [(0, '19.595')] +[2024-07-24 19:15:31,657][05276] Updated weights for policy 0, policy_version 760 (0.0026) +[2024-07-24 19:15:33,635][02885] Fps is (10 sec: 3686.4, 60 sec: 3413.3, 300 sec: 3554.5). Total num frames: 3117056. Throughput: 0: 855.6. Samples: 778142. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-07-24 19:15:33,639][02885] Avg episode reward: [(0, '19.582')] +[2024-07-24 19:15:38,635][02885] Fps is (10 sec: 2867.2, 60 sec: 3208.5, 300 sec: 3554.5). Total num frames: 3129344. Throughput: 0: 808.8. Samples: 782182. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:15:38,642][02885] Avg episode reward: [(0, '19.835')] +[2024-07-24 19:15:43,635][02885] Fps is (10 sec: 3276.7, 60 sec: 3345.1, 300 sec: 3554.5). Total num frames: 3149824. Throughput: 0: 799.8. Samples: 787484. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-07-24 19:15:43,637][02885] Avg episode reward: [(0, '21.094')] +[2024-07-24 19:15:44,375][05276] Updated weights for policy 0, policy_version 770 (0.0021) +[2024-07-24 19:15:48,635][02885] Fps is (10 sec: 4096.0, 60 sec: 3413.3, 300 sec: 3554.5). Total num frames: 3170304. Throughput: 0: 830.1. Samples: 790732. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-07-24 19:15:48,639][02885] Avg episode reward: [(0, '22.713')] +[2024-07-24 19:15:48,649][05263] Saving new best policy, reward=22.713! +[2024-07-24 19:15:53,635][02885] Fps is (10 sec: 3686.5, 60 sec: 3345.1, 300 sec: 3554.5). Total num frames: 3186688. Throughput: 0: 840.3. Samples: 796126. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:15:53,638][02885] Avg episode reward: [(0, '23.073')] +[2024-07-24 19:15:53,640][05263] Saving new best policy, reward=23.073! +[2024-07-24 19:15:56,412][05276] Updated weights for policy 0, policy_version 780 (0.0023) +[2024-07-24 19:15:58,635][02885] Fps is (10 sec: 3276.8, 60 sec: 3345.2, 300 sec: 3554.5). Total num frames: 3203072. Throughput: 0: 809.1. Samples: 800492. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-07-24 19:15:58,637][02885] Avg episode reward: [(0, '22.255')] +[2024-07-24 19:16:03,635][02885] Fps is (10 sec: 3686.3, 60 sec: 3413.4, 300 sec: 3554.5). Total num frames: 3223552. Throughput: 0: 832.8. Samples: 803636. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:16:03,638][02885] Avg episode reward: [(0, '21.593')] +[2024-07-24 19:16:06,352][05276] Updated weights for policy 0, policy_version 790 (0.0029) +[2024-07-24 19:16:08,635][02885] Fps is (10 sec: 3686.4, 60 sec: 3413.3, 300 sec: 3540.6). Total num frames: 3239936. Throughput: 0: 900.7. Samples: 810136. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-07-24 19:16:08,639][02885] Avg episode reward: [(0, '22.264')] +[2024-07-24 19:16:13,635][02885] Fps is (10 sec: 2867.3, 60 sec: 3276.8, 300 sec: 3540.6). Total num frames: 3252224. Throughput: 0: 863.3. Samples: 814168. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) +[2024-07-24 19:16:13,641][02885] Avg episode reward: [(0, '22.956')] +[2024-07-24 19:16:18,635][02885] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 3540.6). Total num frames: 3272704. Throughput: 0: 858.9. Samples: 816794. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:16:18,637][02885] Avg episode reward: [(0, '21.718')] +[2024-07-24 19:16:18,674][05276] Updated weights for policy 0, policy_version 800 (0.0036) +[2024-07-24 19:16:23,635][02885] Fps is (10 sec: 4505.6, 60 sec: 3618.1, 300 sec: 3554.5). Total num frames: 3297280. Throughput: 0: 913.1. Samples: 823272. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:16:23,637][02885] Avg episode reward: [(0, '21.490')] +[2024-07-24 19:16:28,635][02885] Fps is (10 sec: 3686.3, 60 sec: 3481.6, 300 sec: 3540.6). Total num frames: 3309568. Throughput: 0: 903.9. Samples: 828158. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-07-24 19:16:28,638][02885] Avg episode reward: [(0, '22.450')] +[2024-07-24 19:16:28,653][05263] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000808_3309568.pth... +[2024-07-24 19:16:28,782][05263] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000601_2461696.pth +[2024-07-24 19:16:30,632][05276] Updated weights for policy 0, policy_version 810 (0.0024) +[2024-07-24 19:16:33,635][02885] Fps is (10 sec: 2867.2, 60 sec: 3481.6, 300 sec: 3540.6). Total num frames: 3325952. Throughput: 0: 875.0. Samples: 830108. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:16:33,642][02885] Avg episode reward: [(0, '21.423')] +[2024-07-24 19:16:38,635][02885] Fps is (10 sec: 4096.1, 60 sec: 3686.4, 300 sec: 3540.6). Total num frames: 3350528. Throughput: 0: 894.6. Samples: 836384. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:16:38,639][02885] Avg episode reward: [(0, '20.518')] +[2024-07-24 19:16:40,335][05276] Updated weights for policy 0, policy_version 820 (0.0028) +[2024-07-24 19:16:43,635][02885] Fps is (10 sec: 4096.0, 60 sec: 3618.2, 300 sec: 3540.6). Total num frames: 3366912. Throughput: 0: 931.8. Samples: 842422. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:16:43,645][02885] Avg episode reward: [(0, '20.480')] +[2024-07-24 19:16:48,635][02885] Fps is (10 sec: 2867.2, 60 sec: 3481.6, 300 sec: 3540.6). Total num frames: 3379200. Throughput: 0: 905.8. Samples: 844398. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:16:48,641][02885] Avg episode reward: [(0, '20.670')] +[2024-07-24 19:16:52,616][05276] Updated weights for policy 0, policy_version 830 (0.0018) +[2024-07-24 19:16:53,635][02885] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3554.5). Total num frames: 3403776. Throughput: 0: 881.7. Samples: 849812. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-07-24 19:16:53,637][02885] Avg episode reward: [(0, '20.884')] +[2024-07-24 19:16:58,638][02885] Fps is (10 sec: 4504.1, 60 sec: 3686.2, 300 sec: 3554.5). Total num frames: 3424256. Throughput: 0: 932.7. Samples: 856144. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-07-24 19:16:58,642][02885] Avg episode reward: [(0, '20.630')] +[2024-07-24 19:17:03,635][02885] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3540.6). Total num frames: 3436544. Throughput: 0: 927.4. Samples: 858528. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:17:03,642][02885] Avg episode reward: [(0, '21.316')] +[2024-07-24 19:17:04,322][05276] Updated weights for policy 0, policy_version 840 (0.0023) +[2024-07-24 19:17:08,635][02885] Fps is (10 sec: 3277.9, 60 sec: 3618.1, 300 sec: 3554.5). Total num frames: 3457024. Throughput: 0: 880.6. Samples: 862900. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:17:08,637][02885] Avg episode reward: [(0, '22.458')] +[2024-07-24 19:17:13,635][02885] Fps is (10 sec: 4096.0, 60 sec: 3754.7, 300 sec: 3540.6). Total num frames: 3477504. Throughput: 0: 917.5. Samples: 869444. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:17:13,637][02885] Avg episode reward: [(0, '21.491')] +[2024-07-24 19:17:14,252][05276] Updated weights for policy 0, policy_version 850 (0.0023) +[2024-07-24 19:17:18,635][02885] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3554.5). Total num frames: 3493888. Throughput: 0: 946.3. Samples: 872692. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:17:18,637][02885] Avg episode reward: [(0, '21.040')] +[2024-07-24 19:17:23,635][02885] Fps is (10 sec: 2867.2, 60 sec: 3481.6, 300 sec: 3540.6). Total num frames: 3506176. Throughput: 0: 895.9. Samples: 876700. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:17:23,639][02885] Avg episode reward: [(0, '21.054')] +[2024-07-24 19:17:26,699][05276] Updated weights for policy 0, policy_version 860 (0.0030) +[2024-07-24 19:17:28,635][02885] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3554.5). Total num frames: 3530752. Throughput: 0: 890.5. Samples: 882496. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:17:28,642][02885] Avg episode reward: [(0, '20.544')] +[2024-07-24 19:17:33,635][02885] Fps is (10 sec: 4505.6, 60 sec: 3754.7, 300 sec: 3554.5). Total num frames: 3551232. Throughput: 0: 918.4. Samples: 885724. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:17:33,637][02885] Avg episode reward: [(0, '20.733')] +[2024-07-24 19:17:37,626][05276] Updated weights for policy 0, policy_version 870 (0.0025) +[2024-07-24 19:17:38,635][02885] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3554.5). Total num frames: 3563520. Throughput: 0: 914.4. Samples: 890960. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:17:38,639][02885] Avg episode reward: [(0, '20.580')] +[2024-07-24 19:17:43,635][02885] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3554.5). Total num frames: 3584000. Throughput: 0: 887.3. Samples: 896070. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:17:43,638][02885] Avg episode reward: [(0, '21.328')] +[2024-07-24 19:17:47,811][05276] Updated weights for policy 0, policy_version 880 (0.0026) +[2024-07-24 19:17:48,635][02885] Fps is (10 sec: 4096.0, 60 sec: 3754.7, 300 sec: 3554.5). Total num frames: 3604480. Throughput: 0: 908.5. Samples: 899412. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:17:48,638][02885] Avg episode reward: [(0, '20.050')] +[2024-07-24 19:17:53,636][02885] Fps is (10 sec: 3686.1, 60 sec: 3618.1, 300 sec: 3554.5). Total num frames: 3620864. Throughput: 0: 943.6. Samples: 905362. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-07-24 19:17:53,639][02885] Avg episode reward: [(0, '18.849')] +[2024-07-24 19:17:58,635][02885] Fps is (10 sec: 3276.8, 60 sec: 3550.1, 300 sec: 3554.5). Total num frames: 3637248. Throughput: 0: 886.5. Samples: 909336. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-07-24 19:17:58,641][02885] Avg episode reward: [(0, '19.716')] +[2024-07-24 19:18:00,203][05276] Updated weights for policy 0, policy_version 890 (0.0023) +[2024-07-24 19:18:03,635][02885] Fps is (10 sec: 3686.6, 60 sec: 3686.4, 300 sec: 3554.5). Total num frames: 3657728. Throughput: 0: 885.2. Samples: 912526. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2024-07-24 19:18:03,641][02885] Avg episode reward: [(0, '19.977')] +[2024-07-24 19:18:08,635][02885] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3554.5). Total num frames: 3678208. Throughput: 0: 943.0. Samples: 919136. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:18:08,642][02885] Avg episode reward: [(0, '19.452')] +[2024-07-24 19:18:10,497][05276] Updated weights for policy 0, policy_version 900 (0.0013) +[2024-07-24 19:18:13,635][02885] Fps is (10 sec: 3686.5, 60 sec: 3618.1, 300 sec: 3568.4). Total num frames: 3694592. Throughput: 0: 914.7. Samples: 923658. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:18:13,644][02885] Avg episode reward: [(0, '19.590')] +[2024-07-24 19:18:18,635][02885] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3554.5). Total num frames: 3710976. Throughput: 0: 894.9. Samples: 925994. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:18:18,638][02885] Avg episode reward: [(0, '21.980')] +[2024-07-24 19:18:21,785][05276] Updated weights for policy 0, policy_version 910 (0.0017) +[2024-07-24 19:18:23,635][02885] Fps is (10 sec: 4096.0, 60 sec: 3822.9, 300 sec: 3568.4). Total num frames: 3735552. Throughput: 0: 924.0. Samples: 932538. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-07-24 19:18:23,640][02885] Avg episode reward: [(0, '22.373')] +[2024-07-24 19:18:28,641][02885] Fps is (10 sec: 3684.2, 60 sec: 3617.8, 300 sec: 3554.4). Total num frames: 3747840. Throughput: 0: 921.7. Samples: 937550. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:18:28,644][02885] Avg episode reward: [(0, '21.937')] +[2024-07-24 19:18:28,660][05263] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000915_3747840.pth... +[2024-07-24 19:18:28,835][05263] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000706_2891776.pth +[2024-07-24 19:18:33,635][02885] Fps is (10 sec: 2867.2, 60 sec: 3549.9, 300 sec: 3554.5). Total num frames: 3764224. Throughput: 0: 891.2. Samples: 939514. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:18:33,637][02885] Avg episode reward: [(0, '22.521')] +[2024-07-24 19:18:34,278][05276] Updated weights for policy 0, policy_version 920 (0.0045) +[2024-07-24 19:18:38,635][02885] Fps is (10 sec: 3688.6, 60 sec: 3686.4, 300 sec: 3554.5). Total num frames: 3784704. Throughput: 0: 896.5. Samples: 945702. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-07-24 19:18:38,637][02885] Avg episode reward: [(0, '21.319')] +[2024-07-24 19:18:43,635][02885] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3568.4). Total num frames: 3805184. Throughput: 0: 945.7. Samples: 951892. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-07-24 19:18:43,647][02885] Avg episode reward: [(0, '22.520')] +[2024-07-24 19:18:43,967][05276] Updated weights for policy 0, policy_version 930 (0.0026) +[2024-07-24 19:18:48,641][02885] Fps is (10 sec: 3274.9, 60 sec: 3549.5, 300 sec: 3554.4). Total num frames: 3817472. Throughput: 0: 919.0. Samples: 953886. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-07-24 19:18:48,644][02885] Avg episode reward: [(0, '22.627')] +[2024-07-24 19:18:53,635][02885] Fps is (10 sec: 3276.8, 60 sec: 3618.2, 300 sec: 3554.5). Total num frames: 3837952. Throughput: 0: 884.8. Samples: 958954. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:18:53,637][02885] Avg episode reward: [(0, '22.734')] +[2024-07-24 19:18:55,790][05276] Updated weights for policy 0, policy_version 940 (0.0025) +[2024-07-24 19:18:58,635][02885] Fps is (10 sec: 4098.5, 60 sec: 3686.4, 300 sec: 3554.5). Total num frames: 3858432. Throughput: 0: 926.0. Samples: 965330. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:18:58,637][02885] Avg episode reward: [(0, '21.689')] +[2024-07-24 19:19:03,635][02885] Fps is (10 sec: 3686.4, 60 sec: 3618.2, 300 sec: 3568.4). Total num frames: 3874816. Throughput: 0: 932.5. Samples: 967958. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2024-07-24 19:19:03,638][02885] Avg episode reward: [(0, '21.756')] +[2024-07-24 19:19:08,003][05276] Updated weights for policy 0, policy_version 950 (0.0023) +[2024-07-24 19:19:08,635][02885] Fps is (10 sec: 3276.7, 60 sec: 3549.9, 300 sec: 3568.4). Total num frames: 3891200. Throughput: 0: 881.3. Samples: 972196. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2024-07-24 19:19:08,638][02885] Avg episode reward: [(0, '21.105')] +[2024-07-24 19:19:13,635][02885] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3554.5). Total num frames: 3911680. Throughput: 0: 913.5. Samples: 978652. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2024-07-24 19:19:13,639][02885] Avg episode reward: [(0, '19.860')] +[2024-07-24 19:19:18,240][05276] Updated weights for policy 0, policy_version 960 (0.0017) +[2024-07-24 19:19:18,635][02885] Fps is (10 sec: 4096.1, 60 sec: 3686.4, 300 sec: 3568.4). Total num frames: 3932160. Throughput: 0: 942.0. Samples: 981904. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:19:18,641][02885] Avg episode reward: [(0, '18.992')] +[2024-07-24 19:19:23,635][02885] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3568.4). Total num frames: 3944448. Throughput: 0: 897.5. Samples: 986088. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:19:23,638][02885] Avg episode reward: [(0, '19.343')] +[2024-07-24 19:19:28,635][02885] Fps is (10 sec: 3276.8, 60 sec: 3618.5, 300 sec: 3568.4). Total num frames: 3964928. Throughput: 0: 886.3. Samples: 991774. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2024-07-24 19:19:28,637][02885] Avg episode reward: [(0, '18.882')] +[2024-07-24 19:19:29,771][05276] Updated weights for policy 0, policy_version 970 (0.0041) +[2024-07-24 19:19:33,635][02885] Fps is (10 sec: 4505.6, 60 sec: 3754.7, 300 sec: 3568.4). Total num frames: 3989504. Throughput: 0: 913.6. Samples: 994992. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2024-07-24 19:19:33,637][02885] Avg episode reward: [(0, '20.129')] +[2024-07-24 19:19:38,635][02885] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3568.4). Total num frames: 4001792. Throughput: 0: 915.6. Samples: 1000158. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2024-07-24 19:19:38,641][02885] Avg episode reward: [(0, '19.368')] +[2024-07-24 19:19:39,565][05263] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... +[2024-07-24 19:19:39,588][05263] Stopping Batcher_0... +[2024-07-24 19:19:39,588][05263] Loop batcher_evt_loop terminating... +[2024-07-24 19:19:39,588][02885] Component Batcher_0 stopped! +[2024-07-24 19:19:39,645][02885] Component RolloutWorker_w7 stopped! +[2024-07-24 19:19:39,653][05283] Stopping RolloutWorker_w7... +[2024-07-24 19:19:39,653][05283] Loop rollout_proc7_evt_loop terminating... +[2024-07-24 19:19:39,685][02885] Component RolloutWorker_w3 stopped! +[2024-07-24 19:19:39,691][05277] Stopping RolloutWorker_w0... +[2024-07-24 19:19:39,692][05277] Loop rollout_proc0_evt_loop terminating... +[2024-07-24 19:19:39,692][02885] Component RolloutWorker_w0 stopped! +[2024-07-24 19:19:39,689][05281] Stopping RolloutWorker_w3... +[2024-07-24 19:19:39,702][05281] Loop rollout_proc3_evt_loop terminating... +[2024-07-24 19:19:39,706][02885] Component RolloutWorker_w1 stopped! +[2024-07-24 19:19:39,709][05279] Stopping RolloutWorker_w1... +[2024-07-24 19:19:39,716][05279] Loop rollout_proc1_evt_loop terminating... +[2024-07-24 19:19:39,719][02885] Component RolloutWorker_w5 stopped! +[2024-07-24 19:19:39,724][05282] Stopping RolloutWorker_w5... +[2024-07-24 19:19:39,725][05282] Loop rollout_proc5_evt_loop terminating... +[2024-07-24 19:19:39,742][05280] Stopping RolloutWorker_w4... +[2024-07-24 19:19:39,743][05280] Loop rollout_proc4_evt_loop terminating... +[2024-07-24 19:19:39,742][02885] Component RolloutWorker_w4 stopped! +[2024-07-24 19:19:39,755][05276] Weights refcount: 2 0 +[2024-07-24 19:19:39,757][02885] Component InferenceWorker_p0-w0 stopped! +[2024-07-24 19:19:39,757][05276] Stopping InferenceWorker_p0-w0... +[2024-07-24 19:19:39,769][05276] Loop inference_proc0-0_evt_loop terminating... +[2024-07-24 19:19:39,773][05263] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000808_3309568.pth +[2024-07-24 19:19:39,799][05263] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... +[2024-07-24 19:19:39,819][05278] Stopping RolloutWorker_w2... +[2024-07-24 19:19:39,820][05278] Loop rollout_proc2_evt_loop terminating... +[2024-07-24 19:19:39,822][02885] Component RolloutWorker_w2 stopped! +[2024-07-24 19:19:39,829][02885] Component RolloutWorker_w6 stopped! +[2024-07-24 19:19:39,831][05284] Stopping RolloutWorker_w6... +[2024-07-24 19:19:39,832][05284] Loop rollout_proc6_evt_loop terminating... +[2024-07-24 19:19:40,163][02885] Component LearnerWorker_p0 stopped! +[2024-07-24 19:19:40,168][02885] Waiting for process learner_proc0 to stop... +[2024-07-24 19:19:40,173][05263] Stopping LearnerWorker_p0... +[2024-07-24 19:19:40,173][05263] Loop learner_proc0_evt_loop terminating... +[2024-07-24 19:19:41,578][02885] Waiting for process inference_proc0-0 to join... +[2024-07-24 19:19:41,583][02885] Waiting for process rollout_proc0 to join... +[2024-07-24 19:19:43,401][02885] Waiting for process rollout_proc1 to join... +[2024-07-24 19:19:43,406][02885] Waiting for process rollout_proc2 to join... +[2024-07-24 19:19:43,410][02885] Waiting for process rollout_proc3 to join... +[2024-07-24 19:19:43,414][02885] Waiting for process rollout_proc4 to join... +[2024-07-24 19:19:43,418][02885] Waiting for process rollout_proc5 to join... +[2024-07-24 19:19:43,421][02885] Waiting for process rollout_proc6 to join... +[2024-07-24 19:19:43,426][02885] Waiting for process rollout_proc7 to join... +[2024-07-24 19:19:43,430][02885] Batcher 0 profile tree view: +batching: 26.6392, releasing_batches: 0.0283 +[2024-07-24 19:19:43,431][02885] InferenceWorker_p0-w0 profile tree view: wait_policy: 0.0000 - wait_policy_total: 462.6672 -update_model: 9.9923 - weight_update: 0.0024 -one_step: 0.0127 - handle_policy_step: 634.6919 - deserialize: 16.4988, stack: 3.3864, obs_to_device_normalize: 128.3574, forward: 338.0509, send_messages: 30.3013 - prepare_outputs: 85.7805 - to_cpu: 49.2792 -[2024-07-24 18:07:14,400][03928] Learner 0 profile tree view: -misc: 0.0051, prepare_batch: 14.2455 -train: 75.5714 - epoch_init: 0.0145, minibatch_init: 0.0154, losses_postprocess: 0.6793, kl_divergence: 0.8118, after_optimizer: 33.8778 - calculate_losses: 28.0188 - losses_init: 0.0038, forward_head: 1.4569, bptt_initial: 18.6248, tail: 1.2256, advantages_returns: 0.2762, losses: 3.9658 - bptt: 2.1481 - bptt_forward_core: 2.0234 - update: 11.4792 - clip: 1.0309 -[2024-07-24 18:07:14,401][03928] RolloutWorker_w0 profile tree view: -wait_for_trajectories: 0.2901, enqueue_policy_requests: 123.5031, env_step: 889.5694, overhead: 16.4671, complete_rollouts: 7.3169 -save_policy_outputs: 21.4189 - split_output_tensors: 8.8444 -[2024-07-24 18:07:14,402][03928] RolloutWorker_w7 profile tree view: -wait_for_trajectories: 0.4289, enqueue_policy_requests: 126.3101, env_step: 889.9336, overhead: 16.4210, complete_rollouts: 7.5382 -save_policy_outputs: 21.6390 - split_output_tensors: 8.7660 -[2024-07-24 18:07:14,404][03928] Loop Runner_EvtLoop terminating... -[2024-07-24 18:07:14,406][03928] Runner profile tree view: -main_loop: 1178.6859 -[2024-07-24 18:07:14,407][03928] Collected {0: 4005888}, FPS: 3398.6 -[2024-07-24 18:29:36,413][03928] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json -[2024-07-24 18:29:36,414][03928] Overriding arg 'num_workers' with value 1 passed from command line -[2024-07-24 18:29:36,416][03928] Adding new argument 'no_render'=True that is not in the saved config file! -[2024-07-24 18:29:36,418][03928] Adding new argument 'save_video'=True that is not in the saved config file! -[2024-07-24 18:29:36,420][03928] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! -[2024-07-24 18:29:36,422][03928] Adding new argument 'video_name'=None that is not in the saved config file! -[2024-07-24 18:29:36,424][03928] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! -[2024-07-24 18:29:36,426][03928] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! -[2024-07-24 18:29:36,429][03928] Adding new argument 'push_to_hub'=True that is not in the saved config file! -[2024-07-24 18:29:36,430][03928] Adding new argument 'hf_repository'='dergky1/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! -[2024-07-24 18:29:36,432][03928] Adding new argument 'policy_index'=0 that is not in the saved config file! -[2024-07-24 18:29:36,433][03928] Adding new argument 'eval_deterministic'=False that is not in the saved config file! -[2024-07-24 18:29:36,434][03928] Adding new argument 'train_script'=None that is not in the saved config file! -[2024-07-24 18:29:36,435][03928] Adding new argument 'enjoy_script'=None that is not in the saved config file! -[2024-07-24 18:29:36,436][03928] Using frameskip 1 and render_action_repeat=4 for evaluation -[2024-07-24 18:29:36,471][03928] Doom resolution: 160x120, resize resolution: (128, 72) -[2024-07-24 18:29:36,474][03928] RunningMeanStd input shape: (3, 72, 128) -[2024-07-24 18:29:36,477][03928] RunningMeanStd input shape: (1,) -[2024-07-24 18:29:36,493][03928] ConvEncoder: input_channels=3 -[2024-07-24 18:29:36,607][03928] Conv encoder output size: 512 -[2024-07-24 18:29:36,610][03928] Policy head output size: 512 -[2024-07-24 18:29:36,781][03928] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... -[2024-07-24 18:29:37,578][03928] Num frames 100... -[2024-07-24 18:29:37,723][03928] Num frames 200... -[2024-07-24 18:29:37,856][03928] Num frames 300... -[2024-07-24 18:29:37,991][03928] Num frames 400... -[2024-07-24 18:29:38,132][03928] Num frames 500... -[2024-07-24 18:29:38,261][03928] Num frames 600... -[2024-07-24 18:29:38,388][03928] Num frames 700... -[2024-07-24 18:29:38,518][03928] Num frames 800... -[2024-07-24 18:29:38,661][03928] Avg episode rewards: #0: 16.670, true rewards: #0: 8.670 -[2024-07-24 18:29:38,663][03928] Avg episode reward: 16.670, avg true_objective: 8.670 -[2024-07-24 18:29:38,710][03928] Num frames 900... -[2024-07-24 18:29:38,836][03928] Num frames 1000... -[2024-07-24 18:29:38,963][03928] Num frames 1100... -[2024-07-24 18:29:39,092][03928] Num frames 1200... -[2024-07-24 18:29:39,232][03928] Num frames 1300... -[2024-07-24 18:29:39,364][03928] Num frames 1400... -[2024-07-24 18:29:39,501][03928] Avg episode rewards: #0: 13.810, true rewards: #0: 7.310 -[2024-07-24 18:29:39,504][03928] Avg episode reward: 13.810, avg true_objective: 7.310 -[2024-07-24 18:29:39,584][03928] Num frames 1500... -[2024-07-24 18:29:39,781][03928] Num frames 1600... -[2024-07-24 18:29:39,967][03928] Num frames 1700... -[2024-07-24 18:29:40,158][03928] Num frames 1800... -[2024-07-24 18:29:40,348][03928] Num frames 1900... -[2024-07-24 18:29:40,526][03928] Num frames 2000... -[2024-07-24 18:29:40,702][03928] Num frames 2100... -[2024-07-24 18:29:40,898][03928] Num frames 2200... -[2024-07-24 18:29:41,093][03928] Num frames 2300... -[2024-07-24 18:29:41,330][03928] Avg episode rewards: #0: 15.307, true rewards: #0: 7.973 -[2024-07-24 18:29:41,333][03928] Avg episode reward: 15.307, avg true_objective: 7.973 -[2024-07-24 18:29:41,356][03928] Num frames 2400... -[2024-07-24 18:29:41,554][03928] Num frames 2500... -[2024-07-24 18:29:41,746][03928] Num frames 2600... -[2024-07-24 18:29:41,948][03928] Num frames 2700... -[2024-07-24 18:29:42,145][03928] Num frames 2800... -[2024-07-24 18:29:42,281][03928] Num frames 2900... -[2024-07-24 18:29:42,386][03928] Avg episode rewards: #0: 13.340, true rewards: #0: 7.340 -[2024-07-24 18:29:42,388][03928] Avg episode reward: 13.340, avg true_objective: 7.340 -[2024-07-24 18:29:42,479][03928] Num frames 3000... -[2024-07-24 18:29:42,617][03928] Num frames 3100... -[2024-07-24 18:29:42,754][03928] Num frames 3200... -[2024-07-24 18:29:42,903][03928] Num frames 3300... -[2024-07-24 18:29:43,038][03928] Num frames 3400... -[2024-07-24 18:29:43,177][03928] Num frames 3500... -[2024-07-24 18:29:43,316][03928] Num frames 3600... -[2024-07-24 18:29:43,385][03928] Avg episode rewards: #0: 13.016, true rewards: #0: 7.216 -[2024-07-24 18:29:43,387][03928] Avg episode reward: 13.016, avg true_objective: 7.216 -[2024-07-24 18:29:43,507][03928] Num frames 3700... -[2024-07-24 18:29:43,635][03928] Num frames 3800... -[2024-07-24 18:29:43,764][03928] Num frames 3900... -[2024-07-24 18:29:43,901][03928] Num frames 4000... -[2024-07-24 18:29:44,029][03928] Num frames 4100... -[2024-07-24 18:29:44,201][03928] Avg episode rewards: #0: 12.472, true rewards: #0: 6.972 -[2024-07-24 18:29:44,203][03928] Avg episode reward: 12.472, avg true_objective: 6.972 -[2024-07-24 18:29:44,227][03928] Num frames 4200... -[2024-07-24 18:29:44,358][03928] Num frames 4300... -[2024-07-24 18:29:44,491][03928] Num frames 4400... -[2024-07-24 18:29:44,621][03928] Num frames 4500... -[2024-07-24 18:29:44,754][03928] Num frames 4600... -[2024-07-24 18:29:44,898][03928] Num frames 4700... -[2024-07-24 18:29:45,031][03928] Num frames 4800... -[2024-07-24 18:29:45,173][03928] Num frames 4900... -[2024-07-24 18:29:45,304][03928] Num frames 5000... -[2024-07-24 18:29:45,436][03928] Num frames 5100... -[2024-07-24 18:29:45,548][03928] Avg episode rewards: #0: 13.776, true rewards: #0: 7.347 -[2024-07-24 18:29:45,549][03928] Avg episode reward: 13.776, avg true_objective: 7.347 -[2024-07-24 18:29:45,629][03928] Num frames 5200... -[2024-07-24 18:29:45,776][03928] Num frames 5300... -[2024-07-24 18:29:45,935][03928] Num frames 5400... -[2024-07-24 18:29:46,086][03928] Num frames 5500... -[2024-07-24 18:29:46,226][03928] Num frames 5600... -[2024-07-24 18:29:46,362][03928] Num frames 5700... -[2024-07-24 18:29:46,497][03928] Num frames 5800... -[2024-07-24 18:29:46,627][03928] Num frames 5900... -[2024-07-24 18:29:46,757][03928] Num frames 6000... -[2024-07-24 18:29:46,887][03928] Num frames 6100... -[2024-07-24 18:29:47,023][03928] Num frames 6200... -[2024-07-24 18:29:47,135][03928] Avg episode rewards: #0: 14.930, true rewards: #0: 7.805 -[2024-07-24 18:29:47,137][03928] Avg episode reward: 14.930, avg true_objective: 7.805 -[2024-07-24 18:29:47,220][03928] Num frames 6300... -[2024-07-24 18:29:47,349][03928] Num frames 6400... -[2024-07-24 18:29:47,482][03928] Num frames 6500... -[2024-07-24 18:29:47,612][03928] Num frames 6600... -[2024-07-24 18:29:47,793][03928] Avg episode rewards: #0: 14.213, true rewards: #0: 7.436 -[2024-07-24 18:29:47,795][03928] Avg episode reward: 14.213, avg true_objective: 7.436 -[2024-07-24 18:29:47,809][03928] Num frames 6700... -[2024-07-24 18:29:47,941][03928] Num frames 6800... -[2024-07-24 18:29:48,084][03928] Num frames 6900... -[2024-07-24 18:29:48,228][03928] Num frames 7000... -[2024-07-24 18:29:48,363][03928] Num frames 7100... -[2024-07-24 18:29:48,491][03928] Num frames 7200... -[2024-07-24 18:29:48,627][03928] Num frames 7300... -[2024-07-24 18:29:48,759][03928] Num frames 7400... -[2024-07-24 18:29:48,898][03928] Num frames 7500... -[2024-07-24 18:29:49,041][03928] Num frames 7600... -[2024-07-24 18:29:49,176][03928] Num frames 7700... -[2024-07-24 18:29:49,312][03928] Num frames 7800... -[2024-07-24 18:29:49,452][03928] Num frames 7900... -[2024-07-24 18:29:49,585][03928] Num frames 8000... -[2024-07-24 18:29:49,715][03928] Num frames 8100... -[2024-07-24 18:29:49,851][03928] Num frames 8200... -[2024-07-24 18:29:49,982][03928] Num frames 8300... -[2024-07-24 18:29:50,081][03928] Avg episode rewards: #0: 16.924, true rewards: #0: 8.324 -[2024-07-24 18:29:50,082][03928] Avg episode reward: 16.924, avg true_objective: 8.324 -[2024-07-24 18:30:43,566][03928] Replay video saved to /content/train_dir/default_experiment/replay.mp4! + wait_policy_total: 442.0098 +update_model: 9.3368 + weight_update: 0.0030 +one_step: 0.0384 + handle_policy_step: 629.0817 + deserialize: 16.6894, stack: 3.3098, obs_to_device_normalize: 127.2725, forward: 334.8990, send_messages: 30.1582 + prepare_outputs: 85.2321 + to_cpu: 49.7712 +[2024-07-24 19:19:43,433][02885] Learner 0 profile tree view: +misc: 0.0056, prepare_batch: 14.4996 +train: 73.8687 + epoch_init: 0.0143, minibatch_init: 0.0120, losses_postprocess: 0.5615, kl_divergence: 0.6758, after_optimizer: 34.0619 + calculate_losses: 26.7549 + losses_init: 0.0045, forward_head: 1.2823, bptt_initial: 17.5806, tail: 1.1827, advantages_returns: 0.2552, losses: 4.0035 + bptt: 2.1324 + bptt_forward_core: 2.0421 + update: 11.1515 + clip: 0.9352 +[2024-07-24 19:19:43,436][02885] RolloutWorker_w0 profile tree view: +wait_for_trajectories: 0.3334, enqueue_policy_requests: 118.7394, env_step: 873.2198, overhead: 15.5168, complete_rollouts: 8.2662 +save_policy_outputs: 21.5270 + split_output_tensors: 8.7756 +[2024-07-24 19:19:43,438][02885] RolloutWorker_w7 profile tree view: +wait_for_trajectories: 0.3855, enqueue_policy_requests: 118.1929, env_step: 870.2624, overhead: 15.5810, complete_rollouts: 6.8537 +save_policy_outputs: 21.5530 + split_output_tensors: 8.4852 +[2024-07-24 19:19:43,440][02885] Loop Runner_EvtLoop terminating... +[2024-07-24 19:19:43,442][02885] Runner profile tree view: +main_loop: 1150.4710 +[2024-07-24 19:19:43,443][02885] Collected {0: 4005888}, FPS: 3482.0 +[2024-07-24 19:30:16,511][02885] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json +[2024-07-24 19:30:16,514][02885] Overriding arg 'num_workers' with value 1 passed from command line +[2024-07-24 19:30:16,516][02885] Adding new argument 'no_render'=True that is not in the saved config file! +[2024-07-24 19:30:16,518][02885] Adding new argument 'save_video'=True that is not in the saved config file! +[2024-07-24 19:30:16,519][02885] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! +[2024-07-24 19:30:16,521][02885] Adding new argument 'video_name'=None that is not in the saved config file! +[2024-07-24 19:30:16,523][02885] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file! +[2024-07-24 19:30:16,526][02885] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! +[2024-07-24 19:30:16,527][02885] Adding new argument 'push_to_hub'=False that is not in the saved config file! +[2024-07-24 19:30:16,528][02885] Adding new argument 'hf_repository'=None that is not in the saved config file! +[2024-07-24 19:30:16,529][02885] Adding new argument 'policy_index'=0 that is not in the saved config file! +[2024-07-24 19:30:16,530][02885] Adding new argument 'eval_deterministic'=False that is not in the saved config file! +[2024-07-24 19:30:16,532][02885] Adding new argument 'train_script'=None that is not in the saved config file! +[2024-07-24 19:30:16,533][02885] Adding new argument 'enjoy_script'=None that is not in the saved config file! +[2024-07-24 19:30:16,534][02885] Using frameskip 1 and render_action_repeat=4 for evaluation +[2024-07-24 19:30:16,572][02885] Doom resolution: 160x120, resize resolution: (128, 72) +[2024-07-24 19:30:16,576][02885] RunningMeanStd input shape: (3, 72, 128) +[2024-07-24 19:30:16,578][02885] RunningMeanStd input shape: (1,) +[2024-07-24 19:30:16,595][02885] ConvEncoder: input_channels=3 +[2024-07-24 19:30:16,699][02885] Conv encoder output size: 512 +[2024-07-24 19:30:16,700][02885] Policy head output size: 512 +[2024-07-24 19:30:16,878][02885] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... +[2024-07-24 19:30:17,696][02885] Num frames 100... +[2024-07-24 19:30:17,872][02885] Num frames 200... +[2024-07-24 19:30:18,060][02885] Num frames 300... +[2024-07-24 19:30:18,249][02885] Num frames 400... +[2024-07-24 19:30:18,446][02885] Num frames 500... +[2024-07-24 19:30:18,626][02885] Num frames 600... +[2024-07-24 19:30:18,804][02885] Num frames 700... +[2024-07-24 19:30:18,986][02885] Num frames 800... +[2024-07-24 19:30:19,165][02885] Num frames 900... +[2024-07-24 19:30:19,359][02885] Num frames 1000... +[2024-07-24 19:30:19,579][02885] Avg episode rewards: #0: 23.880, true rewards: #0: 10.880 +[2024-07-24 19:30:19,581][02885] Avg episode reward: 23.880, avg true_objective: 10.880 +[2024-07-24 19:30:19,610][02885] Num frames 1100... +[2024-07-24 19:30:19,794][02885] Num frames 1200... +[2024-07-24 19:30:19,983][02885] Num frames 1300... +[2024-07-24 19:30:20,186][02885] Num frames 1400... +[2024-07-24 19:30:20,355][02885] Num frames 1500... +[2024-07-24 19:30:20,483][02885] Num frames 1600... +[2024-07-24 19:30:20,609][02885] Num frames 1700... +[2024-07-24 19:30:20,735][02885] Num frames 1800... +[2024-07-24 19:30:20,802][02885] Avg episode rewards: #0: 19.040, true rewards: #0: 9.040 +[2024-07-24 19:30:20,803][02885] Avg episode reward: 19.040, avg true_objective: 9.040 +[2024-07-24 19:30:20,930][02885] Num frames 1900... +[2024-07-24 19:30:21,066][02885] Num frames 2000... +[2024-07-24 19:30:21,194][02885] Num frames 2100... +[2024-07-24 19:30:21,321][02885] Num frames 2200... +[2024-07-24 19:30:21,461][02885] Num frames 2300... +[2024-07-24 19:30:21,585][02885] Num frames 2400... +[2024-07-24 19:30:21,710][02885] Num frames 2500... +[2024-07-24 19:30:21,843][02885] Num frames 2600... +[2024-07-24 19:30:21,983][02885] Num frames 2700... +[2024-07-24 19:30:22,112][02885] Num frames 2800... +[2024-07-24 19:30:22,255][02885] Avg episode rewards: #0: 20.557, true rewards: #0: 9.557 +[2024-07-24 19:30:22,258][02885] Avg episode reward: 20.557, avg true_objective: 9.557 +[2024-07-24 19:30:22,302][02885] Num frames 2900... +[2024-07-24 19:30:22,440][02885] Num frames 3000... +[2024-07-24 19:30:22,572][02885] Num frames 3100... +[2024-07-24 19:30:22,703][02885] Num frames 3200... +[2024-07-24 19:30:22,834][02885] Num frames 3300... +[2024-07-24 19:30:22,971][02885] Num frames 3400... +[2024-07-24 19:30:23,101][02885] Num frames 3500... +[2024-07-24 19:30:23,232][02885] Num frames 3600... +[2024-07-24 19:30:23,360][02885] Num frames 3700... +[2024-07-24 19:30:23,502][02885] Num frames 3800... +[2024-07-24 19:30:23,630][02885] Num frames 3900... +[2024-07-24 19:30:23,759][02885] Num frames 4000... +[2024-07-24 19:30:23,896][02885] Num frames 4100... +[2024-07-24 19:30:24,027][02885] Num frames 4200... +[2024-07-24 19:30:24,157][02885] Num frames 4300... +[2024-07-24 19:30:24,292][02885] Num frames 4400... +[2024-07-24 19:30:24,422][02885] Num frames 4500... +[2024-07-24 19:30:24,561][02885] Num frames 4600... +[2024-07-24 19:30:24,688][02885] Num frames 4700... +[2024-07-24 19:30:24,852][02885] Avg episode rewards: #0: 26.717, true rewards: #0: 11.967 +[2024-07-24 19:30:24,854][02885] Avg episode reward: 26.717, avg true_objective: 11.967 +[2024-07-24 19:30:24,873][02885] Num frames 4800... +[2024-07-24 19:30:25,007][02885] Num frames 4900... +[2024-07-24 19:30:25,134][02885] Num frames 5000... +[2024-07-24 19:30:25,269][02885] Num frames 5100... +[2024-07-24 19:30:25,401][02885] Num frames 5200... +[2024-07-24 19:30:25,538][02885] Num frames 5300... +[2024-07-24 19:30:25,664][02885] Num frames 5400... +[2024-07-24 19:30:25,791][02885] Num frames 5500... +[2024-07-24 19:30:25,920][02885] Num frames 5600... +[2024-07-24 19:30:26,044][02885] Num frames 5700... +[2024-07-24 19:30:26,146][02885] Avg episode rewards: #0: 26.074, true rewards: #0: 11.474 +[2024-07-24 19:30:26,148][02885] Avg episode reward: 26.074, avg true_objective: 11.474 +[2024-07-24 19:30:26,226][02885] Num frames 5800... +[2024-07-24 19:30:26,386][02885] Num frames 5900... +[2024-07-24 19:30:26,518][02885] Num frames 6000... +[2024-07-24 19:30:26,645][02885] Num frames 6100... +[2024-07-24 19:30:26,770][02885] Num frames 6200... +[2024-07-24 19:30:26,887][02885] Avg episode rewards: #0: 22.915, true rewards: #0: 10.415 +[2024-07-24 19:30:26,889][02885] Avg episode reward: 22.915, avg true_objective: 10.415 +[2024-07-24 19:30:26,961][02885] Num frames 6300... +[2024-07-24 19:30:27,093][02885] Num frames 6400... +[2024-07-24 19:30:27,218][02885] Num frames 6500... +[2024-07-24 19:30:27,344][02885] Num frames 6600... +[2024-07-24 19:30:27,478][02885] Num frames 6700... +[2024-07-24 19:30:27,616][02885] Num frames 6800... +[2024-07-24 19:30:27,742][02885] Num frames 6900... +[2024-07-24 19:30:27,903][02885] Avg episode rewards: #0: 21.550, true rewards: #0: 9.979 +[2024-07-24 19:30:27,906][02885] Avg episode reward: 21.550, avg true_objective: 9.979 +[2024-07-24 19:30:27,929][02885] Num frames 7000... +[2024-07-24 19:30:28,056][02885] Num frames 7100... +[2024-07-24 19:30:28,184][02885] Num frames 7200... +[2024-07-24 19:30:28,309][02885] Num frames 7300... +[2024-07-24 19:30:28,438][02885] Num frames 7400... +[2024-07-24 19:30:28,496][02885] Avg episode rewards: #0: 19.626, true rewards: #0: 9.251 +[2024-07-24 19:30:28,498][02885] Avg episode reward: 19.626, avg true_objective: 9.251 +[2024-07-24 19:30:28,641][02885] Num frames 7500... +[2024-07-24 19:30:28,775][02885] Num frames 7600... +[2024-07-24 19:30:28,902][02885] Num frames 7700... +[2024-07-24 19:30:29,031][02885] Num frames 7800... +[2024-07-24 19:30:29,158][02885] Num frames 7900... +[2024-07-24 19:30:29,286][02885] Num frames 8000... +[2024-07-24 19:30:29,415][02885] Num frames 8100... +[2024-07-24 19:30:29,480][02885] Avg episode rewards: #0: 18.894, true rewards: #0: 9.006 +[2024-07-24 19:30:29,483][02885] Avg episode reward: 18.894, avg true_objective: 9.006 +[2024-07-24 19:30:29,613][02885] Num frames 8200... +[2024-07-24 19:30:29,741][02885] Num frames 8300... +[2024-07-24 19:30:29,868][02885] Num frames 8400... +[2024-07-24 19:30:30,003][02885] Num frames 8500... +[2024-07-24 19:30:30,132][02885] Num frames 8600... +[2024-07-24 19:30:30,255][02885] Num frames 8700... +[2024-07-24 19:30:30,435][02885] Num frames 8800... +[2024-07-24 19:30:30,621][02885] Num frames 8900... +[2024-07-24 19:30:30,809][02885] Num frames 9000... +[2024-07-24 19:30:30,930][02885] Avg episode rewards: #0: 19.033, true rewards: #0: 9.033 +[2024-07-24 19:30:30,932][02885] Avg episode reward: 19.033, avg true_objective: 9.033 +[2024-07-24 19:31:26,493][02885] Replay video saved to /content/train_dir/default_experiment/replay.mp4! +[2024-07-24 19:33:00,770][02885] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json +[2024-07-24 19:33:00,772][02885] Overriding arg 'num_workers' with value 1 passed from command line +[2024-07-24 19:33:00,774][02885] Adding new argument 'no_render'=True that is not in the saved config file! +[2024-07-24 19:33:00,776][02885] Adding new argument 'save_video'=True that is not in the saved config file! +[2024-07-24 19:33:00,777][02885] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! +[2024-07-24 19:33:00,779][02885] Adding new argument 'video_name'=None that is not in the saved config file! +[2024-07-24 19:33:00,780][02885] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! +[2024-07-24 19:33:00,782][02885] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! +[2024-07-24 19:33:00,783][02885] Adding new argument 'push_to_hub'=True that is not in the saved config file! +[2024-07-24 19:33:00,784][02885] Adding new argument 'hf_repository'='dergky1/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! +[2024-07-24 19:33:00,785][02885] Adding new argument 'policy_index'=0 that is not in the saved config file! +[2024-07-24 19:33:00,786][02885] Adding new argument 'eval_deterministic'=False that is not in the saved config file! +[2024-07-24 19:33:00,788][02885] Adding new argument 'train_script'=None that is not in the saved config file! +[2024-07-24 19:33:00,789][02885] Adding new argument 'enjoy_script'=None that is not in the saved config file! +[2024-07-24 19:33:00,790][02885] Using frameskip 1 and render_action_repeat=4 for evaluation +[2024-07-24 19:33:00,821][02885] RunningMeanStd input shape: (3, 72, 128) +[2024-07-24 19:33:00,824][02885] RunningMeanStd input shape: (1,) +[2024-07-24 19:33:00,837][02885] ConvEncoder: input_channels=3 +[2024-07-24 19:33:00,876][02885] Conv encoder output size: 512 +[2024-07-24 19:33:00,877][02885] Policy head output size: 512 +[2024-07-24 19:33:00,899][02885] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... +[2024-07-24 19:33:01,348][02885] Num frames 100... +[2024-07-24 19:33:01,479][02885] Num frames 200... +[2024-07-24 19:33:01,609][02885] Num frames 300... +[2024-07-24 19:33:01,736][02885] Num frames 400... +[2024-07-24 19:33:01,863][02885] Num frames 500... +[2024-07-24 19:33:02,000][02885] Num frames 600... +[2024-07-24 19:33:02,135][02885] Num frames 700... +[2024-07-24 19:33:02,270][02885] Num frames 800... +[2024-07-24 19:33:02,402][02885] Num frames 900... +[2024-07-24 19:33:02,528][02885] Num frames 1000... +[2024-07-24 19:33:02,655][02885] Num frames 1100... +[2024-07-24 19:33:02,781][02885] Num frames 1200... +[2024-07-24 19:33:02,921][02885] Num frames 1300... +[2024-07-24 19:33:03,053][02885] Num frames 1400... +[2024-07-24 19:33:03,189][02885] Num frames 1500... +[2024-07-24 19:33:03,338][02885] Num frames 1600... +[2024-07-24 19:33:03,467][02885] Num frames 1700... +[2024-07-24 19:33:03,559][02885] Avg episode rewards: #0: 45.279, true rewards: #0: 17.280 +[2024-07-24 19:33:03,562][02885] Avg episode reward: 45.279, avg true_objective: 17.280 +[2024-07-24 19:33:03,656][02885] Num frames 1800... +[2024-07-24 19:33:03,785][02885] Num frames 1900... +[2024-07-24 19:33:03,919][02885] Num frames 2000... +[2024-07-24 19:33:04,049][02885] Num frames 2100... +[2024-07-24 19:33:04,180][02885] Num frames 2200... +[2024-07-24 19:33:04,316][02885] Num frames 2300... +[2024-07-24 19:33:04,447][02885] Num frames 2400... +[2024-07-24 19:33:04,575][02885] Num frames 2500... +[2024-07-24 19:33:04,706][02885] Num frames 2600... +[2024-07-24 19:33:04,832][02885] Num frames 2700... +[2024-07-24 19:33:05,004][02885] Avg episode rewards: #0: 35.920, true rewards: #0: 13.920 +[2024-07-24 19:33:05,006][02885] Avg episode reward: 35.920, avg true_objective: 13.920 +[2024-07-24 19:33:05,031][02885] Num frames 2800... +[2024-07-24 19:33:05,160][02885] Num frames 2900... +[2024-07-24 19:33:05,298][02885] Num frames 3000... +[2024-07-24 19:33:05,427][02885] Num frames 3100... +[2024-07-24 19:33:05,554][02885] Num frames 3200... +[2024-07-24 19:33:05,680][02885] Num frames 3300... +[2024-07-24 19:33:05,808][02885] Num frames 3400... +[2024-07-24 19:33:05,948][02885] Num frames 3500... +[2024-07-24 19:33:06,077][02885] Num frames 3600... +[2024-07-24 19:33:06,212][02885] Num frames 3700... +[2024-07-24 19:33:06,346][02885] Num frames 3800... +[2024-07-24 19:33:06,474][02885] Num frames 3900... +[2024-07-24 19:33:06,604][02885] Num frames 4000... +[2024-07-24 19:33:06,717][02885] Avg episode rewards: #0: 33.813, true rewards: #0: 13.480 +[2024-07-24 19:33:06,718][02885] Avg episode reward: 33.813, avg true_objective: 13.480 +[2024-07-24 19:33:06,791][02885] Num frames 4100... +[2024-07-24 19:33:06,952][02885] Num frames 4200... +[2024-07-24 19:33:07,147][02885] Num frames 4300... +[2024-07-24 19:33:07,342][02885] Num frames 4400... +[2024-07-24 19:33:07,528][02885] Num frames 4500... +[2024-07-24 19:33:07,710][02885] Num frames 4600... +[2024-07-24 19:33:07,889][02885] Num frames 4700... +[2024-07-24 19:33:08,081][02885] Num frames 4800... +[2024-07-24 19:33:08,268][02885] Num frames 4900... +[2024-07-24 19:33:08,469][02885] Num frames 5000... +[2024-07-24 19:33:08,659][02885] Num frames 5100... +[2024-07-24 19:33:08,848][02885] Num frames 5200... +[2024-07-24 19:33:09,025][02885] Avg episode rewards: #0: 31.400, true rewards: #0: 13.150 +[2024-07-24 19:33:09,028][02885] Avg episode reward: 31.400, avg true_objective: 13.150 +[2024-07-24 19:33:09,114][02885] Num frames 5300... +[2024-07-24 19:33:09,305][02885] Num frames 5400... +[2024-07-24 19:33:09,452][02885] Num frames 5500... +[2024-07-24 19:33:09,578][02885] Num frames 5600... +[2024-07-24 19:33:09,705][02885] Num frames 5700... +[2024-07-24 19:33:09,833][02885] Num frames 5800... +[2024-07-24 19:33:09,987][02885] Avg episode rewards: #0: 27.136, true rewards: #0: 11.736 +[2024-07-24 19:33:09,990][02885] Avg episode reward: 27.136, avg true_objective: 11.736 +[2024-07-24 19:33:10,035][02885] Num frames 5900... +[2024-07-24 19:33:10,168][02885] Num frames 6000... +[2024-07-24 19:33:10,295][02885] Num frames 6100... +[2024-07-24 19:33:10,423][02885] Num frames 6200... +[2024-07-24 19:33:10,564][02885] Num frames 6300... +[2024-07-24 19:33:10,690][02885] Num frames 6400... +[2024-07-24 19:33:10,813][02885] Num frames 6500... +[2024-07-24 19:33:10,946][02885] Num frames 6600... +[2024-07-24 19:33:11,072][02885] Num frames 6700... +[2024-07-24 19:33:11,204][02885] Num frames 6800... +[2024-07-24 19:33:11,331][02885] Num frames 6900... +[2024-07-24 19:33:11,459][02885] Num frames 7000... +[2024-07-24 19:33:11,594][02885] Num frames 7100... +[2024-07-24 19:33:11,724][02885] Num frames 7200... +[2024-07-24 19:33:11,851][02885] Num frames 7300... +[2024-07-24 19:33:11,988][02885] Num frames 7400... +[2024-07-24 19:33:12,118][02885] Num frames 7500... +[2024-07-24 19:33:12,246][02885] Num frames 7600... +[2024-07-24 19:33:12,377][02885] Num frames 7700... +[2024-07-24 19:33:12,513][02885] Num frames 7800... +[2024-07-24 19:33:12,644][02885] Num frames 7900... +[2024-07-24 19:33:12,768][02885] Avg episode rewards: #0: 32.256, true rewards: #0: 13.257 +[2024-07-24 19:33:12,770][02885] Avg episode reward: 32.256, avg true_objective: 13.257 +[2024-07-24 19:33:12,829][02885] Num frames 8000... +[2024-07-24 19:33:12,960][02885] Num frames 8100... +[2024-07-24 19:33:13,084][02885] Num frames 8200... +[2024-07-24 19:33:13,211][02885] Num frames 8300... +[2024-07-24 19:33:13,341][02885] Num frames 8400... +[2024-07-24 19:33:13,401][02885] Avg episode rewards: #0: 28.431, true rewards: #0: 12.003 +[2024-07-24 19:33:13,403][02885] Avg episode reward: 28.431, avg true_objective: 12.003 +[2024-07-24 19:33:13,530][02885] Num frames 8500... +[2024-07-24 19:33:13,661][02885] Num frames 8600... +[2024-07-24 19:33:13,786][02885] Num frames 8700... +[2024-07-24 19:33:13,918][02885] Num frames 8800... +[2024-07-24 19:33:14,052][02885] Num frames 8900... +[2024-07-24 19:33:14,129][02885] Avg episode rewards: #0: 26.267, true rewards: #0: 11.142 +[2024-07-24 19:33:14,131][02885] Avg episode reward: 26.267, avg true_objective: 11.142 +[2024-07-24 19:33:14,242][02885] Num frames 9000... +[2024-07-24 19:33:14,369][02885] Num frames 9100... +[2024-07-24 19:33:14,503][02885] Num frames 9200... +[2024-07-24 19:33:14,642][02885] Num frames 9300... +[2024-07-24 19:33:14,770][02885] Num frames 9400... +[2024-07-24 19:33:14,906][02885] Num frames 9500... +[2024-07-24 19:33:14,994][02885] Avg episode rewards: #0: 24.691, true rewards: #0: 10.580 +[2024-07-24 19:33:14,996][02885] Avg episode reward: 24.691, avg true_objective: 10.580 +[2024-07-24 19:33:15,097][02885] Num frames 9600... +[2024-07-24 19:33:15,231][02885] Num frames 9700... +[2024-07-24 19:33:15,358][02885] Num frames 9800... +[2024-07-24 19:33:15,489][02885] Num frames 9900... +[2024-07-24 19:33:15,594][02885] Avg episode rewards: #0: 22.838, true rewards: #0: 9.938 +[2024-07-24 19:33:15,596][02885] Avg episode reward: 22.838, avg true_objective: 9.938 +[2024-07-24 19:34:17,803][02885] Replay video saved to /content/train_dir/default_experiment/replay.mp4!