Upload folder using huggingface_hub

.summary/0/events.out.tfevents.1696695316.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:902e041b5c1c430100d9af0f274b74f017bbbf8a0b41af0b5af56485654024c1
+size 33340

.summary/1/events.out.tfevents.1696695316.rhmmedcatt-proliant-ml350-gen10

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e7238c487dd28c7487cd55f018fb9d93d98ca6026bdaf2e69ec3fe13ac6b4cd6
+size 25944

README.md

@@ -15,7 +15,7 @@ model-index:
       type: atari_alien
     metrics:
     - type: mean_reward
-      value: 354.00 +/- 198.66
+      value: 372.00 +/- 246.77
       name: mean_reward
       verified: false
 ---

checkpoint_p0/best_000000608_155648_reward_9.290.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:96dcc549870abaca6379b773facf45b3fe2f7f0c31198c940bb8861013a207d7
+size 20795763

checkpoint_p0/checkpoint_000000704_180224.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:85890b74db653455754b904bbfbac5f84d0d8054544f261959b12bfbb5e08832
+size 20796099

checkpoint_p1/best_000000608_155648_reward_8.640.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:300a812f23c3be557009506362ff5c023a902a07956dc9d1678d3236a979f884
+size 20795763

checkpoint_p1/checkpoint_000000704_180224.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c4cba4a6c2451f53459312418991053e01c3325457e0d163393dbe8454ef8153
+size 20796099

config.json

@@ -146,5 +146,5 @@
   },
   "git_hash": "5fff97c2f535da5987d358cdbe6927cccd43621e",
   "git_repo_name": "not a git repository",
-  "wandb_unique_id": "atari_alien_APPO_20231007_171310_311603"
+  "wandb_unique_id": "atari_alien_APPO_20231007_171513_616259"
 }

replay.mp4

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:bf812b396ea7f2c4b30ed70292246de2db54668f1689c039176f868a9586f02c
-size 1479645
+oid sha256:b1d89dbf51b345e273df907ff92e76737061e0383c0d5cb375eec05ad92d6572
+size 1600856

sf_log.txt

@@ -1,40 +1,40 @@
-[2023-10-07 17:13:16,813][67527] Saving configuration to ./train_atari/atari_alien_APPO/config.json...
-[2023-10-07 17:13:17,130][67527] Rollout worker 0 uses device cpu
-[2023-10-07 17:13:17,131][67527] Rollout worker 1 uses device cpu
-[2023-10-07 17:13:17,132][67527] Rollout worker 2 uses device cpu
-[2023-10-07 17:13:17,132][67527] Rollout worker 3 uses device cpu
-[2023-10-07 17:13:17,133][67527] Rollout worker 4 uses device cpu
-[2023-10-07 17:13:17,133][67527] Rollout worker 5 uses device cpu
-[2023-10-07 17:13:17,134][67527] Rollout worker 6 uses device cpu
-[2023-10-07 17:13:17,135][67527] Rollout worker 7 uses device cpu
-[2023-10-07 17:13:17,135][67527] Rollout worker 8 uses device cpu
-[2023-10-07 17:13:17,136][67527] Rollout worker 9 uses device cpu
-[2023-10-07 17:13:17,136][67527] Rollout worker 10 uses device cpu
-[2023-10-07 17:13:17,137][67527] Rollout worker 11 uses device cpu
-[2023-10-07 17:13:17,137][67527] Rollout worker 12 uses device cpu
-[2023-10-07 17:13:17,138][67527] Rollout worker 13 uses device cpu
-[2023-10-07 17:13:17,138][67527] Rollout worker 14 uses device cpu
-[2023-10-07 17:13:17,139][67527] Rollout worker 15 uses device cpu
-[2023-10-07 17:13:17,139][67527] In synchronous mode, we only accumulate one batch. Setting num_batches_to_accumulate to 1
-[2023-10-07 17:13:17,233][67527] Using GPUs [0] for process 0 (actually maps to GPUs [0])
-[2023-10-07 17:13:17,234][67527] InferenceWorker_p0-w0: min num requests: 2
-[2023-10-07 17:13:17,237][67527] Using GPUs [1] for process 1 (actually maps to GPUs [1])
-[2023-10-07 17:13:17,237][67527] InferenceWorker_p1-w0: min num requests: 2
-[2023-10-07 17:13:17,282][67527] Starting all processes...
-[2023-10-07 17:13:17,283][67527] Starting process learner_proc0
-[2023-10-07 17:13:18,970][67527] Starting process learner_proc1
-[2023-10-07 17:13:18,974][68242] Using GPUs [0] for process 0 (actually maps to GPUs [0])
-[2023-10-07 17:13:18,974][68242] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0
-[2023-10-07 17:13:18,992][68242] Num visible devices: 1
-[2023-10-07 17:13:19,014][68242] Setting fixed seed 1234
-[2023-10-07 17:13:19,016][68242] Using GPUs [0] for process 0 (actually maps to GPUs [0])
-[2023-10-07 17:13:19,016][68242] Initializing actor-critic model on device cuda:0
-[2023-10-07 17:13:19,017][68242] RunningMeanStd input shape: (4, 84, 84)
-[2023-10-07 17:13:19,018][68242] RunningMeanStd input shape: (1,)
-[2023-10-07 17:13:19,036][68242] ConvEncoder: input_channels=4
-[2023-10-07 17:13:19,211][68242] Conv encoder output size: 512
-[2023-10-07 17:13:19,213][68242] Created Actor Critic model with architecture:
-[2023-10-07 17:13:19,213][68242] ActorCriticSharedWeights(
+[2023-10-07 17:15:19,910][76104] Saving configuration to ./train_atari/atari_alien_APPO/config.json...
+[2023-10-07 17:15:20,227][76104] Rollout worker 0 uses device cpu
+[2023-10-07 17:15:20,227][76104] Rollout worker 1 uses device cpu
+[2023-10-07 17:15:20,228][76104] Rollout worker 2 uses device cpu
+[2023-10-07 17:15:20,229][76104] Rollout worker 3 uses device cpu
+[2023-10-07 17:15:20,229][76104] Rollout worker 4 uses device cpu
+[2023-10-07 17:15:20,229][76104] Rollout worker 5 uses device cpu
+[2023-10-07 17:15:20,230][76104] Rollout worker 6 uses device cpu
+[2023-10-07 17:15:20,230][76104] Rollout worker 7 uses device cpu
+[2023-10-07 17:15:20,231][76104] Rollout worker 8 uses device cpu
+[2023-10-07 17:15:20,231][76104] Rollout worker 9 uses device cpu
+[2023-10-07 17:15:20,232][76104] Rollout worker 10 uses device cpu
+[2023-10-07 17:15:20,232][76104] Rollout worker 11 uses device cpu
+[2023-10-07 17:15:20,233][76104] Rollout worker 12 uses device cpu
+[2023-10-07 17:15:20,233][76104] Rollout worker 13 uses device cpu
+[2023-10-07 17:15:20,234][76104] Rollout worker 14 uses device cpu
+[2023-10-07 17:15:20,234][76104] Rollout worker 15 uses device cpu
+[2023-10-07 17:15:20,234][76104] In synchronous mode, we only accumulate one batch. Setting num_batches_to_accumulate to 1
+[2023-10-07 17:15:20,326][76104] Using GPUs [0] for process 0 (actually maps to GPUs [0])
+[2023-10-07 17:15:20,326][76104] InferenceWorker_p0-w0: min num requests: 2
+[2023-10-07 17:15:20,329][76104] Using GPUs [1] for process 1 (actually maps to GPUs [1])
+[2023-10-07 17:15:20,330][76104] InferenceWorker_p1-w0: min num requests: 2
+[2023-10-07 17:15:20,374][76104] Starting all processes...
+[2023-10-07 17:15:20,375][76104] Starting process learner_proc0
+[2023-10-07 17:15:22,041][76104] Starting process learner_proc1
+[2023-10-07 17:15:22,045][76817] Using GPUs [0] for process 0 (actually maps to GPUs [0])
+[2023-10-07 17:15:22,045][76817] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0
+[2023-10-07 17:15:22,074][76817] Num visible devices: 1
+[2023-10-07 17:15:22,094][76817] Setting fixed seed 1234
+[2023-10-07 17:15:22,095][76817] Using GPUs [0] for process 0 (actually maps to GPUs [0])
+[2023-10-07 17:15:22,095][76817] Initializing actor-critic model on device cuda:0
+[2023-10-07 17:15:22,096][76817] RunningMeanStd input shape: (4, 84, 84)
+[2023-10-07 17:15:22,096][76817] RunningMeanStd input shape: (1,)
+[2023-10-07 17:15:22,107][76817] ConvEncoder: input_channels=4
+[2023-10-07 17:15:22,261][76817] Conv encoder output size: 512
+[2023-10-07 17:15:22,263][76817] Created Actor Critic model with architecture:
+[2023-10-07 17:15:22,263][76817] ActorCriticSharedWeights(
   (obs_normalizer): ObservationNormalizer(
     (running_mean_std): RunningMeanStdDictInPlace(
       (running_mean_std): ModuleDict(
@@ -75,41 +75,41 @@
     (distribution_linear): Linear(in_features=512, out_features=18, bias=True)
   )
 )
-[2023-10-07 17:13:19,747][68242] Using optimizer <class 'torch.optim.adam.Adam'>
-[2023-10-07 17:13:19,747][68242] No checkpoints found
-[2023-10-07 17:13:19,747][68242] Did not load from checkpoint, starting from scratch!
-[2023-10-07 17:13:19,747][68242] Initialized policy 0 weights for model version 0
-[2023-10-07 17:13:19,749][68242] LearnerWorker_p0 finished initialization!
-[2023-10-07 17:13:19,749][68242] Using GPUs [0] for process 0 (actually maps to GPUs [0])
-[2023-10-07 17:13:20,730][67527] Starting all processes...
-[2023-10-07 17:13:20,735][68341] Using GPUs [1] for process 1 (actually maps to GPUs [1])
-[2023-10-07 17:13:20,736][68341] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [1]) for learning process 1
-[2023-10-07 17:13:20,737][67527] Starting process inference_proc0-0
-[2023-10-07 17:13:20,737][67527] Starting process inference_proc1-0
-[2023-10-07 17:13:20,738][67527] Starting process rollout_proc0
-[2023-10-07 17:13:20,738][67527] Starting process rollout_proc1
-[2023-10-07 17:13:20,738][67527] Starting process rollout_proc2
-[2023-10-07 17:13:20,755][68341] Num visible devices: 1
-[2023-10-07 17:13:20,738][67527] Starting process rollout_proc3
-[2023-10-07 17:13:20,739][67527] Starting process rollout_proc4
-[2023-10-07 17:13:20,777][68341] Setting fixed seed 1234
-[2023-10-07 17:13:20,741][67527] Starting process rollout_proc5
-[2023-10-07 17:13:20,779][68341] Using GPUs [0] for process 1 (actually maps to GPUs [1])
-[2023-10-07 17:13:20,779][68341] Initializing actor-critic model on device cuda:0
-[2023-10-07 17:13:20,743][67527] Starting process rollout_proc6
-[2023-10-07 17:13:20,779][68341] RunningMeanStd input shape: (4, 84, 84)
-[2023-10-07 17:13:20,744][67527] Starting process rollout_proc7
-[2023-10-07 17:13:20,780][68341] RunningMeanStd input shape: (1,)
-[2023-10-07 17:13:20,747][67527] Starting process rollout_proc8
-[2023-10-07 17:13:20,749][67527] Starting process rollout_proc9
-[2023-10-07 17:13:20,752][67527] Starting process rollout_proc10
-[2023-10-07 17:13:20,754][67527] Starting process rollout_proc11
-[2023-10-07 17:13:20,755][67527] Starting process rollout_proc12
-[2023-10-07 17:13:20,757][67527] Starting process rollout_proc13
-[2023-10-07 17:13:20,800][68341] ConvEncoder: input_channels=4
-[2023-10-07 17:13:21,270][68341] Conv encoder output size: 512
-[2023-10-07 17:13:21,273][68341] Created Actor Critic model with architecture:
-[2023-10-07 17:13:21,273][68341] ActorCriticSharedWeights(
+[2023-10-07 17:15:22,807][76817] Using optimizer <class 'torch.optim.adam.Adam'>
+[2023-10-07 17:15:22,807][76817] No checkpoints found
+[2023-10-07 17:15:22,807][76817] Did not load from checkpoint, starting from scratch!
+[2023-10-07 17:15:22,808][76817] Initialized policy 0 weights for model version 0
+[2023-10-07 17:15:22,809][76817] LearnerWorker_p0 finished initialization!
+[2023-10-07 17:15:22,809][76817] Using GPUs [0] for process 0 (actually maps to GPUs [0])
+[2023-10-07 17:15:23,740][76104] Starting all processes...
+[2023-10-07 17:15:23,744][76906] Using GPUs [1] for process 1 (actually maps to GPUs [1])
+[2023-10-07 17:15:23,744][76906] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [1]) for learning process 1
+[2023-10-07 17:15:23,762][76906] Num visible devices: 1
+[2023-10-07 17:15:23,796][76906] Setting fixed seed 1234
+[2023-10-07 17:15:23,797][76906] Using GPUs [0] for process 1 (actually maps to GPUs [1])
+[2023-10-07 17:15:23,798][76906] Initializing actor-critic model on device cuda:0
+[2023-10-07 17:15:23,798][76104] Starting process inference_proc0-0
+[2023-10-07 17:15:23,798][76906] RunningMeanStd input shape: (4, 84, 84)
+[2023-10-07 17:15:23,799][76906] RunningMeanStd input shape: (1,)
+[2023-10-07 17:15:23,798][76104] Starting process inference_proc1-0
+[2023-10-07 17:15:23,798][76104] Starting process rollout_proc0
+[2023-10-07 17:15:23,798][76104] Starting process rollout_proc1
+[2023-10-07 17:15:23,798][76104] Starting process rollout_proc2
+[2023-10-07 17:15:23,798][76104] Starting process rollout_proc3
+[2023-10-07 17:15:23,800][76104] Starting process rollout_proc4
+[2023-10-07 17:15:23,810][76906] ConvEncoder: input_channels=4
+[2023-10-07 17:15:23,801][76104] Starting process rollout_proc5
+[2023-10-07 17:15:23,802][76104] Starting process rollout_proc6
+[2023-10-07 17:15:23,803][76104] Starting process rollout_proc7
+[2023-10-07 17:15:23,803][76104] Starting process rollout_proc8
+[2023-10-07 17:15:23,804][76104] Starting process rollout_proc9
+[2023-10-07 17:15:23,805][76104] Starting process rollout_proc10
+[2023-10-07 17:15:23,805][76104] Starting process rollout_proc11
+[2023-10-07 17:15:23,806][76104] Starting process rollout_proc12
+[2023-10-07 17:15:23,807][76104] Starting process rollout_proc13
+[2023-10-07 17:15:24,320][76906] Conv encoder output size: 512
+[2023-10-07 17:15:24,323][76906] Created Actor Critic model with architecture:
+[2023-10-07 17:15:24,323][76906] ActorCriticSharedWeights(
   (obs_normalizer): ObservationNormalizer(
     (running_mean_std): RunningMeanStdDictInPlace(
       (running_mean_std): ModuleDict(
@@ -150,180 +150,177 @@
     (distribution_linear): Linear(in_features=512, out_features=18, bias=True)
   )
 )
-[2023-10-07 17:13:21,871][68341] Using optimizer <class 'torch.optim.adam.Adam'>
-[2023-10-07 17:13:21,872][68341] No checkpoints found
-[2023-10-07 17:13:21,872][68341] Did not load from checkpoint, starting from scratch!
-[2023-10-07 17:13:21,872][68341] Initialized policy 1 weights for model version 0
-[2023-10-07 17:13:21,874][68341] LearnerWorker_p1 finished initialization!
-[2023-10-07 17:13:21,874][68341] Using GPUs [0] for process 1 (actually maps to GPUs [1])
-[2023-10-07 17:13:22,965][67527] Starting process rollout_proc14
-[2023-10-07 17:13:22,969][68536] Worker 7 uses CPU cores [14, 15]
-[2023-10-07 17:13:22,992][67527] Starting process rollout_proc15
-[2023-10-07 17:13:22,997][68540] Worker 11 uses CPU cores [22, 23]
-[2023-10-07 17:13:23,032][68533] Worker 4 uses CPU cores [8, 9]
-[2023-10-07 17:13:23,108][68541] Worker 12 uses CPU cores [24, 25]
-[2023-10-07 17:13:23,152][68537] Worker 8 uses CPU cores [16, 17]
-[2023-10-07 17:13:23,281][68530] Worker 2 uses CPU cores [4, 5]
-[2023-10-07 17:13:23,308][68498] Worker 1 uses CPU cores [2, 3]
-[2023-10-07 17:13:23,312][68497] Worker 0 uses CPU cores [0, 1]
-[2023-10-07 17:13:23,324][68531] Worker 3 uses CPU cores [6, 7]
-[2023-10-07 17:13:23,325][68535] Worker 6 uses CPU cores [12, 13]
-[2023-10-07 17:13:23,325][68538] Worker 9 uses CPU cores [18, 19]
-[2023-10-07 17:13:23,437][68539] Worker 10 uses CPU cores [20, 21]
-[2023-10-07 17:13:23,482][68491] Using GPUs [0] for process 0 (actually maps to GPUs [0])
-[2023-10-07 17:13:23,482][68491] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0
-[2023-10-07 17:13:23,500][68491] Num visible devices: 1
-[2023-10-07 17:13:23,697][68496] Using GPUs [1] for process 1 (actually maps to GPUs [1])
-[2023-10-07 17:13:23,697][68496] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [1]) for inference process 1
-[2023-10-07 17:13:23,702][68534] Worker 5 uses CPU cores [10, 11]
-[2023-10-07 17:13:23,715][68496] Num visible devices: 1
-[2023-10-07 17:13:23,838][68542] Worker 13 uses CPU cores [26, 27]
-[2023-10-07 17:13:24,136][68491] RunningMeanStd input shape: (4, 84, 84)
-[2023-10-07 17:13:24,137][68491] RunningMeanStd input shape: (1,)
-[2023-10-07 17:13:24,148][68491] ConvEncoder: input_channels=4
-[2023-10-07 17:13:24,251][68491] Conv encoder output size: 512
-[2023-10-07 17:13:24,337][68496] RunningMeanStd input shape: (4, 84, 84)
-[2023-10-07 17:13:24,338][68496] RunningMeanStd input shape: (1,)
-[2023-10-07 17:13:24,349][68496] ConvEncoder: input_channels=4
-[2023-10-07 17:13:24,450][68496] Conv encoder output size: 512
-[2023-10-07 17:13:24,966][69313] Worker 14 uses CPU cores [28, 29]
-[2023-10-07 17:13:25,058][67527] Inference worker 0-0 is ready!
-[2023-10-07 17:13:25,059][69345] Worker 15 uses CPU cores [30, 31]
-[2023-10-07 17:13:25,059][67527] Inference worker 1-0 is ready!
-[2023-10-07 17:13:25,060][67527] All inference workers are ready! Signal rollout workers to start!
-[2023-10-07 17:13:25,061][67527] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan, 1: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
-[2023-10-07 17:13:25,762][68541] Decorrelating experience for 0 frames...
-[2023-10-07 17:13:25,766][68497] Decorrelating experience for 0 frames...
-[2023-10-07 17:13:25,780][68533] Decorrelating experience for 0 frames...
-[2023-10-07 17:13:25,783][68537] Decorrelating experience for 0 frames...
-[2023-10-07 17:13:25,784][68536] Decorrelating experience for 0 frames...
-[2023-10-07 17:13:25,853][69345] Decorrelating experience for 0 frames...
-[2023-10-07 17:13:25,981][68541] Decorrelating experience for 128 frames...
-[2023-10-07 17:13:25,982][68497] Decorrelating experience for 128 frames...
-[2023-10-07 17:13:26,003][68533] Decorrelating experience for 128 frames...
-[2023-10-07 17:13:26,007][68537] Decorrelating experience for 128 frames...
-[2023-10-07 17:13:26,008][68536] Decorrelating experience for 128 frames...
-[2023-10-07 17:13:26,075][69345] Decorrelating experience for 128 frames...
-[2023-10-07 17:13:26,089][68542] Decorrelating experience for 0 frames...
-[2023-10-07 17:13:26,105][68534] Decorrelating experience for 0 frames...
-[2023-10-07 17:13:26,278][68535] Decorrelating experience for 0 frames...
-[2023-10-07 17:13:26,306][68542] Decorrelating experience for 128 frames...
-[2023-10-07 17:13:26,318][68530] Decorrelating experience for 0 frames...
-[2023-10-07 17:13:26,321][68539] Decorrelating experience for 0 frames...
-[2023-10-07 17:13:26,325][68534] Decorrelating experience for 128 frames...
-[2023-10-07 17:13:26,326][68540] Decorrelating experience for 0 frames...
-[2023-10-07 17:13:26,327][68498] Decorrelating experience for 0 frames...
-[2023-10-07 17:13:26,329][68538] Decorrelating experience for 0 frames...
-[2023-10-07 17:13:26,333][68531] Decorrelating experience for 0 frames...
-[2023-10-07 17:13:26,421][69313] Decorrelating experience for 0 frames...
-[2023-10-07 17:13:26,681][68535] Decorrelating experience for 128 frames...
-[2023-10-07 17:13:26,734][68539] Decorrelating experience for 128 frames...
-[2023-10-07 17:13:26,735][68540] Decorrelating experience for 128 frames...
-[2023-10-07 17:13:26,735][68530] Decorrelating experience for 128 frames...
-[2023-10-07 17:13:26,740][68531] Decorrelating experience for 128 frames...
-[2023-10-07 17:13:26,740][68498] Decorrelating experience for 128 frames...
-[2023-10-07 17:13:26,745][68538] Decorrelating experience for 128 frames...
-[2023-10-07 17:13:26,828][69313] Decorrelating experience for 128 frames...
-[2023-10-07 17:13:27,914][67527] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 184.4, 1: 326.4. Samples: 1457. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
-[2023-10-07 17:13:27,915][67527] Avg episode reward: [(1, '1.750')]
-[2023-10-07 17:13:32,914][67527] Fps is (10 sec: 2086.5, 60 sec: 2086.5, 300 sec: 2086.5). Total num frames: 16384. Throughput: 0: 260.8, 1: 260.8. Samples: 4096. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0)
-[2023-10-07 17:13:32,915][67527] Avg episode reward: [(0, '8.417'), (1, '3.278')]
-[2023-10-07 17:13:37,221][67527] Heartbeat connected on Batcher_0
-[2023-10-07 17:13:37,224][67527] Heartbeat connected on LearnerWorker_p0
-[2023-10-07 17:13:37,227][67527] Heartbeat connected on Batcher_1
-[2023-10-07 17:13:37,229][67527] Heartbeat connected on LearnerWorker_p1
-[2023-10-07 17:13:37,236][67527] Heartbeat connected on InferenceWorker_p0-w0
-[2023-10-07 17:13:37,239][67527] Heartbeat connected on InferenceWorker_p1-w0
-[2023-10-07 17:13:37,244][67527] Heartbeat connected on RolloutWorker_w0
-[2023-10-07 17:13:37,245][67527] Heartbeat connected on RolloutWorker_w1
-[2023-10-07 17:13:37,251][67527] Heartbeat connected on RolloutWorker_w4
-[2023-10-07 17:13:37,252][67527] Heartbeat connected on RolloutWorker_w2
-[2023-10-07 17:13:37,252][67527] Heartbeat connected on RolloutWorker_w3
-[2023-10-07 17:13:37,256][67527] Heartbeat connected on RolloutWorker_w5
-[2023-10-07 17:13:37,258][67527] Heartbeat connected on RolloutWorker_w6
-[2023-10-07 17:13:37,265][67527] Heartbeat connected on RolloutWorker_w9
-[2023-10-07 17:13:37,267][67527] Heartbeat connected on RolloutWorker_w7
-[2023-10-07 17:13:37,267][67527] Heartbeat connected on RolloutWorker_w8
-[2023-10-07 17:13:37,273][67527] Heartbeat connected on RolloutWorker_w10
-[2023-10-07 17:13:37,276][67527] Heartbeat connected on RolloutWorker_w11
-[2023-10-07 17:13:37,278][67527] Heartbeat connected on RolloutWorker_w12
-[2023-10-07 17:13:37,280][67527] Heartbeat connected on RolloutWorker_w13
-[2023-10-07 17:13:37,280][67527] Heartbeat connected on RolloutWorker_w14
-[2023-10-07 17:13:37,288][67527] Heartbeat connected on RolloutWorker_w15
-[2023-10-07 17:13:37,914][67527] Fps is (10 sec: 6553.6, 60 sec: 5099.1, 300 sec: 5099.1). Total num frames: 65536. Throughput: 0: 599.0, 1: 627.0. Samples: 15757. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0)
-[2023-10-07 17:13:37,915][67527] Avg episode reward: [(0, '7.379'), (1, '4.000')]
-[2023-10-07 17:13:42,914][67527] Fps is (10 sec: 9830.4, 60 sec: 6424.2, 300 sec: 6424.2). Total num frames: 114688. Throughput: 0: 803.0, 1: 813.4. Samples: 28857. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0)
-[2023-10-07 17:13:42,915][67527] Avg episode reward: [(0, '7.439'), (1, '5.736')]
-[2023-10-07 17:13:47,404][68496] Updated weights for policy 1, policy_version 320 (0.0019)
-[2023-10-07 17:13:47,404][68491] Updated weights for policy 0, policy_version 320 (0.0018)
-[2023-10-07 17:13:47,914][67527] Fps is (10 sec: 9830.5, 60 sec: 7169.5, 300 sec: 7169.5). Total num frames: 163840. Throughput: 0: 806.6, 1: 806.6. Samples: 36864. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0)
-[2023-10-07 17:13:47,915][67527] Avg episode reward: [(0, '7.338'), (1, '5.790')]
-[2023-10-07 17:13:52,914][67527] Fps is (10 sec: 9830.3, 60 sec: 7647.1, 300 sec: 7647.1). Total num frames: 212992. Throughput: 0: 915.7, 1: 929.6. Samples: 51396. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0)
-[2023-10-07 17:13:52,915][67527] Avg episode reward: [(0, '7.480'), (1, '6.820')]
-[2023-10-07 17:13:52,916][68242] Saving new best policy, reward=7.480!
-[2023-10-07 17:13:52,916][68341] Saving new best policy, reward=6.820!
-[2023-10-07 17:13:57,914][67527] Fps is (10 sec: 9830.3, 60 sec: 7979.4, 300 sec: 7979.4). Total num frames: 262144. Throughput: 0: 997.4, 1: 997.4. Samples: 65536. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0)
-[2023-10-07 17:13:57,915][67527] Avg episode reward: [(0, '7.480'), (1, '7.570')]
-[2023-10-07 17:13:57,921][68341] Saving new best policy, reward=7.570!
-[2023-10-07 17:14:02,914][67527] Fps is (10 sec: 9830.7, 60 sec: 8224.0, 300 sec: 8224.0). Total num frames: 311296. Throughput: 0: 967.7, 1: 973.8. Samples: 73490. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0)
-[2023-10-07 17:14:02,914][67527] Avg episode reward: [(0, '7.720'), (1, '7.960')]
-[2023-10-07 17:14:02,915][68341] Saving new best policy, reward=7.960!
-[2023-10-07 17:14:02,915][68242] Saving new best policy, reward=7.720!
-[2023-10-07 17:14:04,271][68496] Updated weights for policy 1, policy_version 640 (0.0017)
-[2023-10-07 17:14:04,272][68491] Updated weights for policy 0, policy_version 640 (0.0017)
-[2023-10-07 17:14:07,914][67527] Fps is (10 sec: 9830.6, 60 sec: 8411.4, 300 sec: 8411.4). Total num frames: 360448. Throughput: 0: 1020.5, 1: 1028.6. Samples: 87807. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0)
-[2023-10-07 17:14:07,914][67527] Avg episode reward: [(0, '8.280'), (1, '9.140')]
-[2023-10-07 17:14:07,915][68242] Saving new best policy, reward=8.280!
-[2023-10-07 17:14:07,915][68341] Saving new best policy, reward=9.140!
-[2023-10-07 17:14:10,532][67527] Keyboard interrupt detected in the event loop EvtLoop [Runner_EvtLoop, process=main process 67527], exiting...
-[2023-10-07 17:14:10,534][68542] Stopping RolloutWorker_w13...
-[2023-10-07 17:14:10,534][68534] Stopping RolloutWorker_w5...
-[2023-10-07 17:14:10,534][68531] Stopping RolloutWorker_w3...
-[2023-10-07 17:14:10,534][69313] Stopping RolloutWorker_w14...
-[2023-10-07 17:14:10,534][68540] Stopping RolloutWorker_w11...
-[2023-10-07 17:14:10,534][68537] Stopping RolloutWorker_w8...
-[2023-10-07 17:14:10,534][68533] Stopping RolloutWorker_w4...
-[2023-10-07 17:14:10,534][68538] Stopping RolloutWorker_w9...
-[2023-10-07 17:14:10,534][68542] Loop rollout_proc13_evt_loop terminating...
-[2023-10-07 17:14:10,534][68534] Loop rollout_proc5_evt_loop terminating...
-[2023-10-07 17:14:10,534][68540] Loop rollout_proc11_evt_loop terminating...
-[2023-10-07 17:14:10,534][68531] Loop rollout_proc3_evt_loop terminating...
-[2023-10-07 17:14:10,534][68536] Stopping RolloutWorker_w7...
-[2023-10-07 17:14:10,534][69313] Loop rollout_proc14_evt_loop terminating...
-[2023-10-07 17:14:10,534][68535] Stopping RolloutWorker_w6...
-[2023-10-07 17:14:10,534][68539] Stopping RolloutWorker_w10...
-[2023-10-07 17:14:10,534][68530] Stopping RolloutWorker_w2...
-[2023-10-07 17:14:10,534][68537] Loop rollout_proc8_evt_loop terminating...
-[2023-10-07 17:14:10,534][68498] Stopping RolloutWorker_w1...
-[2023-10-07 17:14:10,534][68533] Loop rollout_proc4_evt_loop terminating...
-[2023-10-07 17:14:10,534][68536] Loop rollout_proc7_evt_loop terminating...
-[2023-10-07 17:14:10,534][69345] Stopping RolloutWorker_w15...
-[2023-10-07 17:14:10,534][68535] Loop rollout_proc6_evt_loop terminating...
-[2023-10-07 17:14:10,534][68539] Loop rollout_proc10_evt_loop terminating...
-[2023-10-07 17:14:10,534][68497] Stopping RolloutWorker_w0...
-[2023-10-07 17:14:10,534][68541] Stopping RolloutWorker_w12...
-[2023-10-07 17:14:10,534][68538] Loop rollout_proc9_evt_loop terminating...
-[2023-10-07 17:14:10,534][68242] Stopping Batcher_0...
-[2023-10-07 17:14:10,534][68341] Stopping Batcher_1...
-[2023-10-07 17:14:10,534][67527] Runner profile tree view:
-main_loop: 53.2518
-[2023-10-07 17:14:10,535][69345] Loop rollout_proc15_evt_loop terminating...
-[2023-10-07 17:14:10,535][68530] Loop rollout_proc2_evt_loop terminating...
-[2023-10-07 17:14:10,535][68498] Loop rollout_proc1_evt_loop terminating...
-[2023-10-07 17:14:10,535][68497] Loop rollout_proc0_evt_loop terminating...
-[2023-10-07 17:14:10,535][68541] Loop rollout_proc12_evt_loop terminating...
-[2023-10-07 17:14:10,535][68242] Loop batcher_evt_loop terminating...
-[2023-10-07 17:14:10,535][67527] Collected {0: 188416, 1: 188416}, FPS: 7076.4
-[2023-10-07 17:14:10,536][68341] Loop batcher_evt_loop terminating...
-[2023-10-07 17:14:10,536][68242] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000000736_188416.pth...
-[2023-10-07 17:14:10,537][68341] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000000736_188416.pth...
-[2023-10-07 17:14:10,568][68242] Stopping LearnerWorker_p0...
-[2023-10-07 17:14:10,568][68242] Loop learner_proc0_evt_loop terminating...
-[2023-10-07 17:14:10,574][68341] Stopping LearnerWorker_p1...
-[2023-10-07 17:14:10,574][68341] Loop learner_proc1_evt_loop terminating...
-[2023-10-07 17:14:10,582][68491] Weights refcount: 2 0
-[2023-10-07 17:14:10,583][68496] Weights refcount: 2 0
-[2023-10-07 17:14:10,584][68491] Stopping InferenceWorker_p0-w0...
-[2023-10-07 17:14:10,584][68491] Loop inference_proc0-0_evt_loop terminating...
-[2023-10-07 17:14:10,585][68496] Stopping InferenceWorker_p1-w0...
-[2023-10-07 17:14:10,585][68496] Loop inference_proc1-0_evt_loop terminating...
+[2023-10-07 17:15:25,138][76906] Using optimizer <class 'torch.optim.adam.Adam'>
+[2023-10-07 17:15:25,139][76906] No checkpoints found
+[2023-10-07 17:15:25,139][76906] Did not load from checkpoint, starting from scratch!
+[2023-10-07 17:15:25,139][76906] Initialized policy 1 weights for model version 0
+[2023-10-07 17:15:25,142][76906] LearnerWorker_p1 finished initialization!
+[2023-10-07 17:15:25,142][76906] Using GPUs [0] for process 1 (actually maps to GPUs [1])
+[2023-10-07 17:15:25,995][76104] Starting process rollout_proc14
+[2023-10-07 17:15:25,998][76104] Starting process rollout_proc15
+[2023-10-07 17:15:26,001][77063] Using GPUs [1] for process 1 (actually maps to GPUs [1])
+[2023-10-07 17:15:26,001][77063] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [1]) for inference process 1
+[2023-10-07 17:15:26,004][77065] Worker 1 uses CPU cores [2, 3]
+[2023-10-07 17:15:26,020][77063] Num visible devices: 1
+[2023-10-07 17:15:26,036][77069] Worker 5 uses CPU cores [10, 11]
+[2023-10-07 17:15:26,059][77064] Worker 0 uses CPU cores [0, 1]
+[2023-10-07 17:15:26,081][77075] Worker 11 uses CPU cores [22, 23]
+[2023-10-07 17:15:26,146][77068] Worker 4 uses CPU cores [8, 9]
+[2023-10-07 17:15:26,276][77077] Worker 13 uses CPU cores [26, 27]
+[2023-10-07 17:15:26,489][77076] Worker 12 uses CPU cores [24, 25]
+[2023-10-07 17:15:26,501][77062] Using GPUs [0] for process 0 (actually maps to GPUs [0])
+[2023-10-07 17:15:26,501][77062] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0
+[2023-10-07 17:15:26,519][77062] Num visible devices: 1
+[2023-10-07 17:15:26,612][77074] Worker 10 uses CPU cores [20, 21]
+[2023-10-07 17:15:26,648][77073] Worker 9 uses CPU cores [18, 19]
+[2023-10-07 17:15:26,654][77070] Worker 6 uses CPU cores [12, 13]
+[2023-10-07 17:15:26,696][77072] Worker 7 uses CPU cores [14, 15]
+[2023-10-07 17:15:26,708][77066] Worker 2 uses CPU cores [4, 5]
+[2023-10-07 17:15:26,780][77071] Worker 8 uses CPU cores [16, 17]
+[2023-10-07 17:15:26,804][77067] Worker 3 uses CPU cores [6, 7]
+[2023-10-07 17:15:26,805][77063] RunningMeanStd input shape: (4, 84, 84)
+[2023-10-07 17:15:26,806][77063] RunningMeanStd input shape: (1,)
+[2023-10-07 17:15:26,819][77063] ConvEncoder: input_channels=4
+[2023-10-07 17:15:26,930][77063] Conv encoder output size: 512
+[2023-10-07 17:15:27,118][77062] RunningMeanStd input shape: (4, 84, 84)
+[2023-10-07 17:15:27,119][77062] RunningMeanStd input shape: (1,)
+[2023-10-07 17:15:27,131][77062] ConvEncoder: input_channels=4
+[2023-10-07 17:15:27,229][77062] Conv encoder output size: 512
+[2023-10-07 17:15:27,861][77789] Worker 14 uses CPU cores [28, 29]
+[2023-10-07 17:15:27,987][77790] Worker 15 uses CPU cores [30, 31]
+[2023-10-07 17:15:28,012][76104] Inference worker 1-0 is ready!
+[2023-10-07 17:15:28,014][76104] Inference worker 0-0 is ready!
+[2023-10-07 17:15:28,015][76104] All inference workers are ready! Signal rollout workers to start!
+[2023-10-07 17:15:28,016][76104] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan, 1: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
+[2023-10-07 17:15:28,708][77070] Decorrelating experience for 0 frames...
+[2023-10-07 17:15:28,708][77074] Decorrelating experience for 0 frames...
+[2023-10-07 17:15:28,708][77068] Decorrelating experience for 0 frames...
+[2023-10-07 17:15:28,712][77064] Decorrelating experience for 0 frames...
+[2023-10-07 17:15:28,713][77071] Decorrelating experience for 0 frames...
+[2023-10-07 17:15:28,717][77076] Decorrelating experience for 0 frames...
+[2023-10-07 17:15:28,748][77072] Decorrelating experience for 0 frames...
+[2023-10-07 17:15:28,811][77789] Decorrelating experience for 0 frames...
+[2023-10-07 17:15:28,910][77790] Decorrelating experience for 0 frames...
+[2023-10-07 17:15:28,927][77070] Decorrelating experience for 128 frames...
+[2023-10-07 17:15:28,931][77064] Decorrelating experience for 128 frames...
+[2023-10-07 17:15:28,932][77074] Decorrelating experience for 128 frames...
+[2023-10-07 17:15:28,938][77071] Decorrelating experience for 128 frames...
+[2023-10-07 17:15:28,938][77068] Decorrelating experience for 128 frames...
+[2023-10-07 17:15:28,948][77076] Decorrelating experience for 128 frames...
+[2023-10-07 17:15:28,967][77066] Decorrelating experience for 0 frames...
+[2023-10-07 17:15:28,971][77072] Decorrelating experience for 128 frames...
+[2023-10-07 17:15:28,996][77073] Decorrelating experience for 0 frames...
+[2023-10-07 17:15:29,004][77065] Decorrelating experience for 0 frames...
+[2023-10-07 17:15:29,027][77789] Decorrelating experience for 128 frames...
+[2023-10-07 17:15:29,129][77790] Decorrelating experience for 128 frames...
+[2023-10-07 17:15:29,220][77073] Decorrelating experience for 128 frames...
+[2023-10-07 17:15:29,230][77065] Decorrelating experience for 128 frames...
+[2023-10-07 17:15:29,256][77066] Decorrelating experience for 128 frames...
+[2023-10-07 17:15:29,280][77075] Decorrelating experience for 0 frames...
+[2023-10-07 17:15:29,281][77077] Decorrelating experience for 0 frames...
+[2023-10-07 17:15:29,287][77067] Decorrelating experience for 0 frames...
+[2023-10-07 17:15:29,289][77069] Decorrelating experience for 0 frames...
+[2023-10-07 17:15:29,679][77077] Decorrelating experience for 128 frames...
+[2023-10-07 17:15:29,681][77075] Decorrelating experience for 128 frames...
+[2023-10-07 17:15:29,687][77067] Decorrelating experience for 128 frames...
+[2023-10-07 17:15:29,687][77069] Decorrelating experience for 128 frames...
+[2023-10-07 17:15:31,088][76104] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 204.8, 1: 74.6. Samples: 858. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
+[2023-10-07 17:15:31,089][76104] Avg episode reward: [(1, '1.300')]
+[2023-10-07 17:15:36,088][76104] Fps is (10 sec: 2029.8, 60 sec: 2029.8, 300 sec: 2029.8). Total num frames: 16384. Throughput: 0: 253.7, 1: 507.5. Samples: 6144. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0)
+[2023-10-07 17:15:36,089][76104] Avg episode reward: [(0, '8.462'), (1, '2.737')]
+[2023-10-07 17:15:40,314][76104] Heartbeat connected on Batcher_0
+[2023-10-07 17:15:40,317][76104] Heartbeat connected on LearnerWorker_p0
+[2023-10-07 17:15:40,320][76104] Heartbeat connected on Batcher_1
+[2023-10-07 17:15:40,322][76104] Heartbeat connected on LearnerWorker_p1
+[2023-10-07 17:15:40,329][76104] Heartbeat connected on InferenceWorker_p0-w0
+[2023-10-07 17:15:40,333][76104] Heartbeat connected on RolloutWorker_w0
+[2023-10-07 17:15:40,334][76104] Heartbeat connected on InferenceWorker_p1-w0
+[2023-10-07 17:15:40,336][76104] Heartbeat connected on RolloutWorker_w1
+[2023-10-07 17:15:40,338][76104] Heartbeat connected on RolloutWorker_w2
+[2023-10-07 17:15:40,343][76104] Heartbeat connected on RolloutWorker_w3
+[2023-10-07 17:15:40,346][76104] Heartbeat connected on RolloutWorker_w4
+[2023-10-07 17:15:40,347][76104] Heartbeat connected on RolloutWorker_w5
+[2023-10-07 17:15:40,351][76104] Heartbeat connected on RolloutWorker_w7
+[2023-10-07 17:15:40,354][76104] Heartbeat connected on RolloutWorker_w6
+[2023-10-07 17:15:40,354][76104] Heartbeat connected on RolloutWorker_w8
+[2023-10-07 17:15:40,357][76104] Heartbeat connected on RolloutWorker_w9
+[2023-10-07 17:15:40,365][76104] Heartbeat connected on RolloutWorker_w11
+[2023-10-07 17:15:40,366][76104] Heartbeat connected on RolloutWorker_w10
+[2023-10-07 17:15:40,368][76104] Heartbeat connected on RolloutWorker_w13
+[2023-10-07 17:15:40,371][76104] Heartbeat connected on RolloutWorker_w12
+[2023-10-07 17:15:40,373][76104] Heartbeat connected on RolloutWorker_w15
+[2023-10-07 17:15:40,391][76104] Heartbeat connected on RolloutWorker_w14
+[2023-10-07 17:15:41,088][76104] Fps is (10 sec: 6553.6, 60 sec: 5013.6, 300 sec: 5013.6). Total num frames: 65536. Throughput: 0: 609.3, 1: 564.7. Samples: 15346. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0)
+[2023-10-07 17:15:41,089][76104] Avg episode reward: [(0, '7.050'), (1, '4.826')]
+[2023-10-07 17:15:46,088][76104] Fps is (10 sec: 9830.4, 60 sec: 6346.3, 300 sec: 6346.3). Total num frames: 114688. Throughput: 0: 823.2, 1: 794.3. Samples: 29231. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0)
+[2023-10-07 17:15:46,089][76104] Avg episode reward: [(0, '6.845'), (1, '5.827')]
+[2023-10-07 17:15:49,886][77062] Updated weights for policy 0, policy_version 320 (0.0016)
+[2023-10-07 17:15:49,886][77063] Updated weights for policy 1, policy_version 320 (0.0016)
+[2023-10-07 17:15:51,088][76104] Fps is (10 sec: 9830.2, 60 sec: 7101.3, 300 sec: 7101.3). Total num frames: 163840. Throughput: 0: 799.6, 1: 970.3. Samples: 40835. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0)
+[2023-10-07 17:15:51,089][76104] Avg episode reward: [(0, '7.137'), (1, '6.040')]
+[2023-10-07 17:15:56,088][76104] Fps is (10 sec: 9830.5, 60 sec: 7587.5, 300 sec: 7587.5). Total num frames: 212992. Throughput: 0: 931.4, 1: 911.5. Samples: 51732. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0)
+[2023-10-07 17:15:56,089][76104] Avg episode reward: [(0, '7.600'), (1, '6.750')]
+[2023-10-07 17:15:56,090][76817] Saving new best policy, reward=7.600!
+[2023-10-07 17:15:56,090][76906] Saving new best policy, reward=6.750!
+[2023-10-07 17:16:01,088][76104] Fps is (10 sec: 9830.6, 60 sec: 7926.5, 300 sec: 7926.5). Total num frames: 262144. Throughput: 0: 995.3, 1: 990.8. Samples: 65683. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0)
+[2023-10-07 17:16:01,089][76104] Avg episode reward: [(0, '7.980'), (1, '7.430')]
+[2023-10-07 17:16:01,097][76817] Saving new best policy, reward=7.980!
+[2023-10-07 17:16:01,097][76906] Saving new best policy, reward=7.430!
+[2023-10-07 17:16:06,088][76104] Fps is (10 sec: 9830.3, 60 sec: 8176.6, 300 sec: 8176.6). Total num frames: 311296. Throughput: 0: 968.3, 1: 1070.3. Samples: 77612. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0)
+[2023-10-07 17:16:06,089][76104] Avg episode reward: [(0, '9.290'), (1, '8.640')]
+[2023-10-07 17:16:06,090][76817] Saving new best policy, reward=9.290!
+[2023-10-07 17:16:06,090][76906] Saving new best policy, reward=8.640!
+[2023-10-07 17:16:06,617][77063] Updated weights for policy 1, policy_version 640 (0.0016)
+[2023-10-07 17:16:06,617][77062] Updated weights for policy 0, policy_version 640 (0.0018)
+[2023-10-07 17:16:10,324][76104] Keyboard interrupt detected in the event loop EvtLoop [Runner_EvtLoop, process=main process 76104], exiting...
+[2023-10-07 17:16:10,326][77068] Stopping RolloutWorker_w4...
+[2023-10-07 17:16:10,326][76906] Stopping Batcher_1...
+[2023-10-07 17:16:10,326][77068] Loop rollout_proc4_evt_loop terminating...
+[2023-10-07 17:16:10,326][77065] Stopping RolloutWorker_w1...
+[2023-10-07 17:16:10,326][77074] Stopping RolloutWorker_w10...
+[2023-10-07 17:16:10,326][77064] Stopping RolloutWorker_w0...
+[2023-10-07 17:16:10,326][77072] Stopping RolloutWorker_w7...
+[2023-10-07 17:16:10,326][77065] Loop rollout_proc1_evt_loop terminating...
+[2023-10-07 17:16:10,326][77074] Loop rollout_proc10_evt_loop terminating...
+[2023-10-07 17:16:10,326][77064] Loop rollout_proc0_evt_loop terminating...
+[2023-10-07 17:16:10,326][77072] Loop rollout_proc7_evt_loop terminating...
+[2023-10-07 17:16:10,326][76817] Stopping Batcher_0...
+[2023-10-07 17:16:10,326][76104] Runner profile tree view:
+main_loop: 49.9517
+[2023-10-07 17:16:10,327][76906] Loop batcher_evt_loop terminating...
+[2023-10-07 17:16:10,327][77075] Stopping RolloutWorker_w11...
+[2023-10-07 17:16:10,327][76104] Collected {0: 180224, 1: 180224}, FPS: 7215.9
+[2023-10-07 17:16:10,327][77075] Loop rollout_proc11_evt_loop terminating...
+[2023-10-07 17:16:10,328][77789] Stopping RolloutWorker_w14...
+[2023-10-07 17:16:10,328][77069] Stopping RolloutWorker_w5...
+[2023-10-07 17:16:10,328][77073] Stopping RolloutWorker_w9...
+[2023-10-07 17:16:10,327][76817] Loop batcher_evt_loop terminating...
+[2023-10-07 17:16:10,328][77789] Loop rollout_proc14_evt_loop terminating...
+[2023-10-07 17:16:10,328][76906] Saving ./train_atari/atari_alien_APPO/checkpoint_p1/checkpoint_000000704_180224.pth...
+[2023-10-07 17:16:10,328][76817] Saving ./train_atari/atari_alien_APPO/checkpoint_p0/checkpoint_000000704_180224.pth...
+[2023-10-07 17:16:10,329][77069] Loop rollout_proc5_evt_loop terminating...
+[2023-10-07 17:16:10,329][77073] Loop rollout_proc9_evt_loop terminating...
+[2023-10-07 17:16:10,330][77071] Stopping RolloutWorker_w8...
+[2023-10-07 17:16:10,330][77071] Loop rollout_proc8_evt_loop terminating...
+[2023-10-07 17:16:10,330][77790] Stopping RolloutWorker_w15...
+[2023-10-07 17:16:10,330][77066] Stopping RolloutWorker_w2...
+[2023-10-07 17:16:10,330][77067] Stopping RolloutWorker_w3...
+[2023-10-07 17:16:10,330][77066] Loop rollout_proc2_evt_loop terminating...
+[2023-10-07 17:16:10,330][77790] Loop rollout_proc15_evt_loop terminating...
+[2023-10-07 17:16:10,331][77067] Loop rollout_proc3_evt_loop terminating...
+[2023-10-07 17:16:10,332][77077] Stopping RolloutWorker_w13...
+[2023-10-07 17:16:10,332][77077] Loop rollout_proc13_evt_loop terminating...
+[2023-10-07 17:16:10,333][77070] Stopping RolloutWorker_w6...
+[2023-10-07 17:16:10,334][77070] Loop rollout_proc6_evt_loop terminating...
+[2023-10-07 17:16:10,339][77076] Stopping RolloutWorker_w12...
+[2023-10-07 17:16:10,339][77076] Loop rollout_proc12_evt_loop terminating...
+[2023-10-07 17:16:10,344][77062] Weights refcount: 2 0
+[2023-10-07 17:16:10,344][77063] Weights refcount: 2 0
+[2023-10-07 17:16:10,345][77063] Stopping InferenceWorker_p1-w0...
+[2023-10-07 17:16:10,345][77062] Stopping InferenceWorker_p0-w0...
+[2023-10-07 17:16:10,346][77063] Loop inference_proc1-0_evt_loop terminating...
+[2023-10-07 17:16:10,346][77062] Loop inference_proc0-0_evt_loop terminating...
+[2023-10-07 17:16:10,367][76906] Stopping LearnerWorker_p1...
+[2023-10-07 17:16:10,368][76906] Loop learner_proc1_evt_loop terminating...
+[2023-10-07 17:16:10,373][76817] Stopping LearnerWorker_p0...
+[2023-10-07 17:16:10,374][76817] Loop learner_proc0_evt_loop terminating...